UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
GDN-CMCF: A Gated Disentangled Network With Cross-Modality Consensus Fusion for Multimodal Named Entity Recognition
Huang, Guoheng1; He, Qin1; Dai, Zihao1; Zhong, Guo2; Yuan, Xiaochen3; Pun, Chi Man4
2023
Source PublicationIEEE Transactions on Computational Social Systems
ISSN2329-924X
Volume11Issue:3Pages:3944-3954
Abstract

Multimodal named entity recognition (MNER) is a crucial task in social systems of artificial intelligence that requires precise identification of named entities in sentences using both visual and textual information. Previous methods have focused on capturing fine-grained visual features and developing complex fusion procedures. However, these approaches overlook the heterogeneity gap and loss of original modality uniqueness that may occur during fusion, leading to incorrect entity identification. This article proposes a novel approach for MNER called a gated disentangled network with cross-modality consensus fusion (GDN-CMCF) to address the above challenges. Specifically, to eliminate cross-modality variation, we propose a cross-modality consensus fusion module that generates a consensus representation by learning inter-and intramodality interactions with a designed commonality constraint. We then introduce a gated disentanglement module to separate modality-relevant features from support and auxiliary modalities, which further filters out extraneous information while retaining the uniqueness of unimodal features. Experimental results on two real public datasets are provided to verify the effectiveness of our proposed GDN-CMCF. The source code of this article can be found at https://github.com/HaoDavis/ GDN-CMCF.

KeywordCommon Space Learning Correlation Feature Disentanglement Feature Extraction Logic Gates Multimodal Named Entity Recognition (Mner) Semantics Social Networking (Online) Task Analysis Visualization
DOI10.1109/TCSS.2023.3323402
URLView the original
Indexed BySCIE
Language英語English
WOS Research AreaComputer Science
WOS SubjectComputer Science, Cybernetics ; Computer Science, Information Systems
WOS IDWOS:001088287500004
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 445 HOES LANE, PISCATAWAY, NJ 08855-4141
Scopus ID2-s2.0-85174828134
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionFaculty of Science and Technology
DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorZhong, Guo; Pun, Chi Man
Affiliation1.School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China
2.School of Information Science and Technology, Guangdong University of Foreign Studies, Guangzhou, China
3.Faculty of Applied Sciences, Macao Polytechnic University, Macao, China
4.Faculty of Science and Technology, University of Macau, Macao, China
Corresponding Author AffilicationFaculty of Science and Technology
Recommended Citation
GB/T 7714
Huang, Guoheng,He, Qin,Dai, Zihao,et al. GDN-CMCF: A Gated Disentangled Network With Cross-Modality Consensus Fusion for Multimodal Named Entity Recognition[J]. IEEE Transactions on Computational Social Systems, 2023, 11(3), 3944-3954.
APA Huang, Guoheng., He, Qin., Dai, Zihao., Zhong, Guo., Yuan, Xiaochen., & Pun, Chi Man (2023). GDN-CMCF: A Gated Disentangled Network With Cross-Modality Consensus Fusion for Multimodal Named Entity Recognition. IEEE Transactions on Computational Social Systems, 11(3), 3944-3954.
MLA Huang, Guoheng,et al."GDN-CMCF: A Gated Disentangled Network With Cross-Modality Consensus Fusion for Multimodal Named Entity Recognition".IEEE Transactions on Computational Social Systems 11.3(2023):3944-3954.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Huang, Guoheng]'s Articles
[He, Qin]'s Articles
[Dai, Zihao]'s Articles
Baidu academic
Similar articles in Baidu academic
[Huang, Guoheng]'s Articles
[He, Qin]'s Articles
[Dai, Zihao]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Huang, Guoheng]'s Articles
[He, Qin]'s Articles
[Dai, Zihao]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.