Explanation guided cross-modal social image clustering
Yan, Xiaoqiang1; Mao, Yiqiao1; Ye, Yangdong1; Yu, Hui2; Wang, Fei-Yue3,4,5
Corresponding AuthorYe, Yangdong(ieydye@zzu.edu.cn)
AbstractThe integration of visual and semantic information has been found to play a role in increasing the accuracy of social image clustering methods. However, existing approaches are limited by the heterogeneity gap between the visual and semantic modalities, and their performances significantly degrade due to the commonly sparse and incomplete tags in semantic modality. To address these problems, we propose a novel clustering framework to discover reasonable categories in unlabeled social images under the guidance of human explanations. First of all, a novel Explanation Generation Model (EGM) is proposed to automatically boost textual information for the sparse and incomplete tags based on an extra lexical database with human knowledge. Then, a novel clustering algorithm called Group Constrained Information Maximization (GCIM) is proposed to learn image categories. In this algorithm, a new type of constraint named group level side information is unprecedentedly defined to bridge the well-known heterogeneity gap between visual and textual modalities. Finally, an interactive draw-and-merge optimization method is proposed to ensure an optimal solution. Extensive experiments on several social image datasets including NUS-Wide, IAPRTC, MIRFlickr, ESP-Game and COCO demonstrate the superiority of the proposed approach to state-of-the-art baselines. (c) 2022 Elsevier Inc. All rights reserved.
KeywordSocial image clustering Human explanation Side information Information maximization Interactive optimization
Indexed BySCI
Funding ProjectNational Natural Science Foundation of China[61906172] ; National Natural Science Foundation of China[62176239] ; Postdoctoral Research Foundation of China[2020M682357] ; EPSRC through project 4D Facial Sensing and Modelling[EP/N025849/1]
Funding OrganizationNational Natural Science Foundation of China ; Postdoctoral Research Foundation of China ; EPSRC through project 4D Facial Sensing and Modelling
WOS Research AreaComputer Science
WOS SubjectComputer Science, Information Systems
WOS IDWOS:000770686400001
Citation statistics
Cited Times:6[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Corresponding AuthorYe, Yangdong
Affiliation1.Zhengzhou Univ, Sch Informat Engn, Zhengzhou 450052, Peoples R China
2.Univ Portsmouth, Sch Creat Technol, Portsmouth PO1 2DJ, Hants, England
3.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
4.Macau Univ Sci & Technol, Inst Syst Engn, Taipa 999078, Macao, Peoples R China
5.Qingdao Acad Intelligent Ind, Qingdao 266109, Peoples R China
Recommended Citation
GB/T 7714
Yan, Xiaoqiang,Mao, Yiqiao,Ye, Yangdong,et al. Explanation guided cross-modal social image clustering[J]. INFORMATION SCIENCES,2022,593:1-16.
APA Yan, Xiaoqiang,Mao, Yiqiao,Ye, Yangdong,Yu, Hui,&Wang, Fei-Yue.(2022).Explanation guided cross-modal social image clustering.INFORMATION SCIENCES,593,1-16.
MLA Yan, Xiaoqiang,et al."Explanation guided cross-modal social image clustering".INFORMATION SCIENCES 593(2022):1-16.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yan, Xiaoqiang]'s Articles
[Mao, Yiqiao]'s Articles
[Ye, Yangdong]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yan, Xiaoqiang]'s Articles
[Mao, Yiqiao]'s Articles
[Ye, Yangdong]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yan, Xiaoqiang]'s Articles
[Mao, Yiqiao]'s Articles
[Ye, Yangdong]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.