Object-difference drived graph convolutional networks for visual question answering
Zhu, Xi1,2; Mao, Zhendong3; Chen, Zhineng4; Li, Yangyang5; Wang, Zhaohui1,2; Wang, Bin6
发表期刊MULTIMEDIA TOOLS AND APPLICATIONS
ISSN1380-7501
2020-03-20
页码19
通讯作者Mao, Zhendong(maozhendong2008@gmail.com)
摘要Visual Question Answering(VQA), an important task to evaluate the cross-modal understanding capability of an Artificial Intelligence model, has been a hot research topic in both computer vision and natural language processing communities. Recently, graph-based models have received growing interest in VQA, for its potential of modeling the relationships between objects as well as its formidable interpretability. Nonetheless, those solutions mainly define the similarity between objects as their semantical relationships, while largely ignoring the critical point that the difference between objects can provide more information for establishing the relationship between nodes in the graph. To achieve this, we propose an object-difference based graph learner, which learns question-adaptive semantic relations by calculating inter-object difference under the guidance of questions. With the learned relationships, the input image can be represented as an object graph encoded with structural dependencies between objects. In addition, existing graph-based models leverage the pre-extracted object boxes by the object detection model as node features for convenience, but they are suffering from the redundancy problem. To reduce the redundant objects, we introduce a soft-attention mechanism to magnify the question-related objects. Moreover, we incorporate our object-difference based graph learner into the soft-attention based Graph Convolutional Networks to capture question-specific objects and their interactions for answer prediction. Our experimental results on the VQA 2.0 dataset demonstrate that our model gives significantly better performance than baseline methods.
关键词Visual question answering Graph convolutional networks Object-difference
DOI10.1007/s11042-020-08790-0
收录类别SCI
语种英语
资助项目National Key Research and Development Program of China[2016QY03D0505] ; National Natural Science Foundation of China[U19A2057]
项目资助者National Key Research and Development Program of China ; National Natural Science Foundation of China
WOS研究方向Computer Science ; Engineering
WOS类目Computer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号WOS:000521018700001
出版者SPRINGER
七大方向——子方向分类智能交互
引用统计
被引频次:19[WOS]   [WOS记录]     [WOS相关记录]
文献类型期刊论文
条目标识符http://ir.ia.ac.cn/handle/173211/38697
专题数字内容技术与服务研究中心_远程智能医疗
通讯作者Mao, Zhendong
作者单位1.Chinese Acad Sci, Inst Informat Engn, Beijing, Peoples R China
2.Univ Chinese Acad Sci, Sch Cyber Secur, Beijing, Peoples R China
3.Univ Sci & Technol China, Hefei, Peoples R China
4.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China
5.China Acad Elect & Informat Technol, Beijing, Peoples R China
6.Xiaomi Inc, Xiaomi AI Lab, Beijing, Peoples R China
推荐引用方式
GB/T 7714
Zhu, Xi,Mao, Zhendong,Chen, Zhineng,et al. Object-difference drived graph convolutional networks for visual question answering[J]. MULTIMEDIA TOOLS AND APPLICATIONS,2020:19.
APA Zhu, Xi,Mao, Zhendong,Chen, Zhineng,Li, Yangyang,Wang, Zhaohui,&Wang, Bin.(2020).Object-difference drived graph convolutional networks for visual question answering.MULTIMEDIA TOOLS AND APPLICATIONS,19.
MLA Zhu, Xi,et al."Object-difference drived graph convolutional networks for visual question answering".MULTIMEDIA TOOLS AND APPLICATIONS (2020):19.
条目包含的文件
条目无相关文件。
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Zhu, Xi]的文章
[Mao, Zhendong]的文章
[Chen, Zhineng]的文章
百度学术
百度学术中相似的文章
[Zhu, Xi]的文章
[Mao, Zhendong]的文章
[Chen, Zhineng]的文章
必应学术
必应学术中相似的文章
[Zhu, Xi]的文章
[Mao, Zhendong]的文章
[Chen, Zhineng]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。