Knowledge Commons of Institute of Automation,CAS
Improving visual question answering using dropout and enhanced question encoder | |
Fang, Zhiwei1,2; Liu, Jing1; Li, Yong3; Qiao, Yanyuan2; Lu, Hanqing1 | |
发表期刊 | PATTERN RECOGNITION |
ISSN | 0031-3203 |
2019-06-01 | |
卷号 | 90期号:1页码:404-414 |
摘要 | Using dropout in Visual Question Answering (VQA) is a common practice to prevent overfitting. However, the current way to use dropout in multi-path networks may cause two problems: the co-adaptations of neurons and the explosion of output variance. In this paper, we propose coherent dropout and siamese dropout mechanism to solve the two problems, respectively. Specifically, in coherent dropout, the relevant dropout layers in multiple paths are forced to work coherently to maximize the ability of preventing neuron co-adaptations. We show that the coherent dropout is simple in implementation but very effective to overcome overfitting. As for the explosion of output variance, we develop a siamese dropout mechanism to explicitly minimize the difference between the two output vectors produced from the same input data during training phase. Such mechanism can reduce the gap between training and inference phases and make the VQA model more robust. With the help of the two techniques, we further design an enhanced question encoder called Multi-path Stacked Residual RNNs which is deeper and wider and more powerful than current shallow question encoder. Extensive experiments are conducted to verify the effectiveness of coherent dropout, siamese dropout and the enhanced question encoder. And the results show that our methods can bring clear improvements to the state-of-the-art VQA models on VQA-vl and VQA-v2 datasets. (C) 2019 Elsevier Ltd. All rights reserved. |
关键词 | Visual question answering Coherent dropout Siamese dropout Enhanced question encoder |
DOI | 10.1016/j.patcog.2019.01.038 |
关键词[WOS] | NETWORKS |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Natural Science Foundation of China[61872366] ; Beijing Municipal Natural Science Foundation[4192059] ; National Natural Science Foundation of China[61872366] ; Beijing Municipal Natural Science Foundation[4192059] |
WOS研究方向 | Computer Science ; Engineering |
WOS类目 | Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic |
WOS记录号 | WOS:000463130400033 |
出版者 | ELSEVIER SCI LTD |
七大方向——子方向分类 | 多模态智能 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/23484 |
专题 | 紫东太初大模型研究中心_图像与视频分析 |
通讯作者 | Liu, Jing |
作者单位 | 1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, 95 Zhongguancun East Rd, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.JD Com, Business Growth BU, Intelligent Advertising Lab, Beijing, Peoples R China |
第一作者单位 | 模式识别国家重点实验室 |
通讯作者单位 | 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | Fang, Zhiwei,Liu, Jing,Li, Yong,et al. Improving visual question answering using dropout and enhanced question encoder[J]. PATTERN RECOGNITION,2019,90(1):404-414. |
APA | Fang, Zhiwei,Liu, Jing,Li, Yong,Qiao, Yanyuan,&Lu, Hanqing.(2019).Improving visual question answering using dropout and enhanced question encoder.PATTERN RECOGNITION,90(1),404-414. |
MLA | Fang, Zhiwei,et al."Improving visual question answering using dropout and enhanced question encoder".PATTERN RECOGNITION 90.1(2019):404-414. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Improving visual que(1624KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论