Relational graph neural network for situation recognition | |
Jing Y(荆雅)1,2,3; Wang JB(王君波)1,2,3,4; Wang W(王威)1,2,3; Wang L(王亮)1,2,3; Tan TN(谭铁牛)1,2,3 | |
发表期刊 | Pattern Recognition |
2020 | |
期号 | 108页码:107544 |
摘要 | Recently, situation recognition as a new challenging task for image understanding has gained great attention, which needs to simultaneously predict the main activity (verb) and its associated objects (noun entities) in a structured and detailed way. Several methods have been proposed to handle this task, but usually they cannot effectively model the relationships between the activity and the objects. In this paper, we propose a Relational Graph Neural Network (RGNN) for situation recognition, which builds a neural graph on the activity and the objects, and models the triplet relationships between the activity and pairs of objects through message passing between graph nodes. Moreover, we propose a two-stage training strategy to optimize the model. A progressive supervised learning is first adopted to obtain an initial prediction for the activity and the objects. Then, the initial predictions are refined by using a policy-gradient method to directly optimize the non-differentiable value-all metric. To verify the effectiveness of our method, we perform extensive experiments on the Imsitu dataset which is currently the only available dataset for situation recognition. Experimental results show that our approach outperforms the state-ofthe-art methods on verb and value metrics, and demonstrates better relationships between the activity and the objects. |
关键词 | Situation recognition Relationship modeling Graph neural network Reinforcement learning |
语种 | 英语 |
七大方向——子方向分类 | 模式识别基础 |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/44448 |
专题 | 模式识别实验室 |
通讯作者 | Wang W(王威) |
作者单位 | 1.Center for Research on Intelligent Perception and Computing, CASIA 2.National Laboratory of Pattern Recognition, CASIA 3.University of Chinese Academy of Sciences 4.Tencent Games |
第一作者单位 | 模式识别国家重点实验室; 中国科学院自动化研究所 |
通讯作者单位 | 模式识别国家重点实验室; 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Jing Y,Wang JB,Wang W,et al. Relational graph neural network for situation recognition[J]. Pattern Recognition,2020(108):107544. |
APA | Jing Y,Wang JB,Wang W,Wang L,&Tan TN.(2020).Relational graph neural network for situation recognition.Pattern Recognition(108),107544. |
MLA | Jing Y,et al."Relational graph neural network for situation recognition".Pattern Recognition .108(2020):107544. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
1-s2.0-S003132032030(3098KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论