Knowledge Commons of Institute of Automation,CAS
STRNet: Triple-stream Spatiotemporal Relation Network for Action Recognition | |
Zhi-Wei Xu1,2; Xiao-Jun Wu1,2; Josef Kittler3 | |
发表期刊 | International Journal of Automation and Computing |
ISSN | 1476-8186 |
2021 | |
卷号 | 18期号:5页码:718-730 |
摘要 | Learning comprehensive spatiotemporal features is crucial for human action recognition. Existing methods tend to model the spatiotemporal feature blocks in an integrate-separate-integrate form, such as appearance-and-relation network (ARTNet) and spatiotemporal and motion network (STM). However, with blocks stacking up, the rear part of the network has poor interpretability. To avoid this problem, we propose a novel architecture called spatial temporal relation network (STRNet), which can learn explicit information of appearance, motion and especially the temporal relation information. Specifically, our STRNet is constructed by three branches, which separates the features into 1) appearance pathway, to obtain spatial semantics, 2) motion pathway, to reinforce the spatiotemporal feature representation, and 3) relation pathway, to focus on capturing temporal relation details of successive frames and to explore long-term representation dependency. In addition, our STRNet does not just simply merge the multi-branch information, but we apply a flexible and effective strategy to fuse the complementary information from multiple pathways. We evaluate our network on four major action recognition benchmarks: Kinetics-400, UCF-101, HMDB-51, and Something-Something v1, demonstrating that the performance of our STRNet achieves the state-of-the-art result on the UCF-101 and HMDB-51 datasets, as well as a comparable accuracy with the state-of-the-art method on Something-Something v1 and Kinetics-400. |
关键词 | Action recognition spatiotemporal relation multi-branch fusion long-term representation video classification |
DOI | 10.1007/s11633-021-1289-9 |
七大方向——子方向分类 | 其他 |
国重实验室规划方向分类 | 其他 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/45459 |
专题 | 学术期刊_Machine Intelligence Research |
作者单位 | 1.School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China 2.Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence, Wuxi 214122, China 3.Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, GU2 7XH, UK |
推荐引用方式 GB/T 7714 | Zhi-Wei Xu,Xiao-Jun Wu,Josef Kittler. STRNet: Triple-stream Spatiotemporal Relation Network for Action Recognition[J]. International Journal of Automation and Computing,2021,18(5):718-730. |
APA | Zhi-Wei Xu,Xiao-Jun Wu,&Josef Kittler.(2021).STRNet: Triple-stream Spatiotemporal Relation Network for Action Recognition.International Journal of Automation and Computing,18(5),718-730. |
MLA | Zhi-Wei Xu,et al."STRNet: Triple-stream Spatiotemporal Relation Network for Action Recognition".International Journal of Automation and Computing 18.5(2021):718-730. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
IJAC-2020-10-291.pdf(1129KB) | 期刊论文 | 出版稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论