Knowledge Commons of Institute of Automation,CAS
View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition | |
Zhang, Pengfei1; Lan, Cuiling3; Xing, Junliang5; Zeng, Wenjun4; Xue, Jianru2; Zheng, Nanning2 | |
发表期刊 | IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE |
ISSN | 0162-8828 |
2019-08-01 | |
卷号 | 41期号:8页码:1963-1978 |
通讯作者 | Lan, Cuiling(culan@microsoft.com) ; Xue, Jianru(jrxue@mail.xjtu.edu.cn) |
摘要 | Skeleton-based human action recognition has recently attracted increasing attention thanks to the accessibility and the popularity of 3D skeleton data. One of the key challenges in action recognition lies in the large variations of action representations when they are captured from different viewpoints. In order to alleviate the effects of view variations, this paper introduces a novel view adaptation scheme, which automatically determines the virtual observation viewpoints over the course of an action in a learning based data driven manner. Instead of re-positioning the skeletons using a fixed human-defined prior criterion, we design two view adaptive neural networks, i.e., VA-RNN and VA-CNN, which are respectively built based on the recurrent neural network (RNN) with the Long Short-term Memory (LSTM) and the convolutional neural network (CNN). For each network, a novel view adaptation module learns and determines the most suitable observation viewpoints, and transforms the skeletons to those viewpoints for the end-to-end recognition with a main classification network. Ablation studies find that the proposed view adaptive models are capable of transforming the skeletons of various views to much more consistent virtual viewpoints. Therefore, the models largely eliminate the influence of the viewpoints, enabling the networks to focus on the learning of action-specific features and thus resulting in superior performance. In addition, we design a two-stream scheme (referred to as VA-fusion) that fuses the scores of the two networks to provide the final prediction, obtaining enhanced performance. Moreover, random rotation of skeleton sequences is employed to improve the robustness of view adaptation models and alleviate overfitting during training. Extensive experimental evaluations on five challenging benchmarks demonstrate the effectiveness of the proposed view-adaptive networks and superior performance over state-of-the-art approaches. |
关键词 | View adaptation skeleton action recognition RNN CNN consistent |
DOI | 10.1109/TPAMI.2019.2896631 |
关键词[WOS] | CLASSIFICATION ; MOTION ; ROBUST |
收录类别 | SCI |
语种 | 英语 |
资助项目 | Natural Science Foundation of China[61672519] ; National Key Research and Development Program of China[2016YFB1001004] ; National Key Research and Development Program of China[2016YFB1001004] ; Natural Science Foundation of China[61672519] |
项目资助者 | National Key Research and Development Program of China ; Natural Science Foundation of China |
WOS研究方向 | Computer Science ; Engineering |
WOS类目 | Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic |
WOS记录号 | WOS:000473598800013 |
出版者 | IEEE COMPUTER SOC |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/26841 |
专题 | 复杂系统认知与决策实验室_智能系统与工程 |
通讯作者 | Lan, Cuiling; Xue, Jianru |
作者单位 | 1.Xi An Jiao Tong Univ, Xian 710049, Shaanxi, Peoples R China 2.Xi An Jiao Tong Univ, Inst Artificial Intelligence & Robot, Xian 710049, Shaanxi, Peoples R China 3.Microsoft Res Asia, Beijing 100080, Peoples R China 4.Microsoft Res Asia, Senior Leadership Team, Beijing 100080, Peoples R China 5.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing, Peoples R China |
推荐引用方式 GB/T 7714 | Zhang, Pengfei,Lan, Cuiling,Xing, Junliang,et al. View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2019,41(8):1963-1978. |
APA | Zhang, Pengfei,Lan, Cuiling,Xing, Junliang,Zeng, Wenjun,Xue, Jianru,&Zheng, Nanning.(2019).View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,41(8),1963-1978. |
MLA | Zhang, Pengfei,et al."View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 41.8(2019):1963-1978. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论