CASIA OpenIR  > 模式识别国家重点实验室  > 生物识别与安全技术研究
A Unified Framework for Multi-Modal Isolated Gesture Recognition; A Unified Framework for Multi-Modal Isolated Gesture Recognition
Jiali Duan1; Jun Wan1; Shuai Zhou2; Xiaoyuan Guo3; Stan Z. Li1
Source PublicationACM Transactions on Multimedia Computing, Communications, and Applications ; ACM Transactions on Multimedia Computing, Communications, and Applications
2017 ; 2017
Volume9Issue:4Pages:39:1-39:17
Abstract  ; In this paper, we focus on isolated gesture recognition and explore different modalities by involving RGB stream, depth stream and saliency stream for inspection. Our goal is to push the boundary of this realm even further by proposing a unified framework which exploits the advantages of multi-modality fusion. Specifically, a spatial-temporal network architecture based on consensus-voting has been proposed to explicitly model the long term structure of the video sequence and to reduce estimation variance when confronted with comprehensive inter-class variations. In addition, a 3D depth-saliency convolutional network is aggregated in parallel to capture subtle motion characteristics. Extensive experiments are done to analyze the performance of each component and our proposed approach achieves the best results on two public benchmarks–ChaLearn IsoGD and RGBD-HuDaAct, outperforming the closest competitor by a margin of over 10% and 15% respectively. We will release our codes to facilitate future research.;   ; In this paper, we focus on isolated gesture recognition and explore different modalities by involving RGB stream, depth stream and saliency stream for inspection. Our goal is to push the boundary of this realm even further by proposing a unified framework which exploits the advantages of multi-modality fusion. Specifically, a spatial-temporal network architecture based on consensus-voting has been proposed to explicitly model the long term structure of the video sequence and to reduce estimation variance when confronted with comprehensive inter-class variations. In addition, a 3D depth-saliency convolutional network is aggregated in parallel to capture subtle motion characteristics. Extensive experiments are done to analyze the performance of each component and our proposed approach achieves the best results on two public benchmarks–ChaLearn IsoGD and RGBD-HuDaAct, outperforming the closest competitor by a margin of over 10% and 15% respectively. We will release our codes to facilitate future research.
KeywordMulti-modal Multi-modal Consensus-voting Consensus-voting 3d Convolution 3d Convolution Isolated Gesture Recognition Isolated Gesture Recognition
WOS IDWOS:000433517100007 ; WOS:000433517100007
Citation statistics
Cited Times:2[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/15302
Collection模式识别国家重点实验室_生物识别与安全技术研究
Corresponding AuthorJun Wan
Affiliation1.Center for Biometrics and Security Research & National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China
2.Macau University of Science and Technology
3.School of Engineering Science, University of Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Jiali Duan,Jun Wan,Shuai Zhou,et al. A Unified Framework for Multi-Modal Isolated Gesture Recognition, A Unified Framework for Multi-Modal Isolated Gesture Recognition[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, ACM Transactions on Multimedia Computing, Communications, and Applications,2017, 2017,9, 9(4):39:1-39:17, 39:1-39:17.
APA Jiali Duan,Jun Wan,Shuai Zhou,Xiaoyuan Guo,&Stan Z. Li.(2017).A Unified Framework for Multi-Modal Isolated Gesture Recognition.ACM Transactions on Multimedia Computing, Communications, and Applications,9(4),39:1-39:17.
MLA Jiali Duan,et al."A Unified Framework for Multi-Modal Isolated Gesture Recognition".ACM Transactions on Multimedia Computing, Communications, and Applications 9.4(2017):39:1-39:17.
Files in This Item: Download All
File Name/Size DocType Version Access License
TOMM2017_isogesture.(6349KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Jiali Duan]'s Articles
[Jun Wan]'s Articles
[Shuai Zhou]'s Articles
Baidu academic
Similar articles in Baidu academic
[Jiali Duan]'s Articles
[Jun Wan]'s Articles
[Shuai Zhou]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Jiali Duan]'s Articles
[Jun Wan]'s Articles
[Shuai Zhou]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: TOMM2017_isogesture.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.