CASIA OpenIR  > 模式识别国家重点实验室  > 图像与视频分析
Egocentric Gesture Recognition Using Recurrent 3D Convolutional Neural Networks with Spatiotemporal Transformer Modules
Cao, Congqi1,2; Zhang, Yifan1,2; Wu, Yi3,4; Lu, Hanqing1,2; Cheng, Jian1,2,5
2017-10-22
Conference NameIEEE International Conference on Computer Vision
Conference Date22-29 Oct. 2017
Conference PlaceVenice, Italy
AbstractGesture is a natural interface in interacting with wearable devices such as VR/AR helmet and glasses. The main challenge of gesture recognition in egocentric vision arises from the global camera motion caused by the spontaneous head movement of the device wearer. In this paper, we address the problem by a novel recurrent 3D convolutional neural network for end-to-end learning. We specially design a spatiotemporal transformer module with recurrent connections between neighboring time slices which can actively transform a 3D feature map into a canonical view in both spatial and temporal dimensions. To validate our method, we introduce a new dataset with sufficient size, variation and reality, which contains 83 gestures designed for interaction with wearable devices, and more than 24,000 RGB-D gesture samples from 50 subjects captured in 6 scenes. On this dataset, we show that the proposed network outperforms competing state-of-the-art algorithms. Moreover, our method can achieve state-of-the-art performance on the challenging GTEA egocentric action dataset.
DOI10.1109/ICCV.2017.406
Indexed ByEI
Citation statistics
Cited Times:1[WOS]   [WOS Record]     [Related Records in WOS]
Document Type会议论文
Identifierhttp://ir.ia.ac.cn/handle/173211/20890
Collection模式识别国家重点实验室_图像与视频分析
Affiliation1.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences
2.University of Chinese Academy of Sciences
3.School of Technology, Nanjing Audit University
4.Department of Medicine, Indiana University, USA
5.CAS Center for Excellence in Brain Science and Intelligence Technology
First Author AffilicationChinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
Recommended Citation
GB/T 7714
Cao, Congqi,Zhang, Yifan,Wu, Yi,et al. Egocentric Gesture Recognition Using Recurrent 3D Convolutional Neural Networks with Spatiotemporal Transformer Modules[C],2017.
Files in This Item: Download All
File Name/Size DocType Version Access License
曹聪琦_ICCV2017_Egocent(702KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Cao, Congqi]'s Articles
[Zhang, Yifan]'s Articles
[Wu, Yi]'s Articles
Baidu academic
Similar articles in Baidu academic
[Cao, Congqi]'s Articles
[Zhang, Yifan]'s Articles
[Wu, Yi]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Cao, Congqi]'s Articles
[Zhang, Yifan]'s Articles
[Wu, Yi]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 曹聪琦_ICCV2017_Egocentric Gesture Recognition Using Recurrent 3D Convolutional Neural Networks with Spatiotemporal Transformer Modules.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.