Knowledge Commons of Institute of Automation，CAS
| Egocentric Gesture Recognition Using Recurrent 3D Convolutional Neural Networks with Spatiotemporal Transformer Modules | |
Cao, Congqi1,2 ; Zhang, Yifan1,2; Wu, Yi3,4; Lu, Hanqing1,2; Cheng, Jian1,2,5
| |
| 2017-10-22 | |
| 会议名称 | IEEE International Conference on Computer Vision |
| 会议日期 | 22-29 Oct. 2017 |
| 会议地点 | Venice, Italy |
| 摘要 | Gesture is a natural interface in interacting with wearable devices such as VR/AR helmet and glasses. The main challenge of gesture recognition in egocentric vision arises from the global camera motion caused by the spontaneous head movement of the device wearer. In this paper, we address the problem by a novel recurrent 3D convolutional neural network for end-to-end learning. We specially design a spatiotemporal transformer module with recurrent connections between neighboring time slices which can actively transform a 3D feature map into a canonical view in both spatial and temporal dimensions. To validate our method, we introduce a new dataset with sufficient size, variation and reality, which contains 83 gestures designed for interaction with wearable devices, and more than 24,000 RGB-D gesture samples from 50 subjects captured in 6 scenes. On this dataset, we show that the proposed network outperforms competing state-of-the-art algorithms. Moreover, our method can achieve state-of-the-art performance on the challenging GTEA egocentric action dataset. |
| DOI | 10.1109/ICCV.2017.406 |
| 收录类别 | EI |
| 引用统计 | |
| 文献类型 | 会议论文 |
| 条目标识符 | http://ir.ia.ac.cn/handle/173211/20890 |
| 专题 | 紫东太初大模型研究中心_图像与视频分析 |
| 作者单位 | 1.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences 2.University of Chinese Academy of Sciences 3.School of Technology, Nanjing Audit University 4.Department of Medicine, Indiana University, USA 5.CAS Center for Excellence in Brain Science and Intelligence Technology |
| 第一作者单位 | 模式识别国家重点实验室 |
| 推荐引用方式 GB/T 7714 | Cao, Congqi,Zhang, Yifan,Wu, Yi,et al. Egocentric Gesture Recognition Using Recurrent 3D Convolutional Neural Networks with Spatiotemporal Transformer Modules[C],2017. |
| 条目包含的文件 | 下载所有文件 | |||||
| 文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
| 曹聪琦_ICCV2017_Egocent（702KB） | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 | ||
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。
修改评论