Knowledge Commons of Institute of Automation,CAS
A Unified Framework for Multi-Modal Isolated Gesture Recognition | |
Duan, Jiali1,2; Wan, Jun1,2; Zhou, Shuai3; Guo, Xiaoyuan4; Li, Stan Z.1,2 | |
发表期刊 | ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS |
ISSN | 1551-6857 |
2018-04-01 | |
卷号 | 14期号:1页码:16 |
通讯作者 | Wan, Jun(jun.wan@ia.ac.cn) |
摘要 | In this article, we focus on isolated gesture recognition and explore different modalities by involving RGB stream, depth stream, and saliency stream for inspection. Our goal is to push the boundary of this realm even further by proposing a unified framework that exploits the advantages of multi-modality fusion. Specifically, a spatial-temporal network architecture based on consensus-voting has been proposed to explicitly model the long-term structure of the video sequence and to reduce estimation variance when confronted with comprehensive inter-class variations. In addition, a three-dimensional depth-saliency convolutional network is aggregated in parallel to capture subtle motion characteristics. Extensive experiments are done to analyze the performance of each component and our proposed approach achieves the best results on two public benchmarks, ChaLearn IsoGD and RGBD-HuDaAct, outperforming the closest competitor by a margin of over 10% and 15%, respectively. Our project and codes will be released at https:// davidsonic.github. io/index/acm_tomm_2017.html. |
关键词 | Multi-modal consensus-voting 3D convolution isolated gesture recognition |
DOI | 10.1145/3131343 |
关键词[WOS] | REAL-TIME |
收录类别 | SCI |
语种 | 英语 |
资助项目 | AuthenMetric RD Funds ; NVIDIA GPU ; Science and Technology Development Fund of Macau[112/2014/A3] ; Chinese National Natural Science Foundation[61673052] ; Chinese National Natural Science Foundation[61572536] ; Chinese National Natural Science Foundation[61572501] ; Chinese National Natural Science Foundation[61473291] ; Chinese National Natural Science Foundation[61502491] ; National Key Research and Development Plan[2016YFC0801002] ; National Key Research and Development Plan[2016YFC0801002] ; Chinese National Natural Science Foundation[61502491] ; Chinese National Natural Science Foundation[61473291] ; Chinese National Natural Science Foundation[61572501] ; Chinese National Natural Science Foundation[61572536] ; Chinese National Natural Science Foundation[61673052] ; Science and Technology Development Fund of Macau[112/2014/A3] ; NVIDIA GPU ; AuthenMetric RD Funds |
项目资助者 | National Key Research and Development Plan ; Chinese National Natural Science Foundation ; Science and Technology Development Fund of Macau ; NVIDIA GPU ; AuthenMetric RD Funds |
WOS研究方向 | Computer Science |
WOS类目 | Computer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods |
WOS记录号 | WOS:000433517100007 |
出版者 | ASSOC COMPUTING MACHINERY |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/15302 |
专题 | 多模态人工智能系统全国重点实验室_生物识别与安全技术 |
通讯作者 | Wan, Jun |
作者单位 | 1.Chinese Acad Sci, CBSR, Inst Automat, Beijing, Peoples R China 2.Chinese Acad Sci, NLPR, Inst Automat, Beijing, Peoples R China 3.Macau Univ Sci & Technol, Taipa, Macao, Peoples R China 4.Univ Chinese Acad Sci, Sch Engn Sci, Beijing, Peoples R China |
第一作者单位 | 模式识别国家重点实验室 |
通讯作者单位 | 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | Duan, Jiali,Wan, Jun,Zhou, Shuai,et al. A Unified Framework for Multi-Modal Isolated Gesture Recognition[J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,2018,14(1):16. |
APA | Duan, Jiali,Wan, Jun,Zhou, Shuai,Guo, Xiaoyuan,&Li, Stan Z..(2018).A Unified Framework for Multi-Modal Isolated Gesture Recognition.ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,14(1),16. |
MLA | Duan, Jiali,et al."A Unified Framework for Multi-Modal Isolated Gesture Recognition".ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 14.1(2018):16. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
TOMM2017_isogesture.(6349KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论