Knowledge Commons of Institute of Automation,CAS
Joint Learning in the Spatio-Temporal and Frequency Domains for Skeleton-Based Action Recognition | |
Guyue, Hu1,2,3; Bo, Cui1,2,3; Shan, Yu1,2,3,4 | |
发表期刊 | IEEE TRANSACTIONS ON MULTIMEDIA |
ISSN | 1520-9210 |
2020-09-01 | |
卷号 | 22期号:9页码:2207-2220 |
文章类型 | Article |
摘要 | Benefiting from its succinctness and robustness, skeleton-based action recognition has recently attracted much attention. Most existing methods utilize local networks (e.g. recurrent network, convolutional network, and graph convolutional network) to extract spatio-temporal dynamics hierarchically. As a consequence, the local and non-local dependencies, which contain more details and semantics respectively, are asynchronously captured in different level of layers. Moreover, existing methods are limited to the spatio-temporal domain and ignore information in the frequency domain. To better extract synchronous detailed and semantic information from multi-domains, we propose a residual frequency attention (rFA) block to focus on discriminative patterns in the frequency domain, and a synchronous local and non-local (SLnL) block to simultaneously capture the details and semantics in the spatio-temporal domain. In addition, to optimize the whole learning processes of the multi-branch network, we put it under a pseudo multi-task learning paradigm. During training, 1) a soft-margin focal loss (SMFL) is proposed to optimize the intra-branch separated learning process, which can automatically conduct data selection and encourage intrinsic margins in classifiers; 2) A mutual learning policy is also proposed to further facilitate the inter-branch collaborative learning process. Eventually, our approach achieves the state-of-the-art performance on several large-scale datasets for skeleton-based action recognition. |
关键词 | Skeleton-based Action Recognition Frequency Attention Synchronous Local and Non-local Learning Soft-margin Focal Loss Pesudo Multi-task Learning |
DOI | 10.1109/TMM.2019.2953325 |
关键词[WOS] | ENSEMBLE |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Key Research and Development Program of China[2017YFA0105203] ; Strategic Priority Research Program of the Chinese Academy of Sciences (CAS)[XDB32040200] ; Hundred-Talent Program of CAS |
项目资助者 | National Key Research and Development Program of China ; Strategic Priority Research Program of the Chinese Academy of Sciences (CAS) ; Hundred-Talent Program of CAS |
WOS研究方向 | Computer Science ; Telecommunications |
WOS类目 | Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications |
WOS记录号 | WOS:000562310200002 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
七大方向——子方向分类 | 医学影像处理与分析 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/40516 |
专题 | 脑图谱与类脑智能实验室_脑网络组研究 |
通讯作者 | Guyue, Hu |
作者单位 | 1.Chinese Acad Sci, Brainnetome Ctr, Inst Automat, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Natl Labo Pattern Recognit, Inst Automat, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 4.CAS Ctr Excellence Brain Sci & Intelligence Techn, Beijing 100190, Peoples R China |
推荐引用方式 GB/T 7714 | Guyue, Hu,Bo, Cui,Shan, Yu. Joint Learning in the Spatio-Temporal and Frequency Domains for Skeleton-Based Action Recognition[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2020,22(9):2207-2220. |
APA | Guyue, Hu,Bo, Cui,&Shan, Yu.(2020).Joint Learning in the Spatio-Temporal and Frequency Domains for Skeleton-Based Action Recognition.IEEE TRANSACTIONS ON MULTIMEDIA,22(9),2207-2220. |
MLA | Guyue, Hu,et al."Joint Learning in the Spatio-Temporal and Frequency Domains for Skeleton-Based Action Recognition".IEEE TRANSACTIONS ON MULTIMEDIA 22.9(2020):2207-2220. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
[TMM2020] Joint lear(4803KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论