Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition

doi:10.1109/TCYB.2017.2756840

	Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition
	Cao, Congqi1,2 ; Zhang, Yifan1,2 ; Zhang, Chunjie2,3 ; Lu, Hanqing1,2
发表期刊	IEEE TRANSACTIONS ON CYBERNETICS
	2018-03-01
卷号	48 期号:3 页码:1095-1108
文章类型	Article
摘要	3-D convolutional neural networks (3-D CNNs) have been established as a powerful tool to simultaneously learn features from both spatial and temporal dimensions, which is suitable to be applied to video-based action recognition. In this paper, we propose not to directly use the activations of fully connected layers of a 3-D CNN as the video feature, but to use selective convolutional layer activations to form a discriminative descriptor for video. It pools the feature on the convolutional layers under the guidance of body joint positions. Two schemes of mapping body joints into convolutional feature maps for pooling are discussed. The body joint positions can be obtained from any off-the-shelf skeleton estimation algorithm. The helpfulness of the body joint guided feature pooling with inaccurate skeleton estimation is systematically evaluated. To make it end-to-end and do not rely on any sophisticated body joint detection algorithm, we further propose a two-stream bilinear model which can learn the guidance from the body joints and capture the spatio-temporal features simultaneously. In this model, the body joint guided feature pooling is conveniently formulated as a bilinear product operation. Experimental results on three real-world datasets demonstrate the effectiveness of body joint guided pooling which achieves promising performance.
关键词	Action Recognition Body Joints Convolutional Networks Feature Pooling Two-stream Bilinear Model
WOS标题词	Science & Technology ; Technology
DOI	10.1109/TCYB.2017.2756840
关键词[WOS]	CLASSIFICATION ; TRAJECTORIES ; HISTOGRAMS ; PARTS
收录类别	SCI
语种	英语
项目资助者	National Natural Science Foundation of China(61332016 ; Youth Innovation Promotion Association CAS ; 61572500)
WOS研究方向	Automation & Control Systems ; Computer Science
WOS类目	Automation & Control Systems ; Computer Science, Artificial Intelligence ; Computer Science, Cybernetics
WOS记录号	WOS:000424826800022
引用统计	被引频次：47[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/20410
专题	紫东太初大模型研究中心_图像与视频分析
作者单位	1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Automat, Res Ctr Brain Inspired Intelligence, Beijing 100190, Peoples R China
第一作者单位	模式识别国家重点实验室
推荐引用方式 GB/T 7714	Cao, Congqi,Zhang, Yifan,Zhang, Chunjie,et al. Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition[J]. IEEE TRANSACTIONS ON CYBERNETICS,2018,48(3):1095-1108.
APA	Cao, Congqi,Zhang, Yifan,Zhang, Chunjie,&Lu, Hanqing.(2018).Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition.IEEE TRANSACTIONS ON CYBERNETICS,48(3),1095-1108.
MLA	Cao, Congqi,et al."Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition".IEEE TRANSACTIONS ON CYBERNETICS 48.3(2018):1095-1108.

条目包含的文件		下载所有文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
曹聪琦_T-Cybernetics_Bo（1731KB）	期刊论文	作者接受稿	开放获取	CC BY-NC-SA	浏览下载