CASIA OpenIR  > 复杂系统管理与控制国家重点实验室  > 先进机器人
Generalized zero-shot emotion recognition from body gestures
Wu, Jinting1,2; Zhang, Yujia1; Sun, Shiying1; Li, Qianzhong1,2; Zhao, Xiaoguang1
Source PublicationAPPLIED INTELLIGENCE
ISSN0924-669X
2021-11-01
Pages19
Abstract

In human-human interaction, body language is one of the most important emotional expressions. However, each emotion category contains abundant emotional body gestures, and basic emotions used in most researches are difficult to describe complex and diverse emotional states. It is costly to collect sufficient samples of all emotional expressions, and new emotions or new body gestures that are not included in the training set may appear during testing. To address the above problems, we design a novel mechanism that treats each emotion category as a collection of multiple body gesture categories to make better use of gesture information for emotion recognition. A Generalized Zero-Shot Learning (GZSL) framework is introduced to recognize both seen and unseen body gesture categories with the help of semantic information, and emotion predictions are further provided based on the relationship between gestures and emotions. This framework consists of two branches. The first branch is a Hierarchical Prototype Network (HPN) which learns the prototypes of body gestures and uses them to calculate the emotion attentive prototypes. This branch aims to obtain predictions on samples of the seen gesture categories. The second branch is a Semantic Auto-Encoder (SAE) which utilizes semantic representations to predict samples of unseen gesture categories. Thresholds are further trained to determine which branch result will be used during testing, and the emotion labels are finally obtained from these results. Comprehensive experiments are conducted on an emotion recognition dataset which contains skeleton data of multiple body gestures, and the performance of our framework is superior to both the traditional emotion classifier and state-of-the-art zero-shot learning methods.

KeywordGeneralized zero-shot learning Emotion recognition Body gesture recognition Prototype learning
DOI10.1007/s10489-021-02927-w
WOS KeywordCLASSIFICATION ; MOVEMENT ; NETWORK
Indexed BySCI
Language英语
Funding ProjectNational Key Research and Development Project of China[2019YFB1310601] ; National Key R&D Program of China[2017YFC0820203] ; National Natural Science Foundation of China[62103410]
Funding OrganizationNational Key Research and Development Project of China ; National Key R&D Program of China ; National Natural Science Foundation of China
WOS Research AreaComputer Science
WOS SubjectComputer Science, Artificial Intelligence
WOS IDWOS:000713535200001
PublisherSPRINGER
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/46354
Collection复杂系统管理与控制国家重点实验室_先进机器人
Corresponding AuthorZhang, Yujia
Affiliation1.Institute of Automation, Chinese Academy of Sciences, Beijing, China
2.School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Wu, Jinting,Zhang, Yujia,Sun, Shiying,et al. Generalized zero-shot emotion recognition from body gestures[J]. APPLIED INTELLIGENCE,2021:19.
APA Wu, Jinting,Zhang, Yujia,Sun, Shiying,Li, Qianzhong,&Zhao, Xiaoguang.(2021).Generalized zero-shot emotion recognition from body gestures.APPLIED INTELLIGENCE,19.
MLA Wu, Jinting,et al."Generalized zero-shot emotion recognition from body gestures".APPLIED INTELLIGENCE (2021):19.
Files in This Item: Download All
File Name/Size DocType Version Access License
Generalized zero-sho(2059KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wu, Jinting]'s Articles
[Zhang, Yujia]'s Articles
[Sun, Shiying]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wu, Jinting]'s Articles
[Zhang, Yujia]'s Articles
[Sun, Shiying]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wu, Jinting]'s Articles
[Zhang, Yujia]'s Articles
[Sun, Shiying]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: Generalized zero-shot emotion recognition from body gestures.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.