Knowledge Commons of Institute of Automation,CAS
Multimodal Emotion Recognition with Transfer Learning of Deep Neural Network | |
Huang, Jian1,3![]() ![]() ![]() | |
发表期刊 | ZTE Communications
![]() |
2017-01 | |
卷号 | 15期号:S2页码:1673-5188 |
摘要 | Due to the lack of large scale emotion databases, it is hard to obtain comparable improvement in multimodal emotion recognition of the deep neural network by deep learning, which has made great progress in other areas. We use transfer learning to improve its performance with pre⁃trained models on large⁃scale data. Audio is encoded using deep speech recognition networks with 500 hours’speech and video is encoded using convolutional neural networks with over 110,000 images. The extracted audio and visual features are fed into Long Short⁃Term Memory to train models respectively. Logistic regression and ensemble method are performed in decision level fusion. The experiment results indicate that 1) audio features extracted from deep speech recognition networks achieve better performance than handcrafted audio features; 2) the visual emotion recognition obtains better performance than audio emotion recognition; 3) the ensemble method gets better performance than logistic regression and prior knowledge from micro F1 value further improves the performance and robustness, achieving accuracy of 67.00% for“happy”, 54.90% for“angry”, and 51.69% for“sad”. |
关键词 | deep neutral network ensemble method multimodal emotion recognition transfer learning |
语种 | 英语 |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/39298 |
专题 | 多模态人工智能系统全国重点实验室_智能交互 |
通讯作者 | Tao, Jianhua |
作者单位 | 1.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China 2.CAS Center for Excellence in Brain Science and Intelligence Technology, Beijing, China 3.School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China |
第一作者单位 | 模式识别国家重点实验室 |
通讯作者单位 | 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | Huang, Jian,Li, Ya,Tao, Jianhua,et al. Multimodal Emotion Recognition with Transfer Learning of Deep Neural Network[J]. ZTE Communications,2017,15(S2):1673-5188. |
APA | Huang, Jian,Li, Ya,Tao, Jianhua,&Yi, Jianyan.(2017).Multimodal Emotion Recognition with Transfer Learning of Deep Neural Network.ZTE Communications,15(S2),1673-5188. |
MLA | Huang, Jian,et al."Multimodal Emotion Recognition with Transfer Learning of Deep Neural Network".ZTE Communications 15.S2(2017):1673-5188. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
ZTE.pdf(828KB) | 期刊论文 | 出版稿 | 开放获取 | CC BY-NC-SA | 浏览 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论