Music Theory-Inspired Acoustic Representation for Speech Emotion Recognition

doi:10.1109/TASLP.2023.3289312

	Music Theory-Inspired Acoustic Representation for Speech Emotion Recognition
	Li, Xingfeng 1; Shi, Xiaohan 2; Hu, Desheng 3; Li, Yongwei 4; Zhang, Qingchen 1; Wang, Zhengxia 5; Unoki, Masashi 6; Akagi, Masato 6
发表期刊	IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING
ISSN	2329-9290
	2023
卷号	31 页码:2534-2547
通讯作者	Li, Xingfeng(lixingfeng@hainanu.edu.cn)
摘要	This research presents a music theory-inspired acoustic representation (hereafter, MTAR) to address improved speech emotion recognition. The recognition of emotion in speech and music is developed in parallel, yet a relatively limited understanding of MTAR for interpreting speech emotions is involved. In the present study, we use music theory to study representative acoustics associated with emotion in speech from vocal emotion expressions and auditory emotion perception domains. In experiments assessing the role and effectiveness of the proposed representation in classifying discrete emotion categories and predicting continuous emotion dimensions, it shows promising performance compared with extensively used features for emotion recognition based on the spectrogram, Mel-spectrogram, Mel-frequency cepstral coefficients, VGGish, and the large baseline feature sets of the INTERSPEECH challenges. This proposal opens up a novel research avenue in developing a computational acoustic representation of speech emotion via music theory.
关键词	Affective computing speech emotion recognition acoustic representation music theory and speech analysis
DOI	10.1109/TASLP.2023.3289312
关键词[WOS]	PERCEPTION ; EXPRESSION ; PATTERNS ; FEATURES ; PITCH ; PERSPECTIVE ; MODALITIES ; KNOWLEDGE ; INTERVALS ; COGNITION
收录类别	SCI
语种	英语
资助项目	Key Research and Development Program of Hainan Province[ZDYF2021GXJS017] ; National Natural Science Foundation of China[82160345] ; National Natural Science Foundation of China[62201571] ; Key Science and Technology Plan Project of Haikou[2021-016]
项目资助者	Key Research and Development Program of Hainan Province ; National Natural Science Foundation of China ; Key Science and Technology Plan Project of Haikou
WOS研究方向	Acoustics ; Engineering
WOS类目	Acoustics ; Engineering, Electrical & Electronic
WOS记录号	WOS:001025466100003
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/53769
专题	多模态人工智能系统全国重点实验室_智能交互
通讯作者	Li, Xingfeng
作者单位	1.Hainan Univ, Grad Sch Comp Sci & Technol, Haikou 570288, Peoples R China 2.Nagoya Univ, Sch Informat Sci, Nagoya 4648601, Japan 3.Taiyuan Univ Technol, Coll Informat & Comp, Taiyuan 030024, Peoples R China 4.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 5.Hainan Univ, Sch Comp Sci & Technol, Haikou 570288, Peoples R China 6.Japan Adv Inst Sci & Technol, Sch Informat Sci, Nomi 9231292, Japan
推荐引用方式 GB/T 7714	Li, Xingfeng,Shi, Xiaohan,Hu, Desheng,et al. Music Theory-Inspired Acoustic Representation for Speech Emotion Recognition[J]. IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING,2023,31:2534-2547.
APA	Li, Xingfeng.,Shi, Xiaohan.,Hu, Desheng.,Li, Yongwei.,Zhang, Qingchen.,...&Akagi, Masato.(2023).Music Theory-Inspired Acoustic Representation for Speech Emotion Recognition.IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING,31,2534-2547.
MLA	Li, Xingfeng,et al."Music Theory-Inspired Acoustic Representation for Speech Emotion Recognition".IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 31(2023):2534-2547.