A multimodal approach of generating 3D human-like talking agent

	A multimodal approach of generating 3D human-like talking agent
	Yang, Minghao; Tao, Jianhua; Mu, Kaihui; Li, Ya; Che, Jianfeng
发表期刊	JOURNAL ON MULTIMODAL USER INTERFACES
	2012-03-01
卷号	5 期号:1-2 页码:61-68
文章类型	Article
摘要	This paper introduces a multimodal framework of generating a 3D human-like talking agent which can communicate with user through speech, lip movement, head motion, facial expression and body animation. In this framework, lip movements are obtained by searching and matching acoustic features which are represented by Mel-frequency cepstral coefficients (MFCC) in audio-visual bimodal database. Head motion is synthesized by visual prosody which maps textual prosodic features into rotational and translational parameters. Facial expression and body animation are generated by transferring motion data to skeleton. A simplified high level Multimodal Marker Language (MML), in which only a few fields are used to coordinate the agent channels, is introduced to drive the agent. The experiments validate the effectiveness of the proposed multimodal framework.
关键词	Multimodal 3d Talking Agent Lip Movement Head Motion Mfcc Facial Expression Gesture Animation
WOS标题词	Science & Technology ; Technology
关键词[WOS]	CHARACTER ANIMATION ; FACE
收录类别	SCI
语种	英语
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Cybernetics
WOS记录号	WOS:000309997800008
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/40914
专题	多模态人工智能系统全国重点实验室_智能交互
推荐引用方式 GB/T 7714	Yang, Minghao,Tao, Jianhua,Mu, Kaihui,et al. A multimodal approach of generating 3D human-like talking agent[J]. JOURNAL ON MULTIMODAL USER INTERFACES,2012,5(1-2):61-68.
APA	Yang, Minghao,Tao, Jianhua,Mu, Kaihui,Li, Ya,&Che, Jianfeng.(2012).A multimodal approach of generating 3D human-like talking agent.JOURNAL ON MULTIMODAL USER INTERFACES,5(1-2),61-68.
MLA	Yang, Minghao,et al."A multimodal approach of generating 3D human-like talking agent".JOURNAL ON MULTIMODAL USER INTERFACES 5.1-2(2012):61-68.