Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units

doi:10.1109/ACCESS.2019.2947201

	Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units
	Zhang, Yong 1; Fan, Yanbo 1; Dong, Weiming2 ; Hu, Bao-Gang2 ; Ji, Qiang 3
发表期刊	IEEE ACCESS
ISSN	2169-3536
	2019
卷号	7 页码:150743-150756
通讯作者	Zhang, Yong(zhangyong201303@gmail.com)
摘要	Facial action units (AUs) are defined to depict movements of facial muscles, which are basic elements to encode facial expressions. Automatic AU intensity estimation is an important task in affective computing. Previous works leverage the representation power of deep neural networks (DNNs) to improve the performance of intensity estimation. However, a large number of intensity annotations are required to train DNNs that contain millions of parameters. But it is expensive and difficult to build a large-scale database with AU intensity annotation since AU annotation requires annotators have strong domain expertise. We propose a novel semi-supervised deep convolutional network that leverages extremely limited AU annotations for AU intensity estimation. It requires only intensity annotations of keyframes of training sequences. Domain knowledge on AUs is leveraged to provide weak supervisory information, including relative appearance similarity, temporal intensity ordering, facial symmetry, and contrastive appearance difference. We also propose a strategy to train a model for joint intensity estimation of multiple AUs under the setting of semi-supervised learning, which greatly improves the efficiency during inference. We perform empirical experiments on two public benchmark expression databases and make comparisons with state-of-the-art methods to demonstrate the effectiveness of the proposed method.
关键词	Gold Estimation Hidden Markov models Training Face Task analysis Neural networks Facial action units intensity estimation deep learning weakly supervised learning
DOI	10.1109/ACCESS.2019.2947201
关键词[WOS]	TRACKING ; MODEL
收录类别	SCI
语种	英语
资助项目	U.S National Science Foundation Award CNS[1629856] ; National Natural Science Foundation of China (NSFC)[61720106006] ; National Key Research and Development Program of China[2018YFC0807500] ; National Key Research and Development Program of China[2018YFC0807500] ; National Natural Science Foundation of China (NSFC)[61720106006] ; U.S National Science Foundation Award CNS[1629856]
项目资助者	National Key Research and Development Program of China ; National Natural Science Foundation of China (NSFC) ; U.S National Science Foundation Award CNS
WOS研究方向	Computer Science ; Engineering ; Telecommunications
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications
WOS记录号	WOS:000497163000031
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/29414
专题	多模态人工智能系统全国重点实验室_多媒体计算
通讯作者	Zhang, Yong
作者单位	1.Tencent AI Lab, Shenzhen 518057, Guangdong, Peoples R China 2.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 3.Rensselaer Polytech Inst, Dept Elect Comp & Syst Engn, Troy, NY 12180 USA
推荐引用方式 GB/T 7714	Zhang, Yong,Fan, Yanbo,Dong, Weiming,et al. Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units[J]. IEEE ACCESS,2019,7:150743-150756.
APA	Zhang, Yong,Fan, Yanbo,Dong, Weiming,Hu, Bao-Gang,&Ji, Qiang.(2019).Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units.IEEE ACCESS,7,150743-150756.
MLA	Zhang, Yong,et al."Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units".IEEE ACCESS 7(2019):150743-150756.