基于深度学习的多光谱人脸表情识别方法研究

CASIA OpenIR > 毕业生 > 硕士学位论文

	基于深度学习的多光谱人脸表情识别方法研究
	王影影1,2
	2018-05-25
学位类型	工程硕士
中文摘要	人脸表情识别就是让计算机识别人脸的各种表情，进而分析表情所代表的情绪、意图等，从而实现更加智能化的人机交互。该技术可以应用于安全驾驶、公共监控、机器人制造以及医学辅助诊断等领域，具有重要的研究意义。然而受光照变化的影响，使用单一的可见光图像无法实现全天候的人脸表情识别，针对该问题，本文研究基于多光谱图像的人脸表情识别，主要采用计算机视觉及深度学习技术，实现全天候的人脸表情监控。本文的主要工作和创新性成果包括：（1）建立了大规模的多光谱人脸表情数据库。设计并搭建多光谱人脸表情图像采集系统，采集明亮、昏暗、可见三种光照强度下的人脸表情图像。每种光照等级下均包含自然、愤怒、高兴、悲伤、惊讶、厌恶和恐惧七种表情，类别之间数量分别均衡。最终建立了共包含64460对可见光和近红外图像的大规模多光谱人脸表情数据库。与现有公开的数据库相比，该数据库数据量大、人脸表情更加自然、光照变化以及头部姿态更加丰富、对算法设计要求高，为基于多光谱图像的人脸表情识别算法的研究和验证提供了数据基础。（2）提出了基于多尺度卷积神经网络特征融合的人头检测算法。为了避免复杂背景的干扰，提取到更加细致的人脸表情特征，需要从整张图像中提取出人头区域。本文基于全卷积神经网络设计人头检测算法，并且融合多尺度的特征进行人头检测，通过实验验证了本算法对人头检测的有效性。（3）提出了基于多光谱图像的人脸表情识别框架。首先通过人头检测算法提取出近红外图像中的人头区域作为表情识别的输入图像，并将该图像作为输入通过生成对抗网络生成可用于表情识别的可见光图像；然后根据人头检测结果提取出可见光图像中的人头区域，并判断可见光图像采集时的光照等级，根据判断结果选择是否将真实的可见光图像用于表情识别；最后将两种或者三种类型的图像分别输入卷积神经网络模型中，在网络的决策层使用加权求和的方式得到最终的识别结果，在自建的多光谱人脸表情数据库上验证了该方法的有效性。（4）提出了基于低秩分解与卷积层合并的深度卷积神经网络加速算法。通过对卷积层的低秩分解降低卷积操作的计算复杂度，然后将符合约束条件的两个相邻的卷积层合并为一个卷积层，进一步降低卷积运算的复杂度，最后将该加速算法应用于表情识别网络，证明了加速算法的有效性。
英文摘要	Facial expression recognition (FER) is to make the computer to recognize kinds of expressions of faces, and then analysis emotions and intentions of them, thus to make the human-computer interaction more intelligent. This technology can be used in many fields such as safe driving, public surveillance, roboteer, medical auxiliary diagnosis, etc. However, affected by the illumination variations, it is impossible to recognize facial expression all-day only by using visible images. To solve this problem, we focus on the research of FER based on multi-spectral images. The basic technologies of this research are based on computer vision and deep learning. Main work and contributions are as following: (1) The multispectral facial expression database is built. The database is composed of near-infrared images and visible images collected by multi-spectral camera in different illumination intensity. The light environment was divided into light, media, dark, and each of them included seven emotions: nature, anger, happiness, sadness, surprise, disgust and fear. These emotions have a balanced distribution. Finally, a large scale multispectral facial expression database containing 64460 visible and near-infrared images is established. Compared with the existing database, multispectral facial expression database has more illumination changes and pose variations changes which makes it’s more difficult to design algorithms. It can be used to validate the accuracy of head detection algorithm and FER algorithm. (2) Person head detection algorithm based on multi-scale representation fusion of deep convolution neural network is proposed. In order to reduce the influence of back-ground and extract more detailed facial expression features, the head area needs to be extracted from the whole image firstly. A person head detection algorithm based on fully convolutional networks is proposed, which has been validated to be helpful to improve the detection accuracy in our experiment. (3) An effective framework of facial expression recognition based on multi-spectral images is proposed. Firstly, fake visible images is generate through GAN by using the head area of near-infrared image which is extracted by the head detection algorithm, two of these kinds images are used to recognize expressions. Secondly, extract the head area of visible image according to the detection result of near-infrared image and decide whether to use for FER. Finally, images chosen to recognize facial expressions are input into the convolution neural network and the weighted sum method is used to get the final result in the decision layer. This proposed framework has been proven effective in the multispectral facial expression database. (4) A model acceleration method based on low-rank decomposition and combination of convolutional layers is proposed. Using low-rank decomposition to reduce the complexity of convolutional networks. Then combine these convolutional layers who meet the conditions of constraint to reduce the number of operation. Finally, this acceleration algorithm is applied to facial expression recognition network. The experimental results illustrate the efficiency of the proposed algorithm.
关键词	深度学习多光谱人脸表情识别人头检测网络加速多光谱人脸表情数据库卷积神经网络
文献类型	学位论文
条目标识符	http://ir.ia.ac.cn/handle/173211/20991
专题	毕业生_硕士学位论文
作者单位	1.中国科学院大学 2.自动化研究所精密感知与控制研究中心
第一作者单位	精密感知与控制研究中心
推荐引用方式 GB/T 7714	王影影. 基于深度学习的多光谱人脸表情识别方法研究[D]. 北京. 中国科学院研究生院,2018.

条目包含的文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
王影影_硕士毕业论文_2015E8014（2504KB）	学位论文		限制开放	CC BY-NC-SA