CASIA OpenIR  > 毕业生  > 博士学位论文
Robust structural feature learning based facial expression recognition
Jain Deepak Kumar
学位类型工程博士
导师Kaiqi Huang
2018
学位授予单位中国科学院研究生院
学位授予地点北京
关键词Facial Expressions Pattern Analysis Random Walk Multi- Angle Properties Micro-expression Facial Action Points
摘要
Facial expression analysis and recognition are one of the most fast developing research areas due to its wide range of applications such as emotion analysis, biometrics, image retrieval etc. Facial expression recently becomes one of the most interesting subjects that face some challenges e.g., multiple illuminations, orientations and numerous variations. In general context, variant methods show different requirements and hinges to develop new models. These methods handle the post-processing and pre-processing of the facial images. The pre-processing models include the detection and extraction of face images using normalization schemes whereas post-processing models extract the relevant features for finding the facial expressions.
For the above challenging problems, it is essential to enhance the robustness of facial expression recognition model with structured features and feature learning to provide effective and discriminative features for facial expression recognition. In the thesis, we proposed three approaches which is divided into three phases:
 The first phase designs a novel hybrid patch based diagonal pattern geometric appearance models which cover efficient feature extraction and co-training labels to increase the accuracy of our model. For the given input RGB-D images, the features are extracted using Geometric Appearance Models (GAM). The selected features are then processed onto Patch-Based Diagonal Pattern (PBDP). At last, Relevance Vector Machine (RVM) is used for classifying the facial expressions with labels neutral or smile. The proposed model is tested on two public datasets (EURECOMM and biographer dataset). Performance metrics such as sensitivity, specificity, precision, recall, jaccard coefficient, dice overlap, kappa coefficient and accuracy were studied. The proposed classifier RVM is compared with the radial basis function kernel(RBF) SVM and other recent approaches which proves that the proposed classifier significantly reduced the error rate. Moreover, the proposed model as compared to RBF kernel based SVM has very low cost, training time and less number of mathematical operations.
 
The second phase concentrates on developing a multi-angle based optimal pattern deep convolution neural network for automatic facial expression recognition systems. The framework has aimed at resolving the projection of complex 3D actions on the image plane and inaccurate class alignment. In specific, sudden illumination changes of images are rectified from multi-angle based optimal configurations. The whole process is carried out in five stages, namely, Extended Boundary Background Subtraction (EBBS), Multi- angle Texture Pattern with spatio temporal matching, Densely Extracted SURF with Local Occupancy Pattern (LOP), Priority Particle Cuckoo Search Optimization and Long Short- Term Memory Convolutional Neural Network. Each process has a unique model which accurately predicts the relevant features to have better classification systems. The proposed algorithm is analyzed on CK+ and MMI datasets with the parameters of recognition rate and F1 measure. Each process has unique model which accurately predicts the relevant features for better classification system.However,the noise removal through the EBBS and novel feature extraction based on the multi-angle texture pattern contributed towards the illumination variation handling and the matching accuracy effectively.
 
 
The final phase concentrates on micro-expressions.The challenging limitations with macro-expression having low performance in involuntary expression handling, and recognition of muscle variations,Due to these limitations observed,we will focus on microexpression to solve the difficulties such as short durations and rapid spontaneous facial expression which are induced due to the detection and analysis of the micro-expression.To face and solve these challenges we proposed a framework to detect spontaneous microexpression clips temporally from a video sequence in this final phase.In this frame work,three techniques are studied Active Shape Modeling (ASM), Random Walk (RW) and the Artificial Neural Network (ANN).To describe the geometric shape of human face.we proposed active shape model using procrustes analysis which are robust to the head movement and light variation.After that random walk model is presented to adopted the features and expressed into ANN model.The integration of random walk with ANN enhance the learning performance of micro-expression. Experimental validation on two spontaneous micro-expression database:Chinese Academy of Sciences micro-Expression (CASME) and Spontaneous micro-expression (SMIC) with the parameters of accuracy, true positive rate, and ROC which proves the effectiveness of the proposed algorithm in terms of accurate detection of facial points with the help of active shape model with procrustes analysis and achieve better recognition rate than existing nonlinear classifiers.
;
Facial expression analysis and recognition are one of the most fast developing research areas due to its wide range of applications such as emotion analysis, biometrics, image retrieval etc. Facial expression recently becomes one of the most interesting subjects that face some challenges e.g., multiple illuminations, orientations and numerous variations. In general context, variant methods show different requirements and hinges to develop new models. These methods handle the post-processing and pre-processing of the facial images. The pre-processing models include the detection and extraction of face images using normalization schemes whereas post-processing models extract the relevant features for finding the facial expressions.
For the above challenging problems, it is essential to enhance the robustness of facial expression recognition model with structured features and feature learning to provide effective and discriminative features for facial expression recognition. In the thesis, we proposed three approaches which is divided into three phases:
 The first phase designs a novel hybrid patch based diagonal pattern geometric appearance models which cover efficient feature extraction and co-training labels to increase the accuracy of our model. For the given input RGB-D images, the features are extracted using Geometric Appearance Models (GAM). The selected features are then processed onto Patch-Based Diagonal Pattern (PBDP). At last, Relevance Vector Machine (RVM) is used for classifying the facial expressions with labels neutral or smile. The proposed model is tested on two public datasets (EURECOMM and biographer dataset). Performance metrics such as sensitivity, specificity, precision, recall, jaccard coefficient, dice overlap, kappa coefficient and accuracy were studied. The proposed classifier RVM is compared with the radial basis function kernel(RBF) SVM and other recent approaches which proves that the proposed classifier significantly reduced the error rate. Moreover, the proposed model as compared to RBF kernel based SVM has very low cost, training time and less number of mathematical operations.
 
The second phase concentrates on developing a multi-angle based optimal pattern deep convolution neural network for automatic facial expression recognition systems. The framework has aimed at resolving the projection of complex 3D actions on the image plane and inaccurate class alignment. In specific, sudden illumination changes of images are rectified from multi-angle based optimal configurations. The whole process is carried out in five stages, namely, Extended Boundary Background Subtraction (EBBS), Multi- angle Texture Pattern with spatio temporal matching, Densely Extracted SURF with Local Occupancy Pattern (LOP), Priority Particle Cuckoo Search Optimization and Long Short- Term Memory Convolutional Neural Network. Each process has a unique model which accurately predicts the relevant features to have better classification systems. The proposed algorithm is analyzed on CK+ and MMI datasets with the parameters of recognition rate and F1 measure. Each process has unique model which accurately predicts the relevant features for better classification system.However,the noise removal through the EBBS and novel feature extraction based on the multi-angle texture pattern contributed towards the illumination variation handling and the matching accuracy effectively.
 
 
The final phase concentrates on micro-expressions.The challenging limitations with macro-expression having low performance in involuntary expression handling, and recognition of muscle variations,Due to these limitations observed,we will focus on microexpression to solve the difficulties such as short durations and rapid spontaneous facial expression which are induced due to the detection and analysis of the micro-expression.To face and solve these challenges we proposed a framework to detect spontaneous microexpression clips temporally from a video sequence in this final phase.In this frame work,three techniques are studied Active Shape Modeling (ASM), Random Walk (RW) and the Artificial Neural Network (ANN).To describe the geometric shape of human face.we proposed active shape model using procrustes analysis which are robust to the head movement and light variation.After that random walk model is presented to adopted the features and expressed into ANN model.The integration of random walk with ANN enhance the learning performance of micro-expression. Experimental validation on two spontaneous micro-expression database:Chinese Academy of Sciences micro-Expression (CASME) and Spontaneous micro-expression (SMIC) with the parameters of accuracy, true positive rate, and ROC which proves the effectiveness of the proposed algorithm in terms of accurate detection of facial points with the help of active shape model with procrustes analysis and achieve better recognition rate than existing nonlinear classifiers.

文献类型学位论文
条目标识符http://ir.ia.ac.cn/handle/173211/21197
专题毕业生_博士学位论文
作者单位Institute of Automation,Chinese Academy of Sciences,Beijing,China
推荐引用方式
GB/T 7714
Jain Deepak Kumar. Robust structural feature learning based facial expression recognition[D]. 北京. 中国科学院研究生院,2018.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
Thesis.pdf(6412KB)学位论文 暂不开放CC BY-NC-SA请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Jain Deepak Kumar]的文章
百度学术
百度学术中相似的文章
[Jain Deepak Kumar]的文章
必应学术
必应学术中相似的文章
[Jain Deepak Kumar]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。