首页 > 论文 > 激光与光电子学进展 > 55卷 > 1期(pp:11002--1)

融合局部特征与深度置信网络的人脸表情识别

Facial Expression Recognition Based on Fusion of Local Features and Deep Belief Network

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

针对传统人脸表情识别(FER)方法所提取的表情特征较为单一, 同时对于表情分类器的选择存在局限性的问题, 提出一种融合局部特征与深度置信网络(DBN)的FER方法。该方法首先从人脸表情图像中切割出眉毛眼睛部位与嘴巴部位这2种包含丰富表情信息的局部表情图像, 对其分别提取包含纹理信息的Log-Gabor特征与包含形状信息的二阶梯度方向直方图特征, 并将这2种特征相融合, 获得更有效的表情特征, 然后利用融合后的特征训练DBN模型, 并用训练后的DBN模型进行表情识别。利用本文方法在三种表情库上进行实验, 识别率可分别达到96.30%、97.39%以及95.73%, 表明本文方法可有效提高人脸表情识别率。

Abstract

The traditional facial expression recognition (FER) methods only extract single expression feature. Meanwhile, the choice of expression classifiers has limitations. To solve these problems, we propose a FER method based on the fusion of local features and deep belief network (DBN). Firstly, the eyebrows and eyes part and mouth part with rich expression information are extracted as local expression images. In order to attain more effective expression features, the Log-Gabor features with texture information and second-order histogram of gradient direction features with shape information are extracted and fused from local expression images. DBN model is trained with fusion features. The trained DBN model is used to recognize the facial expression. The experimental results show that the recognition rates of the proposed method on three databases are 96.30%, 97.39% and 95.73%. The proposed method effectively improves the recognition rate of facial expression.

Newport宣传-MKS新实验室计划
补充资料

中图分类号:TP391

DOI:10.3788/lop55.011002

所属栏目:图像处理

基金项目:国家自然科学基金(61571323)

收稿日期:2017-05-27

修改稿日期:2017-08-07

网络出版日期:--

作者单位    点击查看

王琳琳:天津大学电气自动化与信息工程学院, 天津 300072
刘敬浩:天津大学电气自动化与信息工程学院, 天津 300072
付晓梅:天津大学海洋科学与技术学院, 天津 300072

联系人作者:王琳琳(wanglinlin@tju.edu.cn)

备注:王琳琳(1992-), 女, 硕士研究生, 主要从事模式识别与机器学习方面的研究。E-mail: wanglinlin@tju.edu.cn

【1】Chen J, Takiguchi T, Ariki Y. Facial expression recognition with multithreaded cascade of rotation-invariant HOG[C]. International Conference on Affective Computing and Intelligent Interaction, IEEE, 2015: 636-642.

【2】Negri P, Acevedo D, Mejail M, et al. Facial expression recognition: A comparison between static and dynamic approaches[C]. International Conference on Pattern Recognition Systems, 2016: 1-6.

【3】Liu S S, Tian Y T, Wan C, et al. Facial expression recognition method based on Gabor multi-orientation features fusion and block histogram[J]. Acta Automatica Sinica, 2011, 37(12): 1455-1463.
刘帅师, 田彦涛, 万川. 基于Gabor多方向特征融合与分块直方图的人脸表情识别方法[J]. 自动化学报, 2011, 37(12): 1455-1463.

【4】Ye Z, Bai L, Nian Y J. Hyperspectral image classification algorithm based on Gabor feature and locality-preserving dimensionality reduction[J]. Acta Optica Sinica, 2016, 36(10): 1028003.
叶珍, 白璘, 粘永健. 基于Gabor特征与局部保护降维的高光谱图像分类算法[J]. 光学学报, 2016, 36(10): 1028003.

【5】Liu Y, Wu X J. Image recognition algorithm based on Log-Gabor wavelet and Riemannian manifold learning[J]. Pattern Recognition and Artificial Intelligence, 2015, 28(10): 946-952.
刘元, 吴小俊. 基于Log-Gabor滤波与黎曼流形学习的图像识别算法[J]. 模式识别与人工智能, 2015, 28(10): 946-952.

【6】Yuan W Q, Fan Y G, Ke L. Palmprints recognition method based on the phase consistency combined with Log-Gabor filter[J]. Acta Optica Sinica, 2010, 30(1): 147-152.
苑玮琦, 范永刚, 柯丽. 相位一致性和对数Gabor滤波器相结合的掌纹识别方法[J]. 光学学报, 2010, 30(1): 147-152.

【7】Liu B, Zhang X, Hu C H, et al. Gesture recognition method of hand over face occlusion in color and depth images[J]. Laser & Optoelectronic Progress, 2016, 53(6): 061001.
刘斌, 赵兴, 胡春海, 等. 面向颜色深度图像手脸近距遮挡的手势识别[J]. 激光与光电子学进展, 2016, 53(6): 061001.

【8】Cao H, Yamaguchi K, Naito, et al. Pedestrian recognition using second-order HOG feature[C]. 9th Asian Conference on Computer Vision (ACCV 2009), 2009: 628-634.

【9】Zhang S H, He H, Kong L F. Fusing multi-feature for video occlusion region detection based on graph cut[J]. Acta Optica Sinica, 2015, 35(4): 0415001.
张世辉, 何欢, 孔令富. 融合多特征基于图割实现视频遮挡区域检测[J]. 光学学报, 2015, 35(4): 0415001.

【10】Liu Y, Li Y, Ma X, et al. Facial expression recognition with fusion features extracted from salient facial areas[J]. Sensors, 2017, 17(4): 712.

【11】Zhao L, Wang Z, Zhang G. Facial expression recognition from video sequences based on spatial-temporal motion local binary pattern and Gabor multiorientation fusion histogram[J]. Mathematical Problems in Engineering, 2017, 2017: 1-12.

【12】Jia Q, Gao X, Guo H, et al. Multi-layer sparse representation for weighted LBP-patches based facial expression recognition[J]. Sensors, 2015, 15(3): 6719-6739.

【13】Hu M, Jiang H, Wang X H, et al. Precise local feature description for facial expression recognition[J]. Journal of Image and Graphics, 2014, 19(11): 1613-1622.
胡敏, 江河, 王晓华, 等. 精确局部特征描述的表情识别[J]. 中国图象图形学报, 2014, 19(11): 1613-1622.

【14】Li Y, Mavadati S M, Mahoor M H, et al. A unified probabilistic framework for measuring the intensity of spontaneous facial action units[C]. IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, 2013: 1-7.

【15】Wang Q, Jia K, Liu P. Design and implementation of remote facial expression recognition surveillance system based on PCA and KNN algorithms[C]. International Conference on Intelligent Information Hiding and Multimedia Signal Processing. 2016: 314-317.

【16】Siddiqi M H, Ali R, Idris M, et al. Human facial expression recognition using curvelet feature extraction and normalized mutual information feature selection[J]. Multimedia Tools and Applications, 2016, 75(2): 935-959.

【17】Liu D W, Han L, Han X Y. High spatial resolution remote sensing image classification based on deep learning[J]. Acta Optica Sinica, 2016, 36(4): 0428001.
刘大伟, 韩玲, 韩晓勇. 基于深度学习的高分辨率遥感影像分类研究[J]. 光学学报, 2016, 36(4): 0428001.

【18】Hinton G E, Osindero S, Teh Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7): 1527-1554.

【19】Lv Y, Feng Z, Xu C. Facial expression recognition via deep learning[C]. International Conference on Smart Computing, IEEE, 2015: 303-308.

【20】Liu P, Han S, Meng Z, et al. Facial expression recognition via a boosted deep belief network[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2014: 1805-1812.

【21】Xia J, Pei D, Wang Q Z, et al. Face recognition based on local adaptive ternary derivative pattern coupled with Gabor feature[J]. Laser & Optoelectronics Progress, 2016, 53(11): 111004.
夏军, 裴东, 王全州, 等. 融合Gabor特征的局部自适应三值微分模式的人脸识别[J]. 激光与光电子学进展, 2016, 53(11): 111004.

【22】Zhang Y, Liao Q Z, Luo Y. Head pose estimation fusing the second order HOG and CS-LBP[J]. CAAI Transactions on Intelligent Systems, 2015, 10(5): 741-746.
张毅, 廖巧珍, 罗元. 融合二阶HOG与CS-LBP的头部姿态估计[J]. 智能系统学报, 2015, 10(5): 741-746.

【23】Hinton G E. Training products of experts by minimizing contrastive divergence[J]. Neural Computation, 2002, 14(8): 1771-1800.

【24】Happy S L, Routray A. Robust facial expression classification using shape and appearance features[C]. Eighth International Conference on Advances in Pattern Recognition, 2015: 1-5.

【25】Gu W, Xiang C, Venkatesh Y V, et al. Facial expression recognition using radial encoding of local Gabor features and classifier synthesis[J]. Pattern Recognition, 2012, 45(1): 80-91.

【26】Rivera A R, Castillo J R, Chae O. Local directional number pattern for face analysis: face and expression recognition[J]. IEEE Transactions on Image Processing, 2013, 22(5): 1740-1752.

【27】Ghimire D, Lee J. Extremelearning machine ensemble using bagging for facial expression recognition[J]. Journal of Information Processing Systems, 2014, 10(3): 443-458.

【28】Zhong L, Liu Q, Yang P, et al. Learning multiscale active facial patches for expression analysis[J]. IEEE Transactions on Cybernetics, 2015, 45(8): 1499-1510.

【29】Ahmed F, Bari H, Hossain E. Person-independent facial expression recognition based on compound local binary pattern (CLBP)[J]. International Arab Journal of Information Technology, 2014, 11(2): 195-203.

【30】Li Y Q, Li Y J, Li H B, et al. Fusion of global and local various feature for facial expression recognition[J]. Atca Optica Sinica, 2014, 34(5): 0515001.
李雅倩, 李颖杰, 李海滨, 等. 融合全局与局部多样性特征的人脸表情识别[J]. 光学学报, 2014, 34(5): 0515001.

【31】Ghimire D, Jeong S, Yoon S, et al. Facial expression recognition based on region specific appearance and geometric features[C]. Tenth International Conference on Digital Information Management, IEEE, 2016: 142-147.

引用该论文

Wang Linlin,Liu Jinghao,Fu Xiaomei. Facial Expression Recognition Based on Fusion of Local Features and Deep Belief Network[J]. Laser & Optoelectronics Progress, 2018, 55(1): 011002

王琳琳,刘敬浩,付晓梅. 融合局部特征与深度置信网络的人脸表情识别[J]. 激光与光电子学进展, 2018, 55(1): 011002

被引情况

【1】陈龙,庞彦伟. 一种上下文敏感的多尺度人脸检测方法. 激光与光电子学进展, 2019, 56(4): 41003--1

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF