Human emotions are mental states of feelings that are exposed unconsciously and followed by physical changes in their facial muscles which entail the expressions on the face. Certain emotions commonly expressed by human are happiness, sadness, anger, fear, disgust, surprise, and neutral. For a non-verbal communication, facial expression plays a vital role since it appears because of inner core feelings of a person that reflects on the faces. For the automatic recognition of facial emotions, many methods are used such as Artificial Neural networks, Neuro-fuzzy, Wavelet transformation, etc. However, the existing methods take more time for data classification, low accuracy in the optimization process and high level of error rate. To overcome these concerns, this paper depicts an amphibious operation of Multi Support Vector Machine (SVM) with the Convolutional Neural Networks (CNN). Initially, the characteristics of the pre-processed face image are efficiently extracted by using Local Binary Pattern (LBP), Principal Component Analysis (PCA) and Gray Level Occurrence Matrix (GLCM). In this model, CNN works as a trainable feature extractor, and Multi-SVM performs as a recognizer. The proposed system`s performance is analyzed with various human faces using the MATLAB tool. The results prove that the proposed method surpasses the earlier methods regarding high accuracy with low computation time and low error rate.

Convolutional Neural Networks, Face recognition, Local Binary Pattern, Principal Component Analysis

[1] C. Darwin, The Expression of the Emotions in Man and Animals. D. Appleton and Company, 1899.
[2] D. Keltner and P. Ekman, Handbook of Emotions, ch. 15 - Facial Expression of Emotion, pp. 151–249. Guilford Publications, Inc., 2nd ed., 2000.
[3] E. M. Provost, Y. Shangguan, and C. Busso, "Umeme: University of Michigan emotional McGurk effect data set," IEEE Transactions on Affective Computing, vol. 6, pp. 395–409, Oct 2015.
[4] R. E. Jack, O. G. Garrod, and P. G. Schyns, “Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time,” Current Biology, vol. 24, no. 2, pp. 187 – 192, 2014.
[5] Lopes, A. T., de Aguiar, E., De Souza, A. F., & Oliveira-Santos, T. (2017). Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recognition, 61, 610-628.
[6] R. E. Jack, O. G. Garrod, and P. G. Schyns, “Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time,” Current Biology, vol. 24, no. 2, pp. 187 – 192, 2014.
[7] M. W. Schurgin, J. Nelson, S. Iida, H. Ohira, J. Y. Chiao, and S. L. Franconeri, “Eye movements during emotion recognition in faces,” Journal of Visualization, vol. 14, no. 13, 2014.
[8] I. Goodfellow, Y. Bengio, and A. Courville, “Deep learning.” Book in preparation for MIT Press, 2016.
[9] Pilla Jr, V., Zanellato, A., Bortolini, C., Gamba, H. R., Borba, G. B., & Medeiros, H. Facial Expression Classification Using Convolutional Neural Network and Support Vector Machine.
[10] Azcarate, A., Hageloh, F., Van de Sande, K., & Valenti, R. (2005). Automatic facial emotion recognition. Universiteit van Amsterdam, 1-6.
[11] Cohen, I., Garg, A., & Huang, T. S. (2000, November). Emotion recognition from facial expressions using multilevel HMM. In Neural information processing systems (Vol. 2).
[12] Bartlett, M. S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., & Movellan, J. (2006, April). Fully automatic facial action recognition in spontaneous behavior. In Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on (pp. 223-230). IEEE.
[13] Shan, C., Gong, S., & McOwan, P. W. (2009). Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing, 27(6), 803-816.
[14] Walecki, R.; Rudovic, O. Deep structured learning for facial expression intensity estimation. Image Vis. Comput. 2017, 259, 143–154.
[15] Al-Shabi, M., Cheah, W. P., & Connie, T. (2016). Facial Expression Recognition Using a Hybrid CNN-SIFT Aggregator. arXiv preprint arXiv:1608.02833.
[16] Hasani, B., & Mahoor, M. H. (2017, May). Spatio-temporal facial expression recognition using convolutional neural networks and conditional random fields. In Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference on (pp. 790-795). IEEE.
[17] Yang, B., & Chen, S. (2013). A comparative study on the local binary pattern (LBP) based face recognition: LBP histogram versus LBP image. Neurocomputing, 120, 365-379.
[18] Chamasemani, F. F., & Singh, Y. P. (2011, September). Multi-class support vector machine (SVM) classifiers--an application in hypothyroid detection and classification. In Bio-Inspired Computing: Theories and Applications (BIC-TA), 2011 Sixth International Conference on (pp. 351-356). IEEE.