Open Access Open Access  Restricted Access Subscription or Fee Access

A Novel Approach for Bimodal Human Emotion Recognition

Swagata Sarkar, H. Ranganathan

Abstract


In this paper six basic (Fear, Angry, Happy, Neutral, Sad, Surprise) human emotions are recognized by bimodal fusion techniques. The emotion features are extracted both from facial images and speech signal. All the features collected from both the domain are fused using Discrete Wavelet Transform and Logistic Regression Model to generate single feature set. Both the fused feature set classified by Back Propagation Network. It is observed that the accuracy ofDiscrete Wavelet Transform is 91.67% which is better than Logistic Regression of accuracy 89.17%.


Keywords


Artificial Neural Network, Back Propagation Network, Discrete Wavelet Transform, Human Emotion, Logistic Regression

Full Text:

PDF

References


G. Castellano, L. Kessous, “Multimodal emotion recognition from expressive faces, body gestures and speech”, 2nd International Conference on Affective Computing and Intelligent Interaction, Lisbon, September 2007.

J. Kim, “Bimodal emotion recognition using speech and physiological changes”, Robust Speech Rcognition and Understanding, I-Tech Education and Publishing, Vienna, 2007.

Sebe, I. Cohen, T. Gevers, T.S. Huang, “Multimodal Approaches for Emotion Recognition: A Survey” Internet Imaging VI, SPIE’05, San Jose, USA, January 2005.

C. Busso, Z. Deng, et al, “Analysis of emotion recognition using facial expressions, speech and multimodal information”, Proceedings of the 6th international conference on Multimodal interfaces, State College, USA, 2004

Chen, L.S., Huang, T.S. “Emotional expressions in audiovisual human computer interaction“. Multimedia and Expo, ICME 2000.

Vapnik, V. “The Nature of Statistical Learning Theory”, New York, NY: Springer-Verlag, 1995

Ekman, P., Friesen, W. V. Facial Action Coding System: A Technique for Measurement of Facial Movement. Consulting Psychologists Press Palo Alto, California, 1978.

Lee, Chul Min et al. "Emotion recognition based on phoneme classes", In Interspeech 2004, pg 889-892.

S. Mallat, “A wavelet tour of signal processing”, New York, Academic Press, 1999

M.K Hu, “Visual pattern recognition by moment invariants”, IRE Trans.Inf. Theory, It-8, 1962, pp.179-187.

AgataKołakowska, Agnieszka Landowska, MariuszSzwoch, WioletaSzwoch, Michał R. Wróbel Gdansk .2013.Emotion Recognition and its application in software engineering.IEEE Explore Digital Library.

P. M. Chavan, Manan C .Jadhav, Jinal B. Mashruwala, Aditi K. Nehete and Pooja A. Panjari .2013. Real Time Emotion Recognition through Facial Expressions for Desktop Devices (IJESE) India ISSN: 2319–6378, Vol 1, Issue 7

H. Gunes and M. Piccardi. Bi-modal emotion recognition from expressive face and body gestures. Journal of Network and Computer Applications, 30:1334–1345, 2007.

H. Gunes and M. Piccardi. A bimodal face and body gesture database for automatic analysis of human nonverbal affective behavior. In Proc. of ICPR 2006 the 18th International Conference on Pattern Recognition, Hong Kong, China, November 2006

T. Banziger, H. Pirker, and K. Scherer. Gemep - Geneva multimodal emotion portrayals: A corpus for the study of multimodal emotional expressions. In In L. Deviller et al. (Ed.), Proceedings of LREC’06 Workshop on Corpora for Research on Emotion and Affect, pages 15-019, Genoa. Italy, 2006.

E. Douglas-Cowie, N. Campbell, R. Cowie, and P. Roach. Emotional speech: towards a new generation of databases. Speech Communication, 40:33–60, 2003.

M. Rosenblum, Y. Yacoob, and L. Davis. Human expression recognition from motion using a radial basis function network architecture. IEEE Transactions on Neural Networks, 7(5):1121–1138, 1996.

M. Pantic and L.J.M. Rothkrantz. Automatic analysis of facial expressions: The state of the art. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12):1424–1445, 2000.

M. Pantic and MS Bartlett. Machine analysis of facial expressions. In Face Recognition, K. Delac and M. Grgic, Eds.,Vienna, Austria: I-Tech Education and Publishing, pp. 377- 416, 2007.

G. Castellano, M. Mortillaro, A. Camurri, G. Volpe, and K. Scherer. Automated analysis of body movement in emotionally expressive piano performances. Music Perception, 26(2):103–119, University of California Press, 2008.


Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.