Open Access Open Access  Restricted Access Subscription or Fee Access

Optical Flow Analysis of Emotional Human Facial Expressions – Forensic Application

David K. F. Chow, Sara Lal, Tamara Sztynda

Abstract


Emotional facial expressions have been suggested as indicators of a person’s intended actions when they display a set of specific facial reactions. Our aim was to investigate Optical Flow to identify specific emotions with the long-term objective to create an algorithm that would automate the process. This would remove all allegations of bias to give security personnel an objective way of screening individuals in a crowd who may require closer scrutiny. This paper describes our preliminary studies, using film clips to elicit dynamic natural facial responses. This has identified key elements that need to be evaluated further, such as gender and age of the people under observation, as well as specific facial elements that may be of greater benefit than whole facial images for the purpose of screening crowds for potential perpertrators of anti-social behaviour.


Keywords


Emotion Induction, Lucas-Kanade Optical Flow, Spontaneous Emotional Facial Expression.

Full Text:

PDF

References


F. Sigmund, and S. James, “The Ego and the Id,” New York, W.W. Norton & Company, 1990.

T. Troscianko, A. Holmes, J. Stillman, M. Mirmehdi, D. Wright, and A. Wilson, “What happens next? The predictability of natural behaviour. Viewed through CCTV Cameras,” Perception and Psychophysics, vol. 33, pp. 87-101, 2004. https://doi.org/10.1068/p3402

A.Barnes, and P. H. Ephross, “Emotional and behavioural responses to attacks. The impact of hate violence on victims,” 1994, Available: http://www.socialworkers.org/pressroom/events/911/barnes.asp

S. Bruner, and R. Tagiuri, The perception of people, Massachusetts: Addison-Wesley, 1954.

S. Tomkins, and S. Silvan, Affect, imagery, consciousness, New York: Springer, 1962.

D. Craig, S. Hyde, and C. Patrick, “Genuine, suppressed and faked facial behaviour during exacerbation of chronic low back pain,” Pain, vol. 2, no. 2, pp. 161-171, 199, Aug. 1991. https://doi.org/10.1016/0304-3959(91)90071-5

E. D. Ross, “Facial expressions, pain and nociception – are they related?” Nature Reviews Neurology, vol. 4, pp. 304-305, 2008. https://doi.org/10.1038/ncpneuro0791

R. M. Todd, and A. Anderson, “Six degrees of separation: the amygdala regulates social behaviour and perception,” Nature Neuroscience, vol. 12, pp. 1217-1218, 2009. https://doi.org/10.1038/nn1009-1217

W. Sato, T. Kochiyama, S. Yoshikawa, E. Naito, and M. Matusumara, “Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study,” Cognitive Brain Research, vol. 20, pp. 81-91, 2004. https://doi.org/10.1016/j.cogbrainres.2004.01.008

H. Nakayama, “Development of infant crying behaviour: a longitudinal case study,” Infant Behavior and Development, vol. 33, pp. 463-471, 2010. https://doi.org/10.1016/j.infbeh.2010.05.002

P. Ekman, “Facial expression and emotion,” American Psychologist, vol. 48, pp. 384-392, 1993.

A. Russel, “Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies,” Psychological Bulletin, vol. 115, pp. 102-141, 1994.

P. Ekman, and W. V. Friesen, “The repertoire of nonverbal behaviour: categories, origins, usage, and encoding,” Semiotica, vol. 1, pp. 49-98, 1969.

P. Ekman, and W. V. Friesen, “Measuring facial movement,” Environmental Psychology and Nonverbal Behaviour, vol. 1, pp. 56-75, 1976.

P. Ekman, and W. V. Friesen, Facial action coding system: a technique for the measurement of facial movement, Palo Alto, CA: Consulting Psychologists Press, 1978.

D. Matsumoto, P. Ekman, and A. Fridlund, “Analyzing nonverbal behaviour,” in Practical guide to using video in the behavioural sciences, P. Ekman, A. Fridlund, and D. Matsumoto, Eds. New York: John Wiley & Sons, 1991, pp. 153-164.

J. Cohn, A. J. Zlochower, J. Lien, and T. Kanade, “Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding,” Psychophysiology, vol. 36, pp. 35-43, 1999. https://doi.org/10.1017/S0048577299971184

Pantic, and L. Rothkrantz, “Automatic analysis of facial expressions: the state of the art,” Institute of Electrical and Electronics Engineers Transactions on Pattern Analysis and Machine Intelligence, vol. 22, pp. 1424-1445, 2000.

P. Ekman, and E. L. Rosenberg, What the face reveals – basic and applied studies of spontaneous expression using the facial action coding system (FACS), New York: Oxford University Press, 2005.

H. L. Meiselman, “A review of current state of emotion research in product development.” Food Research International, vol. 76, pp. 192-199, 2015. https://doi.org/10.1016/j.foodres.2015.04.015

C. J. Duthoit, T. Sztynda, S.K.L. Lal, B. T. Jap, and J. I. Agbinya, “Optical flow image analysis of facial expressions of human emotion: forensic applications,” in 1st Int. Conf. on Forensic Applications and Techniques in Telecommunications, Information, and Multimedia and Workshop Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, e-Forensics, 2008, pp. 5:1–5:6.

E. Trucco, and A. Verri, Introductory techniques for 3-d computer vision, New Jersey: Prentice Hall PTR, 1998.

S. Bartlett, P. A. Viola, T. J. Sejnowski, B. A. Golomb, J. Larsen, M. Tourretzki, and M. Hasselmo, “Classifying facial action,” Advances in Neural Information Processing Systems, vol. 8, pp. 823-829, 1996.

L. Xu, J. Chen, and J. Jia, “A segmentation based variational model for accurate optical flow estimation,” in Computer Vision – European Conference on Computer Vision, Berlin, Heidelberg, 2008, pp. 671-684. https://doi.org/10.1007/978-3-540-88682-2_51

B. D. Lucas, and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proc. 7th Intl Joint Conf. on Artificial Intelligence. Aug. 1981, pp. 121-130.

B. D. Lucas, “Generalized image matching by the method of differences,” Ph.D. dissertation, Univ. of Carnegie-Mellon. Univ. of Microfilms International,1984.

J. Gross, and R. Levenson,”Cognition and emotion,” Emotion Elicitation Using Films, vol. 9, pp. 87-108, 1995. https://doi.org/10.1080/02699939508408966

P. Ekman, and W. V. Friesen, “Nonverbal leakage and clues to deception,” Psychiatry, vol. 32, pp. 88-105, 1969.

B. Koen, “The problem of negative emotions,” 2001. Available: http://csml.som.ohio-state.edu/Music829D/Notes/Negative.emotions.html

W. Sato, T. Fujimura, and N. Suzuki, “Enhanced facial EMG activity in response to dynamic facial expressions,” International Journal of Psychophysiology, vol. 70, pp. 70-74, 2008. https://doi.org/10.1016/j.ijpsycho.2008.06.001

P. Philippot, “Inducing and assessing differentiated emotion-feeling states in the laboratory,” Cognition and Emotion, vol. 7, pp. 171-193, 1993. https://doi.org/10.1080/02699939308409183

J. Levinson, Music and meaning, Ithaca and London: Cornell University Press, 1997.

V. T. Visch, and E. S Tan, “Categorising moving objects into film genres: the effect of animacy attribution, emotional response, and the deviation from non-fiction.” Cognition, vol. 110, pp. 265-272, 2009. https://doi.org/10.1016/j.cognition.2008.10.018

D. Wied, K. Hoffman, and D. R. Roskos-Ewoldsen, “Forewarning of graphic portrayal of violence and the experience of suspenseful drama,” Cognition and Emotion, vol. 11, pp. 481-494, 1997. https://doi.org/10.1080/026999397379890

M. Jabbi, M. Swart, and C. Keysers, “Empathy for positive and negative emotions in the gustatory cortex,” Neuroimage, vol. 34, pp. 1744-1753, 2006.

L. Norton, “Running head: elevation, empathy, and forgiveness,” Distinguished majors thesis, Univ. of Virginia, 2009.

J. J. Gross, “Emotion regulation: affective, cognitive, and social consequences,” Psychophysiology, vol. 39, pp. 81-291, 2002. https://doi.org/10.1017/S0048577201393198

J. J. Gross, “Antecedent and response-focused emotion regulation: divergent consequences for experience, expression, and physiology,” Journal of Personality and Social Psychology, vol. 74, pp. 224-237, 1998.

S. A. Moore, L. A. Zoellner, and N. Mollenholt, “Are expressive suppression and cognitive reappraisal associated with stress-related symptoms?” Behaviour Research and Therapy, vol. 46, pp. 993-1000, 2008. https://doi.org/10.1016/j.brat.2008.05.001

B. Mauss, C. L. Cook, J.Y.L. Cheng, and J. J. Gross, “Individual differences in cognitive reappraisal: experiential and physiological responses to an anger provocation,” International Journal of Psychophysiology, vol. 66, pp. 116-124, 2007. https://doi.org/10.1016/j.ijpsycho.2007.03.017

T. Hagemann, R. W. Levenson, and J. J. Gross, “Expressive suppression during an acoustic startle,” Psychophysiology, vol. 43, pp. 104-112, 2006. https://doi.org/10.1111/j.1469-8986.2006.00382.x

E. N. Marieb, and K. Hoehn, Human anatomy and physiology, San Francisco: Pearson Education, 2007.

Ekman, G. Roper, and G. C. Hager, “Deliberate facial movement,” Child Development, vol. 51, pp. 886-891, 1980.

C. G. Kohler, T. Turner, N. M. Stolar, W. B. Bilker, C. M. Brensinger, R. E. Gur, and R. C. Gur, “Differences in facial expressions of four universal emotions,” Psychiatry Research, vol. 128, pp. 235-244, 2004. https://doi.org/10.1016/j.psychres.2004.07.003

D. Matsumoto, and P. Ekman,”Facial expression analysis,” Scholarpedia, vol. 3, pp. 4237-4248, 2008. http://doi.org/10.4249/scholarpedia.4237

A. A. Root, and J. A. Stephens, “Organisation of the central control of muscles of facial expression in man,” Journal of Physiology, vol. 549, pp. 289-298, 2003. https://doi.org/10.1113/jphysiol.2002.035691

B. M. Waller, J. J. Craig, and A. M. Burrows, “Selection for universal facial emotion,” Emotion, vol. 8, pp. 435-439, 2008.


Refbacks

  • There are currently no refbacks.