CFP last date
20 January 2025
Call for Paper
February Edition
IJCA solicits high quality original research papers for the upcoming February edition of the journal. The last date of research paper submission is 20 January 2025

Submit your paper
Know more
The week's pick
Responsive image
SFLA-Based Line Balancing and Task Optimization in Master-Slave Controlled Hanger Transportation Systems for Garment Production

Duc Hoang Nguyen
Random Articles
Reseach Article

Facial Action Unit Recognition from Video Streams with Recurrent Neural Networks

by Hima Vadapalli
journal cover thumbnail

International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 96 - Number 19
Year of Publication: 2014
Authors: Hima Vadapalli
10.5120/16904-6971

Hima Vadapalli . Facial Action Unit Recognition from Video Streams with Recurrent Neural Networks. International Journal of Computer Applications. 96, 19 ( June 2014), 31-39. DOI=10.5120/16904-6971

@article{ 10.5120/16904-6971,
author = { Hima Vadapalli },
title = { Facial Action Unit Recognition from Video Streams with Recurrent Neural Networks },
journal = { International Journal of Computer Applications },
issue_date = { June 2014 },
volume = { 96 },
number = { 19 },
month = { June },
year = { 2014 },
issn = { 0975-8887 },
pages = { 31-39 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume96/number19/16904-6971/ },
doi = { 10.5120/16904-6971 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2024-02-06T22:22:12.636013+05:30
%A Hima Vadapalli
%T Facial Action Unit Recognition from Video Streams with Recurrent Neural Networks
%J International Journal of Computer Applications
%@ 0975-8887
%V 96
%N 19
%P 31-39
%D 2014
%I Foundation of Computer Science (FCS), NY, USA
Abstract

Facial expressions are one of the parameters for accessing individual behavioral processes. Their recognition and verification can be framed as the identification of states of dynamical systems generated by physiological processes. Whereas a snap shot of a dynamical system gives information about its current state, a time series of past states captures its trajectory in state space. The description and recognition of facial expressions using atomic muscle movements, so-called action units provide an extensive framework. The temporal modeling and recognition of these muscle movements promises a broader and more generic approach for recognizing subtle changes on the facial region. This paper proposes the use of recurrent neural networks for modeling facial action unit activity. Recurrent neural networks are able to model actions based on their previous and current states, unlike other dynamic classifiers such as hidden Markov models. A detailed comparative analysis with the recognition performance of a static classifier such as support vector machines suggests that recurrent neural networks gain more knowledge about the action unit activation when presented with a sequence of images. On average our model achieved a positive hit rate of 85. 8% for upper face action units and 84. 9% for lower face action units.

References
  1. Ahmad, A. M. , Ismail, S. , and Samaon, D. F. 2004 Recurrent Neural Network with Backpropagation through Time for Speech Recognition. International Symposium on Communications and Information Technologies 2004.
  2. Ali, G. , Feyzullah, T. , Kader, E. , and Serdar, C. Signature Verification Performance of Elman's Recurrent Neural Network. Technology, vol. 7, pp. 541-547, 2004.
  3. Bartlett, M. S. , Littlewort, G. , Braathen, B. , Sejnowski, T. J. , and Movellan, J. R. A Prototype for Automatic Recognition of Spontaneous Facial Actions. Advances in Neural Information Processing Systems, vol. 15, MIT Press-S. Becker and K Obermayer(eds. ), 2003.
  4. Bartlett, M. S. , Littlewort, G. , Frank, M. , Lainscsek, C. , Fasel, I. , and Movellan, J. Machine Learning Methods for Fully Automatic Recognition of Facial Expression and Facial Actions. Proceedings of the IEEE Conference on Systems, Man and Cybernetics, Netherlands, 2004.
  5. Bartlett, M. S. , Littlewort, G. C. , Frank, M. G. , Lainscsek, C. , Fasel, I. R. , and Movellan, J. R. Automatic Recognition of Facial Actions in Spontaneous Expressions. Journal of Multimedia, vol. 19, no. 6, 2006.
  6. Baum, E. B. , and Haussler, D. What Size Nets Get Valid Generalization. Neural Computation, vol. 1, pp. 151-160, 1989.
  7. Black, M. J. , and Yacoob, Y. Tracking and Recognizing Rigid and Non-Rigid Facial Motions using Local Parametric Models of Image Motion. Proceedings of Fifth International Conference on Computer Vision, pp. 374-381, 1995.
  8. Boser, B. E. , Guyon, I. M. , and Vapnik, V. N. A Training Algorithm for Optimal Marginal Classifiers. D. Haussler Editor, 5th Annual ACM Workshop on COLT, ACM Press, pp. 144-152, 1992.
  9. Donato, G. , Bartlett, M. S. , Hager, J. C. , Ekman, P. , and Sejnoeski, T. J. Classifying Facial Actions. IEEE Transactions on Patterns Analysis and Machine Intelligence, vol. 21, issue 10, pp. 974-989, Oct. 1999.
  10. Ekman, P. , and Friesen, W. The Facial Action Coding System: A Technique For the Measurement of Facial Movement. Consulting Psychologists Press, San Francisco, CA, 1978.
  11. Ekman, P. , Friesen, W. , and Hager, J. C. Facial Action Coding System (FACS). A Human Face, Salt Lake City, 2002.
  12. Elman, J. Finding Structure in Time. Cognitive Science, vol. 14, pp. 179-211, 1990.
  13. Essa, I. A. , and Pentland, A. P. Coding, Analysis, Interpretation and Recognition of Facial Expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 757-763, July 1997.
  14. Fadi, D. , and Franck, D. Facial Expression Recognition in continuous Videos using Linear Discriminant Analysis. Proceedings of IAPR Conference on Machine Vision Applications, pp. 277-280, May 2005.
  15. Fasel, I. , Dahl, R. , Hershey, J. , Fortenberry, B. , Susskind, J. , and Movellan, J. R. Machine Perception Toolbox. Machine Perception Laboratory, University of California San Diego.
  16. Graves, A. , Mayer, C. , Wimmer, M. , Schmidhuber, J. , and Radig, B. Facial Expression Recognition With Recurrent Neural Networks. Proceedings of the International Workshop on Cognition for Technical Systems, Germany, 2008.
  17. Hai Tao, Chen, H. , and Huang, T. Analysis and Compression of Facial Animation Parameters Set (FAPs). IEEE First Workshop on Multimedia Signal Processing, Princeton, USA, pp. 245-250, June 1997.
  18. Hoque, M. E. , McDuff, D. J. , and Picard, R. W. Exploring Temporal Patterns in Classifying Frustrated and Delighted Smiles. IEEE Transactions on Affective Computing, vol. 3, issue 3, 2012.
  19. Ira, C. , Nicu, S. , Ashutosh, G. , Lawrence, S. C. and Thomas, S. H. Facial Expression Recognition from Video Sequences: Temporal and Static Modeling. Computer Vision an Image Understanding, pp. 160-187, 2003.
  20. Jacob, W. , and Christian, W. O. Haar Features for FACS AU Recognition. Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition, 2006.
  21. Jordan, M. I. Attractor Dynamics and Parallelism in a Connectionist Sequential Machine. Proc. of the Ninth Annual Conference of the Cognitive Science Society, Lawrance Erlbaum, pp. 531-546, 1986.
  22. Kanade, T. , Cohn, J. , and Tian, Y. Comprehensive Database for Facial Expression Analysis. Proceedings of 4th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46-53, 2000.
  23. Kobayashi, H. , and Hara, F. Dynamic Recognition of Basic Facial Expressions by Discrete-time Recurrent Neural Network. Proceedings of the International Joint Conference on Neural Networks, pp. 155-158, 1993.
  24. Kondratenko, V. V. and Kuperin, Yu. A. Using Recurrent Neural Networks To Forecasting of Forex. Disordered Systems and Neural Networks, April 2003.
  25. Kotsia, I. and Pitas, I. Facial Expression Recognition in Image Sequences using Geometric Deformation Features and Support Vector Machines. IEEE Transactions on Image Processing, vol. 16, no. 1, pp. 172- 187, 2007.
  26. Krogh, A. , and Hertz, J. A. A Simple Weight Decay Can Improve Generalization. Advances in Neural Information Processing Systems 4, J. E Moody, S J Hanson and R P Lippmann eds. Morgan Kauffmann Publishers, San Mateo CA, pp. 950-957, 1995.
  27. Lien, J. Automatic Recognition of Facial Expressions using Hidden Markov Models and Estimation of Expression Intensity. PhD dissertation, Carnegie Mellon University, Pittsburg, PA, 1998.
  28. Lien, J. J. , Kanade, T. , Cohn, J. , and Li, C. Automated Facial Expressions Based on FACS Action Units. Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 390-395, April 1998.
  29. Ligang, Z. , and Dian, T. Facial Expression Recognition using Facial Movement Features. IEEE Transactions on Affective Computing, 2011.
  30. Littlewort, G. , Bartlett, M. S. , Fasel, I. , Susskind, J. , and Movellan, J. Dynamics of Facial Expression Extracted Automatically from Video. IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Face Processing in Video, vol. 5, pp. 80, June 2004.
  31. Lyons, M. J. , Budynek, J. , Plante, A. , and Akamatsu, S. Classifying Facial Attributes using a 2D Gabor Wavelet Representation and Discriminant Analysis. Proceedings of the 4th International Conference on Automatic Face and Gesture Recognition, Grenoble France, IEEE Computer Society, pp. 202-207, 2000.
  32. Oliver, N. , Pentland, A. , and Berard, F. LAFTER: A Real-time Lips and Face Tracker with Facial Expression Recognition. Pattern Recognition, vol. 33, no. 8, pp. 1369-1382, 2000.
  33. Otsuka, T. , and Ohya, J. Spotting Segments Displaying Facial Expression from Image Sequences using HMM. In IEEE Proceedings of the Second International Conference on Automatic Face and Gesture Recognition (FG98), Nara, Japan, 1998, pp. 442-447, 1998.
  34. Petar S. A. , and Aggelos K. K. Automatic Facial Expression Recognition Using Facial Animation Parameters and Multistream HMMs. In IEEE Transactions on Information Forensics and Security, vol 1, No. 1, pp. 3-11, March, 2006.
  35. Philipp M. , and Rana E. K. Real Time Facial Expression Recognition in Video Using Support Vector Machines. In Proceedings of the 5th International Conference on Multimodal Interfaces, ICMI03, pp. 258-264, 2003.
  36. Rosenblum M. , Yacoob Y. , and Davis L. Human Expression Recognition from Motion using a Radial Basis Function Network Architecture. In IEEE Trans. Neural Networks, Vol 7, No. 5, pp. 1121-1138, 1996.
  37. Smith E. , Bartlett M. S. , and Movellan J. R. Computer Recognition of Facial Actions: A Study of Co-articulation Effects. In Proceedings of the 8th Annual Joint Symposium on Neural Computation, 2001.
  38. Tian Y. , Kanade T. , and Cohn J. F. Recognizing Action Units for Facial Expression Analysis. In IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 23, No. 2, pp. 97-115, 2001.
  39. Tian Y. , Kanade T. , and Cohn J. F. Evaluation of Gabor Wavelet Based Facial Action Unit Recognition in Image Sequences of Increasing Complexity. In Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 229-234, May, 2002.
  40. Theekapun C. , Tokia S. , and Hase H. Facial Expression Recognition from a Partial Face Image by Using Displacement Vector. In the Proceedings of 5th International Conference on Electrical/Electronics, Computer, telecommunications and Information Technology, ECTI-CON, pp. 441-444, 2008
  41. Werbos P. Backpropagation: Past and Future. In Proceedings of the IEEE International Conference on Neural Networks, IEEE Press, pp. 343-353, 1988.
  42. Werbos J. Paul. Back Propagation Through Time: What it Does and How to do it. In Proceedings of the IEEE, Vol 78, No. 10, pp. 1550-1560, October, 1990.
  43. Williams R. J. , and Zisper D. A Learning Algorithm for Continually Running Fully Recurrent Neural Networks. In Neural Computation, Vol 1, No. 2, pp. 270-280, 1989.
  44. Tong, Y. , Chen, J. , and Ji, Q. A Unified Probabilistic Framework for Spontaneous Facial Activity Modeling and Understanding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 2, pp. 258-273,2010.
  45. Tong, Y. , Liao, W. , and Ji, Q. Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships. IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1683-1699, 2007.
  46. Zhang Z. Feature Based Facial Expression Recognition: Sensitivity Analysis and Experiments With a Multi Layer Perception. Technical Re- port 3354, INRIA Sophia Antopolis, 1998.
  47. Zhang Z. , Lyons M. , Schuster M. , and Akamatsu S. Comparison Between Geometry Based and Gabor Wavelets Based Facial Expression Recognition Using Multi Layer Perception. In Proceedings of Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara Japan, IEEE Computer Society, pp. 454-459, 1998.
Index Terms

Computer Science
Information Sciences

Keywords

Computer Vision Face and Gesture Recognition Feature Extraction Neural Nets.

Powered by PhDFocusTM