This paper introduces a face recognition system that contributes the feature and decision fusion in challenging environment. In this work, we investigate the proposed facial recognition system in typical office environments conditions. Though the traditional HMM based facial recognition system is very sensitive to the facial parameters variation, the proposed feature and decision fusion based face recognition is found to be stance and performs well for improving the robustness and naturalness of human-computer-interaction. At first appearance and shape based features are extracted using Active Appearance Model and Active Shape Model. The other task combines appearance and shape based features that have been used by the multiple Discrete Hidden Markov Model classifiers with likelihood ratio based score fusion and majority voting method. The performances of all these uni-modal and multi-modal system performance have been evaluated and compared with each other according to the VALID database.

1. Bin Zhang and Sargur N. Srihari, “CLASS-WISE MULTI-CLASSIFIER COMBINATION BASED ON DEMPSTER-SHAFER THEORY”, Seventh International Conference on Control, Automation, Robotics and Vision (ICARV 2002), Singapur, Dec. 2002.
2. T. K. Ho. A theory of multiple classifier systems and itsapplication to visual word recognition. PhD Dissertation, State University of New York at Buffalo, 1992.
3. A. Samal and P.A. Iyengar, “Automatic recognition and analysis of human faces and facial expressions: a survey,” Patt. Recog,. 25 (1), pp. 65–77, 1992.
4. D. Valentin, H. Abdi, A.J. O_Toole, “G.W. Cottrell, Connectionist models of face processing: a survey,” Patt. Recog,. 27 (9), pp. 1209–1230, 1994.
5. R. Chellappa, C.L. Wilson and S. Sirohey, “Human and machine recognition of faces: a survey,” Proc. IEEE, 83 (5), pp. 705–740, 1995.
6. J. Zhang, Y. Yan and M. Lades, “Face recognition: eigenface, elastic matching, and neural nets,” Proc. IEEE, 85 (9), pp. 1423–1435, 1997.
7. I. Craw, N. Costen, T. Kato and S. Akamatsu, “How should we represent faces for automatic recognition?,” IEEE Trans. Patt. Anal. Mach. Intell., 21 (8), pp. 725–736, 1999.
8. A.M. Burton, V. Bruce, P.J.B. Hancock, “From pixels to people: a model of familiar face recognition,” Cognitive Sci. 23, pp. 1–31, 1999.
9. S. Kong, J. Heo, B. Abidi, J. Paik and M. Abidi, “Recent Advances in Visual and Infrared Face Recognition - A Review,” The Journal of Computer Vision and Image Understanding, Vol. 97, No. 1, pp. 103-135, 2005.
10. K. Parimala Geetha, S. Sundaravadivelu and N. Albert Singh, “Rotation Invariant Face Recognition using Optical Neural Networks,” TENCON 2008 - 2008 IEEE Region 10 Conference, Hyderabad, India, 2009.
11. Kiyomi Nakamura, and Hironobu Takano, “Rotation and Size Independent Face Recognition by the Spreading Associative Neural Network,” International Joint Conference on Neural Networks, Sheraton Vancouver Wall Centre Hotel, Vancouver, BC, Canada, 2006.
12. S. H. Lin, S. Y. Kung, and L. J. Lin., “Face recognition/detection by probabilistic decision-based neural network,” IEEE Transactions on Neural Networks, Special Issue on Artificial Neural Networks and Pattern Recognition, 8(1), 1997.
13. Henry A. Rowley, Shumeet Baluja, and Takeo Kanade, “Neural network based face detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1), 1998.
14. Jianke Zhu, Mang I Vai and Peng Un Mak, ”Gabor Wavelets Transform and Extended Nearest Feature Space Classifier for Face Recognition,” Proceedings of the Third IEEE International Conference on Image and Graphics (ICIG’04), 2004.
15. M. Kirby and L. Sirovich, “Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 103-108, 1990.
16. J. Karhunen, E. Oja, L. Wang, R. Vigario, and J. Joutsensalo, “ A class of neural networks for independent component analysis,” IEEE Trans. Neural Networks, Vol. 8, pp. 486–504, 1997.
17. E. Osuna, R. Freund, and F. Girosi, “Training support vector machines: An application to face detection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 130–136, 1997.
18. M. Turk and A. Pentland, “Eigenfaces for recognition,” J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71–86, 1991.
19. Y.-S. Ryu and S.-Y. Oh, “Simple hybrid classifier for face recognition with adaptively generated virtual data,” Pattern Recognit. Leters., vol. 23, pp. 833–841, 2002.
20. J. Bala, P. Pachowicz, and K. De Jong, “Multistrategy Learning from Engineering Data by Integrating Inductive Generalization and Genetic Algorithms, in Machine Learning: A Multistrategy Approach,” Michalski and G. Tecuci (Eds.), Vol. IV, R.S. Morgan Kaufmann, San Mateo, CA., pp. 121-138, 1994.
21. F. Gruau and D. Whitley, “Adding Learning to the Cellular Development of Neural Networks: Evolution and the Baldwin Effect,” Evolutionary Computation, Vol.1, No.3, pp. 213-234, 1993.
22. H. Vafaie and K. De Jong, “Improving a Rule Induction System Using Genetic Algorithms, in Machine Learning: A Multistrategy Approach,” Michalski and G. Tecuci (Eds.), Vol. IV, R.S., Morgan Kaufmann, San Mateo, CA., pp. 453-469, 1994.
23. Stephen Milborrow and Fred Nicolls, “Locating Facial Features with an Extended Active Shape Model,” available at http://www.milbo.org/stasm-files/locating-facial-features-with-an-extended-asm.pdf.
24. R. Herpers, G. Verghese, K. Derpains and R. McCready, “Detection and tracking of face in real environments,” IEEE Int. Workshop on Recognition, Analysis and Tracking of Face and Gesture in Real- Time Systems, Corfu, Greece, pp. 96-104, 1999.
25. J. Daugman, “Face detection: a survey,” Comput. Vis. Imag. Underst, 83, 3, pp. 236- 274, 2001.
26. Rafael C. Gonzalez and Richard E. Woods, Digital Image Processing. Addison-Wesley, 2002.
27. Stephen Milborrow and Fred Nicolls, “Locating Facial Features with an Extended Active Shape Model,” available at http://www.milbo.org/stasm-files/locating-facial-features-with-an-extended-asm.pdf.
28. R. Herpers, G. Verghese, K. Derpains and R. McCready, “Detection and tracking of face in real environments,” IEEE Int. Workshop on Recognition, Analysis and Tracking of Face and Gesture in Real- Time Systems, Corfu, Greece, pp. 96-104, 1999.
29. J. Daugman, “Face detection: a survey,” Comput. Vis. Imag. Underst, 83, 3, pp. 236- 274, 2001.
30. Rafael C. Gonzalez and Richard E. Woods, Digital Image Processing. Addison-Wesley, 2002.
31. Jong-Seok Lee and Cheol Hoon Park, Speech Recognition, Technologies and Applications, pp. 275-296, I-Tech, Vienna, Austria, 2008.
32. P. A. Devijver, "Baum's forward-backward algorithm revisited", Pattern Recognition Letter, 3, pp. 369-373, 1985.
33. A. Rogozan, P.S. Sathidevi, “Static and dynamic features for improved HMM based visual speech recognition,” 1st International Conference on Intelligent Human Computer Interaction, Allahabad, India, pp. 184-194, 2009.
34. J. S. Lee, C. H. Park, “Adaptive Decision Fusion for Audio-visual speech Recognition”, Speech Recognition, Technologies and Applications, ed. F. Mihelic, J. Zibert, (Vienna, Australia, 2008), pp. 550, 2008.
35. A. Adjoudant, C. Benoit, “On the integratio of auditory and visual parameters in an HMM-based ASR,” Speechreading by Humans and Machines: Models, Systems, and Speech Recognition, Technologies and Applications, ed. D.G. Strok and M. E. Hennecke, (Springer, Berlin, Germany, 1996), pp. 461-472.
36. Nayer Wanas, “Feature Based Architecture for Decision Fusion,” Ph.D. Dissertation, Dept. of Systems Design Engineering, University of Waterloo, Ontario, 2003.
37. R. Battiti and A. Colla. Democracy in neural nets: Voting schemes for classification. Neural Networks, vol. 7, no. 4, pp. 691–707, 1994.
38. C. Ji and S. Ma., “Combinations of weak classifiers,” IEEE Transactions on Neural Networks, Vol. 8, No. 1, pp. 32–42, 1997.
39. L. Lam and C. Suen. Optimal combination of pattern classifiers. Pattern Recognition Letters, vol. 16, pp. 945–954, 1995.
40. L. Xu, A. Krzy´zak, and C. Suen. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man, and Cybernetics, vol. 22, no. 3, pp. 418–435, 1992.
41. Dymitr Ruta and Bogdan Gabrys, “An Overview of Classifier Fusion Methods,” Computing and Information Systems, 7 (2000), University of Paisley, p.1-10, 2002.
42. N. A. Fox, B. A. O'Mullane and R. B. Reilly, “The Realistic Multi-modal VALID database and Visual Speaker Identification Comparison Experiments,” Proc. of the 5th International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA-2005), New York, 2005.

Face Recognition Feature and Decision Fusion Facial Feature Extraction Human Computer Interaction Discrete Hidden Markov Model