CFP last date
20 December 2024
Call for Paper
January Edition
IJCA solicits high quality original research papers for the upcoming January edition of the journal. The last date of research paper submission is 20 December 2024

Submit your paper
Know more
The week's pick
Responsive image
Improved Shuffled Frog Leaping Algorithm with Self-Adaptive Shuffling for Fuzzy Logic PD+G Controller Optimization in Robotic Manipulators

Duc Hoang Nguyen
Random Articles
Reseach Article

Towards Computer Assisted International Sign Language Recognition System: A Systematic Survey

by Mikhaylyna Melnyk, Vira Shadrova, Borys Karwatsky
journal cover thumbnail

International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 89 - Number 17
Year of Publication: 2014
Authors: Mikhaylyna Melnyk, Vira Shadrova, Borys Karwatsky
10.5120/15727-4698

Mikhaylyna Melnyk, Vira Shadrova, Borys Karwatsky . Towards Computer Assisted International Sign Language Recognition System: A Systematic Survey. International Journal of Computer Applications. 89, 17 ( March 2014), 44-51. DOI=10.5120/15727-4698

@article{ 10.5120/15727-4698,
author = { Mikhaylyna Melnyk, Vira Shadrova, Borys Karwatsky },
title = { Towards Computer Assisted International Sign Language Recognition System: A Systematic Survey },
journal = { International Journal of Computer Applications },
issue_date = { March 2014 },
volume = { 89 },
number = { 17 },
month = { March },
year = { 2014 },
issn = { 0975-8887 },
pages = { 44-51 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume89/number17/15727-4698/ },
doi = { 10.5120/15727-4698 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2024-02-06T22:09:54.198233+05:30
%A Mikhaylyna Melnyk
%A Vira Shadrova
%A Borys Karwatsky
%T Towards Computer Assisted International Sign Language Recognition System: A Systematic Survey
%J International Journal of Computer Applications
%@ 0975-8887
%V 89
%N 17
%P 44-51
%D 2014
%I Foundation of Computer Science (FCS), NY, USA
Abstract

There is a number of automated sign language recognition systems proposed in the computer vision literature. The biggest drawback of all these systems is that every nation has their own culture oriented sign language. In other words, everyone needs to develop a specific sign language recognition system for their nation. Although the main building blocks of all signs are gestures and facial expressions in all sign languages, the nation specific requirements make it difficult to design a multinational recognition framework. In this paper, we focus on the advancements in computer assisted sign language recognition systems. More specifically, we discuss if the ongoing research may trigger the start of an international sign language design. We categorize and present a summary of the current sign language recognition systems. In addition, we present a list of publicly available databases that can be used for designing sign language recognition systems.

References
  1. Grimes, G. J. Digital data entry glove interface device. U. S. Patent and Trademark Office, 4,414,537, 1983.
  2. Hall, J. A. The human interface in three dimensional computer art space. Media Lab, Massachusetts Institute of Technology, MSVS Thesis, Cambridge, MA, 1985.
  3. Kadous, M. W. Machine recognition of Auslan signs using PowerGloves: Towards large-lexicon recognition of sign language. Workshop on the Integration of Gesture in Language and Speech, pp. 165-174, 1996.
  4. Vogler, C. and Metaxas, D. , Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods. In Proc. of Systems, Man, and Cybernetics, vol. 1, no. , vol. 1, pp. 156-161, 1997.
  5. Starner, T. and Pentland, A. Real-time american sign language recognition from video using hidden markov models. In Motion-Based Recognition, Springer Netherlands, pp. 227-243, 1997.
  6. Brashear, H. , Junker, H. , Starner, T. , and Lukowicz, P. Using Multiple Sensors for Mobile Sign Language Recognition. 16th International Symposium on Wearable Computers, pp. 45, 2003.
  7. Liang, R. H. and Ouhyoung, M. A real-time continuous gesture recognition system for sign language. In Proc. of Automatic Face and Gesture Recognition, pp. 558-567, 14-16 Apr 1998.
  8. Gao, W. , Ma, J. , Shan, S. , Chen, X. , Zeng, W. , Zhang, H. , and Wang, J. HandTalker: A multimodal dialog system using sign language and 3-D virtual human. In Proc. of Advances in Multimodal Interfaces, pp. 564-571, 2000.
  9. Hernandez-Rebollar, J. L. , Lindeman, R. W. , Kyriakopoulos, N. A multi-class pattern recognition system for practical finger spelling translation. In Proc. of Multimodal Interfaces Conf. , pp 185-190, 2002.
  10. Mehdi, S. A. , and Khan, Y. N. Sign language recognition using sensor gloves. In Proc. of Neural Information Processing, vol. 5, pp. 2204-2206, 2002.
  11. Akyol, S. , and Canzler, U. An information terminal using vision based sign language recognition. In Proc. of ITEA Workshop on Virtual Home Environments, VHE Middleware Consortium, vol. 12, pp. 61-68, 2002.
  12. Kuroda, T. , Tabata, Y. , Goto, A. , Ikuta, H. , and Murakami, M. Consumer price data-glove for sign language recognition. In Proc. of 5th Int. Con. Disability, Virtual Reality Assoc. , Oxford, UK, pp. 253-258, 2004.
  13. Hernandez-Rebollar, J. L. , Kyriakopoulos, N. , Lindeman, R. W. A new instrumented approach for translating ASL into sound and text. In Proc. of Automatic Face and Gesture Recognition, pp. 547-552 2004.
  14. Oz, C. , and Leu, M. C. American Sign Language word recognition with a sensory glove using artificial neural networks. Engineering Applications of Artificial Intelligence, vol. 24(7), pp. 1204-1213, 2011.
  15. Zafrulla, Z. , Brashear, H. , Starner, T. , Hamilton, H. , and Presti, P. American sign language recognition with the kinect. In Proc. of the 13th international conference on multimodal interfaces pp. 279-286, 2011.
  16. Assaleh, K. , Shanableh, T. , and Zourob, M. Low complexity classification system for glove-based Arabic sign language recognition. In Neural Information Processing, pp. 262-268, 2012.
  17. Jeong, E. , Lee, J. , and Kim, D. Finger-gesture Recognition Glove using Velostat. 11th Int. Conf. on Control, Automation and Systems, pp. 206-210, 2011.
  18. Luzanin, O. and Plancak, M. Hand gesture recognition using low-budget data glove and cluster-trained probabilistic neural network. Assembly Automation, vol. 34(1), pp. 94-105, 2014.
  19. Waldron, M. B. , and Kim, S. Isolated ASL sign recognition system for deaf persons. Transactions on Rehabilitation Engineering, vol. 3(3), pp. 261-271, 1995.
  20. Fagiani, M. , Principi, E. , Squartini, S. , and Piazza, F. A New System for Automatic Recognition of Italian Sign Language. In Neural Nets and Surroundings, Springer, pp. 69-79, 2013.
  21. Akyol, S. , and Alvarado, P. Finding relevant image content for mobile sign language recognition. International Conference-Signal Processing, Pattern Recognition and Applications, pp. 48-52, 2001.
  22. Kishore, P. V. V. and Kumar, P. R. Sign language video segmentation with level sets fusing color, texture, boundary and shape features. Journal of Signal and Image Processing, vol. 3, no. 3, 2012.
  23. Haberdar, H. and Albayrak S. A two-stage visual Turkish Sign Language recognition system based on global and local features. Lecture Notes in Artificial Intelligence vol. 4203, Springer-Verlag, pp. 29-37, 2006.
  24. Zhang, L. G. , Chen, Y. , Fang, G. , Chen, X. , and Gao, W. A vision-based sign language recognition system using tied-mixture density HMM. In Proc. of the 6th International Conference on Multimodal Interfaces pp. 198-204, 2004.
  25. Zieren, J. and Kraiss, K. F. Robust person-independent visual sign language recognition. Pattern recognition and image analysis, Springer, pp. 520-528, 2005.
  26. Assan, M. , and Grobel, K. Video-based sign language recognition using hidden markov models. In Gesture and Sign Language in Human-Computer Interaction, pp. 97-109, Springer Berlin Heidelberg, 1998.
  27. Zafrulla, Z. , Brashear, H. , Hamilton, H. , and Starner, T. A novel approach to American Sign Language phrase verification using reversed signing. Computer Vision and Pattern Recognition Workshops, pp. 48-55, 2010.
  28. Feris, R. , Turk, M. , Raskar, R. , Tan, K. H. , and Ohashi, G. Recognition of isolated fingerspelling gestures using depth edges. In Real-Time Vision for Human-Computer Interaction, Springer, pp. 43-56, 2005.
  29. Hienz, H. , Bauer, B. , and Kraiss, K. F. Video-based continuous sign language recognition using statistical methods. In Proc. of . 15th International Conference on Pattern Recognition, vol. 2, pp. 463-466, 2000.
  30. Dreuw, P. , Rybach, D. , Deselaers, T. , Zahedi, M. , and Ney, H. Speech recognition techniques for a sign language recognition system. Hand, 2007.
  31. Holden, E. J. , Lee, G. and Owens, R. Australian sign language recognition. Machine Vision and Applications, 16(5), pp. 312-320, 2005.
  32. Dreuw, P. Steingrube, P. , Deselaers, T. and Ney, H. Smoothed Disparity Maps for Continuous American Sign Language Recognition. Iberian Conf. on Pattern Recognition and Image Analysis, pp. 24-31, 2009.
  33. Wang, H. , Gao, W. , and Shan, S. An approach based on phonemes to large vocabulary Chinese sign language recognition. 5th Conf. on Automatic Face and Gesture Recognition, pp. 411-416, 2002.
  34. Nayak, S. , Sarkar, S. , and Loeding, B. Unsupervised modelling of signs embedded in continuous sentences. In Computer Vision and Pattern Recognition-Workshops, pp. 81-81, 2005.
  35. Bauer, B. and Hienz, H. Relevant features for video-based continuous sign language recognition. Int. Conference on Automatic Face and Gesture Recognition, pp. 440-445, 2000.
  36. Kelly, D. , Reilly Delannoy, J. , Mc Donald, J. , and Markham, C. A framework for continuous multimodal sign language recognition. In Proc. of Intl. Conference on Multimodal interfaces, pp. 351-358, 2009.
  37. Fang, G. , Gao, W. , and Zhao, D. Large-vocabulary continuous sign language recognition based on transition-movement models. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 37(1), pp. 1-9, 2007.
  38. Wilbur, R. and Kak, A. , "Purdue RVL-SLLL American Sign Language Database", School of Electrical and Computer Eng. Tech. Report, Purdue University, 2006.
  39. Dreuw, P. , Neidle, C. , Athitsos, V. , Sclaroff, S. , and Ney, H. Benchmark Databases for Video-Based Automatic Sign Language Recognition. In LREC, 2008.
  40. Johnston, T. , Schembri, A. , Adam, R. , Napier, J. , Thornton, D. Auslan SignBank: the Auslan lexical database. http://www. auslan. org. au/
  41. Schembri, A. , Fenlon, J. , Rentelis, R. , Reynolds, S. , and Cormier, K. Building the British Sign Language Corpus. Language Documentation and Conservation, vol. 7, pp. 136-154, 2013.
  42. Porfirio, A. , Wiggers, K. , Oliveira, S. , and Weingaertner, D. Libras sign language hand configuration recognition based on 3D meshes. IEEE Int. Conf. on Systems, Man, and Cybernetics, 2013.
  43. Efthimiou, E. , Fotinea, S. E. , Hanke, T. , Glauert, J. , Bowden, R. , Braffort, A. , and Goudenove, F. DICTA-SIGN: Sign language recognition, generation and modelling with application in Deaf communication. 4th Workshop on the Representation and Processing of Sign Languages, pp. 80-83, 2010.
  44. Crasborn, O. and Zwitserlood, I. The Corpus NGT: an online corpus for professionals and laymen. 3rd Workshop on the Representation and Processing of Sign Languages, 2008.
  45. von Agris, U. and Kraiss, K. F. Towards a video corpus for signer-independent continuous sign language recognition. In Proc. of Gesture in Human-Computer Interaction and Simulation. Int. Gesture Workshop, 2007.
  46. Efthimiou, E. and Fotinea, S. E. GSLC: creation and annotation of a Greek sign language corpus for HCI. In Universal Access in Human Computer Interaction, Springer Berlin Heidelberg, pp. 657-666, 2007.
  47. Kishore, P. V. V. , Kumar, P. R. , Kumar, E. K. , and Kishore, S. R C. Video Audio Interface for Recognizing Gestures of Indian Sign. International Journal of Image Processing, vol. 5, no. 4, pp. 479, 2011.
  48. Bungerot, J. , Stein, D. , Dreuw, P. , Ney, H. , Morrissey, S. , Way, A. , and van Zijl, L. The ATIS sign language corpus. 2008.
  49. Fagiani, M. , Principi, E. , Squartini, S. and Piazza, F. A New Italian Sign Language Database. In Proc. of International Conference on Brain Inspired Cognitive Systems (BICS), Shenyang, China, Jul. 11-14 2012.
  50. Kim, J. S. , Jang, W. , and Bien, Z. A dynamic gesture recognition system for the Korean sign language (KSL). Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 26(2), pp. 354-359, 1996.
  51. Terrillon, J. C. , Piplr, A. , Niwa, Y. , and Yamamoto, K. Robust Face Detection and Japanese Sign Language Hand Posture Recognition for Human-Computer Interaction in an Intelligent Room. In Proc. of Int'l Conf. Vision Interface, pp. 369-376, 2002.
  52. Al Qodri Maarif, H. , Akmeliawati, R. , Bilal, S. Malaysian Sign Language database for research. In Proc. of Computer and Communication Engineering, pp. 798-801, 3-5 July 2012.
  53. Kausar, S. , Javed, M. Y. , and Sohail, S. Recognition of gestures in Pakistani SL using fuzzy classifier. In Proc. of 8th Conf. on Signal Processing, Comp. Geometry and Artificial Vision, pp. 101-105, 2008.
  54. Karami, A. , Zanj, B. , and Sarkaleh, A. K. Persian sign language (PSL) recognition using wavelet transform and neural networks. Expert Systems with Applications, vol. 38(3), 2011.
  55. San-Segundo, R. , Pardo, J. M. , Ferreiros, J. , Sama, V. , Barra-Chicote, R. , Lucas, J. M. , and García, A. Spoken Spanish generation from sign language. Interacting with Computers, vol. 22(2), pp. 123-139, 2010.
  56. Aran, O. , Ari, I. , Guvensan, A. , Haberdar, H. , Kurt, Z. , Turkmen, I. , Uyar, A. , and Akarun, L. A Database of Non-Manual Signs in Turkish Sign Language. Signal Processing and Communications Applications, 2007.
  57. Tai, J. H. Y. and Tsay, J. S. Taiwan Sign Language Corpus: Digital Dictionary and Database. TELDAP Intl. Conference, Taipei, Taiwan, pp 41-47, 2010.
  58. Hieu, D. V. Vietnamese sign language recognition for hearing impaired people using fuzzy hidden Markov models. Master of Science Thesis, Department of Information Technology, King Mongkut's Univesity of Technology, 2008.
  59. Picone, J. W. Signal modeling techniques in speech recognition. Proceedings of the IEEE, vol. 81, no. 9, pp. 1215-1247, 1993.
  60. Efthimiou, E. , Fotinea, S. E. , Vogler, C. , Hanke, T. , Glauert, J. , Bowden, R. , and Segouat, J. Sign language recognition, generation, and modelling: a research effort with applications in deaf communication. In Universal Access in Human-Computer Interaction. Addressing Diversity, Springer Berlin Heidelberg, pp. 21-30, 2009.
  61. Rabiner, L. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), pp. 257-286, 1989.
  62. Starner, T. E. Visual recognition of American Sign Language using hidden Markov models. MIT, Cambridge, Dept of Brain and Cognitive Sciences, 1995.
  63. Ouhyoung, M. , and Liang, R. H. A sign language recognition system using hidden Markov model and context sensitive search. In Proc. of the ACM Symp. on virtual reality software and technology, pp. 59-66, 1996.
  64. Wang, H. , Leu, M. C. , and Oz, C. American Sign Language Recognition Using Multi-dimensional Hidden Markov Models. Journal of Information Science and Engineering, 22(5), pp. 1109-1123, 2006.
  65. Al-Rousan, M. , Assaleh, K. , and Tala'a, A. Video-based signer-independent Arabic sign language recognition using hidden Markov models. Applied Soft Computing, 9(3), 990-999, 2009.
  66. Haberdar, H. and Albayrak, S. Real Time Isolated Turkish Sign Language Recognition from Video Using Hidden Markov Models with Global Features. ISCIS, Lecture Notes in Computer Science, Springer, pp. 677-687, 2005
  67. Grobel, K. and Assan, M. Isolated sign language recognition using hidden Markov models. IEEE Intl. Conf. on Systems, Man, and Cybernetics, vol. 1, pp. 162-167, 1997.
  68. Hienz, H. , Bauer, B. , and Kraiss, K. F. , "HMM-based continuous sign language recognition using stochastic grammars", in Gesture-Based Communication in Human-Computer Interaction, Springer, pp. 185-196, 1999.
  69. Dreuw, P. , Deselaers, T. , Rybach, D. , Keysers, D. , and Ney, H. Tracking using dynamic programming for appearance-based sign language recognition. 7th International Conference on Automatic Face and Gesture Recognition, pp. 293-298, 2006.
  70. Fang, G. , Gao, X. , Gao, W. , and Chen, Y. A novel approach to automatically extracting basic units from Chinese sign language. In 17th International Conference on Pattern Recognition,. vol. 4, pp. 454-457, 2004.
  71. Hieu, D. V. and Nitsuwat, S. Image preprocessing and trajectory feature extraction based on hidden markov models for sign language recognition. In Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, pp. 501-506, 2008.
  72. Holden, E. J. , Lee, G. , and Owens, R. Australian sign language recognition. Machine Vision and Applications, vol. 16(5), pp. 312-320, 2005.
  73. . Kelly, D. , McDonald, J. , and Markham, C. Evaluation of threshold model HMMS and Conditional Random Fields for recognition of spatiotemporal gestures in sign language. 12th Int. Computer Vision Workshops, 2009.
  74. Eddy, S. R. What is a hidden Markov model?. Nature biotechnology, 22 (10), pp. 1315-1316, 2004.
  75. Evermann, G. , Kershaw, D. , Moore, G. , Odell, J. , Ollason, D. , Valtchev, V. , and Woodland, P. The HTK book, vol. Cambridge: Entropic Cambridge Research Laboratory, 1997.
Index Terms

Computer Science
Information Sciences

Keywords

International sign language sign language recognition deaf community survey of sign language recognition.

Powered by PhDFocusTM