The week's pick
Random Articles
Reseach Article
Real Time Indian Sign Language Detection using LSTM and Keypoint Extraction
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 184 - Number 21 |
| Year of Publication: 2022 |
| Authors: Ezhil Tharsan S., Dharshan S., Dinesh G., Saraswathi S. |
10.5120/ijca2022922235
|
Ezhil Tharsan S., Dharshan S., Dinesh G., Saraswathi S. . Real Time Indian Sign Language Detection using LSTM and Keypoint Extraction. International Journal of Computer Applications. 184, 21 ( Jul 2022), 1-7. DOI=10.5120/ijca2022922235
Abstract
Hearing and speech impaired people find it challenging to converse with ordinary. In India, about 63 million (6.3%) of the population suffers from auditory loss. They communicate through sign languages, which are based on visually conveyed sign patterns, which typically include hand movements. Sign language is not universal and not easy to learn. Many Artificial Intelligence (AI) based systems are present for the Sign Language conversion as a typical solution to this problem. But, most of the existing solutions, use American Standard Sign Language (ASL) . As per the research done for this project, Each country has its own sign language variant and in India, Indian Sign Language (ISL) is being followed. The major goal of the proposed system is to create an ISL Hand Gesture Motion Translation Tool that will assist the hearing and speech challenged community in transforming their voice and ideas into text so that they may communicate with others without huddles. The methodology is to identify isolated words and to design a dynamic hand gesture recognition system for ISL. The proposed system is built using Mediapipe which is used for key-point extraction combined with a Long Short-Term Memory (LSTM), a Recurrent Neural Network (RNN) architecture system design with Dense layers. It is capable to identify ISL Alphabets and words in real time environment.
References
- Bencherif M A. (2021) ‘Arabic Sign Language Recognition System Using 2D Hands and Body Skeleton Data’, IEEE Access, Vol. 9, pp. 59612-59627.
- Dhiman R., Joshi G. and Rama Krishna C. (2021) ‘A deep learning approach for Indian sign language gestures classification with different backgrounds’, Jouranal of Physics: Conference Series, ICMAI, pp. 1742-6596.
- Fernandes L, Dalvi P, Junnarkar and Bansode M. (2020) ‘Convolutional Neural Network based Bidirectional Sign Language Translation System’, Proc. Third International Conference on Smart Systems and Inventive Technology, pp. 769-775.
- Giulio Marin, Fabio Dominio and Pietro Zanuttigh. (2014) ‘Hand Gesture Recognition with Leap Motion and Kinect Devices’, Proc. IEEE International Conference on Image Processing (ICIP), pp. 1565-1569.
- Intwala N., Banerjee A., Meenakshi and Gala N. (2019) ‘Indian Sign Language converter using Convolutional Neural Networks’, Proc. IEEE 5th International Conference for Convergence in Technology (I2CT), pp. 1-5.
- Kar A. and Chatterjee P.S. (2015) ‘An Approach for Minimizing the Time Taken by Video Processing for Translating Sign Language to Simple Sentence in English’, Proc. International Conference on Computational Intelligence and Networks, pp. 172-177.
- Kim C.J. and Park H.M. (2021) ‘Per-frame Sign Language Gloss Recognition’, Proc. International Conference on Information and Communication Technology Convergence, pp. 1125-1127.
- Lee B.G. and Lee S.M. (2018) ‘SmartWearable Hand Device for Sign Language Interpretation System With Sensors Fusion’, IEEE Sensors Journal, Vol. 18, No. 3, pp. 1224-1232.
- Liu T., Zhou W. and Li H. (2016) ‘Sign language recognition with long short-term memory’, Proc. IEEE International Conference on Image Processing (ICIP), pp. 2871-2875.
- Mathavan Suresh Anand, Nagarajan Mohan Kumar and Angappan Kumaresan. (2016) ‘An Efficient Framework for Indian Sign Language Recognition Using Wavelet Transform’, Circuits and Systems and Scientific Research Publishing, Vol. 07, pp. 1874-1883.
- Muthukumar K., Poorani and Gobinath S. (2018) ‘Extraction of Hand Gesture Features for Indian Sign languages using Combined DWT-DCT and Local Binary Pattern’, International Journal of Engineering Technology, Vol. 07, pp. 316- 320.
- Pahuja D. and Jain S. (2020) ‘Recognition of Sign Language Symbols using Templates’, Proc. International Conference on Reliability, pp. 1157-1160.
- Pan W., Zhang X. and Ye Z. (2020) ‘Attention-Based Sign Language Recognition Network Utilizing Keyframe Sampling and Skeletal Features’, IEEE Access, Vol. 08, pp. 215592-215602.
- Shrinidhi Gindi, Amina Bhatkar, Hasan Haider S. and Ramsha Ansari. (2020) ‘Indian Sign Language Translator Using Residual Network and Computer Vision’, International Research Journal of Engineering and Technology, Vol. 07, No.07.
- Vipul Brahmankar, Nitesh Sharma, Saurabh Agrawal, Saleem Ansari, Priyanka Borse and Khalid Al Fatemi. (2021) ‘Indian Sign Language Recognition Using Canny Edge Detection’, International Journal of Advanced Trends in Computer Science and Engineering, Vol. 10, No. 03.
Index Terms
Keywords