The week's pick
Random Articles
Reseach Article
Hybrid Transformer–Recurrent Modelling for Sentiment Analysis in Low Resource Language
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 187 - Number 86 |
| Year of Publication: 2026 |
| Authors: Sheezan Farooq, Rumaan Bashir |
10.5120/ijca2026926485
|
Sheezan Farooq, Rumaan Bashir . Hybrid Transformer–Recurrent Modelling for Sentiment Analysis in Low Resource Language. International Journal of Computer Applications. 187, 86 ( Mar 2026), 14-18. DOI=10.5120/ijca2026926485
Abstract
Sentiment analysis in low-resource languages like Kashmiri is underexplored due to the lack of annotated datasets and computational tools. This research proposes an effective hybrid deep learning model that combines the strengths of XLM-RoBERTa and BiLSTM networks for sentiment analysis of Kashmiri language text. The Kashmiri language, being low-resource and morphologically rich, poses significant challenges for natural language understanding tasks. A major contribution of this work is the creation of a manually annotated sentiment dataset tailored for the Kashmiri language, encompassing positive, negative, and neutral sentiment categories. This dataset serves as a foundational resource for training and evaluating sentiment classification models in this underrepresented language. The hybrid model combines XLM-RoBERTa transformer contextual embeddings with BiLSTM modeling sequences because of its ability to function in low-resource environments. Experimental results demonstrate that the hybrid model achieves state-of-the-art performance with a validation accuracy of 94.7% and an F1-score of 0.94 across all sentiment classes. Additionally, ROC analysis confirms high discriminative ability with an AUC of 0.99 for each class. Experimental findings demonstrate that integrating pre-trained transformers with recurrent models improves sentiment recognition abilities in Kashmiri by a significant degree.
References
- N. C. Dang, M. N. Moreno-García, and F. De la Prieta, “Sentiment analysis based on deep learning: A comparative study,” Electronics (Switzerland), vol. 9, no. 3, Mar. 2020, doi: 10.3390/electronics9030483.
- K. Rakshitha, R. H M, M. Pavithra, A. H D, and M. Hegde, “Sentimental analysis of Indian regional languages on social media,” Global Transitions Proceedings, vol. 2, no. 2, pp. 414–420, Nov. 2021, doi: 10.1016/j.gltp.2021.08.039.
- W. Medhat, A. Hassan, and H. Korashy, “Sentiment analysis algorithms and applications: A survey,” Ain Shams Engineering Journal, vol. 5, no. 4, pp. 1093–1113, Dec. 2014, doi: 10.1016/j.asej.2014.04.011.
- M. R. R. Rana, A. Nawaz, and J. Iqbal, “A survey on sentiment classification algorithms, challenges and applications,” Acta Universitatis Sapientiae, Informatica, vol. 10, no. 1, pp. 58–72, Aug. 2018, doi: 10.2478/ausi-2018-0004.
- N. A. Lone, K. J. Giri, and R. Bashir, “Natural Language Processing Resources for the Kashmiri Language,” Indian J Sci Technol, vol. 15, no. 43, pp. 2275–2281, Nov. 2022, doi: 10.17485/IJST/v15i43.1964.
- S. Anbukkarasi and S. Varadhaganapathy, “Analyzing Sentiment in Tamil Tweets using Deep Neural Network,” in Proceedings of the 4th International Conference on Computing Methodologies and Communication, ICCMC 2020, Institute of Electrical and Electronics Engineers Inc., Mar. 2020, pp. 449–453. doi: 10.1109/ICCMC48092.2020.ICCMC-00084.
- M. R. H. Khan, U. S. Afroz, A. K. M. Masum, S. Abujar, and S. A. Hossain, “Sentiment Analysis from Bengali Depression Dataset using Machine Learning,” in 2020 11th International Conference on Computing, Communication and Networking Technologies, ICCCNT 2020, Institute of Electrical and Electronics Engineers Inc., Jul. 2020. doi: 10.1109/ICCCNT49239.2020.9225511.
- K. Shrivastava and S. Kumar, “A sentiment analysis system for the hindi language by integrating gated recurrent unit with genetic algorithm,” International Arab Journal of Information Technology, vol. 17, no. 6, pp. 954–964, Nov. 2020, doi: 10.34028/iajit/17/6/14.
- M. Ahmad, S. Aftab, and I. Ali, “Sentiment Analysis of Tweets using SVM,” Int J Comput Appl, vol. 177, no. 5, pp. 25–29, Nov. 2017, doi: 10.5120/ijca2017915758.
- N. Dhamayanthi and B. Lavanya, “The Role Of Naïve Bayes, SVM, And Decision Trees In Sentiment Analysis,” Theory and Practice, vol. 2024, no. 4, pp. 6377–6381, 2024, doi: 10.53555/kuey.v30i4.2392.
- J. Y. Le Chan, K. T. Bea, S. M. H. Leow, S. W. Phoong, and W. K. Cheng, “State of the art: a review of sentiment analysis based on sequential transfer learning,” Artif Intell Rev, vol. 56, no. 1, pp. 749–780, Jan. 2023, doi: 10.1007/s10462-022-10183-8.
- L. Khan, A. Amjad, N. Ashraf, and H. T. Chang, “Multi-class sentiment analysis of urdu text using multilingual BERT,” Sci Rep, vol. 12, no. 1, Dec. 2022, doi: 10.1038/s41598-022-09381-9.
- A. S. Talaat, “Sentiment analysis classification system using hybrid BERT models,” J Big Data, vol. 10, no. 1, Dec. 2023, doi: 10.1186/s40537-023-00781-w.
- Md. S. Islam and K. M. Alam, “Sentiment analysis of Bangla language using a new comprehensive dataset BangDSA and the novel feature metric skipBangla-BERT,” Natural Language Processing Journal, vol. 7, p. 100069, Jun. 2024, doi: 10.1016/j.nlp.2024.100069.
- R. Cheruku, K. Hussain, I. Kavati, A. M. Reddy, and K. S. Reddy, “Sentiment classification with modified RoBERTa and recurrent neural networks,” Multimed Tools Appl, vol. 83, no. 10, pp. 29399–29417, Mar. 2024, doi: 10.1007/s11042-023-16833-5.
- M. U. Salur and I. Aydin, “A Novel Hybrid Deep Learning Model for Sentiment Classification,” IEEE Access, vol. 8, pp. 58080–58093, 2020, doi: 10.1109/ACCESS.2020.2982538.
- S. Zhou, Q. Chen, X. Wang, and X. Li, “Hybrid Deep Belief Networks for Semi-supervised Sentiment Classification.”
- J. Blitzer, M. Dredze, and F. Pereira, “Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification,” Association for Computational Linguistics, 2007. [Online]. Available: http://ida.
Index Terms
Keywords