The week's pick
Reseach Article
Assessing the Effectiveness of Various Text Classification Algorithms in Customer Complaint Classification: An Informative Resource for Data Scientists and Data Analysts
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 186 - Number 2 |
| Year of Publication: 2024 |
| Authors: Yehia Helmy, Merna Ashraf, Laila Abdelhamid |
10.5120/ijca2024923346
|
Yehia Helmy, Merna Ashraf, Laila Abdelhamid . Assessing the Effectiveness of Various Text Classification Algorithms in Customer Complaint Classification: An Informative Resource for Data Scientists and Data Analysts. International Journal of Computer Applications. 186, 2 ( Jan 2024), 8-16. DOI=10.5120/ijca2024923346
Abstract
Due to the numerous issues or challenges that aren't always within the company's control. Customers became unhappy. Customer complaint is the method by which they convey their dissatisfaction. Due to the rapid advancement of technology and the various convenient channels available for customers to voice their complaints, including email, web, and chatbots, online complaints have experienced exponential growth. As a result, classifying these complaints under the pertinent issue in time became a difficult task. Selecting the appropriate classification model and Fitting it with the proper training and testing ratios is a crucial topic that always faces researchers. This paper implements and compares the performance of six text classification machine learning algorithms used in multi-classification (SVM, KNN, NB, DT, RF, and GB) under two types of sampling (random and stratified) with the use of various data splitting ratios 50:50,80:20, 60:40, 70:30, and 90:10 on a Complaint Dataset. This paper aims to provide a roadmap for researchers working in the text classification field that helps them select the optimum classification model and splitting ratio. The results demonstrate that DT with an accuracy of 99%, F1-measure of 99%, and runtime of 1 second outperformed all other algorithms. And that the most suitable splitting ratio that fits most algorithms and acts as a secure base to work with is 80:20. It also indicates that using stratified sampling in multi-class text classification produces better results than random sampling.
References
- Ali, M., et al. (2019). Classifying Arabic farmers’ complaints based on crops and diseases using machine learning approaches. Recent Trends in Image Processing and Pattern Recognition: Second International Conference, RTIP2R 2018, Solapur, India, December 21–22, 2018, Revised Selected Papers, Part III 2, Springer.
- Anwar, M. T., et al. (2021). "Automatic Complaints Categorization Using Random Forest and Gradient Boosting." 3(1): 210106.
- Arusada, M. D. N., et al. (2017). Training data optimization strategy for multiclass text classification. 2017 5th International Conference on Information and Communication Technology (ICoIC7), IEEE.
- Bazzan, J., et al. (2023). "An Information Management Model for Addressing Residents’ Complaints through Artificial Intelligence Techniques." 13(3): 737.
- BOZYİĞİT, F., et al. (2022). "Categorization of customer complaints in food industry using machine learning approaches." 5(1): 85-91.
- Choi, C. (2018). Predicting customer complaints in mobile telecom industry using machine learning algorithms, Purdue University.
- Endut, N., et al. (2022). "A Systematic Literature Review on Multi-Label Classification based on Machine Learning Algorithms." 11(2): 658.
- Goncarovs, P. (2019). Active learning svm classification algorithm for complaints management process automatization. 2019 60th International Scientific Conference on Information Technology and Management Science of Riga Technical University (ITMS), IEEE.
- HaCohen-Kerner, Y., et al. (2019). "Automatic classification of complaint letters according to service provider categories." 56(6): 102102.
- Hasan, T., et al. (2020). Machine learning based automatic classification of customer sentiment. 2020 23rd International Conference on Computer and Information Technology (ICCIT), IEEE.
- Kadhim, A. I. J. A. I. R. (2019). "Survey on supervised machine learning techniques for automatic text classification." 52(1): 273-292.
- Kalra, V. and R. Aggarwal (2017). Importance of Text Data Preprocessing & Implementation in RapidMiner. ICITKM.
- Li, L. and W. J. T. v. Li (2019). "Naive Bayesian automatic classification of railway service complaint text based on eigenvalue extraction." 26(3): 778-785.
- Miner, G. (2012). Practical text mining and statistical analysis for non-structured text data applications, Academic Press.
- Muraina, I. (2022). Ideal dataset splitting ratios in machine learning algorithms: general concerns for data scientists and data analysts. 7th International Mardin Artuklu Scientific Research Conference.
- Naseem, U., et al. (2021). "A comprehensive survey on word representation models: From classical to state-of-the-art word representation language models." 20(5): 1-35.
- Sen, P. C., et al. (2020). Supervised classification algorithms in machine learning: A survey and review. Emerging technology in modelling and graphics, Springer: 99-111.
- Tharwat, A. J. A. C. and Informatics (2020). "Classification assessment methods."
- Tufail, A. B., et al. (2020). "Binary classification of Alzheimer’s disease using sMRI imaging modality and deep learning." 33: 1073-1090.
- Xu, S., et al. (2017). Bayesian multinomial Naïve Bayes classifier to text classification. Advanced Multimedia and Ubiquitous Engineering: MUE/FutureTech 2017 11, Springer.
Index Terms
Keywords