Automatic sorting of documents is progressively becoming vital because manual handling and organization of documents is not a feasible solution as it can be very time consuming given the number of documents. One of the machine learning applications – text classification which is employed for document classification is explored in this paper. Generative learning algorithm – Naïve Bayes classifier is discussed in this paper. Documents from the 20 Newsgroups dataset are distributed in two groups. Group 1 consists of relatively unrelated two categories of documents and group 2 consists of relatively similar two categories of documents. Naïve Bayes classifier - Multinomial model is implemented to perform classification on both groups and it is observed that Accuracy can be improved with increasing the training set size for both the groups and Classification accuracy is higher for category of documents with lower similarity.

1. Karl - Michael “Techniques for Improving the performance of Naïve Bayes for text Classification” University of Passau, department of general Linguistics Innstr. 40, 94032 Passau, Germany
2. Kevin P. Murphy, “ Naïve Bayes classifier”, Department of Computer Science, University of British Columbia
3. Andrew McCallum and Kamal Nigam, “A Comparison of Event Models for Naïve Bayes Text Classification ”, In: Learning for Text Categorization: Papers from the AAAI workshop, AAAI pressc(1998) 41 – 48 Technical report Ws – 98 - 05
4. “Text Classification using Naïve Bayes”, Steve Renals, Learning and Data lecture 7, Informatics 2B, http://www.inf.ed.ac.uk/teaching/courses/inf2b/learnnotes/inf2b11-learnlec07-nup.pdf
5. S. A. Noah and F. Ismail, “Automatic Classification of Malay Proverbs using Naïve Bayesian Algorithm”, in Information Technology Journal 7 (7): 1016-1022, 2002 ISSN 1812-5638
6. Carl Liu, “Experiments on Spam Detection with Boosting, SVM and Naïve Bayes”, CMPS 242, Final project, Winter 2008, UCSC
7. “Generative learning algorithm”, lecture notes2 for CS229, Department of Computer Science, University of Stanford. Available online: http://www.stanford.edu/class/cs229/notes/cs229-notes2.pdf
8. George Tzanis, Ioannis Katakis, Ioannis Partalas, Ioannis Vlahavas, “ Modern Applications of Machine Learning”, in Proceedings of the 1st Annual SEERC Doctoral Student Conference – DSC 2006

Naive Bayes Multinomial Model