The dynamic web has increased exponentially over the past few years with more than thousands of documents related to a subject available to the user now. Most of the web documents are unstructured and not in an organized manner and hence user facing more difficult to find relevant documents. A more useful and efficient mechanism is combining clustering with ranking, where clustering can group the similar documents in one place and ranking can be applied to each cluster for viewing the top documents at the beginning.. Besides the particular clustering algorithm, the different term weighting functions applied to the selected features to represent web document is a main aspect in clustering task. Keeping this approach in mind, here we proposed a new mechanism called Tf-Idf based Apriori for clustering the web documents. We then rank the documents in each cluster using Tf-Idf and similarity factor of documents based on the user query. This approach will helps the user to get all his relevant documents in one place and can restrict his search to some top documents of his choice. For experimental purpose, we have taken the Classic3 and Classic4 datasets of Cornell University having more than 10,000 documents and use gensim toolkit to carry out our work. We have compared our approach with traditional apriori algorithm and found that our approach is giving better results for higher minimum support. Our ranking mechanism is also giving a good F-measure of 78%.

1. Naresh Barsagade. Web usage mining and pattern discovery: A survey paper. CSE8331, Dec, 2003.
2. A. Jain, M. N. Murty, and P. Flyn. Data clustering: A review. ACM Computing Surveys, vol.no. 3, pp. 264323, 1999. Patrick Pantel and D. Lin. Discovering Word Senses from Text, SIGKDD’02, July 23-26, 2002, Edmonton, Ablerta, Canada.
3. J. Chen, O. R. Zaiane and R. Goebel. An Unsupervised Approach to Cluster Web Search Results based on Word Sense Communities, 2008 IEEE/WIC/ACM, International Conference on Web Intelligence and Intelligent Agent Technology.
4. Doreswamy and Hemanth K.S. A Novel Design Specification(DSD) based K-mean clustering performance Evaluation on Engineering material’s database, IJCA, Vol 55, No.15, Oct-2012.
5. S.Chakrabarti. Mining the Web: Discovering Knowledge from Hypertext Data. Morgan Kaufmann Publishers 2003.
6. Mansaf Alam and Kishwar Sadf. Web search result clustering using heuristic search and latent semantic indexing, IJCA, Vol 44, No.15, April-2012.
7. Li p, Wang B and Jin W. Improving web document clustering through employing user-related tag expansion techniques.Journal of Computer Science and Technology 27(3):554-556 May-2012. DOI 10.1007/ S11390-012-1243-4.
8. Ingyu Lee and B-Won. An Effective web document clustering algorithms based on bisection and merge, Artif Intell Rev(2011) 36-69-85, DOI 10.1007/S10462- 011-9203-4.
9. R.Thiyagarajan, K.Thangavel and R.Rathipriya. IJCA Vol-86, No.14, Jan-2014.
10. http://www.miislita.com/term-vector/term-vector-3.html
11. http://www.nlp.fi.muni.cz/projekty/gensim/intro.html
12. http://tartarus.org/martin/PorterStemmer/def.txt
13. Introduction to Data Mining by P.Tan, M. Steinbach and Vipin Kumar, Pearson Education, 2006.
14. Khaled M and Mohamed S. Efficient phrased-based document indexing for web document clustering, IEEE Transaction on Knowledge and Data Engineering, Vol 16, No. 10, October-2004.
15. B. Shanmugapriya and M.Punithavalli. A modified projected K-means clustering algorithm with effective distance measure, International Journal of Computer Application, Vol 44, No.8, April-2012.
16. http://www.dataminingresearch.com/index.php/2010/09/ classic3-classic4-datasets
17. Jaroslar Pokorhy, Jozef Smizansky. Page Content Rank, An approach to the web content mining.
18. http://en.wikipedia.org/wiki/F1_score

Apriori Clustering Gensim Ranking Vector Space Model