The week's pick
Reseach Article
A Users Search History based Approach to Manage Revisit Frequency of an Incremental Crawler
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 63 - Number 3 |
| Year of Publication: 2013 |
| Authors: Yadu Nagar, Niraj Singhal |
10.5120/10446-5138
|
Yadu Nagar, Niraj Singhal . A Users Search History based Approach to Manage Revisit Frequency of an Incremental Crawler. International Journal of Computer Applications. 63, 3 ( February 2013), 18-22. DOI=10.5120/10446-5138
Abstract
With the tremendous growth of the Internet, World Wide Web has become a huge source of hyperlinked information contained in hypertext documents. Search engines use web crawlers to collect these documents from web for the purpose of storage and indexing. An incremental crawler visits the web for updating its collection. There is a need to regulate the frequency of the crawler to visit web sites and provide latest information to the user. In this paper a novel approach to manage the revisiting frequency of an incremental crawler based on the users search history is being proposed.
References
- Niraj Singhal, Ashutosh Dixit and A. K. Sharma, "Design of A Priority Based Frequency Regulated Incremental Crawler", Published in International Journal of Computer Applications (IJCA), Volume 1–No. 1, Article 8, Harvard Press US 2010. ISSN: 0975–8887, pp 47-52, Feb 2010.
- A. K. Sharma, J. P. Gupta and D. P. Agarwal, " A novel approach towards management of Volatile Information" Journal of CSI, Vol. 33 No. 1, pp 18-27, Sept 2003.
- Alexandros Ntoulas, Junghoo Cho and Christopher Olston,"What's new on the Web ? The Evolution of the Web from a Search Engine perspective", In Proceedings of the World-Wide Web Conference (WWW), May 2004.
- Arvind Arasu, Junghoo Cho, Hector Garcia-Molina, Andreas Paepcke and Sriram Raghavan "Searching the Web" ACM Transactions on Internet Technology, 1(1): August 2001.
- Brian E. Brewington and George Cybenko. "How dynamic is the web. ", In Proceedings of the Ninth International World-Wide Web Conference, Amsterdam, Netherlands, May 2000.
- Junghoo Cho and Hector Garcia-Molina, "The evolution of the web and implications for an incremental crawler", In Proceedings of the 26th International Conference on Very Large Databases, 2000.
- Junghoo Cho and Hector Garcia-Molina, "Estimating frequency of change", 2000, submitted to VLDB, Research track,2000.
- Sergey Brin and Lawrence Page, "The anatomy of a large scale hyper textual Web search engine". Proceedings of the Seventh International World Wide Web Conference, pp 107-117, April 1998.
- F. Douglis, A. Feldmann, and B. Krishnamurthy," Rate of change and other metrics : a live study of the world wide web" In Proceedings of the USENIX Symposium on Internet Technologies and Systems, Monterey, California, Dec. 1997.
- D. Fetterly, M. Manasse, M. Najork, and J. L. Wiener," A large-scale study of the evolution of web pages", In Proceedings of the Twelfth International World Wide Web Conference, Budapest, Hungary, May 2003.
- Niraj Singhal and Ashutosh Dixit," Need of Search Engines and Role of a Web Crawler",National Conference on Recent Trends in Computers and IT (RTCIT-09),Samalkha, Haryana, 24th-25th April 2009.
- Ashutosh Dixit, Harish Kumar and A. K Sharma, "Self Adjusting Refresh Time Based Architecture For Incremental Web Crawler", International Journal of Computer Science and Network Security (IJCSNS), Vol 8, No12, Dec 2008.
- Niraj Singhal, Ashutosh Dixit and R. P. Agarwal, A. K. Sharma, "Regulating Frequency of a Migrating Web Crawler based on Users Interest", published in International Journal of Engineering and Technology (IJET), Vol. 4, No. 4, Aug-Sep 2012, ISSN : 0975-4024, pp. 246-253.
- Arun Kumar Singh and Niraj Singhal, "A Novel Page Rank Algorithm for Web Mining based on User's Interest", International Journal of Emerging Technology and Advanced Engineering, ISSN 2250-2459, Vol. 2, Issue 9, pp. 395-400, September 2012.
Index Terms
Keywords