Implementation of Mobile Optimized Search Crawler
Keywords:
Search engine, Search Crawler, Keyword density, Mobile-optimized websiteAbstract
The web crawler is the central component of a search engine which works like an indexer, finds out hyperlinks, computes keyword density of each web page and stores the visited links for the future use. Today's world is mobile world! Aodhan Cullen, CEO, StatCounter stated that “Mobile has grown rapidly from 17.1% to 28.5% in past 12 months. Mobile usage has already overtaken desktop in several countries including India, South Africa and Saudi Arabia[1]”. Due to increasing use of mobile, there is a need to develop a search crawler which will mainly focus on the mobile devices and provide them quality results in less time. The proposed system, “Mobile Optimized Search Crawler” is a crawler that mainly focuses on providing quick and relevant results to the mobile users. It gives more priority to the websites which are optimized for mobile devices than the websites which are not. It uses more than 50 factors to determine the relevancy of page, giving more weightage to the mobile-optimized factor so that mobile users get websites which are user-friendly and rich in knowledge.
References
“StatCounter”, http://gs.statcounter.com/press/mobile-internet-usage-soars-by-67-perc , 19 Dec, 2015.
“Search Engine”,
http://websearch.about.com/od/enginesanddirectories/a/searchengine.htm , 15 July, 2015.
Feng Zhao, Jingyu Zhou, Chang Nie, Heqing Huang and Hai Jin, “SmartCrawler : A two-stage crawler for efficiently harvesting deep-web interfaces”, Services Computing, IEEE Transactions, Volume-PP, Issue-99, DOI-10.11.09, 2015.
Mehdi Bahrami, Mukesh Singhal and Zixuan Zhuang, “A Cloud-based Web Crawler Architecture”, Intelligence in Next Generation Networks (ICIN), 2015 18th International Conference, Page No (216-233), 17-19 Feb 2015.
Pavalam S.M, S.V. KumarRaja, M. Jawhar and Felix K. Akorli, “Web crawler in mobile systems”, International Journal of Machine Learning and Computing, Volume-02, Issue-04, Page No (531-534), August 2012.
Vladislav Shkapenyuk and Torsten Suel, “Design and implementation of a high-performance distributed Web crawler”, Data Engineering, 2002. IEEE Proceedings. 18th International Conference, Page No (357-368), 2002.
Hardik P. Trivedi, Gaurav N. Daxini, Jignesh A. Oswal, Vinay D. Gor and Swati Mali, “An Approach to Design Personalized Focused Crawler”, International Journal of Computer Sciences and Engineering, Volume-02, Issue-03, Page No (144-147), March 2014.
“Google Ranking Factors: The Complete List”,
http://www.backlinko.com/google-ranking-factors ,
Feb, 2016.
“Apache Hadoop”, http://www.hadoop.apache.org ,
March, 2016.
“Apache HBase”, http://www.hbase.apache.org ,
March, 2016.
“Apache Spark”, http://www.spark.apache.org ,
March, 2016.
“Java Servlets”,
http://docs.oracle.com/javaee/5/tutorial/doc/bnafd.html ,
March, 2016.
“Java Server Pages”,
http://docs.oracle.com/javaee/5/tutorial/doc/bnagx.html ,
March, 2016.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors contributing to this journal agree to publish their articles under the Creative Commons Attribution 4.0 International License, allowing third parties to share their work (copy, distribute, transmit) and to adapt it, under the condition that the authors are given credit and that in the event of reuse or distribution, the terms of this license are made clear.
