What exactly is a web crawler program? | Sololearn: Learn to code for FREE!
+ 1

What exactly is a web crawler program?

can someone explain to me what a web crawler is and how it is developed??

30th Jan 2017, 2:20 PM
Glenn Bennett
Glenn Bennett - avatar
3 Respuestas
+ 1
ahh well that makes a lot of sense now thank you
30th Jan 2017, 2:36 PM
Glenn Bennett
Glenn Bennett - avatar
0
a crawler is a program, script or whatever that browse in the world wide wide in a automated way, a example of crawler is the google bot, the google bot browse in the world wide web for create a copy of the web pages to be processed for the google search engine later. For example a spammer can use a web crawler for search e-mails in the WWW and later automatically send message for these! PS: sorry for my bad english, i'm brazilian!
30th Jan 2017, 2:34 PM
N1NJ4BR
0
A web crawler is a piece of code written in a particular programming language (c, java, Phyton, php). The main task performed by a web crawler is to extract the information from a single web page or multiple web pages. The information could be the title of the web page or the links contained by that particular web page or any other associated with that web page. Google is a good example which uses web crawler.
18th Feb 2017, 5:49 AM
Yash Sankhala
Yash Sankhala - avatar