New course! Every coder should learn Generative AI!
Try a free lesson+ 16
Python is really good for webscraping and crawling. Its scrapy and beautifulsoup4 libraries are great, easy and intuitive tools for even commercially viable webscraping.
+ 8
Depending on how complex your program is file_get_contents and explode() do the job for me.
+ 6
Disclaimer: I️ have never used a language for web scrapping.
I️ would rule out any language that does not support parallelism as many computer these days have multiple cores.
Most languages will probably be fast enough for your needs, however I️ would recommend c and golang for speed.
+ 4
I agree with Kuba Siekierzyński beautifulsoup is training 1/2 an hour, if you ha e basic knowlesge of Python
+ 2
I have done web scraping with PHP (cURL) and Python (requests, beautifulSoup, Selenium) so far and Python is my preferred choice.
Be aware that good web scraping is most often not about speed. Depending on your needs, it might be to look as human (check your headers, referrers etc.) as possible and to be intentionally slower than possible in order to not being detected if your IP is hitting the target too many times each minute!
+ 2
Since you are asking for "FASTEST", people share good experiences with GO scrappers:
Simple web scraper:
https://schier.co/blog/2015/04/26/a-simple-web-scraper-in-go.html
List of scrapers and related tools:
https://github.com/lorien/awesome-web-scraping/blob/master/golang.md
Web Scraper in Go, similar to BeautifulSoup
https://github.com/anaskhan96/soup
How to speed it up?
https://juliensalinas.com/en/how-to-speed-up-web-scraping-with-go-golang-concurrency/
Go is so much faster than Python or Ruby and uses so much less memory than Java, it’s really the natural choice for both enterprises and startups. — Nate Finch.
+ 2
Kuba Siekierzyński please make a course on web scraping. I really loved your explanation in Google Maps course
+ 1
in my opinion, it's python
+ 1
The speed is not necessary something you really need if by speed refers to the peed of a binary executable compared to scripts.
I have seen many languages used for web scraping but I saw a YouTube tutorial for using Ruby for just that purpose that blow my mind. The thing was not speed but the ability to load different "modules" that was for example behaving like a Firefox browser without the usual interface and the ability to assign what it fetched to suitable variables for processing according to your needs. The simplicity and almost exactly "human" like interaction (which could be tweaked further) from the "targets" perspective made me realize the wast capabilities it presented.
0
Python Is the Best for web scrapping and crawling . and also it is easy to learn as compare to other programming languages like java , c++ etc.
0
google it, you will get perfect answer!
0
Python is the best.
Often have questions like this?
Learn more efficiently, for free:
