Which is best for web scraping beautiful soup /selenium /scrapy !? | Sololearn: Learn to code for FREE!


Which is best for web scraping beautiful soup /selenium /scrapy !?

Actually i learned selenium and beautiful soup but i don't know scrapy May be suggest some web scraping ideas??!!

8/26/2020 4:56:42 PM


1 Answer

New Answer


It is tough to answer questions like this on determining "best". I like them all and it depends exactly what you're trying to do. I like BeautifulSoup for extracting text, the cool thing about it is you can then pass the text into services like AWS and have the results sent through SMS. I like Selenium for automating tasks through a simulated web browser. Scrapy is a little more robust version of BeautifulSoup allowing for pipelines, feed exports, selectors, etc and I believe Scrapy uses BeautifulSoup as its backbone