Web scraping | Sololearn: Learn to code for FREE!


Web scraping

Hello friends! Last few days I was working on developement of some simple scraping codes in python. I was thinking if its possible to make some script challenge where lot of coders can join, post their codes and discuss about given solutions? Other thing, is there any programer who did some web scripts that is willing to share it with us here? I will be very pleased. I am asking this since data extraction and web scraping is very interesting branch of programming and I feel like we don't give enough attention to it!

8/27/2018 9:18:09 AM

Luka Vasiljevic

4 Answers

New Answer


I made a movie web scraper for kodi with python. It uses one scraper to pull the movie informations and 2 scrapers to scrape the embedded source links. There’s only two scrapers for the source links for now. But I could make 5. Make each run in a new thread and have one powerfull scraper.


A year ago, I created a profile list for a Sololearn contest, by web scraping the Sololearn profiles. https://code.sololearn.com/WrHot3955gBr/?ref=app


i watch youtube tuturial videos a lot, there are so many Professional developers making helpful stuffs for free, but my wifi is crazy and I dont want to load the videos every time to waste data. so its not much but a life saver shell script to download youtube videos for me step 1. $sudo apt-get install youtube-dl step 2: save the shell script written below in a folder where u want videos with file name dwnld. sh step 3: in same folder compile a list of links of all youtube videos you wanna download in a single go with file name links. txt step 4: run the script by typing . /dwnld.sh it basically saves me from doing $youtube-dl - f 22 video _ link (enter) hundred times and waiting till one file downloads to start for another I do this everytime when I see a tuturial series, if anyone has better idea please suggest, coz right now the only boring task is copy paste the links. https://code.sololearn.com/c0u56B1Z2wf5/?ref=app


Not mine but someone else's code. https://code.sololearn.com/c9YHb9DfYHUn/?ref=app When i was learning about scraping using nokogiri in ruby i used the below resource among others to learn. https://thaiwood.io/screen-scraping-with-a-saw-a-nokogiri-tutorial-with-examples/