Help me to print useful covid19 data from website using BeautifulSoup
I want to print state vise data of covid19 cases on my terminal screen so I made a program which scrap useful data from website and print it. like this : ' ' ' name of state : Maharashtra #(india) Active cases : xxxxxxxx Cured/discharge : xxxxxxxxx Deaths : xxxxxxx Total cases : xxxxx ' ' ' before some days my program was working fine but now it's not working their is no output. what is the problem? please help me to solve it.. I am stuck..! website from where I am fetching data : view-source : https://www.mohfw.gov.in my attempt : Edited : new code https://code.sololearn.com/cdEM3DJm64E6/?ref=app
11/26/2020 5:28:58 PMRatnapal Shende
19 AnswersNew Answer
Ratnapal Shende JSON is similar to working with a nested dictionary. You could actually pull the file in and store it in a dictionary, then use the data with dictionary methods. It is not as fun as using beautiful soup, but you might be able to manipulate the data with bs4? I have never tried bs4 with json, but it seems it should be possible. There are plenty of json libraries as well. And if you want to learn regex really well, this might be a good project to do that, since the data is structured? https://www.mohfw.gov.in/data/datanew.json Happy Coding! 😎👍
Find error at line 43, remove plus
unwanted lines are removed from code check once Steve Sajeev bro
I can't go more in. Bcs I am running the code in sololearn code playground
Hi Ratnapal Shende the problem or argument happening here is bcs u are not willing to understand what others are trying to say.
Kode Krasher see the new code I edited... please look deep into my probelm their is no class for tr or td and seems like it's hidden May be commented and their is a button in that site when we tap it then it is showing the tds data without reloading the page... what to do now ? I researched a lot but not found any solution for this... please help me!
Ratnapal Shende your code does not run in code playground without the code I posted. I can reach the page and text you are scraping and see the data just fine. requests is not better than urllib in code playground. requests is much larger in size and it has a tendency of timing out on higher latency connections. urllib is much simpler and is more lightweight. If you want to have others test it you will need a try and except addition to your code to load the modules.
Kode Krasher the big problem is what is wrong in my code ? And i am running it in pydroid or pc so i have installed that modules ..don't worry about it !
Ratnapal Shende OK. The only thing I would suggest is double check your syntax on your bs4 lookup, look at the source of the page to be as specific as you can. I use bs4 often, and it is easiest if you look for a class id, or find a table attribute to identify the table to differentiate it from the other tables on the page.
Lothar need your help sir ..i am stuck ...
please do not spam QA! قيس ابو اركان
actually I don't know json sorry for that... I will try to do my best.. THANK YOU SO MUCH FOR GIVING TIME SIR ! Kode Krasher
Ratnapal Shende add the code to import the modules in code playground. I did this awhile ago, but it should get you there if you look at the code and comments... https://code.sololearn.com/ccP7wTmpp5GW/?ref=app
Ratnapal Shende your code doesn't run in code playground because you don't pip install the modules before you try to import them. I am accessing this on an iPhone right now. I can look at it on my PC later to see if I can figure out the issue.
''' Ratnapal Shende replace your import statements with the following code: ''' import os, sys try: from bs4 import BeautifulSoup import requests except: os.system('pip install --disable-pip-version-chec --quiet beautifulsoup4') from bs4 import BeautifulSoup os.system('pip install --disable-pip-version-chec --quiet requests') import requests
Ratnapal Shende once you insert the code I put up above, uncomment your print(soup) line and run it. You will see that you are running out of time in the container. Use urllib instead of requests and try it again. It will probably run then... requests requires more memory and is slower than urllib.
Kode Krasher i have installed the modules perfectly bro why are you giving a code which install the modules? 🙄 and requests module is better than urllib because of easy API and high level.. I want to solve my problem without changing the modules..please kindly understand
Kode Krasher did you read previous chat ? I already said their is no table attribute ,their is no class id ....