+ 1

anybody knows how to overcome or skip a password authentication page on a website?for scraping data...!

password authentication

19th Oct 2016, 6:17 AM
RTJ777
RTJ777 - avatar
1 Answer
+ 1
You can't. If you did, you would have to either log in, or hack into the server and extract the data. If you have accounts with the website, your scraper could log in for you. There are ways to add usernames and passwords to website requests. This is the exact problem that created the "deep web." All of the data located behind pay walls and user authentication and private servers is not accessable; thus a deep ocean of information not accessable through many search engines. Some servers allow google special access so that people can find out about their servers, and pay/sign up for their services.
22nd Oct 2016, 9:43 PM
Nelson Earle
Nelson Earle - avatar