Get context from an external Webpage

Question

I want visulise data from an external webpage(not my own) for example: www.example.com. i need just the contet of 1 div container there but dont know how?

Answer 1

jack here the scrape using your code and exec function with regex
https://code.sololearn.com/WDFKf0oDY1Km/?ref=app

Answer 2

jack To be clear... This is only possible for code not running within a modern browser.
That is... unless the code is being applied to a website on the same origin or there is a cross origin resource sharing (CORS) policy on the target domain for the current domain.

Answer 3

jack
My apologies. Ajax doesn't work in a way to extract data from another web pages rather it's useful to extract data from an api.
You can't get website data just using jQuery get or ajax method coz CORS won't let you do that.
Your try seems well coz we must use cross-domain tools such as anyorigin, allorigins or any other similar to scrape a website.

Answer 4

Use regex to further extract the data

Answer 5

jack You can implement this via any number of server side solutions.
You can search for examples on any of the following:
(with a few options that come to mind)
"HTTP Request PHP"
- cURL
- fopen
- file_get_contents
"HTTP Request .NET"
- WebRequest
"HTTP Request NodeJS"
- request()
- https.get()
- SuperAgent
You get the point.
Note... The response body is a string that will need to be parsed. You can either do this using various string search methods or load via HTML Parsing Library.
You'll need to search for available parsers for your specific language.

Answer 6

jack
Possible? Absolutely!
However, depending on the authentication strategy used on a given website, it can get quite involved.
Most commercial websites implement thorough integration tests that would include the automatation for logging in and doing everything else you would want to simulate a user's experience on a website.

Answer 7

I don't think you can access the exact visual data from a website but sure you can extract data from a website or a link by using jQuery Ajax.

Answer 8

i dont need visuals just a part of a divcontainer

Answer 9

jack api.allorigins.ml seems slow but might be useful to access a small div content of you know the id, class.
That is one way and possibly the only way.

Answer 10

If you just need written data then I think you should go with Ajax.

Answer 11

@jack There is notthing to extract from these elements.
What do you intend to do?

Answer 12

There is no hidden input. Checked from devtools

Answer 13

No, not possible, don't waste your time.

Answer 14

And thanks for your great overview

Answer 15

To access other websites data through IFrame is not possible.
Reason: Same Origin Policy
https://en.m.wikipedia.org/wiki/Same-origin_policy
iFrame data can be accessed only if it is same domain. Or else blocked by the browser itself.

Answer 16

its possible with a little trick but not for the url i want it. here is what i got: https://code.sololearn.com/WyYP43VP3nRZ/?ref=app

Answer 17

jack Show the external website link and the div container you want, I try to scrape it if possible.

Answer 18

Cool thanks Calvin

Answer 19

Can you show me an example

Answer 20

Thx