+ 4

Get context from an external Webpage

I want visulise data from an external webpage(not my own) for example: www.example.com. i need just the contet of 1 div container there but dont know how?

28th Dec 2018, 12:13 PM
jack
jack - avatar
44 Answers
+ 2
jack here the scrape using your code and exec function with regex https://code.sololearn.com/WDFKf0oDY1Km/?ref=app
29th Dec 2018, 1:37 PM
Calviղ
Calviղ - avatar
+ 5
jack To be clear... This is only possible for code not running within a modern browser. That is... unless the code is being applied to a website on the same origin or there is a cross origin resource sharing (CORS) policy on the target domain for the current domain.
30th Dec 2018, 4:23 PM
David Carroll
David Carroll - avatar
+ 3
jack My apologies. Ajax doesn't work in a way to extract data from another web pages rather it's useful to extract data from an api. You can't get website data just using jQuery get or ajax method coz CORS won't let you do that. Your try seems well coz we must use cross-domain tools such as anyorigin, allorigins or any other similar to scrape a website.
28th Dec 2018, 1:23 PM
Шащи Ранжан
Шащи Ранжан - avatar
+ 3
Use regex to further extract the data
28th Dec 2018, 3:20 PM
Calviղ
Calviղ - avatar
+ 3
jack You can implement this via any number of server side solutions. You can search for examples on any of the following: (with a few options that come to mind) "HTTP Request PHP" - cURL - fopen - file_get_contents "HTTP Request .NET" - WebRequest "HTTP Request NodeJS" - request() - https.get() - SuperAgent You get the point. Note... The response body is a string that will need to be parsed. You can either do this using various string search methods or load via HTML Parsing Library. You'll need to search for available parsers for your specific language.
29th Dec 2018, 11:29 PM
David Carroll
David Carroll - avatar
+ 3
jack Possible? Absolutely! However, depending on the authentication strategy used on a given website, it can get quite involved. Most commercial websites implement thorough integration tests that would include the automatation for logging in and doing everything else you would want to simulate a user's experience on a website.
30th Dec 2018, 4:14 PM
David Carroll
David Carroll - avatar
+ 2
I don't think you can access the exact visual data from a website but sure you can extract data from a website or a link by using jQuery Ajax.
28th Dec 2018, 12:44 PM
Шащи Ранжан
Шащи Ранжан - avatar
+ 2
i dont need visuals just a part of a divcontainer
28th Dec 2018, 12:46 PM
jack
jack - avatar
+ 2
jack api.allorigins.ml seems slow but might be useful to access a small div content of you know the id, class. That is one way and possibly the only way.
28th Dec 2018, 12:49 PM
Niush
Niush - avatar
+ 2
If you just need written data then I think you should go with Ajax.
28th Dec 2018, 12:51 PM
Шащи Ранжан
Шащи Ранжан - avatar
+ 2
@jack There is notthing to extract from these elements. What do you intend to do?
29th Dec 2018, 3:57 AM
Calviղ
Calviղ - avatar
+ 2
There is no hidden input. Checked from devtools
29th Dec 2018, 9:47 AM
Calviղ
Calviղ - avatar
+ 2
No, not possible, don't waste your time.
29th Dec 2018, 2:06 PM
Calviղ
Calviղ - avatar
+ 2
And thanks for your great overview
30th Dec 2018, 9:47 AM
jack
jack - avatar
+ 1
To access other websites data through IFrame is not possible. Reason: Same Origin Policy https://en.m.wikipedia.org/wiki/Same-origin_policy iFrame data can be accessed only if it is same domain. Or else blocked by the browser itself.
28th Dec 2018, 12:25 PM
Niush
Niush - avatar
+ 1
its possible with a little trick but not for the url i want it. here is what i got: https://code.sololearn.com/WyYP43VP3nRZ/?ref=app
28th Dec 2018, 12:43 PM
jack
jack - avatar
+ 1
jack Show the external website link and the div container you want, I try to scrape it if possible.
28th Dec 2018, 3:48 PM
Calviղ
Calviղ - avatar
+ 1
Cool thanks Calvin
29th Dec 2018, 1:44 PM
jack
jack - avatar
0
Can you show me an example
28th Dec 2018, 12:54 PM
jack
jack - avatar
0
Thx
28th Dec 2018, 3:32 PM
jack
jack - avatar