+ 4

Get context from an external Webpage

I want visulise data from an external webpage(not my own) for example: www.example.com. i need just the contet of 1 div container there but dont know how?

28th Dec 2018, 12:13 PM
jack
jack - avatar
39 Antworten
+ 2
jack here the scrape using your code and exec function with regex https://code.sololearn.com/WDFKf0oDY1Km/?ref=app
29th Dec 2018, 1:37 PM
CalviŇ▓
CalviŇ▓ - avatar
+ 5
jack To be clear... This is only possible for code not running within a modern browser. That is... unless the code is being applied to a website on the same origin or there is a cross origin resource sharing (CORS) policy on the target domain for the current domain.
30th Dec 2018, 4:23 PM
David Carroll
David Carroll - avatar
+ 3
jack My apologies. Ajax doesn't work in a way to extract data from another web pages rather it's useful to extract data from an api. You can't get website data just using jQuery get or ajax method coz CORS won't let you do that. Your try seems well coz we must use cross-domain tools such as anyorigin, allorigins or any other similar to scrape a website.
28th Dec 2018, 1:23 PM
đĘđ░ĐëđŞ đáđ░đŻđÂđ░đŻ
đĘđ░ĐëđŞ đáđ░đŻđÂđ░đŻ - avatar
+ 3
Use regex to further extract the data
28th Dec 2018, 3:20 PM
CalviŇ▓
CalviŇ▓ - avatar
+ 3
jack You can implement this via any number of server side solutions. You can search for examples on any of the following: (with a few options that come to mind) "HTTP Request PHP" - cURL - fopen - file_get_contents "HTTP Request .NET" - WebRequest "HTTP Request NodeJS" - request() - https.get() - SuperAgent You get the point. Note... The response body is a string that will need to be parsed. You can either do this using various string search methods or load via HTML Parsing Library. You'll need to search for available parsers for your specific language.
29th Dec 2018, 11:29 PM
David Carroll
David Carroll - avatar
+ 3
jack Possible? Absolutely! However, depending on the authentication strategy used on a given website, it can get quite involved. Most commercial websites implement thorough integration tests that would include the automatation for logging in and doing everything else you would want to simulate a user's experience on a website.
30th Dec 2018, 4:14 PM
David Carroll
David Carroll - avatar
+ 2
I don't think you can access the exact visual data from a website but sure you can extract data from a website or a link by using jQuery Ajax.
28th Dec 2018, 12:44 PM
đĘđ░ĐëđŞ đáđ░đŻđÂđ░đŻ
đĘđ░ĐëđŞ đáđ░đŻđÂđ░đŻ - avatar
+ 2
i dont need visuals just a part of a divcontainer
28th Dec 2018, 12:46 PM
jack
jack - avatar
+ 2
If you just need written data then I think you should go with Ajax.
28th Dec 2018, 12:51 PM
đĘđ░ĐëđŞ đáđ░đŻđÂđ░đŻ
đĘđ░ĐëđŞ đáđ░đŻđÂđ░đŻ - avatar
+ 2
@jack There is notthing to extract from these elements. What do you intend to do?
29th Dec 2018, 3:57 AM
CalviŇ▓
CalviŇ▓ - avatar
+ 2
There is no hidden input. Checked from devtools
29th Dec 2018, 9:47 AM
CalviŇ▓
CalviŇ▓ - avatar
+ 2
No, not possible, don't waste your time.
29th Dec 2018, 2:06 PM
CalviŇ▓
CalviŇ▓ - avatar
+ 2
And thanks for your great overview
30th Dec 2018, 9:47 AM
jack
jack - avatar
+ 1
its possible with a little trick but not for the url i want it. here is what i got: https://code.sololearn.com/WyYP43VP3nRZ/?ref=app
28th Dec 2018, 12:43 PM
jack
jack - avatar
+ 1
jack Show the external website link and the div container you want, I try to scrape it if possible.
28th Dec 2018, 3:48 PM
CalviŇ▓
CalviŇ▓ - avatar
+ 1
Cool thanks Calvin
29th Dec 2018, 1:44 PM
jack
jack - avatar
0
Can you show me an example
28th Dec 2018, 12:54 PM
jack
jack - avatar
0
Thx
28th Dec 2018, 3:32 PM
jack
jack - avatar
0
Cool wait could take a bit I'm at the dinner;)
28th Dec 2018, 5:09 PM
jack
jack - avatar
0
The domain called:"www.swcombine.com" Need there : <input name="log" type="hidden".. <input name="hand" id="hand".... <input name="pass" id="pass"....
28th Dec 2018, 5:22 PM
jack
jack - avatar