javascript - How to scrape a live java script webpage in R? -

-

i scrape play play http://stats.statbroadcast.com/statmonitr/?id=107165. link bring "split box" tab. interested in scraping play play tab home stats , visitors stats tab. 1 of problems no matter tab switch url never changes. if use selector gadget css-selector main contents of tabs same well, "#stats". novice @ web scraping , of time can scrape html page package rvest, unfortunately lost how should proceed javascript. have heard of json, not sure how combat issue of tabs having same url.

my main goal able scrape play play, home stats, , visitor stats tab when game live.

any appreciated. please let me know if should provide more info.

you can use rselenuim follows:

require(rselenium) rselenium::startserver() remdr <- remotedriver() remdr$open() remdr$navigate("http://stats.statbroadcast.com/statmonitr/?id=107165")

now firefox window should open can browse normal. doc <- remdr$getpagesource() gives source-code of current webpage. can use rvest scrape code follows:

doc <- remdr$getpagesource()[[1]] require(rvest) current_doc <- read_html(doc)

if want automate "browsing" can eg. navigate "play play"-page follows:

webelem <- remdr$findelement(using = "css selector", '#bb_b6') remdr$mousemovetolocation(webelement = webelem) remdr$click(1)

at end: close remote driver ans shut down selenium-server

#shutdown remdr$close() browseurl("http://localhost:4444/selenium-server/driver/?cmd=shutdownseleniumserver")

for more details see: https://cran.r-project.org/web/packages/rselenium/vignettes/rselenium-basics.html

edit: current_doc caputures website when execute doc <- remdr$getpagesource()[[1]]. not realtime like. 1 time picture.

if want scrape "period i" follows: 1st navigate "play play" (as shown above) - sys.sleep(3) till website loaded - navigate "period i" same way navigated "play play" css-selector.

have @ remote-driver (aka browser window control) if arrived @ "period i" webpage.

after arrived execute doc <- remdr$getpagesource()[[1]] , analyse content.


Comments

Post a Comment

Popular posts from this blog

Hatching array of circles in AutoCAD using c# -

-
i beginner in autocad plugins .i have created array of circle using following code. now, have hatch circles in array .how that??how hatch circles @ once? moreover, how can return hatch method , use hatch objects.sorry language.thanks in advance. [assembly: commandclass(typeof(beamsection.class1))] namespace beamsection { public class class1 { //class1 obj = new class1(); [commandmethod("beamatsupport")] public void addlightweightpolyline() { // current document , database document acdoc = application.documentmanager.mdiactivedocument; database accurdb = acdoc.database; promptpointresult ppr = acdoc.editor.getpoint("\nselect starting point "); var ucs = acdoc.editor.currentusercoordinatesystem; point3d startingpt = ppr.value.transformby(ucs); list<double> radius = new list<double>(); radius.add(0.5);
Read more

ios - UITEXTFIELD InputView Uipicker not working in swift -

-
hello trying pop uipickerview view programmatically when user clicks on textfield. have tried doesn't doing anything. isn't working. nothing happening when click textfield class userprofiletableviewcontroller: uitableviewcontroller,uitextfielddelegate,uipickerviewdelegate,uipickerviewdatasource { var itempicker: uipickerview! = uipickerview() @iboutlet weak var gendertxtfield: uitextfield! var gender = ["male","female"] override func viewdidload() { super.viewdidload() gendertxtfield.delegate = self itempicker!.delegate = self itempicker!.datasource = self itempicker!.backgroundcolor = uicolor.blackcolor() self.gendertxtfield.inputview = itempicker } func numberofcomponentsinpickerview(pickerview: uipickerview) -> int{ return 1 } // returns # of rows in each component.. func pickerview(pickerview: uipickerview, numberofrowsincomponent component:
Read more

Python Pig Latin Translator -

-
so have been working on @ codecademy.com while trying learn python. don't know if blind , can't see why won't work or what. here's code won't let me use. function print variable if had typed in , print "empty" if didn't type anything. print "welcome english pig latin translator!" original = raw_input("what's name?") if len(original) > 0 print original else print "empty" if len(original) > 0 else the above lines contains syntax errors. try: if len(original) > 0: else: note added colons, ( : ).
Read more