Scraping Real Time Visitors from Google Analytics

Using ajax: Scraping Real Time Visitors from Google Analytics on newest questions tagged ajax – Stack Overflow

I have a lot of sites and want to build a dashboard showing the number of real time visitors on each of them on a single page. (would anyone else want this?) Right now the only way to view this information is to open a new tab for each site.

Google doesn’t have a real-time API, so I’m wondering if it is possible to scrape this data. Eduardo Cereto found out that Google transfers the real-time data over the realtime/bind network request. Anyone more savvy have an idea of how I should start? Here’s what I’m thinking:

1. Figure out how to authenticate programmatically

2. Inspect all of the realtime/bind requests to see how they change. Does each request have a unique key? Where does that come from? Below is my breakdown of the request:

   https://www.google.com/analytics/realtime/bind?VER=8

   &key=[What is this? Where does it come from? 21 character lowercase alphanumeric, stays the same each request]

   &ds=[What is this? Where does it come from? 21 character lowercase alphanumeric, stays the same each request]

   &pageId=rt-standard%2Frt-overview

   &q=t%3A0%7C%3A1%3A0%3A%2Ct%3A11%7C%3A1%3A5%3A%2Cot%3A0%3A0%3A4%2Cot%3A0%3A0%3A3%2Ct%3A7%7C%3A1%3A10%3A6%3D%3DREFERRAL%3B%2Ct%3A10%7C%3A1%3A10%3A%2Ct%3A18%7C%3A1%3A10%3A%2Ct%3A4%7C5%7C2%7C%3A1%3A10%3A2!%3Dzz%3B%2C&f

   The q variable URI decodes to this (what the?):
   t:0|:1:0:,t:11|:1:5:,ot:0:0:4,ot:0:0:3,t:7|:1:10:6==REFERRAL;,t:10|:1:10:,t:18|:1:10:,t:4|5|2|:1:10:2!=zz;,&f

   &RID=rpc

   &SID=[What is this? Where does it come from? 16 character uppercase alphanumeric, stays the same each request]

   &CI=0

   &AID=[What is this? Where does it come from? integer, starts at 1, increments weirdly to 150 and then 298]

   &TYPE=xmlhttp

   &zx=[What is this? Where does it come from? 12 character lowercase alphanumeric, changes each request]

   &t=1

3. Inspect all of the realtime/bind responses to see how they change. How does the data come in? It looks like some altered JSON. How many times do I need to connect to get the data? Where is the active visitors on site number in there? Here is a dump of sample data:

   19
   [[151,["noop"]
   ]
   ]
   388
   [[152,["rt",[{"ot:0:0:4":{"timeUnit":"MINUTES","overTimeData":[{"values":[49,53,52,40,42,55,49,41,51,52,47,42,62,82,76,71,81,66,81,86,71,66,65,65,55,51,53,73,71,81],”name”:”Total”}]},”ot:0:0:3″:{“timeUnit”:”SECONDS”,”overTimeData”:[{"values":[0,1,1,1,1,0,1,0,1,1,1,0,2,0,2,2,1,0,0,0,0,0,2,1,1,2,1,2,0,5,1,0,2,1,1,1,2,0,2,1,0,5,1,1,2,0,0,0,0,0,0,0,0,0,1,1,0,3,2,0],”name”:”Total”}]}}]]]
   ]
   388
   [[153,["rt",[{"ot:0:0:4":{"timeUnit":"MINUTES","overTimeData":[{"values":[52,53,52,40,42,55,49,41,51,52,47,42,62,82,76,71,81,66,81,86,71,66,65,65,55,51,53,73,71,81],”name”:”Total”}]},”ot:0:0:3″:{“timeUnit”:”SECONDS”,”overTimeData”:[{"values":[2,1,1,1,1,1,0,1,0,1,1,1,0,2,0,2,2,1,0,0,0,0,0,2,1,1,2,1,2,0,5,1,0,2,1,1,1,2,0,2,1,0,5,1,1,2,0,0,0,0,0,0,0,0,0,1,1,0,3,2],”name”:”Total”}]}}]]]
   ]
   388
   [[154,["rt",[{"ot:0:0:4":{"timeUnit":"MINUTES","overTimeData":[{"values":[53,53,52,40,42,55,49,41,51,52,47,42,62,82,76,71,81,66,81,86,71,66,65,65,55,51,53,73,71,81],”name”:”Total”}]},”ot:0:0:3″:{“timeUnit”:”SECONDS”,”overTimeData”:[{"values":[0,3,1,1,1,1,1,0,1,0,1,1,1,0,2,0,2,2,1,0,0,0,0,0,2,1,1,2,1,2,0,5,1,0,2,1,1,1,2,0,2,1,0,5,1,1,2,0,0,0,0,0,0,0,0,0,1,1,0,3],”name”:”Total”}]}}]]]
   ]

Let me know if you can help with any of the items above!

source: http://stackoverflow.com/questions/11021554/scraping-real-time-visitors-from-google-analytics
Using ajax: using-ajax