http://lists.ausnog.net/pipermail/ausnog/2012-June/013833.html and http://www.scmagazine.com.au/News/305928,telstra-says-its-not-spying-on-users.aspx were recently published saying that Telstra NextG users were seeing some interesting things. (Yes, there’s a Whirlpool post too, but since they block requests from Tor I’m not going to link to them)
Basically, on their servers they were seeing HTTP requests to the same URL as they had just visited with their phone, but from an IP address that certainly wasn’t their phone.
I started to investigate.
I put up a simple HTML page on a standard HTTP server and then got a NextG device to query it. I saw a log that came from a TELSTRA owned block of IPs. I didn’t see any suspicious second request though. Sadness.
Turns out you have to request the URL twice to get this other request. It is after this second request that you get a query from a Rackspace/Slicehost IP (cloud provider, so it is unlikely Rackspace itself is involved any more than as a Cloud provider) with the same URL (although via HTTP/1.0 instead of 1.1). On a subsequent request, I didn’t see a corresponding one from this IP. Also, when accessing this URL from a different NextG device, I did not see a request from the Rackspace/Slicehost IP block.
If I change the content of the file and try to fetch again, it doesn’t download it anew. This suggests that there is not inspection of the content of what’s coming back from the HTTP server.
The User Agent pretends to be Firefox running on Windows. I have not yet found out anything specific about it.
What can we learn from this?
- If you think that putting a URL up and only telling 1 person about it is private you are very, very, very much mistaken
- Telstra is quite possibly spying on you, from servers in the USA, which is under a different set of laws than if it was done in Australia.
- Telstra is sending what websites you visit on your NextG connection to the USA. If you are at all involved in anything that may make the US government unhappy (e.g. disagreeing with it) this may have interesting implications. Further research is needed as to what exactly
- Telstra keeps a record of all URLs as otherwise it could not implement “on the second request”
- The iPhone needs Tor more than ever and it needs it on a system level.
Update: I have been pointed to http://v3.mike.tig.as/onionbrowser/ which is an Open Source Web Browser that uses Tor on iOS.
One of the many reasons why my blog is HTTPS–only, and I religiously run HTTPS Everywhere to force HTTPS on websites that support it.
We know Telstra is evil, but this behaviour is only going to increase, even in ISPs that are less evil than Telstra.
Could this just be some sort of innocuous cache they have going on that grabs popular content so it is faster for others? Though it doesn’t refresh?
Would it have anything to do with the “voluntary” ISP Child Porn Filter?
Seems unlikely since it’s only happening on NextG.
If it were a cache, you’d expect to see it return a cached result at some point rather than fetch from the remote host all the time.
I tested this with my Telstra NextG iPad microsim on a prepaid connection.
Tested with APN, telstra.iph and telstra.internet, Airplane mode between switches, new IP.
Saw no behavior as indicated on ausNog or this blog, loaded urls and watched access.log. Only single accesses from a Telstra IP. Different pool for different APNs.
Whatever they were doing at Telstra, they appear to have stopped.
jeremy – https would not protect you from this issue. GET requests are not protected by encrypted (although the subsequent traffic is)
steve – possibly… but in the usa? what… they ran out of room in their data center? it seems odd – and even if the reasons are completely mundane the appearance of “off” warrants better explanation then has been given to date
http 1.0 usually indicates a proxy is being used, like squid proxy.
e.g. http://www.hangthebankers.com/australian-government-to-track-all-web-usage/
While the Australian government’s desire to track all internet usage is technically infeasible, it would be possible to build up personal profiles by categorising the text in the pages visited. The behaviour you are seeing here looks like it fits with such a scenario. It could be that a small trial is being run on a limited selection of users (it would not be the firs time something was trialled without the public knowing)
Complete speculation, but imagine the transparent proxy stored a list of categories for each page visited. When someone’s phone requests a page, the cache would ignore it the first time (so as to reduce noise), but for repeat visits it would request a cloud service to download the page and categorise it. The cache could then send the user’s identity plus the page categories to a third system which would build up a profile of the user based on the pages they keep going back to. I’ve attempted to draw a diagram to show you what I mean – see http://is.gd/rdl3Fg
Note that this doesn’t require any particular personal information to be sent outside Australia and so would be completely legal.
Are you saying that the proxy only requests pages with a particular content type? Very curious.
Intriguing read, up until the point you said iPhone. I think Telstra collecting metrics on your browsing is the least of you problems. It’s like worrying about your personal details being exposed and then printing them on your t-shirt. I’m not saying Android is safe, I just don’t think Tor will help you on an iPhone. http://blogs.wsj.com/wtk-mobile/ It is still fascinating though and I might do some testing this afternoon. Cheers.
Pingback: An update on Telstra’s surveillance of what you do online | Ramblings
I’ve sent emails in the past to people who have got their MX records pointing to McAfee for email protection. I then see the HTTP request from McAfee’s servers in the US, then later the request from the intended recipient.
Maybe Telstra are using a 3rd party proxy to do scanning and then later blacklisting?
Or they could just be spying on you…
@caseyjohnellis – please explain? HTTPS requests ARE encrypted..
Pingback: Telstra funding censorship in Middle East | Ramblings
Pingback: Tor + Firefox + Twitter + (not rooted) Android = awesome | Ramblings
@Alex – apologies, my comment was VERY sloppy and basically incorrect…
more correctly: HTTPS would not prevent your ISP from being able to tell which web server you were connecting to which yield similar, albeit less detailed, information re your web browsing habits.
you are correct in saying HTTP requests made over SSL (i.e. HTTPS) are encrypted.