#4455 closed change (fixed)
WebRTC circumvention
Reported by: | kzar | Assignee: | kzar |
---|---|---|---|
Priority: | P2 | Milestone: | Adblock-Plus-1.13.3-for-Chrome-Opera |
Module: | Platform | Keywords: | |
Cc: | sebastian, trev, mapx, ameshkov, fhd, greiner, fanboy, Lain_13, dzr156, SMed79, IsraeliAdblocker, Ross | Blocked By: | #5087, #5092 |
Blocking: | Platform: | Chrome | |
Ready: | yes | Confidential: | no |
Tester: | Ross | Verified working: | yes |
Review URL(s): |
https://codereview.adblockplus.org/29401590/ |
Description (last modified by kzar)
Background
Some websites have begun using WebRTC as a way to serve advertising. Neither Firefox nor Chrome allow these connections to be blocked with their extension APIs.
The AdGuard guys have found some example code in the wild, it appears to be using RTCPeerConnection specifically:
new (window.RTCPeerConnection || window.mozRTCPeerConnection || window.webkitRTCPeerConnection)({ iceServers: [{url: "stun:1755001826:443"}] },{ optional: [{RtpDataChannels: !0}] });
It looks like Content Security Policies can't be used to block these connections and that in Chrome WebRTC can't be disabled at all!
Resources
- Related AdGuard issue
- Related discussion in AdGuard forums
- https://www.privacytools.io/webrtc.html
- http://webrtc-security.github.io/
- Injected wrapper in uBO-Extra's content script
- https://bugs.chromium.org/p/chromium/issues/detail?id=707683
What to change
Wrap RTCPeerConnection like we did for WebSocket.
Hints for testers
- Check that WebSocket blocking still works for versions of Chrome < 58.
- Test WebRTC blocking by trying some of the linked websites in this issue, experiment with our devtools pane and make sure the WEBRTC connections can now be blocked.
- Test that WebRTC does not break for websites such as Google Hangouts (important!) and https://test.webrtc.org. Please test with Chrome 41, a new version and some versions of Opera.
- Test that WebSocket connections are only listed once for versions of Chrome >= 58 and that blocking those still works as expected.
Attachments (1)
Change History (66)
comment:1 Changed 3 years ago by trev
comment:2 Changed 3 years ago by trev
Ok, I looked a bit closer into the code in question. This code doesn't actually create a WebRTC connection - transmitting data over one would be a fairly complicated affair. It merely consults a STUN server which is usually being done to determine the host's external IP address. There is currently a known side-effect of deanonymizing users (bug 959893) but that doesn't seem to be what this script is after. Instead, it receives an IP address from the STUN server that doesn't belong to the client - and it uses this IP address to build the pop-up address, something like http://130.211.198.219/MzEyNzQ5MDYxMDk4LzMxMjc1MDQzNzc1OS8wcjRkZWI5bjky (the path here is merely two weirdly encoded random numbers). This doesn't happen for me in Firefox but their code is working in Chrome. So at the moment this contributes to significant obfuscation but blocking isn't too challenging.
This doesn't change the fact that connections to STUN servers should go through content policies and this was already proposed under https://bugzilla.mozilla.org/show_bug.cgi?id=1126187#c4 (not sure why they chose to discuss this in a bug that was resolved as duplicate). We might want to implement a patch for that.
comment:3 Changed 3 years ago by mapx
- Cc fanboy added
comment:4 Changed 3 years ago by mapx
- Cc Lain_13 added
comment:5 Changed 3 years ago by fanboy
New incidents reported on crackberry.com (Reported here: https://forums.lanik.us/viewtopic.php?p=111045#p111045)
So many Chrome Anti-adblock loopholes lately :(
comment:6 follow-up: ↓ 8 Changed 3 years ago by mapx
it seems gorhill fixed it using the same CSP.
A lot of sites are migrating from websocket to webrtc trick.
comment:7 follow-up: ↓ 9 Changed 3 years ago by fanboy
Maybe we should have a $webrtc filter option?
comment:8 in reply to: ↑ 6 Changed 3 years ago by kzar
Replying to mapx:
it seems gorhill fixed it using the same CSP.
So I just took a look at his code and unfortunately it's not as simple as adding another CSP. He's actually wrapping RTCPeerConnection to create a dummy WebSocket connection before opening the WebRTC connections. If the dummy WebSocket connection is blocked by the CSP the WebRTC connection won't be allowed either.
We could also probably wrap the WebRTC API like we're doing for the WebSocket API but it we'd have to be careful about it. I've not looked into this in more detail yet.
Replying to fanboy:
Maybe we should have a $webrtc filter option?
If WebRTC is an increasing privacy + circumvention problem like you guys suggest I wouldn't be against adding a webrtc request type. We could then add wrapper(s) for the WebRTC APIs which used that request type and in turn filters with $webrtc could be written.
Any opinion Wladimir / Sebastian?
comment:9 in reply to: ↑ 7 ; follow-up: ↓ 10 Changed 3 years ago by mapx
Replying to fanboy:
Maybe we should have a $webrtc filter option?
why another option ? better keeping websocket for all this crap and will use the filters already in easylist.
however, the users are complaining why ABP is doing ...nothing (and ubo already patched it)
https://forums.lanik.us/viewtopic.php?p=111399#p111399
comment:10 in reply to: ↑ 9 Changed 3 years ago by kzar
Replying to mapx:
Replying to fanboy:
Maybe we should have a $webrtc filter option?
why another option ? better keeping websocket for all this crap and will use the filters already in easylist.
Well because webrtc connections are not the same as websocket connections. (Remember that with Firefox the extension API allows us to intercept websocket connections without the wrapper and hopefully Chrome will in the future too.)
comment:11 Changed 3 years ago by fanboy
So can we can temp fix for webrtc or something (anything to combat it). Waiting on X and Y won't help the enduser, all they see is ads and in-action.
comment:12 Changed 3 years ago by kzar
Well I need to check with Wladimir and Sebastian before beginning work to wrap the WebRTC APIs. (Since we will likely need to consider those connections as a new webrtc type core changes will be required.) It will probably be quite a bit of work and if we don't agree on an approach before I start it'll end up wasting a bunch of my time. I'll update the issue(s) when I've heard back from them about it.
comment:13 Changed 3 years ago by fanboy
From the current (growing) list of sites using the webrtc circumvention (seen from uBlock-Origin
britannica.com businessnewsdaily.com collegehumor.com dorkly.com investopedia.com laptopmag.com lifeandstylemag.com mashable.com newsarama.com phonearena.com space.com theberry.com thechive.com thepoliticalinsider.com tomsguide.com
comment:14 Changed 3 years ago by mapx
pm message:
webRTC ads (huge adoption) Sent: Mon Feb 13, 2017 10:46 am From: dzr156 To: mapx Hey mapx, recently we see huge adoption of webRTC ads in the top 5 websites in Israel, for example: mako.co.il jpost.com walla.co.il bizportal.co.il calcalist.co.il You guys must do something with that, people uninstall Adblock or moving to ubo. And the guys from the company that provides this solution claims that the "ad blocking is dead". please keep me posted. thnks
comment:15 Changed 3 years ago by mapx
- Cc dzr156 added
comment:16 Changed 3 years ago by kzar
I also received a similar PM, seems like this WebRTC circumvention is increasingly becoming a problem.
Wladimir / Sebastian what do you think to my earlier suggestion?
If WebRTC is an increasing privacy + circumvention problem like you guys suggest I
wouldn't be against adding a webrtc request type. We could then add wrapper(s) for the
WebRTC APIs which used that request type and in turn filters with $webrtc could be
written.
comment:17 Changed 3 years ago by sebastian
So you are talking about comment:8, where you suggest to wrap the Web RTC API, like we did for the WebSocket API, but with a different request type? IIRC, as we did that for WebSockets, it didn't take long until people discovered a way to bypass our wrapper, by initiating the connection from a document loaded through a data: URL, web workers and alike. Won't we have the same problem with Web RTC? Is it possible to leverage CSP to address that?
Also, is there any reason the webRequest API cannot intercept Web RTC connections, other than they just forgot to implement it like they did for WebSockets? It might be worth filing a Chromium bug.
comment:18 Changed 3 years ago by fanboy
Maybe we have a working around in ABP, until which Chromium can fix it. Chromium bugs tend to be slower to get fixed?
comment:19 Changed 3 years ago by sebastian
I'm not against implementing a workaround in Adblock Plus per se. But we should still bring that issue up to the Chromium team, so that they will hopefully lift the limitations on their end, so that we can eventually get rid of the workaround, since using the webRequest API would be more powerful, reliable, maintainable and efficient than any workaround we could implement.
comment:20 follow-up: ↓ 21 Changed 3 years ago by fanboy
Not to push the point, but are we getting a fix or not?
comment:21 in reply to: ↑ 20 Changed 3 years ago by kzar
Replying to fanboy:
Not to push the point, but are we getting a fix or not?
I know this is an important issue for you guys, I see read all your replies here, the private messages and emails I receive about it. I want to start working on this when I have the time, but until then please be patient.
comment:22 Changed 3 years ago by dzr156
Guys, this issue starting to be very urgent. IMO - its putting ABP in a bad light.
Here in Israel, all the big sites (ynet.co.il, walla.co.il, mako.co.il) using webRTC, and the Heb list fourm is officially recommended on ubo (example: https://github.com/easylist/EasyListHebrew/issues/233)
comment:23 Changed 3 years ago by kzar
- Owner set to kzar
Time allowing I hope to start with this next week. I'll keep you all posted.
comment:24 Changed 3 years ago by mapx
- Cc SMed79 added
comment:25 Changed 3 years ago by dzr156
Hi, just update - I find the Extension WebRTC Leak Prevent useful here. https://chrome.google.com/webstore/detail/webrtc-leak-prevent/eiadekoaikejlgdbkbdfeijglgfdalml
maybe we can take some piece of code from this.
comment:26 Changed 3 years ago by kzar
- Description modified (diff)
comment:27 Changed 3 years ago by kzar
- Blocked By 5087 added
comment:28 Changed 3 years ago by kzar
- Description modified (diff)
comment:29 Changed 3 years ago by kzar
- Blocked By 5092 added
Changed 3 years ago by kzar
comment:30 Changed 3 years ago by kzar
Good news, I finally got some time to work on this today! Better news this wasn't too hard, since a lot of the code and lessons learned wrapping WebSocket could be reused here. Even better news I have a (very rough) patch that seems to be working nicely.
The devil is in the details though, so I have some work to do to tidy up my patch and to test everything really does work correctly. I'll keep you posted, and in the mean time I've uploaded a teaser screenshot.
comment:31 Changed 3 years ago by mapx
- Cc IsraeliAdblocker added
comment:32 Changed 3 years ago by kzar
comment:33 Changed 3 years ago by kzar
- Blocked By 5124 added
comment:34 Changed 3 years ago by kzar
- Blocked By 5124 removed
comment:35 Changed 3 years ago by abpbot
A commit referencing this issue has landed:
Issue 4455, 5087, 5092 - Wrap RTCPeerConnection to allow blocking
comment:36 Changed 3 years ago by kzar
- Cc Ross added
- Description modified (diff)
- Milestone set to Adblock-Plus-for-Chrome-Opera-next
- Platform changed from Unknown / Cross platform to Chrome
- Resolution set to fixed
- Status changed from reviewing to closed
comment:37 Changed 3 years ago by fanboy
Does this give us a new $webrtc switch? How does it work?
comment:38 Changed 3 years ago by kzar
Yes exactly, there's now a request type of webrtc. Those connections should show up in the Adblock Plus devtools pane and should be blockable like any others. Check out the screenshot I uploaded to this issue a few days ago to see how it should work.
We're still drafting the blog post about this, I'll post a link to it here when it's ready.
comment:39 Changed 3 years ago by fanboy
Should we commit the filter changes before hand?
Looking at uBo's filters, I've compiled a list of webrtc-ad sites:
https://pastebin.com/raw/RAeCxGzH
No issues on formatting? was just using a similar approach to $websocket.
comment:40 Changed 3 years ago by kzar
Filters with the $webrtc option should just be ignored for versions of Adblock Plus that don't support it yet. So as long as you keep them separate then it should be fine to go ahead yes. I would recommend testing that first however, better safe than sorry.
comment:41 Changed 3 years ago by kzar
FWIW I just tested your filter with the latest code and it worked for me, WebRTC connections were blocked.
comment:42 Changed 3 years ago by fanboy
Great! A win for now, tho no doubt the advertisers will find new/alternative ways to circumvent Adblock Plus.
I'll check on the rest of the Easylist authors before making a large webrtc commit.
comment:43 follow-up: ↓ 44 Changed 3 years ago by fanboy
What's the debug options like on webrtc, will it show up in the blocked items of either browser?
comment:44 in reply to: ↑ 43 Changed 3 years ago by mapx
Replying to fanboy:
What's the debug options like on webrtc, will it show up in the blocked items of either browser?
https://issues.adblockplus.org/attachment/ticket/4455/blocked-webrtc.png
"Those connections should show up in the Adblock Plus devtools pane and should be blockable like any others."
comment:45 Changed 3 years ago by mapx
and probably will land in the google store only after they (google) will fix this issue:
https://bugs.chromium.org/p/chromium/issues/detail?id=129353#c94
comment:46 Changed 3 years ago by dzr156
I've tested it a bit. works fine for http://www.tomsguide.com/. doesn't work for the uponit ads - http://www.ynet.co.il/home/0,7340,L-8,00.html and http://www.walla.co.il/
comment:47 Changed 3 years ago by fanboy
https://issues.adblockplus.org/ticket/242
This should be fixed asap, uponit is taken advantage of this bug in chrome
comment:48 follow-up: ↓ 56 Changed 3 years ago by kzar
This issue is about allowing the blocking of WebRTC connections without breaking legitimate usage such as Google Hangouts. Help with testing that is appreciated, but discussion about other stuff like element hiding circumvention doesn't belong here.
comment:49 Changed 3 years ago by kzar
Whoops sorry, forgot to post a link to the blog post. Here you go: https://adblockplus.org/development-builds/new-filter-type-option-for-webrtc-connections
comment:50 Changed 3 years ago by kzar
- Resolution fixed deleted
- Review URL(s) modified (diff)
- Status changed from closed to reopened
comment:51 Changed 3 years ago by kzar
- Status changed from reopened to reviewing
comment:52 follow-up: ↓ 53 Changed 3 years ago by fanboy
What happened? why the review?
comment:53 in reply to: ↑ 52 Changed 3 years ago by kzar
Replying to fanboy:
What happened? why the review?
Just some more work to harden our RTCPeerConnection wrapper against circumvention, since it's very hard to get this stuff right. Nothing will change with how the filters are applied or anything like that, don't worry.
comment:54 Changed 3 years ago by abpbot
A commit referencing this issue has landed:
Issue 4455 - Further harden the RTCPeerConnection wrapper
comment:55 Changed 3 years ago by kzar
- Resolution set to fixed
- Status changed from reviewing to closed
comment:56 in reply to: ↑ 48 Changed 3 years ago by kzar
Replying to kzar:
This issue is about allowing the blocking of WebRTC connections without breaking legitimate usage such as Google Hangouts. Help with testing that is appreciated, but discussion about other stuff like element hiding circumvention doesn't belong here.
Sorry, after a discussing this with dzr156 via email it turns out I was wrong. There is something else happening there. I can reproduce too, I've opened #5207 for that and I'm investigating.
comment:57 Changed 3 years ago by SMed79
Sorry for the noise but webrtc requests are not blocked for me. for example on :
merriam-webster.com imore.com androidcentral.com closerweekly.com collegehumor.com connectedly.com dorkly.com firstforwomen.com xda-developers.com ...
Do I missed something or i am doing something wrong?
comment:58 Changed 3 years ago by kzar
comment:59 Changed 3 years ago by kzar
Fixes for those issues have landed now and should be included in development builds soon. Could you give those pages a try again and let me know if the WebRTC connections are blocked now? I'd be interested to hear about any website who are still managing to use WebRTC for circumvention, hopefully that shouldn't be possible any longer.
comment:60 Changed 2 years ago by Ross
Done. WebRTC connections can be blocked, appear in the DevTools panel. WebRTC functionality like Google Handouts still works as expected.
ABP 1.13.2.1785
Chrome 49 / 58 / Windows 7
Opera 39 / 45 / Windows 7
comment:61 Changed 2 years ago by Ross
- Tester changed from Unknown to Ross
- Verified working set
comment:62 Changed 2 years ago by kzar
- Resolution fixed deleted
- Status changed from closed to reopened
comment:63 Changed 2 years ago by kzar
- Resolution set to fixed
- Sensitive unset
- Status changed from reopened to closed
comment:64 Changed 2 years ago by dzr156
Hi, could you check this site - http://www.calcalist.co.il/real_estate/articles/0,7340,L-3716614,00.html
Seems like a new way to bypass our new wrapper.
comment:65 Changed 2 years ago by kzar
Please could you start a new issue for that? This issue is now considered closed and has been made public.
For reference, 1755001826 is another way of writing 104.155.51.226 - an IP address assigned to Google, presumably App Engine.