Opened 13 months ago

Closed 4 months ago

Last modified 4 months ago

#4455 closed change (fixed)

WebRTC circumvention

Reported by: kzar Assignee: kzar
Priority: P2 Milestone: Adblock-Plus-1.13.3-for-Chrome-Opera
Module: Platform Keywords:
Cc: sebastian, trev, mapx, ameshkov, fhd, greiner, fanboy, Lain_13, dzr156, SMed79, IsraeliAdblocker, Ross Blocked By: #5087, #5092
Blocking: Platform: Chrome
Ready: yes Confidential: no
Tester: Ross Verified working: yes
Review URL(s):

https://codereview.adblockplus.org/29401590/
https://codereview.adblockplus.org/29420555/

Description (last modified by kzar)

Background

Some websites have begun using WebRTC as a way to serve advertising. Neither Firefox nor Chrome allow these connections to be blocked with their extension APIs.

The AdGuard guys have found some example code in the wild, it appears to be using RTCPeerConnection specifically:

new (window.RTCPeerConnection || window.mozRTCPeerConnection || window.webkitRTCPeerConnection)({
  iceServers: [{url: "stun:1755001826:443"}]
},{
  optional: [{RtpDataChannels: !0}]
});

It looks like Content Security Policies can't be used to block these connections and that in Chrome WebRTC can't be disabled at all!

Resources

What to change

Wrap RTCPeerConnection like we did for WebSocket.

Hints for testers

  • Check that WebSocket blocking still works for versions of Chrome < 58.
  • Test WebRTC blocking by trying some of the linked websites in this issue, experiment with our devtools pane and make sure the WEBRTC connections can now be blocked.
  • Test that WebRTC does not break for websites such as Google Hangouts (important!) and https://test.webrtc.org. Please test with Chrome 41, a new version and some versions of Opera.
  • Test that WebSocket connections are only listed once for versions of Chrome >= 58 and that blocking those still works as expected.

Attachments (1)

blocked-webrtc.png (34.8 KB) - added by kzar 7 months ago.

Download all attachments as: .zip

Change History (66)

comment:1 Changed 13 months ago by trev

For reference, 1755001826 is another way of writing 104.155.51.226 - an IP address assigned to Google, presumably App Engine.

comment:2 Changed 13 months ago by trev

Ok, I looked a bit closer into the code in question. This code doesn't actually create a WebRTC connection - transmitting data over one would be a fairly complicated affair. It merely consults a STUN server which is usually being done to determine the host's external IP address. There is currently a known side-effect of deanonymizing users (bug 959893) but that doesn't seem to be what this script is after. Instead, it receives an IP address from the STUN server that doesn't belong to the client - and it uses this IP address to build the pop-up address, something like http://130.211.198.219/MzEyNzQ5MDYxMDk4LzMxMjc1MDQzNzc1OS8wcjRkZWI5bjky (the path here is merely two weirdly encoded random numbers). This doesn't happen for me in Firefox but their code is working in Chrome. So at the moment this contributes to significant obfuscation but blocking isn't too challenging.

This doesn't change the fact that connections to STUN servers should go through content policies and this was already proposed under https://bugzilla.mozilla.org/show_bug.cgi?id=1126187#c4 (not sure why they chose to discuss this in a bug that was resolved as duplicate). We might want to implement a patch for that.

comment:3 Changed 13 months ago by mapx

  • Cc fanboy added

comment:4 Changed 13 months ago by mapx

  • Cc Lain_13 added

comment:5 Changed 10 months ago by fanboy

New incidents reported on crackberry.com (Reported here: https://forums.lanik.us/viewtopic.php?p=111045#p111045)

So many Chrome Anti-adblock loopholes lately :(

comment:6 follow-up: Changed 10 months ago by mapx

it seems gorhill fixed it using the same CSP.
A lot of sites are migrating from websocket to webrtc trick.

comment:7 follow-up: Changed 10 months ago by fanboy

Maybe we should have a $webrtc filter option?

comment:8 in reply to: ↑ 6 Changed 10 months ago by kzar

Replying to mapx:

it seems gorhill fixed it using the same CSP.

So I just took a look at his code and unfortunately it's not as simple as adding another CSP. He's actually wrapping RTCPeerConnection to create a dummy WebSocket connection before opening the WebRTC connections. If the dummy WebSocket connection is blocked by the CSP the WebRTC connection won't be allowed either.

We could also probably wrap the WebRTC API like we're doing for the WebSocket API but it we'd have to be careful about it. I've not looked into this in more detail yet.

Replying to fanboy:

Maybe we should have a $webrtc filter option?

If WebRTC is an increasing privacy + circumvention problem like you guys suggest I wouldn't be against adding a webrtc request type. We could then add wrapper(s) for the WebRTC APIs which used that request type and in turn filters with $webrtc could be written.

Any opinion Wladimir / Sebastian?

comment:9 in reply to: ↑ 7 ; follow-up: Changed 10 months ago by mapx

Replying to fanboy:

Maybe we should have a $webrtc filter option?

why another option ? better keeping websocket for all this crap and will use the filters already in easylist.

however, the users are complaining why ABP is doing ...nothing (and ubo already patched it)
https://forums.lanik.us/viewtopic.php?p=111399#p111399

comment:10 in reply to: ↑ 9 Changed 10 months ago by kzar

Replying to mapx:

Replying to fanboy:

Maybe we should have a $webrtc filter option?

why another option ? better keeping websocket for all this crap and will use the filters already in easylist.

Well because webrtc connections are not the same as websocket connections. (Remember that with Firefox the extension API allows us to intercept websocket connections without the wrapper and hopefully Chrome will in the future too.)

comment:11 Changed 9 months ago by fanboy

So can we can temp fix for webrtc or something (anything to combat it). Waiting on X and Y won't help the enduser, all they see is ads and in-action.

comment:12 Changed 9 months ago by kzar

Well I need to check with Wladimir and Sebastian before beginning work to wrap the WebRTC APIs. (Since we will likely need to consider those connections as a new webrtc type core changes will be required.) It will probably be quite a bit of work and if we don't agree on an approach before I start it'll end up wasting a bunch of my time. I'll update the issue(s) when I've heard back from them about it.

comment:13 Changed 9 months ago by fanboy

From the current (growing) list of sites using the webrtc circumvention (seen from uBlock-Origin

britannica.com
businessnewsdaily.com
collegehumor.com
dorkly.com
investopedia.com
laptopmag.com
lifeandstylemag.com
mashable.com
newsarama.com
phonearena.com
space.com
theberry.com
thechive.com
thepoliticalinsider.com
tomsguide.com

comment:14 Changed 8 months ago by mapx

pm message:

webRTC ads (huge adoption)
Sent: Mon Feb 13, 2017 10:46 am	
From: dzr156	
To: mapx 

Hey mapx, 

recently we see huge adoption of webRTC ads in the top 5 websites in Israel, for example: 

mako.co.il
jpost.com
walla.co.il
bizportal.co.il
calcalist.co.il

You guys must do something with that, people uninstall Adblock or moving to ubo. And the guys from the company that provides this solution claims that the "ad blocking is dead". 

please keep me posted. 

thnks

comment:15 Changed 8 months ago by mapx

  • Cc dzr156 added

comment:16 Changed 8 months ago by kzar

I also received a similar PM, seems like this WebRTC circumvention is increasingly becoming a problem.

Wladimir / Sebastian what do you think to my earlier suggestion?

If WebRTC is an increasing privacy + circumvention problem like you guys suggest I
wouldn't be against adding a webrtc request type. We could then add wrapper(s) for the
WebRTC APIs which used that request type and in turn filters with $webrtc could be
written.

comment:17 Changed 8 months ago by sebastian

So you are talking about comment:8, where you suggest to wrap the Web RTC API, like we did for the WebSocket API, but with a different request type? IIRC, as we did that for WebSockets, it didn't take long until people discovered a way to bypass our wrapper, by initiating the connection from a document loaded through a data: URL, web workers and alike. Won't we have the same problem with Web RTC? Is it possible to leverage CSP to address that?

Also, is there any reason the webRequest API cannot intercept Web RTC connections, other than they just forgot to implement it like they did for WebSockets? It might be worth filing a Chromium bug.

Last edited 8 months ago by sebastian (previous) (diff)

comment:18 Changed 8 months ago by fanboy

Maybe we have a working around in ABP, until which Chromium can fix it. Chromium bugs tend to be slower to get fixed?

comment:19 Changed 8 months ago by sebastian

I'm not against implementing a workaround in Adblock Plus per se. But we should still bring that issue up to the Chromium team, so that they will hopefully lift the limitations on their end, so that we can eventually get rid of the workaround, since using the webRequest API would be more powerful, reliable, maintainable and efficient than any workaround we could implement.

comment:20 follow-up: Changed 7 months ago by fanboy

Not to push the point, but are we getting a fix or not?

comment:21 in reply to: ↑ 20 Changed 7 months ago by kzar

Replying to fanboy:

Not to push the point, but are we getting a fix or not?

I know this is an important issue for you guys, I see read all your replies here, the private messages and emails I receive about it. I want to start working on this when I have the time, but until then please be patient.

comment:22 Changed 7 months ago by dzr156

Guys, this issue starting to be very urgent. IMO - its putting ABP in a bad light.
Here in Israel, all the big sites (ynet.co.il, walla.co.il, mako.co.il) using webRTC, and the Heb list fourm is officially recommended on ubo (example: https://github.com/easylist/EasyListHebrew/issues/233)

Last edited 7 months ago by dzr156 (previous) (diff)

comment:23 Changed 7 months ago by kzar

  • Owner set to kzar

Time allowing I hope to start with this next week. I'll keep you all posted.

comment:24 Changed 7 months ago by mapx

  • Cc SMed79 added

comment:25 Changed 7 months ago by dzr156

Hi, just update - I find the Extension WebRTC Leak Prevent useful here. https://chrome.google.com/webstore/detail/webrtc-leak-prevent/eiadekoaikejlgdbkbdfeijglgfdalml
maybe we can take some piece of code from this.

comment:26 Changed 7 months ago by kzar

  • Description modified (diff)

comment:27 Changed 7 months ago by kzar

  • Blocked By 5087 added

comment:28 Changed 7 months ago by kzar

  • Description modified (diff)

comment:29 Changed 7 months ago by kzar

  • Blocked By 5092 added

Changed 7 months ago by kzar

comment:30 Changed 7 months ago by kzar

Good news, I finally got some time to work on this today! Better news this wasn't too hard, since a lot of the code and lessons learned wrapping WebSocket could be reused here. Even better news I have a (very rough) patch that seems to be working nicely.

The devil is in the details though, so I have some work to do to tidy up my patch and to test everything really does work correctly. I'll keep you posted, and in the mean time I've uploaded a teaser screenshot.

comment:31 Changed 7 months ago by mapx

  • Cc IsraeliAdblocker added

comment:32 Changed 7 months ago by kzar

  • Component changed from Unknown to Platform
  • Description modified (diff)
  • Priority changed from Unknown to P2
  • Ready set
  • Review URL(s) modified (diff)
  • Status changed from new to reviewing

comment:33 Changed 7 months ago by kzar

  • Blocked By 5124 added

comment:34 Changed 7 months ago by kzar

  • Blocked By 5124 removed

comment:35 Changed 7 months ago by abpbot

A commit referencing this issue has landed:
Issue 4455, 5087, 5092 - Wrap RTCPeerConnection to allow blocking

comment:36 Changed 7 months ago by kzar

  • Cc Ross added
  • Description modified (diff)
  • Milestone set to Adblock-Plus-for-Chrome-Opera-next
  • Platform changed from Unknown / Cross platform to Chrome
  • Resolution set to fixed
  • Status changed from reviewing to closed

comment:37 Changed 7 months ago by fanboy

Does this give us a new $webrtc switch? How does it work?

comment:38 Changed 7 months ago by kzar

Yes exactly, there's now a request type of webrtc. Those connections should show up in the Adblock Plus devtools pane and should be blockable like any others. Check out the screenshot I uploaded to this issue a few days ago to see how it should work.

We're still drafting the blog post about this, I'll post a link to it here when it's ready.

comment:39 Changed 7 months ago by fanboy

Should we commit the filter changes before hand?

Looking at uBo's filters, I've compiled a list of webrtc-ad sites:

https://pastebin.com/raw/RAeCxGzH

No issues on formatting? was just using a similar approach to $websocket.

comment:40 Changed 7 months ago by kzar

Filters with the $webrtc option should just be ignored for versions of Adblock Plus that don't support it yet. So as long as you keep them separate then it should be fine to go ahead yes. I would recommend testing that first however, better safe than sorry.

comment:41 Changed 7 months ago by kzar

FWIW I just tested your filter with the latest code and it worked for me, WebRTC connections were blocked.

comment:42 Changed 7 months ago by fanboy

Great! A win for now, tho no doubt the advertisers will find new/alternative ways to circumvent Adblock Plus.

I'll check on the rest of the Easylist authors before making a large webrtc commit.

Last edited 4 months ago by fanboy (previous) (diff)

comment:43 follow-up: Changed 7 months ago by fanboy

What's the debug options like on webrtc, will it show up in the blocked items of either browser?

comment:44 in reply to: ↑ 43 Changed 7 months ago by mapx

Replying to fanboy:

What's the debug options like on webrtc, will it show up in the blocked items of either browser?

https://issues.adblockplus.org/attachment/ticket/4455/blocked-webrtc.png

"Those connections should show up in the Adblock Plus devtools pane and should be blockable like any others."

comment:45 Changed 7 months ago by mapx

and probably will land in the google store only after they (google) will fix this issue:
https://bugs.chromium.org/p/chromium/issues/detail?id=129353#c94

comment:46 Changed 7 months ago by dzr156

I've tested it a bit. works fine for http://www.tomsguide.com/. doesn't work for the uponit ads - http://www.ynet.co.il/home/0,7340,L-8,00.html and http://www.walla.co.il/

comment:47 Changed 7 months ago by fanboy

https://issues.adblockplus.org/ticket/242

This should be fixed asap, uponit is taken advantage of this bug in chrome

comment:48 follow-up: Changed 6 months ago by kzar

This issue is about allowing the blocking of WebRTC connections without breaking legitimate usage such as Google Hangouts. Help with testing that is appreciated, but discussion about other stuff like element hiding circumvention doesn't belong here.

comment:49 Changed 6 months ago by kzar

Whoops sorry, forgot to post a link to the blog post. Here you go: https://adblockplus.org/development-builds/new-filter-type-option-for-webrtc-connections

comment:50 Changed 6 months ago by kzar

  • Resolution fixed deleted
  • Review URL(s) modified (diff)
  • Status changed from closed to reopened

comment:51 Changed 6 months ago by kzar

  • Status changed from reopened to reviewing

comment:52 follow-up: Changed 6 months ago by fanboy

What happened? why the review?

comment:53 in reply to: ↑ 52 Changed 6 months ago by kzar

Replying to fanboy:

What happened? why the review?

Just some more work to harden our RTCPeerConnection wrapper against circumvention, since it's very hard to get this stuff right. Nothing will change with how the filters are applied or anything like that, don't worry.

comment:54 Changed 6 months ago by abpbot

A commit referencing this issue has landed:
Issue 4455 - Further harden the RTCPeerConnection wrapper

comment:55 Changed 6 months ago by kzar

  • Resolution set to fixed
  • Status changed from reviewing to closed

comment:56 in reply to: ↑ 48 Changed 6 months ago by kzar

Replying to kzar:

This issue is about allowing the blocking of WebRTC connections without breaking legitimate usage such as Google Hangouts. Help with testing that is appreciated, but discussion about other stuff like element hiding circumvention doesn't belong here.

Sorry, after a discussing this with dzr156 via email it turns out I was wrong. There is something else happening there. I can reproduce too, I've opened #5207 for that and I'm investigating.

comment:57 Changed 6 months ago by SMed79

Sorry for the noise but webrtc requests are not blocked for me. for example on :

merriam-webster.com
imore.com
androidcentral.com
closerweekly.com
collegehumor.com
connectedly.com
dorkly.com
firstforwomen.com
xda-developers.com
...

Do I missed something or i am doing something wrong?

comment:58 Changed 5 months ago by kzar

You're right, I can reproduce that for merriam-webster.com at least - it's the same / similar as the circumvention seen in #4586 and #5207. I have a rough fix for those issues working and it works for merriam-webster.com too.

comment:59 Changed 5 months ago by kzar

Fixes for those issues have landed now and should be included in development builds soon. Could you give those pages a try again and let me know if the WebRTC connections are blocked now? I'd be interested to hear about any website who are still managing to use WebRTC for circumvention, hopefully that shouldn't be possible any longer.

comment:60 Changed 4 months ago by Ross

Done. WebRTC connections can be blocked, appear in the DevTools panel. WebRTC functionality like Google Handouts still works as expected.

ABP 1.13.2.1785
Chrome 49 / 58 / Windows 7
Opera 39 / 45 / Windows 7

comment:61 Changed 4 months ago by Ross

  • Tester changed from Unknown to Ross
  • Verified working set

comment:62 Changed 4 months ago by kzar

  • Resolution fixed deleted
  • Status changed from closed to reopened

comment:63 Changed 4 months ago by kzar

  • Resolution set to fixed
  • Sensitive unset
  • Status changed from reopened to closed

comment:64 Changed 4 months ago by dzr156

Hi, could you check this site - http://www.calcalist.co.il/real_estate/articles/0,7340,L-3716614,00.html
Seems like a new way to bypass our new wrapper.

comment:65 Changed 4 months ago by kzar

Please could you start a new issue for that? This issue is now considered closed and has been made public.

Note: See TracTickets for help on using tickets.