Opened 3 years ago

Closed 15 months ago

Last modified 14 months ago

#1727 closed defect (fixed)

WebSocket connections can't be blocked

Reported by: fanboy Assignee: kzar
Priority: P2 Milestone: Adblock-Plus-1.12.2-for-Chrome-Opera-Safari
Module: Platform Keywords:
Cc: sebastian, mapx, greiner, kzar, arthur, fhd, Lain_13, Ross, scheer Blocked By: #4331, #4332
Blocking: Platform: Chrome
Ready: yes Confidential: no
Tester: Unknown Verified working: yes
Review URL(s):

https://codereview.adblockplus.org/29347034/
https://codereview.adblockplus.org/29349737/

Description (last modified by kzar)

Environment

  • Adblock Plus for Chrome 1.11, Chrome 50.
  • EasyList enabled, including the filter ||bulletproofserving.com^$third-party.

How to reproduce

  1. Visit http://www.opensubtitles.org/
  2. Open the Chrome developer tools, click the Network tab and then the WS section.

Observed behaviour

A WebSocket to ws://ws.bulletproofserving.com:6001/ is successfully opened, adverts are displayed on the page.

Expected behaviour

The WebSocket connection/messages should be blocked, there should be no adverts displayed.

Notes

Chrome currently doesn't allow extensions to block WebSockets, Chromium bug #129353 has been open for some time now to address this limitation but little progress seems to have been made.

In the mean time both uBlock Origin and the Adguard browser extension have added a workaround. They use a content script to inject a wrapper for WebSocket into pages. The wrapper performs a dummy web request before WebSocket messages are sent/received. The extension recognises these dummy web requests as representing a WebSocket message. It intercepts and blocks them if the corresponding WebSocket message should be blocked. The WebSocket wrapper then allows / blocks the WebSocket message based on whether the dummy web request was blocked or not.

This is becoming an increasingly important problem as more and more websites are using WebSockets for advertising. We now need to add a similar work around to adblockpluschrome

References

Hints for testers

In supported versions of Chrome, Opera and Safari test the following:

It's important to test as many websites that use WebSockets properly as possible, wrapping WebSocket like this could well cause problems. (For more examples see the discussions linked in the References section above.)

We refactored some of the code added with issue #1677 whilst making these changes. So it's also important to make sure that code to protect our ElemHide stylesheets still works. In case the US version of Yahoo no longer uses this circumvention technique here are the steps I took to test the feature. (All in the console for a webpage.)

  1. Find our stylesheet: var sheet = document.documentElement.shadowRoot.styleSheets[0]; (Note: On some browsers you will need to remove the ".shadowRoot", also note that 0 might be the wrong index. You'll have to experiment to figure out the correct number.)
  2. Make sure you have the correct stylesheet by checking its first rule: sheet.rules[0]; (It should have a whole bunch of selectors to hide advertisements.)
  3. Once you have the correct stylesheet take note of the number of rules it contains: sheet.rules.length; (For me, with just EasyList enabled I saw 87 rules.)
  4. Now try removing a rule: sheet.removeRule(0); sheet.deleteRule(0);
  5. Now check no rules were removed, by checking the number of rules again: sheet.rules.length;
  6. Finally ensure our sheet can't be disabled: sheet.disabled = true; sheet.disabled; (Should display false.)

Finally we also refactored some code relating to YouTube adblocking for older versions of Safari which used Flash. Please test that YouTube adverts do not play when using an old version of Safari, to make sure a regression like #4141 has not resurfaced. (Take a look at that issue for some more testing approaches.)

Change History (35)

comment:1 Changed 3 years ago by sebastian

  • Cc sebastian added
  • Component changed from Unknown to Platform
  • Description modified (diff)
  • Resolution set to worksforme
  • Status changed from new to closed

With no filter subscription but this one filter configured, I get the popup, both with Chrome and Firefox. However when I use EasyList I don't get a popup there with either browser.

However, note that Adblock Plus for Chrome currently treats all requests with type other as if they are of the type object, to work around https://crbug.com/410382, which caused issues like #1372.

comment:2 Changed 3 years ago by fanboy

If its that's the case Chrome isn't blocking wsockd.com outright. There has been reports be a few users regarding this;

https://adblockplus.org/forum/viewtopic.php?f=2&t=26941
http://forums.lanik.us/viewtopic.php?f=62&t=20161

These were fixed by myself by disabling wsockd.com filter in Firefox, would be nice to see if Chrome blocking this script. If left alone wsockd will keep on generating popups for Chrome users.

comment:3 Changed 3 years ago by fanboy

Also wsock isn't limited to just the above site; Also seen on these. (probably far more)

fastpiratebay.eu
viooz.ac
katproxy.com

comment:4 Changed 3 years ago by sebastian

  • Keywords externaldependency added
  • Priority changed from Unknown to P2
  • Resolution worksforme deleted
  • Status changed from closed to reopened
  • Summary changed from Chrome supporting $other to WebSocket connections can't be blocked

It seems that Chrome currently doesn't allow extensions to intercept WebSocket connections. But they plan to implement it by https://crbug.com/129353. Once the webRequest API supports WebSocket connections we have to add the ws:// and wss:// URL schemes in our code.

comment:5 Changed 3 years ago by mapx

  • Cc mapx added

comment:6 Changed 3 years ago by sebastian

  • Priority changed from P2 to P3

comment:7 Changed 20 months ago by Lain_13

http://seasonvar.ru/serial-5296-Inuyasya-2-season.html - here is another example. This time it's marketgid.com ( ws://wsp.marketgid.com:8040/ws ).

Version 0, edited 20 months ago by Lain_13 (next)

comment:8 Changed 18 months ago by mapx

  • Tester set to Unknown
  • Verified working unset

from: https://bugs.chromium.org/p/chromium/issues/detail?id=168175#c49

Maybe we can use new JS features (Proxy, WeakMap) to intercept or wrap around the script execution from Chrome extension. There's an example of a wrapper around WebSocket API created by the developer behind uBlock Origin.
See detail here https://github.com/gorhill/uBlock/issues/1497

comment:9 Changed 18 months ago by mapx

  • Cc greiner kzar added

comment:10 Changed 17 months ago by arthur

  • Cc arthur added

uBlock Origin had some support for blocking WebSockets in version 1.7.0 but it was removed later and a separete extension was created. Not sure how good it works though.

comment:11 Changed 17 months ago by mapx

the guys from adguard improved the approach and the last idea was to add the websockets support back in the extension

https://github.com/gorhill/chromium-websocket-wrapper
https://github.com/AdguardTeam/AdguardBrowserExtension/issues/203

comment:12 Changed 17 months ago by kzar

  • Description modified (diff)
  • Keywords externaldependency removed
  • Priority changed from P3 to P2
  • Ready set

We agree this is an important problem, we'll tackle it as soon as possible. Unfortunately that likely won't be before late June / early July.

comment:13 Changed 16 months ago by fhd

  • Cc fhd added

comment:14 Changed 16 months ago by kzar

  • Owner set to kzar

comment:15 Changed 16 months ago by mapx

  • Cc Lain_13 added

comment:16 Changed 16 months ago by Lain_13

BTW, I've implemented my own stand-alone wrapper-blocker (actually instead of blocking it simulates working websocket).
https://greasyfork.org/en/scripts/19144-websuckit

Rather simple and dirty, but it works.

Last edited 16 months ago by Lain_13 (previous) (diff)

comment:17 Changed 16 months ago by kzar

Thanks Lain_13, interesting to see your approach, it didn't occur to me to use a Proxy object and that actually would be a pretty nice way to wrap WebSocket, preventing access to the original constructor. Unfortunately we support older browser versions (Safari 9 and Chrome < 49) which don't have Proxy support.

I'm on it anyway, will update you guys when I have something I'm happy with!

comment:18 Changed 16 months ago by kzar

Making some progress, little teaser for you guys http://i.imgur.com/QzhFrFq.png :)

comment:19 Changed 16 months ago by kzar

  • Review URL(s) modified (diff)
  • Status changed from reopened to reviewing

comment:20 Changed 16 months ago by kzar

  • Description modified (diff)

comment:21 follow-up: Changed 16 months ago by fanboy

So filters like this should also work?

|ws://nodesocket-$other,domain=thewatchseries.to
|ws://$other,third-party,domain=jpost.com

comment:22 Changed 16 months ago by Lain_13

[moved this comment to code review]

Last edited 16 months ago by Lain_13 (previous) (diff)

comment:23 in reply to: ↑ 21 Changed 16 months ago by kzar

Replying to fanboy:

So filters like this should also work?

Yup, think they should work :)

comment:24 Changed 15 months ago by kzar

  • Description modified (diff)

comment:25 Changed 15 months ago by kzar

  • Description modified (diff)

comment:26 Changed 15 months ago by kzar

  • Description modified (diff)

comment:27 Changed 15 months ago by abpbot

A commit referencing this issue has landed:
Issue 1727 - Prevent circumvention via WebSocket

comment:28 Changed 15 months ago by kzar

  • Cc Ross scheer added
  • Milestone set to Adblock-Plus-for-Chrome-Opera-Safari-next
  • Resolution set to fixed
  • Status changed from reviewing to closed

Heads up Ross / Scott, this one will need a lot of testing!

comment:29 Changed 15 months ago by sebastian

  • Review URL(s) modified (diff)

comment:31 Changed 14 months ago by trev

  • Blocked By 4331 added

comment:32 Changed 14 months ago by trev

  • Blocked By 4332 added

comment:33 Changed 14 months ago by rraceanu

  • Verified working set

Ads circumventing ABP with WebSocket are being blocked.

Win 10 Home x64- 10586
Chrome 52
Opera 39, 41(dev build)

Win 7 x64
Chrome 30,31,42,43,50
Opera 19,23,28,39,36

  • thewatchseries.to/cale.html?r=aHR0cDovL2dvcmlsbGF2aWQuaW4veDBlM252eWk2MXh2

||ws-gateway.com^$third-party (blocked - RuAdlist+Easylist)

  • ikinohd.com/19876-smertelnoe-nasledstvo.html

||brokeloy.com^$third-party (RuAdlist+Easylist)

  • seasonvar.ru/serial-13138-Ostrov_rus.html (WS connections blocked)

||marketgid.com^$third-party (blocked - Easylist)

||psma01.com^$third-party (blocked - Easylist)

  • opensubtitles.org/de (ws connections blocked)

||bulletproofserving.com^$third-party (blocked - Easylist)

  • slickdeals.net (blocked - Easylist)
  • www.websocket.org/echo.html works properly.
  • pwnwin.com/dashboard (header and sidebar loads)
  • twitch.tv (chat loads promptly - 3-5 sec)

Safari 6 OS 10.8 x64

  • ikinohd.com

ws://brokeloy.com:8040/ - index 3 (blocked)
||brokeloy.com^$third-party

  • seasonvar.ru/serial-13138-Ostrov_rus.html ("ws://wsp.marketgid.com:8040/ws") - index 1 (not blocked)

||marketgid.com^$third-party - No ads displayed.

("ws://psma01.com/list") - index 3 (blocked)

||psma01.com^$third-party (blocked - Easylist) - (blocked -Easylist)

  • Opensubtitles.org

("ws://ws.bulletproofserving.com:6004/") - index 3 (blocked)
||bulletproofserving.com^$third-party (blocked - Easylist)

  • Slickdeals.net (blocked - Easylist)
  • Pwnwin.com/dashboard (not supported on Safari 6, page is broken)
  • twitch.tv chat works properly
  • www.websocket.org/echo.html works properly.

Safari 7 - OSX 10.9 x64
Safari 9 - OS 10.11 x64

  • thewatchseries.to/cale.html?r=aHR0cDovL2dvcmlsbGF2aWQuaW4veDBlM252eWk2MXh2

||ws-gateway.com^$third-party (blocked - RuAdlist+Easylist)

  • ikinohd.com

||brokeloy.com^$third-party (blocked - RuAdlist+Easylist)

  • seasonvar.ru/serial-13138-Ostrov_rus.html (WS connections blocked)

||marketgid.com^$third-party (blocked - Easylist)
||psma01.com^$third-party (blocked - Easylist)

  • opensubtitles.org/de (ws connections blocked)

||bulletproofserving.com^$third-party (blocked - Easylist)

  • slickdeals.net (blocked - Easylist)
  • www.websocket.org/echo.html works properly.
  • pwnwin.com/dashboard (header and sidebar loads)
  • twitch.tv (chat loads promptly - 3-5 sec)
Last edited 14 months ago by mapx (previous) (diff)

comment:34 Changed 14 months ago by routehero

Users should be presented with the fact that Adblock Plus will overwrite WebSocket() functionality.

Failing to do so does not allow the user to make informed decisions about what is happening in their browser.

Malicious extension authors could also use this technique. This strikes me as a slippery slope.

comment:35 Changed 14 months ago by Lain_13

Oh man, you came here as well. I'll just leave this link here:
https://bugs.chromium.org/p/chromium/issues/detail?id=129353#c58
Just for the reference. Everyone here already seen this discussion.

Last edited 14 months ago by Lain_13 (previous) (diff)
Note: See TracTickets for help on using tickets.