WebSocket connections can't be blocked

[fanboy]

Environment

• Adblock Plus for Chrome 1.11, Chrome 50.
• EasyList enabled, including the filter ||bulletproofserving.com^$third-party.

How to reproduce

1. Visit ​http://www.opensubtitles.org/
2. Open the Chrome developer tools, click the Network tab and then the WS section.

Observed behaviour

A WebSocket to ws://ws.bulletproofserving.com:6001/ is successfully opened, adverts are displayed on the page.

Expected behaviour

The WebSocket connection/messages should be blocked, there should be no adverts displayed.

Notes

Chrome currently doesn't allow extensions to block WebSockets, ​Chromium bug #129353 has been open for some time now to address this limitation but little progress seems to have been made.

In the mean time both uBlock Origin and the Adguard browser extension have added a workaround. They use a content script to inject a wrapper for WebSocket into pages. The wrapper performs a dummy web request before WebSocket messages are sent/received. The extension recognises these dummy web requests as representing a WebSocket message. It intercepts and blocks them if the corresponding WebSocket message should be blocked. The WebSocket wrapper then allows / blocks the WebSocket message based on whether the dummy web request was blocked or not.

This is becoming an increasingly important problem as more and more websites are using WebSockets for advertising. We now need to add a similar work around to adblockpluschrome

References

• ​https://github.com/AdguardTeam/AdguardBrowserExtension/issues/203
• ​https://github.com/gorhill/uBlock/issues/1604
• ​https://github.com/AdguardTeam/AdguardBrowserExtension/blob/HEAD/Extension/browser/chrome/lib/content-script/websocket.js
• ​https://github.com/gorhill/uBO-WebSocket

Hints for testers

In supported versions of Chrome, Opera and Safari test the following:

• That websites using WebSockets to circumvent us no longer can. For example browse to ​http://www.opensubtitles.org/ and make sure adverts aren't displayed and that some WebSocket connections are blocked. (See ​this screenshot.)
• That websites using WebSockets properly do not break. For example load a stream on ​https://www.twitch.tv/ and make sure the chat window connects straight away and you can see people's messages. That the header and sidebar of ​http://www.pwnwin.com/dashboard load. That this WebSocket demo site still works correctly ​http://www.websocket.org/echo.html

It's important to test as many websites that use WebSockets properly as possible, wrapping WebSocket like this could well cause problems. (For more examples see the discussions linked in the References section above.)

We refactored some of the code added with issue #1677 whilst making these changes. So it's also important to make sure that code to protect our ElemHide stylesheets still works. In case the US version of Yahoo no longer uses this circumvention technique here are the steps I took to test the feature. (All in the console for a webpage.)

1. Find our stylesheet: var sheet = document.documentElement.shadowRoot.styleSheets[0]; (Note: On some browsers you will need to remove the ".shadowRoot", also note that 0 might be the wrong index. You'll have to experiment to figure out the correct number.)
2. Make sure you have the correct stylesheet by checking its first rule: sheet.rules[0]; (It should have a whole bunch of selectors to hide advertisements.)
3. Once you have the correct stylesheet take note of the number of rules it contains: sheet.rules.length; (For me, with just EasyList enabled I saw 87 rules.)
4. Now try removing a rule: sheet.removeRule(0); sheet.deleteRule(0);
5. Now check no rules were removed, by checking the number of rules again: sheet.rules.length;
6. Finally ensure our sheet can't be disabled: sheet.disabled = true; sheet.disabled; (Should display false.)

Finally we also refactored some code relating to YouTube adblocking for older versions of Safari which used Flash. Please test that YouTube adverts do not play when using an old version of Safari, to make sure a regression like #4141 has not resurfaced. (Take a look at that issue for some more testing approaches.)

[sebastian]

With no filter subscription but this one filter configured, I get the popup, both with Chrome and Firefox. However when I use EasyList I don't get a popup there with either browser.

However, note that Adblock Plus for Chrome currently treats all requests with type other as if they are of the type object, to work around ​https://crbug.com/410382, which caused issues like #1372.

[fanboy]

If its that's the case Chrome isn't blocking wsockd.com outright. There has been reports be a few users regarding this;

​https://adblockplus.org/forum/viewtopic.php?f=2&t=26941
​http://forums.lanik.us/viewtopic.php?f=62&t=20161

These were fixed by myself by disabling wsockd.com filter in Firefox, would be nice to see if Chrome blocking this script. If left alone wsockd will keep on generating popups for Chrome users.

[fanboy]

Also wsock isn't limited to just the above site; Also seen on these. (probably far more)

fastpiratebay.eu
viooz.ac
katproxy.com

[sebastian]

It seems that Chrome currently doesn't allow extensions to intercept WebSocket connections. But they plan to implement it by ​https://crbug.com/129353. Once the webRequest API supports WebSocket connections we have to add the ws:// and wss:// URL schemes in our code.

[Lain_13]

​http://seasonvar.ru/serial-5296-Inuyasya-2-season.html - here is another example. This time it's marketgid.com ( ws://wsp.marketgid.com:8040/ws ). Instead of loading popups they are loading scripts and images, and then insert them into page using data:url. It's possible to circumvent by hiding, but if they will also use !important trick it will became a serious issue.

[mapx]

from: ​https://bugs.chromium.org/p/chromium/issues/detail?id=168175#c49

Maybe we can use new JS features (Proxy, WeakMap) to intercept or wrap around the script execution from Chrome extension. There's an example of a wrapper around WebSocket API created by the developer behind uBlock Origin.
See detail here ​https://github.com/gorhill/uBlock/issues/1497

[arthur]

uBlock Origin had some support for blocking WebSockets in ​version 1.7.0 but it was ​removed later and a separete extension ​was created. Not sure how good it works though.

[mapx]

the guys from adguard improved the approach and the last idea was to add the websockets support back in the extension

​https://github.com/gorhill/chromium-websocket-wrapper
​https://github.com/AdguardTeam/AdguardBrowserExtension/issues/203

[kzar]

We agree this is an important problem, we'll tackle it as soon as possible. Unfortunately that likely won't be before late June / early July.

[Lain_13]

BTW, I've implemented my own stand-alone wrapper-blocker (actually instead of blocking it simulates working websocket).
​https://greasyfork.org/en/scripts/19144-websuckit

Rather simple and dirty, but it works.

[kzar]

Thanks Lain_13, interesting to see your approach, it didn't occur to me to use ​a Proxy object and that actually would be a pretty nice way to wrap WebSocket, preventing access to the original constructor. Unfortunately we support older browser versions (​Safari 9 and Chrome < 49) which don't have Proxy support.

I'm on it anyway, will update you guys when I have something I'm happy with!

[kzar]

Making some progress, little teaser for you guys ​http://i.imgur.com/QzhFrFq.png :)

[fanboy]

So filters like this should also work?

|ws://nodesocket-$other,domain=thewatchseries.to
|ws://$other,third-party,domain=jpost.com

[Lain_13]

[moved this comment to code review]

[kzar]

Replying to fanboy:

So filters like this should also work?

Yup, think they should work :)

[abpbot]

A commit referencing this issue has landed:
​Issue 1727 - Prevent circumvention via WebSocket

[kzar]

Heads up Ross / Scott, this one will need a lot of testing!

[abpbot]

A commit referencing this issue has landed:
​Issue 1727 - Fix WebSocket constructor without second argument in Chrome 47

[rraceanu]

Ads circumventing ABP with WebSocket are being blocked.

Win 10 Home x64- 10586
Chrome 52
Opera 39, 41(dev build)

Win 7 x64
Chrome 30,31,42,43,50
Opera 19,23,28,39,36

• thewatchseries.to/cale.html?r=aHR0cDovL2dvcmlsbGF2aWQuaW4veDBlM252eWk2MXh2

||ws-gateway.com^$third-party (blocked - RuAdlist+Easylist)

• ikinohd.com/19876-smertelnoe-nasledstvo.html

||brokeloy.com^$third-party (RuAdlist+Easylist)

• seasonvar.ru/serial-13138-Ostrov_rus.html (WS connections blocked)

||marketgid.com^$third-party (blocked - Easylist)

||psma01.com^$third-party (blocked - Easylist)

• opensubtitles.org/de (ws connections blocked)

||bulletproofserving.com^$third-party (blocked - Easylist)

• slickdeals.net (blocked - Easylist)

• www.websocket.org/echo.html works properly.

• pwnwin.com/dashboard (header and sidebar loads)
• twitch.tv (chat loads promptly - 3-5 sec)

Safari 6 OS 10.8 x64

• ikinohd.com

ws://brokeloy.com:8040/ - index 3 (blocked)
||brokeloy.com^$third-party

• seasonvar.ru/serial-13138-Ostrov_rus.html ("ws://wsp.marketgid.com:8040/ws") - index 1 (not blocked)

||marketgid.com^$third-party - No ads displayed.

("ws://psma01.com/list") - index 3 (blocked)

||psma01.com^$third-party (blocked - Easylist) - (blocked -Easylist)

• Opensubtitles.org

("ws://ws.bulletproofserving.com:6004/") - index 3 (blocked)
||bulletproofserving.com^$third-party (blocked - Easylist)

• Slickdeals.net (blocked - Easylist)

• Pwnwin.com/dashboard (not supported on Safari 6, page is broken)
• twitch.tv chat works properly
• www.websocket.org/echo.html works properly.

Safari 7 - OSX 10.9 x64
Safari 9 - OS 10.11 x64

• thewatchseries.to/cale.html?r=aHR0cDovL2dvcmlsbGF2aWQuaW4veDBlM252eWk2MXh2

||ws-gateway.com^$third-party (blocked - RuAdlist+Easylist)

• ikinohd.com

||brokeloy.com^$third-party (blocked - RuAdlist+Easylist)

• seasonvar.ru/serial-13138-Ostrov_rus.html (WS connections blocked)

||marketgid.com^$third-party (blocked - Easylist)
||psma01.com^$third-party (blocked - Easylist)

• opensubtitles.org/de (ws connections blocked)

||bulletproofserving.com^$third-party (blocked - Easylist)

• slickdeals.net (blocked - Easylist)

• www.websocket.org/echo.html works properly.

• pwnwin.com/dashboard (header and sidebar loads)
• twitch.tv (chat loads promptly - 3-5 sec)

[routehero]

Users should be presented with the fact that Adblock Plus will overwrite WebSocket() functionality.

Failing to do so does not allow the user to make informed decisions about what is happening in their browser.

Malicious extension authors could also use this technique. This strikes me as a slippery slope.

[Lain_13]

Oh man, you came here as well. I'll just leave this link here:
​https://bugs.chromium.org/p/chromium/issues/detail?id=129353#c58
Just for the reference. Everyone here already seen this discussion.

[ricard85perez]

Hello colleagues, I have a problem. When I try to broadcast online with my coworkers via webcam, I get advertising from ​https://www.chatsexocam.com and I can not eliminate it. I have well configured adblockplus and I can not solve this problem and I always have to disconnect the webcam when I have to transmit on facebook or another videoconference page

[kzar]

Replying to ricard85perez:

Hello colleagues, I have a problem. When I try to broadcast online with my coworkers via webcam, I get advertising from chatsexocam.com and I can not eliminate it. I have well configured adblockplus and I can not solve this problem and I always have to disconnect the webcam when I have to transmit on facebook or another videoconference page

Hi Ricard,

It sounds like you could have malware on your computer, I would first of all install a virus scanner and check your computer for viruses and malware.

If that doesn't help you could report the advert that is not being blocked to ​EasyList, perhaps they will be able to add a filter which helps you.

Finally I should mention this is not the right place to report such problems, in the future please instead ask questions on the ​Adblock Plus forums.

Cheers, Dave.