Opened 11 months ago

Last modified 2 weeks ago

#4280 reopened defect

Safari "page's address isn't valid" error when using the content blocking API

Reported by: scheer Assignee: arthur
Priority: P1 Milestone:
Module: Adblock-Plus-for-iOS Keywords: AcceptableAds
Cc: mario, kzar, Ross, scheer, CraftyDeano Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://codereview.adblockplus.org/29349794/
https://codereview.adblockplus.org/29354751/
https://codereview.adblockplus.org/29370756/

Description (last modified by kzar)

Environment

iPhone 6s Plus
iOS 9.3.1
ABP iOS 1.1.0

OR

Safari 9.1 for Mac OS X
An otherwise empty extension that uses this Content Blocker File:

[
	{
		"trigger": {
			"url-filter": "^https?://.*&advid=",
			"resource-type": [
				"document"
			]
		},
		"action": {
			"type": "block"
		}
	}
]

How to reproduce

  1. Visit https://www.google.co.uk/aclk?sa=L&ai=DChcSEwjC6rqt5p_OAhVGuBsKHUoeAG8YABAA&sig=AOD64_2kAoVInCmwfi5kot9P5h-jb1GQ7Q&q=&ved=0ahUKEwjA-bet5p_OAhVlBcAKHVU4CDMQ0QwIIQ&adurl=

OR

  1. Visit https://www.google.de and search 'Samsung Galaxy S7'
  2. Select the Ad with the URL 'www.base.de/galaxy-s7'

OR

  1. Visit https://au.search.yahoo.com and search 'buy car'
  2. Select the ad 'www.AMSVans.com'

Observed behaviour

On iOS this error message is displayed: Safari cannot open the page 'The Error was. 'The URL can't be shown'.

For Safari 9.1 on Mac OS X this message is displayed: "Safari can't open the page "​https://www.google.co.uk/aclick?..." because the page's address isn't valid".

Expected behaviour

No error message displayed, browser is redirected to the advertisement.

Notes

  • The URL redirects to http://bs.serving-sys.com/BurstingPipe/AdServer.bs?cn=ccs&ebcmp=10430353&ebkw=249705620&advid=11899&ebag=9642384&sead=105349998804&ccsdev=c&ccsurl=$$https://www.nissan.co.uk/?&cid=psmh565SKgG_dc|U&cid=psmh565SKgG_dc|U$$ which is then blocked by the content blocking rule. Safari unhelpfully displays the wrong error message, stating that the URL can't be parsed!
  • A possible longer-term work around for this issue is #4327.

Attachments (2)

image1.PNG (76.0 KB) - added by scheer 11 months ago.
Screen Shot 2016-08-01 at 10.30.54.png (50.8 KB) - added by kzar 11 months ago.

Download all attachments as: .zip

Change History (40)

Changed 11 months ago by scheer

comment:1 Changed 11 months ago by scheer

Whilst testing, I also used the same search criteria on www.yahoo.co.uk for 'buy car'. When selecting the ad link for www.nissan.co.uk/car, the advertised URL loads correctly.

comment:2 Changed 11 months ago by scheer

bing.com also works correctly with the above mentioned URL's.

comment:3 Changed 11 months ago by scheer

  • Description modified (diff)

comment:4 Changed 11 months ago by kzar

  • Cc mario added
  • Description modified (diff)
  • Owner set to kzar
  • Platform changed from iOS to Unknown / Cross platform
  • Priority changed from Unknown to P1
  • Ready set
  • Summary changed from 'URL can't be shown' when selecting a particular ad-link from Google search results in ABP iOS to Safari "page's address isn't valid" error when using the content blocking API

This looks like it could be a Safari bug, but before we continue I want to try to figure out which rules trigger the problem.

comment:5 Changed 11 months ago by kzar

  • Description modified (diff)

comment:6 Changed 11 months ago by kzar

  • Description modified (diff)

Changed 11 months ago by kzar

comment:7 Changed 11 months ago by kzar

  • Description modified (diff)
  • Keywords externaldependency added

I have filed a bug with Apple about this, bug number is 27656882.

comment:8 Changed 11 months ago by kzar

  • Owner kzar deleted

I haven't managed to find a quick work around for this problem, I'm not sure what we can do at this point. I've tried removing all rules with filters containing .*&ad but that didn't help. To work around this our end we would likely have to remove huge portions of the rule list, and good luck finding the exact combination of the ~50,000 rules that works!

Apple are hardly responsive to bug reports. Perhaps we could instead work with Google to help them avoid using URLs that trigger the bug?

comment:9 Changed 11 months ago by kzar

  • Description modified (diff)

comment:10 Changed 11 months ago by kzar

  • Description modified (diff)

comment:11 Changed 11 months ago by kzar

  • Cc kzar added

comment:12 Changed 11 months ago by mario

  • Keywords AcceptableAds added

comment:13 Changed 10 months ago by sebastian

I filtered the list for any blocking rule whose regular expression matches this URL, and I found this one, which would essentially block that page. So it's probably not a Safari bug, though the error message is quite misleading.

{
    "trigger": {
        "url-filter": "^https?://.*&adurl=",
        "resource-type": [
            "image",
            "style-sheet",
            "script",
            "font",
            "media",
            "raw",
            "document"
        ]
    },
    "action": {
        "type": "block"
    }
}

Note that there is no distinction between frames and top-level documents with Content Blockers. Both are blocked by the "request-type": ["document"].

Last edited 10 months ago by sebastian (previous) (diff)

comment:14 Changed 10 months ago by kzar

I don't think that's the case, I can reproduce the problem without that rule. (See my notes in the Environment section about how I could reproduce this problem with only the ^https?://.*&advid= blocking rule.)

comment:15 Changed 10 months ago by sebastian

The server returns a redirect to a URL like this, which is blocked by that rule:

http://bs.serving-sys.com/BurstingPipe/AdServer.bs?cn=ccs&ebcmp=10430353&ebkw=249705620&advid=11899&ebag=9642384&sead=105349998804&ccsdev=c&ccsurl=$$https://www.nissan.co.uk/?&cid=psmh565SKgG_dc|U&cid=psmh565SKgG_dc|U$$

comment:16 Changed 10 months ago by kzar

  • Cc Ross scheer added
  • Description modified (diff)
  • Keywords externaldependency removed
  • Owner set to kzar
  • Review URL(s) modified (diff)

Yes, looks like you're right, remind me to completely ignore Safari's error messages in the future...

I've put a change under review to whitelist the domain for that example, but the same problem could still be present for other adverts.
Mario / Ross / Scott - could you guys do some further testing for me and see if you can find any other similarly broken ads? (Please paste the link's address for any that you find, rather than explaining how you performed the search.)

comment:17 Changed 10 months ago by abpbot

A commit referencing this issue has landed:
Issue 4280 - Begin whitelisting domains used by Google redirects

comment:18 Changed 10 months ago by abpbot

A commit referencing this issue has landed:
Issue 4280 - Whitelist further domains used by Google redirections

comment:19 Changed 9 months ago by rraceanu

The issue also occurs on about.com when searching for Iphone 6s

Environment:
Iphone 6
iOS 9.1
ABP 1.1.0 (last update Thursday 15 September)

  1. Open about.com with Safari and ABP enabled
  2. Search for Iphoen 6 and open the link from m.base.de/iphone-6s

The link will open a new tab trying to redirect the user, but the page cannot be opened with the Error "The URL can't be shown". The link for that page is "googleadservices.com"

comment:20 Changed 9 months ago by kzar

Please right click on the troublesome link, select "Copy Link" and paste that.

Unfortunately we're playing whack-a-mole with this issue for now. When one of these links are clicked the browser is redirected multiple times and if any of the redirected URLs are blocked this error shows. Worse still Safari displays the wrong URL when the error is shown, so without the original link we can't do anything!

Thanks for your help testing this.

(For reference #4327 is a possible longer-term solution to this problem.)

comment:22 Changed 9 months ago by kzar

  • Description modified (diff)

comment:23 Changed 9 months ago by arthur

  • Owner changed from kzar to arthur

comment:24 Changed 9 months ago by arthur

  • Review URL(s) modified (diff)
  • Status changed from new to reviewing

comment:25 Changed 9 months ago by mario

  • Cc CraftyDeano added

comment:26 Changed 9 months ago by abpbot

A commit referencing this issue has landed:
Issue 4280 - Whitelist further domains used by Google redirections

comment:27 Changed 9 months ago by arthur

  • Status changed from reviewing to reopened

comment:28 Changed 6 months ago by Bravy02

I still run into the same issue.
iOS 10.2 (iPad Air 2)
Adblock Plus 1.1.0

If I go to www.google.com in Safari and search for "Flatware" and then try to click one of the links near the top under "Shop For Flatware", I'm taken to the error "Safari cannot open the page. The error was: The URL was blocked by a content blocker".

The URL when the error is displayed is "https://www.google.com/aclk?sa=l&ai=DChcSEwiju-jM1_PQAhUDuMAKHUTYB9EYABAZ&sig=AOD64_1THDIeesXV1s1p1w6-6P0h1tICrQ&ctype=5&q=&ved=0ahUKEwjX0eHM1_PQAhWB54MKHS49DQkQwg8IHA&adurl="

I have tried whitelisting google.com and googleadservices.com in my Adblock Plus config but it did not help. Thank you.

comment:29 Changed 6 months ago by arthur

  • Review URL(s) modified (diff)
  • Status changed from reopened to reviewing

comment:30 Changed 6 months ago by abpbot

A commit referencing this issue has landed:
Issue 4280 - Include sub-domains for xg4ken.com click redirects

comment:31 Changed 6 months ago by arthur

  • Status changed from reviewing to reopened

@Bravy02
Please try it with a manual filter update in ~20 minutes.

comment:32 Changed 5 months ago by Bravy02

@arthur
I ran a manual filter update and still run into the same issue on my iPad Air 2 running iOS 10.2. I even removed "google.com" and "googleadservices.com" from the whitelist.

comment:33 follow-up: Changed 5 months ago by arthur

Strange. Could you add the URL of the ad as you did last time?

comment:34 in reply to: ↑ 33 Changed 5 months ago by Bravy02

Replying to arthur:

Strange. Could you add the URL of the ad as you did last time?

Sure thing. If you search www.google.com for "Flatware" and click on the first shopping picture for Walmart it's this URL which is still blocked by a content blocker.

https://www.googleadservices.com/pagead/aclk?sa=L&ai=DChcSEwjHtufOsuLRAhUbVg0KHZBDAVwYABAY&ohost=www.google.com&cid=CAESIuD2pjpXqclT2dlEoCZ5T84d_zE2rYGvRimS6IjhMwI9f6w&sig=AOD64_2l17LGUea1F_qG9IrjmN1jAdxdMw&ctype=5&q=&ved=0ahUKEwjWvOPOsuLRAhVIzIMKHZfFCc0Qwg8IHA&adurl=

Note that if I manually remove "&adurl=" from the end of that URL, this URL works in Safari on my iPad (iOS 10.2.1) with the latest filter list:

https://www.googleadservices.com/pagead/aclk?sa=L&ai=DChcSEwjHtufOsuLRAhUbVg0KHZBDAVwYABAY&ohost=www.google.com&cid=CAESIuD2pjpXqclT2dlEoCZ5T84d_zE2rYGvRimS6IjhMwI9f6w&sig=AOD64_2l17LGUea1F_qG9IrjmN1jAdxdMw&ctype=5&q=&ved=0ahUKEwjWvOPOsuLRAhVIzIMKHZfFCc0Qwg8IHA

Last edited 5 months ago by Bravy02 (previous) (diff)

comment:35 follow-up: Changed 4 months ago by arthur

Sorry for my late reply!

Thanks! I tried to reproduce this on a Mac Mini and iPad Air 2 but couldn't. The links from you work fine and I also tried it with some Walmart ads when searching for "flatware".

Maybe you could try it again after clearing cache and cookies in Safari and re-installing Adblock Plus?

comment:36 in reply to: ↑ 35 Changed 4 months ago by Bravy02

Replying to arthur:

Sorry for my late reply!

Thanks! I tried to reproduce this on a Mac Mini and iPad Air 2 but couldn't. The links from you work fine and I also tried it with some Walmart ads when searching for "flatware".

Maybe you could try it again after clearing cache and cookies in Safari and re-installing Adblock Plus?

No worries, thank you for your help!

Actually I just updated the filter list again (it was last updated 1/27/17) and then disabled/re-enabled AdBlock Plus in the iOS Safari Content Blocker settings and now the Google Shopping Ad URLs are no longer blocked. I also tried the Google Ad URL for Walmart flatware that I previously posted and it works now too!

Let's close this out, thank you again.

comment:37 Changed 4 months ago by arthur

Thanks for confirming!

comment:38 Changed 2 weeks ago by abpbot

A commit referencing this issue has landed:
Issue 4280 - Whitelist further domains used by Google redirections

Note: See TracTickets for help on using tickets.