Add $genericblock filter option that can be used to disable all generic request blocking filters for a website

[arthur]

Background

We need a way to disable all generic request blocking rules for a given website. At the moment we can only disable each generic filter individually which is time consuming and verbose!

(Also see ticket #616 which is very similar and interrelated but for generic element hiding rules instead of generic blocking ones.)

What to change

Add the new $genericblock filter option to the lib/filter... code in the adblockplus repository.

It should match all generic generic request blocking rules for the given domain(s) / sitekey(s) and act as a way to easily disable them.

When a $genericblock filter is applied to a page it should also recursively apply to all child frames.

Generic filter rules count as rules that:

• Do not have a domain / sitekey specified. "Block this request for all domains"
• Have only domain / sitekey exceptions specified. "Block this request for all domains except example.com"

[barbaz]

This could be the effect of the $~elemhide option on whitelist rules (IIUC that option currently has no meaning).

(Can someone please add me to CC list for this issue as I don't see how to do it?)

[Crits]

I think generic rules can be defined as the rules beginning by ||.
I don't think the rules beginning by |http: can be considered as generic rules, and many of them are already used to circumvent anti-adblock anyway.

@barbaz : $~elemhide may not be understandable enough. Why not create a new option altogether?

[barbaz]

Replying to Crits:

I think generic rules can be defined as the rules beginning by ||.

In my enormous custom filter lists, it's the rules beginning with || that are the least generic. Why not define generic rules to be those with no leading anchor (meaning, not starting with | or ||)?
That way, this option is harder to unintentionally "abuse"; in otherwords, this definition of generic rules makes it unlikely that using this option will have the side-effect of whitelisting requests to "evil" domains that the end user does not ever want their computer connecting to.

@barbaz : $~elemhide may not be understandable enough. Why not create a new option altogether?

Could, but then there would be yet another meaningless inverse boolean filter option.
I was thinking that since $elemhide matches pages and disables all element hiding rules on the page, the logical inverse of that option for whitelist rules would be to disable blocking rules on the matched page.

Really, it depends how hard it is to add aditional type options and how often filterlist authors will want to use both $elemhide and this new option on the same page.

[Crits]

About ||: sorry, I of course meant not beginning by ||

[...] another meaningless inverse boolean filter option

Is it a big deal?
$~elemhide meaning whitelisting generic filters is not logical for me, as elements are not the contrary of HTTP requests.

[barbaz]

Replying to Crits:

[...] another meaningless inverse boolean filter option

Is it a big deal?

No big deal at all, it was just a suggestion.

[arthur]

btw it would be great if that would also allow to disable generic blocking rules in an iframe (iframeexample.com/adblock-test.html) on a site (example.com). I know a site that's doing that for quite some time (of course the URL of the iframe constantly changes).

[kzar]

So this issue and #616 have been progressing, if slowly. I've given Arthur a test build of the Firefox extension to try out and it seems we need to give this some more thought. What makes a filter "generic"?

At the moment filters are considered "generic" if there are no domain / sitekeys specified, or if there are only exceptions specified ​Specifically the implementation as it stands looks like this.

It has been suggested that a more useful test might be to just check if the filter text starts with | or not, rules that don't start with | could be considered generic. But as Arthur points out that's not a perfect test either, for example ||adserver.com^$third-party could arguably be considered generic.

(The other issue with this is that we need to check if filters are generic _a lot_, so we should be mindful to perform the minimum work possible in the check.)

So defining what actually makes a filter generic is currently blocking us, there's no point going ahead until we've agreed something that's actually useful for you guys!

Any suggestions?

[greiner]

I quickly talked to Arthur about that and apparently there are the following problems:

• If we block such filters then it would lead to a lot of additional effort in maintainance of the filters because each of those kind of generic filters would need to be explicitily blocked again by adding a specific blocking filter.

  ||adserver.com^$third-party
  @@||example.com^$genericblock
  ||adserver.com^$third-party,domain=example.com
  

• Not blocking such filters would, however, decrease the usefulness of this feature for countering adblocker detection solutions and it would make this filter option more confusing to understand correctly because it would no longer affect all generic filters.

Let's consult more filter authors before making a decision on that.

[famlam]

I assume it isn't an issue for #616?

Personally I'm not at all in favor of having this option for _blocking_ rules:

1. it kills privacy/social lists
2. requires large amounts of specific filters
   • for the site itself in the adblocking list (both first party and third party resources)
   • for the site itself in the social list and the privacy list (which is annoying as EP and social include international filters and are thus also affected by what the filter list for _name your random language_ does)
   • for any third party inline frames (including widgets, social media), which all need their specific filters to cover for them as well
   • therefore adds a very lot of filters that are useless on *all* other sites as other filters already cover for it.
3. sites that are troublesome enough to warrant a $genericblock filter in the first place, probably have no trouble with extending their anti-adblock checks for specific filters; and if the specific filters are used to block ads, then they can just change the path to the ads.
4. (it basically kills my redundancy check as I cannot say if the filter ||thirdpartydomain/ads/css. is redundant due to /ads/css., or should be kept because we have @@||firstpartydomain^$genericblock. This is of course not an issue for ABP, just for me, but it does no longer allow authors to find redundant filters, which are quite often present )

However, since other people (apparently) see the benefits, a relatively simple suggestion I could make, on top of the domain and sitekey check, is:
(with rule = the rule without the options)
rule[0] !== "|" || rule[rule.length - 1] === "^"
(or use String.prototype.startsWith/endsWith; not sure how fast they are).

However, if you wish to deal with
||domain/path^
||domain^path^
||domain*path^
I don't see any fast solutions, other than using rule.indexOf/rule.includes to check for all of them (/, ^ and *) separately...

(Note: don't get me wrong: I do love to get this feature for hiding rules; I just don't like it for blocking rules.)

[kzar]

Replying to famlam:

I assume it isn't an issue for #616?

The issue of how we define a generic filter applies to both #647 and #616, both will make use of the same .isGeneric() filter method. (Sorry it's a bit of a mess with the two separate issues at the moment.) We need to think of a check that will work for both blocking and hiding rules.

Personally I'm not at all in favor of having this option for _blocking_ rules...

Do any of the other filter list maintainers think $genericblock will be useful to them or do you agree with famlam's points? (There's nothing to stop us only implementing $generichide if that's more useful.)

However, since other people (apparently) see the benefits, a relatively simple suggestion I could make...

Thanks for the suggestions, very helpful!

[greiner]

Thanks everyone for your feedback. Since there is cause for a timely solution and the current implementation is almost through the review we are going to release it the way it is currently specified. That means that an exception rule with $genericblock will unblock all resources blocked by filters that don't specify either $domain or $sitekey.

That being said, this feature should be considered experimental and may go away at any point for something more appropriate so please use it sparingly but still wherever it is necessary. Note that a feature with the suggested functionality would require to be named differently no matter what due to it not applying to only some generic filters.

[mapx]

test case ABP firefox:
go to
​http://www.footmercato.net/ligue1/calendrier

add this filter

@@||footmercato.net^$genericblock

open blockable items list ==> I don't see in green all the whitelisted items (even if they are really whitelisted !)

for example this item: ​http://www.badibadou.eu/jk7dlkfe.js?auto=1&advid=5

[kzar]

I think this is to be expected due to the way that it's implemented, when $genericX options are in effect we only search for specific blocking rules. As far as the code is concerned the generic blocking rule never matched, instead of it being matched and then whitelisted.

This was a trade-off we made to avoid having to perform an extra search for matching filters. Otherwise we would have to search for all matching blocking filters, and then later for specific ones if a $genericX whitelist rule was in effect.

[mapx]

a side effect of this approach:
I can click a such (invisible) whitelisted item and create a blocking filter for it.

[kzar]

Well IMHO you should be able to create a blocking filter for those items, the suggested blocking filter should just be a specific one. I think you have a valid point anyway that this is not ideal as it stands.

[mapx]

right, the specific domain filters can be added.

[kzar]

OK, I have filed issue #3162 for this problem mapx. Thanks for reporting it, I'm going to try to see if I can at least fix the suggested filters in this situation.

In the mean time I'm going to finally close this issue :)

​https://hg.adblockplus.org/adblockplus/rev/893426c6a6ab
​https://hg.adblockplus.org/testpages.adblockplus.org/rev/2a068b22299a
​https://hg.adblockplus.org/adblockplustests/rev/449058480108
​​https://hg.adblockplus.org/adblockpluschrome/rev/68f1d78e2832

[trev]

Adblock Plus 2.6.12 is being released from branch, this change is part of it.