Add $generichide filter option that can be used to disable all generic element hiding filters for a website

[Crits]

Background

We need a way to disable all generic element hiding rules for a given website. At the moment we can only disable each generic filter individually which is time consuming and verbose!

• ​See this ABP forum topic for more background.
• Also see ticket #647 which is very similar and interrelated but for generic request blocking rules instead of generic hiding ones.

What to change

Add the new $generichide filter option to the lib/filter... code in the adblockplus repository.

It should act similarly to the existing $elemhide option but instead of matching all element hiding rules it should only match the generic ones.

When a $generichide filter is applied to a page it should also recursively apply to all child frames.

Generic filter rules count as rules that:

• Do not have a domain specified. "Hide this element on all domains"
• Have only domain exceptions specified. "Hide this element on all domains except example.com"

Example generic filters:

• ~example.com##.ad

We also need to update the adblockplus dependency for the adblockpluschrome repository from bdd5cd8bca3a to 893426c6a6ab. The following unrelated changes to adblockplus have happened since then:

• ​d9aea02b4c79 - "Noissue - Fixed typo in Indonesian translation for viewList.label"
• ​a463f8a01e5f - "Issue 3086 - Add mapping for Adblock Browser logo on the first run page"
• ​2d8be2fd147b - "Releasing Adblock Plus 2.6.11"
• ​8a335396ba99 - "Added tag 2.6.11 for changeset 2d8be2fd147b"
• ​098d48f73764 - "Noissue - reverted unintentionally checked in debug changes"
• ​29d524332056 - "Releasing Adblock Plus 2.6.11"
• ​2c481ae80add - "Added tag 2.6.11 for changeset 29d524332056"

• The first simply fixes a mistake in an Indonesian translation, this should not matter to us.
• The second is simply adding a mapping for an ABB related graphic, again not relevant to us.
• Finally the last five focus on releasing version 2.6.11 and should not cause us any problems either.

Finally we need to update the adblockplustests dependency for the adblockpluschrome repository from ae6c7a2cc723 to 449058480108. The following unrelated changes to adblockplustests have happened since then:

• ​afcb18d3311b - "Issue 284 - Unify initial delay for all downloads"
• ​04616196f39b - "Noissue - Fixed syntax error (invalid left-hand expression)"
• ​7e924f798246 - "Noissue - Marked compatible with Firefox 43 & Co."
• ​63747b8afb94 - "Issue 2953 - Cleanup on unload is broken"
• ​c1b619da264f - "Noissue - Fix SVG unit tests"

• The first, second and fifth changes were to tests that aren't included by the Chrome extension test suite. (synchroniser, icon_position and policy respectively)
• The third change was to the metadata.gecko file which again is not used by the Chrome tests.
• The fourth change was to fix a problem with cleanup after some of the asynchronous unit tests. I don't think the issue showed up in any of the Chrome tests, but if anything adding this change should make things run more reliably.

I have also tested that these changes and no additional tests are failing as a result. (9 CSS escaping tests are currently failing but that is unrelated)

[arthur]

Thanks for creating this issue. I will also create one for generic blocking rules as well.

[Crits]

Replying to arthur:

Thanks for creating this issue. I will also create one for generic blocking rules as well.

That would be sooooo great!

[greiner]

Rather than introducing a new option, what about reusing elemhide for that? In such a case <domain>$elemhide would apply to all element hiding filters while <domain>$elemhide=~<domain> would only apply to filters that are not specific to that domain. Would that be sufficient?

However, there might be backwards-compatibility problems so <domain>$elemhide=~<domain> would probably be equal to <domain>$elemhide in older versions that don't support this syntax. Although implementing $generichide would probably have similar issues in this regard.

@trev: Regarding the technical feasibility of this feature, I wonder whether this would be blocked on #521.

[trev]

Although implementing $generichide would probably have similar issues in this regard.

Currently, the typical approach is writing rules like foo$generichide,image,~image to avoid this. For older versions (ones that don't know generichide) this is a rule not matching any types. For newer versions it is equivalent to foo$generichide.

@trev: Regarding the technical feasibility of this feature, I wonder whether this would be blocked on #521.

No, #521 doesn't affect this any more than it affects usual element hiding exceptions for example.

[famlam]

Rather than introducing a new option, what about reusing elemhide for that? In such a case <domain>$elemhide would apply to all element hiding filters while <domain>$elemhide=~<domain> would only apply to filters that are not specific to that domain. Would that be sufficient?

Although I would prefer the re-use of options, I'd rather suggest the use of boolean options. $elemhide=... would give a lot of possibilities that all have to be taken care of, e.g. <domain>$elemhide=~<domain2> or <domain>$elemhide=<domain> and all kind of combinations such as <domain.com>$elemhide=~sub.domain.com|domain.com|~com if we want to exclude certain rules only. Besides, it's slightly confusing as it has a completely different interpretation than the similar-syntax $domain= option.

I can think of a couple of those boolean options. The simplest is ~elemhide, which only purpose currently could be that ~elemhide is shorthand for ~image,image (e.g., match all resource types), although I cannot recall any occurrence of that so far.
Another possibility is ~third-party: adding this would get rid of all domain-specific (=first party) rules, which would be more easily to understand, even though the meaning is slightly different than ~third-party for regular rules (as hiding rules are of course by definition first-party, except for in subframes)
Another option is to completely replace the present elemhide with the newly suggested elemhide. The need for the current behavior is rare: you don't want to whitelist specific rules if you can as well remove those (or, in case of an external filter list that doesn't want to cooperate, you could eventually whitelist them with #@# rules). Requiring the current elemhide behavior on third-party subframes is even more rare, I cannot think of a realistic use case for that. Furthermore, elemhide is only known to list authors and the few users who have read the time to learn the syntax. It's not an option that can be added from within the extension (e.g. EHH or the block-a-resource dialog).

My first and second suggestion can also be ported to blocking rules (e.g. ~document, issue #647). The third suggestion would be specific to hiding rules.

However, there might be backwards-compatibility problems so <domain>$elemhide=~<domain> would probably be equal to <domain>$elemhide in older versions that don't support this syntax. Although implementing $generichide would probably have similar issues in this regard.

Wasn't this dealt with in a way that filters with unknown options were completely discarded, such that we don't even need the $image,~image syntax anymore? At least in EL we do no longer use $image,~image for $popup rules...

[greiner]

Replying to famlam:

Besides, it's slightly confusing as it has a completely different interpretation than the similar-syntax $domain= option.

Isn't that the general problem with the $elemhide filter option? While other options are specifying when a filter should be applied, $elemhide specifies what should be applied. On top of that it defines element hiding behavior within a blocking filter. Adding another element hiding-specific option to blocking filters would make it even more confusing than it already is.

Therefore I'm leaning towards either extending $elemhide in some way or introducing filter options to element hiding filters. While I could also imagine a completely new type of filter for domain-specific options (e.g. $document, $elemhide) rather than resource-specific ones (e.g. $image, $collapse), it appears out of scope for this issue.

My first and second suggestion can also be ported to blocking rules (e.g. ~document, issue #647).

While this might be the way of least resistance in regard to implementation, I can imagine this to make it even more difficult to comprehend what the keyword elemhide actually means (e.g. @@||example.com$elemhide disables all element hiding filters; @@||example.com$~elemhide disables only generic element hiding filters). The same issue applies to $~document.

[famlam]

And what about completely replacing the current $elemhide behavior? As explained in my previous post, I don't think there is much reason to have the present behavior over the suggested behavior. (And it would hardly, if any at all, cause compatibility issues with currently existing filters). Furthermore, you would prevent that all filter lists have to update their filters to the new variant, as well as keeping an old rule for older ABP versions (until most people have updated).

But if you really want to extend $elemhide with options ($elemhide=X), I wouldn't choose the domain as it is rare that $elemhide occurs in a filter not containing a domain already (e.g. @@/ads/*$elemhide would always be way too generic, compared to @@||DOMAIN/ads/$elemhide or @@/ads/*$elemhide,domain=DOMAIN, and @@||site.com^$elemhide=~anyOtherSite.com would be meaningless). That way, you would only add duplicate information to a filter. Better (IMO) might in that case be something like $elemhide=generic (or some other fixed-value-option). I guess code-technically it would also be beneficial for speed, as you have to match the full filter anyway, but have no additional comparisons for the domain of $elemhide.

[trev]

Replying to greiner:

Isn't that the general problem with the $elemhide filter option? While other options are specifying when a filter should be applied, $elemhide specifies what should be applied.

We actually have both kinds of options. We have those making the filter more specific (type options, $domain and similar) and those activating non-default behavior ($document, $collapse).

On top of that it defines element hiding behavior within a blocking filter.

Since it applies to URLs rather than selectors, it actually makes sense.

Therefore I'm leaning towards either extending $elemhide in some way

That would overload the meaning here and make things extremely confusing.

or introducing filter options to element hiding filters.

That would make no sense whatsoever - as I said already, this isn't about selectors.

Actually, both this and #647 could be implemented by means of a new $generic option: @@||example.com^$generic (disable generic filters of any kind on example.com domain).

Replying to famlam:

And what about completely replacing the current $elemhide behavior? As explained in my previous post, I don't think there is much reason to have the present behavior over the suggested behavior. (And it would hardly, if any at all, cause compatibility issues with currently existing filters).

This is indeed an option. It has the advantage that we wouldn't need to support three different whitelisting schemes for element hiding rules ($document, $elemhide and $generic), and conventional filter lists indeed rarely intend to override element hiding rules that have been targeted specifically at a particular domain. The only place where this would become problematic is the Acceptable Ads list, it would have to use specific exceptions for some rules (not sure how many) if we changed $elemhide behavior. My impression is that this would make maintaining that list harder.

[famlam]

Actually, both this and #647 could be implemented by means of a new $generic option: >@@||example.com^$generic (disable generic filters of any kind on example.com domain).

The problem I see with $generic is that it would also more-or-less kill EasyPrivacy and other non-ads-lists on websites where basically only generic element hiding rules have to be disabled.
Many cases where we have to add $elemhide rules due to Anti-Adblock measurements, we only need a few blocking whitelists, and often enough those blocking whitelists are for resources of which the server is blocked (e.g., rules starting with ||, which according to the discussion-in-which-I-do-not-participate in #647 would be non-generic). It's very rare to find Anti-Anti-Tracking or Anti-Anti-Social measurements on websites though, so there is no need to whitelist them altogether.

The only place where this would become problematic is the Acceptable Ads list, it would have to use specific exceptions for some rules (not sure how many) if we changed $elemhide behavior.

Just to sketch the impact: according to my redundancy checker there are ~125 specific hiding rules (in EL+ELD) being made redundant by $elemhide rules in the Acceptable Ads list, which means ~125 #@# rules have to be added to the AA list (+ ones that are triggered by specific rules in other regional lists) if this method is chosen. (Furthermore, off topic, it would allow using custom hiding rules on those AA domains, which is currently impossible).

[trev]

Replying to famlam:

Just to sketch the impact: according to my redundancy checker there are ~125 specific hiding rules (in EL+ELD) being made redundant by $elemhide rules in the Acceptable Ads list, which means ~125 #@# rules have to be added to the AA list (+ ones that are triggered by specific rules in other regional lists) if this method is chosen.

The main problem here isn't the one-time effort of adding these rules, it's keeping them in sync with EasyList & Co. that's problematic.

[Lain_13]

Hi, I'm not sure it have to be a part of this issue, so feel free to create a new one but I think elemhide have to disable collecting hit statistics in FireFox for specific site since it reveals presence of active AdBlock through property -moz-binding (example here: iphones.ru search for ".getPropertyValue('-moz-binding').indexOf('about:')" in the page code). Maybe it have to be a separate option but it will be a fix specific only for Firefox then.

[greiner]

Replying to Lain_13:

Hi, I'm not sure it have to be a part of this issue, so feel free to create a new one but I think elemhide have to disable collecting hit statistics in FireFox for specific site since it reveals presence of active AdBlock through property -moz-binding (example here: iphones.ru search for ".getPropertyValue('-moz-binding').indexOf('about:')" in the page code). Maybe it have to be a separate option but it will be a fix specific only for Firefox then.

Sorry but I couldn't find that in the page code. Please create a new issue if you can provide more details about it. Generally, example code and pages would be quite helpful.

[kzar]

I'm not clear how this ticket differs from #647? (Maybe one of them should be closed?)

Also there are lots of different approaches suggested in the comments of this ticket, could you check the description is accurate and maybe update it @palant / @greiner? Just so it's clear what exactly we're doing before I start. So far there are a couple of quotes in the description, describing things that would be nice but nothing concrete that says exactly what we're going to do.

[arthur]

@kzar
#647 is for blocking rules, this one is for hiding rules.

[greiner]

I reflected the common consens in the issue description.

[kzar]

Quick update: I've implemented $generichide + $genericblock in Firefox + Chrome and added some tests. All changes are now just awaiting review.

[kzar]

​https://hg.adblockplus.org/adblockplus/rev/893426c6a6ab

[kzar]

​https://hg.adblockplus.org/testpages.adblockplus.org/rev/2a068b22299a

[kzar]

​https://hg.adblockplus.org/adblockplustests/rev/449058480108

[kzar]

​https://hg.adblockplus.org/adblockpluschrome/rev/68f1d78e2832