[abp2blocklist] Improve abp2blocklist to support/patch rules which can't be properly translated from Easylist

[mario]

Background

ABP for Safari on OS X (#3687) as well as ABP for Safari on iOS (#3093) utilize Apple's new content blocking filter list format for content blocking. On OS X this feature is experimental whereas on iOS it's the only available filter list format.
These filter lists are generated by ​abp2blocklist.

On both platforms, using those content blocking filter lists, we have multiple instances of erroneous filters which result in false positives, blocked acceptable ads or visible ads whereas those errors can't be reproduced using the original source filter lists (tested only via Easylist).

A list of known problems due to the new filter list format can be found here, containing individual ways of reproduction. Please note that this list is not complete, as only Acceptable Ads have been tested thus far.

Want to change

To be defined. Possible options would for example be to improve list generation or to introduce a patch file which is applied to any generated list containing individual fixes.

[sebastian]

I think the first thing we should do here is generating a new blocking list (using the latest version of our filter lists and abp2blocklist) and re-run the Acceptable Ads tests. I'm positive that this will already fix a fair amount of issues, probably not all though. But that is something we can look into then.

FWIW, applying changes to the generated list is only a last resort. But I agree, if there is no way around, we need a more structured way than how we do it right now, and one that also considers Adblock Plus for Safari on OS X.

[kzar]

Just to add to what Sebastian said, we made an improvement to abp2blocklist today in issue #4326 which should improve things quite a bit. I know that you guys had a test this / last week, but it might be worth regenerating the rules with the latest abp2blocklist and testing again.

[mario]

@sebastian, @kzar, this issue was opened after we gave content blocking a try using a completely new content blocking list generated via abp2blocklist. The new list resolved some of the existing issues, but also introduced a lot more problems. The results can be found ​here* (column BB).
However we'll give it another try now that #4326 is closed. I will report back with the results.

*This document is not public - I'll attach a file containing the results later.

[mjethani]

While I would love to work on this, in terms of priorities it looks like I just won't be able to give any time to this as it is not (directly) related to circumvention.

Removing myself as the owner so someone else can look into it.

[dzhang]

I'll investigate what can be done about this.

[dzhang]

Individual improvements will be tracked at ​https://gitlab.com/eyeo/sandbox/issue/boards?=&label_name[]=abp2blocklist.

[kzar]

Why track changes to the abp2blocklist repository under a GitLab board for the sandbox repository, especially when the changes already have issues in the issue tracker currently used by the ​Platform module which abp2blocklist comes under?

As it happens we already have ​a Hub issue open for moving some of the other Platform (and Automation) repositories over to GitLab, would you like me to request abp2blocklist is included in that migration?

[sebastian]

With Adblock Plus for Safari (legacy extension) on the way out, and Adblock Plus for iOS/macOS being the only consumer left of abp2blocklist, I'd like to transfer the repository to the "Adblock Plus for iOS/macOS" module (leaving it up to the new module owner whether to move it to GitLab).

[sebastian]

abp2blocklist has been ​moved from the Platform module to Adblock Plus for iOS/macOS.