Opened 4 years ago

Closed 4 years ago

Last modified 3 years ago

#3093 closed change (fixed)

Automatically update content blocking lists

Reported by: fhd Assignee: jand
Priority: P2 Milestone: Adblock-Plus-for-iOS-1.1.0
Module: Adblock-Plus-for-iOS/macOS Keywords: salsita
Cc: sven, lisabielik, sebastian, mario Blocked By: #3088
Blocking: Platform: iOS
Ready: no Confidential: no
Tester: Ross Verified working: yes
Review URL(s):

https://github.com/adblockplus/adblockplussafariios/pull/3
https://github.com/adblockplus/adblockplussafariios/pull/6
https://codereview.adblockplus.org/29329333/

Description (last modified by fhd)

Background

We are currently bundling the Safari content blocking lists we use, and need to release a new version of the app to update it. Eventually, we want to generate content blocking lists from arbitrary ABP filter lists on the fly, but for now, we want to download the content blocking lists directly. #3088 is about offering them for download on our infrastructure.

What to change

We are bundling two different content blocking lists right now:

easylist.json - Generated from EasyList.
easylist_with_acceptable_ads.json - A combination of EasyList and the Acceptable Ads list.

We should continue to bundle them, but they should be renamed:

easylist.json -> easylist_content_blocker.json
easylist_with_acceptable_ads.json -> easylist+exceptionrules_content_blocker.json

In addition, an Update Filterlists item should be added, as in the mock up, but the text should actually be Update filter lists.

Triggering this item should download the latest version of each list from https://easylist-downloads.adblockplus.org/easylist_content_blocker.json and https://easylist-downloads.adblockplus.org/easylist+exceptionrules_content_blocker.json respectively, and use those instead of the ones that were bundled with the app.

The same download should also be triggered on a continuous basis, every five days. It should be triggered for the first time when the app first starts.

If the download fails, it should not be retried - we should wait another full five days for the next try. If the download was triggered by the user, a standard error dialog with the title Filter list update failed and the message Failed to update filter lists, please try again later..

We should strip JavaScript comments the JSON file, if there are any. There is a chance we will have to add some later, and we should be able to deal with that for backwards compatibility.

Attachments (2)

Adblock Plus iOS 11 settings.png (133.1 KB) - added by fhd 4 years ago.
Ticket_3093.png (20.9 KB) - added by lisabielik 4 years ago.

Download all attachments as: .zip

Change History (27)

Changed 4 years ago by fhd

comment:1 Changed 4 years ago by fhd

  • Description modified (diff)

comment:2 Changed 4 years ago by fhd

  • Review URL(s) modified (diff)
  • Status changed from new to reviewing

comment:3 Changed 4 years ago by fhd

  • Description modified (diff)
  • Summary changed from Make it possible to refresh content blocking lists to Automatically update content blocking lists

comment:4 Changed 4 years ago by fhd

  • Description modified (diff)

comment:5 Changed 4 years ago by philll

  • Platform changed from Unknown / Cross platform to iOS

Added newly available platform value.

comment:6 Changed 4 years ago by fhd

  • Description modified (diff)

comment:7 Changed 4 years ago by matze

  • Blocked By 3088 added

comment:8 Changed 4 years ago by matze

  • Description modified (diff)

comment:9 Changed 4 years ago by lisabielik

Please see Ticket_3093.png for text changes.

Last edited 4 years ago by lisabielik (previous) (diff)

Changed 4 years ago by lisabielik

comment:10 Changed 4 years ago by philll

  • Cc lisabielik added

Please put texts for changes in the ticket description to not have them hidden in attachments and error-prone image files due to manual copy-typing texts.

comment:11 Changed 4 years ago by mario

  • Keywords salsita added

comment:12 follow-up: Changed 4 years ago by fhd

  • Description modified (diff)

Added an error dialog we have in the code, but not in the issue so far. Lisa, please look over the texts.

comment:13 follow-up: Changed 4 years ago by sebastian

  • Cc sebastian added
  • Ready unset

Removing ready as we don't agree how to transfer the metadata. See 3176#comment:11.

comment:14 Changed 4 years ago by fhd

  • Description modified (diff)

comment:15 in reply to: ↑ 13 Changed 4 years ago by fhd

  • Cc mario added

Replying to sebastian:

Removing ready as we don't agree how to transfer the metadata. See 3176#comment:11.

Yes, true. But then again, we can worry about backwards compatibility later. As long as we don't make a release before we figured it out, we'll be fine. So I've removed the part about stripping out JSON comments, for now we can assume on the client-side that we'll get valid content blocker lists from the backend. We can deal with meta data in a follow-up issue.

comment:16 in reply to: ↑ 12 Changed 4 years ago by lisabielik

Replying to fhd:

Added an error dialog we have in the code, but not in the issue so far. Lisa, please look over the texts.

Please change:

Set updating flag to false of filterlist, which was cancelled by user (user killed application).

TO

Set updating flag to false of filter list, which was cancelled by user (user killed application).

comment:17 follow-up: Changed 4 years ago by fhd

I can only see this text as a comment in the code, it's not user visible, so it doesn't matter too much.

The visible text that has been added is:

Title: Filter list update failed
Message: Failed to update filter lists, please try again later.

comment:18 in reply to: ↑ 17 Changed 4 years ago by lisabielik

Replying to fhd:

I can only see this text as a comment in the code, it's not user visible, so it doesn't matter too much.

The visible text that has been added is:

Title: Filter list update failed
Message: Failed to update filter lists, please try again later.

Please change Message to:

Failed to update filter lists. Please try again later.

comment:19 Changed 4 years ago by fhd

  • Review URL(s) modified (diff)

A new pull request had to be opened, because GitHub. The comments in PR 3 are still relevant however.

Last edited 4 years ago by fhd (previous) (diff)

comment:20 Changed 4 years ago by fhd

PR 6 has been pushed: https://hg.adblockplus.org/adblockplussafariios/rev/c41afeb72aa0

The text change Lisa asked for still needs to happen, but I'll quickly do that in a new PR, that's easier.

comment:21 follow-up: Changed 4 years ago by fhd

  • Review URL(s) modified (diff)

OK, here's a review for the small text changes: https://codereview.adblockplus.org/29329333/

comment:22 in reply to: ↑ 21 Changed 4 years ago by lisabielik

Replying to fhd:

OK, here's a review for the small text changes: https://codereview.adblockplus.org/29329333/

LGTM.

comment:23 Changed 4 years ago by fhd

  • Milestone set to Adblock-Plus-for-iOS-next
  • Resolution set to fixed
  • Status changed from reviewing to closed

comment:24 Changed 4 years ago by fhd

  • Owner set to jand

Assigning to Jan in retrospect - he did ~99.9% of the work here :)

comment:25 Changed 3 years ago by Ross

  • Tester changed from Unknown to Ross
  • Verified working set

The content blocker now requests the lists and uses loads them correctly (both original format and new format with meta data).

iPhone 5S / iOS 9.2
ABP for iOS / 1.0.1 / 201601221200

Note: See TracTickets for help on using tickets.