Opened on 10/06/2015 at 02:37:18 PM

Closed on 11/20/2015 at 09:08:05 AM

#3168 closed change (fixed)

Add a script for generating new content blocker lists

Reported by: fhd Assignee: fhd
Priority: P3 Milestone:
Module: Sitescripts Keywords:
Cc: sebastian, kzar Blocked By:
Blocking: #3176 Platform: Unknown / Cross platform
Ready: no Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

Description (last modified by fhd)


We currently have to create the content blocker lists stored in contentblockerlists by manually downloading/combining the lists, and then running abp2blocklist. We should automate this as much as possible, since it needs to be done frequently, and should eventually happen automatically.

What to change

Add a new script sitescripts.content_blocker_lists.bin.generate_lists, that will:

  1. Download easylist_noadult.txt and exceptionrules.txt.
  2. Fetch abp2blocklist and invoke it to generate easylist_content_blocker.json from easylist_noadult.txt and easylist+exceptionrules_content_blocker.json from the combination of easylist_noadult.txt and exceptionrules.txt.

Attachments (0)

Change History (12)

comment:1 Changed on 10/06/2015 at 02:37:51 PM by fhd

  • Summary changed from [contentblockerlists] Add a script for generating new content blocking lists to [contentblockerlists] Add a script for generating new content blocker lists

comment:2 Changed on 10/06/2015 at 03:19:48 PM by fhd

  • Owner set to fhd

comment:3 Changed on 10/06/2015 at 05:44:06 PM by fhd

  • Priority changed from Unknown to P3
  • Review URL(s) modified (diff)
  • Status changed from new to reviewing

comment:4 Changed on 10/06/2015 at 06:25:39 PM by fhd

  • Cc sebastian added
  • Review URL(s) modified (diff)

Added another patch that avoids using as discussed with Sebastian.

No strong opinion on what's better, mildly leaning towards the new patch now.

comment:5 Changed on 10/13/2015 at 01:55:39 PM by fhd

  • Review URL(s) modified (diff)

Removed the patch that used

comment:6 Changed on 10/20/2015 at 04:07:04 PM by fhd

  • Cc dave added

comment:7 Changed on 10/22/2015 at 06:31:49 PM by fhd

  • Blocking 3176 added

comment:8 Changed on 10/22/2015 at 06:39:40 PM by fhd

  • Description modified (diff)
  • Ready unset
  • Summary changed from [contentblockerlists] Add a script for generating new content blocker lists to Add a script for generating new content blocker lists

Updated the description based on our discussion, what do you think? Suggestions on where in sitescripts the script should be and how it should be called welcome. I was wondering if it should go into the subscriptions module, not sure.

comment:9 Changed on 10/29/2015 at 04:03:00 PM by kzar

  • Cc kzar added; dave removed

comment:10 Changed on 10/30/2015 at 07:11:37 AM by fhd

  • Description modified (diff)

comment:11 Changed on 10/30/2015 at 07:17:59 AM by fhd

  • Review URL(s) modified (diff)

comment:12 Changed on 11/20/2015 at 09:08:05 AM by fhd

  • Resolution set to fixed
  • Status changed from reviewing to closed

Add Comment

Modify Ticket

Change Properties
as closed .
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from fhd.
Note: See TracTickets for help on using tickets.