Opened on 10/06/2015 at 02:37:18 PM
Closed on 11/20/2015 at 09:08:05 AM
#3168 closed change (fixed)
Add a script for generating new content blocker lists
| Reported by: | fhd | Assignee: | fhd |
|---|---|---|---|
| Priority: | P3 | Milestone: | |
| Module: | Sitescripts | Keywords: | |
| Cc: | sebastian, kzar | Blocked By: | |
| Blocking: | #3176 | Platform: | Unknown / Cross platform |
| Ready: | no | Confidential: | no |
| Tester: | Unknown | Verified working: | no |
| Review URL(s): |
https://codereview.adblockplus.org/29328894/ |
||
Description (last modified by fhd)
Background
We currently have to create the content blocker lists stored in contentblockerlists by manually downloading/combining the lists, and then running abp2blocklist. We should automate this as much as possible, since it needs to be done frequently, and should eventually happen automatically.
What to change
Add a new script sitescripts.content_blocker_lists.bin.generate_lists, that will:
- Download easylist_noadult.txt and exceptionrules.txt.
- Fetch abp2blocklist and invoke it to generate easylist_content_blocker.json from easylist_noadult.txt and easylist+exceptionrules_content_blocker.json from the combination of easylist_noadult.txt and exceptionrules.txt.
Attachments (0)
Change History (12)
comment:1 Changed on 10/06/2015 at 02:37:51 PM by fhd
- Summary changed from [contentblockerlists] Add a script for generating new content blocking lists to [contentblockerlists] Add a script for generating new content blocker lists
comment:2 Changed on 10/06/2015 at 03:19:48 PM by fhd
- Owner set to fhd
comment:3 Changed on 10/06/2015 at 05:44:06 PM by fhd
- Priority changed from Unknown to P3
- Review URL(s) modified (diff)
- Status changed from new to reviewing
comment:4 Changed on 10/06/2015 at 06:25:39 PM by fhd
- Cc sebastian added
- Review URL(s) modified (diff)
comment:5 Changed on 10/13/2015 at 01:55:39 PM by fhd
- Review URL(s) modified (diff)
Removed the patch that used ensure_dependencies.py.
comment:6 Changed on 10/20/2015 at 04:07:04 PM by fhd
- Cc dave added
comment:7 Changed on 10/22/2015 at 06:31:49 PM by fhd
- Blocking 3176 added
comment:8 Changed on 10/22/2015 at 06:39:40 PM by fhd
- Description modified (diff)
- Ready unset
- Summary changed from [contentblockerlists] Add a script for generating new content blocker lists to Add a script for generating new content blocker lists
Updated the description based on our discussion, what do you think? Suggestions on where in sitescripts the script should be and how it should be called welcome. I was wondering if it should go into the subscriptions module, not sure.
comment:9 Changed on 10/29/2015 at 04:03:00 PM by kzar
- Cc kzar added; dave removed
comment:10 Changed on 10/30/2015 at 07:11:37 AM by fhd
- Description modified (diff)
comment:11 Changed on 10/30/2015 at 07:17:59 AM by fhd
- Review URL(s) modified (diff)
comment:12 Changed on 11/20/2015 at 09:08:05 AM by fhd
- Resolution set to fixed
- Status changed from reviewing to closed
Added another patch that avoids using ensure_dependencies.py as discussed with Sebastian.
No strong opinion on what's better, mildly leaning towards the new patch now.