Opened 4 years ago
Closed 4 years ago
#3168 closed change (fixed)
Add a script for generating new content blocker lists
| Reported by: | fhd | Assignee: | fhd |
|---|---|---|---|
| Priority: | P3 | Milestone: | |
| Module: | Sitescripts | Keywords: | |
| Cc: | sebastian, kzar | Blocked By: | |
| Blocking: | #3176 | Platform: | Unknown / Cross platform |
| Ready: | no | Confidential: | no |
| Tester: | Unknown | Verified working: | no |
| Review URL(s): |
https://codereview.adblockplus.org/29328894/ |
||
Description (last modified by fhd)
Background
We currently have to create the content blocker lists stored in contentblockerlists by manually downloading/combining the lists, and then running abp2blocklist. We should automate this as much as possible, since it needs to be done frequently, and should eventually happen automatically.
What to change
Add a new script sitescripts.content_blocker_lists.bin.generate_lists, that will:
- Download easylist_noadult.txt and exceptionrules.txt.
- Fetch abp2blocklist and invoke it to generate easylist_content_blocker.json from easylist_noadult.txt and easylist+exceptionrules_content_blocker.json from the combination of easylist_noadult.txt and exceptionrules.txt.
Change History (12)
comment:1 Changed 4 years ago by fhd
- Summary changed from [contentblockerlists] Add a script for generating new content blocking lists to [contentblockerlists] Add a script for generating new content blocker lists
comment:2 Changed 4 years ago by fhd
- Owner set to fhd
comment:3 Changed 4 years ago by fhd
- Priority changed from Unknown to P3
- Review URL(s) modified (diff)
- Status changed from new to reviewing
comment:4 Changed 4 years ago by fhd
- Cc sebastian added
- Review URL(s) modified (diff)
comment:5 Changed 4 years ago by fhd
- Review URL(s) modified (diff)
Removed the patch that used ensure_dependencies.py.
comment:6 Changed 4 years ago by fhd
- Cc dave added
comment:7 Changed 4 years ago by fhd
- Blocking 3176 added
comment:8 Changed 4 years ago by fhd
- Description modified (diff)
- Ready unset
- Summary changed from [contentblockerlists] Add a script for generating new content blocker lists to Add a script for generating new content blocker lists
Updated the description based on our discussion, what do you think? Suggestions on where in sitescripts the script should be and how it should be called welcome. I was wondering if it should go into the subscriptions module, not sure.
comment:9 Changed 4 years ago by kzar
- Cc kzar added; dave removed
comment:10 Changed 4 years ago by fhd
- Description modified (diff)
comment:11 Changed 4 years ago by fhd
- Review URL(s) modified (diff)
comment:12 Changed 4 years ago by fhd
- Resolution set to fixed
- Status changed from reviewing to closed
Added another patch that avoids using ensure_dependencies.py as discussed with Sebastian.
No strong opinion on what's better, mildly leaning towards the new patch now.