Opened 4 years ago

Closed 4 years ago

Last modified 3 years ago

#3756 closed change (fixed)

Filterlist rendering in python-abp

Reported by: kvas Assignee: kvas
Priority: Unknown Milestone:
Module: Sitescripts Keywords:
Cc: Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: no Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

Description

Filterlist rendering library should combine and post-process filterlists the same way combineSubscriptions.py script from sitescripts does it. In particular the following features need to be supported:

  • Including other list from filesystem and via http(s) requests,
  • Rendering the timestamp,
  • Removing duplicate headers and metadata,
  • Inserting the checksum into the rendered list,
  • Producing .TPL version of the filterlist by converting the filters and metadata and removing/commenting unsupported content.

The rendering code should conserve memory and do all the processing in a streaming fashion as much as possible (the checksum feature makes it impossible to be completely stream-oriented, but we should avoid keeping the whole list in memory).

Change History (7)

comment:1 Changed 4 years ago by kvas

  • Blocking 3751 added

comment:2 Changed 4 years ago by kvas

  • Component changed from Unknown to Sitescripts

comment:3 Changed 4 years ago by kvas

  • Sensitive unset

comment:4 Changed 4 years ago by kzar

  • Blocking 3849 added

comment:5 Changed 4 years ago by kzar

  • Blocking 3751 removed
  • Priority changed from P3 to Unknown

comment:6 Changed 4 years ago by kvas

  • Blocking 3849 removed
  • Resolution set to fixed
  • Status changed from new to closed

It's not clear if the rendering functionality will be needed anywhere outside of the rendering script (#3849), so the moment we won't expose it as a public API of the library. The issue was folded into 3849 for simplicity.

comment:7 Changed 3 years ago by abpbot

A commit referencing this issue has landed:
Issue 3756 - Refactor ensure_*state functions

Note: See TracTickets for help on using tickets.