Opened on 11/13/2014 at 08:12:39 PM

Closed on 11/19/2014 at 02:22:47 PM

#1569 closed defect (fixed)

Issue report digest update is inefficient

Reported by: trev Assignee: trev
Priority: P1 Milestone:
Module: Infrastructure Keywords:
Cc: Blocked By:
Blocking: Platform: Unknown
Ready: yes Confidential: no
Tester: Verified working: no
Review URL(s):

http://codereview.adblockplus.org/5694484511719424

Description

Background

When updating issue report digests, we currently need to process 160k issue reports. In order to save memory we process 1000 issue reports at a time. As it turns out, this produced significant stress in the database which had to process 160 rather large queries - the script spends most of the time waiting for the database to retrieve data. Even worse, the memory use of the script is very high regardless as Python garbage collection doesn't seem to be too efficient.

What to change

Process issue reports in steps of 10000, this will allow the script to finish faster and consequently release memory faster.

Attachments (0)

Change History (2)

comment:1 Changed on 11/13/2014 at 08:14:46 PM by trev

  • Review URL(s) modified (diff)
  • Status changed from new to reviewing

comment:2 Changed on 11/19/2014 at 02:22:47 PM by trev

  • Resolution set to fixed
  • Status changed from reviewing to closed

Add Comment

Modify Ticket

Change Properties
Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from trev.
 
Note: See TracTickets for help on using tickets.