Opened 21 months ago

Last modified 16 months ago

#6050 closed defect

[webextension] Frequent large disk writes since 3.0 — at Version 8

Reported by: krestin Assignee:
Priority: P1 Milestone:
Module: Platform Keywords:
Cc: sebastian, kzar, mapx, Ross, jeen, athornburgh, tschuster Blocked By:
Blocking: Platform: Firefox
Ready: no Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://codereview.adblockplus.org/29621569/

Description (last modified by trev)

Environment

Ubuntu 16.04 64-bit
Firefox 57.0 64-bit
Adblock Plus 3.0.1

How to reproduce

  1. Enable Adblock Plus
  2. Run iotop, sort by 'Disk Write' column

Observed behaviour

Command 'firefox [DOM Worker]' writes several MBs every minute. After disabling Adblock Plus the writes are no longer seen. After re-enabling Adblock Plus the writes are seen again.

Expected behaviour

Adblock Plus does not cause excessive disk writes. I don't remember seeing this with versions < 3.0.

Background

See #5769

Change History (10)

comment:1 Changed 21 months ago by mapx

  • Cc trev sebastian kzar mapx added

comment:2 Changed 21 months ago by kzar

  • Cc Ross added
  • Component changed from Unknown to Platform
  • Summary changed from Frequent large disk writes since 3.0 to [webextensions] Frequent large disk writes since 3.0

comment:3 Changed 21 months ago by kzar

  • Summary changed from [webextensions] Frequent large disk writes since 3.0 to [webextension] Frequent large disk writes since 3.0

comment:4 Changed 21 months ago by kzar

Thanks for the great bug report.

Which filter subscriptions and custom filters do you have enabled? Do you still observe the excessive writes when all filters and subscriptions are disabled? If not are you able to narrow down the problem to a specific filter or subset of filters?

comment:5 Changed 21 months ago by krestin

I disabled all subscriptions and the 'firefox [DOM Worker]' writes virtually stopped. Then I enabled 'Adblock Warning Removal List' and 'EasyList' and the writes were still rare and small. Then I enabled 'EasyPrivacy' and the writes became more regular and larger. About half were 20-30 KB and the other half 3-4 MB. I enabled the rest of the subscriptions (Facebook Privacy List and Fanboy lists) and the frequency and size stayed about the same.

For whatever reason it's currently not as regular as what I was seeing before, which was 3-4 MB writes exactly every minute. Now it's more like every 30-120 seconds and some of the writes are only about 20-30 KB.

Edit: I'm seeing this behaviour with Gmail open and an otherwise idle browser. I imagine that background activity in Gmail is triggering the regular writes.

Last edited 21 months ago by krestin (previous) (diff)

comment:6 Changed 21 months ago by mapx

It's definitely something wrong:

  • I'm using windows 10, I opened resource monitor.
  • in my FF 57 (ABP beta build, easylist, easyprivacy, etc) keeping google page
  • when I refresh the page => the storage.js file is rewritten entirely or at least partially - the timestamp changes and in resource monitor I can see a lot of data is written in \browser-extension-data\{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} folder (where storage.js is placed)
  • see the attached file
  • I repeated the test using uBo => storage.js remains with the same timestamp even refreshing the page (even if uBo use storage.js only for some prefs and the filters are written using indexedDB)
  • does ABP write some weird statistic bytes ? If it's about writting the total number of ads is just bad updating such counter for every visited page. It should be something like every 5 minutes (or better: at the end of session) just to update a similar (useless) counter.
Last edited 21 months ago by mapx (previous) (diff)

Changed 21 months ago by mapx

comment:7 Changed 21 months ago by Ross

I've reproduced this too on Ubuntu and Windows (Firefox 57/ABP 3.0.1). Attached screenshot shows updating lists and then general browsing on Windows.

Changed 21 months ago by Ross

comment:8 Changed 21 months ago by trev

  • Description modified (diff)

For reference, what you see is the current state after #5769. The root issue here is that storage implementation in Firefox is very suboptimal, large writes being triggered by a change of a single small value. We could consider IndexedDB again of course, but it might cause more harm than good.

Note: See TracTickets for help on using tickets.