Opened 2 years ago

Last modified 19 months ago

#6050 closed defect

[webextension] Frequent large disk writes since 3.0 — at Version 8

Reported by: krestin Assignee:
Priority: P1 Milestone:
Module: Platform Keywords:
Cc: sebastian, kzar, mapx, Ross, jeen, athornburgh, tschuster Blocked By:
Blocking: Platform: Firefox
Ready: no Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://codereview.adblockplus.org/29621569/

Description (last modified by trev)

Environment

Ubuntu 16.04 64-bit
Firefox 57.0 64-bit
Adblock Plus 3.0.1

How to reproduce

  1. Enable Adblock Plus
  2. Run iotop, sort by 'Disk Write' column

Observed behaviour

Command 'firefox [DOM Worker]' writes several MBs every minute. After disabling Adblock Plus the writes are no longer seen. After re-enabling Adblock Plus the writes are seen again.

Expected behaviour

Adblock Plus does not cause excessive disk writes. I don't remember seeing this with versions < 3.0.

Background

See #5769

Change History (10)

comment:1 Changed 2 years ago by mapx

  • Cc trev sebastian kzar mapx added

comment:2 Changed 2 years ago by kzar

  • Cc Ross added
  • Component changed from Unknown to Platform
  • Summary changed from Frequent large disk writes since 3.0 to [webextensions] Frequent large disk writes since 3.0

comment:3 Changed 2 years ago by kzar

  • Summary changed from [webextensions] Frequent large disk writes since 3.0 to [webextension] Frequent large disk writes since 3.0

comment:4 Changed 2 years ago by kzar

Thanks for the great bug report.

Which filter subscriptions and custom filters do you have enabled? Do you still observe the excessive writes when all filters and subscriptions are disabled? If not are you able to narrow down the problem to a specific filter or subset of filters?

comment:5 Changed 2 years ago by krestin

I disabled all subscriptions and the 'firefox [DOM Worker]' writes virtually stopped. Then I enabled 'Adblock Warning Removal List' and 'EasyList' and the writes were still rare and small. Then I enabled 'EasyPrivacy' and the writes became more regular and larger. About half were 20-30 KB and the other half 3-4 MB. I enabled the rest of the subscriptions (Facebook Privacy List and Fanboy lists) and the frequency and size stayed about the same.

For whatever reason it's currently not as regular as what I was seeing before, which was 3-4 MB writes exactly every minute. Now it's more like every 30-120 seconds and some of the writes are only about 20-30 KB.

Edit: I'm seeing this behaviour with Gmail open and an otherwise idle browser. I imagine that background activity in Gmail is triggering the regular writes.

Last edited 2 years ago by krestin (previous) (diff)

comment:6 Changed 2 years ago by mapx

It's definitely something wrong:

  • I'm using windows 10, I opened resource monitor.
  • in my FF 57 (ABP beta build, easylist, easyprivacy, etc) keeping google page
  • when I refresh the page => the storage.js file is rewritten entirely or at least partially - the timestamp changes and in resource monitor I can see a lot of data is written in \browser-extension-data\{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} folder (where storage.js is placed)
  • see the attached file
  • I repeated the test using uBo => storage.js remains with the same timestamp even refreshing the page (even if uBo use storage.js only for some prefs and the filters are written using indexedDB)
  • does ABP write some weird statistic bytes ? If it's about writting the total number of ads is just bad updating such counter for every visited page. It should be something like every 5 minutes (or better: at the end of session) just to update a similar (useless) counter.
Last edited 2 years ago by mapx (previous) (diff)

Changed 2 years ago by mapx

comment:7 Changed 2 years ago by Ross

I've reproduced this too on Ubuntu and Windows (Firefox 57/ABP 3.0.1). Attached screenshot shows updating lists and then general browsing on Windows.

Changed 2 years ago by Ross

comment:8 Changed 2 years ago by trev

  • Description modified (diff)

For reference, what you see is the current state after #5769. The root issue here is that storage implementation in Firefox is very suboptimal, large writes being triggered by a change of a single small value. We could consider IndexedDB again of course, but it might cause more harm than good.

Note: See TracTickets for help on using tickets.