Opened 5 years ago

Closed 22 months ago

#101 closed change (rejected)

Look into Firefox unit test failures

Reported by: trev Assignee: tschuster
Priority: P4 Milestone:
Module: Adblock-Plus-for-Firefox Keywords: goodfirstbug
Cc: manvel@… Blocked By:
Blocking: Platform: Firefox
Ready: yes Confidential: no
Tester: Verified working: no
Review URL(s):

http://codereview.adblockplus.org/6280417539784704/
http://codereview.adblockplus.org/5390879414026240/

Description

Background

Our adblockplustests Firefox extension (https://adblockplus.org/en/source#unittests) currently consistently reports a bunch of failures. There are a few "allowed request not received by server" (most likely outdated tests no longer triggering a request) and a few pop-up blocking related failures.

What to change

Figure out what those failures are about and fix them.

Change History (17)

comment:1 Changed 5 years ago by philll

  • Owner set to saroyanm
  • Status changed from new to assigned

comment:2 Changed 5 years ago by fhd

  • Priority changed from Unknown to P4
  • Reporter changed from philll to trev

comment:3 Changed 5 years ago by trev

  • Keywords goodfirstbug added
  • Ready set

Some test failures have been resolved by #340 and #341. What's left is the UI tests failing (these are not ready for Australis - need to be adapted or removed), some content policies failures (it seems that Firefox no longer does some requests that the tests expect) and an intermittent failure in the filter storage test (hangs up when running all tests but works fine when running on its own).

comment:4 Changed 5 years ago by trev

  • Owner saroyanm deleted

comment:5 Changed 5 years ago by saroyanm

  • Cc manvel@… added

comment:6 Changed 5 years ago by philll

  • Status changed from assigned to new

The assigned state will be dropped by #403

comment:7 follow-up: Changed 5 years ago by tschuster

  • Platform set to Unknown

So I started looking at this yesterday. We seem to have quite a few test failures that happen intermittently, which makes them harder track down. I also have problems with running single tests.

The easiest problem are the "Icon position" tests, they are just outdated. These tests were written with the icon in the addon-bar, but now our icon is always in the nav-bar, next to the hamburger button.

"Content policy: Dynamically inserted image button" and "Content policy: Web worker" are just not totally correct tests. We need to wait for the load event of the image and worker, because the frame's onload event (where most of the test code executes) can trigger before them.

Notification handling: Target selection is just really slow.

Filter storage read/write: Saving with backups not sure if that test is just super really slow or broken.

comment:8 in reply to: ↑ 7 Changed 5 years ago by trev

Replying to tschuster:

Filter storage read/write: Saving with backups not sure if that test is just super really slow or broken.

I guess you were testing on Linux. There are strange intermittent issues with this test on Linux (error message is thrown), this test works correctly when run individually however. There are no issues on Windows and OS X however.

Note that you can use the Rerun link to run individual tests - or you can use ?test=filterStorage_readwrite to run a single test file only.

comment:9 Changed 5 years ago by philll

  • Platform changed from Unknown to Firefox

comment:11 Changed 5 years ago by trev

  • Owner set to tschuster
  • Review URL(s) modified (diff)

comment:12 Changed 5 years ago by trev

  • Status changed from new to reviewing

comment:13 Changed 5 years ago by tschuster

Pushed fix for icon tests: https://hg.adblockplus.org/adblockplustests/rev/1323a160ed7a

The tests should now usually pass when using a clean Firefox profile, but I am going to do a few test runs before calling this fixed.

comment:14 Changed 5 years ago by trev

Actually, there were some more failures last time I tried. Pop-up blocking tests show issues but that should be #443. Also, I've now seen "saving with backups" test fail on Mac OS X (exception in an async callback, test hangs), previously I've only seen that issue on Linux. This test actually executed correctly when I've rerun it.

comment:15 Changed 5 years ago by tschuster

You are right, "saving with backups" just failed, but passed in the run before.

A promise chain failed to handle a rejection. Did you forget to '.catch', or did you forget to 'return'?
See https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Promise.jsm/Promise

Date: Wed Aug 06 2014 18:40:20 GMT+0200 (CEST)
Full Message: Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIFile.lastModifiedTime]

I will look into that problem. Luckily all other tests seem to be fine. I haven't seen any problem with pop-up blocking before.

comment:16 Changed 5 years ago by tschuster

So from what I can tell "saving with backups" just randomly fails sometimes. However if you keep the tab with test page focused and do nothing else in the browser instance it seems to work most of the time. The failure seems to indicate that cloning the file didn't work for some reason.

comment:17 Changed 22 months ago by trev

  • Resolution set to rejected
  • Status changed from reviewing to closed

Mass-closing all bugs in Adblock Plus for Firefox module, the codebase of Adblock Plus 3.0 belongs into Platform and User-Interface modules. Old bugs are unlikely to still apply.

Note: See TracTickets for help on using tickets.