Opened 15 months ago

Closed 15 months ago

Last modified 14 months ago

#6918 closed change (fixed)

Automate the test pages

Reported by: sebastian Assignee: sebastian
Priority: P3 Milestone: Adblock-Plus-3.4-for-Chrome-Opera-Firefox
Module: Automation Keywords:
Cc: tlucas, Ross, fhd Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: yes Confidential: no
Tester: Ross Verified working: yes
Review URL(s):

https://codereview.adblockplus.org/29871628

Description

Background

Our test pages can currently only be tested manually which takes a lot of time. If we can automate those tests (and later have them even run by our CI), we have a higher chance to catch regressions early.

What to change

Add a new script to test/wrappers/ in adblockpluschrome, automating the following procedure using Selenium WebDriver (for Chrome and Firefox):

  1. Load testpages.adblockplus.org.
  2. Load any sub-pages linked there.
  3. On the sub-pages add the CSS class expected to the <body> element (this will render the page as it is expected to look with the given filters in effect).
  4. Take screenshots of the expected outcome.
  5. For each test case, add the respective filters to Adblock Plus, disable/remove any other filters, reload the page and compare it to the previously taken screenshot.

Out of scope

There are some test cases that cannot be covered by comparing screenshots, but require more complex interactions (e.g. the circumvention test cases and the popup tests). These are out of scope for now.

Change History (4)

comment:1 Changed 15 months ago by sebastian

  • Review URL(s) modified (diff)
  • Status changed from new to reviewing

comment:3 Changed 15 months ago by sebastian

  • Milestone set to Adblock-Plus-for-Chrome-Opera-Firefox-next
  • Resolution set to fixed
  • Status changed from reviewing to closed

comment:4 Changed 14 months ago by Ross

  • Tester changed from Unknown to Ross
  • Verified working set

Automated tests run successfully in Firefox as well as Chromium.

adblockpluschrome / master / 2175:02d136073262

Note: See TracTickets for help on using tickets.