Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions tests/visual-regression/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# Visual Regression Tests in WordPress Core

These tests make use of Playwright, with a setup very similar to that of the e2e tests.
These tests use Playwright to capture full-page screenshots of WordPress admin
screens and compare them against baseline snapshots.

## How to Run the Tests Locally

1. Check out trunk.
2. Run `npm run test:visual` to generate some base snapshots.
3. Check out the feature branch to be tested.
4. Run `npm run test:visual` again. If any tests fail, the diff images can be found in `artifacts/`

1. Start the local environment: `npm run env:start`
2. Generate baseline snapshots: `npm run test:visual -- --update-snapshots`
3. Make your changes.
4. Run `npm run test:visual` — any visual differences will fail and produce
diff images in `artifacts/`.
5. Open the HTML report for a side-by-side comparison (opens automatically
after a local run, or see `artifacts/visual-report/`).
71 changes: 71 additions & 0 deletions tests/visual-regression/config/screenshot.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
/*
* Global stylesheet applied during screenshot capture.
*
* Hides volatile elements that change between environments or runs,
* preventing false positives in visual regression comparisons.
* Applied via Playwright's stylePath config option.
*
* See: https://playwright.dev/docs/test-snapshots#stylepath
*/

/*
* Uses `visibility: hidden` instead of `display: none` to preserve
* each element's layout space. Collapsing elements with `display: none`
* would shift surrounding content and cause false positives elsewhere.
*/

/* WordPress version/update nag in the admin footer. */
#footer-upgrade {
visibility: hidden !important;
}

/* Admin bar user-specific content (Howdy, gravatar). */
#wp-admin-bar-root-default {
visibility: hidden !important;
}

/* Gutenberg plugin menu item — not present in all environments. */
#toplevel_page_gutenberg {
visibility: hidden !important;
}

/* Gravatar images — external service, different per environment. */
.avatar {
visibility: hidden !important;
}

/* Date columns in list tables — relative timestamps shift between runs. */
.column-date {
visibility: hidden !important;
}

/* Dashboard widgets with dynamic counts and activity. */
#dashboard_right_now .inside,
#dashboard_activity .inside {
visibility: hidden !important;
}

/* Update-related timestamps. */
.update-last-checked {
visibility: hidden !important;
}

/* Admin notices — various nags (PHP deprecation, updates, etc.). */
.notice,
.update-nag,
.updated,
.error:not(#error),
#message {
visibility: hidden !important;
}

/* General Settings — live date/time preview changes on every run. */
#local-time,
.example {
visibility: hidden !important;
}

/* Users list table — post counts vary based on test data. */
.column-posts {
visibility: hidden !important;
}
61 changes: 60 additions & 1 deletion tests/visual-regression/playwright.config.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@
/**
* Playwright config for visual regression tests.
*
* Captures full-page screenshots of WordPress admin screens and compares
* them against baseline snapshots. Intended for local use to catch
* unintended visual changes during development.
*
* Usage:
* npm run test:visual -- --update-snapshots # generate baselines
* npm run test:visual # compare against baselines
*
* @see tests/visual-regression/config/screenshot.css for globally hidden elements.
* @see tests/visual-regression/specs/visual-snapshots.test.js for the test spec.
*/

/**
* External dependencies
*/
Expand All @@ -15,9 +30,53 @@ process.env.STORAGE_STATE_PATH ??= path.join(
'storage-states/admin.json'
);

// Reporters:
// - 'list' — prints pass/fail per test in the terminal.
// - 'github' — adds inline PR annotations when running in CI.
// - 'html' — generates a visual report with side-by-side diff images;
// opens automatically after local runs.
const reporter = [
[ 'list' ],
...( process.env.CI ? [ [ 'github' ] ] : [] ),
[
'html',
{
open: process.env.CI ? 'never' : 'always',
outputFolder: path.join(
process.env.WP_ARTIFACTS_PATH,
'visual-report'
),
},
],
];

const config = defineConfig( {
...baseConfig,
globalSetup: undefined,
fullyParallel: true,
// No retries — visual diffs are expected when regressions exist;
// retrying would just re-confirm the same diff.
retries: 0,
// Serialize tests in CI to reduce flakiness from resource contention.
workers: process.env.CI ? 1 : undefined,
reporter,
use: {
...baseConfig.use,
viewport: { width: 1280, height: 720 },
},
expect: {
toHaveScreenshot: {
// Only disables CSS animations/transitions. JavaScript-driven
// animations (e.g. jQuery .animate()) can still cause flakes.
animations: 'disabled',
// Captures the entire scrollable page, not just the viewport.
// The viewport width (1280) still matters — it controls layout.
fullPage: true,
// 1% tolerance — catches real regressions while ignoring
// sub-pixel anti-aliasing differences across environments.
maxDiffPixelRatio: 0.01,
stylePath: path.join( __dirname, 'config', 'screenshot.css' ),
},
},
webServer: {
...baseConfig.webServer,
command: 'npm run env:start',
Expand Down
Loading
Loading