Visual and Accessibility Checks in Playwright

Visual regression tests catch UI changes you didn't intend. Accessibility tests catch barriers you didn't notice. Learn when visual testing helps, when it's noise, and how to integrate axe-core for.

#playwright #e2e #testing #qa-automation #accessibility #a11y

This post is part of the Playwright Essentials series. If you missed the previous post, read Part 6 — Auth, Storage State, and Multi-User Flows first.

Introduction — Automation Can’t Replace Human Judgment, But It Can Flag What Humans Miss

Visual regression testing and accessibility testing sit at an uncomfortable intersection: they’re automatable, but they require subjective judgment. A pixel difference might be a bug or an intentional design change. An accessibility violation flagged by a tool might be a real barrier or a false positive depending on context.

The promise of visual testing is that you’ll catch unintended layout changes before they reach production. The reality is that most teams abandon visual testing within six months because the noise-to-signal ratio is too high. Every time a button moves two pixels, the test fails. Every time a font loads slightly differently in CI, the test fails. Eventually the team stops reviewing visual diffs and just approves everything.

Accessibility testing has the opposite problem: automated tools catch maybe 30–40% of real accessibility issues. They’ll tell you if an image is missing alt text or if a form input lacks a label, but they won’t tell you if your focus order is nonsensical or if your colour contrast fails for users with deuteranopia specifically.

So should you automate these checks? Yes, but with discipline.

This post will show you how to use Playwright’s toHaveScreenshot() for visual regression, when visual tests are worth the maintenance cost, and how to integrate axe-core for automated accessibility checks that actually catch real issues.

Visual Regression Testing with toHaveScreenshot()

Playwright has built-in visual comparison via the toHaveScreenshot() assertion. It works like this:

  1. The first time the test runs, Playwright takes a screenshot and saves it as the baseline.
  2. On subsequent runs, Playwright takes a new screenshot and compares it to the baseline pixel-by-pixel.
  3. If the images differ beyond a configured threshold, the test fails.

Basic Usage

import { test, expect } from '@playwright/test';

test('homepage renders correctly', async ({ page }) => {
  await page.goto('/');
  await expect(page).toHaveScreenshot('homepage.png');
});

The first run generates homepage.png in the tests/__screenshots__/ directory. Subsequent runs compare against it.

If the page changes, the test fails and Playwright generates:

  • homepage-actual.png — what the page looks like now
  • homepage-diff.png — a visual diff highlighting the changes

You review the diff, decide if the change is intentional, and if so, update the baseline:

npx playwright test --update-snapshots

Scoped Screenshots

You can screenshot specific elements instead of the full page:

test('navigation bar renders correctly', async ({ page }) => {
  await page.goto('/');
  const nav = page.locator('nav');
  await expect(nav).toHaveScreenshot('nav.png');
});

This is more stable than full-page screenshots because unrelated changes elsewhere on the page won’t trigger a failure.

Tolerating Minor Differences

Fonts, antialiasing, and browser rendering can vary slightly between environments. You can configure a tolerance threshold:

await expect(page).toHaveScreenshot('homepage.png', {
  maxDiffPixels: 100, // Allow up to 100 pixels to differ
});

Or as a percentage:

await expect(page).toHaveScreenshot('homepage.png', {
  maxDiffPixelRatio: 0.01, // Allow 1% pixel difference
});

:::warning[Don’t Set Tolerance Too High] If you set maxDiffPixelRatio to 5%, you’re allowing 5% of the screen to change without the test failing. That’s enough to miss a broken layout. Start conservative (0.1–0.5%) and adjust based on real flakiness. :::

When Visual Testing Helps

Visual regression tests are high maintenance. They break often, require manual review, and produce false positives. They’re only worth the cost when:

1. You Have a Design System with Reusable Components

If you maintain a component library (buttons, cards, modals, forms) used across dozens of pages, visual regression on those components catches unintended styling changes.

test('primary button matches design', async ({ page }) => {
  await page.goto('/design-system/buttons');
  const button = page.locator('[data-testid="primary-button"]');
  await expect(button).toHaveScreenshot('primary-button.png');
});

If someone accidentally changes the button’s border-radius or padding, this test catches it before it ships.

2. You Have Complex, Dynamic Layouts

Tables with dynamic columns, dashboards with draggable widgets, charts rendered from live data — these are hard to test with functional assertions. A visual test confirms the layout doesn’t collapse.

test('dashboard with 5 widgets renders without overlap', async ({ page }) => {
  await page.goto('/dashboard');
  await page.waitForSelector('[data-testid="widget"]');
  await expect(page).toHaveScreenshot('dashboard-5-widgets.png', {
    maxDiffPixelRatio: 0.02, // Charts may vary slightly
  });
});

3. You Have a Stable Visual Identity

If your brand guidelines are strict and changes to typography, spacing, or colour are reviewed carefully, visual tests enforce that consistency.

When Visual Testing Hurts

Visual tests are not appropriate when:

1. The Page Has Dynamic Content

If your homepage shows a “Latest News” section that changes daily, a visual test will fail daily. You’d need to either:

  • Mock the API to return static data (high effort)
  • Exclude the dynamic region from the screenshot (defeats the purpose)
  • Accept the noise (unsustainable)

2. You Can Assert the Behaviour Functionally

If the test goal is “the login button is visible and clickable,” a visual test is overkill:

// Bad: Visual test for functional behaviour
await expect(page).toHaveScreenshot('login-page.png');

// Good: Functional assertion
await expect(page.locator('button:has-text("Log in")')).toBeVisible();
await page.click('button:has-text("Log in")');

Functional assertions are faster, more stable, and describe intent better.

3. Your CI Environment Renders Differently

If fonts, subpixel rendering, or GPU acceleration differ between your local machine and CI, visual tests will be noisy. Solutions:

  • Use Docker containers with identical rendering environments
  • Disable font anti-aliasing in test config
  • Accept higher tolerance (which reduces test value)

None of these are free. If the engineering effort exceeds the bug-catching value, skip visual tests.

Accessibility Testing with axe-core

Automated accessibility testing is different from visual testing. Accessibility violations are objective: a missing alt attribute, a contrast ratio below 4.5:1, a form input without a <label>. These are measurable defects, not design opinions.

The most widely used accessibility testing library is axe-core, developed by Deque. Playwright integrates with it via @axe-core/playwright.

Setup

Install the package:

npm install --save-dev @axe-core/playwright

Basic Usage

import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test('homepage has no accessibility violations', async ({ page }) => {
  await page.goto('/');

  const accessibilityScanResults = await new AxeBuilder({ page }).analyze();

  expect(accessibilityScanResults.violations).toEqual([]);
});

If the page has accessibility issues, the test fails and outputs:

Expected: []
Received: [
  {
    id: 'label',
    impact: 'critical',
    description: 'Form elements must have labels',
    nodes: [
      {
        html: '<input type="text" name="search">',
        target: ['input[name="search"]'],
      }
    ]
  }
]

This tells you:

  • What the violation is: “Form elements must have labels”
  • Where it is: input[name="search"]
  • How severe it is: critical

Scoping to Specific Regions

You can run axe on a specific part of the page:

test('navigation bar has no a11y violations', async ({ page }) => {
  await page.goto('/');

  const accessibilityScanResults = await new AxeBuilder({ page })
    .include('nav') // Only scan the <nav> element
    .analyze();

  expect(accessibilityScanResults.violations).toEqual([]);
});

This is useful for component-level testing.

Excluding Known Issues

If you have a third-party widget that you can’t fix immediately, you can exclude it from the scan:

const accessibilityScanResults = await new AxeBuilder({ page })
  .exclude('.third-party-chat-widget')
  .analyze();

:::warning[Don’t Overuse Exclusions] Excluding regions should be temporary. If you exclude half your page, the test has no value. Track excluded regions as technical debt and fix them incrementally. :::

Testing Specific WCAG Levels

axe-core supports different WCAG conformance levels. You can test for specific standards:

const accessibilityScanResults = await new AxeBuilder({ page })
  .withTags(['wcag2a', 'wcag2aa']) // WCAG 2.0 Level A and AA
  .analyze();

Common tags:

  • wcag2a — WCAG 2.0 Level A
  • wcag2aa — WCAG 2.0 Level AA (most common legal requirement)
  • wcag21aa — WCAG 2.1 Level AA
  • wcag22aa — WCAG 2.2 Level AA (latest standard)
  • best-practice — Deque’s recommended best practices beyond WCAG

Start with wcag2aa. That’s the standard most regulations reference (ADA in the US, EN 301 549 in the EU).

What Automated Accessibility Testing Misses

Automated tools catch structural issues. They do not catch:

  • Keyboard navigation order — Is the focus order logical?
  • Screen reader announcements — Does the page make sense when read aloud?
  • Cognitive load — Is the interface overwhelming?
  • Context-specific contrast issues — A 4.5:1 contrast ratio might pass WCAG but still be unreadable for some users depending on surrounding colours and text size.

These require manual testing with assistive technologies:

  • Test with keyboard only (Tab, Shift+Tab, Enter, Escape)
  • Test with a screen reader (NVDA on Windows, VoiceOver on macOS, JAWS commercially)
  • Test with browser zoom at 200%
  • Test with high contrast mode enabled

Automated tests are the baseline, not the goal. They catch obvious issues so manual testing can focus on nuanced issues.

Integrating Visual and Accessibility Checks into CI

Both visual and accessibility tests should run in CI, but with different failure policies.

Accessibility Tests: Always Block the Build

Accessibility violations are objective defects. They should fail the build:

test('no a11y violations on checkout page', async ({ page }) => {
  await page.goto('/checkout');
  const results = await new AxeBuilder({ page }).analyze();
  expect(results.violations).toEqual([]);
});

If this fails, the PR should not merge until the violation is fixed.

Visual Tests: Review Required, Don’t Auto-Block

Visual test failures often require human judgment. Configure CI to:

  1. Upload the visual diffs as artifacts
  2. Require manual review before merging
  3. Don’t auto-fail the build unless the diff is catastrophic (e.g., the entire page is white)

In GitHub Actions:

- name: Run Playwright tests
  run: npx playwright test

- name: Upload visual diffs
  if: failure()
  uses: actions/upload-artifact@v3
  with:
    name: playwright-screenshots
    path: test-results/

Reviewers download the artifacts, inspect the diffs, and decide whether to update baselines or fix the code.

Common Pitfalls

Pitfall 1: Visual Tests on Every Page

Don’t screenshot every page in your app. That’s hundreds of baselines to maintain. Instead:

  • Screenshot your design system components
  • Screenshot high-risk pages (checkout, payment, dashboard)
  • Screenshot complex layouts that are hard to assert functionally

Pitfall 2: Ignoring Accessibility Test Failures

It’s tempting to .exclude() accessibility violations to make the build green. Resist this. Each exclusion is a real barrier for real users.

If you can’t fix it immediately:

  1. File a ticket
  2. Add a comment explaining why it’s excluded
  3. Set a deadline to fix it

Pitfall 3: Running axe-core Too Late

Run accessibility scans early in the feature development process, not just before release. Fixing an accessibility issue in design is cheap. Fixing it in production is expensive.

Conclusion

Visual regression testing and accessibility testing are both valuable, but they require discipline to maintain.

Visual testing works best for:

  • Component libraries
  • Complex dynamic layouts
  • Strict brand consistency requirements

Accessibility testing works best for:

  • Every page in your app (it’s cheap to run)
  • High-traffic user journeys
  • Compliance-critical features (checkout, forms, legal pages)

Key takeaways:

  • Use toHaveScreenshot() for scoped visual regression on stable components
  • Set realistic tolerance thresholds to avoid noise
  • Use @axe-core/playwright to catch objective accessibility violations
  • Always block the build on accessibility failures
  • Supplement automated accessibility tests with manual testing

:::tip[Start with Accessibility] If you can only add one type of test, add accessibility tests. They’re low-maintenance, high-value, and catch real defects that affect real users. Visual tests are higher maintenance and require more judgment. :::

Action for this week: Add one accessibility test to your most important user journey (login, checkout, registration). Run it locally. Fix any violations. Then add it to CI and make it a required check. You’ve just made your app more accessible to millions of users.

Next in this series: Part 8 — Parallel Runs, Sharding, and CI-Ready Playwright