1# Visual Regression Testing
2 
3## Table of Contents
4 
51. [Quick Reference](#quick-reference)
62. [Patterns](#patterns)
73. [Decision Guide](#decision-guide)
84. [Anti-Patterns](#anti-patterns)
95. [Troubleshooting](#troubleshooting)
10 
11> **When to use**: Detecting unintended visual changes—layout shifts, style regressions, broken responsive designs—that functional assertions miss.
12 
13## Quick Reference
14 
15```typescript
16// Element screenshot
17await expect(page.getByTestId('product-card')).toHaveScreenshot();
18 
19// Full page screenshot
20await expect(page).toHaveScreenshot('landing-hero.png');
21 
22// Threshold for minor pixel variance
23await expect(page).toHaveScreenshot({ maxDiffPixelRatio: 0.01 });
24 
25// Mask volatile content
26await expect(page).toHaveScreenshot({
27  mask: [page.getByTestId('clock'), page.getByRole('img', { name: 'User photo' })],
28});
29 
30// Disable CSS animations
31await expect(page).toHaveScreenshot({ animations: 'disabled' });
32 
33// Update baselines
34npx playwright test --update-snapshots
35```
36 
37## Patterns
38 
39### Masking Volatile Content
40 
41**Use when**: Page contains timestamps, avatars, ad slots, relative dates, random images, or A/B variants.
42 
43The `mask` option overlays a solid box over specified locators before capturing.
44 
45```typescript
46test('analytics panel with masked dynamic elements', async ({ page }) => {
47  await page.goto('/analytics');
48 
49  await expect(page).toHaveScreenshot('analytics.png', {
50    mask: [
51      page.getByTestId('last-updated'),
52      page.getByTestId('profile-avatar'),
53      page.getByTestId('active-users'),
54      page.locator('.promo-banner'),
55    ],
56    maskColor: '#FF00FF',
57  });
58});
59 
60test('activity stream with relative times', async ({ page }) => {
61  await page.goto('/activity');
62 
63  await expect(page).toHaveScreenshot('activity.png', {
64    mask: [page.locator('time[datetime]')],
65  });
66});
67```
68 
69**Alternative: freeze content with JavaScript** when masking affects layout:
70 
71```typescript
72test('freeze timestamps before capture', async ({ page }) => {
73  await page.goto('/analytics');
74 
75  await page.evaluate(() => {
76    document.querySelectorAll('[data-testid="time-display"]').forEach((el) => {
77      el.textContent = 'Jan 1, 2025 12:00 PM';
78    });
79  });
80 
81  await expect(page).toHaveScreenshot('analytics-frozen.png');
82});
83```
84 
85### Disabling Animations
86 
87**Use when**: Always. CSS animations and transitions are the primary cause of flaky visual diffs.
88 
89```typescript
90test('renders without animation interference', async ({ page }) => {
91  await page.goto('/');
92 
93  await expect(page).toHaveScreenshot('home.png', {
94    animations: 'disabled',
95  });
96});
97```
98 
99**Set globally** in config:
100 
101```typescript
102// playwright.config.ts
103export default defineConfig({
104  expect: {
105    toHaveScreenshot: {
106      animations: 'disabled',
107    },
108  },
109});
110```
111 
112When `animations: 'disabled'` is set, Playwright injects CSS forcing animation/transition duration to 0s, waits for running animations to finish, then captures.
113 
114For JavaScript-driven animations (GSAP, Framer Motion), wait for stability:
115 
116```typescript
117test('page with JS animations', async ({ page }) => {
118  await page.goto('/animated-hero');
119 
120  const heroBanner = page.getByTestId('hero-banner');
121  await heroBanner.waitFor({ state: 'visible' });
122  
123  // Wait for animation to complete by checking for stable state
124  await expect(heroBanner).not.toHaveClass(/animating/);
125 
126  await expect(page).toHaveScreenshot('hero.png', {
127    animations: 'disabled',
128  });
129});
130```
131 
132### Configuring Thresholds
133 
134**Use when**: Minor rendering differences from anti-aliasing, font hinting, or sub-pixel rendering cause false failures.
135 
136| Option | Controls | Typical Value |
137|---|---|---|
138| `maxDiffPixels` | Absolute pixel count that can differ | `100` for pages, `10` for components |
139| `maxDiffPixelRatio` | Fraction of total pixels (0-1) | `0.01` (1%) for pages |
140| `threshold` | Per-pixel color tolerance (0-1) | `0.2` for most UIs, `0.1` for design systems |
141 
142```typescript
143test('control panel allows minor variance', async ({ page }) => {
144  await page.goto('/control-panel');
145 
146  await expect(page).toHaveScreenshot('control-panel.png', {
147    maxDiffPixelRatio: 0.01,
148  });
149});
150 
151test('brand logo renders pixel-perfect', async ({ page }) => {
152  await page.goto('/brand');
153 
154  await expect(page.getByTestId('brand-logo')).toHaveScreenshot('brand-logo.png', {
155    maxDiffPixels: 0,
156    threshold: 0,
157  });
158});
159 
160test('graph allows anti-aliasing differences', async ({ page }) => {
161  await page.goto('/reports');
162 
163  await expect(page.getByTestId('sales-graph')).toHaveScreenshot('sales-graph.png', {
164    threshold: 0.3,
165    maxDiffPixels: 200,
166  });
167});
168```
169 
170**Global thresholds** in config:
171 
172```typescript
173// playwright.config.ts
174export default defineConfig({
175  expect: {
176    toHaveScreenshot: {
177      maxDiffPixelRatio: 0.01,
178      threshold: 0.2,
179      animations: 'disabled',
180    },
181  },
182});
183```
184 
185### CI Configuration
186 
187**Use when**: Running visual tests in CI. Consistent rendering is critical—the same test must produce identical screenshots every time.
188 
189**The problem**: Font rendering and anti-aliasing differ across operating systems. macOS snapshots won't match Linux.
190 
191**The solution**: Run visual tests in Docker using the official Playwright container. Generate and update snapshots from the same container.
192 
193**GitHub Actions with Docker**
194 
195```yaml
196# .github/workflows/visual-tests.yml
197name: Visual Regression Tests
198on: [push, pull_request]
199 
200jobs:
201  visual-tests:
202    runs-on: ubuntu-latest
203    container:
204      image: mcr.microsoft.com/playwright:v1.48.0-noble
205    steps:
206      - uses: actions/checkout@v4
207 
208      - uses: actions/setup-node@v4
209        with:
210          node-version: lts/*
211          cache: npm
212 
213      - run: npm ci
214 
215      - name: Run visual tests
216        run: npx playwright test --project=visual
217        env:
218          HOME: /root
219 
220      - uses: actions/upload-artifact@v4
221        if: failure()
222        with:
223          name: visual-test-report
224          path: playwright-report/
225          retention-days: 14
226```
227 
228**Updating snapshots locally using Docker**:
229 
230```bash
231docker run --rm -v $(pwd):/work -w /work \
232  mcr.microsoft.com/playwright:v1.48.0-noble \
233  npx playwright test --update-snapshots --project=visual
234```
235 
236**Add script to `package.json`**:
237 
238```json
239{
240  "scripts": {
241    "test:visual": "npx playwright test --project=visual",
242    "test:visual:update": "docker run --rm -v $(pwd):/work -w /work mcr.microsoft.com/playwright:v1.48.0-noble npx playwright test --update-snapshots --project=visual"
243  }
244}
245```
246 
247**Platform-agnostic snapshots** (requires Docker for generation):
248 
249```typescript
250// playwright.config.ts
251export default defineConfig({
252  snapshotPathTemplate: '{testDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{ext}',
253  projects: [
254    {
255      name: 'visual',
256      testMatch: '**/*.visual.spec.ts',
257      use: { ...devices['Desktop Chrome'] },
258    },
259  ],
260});
261```
262 
263### Full Page vs Element Screenshots
264 
265**Use when**: Deciding scope. Full page catches layout shifts. Element screenshots isolate components and are more stable.
266 
267```typescript
268test('full page captures layout shifts', async ({ page }) => {
269  await page.goto('/');
270 
271  // Visible viewport
272  await expect(page).toHaveScreenshot('home-viewport.png');
273 
274  // Entire scrollable page
275  await expect(page).toHaveScreenshot('home-full.png', {
276    fullPage: true,
277  });
278});
279 
280test('element screenshot isolates component', async ({ page }) => {
281  await page.goto('/catalog');
282 
283  await expect(page.getByRole('table')).toHaveScreenshot('catalog-table.png');
284  await expect(page.getByTestId('featured-item')).toHaveScreenshot('featured-item.png');
285});
286```
287 
288**Rule of thumb**: Element screenshots for independently changing components. Full page screenshots for key layouts where spacing matters.
289 
290### Responsive Visual Testing
291 
292**Use when**: Application has responsive breakpoints requiring verification at different viewport sizes.
293 
294```typescript
295const breakpoints = [
296  { name: 'phone', width: 375, height: 812 },
297  { name: 'tablet', width: 768, height: 1024 },
298  { name: 'desktop', width: 1440, height: 900 },
299];
300 
301for (const bp of breakpoints) {
302  test(`landing at ${bp.name} (${bp.width}x${bp.height})`, async ({ page }) => {
303    await page.setViewportSize({ width: bp.width, height: bp.height });
304    await page.goto('/');
305 
306    await expect(page).toHaveScreenshot(`landing-${bp.name}.png`, {
307      animations: 'disabled',
308      fullPage: true,
309    });
310  });
311}
312```
313 
314**Alternative: use projects for responsive testing**:
315 
316```typescript
317// playwright.config.ts
318export default defineConfig({
319  projects: [
320    {
321      name: 'desktop',
322      testMatch: '**/*.visual.spec.ts',
323      use: {
324        ...devices['Desktop Chrome'],
325        viewport: { width: 1440, height: 900 },
326      },
327    },
328    {
329      name: 'tablet',
330      testMatch: '**/*.visual.spec.ts',
331      use: { ...devices['iPad (gen 7)'] },
332    },
333    {
334      name: 'mobile',
335      testMatch: '**/*.visual.spec.ts',
336      use: { ...devices['iPhone 14'] },
337    },
338  ],
339});
340```
341 
342### Component Visual Testing
343 
344**Use when**: Testing individual UI components in isolation—buttons, cards, forms, modals. Faster and more stable than full-page screenshots.
345 
346```typescript
347test.describe('Button visual states', () => {
348  test('primary button', async ({ page }) => {
349    await page.goto('/storybook/iframe.html?id=button--primary');
350    const btn = page.getByRole('button');
351    await expect(btn).toHaveScreenshot('btn-primary.png', {
352      animations: 'disabled',
353    });
354  });
355 
356  test('primary button hover', async ({ page }) => {
357    await page.goto('/storybook/iframe.html?id=button--primary');
358    const btn = page.getByRole('button');
359    await btn.hover();
360    await expect(btn).toHaveScreenshot('btn-primary-hover.png', {
361      animations: 'disabled',
362    });
363  });
364 
365  test('button sizes', async ({ page }) => {
366    for (const size of ['small', 'medium', 'large']) {
367      await page.goto(`/storybook/iframe.html?id=button--${size}`);
368      const btn = page.getByRole('button');
369      await expect(btn).toHaveScreenshot(`btn-${size}.png`, {
370        animations: 'disabled',
371      });
372    }
373  });
374});
375```
376 
377**Using a dedicated test harness** instead of Storybook:
378 
379```typescript
380test.describe('Card component', () => {
381  test.beforeEach(async ({ page }) => {
382    await page.goto('/test-harness/card');
383  });
384 
385  test('default state', async ({ page }) => {
386    await expect(page.getByTestId('card')).toHaveScreenshot('card-default.png', {
387      animations: 'disabled',
388    });
389  });
390 
391  test('truncates long content', async ({ page }) => {
392    await page.goto('/test-harness/card?content=long');
393    await expect(page.getByTestId('card')).toHaveScreenshot('card-long.png', {
394      animations: 'disabled',
395    });
396  });
397});
398```
399 
400### Updating Snapshots
401 
402**Use when**: Intentionally changed UI—design refresh, rebrand, new feature. Never update when diff is unexpected.
403 
404```bash
405# Update all snapshots
406npx playwright test --update-snapshots
407 
408# Update for specific file
409npx playwright test tests/landing.spec.ts --update-snapshots
410 
411# Update for specific project
412npx playwright test --project=chromium --update-snapshots
413```
414 
415**Workflow for reviewing changes:**
416 
4171. Run tests and view failures in HTML report:
418   ```bash
419   npx playwright test
420   npx playwright show-report
421   ```
422   The report shows expected, actual, and diff images side-by-side.
423 
4242. If changes are intentional, update:
425   ```bash
426   npx playwright test --update-snapshots
427   ```
428 
4293. Review updated snapshots before committing:
430   ```bash
431   git diff --name-only
432   ```
433 
434**Tag visual tests for selective updates:**
435 
436```typescript
437test('landing visual @visual', async ({ page }) => {
438  await page.goto('/');
439  await expect(page).toHaveScreenshot('landing.png', {
440    animations: 'disabled',
441  });
442});
443```
444 
445```bash
446npx playwright test --grep @visual --update-snapshots
447```
448 
449### Cross-Browser Visual Testing
450 
451**Use when**: Users span Chrome, Firefox, Safari and you need per-browser rendering verification.
452 
453Playwright separates snapshots by project name automatically. Each browser gets its own baseline—browsers render fonts and shadows differently.
454 
455```typescript
456// playwright.config.ts
457export default defineConfig({
458  expect: {
459    toHaveScreenshot: {
460      animations: 'disabled',
461      maxDiffPixelRatio: 0.01,
462    },
463  },
464  projects: [
465    {
466      name: 'chromium',
467      use: { ...devices['Desktop Chrome'] },
468    },
469    {
470      name: 'firefox',
471      use: { ...devices['Desktop Firefox'] },
472    },
473    {
474      name: 'webkit',
475      use: { ...devices['Desktop Safari'] },
476    },
477  ],
478});
479```
480 
481**Strategy**: Run visual tests in a single browser (Chromium on Linux in CI) to minimize snapshot count. Add other browsers only when you have actual cross-browser rendering bugs:
482 
483```typescript
484// playwright.config.ts
485export default defineConfig({
486  projects: [
487    {
488      name: 'visual',
489      testMatch: '**/*.visual.spec.ts',
490      use: { ...devices['Desktop Chrome'] },
491    },
492    {
493      name: 'chromium',
494      testIgnore: '**/*.visual.spec.ts',
495      use: { ...devices['Desktop Chrome'] },
496    },
497    {
498      name: 'firefox',
499      testIgnore: '**/*.visual.spec.ts',
500      use: { ...devices['Desktop Firefox'] },
501    },
502  ],
503});
504```
505 
506## Decision Guide
507 
508| Scenario | Approach | Rationale |
509|---|---|---|
510| Key landing/marketing pages | Full page, `fullPage: true` | Catches layout shifts, spacing, overall harmony |
511| Individual components | Element screenshot | Isolated, fast, immune to unrelated changes |
512| Page with dynamic content | Full page + `mask` | Covers layout while ignoring volatile content |
513| Design system library | Element per variant, zero threshold | Pixel-perfect enforcement |
514| Responsive verification | Screenshot per viewport | Catches breakpoint bugs |
515| Cross-browser consistency | Separate snapshots per browser | Browsers render differently |
516| CI pipeline | Docker container, Linux-only snapshots | Consistent rendering |
517| Threshold: design system | `threshold: 0`, `maxDiffPixels: 0` | Zero tolerance |
518| Threshold: content pages | `maxDiffPixelRatio: 0.01`, `threshold: 0.2` | Minor anti-aliasing variance |
519| Threshold: charts/graphs | `maxDiffPixels: 200`, `threshold: 0.3` | Anti-aliasing on curves varies |
520 
521## Anti-Patterns
522 
523| Don't | Problem | Do Instead |
524|---|---|---|
525| Visual test every page | Massive maintenance, constant false failures | Pick 5-10 key pages and critical components |
526| Skip masking dynamic content | Screenshots differ every run, permanently flaky | Use `mask` for all volatile elements |
527| Run across macOS, Linux, Windows | Font rendering differs, snapshots never match | Standardize on Linux via Docker |
528| Skip Docker in CI | OS updates shift rendering silently | Pin specific Playwright Docker image |
529| Blindly run `--update-snapshots` | Accepts unintentional regressions | Always review diff in HTML report first |
530| Skip `animations: 'disabled'` | CSS transitions create random diffs | Set globally in config |
531| Replace functional assertions with visual tests | Diffs don't tell you *what* broke | Visual tests complement, never replace |
532| Commit snapshots from different platforms | Tests fail for everyone | All team members use same Docker container |
533| Set threshold too high (`0.1`) | 10% pixel change passes, defeats purpose | Start with `0.01`, adjust per-test |
534| Full page on infinite scroll pages | Page height nondeterministic | Element screenshots on above-the-fold content |
535 
536## Troubleshooting
537 
538### "Screenshot comparison failed" on first CI run after local development
539 
540**Cause**: Snapshots generated on macOS locally. CI runs on Linux. Font rendering differs.
541 
542**Fix**: Generate snapshots using Docker:
543 
544```bash
545docker run --rm -v $(pwd):/work -w /work \
546  mcr.microsoft.com/playwright:v1.48.0-noble \
547  npx playwright test --update-snapshots --project=visual
548```
549 
550Commit Linux-generated snapshots.
551 
552### "Expected screenshot to match but X pixels differ"
553 
554**Cause**: Anti-aliasing, font hinting, sub-pixel rendering differences.
555 
556**Fix**: Add tolerance:
557 
558```typescript
559await expect(page).toHaveScreenshot('page.png', {
560  maxDiffPixelRatio: 0.01,
561  threshold: 0.2,
562});
563```
564 
565Check HTML report diff image to determine if it's regression or noise.
566 
567### Visual tests pass locally but fail in CI (even with Docker)
568 
569**Cause**: Different Playwright versions locally vs CI.
570 
571**Fix**: Ensure `package.json` version matches Docker image tag:
572 
573```json
574{
575  "devDependencies": {
576    "@playwright/test": "latest"
577  }
578}
579```
580 
581```yaml
582container:
583  image: mcr.microsoft.com/playwright:v1.48.0-noble
584```
585 
586### Animations cause random diff failures
587 
588**Cause**: CSS animations captured mid-frame.
589 
590**Fix**: Set `animations: 'disabled'` globally:
591 
592```typescript
593// playwright.config.ts
594export default defineConfig({
595  expect: {
596    toHaveScreenshot: {
597      animations: 'disabled',
598    },
599  },
600});
601```
602 
603For JS animations, wait for stable state before capture.
604 
605### Snapshot file names conflict between tests
606 
607**Cause**: Two tests use same screenshot name without unique paths.
608 
609**Fix**: Use explicit unique names:
610 
611```typescript
612await expect(page).toHaveScreenshot('auth-home.png');
613await expect(page).toHaveScreenshot('public-home.png');
614```
615 
616Or customize snapshot path template:
617 
618```typescript
619export default defineConfig({
620  snapshotPathTemplate: '{testDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{ext}',
621});
622```
623 
624### Too many snapshot files to maintain
625 
626**Cause**: Visual tests for every page, browser, viewport.
627 
628**Fix**: Be selective. Visual test only high-risk pages:
629- Landing and marketing pages
630- Design system components
631- Complex layouts (dashboards, data tables)
632- Pages after major refactor
633 
634Skip pages where functional assertions cover key elements.
635