C.W.K.
Stream
Lesson 04 of 04 · published

CI, Sharding, and the Blob-Merge Pattern

~18 min · playwright-advanced, ci, sharding

Level 0Test Curious
0 XP0/32 lessons0/13 achievements
0/100 XP to next level100 XP to go0% complete
"A 30-minute E2E suite splits into 3 × 10-minute parallel shards. Same coverage, one-third the wall-clock time."

Why Sharding

CI feedback time is a feature. Devs wait for green before merging; long CI = slow team. E2E tests grow faster than unit tests as your product matures, so even a well-shaped suite hits the 'too slow' wall eventually. Sharding is the standard answer: split the test set into N pieces, run each piece on a separate CI machine in parallel, merge the results.

Playwright has sharding built in. --shard=1/3 says "run the first third of the discovered tests." --shard=2/3 runs the second. You spawn three CI jobs each with a different shard arg, and you've parallelized.

The Blob Reporter — The Missing Piece

Sharded runs produce N separate reports. The blob reporter (--reporter=blob) writes a compact binary report per shard. A final 'merge' job downloads all the blobs and merges them into one HTML report using npx playwright merge-reports. The team sees one unified report instead of N separate ones.

This is the standard CI pipeline shape:

  1. Test job (matrix of N) — each runs npx playwright test --shard=$INDEX/$TOTAL --reporter=blob, uploads its blob.
  2. Merge job — depends on all test jobs, downloads all blobs, runs npx playwright merge-reports --reporter=html ./all-blob-reports.
  3. Upload job — uploads the merged HTML report as an artifact.

The GitHub Actions Setup

GitHub Actions matrix strategy makes this clean: declare matrix: { shard: [1, 2, 3] } and you get three parallel jobs, each receiving its shard number as ${{ matrix.shard }}. The merge job uses needs: [test] to wait for all matrix legs.

Don't shard until the suite is slow enough to need it. The blob-merge dance adds orchestration complexity. For a sub-10-minute suite, sharding is overhead with no win. For a 30+ minute suite, it's the difference between 'I waited for CI' and 'I forgot CI was running.'

Retry vs Reproducibility

CI is also where retries earn their keep. The retries: 2 setting (CI-only via process.env.CI) catches genuine network blips without masking real flakes — Playwright marks retried tests as 'flaky' in the report so you can see the rate even when CI is green. If the flaky rate creeps up, you have a number to react to.

Artifacts to Always Upload

Whatever else you upload, two things are essential:

  • The HTML report (after merging shards) — for browsing failures.
  • The trace .zip files (from test-results/) — for time-traveling through specific failures.

Set retention to ~30 days so you can compare across runs when a flake reappears.

Local vs CI Configuration

The cleanest CI config flips knobs based on process.env.CI:

  • Workers: 1 in CI (less contention), '50%' locally (use the machine).
  • Retries: 2 in CI (catch network jitter), 0 locally (see flakes immediately).
  • Reporter: 'blob' in CI (for sharding), 'html' locally (for browsing).
  • Trace: 'on-first-retry' (same both — captures forensics for CI failures).

Code

GitHub Actions — sharded + merged·yaml
# .github/workflows/playwright.yml — sharded + merged
name: Playwright Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    timeout-minutes: 30
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false       # let all shards finish even if one fails
      matrix:
        shard: [1, 2, 3]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: lts/*
      - name: Install dependencies
        run: npm ci
      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run shard ${{ matrix.shard }}/3
        run: npx playwright test --shard=${{ matrix.shard }}/3 --reporter=blob

      - name: Upload blob report
        uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: blob-report-${{ matrix.shard }}
          path: blob-report
          retention-days: 7

  merge:
    needs: [test]
    if: ${{ !cancelled() }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: lts/*
      - run: npm ci

      - name: Download all blob reports
        uses: actions/download-artifact@v4
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true

      - name: Merge into one HTML report
        run: npx playwright merge-reports --reporter=html ./all-blob-reports

      - name: Upload merged HTML report
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report
          retention-days: 30
Local vs CI — knobs flipped by `process.env.CI`·typescript
// playwright.config.ts — environment-aware
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,         // fail CI if .only slips through
  workers: process.env.CI ? 1 : '50%',  // 1 per shard in CI
  retries: process.env.CI ? 2 : 0,      // catch network jitter, not flakes

  reporter: process.env.CI
    ? [['blob']]                        // blob for sharding + merging
    : [['html', { open: 'never' }]],    // browseable locally

  use: {
    baseURL: process.env.BASE_URL ?? 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },

  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox',  use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit',   use: { ...devices['Desktop Safari'] } },
  ],
});
Sharding from the CLI — locally and in CI·bash
# Local — single process, watchable
npx playwright test

# Local — simulate sharding for debugging
npx playwright test --shard=1/3   # run only the first shard locally
npx playwright test --shard=2/3
npx playwright test --shard=3/3

# Local — merge the blobs you produced into one HTML report
npx playwright merge-reports --reporter=html ./blob-report-1 ./blob-report-2 ./blob-report-3

# CI — uses the shard arg from the matrix
npx playwright test --shard=${SHARD}/${TOTAL_SHARDS} --reporter=blob
What a sharded + merged run looks like·text
# Workflow result you should see in the GitHub Actions UI

┌──────────────┐
│ test (1/3)   │ Running shard 1 of 3 ............ 9m 42s ✅
├──────────────┤
│ test (2/3)   │ Running shard 2 of 3 ............ 9m 18s ✅
├──────────────┤
│ test (3/3)   │ Running shard 3 of 3 ............10m 04s ✅
└──────────────┘
          │
          ▼
┌──────────────┐
│ merge        │ Downloading 3 blobs, merging .... 0m 47s ✅
└──────────────┘

Total wall time: ~11 minutes  (vs ~30 minutes serial)
Artifacts: playwright-report (browseable HTML), blob-report-{1,2,3}

Click the failing test in the merged report → click 'Trace' → time-travel through the failure.

External links

Exercise

If your project's CI runs Playwright tests today, look at the total wall time. If it's under 5 minutes, sharding is overhead — skip this exercise. If it's over 10 minutes, set up the 3-shard pattern: update CI to use a matrix strategy, switch to the blob reporter in CI, add the merge job. Measure the new wall time — you should see ~1/3 the old time. Confirm the merged HTML report works end-to-end (click into a failure, view its trace).
Hint
If the merge job fails with 'no blob reports found,' the download pattern probably doesn't match. The artifact name MUST be unique per shard (blob-report-${{ matrix.shard }}), and the download pattern needs merge-multiple: true to consolidate them into one folder.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.