Test automation

Playwright GitHub Actions setup: a fast CI pipeline you can trust

A complete Playwright GitHub Actions workflow to copy: sharding, merged HTML reports, retries with traces, secrets for test accounts and what not to cache.

A working Playwright GitHub Actions setup is one workflow file. It checks out your code, sets up Node.js, runs npm ci, installs the browsers with their system dependencies (npx playwright install --with-deps), runs npx playwright test and uploads the HTML report as an artifact. Playwright’s Setting up CI guide shows that file, and npm init playwright@latest offers to create it for you.

That file works on day one. Keeping it fast and trustworthy as the suite grows takes a few more decisions: which URL the tests hit, when a retry is allowed, where the traces go, how to split the suite across machines and merge the results, how test accounts stay secret and what not to cache. Below is a complete workflow that makes those decisions, checked against the Playwright and GitHub docs as of October 2026.

The basic Playwright GitHub Actions workflow

The generated .github/workflows/playwright.yml runs on pushes and pull requests to main (or master), and three details in it matter more than they look:

  • The runner must be able to run browsers. The CI guide gives two ways: the official Docker image, or --with-deps, which installs the operating system packages next to the browsers.
  • The report is uploaded when tests fail. The upload step has if: ${{ !cancelled() }}. Without a status function a step runs only after success, and GitHub’s expressions reference recommends !cancelled() over always() for steps that should run either way.
  • There is no job timeout. The docs set globalTimeout in the config instead. When the runner kills a job at its time limit, no report is written; when Playwright stops the run itself, the reporters still produce one. If you add timeout-minutes anyway, keep it comfortably above globalTimeout.

Playwright config for CI

Most of the CI behavior lives in playwright.config.ts. This excerpt keeps the docs’ basic configuration and changes three things: one retry instead of two, a blob reporter for sharding, and a base URL that can come from the environment.

import { defineConfig } from '@playwright/test';

const CI = !!process.env.CI;
// A deployed preview or staging URL. When it is empty, the app starts locally.
const deployedURL = process.env.PLAYWRIGHT_TEST_BASE_URL;

export default defineConfig({
  fullyParallel: true, // shards split by test, not by file
  forbidOnly: CI, // a forgotten test.only fails the run
  retries: CI ? 1 : 0, // retry in CI only
  workers: CI ? 1 : undefined, // the CI guide's advice; shards add speed
  globalTimeout: 60 * 60 * 1000, // stop with a report instead of hanging
  reporter: CI ? [['dot'], ['blob']] : 'html',
  use: {
    baseURL: deployedURL || 'http://localhost:3000',
    trace: 'on-first-retry',
  },
  webServer: deployedURL
    ? undefined
    : {
        command: 'npm run start', // your app's start command
        url: 'http://localhost:3000',
        reuseExistingServer: !CI,
      },
  // testDir, projects and the rest stay as they are.
});
  • workers: 1 on CI is what the CI guide recommends “to prioritize stability and reproducibility”. The speed comes from sharding instead.
  • fullyParallel: true lets Playwright balance shards by test rather than by file, the mode the sharding guide prefers. It assumes your tests do not depend on each other.
  • The dot reporter keeps each shard’s log short, and blob writes the file that is merged later. Locally you still get the HTML report.

Which URL the tests should hit

A pull request should test its own code. On the pull_request event, GitHub checks out a merge branch, so your tests run against “the merged result, not just the head branch alone” (events reference). That leaves three setups:

  1. Start the app in the job. With PLAYWRIGHT_TEST_BASE_URL empty, the webServer block starts the app from the checked out code and waits for its URL. This is what the workflow below does. It tests exactly the code under review, but every shard starts its own copy of the app.
  2. Test the preview deployment. If your host creates GitHub deployments (the CI guide names Vercel), copy the workflow and follow the docs’ “On deployment” example: trigger on deployment_status, run the test job only if: github.event.deployment_status.state == 'success', and set PLAYWRIGHT_TEST_BASE_URL to ${{ github.event.deployment_status.target_url }} in the test step. Also add needs.test.result != 'skipped' to the merge job’s condition, so the other deployment states do not start it.
  3. Test a shared staging URL. Useful for a nightly run, but it only tests a pull request if staging is running that pull request.

Read the variable in your config, as above. Playwright uses PLAYWRIGHT_TEST_BASE_URL only as the default baseURL (source), so a fixed baseURL in the config wins over it and the deployment run never reaches the deployment.

Retries that do not hide flaky tests

Playwright does not retry failed tests by default. Turn retries on in CI only and keep the count low: one retry is enough to label a test that failed and then passed as flaky, and trace: 'on-first-retry', the CI setting the trace viewer docs recommend, records that retry as a trace. The docs’ basic configuration uses two retries; that works too, but a broken test then runs three times.

A flaky test does not fail the run by default. The HTML report has a filter for flaky tests, and failOnFlakyTests (added in version 1.52) fails the run instead, if you want flakiness to block a merge. Either way, read the label as a bug report, not as a pass. The guide to flaky tests in Playwright covers finding the cause.

The complete workflow: sharded, on every pull request

This is the file we would start from: four shards, a blob report from each and one merged HTML report. The action versions are the current major versions on their GitHub release pages as of October 2026 (actions/checkout@v7, actions/setup-node@v7, actions/upload-artifact@v7 and actions/download-artifact@v8). The Playwright docs still show older majors, and every input used here exists in the current ones.

name: Playwright Tests

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

# On a pull request, a new push cancels the older run. On main, every run
# gets its own group (the run ID), so nothing is cancelled.
concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: lts/*
      - name: Install dependencies
        run: npm ci
      - name: Install Playwright browsers
        run: npx playwright install --with-deps
      - name: Run Playwright tests
        run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
        env:
          TEST_USER_EMAIL: ${{ secrets.TEST_USER_EMAIL }}
          TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}
      - name: Upload blob report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v7
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: blob-report
          retention-days: 1

  merge-reports:
    # Runs after every shard, also when some failed, so the report exists.
    if: ${{ !cancelled() }}
    needs: [test]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: lts/*
      - name: Install dependencies
        run: npm ci
      - name: Download blob reports
        uses: actions/download-artifact@v8
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true
      - name: Merge into one HTML report
        run: npx playwright merge-reports --reporter=html,github ./all-blob-reports
      - name: Upload HTML report
        uses: actions/upload-artifact@v7
        with:
          name: html-report--attempt-${{ github.run_attempt }}
          path: playwright-report
          retention-days: 14

What each part does:

  • Concurrency. On a pull request, a new push cancels the run it replaces. On main, github.head_ref is empty, so the group falls back to the run ID and nothing is cancelled. This is the fallback pattern from GitHub’s concurrency docs.
  • Permissions. contents: read limits the workflow’s token to reading the code.
  • Matrix. fail-fast defaults to true, which cancels the other shards as soon as one fails. Turning it off gives you a complete report.
  • Blob artifacts. Artifact names must be unique, hence the shard number in the name. One day of retention is enough, because only the merge job reads them.
  • Merge job. It runs after all shards, also when some failed, installs the npm packages but no browsers and merges the blob reports. The github reporter adds failure annotations; the docs advise against it inside a matrix, where the same failures multiply, so it runs once here. The run attempt in the name gives each rerun its own report.

One thing the merge job is not: a gate. merge-reports succeeds even when tests failed (merge code), so if you protect main, require the shard checks.

Reading a failure

Download html-report--attempt-1 from the run’s artifacts, unzip it and serve it with npx playwright show-report followed by the folder name: the CI guide notes that the report needs a web server to work. Click the trace icon next to a failed test to step through it. The traces travel inside the report, because blob reports carry attachments such as traces and the merged HTML report keeps them in its data folder.

Secrets for test accounts

Give CI its own test accounts with the least access that works, as GitHub’s secrets docs recommend for any credential, and store their credentials as repository secrets (Settings, then Secrets and variables, then Actions).

  • Pass them as environment variables on the step that needs them, as the workflow does with TEST_USER_EMAIL and TEST_USER_PASSWORD, and read them from process.env in your tests. A workflow only gets the secrets it names, and GitHub advises against passing them on the command line.
  • Expect them to be missing. Secrets are not passed to runs triggered from a fork or by Dependabot, and an unset secret evaluates to an empty string. Make the login step fail with a clear message when the value is empty, so a missing secret does not look like a product bug.
  • Keep the saved login out of git. Playwright’s authentication guide saves the logged in state under playwright/.auth and recommends adding that folder to .gitignore, because the file holds cookies that can impersonate the test account.
  • Treat reports and traces as sensitive. The CI guide warns that traces, HTML reports and logs can contain credentials and tokens. GitHub redacts secrets printed to the job log, but anyone signed in with read access to the repository can download the artifacts. In a public repository, keep retention short or skip uploading traces from runs that log in.

What not to cache

What Cache it? Why
Browsers No Restoring takes about as long as downloading
npm downloads Optional setup-node keys them on the lockfile
playwright/.auth Never Anyone who can open a pull request can read it
Reports and traces No They are artifacts

Playwright’s CI guide does not recommend caching browsers: restoring the cache takes about as long as downloading the binaries, and on Linux the operating system dependencies cannot be cached anyway. If you cache them regardless, key the cache on the Playwright version.

For npm, actions/setup-node caches the download cache with cache: 'npm', keyed on package-lock.json, and never node_modules. It turns caching on by itself when package.json names npm in its packageManager field, and its README suggests package-manager-cache: false in workflows with access to sensitive information when you do not need the cache. GitHub’s caching reference warns that anyone who can open a pull request can read the caches of the base branch, so nothing with credentials goes into one.

The official Docker image option

The Playwright image on Microsoft Artifact Registry ships the browsers and their system dependencies, but not the Playwright package, which npm ci still installs. To use it, run the test job in the container, as the CI guide’s “Via Containers” example does, and delete the browser install step:

container:
  image: mcr.microsoft.com/playwright:v1.64.1-noble
  options: --user 1001

Pin the tag to the exact Playwright version in your lockfile: the Docker docs warn that Playwright cannot find the browsers when the image and the project disagree. As of October 2026 the latest release is 1.64.1, published on October 14, 2026, and the image comes in noble (Ubuntu 24.04), jammy (Ubuntu 22.04) and resolute (Ubuntu 26.04) variants. Bump the tag in the same pull request as the package. In return, the CI guide notes, you get a consistent environment, which matters for screenshot comparisons across operating systems.

Keep it fast as the suite grows

  • More shards. Change shardIndex and shardTotal together. Each shard repeats the setup (checkout, npm ci, browsers), so more shards pay off only while test time is much larger than setup time.
  • Install only what you run. npx playwright install --with-deps chromium installs one browser, and --only-shell skips the full Chromium build when you only run headless (browsers docs).
  • Run fewer browsers on pull requests. Since version 1.64, a project with default: false stays in the config but runs only when selected, so npx playwright test runs your main browser and --project="*" runs all of them, for example in a nightly workflow (release notes).
  • Run the likely failures first. For faster feedback, the CI guide runs npx playwright test --only-changed=origin/$GITHUB_BASE_REF before the full suite, with fetch-depth: 0 on checkout. It is a heuristic that can miss tests, so the full run must still follow.

How we run it

We set up this kind of pipeline in your own repository: plain Playwright and TypeScript tests that run in your CI on every pull request, with unlimited runs that are never billed. Claude agents write and run the tests and propose fixes; a QA engineer reviews every change before it is merged and looks at every failure before it becomes a bug report or a test fix. Before each release, the same results feed a signed Go or No-Go call, and shipping stays your decision. Our open-source sample project runs its suite in the official Docker image, with sharded jobs and blob reports merged into one HTML report, and the Playwright testing page describes the service.

Sources

Researched and drafted with Claude, Anthropic's AI model, and checked against the sources it links. Spotted an error? Tell us and we will correct it.

Keep reading

Every post

Get a Playwright suite that runs on every pull request

Claude agents write and maintain the tests in your repository, and your CI runs them on every pull request. A QA engineer reviews every change and every failure.

Or write to qa@releaseworthy.app. Still deciding after the proposal? We book a Google Meet call for your questions.