Flaky tests in Playwright: causes, fixes and a quarantine policy

Flaky tests in Playwright pass on one run and fail on the next with no change to the code. With retries on, Playwright labels a test “flaky” when it fails first and passes on a retry. The causes come from a short list: fixed waits, shared state, colliding test data, animations, third-party services, an overloaded CI runner and, sometimes, a real race condition in your product.

To handle them, reproduce the flake with --repeat-each, read the trace of the failing run, fix the cause with Playwright’s own tools, and quarantine what you cannot fix today without hiding it. Every Playwright API, flag and option below was checked against the docs for Playwright 1.64, the current release as of October 2026.

What Playwright calls a flaky test

Playwright does not retry failed tests by default. With retries on, the retries guide sorts results into three groups:

• passed: passed on the first run;
• flaky: failed on the first run, passed on a retry;
• failed: failed on the first run and on every retry.

A flaky result does not fail the build: the runner exits with code 0 when every failure recovered on a retry, unless you pass --fail-on-flaky-tests or set failOnFlakyTests (version 1.52 and later). That is the danger. As Martin Fowler wrote in 2011, once a team ignores a test that sometimes fails, that test is useless, and soon failures in the healthy tests get ignored too.

What causes flaky tests

The Google Testing Blog groups the sources of flakiness into four layers: the tests, the framework that runs them, the application and its dependencies, and the operating system and hardware. In a Playwright suite, check these first.

Timing and fixed waits

A test that sleeps for 2 seconds passes when the app answers in 1.9 and fails at 2.1. A check that reads the page once, like expect(await locator.isVisible()).toBe(true), never waits for the page to catch up.

Shared state and test order

Tests that share a page, a session, a module variable or a database record pass in the order they were written and fail alone, in parallel or reordered. Playwright runs test files in parallel by default, with no guaranteed order across files.

Test data collisions

“Two tests that create or edit the same record race with each other,” the parallelism guide warns. With a hardcoded unique email or a coupon that works once, whichever test arrives second fails, and that changes from run to run.

Animations

Before a click, Playwright waits until the element is stable (the same bounding box for at least two animation frames). A test can still catch a transition halfway: a check on a drawer that is sliding in, or a click forced through a fading overlay.

Network and third parties

Payment widgets, chat bubbles, analytics and feature flag services can be slow, rate limited or down, or add cookie banners and overlays. Time is a dependency too: a test that expects today’s date can fail when the run crosses midnight.

Environment and resources

Each worker starts its own browser, and the default number of workers is half the logical CPU cores. On a runner short of CPU or memory, actions slow down until some of them time out. Screenshot tests also need identical operating system and browser versions.

Real bugs in the product

Sometimes the test is right. Google’s list for the application layer includes race conditions, uninitialized variables, memory leaks and slow responses. A double submit, a list rendered before its data arrives, a webhook that lands late: your users can hit these too.

How to find flaky tests in Playwright

Rerun the suspect many times

A test that fails one run in 20 passes most single runs, so one green rerun proves nothing. Run it many times with --repeat-each, and turn retries off so a retry cannot hide a failure:

# 50 runs, default workers, no retries
npx playwright test tests/checkout.spec.ts --repeat-each=50 --retries=0

# The same on one worker
npx playwright test tests/checkout.spec.ts --repeat-each=50 --retries=0 --workers=1

Compare the two:

What you see	Where to look
Fails in parallel, passes on one worker	Shared state, test data, CPU and memory
Fails on one worker too	Timing, animations, network, the product
Fails only on CI	Runner resources, browser and OS versions, third parties
Fails around midnight or month end	Dates and time: pin the clock

Turn on retries and read the flaky label

For everyday CI runs, the trace viewer docs suggest retries: 1 with trace: 'on-first-retry', so a failed test gets one more attempt and that attempt is recorded. Tests that pass on the retry are marked flaky in the output and in the HTML report, which filters by browser and by passed, failed, skipped and flaky tests (npx playwright show-report opens it).

Treat the label as a bug report, not a pass. failOnFlakyTests: !!process.env.CI keeps the evidence from the retry but fails the run. Since version 1.62, retryStrategy: 'isolated' runs retries at the end on a single worker, so a test that then passes alone hints at interference.

Read the trace

A trace holds every action with DOM snapshots before and after it, the action log, console messages, network requests and the error. Open it from the HTML report, with npx playwright show-trace path/to/trace.zip, or at trace.playwright.dev, which loads it in your browser without sending it anywhere.

With on-first-retry the trace comes from the retry, so a passing retry shows a good run. For a hunt, rerun with --retries=1 --trace retain-on-failure-and-retries (version 1.59 and later): it keeps the trace of every failed attempt and every retry, so the failing and the passing run sit side by side. Check the last action before the error, its snapshot, pending or failed requests and console errors.

How to fix flaky tests in Playwright

Timing: let Playwright do the waiting

Before a click, for example, Playwright runs actionability checks: exactly one matching element, visible, stable, receiving events and enabled. Web first assertions such as toHaveText retry until they pass or time out (5 seconds by default). Rely on both and delete every fixed wait.

// Bad: a guessed pause, then a check that reads the page once
await page.getByRole('button', { name: 'Add to cart' }).click();
await page.waitForTimeout(2000);
expect(await page.getByTestId('cart-count').textContent()).toBe('1');

// Good: the click waits for actionability, the assertion retries
await page.getByRole('button', { name: 'Add to cart' }).click();
await expect(page.getByTestId('cart-count')).toHaveText('1');

// State that is not on the page: poll it, with a limit
await expect
  .poll(
    async () => {
      const res = await request.get(`/api/orders/${orderId}`);
      return (await res.json()).status;
    },
    { timeout: 15_000 },
  )
  .toBe('paid');

For steps that must pass together, use expect(async () => { ... }).toPass() with a timeout: by default it has none of its own (0) and ignores the expect timeout. When the next step depends on the server, start page.waitForResponse() before the click and await it after.

Shared state and order: a fresh context and a fixture per test

Playwright already gives each test a fresh browser context with its own cookies and storage. Keep it that way: do not share a page through beforeAll, avoid serial mode (the docs advise against it), and put setup and cleanup in a fixture, which is isolated between tests.

// Bad: one shared page and account, and the second test needs the first
test.describe.configure({ mode: 'serial' });
let page: Page;
test.beforeAll(async ({ browser }) => {
  page = await browser.newPage();
});
test('saves an address', async () => {
  /* signed in as qa@example.com */
});
test('lists the address', async () => {
  /* fails when run on its own */
});

// Good (fixtures.ts): each test gets its own account, deleted afterwards
import { test as base } from '@playwright/test';
import { randomUUID } from 'node:crypto';

type Account = { email: string; password: string };

export const test = base.extend<{ account: Account }>({
  account: async ({ request }, use) => {
    const email = `qa-${randomUUID()}@example.test`;
    const account = { email, password: randomUUID() };
    await request.post('/api/test/accounts', { data: account });
    await use(account);
    await request.delete(`/api/test/accounts/${encodeURIComponent(email)}`);
  },
});

Tests that must touch one shared thing, such as a global setting, can share a lock name (version 1.63 and later), so they never run at the same time.

Test data: unique per test or per worker

Create the data each test changes, with a unique id, as the fixture does. For tests that change server state, the authentication guide recommends one account per parallel worker, keyed on testInfo.parallelIndex. For files, use testInfo.outputPath().

Animations: assert the end state, never force

Never pass force: true to push a click through: it skips the check that the element receives the click, so the click can hit an overlay that is still fading. Assert the end state instead. If your CSS honors prefers-reduced-motion, set reducedMotion: 'reduce' in use. Screenshot assertions already stop CSS animations by default.

Network, third parties and time: control what you do not own

The best practices are blunt: “Only test what you control.” Answer third-party calls with page.route() and route.fulfill(), or replay a recorded HAR file with page.routeFromHAR(), and check the real integration in a few tests that do not block pull requests. For dates, the clock guide recommends page.clock.setFixedTime() first.

Environment: give the runner room

The CI guide recommends one worker on CI “to prioritize stability and reproducibility”, and sharding when that gets slow (see running Playwright in GitHub Actions). If a test fails only on CI, compare its trace timings with a local run before touching the test.

Real bugs: report them, do not dodge them

When the trace shows the product misbehaving (two POST requests for one click, a console error, stale data), file a bug with the trace attached and leave the assertion alone. A test changed to wait out a race hides the bug from you, not from your users. Our post on self-healing tests covers where that line sits.

Make the next failure readable

Wrap the parts of a long test in test.step(). Steps show in the HTML report and the trace viewer, so you see whether the same step breaks every time.

What never to do

• Sleep. page.waitForTimeout() is for debugging: “Tests that wait for time are inherently flaky,” say its docs. Waiting for networkidle is marked as discouraged too.
• Raise timeouts everywhere. The defaults are 30 seconds per test and 5 per assertion. Higher global values slow every failure and let real slowdowns pass. For one known slow test, use test.slow(), which triples its timeout, with a comment.
• Use retries as the fix. Retries collect evidence. By default a pass on retry leaves the run green and the flake in place.
• Mute it. test.skip(), test.fixme(), a commented out test or a permanent --grep-invert stop it from running. Skipped tests are “not supposed to be ever run,” the docs say.
• Weaken it. Swapping toHaveText('Paid') for toBeVisible(), adding a try/catch or forcing a click passes by checking less.

A quarantine policy that keeps flaky tests visible

When a fix will take more than a day, quarantine the test, Fowler’s term for keeping it apart from the healthy suite until it is fixed. Keep it running and owned:

1. Tag it, do not skip it. Add a @quarantine tag and an annotation that links the issue. The HTML report shows annotations, so the link travels with each result.
2. Run it in its own job. The blocking job leaves it out. A second job runs only quarantined tests, may fail, and publishes a report.
3. Give it an owner and a deadline. Fowler suggests capping quarantine by count (his example is 8 tests) or by time (no longer than a week). Use both, and review the list weekly.
4. Prove the fix. Require zero failures in --repeat-each=100 --retries=0. A test that failed one run in 20 passes 100 in a row less than 1% of the time (0.95 to the power of 100 is about 0.006). At one in 100 it still passes about 37% of the time, so rarer flakes need more runs.
5. At the deadline, fix it or delete it on purpose, and list the lost coverage under the risks in your Go/No-Go checklist.

Steps 1 and 2 in code:

// Bad: muted. It never runs again, and nobody owns it
test.skip('pays with a saved card', async ({ page }) => {
  /* ... */
});

// Good: still runs in its own job, tagged and linked to an issue
test(
  'pays with a saved card',
  {
    tag: '@quarantine',
    annotation: {
      type: 'issue',
      description: 'https://github.com/your-org/app/issues/412',
    },
  },
  async ({ page }) => {
    // ...
  },
);

# Blocking job: everything except quarantined tests
npx playwright test --grep-invert @quarantine

# Separate job, allowed to fail: only quarantined tests
npx playwright test --grep @quarantine

How we handle flaky tests

When we maintain a Playwright suite, a Claude agent reruns each flaky test with retries off, collects the evidence for each failure and proposes a fix for the cause as a pull request in your repository. A QA engineer looks at every failure before it becomes a bug report or a test fix and reviews every change before it is merged, and an automatic check rejects any fix that would weaken what a test asserts. Whatever is still open goes into the Go/No-Go report as a risk, not into a skipped test. Our open-source sample project shows the flaky hunt in code, and the Playwright testing page describes the service.

Sources

• Retries, Trace viewer and Command line, Playwright docs
• Assertions, Actionability checks and Best practices, Playwright docs
• Parallelism and Continuous integration, Playwright docs
• Test configuration and release notes, Playwright docs
• Eradicating nondeterminism in tests, Martin Fowler, martinfowler.com
• Test flakiness: one of the main challenges of automated testing, Google Testing Blog

Researched and drafted with Claude, Anthropic's AI model, and checked against the sources it links. Spotted an error? Tell us and we will correct it.