Flaky tests in Playwright: causes, fixes and a quarantine policy
A field guide to flaky tests in Playwright: what causes them, how to find them with repeats, retries and traces, how to fix each cause, and how to quarantine safely.
Web testing
Playwright testing for your web app, done for you
We write, run and maintain Playwright + TypeScript end-to-end tests for your web app. Claude agents do the volume, a QA engineer reviews every change, and the tests live in your repository and run in your CI on every pull request.
What we test in your web app
- The user flows that would hurt most if they broke, end-to-end: sign-up, login, checkout, billing, the core job of your product.
- Chromium, Firefox and WebKit, and phone screen sizes where your users need them.
- Roles and permissions: what each kind of user can and cannot do.
- Error states your users will meet: failed payments, expired sessions, empty and invalid input.
- New features, hands-on, by a person, before they ship.
Playwright, in its own words: a test runner with auto waiting, assertions, tracing and parallelism across Chromium, Firefox and WebKit, where each test gets a fresh browser context (playwright.dev). We write the tests to use those strengths.
Tests your team can read
Locators that mirror how people see the page (a button by its name, a field by its label), assertions that wait for the page instead of fixed sleeps, and test data set up through your API so every test starts clean. A test that fails says what broke, with a trace to show where.
When the product changes, the agents propose updates as pull requests. An automatic check refuses any fix that would weaken what a test asserts, and a person reviews every change before it is merged.
test('checkout with a saved card', async ({ page }) => {
await page.goto('/cart');
await page.getByRole('button', { name: 'Pay now' }).click();
await expect(page.getByText('Order confirmed')).toBeVisible();
});Who does what
The agents write and run, a person reviews and decides
Claude agents
- Draft test plans from your specs, tickets and code
- Write new tests and update tests the product has outgrown
- Run the suites and collect evidence for each failure
- Propose fixes for broken tests
The QA engineer
- Sets the strategy and the priorities with you
- Reviews every change before it is merged
- Confirms each bug and its severity
- Signs the Go or No-Go call before each release
In your CI, on every pull request
The suite runs in the CI you already have, such as GitHub Actions or GitLab CI, on every pull request, with no fees per run. A person looks at every failure before it becomes a bug report or a test fix, and before each release you get a signed Go or No-Go report: what was tested, what failed and what risk is left. Shipping stays your decision.
The guides on running Playwright in GitHub Actions and the Go/No-Go checklist show how that works.
What it costs
US$2,000 a month for up to 500 user flows, across web, API and mobile. Each platform counts on its own, so checkout on web and on iOS is two flows. An API journey counts as one flow. Test runs, AI credits and hours are never billed. New clients start with a 60-day pilot at half price. The pricing page has the calculator and every term.
Where do the tests live?
In your repository, as plain Playwright + TypeScript code, and they run in your own CI. They stay with you if the service ends, and your developers can read and change them like any other code.
Which browsers do you cover?
Chromium, Firefox and WebKit, the three browser engines Playwright drives, and phone screen sizes where your users need them. We agree the list in the test plan, so the suite runs what matters and stays fast.
Can you take over our existing Playwright suite?
Yes. We start by running it and triaging every failure, then fix the flaky tests and fill the gaps in your most critical flows. Nothing is thrown away unless it tests the wrong thing.
Do you need access to production?
No. Tests run against a preview or staging environment with test accounts. Secrets stay in your CI secrets or your vault, never in test code. See security and data handling.
How do you handle flaky tests?
A flaky test is traced to its cause and fixed, not muted or retried until it passes. The guide to flaky tests in Playwright shows the method.
Guides
Guides for Playwright teams
Playwright GitHub Actions setup: a fast CI pipeline you can trust
A complete Playwright GitHub Actions workflow to copy: sharding, merged HTML reports, retries with traces, secrets for test accounts and what not to cache.
Agentic testing with Playwright: how AI agents plan, write and repair tests
How AI agents do QA work with Playwright today: the loop, the tools (test agents, MCP, Agent SDK), where agents go wrong and the guardrails that work.
Get your critical flows tested on every pull request
One email is enough to start: your product, your platforms and how often you release. You get a plain answer on fit and, if it fits, a written proposal for the 60-day pilot with a short video of how we would test your product.
Or write to qa@releaseworthy.app. Still deciding after the proposal? We book a Google Meet call for your questions.