QA for startups: a checklist by stage, and who should do the testing

QA for startups comes down to four habits, not a department: know which user flows must never break, test those flows automatically on every change, have someone other than the author try each new feature, and decide each release on evidence. What changes as the company grows is how much of each habit you need, and who does the work.

Our short answer on who: developers test their own work until customers depend on the product. Once you release every week, someone has to own QA: a first hire, a managed service, or AI tools that an engineer runs. Freelancers and agencies fit work with a clear scope and an end date.

A QA checklist for startups, stage by stage

A startup QA process grows in stages, and each stage keeps everything from the one before. Adopting a later stage’s habits early only costs time.

Before the first paying customer

The goal: the main path through your product works, and you hear about it when it breaks before a prospect does.

• Write down the three to five flows that must never break, such as sign-up, sign-in, the core action of your product and payment.
• Automate one end-to-end smoke test per flow and run it on every pull request. npm init playwright@latest offers to add a GitHub Actions workflow that runs the tests on each push and pull request to your main branch.
• Keep a staging or preview environment with test data, so nobody tests on production.
• Use your payment provider’s test mode and a test inbox, so smoke tests can run a whole flow.
• Turn on error tracking and an uptime check in production.
• Back up the database and test a restore.
• Keep one list of known bugs, each with steps to reproduce.

Change the labels and the heading in this smoke test to match your app:

import { test, expect } from '@playwright/test';

// One smoke test per flow that must never break.
// Set baseURL in playwright.config.ts so goto() can take a path.
test('a visitor can sign up and reach the dashboard', async ({ page }) => {
  await page.goto('/signup');
  await page.getByLabel('Email').fill(`smoke+${Date.now()}@example.com`);
  await page.getByLabel('Password').fill('a long throwaway password');
  await page.getByRole('button', { name: 'Create account' }).click();
  await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
});

Your first customers

The goal now: nothing that worked yesterday breaks unnoticed, and no customer finds the same bug twice.

• Rank your flows by how likely they are to break and how much a failure hurts, and cover the top first. Our risk-based testing guide has a matrix.
• Every bug a customer reports gets a test that reproduces it, merged with the fix.
• Add API tests for the endpoints behind your critical flows. Without a browser, they are cheap to run on every change.
• Test with realistic accounts: each role and permission, an empty account, a large one, and a slow mobile connection.
• Agree on one severity scale: which bugs block a release and which can wait.
• Know how to roll back a release.
• Ask “how was this tested?” on every pull request. On GitHub, a .github/pull_request_template.md file on the default branch fills in every new pull request:

## What changed

## How was this tested?

- [ ] Automated tests added or updated: <link>
- [ ] Tried by hand on: web / iOS / Android / API
- [ ] New feature tried by someone other than the author

## Risk

- Flows this touches:
- How to roll back:

Weekly releases with a few engineers

The goal: every release decision rests on evidence, not on whoever merged last.

• Make the test job a required status check. On GitHub, a protected branch with required status checks blocks merges until the checks pass; admins can bypass it unless you turn that off.
• Fix flaky tests, or quarantine them with an owner and a date. A test that fails at random teaches the team to ignore red builds.
• Give every new feature a session of exploratory testing by someone who did not build it.
• Hold a short Go/No-Go review before each release, against criteria written in advance. Our Go/No-Go release checklist has a template.
• Count the bugs customers find after each release. If that number grows, testing is falling behind.
• Name one person who owns QA, even part time.

A growing team

The goal: quality survives more people, more code and more releases.

• Write a short test strategy: which risks the unit, API and end-to-end tests each cover, who owns each suite, and the release criteria every team applies.
• Each team owns the tests for its features, and a change that breaks another team’s tests does not merge.
• Keep the suite fast enough for every pull request: run it in parallel and push checks down to API or unit tests.
• Keep test data seeded, resettable and away from real customers.
• Plan specialist load, security and accessibility testing before you need it.
• Track the time from a red build to a decision, and how many tests are flaky.

QA for startups: who should do the testing

Whoever does the work, the checklist stays the same; the cost, feedback speed and accountability differ.

Option	Good at	Signs you have outgrown it
Developers test their own work	Speed before you have customers	Customers find bugs before you do
Your first QA hire	Deep product knowledge	One person gates every release
A freelancer	A defined, short project	You need someone every week
A QA agency or outsourcing firm	Capacity and specialist testing	Results arrive after the release
Managed QA with AI agents and a reviewer	The whole job without a hire	Each team wants its own QA engineer
AI testing tools you run yourself	Writing tests faster	Nobody has time to review the output

Developers test their own work

• Cost: no invoice, but every testing hour is a product hour lost, and a developer tests the cases they thought of while coding, the ones least likely to be broken.
• Good at: fast feedback, and the right choice before the first paying customer. Unit tests stay with developers at every stage.
• Outgrown when: customers report bugs before you find them, releases wait because nobody has time to test, or nobody checks how two features behave together.

Your first QA hire

• Cost: the U.S. Bureau of Labor Statistics puts the median annual wage for software quality assurance analysts and testers at $104,300 in May 2025. The lowest 10 percent earned less than $61,440 and the highest 10 percent more than $167,010. These are US wages only: the survey behind them leaves out employer costs of benefits such as health insurance and retirement contributions, and recruiting and tools come on top. We break those costs down in what a QA engineer costs. Add the time to recruit and onboard.
• Good at: knowing the product deeply. The same handbook lists the core duties as creating test plans, identifying project risks, running manual, automated and exploratory tests, and reporting defects.
• Outgrown when: one person gates every release, the automation backlog grows while they test by hand, or a vacation pauses your releases.

A freelancer

• Cost: billed by the hour or by the project. We have no primary source for freelance QA rates, so we give no figure: compare quotes for the same written scope, and budget time to explain your product.
• Good at: a defined project with an end, such as a first test suite, a test pass before a big launch, or a sweep across devices.
• Outgrown when: you need someone every week, or the tests decay after the contract ends and what the freelancer learned leaves with them. Keep the tests in your repository from the first day.

A QA agency or outsourcing firm

• Cost: a contract. Compare billing model, minimum term and notice period, not only the rate, and allow time for the firm to learn your product.
• Good at: capacity you can scale up and down, and specialist work such as load, security or accessibility testing, if the firm offers it.
• Outgrown when: results arrive after the release, bug reports cannot be reproduced, or the tests live in a tool you cannot take with you.

QA outsourcing for startups works best when the scope is written down and the tests stay yours. Before you outsource QA testing, ask:

1. Where do the tests live, and who owns them if we stop?
2. Do they run in our CI on every pull request, or on the firm’s schedule?
3. Who looks at a failure before it becomes a bug report?
4. How many hours overlap with our working day?
5. What access do you need, and can we remove it at any time?

Managed QA with AI agents and a human reviewer

This is what we do, so weigh it accordingly. Claude agents plan, write, run and maintain the tests. A human QA engineer reviews every change, looks at every failure before it becomes a bug report or a test fix, and signs a Go or No-Go call before each release; shipping stays your decision. The tests are plain Playwright, Appium, Maestro or Mobilewright code in your repository, run in your CI on every pull request.

• Cost: one flat monthly price for a set number of user flows, shown on our pricing page, with no fees per run. In week 1 we set up access, rank your flows by risk with you and aim to have the first tests running in your CI.
• Good at: covering web, API and mobile without a hire. Agents propose fixes to broken tests and a person reviews each one, and an automatic check rejects any fix that would weaken what a test asserts. Billing is month-to-month, and you keep every test if you stop.
• Outgrown when: each product team wants a QA engineer in its daily planning; then build a QA team of your own.

It is also the wrong choice in these cases:

• You need a certified audit or specialist testing. Penetration testing, load testing and formal accessibility audits are not part of our standard service. If a regulator or an auditor requires an accredited tester, hire one.
• You need a person on site, for hardware or a network with no outside access. We work remotely.
• Your policy does not allow code to be processed by an outside AI model. Our agents run on Claude through the Anthropic API.
• You have no paying customers yet. Developers and a few smoke tests are enough at that stage.

AI testing tools you run yourself

Playwright, which is open source under the Apache 2.0 license, has shipped three test agents since version 1.56: a planner that explores the app and writes a Markdown test plan, a generator that turns the plan into test files, and a healer that runs the suite and repairs failing tests. npx playwright init-agents --loop=claude sets them up for Claude Code, and the docs list other agent loops. As of October 2026 the newest release is 1.64.

• Cost: Playwright is free; the AI model or coding agent you pair it with has its own cost. The larger cost is engineering time to decide what to test and to review everything the agents write.
• Good at: writing tests faster, for a team that already knows what to test and owns the suite.
• Outgrown when: generated tests merge without review, red builds get healed instead of investigated, or nobody owns the release call.

Per the Playwright docs, the healer reruns a failing test with suggested patches until it passes or its guardrails stop the loop, and skips the test when it believes the feature is broken. A patch that asserts less turns a real bug into a green build, so read every patch and every skip.

When to hire a QA engineer, and when to outsource

Decide by signals, not headcount. Someone must own QA when:

• Customers keep finding bugs before you do.
• Developers spend a day or more testing by hand before each release.
• You release every week, and a bug has already delayed a release or forced a rollback.
• You ship on more than one platform, such as web plus iOS and Android, or a public API.

Hire when QA needs a seat in daily planning or you are building a QA team anyway. Use a freelancer or an agency for scoped projects and specialist testing, managed QA for the whole job every week without a hire, and AI tools when an engineer owns testing and has time to review their output.

How we do it

We run the managed option above: Claude agents, built with the Claude Agent SDK, do the repetitive test work in your repository and CI, and a QA engineer reviews every change and failure, sends a weekly digest in your chat, works in your time zone and signs a Go or No-Go report before each release. How it works shows a week of the service.

Sources

• Software Developers, Quality Assurance Analysts, and Testers, Occupational Outlook Handbook, U.S. Bureau of Labor Statistics
• Frequently Asked Questions, Occupational Employment and Wage Statistics, U.S. Bureau of Labor Statistics
• Playwright Test Agents, Playwright docs
• Release notes, Playwright docs
• Setting up CI, Playwright docs
• Playwright license, Playwright on GitHub
• Creating a pull request template for your repository, GitHub Docs
• About protected branches, GitHub Docs

Researched and drafted with Claude, Anthropic's AI model, and checked against the sources it links. Spotted an error? Tell us and we will correct it.