# Regression testing for AI-generated code: a practical guide

> When a coding agent writes most of the diff, code review alone cannot carry the job of catching regressions. What to add beside it, and how to set it up without a QA team.

Published 2026-10-08, updated 2026-10-09, about FlowQA.

Many teams that adopted coding agents noticed the same thing soon after: the volume of change went up and the number of people who had read each change closely went down.
That is not a complaint about the models.
A person who wrote a change by hand carried a mental model of what it touched; a person who accepted a change from an agent often did not.
Review still matters: it is where a team checks the intent of a change, its design and the parts that are risky to get wrong.
What review was never good at is noticing a behaviour that broke three pages away from the diff, and with agents writing the diff, that gap gets wider.
Tests that drive the product complement the review: the review checks what the change means to do, and the tests check that everything else still works.

## What a regression is, in this world

A regression is a behaviour that worked yesterday and does not work today.
It is rarely in the file the agent edited.
It is in the checkout page that depended on a helper the agent "cleaned up", in the form whose validation moved, in the redirect that now goes one hop further.
Unit tests on the changed function do not see any of this.
The only test that sees it is the one that drives the product the way a person does.

## The three properties a regression test needs

**It has to run on a fresh session.**
A test recorded while you were signed in with your own account, with items already in a basket, passes on your machine and nowhere else.
Every assertion has to hold on a browser that has never seen the site.

**It has to be able to fail.**
A test that opens a page and checks the URL will pass while the page shows an error.
Before you trust a test, break the page the way the test claims to protect it, and watch the test fail.
If it does not, it protects nothing.

**It has to run without a person.**
On deploy, on a schedule, on demand from a pipeline.
A test that is only run when someone remembers is a test that is run on the day after the regression shipped.

## Where the tests come from

Three sources, in the order most teams end up with them:

1. **Recording.**
   Click through the flow once in a real browser and keep the steps.
   Fast, and good for the flows you know matter.
   The weakness is coverage: you only record what you thought of.
2. **Exploration.**
   An agent maps the site, proposes journeys with a goal and a success condition each, and generates a test for every journey you approve.
   The weakness is judgement: it will propose journeys you do not care about, and it needs the two checks above to stop it generating tests that cannot fail.
3. **Bugs.**
   Every bug that reached a person is a regression test waiting to be written: the steps that led there are the test, and the broken behaviour is the assertion.
   Teams that turn bugs into tests stop fixing the same thing twice.

## A setup that works for a small team

- Point FlowQA's AI agent at staging.
  Review the plan it proposes: drop the journeys that would place real orders or send real mail, keep the ones that cover money, sign-in and the main path.
- Accept only tests that passed twice on fresh sessions and failed on a broken page, and read what each one asserts.
  Failing on a broken page shows a test can fail; it does not show the test checks the outcome you care about.
  Some of what the agent proposes will be rejected; that is the point of the gate, not a sign that it is failing.
- Run the suite from your deploy pipeline with a token, and nightly on a schedule.
  Alert the channel where the people who can fix things already are.
- When a test fails on a changed page, decide whether the page changed on purpose.
  If it did, let the agent propose the repair and approve it; if it did not, you have found the regression before a customer did.
- File what slipped through as a bug with the console and the network attached, and turn the bug into a test.
- Keep reviewing the changes themselves.
  The tests tell you that the product still works; the review tells you that the change is the one you wanted.

None of this needs a QA team.
It needs a place where the tests live, a way to run them without a person, and the discipline to reject tests that cannot fail.
