TestFinch

BlogFlowQA

Why your AI coding agent says "done" when the app is broken

A coding agent says done from what it can read, the code and the test output, and it never opened the page. Four ways to let Claude Code, Cursor or Codex check a change in a browser, what each catches and misses, and a loop to give the agent.

A coding agent such as Claude Code, Cursor or Codex says "done" when the code it wrote compiles and the checks it can run have passed. Unless you gave it a browser, the agent never opened the app, so a button that does nothing or a page that renders blank is invisible to it. Give the agent an end-to-end test to run: a test that opens the running app in a browser, does what a person would do, and checks what the page shows.

Why does the agent say done when the page is broken?#

A coding agent works from text. It reads your files, the type checker's output, the linter's output and the results of the unit tests. Every one of those can pass while the page a person opens is broken.

What broke Why the agent saw nothing wrong
The Save button calls a handler that was renamed The handler's own unit test passes, and nothing clicked the button
The form sends company and the server now expects companyName The form and the server each pass their own tests, and nothing sent a request from one to the other
The page throws an error while it loads and stays blank Nothing loaded the page
Sign-in redirects to an address that no longer exists Nothing followed the redirect
A new banner covers the Pay button The code for the button is correct, and only a rendered page shows the overlap

What can a coding agent use to check its own work?#

Option What it catches What it misses Kept for the next change
Unit tests A wrong result from one function Anything between two parts, and anything on the page Yes
A browser the agent drives itself What is wrong today on the page it changed Every page it did not walk, and tomorrow's change No, unless the agent writes it down
Playwright test scripts the agent writes The same journeys, checked the same way on every run Whatever the scripts do not check, plus the upkeep is yours Yes, as code in your repository
A test service the agent calls The same journeys on every run, also when no agent session is open Whatever the tests do not check, and it is one more account Yes, in the service

Unit tests#

A unit test calls one function with known input and checks the output. A unit test cannot tell you that the checkout page works, because no unit test opens the checkout page.

A browser automation MCP server#

MCP, the Model Context Protocol, is how a coding agent is given tools beyond its editor. A browser automation MCP server gives the agent tools to open a page, click, type and read what the page shows. One is Playwright MCP, which its repository describes, as of 18 November 2026, as "a Model Context Protocol (MCP) server that provides browser automation capabilities using Playwright", under the Apache-2.0 license.

With a browser of its own, the agent can open the page it has changed and see that the button does nothing. For looking at a page while building, a browser the agent drives is the best of the four options. A browser the agent drives leaves no test behind. Tomorrow's change gets no check unless you ask for one, and the agent chooses its clicks afresh each time, so two walks are not the same test. Playwright MCP for testing covers when to use which.

Playwright scripts the agent writes#

Playwright's documentation, read on 18 November 2026, calls Playwright Test "an end-to-end test framework for modern web apps". You install it with npm init playwright@latest and run the tests with npx playwright test. An agent can write these scripts, run them and read the failures, and the scripts live in your repository beside the code.

Pick this option when you want tests as code and someone on the team will own them. The scripts are code like any other: selectors, waits, the sign-in setup, the pipeline that runs them and the place the screenshots go are yours to maintain. An agent asked to make a failing script pass can also do it by weakening the check, so read every change to a test file the way you read a change to the product.

A test service the agent calls#

A test service keeps the tests and runs them on its own browsers. The agent saves a test, starts a run and reads the result through an MCP server or a command-line tool. The tests also run when no agent session is open: on a schedule, or after every deploy. The cost is another account, a price past the free plan, and tests that are records in the service and not code in your repository.

A loop to give the agent#

  1. The agent changes the code.
  2. The agent writes or updates an end-to-end test for the journey the change touches. The test ends by checking an outcome a person would see, such as the order number on the confirmation page.
  3. The agent runs the test in a browser.
  4. When the test fails, the agent reads the failing step, the error and the screenshot of the page at that step.
  5. The agent decides which is wrong, the product or the test, fixes that one, and runs the test again.
  6. The agent reports done with the passing run attached.

Have the agent tell you whenever it changes a check, and why. Put what done means in your agent's instructions, for example: "A change to a page is done when its end-to-end test has run and passed. Name the test and link the run."

A worked example#

A composed example: the app and the steps show the shape of the work and are not taken from one customer.

You ask the agent to add a company field to the sign-up form. The agent adds the field, the unit tests pass, and it reports done.

The project has a test named auth/sign-up: open the sign-up page, fill the name, the email and the password, press Create account, and check that the page shows "Welcome". The agent runs it. The run fails at the last step, and the screenshot shows the sign-up form still on screen with "Company is required" under the new field.

The agent made the new field required, and you asked for an optional one. The test is right and the product is wrong. The agent makes the field optional, runs auth/sign-up again, and it passes. Without the run, the first person to sign up without a company would have met that error.

How to set this up with FlowQA#

FlowQA by TestFinch is a test service of the fourth kind. Your coding agent writes tests through the TestFinch MCP server, runs them on a cloud browser, and reads the failing step and its screenshot.

  1. Start free at TestFinch with your site's address.

  2. In TestFinch, open Agents and create a token.

  3. Add the server to your agent with the token. For Claude Code, in your repository:

    claude mcp add --transport http testfinch https://api.testfinch.com/mcp --header "Authorization: Bearer <token>"
    

    The Cursor and Codex versions are in the setup walk-through and in the coding agents doc.

  4. Start a new session and ask the agent to read the flowqa_guide tool.

  5. Ask for a test. The agent saves it with flowqa_save_tests and verify: true, which runs the saved test once on a cloud browser and answers with the verdict: passed or failed, the failing step and its error, the failing screenshot's path and a link to the run. flowqa_get_artifact returns that screenshot.

A cloud browser cannot reach an app that runs only on your computer. For that, the agent uses the TestFinch CLI: testfinch test http://localhost:3000 runs the project's tests on your computer and writes the results and the failing steps' screenshots to files the agent can open.

Every test the agent saves is checked, and the save comes back with findings such as a test with no check, a typed password or a name another test already has. The findings do not show that a test checks the right outcome, so read what each test checks.

What FlowQA does not do#

The TestFinch connector does not hand your agent a live page to click around in. Its tools save, run and read tests, so keep a browser MCP server for looking at a page while building. FlowQA tests web apps in Chromium, at desktop sizes and at emulated phone and tablet sizes. Runs your agent starts on cloud browsers count against your plan exactly as if you started them. The free plan covers 1 project, up to 100 tests and 1,000 cloud test runs a month, and runs on your own computer through the CLI never count against your plan. When a test fails because the page changed on purpose, FlowQA proposes one small repair with its evidence and waits for you to approve it.

FlowQA explores your site, writes the tests, keeps each one only once it fails on a page broken the way its checks describe, and runs them after every change.

See FlowQA