TestFinch

Docs

Repairs: when a test fails on a changed page

FlowQA proposes one small change, proves it twice, and waits for your approval.

When a test fails on a page that changed, FlowQA proposes a repair for approval (spec section 6, step 6). Nothing changes until a person approves it.

What happens

  1. From a failing test in the workbench, "Propose a repair" starts one for the test's latest failing run (POST /v1/projects/:p/qa/tests/:testId/repairs with the execution_id). The repair runs on a cloud worker or, from the desktop app, on the person's own computer.
  2. The engine (packages/flowqa-explorer/src/repair.mjs) replays the test to the failing step with FlowQA's own runner and holds the page there. It gives the strong model the test's steps, the failing step and its error, and what the page shows now: a summary and the interactive elements. The model answers with one small change: a new target, a new value, a corrected assertion, or a new path, or says that no repair fits.
  3. The repaired test is replayed twice on fresh sessions. Only a repair that replays green twice is proposed.
  4. The person sees the step before and after, with the model's reason, and approves or rejects. Approval publishes the test's next revision through the normal save path and charges 5 credits (spec 4.2); rejection leaves the test unchanged.

A failure inside a shared opening the test uses is referred to that opening's own test. Steps of kinds the engine does not repair (for example press) are declined with a reason. A test that moved on since the run (a newer revision) cannot seed a repair; run it again first.

Where it runs

Cloud workers claim queued repairs after explorations (v1/qa/worker/repairs/claim, then heartbeat, complete, model). On the desktop, the same child process that runs explorations runs repairs (@snag/flowqa-explorer/local-worker with kind: 'repair'), relaying its API and model calls through the signed-in account. With a customer's own model key connected, model calls go through the API in both cases (model_via_api).

Limits and billing

Free includes no repairs; a repair needs 5 credits on the workspace, which are charged only when a proposal is approved. Every model call is on the ledger with refType: repair, and the approval writes a repair row. Each repair carries a spend cap (default 1 USD) that stops the engine mid-way.