# When the AI agent asks a person

> Which element, what value, may it follow a host, is it done: how questions and answers travel.

The AI agent runs on its own, but a site has things no map can tell it: a promo code that only the team knows, a link that leaves the site on purpose, a page where two models disagree about whether the goal is reached.
Instead of giving up, the AI agent files a question, parks that journey and carries on with the others.
The person answers from the workbench; the journey resumes where it stopped, and what was answered is kept so the next exploration does not ask again.

## What it asks

| Kind | When | What the person gives |
| --- | --- | --- |
| `what_value` | A form needs a variable nobody provided (`{{promo_code}}`). | The value, whether it is secret, and whether to save it as a project variable for this environment. |
| `which_one` | The AI agent keeps landing on the same page state, or the strong model found nothing to do. | One of the offered elements, or a sentence that steers every later step of the journey, or "give this journey up". |
| `may_i` | An action tried to leave the allowed hosts. | Allow the host (for this run, and for the environment from now on) or stay on the site. |
| `am_i_there` | The decision model says the goal is reached and the reviewing model has disagreed twice. | Yes (the journey ends as done, confirmed by a person) or no, with an optional note. |

Every question carries a screenshot of the page the AI agent is on.
Before it is taken, input values and text that contain any of the project's secret values are blanked, and the values are put back after.
The screenshot is a JPEG small enough to travel on a heartbeat; it is deleted once the question is answered or closed.

Limits keep it from flooding anyone: two questions per journey, ten per exploration.
Past the limit the AI agent does what it did before: refuses the form, avoids the element, ends the journey as stuck.

## How it travels

The engine keeps a desk of questions (`packages/flowqa-explorer/src/questions.mjs`).
A journey that asks calls `ask()`; the engine takes the screenshot, files the question and tells the scheduler, which starts the next journey while this one waits.
The desk's unsent questions ride on the next heartbeat to the API, which stores them (`QaExplorationQuestion`) and returns, on every heartbeat, the answers given so far.
The desk resolves the waiting journey with its answer.

A heartbeat that failed sends its questions again.
When every journey has run or parked, the engine waits up to thirty minutes for the rest of the answers, then expires the desk: parked journeys get `null` and finish the way they would have alone.
Cancelling the exploration expires the desk at once.

The person's side:

- `GET /v1/projects/:p/qa/explorations/:id` lists `questions` and counts `open_questions`; the exploration list carries `open_questions` per row.
- `GET .../questions/:qid/screenshot` returns the image as `{ data, content_type }`.
- `POST .../questions/:qid/answer` takes `choice`, `value` and `secret`, `allow`, `yes`, `note`, `give_up`, and `remember` (default true); the API checks the fields against the question's kind.
- `GET /v1/projects/:p/qa/questions` lists questions across the project's explorations, newest first, at most 100: open ones by default, or `status=answered`, `expired` or `all`.
  Each carries `exploration_id`, `environment` and `exploration_status` beside the question's own fields.

Open questions expire when the exploration ends, whichever way it ends.

## What is kept

With `remember` on (the default):

- a `what_value` answer becomes a project variable for the environment, secret unless the person said otherwise;
- an allowed host joins the environment's allowed hosts in the project configuration;
- a note on a `which_one` or `am_i_there` answer lands under "From the team" on the page's site note (or the README when the page has no note), where the AI agent reads it on every later visit.

If a value cannot be stored (for example the server has no `FLOWQA_INPUT_KEY`), the answer still reaches the engine and the response says why it was not kept.

## In the workbench

The exploration dialog shows open questions above the progress, newest journeys first, each with its screenshot and the controls for its kind.
An answer being typed survives the dialog's polling.
While the dialog is closed, the Explore button carries a count of open questions and a new question raises a toast with a way into the dialog.
Answered questions stay in the dialog as one line each.
An answer a coding agent gave says so under its line: "Answered by Claude Code for Chitrarth".
The app's link to an exploration, `/p/:p/tests?exploration=<id>`, opens the project's Tests with that exploration's dialog open, once; the address then drops the parameter.
A link to an exploration the project does not have opens nothing and says so.

## Answering from a connector

A coding agent connected to TestFinch (see [Coding agents](/docs/coding-agents/)) is one more participant in an exploration.
It reads the open questions with `flowqa_list_questions`, looks at the page with `flowqa_get_question_screenshot`, which returns the screenshot as an image, and answers with `flowqa_answer_question`.
The answer is the same route with the same fields, except the take-over's `steps` and `done`, which only FlowQA Desktop sends.

Every question, in the project list and in `GET .../explorations/:id`, carries two links for the person, and so does the exploration itself:

- `person_link` opens the question where the person answers it: for an exploration on a cloud browser, the project's Tests in the app with the exploration's dialog open (`/p/:p/tests?exploration=<id>`); for an exploration on the person's own computer, FlowQA Desktop (`flowqa://open?project=<p>&action=explore&ids=<id>`), which runs it and offers to take over the browser.
- `web_link` is always the app's link.

`context.takeover_available` is true when the person can take over the browser for that question now: the question is open and asks which element or whether the goal is reached, the exploration runs on their desktop, and the desktop holds its lease.

The agent answers what the code or the screenshot tells it and hands the rest to the person with `person_link`, including a question with `takeover_available` when the person would rather drive.
When the person answers, the question leaves the open list, and the agent carries on.

An answer given through the connector records the agent beside the person: `answered_agent` holds the client's name and version (`claude-code/2.1.0`), or `mcp` for a client that did not announce itself, and `answered_by_name` the person it answered for.
The note it leaves under "From the team" says the same: ``- 2026-10-10 on `/`, answered by Claude Code for Chitrarth: Mugs is the main category.``

A `what_value` answer is secret unless the agent sends `secret: false`.
With `remember`, the answer's `remembered` says how the value was kept: `variable` names the project variable and `secret` says whether it is secret.
A secret answer for a variable that exists as a plain one makes the variable secret: its plain values, in every environment, move to the encrypted store with it, so a secret is never kept in plain form.
A variable that is already secret stays secret, whatever the answer says, and the question shows that answer masked too.
A value or host that could not be kept (a storage error) leaves the answer standing and says why in `remembered.problem`.
Of two answers to one question at once, one is kept and the other answers 409, so its side effects (a variable, an allowed host, a note) apply once.
A secret value never comes back: the engine receives it on its heartbeat, and every read of the question, the answer's own response included, shows `••••••••` in its place; the tool also blanks the value anywhere else in the response, an error included.
A question answered already, or whose exploration ended, answers 409, and the tool returns `closed: true` with a pointer to the open ones.
A screenshot is deleted once its question is answered or closed; asking for it after that answers 404, and the agent is told to decide from the question's text and context.

Any read-and-write credential may start an exploration: a session, an OAuth access token or an API token, through the connector (`flowqa_explore`) or not.
It is billed as if the person started it, with the same refusals when the plan has none left, and a read-only token is refused.
The connector's `X-Snag-Via` header records where a call came from; it is provenance, not proof, and gates nothing.
The tool takes `environment`, `goal`, `start_url` (a path, or a URL on the environment's base URL or its allowed hosts; anything else is refused before the API is called), `opening`, `target` and `budget`.
`flowqa_get_exploration` follows it, with `done` once it completed, failed or was cancelled, and, while the plan waits in `review`, the proposed journeys and a pointer to `person_link`.
Each proposed journey's name, goal, success condition and start page were written from the site, so they come wrapped as untrusted data (`{ untrusted: true, body }`).
While it runs, `progress.current` and each progress journey's `name` come from the site too, and are wrapped the same way; the browser's frame, a journey's live page and its step lines never reach an agent.
`flowqa_review_plan` approves a plan under review, as Generate selected in the app does, and only when the person asked the agent to: `journeys` lists the ids to keep, and without it the plan's own selection is kept.
A journey the plan skipped, such as one that would pay or delete while destructive actions are off, is refused before the API is called, and an exploration that is not in review is refused with what to do instead.
The reply names the kept and dropped journeys, the credits estimate for the kept ones under the allowance the exploration started with, and the next step.
The API applies the rest, for the workbench and the agent alike: two approvals, or an approval and a cancel, that race leave one winner and refuse the other with 409.
An edited name, goal or success condition that would pay, delete or change the account is refused unless the exploration allows destructive actions, by the same rule the planner uses (`isUnaskedOutwardName` in `@snag/flowqa-contracts`); a spare left unticked is checked too, since the engine can run it as a replacement.
A read-only environment allows nothing destructive at approval, whatever the exploration started with: the journeys the planner kept only because destructive actions were allowed are dropped, and generation runs with destructive actions off.
When the kept tests will be charged and the workspace has no credits for one, the approval is refused and the plan stays in review.
`flowqa_cancel_exploration` cancels one that has not ended, as Cancel exploration in the app does: the engine is told on its next heartbeat, the open questions expire, and the phase it was in publishes nothing.
An exploration that already completed, failed or was cancelled comes back as it is, with `cancelled: false` unless it was cancelled; a read-only token does not see the tool, and the API refuses a read-only credential too.

## Runs on the desktop too

A local exploration goes through the same desk: the desktop relays the heartbeat to the API, so questions and answers travel exactly as they do from a cloud worker.

## Taking over the browser (desktop)

On the desktop, a "which element" or "is it done" question offers one more answer: take over the browser.
The exploration child process reports the page the question was asked on, its URL and its cookies and local storage.
The desktop opens the recorder there, signed in as the AI agent is, through the recorder's own auth-state setup, and the person does the steps themselves.
"Hand back" ends the recording and answers the question with the recorded steps, either to keep exploring or with "goal reached".
The AI agent replays those steps on its own page (navigate, tap, fill, select, press, waitFor, back, forward, reload; anything else is dropped), records them in the journey as its own, and carries on, or ends the journey as done and writes the assertions.
A step that cannot be replayed ends the journey as stuck with the reason, and nothing of the hand-back is recorded.

The route is the recorder's: `POST /api/record-edit/takeover` with `{ exploration, question }` opens it, `POST /api/record-edit/hand-back` with `{ session, done }` answers.
On the web both say that the desktop app is needed.
The API side is the ordinary answer route with `steps` (valid FlowQA steps, at most 80) and `done`.

## Before any question: a site behind a gate

A preview password or a sign-in that stands in front of every page is not something the AI agent asks about mid-journey.
The map phase notices it: when the crawl collapses to one page with a password field, the exploration stops with a message naming the page and what it asks for.
The remedy is an opening: record the steps that pass the gate once, keep the password in a secret variable, and choose that opening under Opening when starting the exploration; the map then starts beyond it.
If an opening was chosen and the gate still shows, the message says so and points at the opening's steps and the variable values for that environment.
Nothing is written to the site notes from a map that stopped at a gate.
