# Schedules and CI for FlowQA

> Run tests on a schedule or from your deploy pipeline, and get told when they fail.

FlowQA runs your recorded tests without anyone at the keyboard, on cloud browsers or, from a CI job, on the CI machine.
Two ways to start a run unattended, both set up in the project's **Settings, then Schedules & CI**:

- A **schedule** runs the project's tests at set times (nightly, hourly, weekdays at 9).
- A **CI trigger** gives a pipeline a token that starts a run and reads the verdict.

Both run as the person who created them, with that person's access, and tell the people named on them when a run fails.

Deploying from GitHub?
The GitHub App does this with no workflow file: it runs the regression after every successful deployment and posts a check on the commit; see [the GitHub document](https://testfinch.com/docs/flowqa-github/).
It can also check every pull request against its preview deployment; see [Pull request checks](https://testfinch.com/docs/flowqa-github/#pull-request-checks).
The trigger below stays the way to run on other events, such as every push or a manual dispatch.

## Schedules

A schedule has a name, a cron expression evaluated in a timezone, an environment, the tests to run (every recorded test, or a chosen set), and alert settings.
The runner checks every minute.
If the previous run of a schedule is still queued or running when the next slot arrives, that slot is skipped and the schedule says so; runs never stack up behind each other.

Cron is five fields: minute, hour, day of month, month, day of week.

| Want | Cron | Timezone |
|---|---|---|
| Every night at 03:00 in Kolkata | `0 3 * * *` | `Asia/Kolkata` |
| Weekdays at 09:00 UTC | `0 9 * * 1-5` | `UTC` |
| Every hour | `0 * * * *` | any |
| Monday 07:30 in Berlin | `30 7 * * 1` | `Europe/Berlin` |

## CI trigger

Create a trigger in Schedules & CI.
Its token is shown once; store it as a secret in your CI system (for example `TESTFINCH_TRIGGER_TOKEN`).
The token is the only credential the pipeline needs.

### With the TestFinch CLI

The [TestFinch CLI](https://testfinch.com/docs/testfinch-cli/) starts the run, waits for the verdict and exits with a code your pipeline can branch on: 0 when every test passed, 1 when a test failed or was blocked.
Set the trigger's token as `TESTFINCH_TRIGGER_TOKEN`.

On the CI machine, against a server the job starts:

```yaml
name: TestFinch
on: [pull_request]
jobs:
  testfinch:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22 }
      - run: npm ci
      - run: npx @testfinch/cli@0.1 browsers install --with-deps
      - run: npx @testfinch/cli@0.1 test http://localhost:3000 --local --start "npm run dev" --label "pr ${{ github.event.number }}"
        env:
          TESTFINCH_TRIGGER_TOKEN: ${{ secrets.TESTFINCH_TRIGGER_TOKEN }}
      - if: failure()
        uses: actions/upload-artifact@v4
        with: { name: testfinch-runs, path: .testfinch/runs }
```

On cloud browsers, against an address you deployed, with a personal access token:

```yaml
      - run: npx @testfinch/cli@0.1 test "${{ github.event.deployment_status.environment_url }}" --cloud --json > testfinch-result.json
        env:
          TESTFINCH_TOKEN: ${{ secrets.TESTFINCH_TOKEN }}
          TESTFINCH_PROJECT: prj_...
```

A trigger's token also works for a cloud run: `TESTFINCH_TRIGGER_TOKEN=... npx @testfinch/cli@0.1 test --cloud`.

### Run tests on the CI machine

A trigger runs on cloud browsers until you turn on **Run tests on the CI machine**.
Find it in Schedules & CI, under CI triggers: it is a checkbox when you create the trigger, and a **Run on the CI machine** button on a trigger that exists.
You can turn it off again at any time.

With it on, the CI machine receives each test value, secret ones included, to type them into your site.
Anyone who can read your CI logs or change your pipeline can read them too, so turn it on only for a pipeline you trust.
With it off, a run on the CI machine is refused and says where to turn it on.

A run on the CI machine uses Chromium only, and never counts against your plan.
When you give an address that is not the environment's own, the environment has to allow it.
Site passwords and sign-ins are sent only to the environment's own address, so a test that needs one stops and says so on any other address.

### With curl

Send the trigger token in the `Authorization` header, never in the address, so it stays out of logs.

Start a run:

```sh
curl -sS -X POST "https://api.testfinch.com/v1/qa/triggers/run" \
  -H "authorization: Bearer $TESTFINCH_TRIGGER_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"label":"deploy '"$GITHUB_SHA"'"}'
```

The response lists the executions started (one, unless the project has more than 100 tests):

```json
{ "project_id": "prj_...", "executions": [{ "execution_id": "qex_...", "status": "queued", "environment": "staging", "test_ids": ["smoke/home"] }] }
```

Optional body fields: `environment` (an environment id from the project; the trigger's own when omitted), `tests` (flow ids such as `smoke/home`; the trigger's selection when omitted), `label` (shown with the run in FlowQA).

Poll the verdict:

```sh
curl -sS "https://api.testfinch.com/v1/qa/triggers/executions/qex_..." \
  -H "authorization: Bearer $TESTFINCH_TRIGGER_TOKEN"
```

```json
{ "execution_id": "qex_...", "status": "completed", "done": true, "ok": true,
  "summary": { "total": 12, "passed": 12, "failed": 0, "could_not_run": 0, "flaky": 0 },
  "results": [{ "id": "smoke/home", "status": "pass", "duration_ms": 900, "could_not_run": false, "failed_step": null }] }
```

`done` is true once the run finished, was cancelled or expired.
`ok` is true only when the run completed and every test passed; fail the job on anything else.
`summary.could_not_run` counts the tests whose cloud browser was lost before they could finish, and each such result has `"could_not_run": true`.
They are not failures of your site, and they are not counted in `failed`, but they did not pass either: `ok` is false while any test could not run.
Check `ok`, never `failed == 0`, and run the job again to run them.
A run waits at most 30 minutes for a browser before it expires.

### GitHub Actions with curl

```yaml
name: FlowQA regression
on:
  deployment_status:
jobs:
  flowqa:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - name: Start the run
        id: start
        run: |
          body=$(curl -sS -X POST "https://api.testfinch.com/v1/qa/triggers/run" \
            -H "authorization: Bearer ${{ secrets.TESTFINCH_TRIGGER_TOKEN }}" \
            -H 'content-type: application/json' \
            -d "{\"label\":\"deploy ${GITHUB_SHA::7}\"}")
          echo "execution=$(echo "$body" | jq -r '.executions[0].execution_id')" >> "$GITHUB_OUTPUT"
      - name: Wait for the verdict
        run: |
          for i in $(seq 1 120); do
            v=$(curl -sS "https://api.testfinch.com/v1/qa/triggers/executions/${{ steps.start.outputs.execution }}" \
              -H "authorization: Bearer ${{ secrets.TESTFINCH_TRIGGER_TOKEN }}")
            if [ "$(echo "$v" | jq -r .done)" = "true" ]; then
              echo "$v" | jq -r '.results[] | "\(.status)\t\(.id)\t\(.failed_step.error // "")"'
              [ "$(echo "$v" | jq -r .ok)" = "true" ] && exit 0 || exit 1
            fi
            sleep 15
          done
          echo "FlowQA run did not finish in 30 minutes"; exit 1
```

Replace the `on:` block with `push` or `workflow_run` to run on other events.
A GitLab, CircleCI or Jenkins job is the same two calls.

Pipelines set up before the header put the token in the address instead (`/v1/qa/triggers/<token>/run` and `/v1/qa/triggers/<token>/executions/<id>`).
That form still works but is deprecated: move the token into the header.

## Alerts

Each schedule and trigger names who hears about its runs:

- **Emails**: one or more addresses; the creator's by default.
- **Slack channel**: a channel name such as `#qa-alerts`, posted through the workspace's Slack connection (connected in TestFinch under Settings, then Integrations).
  Slack alerts are part of FlowQA Team and above; on other plans naming a channel is refused, and a channel named on a better plan is kept but not posted to until the workspace upgrades.
- **When**: on failure (default), or after every run.

An alert lists each failing test with the step and error, and links to the test's run history in FlowQA.
Manual runs never alert: the person who started one is watching it.

## Running with a personal access token instead

A pipeline can also use a TestFinch personal access token (Settings, then API tokens) and the regular runs API (`/qa/executions`):
`POST /v1/projects/:projectId/qa/executions` with `target: "cloud"`, then `GET .../executions/:id`.
That path needs the token owner's workspace membership to include FlowQA, and the test revisions in the request must be current.
The trigger token is simpler for CI; the personal token suits scripts that also read the catalogue.
