Docs

Schedules and CI for FlowQA

Run tests on a schedule or from your deploy pipeline, and get told when they fail.

FlowQA runs your recorded tests on cloud browsers without anyone at the keyboard. Two ways to start a run unattended, both set up in the project's Settings, then Schedules & CI:

• A schedule runs the project's tests at set times (nightly, hourly, weekdays at 9).
• A CI trigger gives a pipeline a token that starts a run and reads the verdict.

Both run as the person who created them, with that person's access, and tell the people named on them when a run fails.

Schedules

A schedule has a name, a cron expression evaluated in a timezone, an environment, the tests to run (every recorded test, or a chosen set), and alert settings. The runner checks every minute. If the previous run of a schedule is still queued or running when the next slot arrives, that slot is skipped and the schedule says so; runs never stack up behind each other.

Cron is five fields: minute, hour, day of month, month, day of week.

CI trigger

Create a trigger in Schedules & CI. Its token is shown once; store it as a secret in your CI system (for example FLOWQA_TRIGGER_TOKEN). The token is the only credential the pipeline needs.

Start a run:

curl -sS -X POST "https://api.testfinch.com/v1/qa/triggers/$FLOWQA_TRIGGER_TOKEN/run" \
  -H 'content-type: application/json' \
  -d '{"label":"deploy '"$GITHUB_SHA"'"}'

The response lists the executions started (one, unless the project has more than 100 tests):

{ "project_id": "prj_...", "executions": [{ "execution_id": "qex_...", "status": "queued", "environment": "staging", "test_ids": ["smoke/home"] }] }

Optional body fields: environment (an environment id from the project; the trigger's own when omitted), tests (flow ids such as smoke/home; the trigger's selection when omitted), label (shown with the run in FlowQA).

Poll the verdict:

curl -sS "https://api.testfinch.com/v1/qa/triggers/$FLOWQA_TRIGGER_TOKEN/executions/qex_..."

{ "execution_id": "qex_...", "status": "completed", "done": true, "ok": true,
  "summary": { "total": 12, "passed": 12, "failed": 0, "flaky": 0 },
  "results": [{ "id": "smoke/home", "status": "pass", "duration_ms": 900, "failed_step": null }] }

done is true once the run finished, was cancelled or expired. ok is true only when the run completed and every test passed; fail the job on anything else. A run waits at most 30 minutes for a browser before it expires.

GitHub Actions

name: FlowQA regression
on:
  deployment_status:
jobs:
  flowqa:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - name: Start the run
        id: start
        run: |
          body=$(curl -sS -X POST "https://api.testfinch.com/v1/qa/triggers/${{ secrets.FLOWQA_TRIGGER_TOKEN }}/run" \
            -H 'content-type: application/json' \
            -d "{\"label\":\"deploy ${GITHUB_SHA::7}\"}")
          echo "execution=$(echo "$body" | jq -r '.executions[0].execution_id')" >> "$GITHUB_OUTPUT"
      - name: Wait for the verdict
        run: |
          for i in $(seq 1 120); do
            v=$(curl -sS "https://api.testfinch.com/v1/qa/triggers/${{ secrets.FLOWQA_TRIGGER_TOKEN }}/executions/${{ steps.start.outputs.execution }}")
            if [ "$(echo "$v" | jq -r .done)" = "true" ]; then
              echo "$v" | jq -r '.results[] | "\(.status)\t\(.id)\t\(.failed_step.error // "")"'
              [ "$(echo "$v" | jq -r .ok)" = "true" ] && exit 0 || exit 1
            fi
            sleep 15
          done
          echo "FlowQA run did not finish in 30 minutes"; exit 1

Replace the on: block with push or workflow_run to run on other events. A GitLab, CircleCI or Jenkins job is the same two calls.

Alerts

Each schedule and trigger names who hears about its runs:

• Emails: one or more addresses; the creator's by default.
• Slack channel: a channel name such as #qa-alerts, posted through the workspace's Slack connection (connected in Snag under Settings, then Integrations).
• When: on failure (default), or after every run.

An alert lists each failing test with the step and error, and links to the test's run history in FlowQA. Manual runs never alert: the person who started one is watching it.

Running with a personal access token instead

A pipeline can also use a Snag personal access token (Settings, then API tokens) and the regular execution API: POST /v1/projects/:projectId/qa/executions with target: "cloud", then GET .../executions/:id. That path needs the token owner's workspace membership to include FlowQA, and the test revisions in the request must be current. The trigger token is simpler for CI; the personal token suits scripts that also read the catalog.