Testing

Scheduling

Scheduled test runs execute a test suite automatically on a cron schedule — nightly regression runs, hourly smoke tests, weekday-morning health checks — without anyone clicking Run All. Combined with webhooks for failure notifications and ready-made CI/CD snippets, they turn your suites into continuous monitoring for your API.

Scheduled test runs are available on the Starter plan and above. On the Free plan the schedule dialog shows a Starter Feature notice with a View Plans link.

Where schedules live

Schedules are per suite. Open the suite in the Test Runner, click Suite Settings at the bottom of the sidebar, and scroll to the Schedules section. Before the first schedule you'll see No scheduled runs configured with a Create First Schedule button; afterwards a table lists each schedule with its Name, Schedule (cron expression), Next Run, Last Run, Enabled toggle, and Actions.

Creating a schedule

Click Add Schedule (requires the tests:write permission):

Field Meaning
Name Required — e.g. Daily morning tests.
Description Optional.
Cron Expression Required — standard 5-field cron (minute hour day month weekday), e.g. 0 8 * * 1-5 for 8 AM on weekdays. Invalid expressions are rejected with an inline error.
Timezone The timezone the cron times are interpreted in (default UTC; common zones like Europe/Berlin, America/New_York, Asia/Tokyo are available).
Environment Which environment's base URL, variables, and auth the scheduled run uses — or None (use defaults).
Enabled "Schedule will run automatically when enabled."

The Presets button fills in common schedules, each with a human-readable preview:

Preset Cron
Every hour 0 * * * *
Every 6 hours 0 */6 * * *
Daily at midnight 0 0 * * *
Daily at 8 AM 0 8 * * *
Daily at 6 PM 0 18 * * *
Weekly on Monday 0 0 * * 1
Weekly on Friday 0 18 * * 5
Monthly on 1st 0 0 1 * *
Monthly on 15th 0 0 15 * *
The Create Schedule dialog with cron presets open

Managing schedules

  • Pause / resume with the Enabled toggle in the list.
  • Run now (the play button) triggers an immediate run outside the schedule.
  • Edit reopens the dialog; Delete removes the schedule — "Any pending scheduled runs will be cancelled."

Scheduled runs produce regular test runs: their results appear in the suite's History tab alongside manual runs, with the same results detail, export, and sharing options — see Test Suites.

Notifications on failure: webhooks

Webhooks push run outcomes to your own systems — Slack bridges, incident tooling, chat bots. They live in the Webhooks section of the Suite Configuration sheet and are project-wide: one webhook receives events from every suite in the project, whether the run was scheduled or manual.

Creating a webhook

Click Add Webhook:

Field Meaning
Name Required — e.g. CI/CD Pipeline Webhook.
Description Optional.
Payload URL Required — the HTTPS endpoint that receives the POST ("We recommend using HTTPS for security").
Events At least one of the event types below.
Max Retries How often failed deliveries are retried: No retries, 1, 2, 3 (default), 5, or 10.
Enabled "Webhook will receive events when enabled."
Event Fires when
Test Run Completed (test_run.completed) Any test run finishes.
Test Run Passed (test_run.passed) All assertions pass.
Test Run Failed (test_run.failed) One or more assertions fail.
Test Run Error (test_run.error) A run encounters an execution error.

For failure alerting, subscribe to Test Run Failed and Test Run Error.

Verifying deliveries

When a webhook is created, Routebase shows its Webhook Secret exactly once — "Save this secret now!". Verify it with the secret before trusting a payload. You can regenerate the secret at any time (Regenerate secret), which invalidates the old one.

Every delivery carries three headers:

Header Value
X-Routebase-Signature sha256= followed by the HMAC-SHA256 signature of the raw request body (hex, lowercase), computed with the webhook's secret
X-Routebase-Event The event type, e.g. test_run.failed
X-Routebase-Delivery The delivery's unique id — useful for deduplicating retries

Each webhook row expands into a delivery log — the most recent deliveries with their Status (Pending, Delivered, Failed, Retrying), event, HTTP response, attempt count, and time. Use Send test delivery to fire a test event at your endpoint without waiting for a real run.

Running suites from CI/CD

The CI/CD section of the Suite Configuration sheet generates copy-paste pipeline snippets that trigger the suite from your build system and fail the build when tests fail:

  1. Create an API key under Settings → API Keys and store it as the ROUTEBASE_API_KEY secret in your pipeline — the panel links directly to API key management.
  2. Optionally pick an Environment for the pipeline runs.
  3. Choose your platform tab — GitHub Actions, Azure Pipelines, GitLab CI, Jenkins, or Bitbucket Pipelines — and copy the snippet (each shows the target filename, e.g. .github/workflows/api-tests.yml).

Every snippet does the same thing:

POST <api base URL>/api/cli/run
X-API-Key: $ROUTEBASE_API_KEY
Content-Type: application/json

{ "suiteId": "<this suite>", "environmentId": "<if you picked one>" }

It then parses the JSON response and exits non-zero unless allPassed is true — so a red test run turns your pipeline red. The response also carries the totals (totalTests, passedTests, failedTests), totalElapsedMs, and testRunId — the run is persisted, so it shows up in the suite's History like any other.

If your organization is hosted in the US region, the generated snippet carries one extra header, X-RB-Region: us. A single address serves both regions, and the browser app picks yours from a cookie your pipeline does not have — without the header the request lands in the EU region, where your API key does not exist, and the run fails on authentication rather than on its tests. Keep the header if you edit the snippet. EU organizations need nothing extra; you can check your region under Settings → Organization.

The generator fills the base URL — and the region header where it applies — in for you. Copy the snippet rather than retyping it: those are the parts you cannot infer from this page.

The same run is also one command with the Routebase CLI, which adds JUnit output your CI can render as a native test report:

routebase run "$PROJECT_ID" "$SUITE_ID" --environment staging --format junit --output results.xml

The generated snippets need nothing installed on the runner; the CLI is shorter and reports better. See CLI in CI/CD.

  • Test Suites — the suites your schedules run
  • API Keys — authentication for CI/CD triggers
  • Environments — what a scheduled run targets
  • Webhooks — organization-level webhooks for other event types
  • Monitors — uptime and latency monitoring beyond test runs
  • CLI in CI/CD — the same runs from a command line, with JUnit output