Test Suites

Routebase's Test Runner sends automated requests against your live API and tracks their health over time, so you find out your contract drifted before your consumers do. Work is organized into test suites that group test cases, and each case sends one request and asserts on the response.

The Test Runner page

Open Test Runner from the sidebar (requires the tests:read permission). The page has three panels:

Panel	Contents
Left	The suite tree — collapsible Suites and Scenarios sections with a shared search box (Search suites & scenarios...), plus Suite Settings and project-wide Testing settings buttons at the bottom.
Center	The selected suite, with an Overview / Tests view toggle and the run toolbar.
Right	A collapsible panel with Results and History tabs (toggle with Show Results Panel / Hide Results Panel).

Each suite in the tree carries a status dot summarizing its last run:

Dot	Last run
Green	Passed
Red	Failed or error
Yellow	Running or pending
Gray	Never run or cancelled

Creating suites and folders

Click the + button on the Suites header and choose New Suite or New Folder (requires tests:write).

1. Enter a Name (e.g. User API Tests).
2. Optionally add a Description and Tags, because tags help you organize and filter suites.
3. Click Create Suite.

Folders can be nested and support drag & drop, so you can group suites by service, team, or environment. Right-click a suite for Run, Rename and Delete. Folders offer Rename, New Subfolder and Delete.

The number of test suites is limited by your plan. At the limit, the create button reads Limit Reached and Routebase prompts you to upgrade.

Suite overview

The Overview view is a dashboard for the selected suite:

• Pass Rate shows a color-coded percentage across recent runs, with a trend badge reading Improving, Declining or Stable.
• Total Runs shows a passed, failed and error breakdown.
• Average Duration and Last Run status.
• A Contract Tests card carries linked-test and coverage stats, which Contract Testing covers.

These stats and the run history follow the environment selected in the toolbar. If a suite has runs but none against the current environment, the Overview shows "No runs for this environment" rather than an empty "No test runs yet". It also counts the runs the filter hides.

Project-wide test trends (pass rate over 7/30/90 days) also appear on your dashboard.

Suite configuration

Click Suite Settings at the bottom of the sidebar to open the Suite Configuration sheet. It covers everything a suite needs beyond its cases:

Section	What it does
Authentication	Credentials applied to every case in the suite (see below).
Seeds (pre / post-run)	Setup and cleanup sequences — see Test Data.
Reset snapshot	A known-good state restored before each run — see Test Data.
Schedules	Automated runs on a cron schedule — see Scheduling.
Webhooks (project-wide)	Notifications when runs complete — see Scheduling.
CI/CD	Ready-made pipeline snippets — see Scheduling.
Data Sets	Tabular data for data-driven runs — see Data-Driven Tests.

A suite carries no base URL and no variables of its own. Both come from the environment a run is started against, whose base URL is available to every request as {{baseUrl}}. Values that should differ from run to run come from a data set instead. Authentication is the one thing a suite can define for itself.

Authentication

Suite authentication has three modes. Inherit from Environment is the default and uses whatever the active environment defines, while Custom Auth and No Auth override it. Custom auth supports the full method catalog:

• Basic Auth, Bearer Token and API Key
• OAuth 2.0 and OAuth 1.0
• JWT Bearer and Digest Auth
• AWS Signature V4, Hawk Auth and NTLM Auth

Credential fields accept {{VARIABLE_NAME}} placeholders, so secrets can live in environment variables instead of the suite. Environment-level auth itself is configured per environment, which Project Auth describes.

Project-wide testing settings

The Testing settings button at the bottom of the sidebar opens a settings page that applies to the whole project, independent of any one suite. Today it governs how the Schema Validation assertion treats null values across every test, which Contract Testing explains.

Test cases

A test case pairs one HTTP request with a set of assertions. Click Add Test Case in a suite:

1. Enter a Name (e.g. GET /users returns 200).
2. Choose the method and the request URL, for example https://api.example.com/users. The methods are GET, POST, PUT, PATCH, DELETE, HEAD and OPTIONS, and URLs must start with http://, https:// or a variable placeholder.

Cases run top to bottom within a suite, and you drag to reorder them. Expanding a case opens its editor with four tabs named Request, Assertions, Scripts and Extract.

Right-click a case, either on its card or on its row in the suite tree, for Rename, Duplicate and Delete. Both places also carry a more-actions button, and the whole menu is hidden without tests:write. In the tree you can rename in place, so double-click the name, then Enter saves and Esc discards. Duplicate copies the request with its method, URL, headers, body and scripts, plus every assertion and every extraction, into a new case named <name> (Copy). The spec link is not carried over, so link the copy yourself if it should be a contract test.

Request

• URL supports {{...}} placeholders with inline highlighting and autocomplete. Typing {{ offers the variables earlier cases in this suite extract, each with its origin such as Step 1 · $.data.id, then environment variables, then fixtures. Typing {{fixture. and {{data. drills into fixture paths and data-set columns. Selecting an environment resolves the placeholders.
• Headers takes name and value pairs.
• Body comes with a Content-Type selector offering JSON, Form URL Encoded, Multipart Form, Plain Text and XML. The Variable button inserts a placeholder from three groups, which are Extracted variables for what earlier cases capture, Environment variables and Fixtures. For a case linked to a spec endpoint, Generate Body fills the body from the endpoint's request-body example, or builds one from its schema when the spec carries no example.
• Dynamic values come from the Dynamic menu, which inserts tokens generated at runtime:

Token	Value
{{$timestamp}}	Unix timestamp in milliseconds
{{$isoTimestamp}}	ISO 8601 date/time string
{{$uuid}}	Random UUID v4
{{$randomInt}}	Random number
{{$randomEmail}}	Random email address

• Request settings hold Timeout (ms) with a default of 30000, plus Follow Redirects, SSL Verification and URL Encoding. URL encoding offers Auto (default), Manual and None.

Assertions

Assertions define what "passing" means. Each assertion has a type, an operator, an optional target, and an expected value:

Type	Target	Typical use
Status Code	—	equals 200
Body	—	contains "success"
Header	Header name	Content-Type contains application/json
Latency (ms)	—	< less than 500
JSON Path	JSONPath (e.g. $.path.to.value)	Assert on a specific field
Schema Validation	automatic	Validates the response against the linked endpoint's schema — see Contract Testing

The operators are equals, not equals, contains, not contains, matches regex, exists, < less than**, **> greater than, <= less or equal** and **>= greater or equal. Each assertion has an on/off checkbox, so you can temporarily disable one without deleting it. Expected values accept {{...}} placeholders, including data-set columns.

When the case is linked to a spec endpoint, the JSON Path target field suggests paths from that endpoint's response schema. A Response fields dropdown lists each path with its type, filters as you type, and accepts a selection with Enter or Tab after you navigate with the arrow keys. Free text stays possible at all times, so expressions the schema cannot enumerate still work, and without a link the field is a plain input. The Extract tab uses the same dropdown.

Scripts

The Scripts tab adds JavaScript that runs in a secure sandbox. A Pre-Request Script runs before the request is sent, and a Post-Response Script runs after the response arrives. Scripts can set headers with request.setHeader(...), read environment values, and store variables with rb.setVariable(...) for later cases. The built-in API Reference panel documents the full scripting API. Use Library to insert a saved snippet and Save to add the current script to your library. Script output appears in the Console tab of the results.

Extract

The Extract tab captures values from the response into variables for subsequent test cases and for scenarios. It reads from the JSON body via JSONPath, from a response header, or from the status code. Reference the values later as {{variableName}}. The JSONPath field offers the same Response fields suggestions as a JSON Path assertion when the case is linked to an endpoint.

Importing test cases from your spec

Instead of writing cases by hand, click Import in the suite toolbar to generate them from an API spec. The import is available on the Starter plan and above, and on Free the button shows a lock and an upgrade prompt.

1. Click Import from API Spec, pick a spec, and optionally filter by version and method. Endpoints are grouped by path prefix, with Select All and Deselect All to help you choose.
2. Click Preview Import to review the generated cases, rename them, and uncheck any you do not want.
3. The import creates one case per endpoint with sensible assertions pre-filled. See Contract Testing for what gets generated.

Running tests

Pick the target environment in the toolbar's environment selector, because it determines the base URL, variables and authentication for the run. Switching environments shows which auth type is now active, and a warning appears if the selected environment has no authentication configured.

• Run a single case with the play button on its card.
• Run Selected appears once you select more than one case, as a bulk-action bar reading N selected. This is a filtered suite run rather than a set of isolated single-case runs, so variables extracted by an earlier selected case still flow into a later one.
• Run All executes every case in the suite in order.

Runs require the tests:execute permission. On completion a toast summarizes the outcome, for example "All 12 test cases passed (843ms)".

Live feedback while a run is in flight

A suite run does not make you wait for the summary. Every case starts queued, dimmed and with a pulsing dot. Exactly one case is running at a time, with a spinner ring and a highlighted border. Each card gets its result the moment that case finishes, reading 200 · 128 ms when it passed and 500 · 2/3 failed when it did not.

Failed cases are tinted red. Passed ones deliberately are not tinted green, because in a suite that is mostly green, tinting everything hides the one row that matters. If a run ends before reaching every case, the cases it never got to fall back to not yet run rather than spinning forever.

Functional vs. Performance mode

The toolbar's mode toggle switches between Functional and Performance. Functional is the default, and it makes one assertion-focused pass. The selected mode applies to every run trigger, so a single case, Run Selected and Run All all follow it. In Performance mode a Performance Config popover controls the load profile:

Setting	Range
Iterations	1–10,000
Threads	1–50
Delay between iterations (ms)	0–60,000
On Error	Ignore — Continue on error, Stop — Skip to next iteration, or Abort — Cancel entire run

Iterations run in parallel across threads, while test cases within each iteration run sequentially. Results include throughput in requests per second, P95 response time, error rate, and a Per-Test-Case Breakdown table with Avg, Min, Max, P95, Err% and Count.

Reading results

Run overview (Run All / Run Selected)

After a Run All or Run Selected run, the Results tab opens on a live run-overview table. It carries one row per test case in execution order, and it fills in as each case finishes:

• The case status, its name and its duration.
• Vars in lists the variables that resolved into the request, with secrets masked.
• Extracted lists the variables the case captured for later cases.

Below the table, a summary shows passed, failed and total time, and a progress bar while the run is still going. Click any row to focus that case and load its full detail underneath.

Case statuses

A case ends a run in one of four states. Two of them are neither a pass nor a failure, because the case never got an answer.

Status	What it means
Passed	The request went out and every enabled assertion held.
Failed	The request went out and something was wrong with the answer — a failed assertion, a schema mismatch, a transport error.
Pending	The case is a contract test and the environment does not serve that endpoint yet. Amber pill; does not fail the run — see Contract Testing.
Blocked	The target environment is read-only and this case writes, so nothing was sent. Muted shield instead of a red X — see below.

Pending and blocked cases are counted separately and appear next to the total time as · 2 pending or · 1 blocked. Neither is ever folded into the failed count, and neither turns the run red. The completion toast counts the same way and names what it means, so a suite of eight reads and two blocked writes reports "8 passed · 2 blocked" rather than claiming all ten passed. A run in which nothing was sent, because every case was blocked, is reported as a neutral notice rather than a success, because it confirmed nothing.

Blocked: writing against a read-only environment

An environment can be marked read-only on its detail sheet, under Contract → Read-only environment. The switch needs the Manage governance permission, because a protection switch that everyone who may rename an environment can flip is not a protection switch. Environments in that state carry a Read-only badge in the environment list.

Only GET, HEAD and OPTIONS may be sent to such an environment. Every other method is refused before the URL is parsed and before any socket opens, so a blocked request leaves no trace on the target. That includes verbs Routebase does not know, which count as writes until proven otherwise. The case is reported as Blocked with a message naming both the method and the environment, reading "POST is not allowed against the read-only environment “Production”. Only GET, HEAD and OPTIONS may be sent there."

One consequence is worth knowing before you flip the switch. An API that reads via POST, such as search endpoints, GraphQL or report queries, cannot be exercised against a read-only environment either. The guard goes by the method, not by what the endpoint does with it.

Case detail

The case detail appears for a single-case run, and when you focus a row in the run overview. It carries the case's latest outcome with status code, response size and elapsed time, a per-assertion pass and fail table comparing expected against actual, and these detail tabs:

Tab	Contents
Response	The response body with syntax highlighting.
Headers	Response headers.
Cookies	Cookies set by the response.
Actual Request	Exactly what was sent — after variable substitution.
Console	Pre-request and post-response script output.

A Timing Breakdown splits the request into DNS, Connect, TLS, First Byte and Download phases, which separates network latency from server time.

Run history, export and sharing

The History tab lists past runs with filter tabs All, My Runs, and Shared, plus a compact trend chart. Click a run to load it into the Results tab.

• Export a run as JSON, an HTML Report, or JUnit XML for CI systems.
• Share Test Report generates a link to a run report. Choose the access level, which is either Public for anyone with the link or Team, then choose an expiry of 1 hour, 24 hours, 7 days or 30 days. Active links are listed with copy, Extend by 24 hours and Revoke link actions.

Permissions at a glance

Action	Permission
View suites, cases, and results	tests:read
Create, edit, delete suites and cases	tests:write
Execute runs	tests:execute

All built-in roles include these testing permissions, so Member, Admin and Owner all have them. Project-level team access still applies.

Related

• Test Scenarios — chain cases into multi-step workflows
• Contract Testing — validate responses against your spec
• Data-Driven Tests — run a case across many data rows
• Test Data — fixtures, seeds, and snapshots
• Scheduling — automated runs, webhooks, and CI/CD
• Environments — where your tests run