Files
liqa/docs/scenario.md
T
ars9 8935783371 feat(import): add central yaml import for all exportable kinds
- add POST /import, an import page and the import_yaml mcp tool; items
  are validated up front and imported in dependency order
- switch list page import buttons to the central import
- write scenario exports as separate --- yaml documents per entity
- give each test app its own files dir to avoid cross-suite races
- use node: prefix for node built-in imports
- bump version to 1.18.0
2026-09-30 19:25:03 +03:00

7.5 KiB

Scenario execution

When a scenario run is created, the server creates one ScenarioRunStep per scenario step.

  • Step runs are ordered by step.order.
  • First step run starts as pending; the rest start as waiting.
  • Scheduler tick (@Interval(1000)) picks pending runs and processes steps in order.
  • Steps in the same run share one Playwright browser/context/page.

Execution model

Current scheduler behavior is exec-centric:

  • execCode is required for a step to pass.
  • validateCode is optional and runs after execCode.
  • Legacy step type metadata may still appear in MCP payloads, but run execution is not branched by step type.

Script runtime

execCode and validateCode execute as async JavaScript with a single context argument:

async (context) => {
  // your code here
}

Available in scope:

  • context.page: Playwright Page
  • context.browser: Playwright BrowserContext
  • context.env: shallow copy of environment key/value map
  • context.getEnv(key): required environment value lookup (throws if missing)
  • context.getCredential(alias): credential payload assigned to the scenario alias
  • context.getStepOutput(order): prior step output by absolute order (1, 2, ...) or relative (-1 = previous step)
  • context.dumpDom(selector?): simplified DOM snapshot
  • context.log(...args), context.warn(...args), context.error(...args): structured step logs
  • context.assert(() => boolean, description): runs the arrow function and requires it to return true. Logs description (as a log entry on success, as an error entry on failure or exception), then throws to stop the step if the function returned anything other than true or threw
  • context.runSnippet(name, ...args): execute stored snippet code with the same context. name is the snippet alias for its latest version, or alias@<version> (e.g. login@3) to pin a specific version
  • context.getScenarioFiles(opts?): list files uploaded to the scenario (limit, offset)
  • context.downloadFile(url, opts?): fetch url and save the result as a run artifact (requires a real scenario run — throws when invoked ad hoc, e.g. via the exec_code MCP tool). opts may include method, headers, body, filename. url also accepts a data: URI (data:<mediaType>;base64,<data> or data:<mediaType>,<percent-encoded data>) to save content generated in-script (e.g. a credential JWT) directly, without an actual network fetch.
  • console: proxied to run logs (log, warn, error, etc.)

validateCode additionally receives result in scope, which is the value returned by execCode.

Versioning

Scenarios and snippets are versioned, starting at version 1.

  • A scenario's version goes up when its content changes: name, description, default environment, timeout, or any step change (add, edit, delete, reorder). Credential assignments and uploaded files are not versioned. Saving without changes does not create a new version.
  • A snippet's version goes up when its alias, title, description or code changes.
  • The scenario and snippet pages always show the latest version. Older versions are available via GET /scenarios/:id/versions[/:version] and GET /snippets/:id/versions[/:version].
  • A run executes one scenario version, chosen in the run modal (latest by default) or via version in POST /scenarios/:id/run. The run keeps its own copy of that version's steps, so deleting or editing steps later doesn't change what the run shows. "Repeat run" reuses the run's version.
  • Step code uses the latest version of a snippet by default. context.runSnippet("alias@3") pins version 3 instead. The version is looked up through the snippet's current alias. Pinned versions are listed under "Used Snippets" on the scenario page.

Export / import

Exports hold exactly one version and always include its version number (the latest by default, or ?version=N). Imports require version, and restore that exact version by id + version:

  • If that version already exists, it is left untouched.
  • Otherwise it is stored under that number, and becomes the latest if it is higher than the current latest version.
  • Without an id, a new scenario or snippet is created at the given version.

Import

POST /import (the Import page in the UI, and the Import buttons on the list pages; the import_yaml MCP tool) imports any mix of environments, credentials, snippets and scenarios.

  • Send YAML with Content-Type: application/yaml. A file is a stream of ----separated documents; each document holds one exported item or a list of items. JSON is valid YAML, and a JSON body (one item or an array) is accepted too.
  • Every item needs a kind: environment, credential, snippet or scenario. Snippets and scenarios also need version (see above).
  • All items are validated before anything is imported, so one invalid item means nothing from that request is imported. The error lists every invalid item by its position.
  • Items are imported in dependency order - environments, credentials, snippets, scenarios - regardless of their order in the file, so a scenario can reference an environment or credential from the same file.
  • The response lists what was imported: { imported: [{ kind, id, name, version? }] }.

A scenario export (GET /scenarios/:id/export, optionally with its environment and credentials) is in this format, one document per entity, and can be imported back as is.

---
# Environment: staging
kind: environment
id: 5b0c…
name: staging
data:
  base_url: https://staging.example
---
# Scenario: Login flow
kind: scenario
id: 9e1f…
version: 3
name: Login flow
environmentId: 5b0c…
steps:
  - title: Open login page
    execCode: await context.page.goto(context.getEnv("base_url"));

Playwright traces

A run can record a Playwright trace of its browser context (screenshots and DOM snapshots). When the run finishes, the trace is uploaded to the run's files as trace.zip. Download it from the run detail page, then open it with npx playwright show-trace trace.zip or at https://trace.playwright.dev.

The trace mode is chosen per run: the run modal in the UI, the traceMode field of POST /scenarios/:id/run, and the traceMode argument of the run_scenario MCP tool all accept one of the values below. When it is omitted, the server's PLAYWRIGHT_TRACE_MODE setting is used (exposed via GET /scenarios/run-defaults). "Repeat run" reuses the original run's mode.

  • on: keep the trace of every run
  • retain-on-failure: keep traces only for runs that failed
  • off (default): do not record traces

Traces are off by default because they often reach ~10 MB per run.

Traces are run artifacts, so they expire after FILE_RUN_EXPIRATION_DAYS like other run files.

Validation contract

validateCode may return:

return { success: true, description: 'ok' };
// or
return false;

Interpretation:

  • Boolean: true = pass, false = fail
  • Object: success controls pass/fail; description is persisted as step description

On validation failure, the step fails and the run is marked fail.

Statuses

Run status:

  • pending
  • in_progress
  • pass
  • fail

Run-step status:

  • waiting
  • pending
  • in_progress
  • pass
  • fail
  • cancelled

Example workflow

  1. Create environment
  2. Create credentials and snippets (optional but common)
  3. Create scenario
  4. Add ordered steps (execCode, optional validateCode)
  5. Assign credentials to scenario aliases when needed
  6. Run scenario
  7. Wait for run completion and inspect run logs/outputs