Files
liqa/docs/scenario.md
T
Andrii Arsenin 25064c8b38 fix(runs): disable playwright traces by default, bump to 1.16.1
- default PLAYWRIGHT_TRACE_MODE to off since traces often reach ~10 MB
  per run; traces can still be enabled per run or via the setting
2026-09-30 18:35:26 +03:00

6.0 KiB

Scenario execution

When a scenario run is created, the server creates one ScenarioRunStep per scenario step.

  • Step runs are ordered by step.order.
  • First step run starts as pending; the rest start as waiting.
  • Scheduler tick (@Interval(1000)) picks pending runs and processes steps in order.
  • Steps in the same run share one Playwright browser/context/page.

Execution model

Current scheduler behavior is exec-centric:

  • execCode is required for a step to pass.
  • validateCode is optional and runs after execCode.
  • Legacy step type metadata may still appear in MCP payloads, but run execution is not branched by step type.

Script runtime

execCode and validateCode execute as async JavaScript with a single context argument:

async (context) => {
  // your code here
}

Available in scope:

  • context.page: Playwright Page
  • context.browser: Playwright BrowserContext
  • context.env: shallow copy of environment key/value map
  • context.getEnv(key): required environment value lookup (throws if missing)
  • context.getCredential(alias): credential payload assigned to the scenario alias
  • context.getStepOutput(order): prior step output by absolute order (1, 2, ...) or relative (-1 = previous step)
  • context.dumpDom(selector?): simplified DOM snapshot
  • context.log(...args), context.warn(...args), context.error(...args): structured step logs
  • context.assert(() => boolean, description): runs the arrow function and requires it to return true. Logs description (as a log entry on success, as an error entry on failure or exception), then throws to stop the step if the function returned anything other than true or threw
  • context.runSnippet(name, ...args): execute stored snippet code with the same context. name is the snippet alias for its latest version, or alias@<version> (e.g. login@3) to pin a specific version
  • context.getScenarioFiles(opts?): list files uploaded to the scenario (limit, offset)
  • context.downloadFile(url, opts?): fetch url and save the result as a run artifact (requires a real scenario run — throws when invoked ad hoc, e.g. via the exec_code MCP tool). opts may include method, headers, body, filename. url also accepts a data: URI (data:<mediaType>;base64,<data> or data:<mediaType>,<percent-encoded data>) to save content generated in-script (e.g. a credential JWT) directly, without an actual network fetch.
  • console: proxied to run logs (log, warn, error, etc.)

validateCode additionally receives result in scope, which is the value returned by execCode.

Versioning

Scenarios and snippets are versioned, starting at version 1.

  • A scenario's version goes up when its content changes: name, description, default environment, timeout, or any step change (add, edit, delete, reorder). Credential assignments and uploaded files are not versioned. Saving without changes does not create a new version.
  • A snippet's version goes up when its alias, title, description or code changes.
  • The scenario and snippet pages always show the latest version. Older versions are available via GET /scenarios/:id/versions[/:version] and GET /snippets/:id/versions[/:version].
  • A run executes one scenario version, chosen in the run modal (latest by default) or via version in POST /scenarios/:id/run. The run keeps its own copy of that version's steps, so deleting or editing steps later doesn't change what the run shows. "Repeat run" reuses the run's version.
  • Step code uses the latest version of a snippet by default. context.runSnippet("alias@3") pins version 3 instead. The version is looked up through the snippet's current alias. Pinned versions are listed under "Used Snippets" on the scenario page.

Export / import

Exports hold exactly one version and always include its version number (the latest by default, or ?version=N). Imports require version, and restore that exact version by id + version:

  • If that version already exists, it is left untouched.
  • Otherwise it is stored under that number, and becomes the latest if it is higher than the current latest version.
  • Without an id, a new scenario or snippet is created at the given version.

Playwright traces

A run can record a Playwright trace of its browser context (screenshots and DOM snapshots). When the run finishes, the trace is uploaded to the run's files as trace.zip. Download it from the run detail page, then open it with npx playwright show-trace trace.zip or at https://trace.playwright.dev.

The trace mode is chosen per run: the run modal in the UI, the traceMode field of POST /scenarios/:id/run, and the traceMode argument of the run_scenario MCP tool all accept one of the values below. When it is omitted, the server's PLAYWRIGHT_TRACE_MODE setting is used (exposed via GET /scenarios/run-defaults). "Repeat run" reuses the original run's mode.

  • on: keep the trace of every run
  • retain-on-failure: keep traces only for runs that failed
  • off (default): do not record traces

Traces are off by default because they often reach ~10 MB per run.

Traces are run artifacts, so they expire after FILE_RUN_EXPIRATION_DAYS like other run files.

Validation contract

validateCode may return:

return { success: true, description: 'ok' };
// or
return false;

Interpretation:

  • Boolean: true = pass, false = fail
  • Object: success controls pass/fail; description is persisted as step description

On validation failure, the step fails and the run is marked fail.

Statuses

Run status:

  • pending
  • in_progress
  • pass
  • fail

Run-step status:

  • waiting
  • pending
  • in_progress
  • pass
  • fail
  • cancelled

Example workflow

  1. Create environment
  2. Create credentials and snippets (optional but common)
  3. Create scenario
  4. Add ordered steps (execCode, optional validateCode)
  5. Assign credentials to scenario aliases when needed
  6. Run scenario
  7. Wait for run completion and inspect run logs/outputs