Files
liqa/docs/scenario.md
T
ars9 a328ef5ea7 feat(runs): record playwright traces with per-run trace mode, bump to 1.15.0
- record a trace for each scenario run and upload it as trace.zip
- add PLAYWRIGHT_TRACE_MODE (on, retain-on-failure, off) as the default
- add a trace mode option to the run modal, stored per run and reused
  by repeat run; expose the default via GET /scenarios/run-defaults
- accept traceMode in POST /scenarios/:id/run and run_scenario MCP tool
- share one run modal component between scenario list and detail pages
- git-ignore server/data
2026-09-30 14:58:07 +03:00

4.2 KiB

Scenario execution

When a scenario run is created, the server creates one ScenarioRunStep per scenario step.

  • Step runs are ordered by step.order.
  • First step run starts as pending; the rest start as waiting.
  • Scheduler tick (@Interval(1000)) picks pending runs and processes steps in order.
  • Steps in the same run share one Playwright browser/context/page.

Execution model

Current scheduler behavior is exec-centric:

  • execCode is required for a step to pass.
  • validateCode is optional and runs after execCode.
  • Legacy step type metadata may still appear in MCP payloads, but run execution is not branched by step type.

Script runtime

execCode and validateCode execute as async JavaScript with a single context argument:

async (context) => {
  // your code here
}

Available in scope:

  • context.page: Playwright Page
  • context.browser: Playwright BrowserContext
  • context.env: shallow copy of environment key/value map
  • context.getEnv(key): required environment value lookup (throws if missing)
  • context.getCredential(alias): credential payload assigned to the scenario alias
  • context.getStepOutput(order): prior step output by absolute order (1, 2, ...) or relative (-1 = previous step)
  • context.dumpDom(selector?): simplified DOM snapshot
  • context.log(...args), context.warn(...args), context.error(...args): structured step logs
  • context.assert(() => boolean, description): runs the arrow function and requires it to return true. Logs description (as a log entry on success, as an error entry on failure or exception), then throws to stop the step if the function returned anything other than true or threw
  • context.runSnippet(name, ...args): execute stored snippet code with the same context
  • context.getScenarioFiles(opts?): list files uploaded to the scenario (limit, offset)
  • context.downloadFile(url, opts?): fetch url and save the result as a run artifact (requires a real scenario run — throws when invoked ad hoc, e.g. via the exec_code MCP tool). opts may include method, headers, body, filename. url also accepts a data: URI (data:<mediaType>;base64,<data> or data:<mediaType>,<percent-encoded data>) to save content generated in-script (e.g. a credential JWT) directly, without an actual network fetch.
  • console: proxied to run logs (log, warn, error, etc.)

validateCode additionally receives result in scope, which is the value returned by execCode.

Playwright traces

Each run records a Playwright trace of its browser context (screenshots and DOM snapshots). When the run finishes, the trace is uploaded to the run's files as trace.zip. Download it from the run detail page, then open it with npx playwright show-trace trace.zip or at https://trace.playwright.dev.

The trace mode is chosen per run: the run modal in the UI, the traceMode field of POST /scenarios/:id/run, and the traceMode argument of the run_scenario MCP tool all accept one of the values below. When it is omitted, the server's PLAYWRIGHT_TRACE_MODE setting is used (exposed via GET /scenarios/run-defaults). "Repeat run" reuses the original run's mode.

  • on (default): keep the trace of every run
  • retain-on-failure: keep traces only for runs that failed
  • off: do not record traces

Traces are run artifacts, so they expire after FILE_RUN_EXPIRATION_DAYS like other run files.

Validation contract

validateCode may return:

return { success: true, description: 'ok' };
// or
return false;

Interpretation:

  • Boolean: true = pass, false = fail
  • Object: success controls pass/fail; description is persisted as step description

On validation failure, the step fails and the run is marked fail.

Statuses

Run status:

  • pending
  • in_progress
  • pass
  • fail

Run-step status:

  • waiting
  • pending
  • in_progress
  • pass
  • fail
  • cancelled

Example workflow

  1. Create environment
  2. Create credentials and snippets (optional but common)
  3. Create scenario
  4. Add ordered steps (execCode, optional validateCode)
  5. Assign credentials to scenario aliases when needed
  6. Run scenario
  7. Wait for run completion and inspect run logs/outputs