Files
liqa/docs/scenario.md
T
ars9 daecf3d539
CI / changes (pull_request) Successful in 7s
CI / version (pull_request) Skipped
CI / server (pull_request) Skipped
CI / client (pull_request) Successful in 1m13s
CI / image (pull_request) Skipped
feat(client): add code reference modal to step and snippet editors
- a (?) button next to the Code label on the step and snippet
  create/edit pages opens a reference of the script runtime: globals,
  every context member with examples, and pitfalls such as awaiting
  getStepOutput and assert
- the reference lives in components/modals/code-help.md and is
  rendered as markdown
- Modal takes size="lg" for document-like content; the dialog is
  capped to the viewport and its body scrolls, focusable when lg so
  the keyboard can scroll it
- docs/scenario.md notes that getStepOutput is async and points to
  the in-app reference to keep in sync
2026-10-01 14:45:03 +03:00

11 KiB

Scenario execution

API paths below are relative to the REST prefix /api/v1 (e.g. POST /api/v1/scenarios/:id/run).

Runs

Creating a run (POST /scenarios/:id/run, the Run button, or the run_scenario MCP tool) takes an environment, and optionally a scenario version (latest by default), saveSession and a trace mode.

  • The run gets one step run per step of that scenario version, each with its own copy of the step's title, code and timeout. Editing or deleting steps later doesn't change existing runs.
  • The first step run starts as pending, the rest as waiting.
  • A scheduler tick (@Interval(1000)) picks up pending runs (up to 10 at a time) and executes their steps in order. All steps of a run share one Playwright browser, context and page.
  • A step passes when its code returns, and fails when it throws or times out. Its return value is stored as the step output (JSON) and is available to later steps via context.getStepOutput().
  • When a step fails, the run is marked fail, the remaining steps are cancelled, and a screenshot and the page HTML are saved to the run's files (step-<order>-failure.png / .html).
  • Timeouts: each step gets its own timeoutSeconds, otherwise the scenario's timeoutSeconds, otherwise 60 s. The run as a whole fails once it has run longer than the scenario's timeoutSeconds (default 600 s); this is checked before each step.
  • With saveSession, the run's browser is kept after the run as a saved session named run-<runId> (see Sessions in the UI), instead of being closed.
  • Runs left in_progress when the server stops are marked fail on the next start.

Script runtime

Step code (execCode) and snippet code run as an async JavaScript function body with a single context argument:

async (context) => {
  // your code here
}

Available on context:

  • context.page: Playwright Page
  • context.browser: Playwright BrowserContext
  • context.env: shallow copy of the environment's key/value map
  • context.getEnv(key): environment value lookup (throws if missing)
  • context.getCredential(alias): payload of the credential assigned to the scenario under alias (parsed JSON when the data is JSON; throws if missing)
  • context.getStepOutput(order): async (await it); output of a previous step by absolute order (1, 2, ...) or relative (-1 = previous step); null if there is none
  • context.dumpDom(selector?): simplified DOM snapshot of the page (or of one element)
  • context.log(...args), context.warn(...args), context.error(...args), context.debug(...args): run log entries
  • context.assert(() => boolean, description): runs the arrow function and requires it to return true. Logs description (as a log entry on success, as an error entry on failure or exception), then throws to stop the step if the function returned anything other than true or threw
  • context.runSnippet(name, ...args): runs stored snippet code with the same context; the snippet receives the extra arguments as ...args. name is the snippet alias for its latest version, or alias@<version> (e.g. login@3) to pin a specific version. Snippet start/finish/failure is logged
  • context.getScenarioFiles(opts?): list files uploaded to the scenario (limit, offset), including their path on disk
  • context.downloadFile(url, opts?): fetch url and save the result as a run file (requires a real scenario run — throws when invoked ad hoc, e.g. via the exec_code MCP tool). opts may include method, headers, body, filename. url also accepts a data: URI (data:<mediaType>;base64,<data> or data:<mediaType>,<percent-encoded data>) to save content generated in-script (e.g. a credential JWT) directly, without an actual network fetch.

Also in scope:

  • console: proxied to run logs (log/info → log, warn, error, debug)
  • expect: Playwright's expect (e.g. await expect(context.page.locator("h1")).toHaveText("Done"))

Code is syntax-checked before it runs, and executes in the server process: it's trusted code, not sandboxed.

The step and snippet editors have a (?) button next to the Code label that opens a reference of this runtime in the UI. Its content is client/src/components/modals/code-help.md: keep it in sync with this section when the context API changes.

Run logs

Each run log entry (level, message, createdAt) also carries a context: where the message was generated. Contexts:

  • step:<N>: the step a log entry belongs to, including the automatic "Step N started / passed / failed" entries
  • snippet:<alias>: entries logged while context.runSnippet(alias, ...) is running, including its own automatic "started"/"finished"/"failed" entries. A snippet calling another snippet tags entries with the innermost running snippet's alias
  • assertion:pass / assertion:fail: context.assert()'s own log entry
  • run: run-level entries not tied to a step (e.g. a scenario timeout, or the Playwright trace upload notice)

context.log/warn/error (and console.log/etc.) split their arguments: strings/numbers/booleans are joined into message, while object/array arguments are kept out of it and stored as data (JSON), so message stays a stable string regardless of an object's shape. With one object argument data is that object; with more than one, data is an array of them. E.g. context.log("Saved credential as run file:", fileMeta) logs message: "Saved credential as run file:" with fileMeta as data, shown pretty-printed and separately from the message in the UI.

GET /scenarios/:id/runs/:runId?q=... and the exported CSV/Markdown logs (GET /scenarios/:id/runs/:runId/export) match q against both message and context.

Versioning

Scenarios and snippets are versioned, starting at version 1.

  • A scenario's version goes up when its content changes: name, description, default environment, timeout, or any step change (add, edit, delete, reorder). Credential assignments and uploaded files are not versioned. Saving without changes does not create a new version.
  • A snippet's version goes up when its alias, title, description or code changes.
  • The scenario and snippet pages always show the latest version. Older versions are available via GET /scenarios/:id/versions[/:version] and GET /snippets/:id/versions[/:version].
  • A run executes one scenario version, chosen in the run modal (latest by default) or via version in POST /scenarios/:id/run. The run keeps its own copy of that version's steps, so deleting or editing steps later doesn't change what the run shows. "Repeat run" reuses the run's version.
  • Step code uses the latest version of a snippet by default. context.runSnippet("alias@3") pins version 3 instead. The version is looked up through the snippet's current alias. Pinned versions are listed under "Used Snippets" on the scenario page.

Export / import

Exports hold exactly one version and always include its version number (the latest by default, or ?version=N). Imports require version, and restore that exact version by id + version:

  • If that version already exists, it is left untouched.
  • Otherwise it is stored under that number, and becomes the latest if it is higher than the current latest version.
  • Without an id, a new scenario or snippet is created at the given version.

Import

POST /import (the Import page in the UI, and the Import buttons on the list pages; the import_yaml MCP tool) imports any mix of environments, credentials, snippets and scenarios.

  • Send YAML with Content-Type: application/yaml. A file is a stream of ----separated documents; each document holds one exported item or a list of items. JSON is valid YAML, and a JSON body (one item or an array) is accepted too.
  • Every item needs a kind: environment, credential, snippet or scenario. Snippets and scenarios also need version (see above).
  • All items are validated before anything is imported, so one invalid item means nothing from that request is imported. The error lists every invalid item by its position.
  • Items are imported in dependency order - environments, credentials, snippets, scenarios - regardless of their order in the file, so a scenario can reference an environment or credential from the same file.
  • The response lists what was imported: { imported: [{ kind, id, name, version? }] }.

A scenario export (GET /scenarios/:id/export, optionally with its environment, credentials and used snippets) is in this format, one document per entity, and can be imported back as is. Query parameters: includeEnvironment=true, credentialIds and snippetIds (repeat the parameter or separate ids with commas) and version. A selected snippet is exported once per version the scenario uses: the latest for plain runSnippet("alias") calls, plus each version pinned with alias@N. Snippets used only through other snippets count as used.

---
# Environment: staging
kind: environment
id: 5b0c…
name: staging
data:
  base_url: https://staging.example
---
# Scenario: Login flow
kind: scenario
id: 9e1f…
version: 3
name: Login flow
environmentId: 5b0c…
steps:
  - title: Open login page
    execCode: await context.page.goto(context.getEnv("base_url"));

Playwright traces

A run can record a Playwright trace of its browser context (screenshots and DOM snapshots). When the run finishes, the trace is uploaded to the run's files as trace.zip. Download it from the run detail page, then open it with npx playwright show-trace trace.zip or at https://trace.playwright.dev.

The trace mode is chosen per run: the run modal in the UI, the traceMode field of POST /scenarios/:id/run, and the traceMode argument of the run_scenario MCP tool all accept one of the values below. When it is omitted, the server's PLAYWRIGHT_TRACE_MODE setting is used (exposed via GET /scenarios/run-defaults). "Repeat run" reuses the original run's mode.

  • on: keep the trace of every run
  • retain-on-failure: keep traces only for runs that failed
  • off (default): do not record traces

Traces are off by default because they often reach ~10 MB per run.

Traces are run artifacts, so they expire after FILE_RUN_EXPIRATION_DAYS like other run files.

Statuses

Run status:

  • pending
  • in_progress
  • pass
  • fail

Run-step status:

  • waiting
  • pending
  • in_progress
  • pass
  • fail
  • cancelled

Example workflow

  1. Create environment
  2. Create credentials and snippets (optional but common)
  3. Create scenario
  4. Add ordered steps (execCode, optional title and timeoutSeconds)
  5. Assign credentials to scenario aliases when needed
  6. Run scenario
  7. Wait for run completion and inspect run logs/outputs