- snapshot scenario and snippet content into version tables on every
content change; live tables keep holding the latest version
- record the scenario version on each run and copy its steps into the
run steps, so runs execute and show the version they were created with
- keep run step history when a step is deleted (FK set null)
- pin a snippet version with runSnippet('alias@N')
- add version picker to the run modal and version info across the UI
- add version list/get endpoints and MCP tools
- include version in exports and require it on import, restoring the
exact version by id + version
5.9 KiB
Scenario execution
When a scenario run is created, the server creates one ScenarioRunStep per scenario step.
- Step runs are ordered by
step.order. - First step run starts as
pending; the rest start aswaiting. - Scheduler tick (
@Interval(1000)) picks pending runs and processes steps in order. - Steps in the same run share one Playwright browser/context/page.
Execution model
Current scheduler behavior is exec-centric:
execCodeis required for a step to pass.validateCodeis optional and runs afterexecCode.- Legacy step type metadata may still appear in MCP payloads, but run execution is not branched by step type.
Script runtime
execCode and validateCode execute as async JavaScript with a single context argument:
async (context) => {
// your code here
}
Available in scope:
context.page: PlaywrightPagecontext.browser: PlaywrightBrowserContextcontext.env: shallow copy of environment key/value mapcontext.getEnv(key): required environment value lookup (throws if missing)context.getCredential(alias): credential payload assigned to the scenario aliascontext.getStepOutput(order): prior step output by absolute order (1,2, ...) or relative (-1= previous step)context.dumpDom(selector?): simplified DOM snapshotcontext.log(...args),context.warn(...args),context.error(...args): structured step logscontext.assert(() => boolean, description): runs the arrow function and requires it to returntrue. Logsdescription(as alogentry on success, as anerrorentry on failure or exception), then throws to stop the step if the function returned anything other thantrueor threwcontext.runSnippet(name, ...args): execute stored snippet code with the same context.nameis the snippet alias for its latest version, oralias@<version>(e.g.login@3) to pin a specific versioncontext.getScenarioFiles(opts?): list files uploaded to the scenario (limit,offset)context.downloadFile(url, opts?): fetchurland save the result as a run artifact (requires a real scenario run — throws when invoked ad hoc, e.g. via theexec_codeMCP tool).optsmay includemethod,headers,body,filename.urlalso accepts adata:URI (data:<mediaType>;base64,<data>ordata:<mediaType>,<percent-encoded data>) to save content generated in-script (e.g. a credential JWT) directly, without an actual network fetch.console: proxied to run logs (log,warn,error, etc.)
validateCode additionally receives result in scope, which is the value returned by execCode.
Versioning
Scenarios and snippets are versioned, starting at version 1.
- A scenario's version goes up when its content changes: name, description, default environment, timeout, or any step change (add, edit, delete, reorder). Credential assignments and uploaded files are not versioned. Saving without changes does not create a new version.
- A snippet's version goes up when its alias, title, description or code changes.
- The scenario and snippet pages always show the latest version. Older versions are available via
GET /scenarios/:id/versions[/:version]andGET /snippets/:id/versions[/:version]. - A run executes one scenario version, chosen in the run modal (latest by default) or via
versioninPOST /scenarios/:id/run. The run keeps its own copy of that version's steps, so deleting or editing steps later doesn't change what the run shows. "Repeat run" reuses the run's version. - Step code uses the latest version of a snippet by default.
context.runSnippet("alias@3")pins version 3 instead. The version is looked up through the snippet's current alias. Pinned versions are listed under "Used Snippets" on the scenario page.
Export / import
Exports hold exactly one version and always include its version number (the latest by default, or ?version=N). Imports require version, and restore that exact version by id + version:
- If that version already exists, it is left untouched.
- Otherwise it is stored under that number, and becomes the latest if it is higher than the current latest version.
- Without an
id, a new scenario or snippet is created at the given version.
Playwright traces
Each run records a Playwright trace of its browser context (screenshots and DOM snapshots). When the run finishes, the trace is uploaded to the run's files as trace.zip. Download it from the run detail page, then open it with npx playwright show-trace trace.zip or at https://trace.playwright.dev.
The trace mode is chosen per run: the run modal in the UI, the traceMode field of POST /scenarios/:id/run, and the traceMode argument of the run_scenario MCP tool all accept one of the values below. When it is omitted, the server's PLAYWRIGHT_TRACE_MODE setting is used (exposed via GET /scenarios/run-defaults). "Repeat run" reuses the original run's mode.
on(default): keep the trace of every runretain-on-failure: keep traces only for runs that failedoff: do not record traces
Traces are run artifacts, so they expire after FILE_RUN_EXPIRATION_DAYS like other run files.
Validation contract
validateCode may return:
return { success: true, description: 'ok' };
// or
return false;
Interpretation:
- Boolean:
true= pass,false= fail - Object:
successcontrols pass/fail;descriptionis persisted as step description
On validation failure, the step fails and the run is marked fail.
Statuses
Run status:
pendingin_progresspassfail
Run-step status:
waitingpendingin_progresspassfailcancelled
Example workflow
- Create environment
- Create credentials and snippets (optional but common)
- Create scenario
- Add ordered steps (
execCode, optionalvalidateCode) - Assign credentials to scenario aliases when needed
- Run scenario
- Wait for run completion and inspect run logs/outputs