docs: bring docs up to date and drop unused keys setting

- rewrite architecture, development, mcp and scenario docs to match the
  current code: single container, all mcp tools, runs, versioning,
  timeouts, settings, client routing and dev setup
- rewrite README as a general overview without liquio-specific wording
- remove the unused KEYS_DIR setting and ./keys compose mount
- remove the unused result parameter from step code execution
- complete .env.example with the file storage settings and fix the
  DB_PATH default
This commit is contained in:
Andrii Arsenin committed 2026-09-30 21:18:41 +03:00
1 parent 0d621a8bdb
commit 522c49d7c2
12 files changed
+239 -207

No files matched your search

+8 -4
View File
@@ -1,11 +1,15 @@
NODE_ENV=development
PORT=3000
# Directory (relative to cwd) containing *.json key descriptors
KEYS_DIR=keys
# SQLite database path
DB_PATH=data/sessions.db
DB_PATH=data/liqa.db
# Stored scenario files and run artifacts (downloads, failure dumps, traces)
FILES_DIR=data/files
# Days to keep run artifacts before they are deleted
FILE_RUN_EXPIRATION_DAYS=7
# When expired files are deleted (cron expression)
FILE_CLEANUP_CRON=0 3 * * *
# Chromium binary to use instead of Playwright's own (not needed in Docker,
# which ships Playwright's chromium-headless-shell)
+2 -2
View File
@@ -1,6 +1,6 @@
# liqa
liqa (liquio-qa-bot) is a QA automation service for Liquio environments. It stores **scenarios** (ordered Playwright steps), **snippets** (reusable code run via `context.runSnippet`), **environments**, **credentials** and **sessions**, and runs scenarios in headless Chromium. Scenarios and snippets are versioned; each run records the scenario version it executed.
liqa is a browser automation and QA service for web applications. It stores **scenarios** (ordered Playwright steps), **snippets** (reusable code run via `context.runSnippet`), **environments**, **credentials** and **sessions**, and runs scenarios in headless Chromium. Scenarios and snippets are versioned; each run records the scenario version it executed.
It is an npm workspace with two packages:
@@ -20,7 +20,7 @@ It is an npm workspace with two packages:
## Default deployment (docker compose)
`compose.yaml` at the repo root runs a single `liqa` container, built from the root `Dockerfile`, at http://localhost:13000. The server serves everything: the UI at `/` (the built client, via `CLIENT_DIR`), the REST API at `/api/v1`, MCP at `/mcp`, Swagger at `/docs`. It reads `.env` and mounts `./data` (SQLite DB `data/liqa.db` + stored files) and `./keys`.
`compose.yaml` at the repo root runs a single `liqa` container, built from the root `Dockerfile`, at http://localhost:13000. The server serves everything: the UI at `/` (the built client, via `CLIENT_DIR`), the REST API at `/api/v1`, MCP at `/mcp`, Swagger at `/docs`. It reads `.env` and mounts `./data` (SQLite DB `data/liqa.db` + stored files).
Rebuild and restart: `docker compose up -d --build`. Check with `docker compose ps`, `curl localhost:13000/health` and `docker compose logs liqa`.
+1 -1
View File
@@ -79,7 +79,7 @@ COPY --from=client-build /app/client/dist ./client
# Serve the built client at / (see server/src/common/client-static.ts)
ENV CLIENT_DIR=client
# Runtime directories (keys and SQLite DB mounted via volumes)
# Runtime directory (SQLite DB and stored files, mounted as a volume)
RUN mkdir -p data
EXPOSE 3000
+27 -34
View File
@@ -1,46 +1,39 @@
# liquio-qa-bot
# liqa
A headless QA automation service for Liquio environments. Exposes an **MCP (Model Context Protocol)** server that lets AI agents define, run, and monitor multi-step browser test scenarios using [Playwright](https://playwright.dev/).
A browser automation and QA service for web applications. It stores multi-step browser scenarios, runs them with [Playwright](https://playwright.dev/) in headless Chromium, and keeps their results, logs and artifacts. Scenarios are managed from a web UI, a REST API, or an **MCP (Model Context Protocol)** server that lets AI agents define, run and debug them.
---
## Quick start
```bash
cp .env.example .env
docker compose up -d --build
```
Everything is served from one container at **http://localhost:13000**: the UI at `/`, the REST API at `/api/v1`, MCP at `/mcp`, Swagger at `/docs`. Data is kept in `./data/`.
---
## Overview
The bot manages four core concepts:
| Concept | Description |
|---|---|
| **Environment** | A named set of URLs for a deployment target (e.g. `liquio-diia-stg`) |
| **Key** | A JSON descriptor in `keys/` that points to an EDS key file + password |
| **Session** | Saved browser state (cookies + localStorage) produced by a login |
| **Scenario** | An ordered list of steps that run in sequence against a shared browser instance |
| **Scenario** | An ordered list of steps (async Playwright JS) that run in sequence in one browser. Versioned: every content change creates a new version |
| **Run** | One execution of a scenario version in an environment, with per-step status and output, logs, and run files (downloads, failure screenshots, Playwright traces) |
| **Snippet** | Reusable, versioned code called from steps with `context.runSnippet("alias")` (or `"alias@N"` for a specific version) |
| **Environment** | A named set of key/value settings for a deployment target (URLs etc.), read with `context.getEnv(key)` |
| **Credential** | Secret data (e.g. a key file and password), assigned to scenarios under an alias and read with `context.getCredential(alias)` |
| **Session** | Saved browser state (cookies + localStorage), e.g. kept from a run with "save session" |
Scenarios, snippets, environments and credentials can be exported as YAML and imported back (one file may hold several of them).
---
## Architecture
## Documentation
See [docs/architecture.md](docs/architecture.md).
---
## Key descriptors
See [docs/key-descriptors.md](docs/key-descriptors.md).
---
## MCP tools
See [docs/mcp.md](docs/mcp.md).
---
## Scenario execution
See [docs/scenario.md](docs/scenario.md).
---
## Development
See [docs/development.md](docs/development.md) for setup, running, testing, linting, and Docker instructions.
- [docs/architecture.md](docs/architecture.md) - components, data model, client
- [docs/scenario.md](docs/scenario.md) - runs, the script `context` API, versioning, import/export, Playwright traces
- [docs/mcp.md](docs/mcp.md) - MCP tool reference
- [docs/development.md](docs/development.md) - setup, configuration, running, testing, linting, Docker
- [CHANGELOG.md](CHANGELOG.md) - release notes
-2
View File
@@ -10,6 +10,4 @@ services:
volumes:
# persist SQLite database across restarts
- ./data:/app/data
# mount key files into the container
- ./keys:/app/keys
restart: unless-stopped
+65 -37
View File
@@ -1,50 +1,78 @@
# Architecture
```
┌──────────────────────────────────────────────────────┐
│ Browser (React SPA - client/) │
│ │
│ Environments Credentials Snippets │
│ Sessions Scenarios Runs │
└─────────────────────────────┬────────────────────────┘
│ HTTP REST (:13000)
│
MCP client (AI agent / VS Code Copilot)
│ HTTP /mcp (Streamable MCP)
▼
┌──────────────────────────────────────────────────────┐
│ NestJS app (server/src/) │
│ │
│ McpService all MCP tools │
│ SessionService session persistence │
│ SessionContextService live browser contexts │
│ EnvironmentService environment CRUD + import │
│ CredentialService credential CRUD + import │
│ SnippetService snippet CRUD + import │
│ BrowserService ad-hoc open/exec │
│ CodeExecutorService sandboxed JS runtime │
│ ScenarioService scenarios/steps/runs CRUD │
│ ScenarioSchedulerService @Interval(1 s) run worker │
└───────────────────────┬──────────────────────────────┘
│ better-sqlite3 (TypeORM)
▼
data/sessions.db
Browser (React SPA) MCP client (AI agent, IDE)
│ / and /api/v1 │ /mcp (Streamable HTTP)
└──────────────┬──────────────────┘
▼
┌─────────────────────────────────────────────────────────────┐
│ liqa container (:13000 → 3000) NestJS app, server/src/ │
│ │
│ Routes / built client (ServeStaticModule) │
│ /api/v1 REST API /docs Swagger UI │
│ /mcp MCP tools /health health check │
│ │
│ McpService MCP tools → services below │
│ ScenarioService scenarios, steps, versions, runs │
│ ScenarioSchedulerService @Interval(1 s) run worker │
│ E2eExportService standalone Playwright test zip │
│ SnippetService snippets + versions │
│ EnvironmentService environments │
│ CredentialService credentials │
│ ImportService central YAML import │
│ FileStorageService scenario files + run artifacts │
│ SessionService saved browser sessions │
│ SessionContextService live Playwright contexts │
│ SessionSchedulerService @Interval(60 s) session cleanup │
│ BrowserService ad-hoc open / exec │
│ CodeExecutorService runs step / snippet JS │
└───────────────┬─────────────────────────────┬───────────────┘
│ TypeORM (better-sqlite3) │ Playwright
▼ ▼
data/liqa.db, data/files/ chromium-headless-shell
```
- **McpService** registers MCP tools and delegates to session, environment, browser, and scenario services.
- **SessionService** persists session records in SQLite. **SessionContextService** manages live Playwright contexts for open sessions.
- **EnvironmentService**, **CredentialService**, and **SnippetService** provide CRUD plus import/export upserts.
- **BrowserService** handles stateless ad-hoc automation (`/open`, `/exec`) and session-aware browsing.
- **CodeExecutorService** validates and executes user JS with `page`, `context`, and helper APIs.
- **ScenarioService** manages scenarios, steps, credential aliases, runs, logs, and scenario import/export.
- **ScenarioSchedulerService** polls pending runs every second and executes step runs to completion.
Everything runs in one container built from the root `Dockerfile`: the server serves the API, MCP and Swagger, and also the built client. See [development.md](development.md) for running it.
## Server components (`server/src/`)
- **McpService** (`mcp/`) registers the MCP tools and delegates to the services below. Each request gets a fresh MCP server over the Streamable HTTP transport.
- **ScenarioService** (`scenario/`) manages scenarios, steps, credential aliases, versions, runs, run logs, scenario/run files, and scenario export/import.
- **ScenarioSchedulerService** picks up pending runs every second (up to 10 in parallel) and executes their steps in order in one Playwright browser per run. It handles step and scenario timeouts, failure page dumps (screenshot + HTML), Playwright traces, and saving the browser as a session (`saveSession`).
- **E2eExportService** builds a standalone Playwright test project (zip) from a scenario, bundling the snippets, credentials and files it uses.
- **SnippetService** (`snippet/`), **EnvironmentService** (`environment/`) and **CredentialService** (`credential/`) provide CRUD plus export/import.
- **ImportService** (`import/`) implements `POST /import`: validates a YAML stream of mixed items, then imports them in dependency order (environments, credentials, snippets, scenarios).
- **FileStorageService** (`file/`) stores uploaded scenario files and run artifacts under `FILES_DIR`, and deletes expired files on a cron schedule (`FILE_CLEANUP_CRON`).
- **SessionService** (`session/`) persists saved sessions (cookies, localStorage); **SessionContextService** keeps their live Playwright contexts; **SessionSchedulerService** closes idle sessions and deletes old closed ones every minute.
- **BrowserService** (`browser/`) handles ad-hoc automation (`POST /api/v1/open`, `POST /api/v1/exec`, and the `open_url` / `exec_code` MCP tools), optionally in a saved session.
- **CodeExecutorService** (`code-executor/`) checks step and snippet code with acorn, then runs it with the `context` API (page, environment values, credentials, snippets, files, assertions, logs). It is not a security sandbox: step code is trusted.
Cross-cutting pieces:
- `config/app.config.ts` - all settings, validated at startup (see `.env.example`).
- `common/app-package.ts` - app name and version, read from the workspace root `package.json`.
- `common/client-static.ts` - serving the built client, with the `index.html` fallback for client-side routes.
- `common/body-parsers.ts` - YAML request bodies (`application/yaml`) for the import endpoint.
- `interceptors/` - a trace id per request (kept in `AsyncLocalStorage` and added to log lines) and request/response logging; `filters/` - consistent error responses.
## Data
SQLite via TypeORM with `synchronize: true`: entity changes alter the schema on startup, and there are no migrations. The main tables:
- `scenarios` / `scenario_steps` hold the **latest** version of each scenario; `scenario_versions` holds a snapshot of every version (content + steps).
- `snippets` holds the latest version of each snippet; `snippet_versions` holds every version. Step code pins a version with `runSnippet("alias@N")`.
- `scenario_runs` record the scenario version, environment, trace mode and status; `scenario_run_steps` keep their own copy of each step's code, so a run always shows what it executed; `scenario_run_logs` hold ordered run logs.
- `files` + `scenario_files` / `scenario_run_files` map stored files to scenarios and runs.
- `environments`, `credentials`, `scenario_credentials` (alias → credential per scenario), `sessions`.
See [scenario.md](scenario.md) for versioning, runs and import/export rules.
## Client (`client/`)
A React 19 + TypeScript SPA served by the Vite dev server during development. In the Docker image the server serves the built client itself (`ServeStaticModule`, see `server/src/common/client-static.ts`), from the same origin as the API.
A React 19 + TypeScript SPA. During development it's served by the Vite dev server; in the Docker image the server serves the built client from the same origin as the API.
The client calls the server REST endpoints directly — there is no separate BFF layer. Routing uses `BrowserRouter`; the server answers any GET outside `/api`, `/mcp`, `/docs` and `/health` with the client's `index.html`, so client-side routes survive a page refresh.
The sidebar footer contains the theme switcher and the app version (injected from the workspace root `package.json` at build time).
Pages: scenarios (with steps, versions, credentials, used snippets and files), runs, snippets, credentials, environments, sessions, and the central import page. The sidebar footer contains the theme switcher and the app version (injected from the workspace root `package.json` at build time).
See [development.md](development.md) for the full client tech stack and directory layout.
+37 -27
View File
@@ -16,20 +16,26 @@ Install all dependencies from the repo root:
```bash
npm install
npx playwright install chromium # first time only (server)
npx playwright install --only-shell chromium # first time only: headless Chromium for the server
```
Copy `.env.example` to `.env` and fill in the required values before running.
### Environment variables (server)
All settings are validated at startup by `server/src/config/app.config.ts`.
| Variable | Default | Description |
|---|---|---|
| `NODE_ENV` | `development` | Shown in the startup log |
| `PORT` | `3000` | HTTP port |
| `KEYS_DIR` | `keys` | Directory for key material used by credential/snippet workflows |
| `DB_PATH` | `data/sessions.db` | SQLite database file |
| `DB_PATH` | `data/liqa.db` | SQLite database file |
| `FILES_DIR` | `data/files` | Where scenario files and run artifacts are stored |
| `FILE_RUN_EXPIRATION_DAYS` | `7` | How long run artifacts (downloads, failure dumps, traces) are kept |
| `FILE_CLEANUP_CRON` | `0 3 * * *` | When expired files are deleted |
| `SESSION_IDLE_TIMEOUT_MINUTES` | `30` | Idle timeout before open sessions are closed |
| `SESSION_DELETE_CLOSED_DAYS` | `7` | Retention period for closed sessions |
| `PLAYWRIGHT_TRACE_MODE` | `off` | Default Playwright trace mode for runs: `on`, `retain-on-failure` or `off` (see [scenario.md](scenario.md)) |
| `CLIENT_DIR` | *(empty)* | Directory with the built client to serve at `/` (the Docker image sets it to `client`); empty means the server doesn't serve the UI |
| `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` | _(Playwright default)_ | Path to a Chromium binary to use instead of Playwright's own. Not set in Docker: the image ships Playwright's `chromium-headless-shell` (matching the Playwright version) under `PLAYWRIGHT_BROWSERS_PATH=/ms-playwright`. |
@@ -62,13 +68,12 @@ The server listens on port **3000** by default.
npm -w client run dev
```
The Vite dev server runs on **http://localhost:5173** and proxies these API prefixes to `localhost:13000` (or `localhost:3000` when running server locally):
The Vite dev server runs on **http://localhost:5173**. It has no API proxy: the client calls `VITE_API_URL` (default `/api/v1` on its own origin), and the server allows CORS from `http://localhost:5173`. So point it at a running server:
- `/environments`
- `/credentials`
- `/snippets`
- `/sessions`
- `/scenarios`
```bash
npm -w client run dev:compose-api # API of the compose container (localhost:13000)
VITE_API_URL=http://localhost:3000/api/v1 npm -w client run dev # API of a locally started server
```
**Production build**:
@@ -100,7 +105,7 @@ Watch mode:
npm -w server run test:watch
```
Enable verbose NestJS log output during tests:
Print the NestJS logs of the test app (they're silenced by default; `DEBUG=test`):
```bash
npm -w server run test:debug
@@ -198,7 +203,7 @@ Everything is served by the single `liqa` container at **http://localhost:13000*
When switching from the old two-container setup (`server` + `client`), remove the old containers with `docker compose up -d --build --remove-orphans`.
SQLite data is persisted in `./data/` and key files are mounted from `./keys/` — both directories are volume-mounted into the container.
The SQLite database and stored files are persisted in `./data/`, which is volume-mounted into the container.
Stop and remove containers:
@@ -216,7 +221,7 @@ docker compose logs -f
## Client overview
The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provides a browser UI for managing environments, credentials, snippets, sessions, scenarios, and runs.
The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provides a browser UI for managing scenarios, runs, snippets, credentials, environments and sessions, and for importing exported YAML.
### Tech stack
@@ -224,9 +229,13 @@ The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provid
|---|---|
| React 19 + TypeScript | UI framework |
| Vite 6 | Dev server and bundler |
| react-router-dom (HashRouter) | Client-side routing |
| react-router-dom 7 (`BrowserRouter`) | Client-side routing |
| @tanstack/react-query | Data fetching (cross-scenario runs list) |
| i18next + react-i18next | Internationalisation |
| lucide-react | Icon library |
| @monaco-editor/react | Code editor for steps and snippets |
| react-markdown + highlight.js | Markdown descriptions and code highlighting |
| yaml | YAML export of entities |
| moment | Relative timestamp formatting |
| ESLint 10 + Prettier | Linting and formatting |
| Storybook 10 + Vitest | Component development and testing |
@@ -236,29 +245,30 @@ The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provid
```
client/
src/
api/ API client modules (environments, credentials, snippets, sessions, scenarios)
hooks/ useTheme (light/dark persistence)
api/ API client (client.ts) and types (types.ts)
components/ modals/ - run, export, export logs and delete confirmation modals
hooks/ useTheme, usePageTitle, useRunDefaults, useImportFile
i18n/ i18next bootstrap + locales/en.json
pages/ Environment/Credential/Snippet/Session/Scenario/Run pages
ui/ Reusable component library
Badge, Breadcrumbs, Button, Card, Input, Select,
SidePanel, Table, ThemeSwitcher, Timestamp
lib/ highlight.js setup, API error toast events
pages/ scenario/, run/, snippet/, credential/, environment/, session/, import/
ui/ Reusable component library (Badge, Button, Card, CodeEditor, Modal,
Select, SidePanel, Table, Toast, ... - see ui/index.ts)
.storybook/
stories/ *.stories.tsx for all ui/ components
stories/ *.stories.tsx for ui/ components
```
### Routing
Hash-based routing (`/#/path`) avoids conflicts with the Vite dev-server proxy. Default route redirects to `/#/scenarios`.
`BrowserRouter` with plain paths. The root and unknown paths redirect to `/scenarios`. The server answers these paths with the client's `index.html` (see [architecture.md](architecture.md)), so they work on refresh.
| Route | Page |
|---|---|
| `/#/environments` | Environment list |
| `/#/credentials` | Credential list |
| `/#/snippets` | Snippet list |
| `/#/sessions` | Session list |
| `/#/scenarios` | Scenario list |
| `/#/runs` | Cross-scenario runs list |
| `/scenarios` | Scenario list |
| `/scenarios/:id` | Scenario details (steps, versions, credentials, used snippets, files) |
| `/scenarios/:id/runs`, `/scenarios/:id/runs/:runId` | Runs of a scenario, run details |
| `/runs` | Cross-scenario runs list |
| `/snippets`, `/credentials`, `/environments`, `/sessions` | Entity lists (plus `/:id`, `/new`, `/:id/edit` where applicable) |
| `/import` | Central import |
### Theming
+66 -44
View File
@@ -1,94 +1,116 @@
# MCP tools
The MCP endpoint is exposed at `/mcp` (Streamable HTTP transport).
The MCP endpoint is exposed at `/mcp` (Streamable HTTP transport), e.g. `http://localhost:13000/mcp` with the compose setup. Each request is served by a fresh MCP server, so there is no MCP session state between calls.
The server currently registers the following tools.
The server registers the following tools. All entity IDs are UUIDs; list tools are paginated (`page`, `limit`, and where noted `orderBy` / `orderDir`). Errors are returned as MCP tool errors (`isError: true`) with the message as text.
## Sessions
| Tool | Description |
|---|---|
| `list_sessions` | List stored sessions (paginated) |
| `delete_session` | Delete a session by UUID |
| `list_sessions` | List saved browser sessions (`orderBy`/`orderDir`) |
| `delete_session` | Delete a session by `id` |
## Environments
| Tool | Description |
|---|---|
| `list_environments` | List environments (paginated) |
| `get_environment` | Get one environment by UUID |
| `create_environment` | Create environment (`name`, `urls`) |
| `update_environment` | Update environment (`id`, optional `name`/`urls`) |
| `delete_environment` | Delete environment by UUID |
| `list_environments` | List environments (`orderBy`/`orderDir`) |
| `get_environment` | Get one environment by `id` |
| `create_environment` | Create an environment: `name`, optional `description`, `data` |
| `update_environment` | Update `name`, `description` and/or `data` of environment `id` |
| `delete_environment` | Delete an environment by `id` |
`urls` is a free-form key/value map. Common keys: `id_url`, `cabinet_url`, `admin_url`.
`data` is a free-form map of string keys to string values, read in step code with `context.getEnv(key)`.
## Credentials
| Tool | Description |
|---|---|
| `list_credentials` | List credentials (`orderBy`/`orderDir`). Returns the full `data`, secrets included |
| `get_credential` | Get one credential by `id` (full `data`) |
| `create_credential` | Create a credential: `name`, optional `data` (JSON-encoded string) |
| `update_credential` | Update `name` and/or `data`; omitted fields are left unchanged |
| `delete_credential` | Delete a credential; also removes it from scenarios it's assigned to |
## Browser (ad-hoc)
| Tool | Description |
|---|---|
| `open_url` | Open URL with optional `sessionName`, return page/title/content |
| `exec_code` | Execute Playwright JS — script receives a single `context` argument (see `scenario.md`). Optional `environmentId` (UUID) resolves environment from DB for `context.getEnv()`. Optional `credentials` (`{ alias: credentialUUID }`) resolves credentials from DB for `context.getCredential(alias)`. |
| `open_url` | Open `url`, optionally in saved session `sessionName`; returns title and content (`readerMode` for readable text, `selector` to limit to one element) |
| `exec_code` | Run Playwright JS that receives a single `context` argument (see [scenario.md](scenario.md)), optionally after opening `url` and/or in session `sessionName`. `environmentId` loads an environment for `context.getEnv()`; `credentials` (`{ alias: credentialId }`) loads credentials for `context.getCredential(alias)` |
## Scenarios
| Tool | Description |
|---|---|
| `list_scenarios` | List scenarios (paginated) |
| `get_scenario` | Get scenario with steps |
| `create_scenario` | Create scenario by `name` |
| `update_scenario` | Rename scenario |
| `delete_scenario` | Delete scenario |
| `create_scenario_step` | Add step to scenario |
| `list_scenarios` | List scenarios (`orderBy`/`orderDir`) |
| `get_scenario` | Get a scenario (latest version) with its steps |
| `create_scenario` | Create a scenario: `name`, optional `description`, `environmentId` (default environment), `timeoutSeconds` |
| `update_scenario` | Update scenario fields; bumps the version when content changes |
| `delete_scenario` | Delete a scenario with its steps, versions and runs |
| `create_scenario_step` | Append a step: `execCode`, optional `title` and `timeoutSeconds`. The schema also requires `order` and `type`, but both are ignored: new steps always go last (use `update_scenario_step` with `order` to move one) |
| `get_scenario_step` | Get one step |
| `update_scenario_step` | Update step fields |
| `delete_scenario_step` | Delete step |
| `list_scenario_runs` | List runs for a scenario |
| `update_scenario_step` | Update step fields (including `order` to move it) |
| `delete_scenario_step` | Delete a step |
| `list_scenario_versions` | List a scenario's versions, newest first |
| `get_scenario_version` | Get one scenario version with its steps |
| `run_scenario` | Trigger run for scenario; optional `version` (defaults to latest) and `traceMode` |
| `get_scenario_run` | Get run with step runs and logs |
| `wait_for_scenario_run` | Wait until run is `pass` or `fail` |
| `export_scenario` | Export one scenario version (optional `version`, defaults to latest) |
| `import_scenario` | Import one scenario version; `version` is required (see "Versioning" in `scenario.md`) |
| `import_yaml` | Import any mix of environments, credentials, snippets and scenarios from YAML (see "Import" in `scenario.md`) |
| `list_scenario_credentials` | List credentials assigned to a scenario (alias → credential) |
| `add_scenario_credential` | Assign credential `credentialId` to a scenario under `alias` |
| `remove_scenario_credential` | Remove an assignment by `scenarioCredentialId` (the assignment id, not the credential id) |
| `export_scenario` | Export one scenario version (optional `version`, defaults to the latest) |
| `import_scenario` | Import one scenario version; `version` is required (see "Versioning" in [scenario.md](scenario.md)) |
| `import_yaml` | Import any mix of environments, credentials, snippets and scenarios from YAML (see "Import" in [scenario.md](scenario.md)) |
| `export_e2e_test` | Export a scenario as a standalone Playwright test project (zip, base64) |
Changes to scenario content (fields and steps) create a new scenario version; credential assignments and files are not versioned.
## Runs
| Tool | Description |
|---|---|
| `run_scenario` | Start a run of scenario `id` in `environmentId`; optional `version` (defaults to the latest), `saveSession`, `traceMode` (`on`, `retain-on-failure`, `off`; defaults to `PLAYWRIGHT_TRACE_MODE`) |
| `list_scenario_runs` | List runs of a scenario, optionally filtered by `status` |
| `list_all_runs` | List runs across all scenarios, optionally filtered by `status` |
| `get_scenario_run` | Get a run with its step runs (outputs) and logs |
| `wait_for_scenario_run` | Wait until a run is `pass` or `fail` (up to 5 minutes), then return it |
## Snippets
| Tool | Description |
|---|---|
| `list_snippets` | List snippets (paginated) |
| `list_snippets` | List snippets (`orderBy`/`orderDir`) |
| `get_snippet` | Get one snippet (latest version) |
| `create_snippet` | Create snippet (`alias`, `title`, `code`) |
| `create_snippet` | Create a snippet: `alias`, `title`, `code`, optional `description` |
| `update_snippet` | Update snippet fields; bumps its version when content changes |
| `delete_snippet` | Delete snippet and all its versions |
| `delete_snippet` | Delete a snippet and all its versions |
| `list_snippet_versions` | List a snippet's versions, newest first |
| `get_snippet_version` | Get one snippet version |
## Scenario Files
Step code runs a snippet with `context.runSnippet("alias", ...args)` (latest version) or `context.runSnippet("alias@3", ...args)` (version 3).
## Scenario files
| Tool | Description |
|---|---|
| `upload_scenario_file` | Upload a file to a scenario; accepts optional `expiresAt` (ISO 8601 date string) |
| `list_scenario_files` | List scenario files (paginated) |
| `get_scenario_file_content` | Retrieve file content as base64 with metadata |
| `upload_scenario_file` | Upload a file to a scenario: `name`, base64 `contentBase64`, optional `mimeType` and `expiresAt` (ISO 8601) |
| `list_scenario_files` | List a scenario's files (`limit`, `offset`) |
| `get_scenario_file_content` | Get a file's content as base64, with metadata |
## Run Artifact Files
## Run files
| Tool | Description |
|---|---|
| `list_run_files` | List files (artifacts) created during a scenario run (paginated) |
| `get_run_file_content` | Retrieve run artifact content as base64 with metadata; run artifact content can't be modified and inherits expiry from the run artifact subsystem |
| `delete_run_file` | Delete a file from a run; the stored file is removed once no scenario or run references it (also `DELETE /scenarios/:id/runs/:runId/files/:fileId` and the delete button on the run detail page) |
| `list_run_files` | List files created during a run (`limit`, `offset`): downloads, failure screenshots/HTML, `trace.zip` |
| `get_run_file_content` | Get a run file's content as base64, with metadata |
| `delete_run_file` | Delete a file from a run; the stored file is removed once no scenario or run references it (also `DELETE /api/v1/scenarios/:id/runs/:runId/files/:fileId` and the delete button on the run detail page) |
## Binary Content Handling
## Binary content handling
File tools transport binary payloads as base64-encoded strings in a `contentBase64` field with an accompanying `encoding: "base64"` marker in the response. When retrieving file content via `get_scenario_file_content` or `get_run_file_content`, decode the base64 to recover the original bytes.
File tools carry binary content as base64 strings in a `contentBase64` field, with `encoding: "base64"` in the response. Decode it to get the original bytes.
Scenario files uploaded via `upload_scenario_file` must have their content pre-encoded as base64. Run artifacts are created implicitly through scenario execution (via `context.downloadFile()` during step execution) and cannot be uploaded via MCP.
Content for `upload_scenario_file` must be base64-encoded. Run files are only created by runs themselves (`context.downloadFile()`, failure page dumps, traces) and can't be uploaded.
## Notes
- Tool IDs and entity IDs are UUIDs.
- `create_scenario_step` schema still accepts legacy `type`/`sessionName` fields for compatibility. Current scheduler executes `execCode` plus optional `validateCode` and does not branch by step type.
- The step `type` field (`login`, `exec`, `sign`) in `create_scenario_step`, `update_scenario_step` and `import_scenario` is a legacy field kept for compatibility; it's ignored, and every step runs its `execCode`.
+31 -42
View File
@@ -1,23 +1,23 @@
# Scenario execution
When a scenario run is created, the server creates one `ScenarioRunStep` per scenario step.
API paths below are relative to the REST prefix `/api/v1` (e.g. `POST /api/v1/scenarios/:id/run`).
- Step runs are ordered by `step.order`.
- First step run starts as `pending`; the rest start as `waiting`.
- Scheduler tick (`@Interval(1000)`) picks pending runs and processes steps in order.
- Steps in the same run share one Playwright browser/context/page.
## Runs
## Execution model
Creating a run (`POST /scenarios/:id/run`, the Run button, or the `run_scenario` MCP tool) takes an environment, and optionally a scenario version (latest by default), `saveSession` and a trace mode.
Current scheduler behavior is exec-centric:
- `execCode` is required for a step to pass.
- `validateCode` is optional and runs after `execCode`.
- Legacy step type metadata may still appear in MCP payloads, but run execution is not branched by step type.
- The run gets one step run per step of that scenario version, each with its own copy of the step's title, code and timeout. Editing or deleting steps later doesn't change existing runs.
- The first step run starts as `pending`, the rest as `waiting`.
- A scheduler tick (`@Interval(1000)`) picks up pending runs (up to 10 at a time) and executes their steps in order. All steps of a run share one Playwright browser, context and page.
- A step passes when its code returns, and fails when it throws or times out. Its return value is stored as the step output (JSON) and is available to later steps via `context.getStepOutput()`.
- When a step fails, the run is marked `fail`, the remaining steps are `cancelled`, and a screenshot and the page HTML are saved to the run's files (`step-<order>-failure.png` / `.html`).
- Timeouts: each step gets its own `timeoutSeconds`, otherwise the scenario's `timeoutSeconds`, otherwise 60 s. The run as a whole fails once it has run longer than the scenario's `timeoutSeconds` (default 600 s); this is checked before each step.
- With `saveSession`, the run's browser is kept after the run as a saved session named `run-<runId>` (see Sessions in the UI), instead of being closed.
- Runs left `in_progress` when the server stops are marked `fail` on the next start.
## Script runtime
`execCode` and `validateCode` execute as async JavaScript with a single `context` argument:
Step code (`execCode`) and snippet code run as an async JavaScript function body with a single `context` argument:
```js
async (context) => {
@@ -25,23 +25,29 @@ async (context) => {
}
```
Available in scope:
Available on `context`:
- `context.page`: Playwright `Page`
- `context.browser`: Playwright `BrowserContext`
- `context.env`: shallow copy of environment key/value map
- `context.getEnv(key)`: required environment value lookup (throws if missing)
- `context.getCredential(alias)`: credential payload assigned to the scenario alias
- `context.getStepOutput(order)`: prior step output by absolute order (`1`, `2`, ...) or relative (`-1` = previous step)
- `context.dumpDom(selector?)`: simplified DOM snapshot
- `context.log(...args)`, `context.warn(...args)`, `context.error(...args)`: structured step logs
- `context.env`: shallow copy of the environment's key/value map
- `context.getEnv(key)`: environment value lookup (throws if missing)
- `context.getCredential(alias)`: payload of the credential assigned to the scenario under `alias` (parsed JSON when the data is JSON; throws if missing)
- `context.getStepOutput(order)`: output of a previous step by absolute order (`1`, `2`, ...) or relative (`-1` = previous step); `null` if there is none
- `context.dumpDom(selector?)`: simplified DOM snapshot of the page (or of one element)
- `context.log(...args)`, `context.warn(...args)`, `context.error(...args)`: run log entries
- `context.assert(() => boolean, description)`: runs the arrow function and requires it to return `true`. Logs `description` (as a `log` entry on success, as an `error` entry on failure or exception), then throws to stop the step if the function returned anything other than `true` or threw
- `context.runSnippet(name, ...args)`: execute stored snippet code with the same context. `name` is the snippet alias for its latest version, or `alias@<version>` (e.g. `login@3`) to pin a specific version
- `context.getScenarioFiles(opts?)`: list files uploaded to the scenario (`limit`, `offset`)
- `context.downloadFile(url, opts?)`: fetch `url` and save the result as a run artifact (requires a real scenario run — throws when invoked ad hoc, e.g. via the `exec_code` MCP tool). `opts` may include `method`, `headers`, `body`, `filename`. `url` also accepts a `data:` URI (`data:<mediaType>;base64,<data>` or `data:<mediaType>,<percent-encoded data>`) to save content generated in-script (e.g. a credential JWT) directly, without an actual network fetch.
- `console`: proxied to run logs (`log`, `warn`, `error`, etc.)
- `context.runSnippet(name, ...args)`: runs stored snippet code with the same context; the snippet receives the extra arguments as `...args`. `name` is the snippet alias for its latest version, or `alias@<version>` (e.g. `login@3`) to pin a specific version. Snippet start/finish/failure is logged
- `context.getScenarioFiles(opts?)`: list files uploaded to the scenario (`limit`, `offset`), including their path on disk
- `context.downloadFile(url, opts?)`: fetch `url` and save the result as a run file (requires a real scenario run — throws when invoked ad hoc, e.g. via the `exec_code` MCP tool). `opts` may include `method`, `headers`, `body`, `filename`. `url` also accepts a `data:` URI (`data:<mediaType>;base64,<data>` or `data:<mediaType>,<percent-encoded data>`) to save content generated in-script (e.g. a credential JWT) directly, without an actual network fetch.
`validateCode` additionally receives `result` in scope, which is the value returned by `execCode`.
Also in scope:
- `console`: proxied to run logs (`log`, `warn`, `error`, etc.)
- `expect`: Playwright's `expect` (e.g. `await expect(context.page.locator("h1")).toHaveText("Done")`)
Code is syntax-checked before it runs, and executes in the server process: it's trusted code, not sandboxed.
Step run logs also include automatic "Step N started / passed / failed" entries.
## Versioning
@@ -107,23 +113,6 @@ Traces are off by default because they often reach ~10 MB per run.
Traces are run artifacts, so they expire after `FILE_RUN_EXPIRATION_DAYS` like other run files.
## Validation contract
`validateCode` may return:
```js
return { success: true, description: 'ok' };
// or
return false;
```
Interpretation:
- Boolean: `true` = pass, `false` = fail
- Object: `success` controls pass/fail; `description` is persisted as step description
On validation failure, the step fails and the run is marked `fail`.
## Statuses
Run status:
@@ -147,7 +136,7 @@ Run-step status:
1. Create environment
2. Create credentials and snippets (optional but common)
3. Create scenario
4. Add ordered steps (`execCode`, optional `validateCode`)
4. Add ordered steps (`execCode`, optional `title` and `timeoutSeconds`)
5. Assign credentials to scenario aliases when needed
6. Run scenario
7. Wait for run completion and inspect run logs/outputs
@@ -74,7 +74,6 @@ export interface ExecContext {
credentials?: Record<string, unknown>;
environment?: EnvironmentData | null;
snippets?: Record<string, string> | null;
result?: unknown;
scenarioId?: string;
runId?: string;
fileService?: FileStorageService;
@@ -115,7 +114,6 @@ export class CodeExecutorService {
credentials,
environment,
snippets,
result,
scenarioId,
runId,
fileService,
@@ -278,12 +276,11 @@ export class CodeExecutorService {
},
};
// `console`, `result`, and `expect` are injected as named parameters so
// existing scripts that reference them directly still work.
// `console` and `expect` are injected as named parameters so scripts
// can reference them directly.
const fn = new Function(
"context",
"console",
"result",
"expect",
`return (async (context) => { ${code} })(context)`,
);
@@ -291,7 +288,6 @@ export class CodeExecutorService {
const execResult = await fn(
scriptContext,
fakeConsole,
result,
playwrightExpect,
);
return { result: execResult };
@@ -46,11 +46,6 @@ export class ExecContextBuilder {
return this;
}
result(result: unknown): this {
this.ctx.result = result;
return this;
}
scenarioId(scenarioId: string): this {
this.ctx.scenarioId = scenarioId;
return this;
-3
View File
@@ -31,9 +31,6 @@ export class AppConfig {
@IsString()
APP_VERSION: string = appPackage().version;
@IsString()
KEYS_DIR: string = "keys";
@IsString()
DB_PATH: string = "data/liqa.db";