ars9/liqa
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: refresh architecture and runtime documentation

Browse Source 
- align architecture, development, MCP, and scenario docs with current code

- remove obsolete key descriptor document and stale keys/auth references
This commit is contained in:
ars9
2026-04-10 20:40:20 +03:00
parent 0e4a2e1819
commit 1627733701
5 changed files with 150 additions and 149 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+26 -26
View File
@@ -1,45 +1,43 @@
# Architecture
```
┌─────────────────────────────────────────┐
│ Browser (React SPA — client/) │
┌──────────────────────────────────────────────────────┐
│ Browser (React SPA - client/) │
│ │
│ EnvironmentsPage KeysPage │
│ SessionsPage ScenariosPage │
└───────────────────┬─────────────────────┘
│ Environments Credentials Snippets │
│ Sessions Scenarios Runs │
└─────────────────────────────┬────────────────────────┘
│ HTTP REST (:13000)
│
MCP client (AI agent / VS Code Copilot)
│ HTTP /mcp (Streamable MCP)
▼
┌──────────────────────────────────────┐
┌──────────────────────────────────────────────────────┐
│ NestJS app (server/src/) │
│ │
│ McpService ← all MCP tools │
│ │
│ AuthService login + sessions │
│ SessionService session CRUD │
│ EnvironmentService environment CRUD │
│ BrowserService ad-hoc browsing │
│ CodeExecutorService sandboxed JS │
│ │
│ ScenarioService scenario CRUD │
│ ScenarioSchedulerService │
│ └─ @Interval(1 s) tick loop │
└──────────────────┬───────────────────┘
│ McpService all MCP tools │
│ SessionService session persistence │
│ SessionContextService live browser contexts │
│ EnvironmentService environment CRUD + import │
│ CredentialService credential CRUD + import │
│ SnippetService snippet CRUD + import │
│ BrowserService ad-hoc open/exec │
│ CodeExecutorService sandboxed JS runtime │
│ ScenarioService scenarios/steps/runs CRUD │
│ ScenarioSchedulerService @Interval(1 s) run worker │
└───────────────────────┬──────────────────────────────┘
│ better-sqlite3 (TypeORM)
▼
data/sessions.db
```
- **McpService** — registers every MCP tool and delegates to domain services.
- **AuthService** — drives a Playwright browser to log in via a key descriptor; saves cookies + localStorage as a `Session`.
- **SessionService** — stores and retrieves saved browser sessions.
- **EnvironmentService** — stores named environments (URL maps) used by auth and scenarios.
- **BrowserService** — ad-hoc, stateless `open_url` / `exec_code` operations outside the scenario scheduler.
- **CodeExecutorService** — compiles and runs user-supplied Playwright JS inside a sandboxed `async` function with `page`, `context`, and `helpers` in scope.
- **ScenarioService** — CRUD for scenarios, steps, runs, and run logs; import/export.
- **ScenarioSchedulerService** — polls for pending step runs every second and executes them. All steps in the same run share **one browser instance** (see [scenario.md](scenario.md)).
- **McpService** registers MCP tools and delegates to session, environment, browser, and scenario services.
- **SessionService** persists session records in SQLite. **SessionContextService** manages live Playwright contexts for open sessions.
- **EnvironmentService**, **CredentialService**, and **SnippetService** provide CRUD plus import/export upserts.
- **BrowserService** handles stateless ad-hoc automation (`/open`, `/exec`) and session-aware browsing.
- **CodeExecutorService** validates and executes user JS with `page`, `context`, and helper APIs.
- **ScenarioService** manages scenarios, steps, credential aliases, runs, logs, and scenario import/export.
- **ScenarioSchedulerService** polls pending runs every second and executes step runs to completion.
## Client (`client/`)
@@ -47,4 +45,6 @@ A React 19 + TypeScript SPA served separately by Vite during development. In Doc
The client calls the server REST endpoints directly — there is no separate BFF layer. Routing uses HashRouter so the Vite proxy and server-side routes are never ambiguous.
The sidebar footer contains the theme switcher and the app version (injected from the workspace root `package.json` at build time).
See [development.md](development.md) for the full client tech stack and directory layout.
docs/development.md
+19 -7
View File
@@ -26,8 +26,10 @@ Copy `.env.example` to `.env` and fill in the required values before running.
| Variable | Default | Description |
|---|---|---|
| `PORT` | `3000` | HTTP port |
| `KEYS_DIR` | `keys` | Directory containing `*.json` key descriptors |
| `KEYS_DIR` | `keys` | Directory for key material used by credential/snippet workflows |
| `DB_PATH` | `data/sessions.db` | SQLite database file |
| `SESSION_IDLE_TIMEOUT_MINUTES` | `30` | Idle timeout before open sessions are closed |
| `SESSION_DELETE_CLOSED_DAYS` | `7` | Retention period for closed sessions |
| `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` | _(Playwright default)_ | Path to Chromium binary. Set automatically in Docker (`/usr/bin/chromium`). |
---
@@ -59,7 +61,13 @@ The server listens on port **3000** by default.
npm -w client run dev
```
The Vite dev server runs on **http://localhost:5173** and proxies all API paths (`/environments`, `/sessions`, `/scenarios`, `/keys`, `/login`, `/mcp`) to the server at `localhost:3000`.
The Vite dev server runs on **http://localhost:5173** and proxies these API prefixes to `localhost:13000` (or `localhost:3000` when running server locally):
- `/environments`
- `/credentials`
- `/snippets`
- `/sessions`
- `/scenarios`
**Production build**:
@@ -201,7 +209,7 @@ docker compose logs -f
## Client overview
The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provides a browser UI for managing environments, sessions, keys, and scenarios.
The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provides a browser UI for managing environments, credentials, snippets, sessions, scenarios, and runs.
### Tech stack
@@ -221,10 +229,10 @@ The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provid
```
client/
src/
api/ API client modules (environments, keys, sessions, scenarios)
api/ API client modules (environments, credentials, snippets, sessions, scenarios)
hooks/ useTheme (light/dark persistence)
i18n/ i18next bootstrap + locales/en.json
pages/ EnvironmentsPage, KeysPage, SessionsPage, ScenariosPage
pages/ Environment/Credential/Snippet/Session/Scenario/Run pages
ui/ Reusable component library
Badge, Breadcrumbs, Button, Card, Input, Select,
SidePanel, Table, ThemeSwitcher, Timestamp
@@ -239,10 +247,14 @@ Hash-based routing (`/#/path`) avoids conflicts with the Vite dev-server proxy.
| Route | Page |
|---|---|
| `/#/environments` | Environment list |
| `/#/keys` | Key list |
| `/#/credentials` | Credential list |
| `/#/snippets` | Snippet list |
| `/#/sessions` | Session list |
| `/#/scenarios` | Scenario list |
| `/#/runs` | Cross-scenario runs list |
### Theming
Light/dark theme is toggled by the `ThemeSwitcher` button in the sidebar header. The selection is persisted to `localStorage` and applied as `data-theme` on `<html>` before React mounts (anti-FOUC inline script in `index.html`).
Light/dark theme is toggled by the `ThemeSwitcher` button in the sidebar footer. The selection is persisted to `localStorage` and applied as `data-theme` on `<html>` before React mounts (anti-FOUC inline script in `index.html`).
The sidebar footer also shows the app version read from the workspace root `package.json` and injected via Vite `define`.
docs/key-descriptors.md
-20
View File
@@ -1,20 +0,0 @@
# Key descriptors
Each file in `KEYS_DIR` is named `<keyId>.json` and takes one of two forms:
**File key (EDS):**
```json
{
"keyFile": "path/to/key.dat",
"password": "secret"
}
```
`keyFile` is resolved relative to `KEYS_DIR`.
**Login + password:**
```json
{
"login": "user@example.com",
"password": "secret"
}
```
docs/mcp.md
+35 -28
View File
@@ -1,49 +1,56 @@
# MCP tools
## Auth / session
The MCP endpoint is exposed at `/mcp` (Streamable HTTP transport).
The server currently registers the following tools.
## Sessions
| Tool | Description |
|---|---|
| `list_keys` | List available key IDs from `KEYS_DIR` |
| `login` | Log in with a key against an environment; stores resulting session |
| `list_sessions` | Paginated list of saved sessions |
| `delete_session` | Delete a session by ID |
| `list_sessions` | List stored sessions (paginated) |
| `delete_session` | Delete a session by UUID |
## Environments
| Tool | Description |
|---|---|
| `list_environments` | Paginated list of environments |
| `get_environment` | Get one environment by ID |
| `create_environment` | Create an environment (`name` + `urls` map) |
| `update_environment` | Update name / urls |
| `delete_environment` | Delete by ID |
| `list_environments` | List environments (paginated) |
| `get_environment` | Get one environment by UUID |
| `create_environment` | Create environment (`name`, `urls`) |
| `update_environment` | Update environment (`id`, optional `name`/`urls`) |
| `delete_environment` | Delete environment by UUID |
`urls` is a free-form map. Recognised keys: `id_url` (auth server), `cabinet_url` (cabinet redirect target), `admin_url`.
`urls` is a free-form key/value map. Common keys: `id_url`, `cabinet_url`, `admin_url`.
## Browser (ad-hoc, stateless)
## Browser (ad-hoc)
| Tool | Description |
|---|---|
| `open_url` | Navigate to a URL and return page content / reader-mode text |
| `exec_code` | Run Playwright JS in a session's browser context |
| `open_url` | Open URL with optional `sessionName`, return page/title/content |
| `exec_code` | Execute Playwright JS with `page` and `context` in scope |
## Scenarios
| Tool | Description |
|---|---|
| `list_scenarios` | Paginated list |
| `get_scenario` | Fetch scenario with all steps |
| `create_scenario` | Create a new scenario by name |
| `update_scenario` | Rename |
| `delete_scenario` | Delete (cascades to steps and runs) |
| `create_scenario_step` | Add a step |
| `get_scenario_step` | Fetch one step |
| `list_scenarios` | List scenarios (paginated) |
| `get_scenario` | Get scenario with steps |
| `create_scenario` | Create scenario by `name` |
| `update_scenario` | Rename scenario |
| `delete_scenario` | Delete scenario |
| `create_scenario_step` | Add step to scenario |
| `get_scenario_step` | Get one step |
| `update_scenario_step` | Update step fields |
| `delete_scenario_step` | Remove a step |
| `list_scenario_runs` | Paginated run history (with optional status filter) |
| `run_scenario` | Trigger an immediate run |
| `get_scenario_run` | Get a run with all step runs and their outputs |
| `wait_for_scenario_run` | Block until run reaches `pass`/`fail`, return full run |
| `export_scenario` | Export scenario as a portable JSON payload |
| `import_scenario` | Import a scenario from an export payload |
| `delete_scenario_step` | Delete step |
| `list_scenario_runs` | List runs for a scenario |
| `run_scenario` | Trigger run for scenario |
| `get_scenario_run` | Get run with step runs and logs |
| `wait_for_scenario_run` | Wait until run is `pass` or `fail` |
| `export_scenario` | Export scenario payload |
| `import_scenario` | Import scenario payload |
## Notes
- Tool IDs and entity IDs are UUIDs.
- `create_scenario_step` schema still accepts legacy `type`/`sessionName` fields for compatibility. Current scheduler executes `execCode` plus optional `validateCode` and does not branch by step type.
docs/scenario.md
+60 -58
View File
@@ -1,80 +1,82 @@
# Scenario execution
When a scenario is run, the scheduler creates a `ScenarioRun` with one `ScenarioRunStep` per step. Steps are executed **in order** (by `step.order`), one per scheduler tick.
When a scenario run is created, the server creates one `ScenarioRunStep` per scenario step.
**All steps in the same run share a single Playwright browser instance.** The browser is created on the first `exec` or `sign` step (using session cookies + localStorage), reused for every subsequent step, and closed when the run finishes (pass or fail).
- Step runs are ordered by `step.order`.
- First step run starts as `pending`; the rest start as `waiting`.
- Scheduler tick (`@Interval(1000)`) picks pending runs and processes steps in order.
- Steps in the same run share one Playwright browser/context/page.
## Step types
## Execution model
### `login`
Current scheduler behavior is exec-centric:
Authenticates in a **separate, ephemeral browser** and saves the resulting session to the database. Subsequent exec/sign steps load that session into the shared browser.
- `execCode` is required for a step to pass.
- `validateCode` is optional and runs after `execCode`.
- Legacy step type metadata may still appear in MCP payloads, but run execution is not branched by step type.
`execCode` must be a JSON object:
```json
{ "keyId": "3137411915", "environmentName": "liquio-diia-stg" }
```
## Script runtime
### `exec`
`execCode` and `validateCode` execute as async JavaScript with:
Runs arbitrary Playwright JS on the **shared page**. The page carries over state from the previous step (URL, form inputs, cookies, etc.).
- `page`: Playwright `Page`
- `context`: Playwright `BrowserContext`
- `helpers`: utility object
- `console`: proxied to run logs (`log`, `warn`, `error`, etc.)
`execCode` is executed as the body of an `async` function with these variables in scope:
`validateCode` additionally receives `result`, which is the value returned by `execCode`.
- `page` — current Playwright `Page`
- `context` — current Playwright `BrowserContext`
- `helpers.dumpDom(selector?)` — returns a simplified DOM string
- `helpers.getStepOutput(order)` — returns the JSON-parsed `output` of a previous step. Positive values are absolute step orders; negative values are relative to the current step (`-1` = previous step).
## Available helpers
Any value returned from `execCode` is JSON-serialised and stored as the `output` field on the `ScenarioRunStep` record. Subsequent steps can read it via `helpers.getStepOutput()`.
- `helpers.dumpDom(selector?)`: simplified DOM snapshot
- `helpers.log(...args)`, `helpers.warn(...args)`, `helpers.error(...args)`: structured step logs
- `helpers.getStepOutput(order)`: prior step output by absolute order (`0`, `1`, ...) or relative (`-1` previous step)
- `helpers.getCredential(alias)`: credential payload assigned to the scenario alias
- `helpers.env`: shallow copy of environment URL map
- `helpers.getEnvUrl(key)`: required environment URL lookup (throws if missing)
- `helpers.runSnippet(name, ...args)`: execute stored snippet code in the same page/context/helpers scope
## Validation contract
`validateCode` may return:
```js
// example execCode
await page.goto('https://cabinet.example.com/tasks/create/987825');
await page.locator('input[name="institution"]').fill('caltech');
await page.locator('button:has-text("Далі")').click();
await page.waitForLoadState('networkidle');
return page.url(); // stored in ScenarioRunStep.output
```
### `sign`
Performs EDS signing on the **current shared page** using a key from `KEYS_DIR`. The step clicks the sign widget, selects the file key method, uploads the key file, enters the password, and submits — all without requiring the step author to handle key material in their code.
`execCode` must be a JSON object:
```json
{ "keyId": "3137411915" }
```
## Validation code
Any step type optionally accepts `validateCode` — Playwright JS that runs on the shared page **after** the main action and must return a result evaluated as:
```js
return { success: true, description: 'Application created' };
return { success: true, description: 'ok' };
// or
return { success: false, description: 'Expected success banner not found' };
// or simply
return someBoolean;
return false;
```
If `success` is false the step (and the whole run) fails with the provided `description`.
Interpretation:
## Run statuses
- Boolean: `true` = pass, `false` = fail
- Object: `success` controls pass/fail; `description` is persisted as step description
`pending` → `in_progress` → `pass` | `fail`
On validation failure, the step fails and the run is marked `fail`.
Individual step runs also expose `waiting` (not yet reached) and `cancelled` (run failed before this step).
## Statuses
## Adding a new scenario (example workflow)
Run status:
```
1. create_environment (if not already present)
2. create_scenario → get scenarioId
3. create_scenario_step type=login, order=0
4. create_scenario_step type=exec, order=1 (navigate + fill form)
5. create_scenario_step type=sign, order=2
6. create_scenario_step type=exec, order=3 (validateCode checks success banner)
7. run_scenario → get runId
8. wait_for_scenario_run (blocks until pass/fail, returns full run with outputs)
```
- `pending`
- `in_progress`
- `pass`
- `fail`
Run-step status:
- `waiting`
- `pending`
- `in_progress`
- `pass`
- `fail`
- `cancelled`
## Example workflow
1. Create environment
2. Create credentials and snippets (optional but common)
3. Create scenario
4. Add ordered steps (`execCode`, optional `validateCode`)
5. Assign credentials to scenario aliases when needed
6. Run scenario
7. Wait for run completion and inspect run logs/outputs