ars9/liqa
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: refresh architecture and runtime documentation

Browse Source 
- align architecture, development, MCP, and scenario docs with current code

- remove obsolete key descriptor document and stale keys/auth references
This commit is contained in:
ars9
2026-04-10 20:40:20 +03:00
parent 0e4a2e1819
commit 1627733701
5 changed files with 150 additions and 149 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+35 -35
View File
@@ -1,45 +1,43 @@
# Architecture # Architecture
``` ```
┌─────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────┐
│ Browser (React SPA — client/) │ │ Browser (React SPA - client/) │
│ │ │ │
│ EnvironmentsPage KeysPage │ │ Environments Credentials Snippets │
│ SessionsPage ScenariosPage │ │ Sessions Scenarios Runs │
└───────────────────┬─────────────────────┘ └─────────────────────────────┬────────────────────────┘
│ HTTP REST (:13000) │ HTTP REST (:13000)
│ │
MCP client (AI agent / VS Code Copilot) MCP client (AI agent / VS Code Copilot)
│ HTTP /mcp (Streamable MCP) │ HTTP /mcp (Streamable MCP)
▼ ▼
┌──────────────────────────────────────┐ ┌──────────────────────────────────────────────────────┐
│ NestJS app (server/src/) │ │ NestJS app (server/src/) │
│ │ │ │
│ McpService ← all MCP tools │ │ McpService all MCP tools │
│ │ │ SessionService session persistence │
│ AuthService login + sessions │ │ SessionContextService live browser contexts │
│ SessionService session CRUD │ │ EnvironmentService environment CRUD + import │
│ EnvironmentService environment CRUD │ │ CredentialService credential CRUD + import │
│ BrowserService ad-hoc browsing │ │ SnippetService snippet CRUD + import │
│ CodeExecutorService sandboxed JS │ │ BrowserService ad-hoc open/exec │
│ │ │ CodeExecutorService sandboxed JS runtime │
│ ScenarioService scenario CRUD │ │ ScenarioService scenarios/steps/runs CRUD │
│ ScenarioSchedulerService │ │ ScenarioSchedulerService @Interval(1 s) run worker │
│ └─ @Interval(1 s) tick loop │ └───────────────────────┬──────────────────────────────┘
└──────────────────┬───────────────────┘ │ better-sqlite3 (TypeORM)
│ better-sqlite3 (TypeORM) ▼
▼ data/sessions.db
data/sessions.db
``` ```
- **McpService** — registers every MCP tool and delegates to domain services. - **McpService** registers MCP tools and delegates to session, environment, browser, and scenario services.
- **AuthService** — drives a Playwright browser to log in via a key descriptor; saves cookies + localStorage as a `Session`. - **SessionService** persists session records in SQLite. **SessionContextService** manages live Playwright contexts for open sessions.
- **SessionService** — stores and retrieves saved browser sessions. - **EnvironmentService**, **CredentialService**, and **SnippetService** provide CRUD plus import/export upserts.
- **EnvironmentService** — stores named environments (URL maps) used by auth and scenarios. - **BrowserService** handles stateless ad-hoc automation (`/open`, `/exec`) and session-aware browsing.
- **BrowserService** — ad-hoc, stateless `open_url` / `exec_code` operations outside the scenario scheduler. - **CodeExecutorService** validates and executes user JS with `page`, `context`, and helper APIs.
- **CodeExecutorService** — compiles and runs user-supplied Playwright JS inside a sandboxed `async` function with `page`, `context`, and `helpers` in scope. - **ScenarioService** manages scenarios, steps, credential aliases, runs, logs, and scenario import/export.
- **ScenarioService** — CRUD for scenarios, steps, runs, and run logs; import/export. - **ScenarioSchedulerService** polls pending runs every second and executes step runs to completion.
- **ScenarioSchedulerService** — polls for pending step runs every second and executes them. All steps in the same run share **one browser instance** (see [scenario.md](scenario.md)).
## Client (`client/`) ## Client (`client/`)
@@ -47,4 +45,6 @@ A React 19 + TypeScript SPA served separately by Vite during development. In Doc
The client calls the server REST endpoints directly — there is no separate BFF layer. Routing uses HashRouter so the Vite proxy and server-side routes are never ambiguous. The client calls the server REST endpoints directly — there is no separate BFF layer. Routing uses HashRouter so the Vite proxy and server-side routes are never ambiguous.
The sidebar footer contains the theme switcher and the app version (injected from the workspace root `package.json` at build time).
See [development.md](development.md) for the full client tech stack and directory layout. See [development.md](development.md) for the full client tech stack and directory layout.
docs/development.md
+20 -8
View File
@@ -26,8 +26,10 @@ Copy `.env.example` to `.env` and fill in the required values before running.
| Variable | Default | Description | | Variable | Default | Description |
|---|---|---| |---|---|---|
| `PORT` | `3000` | HTTP port | | `PORT` | `3000` | HTTP port |
| `KEYS_DIR` | `keys` | Directory containing `*.json` key descriptors | | `KEYS_DIR` | `keys` | Directory for key material used by credential/snippet workflows |
| `DB_PATH` | `data/sessions.db` | SQLite database file | | `DB_PATH` | `data/sessions.db` | SQLite database file |
| `SESSION_IDLE_TIMEOUT_MINUTES` | `30` | Idle timeout before open sessions are closed |
| `SESSION_DELETE_CLOSED_DAYS` | `7` | Retention period for closed sessions |
| `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` | _(Playwright default)_ | Path to Chromium binary. Set automatically in Docker (`/usr/bin/chromium`). | | `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` | _(Playwright default)_ | Path to Chromium binary. Set automatically in Docker (`/usr/bin/chromium`). |
--- ---
@@ -59,7 +61,13 @@ The server listens on port **3000** by default.
npm -w client run dev npm -w client run dev
``` ```
The Vite dev server runs on **http://localhost:5173** and proxies all API paths (`/environments`, `/sessions`, `/scenarios`, `/keys`, `/login`, `/mcp`) to the server at `localhost:3000`. The Vite dev server runs on **http://localhost:5173** and proxies these API prefixes to `localhost:13000` (or `localhost:3000` when running server locally):
- `/environments`
- `/credentials`
- `/snippets`
- `/sessions`
- `/scenarios`
**Production build**: **Production build**:
@@ -201,7 +209,7 @@ docker compose logs -f
## Client overview ## Client overview
The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provides a browser UI for managing environments, sessions, keys, and scenarios. The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provides a browser UI for managing environments, credentials, snippets, sessions, scenarios, and runs.
### Tech stack ### Tech stack
@@ -221,11 +229,11 @@ The client (`client/`) is a React 19 + TypeScript SPA built with Vite. It provid
``` ```
client/ client/
src/ src/
api/ API client modules (environments, keys, sessions, scenarios) api/ API client modules (environments, credentials, snippets, sessions, scenarios)
hooks/ useTheme (light/dark persistence) hooks/ useTheme (light/dark persistence)
i18n/ i18next bootstrap + locales/en.json i18n/ i18next bootstrap + locales/en.json
pages/ EnvironmentsPage, KeysPage, SessionsPage, ScenariosPage pages/ Environment/Credential/Snippet/Session/Scenario/Run pages
ui/ Reusable component library ui/ Reusable component library
Badge, Breadcrumbs, Button, Card, Input, Select, Badge, Breadcrumbs, Button, Card, Input, Select,
SidePanel, Table, ThemeSwitcher, Timestamp SidePanel, Table, ThemeSwitcher, Timestamp
.storybook/ .storybook/
@@ -239,10 +247,14 @@ Hash-based routing (`/#/path`) avoids conflicts with the Vite dev-server proxy.
| Route | Page | | Route | Page |
|---|---| |---|---|
| `/#/environments` | Environment list | | `/#/environments` | Environment list |
| `/#/keys` | Key list | | `/#/credentials` | Credential list |
| `/#/snippets` | Snippet list |
| `/#/sessions` | Session list | | `/#/sessions` | Session list |
| `/#/scenarios` | Scenario list | | `/#/scenarios` | Scenario list |
| `/#/runs` | Cross-scenario runs list |
### Theming ### Theming
Light/dark theme is toggled by the `ThemeSwitcher` button in the sidebar header. The selection is persisted to `localStorage` and applied as `data-theme` on `<html>` before React mounts (anti-FOUC inline script in `index.html`). Light/dark theme is toggled by the `ThemeSwitcher` button in the sidebar footer. The selection is persisted to `localStorage` and applied as `data-theme` on `<html>` before React mounts (anti-FOUC inline script in `index.html`).
The sidebar footer also shows the app version read from the workspace root `package.json` and injected via Vite `define`.
docs/key-descriptors.md
-20
View File
@@ -1,20 +0,0 @@
# Key descriptors
Each file in `KEYS_DIR` is named `<keyId>.json` and takes one of two forms:
**File key (EDS):**
```json
{
"keyFile": "path/to/key.dat",
"password": "secret"
}
```
`keyFile` is resolved relative to `KEYS_DIR`.
**Login + password:**
```json
{
"login": "user@example.com",
"password": "secret"
}
```
docs/mcp.md
+35 -28
View File
@@ -1,49 +1,56 @@
# MCP tools # MCP tools
## Auth / session The MCP endpoint is exposed at `/mcp` (Streamable HTTP transport).
The server currently registers the following tools.
## Sessions
| Tool | Description | | Tool | Description |
|---|---| |---|---|
| `list_keys` | List available key IDs from `KEYS_DIR` | | `list_sessions` | List stored sessions (paginated) |
| `login` | Log in with a key against an environment; stores resulting session | | `delete_session` | Delete a session by UUID |
| `list_sessions` | Paginated list of saved sessions |
| `delete_session` | Delete a session by ID |
## Environments ## Environments
| Tool | Description | | Tool | Description |
|---|---| |---|---|
| `list_environments` | Paginated list of environments | | `list_environments` | List environments (paginated) |
| `get_environment` | Get one environment by ID | | `get_environment` | Get one environment by UUID |
| `create_environment` | Create an environment (`name` + `urls` map) | | `create_environment` | Create environment (`name`, `urls`) |
| `update_environment` | Update name / urls | | `update_environment` | Update environment (`id`, optional `name`/`urls`) |
| `delete_environment` | Delete by ID | | `delete_environment` | Delete environment by UUID |
`urls` is a free-form map. Recognised keys: `id_url` (auth server), `cabinet_url` (cabinet redirect target), `admin_url`. `urls` is a free-form key/value map. Common keys: `id_url`, `cabinet_url`, `admin_url`.
## Browser (ad-hoc, stateless) ## Browser (ad-hoc)
| Tool | Description | | Tool | Description |
|---|---| |---|---|
| `open_url` | Navigate to a URL and return page content / reader-mode text | | `open_url` | Open URL with optional `sessionName`, return page/title/content |
| `exec_code` | Run Playwright JS in a session's browser context | | `exec_code` | Execute Playwright JS with `page` and `context` in scope |
## Scenarios ## Scenarios
| Tool | Description | | Tool | Description |
|---|---| |---|---|
| `list_scenarios` | Paginated list | | `list_scenarios` | List scenarios (paginated) |
| `get_scenario` | Fetch scenario with all steps | | `get_scenario` | Get scenario with steps |
| `create_scenario` | Create a new scenario by name | | `create_scenario` | Create scenario by `name` |
| `update_scenario` | Rename | | `update_scenario` | Rename scenario |
| `delete_scenario` | Delete (cascades to steps and runs) | | `delete_scenario` | Delete scenario |
| `create_scenario_step` | Add a step | | `create_scenario_step` | Add step to scenario |
| `get_scenario_step` | Fetch one step | | `get_scenario_step` | Get one step |
| `update_scenario_step` | Update step fields | | `update_scenario_step` | Update step fields |
| `delete_scenario_step` | Remove a step | | `delete_scenario_step` | Delete step |
| `list_scenario_runs` | Paginated run history (with optional status filter) | | `list_scenario_runs` | List runs for a scenario |
| `run_scenario` | Trigger an immediate run | | `run_scenario` | Trigger run for scenario |
| `get_scenario_run` | Get a run with all step runs and their outputs | | `get_scenario_run` | Get run with step runs and logs |
| `wait_for_scenario_run` | Block until run reaches `pass`/`fail`, return full run | | `wait_for_scenario_run` | Wait until run is `pass` or `fail` |
| `export_scenario` | Export scenario as a portable JSON payload | | `export_scenario` | Export scenario payload |
| `import_scenario` | Import a scenario from an export payload | | `import_scenario` | Import scenario payload |
## Notes
- Tool IDs and entity IDs are UUIDs.
- `create_scenario_step` schema still accepts legacy `type`/`sessionName` fields for compatibility. Current scheduler executes `execCode` plus optional `validateCode` and does not branch by step type.
docs/scenario.md
+60 -58
View File
@@ -1,80 +1,82 @@
# Scenario execution # Scenario execution
When a scenario is run, the scheduler creates a `ScenarioRun` with one `ScenarioRunStep` per step. Steps are executed **in order** (by `step.order`), one per scheduler tick. When a scenario run is created, the server creates one `ScenarioRunStep` per scenario step.
**All steps in the same run share a single Playwright browser instance.** The browser is created on the first `exec` or `sign` step (using session cookies + localStorage), reused for every subsequent step, and closed when the run finishes (pass or fail). - Step runs are ordered by `step.order`.
- First step run starts as `pending`; the rest start as `waiting`.
- Scheduler tick (`@Interval(1000)`) picks pending runs and processes steps in order.
- Steps in the same run share one Playwright browser/context/page.
## Step types ## Execution model
### `login` Current scheduler behavior is exec-centric:
Authenticates in a **separate, ephemeral browser** and saves the resulting session to the database. Subsequent exec/sign steps load that session into the shared browser. - `execCode` is required for a step to pass.
- `validateCode` is optional and runs after `execCode`.
- Legacy step type metadata may still appear in MCP payloads, but run execution is not branched by step type.
`execCode` must be a JSON object: ## Script runtime
```json
{ "keyId": "3137411915", "environmentName": "liquio-diia-stg" }
```
### `exec` `execCode` and `validateCode` execute as async JavaScript with:
Runs arbitrary Playwright JS on the **shared page**. The page carries over state from the previous step (URL, form inputs, cookies, etc.). - `page`: Playwright `Page`
- `context`: Playwright `BrowserContext`
- `helpers`: utility object
- `console`: proxied to run logs (`log`, `warn`, `error`, etc.)
`execCode` is executed as the body of an `async` function with these variables in scope: `validateCode` additionally receives `result`, which is the value returned by `execCode`.
- `page` — current Playwright `Page` ## Available helpers
- `context` — current Playwright `BrowserContext`
- `helpers.dumpDom(selector?)` — returns a simplified DOM string
- `helpers.getStepOutput(order)` — returns the JSON-parsed `output` of a previous step. Positive values are absolute step orders; negative values are relative to the current step (`-1` = previous step).
Any value returned from `execCode` is JSON-serialised and stored as the `output` field on the `ScenarioRunStep` record. Subsequent steps can read it via `helpers.getStepOutput()`. - `helpers.dumpDom(selector?)`: simplified DOM snapshot
- `helpers.log(...args)`, `helpers.warn(...args)`, `helpers.error(...args)`: structured step logs
- `helpers.getStepOutput(order)`: prior step output by absolute order (`0`, `1`, ...) or relative (`-1` previous step)
- `helpers.getCredential(alias)`: credential payload assigned to the scenario alias
- `helpers.env`: shallow copy of environment URL map
- `helpers.getEnvUrl(key)`: required environment URL lookup (throws if missing)
- `helpers.runSnippet(name, ...args)`: execute stored snippet code in the same page/context/helpers scope
## Validation contract
`validateCode` may return:
```js ```js
// example execCode return { success: true, description: 'ok' };
await page.goto('https://cabinet.example.com/tasks/create/987825');
await page.locator('input[name="institution"]').fill('caltech');
await page.locator('button:has-text("Далі")').click();
await page.waitForLoadState('networkidle');
return page.url(); // stored in ScenarioRunStep.output
```
### `sign`
Performs EDS signing on the **current shared page** using a key from `KEYS_DIR`. The step clicks the sign widget, selects the file key method, uploads the key file, enters the password, and submits — all without requiring the step author to handle key material in their code.
`execCode` must be a JSON object:
```json
{ "keyId": "3137411915" }
```
## Validation code
Any step type optionally accepts `validateCode` — Playwright JS that runs on the shared page **after** the main action and must return a result evaluated as:
```js
return { success: true, description: 'Application created' };
// or // or
return { success: false, description: 'Expected success banner not found' }; return false;
// or simply
return someBoolean;
``` ```
If `success` is false the step (and the whole run) fails with the provided `description`. Interpretation:
## Run statuses - Boolean: `true` = pass, `false` = fail
- Object: `success` controls pass/fail; `description` is persisted as step description
`pending` → `in_progress` → `pass` | `fail` On validation failure, the step fails and the run is marked `fail`.
Individual step runs also expose `waiting` (not yet reached) and `cancelled` (run failed before this step). ## Statuses
## Adding a new scenario (example workflow) Run status:
``` - `pending`
1. create_environment (if not already present) - `in_progress`
2. create_scenario → get scenarioId - `pass`
3. create_scenario_step type=login, order=0 - `fail`
4. create_scenario_step type=exec, order=1 (navigate + fill form)
5. create_scenario_step type=sign, order=2 Run-step status:
6. create_scenario_step type=exec, order=3 (validateCode checks success banner)
7. run_scenario → get runId - `waiting`
8. wait_for_scenario_run (blocks until pass/fail, returns full run with outputs) - `pending`
``` - `in_progress`
- `pass`
- `fail`
- `cancelled`
## Example workflow
1. Create environment
2. Create credentials and snippets (optional but common)
3. Create scenario
4. Add ordered steps (`execCode`, optional `validateCode`)
5. Assign credentials to scenario aliases when needed
6. Run scenario
7. Wait for run completion and inspect run logs/outputs