feat(scenario): shared browser per run, sign step type, key loading helper
- all steps in a run share one browser/page instance so wizard state persists
- add 'sign' step type: execCode is {keyId} and signs via EDS widget on current page
- extract AuthService.loadKeyDescriptor() to deduplicate key file parsing
- extend StepType, DTOs, and MCP enums to include 'sign'
This commit is contained in:
@@ -0,0 +1,229 @@
|
||||
# liquio-qa-bot
|
||||
|
||||
A headless QA automation service for Liquio environments. Exposes an **MCP (Model Context Protocol)** server that lets AI agents define, run, and monitor multi-step browser test scenarios using [Playwright](https://playwright.dev/).
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The bot manages four core concepts:
|
||||
|
||||
| Concept | Description |
|
||||
|---|---|
|
||||
| **Environment** | A named set of URLs for a deployment target (e.g. `liquio-diia-stg`) |
|
||||
| **Key** | A JSON descriptor in `keys/` that points to an EDS key file + password |
|
||||
| **Session** | Saved browser state (cookies + localStorage) produced by a login |
|
||||
| **Scenario** | An ordered list of steps that run in sequence against a shared browser instance |
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
MCP client (AI agent / VS Code Copilot)
|
||||
│ HTTP (Streamable MCP)
|
||||
▼
|
||||
┌──────────────────────────────────┐
|
||||
│ NestJS app (src/) │
|
||||
│ │
|
||||
│ McpService ← registers │
|
||||
│ AuthService all tools │
|
||||
│ BrowserService │
|
||||
│ ScenarioService │
|
||||
│ ScenarioSchedulerService │
|
||||
│ └─ @Interval(1s) tick loop │
|
||||
└──────────┬───────────────────────┘
|
||||
│ better-sqlite3 (TypeORM)
|
||||
▼
|
||||
data/sessions.db
|
||||
```
|
||||
|
||||
- **McpService** — registers every MCP tool and delegates to domain services.
|
||||
- **AuthService** — logs in via Playwright using a key descriptor, saves the resulting session.
|
||||
- **ScenarioSchedulerService** — polls for pending step runs every second and executes them. All steps belonging to the same run share **one browser instance** (see [Scenario execution](#scenario-execution)).
|
||||
- **CodeExecutorService** — compiles and runs user-supplied Playwright JS inside a sandboxed `async` function with `page`, `context`, and `helpers` in scope.
|
||||
|
||||
---
|
||||
|
||||
## Environment variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `PORT` | `3000` | HTTP port |
|
||||
| `KEYS_DIR` | `keys` | Directory containing `*.json` key descriptors |
|
||||
| `DB_PATH` | `data/sessions.db` | SQLite database file |
|
||||
| `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` | _(Playwright default)_ | Path to Chromium binary. Set automatically in Docker (`/usr/bin/chromium`). |
|
||||
|
||||
Copy `.env.example` to `.env` and adjust.
|
||||
|
||||
---
|
||||
|
||||
## Running locally
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npx playwright install chromium # first time only
|
||||
npm run start:dev
|
||||
```
|
||||
|
||||
The MCP endpoint is available at `http://localhost:3000/mcp`.
|
||||
|
||||
With Docker:
|
||||
|
||||
```bash
|
||||
docker build -t liquio-qa-bot .
|
||||
docker run -p 3000:3000 -v $(pwd)/keys:/app/keys -v $(pwd)/data:/app/data liquio-qa-bot
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key descriptors
|
||||
|
||||
Each file in `KEYS_DIR` is named `<keyId>.json` and takes one of two forms:
|
||||
|
||||
**File key (EDS):**
|
||||
```json
|
||||
{
|
||||
"keyFile": "path/to/key.dat",
|
||||
"password": "secret"
|
||||
}
|
||||
```
|
||||
`keyFile` is resolved relative to `KEYS_DIR`.
|
||||
|
||||
**Login + password:**
|
||||
```json
|
||||
{
|
||||
"login": "user@example.com",
|
||||
"password": "secret"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## MCP tools
|
||||
|
||||
### Auth / session
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `list_keys` | List available key IDs from `KEYS_DIR` |
|
||||
| `login` | Log in with a key against an environment; stores resulting session |
|
||||
| `list_sessions` | Paginated list of saved sessions |
|
||||
| `delete_session` | Delete a session by ID |
|
||||
|
||||
### Environments
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `list_environments` | Paginated list of environments |
|
||||
| `get_environment` | Get one environment by ID |
|
||||
| `create_environment` | Create an environment (`name` + `urls` map) |
|
||||
| `update_environment` | Update name / urls |
|
||||
| `delete_environment` | Delete by ID |
|
||||
|
||||
`urls` is a free-form map. Recognised keys: `id_url` (auth server), `cabinet_url` (cabinet redirect target), `admin_url`.
|
||||
|
||||
### Browser (ad-hoc, stateless)
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `open_url` | Navigate to a URL and return page content / reader-mode text |
|
||||
| `exec_code` | Run Playwright JS in a session's browser context |
|
||||
|
||||
### Scenarios
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `list_scenarios` | Paginated list |
|
||||
| `get_scenario` | Fetch scenario with all steps |
|
||||
| `create_scenario` | Create a new scenario by name |
|
||||
| `update_scenario` | Rename |
|
||||
| `delete_scenario` | Delete (cascades to steps and runs) |
|
||||
| `create_scenario_step` | Add a step |
|
||||
| `get_scenario_step` | Fetch one step |
|
||||
| `update_scenario_step` | Update step fields |
|
||||
| `delete_scenario_step` | Remove a step |
|
||||
| `list_scenario_runs` | Paginated run history (with optional status filter) |
|
||||
| `run_scenario` | Trigger an immediate run |
|
||||
|
||||
---
|
||||
|
||||
## Scenario execution
|
||||
|
||||
When a scenario is run, the scheduler creates a `ScenarioRun` with one `ScenarioRunStep` per step. Steps are executed **in order** (by `step.order`), one per scheduler tick.
|
||||
|
||||
**All steps in the same run share a single Playwright browser instance.** The browser is created on the first `exec` or `sign` step (using session cookies + localStorage), reused for every subsequent step, and closed when the run finishes (pass or fail).
|
||||
|
||||
### Step types
|
||||
|
||||
#### `login`
|
||||
|
||||
Authenticates in a **separate, ephemeral browser** and saves the resulting session to the database. Subsequent exec/sign steps load that session into the shared browser.
|
||||
|
||||
`execCode` must be a JSON object:
|
||||
```json
|
||||
{ "keyId": "3137411915", "environmentName": "liquio-diia-stg" }
|
||||
```
|
||||
|
||||
#### `exec`
|
||||
|
||||
Runs arbitrary Playwright JS on the **shared page**. The page carries over state from the previous step (URL, form inputs, cookies, etc.).
|
||||
|
||||
`execCode` is executed as the body of an `async` function with these variables in scope:
|
||||
|
||||
- `page` — current Playwright `Page`
|
||||
- `context` — current Playwright `BrowserContext`
|
||||
- `helpers.dumpDom(selector?)` — returns a simplified DOM string
|
||||
|
||||
```js
|
||||
// example execCode
|
||||
await page.goto('https://cabinet.example.com/tasks/create/987825');
|
||||
await page.locator('input[name="institution"]').fill('caltech');
|
||||
await page.locator('button:has-text("Далі")').click();
|
||||
await page.waitForLoadState('networkidle');
|
||||
return page.url();
|
||||
```
|
||||
|
||||
#### `sign`
|
||||
|
||||
Performs EDS signing on the **current shared page** using a key from `KEYS_DIR`. The step clicks the sign widget, selects the file key method, uploads the key file, enters the password, and submits — all without requiring the step author to handle key material in their code.
|
||||
|
||||
`execCode` must be a JSON object:
|
||||
```json
|
||||
{ "keyId": "3137411915" }
|
||||
```
|
||||
|
||||
### Validation code
|
||||
|
||||
Any step type optionally accepts `validateCode` — Playwright JS that runs on the shared page **after** the main action and must return a result evaluated as:
|
||||
|
||||
```js
|
||||
return { success: true, description: 'Application created' };
|
||||
// or
|
||||
return { success: false, description: 'Expected success banner not found' };
|
||||
// or simply
|
||||
return someBoolean;
|
||||
```
|
||||
|
||||
If `success` is false the step (and the whole run) fails with the provided `description`.
|
||||
|
||||
### Run statuses
|
||||
|
||||
`pending` → `in_progress` → `pass` | `fail`
|
||||
|
||||
Individual step runs also expose `waiting` (not yet reached) and `cancelled` (run failed before this step).
|
||||
|
||||
---
|
||||
|
||||
## Adding a new scenario (example workflow)
|
||||
|
||||
```
|
||||
1. create_environment (if not already present)
|
||||
2. create_scenario → get scenarioId
|
||||
3. create_scenario_step type=login, order=0
|
||||
4. create_scenario_step type=exec, order=1 (navigate + fill form)
|
||||
5. create_scenario_step type=sign, order=2
|
||||
6. create_scenario_step type=exec, order=3 (validateCode checks success banner)
|
||||
7. run_scenario
|
||||
8. list_scenario_runs (poll until status != in_progress)
|
||||
```
|
||||
Reference in New Issue
Block a user