feat(scenario): step output, getStepOutput helper, export/import MCP tools, doc restructure
- add output column to ScenarioRunStepEntity; exec step return value is JSON-serialised and stored - expose helpers.getStepOutput(order) in exec code; negative order is relative to current step - add get_scenario_run, wait_for_scenario_run, export_scenario, import_scenario MCP tools - move Architecture, Key descriptors, MCP tools, Scenario, Development docs to docs/ - delete CONTRIBUTING.md (replaced by docs/development.md)
This commit is contained in:
@@ -19,211 +19,28 @@ The bot manages four core concepts:
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
MCP client (AI agent / VS Code Copilot)
|
||||
│ HTTP (Streamable MCP)
|
||||
▼
|
||||
┌──────────────────────────────────┐
|
||||
│ NestJS app (src/) │
|
||||
│ │
|
||||
│ McpService ← registers │
|
||||
│ AuthService all tools │
|
||||
│ BrowserService │
|
||||
│ ScenarioService │
|
||||
│ ScenarioSchedulerService │
|
||||
│ └─ @Interval(1s) tick loop │
|
||||
└──────────┬───────────────────────┘
|
||||
│ better-sqlite3 (TypeORM)
|
||||
▼
|
||||
data/sessions.db
|
||||
```
|
||||
|
||||
- **McpService** — registers every MCP tool and delegates to domain services.
|
||||
- **AuthService** — logs in via Playwright using a key descriptor, saves the resulting session.
|
||||
- **ScenarioSchedulerService** — polls for pending step runs every second and executes them. All steps belonging to the same run share **one browser instance** (see [Scenario execution](#scenario-execution)).
|
||||
- **CodeExecutorService** — compiles and runs user-supplied Playwright JS inside a sandboxed `async` function with `page`, `context`, and `helpers` in scope.
|
||||
|
||||
---
|
||||
|
||||
## Environment variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `PORT` | `3000` | HTTP port |
|
||||
| `KEYS_DIR` | `keys` | Directory containing `*.json` key descriptors |
|
||||
| `DB_PATH` | `data/sessions.db` | SQLite database file |
|
||||
| `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` | _(Playwright default)_ | Path to Chromium binary. Set automatically in Docker (`/usr/bin/chromium`). |
|
||||
|
||||
Copy `.env.example` to `.env` and adjust.
|
||||
|
||||
---
|
||||
|
||||
## Running locally
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npx playwright install chromium # first time only
|
||||
npm run start:dev
|
||||
```
|
||||
|
||||
The MCP endpoint is available at `http://localhost:3000/mcp`.
|
||||
|
||||
With Docker:
|
||||
|
||||
```bash
|
||||
docker build -t liquio-qa-bot .
|
||||
docker run -p 3000:3000 -v $(pwd)/keys:/app/keys -v $(pwd)/data:/app/data liquio-qa-bot
|
||||
```
|
||||
See [docs/architecture.md](docs/architecture.md).
|
||||
|
||||
---
|
||||
|
||||
## Key descriptors
|
||||
|
||||
Each file in `KEYS_DIR` is named `<keyId>.json` and takes one of two forms:
|
||||
|
||||
**File key (EDS):**
|
||||
```json
|
||||
{
|
||||
"keyFile": "path/to/key.dat",
|
||||
"password": "secret"
|
||||
}
|
||||
```
|
||||
`keyFile` is resolved relative to `KEYS_DIR`.
|
||||
|
||||
**Login + password:**
|
||||
```json
|
||||
{
|
||||
"login": "user@example.com",
|
||||
"password": "secret"
|
||||
}
|
||||
```
|
||||
See [docs/key-descriptors.md](docs/key-descriptors.md).
|
||||
|
||||
---
|
||||
|
||||
## MCP tools
|
||||
|
||||
### Auth / session
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `list_keys` | List available key IDs from `KEYS_DIR` |
|
||||
| `login` | Log in with a key against an environment; stores resulting session |
|
||||
| `list_sessions` | Paginated list of saved sessions |
|
||||
| `delete_session` | Delete a session by ID |
|
||||
|
||||
### Environments
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `list_environments` | Paginated list of environments |
|
||||
| `get_environment` | Get one environment by ID |
|
||||
| `create_environment` | Create an environment (`name` + `urls` map) |
|
||||
| `update_environment` | Update name / urls |
|
||||
| `delete_environment` | Delete by ID |
|
||||
|
||||
`urls` is a free-form map. Recognised keys: `id_url` (auth server), `cabinet_url` (cabinet redirect target), `admin_url`.
|
||||
|
||||
### Browser (ad-hoc, stateless)
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `open_url` | Navigate to a URL and return page content / reader-mode text |
|
||||
| `exec_code` | Run Playwright JS in a session's browser context |
|
||||
|
||||
### Scenarios
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `list_scenarios` | Paginated list |
|
||||
| `get_scenario` | Fetch scenario with all steps |
|
||||
| `create_scenario` | Create a new scenario by name |
|
||||
| `update_scenario` | Rename |
|
||||
| `delete_scenario` | Delete (cascades to steps and runs) |
|
||||
| `create_scenario_step` | Add a step |
|
||||
| `get_scenario_step` | Fetch one step |
|
||||
| `update_scenario_step` | Update step fields |
|
||||
| `delete_scenario_step` | Remove a step |
|
||||
| `list_scenario_runs` | Paginated run history (with optional status filter) |
|
||||
| `run_scenario` | Trigger an immediate run |
|
||||
See [docs/mcp.md](docs/mcp.md).
|
||||
|
||||
---
|
||||
|
||||
## Scenario execution
|
||||
|
||||
When a scenario is run, the scheduler creates a `ScenarioRun` with one `ScenarioRunStep` per step. Steps are executed **in order** (by `step.order`), one per scheduler tick.
|
||||
|
||||
**All steps in the same run share a single Playwright browser instance.** The browser is created on the first `exec` or `sign` step (using session cookies + localStorage), reused for every subsequent step, and closed when the run finishes (pass or fail).
|
||||
|
||||
### Step types
|
||||
|
||||
#### `login`
|
||||
|
||||
Authenticates in a **separate, ephemeral browser** and saves the resulting session to the database. Subsequent exec/sign steps load that session into the shared browser.
|
||||
|
||||
`execCode` must be a JSON object:
|
||||
```json
|
||||
{ "keyId": "3137411915", "environmentName": "liquio-diia-stg" }
|
||||
```
|
||||
|
||||
#### `exec`
|
||||
|
||||
Runs arbitrary Playwright JS on the **shared page**. The page carries over state from the previous step (URL, form inputs, cookies, etc.).
|
||||
|
||||
`execCode` is executed as the body of an `async` function with these variables in scope:
|
||||
|
||||
- `page` — current Playwright `Page`
|
||||
- `context` — current Playwright `BrowserContext`
|
||||
- `helpers.dumpDom(selector?)` — returns a simplified DOM string
|
||||
|
||||
```js
|
||||
// example execCode
|
||||
await page.goto('https://cabinet.example.com/tasks/create/987825');
|
||||
await page.locator('input[name="institution"]').fill('caltech');
|
||||
await page.locator('button:has-text("Далі")').click();
|
||||
await page.waitForLoadState('networkidle');
|
||||
return page.url();
|
||||
```
|
||||
|
||||
#### `sign`
|
||||
|
||||
Performs EDS signing on the **current shared page** using a key from `KEYS_DIR`. The step clicks the sign widget, selects the file key method, uploads the key file, enters the password, and submits — all without requiring the step author to handle key material in their code.
|
||||
|
||||
`execCode` must be a JSON object:
|
||||
```json
|
||||
{ "keyId": "3137411915" }
|
||||
```
|
||||
|
||||
### Validation code
|
||||
|
||||
Any step type optionally accepts `validateCode` — Playwright JS that runs on the shared page **after** the main action and must return a result evaluated as:
|
||||
|
||||
```js
|
||||
return { success: true, description: 'Application created' };
|
||||
// or
|
||||
return { success: false, description: 'Expected success banner not found' };
|
||||
// or simply
|
||||
return someBoolean;
|
||||
```
|
||||
|
||||
If `success` is false the step (and the whole run) fails with the provided `description`.
|
||||
|
||||
### Run statuses
|
||||
|
||||
`pending` → `in_progress` → `pass` | `fail`
|
||||
|
||||
Individual step runs also expose `waiting` (not yet reached) and `cancelled` (run failed before this step).
|
||||
See [docs/scenario.md](docs/scenario.md).
|
||||
|
||||
---
|
||||
|
||||
## Adding a new scenario (example workflow)
|
||||
## Development
|
||||
|
||||
```
|
||||
1. create_environment (if not already present)
|
||||
2. create_scenario → get scenarioId
|
||||
3. create_scenario_step type=login, order=0
|
||||
4. create_scenario_step type=exec, order=1 (navigate + fill form)
|
||||
5. create_scenario_step type=sign, order=2
|
||||
6. create_scenario_step type=exec, order=3 (validateCode checks success banner)
|
||||
7. run_scenario
|
||||
8. list_scenario_runs (poll until status != in_progress)
|
||||
```
|
||||
See [docs/development.md](docs/development.md) for setup, running, testing, linting, and Docker instructions.
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
# Architecture
|
||||
|
||||
```
|
||||
MCP client (AI agent / VS Code Copilot)
|
||||
│ HTTP /mcp (Streamable MCP)
|
||||
▼
|
||||
┌──────────────────────────────────────┐
|
||||
│ NestJS app (src/) │
|
||||
│ │
|
||||
│ McpService ← all MCP tools │
|
||||
│ │
|
||||
│ AuthService login + sessions │
|
||||
│ SessionService session CRUD │
|
||||
│ EnvironmentService environment CRUD │
|
||||
│ BrowserService ad-hoc browsing │
|
||||
│ CodeExecutorService sandboxed JS │
|
||||
│ │
|
||||
│ ScenarioService scenario CRUD │
|
||||
│ ScenarioSchedulerService │
|
||||
│ └─ @Interval(1 s) tick loop │
|
||||
└──────────────────┬───────────────────┘
|
||||
│ better-sqlite3 (TypeORM)
|
||||
▼
|
||||
data/sessions.db
|
||||
```
|
||||
|
||||
- **McpService** — registers every MCP tool and delegates to domain services.
|
||||
- **AuthService** — drives a Playwright browser to log in via a key descriptor; saves cookies + localStorage as a `Session`.
|
||||
- **SessionService** — stores and retrieves saved browser sessions.
|
||||
- **EnvironmentService** — stores named environments (URL maps) used by auth and scenarios.
|
||||
- **BrowserService** — ad-hoc, stateless `open_url` / `exec_code` operations outside the scenario scheduler.
|
||||
- **CodeExecutorService** — compiles and runs user-supplied Playwright JS inside a sandboxed `async` function with `page`, `context`, and `helpers` in scope.
|
||||
- **ScenarioService** — CRUD for scenarios, steps, runs, and run logs; import/export.
|
||||
- **ScenarioSchedulerService** — polls for pending step runs every second and executes them. All steps in the same run share **one browser instance** (see [scenario.md](scenario.md)).
|
||||
@@ -1,4 +1,4 @@
|
||||
# Contributing
|
||||
# Development
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -9,10 +9,20 @@ Install dependencies:
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npx playwright install chromium # first time only
|
||||
```
|
||||
|
||||
Copy `.env.example` to `.env` and fill in the required values before running.
|
||||
|
||||
### Environment variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `PORT` | `3000` | HTTP port |
|
||||
| `KEYS_DIR` | `keys` | Directory containing `*.json` key descriptors |
|
||||
| `DB_PATH` | `data/sessions.db` | SQLite database file |
|
||||
| `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` | _(Playwright default)_ | Path to Chromium binary. Set automatically in Docker (`/usr/bin/chromium`). |
|
||||
|
||||
---
|
||||
|
||||
## Starting the application
|
||||
@@ -0,0 +1,20 @@
|
||||
# Key descriptors
|
||||
|
||||
Each file in `KEYS_DIR` is named `<keyId>.json` and takes one of two forms:
|
||||
|
||||
**File key (EDS):**
|
||||
```json
|
||||
{
|
||||
"keyFile": "path/to/key.dat",
|
||||
"password": "secret"
|
||||
}
|
||||
```
|
||||
`keyFile` is resolved relative to `KEYS_DIR`.
|
||||
|
||||
**Login + password:**
|
||||
```json
|
||||
{
|
||||
"login": "user@example.com",
|
||||
"password": "secret"
|
||||
}
|
||||
```
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
# MCP tools
|
||||
|
||||
## Auth / session
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `list_keys` | List available key IDs from `KEYS_DIR` |
|
||||
| `login` | Log in with a key against an environment; stores resulting session |
|
||||
| `list_sessions` | Paginated list of saved sessions |
|
||||
| `delete_session` | Delete a session by ID |
|
||||
|
||||
## Environments
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `list_environments` | Paginated list of environments |
|
||||
| `get_environment` | Get one environment by ID |
|
||||
| `create_environment` | Create an environment (`name` + `urls` map) |
|
||||
| `update_environment` | Update name / urls |
|
||||
| `delete_environment` | Delete by ID |
|
||||
|
||||
`urls` is a free-form map. Recognised keys: `id_url` (auth server), `cabinet_url` (cabinet redirect target), `admin_url`.
|
||||
|
||||
## Browser (ad-hoc, stateless)
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `open_url` | Navigate to a URL and return page content / reader-mode text |
|
||||
| `exec_code` | Run Playwright JS in a session's browser context |
|
||||
|
||||
## Scenarios
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `list_scenarios` | Paginated list |
|
||||
| `get_scenario` | Fetch scenario with all steps |
|
||||
| `create_scenario` | Create a new scenario by name |
|
||||
| `update_scenario` | Rename |
|
||||
| `delete_scenario` | Delete (cascades to steps and runs) |
|
||||
| `create_scenario_step` | Add a step |
|
||||
| `get_scenario_step` | Fetch one step |
|
||||
| `update_scenario_step` | Update step fields |
|
||||
| `delete_scenario_step` | Remove a step |
|
||||
| `list_scenario_runs` | Paginated run history (with optional status filter) |
|
||||
| `run_scenario` | Trigger an immediate run |
|
||||
| `get_scenario_run` | Get a run with all step runs and their outputs |
|
||||
| `wait_for_scenario_run` | Block until run reaches `pass`/`fail`, return full run |
|
||||
| `export_scenario` | Export scenario as a portable JSON payload |
|
||||
| `import_scenario` | Import a scenario from an export payload |
|
||||
@@ -0,0 +1,80 @@
|
||||
# Scenario execution
|
||||
|
||||
When a scenario is run, the scheduler creates a `ScenarioRun` with one `ScenarioRunStep` per step. Steps are executed **in order** (by `step.order`), one per scheduler tick.
|
||||
|
||||
**All steps in the same run share a single Playwright browser instance.** The browser is created on the first `exec` or `sign` step (using session cookies + localStorage), reused for every subsequent step, and closed when the run finishes (pass or fail).
|
||||
|
||||
## Step types
|
||||
|
||||
### `login`
|
||||
|
||||
Authenticates in a **separate, ephemeral browser** and saves the resulting session to the database. Subsequent exec/sign steps load that session into the shared browser.
|
||||
|
||||
`execCode` must be a JSON object:
|
||||
```json
|
||||
{ "keyId": "3137411915", "environmentName": "liquio-diia-stg" }
|
||||
```
|
||||
|
||||
### `exec`
|
||||
|
||||
Runs arbitrary Playwright JS on the **shared page**. The page carries over state from the previous step (URL, form inputs, cookies, etc.).
|
||||
|
||||
`execCode` is executed as the body of an `async` function with these variables in scope:
|
||||
|
||||
- `page` — current Playwright `Page`
|
||||
- `context` — current Playwright `BrowserContext`
|
||||
- `helpers.dumpDom(selector?)` — returns a simplified DOM string
|
||||
- `helpers.getStepOutput(order)` — returns the JSON-parsed `output` of a previous step. Positive values are absolute step orders; negative values are relative to the current step (`-1` = previous step).
|
||||
|
||||
Any value returned from `execCode` is JSON-serialised and stored as the `output` field on the `ScenarioRunStep` record. Subsequent steps can read it via `helpers.getStepOutput()`.
|
||||
|
||||
```js
|
||||
// example execCode
|
||||
await page.goto('https://cabinet.example.com/tasks/create/987825');
|
||||
await page.locator('input[name="institution"]').fill('caltech');
|
||||
await page.locator('button:has-text("Далі")').click();
|
||||
await page.waitForLoadState('networkidle');
|
||||
return page.url(); // stored in ScenarioRunStep.output
|
||||
```
|
||||
|
||||
### `sign`
|
||||
|
||||
Performs EDS signing on the **current shared page** using a key from `KEYS_DIR`. The step clicks the sign widget, selects the file key method, uploads the key file, enters the password, and submits — all without requiring the step author to handle key material in their code.
|
||||
|
||||
`execCode` must be a JSON object:
|
||||
```json
|
||||
{ "keyId": "3137411915" }
|
||||
```
|
||||
|
||||
## Validation code
|
||||
|
||||
Any step type optionally accepts `validateCode` — Playwright JS that runs on the shared page **after** the main action and must return a result evaluated as:
|
||||
|
||||
```js
|
||||
return { success: true, description: 'Application created' };
|
||||
// or
|
||||
return { success: false, description: 'Expected success banner not found' };
|
||||
// or simply
|
||||
return someBoolean;
|
||||
```
|
||||
|
||||
If `success` is false the step (and the whole run) fails with the provided `description`.
|
||||
|
||||
## Run statuses
|
||||
|
||||
`pending` → `in_progress` → `pass` | `fail`
|
||||
|
||||
Individual step runs also expose `waiting` (not yet reached) and `cancelled` (run failed before this step).
|
||||
|
||||
## Adding a new scenario (example workflow)
|
||||
|
||||
```
|
||||
1. create_environment (if not already present)
|
||||
2. create_scenario → get scenarioId
|
||||
3. create_scenario_step type=login, order=0
|
||||
4. create_scenario_step type=exec, order=1 (navigate + fill form)
|
||||
5. create_scenario_step type=sign, order=2
|
||||
6. create_scenario_step type=exec, order=3 (validateCode checks success banner)
|
||||
7. run_scenario → get runId
|
||||
8. wait_for_scenario_run (blocks until pass/fail, returns full run with outputs)
|
||||
```
|
||||
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"name": "Diia STG — Create certificate application (987825)",
|
||||
"steps": [
|
||||
{
|
||||
"order": 0,
|
||||
"type": "login",
|
||||
"sessionName": "diia-stg-session",
|
||||
"execCode": "{\"keyId\": \"3273334361\", \"environmentName\": \"liquio-diia-stg\"}",
|
||||
"validateCode": null
|
||||
},
|
||||
{
|
||||
"order": 1,
|
||||
"type": "exec",
|
||||
"sessionName": "diia-stg-session",
|
||||
"execCode": "// Navigate to the tasks/services list for this user\nawait page.goto(\"https://cabinet-liquio-diia-stg.kitsoft.ua/tasks\", { waitUntil: \"networkidle\" });\nhelpers.log(\"URL: \" + page.url());\nhelpers.log(\"title: \" + await page.title());\n\nconst headings = await page.locator(\"h1, h2, h3\").allTextContents();\nhelpers.log(\"headings: \" + JSON.stringify(headings));\n\n// Find any links that look like task/service links\nconst links = await page.locator(\"a[href*=task], a[href*=service], a[href*=create]\").all();\nconst linkData = [];\nfor (const a of links.slice(0, 20)) {\n linkData.push({ href: await a.getAttribute(\"href\"), text: (await a.textContent()).trim().slice(0, 80) });\n}\nhelpers.log(\"task links: \" + JSON.stringify(linkData));\n\nconst bodyText = await page.locator(\"body\").innerText();\nhelpers.log(\"body (first 800): \" + bodyText.slice(0, 800));\n\nreturn page.url();\n",
|
||||
"validateCode": null
|
||||
},
|
||||
{
|
||||
"order": 2,
|
||||
"type": "exec",
|
||||
"sessionName": "diia-stg-session",
|
||||
"execCode": "helpers.log(\"url=\" + page.url());\nconst btns = await page.locator(\"button\").all();\nconst labels = [];\nfor (const b of btns) { const t = (await b.textContent()) || \"\"; const vis = await b.isVisible(); if (vis) labels.push(t.trim()); }\nhelpers.log(\"visible buttons: \" + JSON.stringify(labels));\nreturn labels;",
|
||||
"validateCode": null
|
||||
},
|
||||
{
|
||||
"order": 3,
|
||||
"type": "sign",
|
||||
"sessionName": "diia-stg-session",
|
||||
"execCode": "{\"keyId\": \"3273334361\"}",
|
||||
"validateCode": null
|
||||
},
|
||||
{
|
||||
"order": 4,
|
||||
"type": "exec",
|
||||
"sessionName": "diia-stg-session",
|
||||
"execCode": "return page.url();",
|
||||
"validateCode": "\nconst url = page.url();\nconst successEl = await page.locator('.success, [data-test=\"success\"], .task-success, .done-message').count();\nconst successText = /успішно|success|заяву подано|виконано/i.test(await page.locator('body').innerText());\nreturn {\n success: successEl > 0 || successText,\n description: `Final URL: ${url}`,\n};\n"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -46,6 +46,7 @@ export class CodeExecutorService {
|
||||
context: BrowserContext,
|
||||
code: string,
|
||||
log?: ScriptLogger,
|
||||
getStepOutput?: (order: number) => Promise<unknown>,
|
||||
): Promise<ExecResult> {
|
||||
const scriptLog: ScriptLogger =
|
||||
log ?? ((level, msg) => this.logger[level](msg));
|
||||
@@ -60,6 +61,7 @@ export class CodeExecutorService {
|
||||
log: (...args: unknown[]) => scriptLog("log", toStr(args)),
|
||||
warn: (...args: unknown[]) => scriptLog("warn", toStr(args)),
|
||||
error: (...args: unknown[]) => scriptLog("error", toStr(args)),
|
||||
getStepOutput: getStepOutput ?? (() => Promise.resolve(null)),
|
||||
};
|
||||
|
||||
const fakeConsole = {
|
||||
|
||||
@@ -731,6 +731,134 @@ export class McpService {
|
||||
},
|
||||
);
|
||||
|
||||
server.registerTool(
|
||||
"get_scenario_run",
|
||||
{
|
||||
description:
|
||||
"Get a specific scenario run with all step runs and their outputs",
|
||||
inputSchema: {
|
||||
scenarioId: z.number().int().describe("Scenario ID"),
|
||||
runId: z.number().int().describe("Run ID"),
|
||||
},
|
||||
},
|
||||
async ({ scenarioId, runId }) => {
|
||||
try {
|
||||
const run = await this.scenarioService.findRun(scenarioId, runId);
|
||||
return {
|
||||
content: [{ type: "text" as const, text: JSON.stringify(run) }],
|
||||
};
|
||||
} catch (err) {
|
||||
return {
|
||||
isError: true,
|
||||
content: [{ type: "text" as const, text: (err as Error).message }],
|
||||
};
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
server.registerTool(
|
||||
"wait_for_scenario_run",
|
||||
{
|
||||
description:
|
||||
"Block until a scenario run reaches pass or fail (max 5 min), then return the full run with step outputs",
|
||||
inputSchema: {
|
||||
scenarioId: z.number().int().describe("Scenario ID"),
|
||||
runId: z.number().int().describe("Run ID"),
|
||||
},
|
||||
},
|
||||
async ({ scenarioId, runId }) => {
|
||||
try {
|
||||
const run = await this.scenarioService.waitForRun(scenarioId, runId);
|
||||
return {
|
||||
content: [{ type: "text" as const, text: JSON.stringify(run) }],
|
||||
};
|
||||
} catch (err) {
|
||||
return {
|
||||
isError: true,
|
||||
content: [{ type: "text" as const, text: (err as Error).message }],
|
||||
};
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
server.registerTool(
|
||||
"export_scenario",
|
||||
{
|
||||
description:
|
||||
"Export a scenario as a portable JSON payload (name + steps)",
|
||||
inputSchema: {
|
||||
id: z.number().int().describe("Scenario ID to export"),
|
||||
},
|
||||
},
|
||||
async ({ id }) => {
|
||||
try {
|
||||
const exported = await this.scenarioService.exportScenario(id);
|
||||
return {
|
||||
content: [
|
||||
{ type: "text" as const, text: JSON.stringify(exported) },
|
||||
],
|
||||
};
|
||||
} catch (err) {
|
||||
return {
|
||||
isError: true,
|
||||
content: [{ type: "text" as const, text: (err as Error).message }],
|
||||
};
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
server.registerTool(
|
||||
"import_scenario",
|
||||
{
|
||||
description:
|
||||
"Import a scenario from an export payload, creating a new scenario with all its steps",
|
||||
inputSchema: {
|
||||
name: z.string().describe("Scenario name"),
|
||||
steps: z
|
||||
.array(
|
||||
z.object({
|
||||
order: z.number().int().min(0).describe("Execution order"),
|
||||
type: z
|
||||
.enum(["login", "exec", "sign"])
|
||||
.describe("Step type"),
|
||||
sessionName: z.string().describe("Session name"),
|
||||
execCode: z
|
||||
.string()
|
||||
.nullable()
|
||||
.optional()
|
||||
.describe("Exec/sign code"),
|
||||
validateCode: z
|
||||
.string()
|
||||
.nullable()
|
||||
.optional()
|
||||
.describe("Validation code"),
|
||||
}),
|
||||
)
|
||||
.describe("Ordered list of steps"),
|
||||
},
|
||||
},
|
||||
async ({ name, steps }) => {
|
||||
try {
|
||||
const scenario = await this.scenarioService.importScenario({
|
||||
name,
|
||||
steps: steps as Parameters<
|
||||
typeof this.scenarioService.importScenario
|
||||
>[0]["steps"],
|
||||
});
|
||||
return {
|
||||
content: [
|
||||
{ type: "text" as const, text: JSON.stringify(scenario) },
|
||||
],
|
||||
};
|
||||
} catch (err) {
|
||||
return {
|
||||
isError: true,
|
||||
content: [{ type: "text" as const, text: (err as Error).message }],
|
||||
};
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
// ── Transport ─────────────────────────────────────────────────────────────
|
||||
}
|
||||
|
||||
|
||||
@@ -48,6 +48,9 @@ export class ScenarioRunStepEntity {
|
||||
@Column({ type: "text", nullable: true })
|
||||
description: string | null;
|
||||
|
||||
@Column({ type: "text", nullable: true })
|
||||
output: string | null;
|
||||
|
||||
@CreateDateColumn()
|
||||
createdAt: Date;
|
||||
|
||||
|
||||
@@ -63,6 +63,24 @@ export class ScenarioSchedulerService {
|
||||
};
|
||||
}
|
||||
|
||||
private makeGetStepOutput(
|
||||
stepRun: ScenarioRunStepEntity,
|
||||
): (order: number) => Promise<unknown> {
|
||||
return async (order: number) => {
|
||||
const targetOrder = order < 0 ? stepRun.order + order : order;
|
||||
if (targetOrder < 0) return null;
|
||||
const sr = await this.runStepRepo.findOne({
|
||||
where: { runId: stepRun.runId, order: targetOrder },
|
||||
});
|
||||
if (!sr?.output) return null;
|
||||
try {
|
||||
return JSON.parse(sr.output) as unknown;
|
||||
} catch {
|
||||
return sr.output;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
// ── Job: pick up pending runs and process each to completion ─────────────
|
||||
|
||||
@Interval(1000)
|
||||
@@ -223,6 +241,7 @@ export class ScenarioSchedulerService {
|
||||
context,
|
||||
step.validateCode,
|
||||
this.stepLogger(stepRun.id, stepRun.runId),
|
||||
this.makeGetStepOutput(stepRun),
|
||||
);
|
||||
const vr = this.parseValidateResult(result);
|
||||
if (!vr.success) throw new Error(vr.description ?? "Validation failed");
|
||||
@@ -244,11 +263,13 @@ export class ScenarioSchedulerService {
|
||||
stepRun.runId,
|
||||
step.sessionName,
|
||||
);
|
||||
await this.codeExecutor.execute(
|
||||
const getStepOutput = this.makeGetStepOutput(stepRun);
|
||||
const { result: execOutput } = await this.codeExecutor.execute(
|
||||
page,
|
||||
context,
|
||||
step.execCode,
|
||||
this.stepLogger(stepRun.id, stepRun.runId),
|
||||
getStepOutput,
|
||||
);
|
||||
this.logger.log(`StepRun #${stepRun.id}: exec OK`);
|
||||
|
||||
@@ -259,12 +280,13 @@ export class ScenarioSchedulerService {
|
||||
context,
|
||||
step.validateCode,
|
||||
this.stepLogger(stepRun.id, stepRun.runId),
|
||||
getStepOutput,
|
||||
);
|
||||
const vr = this.parseValidateResult(result);
|
||||
if (!vr.success) throw new Error(vr.description ?? "Validation failed");
|
||||
}
|
||||
|
||||
await this.passStepRun(stepRun, null);
|
||||
await this.passStepRun(stepRun, null, execOutput);
|
||||
}
|
||||
|
||||
// ── Sign step ──────────────────────────────────────────────────────────────
|
||||
@@ -296,6 +318,7 @@ export class ScenarioSchedulerService {
|
||||
context,
|
||||
step.validateCode,
|
||||
this.stepLogger(stepRun.id, stepRun.runId),
|
||||
this.makeGetStepOutput(stepRun),
|
||||
);
|
||||
const vr = this.parseValidateResult(result);
|
||||
if (!vr.success) throw new Error(vr.description ?? "Validation failed");
|
||||
@@ -324,9 +347,12 @@ export class ScenarioSchedulerService {
|
||||
private async passStepRun(
|
||||
stepRun: ScenarioRunStepEntity,
|
||||
description: string | null,
|
||||
output: unknown = null,
|
||||
): Promise<void> {
|
||||
stepRun.status = "pass";
|
||||
stepRun.description = description;
|
||||
stepRun.output =
|
||||
output !== null && output !== undefined ? JSON.stringify(output) : null;
|
||||
await this.runStepRepo.save(stepRun);
|
||||
this.logger.log(`StepRun #${stepRun.id} → pass`);
|
||||
|
||||
|
||||
@@ -1,15 +1,25 @@
|
||||
export const chromium = {
|
||||
launch: jest.fn().mockResolvedValue({
|
||||
newContext: jest.fn().mockResolvedValue({
|
||||
newPage: jest.fn().mockResolvedValue({
|
||||
const makePage = () => ({
|
||||
goto: jest.fn(),
|
||||
content: jest.fn().mockResolvedValue("<html></html>"),
|
||||
title: jest.fn().mockReturnValue(""),
|
||||
url: jest.fn().mockReturnValue(""),
|
||||
evaluate: jest.fn(),
|
||||
close: jest.fn(),
|
||||
}),
|
||||
}),
|
||||
});
|
||||
|
||||
const makeContext = () => ({
|
||||
newPage: jest.fn().mockResolvedValue(makePage()),
|
||||
addCookies: jest.fn().mockResolvedValue(undefined),
|
||||
addInitScript: jest.fn().mockResolvedValue(undefined),
|
||||
});
|
||||
|
||||
const makeBrowser = () => ({
|
||||
newContext: jest
|
||||
.fn()
|
||||
.mockImplementation(() => Promise.resolve(makeContext())),
|
||||
close: jest.fn(),
|
||||
}),
|
||||
});
|
||||
|
||||
export const chromium = {
|
||||
launch: jest.fn().mockImplementation(() => Promise.resolve(makeBrowser())),
|
||||
};
|
||||
|
||||
@@ -139,7 +139,8 @@ function makePage(rootEl: FakeEl) {
|
||||
|
||||
const evaluate = jest
|
||||
.fn()
|
||||
.mockImplementation((fn: (...args: unknown[]) => unknown, args: unknown) => {
|
||||
.mockImplementation(
|
||||
(fn: (...args: unknown[]) => unknown, args: unknown) => {
|
||||
const exec = new Function(
|
||||
"document",
|
||||
"window",
|
||||
@@ -148,7 +149,8 @@ function makePage(rootEl: FakeEl) {
|
||||
`return (${fn.toString()})(__args__)`,
|
||||
);
|
||||
return Promise.resolve(exec(fakeDocument, fakeWindow, fakeNode, args));
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
return { evaluate } as unknown as import("playwright").Page;
|
||||
}
|
||||
|
||||
@@ -117,7 +117,9 @@ describe("EnvironmentController", () => {
|
||||
const res = await request(app.getHttpServer())
|
||||
.get("/environments?orderBy=name&orderDir=ASC")
|
||||
.expect(200);
|
||||
const names: string[] = res.body.data.map((e: { name: string }) => e.name);
|
||||
const names: string[] = res.body.data.map(
|
||||
(e: { name: string }) => e.name,
|
||||
);
|
||||
expect(names).toEqual([...names].sort());
|
||||
});
|
||||
|
||||
@@ -125,7 +127,9 @@ describe("EnvironmentController", () => {
|
||||
const res = await request(app.getHttpServer())
|
||||
.get("/environments?orderBy=name&orderDir=DESC")
|
||||
.expect(200);
|
||||
const names: string[] = res.body.data.map((e: { name: string }) => e.name);
|
||||
const names: string[] = res.body.data.map(
|
||||
(e: { name: string }) => e.name,
|
||||
);
|
||||
expect(names).toEqual([...names].sort().reverse());
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
/**
|
||||
* Integration tests for ScenarioRunStepEntity.output column and the
|
||||
* helpers.getStepOutput() API available inside exec step scripts.
|
||||
*
|
||||
* Strategy:
|
||||
* - Seed a session row so the scheduler can create a browser context.
|
||||
* - Create a scenario + steps with controlled return values.
|
||||
* - Call ScenarioSchedulerService.pickUpPendingRuns() directly to process
|
||||
* the run synchronously (no real timers needed).
|
||||
* - Assert step-run output persisted and getStepOutput() returns it.
|
||||
*/
|
||||
|
||||
import { INestApplication } from "@nestjs/common";
|
||||
import { DataSource } from "typeorm";
|
||||
import { buildTestApp } from "./app.harness";
|
||||
import { ScenarioService } from "../src/scenario/scenario.service";
|
||||
import { ScenarioSchedulerService } from "../src/scenario/scenario-scheduler.service";
|
||||
|
||||
describe("ScenarioRunStepEntity.output + helpers.getStepOutput", () => {
|
||||
let app: INestApplication;
|
||||
let dataSource: DataSource;
|
||||
let scenarioService: ScenarioService;
|
||||
let scheduler: ScenarioSchedulerService;
|
||||
|
||||
beforeAll(async () => {
|
||||
app = await buildTestApp();
|
||||
dataSource = app.get(DataSource);
|
||||
scenarioService = app.get(ScenarioService);
|
||||
scheduler = app.get(ScenarioSchedulerService);
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await app.close();
|
||||
});
|
||||
|
||||
/** Seed a minimal session so the scheduler can open a browser context. */
|
||||
async function seedSession(name = "output-test-session"): Promise<void> {
|
||||
await dataSource.query(
|
||||
`INSERT OR IGNORE INTO sessions (sessionName, token, cookies, localStorage)
|
||||
VALUES ('${name}', 'tok', '[]', '{}')`,
|
||||
);
|
||||
}
|
||||
|
||||
/** Create a scenario and return its id. */
|
||||
async function createScenario(name = "output-scenario"): Promise<number> {
|
||||
const sc = await scenarioService.create({ name });
|
||||
return sc.id;
|
||||
}
|
||||
|
||||
/** Create a step and return its id. */
|
||||
async function createStep(
|
||||
scenarioId: number,
|
||||
order: number,
|
||||
execCode: string,
|
||||
sessionName = "output-test-session",
|
||||
): Promise<number> {
|
||||
const step = await scenarioService.createStep(scenarioId, {
|
||||
order,
|
||||
type: "exec",
|
||||
sessionName,
|
||||
execCode,
|
||||
});
|
||||
return step.id;
|
||||
}
|
||||
|
||||
/** Trigger a run and process it to completion via the scheduler. */
|
||||
async function runScenario(scenarioId: number): Promise<number> {
|
||||
const run = await scenarioService.createRun(scenarioId);
|
||||
// Drive the scheduler directly — keeps tests synchronous and fast.
|
||||
await scheduler.pickUpPendingRuns();
|
||||
// Wait for the run to reach a terminal state (max 10 s).
|
||||
const result = await scenarioService.waitForRun(scenarioId, run.id, 10_000);
|
||||
return result.id;
|
||||
}
|
||||
|
||||
// ── output column ──────────────────────────────────────────────────────────
|
||||
|
||||
describe("output column", () => {
|
||||
it("stores the return value of an exec script as JSON", async () => {
|
||||
await seedSession();
|
||||
const scId = await createScenario("output-basic");
|
||||
await createStep(scId, 0, "return 42;");
|
||||
|
||||
const runId = await runScenario(scId);
|
||||
|
||||
const row = await dataSource.query(
|
||||
`SELECT output, status FROM scenario_run_steps WHERE runId = ${runId}`,
|
||||
);
|
||||
expect(row[0].status).toBe("pass");
|
||||
expect(JSON.parse(row[0].output as string)).toBe(42);
|
||||
});
|
||||
|
||||
it("stores object return values as JSON", async () => {
|
||||
await seedSession();
|
||||
const scId = await createScenario("output-object");
|
||||
await createStep(scId, 0, 'return { foo: "bar", n: 7 };');
|
||||
|
||||
const runId = await runScenario(scId);
|
||||
|
||||
const row = await dataSource.query(
|
||||
`SELECT output FROM scenario_run_steps WHERE runId = ${runId}`,
|
||||
);
|
||||
expect(JSON.parse(row[0].output as string)).toEqual({ foo: "bar", n: 7 });
|
||||
});
|
||||
|
||||
it("stores null when the script returns undefined/nothing", async () => {
|
||||
await seedSession();
|
||||
const scId = await createScenario("output-undefined");
|
||||
await createStep(scId, 0, "const x = 1;");
|
||||
|
||||
const runId = await runScenario(scId);
|
||||
|
||||
const row = await dataSource.query(
|
||||
`SELECT output FROM scenario_run_steps WHERE runId = ${runId}`,
|
||||
);
|
||||
expect(row[0].output).toBeNull();
|
||||
});
|
||||
|
||||
it("output is exposed on the stepRuns inside GET /scenarios/:id/run/:runId via findRun", async () => {
|
||||
await seedSession();
|
||||
const scId = await createScenario("output-findrun");
|
||||
await createStep(scId, 0, "return 'hello';");
|
||||
|
||||
const runId = await runScenario(scId);
|
||||
const run = await scenarioService.findRun(scId, runId);
|
||||
|
||||
expect(run.stepRuns[0].output).toBe(JSON.stringify("hello"));
|
||||
});
|
||||
});
|
||||
|
||||
// ── helpers.getStepOutput ──────────────────────────────────────────────────
|
||||
|
||||
describe("helpers.getStepOutput()", () => {
|
||||
it("returns output of a previous step by absolute order", async () => {
|
||||
await seedSession();
|
||||
const scId = await createScenario("getStepOutput-absolute");
|
||||
await createStep(scId, 0, "return 99;");
|
||||
// Step 1 reads step 0's output via absolute index 0
|
||||
await createStep(scId, 1, "return await helpers.getStepOutput(0);");
|
||||
|
||||
const runId = await runScenario(scId);
|
||||
const rows = await dataSource.query(
|
||||
`SELECT "order", output FROM scenario_run_steps WHERE runId = ${runId} ORDER BY "order"`,
|
||||
);
|
||||
|
||||
expect(JSON.parse(rows[0].output as string)).toBe(99);
|
||||
expect(JSON.parse(rows[1].output as string)).toBe(99);
|
||||
});
|
||||
|
||||
it("returns output of the previous step using relative index -1", async () => {
|
||||
await seedSession();
|
||||
const scId = await createScenario("getStepOutput-relative");
|
||||
await createStep(scId, 0, 'return "step-zero";');
|
||||
// Step 1 uses relative index -1 to reference step 0
|
||||
await createStep(scId, 1, "return await helpers.getStepOutput(-1);");
|
||||
|
||||
const runId = await runScenario(scId);
|
||||
const rows = await dataSource.query(
|
||||
`SELECT "order", output FROM scenario_run_steps WHERE runId = ${runId} ORDER BY "order"`,
|
||||
);
|
||||
|
||||
expect(JSON.parse(rows[1].output as string)).toBe("step-zero");
|
||||
});
|
||||
|
||||
it("returns null for a step that does not exist", async () => {
|
||||
await seedSession();
|
||||
const scId = await createScenario("getStepOutput-missing");
|
||||
// Step 0 tries to read step order 99 which does not exist
|
||||
await createStep(scId, 0, "return await helpers.getStepOutput(99);");
|
||||
|
||||
const runId = await runScenario(scId);
|
||||
const row = await dataSource.query(
|
||||
`SELECT output FROM scenario_run_steps WHERE runId = ${runId}`,
|
||||
);
|
||||
|
||||
expect(row[0].output).toBeNull();
|
||||
});
|
||||
|
||||
it("chains output across three steps", async () => {
|
||||
await seedSession();
|
||||
const scId = await createScenario("getStepOutput-chain");
|
||||
await createStep(scId, 0, "return [1, 2];");
|
||||
await createStep(
|
||||
scId,
|
||||
1,
|
||||
"const prev = await helpers.getStepOutput(-1); return [...prev, 3];",
|
||||
);
|
||||
await createStep(
|
||||
scId,
|
||||
2,
|
||||
"const prev = await helpers.getStepOutput(-1); return [...prev, 4];",
|
||||
);
|
||||
|
||||
const runId = await runScenario(scId);
|
||||
const rows = await dataSource.query(
|
||||
`SELECT "order", output FROM scenario_run_steps WHERE runId = ${runId} ORDER BY "order"`,
|
||||
);
|
||||
|
||||
expect(JSON.parse(rows[0].output as string)).toEqual([1, 2]);
|
||||
expect(JSON.parse(rows[1].output as string)).toEqual([1, 2, 3]);
|
||||
expect(JSON.parse(rows[2].output as string)).toEqual([1, 2, 3, 4]);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -106,7 +106,9 @@ describe("ScenarioController", () => {
|
||||
const res = await request(app.getHttpServer())
|
||||
.get("/scenarios?orderBy=name&orderDir=ASC")
|
||||
.expect(200);
|
||||
const names: string[] = res.body.data.map((s: { name: string }) => s.name);
|
||||
const names: string[] = res.body.data.map(
|
||||
(s: { name: string }) => s.name,
|
||||
);
|
||||
expect(names).toEqual([...names].sort());
|
||||
});
|
||||
|
||||
@@ -114,7 +116,9 @@ describe("ScenarioController", () => {
|
||||
const res = await request(app.getHttpServer())
|
||||
.get("/scenarios?orderBy=name&orderDir=DESC")
|
||||
.expect(200);
|
||||
const names: string[] = res.body.data.map((s: { name: string }) => s.name);
|
||||
const names: string[] = res.body.data.map(
|
||||
(s: { name: string }) => s.name,
|
||||
);
|
||||
expect(names).toEqual([...names].sort().reverse());
|
||||
});
|
||||
|
||||
@@ -336,7 +340,9 @@ describe("ScenarioController", () => {
|
||||
expect(Array.isArray(res.body.stepRuns)).toBe(true);
|
||||
expect(res.body.stepRuns).toHaveLength(3);
|
||||
|
||||
const statuses = res.body.stepRuns.map((s: { status: string }) => s.status);
|
||||
const statuses = res.body.stepRuns.map(
|
||||
(s: { status: string }) => s.status,
|
||||
);
|
||||
expect(statuses[0]).toBe("pending");
|
||||
expect(statuses[1]).toBe("waiting");
|
||||
expect(statuses[2]).toBe("waiting");
|
||||
|
||||
@@ -49,7 +49,9 @@ describe("SessionController", () => {
|
||||
const res = await request(app.getHttpServer())
|
||||
.get("/sessions")
|
||||
.expect(200);
|
||||
const names = res.body.data.map((s: { sessionName: string }) => s.sessionName);
|
||||
const names = res.body.data.map(
|
||||
(s: { sessionName: string }) => s.sessionName,
|
||||
);
|
||||
expect(names).toContain("visible-session");
|
||||
});
|
||||
|
||||
@@ -97,7 +99,9 @@ describe("SessionController", () => {
|
||||
const res = await request(app.getHttpServer())
|
||||
.get("/sessions?orderBy=sessionName&orderDir=ASC")
|
||||
.expect(200);
|
||||
const names: string[] = res.body.data.map((s: { sessionName: string }) => s.sessionName);
|
||||
const names: string[] = res.body.data.map(
|
||||
(s: { sessionName: string }) => s.sessionName,
|
||||
);
|
||||
expect(names).toEqual([...names].sort());
|
||||
});
|
||||
|
||||
@@ -105,7 +109,9 @@ describe("SessionController", () => {
|
||||
const res = await request(app.getHttpServer())
|
||||
.get("/sessions?orderBy=sessionName&orderDir=DESC")
|
||||
.expect(200);
|
||||
const names: string[] = res.body.data.map((s: { sessionName: string }) => s.sessionName);
|
||||
const names: string[] = res.body.data.map(
|
||||
(s: { sessionName: string }) => s.sessionName,
|
||||
);
|
||||
expect(names).toEqual([...names].sort().reverse());
|
||||
});
|
||||
|
||||
@@ -128,7 +134,9 @@ describe("SessionController", () => {
|
||||
const res = await request(app.getHttpServer())
|
||||
.get("/sessions")
|
||||
.expect(200);
|
||||
const names = res.body.data.map((sess: { sessionName: string }) => sess.sessionName);
|
||||
const names = res.body.data.map(
|
||||
(sess: { sessionName: string }) => sess.sessionName,
|
||||
);
|
||||
expect(names).not.toContain("delete-me-session");
|
||||
});
|
||||
|
||||
|
||||
Reference in New Issue
Block a user