Merge pull request 'feat(examples): add self-test example' (#10) from feat/examples into master
CI / changes (push) Successful in 6s
CI / client (push) Skipped
CI / server (push) Successful in 1m6s
CI / image (push) Skipped

Reviewed-on: #10
This commit was merged in pull request #10.
This commit is contained in:
ars9 committed 2026-10-02 15:42:19 +03:00
commit 0c11e11c27
7 files changed
+433

No files matched your search

+1
View File
@@ -14,6 +14,7 @@ It is an npm workspace with two packages:
- [docs/development.md](docs/development.md) - local setup, running, testing, Docker - [docs/development.md](docs/development.md) - local setup, running, testing, Docker
- [docs/scenario.md](docs/scenario.md) - scenario execution, script `context` API, versioning, export/import, Playwright traces - [docs/scenario.md](docs/scenario.md) - scenario execution, script `context` API, versioning, export/import, Playwright traces
- [docs/mcp.md](docs/mcp.md) - MCP tool reference - [docs/mcp.md](docs/mcp.md) - MCP tool reference
- [examples/](examples/README.md) - ready-to-import example resources (YAML); `server/test/examples.spec.ts` imports each file, so keep them in sync with the import format
- [CHANGELOG.md](CHANGELOG.md) - release notes per version - [CHANGELOG.md](CHANGELOG.md) - release notes per version
- [Dockerfile](Dockerfile) - single image: builds client and server, runtime serves both - [Dockerfile](Dockerfile) - single image: builds client and server, runtime serves both
- [.env.example](.env.example) - all configuration variables (validated in `server/src/config/app.config.ts`) - [.env.example](.env.example) - all configuration variables (validated in `server/src/config/app.config.ts`)
+3
View File
@@ -14,6 +14,8 @@ No configuration is needed: every setting has a default. To override one, copy `
Everything is served from one container at **http://localhost:13000**: the UI at `/`, the REST API at `/api/v1`, MCP at `/mcp`, Swagger at `/docs`. Data is kept in `./data/`. Everything is served from one container at **http://localhost:13000**: the UI at `/`, the REST API at `/api/v1`, MCP at `/mcp`, Swagger at `/docs`. Data is kept in `./data/`.
To see it work, import and run the [self-test example](examples/self-test/): a scenario that tests liqa itself using an environment, a credential and snippets.
--- ---
## Installation ## Installation
@@ -49,6 +51,7 @@ Scenarios, snippets, environments and credentials can be exported as YAML and im
- [docs/scenario.md](docs/scenario.md) - runs, the script `context` API, versioning, import/export, Playwright traces - [docs/scenario.md](docs/scenario.md) - runs, the script `context` API, versioning, import/export, Playwright traces
- [docs/mcp.md](docs/mcp.md) - MCP tool reference - [docs/mcp.md](docs/mcp.md) - MCP tool reference
- [docs/development.md](docs/development.md) - setup, configuration, running, testing, linting, Docker - [docs/development.md](docs/development.md) - setup, configuration, running, testing, linting, Docker
- [examples/](examples/) - ready-to-import example scenarios
- [CHANGELOG.md](CHANGELOG.md) - release notes - [CHANGELOG.md](CHANGELOG.md) - release notes
--- ---
+2
View File
@@ -155,3 +155,5 @@ Run-step status:
5. Assign credentials to scenario aliases when needed 5. Assign credentials to scenario aliases when needed
6. Run scenario 6. Run scenario
7. Wait for run completion and inspect run logs/outputs 7. Wait for run completion and inspect run logs/outputs
The [self-test example](../examples/self-test/) follows these steps through the REST API, and can be imported as a complete, working scenario.
+9
View File
@@ -0,0 +1,9 @@
# Examples
Ready-to-import liqa resources. Each example is a directory with a YAML file to import (the Import page in the UI, `POST /api/v1/import`, or the `import_yaml` MCP tool; see [Import](../docs/scenario.md#import)) and a README on how to run it.
| Example | What it shows |
|---|---|
| [self-test](self-test/) | A scenario that tests the liqa instance it runs on: creates a scenario through the REST API, runs it, checks the run and its page in the UI, and cleans up. Uses an environment, a credential, snippets and assertions |
`server/test/examples.spec.ts` imports every YAML file in this directory and checks that its code parses, so the examples keep up with the import format.
+55
View File
@@ -0,0 +1,55 @@
# Self-test
A scenario that tests the liqa instance it runs on. It drives liqa through its own REST API and web UI: it creates a small scenario, runs it, checks the result and deletes it again. It shows the main liqa features in one place:
- an **environment** with the URLs and test values the steps read with `context.getEnv()`
- a **credential**, assigned to the scenario as `fixture` and read with `context.getCredential()`
- three **snippets**, called with `context.runSnippet()`, one of them from the other two
- **assertions** (`context.assert()` and Playwright's `expect`), step outputs (`context.getStepOutput()`) and run logs
Everything is in [self-test.yaml](self-test.yaml).
## Run it
1. Start liqa with the default docker compose deployment (`docker compose up -d --build` at the repo root). It's served at http://localhost:13000.
2. Import [self-test.yaml](self-test.yaml): open **Import** in the UI and pick the file, or run
```bash
curl -X POST http://localhost:13000/api/v1/import \
-H 'Content-Type: application/yaml' \
--data-binary @examples/self-test/self-test.yaml
```
Every item has a fixed id, so importing the file again updates the same resources instead of adding copies.
3. Open the **liqa self-test** scenario and click **Run**. Its default environment, `liqa-self-test`, is preselected.
The run takes a few seconds. The run page shows every assertion as a log entry, and the target scenario's run while it exists.
## What's in the file
| Kind | Name | Purpose |
|---|---|---|
| Environment | `liqa-self-test` | `liqa_url` (REST API base), `ui_url` (web UI base), `temp_prefix` (name prefix of the temporary resources), `greeting` (test value) |
| Credential | `liqa-self-test-fixture` | Test data (`username`, `token`) that the self-test passes on to the scenario it creates. It's not a real secret |
| Snippet | `liqa-api` | `runSnippet("liqa-api", method, path, body?)`: calls `liqa_url` + `/api/v1` + `path`, returns the parsed JSON and throws on an error status |
| Snippet | `liqa-run-and-wait` | `runSnippet("liqa-run-and-wait", scenarioId, environmentId)`: starts a run and returns it once it has finished. Uses `liqa-api` |
| Snippet | `self-test-cleanup` | Deletes every scenario, credential and environment whose name starts with `temp_prefix` and returns how many it deleted. Uses `liqa-api` |
| Scenario | `liqa self-test` | The self-test itself (below) |
The scenario's steps:
1. **Check the instance is up**: checks `GET /health`, then runs `self-test-cleanup` to remove anything a failed earlier run left behind (and logs a warning if it found some).
2. **Create the target scenario**: creates a temporary environment, a temporary credential (holding the `fixture` credential's data) and the target scenario `[self-test] target scenario`, and assigns the credential to it as `secret`. The target has two steps: one reads `greeting` and the `secret` token, the other renders the greeting on a page and checks it with `expect`.
3. **Run the target scenario**: runs it with `liqa-run-and-wait` and checks the result: the run and both steps passed, the step outputs hold the greeting and the token's length, and the target logged its passing assertion.
4. **Check the run in the UI**: opens the target's run page at `ui_url` and checks that it shows the scenario name and the assertion.
5. **Clean up**: runs `self-test-cleanup` again, and checks that it deleted the three temporary resources and that the target scenario is gone.
## Addresses
Steps run inside the liqa server, so `liqa_url` and `ui_url` are addresses as seen from the server, not from your machine. In the docker compose deployment the server listens on port 3000 inside the container, published as 13000 on the host, so both are `http://localhost:3000`.
When running the server locally (`npm run start:dev`, see [docs/development.md](../../docs/development.md)), `liqa_url` stays `http://localhost:3000`. The UI is then served by the Vite dev server: set `ui_url` to `http://localhost:5173`.
## When it fails
A failed step stops the run, so the temporary resources (named `[self-test] ...`) are left in place to inspect. The next self-test run deletes them in its first step.
+280
View File
@@ -0,0 +1,280 @@
# liqa self-test example: a scenario that tests the liqa instance it runs on.
#
# Import this file (Import page in the UI, POST /api/v1/import with
# Content-Type: application/yaml, or the import_yaml MCP tool), then run the
# "liqa self-test" scenario. See README.md next to this file.
#
# Every item has a fixed id, so importing the file again updates the same
# resources instead of creating duplicates.
---
# Environment: where the liqa under test is reachable from the liqa server
kind: environment
id: 418eacb0-ab16-4ad6-ab50-c0274fdc49f1
name: liqa-self-test
description: >-
The liqa instance the self-test scenario runs against. Steps run inside the
liqa server (the Docker container), so the compose deployment published at
http://localhost:13000 is reached at its in-container port 3000.
data:
# REST API base (without /api/v1)
liqa_url: http://localhost:3000
# Web UI base; use http://localhost:5173 with the Vite dev server
ui_url: http://localhost:3000
# Name prefix of the temporary resources the self-test creates and deletes
temp_prefix: "[self-test] "
# Value the target scenario reads from its own (temporary) environment
greeting: Hello from the liqa self-test
---
# Credential: test data handed over to the temporary target scenario
kind: credential
id: b0c9ccab-6743-4ac1-9017-d99e0f5d7474
name: liqa-self-test-fixture
data: '{"username":"self-test-user","token":"not-a-real-secret-0123456789"}'
---
# Snippet: liqa-api
kind: snippet
id: f9d2612c-266c-411a-9483-5d860599f208
version: 1
alias: liqa-api
title: Call the liqa REST API
description: >-
`context.runSnippet("liqa-api", method, path, body?)` sends a request to
`liqa_url` + `/api/v1` + path and returns the parsed JSON response (`null`
for an empty body). Throws on a non-2xx status.
code: |
const [method, path, body] = args;
const url = context.getEnv("liqa_url") + "/api/v1" + path;
const response = await context.page.request.fetch(url, {
method,
data: body,
// POST /scenarios/:id/run/:runId/wait blocks until the run finishes
timeout: 120_000,
failOnStatusCode: false,
});
const text = await response.text();
if (!response.ok()) {
throw new Error(`${method} ${path} returned ${response.status()}: ${text}`);
}
return text ? JSON.parse(text) : null;
---
# Snippet: liqa-run-and-wait
kind: snippet
id: b36c1075-2700-421f-a2e7-ab1a80adb7e7
version: 1
alias: liqa-run-and-wait
title: Run a scenario and wait for it
description: >-
`context.runSnippet("liqa-run-and-wait", scenarioId, environmentId)` starts
a run of the scenario's latest version and returns the finished run (with
its step runs and logs). Uses the liqa-api snippet.
code: |
const [scenarioId, environmentId] = args;
const run = await context.runSnippet(
"liqa-api",
"POST",
`/scenarios/${scenarioId}/run`,
{ environmentId },
);
context.log(`Started run ${run.id}, waiting for it to finish`);
return context.runSnippet(
"liqa-api",
"POST",
`/scenarios/${scenarioId}/run/${run.id}/wait`,
);
---
# Snippet: self-test-cleanup
kind: snippet
id: f99e3b17-d28d-4a80-8ae4-f7e8252e0be5
version: 1
alias: self-test-cleanup
title: Delete temporary self-test resources
description: >-
Deletes every scenario, credential and environment whose name starts with
the `temp_prefix` environment value, and returns how many were deleted.
Uses the liqa-api snippet.
code: |
const prefix = context.getEnv("temp_prefix");
let deleted = 0;
// Scenarios first: they reference the credentials and environments
for (const path of ["/scenarios", "/credentials", "/environments"]) {
const page = await context.runSnippet("liqa-api", "GET", `${path}?limit=1000`);
for (const item of page.data.filter((i) => i.name.startsWith(prefix))) {
await context.runSnippet("liqa-api", "DELETE", `${path}/${item.id}`);
context.log(`Deleted ${path.slice(1, -1)} "${item.name}"`);
deleted += 1;
}
}
return deleted;
---
# Scenario: liqa self-test
kind: scenario
id: 571b2894-dacc-4428-b459-4e3fefac5908
version: 1
name: liqa self-test
description: |
Tests the liqa instance it runs on, through its REST API and web UI:
1. checks that the instance is up and removes leftovers of earlier runs
2. creates a temporary environment, credential and two-step target scenario
3. runs the target scenario, waits for it and checks its step outputs and logs
4. opens the finished run in the web UI
5. deletes the temporary resources
Uses the `liqa-api`, `liqa-run-and-wait` and `self-test-cleanup` snippets and
the `fixture` credential. See `examples/self-test/README.md`.
environmentId: 418eacb0-ab16-4ad6-ab50-c0274fdc49f1
timeoutSeconds: 300
credentials:
- credentialId: b0c9ccab-6743-4ac1-9017-d99e0f5d7474
alias: fixture
steps:
- title: Check the instance is up
execCode: |
const response = await context.page.request.get(
context.getEnv("liqa_url") + "/health",
);
const health = await response.json();
context.assert(() => response.ok(), "Health endpoint responds", {
status: response.status(),
});
context.assert(() => health.status === "ok", "Health status is ok", health);
// A failed earlier run may have left its temporary resources behind
const deleted = await context.runSnippet("self-test-cleanup");
if (deleted > 0) {
context.warn(`Deleted ${deleted} leftover resource(s) of an earlier run`);
}
return health;
- title: Create the target scenario
execCode: |
const prefix = context.getEnv("temp_prefix");
const api = (method, path, body) =>
context.runSnippet("liqa-api", method, path, body);
const environment = await api("POST", "/environments", {
name: `${prefix}environment`,
data: { greeting: context.getEnv("greeting") },
});
// Hand the fixture credential over to the target scenario
const fixture = context.getCredential("fixture");
const credential = await api("POST", "/credentials", {
name: `${prefix}credential`,
data: JSON.stringify(fixture),
});
const scenario = await api("POST", "/scenarios", {
name: `${prefix}target scenario`,
description: "Temporary scenario created by the liqa self-test",
environmentId: environment.id,
timeoutSeconds: 60,
});
await api("POST", `/scenarios/${scenario.id}/credentials`, {
credentialId: credential.id,
alias: "secret",
});
await api("POST", `/scenarios/${scenario.id}/steps`, {
title: "Read the environment and credential",
execCode: `
const secret = context.getCredential("secret");
context.assert(() => typeof secret.token === "string", "Credential has a token");
// Return the token length, not the token: step outputs are not secret
return { greeting: context.getEnv("greeting"), tokenLength: secret.token.length };
`,
});
await api("POST", `/scenarios/${scenario.id}/steps`, {
title: "Render the greeting",
execCode: `
const { greeting } = await context.getStepOutput(-1);
await context.page.setContent("<h1>" + greeting + "</h1>");
await expect(context.page.locator("h1")).toHaveText(greeting);
return { heading: await context.page.locator("h1").textContent() };
`,
});
const created = await api("GET", `/scenarios/${scenario.id}`);
context.assert(
() => created.steps.length === 2,
"Target scenario has 2 steps",
{ steps: created.steps.length },
);
context.log(`Created target scenario ${scenario.id}`);
return {
scenarioId: scenario.id,
scenarioName: scenario.name,
environmentId: environment.id,
credentialId: credential.id,
};
- title: Run the target scenario
timeoutSeconds: 150
execCode: |
const { scenarioId, environmentId } = await context.getStepOutput(-1);
const run = await context.runSnippet(
"liqa-run-and-wait",
scenarioId,
environmentId,
);
context.assert(() => run.status === "pass", "Target run passed", {
status: run.status,
errors: run.logs.filter((l) => l.level === "error").map((l) => l.message),
});
context.assert(
() => run.stepRuns.every((s) => s.status === "pass"),
"All target steps passed",
run.stepRuns.map((s) => ({ order: s.order, status: s.status })),
);
const [read, render] = run.stepRuns.map((s) => JSON.parse(s.output));
const greeting = context.getEnv("greeting");
context.assert(
() => read.greeting === greeting,
"Target read the greeting from its environment",
read,
);
context.assert(
() => read.tokenLength === context.getCredential("fixture").token.length,
"Target read the token from its credential",
);
context.assert(
() => render.heading === greeting,
"Target rendered the greeting",
render,
);
context.assert(
() =>
run.logs.some(
(l) =>
l.context === "assertion:pass" &&
l.message === "Credential has a token",
),
"Target logged its passed assertion",
);
return { runId: run.id };
- title: Check the run in the UI
execCode: |
const { scenarioId, scenarioName } = await context.getStepOutput(2);
const { runId } = await context.getStepOutput(3);
const page = context.page;
await page.goto(
`${context.getEnv("ui_url")}/scenarios/${scenarioId}/runs/${runId}`,
);
await expect(page.getByText(scenarioName).first()).toBeVisible();
await expect(page.getByText("Credential has a token").first()).toBeVisible();
context.log(`Run page checked: ${page.url()}`);
return { url: page.url() };
- title: Clean up
execCode: |
const { scenarioId } = await context.getStepOutput(2);
const deleted = await context.runSnippet("self-test-cleanup");
context.assert(() => deleted === 3, "Deleted the 3 temporary resources", {
deleted,
});
const response = await context.page.request.get(
`${context.getEnv("liqa_url")}/api/v1/scenarios/${scenarioId}`,
);
context.assert(
() => response.status() === 404,
"Target scenario is gone",
{ status: response.status() },
);
return { deleted };
+83
View File
@@ -0,0 +1,83 @@
/**
* Keeps the files in `examples/` importable: every YAML file there must pass
* `POST /import`, and all its step and snippet code must parse.
*/
import * as fs from "node:fs";
import * as path from "node:path";
import { INestApplication } from "@nestjs/common";
import request from "supertest";
import { buildTestApp } from "./app.harness";
import { ImportService } from "../src/import/import.service";
const EXAMPLES_DIR = path.resolve(__dirname, "../../examples");
function findYamlFiles(dir: string): string[] {
return fs.readdirSync(dir, { withFileTypes: true }).flatMap((entry) => {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) return findYamlFiles(full);
return /\.ya?ml$/.test(entry.name) ? [full] : [];
});
}
const AsyncFunction = Object.getPrototypeOf(async () => {})
.constructor as new (...args: string[]) => unknown;
interface ExampleItem {
kind: string;
code?: string;
steps?: { title?: string; execCode?: string }[];
}
describe("Examples", () => {
let app: INestApplication;
let importService: ImportService;
beforeAll(async () => {
app = await buildTestApp();
importService = app.get(ImportService);
});
afterAll(async () => {
await app.close();
});
const files = findYamlFiles(EXAMPLES_DIR);
it("has example files", () => {
expect(files.length).toBeGreaterThan(0);
});
describe.each(files.map((f) => [path.relative(EXAMPLES_DIR, f), f]))(
"%s",
(_name, file) => {
const yaml = fs.readFileSync(file, "utf8");
it("imports", async () => {
const res = await request(app.getHttpServer())
.post("/import")
.set("Content-Type", "application/yaml")
.send(yaml)
.expect(201);
expect(res.body.imported.length).toBeGreaterThan(0);
});
it("has code that parses", () => {
const items = importService.parseYaml(yaml) as ExampleItem[];
for (const item of items) {
// Same shape as the executor: snippets get `...args`
if (item.code != null) {
expect(
() => new AsyncFunction("context", "...args", item.code!),
).not.toThrow();
}
for (const step of item.steps ?? []) {
if (step.execCode == null) continue;
expect(
() => new AsyncFunction("context", step.execCode!),
).not.toThrow();
}
}
});
},
);
});