Files
liqa/docs/development.md
T
c0fe15dc01
CI / changes (push) Successful in 4s
CI / server (push) Failing after 1h44m8s
CI / client (push) Successful in 1m52s
CI / image (push) Skipped
Keep test output quiet with opt-in diagnostic logs (#25)
Test runs stay quiet by default. Diagnostic logs can be enabled explicitly when troubleshooting, while normal application logging and saved scenario output remain available.

Resolves #22.

---------

Co-authored-by: ars9 <ars9@pm.me>
Co-authored-by: agent <agent@halstation.halstation.local>
Reviewed-on: #25
Co-authored-by: agent-codex <agent-codex@ars9.space>
2026-11-11 22:23:44 +02:00

22 KiB

Development

Prerequisites

  • Node.js 24+ (required by engines in package.json; the Docker image uses Node.js 24)
  • Docker + Docker Compose

The repository is a npm workspace monorepo with two packages:

Workspace Directory Description
server server/ NestJS API + MCP server
client client/ React 19 + Vite admin UI

Install all dependencies from the repo root:

npm install
npx playwright install --only-shell chromium   # first time only: headless Chromium for the server

No .env is needed: every setting below has a default. To override settings, copy .env.example to .env and uncomment the ones you need. Docker Compose loads .env when it exists.

Environment variables (server)

All settings are validated at startup by server/src/config/app.config.ts.

Variable Default Description
NODE_ENV development production in the Docker image. Shown in the startup log, and selects the default CORS origins
CORS_ORIGINS (empty) Comma-separated origins allowed to call the API cross-origin. When empty: the Vite dev server (http://localhost:5173, http://127.0.0.1:5173) unless NODE_ENV=production, then the compose port (http://localhost:13000, http://127.0.0.1:13000)
PORT 3000 HTTP port
LOG_MODE console console preserves the default server output; debug routes application diagnostics through the debug package and only emits namespaces enabled by DEBUG (for example, DEBUG=liqa:*).
DB_PATH data/liqa.db SQLite database file
FILES_DIR data/files Where scenario files and run artifacts are stored
FILE_RUN_EXPIRATION_DAYS 7 How long run artifacts (downloads, failure dumps, traces) are kept
FILE_CLEANUP_CRON 0 3 * * * When expired files are deleted
SESSION_IDLE_TIMEOUT_MINUTES 30 Idle timeout before open sessions are closed
SESSION_DELETE_CLOSED_DAYS 7 Retention period for closed sessions
PLAYWRIGHT_TRACE_MODE off Default Playwright trace mode for runs: on, retain-on-failure or off (see scenario.md)
RUN_LOG_DATA_COLLAPSE_BYTES 256 Run log data blocks whose pretty-printed JSON is over this many bytes start collapsed in the UI (exposed via /manifest.json)
RUN_LOG_DATA_COLLAPSE_LINES 5 Same, for the number of lines
CLIENT_DIR (empty) Directory with the built client to serve at / (the Docker image sets it to client); empty means the server doesn't serve the UI
PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH (Playwright default) Path to a Chromium binary to use instead of Playwright's own. Not set in Docker: the image ships Playwright's chromium-headless-shell (matching the Playwright version) under PLAYWRIGHT_BROWSERS_PATH=/ms-playwright.
AUTH_ENABLED false Opt-in account authentication. With the default false, the existing UI, REST API and MCP remain available without accounts.
AUTH_JWT_SECRET (empty) Required and non-empty when AUTH_ENABLED=true. Keep it private and stable across restarts; changing it invalidates existing tokens.
AUTH_COOKIE_NAME liqa_auth Name of the browser's HttpOnly session cookie.
AUTH_JWT_EXPIRES_IN 24h Default lifetime for browser sessions and CLI-issued tokens. Positive durations use s, m, h or d.
AUTH_REVOKED_TOKEN_CLEANUP_CRON 0 3 * * * When expired logout revocations are cleaned. Cleanup also runs once at startup and is disabled when auth is off.

Starting the application

Server

Development (watch mode, restarts on file changes):

npm -w server run start:dev

Production build, then start:

npm -w server run build
npm -w server run start:prod

The server listens on port 3000 by default.

Client

Development (Vite dev server with HMR):

npm -w client run dev

The Vite dev server runs on http://localhost:5173. It has no API proxy: the client calls VITE_API_URL (default /api/v1 on its own origin), and the server allows CORS from http://localhost:5173. So point it at a running server:

npm -w client run dev:compose-api                          # API of the compose container (localhost:13000)
VITE_API_URL=http://localhost:3000/api/v1 npm -w client run dev   # API of a locally started server

Production build:

npm -w client run build

The built files are emitted to client/dist/.


Database migrations

The SQLite schema is managed by TypeORM migrations in server/src/database/migrations/; synchronize is off.

  • Docker: the container applies pending migrations before starting the server.

  • Locally: build, then apply them before starting the server (it refuses to start while migrations are pending):

    npm -w server run build
    npm -w server run migration:run    # apply pending migrations to DB_PATH
    npm -w server run migration:show   # list migrations and whether they're applied
    

The scripts run from server/, so DB_PATH (default data/liqa.db) is relative to it, and they read server/.env if present, like the server does.

Enabling authentication

Authentication is disabled by default, so a fresh install needs no auth settings. For Docker Compose, set these in the project .env (keep it out of version control):

AUTH_ENABLED=true
AUTH_JWT_SECRET=<private-random-secret>

Use a long, randomly generated secret from your secret manager; do not use the placeholder literally or commit a real value. AUTH_ENABLED accepts only true or false. When enabled, startup fails if the signing secret is blank or the configured token lifetime/cleanup schedule is invalid. Rebuild/restart the container after changing its configuration. Compose applies pending migrations before starting the server. For local development, export the same variables in the server process environment (or put them in server/.env); run npm -w server run build and npm -w server run migration:run before account commands or server startup. Migrations add the user and revoked-token tables and are required even when authentication is disabled.

When auth is enabled, REST and MCP controller routes require a valid bearer token or browser cookie except GET /health, GET /healthz, GET /manifest.json, and POST /api/v1/auth/login. Login is public so a browser can establish its session. The manifest exposes whether auth is enabled; health endpoints return service status. The UI assets and Swagger UI at /docs remain reachable, while API operations invoked from Swagger require authentication. MCP at /mcp and the run-events stream are protected. When auth is disabled, the guard allows existing requests through.

Operator account commands

Authentication is opt-in. To manage accounts or issue tokens, set AUTH_ENABLED=true and AUTH_JWT_SECRET in the same environment used by the server. Apply database migrations before using the commands. Passwords are entered interactively without terminal echo; they are never accepted as command-line arguments. The CLI uses the same DB_PATH as the server; locally it resolves the default data/liqa.db relative to server/, while inside the container it uses the configured path (normally /app/data/liqa.db, persisted by the ./data:/app/data volume).

Build the server and apply migrations locally, then run commands from the server workspace:

npm -w server run build
npm -w server run migration:run
npm -w server run cli -- user:create
npm -w server run cli -- user:get alice
npm -w server run cli -- user:token alice       # uses AUTH_JWT_EXPIRES_IN (default 24h)
npm -w server run cli -- user:token alice 30m   # optional lifetime
npm -w server run cli -- user:delete alice

The container image does not include npm or npx. Run the built CLI with Node inside the running service; it uses that container's configured DB_PATH and exits without starting the web server or schedulers:

docker compose exec liqa node dist/cli.js user:create
docker compose exec liqa node dist/cli.js user:get alice
docker compose exec liqa node dist/cli.js user:token alice 30m
docker compose exec liqa node dist/cli.js user:delete alice

user:create asks for a username and two matching passwords; password entry is hidden and requires an interactive terminal. A duplicate username, blank username/password, mismatch or invalid input fails with a nonzero exit. The other commands require an existing username; lookup output contains only the account id, username and timestamps. Unknown usernames fail with No account found for that username. and a nonzero exit. A token duration must be a positive number followed by s, m, h or d (for example 30m, 1h, 7d); omitted duration uses AUTH_JWT_EXPIRES_IN (24h by default). Token output is a bearer credential: handle it as a secret and do not paste it into logs or shared history. User deletion removes the account, so the server rejects that user's previously issued tokens on their next authentication. General CLI failures print a concise hint to check auth settings, DB_PATH and migrations.

Browser sessions, logout and development origins

With auth enabled, the UI asks for credentials at /login. POST /api/v1/auth/login sets a signed session in an HttpOnly, SameSite=Strict cookie (default name liqa_auth) whose expiry matches the JWT lifetime. The cookie gets the Secure attribute when the request is HTTPS, including when the server sees X-Forwarded-Proto: https; serve production traffic over HTTPS and configure trusted proxy headers accordingly. JavaScript cannot read the token. POST /api/v1/auth/logout revokes the presented token and clears the cookie. Revoked tokens remain rejected until their expiration; deleted accounts also cause their tokens to be rejected on the next authentication.

Expired revocation rows are removed at startup and by AUTH_REVOKED_TOKEN_CLEANUP_CRON (default 0 3 * * *, daily at 03:00 server time). The cleanup is not scheduled when authentication is disabled. Browser run notifications use the authenticated GET /api/v1/scenarios/runs/events event stream.

For local Vite development, the server's default CORS origins are http://localhost:5173 and http://127.0.0.1:5173 outside production. Vite has no API proxy: set VITE_API_URL=http://localhost:3000/api/v1 when using a local server, or use npm -w client run dev:compose-api to target http://localhost:13000. The API's default CORS list also permits the compose origins http://localhost:13000 and http://127.0.0.1:13000 in production. Keep the UI and API on the same host name (localhost with localhost, or 127.0.0.1 with 127.0.0.1) so the strict cookie remains same-site. For a different origin, set CORS_ORIGINS to the exact comma-separated UI origins; use HTTPS in deployed environments.

Changing the schema (adding or changing an entity):

  1. Change the entity and build (npm -w server run build).

  2. Make sure your local database is on the latest migration (migration:run).

  3. Generate a migration from the difference between the entities and that database:

    npm -w server run migration:generate -- src/database/migrations/AddSomething
    
  4. Review the generated file, then add its class to MIGRATIONS in src/database/migrations/index.ts (they're listed explicitly, oldest first).

  5. Run the tests: test/migrations.spec.ts checks that the migrations produce exactly the schema the entities describe, and fails if an entity change has no migration.

The initial migration (Initial…) adopts databases created before migrations existed (by synchronize): if the tables already exist, it's only recorded as applied.

Running tests

Server (Jest)

npm -w server run test

Run a single spec file:

npm -w server run test -- --no-coverage test/scenario.controller.spec.ts

Watch mode:

npm -w server run test:watch

Server application logs use debug output during tests. They remain quiet unless the test namespace is enabled. To see NestJS and application logs:

npm -w server run test:debug

Client (Vitest + Storybook)

Storybook interaction tests (runs all *.stories.tsx files):

npm -w client run test:storybook

Launch Storybook dev server (with live preview):

npm -w client run storybook

Formatting code

Both workspaces use Prettier.

Server — rewrites src/**/*.ts and test/**/*.ts:

npm -w server run format

Client — rewrites src/**/*.{ts,tsx} and .storybook/**/*.{ts,tsx}:

npm -w client run format

Linting

Both workspaces use ESLint with typescript-eslint. The client also includes eslint-plugin-react-hooks and eslint-plugin-react-refresh.

Server:

npm -w server run lint        # report issues
npm -w server run lint:fix    # report and auto-fix

Client:

npm -w client run lint        # report issues
npm -w client run lint:fix    # report and auto-fix

Both workspaces target zero errors. Run lint before committing.


Building the container

The root Dockerfile builds a single image with the whole application: it builds the client and the server in separate stages, and the runtime image runs the server, which also serves the built client (CLIENT_DIR).

The runtime image uses Playwright's headless-only Chromium build (chromium-headless-shell) with just the shared libraries it links against, plus DejaVu and Liberation fonts (Liberation is metric-compatible with Arial/Times, so pages lay out as they would with those fonts). All browser launches are headless, so the full desktop Chromium isn't installed. If a new Playwright version needs another library, ldd on the chrome-headless-shell binary in the image lists what's missing.

To keep the image small, production dependencies and the browser are installed in a separate runtime-deps stage, which drops what nothing loads at runtime (source maps, TypeScript sources and declarations, docs, TypeORM's browser build, better-sqlite3's C sources, and Playwright's ffmpeg, which is only used for video recording). The runtime stage is plain debian:bookworm-slim with only the node binary copied in, so the image has no npm, npx, yarn or corepack. If a package starts failing in the image only, check whether it reads one of the deleted file types at runtime.

docker compose build

To rebuild without the layer cache:

docker compose build --no-cache

Licenses

The project is MIT-licensed (LICENSE); the license field in all three package.json files says the same.

The image ships third-party code: the server's production dependencies, the client's production dependencies (bundled into its build), the Node.js binary, Chromium and Debian packages. Their notices are in the image as follows:

  • npm packages and Node.js: the licenses stage of the Dockerfile runs scripts/third-party-licenses.mjs, which writes the project's LICENSE followed by every production dependency of both workspaces with its license files (or, for the few packages that ship none, the declared license, author and source). Node.js's LICENSE for the exact version in the image is fetched from GitHub and appended. The result is /app/client/licenses.txt, which the app serves at /licenses.txt. The build fails if a package has no license information at all.
  • Server packages' own license files stay in node_modules: the slimming step deletes Markdown files except LICENSE*, NOTICE* and COPYING*.
  • Chromium: LICENSE.headless_shell next to the binary under /ms-playwright.
  • Debian packages: /usr/share/doc/*/copyright.

The image also has the OCI labels org.opencontainers.image.licenses=MIT, source, title and version (CI passes the version as the VERSION build arg).

To check the notices locally, or to review a new dependency's license, run from the repository root:

node scripts/third-party-licenses.mjs > /tmp/licenses.txt

Before adding a dependency, check its license is compatible with MIT distribution (permissive licenses such as MIT, ISC, BSD, Apache-2.0 are; copyleft ones such as GPL or AGPL need a closer look).


CI (Gitea Actions)

.gitea/workflows/ci.yml runs on pushes to master, on pull requests and on release tags (vX.Y.Z):

  • changes (not on tags): detects which of server/ and client/ changed (via dorny/paths-filter), also treating root package.json/package-lock.json and the workflow file itself as affecting both.
  • server (skipped if nothing under server/ changed; always runs on tags): lint (no warnings allowed), type-check, tests, build.
  • client (skipped if nothing under client/ changed; always runs on tags): lint (no warnings allowed), build, Storybook tests in headless Chromium.
  • image (release tags only): after the server and client jobs pass, checks that the tag matches the root package.json version (v<version>), builds the Docker image and pushes it to the Gitea container registry set by REGISTRY at the top of the workflow (git.hq.ars9.space) as <registry>/<owner>/<repo>:<version> and :latest. Merges to master don't build images.

The image job logs in as the repository owner with the REGISTRY_TOKEN secret: a Gitea access token with package write access (Settings → Applications → token with write:package), added under the repository's Settings → Actions → Secrets. It runs docker on the runner, so the runner must give jobs access to a Docker daemon (act_runner mounts the host's Docker socket into job containers by default).

Jobs clone the repository from the address the runner was registered with. If that's an internal hostname (such as http://gitea:3000 from Gitea's compose network), job containers can't resolve it: either register the runner with the public URL, or set container.network in the act_runner config to Gitea's Docker network so jobs join it.


Running with Docker Compose

Start (detached):

docker compose up -d

Build and start in one step:

docker compose up -d --build

Everything is served by the single liqa container at http://localhost:13000: the UI at /, the REST API at /api/v1, MCP at /mcp, Swagger at /docs, health at /health and the client's public configuration at /manifest.json. Unknown paths outside those routes get the client's index.html, so client-side routes survive a page refresh.

When switching from the old two-container setup (server + client), remove the old containers with docker compose up -d --build --remove-orphans.

The SQLite database and stored files are persisted in ./data/, which is volume-mounted into the container.

Stop and remove containers:

docker compose down

View logs:

docker compose logs -f

Client overview

The client (client/) is a React 19 + TypeScript SPA built with Vite. It provides a browser UI for managing scenarios, runs, snippets, credentials, environments and sessions, and for importing exported YAML.

Tech stack

Library Purpose
React 19 + TypeScript UI framework
Vite 6 Dev server and bundler
react-router-dom 7 (BrowserRouter) Client-side routing
@tanstack/react-query Data fetching (cross-scenario runs list)
i18next + react-i18next Internationalisation
lucide-react Icon library
@monaco-editor/react Code editor for steps and snippets
react-markdown + highlight.js Markdown descriptions and code highlighting
yaml YAML export of entities
moment Relative timestamp formatting
ESLint 10 + Prettier Linting and formatting
Storybook 10 + Vitest Component development and testing

Directory layout

client/
  src/
    api/          API client (client.ts) and types (types.ts)
    components/   modals/ - run, export, export logs and delete confirmation modals
    hooks/        useTheme, usePageTitle, useRunDefaults, useImportFile, useRunNotifications
    i18n/         i18next bootstrap + locales/en.json
    lib/          highlight.js setup, API error toast events
    pages/        scenario/, run/, snippet/, credential/, environment/, session/, import/
    ui/           Reusable component library (Badge, Button, Card, CodeEditor, Modal,
                  Select, SidePanel, Table, Toast, ... - see ui/index.ts)
  .storybook/
    stories/      *.stories.tsx for ui/ components

Routing

BrowserRouter with plain paths. The root and unknown paths redirect to /scenarios. The server answers these paths with the client's index.html (see architecture.md), so they work on refresh.

Route Page
/scenarios Scenario list
/scenarios/:id Scenario details (steps, versions, credentials, used snippets, files)
/scenarios/:id/runs, /scenarios/:id/runs/:runId Runs of a scenario, run details
/runs Cross-scenario runs list
/snippets, /credentials, /environments, /sessions Entity lists (plus /:id, /new, /:id/edit where applicable)
/import Central import

Theming

Light/dark theme is toggled by the ThemeSwitcher button in the sidebar footer. The selection is persisted to localStorage and applied as data-theme on <html> before React mounts (anti-FOUC inline script in index.html).

The sidebar footer also shows the server's app version from /manifest.json (see architecture.md), falling back to the client build's version (read from the workspace root package.json and injected via Vite define). If the two differ, the version's tooltip suggests reloading the page.