Compare commits

..
2 Commits
Author SHA1 Message Date
ars9 8aadbd25fb chore(deps): update dependencies and bring storybook up to date
- update client and server dependencies within their ranges, keeping
  all playwright packages aligned at 1.63
- fix code flagged by react-hooks 7.1: refs written during render and
  synchronous setState in effects
- add stories for the ui components that had none, with interaction
  tests, and cover all DescriptionList layouts
- drop the unused mdx stories pattern
- run the storybook tests as part of the pre-commit checks
- bump version to 1.20.1
2026-09-30 22:05:21 +03:00
ars9 9f341719ef feat(database): manage the schema with typeorm migrations
- replace synchronize with migrations, starting with a generated initial
  migration that adopts databases created by synchronize
- apply pending migrations in the docker container before the server,
  and refuse to start the server while migrations are pending
- share one entity list and data source between app, migration runner,
  typeorm cli and tests; tests run the real migrations
- add migration:run, migration:show and migration:generate scripts and
  a test that fails when entities and migrations differ
- exit with code 1 when startup fails
- bump version to 1.20.0
2026-09-30 21:45:42 +03:00
33 changed files with 5571 additions and 3693 deletions

No files matched your search

+36
View File
@@ -2,6 +2,42 @@
All notable changes to this project will be documented in this file.
## 1.20.1 - 2026-09-30
Changes since 1.20.0:
### Added
- Storybook stories for AutoRefreshIndicator, CodeBlock, CodeEditor, MarkdownContent, Modal, Notification, Search, Textarea, Toast and UuidBadge, with interaction tests where the component has behavior. The DescriptionList story covers all four layouts.
### Changed
- Updated all client and server dependencies to the latest versions within their ranges, including React 19.3, Storybook 10.6, NestJS 11.2, TypeORM 0.3.31 and Playwright 1.63 (the Docker image now ships HeadlessChrome 153).
- The Storybook tests are part of the pre-commit checks.
### Fixed
- Code flagged by the stricter React hooks lint rules: the runs list no longer writes refs during render, and the scenario list, scenario files and sessions list no longer set state synchronously inside effects.
## 1.20.0 - 2026-09-30
Changes since 1.19.3:
### Added
- Database migrations: the SQLite schema is now managed by TypeORM migrations (`server/src/database/migrations/`) instead of `synchronize`. The Docker container applies pending migrations before starting the server, and the server refuses to start while migrations are pending.
- `npm -w server run migration:run`, `migration:show` and `migration:generate` scripts; the workflow for schema changes is described in `docs/development.md`.
- A test that fails when an entity changes without a matching migration.
### Changed
- Databases created before migrations (by `synchronize`) are adopted automatically: the initial migration is recorded as applied without changing them.
- Tests now build their database from the migrations instead of `synchronize`.
### Fixed
- Startup failures were logged but the process exited with code 0; they now exit with code 1.
## 1.19.3 - 2026-09-30
Changes since 1.19.2:
+4 -3
View File
@@ -4,7 +4,7 @@ liqa is a browser automation and QA service for web applications. It stores **sc
It is an npm workspace with two packages:
- `server/` - NestJS API + scheduler + MCP server, SQLite via TypeORM (`synchronize: true`, so entity changes alter the schema on startup - there are no migrations). REST API under `/api/v1`, Swagger UI at `/docs`, MCP at `/mcp`, health at `/health`.
- `server/` - NestJS API + scheduler + MCP server, SQLite via TypeORM; the schema is managed by migrations in `server/src/database/migrations/` (see "Database migrations" in `docs/development.md`). Any entity change needs a generated migration - `test/migrations.spec.ts` fails otherwise. REST API under `/api/v1`, Swagger UI at `/docs`, MCP at `/mcp`, health at `/health`.
- `client/` - React 19 + Vite SPA (i18n strings in `client/src/i18n/locales/en.json`).
## Docs
@@ -24,7 +24,7 @@ It is an npm workspace with two packages:
Rebuild and restart: `docker compose up -d --build`. Check with `docker compose ps`, `curl localhost:13000/health` and `docker compose logs liqa`.
The live database is `data/liqa.db`. Because the schema is synced on startup, back it up before deploying entity changes.
The live database is `data/liqa.db`. The container applies pending migrations before starting the server, so back it up before deploying a release with new migrations.
## Code quality
@@ -34,10 +34,11 @@ Before committing, run and pass all of these (and report the results):
# server
cd server && npm run lint && npx tsc --noEmit -p tsconfig.json && npm test && npm run build
# client
cd client && npm run lint && npm run build
cd client && npm run lint && npm run build && npm run test:storybook
```
- Lint must stay clean in both packages (no errors, no warnings).
- `test:storybook` runs every story in `client/.storybook/stories/` in headless Chromium (needs `npx playwright install --only-shell chromium` once, and again after Playwright updates). Add or update a story when adding or changing a `client/src/ui/` component.
- Don't run prettier over whole existing files; several aren't prettier-clean and it produces unrelated diffs. Format only new files or the lines you changed.
- Server tests use an in-memory DB, a Playwright mock (`server/test/__mocks__/playwright.ts`), and a temporary files directory per test app (see `server/test/app.harness.ts`).
- Import Node.js built-in modules with the `node:` prefix (`node:fs`, `node:path`, `node:crypto`, ...).
+3 -1
View File
@@ -87,4 +87,6 @@ RUN mkdir -p data
EXPOSE 3000
CMD ["node", "dist/main"]
# Apply pending database migrations, then start the server (which refuses to
# start while migrations are pending)
CMD ["sh", "-c", "node dist/database/migrate && exec node dist/main"]
+1 -1
View File
@@ -12,7 +12,7 @@ function getAbsolutePath(value: string) {
return dirname(fileURLToPath(import.meta.resolve(`${value}/package.json`)));
}
const config: StorybookConfig = {
stories: ['./stories/**/*.mdx', './stories/**/*.stories.@(js|jsx|mjs|ts|tsx)'],
stories: ['./stories/**/*.stories.@(js|jsx|mjs|ts|tsx)'],
addons: [
getAbsolutePath('@storybook/addon-vitest'),
getAbsolutePath('@storybook/addon-a11y'),
@@ -0,0 +1,25 @@
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect, fn, userEvent, within } from 'storybook/test';
import { AutoRefreshIndicator } from '../../src/ui/AutoRefreshIndicator/AutoRefreshIndicator';
const meta: Meta<typeof AutoRefreshIndicator> = {
title: 'UI/AutoRefreshIndicator',
component: AutoRefreshIndicator,
parameters: { layout: 'centered' },
tags: ['autodocs'],
args: { onClick: fn() },
};
export default meta;
type Story = StoryObj<typeof AutoRefreshIndicator>;
export const Active: Story = { args: { active: true, pulseKey: 1 } };
export const Idle: Story = { args: { active: false } };
export const Errored: Story = { args: { active: false, error: true } };
export const ClickToRefresh: Story = {
args: { active: false },
play: async ({ args, canvasElement }) => {
await userEvent.click(within(canvasElement).getByRole('button'));
await expect(args.onClick).toHaveBeenCalled();
},
};
@@ -0,0 +1,33 @@
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect } from 'storybook/test';
import { CodeBlock } from '../../src/ui/CodeBlock/CodeBlock';
const meta: Meta<typeof CodeBlock> = {
title: 'UI/CodeBlock',
component: CodeBlock,
parameters: { layout: 'padded' },
tags: ['autodocs'],
};
export default meta;
type Story = StoryObj<typeof CodeBlock>;
export const JavaScript: Story = {
args: {
language: 'javascript',
code: "const url = context.getEnv('BASE_URL');\nawait context.page.goto(url);\nreturn await context.page.title();",
},
play: async ({ canvasElement }) => {
// Highlighting splits the code into many spans: check the whole text and
// that highlighting actually happened
const code = canvasElement.querySelector('code');
await expect(code).toHaveTextContent('await context.page.goto(url);');
await expect(code?.querySelector('.hljs-keyword')).not.toBeNull();
},
};
export const Json: Story = {
args: {
language: 'json',
code: JSON.stringify({ status: 'pass', output: { title: 'Dashboard' } }, null, 2),
},
};
@@ -0,0 +1,35 @@
import React, { useState } from 'react';
import type { Meta, StoryObj } from '@storybook/react-vite';
import { CodeEditor, type CodeEditorProps } from '../../src/ui/CodeEditor/CodeEditor';
// Monaco is loaded from a CDN at runtime, so these stories only check that the
// editor mounts; they have no interaction tests.
function ControlledEditor(
props: Omit<CodeEditorProps, 'value' | 'onChange'> & { initial: string },
) {
const { initial, ...rest } = props;
const [value, setValue] = useState(initial);
return <CodeEditor {...rest} value={value} onChange={setValue} />;
}
const meta: Meta<typeof ControlledEditor> = {
title: 'UI/CodeEditor',
component: ControlledEditor,
parameters: { layout: 'padded' },
tags: ['autodocs'],
};
export default meta;
type Story = StoryObj<typeof ControlledEditor>;
export const JavaScript: Story = {
args: {
language: 'javascript',
rows: 8,
initial:
"await context.page.goto(context.getEnv('BASE_URL'));\nreturn await context.page.title();",
},
};
export const WithError: Story = {
args: { language: 'javascript', rows: 4, error: true, initial: 'await context.page.goto(' },
};
@@ -15,7 +15,7 @@ const meta: Meta<typeof DescriptionList> = {
),
],
argTypes: {
layout: { control: 'select', options: ['inline', 'compact'] },
layout: { control: 'select', options: ['inline', 'compact', 'comfortable', 'grid'] },
},
};
export default meta;
@@ -35,6 +35,14 @@ export const Compact: Story = {
args: { items: urlItems, layout: 'compact' },
};
export const Comfortable: Story = {
args: { items: urlItems, layout: 'comfortable' },
};
export const Grid: Story = {
args: { items: urlItems, layout: 'grid' },
};
export const WithLinks: Story = {
args: {
layout: 'inline',
@@ -0,0 +1,35 @@
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect, within } from 'storybook/test';
import { MarkdownContent } from '../../src/ui/MarkdownContent/MarkdownContent';
const meta: Meta<typeof MarkdownContent> = {
title: 'UI/MarkdownContent',
component: MarkdownContent,
parameters: { layout: 'padded' },
tags: ['autodocs'],
};
export default meta;
type Story = StoryObj<typeof MarkdownContent>;
export const Description: Story = {
args: {
content: [
'## Login flow',
'',
'Logs in with the **test user** and checks the dashboard.',
'',
'- Uses the `BASE_URL` environment value',
'- Needs the `user` credential alias',
'',
'```js',
"await context.page.goto(context.getEnv('BASE_URL'));",
'```',
].join('\n'),
},
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(canvas.getByRole('heading', { name: 'Login flow' })).toBeInTheDocument();
await expect(canvas.getByText('test user').tagName).toBe('STRONG');
await expect(canvas.getAllByRole('listitem')).toHaveLength(2);
},
};
@@ -0,0 +1,63 @@
import React, { useState } from 'react';
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect, userEvent, within } from 'storybook/test';
import { Modal } from '../../src/ui/Modal/Modal';
import { Button } from '../../src/ui/Button/Button';
function ModalDemo({ title, body }: { title: string; body: string }) {
const [open, setOpen] = useState(false);
return (
<>
<Button onClick={() => setOpen(true)}>Open modal</Button>
<Modal
open={open}
title={title}
onClose={() => setOpen(false)}
footer={
<>
<Button variant="secondary" onClick={() => setOpen(false)}>
Cancel
</Button>
<Button onClick={() => setOpen(false)}>Confirm</Button>
</>
}
>
{body}
</Modal>
</>
);
}
const meta: Meta<typeof ModalDemo> = {
title: 'UI/Modal',
component: ModalDemo,
parameters: { layout: 'centered' },
tags: ['autodocs'],
args: { title: 'Delete scenario', body: 'This deletes the scenario, its versions and its runs.' },
};
export default meta;
type Story = StoryObj<typeof ModalDemo>;
export const Default: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(canvas.getByRole('button', { name: 'Open modal' }));
const dialog = canvas.getByRole('dialog', { name: 'Delete scenario' });
await expect(dialog).toHaveTextContent('its versions and its runs');
await userEvent.click(within(dialog).getByRole('button', { name: 'Cancel' }));
await expect(canvas.queryByRole('dialog')).not.toBeInTheDocument();
},
};
export const ClosesOnEscapeAndBackdrop: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(canvas.getByRole('button', { name: 'Open modal' }));
await userEvent.keyboard('{Escape}');
await expect(canvas.queryByRole('dialog')).not.toBeInTheDocument();
await userEvent.click(canvas.getByRole('button', { name: 'Open modal' }));
await userEvent.click(canvas.getByRole('presentation'));
await expect(canvas.queryByRole('dialog')).not.toBeInTheDocument();
},
};
@@ -0,0 +1,30 @@
import type { Meta, StoryObj } from '@storybook/react-vite';
import { Notification } from '../../src/ui/Notification/Notification';
const meta: Meta<typeof Notification> = {
title: 'UI/Notification',
component: Notification,
parameters: { layout: 'padded' },
tags: ['autodocs'],
argTypes: {
variant: { control: 'select', options: ['info', 'success', 'error', 'warning', 'note'] },
},
};
export default meta;
type Story = StoryObj<typeof Notification>;
export const Info: Story = {
args: {
variant: 'info',
title: 'Traces are off',
children: 'Enable them per run in the run modal.',
},
};
export const Success: Story = { args: { variant: 'success', children: 'Scenario imported.' } };
export const Error: Story = {
args: { variant: 'error', title: 'Run failed', children: 'Step 2 timed out after 60s.' },
};
export const Warning: Story = {
args: { variant: 'warning', children: 'The environment of this run no longer exists.' },
};
export const Note: Story = { args: { variant: 'note', children: 'Pinned snippet version: v3.' } };
@@ -0,0 +1,34 @@
import React, { useState } from 'react';
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect, userEvent, within } from 'storybook/test';
import { Search } from '../../src/ui/Search/Search';
function ControlledSearch({ placeholder }: { placeholder?: string }) {
const [value, setValue] = useState('');
return (
<div>
<Search value={value} onChange={setValue} placeholder={placeholder} />
<p data-testid="query">{value}</p>
</div>
);
}
const meta: Meta<typeof ControlledSearch> = {
title: 'UI/Search',
component: ControlledSearch,
parameters: { layout: 'centered' },
tags: ['autodocs'],
};
export default meta;
type Story = StoryObj<typeof ControlledSearch>;
export const Default: Story = {};
export const Typing: Story = {
args: { placeholder: 'Filter logs…' },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.type(canvas.getByPlaceholderText('Filter logs…'), 'timeout');
await expect(canvas.getByTestId('query')).toHaveTextContent('timeout');
},
};
@@ -0,0 +1,37 @@
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect, within } from 'storybook/test';
import { Textarea } from '../../src/ui/Textarea/Textarea';
const meta: Meta<typeof Textarea> = {
title: 'UI/Textarea',
component: Textarea,
parameters: { layout: 'centered' },
tags: ['autodocs'],
decorators: [
(Story) => (
<div style={{ width: 360 }}>
<Story />
</div>
),
],
};
export default meta;
type Story = StoryObj<typeof Textarea>;
export const Default: Story = {
args: { label: 'Description', placeholder: 'Describe this scenario', rows: 4 },
};
export const WithHint: Story = {
args: { label: 'Description', hint: 'Markdown is supported.', rows: 4 },
};
export const WithError: Story = {
args: { label: 'Description', error: 'Description is too long.', rows: 4 },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const textarea = canvas.getByLabelText('Description');
await expect(textarea).toHaveAttribute('aria-invalid', 'true');
await expect(canvas.getByRole('alert')).toHaveTextContent('Description is too long.');
},
};
@@ -0,0 +1,41 @@
import React from 'react';
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect, userEvent, within } from 'storybook/test';
import { ToastProvider } from '../../src/ui/Toast/ToastProvider';
import { useToast } from '../../src/ui/Toast/useToast';
import { Button } from '../../src/ui/Button/Button';
function ToastButtons() {
const toast = useToast();
return (
<div style={{ display: 'flex', gap: 8 }}>
<Button onClick={() => toast.success('Scenario run started')}>Success</Button>
<Button variant="danger" onClick={() => toast.error('Environment not found')}>
Error
</Button>
<Button variant="secondary" onClick={() => toast.info('Traces are off')}>
Info
</Button>
</div>
);
}
const meta: Meta<typeof ToastProvider> = {
title: 'UI/Toast',
component: ToastProvider,
parameters: { layout: 'centered' },
tags: ['autodocs'],
args: { durationMs: 3500, children: <ToastButtons /> },
};
export default meta;
type Story = StoryObj<typeof ToastProvider>;
export const Default: Story = {};
export const ShowsToast: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(canvas.getByRole('button', { name: 'Success' }));
await expect(await within(document.body).findByText('Scenario run started')).toBeVisible();
},
};
@@ -0,0 +1,21 @@
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect, within } from 'storybook/test';
import { UuidBadge } from '../../src/ui/UuidBadge/UuidBadge';
const meta: Meta<typeof UuidBadge> = {
title: 'UI/UuidBadge',
component: UuidBadge,
parameters: { layout: 'centered' },
tags: ['autodocs'],
};
export default meta;
type Story = StoryObj<typeof UuidBadge>;
export const Default: Story = {
args: { id: '61d2b1c2-7664-4dca-974f-436c2ee90b2e' },
play: async ({ canvasElement }) => {
// Shows a shortened id and keeps the full one in the tooltip
const badge = within(canvasElement).getByTitle('61d2b1c2-7664-4dca-974f-436c2ee90b2e');
await expect(badge).toHaveTextContent('61d2…0b2e');
},
};
+6 -4
View File
@@ -68,10 +68,12 @@ export function RunsPage() {
const sortDirRef = useRef(sortDir);
const pageRef = useRef(page);
const pageSizeRef = useRef(pageSize);
sortByRef.current = sortBy;
sortDirRef.current = sortDir;
pageRef.current = page;
pageSizeRef.current = pageSize;
useEffect(() => {
sortByRef.current = sortBy;
sortDirRef.current = sortDir;
pageRef.current = page;
pageSizeRef.current = pageSize;
}, [sortBy, sortDir, page, pageSize]);
const load = useCallback(
(orderBy: string, orderDir: SortDirection, p: number, limit: number) => {
@@ -98,7 +98,7 @@ export function ScenarioDetailPage() {
// Files state
const [files, setFiles] = useState<FileMetadata[]>([]);
const [filesLoading, setFilesLoading] = useState(false);
const [filesLoading, setFilesLoading] = useState(true);
const [uploadOpen, setUploadOpen] = useState(false);
const [uploadFile, setUploadFile] = useState<File | null>(null);
const [uploadExpiresAt, setUploadExpiresAt] = useState('');
@@ -106,9 +106,10 @@ export function ScenarioDetailPage() {
const [usedSnippets, setUsedSnippets] = useState<UsedSnippet[]>([]);
const loadFiles = useCallback(async () => {
// Fetches the files; `filesLoading` starts as true for the fetch from the
// load effect, and `loadFiles` sets it for reloads (e.g. after an upload)
const fetchFiles = useCallback(async () => {
if (!id) return;
setFilesLoading(true);
try {
const result = await scenarioFiles.list(id);
setFiles(result.items);
@@ -117,6 +118,11 @@ export function ScenarioDetailPage() {
}
}, [id]);
const loadFiles = async () => {
setFilesLoading(true);
await fetchFiles();
};
useEffect(() => {
if (!id) return;
void scenarios
@@ -126,7 +132,7 @@ export function ScenarioDetailPage() {
setScenarioCreds(s.scenarioCredentials ?? []);
})
.finally(() => setLoading(false));
void loadFiles();
void fetchFiles();
scenarios
.usedSnippets(id)
.then(setUsedSnippets)
@@ -141,7 +147,7 @@ export function ScenarioDetailPage() {
setEnvs(r.data);
})
.catch(() => {});
}, [id, loadFiles]);
}, [id, fetchFiles]);
const openRunModal = () => {
// Preselect the scenario's default environment (or the first one) when the
+10 -4
View File
@@ -68,9 +68,10 @@ export function ScenariosPage() {
const [pageSize, setPageSize] = useState(10);
const [total, setTotal] = useState(0);
const load = useCallback(
// Fetches a page; `loading` starts as true for the initial fetch from the
// effect, and `load` sets it for later ones (no setState inside the effect)
const fetchPage = useCallback(
(orderBy: string, orderDir: SortDirection, p: number, limit: number) => {
setLoading(true);
Promise.all([
scenarios.list(p, limit, orderBy, orderDir === 'asc' ? 'ASC' : 'DESC'),
environments.list(1, 200),
@@ -88,9 +89,14 @@ export function ScenariosPage() {
[toast],
);
const load = (orderBy: string, orderDir: SortDirection, p: number, limit: number) => {
setLoading(true);
fetchPage(orderBy, orderDir, p, limit);
};
useEffect(() => {
load('updatedAt', 'desc', 1, 10);
}, [load]);
fetchPage('updatedAt', 'desc', 1, 10);
}, [fetchPage]);
const handleSort = (key: string, dir: SortDirection) => {
setSortBy(key);
+8 -3
View File
@@ -23,7 +23,9 @@ export function SessionsPage() {
usePageTitle(t('sessions.title'));
const navigate = useNavigate();
const [items, setItems] = useState<Session[]>([]);
const [loading, setLoading] = useState(true);
// Loading until the current query's result has arrived (derived, so the
// loading effect doesn't have to set state synchronously)
const [loadedQuery, setLoadedQuery] = useState<string | null>(null);
const [deleteId, setDeleteId] = useState<string | null>(null);
const [deleting, setDeleting] = useState(false);
const [sortBy, setSortBy] = useState('id');
@@ -32,15 +34,18 @@ export function SessionsPage() {
const [pageSize, setPageSize] = useState(10);
const [total, setTotal] = useState(0);
const query = `${sortBy}|${sortDir}|${page}|${pageSize}`;
const loading = loadedQuery !== query;
const load = useCallback((orderBy: string, orderDir: SortDirection, p: number, limit: number) => {
setLoading(true);
const key = `${orderBy}|${orderDir}|${p}|${limit}`;
sessions
.list(p, limit, orderBy, orderDir === 'asc' ? 'ASC' : 'DESC')
.then((res) => {
setItems(res.data);
setTotal(res.total);
})
.finally(() => setLoading(false));
.finally(() => setLoadedQuery(key));
}, []);
useEffect(() => {
+2 -1
View File
@@ -50,6 +50,7 @@ Everything runs in one container built from the root `Dockerfile`: the server se
Cross-cutting pieces:
- `config/app.config.ts` - all settings, validated at startup (see `.env.example`).
- `database/` - the entity list, TypeORM data source options (shared by the app, the migration runner and the TypeORM CLI), the migration runner and the migrations.
- `common/app-package.ts` - app name and version, read from the workspace root `package.json`.
- `common/client-static.ts` - serving the built client, with the `index.html` fallback for client-side routes.
- `common/body-parsers.ts` - YAML request bodies (`application/yaml`) for the import endpoint.
@@ -57,7 +58,7 @@ Cross-cutting pieces:
## Data
SQLite via TypeORM with `synchronize: true`: entity changes alter the schema on startup, and there are no migrations. The main tables:
SQLite via TypeORM. The schema is managed by migrations (`server/src/database/migrations/`, registered in `migrations/index.ts`); `synchronize` is off. The Docker container applies pending migrations (`node dist/database/migrate`) before starting the server, and the server refuses to start while migrations are pending. See "Database migrations" in [development.md](development.md). The main tables:
- `scenarios` / `scenario_steps` hold the **latest** version of each scenario; `scenario_versions` holds a snapshot of every version (content + steps).
- `snippets` holds the latest version of each snippet; `snippet_versions` holds every version. Step code pins a version with `runSnippet("alias@N")`.
+30
View File
@@ -86,6 +86,36 @@ The built files are emitted to `client/dist/`.
---
## Database migrations
The SQLite schema is managed by TypeORM migrations in `server/src/database/migrations/`; `synchronize` is off.
- **Docker:** the container applies pending migrations before starting the server.
- **Locally:** build, then apply them before starting the server (it refuses to start while migrations are pending):
```bash
npm -w server run build
npm -w server run migration:run # apply pending migrations to DB_PATH
npm -w server run migration:show # list migrations and whether they're applied
```
The scripts run from `server/`, so `DB_PATH` (default `data/liqa.db`) is relative to it, and they read `server/.env` if present, like the server does.
**Changing the schema** (adding or changing an entity):
1. Change the entity and build (`npm -w server run build`).
2. Make sure your local database is on the latest migration (`migration:run`).
3. Generate a migration from the difference between the entities and that database:
```bash
npm -w server run migration:generate -- src/database/migrations/AddSomething
```
4. Review the generated file, then add its class to `MIGRATIONS` in `src/database/migrations/index.ts` (they're listed explicitly, oldest first).
5. Run the tests: `test/migrations.spec.ts` checks that the migrations produce exactly the schema the entities describe, and fails if an entity change has no migration.
The initial migration (`Initial…`) adopts databases created before migrations existed (by `synchronize`): if the tables already exist, it's only recorded as applied.
## Running tests
### Server (Jest)
+4629 -3596
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "liqa",
"version": "1.19.3",
"version": "1.20.1",
"private": true,
"engines": {
"node": ">=24"
+3
View File
@@ -7,6 +7,9 @@
"start": "nest start",
"start:dev": "nest start --watch",
"start:prod": "node dist/main",
"migration:run": "node dist/database/migrate",
"migration:show": "typeorm migration:show -d dist/database/data-source.js",
"migration:generate": "typeorm migration:generate -d dist/database/data-source.js",
"format": "prettier --write \"src/**/*.ts\" \"test/**/*.ts\"",
"lint": "eslint \"src/**/*.ts\" \"test/**/*.ts\"",
"lint:fix": "eslint --fix \"src/**/*.ts\" \"test/**/*.ts\"",
+3 -37
View File
@@ -5,29 +5,15 @@ import { ServeStaticModule } from "@nestjs/serve-static";
import { TypeOrmModule } from "@nestjs/typeorm";
import { BrowserModule } from "./browser/browser.module";
import { clientStaticOptions } from "./common/client-static";
import { dataSourceOptions } from "./database/data-source";
import { AppConfig, validateAppConfig } from "./config/app.config";
import { CredentialEntity } from "./credential/credential.entity";
import { CredentialModule } from "./credential/credential.module";
import { EnvironmentEntity } from "./environment/environment.entity";
import { EnvironmentModule } from "./environment/environment.module";
import { FileEntity } from "./file/file.entity";
import { FileModule } from "./file/file.module";
import { ScenarioFileEntity } from "./file/scenario-file.entity";
import { ScenarioRunFileEntity } from "./file/scenario-run-file.entity";
import { HealthModule } from "./health/health.module";
import { ImportModule } from "./import/import.module";
import { McpModule } from "./mcp/mcp.module";
import { ScenarioCredentialEntity } from "./scenario/scenario-credential.entity";
import { ScenarioRunLogEntity } from "./scenario/scenario-run-log.entity";
import { ScenarioRunStepEntity } from "./scenario/scenario-run-step.entity";
import { ScenarioRunEntity } from "./scenario/scenario-run.entity";
import { ScenarioStepEntity } from "./scenario/scenario-step.entity";
import { ScenarioVersionEntity } from "./scenario/scenario-version.entity";
import { SnippetVersionEntity } from "./snippet/snippet-version.entity";
import { ScenarioEntity } from "./scenario/scenario.entity";
import { ScenarioModule } from "./scenario/scenario.module";
import { SessionEntity } from "./session/session.entity";
import { SnippetEntity } from "./snippet/snippet.entity";
import { SnippetModule } from "./snippet/snippet.module";
@Module({
@@ -45,28 +31,8 @@ import { SnippetModule } from "./snippet/snippet.module";
}),
TypeOrmModule.forRootAsync({
inject: [ConfigService],
useFactory: (config: ConfigService<AppConfig, true>) => ({
type: "better-sqlite3",
database: config.get("DB_PATH"),
entities: [
SessionEntity,
EnvironmentEntity,
CredentialEntity,
ScenarioEntity,
ScenarioStepEntity,
ScenarioVersionEntity,
ScenarioRunEntity,
ScenarioRunStepEntity,
ScenarioRunLogEntity,
ScenarioCredentialEntity,
SnippetEntity,
SnippetVersionEntity,
FileEntity,
ScenarioFileEntity,
ScenarioRunFileEntity,
],
synchronize: true,
}),
useFactory: (config: ConfigService<AppConfig, true>) =>
dataSourceOptions(config.get("DB_PATH")),
}),
BrowserModule,
EnvironmentModule,
+31
View File
@@ -0,0 +1,31 @@
import * as fs from "node:fs";
import { DataSource, type DataSourceOptions } from "typeorm";
import { validateAppConfig } from "../config/app.config";
import { ENTITIES } from "./entities";
import { MIGRATIONS } from "./migrations";
/**
* TypeORM options shared by the app, the migration runner and the TypeORM
* CLI. The schema is managed by migrations only (`synchronize: false`).
*/
export function dataSourceOptions(database: string): DataSourceOptions {
return {
type: "better-sqlite3",
database,
entities: ENTITIES,
migrations: MIGRATIONS,
synchronize: false,
};
}
// Outside Nest (migration runner, TypeORM CLI) nothing loads .env; load it
// like ConfigModule does: optional, and without overriding set variables
if (fs.existsSync(".env")) process.loadEnvFile(".env");
/**
* Data source for the TypeORM CLI and `migrate.ts`, using DB_PATH from the
* environment or .env (default `data/liqa.db`).
*/
export default new DataSource(
dataSourceOptions(validateAppConfig(process.env).DB_PATH),
);
+34
View File
@@ -0,0 +1,34 @@
import { CredentialEntity } from "../credential/credential.entity";
import { EnvironmentEntity } from "../environment/environment.entity";
import { FileEntity } from "../file/file.entity";
import { ScenarioFileEntity } from "../file/scenario-file.entity";
import { ScenarioRunFileEntity } from "../file/scenario-run-file.entity";
import { ScenarioCredentialEntity } from "../scenario/scenario-credential.entity";
import { ScenarioRunLogEntity } from "../scenario/scenario-run-log.entity";
import { ScenarioRunStepEntity } from "../scenario/scenario-run-step.entity";
import { ScenarioRunEntity } from "../scenario/scenario-run.entity";
import { ScenarioStepEntity } from "../scenario/scenario-step.entity";
import { ScenarioVersionEntity } from "../scenario/scenario-version.entity";
import { ScenarioEntity } from "../scenario/scenario.entity";
import { SessionEntity } from "../session/session.entity";
import { SnippetVersionEntity } from "../snippet/snippet-version.entity";
import { SnippetEntity } from "../snippet/snippet.entity";
/** Every TypeORM entity of the app (used by the app, the migrations CLI and tests). */
export const ENTITIES = [
SessionEntity,
EnvironmentEntity,
CredentialEntity,
ScenarioEntity,
ScenarioStepEntity,
ScenarioVersionEntity,
ScenarioRunEntity,
ScenarioRunStepEntity,
ScenarioRunLogEntity,
ScenarioCredentialEntity,
SnippetEntity,
SnippetVersionEntity,
FileEntity,
ScenarioFileEntity,
ScenarioRunFileEntity,
];
+24
View File
@@ -0,0 +1,24 @@
import dataSource from "./data-source";
/**
* Applies pending migrations to the database at DB_PATH, then exits. Runs
* before the server in the Docker image (`node dist/database/migrate`).
*/
async function migrate(): Promise<void> {
await dataSource.initialize();
try {
const applied = await dataSource.runMigrations({ transaction: "each" });
console.log(
applied.length > 0
? `Applied ${applied.length} migration(s): ${applied.map((m) => m.name).join(", ")}`
: "Database schema is up to date",
);
} finally {
await dataSource.destroy();
}
}
migrate().catch((err: unknown) => {
console.error("Migration failed:", err);
process.exit(1);
});
@@ -0,0 +1,277 @@
import { MigrationInterface, QueryRunner } from "typeorm";
/**
* Initial schema. Databases created before migrations existed (by TypeORM
* `synchronize`) already have this schema: for them this migration only gets
* recorded as applied, and later migrations run from there.
*/
export class Initial1790793447827 implements MigrationInterface {
name = "Initial1790793447827";
public async up(queryRunner: QueryRunner): Promise<void> {
if (await queryRunner.hasTable("scenarios")) return;
await queryRunner.query(
`CREATE TABLE "credentials" ("id" varchar PRIMARY KEY NOT NULL, "name" varchar NOT NULL, "data" text, "lastUsedAt" datetime, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`CREATE TABLE "environments" ("id" varchar PRIMARY KEY NOT NULL, "name" varchar NOT NULL, "description" text, "data" text NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_9d00da4836959b86d7590f60c57" UNIQUE ("name"))`,
);
await queryRunner.query(
`CREATE TABLE "files" ("id" varchar PRIMARY KEY NOT NULL, "originalName" varchar NOT NULL, "mimeType" varchar NOT NULL, "size" integer NOT NULL, "sha256" varchar NOT NULL, "filePath" varchar NOT NULL, "expiresAt" datetime, "createdAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`CREATE TABLE "scenario_credentials" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "credentialId" varchar NOT NULL, "alias" varchar NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_02756536236c408dbb58b3fc964" UNIQUE ("scenarioId", "alias"))`,
);
await queryRunner.query(
`CREATE TABLE "scenario_steps" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "order" integer NOT NULL DEFAULT (0), "title" text, "execCode" text, "timeoutSeconds" integer, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`CREATE TABLE "scenarios" ("id" varchar PRIMARY KEY NOT NULL, "name" varchar NOT NULL, "description" text, "environmentId" varchar, "timeoutSeconds" integer, "version" integer NOT NULL DEFAULT (1), "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`CREATE TABLE "scenario_files" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "fileId" varchar NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`CREATE TABLE "sessions" ("id" varchar PRIMARY KEY NOT NULL, "sessionName" varchar NOT NULL, "token" text NOT NULL, "cookies" text NOT NULL, "localStorage" text NOT NULL DEFAULT ('{}'), "status" varchar NOT NULL DEFAULT ('closed'), "lastUsedAt" datetime, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_b8f17f1ce30f015f494188c50bb" UNIQUE ("sessionName"))`,
);
await queryRunner.query(
`CREATE TABLE "scenario_run_steps" ("id" varchar PRIMARY KEY NOT NULL, "runId" varchar NOT NULL, "scenarioStepId" varchar, "title" text, "execCode" text, "timeoutSeconds" integer, "status" text NOT NULL DEFAULT ('waiting'), "order" integer NOT NULL DEFAULT (0), "description" text, "output" text, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`CREATE TABLE "scenario_runs" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "environmentId" varchar, "sessionId" varchar, "scenarioVersion" integer, "status" text NOT NULL DEFAULT ('pending'), "saveSession" boolean NOT NULL DEFAULT (0), "traceMode" text, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`CREATE TABLE "scenario_run_files" ("id" varchar PRIMARY KEY NOT NULL, "runId" varchar NOT NULL, "fileId" varchar NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`CREATE TABLE "scenario_run_logs" ("id" varchar PRIMARY KEY NOT NULL, "runId" varchar NOT NULL, "stepRunId" varchar, "level" text NOT NULL DEFAULT ('log'), "message" text NOT NULL, "seq" integer NOT NULL DEFAULT (0), "createdAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`CREATE TABLE "scenario_versions" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "version" integer NOT NULL, "name" varchar NOT NULL, "description" text, "environmentId" text, "timeoutSeconds" integer, "steps" text NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_e1a56dce2b7e0379bb5bbdf8c1f" UNIQUE ("scenarioId", "version"))`,
);
await queryRunner.query(
`CREATE TABLE "snippets" ("id" varchar PRIMARY KEY NOT NULL, "alias" varchar, "title" varchar NOT NULL DEFAULT (''), "description" text, "code" text NOT NULL, "version" integer NOT NULL DEFAULT (1), "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_bd062c68591ae917b720eb3427c" UNIQUE ("alias"))`,
);
await queryRunner.query(
`CREATE TABLE "snippet_versions" ("id" varchar PRIMARY KEY NOT NULL, "snippetId" varchar NOT NULL, "version" integer NOT NULL, "alias" varchar NOT NULL, "title" varchar NOT NULL DEFAULT (''), "description" text, "code" text NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_2487f4aabeb20c4192ee30e09a2" UNIQUE ("snippetId", "version"))`,
);
await queryRunner.query(
`CREATE TABLE "temporary_scenario_credentials" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "credentialId" varchar NOT NULL, "alias" varchar NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_02756536236c408dbb58b3fc964" UNIQUE ("scenarioId", "alias"), CONSTRAINT "FK_37f19dadd8e524b22a807ece1a9" FOREIGN KEY ("scenarioId") REFERENCES "scenarios" ("id") ON DELETE CASCADE ON UPDATE NO ACTION, CONSTRAINT "FK_96676b988693d5ca2923782530c" FOREIGN KEY ("credentialId") REFERENCES "credentials" ("id") ON DELETE CASCADE ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_scenario_credentials"("id", "scenarioId", "credentialId", "alias", "createdAt") SELECT "id", "scenarioId", "credentialId", "alias", "createdAt" FROM "scenario_credentials"`,
);
await queryRunner.query(`DROP TABLE "scenario_credentials"`);
await queryRunner.query(
`ALTER TABLE "temporary_scenario_credentials" RENAME TO "scenario_credentials"`,
);
await queryRunner.query(
`CREATE TABLE "temporary_scenario_steps" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "order" integer NOT NULL DEFAULT (0), "title" text, "execCode" text, "timeoutSeconds" integer, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "FK_4bab3637cc3bf37677bb01c7446" FOREIGN KEY ("scenarioId") REFERENCES "scenarios" ("id") ON DELETE CASCADE ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_scenario_steps"("id", "scenarioId", "order", "title", "execCode", "timeoutSeconds", "createdAt", "updatedAt") SELECT "id", "scenarioId", "order", "title", "execCode", "timeoutSeconds", "createdAt", "updatedAt" FROM "scenario_steps"`,
);
await queryRunner.query(`DROP TABLE "scenario_steps"`);
await queryRunner.query(
`ALTER TABLE "temporary_scenario_steps" RENAME TO "scenario_steps"`,
);
await queryRunner.query(
`CREATE TABLE "temporary_scenarios" ("id" varchar PRIMARY KEY NOT NULL, "name" varchar NOT NULL, "description" text, "environmentId" varchar, "timeoutSeconds" integer, "version" integer NOT NULL DEFAULT (1), "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "FK_ffdcf9186d95114f0dab068e20a" FOREIGN KEY ("environmentId") REFERENCES "environments" ("id") ON DELETE SET NULL ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_scenarios"("id", "name", "description", "environmentId", "timeoutSeconds", "version", "createdAt", "updatedAt") SELECT "id", "name", "description", "environmentId", "timeoutSeconds", "version", "createdAt", "updatedAt" FROM "scenarios"`,
);
await queryRunner.query(`DROP TABLE "scenarios"`);
await queryRunner.query(
`ALTER TABLE "temporary_scenarios" RENAME TO "scenarios"`,
);
await queryRunner.query(
`CREATE TABLE "temporary_scenario_files" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "fileId" varchar NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "FK_c41719984235f8984d1c6b92ef5" FOREIGN KEY ("scenarioId") REFERENCES "scenarios" ("id") ON DELETE CASCADE ON UPDATE NO ACTION, CONSTRAINT "FK_67013a49e7948fb530a7d956e2b" FOREIGN KEY ("fileId") REFERENCES "files" ("id") ON DELETE CASCADE ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_scenario_files"("id", "scenarioId", "fileId", "createdAt") SELECT "id", "scenarioId", "fileId", "createdAt" FROM "scenario_files"`,
);
await queryRunner.query(`DROP TABLE "scenario_files"`);
await queryRunner.query(
`ALTER TABLE "temporary_scenario_files" RENAME TO "scenario_files"`,
);
await queryRunner.query(
`CREATE TABLE "temporary_scenario_run_steps" ("id" varchar PRIMARY KEY NOT NULL, "runId" varchar NOT NULL, "scenarioStepId" varchar, "title" text, "execCode" text, "timeoutSeconds" integer, "status" text NOT NULL DEFAULT ('waiting'), "order" integer NOT NULL DEFAULT (0), "description" text, "output" text, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "FK_213c27405af3433011121babb58" FOREIGN KEY ("runId") REFERENCES "scenario_runs" ("id") ON DELETE CASCADE ON UPDATE NO ACTION, CONSTRAINT "FK_e84cd4c7725bdc5dfd09d18d92f" FOREIGN KEY ("scenarioStepId") REFERENCES "scenario_steps" ("id") ON DELETE SET NULL ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_scenario_run_steps"("id", "runId", "scenarioStepId", "title", "execCode", "timeoutSeconds", "status", "order", "description", "output", "createdAt", "updatedAt") SELECT "id", "runId", "scenarioStepId", "title", "execCode", "timeoutSeconds", "status", "order", "description", "output", "createdAt", "updatedAt" FROM "scenario_run_steps"`,
);
await queryRunner.query(`DROP TABLE "scenario_run_steps"`);
await queryRunner.query(
`ALTER TABLE "temporary_scenario_run_steps" RENAME TO "scenario_run_steps"`,
);
await queryRunner.query(
`CREATE TABLE "temporary_scenario_runs" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "environmentId" varchar, "sessionId" varchar, "scenarioVersion" integer, "status" text NOT NULL DEFAULT ('pending'), "saveSession" boolean NOT NULL DEFAULT (0), "traceMode" text, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "FK_1d2c2595dfff133de062f9c692a" FOREIGN KEY ("scenarioId") REFERENCES "scenarios" ("id") ON DELETE CASCADE ON UPDATE NO ACTION, CONSTRAINT "FK_5076ffbd1b4fa71f5de011eba1f" FOREIGN KEY ("environmentId") REFERENCES "environments" ("id") ON DELETE SET NULL ON UPDATE NO ACTION, CONSTRAINT "FK_068b01cdfa49095433f245d6708" FOREIGN KEY ("sessionId") REFERENCES "sessions" ("id") ON DELETE SET NULL ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_scenario_runs"("id", "scenarioId", "environmentId", "sessionId", "scenarioVersion", "status", "saveSession", "traceMode", "createdAt", "updatedAt") SELECT "id", "scenarioId", "environmentId", "sessionId", "scenarioVersion", "status", "saveSession", "traceMode", "createdAt", "updatedAt" FROM "scenario_runs"`,
);
await queryRunner.query(`DROP TABLE "scenario_runs"`);
await queryRunner.query(
`ALTER TABLE "temporary_scenario_runs" RENAME TO "scenario_runs"`,
);
await queryRunner.query(
`CREATE TABLE "temporary_scenario_run_files" ("id" varchar PRIMARY KEY NOT NULL, "runId" varchar NOT NULL, "fileId" varchar NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "FK_5fc35a4bfc42d7566a0f7706902" FOREIGN KEY ("runId") REFERENCES "scenario_runs" ("id") ON DELETE CASCADE ON UPDATE NO ACTION, CONSTRAINT "FK_637619025bdb0b6a89fae6e178d" FOREIGN KEY ("fileId") REFERENCES "files" ("id") ON DELETE CASCADE ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_scenario_run_files"("id", "runId", "fileId", "createdAt") SELECT "id", "runId", "fileId", "createdAt" FROM "scenario_run_files"`,
);
await queryRunner.query(`DROP TABLE "scenario_run_files"`);
await queryRunner.query(
`ALTER TABLE "temporary_scenario_run_files" RENAME TO "scenario_run_files"`,
);
await queryRunner.query(
`CREATE TABLE "temporary_scenario_run_logs" ("id" varchar PRIMARY KEY NOT NULL, "runId" varchar NOT NULL, "stepRunId" varchar, "level" text NOT NULL DEFAULT ('log'), "message" text NOT NULL, "seq" integer NOT NULL DEFAULT (0), "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "FK_dde48478a42823e583a360dd468" FOREIGN KEY ("runId") REFERENCES "scenario_runs" ("id") ON DELETE CASCADE ON UPDATE NO ACTION, CONSTRAINT "FK_62d82b9fce3a6a2947ed1136c67" FOREIGN KEY ("stepRunId") REFERENCES "scenario_run_steps" ("id") ON DELETE SET NULL ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_scenario_run_logs"("id", "runId", "stepRunId", "level", "message", "seq", "createdAt") SELECT "id", "runId", "stepRunId", "level", "message", "seq", "createdAt" FROM "scenario_run_logs"`,
);
await queryRunner.query(`DROP TABLE "scenario_run_logs"`);
await queryRunner.query(
`ALTER TABLE "temporary_scenario_run_logs" RENAME TO "scenario_run_logs"`,
);
await queryRunner.query(
`CREATE TABLE "temporary_scenario_versions" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "version" integer NOT NULL, "name" varchar NOT NULL, "description" text, "environmentId" text, "timeoutSeconds" integer, "steps" text NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_e1a56dce2b7e0379bb5bbdf8c1f" UNIQUE ("scenarioId", "version"), CONSTRAINT "FK_f5ccfb45c7847cfab74cf4f17b6" FOREIGN KEY ("scenarioId") REFERENCES "scenarios" ("id") ON DELETE CASCADE ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_scenario_versions"("id", "scenarioId", "version", "name", "description", "environmentId", "timeoutSeconds", "steps", "createdAt") SELECT "id", "scenarioId", "version", "name", "description", "environmentId", "timeoutSeconds", "steps", "createdAt" FROM "scenario_versions"`,
);
await queryRunner.query(`DROP TABLE "scenario_versions"`);
await queryRunner.query(
`ALTER TABLE "temporary_scenario_versions" RENAME TO "scenario_versions"`,
);
await queryRunner.query(
`CREATE TABLE "temporary_snippet_versions" ("id" varchar PRIMARY KEY NOT NULL, "snippetId" varchar NOT NULL, "version" integer NOT NULL, "alias" varchar NOT NULL, "title" varchar NOT NULL DEFAULT (''), "description" text, "code" text NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_2487f4aabeb20c4192ee30e09a2" UNIQUE ("snippetId", "version"), CONSTRAINT "FK_0d3a004957c9553619be11635fd" FOREIGN KEY ("snippetId") REFERENCES "snippets" ("id") ON DELETE CASCADE ON UPDATE NO ACTION)`,
);
await queryRunner.query(
`INSERT INTO "temporary_snippet_versions"("id", "snippetId", "version", "alias", "title", "description", "code", "createdAt") SELECT "id", "snippetId", "version", "alias", "title", "description", "code", "createdAt" FROM "snippet_versions"`,
);
await queryRunner.query(`DROP TABLE "snippet_versions"`);
await queryRunner.query(
`ALTER TABLE "temporary_snippet_versions" RENAME TO "snippet_versions"`,
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "snippet_versions" RENAME TO "temporary_snippet_versions"`,
);
await queryRunner.query(
`CREATE TABLE "snippet_versions" ("id" varchar PRIMARY KEY NOT NULL, "snippetId" varchar NOT NULL, "version" integer NOT NULL, "alias" varchar NOT NULL, "title" varchar NOT NULL DEFAULT (''), "description" text, "code" text NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_2487f4aabeb20c4192ee30e09a2" UNIQUE ("snippetId", "version"))`,
);
await queryRunner.query(
`INSERT INTO "snippet_versions"("id", "snippetId", "version", "alias", "title", "description", "code", "createdAt") SELECT "id", "snippetId", "version", "alias", "title", "description", "code", "createdAt" FROM "temporary_snippet_versions"`,
);
await queryRunner.query(`DROP TABLE "temporary_snippet_versions"`);
await queryRunner.query(
`ALTER TABLE "scenario_versions" RENAME TO "temporary_scenario_versions"`,
);
await queryRunner.query(
`CREATE TABLE "scenario_versions" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "version" integer NOT NULL, "name" varchar NOT NULL, "description" text, "environmentId" text, "timeoutSeconds" integer, "steps" text NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_e1a56dce2b7e0379bb5bbdf8c1f" UNIQUE ("scenarioId", "version"))`,
);
await queryRunner.query(
`INSERT INTO "scenario_versions"("id", "scenarioId", "version", "name", "description", "environmentId", "timeoutSeconds", "steps", "createdAt") SELECT "id", "scenarioId", "version", "name", "description", "environmentId", "timeoutSeconds", "steps", "createdAt" FROM "temporary_scenario_versions"`,
);
await queryRunner.query(`DROP TABLE "temporary_scenario_versions"`);
await queryRunner.query(
`ALTER TABLE "scenario_run_logs" RENAME TO "temporary_scenario_run_logs"`,
);
await queryRunner.query(
`CREATE TABLE "scenario_run_logs" ("id" varchar PRIMARY KEY NOT NULL, "runId" varchar NOT NULL, "stepRunId" varchar, "level" text NOT NULL DEFAULT ('log'), "message" text NOT NULL, "seq" integer NOT NULL DEFAULT (0), "createdAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`INSERT INTO "scenario_run_logs"("id", "runId", "stepRunId", "level", "message", "seq", "createdAt") SELECT "id", "runId", "stepRunId", "level", "message", "seq", "createdAt" FROM "temporary_scenario_run_logs"`,
);
await queryRunner.query(`DROP TABLE "temporary_scenario_run_logs"`);
await queryRunner.query(
`ALTER TABLE "scenario_run_files" RENAME TO "temporary_scenario_run_files"`,
);
await queryRunner.query(
`CREATE TABLE "scenario_run_files" ("id" varchar PRIMARY KEY NOT NULL, "runId" varchar NOT NULL, "fileId" varchar NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`INSERT INTO "scenario_run_files"("id", "runId", "fileId", "createdAt") SELECT "id", "runId", "fileId", "createdAt" FROM "temporary_scenario_run_files"`,
);
await queryRunner.query(`DROP TABLE "temporary_scenario_run_files"`);
await queryRunner.query(
`ALTER TABLE "scenario_runs" RENAME TO "temporary_scenario_runs"`,
);
await queryRunner.query(
`CREATE TABLE "scenario_runs" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "environmentId" varchar, "sessionId" varchar, "scenarioVersion" integer, "status" text NOT NULL DEFAULT ('pending'), "saveSession" boolean NOT NULL DEFAULT (0), "traceMode" text, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`INSERT INTO "scenario_runs"("id", "scenarioId", "environmentId", "sessionId", "scenarioVersion", "status", "saveSession", "traceMode", "createdAt", "updatedAt") SELECT "id", "scenarioId", "environmentId", "sessionId", "scenarioVersion", "status", "saveSession", "traceMode", "createdAt", "updatedAt" FROM "temporary_scenario_runs"`,
);
await queryRunner.query(`DROP TABLE "temporary_scenario_runs"`);
await queryRunner.query(
`ALTER TABLE "scenario_run_steps" RENAME TO "temporary_scenario_run_steps"`,
);
await queryRunner.query(
`CREATE TABLE "scenario_run_steps" ("id" varchar PRIMARY KEY NOT NULL, "runId" varchar NOT NULL, "scenarioStepId" varchar, "title" text, "execCode" text, "timeoutSeconds" integer, "status" text NOT NULL DEFAULT ('waiting'), "order" integer NOT NULL DEFAULT (0), "description" text, "output" text, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`INSERT INTO "scenario_run_steps"("id", "runId", "scenarioStepId", "title", "execCode", "timeoutSeconds", "status", "order", "description", "output", "createdAt", "updatedAt") SELECT "id", "runId", "scenarioStepId", "title", "execCode", "timeoutSeconds", "status", "order", "description", "output", "createdAt", "updatedAt" FROM "temporary_scenario_run_steps"`,
);
await queryRunner.query(`DROP TABLE "temporary_scenario_run_steps"`);
await queryRunner.query(
`ALTER TABLE "scenario_files" RENAME TO "temporary_scenario_files"`,
);
await queryRunner.query(
`CREATE TABLE "scenario_files" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "fileId" varchar NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`INSERT INTO "scenario_files"("id", "scenarioId", "fileId", "createdAt") SELECT "id", "scenarioId", "fileId", "createdAt" FROM "temporary_scenario_files"`,
);
await queryRunner.query(`DROP TABLE "temporary_scenario_files"`);
await queryRunner.query(
`ALTER TABLE "scenarios" RENAME TO "temporary_scenarios"`,
);
await queryRunner.query(
`CREATE TABLE "scenarios" ("id" varchar PRIMARY KEY NOT NULL, "name" varchar NOT NULL, "description" text, "environmentId" varchar, "timeoutSeconds" integer, "version" integer NOT NULL DEFAULT (1), "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`INSERT INTO "scenarios"("id", "name", "description", "environmentId", "timeoutSeconds", "version", "createdAt", "updatedAt") SELECT "id", "name", "description", "environmentId", "timeoutSeconds", "version", "createdAt", "updatedAt" FROM "temporary_scenarios"`,
);
await queryRunner.query(`DROP TABLE "temporary_scenarios"`);
await queryRunner.query(
`ALTER TABLE "scenario_steps" RENAME TO "temporary_scenario_steps"`,
);
await queryRunner.query(
`CREATE TABLE "scenario_steps" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "order" integer NOT NULL DEFAULT (0), "title" text, "execCode" text, "timeoutSeconds" integer, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), "updatedAt" datetime NOT NULL DEFAULT (datetime('now')))`,
);
await queryRunner.query(
`INSERT INTO "scenario_steps"("id", "scenarioId", "order", "title", "execCode", "timeoutSeconds", "createdAt", "updatedAt") SELECT "id", "scenarioId", "order", "title", "execCode", "timeoutSeconds", "createdAt", "updatedAt" FROM "temporary_scenario_steps"`,
);
await queryRunner.query(`DROP TABLE "temporary_scenario_steps"`);
await queryRunner.query(
`ALTER TABLE "scenario_credentials" RENAME TO "temporary_scenario_credentials"`,
);
await queryRunner.query(
`CREATE TABLE "scenario_credentials" ("id" varchar PRIMARY KEY NOT NULL, "scenarioId" varchar NOT NULL, "credentialId" varchar NOT NULL, "alias" varchar NOT NULL, "createdAt" datetime NOT NULL DEFAULT (datetime('now')), CONSTRAINT "UQ_02756536236c408dbb58b3fc964" UNIQUE ("scenarioId", "alias"))`,
);
await queryRunner.query(
`INSERT INTO "scenario_credentials"("id", "scenarioId", "credentialId", "alias", "createdAt") SELECT "id", "scenarioId", "credentialId", "alias", "createdAt" FROM "temporary_scenario_credentials"`,
);
await queryRunner.query(`DROP TABLE "temporary_scenario_credentials"`);
await queryRunner.query(`DROP TABLE "snippet_versions"`);
await queryRunner.query(`DROP TABLE "snippets"`);
await queryRunner.query(`DROP TABLE "scenario_versions"`);
await queryRunner.query(`DROP TABLE "scenario_run_logs"`);
await queryRunner.query(`DROP TABLE "scenario_run_files"`);
await queryRunner.query(`DROP TABLE "scenario_runs"`);
await queryRunner.query(`DROP TABLE "scenario_run_steps"`);
await queryRunner.query(`DROP TABLE "sessions"`);
await queryRunner.query(`DROP TABLE "scenario_files"`);
await queryRunner.query(`DROP TABLE "scenarios"`);
await queryRunner.query(`DROP TABLE "scenario_steps"`);
await queryRunner.query(`DROP TABLE "scenario_credentials"`);
await queryRunner.query(`DROP TABLE "files"`);
await queryRunner.query(`DROP TABLE "environments"`);
await queryRunner.query(`DROP TABLE "credentials"`);
}
}
+7
View File
@@ -0,0 +1,7 @@
import { Initial1790793447827 } from "./1790793447827-Initial";
/**
* All migrations, oldest first. Listed explicitly (instead of a glob) so the
* same list works from compiled JS, ts-jest and the TypeORM CLI.
*/
export const MIGRATIONS = [Initial1790793447827];
+19 -1
View File
@@ -1,6 +1,7 @@
import { RequestMethod, ValidationPipe } from "@nestjs/common";
import { ConfigService } from "@nestjs/config";
import { NestFactory } from "@nestjs/core";
import { DataSource } from "typeorm";
import type { NestExpressApplication } from "@nestjs/platform-express";
import { DocumentBuilder, SwaggerModule } from "@nestjs/swagger";
import { AppModule } from "./app.module";
@@ -36,6 +37,15 @@ async function bootstrap() {
logger: new TraceLogger("Bootstrap", { timestamp: true }),
});
// The schema is managed by migrations; refuse to run against an outdated one.
// Checked before app.init(), i.e. before any module touches the tables.
if (await app.get(DataSource).showMigrations()) {
throw new Error(
"The database has pending migrations. Run `npm -w server run migration:run` " +
"(the Docker image runs them automatically before starting the server).",
);
}
app.setGlobalPrefix(API_PREFIX, {
exclude: [
{ path: "health", method: RequestMethod.ALL },
@@ -77,4 +87,12 @@ async function bootstrap() {
logger.log(`MCP endpoint available at http://localhost:${port}/mcp`);
}
bootstrap();
// Startup failures (pending migrations, port in use, ...) must stop the
// process with an error code; the handlers above only keep a running server alive
bootstrap().catch((err: unknown) => {
processLogger.error(
`Startup failed: ${err instanceof Error ? err.message : String(err)}`,
err instanceof Error ? err.stack : undefined,
);
process.exit(1);
});
+4 -35
View File
@@ -9,6 +9,7 @@ import type { NestExpressApplication } from "@nestjs/platform-express";
import { ScheduleModule } from "@nestjs/schedule";
import { configureBodyParsers } from "../src/common/body-parsers";
import { dataSourceOptions } from "../src/database/data-source";
import { ImportModule } from "../src/import/import.module";
jest.mock("@nestjs/common", () => {
@@ -42,25 +43,10 @@ import { SessionModule } from "../src/session/session.module";
import { EnvironmentModule } from "../src/environment/environment.module";
import { ScenarioModule } from "../src/scenario/scenario.module";
import { McpModule } from "../src/mcp/mcp.module";
import { SessionEntity } from "../src/session/session.entity";
import { EnvironmentEntity } from "../src/environment/environment.entity";
import { ScenarioEntity } from "../src/scenario/scenario.entity";
import { ScenarioStepEntity } from "../src/scenario/scenario-step.entity";
import { ScenarioVersionEntity } from "../src/scenario/scenario-version.entity";
import { SnippetVersionEntity } from "../src/snippet/snippet-version.entity";
import { ScenarioRunEntity } from "../src/scenario/scenario-run.entity";
import { ScenarioRunStepEntity } from "../src/scenario/scenario-run-step.entity";
import { ScenarioRunLogEntity } from "../src/scenario/scenario-run-log.entity";
import { ScenarioCredentialEntity } from "../src/scenario/scenario-credential.entity";
import { CredentialEntity } from "../src/credential/credential.entity";
import { SnippetEntity } from "../src/snippet/snippet.entity";
import { HealthController } from "../src/health/health.controller";
import { HttpExceptionFilter } from "../src/filters/http-exception.filter";
import { LoggingInterceptor } from "../src/interceptors/logging.interceptor";
import { validateAppConfig } from "../src/config/app.config";
import { FileEntity } from "../src/file/file.entity";
import { ScenarioFileEntity } from "../src/file/scenario-file.entity";
import { ScenarioRunFileEntity } from "../src/file/scenario-run-file.entity";
export async function buildTestApp(): Promise<INestApplication> {
// Each test app stores files in its own temp dir: suites run in parallel and
@@ -75,27 +61,10 @@ export async function buildTestApp(): Promise<INestApplication> {
ignoreEnvFile: true,
validate: validateAppConfig,
}),
// The real migrations, not synchronize, so tests use the schema production gets
TypeOrmModule.forRoot({
type: "better-sqlite3",
database: ":memory:",
entities: [
SessionEntity,
EnvironmentEntity,
ScenarioEntity,
ScenarioStepEntity,
ScenarioVersionEntity,
ScenarioRunEntity,
ScenarioRunStepEntity,
ScenarioRunLogEntity,
ScenarioCredentialEntity,
CredentialEntity,
SnippetEntity,
SnippetVersionEntity,
FileEntity,
ScenarioFileEntity,
ScenarioRunFileEntity,
],
synchronize: true,
...dataSourceOptions(":memory:"),
migrationsRun: true,
}),
ScheduleModule.forRoot(),
BrowserModule,
+65
View File
@@ -0,0 +1,65 @@
/**
* The schema is managed by migrations. These tests fail when an entity
* changes without a matching migration, and cover adopting databases that
* were created by `synchronize` before migrations existed.
*/
import { DataSource } from "typeorm";
import { dataSourceOptions } from "../src/database/data-source";
import { MIGRATIONS } from "../src/database/migrations";
async function pendingSchemaChanges(dataSource: DataSource): Promise<string[]> {
const sql = await dataSource.driver.createSchemaBuilder().log();
return sql.upQueries.map((q) => q.query);
}
describe("Database migrations", () => {
it("migrations produce exactly the schema the entities describe", async () => {
const dataSource = new DataSource(dataSourceOptions(":memory:"));
await dataSource.initialize();
try {
await dataSource.runMigrations();
expect(await dataSource.showMigrations()).toBe(false);
// If this fails, generate a migration: see "Database migrations" in docs/development.md
expect(await pendingSchemaChanges(dataSource)).toEqual([]);
} finally {
await dataSource.destroy();
}
});
it("adopts a database created by synchronize", async () => {
const dataSource = new DataSource({
...dataSourceOptions(":memory:"),
synchronize: true,
});
await dataSource.initialize();
try {
await dataSource.query(
`INSERT INTO scenarios (id, name, version) VALUES ('11111111-1111-4111-8111-111111111111', 'kept', 1)`,
);
const applied = await dataSource.runMigrations();
expect(applied.map((m) => m.name)).toEqual(MIGRATIONS.map((m) => m.name));
expect(await dataSource.showMigrations()).toBe(false);
expect(await pendingSchemaChanges(dataSource)).toEqual([]);
const [row] = await dataSource.query(`SELECT name FROM scenarios`);
expect(row.name).toBe("kept");
} finally {
await dataSource.destroy();
}
});
it("can revert the initial migration", async () => {
const dataSource = new DataSource(dataSourceOptions(":memory:"));
await dataSource.initialize();
try {
await dataSource.runMigrations();
await dataSource.undoLastMigration();
const tables = await dataSource.query(
`SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT IN ('migrations', 'sqlite_sequence')`,
);
expect(tables).toEqual([]);
} finally {
await dataSource.destroy();
}
});
});