Compare commits

..
8 Commits
Author SHA1 Message Date
ars9 385f51f3ba fix(scenario-logs): fix out-of-order run log persistence
- Add seq column assigned in call order to fix logs racing each other
  as fire-and-forget writes (e.g. a snippet's "started" entry could be
  saved after its "finished" entry)
- Log snippet start/finish/failure via context.runSnippet, matching
  existing step start/pass/fail logging
- Rework Markdown log export to per-entry sections instead of a table
- Version bump: 1.10.0 -> 1.10.1
2026-09-15 13:17:29 +03:00
ars9 540a7ec96b feat(logs): add scenario run logs export
- Backend: logs export endpoint for CSV and Markdown formats
- Frontend: ExportLogsModal with download button on run detail page
- Updated i18n and CHANGELOG
- Version bump: 1.9.1 -> 1.10.0
2026-09-15 13:04:59 +03:00
ars9 4e79d3cd81 feat(scenario): add automatic step boundary logging to run logs
- step start, pass, and fail events logged automatically
- step boundaries visible in run logs without explicit context.log calls
- version bumped from 1.9.0 to 1.9.1
2026-09-15 12:53:33 +03:00
ars9 cbfd4404c1 feat(scenario): add context.assert() API for script assertions
- Add context.assert(fn, description) for function assertions
- Logs outcome on success/failure, throws on assertion failure
- Bump package version from 1.8.0 to 1.9.0
2026-09-15 12:35:17 +03:00
ars9andClaude Sonnet 5 50820b8b40 feat(scenario): let context.downloadFile save data: URLs as run artifacts
Scenario steps can now persist in-script generated content (e.g. an
obtained VC's JWT) as a run file via context.downloadFile('data:...'),
without needing a network fetch. Bumps to 1.8.0.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 10:37:22 +03:00
ars9 502a3e4f2c 1.7.1 2026-04-21 18:09:49 +03:00
ars9 b57b55f3e1 feat(mcp): add file tools for scenarios and runs
- expose scenario file upload, listing, and content retrieval over MCP\n- expose run artifact listing and content retrieval with end-to-end coverage
2026-04-21 18:00:41 +03:00
ars9 06c662042b doc(changelog): add changelog 2026-04-21 17:09:05 +03:00
19 changed files with 1164 additions and 12 deletions

No files matched your search

+93
View File
@@ -0,0 +1,93 @@
# Changelog
All notable changes to this project will be documented in this file.
## 1.10.1 - 2026-09-15
Changes since 1.10.0:
### Added
- `context.runSnippet()` now logs "Snippet started"/"finished"/"failed" entries, matching the automatic step start/pass/fail logging.
### Changed
- Reworked the Markdown log export format: each log entry is now its own `## [level] timestamp` section with the message as its body, instead of a single large table.
### Fixed
- Run logs could persist out of call order (e.g. a snippet's "started" log appearing after its "finished" log) because writes were fire-and-forget and raced against each other. Log entries now carry a per-run sequence number assigned in call order, and queries/exports sort by it.
## 1.10.0 - 2026-09-15
Changes since 1.9.1:
### Added
- Added a "Download logs" button on the scenario run detail page: opens a modal to choose Markdown (default) or CSV, then downloads the run's logs in that format.
- Added `GET /scenarios/:id/run/:runId/logs/export?format=csv|md` backend endpoint to export a run's logs.
## 1.9.1 - 2026-09-15
Changes since 1.9.0:
### Added
- Scenario run logs now include an automatic entry when a step starts and when it finishes (pass or fail), so step boundaries are visible in run logs even for steps that don't call `context.log`/`context.assert` themselves.
## 1.9.0 - 2026-09-15
Changes since 1.8.0:
### Added
- Added `context.assert(fn, description)` for scenario/snippet script steps: runs the given function, requires it to return `true`, logs the outcome (`log` on success, `error` on failure or exception), and throws to stop the step on failure.
### Changed
- Documented `context.assert` in `docs/scenario.md`.
## 1.8.0 - 2026-09-15
Changes since 1.7.1:
### Added
- `context.downloadFile()` now accepts `data:` URIs, so scenario steps can save in-script generated content (e.g. a credential JWT) as a run artifact without a network fetch.
### Changed
- Documented the full scenario step `context` file API (`getScenarioFiles`, `downloadFile`) in `docs/scenario.md`.
## 1.7.1 - 2026-04-21
Changes since 1.7.0:
### Added
- Added MCP tools to upload, list, and fetch scenario files using base64 content payloads.
- Added MCP tools to list run artifacts and fetch run artifact content with metadata.
### Changed
- Documented MCP binary content handling for scenario files and run artifacts.
## 1.7.0 - 2026-04-21
Changes since 1.6.1:
### Added
- Added scenario file storage and run artifact support on the backend.
- Added file management UI for scenarios and run details, including artifact lists.
- Added snippet CRUD tools to the MCP server.
### Changed
- Added dynamic page titles based on the active client content.
- Extended the scenarios list to show the latest run status and timestamp.
### Fixed
- Exposed file expiry metadata in scenario and run file responses.
- Disabled expired download actions in the UI while keeping the action visible.
+3
View File
@@ -292,6 +292,9 @@ export const runs = {
const qs = q?.trim() ? `?q=${encodeURIComponent(q.trim())}` : '';
return request(`/scenarios/${scenarioId}/run/${runId}${qs}`);
},
exportLogs(scenarioId: string, runId: string, format: 'csv' | 'md'): Promise<string> {
return request(`/scenarios/${scenarioId}/run/${runId}/logs/export?format=${format}`);
},
};
// ── Scenario Credentials ──────────────────────────────────────────────────────
@@ -0,0 +1,27 @@
.section {
display: flex;
flex-direction: column;
gap: 0.5rem;
}
.sectionTitle {
font-size: 0.8125rem;
font-weight: 600;
color: var(--text-muted, #888);
margin: 0;
text-transform: uppercase;
letter-spacing: 0.04em;
}
.radioRow {
display: flex;
align-items: center;
gap: 0.5rem;
font-size: 0.875rem;
cursor: pointer;
}
.radioRow input[type='radio'] {
cursor: pointer;
accent-color: var(--color-primary);
}
@@ -0,0 +1,68 @@
import { useTranslation } from 'react-i18next';
import { X, Download } from 'lucide-react';
import { Modal, Button } from '../../ui';
import styles from './ExportLogsModal.module.css';
export type LogExportFormat = 'md' | 'csv';
export interface ExportLogsModalProps {
open: boolean;
format: LogExportFormat;
onFormatChange: (format: LogExportFormat) => void;
onClose: () => void;
onExport: () => void;
isExporting: boolean;
}
export function ExportLogsModal({
open,
format,
onFormatChange,
onClose,
onExport,
isExporting,
}: ExportLogsModalProps) {
const { t } = useTranslation();
return (
<Modal
open={open}
title={t('runs.export_logs_modal_title')}
onClose={() => !isExporting && onClose()}
footer={
<>
<Button variant="secondary" onClick={onClose} disabled={isExporting}>
<X size={14} />
{t('common.button_cancel')}
</Button>
<Button variant="primary" onClick={onExport} disabled={isExporting}>
<Download size={14} />
{t('runs.action_export')}
</Button>
</>
}
>
<div className={styles.section}>
<h4 className={styles.sectionTitle}>{t('runs.export_logs_format')}</h4>
<label className={styles.radioRow}>
<input
type="radio"
name="log-export-format"
checked={format === 'md'}
onChange={() => onFormatChange('md')}
/>
<span>{t('runs.export_logs_format_md')}</span>
</label>
<label className={styles.radioRow}>
<input
type="radio"
name="log-export-format"
checked={format === 'csv'}
onChange={() => onFormatChange('csv')}
/>
<span>{t('runs.export_logs_format_csv')}</span>
</label>
</div>
</Modal>
);
}
+2
View File
@@ -4,3 +4,5 @@ export { ExportScenarioModal } from './ExportScenarioModal';
export type { ExportScenarioModalProps } from './ExportScenarioModal';
export { RunScenarioModal } from './RunScenarioModal';
export type { RunScenarioModalProps } from './RunScenarioModal';
export { ExportLogsModal } from './ExportLogsModal';
export type { ExportLogsModalProps, LogExportFormat } from './ExportLogsModal';
+7 -1
View File
@@ -257,7 +257,13 @@
"step_description": "Result",
"log_level": "Level",
"log_message": "Message",
"log_time": "Time"
"log_time": "Time",
"action_download_logs": "Download logs",
"export_logs_modal_title": "Export logs",
"export_logs_format": "Format",
"export_logs_format_md": "Markdown (.md)",
"export_logs_format_csv": "CSV (.csv)",
"action_export": "Export"
},
"snippets": {
"title": "Snippets",
+45 -5
View File
@@ -14,6 +14,7 @@ import type {
FileMetadata,
} from '../../api';
import { scenarios } from '../../api';
import { ExportLogsModal, type LogExportFormat } from '../../components/modals';
import {
AutoRefreshIndicator,
Badge,
@@ -84,6 +85,9 @@ export function RunDetailPage() {
const [artifacts, setArtifacts] = useState<FileMetadata[]>([]);
const [artifactsTotal, setArtifactsTotal] = useState(0);
const [artifactsLoading, setArtifactsLoading] = useState(false);
const [exportLogsOpen, setExportLogsOpen] = useState(false);
const [exportLogsFormat, setExportLogsFormat] = useState<LogExportFormat>('md');
const [exportingLogs, setExportingLogs] = useState(false);
const logSearchRef = useRef('');
const LOG_PAGE_SIZE = 25;
const pollRef = useRef<ReturnType<typeof setInterval> | null>(null);
@@ -106,6 +110,27 @@ export function RunDetailPage() {
const isFileExpired = (file: FileMetadata) =>
file.expiresAt != null && new Date(file.expiresAt).getTime() <= Date.now();
const handleExportLogs = async () => {
if (!id || !runId) return;
setExportingLogs(true);
try {
const content = await runs.exportLogs(id, runId, exportLogsFormat);
const mimeType = exportLogsFormat === 'csv' ? 'text/csv' : 'text/markdown';
const blob = new Blob([content], { type: mimeType });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = `run-${runId}-logs.${exportLogsFormat}`;
a.click();
URL.revokeObjectURL(url);
setExportLogsOpen(false);
} catch {
// error toast is already emitted by the API client
} finally {
setExportingLogs(false);
}
};
const manualRefresh = () => {
if (!id || !runId) return;
runs
@@ -436,11 +461,17 @@ export function RunDetailPage() {
<div className={styles.stepsSection}>
<div className={styles.sectionHeadingRow}>
<h2 className={styles.sectionHeading}>{t('runs.logs_heading')}</h2>
<Search
value={logSearch}
onChange={setLogSearch}
placeholder={t('runs.logs_search_placeholder')}
/>
<div style={{ display: 'flex', alignItems: 'center', gap: '8px' }}>
<Search
value={logSearch}
onChange={setLogSearch}
placeholder={t('runs.logs_search_placeholder')}
/>
<Button variant="secondary" size="sm" onClick={() => setExportLogsOpen(true)}>
<Download size={14} />
{t('runs.action_download_logs')}
</Button>
</div>
</div>
<Table
columns={logColumns}
@@ -523,6 +554,15 @@ export function RunDetailPage() {
)}
</>
)}
<ExportLogsModal
open={exportLogsOpen}
format={exportLogsFormat}
onFormatChange={setExportLogsFormat}
onClose={() => setExportLogsOpen(false)}
onExport={handleExportLogs}
isExporting={exportingLogs}
/>
</div>
);
}
+22
View File
@@ -50,7 +50,29 @@ The server currently registers the following tools.
| `export_scenario` | Export scenario payload |
| `import_scenario` | Import scenario payload |
## Scenario Files
| Tool | Description |
|---|---|
| `upload_scenario_file` | Upload a file to a scenario; accepts optional `expiresAt` (ISO 8601 date string) |
| `list_scenario_files` | List scenario files (paginated) |
| `get_scenario_file_content` | Retrieve file content as base64 with metadata |
## Run Artifact Files
| Tool | Description |
|---|---|
| `list_run_files` | List files (artifacts) created during a scenario run (paginated) |
| `get_run_file_content` | Retrieve run artifact content as base64 with metadata; run artifacts are read-only and inherit expiry from the run artifact subsystem |
## Binary Content Handling
File tools transport binary payloads as base64-encoded strings in a `contentBase64` field with an accompanying `encoding: "base64"` marker in the response. When retrieving file content via `get_scenario_file_content` or `get_run_file_content`, decode the base64 to recover the original bytes.
Scenario files uploaded via `upload_scenario_file` must have their content pre-encoded as base64. Run artifacts are created implicitly through scenario execution (via `context.downloadFile()` during step execution) and cannot be uploaded via MCP.
## Notes
- Tool IDs and entity IDs are UUIDs.
- `create_scenario_step` schema still accepts legacy `type`/`sessionName` fields for compatibility. Current scheduler executes `execCode` plus optional `validateCode` and does not branch by step type.
+3
View File
@@ -35,7 +35,10 @@ Available in scope:
- `context.getStepOutput(order)`: prior step output by absolute order (`1`, `2`, ...) or relative (`-1` = previous step)
- `context.dumpDom(selector?)`: simplified DOM snapshot
- `context.log(...args)`, `context.warn(...args)`, `context.error(...args)`: structured step logs
- `context.assert(() => boolean, description)`: runs the arrow function and requires it to return `true`. Logs `description` (as a `log` entry on success, as an `error` entry on failure or exception), then throws to stop the step if the function returned anything other than `true` or threw
- `context.runSnippet(name, ...args)`: execute stored snippet code with the same context
- `context.getScenarioFiles(opts?)`: list files uploaded to the scenario (`limit`, `offset`)
- `context.downloadFile(url, opts?)`: fetch `url` and save the result as a run artifact (requires a real scenario run — throws when invoked ad hoc, e.g. via the `exec_code` MCP tool). `opts` may include `method`, `headers`, `body`, `filename`. `url` also accepts a `data:` URI (`data:<mediaType>;base64,<data>` or `data:<mediaType>,<percent-encoded data>`) to save content generated in-script (e.g. a credential JWT) directly, without an actual network fetch.
- `console`: proxied to run logs (`log`, `warn`, `error`, etc.)
`validateCode` additionally receives `result` in scope, which is the value returned by `execCode`.
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "liqa",
"version": "1.7.0",
"version": "1.7.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "liqa",
"version": "1.7.0",
"version": "1.7.1",
"workspaces": [
"server",
"client"
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "liqa",
"version": "1.7.0",
"version": "1.10.1",
"private": true,
"workspaces": [
"server",
@@ -42,6 +42,15 @@ export interface ScriptContext {
log: (...args: unknown[]) => void;
warn: (...args: unknown[]) => void;
error: (...args: unknown[]) => void;
/**
* Runs `fn` and requires it to return `true`. Logs the outcome under
* `description`, then throws if `fn` returned anything other than `true`
* or threw an exception.
*/
assert: (
fn: () => boolean | Promise<boolean>,
description: string,
) => Promise<void>;
/** Runs a named snippet with the same context, plus any extra positional args. */
runSnippet: (alias: string, ...args: unknown[]) => Promise<unknown>;
/** Returns metadata + absolute disk path for files attached to the current scenario. */
@@ -141,6 +150,28 @@ export class CodeExecutorService {
log: (...args: unknown[]) => scriptLog("log", toStr(args)),
warn: (...args: unknown[]) => scriptLog("warn", toStr(args)),
error: (...args: unknown[]) => scriptLog("error", toStr(args)),
assert: async (
fn: () => boolean | Promise<boolean>,
description: string,
): Promise<void> => {
let passed: boolean;
let failureReason: string | undefined;
try {
passed = (await fn()) === true;
} catch (err) {
passed = false;
failureReason = (err as Error).message;
}
if (passed) {
scriptLog("log", `Assertion passed: ${description}`);
return;
}
const message = failureReason
? `Assertion failed: ${description} (${failureReason})`
: `Assertion failed: ${description}`;
scriptLog("error", message);
throw new Error(message);
},
getStepOutput: getStepOutput ?? (() => Promise.resolve(null)),
getCredential: (alias: string): unknown => {
if (!(alias in credMap)) {
@@ -168,6 +199,7 @@ export class CodeExecutorService {
if (snippetCode == null) {
throw new Error(`Snippet alias "${alias}" not found`);
}
scriptLog("log", `Snippet "${alias}" started`);
const snippetFn = new Function(
"context",
"console",
@@ -175,7 +207,22 @@ export class CodeExecutorService {
"expect",
`return (async (context, ...args) => { ${snippetCode} })(context, ...snippetArgs)`,
);
return snippetFn(scriptContext, fakeConsole, args, playwrightExpect);
try {
const snippetResult = await snippetFn(
scriptContext,
fakeConsole,
args,
playwrightExpect,
);
scriptLog("log", `Snippet "${alias}" finished`);
return snippetResult;
} catch (err) {
scriptLog(
"error",
`Snippet "${alias}" failed: ${(err as Error).message}`,
);
throw err;
}
},
getScenarioFiles: async (opts?: { limit?: number; offset?: number }) => {
if (!fileService || !scenarioId) {
+16
View File
@@ -5,6 +5,18 @@ import { URL } from "url";
const MAX_FILE_SIZE = 100 * 1024 * 1024; // 100 MB
const TIMEOUT_MS = 30000; // 30 seconds
function decodeDataUrl(urlStr: string): Buffer {
const match = /^data:([^,]*),(.*)$/s.exec(urlStr);
if (!match) {
throw new Error("Malformed data: URL");
}
const [, meta, data] = match;
if (meta.endsWith(";base64")) {
return Buffer.from(data, "base64");
}
return Buffer.from(decodeURIComponent(data), "utf-8");
}
export async function downloadFile(
urlStr: string,
options?: {
@@ -13,6 +25,10 @@ export async function downloadFile(
body?: string;
},
): Promise<Buffer> {
if (urlStr.startsWith("data:")) {
return decodeDataUrl(urlStr);
}
const url = new URL(urlStr);
const protocol = url.protocol === "https:" ? https : http;
const method = options?.method ?? "GET";
+256
View File
@@ -1062,6 +1062,262 @@ export class McpService {
},
);
// ── Scenario Files ────────────────────────────────────────────────────────
server.registerTool(
"upload_scenario_file",
{
description:
"Upload a file to a scenario. Content must be base64-encoded. Returns file metadata.",
inputSchema: {
scenarioId: z.uuid().describe("Scenario ID"),
name: z.string().describe("File name"),
contentBase64: z
.string()
.describe("Base64-encoded file content"),
mimeType: z
.string()
.optional()
.describe("MIME type (e.g., text/plain, application/json)"),
expiresAt: z
.string()
.optional()
.describe("ISO 8601 expiration date"),
},
},
async ({ scenarioId, name, contentBase64, mimeType, expiresAt }) => {
try {
// Validate base64 format
if (!/^[A-Za-z0-9+/]*={0,2}$/.test(contentBase64)) {
return {
isError: true,
content: [
{
type: "text" as const,
text: "Invalid base64 encoding",
},
],
};
}
// Decode base64 to buffer
let buffer: Buffer;
try {
buffer = Buffer.from(contentBase64, "base64");
// Verify the base64 can be re-encoded to match the original
if (buffer.toString("base64") !== contentBase64) {
return {
isError: true,
content: [
{
type: "text" as const,
text: "Invalid base64 encoding",
},
],
};
}
} catch {
return {
isError: true,
content: [
{
type: "text" as const,
text: "Invalid base64 encoding",
},
],
};
}
// Create a mock Multer file object for the service
const file = {
buffer,
originalname: name,
mimetype: mimeType || "application/octet-stream",
} as any;
// Call the service method
const result = await this.scenarioService.uploadFile(
scenarioId,
file,
expiresAt,
);
return {
content: [{ type: "text" as const, text: JSON.stringify(result) }],
};
} catch (err) {
return {
isError: true,
content: [{ type: "text" as const, text: (err as Error).message }],
};
}
},
);
server.registerTool(
"list_scenario_files",
{
description:
"List files uploaded to a scenario (paginated)",
inputSchema: {
scenarioId: z.uuid().describe("Scenario ID"),
limit: z
.number()
.int()
.min(1)
.optional()
.describe("Items per page (default 20)"),
offset: z
.number()
.int()
.min(0)
.optional()
.describe("Skip count (default 0)"),
},
},
async ({ scenarioId, limit, offset }) => {
try {
const result = await this.scenarioService.listScenarioFiles(
scenarioId,
limit,
offset,
);
return {
content: [{ type: "text" as const, text: JSON.stringify(result) }],
};
} catch (err) {
return {
isError: true,
content: [{ type: "text" as const, text: (err as Error).message }],
};
}
},
);
server.registerTool(
"get_scenario_file_content",
{
description:
"Get a scenario file's content as base64 with metadata",
inputSchema: {
scenarioId: z.uuid().describe("Scenario ID"),
fileId: z.uuid().describe("File ID"),
},
},
async ({ scenarioId, fileId }) => {
try {
const { file, contentBuffer } =
await this.scenarioService.getScenarioFileContentAsBuffer(
scenarioId,
fileId,
);
const contentBase64 = contentBuffer.toString("base64");
return {
content: [
{
type: "text" as const,
text: JSON.stringify({
file,
contentBase64,
encoding: "base64",
}),
},
],
};
} catch (err) {
return {
isError: true,
content: [{ type: "text" as const, text: (err as Error).message }],
};
}
},
);
// ── Run Artifact Files ────────────────────────────────────────────────────
server.registerTool(
"list_run_files",
{
description:
"List files (artifacts) created during a scenario run (paginated)",
inputSchema: {
scenarioId: z.uuid().describe("Scenario ID"),
runId: z.uuid().describe("Run ID"),
limit: z
.number()
.int()
.min(1)
.optional()
.describe("Items per page (default 20)"),
offset: z
.number()
.int()
.min(0)
.optional()
.describe("Skip count (default 0)"),
},
},
async ({ scenarioId, runId, limit, offset }) => {
try {
const result = await this.scenarioService.listRunFiles(
scenarioId,
runId,
limit,
offset,
);
return {
content: [{ type: "text" as const, text: JSON.stringify(result) }],
};
} catch (err) {
return {
isError: true,
content: [{ type: "text" as const, text: (err as Error).message }],
};
}
},
);
server.registerTool(
"get_run_file_content",
{
description:
"Get a run artifact file's content as base64 with metadata",
inputSchema: {
scenarioId: z.uuid().describe("Scenario ID"),
runId: z.uuid().describe("Run ID"),
fileId: z.uuid().describe("File ID"),
},
},
async ({ scenarioId, runId, fileId }) => {
try {
const { file, contentBuffer } =
await this.scenarioService.getRunFileContentAsBuffer(
scenarioId,
runId,
fileId,
);
const contentBase64 = contentBuffer.toString("base64");
return {
content: [
{
type: "text" as const,
text: JSON.stringify({
file,
contentBase64,
encoding: "base64",
}),
},
],
};
} catch (err) {
return {
isError: true,
content: [{ type: "text" as const, text: (err as Error).message }],
};
}
},
);
// ── Snippets ──────────────────────────────────────────────────────────────
server.registerTool(
@@ -39,6 +39,11 @@ export class ScenarioRunLogEntity {
@Column({ type: "text" })
message: string;
// Assigned in call order per run, since concurrent unawaited writes can
// otherwise persist out of order (createdAt alone isn't a reliable tiebreaker).
@Column({ type: "int", default: 0 })
seq: number;
@CreateDateColumn()
createdAt: Date;
}
@@ -46,6 +46,9 @@ export class ScenarioSchedulerService implements OnModuleInit {
private readonly runScenarioTimeouts = new Map<string, number | null>();
// Cache scenarioId per run
private readonly runScenarioIds = new Map<string, string>();
// Monotonic log sequence per run, since persistLog writes are unawaited and
// can otherwise resolve out of call order
private readonly runLogSeq = new Map<string, number>();
private static readonly DEFAULT_SCENARIO_TIMEOUT_SEC = 600;
private static readonly DEFAULT_STEP_TIMEOUT_SEC = 60;
@@ -138,8 +141,10 @@ export class ScenarioSchedulerService implements OnModuleInit {
level: "log" | "warn" | "error",
message: string,
): void {
const seq = (this.runLogSeq.get(runId) ?? 0) + 1;
this.runLogSeq.set(runId, seq);
void this.runLogRepo.save(
this.runLogRepo.create({ runId, stepRunId, level, message }),
this.runLogRepo.create({ runId, stepRunId, level, message, seq }),
);
}
@@ -216,6 +221,7 @@ export class ScenarioSchedulerService implements OnModuleInit {
this.runEnvironments.delete(run.id);
this.runScenarioTimeouts.delete(run.id);
this.runScenarioIds.delete(run.id);
this.runLogSeq.delete(run.id);
this.logger.error(
`Run #${run.id}: setup failed — ${String(err)}`,
);
@@ -279,6 +285,7 @@ export class ScenarioSchedulerService implements OnModuleInit {
this.runEnvironments.delete(runId);
this.runScenarioTimeouts.delete(runId);
this.runScenarioIds.delete(runId);
this.runLogSeq.delete(runId);
await this.maybePreserveSession(runId);
}
}
@@ -345,6 +352,12 @@ export class ScenarioSchedulerService implements OnModuleInit {
this.logger.log(
`StepRun #${stepRun.id} (run #${stepRun.runId}, step #${step.id} order=${step.order}) → in_progress`,
);
this.persistLog(
stepRun.runId,
stepRun.id,
"log",
`Step ${step.order} started${step.title ? `: ${step.title}` : ""}`,
);
try {
if (!step.execCode) throw new Error("step has no execCode");
@@ -453,6 +466,12 @@ export class ScenarioSchedulerService implements OnModuleInit {
output !== null && output !== undefined ? JSON.stringify(output) : null;
await this.runStepRepo.save(stepRun);
this.logger.log(`StepRun #${stepRun.id} → pass`);
this.persistLog(
stepRun.runId,
stepRun.id,
"log",
`Step ${(stepRun.scenarioStep as ScenarioStepEntity | undefined)?.order ?? "?"} passed`,
);
// Find the next waiting step in this run (next by order)
const nextStep = await this.runStepRepo.findOne({
@@ -489,6 +508,12 @@ export class ScenarioSchedulerService implements OnModuleInit {
stepRun.description = description;
await this.runStepRepo.save(stepRun);
this.logger.log(`StepRun #${stepRun.id} → fail: ${description}`);
this.persistLog(
stepRun.runId,
stepRun.id,
"error",
`Step ${(stepRun.scenarioStep as ScenarioStepEntity | undefined)?.order ?? "?"} failed: ${description}`,
);
// Cancel all remaining waiting/pending step runs in this run
await this.runStepRepo
@@ -260,6 +260,29 @@ export class ScenarioController {
return this.scenarioService.findRun(id, runId, q);
}
@Get(":id/run/:runId/logs/export")
@ApiOperation({ summary: "Export run logs as CSV or Markdown" })
@ApiResponse({ status: 200 })
@ApiResponse({ status: 404, description: "Scenario or run not found" })
async exportRunLogs(
@Param("id", ParseUUIDPipe) id: string,
@Param("runId", ParseUUIDPipe) runId: string,
@Query("format") format: string | undefined,
@Res({ passthrough: true }) res: Response,
) {
const fmt = format === "csv" ? "csv" : "md";
const content = await this.scenarioService.exportRunLogs(id, runId, fmt);
const contentType =
fmt === "csv" ? "text/csv; charset=utf-8" : "text/markdown; charset=utf-8";
const ext = fmt === "csv" ? "csv" : "md";
res.setHeader("Content-Type", contentType);
res.setHeader(
"Content-Disposition",
`attachment; filename="run-${runId}-logs.${ext}"`,
);
return content;
}
@Post(":id/run/:runId/wait")
@HttpCode(200)
@ApiOperation({
+127 -1
View File
@@ -5,6 +5,7 @@ import {
NotFoundException,
} from "@nestjs/common";
import { InjectRepository } from "@nestjs/typeorm";
import * as fs from "fs/promises";
import * as path from "path";
import { Like, Repository } from "typeorm";
import {
@@ -401,11 +402,49 @@ export class ScenarioService {
);
const logs = await this.runLogRepo.find({
where: q?.trim() ? { runId, message: Like(`%${q.trim()}%`) } : { runId },
order: { createdAt: "ASC" },
order: { seq: "ASC", createdAt: "ASC" },
});
return Object.assign(run, { logs });
}
async exportRunLogs(
scenarioId: string,
runId: string,
format: "csv" | "md",
): Promise<string> {
await this.findOne(scenarioId); // 404 guard
const run = await this.runRepo.findOne({
where: { id: runId, scenarioId },
});
if (!run)
throw new NotFoundException(
`Run ${runId} not found in scenario ${scenarioId}`,
);
const logs = await this.runLogRepo.find({
where: { runId },
order: { seq: "ASC", createdAt: "ASC" },
});
if (format === "csv") {
const escapeCsv = (value: string): string =>
/[",\n]/.test(value) ? `"${value.replace(/"/g, '""')}"` : value;
const rows = [
["Timestamp", "Level", "Message"].map(escapeCsv).join(","),
...logs.map((l) =>
[l.createdAt.toISOString(), l.level, l.message]
.map((v) => escapeCsv(String(v)))
.join(","),
),
];
return rows.join("\n");
}
const sections = logs.map(
(l) => `## [${l.level}] ${l.createdAt.toISOString()}\n\n${l.message}`,
);
return [`# Run ${runId} logs`, ...sections].join("\n\n");
}
async waitForRun(
scenarioId: string,
runId: string,
@@ -768,6 +807,47 @@ export class ScenarioService {
res.sendFile(fullPath);
}
/**
* Get scenario file content as a buffer (for MCP use).
* Returns metadata plus raw bytes without streaming to Response.
*/
async getScenarioFileContentAsBuffer(
scenarioId: string,
fileId: string,
): Promise<{ file: { id: string; name: string; mimeType: string; size: number; sha256: string; expiresAt: Date | null; createdAt: Date }; contentBuffer: Buffer }> {
// Verify scenario exists
await this.findOne(scenarioId);
// Check if file is linked to this scenario
const link = await this.scenarioFileRepo.findOne({
where: { scenarioId, fileId },
relations: ["file"],
});
if (!link) {
throw new NotFoundException(`File ${fileId} not found in scenario ${scenarioId}`);
}
const file = link.file;
const fullPath = path.resolve(this.fileStorageService.getAbsolutePath(file.filePath));
// Read file content as buffer
const contentBuffer = await fs.readFile(fullPath);
return {
file: {
id: file.id,
name: file.originalName,
mimeType: file.mimeType,
size: file.size,
sha256: file.sha256,
expiresAt: file.expiresAt,
createdAt: file.createdAt,
},
contentBuffer,
};
}
async listRunFiles(
scenarioId: string,
runId: string,
@@ -833,4 +913,50 @@ export class ScenarioService {
res.sendFile(fullPath);
}
/**
* Get run artifact file content as a buffer (for MCP use).
* Returns metadata plus raw bytes without streaming to Response.
*/
async getRunFileContentAsBuffer(
scenarioId: string,
runId: string,
fileId: string,
): Promise<{ file: { id: string; name: string; mimeType: string; size: number; sha256: string; expiresAt: Date | null; createdAt: Date }; contentBuffer: Buffer }> {
// Verify scenario and run exist
await this.findOne(scenarioId);
const run = await this.runRepo.findOneBy({ id: runId, scenarioId });
if (!run) {
throw new NotFoundException(`Run ${runId} not found in scenario ${scenarioId}`);
}
// Check if file is linked to this run
const link = await this.scenarioRunFileRepo.findOne({
where: { runId, fileId },
relations: ["file"],
});
if (!link) {
throw new NotFoundException(`File ${fileId} not found in run ${runId}`);
}
const file = link.file;
const fullPath = path.resolve(this.fileStorageService.getAbsolutePath(file.filePath));
// Read file content as buffer
const contentBuffer = await fs.readFile(fullPath);
return {
file: {
id: file.id,
name: file.originalName,
mimeType: file.mimeType,
size: file.size,
sha256: file.sha256,
expiresAt: file.expiresAt,
createdAt: file.createdAt,
},
contentBuffer,
};
}
}
+390
View File
@@ -1,6 +1,14 @@
import { INestApplication } from "@nestjs/common";
import { getRepositoryToken } from "@nestjs/typeorm";
import request from "supertest";
import * as http from "http";
import { Repository } from "typeorm";
import { buildTestApp } from "./app.harness";
import { ScenarioEntity } from "../src/scenario/scenario.entity";
import { ScenarioStepEntity } from "../src/scenario/scenario-step.entity";
import { ScenarioRunEntity } from "../src/scenario/scenario-run.entity";
import { ScenarioRunStepEntity } from "../src/scenario/scenario-run-step.entity";
import { ScenarioSchedulerService } from "../src/scenario/scenario-scheduler.service";
/**
* MCP controller integration tests.
@@ -16,15 +24,65 @@ import { buildTestApp } from "./app.harness";
*/
describe("McpController", () => {
let app: INestApplication;
let testServer: http.Server;
let testServerUrl: string;
let scheduler: ScenarioSchedulerService;
let scenarioRepo: Repository<ScenarioEntity>;
let stepRepo: Repository<ScenarioStepEntity>;
let runRepo: Repository<ScenarioRunEntity>;
let runStepRepo: Repository<ScenarioRunStepEntity>;
beforeAll(async () => {
app = await buildTestApp();
scheduler = app.get(ScenarioSchedulerService);
scenarioRepo = app.get<Repository<ScenarioEntity>>(
getRepositoryToken(ScenarioEntity),
);
stepRepo = app.get<Repository<ScenarioStepEntity>>(
getRepositoryToken(ScenarioStepEntity),
);
runRepo = app.get<Repository<ScenarioRunEntity>>(
getRepositoryToken(ScenarioRunEntity),
);
runStepRepo = app.get<Repository<ScenarioRunStepEntity>>(
getRepositoryToken(ScenarioRunStepEntity),
);
// Start a test HTTP server for downloadFile tests
testServer = await createTestServer();
testServerUrl = `http://localhost:${(testServer.address() as any).port}`;
});
afterAll(async () => {
if (testServer) {
testServer.close();
}
await app.close();
});
// ── helpers ────────────────────────────────────────────────────────────────
function createTestServer(): Promise<http.Server> {
return new Promise((resolve) => {
const server = http.createServer((req, res) => {
if (req.url === "/test-file.bin") {
res.writeHead(200, { "Content-Type": "application/octet-stream" });
res.end(Buffer.from("test downloaded file content"));
} else if (req.url === "/test.pdf") {
res.writeHead(200, { "Content-Type": "application/pdf" });
res.end(Buffer.from("fake pdf content"));
} else {
res.writeHead(404);
res.end();
}
});
server.listen(0, "localhost", () => {
resolve(server);
});
});
}
/** Parse the JSON-RPC payload from an SSE response body. */
function parseSse(text: string): Record<string, unknown> {
const match = text.match(/^data:\s*(.+)$/m);
@@ -47,6 +105,49 @@ describe("McpController", () => {
return { status: res.status, rpc: parseSse(res.text) };
}
async function createScenario(name: string) {
return scenarioRepo.save(scenarioRepo.create({ name }));
}
async function createStep(scenarioId: string, execCode: string, order = 0) {
return stepRepo.save(
stepRepo.create({
scenarioId,
order,
execCode,
}),
);
}
async function createRun(scenarioId: string) {
return runRepo.save(runRepo.create({ scenarioId, status: "pending" }));
}
async function createRunStep(runId: string, scenarioStepId: string, order = 0) {
return runStepRepo.save(
runStepRepo.create({
runId,
scenarioStepId,
order,
status: "pending",
}),
);
}
async function waitForRunCompletion(runId: string, maxAttempts = 50) {
for (let index = 0; index < maxAttempts; index += 1) {
await scheduler.pickUpPendingRuns();
await new Promise((resolve) => setTimeout(resolve, 100));
const run = await runRepo.findOneBy({ id: runId });
if (run && (run.status === "pass" || run.status === "fail")) {
return run;
}
}
throw new Error(`Run ${runId} did not complete within timeout`);
}
// ── Connectivity ───────────────────────────────────────────────────────────
describe("POST /mcp — connectivity", () => {
@@ -285,6 +386,295 @@ describe("McpController", () => {
});
});
// ── Scenario File Tools ────────────────────────────────────────────────────
describe("upload_scenario_file", () => {
it("uploads a small text file via MCP and returns metadata", async () => {
const scRpc = (
await mcpCall("create_scenario", { name: "mcp-file-upload-test" })
).rpc;
const sc = JSON.parse(
(scRpc.result as { content: { text: string }[] }).content[0].text,
) as { id: string };
const content = "Hello, MCP!";
const contentBase64 = Buffer.from(content).toString("base64");
const { status, rpc } = await mcpCall("upload_scenario_file", {
scenarioId: sc.id,
name: "test.txt",
contentBase64,
mimeType: "text/plain",
});
expect(status).toBe(200);
const result = rpc.result as { content: { text: string }[] };
const uploaded = JSON.parse(result.content[0].text) as {
id: string;
name: string;
mimeType: string;
size: number;
};
expect(uploaded.name).toBe("test.txt");
expect(uploaded.mimeType).toBe("text/plain");
expect(uploaded.size).toBe(content.length);
expect(typeof uploaded.id).toBe("string");
});
it("returns MCP error for invalid base64", async () => {
const scRpc = (
await mcpCall("create_scenario", {
name: "mcp-file-bad-base64-test",
})
).rpc;
const sc = JSON.parse(
(scRpc.result as { content: { text: string }[] }).content[0].text,
) as { id: string };
const { status, rpc } = await mcpCall("upload_scenario_file", {
scenarioId: sc.id,
name: "test.txt",
contentBase64: "!@#$%^&*()",
});
expect(status).toBe(200);
const result = rpc.result as { isError: boolean };
expect(result.isError).toBe(true);
});
});
describe("list_scenario_files", () => {
it("returns uploaded files with metadata", async () => {
const scRpc = (
await mcpCall("create_scenario", { name: "mcp-file-list-test" })
).rpc;
const sc = JSON.parse(
(scRpc.result as { content: { text: string }[] }).content[0].text,
) as { id: string };
// Upload a file
const content = "Test content";
const contentBase64 = Buffer.from(content).toString("base64");
await mcpCall("upload_scenario_file", {
scenarioId: sc.id,
name: "test.txt",
contentBase64,
});
// List files
const { status, rpc } = await mcpCall("list_scenario_files", {
scenarioId: sc.id,
});
expect(status).toBe(200);
const result = rpc.result as { content: { text: string }[] };
const list = JSON.parse(result.content[0].text) as {
items: Array<{ id: string; name: string }>;
total: number;
};
expect(list.items.length).toBeGreaterThan(0);
expect(list.items[0].name).toBe("test.txt");
expect(typeof list.total).toBe("number");
});
it("returns MCP error for non-existent scenario", async () => {
const { status, rpc } = await mcpCall("list_scenario_files", {
scenarioId: "00000000-0000-0000-0000-000000000000",
});
expect(status).toBe(200);
const result = rpc.result as { isError: boolean };
expect(result.isError).toBe(true);
});
});
describe("get_scenario_file_content", () => {
it("retrieves file content as base64 with metadata", async () => {
const scRpc = (
await mcpCall("create_scenario", { name: "mcp-file-content-test" })
).rpc;
const sc = JSON.parse(
(scRpc.result as { content: { text: string }[] }).content[0].text,
) as { id: string };
// Upload a file
const originalContent = "Hello, this is test content!";
const contentBase64 = Buffer.from(originalContent).toString("base64");
const uploadRpc = (
await mcpCall("upload_scenario_file", {
scenarioId: sc.id,
name: "content-test.txt",
contentBase64,
mimeType: "text/plain",
})
).rpc;
const uploaded = JSON.parse(
(uploadRpc.result as { content: { text: string }[] }).content[0].text,
) as { id: string };
// Retrieve content
const { status, rpc } = await mcpCall("get_scenario_file_content", {
scenarioId: sc.id,
fileId: uploaded.id,
});
expect(status).toBe(200);
const result = rpc.result as { content: { text: string }[] };
const response = JSON.parse(result.content[0].text) as {
file: { id: string; name: string; size: number };
contentBase64: string;
encoding: string;
};
// Verify roundtrip
const retrievedContent = Buffer.from(response.contentBase64, "base64").toString(
"utf8",
);
expect(retrievedContent).toBe(originalContent);
expect(response.file.name).toBe("content-test.txt");
expect(response.file.size).toBe(originalContent.length);
expect(response.encoding).toBe("base64");
});
it("returns MCP error for unknown file ID", async () => {
const scRpc = (
await mcpCall("create_scenario", {
name: "mcp-file-unknown-id-test",
})
).rpc;
const sc = JSON.parse(
(scRpc.result as { content: { text: string }[] }).content[0].text,
) as { id: string };
const { status, rpc } = await mcpCall("get_scenario_file_content", {
scenarioId: sc.id,
fileId: "00000000-0000-0000-0000-000000000000",
});
expect(status).toBe(200);
const result = rpc.result as { isError: boolean };
expect(result.isError).toBe(true);
});
});
// ── Run Artifact Tools ─────────────────────────────────────────────────────
async function createRunArtifactFixture(name: string) {
const scenario = await createScenario(name);
const step = await createStep(
scenario.id,
`return await context.downloadFile('${testServerUrl}/test-file.bin', { filename: 'artifact.bin' });`,
);
const run = await createRun(scenario.id);
await createRunStep(run.id, step.id);
const completedRun = await waitForRunCompletion(run.id);
expect(completedRun.status).toBe("pass");
return { sc: scenario, run: completedRun };
}
describe("list_run_files", () => {
it("returns artifacts created during a run with expiry metadata", async () => {
const { sc, run } = await createRunArtifactFixture("mcp-run-files-test");
const { status, rpc } = await mcpCall("list_run_files", {
scenarioId: sc.id,
runId: run.id,
});
expect(status).toBe(200);
const result = rpc.result as { content: { text: string }[] };
const list = JSON.parse(result.content[0].text) as {
items: Array<{
id: string;
name: string;
mimeType: string;
expiresAt: string | null;
}>;
total: number;
};
expect(list.total).toBe(1);
expect(list.items).toHaveLength(1);
expect(list.items[0].name).toBe("artifact.bin");
expect(list.items[0].mimeType).toBe("application/octet-stream");
expect(list.items[0].expiresAt).not.toBeNull();
});
it("returns MCP error for non-existent scenario", async () => {
const { status, rpc } = await mcpCall("list_run_files", {
scenarioId: "00000000-0000-0000-0000-000000000000",
runId: "00000000-0000-0000-0000-000000000000",
});
expect(status).toBe(200);
const result = rpc.result as { isError: boolean };
expect(result.isError).toBe(true);
});
});
describe("get_run_file_content", () => {
it("returns run artifact bytes as base64 for a valid file", async () => {
const { sc, run } = await createRunArtifactFixture("mcp-run-file-content-test");
const listRpc = (
await mcpCall("list_run_files", {
scenarioId: sc.id,
runId: run.id,
})
).rpc;
const list = JSON.parse(
(listRpc.result as { content: { text: string }[] }).content[0].text,
) as {
items: Array<{ id: string; name: string; mimeType: string }>;
};
const file = list.items[0];
const { status, rpc } = await mcpCall("get_run_file_content", {
scenarioId: sc.id,
runId: run.id,
fileId: file.id,
});
expect(status).toBe(200);
const result = rpc.result as { content: { text: string }[] };
const response = JSON.parse(result.content[0].text) as {
file: { id: string; name: string; mimeType: string; expiresAt: string | null };
contentBase64: string;
encoding: string;
};
expect(response.file.id).toBe(file.id);
expect(response.file.name).toBe("artifact.bin");
expect(response.file.mimeType).toBe("application/octet-stream");
expect(response.file.expiresAt).not.toBeNull();
expect(response.encoding).toBe("base64");
expect(Buffer.from(response.contentBase64, "base64").toString("utf8")).toBe(
"test downloaded file content",
);
});
it("returns MCP error for mismatched run and file ids", async () => {
const { sc: firstScenario, run: firstRun } = await createRunArtifactFixture(
"mcp-run-file-mismatch-a",
);
const { sc: secondScenario, run: secondRun } = await createRunArtifactFixture(
"mcp-run-file-mismatch-b",
);
const firstListRpc = (
await mcpCall("list_run_files", {
scenarioId: firstScenario.id,
runId: firstRun.id,
})
).rpc;
const firstList = JSON.parse(
(firstListRpc.result as { content: { text: string }[] }).content[0].text,
) as { items: Array<{ id: string }> };
const { status, rpc } = await mcpCall("get_run_file_content", {
scenarioId: secondScenario.id,
runId: secondRun.id,
fileId: firstList.items[0].id,
});
expect(status).toBe(200);
const result = rpc.result as { isError: boolean };
expect(result.isError).toBe(true);
});
});
// ── Snippet CRUD tools ────────────────────────────────────────────────────
describe("list_snippets", () => {