A run from start to finish

A run is what kasm-session-runtime run does with a scenario: wait for the desktop and the app, act, check, decide a verdict, and write the evidence. It is one process inside the session. Every stage has a time limit, and a run always ends with a report. This page describes a run under run. Under mcp there are no stages: the caller decides what happens and when.

The stages

Stage	What happens	Needs a model
Boot wait	Waits until the X11 display accepts a connection, up to 60 seconds by default.	No
Launch	Optional. If the scenario has a `launch:` command, runs it once, detached.	No
App wait	Polls until a process matching the scenario’s `process:` is running and a window matching `window:` is on screen, up to `launch_timeout_s` (60 by default).	No
Settle	Waits `settle_s` (5 by default) for splash screens and first paint, checks the app did not crash straight away, and notes any in-app updater still on screen.	No
Accessibility probe	Lists the controls the app publishes over AT-SPI. Recorded for reference; it never decides the verdict. Skipped with `atspi: false`.	No
First look	Takes the post-launch screenshot, compares it with a baseline screenshot if one exists, and looks for an error-dialog window through X11. With a model, also judges whether the app rendered and whether an error is showing.	Only for the judgement
Browser attach	For scenarios that use the Chrome DevTools Protocol (CDP), connects to the browser.	No
Steps	Runs the scenario’s `steps:` in order, with each step’s checks. A failing step ends the run.	Only for visual checks and grounded clicks
Plan	Replays a compiled `plan:`. A step whose postcondition does not hold hands over to the goal loop.	No
Goal loop	For a `goal:` scenario, the guarded agentic loop works towards the goal.	Yes
Final checks	Confirms the app’s process is still running and its window is still present, then combines the loop’s own verdict with these checks.	No
Report	Writes the evidence to `KASM_RUNTIME_OUT`, with `DONE` last.	No

When the app never appears, or a step shows the app failing, the run goes to diagnosis instead: it re-checks the process to tell a crash from a crash loop, saves 99-failure.png, collects log tails and, with a model, asks for a description of the failure screen. Then it writes the verdict and the report.

Desktop and agentic modes

Desktop mode (desktop: true) is for an image that carries a desktop of tools instead of one app. There is no app to wait for: the run settles on the desktop, and the steps spawn, observe and kill each tool in turn. The end-of-run process and window checks are skipped.
Agentic mode (goal:) replaces scripted steps with the guarded loop, or runs it after them. Everything before the loop still runs, and the deterministic checks still decide the verdict. The model’s own verdict can add a concern, and it can fail the run only when a deterministic signal or a failed check agrees with it.

Verdicts

Each run gets one of four verdicts. The window, process and log checks decide it, together with the checks the scenario itself asked for. A visual check the scenario asked for (observe:, expect:, a screen postcondition) fails its step when the model answers that the expectation does not hold, and a first look judged as not rendered or showing an error fails the run. A model that cannot be reached never fails a run by itself, and nothing a model says turns a failure into a pass.

Verdict	Meaning	Exit code
`pass`	The checks are clean and nothing is flagged	0
`pass-with-concerns`	It worked, but something needs a person’s attention, such as a first-run prompt, baseline drift, or a claim the model made that nothing confirmed	0
`app-failed`	The app is broken: it never launched, crashed, showed an error dialog, or an expected element or condition was absent	1
`test-error`	The runtime could not perform the test: no X display, an unwritable output directory, the deadline passed, or a required backend was unavailable	2

An app-failed verdict points at the application, and a test-error at the setup around it.

Deadlines and interruptions

KASM_RUNTIME_DEADLINE_S (600 by default) bounds the run. The last 30 seconds are reserved for diagnosis and the report. A SIGTERM or SIGINT ends the run early: the wait in progress is cancelled, the report is still written, and the verdict is test-error with outcome interrupted.

Output of a run

In KASM_RUNTIME_OUT: report.json, report.md, trace.jsonl, the screenshots, log tails, a baseline candidate when no baseline existed, and finally DONE, which contains the verdict. DONE is written only once everything else has been written and synced, so an orchestrator can wait for it and then read the rest. See Report.