Scenario YAML

A scenario file says which app to test, what to do in it and what must be true afterwards. It is one YAML document. run takes scenario files; the no-argument invocation takes one base64-encoded in KASM_RUNTIME_SCENARIO_B64. For a walk-through, see Write a scenario.

Unknown fields, unknown action words and a second YAML document are rejected when the scenario loads. An empty scenario is valid, and runs launch, settle, one observation and the verdict.

Top level

Field	Default	What it does	Needs a model
app	KASM_RUNTIME_APP	The app under test	No
window	app	A substring matched against window titles (_NET_WM_NAME) and classes (WM_CLASS) to find the app’s window	No
process	app	A substring matched against process command lines in /proc/*/cmdline	No
launch_timeout_s	60	Seconds to wait for the window and process before the run is app-failed. Positive whole number	No
settle_s	5	Seconds to wait after launch before the first judgement. 0 skips settling	No
launch	none	A command run once, detached, at the start of the launch wait. See launch	No
steps	none	Scripted steps, at most 40. See A step	Only for observe, expect, click_grounded and screen
plan	none	A compiled replay run after steps: actions with mandatory postconditions. If a step fails or a postcondition does not hold, the agent loop takes over from there. Requires goal. At most 40 steps. See Plans and postconditions	No, until it falls back to the goal
goal	none	A natural-language goal that runs the runtime’s agent loop, after any steps	Yes
provides	none	Named values the model may ask to have typed, with $VAR expanded. The model never composes typed text	Used with goal
agentic_max_iterations	runner.max_iterations (25)	This scenario’s cap on model calls. Positive	Used with goal
checks.log_fatal	none	Substrings which, found in the collected logs, fail the run	No
desktop	false	The image is a desktop of many tools rather than one app. Nothing is launched or waited for; steps spawn, check and kill each tool. Cannot be combined with launch	No
atspi	true	Whether the run may touch the accessibility bus. false means no probe and no connection, and rejects click_element, expect_state and state postconditions	No
cdp	false	Attach to a Chromium-family browser over the Chrome DevTools Protocol. Turned on by any cdp_* step, or url or dom postcondition, in steps or plan. A declared tool with a url or dom postcondition needs it set	No
cdp_targets	none	Named CSS selectors the model may use in a goal. Requires cdp	Used with goal
cdp_waits	none	Named JavaScript predicates the model may wait on. Requires cdp	Used with goal
click_targets	none	Named window-relative positions, [x, y] with each between 0 and 1	No
wait_conditions	none	Named holds for wait_while. See A wait condition	No
shell	none	Named literal shell commands for shell steps and declared tools	No
tools	none	Tools the model may call, composed from shell, wait_while and launch. Requires goal. See A declared tool	Used with goal
rules	none	Patterns over captured output that offer a declared tool to the model. Requires goal. See A rule	Used with goal
policy	shipped procedure	Where the loop’s workflow procedure comes from. See policy	Used with goal
grounding_audit	false	Turns the run into a measurement of the model’s ability to locate controls, scored against the rectangles the app publishes over AT-SPI. Clicks nothing. Cannot be combined with goal, steps or plan	Yes
grounding_audit_describe	name	How controls are described in an audit: name, or name_role (name plus role)	Yes
needs	none	Scheduler requirements, a map of strings. Accepted and ignored by the runtime	No

Steps

Each entry in steps or plan has either one action or one observe.

Field	What it does	Needs a model
action	One action from the vocabulary below	Only click_grounded
observe	A description judged against a fresh screenshot. Not allowed in plan	Yes. Without one it is recorded as not judged and the step passes
expect	A description that must be judged true after the action. Required with click_grounded	Yes. Without one it is recorded as not judged, except after click_grounded
expect_state	Asks the app over AT-SPI whether a named control is in a state: <element> [not] <state>, for example Bold checked or Bold not checked	No
post	Postconditions waited for after the action. Optional in steps, at least one in each plan step. See Postconditions	Only screen
target	Who receives the action: window "<substring>" activates that window first; browser addresses the attached tab. cdp_* actions cannot target a window, and only cdp_* actions may target browser	No

The expect_state states are checked, enabled, expanded, focused, pressed, selected, sensitive and showing. The element name is written unquoted.

Actions

Arguments in double quotes are taken verbatim, with no backslash escapes. A selector that needs quotes uses CSS’s single-quote form: a[title='x'].

Action	What it does	Rung	Needs a model
key "<chord>"	Presses a key or chord, such as key "ctrl+s"		No
type "<text>"	Types the text verbatim, with $VAR expanded		No
click center	Clicks the middle of the app’s window	center	No
click "<name>"	Clicks a position declared in click_targets	declared	No
click <fx> <fy>	Clicks at window-relative fractions, each 0 to 1	literal	No
click_element "<name>"	Clicks a control the app names over AT-SPI; the app performs it or publishes where it is	element	No
click_grounded "<description>"	Asks the model where something is and clicks there. expect is required	grounded	Yes. Without one the step fails
wait <seconds>	Waits. Zero or more; stops at the deadline reserve		No
wait_while "<name>"	Holds on a condition declared in wait_conditions		No
shell "<name>"	Runs a command declared in shell. Output is capped		No
spawn "<cmd>"	Starts a program on the desktop		No
kill "<pattern>"	Stops processes whose full command line matches a POSIX extended regular expression, as pkill -f does		No
cdp_navigate "<url>"	Loads a URL in the attached tab		No
cdp_attach "<substring>"	Attaches to the tab whose URL or title contains the substring, ignoring case		No
cdp_query "<selector>"	Checks that a CSS selector matches an element, and records the first match; fails the step if none appears		No
cdp_click "<selector>"	Clicks an element by CSS selector		No
cdp_click_described "<name>"	Clicks an element by its accessible name or handle, as the page’s element list reports it		No
cdp_fill "<selector>" "<value>"	Fills a field. $VAR is expanded; only the value’s length is recorded		No
cdp_wait "<predicate>" [seconds]	Waits for a JavaScript expression to be true, or fails the step		No
cdp_url	Records the tab’s current URL		No

A chord joins modifiers and one key with +. Modifiers: ctrl or control, alt or meta, shift, super, win or cmd. The key is one printable character, f1 to f12, or one of return, enter, tab, escape, esc, space, backspace, delete, del, insert, home, end, pageup, prior, pagedown, next, left, up, right, down, minus, plus, equal, comma, period, slash.

The rung column is recorded on each report row. See How an action reaches the app.

Environment references

type, cdp_fill and provides: values expand $NAME and ${NAME} from the environment when the step runs. $$ is a literal $. A variable that is not set fails the step; it is never typed as blank. An unterminated ${ or an empty ${} is rejected at load. Values of KASM_RUNTIME_SECRET_* variables are redacted from all output; see Secrets and evidence.

Postconditions

Each entry in a post list sets exactly one kind. They are listed from the strongest evidence to the weakest.

Field	What must be true	Needs a model
url	The tab’s URL contains this substring. Turns on cdp	No
dom	This JavaScript expression is true in the page. Turns on cdp	No
state	An AT-SPI control is in a state, written as for expect_state	No
window	A window whose title or class contains this exists	No
no_window	No such window exists	No
process	A process whose command line contains this is running	No
no_process	No such process is running	No
file	This path exists	No
screen	The model judges this description true, once, on a settled screenshot	Yes. Without one it is recorded as not judged and holds
timeout_s	How long a deterministic kind is re-checked before it fails. Default 10 seconds. Not allowed with screen

launch

Field	Default	What it does
cmd	required	The command, run once, detached. Its output goes to ~/.xsession-errors
env	none	Extra environment for the command
on_missing_window	fail	fail: a launch timeout ends the run app-failed. continue: the agent loop starts instead, and the end-of-run window and process check still decides the verdict. continue requires goal and cannot be combined with cdp
on_error_dialog	fail	fail: an error dialog after launch ends the run app-failed. continue: the agent loop starts and may dismiss it; a dialog still up at the end is app-failed. continue requires goal

Wait conditions

Every field of a wait condition is required.

Field	What it does
kind	growing: hold while a path’s size keeps rising. process: hold while a matching process exists
path	For growing: the file or directory watched
match	For process: a substring of the command line
timeout_s	The whole hold’s ceiling, in seconds. Positive
poll_s	Seconds between samples. Positive
quiet_polls	Consecutive quiet samples needed to release. Positive whole number

Declared tools

Every field is required except commands, then, terminal and post. A tool’s name cannot be one of the built-in tool names listed on Operator file.

Field	What it does
description	What the model is told the tool does
params	Named parameters, each {enum: [...]} with at least one value and no duplicates. There is no free-text parameter. Use {} for none
run	The primitive: shell, wait_while or launch
commands	For run: shell: one literal command per value of a name parameter, which is then required
then	relaunch, or omitted
prerequisites	Tools that must have been called first: a bare name, or {tool: <name>, match_arg: <param>} to require the same argument value. Use [] for none
side_effects	true or false; must be written either way
terminal	Whether a successful call ends the loop. Default false
budget	How many times the tool may be called in a run. Positive
on_error	A map from failure condition to class: malformed, resolution or environment
post	Postconditions the tool promises. One that does not hold is reported to the model as a resolution error

run: wait_while also needs a name parameter whose values are declared in wait_conditions. run: launch needs a top-level launch block.

on_error conditions by primitive

run	Conditions
shell	nonzero_exit, missing_command, timeout
wait_while	timeout
launch	nonzero_exit

Rules

Field	What it does
match	A regular expression over captured output, including the launch’s log lines. Required
offer.tool	The declared tool offered to the model when it matches. Required
offer.name	For a tool with a name parameter, which value to offer

policy

Exactly one of skill and inline is set. The runtime’s own guardrail text is always placed first and cannot be overridden.

Field	What it does
skill	The name of a procedure built into the binary. The binary ships one: generic-gui-test
inline	The procedure text, in the scenario
vars	Values for the procedure’s {{.name}} placeholders. A placeholder with no value fails at load
tools	The tool names the procedure refers to, checked against what the scenario provides
literal	With inline only: use the text as written, braces included. Cannot be combined with vars

Example

A scripted scenario that needs no model (optional):

app: chrome
steps:
  - action: cdp_navigate "https://example.test/login"
  - action: cdp_fill "#username" "$KASM_RUNTIME_SECRET_USER"
  - action: cdp_fill "#password" "$KASM_RUNTIME_SECRET_PW"
  - action: cdp_click "button[type='submit']"
    post:
      - url: "/dashboard"
        timeout_s: 20
checks:
  log_fatal: ["GPU process crashed"]

Next

• Write a scenario
• Report
• Plans and postconditions