Write a scenario

A scenario is a YAML file that says which application is under test, what to do to it, and how to check each step. Kasm Session Runtime validates the whole file before anything runs. An unknown field or action word is rejected at load, before the session is touched.

A scripted scenario needs no vision model. In this guide only observe: steps, expect: checks, the click_grounded action and the screen postcondition use one, and the tables mark them. With no model configured the runtime runs in deterministic-only mode: the window and process probes decide the verdict, and a check that needs a judgement is recorded as not judged.

This guide builds a scripted scenario. A goal: scenario, where the runtime’s own loop decides the steps, needs a model; see Plans and postconditions. Every field is listed in the scenario YAML reference.

Before you start

• A Kasm session with the runtime in it. See Enable the runtime on a workspace.
• The application’s window title (or window class) and the name its process runs under.
• For a Chromium-family browser: the browser started with a debugging port, set through APP_ARGS (step 3).

1. Name the application

app: chrome
window: "Chrom"         # substring of the window title or class; matches Chrome and Chromium
process: "chrome"       # substring of the process command line
launch_timeout_s: 90    # default 60
settle_s: 5             # default 5

Field	What it does
app	The app under test. Usually left out; KASM_RUNTIME_APP supplies it
window	Matched against window titles and classes to find the app’s window. Defaults to app
process	Matched against process command lines. Defaults to app
launch_timeout_s	How long to wait for the window and process before the run is app-failed. Raise it for slow starters
settle_s	How long to wait after launch before the first check

If the image does not start the app itself, add a launch: block with the command. Its output goes to ~/.xsession-errors:

launch:
  cmd: "blender --python-expr ''"
  env: {BLENDER_USER_CONFIG: /tmp/cfg}

2. Write the steps

steps: is a list of at most 40 entries. Each entry has either an action or an observe.

Choose each action from as high in this table as the application allows. In the higher rows the application or the author supplies the target, which makes them cheaper and more reliable. The runtime records the backend that ran each step and, for clicks, its rung. See How an action reaches the app.

Action	What it does	Needs a model
cdp_navigate "<url>"	Loads a URL in the attached browser tab	No
cdp_query "<selector>"	Checks an element exists and is visible; fails if not	No
cdp_click "<selector>"	Clicks an element through the DOM	No
cdp_click_described "<name>"	Clicks the element with this accessible name, matched against the page’s interactive elements	No
cdp_fill "<selector>" "<value>"	Fills a field and fires input and change. The report records the length, never the value	No
cdp_wait "<js predicate>" [seconds]	Waits for a JavaScript expression to be true; fails with the page’s visible candidates if it never is	No
cdp_url	Records the page’s location.href	No
cdp_attach "<tab substring>"	Switches to the tab whose URL contains this	No
key "<chord>"	Presses a key chord: ctrl, alt, shift, super plus a key	No
type "<text>"	Types text verbatim. $VAR and ${VAR} are expanded from the environment	No
click_element "<name>"	Clicks a control the application publishes over AT-SPI, by its name	No
click "<name>"	Clicks a point declared in click_targets	No
click <fx> <fy>	Clicks at fractions of the window, each from 0 to 1	No
click center	Clicks the middle of the app’s window	No
click_grounded "<description>"	Asks the vision model where the described thing is, then clicks it. Requires expect: on the same step	Yes
wait <seconds>	Waits	No
wait_while "<name>"	Holds while a condition declared in wait_conditions is true	No
shell "<name>"	Runs a command declared in shell. Output is capped	No
spawn "<cmd>"	Starts a tool, for desktop: true scenarios	No
kill "<pattern>"	Stops processes whose command line matches this regular expression	No

Quoted arguments are taken verbatim, with no backslash escapes. A CSS selector that needs a quote uses the single-quote form: a[title='x'].

click "<name>", wait_while and shell refer to names you declare elsewhere in the file:

click_targets:              # window-relative fractions, authored with a screenshot in hand
  render-button: [0.5, 0.12]
shell:
  list-renders: "ls ~/renders"
wait_conditions:
  render-in-progress:
    kind: process           # or growing, with path:
    match: "blender-render"
    timeout_s: 300
    poll_s: 2
    quiet_polls: 3

When a native step and a browser tab are both in play, name the target on the step. Otherwise a key chord goes to whichever window has focus:

  - action: key "ctrl+alt+t"
    target: window "Terminal"

3. Browser scenarios

Set cdp: true, or use any cdp_* action, which implies it. The runtime then attaches to the browser over the Chrome DevTools Protocol at http://127.0.0.1:9222.

Chrome must be started with a debugging port. Set APP_ARGS on the session:

--start-maximized --test-type --ignore-certificate-errors --remote-debugging-port=9222 --remote-debugging-address=127.0.0.1 --remote-allow-origins=* --user-data-dir=/home/kasm-user/.config/chrome-cdp-profile

• --user-data-dir is required on Chrome 136 and later. Without it Chrome ignores --remote-debugging-port and gives no error.
• APP_ARGS replaces the image’s default arguments, so --start-maximized is restated.
• Navigate with cdp_navigate. Chrome is already running in these images, and a URL on a command line goes to the running instance without the flags.

4. Check each step

A step can carry checks. Use the strongest one the application offers.

Check	What it asks	Needs a model
post: with url, dom, state, window, no_window, process, no_process or file	Waits up to timeout_s (default 10) for a fact the browser, the app or the system reports	No
expect_state: Bold checked	Asks the app over AT-SPI whether a control is in a state. States: checked, enabled, expanded, focused, pressed, selected, sensitive, showing; write not before the state to negate it	No
cdp_wait as the next step	A JavaScript predicate that must become true	No
post: with screen	The vision model judges a description true once, on a settled screenshot	Yes
expect: "<description>"	The vision model judges the description true after the action	Yes
observe: "<description>" as a step	The vision model judges a fresh screenshot	Yes

Exactly one kind goes in each post: entry:

  - action: cdp_click "button.login"
    post:
      - url: "#/dashboard"
      - dom: "document.getElementById('who').textContent === 'demo-user'"
        timeout_s: 15

To fail the run when a known-fatal line appears in the log:

checks:
  log_fatal: ["EGL error", "GPU process crashed"]

5. Keep secrets out of the file

Put credentials in environment variables named KASM_RUNTIME_SECRET_* and refer to them with $:

  - action: cdp_fill "#password" "$KASM_RUNTIME_SECRET_PW"

Values of KASM_RUNTIME_SECRET_* variables, and values filled from a $VAR reference, are replaced with [REDACTED] in everything the runtime writes. A password drawn on screen still appears in screenshots. See Secrets and evidence.

6. Run it

Save the file into the session and run it as the session user from a terminal in the session, or through the Kasm API’s exec call:

export KASM_RUNTIME_APP=chrome
export KASM_RUNTIME_OUT=$HOME/out
export KASM_RUNTIME_DEADLINE_S=300
export DISPLAY=:1
kasm-session-runtime run scenario.yaml

On stock images built with the SDK layer the binary is /dockerstartup/kasm-session-runtime.

KASM_RUNTIME_DEADLINE_S is the whole run’s limit (default 600); the last 30 seconds are kept for diagnosis and the report. Several files can go on one command line; they share the session and the deadline, and the worst verdict decides the exit code. Leave KASM_RUNTIME_VISION_ENDPOINT unset for a model-free run.

Check it works

The exit code follows the verdict.

Exit code	Verdict
0	pass or pass-with-concerns
1	app-failed
2	test-error

Each scenario writes to $KASM_RUNTIME_OUT/<name>/, where <name> comes from the file name: report.md with the verdict and reasons first, report.json, screenshots/, and DONE last. summary.json at the top holds the worst-of verdict. Every step row in report.md names the backend that ran it (cdp, atspi, xtest, vision, shell, none). A run with no model says deterministic_only=true.

A complete browser scenario that needs no model, from the 01-cdp-chrome example in the SDK:

app: chrome
window: "Chrom"
process: "chrome"
cdp: true
launch_timeout_s: 90

steps:
  - action: cdp_navigate "http://127.0.0.1:8901/login.html"
  - action: cdp_query "#username"
  - action: cdp_fill "#username" "demo-user"
  - action: cdp_fill "#password" "demo-password"
  - action: cdp_click "button.login"
  - action: cdp_wait "location.hash === '#/dashboard'" 15
  - action: cdp_wait "document.getElementById('who').textContent === 'demo-user'" 10
  - action: cdp_url

Troubleshooting

Load errors

A file that fails to load gives verdict test-error and exit code 2. The message is in summary.json and the scenario’s report.

Message	Fix
unknown action "..."	Use a word from the table in step 2. The message lists the vocabulary
needs either `action` or `observe` / has both `action` and `observe`; pick one	Give the step exactly one
click_grounded requires `expect:`	Add expect: saying what the click should change
click "..." references no declared target; add it to click_targets	Declare the name, or write the fractions in the step
fractions must be within 0..1	Fractions are of the window, from 0 to 1
scenario has N steps; the maximum is 40	Split the scenario
expect_state must end with a state name	End with one of the states listed in step 4
cdp_targets/cdp_waits are declared but the scenario does not enable cdp: true	Add cdp: true

Failures at run time

Symptom	Cause
Every cdp_* step fails; CDP is not there	Chrome started without --user-data-dir and ignored the debugging port
Navigation lands on chrome-error://chromewebdata/	The Kasm browser images block file:// by policy. Serve the page over loopback HTTP
A login “times out” although it worked	The page never changes document.title. Check location.hash or location.href with cdp_wait or a url postcondition
no element matches "..." with a list of candidates	The selector is stale; the candidates show what the page has now

Next

• Scenario YAML reference
• Plans and postconditions
• Serve a vision model