Operator file

The operator file is a YAML file of tuning that belongs to a deployment rather than to one test. It sets how many turns the runtime’s agent loop may take, how the model is called, how long the runtime waits for the desktop, and how much evidence reaches the model and the report. KASM_RUNTIME_CONFIG_FILE names it. Two people running the same scenario on different hosts then run the same test.

Most deployments never need one. The usual reasons to reach for it are a new model (start with model, max_tokens and sampling), an app that is slow to paint or to appear on the accessibility bus (timing), or a run that keeps flagging something harmless (findings, baseline).

Several blocks only matter when a model is configured: runner, sampling, function_calling, context, max_tokens and model tune the model calls and the runtime’s own agent loop. With KASM_RUNTIME_VISION_ENDPOINT empty they have nothing to act on. The other blocks apply to every run.

Loading the file

• Every leaf is optional. No file, or an empty one, gives the shipped behaviour.
• Every leaf also has an environment override, KASM_RUNTIME_<BLOCK>_<FIELD> upper-cased, with nested blocks joined the same way: timing.input.keystroke_ms is KASM_RUNTIME_TIMING_INPUT_KEYSTROKE_MS. List leaves are comma-separated in the environment. The full list is on Environment variables.
• Precedence: environment, then file, then default.
• A leaf that is out of range or the wrong type logs a warning and keeps the value it had: the default, or the file’s value when a bad override sits on top of it.
• A file that cannot be read, or does not decode, stops the run before it starts. This includes an unknown key, a YAML syntax error and two YAML documents in one file.
• KASM_RUNTIME_CONFIG_STRICT=true turns every warning into a fatal error at start-up.
• The values in force are recorded in report.json at agent.operator_config, with the file’s path but never its contents. See Report.

A typical file is a few lines. For example, for a hosted model that thinks before it answers (optional):

model:
  timeout_s: 180
max_tokens:
  judge: 4000
  tools: 1500

runner: the agent loop’s budgets

Override: KASM_RUNTIME_RUNNER_<LEAF>. These apply to the runtime’s agent loop in goal: scenarios, and grounding_attempts also to scripted click_grounded steps. Both need a model.

Leaf	Default	What it does	When to change it
max_iterations	25	The most model calls one agent loop may make. A scenario’s agentic_max_iterations overrides it. Positive	Raise it as the deployment default for long remediation workflows
max_retries_per_step	3	How many malformed or missing tool calls the loop retries before it degrades. Zero or more	Raise it for a model that often fumbles the call format
max_tool_errors	0 (unbounded)	How many consecutive tool execution failures the loop tolerates before it degrades. A success resets the count. Zero means no limit. Zero or more	Set it when a failing tool should end the loop rather than be retried to the iteration cap
reasoning_replay	full	How much of the model’s own earlier turns is sent back: full, keep-last or none	To shrink the prompt for a small context window
rescue_enabled	true	In injected function calling only: accept a well-formed JSON call even when the reply’s code fence is wrong	Turn it off to measure a model’s format compliance strictly
respond	false	Adds a respond tool so the model can say something without ending the loop	For a model that keeps replying in prose. It changes every scenario’s prompt and tool list
max_hold_s	0 (unbounded)	Total seconds a run may spend in waiting tools: wait, wait_while, cdp_wait, and declared tools with run: wait_while. Zero or more	When a model might spend the deadline waiting
escalation_pause_s	absent	When the model calls escalate, how long the run pauses for a person. Absent: until the deadline, less the reserve. 0: no pause. N: at most N seconds. Zero or more	Set 0 for unattended runs so an escalation does not hold the session
grounding_attempts	2	How many times one step asks the model to locate an element before failing, for click_grounded steps and the agent’s click_grounded tool. At least 1	Raise it for a model whose pointing is noisy but usually right on a retry
max_context_recoveries	3	How many times per run the loop may drop the previous tool result and retry after the server says the prompt is too large. Zero turns recovery off	Raise it for a small context window with large tool results
wait_tool_max_s	30	The most seconds one call of the model’s wait tool may ask for. At least 1	Raise it when installs or downloads take minutes
ask_human_budget_s	600	Total seconds ask_human may spend waiting for a person across the run. 0 means unattended: it returns at once. Zero or more	Set 0 for batch runs; raise it for attended sessions
ask_human_timeout_s	300	The most seconds one ask_human question waits. Positive	Lower it when the model should move on quickly

sampling: sampling parameters for tool-call turns

Override: KASM_RUNTIME_SAMPLING_<LEAF>. These apply to the agent loop’s tool-call turns. Screenshot judgements are always sent at temperature 0, and a retried judgement at 0.7.

Leaf	Default	What it does	When to change it
temperature	0	Randomness. Finite and zero or more	Rarely: a test harness seldom wants variety
top_p	not sent	Nucleus sampling cutoff, 0 to 1	To match a model’s recommended serving settings
top_k	not sent	Candidate-token cutoff. Zero or more	Same
min_p	not sent	Minimum relative probability, 0 to 1	Same

function_calling: how tools reach the model

Override: KASM_RUNTIME_FUNCTION_CALLING. It is a single top-level value.

Leaf	Default	What it does	When to change it
function_calling	native	native sends the OpenAI tools field and reads structured calls back. injected writes the tool list into the prompt and parses a fenced JSON call from the reply	When the serving backend’s native tool calling is poor or missing. Worth measuring per deployment

context: the context window budget

Override: KASM_RUNTIME_CONTEXT_<LEAF>. The runtime has no tokenizer, and estimates size as the conversation’s JSON length divided by chars_per_token.

Leaf	Default	What it does	When to change it
strategy	sliding_window	sliding_window drops the oldest tool results first. tiered cuts in order as thresholds pass: retry nudges, then truncated tool results, then dropped tool results, then the model’s reasoning text. none never compacts	none when early evidence stays relevant; tiered for a small window
budget_tokens	6000	The estimated size above which compaction runs. Positive	Match it to the model’s window, less room for the prompt, tools and reply
keep_recent	0	How many of the latest loop iterations are never compacted. Zero or more	When the model needs its last few observations intact
phase_thresholds	[0.6, 0.75, 0.9]	tiered only: the budget fractions at which the three cuts start. Exactly three, strictly ascending, between 0 and 1. An empty list restores the default	Earlier for a small window, later when early evidence matters
warn_at	none	Budget fractions at which the model is told, once each, that context is running low. Strictly ascending, between 0 and 1. An empty list means no warnings	When the model should start wrapping up before compaction
chars_per_token	4	The divisor of the size estimate. Positive	2 or 3 for CJK text, code-heavy output, or a tokenizer that splits finely
compact_truncate_chars	200	tiered only: the length in bytes older tool results are cut to at the second threshold. Positive	Raise it if the model loses track of earlier evidence

max_tokens: reply caps

Override: KASM_RUNTIME_MAX_TOKENS_<LEAF>. The pair in force is also recorded at agent.vision.max_tokens.

Leaf	Default	What it does	When to change it
judge	2000	The reply cap for a screenshot judgement or grounding call. A thinking model that hits the cap can return an empty reply. Positive	Raise it when judgements come back empty
tools	500	The reply cap for a tool-call turn. Positive	Raise it when calls arrive truncated, or respond messages are cut off

model: the model server connection

Override: KASM_RUNTIME_MODEL_<LEAF>. Recorded at agent.vision.model_transport.

Leaf	Default	What it does	When to change it
timeout_s	90	Seconds one request may take before it counts as a failed try. Positive	Raise it for hosted or thinking models
attempts	3	Tries per call, the first included. Server errors, throttling, timeouts, gateway errors, transport errors and empty judgement replies are retried; a bad request is not. At least 1	Raise it on a flaky network; 1 to measure raw reliability
backoff_s	2	The pause before retry n is n times this. A Retry-After header is honoured, up to a minute. Zero or more	Raise it for a rate-limited API; 0 for a local server
disable_after	2	How many judgement calls in a row may fail every try before the runtime stops using the model for the rest of the run and finishes deterministic-only. At least 1	Raise it for a shared server with occasional bad minutes

baseline: acceptable visual drift

Override: KASM_RUNTIME_BASELINE_<LEAF>. When a good-* image exists in KASM_RUNTIME_BASELINE, the post-launch screenshot is compared with it by a 64-bit perceptual hash. The distance is the count of differing bits: 0 is identical.

Leaf	Default	What it does	When to change it
similar_max	10	The largest distance labelled similar. Anything above it is also a run concern. 0 to 63	Lower it where small changes matter; raise it for screens that vary, such as feeds or clocks
drifted_max	20	The largest distance labelled drifted. Above it the label is very-different. 0 to 64, and at least similar_max. An inconsistent pair falls back to the last consistent one	Raise it with similar_max for busy or animated apps

To turn the comparison off, leave KASM_RUNTIME_BASELINE without a good-* image.

findings: window titles that count as a problem

Override: KASM_RUNTIME_FINDINGS_<LEAF>, comma-separated. Matching ignores case and matches anywhere in a window’s title or class. A list you set replaces the shipped one; an empty list turns that finding off.

Leaf	Default	What it does	When to change it
updater_markers	updater, checking for update, downloading update, installing update, update available, software update	Reports an in-app updater running at launch	Add your app’s wording; empty it for an app whose window always says “Update”
first_run_markers	keyring, choose password, gcr-prompt, gnome-keyring, legal notice	Reports a first-run prompt blocking the app. Reported, never dismissed	Add licence or welcome wording; translate for a non-English desktop
error_dialog_markers	error, crash, segfault, problem report	Reports a window other than the app’s own as an error dialog	Translate for a non-English desktop; trim when an ordinary window contains one of the words

timing: the runtime’s own waits

Override: KASM_RUNTIME_TIMING_<SUB>_<LEAF>. These are pauses the runtime takes on its own account. A scenario’s settle_s, a timeout_s or a wait_while condition is separate and wins for its own step. Millisecond pauses are zero or more; poll intervals (poll_ms, post_poll_ms) are at least 1; both are at most 24 hours. Seconds are positive unless the row says zero is allowed. Counts are at least 1.

timing.input

Leaf	Default	What it does	When to change it
activate_settle_ms	150	Pause after focusing a window before input follows	Raise it when keystrokes land in the wrong window
keystroke_ms	20	Gap between typed characters	Raise it when characters go missing
click_ms	50	Pause between pointer move and press, and between press and release	Raise it when clicks are ignored or read as drags
chord_settle_ms	50	Pause after a key combination is released	Raise it when the next step looks too early

timing.settle

Leaf	Default	What it does	When to change it
poll_ms	150	Gap between compared frames while waiting for repainting to stop. Clamped to max_s if longer	Lower it on a fast host
max_changed_fraction	0.001	Fraction of pixels that may differ between frames and still count as settled. Above 0 and below 1	Raise it for a spinner or clock that never settles
max_s	2	Ceiling on one settle wait	Raise it for slow repaints
post_poll_ms	250	How often a deterministic postcondition is re-checked	Raise it under heavy host load
kill_settle_ms	500	Pause after signalling a process, so its windows close	Raise it when no_window fails right after a kill

timing.cdp

Leaf	Default	What it does	When to change it
dial_s	15	Time for each stage of attaching to the browser: discovery, then the WebSocket	Raise it when the browser is still starting
call_s	25	Time for one protocol command	Raise it for heavy pages
resolve_s	5	Time to find an element before reporting it not found	Raise it for slow single-page apps
poll_ms	250	Re-query interval inside find, wait and navigate loops	Lower it to react faster
default_wait_s	30	Ceiling on a browser wait or navigation that gave no timeout	Raise it for slow sites
read_limit_mib	16	Largest single message accepted from the browser, at most 4096	Raise it when large pages drop the connection

timing.atspi

Leaf	Default	What it does	When to change it
probe_budget_s	50	Ceiling on the start-up probe of the app’s accessible elements	Lower it for apps with no accessibility support
probe_app_wait_s	20	How long the probe waits for the app to appear on the bus	Raise it for large apps on slow hosts
app_wait_after_enable_s	30	The wait used when the runtime had to switch accessibility on itself	Rarely
action_budget_s	120	Ceiling on one accessibility action, such as click_element or a state check	Lower it for short deadlines
action_app_wait_s	90	How long an action waits for the app on the bus	Raise it on loaded hosts
max_nodes	1500	Most elements walked in an app’s tree	Raise it for very rich apps
max_depth	16	How deep the walk goes	Raise it when deeply nested controls are missing
max_actionable	150	Most actionable elements reported and offered to the model	Raise it for apps with hundreds of controls

timing.lifecycle

Leaf	Default	What it does	When to change it
run_boot_timeout_s	60	Under run, how long to wait for the X display	Raise it for heavy images
mcp_boot_timeout_s	30	Under mcp, the same wait	Raise it when mcp starts during container boot
wrap_up_reserve_s	30	The end of the deadline kept for diagnosis and the report. Zero allowed. Must be below the deadline, or it is reset	Raise it for a slow judge; 0 for deterministic-only runs
window_only_grace_s	10	How long to keep looking for the process once the window has appeared, before accepting the window alone. Zero allowed	Raise it for apps whose process appears after a splash window
crash_recheck_dwell_s	3	Least time after first sighting before re-checking the process is alive	Raise it for apps that crash a few seconds in
crash_rescan_s	2	Pause between the two process scans that detect a crash loop. Zero allowed	Raise it for slow supervisors

evidence: evidence sizes

Override: KASM_RUNTIME_EVIDENCE_<LEAF>. Every leaf is positive; jpeg_quality is 1 to 100. Larger values give more detail and cost more prompt and disk.

Leaf	Default	What it does	When to change it
shell_output_chars	2000	Bytes of a declared command’s output kept	Raise it when the useful line is cut off
tool_result_chars	300	Bytes of each tool result, and of the model’s text and arguments, kept in the report and trace. The model sees the full result	Raise it when step rows are too short to read, or compile-plan reads truncated arguments
page_text_chars	4000	Page text given to the model with a browser attached, in UTF-16 code units	Raise it for long pages
page_elements	20	Interactive elements listed with that text	Raise it for busy pages
element_text_chars	30	Text kept per listed element, in UTF-16 code units	Raise it for long labels
listed_candidates	20	Elements named in a not-found or ambiguous-match error	Raise it when the wanted control is not in the list
accessible_name_chars	260	Bytes of one element’s accessible name in a browser listing	Raise it for long names that differ near the end
log_tail_lines	200	Trailing lines of each log collected, written to logs/, and scanned by checks.log_fatal and rules:	Raise it when a fatal line sits further back
report_log_lines	40	How many of those lines report.md shows	Raise it for more context in the report
jpeg_quality	85	Quality of saved screenshots and of the square-padded image sent to the model	Lower it to save disk; raise it for small text
trace_replay_events	4096	Recent trace events kept in memory for a late sidebar to catch up	Raise it for very long runs
inbox_message_chars	4000	Most characters in one inbox message. Longer ones are refused with HTTP 413 and never truncated	Raise it for pasted instructions or logs
inbox_attachment_bytes	8388608	Most bytes in one inbox attachment (8 MiB)	Raise it for large documents
inbox_context_chars	16000	Most inbox text the agent loop keeps in the model’s context at once	Raise it with context.budget_tokens
inbox_summary_chars	2000	Size of the summary that older inbox messages are folded into	Rarely
inbox_drain_max	8	Pending inbox messages one model request may take	Lower it so bursts are answered in turn

tools: built-in tools the model is never offered

Override: KASM_RUNTIME_TOOLS_DISABLED, comma-separated. This narrows what the runtime’s own agent loop can call. A scenario’s declared tools are never affected.

Leaf	Default	What it does	When to change it
disabled	empty	Built-in tool names never offered to the model. An unknown name, or one whose removal strands a tool that depends on it, stops the run before the app launches	To withhold tools a goal does not need, or one a model misuses

The built-in names are observe_screen, press_key, click, type_text, wait, check_expectation, record_verdict, escalate, ask_human, respond, click_grounded, click_element, wait_while, cdp_url, cdp_wait, cdp_query, cdp_click, cdp_fill, cdp_list_interactive, cdp_click_described. The input tools require observe_screen first, so observe_screen can be disabled only together with press_key, click, type_text and click_grounded.

Next

• Environment variables
• Report
• Vision models