Skip to content
KasmKasmKasm Session Runtime
Draft, pre-release documentation

When you need a model

Kasm Session Runtime does not need a model. Scripted steps, browser control, accessibility actions, the checks that decide pass or fail, and the MCP and HTTP interface for your own agent work with no model configured. A model adds judging what is on screen, finding an element on screen from a description, and the runtime’s own agentic loop.

The model endpoint

Section titled “The model endpoint”

The model is one endpoint, set with KASM_RUNTIME_VISION_ENDPOINT: any OpenAI-compatible chat-completions server, on your own hardware or hosted. The runtime sends it three kinds of request.

  • Judging screenshots: does the screen show the app, an error dialog, or what a step expected.
  • Locating an element on screen (grounding): where is “the Render menu”, as a coordinate to click.
  • Running the agentic loop for scenarios that give a goal: instead of steps.

With the variable empty, the runtime runs in deterministic-only mode. report.json records agent.deterministic_only: true, and report.md notes that no visual judgement was made.

Outside agents

Section titled “Outside agents”

When an agent such as Claude Code drives the session over MCP or HTTP, your agent does the reasoning. It takes screenshots with screen.capture, reads the page or the accessibility tree, and acts. Leave KASM_RUNTIME_VISION_ENDPOINT empty unless you want your agent to call the runtime’s own screen.judge, screen.ground or screen.ask. See Using it with your own agent.

Features with no model

Section titled “Features with no model”
Area Works without a model Needs a model
Scripted actions key, type, click center, click "<declared-target>", click <fx> <fy>, click_element, wait, wait_while, shell, spawn, kill click_grounded
Browser (CDP) Every cdp_* step, including cdp_wait Nothing
Checks after a step expect_state, and post: conditions of kind url, dom, state, window, no_window, process, no_process, file observe:, expect:, and post: conditions of kind screen
Pass or fail Window and process checks, checks.log_fatal, error-dialog windows found through X11, comparison with baseline screenshots Judging from the screenshot whether the app rendered or shows an error; the description of a failure screen
Goals and plans Replaying a plan: whose postconditions hold; compile-plan goal: scenarios; a plan’s fallback to the loop when a postcondition fails
MCP and HTTP screen.capture, windows.list, input.*, dom.* (including dom.click_described), a11y.*, proc.*, launch, shell, wait_while, file transfer, the /cdp/ proxy screen.judge, screen.ground, screen.ask
Evidence and people Screenshots, report.json, report.md, the trace, the sidebar, the inbox, input observation Nothing

dom.click_described and cdp_click_described match your description against the names of the page’s interactive elements as text, without a model.

Model features with no model configured

Section titled “Model features with no model configured”
  • An observe: or expect: step, or a screen postcondition, still takes its screenshot. The step’s row reads not-judged (deterministic-only mode) and the step does not fail.
  • A scripted click_grounded step cannot run, and the run ends as test-error: the runtime could not perform the test.
  • A goal: scenario runs its launch checks and setup steps, then skips the loop. The report notes that the loop was degraded and adds a concern that the goal was not verified. The deterministic checks decide the verdict, and the concern caps it at pass-with-concerns.
  • A plan: that completes passes on its postconditions without a model. If a plan step fails, the loop it hands over to is skipped as above.
  • screen.judge, screen.ground and screen.ask return an error saying the vision endpoint is unavailable.

Model failure during a run

Section titled “Model failure during a run”

Each model call is tried up to three times (model.attempts), with a pause between tries. After two judgement calls in a row fail on every try (model.disable_after), the runtime stops calling the model for the rest of the run and continues as if none were configured. Later judgements read as not judged. A grounded click that has already landed but cannot be checked is recorded as clicked and unverified, with a concern, and is not repeated. An agentic loop whose model call fails stops and is marked degraded.

In the report, agent.deterministic_only is true and report.md carries the note “vision endpoint failed repeatedly – degraded to deterministic-only”. An unreachable model does not by itself fail a run. Both settings are in the operator file; see Operator file.

Next

Section titled “Next”