Kasm Session Runtime
Any application · Any model · Observable · Repeatability & reliability
Kasm Session Runtime is a static binary that runs inside any Kasm session as the session user and drives real applications. A scripted scenario, an agent you already use, or the runtime’s own loop with a vision model can each drive a session. Traceability is a first-class concern, so every run leaves a report, screenshots, logs and a verdict. If you provide an optional vision model it adds visual judgement, vision-guided clicks and the runtime’s own agent loop. See When you need a model.
Why we built this
Section titled “Why we built this”We needed AI automation that is reliable, repeatable, and cheap enough to run in our CI/CD pipelines. The agents and computer-use tools we tried work a task out once, with a large model reasoning on every step.
- The model should only do what a script can’t. Scripted steps, browser
control and the check after each step run with no model. A
goal:hands one stretch of a task to the runtime’s loop, and a run the loop worked out compiles back into plain steps with their checks. Mixing the two gives the flexibility of an agent with the model’s part kept small: in our measurement 8 of 8 compiled plans replayed with no model call. See Plans and postconditions. - It has to run on models we can afford. The loop completed a real web task, 9 of 9 categories, on a 12B model served from one consumer GPU, and screenshots can be judged by a 4B model on a CPU. A replayed plan needs no model at all. See Vision models.
- Sometimes you have to watch the agent work. A Kasm session is an isolated container with a real desktop. A person watches from the browser, sees each step as it runs, can message the agent and can stop it. See People in the loop.
- Air-gapped and closed networks need automation too. The runtime talks to any OpenAI-compatible endpoint, including one on your own GPU, so a whole run can finish inside the network. See Vision models.
- A pass has to be one we can trust. Window and process probes decide whether the application launched and stayed up. A model can fail a check the scenario asked it to make, and it cannot overrule a failure. See What the model can do.
- We have to know what the agent actually did. Every step in the report names the backend that ran it and, for clicks, how the coordinates were resolved. Secrets from the environment are redacted from every text file the run writes. Screenshots are not redacted. See Secrets and evidence.
- Not everything we test is a web page. The runtime uses the Chrome DevTools Protocol where there is a browser, the AT-SPI accessibility bus where an application publishes one, and X11 input where there is only a screen. Our image test bench runs 26 applications through it on a CPU runner in CI. See How an action reaches the app.
- Teams want to keep the agent they already use. Claude Code, Codex or your own agent drives a Kasm session over MCP, with no model configured in the runtime. See Using it with your own agent.
Research we drew on
Section titled “Research we drew on”- Forge (Zambelli): guardrails around tool calling that let small local models complete agentic workflows.
- Agent JIT compilation (Winston, Wang, Mirhoseini and Kozyrakis): postconditions and compiled plans.
- D2Snap (Schiepanski and Piël): DOM downsampling for web agents, which we measured against the runtime’s own element listing and did not adopt.