studio: agent window, a chat with a model that edits the scene
Deploy web Studio / deploy (push) Successful in 3m3s

✦ Agent in the Studio's top bar opens a floating chat window. The model
(any OpenAI-compatible endpoint with tool calling; default B4L's inference
server, https://inference.p0.contact/v1) gets the 15 MCP tools plus two page
tools (select_object, set_view) and the Studio's state (selection, level,
rotation) in its system prompt. Answers and reasoning stream in; tool calls
show as rows with arguments and results; object ids link to the object;
Stop ends a turn; Undo takes the agent's edits back. The window moves by
its title bar and resizes at its top-left corner.

- studio/agent.js: shared by both front ends (waits for studio.js's
  isokit:studio event); MCP over /mcp or isokit.mcp in the web Studio
- isokit studio: /api/agent/{config,models,chat} relay the calls on their
  own threads (ureq, chunked and flushed per read), so the key stays on the
  machine and the endpoint needs no CORS; settings from ISOKIT_LLM_URL /
  _KEY / _MODEL or agent.json in the user's config folder
- web Studio: calls the endpoint from the page with a key kept in the
  browser; build_web.py --agent-url names a same-site relay as default; the
  deploy builds with /llm/v1, which deploy/Caddyfile forwards to the
  inference server (its nginx refuses the CORS preflight)
- tools/mock_llm.py + tools/agent_checks.py: the agent window checks in both
  check_studio.py and check_web.py
This commit is contained in:
grabowski committed 2026-10-23 19:35:53 +07:00
1 parent c647f39059
commit 1a1b62d475
21 files changed
+1690 -25

No files matched your search

+37 -4
View File
@@ -9,6 +9,8 @@ click for details. One Rust binary does everything:
- **Web Studio**: the same Studio running entirely in the browser, with the
engine compiled to WebAssembly. Nothing to install: projects live in the
browser, or in a folder on disk (Chrome / Edge) that agents edit alongside.
- **Agent window**: chat with a language model right in the Studio (by default
B4L's inference server); it edits the scene with the MCP tools while you watch.
- **MCP server**: AI agents (Hermes and others) edit the same scene with the
same operations; their changes show up live in an open Studio.
- **Build**: a self-contained HTML page with every view prerendered, or
@@ -80,6 +82,33 @@ MCP tools also run inside the page: `await isokit.mcp({jsonrpc: "2.0", id: 1,
method: "tools/list"})`, and browsers with WebMCP (`navigator.modelContext`)
get them as page tools.
### Agent window
**✦ Agent** in the Studio's top bar opens a chat with a language model that
edits the scene for you: "plant a row of trees along the road", "make the
workshop one storey taller", "what is on this site?". It works with the 15 MCP
tools below plus two page tools (point at an object, turn the view), streams
its answer and its reasoning, and shows each tool call (click one for the
arguments and the result, click an object id to select it). Every edit lands
on the map at once, and Undo takes it back. Stop ends a turn; the window moves
by its title bar and resizes at its top-left corner.
Any OpenAI-compatible endpoint with tool calling works (vLLM, llama.cpp,
Ollama, LM Studio, hosted APIs). The default is B4L's inference server,
`https://inference.p0.contact/v1`; ⚙ sets endpoint, key, model and whether
the model thinks first.
- `isokit studio` passes the calls through its own server (`/api/agent/*`), so
the key never reaches the page and the endpoint needs no CORS. Settings:
`ISOKIT_LLM_URL`, `ISOKIT_LLM_KEY`, `ISOKIT_LLM_MODEL`, or the window's ⚙,
which saves them in `%APPDATA%\isokit\agent.json` (`~/.config/isokit` elsewhere).
- The web Studio calls the endpoint from the page; the key stays in that
browser. The endpoint must allow the page (CORS, preflight included), or the
site relays it: https://isokit.buildfor.life/ is built with
`--agent-url /llm/v1`, a same-site path that Caddy forwards to B4L's
inference server (`deploy/Caddyfile`). The relay adds no key: each browser
enters its own once.
### Agents (MCP)
While the Studio runs, it also serves MCP over HTTP at `http://127.0.0.1:7878/mcp`.
@@ -127,7 +156,7 @@ storey.
| Command | |
|---|---|
| `isokit new <dir>` | starter project |
| `isokit studio [dir] [--port N] [--no-open]` | GUI editor + `/mcp` + `/view` live preview |
| `isokit studio [dir] [--port N] [--no-open]` | GUI editor + `/mcp` + `/view` live preview + the agent window's link to the model |
| `isokit mcp [dir]` | MCP over stdio |
| `isokit build [dir] [-o file]` | self-contained HTML (all levels × 4 rotations) |
| `isokit render [dir] -o x.svg [--level gf] [--rot 2] [--light]` | one view as SVG |
@@ -144,7 +173,7 @@ crates/isokit-project project folder, document ops, export, check, Workspace (u
crates/isokit-cli the `isokit` binary: Studio HTTP server, MCP stdio, build/render/check
crates/isokit-wasm the engine for the browser (wasm-bindgen): a project in memory behind the Studio API
viewer/ isokit-view.js/.css: pan, zoom, levels, rotation, tooltips (any page can embed it)
studio/ the Studio GUI (served by `isokit studio`, and by the web Studio)
studio/ the Studio GUI (served by `isokit studio`, and by the web Studio); agent.js = agent window
web/ web Studio shell: WASM backend for studio.js, projects, folders, zip, code editor
examples/demo demo project
deploy/Caddyfile the web Studio's vhost on the buildfor.life web server (s1)
@@ -165,12 +194,16 @@ Plans: [docs/ROADMAP.md](docs/ROADMAP.md).
```bash
cargo test # engine, runtime, ops, demo, MCP
cargo clippy --all-targets -- -D warnings
isokit studio examples/demo --port 7890 --no-open &
uv run --with playwright python tools/check_studio.py <shot-dir> http://127.0.0.1:7890 # GUI + MCP end to end (Edge)
ISOKIT_LLM_URL=http://127.0.0.1:7932/v1 ISOKIT_LLM_KEY=test-key ISOKIT_CONFIG_DIR=<scratch> \
isokit studio <copy of examples/demo> --port 7890 --no-open & # agent window on tools/mock_llm.py
uv run --with playwright python tools/check_studio.py <shot-dir> http://127.0.0.1:7890 # GUI + MCP + agent window end to end (Edge)
cargo clippy -p isokit-wasm --target wasm32-unknown-unknown -- -D warnings
python tools/build_web.py && uv run --with playwright python tools/check_web.py <shot-dir> # web Studio end to end
```
The agent window checks run on `tools/mock_llm.py`, a scripted stand-in for an
OpenAI-compatible server that the check scripts start on port 7932.
`python tools/build_web.py --skip-cargo` re-copies the front end (studio/,
viewer/, web/) without rebuilding the WASM. Code that runs in the browser must
not use `std::time` (it traps on wasm32-unknown-unknown): use `web_time`.