studio: agent window, a chat with a model that edits the scene
Deploy web Studio / deploy (push) Successful in 3m3s
Deploy web Studio / deploy (push) Successful in 3m3s
✦ Agent in the Studio's top bar opens a floating chat window. The model (any OpenAI-compatible endpoint with tool calling; default B4L's inference server, https://inference.p0.contact/v1) gets the 15 MCP tools plus two page tools (select_object, set_view) and the Studio's state (selection, level, rotation) in its system prompt. Answers and reasoning stream in; tool calls show as rows with arguments and results; object ids link to the object; Stop ends a turn; Undo takes the agent's edits back. The window moves by its title bar and resizes at its top-left corner. - studio/agent.js: shared by both front ends (waits for studio.js's isokit:studio event); MCP over /mcp or isokit.mcp in the web Studio - isokit studio: /api/agent/{config,models,chat} relay the calls on their own threads (ureq, chunked and flushed per read), so the key stays on the machine and the endpoint needs no CORS; settings from ISOKIT_LLM_URL / _KEY / _MODEL or agent.json in the user's config folder - web Studio: calls the endpoint from the page with a key kept in the browser; build_web.py --agent-url names a same-site relay as default; the deploy builds with /llm/v1, which deploy/Caddyfile forwards to the inference server (its nginx refuses the CORS preflight) - tools/mock_llm.py + tools/agent_checks.py: the agent window checks in both check_studio.py and check_web.py
This commit is contained in:
1 parent
c647f39059
commit
1a1b62d475
21 files changed
+1690
-25
No files matched your search
@@ -9,6 +9,8 @@ click for details. One Rust binary does everything:
|
||||
- **Web Studio**: the same Studio running entirely in the browser, with the
|
||||
engine compiled to WebAssembly. Nothing to install: projects live in the
|
||||
browser, or in a folder on disk (Chrome / Edge) that agents edit alongside.
|
||||
- **Agent window**: chat with a language model right in the Studio (by default
|
||||
B4L's inference server); it edits the scene with the MCP tools while you watch.
|
||||
- **MCP server**: AI agents (Hermes and others) edit the same scene with the
|
||||
same operations; their changes show up live in an open Studio.
|
||||
- **Build**: a self-contained HTML page with every view prerendered, or
|
||||
@@ -80,6 +82,33 @@ MCP tools also run inside the page: `await isokit.mcp({jsonrpc: "2.0", id: 1,
|
||||
method: "tools/list"})`, and browsers with WebMCP (`navigator.modelContext`)
|
||||
get them as page tools.
|
||||
|
||||
### Agent window
|
||||
|
||||
**✦ Agent** in the Studio's top bar opens a chat with a language model that
|
||||
edits the scene for you: "plant a row of trees along the road", "make the
|
||||
workshop one storey taller", "what is on this site?". It works with the 15 MCP
|
||||
tools below plus two page tools (point at an object, turn the view), streams
|
||||
its answer and its reasoning, and shows each tool call (click one for the
|
||||
arguments and the result, click an object id to select it). Every edit lands
|
||||
on the map at once, and Undo takes it back. Stop ends a turn; the window moves
|
||||
by its title bar and resizes at its top-left corner.
|
||||
|
||||
Any OpenAI-compatible endpoint with tool calling works (vLLM, llama.cpp,
|
||||
Ollama, LM Studio, hosted APIs). The default is B4L's inference server,
|
||||
`https://inference.p0.contact/v1`; ⚙ sets endpoint, key, model and whether
|
||||
the model thinks first.
|
||||
|
||||
- `isokit studio` passes the calls through its own server (`/api/agent/*`), so
|
||||
the key never reaches the page and the endpoint needs no CORS. Settings:
|
||||
`ISOKIT_LLM_URL`, `ISOKIT_LLM_KEY`, `ISOKIT_LLM_MODEL`, or the window's ⚙,
|
||||
which saves them in `%APPDATA%\isokit\agent.json` (`~/.config/isokit` elsewhere).
|
||||
- The web Studio calls the endpoint from the page; the key stays in that
|
||||
browser. The endpoint must allow the page (CORS, preflight included), or the
|
||||
site relays it: https://isokit.buildfor.life/ is built with
|
||||
`--agent-url /llm/v1`, a same-site path that Caddy forwards to B4L's
|
||||
inference server (`deploy/Caddyfile`). The relay adds no key: each browser
|
||||
enters its own once.
|
||||
|
||||
### Agents (MCP)
|
||||
|
||||
While the Studio runs, it also serves MCP over HTTP at `http://127.0.0.1:7878/mcp`.
|
||||
@@ -127,7 +156,7 @@ storey.
|
||||
| Command | |
|
||||
|---|---|
|
||||
| `isokit new <dir>` | starter project |
|
||||
| `isokit studio [dir] [--port N] [--no-open]` | GUI editor + `/mcp` + `/view` live preview |
|
||||
| `isokit studio [dir] [--port N] [--no-open]` | GUI editor + `/mcp` + `/view` live preview + the agent window's link to the model |
|
||||
| `isokit mcp [dir]` | MCP over stdio |
|
||||
| `isokit build [dir] [-o file]` | self-contained HTML (all levels × 4 rotations) |
|
||||
| `isokit render [dir] -o x.svg [--level gf] [--rot 2] [--light]` | one view as SVG |
|
||||
@@ -144,7 +173,7 @@ crates/isokit-project project folder, document ops, export, check, Workspace (u
|
||||
crates/isokit-cli the `isokit` binary: Studio HTTP server, MCP stdio, build/render/check
|
||||
crates/isokit-wasm the engine for the browser (wasm-bindgen): a project in memory behind the Studio API
|
||||
viewer/ isokit-view.js/.css: pan, zoom, levels, rotation, tooltips (any page can embed it)
|
||||
studio/ the Studio GUI (served by `isokit studio`, and by the web Studio)
|
||||
studio/ the Studio GUI (served by `isokit studio`, and by the web Studio); agent.js = agent window
|
||||
web/ web Studio shell: WASM backend for studio.js, projects, folders, zip, code editor
|
||||
examples/demo demo project
|
||||
deploy/Caddyfile the web Studio's vhost on the buildfor.life web server (s1)
|
||||
@@ -165,12 +194,16 @@ Plans: [docs/ROADMAP.md](docs/ROADMAP.md).
|
||||
```bash
|
||||
cargo test # engine, runtime, ops, demo, MCP
|
||||
cargo clippy --all-targets -- -D warnings
|
||||
isokit studio examples/demo --port 7890 --no-open &
|
||||
uv run --with playwright python tools/check_studio.py <shot-dir> http://127.0.0.1:7890 # GUI + MCP end to end (Edge)
|
||||
ISOKIT_LLM_URL=http://127.0.0.1:7932/v1 ISOKIT_LLM_KEY=test-key ISOKIT_CONFIG_DIR=<scratch> \
|
||||
isokit studio <copy of examples/demo> --port 7890 --no-open & # agent window on tools/mock_llm.py
|
||||
uv run --with playwright python tools/check_studio.py <shot-dir> http://127.0.0.1:7890 # GUI + MCP + agent window end to end (Edge)
|
||||
cargo clippy -p isokit-wasm --target wasm32-unknown-unknown -- -D warnings
|
||||
python tools/build_web.py && uv run --with playwright python tools/check_web.py <shot-dir> # web Studio end to end
|
||||
```
|
||||
|
||||
The agent window checks run on `tools/mock_llm.py`, a scripted stand-in for an
|
||||
OpenAI-compatible server that the check scripts start on port 7932.
|
||||
|
||||
`python tools/build_web.py --skip-cargo` re-copies the front end (studio/,
|
||||
viewer/, web/) without rebuilding the WASM. Code that runs in the browser must
|
||||
not use `std::time` (it traps on wasm32-unknown-unknown): use `web_time`.
|
||||
|
||||
Reference in new issue
Block a user