Found while recording a demo with Qwen3.8 on the inference server:
- When the model wrote its answer and then called select_object, it was
asked again with nothing left to say and rambled ("the user hasn't sent
a message yet"), sometimes in Chinese. A step of page tools only
(select_object, set_view) after a written answer now ends the turn; the
system prompt says so, and to answer in the language of the latest message.
- The system prompt names the world directions of the user's screen right
and up, and the 2:1 foreshortening, so "left of the hall" and shapes meant
to be read on screen come out right (asked for a giant W of people, the
model now draws it upright and readable).
- mock_llm.py "show me X" + check: answer and select_object in one step end
the turn without another request, and the history stays valid.
✦ Agent in the Studio's top bar opens a floating chat window. The model
(any OpenAI-compatible endpoint with tool calling; default B4L's inference
server, https://inference.p0.contact/v1) gets the 15 MCP tools plus two page
tools (select_object, set_view) and the Studio's state (selection, level,
rotation) in its system prompt. Answers and reasoning stream in; tool calls
show as rows with arguments and results; object ids link to the object;
Stop ends a turn; Undo takes the agent's edits back. The window moves by
its title bar and resizes at its top-left corner.
- studio/agent.js: shared by both front ends (waits for studio.js's
isokit:studio event); MCP over /mcp or isokit.mcp in the web Studio
- isokit studio: /api/agent/{config,models,chat} relay the calls on their
own threads (ureq, chunked and flushed per read), so the key stays on the
machine and the endpoint needs no CORS; settings from ISOKIT_LLM_URL /
_KEY / _MODEL or agent.json in the user's config folder
- web Studio: calls the endpoint from the page with a key kept in the
browser; build_web.py --agent-url names a same-site relay as default; the
deploy builds with /llm/v1, which deploy/Caddyfile forwards to the
inference server (its nginx refuses the CORS preflight)
- tools/mock_llm.py + tools/agent_checks.py: the agent window checks in both
check_studio.py and check_web.py
grabowski