studio: agent window, a chat with a model that edits the scene
Deploy web Studio / deploy (push) Successful in 3m3s
Deploy web Studio / deploy (push) Successful in 3m3s
✦ Agent in the Studio's top bar opens a floating chat window. The model (any OpenAI-compatible endpoint with tool calling; default B4L's inference server, https://inference.p0.contact/v1) gets the 15 MCP tools plus two page tools (select_object, set_view) and the Studio's state (selection, level, rotation) in its system prompt. Answers and reasoning stream in; tool calls show as rows with arguments and results; object ids link to the object; Stop ends a turn; Undo takes the agent's edits back. The window moves by its title bar and resizes at its top-left corner. - studio/agent.js: shared by both front ends (waits for studio.js's isokit:studio event); MCP over /mcp or isokit.mcp in the web Studio - isokit studio: /api/agent/{config,models,chat} relay the calls on their own threads (ureq, chunked and flushed per read), so the key stays on the machine and the endpoint needs no CORS; settings from ISOKIT_LLM_URL / _KEY / _MODEL or agent.json in the user's config folder - web Studio: calls the endpoint from the page with a key kept in the browser; build_web.py --agent-url names a same-site relay as default; the deploy builds with /llm/v1, which deploy/Caddyfile forwards to the inference server (its nginx refuses the CORS preflight) - tools/mock_llm.py + tools/agent_checks.py: the agent window checks in both check_studio.py and check_web.py
This commit is contained in:
1 parent
c647f39059
commit
1a1b62d475
21 files changed
+1690
-25
No files matched your search
+15
-2
@@ -7,8 +7,9 @@ Serves the build on a local port and checks: the demo loads from the bundled
|
||||
files, placing / dragging / props / undo, persistence across reloads, the code
|
||||
editor (script error and repair), new / export / import (.zip, stored and
|
||||
deflated), page build and preview, MCP in the page and WebMCP, two tabs on one
|
||||
project, and a folder project (the origin-private file system stands in for a
|
||||
folder picked with showDirectoryPicker: same handle API) following outside edits.
|
||||
project, a folder project (the origin-private file system stands in for a
|
||||
folder picked with showDirectoryPicker: same handle API) following outside
|
||||
edits, and the agent window on a mock model (tools/mock_llm.py, started here).
|
||||
"""
|
||||
import io
|
||||
import json
|
||||
@@ -20,6 +21,8 @@ from pathlib import Path
|
||||
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
import agent_checks
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
OUT = Path(sys.argv[1]) if len(sys.argv) > 1 else ROOT / "dist" / "web-shots"
|
||||
DIST = Path(sys.argv[2]) if len(sys.argv) > 2 else ROOT / "dist" / "web"
|
||||
@@ -60,6 +63,7 @@ WEBMCP_STUB = "navigator.modelContext = { provideContext: function (c) { window.
|
||||
|
||||
srv = subprocess.Popen([sys.executable, "-m", "http.server", str(PORT), "--bind", "127.0.0.1", "--directory", str(DIST)],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
mock = agent_checks.start_mock()
|
||||
time.sleep(1.0)
|
||||
try:
|
||||
with sync_playwright() as p:
|
||||
@@ -186,6 +190,12 @@ try:
|
||||
return t.execute({id: "agent-lamp", tag: "via WebMCP"}); }""")
|
||||
check(r["isError"] is False and obj(pg, "agent-lamp")["tag"] == "via WebMCP", "a WebMCP tool call edits the scene")
|
||||
|
||||
# ---------------------------------------------------------------- agent window (mock model, key kept in the browser)
|
||||
pg.evaluate("(u) => localStorage.setItem('isokit-agent', JSON.stringify({url: u, key: 'test-key'}))", agent_checks.MOCK + "/v1")
|
||||
agent_checks.run(pg, lambda: doc(pg)["objects"], lambda n: shot(pg, "web-" + n), check, can_set_key=True)
|
||||
stored = pg.evaluate("() => JSON.parse(localStorage.getItem('isokit-agent'))")
|
||||
check(stored.get("key") == "test-key" and not stored.get("model"), f"settings stay in this browser {stored}")
|
||||
|
||||
# ---------------------------------------------------------------- two tabs on one project
|
||||
pg2 = ctx.new_page()
|
||||
watch(pg2)
|
||||
@@ -309,10 +319,13 @@ try:
|
||||
pg.wait_for_timeout(400)
|
||||
shot(pg, "web-narrow.jpg")
|
||||
|
||||
# the browser logs the 401 of the agent window's wrong-key check
|
||||
errs = [e for e in errs if "status of 401" not in e]
|
||||
check(not errs, f"no page errors {errs[:5]}")
|
||||
b.close()
|
||||
finally:
|
||||
srv.terminate()
|
||||
mock.terminate()
|
||||
|
||||
print("\n%d failure(s)" % len(fails))
|
||||
sys.exit(1 if fails else 0)
|
||||
Reference in new issue
Block a user