studio: agent window, a chat with a model that edits the scene
Deploy web Studio / deploy (push) Successful in 3m3s
Deploy web Studio / deploy (push) Successful in 3m3s
✦ Agent in the Studio's top bar opens a floating chat window. The model (any OpenAI-compatible endpoint with tool calling; default B4L's inference server, https://inference.p0.contact/v1) gets the 15 MCP tools plus two page tools (select_object, set_view) and the Studio's state (selection, level, rotation) in its system prompt. Answers and reasoning stream in; tool calls show as rows with arguments and results; object ids link to the object; Stop ends a turn; Undo takes the agent's edits back. The window moves by its title bar and resizes at its top-left corner. - studio/agent.js: shared by both front ends (waits for studio.js's isokit:studio event); MCP over /mcp or isokit.mcp in the web Studio - isokit studio: /api/agent/{config,models,chat} relay the calls on their own threads (ureq, chunked and flushed per read), so the key stays on the machine and the endpoint needs no CORS; settings from ISOKIT_LLM_URL / _KEY / _MODEL or agent.json in the user's config folder - web Studio: calls the endpoint from the page with a key kept in the browser; build_web.py --agent-url names a same-site relay as default; the deploy builds with /llm/v1, which deploy/Caddyfile forwards to the inference server (its nginx refuses the CORS preflight) - tools/mock_llm.py + tools/agent_checks.py: the agent window checks in both check_studio.py and check_web.py
This commit is contained in:
1 parent
c647f39059
commit
1a1b62d475
21 files changed
+1690
-25
No files matched your search
+19
-1
@@ -1,10 +1,18 @@
|
||||
"""Functional test of the isokit Studio (GUI) and its MCP endpoint, headless Edge."""
|
||||
"""Functional test of the isokit Studio (GUI), its MCP endpoint and the agent
|
||||
window, headless Edge. The agent window runs on a mock model this script starts:
|
||||
|
||||
ISOKIT_LLM_URL=http://127.0.0.1:7932/v1 ISOKIT_LLM_KEY=test-key ISOKIT_CONFIG_DIR=<scratch> \
|
||||
isokit studio <copy of examples/demo> --port 7890 --no-open
|
||||
uv run --with playwright python tools/check_studio.py <screenshot dir> http://127.0.0.1:7890
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
import agent_checks
|
||||
|
||||
BASE = sys.argv[2] if len(sys.argv) > 2 else "http://127.0.0.1:7878"
|
||||
OUT = Path(sys.argv[1])
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
@@ -36,6 +44,7 @@ def obj(id_):
|
||||
return next((o for o in doc()["objects"] if o["id"] == id_), None)
|
||||
|
||||
|
||||
mock = agent_checks.start_mock()
|
||||
with sync_playwright() as p:
|
||||
b = p.chromium.launch(channel="msedge", headless=True)
|
||||
pg = b.new_page(viewport={"width": 1500, "height": 950})
|
||||
@@ -140,8 +149,17 @@ with sync_playwright() as p:
|
||||
pg.locator('[data-a="rotr"]').click()
|
||||
pg.wait_for_timeout(600)
|
||||
pg.screenshot(path=str(OUT / "studio-gf.jpg"), type="jpeg", quality=80)
|
||||
# the agent window, on the mock model (the Studio must run with ISOKIT_LLM_URL=<mock>/v1, see the docstring)
|
||||
cfg = get("/api/agent/config")
|
||||
if agent_checks.MOCK not in cfg["url"]:
|
||||
check(False, "agent window checks need the Studio started with ISOKIT_LLM_URL=" + agent_checks.MOCK + "/v1 (found " + cfg["url"] + ")")
|
||||
else:
|
||||
check(not cfg.get("key") and cfg["has_key"] and cfg["key_from"] == "env", "/api/agent/config says a key is set, without the key")
|
||||
pg.locator('.iso-seg button[data-level="site"]').click()
|
||||
agent_checks.run(pg, lambda: doc()["objects"], lambda n: pg.screenshot(path=str(OUT / n), type="jpeg", quality=80), check, can_set_key=False)
|
||||
check(not errs, f"no script errors {errs}")
|
||||
b.close()
|
||||
|
||||
mock.terminate()
|
||||
print("FAILED:" if fails else "ALL PASSED", fails)
|
||||
sys.exit(1 if fails else 0)
|
||||
Reference in new issue
Block a user