agent window: screen directions, pointing ends the turn
Deploy web Studio / deploy (push) Successful in 3m33s

Found while recording a demo with Qwen3.8 on the inference server:

- When the model wrote its answer and then called select_object, it was
  asked again with nothing left to say and rambled ("the user hasn't sent
  a message yet"), sometimes in Chinese. A step of page tools only
  (select_object, set_view) after a written answer now ends the turn; the
  system prompt says so, and to answer in the language of the latest message.
- The system prompt names the world directions of the user's screen right
  and up, and the 2:1 foreshortening, so "left of the hall" and shapes meant
  to be read on screen come out right (asked for a giant W of people, the
  model now draws it upright and readable).
- mock_llm.py "show me X" + check: answer and select_object in one step end
  the turn without another request, and the history stays valid.
This commit is contained in:
grabowski committed 2026-10-23 20:29:17 +07:00
1 parent 36f7a9f440
commit 84309a7a08
3 files changed
+34 -3

No files matched your search

+8
View File
@@ -7,6 +7,8 @@ Wants the key `test-key`. Answers like a model that uses tools:
list_objects {type: tree} and add_object {type: tree, x: X, y: Y, name: "Agent tree"}
- tool results -> a short answer naming the new object's id in backticks
- a user message containing "slow" -> a long, slow answer (to test Stop)
- a user message "show me X" -> an answer and a select_object call for X in one step
(that ends the turn: no request follows)
- anything else -> a plain answer
Streams (SSE) when asked to. Every page origin may call it (CORS, also the
preflight). GET /_requests lists the requests seen (headers and bodies).
@@ -76,6 +78,12 @@ class H(BaseHTTPRequestHandler):
if last.get("role") == "user" and "slow" in last.get("content", ""):
deltas = [{"content": "word%d " % i} for i in range(80)]
return self.stream(body, deltas, finish, pause=0.1)
show = re.search(r"show me (\S+)", last.get("content", "")) if last.get("role") == "user" else None
if show:
deltas = [{"content": "Here is `%s`." % show.group(1)},
{"tool_calls": [{"index": 0, "id": "call_s", "type": "function",
"function": {"name": "select_object", "arguments": json.dumps({"id": show.group(1)})}}]}]
return self.stream(body, deltas, "tool_calls")
m = re.search(r"tree at (-?[\d.]+),\s*(-?[\d.]+)", last.get("content", "")) if last.get("role") == "user" else None
if m:
x, y = float(m.group(1)), float(m.group(2))