agent window: screen directions, pointing ends the turn
Deploy web Studio / deploy (push) Successful in 3m33s
Deploy web Studio / deploy (push) Successful in 3m33s
Found while recording a demo with Qwen3.8 on the inference server:
- When the model wrote its answer and then called select_object, it was
asked again with nothing left to say and rambled ("the user hasn't sent
a message yet"), sometimes in Chinese. A step of page tools only
(select_object, set_view) after a written answer now ends the turn; the
system prompt says so, and to answer in the language of the latest message.
- The system prompt names the world directions of the user's screen right
and up, and the 2:1 foreshortening, so "left of the hall" and shapes meant
to be read on screen come out right (asked for a giant W of people, the
model now draws it upright and readable).
- mock_llm.py "show me X" + check: answer and select_object in one step end
the turn without another request, and the history stays valid.
This commit is contained in:
1 parent
36f7a9f440
commit
84309a7a08
3 files changed
+34
-3
No files matched your search
@@ -81,6 +81,17 @@ def run(pg, objects, shot, check, can_set_key):
|
||||
pg.wait_for_function("() => !document.querySelector('.ag-busy')", timeout=10000)
|
||||
check(('"id":"%s"' % tid) in chats()[-1]["body"]["messages"][0]["content"], "the system prompt names the selection")
|
||||
|
||||
# an answer that ends by pointing at an object ends the turn there
|
||||
n_chat = len(chats())
|
||||
pg.locator("#agtext").fill("show me " + tid)
|
||||
pg.locator("#agtext").press("Enter")
|
||||
pg.wait_for_function("() => !document.querySelector('.ag-busy')", timeout=10000)
|
||||
pg.wait_for_timeout(600)
|
||||
check(len(chats()) == n_chat + 1 and ("id " + tid) in pg.locator("#right .st-sub").inner_text(),
|
||||
"answer + select_object ends the turn (no extra request), the object is selected")
|
||||
tail = pg.evaluate("() => __agent.A.msgs.slice(-2).map(m => m.role + ':' + (m.tool_call_id || (m.tool_calls || []).length))")
|
||||
check(tail == ["assistant:1", "tool:call_s"], f"the history stays valid for the next request {tail}")
|
||||
|
||||
# Undo takes the agent's edit back
|
||||
pg.locator("#undo").click()
|
||||
pg.wait_for_timeout(700)
|
||||
|
||||
Reference in new issue
Block a user