test(e2e): run the client-apps OpenAI cells on gpt-5.6 and accept a Codex preamble message

This commit is contained in:
mateo-berri 2026-09-22 12:42:11 -07:00
parent 348ecf6556
commit e95ed75614
2 changed files with 5 additions and 4 deletions

View file

@ -42,7 +42,7 @@ def anthropic_model(proxy: ProxyClient) -> Iterator[str]:
@pytest.fixture(scope="session")
def openai_model(proxy: ProxyClient) -> Iterator[str]:
yield from _suite_model(proxy, "openai/gpt-5.5", "OPENAI_API_KEY")
yield from _suite_model(proxy, "openai/gpt-5.6", "OPENAI_API_KEY")
def _suite_model(proxy: ProxyClient, provider_model: str, env_var: str) -> Iterator[str]:

View file

@ -2,7 +2,8 @@
`codex exec --json` turn against the proxy on a virtual key through a custom
model provider (`wire_api = "responses"`, so `/v1/responses`): it runs a shell
command, reads its output, and answers. Client side the JSONL events carry the
completed command execution and the agent message; proxy side the spend rows
completed command execution and the agent message (the last one: Codex sometimes
sends a one-line preamble before running the command); proxy side the spend rows
carry real spend and `stream: true` (Codex always streams the Responses API).
The echoed word is unique per run: the CI stack caches responses, and an
identical prompt across runs would be served a $0 cache-hit row instead of a
@ -47,8 +48,8 @@ class TestCodexCli:
assert word in commands[0].aggregated_output, commands[0]
messages = [item for item in completed_items if isinstance(item, CodexAgentMessage)]
assert len(messages) == 1, events
assert word in messages[0].text, messages[0]
assert messages, events
assert word in messages[-1].text, messages
turns = [event for event in events if isinstance(event, CodexTurnCompleted)]
assert len(turns) == 1, events