Merge main and move the Codex compatibility work onto main's tests/e2e/codex
suite, replacing the parallel tests/e2e/client_apps suite and its harness tests.
A makereport hook in codex/compat_results.py appends one JSON line per decided
cell when CODEX_COMPAT_RESULTS_PATH is set, and codex/matrix_builder.py builds a
table in the Claude Code matrix shape and exits 3 when a cell that passed no
longer does. The row id comes from the test file name and the label from the
class name, so a new row needs no other edit.
The suite now pins @openai/codex 0.166.0 in its own package.json and lockfile,
adds Gemini on Vertex AI as a setup, and adds prompt caching, reasoning effort,
vision, structured output and web search rows. Web search carries strict xfails
where Codex or LiteLLM leaves the search tool out, and auto-compaction now also
expects Vertex AI to fail, for the same bridge gap Claude on Bedrock has
* test(e2e): add the Codex file edit and long session rows
The file edit row has Codex change a random value into config.txt with the patch tool and checks the file on disk. The long session row has it follow a chain of random file names one cat at a time. new_workspace now returns the resolved path, because Codex reports patched files by their real path (/private/var on macOS).
* test(e2e): make the Codex long session prove every earlier command output reached the model
* test(e2e): Codex auto-compaction through LiteLLM (#49389)
* test(e2e): add the Codex auto-compaction row
Codex reads a random value, compacts through the gateway with its remote compaction forced on, and is asked for the value after the file holding it is deleted. The row reads the compaction from the thread's rollout file and marks Claude on Bedrock as a strict expected failure, since the Responses-to-chat bridge ignores the compaction_trigger item
* refactor(e2e): validate Codex rollout lines with the models directly
* test(e2e): drive the real Codex CLI through the gateway and add the command round trip row
Adds tests/e2e/codex: a harness that runs codex exec --json with a throwaway HOME and CODEX_HOME, a custom responses provider pointed at the gateway, the model list read from the gateway's /v1/models, the sandbox on and approvals off. Each test registers its setup's deployment via /model/new and calls it with a key scoped to it, both deleted on teardown. The first row checks that a value only a command can print comes back through the gateway on all five setups
* test(e2e): type the Codex self-test event helper as a Mapping
* test(e2e): clean up Codex test keys even when setup fails, and fail runs where Codex exits with an error
* test(e2e): record a step when the Codex harness sets up its home and workspace
The suite rules ask every public harness helper that does something to record a step, so a failed run shows whether it died while setting up Codex or while running it
* Update run.py
wip