scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-15 09:12:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 311
litellm/tests/e2e_harness/codex
History
Exact
mateo-berri fa992071f3 test(e2e): build a Codex compatibility table from tests/e2e/codex and add Vertex AI and five rows 
Merge main and move the Codex compatibility work onto main's tests/e2e/codex
suite, replacing the parallel tests/e2e/client_apps suite and its harness tests.

A makereport hook in codex/compat_results.py appends one JSON line per decided
cell when CODEX_COMPAT_RESULTS_PATH is set, and codex/matrix_builder.py builds a
table in the Claude Code matrix shape and exits 3 when a cell that passed no
longer does. The row id comes from the test file name and the label from the
class name, so a new row needs no other edit.

The suite now pins @openai/codex 0.166.0 in its own package.json and lockfile,
adds Gemini on Vertex AI as a setup, and adds prompt caching, reasoning effort,
vision, structured output and web search rows. Web search carries strict xfails
where Codex or LiteLLM leaves the search tool out, and auto-compaction now also
expects Vertex AI to fail, for the same bridge gap Claude on Bedrock has
2026-11-03 03:10:05 -08:00
..
test_codex_rollout.py test(e2e): add Codex file edit, long session and auto-compaction rows through LiteLLM (#49386) 2026-11-03 01:52:29 +00:00
test_codex_run.py test(e2e): build a Codex compatibility table from tests/e2e/codex and add Vertex AI and five rows 2026-11-03 03:10:05 -08:00
test_compat_results.py test(e2e): build a Codex compatibility table from tests/e2e/codex and add Vertex AI and five rows 2026-11-03 03:10:05 -08:00
test_matrix_builder.py test(e2e): build a Codex compatibility table from tests/e2e/codex and add Vertex AI and five rows 2026-11-03 03:10:05 -08:00