litellm/tests/e2e_harness
ryan-crabbe-berri 50af27eaa6
test(e2e): add Codex file edit, long session and auto-compaction rows through LiteLLM (#49386)
* test(e2e): add the Codex file edit and long session rows

The file edit row has Codex change a random value into config.txt with the patch tool and checks the file on disk. The long session row has it follow a chain of random file names one cat at a time. new_workspace now returns the resolved path, because Codex reports patched files by their real path (/private/var on macOS).

* test(e2e): make the Codex long session prove every earlier command output reached the model

* test(e2e): Codex auto-compaction through LiteLLM (#49389)

* test(e2e): add the Codex auto-compaction row

Codex reads a random value, compacts through the gateway with its remote compaction forced on, and is asked for the value after the file holding it is deleted. The row reads the compaction from the thread's rollout file and marks Claude on Bedrock as a strict expected failure, since the Responses-to-chat bridge ignores the compaction_trigger item

* refactor(e2e): validate Codex rollout lines with the models directly
2026-11-03 01:52:29 +00:00
..
batches test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
claude_code test(e2e): share the coding-CLI harness between suites (#49370) 2026-11-02 13:50:56 -08:00
cli_harness test(e2e): share the coding-CLI harness between suites (#49370) 2026-11-02 13:50:56 -08:00
codex test(e2e): add Codex file edit, long session and auto-compaction rows through LiteLLM (#49386) 2026-11-03 01:52:29 +00:00
coverage_registry test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
guardrails test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
logging test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
perf test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
AGENTS.md test(e2e): add Codex file edit, long session and auto-compaction rows through LiteLLM (#49386) 2026-11-03 01:52:29 +00:00
pytest.ini test(e2e): attach perf case metrics and limits as JUnit properties (#46774) 2026-10-15 19:12:07 -07:00
test_e2e_http.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_fixture_bundle.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_fixture_canonical.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_fixture_mode.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_idp.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_junit_properties.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_provider_edge.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_proxy_client.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_stack_lock.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00