litellm/tests/e2e
Kerry Lu cc1d2c66c8 test(e2e): bound chaos latency and log volume with flat ceilings
A ratio against the healthy phase cannot bound either metric. Once the Redis
circuit breaker opens, a request skips Redis instead of waiting on its socket
timeout, so the chaos phase can measure cheaper than the baseline it is compared
against: local runs came in at 0.61x baseline p90 while a log-bytes ratio read
724x. Splitting Budget into RatioBudget and AbsoluteBudget lets RSS and CPU keep
the ratio they need, since both are machine-shaped, while latency and log volume
get the wall-clock ceiling a user actually cares about.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-11 13:38:01 -07:00
..
a2a test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
access_control test(e2e): require a 200 inside the regenerate grace window and drop the helper docstrings 2026-09-05 11:53:22 -07:00
batches fix(files): preserve managed deletion routing and response identity 2026-09-07 16:59:53 -07:00
claude_code fix(e2e): wait for every gateway before using a new model and keep the network rerun 2026-09-05 16:10:40 -07:00
coverage_registry test(load): drive /v1/messages alongside /chat/completions in the Redis chaos test 2026-09-11 12:28:02 -07:00
gateway test(e2e): address Redis chaos PR review, add log-bytes budget 2026-09-11 12:40:25 -07:00
guardrails test: stop guardrail retries at the polling deadline 2026-09-08 17:02:00 -07:00
llm_translation fix(mcp): honor an explicit null on toolset update, cover MCP lifecycle e2e (#40022) 2026-09-08 22:50:13 -07:00
load test(e2e): bound chaos latency and log volume with flat ceilings 2026-09-11 13:38:01 -07:00
logging fix(mcp): honor an explicit null on toolset update, cover MCP lifecycle e2e (#40022) 2026-09-08 22:50:13 -07:00
management revert(model-management): roll back #40047 2026-09-09 00:06:26 -07:00
mcp fix(mcp): honor an explicit null on toolset update, cover MCP lifecycle e2e (#40022) 2026-09-08 22:50:13 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management test(e2e): drop the unused batch list client and fail loudly on a user teardown miss 2026-09-09 18:34:15 -07:00
router test(e2e): rewrite the Redis timeout test as a locust chaos load test 2026-09-10 21:55:23 -07:00
ui test(e2e/ui): cover key budget window, non-admin model scope edit, and key blocking (#40027) 2026-09-08 22:49:54 -07:00
CLAUDE.md test(e2e): bound chaos latency and log volume with flat ceilings 2026-09-11 13:38:01 -07:00
conftest.py test(load): pause Redis outright and budget the chaos phase against the baseline 2026-09-11 11:06:06 -07:00
CONTRIBUTING.md test(e2e): rewrite the Redis timeout test as a locust chaos load test 2026-09-10 21:55:23 -07:00
e2e_config.py test(e2e): rewrite the Redis timeout test as a locust chaos load test 2026-09-10 21:55:23 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py fix(mcp): honor an explicit null on toolset update, cover MCP lifecycle e2e (#40022) 2026-09-08 22:50:13 -07:00
fixture_bundle.py feat(e2e): record and replay streamed provider responses chunk-for-chunk 2026-08-24 12:51:44 -07:00
fixture_canonical.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
junit_properties.py Refuse a source path carrying a colon 2026-09-01 16:02:54 -07:00
lifecycle.py fix(e2e): clean up batch files reliably and expire Azure inputs 2026-09-07 12:15:06 -07:00
models.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_e2e_redis_timeout 2026-09-11 11:38:22 -07:00
otel_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
provider_edge.py fix(e2e): record a streamed chunk only after its downstream write lands 2026-08-24 13:26:58 -07:00
proxy_client.py revert(model-management): roll back #40047 2026-09-09 00:06:26 -07:00
pytest.ini test(load): pause Redis outright and budget the chaos phase against the baseline 2026-09-11 11:06:06 -07:00
test_e2e_http.py fix(mcp): honor an explicit null on toolset update, cover MCP lifecycle e2e (#40022) 2026-09-08 22:50:13 -07:00
test_fixture_bundle.py feat(e2e): record and replay streamed provider responses chunk-for-chunk 2026-08-24 12:51:44 -07:00
test_fixture_canonical.py test(e2e): pin query params and multipart form fields as replay match-key identity 2026-08-20 15:59:34 -04:00
test_fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_junit_properties.py test(e2e): read JUnit properties off the real collected pytest Item 2026-09-01 19:12:07 -07:00
test_provider_edge.py fix(e2e): record a streamed chunk only after its downstream write lands 2026-08-24 13:26:58 -07:00
test_proxy_client.py revert(model-management): roll back #40047 2026-09-09 00:06:26 -07:00
transport.py revert(model-management): roll back #40047 2026-09-09 00:06:26 -07:00