litellm/tests/e2e
Yuneng Jiang a3d18c5468
test(e2e): make the streaming spend test judge its own stream
test_streaming_chat_completion_tracks_spend asserted only result.ok, which for
a stream is just the HTTP 200 the proxy commits before the upstream finishes. A
stream that breaks mid-flight comes back as a 200 carrying an SSE error frame
and no [DONE], so the only symptom was a missing spend row E2E_POLL_TIMEOUT
later, blaming spend tracking for a failure on the streaming path. It now
asserts is_streaming, chunks, stream_error and stream_done, the set its passing
sibling already used and the last streaming test in the suite that did not

ProxyClient's spend-log poller had the matching hole on the read side: every
non-success /spend/logs outcome became an empty list, so a window where the
endpoint 500'd or timed out on every read was indistinguishable from a row that
was never written. One successful read anywhere in the window still trusts the
emptiness, so a transient blip polls through, but a window with no successful
read now raises and carries the read that failed
2026-08-12 10:54:59 -07:00
..
a2a test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
access_control test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
batches test(e2e): skip the batch rate-limiter spend-row test pending LIT-5027 (#35301) 2026-07-31 01:20:04 +00:00
claude_code Merge pull request #36499 from BerriAI/litellm_bedrock_converse_pdf_text_block_fix 2026-08-11 12:59:00 -07:00
coverage_registry test(e2e): cover google-native generateContent framing and prometheus queue time (#34650) 2026-08-12 01:28:26 +00:00
guardrails test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
llm_translation test(e2e): cover google-native generateContent framing and prometheus queue time (#34650) 2026-08-12 01:28:26 +00:00
load test(e2e): move load/perf testing out of the main suite and drop the vllm passthrough test (#35820) 2026-08-04 14:12:25 -07:00
logging test(e2e): cover google-native generateContent framing and prometheus queue time (#34650) 2026-08-12 01:28:26 +00:00
management test(e2e): add reproducers for passthrough and model budget gaps (#34657) 2026-08-11 18:15:34 -07:00
mcp test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management test(e2e): make the streaming spend test judge its own stream 2026-08-12 10:54:59 -07:00
router test(e2e): add reliability suite covering fallback, timeout, and cache behavior (#34023) 2026-07-20 23:06:46 +00:00
ui test(e2e): self-seed the ui suite's password-login users in global setup 2026-08-04 17:38:38 -07:00
CLAUDE.md test(e2e): move load/perf testing out of the main suite and drop the vllm passthrough test (#35820) 2026-08-04 14:12:25 -07:00
conftest.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
CONTRIBUTING.md build(lint): rename make pre-commit to make check with a working-tree fallback 2026-08-08 03:25:35 -07:00
e2e_config.py test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
models.py test(e2e): add reproducers for passthrough and model budget gaps (#34657) 2026-08-11 18:15:34 -07:00
otel_client.py test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
proxy_client.py test(e2e): make the streaming spend test judge its own stream 2026-08-12 10:54:59 -07:00
pytest.ini test(e2e): add weekly session-anomaly load test against real providers 2026-07-21 14:54:02 -07:00
test_e2e_http.py test(e2e): retry provider-transient statuses at the transport with bounded backoff (#35824) 2026-08-04 14:57:42 -07:00
test_proxy_client.py test(e2e): make the streaming spend test judge its own stream 2026-08-12 10:54:59 -07:00
transport.py test(e2e): bound the post-/model/new servable wait at 40s 2026-07-28 17:47:19 -07:00