scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36
litellm/tests/e2e
History
yuneng-jiang add2d23df2
test(e2e): bypass the proxy response cache in the mid-conversation system and fallback tests (#37915) 
The mid-conversation system tests prime the prompt cache by re-sending an
identical /v1/messages body until its usage shows the full prefix read back
three times in a row. The e2e stack runs with the litellm response cache on,
so every resend after the first is served from redis with the first call's
usage and the streak can never form; the three unflagged-model tests have
failed on every litellm-e2e build since the consecutive-read check landed.
Send cache: {"no-cache": true} on RichMessagesRequest, as test_cache_control
already does, so each resend reaches the provider.

The two fallback tests sent the same "say hi" / max_tokens=16 body to the
gpt-5.5 fallback, so one empty (finish_reason=length) completion served the
second test from the response cache and failed both. Give each test a unique
prompt and leave gpt-5.5 enough tokens to emit text.
2026-08-21 22:07:16 -07:00
..
a2a test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
access_control fix(auth): resolve bare model names against wildcard deployments in model access groups (#37492) 2026-08-19 15:33:29 -07:00
batches test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
claude_code test(e2e): replay a real tool-search assistant turn back to Bedrock Invoke (#36856) 2026-08-17 11:59:26 -07:00
coverage_registry Merge pull request #37903 from BerriAI/litellm_lit_5902_ws_passthrough_e2e_pin 2026-08-21 18:25:19 -07:00
guardrails test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
llm_translation test(e2e): bypass the proxy response cache in the mid-conversation system and fallback tests (#37915) 2026-08-21 22:07:16 -07:00
load test(e2e): move load/perf testing out of the main suite and drop the vllm passthrough test (#35820) 2026-08-04 14:12:25 -07:00
logging test(e2e): assert provider error shape instead of pinned prose 2026-08-15 16:08:52 -07:00
management test(e2e): add reproducers for passthrough and model budget gaps (#34657) 2026-08-11 18:15:34 -07:00
mcp test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management Merge pull request #37607 from BerriAI/litellm_lit_5869_cost_e2e_pins 2026-08-20 16:15:08 -07:00
router test(e2e): bypass the proxy response cache in the mid-conversation system and fallback tests (#37915) 2026-08-21 22:07:16 -07:00
ui refactor(ui): migrate the last antd components off antd onto shadcn (#37569) 2026-08-20 03:01:07 +00:00
CLAUDE.md test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
conftest.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
CONTRIBUTING.md test(e2e): record and replay the non-streaming provider flows 2026-08-21 18:50:41 -07:00
e2e_config.py [e2e] Pin the OpenAI websocket passthrough prefixes 2026-08-21 17:26:18 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
fixture_bundle.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
fixture_canonical.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): pin openai_passthrough routing, cost logging, and file list isolation 2026-08-20 03:03:35 -07:00
models.py Merge pull request #37607 from BerriAI/litellm_lit_5869_cost_e2e_pins 2026-08-20 16:15:08 -07:00
otel_client.py test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
provider_edge.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
proxy_client.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
pytest.ini test(e2e): add weekly session-anomaly load test against real providers 2026-07-21 14:54:02 -07:00
test_e2e_http.py test(e2e): retry provider-transient statuses at the transport with bounded backoff (#35824) 2026-08-04 14:57:42 -07:00
test_fixture_bundle.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_fixture_canonical.py test(e2e): pin query params and multipart form fields as replay match-key identity 2026-08-20 15:59:34 -04:00
test_fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_provider_edge.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
transport.py test(e2e): bound the post-/model/new servable wait at 40s 2026-07-28 17:47:19 -07:00