litellm/tests/e2e
Yuneng Jiang 4717cb37b8
test(e2e): serve the vision image from our own fixture
The two vision tests pointed at a Wikipedia-hosted cat photo, so every run
depended on upload.wikimedia.org staying up and unthrottled. It throttled,
and the 429 surfaced as a bedrock APIConnectionError, which reads as a
gateway failure rather than what it was.

The image is now a fixture in the repo, passed as a data URL. That also puts
the two providers on the same bytes: litellm downloads the image itself for
bedrock, while openai is handed the link and fetches it from its own servers,
so the hosted URL quietly meant the two tests were not testing the same thing.

The image was generated for this repo rather than borrowed, so nothing here
carries a third-party license. Also drops a stale comment about openai prompt
caching that sat above the vision helper; no caching test uses it.

(cherry picked from commit ff418ffb9c)
2026-08-29 16:54:13 -07:00
..
a2a test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
access_control fix(auth): resolve bare model names against wildcard deployments in model access groups (#37492) 2026-08-19 15:33:29 -07:00
batches test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
claude_code test(e2e): replay a real tool-search assistant turn back to Bedrock Invoke (#36856) 2026-08-17 11:59:26 -07:00
coverage_registry Merge pull request #37903 from BerriAI/litellm_lit_5902_ws_passthrough_e2e_pin 2026-08-21 18:25:19 -07:00
guardrails test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
llm_translation test(e2e): serve the vision image from our own fixture 2026-08-29 16:54:13 -07:00
load test(e2e): move load/perf testing out of the main suite and drop the vllm passthrough test (#35820) 2026-08-04 14:12:25 -07:00
logging test(e2e): assert provider error shape instead of pinned prose 2026-08-15 16:08:52 -07:00
management test(e2e): add reproducers for passthrough and model budget gaps (#34657) 2026-08-11 18:15:34 -07:00
mcp test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
router test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
ui Merge pull request #38782 from BerriAI/litellm_/logs-reopen-shadcn-migration-2c9526 2026-08-29 15:15:11 -07:00
CLAUDE.md test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
conftest.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
CONTRIBUTING.md test(e2e): record and replay the non-streaming provider flows 2026-08-21 18:50:41 -07:00
e2e_config.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
fixture_bundle.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
fixture_canonical.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): pin openai_passthrough routing, cost logging, and file list isolation 2026-08-20 03:03:35 -07:00
models.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
otel_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
provider_edge.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
proxy_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
pytest.ini test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
test_e2e_http.py test(e2e): retry provider-transient statuses at the transport with bounded backoff (#35824) 2026-08-04 14:57:42 -07:00
test_fixture_bundle.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_fixture_canonical.py test(e2e): pin query params and multipart form fields as replay match-key identity 2026-08-20 15:59:34 -04:00
test_fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_provider_edge.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
transport.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00