litellm/tests/e2e
Yuneng Jiang f1e7d9eef7
fix(e2e): move the vertex realtime suite off the retired Live preview model
Google withdrew gemini-live-2.5-flash-preview-native-audio-09-2025 from the
Vertex Live API. Every session dies at setup:

  received 1007 (invalid frame payload data)
  gemini-live-2.5-flash-preview-native-audio-09-2025 is not supported in the live api.

The client sees session.created (the proxy synthesizes it on connect) and then
nothing, so both vertex_ai realtime tests time out waiting for session.updated.

Confirmed by probing the Vertex Live endpoint directly with the e2e stack's own
credentials:

  gemini-live-2.5-flash-preview-native-audio-09-2025 -> 1007, not supported
  gemini-live-2.5-flash-native-audio                 -> setupComplete

so this swaps to the non-preview sibling, which is the same native-audio class
and is what the cost map already carries for vertex_ai.

Not a litellm regression. The suspicion fell on #38395 because it removed the
native-audio speechConfig strip, but the setup payload this suite sends is
byte-identical either side of that change: the strip only fires when a client
sends a voice, and the e2e SessionConfig has no voice field. Google's rejection
names the model, not a field.

The gemini (Google AI Studio) provider keeps the -09-2025 id, which still works
there; only the Vertex endpoint dropped it.

(cherry picked from commit a215ecaf3d)
2026-08-29 16:54:13 -07:00
..
a2a test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
access_control fix(auth): resolve bare model names against wildcard deployments in model access groups (#37492) 2026-08-19 15:33:29 -07:00
batches test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
claude_code test(e2e): replay a real tool-search assistant turn back to Bedrock Invoke (#36856) 2026-08-17 11:59:26 -07:00
coverage_registry Merge pull request #37903 from BerriAI/litellm_lit_5902_ws_passthrough_e2e_pin 2026-08-21 18:25:19 -07:00
guardrails test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
llm_translation fix(e2e): move the vertex realtime suite off the retired Live preview model 2026-08-29 16:54:13 -07:00
load test(e2e): move load/perf testing out of the main suite and drop the vllm passthrough test (#35820) 2026-08-04 14:12:25 -07:00
logging test(e2e): assert provider error shape instead of pinned prose 2026-08-15 16:08:52 -07:00
management test(e2e): add reproducers for passthrough and model budget gaps (#34657) 2026-08-11 18:15:34 -07:00
mcp test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
router test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
ui Merge pull request #38782 from BerriAI/litellm_/logs-reopen-shadcn-migration-2c9526 2026-08-29 15:15:11 -07:00
CLAUDE.md test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
conftest.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
CONTRIBUTING.md test(e2e): record and replay the non-streaming provider flows 2026-08-21 18:50:41 -07:00
e2e_config.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
fixture_bundle.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
fixture_canonical.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): pin openai_passthrough routing, cost logging, and file list isolation 2026-08-20 03:03:35 -07:00
models.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
otel_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
provider_edge.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
proxy_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
pytest.ini test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
test_e2e_http.py test(e2e): retry provider-transient statuses at the transport with bounded backoff (#35824) 2026-08-04 14:57:42 -07:00
test_fixture_bundle.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_fixture_canonical.py test(e2e): pin query params and multipart form fields as replay match-key identity 2026-08-20 15:59:34 -04:00
test_fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_provider_edge.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
transport.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00