mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
4725cb4661
* test(e2e): cover google-native generateContent framing and prometheus queue time
Adds live coverage for three shipped regressions that had none, all reached
through surfaces a customer drives from Google SDKs and operator dashboards.
The managed google-native route (`/v1beta/models/{model}:generateContent`) had
no harness support at all, so EndpointsClient gains generate_content and
stream_generate_content plus the request body models, and a new suite asserts
the two contracts that broke there: the response carries
x-litellm-response-cost so SDK traffic reconciles against spend (LIT-4076), and
the stream relays single-prefixed SSE frames with no OpenAI [DONE] terminator.
A doubled `data:` prefix, a leaked bytes literal, or the [DONE] sentinel each
fail the stream test; [DONE] absence is only asserted once real content has
arrived, because a first-chunk upstream error legitimately falls back to the
OpenAI error shape and does emit it.
The prometheus test pins litellm_request_queue_time_seconds to an actual
observation on our own key's series rather than to the family merely existing,
which is the distinction the original regression turned on: the histogram stayed
registered while nothing was ever written to it (LIT-2034).
Each assertion was mutation-checked against the live proxy; inverting the
[DONE] expectation, the cost-header expectation, or the metric name fails the
corresponding test.
* refactor(e2e): simplify google native coverage
|
||
|---|---|---|
| .. | ||
| realtime | ||
| conftest.py | ||
| endpoints_client.py | ||
| LLM_TRANSLATION_COVERAGE_MATRIX.md | ||
| passthrough_client.py | ||
| test_audio_speech_e2e.py | ||
| test_audio_transcriptions_e2e.py | ||
| test_bedrock_native_e2e.py | ||
| test_bedrock_web_search_server_tool_e2e.py | ||
| test_cache_control.py | ||
| test_chat_completions_contract_e2e.py | ||
| test_chat_completions_regression_e2e.py | ||
| test_chat_stream_contract_e2e.py | ||
| test_completions_endpoint_e2e.py | ||
| test_credential_messages_e2e.py | ||
| test_custom_pricing_e2e.py | ||
| test_deepseek_reasoning_e2e.py | ||
| test_embeddings_endpoint_e2e.py | ||
| test_files_batches_contract_e2e.py | ||
| test_google_native_e2e.py | ||
| test_image_edits_e2e.py | ||
| test_image_generation_e2e.py | ||
| test_messages_azure_foundry_e2e.py | ||
| test_messages_e2e.py | ||
| test_messages_mid_conversation_system_e2e.py | ||
| test_messages_mid_conversation_system_native_providers_e2e.py | ||
| test_moderations_e2e.py | ||
| test_ocr_rust_e2e.py | ||
| test_passthrough_e2e.py | ||
| test_passthrough_headers_e2e.py | ||
| test_provider_features_e2e.py | ||
| test_realtime_http_e2e.py | ||
| test_rerank_e2e.py | ||
| test_responses_bridge_streaming_e2e.py | ||
| test_responses_e2e.py | ||
| test_responses_retrieve_e2e.py | ||
| test_vector_stores_e2e.py | ||
| test_vertex_passthrough_e2e.py | ||