mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-07 02:59:05 +00:00
a780d4e4e3
* test(e2e): cover customer chat/messages cost + streaming paths Fills five uncovered P0 registry cells matching the customer's confirmed stack (OpenAI SDK, Bedrock, /v1/messages) and their per-request cost dependency: - /v1/messages logs cost that matches the x-litellm-response-cost header (LIT-4076) - OpenAI /chat/completions streams real content, and a non-streamed call is costed - Bedrock Converse /chat/completions returns real content non-streamed and streamed The streaming checks aggregate delta content and parse every chunk as JSON, so a clean-but-empty stream or a truncated chunk fails instead of passing on a bare 200. * test(e2e): add tool-use coverage for openai, bedrock converse, anthropic responses Function-calling regression guards on the paths the customer's agentic SDK usage exercises: OpenAI and Bedrock Converse /chat/completions, and Anthropic /v1/responses. The model is forced to call a weather tool and the test asserts the returned tool call names the function and carries JSON-parseable arguments with the expected field, so a dropped tool_call or malformed argument JSON fails instead of passing on a bare 200. Adds a minimal tool_calls field to the response OutMessage. * test(e2e): cover bedrock converse responses + thinking Adds llm.responses.bedrock_converse.basic/tool_use and llm.chat_completions.bedrock_converse.thinking. The thinking test enables extended thinking and requires reasoning_content plus a real answer, so a path that drops the reasoning block fails rather than passing. * test(e2e): cover bedrock embeddings + openai structured output and reasoning Bedrock Titan embeddings return a real vector; OpenAI structured output must yield schema-conforming JSON with the correct extracted values (age==42, not just valid JSON); an OpenAI reasoning call must report reasoning tokens, so a non-reasoning fallback fails. Adds response_format to ChatBody and reasoning-token details to Usage. * test(e2e): cover vision + streaming tool calls on openai and bedrock converse Vision on both providers must describe the image (not just 200); the streamed OpenAI tool call is reassembled from its fragments and its argument JSON parsed, so a stream that never completes the call or splits its JSON fails. Extends ChatMessage content to a typed text/image union. * test(e2e): cover openai prompt caching hit on repeated large prefix A repeated large-prefix prompt must report cached prompt tokens on the second call, so a cache regression that stops reusing the prefix (and silently re-bills full input) fails here. * test(e2e): cover openai audio speech + bedrock rerank and image generation Marks the OpenAI TTS cell and adds Bedrock Titan rerank (top_n honored, scored) and Bedrock Titan image generation (returns b64/url), the customer's non-chat AWS surfaces. * test(e2e): cover end-user (customer) create persistence mgmt.end_user.new.happy_path: create an end-user via /customer/new and confirm /customer/info reports it, the end-user-identity surface the customer relies on for per-customer controls. Adds customer models + management-client methods. * test(e2e): enforce key model allow-list on the passthrough route other.auth.passthrough.model_allowlist_enforced: a key scoped to gemini must be denied a claude call through the anthropic passthrough route (403), so custom-auth scoping is not bypassable by going through passthrough instead of /chat/completions. * test(e2e): address Greptile - assert stream data events, correlate messages spend by key - streaming: assert len(stream_events) > 1 instead of chunks > 1, since chunks counts the terminal data: [DONE] marker and would pass a single content event - messages cost: correlate the spend row by the unique scoped key rather than the Anthropic response id, which need not equal the proxy spend-log request_id |
||
|---|---|---|
| .. | ||
| realtime | ||
| conftest.py | ||
| endpoints_client.py | ||
| LLM_TRANSLATION_COVERAGE_MATRIX.md | ||
| passthrough_client.py | ||
| test_audio_speech_e2e.py | ||
| test_audio_transcriptions_e2e.py | ||
| test_cache_control.py | ||
| test_chat_completions_regression_e2e.py | ||
| test_credential_messages_e2e.py | ||
| test_custom_pricing_e2e.py | ||
| test_deepseek_reasoning_e2e.py | ||
| test_embeddings_endpoint_e2e.py | ||
| test_image_generation_e2e.py | ||
| test_messages_azure_foundry_e2e.py | ||
| test_messages_e2e.py | ||
| test_messages_mid_conversation_system_e2e.py | ||
| test_messages_mid_conversation_system_native_providers_e2e.py | ||
| test_moderations_e2e.py | ||
| test_ocr_rust_e2e.py | ||
| test_passthrough_e2e.py | ||
| test_passthrough_headers_e2e.py | ||
| test_provider_features_e2e.py | ||
| test_rerank_e2e.py | ||
| test_responses_e2e.py | ||
| test_responses_metadata_e2e.py | ||
| test_vertex_passthrough_e2e.py | ||