Commit graph

5398 commits

Author SHA1 Message Date
Sameer Kankute
8b7c36c271
Merge pull request #19135 from BerriAI/litellm_websocker_realtime_fix
Fix model name from query param in realtime request
2026-01-15 22:26:28 +05:30
Sameer Kankute
8c454eb74f Fix: mock test tests 2026-01-15 22:02:51 +05:30
Sameer Kankute
83e33944ef Fix: mock test tests 2026-01-15 22:02:42 +05:30
Sameer Kankute
890fa85a33 Fix: response enterprise tests 2026-01-15 21:25:22 +05:30
Sameer Kankute
b4ee1b2589 Fix: response enterprise tests 2026-01-15 19:53:04 +05:30
Sameer Kankute
4bdda9cc28 Fix: tests/test_litellm/proxy/test_proxy_server.py::test_embedding_input_array_of_tokens 2026-01-15 19:46:35 +05:30
Sameer Kankute
f28d951202 Fix: tests/test_litellm/proxy/test_litellm_pre_call_utils.py::test_embedding_header_forwarding_with_model_group 2026-01-15 19:41:16 +05:30
Sameer Kankute
3af994a32a Fix: test router 2026-01-15 19:21:31 +05:30
Sameer Kankute
83cdfd886a
Merge pull request #19059 from BerriAI/litellm_openrouter_image_gen
Add openrouter support for image/generation endpoints
2026-01-15 15:48:00 +05:30
Sameer Kankute
e9a2999f61 Fix: handling of model name in query param 2026-01-15 15:06:37 +05:30
YutaSaito
f541bc0261
Merge pull request #19129 from BerriAI/litellm_fix_mcp_chat_completions
[fix] mcp chat completions
2026-01-15 18:21:37 +09:00
Yuta Saito
1c2942d808 test: add mcp completions test 2026-01-15 15:47:45 +09:00
Yuta Saito
ba43f742ab fix: event loop nesting issue in MCP chat completions with streaming
Problem: When using MCP tools with  and , the following error occurred:

RuntimeError: Timeout context manager should be used inside a task
2026-01-15 15:36:56 +09:00
Sameer Kankute
dca42047b9
Merge pull request #19068 from BerriAI/litellm_responses_caching_support
[Fix] Add support for caching for responses API
2026-01-15 08:38:31 +05:30
yuneng-jiang
27bc5c1fca
Merge pull request #19120 from BerriAI/litellm_yj_triage_branch
[Fix] Azure Storage Circular Reference Error
2026-01-14 18:08:38 -08:00
Ishaan Jaff
458f773861
[Feat] Claude Code - Add support for Prompt Caching with Bedrock Converse (#19123)
* init BaseAnthropicMessagesPromptCachingTest

* fix UsageDelta

* fix: _create_initial_usage_delta

* TestBedrockInvokePromptCaching

* translate_anthropic_messages_to_openai wiht cache control

* fix translate_anthropic_messages_to_openai
2026-01-14 18:05:10 -08:00
Ishaan Jaff
e67e9e7054
[Fix] Containers API - Allow routing to regional endpoints (#19118)
* fix get_complete_url

* fix url resolution containers API

* TestContainerRegionalApiBase
2026-01-14 17:28:18 -08:00
yuneng-jiang
97c39dbb7a Fixing circular reference for azure storage 2026-01-14 16:11:55 -08:00
Ishaan Jaff
62187103b4
[Fix] Containers API - Container API routes return 401 for non-admin users - routes missing from openai_routes (#19115)
* test_containers_routes_are_llm_api_routes

* allow /containers/* API
2026-01-14 15:14:03 -08:00
Ishaan Jaff
06ded8750e
[Fix] Claude Code (/messages) - Litellm fix claude code Bedrock Invoke usage, request signing (#19111)
* test_should_not_fail_with_forwarded_headers_bedrock_invoke_messages

* use common get_request_headers for BaseAWS

* fix get_request_headers

* test_should_not_fail_with_forwarded_headers_bedrock_invoke_messages
2026-01-14 14:51:50 -08:00
yuneng-jiang
dcac090de0
Merge pull request #19053 from BerriAI/litellm_intern_user_usage
[Fix] /team/daily/activity Show Internal Users Their Spend Only
2026-01-14 13:47:34 -08:00
Ishaan Jaff
747829dadb
[Fix] Claude Code + Bedrock Converse Usage - ensure budget tokens are passed to converse api correctly (#19107)
* test_bedrock_converse_budget_tokens_preserved

* test_openai_model_with_thinking_converts_to_reasoning_effort

* fix translate_anthropic_thinking_to_reasoning_effort

* test_bedrock_converse_budget_tokens_preserved

* test_anthropic_messages_bedrock_converse_with_thinking
2026-01-14 12:02:27 -08:00
Rayan Pal
f880ea537f
fix(vertex): add type object to tool schemas missing type field (#19103)
Tools with no parameters (like EnterPlanMode from Anthropic Agents SDK)
send schemas with only $schema and no type field. Gemini rejects these
with "functionDeclaration parameters schema should be of type OBJECT".

Adds type: object when schema has no type and no anyOf/oneOf/allOf.
2026-01-15 00:43:35 +05:30
Peter Golm
c215b3a79f
fix: preserve llm_provider-* headers in error responses (#19020)
Extract and preserve provider-specific headers (llm_provider-*) when
handling error responses from LLM providers. This ensures that useful
debugging information from providers is available even when requests
fail with BadRequestError or similar exceptions.
2026-01-14 22:49:39 +05:30
Sameer Kankute
cd2d381dd4
Merge pull request #19042 from BerriAI/litellm_staging_01_13_2026
Staging 01/13/2026
2026-01-14 21:29:09 +05:30
Sameer Kankute
4aadc0d41f
Merge pull request #19074 from BerriAI/litellm_19046-bug-retry-policies-are-not-applied-on-responses-calls
Add retry policy support to responses API
2026-01-14 17:56:11 +05:30
Sameer Kankute
26d8ef734c
Merge pull request #19070 from BerriAI/litellm_19066-bug-gemini-image-generation-returns-incorrect-prompt_tokens_details
Fix: [Bug]: Gemini Image Generation Returns Incorrect prompt_tokens_d…
2026-01-14 17:55:09 +05:30
Sameer Kankute
49d873e894
Merge pull request #19067 from BerriAI/litellm_caching_fix
Fix: anthropic token counter with thinking
2026-01-14 17:52:56 +05:30
Sameer Kankute
b677ea1113
Merge pull request #19064 from BerriAI/litellm_openai_content_none_fix
Fix: Responses content can't be none
2026-01-14 17:52:31 +05:30
Sameer Kankute
7dbf09cb12 Fix all 130126 tests 2026-01-14 17:47:03 +05:30
Sameer Kankute
b4a48f7996
Revert "feat(gemini): add opt-in support for responseJsonSchema (#18147)"
This reverts commit 4e417f9ef1.
2026-01-14 17:06:13 +05:30
Sameer Kankute
ff467c797d
Merge pull request #19079 from BerriAI/main
merge main
2026-01-14 16:44:31 +05:30
rohan.w.charles
bc88dbfbc9 fix: implement mid-stream fallback handling based on user preference 2026-01-14 16:28:51 +05:30
Sameer Kankute
52d3c9dcfc Add retry policy support to responses API 2026-01-14 14:55:17 +05:30
Sameer Kankute
b2d4f67e17
Merge pull request #19023 from eagle-p/feat/guardrail-clean-error-message
fix(guardrails): use clean error messages for blocked requests
2026-01-14 14:03:16 +05:30
Sameer Kankute
11cc8dc9d2
Merge pull request #19037 from Jetemple/json-log-fix
fix: enable JSON logging via configuration and add regression test
2026-01-14 13:51:36 +05:30
Sameer Kankute
6991342dc4 Fix: [Bug]: Gemini Image Generation Returns Incorrect prompt_tokens_details 2026-01-14 13:47:46 +05:30
Sameer Kankute
333947e236 Add support for caching for responses API 2026-01-14 13:33:07 +05:30
Sameer Kankute
0563ce42a4 Fix: anthropic token counter with thinking 2026-01-14 12:59:23 +05:30
Sameer Kankute
1b34d17935 Fix: Responses content can't be none 2026-01-14 12:13:18 +05:30
YutaSaito
8d204f0a84
Merge pull request #19060 from BerriAI/litellm_fix_anthropic-during-call-error
[fix] anthropic during call error
2026-01-14 15:13:09 +09:00
Sameer Kankute
2b13c9aba2 Add tests for openrouter 2026-01-14 10:17:01 +05:30
yuneng-jiang
39f56abd8d
Merge pull request #18878 from BerriAI/litellm_hakhundov_contribution
[Fix] Feature Flag for Fix SCIM GET /Users error and enforce SCIM 2.0 compliance
2026-01-13 20:46:28 -08:00
Yuta Saito
82ed6283fa test: reorganize unified guardrail tests into nested classes 2026-01-14 13:38:43 +09:00
Yuta Saito
c5ced033c9 fix: anthropic during call guardrail error 2026-01-14 13:37:01 +09:00
YutaSaito
95b419d65b
Merge pull request #18938 from BerriAI/litellm_fix_generate_two_telemetry
[fix] generate two telemetry in responses
2026-01-14 12:35:23 +09:00
Ishaan Jaff
1b00576711
[Feat] New Model - Azure Model Router on LiteLLM AI Gateway (#19054)
* fix - azure model router integration

* fix:_check_provider_match

* fix:_get_response_model

* tests azure model router

* test_azure_ai_model_router_streaming_model_in_chunk

* fix LlmProviders.AZURE.value

* test_azure_ai_model_router_streaming_cost_with_stream_options

* def test_get_model_from_chunks_azure_model_router():

* _get_model_from_chunks

* docs azure model router

* azure model router
2026-01-13 18:31:43 -08:00
Yuta Saito
2814d3dd3d test: cover success handler sync/async logging branches 2026-01-14 10:52:14 +09:00
YutaSaito
a66e007574
Merge pull request #19051 from BerriAI/litellm_fix_mcp-rest-auth-checks
[fix] mcp rest auth checks
2026-01-14 10:39:28 +09:00
Alexsander Hamir
15c3bc219b
[Refactor] Add CI enforcement for O(1) operations in _get_model_cost_key to prevent performance regressions (#19052)
* Optimize _get_model_cost_key to avoid expensive scans

- Remove expensive O(n) scan fallback that was causing 42.87% CPU overhead
- Only scan when size mismatch detected (O(1) check)
- Add warning in docstring: Only O(1) lookup operations are acceptable
- Clean up comments to be more concise
- Keep stale entry rebuild for pop() case (only triggers when stale entry found)

This fixes the performance issue where the scan was being triggered on every
failed lookup, causing severe CPU overhead during router operations.

* Add code quality check to enforce O(1) operations in _get_model_cost_key

- Add check_get_model_cost_key_performance.py to statically analyze _get_model_cost_key
- Detects O(n) operations (loops, comprehensions, problematic function calls)
- Recursively checks called functions to find nested O(n) operations
- Allows conditional O(n) rebuilds in helper functions (_rebuild_model_cost_lowercase_map, _handle_stale_map_entry_rebuild, _handle_new_key_with_scan)

* Integrate _get_model_cost_key performance check into CI pipeline

- Add check_get_model_cost_key_performance.py to check_code_and_doc_quality job
- Ensures O(1) requirement is enforced in CI to prevent performance regressions

* Remove unused performance test and clean up utils.py

- Remove test_get_model_info_performance.py (no longer needed)
- Remove extra blank line in utils.py

* Document allowed helper functions and exception process in _get_model_cost_key

- Add documentation listing allowed helper functions with O(n) operations
- Explain why these are acceptable (conditionally called)
- Add instructions for adding new exceptions to check_get_model_cost_key_performance.py

* Fix docstring detection and type checker error in performance check

- Add proper docstring tracking to skip docstring content (fixes false positive for 'map' in docstring)
- Add None check for docstring_quote to fix type checker error
- Restore _handle_new_key_with_scan to allowed_helpers list

* Remove check_get_model_cost_key_performance from CI pipeline

- Temporarily remove the performance check from CI to avoid blocking builds

* Restore performance check and remove memory leak tests from CI

- Add back check_get_model_cost_key_performance.py to CI pipeline
- Remove memory_leak_tests job that was causing port conflicts

* Remove extra blank line in CI config
2026-01-13 17:08:03 -08:00