Commit graph

6027 commits

Author SHA1 Message Date
Cesar Garcia
2ef8bbdf6a
fix: add xiaomi_mimo to LlmProviders enum to fix router support (#18819)
Added XIAOMI_MIMO to the LlmProviders enum in types/utils.py.
The provider was already configured in providers.json but was
missing from the enum, causing "Unsupported provider" errors
when using it in Router/Proxy configurations.

Also added comprehensive unit tests to prevent regression.
2026-01-08 23:45:57 +05:30
Cesar Garcia
1c1ee8de46
Mask extra header secrets in model info (#18822) 2026-01-08 23:42:05 +05:30
Chongshun
60edf13a21
feat(tag-routing): support toggling tag matching between ANY and ALL (#18776) 2026-01-08 23:39:03 +05:30
Constantine
3ebec39b74
fix(proxy): use async anthropic client to prevent event loop blocking (#18435)
Fixes #16716.
Previously, synchronous Anthropic client was used for token counting, which blocked the event loop.
This change switches to AsyncAnthropic and caches the client instance.
2026-01-08 23:26:46 +05:30
Cesar Garcia
cfda03ebe1
fix(gemini): support snake_case for google_search tool parameters (#18451)
* fix(gemini): support snake_case for google_search tool parameters

Add snake_case aliases for Gemini tool names to match the pattern
already used by other tools (url_context, google_maps, code_execution):
- google_search -> googleSearch
- google_search_retrieval -> googleSearchRetrieval
- enterprise_web_search -> enterpriseWebSearch

* test(gemini): add tests for snake_case google_search tool aliases

* refactor(gemini): simplify get_tool_value calls formatting
2026-01-08 23:25:36 +05:30
Ishaan Jaffer
10ec499369 responses API fixes 2026-01-08 19:16:23 +05:30
Ishaan Jaffer
ebf09218d5 TestManusResponsesAPITest 2026-01-08 18:59:46 +05:30
Ishaan Jaff
b482d336b3
[Feat] New provider - Manus API on /responses, GET /responses (#18804)
* init ManusResponsesAPIConfig

* init MANUS ApI

* init MANUS create responses

* init MANUS

* test_extract_agent_profile

* transform_get_response_api_request

* test fix

* fixes non stream

* fix streaming

* add MANUSConfig

* test_multiturn_responses_api

* code QA check

* add manus

* Potential fix for code scanning alert no. 3961: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-01-08 18:37:42 +05:30
Ishaan Jaffer
58e6ef7d93 TestAzureAIFlux2ImageEdit 2026-01-08 18:23:05 +05:30
Sameer Kankute
5241b27bea fix: test_video_status_basic 2026-01-08 17:30:58 +05:30
Sameer Kankute
f5b5073649 fix: test_video_status_async 2026-01-08 17:20:38 +05:30
Sameer Kankute
62d860ea7d fix: litellm/tests/test_litellm/test_responses_id_security.py 2026-01-08 17:01:44 +05:30
Sameer Kankute
0a9861c3ec Fix: test_token_counter_lazy_imports 2026-01-08 16:44:35 +05:30
Sameer Kankute
9d5eb60ff1 Fix: test_text_format_to_text_conversion - properly mock handler to avoid API calls 2026-01-08 16:37:12 +05:30
Sameer Kankute
b6e011309a Fix: test_spend_logs_payload_success_log_with_router 2026-01-08 16:07:00 +05:30
Sameer Kankute
c023c69eae
Merge pull request #18739 from BerriAI/litellm_fix_deepinfra_tool_call
[Fix]: Tool content should be string for Deepinfra
2026-01-08 15:41:56 +05:30
Sameer Kankute
c13bc21520
Merge pull request #18754 from BerriAI/litellm_add_annotations_responses_bridge
Add annotations to completions responses API bridge
2026-01-08 15:30:16 +05:30
Sameer Kankute
0eb20ad500
Merge pull request #18745 from BerriAI/litellm_fix_gemini_audio_file_upload
Fix: Gemini generate content request with audio file id
2026-01-08 15:30:09 +05:30
Sameer Kankute
df7337fd38
Merge pull request #18800 from BerriAI/litellm_add_model_id_support_bedrock
Add support for model id in bedrock passthrough
2026-01-08 15:29:47 +05:30
Sameer Kankute
e0662d53a2
Merge pull request #18799 from BerriAI/litellm_fix_claude_usage_object
Fix: Incomplete usage in response object passed
2026-01-08 15:29:26 +05:30
Sameer Kankute
4abc94119f
Merge pull request #18797 from BerriAI/litellm_bedrock_kimi2_model
[Feat]Add bedrock kimi2 model support
2026-01-08 15:29:12 +05:30
Sameer Kankute
c05e148488
Merge pull request #18809 from BerriAI/main
merge main
2026-01-08 15:28:13 +05:30
Sameer Kankute
c95544959b
Merge pull request #18808 from BerriAI/main
merge main
2026-01-08 15:25:50 +05:30
Sameer Kankute
6941a29ad3
Merge pull request #18807 from BerriAI/main
merge main
2026-01-08 15:25:13 +05:30
Sameer Kankute
ad501048f3 Add support for Vertex AI API keys 2026-01-08 15:22:14 +05:30
Yuta Saito
271ee0959b test: focus 2026-01-08 16:53:09 +09:00
Emerson Gomes
6c00f6f342
Add support to zai glm-4.7 model in Vertex (#18782)
* Add support to zai glm-4.7 model in Vertex

* Avoid failed on missing 'created' streaming chunk key
2026-01-08 13:20:02 +05:30
Sameer Kankute
c78bf8cfcd Add support for model id in bedrock passthrough 2026-01-08 13:03:58 +05:30
Sameer Kankute
7e98843d97 Fix: Incomplete usage in response object passed 2026-01-08 10:57:07 +05:30
Sameer Kankute
af6883712e Add tests for kimi 2 bedrock model 2026-01-08 10:07:33 +05:30
yuneng-jiang
51759424a6 Key and Team Routing Setting 2026-01-07 17:17:30 -08:00
Alexsander Hamir
2f803171d6
refactor(prometheus): skip metrics for invalid API key requests (#18788) 2026-01-07 17:14:14 -08:00
Benedikt Óskarsson
81fefc69c9
fix(bedrock): handle thinking with tool calls for Claude 4 models 2026-01-08 00:48:17 +00:00
yuneng-jiang
e285e2b91d
Merge pull request #18675 from BerriAI/litellm_key_team_routing_config
[Feature] Allow Add and Update for router_settings on Keys and Teams
2026-01-07 16:24:04 -08:00
yuneng-jiang
d01c48ec5e User metrics for promethus 2026-01-07 15:05:09 -08:00
yuneng-jiang
1c84af8ae4 normalize proxy config callbacks 2026-01-07 12:22:57 -08:00
Elkhan Eminov
bae625bdc6
OpenRouter embeddings API support (#18391)
* support for OpenRouter embeddings

* add bearer

* add content header
2026-01-08 00:57:31 +05:30
Danial Khan
f126f75995
feat: pass server_tool_use and tool_search_tool_result blocks to anthropic (#18770)
* feat: Allow message types of server_tool_use and tool_search_tool_result to reach anthropic

* test: anthropic server tool use pass through testing
2026-01-08 00:47:28 +05:30
yuneng-jiang
140ce5fce2 Merge remote-tracking branch 'origin' into litellm_key_team_routing_config 2026-01-07 11:08:01 -08:00
yuneng-jiang
943445dd0f Adding test 2026-01-07 11:00:07 -08:00
yuneng-jiang
9a49bb293f Merge remote-tracking branch 'origin' into litellm_key_team_routing_config 2026-01-07 10:55:38 -08:00
yuneng-jiang
7da8d5fcd5 Merge remote-tracking branch 'origin' into litellm_endpoint_usage_breakdown 2026-01-07 10:25:43 -08:00
Ishaan Jaff
929af510fa
[Feat] New provider - Add Azure BFL FLux for image edits (#18766)
* add azure_ai/flux.2-pro

* get_flux2_image_generation_url

* azure_client_params

* docs

* add Image Editing

* add azure ai image edits

* AzureFoundryFlux2ImageEditConfig

* TestAzureAIFlux2ImageEdit
2026-01-07 23:28:39 +05:30
Abliteration AI
dc4ce7c5a2
feat: Add abliteration.ai provider (#18678)
* feat: Add abliteration.ai provider

* adding signoz integration to observability docs

* Fixing build

* Adding timeout for flaky test

* Fixing e2e

* add team member budget duration in team/update

* Reusable Duration Select and update team member budget UI

---------

Co-authored-by: Goutham Karthi <goutham@signoz.io>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: YutaSaito <36355491+uc4w6c@users.noreply.github.com>
2026-01-07 21:46:54 +05:30
tianduo-fh
f7212d84d5
fix: prevent duplicate User-Agent tags in request_tags (#18723)
The `_get_request_tags` function was returning a reference to the
original tags list from metadata, then mutating it with `.extend()`.
This caused duplicate User-Agent tags when the function was called
multiple times during a single request lifecycle (e.g., by logging,
prometheus, and guardrails).

The fix uses `.copy()` to create a new list before extending, ensuring
the original metadata tags are not mutated.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Tianduo Zhai <tzhai@firsthandadmins-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 21:42:27 +05:30
kothamah
1b8708fccc
Litellm embeddings calltype fix for guardrail precallhook (#18740)
* adding signoz integration to observability docs

* Fixing build

* Adding timeout for flaky test

* Fixing e2e

* add team member budget duration in team/update

* Reusable Duration Select and update team member budget UI

* feat: allow configuring project name for OpenTelemetry service name

* docs: sets ARIZE_PROJECT_NAME

* added valid callType for bedrock guardrail pre hook

This is to resolve the error when bedrock guardrails are enabled and invoke the embedding models.   {"error":{"message":"'embeddings' is not a valid CallTypes","type":"None","param":"None","code":"500"}}*

* updated the test case to reflect valid callType

---------

Co-authored-by: Goutham Karthi <goutham@signoz.io>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: YutaSaito <36355491+uc4w6c@users.noreply.github.com>
Co-authored-by: Yuta Saito <uc4w6c@bma.biglobe.ne.jp>
2026-01-07 21:40:36 +05:30
Harshit Jain
92f7789f10
feat(prometheus): add caching metrics (#18755) 2026-01-07 21:29:04 +05:30
Kris Xia
91b5c66cf2
fix(proxy): return json error response instead of sse format for initial streaming errors (#18757)
* adding signoz integration to observability docs

* Fixing build

* Adding timeout for flaky test

* Fixing e2e

* fix(proxy): return json error response instead of sse format for initial streaming errors

when the first chunk of a streaming response contains an error,
return a standard json error response instead of sse format.
this ensures clients receive properly formatted error responses
before the stream actually begins.

- rename create_streaming_response to create_response
- add logic to detect error in first chunk and return JSONResponse
- add _extract_error_from_sse_chunk helper function
- update all call sites to use the new function name
- update tests to reflect the function rename

* test(proxy): add comprehensive tests for error extraction from sse chunks

- Add new test class TestExtractErrorFromSSEChunk with 10 test cases
- Update existing tests to verify JSONResponse returned for initial streaming errors
- Add tests for error code as string, bytes input, invalid JSON, and edge cases
- Verify correct error format extraction from SSE chunks

---------

Co-authored-by: Goutham Karthi <goutham@signoz.io>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: YutaSaito <36355491+uc4w6c@users.noreply.github.com>
2026-01-07 21:26:47 +05:30
Ishaan Jaffer
6122ff9fce TestEncryptResponseId 2026-01-07 21:03:22 +05:30
Ishaan Jaffer
b855ad5541 test_encode_decode_helpers_roundtrip_in_cache_context 2026-01-07 18:23:17 +05:30