scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8
31779 commits 3745 branches 1395 tags 3.9 GiB
Commit graph

6027 commits

Author SHA1 Message Date
Will Chen
9f57eb3e74
Fix Azure AI costs for Anthropic models (#19530) 
* Fix Azure AI cost calculation

* fixup
 2026-01-21 21:10:27 -08:00
Emerson Gomes
a3f7f5858b
Fix date overflow/division by zero in proxy utils (#19527) 
* Fix date overflow/division by zero in proxy utils

* Fix projected spend calculation

* Strengthen projected spend tests
 2026-01-21 21:09:57 -08:00
Yogeshwaran Ravichandran
ab274ac3c4
fix(azure response api): flatten tools for responses api to support nested definitions (#19526) 
The Azure Responses API uses a different schema (flattened) for tools compared to the standard OpenAI/Azure Chat Completions API (nested). This caused a `BadRequestError` when users passed standard tool definitions.

Changes:
- Implemented tool flattening logic in `AzureOpenAIResponsesAPIConfig.transform_responses_api_request`.
- Added comprehensive unit tests in test_azure_transformation.py to verify nested-to-flat transformation, pass-through of flat tools, and immutability.
- Ensures cross-provider compatibility for tool definitions.

Fixes #19523
 2026-01-21 21:08:28 -08:00
Emerson Gomes
c8669cf8fa
Fix Azure RPM calculation formula (#19513) 
* Fix Azure RPM calculation formula

* updated test
 2026-01-21 21:03:23 -08:00
João Dinis Ferreira
60840ea292
fix(bedrock): correct streaming choice index for tool calls (#19506) 
Bedrock's contentBlockIndex identifies content blocks within a message
(text=0, tool_call=1), not OpenAI's choice index (which varies with n>1).
This caused OpenAI SDK's ChatCompletionAccumulator to fail when tool call
chunks arrived on index 1 while finish_reason arrived on index 0.

Bedrock doesn't support n>1 (no such parameter exists):
https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InferenceConfiguration.html

OpenAI choice index spec:
https://platform.openai.com/docs/api-reference/chat/streaming
 2026-01-21 20:57:14 -08:00
Harshit Jain
22000f3beb
fix: add case-insensitive support for guardrail mode and actions (#19480) 2026-01-21 20:52:57 -08:00
Ishaan Jaff
a117509f93
[Fix] UI Cost Estimator - Fix model dropdown (#19529) 
* add cost estimator

* ui fix show errors

* test_estimate_cost_resolves_router_model_alias
 2026-01-21 20:19:40 -08:00
Harshit Jain
746414eb9b
Fix/per service ssl override v2 (#19538) 
* refactor(ssl): support per-service SSL verification overrides

* add test cases for ssl
 2026-01-21 20:10:04 -08:00
davida-ps
7777aeb695
fixing prompt-security's guardrail implementation (#19374) 
* Consolidated change

* fix(prompt_security): update message processing to persist sanitized files and filter for API calls

* fix per krrishdholakia suggestion
 2026-01-21 20:09:40 -08:00
Ishaan Jaff
ab606c9a73
[Feat] Add Structured output for /v1/messages with Anthropic API, Azure Anthropic API, Bedrock Converse (#19545) 
* fix: add AnthropicMessagesRequestOptionalParams

* add _update_headers_with_anthropic_beta

* fix output format tests

* test_structured_output_e2e

* TestAnthropicAPIStructuredOutput

* test_structured_output_e2e

* fix BASE

* TestAzureAnthropicStructuredOutput

* fix: Bedrock Converse

* add nthropic Messages Pass-Through Architecture

* fix: bedrock invoke output_format

* fix: transform_anthropic_messages_request for vertex anthropic

* TestBedrockInvokeStructuredOutput

* docs anthropic vertex

* docs fix

* docs fix
 2026-01-21 20:09:18 -08:00
jay prajapati
363b0cc132
fix(azure): preserve content_policy_violation details for images (#19328) (#19372) 
Azure OpenAI Images (DALL·E 3) returns policy violations as a structured payload under body["error"], including inner_error.content_filter_results and revised_prompt.

LiteLLM previously:
- Failed to extract nested error messages (get_error_message only handled body["message"])
- Missed policy violation detection when error strings were generic
- Dropped inner_error details when raising ContentPolicyViolationError

This change:
- Extracts nested Azure error fields (code/type/message + inner_error)
- Detects policy violations via structured error codes
- Passes an OpenAI-style error body + provider_specific_fields to preserve details

Tests:
- python3 -m pytest tests/test_litellm/llms/azure/test_azure_exception_mapping.py
- python3 -m pytest tests/test_litellm/litellm_core_utils/test_exception_mapping_utils.py

Fixes #19328
 2026-01-21 20:06:51 -08:00
jay prajapati
0e738a5027
fix(mcp): forward static_headers to MCP servers (#19341) (#19366) 
Forward static_headers from /mcp-rest/test/* routes into the MCP client so headers are present during session.initialize() and tool discovery.

Also add a shared merge_mcp_headers() helper to keep header precedence consistent and ensure OpenAPI-to-MCP generated tools include static_headers.

Tests:
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_rest_endpoints.py
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py -k register_openapi_tools_includes_static_headers

Fixes #19341

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
 2026-01-21 19:30:55 -08:00
Ishaan Jaff
d5e912322f
[Fix] VertexAI Pass through - Ensure only anthropic betas are forwarded down to LLM API (#19542) 
* fix ALLOWED_VERTEX_AI_PASSTHROUGH_HEADERS

* test_vertex_passthrough_forwards_anthropic_beta_header

* fix test_vertex_passthrough_forwards_anthropic_beta_header

* test_vertex_passthrough_does_not_forward_litellm_auth_token

* fix utils

* Using Anthropic Beta Features on Vertex AI

* test_forward_headers_from_request_x_pass_prefix
 2026-01-21 19:12:04 -08:00
yuneng-jiang
6b6785bc4f
Merge pull request #19539 from BerriAI/litellm_models_scope 
[Feature] Adding Optional scope Param to /models
 2026-01-21 17:41:22 -08:00
yuneng-jiang
c6b157832b
Merge pull request #19296 from BerriAI/litellm_esca_reissue 
[Reissue: Fix] /user/new Privilege Escalation
 2026-01-21 16:46:34 -08:00
yuneng-jiang
6723b30d03 Adding scope to /models 2026-01-21 16:40:31 -08:00
Yuta Saito
898cc3ff4f test: update langfuse trace_id tests to use litellm_trace_id 2026-01-22 06:19:43 +09:00
YutaSaito
4a14a53ae8
Merge pull request #19469 from BerriAI/litellm_feat_mcp_spendlogs 
[feat] mcp spendlogs
 2026-01-22 05:29:21 +09:00
Ishaan Jaff
5cb5969a26
[Fix] LiteLLM VertexAI Pass through - ensuring incoming headers are forwarded down to target (#19524) 
* test_vertex_passthrough_forwards_anthropic_beta_header

* add_incoming_headers
 2026-01-21 12:01:33 -08:00
mubashir1osmani
410daf6e6d added tests 2026-01-21 14:41:35 -05:00
yuneng-jiang
d0e35751a1 Fixing tests and linting 2026-01-21 11:02:39 -08:00
yuneng-jiang
b5a7d2ab34 Paginating model/info endpoint 2026-01-21 10:44:18 -08:00
John Greek
aa4b0e0149
Fix duplicate test_handler.py filenames causing pytest collection errors (#19385) 2026-01-21 08:47:50 -08:00
Sameer Kankute
95dc14e400
Merge pull request #19389 from BerriAI/litellm_timeout_fix 
Fix: total timeout is not respected
 2026-01-21 19:16:11 +05:30
Sameer Kankute
e758dd0a59
Merge pull request #19472 from BerriAI/litellm_fix_chat_completion_responses_streaming 
Fix: tool call streaming in chat completion bridge
 2026-01-21 19:15:53 +05:30
Sameer Kankute
540370a1aa
Merge pull request #19479 from BerriAI/litellm_sarvam_int 
Add support for sarvam models
 2026-01-21 19:03:52 +05:30
Sameer Kankute
b6691baf24
Merge pull request #19482 from BerriAI/litellm_anthropic_max_length 
Filter out unsupported fields from JSON schema for Anthropic's output_format API.
 2026-01-21 18:57:16 +05:30
Sameer Kankute
e315e4cd7c fix litellm/tests/litellm_utils_tests/test_utils.py 2026-01-21 18:50:02 +05:30
Sameer Kankute
aeb38ffa9b
Merge pull request #19497 from BerriAI/main 
merge main
 2026-01-21 18:43:27 +05:30
Sameer Kankute
d879dcdcef
Revert "Fix/prisma schema permission (#19391)" 
This reverts commit 75ee0d126c.
 2026-01-21 17:19:32 +05:30
Sameer Kankute
a1aba2ed8d
Merge pull request #19491 from BerriAI/main 
merge main 20 1 25
 2026-01-21 16:40:48 +05:30
Sameer Kankute
c571a02f3b Filter out unsupported fields from JSON schema for Anthropic's output_format API. 2026-01-21 15:48:22 +05:30
Sameer Kankute
3a0d166eb0 Fix: tool call streaming in chat completino brigde 2026-01-21 12:14:02 +05:30
Yuta Saito
caf5f7f8ae test: add test 2026-01-21 14:51:56 +09:00
Sameer Kankute
a5ea08a0bf Fix test_default_api_base failing because of chatgpt as provider 2026-01-21 09:32:38 +05:30
Ryne Carbone
15013cec4b
feat(gemini): add file content support in tool results (#19416) 
Add support for 'file' and 'input_file' content types in
convert_to_gemini_tool_call_result(). File content in tool
results was previously silently dropped.

Supports base64 data URIs and HTTP URLs, matching the existing
image handling pattern. Enables PDF, audio, video, and other
file types as inline_data for Gemini.
 2026-01-20 19:54:12 -08:00
Cesar Garcia
b4ed387d24
fix(vertex_ai): handle reasoning_effort as dict from OpenAI Agents SDK (#19419) 
The OpenAI Agents SDK (v0.6.9+) now passes reasoning_effort as a dict
when summary is specified: {"effort": "high", "summary": "auto"}

This change extracts the "effort" value from the dict for Vertex AI,
which only supports thinkingLevel (not summary).

Before: reasoning_effort={"effort": "high"} was silently ignored
After: reasoning_effort={"effort": "high"} correctly maps to thinkingLevel

Fixes #19411
 2026-01-20 19:31:25 -08:00
Sameer Kankute
0b9f6b543f Fixes test_aaabasic_gcs_logger 2026-01-21 08:58:16 +05:30
Sampson
09941dd1d1
add search provider for brave search api (#19433) 
* add search provider for brave search api

Introduces a minimal implementation of the Brave Search API as a search provider. Additionally, this PR introduces a test file to ensure the provider works properly, and numerous other smaller changes (e.g., changes to docs to mention the new option).

* Update transformation.py
 2026-01-20 19:23:29 -08:00
Kamil Jopek
ce722ab763
Make grpc dependency optional (#19447) 
* Make grpc optional and document gRPC OTEL setup

* Add tests for missing OTLP gRPC imports
 2026-01-20 19:03:52 -08:00
Harshit Jain
3c3148789a
fix: resolve Read-only file system error in non-root images (#19449) 2026-01-20 19:00:52 -08:00
Harshit Jain
b36e704e06
fix: ensure auto-rotation updates existing AWS secret instead of creating new one (#19455) 2026-01-20 18:30:36 -08:00
Ishaan Jaffer
0d75e506c5 fix mock_handler 2026-01-20 18:01:22 -08:00
Ishaan Jaffer
b9de10bd27 test token ctr 2026-01-20 17:53:53 -08:00
yuneng-jiang
96b2d134a7
Revert "[Infra] Changing Google Tests to use Gemini 3 Flash Preview" 2026-01-20 17:32:10 -08:00
Ishaan Jaff
ddebdd47bc
[Feat] Add Support for Claude Code Max/OAuth 2 on LiteLLM AI Gateway (#19453) 
* fix count_tokens_with_anthropic_api

* remove outdated file

* fix ANTHROPIC_TOKEN_COUNTING_BETA_VERSION

* refactor: get_token_counter

* init test suite for token counter

* init token counters

* fix: fix pyrightI

* fix Code QA issues

* feat: add OAUTH handling ant

* feat: Oauth handling Ant

* test anthopic common utils

* fix code QA

* docs
 2026-01-20 17:21:17 -08:00
yuneng-jiang
351e3a5f3c
Merge pull request #19456 from BerriAI/litellm_cicd_fix_yj_05 
[Infra] Changing Google Tests to use Gemini 3 Flash Preview
 2026-01-20 17:15:03 -08:00
yuneng-jiang
0dc92a6979 changing to gemini 3 flash preview 2026-01-20 17:14:06 -08:00
yuneng-jiang
65829411e0 increasing time for spend tracking 2026-01-20 16:31:25 -08:00
yuneng-jiang
d5305c6a61
Merge pull request #19450 from BerriAI/litellm_cicd_fix_yj_02 
[Infra] Fix test_route_checks
 2026-01-20 16:22:20 -08:00