scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-08 22:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 78
31415 commits 4150 branches 1402 tags 2.2 GiB
Commit graph

5891 commits

Author SHA1 Message Date
Sameer Kankute
842e5b3ad6
Merge pull request #19560 from BerriAI/litellm_bedrock_invoke_structured_output 
[Feat] Add support for output formatfor bedrock invoke via v1/messages
 2026-01-22 19:44:48 +05:30
Sameer Kankute
c78c878822
Merge pull request #19558 from BerriAI/litellm_gemini_vertexai_mapping 
Add custom vertex ai mapping to the output
 2026-01-22 19:44:32 +05:30
Sameer Kankute
73715ab417
Merge pull request #19556 from BerriAI/litellm_fix_gemini_batch_jan22 
Fix: generation config empty for batch
 2026-01-22 19:44:26 +05:30
Sameer Kankute
f312bf23d0 Fix:test_multiple_function_call 2026-01-22 19:34:54 +05:30
Sameer Kankute
991fee056f Fix batch tests 2026-01-22 19:23:32 +05:30
Yuta Saito
8a622c51f5 feat: Add MCP tools response to chat completions 2026-01-22 19:23:32 +05:30
Sameer Kankute
b729622bf5 Fix: generationConfig removal from tests 2026-01-22 19:00:37 +05:30
Sameer Kankute
110e2c69d4 Fix : test_anthropic_via_responses_api 2026-01-22 18:28:56 +05:30
Sameer Kankute
caab7821bd Fix: imagegeneration@006 has been deprecated 2026-01-22 18:24:59 +05:30
Sameer Kankute
ad1edd38d5
Merge branch 'main' into litellm_staging_01_21_2026 2026-01-22 17:56:40 +05:30
Sameer Kankute
4d20c8fbc0 feat: Limit stop sequence as per openai spec 2026-01-22 17:52:13 +05:30
Sameer Kankute
24faca9bcf Add support for output formatfor bedrock invoke via v1/messages 2026-01-22 16:36:03 +05:30
Sameer Kankute
18240662db Add custom vertex ai mapping to the output 2026-01-22 15:18:24 +05:30
Yuta Saito
ed67bf2705 feat: Add MCP tools response to chat completions 2026-01-22 15:32:04 +09:00
Will Chen
9f57eb3e74
Fix Azure AI costs for Anthropic models (#19530) 
* Fix Azure AI cost calculation

* fixup
 2026-01-21 21:10:27 -08:00
Emerson Gomes
a3f7f5858b
Fix date overflow/division by zero in proxy utils (#19527) 
* Fix date overflow/division by zero in proxy utils

* Fix projected spend calculation

* Strengthen projected spend tests
 2026-01-21 21:09:57 -08:00
Yogeshwaran Ravichandran
ab274ac3c4
fix(azure response api): flatten tools for responses api to support nested definitions (#19526) 
The Azure Responses API uses a different schema (flattened) for tools compared to the standard OpenAI/Azure Chat Completions API (nested). This caused a `BadRequestError` when users passed standard tool definitions.

Changes:
- Implemented tool flattening logic in `AzureOpenAIResponsesAPIConfig.transform_responses_api_request`.
- Added comprehensive unit tests in test_azure_transformation.py to verify nested-to-flat transformation, pass-through of flat tools, and immutability.
- Ensures cross-provider compatibility for tool definitions.

Fixes #19523
 2026-01-21 21:08:28 -08:00
Emerson Gomes
c8669cf8fa
Fix Azure RPM calculation formula (#19513) 
* Fix Azure RPM calculation formula

* updated test
 2026-01-21 21:03:23 -08:00
João Dinis Ferreira
60840ea292
fix(bedrock): correct streaming choice index for tool calls (#19506) 
Bedrock's contentBlockIndex identifies content blocks within a message
(text=0, tool_call=1), not OpenAI's choice index (which varies with n>1).
This caused OpenAI SDK's ChatCompletionAccumulator to fail when tool call
chunks arrived on index 1 while finish_reason arrived on index 0.

Bedrock doesn't support n>1 (no such parameter exists):
https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InferenceConfiguration.html

OpenAI choice index spec:
https://platform.openai.com/docs/api-reference/chat/streaming
 2026-01-21 20:57:14 -08:00
Harshit Jain
22000f3beb
fix: add case-insensitive support for guardrail mode and actions (#19480) 2026-01-21 20:52:57 -08:00
Ishaan Jaff
a117509f93
[Fix] UI Cost Estimator - Fix model dropdown (#19529) 
* add cost estimator

* ui fix show errors

* test_estimate_cost_resolves_router_model_alias
 2026-01-21 20:19:40 -08:00
Harshit Jain
746414eb9b
Fix/per service ssl override v2 (#19538) 
* refactor(ssl): support per-service SSL verification overrides

* add test cases for ssl
 2026-01-21 20:10:04 -08:00
davida-ps
7777aeb695
fixing prompt-security's guardrail implementation (#19374) 
* Consolidated change

* fix(prompt_security): update message processing to persist sanitized files and filter for API calls

* fix per krrishdholakia suggestion
 2026-01-21 20:09:40 -08:00
Ishaan Jaff
ab606c9a73
[Feat] Add Structured output for /v1/messages with Anthropic API, Azure Anthropic API, Bedrock Converse (#19545) 
* fix: add AnthropicMessagesRequestOptionalParams

* add _update_headers_with_anthropic_beta

* fix output format tests

* test_structured_output_e2e

* TestAnthropicAPIStructuredOutput

* test_structured_output_e2e

* fix BASE

* TestAzureAnthropicStructuredOutput

* fix: Bedrock Converse

* add nthropic Messages Pass-Through Architecture

* fix: bedrock invoke output_format

* fix: transform_anthropic_messages_request for vertex anthropic

* TestBedrockInvokeStructuredOutput

* docs anthropic vertex

* docs fix

* docs fix
 2026-01-21 20:09:18 -08:00
jay prajapati
363b0cc132
fix(azure): preserve content_policy_violation details for images (#19328) (#19372) 
Azure OpenAI Images (DALL·E 3) returns policy violations as a structured payload under body["error"], including inner_error.content_filter_results and revised_prompt.

LiteLLM previously:
- Failed to extract nested error messages (get_error_message only handled body["message"])
- Missed policy violation detection when error strings were generic
- Dropped inner_error details when raising ContentPolicyViolationError

This change:
- Extracts nested Azure error fields (code/type/message + inner_error)
- Detects policy violations via structured error codes
- Passes an OpenAI-style error body + provider_specific_fields to preserve details

Tests:
- python3 -m pytest tests/test_litellm/llms/azure/test_azure_exception_mapping.py
- python3 -m pytest tests/test_litellm/litellm_core_utils/test_exception_mapping_utils.py

Fixes #19328
 2026-01-21 20:06:51 -08:00
jay prajapati
0e738a5027
fix(mcp): forward static_headers to MCP servers (#19341) (#19366) 
Forward static_headers from /mcp-rest/test/* routes into the MCP client so headers are present during session.initialize() and tool discovery.

Also add a shared merge_mcp_headers() helper to keep header precedence consistent and ensure OpenAPI-to-MCP generated tools include static_headers.

Tests:
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_rest_endpoints.py
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py -k register_openapi_tools_includes_static_headers

Fixes #19341

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
 2026-01-21 19:30:55 -08:00
Ishaan Jaff
d5e912322f
[Fix] VertexAI Pass through - Ensure only anthropic betas are forwarded down to LLM API (#19542) 
* fix ALLOWED_VERTEX_AI_PASSTHROUGH_HEADERS

* test_vertex_passthrough_forwards_anthropic_beta_header

* fix test_vertex_passthrough_forwards_anthropic_beta_header

* test_vertex_passthrough_does_not_forward_litellm_auth_token

* fix utils

* Using Anthropic Beta Features on Vertex AI

* test_forward_headers_from_request_x_pass_prefix
 2026-01-21 19:12:04 -08:00
yuneng-jiang
6b6785bc4f
Merge pull request #19539 from BerriAI/litellm_models_scope 
[Feature] Adding Optional scope Param to /models
 2026-01-21 17:41:22 -08:00
yuneng-jiang
c6b157832b
Merge pull request #19296 from BerriAI/litellm_esca_reissue 
[Reissue: Fix] /user/new Privilege Escalation
 2026-01-21 16:46:34 -08:00
yuneng-jiang
6723b30d03 Adding scope to /models 2026-01-21 16:40:31 -08:00
Yuta Saito
898cc3ff4f test: update langfuse trace_id tests to use litellm_trace_id 2026-01-22 06:19:43 +09:00
YutaSaito
4a14a53ae8
Merge pull request #19469 from BerriAI/litellm_feat_mcp_spendlogs 
[feat] mcp spendlogs
 2026-01-22 05:29:21 +09:00
Ishaan Jaff
5cb5969a26
[Fix] LiteLLM VertexAI Pass through - ensuring incoming headers are forwarded down to target (#19524) 
* test_vertex_passthrough_forwards_anthropic_beta_header

* add_incoming_headers
 2026-01-21 12:01:33 -08:00
mubashir1osmani
410daf6e6d added tests 2026-01-21 14:41:35 -05:00
yuneng-jiang
d0e35751a1 Fixing tests and linting 2026-01-21 11:02:39 -08:00
yuneng-jiang
b5a7d2ab34 Paginating model/info endpoint 2026-01-21 10:44:18 -08:00
John Greek
aa4b0e0149
Fix duplicate test_handler.py filenames causing pytest collection errors (#19385) 2026-01-21 08:47:50 -08:00
Sameer Kankute
95dc14e400
Merge pull request #19389 from BerriAI/litellm_timeout_fix 
Fix: total timeout is not respected
 2026-01-21 19:16:11 +05:30
Sameer Kankute
e758dd0a59
Merge pull request #19472 from BerriAI/litellm_fix_chat_completion_responses_streaming 
Fix: tool call streaming in chat completion bridge
 2026-01-21 19:15:53 +05:30
Sameer Kankute
540370a1aa
Merge pull request #19479 from BerriAI/litellm_sarvam_int 
Add support for sarvam models
 2026-01-21 19:03:52 +05:30
Sameer Kankute
b6691baf24
Merge pull request #19482 from BerriAI/litellm_anthropic_max_length 
Filter out unsupported fields from JSON schema for Anthropic's output_format API.
 2026-01-21 18:57:16 +05:30
Sameer Kankute
e315e4cd7c fix litellm/tests/litellm_utils_tests/test_utils.py 2026-01-21 18:50:02 +05:30
Sameer Kankute
aeb38ffa9b
Merge pull request #19497 from BerriAI/main 
merge main
 2026-01-21 18:43:27 +05:30
Sameer Kankute
d879dcdcef
Revert "Fix/prisma schema permission (#19391)" 
This reverts commit 75ee0d126c.
 2026-01-21 17:19:32 +05:30
Sameer Kankute
a1aba2ed8d
Merge pull request #19491 from BerriAI/main 
merge main 20 1 25
 2026-01-21 16:40:48 +05:30
Sameer Kankute
c571a02f3b Filter out unsupported fields from JSON schema for Anthropic's output_format API. 2026-01-21 15:48:22 +05:30
Sameer Kankute
3a0d166eb0 Fix: tool call streaming in chat completino brigde 2026-01-21 12:14:02 +05:30
Yuta Saito
caf5f7f8ae test: add test 2026-01-21 14:51:56 +09:00
Sameer Kankute
a5ea08a0bf Fix test_default_api_base failing because of chatgpt as provider 2026-01-21 09:32:38 +05:30
Ryne Carbone
15013cec4b
feat(gemini): add file content support in tool results (#19416) 
Add support for 'file' and 'input_file' content types in
convert_to_gemini_tool_call_result(). File content in tool
results was previously silently dropped.

Supports base64 data URIs and HTTP URLs, matching the existing
image handling pattern. Enables PDF, audio, video, and other
file types as inline_data for Gemini.
 2026-01-20 19:54:12 -08:00