scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-24 00:52:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 91
30984 commits 2663 branches 1420 tags 3.4 GiB
Commit graph

2456 commits

Author SHA1 Message Date
yuneng-jiang
5e395db1dc
Merge pull request #19604 from BerriAI/litellm_team_update_org 
[Fix] Team Update with Organization having All Proxy Models
 2026-01-24 09:09:55 -08:00
yuneng-jiang
f88a32de05
Merge pull request #19622 from BerriAI/litellm_ui_model_backend 
[Feature] UI - Models Page: Model Search
 2026-01-24 09:09:03 -08:00
yuneng-jiang
b44ac6c682 Fixing ruff check 2026-01-24 09:08:29 -08:00
yuneng-jiang
63166c3acc fixing arize tests 2026-01-23 23:13:21 -08:00
Ishaan Jaff
a870722f65
[Feat] UI + Backend - Allow adding policies on Keys/Teams + Viewing on Info panels (#19688) 
* ui for policy mgmt

* test_add_guardrails_from_policy_engine_accepts_dynamic_policies_and_pops_from_data
 2026-01-23 19:03:44 -08:00
ryan-crabbe
d67d12fc54
perf: Add LRU caching to get_model_info for faster cost lookups (#19606) 
- Add @lru_cache decorator to get_model_info() and _cached_get_model_info_helper()
- Update _invalidate_model_cost_lowercase_map() to clear these caches when model_cost changes
- Update test to call cache invalidation after modifying litellm.model_cost

Reduces get_model_cost_information from 46% to <1% of request handling time.
 2026-01-23 17:26:45 -08:00
yuneng-jiang
8b5b343841 attempt fix flaky tests 2026-01-23 12:10:08 -08:00
xqe2011
ca8c2c3938
fix #19620: SSO user roles are not updated for existing users (#19621) 
* Fix: SSO user roles are not updated for existing users
Fixes #19620

* Refactor: Remove redundant user_info retrieval in SSOAuthenticationHandler

* Test: add new tests for user creation and updates in get_user_info_from_db
 2026-01-23 09:05:29 -08:00
Sameer Kankute
9894721285
Merge pull request #19548 from BerriAI/litellm_staging_01_22_2026 
Litellm staging 01 22 2026
 2026-01-23 20:03:11 +05:30
YutaSaito
8ac1d96d90
Merge pull request #19634 from BerriAI/litellm_feat_hashicorp_rotate 
[feat] hashicorp vault rotate support
 2026-01-23 21:08:55 +09:00
Sameer Kankute
12463809bd
Merge pull request #19638 from BerriAI/main 
merge main in stagin 1 22 26
 2026-01-23 14:54:17 +05:30
Yuta Saito
695fbf4ec5 feat: hashicorp vault rotate support 2026-01-23 17:32:55 +09:00
Yuta Saito
919033a6d0 fix: include tool arguments in proxy_server_request for spend logs callbacks 2026-01-23 16:36:37 +09:00
YutaSaito
12bc66aa5b
Merge pull request #19623 from BerriAI/litellm_fix_completions_mcp_output_ordering 
[fix] completions mcp output ordering
 2026-01-23 15:56:02 +09:00
Yuta Saito
6a60b3d848 test: completions mcp output test 2026-01-23 15:17:14 +09:00
yuneng-jiang
3ee7aab5f2 All Models Backend Search 2026-01-22 22:00:22 -08:00
John Greek
26a2c90818
[Fix] Anthropic models on Azure AI cache pricing (#19532) (#19614) 2026-01-22 20:00:40 -08:00
Harshit Jain
69c8698e62
fix: pass through endpoints update registry (#19420) 
* fix: pass through endpoints update registry

* add test case, fix lint error and comment to avoid confusion

* fix pass through endpoints test case
 2026-01-22 19:57:48 -08:00
Ishaan Jaff
c23e4b87dc
[Feat] New LiteLLM Policy engine - create policies to manage guardrails, conditions - permissions per Key, Team (#19612) 
* init PolicyMatcher

* TestPolicyMatcherGetMatchingPolicies

* TestPolicyMatcherGetMatchingPolicies

* feat: init PolicyResolver

* init resolver types

* init policy from config

* inint PolicyValidator

* validate policy

* init Architecture Diagram

* test_add_guardrails_from_policy_engine

* init _init_policy_engine

* test updates

* test fixws

* new attachment config

* simplify types

* TestPolicyResolverInheritance

* fix policy resolver

* fix policies

* fix applied policy

* docs fix

* docs fix

* fix linting + QA checks

* fix linting + QA fixes

* test fixes
 2026-01-22 19:49:53 -08:00
yuneng-jiang
f78fc4e0fe Fix org all proxy model case 2026-01-22 15:32:10 -08:00
mpcusack-altos
88f8f49e1d
fix(websearch_interception): filter internal kwargs before follow-up request (#19577) 
The websearch interception handler was passing internal flags like
`_websearch_interception_converted_stream` to the follow-up LLM request.
This caused "Extra inputs are not permitted" errors from providers like
Bedrock that use strict Pydantic validation.

Fix: Filter out all kwargs starting with `_websearch_interception` prefix
before making the follow-up anthropic_messages.acreate() call.
 2026-01-22 10:42:20 -08:00
Eric Cao
a51835dfcc
Metrics prometheus user team count (#19520) 
* add user count and team count prometheus metrics

* rebase

* revert mistaken deletion
 2026-01-22 08:17:15 -08:00
Sameer Kankute
842e5b3ad6
Merge pull request #19560 from BerriAI/litellm_bedrock_invoke_structured_output 
[Feat] Add support for output formatfor bedrock invoke via v1/messages
 2026-01-22 19:44:48 +05:30
Sameer Kankute
c78c878822
Merge pull request #19558 from BerriAI/litellm_gemini_vertexai_mapping 
Add custom vertex ai mapping to the output
 2026-01-22 19:44:32 +05:30
Sameer Kankute
73715ab417
Merge pull request #19556 from BerriAI/litellm_fix_gemini_batch_jan22 
Fix: generation config empty for batch
 2026-01-22 19:44:26 +05:30
Sameer Kankute
f312bf23d0 Fix:test_multiple_function_call 2026-01-22 19:34:54 +05:30
Sameer Kankute
991fee056f Fix batch tests 2026-01-22 19:23:32 +05:30
Yuta Saito
8a622c51f5 feat: Add MCP tools response to chat completions 2026-01-22 19:23:32 +05:30
Sameer Kankute
ad1edd38d5
Merge branch 'main' into litellm_staging_01_21_2026 2026-01-22 17:56:40 +05:30
Sameer Kankute
24faca9bcf Add support for output formatfor bedrock invoke via v1/messages 2026-01-22 16:36:03 +05:30
Sameer Kankute
18240662db Add custom vertex ai mapping to the output 2026-01-22 15:18:24 +05:30
Yuta Saito
ed67bf2705 feat: Add MCP tools response to chat completions 2026-01-22 15:32:04 +09:00
Will Chen
9f57eb3e74
Fix Azure AI costs for Anthropic models (#19530) 
* Fix Azure AI cost calculation

* fixup
 2026-01-21 21:10:27 -08:00
Emerson Gomes
a3f7f5858b
Fix date overflow/division by zero in proxy utils (#19527) 
* Fix date overflow/division by zero in proxy utils

* Fix projected spend calculation

* Strengthen projected spend tests
 2026-01-21 21:09:57 -08:00
Yogeshwaran Ravichandran
ab274ac3c4
fix(azure response api): flatten tools for responses api to support nested definitions (#19526) 
The Azure Responses API uses a different schema (flattened) for tools compared to the standard OpenAI/Azure Chat Completions API (nested). This caused a `BadRequestError` when users passed standard tool definitions.

Changes:
- Implemented tool flattening logic in `AzureOpenAIResponsesAPIConfig.transform_responses_api_request`.
- Added comprehensive unit tests in test_azure_transformation.py to verify nested-to-flat transformation, pass-through of flat tools, and immutability.
- Ensures cross-provider compatibility for tool definitions.

Fixes #19523
 2026-01-21 21:08:28 -08:00
João Dinis Ferreira
60840ea292
fix(bedrock): correct streaming choice index for tool calls (#19506) 
Bedrock's contentBlockIndex identifies content blocks within a message
(text=0, tool_call=1), not OpenAI's choice index (which varies with n>1).
This caused OpenAI SDK's ChatCompletionAccumulator to fail when tool call
chunks arrived on index 1 while finish_reason arrived on index 0.

Bedrock doesn't support n>1 (no such parameter exists):
https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InferenceConfiguration.html

OpenAI choice index spec:
https://platform.openai.com/docs/api-reference/chat/streaming
 2026-01-21 20:57:14 -08:00
Harshit Jain
746414eb9b
Fix/per service ssl override v2 (#19538) 
* refactor(ssl): support per-service SSL verification overrides

* add test cases for ssl
 2026-01-21 20:10:04 -08:00
davida-ps
7777aeb695
fixing prompt-security's guardrail implementation (#19374) 
* Consolidated change

* fix(prompt_security): update message processing to persist sanitized files and filter for API calls

* fix per krrishdholakia suggestion
 2026-01-21 20:09:40 -08:00
jay prajapati
363b0cc132
fix(azure): preserve content_policy_violation details for images (#19328) (#19372) 
Azure OpenAI Images (DALL·E 3) returns policy violations as a structured payload under body["error"], including inner_error.content_filter_results and revised_prompt.

LiteLLM previously:
- Failed to extract nested error messages (get_error_message only handled body["message"])
- Missed policy violation detection when error strings were generic
- Dropped inner_error details when raising ContentPolicyViolationError

This change:
- Extracts nested Azure error fields (code/type/message + inner_error)
- Detects policy violations via structured error codes
- Passes an OpenAI-style error body + provider_specific_fields to preserve details

Tests:
- python3 -m pytest tests/test_litellm/llms/azure/test_azure_exception_mapping.py
- python3 -m pytest tests/test_litellm/litellm_core_utils/test_exception_mapping_utils.py

Fixes #19328
 2026-01-21 20:06:51 -08:00
jay prajapati
0e738a5027
fix(mcp): forward static_headers to MCP servers (#19341) (#19366) 
Forward static_headers from /mcp-rest/test/* routes into the MCP client so headers are present during session.initialize() and tool discovery.

Also add a shared merge_mcp_headers() helper to keep header precedence consistent and ensure OpenAPI-to-MCP generated tools include static_headers.

Tests:
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_rest_endpoints.py
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py -k register_openapi_tools_includes_static_headers

Fixes #19341

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
 2026-01-21 19:30:55 -08:00
Ishaan Jaff
d5e912322f
[Fix] VertexAI Pass through - Ensure only anthropic betas are forwarded down to LLM API (#19542) 
* fix ALLOWED_VERTEX_AI_PASSTHROUGH_HEADERS

* test_vertex_passthrough_forwards_anthropic_beta_header

* fix test_vertex_passthrough_forwards_anthropic_beta_header

* test_vertex_passthrough_does_not_forward_litellm_auth_token

* fix utils

* Using Anthropic Beta Features on Vertex AI

* test_forward_headers_from_request_x_pass_prefix
 2026-01-21 19:12:04 -08:00
yuneng-jiang
6b6785bc4f
Merge pull request #19539 from BerriAI/litellm_models_scope 
[Feature] Adding Optional scope Param to /models
 2026-01-21 17:41:22 -08:00
yuneng-jiang
c6b157832b
Merge pull request #19296 from BerriAI/litellm_esca_reissue 
[Reissue: Fix] /user/new Privilege Escalation
 2026-01-21 16:46:34 -08:00
yuneng-jiang
6723b30d03 Adding scope to /models 2026-01-21 16:40:31 -08:00
YutaSaito
4a14a53ae8
Merge pull request #19469 from BerriAI/litellm_feat_mcp_spendlogs 
[feat] mcp spendlogs
 2026-01-22 05:29:21 +09:00
Ishaan Jaff
5cb5969a26
[Fix] LiteLLM VertexAI Pass through - ensuring incoming headers are forwarded down to target (#19524) 
* test_vertex_passthrough_forwards_anthropic_beta_header

* add_incoming_headers
 2026-01-21 12:01:33 -08:00
yuneng-jiang
d0e35751a1 Fixing tests and linting 2026-01-21 11:02:39 -08:00
yuneng-jiang
b5a7d2ab34 Paginating model/info endpoint 2026-01-21 10:44:18 -08:00
John Greek
aa4b0e0149
Fix duplicate test_handler.py filenames causing pytest collection errors (#19385) 2026-01-21 08:47:50 -08:00
Sameer Kankute
e758dd0a59
Merge pull request #19472 from BerriAI/litellm_fix_chat_completion_responses_streaming 
Fix: tool call streaming in chat completion bridge
 2026-01-21 19:15:53 +05:30