Commit graph

3607 commits

Author SHA1 Message Date
Miguel Armenta
750fc4a980
azure content enhancement... (#22581)
* azure content enhancement...

* rafactored to increase confidence score

* improvements based on additional feedback

* removed unused import

* Force-split any word longer than max length allowed

* preserve whitespace in text splitting

* moving common initialization to base class

* consolidate enforcement into async_make_request as single point, remove redundant caller-side checks, extract shared init/HTTP logic into base, and fix stale log messages

* clean up

* clean up tests
2026-03-04 10:22:30 -08:00
ryan-crabbe
0df36582de
Merge pull request #22728 from BerriAI/litellm_batch_expiry_validation_followup
fix(proxy): improve team expiry enforcement validation
2026-03-04 10:16:02 -08:00
Sameer Kankute
23d312dbd2
Merge pull request #22771 from BerriAI/litellm_responses_websocket_2
Add support for responses websocket for all providers
2026-03-04 22:12:12 +05:30
Julio Quinteros Pro
4ec92ba924 fix: add new model_prices properties to validation schema
Add cache_read_input_token_cost_per_audio_token, supports_code_execution,
and supports_file_search to the JSON schema used by the model prices
validation test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 11:37:02 -03:00
Julio Quinteros Pro
a7e2bc3a92
Merge pull request #22784 from BerriAI/fix/responses-id-security-test
Fix responses ID security test for new request_cache parameter
2026-03-04 11:34:23 -03:00
Julio Quinteros Pro
f4e8c02ba2
Merge pull request #22781 from BerriAI/fix/tool-management-endpoint-tests
Fix tool management tests using wrong field name call_policy
2026-03-04 11:34:04 -03:00
Julio Quinteros Pro
c0ac788709 Fix responses ID security test for new request_cache parameter
The _encrypt_response_id method now receives request_cache=None as a
keyword argument from async_post_call_success_hook. Updated the mock
assertion to expect this parameter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 11:29:51 -03:00
Julio Quinteros Pro
1ec6502f88 Fix tool management tests using wrong field name call_policy
Tests used call_policy throughout but the actual API model uses
input_policy and output_policy. Updated _make_tool_row helper,
list filter query param, and policy update request/response assertions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 11:20:17 -03:00
Julio Quinteros Pro
d22996ee87 Exclude aresponses_websocket from Azure SDK client init test
The aresponses_websocket CallType was recently added but not included
in the test exclusion list. It uses WebSocket passthrough (not Azure SDK
client initialization), so it correctly doesn't call
initialize_azure_sdk_client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 11:13:57 -03:00
Sameer Kankute
d143df8591 Add tests for resp
onses websocket
2026-03-04 18:27:06 +05:30
Sameer Kankute
4e229b7b21
Revert "fix(anthropic): remove hardcoded reasoning summary in adapter" 2026-03-04 17:52:49 +05:30
Sameer Kankute
7d790b39be
Merge pull request #22765 from BerriAI/main
merge main for 030326
2026-03-04 17:40:42 +05:30
Harshit Jain
41b149ee93
Merge pull request #22678 from Harshit28j/litellm_custom_auth_opt_in
fix(proxy): make common_checks opt-in for custom auth
2026-03-04 14:53:44 +05:30
Peter Dave Hello
007bea10b8
Add Support for OpenAI's Chat-GPT 5.3 Chat model (#22693)
Reference:
- https://openai.com/index/gpt-5-3-instant/
- https://developers.openai.com/api/docs/models/gpt-5.3-chat-latest
2026-03-03 20:27:05 -08:00
Ishaan Jaff
7befe3c78f
feat(proxy): add key_alias, key_hash, requested_model DD APM span tags (#22710)
* feat(proxy): add key_alias, key_hash, requested_model tags to DD APM spans

* refactor(proxy): consolidate DD APM tag helpers into DDSpanTagger class

* refactor(proxy): move DDSpanTagger to its own file litellm/proxy/dd_span_tagger.py
2026-03-03 20:22:59 -08:00
Ishaan Jaff
1f412bc6d8
[Feat] Add Tool Policies for AI Gateway (#22732)
* fix: fix ui render

* fix: fix minor bugs

* refactor: use prisma functions instead of raw sql (safer)

* fix(add-new-tiles-to-tool-policies): allow developer to see what's available

* feat: ensure tool allowlist runs correctly for tool names + mcp's

* refactor: more ui improvements

* feat: working key tool blocking

* feat(tools): show tool logs

* refactor: backend code improvements

* refactor: improve log viewer for tools

* fix: address PR review feedback for tool access control

- Add missing blocked_tools column to root schema.prisma (schema drift)
- Invalidate ToolPolicyRegistry after policy mutations so changes take effect immediately
- Remove dead code: unused get_effective_policies, get_tool_policies_cached, and helpers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: race condition in permission resolution and remove duplicate allowlist check

- Use atomic update_many with object_permission_id=None to prevent concurrent
  requests from creating orphaned permission rows and losing tool blocks
- Remove duplicate allowed_tools enforcement from guardrail (already enforced
  in auth layer via check_tools_allowlist)
- Move inline uuid import to module level

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* update to account for  userAgent

* UI - Add ToolDetails

* input/output policy

* LiteLLM_PolicyAttachmentTable

* LiteLLM_PolicyAttachmentTable

* fix: add _enqueue_tool_registry_upsert

* fix: tool mgmt endpoints

* tool mgmt endpoints

* Update tests/test_litellm/proxy/db/test_tool_registry_writer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update tests/test_litellm/proxy/db/test_tool_registry_writer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update tests/test_litellm/proxy/db/test_tool_registry_writer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: sync root schema.prisma and fix test_tool_registry_writer for input/output policy

- Migrate root schema.prisma LiteLLM_ToolTable from call_policy to
  input_policy/output_policy, add missing user_agent and last_used_at columns
  (now consistent with litellm/proxy/schema.prisma and litellm-proxy-extras)
- Fix SpendLogToolIndex comment across all three schema files
- Fix all call_policy references in test_tool_registry_writer.py:
  swapped update_tool_policy arguments, wrong get_tools_by_names return type
  assertions, _mock_tool_row setting call_policy instead of input_policy

Addresses Greptile review feedback on PR #22732.

Made-with: Cursor

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 20:22:20 -08:00
Sameer Kankute
120201049e
Merge pull request #22666 from Point72/ephrimstanley/batch-fixes-mar3
Managed batches - Address PR bot comments from #22464
2026-03-04 09:06:41 +05:30
Sameer Kankute
7a83acf086
Merge pull request #22620 from OiPunk/codex/litellm-22619-deepinfra-function-calling
fix: add missing supports_function_calling for deepinfra models
2026-03-04 08:51:21 +05:30
Krish Dholakia
90eb6729d5
Agent Tracing - support context_id based trace id propogation + nested llm calls (#22626)
* style(ui/): distinguish agent calls from llm calls on ui

* feat: initial grouping working

* feat: set stable contextid for a2a calls - allows for easily passing to downstream llm/mcp calls

* feat(a2a_endpoints.py): fix tracing to avoid recreating logging objects for the same call

allows stable trace id usage

* fix(guardrail_endpoints): handle string ui_type values in _build_field_dict

_build_field_dict unconditionally called .value on ui_type, which crashes
for guardrail configs that use plain strings (e.g. BlockCodeExecutionGuardrailConfigModel
uses "multiselect" and "percentage"). Now checks with hasattr before calling .value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: propagate trace/session id from headers in MCP server calls

Cherry-picked mcp_server/server.py fixes from 6feb9bab: adds
get_chain_id_from_headers to extract x-litellm-trace-id /
x-litellm-session-id from raw headers, and uses it in call_tool
and list_tools to keep spend logs and tracing consistent with A2A.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:19:12 -08:00
Cesar Garcia
4ab79451d9
Merge pull request #22552 from shanemort1982/fix/register-model-custom-pricing-fields
fix: pass all custom pricing fields to register_model in completion() and embedding()
2026-03-03 22:40:45 -03:00
Ryan Crabbe
52ec73c07d fix(proxy): improve team expiry enforcement validation
- Change status codes from 400 to 500 for team metadata misconfig errors
  (callers can't fix admin-set config, 400 is misleading)
- Add anchor value validation to batch endpoint (matching files endpoint)
- Coerce seconds to int to handle string values from metadata
- Add error-path tests: missing keys, invalid anchor, status code assertions
- Add happy-path test: team injects expiry when caller sends nothing
2026-03-03 17:29:39 -08:00
yuneng-jiang
ab101671c7
Merge pull request #22725 from BerriAI/litellm_team_admin_reset_spend
[Fix] Team Admin Reset Key Spend
2026-03-03 16:50:46 -08:00
yuneng-jiang
0a1b2635d7 fix: allow team admins to access /key/{key}/reset_spend route
The route-level auth check was blocking internal_user role (team admins)
from reaching /key/{key}/reset_spend because KEY_RESET_SPEND was missing
from key_management_routes. Added it so team admins pass the route check
and the endpoint's existing _check_proxy_or_team_admin_for_key enforces
actual authorization.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:54:34 -08:00
milan-berri
98b9bc8b72
fix: resolve base_model in /cost/estimate for Azure custom deployments (#22724)
The _resolve_model_for_cost_lookup function was only checking
litellm_params.model when resolving model names from the router.
For Azure custom deployment names (e.g. azure/openai/gpt-5.3-codex),
this deployment name doesn't exist in the model cost map, so cost
returned /bin/zsh.

Now checks model_info.base_model and litellm_params.base_model first,
falling back to litellm_params.model only if no base_model is set.
This matches how the router resolves base_model everywhere else.
2026-03-03 15:43:02 -08:00
Julio Quinteros Pro
2415035fcd
Merge pull request #22719 from BerriAI/fix/vertex-response-format-test
fix: account for vertex_ai schema diff in response_format test
2026-03-03 19:59:24 -03:00
Julio Quinteros Pro
5dfa96f62f
Merge pull request #22718 from BerriAI/fix/jwt-test-mock-robustness
fix: use direct AsyncMock in JWT OIDC discovery tests
2026-03-03 19:59:15 -03:00
Julio Quinteros Pro
a07d041881 fix: apply same AsyncMock pattern to remaining OIDC discovery test
Address Greptile review: test_resolve_jwks_url_resolves_oidc_discovery_document
also used the inconsistent patch.object pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:56:51 -03:00
Julio Quinteros Pro
9b92ea16ab fix: update response_format test for vertex_ai's intentional schema diff
Vertex AI / Gemini uses Pydantic's model_json_schema() which omits
additionalProperties: False (Gemini rejects it). The test expected
the same schema for all providers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:55:18 -03:00
Julio Quinteros Pro
eb658693a3 fix: use direct AsyncMock assignment instead of patch.object in JWT tests
The patch.object with new_callable=AsyncMock can behave inconsistently
across Python versions, causing mock_response.status_code to return a
MagicMock instead of the assigned value. Direct assignment is simpler
and more reliable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:51:09 -03:00
Julio Quinteros Pro
e6a1194c28
Merge pull request #22717 from BerriAI/fix/types-utils-video-tokens-test
fix: add video_tokens to completion_tokens_details test
2026-03-03 19:48:44 -03:00
Julio Quinteros Pro
2943df96ba fix: add video_tokens to expected completion_tokens_details in test
The CompletionTokensDetailsWrapper type now includes video_tokens field,
but this test's expected dict was not updated to include it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:46:20 -03:00
Julio Quinteros Pro
fc9d06ceca
Merge pull request #22716 from BerriAI/fix/vertex-function-response-tests
fix: update vertex AI tests for function_response role=user
2026-03-03 19:44:54 -03:00
Julio Quinteros Pro
5a0aba9fb7 fix: update vertex AI tests to expect role=user on function_response messages
The Gemini API requires role="user" on function_response content blocks
(added in commit 273cf12afa), but these tests were never updated to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:40:28 -03:00
Julio Quinteros Pro
2c5c38333d
Merge pull request #22701 from BerriAI/fix/streaming-and-azure-gpt5-test-failures
Fix Anthropic streaming sync and Azure GPT-5.1 logprobs tests
2026-03-03 19:36:44 -03:00
ryan-crabbe
8aa7d53864
Merge pull request #22705 from BerriAI/litellm_feat_batch_expiry_setting_teams
Litellm feat batch expiry setting teams
2026-03-03 14:36:12 -08:00
Julio Quinteros Pro
6b4bc99202 Fix Anthropic streaming sync __next__ and Azure GPT-5.1 logprobs
Two independent fixes for pre-existing test failures on main:

1. Anthropic streaming: The sync __next__ method used a simple
   holding_chunk pattern that lost chunks when multiple events needed
   to be returned. Refactored to use the same chunk_queue approach as
   the async __anext__ method. Also fixed tests that used ModelResponse
   (which defaults finish_reason to 'stop') instead of ModelResponseStream.

2. Azure GPT-5.1 logprobs: The base OpenAI class includes logprobs for
   gpt-5.1+ models, but Azure hasn't verified support for gpt-5.1.
   Added explicit removal of logprobs/top_logprobs for gpt-5.1 (non-5.2)
   models in the Azure config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:16:11 -03:00
Ryan Crabbe
08613b24cb feat(proxy): add team-level file expiry enforcement 2026-03-03 11:03:14 -08:00
Ryan Crabbe
3d15bcdb11 feat(proxy): add team-level batch output expiry enforcement 2026-03-03 10:58:31 -08:00
yuneng-jiang
4c1b15d685
Merge pull request #22687 from BerriAI/litellm_fix_spend_log_cleanup
[Fix] Spend Log Cleanup: lock tracking, integer retention, skip log level
2026-03-03 10:48:20 -08:00
Ryan Crabbe
43cec8c980 feat(batches): support output_expires_after passthrough 2026-03-03 10:46:45 -08:00
Cesar Garcia
8a1e915bb6
Merge pull request #22403 from shivaaang/fix/openrouter-image-edit
feat(openrouter): add image edit support for OpenRouter models
2026-03-03 15:31:04 -03:00
Cesar Garcia
a8b5a876bf
Merge pull request #21491 from Chesars/fix/20998-remove-hardcoded-reasoning-summary
fix(anthropic): remove hardcoded reasoning summary in adapter
2026-03-03 15:29:11 -03:00
Chesars
833c1bc45d merge: resolve conflict with staging, remove hardcoded summary from reasoning test 2026-03-03 15:28:45 -03:00
Cesar Garcia
b2c7d2e049
Merge pull request #21577 from Chesars/fix/gemini-streaming-tool-calls-finish-reason
fix(gemini): correct streaming finish_reason for tool calls
2026-03-03 15:25:24 -03:00
Cesar Garcia
da941e4261
Merge pull request #22589 from Chesars/fix/vertex-preserve-any-type-schema
fix(vertex): preserve type schema semantics for JsonValuefields
2026-03-03 15:19:16 -03:00
yuneng-jiang
224c617119 Fix spend log cleanup: lock tracking, integer retention, skip log level
- Only release distributed lock in finally if it was actually acquired;
  prevents spurious Redis release_lock calls on early returns
- Treat bare integer maximum_spend_logs_retention_period as days (e.g. 3 → "3d")
  instead of silently failing with a ValueError
- Elevate "Skipping cleanup" log from info to error so misconfigured
  retention settings are visible without verbose logging
- Add tests for all three fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 10:12:08 -08:00
yuneng-jiang
76e3dba0f8 fix mcp server created_at and updated_at timestamps being overwritten with current time
- Add created_at field to MCPServer type (was missing)
- Map created_at from LiteLLM_MCPServerTable in build_mcp_server_from_table()
- Use server.created_at and server.updated_at instead of datetime.now() in _build_mcp_server_table() and health check table builder
- Add regression tests to verify timestamps are preserved through round-trip conversions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 09:41:45 -08:00
Harshit28j
b44755db96 fix(proxy): make common_checks opt-in for custom auth via custom_auth_run_common_checks
Replaces the skip_route_check approach from PR #22662 with a configurable
opt-in flag. By default, common_checks() is not run for custom auth flows,
preserving backwards compatibility with pre-#22164 behavior.

Users who want budget/team/route enforcement on custom auth can enable it:
  general_settings:
    custom_auth_run_common_checks: true

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 22:50:34 +05:30
Sameer Kankute
daa0397f30
Merge pull request #22674 from BerriAI/litellm_gemini-3.1-flash-lite-preview
Day 0 gemini 3.1 flash lite preview support
2026-03-03 22:23:17 +05:30
Ephrim Stanley
b83373d29c Managed batches - Address PR bot comments from #22464 2026-03-03 11:01:49 -05:00