Commit graph

34202 commits

Author SHA1 Message Date
Yuneng Jiang
da516a11fe
chore: fixes 2026-04-04 22:56:10 -07:00
Cursor Agent
fe223453d3 feat(ui): wrap left nav items in <a> tags for open-in-new-tab support
Nav items are now rendered as <a> elements with proper href attributes,
enabling right-click → 'Open in new tab', Ctrl/Cmd+click, and
middle-click to open any sidebar page in a new browser tab.

Normal clicks continue to use SPA navigation (no full page reload).

Applied to both leftnav.tsx (query-param routing) and Sidebar2.tsx
(Next.js file-based routing).

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-04 03:46:15 +00:00
Cursor Agent
0bc706d71c Add minimal dev_config.yaml for proxy development
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-04 03:06:17 +00:00
Krish Dholakia
90eb6729d5
Agent Tracing - support context_id based trace id propogation + nested llm calls (#22626)
* style(ui/): distinguish agent calls from llm calls on ui

* feat: initial grouping working

* feat: set stable contextid for a2a calls - allows for easily passing to downstream llm/mcp calls

* feat(a2a_endpoints.py): fix tracing to avoid recreating logging objects for the same call

allows stable trace id usage

* fix(guardrail_endpoints): handle string ui_type values in _build_field_dict

_build_field_dict unconditionally called .value on ui_type, which crashes
for guardrail configs that use plain strings (e.g. BlockCodeExecutionGuardrailConfigModel
uses "multiselect" and "percentage"). Now checks with hasattr before calling .value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: propagate trace/session id from headers in MCP server calls

Cherry-picked mcp_server/server.py fixes from 6feb9bab: adds
get_chain_id_from_headers to extract x-litellm-trace-id /
x-litellm-session-id from raw headers, and uses it in call_tool
and list_tools to keep spend logs and tracing consistent with A2A.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:19:12 -08:00
yuneng-jiang
ba7a6d9bfd
Merge pull request #22476 from BerriAI/litellm_audit_pagination_fix
[Fix] UI - Audit Logs: Server-side pagination, filtering, and drawer view
2026-03-03 16:52:33 -08:00
yuneng-jiang
ab101671c7
Merge pull request #22725 from BerriAI/litellm_team_admin_reset_spend
[Fix] Team Admin Reset Key Spend
2026-03-03 16:50:46 -08:00
yuneng-jiang
5fa0c6e994 fix(ui): Copyable JSON blocks in audit drawer; custom table spinner
- Replace plain JsonBlock with CopyableJsonBlock: header row with label
  and copy icon (CheckOutlined on success), pre block below — matches
  the spend logs RequestResponsePanel pattern
- Key-table rows use the same card chrome for visual consistency
- Remove separate "Changes" section label (now redundant with block headers)
- Table loading spinner replaced with custom Spin + LoadingOutlined

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 16:00:46 -08:00
yuneng-jiang
326ff422f1 fix(ui): Audit logs table and drawer polish
Table:
- Use DefaultProxyAdminTag for changed_by column
- Remove tooltips on Object ID and API Key columns
- Rename API Key column to API Key (Hash)
- Move pagination controls to upper-right of filter bar; add icon-only refresh button

Drawer:
- Object ID is now copyable
- API Key (Hash) is copyable and no longer truncated
- Changed By uses DefaultProxyAdminTag
- Expand JSON view boxes from max-h-72 to max-h-96
- Remove unnecessary vertical scrollbar (drop overflow-auto h-full from body div;
  use flex column layout so header and content flow naturally)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:54:46 -08:00
yuneng-jiang
0a1b2635d7 fix: allow team admins to access /key/{key}/reset_spend route
The route-level auth check was blocking internal_user role (team admins)
from reaching /key/{key}/reset_spend because KEY_RESET_SPEND was missing
from key_management_routes. Added it so team admins pass the route check
and the endpoint's existing _check_proxy_or_team_admin_for_key enforces
actual authorization.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:54:34 -08:00
milan-berri
98b9bc8b72
fix: resolve base_model in /cost/estimate for Azure custom deployments (#22724)
The _resolve_model_for_cost_lookup function was only checking
litellm_params.model when resolving model names from the router.
For Azure custom deployment names (e.g. azure/openai/gpt-5.3-codex),
this deployment name doesn't exist in the model cost map, so cost
returned /bin/zsh.

Now checks model_info.base_model and litellm_params.base_model first,
falling back to litellm_params.model only if no base_model is set.
This matches how the router resolves base_model everywhere else.
2026-03-03 15:43:02 -08:00
yuneng-jiang
661f1e16cf
Merge pull request #22715 from BerriAI/litellm_reset_key_spend
[Feature] UI - Virtual Keys: Manual Spend Reset
2026-03-03 15:40:09 -08:00
yuneng-jiang
5b2110ddb5 Polish Reset Spend button and modal
- Move Reset Spend button after Regenerate Key in header
- Make modal OK button danger style with text "Reset"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:16:47 -08:00
yuneng-jiang
98b91080ca Update Reset Spend button style and modal copy
- Use TransactionOutlined icon and danger style to match Delete Key button
- Rewrite modal description: remove assumption about key being blocked,
  clarify spend history is preserved in logs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:12:37 -08:00
Julio Quinteros Pro
d9a349f7d2
Merge pull request #22720 from BerriAI/fix/proxy-utils-mock-request-state
fix: use real State in mock_request for proxy_utils tests
2026-03-03 20:10:53 -03:00
Julio Quinteros Pro
740cdc5c20 fix: use real State object in mock_request to fix _safe_get_request_headers
The _safe_get_request_headers caching (commit e7175a52) uses
request.state._cached_headers. With Mock(spec=Request), getattr on
state returns a Mock (truthy), causing RedactedDict to receive a Mock
instead of a dict. Using a real starlette State object fixes this.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:05:20 -03:00
Julio Quinteros Pro
2415035fcd
Merge pull request #22719 from BerriAI/fix/vertex-response-format-test
fix: account for vertex_ai schema diff in response_format test
2026-03-03 19:59:24 -03:00
Julio Quinteros Pro
5dfa96f62f
Merge pull request #22718 from BerriAI/fix/jwt-test-mock-robustness
fix: use direct AsyncMock in JWT OIDC discovery tests
2026-03-03 19:59:15 -03:00
Julio Quinteros Pro
a07d041881 fix: apply same AsyncMock pattern to remaining OIDC discovery test
Address Greptile review: test_resolve_jwks_url_resolves_oidc_discovery_document
also used the inconsistent patch.object pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:56:51 -03:00
Julio Quinteros Pro
9b92ea16ab fix: update response_format test for vertex_ai's intentional schema diff
Vertex AI / Gemini uses Pydantic's model_json_schema() which omits
additionalProperties: False (Gemini rejects it). The test expected
the same schema for all providers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:55:18 -03:00
Julio Quinteros Pro
eb658693a3 fix: use direct AsyncMock assignment instead of patch.object in JWT tests
The patch.object with new_callable=AsyncMock can behave inconsistently
across Python versions, causing mock_response.status_code to return a
MagicMock instead of the assigned value. Direct assignment is simpler
and more reliable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:51:09 -03:00
Julio Quinteros Pro
e6a1194c28
Merge pull request #22717 from BerriAI/fix/types-utils-video-tokens-test
fix: add video_tokens to completion_tokens_details test
2026-03-03 19:48:44 -03:00
Julio Quinteros Pro
2943df96ba fix: add video_tokens to expected completion_tokens_details in test
The CompletionTokensDetailsWrapper type now includes video_tokens field,
but this test's expected dict was not updated to include it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:46:20 -03:00
Julio Quinteros Pro
fc9d06ceca
Merge pull request #22716 from BerriAI/fix/vertex-function-response-tests
fix: update vertex AI tests for function_response role=user
2026-03-03 19:44:54 -03:00
Julio Quinteros Pro
5a0aba9fb7 fix: update vertex AI tests to expect role=user on function_response messages
The Gemini API requires role="user" on function_response content blocks
(added in commit 273cf12afa), but these tests were never updated to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:40:28 -03:00
yuneng-jiang
5da7fa9ac1 [Feature] UI - Virtual Keys: Add manual spend reset to unblock keys
Adds a "Reset Spend" button to the key detail view so proxy admins and team
admins can immediately reset a key's spend to $0, unblocking keys that have
hit their budget limit without waiting for the next scheduled budget reset.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:39:39 -08:00
Julio Quinteros Pro
2c5c38333d
Merge pull request #22701 from BerriAI/fix/streaming-and-azure-gpt5-test-failures
Fix Anthropic streaming sync and Azure GPT-5.1 logprobs tests
2026-03-03 19:36:44 -03:00
ryan-crabbe
8aa7d53864
Merge pull request #22705 from BerriAI/litellm_feat_batch_expiry_setting_teams
Litellm feat batch expiry setting teams
2026-03-03 14:36:12 -08:00
ryan-crabbe
c8e6428eb7
Update litellm/proxy/openai_files_endpoints/files_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 14:35:58 -08:00
Ryan Crabbe
35d2bc382f fix(batches): suppress PLR0915 lint for create_batch dispatch function 2026-03-03 14:28:25 -08:00
yuneng-jiang
657a60ea5b fix(audit): AND semantics for combined JSON filters; remove unused allTeams prop
- Fix object_team_id + object_key_hash combining incorrectly as OR — each
  filter now adds an AND clause wrapping an internal OR over before_value
  and updated_values, so both conditions must be satisfied simultaneously
- Rename helper to _build_json_field_or_condition to reflect its purpose
- Remove allTeams from AuditLogsProps and its call site in index.tsx

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:24:55 -08:00
Ryan Crabbe
903ade4a1b fix(proxy): add anchor validation for file expiry, key validation for batch expiry
Validate anchor is "created_at" in enforced_file_expires_after (matching
user-provided path). Add key existence validation to batch endpoint for
enforced_batch_output_expires_after.
2026-03-03 13:51:41 -08:00
yuneng-jiang
2d26209d80
Merge pull request #22698 from BerriAI/litellm_test_projects_hooks
[Test] UI - Projects: Add unit tests for project hooks
2026-03-03 13:42:28 -08:00
Ryan Crabbe
d6ad312a4c fix(proxy): validate enforced_file_expires_after keys before access
Add key validation for enforced_file_expires_after to return a clear
400 error instead of an unhandled KeyError 500.
2026-03-03 13:37:11 -08:00
Julio Quinteros Pro
ab718444c5 Remove dead pending_new_content_block attribute
Cleanup per review: this class attribute is no longer used after the
__next__ refactor to queue-based approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:28:34 -03:00
Julio Quinteros Pro
fbf916d71e
Merge pull request #22699 from BerriAI/fix/mypy-count-tokens-signature
Fix mypy override errors in count_tokens signatures
2026-03-03 18:28:14 -03:00
Julio Quinteros Pro
6b4bc99202 Fix Anthropic streaming sync __next__ and Azure GPT-5.1 logprobs
Two independent fixes for pre-existing test failures on main:

1. Anthropic streaming: The sync __next__ method used a simple
   holding_chunk pattern that lost chunks when multiple events needed
   to be returned. Refactored to use the same chunk_queue approach as
   the async __anext__ method. Also fixed tests that used ModelResponse
   (which defaults finish_reason to 'stop') instead of ModelResponseStream.

2. Azure GPT-5.1 logprobs: The base OpenAI class includes logprobs for
   gpt-5.1+ models, but Azure hasn't verified support for gpt-5.1.
   Added explicit removal of logprobs/top_logprobs for gpt-5.1 (non-5.2)
   models in the Azure config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:16:11 -03:00
Julio Quinteros Pro
279b4f16cb Fix mypy override errors in count_tokens signatures
Replace **kwargs with explicit tools and system parameters to match
the BaseTokenCounter.count_tokens abstract method signature.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:15:27 -03:00
Harshit Jain
b62a4be6b6
Merge pull request #22676 from BerriAI/litellm_feat_guardrail-list
feat: guardrail-mode-default-list
2026-03-04 02:36:57 +05:30
yuneng-jiang
4edb1e00c6 [Test] UI - Add unit tests for project hooks
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 12:36:26 -08:00
Harshit28j
d661419109 fix: support list of modes in Mode.default for tag-based guardrails 2026-03-04 01:50:28 +05:30
Ryan Crabbe
08613b24cb feat(proxy): add team-level file expiry enforcement 2026-03-03 11:03:14 -08:00
Ryan Crabbe
3d15bcdb11 feat(proxy): add team-level batch output expiry enforcement 2026-03-03 10:58:31 -08:00
yuneng-jiang
4c1b15d685
Merge pull request #22687 from BerriAI/litellm_fix_spend_log_cleanup
[Fix] Spend Log Cleanup: lock tracking, integer retention, skip log level
2026-03-03 10:48:20 -08:00
Ryan Crabbe
43cec8c980 feat(batches): support output_expires_after passthrough 2026-03-03 10:46:45 -08:00
yuneng-jiang
a1ba6c9fa6 Fix log levels: info for unconfigured, warning for misconfigured
Suppress noisy error log fired every cron tick when spend log cleanup
is simply not configured. _should_delete_spend_logs already logs the
specific reason at the right level (info for None, warning for
invalid value), so the redundant blanket error log in
cleanup_old_spend_logs is removed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 10:21:31 -08:00
yuneng-jiang
224c617119 Fix spend log cleanup: lock tracking, integer retention, skip log level
- Only release distributed lock in finally if it was actually acquired;
  prevents spurious Redis release_lock calls on early returns
- Treat bare integer maximum_spend_logs_retention_period as days (e.g. 3 → "3d")
  instead of silently failing with a ValueError
- Elevate "Skipping cleanup" log from info to error so misconfigured
  retention settings are visible without verbose logging
- Add tests for all three fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 10:12:08 -08:00
yuneng-jiang
b4980340fb
Merge pull request #22680 from BerriAI/litellm_fix_mcp_server_dates
[Fix] MCP server: preserve created_at and updated_at timestamps
2026-03-03 09:53:37 -08:00
yuneng-jiang
76e3dba0f8 fix mcp server created_at and updated_at timestamps being overwritten with current time
- Add created_at field to MCPServer type (was missing)
- Map created_at from LiteLLM_MCPServerTable in build_mcp_server_from_table()
- Use server.created_at and server.updated_at instead of datetime.now() in _build_mcp_server_table() and health check table builder
- Add regression tests to verify timestamps are preserved through round-trip conversions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 09:41:45 -08:00
Harshit28j
22e682b1e8 feat: guardrail-mode-default-list 2026-03-03 22:45:32 +05:30
Sameer Kankute
afc7b87b36
Merge pull request #22656 from BerriAI/litellm_add_vector_store_Ui
Add support for Attaching knowledge base to model via UI
2026-03-03 22:25:59 +05:30