Commit graph

52881 commits

Author SHA1 Message Date
kerry
4036b769a9 fix(proxy): count unprefixed release bullets and coalesce concurrent latest release fetches
Unprefixed release bullets now count as other_updates, concurrent cache misses share one upstream GitHub request through an injected asyncio.Lock, and the dashboard upgrade banner is announced as status rather than alert

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:22:15 +00:00
kerry
f836bb481d test(integration): keep cost diagnostics and widen shard timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:19:11 +00:00
kerry
1961cbcb6c fix(timing): subtract every provider attempt from receive-anchored overhead
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:18:13 +00:00
ryan-crabbe-berri
1073b9eff7
Merge pull request #41354 from BerriAI/litellm_lit_3269_project_spend_tracking
fix(proxy): track project spend and enforce project budgets additively
2026-09-18 17:17:28 -07:00
kerry
69f9106759 test(integration): move scripted-provider cost suite into cost shard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:16:12 +00:00
kerry
92d82841fd chore(model_info): backfill reseller Gemini entries from provider catalogs and prune retired ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:14:40 +00:00
yassin
c32309fb2d feat(ui): show MCP allowed clients as cards edited in a dialog
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:13:44 +00:00
mateo-berri
12120fe59b refactor(bedrock): inline maxTokens clamp and cover inference-profile ARNs in tests 2026-09-18 17:11:29 -07:00
kerry
83d89aa134 test(integration): cover off-peak pricing on a live proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:10:22 +00:00
kerry
37da5b6f4d fix(bedrock): sign batch retrieve and cancel with deployment credentials when AWS_BEARER_TOKEN_BEDROCK is set
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:10:17 +00:00
Yujong Lee
b6b5ef00cc chore: merge main into litellm_rust_http_pool_ocr
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:10:17 +00:00
mateo-berri
febe9aec65 fix(responses): book a rejected WebSocket connection as a failed request 2026-09-18 17:10:01 -07:00
joshua
7410d00d2e merge(main): resolve mcp server.py import conflict and relock
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:08:33 +00:00
yucheng-berri
500e880a40
Merge pull request #41895 from BerriAI/litellm_openai_moderations_model_default
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 17:06:41 -07:00
ryan-crabbe-berri
4bb1ae115b
Merge pull request #41347 from BerriAI/litellm_team_member_budget_link_default
fix(team): apply team_member_budget updates to members still on the team default
2026-09-18 17:06:25 -07:00
kerry
6d532f2549 Merge remote-tracking branch 'origin/main' into litellm_upgrade_banner_changelog_stats 2026-09-19 00:04:07 +00:00
jesus
82e3f3980d refactor(auth): resolve org identity through an auth_checks helper
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:59:33 +00:00
mateo-berri
dc11c34e3a Merge remote-tracking branch 'origin/main' into litellm_vertex_chirp3_streaming_stt 2026-09-18 16:56:02 -07:00
mateo-berri
3d7a771ea7 fix(vertex_ai): apply finals before interims and refresh the token per stream 2026-09-18 16:53:06 -07:00
joshua
8d8efe7203 style(mcp): satisfy lint and type budgets for the SDK 2 port
Format the ported files, annotate mutable wire payloads, give the e2e
OAuth client the SDK 2 httpx2/AuthorizationCodeResult API, tighten the
transport-streams alias to the two-stream SDK 2 shape, and add a
test-quality reason for the MockTransport factory injection.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:49:23 +00:00
joshua
8d8a2c3742 ci(mcp): add dependency-resolution workflow for the SDK 2 floor
New matrix job across Python 3.10-3.14 verifies uv.lock against the
declared floors, installs the locked mcp+proxy extras and runs the MCP
unit suites, then resolves the same extras with uv's lowest-direct
strategy into a clean venv and runs scripts/check_mcp_sdk_install.py to
prove the floor still imports the SDK 2 API surface.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:49:19 +00:00
Moe Khalil
e0b2c51144 fix(auto-router): validate JEV usage and clear stale context
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:41:20 +00:00
Mateo Wang
ff7dc86947
Merge pull request #41892 from BerriAI/litellm_gemini_contentless_candidate_finish_reason
Some checks failed
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Issue label sync / sync-issue-labels-tests (push) Has been cancelled
Issue label sync / sync-issue-labels (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
VS Code Extension / vscode-extension (push) Has been cancelled
fix(gemini): preserve candidates with finishReason and no content (#40477)
2026-09-18 16:34:46 -07:00
joshua
0d2963fe89 test(mcp): update MCP suites for SDK2 handler signatures and ctx var
Call handlers with ServerRequestContext and params models, seed the
litellm contextvar instead of the removed SDK request_ctx, forward
headers/auth through the httpx2 MockTransport factory, and add
regressions for handler registration, context propagation, and modern
protocol-version rejection.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:34:30 +00:00
joshua
783038010b refactor(mcp): register SDK2 request handlers and drop request_ctx ContextVar
Port the proxy MCP server off the removed SDK1 decorator API. Handlers now
take (ctx, params), are registered via add_request_handler, and return full
result models. Request-scoped session/context propagation moves to a
litellm-owned active_mcp_request_ctx_var ContextVar set at handler entry.
Reject MCP-Protocol-Version values outside the SDK2 handshake set with a
400 before session-manager delegation. Fold SDK2 MCPError-wrapped parse
and content-type failures into the existing connection diagnostics.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:34:30 +00:00
yucheng
c7028761aa fix(proxy): keep queued moderation running past a V1 pre_call guardrail
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
A V1 CustomGuardrail with moderation_check pre_call returned out of
during_call_hook before asyncio.gather, abandoning already-queued
CustomLogger moderation coroutines and skipping every later callback.
Skip only that guardrail instead.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:33:29 +00:00
Yujong Lee
5a474fd799 refactor(rust): inject VertexAuth into OcrClient so the bridge keeps one token cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:31:27 +00:00
yucheng
c4272d894f style(team): wrap the roster audit helper comprehensions at 120 columns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:29:11 +00:00
yucheng
209eba6718 feat(team): carry team_alias on member add, delete and role-change audit payloads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:28:39 +00:00
Yujong Lee
abb9618971 feat(rust): add litellm-http client pool and inject it into the OCR route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:28:11 +00:00
mateo-berri
c6c8aed3f8 fix(proxy): drop only the daily spend batch whose failure cannot be re-sent, requeue the unsent ones 2026-09-18 16:22:04 -07:00
yucheng
e5d02fecf7 chore: merge litellm_openai_moderations_model_default (#41895) into litellm_prompt_injection_llm_api_check_dispatch 2026-09-18 23:19:21 +00:00
mateo-berri
01d8d3c218 fix(claude_code_gateway): wrap managed settings in the uuid, checksum, settings envelope the client requires
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-18 16:18:15 -07:00
yujonglee
018f640b30
Merge pull request #41885 from BerriAI/litellm_rust_callback_contract
refactor(rust): formalize legacy callback contract
2026-09-18 16:16:02 -07:00
Mateo Wang
ec05cd0128
Merge pull request #41887 from BerriAI/litellm_gemma_4_26b_maas_context_window
fix: set vertex gemma-4-26b-a4b-it-maas context window to 262144
2026-09-18 16:14:56 -07:00
mateo-berri
f143a46cdf chore: merge main into litellm_lit8064_unpin_derived_pricing 2026-09-18 16:11:37 -07:00
ryan-crabbe-berri
4ab23d7343
Merge pull request #41894 from BerriAI/litellm_remove_agent_shin
ci: remove the dead Agent Shin triage workflows and scripts
2026-09-18 16:10:35 -07:00
yucheng-berri
8e93031c19
Merge pull request #41786 from BerriAI/litellm_passthrough_xpass_trace
Pass-through requests inject the proxy span into upstream headers since #40669, which
replaced an explicit x-pass-traceparent with an unrelated trace and dropped its
x-pass-tracestate. Keep the caller's context when the carrier already names a
different trace, and keep the proxy child span for same-trace or missing headers.

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 16:07:28 -07:00
yucheng
1704aeebb4 fix(enterprise): resolve openai_moderations model at call time and default to omni-moderation-latest
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:06:47 +00:00
mateo-berri
4356fc58d8 chore(proxy): restore the CI-generated lazy OpenAPI snapshot 2026-09-18 16:05:57 -07:00
mateo-berri
5db2a0c885 test(proxy): type the sqlstate test parameters 2026-09-18 16:05:42 -07:00
mateo-berri
87ac68709c fix(claude_code_gateway): single-use device codes across replicas, protobuf telemetry, CLI user route access 2026-09-18 16:05:16 -07:00
Mateo Wang
3490754e65
Merge pull request #41871 from BerriAI/litellm_bedrock_eager_input_streaming
feat: honor eager_input_streaming on Bedrock and Anthropic Claude tools
2026-09-18 16:05:10 -07:00
ryan
b4c5f6fa44 Merge remote-tracking branch 'origin/main' into litellm_team_member_budget_link_default
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm/proxy/management_helpers/utils.py
#	tests/test_litellm/proxy/management_helpers/test_management_helpers_utils.py
2026-09-18 23:05:03 +00:00
Mateo Wang
ec9435cbf4
Merge pull request #41881 from BerriAI/litellm_responses_ws_deployment_defaults
fix(responses): merge deployment litellm_params into native websocket response.create frames
2026-09-18 16:04:39 -07:00
mateo-berri
a4624b6c6c fix(gemini): derive the finish reason key set from the Candidates type
Candidates.finishReason listed eleven values while the mapping key set
carried twenty-one, so typed fixtures could not spell the reasons this
PR handles. GeminiFinishReason is now the one list, the key set derives
from it, and a test checks every documented reason has an explicit
mapping instead of falling through to "stop"
2026-09-18 16:03:45 -07:00
mateo-berri
8005856411 ci(build_and_test): seed the routing strategy through /config/update 2026-09-18 16:03:27 -07:00
yucheng
458943a7ac chore: merge main into litellm_prompt_injection_llm_api_check_dispatch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:02:31 +00:00
yucheng-berri
711a1924d4
Merge pull request #41583 from BerriAI/litellm_applied_guardrails_blocker
* fix(proxy): name the blocking guardrail in x-litellm-applied-guardrails

When a guardrail hook raises, the common ProxyLogging dispatch (sequential and parallel pre_call, pipeline block, during_call and post_call metrics wrapper, streaming iterator wrapper) now records that guardrail in applied_guardrails before re-raising, and pre_call_hook folds request-declared guardrails in on its raising path. Buffered streams rebuild their response headers after the first chunk so a post_call block reached while buffering carries the blocker too

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): attribute only the raising layer in stream and pipeline blocks

The streaming wrapper caught every exception crossing its boundary and named its own
callback, so a block by an inner guardrail or a provider stream failure also named every
outer guardrail. The wrapper now runs the hook over an upstream boundary that remembers
the exception it raised, and skips attribution when the same exception passes through

Pipeline blocks converted from SensitiveDataRouteException or ModifyResponseException into
a generic guardrail_pipeline_error now still record the blocking step's guardrail

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): drop explanatory docstrings from the stream attribution helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 16:00:27 -07:00
ryan
139445179a ci: remove the dead Agent Shin triage workflows and scripts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:59:32 +00:00