Commit graph

571 commits

Author SHA1 Message Date
yassin
e48dde7b9b fix(tests): drop leftover merge markers in test_handle_jwt
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 16:28:13 +00:00
yassin
5fee1c8710 Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/auth/test_handle_jwt.py
2026-09-16 16:27:18 +00:00
Joshua Valluru
e035682ed1 refactor(auth): separate JWT identity and OAuth authorization 2026-09-15 22:15:57 -07:00
Joshua Valluru
97211bc356 fix(mcp): authorize per-user OAuth credential writes 2026-09-15 19:00:47 -07:00
yassin
15f2e25e8a refactor(proxy): replace configurable model access denied message with a fixed clean client message
Drop the model_access_denied_message setting, its {model} template, the DB
override entry and the Admin UI field. Model access denials now always return
the fixed client message while the allowlist diagnostic is logged at the final
HTTP, realtime and MCP boundaries

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 01:44:36 +00:00
Joshua Valluru
ece1de73b4 Merge remote-tracking branch 'origin/main' into litellm_fix_mcp_jwt_oauth_persistence 2026-09-15 18:36:02 -07:00
yassin
14d239fc84 test(proxy): pin JWT scope denial message shape through auth exception conversion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 00:46:59 +00:00
Yassin Kortam
7eeba69016
Merge pull request #41316 from BerriAI/litellm_nvidia_nim_infer_passthrough
feat(proxy): add /nvidia_nim passthrough route for NIM object detection and OCR /v1/infer
2026-09-15 17:28:25 -07:00
Yassin Kortam
e4a7d2aa0b
Merge pull request #41302 from BerriAI/litellm_fix_key_model_rpm_override_precedence
fix(proxy): key model rpm/tpm override takes precedence over team model limit
2026-09-15 17:24:38 -07:00
Joshua Valluru
72a4824acb fix(mcp): preserve JWT agent validation after main merge 2026-09-15 17:20:06 -07:00
Joshua Valluru
53318796fd fix(mcp): separate JWT identity lookup from request authorization 2026-09-15 17:13:10 -07:00
yassin
b7af51cc4a Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message 2026-09-16 00:12:53 +00:00
yassin
f60a603519 fix(proxy): log configured model access denials at the final response boundary
Post-auth denials from can_key_call_resolved_model (per-request alias
rewrite, MCP sampling, realtime) never reach the auth exception handler,
so the internal allowlist reason was dropped when
model_access_denied_message was set. Log it once from the ProxyException
response handler and the realtime rejection path instead, and convert
JWT ModelAccessDeniedHTTPException into the specialized ProxyException so
the same boundary covers it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 00:12:49 +00:00
Yassin Kortam
24153b5f29
Merge pull request #41308 from BerriAI/litellm_resolve_model_group_alias_before_auth 2026-09-15 16:51:28 -07:00
yassin
7267c6bed7 Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message 2026-09-15 23:36:30 +00:00
yassin
b99f0c812f fix(proxy): log configured model access denial only at the auth error boundary
Move the WARNING that carries the internal denial detail out of the message formatter and into the auth exception handler. The denial exceptions now carry internal_message so access-group probes and fallback paths that catch and recover from the denial no longer log a false denial for an allowed request

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:36:12 +00:00
yassin
771b2509d1 fix(proxy): reject mixed NIM model groups and strip the deployment model before the group in /nvidia_nim URLs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:34:22 +00:00
yassin
c007fb9928 fix(proxy): skip alias rewrite only for dispatched pass-through handlers
Match the pass-through skip to what FastAPI actually dispatched (the user-defined
endpoint marker or a provider handler's {endpoint:path} param) instead of the
mapped route prefixes, which also cover native routes such as /openai/v1/responses
and /cursor/chat/completions. Wrap the added test lines to the 120-column limit.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:00:08 +00:00
yassin
b50a22b370 fix(proxy): restrict /nvidia_nim route to NIM-backed model groups and inject router in tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:55:14 +00:00
Yassin Kortam
9bb83fdaea
Merge pull request #41281 from BerriAI/litellm_lit_7417_jwt_key_mapping_issuer_scope
fix(jwt-auth): scope JWT key mappings by issuer to prevent cross-issuer collisions
2026-09-15 15:51:50 -07:00
yassin
d4d8cc9092 fix(proxy): apply access denied message to JWT paths, sanitize denial log, await dashboard saves
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:44:03 +00:00
yassin
5d6e367d56 fix(proxy): keep alias rewrite off pass-through bodies and auth-merged params
Skip router_settings.model_group_alias resolution on registered pass-through
routes, rebuild the rewritten body from the cached client payload instead of the
auth-enriched request_data, and centralize the resolved-scope sentinel in
litellm/constants.py

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:32:11 +00:00
yassin
69be041da7 feat(proxy): add /nvidia_nim passthrough route for NIM object detection and OCR /v1/infer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:11:08 +00:00
yassin
bae2bf003e fix(proxy): resolve router_settings.model_group_alias before key/team model auth
Key and team router_settings.model_group_alias aliases were resolved only after the key/team model allowlist checks ran, so a key allowed the alias target was denied when it requested the alias. Resolve the alias during auth and rewrite the request body to the target before the allowlist checks. The alias the client sent is kept in the request scope so the response model still echoes it.

Resolves LIT-3054

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:06:55 +00:00
yassin
168b5bc4fb feat(proxy): configurable client-facing model access denied message
Add litellm_settings.model_access_denied_message, a template ({model} placeholder) returned to clients instead of the detailed "can only access models=[...]" text on key/team/user/org/project and team-member model access denials. The full denial reason is still written to the proxy logs at WARNING. Unset keeps the existing detailed message, status codes and error types are unchanged.

Expose the new setting and the existing expose_router_debug_in_errors flag in the Admin UI general settings (String editor, Boolean toggle with an explicit True default) and allow both as safe DB overrides so they persist and propagate across workers.

Resolves LIT-5283

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:05:08 +00:00
yassin
51a4cb9fdd fix(proxy): key model rpm/tpm override takes precedence over team model limit
A key inside a team with model_rpm_limit / model_tpm_limit in team metadata could not
override those limits for itself: the v3 limiter always added the team's per-model
descriptor next to the key's, so the tighter team limit won. The docs already say the
resolution order is key metadata > key model_max_budget > team metadata

get_key_own_model_rate_limit returns only what the key sets on itself, and the team
descriptor now carries only the metrics the key does not override, so an rpm-only
override still leaves the team tpm pool enforced

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 21:48:06 +00:00
Yassin Kortam
e54b93017b fix(jwt-auth): scope JWT key mappings by issuer to prevent cross-issuer collisions 2026-09-15 21:02:17 +00:00
yassin
fad11fa66e fix(proxy): keep client User-Agent on auth failure spend logs
Auth gate rejections are raised before add_litellm_data_to_request stamps the caller User-Agent and SpendLogsMetadata dropped the field, so failure spend logs and prometheus labels could not identify an abusive client. Stamp requester_ip_address and user_agent on the failure hook payload and carry user_agent through spend log metadata. Request scopes without a headers entry are tolerated.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 21:02:06 +00:00
Yassin Kortam
e6af62aaac
Merge pull request #40997 from BerriAI/litellm_batch_tpd_limit
feat(proxy): add tpd_limit (tokens per day) for batch submissions
2026-09-15 13:39:45 -07:00
Yassin Kortam
367393405c
Merge pull request #41170 from BerriAI/litellm_prometheus_401_failed_requests_metric
fix(prometheus): count 401 auth failures in litellm_proxy_failed_requests_metric
2026-09-15 13:13:49 -07:00
Yassin Kortam
a7b894f91b
Merge pull request #41271 from BerriAI/litellm_org_zero_max_budget
fix(proxy): enforce organization budgets when max_budget is 0
2026-09-15 12:47:18 -07:00
Yassin Kortam
60808520df
Merge pull request #40904 from BerriAI/litellm_jwt_agent_id_claim
feat(proxy): bind JWT claims to registered agents via agent_id_jwt_field
2026-09-15 12:46:18 -07:00
yassin
d350aaf8bd Merge remote-tracking branch 'origin/main' into litellm_prometheus_401_failed_requests_metric 2026-09-15 19:43:47 +00:00
Yassin Kortam
501be3143d fix(proxy): enforce organization budgets when max_budget is 0
_organization_max_budget_check returned early whenever org_max_budget
was <= 0, so an organization with an explicit max_budget of 0 was
treated as unlimited instead of zero allowance. Key, team, and user
budget checks already skip only on None; align organization budgets
with that convention.

validate_team_org_change had the same defect in a different shape: it
used a truthy check on the org's max_budget when validating a team
move, so an explicit 0 there silently skipped the guard too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 11:49:05 -07:00
tin-berri
3ac79757f4
Merge pull request #41175 from BerriAI/litellm_team_member_auto_routers
feat(auto-router): allow opted-in team members to manage their routers
2026-09-15 11:19:58 -07:00
Devin AI
79450121f8 test(proxy): document access group test seam
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 17:00:11 +00:00
Devin AI
56d0f953f5 fix(proxy): list directly assigned team models in model access errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 16:33:41 +00:00
Tin Chi Lo
109ca70f66 feat(auto-router): allow opted-in team members to manage their routers 2026-09-14 23:56:55 -07:00
yassin
b9dd397746 fix(prometheus): count 401 auth failures in litellm_proxy_failed_requests_metric
Invalid or unknown virtual keys were filtered out of the proxy failed and total
request counters entirely. Count them with hashed_api_key unset so caller-chosen
key strings cannot create unbounded label series, and normalize the request
route on the auth failure path so dynamic path ids do not leak into the route
label either.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 02:09:00 +00:00
tin-berri
c626ff098b
Merge pull request #40877 from BerriAI/litellm_lit7658_cache_cost_v0_fresh
feat(proxy): predict prompt-cache costs across deployments
2026-09-14 16:25:41 -07:00
yassin
df5e193645 Merge remote-tracking branch 'origin/main' into pr-40997
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/litellm-dashboard/src/components/templates/key_edit_view.tsx
2026-09-14 22:07:48 +00:00
Yassin Kortam
24bfd5fba1
Merge pull request #41102 from BerriAI/litellm_team_membership_once_main
fix(auth): load team membership once per request and skip prisma on an L1 hit
2026-09-14 14:46:06 -07:00
yassin
db8dfe93a5 fix(auth): wait for the in-flight membership load before evicting its cache key on invalidation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:10:43 +00:00
yassin
4d4d3fb18a fix(proxy): bind agent registry into JWTHandler and keep persisted agent id on AUTO_REGISTER race
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:09:14 +00:00
yassin
ffeea30f23 test(auth): cover a stale membership write landing after a fresh reload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:55:22 +00:00
yassin
91c964a338 fix(auth): evict the membership cache entry when invalidation lands during the cache write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:46:09 +00:00
Tin Chi Lo
3177c37e22 fix(cli): show routed models and session stats for LLM API keys 2026-09-14 13:36:00 -07:00
yassin
4435aa601d fix(proxy): keep JWT agent binding through AUTO_REGISTER key creation
The virtual key created by AUTO_REGISTER replaced the JWT principal without
the agent_id auth_builder had resolved from agent_id_jwt_field, so agent
policies were skipped on that request and every later mapped-key request.
Pass the bound agent_id into generate_key_helper_fn and onto the returned
principal.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:28:02 +00:00
yassin
4ac168b3fc fix(auth): drop in-flight membership load on invalidation so it cannot repopulate the cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:13:39 +00:00
yassin
486b328d99 Merge remote-tracking branch 'origin/main' into litellm_jwt_agent_id_claim 2026-09-14 19:34:14 +00:00