Commit graph

50471 commits

Author SHA1 Message Date
yassin
abc85ba607 fix(proxy): leave the realtime max_parallel_requests slot to the limiter failure callback when a refusal was logged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:47:37 +00:00
yassin
91c964a338 fix(auth): evict the membership cache entry when invalidation lands during the cache write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:46:09 +00:00
yucheng
7ebb169a4d fix(guardrails): coalesce usage index rows per request and guardrail, keeping policy linkage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:45:47 +00:00
Yassin Kortam
161ae2f3b4
Merge pull request #41107 from BerriAI/litellm_hide_credentials_hint_when_ui_password_set
fix(proxy): hide default credentials login hint when UI_PASSWORD is set
2026-09-14 13:44:09 -07:00
Yassin Kortam
43c3dbb312
Merge pull request #41118 from BerriAI/litellm_prometheus_proxy_provider_fallback_constant
refactor(prometheus): source PROXY_LLM_PROVIDER_FALLBACK from litellm.constants
2026-09-14 13:40:54 -07:00
Yassin Kortam
97ddb9494e
Merge pull request #40930 from BerriAI/litellm_flat_retry_records
fix(router): record flat retry attempts and cap retries from attempted_retries
2026-09-14 13:37:10 -07:00
yucheng
c3f52fe0d5 fix(guardrails): key not_run index rows by the sibling evaluation's guardrail_id
A not_run entry from the base guardrail only carries guardrail_name, while the
content filter's evaluated entry carries guardrail_id. Keyed apart, one request
listed twice in the monitor for a logging_only guardrail (Not run and Passed).
Resolve the id from a same-name sibling in the payload so the severity pick applies

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:37:09 +00:00
Tin Chi Lo
3177c37e22 fix(cli): show routed models and session stats for LLM API keys 2026-09-14 13:36:00 -07:00
yassin
c8440b5638 fix(proxy): move credentials hint helper into a leaf html_forms module to clear CodeQL cyclic import
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:34:58 +00:00
Yassin Kortam
856aedd14a
Merge pull request #40817 from BerriAI/litellm_redis_timeout_log_throttle
fix(redis): log a timeout streak once per interval instead of one line per cache call
2026-09-14 13:34:45 -07:00
yassin
9a62a5ebee refactor(prometheus): source PROXY_LLM_PROVIDER_FALLBACK from litellm.constants
Importing the fallback from litellm.proxy.hooks.rate_limiter_utils at the top
of litellm/integrations/prometheus.py closed an import cycle CodeQL flagged on
#41059 (rate_limiter_utils -> litellm -> ... -> prometheus). Hoist the constant
into litellm/constants.py so both modules read it from a leaf module.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:29:35 +00:00
Yassin Kortam
ab09abc3c8
Merge pull request #41115 from BerriAI/litellm_fix_wrapper_async_double_sync_success_handler_main
fix(utils): stop wrapper_async submitting the sync success handler twice
2026-09-14 13:28:16 -07:00
Yassin Kortam
963522207d
Merge pull request #41114 from BerriAI/litellm_model_info_skip_jsonable_encoder_main
perf(proxy): serialize /model/info listing once with orjson
2026-09-14 13:28:06 -07:00
yassin
4435aa601d fix(proxy): keep JWT agent binding through AUTO_REGISTER key creation
The virtual key created by AUTO_REGISTER replaced the JWT principal without
the agent_id auth_builder had resolved from agent_id_jwt_field, so agent
policies were skipped on that request and every later mapped-key request.
Pass the bound agent_id into generate_key_helper_fn and onto the returned
principal.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:28:02 +00:00
yassin
336ead5106 fix(proxy): import UserNotFoundError in grants resolver from the types module
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:23:54 +00:00
yassin
1b31be1a9c fix(proxy): leave the realtime max_parallel slot to the success callback when one is enqueued
Releasing the slot unconditionally from the route raced the limiter's own
success handler on the logging worker: both could read the same stashed
acquisition before either cleared it, and under the integer in-memory
fallback that double-decrements the counter. The route now releases only on
exits without a success callback (pre-call rejection, pre-call cancellation,
and Phase 2 exits without the success stamp), matching the HTTP disconnect
path's ownership rule.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:23:23 +00:00
yassin
4ac168b3fc fix(auth): drop in-flight membership load on invalidation so it cannot repopulate the cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:13:39 +00:00
yassin
d7900df73f chore(constants): drop the restating comment above the Bedrock realtime scope keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:13:02 +00:00
yassin
4fee71b7ad fix(proxy): define UserNotFoundError in an import-free types module so no importer sits in the CodeQL cycle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:10:36 +00:00
yassin
a4e34d6e1b test(caching): wrap the DualCache fixture line to the 120 character limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:05:14 +00:00
Yassin Kortam
34fe9f71d5
Merge pull request #41059 from BerriAI/litellm_prometheus_pre_call_rate_limit_api_provider
fix(prometheus): label pre-call rate limit failures with the resolved api_provider
2026-09-14 13:02:07 -07:00
ryan
2d3b63fb5a fix(proxy): report parent org models on /team/info only to team managers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:01:47 +00:00
yassin
bea22df6c2 fix(caching): keep generic add_cache failures at ERROR unless the backend is Redis
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:59:10 +00:00
yassin
9e870ffd00 fix(proxy): resolve CodeQL findings on team list, sanitize log args and move UserNotFoundError out of the import cycle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:57:50 +00:00
yassin
21cd52d508 fix(bedrock/realtime): declare the client websocket scope as a protocol attribute
CodeQL py/ineffectual-statement flags the bare ellipsis body of the
@property declaration on the RealtimeClientWebSocket protocol. A plain
attribute annotation states the same structural contract without an
expression statement.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:55:06 +00:00
yassin
10be7e01d4 fix(router): import retry helpers inside their functions to break CodeQL import cycles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:54:00 +00:00
yassin
845596063d fix(prometheus): label pre-call rate limit failures with the resolved api_provider
Pre-call limiters reject before a deployment is attached to request_data, so
the failure hook could not resolve api_provider for router aliases and emitted
api_provider="None" on litellm_proxy_failed_requests_metric_total and
litellm_proxy_total_requests_metric_total. Fall back to the provider the
limiter already resolved onto RateLimitError.llm_provider, keeping request
data as the first source and ignoring the proxy placeholder.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:50:21 +00:00
yassin
81ffc3125f fix(proxy): release realtime max_parallel_requests slot when the task is cancelled during pre-call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:50:01 +00:00
yassin
caaf368652 fix(proxy): move credentials hint helper into discovery module to break CodeQL import cycle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:48:45 +00:00
yassin
39f6ac4788 perf(proxy): serialize /model/info listing once with orjson
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:47:51 +00:00
yassin
d13e8dcae2 fix(utils): stop wrapper_async submitting the sync success handler twice
_client_async_logging_helper re-submitted logging_obj.success_handler to the
executor after _dispatch_success_logging had already done so, running the same
success pipeline twice per async request and racing on shared logging state.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:47:43 +00:00
Yassin Kortam
b94b8bca21
Merge pull request #41096 from BerriAI/litellm_disable_access_log_paths
feat(proxy): honor LITELLM_DISABLE_ACCESS_LOG_PATHS to drop noisy uvicorn access log lines
2026-09-14 12:41:34 -07:00
Yassin Kortam
687cb6cb7a
Merge pull request #40992 from BerriAI/litellm_model_info_team_alias
fix(ui): show the team alias on the model info page and in its raw JSON
2026-09-14 12:40:02 -07:00
Yassin Kortam
f67cc62032
Merge pull request #40995 from BerriAI/litellm_all_deployments_cooldown_error
fix(router): name the all-deployments-in-cooldown error on 429 responses
2026-09-14 12:39:42 -07:00
ryan
9cdfe311e9 test: assert only the org ceiling /team/info returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:39:18 +00:00
yassin
c4354c248a fix(redis): route the per-TTL pipeline write timeout log through the shared throttle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:38:14 +00:00
ryan
5ddb0fe691 fix(ui): keep the team model select loading until /team/info reports the org ceiling
useTeam seeds its cache from the team list, which has no organization_models, so the select briefly rendered unfiltered. Also drop the Prisma include assertion from the backend test.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:38:05 +00:00
Devin AI
2a06ece58c feat(models): mark Fireworks GLM 5.3 Fast router as supporting reasoning and response schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:37:25 +00:00
Yassin Kortam
7c55c4ba7d
Merge pull request #40925 from BerriAI/litellm_proxy_error_headers_from_litellm_response_headers
fix(proxy): forward provider request id headers on mapped error responses
2026-09-14 12:36:59 -07:00
Yassin Kortam
d541792b30
Merge pull request #41106 from BerriAI/litellm_allowed_file_extensions
feat(proxy): add general_settings.allowed_file_extensions for /v1/files uploads
2026-09-14 12:34:19 -07:00
yassin
486b328d99 Merge remote-tracking branch 'origin/main' into litellm_jwt_agent_id_claim 2026-09-14 19:34:14 +00:00
yassin
35cff94924 fix(proxy): resolve the agent registry lazily in JWT auth to break the import cycle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:34:14 +00:00
ryan-crabbe-berri
3f42a4fc98 test: suppress TQ008 on the two module-state patches
team_info resolves prisma_client and get_all_team_memberships from module
state, so there is no collaborator to inject; the two sibling team_info tests
patch the same way.

Claude-Session: https://claude.ai/code/session_011Tn3657NkV6ojLqewL64Kb
2026-09-14 19:30:44 +00:00
ryan-crabbe-berri
b6a5563d05 fix(ui): let team admins grant a team all proxy models
The team edit form only offered "All Proxy Models" when the dashboard could
read the parent organization, and /organization/info 403s for anyone who is
not a proxy admin or an admin of that org. A team admin with the internal_user
proxy role therefore saw only "No Default Models", which is the opposite of
what they wanted, and had no way to grant their team everything on the proxy.

/team/info now reports the parent org's model ceiling as organization_models,
which the same authorization already admits, and ModelSelect reads the ceiling
from the team it is editing before falling back to the organization. That also
makes the individual model list respect the org's allow-list instead of
listing every proxy model to a caller whose save would be rejected.

useTeam was typed as Team while returning the /team/info envelope, so its one
other caller unwrapped it behind a cast. It now returns the team itself.

Claude-Session: https://claude.ai/code/session_011Tn3657NkV6ojLqewL64Kb
2026-09-14 19:30:44 +00:00
yassin
f9d0cd6409 chore(proxy): drop explanatory docstrings from credentials hint helper and tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:29:30 +00:00
yassin
ebcd9bcb18 fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:29:11 +00:00
yassin
ff02390685 test(auth): drop docstrings from team membership tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:26:14 +00:00
yassin
9ff0dea139 fix(proxy): define credentials hint helper ahead of proxy_server imports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:25:03 +00:00
Devin AI
834f187f4c feat(models): add Together, Fireworks, Nebius, Cohere, Bedrock models and Inception cache pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:20:36 +00:00
Yujong Lee
27981c7d20 fix: preserve rust setting in trace parity 2026-09-14 12:17:15 -07:00