Closes the six gitpython advisories flagged on litellm_internal_staging (GHSA-jm78-9fvv-mhgr and GHSA-wvpp-8hx9-p66j at 8.8, GHSA-hmq2-w58f-27jc at 8.2, GHSA-4gmw-gg2m-w46p at 8.1, GHSA-9rj7-rf2p-w77r at 7.5, GHSA-hh9p-6wh2-4mfc at 6.5). gitpython comes in transitively through mlflow-skinny, so this is a lock-only change re-derived with 'uv lock --upgrade-package gitpython'.
The seventh finding, GHSA-fwg2-594c-jp42 on pypdf, cannot be fixed the same way today: pypdf 6.15.0 published 2026-08-06 and the repo pins exclude-newer to a 3 day window, so uv will not resolve it before 2026-08-09. Rather than widen that window, the advisory gets a short dated IgnoredVulns entry that expires 2026-08-12, which leaves a hard deadline to land the real bump. It is a local, user-interaction denial of service on crafted CID font widths at CVSS 4.8, so a couple of days of exposure in the lock is acceptable.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): derive config agent ids from agent_name so grants survive secret rotation
Config-defined A2A agents were identified by a sha256 of the whole resolved
config entry, secrets included, so rotating an os.environ secret re-minted the
agent_id on restart and orphaned every object_permission.agents grant while
grant-less keys kept access (LIT-5144). The id now hashes only agent_name, and
the old full-entry hash is kept as a legacy alias: permission checks,
GET /v1/agents filtering, spend and key attachment, and public_agent_groups all
normalize legacy ids so pre-upgrade grants keep working
* fix(proxy): persist stable agent ids into stored grants at startup
The runtime alias only translates a legacy grant while the current config
still hashes to it, so a secret rotation after upgrading would orphan the
grant, and an orphaned grant intersecting a stable team grant collapses to
an empty list that downstream reads as allow-all. Rewriting the stored ids
once at boot removes both. This cannot be a SQL migration because only the
running proxy can recompute the legacy hash from resolved config secrets
* fix(proxy): make the grant id migration a compare-and-swap
A grant edited between the migration's read and write kept the stale
snapshot. The update now predicates on the agents array read at scan time
via update_many, so a concurrently modified row is skipped and the runtime
alias covers it until the next boot retries
* fix(proxy): retry the grant id migration and stay within the LIT002 ceiling
The one-shot startup task now retries up to three times with a short delay
so a transient DB error at boot cannot leave a legacy grant unmigrated
until an operator's next restart is the rotation itself. The new list
constructions in the migration and the alias-expanded agent id lookups are
tuples now, keeping the branch under the mutable-collection budget
* fix(proxy): count compare-and-swap misses in the grant id migration
migrate_legacy_grant_ids now returns rewritten and missed counts from the
update_many results instead of reporting scanned rows as migrated, and the
startup task retries while any rows remain unmigrated, not just on errors
* fix(lint): clear basedpyright budget breaches in agent id aliasing
Adds a User Flow section right below the TLDR so every PR describes the same end user doing the same task before and after the change, plus comment instructions and a worked example so contributors can write it without any local tooling.
Making Sentinel follow AZURE_AUTHORITY_HOST is a breaking change for a
deployment that sets that variable for Azure OpenAI or the azure_storage
callback while keeping a commercial Sentinel workspace. That deployment had no
opt-out, because the proxy constructs the logger with no arguments and the
authority_host parameter is reachable only from the SDK.
Resolve the authority from AZURE_SENTINEL_AUTHORITY_HOST before falling back to
AZURE_AUTHORITY_HOST, matching how tenant id, client id and client secret
already resolve in this constructor.
* docs(keys): document /key/info fields and clarify budget_reset_at is the next reset
* docs(keys): drop wrong budget window start math, note reset boundary alignment
PR #36166 added tests/test_litellm/proxy/credential_endpoints/test_endpoints.py
but no CI job invokes it, so the CI Coverage guard failed on
litellm_internal_staging. Add the directory to the proxy-endpoints
job's test-path list so pytest actually runs the new tests and the
coverage assertion is satisfied.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
#35978 stopped the pooled A2A client replaying one upstream's Set-Cookie to
another by installing a blocking policy on that client's httpx cookie jar. That
covers only one of the two jars on the request path. AiohttpTransport is the
default transport unless it is explicitly disabled, and the aiohttp ClientSession
behind it keeps its own cookie jar which no httpx-level assertion can observe, so
the leak is still live on the default path: a live proxy on that commit still
delivers agent-alpha's session cookie to agent-beta's card fetch and JSON-RPC
call.
The reason it looked fixed is that aiohttp's default CookieJar is built with
unsafe=False and refuses to store cookies for IP hosts, so a proof addressed to
127.0.0.1 comes back clean whether or not that jar is blocked.
Cookie persistence is now blocked where the clients are built rather than at one
call site: blocked_cookie_jar() gives every httpx client, async and sync, a jar
whose DefaultCookiePolicy(allowed_domains=()) rejects every domain in both
directions, and both ClientSession constructions litellm owns, the transport's
session factory and the proxy's shared startup session, get a DummyCookieJar.
LiteLLM reads a response cookie nowhere, and an explicitly supplied Cookie header
still goes out, so passthrough forwarding and an agent's extra_headers are
unaffected. The A2A-scoped policy #35978 added is removed, since it is now dead.
The two suites that drive the aiohttp session factory synchronously mock
ClientSession because a real one needs a running event loop; DummyCookieJar has
the same requirement, so they mock it for the same reason.
* feat(proxy): add apply_user_budget_to_team_keys opt-in
PR #32005 made a user's personal max_budget apply to their team-scoped keys
too, and PR #35271 reverted the whole thing (behavior plus the
skip_user_budget_on_team_key opt-out) because that flipped the default for
everyone. This brings the behavior back the other way round: default is
unchanged, and general_settings.apply_user_budget_to_team_keys opts a
deployment into charging the key owner's personal budget on team keys.
The flag reaches all three personal-budget gates so an opted-in deployment
enforces consistently: the read-time check in common_checks, the optimistic
reservation counter in _get_budget_counters, and the _PROXY_MaxBudgetLimiter
pre-call hook. It is also in the /config/list allowed args and, unlike the
reverted flag, in the _update_general_settings propagation allowlist, so the
Admin UI General Settings toggle actually takes effect at runtime; an explicit
YAML value still wins over the DB value on reload.
get_config_list's allowed_args moves to a module-level frozen mapping of
field name to type string, dropping 18 LIT002 violations and rebuilding one
less dict per request.
* style(proxy): drop explanatory comments from the budget flag paths
* fix(proxy): invalidate cached project object on /project/update and /project/delete
The auth path reads projects cache-first via get_project_object with a 60s
TTL and no freshness check, but no project write endpoint ever evicted the
project_id:{id} cache entry. A project cached before /project/update added a
model allowlist kept an empty models list in cache, so _run_project_checks
skipped can_project_access_model and project-bound keys could call team
models outside the project allowlist until the TTL expired. The same
staleness applied to blocked status and budget fields, and /project/delete
left the deleted project enforceable from cache.
Evict the cache entry after the DB write in update_project and
delete_project via a shared delete_cached_project_object helper, with the
cache key derivation shared with get_project_object.
* fix(proxy): broadcast project cache invalidation to all workers and make eviction best-effort
Single-worker eviction leaves every other worker serving its in-memory copy
of the mutated project until the 60s TTL expires, so a project allowlist
change was still bypassable on multi-worker deployments. Add a coordination
Redis pub/sub channel (litellm_proxy.auth_cache_invalidation): project
eviction publishes the cache key and a per-worker subscriber deletes the
local in-memory entry, with the next auth read refetching from the DB.
Subscriber starts on any deployment with a coordination Redis and falls back
to the TTL when none is configured.
Also wrap the eviction in a best-effort catch: the DB write has already
committed when eviction runs, so a cache backend error must not turn a
successful update into a 500 or abort the remaining ids in /project/delete.
* fix(lint): sort auth cache invalidation import and suppress best-effort shutdown catch
The strict-budget gate flagged the new import block as un-sorted (I001) and
the broad except in stop_auth_cache_invalidation_subscriber (BLE001); the
catch is intentional since a failing stop must not break proxy shutdown, so
it carries a named suppression instead of counting against the budget.
A batch or fine-tuning job is created from a file the caller already uploaded,
and that file only exists under the credentials of the deployment that stored
it. When the router fell back to a different model group it handed that file id
to a provider that has never seen it, so the caller got the second provider's
complaint about the file id instead of the error that explains what was actually
wrong with their request.
run_async_fallback now skips fallback targets outside the original model group
whenever the request carries input_file_id or training_file. Order-based
fallbacks stay inside the group, so retrying across deployments still works.
The same handler also crashed with "'NoneType' object has no attribute 'update'"
whenever a fallback fired on a request with metadata set to None, which
/v1/batches always does when the caller sends no metadata, turning the provider's
400 into a 500. Record the model group with a merge instead of setdefault, and
write it to litellm_metadata on the endpoints that use it so the router's
bookkeeping no longer lands in the metadata stored on the provider's batch.
* fix(proxy): return the real status code when a credential update is rejected
update_credential ended its except clause with 'return handle_exception_on_proxy(e)'. Returning the exception makes it the response body, so FastAPI answers 200 and every rejection on this route reads as a successful write to any caller that checks the status; the admin dashboard's API client is one. Patching a name that does not exist answered 200 with the real 404 buried in the body.
The sibling handlers in this file already raise. The route had no test coverage, which is why it survived.
* Update tests/test_litellm/proxy/credential_endpoints/test_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Guards _declared_query_params against a regression in the get_flat_params
migration: the flatten step returns path, query, header and cookie params
together, so a dropped ParamTypes.query filter would wrongly treat path or
header names as declared query params and accept unknown ones. Removing the
filter fails these tests.
fastapi 0.140.7 removed get_flat_dependant(), which broke the import in
management_v1/common.py and took down every /management/v1 route. Switch to
get_flat_params() and filter to ParamTypes.query so unknown-query-param
rejection keeps matching the old behavior.
The end-to-end extras test now injects a MagicMock(spec=HTTPHandler) via
the client parameter instead of patching post on a real handler, and the
docstrings on the new regression tests are removed, addressing the
remaining Greptile review feedback.
Mirror the chat extras translation for /v1/completions, adapted to the
typed OpenAI SDK: anything completions.create() rejects (reasoning_effort,
response_format, fireworks-native extras) rides inside extra_body, which
the SDK merges server-side. Top-level reasoning_effort and response_format
are moved into extra_body (they raised TypeError before), truncate
aliases, chat_template_kwargs effort keys, and guided_* resolve into
extra_body fields, and the strip set removes the rest. Verified live:
/v1/completions rejects prompt_truncate_len, so both truncate names are
stripped on this path rather than renamed.
The Azure Sentinel logger hardcoded the commercial Entra authority and the
commercial Azure Monitor audience, so Log Analytics ingestion could not work in
Azure Government even when the ingestion endpoint pointed at a sovereign Data
Collection Endpoint.
Resolve the authority from AZURE_AUTHORITY_HOST and derive the matching Logs
Ingestion audience from it. Moving only the token URL is not enough: sovereign
Entra would then be asked for a token scoped to the commercial audience, which
the sovereign endpoint rejects.
The http handler merges extra_body after transform_request, so a
response_format nested in an explicit extra_body would silently clobber
the explicit top-level response_format. Drop the nested copy with a
debug log so the top-level value wins, closing the precedence hole in
the guided-param native-wins path.
Streaming /chat/completions and /v1/responses emit nothing, not even response headers, until the upstream yields its first chunk, so an ingress with an idle read timeout (nginx proxy-read-timeout) drops long time-to-first-token streams.
Reuses the existing Anthropic keepalive wrapper with a configurable ping payload, emitting an SSE comment on the OpenAI-shaped routes so conformant clients ignore it. Off unless litellm_settings.sse_keepalive_ping_interval_seconds is set.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
test_get_deployment_credentials_with_provider_bedrock_batch_fields already
covers s3_encryption_key_id on the base branch, and the new test passes with
every production file in this branch reverted, so it guards nothing.
create_websocket_passthrough_route existed but /openai and
/openai_passthrough only registered HTTP methods, so WS upgrades were
rejected at routing. Add catch-all websocket routes mirroring the HTTP
passthrough target construction.
Fixes#36088
An intercepted web search called litellm.asearch() with only the search tool's litellm_params, so the search request carried no owner. The proxy's spend hook skips any call with no key, user or team attached, so the search's provider cost never reached SpendLogs; it was missing from the Logs page and never counted against the caller's budget. The same path never ran the rate limiter either, so an intercepted search was free of the key's RPM/TPM limits.
The search now carries the originating key's attribution metadata (key hash, alias, user, team, org, plus model_group set to the resolved search tool) and runs the caller's rate limit checks before hitting the provider, matching what a direct /v1/search request gets. SDK calls with no proxy auth context are unchanged.
Resolves LIT-5033
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The staging merge tightened the LIT002 ceiling, so the three mutable dict
literals this branch added now breach it. Build the S3 request headers as
MappingProxyType and resolve the encryption key from a tuple of sources.
