* feat(proxy): return per-group model provenance on /team/info
/team/info now carries access_group_details, one entry per resolved access
group with its id, name, and model list, so the UI can attribute each
inherited model to the group granting it. The batch resolver returns the
access group rows keyed by id instead of a stringly dict of lists, and the
team member budget helper returns a copy instead of mutating its parameter.
Type discipline and basedpyright budgets ratchet down accordingly.
* feat(ui): allow group-only teams and show model provenance on hover
Team create and edit no longer require a model selection: an empty
selection is saved as the no-default-models sentinel, never as a bare
empty list, since an empty team model list means unrestricted access.
The team info Models card now renders every badge with a hover tooltip
naming how the team got that model: directly, via named access groups,
or both, and group-granted badges stay visible when the direct list is
empty or a sentinel.
* refactor(proxy): dedupe access group ids and return copies instead of mutating
Duplicate access_group_ids no longer amplify the /team/info response: ids
collapse order-preserving before provenance is built, pinned by a regression
test. The resolver returns a model_copy rather than mutating its parameter,
and the team create call sends a new object instead of reassigning
formValues.models. Budgets ratchet down further with the mutation removal.
* fix(proxy): resolve entity access groups in the model listing endpoints
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): reuse the fetched team object when listing models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover key-level access group resolution in model listing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(websearch): restore snippet text in native web_search_tool_result blocks (LIT-5315)
The build_web_search_tool_result_block method copied url/title/page_age but
hardcoded encrypted_content to empty string, never reading SearchResult.snippet.
This left every native block content-free, forcing clients to web_fetch each
result to recover evidence—the reported symptom.
The Anthropic spec carries page text only in encrypted_content (an opaque
server-issued blob we cannot mint), so snippet is emitted as an additive key
alongside the spec fields. encrypted_content stays empty rather than holding
plaintext, which would assert encryption semantics that don't hold.
The anthropic SDK's BaseModel sets extra='allow', so the additive snippet key
survives SDK parsing. litellm has no typed model for web_search_result at all,
so nothing drops it internally. Turn-2 replay behavior is unaffected: the
empty encrypted_content already exists today.
Tests:
- Updated test_shape_with_results to assert snippet present
- Added test_snippet_carried_for_every_result to cover multi-result ordering
- Added test_missing_snippet_degrades_to_empty_string for edge case
- Mutation check: reverting source-only yields 3 test failures, restored to 117 passed
Fixes: LIT-5315
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(websearch): make synthesized web_search blocks replayable by native clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(websearch): flatten a resultless replayed search block so Bedrock accepts the next turn
The flatten added for LIT-5315 bails when the replayed web_search_tool_result
carries an empty content list, but that is exactly what the interceptor emits
when a search legitimately returns nothing and when a search raises. The block
survived into the outbound body, Bedrock rejected the tag, and the conversation
died on the following turn just as it did before the flatten existed.
An empty content list has no encrypted_content to respect and no evidence to
preserve, so it flattens safely, and its paired server_tool_use goes with it.
The rendered text now says so explicitly rather than emitting a bare header.
Adds the multi-turn replay coverage that existed nowhere: the outbound Bedrock
invoke body is asserted free of both block types, parametrized over the
results-present and resultless cases, and built from the interceptor's own
builder so the fixture cannot drift from what it emits.
Resolves LIT-5320
* test(websearch): pin flatten idempotency for the agentic-loop re-entry
The agentic loop re-enters the same /v1/messages entry point for its follow-up
call and hands it the original client history, so the flatten runs again over
already-flattened messages once per iteration. Bedrock always takes that path,
since its config reports web search as natively handled and the short-circuit
is skipped.
A pass that appended the rendered text instead of replacing the block would
duplicate the evidence on every iteration and re-ship the unsupported tag, and
no existing single-pass test sees it. Mutation checked: keeping the original
block alongside the rendered text fails this test on its own.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* feat(auto-router): track turns per complexity tier (LIT-5302)
Stamps complexity tier at decision time (rollup never re-derives from routed
model, since tier->model mapping is mutable config). Records per-tier turn
counts in LiteLLM_AutoRouterSession.tier_turns (jsonb), rolls up per router
in benchmarks SQL via jsonb_object_agg, returns on AutoRouterBenchmarkGroup
for dashboard turns/share metrics.
Addresses Greptile/Bugbot findings:
- Missing _SessionAggRow.tier_turns field: added with field_validator to
parse jsonb text cast and handle NULL. Would 500 every benchmarks read.
- Missing ::text cast on tier parameter: Postgres fails type inference on
parameterized CASE/IS NULL without explicit cast. Added to all usages.
- Docstring false claim (only complexity routers produce tiers): quality
router stamps numeric tier '1'/'2'/'3'. Per-type grouping in SQL prevents
cross-contamination. Rewrote docstring to clarify isolation.
- Comment convention violations: stripped per CLAUDE.md rule.
- Test gaps: 8 unit tests for extraction/validation/aggregation, 7 behavior
tests for SQL semantics against real Postgres. 12 mutations killed.
Fixed fragile complexity_router test that broke on nested function calls.
No API change; extends existing GET /auto_router/benchmarks response only.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(auto-router): address review findings on tier turns tracking
- Guard router_type update so a mid-session reconfigure can't pool
foreign tier names into tier_turns
- Keep pinned turns attributed to the tier that actually serves them
- Drop stray -- AlterTable comment from hand-written migration
- Drop the now-unnecessary ::text/json.loads round-trip; prisma
already returns tier_turns as a parsed dict
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(auto-router): satisfy type-discipline lint gate
- tier_turns fields: dict[str, int] -> Mapping[str, int] (LIT001,
mutable collection in annotation); these are read-only after
construction
- _summed_agg_row: {} -> MappingProxyType({}) (LIT002, mutable dict
literal)
- default-fallback branch: replace the reassigned-without-Final
fallback_tier with a Final default_model_first flag and a single
ternary assignment (LIT010)
Verified locally: type_discipline_gate.py, ruff_strict_gate.py, and
type_check_gate.py all pass against the litellm_internal_staging
merge-base; full test_complexity_router.py (374), auto_router
management-endpoint tests (26), db-layer rollup tests (31), and the
live-Postgres proxy_behavior rollup suite (17) all pass.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat(ui): resolve user email/alias in usage export instead of raw user id
* test(ui): cover email/alias resolution in usage export data builders
* chore(ui): drop explanatory comment per repo comment policy
GHSA-2v37-7h3g-55p8 (CVE-2026-67213) rates 8.2 against nanoid 3.3.16 and reds osv-scan on every PR into staging. Custom generators can loop indefinitely when size is zero, so a caller that passes through a zero size hangs the process.
nanoid is transitive through the dashboard's toolchain and 3.3.17 is a patch release that published 08-03, so it already clears the .npmrc min-release-age=3 guard. The lock diff is the version, resolved url, and integrity hash for that one package.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
PR #36020 changed AgentRegistry.get_agent_list to return tuple[AgentResponse, ...], and PR #35163 added a test asserting the result equals []. Both were green on their own branches and only collided once they were both on litellm_internal_staging, so proxy-server has been failing on every PR since with 'assert () == []'.
Nothing user-facing was wrong: get_agents only iterates the result and rebuilds it with comprehensions, and FastAPI serializes a tuple to the same JSON array. The test expectation was simply stale, so it now compares against ().
The get_agents local was still annotated list[AgentResponse] while two branches assign the registry tuple straight into it, so it widens to Sequence[AgentResponse]. That covers both the tuple and the list branches without pretending the value is mutable.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): deny when agent grants resolve to nothing
`get_allowed_agents` returned a plain list where the empty value meant both
"this caller was never restricted" and "this caller's grants resolved to
nothing". Downstream read either as allow-all, so a key restricted to one
agent inside a team restricted to another reached every agent on the proxy,
and an access group that resolved to no agents did the same.
Replace it with `resolve_agent_access`, returning a tagged
UnrestrictedAgentAccess | RestrictedAgentAccess. Only a caller with no grant
anywhere is unrestricted; an empty restricted set denies. Access group lookup
failures now propagate to the key/team resolvers so a DB error still fails
open exactly as before, while a group that genuinely resolves to nothing
denies.
* style(proxy): drop redundant comments from the agent access match
GHSA-fp3f-mc75-235c was published at 19:29 UTC today, hours after #36212 landed, and covers a different pypdf issue (large memory use on big /ToUnicode streams) that lands on the same 6.15.0 fix. It reds osv-scan on every PR into staging again.
pypdf 6.15.0 is still inside the repo's P3D exclude-newer window until 08-09, so uv cannot lock it yet and widening exclude-newer to pull it in early would weaken the freshness guard for every package. This gets the same 08-12 ignoreUntil as the first pypdf entry so both drop together in the bump.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Closes the six gitpython advisories flagged on litellm_internal_staging (GHSA-jm78-9fvv-mhgr and GHSA-wvpp-8hx9-p66j at 8.8, GHSA-hmq2-w58f-27jc at 8.2, GHSA-4gmw-gg2m-w46p at 8.1, GHSA-9rj7-rf2p-w77r at 7.5, GHSA-hh9p-6wh2-4mfc at 6.5). gitpython comes in transitively through mlflow-skinny, so this is a lock-only change re-derived with 'uv lock --upgrade-package gitpython'.
The seventh finding, GHSA-fwg2-594c-jp42 on pypdf, cannot be fixed the same way today: pypdf 6.15.0 published 2026-08-06 and the repo pins exclude-newer to a 3 day window, so uv will not resolve it before 2026-08-09. Rather than widen that window, the advisory gets a short dated IgnoredVulns entry that expires 2026-08-12, which leaves a hard deadline to land the real bump. It is a local, user-interaction denial of service on crafted CID font widths at CVSS 4.8, so a couple of days of exposure in the lock is acceptable.
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Closes the six gitpython advisories flagged on litellm_internal_staging (GHSA-jm78-9fvv-mhgr and GHSA-wvpp-8hx9-p66j at 8.8, GHSA-hmq2-w58f-27jc at 8.2, GHSA-4gmw-gg2m-w46p at 8.1, GHSA-9rj7-rf2p-w77r at 7.5, GHSA-hh9p-6wh2-4mfc at 6.5). gitpython comes in transitively through mlflow-skinny, so this is a lock-only change re-derived with 'uv lock --upgrade-package gitpython'.
The seventh finding, GHSA-fwg2-594c-jp42 on pypdf, cannot be fixed the same way today: pypdf 6.15.0 published 2026-08-06 and the repo pins exclude-newer to a 3 day window, so uv will not resolve it before 2026-08-09. Rather than widen that window, the advisory gets a short dated IgnoredVulns entry that expires 2026-08-12, which leaves a hard deadline to land the real bump. It is a local, user-interaction denial of service on crafted CID font widths at CVSS 4.8, so a couple of days of exposure in the lock is acceptable.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): derive config agent ids from agent_name so grants survive secret rotation
Config-defined A2A agents were identified by a sha256 of the whole resolved
config entry, secrets included, so rotating an os.environ secret re-minted the
agent_id on restart and orphaned every object_permission.agents grant while
grant-less keys kept access (LIT-5144). The id now hashes only agent_name, and
the old full-entry hash is kept as a legacy alias: permission checks,
GET /v1/agents filtering, spend and key attachment, and public_agent_groups all
normalize legacy ids so pre-upgrade grants keep working
* fix(proxy): persist stable agent ids into stored grants at startup
The runtime alias only translates a legacy grant while the current config
still hashes to it, so a secret rotation after upgrading would orphan the
grant, and an orphaned grant intersecting a stable team grant collapses to
an empty list that downstream reads as allow-all. Rewriting the stored ids
once at boot removes both. This cannot be a SQL migration because only the
running proxy can recompute the legacy hash from resolved config secrets
* fix(proxy): make the grant id migration a compare-and-swap
A grant edited between the migration's read and write kept the stale
snapshot. The update now predicates on the agents array read at scan time
via update_many, so a concurrently modified row is skipped and the runtime
alias covers it until the next boot retries
* fix(proxy): retry the grant id migration and stay within the LIT002 ceiling
The one-shot startup task now retries up to three times with a short delay
so a transient DB error at boot cannot leave a legacy grant unmigrated
until an operator's next restart is the rotation itself. The new list
constructions in the migration and the alias-expanded agent id lookups are
tuples now, keeping the branch under the mutable-collection budget
* fix(proxy): count compare-and-swap misses in the grant id migration
migrate_legacy_grant_ids now returns rewritten and missed counts from the
update_many results instead of reporting scanned rows as migrated, and the
startup task retries while any rows remain unmigrated, not just on errors
* fix(lint): clear basedpyright budget breaches in agent id aliasing
Adds a User Flow section right below the TLDR so every PR describes the same end user doing the same task before and after the change, plus comment instructions and a worked example so contributors can write it without any local tooling.
Making Sentinel follow AZURE_AUTHORITY_HOST is a breaking change for a
deployment that sets that variable for Azure OpenAI or the azure_storage
callback while keeping a commercial Sentinel workspace. That deployment had no
opt-out, because the proxy constructs the logger with no arguments and the
authority_host parameter is reachable only from the SDK.
Resolve the authority from AZURE_SENTINEL_AUTHORITY_HOST before falling back to
AZURE_AUTHORITY_HOST, matching how tenant id, client id and client secret
already resolve in this constructor.
* docs(keys): document /key/info fields and clarify budget_reset_at is the next reset
* docs(keys): drop wrong budget window start math, note reset boundary alignment
PR #36166 added tests/test_litellm/proxy/credential_endpoints/test_endpoints.py
but no CI job invokes it, so the CI Coverage guard failed on
litellm_internal_staging. Add the directory to the proxy-endpoints
job's test-path list so pytest actually runs the new tests and the
coverage assertion is satisfied.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
#35978 stopped the pooled A2A client replaying one upstream's Set-Cookie to
another by installing a blocking policy on that client's httpx cookie jar. That
covers only one of the two jars on the request path. AiohttpTransport is the
default transport unless it is explicitly disabled, and the aiohttp ClientSession
behind it keeps its own cookie jar which no httpx-level assertion can observe, so
the leak is still live on the default path: a live proxy on that commit still
delivers agent-alpha's session cookie to agent-beta's card fetch and JSON-RPC
call.
The reason it looked fixed is that aiohttp's default CookieJar is built with
unsafe=False and refuses to store cookies for IP hosts, so a proof addressed to
127.0.0.1 comes back clean whether or not that jar is blocked.
Cookie persistence is now blocked where the clients are built rather than at one
call site: blocked_cookie_jar() gives every httpx client, async and sync, a jar
whose DefaultCookiePolicy(allowed_domains=()) rejects every domain in both
directions, and both ClientSession constructions litellm owns, the transport's
session factory and the proxy's shared startup session, get a DummyCookieJar.
LiteLLM reads a response cookie nowhere, and an explicitly supplied Cookie header
still goes out, so passthrough forwarding and an agent's extra_headers are
unaffected. The A2A-scoped policy #35978 added is removed, since it is now dead.
The two suites that drive the aiohttp session factory synchronously mock
ClientSession because a real one needs a running event loop; DummyCookieJar has
the same requirement, so they mock it for the same reason.
* feat(proxy): add apply_user_budget_to_team_keys opt-in
PR #32005 made a user's personal max_budget apply to their team-scoped keys
too, and PR #35271 reverted the whole thing (behavior plus the
skip_user_budget_on_team_key opt-out) because that flipped the default for
everyone. This brings the behavior back the other way round: default is
unchanged, and general_settings.apply_user_budget_to_team_keys opts a
deployment into charging the key owner's personal budget on team keys.
The flag reaches all three personal-budget gates so an opted-in deployment
enforces consistently: the read-time check in common_checks, the optimistic
reservation counter in _get_budget_counters, and the _PROXY_MaxBudgetLimiter
pre-call hook. It is also in the /config/list allowed args and, unlike the
reverted flag, in the _update_general_settings propagation allowlist, so the
Admin UI General Settings toggle actually takes effect at runtime; an explicit
YAML value still wins over the DB value on reload.
get_config_list's allowed_args moves to a module-level frozen mapping of
field name to type string, dropping 18 LIT002 violations and rebuilding one
less dict per request.
* style(proxy): drop explanatory comments from the budget flag paths
* fix(proxy): invalidate cached project object on /project/update and /project/delete
The auth path reads projects cache-first via get_project_object with a 60s
TTL and no freshness check, but no project write endpoint ever evicted the
project_id:{id} cache entry. A project cached before /project/update added a
model allowlist kept an empty models list in cache, so _run_project_checks
skipped can_project_access_model and project-bound keys could call team
models outside the project allowlist until the TTL expired. The same
staleness applied to blocked status and budget fields, and /project/delete
left the deleted project enforceable from cache.
Evict the cache entry after the DB write in update_project and
delete_project via a shared delete_cached_project_object helper, with the
cache key derivation shared with get_project_object.
* fix(proxy): broadcast project cache invalidation to all workers and make eviction best-effort
Single-worker eviction leaves every other worker serving its in-memory copy
of the mutated project until the 60s TTL expires, so a project allowlist
change was still bypassable on multi-worker deployments. Add a coordination
Redis pub/sub channel (litellm_proxy.auth_cache_invalidation): project
eviction publishes the cache key and a per-worker subscriber deletes the
local in-memory entry, with the next auth read refetching from the DB.
Subscriber starts on any deployment with a coordination Redis and falls back
to the TTL when none is configured.
Also wrap the eviction in a best-effort catch: the DB write has already
committed when eviction runs, so a cache backend error must not turn a
successful update into a 500 or abort the remaining ids in /project/delete.
* fix(lint): sort auth cache invalidation import and suppress best-effort shutdown catch
The strict-budget gate flagged the new import block as un-sorted (I001) and
the broad except in stop_auth_cache_invalidation_subscriber (BLE001); the
catch is intentional since a failing stop must not break proxy shutdown, so
it carries a named suppression instead of counting against the budget.
* fix(proxy): return the real status code when a credential update is rejected
update_credential ended its except clause with 'return handle_exception_on_proxy(e)'. Returning the exception makes it the response body, so FastAPI answers 200 and every rejection on this route reads as a successful write to any caller that checks the status; the admin dashboard's API client is one. Patching a name that does not exist answered 200 with the real 404 buried in the body.
The sibling handlers in this file already raise. The route had no test coverage, which is why it survived.
* Update tests/test_litellm/proxy/credential_endpoints/test_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
The Azure Sentinel logger hardcoded the commercial Entra authority and the
commercial Azure Monitor audience, so Log Analytics ingestion could not work in
Azure Government even when the ingestion endpoint pointed at a sovereign Data
Collection Endpoint.
Resolve the authority from AZURE_AUTHORITY_HOST and derive the matching Logs
Ingestion audience from it. Moving only the token URL is not enough: sovereign
Entra would then be asked for a token scoped to the commercial audience, which
the sovereign endpoint rejects.
test_get_deployment_credentials_with_provider_bedrock_batch_fields already
covers s3_encryption_key_id on the base branch, and the new test passes with
every production file in this branch reverted, so it guards nothing.
The staging merge tightened the LIT002 ceiling, so the three mutable dict
literals this branch added now breach it. Build the S3 request headers as
MappingProxyType and resolve the encryption key from a tuple of sources.
