* feat(proxy): add apply_user_budget_to_team_keys opt-in
PR #32005 made a user's personal max_budget apply to their team-scoped keys
too, and PR #35271 reverted the whole thing (behavior plus the
skip_user_budget_on_team_key opt-out) because that flipped the default for
everyone. This brings the behavior back the other way round: default is
unchanged, and general_settings.apply_user_budget_to_team_keys opts a
deployment into charging the key owner's personal budget on team keys.
The flag reaches all three personal-budget gates so an opted-in deployment
enforces consistently: the read-time check in common_checks, the optimistic
reservation counter in _get_budget_counters, and the _PROXY_MaxBudgetLimiter
pre-call hook. It is also in the /config/list allowed args and, unlike the
reverted flag, in the _update_general_settings propagation allowlist, so the
Admin UI General Settings toggle actually takes effect at runtime; an explicit
YAML value still wins over the DB value on reload.
get_config_list's allowed_args moves to a module-level frozen mapping of
field name to type string, dropping 18 LIT002 violations and rebuilding one
less dict per request.
* style(proxy): drop explanatory comments from the budget flag paths
* fix(proxy): invalidate cached project object on /project/update and /project/delete
The auth path reads projects cache-first via get_project_object with a 60s
TTL and no freshness check, but no project write endpoint ever evicted the
project_id:{id} cache entry. A project cached before /project/update added a
model allowlist kept an empty models list in cache, so _run_project_checks
skipped can_project_access_model and project-bound keys could call team
models outside the project allowlist until the TTL expired. The same
staleness applied to blocked status and budget fields, and /project/delete
left the deleted project enforceable from cache.
Evict the cache entry after the DB write in update_project and
delete_project via a shared delete_cached_project_object helper, with the
cache key derivation shared with get_project_object.
* fix(proxy): broadcast project cache invalidation to all workers and make eviction best-effort
Single-worker eviction leaves every other worker serving its in-memory copy
of the mutated project until the 60s TTL expires, so a project allowlist
change was still bypassable on multi-worker deployments. Add a coordination
Redis pub/sub channel (litellm_proxy.auth_cache_invalidation): project
eviction publishes the cache key and a per-worker subscriber deletes the
local in-memory entry, with the next auth read refetching from the DB.
Subscriber starts on any deployment with a coordination Redis and falls back
to the TTL when none is configured.
Also wrap the eviction in a best-effort catch: the DB write has already
committed when eviction runs, so a cache backend error must not turn a
successful update into a 500 or abort the remaining ids in /project/delete.
* fix(lint): sort auth cache invalidation import and suppress best-effort shutdown catch
The strict-budget gate flagged the new import block as un-sorted (I001) and
the broad except in stop_auth_cache_invalidation_subscriber (BLE001); the
catch is intentional since a failing stop must not break proxy shutdown, so
it carries a named suppression instead of counting against the budget.
* fix(proxy): return the real status code when a credential update is rejected
update_credential ended its except clause with 'return handle_exception_on_proxy(e)'. Returning the exception makes it the response body, so FastAPI answers 200 and every rejection on this route reads as a successful write to any caller that checks the status; the admin dashboard's API client is one. Patching a name that does not exist answered 200 with the real 404 buried in the body.
The sibling handlers in this file already raise. The route had no test coverage, which is why it survived.
* Update tests/test_litellm/proxy/credential_endpoints/test_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Guards _declared_query_params against a regression in the get_flat_params
migration: the flatten step returns path, query, header and cookie params
together, so a dropped ParamTypes.query filter would wrongly treat path or
header names as declared query params and accept unknown ones. Removing the
filter fails these tests.
The Azure Sentinel logger hardcoded the commercial Entra authority and the
commercial Azure Monitor audience, so Log Analytics ingestion could not work in
Azure Government even when the ingestion endpoint pointed at a sovereign Data
Collection Endpoint.
Resolve the authority from AZURE_AUTHORITY_HOST and derive the matching Logs
Ingestion audience from it. Moving only the token URL is not enough: sovereign
Entra would then be asked for a token scoped to the commercial audience, which
the sovereign endpoint rejects.
test_get_deployment_credentials_with_provider_bedrock_batch_fields already
covers s3_encryption_key_id on the base branch, and the new test passes with
every production file in this branch reverted, so it guards nothing.
An intercepted web search called litellm.asearch() with only the search tool's litellm_params, so the search request carried no owner. The proxy's spend hook skips any call with no key, user or team attached, so the search's provider cost never reached SpendLogs; it was missing from the Logs page and never counted against the caller's budget. The same path never ran the rate limiter either, so an intercepted search was free of the key's RPM/TPM limits.
The search now carries the originating key's attribution metadata (key hash, alias, user, team, org, plus model_group set to the resolved search tool) and runs the caller's rate limit checks before hitting the provider, matching what a direct /v1/search request gets. SDK calls with no proxy auth context are unchanged.
Resolves LIT-5033
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
create_a2a_client took the raw client off a process-wide cached handler and
called headers.update() on it, then leaned on folding the header set into the
cache key (through the unrelated disable_aiohttp_transport field) to keep one
caller's credentials away from the next.
Per-caller headers now ride with each request through the a2a SDK's call
context, and the agent card fetch gets them through resolver_http_kwargs, so
the shared client is never written to and its cache key no longer varies by
header set. Since the proxy puts a fresh trace id in every request's headers,
that key previously changed on every call, giving each request its own httpx
client and flushing the 200-entry client cache that every other provider
shares. All A2A callers on one timeout now reuse a single pooled client.
Sharing that client also means sharing its httpx cookie jar, which httpx fills
from every Set-Cookie and replays on any later request to a matching domain, so
one agent's session cookie would arrive at another agent on the same host. The
pooled client now carries a cookie policy that stores and sends nothing, which
neither litellm nor the a2a SDK relies on: the SDK's auth interceptor skips
cookie-borne API keys outright.
* fix(proxy): re-assert the authenticated identity on passthrough requests
The passthrough merges the client's litellm_metadata into the request metadata
and then re-asserts only user_api_key and the parent span. Every other identity
field the spend and budget pipeline reads stays whatever the request body set,
so a body carrying user_api_key_user_id, user_api_key_team_id,
user_api_key_org_id or user_api_key_end_user_id charges that user, team, org or
end user instead of the caller.
Re-assert the whole sanitized identity after the merge, so the client's copy of
any of those fields is overwritten by the authenticated key's own values.
* test(passthrough): assert no authenticated identity field is client settable
The existing regression names seven fields; the re-assertion covers every field
get_sanitized_user_information_from_key returns, which is twenty today. Derive
the set from the helper so a field added to StandardLoggingUserAPIKeyMetadata is
covered without touching the test.
Two of the twenty were not covered before, including user_api_key_hash, which is
distinct from user_api_key and was client settable.
get_litellm_params returned metadata=None whenever only litellm_metadata was
supplied, which overwrote the fallback function_setup had already applied and
left litellm_params["metadata"] empty. On the /v1/responses
completion-transformation bridge, used by every provider without a native
Responses API config, and on /v1/messages, that discarded the caller's trace
fields a second time after the proxy had promoted them.
Resolve metadata to a copy of litellm_metadata when metadata is empty, guarding
on isinstance because the proxy leaves an unparseable litellm_metadata string in
place and a null metadata would otherwise suppress the backfill and break the
merge. update_from_kwargs copies rather than aliases for the same reason: on
these routes it is handed the caller's provider-bound dict and would otherwise
write user_api_key_auth into it.
The shared `timeout` guardrail param already parsed into LitellmParams, but
the Zscaler initializer never forwarded it and _send_request hardcoded a 5
second constant, so a configured value was silently ignored and slow scans
failed with `Timeout passed=5` regardless of config.
Forward litellm_params.timeout through to the HTTP call, keep 5 seconds as
the default, fall back to it for non-positive values, and declare the field
on the config model so the dashboard renders it.
* fix(proxy): promote caller metadata trace fields into litellm_metadata
Routes in LITELLM_METADATA_ROUTES keep the caller's metadata as a provider
passthrough field and track proxy state in litellm_metadata, which is the dict
the logging integrations read. The caller's trace_id, session_id, trace_user_id
and trace_metadata therefore never reached any callback on /v1/responses,
/v1/messages, /v1/batches or /v1/files, and mask_input / mask_output were
dropped with them so a caller asking for redaction had their prompt logged in
full.
Promote an explicit allow-list of those fields from the requester_metadata
snapshot into litellm_metadata, never overwriting a value already set so
header-derived ids keep precedence. Trace-mutation controls (existing_trace_id,
update_trace_keys) and trace_public are deliberately excluded: langfuse applies
them to an arbitrary caller-chosen trace with no ownership check. tags is
excluded because per-tag budget enforcement runs earlier, at auth time.
This covers providers with a native Responses API config. Providers reaching
/v1/responses through the chat-completions bridge need the companion change to
get_litellm_params.
* ci: retrigger workflows
* warn at startup when a proxy-wide budget is set but no DB is connected
litellm.max_budget is only enforced via DB-loaded global spend, so a DB-less proxy silently ignores it. Log a one-time startup warning.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): inject max_budget into DB-less budget warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover DB-less budget warning startup call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): pin DB-less budget warning call site
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): stabilize budget warning call-site pin
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: tin <tin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
PR #35956 added the max_input_chars passthrough to the AutoRouter
constructor but left this mock assertion in tests/router_unit_tests
unchanged, so test_init_auto_router_deployment_success has been failing
on litellm_internal_staging ever since.
The passthrough itself is intentional and its behaviour is already
covered by TestAutoRouterMaxInputCharsWiring in tests/test_litellm, so
only the stale expected kwargs need updating. Assert the shared constant
rather than the literal 2000 so tuning the default does not break this
test again.
When store_model_in_db is true, general_settings are persisted to the
LiteLLM_Config DB table. On subsequent startups and periodic reloads,
_add_general_settings_from_db_config() unconditionally overwrites the
in-memory general_settings with DB-cached values, including
store_prompts_in_spend_logs.
This means a YAML config change (e.g. store_prompts_in_spend_logs: false)
deployed via CI/CD has no effect because the stale DB value (true) always
wins. The admin must manually update via /config/update API after every
deploy, defeating config-as-code.
Fix: track which general_settings keys were explicitly set in YAML at
startup (_yaml_general_settings_keys). During DB config merge, prefer the
YAML value for tracked keys. The DB value is only used as fallback when
YAML does not set the key, preserving the admin UI's ability to change
settings at runtime.
Steps to reproduce:
1. Start proxy with store_model_in_db: true, store_prompts_in_spend_logs: true
2. Change YAML to store_prompts_in_spend_logs: false, restart
3. Send a request, query LiteLLM_SpendLogs - prompts still stored
4. Check LiteLLM_Config table - DB still has true, overriding YAML
Slack thread: https://dataset-jsonhackathon.slack.com/archives/C0ACUS7LM29/p1785835131860139
GET /v1/files filters data down to the caller's own managed files but left first_id and last_id as the upstream page's, so a non-owner got back file ids belonging to other users even with an empty data array
Revert the LIT001 build-clause inserts, phrase the LIT002 menu as
'or (if it really must be dynamic) a MappingProxyType wrapping a dict
literal or comprehension', and fold the two freezing-wrapper exemption
sentences into one that names MappingProxyType beside tuple/frozenset.
LIT001/LIT002 and the typing.Dict ban all steered dict-shaped values to
frozen dataclasses or suppression even though the checker already accepts
MappingProxyType as a freezing wrapper; the messages now name it so the
dict-shaped freeze path is actually discoverable at fix time.
Storing a baseline used to prune every other cache entry, so gate runs in
concurrent worktrees kept evicting each other's baselines and forcing full
recomputes: this bit six times across two nights of benchmarking. The store
now writes alongside existing entries and evicts only the oldest beyond
eight, keyed as before by merge-base and environment fingerprint, so
parallel worktrees' baselines simply coexist
