The core now decides when an over-budget team member may fall back and walks
the router's fallback targets in order, skipping the requested model, through
its own TeamMemberStep and driver. The proxy handler only reads the targets
and vets each one, so _try_team_member_model_budget_fallback and its
per-model helper are gone
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The core now owns which models get checked, when an over-budget key may fall
back to its budget_fallbacks, and that the user check sees the rerouted model.
The virtual-key chain runs it inline, custom auth through check_model_budgets
and JWT through check_user_model_budgets, so the three paths share one walk.
A budget refusal is now a ModelBudgetExceeded denial that the proxy turns back
into the same BudgetExceededError
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The core now decides the soft budget alert, when per-model budgets run, the
team and project context load, the proxy budget alert and the missing-token
denial, so the builder only projects the grant. One projection serves both
the request builder and internal jobs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The key checks that ran first in validate_resolved_virtual_key now run in
the Python core, in the same order: blocked key, model access, deactivated
owner, budget exemption, team member budget, expiry, budget alerts and the
key's own max budget or throttle. Lookups, counters and alerts are effects
the proxy answers, and internal jobs run the same checks through
start_resolved_key.
The soft budget, per-model budgets and the team, project and global spend
lookups stay in _finish_virtual_key_auth for a later pass. The source-shape
tests now read the functions that replaced validate_resolved_virtual_key
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Python core now decides the whole virtual key resolution path. It asks
the proxy to resolve the end user, the presented key (preresolved, cached or
UI login), evict an expired cached admin key and load an uncached key from
the DB, each as a typed effect with its own continuation. It decides the
admin key expiry, master-key-only routes, the no-DB case and the sk- prefix
check itself, so the builder no longer has a legacy fallback.
The budget and team checks in validate_resolved_virtual_key still run after
Allow(VirtualKey) and move into the core in later passes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the AuthCore interface in litellm._auth and a Python core that decides
the no-master-key, missing key, malformed key, /user/auth and master key
cases. Everything else defers to the legacy builder. The builder projects
the outcome into UserAPIKeyAuth or the existing error contract, and the
end-user resolution it shared with the master key path is extracted.
A master key request no longer inherits end-user limits from an earlier
master key request, since the master key is no longer read back from the
key cache
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move scheme stripping, key-source precedence, websocket key selection,
hash_token and is_master_key into litellm/_auth as the interface the Rust
auth port will mirror. Authorization schemes now match case-insensitively
like FastAPI's HTTPBearer. Pass-through endpoint keys join the precedence
table instead of a separate fallback, and the dead custom-header helper
plus the duplicated hash_token and is_master_key bodies are gone
* fix(zscaler_ai_guard): scan only the last user turn on requests
* fix(zscaler_ai_guard): keep scanning A2A requests and refuse scan_only_tool_results
* fix(zscaler_ai_guard): join the last user turn's text parts with spaces like before
* fix(zscaler_ai_guard): keep scanning tool results when scan_only_tool_results is set
* fix(zscaler_ai_guard): scan the latest user turn when assistant or tool messages follow it
* fix(zscaler_ai_guard): skip the scan when the latest user turn is only whitespace
A blank latest user turn was sent to Zscaler on its own, which answers
Content is empty, so the proxy returned 500 where main returned 200
---------
Co-authored-by: Arnav Maiti <amaiti@zscaler.com>
* fix(anthropic): report temperature/top_p as supported only when the model keeps them
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(vertex_ai): accept the generic unsupported-param message for Claude batch rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(accounting): keep spend commits working when prometheus_client is not installed
The delivery worker imported the ledger metrics module, which imported
prometheus_client at module level. update_database imports the worker
outside its try block, so installs without the proxy-runtime extra failed
every spend commit. The metrics now register only when prometheus_client
is importable and are skipped otherwise
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style(accounting): note why prometheus_client can be missing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(openai): authenticate raw chat requests with the deployment api_key only, never OPENAI_ADMIN_KEY
openai 3.x request options default to bearer_auth plus admin_api_key_auth, so a deployment whose api_key is empty sent Authorization: Bearer <OPENAI_ADMIN_KEY> to its api_base. Pass security={"bearer_auth": True} when the installed SDK's make_request_options accepts it (openai 2.x has no admin key fallback and no security kwarg)
* test(openai): assert the admin key never reaches the wire whether the SDK sends unsigned or refuses
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): stop the wire peer from dropping requests whose bytes arrive late
The scripted wire peer set a fixed 5 s socket timeout, so a request whose
bytes reached it late on a loaded rig was dropped with no reply and the
proxy answered 500 Server disconnected. Remove the timeout and, at
teardown, stop reading on every accepted connection so idle handlers exit
at once. A request still arriving when teardown starts never reaches the
test's respond callback.
* fix(tests): bound wire peer teardown on unread replies and answer complete requests
* fix(tests): poll wire peer reads so teardown keeps queued requests and only cuts stuck calls
* fix(tests): write wire peer replies in pieces so teardown never cuts a client still reading
* fix(tests): give a wire peer reply the whole teardown grace when its write blocked before teardown
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(accounting): keep one undecodable delivery from blocking the ledger
A request whose metadata carried a 401-digit tokens_saved stored that number in
its delivery payload. The query engine could not decode it when the worker
claimed the batch, so every later delivery on the proxy stayed pending
The worker now claims the payload as text and decodes each row in Python. A row
that still cannot be decoded is parked alone while the rest of the batch
applies. The payload builder also clamps integers to the JSON-safe range, so
caller-supplied counts can no longer reach the engine at that size
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): take spend locks in one order so the ledger cannot deadlock
A ledger delivery updates many spend tables in one transaction, so it can
deadlock with any writer that locks the same rows in another order. It took the
team advisory lock after the user and team rows, while team roster writers take
it first
The delivery now takes every team advisory lock before any row, then writes
tables in one canonical order with ids sorted. It also runs under the rollup
lock_timeout, applies 500 deliveries per batch by default, and spreads retries
with jitter
The writers that could still collide with it now follow the same order. The
budget reset cascade, model allowlist rename, global spend reset, bulk user
update, org delete and the delete-by-list paths lock their rows in key order
first, and access group endpoints update teams and keys in one sorted pass
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): prove a ledger delivery waits instead of deadlocking
Real Postgres tests hold the first lock of a team roster change, an access group
update and a budget reset cascade, start a delivery, and let the other writer
finish. The delivery must wait and then charge once. Before the lock order fix
each case ends in a Postgres deadlock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(accounting)!: make the SpendLogs ledger the default accounting path
use_spend_logs_ledger now defaults to true, so every request is charged through
LiteLLM_SpendLogDelivery. Setting it to false rolls back to the in-memory and
Redis spend queues, which stay in place. Tests that cover those legacy writers
pin the setting off, and the rest check the queued charge the ledger stores
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): let the mocked client open a transaction for key and team deletes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): take team locks first when a model rename rewrites allowlists
A rename locked team rows before user rows while a roster change holds the team
advisory lock and a user row, so the two could deadlock. The rename now takes the
advisory lock for every team that lists the model, sorted, before any row lock.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): drain a backlog in one worker run, one batch per transaction
One run applied a single fresh batch, so at 500 per batch charges fell behind any
load above that rate. The worker now keeps claiming fresh batches while they come
back full, for up to five seconds, and each batch still commits on its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): queue only the charge when a request's spend log is suppressed
With logging off for a request, the ledger still queued the full row, so a failed
flush or a shutdown could leave its prompt and response in Redis. The queue now
holds only the fields the charge needs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style(proxy): format the rename team listing query
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): remove expired delivery receipts as fast as the worker applies them
The worker now applies several batches per run but still removed one cleanup batch
of applied receipts, so the table could outgrow its retention window. Cleanup now
repeats while each pass removes a full batch, for up to five seconds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): retry a model rename when a team starts listing the model mid-lock
The team advisory locks come from a snapshot, so a team that gained the old name
before its row lock was locked without its advisory lock. That attempt now commits
without writing and the next one locks the fresh set of teams.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
