Adds a persistent total_spend column to LiteLLM_VerificationToken and LiteLLM_DeletedVerificationToken, incremented in the same write as spend and left alone by budget resets. Surfaces it on /key/info, /key/list and the Admin UI Virtual Keys table and key detail view
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The four .parentElement reads in the new lifecycle tests pushed
testing-library/no-node-access to 712 against a 707 budget, failing
frontend-lint. The rows now carry data-testid="lifecycle-row" and the test
picks a row with within(), which keeps the assertion tied to the specific row
rather than the whole panel and takes the count back to 707.
#39050 changed RequestLifecycle from sorting every entry with
(a.start_time ?? 0) to filtering on isTimed, which drops any entry whose
start_time/end_time are null. That was the right call for the not_run entries
the PR introduced, but it also drops entries that DID run and simply carry no
timing, and those are pre-existing: add_standard_logging_guardrail_information_to_request_data
defaults start_time, end_time and duration to None, and the conduct guardrail
passes none of them. One such entry used to draw the whole four-row lifecycle
and now draws nothing, so an admin opening that log sees an empty
Request Lifecycle panel.
An entry now stays on the lifecycle when it is timed OR when it ran, so not_run
keeps the exclusion #39050 wanted and every other shape comes back. Offsets are
number | null and render as an em dash rather than a fabricated T+0ms, which is
what a null minus a null used to produce on the base. Entries without timing
sort after the timed ones and the base time comes from the timed entries, so
real offsets are unchanged.
The two new tests fail on the base component and pass here; #39050's own
not_run tests keep passing untouched, which is what makes this additive rather
than a revert.
Overlapping routing_groups persisted from the Admin UI raised inside
Router._init_routing_groups during the DB config reconcile, which skipped
loading SSO, guardrails and the other DB-backed settings while leaving the
proxy healthy. /config/update now returns 400 for overlapping models,
duplicate names, the reserved default name and unknown strategies before
writing, the Router builds every group selector before replacing its state
so a rejected update keeps the previous groups routing, and the proxy applies
routing_groups separately from the other router settings so an already
persisted invalid value is logged and skipped instead of aborting the
reconcile. The Admin UI modal blocks picking a model another group owns.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A team can now carry a per-model budget map that every key on the team
inherits. A key's own model_max_budget entry for the same model takes
precedence, so it is gated on and billed to the key alone.
Backend: NewTeamRequest/UpdateTeamRequest accept model_max_budget (validated
like the key-level field, enterprise gated); the value is hydrated onto
UserAPIKeyAuth via the token view, TeamGrants and the carried budget state;
_check_team_model_budget enforces it in the centralized common checks; the
limiter meters spend under team_model_spend:<team>:<model>:<duration> and
skips the team counter when the key overrides; /team/update lets only a
proxy admin raise, re-window or drop a cap; /team/info exposes usage.
The Anthropic context-management compaction summary subrequest runs the
same team gate. Both fallback token-view SQL definitions project the column.
UI: team create and edit forms reuse the key-level ModelMaxBudgetEditor,
premium gated, sending {} to clear and omitting unchanged fields.
A key entry overrides the team cap only when it spend-gates the model
(non-negative max_budget); a row that only carries tpm/rpm limits or a
negative cap leaves the team cap in force.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replace the username-global lockout with counters keyed by source address and by
source/username pair. Each has a fixed counting window (60s) and a separate
block TTL (300s). Blocks are soft: a correct password still signs in, wrong
passwords from a blocked key take one of 5 held slots per worker and are held
30s before a 429. Once a pair is blocked its failures stop counting against the
source. The source scope runs only when trusted_proxy_ranges is set, IPv6 is
grouped by /64, and per-source limits accept IP and CIDR overrides with
longest-prefix matching. Redis is authoritative through one Lua script per
failure, with bounded per-worker fallback when Redis raises.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A loading flag only flips once the fetch effect runs, so the render right after a
date or filter change still reported the previous range as loaded and let an export
read its rows. Stamp the completed range on the hook and compare it during render
instead, the way the tiles already do.
Also stop the failure banner claiming a page loaded when the first request is what
failed, which left it reading 1/1.
A JWT key mapping can now name its virtual key by the SHA-256 hash the proxy
already stores, instead of only by the plaintext key.
litellm_key makes its generated key write-only so raw keys stay out of Terraform
state, and write-only attributes cannot be referenced at all, so the natural
wiring fails while planning, in every apply ordering:
Error: Missing required argument
with litellm_jwt_key_mapping.example
key = litellm_key.example.key
The argument "key" is required, but no definition was found.
The only way out today is supplying the plaintext from a variable or a secret
manager, which means the mapped key cannot be one the proxy generated and the
configuration has to carry a credential. The value the mapping stores is
hash_token(key), which is the same hash litellm_key already exports as
token_id, and a hash is not a credential, so accepting it closes the gap:
resource "litellm_jwt_key_mapping" "service" {
jwt_claim_name = "client_id"
jwt_claim_value = "reporting-service"
token_id = litellm_key.service.token_id
}
CreateJWTKeyMappingRequest and UpdateJWTKeyMappingRequest gain an optional
token. Create requires exactly one of key or token, update accepts at most one,
and omitting both still leaves the mapped key alone. A supplied token must be 64
lowercase hex characters, because hash_token() hashes unconditionally and a
plaintext key sent as token would be stored as a hash of a hash, then silently
match nothing at auth time. Both rejections are 400s raised before the row is
written.
On the provider side, key becomes Optional with ExactlyOneOf{key, token_id} and
token_id is added next to it. token_id is not marked sensitive since a hash is
not a credential, both fields are omitempty on the wire so the proxy receives
only the one that was configured, and a failed update reverts token_id for the
same reason it already reverts key.
key keeps working unchanged and existing state is untouched. The only change to
it is Required to Optional, which no existing configuration can violate.
Roll request_duration_ms of successful, non-internal requests into the
daily spend tables as total_response_time_ms plus timed_requests, expose
both through the daily activity endpoints, and derive the average in the
Usage -> Model Activity view of the Admin UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Persist and expose the lifecycle of API keys so spend, audit and FinOps
workflows can still resolve a key after it is revoked, expires or is
deleted.
/key/list?status= now accepts active, expired and revoked next to the
existing deleted value. revoked means blocked=true, expired means not
blocked with a past expiry, active is the rest, so the three values
partition the live key table. deleted keeps reading the
LiteLLM_DeletedVerificationToken archive.
/key/info falls back to that archive when the key is no longer in the
live table, running the same owner/team/org authorization check, and
every response now carries a derived status field. The hashed token is
still stripped.
The Virtual Keys page gets a Status filter (URL-persisted) and a Deleted
badge that shows when and by whom the key was deleted.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
