Post-auth denials from can_key_call_resolved_model (per-request alias
rewrite, MCP sampling, realtime) never reach the auth exception handler,
so the internal allowlist reason was dropped when
model_access_denied_message was set. Log it once from the ProxyException
response handler and the realtime rejection path instead, and convert
JWT ModelAccessDeniedHTTPException into the specialized ProxyException so
the same boundary covers it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replace the username-global lockout with counters keyed by source address and by
source/username pair. Each has a fixed counting window (60s) and a separate
block TTL (300s). Blocks are soft: a correct password still signs in, wrong
passwords from a blocked key take one of 5 held slots per worker and are held
30s before a 429. Once a pair is blocked its failures stop counting against the
source. The source scope runs only when trusted_proxy_ranges is set, IPv6 is
grouped by /64, and per-source limits accept IP and CIDR overrides with
longest-prefix matching. Redis is authoritative through one Lua script per
failure, with bounded per-worker fallback when Redis raises.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The write path no longer dual-writes the global table. The cron rolls up closed UTC days
only, so a pod still flushing the current day can never leave the global table short. The
key-free arm reads days through the marker from the global table and later days from
LiteLLM_DailyUserSpend in one UNION ALL, and the marker comes from the config cache
rather than a per-request database lookup.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Move the WARNING that carries the internal denial detail out of the message formatter and into the auth exception handler. The denial exceptions now carry internal_message so access-group probes and fallback paths that catch and recover from the denial no longer log a false denial for an allowed request
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The merge with litellm_internal_staging resolved the test file by taking this
branch's copy whole, which discarded the two tests staging had appended to the
same end-of-file region:
test_delete_evicts_cache_after_row_is_gone
test_update_evicts_old_and_new_cache_keys_after_write
Both sides only appended, so the conflict was additive and nothing had to be
chosen between them. The eviction code those tests cover did survive in
jwt_key_mapping_endpoints.py, so the branch was shipping it untested.
Appending staging's block restores them alongside the nine resolver tests here.
The file is now a superset of both sides, verified line by line, and the test
quality count stays at the baseline of 28 because staging's block carries its
own test-quality-ok suppressions.
A JWT key mapping can now name its virtual key by the SHA-256 hash the proxy
already stores, instead of only by the plaintext key.
litellm_key makes its generated key write-only so raw keys stay out of Terraform
state, and write-only attributes cannot be referenced at all, so the natural
wiring fails while planning, in every apply ordering:
Error: Missing required argument
with litellm_jwt_key_mapping.example
key = litellm_key.example.key
The argument "key" is required, but no definition was found.
The only way out today is supplying the plaintext from a variable or a secret
manager, which means the mapped key cannot be one the proxy generated and the
configuration has to carry a credential. The value the mapping stores is
hash_token(key), which is the same hash litellm_key already exports as
token_id, and a hash is not a credential, so accepting it closes the gap:
resource "litellm_jwt_key_mapping" "service" {
jwt_claim_name = "client_id"
jwt_claim_value = "reporting-service"
token_id = litellm_key.service.token_id
}
CreateJWTKeyMappingRequest and UpdateJWTKeyMappingRequest gain an optional
token. Create requires exactly one of key or token, update accepts at most one,
and omitting both still leaves the mapped key alone. A supplied token must be 64
lowercase hex characters, because hash_token() hashes unconditionally and a
plaintext key sent as token would be stored as a hash of a hash, then silently
match nothing at auth time. Both rejections are 400s raised before the row is
written.
On the provider side, key becomes Optional with ExactlyOneOf{key, token_id} and
token_id is added next to it. token_id is not marked sensitive since a hash is
not a credential, both fields are omitempty on the wire so the proxy receives
only the one that was configured, and a failed update reverts token_id for the
same reason it already reverts key.
key keeps working unchanged and existing state is untouched. The only change to
it is Required to Optional, which no existing configuration can violate.
Adds a daily spend table without api_key or user_id, written atomically alongside
LiteLLM_DailyUserSpend from the batched writer, reconciled from history by a
scheduled job that advances a marker in LiteLLM_Config, and read by the key-free
arm of the aggregated usage query once the marker covers the requested range.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Match the pass-through skip to what FastAPI actually dispatched (the user-defined
endpoint marker or a provider handler's {endpoint:path} param) instead of the
mapped route prefixes, which also cover native routes such as /openai/v1/responses
and /cursor/chat/completions. Wrap the added test lines to the 120-column limit.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The recursion is bounded to depth 1: split parts each carry a single
service_tier, so the recursive call's partition has one key and the split
helper returns None.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The logging object built for aresponses_websocket sessions dropped the
service_tier carried by each billable response.completed/response.incomplete
event, so sessions were priced at the default tier. The logging object now
carries the tier when the session is single-tier, and completion_cost splits
mixed-tier sessions into one object per tier before pricing. Fixes#41299
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Route Vertex and Gemini /interactions create responses through the shared
Interactions usage transformation so SpendLogs carry per-modality tokens and
cost instead of zeros, and carry parameters.sampleCount into Veo usage as
video_count so duration-based video cost is multiplied by the number of
generated videos
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
