The merge with litellm_internal_staging resolved the test file by taking this
branch's copy whole, which discarded the two tests staging had appended to the
same end-of-file region:
test_delete_evicts_cache_after_row_is_gone
test_update_evicts_old_and_new_cache_keys_after_write
Both sides only appended, so the conflict was additive and nothing had to be
chosen between them. The eviction code those tests cover did survive in
jwt_key_mapping_endpoints.py, so the branch was shipping it untested.
Appending staging's block restores them alongside the nine resolver tests here.
The file is now a superset of both sides, verified line by line, and the test
quality count stays at the baseline of 28 because staging's block carries its
own test-quality-ok suppressions.
A JWT key mapping can now name its virtual key by the SHA-256 hash the proxy
already stores, instead of only by the plaintext key.
litellm_key makes its generated key write-only so raw keys stay out of Terraform
state, and write-only attributes cannot be referenced at all, so the natural
wiring fails while planning, in every apply ordering:
Error: Missing required argument
with litellm_jwt_key_mapping.example
key = litellm_key.example.key
The argument "key" is required, but no definition was found.
The only way out today is supplying the plaintext from a variable or a secret
manager, which means the mapped key cannot be one the proxy generated and the
configuration has to carry a credential. The value the mapping stores is
hash_token(key), which is the same hash litellm_key already exports as
token_id, and a hash is not a credential, so accepting it closes the gap:
resource "litellm_jwt_key_mapping" "service" {
jwt_claim_name = "client_id"
jwt_claim_value = "reporting-service"
token_id = litellm_key.service.token_id
}
CreateJWTKeyMappingRequest and UpdateJWTKeyMappingRequest gain an optional
token. Create requires exactly one of key or token, update accepts at most one,
and omitting both still leaves the mapped key alone. A supplied token must be 64
lowercase hex characters, because hash_token() hashes unconditionally and a
plaintext key sent as token would be stored as a hash of a hash, then silently
match nothing at auth time. Both rejections are 400s raised before the row is
written.
On the provider side, key becomes Optional with ExactlyOneOf{key, token_id} and
token_id is added next to it. token_id is not marked sensitive since a hash is
not a credential, both fields are omitempty on the wire so the proxy receives
only the one that was configured, and a failed update reverts token_id for the
same reason it already reverts key.
key keeps working unchanged and existing state is untouched. The only change to
it is Required to Optional, which no existing configuration can violate.
Evicting before the mutation commits left a race: a concurrent JWT
request could re-cache the old mapping between the eviction and the
commit, keeping a deleted or renamed claim authorized until the cache
TTL expired. Flagged by review on PR #39808.
Every repository handed its `.table` back untyped, so a dozen modules had
each grown a private `_PrismaTableActions` Protocol to paper over it. They
had drifted: some declared `update` as returning the row, others the row or
None, and none agreed on whether `find_many` was covariant
Replace all of them with a single `TableActions[RowT_co]` in
`litellm/repositories/prisma_protocols.py`, keyed to the prisma row each
repository is bound to. Query inputs stay `Mapping[str, object]` so callers
keep passing plain dicts, and `find_many` returns `Sequence` so the row type
stays covariant
Typing the nullable returns honestly surfaced paths that were already
crashing. A team admin could never edit or delete a memory entry owned by
their team: the write-auth check fed a raw prisma row to a helper that
expects the domain model, so `members_with_roles` arrived as plain dicts and
the request died as a 500 instead of applying the edit. Non-admin members hit
the same 500 in place of the 403 they were owed, so refusal and breakage were
indistinguishable. `/v2/model/info?user_models_only=true` dereferenced a
missing user row rather than returning the 400 the route already had, three
team routes dereferenced a team deleted between the read and the write, and
the agent registry dereferenced a missing agent instead of naming it
basedpyright drops 2,132 errors, 1,454 of them reportAny and 73
reportExplicitAny. The dashboard's generated types pick up `string[]` where
they had `unknown[]` for a team's members, admins and models
Introduce a single, typed caller identity that is resolved once at the auth
boundary and read by reference downstream, instead of being re-derived from a
50-field key object or rebuilt from request metadata.
What this adds (litellm/proxy/auth/resolvers/), organized by responsibility:
- Principal: a small, frozen, identity-only value type (user / organization /
teams / project / end-user / roles / scopes / network), with its sub-models
and the role mapping. No budget or policy state; those stay on the key object.
- DbIdentityStore: the auth flow's resolver, owning both halves of resolving a
caller. resolve_key does the one combined_view lookup (cache, then DB via the
shared lower-level helpers, then write-back) and returns the key object, which
still flows for budget / rate-limit / policy unchanged. principal_from_key
projects the identity slice of that key object into a Principal, issuing no
lookup. user_api_key_auth resolves every key through the store rather than
calling get_key_object directly; auth_checks.get_key_object stays as the legacy
entrypoint for its other callers until they migrate.
- network: the X-Forwarded-For / trusted-proxy CIDR primitives live here in one
place. trusted_proxy_utils now imports them rather than keeping a second copy.
At the seam, user_api_key_auth projects one per-request Principal off the
resolved key object and stamps the request network context onto it once
(X-Forwarded-For is trusted only when trusted_proxy_ranges is configured). It is
attached to request.state.principal for the downstream consumers later phases
add. The projection is additive and defensive: a failure never rejects an
already-authenticated request, and a missing principal must be treated as deny
by any future reader. The Principal is always identifiable (credential_ref and a
stable subject off the token), never anonymous.
This is additive and changes no behavior today; it is the identity foundation
the spend-attribution and authorization phases build on.
* restore an explicit no-match policy
* fix(jwt): fix AUTO_REGISTER sentinel bypass, race condition, and inline import comment
- AUTO_REGISTER now evicts stale __NO_MAPPING__ sentinel instead of silently
returning None when cached under a prior fallback_team_mapping config
- Race condition in _auto_register_jwt_mapping: catch P2002 unique-constraint
violation on concurrent creates, fetch the winning mapping, proceed cleanly
- Added comment on inline generate_key_helper_fn import explaining the circular
dependency (key_management_endpoints imports user_api_key_auth at line 51)
- 3 new tests: stale sentinel eviction, race condition winner fallback, and the
existing auto_register happy path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(jwt): cache __NO_MAPPING__ sentinel before raising 403 in REJECT mode
REJECT mode was raising HTTPException immediately on a DB miss without writing
the __NO_MAPPING__ sentinel, causing every subsequent rejected request to
re-query the DB. Write the sentinel first so repeated rejections are served
from cache within virtual_key_mapping_cache_ttl.
Adds test asserting DB is not hit on the second reject after a cache-warm miss.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(jwt): enforce no-match policy when prisma_client is None
The early `if prisma_client is None: return None` guard ran before the
no-match policy check, silently bypassing REJECT and AUTO_REGISTER — every
JWT client fell through to team auth regardless of configuration.
Fix: treat prisma_client=None as a definitive DB miss and fall through to the
same policy block as a real miss. REJECT now raises 403, AUTO_REGISTER raises
500 with a clear message (can't create keys without a DB), FALLBACK_TEAM_MAPPING
returns None unchanged.
Adds three tests: REJECT/403 with no DB, FALLBACK returns None with no DB,
AUTO_REGISTER/500 with no DB.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(jwt): consistent AUTO_REGISTER on cached sentinel; clean up race orphans
Addresses Greptile review on PR #25570 cherry-pick.
1. Inconsistent AUTO_REGISTER when __NO_MAPPING__ sentinel is cached:
The cached-sentinel branch silently returned None when prisma_client was
None, while the fresh path raised HTTP 500 under the same config. Same
request, different access-control outcome depending on cache state. Both
paths now raise the same 500.
2. Orphaned virtual keys from race-condition losers:
On unique-constraint conflict, generate_key_helper_fn had already persisted
an unrestricted virtual key in LiteLLM_VerificationToken with the cleartext
in request memory. Under sustained concurrency these accumulated
indefinitely. The loser now deletes its orphan before falling back to the
winner's mapping; failure to delete is logged but does not fail the request.
Also corrects a latent FK bug surfaced while fixing #2: the mapping row was
storing the plaintext key in LiteLLM_JWTKeyMapping.token, but that column FKs
to the hashed LiteLLM_VerificationToken.token — now hashed at the call site.
Tests:
- updated test_auto_register_creates_key_and_mapping to assert the hashed
token is stored, not the plaintext
- updated test_auto_register_race_condition_unique_conflict to assert the
orphan is deleted with the correct hashed token
- added test_auto_register_raises_500_when_sentinel_cached_and_no_db
- added test_auto_register_race_conflict_tolerates_delete_failure
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(jwt): close REJECT bypass when JWT omits the configured claim field
A JWT presented without the configured `virtual_key_claim_field` previously
returned None at the `claim_value is None` guard before the
`unregistered_jwt_client_behavior` check ran. A caller who knows the configured
claim-field name could bypass REJECT by simply omitting that field and falling
through to team-based JWT auth.
Apply the no-match policy on a missing claim:
- REJECT → 403
- AUTO_REGISTER → 403 (no stable identity to map; refuse rather than
create a sentinel-keyed record)
- FALLBACK_TEAM_MAPPING → return None (unchanged, backward-compatible)
Adds three tests covering each branch of the missing-claim path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(jwt): AUTO_REGISTER inherits team_id so keys are bounded by team limits
Auto-registered virtual keys were created with no team, model, route, rate, or
budget constraints — broader access than the standard team-based JWT auth path
the same client would have taken. Under AUTO_REGISTER, resolve the team_id
from the JWT (via the operator-configured team_id_jwt_field / team_id_default)
and stamp it on the new key. Downstream auth then applies the team's
budget/models/tpm/rpm/allowed_routes via the existing virtual-key flow.
Policy when team_id_jwt_field is configured:
- JWT carries team claim → stamp resolved team_id
- JWT lacks claim + team_id_default set → stamp default
- JWT lacks claim + no default → 403 (refuse to create an unbounded key)
When neither team_id_jwt_field nor team_id_default is configured, the
operator has explicitly opted out of team-based limits — the auto-created
key has no team_id (matches what team-auth would do in the same config).
Adds 4 tests covering each branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(jwt): make AUTO_REGISTER functional in prod; raise on missing winner
Two correctness fixes flagged by Greptile on the AUTO_REGISTER path:
1. generate_key_helper_fn was called without table_name="key". Without that,
the helper falls into the user-upsert branch (table_name in (None, "user"))
and tries to insert into LiteLLM_UserTable with user_id=None, which hits
the NOT NULL @id constraint. AUTO_REGISTER would never have succeeded in
production. Now passes table_name="key" explicitly, matching the
/key/generate caller.
2. When the race loser refetches the winner's mapping and gets None (winner
row concurrently deleted), the previous code returned None — and the
caller in _resolve_jwt_to_virtual_key then fell through to less-
restrictive team-based JWT auth, silently bypassing the configured
AUTO_REGISTER policy. Now raises HTTP 503 so the caller retries against
a stable state rather than getting unintended fallback access.
Adds one test for the 503 winner-vanishes path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(jwt): defer AUTO_REGISTER until JWT policy is enforced by auth_builder
Closes the JWT policy bypass on the AUTO_REGISTER path flagged by veria-ai.
Before: when unregistered_jwt_client_behavior=auto_register and the JWT's
claim was unmapped, _resolve_jwt_to_virtual_key validated the JWT signature
and then immediately created a virtual key + mapping. JWTAuthManager.auth_builder
never ran for the first request (the new key short-circuited the team-auth
path), and every subsequent request hit the cached mapping — so custom_validate,
RBAC, scope_mappings, and user_allowed_email_domain were never enforced for
auto-registered clients.
After: _resolve_jwt_to_virtual_key returns a _PendingAutoRegister signal
instead of creating the key. The caller in _user_api_key_auth_builder runs
JWTAuthManager.auth_builder, then — only on a validated, policy-passing
result — calls _auto_register_jwt_mapping with the team_id / user_id from
that result. The created key inherits team + user limits from the validated
identity, and future cache hits load that already-policy-checked key.
Also drops the interim _resolve_inherited_team_id helper that pulled team_id
from raw JWT claims — same bypass risk; team_id now comes exclusively from
auth_builder.
Tests:
- Rewrote two existing tests to assert _resolve_jwt_to_virtual_key returns
_PendingAutoRegister (no key created yet) for both the fresh-DB-miss
and stale-sentinel branches
- Added a contract test that _auto_register_jwt_mapping stamps the
validated team_id/user_id onto generate_key_helper_fn
- Removed four stale team-binding tests that exercised the prior
raw-claim helper
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Update user_api_key_auth.py
* fix(jwt): cache proxy-admin AUTO_REGISTER path to avoid repeated DB lookups
Cache-miss regression introduced by the deferred-auto-register refactor:
when a JWT under AUTO_REGISTER resolved to a proxy admin, the is_proxy_admin
early-return in _user_api_key_auth_builder ran *before* the pending
auto-register cache-write block. Result: no cache entry, so every
subsequent proxy-admin request re-queried get_jwt_key_mapping_object
indefinitely.
Fix: write a __JWT_PROXY_ADMIN__ sentinel to user_api_key_cache before the
early return when a pending auto-register existed. _resolve_jwt_to_virtual_key
treats that sentinel as "skip mapping, fall through to auth_builder", so
future requests from the same JWT identity hit the cache instead of the DB.
auth_builder still runs full JWT policy on every request — only the
mapping DB lookup is short-circuited.
Adds one test asserting the sentinel cache-hit returns None without
hitting prisma_client.db.litellm_jwtkeymapping.find_first.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(proxy): stamp org context on JWT auto-registered keys
AUTO_REGISTER keys were created with team_id and user_id only, so org budget checks were skipped after switching to the key-scoped path.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
- Remove token field from JWTKeyMappingResponse to prevent hashed key exposure
- Use _to_response() helper on all CRUD endpoints to control returned fields
- Return 409 for unique constraint violations, 400 for FK violations, 404 for not found
- Add response_model to endpoint decorators
- Add 8 new unit tests covering error handling and token redaction
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
