JWT auth caches the user row before it adds the user to the JWT's team, and admission checks the credential's team against the cached row on whichever worker takes the next request. On a two-worker gateway the credential minted for a newly joined team answered 403 "not in your team memberships" until the management-object TTL ran out, because /team/member_add only evicted the membership spend sentinel. The add now evicts the added members' cached user rows and broadcasts the eviction to the other workers, the way /team/member_delete already did
The mint test now also covers a user SCIM deactivated after the cache last saw them active: the database read refuses the mint while the cached row still says active
The token exchange mint keeps reading the user row from the database, since JWT auth caches the user it creates before adding it to the JWT's team and a mint off that cached row refused the first exchange for a new user. Introspection and the refresh revalidation go back to the cache read, so a resource server calling /introspect per request pays no database read.
JWT auth caches the user it creates before it adds that user to the JWT's
team, and adding a team member never evicts the cached user row, so the mint
read a row with no teams and refused the very first token exchange for a
never-seen user as not a member. The loader now reads the row from the
database and leaves the fresh row in the cache for the requests the
credential makes next
The exchange refused the very user JWT auth upserts (its row has no
user_role) as "no longer active". The credential now carries the role the
proxy already enforces for that user on every request, internal_user when
the row has none, the same rule _get_user_role applies on the data plane.
A rejected subject_token no longer echoes JWT auth's wording on the public
/token endpoint: the response is a fixed invalid_request and the reason
goes to the proxy log, since that wording can name the JWKS URL or relay
the IdP's reply.
The exchange grant is listed on /register, /.well-known/litellm-cli-auth,
and the aggregate authorization-server metadata only when JWT auth is on,
backed by a database, and licensed, so a client never selects a grant the
gateway would then refuse.
A registered gateway DCR client can now POST /token with
grant_type=urn:ietf:params:oauth:grant-type:token-exchange and an IdP
JWT as subject_token. The gateway proves the JWT the way its JWT auth
does, resolves the user and team, and answers with the proxy-API
credential and a refresh token, so a fresh laptop with only an IdP login
gets a gateway key without a browser round trip.
"/token" joins mcp_inference_routes so the default JWT team allowlist
reaches the exchange, and the JWT auth builder accepts any header
mapping so the request headers pass through unchanged.
Trimming the streamed buffer to the retained tail could drop a category
exception phrase that suppresses a later keyword, or the identifier word
of an unfinished sentence that a conditional category pairs with a later
block word. Refuse the cut while either would leave the buffer so the
bounded scan masks and blocks exactly like a scan of the full text
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The streaming post-call hook rescanned the whole accumulated choice buffer on every chunk, so scan cost grew quadratically with output length. Keep a bounded per-choice buffer instead: once it exceeds twice the scan context, drop the head when masking the head and tail separately yields the same output as masking the whole buffer, so no pattern, phrase or exception straddles the cut. Detections from the dropped head are kept and merged, deduplicated, into the final log row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A body that serializes litellm_trace_id as null or an empty string carries no identity, so it must not
block the server span fallback. Also mark the nested metadata write as an out-param store
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
UserAPIKeyAuth.parent_otel_span is Any at runtime (opentelemetry is an optional extra), so the OTel
trace-id fallback must only format an int trace id, otherwise an object that merely quacks like a span
turns the whole request into a 500
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
When the otel callback is enabled and the client sends no trace or session identity, the request now inherits the W3C trace id of the proxy's server span as litellm_trace_id and metadata.trace_id. The missing_session_id policy and SpendLogs then persist that value as session_id, so a trace in the OTel backend and its row in the Logs UI carry the same id. Explicit x-litellm-trace-id, traceparent, body metadata.trace_id and litellm_trace_id keep priority.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The first cache-enabled litellm-e2e build (211) showed three gaps in the shared provider cache:
Every OpenAI response carries Cloudflare bot-management Set-Cookie headers, and the capture rejected any response with Set-Cookie, so no OpenAI response was ever recorded (179 of 372 misses rejected). The edge already withholds Set-Cookie from the proxy, so drop it before validating and storing instead of rejecting.
The provider prompt-caching tests need fresh provider state: a replayed priming response reports cache creation rather than a cache read, and the TPM test then trips the key limit. Mark both modules provider_live.
TestApiBaseSeam::test_live_mode_returns_none ran inside the cache-enabled runner and saw the shared edge; isolate it from E2E_PROVIDER_CACHE.
