* fix(proxy): make /team/member_delete's four cleanups atomic
The team roster update, the user.teams update, the team membership
delete, and the team-scoped verification token delete ran as four
sequential writes with no transaction around them, so a failure
between any two left the removal half applied. Thread a single
prisma transaction through all four writes, following the same
tx.<table> pattern /team/member_add and /team/member_update already
use, so either all four land or none do.
* fix(team): serialize member_add, member_delete, and delete under the team's advisory lock
/team/member_add validated a team exists and then wrote the user's teams array and
a membership row without holding anything across that gap, so a /team/delete could
commit its reference sweeps in between and leave a member pointing at a team id that
no longer exists. The write path already re-read members_with_roles under a row lock
before this change, but SELECT ... FOR UPDATE can deadlock with the access-group
endpoints, which lock an access group and then a team.
member_add now takes pg_advisory_xact_lock(hashtext(team_id)) before re-reading the
team and only writes if it is still there, so a delete that already committed is
visible before any write happens. delete_team takes the same lock around its own
row delete and reference sweep, so the two requests can never interleave: whichever
acquires the lock first runs to completion before the other's read can proceed.
Dropping the row lock from member_add's read also dropped the incidental protection
it gave against a concurrent member_delete, which still wrote from the snapshot it
validated against, unlocked, and could silently overwrite whatever member_add had
just committed. member_delete now takes the same advisory lock and re-reads the
roster under it before computing its own write, so it can never resurrect a member
by overwriting from stale data.
Resolves LIT-5544
* fix(team): run member writes on the advisory lock's transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): keep member writes on the lock holder's connection after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): keep the transactional member create an upsert on user_id
The transaction path was creating the email-identified user row outright, where the
regular client path upserts on user_id. Share one upsert helper between both member
paths so the create stays idempotent on the lock holder's connection.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): read member_delete's user and key rows on the lock-holding transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(scim): return user_id as Group members[].value on transformed group responses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: format scim transformation tests per ruff
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: drop redundant assertion comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: retrigger ci
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): don't show a stale provider prompt-cache chip on a response-cache hit
The playground's non-streaming chat completion and responses paths replayed a cache hit's original usage payload verbatim, so ResponseMetrics kept rendering the provider's prompt-cache-write/read chips using token counts from the original request. Detect the hit via the x-litellm-cache-key response header and render a Response Cache indicator instead.
* fix(dashboard): expose x-litellm-cache-key through CORS for the playground cache-hit indicator
* fix(proxy): stop expected 4xx responses from saturating worker CPU on failure logging
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep threaded sync failure handler so CustomLogger sync callbacks still run
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: remove redundant comments per review
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The model Azure Model Router served was recovered by checking whether the text
"model_router" or "model-router" appeared in a model string. Spend logs applied that
check to the litellm model path, where the route prefix guarantees a match, but the
proxy applied it to the client's model group alias, which carries no prefix. A model
group named anything else therefore lost the selected model in both the response and
the spend row.
AzureModelRouterConfig now stamps the served model onto _hidden_params, and the spend
log payload and the proxy's response restamping read that stamp. The name heuristic
survives as a fallback for callers with no response in hand, routed through
get_azure_ai_route so it lives in one place.
* fix(proxy): reset a stuck team member's budget
A per-team-member budget check reads a cross-pod spend counter that
nothing ever invalidates. Once a member exceeds their per-member
budget, resetting the key's spend, raising the user's or the team's
own budget, or issuing a new key all leave the member stuck, because
none of them touch this counter or its cached membership object.
Add POST /team/{team_id}/member/{user_id}/reset_spend to reset a
member's tracked spend, and invalidate the same cached state from
/team/member_update when it raises a member's own budget, so that
path also takes effect immediately instead of waiting on the
membership cache's TTL. Name the entity in the check's error message
so a stuck member is diagnosable from the 429 body alone.
* fix(proxy): close reset-vs-floor-read race and surface double Redis write failure on member spend reset
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): broadcast spend reset as a SET so the handler's self-delivered message cannot erase the reset guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): omit null fields from the invalidation message so plain evictions keep the old wire format
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
/global/activity/cache_hits now returns an error_breakdown: failed spend
logs bucketed per call_type by error code and error class, read from
metadata->error_information. Clicking a red failed-requests segment on
the cache activity chart opens a per-code bar chart; hovering a bar
lists the error classes behind that code.
The video edit endpoint parsed the multipart body but dropped the uploaded
source video, only normalizing it to an id. When a raw file is uploaded it now
flows through videos.main -> the http handler -> the provider transform, which
emits multipart/form-data with the source video as a file part, matching the
official OpenAI SDK's videos.edit wire format. Edit-by-id still egresses JSON.
On mapped pass-through routes, of which /vertex_ai is one,
user_api_key_auth accepts the caller key from a header literally named
litellm_user_api_key and applies it last, so it overrides every other source.
The credential-less filter neither dropped it nor resolved the caller key from
it, so a virtual key there reached Google past a real x-goog-api-key, and a
bring-your-own Authorization could be stripped when auth actually came from that
header. Drop it by name and resolve it at highest precedence.
The resolver placed both operator-configured key headers at the top of its
precedence, but user_api_key_auth only overrides with litellm_key_header_name;
a pass_through_endpoints litellm_user_api_key is checked last. So a request that
authenticated via Authorization while also sending a pass-through header could
have the wrong value chosen, leaving the authenticated Authorization key
forwarded. Order the resolver exactly like get_api_key: override first, built-in
headers next, pass-through header last.
user_api_key_auth also accepts the caller key from a pass_through_endpoints
entry's headers.litellm_user_api_key, not just litellm_key_header_name. Drop
every operator-configured caller-key header by name and treat them as
top-precedence caller-key sources, so a virtual key sent through one is never
forwarded to Google.
The LIT-4761 streaming-classification tests passed only the bring-your-own
Google OAuth token in Authorization and mocked get_litellm_virtual_key, a shape
that cannot authenticate in production. The credential-less filter now resolves
the caller key by auth precedence, so a lone Authorization value reads as the
key and is stripped. Send the virtual key in x-litellm-api-key, matching a real
request, so Authorization is preserved and the classification assertions run.
The filter's own Bearer-only stripping missed the other schemes
user_api_key_auth accepts, so a virtual key echoed as `Authorization: Basic
<key>` alongside a higher-precedence auth header did not match the caller key
and was forwarded to Google. Reuse the auth module's _get_bearer_token so the
comparison strips exactly what authentication does (Bearer / bearer / Basic /
AWS4-HMAC-SHA256), falling back to the raw value for a bare token.
The credential-less filter derived the caller key only from x-litellm-api-key,
Authorization, and the custom header, but the route authenticates through
Depends(user_api_key_auth), which also accepts the key from x-goog-api-key. A
virtual key sent only in x-goog-api-key therefore authenticated yet was kept as
a preserved upstream header and forwarded to Google. Resolve the caller key by
the same precedence get_api_key uses and value-strip exactly that, so a key in
x-goog-api-key is stripped while a real Google key alongside a higher-precedence
virtual key is preserved.
The hand-rolled drop set missed Ocp-Apim-Subscription-Key, so a caller
Azure APIM secret in that header was forwarded to Google on the
credential-less branch. Derive the name-drop set from the canonical
SpecialHeaders.litellm_credential_header_names(), minus Authorization and
x-goog-api-key which double as real Google credentials and are value-stripped
instead. New credential headers added there are now dropped automatically.
Uploaded files reaching the RAG ingest path were trusted by client
filename and content-type, so archives and executable scripts were
ingested and malicious content was never screened. Enforce controls at
the upload boundary before the file leaves the proxy:
- classify content by magic bytes and a strict UTF-8 decode, never by
the client filename or content-type
- allowlist PDF and UTF-8 text; reject archives and executables/scripts
- cap upload size (512MB) via a bounded read
- run every accepted upload through a dependency-injected malware
scanner, failing closed on scan error; the default scanner flags the
EICAR test file so the hook is validated end to end
- give accepted uploads a server-generated filename so the client name
never reaches storage
- set Content-Disposition attachment and X-Content-Type-Options nosniff
on vector-store file downloads
The router hop _ageneric_api_call_with_fallbacks canonicalises the passthrough
call type onto litellm_metadata, and the cost callback reads spend attribution
from that bucket while only backfilling user_api_key* keys from metadata. The
helper was building on metadata, so agent_id and user_api_end_user_max_budget
were silently dropped before the callback ever saw them. Build and pass the
attribution under litellm_metadata so every field survives.
user_api_key_auth also authenticates a caller from the operator-configured
general_settings.litellm_key_header_name, reading that header straight off
the request, so a virtual key sent there survived the credential-less Vertex
forwarding filter and reached Google alongside a real bring-your-own
credential. Value-strip every header whose value matches the caller's key
from any accepted source, including that custom header.
On the credential-less Vertex passthrough branch, drop every header that
can only carry LiteLLM caller auth (x-litellm-api-key, api-key, x-api-key)
by name, since Google never consumes them, and strip the virtual key by
value from Authorization / x-goog-api-key, which may instead hold a genuine
bring-your-own Google credential. This closes the residual leak where a
distinct caller secret in api-key or x-api-key still reached upstream.
Adds a regression asserting the value-based strip also drops the caller's
virtual key when it is duplicated into the api-key and x-api-key headers,
while a genuine bring-your-own Google credential still forwards.
Add an endpoint-level regression test asserting can_user_make_model_call
receives the litellm_params after health_check_params are merged in, so the
merge-before-auth ordering cannot silently regress and let a request smuggle
a field past authorization.
The credential-less Vertex passthrough dropped the caller's LiteLLM
virtual key only from Authorization by exact match. A caller who sent
the same key in x-goog-api-key (which doubles as a real Google
credential) had it accepted as a credential and forwarded upstream.
Drop the virtual key by value across every forwarded header, normalizing
any Bearer prefix, so no header name carries it to Google.
Pulls in the detect-changes CI action and the test-unit job timeout bump, which clears the red lint and code-quality checks on this PR
The merged, tightened lint budgets flag this PR's own code, so this merge also makes video_reference_to_id a pure function instead of a helper that mutates its input dict, and rewrites the form-body regression test to call the video_edit and video_extension handlers directly rather than patching an internal class method. Adds pure-logic unit tests for video_reference_to_id
Move _update_litellm_params_for_health_check before can_user_make_model_call
so health_check_params cannot retarget the probe after the auth check. Type
the Pegasus test helper signature and drop the redundant test narrative.
When no Vertex credential is configured (no default_vertex_config, no matching
use_in_pass_through deployment, no vector-store credential), the Vertex passthrough
took the bring-your-own-credentials branch and forwarded the entire incoming header
set upstream to Google. That set included whichever header carried the caller's
LiteLLM virtual key: x-litellm-api-key, or Authorization when get_litellm_virtual_key
read the key from there. The proxy's own secret was sent to a third-party provider.
The credential-less branch now drops x-litellm-api-key and the Authorization value
that equals the virtual key, keeping a genuine bring-your-own Google credential
(an OAuth token in Authorization, or x-goog-api-key) so real BYO passthrough still
works. When neither survives, the request fails with a clean 401 telling the operator
no credential is configured, instead of forwarding the virtual key.
Regression coverage in the mapped test path asserts the 401-and-never-forwarded
behavior for both leak vectors and that a real Google credential still passes through
with the virtual key stripped.
The /vllm and /azure router-model passthrough branches called
llm_router.allm_passthrough_route directly with no request metadata,
so the cost callback saw no user_api_key and no
user_api_key_budget_reservation. Spend for a budgeted virtual key hit
neither the key's spend nor the spend logs, and the reservation minted
at auth into the shared Redis counter was never released, drifting the
counter up until the key falsely tripped BudgetExceededError.
Thread the authenticated key's attribution metadata into both calls via
the same builder add_litellm_data_to_request uses, so the cost callback
attributes spend and reconciles the reservation. Regression tests cover
both branches.
POST /v1/videos without an input_reference file now goes out as
multipart/form-data the way the OpenAI SDK always sends it, instead of a
JSON body that OpenAI-compatible backends (SGLang Diffusion, vLLM-Omni)
reject; gemini, vertex, and runwayml keep their JSON bodies
/v1/images/edits on the openai/azure/openai-compatible path now forwards
unknown provider params (e.g. seed) and honors extra_body, matching
/v1/images/generations, and aimage_edit forwards
extra_headers/extra_query/extra_body instead of dropping them
Generic pass-through no longer downgrades a file-less multipart form to
application/x-www-form-urlencoded
