* fix(mcp): strip root_path before matching the per-server MCP route spelling
The 401 challenge for a gateway-managed oauth2 MCP server advertises the
protected-resource metadata URL in the spelling the client connected on, so a
strict RFC 9728 section 3 client lands on a document whose `resource` equals the
URL it actually called. That spelling test compared `_original_path` against the
root-relative `/{server}/mcp` shape, but `_original_path` and `scope["path"]`
are raw request-line paths that still carry the deployment's `root_path`
On a SERVER_ROOT_PATH deployment the prefix therefore made the legacy test fail
and every request fell through to the standard `/mcp/{server}` branch. A client
connecting on `/litellm/github/mcp` was pointed at the standard-pattern
document, which serves `resource = {base}/litellm/mcp/github`; that is not the
URL the client called, so a strict client aborts discovery before the MCP
request fires
Route the path through `get_route_relative_request_path` first, which removes
`root_path` on a segment boundary the same way
`litellm.proxy.auth.auth_utils.get_request_route` already does for the rest of
the MCP auth path, so `/litellmfoo` is not truncated under `root_path=/litellm`
* fix(mcp): make the gateway-managed 401 challenge root-path aware
The gateway-managed authorization_code challenge in process_mcp_request
built its AS-metadata URL from two root-path-unaware pieces:
- it matched the caller's spelling against `scope["_original_path"]`, a
raw request-line path that still carries the deployment prefix, so on a
SERVER_ROOT_PATH deployment the `/mcp/{server}` branch never matched and
every request fell through to the legacy one-segment form
- it hardcoded `/.well-known/oauth-authorization-server` without the
root-path segment the discovery route decorators bake in, so the URL
404'd under a sub-path deployment regardless of which branch was taken
Route the spelling match through get_route_relative_request_path and the
well-known root through well_known_root_suffix, the same two helpers the
discovery route registrations derive their paths from, so the advertised
URL cannot drift from the route that serves it.
Root-mounted deployments are unaffected: both helpers are no-ops when
SERVER_ROOT_PATH is unset.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
A ConnectionError/TimeoutError on one node of the async Redis Cluster client
made redis-py tear down every node's connections and force every other
concurrent caller through the shared reinit lock, turning one client-side
timeout under event-loop saturation into a proxy-wide latency spike while
Redis itself stayed healthy. Confirmed live against a local 3-master
cluster: pausing one node made 100% of concurrent commands to the other
two, untouched nodes stall for the full pause duration; after this change,
zero.
LiteLLMAsyncRedisCluster overrides only the ConnectionError/TimeoutError
branch of _execute_command to reset the one node that failed, mirroring
what a plain non-cluster Redis client already does when a pooled
connection errors. Every other branch (MOVED, ASK, CLUSTERDOWN,
slot-not-covered) is unchanged, since those already carry real evidence
the topology changed.
* feat(ui): add per-key Savings tab to key detail page
Adds a "Savings" tab to the key detail view, showing the same four metrics
and time-series chart as the proxy-wide Cost Optimization view, but scoped
to a single API key.
For org admins, the tab shows the key's full savings across all requests.
Non-admins see only their own requests on the key, with a scope note
explaining the limitation.
Root cause: userDailyActivityCall and userDailyActivityAggregatedCall
never forwarded an api_key query parameter to the backend, even though
both handlers already accept and filter by it.
Changes:
- networking.tsx: Add optional apiKey param to both daily activity call
wrappers (appended to variadic options tuple for backward compatibility).
- costOptimizationUtils.ts: Extract shared metrics helpers (compressionOf,
cachingOf, autorouterOf, savedTokensOf, cacheHitRatio) and shortDate
so both UsageTab and KeySavingsTab use the same formulas and prevent
divergence.
- useDailyActivityRange.ts: Refactor into useScopedDailyActivityRange(
accessToken, scope: {userId, apiKey?}) for reuse-by-parameter unbundling.
Role resolution stays at the entry point (useDailyActivityRange), not in
a scoped caller. Update test expectations for new 6-arg tuple.
- UsageTab.tsx: Simplify by importing extracted helpers and SummaryCard
component instead of defining them inline. No behavioral change.
- key_info_view.tsx: Insert "Savings" tab trigger between "Overview" and
"Settings"; wire TabsContent to new KeySavingsTab component with lazy
mounting (no keepMounted) to defer daily-activity fetch until tab opened.
- NEW: components/shared/SummaryCard.tsx — Shared presenter for four-tile
summary row (label + value + hint + optional info popover). Extracted
from UsageTab so both surfaces show identical tile layout without CSS
divergence.
- NEW: components/templates/KeySavingsTab.tsx — Per-key view with admin/
non-admin scope branching, empty-state messaging, same chart toggles
and info popovers as UsageTab.
- NEW: components/templates/KeySavingsTab.test.tsx — 7 tests covering mount,
loading state, empty state, scoping, and scope-note visibility.
Authorization: No new permission check. Both backends gate api_key filter
by the same user role check that governs the request itself. Non-admins
must send their own user_id and can only see their own keys.
Tests: 6121 pass (1 pre-existing failure unrelated to this change).
Prior art / collision note:
- PR #37570 (budgets tab) lands in same TabsList hunks as "Savings" tab,
but different tab names so conflict trivial if both merge.
- PR #37659 (my own) adds progress/cancelled/cancel to DailyActivityRange,
but this PR uses stable three-field interface from staging.
* fix(ui): scope spend view by the backend's admin-view contract, not all_admin_roles
Greptile flagged org admin handling on the key savings tab. The live bug it
described does not fire today: useAuthorized supplies session-role labels and
all_admin_roles only carries the raw org_admin spelling, so an org admin was
already scoped. That safety was accidental, so replace the predicate with
spendScopeUserId / hasProxyWideSpendView in utils/roles.ts, mirroring the
backend's user_api_key_has_admin_view (proxy admin and admin viewer only, org
admin excluded in both spellings), and use it in both useDailyActivityRange
and KeySavingsTab
Reclassify the KeySavingsTab render test as an integration test per the
repo's unit/integration split, move scope-resolution coverage to roles.test.ts
as a full role matrix, use real session-role values instead of raw ones, and
assert tile totals against non-empty metrics. Replace the nested ternary in
the chart body (frontend-lint error) with flat conditional rendering
* fix(ui): show auto-router savings as the fourth key-savings tile
Cache hit rate had displaced auto-router savings from the fourth slot,
diverging from the org-wide Cost Optimization page's tile order. Match
it: Total / Compression / Prompt caching / Auto-router, with cache hit
rate as a fifth tile.
* fix(ui): drop cache hit rate from the key savings tiles
Keep the four tiles this page is meant to show: total, compression,
prompt caching, and auto-router savings.
* fix(ui): stop an empty api_key from widening a key-scoped activity read
The paginated and aggregated daily-activity wrappers disagreed on an
empty filter value: the paginated one appended it, the aggregated one
coerced it to undefined with || and dropped it. Since the aggregated
call is the one tried first, an empty key hash would have silently
turned a key-scoped read into a proxy-wide one and reported every
key's savings as this key's. Use ?? so both send the filter through
and it matches nothing instead.
* style(ui): satisfy prettier and the inline-object lint rule in key savings tests
* refactor(ui): drop the cacheHitRatio extraction left over from the removed tile
* fix(ui): pass daily-activity filters raw so both transports agree at the null boundary
* refactor(ui): share the savings tiles and totals between both surfaces
The per-key Savings tab and the proxy-wide Cost Optimization tab carried a byte-identical
four-tile block, three long metric-definition strings included, and five identical useMemo
totals. Both now render SavingsTiles and total through useSavingsTotals, so the donut cannot
slice numbers the tile above it disagrees with.
* docs(ui): say request, not mount, in the savings tab comment
The comment claimed mounting eagerly would fire the rollup sweep, which reads as a claim about
the bundle. Only the request is deferred; the module ships with the key page either way.
* test(ui): pin the daily-activity args array against the real caller signatures
The sibling unit test mocks networking, so it checks the positional array against itself and
stays green when the array and a networking signature drift apart. Swapping user_id and api_key
in the aggregated signature alone passes there and fails here on user_id=hash-abc.
* style(ui): hoist the daily-activity query options out of the call argument
The four-property object literal tripped local/no-large-inline-object-arg. The violation predates
this branch, which only moved the line into the annotated range, and the rule count drops 550 to 549.
* fix(ui): drive auto-router usage from the shared cost-optimization time picker
* fix(ui): extend a live-ending benchmarks range to the current UTC day
* fix(router): don't log 'Could not identify azure model' when the deployment name resolves from the cost map
get_router_model_info already falls back to resolving the azure
deployment's model name against the model cost map when base_model is
unset — and for deployments named after real azure models (e.g.
azure/gpt-4o) that resolution returns correct max tokens and costs. The
unconditional ERROR was therefore spurious for exactly the deployments
that need no operator action, and on busy proxies it logs thousands of
times per day per multi-deployment group.
Log at debug when the fallback entry carries usable limits/costs
(membership alone is not enough: Router init auto-registers every
deployment name as a zeroed stub), keep the ERROR otherwise.
Fixes#33172
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(router): use consistent positive checks in azure base_model fallback gate
Review follow-up: token-limit fields used 'is not None' while the cost
field used '> 0' — a cost-map entry explicitly storing 0 limits could
suppress the error log without carrying usable resolution data. All
three checks now require a positive value.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(router): trim fallback gate comment and reuse the shared local_model_cost_map fixture
---------
Co-authored-by: Mihidum Hettiyahandi <55163074+mihidumh@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A test that asserts on the error inside its own except block passes when the
call stops raising, because nothing runs the handler. That is the exact case
the test exists to catch, so the regression lands green.
Rewrites all 111 such blocks into pytest.raises, which fails when the call
succeeds, and selects PT017 in ruff-tests.toml so no new one lands.
* fix(vertex_ai): convert messages to contents in gemini count_tokens
acount_tokens passed contents=None to the Vertex Gemini countTokens
endpoint when called with messages=, causing a silent zero token count.
The Gemini branch of VertexAITokenCounter.count_tokens never read the
messages parameter, so the request body was {"contents": null}, which
Vertex accepts with HTTP 200 and no totalTokens field.
Convert messages to Gemini contents format using the existing
_gemini_convert_messages_with_history helper when contents is None.
Treat a response without totalTokens as a failure so the caller falls
back to local token counting instead of returning a silent zero.
Fixes#36921
* style: apply ruff format to common_utils.py
Resolves lint CI failure on PR #36981.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: suppress LIT002 on messages fallback for Gemini token counter
Adds `# mutable-ok:` suppression on the `messages or []` fallback passed
to `_gemini_convert_messages_with_history`. The [] is a None-fallback;
the helper signature requires list[AllMessageValues], so a tuple would
violate the type contract. Resolves type-discipline-budget CI failure
on PR #36981.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: suppress reportPrivateUsage on _gemini_convert_messages_with_history import
Adds `# pyright: ignore[reportPrivateUsage]` on the import of the
shared `_gemini_convert_messages_with_history` helper. The function is
already used by gemini/chat, context_caching, and
vertex_and_google_ai_studio_gemini; reusing it here avoids duplicating
the OpenAI-to-Gemini message conversion. Resolves basedpyright budget
CI failure on PR #36981.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): resolve guardrail pipelines from the canonical metadata bucket
Policy-resolved pipelines are stored in litellm_metadata on routes like /v1/messages, but the pre_call reader fell back to the caller-supplied metadata field first, so a request that sends its own top-level metadata (Claude Code sends metadata.user_id) skipped every pipeline-managed guardrail.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): drive the pipeline regression through a registered guardrail
Exercise the real executor with a guardrail in litellm.callbacks instead of patching PipelineExecutor.execute_steps at class scope.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): read pipeline state from the bucket the policy engine wrote
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): type the policy pipeline state accessors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): annotate policy pipeline state casts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Mantle reports context overflow as a structured 400 validation_error rather than
the plain-text patterns Bedrock itself uses, so callers such as Claude Code that
key reactive compaction off the phrase "prompt is too long" never see it. Detect
the pattern and normalize the message to that phrase.
The dcr_bridge oauth_delegate connect flow completed for a signed-in user
with no litellm-side grant to the target server: every leg returned 200,
the DCR client showed connected, and tools/list then fail-closed to an
empty list with the upstream never contacted (#36358). The authorize leg
now admits the user the way MCP egress will (same reload_admitted_user
constructor, same get_allowed_mcp_servers resolver) and refuses with an
RFC 6749 access_denied redirect naming the remedy, before any upstream
OAuth runs or an envelope is minted. Availability faults (5xx) propagate;
unknown or deactivated users deny fail-closed
Promotes MCPRequestHandler reload_admitted_user to public: it already had
a cross-module consumer in ui_session_utils, and this gate adds a second,
so the private name no longer reflected its use. Ratchets the freed
reportPrivateUsage budget headroom down
A name bound twice keeps only the second binding. In `tests/` that is nearly
always a repeated import, harmless but misleading, and the same rule is what
catches the cases that are not harmless: a local that shadows an import the
module still calls, and a second `def test_x` that quietly replaces the first.
311 of the 344 sites were repeated imports and came out with ruff's own fix.
The remaining 33 needed a decision. Four modules imported a name they never
used because a local definition below already shadowed it. Two comprehensions
bound `call` over `unittest.mock.call`, which those modules import and use.
One test rebound the two module handles its nested reload closure had captured.
One class attribute shadowed an unused `status` import.
The load-test fixtures move to a conftest, which is how pytest is meant to share
them, so the test module no longer imports three fixture names it never calls.
The nine `prisma_client` parameters keep a narrow `noqa`: pytest resolves that
fixture by name before the body runs, so the parameter never shadows anything.
Adds ownership of litellm-proxy-extras/litellm_proxy_extras/migrations so
schema migration changes get a review request.
Also repoints the two existing entries at @yuneng-berri. GitHub's CODEOWNERS
validator was rejecting @yuneng-jiang as an unknown owner, which left the
/ui/ and _experimental/out/ rules inert.
The sweep ran unbounded whenever no config.yaml deployment was present,
deleting the day's sentinel rows for deployments absent from the run's own
view. A written charge records capacity that was reserved, so the only rows
a run may retract are the ones it can reassess: a deployment it scanned and
then declined to charge, because the window closed or the PTU config was
removed. It is now always bounded to the ids it scanned
A config deployment is otherwise keyed by a hash of its resolved
litellm_params, so rotating a credential or editing an endpoint mints a
second identity and the catch-up bills the reservation again under it. Flat
cost is keyed by that id and a written charge is never retracted, so the
duplicate is permanent.
The id is read before set_model_list mints one, or the rule would inspect
the value it is meant to reject. Duplicates are counted once per config
entry across the whole file, so the check is order-independent and an
organization fan-out cannot collide with itself. The refusal names the id
the deployment already uses, since inventing a fresh one starts exactly the
second identity this prevents
* fix(proxy): scan batch records with the content hooks that are not guardrails
Guardrails were made to run on batch uploads by scanning each record through the pre-call hook
with the walk limited to guardrails. That limit exists because the same branch carries the rate
limiters and budget accounting, which must count an upload once rather than once per line. It
also excluded every enforcement hook written as a plain CustomLogger, so prompt-injection
detection, Azure content safety, banned keywords and the blocked-user check never saw a batch
record at all. Content that is a hard 400 online reached the provider verbatim through batch.
A CustomLogger now declares whether its pre-call hook judges the payload or merely counts the
request. The four that judge it opt in, the walk admits them, and both short-circuits learn
about them, including the one that decides whether the file is streamed off disk in the first
place: a proxy configured only with one of these hooks was skipping the scan entirely. Nothing
that counts a request is marked, so an upload still costs one slot and one budget check.
* refactor(proxy): drop the per-hook comment the attribute contract already states
* test(proxy): make the classification a ledger, and pin the wiring with a real hook
The classification test listed the two non-enterprise hooks by hand, so unmarking either
enterprise one changed nothing and the mutation matrix passed with both surviving. It now walks
the hook registries and fails on any pre-call CustomLogger that is on neither side, which also
gives the flag the forcing function it lacked: an enforcement hook added later would otherwise
default to off and silently skip batch records, which is the bug being fixed here.
Nothing exercised the path the bug actually lived on either, since every test raised its own
exception rather than a real hook's. One test now drives the shipped prompt-injection hook
through the scan, which pins the part no synthetic exception reaches: a chained exception reads
as a failure to judge, so refactoring any of these hooks to `raise ... from` would turn every
per-record drop into an aborted upload.
Also records why a hook that rewrites the payload for routing stays unmarked, and that only the
leaf class is consulted.
* test(proxy): set the callback list through monkeypatch rather than writing the global
* fix(proxy): read batch records the same way the upload validation does
The upload validation parses each JSONL line as bytes, where the json module sniffs the
encoding itself and accepts a leading byte order mark or a lone surrogate. The guardrail scan
that runs immediately after decoded each line to text first, which is stricter, so a file the
validation had just accepted could fail the scan. A `.jsonl` written by any of the editors that
emit a BOM, which includes PowerShell's Out-File and classic Notepad, uploaded fine until a
pre_call guardrail was configured and then returned 500 with a decode error and no indication
of which line or why. The scan now parses the same bytes the validation did, and an untouched
record is copied through as the bytes it arrived as rather than re-encoded.
A numeric custom_id was reported as null. The spec asks for a string, but callers do send
numbers, and null leaves the one field a caller reconciles on empty for exactly the records
that need it.
* fix(proxy): read the load-balancing record the same way, so a byte order mark keeps its routing
The first record is parsed to pick a deployment when batch load balancing is on, and it was
decoded to text before parsing, which rejects a leading byte order mark. The lookup returns None
on any parse failure, so such a file silently lost its routing and went to the default provider
rather than the configured one. That was already reachable for an upload no guardrail changed,
since the original bytes are passed straight through, and preserving the mark through a rewrite
widens it. Parsed as bytes now, like the validation and the scan.
* fix(proxy): find the routing record past a blank first line
The upload validation and the guardrail scan both skip blank lines, but deployment selection
read only the first physical line, so a file starting with a blank line lost its routing model
and went to the default provider rather than the configured one. It now skips blanks the way
the other two readers do, reading lazily so a large file is not read past its first record.
* fix(proxy): do not crash deployment selection on a record whose body is not an object
The upload validation checks that a record has a `body`, not that it is an object, so a record
can carry a string or a list there. Deployment selection called `.get` on it unconditionally and
raised, returning 500. That was already reachable for a plain file, and reading past a byte order
mark or a blank first line widened it to files that previously fell through to the default
provider instead. A record whose body names no readable model now resolves to no model, which is
the same answer the default-provider branch already handled.
* fix(proxy): keep a custom_id that cannot be encoded from failing the whole upload
A record identifier is echoed back in the create response. JSON parses a lone surrogate happily
but it cannot be encoded again, so a file the upload validation accepts returned 500 from the
response renderer rather than a report. Unencodable characters are replaced, which leaves every
ordinary identifier untouched and keeps a pathological one reconcilable.
This predates the reader change; reading past a byte order mark only altered which error the
same file produced first.
* fix(proxy): treat a url the parser rejects as one we do not recognize
Resolving a record's call type from its url runs the url through urlsplit, which raises on a few
malformed authorities such as an unclosed bracket. That happens before the try that wraps the
guardrail call, so it escaped the scan and returned 500 on a file the upload validation had just
accepted. An unreadable url is simply one we cannot recognize, which the body-shape fallback
already handles, so the record is still scanned rather than lost.
Reachable on staging today for a proxy running any guardrail. Enabling the scan for a proxy that
runs only a content-enforcing CustomLogger widens it to that configuration too, which is why it
is fixed here rather than left.
The auto-routed model group was only reachable through the
x-litellm-model-id response header. SDK and framework callers that do not
expose response headers had no way to read it, and under streaming there
was no body surface at all.
The response body `model` field is deliberately restamped back to the
client-requested alias on both paths, which is correct OpenAI semantics,
so this adds a separate namespaced `router_model_name` key instead of
redefining `model`. The key is written on non-streaming bodies and on
every SSE chunk, including the streaming fast path, and is emitted only
when an auto-routing strategy actually selected the deployment.
After a mid-stream fallback moves the request off the group the router
picked, the key is omitted rather than continuing to claim the original
tier. The router marker already supports per-chunk fallback signals via
`x-litellm-attempted-fallbacks` headers; this wires that signal into
the gate so no stale tier is claimed after a fallback fires.
Also removes a redundant function-local import in the streaming
generator that shadowed the module-level one for the whole function.
The Prisma query engine is a separate process whose resident memory grows with
what it is asked to hold and glibc never returns it, so a pod's memory floor
ratchets up to its worst statement and stays there for the life of the worker.
#34956 bounded a spend-log flush by payload bytes, which caps that floor when
prompts are stored and does nothing when they are not: rows carrying only
attribution metadata run about 1.2 KB, so a 1000-row statement is roughly
1.2 MB, the 2 MB byte budget never binds, and every statement stays at 1000
rows forever.
The engine charges per row as well as per byte. Measured on a container running
the same engine build (5.4.2) against real Postgres, with rows shaped like a
store_prompts_in_spend_logs=false deployment, writing the same 200,000 rows:
rows/statement engine RSS still resident after the flush
1000 179 MB
500 91 MB
250 41 MB
100 19 MB
None of those statements came near the byte budget, so the whole difference is
row count. The floor is a plateau rather than a leak: 1,000,000 rows written at
1000 per statement settles around 229 MB and stops climbing.
Adds SPEND_LOG_WRITE_BATCH_MAX_ROWS, default 100, applied alongside the
existing byte budget so whichever binds first splits the statement. Both are
needed, since bytes are what track a prompt-carrying row and rows are what
track the engine's per-row bookkeeping.
One consequence worth naming: a flush now issues more statements, and a
statement that fails under a poison flood costs one insert before any
isolation runs, so the irreducible floor rises by the statement count. The
isolation budget still caps the amplification on top of that, and the tests
assert the bound derived from the configured row cap rather than a constant.
Per-model budgets were three separate things pretending to be one. The
enforcement check, the post-call increment and the info endpoints each derived
their own cache key, so a budget could refuse traffic at 429 while /key/info
reported zero usage, and a Bedrock model id never matched a budget keyed on the
bare family name. /user/new echoed a model_max_budget back and stored an empty
dict, and nothing enforced a user-scoped per-model budget at all.
One owner now builds the counter key from the configured budget model, and
enforcement, the increment and the info endpoints all read it. Bedrock ids
resolve through the model-cost map. Auth carries the user's budget onto the
token on every branch that reaches the spend hook, including JWT and
auto-registration. Native passthrough attaches the three budget metadata keys
its StandardLoggingUserAPIKeyMetadata does not carry, so /anthropic/... and
/bedrock/... traffic is counted and capped like /v1/chat/completions.
The dashboard gains the per-model budget editor it never had, on the key create,
key edit and internal-user edit forms. It is read-only without an enterprise
license, matching the write gate the proxy already enforces, and an untouched
budget is left out of an update so an unrelated edit cannot trip that gate.
The editor hydrates from either BudgetConfig spelling, since model_max_budget is
a plain dict that the proxy stores exactly as the client sent it, and it carries
through the fields it does not model. Without both, editing one model would drop
another model row entirely and silently discard its tpm_limit and rpm_limit.
/user/info refreshes its local copy of the user field by field after a save, so
model_max_budget joins that list. Left out, a saved cap read back as the old one
when the form was reopened, and clearing the row to recover would then wipe the
value that had actually persisted.
A zero-dollar cap is the strictest limit expressible, not the absence of one,
so it is enforced rather than skipped on falsiness, spend exactly at the cap is
refused the way every sibling budget check already refuses it, and a counter
that was never written reads as zero spend rather than as unknown. The usage
endpoints read every counter in one batched lookup, so a large model_max_budget
cannot fan out into one concurrent cache call per configured model.
Every auth path honours the same zero-cost skip flag, so none of them can refuse
a free request that another serves. The custom-auth helper gains the flag it
never had, which also changes its pre-existing key and end-user checks.
The compaction summary gate checks the user scope alongside the key and end-user
ones. This file propagates all three budgets into the summary subrequest, so
enforcing only two let compaction increment a counter it could not be refused by.
Custom auth attaches the user's budget to the token unconditionally, since the
post-call spend hook reads it there: gating the attach on the same condition as
enforcement left the counter uncharged whenever the request was not itself
enforceable. An entry that will not validate is skipped rather than raised on,
so one malformed scope cannot abort every other scope's increment or turn a
config typo into a 500.
The edit forms re-seed the budget editor when a different key or user is loaded.
Its rows are seeded once and cannot re-read their own value prop, so without
this a save wrote the previously loaded record's budgets onto the current one.
Only the built-in provider pass-through routes carry the budget metadata.
get_model_from_request deliberately resolves no model for a user-defined
pass-through, since its body is forwarded verbatim and names an upstream model,
so attaching there would charge a counter nothing on that route can refuse.
Master-key auth stamps the stable alias litellm_proxy_master_key instead of the
raw key, so spend logs carry a readable, non-secret identifier for those rows.
The new redaction path only recognized sha256 and hashed-jwt shapes, so it
hashed that alias and broke continuity with every master-key row written
before this change. The alias joins the recognized non-secret values, still
behind the same provenance gate, so a caller who sends the alias string as
their own bearer token still gets it hashed.
The proxy conftest already snapshots master_key and prisma_client around every
test, because a value left behind on litellm.proxy.proxy_server poisons the rest
of the xdist worker. llm_router has the same problem. The PTU rollup reads the
running router out of sys.modules, so a router a sibling test left behind lands
in its deployment scan and three test_ptu_flat_cost_rollup tests fail or pass
depending on how xdist happens to split the shard.
The hashed-jwt branch trusted the value's shape alone, so a caller-supplied key in that shape was stored unhashed. Both pass-throughs now require the value to match the auth-time user_api_key_hash, and the shape check is a full match.
The spend-log helper no longer treats a 64-hex shape as proof a value was already hashed, so this case has to say where the hash came from. Reconciles the test that came in with #31799 against that change.
`pytest.raises(Exception)` with no `match=` passes on any error that broad. A
TypeError from a refactor, a botched fixture, an import that moved: all of them
read as the rejection the test claims to police, so the test goes green for the
wrong reason and stays green after the behaviour it guards is gone.
PT011 closes that gap for the 317 sites B017 could not reach, because B017 only
fires on a single-statement body with no `as e` binding. Each pattern here is the
message the code actually raised, recorded by running the sites under a plugin
that logged the concrete type and text per call site, so the assertions describe
observed behaviour rather than a guess. Where a site raises more than one message
across its parametrize cases, the pattern is an alternation of what was seen;
where the exception carries an empty `str()` and puts the text on `.message`, the
site keeps a narrow `noqa` with the reason.
PT014 removes four parametrize cases that were listed twice. The duplicate re-runs
an assertion that already passed, and it usually marks a case someone meant to
vary and forgot to edit.
sagemaker_chat never put X-Amzn-SageMaker-Inference-Component on the request, so any endpoint
backed by inference components answered 400 INFERENCE_COMPONENT_NAME_MISSING and the call never
reached the container. The legacy sagemaker provider has built that header from model_id since
#8889, and this brings the chat provider in line. It goes on in validate_environment, which runs
before the request is SigV4-signed, so the signature covers it
The request body also always named the endpoint rather than the served model, which containers
that validate the body's model answer with a 404. hf_model_name now becomes the body's model
when it is set, and endpoints that do not set it keep sending exactly what they send today
Add sync and async regression tests for the BaseLLMHTTPHandler streaming
path, which forwards provider response headers for the ~30 providers that
ride the generic handler and had no coverage. Also drop redundant setup
prose from the moonshot invoke test docstring.
`members_with_roles` is a denormalized JSON snapshot written at add-time.
`_update_team_members_list` backfilled `user_id` from `user_email` but never
the reverse, so a member added by `user_id` alone was stored with
`user_email=None` permanently - and `/team/info` returns that blob verbatim
with no join to `LiteLLM_UserTable`, so the Admin UI's member table renders
"-" for a user that plainly has an email.
Fix both ends:
- write path: `_resolve_member_identity` resolves identity both ways off the
user rows the add just touched, so new roster entries stop being born blank.
- read path: `/team/info` fills blank emails from `LiteLLM_UserTable` in one
indexed `user_id IN (...)` query, repairing rows already in the database.
Members that already carry an email are passed through untouched and cost
no query, so this only ever turns a null into the right value.
