Both failures are from this branch, not pre-existing
The component allowlist test asserts the gateway and backend route sets union to
the whole app, so any route on neither is a 404 on both pods. Allowlist the
`/management/v1/` prefix on the backend, next to the other control plane
entries, so every resource that moves under it later is covered without a
per-resource edit
The otel handler test builds its request as a SimpleNamespace carrying only
`state`. The validation handler now reads `request.url.path` to decide whether
the caller is on a surface with its own error contract, so the fake needs a url;
a real Request always has one, which is why the handler does not guard for it
The control plane branch returns early, and nothing covered that it still closes
the dangling SERVER span first, so those requests would have leaked a span
apiece. Added a case that pins it; removing the close call fails it
uv selects an interpreter before resolving dependencies, so with no
--python request the stock macOS /usr/bin/python3 (3.9.6) satisfies the
unconstrained request and resolution then fails against litellm's
requires-python (>=3.10,<3.15) instead of downloading a managed Python.
Request the requires-python range explicitly in install-cli.sh and
install.sh so uv reuses a compatible system interpreter when present and
downloads a managed one otherwise. The manual-fallback hint in the die
message carries the same flag so it no longer reproduces the failure.
Replace Any-typed seams with real types in the files carrying the highest
reportAny/reportExplicitAny density: typed Prisma read helpers in the MCP
db layer and verification token repository, TypedDicts for OAuth credential
payloads and aggregated spend rows, a DailySpendRecord protocol for the
daily activity endpoints, and concrete request/response types in the
volcengine, openai evals, azure batches, azure_ai count_tokens, and ocr
transformation modules. Modernize touched annotations to PEP 604/585 forms.
No casts, no type: ignore, no noqa, no new Any annotations, no behavior
changes. Whole-tree basedpyright: reportAny 27,005 -> 24,427,
reportExplicitAny 7,439 -> 7,280, no rule increased anywhere. Budgets
ratcheted: basedpyright -2,869, ruff-strict -1,505, type-discipline -167.
Three fixes from an adversarial review of this branch, each at the owning
seam rather than the report site.
The flush retried DB_CONNECTION_ERROR_TYPES, which includes ReadTimeout.
A ReadTimeout is the committed-but-unacked case: the review reproduced the
engine abandoning the transaction open on the pooled connection, the retry
stacking its statements into it, and one commit applying both increment
sets while the flush reports success. The retry now covers only
ConnectError, the one failure that proves the statements never reached the
database; post-send failures drop the batch with an error log. The
docstring no longer claims an idempotency the pattern does not have. The
same hazard exists in the untouched daily spend writer and is left for its
own change.
get_tool_calls_from_response read choices[0] only, so a tool invoked in a
later choice of an n>1 response earned spend but never reached the rollup,
the index, or the registry. Choice scope is now an explicit parameter:
accounting passes include_all_choices=True because every choice costs
money; guardrails keep the primary-choice default because they rebuild the
primary assistant message. First multi-choice fixtures in the suite pin
both scopes.
maxBarSize=64 had been added to the shared BarChart unconditionally,
resizing every existing consumer. It is now a prop; only the tool spend
charts opt in. The legend flex-wrap changes stay global because clipping
overflow was a defect, not a preference.
The RFC 9457 `type` was `https://docs.litellm.ai/errors/<slug>`, copied from the
standard's own error example. That path is a 404 and there is no docs section
behind it, so every error body shipped a broken link
RFC 9457 only requires `type` to identify the problem type; it encourages, but
does not require, that dereferencing it yield documentation. An https URI makes a
promise we are not keeping, so use `urn:litellm:error:<slug>` instead, which
carries the same machine-readable identity with nothing to resolve. Switching to
an https base later is a contract change for anyone matching on `type`, so that
should wait for pages that actually exist
A test pins the identifier against regressing to an https docs URL, since the
existing assertion built the expected value from the same constant and would have
stayed green whatever it held
`/customer/aliases` shipped two days ago and has not been in a release, so its
wire contract is still free to change. This lands it on the control-plane
contract before that stops being true, since after a release the path, the param
names and the envelope would all need a permanent legacy adapter
The endpoint becomes `GET /management/v1/spend_logs/end_users`. It is a facet,
the distinct values one column takes over a filtered query on a resource, not an
entity collection; naming it after `customers` implied it listed the end-user
table when it actually reads spend logs, which is a different row set. Serving it
under the parent resource means its filters are the parent's filters, so the
dropdown offers exactly the values the logs table can show without two endpoints
having to keep agreeing on that
Contract changes: `size` becomes `page_size`, `search` becomes `q`, the window
moves from flat `start_date` / `end_date` to `filter[startTime][gte]` / `[lte]`,
and the body becomes `{data, meta, links}`. Unknown query params are now a 400
rather than being silently dropped, because an ignored filter over-returns data.
Errors are RFC 9457 problem documents on this prefix only; every other route
keeps the shape its callers already parse
`links` is what makes the rest deferrable. The dashboard hook follows the
server's `links.next` instead of computing `page + 1`, so moving this to cursor
pagination later changes the links and nothing the client does. That matters
because the inner scan is a sliding window, so offset paging can currently skip
or repeat an end user across pages; the fix is a follow-up, and the hypermedia
means it will not be a breaking one
Cursor mode, `sort`, `include`, ETag / `If-None-Match` and the generic `ListSpec`
framework are all deliberately out of scope here. They are additive or internal,
so none of them needs to beat the release
The Team dropdown popup is pinned to the trigger width via
w-(--anchor-width) and clips its overflow, while Base UI's ItemText
wrapper is flex-1 shrink-0 with min-width: auto, so it sizes itself to
the full nowrap label and simply overflows the popup. Teams without a
team_alias render their 36-char id, so those options were sliced
mid-character with no ellipsis.
Clears min-width: auto off the text wrapper and truncates the label at
the call site. The underlying gap is in the shared Select primitive,
which any long-labelled select in the dashboard will hit; that is left
for a separate change.
GET /v1/tool/spend served the Cost Optimization card with two raw queries
over LiteLLM_SpendLogToolIndex x LiteLLM_SpendLogs on every dashboard load;
the totals query's driving scan was all of SpendLogs in the window. Both
per-request tables reach 1M+ rows at customer scale, so the card cost
O(traffic) per view and had to be capped at 30 days.
The index writer also mined proxy_server_request.tools, i.e. tools DECLARED
in the request body, attributing each request's full spend to tools that
never ran; and all non-MCP mining ran against payload fields that are '{}'
unless store_prompts_in_spend_logs is enabled, so non-MCP coverage silently
depended on a privacy setting.
Now the spend writer builds a ToolUsageTransaction at request time from
invoked tools only, resolved by the shared get_tool_calls_from_response
normalizer so every response surface (chat completions, Responses API,
Anthropic Messages) is covered; the tool registry's response arm delegates
to the same owner. Transactions queue beside the spend-log queue and the
flush job writes index rows plus a new LiteLLM_DailyToolSpend rollup
(date, tool_name PK) in one transaction, retrying connection errors with
backoff (a failed batch commits nothing, so the retry cannot double-count)
and dropping the batch with an error log on anything else.
The endpoint aggregates in SQL: by_tool is the top TOOL_SPEND_TOP_TOOLS
tools by spend via group_by and daily covers only those tools, so the
response is bounded by days x TOOL_SPEND_TOP_TOOLS regardless of range or
tool-name cardinality; the 30-day clamp is gone. total_spend is dropped
from the response; it was never rendered and its deduplicated semantics
are not computable from a rollup. Spend-log retention deliberately does
not touch the rollup, so tool spend history outlives per-request rows.
* fix(guardrails): resolve judge_model credentials via Router in llm_as_a_judge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): wire llm_router into DB-backed judge guardrail init paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): assert patch endpoint forwards llm_router to sync
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(guardrails): resolve judge Router lazily and fix wildcard/alias dispatch
Resolve the proxy Router at judge-call time via an injected provider instead of
capturing it at construction, so a DB-backed judge guardrail created before the
Router exists no longer captures None permanently. Select the Router path with
router.get_model_list(model_name=judge_model) so wildcard routes and
model_group_alias keys resolve, not just literal deployment names. Isolate the
judge call from user-traffic routing with num_retries=0 and fallbacks=[].
Revert the llm_router threading through the DB sync/reinit/create/approve/patch
paths since the lazy provider makes it unnecessary. Replace mocked-Router tests
with real Router coverage for plain deployments, model_group_alias, and wildcard
routes, plus lazy per-call resolution.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): harden judge verdict parsing and guard proxy import
Strip markdown fences and surrounding prose before json.loads so fencing-prone
judge models evaluate instead of failing open, guard the proxy_server import in
_default_router_provider so an unimportable proxy falls back to the SDK, and
snapshot/restore global callback lists in the DB-path judge registry tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): reject non-object judge verdicts instead of failing open as success
* fix(guardrails): route hidden model_group_alias judge models through the Router
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <yucheng@berri.ai>
The shadcn separator primitive ships `data-vertical:self-stretch` so a bare
vertical divider fills its row, but every call site overrides the height with
`h-5`. A definite cross size makes `align-self: stretch` behave as
`flex-start`, so the dividers rendered flush with the top of their flex line
instead of centered: 0px above and 18px below in the dashboard header, 0px
above and 12px below in the models table toolbar
Routes the three vertical dividers through a ToolbarSeparator that pairs the
fixed height with a same-variant `data-vertical:self-center`. Matching the
variant is what matters; tailwind-merge then drops the conflicting class
outright, whereas a plain `self-center` ties on specificity (the variant is
defined with `:where()`) and loses on utility order. The CLI-managed primitive
is left untouched
DialogContent's close button is absolutely positioned 16px from the right
edge at 32px wide, so it overlays the rightmost 24px of the p-6 content
box. The justify-between header pins "+ Custom Server" to that same edge
and, being out of flow, the close button reserves nothing. Give the action
a right margin that clears it; keeping the margin on the button rather
than the row leaves the header rule full-bleed
Vertex passthrough classified any target URL containing "stream" as a streaming
request. `:streamRawPredict` carries that substring, so a unary Claude-on-Vertex
call whose body omits `stream` was routed through the streaming logging path.
That path never consults the response content-type, so a complete
`"type": "message"` JSON body was handed to the Anthropic SSE chunk parser,
which recognises none of it; the spend log recorded 0 prompt tokens,
0 completion tokens and zero cost
Streaming for the rawPredict family now comes from the request body, which is
what the Anthropic Messages contract uses for those endpoints. The
generateContent family keeps its URL signal because the Gemini REST body has no
`stream` field, and `?alt=sse` is still appended for every request that is
classified as streaming, so Gemini framing and its usage parsing are unchanged
Both passthrough streaming predicates read `.get("stream")` off a body that is
only annotated as a dict; `_read_request_body` returns whatever the JSON parser
produced, so an array body raised AttributeError. The two predicates are now one
owner that answers False for any non-object body, which covers the vertex,
mistral, anthropic, vllm and azure passthrough routes
The MCP Servers page was the only page-level tab bar using the segmented
(pill) TabsList stretched with w-full, which rendered a full-width grey
bar with a lone pill on the left. Every other page-level tab bar
(budgets, vector stores, access groups, organizations, routing groups,
API reference) uses the underlined line variant, so use that here too.
The shadcn migration carried the antd modal's 1000px width over as an
unprefixed max-w-[1000px], which tailwind-merge keeps alongside the
DialogContent base class sm:max-w-md; the responsive variant wins from
640px up, so the dialog rendered at 448px. Prefix the override so the
merge drops the base clamp
Cache web identity STS credentials in the shared IAM cache (restores the
pre-v1.85.0 behavior removed by #27125) and cap the Google OIDC token cache
TTL at the token's own exp claim minus a 60s margin, never caching an
already-expired token
* fix(e2e): stop the cache-settings test from persisting a degraded Redis config
TestCacheSettings.test_update_persists_cache_backend_to_get read the live cache
settings and wrote them back, intending a no-op. Its capture modelled only
type/host/port, so on a TLS cluster the write-back silently dropped `ssl` and
`redis_startup_nodes`.
That is not recoverable on its own. `/cache/settings` persists what it receives
into LiteLLM_CacheConfig, that row outranks the YAML `cache_params`, and
init_cache_settings_in_db re-applies it on a timer, so a restart does not clear
it. The proxy ends up driving a TLS-only cluster endpoint as a plaintext
standalone node and every Redis call blocks to socket timeout.
On the affected deployment that took out rate limiting entirely (the v3 limiter
is a Lua script on Redis with no DB fallback), Redis-only budget levels (tag,
per-model, team-member, per-window), spend tracking, `ResetBudgetJob` (which
self-starved at 54 skipped runs per 15 min), and `ProxyConfig.add_deployment`,
whose last statement syncs guardrails and never ran. 60 of 72 failures in one
run traced back here.
The settings blob is now round-tripped verbatim via a RootModel over an
exhaustive value union, so a subset cannot be written. Two guards make a
regression fail loudly at this test instead of silently downstream:
- refuse to write when GET reports redis_type=cluster but omits
redis_startup_nodes, which is the exact precondition for persisting a
downgrade. GET resolves the stored row overlaid with REDIS_* env and never
reads YAML, so a cluster configured only in YAML cannot round-trip here
- compare /cache/ping before and after, so a write that breaks connectivity
fails this test rather than every suite that follows
The underlying product defect is filed as LIT-4816: GET cannot express the
effective config, and a partial POST is allowed to downgrade transport. This
change only stops the suite from triggering it; the Admin UI can still do so.
basedpyright clean (0 errors) under the e2e gate.
* fix(e2e): scope the bedrock guardrail per request and send OpenAI's current token param
Two failures that had nothing to do with the guardrail or route under test.
create_bedrock_guardrail registered with default_on=True, which applies the
guardrail to every request the proxy serves. The upstream ApplyGuardrail call was
answering 403, and that came back to unrelated traffic as
`403 Bedrock guardrail request failed`, failing three a2a tests and a passthrough
headers test alongside the bedrock one. The harness already supports the
per-request `guardrails` selector, so the guardrail is now registered opted out of
default_on and selected by the test that wants it. A broken upstream guardrail
fails its own test instead of whatever else is running.
Note this only contains the blast radius; the 403 itself still needs the
bedrock:ApplyGuardrail permission (or a valid guardrail identifier) on the
deployment, so test_bedrock_pre_call_blocks_harmful_prompt can still fail on its
own until that is sorted.
The OpenAI passthrough body sent `max_tokens`, which newer models reject with
"Unsupported parameter: 'max_tokens' is not supported with this model. Use
'max_completion_tokens' instead." Passthrough forwards the body untranslated, so
drop_params does not apply and the body has to satisfy OpenAI's contract
directly. vllm_chat keeps max_tokens, which vLLM accepts.
basedpyright clean (0 errors) under the e2e gate.
* fix(e2e): drop the pinned a2a api_key that broke every message/send
#34512 pinned `api_key="os.environ/ANTHROPIC_API_KEY"` on the a2a bridge agent.
The a2a bridge forwards the agent's litellm_params straight into
litellm.acompletion() without expanding "os.environ/" indirection, so that literal
string was sent upstream as x-api-key and every message/send failed with
`AnthropicException - {"type":"authentication_error","message":"invalid x-api-key"}`.
Omitting api_key restores the normal provider resolution: litellm reads
ANTHROPIC_API_KEY from the proxy's own environment for this provider, which is what
the agent-owner flow depends on and what the suite did before #34512.
Verified against a live proxy, same agent shape each time:
api_key omitted -> message/send 200
api_key "os.environ/ANTHROPIC_API_KEY" -> message/send 500 invalid x-api-key
api_key <literal key> -> message/send 200
and the key itself is valid (direct call to api.anthropic.com returns 200), so this
was indirection that never got expanded rather than a bad credential.
This accounts for four failures (test_semver_protocol_version_registers_and_serves,
test_message_send_runs_completion_bridge, test_pinned_v0_3_serves_flat_message_shape,
test_pinned_v1_0_serves_nested_message_shape). They were previously reported as
`403 Bedrock guardrail request failed`, because a default_on Bedrock guardrail
short-circuited the request before it ever reached the bridge and hid this.
The bridge silently ignoring "os.environ/" in agent params is a product defect in
its own right, filed separately; anyone configuring an agent credential that way
through the UI hits the same wall.
basedpyright clean (0 errors) under the e2e gate.
* test(e2e): make the load suite less aggressive against a shared proxy
750 users at spawn rate 50 saturated the request path hard enough to distort the
latency-sensitive suites sharing the same proxy, and it spends real provider money
at that rate. Drop to 200 users at spawn rate 20.
The RPS floor moves with the user count rather than staying put, so the assertion
keeps its meaning instead of becoming a formality: 355 RPS over 750 users is
~0.47 RPS/user, and 90 over 200 holds that same per-user expectation with a
similar pass margin. A request-path regression still trips it.
All four knobs stay env-overridable (E2E_LOAD_USERS, E2E_LOAD_SPAWN_RATE,
E2E_LOAD_DURATION_SECONDS, E2E_LOAD_MIN_RPS) for a deliberate load run.
Note the recorded failure for this test was "no requests completed in 60s", which
was the gateway wedged on unreachable Redis rather than a throughput regression;
this change is about not perturbing its neighbours, not about that failure.
* fix(e2e): make the reasoning-tokens assertion exercise a request that reasons
test_openai_chat_reasoning_reports_reasoning_tokens asked "A train travels 60 miles
in 1.5 hours. What is its average speed in mph?" at reasoning_effort="low", then
asserted reasoning_tokens > 0. The model answers that directly without reasoning, so
0 is correct behavior and the assertion was testing the model's discretion rather
than litellm's reporting.
Verified against a live proxy on a dedicated openai/gpt-5.6 deployment, matching how
the test provisions its model:
reasoning_effort=low, one-step arithmetic -> reasoning_tokens=0
reasoning_effort=high, the prompt used here -> reasoning_tokens=114
Raised to high effort with a prompt that requires a proof plus a search, so the
field under test is actually populated and the assertion fails only if litellm stops
surfacing it.
While confirming this I also checked prompt caching, which needed no change:
cached_tokens comes back 3615 of 3618 prompt tokens on a repeated large prefix
against a dedicated deployment. An earlier reading of 0 was an artifact of probing a
fan-out alias whose requests land on different deployments, not a caching defect.
* test(e2e): skip the files-list test while LIT-4820 is open
GET /v1/files does not include a just-uploaded file. The upload returns 200 and
GET /v1/files/{id} resolves it, but the listing never contains it: the returned set
stays fixed at 27 entries whose newest created_at is roughly ten hours older than
the upload, on both the managed (/v1/files?model=) and provider-scoped
(/openai/v1/files) routes. Polled for 40s, so not an eventual-consistency window.
Filed as LIT-4820. Skipping keeps a known, ticketed product bug from holding the
suite red and masking a new regression somewhere else in the same test.
The assertion is left exactly as it was on purpose. It encodes the contract we
actually want, that a file retrievable by id is also enumerable, and anything that
lists files (a UI picker, cleanup tooling that lists then deletes and would
therefore leak provider-side files) depends on it. Relaxing it to get green would
delete the signal. The skip reason says so and links the ticket, and the ticket
records that removing this marker is part of its definition of done.
Matches the existing pattern in this file, where test_unified_file_and_batch_create
skips with a reason citing LIT-3266.
While skipped, the registry cell llm.files.openai.list.nonstream.works has no
passing covering test, so files-list coverage reports as uncovered rather than
passing, which is the honest state.
* fix(e2e): parse Sentinel node lists in the cache-settings model
The value union covered scalar lists and lists of mappings, but not lists of
lists. `redis_startup_nodes` holds host/port mappings while `sentinel_nodes` holds
positional pairs (CACHE_SETTINGS_FIELDS documents `[['localhost', 26379]]`), so on
a Sentinel deployment pydantic rejected the response:
sentinel_nodes.list[dict[str,...]].1
Input should be a valid dictionary [input_value=['localhost', 26380]]
The round-trip test reads GET /cache/settings before it writes anything, so that
rejection failed the test at the read, before any assertion ran. A Sentinel
deployment would have looked like a broken cache-settings route rather than a
model too narrow to parse a documented shape.
A list element may now be a scalar, a list or a mapping, which covers both node
shapes without special-casing either and tolerates a heterogeneous list instead of
rejecting the whole response.
Adds TestCacheSettingsModel, harness-level with no `e2e` marker so it runs without
a proxy, covering all four backend shapes (cluster mappings, sentinel pairs, plain
node, url mode with a null discrete field) plus transport() key selection.
Confirmed it fails on the previous union and passes on this one:
old union -> 1 failed, 4 passed (the sentinel case)
new union -> 5 passed
* test(e2e): remove the cache-settings round-trip test
The test could not fail for the thing it claimed to test, and could break the
deployment it ran against. Both halves of that are worth stating.
It read the live settings, wrote back identical values, and asserted the read-back
matched. If POST /cache/settings were a complete no-op that returned 200 and touched
nothing, GET would still return the values read a moment earlier and the test would
pass. It verified that GET is stable, not that the route persists anything.
Against that, /cache/settings persists what it receives into LiteLLM_CacheConfig,
that row outranks YAML cache_params, and init_cache_settings_in_db re-applies it on a
timer. A write that omits ssl or redis_startup_nodes converts a TLS cluster into a
plaintext standalone client and every later Redis call blocks to socket timeout. On
2026-07-25 that failed 60 of 72 tests in one run: rate limiting stopped enforcing,
Redis-only budgets admitted billable over-budget spend, ResetBudgetJob self-starved,
and guardrail sync never ran.
Guarding the previous shape was not sufficient. Writing the blob verbatim plus a
cluster precondition and a /cache/ping check narrowed the hazard but did not remove
it, because GET cannot express the effective config: it resolves the stored row
overlaid with REDIS_* env and never reads YAML. On a fresh deploy it cannot see
YAML's ssl to echo back, so a TLS non-cluster deployment could still have a row
written that drops it. No round-trip through this route is safe on a shared proxy.
Removed with the models and helpers it owned, and TestCacheSettingsModel with them
since it existed only to protect that parsing.
The registry row mgmt.cache_settings.update.happy_path stays, now carrying the
rationale for why it is deliberately uncovered and what a safe test would require
(an isolated proxy, or LIT-4816 fixed so a partial write cannot downgrade
transport). Coverage therefore reports this cell as a gap, which is the honest
state. Collector passes --strict; the module still collects 11 tests.
The "Savings over time" chart plotted a single floating dot for short
ranges: the daily rollup keys spend by YYYY-MM-DD, so a one-day range is
one point by construction. Rather than stand up an hourly SpendLogs data
source, read that same daily rollup and make the cumulative line legible.
- Cumulative | Per day toggle. Cumulative accumulates within the range;
Per day shows the raw stacked bars.
- Cumulative prepends a synthetic $0 point at the range start
(withStartAnchor) so the line rises from zero to the running total
instead of floating. An empty series is left untouched so the chart's
own "No data" state shows.
- Order the daily series oldest-first (the rollup arrives newest-first)
so the axis reads left to right and the total accumulates forward.
- Header legend, dots on small series, and a "No data" guard on BarChart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Anthropic cache_control breakpoints are positional: each one caches the
prefix ending at the part that carries it. Compresr flattened every text
part of a message into one string and wrote the compressed result back
into the first text part only, which dropped every later breakpoint and,
when a non-text part sat between text parts, moved the trailing text to
the other side of it.
The positional invariant now has one owner. guardrail_hooks/content_text.py
holds content_to_text alongside is_all_text_parts and
merge_rewritten_text_parts, so a compressed string is only ever written
back over a contiguous run of text parts, and the merged part carries the
last declared breakpoint and its TTL.
Compresr consumes that owner at both ends: _select_targets no longer
selects a row holding a non-text part, and _replace_text_in_content
returns such a row unchanged rather than merging across it. Rows whose
content is a plain string are unaffected.
Mixed rows therefore stop being compressed, which is a deliberate trade;
no single-string write-back can preserve a breakpoint across a non-text
part, so the alternative is silently caching a different prefix than the
caller configured.
Running the proxy as a script (python litellm/proxy/proxy_cli.py) puts
litellm/proxy at sys.path[0], so `import a2a` resolved to the internal
litellm/proxy/a2a package instead of the a2a SDK. The optional-import guard in
litellm/a2a_protocol/card_resolver.py swallowed the resulting ModuleNotFoundError
and left its None fallback in place, so the module-level class statement raised
TypeError: NoneType takes no arguments and every proxied A2A agent call failed
with JSON-RPC -32603.
Move the script directory to the end of sys.path instead of removing it; the
sibling-import fallbacks in proxy_cli (from proxy_server import ...) still
need the entry to resolve.
Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com>
* fix(auth): route JWT default-team into memberships instead of the create payload
JWT auto-provisioning (get_user_object with user_id_upsert) merged
litellm.default_internal_user_params verbatim into the Prisma user create,
including a teams key. When a default team is configured through the Admin
UI it is stored as a list of NewUserRequestTeam objects, but the user
table's teams column is String[], so the create raised a Prisma type error
and every JWT-authenticated request 401'd with the user never created.
Mirror the /user/new path: strip teams (and available_teams) out of the
create payload, then route the configured default team through
check_if_default_team_set / add_new_user_to_default_team so provisioned
users get real membership rows. Reuse the synthetic PROXY_ADMIN
UserAPIKeyAuth pattern already used by the team-upsert path to satisfy the
membership permission gate, and import the helpers lazily to avoid the
auth_checks <-> internal_user_endpoints import cycle.
* fix(auth): propagate max_budget_in_team when adding users to default teams
* fix: use pipe union instead of Optional for UP045 budget
* fix(proxy): enforce global max_budget against the resettable proxy budget row
The global proxy budget check compared litellm.max_budget against
SUM(spend) from the MonthlyGlobalSpend view, whose window is hardcoded
to a trailing 30 days. litellm.budget_duration was stored and reset on
a user row that enforcement never read, and startup budgeted the admin
user's own row (default_user_id) instead of the litellm-proxy-budget
aggregate row the spend writer increments per request. Net effect: 1d,
7d and 30d all behaved as a trailing 30 day cap that never reset on the
configured duration.
Startup now upserts the budget onto the litellm-proxy-budget row (and
zeroes lifetime accrual when first putting a row on a reset schedule),
enforcement loads global spend from that row, and ResetBudgetJob drops
the cached global spend accumulator when it resets that row so the cap
unblocks immediately after each window.
Fixes https://github.com/BerriAI/litellm/issues/31292
* refactor(proxy): address review nits on global proxy budget fix
Drop the redundant litellm_proxy_budget_name parameter from
_upsert_proxy_budget_with_reset_at_backfill; its only caller always passed
LITELLM_PROXY_BUDGET_NAME, and any other value would write the budget to a
row enforcement never reads.
Introduce GLOBAL_PROXY_SPEND_CACHE_KEY in constants.py and use it at every
site that previously built the key from litellm_proxy_admin_name (auth
loads, spend-writer increments, startup warm, reset-job invalidation), so
the reader and invalidator can no longer drift apart. The literal key value
is unchanged. Also drop the now-pointless litellm_proxy_admin_name
parameter from _warm_global_spend_cache and the proxy_server import from
the reset-job helper.
The /openai/{endpoint} passthrough forwards a raw OpenAI-format request to
api.openai.com (or OPENAI_API_BASE) with the proxy's OPENAI_API_KEY swapped in,
and still logs a costed pass_through_endpoint SpendLogs row. Nothing exercised
that path end to end.
Adds a live /openai/v1/chat/completions passthrough test that asserts a 2xx
completion and a costed row with custom_llm_provider=openai, mirroring the
gemini and anthropic passthrough cost tests, and registers
llm.chat_completions.openai.passthrough.nonstream.cost_logged.
Swap the antd Collapse "How savings are calculated" panel for click-triggered
shadcn Popovers on each SummaryCard, so the explanation sits next to the
metric it describes instead of in one combined block.
* fix(proxy): hash caller-supplied key in key update audit log object_id
* test: bound audit-log wait to the captured task instead of gathering the loop
