Resolves the test-file conflict by keeping both sides, extends the
finish-reason gate to trace-bearing metadata events so guardrail trace
chunks keep their pre-regression delta shape, parametrizes the
regression test over tool-call, mixed, and reasoning streams, and
repairs the one ant-design icon usage the lucide-react migration left
behind in skill_detail.tsx (semantic conflict on the base branch)
The playground message bubble painted its fill, border and avatar circle from
inline hex values, so in dark mode both bubbles stayed near-white while the text
inherited the dark foreground: the message body was unreadable. The MCP-events
placeholder bubble in ChatUI carried the same three fills.
They move onto the tokens the rest of the sweep already uses, so the assistant
surface is bg-card over border-border and the user surface is the info tint at
the same weight the other selected-state surfaces take. Light mode keeps the
same colour family it had.
The regression test asserts the token classes and that no inline style survives
on either surface, which is the exact shape the bug took.
Regenerating a key from the key info page left the ?key= query param on the
old hash, so dismissing the dialog or reloading landed on a key that no longer
exists and the page rendered "Key not found".
Two defects had to line up. POST /key/{key}/regenerate returns the rotated
hash in token_id and leaves token null, but RegenerateKeyModal read
response.token || response.key_id, neither of which the endpoint populates, so
it always reported the old hash back to its parent. And KeyInfoView's
onKeyDataUpdate prop had no caller anywhere in the tree: VirtualKeysTable owns
the ?key= param and mounts the view but never passed it, so even a correct
hash went nowhere.
VirtualKeysTable now handles the update by pointing ?key= at the rotated hash
and refetching. KeyInfoView holds that callback until the regenerate dialog is
dismissed rather than firing it on the API response, because swapping the
selected key mid-dialog unmounts the view and tears down the one-time
plaintext key before the user can copy it.
* fix(ui): render team and org tpm/rpm limits of 0 as 0 instead of Unlimited
A tpm_limit or rpm_limit of 0 is a hard block on the backend (every request 429s) and only null means unlimited, but the team and organization views rendered both as "Unlimited" (and a team-member limit of 0 as "No Limit") because every display site used a falsy || fallback. The team member edit dialog also seeded its form with `tpm_limit || null`, so opening Edit Member on a member stored with 0 and clicking Save sent null to /team/member_update and silently turned the hard block into unlimited
Every limit display site in TeamInfo, organization_view, the organizations list cell and the team members table now uses a nullish check, and both member form seeding paths keep 0 for max_budget_in_team, tpm_limit and rpm_limit. Regression tests cover each site and the existing memberFormValues test that asserted 0 -> null is flipped to assert 0 survives
Resolves LIT-5760
* test(ui): assert a stored 0 member limit survives an untouched save
The EditMembership integration test named the old 0 -> null collapse as the expected payload, so the related-tests CI job went red once the form kept 0. It now asserts 0 survives and only the empty budget_duration collapses to null. The TeamMemberTab fixture is built with a map instead of mutating the nested membership
The tooltip popup is an inline-flex row, so the four sibling blocks passed as a fragment laid out side by side in four columns. Wrap them in a single flex-col container instead.
The inline code samples also used bg-muted, which is defined against the page surface, not the inverted tooltip surface, so they rendered as near-white chips carrying near-white text. Tint them from the popup's own token instead.
The virtual key shown after creating a key sits in a div with a
hardcoded #f8f8f8 inline background, so in dark mode the box keeps
the light background while the key text inherits the light foreground
color, leaving the key nearly unreadable. Swap the inline styles for
the bg-muted and text-foreground tokens, which resolve per theme.
* fix(terraform): add soft_budget, tags, and soft_budget_alerting_emails to litellm_team
The team resource rejected soft_budget and tags at plan time and had no way
to express the list-valued metadata.soft_budget_alerting_emails the proxy
reads for soft-budget alerts, even though /team/new and /team/update accept
all three. Add the attributes, forward them in buildTeamData (alert emails
merged under metadata, where the proxy stores them), and send the full
metadata map whenever either half changes because /team/update replaces
metadata wholesale.
Read was decoding /team/info as if the team fields were top-level, but the
proxy nests them under team_info, so every attribute silently fell back to
prior state. Decode the envelope and split the proxy's metadata back into
tags / soft_budget_alerting_emails / string metadata, dropping the
server-managed team_member_budget_id.
Verified with OpenTofu plan/apply against a live proxy: the attributes are
accepted, land on the proxy, refresh into state, re-plan clean, propagate
on update, and clear when removed from HCL.
* fix(terraform): clear litellm_team.soft_budget in state when the proxy returns null
Read only wrote soft_budget when the proxy returned a value, so a soft
budget cleared outside Terraform stayed in state and never surfaced as
drift. Set it from the response unconditionally so a null clears it.
`exception_type` decides the class and status a caller sees for every provider
failure, across 190 raise sites, and the tests for it were written one incident
at a time. Nothing said what a plain 401 from any given provider should be, so
mutating a raise site went unnoticed: swapping the class at each of the 190 in
turn, the mapped test file caught 15.
Adds two tables asserted end to end through `exception_type`: 25 providers by
the 9 upstream statuses, and the three error shapes the router branches on
(a full context window, a content policy block, a timeout). The same 190
mutants now fail 97 of them.
The tables record today's behavior, uneven where it is uneven. cloudflare,
ollama and vllm map no status at all, so every failure reaches the caller as a
500. A full context window is recognised by 15 of the 25, and a content policy
block by 11, which bounds where `context_window_fallbacks` and the content
policy retry policy can fire.
proxy-endpoints and proxy-infra are the unit tier's critical path at 358s and
325s of pytest, measured on staging 2026-08-21, and both run two xdist workers
on a four-vCPU runner. proxy-server already runs four. This is the cheaper half
of splitting them: no second job, so no second setup to pay for.
tests/enterprise is 13 files and 244 tests that only CircleCI runs, and CircleCI
gates nothing: it triggers on PR labeled events, none of its jobs are required,
and red runs get merged past. So the suite that covers the enterprise package's
guardrails, auth and management endpoints has had no say in whether a change
lands.
Measured on 2026-08-21 with every credential stripped from the environment: 240
passed, 4 skipped, nothing failed. It needs no provider key, so it can be a
required shard rather than a scheduled lane, unlike the other CircleCI suites in
this group, which each carry a live-API minority.
The CircleCI job is removed in the same commit so the suite runs once, not twice.
* fix(ci): make the migration DDL guard run, and stop it reading comments as SQL
TestMigrationSQLIdempotency requires guarded DDL across litellm-proxy-extras
and has never run in any job, so the convention eroded quietly. Four of its
assertions fail today, and it was allowlisted rather than wired up because
fixing the migrations is not an option: Prisma checksums an applied migration,
so editing one breaks `migrate deploy` for every existing install.
Two things were wrong with the guard itself. It scanned raw lines, so Prisma's
own `-- CREATE INDEX CONCURRENTLY ...` explanations counted as the statements
they describe, which is two of the reported migrations. And it had no way to
say "these predate the rule", so the only options were editing immutable files
or leaving the whole file unrun.
Comments are now stripped before matching, on the drop-column rule too, and the
migrations that already violate are named once in _PRE_GUARD_MIGRATIONS. The
rules bind everything after them, so a new migration with bare CREATE TABLE,
ADD COLUMN, CREATE INDEX or an unguarded ADD CONSTRAINT now fails a check
instead of landing unnoticed.
That set is 14 migrations, not the 13 previously recorded, measured after
comment-stripping. It can only shrink: a test fails if an entry names no
migration on disk, and another fails if an entry no longer violates anything.
The file now runs as a proxy-extras shard and comes off the coverage allowlist.
* fix(ci): strip block comments in the migration guard too
Prisma opens a destructive migration with a /* Warnings: You are about to
drop the column ... */ header. Nothing in the tree trips a rule on that text
today, but it is prose about a statement rather than the statement, and the
line-comment fix left the class open. Bodies are blanked rather than removed
so the reported line number still points at the real statement.
The allowlist recorded eight files in tests/local_testing, 118 tests, that
every job globbing that directory then deselects: local_testing_part1 and
part2 carry `-k "... and not caching and not cache"`, and the other three keep
one unrelated keyword each. They counted as covered while running nowhere.
Five of the eight need nothing. Measured with no provider credentials and no
Redis: test_cache_preset_key, test_caching_handler, test_prompt_caching,
test_responses_stream_cache_keys and test_unit_test_caching pass, 45 tests
together, and they now run as a caching-local shard. The other three stay
allowlisted with what they actually need recorded rather than a question:
test_caching wants Redis and a provider key for 37 of its 65, disk-cache wants
OPENAI_API_KEY for 2 of 4, gcs-cache wants GCS credentials for all 4.
Taking them off the allowlist exposed a gap in the slice guard itself: it
reasoned only about CircleCI `-k` expressions, so a file every slice drops read
as unrun even when a workflow names it outright. It now credits workflow
test-paths the way the census already does, and only workflows, so a tree only
CircleCI globs is still reported.
codecov.yaml has carried an `Enterprise` component scoped to `enterprise/**`
since it was written, and it has never received a line of data. Every one of
the 19 coverage invocations across the unit base, the MCP workflow and the
CircleCI config passes `--cov=./litellm` and nothing else, so 11,203 lines of
paid-customer code sat outside the measured universe while the reported number
described only the rest.
litellm-enterprise is a uv workspace member and a direct dependency, so every
job that syncs already has it installed and importable; only the measurement
was missing.
Measured on tests/test_litellm/enterprise, the shard that exercises this code:
0 enterprise files in the report before, 142 after, at `enterprise/...` paths
that match the component's existing glob. That shard alone puts enterprise at
30.8%, which nudged its own total from 24.09% to 24.20% rather than down. The
aggregate direction across every shard is not knowable until they all report,
and a drop there is the instrument working, not a regression.
* feat(ci): gate patching of SDK internals in tests as TQ008
TQ002 catches the narrowest symptom of the suite's dominant mocking idiom,
patch X then assert only that X was called. The idiom itself is wider: tests
reach for litellm's own functions instead of faking the wire, so they pin how
the code is wired rather than what it does, and a test that patches internals
but makes weak real assertions trips nothing today.
TQ008 counts patch targets rooted at `litellm`, both the dotted string form and
the attribute chain handed to patch.object, and ratchets like every other rule.
Mocking anything outside the SDK is untouched: respx, httpx transports and
third-party clients do not trip it, which is the point, since those are the
patterns this is meant to move the suite toward.
Seeded at 9,643, in line with the ~9.4k patch sites an independent grep found
in the mirror. The burn-down horizon is long; the value here is stopping the
flow rather than clearing the stock.
Five existing rule tests patched `litellm.completion` incidentally and now
report TQ008 alongside what they were pinning. Their expected values are
updated to the accurate pair rather than loosened, so they keep failing on a
regression in either rule.
* test: add TQ008 to the shipped-budget rule canary
* fix(ci): resolve imported SDK names in TQ008
patch.object(handler.OpenAIChatCompletion, ...) after a from-import reaches the
same internal as the dotted string form, but the rule only saw the bare local
name and let it through. Import bindings are now resolved to the path they
stand for, so the aliased, renamed and from-imported forms all read alike and
the reported target is the real one.
That is 1,496 patches the ratchet could not see, so the TQ008 limit moves from
9,643 to 11,139. Third-party names and locals with no SDK import behind them
stay unflagged.
* ci: port the Postgres suites off CircleCI onto service containers
proxy_behavior_tests, proxy_security_tests and schema_migration_check were
near-identical CircleCI jobs: a Postgres sidecar, a schema seed, and one pytest
tree each. They ran nowhere else, and CircleCI holds none of the branch
ruleset's required checks, so the signal they produced gated nothing.
test-postgres.yml runs the same three trees on a Postgres service container as
one matrix, keeping each suite's own seeding rather than normalising it: the
behavior and security trees keep `prisma db push`, and the migration tree keeps
an empty database, which is what it needs to apply every committed migration
itself.
Their CircleCI definitions and workflow entries go with them, taking the config
from 47 jobs to 44. assert_ci_coverage.py stays green: dropping the new
workflow fails the census on exactly these trees, so the coverage moved rather
than went missing.
auth_ui_unit_tests is deliberately left behind. Ported, two of its
tests fail because prepare_metadata_fields refuses enterprise-only keys without
LITELLM_LICENSE, which exists as a CircleCI project variable and has no
GitHub Actions secret. Creating that secret is a human action, so the job stays
on CircleCI until it exists rather than shipping a red shard or quietly
deselecting the two tests.
* chore(ci): drop the narrative header from test-postgres.yml
* perf(ci): fan the budget checkers out across cores
check_type_discipline.py and check_test_quality.py each walk a few thousand
files and parse every one, single-threaded. In the lint job those two steps
measure 2.3 and 1.6 minutes, second and third behind dependency install, and
lint is the slowest required check on 9 of the last 10 merged staging PRs.
check_file is already pure per-file work, so the walk fans out over a process
pool with no change to what either rule reports. Callers sort, which is what
keeps output order stable when results land out of order. Runs below
PARALLEL_MIN_PATHS stay serial rather than pay for process startup, and the
worker count is capped so a large runner does not oversubscribe.
Measured locally over the same trees, output byte-identical both times:
type-discipline 17.8s -> 3.0s over litellm/ (78,768 report lines), test-quality
14.4s -> 2.3s over tests/ (6,321 report lines), per-rule counts unchanged.
* test(ci): type the fan-out helpers and skip the comparison on one core
The lint job installs its dependencies from scratch on every run. That step
measures 2.8 minutes of a job whose p50 is 9.5, and lint is the slowest
required check on 9 of the last 10 merged staging PRs, so it sets the
critical path for the whole PR.
_test-unit-base.yml already caches ~/.cache/uv and .venv keyed on uv.lock.
This mirrors that block. The key carries its own `lint` namespace rather
than sharing the unit tier's: the two jobs sync different group sets
(proxy-dev + e2e-dev here, ci + proxy-dev + four extras there), so a shared
.venv entry would be pruned and rebuilt on alternating runs.
The 20% promotion that runs to 2027-01-31 covers every gemini model, not
just the 2.5 pair, so a constant naming two of them implied the other four
were exempt. Six covered entries live in the registry: two store the
discounted rate and four store list, which is a pre-existing overcharge
this branch does not touch, since it only adds cache fields and derives
them from each entry's own input rate. Name both groups for what they
store, pin the expiry, and tighten the tolerance to 2e-4.
Databricks documents databricks-claude-fable-5 as accepting text only,
where every sibling Claude endpoint accepts text and image. An earlier
commit flipped the flag to true on the reasoning that fable-5 was the
only entry of its generation declaring false, which had it backwards:
it is the only one because the endpoint really does refuse images.
Advertising vision here would surface the model in capability filters
and hand the caller a provider-side rejection.
The two gemini 2.5 entries price a factor of 1.25 under the published DBU
table because the published figures exclude a 20% promotion that runs to
2027-01-31. The previous constant name and test called them an older vintage
awaiting a refresh, which would have led a future reader to scale them up and
overcharge. Pin the discount and the cache relationship instead.
`queued` is the Interactions API's not-started-yet status. It was in neither
the pollable set nor the terminal one, so a create returning it got no poll
task, counted as a response with nothing to charge for, and released its
budget reservation: billed nowhere, alerting nobody.
Poll it alongside `in_progress`, and pin the union of the pollable and
terminal sets against the generated spec enum so a status Google adds later
fails CI rather than shipping another unbilled path. A give-up on a status in
neither set now names the status and logs at error, instead of the warning
that reads as an interaction merely still running.
Also drop `model_id` and `litellm_model_name` from the settled body next to
the foreign `response_cost` already dropped there. All three come from the
poll's own throwaway client call, and left in place the two identity fields
overwrite the create's real deployment in the payload every logging
integration reads.
Rewrites the callback's per-status test to assert the observable outcome
(reservation held vs released) across all eight statuses rather than
comparing the gate to the function it delegates to, and pins the shipped
5-10-20-40-60 poll backoff and its timeout cutoff.
A parenthesised VALUES list ended the search for an insert's row source
only when no group followed it, so a RETURNING or an ON CONFLICT DO
UPDATE carrying a subquery was read as the rows the insert copies. A
writable CTE bounded by its own VALUES list was handed the query the
statement ends with for the same reason: the WITH branch read the whole
statement rather than the part holding the insert.
A CREATE FUNCTION or CREATE PROCEDURE body was scanned as if it ran at
boot, but defining a routine only stores it. The body is now read when
the same migration names the routine somewhere else, so a migration that
defines a backfill and then runs it is still caught, and one whose name
needed quoting is read either way since quoting is blanked at the call
sites too.
main() had no test, so neither its exit codes nor the branch the CI gate
reads were pinned; a mutant returning 0 on a violation passed the whole
suite. Its four outcomes now have tests, along with both directions of
each fix above.
databricks-claude-fable-5 was the only fable-5 entry in the registry
declaring supports_vision false, and the only one of the five new
entries to do so.
Only the five new models were pinned against the published DBU table,
so the 26 cache literals added to pre-existing entries were checked by
nothing independent. Extend the table to all 33 entries carrying cache
rates and assert both cache fields against it for the 31 that take the
published rates, leaving the two older-vintage gemini-2-5 entries to
their existing guard.
Also widen the cache-declaration guard to both cache fields, and
replace the single-model equals-input assertion with one that covers
all 14 entries publishing no cache rates.
Cache rates were derived as ratios of the dollar input rate (1.25x write,
0.1x read) while input and output derive from the published DBU table
times $0.070. Databricks publishes cache write and cache read DBU per
model, and those are not exact multiples of the input DBU, so the two
rules disagreed by up to 0.1 percent.
Rewrites 43 cache literals across 31 entries to published_cache_DBU x
$0.070. Skips databricks-gemini-2-5-pro and databricks-gemini-2-5-flash,
whose input and output rates predate the current table by a 1.25x
increase; their cache rates stay tied to their own input rate so each
entry remains internally consistent.
Replaces the ratio assertions with a test pinning the absolute published
DBU figures for the five new models, and adds a test pinning the
older-vintage exception. Corrects the metadata note on the five new
entries, which claimed the reference-only *_dbu_cost_per_token fields
drive cost calculation.
Silencing the cost-tracking alert for every usage-less interaction response
went one status too far. An interaction that stopped at failed, cancelled,
incomplete or budget_exceeded genuinely has nothing to charge for, so alerting
on it is noise. completed and requires_action are different: both mean the
model produced output, so a usage block is always expected, and one arriving
without it means the charge for real work was lost. That is precisely the case
failed_tracking_alert exists to surface, and swallowing it would let an
operator's interactions bill nothing with no signal that anything was wrong.
The status knowledge lives next to the other status predicates rather than in
the proxy callback. The reservation is still released on both paths, since
suppressing the alert was never what freed it.
The poll fetches the terminal interaction through its own client call, which
stamps a response cost computed by a throwaway logging object holding none of
the original request's deployment context: no model_info, no router model_id,
no deployment litellm_params. Carrying that cost into the settlement event
billed custom-priced deployments at the wrong rate, and it also satisfied the
"already calculated" shortcut in _response_cost_calculator's caller, so the
settlement never repriced and never built a cost breakdown. The zeros stamped
by the usage-less create survived into the spend log row and the OTEL span.
Dropping the imported cost before re-emitting makes the settlement price the
settled body itself, against the deployment that served the create.
The figures came out of a float product of the DBU rate and the dollar
rate, so the map carried representation tails such as
2.9999900000000006e-07 where the derivation only means 2.99999e-07.
Trim the 44 values this change adds or edits to the shortest literal that
round-trips to the same figure, leaving every pre-existing value alone.
The largest move is under 1e-15 relative, so no billed amount changes.
The tightened gate correctly stopped deferring the reservation release for
InteractionsAPIResponses the scheduler will not poll (terminal status, or
in_progress without an id), but the response then fell through into the
generic 'Cost tracking failed' raise and the failed_tracking_alert path.
A create returning failed, cancelled, requires_action, incomplete,
budget_exceeded, or an id-less in_progress without usage therefore released
its reservation as intended and, in the same breath, alerted operators for a
legitimate no-usage response, both creating noise and masking real
cost-tracking failures.
The two gates are now nested under a single 'unbilled interaction response'
outer check, so any InteractionsAPIResponse with no usage takes either the
defer path (pollable, polling on) or the release-and-return path, and none
of them fall through to the generic failure raise. The two regression tests
also now assert failed_tracking_alert is not called, closing the observation
gap the report flagged.
The new supports_prompt_caching assertion reads a capability this branch
adds to the registry, so it only holds against the bundled map. CI leaves
LITELLM_LOCAL_MODEL_COST_MAP unset and fetches main's copy, which lags the
branch until merge, so the test failed there while passing locally.
Use the local_model_cost_map fixture the repo already provides, matching
what the other two test files in this change do.
The shared cost calculator treats a missing cache rate as free, so routing
Databricks through it billed cached tokens at zero on the 14 entries that
publish no cache pricing. On a 10,000 token prompt with 8,000 cache reads
that is $0.0010000 against the correct $0.0050001, a fivefold undercharge.
Those entries now declare cache rates equal to their input rate, which is
what a model with no caching discount should charge, and a test pins every
priced Databricks entry to declaring cache rates so no future entry can
regress into it.
Also repoints the provider-neutral generalization test off an id the new
Opus 5 entry now shadows, adds backup-to-main parity tests for the five new
entries, pins that Databricks Claude is never auto-injected with cache
control despite reporting caching support, and trims the Sonnet 5 pricing
note, which is served on an unauthenticated route.
Both leave a background interaction's pre-call reservation open, so the
serving process keeps refusing traffic on the key at the estimated cost
while its recorded spend stays near zero.
A raise from the completion event propagated out with the settlement gate
already claimed, and nothing retries a claim that is set, so the reservation
was never released. Billing now releases it on the way out.
`requires_action` was missing from the terminal set. It is terminal for the
interaction it names: the API has no operation that resumes one, and a caller
answers a tool request by creating a new interaction whose
`previous_interaction_id` points at it. A function-calling background create
that stopped there was polled until the 3600s timeout, losing the tokens it
had already spent producing the tool request and holding its reservation open
for that whole window.
