Commit graph

43912 commits

Author SHA1 Message Date
Mateo Wang
e51addb802
Merge pull request #37628 from BerriAI/litellm_lit5876_openai_prompt_cache_breakpoint
feat(prompt-caching): map cache_control_injection_points to OpenAI prompt_cache_breakpoint on GPT-5.6+ targets
2026-08-20 10:24:43 -07:00
Mateo Wang
ec810696e4
Merge pull request #37626 from BerriAI/litellm_lit5874_native_cli_oauth_pkce
feat(proxy): native CLI login with OAuth authorization code + PKCE
2026-08-20 10:24:41 -07:00
yuneng-jiang
ffab5a39d0
feat(ci): ratchet the test suite's zero-assert, mock-echo and global-state debt (#37588)
* feat(ci): ratchet the test suite's zero-assert, mock-echo and global-state debt

The suite's dominant failure mode is tests that cannot fail for the reason anyone
would want them to. The testing-strategy audit measured five shapes of it, and
nothing mechanical stops any of them from reproducing, so they keep reproducing.

`scripts/check_test_quality.py` is an AST checker for those five, emitting the
same `path:line: CODE message` contract as `scripts/check_type_discipline.py`:

  TQ001  a collectible test with no assertion of any kind
  TQ002  mock-echo, where every assertion only inspects the mock that was patched
  TQ003  sys.path.insert inside the test tree
  TQ004  raw `os.environ[...] =`, which leaks into whatever runs next
  TQ005  `litellm.<attr> =`, the process-wide leak the 491-line conftest undoes

`scripts/test_quality_gate.py` caps each rule against test-quality-budget.json,
seeded at exactly today's count, and fails only when a rule is both over its
limit and higher than the base being merged into, so a change is blamed for what
it adds and never for drift already in the base. `--update` lowers a limit by
what a branch cleared, so the ceilings only ever fall. It runs in the existing
required lint job, which means it enforces without a ruleset change.

TQ001 follows assertions into helpers defined in the same module, transitively.
Without that it flagged 111 tests in tests/e2e, the harness this program holds up
as the reference, because that suite factors its assertions into shared helpers
(`assert_auth_denied(result, ...)`). Following them leaves 25, all of which reach
their assertions across a module boundary; those are grandfathered and documented
rather than papered over.

The seeded counts land within about 10% of the audit's independent numbers for
every rule measured on the same subtree, which is the cross-check that the
definitions here match the ones the audit pinned.

* fix(ci): resolve test helpers per scope, not by bare name

The helper walk keyed every function in a module by its bare name, so two
same-named helpers in different classes collided and the last one parsed won.
A test calling `self._check()` could be cleared by a `_check` belonging to a
different class, or flagged because of one.

Resolution is now scoped: a bare name looks up the module-level functions, and
`self.<name>` looks up the enclosing class's own methods and no other class's.
Recursion is tracked by function identity rather than by name, so the cycle
guard cannot be confused by the same collision.

This surfaced one real zero-assert test that a same-named helper elsewhere had
been clearing, so TQ001 seeds at 750 rather than 749.

The test module has to register itself in sys.modules before exec_module:
`@dataclass(slots=True)` rebuilds its class through `sys.modules[__module__]`,
and Scope fails to construct without it. Recorded at the call site, since it
reads like avoidable global mutation otherwise.

* fix: register test-quality-budget.json with the ratchet alarm

The repo keeps one census over its budget files: every *-budget.json on disk
must appear in DEFAULT_BUDGETS, or its ceilings can be raised with no signal.
tests/test_litellm/test_budget_ratchet_check.py asserts that set equality and
caught the new budget on the way in.

Registering it also turns the alarm on for TQ001-TQ005, so a later PR cannot
quietly raise a test-quality ceiling. The file already uses the {limit: N}
schema the ratchet reads, so no other change was needed.
2026-08-20 10:08:49 -07:00
yuneng-jiang
7de8f458e0
test: retire tests/old_proxy_tests, which holds no tests (#37605)
* test: retire tests/old_proxy_tests, which holds no tests

Twenty files named test_*.py, and pytest collects nothing from any of them:

  uv run pytest tests/old_proxy_tests --collect-only -q
  no tests collected, 16 errors in 114.17s

They are manual snippets against a running proxy, written at module level with
no test function, no assertion and no entry point, so the only thing the name
buys them is a place on the coverage allowlist. Sixteen of the twenty cannot
even be imported in this environment, wanting langchain, llama_index or
google.api_core, and ten still point at 0.0.0.0:8000, which stopped being the
proxy's default port some time ago.

Nothing outside the directory refers to it apart from the allowlist entry, which
goes with it. The other loose contents go too: five load_test_*.py scripts, a
bursty variant, two committed log files, an essay fixture and a stray .js
snippet.

Allowlist paths 88 -> 68, test files 2422 -> 2402, and no job loses anything it
was running. Recoverable from history if a snippet turns out to be someone's
habit.

* test: drop the retired old_proxy_tests paths from the coverage allowlist
2026-08-20 10:08:28 -07:00
yuneng-jiang
5b1a9563d6
chore(ci): close the test-census blind spots and move scripts out of workflows/ (#37586)
The agent job's CircleCI glob collected `tests/agent_tests/**/test_*.py` and then
piped it through `grep -v` to drop `local_only_agent_tests/`. `assert_ci_coverage.py`
reads the glob but not the pipeline, so those two files looked covered and were
invisible to the census. The glob now excludes them structurally and they carry an
allowlist entry instead, which is a decision on the record rather than a hidden
filter. The collected file set is unchanged: `tests/agent_tests/` holds exactly one
CI-runnable test at the top level.

`tests/scim_tests/` held a single JSON fixture and no tests, referenced from nowhere.

`.github/workflows/` is for workflows. Both stray scripts move to `.github/scripts/`
with their callers updated: the price-file updater is invoked by
`auto_update_price_and_context_window.yml`, and the translation-report runner by
`make test-llm-translation`. The audit listed the latter as orphaned, but Makefile
line 317 still runs it, so it moves rather than being deleted.

The rollout heads-up workflow was a deliberate one-shot for the agent-shin rollout.
That rollout is done, the triage and auto-close workflows have been running daily
since June, so the pre-flip warning window is long past. Its script and dedicated
test go with it, and the sibling workflow-invariant test drops its entry.
2026-08-20 10:07:14 -07:00
yuneng-jiang
68deda3da7
refactor(ci): fold the nine thin unit-shard callers into one matrix (#37590)
Nine workflow files existed only to make a single call to _test-unit-base.yml
with a different test-path. Adding a shard meant adding a file; changing
anything shared meant editing nine. One matrix caller replaces them, so a shard
is now one entry.

Check names are unchanged, which is the whole constraint. A reusable-workflow
job reports as "<job name> / <inner job name>", so setting `name` to the shard id
alone reproduces today's context strings exactly: the eleven the matrix produces
are eleven of the twenty-three "/ Run tests" contexts the branch ruleset
requires, matched string for string. No ruleset edit is needed and none should
be made for this.

Every matrix entry states its timeouts even where they equal the base defaults.
An absent matrix key renders as an empty string rather than falling back, and an
empty string is not a number, so a partially-specified entry would fail the call.

tests/proxy_unit_tests keeps test-unit-proxy-db.yml. It is already a matrix and
its shard-coverage guard reads that file by name, so folding it in belongs with
generalising that guard into assert_ci_coverage.py rather than here. Its twelve
shards are the remaining required contexts.

test-unit-documentation.yml stays too: it does not call the base workflow.
2026-08-20 10:07:02 -07:00
Mateo Wang
fad8116cdc
Merge pull request #37615 from BerriAI/litellm_autorouter_marker_region_leak
fix(router): routed deployment's own litellm_params beat forwarded auto_router marker params
2026-08-20 09:06:51 -07:00
mateo-berri
64e7ab1beb fix(cli): keep the login record when the proxy cannot record a logout's revocation
A 503 from POST /revoke means the proxy could not write the single-use record, so clearing the local record left a live refresh token nobody could revoke and a hint to retry with nothing left to retry. lite logout now keeps the record, exits 1, and asks to be run again shortly. A refused or unreachable revocation still clears the record and warns as before, and a re-login that replaces a record keeps its existing warning because the new record already stands
2026-08-20 08:16:30 -07:00
mateo-berri
238609ea5d fix(mcp): answer 503 when a refresh-token burn cannot be recorded
revoke_refresh_token discarded the single-use claim result, so a revocation that arrived while Redis was unreachable answered 200 and left the refresh token live. The token endpoint reported the same outage as invalid_grant "already used". The guard now reports first, replayed, or unavailable, and both endpoints answer 503 temporarily_unavailable for an outage (RFC 7009 section 2.2.1, RFC 6749 section 5.2), which the CLI surfaces as a one-line warning while keeping the key it has
2026-08-20 07:56:38 -07:00
mateo-berri
a20cfc5f0e fix(cli): let lite up trust the key the group already renewed
The lite group resolves the stored key once for every command and renews a --pkce key on the way in. lite up then asked the token file again, so every start sent a second refresh to the proxy, and once the refresh token was burned the refusal printed twice. _ensure_fresh_login now reuses the key the group resolved when the group read it from the token file, and only re-reads the file after the interactive login it starts itself. Also covers print-token through the group with a renewing session in the tests
2026-08-20 07:40:17 -07:00
mateo-berri
36e9e813a7 fix(cli): renew the key once per lite auth print-token run
The cli group already resolves the stored key for the server it was pointed
at, so print-token re-ran the renewal and, when the refresh token had been
revoked, posted to /token twice and printed the reason twice. print-token now
reuses the group's result whenever the stored record was issued for that
server and no --api-key or LITELLM_PROXY_API_KEY took precedence, and only
resolves the key itself when invoked bare for a different server.
2026-08-20 07:29:04 -07:00
mateo-berri
44d3fc7ec8 fix(cli): name lite login --pkce and the refresh failure when a key cannot be renewed
A PKCE credential whose renewal is refused (for example after lite logout ran
on another copy of it) used to fail lite auth print-token with the classic
'Token expired. Run lite login again' hint and no reason, while lite whoami
already named lite login --pkce. fresh_api_key now reports why a renewal
failed through a warn callback whenever no sibling rotation rescued it, the
CLI prints that reason on stderr, and the expiry hint names the command that
produced the credential. Both READMEs document the admin revocation semantics
and the Redis precondition for refresh single use on several workers.
2026-08-20 07:21:42 -07:00
mateo-berri
39bdacb3a6 fix(cli): revoke the previous login's refresh token when lite login replaces a stored record 2026-08-20 06:26:57 -07:00
mateo-berri
218a3ec370 fix(cli): never follow a redirect when posting to the proxy's OAuth endpoints
Discovery checks that every endpoint sits on the proxy origin, but the CLI's
requests.Session followed redirects, and requests replays a POST body on
307 and 308, so a token or revocation endpoint answering with one of those
would have sent the code and verifier, or the refresh token, wherever
Location pointed. Every POST now goes out with allow_redirects=False and a
3xx answer fails the command with a message naming where it pointed
2026-08-20 06:03:13 -07:00
mateo-berri
e16cf5ed3f Fall back to the GPT version rule when the cost map carries no breakpoint flag
A proxy on the default remote cost map never produced a prompt cache
breakpoint: the published map has the gpt-5.6 entries without
supports_prompt_cache_breakpoint, so the model-map gate returned False
for every listed model and only LITELLM_LOCAL_MODEL_COST_MAP=True (the
repo .env, hence the passing unit tests) made the feature work. The hook
now honors the flag when the entry carries one, True or False, and
otherwise applies the GPT-5.6+ version rule to the model name, so a map
that lags the flag still gets the OpenAI dialect. The model-map tests
pin litellm.model_cost to the bundled backup map and a new test drives
the hook against an unflagged gpt-5.6 entry.

completion() and acompletion() take base_url as an alias for api_base
that only lands on api_base after the cache control hook ran, so a
GPT-5.6 call at a non-OpenAI gateway given through base_url still got
the dialect. Both seed calls and the unstamped request-params read now
look at base_url too.

ResponsesAPIRequestUtils.merge_prompt_management_input reshaped hook
output in place, retyping text parts to input_text on the caller's own
message objects. The merge now shapes a copy of each message as it
emits it, so the identity-based merge keeps working on the hook's
objects and nothing the hook or the client owns is mutated.
2026-08-20 05:47:19 -07:00
mateo-berri
17455f4086 fix(proxy): refuse a teamless native-client grant for a user who has teams
The consent page offers the team picker, but a form posted without a team
sealed a teamless grant and the token endpoint minted an unscoped
credential for a team member, escaping the team attribution classic lite
login always applies. The minter now refuses such a grant on redemption
and refresh alike; memberships whose team rows are gone still count as no
team so they cannot lock a user out
2026-08-20 05:37:35 -07:00
mateo-berri
c011b4b56f fix(cli): tell whoami's expired PKCE key to sign in again instead of promising a renewal 2026-08-20 05:22:26 -07:00
mateo-berri
d9aaa95978 Gate OpenAI prompt cache breakpoints on the real target and carry them through /v1/responses
The cache control hook also runs on litellm.responses() input. On a
GPT-5.6 deployment it wrapped a string-content item into a chat-shaped
{"type": "text"} part, which the Responses API rejects, and it never
marked input_text, input_image or input_file parts, so no breakpoint and
no prompt_cache_options reached the provider. Add the Responses part
types to the eligible block set and translate chat-shaped text parts on
non-assistant items to input_text in
ResponsesAPIRequestUtils.merge_prompt_management_input, which both the
async and the sync prompt management sites go through.

The dialect also fired for any GPT-5.6 name that resolved to provider
openai, including deployments pointed at a custom api_base that does not
understand prompt_cache_breakpoint. Decide it once per request from the
provider, the model map and the resolved api_base (request, then
litellm.api_base, then OPENAI_BASE_URL / OPENAI_API_BASE): only
api.openai.com and *.api.openai.com hosts speak the dialect, a top-level
prompt_cache_options opts a custom target in, and litellm_proxy/ targets
never get it. maybe_seed_default_injection_points takes api_base and
stamps the finished decision on the points as _litellm_openai_dialect so
the sync completion() path, whose hook params do not carry api_base,
honors it; maybe_inject_cache_control takes api_base from the
/v1/messages handler.

Eligibility now comes from a supports_prompt_cache_breakpoint model map
flag on the OpenAI gpt-5.6 entries, exposed through
litellm.utils.supports_prompt_cache_breakpoint, with the GPT version rule
kept only for models the map does not know. The OpenAI dialect no longer
reserves a slot for tool_config points, which OpenAI has no cache block
for, and with_prompt_cache_breakpoint plus the chat bridge helper return
a new block instead of mutating their input.
2026-08-20 05:17:13 -07:00
mateo-berri
52c94327bd fix(cli): renew a --pkce key wherever lite checks freshness and show its expiry in whoami 2026-08-20 05:17:00 -07:00
mateo-berri
fb123b33de fix(cli): bind the sibling refresh fallback to the same user and team 2026-08-20 04:41:43 -07:00
mateo-berri
912206832c fix(cli): pin PKCE discovery to the typed proxy and ignore foreign sibling records
The discovery document is accepted only when its issuer is the --base-url the user
typed and every endpoint and the resource share that origin (RFC 8414 section 3.3),
so a tampered or redirected document can no longer point the code, verifier, or
refresh token at another host. After a failed refresh the re-read token record is
used only when it continues the same credential (same proxy, token endpoint, and
resource) and has not expired, so a concurrent login against a different proxy can
never hand this one its key
2026-08-20 04:29:01 -07:00
mateo-berri
5f6d22e792 Map cache_control_injection_points to OpenAI prompt_cache_breakpoint on GPT-5.6+ targets
When the resolved deployment is provider openai and the model is GPT-5.6 or
newer, the cache control hook now writes prompt_cache_breakpoint on the
targeted content block and sets prompt_cache_options to explicit mode unless
the caller already passed one. The /v1/messages bridges carry the marker
through (the Responses bridge moves a marked system prompt into a developer
message, since top-level instructions cannot hold one). Breakpoint counting
and the stand-down check recognise both marker kinds, and client breakpoints
already present in messages are no longer subtracted from the cap twice.

Fixes #37509
2026-08-20 04:16:22 -07:00
mateo-berri
2c691d3820 feat(proxy): native CLI login with OAuth authorization code + PKCE
The proxy's OAuth authorization server (dynamic registration, PKCE S256,
loopback redirects, single-use codes, refresh rotation) gains a proxy-API
audience: /authorize?resource=<proxy origin> renders a consent page with
team selection and /token mints the same per-user credential lite login
mints, so a native CLI can sign a user in through the system browser and
call /v1/* with user and team attribution. Adds GET /.well-known/litellm-cli-auth
as the versioned discovery contract for non-Python clients, POST /revoke
(RFC 7009) for logout, and lite login --pkce, lite logout, and
lite auth print-token on the CLI side. Proxy-API grants only ever redirect
to a loopback address and the server never picks a team on the user's behalf.

Fixes #37332
2026-08-20 04:10:10 -07:00
Mateo Wang
6d47468dae
Merge pull request #36968 from oneKn8/fix-anthropic-mid-conversation-system-cache
fix: preserve prompt cache for mid-conversation system on unflagged Claude models
2026-08-20 03:33:40 -07:00
mateo-berri
7683affdff fix(router): carry forwarded auto_router marker params per request and keep tier overrides and marker flags
The forwarded-keys record moves off the shared metadata dict onto the
per-request kwargs, where _update_kwargs_with_deployment consumes it, so
sibling requests that share a metadata dict (abatch_completion) can no
longer clear it mid-routing. Per-tier litellm_params from the hook
response are never treated as forwarded marker params, and a deployment
only beats a forwarded value when it sets its own, not when it carries a
LiteLLM_Params default such as merge_reasoning_content_in_choices=False.
2026-08-20 03:23:28 -07:00
mateo-berri
e11399fc56 test(e2e): pin the unflagged Vertex cache test to us-east5 2026-08-20 03:13:14 -07:00
Mateo Wang
952c6d5675
Merge pull request #37517 from BerriAI/devin_ai_bedrock_grok_4_6_cost_map
feat: add bedrock grok 4.6 to model cost map
2026-08-20 03:03:44 -07:00
mateo-berri
2604f888c4 fix(router): let the routed deployment's own litellm_params beat forwarded auto_router marker params
An `auto_router/<alias>` marker entry's litellm_params (for example
`aws_region_name: eu-west-3`) were forwarded onto every routed call with
setdefault and then won the `{**litellm_params, **kwargs}` merge against
the selected tier's own values, so a Bedrock tier pinned to us-east-1 was
called in eu-west-3 and failed with 400.

The hook now records which keys it actually forwarded on the request's
metadata bucket, and `_update_kwargs_with_deployment` drops every
forwarded key the selected deployment defines itself, so marker params
only fill gaps a tier leaves open. Request-supplied values still win over
both. The stamp is stripped from logged metadata like its siblings.

Fixes #37613
2026-08-20 03:03:10 -07:00
mateo-berri
155ef8cb11 test(e2e): require consecutive cache reads before the reminder turn 2026-08-20 03:03:08 -07:00
mateo-berri
fc98bab180 Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into fix-anthropic-mid-conversation-system-cache 2026-08-20 02:55:51 -07:00
mateo-berri
fe7ada15e2 fix: keep tool_use and tool_result adjacent when converting mid-conversation system turns
On models without supports_mid_conversation_system, a system entry between
an assistant tool_use turn and the user tool_result turn became a user turn
in that position and the provider rejected the request ("tool_use ids were
found without tool_result blocks immediately after"). That run of entries
now goes right after the tool_result turn, where consecutive user turns
merge upstream. The converted turn also carries only role and content, as
the hoist did, so an entry with extra keys no longer 400s with "Extra
inputs are not permitted".

The e2e cache priming re-sends the identical first turn until its own cache
entry reads back before the reminder turn goes out, since Vertex can take a
few seconds to serve a freshly written entry.
2026-08-20 02:43:10 -07:00
Mateo Wang
6fcdea03b0
Merge pull request #36969 from oneKn8/fix-cost-map-mid-conversation-flag
fix: add supports_mid_conversation_system to bare first-party Claude cost-map keys
2026-08-20 02:04:56 -07:00
yuneng-jiang
5290150a05
fix(ui): keep semantic button colours on hover after the no-op hover cleanup (#37580)
PR #37579 read `text-X hover:text-X` on a shadcn Button as dead weight and
removed the hover half. On the ghost and outline variants it was not dead: both
carry their own `hover:text-foreground`, and the duplicate in the className was
the thing displacing it through tailwind-merge. Dropping it handed the hover
back to the variant, so the Remove button in a team's logging settings, the
chat storage banner's dismiss control, and the collapsed enterprise-usage rail
all lose their colour the moment you point at them.

Each of the three now carries a distinct hover value, following the alpha-step
idiom the rest of that migration used, which restores the colour and keeps
`local/no-noop-hover-variant` satisfied.

Every other hover utility that PR dropped sits on a plain element or a variant
with no competing `hover:text-`, so those stay as they are.
2026-08-19 23:13:08 -07:00
yuneng-jiang
6811f1d37f
fix(ci): run the full dashboard suite when a change reaches outside src/ (#37563)
The UI unit test job narrows a pull request to `vitest related <changed
files>`. `related` maps a file to the tests that import it, so a file no
test imports maps to nothing, and `--passWithNoTests` turns that empty
selection into a green job. package.json, package-lock.json, the Vitest,
Tailwind and TypeScript configs and tests/setupTests.ts are all in that
category even though each of them can change the behaviour of every test
in the suite, so a dashboard dependency bump merged having run no unit
tests at all and only got real coverage later, from the full run on the
push to litellm_internal_staging.

Keep `related` for the common case where a pull request only touches
files under src/, and fall back to the full suite as soon as one changed
file sits outside it. The decision lives in
.github/scripts/select_ui_test_scope.sh so it can be tested on its own,
next to the existing classify_changes.sh gate.
2026-08-19 23:09:03 -07:00
yuneng-jiang
93c1461074
fix(ui): restore hover feedback and dark-mode variants lost in the token migration (#37579)
* fix(ui): restore hover feedback and dark-mode variants lost in the token migration

PR #37576 mapped hardcoded Tailwind palette classes onto semantic tokens. Two-tone
hover pairs collapsed onto a single token, so 116 hover utilities across 49 files
became identical to their base class and produced no visible feedback, and in seven
files a dark: variant was dropped while its hardcoded light partner survived, leaving
those elements stuck light in dark mode.

Hover states now follow the alpha-step idiom the shadcn primitives already use
(hover:bg-primary/80, hover:bg-success/20): a duplicated hover:text-X or hover:bg-X
becomes /80, hover:border-border becomes hover:border-ring, and a duplicate is
dropped where another hover utility on the element already carries the change. One
transition-colors that no longer animated anything is removed.

For the dark-mode gaps, indigo maps onto info and amber onto warning. There is no
purple token in globals.css, so the purple sites keep their palette classes and get
their dark: partner back.

* fix(ui): add an eslint rule that fails a hover: utility identical to its base

The token migration collapsed two-tone hover pairs by hand, so nothing catches
the next one. `local/no-noop-hover-variant` reads every string literal and
template chunk and errors when a `hover:X` sits alongside a bare `X`, which is
exactly the shape that renders no hover feedback. It ships at error with no
suppression baseline, so the eleven sites that already carried a dead hover
before the migration are fixed here too.

The rule reads one class string at a time, so a base class supplied by a
different ternary branch than its hover partner is left alone: a selected row
whose resting colour already matches its hover colour is deliberate, not a bug.
2026-08-19 22:52:36 -07:00
Mateo Wang
47a7e1742e
Merge pull request #37539 from BerriAI/litellm_batch_enqueued_token_limit
feat(proxy): enqueued-token rate limiting for batches with refund on completion and cancellation
2026-08-19 22:22:05 -07:00
yucheng-berri
8cf0b50125
fix(ptu): hand the prune a plain delete filter the query builder can serialise (#37571)
* fix(ptu): hand the prune a plain delete filter the query builder can serialise

The bounded sweep built its predicate as a read-only mapping view, which the query
builder refuses to serialise, so the nightly job raised as soon as a config-declared
deployment was priced. The charges were already written by then, which is why the run
looked like it had produced its rows.

The in-memory table these tests run against accepts any mapping, so only a live run
caught it. A predicate builder now returns a plain dict and is asserted as one, and the
catch-up pass has a test covering a config-declared reservation.

* refactor(ptu): build the prune predicate in one shot

Both filter shapes are known upfront, so the bounded one is constructed
directly rather than by mutating a value already declared Final.

The catch-up test took two independent clock reads, which disagree across
UTC midnight; it now derives both the reservation start and the expected
last charged day from a single read, matching the three sibling tests.
2026-08-19 22:18:58 -07:00
ryan-crabbe-berri
57b328ff96
refactor(ui): map hardcoded Tailwind palette classes onto semantic tokens (#37576)
* refactor(ui): map hardcoded Tailwind palette classes onto semantic tokens

The dashboard painted itself with literal palette utilities (text-gray-500,
bg-blue-50, border-red-200) that resolve to one fixed color regardless of
theme, so the shadcn token layer and its .dark block could never take effect.

A codemod (scripts/codemod-color-tokens.mjs) rewrites 3,232 of those across
254 files onto the existing token scale: neutrals become foreground /
muted-foreground / muted / border / card, and red, green, amber and blue
collapse into destructive, success, warning and info, with the pale -50 to
-300 tints expressed as opacity modifiers on the same token. Hover and focus
variants map to accent so they lift rather than recess. 210 now-redundant
dark: variants are dropped since the tokens carry both modes.

The .dark palette is retuned to a neutral gray ramp with the sidebar recessed
below the content canvas, replacing the blue-tinted shadcn default where the
sidebar read as a full-height card floating on a near-black page.

Nothing sets the .dark class yet, so light mode is unchanged and dark mode
stays inert until a theme toggle lands.

* chore(ui): drop the one-shot color-token codemod script
2026-08-19 22:11:44 -07:00
ryan-crabbe-berri
7b574b9df6
chore(ui): drop the antd dependency and its leftovers (#37574)
Some checks failed
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Publish basedpyright base counts / publish (push) Waiting to run
Code Quality Checks / code-quality (push) Waiting to run
UI Unit Tests / ui-unit-tests (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Unit Tests: Core Utilities / core-utils (push) Has been cancelled
Unit Tests: Enterprise, Google GenAI & Routing / enterprise-routing (push) Has been cancelled
Unit Tests: Integrations (Callbacks & Logging) / integrations (push) Has been cancelled
Unit Tests: LLM Provider Transformations / Vertex AI (push) Has been cancelled
Unit Tests: LLM Provider Transformations / All Other Providers (push) Has been cancelled
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Has been cancelled
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Has been cancelled
Unit Tests: Proxy Infrastructure / proxy-infra (push) Has been cancelled
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Has been cancelled
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Has been cancelled
Unit Tests: Proxy API Endpoints / proxy-server (push) Has been cancelled
Nothing in the dashboard renders antd any more, so the package and the
scaffolding around it can go. This removes `antd` and
`@ant-design/cssinjs` from package.json, deletes the global StyleProvider
the root layout wrapped every page in, drops the `antd` cascade layer and
the z-index override that lifted Base UI popups over an antd Modal, and
retires the lint rules that policed antd imports and antd class selectors
in tests.

Fifteen test files still carried `vi.mock("antd", ...)` factories for
components that stopped importing antd during the migration. They were
inert, and they resolve the real module, so they would have broken the
moment the package left node_modules.

The compatibility shims keep their behaviour and lose the antd name:
`antdRules`/`antdRequired` become `validatorRules`/`requiredRule`,
`isAntdUrl` becomes `isValidUrl`, and `ABOVE_ANTD_MODAL` becomes
`NESTED_DIALOG_LAYER`. Comments that explain why a contract looks the way
it does still name antd, because that history is the reason.
2026-08-19 20:44:27 -07:00
ryan-crabbe-berri
0b374541bb
refactor(ui): migrate the last antd components off antd onto shadcn (#37569)
Converts the remaining dashboard components that still imported antd: admin panel, agents, MCP toolsets, policies, prompts, bulk user edit, create user, plugin settings, teams, add model, auto router, cloudzero export, BYOK credentials, credential modal, onboarding link, create key and routing groups.

Primitives map onto the house shadcn set: Typography onto semantic tags, Select onto ui/select, SearchSelect or MultiSelect, Input onto ui/input, Tooltip onto SimpleTooltip, Card, Table, Tabs, Switch, Checkbox, Radio, Tag onto Badge, Divider onto Separator, Spin onto UiLoadingSpinner, Modal onto Dialog, message onto toast, and Space, Row, Col, Flex and Layout onto flex containers.
2026-08-20 03:01:07 +00:00
mateo-berri
d5ac49588a Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_batch_enqueued_token_limit 2026-08-19 19:31:46 -07:00
hiraku-miyoshi
2a771caf02 fix(proxy): clamp reservation record TTL so stale records never outlive their counters 2026-08-19 19:12:22 -07:00
yucheng-berri
c2b3c4b1e4
feat(ptu): accrue flat cost for PTU deployments declared in config.yaml (#37556)
* feat(ptu): accrue flat cost for PTU deployments declared in config.yaml

The flat-cost rollup reads deployments from LiteLLM_ProxyModelTable, and config.yaml
models never reach that table by design, so a PTU deployment declared there accrued no
flat cost at all while still billing its traffic per token. The provider bills the
reservation whichever file declared it.

The rollup now also reads the deployments the router holds that no database row owns,
identified by db_model, skipping the per-request credential clones that carry
original_model_id and reuse their source's PTU config under a fresh id. Registering such
a deployment zeroes its pricing, since reserved capacity already pays for the traffic it
serves, and leaving a rate unset falls back to the public cost map, which makes the double
charge the default rather than an opt-in.

The rules both halves apply now live in one module. The rollup's test for what it will
charge and the router's test for what to zero have to agree, or a deployment one accepts
and the other declines serves its traffic for free. That module also owns the fields the
write endpoints already zero, so the two paths cannot drift: tiered_pricing is emptied
rather than zeroed because its tiers outrank the rates beside them, the search context
table is written zeroed because an absent one means the provider default, and any further
rate the deployment itself declares is zeroed alongside the standing set.

The prune is bounded to the deployments a run scanned, but only for a run that priced a
config-declared deployment. Deciding a row is garbage on staleness alone stays correct
while every run derives its charges from the same table, so a database-only run sweeps
exactly as it did before; once one host's charges come from a file the others cannot read,
a row it never considered is not evidence of anything.

Behaviour change worth calling out: a zeroed deployment sorts ahead of an unpriced sibling
in QualityRouter's cost tiebreak, where an unset rate previously sorted last. Reserved
capacity really is the cheaper choice, but the ordering moves.

* refactor(ptu): drop a Final rebind and two redundant isinstance guards

The basedpyright budget rejected reassigning a Final in the datetime coercion and
two isinstance calls the router entry's own type already guarantees. Filtering the
built records rather than the raw entries removes both guards and leaves
_router_deployment as the single validator.
2026-08-19 19:08:14 -07:00
yuneng-jiang
a0f367fcd1
Merge pull request #36897 from BerriAI/litellm_standard_page_header
feat(ui): standardize the Teams page header
2026-08-19 18:52:55 -07:00
devin-ai-integration[bot]
3a04860122
feat(proxy)!: default audit logs on for enterprise licenses (#37518)
* feat(proxy): enable audit logs by premium license

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): support premium audit logging mocks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): disable audit logging for key rotation mocks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 18:49:21 -07:00
yuneng-jiang
0edd245545
fix(ui): render optional array and object MCP tool parameters as JSON inputs (#37548)
* fix(ui): render optional array and object MCP tool parameters as JSON inputs

A Python signature like `tags: list[str] | None = None` serialises to
`{"anyOf": [{"type": "array"}, {"type": "null"}]}` with no top-level
`type`, so the tool test panel's control dispatch fell through to the
generic text input. Whatever the user typed was sent verbatim, and the
server rejected it as the wrong type.

Resolve a property to its single non-null union member before choosing a
control, validating, seeding defaults, and coercing the submitted value,
so all four agree and an optional array or object gets the same JSON
textarea a required one already got.

* fix(ui): keep a null-defaulted optional MCP parameter out of the call

A parameter declared `list[str] | None = None` carries `default: null`,
which means the caller should send nothing. Seeding its editor with an
empty container made the field non-blank, so an untouched parameter was
submitted as `[]` or `{}` instead of being omitted.

Treat an explicit null default as "no value" everywhere it is read: the
editor starts blank and shows its placeholder, and the submitted
arguments leave the key out entirely.
2026-08-19 18:48:43 -07:00
Yuneng Jiang
f99eec5ecb
Merge branch 'litellm_internal_staging' into litellm_standard_page_header
Teams.tsx and Teams.test.tsx both conflicted with staging's antd -> shadcn
migration of the team create form.

Teams.tsx: took staging's rewritten import block and dropped `theme` from the
antd import, since this branch replaced `<Content style={{ padding: token... }}>`
with the Tailwind inset. Dropped both `const { Text } = Typography` (staging
removed its last use) and `const { token } = theme.useToken()` (this branch
removed its last use).

Teams.test.tsx: took this branch's PageHeader-shaped assertions over staging's
older tab-bar lookup, and restored the `within` import that staging had dropped.

Removed the `toHaveClass` snapshot of the antd tab-bar Tailwind classes and the
`.closest(".ant-tabs")` lookup: staging added local/no-antd-class-selectors as a
zero-violation error rule, and those assertions are inert in jsdom anyway. Every
behavioural assertion in that test is unchanged.
2026-08-19 18:45:04 -07:00
yuneng-jiang
2672b36dc3
fix(ui): clear pass-through header rows when the create modal is reopened (#37549)
KeyValueInput and QueryParamInput each seeded a private copy of their rows
from the value prop with a one-time useState initializer. The antd form they
were written for hid that: rc-field-form bumps an internal resetCount key on
resetFields, which remounts a Field's children, so the private copy was thrown
away on every reset. react-hook-form's reset does not remount, and the modal is
hidden rather than unmounted, so after Cancel the rows stayed on screen holding
the old values while the form value went back to empty.

The visible cost was a blocked create flow. A leaked header row made the modal
look configured, but the form value behind it was gone, so submitting a fresh
path and target was refused with "Please configure the headers" and no request
was sent. Typing one character into the leaked row put a value back and the
submit went through, which is not something a user can guess.

Both inputs are now controlled off the value prop, which is an array of pairs
rather than a record. A record cannot represent a row whose name is still empty,
which is the reason the private copy existed: two blank rows collapse into one
and a half-typed row disappears as it is typed. With pairs the field value is
the editable shape, the second source of truth is gone, and a form reset clears
the rows like every other field. add_pass_through converts to a record at submit,
so the request payload is unchanged.

Headers now require at least one row with a non-empty name. Previously that was
enforced by accident, because adding a row did not notify the form at all.
2026-08-19 18:44:01 -07:00
Mateo Wang
8922aaab95
fix(anthropic): log partial stream spend when a /v1/messages client disconnects mid-stream (#37558) 2026-08-19 18:43:46 -07:00
devin-ai-integration[bot]
f5cfa84220
feat(router): allow per-tier litellm_params in complexity autorouter config (#37064)
* feat(router): support complexity tier request params

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): make complexity tier params immutable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(router): simplify complexity tier overlays

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): preserve plain tier config round trips

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): mask tier params in routing decisions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-20 01:39:02 +00:00