* feat(ci): ratchet tests that skip themselves when a credential is absent
* docs(ci): name the new rule where the gate's rules are listed
* fix(ci): require the condition to test for absence before TQ006 fires
* test: settle three allowlist entries that were open questions
The allowlist is meant to hold decisions, not deferrals, so an entry reading
'needs moving' or 'referenced by no job' is a gap wearing an exemption. These
three each get an answer.
The two prompt-factory tests move into the mirror, which is what their own entry
said they needed. Both were passing the whole time, so the 23 tests they hold
start running and the entry goes away rather than getting reworded.
test_aio_http_image_conversion.py is not a test. It fetches live image URLs,
times aiohttp against httpx, prints the ratio, and asserts nothing, and pytest
cannot collect it because its functions take arguments rather than fixtures.
Running it beside its siblings would buy CI a network dependency and a number
nothing reads, so it stays exempt with that written down.
test_litellm_proxy_extras_utils.py stays exempt with a measured reason. 24 of
its 28 tests pass; the 4 in TestMigrationSQLIdempotency fail because nine
migrations from 2026-04 onward use bare CREATE TABLE, ADD COLUMN and CREATE
INDEX where that file requires guarded forms. The convention eroded quietly
precisely because the test enforcing it has never run. Wiring it up is blocked
on what to do about those migrations, and editing them is not the answer, since
Prisma checksums an applied migration and a changed one breaks migrate deploy
for existing installs.
Allowlist entries 10 -> 9, paths 88 -> 86.
* docs(ci): correct the migration count in the proxy-extras allowlist reason
* fix(ui): move the policy flow builder onto theme tokens
The flow builder carried its own private palette: 126 raw literals across a
1644-line file, hardcoded into React inline style objects and SVG presentation
attributes. Inline styles beat every class, so the whole page, its version
sidebar, its step cards and its test panel stayed light no matter what the
theme said.
Each literal now resolves through the token it was already imitating. The greys
map onto card, muted, border, muted-foreground and foreground; the indigo and
blue accents onto info; the pass, fail and API-failure accents onto success,
destructive and warning; and the pale status washes become a color-mix of the
same token so they track it in both themes. Six icons carried their colour as
an SVG presentation attribute, where custom properties do not substitute, so
those switch to currentColor with the token set alongside.
Light mode is not byte-identical, and that is the point: the file stops keeping
a second palette. Of the mappings, card, muted and border land on the exact same
rgb they had, covering most of the file. The rest snap to the dashboard's
canonical shade, which mostly means slightly darker text and deeper status
colours: the gray-400 labels pick up real contrast, the soft red on the fail
icon becomes the destructive red every other failure indicator uses, and the
indigo accent becomes the blue that info resolves to.
Verified in a browser on both themes. In dark mode nothing on the page paints a
light background any more; the six that still do are shadcn's inverted primary
buttons and badges, which are meant to.
* fix(ui): token the flow builder test textarea fill
The quick-chat textarea is the one bare form control left in the file, so the
@tailwindcss/forms base layer still paints it `background-color: #fff`. The
inline style overrode the plugin's border but not its fill, which left a white
box inside the now-dark test panel, and its text inherits the near-white
foreground, so the typed message was invisible in dark mode.
Pin both halves of the pair on the element the plugin styles: the card token it
sits on, and the foreground token it was already inheriting.
* fix(ui): make inline styles and code blocks follow the theme
Two families of colour that a stylesheet never gets to see, so dark mode could
not reach them.
The log details drawer paints most of its chrome through React inline style
objects holding raw hex: #f0f0f0 borders, #fafafa panels, #262626 body text,
the antd-era role accents on message cards, and a green/red guardrail summary
pill. Inline styles win over any class, so the drawer stayed light on a dark
page. Every one of those literals becomes the var(--color-*) it was already
imitating, which costs nothing in light mode and now tracks the theme. The
guardrail pill keeps its layout inline and moves its three colours onto the
success and destructive tokens the rest of the dashboard uses.
The eleven code blocks pass a prism stylesheet as a prop, so the theme has to be
picked in JavaScript. There is no dark-mode toggle in the app yet, only the
`dark` class the design system keys off, so useIsDarkMode subscribes to that
class through useSyncExternalStore and useSyntaxTheme swaps in oneDark when it
is set. Each call site keeps the light stylesheet it already had, including the
two that were relying on the prism default and now name it, so light mode is
unchanged everywhere.
Six of those call sites were casting the stylesheet to `any` or re-declaring its
type to get past the prop signature; the hook returns the right type, so the
casts are gone.
* fix(ui): let the markdown code renderer keep its own syntax theme
The three ReactMarkdown code renderers spread the remaining code element
props after style, so the incoming style attribute widened the prop type
and next build's type check rejected the hook's return value. The old
`coy as any` cast hid the same conflict. Spreading first lets the
explicit props win, which is what every one of these call sites meant.
* test(ui): cover the dark-mode hooks that pick a syntax stylesheet
useIsDarkMode carries the only real logic in this change: an external
store over the root element's class list. Cover the three things that can
regress, the class already being present at mount, the class being
toggled later, and the observer being disconnected on unmount, then cover
useSyntaxTheme handing back the caller's own stylesheet in light mode and
oneDark in dark. The assertions are on which stylesheet object comes
back, by identity, not on any colour it holds.
* refactor(ui): drop the last stylesheet cast in the chat code renderer
This was the one markdown code renderer still spreading the code element
props over its style, so an incoming style attribute would have won over
the theme, and the cast on the spread was what kept that compiling.
Spreading first lets the theme win and the cast go.
* fix(ui): give status colours a readable foreground and drop the muted 70% step
The four status tokens are lightened for dark mode, which is correct when they are used as text
and wrong for the 27 places that use them as a background under `text-white`. Every one of those
passes in light and fails in dark: success 1.78:1, warning 1.72:1, info 2.64:1, destructive
2.89:1. The cause is not 27 authoring mistakes, it is that no `--success-foreground` and no
sibling ever existed, so `text-white` was the only thing available to write. Adding the four
companions and registering them in `@theme` makes the correct pairing expressible, and the call
sites then read `text-success-foreground` instead of a hardcoded colour. Dark lands at 9.98, 10.31,
6.72 and 6.15.
Light is deliberately pure white rather than the near-white the other `-foreground` tokens use, so
the four ratios stay at exactly the 4.95, 5.03, 5.25 and 4.77 they are today instead of drifting
down to 4.73, 4.81, 5.02 and 4.56.
Separately `text-muted-foreground/70` measures 2.75:1 on a light page and 4.31:1 on a dark one,
so the same 183 occurrences fail AA in light and sit under it in dark. Dropping the opacity step
takes them to 4.84:1 and 7.34:1. The identical step on the placeholder base rule goes with them,
which is what put every input's placeholder at 2.75:1 in light.
Residual, not addressed here: `text-muted-foreground` over `bg-muted` reaches 4.39:1 in light,
still short of 4.5. Closing that needs `--muted-foreground` itself to move, which changes every
secondary label in the product and is a design call rather than a defect fix.
* fix(ui): finish the status-foreground swap and repoint no-op muted hovers
Four sites still forced text-white on a status fill because the class sat on
a child element rather than on the filled container, so the earlier sweep did
not reach them. The compliance quick-test bubble was worse: it paired bg-info
with text-success-foreground and its paragraph kept text-white on top, so the
dark-theme contrast the PR set out to fix was still reachable there
Dropping the /70 step also turned 21 existing "text-muted-foreground/70
hover:text-muted-foreground" pairs into hovers that change nothing, which
local/no-noop-hover-variant flags as an error. The affordance was "brighten on
hover", so these now hover to text-foreground, matching the 74 places that
already spell it that way
The remaining churn is prettier reflowing the handful of lines whose length
changed, since the token names are longer than text-white
* fix(ui): let the approve/reject confirm button pick the token its fill uses
Both submission review dialogs put text-success-foreground on the shared
button class while the fill below it swings between bg-success for Approve and
bg-destructive for Reject, so Reject drew a success token over a destructive
fill. The two tokens resolve to the same value today, so nothing looks wrong,
but the pairing only holds by coincidence and would break the moment either
token moves. Moving the token into the branch makes it track the fill
* fix(ui): drop the last 70% placeholders, still live on the legacy utility
Four inputs spell their placeholder colour with Tailwind's older
placeholder-<colour> utility rather than placeholder:text-<colour>, so the
sweep that dropped the 70% step passed over them. Tailwind 4.3 still emits
that utility, and utilities sit after base in the layer order, so those four
kept overriding the new input::placeholder rule and kept rendering at 70% in
dark mode, which is the contrast failure this PR set out to close
They now spell it the same way as the three placeholders the PR already
converted, which both removes the step and settles on one spelling
* fix(ui): make dark-mode form controls visible
Two dark-mode defects left form controls without any visual boundary or fill.
`--input` and `--border` share one value in `.dark`, oklch(0.309), which resolves to
rgb(48,48,48). Against `--background` (33) that is a 15-step stroke, and against `--popover` (42)
it collapses to 6 steps out of 255, so a control inside any dialog is effectively undrawn. The
controls also use `bg-transparent`, so there is no fill cue either and only the placeholder text
renders. Measured 1.09:1 against the dialog surface where WCAG 1.4.11 asks for 3.0:1 on the
boundary of a user interface component. Splitting `--input` off at oklch(0.56) restores 3.07:1
without touching `--border`, which stays where it is because it draws decorative separators rather
than control boundaries. 91 controls across 19 routes were measured at the collapsed value, every
one with an identical stroke and surface, so a single token covers all of them.
Separately, `@tailwindcss/forms` paints a white fill on every bare control. The block above
already neutralises that for `combobox-chip-input`, but its audit covered `components/ui` only,
and hand-rolled controls elsewhere still render white on a dark page: typed text lands at 1.11:1
and native selects at 2.19:1 on `/model-hub-table`, `/playground`, `/guardrails`, `/mcp-servers`
and `/models-and-endpoints`. Tracking `--background` fixes those at 14.51:1 and 7.34:1.
Light mode is unchanged by both. The token edit is scoped to `.dark`, and `--background` in
`:root` is the same white the plugin was already painting, verified control-by-control on a dev
server: backgrounds stay rgb(255,255,255) and ratios stay 20.13:1 and 4.84:1.
* fix(ui): keep the combobox chip input transparent under the bare-control fill
The new base rule matched at (0,2,1) while the combobox chip-input override
sits at (0,1,0), so ComboboxChipsInput lost its transparent background and
painted an opaque page-colored rectangle inside the chips container, which
carries its own bg-transparent / dark:bg-input/30 fill.
Folding the exclusions into one :not() list adds the chip input and drops the
selector to (0,1,1). Every @tailwindcss/forms base selector is wrapped in
:where(), so it lands at (0,0,1); (0,1,1) still outweighs it and bare inputs,
textareas and selects keep the fill this PR gives them.
Stacked on the multi-key shadow eval backend. The key picker becomes a
paginated multi-select with chips, built on the base-ui combobox chips
primitives, with the pagination and debounced-search logic extracted into a
shared usePaginatedCombobox hook that PaginatedSearchSelect now also uses.
The detail view gains a per key table showing each key's own status, judged
turns against its budget, and win rates from the by_key slice, and the job
headline pluralises to "N keys" for multi-key jobs
tests/proxy_unit_tests had a 30-line YAML parser inlined in its workflow that
failed the run when a test file there belonged to no shard. tests/test_litellm
is sharded the same way, with no catch-all bucket, and had no such guard: a new
directory under it (or under its proxy subtree) is collected by nothing and runs
nowhere, and the coverage census cannot see it because a token like
tests/test_litellm/test_*.py already answers 'yes, that tree runs'.
The two questions differ. The census asks whether a file runs at all, so an
ancestor path standing in for everything beneath it is a fine answer. Shard
assignment asks which shard owns a child, and there that same ancestor path is
precisely the bug. _token_covers keeps the first meaning; _token_names adds the
second, and the guard now walks a list of sharded trees rather than one hardcoded
directory. Both read the same test-path keys, so there is one workflow parser.
A directory needs a shard when it holds a test file, not when it is named test_*.
That drops the hardcoded test_configs exception and keeps fixture directories
like expected_fine_tuning_api out on their own merits.
The job keeps its name and its workflow, since assert-shard-coverage is a
required status check on litellm_internal_staging.
Verified red-first: a planted directory under tests/test_litellm, a planted
directory under tests/test_litellm/proxy, and a planted file under
tests/proxy_unit_tests each fail the guard, while a fixture-only directory does
not. 327 children across the three trees are assigned today.
Python binds a name once per scope, so when a module or class defines the same
test twice only the last one exists. The earlier definitions are unreachable:
pytest never collects them, and nothing that references them can fail.
A sweep in August cleared nine of these. Five have appeared since, which is the
argument for a rule rather than another sweep.
Each survivor is the better version, so nothing is lost. The two SQS logger
twins additionally stub `asyncio.create_task`, which the shadowed copies did
not. The cost-calculator duplicate is a two-line stub that also takes a
`model_item` parameter no fixture supplies, so it could not have run even
unshadowed. The two `test_prompt_caching` bodies are both `pass`.
Collecting the four files reports 416 tests before and after.
`tests/proxy_unit_tests/conftest copy.py` goes with them. pytest only loads a
file named exactly `conftest.py`, nothing imports this one, and the space in the
name says what it was.
* feat(ci): ratchet the test suite's zero-assert, mock-echo and global-state debt
The suite's dominant failure mode is tests that cannot fail for the reason anyone
would want them to. The testing-strategy audit measured five shapes of it, and
nothing mechanical stops any of them from reproducing, so they keep reproducing.
`scripts/check_test_quality.py` is an AST checker for those five, emitting the
same `path:line: CODE message` contract as `scripts/check_type_discipline.py`:
TQ001 a collectible test with no assertion of any kind
TQ002 mock-echo, where every assertion only inspects the mock that was patched
TQ003 sys.path.insert inside the test tree
TQ004 raw `os.environ[...] =`, which leaks into whatever runs next
TQ005 `litellm.<attr> =`, the process-wide leak the 491-line conftest undoes
`scripts/test_quality_gate.py` caps each rule against test-quality-budget.json,
seeded at exactly today's count, and fails only when a rule is both over its
limit and higher than the base being merged into, so a change is blamed for what
it adds and never for drift already in the base. `--update` lowers a limit by
what a branch cleared, so the ceilings only ever fall. It runs in the existing
required lint job, which means it enforces without a ruleset change.
TQ001 follows assertions into helpers defined in the same module, transitively.
Without that it flagged 111 tests in tests/e2e, the harness this program holds up
as the reference, because that suite factors its assertions into shared helpers
(`assert_auth_denied(result, ...)`). Following them leaves 25, all of which reach
their assertions across a module boundary; those are grandfathered and documented
rather than papered over.
The seeded counts land within about 10% of the audit's independent numbers for
every rule measured on the same subtree, which is the cross-check that the
definitions here match the ones the audit pinned.
* fix(ci): resolve test helpers per scope, not by bare name
The helper walk keyed every function in a module by its bare name, so two
same-named helpers in different classes collided and the last one parsed won.
A test calling `self._check()` could be cleared by a `_check` belonging to a
different class, or flagged because of one.
Resolution is now scoped: a bare name looks up the module-level functions, and
`self.<name>` looks up the enclosing class's own methods and no other class's.
Recursion is tracked by function identity rather than by name, so the cycle
guard cannot be confused by the same collision.
This surfaced one real zero-assert test that a same-named helper elsewhere had
been clearing, so TQ001 seeds at 750 rather than 749.
The test module has to register itself in sys.modules before exec_module:
`@dataclass(slots=True)` rebuilds its class through `sys.modules[__module__]`,
and Scope fails to construct without it. Recorded at the call site, since it
reads like avoidable global mutation otherwise.
* fix: register test-quality-budget.json with the ratchet alarm
The repo keeps one census over its budget files: every *-budget.json on disk
must appear in DEFAULT_BUDGETS, or its ceilings can be raised with no signal.
tests/test_litellm/test_budget_ratchet_check.py asserts that set equality and
caught the new budget on the way in.
Registering it also turns the alarm on for TQ001-TQ005, so a later PR cannot
quietly raise a test-quality ceiling. The file already uses the {limit: N}
schema the ratchet reads, so no other change was needed.
* test: retire tests/old_proxy_tests, which holds no tests
Twenty files named test_*.py, and pytest collects nothing from any of them:
uv run pytest tests/old_proxy_tests --collect-only -q
no tests collected, 16 errors in 114.17s
They are manual snippets against a running proxy, written at module level with
no test function, no assertion and no entry point, so the only thing the name
buys them is a place on the coverage allowlist. Sixteen of the twenty cannot
even be imported in this environment, wanting langchain, llama_index or
google.api_core, and ten still point at 0.0.0.0:8000, which stopped being the
proxy's default port some time ago.
Nothing outside the directory refers to it apart from the allowlist entry, which
goes with it. The other loose contents go too: five load_test_*.py scripts, a
bursty variant, two committed log files, an essay fixture and a stray .js
snippet.
Allowlist paths 88 -> 68, test files 2422 -> 2402, and no job loses anything it
was running. Recoverable from history if a snippet turns out to be someone's
habit.
* test: drop the retired old_proxy_tests paths from the coverage allowlist
The agent job's CircleCI glob collected `tests/agent_tests/**/test_*.py` and then
piped it through `grep -v` to drop `local_only_agent_tests/`. `assert_ci_coverage.py`
reads the glob but not the pipeline, so those two files looked covered and were
invisible to the census. The glob now excludes them structurally and they carry an
allowlist entry instead, which is a decision on the record rather than a hidden
filter. The collected file set is unchanged: `tests/agent_tests/` holds exactly one
CI-runnable test at the top level.
`tests/scim_tests/` held a single JSON fixture and no tests, referenced from nowhere.
`.github/workflows/` is for workflows. Both stray scripts move to `.github/scripts/`
with their callers updated: the price-file updater is invoked by
`auto_update_price_and_context_window.yml`, and the translation-report runner by
`make test-llm-translation`. The audit listed the latter as orphaned, but Makefile
line 317 still runs it, so it moves rather than being deleted.
The rollout heads-up workflow was a deliberate one-shot for the agent-shin rollout.
That rollout is done, the triage and auto-close workflows have been running daily
since June, so the pre-flip warning window is long past. Its script and dedicated
test go with it, and the sibling workflow-invariant test drops its entry.
Nine workflow files existed only to make a single call to _test-unit-base.yml
with a different test-path. Adding a shard meant adding a file; changing
anything shared meant editing nine. One matrix caller replaces them, so a shard
is now one entry.
Check names are unchanged, which is the whole constraint. A reusable-workflow
job reports as "<job name> / <inner job name>", so setting `name` to the shard id
alone reproduces today's context strings exactly: the eleven the matrix produces
are eleven of the twenty-three "/ Run tests" contexts the branch ruleset
requires, matched string for string. No ruleset edit is needed and none should
be made for this.
Every matrix entry states its timeouts even where they equal the base defaults.
An absent matrix key renders as an empty string rather than falling back, and an
empty string is not a number, so a partially-specified entry would fail the call.
tests/proxy_unit_tests keeps test-unit-proxy-db.yml. It is already a matrix and
its shard-coverage guard reads that file by name, so folding it in belongs with
generalising that guard into assert_ci_coverage.py rather than here. Its twelve
shards are the remaining required contexts.
test-unit-documentation.yml stays too: it does not call the base workflow.
A 503 from POST /revoke means the proxy could not write the single-use record, so clearing the local record left a live refresh token nobody could revoke and a hint to retry with nothing left to retry. lite logout now keeps the record, exits 1, and asks to be run again shortly. A refused or unreachable revocation still clears the record and warns as before, and a re-login that replaces a record keeps its existing warning because the new record already stands
revoke_refresh_token discarded the single-use claim result, so a revocation that arrived while Redis was unreachable answered 200 and left the refresh token live. The token endpoint reported the same outage as invalid_grant "already used". The guard now reports first, replayed, or unavailable, and both endpoints answer 503 temporarily_unavailable for an outage (RFC 7009 section 2.2.1, RFC 6749 section 5.2), which the CLI surfaces as a one-line warning while keeping the key it has
The lite group resolves the stored key once for every command and renews a --pkce key on the way in. lite up then asked the token file again, so every start sent a second refresh to the proxy, and once the refresh token was burned the refusal printed twice. _ensure_fresh_login now reuses the key the group resolved when the group read it from the token file, and only re-reads the file after the interactive login it starts itself. Also covers print-token through the group with a renewing session in the tests
The cli group already resolves the stored key for the server it was pointed
at, so print-token re-ran the renewal and, when the refresh token had been
revoked, posted to /token twice and printed the reason twice. print-token now
reuses the group's result whenever the stored record was issued for that
server and no --api-key or LITELLM_PROXY_API_KEY took precedence, and only
resolves the key itself when invoked bare for a different server.
A PKCE credential whose renewal is refused (for example after lite logout ran
on another copy of it) used to fail lite auth print-token with the classic
'Token expired. Run lite login again' hint and no reason, while lite whoami
already named lite login --pkce. fresh_api_key now reports why a renewal
failed through a warn callback whenever no sibling rotation rescued it, the
CLI prints that reason on stderr, and the expiry hint names the command that
produced the credential. Both READMEs document the admin revocation semantics
and the Redis precondition for refresh single use on several workers.
Discovery checks that every endpoint sits on the proxy origin, but the CLI's
requests.Session followed redirects, and requests replays a POST body on
307 and 308, so a token or revocation endpoint answering with one of those
would have sent the code and verifier, or the refresh token, wherever
Location pointed. Every POST now goes out with allow_redirects=False and a
3xx answer fails the command with a message naming where it pointed
A proxy on the default remote cost map never produced a prompt cache
breakpoint: the published map has the gpt-5.6 entries without
supports_prompt_cache_breakpoint, so the model-map gate returned False
for every listed model and only LITELLM_LOCAL_MODEL_COST_MAP=True (the
repo .env, hence the passing unit tests) made the feature work. The hook
now honors the flag when the entry carries one, True or False, and
otherwise applies the GPT-5.6+ version rule to the model name, so a map
that lags the flag still gets the OpenAI dialect. The model-map tests
pin litellm.model_cost to the bundled backup map and a new test drives
the hook against an unflagged gpt-5.6 entry.
completion() and acompletion() take base_url as an alias for api_base
that only lands on api_base after the cache control hook ran, so a
GPT-5.6 call at a non-OpenAI gateway given through base_url still got
the dialect. Both seed calls and the unstamped request-params read now
look at base_url too.
ResponsesAPIRequestUtils.merge_prompt_management_input reshaped hook
output in place, retyping text parts to input_text on the caller's own
message objects. The merge now shapes a copy of each message as it
emits it, so the identity-based merge keeps working on the hook's
objects and nothing the hook or the client owns is mutated.
The consent page offers the team picker, but a form posted without a team
sealed a teamless grant and the token endpoint minted an unscoped
credential for a team member, escaping the team attribution classic lite
login always applies. The minter now refuses such a grant on redemption
and refresh alike; memberships whose team rows are gone still count as no
team so they cannot lock a user out
The cache control hook also runs on litellm.responses() input. On a
GPT-5.6 deployment it wrapped a string-content item into a chat-shaped
{"type": "text"} part, which the Responses API rejects, and it never
marked input_text, input_image or input_file parts, so no breakpoint and
no prompt_cache_options reached the provider. Add the Responses part
types to the eligible block set and translate chat-shaped text parts on
non-assistant items to input_text in
ResponsesAPIRequestUtils.merge_prompt_management_input, which both the
async and the sync prompt management sites go through.
The dialect also fired for any GPT-5.6 name that resolved to provider
openai, including deployments pointed at a custom api_base that does not
understand prompt_cache_breakpoint. Decide it once per request from the
provider, the model map and the resolved api_base (request, then
litellm.api_base, then OPENAI_BASE_URL / OPENAI_API_BASE): only
api.openai.com and *.api.openai.com hosts speak the dialect, a top-level
prompt_cache_options opts a custom target in, and litellm_proxy/ targets
never get it. maybe_seed_default_injection_points takes api_base and
stamps the finished decision on the points as _litellm_openai_dialect so
the sync completion() path, whose hook params do not carry api_base,
honors it; maybe_inject_cache_control takes api_base from the
/v1/messages handler.
Eligibility now comes from a supports_prompt_cache_breakpoint model map
flag on the OpenAI gpt-5.6 entries, exposed through
litellm.utils.supports_prompt_cache_breakpoint, with the GPT version rule
kept only for models the map does not know. The OpenAI dialect no longer
reserves a slot for tool_config points, which OpenAI has no cache block
for, and with_prompt_cache_breakpoint plus the chat bridge helper return
a new block instead of mutating their input.
The discovery document is accepted only when its issuer is the --base-url the user
typed and every endpoint and the resource share that origin (RFC 8414 section 3.3),
so a tampered or redirected document can no longer point the code, verifier, or
refresh token at another host. After a failed refresh the re-read token record is
used only when it continues the same credential (same proxy, token endpoint, and
resource) and has not expired, so a concurrent login against a different proxy can
never hand this one its key
When the resolved deployment is provider openai and the model is GPT-5.6 or
newer, the cache control hook now writes prompt_cache_breakpoint on the
targeted content block and sets prompt_cache_options to explicit mode unless
the caller already passed one. The /v1/messages bridges carry the marker
through (the Responses bridge moves a marked system prompt into a developer
message, since top-level instructions cannot hold one). Breakpoint counting
and the stand-down check recognise both marker kinds, and client breakpoints
already present in messages are no longer subtracted from the cap twice.
Fixes#37509
The proxy's OAuth authorization server (dynamic registration, PKCE S256,
loopback redirects, single-use codes, refresh rotation) gains a proxy-API
audience: /authorize?resource=<proxy origin> renders a consent page with
team selection and /token mints the same per-user credential lite login
mints, so a native CLI can sign a user in through the system browser and
call /v1/* with user and team attribution. Adds GET /.well-known/litellm-cli-auth
as the versioned discovery contract for non-Python clients, POST /revoke
(RFC 7009) for logout, and lite login --pkce, lite logout, and
lite auth print-token on the CLI side. Proxy-API grants only ever redirect
to a loopback address and the server never picks a team on the user's behalf.
Fixes#37332
Both /v1/messages bridges (Responses API adapter for openai/* and the
chat-completions adapter) now derive prompt_cache_key from the first 64
characters of metadata.user_id, next to the existing user mapping. The
chat bridge only sets it when the resolved provider advertises
prompt_cache_key in its supported params, so providers that reject
unknown params are unaffected. A prompt_cache_key sent explicitly by the
client always wins over the derived value.
Fixes#37508
The forwarded-keys record moves off the shared metadata dict onto the
per-request kwargs, where _update_kwargs_with_deployment consumes it, so
sibling requests that share a metadata dict (abatch_completion) can no
longer clear it mid-routing. Per-tier litellm_params from the hook
response are never treated as forwarded marker params, and a deployment
only beats a forwarded value when it sets its own, not when it carries a
LiteLLM_Params default such as merge_reasoning_content_in_choices=False.
- accept the Live SDK's models/<id> and LiteLLM's vertex_ai/<id> when rewriting the setup model
- keep a dict service account intact instead of stringifying it
- treat same-target deployments holding different credentials as ambiguous
- guard both websocket states before every close so a second close cannot raise
- build the sendable close codes from the public CloseCode enum
An `auto_router/<alias>` marker entry's litellm_params (for example
`aws_region_name: eu-west-3`) were forwarded onto every routed call with
setdefault and then won the `{**litellm_params, **kwargs}` merge against
the selected tier's own values, so a Bedrock tier pinned to us-east-1 was
called in eu-west-3 and failed with 400.
The hook now records which keys it actually forwarded on the request's
metadata bucket, and `_update_kwargs_with_deployment` drops every
forwarded key the selected deployment defines itself, so marker params
only fill gaps a tier leaves open. Request-supplied values still win over
both. The stamp is stripped from logged metadata like its siblings.
Fixes#37613
On models without supports_mid_conversation_system, a system entry between
an assistant tool_use turn and the user tool_result turn became a user turn
in that position and the provider rejected the request ("tool_use ids were
found without tool_result blocks immediately after"). That run of entries
now goes right after the tool_result turn, where consecutive user turns
merge upstream. The converted turn also carries only role and content, as
the hoist did, so an entry with extra keys no longer 400s with "Extra
inputs are not permitted".
The e2e cache priming re-sends the identical first turn until its own cache
entry reads back before the reminder turn goes out, since Vertex can take a
few seconds to serve a freshly written entry.
Take the resolver and its timeout as parameters of the bounded helper and
bind the vertex one once at module level, so the timeout tests drive an
injected fake instead of patching a shared singleton.
