* fix(ui): draw one Per Day savings bar per date on Cost Optimization
The page paged /user/daily/activity over raw rows, so a date spanning
pages arrived N times with partial metrics and rendered as N thin bars.
Switch to the single-shot aggregated endpoint, thread
include_current_utc_day through it to keep the live-end extension from
PR #36051, and merge the paginated fallback by date.
* fix(ui): keep aggregated call at four params and mock it in view tests
Trailing userId and includeCurrentUtcDay ride a named rest tuple so the
eslint max-params baseline stays at 23, and the CostOptimizationView
suites mock the new networking export their render now reaches.
* fix(ui): make hardcoded palette surfaces theme-aware
Twenty-one dashboard files painted fills from the raw Tailwind palette with no
dark counterpart, so in dark mode they rendered as near-white islands carrying
dark text: unreadable. The route sweep caught them on teams, access-groups,
policies, users, skills, guardrails-monitor, logs, compliance, playground,
fallbacks and the AI hub.
Where the hue already had a semantic token, the surface moves onto it. Every one
of these lines had a token on its border and a palette class on its fill, so
this finishes a migration that had stalled halfway: bg-blue-50 next to
border-info/20 becomes bg-info, bg-gray-50 becomes bg-muted, DocLink's bg-white
becomes bg-card, and the Alert error variant drops text-red-800 and text-red-600
for the destructive token its sibling variants already use.
Purple, violet and indigo have no token in the system, which is exactly why the
maps in PluginTableColumns, GuardrailsOverview, AccessGroupsTableColumns and
teamTableColumns had migrated every other entry and left those behind. Rather
than mint a brand token here, they take the dark palette step, matching what
TeamGuardrailsTab, add_agent_form, MCPToolsetsTab and mcp_connect already do.
Gradient stops get the same treatment since bg-linear stops have no token form.
Light mode is unchanged apart from the four surfaces that moved onto a token,
and those stay inside the same colour family.
The #1e1e1e code slabs in guardrail_info and CustomCodeModal are deliberately
left alone: they are intentionally dark editors in both themes, and their
gray-200 text stays legible either way.
* fix(ui): give dark surfaces a readable foreground step
The dark fills added for the purple and indigo surfaces left three nested
foregrounds on their original light-palette step, so the text and icon sitting
on those new fills dropped below readable contrast in dark mode.
text-purple-800 on purple-950 measured 1.72:1, text-indigo-600 on indigo-950
2.54:1, and text-purple-600 on the blue-950 gradient stop 2.73:1. Each now
takes the purple-300 / indigo-300 step this PR already uses elsewhere, which
lands them at 8.48:1, 8.02:1 and 8.31:1.
The pricing calculator renders the same cost expression twice, so both copies
move together rather than leaving one half-migrated.
* fix(ui): keep the guardrail chip remove button visible on hover
The chip itself moved to the dark indigo fill, but its remove button still
darkened to indigo-900 on hover, which against indigo-950 measures 1.40:1 and
makes the X vanish under the cursor in dark mode.
Dark mode now brightens to indigo-100 on hover instead, mirroring the light
theme where hover darkens away from the resting colour.
* test: run the 30 test files stranded in the second mirror
tests/litellm sat beside tests/test_litellm, which is the mirror the repo
convention names, and no job collected it. The allowlist called the directory
unresolved and assumed it was a duplicate. It is not: 30 of its 34 files have no
counterpart in the real mirror, so they are tests nobody has run since they were
written, not copies of tests that run elsewhere.
Moving them in is byte-identical, and it is what makes them run. Every one is
now claimed by a shard's test-path rather than by an allowlist entry, and the
216 tests they hold pass. Directories that needed to become packages did, since
several files are named test_transformation.py and pytest cannot import two of
those from non-package directories in one session.
Never running is why three assertions had drifted away from the code:
* nvidia.nemotron-super-3-120b max_output_tokens, 32000 -> 32768
* sambanova/MiniMax-M2.7 max_input_tokens, 204800 -> 196608
* the Vertex text-to-speech handler moved from data= to json=, so the test
reads the decoded body off the json kwarg instead of parsing the data one
The first two follow model_prices_and_context_window.json, which the catalog
sync keeps current; the third follows the handler. In all three the test was the
stale side.
The lint workflow ran test_no_hardcoded_secrets.py by path and now points at the
new one.
Four files stay behind. Each shares a filename with a live test whose contents
are disjoint from it, so landing those means merging test bodies, which is a
content review rather than a move. The allowlist entry now names those four and
records how many tests each would bring, in place of calling the whole
directory unresolved.
* fix(ci): keep the secret scan out of the mirror's conftest
The secret-scan job runs pytest under uv run --no-project, so its environment
holds pytest and nothing else. That worked while the file sat in tests/litellm,
which has no conftest, and broke the moment it moved into tests/test_litellm,
whose conftest imports litellm on collection: ModuleNotFoundError: No module
named 'dotenv', before a single test ran.
The file is a repo-wide static scan that imports only base64, os, re and pytest,
so it belongs with the other repo-wide checks in tests/code_coverage_tests,
which has no conftest, rather than in the package mirror. Installing the full
dependency set into a 15-second job to satisfy a conftest it does not use would
be the wrong trade.
Verified with the job's exact command:
uv run --no-project --with 'pytest==9.0.2' pytest \
tests/code_coverage_tests/test_no_hardcoded_secrets.py -q
1 passed in 0.47s
* feat(ci): catch files a -k expression deselects from every job
The coverage census asks whether some job names a file. It cannot ask what that
job's -k then does with it, and the gap is not hypothetical: tests/local_testing
is globbed by five jobs, two of which carry
-k "... and not router and not assistants and not langfuse and not caching and not cache"
while the other three keep one keyword each. Any file whose path holds an
excluded term is dropped by the first two and matched by none of the rest, so it
runs nowhere while the census counts it as covered. 118 tests across eight
caching files sit in exactly that hole today.
The new mode reads the same CircleCI jobs the census already parses and asks
whether each globbed file survives its job's selector. Two facts about -k make
that decidable without running pytest: it matches an item's own name and its
parents', so a term appearing in the module path deselects the whole file; and
the names it can match are otherwise the classes and functions in the file,
which ast reads. A positive term is therefore satisfied by the path or by a name
inside, which is what keeps a langfuse-named test inside test_logging.py from
being reported.
Where the parser is unsure it stays quiet. An expression with or, parentheses,
or a negated group is left unmodelled and its job is treated as claiming
everything it globs, so an unparsed selector can never raise a false alarm.
Glob translation learned character classes, without which
tests/local_testing/**/test_[a-mA-M]*.py matches nothing and the guard would
report that whole directory. The census and shard counts are unchanged by it,
2423 files and 327 shard children before and after.
Validated against the real thing: collecting tests/local_testing under each
job's own selector leaves 175 of 1577 tests unselected, in exactly the ten files
this check derives statically, no more and no fewer. Two of the ten are named
outright by other jobs, which the check credits, leaving the eight now recorded
in the allowlist as a decision rather than an accident.
Verified red-first: dropping one of those eight from the allowlist reports it,
and adding 'and not embedding' to the two part jobs reports test_embedding.py
and test_get_optional_params_embeddings.py.
* fix(ci): keep the slice guard from pairing one command's -k with another's glob
Two accuracy notes from review, both about the parser's model rather than its
current verdicts.
A job that runs several pytest commands offers no way to tell which glob a -k
belongs to, since both are read out of the same flattened job text. Combining
them could pair one command's exclusion with another command's glob and report a
file that in fact runs. Such a job is now left unmodelled, which means it claims
everything it globs, matching how the parser already treats an expression it
cannot read. Only one job in the config has two globs today and it carries no
-k at all, so no verdict changes.
The second is a deliberate limit, now stated where it lives: an excluded term is
only honoured when it sits in the module path, because that is the case that
takes the whole file with it. A term matching one function inside drops that
test and leaves the file running, and reporting it would be a false alarm.
Answering per-test instead would need a baseline of test ids that churns on
every rename, for a smaller failure than a file going dark.
Both are pinned by tests.
* feat(ci): ratchet tests that skip themselves when a credential is absent
* docs(ci): name the new rule where the gate's rules are listed
* fix(ci): require the condition to test for absence before TQ006 fires
* test: settle three allowlist entries that were open questions
The allowlist is meant to hold decisions, not deferrals, so an entry reading
'needs moving' or 'referenced by no job' is a gap wearing an exemption. These
three each get an answer.
The two prompt-factory tests move into the mirror, which is what their own entry
said they needed. Both were passing the whole time, so the 23 tests they hold
start running and the entry goes away rather than getting reworded.
test_aio_http_image_conversion.py is not a test. It fetches live image URLs,
times aiohttp against httpx, prints the ratio, and asserts nothing, and pytest
cannot collect it because its functions take arguments rather than fixtures.
Running it beside its siblings would buy CI a network dependency and a number
nothing reads, so it stays exempt with that written down.
test_litellm_proxy_extras_utils.py stays exempt with a measured reason. 24 of
its 28 tests pass; the 4 in TestMigrationSQLIdempotency fail because nine
migrations from 2026-04 onward use bare CREATE TABLE, ADD COLUMN and CREATE
INDEX where that file requires guarded forms. The convention eroded quietly
precisely because the test enforcing it has never run. Wiring it up is blocked
on what to do about those migrations, and editing them is not the answer, since
Prisma checksums an applied migration and a changed one breaks migrate deploy
for existing installs.
Allowlist entries 10 -> 9, paths 88 -> 86.
* docs(ci): correct the migration count in the proxy-extras allowlist reason
* fix(ui): move the policy flow builder onto theme tokens
The flow builder carried its own private palette: 126 raw literals across a
1644-line file, hardcoded into React inline style objects and SVG presentation
attributes. Inline styles beat every class, so the whole page, its version
sidebar, its step cards and its test panel stayed light no matter what the
theme said.
Each literal now resolves through the token it was already imitating. The greys
map onto card, muted, border, muted-foreground and foreground; the indigo and
blue accents onto info; the pass, fail and API-failure accents onto success,
destructive and warning; and the pale status washes become a color-mix of the
same token so they track it in both themes. Six icons carried their colour as
an SVG presentation attribute, where custom properties do not substitute, so
those switch to currentColor with the token set alongside.
Light mode is not byte-identical, and that is the point: the file stops keeping
a second palette. Of the mappings, card, muted and border land on the exact same
rgb they had, covering most of the file. The rest snap to the dashboard's
canonical shade, which mostly means slightly darker text and deeper status
colours: the gray-400 labels pick up real contrast, the soft red on the fail
icon becomes the destructive red every other failure indicator uses, and the
indigo accent becomes the blue that info resolves to.
Verified in a browser on both themes. In dark mode nothing on the page paints a
light background any more; the six that still do are shadcn's inverted primary
buttons and badges, which are meant to.
* fix(ui): token the flow builder test textarea fill
The quick-chat textarea is the one bare form control left in the file, so the
@tailwindcss/forms base layer still paints it `background-color: #fff`. The
inline style overrode the plugin's border but not its fill, which left a white
box inside the now-dark test panel, and its text inherits the near-white
foreground, so the typed message was invisible in dark mode.
Pin both halves of the pair on the element the plugin styles: the card token it
sits on, and the foreground token it was already inheriting.
* fix(ui): make inline styles and code blocks follow the theme
Two families of colour that a stylesheet never gets to see, so dark mode could
not reach them.
The log details drawer paints most of its chrome through React inline style
objects holding raw hex: #f0f0f0 borders, #fafafa panels, #262626 body text,
the antd-era role accents on message cards, and a green/red guardrail summary
pill. Inline styles win over any class, so the drawer stayed light on a dark
page. Every one of those literals becomes the var(--color-*) it was already
imitating, which costs nothing in light mode and now tracks the theme. The
guardrail pill keeps its layout inline and moves its three colours onto the
success and destructive tokens the rest of the dashboard uses.
The eleven code blocks pass a prism stylesheet as a prop, so the theme has to be
picked in JavaScript. There is no dark-mode toggle in the app yet, only the
`dark` class the design system keys off, so useIsDarkMode subscribes to that
class through useSyncExternalStore and useSyntaxTheme swaps in oneDark when it
is set. Each call site keeps the light stylesheet it already had, including the
two that were relying on the prism default and now name it, so light mode is
unchanged everywhere.
Six of those call sites were casting the stylesheet to `any` or re-declaring its
type to get past the prop signature; the hook returns the right type, so the
casts are gone.
* fix(ui): let the markdown code renderer keep its own syntax theme
The three ReactMarkdown code renderers spread the remaining code element
props after style, so the incoming style attribute widened the prop type
and next build's type check rejected the hook's return value. The old
`coy as any` cast hid the same conflict. Spreading first lets the
explicit props win, which is what every one of these call sites meant.
* test(ui): cover the dark-mode hooks that pick a syntax stylesheet
useIsDarkMode carries the only real logic in this change: an external
store over the root element's class list. Cover the three things that can
regress, the class already being present at mount, the class being
toggled later, and the observer being disconnected on unmount, then cover
useSyntaxTheme handing back the caller's own stylesheet in light mode and
oneDark in dark. The assertions are on which stylesheet object comes
back, by identity, not on any colour it holds.
* refactor(ui): drop the last stylesheet cast in the chat code renderer
This was the one markdown code renderer still spreading the code element
props over its style, so an incoming style attribute would have won over
the theme, and the cast on the spread was what kept that compiling.
Spreading first lets the theme win and the cast go.
* fix(ui): give status colours a readable foreground and drop the muted 70% step
The four status tokens are lightened for dark mode, which is correct when they are used as text
and wrong for the 27 places that use them as a background under `text-white`. Every one of those
passes in light and fails in dark: success 1.78:1, warning 1.72:1, info 2.64:1, destructive
2.89:1. The cause is not 27 authoring mistakes, it is that no `--success-foreground` and no
sibling ever existed, so `text-white` was the only thing available to write. Adding the four
companions and registering them in `@theme` makes the correct pairing expressible, and the call
sites then read `text-success-foreground` instead of a hardcoded colour. Dark lands at 9.98, 10.31,
6.72 and 6.15.
Light is deliberately pure white rather than the near-white the other `-foreground` tokens use, so
the four ratios stay at exactly the 4.95, 5.03, 5.25 and 4.77 they are today instead of drifting
down to 4.73, 4.81, 5.02 and 4.56.
Separately `text-muted-foreground/70` measures 2.75:1 on a light page and 4.31:1 on a dark one,
so the same 183 occurrences fail AA in light and sit under it in dark. Dropping the opacity step
takes them to 4.84:1 and 7.34:1. The identical step on the placeholder base rule goes with them,
which is what put every input's placeholder at 2.75:1 in light.
Residual, not addressed here: `text-muted-foreground` over `bg-muted` reaches 4.39:1 in light,
still short of 4.5. Closing that needs `--muted-foreground` itself to move, which changes every
secondary label in the product and is a design call rather than a defect fix.
* fix(ui): finish the status-foreground swap and repoint no-op muted hovers
Four sites still forced text-white on a status fill because the class sat on
a child element rather than on the filled container, so the earlier sweep did
not reach them. The compliance quick-test bubble was worse: it paired bg-info
with text-success-foreground and its paragraph kept text-white on top, so the
dark-theme contrast the PR set out to fix was still reachable there
Dropping the /70 step also turned 21 existing "text-muted-foreground/70
hover:text-muted-foreground" pairs into hovers that change nothing, which
local/no-noop-hover-variant flags as an error. The affordance was "brighten on
hover", so these now hover to text-foreground, matching the 74 places that
already spell it that way
The remaining churn is prettier reflowing the handful of lines whose length
changed, since the token names are longer than text-white
* fix(ui): let the approve/reject confirm button pick the token its fill uses
Both submission review dialogs put text-success-foreground on the shared
button class while the fill below it swings between bg-success for Approve and
bg-destructive for Reject, so Reject drew a success token over a destructive
fill. The two tokens resolve to the same value today, so nothing looks wrong,
but the pairing only holds by coincidence and would break the moment either
token moves. Moving the token into the branch makes it track the fill
* fix(ui): drop the last 70% placeholders, still live on the legacy utility
Four inputs spell their placeholder colour with Tailwind's older
placeholder-<colour> utility rather than placeholder:text-<colour>, so the
sweep that dropped the 70% step passed over them. Tailwind 4.3 still emits
that utility, and utilities sit after base in the layer order, so those four
kept overriding the new input::placeholder rule and kept rendering at 70% in
dark mode, which is the contrast failure this PR set out to close
They now spell it the same way as the three placeholders the PR already
converted, which both removes the step and settles on one spelling
* fix(ui): make dark-mode form controls visible
Two dark-mode defects left form controls without any visual boundary or fill.
`--input` and `--border` share one value in `.dark`, oklch(0.309), which resolves to
rgb(48,48,48). Against `--background` (33) that is a 15-step stroke, and against `--popover` (42)
it collapses to 6 steps out of 255, so a control inside any dialog is effectively undrawn. The
controls also use `bg-transparent`, so there is no fill cue either and only the placeholder text
renders. Measured 1.09:1 against the dialog surface where WCAG 1.4.11 asks for 3.0:1 on the
boundary of a user interface component. Splitting `--input` off at oklch(0.56) restores 3.07:1
without touching `--border`, which stays where it is because it draws decorative separators rather
than control boundaries. 91 controls across 19 routes were measured at the collapsed value, every
one with an identical stroke and surface, so a single token covers all of them.
Separately, `@tailwindcss/forms` paints a white fill on every bare control. The block above
already neutralises that for `combobox-chip-input`, but its audit covered `components/ui` only,
and hand-rolled controls elsewhere still render white on a dark page: typed text lands at 1.11:1
and native selects at 2.19:1 on `/model-hub-table`, `/playground`, `/guardrails`, `/mcp-servers`
and `/models-and-endpoints`. Tracking `--background` fixes those at 14.51:1 and 7.34:1.
Light mode is unchanged by both. The token edit is scoped to `.dark`, and `--background` in
`:root` is the same white the plugin was already painting, verified control-by-control on a dev
server: backgrounds stay rgb(255,255,255) and ratios stay 20.13:1 and 4.84:1.
* fix(ui): keep the combobox chip input transparent under the bare-control fill
The new base rule matched at (0,2,1) while the combobox chip-input override
sits at (0,1,0), so ComboboxChipsInput lost its transparent background and
painted an opaque page-colored rectangle inside the chips container, which
carries its own bg-transparent / dark:bg-input/30 fill.
Folding the exclusions into one :not() list adds the chip input and drops the
selector to (0,1,1). Every @tailwindcss/forms base selector is wrapped in
:where(), so it lands at (0,0,1); (0,1,1) still outweighs it and bare inputs,
textareas and selects keep the fill this PR gives them.
Base landed the native CLI OAuth + PKCE login, which added its own token
storage and a silent refresh that wrote the key straight to token.json.
This branch had already moved that secret into the OS keychain, so the two
had to be joined rather than picked between.
auth.py now keeps one pair of record helpers, load_token and save_token,
that read and write through the vault and hand the PKCE layer the plain
mapping it works with. fresh_api_key and revoke_stored_credential get
vault-bound save and reload callables, so a renewed key is stored in the
keychain like any other and a sibling process's rotation is still seen.
login goes through _replace_stored_token on both paths, so the credential
it replaces is revoked on the proxy and the user is still told where the
new one landed. logout revokes first, then reports what the clear actually
managed to do.
Mistral's live /v1/models reports max_context_length 1048576 and capabilities.reasoning
true for zai-glm-5-2, and its docs price cached input at $0.14/M. Without
cache_read_input_token_cost LiteLLM billed every cached prompt token at $0, so a repeat
request against a 21k-token cached prefix logged $0.0000135 instead of its real cost.
Mistral also serves the model under the short glm-5-2 name, which had no cost map entry
at all and therefore no pricing, so add it alongside.
Stacked on the multi-key shadow eval backend. The key picker becomes a
paginated multi-select with chips, built on the base-ui combobox chips
primitives, with the pagination and debounced-search logic extracted into a
shared usePaginatedCombobox hook that PaginatedSearchSelect now also uses.
The detail view gains a per key table showing each key's own status, judged
turns against its budget, and win rates from the by_key slice, and the job
headline pluralises to "N keys" for multi-key jobs
tests/proxy_unit_tests had a 30-line YAML parser inlined in its workflow that
failed the run when a test file there belonged to no shard. tests/test_litellm
is sharded the same way, with no catch-all bucket, and had no such guard: a new
directory under it (or under its proxy subtree) is collected by nothing and runs
nowhere, and the coverage census cannot see it because a token like
tests/test_litellm/test_*.py already answers 'yes, that tree runs'.
The two questions differ. The census asks whether a file runs at all, so an
ancestor path standing in for everything beneath it is a fine answer. Shard
assignment asks which shard owns a child, and there that same ancestor path is
precisely the bug. _token_covers keeps the first meaning; _token_names adds the
second, and the guard now walks a list of sharded trees rather than one hardcoded
directory. Both read the same test-path keys, so there is one workflow parser.
A directory needs a shard when it holds a test file, not when it is named test_*.
That drops the hardcoded test_configs exception and keeps fixture directories
like expected_fine_tuning_api out on their own merits.
The job keeps its name and its workflow, since assert-shard-coverage is a
required status check on litellm_internal_staging.
Verified red-first: a planted directory under tests/test_litellm, a planted
directory under tests/test_litellm/proxy, and a planted file under
tests/proxy_unit_tests each fail the guard, while a fixture-only directory does
not. 327 children across the three trees are assigned today.
Python binds a name once per scope, so when a module or class defines the same
test twice only the last one exists. The earlier definitions are unreachable:
pytest never collects them, and nothing that references them can fail.
A sweep in August cleared nine of these. Five have appeared since, which is the
argument for a rule rather than another sweep.
Each survivor is the better version, so nothing is lost. The two SQS logger
twins additionally stub `asyncio.create_task`, which the shadowed copies did
not. The cost-calculator duplicate is a two-line stub that also takes a
`model_item` parameter no fixture supplies, so it could not have run even
unshadowed. The two `test_prompt_caching` bodies are both `pass`.
Collecting the four files reports 416 tests before and after.
`tests/proxy_unit_tests/conftest copy.py` goes with them. pytest only loads a
file named exactly `conftest.py`, nothing imports this one, and the space in the
name says what it was.
* feat(ci): ratchet the test suite's zero-assert, mock-echo and global-state debt
The suite's dominant failure mode is tests that cannot fail for the reason anyone
would want them to. The testing-strategy audit measured five shapes of it, and
nothing mechanical stops any of them from reproducing, so they keep reproducing.
`scripts/check_test_quality.py` is an AST checker for those five, emitting the
same `path:line: CODE message` contract as `scripts/check_type_discipline.py`:
TQ001 a collectible test with no assertion of any kind
TQ002 mock-echo, where every assertion only inspects the mock that was patched
TQ003 sys.path.insert inside the test tree
TQ004 raw `os.environ[...] =`, which leaks into whatever runs next
TQ005 `litellm.<attr> =`, the process-wide leak the 491-line conftest undoes
`scripts/test_quality_gate.py` caps each rule against test-quality-budget.json,
seeded at exactly today's count, and fails only when a rule is both over its
limit and higher than the base being merged into, so a change is blamed for what
it adds and never for drift already in the base. `--update` lowers a limit by
what a branch cleared, so the ceilings only ever fall. It runs in the existing
required lint job, which means it enforces without a ruleset change.
TQ001 follows assertions into helpers defined in the same module, transitively.
Without that it flagged 111 tests in tests/e2e, the harness this program holds up
as the reference, because that suite factors its assertions into shared helpers
(`assert_auth_denied(result, ...)`). Following them leaves 25, all of which reach
their assertions across a module boundary; those are grandfathered and documented
rather than papered over.
The seeded counts land within about 10% of the audit's independent numbers for
every rule measured on the same subtree, which is the cross-check that the
definitions here match the ones the audit pinned.
* fix(ci): resolve test helpers per scope, not by bare name
The helper walk keyed every function in a module by its bare name, so two
same-named helpers in different classes collided and the last one parsed won.
A test calling `self._check()` could be cleared by a `_check` belonging to a
different class, or flagged because of one.
Resolution is now scoped: a bare name looks up the module-level functions, and
`self.<name>` looks up the enclosing class's own methods and no other class's.
Recursion is tracked by function identity rather than by name, so the cycle
guard cannot be confused by the same collision.
This surfaced one real zero-assert test that a same-named helper elsewhere had
been clearing, so TQ001 seeds at 750 rather than 749.
The test module has to register itself in sys.modules before exec_module:
`@dataclass(slots=True)` rebuilds its class through `sys.modules[__module__]`,
and Scope fails to construct without it. Recorded at the call site, since it
reads like avoidable global mutation otherwise.
* fix: register test-quality-budget.json with the ratchet alarm
The repo keeps one census over its budget files: every *-budget.json on disk
must appear in DEFAULT_BUDGETS, or its ceilings can be raised with no signal.
tests/test_litellm/test_budget_ratchet_check.py asserts that set equality and
caught the new budget on the way in.
Registering it also turns the alarm on for TQ001-TQ005, so a later PR cannot
quietly raise a test-quality ceiling. The file already uses the {limit: N}
schema the ratchet reads, so no other change was needed.
* test: retire tests/old_proxy_tests, which holds no tests
Twenty files named test_*.py, and pytest collects nothing from any of them:
uv run pytest tests/old_proxy_tests --collect-only -q
no tests collected, 16 errors in 114.17s
They are manual snippets against a running proxy, written at module level with
no test function, no assertion and no entry point, so the only thing the name
buys them is a place on the coverage allowlist. Sixteen of the twenty cannot
even be imported in this environment, wanting langchain, llama_index or
google.api_core, and ten still point at 0.0.0.0:8000, which stopped being the
proxy's default port some time ago.
Nothing outside the directory refers to it apart from the allowlist entry, which
goes with it. The other loose contents go too: five load_test_*.py scripts, a
bursty variant, two committed log files, an essay fixture and a stray .js
snippet.
Allowlist paths 88 -> 68, test files 2422 -> 2402, and no job loses anything it
was running. Recoverable from history if a snippet turns out to be someone's
habit.
* test: drop the retired old_proxy_tests paths from the coverage allowlist
The agent job's CircleCI glob collected `tests/agent_tests/**/test_*.py` and then
piped it through `grep -v` to drop `local_only_agent_tests/`. `assert_ci_coverage.py`
reads the glob but not the pipeline, so those two files looked covered and were
invisible to the census. The glob now excludes them structurally and they carry an
allowlist entry instead, which is a decision on the record rather than a hidden
filter. The collected file set is unchanged: `tests/agent_tests/` holds exactly one
CI-runnable test at the top level.
`tests/scim_tests/` held a single JSON fixture and no tests, referenced from nowhere.
`.github/workflows/` is for workflows. Both stray scripts move to `.github/scripts/`
with their callers updated: the price-file updater is invoked by
`auto_update_price_and_context_window.yml`, and the translation-report runner by
`make test-llm-translation`. The audit listed the latter as orphaned, but Makefile
line 317 still runs it, so it moves rather than being deleted.
The rollout heads-up workflow was a deliberate one-shot for the agent-shin rollout.
That rollout is done, the triage and auto-close workflows have been running daily
since June, so the pre-flip warning window is long past. Its script and dedicated
test go with it, and the sibling workflow-invariant test drops its entry.
Nine workflow files existed only to make a single call to _test-unit-base.yml
with a different test-path. Adding a shard meant adding a file; changing
anything shared meant editing nine. One matrix caller replaces them, so a shard
is now one entry.
Check names are unchanged, which is the whole constraint. A reusable-workflow
job reports as "<job name> / <inner job name>", so setting `name` to the shard id
alone reproduces today's context strings exactly: the eleven the matrix produces
are eleven of the twenty-three "/ Run tests" contexts the branch ruleset
requires, matched string for string. No ruleset edit is needed and none should
be made for this.
Every matrix entry states its timeouts even where they equal the base defaults.
An absent matrix key renders as an empty string rather than falling back, and an
empty string is not a number, so a partially-specified entry would fail the call.
tests/proxy_unit_tests keeps test-unit-proxy-db.yml. It is already a matrix and
its shard-coverage guard reads that file by name, so folding it in belongs with
generalising that guard into assert_ci_coverage.py rather than here. Its twelve
shards are the remaining required contexts.
test-unit-documentation.yml stays too: it does not call the base workflow.
A 503 from POST /revoke means the proxy could not write the single-use record, so clearing the local record left a live refresh token nobody could revoke and a hint to retry with nothing left to retry. lite logout now keeps the record, exits 1, and asks to be run again shortly. A refused or unreachable revocation still clears the record and warns as before, and a re-login that replaces a record keeps its existing warning because the new record already stands
revoke_refresh_token discarded the single-use claim result, so a revocation that arrived while Redis was unreachable answered 200 and left the refresh token live. The token endpoint reported the same outage as invalid_grant "already used". The guard now reports first, replayed, or unavailable, and both endpoints answer 503 temporarily_unavailable for an outage (RFC 7009 section 2.2.1, RFC 6749 section 5.2), which the CLI surfaces as a one-line warning while keeping the key it has
The lite group resolves the stored key once for every command and renews a --pkce key on the way in. lite up then asked the token file again, so every start sent a second refresh to the proxy, and once the refresh token was burned the refusal printed twice. _ensure_fresh_login now reuses the key the group resolved when the group read it from the token file, and only re-reads the file after the interactive login it starts itself. Also covers print-token through the group with a renewing session in the tests
The cli group already resolves the stored key for the server it was pointed
at, so print-token re-ran the renewal and, when the refresh token had been
revoked, posted to /token twice and printed the reason twice. print-token now
reuses the group's result whenever the stored record was issued for that
server and no --api-key or LITELLM_PROXY_API_KEY took precedence, and only
resolves the key itself when invoked bare for a different server.
A PKCE credential whose renewal is refused (for example after lite logout ran
on another copy of it) used to fail lite auth print-token with the classic
'Token expired. Run lite login again' hint and no reason, while lite whoami
already named lite login --pkce. fresh_api_key now reports why a renewal
failed through a warn callback whenever no sibling rotation rescued it, the
CLI prints that reason on stderr, and the expiry hint names the command that
produced the credential. Both READMEs document the admin revocation semantics
and the Redis precondition for refresh single use on several workers.
The stamp both orders the two stores and drives is_cli_token_fresh, and
nothing tied the two together, so a login that inherits a stamp from the
future could stop being a deliberate trade without anything failing.
Also corrects the lint-format-check-changed comment: git pathspecs match
recursively, so the target checks a superset of the CI step rather than
an identical set.
Discovery checks that every endpoint sits on the proxy origin, but the CLI's
requests.Session followed redirects, and requests replays a POST body on
307 and 308, so a token or revocation endpoint answering with one of those
would have sent the code and verifier, or the refresh token, wherever
Location pointed. Every POST now goes out with allow_redirects=False and a
3xx answer fails the command with a message naming where it pointed
A proxy on the default remote cost map never produced a prompt cache
breakpoint: the published map has the gpt-5.6 entries without
supports_prompt_cache_breakpoint, so the model-map gate returned False
for every listed model and only LITELLM_LOCAL_MODEL_COST_MAP=True (the
repo .env, hence the passing unit tests) made the feature work. The hook
now honors the flag when the entry carries one, True or False, and
otherwise applies the GPT-5.6+ version rule to the model name, so a map
that lags the flag still gets the OpenAI dialect. The model-map tests
pin litellm.model_cost to the bundled backup map and a new test drives
the hook against an unflagged gpt-5.6 entry.
completion() and acompletion() take base_url as an alias for api_base
that only lands on api_base after the cache control hook ran, so a
GPT-5.6 call at a non-OpenAI gateway given through base_url still got
the dialect. Both seed calls and the unstamped request-params read now
look at base_url too.
ResponsesAPIRequestUtils.merge_prompt_management_input reshaped hook
output in place, retyping text parts to input_text on the caller's own
message objects. The merge now shapes a copy of each message as it
emits it, so the identity-based merge keeps working on the hook's
objects and nothing the hook or the client owns is mutated.
The consent page offers the team picker, but a form posted without a team
sealed a teamless grant and the token endpoint minted an unscoped
credential for a team member, escaping the team attribution classic lite
login always applies. The minter now refuses such a grant on redemption
and refresh alike; memberships whose team rows are gone still count as no
team so they cannot lock a user out
FakeSecretVault could only stand in for a discarding backend by passing
KeyringDiscardsWrites as its `failure`, which also made read() and erase()
hand it back. Neither SecretRead nor SecretErase admits that outcome and the
real KeyringVault never produces it there, so the login path's match was
falling through on a value it can never see. Give the double a `discards`
flag that reports it from write() alone, which is what the null backend does.
Also widen lint-format-check-changed's pathspec. Git wildmatch runs without
FNM_PATHNAME here, so 'litellm/**/*.py' still requires an intermediate
directory and silently skipped all 21 top-level modules, litellm/__init__.py
and litellm/main.py among them. All 21 already pass ruff format.
The cache control hook also runs on litellm.responses() input. On a
GPT-5.6 deployment it wrapped a string-content item into a chat-shaped
{"type": "text"} part, which the Responses API rejects, and it never
marked input_text, input_image or input_file parts, so no breakpoint and
no prompt_cache_options reached the provider. Add the Responses part
types to the eligible block set and translate chat-shaped text parts on
non-assistant items to input_text in
ResponsesAPIRequestUtils.merge_prompt_management_input, which both the
async and the sync prompt management sites go through.
The dialect also fired for any GPT-5.6 name that resolved to provider
openai, including deployments pointed at a custom api_base that does not
understand prompt_cache_breakpoint. Decide it once per request from the
provider, the model map and the resolved api_base (request, then
litellm.api_base, then OPENAI_BASE_URL / OPENAI_API_BASE): only
api.openai.com and *.api.openai.com hosts speak the dialect, a top-level
prompt_cache_options opts a custom target in, and litellm_proxy/ targets
never get it. maybe_seed_default_injection_points takes api_base and
stamps the finished decision on the points as _litellm_openai_dialect so
the sync completion() path, whose hook params do not carry api_base,
honors it; maybe_inject_cache_control takes api_base from the
/v1/messages handler.
Eligibility now comes from a supports_prompt_cache_breakpoint model map
flag on the OpenAI gpt-5.6 entries, exposed through
litellm.utils.supports_prompt_cache_breakpoint, with the GPT version rule
kept only for models the map does not know. The OpenAI dialect no longer
reserves a slot for tool_config points, which OpenAI has no cache block
for, and with_prompt_cache_breakpoint plus the chat bridge helper return
a new block instead of mutating their input.
A login the keychain took but the token file could not record leaves the
keychain naming a later sign-in than the file does. Reading only the file
then stamps the next login below that keychain entry, and a clock that
went back far enough puts the superseded credential back in use.
The comment above the extra named cryptography as one of the heavy imports a
thin install leaves out. That stopped being true when keyring joined the
extra: on Linux it reaches the Secret Service through secretstorage, which
depends on cryptography.
The stamp in the keychain entry is what decides that secret against one still
sitting in the token file, and it came straight off the wall clock. A clock
that stepped backwards between two logins therefore handed the win to the
older of them: a login the keychain took but the token file could not be
pointed at was resolved back to the credential it replaced, and the fresh one
was erased from the keychain on the way past.
save_cli_token now reads the stamp already on disk and pins the new sign-in
just above it, so the ordering never depends on the clock having moved
forwards. On a clock that did, this changes nothing.
