Azure Database for PostgreSQL Flexible Server takes a Microsoft Entra ID access
token as the connection password, and those tokens last about an hour, so a
proxy pointed at one dies shortly after boot unless something keeps minting
fresh ones
Set AZURE_POSTGRESQL_AUTH=True (or pass --azure_postgresql_auth) alongside
DATABASE_HOST, DATABASE_USER, and DATABASE_NAME, and the proxy mints a token at
startup, assembles the connection URL around it, and refreshes it in the
background for as long as the process runs. That is the same shape
IAM_TOKEN_DB_AUTH already had for AWS RDS, so the two now share one code path:
a tagged union picks the minting strategy once, and the wrapper, the read
replica, and the refresh loop all read the choice off it instead of each
guessing from the environment. Setting both toggles is a startup error, in the
chart as well as in Python
The helm chart gets database.writer.useAzureEntraAuth and the matching reader
knob next to the existing useIAMAuth
Fixes#29661
Co-authored-by: David Balatoni <balcsida@gmail.com>
* test: replace blind sleeps with deadline waits in callback and caching tests
tests/local_testing/test_custom_callback_input.py slept a fixed 1-3s after
every call and then asserted the callback handler recorded no errors. Because
the handler only appends to `states` when a callback actually fires, an assert
of `len(errors) == 0` passes just as happily when nothing fired at all, so the
sleep was buying flakiness in exchange for a vacuous check. The async tests
were worse: `time.sleep` blocks the event loop, so the success/failure tasks
scheduled on it could not run before the assertion.
Adds tests/_wait_helpers.py with `wait_until` / `await_until`, which poll a
predicate against a deadline, and converts all 17 sites to wait on the thing
the test actually cares about (the terminal state landing in `states`, or the
patched log hook being called). The waits assert the callback fired, so these
tests now fail on a dropped callback instead of passing silently.
The three sleeps in test_caching_handler.py sat between `sync_set_cache` and
`_sync_get_cache`, both fully synchronous against a local in-memory cache, so
they are just deleted.
* fix(test): wait on the priming call's own logging in the cache-hit test
The 3s sleep in test_logging_async_cache_hit_sync_call was not waiting for the
cache write, which lands before the stream iterator is exhausted. It was
waiting for the priming call's success callback to drain, so the handler
installed right after it only ever sees the second, cache-hit call. Waiting on
a populated cache_dict let the priming call's still-pending log_success_event
reach the new mock, and the test then read cache_hit off the wrong payload.
Waits on the priming handler's own sync_success state instead.
`lite login --pkce` mints a refresh token that buys a fresh key from the
proxy on demand, so it is the credential just as much as the key is. Moving
the key into the keychain left it behind in ~/.litellm/token.json, where any
process running as the user can read it and renew the login for itself.
It now travels with the key: `save_cli_token` writes both into the keychain
entry, the token file keeps only metadata, and `lite logout` takes it out of
the file whether or not the keychain answers.
Upgrading finds one sign-in split across the two stores, the key already in
the keychain and the refresh token still on disk. That case rejoins the two
halves into a single entry before scrubbing the file, so the write never
replaces a live key with nothing, and a machine that refuses the scrub keeps
what it has rather than having the key rolled back out from under it.
* feat(otel): route Phoenix traces to per-key/team projects under otel v2
The v2 arize_phoenix preset read PHOENIX_PROJECT_NAME once at startup into a
static resource attribute, silently dropping the per-key/team project routing
v1 supported. Route it via Phoenix's x-project-name OTLP/HTTP header instead:
the env var stays the global default, and a phoenix_project_name (or
phoenix_project_name_override) in key/team metadata sends that key's traces
to the named project.
The project comes only from user_api_key_auth_metadata (server-set at auth),
never from client request metadata or StandardCallbackDynamicParams, since
choosing the telemetry destination is a data-exfiltration primitive. The
header is appended to the exporter's static headers rather than replacing
them, so the preset's Authorization survives, and it is gated to OTLP/HTTP
exporters because Phoenix only reads it on /v1/traces.
Also unban the bare phoenix_project_name fields from the request-body gate:
the proxy integrations ignore them (only user_api_key_auth_metadata routes,
and that stays banned), so rejecting them just broke SDK-style callers.
* fix(otel): root project-routed Phoenix spans in their own trace
Phoenix assigns a whole trace to one project by whichever span arrives
first. The request's auth/db/root spans always export through the default
provider without the project header, so a project-routed LLM span parented
into that trace got dragged back into the default project and the header
did nothing (verified against a live Phoenix instance). Detach the routed
span into its own trace with a link back to the request trace, mirroring
how the v1 Phoenix logger exported each request under its own local parent.
* fix(otel): drain in-flight spans before shutting down evicted providers
LRU eviction shut a routed provider down immediately, but an LLM span
opened at pre_call stays open until the later success or failure callback;
with more than 256 overlapping credential/project routes that in-flight
span was silently dropped instead of exported. Refcount open spans per
provider (hold at span open, release when the carrier is removed on close,
carrier-map eviction, or MCP stray-carrier cleanup) and defer a retired
provider's shutdown until its last open span closes.
* fix(otel): take the provider hold inside route_for to close the eviction race
pre_call can run on thread-pool workers, so between route_for returning a
provider and the caller recording its open span, a concurrent request could
overflow the LRU and shut that provider down with a zero span count, dropping
the routed trace. route_for now increments the open-span count in the same
locked critical section as the cache update and hands back an already-held
provider; every caller releases it once its span has landed. The lock also
makes the cache mutations safe under that same thread-pool concurrency.
* fix(otel): skip tenant routing on deferred pre_call
route_for ran before the recordable-parent check, so a thread-pool
pre_call still built or LRU-touched a tenant provider and could evict
an idle one even though the hold was released immediately and close
re-routed. Only route when the span actually opens
* add somethign
* Revert "add somethign"
This reverts commit 2f2cf84c5a.
* fix(otel): cap retired tenant providers draining open spans
* docs(otel): justify the retired-provider cap
The bundled logo is a JPEG, so it carries no alpha and its white
background renders as a bright slab against a dark sidebar. Making it
transparent alone would not be enough either: the wordmark is near-black
and would disappear on dark.
Adds logo_dark.png, derived from the light logo. The sky-blue disc and
train are kept as they are behind a circular alpha mask, and the
wordmark's antialiasing is un-flattened from white into straight alpha
and repainted in the dark theme's own foreground colour. Both files are
1000x257, so swapping between them cannot shift the sidebar header.
/get_image gains a theme query param. The default response is byte for
byte what it was, and a logo configured through UI_LOGO_PATH is served
unchanged in both themes, since custom logos have no dark variant yet.
* feat(complexity_router): add business classification rubric preset
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate api schema for business rubric
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): suppress preexisting antd import violations in touched files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: tin <tin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): draw one Per Day savings bar per date on Cost Optimization
The page paged /user/daily/activity over raw rows, so a date spanning
pages arrived N times with partial metrics and rendered as N thin bars.
Switch to the single-shot aggregated endpoint, thread
include_current_utc_day through it to keep the live-end extension from
PR #36051, and merge the paginated fallback by date.
* fix(ui): keep aggregated call at four params and mock it in view tests
Trailing userId and includeCurrentUtcDay ride a named rest tuple so the
eslint max-params baseline stays at 23, and the CostOptimizationView
suites mock the new networking export their render now reaches.
Combines the model-cost-map data from #35911, #36017, #36080, #36113, #36188, #36444, #37029, #37252 and #37632 onto current litellm_internal_staging, merged per entry field so older branches no longer revert fields the base has gained since they were opened. Drops the Gemini deprecation dates from #36188 and the text-embedding-004 date from #36080 that the official docs contradict.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: run the 30 test files stranded in the second mirror
tests/litellm sat beside tests/test_litellm, which is the mirror the repo
convention names, and no job collected it. The allowlist called the directory
unresolved and assumed it was a duplicate. It is not: 30 of its 34 files have no
counterpart in the real mirror, so they are tests nobody has run since they were
written, not copies of tests that run elsewhere.
Moving them in is byte-identical, and it is what makes them run. Every one is
now claimed by a shard's test-path rather than by an allowlist entry, and the
216 tests they hold pass. Directories that needed to become packages did, since
several files are named test_transformation.py and pytest cannot import two of
those from non-package directories in one session.
Never running is why three assertions had drifted away from the code:
* nvidia.nemotron-super-3-120b max_output_tokens, 32000 -> 32768
* sambanova/MiniMax-M2.7 max_input_tokens, 204800 -> 196608
* the Vertex text-to-speech handler moved from data= to json=, so the test
reads the decoded body off the json kwarg instead of parsing the data one
The first two follow model_prices_and_context_window.json, which the catalog
sync keeps current; the third follows the handler. In all three the test was the
stale side.
The lint workflow ran test_no_hardcoded_secrets.py by path and now points at the
new one.
Four files stay behind. Each shares a filename with a live test whose contents
are disjoint from it, so landing those means merging test bodies, which is a
content review rather than a move. The allowlist entry now names those four and
records how many tests each would bring, in place of calling the whole
directory unresolved.
* fix(ci): keep the secret scan out of the mirror's conftest
The secret-scan job runs pytest under uv run --no-project, so its environment
holds pytest and nothing else. That worked while the file sat in tests/litellm,
which has no conftest, and broke the moment it moved into tests/test_litellm,
whose conftest imports litellm on collection: ModuleNotFoundError: No module
named 'dotenv', before a single test ran.
The file is a repo-wide static scan that imports only base64, os, re and pytest,
so it belongs with the other repo-wide checks in tests/code_coverage_tests,
which has no conftest, rather than in the package mirror. Installing the full
dependency set into a 15-second job to satisfy a conftest it does not use would
be the wrong trade.
Verified with the job's exact command:
uv run --no-project --with 'pytest==9.0.2' pytest \
tests/code_coverage_tests/test_no_hardcoded_secrets.py -q
1 passed in 0.47s
* feat(ci): catch files a -k expression deselects from every job
The coverage census asks whether some job names a file. It cannot ask what that
job's -k then does with it, and the gap is not hypothetical: tests/local_testing
is globbed by five jobs, two of which carry
-k "... and not router and not assistants and not langfuse and not caching and not cache"
while the other three keep one keyword each. Any file whose path holds an
excluded term is dropped by the first two and matched by none of the rest, so it
runs nowhere while the census counts it as covered. 118 tests across eight
caching files sit in exactly that hole today.
The new mode reads the same CircleCI jobs the census already parses and asks
whether each globbed file survives its job's selector. Two facts about -k make
that decidable without running pytest: it matches an item's own name and its
parents', so a term appearing in the module path deselects the whole file; and
the names it can match are otherwise the classes and functions in the file,
which ast reads. A positive term is therefore satisfied by the path or by a name
inside, which is what keeps a langfuse-named test inside test_logging.py from
being reported.
Where the parser is unsure it stays quiet. An expression with or, parentheses,
or a negated group is left unmodelled and its job is treated as claiming
everything it globs, so an unparsed selector can never raise a false alarm.
Glob translation learned character classes, without which
tests/local_testing/**/test_[a-mA-M]*.py matches nothing and the guard would
report that whole directory. The census and shard counts are unchanged by it,
2423 files and 327 shard children before and after.
Validated against the real thing: collecting tests/local_testing under each
job's own selector leaves 175 of 1577 tests unselected, in exactly the ten files
this check derives statically, no more and no fewer. Two of the ten are named
outright by other jobs, which the check credits, leaving the eight now recorded
in the allowlist as a decision rather than an accident.
Verified red-first: dropping one of those eight from the allowlist reports it,
and adding 'and not embedding' to the two part jobs reports test_embedding.py
and test_get_optional_params_embeddings.py.
* fix(ci): keep the slice guard from pairing one command's -k with another's glob
Two accuracy notes from review, both about the parser's model rather than its
current verdicts.
A job that runs several pytest commands offers no way to tell which glob a -k
belongs to, since both are read out of the same flattened job text. Combining
them could pair one command's exclusion with another command's glob and report a
file that in fact runs. Such a job is now left unmodelled, which means it claims
everything it globs, matching how the parser already treats an expression it
cannot read. Only one job in the config has two globs today and it carries no
-k at all, so no verdict changes.
The second is a deliberate limit, now stated where it lives: an excluded term is
only honoured when it sits in the module path, because that is the case that
takes the whole file with it. A term matching one function inside drops that
test and leaves the file running, and reporting it would be a false alarm.
Answering per-test instead would need a baseline of test ids that churns on
every rename, for a smaller failure than a file going dark.
Both are pinned by tests.
* feat(ci): ratchet tests that skip themselves when a credential is absent
* docs(ci): name the new rule where the gate's rules are listed
* fix(ci): require the condition to test for absence before TQ006 fires
* test: settle three allowlist entries that were open questions
The allowlist is meant to hold decisions, not deferrals, so an entry reading
'needs moving' or 'referenced by no job' is a gap wearing an exemption. These
three each get an answer.
The two prompt-factory tests move into the mirror, which is what their own entry
said they needed. Both were passing the whole time, so the 23 tests they hold
start running and the entry goes away rather than getting reworded.
test_aio_http_image_conversion.py is not a test. It fetches live image URLs,
times aiohttp against httpx, prints the ratio, and asserts nothing, and pytest
cannot collect it because its functions take arguments rather than fixtures.
Running it beside its siblings would buy CI a network dependency and a number
nothing reads, so it stays exempt with that written down.
test_litellm_proxy_extras_utils.py stays exempt with a measured reason. 24 of
its 28 tests pass; the 4 in TestMigrationSQLIdempotency fail because nine
migrations from 2026-04 onward use bare CREATE TABLE, ADD COLUMN and CREATE
INDEX where that file requires guarded forms. The convention eroded quietly
precisely because the test enforcing it has never run. Wiring it up is blocked
on what to do about those migrations, and editing them is not the answer, since
Prisma checksums an applied migration and a changed one breaks migrate deploy
for existing installs.
Allowlist entries 10 -> 9, paths 88 -> 86.
* docs(ci): correct the migration count in the proxy-extras allowlist reason
Base landed the native CLI OAuth + PKCE login, which added its own token
storage and a silent refresh that wrote the key straight to token.json.
This branch had already moved that secret into the OS keychain, so the two
had to be joined rather than picked between.
auth.py now keeps one pair of record helpers, load_token and save_token,
that read and write through the vault and hand the PKCE layer the plain
mapping it works with. fresh_api_key and revoke_stored_credential get
vault-bound save and reload callables, so a renewed key is stored in the
keychain like any other and a sibling process's rotation is still seen.
login goes through _replace_stored_token on both paths, so the credential
it replaces is revoked on the proxy and the user is still told where the
new one landed. logout revokes first, then reports what the clear actually
managed to do.
Mistral's live /v1/models reports max_context_length 1048576 and capabilities.reasoning
true for zai-glm-5-2, and its docs price cached input at $0.14/M. Without
cache_read_input_token_cost LiteLLM billed every cached prompt token at $0, so a repeat
request against a 21k-token cached prefix logged $0.0000135 instead of its real cost.
Mistral also serves the model under the short glm-5-2 name, which had no cost map entry
at all and therefore no pricing, so add it alongside.
tests/proxy_unit_tests had a 30-line YAML parser inlined in its workflow that
failed the run when a test file there belonged to no shard. tests/test_litellm
is sharded the same way, with no catch-all bucket, and had no such guard: a new
directory under it (or under its proxy subtree) is collected by nothing and runs
nowhere, and the coverage census cannot see it because a token like
tests/test_litellm/test_*.py already answers 'yes, that tree runs'.
The two questions differ. The census asks whether a file runs at all, so an
ancestor path standing in for everything beneath it is a fine answer. Shard
assignment asks which shard owns a child, and there that same ancestor path is
precisely the bug. _token_covers keeps the first meaning; _token_names adds the
second, and the guard now walks a list of sharded trees rather than one hardcoded
directory. Both read the same test-path keys, so there is one workflow parser.
A directory needs a shard when it holds a test file, not when it is named test_*.
That drops the hardcoded test_configs exception and keeps fixture directories
like expected_fine_tuning_api out on their own merits.
The job keeps its name and its workflow, since assert-shard-coverage is a
required status check on litellm_internal_staging.
Verified red-first: a planted directory under tests/test_litellm, a planted
directory under tests/test_litellm/proxy, and a planted file under
tests/proxy_unit_tests each fail the guard, while a fixture-only directory does
not. 327 children across the three trees are assigned today.
Python binds a name once per scope, so when a module or class defines the same
test twice only the last one exists. The earlier definitions are unreachable:
pytest never collects them, and nothing that references them can fail.
A sweep in August cleared nine of these. Five have appeared since, which is the
argument for a rule rather than another sweep.
Each survivor is the better version, so nothing is lost. The two SQS logger
twins additionally stub `asyncio.create_task`, which the shadowed copies did
not. The cost-calculator duplicate is a two-line stub that also takes a
`model_item` parameter no fixture supplies, so it could not have run even
unshadowed. The two `test_prompt_caching` bodies are both `pass`.
Collecting the four files reports 416 tests before and after.
`tests/proxy_unit_tests/conftest copy.py` goes with them. pytest only loads a
file named exactly `conftest.py`, nothing imports this one, and the space in the
name says what it was.
* feat(ci): ratchet the test suite's zero-assert, mock-echo and global-state debt
The suite's dominant failure mode is tests that cannot fail for the reason anyone
would want them to. The testing-strategy audit measured five shapes of it, and
nothing mechanical stops any of them from reproducing, so they keep reproducing.
`scripts/check_test_quality.py` is an AST checker for those five, emitting the
same `path:line: CODE message` contract as `scripts/check_type_discipline.py`:
TQ001 a collectible test with no assertion of any kind
TQ002 mock-echo, where every assertion only inspects the mock that was patched
TQ003 sys.path.insert inside the test tree
TQ004 raw `os.environ[...] =`, which leaks into whatever runs next
TQ005 `litellm.<attr> =`, the process-wide leak the 491-line conftest undoes
`scripts/test_quality_gate.py` caps each rule against test-quality-budget.json,
seeded at exactly today's count, and fails only when a rule is both over its
limit and higher than the base being merged into, so a change is blamed for what
it adds and never for drift already in the base. `--update` lowers a limit by
what a branch cleared, so the ceilings only ever fall. It runs in the existing
required lint job, which means it enforces without a ruleset change.
TQ001 follows assertions into helpers defined in the same module, transitively.
Without that it flagged 111 tests in tests/e2e, the harness this program holds up
as the reference, because that suite factors its assertions into shared helpers
(`assert_auth_denied(result, ...)`). Following them leaves 25, all of which reach
their assertions across a module boundary; those are grandfathered and documented
rather than papered over.
The seeded counts land within about 10% of the audit's independent numbers for
every rule measured on the same subtree, which is the cross-check that the
definitions here match the ones the audit pinned.
* fix(ci): resolve test helpers per scope, not by bare name
The helper walk keyed every function in a module by its bare name, so two
same-named helpers in different classes collided and the last one parsed won.
A test calling `self._check()` could be cleared by a `_check` belonging to a
different class, or flagged because of one.
Resolution is now scoped: a bare name looks up the module-level functions, and
`self.<name>` looks up the enclosing class's own methods and no other class's.
Recursion is tracked by function identity rather than by name, so the cycle
guard cannot be confused by the same collision.
This surfaced one real zero-assert test that a same-named helper elsewhere had
been clearing, so TQ001 seeds at 750 rather than 749.
The test module has to register itself in sys.modules before exec_module:
`@dataclass(slots=True)` rebuilds its class through `sys.modules[__module__]`,
and Scope fails to construct without it. Recorded at the call site, since it
reads like avoidable global mutation otherwise.
* fix: register test-quality-budget.json with the ratchet alarm
The repo keeps one census over its budget files: every *-budget.json on disk
must appear in DEFAULT_BUDGETS, or its ceilings can be raised with no signal.
tests/test_litellm/test_budget_ratchet_check.py asserts that set equality and
caught the new budget on the way in.
Registering it also turns the alarm on for TQ001-TQ005, so a later PR cannot
quietly raise a test-quality ceiling. The file already uses the {limit: N}
schema the ratchet reads, so no other change was needed.
* test: retire tests/old_proxy_tests, which holds no tests
Twenty files named test_*.py, and pytest collects nothing from any of them:
uv run pytest tests/old_proxy_tests --collect-only -q
no tests collected, 16 errors in 114.17s
They are manual snippets against a running proxy, written at module level with
no test function, no assertion and no entry point, so the only thing the name
buys them is a place on the coverage allowlist. Sixteen of the twenty cannot
even be imported in this environment, wanting langchain, llama_index or
google.api_core, and ten still point at 0.0.0.0:8000, which stopped being the
proxy's default port some time ago.
Nothing outside the directory refers to it apart from the allowlist entry, which
goes with it. The other loose contents go too: five load_test_*.py scripts, a
bursty variant, two committed log files, an essay fixture and a stray .js
snippet.
Allowlist paths 88 -> 68, test files 2422 -> 2402, and no job loses anything it
was running. Recoverable from history if a snippet turns out to be someone's
habit.
* test: drop the retired old_proxy_tests paths from the coverage allowlist
The agent job's CircleCI glob collected `tests/agent_tests/**/test_*.py` and then
piped it through `grep -v` to drop `local_only_agent_tests/`. `assert_ci_coverage.py`
reads the glob but not the pipeline, so those two files looked covered and were
invisible to the census. The glob now excludes them structurally and they carry an
allowlist entry instead, which is a decision on the record rather than a hidden
filter. The collected file set is unchanged: `tests/agent_tests/` holds exactly one
CI-runnable test at the top level.
`tests/scim_tests/` held a single JSON fixture and no tests, referenced from nowhere.
`.github/workflows/` is for workflows. Both stray scripts move to `.github/scripts/`
with their callers updated: the price-file updater is invoked by
`auto_update_price_and_context_window.yml`, and the translation-report runner by
`make test-llm-translation`. The audit listed the latter as orphaned, but Makefile
line 317 still runs it, so it moves rather than being deleted.
The rollout heads-up workflow was a deliberate one-shot for the agent-shin rollout.
That rollout is done, the triage and auto-close workflows have been running daily
since June, so the pre-flip warning window is long past. Its script and dedicated
test go with it, and the sibling workflow-invariant test drops its entry.
A 503 from POST /revoke means the proxy could not write the single-use record, so clearing the local record left a live refresh token nobody could revoke and a hint to retry with nothing left to retry. lite logout now keeps the record, exits 1, and asks to be run again shortly. A refused or unreachable revocation still clears the record and warns as before, and a re-login that replaces a record keeps its existing warning because the new record already stands
revoke_refresh_token discarded the single-use claim result, so a revocation that arrived while Redis was unreachable answered 200 and left the refresh token live. The token endpoint reported the same outage as invalid_grant "already used". The guard now reports first, replayed, or unavailable, and both endpoints answer 503 temporarily_unavailable for an outage (RFC 7009 section 2.2.1, RFC 6749 section 5.2), which the CLI surfaces as a one-line warning while keeping the key it has
The lite group resolves the stored key once for every command and renews a --pkce key on the way in. lite up then asked the token file again, so every start sent a second refresh to the proxy, and once the refresh token was burned the refusal printed twice. _ensure_fresh_login now reuses the key the group resolved when the group read it from the token file, and only re-reads the file after the interactive login it starts itself. Also covers print-token through the group with a renewing session in the tests
The cli group already resolves the stored key for the server it was pointed
at, so print-token re-ran the renewal and, when the refresh token had been
revoked, posted to /token twice and printed the reason twice. print-token now
reuses the group's result whenever the stored record was issued for that
server and no --api-key or LITELLM_PROXY_API_KEY took precedence, and only
resolves the key itself when invoked bare for a different server.
A PKCE credential whose renewal is refused (for example after lite logout ran
on another copy of it) used to fail lite auth print-token with the classic
'Token expired. Run lite login again' hint and no reason, while lite whoami
already named lite login --pkce. fresh_api_key now reports why a renewal
failed through a warn callback whenever no sibling rotation rescued it, the
CLI prints that reason on stderr, and the expiry hint names the command that
produced the credential. Both READMEs document the admin revocation semantics
and the Redis precondition for refresh single use on several workers.
The stamp both orders the two stores and drives is_cli_token_fresh, and
nothing tied the two together, so a login that inherits a stamp from the
future could stop being a deliberate trade without anything failing.
Also corrects the lint-format-check-changed comment: git pathspecs match
recursively, so the target checks a superset of the CI step rather than
an identical set.
Discovery checks that every endpoint sits on the proxy origin, but the CLI's
requests.Session followed redirects, and requests replays a POST body on
307 and 308, so a token or revocation endpoint answering with one of those
would have sent the code and verifier, or the refresh token, wherever
Location pointed. Every POST now goes out with allow_redirects=False and a
3xx answer fails the command with a message naming where it pointed
A proxy on the default remote cost map never produced a prompt cache
breakpoint: the published map has the gpt-5.6 entries without
supports_prompt_cache_breakpoint, so the model-map gate returned False
for every listed model and only LITELLM_LOCAL_MODEL_COST_MAP=True (the
repo .env, hence the passing unit tests) made the feature work. The hook
now honors the flag when the entry carries one, True or False, and
otherwise applies the GPT-5.6+ version rule to the model name, so a map
that lags the flag still gets the OpenAI dialect. The model-map tests
pin litellm.model_cost to the bundled backup map and a new test drives
the hook against an unflagged gpt-5.6 entry.
completion() and acompletion() take base_url as an alias for api_base
that only lands on api_base after the cache control hook ran, so a
GPT-5.6 call at a non-OpenAI gateway given through base_url still got
the dialect. Both seed calls and the unstamped request-params read now
look at base_url too.
ResponsesAPIRequestUtils.merge_prompt_management_input reshaped hook
output in place, retyping text parts to input_text on the caller's own
message objects. The merge now shapes a copy of each message as it
emits it, so the identity-based merge keeps working on the hook's
objects and nothing the hook or the client owns is mutated.
The consent page offers the team picker, but a form posted without a team
sealed a teamless grant and the token endpoint minted an unscoped
credential for a team member, escaping the team attribution classic lite
login always applies. The minter now refuses such a grant on redemption
and refresh alike; memberships whose team rows are gone still count as no
team so they cannot lock a user out
FakeSecretVault could only stand in for a discarding backend by passing
KeyringDiscardsWrites as its `failure`, which also made read() and erase()
hand it back. Neither SecretRead nor SecretErase admits that outcome and the
real KeyringVault never produces it there, so the login path's match was
falling through on a value it can never see. Give the double a `discards`
flag that reports it from write() alone, which is what the null backend does.
Also widen lint-format-check-changed's pathspec. Git wildmatch runs without
FNM_PATHNAME here, so 'litellm/**/*.py' still requires an intermediate
directory and silently skipped all 21 top-level modules, litellm/__init__.py
and litellm/main.py among them. All 21 already pass ruff format.
The cache control hook also runs on litellm.responses() input. On a
GPT-5.6 deployment it wrapped a string-content item into a chat-shaped
{"type": "text"} part, which the Responses API rejects, and it never
marked input_text, input_image or input_file parts, so no breakpoint and
no prompt_cache_options reached the provider. Add the Responses part
types to the eligible block set and translate chat-shaped text parts on
non-assistant items to input_text in
ResponsesAPIRequestUtils.merge_prompt_management_input, which both the
async and the sync prompt management sites go through.
The dialect also fired for any GPT-5.6 name that resolved to provider
openai, including deployments pointed at a custom api_base that does not
understand prompt_cache_breakpoint. Decide it once per request from the
provider, the model map and the resolved api_base (request, then
litellm.api_base, then OPENAI_BASE_URL / OPENAI_API_BASE): only
api.openai.com and *.api.openai.com hosts speak the dialect, a top-level
prompt_cache_options opts a custom target in, and litellm_proxy/ targets
never get it. maybe_seed_default_injection_points takes api_base and
stamps the finished decision on the points as _litellm_openai_dialect so
the sync completion() path, whose hook params do not carry api_base,
honors it; maybe_inject_cache_control takes api_base from the
/v1/messages handler.
Eligibility now comes from a supports_prompt_cache_breakpoint model map
flag on the OpenAI gpt-5.6 entries, exposed through
litellm.utils.supports_prompt_cache_breakpoint, with the GPT version rule
kept only for models the map does not know. The OpenAI dialect no longer
reserves a slot for tool_config points, which OpenAI has no cache block
for, and with_prompt_cache_breakpoint plus the chat bridge helper return
a new block instead of mutating their input.
A login the keychain took but the token file could not record leaves the
keychain naming a later sign-in than the file does. Reading only the file
then stamps the next login below that keychain entry, and a clock that
went back far enough puts the superseded credential back in use.
The stamp in the keychain entry is what decides that secret against one still
sitting in the token file, and it came straight off the wall clock. A clock
that stepped backwards between two logins therefore handed the win to the
older of them: a login the keychain took but the token file could not be
pointed at was resolved back to the credential it replaced, and the fresh one
was erased from the keychain on the way past.
save_cli_token now reads the stamp already on disk and pins the new sign-in
just above it, so the ordering never depends on the clock having moved
forwards. On a clock that did, this changes nothing.
