The discovery document is accepted only when its issuer is the --base-url the user
typed and every endpoint and the resource share that origin (RFC 8414 section 3.3),
so a tampered or redirected document can no longer point the code, verifier, or
refresh token at another host. After a failed refresh the re-read token record is
used only when it continues the same credential (same proxy, token endpoint, and
resource) and has not expired, so a concurrent login against a different proxy can
never hand this one its key
The proxy's OAuth authorization server (dynamic registration, PKCE S256,
loopback redirects, single-use codes, refresh rotation) gains a proxy-API
audience: /authorize?resource=<proxy origin> renders a consent page with
team selection and /token mints the same per-user credential lite login
mints, so a native CLI can sign a user in through the system browser and
call /v1/* with user and team attribution. Adds GET /.well-known/litellm-cli-auth
as the versioned discovery contract for non-Python clients, POST /revoke
(RFC 7009) for logout, and lite login --pkce, lite logout, and
lite auth print-token on the CLI side. Proxy-API grants only ever redirect
to a loopback address and the server never picks a team on the user's behalf.
Fixes#37332
`lite up` treats a token record whose key the keychain would not hand over as no
login at all, and that clause had no test: every existing freshness test passed a
record carrying a real key, so deleting the clause left the whole suite green
The base install smoke check now also asserts keyring is absent, which is what
makes the lazy import in cli_keyring meaningful. keyring ships in the cli extra
only, so a plain `pip install litellm` must not be able to reach it
- accept the Live SDK's models/<id> and LiteLLM's vertex_ai/<id> when rewriting the setup model
- keep a dict service account intact instead of stringifying it
- treat same-target deployments holding different credentials as ambiguous
- guard both websocket states before every close so a second close cannot raise
- build the sendable close codes from the public CloseCode enum
`lite whoami` led with "Authenticated" whenever a token file was on disk, even when the
keychain holding the credential would not give it up. The notice about that sat below the
account lines, so the session read as a working one and sent the user looking for the
problem anywhere but the keychain
Taking the secret out of ~/.litellm/token.json stages a replacement and moves it into
place, which needs room for a second file and a directory that will accept a new entry.
A full disk refuses the first and a read-only ~/.litellm the second, and logout gave up
there: it removed the file when it could, dropping the record that the keychain had never
been confirmed clear, so the logout after it reported a clean keychain it never checked
Shortening the file already in place needs neither, so the logout scrub and the legacy
migration now fall back to overwriting it where it lies. On a read-only ~/.litellm the
logout the user asked for now happens, instead of coming back with instructions to delete
the file by hand
A ~/.litellm that has gone read-only, or one left root-owned by a sudo login,
refuses both the scrubbed rewrite and the removal. The removal was unguarded,
so 'lite logout' ended in a PermissionError traceback with the credential still
readable in the file. It now comes back as an outcome the command reports,
naming the file and what to do about it, and a file that holds no secret is
still not worth alarming anyone over.
A logout run from an install without the keyring package treated a token file
holding its own secret as proof that no keychain entry could exist. That only
holds for the login which wrote the file. A login before it may have had the
package and put its credential in the keychain, where it outlives both the
uninstall and the file that replaced it, so logout reported a clean sweep over
a live credential. Every keychain that cannot be reached is now treated the
same way, and the message says the keychain went unchecked rather than
asserting what is in it.
A logout that could not reach the keychain deleted the token file whenever it
still held its own secret, and the next logout read that missing file as proof
the keychain was clean. It answered the warning the first run had just issued
with "Logged out successfully" while the entry an earlier login left behind was
still live. The file is the only record that something may still be in there,
which is what `_nothing_left_behind` already says it relies on, so keep it and
take only the secret out.
A keychain that did answer is a different case. `SecretStranded` means the entry
is confirmed there and would not delete, and that needs no note in the file,
while keeping one lets every later command read the credential straight back out
of the keychain, which makes "Logged out locally" untrue. That one drops the
file, as it did before.
The secret still goes first either way: a copy that cannot be replaced with a
secret-free one is removed rather than kept.
A /v1/realtime connection to a Vertex AI Live model accepted the WebSocket
upgrade and then went silent: a stalled Google OAuth token fetch blocked the
handler before any session event, and the eventual failure closed the socket
with a bare 1011 and no error event, so callers saw an open socket, no frames,
and no reason.
Bound the pre-session token fetch with
REALTIME_CREDENTIAL_RESOLUTION_TIMEOUT_SECONDS (20s default) and, on any
realtime failure, send an OpenAI-style error event before closing with a reason
that names the failure. Close reasons are truncated by bytes, not characters,
since an over-long reason makes the close frame itself fail.
The /vertex_ai/live WebSocket passthrough only ever looked at
default_vertex_config and the DEFAULT_VERTEXAI_* env vars, so a proxy whose
Vertex credentials live in the DB as a model entry with use_in_pass_through
had nothing to authenticate with. The upgrade still succeeded and the socket
then closed with a bare 1000 on the first client frame, which gave the client
no way to tell a misconfiguration from a normal end of session.
Credentials now also resolve from the router deployments flagged
use_in_pass_through, preferring the one matching the requested model, and a
failure to mint an access token closes 1011 with a reason naming both ways to
configure it. Upstream closes other than a plain 1000 are relayed to the client
with their code and reason, so Google's own errors reach the caller. The setup
frame's model is rewritten to the full projects/.../publishers/google/models
resource path, which is what Vertex expects and what lets a bare model id or a
gateway alias work over this route.
Three ways the credential commands could mislead or hang.
`lite logout` on a machine that never logged in warned that a credential may
be stranded in a keychain it could not check, and told the user to install
keyring to go clear it. There was nothing there. A missing token file is now
read as the evidence it is, because logout keeps a secret-free file behind
whenever the keychain is left unconfirmed, so a later run can tell a machine
with a credential it cannot reach apart from one that never had a login. That
holds on the LITELLM_CLI_DISABLE_KEYRING path too.
`KeyringDiscardsWrites` was handled on the read and erase paths, which cannot
produce it: the null backend returns None from `get_password` rather than
raising, so only a write ever detects it. It now lives on `SecretWrite` alone
and the unreachable arms are gone.
`keyring.set_password` blocks forever under a HOME with no usable login
keychain, which is what containers, CI images, `sudo -H`, and service accounts
run with, and reads answer normally there so nothing cheaper tells them apart.
`lite login` never touched a keychain before this, so a sign-in that simply
never returns would be a new way for it to fail. Writes are pre-flighted with
a throwaway value on a bounded wait, and a keychain that stays silent falls
back to the token file. The real credential is never the thing handed to a
call that might land long after we stopped waiting.
Saving also stages the token file before the keychain is given anything, since
the file is the half a read-only or full directory refuses. A save that cannot
land now leaves both stores as it found them, which matters most when the
login it failed to replace still works.
* fix(ptu): hand the prune a plain delete filter the query builder can serialise
The bounded sweep built its predicate as a read-only mapping view, which the query
builder refuses to serialise, so the nightly job raised as soon as a config-declared
deployment was priced. The charges were already written by then, which is why the run
looked like it had produced its rows.
The in-memory table these tests run against accepts any mapping, so only a live run
caught it. A predicate builder now returns a plain dict and is asserted as one, and the
catch-up pass has a test covering a config-declared reservation.
* refactor(ptu): build the prune predicate in one shot
Both filter shapes are known upfront, so the bounded one is constructed
directly rather than by mutating a value already declared Final.
The catch-up test took two independent clock reads, which disagree across
UTC midnight; it now derives both the reservation start and the expected
last charged day from a single read, matching the three sibling tests.
A keyring backend can accept a write and keep nothing. That is exactly what
`keyring --disable` and PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring
select, and it raises nothing to distinguish itself, so `lite login` was
handing the credential to a black hole, scrubbing its own copy from
token.json, and printing a success message over a login that no longer
worked. Reading the value back is the only way to tell that backend apart
from a keychain that really stored the secret.
The same rule closes the rest of the gaps. A credential the token file will
not record is taken back out of the keychain instead of being left live on a
machine with no record of it, and is reported rather than raised. The
migration stages its scrubbed file before the keychain is handed anything,
so a directory that will not accept the rewrite stops the move rather than
leaving the secret in two places. Logout no longer reads a key in the file
as proof that the keychain is clear, which was never sound across two
separate runs, and only draws that conclusion when the `keyring` package is
missing outright, where nothing could have reached a keychain at all.
Migration moved the secret into the keychain and then suppressed any OSError from
rewriting token.json, so a file that could not be rewritten kept the credential in
cleartext while every command reported success. That file is now removed instead:
signing in again costs one command, a stranded live credential costs the credential
`lite logout` also reported a clean logout whenever the keyring package was missing,
on the reasoning that an install without it could never have stored anything. The
entry belongs to the OS, so a keychain-backed login survives a logout run from a venv
without the cli extra. erase() now reports which keychain state applies, and logout
warns with the advice that fixes each one, staying quiet for file-backed logins whose
token file still carries its own secret
Also pins the migration path's tightening of a world-readable legacy token.json, and
moves the logout tests off patch() onto the injected vault
lite ships with every install of litellm, but the keyring package it needs
for keychain storage only ships with the cli extra. Such a user on a Mac was
told 'No OS keychain available' about a machine that plainly has one, with
nothing pointing at the missing package.
The vault now reports which of the three unusable states it is in, so login
can point at the install, name the kill switch, or report a genuinely absent
keychain.
* feat(ptu): accrue flat cost for PTU deployments declared in config.yaml
The flat-cost rollup reads deployments from LiteLLM_ProxyModelTable, and config.yaml
models never reach that table by design, so a PTU deployment declared there accrued no
flat cost at all while still billing its traffic per token. The provider bills the
reservation whichever file declared it.
The rollup now also reads the deployments the router holds that no database row owns,
identified by db_model, skipping the per-request credential clones that carry
original_model_id and reuse their source's PTU config under a fresh id. Registering such
a deployment zeroes its pricing, since reserved capacity already pays for the traffic it
serves, and leaving a rate unset falls back to the public cost map, which makes the double
charge the default rather than an opt-in.
The rules both halves apply now live in one module. The rollup's test for what it will
charge and the router's test for what to zero have to agree, or a deployment one accepts
and the other declines serves its traffic for free. That module also owns the fields the
write endpoints already zero, so the two paths cannot drift: tiered_pricing is emptied
rather than zeroed because its tiers outrank the rates beside them, the search context
table is written zeroed because an absent one means the provider default, and any further
rate the deployment itself declares is zeroed alongside the standing set.
The prune is bounded to the deployments a run scanned, but only for a run that priced a
config-declared deployment. Deciding a row is garbage on staleness alone stays correct
while every run derives its charges from the same table, so a database-only run sweeps
exactly as it did before; once one host's charges come from a file the others cannot read,
a row it never considered is not evidence of anything.
Behaviour change worth calling out: a zeroed deployment sorts ahead of an unpriced sibling
in QualityRouter's cost tiebreak, where an unset rate previously sorted last. Reserved
capacity really is the cheaper choice, but the ordering moves.
* refactor(ptu): drop a Final rebind and two redundant isinstance guards
The basedpyright budget rejected reassigning a Final in the datetime coercion and
two isinstance calls the router entry's own type already guarantees. Filtering the
built records rather than the raw entries removes both guards and leaves
_router_deployment as the single validator.
lite login used to write the minted cli-session key in cleartext to
~/.litellm/token.json. The secret material (key plus any JWT) now goes
to the OS keychain through the optional keyring package, with the 0600
file kept for non-secret metadata and as the fallback on headless boxes.
Legacy plaintext files keep authenticating and are migrated into the
keychain, then scrubbed, on first read. A secret still on disk always
outranks the keychain entry, so a failed keychain write can never
resurrect a stale key. LITELLM_PROXY_API_KEY and --api-key precedence
is unchanged, lite logout clears both stores and warns when the
keychain will not release the entry, and ~/.litellm is created 0700
(tightened from 0755 where an older CLI left it broader).
LITELLM_CLI_DISABLE_KEYRING=1 forces the file fallback.
The field replaces the standard RPM/TPM checks for batch submissions, so a
key holder or team admin writing it could pick their own batch quota.
Mirrors the output-token-estimate admin gate: change-based, so resending
the stored value stays allowed, and enforced on key generate, update, bulk
team-key update, regenerate, and team new/update.
`ProxyConfig.get_config()` walked the parsed config and replaced every
`os.environ/<KEY>` string with `get_secret(value)` before anything initialized
the secret manager, so a key held only by the manager resolved to `None` and
that `None` was written back into the config. The later fallback in
`load_config` could not recover it, because the key now existed with a `None`
value.
Hoist the initialization into `get_config()`, ahead of the resolution pass, so
every entrypoint gets it: the CLI already did this itself, but the microservice
entrypoints (`gateway/main.py`, `backend/main.py`) uvicorn the app directly and
bypass the CLI. `load_config`'s own call is now redundant and is dropped, so
startup builds the manager once instead of building one and discarding it.
`get_config()` also runs on management-endpoint request paths, so this returns
early once a manager exists rather than rebuilding the client per request.
Also warn when a reference the manager would have been asked for resolves to
`None`. The reporter had no log line at all to work from. `get_secret` only
reaches the manager when reads are enabled and the name is in `hosted_keys`, so
`secret_manager_would_be_consulted` mirrors that gate and keeps the warning off
env-only references, which are expected rather than an error.
A Redis over-limit verdict now survives a failing rollback DECRBY instead of
escaping into the in-memory fallback and granting tokens the counter already
rejected; the unrolled increments expire with the TTL. pop_reservation now
falls through to the local record when the Redis pop succeeds but finds
nothing, so a reservation saved in memory after a transient Redis save
failure still refunds on cancel or completion.
In-memory grants now record an owner token, and a refund only debits local
counters when the popping worker is the one that granted them, so a terminal
response handled elsewhere can no longer shrink another worker's unrelated
fallback reservations. A Redis-granted refund that fails no longer falls back
to decrementing local counters either: the leaked Redis increments expire
with the TTL and only tighten the allowance.
Auth and pass-through failures reach post_call_failure_hook with the raw request body unstripped, so a client-supplied standard_logging_object could feed the new attribution fallback when the logging object carries none. Pop the key before the lift so only the logging object may supply it. Also coalesce a None recovered cost to 0.0 so the lift always overwrites any client-supplied response_cost, matching the merge base's clobber semantics.
Reserve-script failures now roll back the scopes already incremented before
re-raising into the in-memory fallback, so a partial redis outage no longer
leaks counter increments that shrink the shared allowance. Reservations
record which backend granted them, so a refund never debits redis counters
an in-memory grant did not charge. Terminal-status matching is now
case-insensitive because the Bedrock async-invoke retrieve path returns raw
AWS-cased statuses like Completed.
* fix(auth): resolve bare model names against wildcard deployments in model access groups
* test(e2e): cover model access group permission checks on keys and teams
`lite up` already patches ~/.claude/settings.json, but only for as long as it
runs in the foreground, and it restores the original file on exit. Users
proxying Claude Code through LiteLLM therefore have to re-wire it by hand after
every login.
--config-claude makes that write persistent. It reuses the settings shape
`lite up` writes (env.ANTHROPIC_BASE_URL plus an apiKeyHelper invocation),
preserves every unrelated key, creates the file when missing, and writes it
atomically with owner-only permissions. Plain `lite login` is unchanged.
Reaching the credential through apiKeyHelper rather than copying it into the
file means a later login refreshes it with no further action, and keeps the
short-lived CLI token out of settings.json entirely.
The shared parts of the settings-file handling move from up.py into a new
claude_settings.py, since up.py imports auth.py and so auth.py cannot import
up.py back. That module now also owns the registry of commands that can be
temporarily managing the file, so the persistent write refuses while either
`lite up` or `lite autoroute up` holds a backup it would later restore over
this write.
Because this write has no backup and no `lite down`, it is stricter than
`lite up` about the user's file: it writes through a symlinked settings.json
rather than replacing the link with a regular file, and it refuses rather than
silently discarding a non-object `env` value.
Also fixes the apiKeyHelper command itself: --base-url belongs to the
top-level `lite` group, so `lite auth print-token --base-url X` is rejected by
click with "No such option". Every settings file `lite up` has written carries
that malformed command, which makes the helper return nothing and every Claude
Code request lose its token. The existing tests only string-matched the
generated command, so the new tests parse it through the real CLI instead.
