`lite up` already patches ~/.claude/settings.json, but only for as long as it
runs in the foreground, and it restores the original file on exit. Users
proxying Claude Code through LiteLLM therefore have to re-wire it by hand after
every login.
--config-claude makes that write persistent. It reuses the settings shape
`lite up` writes (env.ANTHROPIC_BASE_URL plus an apiKeyHelper invocation),
preserves every unrelated key, creates the file when missing, and writes it
atomically with owner-only permissions. Plain `lite login` is unchanged.
Reaching the credential through apiKeyHelper rather than copying it into the
file means a later login refreshes it with no further action, and keeps the
short-lived CLI token out of settings.json entirely.
The shared parts of the settings-file handling move from up.py into a new
claude_settings.py, since up.py imports auth.py and so auth.py cannot import
up.py back. That module now also owns the registry of commands that can be
temporarily managing the file, so the persistent write refuses while either
`lite up` or `lite autoroute up` holds a backup it would later restore over
this write.
Because this write has no backup and no `lite down`, it is stricter than
`lite up` about the user's file: it writes through a symlinked settings.json
rather than replacing the link with a regular file, and it refuses rather than
silently discarding a non-object `env` value.
Also fixes the apiKeyHelper command itself: --base-url belongs to the
top-level `lite` group, so `lite auth print-token --base-url X` is rejected by
click with "No such option". Every settings file `lite up` has written carries
that malformed command, which makes the helper return nothing and every Claude
Code request lose its token. The existing tests only string-matched the
generated command, so the new tests parse it through the real CLI instead.
The trailing-slash normalization test used gateway.litellm-sandbox.ai as
its base URL. Swap it for gateway.example.com so the test file does not
reference a real-looking hostname. The test is fully mocked, so the host
value has no effect on what is exercised.
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* fix(cli): hide codex and opencode from the lite command listings
They stay registered and invokable, so existing `lite codex` users keep
working; they just no longer show up in `lite --help` or the interactive
shell's command list.
* feat(cli): make the hidden lite command list configurable
codex and opencode are supported, so hardcoding them as hidden was wrong. Let deployments curate their own listing with `lite config set hidden_commands codex,opencode` instead; nothing is hidden by default and hidden commands stay invokable.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
os.exec* has no process-replacement semantics on Windows, so `lite claude`
printed its routing line and returned to the prompt while Claude Code was left
detached without a usable console. Windows now spawns the agent, waits for it,
and exits with the child's status. Batch shims such as the npm-installed
claude.cmd go through cmd.exe because CreateProcess cannot run them directly,
and that command line is emitted verbatim with every token quoted so a spaced
path or an argument holding a shell metacharacter cannot be re-parsed by the
command processor. POSIX keeps using os.execvpe unchanged.
Python keeps only the last binding for a name, so when a file defines the same
test twice the earlier one is unreachable. pytest cannot collect a function that
no longer exists, so nothing reports it and the file still looks like it covers
the scenario.
These ten are cases where the two definitions have different bodies, meaning a
real test was replaced rather than duplicated. Each is renamed to say what it
actually covers, which makes it reachable again:
- test_gemini_frequency_penalty: the dead copy checks the parameter is listed in
get_supported_openai_params for vertex_ai; the survivor checks get_optional_params
maps a value for gemini. Different function and different provider.
- test_async_log_success_event_adds_to_queue and the failure variant: the dead
copies run without mocking asyncio.create_task, so they exercise the real task
path the survivors mock out.
- test_async_send_batch_triggers_tasks: the dead copy asserts send is not awaited
directly; the survivor asserts create_task was called.
- test_model_id_in_required_metrics: the dead copy checks the model_id label on
twelve further metrics the survivor dropped.
- test_anthropic_messages_pt_file_block_preserves_cache_control: the dead copy
passes model and llm_provider explicitly and uses real base64 PDF content.
- test_translate_streaming_openai_chunk_to_anthropic_with_thinking: the dead copy
covers thinking_delta; the survivor covers signature_delta.
- test_client_initialization and test_client_without_api_key: the dead copies
assert the resource clients are wired with the right base URL and key; the
survivors only construct the object.
- test_client_initialization_strips_trailing_slash: the dead copy constructs
ModelsManagementClient directly rather than going through Client.
Verification: collecting the seven touched files gives 401 node IDs before and
411 after, the ten new names and nothing else, with nothing lost. All ten pass.
Running the touched files in full gives 299 passed, and test_optional_params.py
goes from 111 passed to 112.
Two further shadowed definitions were left alone rather than renamed: the dead
copies of test_prompt_caching and test_cost_calculator_with_base_model_with_router
have no assertions at all, one being a bare pass and the other a lone import, so
restoring them would add tests that cannot fail.
* feat(cli): read base_url from persistent config file
Adds a lite config command group (set/get/unset) backed by
~/.litellm/config.json so users no longer need to export
LITELLM_PROXY_URL in every shell session. Resolution order is
--base-url flag, then LITELLM_PROXY_URL, then the config file,
then the localhost default. A config-file base_url counts as an
explicit server choice for lite auth print-token, matching the
env var semantics it replaces.
* fix(cli): harden config persistence after review feedback
Rejects base_url values containing a query string or fragment,
including bare trailing ? or # which parse as empty but still
corrupt every joined request URL. Writes config.json and token.json
atomically through a shared write_private_json helper (0600 at
creation, fsync, os.replace) so an interrupted save can no longer
truncate the file or leave it world-readable. Warns on stderr when
an existing config file is invalid instead of silently ignoring it,
including invalid UTF-8. Resolves the eager --version flag through
the same env, config file, default chain as every other command,
and reads the config file once per invocation so base_url and
base_url_explicit always come from the same snapshot.
* fix(cli): resolve --version after option parsing
The eager --version callback ran before --base-url and --api-key were
parsed, so it could not see an explicitly named server. Combined with
the env fallback added for config-file support, that sent the resolved
API key to whichever server the config file pointed at even when the
user named a different one on the command line. Making the flag a
normal option and handling it in the group callback gives the version
request the same flag, env, config, default precedence as every other
command, and lets the stored-token lookup stay origin-checked.
_load_persisted_master_key documents leniency on any unreadable prior
state but only caught parse errors; a permissions failure or non-UTF-8
bytes in config.yaml crashed configure instead of skipping the
carry-forward. Catch OSError and UnicodeDecodeError too and pin the
undecodable-file case with a regression test.
lite autoroute up minted a fresh master key and picked a fresh OS-ephemeral
port on every run, so any client configured against one session (an
already-open Claude Code session, a hand-configured script) broke on the
next run. The port is now a stable default (5483, overridable with --port)
that refuses loudly when busy or when 4000 is requested, since proxy_cli
silently rebinds a busy 4000 to a random port. The master key is minted
once, persisted in the generated config.yaml, reused by every later up,
and carried forward when configure regenerates the config.
* fix(cli): force UTF-8 output so emoji don't crash the CLI on Windows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cli): drop dead flush calls flagged by review
* fix(cli): replace non-ASCII CLI output with ASCII so legacy Windows consoles don't crash
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(cli): add `lite up`/`lite down` to ambiently route Claude Code through the proxy
Patches ~/.claude/settings.json in place (env.ANTHROPIC_BASE_URL + apiKeyHelper
via `lite auth print-token`) so any `claude` session started afterward, from
any terminal, routes through the local LiteLLM proxy with no wrapper command
needed, unlike the existing `lite claude` subprocess-exec approach. Backs up
the original file first and restores it on Ctrl-C/SIGTERM, or via `lite down`
after an unclean exit. Cursor is not supported: no equivalent file-based config
to patch.
* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy (#33249)
* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy
Lets a customer try litellm's complexity_router against models they already
have on their existing, unmodified production proxy, with no config.yaml
edits and no new infra. lite autoroute configure discovers accessible
models via /model_group/info and walks through tier assignment (plus
optional LLM classifier / semantic matching / adaptive selection); every
referenced model becomes its own litellm_proxy/<name> deployment forwarding
back to the real proxy with the real key, so every actual call, routed
completions, classifier calls, embedding calls, still lands on their real
proxy. lite autoroute up launches that generated config as an ephemeral
local proxy, patches ~/.claude/settings.json to point Claude Code at it, and
streams routing decisions live; Ctrl-C/SIGTERM (or lite autoroute down
after an unclean exit) restores everything.
Also adds lite model-groups list (a thin CLI wrapper over the existing
ModelGroupsManagementClient), and generalizes up.py's settings-backup/restore
helpers to take explicit paths so this feature can reuse them instead of
duplicating the logic.
Depends on litellm_lite_up_down (#33231) for that generalization.
* feat(cli): allow multiple models per autoroute tier
complexity_router already supports a pool of models per tier (randomly
picked per request; adaptive mode specifically needs a pool to choose
within), but the configure wizard only ever let you assign one. Tiers are
now a tuple of model names; the wizard prompt accepts comma-separated
indices to pick more than one per tier.
* feat(cli): fuzzy model picker and auto-route Claude Code to autorouter
Numbered-index selection didn't scale past a handful of models, so switch
the tier picker to InquirerPy's fzf-style fuzzy search. Also set
ANTHROPIC_DEFAULT_{SONNET,HAIKU,OPUS}_MODEL to "autorouter" in Claude
Code's settings, since Router resolves auto-router deployments by literal
model name with no wildcard support, so a "*" catch-all model_name would
never match real traffic.
* feat(cli): allow installing lite CLI from source via LITELLM_CLI_REF
Lets testers try an unreleased branch's CLI changes with the same
curl-piped installer, instead of waiting for a PyPI release.
* fix(ci): modernize type hints to clear ruff strict-rule budget
* fix(ci): bump httplib2 and setuptools to patched versions
Clears osv-scan findings for PYSEC-2026-3444 and PYSEC-2026-3447.
* fix(cli): write autoroute's secret-bearing files with mode 0600
commands.py wrote config.yaml (embeds the real proxy key) and Claude
Code's settings.json (embeds the ephemeral proxy's master key) with
plain open(), landing at the umask-derived default (commonly 0644)
until a later chmod call caught up. That window, and the missed case
where settings.json already exists (chmod never ran at all there),
left a credential-bearing file readable by another local account.
secure_create() fixes the mode via fchmod on the fd before any
content is written, covering both the brand-new-file and
already-exists cases, and commands.py/wizard.py now route their
sensitive writes through it.
* docs(cli): warn that a stale Claude Code session can leak to a squatted port
lite autoroute up's master key is embedded statically (unlike lite up's
apiKeyHelper, resolved per request), so a Claude Code session still
running after teardown keeps sending it, along with prompt content, to
a now-unbound loopback port that another local account can bind. This
is the same one-time-patch tradeoff lite up already accepts, just with
a static secret instead of a re-resolved one -- document it in the
README's Caveats section and surface it in the teardown message itself.
* fix(cli): address greptile review feedback on autoroute PR
- terminate the ephemeral proxy child process when its health check
fails, instead of leaking an orphaned, unrecoverable process bound
to the port
- replace bare assert isinstance checks (no-ops under python -O) with
click.ClickException in the model-groups list and configure wizard
code paths
- close launch_proxy's log file handle once the child process has
inherited its fd, instead of leaking it
- add build_generated_proxy_config to config.py's __all__
* fix(cli): close TOCTOU window in lite up's settings backup write
write_backup wrote the backup (which can embed the original
apiKeyHelper/settings content) with plain open() + a chmod call after
the fact -- the same permissive-until-corrected window already fixed
for autoroute's config.yaml and Claude settings writes, and missed
entirely when the backup file already exists with broader permissions.
Moves secure_create (atomic-enough 0600 via fchmod before any content
is written) to up.py, the module both lite up and lite autoroute
share, and has autoroute/process.py import it from there instead of
keeping its own copy.
* fix(cli): refuse autoroute up when a stale backup exists from a crash
The pid-record check only catches a still-live duplicate process; a
SIGKILL'd `up` leaves no live pid but does leave AUTOROUTE_BACKUP_PATH
behind. Without this guard, a fresh `up` overwrote that backup with
the currently-patched Claude settings instead of the true originals,
so `down`/Ctrl-C would restore the wrong content permanently. up.py's
`lite up` already guards the analogous case; mirror it here.
* fix(cli): bind the ephemeral autoroute proxy to loopback only
proxy_cli.py defaults --host to 0.0.0.0 when not passed explicitly.
launch_proxy never passed it, so the ephemeral proxy -- despite every
base_url in this module being built from 127.0.0.1 -- was actually
reachable from other hosts on the network, including its
unauthenticated-until-config-lands routes before the master key is
wired in.
* docs(cli): show curl install for the autoroute QA flow
Points readers at scripts/install-cli.sh's curl one-liner instead of
assuming uv/pip is already set up, and documents the LITELLM_CLI_REF
override for trying an unreleased branch or commit.
* fix(cli): surface a clean error on an empty or corrupt autoroute config
A configure run killed between secure_create's O_TRUNC and the write
completing leaves an empty config.yaml on disk. The next up read that
via yaml.safe_load (None) into the generated-config TypeAdapter
uncaught, surfacing a raw pydantic.ValidationError instead of pointing
the user back at `lite autoroute configure`.
* fix(cli): bind lite up's apiKeyHelper to the proxy it was started against
_ensure_fresh_login only checked token freshness, not which proxy the
cached token belonged to, and resolve_api_key_helper built a bare
`lite auth print-token` command with no --base-url. A user logged into
proxy A who ran `up --base-url proxy-b` (or LITELLM_PROXY_URL=proxy-b)
would silently get proxy A's real token wired into Claude Code's
apiKeyHelper; since apiKeyHelper is invoked bare, print-token's
existing origin check never engaged, so proxy B -- attacker-controlled
or not -- received every subsequent request's Authorization header
carrying proxy A's credential.
_ensure_fresh_login now requires the cached token's base_url to match
before treating it as usable, forcing a fresh login for the selected
proxy otherwise. resolve_api_key_helper now takes that base_url and
threads it through as an explicit --base-url, so print-token's
existing (but previously unreachable in the apiKeyHelper flow)
base_url_explicit check actually enforces the match at request time
too.
* fix(cli): surface clean errors instead of raw tracebacks in lite up/down
load_json_or_empty and read_backup both delegate to pydantic's
validate_json, which raises ValidationError on invalid JSON or a
non-object root -- neither up() nor down() caught it, so a corrupt
settings or backup file surfaced an unformatted Python traceback
instead of a clean CLI error. Both now convert to UpError, and down()
(previously uncaught entirely) and up()'s teardown path now handle it.
restore_claude_settings also gained a parent.mkdir guard before
rewriting CLAUDE_SETTINGS_PATH: if ~/.claude/ was removed while `lite
up` was running, the restore would crash before deleting the backup
file, permanently stranding it and breaking every future `lite down`.
* docs(cli): call out env-var auth for autoroute commands
* fix(cli): clean up leaked proxy and surface clean errors in autoroute
Three related gaps, all following an UpError getting raised somewhere
that wasn't catching it yet:
- up() left the just-launched ephemeral proxy running with no pid
record if load_json_or_empty/write_backup/secure_create raised after
the health check passed, mirroring the existing ProcessLaunchError
cleanup for the health-check-failure branch.
- _teardown() didn't catch restore_claude_settings raising UpError
(e.g. a corrupt backup at stop time), which would otherwise escape
to Click as an unhandled error in the normal-exit path, or print
"Error in atexit" in the atexit path. up.py's own _restore_once
handles the identical case the same way.
- read_pid_record let a corrupt PID file surface a raw
pydantic.ValidationError instead of a clean message, and did so in
down(), the command specifically meant for crash recovery. down()
now clears an unreadable pid record and continues cleanup instead of
aborting, since a corrupt pid file must never block the one command
meant to recover from exactly this kind of crash.
* docs(cli): warn against running lite up and lite autoroute up together
* fix(proxy): tell outdated litellm CLIs to upgrade when CLI SSO login id is legacy sk- format
* fix(cli): surface server error detail when SSO login polling fails and stop on permanent 4xx
* fix(cli): exhaustive, actionable error handling across the CLI SSO login flow
* feat: add silent CLI token refresh for apiKeyHelper support
lite auth print-token prints a valid proxy credential for use as Claude
Code's apiKeyHelper, transparently refreshing it first if the cached JWT
is stale. This unblocks MDM-managed apiKeyHelper deployments (managed
via `lite auth print-token`) that need silent mid-session credential
rotation without restarting the client.
Refresh capability is backed by a virtual key minted with an empty model
list and cli_refresh metadata, kept strictly separate from the actual
(short-lived, real-model-scoped) call credential -- so a leak of the
credential that flows through every LLM request and subprocess env var
can't also self-renew. The refresh flow is single-use: /sso/cli/refresh
mints a fresh JWT + refresh token pair and blocks the presented refresh
token immediately, so a replay can't mint a second pair from it.
Server: /sso/cli/refresh (rotate) and /sso/cli/logout (revoke) endpoints.
lite login now also stores a refresh token; lite logout revokes it
server-side instead of only clearing the local file.
* fix: allow non-admin users to hit CLI refresh routes; resolve apiKeyHelper base_url from token.json
Found via a live end-to-end test against a real proxy + real Claude Code
session: /sso/cli/refresh and /sso/cli/logout were unreachable for any
non-proxy-admin caller, since Depends(user_api_key_auth) pulls in a
route-RBAC gate that 403s any route not on an explicit allowlist. That
made the feature unusable for actual end users, who authenticate as
internal_user. Add both routes to internal_user_routes; the handlers
already do their own fine-grained check (metadata.cli_refresh) same as
/key/block does today.
Also: `lite auth print-token` required an explicit --base-url/
LITELLM_PROXY_URL matching the stored token's origin, defaulting to
localhost:4000 otherwise. But apiKeyHelper is configured bare (no
flags), so this always mismatched a real deployment. Track whether
--base-url was explicitly passed (via click's ParameterSource) and, if
not, resolve the server from token.json directly instead of the CLI
default.
* test: mock refresh-token minting in test_cli_poll_key_tolerates_missing_user_row
Landed on litellm_internal_staging after this branch's refresh-key minting
change; needs the same mock as the other cli_poll_key tests since minting
now runs unconditionally whenever a JWT is generated.
* fix(ci): update test_cli_auth.py for refresh_token contract, regenerate schema.d.ts
_poll_for_authentication now always includes "refresh_token" in its
returned dict, and _handle_team_selection_during_polling returns a dict
instead of a bare JWT string -- test_cli_auth.py predates this branch's
refresh-token work and still asserted the old shapes.
schema.d.ts regenerated via `npm run gen:api` to pick up the new
/sso/cli/refresh and /sso/cli/logout routes (plus unrelated drift from
other PRs merged since it was last generated).
* fix(ci): apply CI's own schema.d.ts diff (enterprise routes I can't generate locally)
Local `npm run gen:api` only sees OSS routes -- this machine's
litellm_enterprise editable install points at a now-deleted temp
directory, so it silently drops enterprise-only routes from the spec.
Applied the exact diff CI's own generation produced instead of
re-running the generator locally.
* fix: close refresh-token race, fail closed on DB down, fix logout base_url
Addresses Greptile review findings on the CLI refresh-token PR:
- cli_refresh_token minted a new JWT + refresh token BEFORE blocking the
presented one. Two concurrent requests bearing the same refresh token
could both pass auth and both mint fresh pairs, yielding four live
credentials from one consumed token. Now the presented token is
consumed atomically first via update_many (only succeeding if it flips
blocked from False/None to True); the loser gets count=0 and is
rejected before anything is minted.
- When prisma_client is None, refresh silently returned a new JWT
without ever being able to mark the presented token consumed, leaving
it valid indefinitely. Now fails closed with a 500 instead.
- `lite logout` sent its revocation POST to ctx.obj["base_url"], which
defaults to localhost:4000 when --base-url isn't passed -- the same
bug print_token had before the base_url_explicit fix, just missed
here. Now resolves the same way: trust the stored token's origin
unless the caller explicitly overrode --base-url.
* fix(ci): satisfy ruff format and narrow token_data type in logout
* fix(security): never trust refresh-token metadata for authorization
Addresses a real privilege-escalation path Veria flagged: cli_refresh_token
read team_id, team_alias, and max_budget straight off the presented
token's own metadata and used them to authorize the new JWT. Since any
authenticated user can self-mint a virtual key with arbitrary metadata
via the ordinary /key/generate endpoint, a self-forged key with
{"cli_refresh": true, "team_id": "<any-team>", "max_budget": 999999999}
would sail through _require_cli_refresh_token's only check
(metadata.cli_refresh == True) and get a JWT scoped to a team the
caller never belonged to, with a budget it never had -- full
cross-team / budget bypass, and a removed team member could keep
refreshing team-scoped sessions indefinitely.
Metadata's team_id is now treated as an untrusted UX hint only: honored
solely if the CALLER (identified by the authenticated key's own
user_id, not client input) is a current member per a fresh
get_user_object lookup. team_alias and max_budget are never read back
from metadata at all -- team_alias comes from a live get_team_object
lookup and max_budget is recomputed with the exact same capping logic
the initial SSO login poll uses. _mint_cli_refresh_token no longer
accepts or stores team_alias/max_budget, only the team_id hint.
Added regression tests proving: a forged/stale team_id is dropped
(falls back to no team, not silently honored), and a forged max_budget
in metadata never reaches the issued JWT.
* fix(ci): catch HTTPException specifically instead of bare Exception (BLE001)
* fix: un-consume refresh token if minting the replacement fails
Greptile flagged a real reliability gap: cli_refresh_token blocks the
presented token atomically, then does several more DB calls before
returning a replacement (user lookup, team lookup, JWT mint, new
refresh-key mint). Since this endpoint exists specifically for fully
unattended apiKeyHelper operation, a single transient failure in that
window (DB hiccup, etc.) permanently stranded the user: their old
token was already dead and no new one was issued, with no recovery
path short of a full interactive browser re-login.
Wrap that window in try/except; on any failure, best-effort revert the
consumed token back to usable (blocked=False) before re-raising, so a
retry can succeed. Standard compensating-action pattern since
generate_key_helper_fn doesn't take an injectable transaction, so
wrapping the whole thing in a real DB transaction isn't practical here.
* fix(security): refresh key had unrestricted model access, not none
Critical bug: _mint_cli_refresh_token used models=[] intending "no LLM
access", but that's backwards in this codebase. Per
_check_model_access_helper: `len(filtered_models) == 0 and len(models)
== 0` -> all_model_access = True. An empty models list on a key with no
team_id means UNRESTRICTED access to every model, not zero access. The
CLI refresh token -- meant to be usable for nothing but silently
exchanging itself for a new JWT -- was actually a fully unrestricted
API key for its entire 90-day lifetime, completely undermining the
whole point of keeping it separate from the short-lived call
credential.
Fixed with two independent layers: allowed_routes hard-restricts the
key to exactly /sso/cli/refresh and /sso/cli/logout (the real enforced
boundary, checked in the shared user_api_key_auth dependency for every
route); models is set to an unmatchable sentinel string as
defense-in-depth in case any code path only consults the models field.
Added an end-to-end regression test that exercises the actual
model-access-control function against a key shaped like the minted
refresh token, rather than only asserting on what arguments were passed
to the key-generation call -- the latter kind of test is exactly what
let the original bug ship, since asserting `models == []` is equally
consistent with "no access" and "unrestricted access" without checking
what the access-control code actually does with that shape.
Also: the compensating-rollback added for reliability un-blocked a
consumed refresh token even when the underlying user no longer exists.
That's a permanent, intentional rejection, not a transient failure --
un-blocking it would let a stale refresh token become valid again for a
different account if the user_id is ever reused/re-registered. Moved
the user-existence check outside the rollback-on-failure block so it
stays permanently blocked.
* refactor: rotate CLI refresh tokens via regenerate_key_fn instead of hand-rolled consume/rollback
The refresh token is already a plain litellm virtual key, so rotation can
delegate to the same atomic DB update /key/regenerate uses instead of a
bespoke update_many + compensating-rollback dance. This makes silent CLI
refresh an Enterprise feature, same as regular key regeneration.
* refactor: replace CLI stateless JWT + refresh-key pair with one self-rotating virtual key
The CLI previously minted two credentials on login: a stateless self-signed
JWT for LLM calls, and a separate DB-backed refresh-only key (scoped away
from ever calling an LLM) just to authorize minting a new JWT. Collapse
this into a single real virtual key, used directly as the LLM bearer token
and re-presented to /sso/cli/refresh to rotate its own secret in place.
This also means the CLI session key now shows up in the Admin UI's Keys
page and can be revoked/regenerated like any other key, rather than being
an invisible, unmanageable stateless token.
* refactor: drop silent CLI refresh, key just expires and requires re-login
/sso/cli/refresh only ever benefited Enterprise deployments (regenerate_key_fn's
gate), while everyone else already fell through to "re-run lite login" on
failure. Cut the endpoint, the rotation logic, and the client-side refresh
path entirely; print-token now just prints the cached key until it hits its
LITELLM_CLI_JWT_EXPIRATION_HOURS duration, then fails fast telling the user
to log in again. Session key itself is unaffected: still a real, revocable
virtual key visible in the Keys UI, `lite logout` still revokes it directly.
* fix(ci): regenerate schema.d.ts after removing /sso/cli/refresh route
* revert: go back to stateless JWT, keep only lite auth print-token
The virtual-key redesign (revocable, Keys-UI-visible credential) wasn't
needed just to support print-token, and cost real server-side surface
(a mint path, a logout-revoke endpoint, migrated tests/docs) for a property
this repo doesn't need yet. Reverting cli_poll_key/_types.py/schema.d.ts
back to the original stateless-JWT design; the only durable addition from
this whole effort is `lite auth print-token` (reads the cached credential,
prints it while fresh, fails with a clear message once it's past
LITELLM_CLI_JWT_EXPIRATION_HOURS) plus the base_url_explicit plumbing it
needs. `lite logout` goes back to clearing the local file only, since a
stateless JWT can't be revoked server-side.
* refactor: move CLI token freshness check to cli_token_utils, drop unnecessary renames
Addresses review: the freshness check is a pure token-shape/timestamp
util, not command logic, so it belongs alongside the other SDK-level
CLI token helpers (load_cli_token, get_litellm_gateway_api_key) rather
than in commands/auth.py. Also reverted a few incidental jwt_token/
session_key variable and string renames that weren't load-bearing.
A trailing slash on --base-url (or LITELLM_PROXY_URL) produced
double-slash URLs like https://host//sso/cli/start, which 404s. Normalize
once in the CLI's top-level group callback so every subcommand benefits.
* fix(proxy/client): redact api key from key/info client error messages
The keys management client builds GET /key/info?key=<key> and lets the
requests HTTPError propagate. str(HTTPError) renders the failing request URL
verbatim ("... for url: .../key/info?key=sk-..."), so any caller that logs the
exception leaks the full key; the 401 branch leaked the same way through
UnauthorizedError(str(orig_exception))
Redact both branches with the existing redact_secrets helper so the
secret-bearing query param is scrubbed to ?REDACTED while the status code,
reason, and response object are preserved. Server-side responses already mask
the key, so this closes the remaining client-side surface
* fix: preserve key info unauthorized response
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat(cli): add `litellm-proxy run -- <agent>` to wrap coding agents through the proxy
Wraps Claude Code, Codex, OpenCode, and any other coding agent so all of its
LLM traffic routes through a LiteLLM proxy, with the agent-vault style of "just
works" DX: one `run -- <agent>` command, auto SSO login when interactive,
env-key "agent mode" for containers/CI, and a fail-fast key check against the
proxy so bad credentials error immediately instead of deep inside the agent.
The wrapped binary is detected by name to pick the right variables. Claude Code
gets ANTHROPIC_BASE_URL (the bare proxy root, so it appends /v1/messages) and
ANTHROPIC_AUTH_TOKEN, with any stray ANTHROPIC_API_KEY cleared so the proxy
token wins. Codex and OpenCode get OPENAI_BASE_URL (proxy + /v1) and
OPENAI_API_KEY. Unrecognized commands get both sets so they work either way.
`litellm-proxy claude-code` remains as a shortcut for `run -- claude`.
The core logic is split into dependency-injected helpers (agent_profile,
build_agent_env, verify_proxy_key, run_agent) so env wiring, the preflight, and
the launch handoff are unit-tested without monkeypatching, alongside CliRunner
tests for auth resolution, agent mode, and auto-login. Mutation-tested the env
profiles, preflight, and agent-mode branch to confirm the tests fail when the
behavior is broken.
https://claude.ai/code/session_0154VpLXW7mMvk5wfbgPRJa6
* Make each coding agent its own litellm-proxy command
Replace the `run -- <agent>` interface and the `claude-code` shortcut with
top-level commands generated per known agent, so launching is just
`litellm-proxy claude`, `litellm-proxy codex`, or `litellm-proxy opencode`,
with everything after the agent name forwarded straight to it. This drops the
ceremony of `run --` and cuts typing.
The `--model`/`--small-fast-model` wrapper flags are gone; pass the agent's
own model flag instead, or export the model env vars (the wrapper preserves
what you already have set), which keeps the surface minimal and avoids
intercepting flags the agent owns. Rename the module to agents.py to match.
* fix(cli): route `litellm-proxy codex` through the proxy via a custom provider
Codex ignores OPENAI_BASE_URL (it always dials api.openai.com over the
Responses WebSocket transport), so the OpenAI env profile alone left
`litellm-proxy codex` talking to OpenAI directly instead of the proxy. Point
Codex at the proxy with a custom provider passed as `-c` config overrides, and
force the HTTP/SSE Responses transport with supports_websockets=false since the
proxy does not speak the Responses WebSocket protocol. The provider reads its
key from OPENAI_API_KEY, which the agent env already exports.
The overrides are injected ahead of the user's args so they precede Codex's
subcommand. Claude Code and OpenCode are unaffected; they honor the exported
env vars. Adds regression tests for the per-agent launch args and the
injection ordering.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* Rename litellm-proxy CLI command to lite
The proxy management CLI was invoked as litellm-proxy, which is a lot to
type for an everyday command. Rename the console script entry point to
lite and update the in-CLI usage examples, help text, error messages and
docs to match.
* fix(sso): stop CLI auth success page from hanging on "Closing..."
The CLI opens the SSO success page with webbrowser.open, so the tab is
not script-opened and the browser refuses window.close(). The countdown
would end on "Closing..." and the tab would sit there forever.
Drop the countdown and just show "You can now close this window and
return to your terminal." from the start, while still attempting
window.close() once so the tab auto-closes in the rare case the browser
allows it. Add a regression test asserting the manual-close instruction
is always present and the misleading countdown/"Closing..." text is gone.
* fix(cli): reattach controlling terminal after SSO login, keep litellm-proxy alias
When the first `lite claude` has to log in via browser SSO, completing the login could
leave stdin detached from the terminal, so a TUI agent like Claude Code would start in
non-interactive mode and exit with "Input must be provided". The wrapper now reopens the
controlling terminal onto stdin just before handoff when the session started interactively;
piped or redirected input is detected up front and left alone, so agent-mode and
non-interactive use are unchanged.
Also keep the `litellm-proxy` console script as an alias for `lite` so existing scripts and
CI that invoke `litellm-proxy` keep working; both names map to the same CLI.
* feat(install): make the curl installer need only curl, not a pre-existing Python
The installer now lets uv provision a managed Python 3.13 when no suitable
interpreter is found, instead of aborting. The minimum is also bumped from
3.9 to 3.10 to match the package's requires-python (>=3.10), so a system
Python 3.9 is no longer selected only for uv tool install to reject it.
* feat(cli): add thin litellm[cli] install path (install-cli.sh + brew) for the lite CLI
On a developer laptop the `lite` CLI only needs `lite login` and running coding
agents through a proxy, but the sole install path was `litellm[proxy]`, which
drags in the whole server tree (fastapi, uvicorn, boto3, polars, cryptography,
litellm-enterprise). The CLI's heavy imports are all guarded, so it runs on the
base SDK plus just rich, pyyaml and requests.
Add a `cli` extra carrying exactly those three, a `scripts/install-cli.sh` curl
one-liner that installs `litellm[cli]`, and a `BerriAI/homebrew-litellm` tap
formula with a release runbook under `packaging/homebrew/`. The installer passes
no `--python`, so uv honours litellm's requires-python and provisions a managed
interpreter, skipping a too-old (3.9) or too-new (3.14+) system Python instead
of failing to resolve.
A pyproject thin-contract test asserts the `cli` extra keeps the deps the CLI
imports and never leaks a server-only dependency from `proxy`, so the laptop
install cannot silently re-bloat
* fix(install): let uv pick the Python via --python-preference system
Both installers detected a system Python with a floor-only check and forced it
with `uv tool install --python <interp>`. On a host whose only Python is outside
litellm's requires-python (a too-old 3.9 or, increasingly, a too-new 3.14) that
forced an incompatible interpreter and the resolve failed. Drop the detection and
pass `--python-preference system`: uv reuses a compatible system Python when
present and downloads a managed one otherwise, always honouring requires-python
* test(router): filter aiohttp unclosed-session gc noise in test_async_fallbacks
test_async_fallbacks asserts the last three captured log records are the
router's fallback messages. Under the litellm_router_testing job (pytest -k
router -n 4) many router tests share the module-level in_memory_llm_clients_cache
(max 200, ttl 3600s). Older cached OpenAI/Azure clients get evicted while their
aiohttp ClientSession is still open, and when the gc reclaims them aiohttp emits
"Unclosed client session"/"Unclosed connector" through the asyncio logger.
Those records land in caplog mid-test and push the expected router logs out of
the last-three window, so the assertion flips to failing non-deterministically.
These warnings are async cleanup noise, not router debug logs, so filter them
out exactly like the existing leaked-task warnings before asserting order. The
assertion on the three router fallback messages is unchanged.
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
* fix: add expected_base_url origin check to get_litellm_gateway_api_key
* fix: scope get_stored_api_key and save base_url on login
* fix: pass base_url to get_stored_api_key in CLI entrypoint
* fix: scope ProxyClient stored key to base_url
* test: add expected_base_url coverage for get_stored_api_key
* fix: initialize self.http with resolved api_key not raw param
* fix: black formatting in client.py and test_auth_commands.py
* build: migrate packaging metadata to uv
* ci: move automation and local tooling to uv
* docker: migrate image builds and runtime setup to uv
* docs: update install and deployment guidance for uv
* chore: align auxiliary scripts and tests with uv
* test: harden test_litellm isolation
* fix: keep release and health check images self-contained
* build: pin uv tooling and health check deps
* test: isolate bedrock image request formatting from suite state
* test: cover sandbox executor requirements flow
* ci: fix circleci no-op command steps
* ci: fix circleci publish workflow parsing
* fix: stabilize remaining uv migration CI checks
* ci: increase matrix test timeout headroom
* fix: restore published docker and license coverage
* fix: restore proxy runtime build parity
* fix: restore proxy extras parity and venv migrations
* ci: persist uv path across circleci steps
* fix: keep psycopg binary in default test env
* docker: preserve prisma cache across stages
* test: run local proxy checks through uv python
* build: restore runtime deps moved into ci
* build: refresh uv lock after upstream merge
* fix: restore module import in test_check_migration after merge
The conflict resolution imported only the function but the test body
references check_migration as a module throughout.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching
- Move google-generativeai, Pillow, tenacity back to ci group (they are
lazily imported and bloat the base SDK install needlessly)
- Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant
in Docker where system Node.js is already installed via apk)
- Remove all nodejs-wheel node replacement and venv npm patching blocks
from Dockerfiles since the wheel is no longer installed
- Add --no-default-groups to CodSpeed benchmark workflow so the benchmark
environment matches the old minimal pip install footprint
- Apply standard uv two-phase Docker pattern: copy metadata first, install
deps (cached layer), then copy source and install project
- Replace CircleCI enterprise no-op with proper uv sync command
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate uv.lock after removing nodejs-wheel-binaries
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): use cache/restore instead of cache to prevent cache poisoning
The old workflow used actions/cache/restore (read-only). The uv migration
changed it to actions/cache (read-write), which zizmor flags as a cache
poisoning risk. Restore the safer read-only variant.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert
The setup-uv action enables caching by default, which zizmor flags as a
cache poisoning risk. Disable it since we already use a read-only
cache/restore step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): disable setup-uv cache in publish workflow
Silences zizmor cache-poisoning alert. Publishing workflow runs
infrequently on protected branches so caching adds no real benefit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(test): remove duplicate verbose_logger mock in test_check_migration
The logger was patched twice — first via mocker.patch() then via
mocker.patch.object(autospec=True). The second call fails because
autospec cannot inspect an already-mocked attribute. Remove the
redundant first patch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): free disk space before Docker build in test-server-root-path
The Dockerfile.non_root build ran out of disk on the CI runner. Remove
Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The test_keys_delete_error_handling test was failing with:
- ConnectionError when the mock wasn't properly applied
- The test was checking str(result.exception) without first verifying exception exists
This fix adds an explicit check that result.exception is not None before
attempting to convert it to string, preventing potential AttributeError
and making the test more robust.
