3 commits

Author	SHA1	Message	Date
Tin Chi Lo	b0899923f8	fix(install): pass an explicit Python version request to uv tool install ... uv selects an interpreter before resolving dependencies, so with no --python request the stock macOS /usr/bin/python3 (3.9.6) satisfies the unconstrained request and resolution then fails against litellm's requires-python (>=3.10,<3.15) instead of downloading a managed Python. Request the requires-python range explicitly in install-cli.sh and install.sh so uv reuses a compatible system interpreter when present and downloads a managed one otherwise. The manual-fallback hint in the die message carries the same flag so it no longer reproduces the failure.	2026-07-26 21:02:35 -07:00
Krrish Dholakia	cf90445574	feat(cli): add lite up/down to ambiently route Claude Code through the proxy (#33231) ... * feat(cli): add `lite up`/`lite down` to ambiently route Claude Code through the proxy Patches ~/.claude/settings.json in place (env.ANTHROPIC_BASE_URL + apiKeyHelper via `lite auth print-token`) so any `claude` session started afterward, from any terminal, routes through the local LiteLLM proxy with no wrapper command needed, unlike the existing `lite claude` subprocess-exec approach. Backs up the original file first and restores it on Ctrl-C/SIGTERM, or via `lite down` after an unclean exit. Cursor is not supported: no equivalent file-based config to patch. * feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy (#33249) * feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy Lets a customer try litellm's complexity_router against models they already have on their existing, unmodified production proxy, with no config.yaml edits and no new infra. lite autoroute configure discovers accessible models via /model_group/info and walks through tier assignment (plus optional LLM classifier / semantic matching / adaptive selection); every referenced model becomes its own litellm_proxy/<name> deployment forwarding back to the real proxy with the real key, so every actual call, routed completions, classifier calls, embedding calls, still lands on their real proxy. lite autoroute up launches that generated config as an ephemeral local proxy, patches ~/.claude/settings.json to point Claude Code at it, and streams routing decisions live; Ctrl-C/SIGTERM (or lite autoroute down after an unclean exit) restores everything. Also adds lite model-groups list (a thin CLI wrapper over the existing ModelGroupsManagementClient), and generalizes up.py's settings-backup/restore helpers to take explicit paths so this feature can reuse them instead of duplicating the logic. Depends on litellm_lite_up_down (#33231) for that generalization. * feat(cli): allow multiple models per autoroute tier complexity_router already supports a pool of models per tier (randomly picked per request; adaptive mode specifically needs a pool to choose within), but the configure wizard only ever let you assign one. Tiers are now a tuple of model names; the wizard prompt accepts comma-separated indices to pick more than one per tier. * feat(cli): fuzzy model picker and auto-route Claude Code to autorouter Numbered-index selection didn't scale past a handful of models, so switch the tier picker to InquirerPy's fzf-style fuzzy search. Also set ANTHROPIC_DEFAULT_{SONNET,HAIKU,OPUS}_MODEL to "autorouter" in Claude Code's settings, since Router resolves auto-router deployments by literal model name with no wildcard support, so a "*" catch-all model_name would never match real traffic. * feat(cli): allow installing lite CLI from source via LITELLM_CLI_REF Lets testers try an unreleased branch's CLI changes with the same curl-piped installer, instead of waiting for a PyPI release. * fix(ci): modernize type hints to clear ruff strict-rule budget * fix(ci): bump httplib2 and setuptools to patched versions Clears osv-scan findings for PYSEC-2026-3444 and PYSEC-2026-3447. * fix(cli): write autoroute's secret-bearing files with mode 0600 commands.py wrote config.yaml (embeds the real proxy key) and Claude Code's settings.json (embeds the ephemeral proxy's master key) with plain open(), landing at the umask-derived default (commonly 0644) until a later chmod call caught up. That window, and the missed case where settings.json already exists (chmod never ran at all there), left a credential-bearing file readable by another local account. secure_create() fixes the mode via fchmod on the fd before any content is written, covering both the brand-new-file and already-exists cases, and commands.py/wizard.py now route their sensitive writes through it. * docs(cli): warn that a stale Claude Code session can leak to a squatted port lite autoroute up's master key is embedded statically (unlike lite up's apiKeyHelper, resolved per request), so a Claude Code session still running after teardown keeps sending it, along with prompt content, to a now-unbound loopback port that another local account can bind. This is the same one-time-patch tradeoff lite up already accepts, just with a static secret instead of a re-resolved one -- document it in the README's Caveats section and surface it in the teardown message itself. * fix(cli): address greptile review feedback on autoroute PR - terminate the ephemeral proxy child process when its health check fails, instead of leaking an orphaned, unrecoverable process bound to the port - replace bare assert isinstance checks (no-ops under python -O) with click.ClickException in the model-groups list and configure wizard code paths - close launch_proxy's log file handle once the child process has inherited its fd, instead of leaking it - add build_generated_proxy_config to config.py's __all__ * fix(cli): close TOCTOU window in lite up's settings backup write write_backup wrote the backup (which can embed the original apiKeyHelper/settings content) with plain open() + a chmod call after the fact -- the same permissive-until-corrected window already fixed for autoroute's config.yaml and Claude settings writes, and missed entirely when the backup file already exists with broader permissions. Moves secure_create (atomic-enough 0600 via fchmod before any content is written) to up.py, the module both lite up and lite autoroute share, and has autoroute/process.py import it from there instead of keeping its own copy. * fix(cli): refuse autoroute up when a stale backup exists from a crash The pid-record check only catches a still-live duplicate process; a SIGKILL'd `up` leaves no live pid but does leave AUTOROUTE_BACKUP_PATH behind. Without this guard, a fresh `up` overwrote that backup with the currently-patched Claude settings instead of the true originals, so `down`/Ctrl-C would restore the wrong content permanently. up.py's `lite up` already guards the analogous case; mirror it here. * fix(cli): bind the ephemeral autoroute proxy to loopback only proxy_cli.py defaults --host to 0.0.0.0 when not passed explicitly. launch_proxy never passed it, so the ephemeral proxy -- despite every base_url in this module being built from 127.0.0.1 -- was actually reachable from other hosts on the network, including its unauthenticated-until-config-lands routes before the master key is wired in. * docs(cli): show curl install for the autoroute QA flow Points readers at scripts/install-cli.sh's curl one-liner instead of assuming uv/pip is already set up, and documents the LITELLM_CLI_REF override for trying an unreleased branch or commit. * fix(cli): surface a clean error on an empty or corrupt autoroute config A configure run killed between secure_create's O_TRUNC and the write completing leaves an empty config.yaml on disk. The next up read that via yaml.safe_load (None) into the generated-config TypeAdapter uncaught, surfacing a raw pydantic.ValidationError instead of pointing the user back at `lite autoroute configure`. * fix(cli): bind lite up's apiKeyHelper to the proxy it was started against _ensure_fresh_login only checked token freshness, not which proxy the cached token belonged to, and resolve_api_key_helper built a bare `lite auth print-token` command with no --base-url. A user logged into proxy A who ran `up --base-url proxy-b` (or LITELLM_PROXY_URL=proxy-b) would silently get proxy A's real token wired into Claude Code's apiKeyHelper; since apiKeyHelper is invoked bare, print-token's existing origin check never engaged, so proxy B -- attacker-controlled or not -- received every subsequent request's Authorization header carrying proxy A's credential. _ensure_fresh_login now requires the cached token's base_url to match before treating it as usable, forcing a fresh login for the selected proxy otherwise. resolve_api_key_helper now takes that base_url and threads it through as an explicit --base-url, so print-token's existing (but previously unreachable in the apiKeyHelper flow) base_url_explicit check actually enforces the match at request time too. * fix(cli): surface clean errors instead of raw tracebacks in lite up/down load_json_or_empty and read_backup both delegate to pydantic's validate_json, which raises ValidationError on invalid JSON or a non-object root -- neither up() nor down() caught it, so a corrupt settings or backup file surfaced an unformatted Python traceback instead of a clean CLI error. Both now convert to UpError, and down() (previously uncaught entirely) and up()'s teardown path now handle it. restore_claude_settings also gained a parent.mkdir guard before rewriting CLAUDE_SETTINGS_PATH: if ~/.claude/ was removed while `lite up` was running, the restore would crash before deleting the backup file, permanently stranding it and breaking every future `lite down`. * docs(cli): call out env-var auth for autoroute commands * fix(cli): clean up leaked proxy and surface clean errors in autoroute Three related gaps, all following an UpError getting raised somewhere that wasn't catching it yet: - up() left the just-launched ephemeral proxy running with no pid record if load_json_or_empty/write_backup/secure_create raised after the health check passed, mirroring the existing ProcessLaunchError cleanup for the health-check-failure branch. - _teardown() didn't catch restore_claude_settings raising UpError (e.g. a corrupt backup at stop time), which would otherwise escape to Click as an unhandled error in the normal-exit path, or print "Error in atexit" in the atexit path. up.py's own _restore_once handles the identical case the same way. - read_pid_record let a corrupt PID file surface a raw pydantic.ValidationError instead of a clean message, and did so in down(), the command specifically meant for crash recovery. down() now clears an unreadable pid record and continues cleanup instead of aborting, since a corrupt pid file must never block the one command meant to recover from exactly this kind of crash. * docs(cli): warn against running lite up and lite autoroute up together	2026-07-15 21:46:02 -07:00
Mateo Wang	20e453f698	feat(cli): per-agent lite claude / codex / opencode commands that wrap coding agents through the proxy (#29850) ... * feat(cli): add `litellm-proxy run -- <agent>` to wrap coding agents through the proxy Wraps Claude Code, Codex, OpenCode, and any other coding agent so all of its LLM traffic routes through a LiteLLM proxy, with the agent-vault style of "just works" DX: one `run -- <agent>` command, auto SSO login when interactive, env-key "agent mode" for containers/CI, and a fail-fast key check against the proxy so bad credentials error immediately instead of deep inside the agent. The wrapped binary is detected by name to pick the right variables. Claude Code gets ANTHROPIC_BASE_URL (the bare proxy root, so it appends /v1/messages) and ANTHROPIC_AUTH_TOKEN, with any stray ANTHROPIC_API_KEY cleared so the proxy token wins. Codex and OpenCode get OPENAI_BASE_URL (proxy + /v1) and OPENAI_API_KEY. Unrecognized commands get both sets so they work either way. `litellm-proxy claude-code` remains as a shortcut for `run -- claude`. The core logic is split into dependency-injected helpers (agent_profile, build_agent_env, verify_proxy_key, run_agent) so env wiring, the preflight, and the launch handoff are unit-tested without monkeypatching, alongside CliRunner tests for auth resolution, agent mode, and auto-login. Mutation-tested the env profiles, preflight, and agent-mode branch to confirm the tests fail when the behavior is broken. https://claude.ai/code/session_0154VpLXW7mMvk5wfbgPRJa6 * Make each coding agent its own litellm-proxy command Replace the `run -- <agent>` interface and the `claude-code` shortcut with top-level commands generated per known agent, so launching is just `litellm-proxy claude`, `litellm-proxy codex`, or `litellm-proxy opencode`, with everything after the agent name forwarded straight to it. This drops the ceremony of `run --` and cuts typing. The `--model`/`--small-fast-model` wrapper flags are gone; pass the agent's own model flag instead, or export the model env vars (the wrapper preserves what you already have set), which keeps the surface minimal and avoids intercepting flags the agent owns. Rename the module to agents.py to match. * fix(cli): route `litellm-proxy codex` through the proxy via a custom provider Codex ignores OPENAI_BASE_URL (it always dials api.openai.com over the Responses WebSocket transport), so the OpenAI env profile alone left `litellm-proxy codex` talking to OpenAI directly instead of the proxy. Point Codex at the proxy with a custom provider passed as `-c` config overrides, and force the HTTP/SSE Responses transport with supports_websockets=false since the proxy does not speak the Responses WebSocket protocol. The provider reads its key from OPENAI_API_KEY, which the agent env already exports. The overrides are injected ahead of the user's args so they precede Codex's subcommand. Claude Code and OpenCode are unaffected; they honor the exported env vars. Adds regression tests for the per-agent launch args and the injection ordering. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Rename litellm-proxy CLI command to lite The proxy management CLI was invoked as litellm-proxy, which is a lot to type for an everyday command. Rename the console script entry point to lite and update the in-CLI usage examples, help text, error messages and docs to match. * fix(sso): stop CLI auth success page from hanging on "Closing..." The CLI opens the SSO success page with webbrowser.open, so the tab is not script-opened and the browser refuses window.close(). The countdown would end on "Closing..." and the tab would sit there forever. Drop the countdown and just show "You can now close this window and return to your terminal." from the start, while still attempting window.close() once so the tab auto-closes in the rare case the browser allows it. Add a regression test asserting the manual-close instruction is always present and the misleading countdown/"Closing..." text is gone. * fix(cli): reattach controlling terminal after SSO login, keep litellm-proxy alias When the first `lite claude` has to log in via browser SSO, completing the login could leave stdin detached from the terminal, so a TUI agent like Claude Code would start in non-interactive mode and exit with "Input must be provided". The wrapper now reopens the controlling terminal onto stdin just before handoff when the session started interactively; piped or redirected input is detected up front and left alone, so agent-mode and non-interactive use are unchanged. Also keep the `litellm-proxy` console script as an alias for `lite` so existing scripts and CI that invoke `litellm-proxy` keep working; both names map to the same CLI. * feat(install): make the curl installer need only curl, not a pre-existing Python The installer now lets uv provision a managed Python 3.13 when no suitable interpreter is found, instead of aborting. The minimum is also bumped from 3.9 to 3.10 to match the package's requires-python (>=3.10), so a system Python 3.9 is no longer selected only for uv tool install to reject it. * feat(cli): add thin litellm[cli] install path (install-cli.sh + brew) for the lite CLI On a developer laptop the `lite` CLI only needs `lite login` and running coding agents through a proxy, but the sole install path was `litellm[proxy]`, which drags in the whole server tree (fastapi, uvicorn, boto3, polars, cryptography, litellm-enterprise). The CLI's heavy imports are all guarded, so it runs on the base SDK plus just rich, pyyaml and requests. Add a `cli` extra carrying exactly those three, a `scripts/install-cli.sh` curl one-liner that installs `litellm[cli]`, and a `BerriAI/homebrew-litellm` tap formula with a release runbook under `packaging/homebrew/`. The installer passes no `--python`, so uv honours litellm's requires-python and provisions a managed interpreter, skipping a too-old (3.9) or too-new (3.14+) system Python instead of failing to resolve. A pyproject thin-contract test asserts the `cli` extra keeps the deps the CLI imports and never leaks a server-only dependency from `proxy`, so the laptop install cannot silently re-bloat * fix(install): let uv pick the Python via --python-preference system Both installers detected a system Python with a floor-only check and forced it with `uv tool install --python <interp>`. On a host whose only Python is outside litellm's requires-python (a too-old 3.9 or, increasingly, a too-new 3.14) that forced an incompatible interpreter and the resolve failed. Drop the detection and pass `--python-preference system`: uv reuses a compatible system Python when present and downloads a managed one otherwise, always honouring requires-python * test(router): filter aiohttp unclosed-session gc noise in test_async_fallbacks test_async_fallbacks asserts the last three captured log records are the router's fallback messages. Under the litellm_router_testing job (pytest -k router -n 4) many router tests share the module-level in_memory_llm_clients_cache (max 200, ttl 3600s). Older cached OpenAI/Azure clients get evicted while their aiohttp ClientSession is still open, and when the gc reclaims them aiohttp emits "Unclosed client session"/"Unclosed connector" through the asyncio logger. Those records land in caplog mid-test and push the expected router logs out of the last-three window, so the assertion flips to failing non-deterministically. These warnings are async cleanup noise, not router debug logs, so filter them out exactly like the existing leaked-task warnings before asserting order. The assertion on the three router fallback messages is unchanged. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>	2026-06-10 13:52:26 -07:00