litellm/scripts/install-cli.sh
Krrish Dholakia cf90445574
feat(cli): add lite up/down to ambiently route Claude Code through the proxy (#33231)
* feat(cli): add `lite up`/`lite down` to ambiently route Claude Code through the proxy

Patches ~/.claude/settings.json in place (env.ANTHROPIC_BASE_URL + apiKeyHelper
via `lite auth print-token`) so any `claude` session started afterward, from
any terminal, routes through the local LiteLLM proxy with no wrapper command
needed, unlike the existing `lite claude` subprocess-exec approach. Backs up
the original file first and restores it on Ctrl-C/SIGTERM, or via `lite down`
after an unclean exit. Cursor is not supported: no equivalent file-based config
to patch.

* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy (#33249)

* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy

Lets a customer try litellm's complexity_router against models they already
have on their existing, unmodified production proxy, with no config.yaml
edits and no new infra. lite autoroute configure discovers accessible
models via /model_group/info and walks through tier assignment (plus
optional LLM classifier / semantic matching / adaptive selection); every
referenced model becomes its own litellm_proxy/<name> deployment forwarding
back to the real proxy with the real key, so every actual call, routed
completions, classifier calls, embedding calls, still lands on their real
proxy. lite autoroute up launches that generated config as an ephemeral
local proxy, patches ~/.claude/settings.json to point Claude Code at it, and
streams routing decisions live; Ctrl-C/SIGTERM (or lite autoroute down
after an unclean exit) restores everything.

Also adds lite model-groups list (a thin CLI wrapper over the existing
ModelGroupsManagementClient), and generalizes up.py's settings-backup/restore
helpers to take explicit paths so this feature can reuse them instead of
duplicating the logic.

Depends on litellm_lite_up_down (#33231) for that generalization.

* feat(cli): allow multiple models per autoroute tier

complexity_router already supports a pool of models per tier (randomly
picked per request; adaptive mode specifically needs a pool to choose
within), but the configure wizard only ever let you assign one. Tiers are
now a tuple of model names; the wizard prompt accepts comma-separated
indices to pick more than one per tier.

* feat(cli): fuzzy model picker and auto-route Claude Code to autorouter

Numbered-index selection didn't scale past a handful of models, so switch
the tier picker to InquirerPy's fzf-style fuzzy search. Also set
ANTHROPIC_DEFAULT_{SONNET,HAIKU,OPUS}_MODEL to "autorouter" in Claude
Code's settings, since Router resolves auto-router deployments by literal
model name with no wildcard support, so a "*" catch-all model_name would
never match real traffic.

* feat(cli): allow installing lite CLI from source via LITELLM_CLI_REF

Lets testers try an unreleased branch's CLI changes with the same
curl-piped installer, instead of waiting for a PyPI release.

* fix(ci): modernize type hints to clear ruff strict-rule budget

* fix(ci): bump httplib2 and setuptools to patched versions

Clears osv-scan findings for PYSEC-2026-3444 and PYSEC-2026-3447.

* fix(cli): write autoroute's secret-bearing files with mode 0600

commands.py wrote config.yaml (embeds the real proxy key) and Claude
Code's settings.json (embeds the ephemeral proxy's master key) with
plain open(), landing at the umask-derived default (commonly 0644)
until a later chmod call caught up. That window, and the missed case
where settings.json already exists (chmod never ran at all there),
left a credential-bearing file readable by another local account.

secure_create() fixes the mode via fchmod on the fd before any
content is written, covering both the brand-new-file and
already-exists cases, and commands.py/wizard.py now route their
sensitive writes through it.

* docs(cli): warn that a stale Claude Code session can leak to a squatted port

lite autoroute up's master key is embedded statically (unlike lite up's
apiKeyHelper, resolved per request), so a Claude Code session still
running after teardown keeps sending it, along with prompt content, to
a now-unbound loopback port that another local account can bind. This
is the same one-time-patch tradeoff lite up already accepts, just with
a static secret instead of a re-resolved one -- document it in the
README's Caveats section and surface it in the teardown message itself.

* fix(cli): address greptile review feedback on autoroute PR

- terminate the ephemeral proxy child process when its health check
  fails, instead of leaking an orphaned, unrecoverable process bound
  to the port
- replace bare assert isinstance checks (no-ops under python -O) with
  click.ClickException in the model-groups list and configure wizard
  code paths
- close launch_proxy's log file handle once the child process has
  inherited its fd, instead of leaking it
- add build_generated_proxy_config to config.py's __all__

* fix(cli): close TOCTOU window in lite up's settings backup write

write_backup wrote the backup (which can embed the original
apiKeyHelper/settings content) with plain open() + a chmod call after
the fact -- the same permissive-until-corrected window already fixed
for autoroute's config.yaml and Claude settings writes, and missed
entirely when the backup file already exists with broader permissions.

Moves secure_create (atomic-enough 0600 via fchmod before any content
is written) to up.py, the module both lite up and lite autoroute
share, and has autoroute/process.py import it from there instead of
keeping its own copy.

* fix(cli): refuse autoroute up when a stale backup exists from a crash

The pid-record check only catches a still-live duplicate process; a
SIGKILL'd `up` leaves no live pid but does leave AUTOROUTE_BACKUP_PATH
behind. Without this guard, a fresh `up` overwrote that backup with
the currently-patched Claude settings instead of the true originals,
so `down`/Ctrl-C would restore the wrong content permanently. up.py's
`lite up` already guards the analogous case; mirror it here.

* fix(cli): bind the ephemeral autoroute proxy to loopback only

proxy_cli.py defaults --host to 0.0.0.0 when not passed explicitly.
launch_proxy never passed it, so the ephemeral proxy -- despite every
base_url in this module being built from 127.0.0.1 -- was actually
reachable from other hosts on the network, including its
unauthenticated-until-config-lands routes before the master key is
wired in.

* docs(cli): show curl install for the autoroute QA flow

Points readers at scripts/install-cli.sh's curl one-liner instead of
assuming uv/pip is already set up, and documents the LITELLM_CLI_REF
override for trying an unreleased branch or commit.

* fix(cli): surface a clean error on an empty or corrupt autoroute config

A configure run killed between secure_create's O_TRUNC and the write
completing leaves an empty config.yaml on disk. The next up read that
via yaml.safe_load (None) into the generated-config TypeAdapter
uncaught, surfacing a raw pydantic.ValidationError instead of pointing
the user back at `lite autoroute configure`.

* fix(cli): bind lite up's apiKeyHelper to the proxy it was started against

_ensure_fresh_login only checked token freshness, not which proxy the
cached token belonged to, and resolve_api_key_helper built a bare
`lite auth print-token` command with no --base-url. A user logged into
proxy A who ran `up --base-url proxy-b` (or LITELLM_PROXY_URL=proxy-b)
would silently get proxy A's real token wired into Claude Code's
apiKeyHelper; since apiKeyHelper is invoked bare, print-token's
existing origin check never engaged, so proxy B -- attacker-controlled
or not -- received every subsequent request's Authorization header
carrying proxy A's credential.

_ensure_fresh_login now requires the cached token's base_url to match
before treating it as usable, forcing a fresh login for the selected
proxy otherwise. resolve_api_key_helper now takes that base_url and
threads it through as an explicit --base-url, so print-token's
existing (but previously unreachable in the apiKeyHelper flow)
base_url_explicit check actually enforces the match at request time
too.

* fix(cli): surface clean errors instead of raw tracebacks in lite up/down

load_json_or_empty and read_backup both delegate to pydantic's
validate_json, which raises ValidationError on invalid JSON or a
non-object root -- neither up() nor down() caught it, so a corrupt
settings or backup file surfaced an unformatted Python traceback
instead of a clean CLI error. Both now convert to UpError, and down()
(previously uncaught entirely) and up()'s teardown path now handle it.

restore_claude_settings also gained a parent.mkdir guard before
rewriting CLAUDE_SETTINGS_PATH: if ~/.claude/ was removed while `lite
up` was running, the restore would crash before deleting the backup
file, permanently stranding it and breaking every future `lite down`.

* docs(cli): call out env-var auth for autoroute commands

* fix(cli): clean up leaked proxy and surface clean errors in autoroute

Three related gaps, all following an UpError getting raised somewhere
that wasn't catching it yet:

- up() left the just-launched ephemeral proxy running with no pid
  record if load_json_or_empty/write_backup/secure_create raised after
  the health check passed, mirroring the existing ProcessLaunchError
  cleanup for the health-check-failure branch.
- _teardown() didn't catch restore_claude_settings raising UpError
  (e.g. a corrupt backup at stop time), which would otherwise escape
  to Click as an unhandled error in the normal-exit path, or print
  "Error in atexit" in the atexit path. up.py's own _restore_once
  handles the identical case the same way.
- read_pid_record let a corrupt PID file surface a raw
  pydantic.ValidationError instead of a clean message, and did so in
  down(), the command specifically meant for crash recovery. down()
  now clears an unreadable pid record and continues cleanup instead of
  aborting, since a corrupt pid file must never block the one command
  meant to recover from exactly this kind of crash.

* docs(cli): warn against running lite up and lite autoroute up together
2026-07-15 21:46:02 -07:00

141 lines
6.5 KiB
Bash
Executable file

#!/usr/bin/env bash
# LiteLLM CLI Installer (the thin `lite` client)
# Usage: curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install-cli.sh | sh
#
# Installs only litellm[cli]: the `lite` command for authenticating to a LiteLLM
# proxy and running coding agents (lite claude / codex / opencode) through it.
# None of the proxy server runtime is pulled in. To run a proxy server instead,
# use scripts/install.sh, which installs litellm[proxy].
#
# Needs only curl: uv is bootstrapped if missing, and uv provisions a compatible
# Python itself (honouring litellm's requires-python), downloading a managed one
# when the host has no suitable interpreter.
#
# To try an unreleased branch instead of the latest PyPI release (for example, to
# QA a CLI feature before it ships), set LITELLM_CLI_REF to a branch, tag, or commit:
# curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/<branch>/scripts/install-cli.sh | \
# LITELLM_CLI_REF=<branch> sh
#
# NOTE: set -e without pipefail for POSIX sh compatibility (dash on Ubuntu/Debian
# ignores the shebang when invoked as `sh` and does not support `pipefail`).
set -eu
# Defaults to the PyPI release; LITELLM_CLI_REF opts into installing from source instead.
if [ -n "${LITELLM_CLI_REF:-}" ]; then
LITELLM_PACKAGE="litellm[cli] @ git+https://github.com/BerriAI/litellm.git@${LITELLM_CLI_REF}"
else
LITELLM_PACKAGE="litellm[cli]"
fi
UV_VERSION="0.10.9"
# ── colours ────────────────────────────────────────────────────────────────
if [ -t 1 ]; then
BOLD='\033[1m'
GREEN='\033[38;2;78;186;101m'
GREY='\033[38;2;153;153;153m'
RESET='\033[0m'
else
BOLD='' GREEN='' GREY='' RESET=''
fi
info() { printf "${GREY} %s${RESET}\n" "$*"; }
success() { printf "${GREEN} ✔ %s${RESET}\n" "$*"; }
header() { printf "${BOLD} %s${RESET}\n" "$*"; }
die() { printf "\n Error: %s\n\n" "$*" >&2; exit 1; }
# ── banner ─────────────────────────────────────────────────────────────────
echo ""
cat << 'EOF'
██╗ ██╗████████╗███████╗
██║ ██║╚══██╔══╝██╔════╝
██║ ██║ ██║ █████╗
██║ ██║ ██║ ██╔══╝
███████╗██║ ██║ ███████╗
╚══════╝╚═╝ ╚═╝ ╚══════╝
EOF
printf " ${BOLD}LiteLLM CLI Installer${RESET} ${GREY}the thin 'lite' client for your proxy${RESET}\n\n"
# ── OS detection ───────────────────────────────────────────────────────────
OS="$(uname -s)"
ARCH="$(uname -m)"
case "$OS" in
Darwin) PLATFORM="macOS ($ARCH)" ;;
Linux) PLATFORM="Linux ($ARCH)" ;;
*) die "Unsupported OS: $OS. LiteLLM supports macOS and Linux." ;;
esac
info "Platform: $PLATFORM"
# ── uv detection / install ────────────────────────────────────────────────
UV_BIN=""
CURRENT_UV_VERSION=""
for candidate in uv "$HOME/.local/bin/uv"; do
if command -v "$candidate" >/dev/null 2>&1; then
UV_BIN="$(command -v "$candidate")"
break
elif [ -x "$candidate" ]; then
UV_BIN="$candidate"
break
fi
done
if [ -n "$UV_BIN" ]; then
CURRENT_UV_VERSION="$("$UV_BIN" --version 2>/dev/null | awk '{print $2}' | head -1 || true)"
fi
if [ -z "$UV_BIN" ] || [ "${CURRENT_UV_VERSION:-}" != "$UV_VERSION" ]; then
header "Installing uv…"
if [ -n "${CURRENT_UV_VERSION:-}" ]; then
info "Upgrading uv from ${CURRENT_UV_VERSION} to ${UV_VERSION}"
fi
curl -LsSf "https://astral.sh/uv/${UV_VERSION}/install.sh" | env UV_NO_MODIFY_PATH=1 sh \
|| die "uv installation failed. Try manually: curl -LsSf https://astral.sh/uv/${UV_VERSION}/install.sh | sh"
UV_BIN="$HOME/.local/bin/uv"
fi
# ── install ────────────────────────────────────────────────────────────────
# --python-preference system: reuse a compatible system Python when present,
# otherwise download a managed one. Either way uv honours litellm's requires-python,
# so a too-old (3.9) or too-new (3.14+) system Python is skipped, not forced.
echo ""
if [ -n "${LITELLM_CLI_REF:-}" ]; then
header "Installing litellm[cli] from ${LITELLM_CLI_REF}"
else
header "Installing litellm[cli]…"
fi
echo ""
"$UV_BIN" tool install --python-preference system --force "${LITELLM_PACKAGE}" \
|| die "uv tool install failed. Try manually: $UV_BIN tool install '${LITELLM_PACKAGE}'"
# ── find the lite binary installed by uv tool ──────────────────────────────
SCRIPTS_DIR="$("$UV_BIN" tool dir --bin)"
LITE_BIN="${SCRIPTS_DIR}/lite"
if [ ! -x "$LITE_BIN" ]; then
die "lite binary not found after install. Try: $UV_BIN tool install '${LITELLM_PACKAGE}'"
fi
# ── success banner ─────────────────────────────────────────────────────────
echo ""
success "LiteLLM CLI installed"
installed_ver="$("$LITE_BIN" --version 2>&1 | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1 || true)"
[ -n "$installed_ver" ] && info "Version: $installed_ver"
# ── PATH hint ──────────────────────────────────────────────────────────────
if ! command -v lite >/dev/null 2>&1; then
info "Note: add lite to your PATH: export PATH=\"\$PATH:${SCRIPTS_DIR}\""
fi
# ── next steps ─────────────────────────────────────────────────────────────
echo ""
header "Next steps:"
echo ""
info " export LITELLM_PROXY_URL=https://your-proxy # point at your gateway"
info " lite login # authenticate via SSO"
info " lite claude # run Claude Code through the proxy"
echo ""
info "Docs: https://docs.litellm.ai/docs/proxy/management_cli"
echo ""