litellm/scripts/install-cli.sh
Krrish Dholakia 52591ba1b5
feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy (#33249)
* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy

Lets a customer try litellm's complexity_router against models they already
have on their existing, unmodified production proxy, with no config.yaml
edits and no new infra. lite autoroute configure discovers accessible
models via /model_group/info and walks through tier assignment (plus
optional LLM classifier / semantic matching / adaptive selection); every
referenced model becomes its own litellm_proxy/<name> deployment forwarding
back to the real proxy with the real key, so every actual call, routed
completions, classifier calls, embedding calls, still lands on their real
proxy. lite autoroute up launches that generated config as an ephemeral
local proxy, patches ~/.claude/settings.json to point Claude Code at it, and
streams routing decisions live; Ctrl-C/SIGTERM (or lite autoroute down
after an unclean exit) restores everything.

Also adds lite model-groups list (a thin CLI wrapper over the existing
ModelGroupsManagementClient), and generalizes up.py's settings-backup/restore
helpers to take explicit paths so this feature can reuse them instead of
duplicating the logic.

Depends on litellm_lite_up_down (#33231) for that generalization.

* feat(cli): allow multiple models per autoroute tier

complexity_router already supports a pool of models per tier (randomly
picked per request; adaptive mode specifically needs a pool to choose
within), but the configure wizard only ever let you assign one. Tiers are
now a tuple of model names; the wizard prompt accepts comma-separated
indices to pick more than one per tier.

* feat(cli): fuzzy model picker and auto-route Claude Code to autorouter

Numbered-index selection didn't scale past a handful of models, so switch
the tier picker to InquirerPy's fzf-style fuzzy search. Also set
ANTHROPIC_DEFAULT_{SONNET,HAIKU,OPUS}_MODEL to "autorouter" in Claude
Code's settings, since Router resolves auto-router deployments by literal
model name with no wildcard support, so a "*" catch-all model_name would
never match real traffic.

* feat(cli): allow installing lite CLI from source via LITELLM_CLI_REF

Lets testers try an unreleased branch's CLI changes with the same
curl-piped installer, instead of waiting for a PyPI release.

* fix(ci): modernize type hints to clear ruff strict-rule budget

* fix(ci): bump httplib2 and setuptools to patched versions

Clears osv-scan findings for PYSEC-2026-3444 and PYSEC-2026-3447.

* fix(cli): write autoroute's secret-bearing files with mode 0600

commands.py wrote config.yaml (embeds the real proxy key) and Claude
Code's settings.json (embeds the ephemeral proxy's master key) with
plain open(), landing at the umask-derived default (commonly 0644)
until a later chmod call caught up. That window, and the missed case
where settings.json already exists (chmod never ran at all there),
left a credential-bearing file readable by another local account.

secure_create() fixes the mode via fchmod on the fd before any
content is written, covering both the brand-new-file and
already-exists cases, and commands.py/wizard.py now route their
sensitive writes through it.

* docs(cli): warn that a stale Claude Code session can leak to a squatted port

lite autoroute up's master key is embedded statically (unlike lite up's
apiKeyHelper, resolved per request), so a Claude Code session still
running after teardown keeps sending it, along with prompt content, to
a now-unbound loopback port that another local account can bind. This
is the same one-time-patch tradeoff lite up already accepts, just with
a static secret instead of a re-resolved one -- document it in the
README's Caveats section and surface it in the teardown message itself.

* fix(cli): address greptile review feedback on autoroute PR

- terminate the ephemeral proxy child process when its health check
  fails, instead of leaking an orphaned, unrecoverable process bound
  to the port
- replace bare assert isinstance checks (no-ops under python -O) with
  click.ClickException in the model-groups list and configure wizard
  code paths
- close launch_proxy's log file handle once the child process has
  inherited its fd, instead of leaking it
- add build_generated_proxy_config to config.py's __all__

* fix(cli): close TOCTOU window in lite up's settings backup write

write_backup wrote the backup (which can embed the original
apiKeyHelper/settings content) with plain open() + a chmod call after
the fact -- the same permissive-until-corrected window already fixed
for autoroute's config.yaml and Claude settings writes, and missed
entirely when the backup file already exists with broader permissions.

Moves secure_create (atomic-enough 0600 via fchmod before any content
is written) to up.py, the module both lite up and lite autoroute
share, and has autoroute/process.py import it from there instead of
keeping its own copy.

* fix(cli): refuse autoroute up when a stale backup exists from a crash

The pid-record check only catches a still-live duplicate process; a
SIGKILL'd `up` leaves no live pid but does leave AUTOROUTE_BACKUP_PATH
behind. Without this guard, a fresh `up` overwrote that backup with
the currently-patched Claude settings instead of the true originals,
so `down`/Ctrl-C would restore the wrong content permanently. up.py's
`lite up` already guards the analogous case; mirror it here.

* fix(cli): bind the ephemeral autoroute proxy to loopback only

proxy_cli.py defaults --host to 0.0.0.0 when not passed explicitly.
launch_proxy never passed it, so the ephemeral proxy -- despite every
base_url in this module being built from 127.0.0.1 -- was actually
reachable from other hosts on the network, including its
unauthenticated-until-config-lands routes before the master key is
wired in.
2026-07-15 14:07:52 -07:00

141 lines
6.5 KiB
Bash
Executable file

#!/usr/bin/env bash
# LiteLLM CLI Installer (the thin `lite` client)
# Usage: curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install-cli.sh | sh
#
# Installs only litellm[cli]: the `lite` command for authenticating to a LiteLLM
# proxy and running coding agents (lite claude / codex / opencode) through it.
# None of the proxy server runtime is pulled in. To run a proxy server instead,
# use scripts/install.sh, which installs litellm[proxy].
#
# Needs only curl: uv is bootstrapped if missing, and uv provisions a compatible
# Python itself (honouring litellm's requires-python), downloading a managed one
# when the host has no suitable interpreter.
#
# To try an unreleased branch instead of the latest PyPI release (for example, to
# QA a CLI feature before it ships), set LITELLM_CLI_REF to a branch, tag, or commit:
# curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/<branch>/scripts/install-cli.sh | \
# LITELLM_CLI_REF=<branch> sh
#
# NOTE: set -e without pipefail for POSIX sh compatibility (dash on Ubuntu/Debian
# ignores the shebang when invoked as `sh` and does not support `pipefail`).
set -eu
# Defaults to the PyPI release; LITELLM_CLI_REF opts into installing from source instead.
if [ -n "${LITELLM_CLI_REF:-}" ]; then
LITELLM_PACKAGE="litellm[cli] @ git+https://github.com/BerriAI/litellm.git@${LITELLM_CLI_REF}"
else
LITELLM_PACKAGE="litellm[cli]"
fi
UV_VERSION="0.10.9"
# ── colours ────────────────────────────────────────────────────────────────
if [ -t 1 ]; then
BOLD='\033[1m'
GREEN='\033[38;2;78;186;101m'
GREY='\033[38;2;153;153;153m'
RESET='\033[0m'
else
BOLD='' GREEN='' GREY='' RESET=''
fi
info() { printf "${GREY} %s${RESET}\n" "$*"; }
success() { printf "${GREEN} ✔ %s${RESET}\n" "$*"; }
header() { printf "${BOLD} %s${RESET}\n" "$*"; }
die() { printf "\n Error: %s\n\n" "$*" >&2; exit 1; }
# ── banner ─────────────────────────────────────────────────────────────────
echo ""
cat << 'EOF'
██╗ ██╗████████╗███████╗
██║ ██║╚══██╔══╝██╔════╝
██║ ██║ ██║ █████╗
██║ ██║ ██║ ██╔══╝
███████╗██║ ██║ ███████╗
╚══════╝╚═╝ ╚═╝ ╚══════╝
EOF
printf " ${BOLD}LiteLLM CLI Installer${RESET} ${GREY}the thin 'lite' client for your proxy${RESET}\n\n"
# ── OS detection ───────────────────────────────────────────────────────────
OS="$(uname -s)"
ARCH="$(uname -m)"
case "$OS" in
Darwin) PLATFORM="macOS ($ARCH)" ;;
Linux) PLATFORM="Linux ($ARCH)" ;;
*) die "Unsupported OS: $OS. LiteLLM supports macOS and Linux." ;;
esac
info "Platform: $PLATFORM"
# ── uv detection / install ────────────────────────────────────────────────
UV_BIN=""
CURRENT_UV_VERSION=""
for candidate in uv "$HOME/.local/bin/uv"; do
if command -v "$candidate" >/dev/null 2>&1; then
UV_BIN="$(command -v "$candidate")"
break
elif [ -x "$candidate" ]; then
UV_BIN="$candidate"
break
fi
done
if [ -n "$UV_BIN" ]; then
CURRENT_UV_VERSION="$("$UV_BIN" --version 2>/dev/null | awk '{print $2}' | head -1 || true)"
fi
if [ -z "$UV_BIN" ] || [ "${CURRENT_UV_VERSION:-}" != "$UV_VERSION" ]; then
header "Installing uv…"
if [ -n "${CURRENT_UV_VERSION:-}" ]; then
info "Upgrading uv from ${CURRENT_UV_VERSION} to ${UV_VERSION}"
fi
curl -LsSf "https://astral.sh/uv/${UV_VERSION}/install.sh" | env UV_NO_MODIFY_PATH=1 sh \
|| die "uv installation failed. Try manually: curl -LsSf https://astral.sh/uv/${UV_VERSION}/install.sh | sh"
UV_BIN="$HOME/.local/bin/uv"
fi
# ── install ────────────────────────────────────────────────────────────────
# --python-preference system: reuse a compatible system Python when present,
# otherwise download a managed one. Either way uv honours litellm's requires-python,
# so a too-old (3.9) or too-new (3.14+) system Python is skipped, not forced.
echo ""
if [ -n "${LITELLM_CLI_REF:-}" ]; then
header "Installing litellm[cli] from ${LITELLM_CLI_REF}"
else
header "Installing litellm[cli]…"
fi
echo ""
"$UV_BIN" tool install --python-preference system --force "${LITELLM_PACKAGE}" \
|| die "uv tool install failed. Try manually: $UV_BIN tool install '${LITELLM_PACKAGE}'"
# ── find the lite binary installed by uv tool ──────────────────────────────
SCRIPTS_DIR="$("$UV_BIN" tool dir --bin)"
LITE_BIN="${SCRIPTS_DIR}/lite"
if [ ! -x "$LITE_BIN" ]; then
die "lite binary not found after install. Try: $UV_BIN tool install '${LITELLM_PACKAGE}'"
fi
# ── success banner ─────────────────────────────────────────────────────────
echo ""
success "LiteLLM CLI installed"
installed_ver="$("$LITE_BIN" --version 2>&1 | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1 || true)"
[ -n "$installed_ver" ] && info "Version: $installed_ver"
# ── PATH hint ──────────────────────────────────────────────────────────────
if ! command -v lite >/dev/null 2>&1; then
info "Note: add lite to your PATH: export PATH=\"\$PATH:${SCRIPTS_DIR}\""
fi
# ── next steps ─────────────────────────────────────────────────────────────
echo ""
header "Next steps:"
echo ""
info " export LITELLM_PROXY_URL=https://your-proxy # point at your gateway"
info " lite login # authenticate via SSO"
info " lite claude # run Claude Code through the proxy"
echo ""
info "Docs: https://docs.litellm.ai/docs/proxy/management_cli"
echo ""