mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
feat(cli): add lite up/down to ambiently route Claude Code through the proxy (#33231)
* feat(cli): add `lite up`/`lite down` to ambiently route Claude Code through the proxy Patches ~/.claude/settings.json in place (env.ANTHROPIC_BASE_URL + apiKeyHelper via `lite auth print-token`) so any `claude` session started afterward, from any terminal, routes through the local LiteLLM proxy with no wrapper command needed, unlike the existing `lite claude` subprocess-exec approach. Backs up the original file first and restores it on Ctrl-C/SIGTERM, or via `lite down` after an unclean exit. Cursor is not supported: no equivalent file-based config to patch. * feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy (#33249) * feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy Lets a customer try litellm's complexity_router against models they already have on their existing, unmodified production proxy, with no config.yaml edits and no new infra. lite autoroute configure discovers accessible models via /model_group/info and walks through tier assignment (plus optional LLM classifier / semantic matching / adaptive selection); every referenced model becomes its own litellm_proxy/<name> deployment forwarding back to the real proxy with the real key, so every actual call, routed completions, classifier calls, embedding calls, still lands on their real proxy. lite autoroute up launches that generated config as an ephemeral local proxy, patches ~/.claude/settings.json to point Claude Code at it, and streams routing decisions live; Ctrl-C/SIGTERM (or lite autoroute down after an unclean exit) restores everything. Also adds lite model-groups list (a thin CLI wrapper over the existing ModelGroupsManagementClient), and generalizes up.py's settings-backup/restore helpers to take explicit paths so this feature can reuse them instead of duplicating the logic. Depends on litellm_lite_up_down (#33231) for that generalization. * feat(cli): allow multiple models per autoroute tier complexity_router already supports a pool of models per tier (randomly picked per request; adaptive mode specifically needs a pool to choose within), but the configure wizard only ever let you assign one. Tiers are now a tuple of model names; the wizard prompt accepts comma-separated indices to pick more than one per tier. * feat(cli): fuzzy model picker and auto-route Claude Code to autorouter Numbered-index selection didn't scale past a handful of models, so switch the tier picker to InquirerPy's fzf-style fuzzy search. Also set ANTHROPIC_DEFAULT_{SONNET,HAIKU,OPUS}_MODEL to "autorouter" in Claude Code's settings, since Router resolves auto-router deployments by literal model name with no wildcard support, so a "*" catch-all model_name would never match real traffic. * feat(cli): allow installing lite CLI from source via LITELLM_CLI_REF Lets testers try an unreleased branch's CLI changes with the same curl-piped installer, instead of waiting for a PyPI release. * fix(ci): modernize type hints to clear ruff strict-rule budget * fix(ci): bump httplib2 and setuptools to patched versions Clears osv-scan findings for PYSEC-2026-3444 and PYSEC-2026-3447. * fix(cli): write autoroute's secret-bearing files with mode 0600 commands.py wrote config.yaml (embeds the real proxy key) and Claude Code's settings.json (embeds the ephemeral proxy's master key) with plain open(), landing at the umask-derived default (commonly 0644) until a later chmod call caught up. That window, and the missed case where settings.json already exists (chmod never ran at all there), left a credential-bearing file readable by another local account. secure_create() fixes the mode via fchmod on the fd before any content is written, covering both the brand-new-file and already-exists cases, and commands.py/wizard.py now route their sensitive writes through it. * docs(cli): warn that a stale Claude Code session can leak to a squatted port lite autoroute up's master key is embedded statically (unlike lite up's apiKeyHelper, resolved per request), so a Claude Code session still running after teardown keeps sending it, along with prompt content, to a now-unbound loopback port that another local account can bind. This is the same one-time-patch tradeoff lite up already accepts, just with a static secret instead of a re-resolved one -- document it in the README's Caveats section and surface it in the teardown message itself. * fix(cli): address greptile review feedback on autoroute PR - terminate the ephemeral proxy child process when its health check fails, instead of leaking an orphaned, unrecoverable process bound to the port - replace bare assert isinstance checks (no-ops under python -O) with click.ClickException in the model-groups list and configure wizard code paths - close launch_proxy's log file handle once the child process has inherited its fd, instead of leaking it - add build_generated_proxy_config to config.py's __all__ * fix(cli): close TOCTOU window in lite up's settings backup write write_backup wrote the backup (which can embed the original apiKeyHelper/settings content) with plain open() + a chmod call after the fact -- the same permissive-until-corrected window already fixed for autoroute's config.yaml and Claude settings writes, and missed entirely when the backup file already exists with broader permissions. Moves secure_create (atomic-enough 0600 via fchmod before any content is written) to up.py, the module both lite up and lite autoroute share, and has autoroute/process.py import it from there instead of keeping its own copy. * fix(cli): refuse autoroute up when a stale backup exists from a crash The pid-record check only catches a still-live duplicate process; a SIGKILL'd `up` leaves no live pid but does leave AUTOROUTE_BACKUP_PATH behind. Without this guard, a fresh `up` overwrote that backup with the currently-patched Claude settings instead of the true originals, so `down`/Ctrl-C would restore the wrong content permanently. up.py's `lite up` already guards the analogous case; mirror it here. * fix(cli): bind the ephemeral autoroute proxy to loopback only proxy_cli.py defaults --host to 0.0.0.0 when not passed explicitly. launch_proxy never passed it, so the ephemeral proxy -- despite every base_url in this module being built from 127.0.0.1 -- was actually reachable from other hosts on the network, including its unauthenticated-until-config-lands routes before the master key is wired in. * docs(cli): show curl install for the autoroute QA flow Points readers at scripts/install-cli.sh's curl one-liner instead of assuming uv/pip is already set up, and documents the LITELLM_CLI_REF override for trying an unreleased branch or commit. * fix(cli): surface a clean error on an empty or corrupt autoroute config A configure run killed between secure_create's O_TRUNC and the write completing leaves an empty config.yaml on disk. The next up read that via yaml.safe_load (None) into the generated-config TypeAdapter uncaught, surfacing a raw pydantic.ValidationError instead of pointing the user back at `lite autoroute configure`. * fix(cli): bind lite up's apiKeyHelper to the proxy it was started against _ensure_fresh_login only checked token freshness, not which proxy the cached token belonged to, and resolve_api_key_helper built a bare `lite auth print-token` command with no --base-url. A user logged into proxy A who ran `up --base-url proxy-b` (or LITELLM_PROXY_URL=proxy-b) would silently get proxy A's real token wired into Claude Code's apiKeyHelper; since apiKeyHelper is invoked bare, print-token's existing origin check never engaged, so proxy B -- attacker-controlled or not -- received every subsequent request's Authorization header carrying proxy A's credential. _ensure_fresh_login now requires the cached token's base_url to match before treating it as usable, forcing a fresh login for the selected proxy otherwise. resolve_api_key_helper now takes that base_url and threads it through as an explicit --base-url, so print-token's existing (but previously unreachable in the apiKeyHelper flow) base_url_explicit check actually enforces the match at request time too. * fix(cli): surface clean errors instead of raw tracebacks in lite up/down load_json_or_empty and read_backup both delegate to pydantic's validate_json, which raises ValidationError on invalid JSON or a non-object root -- neither up() nor down() caught it, so a corrupt settings or backup file surfaced an unformatted Python traceback instead of a clean CLI error. Both now convert to UpError, and down() (previously uncaught entirely) and up()'s teardown path now handle it. restore_claude_settings also gained a parent.mkdir guard before rewriting CLAUDE_SETTINGS_PATH: if ~/.claude/ was removed while `lite up` was running, the restore would crash before deleting the backup file, permanently stranding it and breaking every future `lite down`. * docs(cli): call out env-var auth for autoroute commands * fix(cli): clean up leaked proxy and surface clean errors in autoroute Three related gaps, all following an UpError getting raised somewhere that wasn't catching it yet: - up() left the just-launched ephemeral proxy running with no pid record if load_json_or_empty/write_backup/secure_create raised after the health check passed, mirroring the existing ProcessLaunchError cleanup for the health-check-failure branch. - _teardown() didn't catch restore_claude_settings raising UpError (e.g. a corrupt backup at stop time), which would otherwise escape to Click as an unhandled error in the normal-exit path, or print "Error in atexit" in the atexit path. up.py's own _restore_once handles the identical case the same way. - read_pid_record let a corrupt PID file surface a raw pydantic.ValidationError instead of a clean message, and did so in down(), the command specifically meant for crash recovery. down() now clears an unreadable pid record and continues cleanup instead of aborting, since a corrupt pid file must never block the one command meant to recover from exactly this kind of crash. * docs(cli): warn against running lite up and lite autoroute up together
This commit is contained in:
Krrish Dholakia
GitHub
parent
f6516e7be5
commit
cf90445574
23 changed files with 2869 additions and 75 deletions
|
|
@ -471,6 +471,107 @@ The token minted by `lite login` is a short-lived, per-session agent credential,
|
||||||
|
|
||||||
The credential is short-lived by design (default 24h, configurable via `LITELLM_CLI_JWT_EXPIRATION_HOURS`); run `lite login` again to refresh it, which also re-reads your latest team and user settings. It does not appear in the Keys UI and cannot be rotated or revoked mid-session. `lite auth print-token` (usable as Claude Code's `apiKeyHelper`) prints it while it's still fresh and fails once it expires -- there is no silent renewal, so a long-running session needs a fresh `lite login` once a day. `lite claude`, `lite codex`, and `lite opencode` work with it on a default deployment; `EXPERIMENTAL_UI_LOGIN` is not required. If you need a long-lived, rotatable key that shows up in the Keys UI, create a dedicated virtual key in the dashboard and pass it via `--api-key` or `LITELLM_PROXY_API_KEY` instead.
|
The credential is short-lived by design (default 24h, configurable via `LITELLM_CLI_JWT_EXPIRATION_HOURS`); run `lite login` again to refresh it, which also re-reads your latest team and user settings. It does not appear in the Keys UI and cannot be rotated or revoked mid-session. `lite auth print-token` (usable as Claude Code's `apiKeyHelper`) prints it while it's still fresh and fails once it expires -- there is no silent renewal, so a long-running session needs a fresh `lite login` once a day. `lite claude`, `lite codex`, and `lite opencode` work with it on a default deployment; `EXPERIMENTAL_UI_LOGIN` is not required. If you need a long-lived, rotatable key that shows up in the Keys UI, create a dedicated virtual key in the dashboard and pass it via `--api-key` or `LITELLM_PROXY_API_KEY` instead.
|
||||||
|
|
||||||
|
### Route Every Claude Code Session Through the Proxy
|
||||||
|
|
||||||
|
`lite claude` wraps a single invocation, but `lite up` goes further: it patches `~/.claude/settings.json`, Claude Code's own config file, so that every Claude Code session started afterward -- from any terminal, launched normally with just `claude`, no wrapper needed -- routes through your LiteLLM proxy. It sets `env.ANTHROPIC_BASE_URL` to the proxy URL and `apiKeyHelper` to a `lite auth print-token` invocation, drops any stray static `ANTHROPIC_API_KEY` so the helper-issued token wins, and leaves every other setting in the file untouched. It backs up the original file before patching it.
|
||||||
|
|
||||||
|
Two things need to already be true: you've run `lite login`, since the apiKeyHelper depends on that stored token, and the proxy is already reachable, since `lite up` does not start one for you.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lite login
|
||||||
|
litellm --config litellm/proxy/dev_config.yaml &
|
||||||
|
lite up
|
||||||
|
```
|
||||||
|
|
||||||
|
`lite up` runs in the foreground and blocks. Press Ctrl-C to stop it, which restores the original settings file and exits. If the process is ever killed uncleanly instead -- `kill -9`, a crash -- the settings file is left patched, and `lite down` is the manual recovery path: run it at any later point to restore from the same backup.
|
||||||
|
|
||||||
|
This is a one-time file patch and restore, not a live traffic interceptor. A Claude Code session already running before `lite up` started keeps whatever `ANTHROPIC_BASE_URL` and token it loaded at its own startup, and a session still running when `lite up` stops keeps routing through the proxy until it exits; only sessions *started* while the patch is in effect are affected, and only *new* sessions after a restore go back to Anthropic directly.
|
||||||
|
|
||||||
|
Cursor is not supported: it has no equivalent file-based config to hot-patch this way, since its model routing lives in its own app storage and is configured through its GUI.
|
||||||
|
|
||||||
|
### QA Complexity-Based Auto-Routing Against Your Real Proxy
|
||||||
|
|
||||||
|
`lite autoroute` lets you try LiteLLM's complexity-based auto-routing -- picking a cheaper or more expensive model depending on how complex a prompt looks -- against models your key already has access to on your real, running proxy, without editing that proxy's `config.yaml` and without any real request ever bypassing it. It builds a second, throwaway proxy locally that forwards every request back to your real proxy, and points Claude Code at that local proxy for the duration of the session.
|
||||||
|
|
||||||
|
#### Install the CLI
|
||||||
|
|
||||||
|
If you don't already have the `lite` command, install it with a single curl command -- no existing Python tooling required, `uv` is bootstrapped automatically if missing:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install-cli.sh | sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This installs only `litellm[cli]`, the thin client (`lite`), not the full proxy server. To try an unreleased branch or commit instead of the latest PyPI release, set `LITELLM_CLI_REF`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/<branch-or-commit>/scripts/install-cli.sh | \
|
||||||
|
LITELLM_CLI_REF=<branch-or-commit> sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Point the CLI at your real proxy and key before running any `lite model-groups` or `lite autoroute` command -- like every other command in this CLI, they read `LITELLM_PROXY_URL`/`LITELLM_PROXY_API_KEY` (or `--base-url`/`--api-key`), no `lite login` required:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export LITELLM_PROXY_URL=http://localhost:4000
|
||||||
|
export LITELLM_PROXY_API_KEY=sk-...
|
||||||
|
```
|
||||||
|
|
||||||
|
#### List Your Accessible Model Groups
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lite model-groups list [--format table|json]
|
||||||
|
```
|
||||||
|
|
||||||
|
Lists the model groups your key can reach on the proxy, via `/model_group/info`, along with each group's mode (`chat`, `embedding`, etc.) and per-token pricing. This is also what `lite autoroute configure` uses internally to discover what it can offer you.
|
||||||
|
|
||||||
|
#### Configure the Auto-Router
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lite autoroute configure
|
||||||
|
```
|
||||||
|
|
||||||
|
An interactive wizard. It runs the same model-group discovery as above, splits the results into chat-capable and embedding-capable pools, and asks you to assign one or more models from the chat pool to each of the four complexity tiers -- SIMPLE, MEDIUM, COMPLEX, REASONING. Each tier's picker is a type-to-filter fuzzy search (fzf-style) rather than a scrollable numbered list, so it stays usable even with hundreds of model groups: type a substring to narrow the list, tab to toggle a model into the selection, enter to confirm (assigning more than one model to a tier is exactly when this matters -- complexity_router picks randomly among a tier's pool per request, and adaptive mode specifically depends on having more than one candidate to choose from). From there it optionally offers: classifying prompt complexity with an LLM (again picked from your discovered pool) instead of the free built-in heuristic scorer, semantic keyword matching for tier assignment (needs an embedding model from the pool), and adaptive (bandit-based) selection layered on top of tiering.
|
||||||
|
|
||||||
|
The wizard writes the result to `~/.litellm/autorouter/config.yaml` with `0600` permissions, since the file embeds your real proxy API key. Every model referenced anywhere in that config -- tier targets, the classifier model, the embedding model -- becomes its own `litellm_proxy/<model-name>` deployment whose `api_base` and `api_key` point back at your real proxy. That is the trick that keeps your real proxy's config untouched: every actual network call this generates, whether it is the routed completion, an LLM-classifier call, or an embedding call, forwards transparently through your real, already-running proxy with your real key.
|
||||||
|
|
||||||
|
You do not need to tell Claude Code to request `autorouter` by name yourself: `lite autoroute up` also sets `ANTHROPIC_DEFAULT_SONNET_MODEL`, `ANTHROPIC_DEFAULT_HAIKU_MODEL`, and `ANTHROPIC_DEFAULT_OPUS_MODEL` to `autorouter` in `~/.claude/settings.json`, so every one of Claude Code's own model tiers requests it directly regardless of `/model` or whatever it defaults to otherwise. (A bare `model_name: "*"` deployment looks like the obvious way to catch any request instead, but litellm's Router looks up auto-router deployments by the literal requested model string with no wildcard resolution, so a `"*"` entry would never actually match real traffic -- these env var overrides are what makes it work.)
|
||||||
|
|
||||||
|
You must run `configure` at least once before `up`; running `up` first fails with a clear error telling you to configure first.
|
||||||
|
|
||||||
|
#### Launch the Ephemeral Auto-Router Proxy
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lite autoroute up
|
||||||
|
```
|
||||||
|
|
||||||
|
Starts a local, throwaway litellm proxy on a random free port, running the config `configure` generated, with a freshly-minted random API key baked in for this session only (your real proxy key never leaves the generated config -- it only appears there, forwarding to your real proxy). It waits for the ephemeral proxy to report healthy, then patches `~/.claude/settings.json` the same way `lite up` does, except with a static `ANTHROPIC_AUTH_TOKEN` env var instead of an `apiKeyHelper`, since this key is short-lived and self-issued rather than something needing SSO refresh. Any `claude` session started afterward, from any terminal, routes through the ephemeral proxy.
|
||||||
|
|
||||||
|
`lite autoroute up` runs in the foreground and streams the ephemeral proxy's own log file into your terminal, so you can watch its routing decisions -- which tier and model got picked for each request -- as you use Claude Code normally. Press Ctrl-C (or send SIGTERM) to stop it; this kills the child proxy process and restores your original Claude Code settings, in that order.
|
||||||
|
|
||||||
|
#### Recover From an Unclean Shutdown
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lite autoroute down
|
||||||
|
```
|
||||||
|
|
||||||
|
If the `lite autoroute up` process dies uncleanly -- `kill -9`, a crash -- rather than being stopped with Ctrl-C, `down` is the manual recovery path: it kills any leftover ephemeral proxy process found via a recorded pid file and restores Claude Code's settings from whatever backup is on disk.
|
||||||
|
|
||||||
|
#### Example
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lite autoroute configure
|
||||||
|
lite autoroute up
|
||||||
|
# use Claude Code as normal in another terminal; routing decisions stream live
|
||||||
|
lite autoroute down # only needed if `up` was killed uncleanly instead of Ctrl-C'd
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Caveats
|
||||||
|
|
||||||
|
Adaptive mode's learned state does not persist across `lite autoroute up` sessions -- there is no local database, so every session starts adaptive selection cold. A Claude Code session already running before `up` started, or still running when it stops, keeps whatever settings it loaded at its own startup; like `lite up`, this is a one-time file patch and restore, not a live traffic interceptor. Only Claude Code is supported, for the same reason as `lite up`: no other supported agent (for example Cursor) has an equivalent hot-patchable config file.
|
||||||
|
|
||||||
|
A session that outlives `up` (or is still running the moment you stop it) keeps sending requests, master key included, to that now-freed loopback port until you restart it. Once the ephemeral proxy process exits, nothing stops another local account on the same machine from binding that same port and receiving those requests instead -- unlike `lite up`'s `apiKeyHelper`, which is re-resolved per request, `autoroute`'s master key is a static value, so whoever receives them gets a live-looking token along with the prompt content. Restart any Claude Code session before you consider the machine clean, run `lite autoroute down` promptly rather than leaving a stopped session's settings patched, and do not run `lite autoroute up` on a shared or multi-tenant host.
|
||||||
|
|
||||||
|
Do not run `lite up` and `lite autoroute up` at the same time. Each patches `~/.claude/settings.json` and keeps its own separate backup, with no coordination between them: whichever one you stop or crash out of last is the one whose backup gets restored, which can silently leave the *other* mode's settings (a static master key and a now-dead loopback URL, or a stale `apiKeyHelper`) active. Run `lite down` or `lite autoroute down` (whichever applies) before switching to the other mode.
|
||||||
|
|
||||||
## Environment Variables
|
## Environment Variables
|
||||||
|
|
||||||
The CLI respects the following environment variables:
|
The CLI respects the following environment variables:
|
||||||
|
|
|
||||||
|
|
@ -212,7 +212,7 @@ def _is_interactive() -> bool:
|
||||||
return sys.stdin.isatty()
|
return sys.stdin.isatty()
|
||||||
|
|
||||||
|
|
||||||
def _resolve_api_key(ctx: click.Context) -> str:
|
def resolve_api_key(ctx: click.Context) -> str:
|
||||||
base_url = ctx.obj["base_url"]
|
base_url = ctx.obj["base_url"]
|
||||||
api_key = ctx.obj.get("api_key")
|
api_key = ctx.obj.get("api_key")
|
||||||
if api_key:
|
if api_key:
|
||||||
|
|
@ -238,7 +238,7 @@ _SKIP_VERIFY_HELP = "Skip the pre-launch key check against the proxy."
|
||||||
def _launch(ctx: click.Context, binary: str, args: Sequence[str], *, skip_verify: bool) -> None:
|
def _launch(ctx: click.Context, binary: str, args: Sequence[str], *, skip_verify: bool) -> None:
|
||||||
base_url = ctx.obj["base_url"]
|
base_url = ctx.obj["base_url"]
|
||||||
started_interactive = _is_interactive()
|
started_interactive = _is_interactive()
|
||||||
api_key = _resolve_api_key(ctx)
|
api_key = resolve_api_key(ctx)
|
||||||
|
|
||||||
display_name, _ = agent_profile(binary)
|
display_name, _ = agent_profile(binary)
|
||||||
click.echo(f"litellm: routing {display_name} through proxy at {base_url.rstrip('/')}")
|
click.echo(f"litellm: routing {display_name} through proxy at {base_url.rstrip('/')}")
|
||||||
|
|
@ -288,5 +288,6 @@ __all__ = [
|
||||||
"agent_launch_args",
|
"agent_launch_args",
|
||||||
"verify_proxy_key",
|
"verify_proxy_key",
|
||||||
"agent_profile",
|
"agent_profile",
|
||||||
|
"resolve_api_key",
|
||||||
"AgentRunError",
|
"AgentRunError",
|
||||||
]
|
]
|
||||||
|
|
|
||||||
0
litellm/proxy/client/cli/commands/autoroute/__init__.py
Normal file
0
litellm/proxy/client/cli/commands/autoroute/__init__.py
Normal file
196
litellm/proxy/client/cli/commands/autoroute/commands.py
Normal file
196
litellm/proxy/client/cli/commands/autoroute/commands.py
Normal file
|
|
@ -0,0 +1,196 @@
|
||||||
|
import atexit
|
||||||
|
import json
|
||||||
|
import secrets
|
||||||
|
import signal
|
||||||
|
import threading
|
||||||
|
from types import FrameType
|
||||||
|
|
||||||
|
import click
|
||||||
|
import yaml
|
||||||
|
from pydantic import JsonValue, TypeAdapter, ValidationError
|
||||||
|
|
||||||
|
from ..up import CLAUDE_SETTINGS_PATH, UpError, load_json_or_empty, restore_claude_settings, write_backup
|
||||||
|
from ..up import BackupRecord as ClaudeBackupRecord
|
||||||
|
from .process import (
|
||||||
|
AUTOROUTE_DIR,
|
||||||
|
CONFIG_PATH,
|
||||||
|
LOG_PATH,
|
||||||
|
PidRecord,
|
||||||
|
ProcessLaunchError,
|
||||||
|
allocate_free_port,
|
||||||
|
clear_pid_record,
|
||||||
|
is_running,
|
||||||
|
launch_proxy,
|
||||||
|
poll_liveliness,
|
||||||
|
read_pid_record,
|
||||||
|
secure_create,
|
||||||
|
stream_log,
|
||||||
|
terminate,
|
||||||
|
write_pid_record,
|
||||||
|
)
|
||||||
|
from .settings import merge_claude_settings_static_token
|
||||||
|
from .wizard import run_configure_wizard
|
||||||
|
|
||||||
|
AUTOROUTE_BACKUP_PATH = AUTOROUTE_DIR / "claude_settings_backup.json"
|
||||||
|
|
||||||
|
_GENERATED_CONFIG_ADAPTER = TypeAdapter(dict[str, JsonValue])
|
||||||
|
|
||||||
|
|
||||||
|
def _mint_and_embed_master_key() -> str:
|
||||||
|
"""Generate a fresh key for this session and write it into the generated config.yaml.
|
||||||
|
|
||||||
|
Must go under general_settings, not litellm_settings -- the proxy server only ever
|
||||||
|
reads general_settings.master_key (proxy_server.py:4530) to authenticate requests. A
|
||||||
|
key placed under litellm_settings is silently ignored, leaving the ephemeral proxy with
|
||||||
|
no real auth: any request reaches it regardless of the token Claude Code sends.
|
||||||
|
"""
|
||||||
|
master_key = secrets.token_urlsafe(32)
|
||||||
|
with open(CONFIG_PATH, "r") as f:
|
||||||
|
try:
|
||||||
|
generated = _GENERATED_CONFIG_ADAPTER.validate_python(yaml.safe_load(f))
|
||||||
|
except (yaml.YAMLError, ValidationError):
|
||||||
|
raise click.ClickException(
|
||||||
|
f"{CONFIG_PATH} is empty or corrupt. Run `lite autoroute configure` again to regenerate it."
|
||||||
|
)
|
||||||
|
general_settings = generated.get("general_settings")
|
||||||
|
updated_settings: dict[str, JsonValue] = {
|
||||||
|
**(general_settings if isinstance(general_settings, dict) else {}),
|
||||||
|
"master_key": master_key,
|
||||||
|
}
|
||||||
|
updated: dict[str, JsonValue] = {**generated, "general_settings": updated_settings}
|
||||||
|
with secure_create(CONFIG_PATH) as f:
|
||||||
|
yaml.safe_dump(updated, f, sort_keys=False)
|
||||||
|
return master_key
|
||||||
|
|
||||||
|
|
||||||
|
@click.group(name="autoroute")
|
||||||
|
def autoroute_group() -> None:
|
||||||
|
"""QA complexity-based auto-routing against models your key can already use"""
|
||||||
|
|
||||||
|
|
||||||
|
@autoroute_group.command("configure")
|
||||||
|
@click.pass_context
|
||||||
|
def configure(ctx: click.Context) -> None:
|
||||||
|
"""Discover accessible models and generate an ephemeral auto-router config"""
|
||||||
|
run_configure_wizard(ctx)
|
||||||
|
|
||||||
|
|
||||||
|
@autoroute_group.command("up")
|
||||||
|
def up() -> None:
|
||||||
|
"""Launch the ephemeral auto-router proxy and route Claude Code through it"""
|
||||||
|
if not CONFIG_PATH.exists():
|
||||||
|
raise click.ClickException("No config found. Run `lite autoroute configure` first.")
|
||||||
|
|
||||||
|
try:
|
||||||
|
existing_pid = read_pid_record()
|
||||||
|
except UpError as e:
|
||||||
|
raise click.ClickException(str(e))
|
||||||
|
if existing_pid is not None and is_running(existing_pid.pid):
|
||||||
|
raise click.ClickException(
|
||||||
|
"An ephemeral proxy is already running (lite autoroute up looks already active). "
|
||||||
|
"Run `lite autoroute down` first."
|
||||||
|
)
|
||||||
|
|
||||||
|
if AUTOROUTE_BACKUP_PATH.exists():
|
||||||
|
raise click.ClickException(
|
||||||
|
f"{AUTOROUTE_BACKUP_PATH} already exists -- `lite autoroute up` looks like it's already "
|
||||||
|
"running (or crashed without cleanup). Run `lite autoroute down` first."
|
||||||
|
)
|
||||||
|
|
||||||
|
master_key = _mint_and_embed_master_key()
|
||||||
|
port = allocate_free_port()
|
||||||
|
base_url = f"http://127.0.0.1:{port}"
|
||||||
|
process = launch_proxy(CONFIG_PATH, port, LOG_PATH)
|
||||||
|
write_pid_record(PidRecord(pid=process.pid, port=port, config_path=str(CONFIG_PATH), log_path=str(LOG_PATH)))
|
||||||
|
|
||||||
|
try:
|
||||||
|
poll_liveliness(base_url, LOG_PATH, process)
|
||||||
|
except ProcessLaunchError as e:
|
||||||
|
terminate(process.pid)
|
||||||
|
clear_pid_record()
|
||||||
|
raise click.ClickException(str(e))
|
||||||
|
|
||||||
|
try:
|
||||||
|
original_existed = CLAUDE_SETTINGS_PATH.exists()
|
||||||
|
original_settings = load_json_or_empty(CLAUDE_SETTINGS_PATH)
|
||||||
|
write_backup(
|
||||||
|
ClaudeBackupRecord(existed=original_existed, content=original_settings if original_existed else None),
|
||||||
|
AUTOROUTE_BACKUP_PATH,
|
||||||
|
)
|
||||||
|
merged = merge_claude_settings_static_token(original_settings, base_url, master_key)
|
||||||
|
CLAUDE_SETTINGS_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with secure_create(CLAUDE_SETTINGS_PATH) as f:
|
||||||
|
json.dump(merged, f, indent=2)
|
||||||
|
except UpError as e:
|
||||||
|
terminate(process.pid)
|
||||||
|
clear_pid_record()
|
||||||
|
raise click.ClickException(str(e))
|
||||||
|
|
||||||
|
click.echo(f"litellm: ephemeral auto-router proxy up at {base_url} (pid {process.pid})")
|
||||||
|
click.echo("Claude Code sessions started now will route through it. Press Ctrl-C to stop and restore.")
|
||||||
|
|
||||||
|
stop_event = threading.Event()
|
||||||
|
restored = threading.Lock()
|
||||||
|
|
||||||
|
def _teardown() -> None:
|
||||||
|
if not restored.acquire(blocking=False):
|
||||||
|
return
|
||||||
|
terminate(process.pid)
|
||||||
|
clear_pid_record()
|
||||||
|
try:
|
||||||
|
restore_claude_settings(CLAUDE_SETTINGS_PATH, AUTOROUTE_BACKUP_PATH)
|
||||||
|
except UpError as e:
|
||||||
|
# Runs from atexit/a signal handler too, outside Click's own exception
|
||||||
|
# handling -- raising here would only produce an unhandled-exception
|
||||||
|
# warning on stderr, not a clean message.
|
||||||
|
click.echo(str(e), err=True)
|
||||||
|
return
|
||||||
|
click.echo("\nStopped ephemeral proxy and restored Claude Code settings.")
|
||||||
|
click.echo(
|
||||||
|
f"Restart any Claude Code session still open from this session, or another local account could "
|
||||||
|
f"bind the now-free port {port} and receive its requests. Do not use `lite autoroute up` on a "
|
||||||
|
f"shared or multi-tenant host."
|
||||||
|
)
|
||||||
|
|
||||||
|
def _handle_signal(_signum: int, _frame: FrameType | None) -> None:
|
||||||
|
stop_event.set()
|
||||||
|
|
||||||
|
signal.signal(signal.SIGINT, _handle_signal)
|
||||||
|
signal.signal(signal.SIGTERM, _handle_signal)
|
||||||
|
atexit.register(_teardown)
|
||||||
|
|
||||||
|
log_thread = threading.Thread(target=stream_log, args=(LOG_PATH, stop_event), daemon=True)
|
||||||
|
log_thread.start()
|
||||||
|
|
||||||
|
stop_event.wait()
|
||||||
|
_teardown()
|
||||||
|
|
||||||
|
|
||||||
|
@autoroute_group.command("down")
|
||||||
|
def down() -> None:
|
||||||
|
"""Restore Claude Code settings and stop a leftover ephemeral proxy, if any"""
|
||||||
|
try:
|
||||||
|
record: PidRecord | None = read_pid_record()
|
||||||
|
except UpError as e:
|
||||||
|
# down is the crash-recovery path -- a corrupt pid record must not block it; clear the
|
||||||
|
# unusable record and keep going rather than leaving the user with no way to clean up.
|
||||||
|
click.echo(f"{e} Clearing it and continuing cleanup.", err=True)
|
||||||
|
record = None
|
||||||
|
if record is not None and is_running(record.pid):
|
||||||
|
terminate(record.pid)
|
||||||
|
click.echo(f"Stopped leftover ephemeral proxy (pid {record.pid}).")
|
||||||
|
clear_pid_record()
|
||||||
|
|
||||||
|
try:
|
||||||
|
restored = restore_claude_settings(CLAUDE_SETTINGS_PATH, AUTOROUTE_BACKUP_PATH)
|
||||||
|
except UpError as e:
|
||||||
|
raise click.ClickException(str(e))
|
||||||
|
if restored is None:
|
||||||
|
click.echo("Nothing to restore.")
|
||||||
|
elif restored.existed:
|
||||||
|
click.echo(f"Restored {CLAUDE_SETTINGS_PATH} to its original contents.")
|
||||||
|
else:
|
||||||
|
click.echo(f"Removed {CLAUDE_SETTINGS_PATH} (it did not exist before `lite autoroute up`).")
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["autoroute_group"]
|
||||||
237
litellm/proxy/client/cli/commands/autoroute/config.py
Normal file
237
litellm/proxy/client/cli/commands/autoroute/config.py
Normal file
|
|
@ -0,0 +1,237 @@
|
||||||
|
from typing import Literal, Union
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field, JsonValue, TypeAdapter
|
||||||
|
|
||||||
|
TIER_NAMES: tuple[str, ...] = ("SIMPLE", "MEDIUM", "COMPLEX", "REASONING")
|
||||||
|
AUTOROUTER_MODEL_NAME = "autorouter"
|
||||||
|
|
||||||
|
|
||||||
|
class ConfigGenerationError(Exception):
|
||||||
|
"""Raised when an AutorouteConfig references a model the discovery step didn't find."""
|
||||||
|
|
||||||
|
|
||||||
|
class DiscoveredModel(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
name: str
|
||||||
|
mode: str = "chat"
|
||||||
|
input_cost_per_token: float | None = None
|
||||||
|
output_cost_per_token: float | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class _RawModelGroup(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
model_group: str
|
||||||
|
# Optional: some real deployments return an explicit `"mode": null` for models that
|
||||||
|
# were registered without a mode (seen for embedding models like voyage-4-large).
|
||||||
|
# ModelGroupInfo's own "chat" default (litellm/types/router.py) only applies when the
|
||||||
|
# key is missing entirely, not when it's present as null, so this must tolerate None.
|
||||||
|
mode: str | None = "chat"
|
||||||
|
input_cost_per_token: float | None = None
|
||||||
|
output_cost_per_token: float | None = None
|
||||||
|
|
||||||
|
|
||||||
|
_RAW_MODEL_GROUPS_ADAPTER = TypeAdapter(list[_RawModelGroup])
|
||||||
|
|
||||||
|
|
||||||
|
def parse_discovered_models(raw: list[JsonValue]) -> tuple[DiscoveredModel, ...]:
|
||||||
|
"""Validate a raw `/model_group/info` response into typed models."""
|
||||||
|
parsed = _RAW_MODEL_GROUPS_ADAPTER.validate_python(raw)
|
||||||
|
return tuple(
|
||||||
|
DiscoveredModel(
|
||||||
|
name=group.model_group,
|
||||||
|
# A null mode means the server genuinely doesn't know what this model does;
|
||||||
|
# "unknown" (rather than guessing "chat") keeps it out of both chat_models()
|
||||||
|
# and embedding_models() instead of risking a wrong-mode deployment.
|
||||||
|
mode=group.mode or "unknown",
|
||||||
|
input_cost_per_token=group.input_cost_per_token,
|
||||||
|
output_cost_per_token=group.output_cost_per_token,
|
||||||
|
)
|
||||||
|
for group in parsed
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def chat_models(models: tuple[DiscoveredModel, ...]) -> tuple[DiscoveredModel, ...]:
|
||||||
|
return tuple(m for m in models if m.mode == "chat")
|
||||||
|
|
||||||
|
|
||||||
|
def embedding_models(models: tuple[DiscoveredModel, ...]) -> tuple[DiscoveredModel, ...]:
|
||||||
|
return tuple(m for m in models if m.mode == "embedding")
|
||||||
|
|
||||||
|
|
||||||
|
class HeuristicClassifier(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
kind: Literal["heuristic"] = "heuristic"
|
||||||
|
|
||||||
|
|
||||||
|
class LLMClassifier(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
kind: Literal["llm"] = "llm"
|
||||||
|
model: str
|
||||||
|
timeout_ms: int = 3000
|
||||||
|
|
||||||
|
|
||||||
|
ClassifierChoice = Union[HeuristicClassifier, LLMClassifier]
|
||||||
|
|
||||||
|
|
||||||
|
class NoSemanticMatching(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
kind: Literal["none"] = "none"
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticMatching(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
kind: Literal["semantic"] = "semantic"
|
||||||
|
embedding_model: str
|
||||||
|
match_threshold: float = 0.5
|
||||||
|
|
||||||
|
|
||||||
|
SemanticMatchingChoice = Union[NoSemanticMatching, SemanticMatching]
|
||||||
|
|
||||||
|
# Satisfies complexity_router's "semantic matching requires non-empty keyword_tier_rules"
|
||||||
|
# invariant with a sane starting point; the generated config.yaml can be hand-edited afterward.
|
||||||
|
_DEFAULT_KEYWORD_TIER_RULES: tuple[dict[str, JsonValue], ...] = (
|
||||||
|
{"keywords": ["hi", "hello", "thanks"], "tier": "SIMPLE"},
|
||||||
|
{"keywords": ["explain", "how does"], "tier": "MEDIUM"},
|
||||||
|
{"keywords": ["refactor", "implement", "debug"], "tier": "COMPLEX"},
|
||||||
|
{"keywords": ["step by step", "think through", "prove"], "tier": "REASONING"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class AutorouteConfig(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
base_url: str
|
||||||
|
api_key: str
|
||||||
|
# Each tier maps to a pool of one or more models; complexity_router picks randomly among
|
||||||
|
# them per request (or, in adaptive mode, learns which to prefer within the pool).
|
||||||
|
tiers: dict[str, tuple[str, ...]]
|
||||||
|
default_model: str
|
||||||
|
classifier: ClassifierChoice = Field(default_factory=HeuristicClassifier)
|
||||||
|
semantic_matching: SemanticMatchingChoice = Field(default_factory=NoSemanticMatching)
|
||||||
|
adaptive: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
def validate_config(config: AutorouteConfig, discovered: tuple[DiscoveredModel, ...]) -> None:
|
||||||
|
"""Raise ConfigGenerationError if config references a model discovery didn't return."""
|
||||||
|
chat_names: frozenset[str] = frozenset(m.name for m in chat_models(discovered))
|
||||||
|
embedding_names: frozenset[str] = frozenset(m.name for m in embedding_models(discovered))
|
||||||
|
|
||||||
|
for tier, models in config.tiers.items():
|
||||||
|
for model in models:
|
||||||
|
if model not in chat_names:
|
||||||
|
raise ConfigGenerationError(f"Tier {tier} references unknown chat model '{model}'")
|
||||||
|
|
||||||
|
if config.default_model not in chat_names:
|
||||||
|
raise ConfigGenerationError(f"default_model '{config.default_model}' is not a known chat model")
|
||||||
|
|
||||||
|
if isinstance(config.classifier, LLMClassifier) and config.classifier.model not in chat_names:
|
||||||
|
raise ConfigGenerationError(f"classifier model '{config.classifier.model}' is not a known chat model")
|
||||||
|
|
||||||
|
if (
|
||||||
|
isinstance(config.semantic_matching, SemanticMatching)
|
||||||
|
and config.semantic_matching.embedding_model not in embedding_names
|
||||||
|
):
|
||||||
|
raise ConfigGenerationError(
|
||||||
|
f"embedding model '{config.semantic_matching.embedding_model}' is not a known embedding model"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _litellm_proxy_deployment(name: str, base_url: str, api_key: str) -> dict[str, JsonValue]:
|
||||||
|
return {
|
||||||
|
"model_name": name,
|
||||||
|
"litellm_params": {
|
||||||
|
"model": f"litellm_proxy/{name}",
|
||||||
|
"api_base": base_url,
|
||||||
|
"api_key": api_key,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_generated_model_list(config: AutorouteConfig) -> list[JsonValue]:
|
||||||
|
"""Build the model_list for the ephemeral proxy's config.yaml.
|
||||||
|
|
||||||
|
Every real model referenced anywhere (tier targets, classifier, embedding) is deduplicated
|
||||||
|
to exactly one `litellm_proxy/<name>` deployment forwarding to the customer's real proxy,
|
||||||
|
plus one `auto_router/complexity_router` deployment tying the tiers together.
|
||||||
|
"""
|
||||||
|
referenced_names = {model for models in config.tiers.values() for model in models}
|
||||||
|
referenced_names.add(config.default_model)
|
||||||
|
if isinstance(config.classifier, LLMClassifier):
|
||||||
|
referenced_names.add(config.classifier.model)
|
||||||
|
if isinstance(config.semantic_matching, SemanticMatching):
|
||||||
|
referenced_names.add(config.semantic_matching.embedding_model)
|
||||||
|
|
||||||
|
proxy_deployments = [
|
||||||
|
_litellm_proxy_deployment(name, config.base_url, config.api_key) for name in sorted(referenced_names)
|
||||||
|
]
|
||||||
|
|
||||||
|
complexity_router_config: dict[str, JsonValue] = {
|
||||||
|
"tiers": {tier: list(models) for tier, models in config.tiers.items()},
|
||||||
|
"default_model": config.default_model,
|
||||||
|
}
|
||||||
|
if isinstance(config.classifier, LLMClassifier):
|
||||||
|
complexity_router_config["classifier_type"] = "llm"
|
||||||
|
complexity_router_config["classifier_llm_config"] = {
|
||||||
|
"model": config.classifier.model,
|
||||||
|
"timeout_ms": config.classifier.timeout_ms,
|
||||||
|
}
|
||||||
|
if isinstance(config.semantic_matching, SemanticMatching):
|
||||||
|
complexity_router_config["semantic_keyword_matching"] = True
|
||||||
|
complexity_router_config["embedding_model"] = config.semantic_matching.embedding_model
|
||||||
|
complexity_router_config["match_threshold"] = config.semantic_matching.match_threshold
|
||||||
|
complexity_router_config["keyword_tier_rules"] = list(_DEFAULT_KEYWORD_TIER_RULES)
|
||||||
|
if config.adaptive:
|
||||||
|
complexity_router_config["adaptive"] = True
|
||||||
|
|
||||||
|
auto_router_litellm_params: dict[str, JsonValue] = {
|
||||||
|
"model": "auto_router/complexity_router",
|
||||||
|
"complexity_router_config": complexity_router_config,
|
||||||
|
}
|
||||||
|
# A bare "*" model_name looks like the obvious way to catch every request Claude Code
|
||||||
|
# might send regardless of which model it thinks it's using, but Router's auto-router
|
||||||
|
# registry is keyed by the literal requested model string (router.py:10711-10717), not
|
||||||
|
# resolved through pattern/wildcard matching first -- so a "*" entry here would only ever
|
||||||
|
# match a client that literally sends model="*", never an actual wildcard catch-all. Callers
|
||||||
|
# instead need to make Claude Code request this "autorouter" name directly (see
|
||||||
|
# ANTHROPIC_DEFAULT_*_MODEL in settings.py's merge_claude_settings_static_token).
|
||||||
|
return [
|
||||||
|
*proxy_deployments,
|
||||||
|
{"model_name": AUTOROUTER_MODEL_NAME, "litellm_params": auto_router_litellm_params},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def build_generated_proxy_config(config: AutorouteConfig, master_key: str) -> dict[str, JsonValue]:
|
||||||
|
"""Full config.yaml content for the ephemeral proxy, including its own auth key.
|
||||||
|
|
||||||
|
master_key must live under general_settings, not litellm_settings -- the proxy server
|
||||||
|
only ever reads general_settings.master_key (proxy_server.py:4530) to authenticate
|
||||||
|
requests; a key placed under litellm_settings is silently ignored, leaving the proxy
|
||||||
|
with no real auth at all.
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"model_list": build_generated_model_list(config),
|
||||||
|
"general_settings": {"master_key": master_key},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AUTOROUTER_MODEL_NAME",
|
||||||
|
"TIER_NAMES",
|
||||||
|
"AutorouteConfig",
|
||||||
|
"ClassifierChoice",
|
||||||
|
"ConfigGenerationError",
|
||||||
|
"DiscoveredModel",
|
||||||
|
"HeuristicClassifier",
|
||||||
|
"LLMClassifier",
|
||||||
|
"NoSemanticMatching",
|
||||||
|
"SemanticMatching",
|
||||||
|
"SemanticMatchingChoice",
|
||||||
|
"build_generated_model_list",
|
||||||
|
"build_generated_proxy_config",
|
||||||
|
"chat_models",
|
||||||
|
"embedding_models",
|
||||||
|
"parse_discovered_models",
|
||||||
|
"validate_config",
|
||||||
|
]
|
||||||
174
litellm/proxy/client/cli/commands/autoroute/process.py
Normal file
174
litellm/proxy/client/cli/commands/autoroute/process.py
Normal file
|
|
@ -0,0 +1,174 @@
|
||||||
|
import contextlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import signal
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import click
|
||||||
|
import requests
|
||||||
|
from pydantic import TypeAdapter, ValidationError
|
||||||
|
|
||||||
|
from ..up import UpError, secure_create
|
||||||
|
|
||||||
|
AUTOROUTE_DIR = Path.home() / ".litellm" / "autorouter"
|
||||||
|
CONFIG_PATH = AUTOROUTE_DIR / "config.yaml"
|
||||||
|
LOG_PATH = AUTOROUTE_DIR / "proxy.log"
|
||||||
|
PID_RECORD_PATH = AUTOROUTE_DIR / "proxy.pid.json"
|
||||||
|
|
||||||
|
|
||||||
|
class ProcessLaunchError(Exception):
|
||||||
|
"""Raised when the ephemeral proxy subprocess fails to come up healthy."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class PidRecord:
|
||||||
|
pid: int
|
||||||
|
port: int
|
||||||
|
config_path: str
|
||||||
|
log_path: str
|
||||||
|
|
||||||
|
|
||||||
|
_PID_RECORD_ADAPTER = TypeAdapter(PidRecord)
|
||||||
|
|
||||||
|
|
||||||
|
def allocate_free_port() -> int:
|
||||||
|
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||||
|
sock.bind(("127.0.0.1", 0))
|
||||||
|
return int(sock.getsockname()[1])
|
||||||
|
|
||||||
|
|
||||||
|
def launch_proxy(config_path: Path, port: int, log_path: Path) -> "subprocess.Popen[bytes]":
|
||||||
|
log_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(log_path, "w") as log_file:
|
||||||
|
return subprocess.Popen(
|
||||||
|
[
|
||||||
|
sys.executable,
|
||||||
|
"-m",
|
||||||
|
"litellm.proxy.proxy_cli",
|
||||||
|
"--config",
|
||||||
|
str(config_path),
|
||||||
|
"--port",
|
||||||
|
str(port),
|
||||||
|
"--host",
|
||||||
|
"127.0.0.1",
|
||||||
|
],
|
||||||
|
stdout=log_file,
|
||||||
|
stderr=subprocess.STDOUT,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _tail(log_path: Path, lines: int = 40) -> str:
|
||||||
|
if not log_path.exists():
|
||||||
|
return "(no log output captured)"
|
||||||
|
return "\n".join(log_path.read_text(errors="replace").splitlines()[-lines:])
|
||||||
|
|
||||||
|
|
||||||
|
def poll_liveliness(base_url: str, log_path: Path, process: "subprocess.Popen[bytes]", timeout: float = 30.0) -> None:
|
||||||
|
"""Poll /health/liveliness until it responds, the process dies, or timeout elapses."""
|
||||||
|
deadline = time.monotonic() + timeout
|
||||||
|
url = base_url.rstrip("/") + "/health/liveliness"
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
if process.poll() is not None:
|
||||||
|
raise ProcessLaunchError(
|
||||||
|
f"Ephemeral proxy exited early (code {process.returncode}). Last log lines:\n{_tail(log_path)}"
|
||||||
|
)
|
||||||
|
with contextlib.suppress(requests.RequestException):
|
||||||
|
if requests.get(url, timeout=2).status_code == 200:
|
||||||
|
return
|
||||||
|
time.sleep(0.5)
|
||||||
|
raise ProcessLaunchError(
|
||||||
|
f"Ephemeral proxy never became healthy within {timeout}s. Last log lines:\n{_tail(log_path)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def write_pid_record(record: PidRecord, path: Path | None = None) -> None:
|
||||||
|
resolved_path = path if path is not None else PID_RECORD_PATH
|
||||||
|
resolved_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(resolved_path, "w") as f:
|
||||||
|
json.dump(
|
||||||
|
{"pid": record.pid, "port": record.port, "config_path": record.config_path, "log_path": record.log_path},
|
||||||
|
f,
|
||||||
|
indent=2,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def read_pid_record(path: Path | None = None) -> PidRecord | None:
|
||||||
|
resolved_path = path if path is not None else PID_RECORD_PATH
|
||||||
|
if not resolved_path.exists():
|
||||||
|
return None
|
||||||
|
with open(resolved_path, "r") as f:
|
||||||
|
content = f.read()
|
||||||
|
try:
|
||||||
|
return _PID_RECORD_ADAPTER.validate_json(content)
|
||||||
|
except ValidationError:
|
||||||
|
raise UpError(f"{resolved_path} contains invalid or unexpected JSON; cannot proceed safely.")
|
||||||
|
|
||||||
|
|
||||||
|
def clear_pid_record(path: Path | None = None) -> None:
|
||||||
|
resolved_path = path if path is not None else PID_RECORD_PATH
|
||||||
|
resolved_path.unlink(missing_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
def is_running(pid: int) -> bool:
|
||||||
|
try:
|
||||||
|
os.kill(pid, 0)
|
||||||
|
except ProcessLookupError:
|
||||||
|
return False
|
||||||
|
except PermissionError:
|
||||||
|
return True
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def terminate(pid: int, grace_period: float = 5.0) -> None:
|
||||||
|
"""Terminate a process by pid, escalating from SIGTERM to SIGKILL if needed."""
|
||||||
|
if not is_running(pid):
|
||||||
|
return
|
||||||
|
with contextlib.suppress(ProcessLookupError):
|
||||||
|
os.kill(pid, signal.SIGTERM)
|
||||||
|
deadline = time.monotonic() + grace_period
|
||||||
|
while time.monotonic() < deadline and is_running(pid):
|
||||||
|
time.sleep(0.2)
|
||||||
|
if is_running(pid):
|
||||||
|
with contextlib.suppress(ProcessLookupError):
|
||||||
|
os.kill(pid, signal.SIGKILL)
|
||||||
|
|
||||||
|
|
||||||
|
def stream_log(log_path: Path, stop_event: threading.Event) -> None:
|
||||||
|
"""Print new lines appended to log_path until stop_event is set. Blocks the calling thread."""
|
||||||
|
while not log_path.exists() and not stop_event.is_set():
|
||||||
|
time.sleep(0.1)
|
||||||
|
if stop_event.is_set() or not log_path.exists():
|
||||||
|
return
|
||||||
|
with open(log_path, "r") as f:
|
||||||
|
while not stop_event.is_set():
|
||||||
|
line = f.readline()
|
||||||
|
if line:
|
||||||
|
click.echo(line, nl=False)
|
||||||
|
else:
|
||||||
|
time.sleep(0.2)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AUTOROUTE_DIR",
|
||||||
|
"CONFIG_PATH",
|
||||||
|
"LOG_PATH",
|
||||||
|
"PID_RECORD_PATH",
|
||||||
|
"PidRecord",
|
||||||
|
"ProcessLaunchError",
|
||||||
|
"allocate_free_port",
|
||||||
|
"clear_pid_record",
|
||||||
|
"is_running",
|
||||||
|
"launch_proxy",
|
||||||
|
"poll_liveliness",
|
||||||
|
"read_pid_record",
|
||||||
|
"secure_create",
|
||||||
|
"stream_log",
|
||||||
|
"terminate",
|
||||||
|
"write_pid_record",
|
||||||
|
]
|
||||||
46
litellm/proxy/client/cli/commands/autoroute/settings.py
Normal file
46
litellm/proxy/client/cli/commands/autoroute/settings.py
Normal file
|
|
@ -0,0 +1,46 @@
|
||||||
|
from pydantic import JsonValue
|
||||||
|
|
||||||
|
from .config import AUTOROUTER_MODEL_NAME
|
||||||
|
|
||||||
|
ENV_KEY = "env"
|
||||||
|
API_KEY_HELPER_KEY = "apiKeyHelper"
|
||||||
|
ANTHROPIC_API_KEY_KEY = "ANTHROPIC_API_KEY"
|
||||||
|
ANTHROPIC_AUTH_TOKEN_KEY = "ANTHROPIC_AUTH_TOKEN"
|
||||||
|
ANTHROPIC_BASE_URL_KEY = "ANTHROPIC_BASE_URL"
|
||||||
|
# Force every one of Claude Code's own model tiers to request the auto-router by name.
|
||||||
|
# Router's auto-router registry is keyed by the literal requested model string
|
||||||
|
# (litellm/router.py:10711-10717) with no wildcard/pattern resolution, so a bare "*"
|
||||||
|
# model_name can never work as a catch-all -- these overrides are what actually makes
|
||||||
|
# Claude Code send "autorouter" regardless of /model or its own version-specific defaults.
|
||||||
|
ANTHROPIC_DEFAULT_MODEL_ENV_KEYS = (
|
||||||
|
"ANTHROPIC_DEFAULT_SONNET_MODEL",
|
||||||
|
"ANTHROPIC_DEFAULT_HAIKU_MODEL",
|
||||||
|
"ANTHROPIC_DEFAULT_OPUS_MODEL",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def merge_claude_settings_static_token(
|
||||||
|
settings: dict[str, JsonValue], base_url: str, auth_token: str
|
||||||
|
) -> dict[str, JsonValue]:
|
||||||
|
"""Return a new settings dict wired to a local ephemeral proxy with a static token.
|
||||||
|
|
||||||
|
Unlike up.py's merge_claude_settings (which sets apiKeyHelper for a long-lived, real
|
||||||
|
remote proxy needing refreshable SSO tokens), this proxy is ephemeral and its key was just
|
||||||
|
minted for this session, so a plain env var is simpler and correct. Any existing
|
||||||
|
apiKeyHelper is cleared so it can't fight with the static token.
|
||||||
|
"""
|
||||||
|
raw_env = settings.get(ENV_KEY, {})
|
||||||
|
base_env = raw_env if isinstance(raw_env, dict) else {}
|
||||||
|
env: dict[str, JsonValue] = {
|
||||||
|
**base_env,
|
||||||
|
ANTHROPIC_BASE_URL_KEY: base_url.rstrip("/"),
|
||||||
|
ANTHROPIC_AUTH_TOKEN_KEY: auth_token,
|
||||||
|
**{key: AUTOROUTER_MODEL_NAME for key in ANTHROPIC_DEFAULT_MODEL_ENV_KEYS},
|
||||||
|
}
|
||||||
|
env.pop(ANTHROPIC_API_KEY_KEY, None)
|
||||||
|
merged: dict[str, JsonValue] = {**settings, ENV_KEY: env}
|
||||||
|
merged.pop(API_KEY_HELPER_KEY, None)
|
||||||
|
return merged
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["merge_claude_settings_static_token"]
|
||||||
128
litellm/proxy/client/cli/commands/autoroute/wizard.py
Normal file
128
litellm/proxy/client/cli/commands/autoroute/wizard.py
Normal file
|
|
@ -0,0 +1,128 @@
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import click
|
||||||
|
import yaml
|
||||||
|
from InquirerPy import inquirer
|
||||||
|
from InquirerPy.base.control import Choice
|
||||||
|
|
||||||
|
from .... import Client
|
||||||
|
from .config import (
|
||||||
|
TIER_NAMES,
|
||||||
|
AutorouteConfig,
|
||||||
|
ConfigGenerationError,
|
||||||
|
DiscoveredModel,
|
||||||
|
HeuristicClassifier,
|
||||||
|
LLMClassifier,
|
||||||
|
NoSemanticMatching,
|
||||||
|
SemanticMatching,
|
||||||
|
build_generated_model_list,
|
||||||
|
chat_models,
|
||||||
|
embedding_models,
|
||||||
|
parse_discovered_models,
|
||||||
|
validate_config,
|
||||||
|
)
|
||||||
|
from .process import CONFIG_PATH, secure_create
|
||||||
|
|
||||||
|
|
||||||
|
def _is_interactive() -> bool:
|
||||||
|
return sys.stdin.isatty()
|
||||||
|
|
||||||
|
|
||||||
|
def _fuzzy_pick(models: tuple[DiscoveredModel, ...], prompt_label: str, multiselect: bool) -> list[str]:
|
||||||
|
"""Type-to-filter picker over a (possibly huge) model pool, using InquirerPy's fzf-style fuzzy prompt.
|
||||||
|
|
||||||
|
A plain numbered table + typed index does not scale past a handful of models -- proxies with
|
||||||
|
hundreds of model groups made that interaction unusable. This lets the user narrow the pool by
|
||||||
|
typing a substring instead of scrolling/counting.
|
||||||
|
|
||||||
|
Assumes the caller already checked interactivity (run_configure_wizard does, once, up front) --
|
||||||
|
checking here too would check the wrong thing under test, where InquirerPy is driven through its
|
||||||
|
own injected input/output rather than the real process stdin.
|
||||||
|
"""
|
||||||
|
choices = [Choice(value=model.name, name=model.name) for model in models]
|
||||||
|
toggle_hint = "tab to toggle, " if multiselect else ""
|
||||||
|
while True:
|
||||||
|
result = inquirer.fuzzy(
|
||||||
|
message=f"{prompt_label}: type to filter, {toggle_hint}enter to confirm",
|
||||||
|
choices=choices,
|
||||||
|
multiselect=multiselect,
|
||||||
|
max_height="70%",
|
||||||
|
).execute()
|
||||||
|
selected = result if multiselect else [result]
|
||||||
|
if selected:
|
||||||
|
return selected
|
||||||
|
click.echo("Select at least one model.")
|
||||||
|
|
||||||
|
|
||||||
|
def _render_and_prompt_for_model(models: tuple[DiscoveredModel, ...], prompt_label: str) -> str:
|
||||||
|
return _fuzzy_pick(models, prompt_label, multiselect=False)[0]
|
||||||
|
|
||||||
|
|
||||||
|
def _render_and_prompt_for_models(models: tuple[DiscoveredModel, ...], prompt_label: str) -> tuple[str, ...]:
|
||||||
|
return tuple(_fuzzy_pick(models, prompt_label, multiselect=True))
|
||||||
|
|
||||||
|
|
||||||
|
def run_configure_wizard(ctx: click.Context) -> Path:
|
||||||
|
"""Discover the caller's accessible models, walk them through tier assignment, write config."""
|
||||||
|
base_url = ctx.obj["base_url"]
|
||||||
|
api_key = ctx.obj["api_key"]
|
||||||
|
client = Client(base_url=base_url, api_key=api_key)
|
||||||
|
|
||||||
|
raw_groups = client.model_groups.info()
|
||||||
|
if not isinstance(raw_groups, list):
|
||||||
|
raise click.ClickException(
|
||||||
|
f"Unexpected response from /model_group/info: expected a list, got {type(raw_groups).__name__}"
|
||||||
|
)
|
||||||
|
discovered = parse_discovered_models(raw_groups)
|
||||||
|
chat_pool = chat_models(discovered)
|
||||||
|
embedding_pool = embedding_models(discovered)
|
||||||
|
|
||||||
|
if not chat_pool:
|
||||||
|
raise click.ClickException("Your key has no chat-capable models available on this proxy.")
|
||||||
|
|
||||||
|
if not _is_interactive():
|
||||||
|
raise click.ClickException("`lite autoroute configure` requires an interactive terminal.")
|
||||||
|
|
||||||
|
click.echo("Assign model(s) to each complexity tier (from what your key can access):")
|
||||||
|
tiers = {tier: _render_and_prompt_for_models(chat_pool, tier) for tier in TIER_NAMES}
|
||||||
|
default_model = tiers["MEDIUM"][0]
|
||||||
|
|
||||||
|
classifier = HeuristicClassifier()
|
||||||
|
if click.confirm("\nUse an LLM classifier instead of the free heuristic scorer?", default=False):
|
||||||
|
classifier_model = _render_and_prompt_for_model(chat_pool, "LLM classifier")
|
||||||
|
classifier = LLMClassifier(model=classifier_model)
|
||||||
|
|
||||||
|
semantic_matching = NoSemanticMatching()
|
||||||
|
if embedding_pool and click.confirm("\nEnable semantic keyword matching?", default=False):
|
||||||
|
embedding_model = _render_and_prompt_for_model(embedding_pool, "semantic embeddings")
|
||||||
|
semantic_matching = SemanticMatching(embedding_model=embedding_model)
|
||||||
|
|
||||||
|
adaptive = click.confirm("\nEnable adaptive (bandit) selection on top of tiering?", default=False)
|
||||||
|
|
||||||
|
config = AutorouteConfig(
|
||||||
|
base_url=base_url,
|
||||||
|
api_key=api_key,
|
||||||
|
tiers=tiers,
|
||||||
|
default_model=default_model,
|
||||||
|
classifier=classifier,
|
||||||
|
semantic_matching=semantic_matching,
|
||||||
|
adaptive=adaptive,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
validate_config(config, discovered)
|
||||||
|
except ConfigGenerationError as e:
|
||||||
|
raise click.ClickException(str(e))
|
||||||
|
|
||||||
|
model_list = build_generated_model_list(config)
|
||||||
|
CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with secure_create(CONFIG_PATH) as f:
|
||||||
|
yaml.safe_dump({"model_list": model_list}, f, sort_keys=False)
|
||||||
|
|
||||||
|
click.echo(f"\nWrote {CONFIG_PATH}")
|
||||||
|
for tier, models in tiers.items():
|
||||||
|
click.echo(f" {tier}: {', '.join(models)}")
|
||||||
|
return CONFIG_PATH
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["run_configure_wizard"]
|
||||||
57
litellm/proxy/client/cli/commands/model_groups.py
Normal file
57
litellm/proxy/client/cli/commands/model_groups.py
Normal file
|
|
@ -0,0 +1,57 @@
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
import click
|
||||||
|
import rich
|
||||||
|
import rich.table
|
||||||
|
|
||||||
|
from ... import Client
|
||||||
|
|
||||||
|
|
||||||
|
def create_client(ctx: click.Context) -> Client:
|
||||||
|
return Client(base_url=ctx.obj["base_url"], api_key=ctx.obj["api_key"])
|
||||||
|
|
||||||
|
|
||||||
|
@click.group(name="model-groups")
|
||||||
|
def model_groups() -> None:
|
||||||
|
"""Inspect model groups your key can access on the proxy"""
|
||||||
|
|
||||||
|
|
||||||
|
@model_groups.command("list")
|
||||||
|
@click.option(
|
||||||
|
"--format",
|
||||||
|
"output_format",
|
||||||
|
type=click.Choice(["table", "json"]),
|
||||||
|
default="table",
|
||||||
|
help="Output format (table or json)",
|
||||||
|
)
|
||||||
|
@click.pass_context
|
||||||
|
def list_model_groups(ctx: click.Context, output_format: Literal["table", "json"]) -> None:
|
||||||
|
"""List model groups accessible to your key, with mode and pricing"""
|
||||||
|
client = create_client(ctx)
|
||||||
|
groups = client.model_groups.info()
|
||||||
|
if not isinstance(groups, list):
|
||||||
|
raise click.ClickException(
|
||||||
|
f"Unexpected response from /model_group/info: expected a list, got {type(groups).__name__}"
|
||||||
|
)
|
||||||
|
|
||||||
|
if output_format == "json":
|
||||||
|
rich.print_json(data=groups)
|
||||||
|
return
|
||||||
|
|
||||||
|
table = rich.table.Table(title="Accessible Model Groups")
|
||||||
|
table.add_column("Model", style="cyan")
|
||||||
|
table.add_column("Mode", style="green")
|
||||||
|
table.add_column("Input $/token", style="yellow")
|
||||||
|
table.add_column("Output $/token", style="yellow")
|
||||||
|
|
||||||
|
for group in groups:
|
||||||
|
table.add_row(
|
||||||
|
str(group.get("model_group", "")),
|
||||||
|
str(group.get("mode", "chat")),
|
||||||
|
str(group.get("input_cost_per_token", "")),
|
||||||
|
str(group.get("output_cost_per_token", "")),
|
||||||
|
)
|
||||||
|
rich.print(table)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["model_groups"]
|
||||||
283
litellm/proxy/client/cli/commands/up.py
Normal file
283
litellm/proxy/client/cli/commands/up.py
Normal file
|
|
@ -0,0 +1,283 @@
|
||||||
|
import atexit
|
||||||
|
import contextlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import shlex
|
||||||
|
import shutil
|
||||||
|
import signal
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from types import FrameType
|
||||||
|
from typing import IO, Iterator, Mapping
|
||||||
|
|
||||||
|
import click
|
||||||
|
from pydantic import JsonValue, TypeAdapter, ValidationError
|
||||||
|
|
||||||
|
from litellm.litellm_core_utils.cli_token_utils import is_cli_token_fresh
|
||||||
|
|
||||||
|
from .agents import AgentRunError, resolve_api_key, verify_proxy_key
|
||||||
|
from .auth import load_token, login
|
||||||
|
|
||||||
|
ENV_KEY = "env"
|
||||||
|
API_KEY_HELPER_KEY = "apiKeyHelper"
|
||||||
|
ANTHROPIC_BASE_URL_KEY = "ANTHROPIC_BASE_URL"
|
||||||
|
ANTHROPIC_API_KEY_KEY = "ANTHROPIC_API_KEY"
|
||||||
|
|
||||||
|
CLAUDE_SETTINGS_PATH = Path.home() / ".claude" / "settings.json"
|
||||||
|
BACKUP_PATH = Path.home() / ".litellm" / "claude_settings_backup.json"
|
||||||
|
|
||||||
|
|
||||||
|
class UpError(Exception):
|
||||||
|
"""Raised for any user-actionable failure while starting/stopping interception."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class BackupRecord:
|
||||||
|
"""Snapshot of ~/.claude/settings.json taken right before `lite up` patches it."""
|
||||||
|
|
||||||
|
existed: bool
|
||||||
|
content: dict[str, JsonValue] | None
|
||||||
|
|
||||||
|
|
||||||
|
_SETTINGS_ADAPTER = TypeAdapter(dict[str, JsonValue])
|
||||||
|
_BACKUP_RECORD_ADAPTER = TypeAdapter(BackupRecord)
|
||||||
|
|
||||||
|
|
||||||
|
def load_json_or_empty(path: Path) -> dict[str, JsonValue]:
|
||||||
|
if not path.exists():
|
||||||
|
return {}
|
||||||
|
with open(path, "r") as f:
|
||||||
|
content = f.read()
|
||||||
|
if not content.strip():
|
||||||
|
return {}
|
||||||
|
try:
|
||||||
|
return _SETTINGS_ADAPTER.validate_json(content)
|
||||||
|
except ValidationError:
|
||||||
|
raise UpError(f"{path} contains invalid JSON (or its root is not an object); cannot proceed safely.")
|
||||||
|
|
||||||
|
|
||||||
|
def merge_claude_settings(
|
||||||
|
settings: Mapping[str, JsonValue], base_url: str, api_key_helper: str
|
||||||
|
) -> dict[str, JsonValue]:
|
||||||
|
"""Return a new settings dict wired to route Claude Code through the proxy.
|
||||||
|
|
||||||
|
Only env.ANTHROPIC_BASE_URL and the top-level apiKeyHelper are overridden; a
|
||||||
|
stray env.ANTHROPIC_API_KEY is dropped so it cannot outrank the helper-issued
|
||||||
|
token (same reasoning as build_agent_env in agents.py). Every other key is
|
||||||
|
preserved untouched.
|
||||||
|
"""
|
||||||
|
raw_env = settings.get(ENV_KEY, {})
|
||||||
|
base_env = raw_env if isinstance(raw_env, dict) else {}
|
||||||
|
env = {**base_env, ANTHROPIC_BASE_URL_KEY: base_url.rstrip("/")}
|
||||||
|
env.pop(ANTHROPIC_API_KEY_KEY, None)
|
||||||
|
return {**settings, ENV_KEY: env, API_KEY_HELPER_KEY: api_key_helper}
|
||||||
|
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def secure_create(path: Path) -> Iterator[IO[str]]:
|
||||||
|
"""Open path for writing with mode 0600 fixed up before any content is written.
|
||||||
|
|
||||||
|
A plain `open(path, "w")` creates a *new* file at the umask-derived default (commonly 0644)
|
||||||
|
and leaves it world- or group-readable until a later `chmod` call catches up -- a real window
|
||||||
|
in which a file holding a credential is readable by another local account. Passing the mode to
|
||||||
|
`os.open` closes that window for a brand-new file, but `O_CREAT`'s mode argument is only
|
||||||
|
applied on creation: if the file already exists its old, broader permissions carry over
|
||||||
|
untouched. `os.fchmod` right after opening -- before a single byte of the new content is
|
||||||
|
written -- covers both cases.
|
||||||
|
"""
|
||||||
|
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||||
|
os.fchmod(fd, 0o600)
|
||||||
|
f: IO[str] = os.fdopen(fd, "w")
|
||||||
|
try:
|
||||||
|
yield f
|
||||||
|
finally:
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
|
||||||
|
def write_backup(record: BackupRecord, backup_path: Path | None = None) -> None:
|
||||||
|
path = backup_path if backup_path is not None else BACKUP_PATH
|
||||||
|
path.parent.mkdir(exist_ok=True)
|
||||||
|
with secure_create(path) as f:
|
||||||
|
json.dump({"existed": record.existed, "content": record.content}, f, indent=2)
|
||||||
|
|
||||||
|
|
||||||
|
def read_backup(backup_path: Path | None = None) -> BackupRecord | None:
|
||||||
|
path = backup_path if backup_path is not None else BACKUP_PATH
|
||||||
|
if not path.exists():
|
||||||
|
return None
|
||||||
|
with open(path, "r") as f:
|
||||||
|
content = f.read()
|
||||||
|
try:
|
||||||
|
return _BACKUP_RECORD_ADAPTER.validate_json(content)
|
||||||
|
except ValidationError:
|
||||||
|
raise UpError(f"{path} contains invalid or unexpected JSON; cannot restore from it safely.")
|
||||||
|
|
||||||
|
|
||||||
|
def restore_claude_settings(settings_path: Path | None = None, backup_path: Path | None = None) -> BackupRecord | None:
|
||||||
|
"""Restore settings_path from the backup at backup_path, then delete the backup.
|
||||||
|
|
||||||
|
Returns the restored record, or None if there was nothing to restore.
|
||||||
|
"""
|
||||||
|
resolved_settings_path = settings_path if settings_path is not None else CLAUDE_SETTINGS_PATH
|
||||||
|
resolved_backup_path = backup_path if backup_path is not None else BACKUP_PATH
|
||||||
|
record = read_backup(resolved_backup_path)
|
||||||
|
if record is None:
|
||||||
|
return None
|
||||||
|
if record.existed and record.content is not None:
|
||||||
|
resolved_settings_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(resolved_settings_path, "w") as f:
|
||||||
|
json.dump(record.content, f, indent=2)
|
||||||
|
elif resolved_settings_path.exists():
|
||||||
|
resolved_settings_path.unlink()
|
||||||
|
resolved_backup_path.unlink()
|
||||||
|
return record
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_api_key_helper(base_url: str) -> str:
|
||||||
|
"""Build the shell command Claude Code should run for its apiKeyHelper.
|
||||||
|
|
||||||
|
Resolves `lite` to an absolute path so the helper works regardless of the
|
||||||
|
PATH visible to whatever subprocess Claude Code spawns it from. Passing
|
||||||
|
--base-url explicitly (rather than relying on the bare invocation Claude
|
||||||
|
Code would otherwise use) makes `print-token` enforce that the cached
|
||||||
|
token was actually issued for this proxy -- without it, a token minted
|
||||||
|
for a different, previously-logged-into proxy would be handed to
|
||||||
|
whichever server `up` currently points at.
|
||||||
|
"""
|
||||||
|
lite_path = shutil.which("lite")
|
||||||
|
if lite_path is None:
|
||||||
|
raise UpError(
|
||||||
|
"Could not find `lite` on your PATH. Claude Code's apiKeyHelper needs "
|
||||||
|
"an absolute path to it, so `lite up` cannot continue."
|
||||||
|
)
|
||||||
|
return f"{shlex.quote(lite_path)} auth print-token --base-url {shlex.quote(base_url)}"
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_fresh_login(ctx: click.Context) -> None:
|
||||||
|
base_url = ctx.obj["base_url"].rstrip("/")
|
||||||
|
token_data = load_token()
|
||||||
|
if token_data and token_data.get("base_url") == base_url and is_cli_token_fresh(token_data):
|
||||||
|
return
|
||||||
|
|
||||||
|
if not sys.stdin.isatty():
|
||||||
|
raise UpError(
|
||||||
|
"No fresh LiteLLM login found for this proxy. Run `lite login` first (apiKeyHelper "
|
||||||
|
"reads this token on every Claude Code request)."
|
||||||
|
)
|
||||||
|
|
||||||
|
click.echo("No fresh LiteLLM login found for this proxy; starting login...")
|
||||||
|
ctx.invoke(login)
|
||||||
|
token_data = load_token()
|
||||||
|
if not token_data or token_data.get("base_url") != base_url or not is_cli_token_fresh(token_data):
|
||||||
|
raise UpError("Login did not produce a usable token; cannot start `lite up`.")
|
||||||
|
|
||||||
|
|
||||||
|
def _restore_and_report() -> None:
|
||||||
|
record = restore_claude_settings()
|
||||||
|
if record is None:
|
||||||
|
click.echo("Nothing to restore.")
|
||||||
|
return
|
||||||
|
if record.existed:
|
||||||
|
click.echo(f"Restored {CLAUDE_SETTINGS_PATH} to its original contents.")
|
||||||
|
else:
|
||||||
|
click.echo(f"Removed {CLAUDE_SETTINGS_PATH} (it did not exist before `lite up`).")
|
||||||
|
|
||||||
|
|
||||||
|
@click.command(name="up")
|
||||||
|
@click.pass_context
|
||||||
|
def up(ctx: click.Context) -> None:
|
||||||
|
"""Route every Claude Code session through your LiteLLM proxy until stopped.
|
||||||
|
|
||||||
|
Patches ~/.claude/settings.json so Claude Code picks up the proxy on its own
|
||||||
|
next startup, from any terminal -- no need to launch it through `lite`.
|
||||||
|
Press Ctrl-C to stop and restore your original settings. Assumes the proxy
|
||||||
|
is already running (this does not start one for you). Cursor is not
|
||||||
|
supported: it has no equivalent file-based config to patch.
|
||||||
|
"""
|
||||||
|
base_url = ctx.obj["base_url"]
|
||||||
|
|
||||||
|
try:
|
||||||
|
_ensure_fresh_login(ctx)
|
||||||
|
api_key = resolve_api_key(ctx)
|
||||||
|
verify_proxy_key(base_url, api_key)
|
||||||
|
|
||||||
|
if BACKUP_PATH.exists():
|
||||||
|
raise UpError(
|
||||||
|
f"{BACKUP_PATH} already exists -- `lite up` looks like it's already "
|
||||||
|
"running (or crashed without cleanup). Run `lite down` first."
|
||||||
|
)
|
||||||
|
|
||||||
|
api_key_helper = resolve_api_key_helper(base_url)
|
||||||
|
original_existed = CLAUDE_SETTINGS_PATH.exists()
|
||||||
|
original_settings = load_json_or_empty(CLAUDE_SETTINGS_PATH)
|
||||||
|
write_backup(
|
||||||
|
BackupRecord(
|
||||||
|
existed=original_existed,
|
||||||
|
content=original_settings if original_existed else None,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
CLAUDE_SETTINGS_PATH.parent.mkdir(exist_ok=True)
|
||||||
|
merged = merge_claude_settings(original_settings, base_url, api_key_helper)
|
||||||
|
with open(CLAUDE_SETTINGS_PATH, "w") as f:
|
||||||
|
json.dump(merged, f, indent=2)
|
||||||
|
except (AgentRunError, UpError) as e:
|
||||||
|
raise click.ClickException(str(e))
|
||||||
|
|
||||||
|
click.echo(f"litellm: routing Claude Code through proxy at {base_url.rstrip('/')}")
|
||||||
|
click.echo("Press Ctrl-C to stop and restore your original settings.")
|
||||||
|
|
||||||
|
stop_event = threading.Event()
|
||||||
|
restored = threading.Lock()
|
||||||
|
|
||||||
|
def _handle_signal(_signum: int, _frame: FrameType | None) -> None:
|
||||||
|
stop_event.set()
|
||||||
|
|
||||||
|
def _restore_once() -> None:
|
||||||
|
if not restored.acquire(blocking=False):
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
_restore_and_report()
|
||||||
|
except UpError as e:
|
||||||
|
# Runs from atexit/a signal handler, outside Click's own exception
|
||||||
|
# handling -- raising here would only produce an unhandled-exception
|
||||||
|
# warning on stderr, not a clean message.
|
||||||
|
click.echo(str(e), err=True)
|
||||||
|
|
||||||
|
signal.signal(signal.SIGINT, _handle_signal)
|
||||||
|
signal.signal(signal.SIGTERM, _handle_signal)
|
||||||
|
atexit.register(_restore_once)
|
||||||
|
|
||||||
|
stop_event.wait()
|
||||||
|
_restore_once()
|
||||||
|
|
||||||
|
|
||||||
|
@click.command(name="down")
|
||||||
|
def down() -> None:
|
||||||
|
"""Restore ~/.claude/settings.json if a `lite up` session left it patched.
|
||||||
|
|
||||||
|
Use this after a `lite up` process was killed uncleanly (e.g. `kill -9`)
|
||||||
|
instead of stopped with Ctrl-C.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
_restore_and_report()
|
||||||
|
except UpError as e:
|
||||||
|
raise click.ClickException(str(e))
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"BACKUP_PATH",
|
||||||
|
"CLAUDE_SETTINGS_PATH",
|
||||||
|
"BackupRecord",
|
||||||
|
"UpError",
|
||||||
|
"down",
|
||||||
|
"load_json_or_empty",
|
||||||
|
"merge_claude_settings",
|
||||||
|
"read_backup",
|
||||||
|
"resolve_api_key_helper",
|
||||||
|
"restore_claude_settings",
|
||||||
|
"up",
|
||||||
|
"write_backup",
|
||||||
|
]
|
||||||
|
|
@ -9,15 +9,18 @@ from litellm.proxy.client.health import HealthManagementClient
|
||||||
|
|
||||||
from .commands.agents import agent_commands
|
from .commands.agents import agent_commands
|
||||||
from .commands.auth import auth_group, get_stored_api_key, login, logout, whoami
|
from .commands.auth import auth_group, get_stored_api_key, login, logout, whoami
|
||||||
|
from .commands.autoroute.commands import autoroute_group
|
||||||
from .commands.chat import chat
|
from .commands.chat import chat
|
||||||
from .commands.credentials import credentials
|
from .commands.credentials import credentials
|
||||||
from .commands.encryption import encryption
|
from .commands.encryption import encryption
|
||||||
from .commands.http import http
|
from .commands.http import http
|
||||||
from .commands.keys import keys
|
from .commands.keys import keys
|
||||||
|
from .commands.model_groups import model_groups
|
||||||
|
|
||||||
# local imports
|
# local imports
|
||||||
from .commands.models import models
|
from .commands.models import models
|
||||||
from .commands.teams import teams
|
from .commands.teams import teams
|
||||||
|
from .commands.up import down, up
|
||||||
from .commands.users import users
|
from .commands.users import users
|
||||||
from .interface import interactive_shell
|
from .interface import interactive_shell
|
||||||
|
|
||||||
|
|
@ -131,6 +134,13 @@ cli.add_command(users)
|
||||||
# Add a top-level command per coding agent (claude, codex, opencode, ...)
|
# Add a top-level command per coding agent (claude, codex, opencode, ...)
|
||||||
for agent_command in agent_commands():
|
for agent_command in agent_commands():
|
||||||
cli.add_command(agent_command)
|
cli.add_command(agent_command)
|
||||||
|
# Add the up/down commands (route Claude Code through the local LiteLLM proxy)
|
||||||
|
cli.add_command(up)
|
||||||
|
cli.add_command(down)
|
||||||
|
# Add the model-groups command group (discover models your key can access)
|
||||||
|
cli.add_command(model_groups)
|
||||||
|
# Add the autoroute command group (QA auto-routing against your real proxy)
|
||||||
|
cli.add_command(autoroute_group, name="autoroute")
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|
|
||||||
|
|
@ -66,6 +66,7 @@ proxy = [
|
||||||
"litellm-enterprise==0.1.50",
|
"litellm-enterprise==0.1.50",
|
||||||
"RestrictedPython>=8.1,<9.0",
|
"RestrictedPython>=8.1,<9.0",
|
||||||
"rich>=13.9.4,<14.0",
|
"rich>=13.9.4,<14.0",
|
||||||
|
"InquirerPy>=0.3.4,<1.0",
|
||||||
"polars>=1.38.1,<2.0",
|
"polars>=1.38.1,<2.0",
|
||||||
"soundfile>=0.12.1,<1.0",
|
"soundfile>=0.12.1,<1.0",
|
||||||
"pyroscope-io>=0.8.16,<1.0; sys_platform != 'win32'",
|
"pyroscope-io>=0.8.16,<1.0; sys_platform != 'win32'",
|
||||||
|
|
@ -74,11 +75,12 @@ proxy = [
|
||||||
]
|
]
|
||||||
# Thin client install for the `lite` CLI on developer laptops. The CLI's heavy
|
# Thin client install for the `lite` CLI on developer laptops. The CLI's heavy
|
||||||
# imports (fastapi, cryptography, ...) are all guarded, so it runs on the base
|
# imports (fastapi, cryptography, ...) are all guarded, so it runs on the base
|
||||||
# SDK plus just these three; none of the server runtime in `proxy` is pulled in.
|
# SDK plus just these four; none of the server runtime in `proxy` is pulled in.
|
||||||
cli = [
|
cli = [
|
||||||
"rich>=13.9.4,<14.0",
|
"rich>=13.9.4,<14.0",
|
||||||
"pyyaml>=6.0.3,<7.0",
|
"pyyaml>=6.0.3,<7.0",
|
||||||
"requests>=2.32.0,<3.0",
|
"requests>=2.32.0,<3.0",
|
||||||
|
"InquirerPy>=0.3.4,<1.0",
|
||||||
]
|
]
|
||||||
extra_proxy = [
|
extra_proxy = [
|
||||||
"prisma>=0.11.0,<1.0",
|
"prisma>=0.11.0,<1.0",
|
||||||
|
|
|
||||||
|
|
@ -11,12 +11,21 @@
|
||||||
# Python itself (honouring litellm's requires-python), downloading a managed one
|
# Python itself (honouring litellm's requires-python), downloading a managed one
|
||||||
# when the host has no suitable interpreter.
|
# when the host has no suitable interpreter.
|
||||||
#
|
#
|
||||||
|
# To try an unreleased branch instead of the latest PyPI release (for example, to
|
||||||
|
# QA a CLI feature before it ships), set LITELLM_CLI_REF to a branch, tag, or commit:
|
||||||
|
# curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/<branch>/scripts/install-cli.sh | \
|
||||||
|
# LITELLM_CLI_REF=<branch> sh
|
||||||
|
#
|
||||||
# NOTE: set -e without pipefail for POSIX sh compatibility (dash on Ubuntu/Debian
|
# NOTE: set -e without pipefail for POSIX sh compatibility (dash on Ubuntu/Debian
|
||||||
# ignores the shebang when invoked as `sh` and does not support `pipefail`).
|
# ignores the shebang when invoked as `sh` and does not support `pipefail`).
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
# NOTE: before merging, this must stay as "litellm[cli]" to install from PyPI.
|
# Defaults to the PyPI release; LITELLM_CLI_REF opts into installing from source instead.
|
||||||
LITELLM_PACKAGE="litellm[cli]"
|
if [ -n "${LITELLM_CLI_REF:-}" ]; then
|
||||||
|
LITELLM_PACKAGE="litellm[cli] @ git+https://github.com/BerriAI/litellm.git@${LITELLM_CLI_REF}"
|
||||||
|
else
|
||||||
|
LITELLM_PACKAGE="litellm[cli]"
|
||||||
|
fi
|
||||||
UV_VERSION="0.10.9"
|
UV_VERSION="0.10.9"
|
||||||
|
|
||||||
# ── colours ────────────────────────────────────────────────────────────────
|
# ── colours ────────────────────────────────────────────────────────────────
|
||||||
|
|
@ -90,7 +99,11 @@ fi
|
||||||
# otherwise download a managed one. Either way uv honours litellm's requires-python,
|
# otherwise download a managed one. Either way uv honours litellm's requires-python,
|
||||||
# so a too-old (3.9) or too-new (3.14+) system Python is skipped, not forced.
|
# so a too-old (3.9) or too-new (3.14+) system Python is skipped, not forced.
|
||||||
echo ""
|
echo ""
|
||||||
header "Installing litellm[cli]…"
|
if [ -n "${LITELLM_CLI_REF:-}" ]; then
|
||||||
|
header "Installing litellm[cli] from ${LITELLM_CLI_REF}…"
|
||||||
|
else
|
||||||
|
header "Installing litellm[cli]…"
|
||||||
|
fi
|
||||||
echo ""
|
echo ""
|
||||||
|
|
||||||
"$UV_BIN" tool install --python-preference system --force "${LITELLM_PACKAGE}" \
|
"$UV_BIN" tool install --python-preference system --force "${LITELLM_PACKAGE}" \
|
||||||
|
|
|
||||||
300
tests/test_litellm/proxy/client/cli/autoroute/test_commands.py
Normal file
300
tests/test_litellm/proxy/client/cli/autoroute/test_commands.py
Normal file
|
|
@ -0,0 +1,300 @@
|
||||||
|
import json
|
||||||
|
import stat
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
from click.testing import CliRunner
|
||||||
|
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute import commands as commands_module
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute import process as process_module
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute.commands import down, up
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute.process import PidRecord, ProcessLaunchError, write_pid_record
|
||||||
|
from litellm.proxy.client.cli.commands.up import BackupRecord as ClaudeBackupRecord
|
||||||
|
from litellm.proxy.client.cli.commands.up import write_backup
|
||||||
|
|
||||||
|
|
||||||
|
class FakeProcess:
|
||||||
|
def __init__(self, pid: int):
|
||||||
|
self.pid = pid
|
||||||
|
self.returncode: Optional[int] = None
|
||||||
|
|
||||||
|
def poll(self) -> Optional[int]:
|
||||||
|
return self.returncode
|
||||||
|
|
||||||
|
|
||||||
|
def _patch_paths(monkeypatch, tmp_path):
|
||||||
|
config_path = tmp_path / "config.yaml"
|
||||||
|
log_path = tmp_path / "proxy.log"
|
||||||
|
claude_settings_path = tmp_path / "claude_settings.json"
|
||||||
|
backup_path = tmp_path / "backup.json"
|
||||||
|
pid_record_path = tmp_path / "pid.json"
|
||||||
|
|
||||||
|
monkeypatch.setattr(commands_module, "CONFIG_PATH", config_path)
|
||||||
|
monkeypatch.setattr(commands_module, "LOG_PATH", log_path)
|
||||||
|
monkeypatch.setattr(commands_module, "CLAUDE_SETTINGS_PATH", claude_settings_path)
|
||||||
|
monkeypatch.setattr(commands_module, "AUTOROUTE_BACKUP_PATH", backup_path)
|
||||||
|
monkeypatch.setattr(process_module, "PID_RECORD_PATH", pid_record_path)
|
||||||
|
|
||||||
|
return config_path, log_path, claude_settings_path, backup_path, pid_record_path
|
||||||
|
|
||||||
|
|
||||||
|
def _silence_signal_handling(monkeypatch):
|
||||||
|
monkeypatch.setattr(commands_module.signal, "signal", lambda *a, **k: None)
|
||||||
|
monkeypatch.setattr(commands_module.atexit, "register", lambda *a, **k: None)
|
||||||
|
monkeypatch.setattr(commands_module, "stream_log", lambda *a, **k: None)
|
||||||
|
|
||||||
|
|
||||||
|
class TestUpCommand:
|
||||||
|
def setup_method(self):
|
||||||
|
self.runner = CliRunner()
|
||||||
|
|
||||||
|
def test_refuses_when_never_configured(self, monkeypatch, tmp_path):
|
||||||
|
_patch_paths(monkeypatch, tmp_path)
|
||||||
|
|
||||||
|
result = self.runner.invoke(up)
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "lite autoroute configure" in result.output
|
||||||
|
|
||||||
|
def test_surfaces_clean_error_on_empty_config_file(self, monkeypatch, tmp_path):
|
||||||
|
"""A `configure` killed between secure_create's O_TRUNC and the write completing leaves an
|
||||||
|
empty config.yaml on disk -- yaml.safe_load(empty) returns None, and validating None as the
|
||||||
|
generated-config model raises a raw pydantic.ValidationError if uncaught."""
|
||||||
|
config_path, _log_path, _settings_path, _backup_path, _pid_record_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
config_path.write_text("")
|
||||||
|
|
||||||
|
result = self.runner.invoke(up)
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert result.exception is None or isinstance(result.exception, SystemExit)
|
||||||
|
assert "lite autoroute configure" in result.output
|
||||||
|
|
||||||
|
def test_refuses_when_pid_record_exists_and_process_still_running(self, monkeypatch, tmp_path):
|
||||||
|
config_path, _log_path, _settings_path, _backup_path, pid_record_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
config_path.write_text(yaml.safe_dump({"model_list": []}))
|
||||||
|
write_pid_record(
|
||||||
|
PidRecord(pid=123, port=4000, config_path=str(config_path), log_path="/tmp/proxy.log"), pid_record_path
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(commands_module, "is_running", lambda pid: True)
|
||||||
|
|
||||||
|
result = self.runner.invoke(up)
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "already running" in result.output
|
||||||
|
assert "lite autoroute down" in result.output
|
||||||
|
assert config_path.read_text() == yaml.safe_dump({"model_list": []})
|
||||||
|
|
||||||
|
def test_refuses_when_backup_exists_after_an_unclean_crash(self, monkeypatch, tmp_path):
|
||||||
|
"""A prior `up` that was SIGKILL'd leaves no live pid but does leave a stale backup file.
|
||||||
|
|
||||||
|
Without this guard, a fresh `up` would overwrite that backup with the currently-patched
|
||||||
|
(not original) Claude settings, so `down`/Ctrl-C would restore the wrong content forever.
|
||||||
|
"""
|
||||||
|
config_path, _log_path, claude_settings_path, backup_path, _pid_record_path = _patch_paths(
|
||||||
|
monkeypatch, tmp_path
|
||||||
|
)
|
||||||
|
config_path.write_text(yaml.safe_dump({"model_list": []}))
|
||||||
|
claude_settings_path.write_text(json.dumps({"env": {"ANTHROPIC_AUTH_TOKEN": "stale-patched-token"}}))
|
||||||
|
write_backup(ClaudeBackupRecord(existed=True, content={"theme": "dark"}), backup_path)
|
||||||
|
|
||||||
|
result = self.runner.invoke(up)
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "already exists" in result.output
|
||||||
|
assert "lite autoroute down" in result.output
|
||||||
|
assert json.loads(backup_path.read_text())["content"] == {"theme": "dark"}
|
||||||
|
|
||||||
|
def test_happy_path_patches_settings_then_restores_everything_on_stop(self, monkeypatch, tmp_path):
|
||||||
|
config_path, log_path, claude_settings_path, backup_path, pid_record_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
config_path.write_text(yaml.safe_dump({"model_list": []}))
|
||||||
|
original_settings = {"theme": "dark"}
|
||||||
|
claude_settings_path.write_text(json.dumps(original_settings))
|
||||||
|
_silence_signal_handling(monkeypatch)
|
||||||
|
|
||||||
|
fake_process = FakeProcess(pid=99999)
|
||||||
|
terminate_calls = []
|
||||||
|
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: fake_process)
|
||||||
|
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||||
|
monkeypatch.setattr(commands_module, "allocate_free_port", lambda: 54321)
|
||||||
|
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: terminate_calls.append(pid))
|
||||||
|
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fixed-master-key")
|
||||||
|
|
||||||
|
captured = {}
|
||||||
|
|
||||||
|
def fake_wait(self, timeout=None):
|
||||||
|
captured["settings"] = json.loads(claude_settings_path.read_text())
|
||||||
|
captured["backup_existed"] = backup_path.exists()
|
||||||
|
captured["settings_mode"] = stat.S_IMODE(claude_settings_path.stat().st_mode)
|
||||||
|
return True
|
||||||
|
|
||||||
|
monkeypatch.setattr("threading.Event.wait", fake_wait)
|
||||||
|
|
||||||
|
result = self.runner.invoke(up)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert captured["backup_existed"] is True
|
||||||
|
assert captured["settings"]["theme"] == "dark"
|
||||||
|
assert captured["settings"]["env"]["ANTHROPIC_BASE_URL"] == "http://127.0.0.1:54321"
|
||||||
|
assert captured["settings"]["env"]["ANTHROPIC_AUTH_TOKEN"] == "fixed-master-key"
|
||||||
|
assert "apiKeyHelper" not in captured["settings"]
|
||||||
|
assert captured["settings_mode"] == 0o600
|
||||||
|
|
||||||
|
assert terminate_calls == [99999]
|
||||||
|
assert not pid_record_path.exists()
|
||||||
|
assert not backup_path.exists()
|
||||||
|
assert json.loads(claude_settings_path.read_text()) == original_settings
|
||||||
|
|
||||||
|
written_config = yaml.safe_load(config_path.read_text())
|
||||||
|
assert written_config["general_settings"]["master_key"] == "fixed-master-key"
|
||||||
|
assert stat.S_IMODE(config_path.stat().st_mode) == 0o600
|
||||||
|
|
||||||
|
def test_teardown_reports_clean_error_when_backup_is_corrupt(self, monkeypatch, tmp_path):
|
||||||
|
"""A corrupt backup at teardown time (e.g. a concurrent process wrote garbage to it) must
|
||||||
|
not crash the whole command -- _restore_once in up.py handles the identical case in
|
||||||
|
lite up the same way, echoing the error instead of propagating it."""
|
||||||
|
config_path, _log_path, claude_settings_path, backup_path, pid_record_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
config_path.write_text(yaml.safe_dump({"model_list": []}))
|
||||||
|
claude_settings_path.write_text(json.dumps({"theme": "dark"}))
|
||||||
|
_silence_signal_handling(monkeypatch)
|
||||||
|
|
||||||
|
fake_process = FakeProcess(pid=11111)
|
||||||
|
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: fake_process)
|
||||||
|
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||||
|
monkeypatch.setattr(commands_module, "allocate_free_port", lambda: 65432)
|
||||||
|
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: None)
|
||||||
|
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fixed-master-key")
|
||||||
|
|
||||||
|
def fake_wait(self, timeout=None):
|
||||||
|
backup_path.write_text("not json at all {{{")
|
||||||
|
return True
|
||||||
|
|
||||||
|
monkeypatch.setattr("threading.Event.wait", fake_wait)
|
||||||
|
|
||||||
|
result = self.runner.invoke(up)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "invalid or unexpected JSON" in result.output
|
||||||
|
assert not pid_record_path.exists()
|
||||||
|
|
||||||
|
def test_surfaces_clean_error_and_cleans_up_when_health_check_fails(self, monkeypatch, tmp_path):
|
||||||
|
config_path, _log_path, claude_settings_path, backup_path, pid_record_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
config_path.write_text(yaml.safe_dump({"model_list": []}))
|
||||||
|
original_settings = {"theme": "dark"}
|
||||||
|
claude_settings_path.write_text(json.dumps(original_settings))
|
||||||
|
|
||||||
|
fake_process = FakeProcess(pid=555)
|
||||||
|
terminate_calls = []
|
||||||
|
|
||||||
|
def _raise_launch_error(*args, **kwargs):
|
||||||
|
raise ProcessLaunchError("boom: proxy never became healthy")
|
||||||
|
|
||||||
|
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: fake_process)
|
||||||
|
monkeypatch.setattr(commands_module, "poll_liveliness", _raise_launch_error)
|
||||||
|
monkeypatch.setattr(commands_module, "allocate_free_port", lambda: 12345)
|
||||||
|
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: terminate_calls.append(pid))
|
||||||
|
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fixed-master-key")
|
||||||
|
|
||||||
|
result = self.runner.invoke(up)
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "boom" in result.output
|
||||||
|
assert terminate_calls == [555]
|
||||||
|
assert not pid_record_path.exists()
|
||||||
|
assert not backup_path.exists()
|
||||||
|
assert json.loads(claude_settings_path.read_text()) == original_settings
|
||||||
|
|
||||||
|
def test_terminates_ephemeral_proxy_when_claude_settings_is_corrupt(self, monkeypatch, tmp_path):
|
||||||
|
"""The health check can pass and the proxy can come up fine, but if
|
||||||
|
~/.claude/settings.json turns out to be corrupt, the just-started proxy must not be left
|
||||||
|
running with no pid record -- exactly the leak `lite autoroute down` exists to clean up."""
|
||||||
|
config_path, _log_path, claude_settings_path, backup_path, pid_record_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
config_path.write_text(yaml.safe_dump({"model_list": []}))
|
||||||
|
claude_settings_path.write_text("not json at all {{{")
|
||||||
|
|
||||||
|
fake_process = FakeProcess(pid=777)
|
||||||
|
terminate_calls = []
|
||||||
|
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: fake_process)
|
||||||
|
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||||
|
monkeypatch.setattr(commands_module, "allocate_free_port", lambda: 23456)
|
||||||
|
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: terminate_calls.append(pid))
|
||||||
|
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fixed-master-key")
|
||||||
|
|
||||||
|
result = self.runner.invoke(up)
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "invalid JSON" in result.output
|
||||||
|
assert terminate_calls == [777]
|
||||||
|
assert not pid_record_path.exists()
|
||||||
|
assert not backup_path.exists()
|
||||||
|
|
||||||
|
|
||||||
|
class TestDownCommand:
|
||||||
|
def setup_method(self):
|
||||||
|
self.runner = CliRunner()
|
||||||
|
|
||||||
|
def test_restores_settings_and_terminates_when_process_still_running(self, monkeypatch, tmp_path):
|
||||||
|
_config_path, _log_path, claude_settings_path, backup_path, pid_record_path = _patch_paths(
|
||||||
|
monkeypatch, tmp_path
|
||||||
|
)
|
||||||
|
original_settings = {"theme": "dark"}
|
||||||
|
write_backup(ClaudeBackupRecord(existed=True, content=original_settings), backup_path)
|
||||||
|
claude_settings_path.write_text(json.dumps({"env": {"ANTHROPIC_AUTH_TOKEN": "fixed-master-key"}}))
|
||||||
|
write_pid_record(PidRecord(pid=777, port=1234, config_path="c", log_path="l"), pid_record_path)
|
||||||
|
|
||||||
|
terminate_calls = []
|
||||||
|
monkeypatch.setattr(commands_module, "is_running", lambda pid: True)
|
||||||
|
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: terminate_calls.append(pid))
|
||||||
|
|
||||||
|
result = self.runner.invoke(down)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "Stopped leftover ephemeral proxy" in result.output
|
||||||
|
assert "Restored" in result.output
|
||||||
|
assert terminate_calls == [777]
|
||||||
|
assert not pid_record_path.exists()
|
||||||
|
assert not backup_path.exists()
|
||||||
|
assert json.loads(claude_settings_path.read_text()) == original_settings
|
||||||
|
|
||||||
|
def test_is_a_clean_no_op_when_nothing_is_running_and_no_backup_exists(self, monkeypatch, tmp_path):
|
||||||
|
_config_path, _log_path, claude_settings_path, _backup_path, _pid_record_path = _patch_paths(
|
||||||
|
monkeypatch, tmp_path
|
||||||
|
)
|
||||||
|
|
||||||
|
result = self.runner.invoke(down)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "Nothing to restore." in result.output
|
||||||
|
assert not claude_settings_path.exists()
|
||||||
|
|
||||||
|
def test_clears_a_corrupt_pid_record_and_still_restores_settings(self, monkeypatch, tmp_path):
|
||||||
|
"""down is specifically the crash-recovery path -- a pid file truncated by a mid-write
|
||||||
|
crash must not block it from clearing the record and restoring Claude settings anyway."""
|
||||||
|
_config_path, _log_path, claude_settings_path, backup_path, pid_record_path = _patch_paths(
|
||||||
|
monkeypatch, tmp_path
|
||||||
|
)
|
||||||
|
pid_record_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
pid_record_path.write_text("not json at all {{{")
|
||||||
|
original_settings = {"theme": "dark"}
|
||||||
|
write_backup(ClaudeBackupRecord(existed=True, content=original_settings), backup_path)
|
||||||
|
claude_settings_path.write_text(json.dumps({"env": {"ANTHROPIC_AUTH_TOKEN": "fixed-master-key"}}))
|
||||||
|
|
||||||
|
result = self.runner.invoke(down)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "invalid or unexpected JSON" in result.output
|
||||||
|
assert "Restored" in result.output
|
||||||
|
assert not pid_record_path.exists()
|
||||||
|
assert not backup_path.exists()
|
||||||
|
assert json.loads(claude_settings_path.read_text()) == original_settings
|
||||||
|
|
||||||
|
def test_surfaces_clean_error_when_backup_is_corrupt(self, monkeypatch, tmp_path):
|
||||||
|
_config_path, _log_path, _claude_settings_path, backup_path, _pid_record_path = _patch_paths(
|
||||||
|
monkeypatch, tmp_path
|
||||||
|
)
|
||||||
|
backup_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
backup_path.write_text("not json at all {{{")
|
||||||
|
|
||||||
|
result = self.runner.invoke(down)
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "invalid or unexpected JSON" in result.output
|
||||||
181
tests/test_litellm/proxy/client/cli/autoroute/test_config.py
Normal file
181
tests/test_litellm/proxy/client/cli/autoroute/test_config.py
Normal file
|
|
@ -0,0 +1,181 @@
|
||||||
|
from typing import Any, Dict, Tuple
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute.config import (
|
||||||
|
AutorouteConfig,
|
||||||
|
ConfigGenerationError,
|
||||||
|
DiscoveredModel,
|
||||||
|
HeuristicClassifier,
|
||||||
|
LLMClassifier,
|
||||||
|
NoSemanticMatching,
|
||||||
|
SemanticMatching,
|
||||||
|
build_generated_model_list,
|
||||||
|
build_generated_proxy_config,
|
||||||
|
chat_models,
|
||||||
|
embedding_models,
|
||||||
|
parse_discovered_models,
|
||||||
|
validate_config,
|
||||||
|
)
|
||||||
|
|
||||||
|
DISCOVERED: Tuple[DiscoveredModel, ...] = (
|
||||||
|
DiscoveredModel(name="gpt-4o-mini", mode="chat"),
|
||||||
|
DiscoveredModel(name="gpt-4o", mode="chat"),
|
||||||
|
DiscoveredModel(name="o1", mode="chat"),
|
||||||
|
DiscoveredModel(name="text-embedding-3-small", mode="embedding"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _base_config(**overrides: Any) -> AutorouteConfig:
|
||||||
|
defaults: Dict[str, Any] = {
|
||||||
|
"base_url": "http://real-proxy.internal:4000",
|
||||||
|
"api_key": "sk-real-key",
|
||||||
|
"tiers": {
|
||||||
|
"SIMPLE": ("gpt-4o-mini",),
|
||||||
|
"MEDIUM": ("gpt-4o",),
|
||||||
|
"COMPLEX": ("gpt-4o",),
|
||||||
|
"REASONING": ("o1",),
|
||||||
|
},
|
||||||
|
"default_model": "gpt-4o",
|
||||||
|
}
|
||||||
|
defaults.update(overrides)
|
||||||
|
return AutorouteConfig(**defaults)
|
||||||
|
|
||||||
|
|
||||||
|
class TestParseDiscoveredModels:
|
||||||
|
def test_parses_valid_raw_list_into_typed_tuple(self):
|
||||||
|
raw = [
|
||||||
|
{
|
||||||
|
"model_group": "gpt-4o",
|
||||||
|
"mode": "chat",
|
||||||
|
"input_cost_per_token": 0.01,
|
||||||
|
"output_cost_per_token": 0.02,
|
||||||
|
},
|
||||||
|
{"model_group": "text-embedding-3-small", "mode": "embedding"},
|
||||||
|
]
|
||||||
|
result = parse_discovered_models(raw)
|
||||||
|
assert result == (
|
||||||
|
DiscoveredModel(name="gpt-4o", mode="chat", input_cost_per_token=0.01, output_cost_per_token=0.02),
|
||||||
|
DiscoveredModel(name="text-embedding-3-small", mode="embedding"),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_ignores_unknown_extra_fields(self):
|
||||||
|
raw = [{"model_group": "gpt-4o", "mode": "chat", "totally_unknown_field": "whatever"}]
|
||||||
|
result = parse_discovered_models(raw)
|
||||||
|
assert result == (DiscoveredModel(name="gpt-4o", mode="chat"),)
|
||||||
|
|
||||||
|
def test_missing_mode_defaults_to_chat(self):
|
||||||
|
raw = [{"model_group": "gpt-4o"}]
|
||||||
|
result = parse_discovered_models(raw)
|
||||||
|
assert result[0].mode == "chat"
|
||||||
|
|
||||||
|
|
||||||
|
class TestChatAndEmbeddingFiltering:
|
||||||
|
def test_filters_by_mode(self):
|
||||||
|
models = (
|
||||||
|
DiscoveredModel(name="gpt-4o", mode="chat"),
|
||||||
|
DiscoveredModel(name="text-embedding-3-small", mode="embedding"),
|
||||||
|
DiscoveredModel(name="claude", mode="chat"),
|
||||||
|
)
|
||||||
|
assert chat_models(models) == (models[0], models[2])
|
||||||
|
assert embedding_models(models) == (models[1],)
|
||||||
|
|
||||||
|
|
||||||
|
class TestBuildGeneratedModelList:
|
||||||
|
def test_dedups_model_used_in_multiple_roles(self):
|
||||||
|
config = _base_config(classifier=LLMClassifier(model="gpt-4o"))
|
||||||
|
model_list = build_generated_model_list(config)
|
||||||
|
gpt4o_entries = [m for m in model_list if m["model_name"] == "gpt-4o"]
|
||||||
|
assert len(gpt4o_entries) == 1
|
||||||
|
|
||||||
|
def test_every_proxy_deployment_points_back_at_customer_proxy(self):
|
||||||
|
config = _base_config()
|
||||||
|
model_list = build_generated_model_list(config)
|
||||||
|
proxy_entries = [m for m in model_list if m["model_name"] not in ("autorouter", "*")]
|
||||||
|
names = {m["model_name"] for m in proxy_entries}
|
||||||
|
assert names == {"gpt-4o-mini", "gpt-4o", "o1"}
|
||||||
|
for entry in proxy_entries:
|
||||||
|
assert entry["litellm_params"]["model"] == f"litellm_proxy/{entry['model_name']}"
|
||||||
|
assert entry["litellm_params"]["api_base"] == config.base_url
|
||||||
|
assert entry["litellm_params"]["api_key"] == config.api_key
|
||||||
|
|
||||||
|
def test_no_wildcard_deployment_is_generated(self):
|
||||||
|
# A bare "*" model_name looks like the obvious catch-all, but Router's auto-router
|
||||||
|
# registry is keyed by the literal requested model string with no wildcard resolution
|
||||||
|
# (litellm/router.py:10711-10717), so a "*" entry here would silently never match real
|
||||||
|
# traffic. Regression guard: don't reintroduce it.
|
||||||
|
config = _base_config()
|
||||||
|
model_list = build_generated_model_list(config)
|
||||||
|
assert not any(m["model_name"] == "*" for m in model_list)
|
||||||
|
|
||||||
|
def test_complexity_router_config_reflects_llm_classifier(self):
|
||||||
|
config = _base_config(classifier=LLMClassifier(model="gpt-4o", timeout_ms=1234))
|
||||||
|
autorouter = next(m for m in build_generated_model_list(config) if m["model_name"] == "autorouter")
|
||||||
|
router_config = autorouter["litellm_params"]["complexity_router_config"]
|
||||||
|
assert router_config["classifier_type"] == "llm"
|
||||||
|
assert router_config["classifier_llm_config"] == {"model": "gpt-4o", "timeout_ms": 1234}
|
||||||
|
assert "semantic_keyword_matching" not in router_config
|
||||||
|
assert "adaptive" not in router_config
|
||||||
|
|
||||||
|
def test_complexity_router_config_reflects_semantic_matching(self):
|
||||||
|
config = _base_config(
|
||||||
|
semantic_matching=SemanticMatching(embedding_model="text-embedding-3-small", match_threshold=0.7)
|
||||||
|
)
|
||||||
|
autorouter = next(m for m in build_generated_model_list(config) if m["model_name"] == "autorouter")
|
||||||
|
router_config = autorouter["litellm_params"]["complexity_router_config"]
|
||||||
|
assert router_config["semantic_keyword_matching"] is True
|
||||||
|
assert router_config["embedding_model"] == "text-embedding-3-small"
|
||||||
|
assert router_config["match_threshold"] == 0.7
|
||||||
|
assert router_config["keyword_tier_rules"]
|
||||||
|
assert "classifier_type" not in router_config
|
||||||
|
|
||||||
|
def test_complexity_router_config_reflects_adaptive(self):
|
||||||
|
config = _base_config(adaptive=True)
|
||||||
|
autorouter = next(m for m in build_generated_model_list(config) if m["model_name"] == "autorouter")
|
||||||
|
assert autorouter["litellm_params"]["complexity_router_config"]["adaptive"] is True
|
||||||
|
|
||||||
|
def test_default_classifier_and_semantic_matching_add_no_extra_keys(self):
|
||||||
|
config = _base_config(classifier=HeuristicClassifier(), semantic_matching=NoSemanticMatching())
|
||||||
|
autorouter = next(m for m in build_generated_model_list(config) if m["model_name"] == "autorouter")
|
||||||
|
router_config = autorouter["litellm_params"]["complexity_router_config"]
|
||||||
|
assert set(router_config.keys()) == {"tiers", "default_model"}
|
||||||
|
|
||||||
|
|
||||||
|
class TestBuildGeneratedProxyConfig:
|
||||||
|
def test_embeds_master_key_under_general_settings(self):
|
||||||
|
config = _base_config()
|
||||||
|
proxy_config = build_generated_proxy_config(config, "sk-master-123")
|
||||||
|
assert proxy_config["general_settings"] == {"master_key": "sk-master-123"}
|
||||||
|
assert proxy_config["model_list"] == build_generated_model_list(config)
|
||||||
|
|
||||||
|
|
||||||
|
class TestValidateConfig:
|
||||||
|
def test_passes_for_fully_valid_config(self):
|
||||||
|
validate_config(_base_config(), DISCOVERED)
|
||||||
|
|
||||||
|
def test_raises_for_tier_referencing_unknown_model(self):
|
||||||
|
config = _base_config(
|
||||||
|
tiers={
|
||||||
|
"SIMPLE": ("unknown-model",),
|
||||||
|
"MEDIUM": ("gpt-4o",),
|
||||||
|
"COMPLEX": ("gpt-4o",),
|
||||||
|
"REASONING": ("o1",),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
with pytest.raises(ConfigGenerationError, match="unknown-model"):
|
||||||
|
validate_config(config, DISCOVERED)
|
||||||
|
|
||||||
|
def test_raises_for_unknown_default_model(self):
|
||||||
|
config = _base_config(default_model="unknown-model")
|
||||||
|
with pytest.raises(ConfigGenerationError, match="unknown-model"):
|
||||||
|
validate_config(config, DISCOVERED)
|
||||||
|
|
||||||
|
def test_raises_for_unknown_llm_classifier_model(self):
|
||||||
|
config = _base_config(classifier=LLMClassifier(model="unknown-model"))
|
||||||
|
with pytest.raises(ConfigGenerationError, match="unknown-model"):
|
||||||
|
validate_config(config, DISCOVERED)
|
||||||
|
|
||||||
|
def test_raises_for_unknown_semantic_embedding_model(self):
|
||||||
|
config = _base_config(semantic_matching=SemanticMatching(embedding_model="unknown-embedding"))
|
||||||
|
with pytest.raises(ConfigGenerationError, match="unknown-embedding"):
|
||||||
|
validate_config(config, DISCOVERED)
|
||||||
139
tests/test_litellm/proxy/client/cli/autoroute/test_process.py
Normal file
139
tests/test_litellm/proxy/client/cli/autoroute/test_process.py
Normal file
|
|
@ -0,0 +1,139 @@
|
||||||
|
import os
|
||||||
|
import socket
|
||||||
|
from typing import Optional
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute import process as process_module
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute.process import (
|
||||||
|
PidRecord,
|
||||||
|
ProcessLaunchError,
|
||||||
|
UpError,
|
||||||
|
allocate_free_port,
|
||||||
|
clear_pid_record,
|
||||||
|
is_running,
|
||||||
|
launch_proxy,
|
||||||
|
poll_liveliness,
|
||||||
|
read_pid_record,
|
||||||
|
write_pid_record,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeProcess:
|
||||||
|
def __init__(self, returncode: Optional[int] = None):
|
||||||
|
self.returncode = returncode
|
||||||
|
|
||||||
|
def poll(self) -> Optional[int]:
|
||||||
|
return self.returncode
|
||||||
|
|
||||||
|
|
||||||
|
class FakeResponse:
|
||||||
|
def __init__(self, status_code: int):
|
||||||
|
self.status_code = status_code
|
||||||
|
|
||||||
|
|
||||||
|
def test_allocate_free_port_returns_a_bindable_port():
|
||||||
|
port = allocate_free_port()
|
||||||
|
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||||
|
sock.bind(("127.0.0.1", port))
|
||||||
|
|
||||||
|
|
||||||
|
class TestLaunchProxy:
|
||||||
|
def test_binds_loopback_only_not_all_interfaces(self, tmp_path):
|
||||||
|
"""proxy_cli.py's own --host default is 0.0.0.0 -- without an explicit override here, the
|
||||||
|
ephemeral proxy would be reachable from other hosts on the network despite base_url always
|
||||||
|
being built from 127.0.0.1, exposing its unauthenticated-until-master-key-lands routes."""
|
||||||
|
config_path = tmp_path / "config.yaml"
|
||||||
|
log_path = tmp_path / "proxy.log"
|
||||||
|
|
||||||
|
with patch.object(process_module.subprocess, "Popen") as mock_popen:
|
||||||
|
launch_proxy(config_path, 12345, log_path)
|
||||||
|
|
||||||
|
args = mock_popen.call_args[0][0]
|
||||||
|
assert "--host" in args
|
||||||
|
assert args[args.index("--host") + 1] == "127.0.0.1"
|
||||||
|
|
||||||
|
|
||||||
|
class TestPidRecordRoundTrip:
|
||||||
|
def test_write_then_read_round_trips(self, tmp_path):
|
||||||
|
path = tmp_path / "pid.json"
|
||||||
|
record = PidRecord(pid=123, port=4000, config_path="/tmp/config.yaml", log_path="/tmp/proxy.log")
|
||||||
|
|
||||||
|
write_pid_record(record, path)
|
||||||
|
|
||||||
|
assert read_pid_record(path) == record
|
||||||
|
|
||||||
|
def test_read_missing_file_returns_none(self, tmp_path):
|
||||||
|
assert read_pid_record(tmp_path / "missing.json") is None
|
||||||
|
|
||||||
|
def test_read_raises_clean_error_on_corrupt_content(self, tmp_path):
|
||||||
|
path = tmp_path / "pid.json"
|
||||||
|
path.write_text("not json at all {{{")
|
||||||
|
|
||||||
|
with pytest.raises(UpError, match="invalid or unexpected JSON"):
|
||||||
|
read_pid_record(path)
|
||||||
|
|
||||||
|
def test_clear_removes_an_existing_record(self, tmp_path):
|
||||||
|
path = tmp_path / "pid.json"
|
||||||
|
write_pid_record(PidRecord(pid=1, port=1, config_path="a", log_path="b"), path)
|
||||||
|
assert path.exists()
|
||||||
|
|
||||||
|
clear_pid_record(path)
|
||||||
|
|
||||||
|
assert not path.exists()
|
||||||
|
|
||||||
|
def test_clear_missing_file_is_a_no_op(self, tmp_path):
|
||||||
|
clear_pid_record(tmp_path / "missing.json")
|
||||||
|
|
||||||
|
def test_write_creates_parent_directories(self, tmp_path):
|
||||||
|
path = tmp_path / "nested" / "dir" / "pid.json"
|
||||||
|
|
||||||
|
write_pid_record(PidRecord(pid=1, port=1, config_path="a", log_path="b"), path)
|
||||||
|
|
||||||
|
assert path.exists()
|
||||||
|
|
||||||
|
|
||||||
|
class TestIsRunning:
|
||||||
|
def test_current_process_is_running(self):
|
||||||
|
assert is_running(os.getpid()) is True
|
||||||
|
|
||||||
|
def test_huge_unlikely_pid_is_not_running(self):
|
||||||
|
assert is_running(2**30) is False
|
||||||
|
|
||||||
|
def test_permission_error_from_kill_is_treated_as_running(self, monkeypatch):
|
||||||
|
def fake_kill(pid: int, sig: int) -> None:
|
||||||
|
raise PermissionError("not permitted to signal this pid")
|
||||||
|
|
||||||
|
monkeypatch.setattr(process_module.os, "kill", fake_kill)
|
||||||
|
|
||||||
|
assert is_running(999) is True
|
||||||
|
|
||||||
|
|
||||||
|
class TestPollLiveliness:
|
||||||
|
def test_succeeds_when_health_check_returns_200_quickly(self, monkeypatch, tmp_path):
|
||||||
|
monkeypatch.setattr(process_module.requests, "get", lambda url, timeout: FakeResponse(200))
|
||||||
|
|
||||||
|
poll_liveliness("http://127.0.0.1:4000", tmp_path / "proxy.log", FakeProcess(), timeout=5.0)
|
||||||
|
|
||||||
|
def test_raises_with_log_tail_when_timeout_elapses(self, monkeypatch, tmp_path):
|
||||||
|
log_path = tmp_path / "proxy.log"
|
||||||
|
log_path.write_text("line one\nline two\nline three\n")
|
||||||
|
monkeypatch.setattr(process_module.requests, "get", lambda url, timeout: FakeResponse(500))
|
||||||
|
monkeypatch.setattr(process_module.time, "sleep", lambda seconds: None)
|
||||||
|
|
||||||
|
with pytest.raises(ProcessLaunchError) as exc_info:
|
||||||
|
poll_liveliness("http://127.0.0.1:4000", log_path, FakeProcess(), timeout=0.05)
|
||||||
|
|
||||||
|
assert "never became healthy" in str(exc_info.value)
|
||||||
|
assert "line three" in str(exc_info.value)
|
||||||
|
|
||||||
|
def test_raises_immediately_when_process_already_exited(self, tmp_path):
|
||||||
|
log_path = tmp_path / "proxy.log"
|
||||||
|
log_path.write_text("crash log line")
|
||||||
|
|
||||||
|
with pytest.raises(ProcessLaunchError) as exc_info:
|
||||||
|
poll_liveliness("http://127.0.0.1:4000", log_path, FakeProcess(returncode=1), timeout=5.0)
|
||||||
|
|
||||||
|
assert "exited early" in str(exc_info.value)
|
||||||
|
assert "crash log line" in str(exc_info.value)
|
||||||
|
|
@ -0,0 +1,56 @@
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute.settings import (
|
||||||
|
ANTHROPIC_DEFAULT_MODEL_ENV_KEYS,
|
||||||
|
merge_claude_settings_static_token,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_preserves_unrelated_top_level_keys():
|
||||||
|
merged = merge_claude_settings_static_token({"theme": "dark"}, "http://127.0.0.1:4000", "token-abc")
|
||||||
|
assert merged["theme"] == "dark"
|
||||||
|
|
||||||
|
|
||||||
|
def test_preserves_unrelated_env_keys():
|
||||||
|
settings = {"env": {"SOME_OTHER_VAR": "value"}}
|
||||||
|
merged = merge_claude_settings_static_token(settings, "http://127.0.0.1:4000", "token-abc")
|
||||||
|
assert merged["env"]["SOME_OTHER_VAR"] == "value"
|
||||||
|
|
||||||
|
|
||||||
|
def test_sets_base_url_and_auth_token():
|
||||||
|
merged = merge_claude_settings_static_token({}, "http://127.0.0.1:4000/", "token-abc")
|
||||||
|
assert merged["env"]["ANTHROPIC_BASE_URL"] == "http://127.0.0.1:4000"
|
||||||
|
assert merged["env"]["ANTHROPIC_AUTH_TOKEN"] == "token-abc"
|
||||||
|
|
||||||
|
|
||||||
|
def test_drops_stray_api_key():
|
||||||
|
settings = {"env": {"ANTHROPIC_API_KEY": "leaked-key"}}
|
||||||
|
merged = merge_claude_settings_static_token(settings, "http://127.0.0.1:4000", "token-abc")
|
||||||
|
assert "ANTHROPIC_API_KEY" not in merged["env"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_removes_existing_api_key_helper():
|
||||||
|
settings = {"apiKeyHelper": "/usr/local/bin/lite auth print-token"}
|
||||||
|
merged = merge_claude_settings_static_token(settings, "http://127.0.0.1:4000", "token-abc")
|
||||||
|
assert "apiKeyHelper" not in merged
|
||||||
|
|
||||||
|
|
||||||
|
def test_does_not_mutate_input():
|
||||||
|
settings = {"env": {"FOO": "bar"}, "apiKeyHelper": "old-helper"}
|
||||||
|
merge_claude_settings_static_token(settings, "http://127.0.0.1:4000", "token-abc")
|
||||||
|
assert settings == {"env": {"FOO": "bar"}, "apiKeyHelper": "old-helper"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_forces_all_claude_code_default_model_tiers_to_the_autorouter():
|
||||||
|
# A bare "*" model_name deployment looks like the obvious way to catch every request
|
||||||
|
# regardless of which model Claude Code thinks it's using, but Router's auto-router
|
||||||
|
# registry is keyed by the literal requested model string with no wildcard resolution
|
||||||
|
# (litellm/router.py:10711-10717) -- so the only reliable way to make every one of Claude
|
||||||
|
# Code's own tiers hit the auto-router is to override the env vars it reads per tier.
|
||||||
|
merged = merge_claude_settings_static_token({}, "http://127.0.0.1:4000", "token-abc")
|
||||||
|
for key in ANTHROPIC_DEFAULT_MODEL_ENV_KEYS:
|
||||||
|
assert merged["env"][key] == "autorouter"
|
||||||
|
|
||||||
|
|
||||||
|
def test_overrides_a_preexisting_default_model_env_var():
|
||||||
|
settings = {"env": {"ANTHROPIC_DEFAULT_SONNET_MODEL": "claude-opus-4-8"}}
|
||||||
|
merged = merge_claude_settings_static_token(settings, "http://127.0.0.1:4000", "token-abc")
|
||||||
|
assert merged["env"]["ANTHROPIC_DEFAULT_SONNET_MODEL"] == "autorouter"
|
||||||
293
tests/test_litellm/proxy/client/cli/autoroute/test_wizard.py
Normal file
293
tests/test_litellm/proxy/client/cli/autoroute/test_wizard.py
Normal file
|
|
@ -0,0 +1,293 @@
|
||||||
|
import asyncio
|
||||||
|
from typing import Any, Dict, List, Tuple
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import click
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
from click.testing import CliRunner
|
||||||
|
from InquirerPy.base.control import Choice
|
||||||
|
from prompt_toolkit.application import create_app_session
|
||||||
|
from prompt_toolkit.input import create_pipe_input
|
||||||
|
from prompt_toolkit.output import DummyOutput
|
||||||
|
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute import wizard as wizard_module
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute.config import DiscoveredModel
|
||||||
|
from litellm.proxy.client.cli.commands.autoroute.wizard import run_configure_wizard
|
||||||
|
|
||||||
|
CHAT_AND_EMBEDDING_GROUPS: List[Dict[str, Any]] = [
|
||||||
|
{"model_group": "gpt-4o-mini", "mode": "chat", "input_cost_per_token": 0.01, "output_cost_per_token": 0.02},
|
||||||
|
{"model_group": "gpt-4o", "mode": "chat", "input_cost_per_token": 0.01, "output_cost_per_token": 0.02},
|
||||||
|
{"model_group": "claude-opus", "mode": "chat"},
|
||||||
|
{"model_group": "o1", "mode": "chat"},
|
||||||
|
{"model_group": "text-embedding-3-small", "mode": "embedding"},
|
||||||
|
]
|
||||||
|
|
||||||
|
CHAT_ONLY_GROUPS: List[Dict[str, Any]] = [
|
||||||
|
{"model_group": "gpt-4o-mini", "mode": "chat"},
|
||||||
|
{"model_group": "gpt-4o", "mode": "chat"},
|
||||||
|
{"model_group": "claude-opus", "mode": "chat"},
|
||||||
|
{"model_group": "o1", "mode": "chat"},
|
||||||
|
]
|
||||||
|
|
||||||
|
EMBEDDING_ONLY_GROUPS: List[Dict[str, Any]] = [
|
||||||
|
{"model_group": "text-embedding-3-small", "mode": "embedding"},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@click.command()
|
||||||
|
@click.pass_context
|
||||||
|
def _invoke_wizard(ctx: click.Context) -> None:
|
||||||
|
run_configure_wizard(ctx)
|
||||||
|
|
||||||
|
|
||||||
|
def _run(
|
||||||
|
tmp_path,
|
||||||
|
raw_groups: List[Dict[str, Any]],
|
||||||
|
tier_picks: Dict[str, Tuple[str, ...]],
|
||||||
|
input_str: str,
|
||||||
|
classifier_pick: str = "",
|
||||||
|
embedding_pick: str = "",
|
||||||
|
):
|
||||||
|
"""Drives run_configure_wizard's orchestration logic (discovery, validation, config writing,
|
||||||
|
classifier/semantic/adaptive branching) by mocking the fuzzy picker itself, since that widget
|
||||||
|
is a real prompt_toolkit application tested separately in TestFuzzyPickWidget. CliRunner's
|
||||||
|
injected input still drives the plain click.confirm() y/n prompts."""
|
||||||
|
config_path = tmp_path / "config.yaml"
|
||||||
|
runner = CliRunner()
|
||||||
|
|
||||||
|
def _fake_prompt_for_models(models, prompt_label):
|
||||||
|
return tier_picks[prompt_label]
|
||||||
|
|
||||||
|
def _fake_prompt_for_model(models, prompt_label):
|
||||||
|
if prompt_label == "LLM classifier":
|
||||||
|
return classifier_pick
|
||||||
|
if prompt_label == "semantic embeddings":
|
||||||
|
return embedding_pick
|
||||||
|
raise AssertionError(f"unexpected single-pick prompt_label {prompt_label!r}")
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(wizard_module, "Client") as mock_client_cls,
|
||||||
|
patch.object(wizard_module, "CONFIG_PATH", config_path),
|
||||||
|
patch.object(wizard_module, "_is_interactive", return_value=True),
|
||||||
|
patch.object(wizard_module, "_render_and_prompt_for_models", side_effect=_fake_prompt_for_models),
|
||||||
|
patch.object(wizard_module, "_render_and_prompt_for_model", side_effect=_fake_prompt_for_model),
|
||||||
|
):
|
||||||
|
mock_client_cls.return_value.model_groups.info.return_value = raw_groups
|
||||||
|
result = runner.invoke(
|
||||||
|
_invoke_wizard,
|
||||||
|
obj={"base_url": "http://localhost:4000", "api_key": "sk-test"},
|
||||||
|
input=input_str,
|
||||||
|
)
|
||||||
|
return result, config_path
|
||||||
|
|
||||||
|
|
||||||
|
def _router_config(config_path) -> Dict[str, Any]:
|
||||||
|
written = yaml.safe_load(config_path.read_text())
|
||||||
|
autorouter = next(m for m in written["model_list"] if m["model_name"] == "autorouter")
|
||||||
|
return autorouter["litellm_params"]["complexity_router_config"]
|
||||||
|
|
||||||
|
|
||||||
|
_SIMPLE_TIER_PICKS: Dict[str, Tuple[str, ...]] = {
|
||||||
|
"SIMPLE": ("gpt-4o-mini",),
|
||||||
|
"MEDIUM": ("gpt-4o",),
|
||||||
|
"COMPLEX": ("claude-opus",),
|
||||||
|
"REASONING": ("o1",),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class TestRunConfigureWizardHappyPath:
|
||||||
|
def test_assigns_tiers_and_declines_everything(self, tmp_path):
|
||||||
|
result, config_path = _run(tmp_path, CHAT_AND_EMBEDDING_GROUPS, _SIMPLE_TIER_PICKS, input_str="n\nn\nn\n")
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
router_config = _router_config(config_path)
|
||||||
|
assert router_config["tiers"] == {
|
||||||
|
"SIMPLE": ["gpt-4o-mini"],
|
||||||
|
"MEDIUM": ["gpt-4o"],
|
||||||
|
"COMPLEX": ["claude-opus"],
|
||||||
|
"REASONING": ["o1"],
|
||||||
|
}
|
||||||
|
assert router_config["default_model"] == "gpt-4o"
|
||||||
|
assert "classifier_type" not in router_config
|
||||||
|
assert "classifier_llm_config" not in router_config
|
||||||
|
assert "semantic_keyword_matching" not in router_config
|
||||||
|
assert "adaptive" not in router_config
|
||||||
|
|
||||||
|
def test_assigns_multiple_models_to_a_single_tier(self, tmp_path):
|
||||||
|
tier_picks = {**_SIMPLE_TIER_PICKS, "SIMPLE": ("gpt-4o-mini", "gpt-4o")}
|
||||||
|
result, config_path = _run(tmp_path, CHAT_AND_EMBEDDING_GROUPS, tier_picks, input_str="n\nn\nn\n")
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
router_config = _router_config(config_path)
|
||||||
|
assert router_config["tiers"]["SIMPLE"] == ["gpt-4o-mini", "gpt-4o"]
|
||||||
|
assert router_config["default_model"] == "gpt-4o"
|
||||||
|
|
||||||
|
def test_writes_config_file_with_restricted_permissions(self, tmp_path):
|
||||||
|
result, config_path = _run(tmp_path, CHAT_AND_EMBEDDING_GROUPS, _SIMPLE_TIER_PICKS, input_str="n\nn\nn\n")
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert config_path.exists()
|
||||||
|
assert oct(config_path.stat().st_mode)[-3:] == "600"
|
||||||
|
|
||||||
|
def test_no_embedding_pool_skips_semantic_prompt_entirely(self, tmp_path):
|
||||||
|
result, config_path = _run(tmp_path, CHAT_ONLY_GROUPS, _SIMPLE_TIER_PICKS, input_str="n\nn\n")
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
router_config = _router_config(config_path)
|
||||||
|
assert "semantic_keyword_matching" not in router_config
|
||||||
|
|
||||||
|
|
||||||
|
class TestRunConfigureWizardLLMClassifier:
|
||||||
|
def test_accepting_llm_classifier_records_chosen_model(self, tmp_path):
|
||||||
|
result, config_path = _run(
|
||||||
|
tmp_path, CHAT_AND_EMBEDDING_GROUPS, _SIMPLE_TIER_PICKS, input_str="y\nn\nn\n", classifier_pick="gpt-4o"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
router_config = _router_config(config_path)
|
||||||
|
assert router_config["classifier_type"] == "llm"
|
||||||
|
assert router_config["classifier_llm_config"]["model"] == "gpt-4o"
|
||||||
|
|
||||||
|
|
||||||
|
class TestRunConfigureWizardSemanticMatching:
|
||||||
|
def test_accepting_semantic_matching_records_embedding_model(self, tmp_path):
|
||||||
|
result, config_path = _run(
|
||||||
|
tmp_path,
|
||||||
|
CHAT_AND_EMBEDDING_GROUPS,
|
||||||
|
_SIMPLE_TIER_PICKS,
|
||||||
|
input_str="n\ny\nn\n",
|
||||||
|
embedding_pick="text-embedding-3-small",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
router_config = _router_config(config_path)
|
||||||
|
assert router_config["semantic_keyword_matching"] is True
|
||||||
|
assert router_config["embedding_model"] == "text-embedding-3-small"
|
||||||
|
|
||||||
|
|
||||||
|
class TestRunConfigureWizardAdaptive:
|
||||||
|
def test_accepting_adaptive_sets_adaptive_flag(self, tmp_path):
|
||||||
|
result, config_path = _run(tmp_path, CHAT_AND_EMBEDDING_GROUPS, _SIMPLE_TIER_PICKS, input_str="n\nn\ny\n")
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
router_config = _router_config(config_path)
|
||||||
|
assert router_config["adaptive"] is True
|
||||||
|
|
||||||
|
|
||||||
|
class TestRunConfigureWizardNoChatModels:
|
||||||
|
def test_fails_cleanly_without_prompting_when_no_chat_models(self, tmp_path):
|
||||||
|
result, config_path = _run(tmp_path, EMBEDDING_ONLY_GROUPS, {}, input_str="")
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "no chat-capable models" in result.output.lower()
|
||||||
|
assert not config_path.exists()
|
||||||
|
|
||||||
|
def test_surfaces_clean_error_when_response_is_not_a_list(self, tmp_path):
|
||||||
|
result, config_path = _run(tmp_path, {"data": CHAT_AND_EMBEDDING_GROUPS}, {}, input_str="")
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert result.exception is None or not isinstance(result.exception, AssertionError)
|
||||||
|
assert "Unexpected response from /model_group/info" in result.output
|
||||||
|
assert not config_path.exists()
|
||||||
|
|
||||||
|
|
||||||
|
class TestRunConfigureWizardNotInteractive:
|
||||||
|
def test_fails_cleanly_when_not_a_tty(self, tmp_path):
|
||||||
|
config_path = tmp_path / "config.yaml"
|
||||||
|
runner = CliRunner()
|
||||||
|
with (
|
||||||
|
patch.object(wizard_module, "Client") as mock_client_cls,
|
||||||
|
patch.object(wizard_module, "CONFIG_PATH", config_path),
|
||||||
|
patch.object(wizard_module, "_is_interactive", return_value=False),
|
||||||
|
):
|
||||||
|
mock_client_cls.return_value.model_groups.info.return_value = CHAT_AND_EMBEDDING_GROUPS
|
||||||
|
result = runner.invoke(_invoke_wizard, obj={"base_url": "http://localhost:4000", "api_key": "sk-test"})
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "interactive terminal" in result.output
|
||||||
|
assert not config_path.exists()
|
||||||
|
|
||||||
|
|
||||||
|
def _drive_fuzzy_pick(
|
||||||
|
models: Tuple[DiscoveredModel, ...],
|
||||||
|
prompt_label: str,
|
||||||
|
multiselect: bool,
|
||||||
|
key_events: List[Tuple[str, float]],
|
||||||
|
) -> List[str]:
|
||||||
|
"""Drives the real InquirerPy fuzzy prompt through prompt_toolkit's own test input/output,
|
||||||
|
exercising the actual widget (filtering, tab-to-toggle, enter-to-confirm) rather than mocking
|
||||||
|
it away. asyncio.to_thread propagates the create_app_session context into the worker thread
|
||||||
|
running _fuzzy_pick's synchronous .execute() call."""
|
||||||
|
|
||||||
|
async def _run() -> List[str]:
|
||||||
|
with create_pipe_input() as pipe_input:
|
||||||
|
with create_app_session(input=pipe_input, output=DummyOutput()):
|
||||||
|
task = asyncio.ensure_future(
|
||||||
|
asyncio.to_thread(wizard_module._fuzzy_pick, models, prompt_label, multiselect)
|
||||||
|
)
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
for text, delay in key_events:
|
||||||
|
pipe_input.send_text(text)
|
||||||
|
await asyncio.sleep(delay)
|
||||||
|
return await task
|
||||||
|
|
||||||
|
return asyncio.run(_run())
|
||||||
|
|
||||||
|
|
||||||
|
class TestFuzzyPickWidget:
|
||||||
|
def _models(self) -> Tuple[DiscoveredModel, ...]:
|
||||||
|
return tuple(DiscoveredModel(name=f"model-{i}") for i in range(20))
|
||||||
|
|
||||||
|
def test_single_select_filters_and_returns_highlighted_match(self):
|
||||||
|
result = _drive_fuzzy_pick(
|
||||||
|
self._models(), "test", multiselect=False, key_events=[("model-13", 0.3), ("\r", 0.1)]
|
||||||
|
)
|
||||||
|
assert result == ["model-13"]
|
||||||
|
|
||||||
|
def test_multiselect_requires_tab_to_toggle_before_enter(self):
|
||||||
|
result = _drive_fuzzy_pick(
|
||||||
|
self._models(), "test", multiselect=True, key_events=[("model-7", 0.3), ("\t", 0.1), ("\r", 0.1)]
|
||||||
|
)
|
||||||
|
assert result == ["model-7"]
|
||||||
|
|
||||||
|
def test_multiselect_can_pick_more_than_one_across_filters(self):
|
||||||
|
result = _drive_fuzzy_pick(
|
||||||
|
self._models(),
|
||||||
|
"test",
|
||||||
|
multiselect=True,
|
||||||
|
key_events=[
|
||||||
|
("model-3", 0.3),
|
||||||
|
("\t", 0.1),
|
||||||
|
*[("\x7f", 0.02) for _ in range("model-3".__len__())],
|
||||||
|
("model-15", 0.3),
|
||||||
|
("\t", 0.1),
|
||||||
|
("\r", 0.1),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
assert set(result) == {"model-3", "model-15"}
|
||||||
|
|
||||||
|
def test_choice_wraps_name_and_value_to_the_same_model_name(self):
|
||||||
|
model = DiscoveredModel(name="only-model")
|
||||||
|
choice = Choice(value=model.name, name=model.name)
|
||||||
|
assert choice.value == choice.name == "only-model"
|
||||||
|
|
||||||
|
|
||||||
|
class TestRenderAndPromptForModelWrappers:
|
||||||
|
def test_single_pick_wrapper_returns_bare_string(self):
|
||||||
|
with patch.object(wizard_module, "_fuzzy_pick", return_value=["model-a"]) as mock_pick:
|
||||||
|
result = wizard_module._render_and_prompt_for_model((), "tier")
|
||||||
|
assert result == "model-a"
|
||||||
|
mock_pick.assert_called_once_with((), "tier", multiselect=False)
|
||||||
|
|
||||||
|
def test_multi_pick_wrapper_returns_tuple(self):
|
||||||
|
with patch.object(wizard_module, "_fuzzy_pick", return_value=["model-a", "model-b"]) as mock_pick:
|
||||||
|
result = wizard_module._render_and_prompt_for_models((), "tier")
|
||||||
|
assert result == ("model-a", "model-b")
|
||||||
|
mock_pick.assert_called_once_with((), "tier", multiselect=True)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("isatty_value", [True, False])
|
||||||
|
def test_is_interactive_reflects_stdin_isatty(isatty_value):
|
||||||
|
with patch.object(wizard_module.sys.stdin, "isatty", return_value=isatty_value):
|
||||||
|
assert wizard_module._is_interactive() is isatty_value
|
||||||
|
|
@ -797,14 +797,18 @@ class TestPrintTokenCommand:
|
||||||
verbatim as the bearer token, so any diagnostic text on stdout would
|
verbatim as the bearer token, so any diagnostic text on stdout would
|
||||||
corrupt authentication.
|
corrupt authentication.
|
||||||
|
|
||||||
apiKeyHelper is configured as a bare command (managed-settings.json sets
|
`lite up` now writes `apiKeyHelper` with an explicit `--base-url` bound
|
||||||
just `"apiKeyHelper": "lite auth print-token"`, no --base-url flag) --
|
to whatever proxy it was pointed at (resolve_api_key_helper), so
|
||||||
so in the common case ctx.obj has no explicit base_url at all, and the
|
print-token enforces that the cached token was actually issued for that
|
||||||
command must resolve the server from whatever `lite login` stored in
|
server -- a token minted for a different, previously-logged-into proxy
|
||||||
token.json, not from a CLI default. `--base-url`/`LITELLM_PROXY_URL`
|
must never be handed to whichever server the helper is invoked for.
|
||||||
only matters when a caller explicitly overrides it (tracked via
|
Settings patched by an older `lite up`, or a manually-configured
|
||||||
ctx.obj["base_url_explicit"], set by the `cli` group from
|
apiKeyHelper, can still invoke this bare (no --base-url at all); that
|
||||||
click's ParameterSource).
|
case falls back to trusting whatever `lite login` stored in token.json,
|
||||||
|
since there is no explicit target to check it against. `--base-url`/
|
||||||
|
`LITELLM_PROXY_URL` only enforces the match when a caller explicitly
|
||||||
|
passes it (tracked via ctx.obj["base_url_explicit"], set by the `cli`
|
||||||
|
group from click's ParameterSource).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def setup_method(self):
|
def setup_method(self):
|
||||||
|
|
@ -818,8 +822,9 @@ class TestPrintTokenCommand:
|
||||||
assert "Not authenticated" in result.output
|
assert "Not authenticated" in result.output
|
||||||
|
|
||||||
def test_bare_invocation_resolves_server_from_stored_token(self):
|
def test_bare_invocation_resolves_server_from_stored_token(self):
|
||||||
"""The apiKeyHelper's real invocation shape: no --base-url given at
|
"""The legacy/manual invocation shape: no --base-url given at all
|
||||||
all. Must use token.json's own base_url, not a hardcoded default."""
|
(e.g. settings patched before resolve_api_key_helper started binding
|
||||||
|
one). Must use token.json's own base_url, not a hardcoded default."""
|
||||||
with (
|
with (
|
||||||
patch(
|
patch(
|
||||||
"litellm.proxy.client.cli.commands.auth.load_token",
|
"litellm.proxy.client.cli.commands.auth.load_token",
|
||||||
|
|
@ -839,7 +844,10 @@ class TestPrintTokenCommand:
|
||||||
|
|
||||||
def test_explicit_base_url_mismatch_fails_cleanly(self):
|
def test_explicit_base_url_mismatch_fails_cleanly(self):
|
||||||
"""When the caller *does* explicitly pass --base-url, a token issued
|
"""When the caller *does* explicitly pass --base-url, a token issued
|
||||||
for a different server must never be printed."""
|
for a different server must never be printed. This is the exact
|
||||||
|
scenario `lite up`'s own bound --base-url now guards against: a
|
||||||
|
token minted for proxy A must not reach a helper invocation aimed
|
||||||
|
at proxy B, even though the token itself is otherwise fresh."""
|
||||||
with patch(
|
with patch(
|
||||||
"litellm.proxy.client.cli.commands.auth.load_token",
|
"litellm.proxy.client.cli.commands.auth.load_token",
|
||||||
return_value={
|
return_value={
|
||||||
|
|
@ -856,6 +864,25 @@ class TestPrintTokenCommand:
|
||||||
assert result.exit_code != 0
|
assert result.exit_code != 0
|
||||||
assert "sk-should-not-print" not in result.output
|
assert "sk-should-not-print" not in result.output
|
||||||
|
|
||||||
|
def test_explicit_base_url_match_prints_token(self):
|
||||||
|
"""`lite up`'s own bound invocation shape: --base-url matching the token's origin
|
||||||
|
must succeed exactly like the bare/legacy invocation does."""
|
||||||
|
with patch(
|
||||||
|
"litellm.proxy.client.cli.commands.auth.load_token",
|
||||||
|
return_value={
|
||||||
|
"base_url": "http://localhost:4000",
|
||||||
|
"key": "sk-matches",
|
||||||
|
"timestamp": time.time(),
|
||||||
|
},
|
||||||
|
):
|
||||||
|
result = self.runner.invoke(
|
||||||
|
print_token,
|
||||||
|
obj={"base_url": "http://localhost:4000", "base_url_explicit": True},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.exit_code == 0
|
||||||
|
assert result.output.strip() == "sk-matches"
|
||||||
|
|
||||||
def test_fresh_cached_key_printed_without_network_call(self):
|
def test_fresh_cached_key_printed_without_network_call(self):
|
||||||
"""A recently-issued key should be printed straight from cache -- no
|
"""A recently-issued key should be printed straight from cache -- no
|
||||||
refresh call on every single invocation (apiKeyHelper gets called
|
refresh call on every single invocation (apiKeyHelper gets called
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,114 @@
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from click.testing import CliRunner
|
||||||
|
|
||||||
|
from litellm.proxy.client.cli import cli
|
||||||
|
|
||||||
|
SAMPLE_MODEL_GROUPS: List[Dict[str, Any]] = [
|
||||||
|
{
|
||||||
|
"model_group": "gpt-4o",
|
||||||
|
"mode": "chat",
|
||||||
|
"input_cost_per_token": 0.01,
|
||||||
|
"output_cost_per_token": 0.02,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"model_group": "text-embedding-3-small",
|
||||||
|
"mode": "embedding",
|
||||||
|
"input_cost_per_token": 0.0001,
|
||||||
|
"output_cost_per_token": None,
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_client():
|
||||||
|
with patch("litellm.proxy.client.cli.commands.model_groups.Client") as MockClient:
|
||||||
|
yield MockClient
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def cli_runner():
|
||||||
|
return CliRunner()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def mock_env():
|
||||||
|
with patch.dict(
|
||||||
|
os.environ,
|
||||||
|
{
|
||||||
|
"LITELLM_PROXY_URL": "http://localhost:4000",
|
||||||
|
"LITELLM_PROXY_API_KEY": "sk-test",
|
||||||
|
},
|
||||||
|
):
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_table_format_shows_model_names_and_modes(mock_client, cli_runner):
|
||||||
|
mock_client.return_value.model_groups.info.return_value = SAMPLE_MODEL_GROUPS
|
||||||
|
|
||||||
|
result = cli_runner.invoke(cli, ["model-groups", "list"])
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "gpt-4o" in result.output
|
||||||
|
assert "chat" in result.output
|
||||||
|
assert "text-embedding-3-small" in result.output
|
||||||
|
assert "embedding" in result.output
|
||||||
|
assert "0.01" in result.output
|
||||||
|
assert "0.02" in result.output
|
||||||
|
|
||||||
|
mock_client.assert_called_once_with(base_url="http://localhost:4000", api_key="sk-test")
|
||||||
|
mock_client.return_value.model_groups.info.assert_called_once()
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_table_format_defaults_missing_mode_to_chat(mock_client, cli_runner):
|
||||||
|
mock_client.return_value.model_groups.info.return_value = [{"model_group": "some-model"}]
|
||||||
|
|
||||||
|
result = cli_runner.invoke(cli, ["model-groups", "list"])
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "some-model" in result.output
|
||||||
|
assert "chat" in result.output
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_json_format_round_trips_raw_data(mock_client, cli_runner):
|
||||||
|
mock_client.return_value.model_groups.info.return_value = SAMPLE_MODEL_GROUPS
|
||||||
|
|
||||||
|
result = cli_runner.invoke(cli, ["model-groups", "list", "--format", "json"])
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert json.loads(result.output) == SAMPLE_MODEL_GROUPS
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_with_custom_base_url_and_api_key(mock_client, cli_runner):
|
||||||
|
mock_client.return_value.model_groups.info.return_value = []
|
||||||
|
|
||||||
|
result = cli_runner.invoke(
|
||||||
|
cli,
|
||||||
|
["--base-url", "http://custom.server:8000", "--api-key", "custom-key", "model-groups", "list"],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
mock_client.assert_called_once_with(base_url="http://custom.server:8000", api_key="custom-key")
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_error_handling(mock_client, cli_runner):
|
||||||
|
mock_client.return_value.model_groups.info.side_effect = Exception("API Error")
|
||||||
|
|
||||||
|
result = cli_runner.invoke(cli, ["model-groups", "list"])
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "API Error" in str(result.exception)
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_surfaces_clean_error_when_response_is_not_a_list(mock_client, cli_runner):
|
||||||
|
mock_client.return_value.model_groups.info.return_value = {"data": SAMPLE_MODEL_GROUPS}
|
||||||
|
|
||||||
|
result = cli_runner.invoke(cli, ["model-groups", "list"])
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert result.exception is None or not isinstance(result.exception, AssertionError)
|
||||||
|
assert "Unexpected response from /model_group/info" in result.output
|
||||||
398
tests/test_litellm/proxy/client/cli/test_up_commands.py
Normal file
398
tests/test_litellm/proxy/client/cli/test_up_commands.py
Normal file
|
|
@ -0,0 +1,398 @@
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
import stat
|
||||||
|
import sys
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import click
|
||||||
|
import pytest
|
||||||
|
from click.testing import CliRunner
|
||||||
|
|
||||||
|
from litellm.proxy.client.cli.commands import up as up_module
|
||||||
|
from litellm.proxy.client.cli.commands.agents import AgentRunError
|
||||||
|
from litellm.proxy.client.cli.commands.up import (
|
||||||
|
BackupRecord,
|
||||||
|
UpError,
|
||||||
|
_ensure_fresh_login,
|
||||||
|
down,
|
||||||
|
load_json_or_empty,
|
||||||
|
merge_claude_settings,
|
||||||
|
read_backup,
|
||||||
|
resolve_api_key_helper,
|
||||||
|
restore_claude_settings,
|
||||||
|
up,
|
||||||
|
write_backup,
|
||||||
|
)
|
||||||
|
|
||||||
|
UP_MODULE = "litellm.proxy.client.cli.commands.up"
|
||||||
|
|
||||||
|
|
||||||
|
def _patch_paths(monkeypatch, tmp_path):
|
||||||
|
settings_path = tmp_path / "claude_settings.json"
|
||||||
|
backup_path = tmp_path / "backup.json"
|
||||||
|
monkeypatch.setattr(up_module, "CLAUDE_SETTINGS_PATH", settings_path)
|
||||||
|
monkeypatch.setattr(up_module, "BACKUP_PATH", backup_path)
|
||||||
|
return settings_path, backup_path
|
||||||
|
|
||||||
|
|
||||||
|
class TestMergeClaudeSettings:
|
||||||
|
def test_preserves_unrelated_top_level_keys(self):
|
||||||
|
merged = merge_claude_settings({"theme": "dark"}, "http://localhost:4000", "helper")
|
||||||
|
assert merged["theme"] == "dark"
|
||||||
|
|
||||||
|
def test_preserves_unrelated_env_keys(self):
|
||||||
|
settings = {"env": {"SOME_OTHER_VAR": "value"}}
|
||||||
|
merged = merge_claude_settings(settings, "http://localhost:4000", "helper")
|
||||||
|
assert merged["env"]["SOME_OTHER_VAR"] == "value"
|
||||||
|
|
||||||
|
def test_overrides_base_url_and_helper(self):
|
||||||
|
settings = {
|
||||||
|
"env": {"ANTHROPIC_BASE_URL": "https://old.example.com"},
|
||||||
|
"apiKeyHelper": "old-helper",
|
||||||
|
}
|
||||||
|
merged = merge_claude_settings(settings, "http://localhost:4000/", "new-helper")
|
||||||
|
assert merged["env"]["ANTHROPIC_BASE_URL"] == "http://localhost:4000"
|
||||||
|
assert merged["apiKeyHelper"] == "new-helper"
|
||||||
|
|
||||||
|
def test_drops_stray_api_key(self):
|
||||||
|
settings = {"env": {"ANTHROPIC_API_KEY": "leaked-key"}}
|
||||||
|
merged = merge_claude_settings(settings, "http://localhost:4000", "helper")
|
||||||
|
assert "ANTHROPIC_API_KEY" not in merged["env"]
|
||||||
|
|
||||||
|
def test_works_from_empty_settings(self):
|
||||||
|
merged = merge_claude_settings({}, "http://localhost:4000", "helper")
|
||||||
|
assert merged["env"] == {"ANTHROPIC_BASE_URL": "http://localhost:4000"}
|
||||||
|
assert merged["apiKeyHelper"] == "helper"
|
||||||
|
|
||||||
|
def test_does_not_mutate_input(self):
|
||||||
|
settings = {"env": {"FOO": "bar"}}
|
||||||
|
merge_claude_settings(settings, "http://localhost:4000", "helper")
|
||||||
|
assert settings == {"env": {"FOO": "bar"}}
|
||||||
|
|
||||||
|
|
||||||
|
class TestLoadJsonOrEmpty:
|
||||||
|
def test_returns_empty_dict_when_file_does_not_exist(self, tmp_path):
|
||||||
|
assert load_json_or_empty(tmp_path / "missing.json") == {}
|
||||||
|
|
||||||
|
def test_returns_empty_dict_when_file_is_empty(self, tmp_path):
|
||||||
|
path = tmp_path / "settings.json"
|
||||||
|
path.write_text("")
|
||||||
|
assert load_json_or_empty(path) == {}
|
||||||
|
|
||||||
|
def test_returns_empty_dict_when_file_is_whitespace_only(self, tmp_path):
|
||||||
|
path = tmp_path / "settings.json"
|
||||||
|
path.write_text(" \n")
|
||||||
|
assert load_json_or_empty(path) == {}
|
||||||
|
|
||||||
|
def test_parses_real_content(self, tmp_path):
|
||||||
|
path = tmp_path / "settings.json"
|
||||||
|
path.write_text(json.dumps({"theme": "dark"}))
|
||||||
|
assert load_json_or_empty(path) == {"theme": "dark"}
|
||||||
|
|
||||||
|
def test_raises_clean_error_on_invalid_json(self, tmp_path):
|
||||||
|
path = tmp_path / "settings.json"
|
||||||
|
path.write_text("not json at all {{{")
|
||||||
|
with pytest.raises(UpError, match="invalid JSON"):
|
||||||
|
load_json_or_empty(path)
|
||||||
|
|
||||||
|
def test_raises_clean_error_on_non_object_root(self, tmp_path):
|
||||||
|
path = tmp_path / "settings.json"
|
||||||
|
path.write_text(json.dumps([1, 2, 3]))
|
||||||
|
with pytest.raises(UpError, match="invalid JSON"):
|
||||||
|
load_json_or_empty(path)
|
||||||
|
|
||||||
|
|
||||||
|
class TestBackupRoundTrip:
|
||||||
|
def test_restores_original_content_when_file_existed(self, monkeypatch, tmp_path):
|
||||||
|
settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
original = {"apiKeyHelper": "old-helper", "theme": "dark"}
|
||||||
|
settings_path.write_text(json.dumps(original))
|
||||||
|
|
||||||
|
write_backup(BackupRecord(existed=True, content=original))
|
||||||
|
settings_path.write_text(json.dumps({"apiKeyHelper": "lite-helper"}))
|
||||||
|
|
||||||
|
restored = restore_claude_settings()
|
||||||
|
|
||||||
|
assert restored is not None
|
||||||
|
assert restored.existed is True
|
||||||
|
assert json.loads(settings_path.read_text()) == original
|
||||||
|
assert not backup_path.exists()
|
||||||
|
|
||||||
|
def test_deletes_settings_file_when_it_did_not_exist_before(self, monkeypatch, tmp_path):
|
||||||
|
settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
write_backup(BackupRecord(existed=False, content=None))
|
||||||
|
settings_path.write_text(json.dumps({"apiKeyHelper": "lite-helper"}))
|
||||||
|
|
||||||
|
restored = restore_claude_settings()
|
||||||
|
|
||||||
|
assert restored is not None
|
||||||
|
assert restored.existed is False
|
||||||
|
assert not settings_path.exists()
|
||||||
|
assert not backup_path.exists()
|
||||||
|
|
||||||
|
def test_no_backup_is_a_no_op_returning_none(self, monkeypatch, tmp_path):
|
||||||
|
settings_path, _backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
assert restore_claude_settings() is None
|
||||||
|
assert not settings_path.exists()
|
||||||
|
|
||||||
|
def test_recreates_claude_dir_if_it_was_deleted_while_up_was_running(self, monkeypatch, tmp_path):
|
||||||
|
"""If ~/.claude/ is removed while `lite up` holds it open, restoring must recreate the
|
||||||
|
directory rather than crash with FileNotFoundError and strand the backup file, which
|
||||||
|
would otherwise permanently break every future `lite down`."""
|
||||||
|
claude_dir = tmp_path / "claude_dir"
|
||||||
|
settings_path = claude_dir / "settings.json"
|
||||||
|
backup_path = tmp_path / "backup.json"
|
||||||
|
monkeypatch.setattr(up_module, "CLAUDE_SETTINGS_PATH", settings_path)
|
||||||
|
monkeypatch.setattr(up_module, "BACKUP_PATH", backup_path)
|
||||||
|
original = {"theme": "dark"}
|
||||||
|
claude_dir.mkdir(parents=True)
|
||||||
|
write_backup(BackupRecord(existed=True, content=original))
|
||||||
|
shutil.rmtree(claude_dir)
|
||||||
|
|
||||||
|
restored = restore_claude_settings()
|
||||||
|
|
||||||
|
assert restored is not None
|
||||||
|
assert json.loads(settings_path.read_text()) == original
|
||||||
|
assert not backup_path.exists()
|
||||||
|
|
||||||
|
def test_read_backup_round_trips_write_backup(self, monkeypatch, tmp_path):
|
||||||
|
_patch_paths(monkeypatch, tmp_path)
|
||||||
|
write_backup(BackupRecord(existed=True, content={"a": 1}))
|
||||||
|
assert read_backup() == BackupRecord(existed=True, content={"a": 1})
|
||||||
|
|
||||||
|
def test_read_backup_missing_file_returns_none(self, monkeypatch, tmp_path):
|
||||||
|
_patch_paths(monkeypatch, tmp_path)
|
||||||
|
assert read_backup() is None
|
||||||
|
|
||||||
|
def test_read_backup_raises_clean_error_on_corrupt_content(self, monkeypatch, tmp_path):
|
||||||
|
_settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
backup_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
backup_path.write_text("not json at all {{{")
|
||||||
|
|
||||||
|
with pytest.raises(UpError, match="invalid or unexpected JSON"):
|
||||||
|
read_backup()
|
||||||
|
|
||||||
|
def test_write_backup_restricts_permissions_for_a_new_file(self, monkeypatch, tmp_path):
|
||||||
|
_settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
write_backup(BackupRecord(existed=True, content={"a": 1}))
|
||||||
|
assert stat.S_IMODE(backup_path.stat().st_mode) == 0o600
|
||||||
|
|
||||||
|
def test_write_backup_restricts_permissions_of_a_preexisting_permissive_file(self, monkeypatch, tmp_path):
|
||||||
|
_settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
backup_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
backup_path.write_text("{}")
|
||||||
|
backup_path.chmod(0o644)
|
||||||
|
|
||||||
|
write_backup(BackupRecord(existed=True, content={"a": 1}))
|
||||||
|
|
||||||
|
assert stat.S_IMODE(backup_path.stat().st_mode) == 0o600
|
||||||
|
|
||||||
|
def test_backup_file_always_removed_after_restore(self, monkeypatch, tmp_path):
|
||||||
|
_settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
write_backup(BackupRecord(existed=False, content=None))
|
||||||
|
assert backup_path.exists()
|
||||||
|
|
||||||
|
restore_claude_settings()
|
||||||
|
|
||||||
|
assert not backup_path.exists()
|
||||||
|
|
||||||
|
|
||||||
|
class TestResolveApiKeyHelper:
|
||||||
|
def test_returns_helper_command_bound_to_the_selected_proxy(self, monkeypatch):
|
||||||
|
monkeypatch.setattr(shutil, "which", lambda name: "/usr/local/bin/lite")
|
||||||
|
helper = resolve_api_key_helper("http://localhost:4000")
|
||||||
|
assert helper == "/usr/local/bin/lite auth print-token --base-url http://localhost:4000"
|
||||||
|
|
||||||
|
def test_quotes_a_base_url_containing_shell_metacharacters(self, monkeypatch):
|
||||||
|
monkeypatch.setattr(shutil, "which", lambda name: "/usr/local/bin/lite")
|
||||||
|
helper = resolve_api_key_helper("http://example.com/path; rm -rf /")
|
||||||
|
assert helper == "/usr/local/bin/lite auth print-token --base-url 'http://example.com/path; rm -rf /'"
|
||||||
|
|
||||||
|
def test_raises_when_lite_not_on_path(self, monkeypatch):
|
||||||
|
monkeypatch.setattr(shutil, "which", lambda name: None)
|
||||||
|
with pytest.raises(UpError, match="Could not find `lite`"):
|
||||||
|
resolve_api_key_helper("http://localhost:4000")
|
||||||
|
|
||||||
|
|
||||||
|
def _make_ctx(base_url):
|
||||||
|
return click.Context(click.Command("test"), obj={"base_url": base_url})
|
||||||
|
|
||||||
|
|
||||||
|
class TestEnsureFreshLogin:
|
||||||
|
"""A token that is fresh but was issued for a *different* proxy must not be trusted: without
|
||||||
|
this check, a user logged into proxy A who runs `up --base-url proxy-b` would silently get an
|
||||||
|
apiKeyHelper wired up around proxy A's real token, which print-token would then hand to proxy B."""
|
||||||
|
|
||||||
|
def test_reuses_a_fresh_token_issued_for_the_same_proxy(self, monkeypatch):
|
||||||
|
monkeypatch.setattr(up_module, "load_token", lambda: {"key": "sk-a", "base_url": "http://proxy-a:4000"})
|
||||||
|
monkeypatch.setattr(up_module, "is_cli_token_fresh", lambda token_data: True)
|
||||||
|
login_calls = []
|
||||||
|
monkeypatch.setattr(up_module, "login", lambda ctx: login_calls.append(ctx))
|
||||||
|
|
||||||
|
_ensure_fresh_login(_make_ctx("http://proxy-a:4000"))
|
||||||
|
|
||||||
|
assert login_calls == []
|
||||||
|
|
||||||
|
def test_forces_a_fresh_login_when_the_cached_token_is_for_a_different_proxy(self, monkeypatch):
|
||||||
|
monkeypatch.setattr(up_module.sys.stdin, "isatty", lambda: True)
|
||||||
|
tokens = iter(
|
||||||
|
[
|
||||||
|
{"key": "sk-a", "base_url": "http://proxy-a:4000"},
|
||||||
|
{"key": "sk-b", "base_url": "http://proxy-b:4000"},
|
||||||
|
]
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(up_module, "load_token", lambda: next(tokens))
|
||||||
|
monkeypatch.setattr(up_module, "is_cli_token_fresh", lambda token_data: True)
|
||||||
|
login_calls = []
|
||||||
|
|
||||||
|
@click.pass_context
|
||||||
|
def fake_login(ctx):
|
||||||
|
login_calls.append(ctx.obj["base_url"])
|
||||||
|
|
||||||
|
monkeypatch.setattr(up_module, "login", fake_login)
|
||||||
|
|
||||||
|
_ensure_fresh_login(_make_ctx("http://proxy-b:4000"))
|
||||||
|
|
||||||
|
assert login_calls == ["http://proxy-b:4000"]
|
||||||
|
|
||||||
|
def test_fails_cleanly_non_interactively_when_only_a_different_proxys_token_is_cached(self, monkeypatch):
|
||||||
|
monkeypatch.setattr(up_module.sys.stdin, "isatty", lambda: False)
|
||||||
|
monkeypatch.setattr(up_module, "load_token", lambda: {"key": "sk-a", "base_url": "http://proxy-a:4000"})
|
||||||
|
monkeypatch.setattr(up_module, "is_cli_token_fresh", lambda token_data: True)
|
||||||
|
|
||||||
|
with pytest.raises(UpError, match="lite login"):
|
||||||
|
_ensure_fresh_login(_make_ctx("http://proxy-b:4000"))
|
||||||
|
|
||||||
|
|
||||||
|
class TestUpCommand:
|
||||||
|
def setup_method(self):
|
||||||
|
self.runner = CliRunner()
|
||||||
|
|
||||||
|
def test_refuses_double_start_without_touching_settings_file(self, monkeypatch, tmp_path):
|
||||||
|
settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
existing_backup = {"existed": False, "content": None}
|
||||||
|
backup_path.write_text(json.dumps(existing_backup))
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch(f"{UP_MODULE}.load_token", return_value={"key": "sk-fresh", "base_url": "http://localhost:4000"}),
|
||||||
|
patch(f"{UP_MODULE}.is_cli_token_fresh", return_value=True),
|
||||||
|
patch(f"{UP_MODULE}.resolve_api_key", return_value="sk-fresh"),
|
||||||
|
patch(f"{UP_MODULE}.verify_proxy_key"),
|
||||||
|
):
|
||||||
|
result = self.runner.invoke(up, obj={"base_url": "http://localhost:4000"})
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "already" in result.output
|
||||||
|
assert "lite down" in result.output
|
||||||
|
assert not settings_path.exists()
|
||||||
|
assert json.loads(backup_path.read_text()) == existing_backup
|
||||||
|
|
||||||
|
def test_no_fresh_login_non_interactive_fails_cleanly(self, monkeypatch, tmp_path):
|
||||||
|
_patch_paths(monkeypatch, tmp_path)
|
||||||
|
monkeypatch.setattr(sys.stdin, "isatty", lambda: False)
|
||||||
|
|
||||||
|
with patch(f"{UP_MODULE}.load_token", return_value=None):
|
||||||
|
result = self.runner.invoke(up, obj={"base_url": "http://localhost:4000"})
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "lite login" in result.output
|
||||||
|
|
||||||
|
def test_unreachable_proxy_fails_cleanly(self, monkeypatch, tmp_path):
|
||||||
|
_patch_paths(monkeypatch, tmp_path)
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch(f"{UP_MODULE}.load_token", return_value={"key": "sk-fresh", "base_url": "http://localhost:4000"}),
|
||||||
|
patch(f"{UP_MODULE}.is_cli_token_fresh", return_value=True),
|
||||||
|
patch(f"{UP_MODULE}.resolve_api_key", return_value="sk-fresh"),
|
||||||
|
patch(
|
||||||
|
f"{UP_MODULE}.verify_proxy_key",
|
||||||
|
side_effect=AgentRunError("Could not reach the LiteLLM proxy at http://localhost:4000"),
|
||||||
|
),
|
||||||
|
):
|
||||||
|
result = self.runner.invoke(up, obj={"base_url": "http://localhost:4000"})
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert "Could not reach the LiteLLM proxy" in result.output
|
||||||
|
|
||||||
|
def test_happy_path_writes_settings_and_backup_then_restores_on_stop(self, monkeypatch, tmp_path):
|
||||||
|
settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
original = {"theme": "dark"}
|
||||||
|
settings_path.write_text(json.dumps(original))
|
||||||
|
|
||||||
|
captured = {}
|
||||||
|
|
||||||
|
def fake_wait(self, timeout=None):
|
||||||
|
captured["settings"] = json.loads(settings_path.read_text())
|
||||||
|
captured["backup_existed"] = backup_path.exists()
|
||||||
|
return True
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch(f"{UP_MODULE}.load_token", return_value={"key": "sk-fresh", "base_url": "http://localhost:4000"}),
|
||||||
|
patch(f"{UP_MODULE}.is_cli_token_fresh", return_value=True),
|
||||||
|
patch(f"{UP_MODULE}.resolve_api_key", return_value="sk-fresh"),
|
||||||
|
patch(f"{UP_MODULE}.verify_proxy_key"),
|
||||||
|
patch(
|
||||||
|
f"{UP_MODULE}.resolve_api_key_helper",
|
||||||
|
return_value="/usr/local/bin/lite auth print-token",
|
||||||
|
),
|
||||||
|
patch(f"{UP_MODULE}.signal.signal"),
|
||||||
|
patch(f"{UP_MODULE}.atexit.register"),
|
||||||
|
patch("threading.Event.wait", new=fake_wait),
|
||||||
|
):
|
||||||
|
result = self.runner.invoke(up, obj={"base_url": "http://localhost:4000"})
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert captured["backup_existed"] is True
|
||||||
|
assert captured["settings"]["theme"] == "dark"
|
||||||
|
assert captured["settings"]["env"]["ANTHROPIC_BASE_URL"] == "http://localhost:4000"
|
||||||
|
assert captured["settings"]["apiKeyHelper"] == "/usr/local/bin/lite auth print-token"
|
||||||
|
assert json.loads(settings_path.read_text()) == original
|
||||||
|
assert not backup_path.exists()
|
||||||
|
|
||||||
|
|
||||||
|
class TestDownCommand:
|
||||||
|
def setup_method(self):
|
||||||
|
self.runner = CliRunner()
|
||||||
|
|
||||||
|
def test_restores_when_backup_exists(self, monkeypatch, tmp_path):
|
||||||
|
settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
original = {"apiKeyHelper": "old-helper"}
|
||||||
|
write_backup(BackupRecord(existed=True, content=original))
|
||||||
|
settings_path.write_text(json.dumps({"apiKeyHelper": "lite-helper"}))
|
||||||
|
|
||||||
|
result = self.runner.invoke(down)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "Restored" in result.output
|
||||||
|
assert json.loads(settings_path.read_text()) == original
|
||||||
|
assert not backup_path.exists()
|
||||||
|
|
||||||
|
def test_removes_settings_file_when_it_did_not_exist_before(self, monkeypatch, tmp_path):
|
||||||
|
settings_path, _backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
write_backup(BackupRecord(existed=False, content=None))
|
||||||
|
settings_path.write_text(json.dumps({"apiKeyHelper": "lite-helper"}))
|
||||||
|
|
||||||
|
result = self.runner.invoke(down)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "Removed" in result.output
|
||||||
|
assert not settings_path.exists()
|
||||||
|
|
||||||
|
def test_prints_nothing_to_restore_when_no_backup(self, monkeypatch, tmp_path):
|
||||||
|
_patch_paths(monkeypatch, tmp_path)
|
||||||
|
|
||||||
|
result = self.runner.invoke(down)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "Nothing to restore." in result.output
|
||||||
|
|
||||||
|
def test_surfaces_clean_error_on_a_corrupt_backup_file(self, monkeypatch, tmp_path):
|
||||||
|
_settings_path, backup_path = _patch_paths(monkeypatch, tmp_path)
|
||||||
|
backup_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
backup_path.write_text("not json at all {{{")
|
||||||
|
|
||||||
|
result = self.runner.invoke(down)
|
||||||
|
|
||||||
|
assert result.exit_code != 0
|
||||||
|
assert result.exception is None or isinstance(result.exception, SystemExit)
|
||||||
|
assert "invalid or unexpected JSON" in result.output
|
||||||
154
uv.lock
generated
154
uv.lock
generated
|
|
@ -10,7 +10,7 @@ resolution-markers = [
|
||||||
]
|
]
|
||||||
|
|
||||||
[options]
|
[options]
|
||||||
exclude-newer = "2026-07-13T00:19:39.570486Z"
|
exclude-newer = "2026-07-13T03:38:04.421387Z"
|
||||||
exclude-newer-span = "P3D"
|
exclude-newer-span = "P3D"
|
||||||
|
|
||||||
[manifest]
|
[manifest]
|
||||||
|
|
@ -222,9 +222,9 @@ name = "aiologic"
|
||||||
version = "0.17.0"
|
version = "0.17.0"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "sniffio", marker = "python_full_version < '3.13'" },
|
{ name = "sniffio" },
|
||||||
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
|
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
|
||||||
{ name = "wrapt", marker = "python_full_version < '3.13'" },
|
{ name = "wrapt" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/53/a7/809482759f40079f4c4328c7318bf569ae25d457f5017aad30a1b9aafedc/aiologic-0.17.0.tar.gz", hash = "sha256:65aa058e858c94cd208badb188e7f00b54dcabb3ba85b34f794db98074d108b9", size = 251625, upload-time = "2026-06-14T12:24:35.367Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/53/a7/809482759f40079f4c4328c7318bf569ae25d457f5017aad30a1b9aafedc/aiologic-0.17.0.tar.gz", hash = "sha256:65aa058e858c94cd208badb188e7f00b54dcabb3ba85b34f794db98074d108b9", size = 251625, upload-time = "2026-06-14T12:24:35.367Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -516,14 +516,14 @@ name = "aurelio-sdk"
|
||||||
version = "0.0.19"
|
version = "0.0.19"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "aiofiles", marker = "python_full_version < '3.14'" },
|
{ name = "aiofiles" },
|
||||||
{ name = "aiohttp", marker = "python_full_version < '3.14'" },
|
{ name = "aiohttp" },
|
||||||
{ name = "colorlog", marker = "python_full_version < '3.14'" },
|
{ name = "colorlog" },
|
||||||
{ name = "pydantic", marker = "python_full_version < '3.14'" },
|
{ name = "pydantic" },
|
||||||
{ name = "python-dotenv", marker = "python_full_version < '3.14'" },
|
{ name = "python-dotenv" },
|
||||||
{ name = "requests", marker = "python_full_version < '3.14'" },
|
{ name = "requests" },
|
||||||
{ name = "requests-toolbelt", marker = "python_full_version < '3.14'" },
|
{ name = "requests-toolbelt" },
|
||||||
{ name = "tornado", marker = "python_full_version < '3.14'" },
|
{ name = "tornado" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/27/0e/c2e369ad173fb3d76448e46d10beb3dcc53388318933ddf8169a3f21a810/aurelio_sdk-0.0.19.tar.gz", hash = "sha256:14107e7440ff2efd0b4a08c52fb595e7680bd4bc973a0ddfb3b64157c6666b91", size = 15258, upload-time = "2025-03-24T14:37:32.203Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/27/0e/c2e369ad173fb3d76448e46d10beb3dcc53388318933ddf8169a3f21a810/aurelio_sdk-0.0.19.tar.gz", hash = "sha256:14107e7440ff2efd0b4a08c52fb595e7680bd4bc973a0ddfb3b64157c6666b91", size = 15258, upload-time = "2025-03-24T14:37:32.203Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -1047,7 +1047,7 @@ name = "coloredlogs"
|
||||||
version = "15.0.1"
|
version = "15.0.1"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "humanfriendly", marker = "python_full_version < '3.14'" },
|
{ name = "humanfriendly" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/cc/c7/eed8f27100517e8c0e6b923d5f0845d0cb99763da6fdee00478f91db7325/coloredlogs-15.0.1.tar.gz", hash = "sha256:7c991aa71a4577af2f82600d8f8f3a89f936baeaf9b50a9c197da014e5bf16b0", size = 278520, upload-time = "2021-06-11T10:22:45.202Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/cc/c7/eed8f27100517e8c0e6b923d5f0845d0cb99763da6fdee00478f91db7325/coloredlogs-15.0.1.tar.gz", hash = "sha256:7c991aa71a4577af2f82600d8f8f3a89f936baeaf9b50a9c197da014e5bf16b0", size = 278520, upload-time = "2021-06-11T10:22:45.202Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -1059,7 +1059,7 @@ name = "colorlog"
|
||||||
version = "6.10.1"
|
version = "6.10.1"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "colorama", marker = "python_full_version < '3.14' and sys_platform == 'win32'" },
|
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/a2/61/f083b5ac52e505dfc1c624eafbf8c7589a0d7f32daa398d2e7590efa5fda/colorlog-6.10.1.tar.gz", hash = "sha256:eb4ae5cb65fe7fec7773c2306061a8e63e02efc2c72eba9d27b0fa23c94f1321", size = 17162, upload-time = "2025-10-16T16:14:11.978Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/a2/61/f083b5ac52e505dfc1c624eafbf8c7589a0d7f32daa398d2e7590efa5fda/colorlog-6.10.1.tar.gz", hash = "sha256:eb4ae5cb65fe7fec7773c2306061a8e63e02efc2c72eba9d27b0fa23c94f1321", size = 17162, upload-time = "2025-10-16T16:14:11.978Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -1420,7 +1420,7 @@ name = "culsans"
|
||||||
version = "0.11.0"
|
version = "0.11.0"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "aiologic", marker = "python_full_version < '3.13'" },
|
{ name = "aiologic" },
|
||||||
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
|
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/d9/e3/49afa1bc180e0d28008ec6bcdf82a4072d1c7a41032b5b759b60814ca4b0/culsans-0.11.0.tar.gz", hash = "sha256:0b43d0d05dce6106293d114c86e3fb4bfc63088cfe8ff08ed3fe36891447fe33", size = 107546, upload-time = "2025-12-31T23:15:38.196Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/d9/e3/49afa1bc180e0d28008ec6bcdf82a4072d1c7a41032b5b759b60814ca4b0/culsans-0.11.0.tar.gz", hash = "sha256:0b43d0d05dce6106293d114c86e3fb4bfc63088cfe8ff08ed3fe36891447fe33", size = 107546, upload-time = "2025-12-31T23:15:38.196Z" }
|
||||||
|
|
@ -2966,7 +2966,7 @@ name = "humanfriendly"
|
||||||
version = "10.0"
|
version = "10.0"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "pyreadline3", marker = "python_full_version < '3.14' and sys_platform == 'win32'" },
|
{ name = "pyreadline3", marker = "sys_platform == 'win32'" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/cc/3f/2c29224acb2e2df4d2046e4c73ee2662023c58ff5b113c4c1adac0886c43/humanfriendly-10.0.tar.gz", hash = "sha256:6b0b831ce8f15f7300721aa49829fc4e83921a9a301cc7f606be6686a2288ddc", size = 360702, upload-time = "2021-09-17T21:40:43.31Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/cc/3f/2c29224acb2e2df4d2046e4c73ee2662023c58ff5b113c4c1adac0886c43/humanfriendly-10.0.tar.gz", hash = "sha256:6b0b831ce8f15f7300721aa49829fc4e83921a9a301cc7f606be6686a2288ddc", size = 360702, upload-time = "2021-09-17T21:40:43.31Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -3049,6 +3049,19 @@ wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "inquirerpy"
|
||||||
|
version = "0.3.4"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "pfzy" },
|
||||||
|
{ name = "prompt-toolkit" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/64/73/7570847b9da026e07053da3bbe2ac7ea6cde6bb2cbd3c7a5a950fa0ae40b/InquirerPy-0.3.4.tar.gz", hash = "sha256:89d2ada0111f337483cb41ae31073108b2ec1e618a49d7110b0d7ade89fc197e", size = 44431, upload-time = "2022-06-27T23:11:20.598Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ce/ff/3b59672c47c6284e8005b42e84ceba13864aa0f39f067c973d1af02f5d91/InquirerPy-0.3.4-py3-none-any.whl", hash = "sha256:c65fdfbac1fa00e3ee4fb10679f4d3ed7a012abf4833910e63c295827fe2a7d4", size = 67677, upload-time = "2022-06-27T23:11:17.723Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "isodate"
|
name = "isodate"
|
||||||
version = "0.7.2"
|
version = "0.7.2"
|
||||||
|
|
@ -3280,10 +3293,10 @@ name = "jsonschema-path"
|
||||||
version = "0.3.4"
|
version = "0.3.4"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "pathable", marker = "python_full_version < '3.14'" },
|
{ name = "pathable" },
|
||||||
{ name = "pyyaml", marker = "python_full_version < '3.14'" },
|
{ name = "pyyaml" },
|
||||||
{ name = "referencing", marker = "python_full_version < '3.14'" },
|
{ name = "referencing" },
|
||||||
{ name = "requests", marker = "python_full_version < '3.14'" },
|
{ name = "requests" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/6e/45/41ebc679c2a4fced6a722f624c18d658dee42612b83ea24c1caf7c0eb3a8/jsonschema_path-0.3.4.tar.gz", hash = "sha256:8365356039f16cc65fddffafda5f58766e34bebab7d6d105616ab52bc4297001", size = 11159, upload-time = "2025-01-24T14:33:16.547Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/6e/45/41ebc679c2a4fced6a722f624c18d658dee42612b83ea24c1caf7c0eb3a8/jsonschema_path-0.3.4.tar.gz", hash = "sha256:8365356039f16cc65fddffafda5f58766e34bebab7d6d105616ab52bc4297001", size = 11159, upload-time = "2025-01-24T14:33:16.547Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -3754,6 +3767,7 @@ caching = [
|
||||||
{ name = "diskcache" },
|
{ name = "diskcache" },
|
||||||
]
|
]
|
||||||
cli = [
|
cli = [
|
||||||
|
{ name = "inquirerpy" },
|
||||||
{ name = "pyyaml" },
|
{ name = "pyyaml" },
|
||||||
{ name = "requests" },
|
{ name = "requests" },
|
||||||
{ name = "rich" },
|
{ name = "rich" },
|
||||||
|
|
@ -3789,6 +3803,7 @@ proxy = [
|
||||||
{ name = "fastapi-sso" },
|
{ name = "fastapi-sso" },
|
||||||
{ name = "granian" },
|
{ name = "granian" },
|
||||||
{ name = "gunicorn" },
|
{ name = "gunicorn" },
|
||||||
|
{ name = "inquirerpy" },
|
||||||
{ name = "litellm-enterprise" },
|
{ name = "litellm-enterprise" },
|
||||||
{ name = "litellm-proxy-extras" },
|
{ name = "litellm-proxy-extras" },
|
||||||
{ name = "mcp" },
|
{ name = "mcp" },
|
||||||
|
|
@ -3966,6 +3981,8 @@ requires-dist = [
|
||||||
{ name = "gunicorn", marker = "extra == 'proxy'", specifier = ">=23.0.0,<24.0" },
|
{ name = "gunicorn", marker = "extra == 'proxy'", specifier = ">=23.0.0,<24.0" },
|
||||||
{ name = "httpx", specifier = ">=0.28.0,<1.0" },
|
{ name = "httpx", specifier = ">=0.28.0,<1.0" },
|
||||||
{ name = "importlib-metadata", specifier = ">=8.0.0,<9.0" },
|
{ name = "importlib-metadata", specifier = ">=8.0.0,<9.0" },
|
||||||
|
{ name = "inquirerpy", marker = "extra == 'cli'", specifier = ">=0.3.4,<1.0" },
|
||||||
|
{ name = "inquirerpy", marker = "extra == 'proxy'", specifier = ">=0.3.4,<1.0" },
|
||||||
{ name = "jinja2", specifier = ">=3.1.6,<4.0" },
|
{ name = "jinja2", specifier = ">=3.1.6,<4.0" },
|
||||||
{ name = "jsonschema", specifier = ">=4.0.0,<5.0" },
|
{ name = "jsonschema", specifier = ">=4.0.0,<5.0" },
|
||||||
{ name = "langfuse", marker = "extra == 'proxy-runtime'", specifier = ">=2.59.7,<3.0" },
|
{ name = "langfuse", marker = "extra == 'proxy-runtime'", specifier = ">=2.59.7,<3.0" },
|
||||||
|
|
@ -4464,7 +4481,7 @@ version = "0.4.1"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },
|
{ name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },
|
||||||
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12' and python_full_version < '3.14'" },
|
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/fd/15/76f86faa0902836cc133939732f7611ace68cf54148487a99c539c272dc8/ml_dtypes-0.4.1.tar.gz", hash = "sha256:fad5f2de464fd09127e49b7fd1252b9006fb43d2edc1ff112d390c324af5ca7a", size = 692594, upload-time = "2024-09-13T19:07:11.624Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/fd/15/76f86faa0902836cc133939732f7611ace68cf54148487a99c539c272dc8/ml_dtypes-0.4.1.tar.gz", hash = "sha256:fad5f2de464fd09127e49b7fd1252b9006fb43d2edc1ff112d390c324af5ca7a", size = 692594, upload-time = "2024-09-13T19:07:11.624Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -4963,14 +4980,14 @@ name = "openapi-core"
|
||||||
version = "0.22.0"
|
version = "0.22.0"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "isodate", marker = "python_full_version < '3.14'" },
|
{ name = "isodate" },
|
||||||
{ name = "jsonschema", marker = "python_full_version < '3.14'" },
|
{ name = "jsonschema" },
|
||||||
{ name = "jsonschema-path", marker = "python_full_version < '3.14'" },
|
{ name = "jsonschema-path" },
|
||||||
{ name = "more-itertools", marker = "python_full_version < '3.14'" },
|
{ name = "more-itertools" },
|
||||||
{ name = "openapi-schema-validator", marker = "python_full_version < '3.14'" },
|
{ name = "openapi-schema-validator" },
|
||||||
{ name = "openapi-spec-validator", marker = "python_full_version < '3.14'" },
|
{ name = "openapi-spec-validator" },
|
||||||
{ name = "typing-extensions", marker = "python_full_version < '3.14'" },
|
{ name = "typing-extensions" },
|
||||||
{ name = "werkzeug", marker = "python_full_version < '3.14'" },
|
{ name = "werkzeug" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/fd/65/ee75f25b9459a02df6f713f8ffde5dacb57b8b4e45145cde4cab28b5abba/openapi_core-0.22.0.tar.gz", hash = "sha256:b30490dfa74e3aac2276105525590135212352f5dd7e5acf8f62f6a89ed6f2d0", size = 109242, upload-time = "2025-12-22T19:19:49.608Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/fd/65/ee75f25b9459a02df6f713f8ffde5dacb57b8b4e45145cde4cab28b5abba/openapi_core-0.22.0.tar.gz", hash = "sha256:b30490dfa74e3aac2276105525590135212352f5dd7e5acf8f62f6a89ed6f2d0", size = 109242, upload-time = "2025-12-22T19:19:49.608Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -4982,9 +4999,9 @@ name = "openapi-schema-validator"
|
||||||
version = "0.6.3"
|
version = "0.6.3"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "jsonschema", marker = "python_full_version < '3.14'" },
|
{ name = "jsonschema" },
|
||||||
{ name = "jsonschema-specifications", marker = "python_full_version < '3.14'" },
|
{ name = "jsonschema-specifications" },
|
||||||
{ name = "rfc3339-validator", marker = "python_full_version < '3.14'" },
|
{ name = "rfc3339-validator" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/8b/f3/5507ad3325169347cd8ced61c232ff3df70e2b250c49f0fe140edb4973c6/openapi_schema_validator-0.6.3.tar.gz", hash = "sha256:f37bace4fc2a5d96692f4f8b31dc0f8d7400fd04f3a937798eaf880d425de6ee", size = 11550, upload-time = "2025-01-10T18:08:22.268Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/8b/f3/5507ad3325169347cd8ced61c232ff3df70e2b250c49f0fe140edb4973c6/openapi_schema_validator-0.6.3.tar.gz", hash = "sha256:f37bace4fc2a5d96692f4f8b31dc0f8d7400fd04f3a937798eaf880d425de6ee", size = 11550, upload-time = "2025-01-10T18:08:22.268Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -4996,10 +5013,10 @@ name = "openapi-spec-validator"
|
||||||
version = "0.7.2"
|
version = "0.7.2"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "jsonschema", marker = "python_full_version < '3.14'" },
|
{ name = "jsonschema" },
|
||||||
{ name = "jsonschema-path", marker = "python_full_version < '3.14'" },
|
{ name = "jsonschema-path" },
|
||||||
{ name = "lazy-object-proxy", marker = "python_full_version < '3.14'" },
|
{ name = "lazy-object-proxy" },
|
||||||
{ name = "openapi-schema-validator", marker = "python_full_version < '3.14'" },
|
{ name = "openapi-schema-validator" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/82/af/fe2d7618d6eae6fb3a82766a44ed87cd8d6d82b4564ed1c7cfb0f6378e91/openapi_spec_validator-0.7.2.tar.gz", hash = "sha256:cc029309b5c5dbc7859df0372d55e9d1ff43e96d678b9ba087f7c56fc586f734", size = 36855, upload-time = "2025-06-07T14:48:56.299Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/82/af/fe2d7618d6eae6fb3a82766a44ed87cd8d6d82b4564ed1c7cfb0f6378e91/openapi_spec_validator-0.7.2.tar.gz", hash = "sha256:cc029309b5c5dbc7859df0372d55e9d1ff43e96d678b9ba087f7c56fc586f734", size = 36855, upload-time = "2025-06-07T14:48:56.299Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -5862,6 +5879,15 @@ wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/7d/eb/b6260b31b1a96386c0a880edebe26f89669098acea8e0318bff6adb378fd/pathable-0.4.4-py3-none-any.whl", hash = "sha256:5ae9e94793b6ef5a4cbe0a7ce9dbbefc1eec38df253763fd0aeeacf2762dbbc2", size = 9592, upload-time = "2025-01-10T18:43:11.88Z" },
|
{ url = "https://files.pythonhosted.org/packages/7d/eb/b6260b31b1a96386c0a880edebe26f89669098acea8e0318bff6adb378fd/pathable-0.4.4-py3-none-any.whl", hash = "sha256:5ae9e94793b6ef5a4cbe0a7ce9dbbefc1eec38df253763fd0aeeacf2762dbbc2", size = 9592, upload-time = "2025-01-10T18:43:11.88Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pfzy"
|
||||||
|
version = "0.3.4"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/d9/5a/32b50c077c86bfccc7bed4881c5a2b823518f5450a30e639db5d3711952e/pfzy-0.3.4.tar.gz", hash = "sha256:717ea765dd10b63618e7298b2d98efd819e0b30cd5905c9707223dceeb94b3f1", size = 8396, upload-time = "2022-01-28T02:26:17.946Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8c/d7/8ff98376b1acc4503253b685ea09981697385ce344d4e3935c2af49e044d/pfzy-0.3.4-py3-none-any.whl", hash = "sha256:5f50d5b2b3207fa72e7ec0ef08372ef652685470974a107d0d4999fc5a903a96", size = 8537, upload-time = "2022-01-28T02:26:16.047Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pillow"
|
name = "pillow"
|
||||||
version = "12.3.0"
|
version = "12.3.0"
|
||||||
|
|
@ -6075,6 +6101,18 @@ wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/c7/98/745b810d822103adca2df8decd4c0bbe839ba7ad3511af3f0d09692fc0f0/prometheus_client-0.20.0-py3-none-any.whl", hash = "sha256:cde524a85bce83ca359cc837f28b8c0db5cac7aa653a588fd7e84ba061c329e7", size = 54474, upload-time = "2024-02-14T15:55:03.957Z" },
|
{ url = "https://files.pythonhosted.org/packages/c7/98/745b810d822103adca2df8decd4c0bbe839ba7ad3511af3f0d09692fc0f0/prometheus_client-0.20.0-py3-none-any.whl", hash = "sha256:cde524a85bce83ca359cc837f28b8c0db5cac7aa653a588fd7e84ba061c329e7", size = 54474, upload-time = "2024-02-14T15:55:03.957Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "prompt-toolkit"
|
||||||
|
version = "3.0.52"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "wcwidth" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/a1/96/06e01a7b38dce6fe1db213e061a4602dd6032a8a97ef6c1a862537732421/prompt_toolkit-3.0.52.tar.gz", hash = "sha256:28cde192929c8e7321de85de1ddbe736f1375148b02f2e17edd840042b1be855", size = 434198, upload-time = "2025-08-27T15:24:02.057Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/84/03/0d3ce49e2505ae70cf43bc5bb3033955d2fc9f932163e84dc0779cc47f48/prompt_toolkit-3.0.52-py3-none-any.whl", hash = "sha256:9aac639a3bbd33284347de5ad8d68ecc044b91a762dc39b7c21095fcd6a19955", size = 391431, upload-time = "2025-08-27T15:23:59.498Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "propcache"
|
name = "propcache"
|
||||||
version = "0.5.2"
|
version = "0.5.2"
|
||||||
|
|
@ -7125,16 +7163,16 @@ name = "redisvl"
|
||||||
version = "0.4.1"
|
version = "0.4.1"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "coloredlogs", marker = "python_full_version < '3.14'" },
|
{ name = "coloredlogs" },
|
||||||
{ name = "ml-dtypes", marker = "python_full_version < '3.14'" },
|
{ name = "ml-dtypes" },
|
||||||
{ name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },
|
{ name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },
|
||||||
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12' and python_full_version < '3.14'" },
|
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },
|
||||||
{ name = "pydantic", marker = "python_full_version < '3.14'" },
|
{ name = "pydantic" },
|
||||||
{ name = "python-ulid", marker = "python_full_version < '3.14'" },
|
{ name = "python-ulid" },
|
||||||
{ name = "pyyaml", marker = "python_full_version < '3.14'" },
|
{ name = "pyyaml" },
|
||||||
{ name = "redis", marker = "python_full_version < '3.14'" },
|
{ name = "redis" },
|
||||||
{ name = "tabulate", marker = "python_full_version < '3.14'" },
|
{ name = "tabulate" },
|
||||||
{ name = "tenacity", marker = "python_full_version < '3.14'" },
|
{ name = "tenacity" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/21/33/ab14865a0b2a31b1d003c29e7e8ea3a7a2f2c8ecb24e58e58d606e1f031b/redisvl-0.4.1.tar.gz", hash = "sha256:fd6a36426ba94792c0efca20915c31232d4ee3cc58eb23794a62c142696401e6", size = 77688, upload-time = "2025-02-21T22:51:41.389Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/21/33/ab14865a0b2a31b1d003c29e7e8ea3a7a2f2c8ecb24e58e58d606e1f031b/redisvl-0.4.1.tar.gz", hash = "sha256:fd6a36426ba94792c0efca20915c31232d4ee3cc58eb23794a62c142696401e6", size = 77688, upload-time = "2025-02-21T22:51:41.389Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -7368,7 +7406,7 @@ name = "rfc3339-validator"
|
||||||
version = "0.1.4"
|
version = "0.1.4"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "six", marker = "python_full_version < '3.14'" },
|
{ name = "six" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/28/ea/a9387748e2d111c3c2b275ba970b735e04e15cdb1eb30693b6b5708c4dbd/rfc3339_validator-0.1.4.tar.gz", hash = "sha256:138a2abdf93304ad60530167e51d2dfb9549521a836871b88d7f4695d0022f6b", size = 5513, upload-time = "2021-05-12T16:37:54.178Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/28/ea/a9387748e2d111c3c2b275ba970b735e04e15cdb1eb30693b6b5708c4dbd/rfc3339_validator-0.1.4.tar.gz", hash = "sha256:138a2abdf93304ad60530167e51d2dfb9549521a836871b88d7f4695d0022f6b", size = 5513, upload-time = "2021-05-12T16:37:54.178Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
@ -7817,20 +7855,20 @@ name = "semantic-router"
|
||||||
version = "0.1.15"
|
version = "0.1.15"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "aiohttp", marker = "python_full_version < '3.14'" },
|
{ name = "aiohttp" },
|
||||||
{ name = "aurelio-sdk", marker = "python_full_version < '3.14'" },
|
{ name = "aurelio-sdk" },
|
||||||
{ name = "colorama", marker = "python_full_version < '3.14'" },
|
{ name = "colorama" },
|
||||||
{ name = "colorlog", marker = "python_full_version < '3.14'" },
|
{ name = "colorlog" },
|
||||||
{ name = "litellm", marker = "python_full_version < '3.14'" },
|
{ name = "litellm" },
|
||||||
{ name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },
|
{ name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },
|
||||||
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12' and python_full_version < '3.14'" },
|
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },
|
||||||
{ name = "openai", marker = "python_full_version < '3.14'" },
|
{ name = "openai" },
|
||||||
{ name = "pydantic", marker = "python_full_version < '3.14'" },
|
{ name = "pydantic" },
|
||||||
{ name = "pyyaml", marker = "python_full_version < '3.14'" },
|
{ name = "pyyaml" },
|
||||||
{ name = "regex", marker = "python_full_version < '3.14'" },
|
{ name = "regex" },
|
||||||
{ name = "tiktoken", marker = "python_full_version < '3.14'" },
|
{ name = "tiktoken" },
|
||||||
{ name = "tornado", marker = "python_full_version < '3.14'" },
|
{ name = "tornado" },
|
||||||
{ name = "urllib3", marker = "python_full_version < '3.14'" },
|
{ name = "urllib3" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/dc/a9/1a689e916e8b280f1fd8fb335cc059be626a22fe4533baa045d32fcd6de5/semantic_router-0.1.15.tar.gz", hash = "sha256:328256ddc3c2b713101ec69561d6585aecbf1198ea3461e1486289d8c3a35288", size = 95605, upload-time = "2026-05-23T12:58:15.444Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/dc/a9/1a689e916e8b280f1fd8fb335cc059be626a22fe4533baa045d32fcd6de5/semantic_router-0.1.15.tar.gz", hash = "sha256:328256ddc3c2b713101ec69561d6585aecbf1198ea3461e1486289d8c3a35288", size = 95605, upload-time = "2026-05-23T12:58:15.444Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue