mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-07 08:26:10 +00:00
Merge pull request #34026 from BerriAI/litellm_lit4608_autoroute_fixed_port_key
fix(cli): stable port and persisted master key for lite autoroute up
This commit is contained in:
tin-berri
GitHub
commit
2ea09a9969
10 changed files with 371 additions and 31 deletions
|
|
@ -545,7 +545,7 @@ You must run `configure` at least once before `up`; running `up` first fails wit
|
|||
lite autoroute up
|
||||
```
|
||||
|
||||
Starts a local, throwaway litellm proxy on a random free port, running the config `configure` generated, with a freshly-minted random API key baked in for this session only (your real proxy key never leaves the generated config -- it only appears there, forwarding to your real proxy). It waits for the ephemeral proxy to report healthy, then patches `~/.claude/settings.json` the same way `lite up` does, except with a static `ANTHROPIC_AUTH_TOKEN` env var instead of an `apiKeyHelper`, since this key is short-lived and self-issued rather than something needing SSO refresh. Any `claude` session started afterward, from any terminal, routes through the ephemeral proxy.
|
||||
Starts a local, throwaway litellm proxy on `127.0.0.1:5483` (override with `--port`), running the config `configure` generated, with a self-issued API key baked in (your real proxy key never leaves the generated config -- it only appears there, forwarding to your real proxy). Both the port and the key are stable across runs: the key is minted once, persisted inside the generated config, and reused by every later `up` (and carried forward when you re-run `configure`), so anything you configured against one session keeps working in the next. If the port is already taken, `up` refuses with a clear error instead of silently moving to another one. It waits for the ephemeral proxy to report healthy, then patches `~/.claude/settings.json` the same way `lite up` does, except with a static `ANTHROPIC_AUTH_TOKEN` env var instead of an `apiKeyHelper`, since this key is self-issued rather than something needing SSO refresh. Any `claude` session started afterward, from any terminal, routes through the ephemeral proxy.
|
||||
|
||||
`lite autoroute up` runs in the foreground and streams the ephemeral proxy's own log file into your terminal, so you can watch its routing decisions -- which tier and model got picked for each request -- as you use Claude Code normally. Press Ctrl-C (or send SIGTERM) to stop it; this kills the child proxy process and restores your original Claude Code settings, in that order.
|
||||
|
||||
|
|
@ -570,7 +570,7 @@ lite autoroute down # only needed if `up` was killed uncleanly instead of Ctrl
|
|||
|
||||
Adaptive mode's learned state does not persist across `lite autoroute up` sessions -- there is no local database, so every session starts adaptive selection cold. A Claude Code session already running before `up` started, or still running when it stops, keeps whatever settings it loaded at its own startup; like `lite up`, this is a one-time file patch and restore, not a live traffic interceptor. Only Claude Code is supported, for the same reason as `lite up`: no other supported agent (for example Cursor) has an equivalent hot-patchable config file.
|
||||
|
||||
A session that outlives `up` (or is still running the moment you stop it) keeps sending requests, master key included, to that now-freed loopback port until you restart it. Once the ephemeral proxy process exits, nothing stops another local account on the same machine from binding that same port and receiving those requests instead -- unlike `lite up`'s `apiKeyHelper`, which is re-resolved per request, `autoroute`'s master key is a static value, so whoever receives them gets a live-looking token along with the prompt content. Restart any Claude Code session before you consider the machine clean, run `lite autoroute down` promptly rather than leaving a stopped session's settings patched, and do not run `lite autoroute up` on a shared or multi-tenant host.
|
||||
A session that outlives `up` (or is still running the moment you stop it) keeps sending requests, master key included, to that now-freed loopback port until you restart it. Once the ephemeral proxy process exits, nothing stops another local account on the same machine from binding that same port and receiving those requests instead -- and since the port is a fixed, predictable default and the master key is a static value that persists across sessions (unlike `lite up`'s `apiKeyHelper`, which is re-resolved per request), whoever receives them gets a live-looking token along with the prompt content. Restart any Claude Code session before you consider the machine clean, run `lite autoroute down` promptly rather than leaving a stopped session's settings patched, and do not run `lite autoroute up` on a shared or multi-tenant host. To rotate the persisted key, delete the `master_key` line from `~/.litellm/autorouter/config.yaml`; the next `up` mints a fresh one (deleting the whole file works too, but then `configure` must be re-run first).
|
||||
|
||||
Do not run `lite up` and `lite autoroute up` at the same time. Each patches `~/.claude/settings.json` and keeps its own separate backup, with no coordination between them: whichever one you stop or crash out of last is the one whose backup gets restored, which can silently leave the *other* mode's settings (a static master key and a now-dead loopback URL, or a stale `apiKeyHelper`) active. Run `lite down` or `lite autoroute down` (whichever applies) before switching to the other mode.
|
||||
|
||||
|
|
|
|||
|
|
@ -11,14 +11,16 @@ from pydantic import JsonValue, TypeAdapter, ValidationError
|
|||
|
||||
from ..up import CLAUDE_SETTINGS_PATH, UpError, load_json_or_empty, restore_claude_settings, write_backup
|
||||
from ..up import BackupRecord as ClaudeBackupRecord
|
||||
from .config import master_key_from_config
|
||||
from .process import (
|
||||
AUTOROUTE_DIR,
|
||||
CONFIG_PATH,
|
||||
DEFAULT_AUTOROUTE_PORT,
|
||||
LOG_PATH,
|
||||
PidRecord,
|
||||
ProcessLaunchError,
|
||||
allocate_free_port,
|
||||
clear_pid_record,
|
||||
is_port_available,
|
||||
is_running,
|
||||
launch_proxy,
|
||||
missing_proxy_runtime_modules,
|
||||
|
|
@ -37,15 +39,15 @@ AUTOROUTE_BACKUP_PATH = AUTOROUTE_DIR / "claude_settings_backup.json"
|
|||
_GENERATED_CONFIG_ADAPTER = TypeAdapter(dict[str, JsonValue])
|
||||
|
||||
|
||||
def _mint_and_embed_master_key() -> str:
|
||||
"""Generate a fresh key for this session and write it into the generated config.yaml.
|
||||
def _ensure_master_key() -> str:
|
||||
"""Reuse the master key already persisted in the generated config.yaml, minting one only when absent.
|
||||
|
||||
Must go under general_settings, not litellm_settings -- the proxy server only ever
|
||||
reads general_settings.master_key (proxy_server.py:4530) to authenticate requests. A
|
||||
key placed under litellm_settings is silently ignored, leaving the ephemeral proxy with
|
||||
no real auth: any request reaches it regardless of the token Claude Code sends.
|
||||
The generated config is the single home of the key: the proxy server authenticates against
|
||||
general_settings.master_key only (a key under litellm_settings is silently ignored, which
|
||||
would leave the ephemeral proxy with no real auth), and the file is written 0600 via
|
||||
secure_create. Reusing that persisted value keeps the key stable across `up` runs, so a
|
||||
client configured against one session keeps working in the next.
|
||||
"""
|
||||
master_key = secrets.token_urlsafe(32)
|
||||
with open(CONFIG_PATH, "r") as f:
|
||||
try:
|
||||
generated = _GENERATED_CONFIG_ADAPTER.validate_python(yaml.safe_load(f))
|
||||
|
|
@ -53,6 +55,10 @@ def _mint_and_embed_master_key() -> str:
|
|||
raise click.ClickException(
|
||||
f"{CONFIG_PATH} is empty or corrupt. Run `lite autoroute configure` again to regenerate it."
|
||||
)
|
||||
persisted = master_key_from_config(generated)
|
||||
if persisted is not None:
|
||||
return persisted
|
||||
master_key = secrets.token_urlsafe(32)
|
||||
general_settings = generated.get("general_settings")
|
||||
updated_settings: dict[str, JsonValue] = {
|
||||
**(general_settings if isinstance(general_settings, dict) else {}),
|
||||
|
|
@ -77,7 +83,14 @@ def configure(ctx: click.Context) -> None:
|
|||
|
||||
|
||||
@autoroute_group.command("up")
|
||||
def up() -> None:
|
||||
@click.option(
|
||||
"--port",
|
||||
type=click.IntRange(1, 65535),
|
||||
default=DEFAULT_AUTOROUTE_PORT,
|
||||
show_default=True,
|
||||
help="Loopback port for the ephemeral proxy; stable across runs so configured clients keep working.",
|
||||
)
|
||||
def up(port: int) -> None:
|
||||
"""Launch the ephemeral auto-router proxy and route Claude Code through it"""
|
||||
if not CONFIG_PATH.exists():
|
||||
raise click.ClickException("No config found. Run `lite autoroute configure` first.")
|
||||
|
|
@ -108,8 +121,19 @@ def up() -> None:
|
|||
"running (or crashed without cleanup). Run `lite autoroute down` first."
|
||||
)
|
||||
|
||||
master_key = _mint_and_embed_master_key()
|
||||
port = allocate_free_port()
|
||||
if port == 4000:
|
||||
raise click.ClickException(
|
||||
"Port 4000 is the litellm proxy's own default and its launcher silently rebinds it to a random "
|
||||
"port when busy; pick a different --port."
|
||||
)
|
||||
|
||||
if not is_port_available(port):
|
||||
raise click.ClickException(
|
||||
f"Port {port} on 127.0.0.1 is already in use. If a previous `lite autoroute up` is still "
|
||||
"running or crashed, run `lite autoroute down`; otherwise pick a different port with --port."
|
||||
)
|
||||
|
||||
master_key = _ensure_master_key()
|
||||
base_url = f"http://127.0.0.1:{port}"
|
||||
process = launch_proxy(CONFIG_PATH, port, LOG_PATH)
|
||||
write_pid_record(PidRecord(pid=process.pid, port=port, config_path=str(CONFIG_PATH), log_path=str(LOG_PATH)))
|
||||
|
|
|
|||
|
|
@ -226,6 +226,24 @@ def build_generated_proxy_config(config: AutorouteConfig, master_key: str) -> di
|
|||
}
|
||||
|
||||
|
||||
def master_key_from_config(config: dict[str, JsonValue]) -> str | None:
|
||||
"""The master key persisted in a generated config, or None when absent or blank.
|
||||
|
||||
Single definition of "this config already has a usable key", shared by `up` (reuse
|
||||
instead of minting) and the configure wizard (carry the key forward on rewrite) so the
|
||||
two sites can never disagree on what counts as one. Returned verbatim, never stripped:
|
||||
the proxy authenticates against the exact bytes under general_settings.master_key, so a
|
||||
normalized copy here would diverge from what the proxy expects.
|
||||
"""
|
||||
general_settings = config.get("general_settings")
|
||||
if not isinstance(general_settings, dict):
|
||||
return None
|
||||
master_key = general_settings.get("master_key")
|
||||
if isinstance(master_key, str) and master_key.strip():
|
||||
return master_key
|
||||
return None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"AUTOROUTER_MODEL_NAME",
|
||||
"TIER_NAMES",
|
||||
|
|
@ -244,6 +262,7 @@ __all__ = [
|
|||
"build_generated_proxy_config",
|
||||
"chat_models",
|
||||
"embedding_models",
|
||||
"master_key_from_config",
|
||||
"parse_discovered_models",
|
||||
"validate_config",
|
||||
]
|
||||
|
|
|
|||
|
|
@ -52,10 +52,17 @@ def missing_proxy_runtime_modules() -> tuple[str, ...]:
|
|||
return tuple(name for name in _PROXY_RUNTIME_MODULES if importlib.util.find_spec(name) is None)
|
||||
|
||||
|
||||
def allocate_free_port() -> int:
|
||||
DEFAULT_AUTOROUTE_PORT = 5483
|
||||
|
||||
|
||||
def is_port_available(port: int) -> bool:
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.bind(("127.0.0.1", 0))
|
||||
return int(sock.getsockname()[1])
|
||||
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
try:
|
||||
sock.bind(("127.0.0.1", port))
|
||||
except OSError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def launch_proxy(config_path: Path, port: int, log_path: Path) -> "subprocess.Popen[bytes]":
|
||||
|
|
@ -172,12 +179,13 @@ def stream_log(log_path: Path, stop_event: threading.Event) -> None:
|
|||
__all__ = [
|
||||
"AUTOROUTE_DIR",
|
||||
"CONFIG_PATH",
|
||||
"DEFAULT_AUTOROUTE_PORT",
|
||||
"LOG_PATH",
|
||||
"PID_RECORD_PATH",
|
||||
"PidRecord",
|
||||
"ProcessLaunchError",
|
||||
"allocate_free_port",
|
||||
"clear_pid_record",
|
||||
"is_port_available",
|
||||
"is_running",
|
||||
"launch_proxy",
|
||||
"missing_proxy_runtime_modules",
|
||||
|
|
|
|||
|
|
@ -25,9 +25,9 @@ def merge_claude_settings_static_token(
|
|||
"""Return a new settings dict wired to a local ephemeral proxy with a static token.
|
||||
|
||||
Unlike up.py's merge_claude_settings (which sets apiKeyHelper for a long-lived, real
|
||||
remote proxy needing refreshable SSO tokens), this proxy is ephemeral and its key was just
|
||||
minted for this session, so a plain env var is simpler and correct. Any existing
|
||||
apiKeyHelper is cleared so it can't fight with the static token.
|
||||
remote proxy needing refreshable SSO tokens), this proxy is ephemeral and its key is the
|
||||
locally persisted autoroute master key, so a plain env var is simpler and correct. Any
|
||||
existing apiKeyHelper is cleared so it can't fight with the static token.
|
||||
"""
|
||||
raw_env = settings.get(ENV_KEY, {})
|
||||
base_env = raw_env if isinstance(raw_env, dict) else {}
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ import click
|
|||
import yaml
|
||||
from InquirerPy import inquirer
|
||||
from InquirerPy.base.control import Choice
|
||||
from pydantic import JsonValue, TypeAdapter, ValidationError
|
||||
|
||||
from .... import Client
|
||||
from .config import (
|
||||
|
|
@ -21,6 +22,7 @@ from .config import (
|
|||
build_generated_model_list,
|
||||
chat_models,
|
||||
embedding_models,
|
||||
master_key_from_config,
|
||||
parse_discovered_models,
|
||||
validate_config,
|
||||
)
|
||||
|
|
@ -84,6 +86,25 @@ def _prompt_for_keyword_tier_rules() -> tuple[KeywordTierRule, ...]:
|
|||
return tuple(_rule_for(tier) for tier in TIER_NAMES)
|
||||
|
||||
|
||||
_RAW_CONFIG_ADAPTER = TypeAdapter(dict[str, JsonValue])
|
||||
|
||||
|
||||
def _load_persisted_master_key(config_path: Path) -> str | None:
|
||||
"""The master key from an existing generated config, so a rewrite carries it forward.
|
||||
|
||||
Lenient on a missing or corrupt file: configure is the regeneration path, so it must
|
||||
succeed from any prior state; a key that cannot be read is simply not carried and `up`
|
||||
mints a fresh one.
|
||||
"""
|
||||
if not config_path.exists():
|
||||
return None
|
||||
try:
|
||||
raw = _RAW_CONFIG_ADAPTER.validate_python(yaml.safe_load(config_path.read_text()))
|
||||
except (OSError, UnicodeDecodeError, yaml.YAMLError, ValidationError):
|
||||
return None
|
||||
return master_key_from_config(raw)
|
||||
|
||||
|
||||
def run_configure_wizard(ctx: click.Context) -> Path:
|
||||
"""Discover the caller's accessible models, walk them through tier assignment, write config."""
|
||||
base_url = ctx.obj["base_url"]
|
||||
|
|
@ -137,9 +158,15 @@ def run_configure_wizard(ctx: click.Context) -> Path:
|
|||
raise click.ClickException(str(e))
|
||||
|
||||
model_list = build_generated_model_list(config)
|
||||
persisted_master_key = _load_persisted_master_key(CONFIG_PATH)
|
||||
generated: dict[str, JsonValue] = (
|
||||
{"model_list": model_list, "general_settings": {"master_key": persisted_master_key}}
|
||||
if persisted_master_key is not None
|
||||
else {"model_list": model_list}
|
||||
)
|
||||
CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
with secure_create(CONFIG_PATH) as f:
|
||||
yaml.safe_dump({"model_list": model_list}, f, sort_keys=False)
|
||||
yaml.safe_dump(generated, f, sort_keys=False)
|
||||
|
||||
click.echo(f"\nWrote {CONFIG_PATH}")
|
||||
for tier, models in tiers.items():
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import json
|
||||
import socket
|
||||
import stat
|
||||
from typing import Optional
|
||||
|
||||
|
|
@ -62,6 +63,7 @@ class TestUpCommand:
|
|||
generated-config model raises a raw pydantic.ValidationError if uncaught."""
|
||||
config_path, _log_path, _settings_path, _backup_path, _pid_record_path = _patch_paths(monkeypatch, tmp_path)
|
||||
config_path.write_text("")
|
||||
monkeypatch.setattr(commands_module, "is_port_available", lambda port: True)
|
||||
|
||||
result = self.runner.invoke(up)
|
||||
|
||||
|
|
@ -134,7 +136,7 @@ class TestUpCommand:
|
|||
terminate_calls = []
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: fake_process)
|
||||
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||
monkeypatch.setattr(commands_module, "allocate_free_port", lambda: 54321)
|
||||
monkeypatch.setattr(commands_module, "is_port_available", lambda port: True)
|
||||
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: terminate_calls.append(pid))
|
||||
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fixed-master-key")
|
||||
|
||||
|
|
@ -153,7 +155,7 @@ class TestUpCommand:
|
|||
assert result.exit_code == 0, result.output
|
||||
assert captured["backup_existed"] is True
|
||||
assert captured["settings"]["theme"] == "dark"
|
||||
assert captured["settings"]["env"]["ANTHROPIC_BASE_URL"] == "http://127.0.0.1:54321"
|
||||
assert captured["settings"]["env"]["ANTHROPIC_BASE_URL"] == "http://127.0.0.1:5483"
|
||||
assert captured["settings"]["env"]["ANTHROPIC_AUTH_TOKEN"] == "fixed-master-key"
|
||||
assert "apiKeyHelper" not in captured["settings"]
|
||||
assert captured["settings_mode"] == 0o600
|
||||
|
|
@ -179,7 +181,7 @@ class TestUpCommand:
|
|||
fake_process = FakeProcess(pid=11111)
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: fake_process)
|
||||
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||
monkeypatch.setattr(commands_module, "allocate_free_port", lambda: 65432)
|
||||
monkeypatch.setattr(commands_module, "is_port_available", lambda port: True)
|
||||
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: None)
|
||||
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fixed-master-key")
|
||||
|
||||
|
|
@ -209,7 +211,7 @@ class TestUpCommand:
|
|||
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: fake_process)
|
||||
monkeypatch.setattr(commands_module, "poll_liveliness", _raise_launch_error)
|
||||
monkeypatch.setattr(commands_module, "allocate_free_port", lambda: 12345)
|
||||
monkeypatch.setattr(commands_module, "is_port_available", lambda port: True)
|
||||
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: terminate_calls.append(pid))
|
||||
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fixed-master-key")
|
||||
|
||||
|
|
@ -234,7 +236,7 @@ class TestUpCommand:
|
|||
terminate_calls = []
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: fake_process)
|
||||
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||
monkeypatch.setattr(commands_module, "allocate_free_port", lambda: 23456)
|
||||
monkeypatch.setattr(commands_module, "is_port_available", lambda port: True)
|
||||
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: terminate_calls.append(pid))
|
||||
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fixed-master-key")
|
||||
|
||||
|
|
@ -246,6 +248,197 @@ class TestUpCommand:
|
|||
assert not pid_record_path.exists()
|
||||
assert not backup_path.exists()
|
||||
|
||||
def test_up_uses_the_same_port_and_master_key_across_runs(self, monkeypatch, tmp_path):
|
||||
"""The LIT-4607/LIT-4608 regression: a client configured against one session must keep
|
||||
working in the next, so consecutive runs must patch settings with an identical base URL
|
||||
and auth token, and the key must be minted exactly once."""
|
||||
config_path, _log_path, claude_settings_path, _backup_path, _pid_record_path = _patch_paths(
|
||||
monkeypatch, tmp_path
|
||||
)
|
||||
config_path.write_text(yaml.safe_dump({"model_list": []}))
|
||||
claude_settings_path.write_text(json.dumps({"theme": "dark"}))
|
||||
_silence_signal_handling(monkeypatch)
|
||||
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: FakeProcess(pid=42424))
|
||||
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||
monkeypatch.setattr(commands_module, "is_port_available", lambda port: True)
|
||||
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: None)
|
||||
|
||||
mint_calls = []
|
||||
|
||||
def _mint(n):
|
||||
mint_calls.append(n)
|
||||
return f"minted-key-{len(mint_calls)}"
|
||||
|
||||
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", _mint)
|
||||
|
||||
run_index = {"current": 0}
|
||||
captured = {}
|
||||
|
||||
def fake_wait(self, timeout=None):
|
||||
captured[run_index["current"]] = json.loads(claude_settings_path.read_text())["env"]
|
||||
return True
|
||||
|
||||
monkeypatch.setattr("threading.Event.wait", fake_wait)
|
||||
|
||||
first = self.runner.invoke(up)
|
||||
run_index["current"] = 1
|
||||
second = self.runner.invoke(up)
|
||||
|
||||
assert first.exit_code == 0, first.output
|
||||
assert second.exit_code == 0, second.output
|
||||
assert sorted(captured) == [0, 1]
|
||||
assert captured[0]["ANTHROPIC_BASE_URL"] == captured[1]["ANTHROPIC_BASE_URL"]
|
||||
assert captured[0]["ANTHROPIC_AUTH_TOKEN"] == captured[1]["ANTHROPIC_AUTH_TOKEN"]
|
||||
assert mint_calls == [32]
|
||||
|
||||
def test_up_reuses_a_master_key_already_persisted_in_the_config(self, monkeypatch, tmp_path):
|
||||
config_path, _log_path, claude_settings_path, _backup_path, _pid_record_path = _patch_paths(
|
||||
monkeypatch, tmp_path
|
||||
)
|
||||
original_config = yaml.safe_dump({"model_list": [], "general_settings": {"master_key": "persisted-key"}})
|
||||
config_path.write_text(original_config)
|
||||
claude_settings_path.write_text(json.dumps({"theme": "dark"}))
|
||||
_silence_signal_handling(monkeypatch)
|
||||
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: FakeProcess(pid=31313))
|
||||
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||
monkeypatch.setattr(commands_module, "is_port_available", lambda port: True)
|
||||
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: None)
|
||||
|
||||
def _fail_mint(n):
|
||||
raise AssertionError("a persisted master key must be reused, never re-minted")
|
||||
|
||||
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", _fail_mint)
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_wait(self, timeout=None):
|
||||
captured["env"] = json.loads(claude_settings_path.read_text())["env"]
|
||||
captured["config_text"] = config_path.read_text()
|
||||
return True
|
||||
|
||||
monkeypatch.setattr("threading.Event.wait", fake_wait)
|
||||
|
||||
result = self.runner.invoke(up)
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert captured["env"]["ANTHROPIC_AUTH_TOKEN"] == "persisted-key"
|
||||
assert captured["config_text"] == original_config
|
||||
|
||||
def test_up_mints_a_fresh_key_when_the_persisted_master_key_is_blank(self, monkeypatch, tmp_path):
|
||||
config_path, _log_path, claude_settings_path, _backup_path, _pid_record_path = _patch_paths(
|
||||
monkeypatch, tmp_path
|
||||
)
|
||||
config_path.write_text(yaml.safe_dump({"model_list": [], "general_settings": {"master_key": " "}}))
|
||||
claude_settings_path.write_text(json.dumps({"theme": "dark"}))
|
||||
_silence_signal_handling(monkeypatch)
|
||||
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", lambda *a, **k: FakeProcess(pid=21212))
|
||||
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||
monkeypatch.setattr(commands_module, "is_port_available", lambda port: True)
|
||||
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: None)
|
||||
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fresh-minted-key")
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_wait(self, timeout=None):
|
||||
captured["env"] = json.loads(claude_settings_path.read_text())["env"]
|
||||
return True
|
||||
|
||||
monkeypatch.setattr("threading.Event.wait", fake_wait)
|
||||
|
||||
result = self.runner.invoke(up)
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert captured["env"]["ANTHROPIC_AUTH_TOKEN"] == "fresh-minted-key"
|
||||
written_config = yaml.safe_load(config_path.read_text())
|
||||
assert written_config["general_settings"]["master_key"] == "fresh-minted-key"
|
||||
|
||||
def test_port_override_reaches_settings_launch_and_pid_record(self, monkeypatch, tmp_path):
|
||||
"""A --port override must flow to every consumer of the port; a hardcoded default in any
|
||||
one of them would leave the patched settings pointing somewhere the proxy is not."""
|
||||
config_path, _log_path, claude_settings_path, _backup_path, pid_record_path = _patch_paths(
|
||||
monkeypatch, tmp_path
|
||||
)
|
||||
config_path.write_text(yaml.safe_dump({"model_list": []}))
|
||||
claude_settings_path.write_text(json.dumps({"theme": "dark"}))
|
||||
_silence_signal_handling(monkeypatch)
|
||||
|
||||
launched_ports = []
|
||||
|
||||
def _fake_launch(config, port, log):
|
||||
launched_ports.append(port)
|
||||
return FakeProcess(pid=61616)
|
||||
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", _fake_launch)
|
||||
monkeypatch.setattr(commands_module, "poll_liveliness", lambda *a, **k: None)
|
||||
monkeypatch.setattr(commands_module, "is_port_available", lambda port: True)
|
||||
monkeypatch.setattr(commands_module, "terminate", lambda pid, **k: None)
|
||||
monkeypatch.setattr(commands_module.secrets, "token_urlsafe", lambda n: "fixed-master-key")
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_wait(self, timeout=None):
|
||||
captured["env"] = json.loads(claude_settings_path.read_text())["env"]
|
||||
captured["pid_record"] = json.loads(pid_record_path.read_text())
|
||||
return True
|
||||
|
||||
monkeypatch.setattr("threading.Event.wait", fake_wait)
|
||||
|
||||
result = self.runner.invoke(up, ["--port", "6111"])
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert captured["env"]["ANTHROPIC_BASE_URL"] == "http://127.0.0.1:6111"
|
||||
assert launched_ports == [6111]
|
||||
assert captured["pid_record"]["port"] == 6111
|
||||
|
||||
def test_up_rejects_port_4000_which_the_child_proxy_rebinds_unpredictably(self, monkeypatch, tmp_path):
|
||||
"""proxy_cli special-cases a busy port 4000 by silently rebinding to a random port,
|
||||
which would desync base_url from the child; up must refuse 4000 outright."""
|
||||
config_path, _log_path, _settings_path, backup_path, _pid_record_path = _patch_paths(monkeypatch, tmp_path)
|
||||
config_path.write_text(yaml.safe_dump({"model_list": []}))
|
||||
|
||||
def _fail_launch(*args, **kwargs):
|
||||
raise AssertionError("launch_proxy must not run for port 4000")
|
||||
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", _fail_launch)
|
||||
|
||||
result = self.runner.invoke(up, ["--port", "4000"])
|
||||
|
||||
assert result.exit_code != 0
|
||||
assert "4000" in result.output
|
||||
assert not backup_path.exists()
|
||||
|
||||
def test_up_refuses_when_the_port_is_busy_without_touching_any_state(self, monkeypatch, tmp_path):
|
||||
"""A busy port must fail loudly before anything is minted, launched, or patched --
|
||||
never silently move to another port (the pre-fix behavior this ticket removes)."""
|
||||
config_path, _log_path, claude_settings_path, backup_path, _pid_record_path = _patch_paths(
|
||||
monkeypatch, tmp_path
|
||||
)
|
||||
original_config = yaml.safe_dump({"model_list": []})
|
||||
config_path.write_text(original_config)
|
||||
claude_settings_path.write_text(json.dumps({"theme": "dark"}))
|
||||
|
||||
def _fail_launch(*args, **kwargs):
|
||||
raise AssertionError("launch_proxy must not run when the port is busy")
|
||||
|
||||
monkeypatch.setattr(commands_module, "launch_proxy", _fail_launch)
|
||||
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.bind(("127.0.0.1", 0))
|
||||
sock.listen(1)
|
||||
busy_port = sock.getsockname()[1]
|
||||
result = self.runner.invoke(up, ["--port", str(busy_port)])
|
||||
|
||||
assert result.exit_code != 0
|
||||
assert str(busy_port) in result.output
|
||||
assert "lite autoroute down" in result.output
|
||||
assert "--port" in result.output
|
||||
assert config_path.read_text() == original_config
|
||||
assert not backup_path.exists()
|
||||
assert json.loads(claude_settings_path.read_text()) == {"theme": "dark"}
|
||||
|
||||
|
||||
class TestDownCommand:
|
||||
def setup_method(self):
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@ from litellm.proxy.client.cli.commands.autoroute.config import (
|
|||
build_generated_proxy_config,
|
||||
chat_models,
|
||||
embedding_models,
|
||||
master_key_from_config,
|
||||
parse_discovered_models,
|
||||
validate_config,
|
||||
)
|
||||
|
|
@ -206,3 +207,24 @@ class TestValidateConfig:
|
|||
config = _base_config(semantic_matching=SemanticMatching(embedding_model="unknown-embedding"))
|
||||
with pytest.raises(ConfigGenerationError, match="unknown-embedding"):
|
||||
validate_config(config, DISCOVERED)
|
||||
|
||||
|
||||
class TestMasterKeyFromConfig:
|
||||
def test_returns_a_persisted_key_verbatim(self):
|
||||
assert master_key_from_config({"general_settings": {"master_key": " sk-abc "}}) == " sk-abc "
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"config",
|
||||
[
|
||||
{},
|
||||
{"general_settings": None},
|
||||
{"general_settings": "not-a-dict"},
|
||||
{"general_settings": {}},
|
||||
{"general_settings": {"master_key": None}},
|
||||
{"general_settings": {"master_key": 123}},
|
||||
{"general_settings": {"master_key": ""}},
|
||||
{"general_settings": {"master_key": " "}},
|
||||
],
|
||||
)
|
||||
def test_returns_none_when_absent_or_unusable(self, config):
|
||||
assert master_key_from_config(config) is None
|
||||
|
|
|
|||
|
|
@ -10,8 +10,8 @@ from litellm.proxy.client.cli.commands.autoroute.process import (
|
|||
PidRecord,
|
||||
ProcessLaunchError,
|
||||
UpError,
|
||||
allocate_free_port,
|
||||
clear_pid_record,
|
||||
is_port_available,
|
||||
is_running,
|
||||
launch_proxy,
|
||||
missing_proxy_runtime_modules,
|
||||
|
|
@ -34,10 +34,19 @@ class FakeResponse:
|
|||
self.status_code = status_code
|
||||
|
||||
|
||||
def test_allocate_free_port_returns_a_bindable_port():
|
||||
port = allocate_free_port()
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.bind(("127.0.0.1", port))
|
||||
class TestIsPortAvailable:
|
||||
def test_true_for_a_free_port(self):
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.bind(("127.0.0.1", 0))
|
||||
free_port = sock.getsockname()[1]
|
||||
assert is_port_available(free_port) is True
|
||||
|
||||
def test_false_while_another_socket_holds_the_port(self):
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.bind(("127.0.0.1", 0))
|
||||
sock.listen(1)
|
||||
held_port = sock.getsockname()[1]
|
||||
assert is_port_available(held_port) is False
|
||||
|
||||
|
||||
class TestLaunchProxy:
|
||||
|
|
|
|||
|
|
@ -130,6 +130,44 @@ class TestRunConfigureWizardHappyPath:
|
|||
assert config_path.exists()
|
||||
assert oct(config_path.stat().st_mode)[-3:] == "600"
|
||||
|
||||
|
||||
class TestRunConfigureWizardMasterKeyCarryForward:
|
||||
def test_rewrite_preserves_a_persisted_master_key(self, tmp_path):
|
||||
"""Reconfiguring must not rotate the key `up` persisted, or every client configured
|
||||
against the running setup breaks the moment the user re-runs the wizard."""
|
||||
(tmp_path / "config.yaml").write_text(
|
||||
yaml.safe_dump({"model_list": [], "general_settings": {"master_key": "persisted-key"}})
|
||||
)
|
||||
|
||||
result, config_path = _run(tmp_path, CHAT_AND_EMBEDDING_GROUPS, _SIMPLE_TIER_PICKS, input_str="n\nn\nn\n")
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
written = yaml.safe_load(config_path.read_text())
|
||||
assert written["general_settings"] == {"master_key": "persisted-key"}
|
||||
assert any(m["model_name"] == "autorouter" for m in written["model_list"])
|
||||
|
||||
def test_fresh_configure_writes_no_general_settings(self, tmp_path):
|
||||
result, config_path = _run(tmp_path, CHAT_AND_EMBEDDING_GROUPS, _SIMPLE_TIER_PICKS, input_str="n\nn\nn\n")
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "general_settings" not in yaml.safe_load(config_path.read_text())
|
||||
|
||||
def test_corrupt_prior_config_does_not_block_reconfigure(self, tmp_path):
|
||||
(tmp_path / "config.yaml").write_text("::: {{{ not yaml")
|
||||
|
||||
result, config_path = _run(tmp_path, CHAT_AND_EMBEDDING_GROUPS, _SIMPLE_TIER_PICKS, input_str="n\nn\nn\n")
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "general_settings" not in yaml.safe_load(config_path.read_text())
|
||||
|
||||
def test_undecodable_prior_config_does_not_block_reconfigure(self, tmp_path):
|
||||
(tmp_path / "config.yaml").write_bytes(b"\xff\xfe\x00 not utf-8")
|
||||
|
||||
result, config_path = _run(tmp_path, CHAT_AND_EMBEDDING_GROUPS, _SIMPLE_TIER_PICKS, input_str="n\nn\nn\n")
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "general_settings" not in yaml.safe_load(config_path.read_text())
|
||||
|
||||
def test_no_embedding_pool_skips_semantic_prompt_entirely(self, tmp_path):
|
||||
result, config_path = _run(tmp_path, CHAT_ONLY_GROUPS, _SIMPLE_TIER_PICKS, input_str="n\nn\n")
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue