Staging moved the heavy entrypoints behind scripts/gate_slot_lock.py, so
`lint` and `check` now only queue for a machine-wide slot and delegate to
`lint-inner` / `check-inner`. Keep that indirection and hang this branch's
changes off the inner targets: the parallel setup sub-make moves to
lint-inner, and check-inner depends on bootstrap-python rather than the
full bootstrap.
test_lint_overlaps_env_sync_with_base_fetch drives the real `lint` target,
so its sandbox now carries gate_slot_lock.py and runs with
LITELLM_GATE_SLOTS=0.
Every TypedDict field must carry a ReadOnly[...] qualifier (PEP 705),
nesting freely with Required/NotRequired/Annotated. Detection covers the
class form (including same-module transitive subclasses) and the
functional form. The 4519 existing violations across litellm/ are
grandfathered via type-discipline-budget.json; suppress deliberate
writable keys with # writable-ok: <reason>.
* fix(scripts): stop type-discipline checker reading Literal strings as forward refs
The checker re-parsed every string constant inside an annotation as a
forward reference, so Literal["list"] was counted as the mutable list
type. Skip Literal subtrees and ratchet the LIT001 ceiling down to the
corrected count.
* fix(proxy): keep lazy openapi snapshot fragments for transitively imported features
generate_snapshot skipped register_fn for any feature module already in
sys.modules, so a module pulled in transitively by an earlier feature
never mounted its routes and its fragment silently vanished on regen
(vector_store_management). Route collection also matched path_prefixes
only, dropping suffix-matched routes from fragments. Register every
feature and collect routes with feat.matches, mirroring the runtime
loader.
* feat(proxy): add GET /v1/indexes to list vector store indexes
/v1/indexes was POST-only, so indexes created through it could never be
viewed again. Add an admin-only list endpoint returning the stored index
rows newest first, fix the stale index_create docstring curl, and
regenerate the lazy openapi snapshot and dashboard schema types.
* chore(proxy): defer lazy openapi snapshot catch-up regen to a follow-up
Reverts _lazy_openapi_snapshot.json and schema.d.ts to the staging
versions. The snapshot was months stale, so regenerating it here buried
the actual change under ten thousand generated lines. A follow-up will
land the regen together with CI enforcement that keeps the snapshot
current. Until then GET /v1/indexes is served but absent from the
dashboard's generated types, which the UI step needs anyway.
* fix(proxy): use Annotated dependency to avoid new B008 violation
`make pre-commit` became `make check` on staging, with a working-tree fallback
when nothing is staged, so this branch's three changes move onto that shape:
- `check` depends on `bootstrap-python`, not `bootstrap`
- the script provisions the dashboard off `ui_prettier_changed` /
`ui_eslint_changed` / `spec_files`, matching the new scope variables, so a
deleted dashboard file still provisions before the lint that inspects it
- the skip-on-failed-provisioning guard rides on the same variables
Staging's rewrite of `lint_dashboard` dropped the traps that keep the eslint
report from outliving a Ctrl-C, so those come back with the merge.
Splitting bootstrap introduced a failure path the old target could not reach:
`make bootstrap-dashboard` can now fail on its own while the run continues.
It set status=1 and fell through into the dashboard lint and gen:api blocks
anyway, which then ran against a stale or absent node_modules and printed a
second failure ("format with npm run format") on top of the real cause. On a
box whose node is below the dashboard's engines floor, that misleading advice
is the last thing on screen and the actual reason has scrolled away.
Skip both node blocks when provisioning fails and say so once. The Python
block shares nothing with the node toolchain, so it still runs.
Threading looked like the big win here and did not hold up. Three problems,
any one of which disqualifies it:
It changes the answer. Partitioning files across threads reorders
order-dependent inferences, so the tree totals move with the width (149330
serial, 149325 threaded). A gate whose whole job is counting violations must
not have its verdict depend on the host.
There is no portable width. Pinning a constant keeps counts comparable but
tunes the gate for one machine. Letting it follow nproc is correct per host,
yet the width is in the cache fingerprint, so every distinct core count gets
its own key and nobody can reuse CI's precomputed base artifact. Everyone
then pays the cold path.
The speed was noise. On four cores, --threads 4 ranged 84-114s against
137-151s serial, a spread that swallows the median I originally quoted, and
on a contributor's laptop width 8 ran 724s against 113s serial, roughly 6x
slower, while making the machine unusable.
Serial restores the original cache key, so the existing base entry and CI
artifact stay valid. The surviving win in this branch is the memoized path
resolution, which is measurable in isolation and host-independent.
Greptile flagged the explanatory comments this branch added as a violation of
the repo convention against new source comments, and it is right.
The Python rationale moves into docstrings on the three memoized helpers, which
is what the rest of these gates already use, and the thread-width constant is
described in the module docstring that documents every other measurement
parameter. The Makefile and shell comments go entirely.
The one constraint a comment was carrying that nothing else did, that the
dashboard is provisioned once rather than once per node block, is now a test
instead: it fails if the provisioning moves inside dashboard_checks and
genapi_checks, where two npm installs would race in the same directory.
The header line listing what a dashboard-staged commit runs is updated rather
than removed, so it does not go stale.
Profiling `make pre-commit` on a staged litellm/*.py change showed basedpyright
was ~90% of it: 137s of a 157s type-check gate, on a run that took ~152s total.
Three changes, all measured:
Run basedpyright across worker threads instead of one. The width is pinned to a
constant rather than following the host's core count, because partitioning files
across threads reorders a few order-dependent inferences: serial, 2-thread and
4-thread passes disagree on a handful of diagnostics, while passes at the same
width are byte-identical run after run. An auto width would make a 16-core
laptop and a 4-core runner report different totals for the same tree, so the
width joins the dependency-group set in the environment fingerprint and any
cache entry or CI artifact recorded at another width is recomputed, never
matched. 137s -> 87s over this tree.
Memoize the path resolution in all three budget gates. Each called
`Path.resolve()`, a filesystem round trip, once per violation rather than once
per file: 149k realpath walks for 2.2k files in the basedpyright gate alone,
6.6s -> 0.9s there, 5.0s -> 2.8s for the ruff strict gate.
Split `bootstrap` into `bootstrap-python` and `bootstrap-dashboard`, and have
`pre-commit` take only the Python half. The dashboard's npm install is seconds
a Python-only commit has no use for, and on a machine whose node predates the
dashboard's engines floor it fails outright, which blocked `make pre-commit`
entirely for changes that never touch the dashboard. The script now tops the
dashboard up itself for the commits that reach it, after the Python block forks
so the install overlaps that lint and before both node blocks fork so two npm
installs cannot race.
Also fixes a temp file the dashboard job leaked on Ctrl-C: bash skips EXIT traps
on an uncaught fatal signal, so the eslint report survived the interrupt.
Measured over 3 reps each, staged litellm/*.py change, 4-core box:
before 178.9 / 148.8 / 152.0s (median 152.0)
after 106.2 / 99.2 / 94.8s (median 99.2)
Cold path, where the base tree has to be measured too, 5m33s -> 3m17s.
A budget rule that graduates into a config's hard-fail select list rightly
leaves the budget file, but the ratchet guard read any disappearance as a
silently raised ceiling. Teach it the pairing between ruff-strict-budget.json
and ruff.toml: a dropped rule is excused only when the paired config's
lint.extend-select (minus lint.ignore) now hard-fails it, so deleting a rule
without graduating it still trips the guard.
Revert the LIT001 build-clause inserts, phrase the LIT002 menu as
'or (if it really must be dynamic) a MappingProxyType wrapping a dict
literal or comprehension', and fold the two freezing-wrapper exemption
sentences into one that names MappingProxyType beside tuple/frozenset.
LIT001/LIT002 and the typing.Dict ban all steered dict-shaped values to
frozen dataclasses or suppression even though the checker already accepts
MappingProxyType as a freezing wrapper; the messages now name it so the
dict-shaped freeze path is actually discoverable at fix time.
Storing a baseline used to prune every other cache entry, so gate runs in
concurrent worktrees kept evicting each other's baselines and forcing full
recomputes: this bit six times across two nights of benchmarking. The store
now writes alongside existing entries and evicts only the oldest beyond
eight, keyed as before by merge-base and environment fingerprint, so
parallel worktrees' baselines simply coexist
prisma resolves its prisma-client-py generator through a plain /bin/sh PATH
lookup, never through the interpreter that ran prisma generate, so the gate's
generate step landed the client in whatever venv the caller had on PATH: the
owned env never received one, every gate run regenerated, the caller's venv
was mutated instead, and any invocation without a venv on PATH (the rewritten
publisher workflow) failed outright
The generate now runs with the target interpreter's bin directory pinned to
the front of the child PATH. The prisma schema joins the environment
fingerprint so clientless counts recorded before this commit can never be
compared against clientful ones, a cold provision announces itself on stderr
instead of sitting silent for two minutes, and the CI gate step reuses the
job's prisma binary cache
The gate previously measured whatever environment the caller happened to
have. Locally that is the fat bootstrap venv (--extra proxy pulls in
fastapi-sso, whose type info flips a reportUnnecessaryIsInstance
diagnostic in ui_sso.py), while CI's publisher venv only has the
proxy-dev and e2e-dev groups, so identical trees measured 866 locally vs
865 in CI and every local gate run breached by a phantom +1
scripts/type_check_gate.py now provisions .venv-typecheck itself: a
frozen uv sync of the canonical proxy-dev and e2e-dev groups, the
interpreter pinned to pyrightconfig.json's pythonVersion, plus the
generated Prisma client. Every measurement pass is pinned to that env
with --pythonpath, because basedpyright auto-detects a .venv in the
project root and that auto-detection beats both PATH order and
VIRTUAL_ENV, so the CLI flag is the only pin that actually works. The
dependency-group set is folded into the environment fingerprint, so
artifacts or caches recorded under a different group set never match
and the gate falls back to computing base counts locally instead of
comparing mismatched environments
The publisher workflow drops its own install and prisma steps and lets
the script build the measurement env, and the node heap for the
full-tree pass drops from 12GB to 8GB (peak RSS measured at 5.4GB)
* feat(pre-commit): save full lint output to a per-worktree log file
* docs(claude): point agents at the pre-commit log instead of rerunning
* fix(pre-commit): warn when the log cannot be created or fully written
The dashboard pins engines node >=24.14.1 with engine-strict, so make
bootstrap dies with EBADENGINE on any shell whose default node is older.
Wrap the npm install in scripts/with_dashboard_node.sh: it execs the
command as-is when node already meets the floor, otherwise activates the
.nvmrc version via nvm or fnm, and fails fast with install instructions
when neither manager exists
The stale-types failure already writes the regenerated schema.d.ts to the
working tree, and staging it cannot introduce a new failure: the file is
listed in .prettierignore and the eslint config ignores, so no lint pass
sees it, and gen:api derives it purely from the Python proxy code, so a
second regeneration is a no-op. The only reason left to re-run is when
other checks also failed, so say exactly that in the script message and
CLAUDE.md instead of prescribing an unconditional re-run.
The 12 GB NODE_OPTIONS setting lived only in the Makefile export and the
CI env line, so any hand-run gate pipeline forgot it and node OOMed at
the ~4 GB default after 80 seconds, with || true feeding the gate empty
output. The gate now spawns basedpyright itself for both the head and
base passes, appends the heap flag last so it wins node's last-flag-wins
resolution while preserving other caller flags, and fails loudly on
crash exit codes instead of reading them as zero errors.
Module-level names in litellm/__init__.py are the SDK's documented config
surface: users assign litellm.api_key and friends directly, and the proxy
rebinds them via setattr from litellm_settings. The package ships py.typed,
so the Final sweep made every such documented assignment a mypy error
("Cannot assign to final name") in downstream codebases. Strip Final from
the module scope of that file, keep it on function locals, and teach LIT010
that the config surface's module scope is exempt so the gate stays green
without suppression comments
uv selects an interpreter before resolving dependencies, so with no
--python request the stock macOS /usr/bin/python3 (3.9.6) satisfies the
unconstrained request and resolution then fails against litellm's
requires-python (>=3.10,<3.15) instead of downloading a managed Python.
Request the requires-python range explicitly in install-cli.sh and
install.sh so uv reuses a compatible system interpreter when present and
downloads a managed one otherwise. The manual-fallback hint in the die
message carries the same flag so it no longer reproduces the failure.
Mirrors the new test-code-quality.yml step locally so a green pre-commit stays predictive: the sub-second checker fires only when tests/e2e Python files are staged, matching the script's staged-file gating for every other block.
* feat(cli): add `lite up`/`lite down` to ambiently route Claude Code through the proxy
Patches ~/.claude/settings.json in place (env.ANTHROPIC_BASE_URL + apiKeyHelper
via `lite auth print-token`) so any `claude` session started afterward, from
any terminal, routes through the local LiteLLM proxy with no wrapper command
needed, unlike the existing `lite claude` subprocess-exec approach. Backs up
the original file first and restores it on Ctrl-C/SIGTERM, or via `lite down`
after an unclean exit. Cursor is not supported: no equivalent file-based config
to patch.
* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy (#33249)
* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy
Lets a customer try litellm's complexity_router against models they already
have on their existing, unmodified production proxy, with no config.yaml
edits and no new infra. lite autoroute configure discovers accessible
models via /model_group/info and walks through tier assignment (plus
optional LLM classifier / semantic matching / adaptive selection); every
referenced model becomes its own litellm_proxy/<name> deployment forwarding
back to the real proxy with the real key, so every actual call, routed
completions, classifier calls, embedding calls, still lands on their real
proxy. lite autoroute up launches that generated config as an ephemeral
local proxy, patches ~/.claude/settings.json to point Claude Code at it, and
streams routing decisions live; Ctrl-C/SIGTERM (or lite autoroute down
after an unclean exit) restores everything.
Also adds lite model-groups list (a thin CLI wrapper over the existing
ModelGroupsManagementClient), and generalizes up.py's settings-backup/restore
helpers to take explicit paths so this feature can reuse them instead of
duplicating the logic.
Depends on litellm_lite_up_down (#33231) for that generalization.
* feat(cli): allow multiple models per autoroute tier
complexity_router already supports a pool of models per tier (randomly
picked per request; adaptive mode specifically needs a pool to choose
within), but the configure wizard only ever let you assign one. Tiers are
now a tuple of model names; the wizard prompt accepts comma-separated
indices to pick more than one per tier.
* feat(cli): fuzzy model picker and auto-route Claude Code to autorouter
Numbered-index selection didn't scale past a handful of models, so switch
the tier picker to InquirerPy's fzf-style fuzzy search. Also set
ANTHROPIC_DEFAULT_{SONNET,HAIKU,OPUS}_MODEL to "autorouter" in Claude
Code's settings, since Router resolves auto-router deployments by literal
model name with no wildcard support, so a "*" catch-all model_name would
never match real traffic.
* feat(cli): allow installing lite CLI from source via LITELLM_CLI_REF
Lets testers try an unreleased branch's CLI changes with the same
curl-piped installer, instead of waiting for a PyPI release.
* fix(ci): modernize type hints to clear ruff strict-rule budget
* fix(ci): bump httplib2 and setuptools to patched versions
Clears osv-scan findings for PYSEC-2026-3444 and PYSEC-2026-3447.
* fix(cli): write autoroute's secret-bearing files with mode 0600
commands.py wrote config.yaml (embeds the real proxy key) and Claude
Code's settings.json (embeds the ephemeral proxy's master key) with
plain open(), landing at the umask-derived default (commonly 0644)
until a later chmod call caught up. That window, and the missed case
where settings.json already exists (chmod never ran at all there),
left a credential-bearing file readable by another local account.
secure_create() fixes the mode via fchmod on the fd before any
content is written, covering both the brand-new-file and
already-exists cases, and commands.py/wizard.py now route their
sensitive writes through it.
* docs(cli): warn that a stale Claude Code session can leak to a squatted port
lite autoroute up's master key is embedded statically (unlike lite up's
apiKeyHelper, resolved per request), so a Claude Code session still
running after teardown keeps sending it, along with prompt content, to
a now-unbound loopback port that another local account can bind. This
is the same one-time-patch tradeoff lite up already accepts, just with
a static secret instead of a re-resolved one -- document it in the
README's Caveats section and surface it in the teardown message itself.
* fix(cli): address greptile review feedback on autoroute PR
- terminate the ephemeral proxy child process when its health check
fails, instead of leaking an orphaned, unrecoverable process bound
to the port
- replace bare assert isinstance checks (no-ops under python -O) with
click.ClickException in the model-groups list and configure wizard
code paths
- close launch_proxy's log file handle once the child process has
inherited its fd, instead of leaking it
- add build_generated_proxy_config to config.py's __all__
* fix(cli): close TOCTOU window in lite up's settings backup write
write_backup wrote the backup (which can embed the original
apiKeyHelper/settings content) with plain open() + a chmod call after
the fact -- the same permissive-until-corrected window already fixed
for autoroute's config.yaml and Claude settings writes, and missed
entirely when the backup file already exists with broader permissions.
Moves secure_create (atomic-enough 0600 via fchmod before any content
is written) to up.py, the module both lite up and lite autoroute
share, and has autoroute/process.py import it from there instead of
keeping its own copy.
* fix(cli): refuse autoroute up when a stale backup exists from a crash
The pid-record check only catches a still-live duplicate process; a
SIGKILL'd `up` leaves no live pid but does leave AUTOROUTE_BACKUP_PATH
behind. Without this guard, a fresh `up` overwrote that backup with
the currently-patched Claude settings instead of the true originals,
so `down`/Ctrl-C would restore the wrong content permanently. up.py's
`lite up` already guards the analogous case; mirror it here.
* fix(cli): bind the ephemeral autoroute proxy to loopback only
proxy_cli.py defaults --host to 0.0.0.0 when not passed explicitly.
launch_proxy never passed it, so the ephemeral proxy -- despite every
base_url in this module being built from 127.0.0.1 -- was actually
reachable from other hosts on the network, including its
unauthenticated-until-config-lands routes before the master key is
wired in.
* docs(cli): show curl install for the autoroute QA flow
Points readers at scripts/install-cli.sh's curl one-liner instead of
assuming uv/pip is already set up, and documents the LITELLM_CLI_REF
override for trying an unreleased branch or commit.
* fix(cli): surface a clean error on an empty or corrupt autoroute config
A configure run killed between secure_create's O_TRUNC and the write
completing leaves an empty config.yaml on disk. The next up read that
via yaml.safe_load (None) into the generated-config TypeAdapter
uncaught, surfacing a raw pydantic.ValidationError instead of pointing
the user back at `lite autoroute configure`.
* fix(cli): bind lite up's apiKeyHelper to the proxy it was started against
_ensure_fresh_login only checked token freshness, not which proxy the
cached token belonged to, and resolve_api_key_helper built a bare
`lite auth print-token` command with no --base-url. A user logged into
proxy A who ran `up --base-url proxy-b` (or LITELLM_PROXY_URL=proxy-b)
would silently get proxy A's real token wired into Claude Code's
apiKeyHelper; since apiKeyHelper is invoked bare, print-token's
existing origin check never engaged, so proxy B -- attacker-controlled
or not -- received every subsequent request's Authorization header
carrying proxy A's credential.
_ensure_fresh_login now requires the cached token's base_url to match
before treating it as usable, forcing a fresh login for the selected
proxy otherwise. resolve_api_key_helper now takes that base_url and
threads it through as an explicit --base-url, so print-token's
existing (but previously unreachable in the apiKeyHelper flow)
base_url_explicit check actually enforces the match at request time
too.
* fix(cli): surface clean errors instead of raw tracebacks in lite up/down
load_json_or_empty and read_backup both delegate to pydantic's
validate_json, which raises ValidationError on invalid JSON or a
non-object root -- neither up() nor down() caught it, so a corrupt
settings or backup file surfaced an unformatted Python traceback
instead of a clean CLI error. Both now convert to UpError, and down()
(previously uncaught entirely) and up()'s teardown path now handle it.
restore_claude_settings also gained a parent.mkdir guard before
rewriting CLAUDE_SETTINGS_PATH: if ~/.claude/ was removed while `lite
up` was running, the restore would crash before deleting the backup
file, permanently stranding it and breaking every future `lite down`.
* docs(cli): call out env-var auth for autoroute commands
* fix(cli): clean up leaked proxy and surface clean errors in autoroute
Three related gaps, all following an UpError getting raised somewhere
that wasn't catching it yet:
- up() left the just-launched ephemeral proxy running with no pid
record if load_json_or_empty/write_backup/secure_create raised after
the health check passed, mirroring the existing ProcessLaunchError
cleanup for the health-check-failure branch.
- _teardown() didn't catch restore_claude_settings raising UpError
(e.g. a corrupt backup at stop time), which would otherwise escape
to Click as an unhandled error in the normal-exit path, or print
"Error in atexit" in the atexit path. up.py's own _restore_once
handles the identical case the same way.
- read_pid_record let a corrupt PID file surface a raw
pydantic.ValidationError instead of a clean message, and did so in
down(), the command specifically meant for crash recovery. down()
now clears an unreadable pid record and continues cleanup instead of
aborting, since a corrupt pid file must never block the one command
meant to recover from exactly this kind of crash.
* docs(cli): warn against running lite up and lite autoroute up together
The eslint-metrics.json snapshot duplicated the violation counts already
enforced by eslint-budgets.json. Keeping it current added a CI drift check,
a pre-commit regenerate-and-flag step, and a standalone npm run lint:metrics
script, none of which caught anything the budget gate did not, yet all of
which failed noisily whenever the snapshot went stale. This drops the file
and that machinery while leaving eslint-budgets.json as the actual ratchet
gate
