The MCP client's HTTP transport needs mcp>=1.24.0 for streamable_http_client,
but a base litellm install declares no mcp constraint and no extra existed to
pin one, so environments carrying an older mcp fail at connect time with
'streamable_http_client is not available. Please install mcp with HTTP
support.', which names no version floor and no installable remedy.
Add a litellm[mcp] extra matching the proxy extra's mcp>=1.28.1,<2.0 and
replace the vague ImportError with one naming the required floor, the
installed mcp version, and the pip commands that fix it.
* fix(ci): stop the mutation report publishing a score it never measured
Run 32475268575 was the first dispatch of this workflow since May. Every setup
step passed and mutmut generated all 48 mutant files, so the suspected
zero-mutants bug is not what stops it. It dies in the stats phase, where mutmut
times the configured test set once up front. That set included
tests/proxy_behavior/management/, a behaviour tier that talks to a real
seeded database, so the run ended having mutated nothing.
Narrow tests_dir to the unit tier that maps to paths_to_mutate. Run 32476663383
proved a Postgres service is not enough on its own: with a schema but no seed
rows the same test fails on a foreign key instead, and a mutation score is only
meaningful against the tests that claim to cover the mutated code.
The second half is the one that matters. With no results at all,
mutation_report.py printed "No surviving mutants, the test suite caught every
mutation" and exited 0, so a run that mutated nothing published a perfect score.
It now separates no survivors from no results, says which it got, and exits 1.
* fix(ci): count mutmut's multi-word verdicts as results
The verdict capture was `\w+`, so it matched only single-word statuses. mutmut's
status_by_exit_code table has four that are not: `no tests`, `not checked`,
`caught by type check` and `check was interrupted by user`. A finished run made
entirely of those parsed as zero results, which is exactly the state this script
now treats as an unfinished run, so it would have failed a run that had in fact
completed.
The regression test asserting `reported == 2` on a three-verdict fixture was
codifying that, and now asserts 3. A second test walks all four multi-word
statuses and checks the report does not call the run unfinished.
Caught by Greptile on #37825.
* fix(ci): keep the saml tests out of the mutmut stats phase
Run 32477695014 got past the database blocker and ran 208 of the configured
tests, then ended on one error: test_saml_sso.py builds an x509 certificate in
a fixture, and inside mutmut's mutants/ sandbox cryptography's hash classes are
imported under a second identity, so .sign() rejects the SHA256 instance with
"Algorithm must be a registered hash algorithm".
That is a property of the sandbox, not of the tests or the code being mutated,
and one erroring test ends the stats phase before a single mutant runs.
* fix(ci): only claim a clean sweep when something was shown to be killed
`mutmut results` skips killed mutants by design, so its silence means either
that everything was killed or that nothing ran. Counting the verdicts it does
print cannot tell those apart, which left the report still able to say the suite
caught every mutation on a run whose mutants were all `no tests` or
`not checked`.
The clean-sweep sentence is now gated on mutmut-cicd-stats.json reporting a
non-zero killed count, which is the only signal that positively distinguishes
the two. Without it the report says so in as many words and main returns 1. A
run with zero kills and a stats file says that too.
The test asserting a non-killed run was not called unfinished was codifying the
same confusion; it is replaced by three that pin each branch.
Caught by Greptile on #37825.
* fix(ci): treat stats that count survivors the report never listed as untrusted
clean_sweep_is_provable passed on any positive kill count, so a stats file
reporting 48 killed and 3 survived, next to a `mutmut results` that listed no
survivors, still published a clean sweep. The two sources contradict each other
there, and neither one is worth believing. It now requires the stats file to
agree that nothing survived, and the report says which disagreement it found.
* fix(ci): refuse a clean sweep while mutants never reached the tests
A run can end with kills, no survivors, and a pile of mutants marked no tests,
skipped, suspicious, timeout or segfault. Those never got put in front of the
suite, so "caught every mutation" says more than the run measured. The verdict
now names which of them it found and withholds the pass, and the status list
those five come from is one constant the summary and the verdict share.
* fix(ci): read anything that is not a kill or a survivor as unresolved
The unresolved statuses were a list of five, so a run ending in a status the
reporter had never met, "check was interrupted by user" among them, still
counted as a clean sweep. The rule is now the other way round: killed, survived
and total are the keys with a meaning here, and every other non-zero count is a
mutant that did not reach the tests, whatever mutmut chose to call it.
The comment above the extra named cryptography as one of the heavy imports a
thin install leaves out. That stopped being true when keyring joined the
extra: on Linux it reaches the Secret Service through secretstorage, which
depends on cryptography.
Three ways the credential commands could mislead or hang.
`lite logout` on a machine that never logged in warned that a credential may
be stranded in a keychain it could not check, and told the user to install
keyring to go clear it. There was nothing there. A missing token file is now
read as the evidence it is, because logout keeps a secret-free file behind
whenever the keychain is left unconfirmed, so a later run can tell a machine
with a credential it cannot reach apart from one that never had a login. That
holds on the LITELLM_CLI_DISABLE_KEYRING path too.
`KeyringDiscardsWrites` was handled on the read and erase paths, which cannot
produce it: the null backend returns None from `get_password` rather than
raising, so only a write ever detects it. It now lives on `SecretWrite` alone
and the unreachable arms are gone.
`keyring.set_password` blocks forever under a HOME with no usable login
keychain, which is what containers, CI images, `sudo -H`, and service accounts
run with, and reads answer normally there so nothing cheaper tells them apart.
`lite login` never touched a keychain before this, so a sign-in that simply
never returns would be a new way for it to fail. Writes are pre-flighted with
a throwaway value on a bounded wait, and a keychain that stays silent falls
back to the token file. The real credential is never the thing handed to a
call that might land long after we stopped waiting.
Saving also stages the token file before the keychain is given anything, since
the file is the half a read-only or full directory refuses. A save that cannot
land now leaves both stores as it found them, which matters most when the
login it failed to replace still works.
lite login used to write the minted cli-session key in cleartext to
~/.litellm/token.json. The secret material (key plus any JWT) now goes
to the OS keychain through the optional keyring package, with the 0600
file kept for non-secret metadata and as the fallback on headless boxes.
Legacy plaintext files keep authenticating and are migrated into the
keychain, then scrubbed, on first read. A secret still on disk always
outranks the keychain entry, so a failed keychain write can never
resurrect a stale key. LITELLM_PROXY_API_KEY and --api-key precedence
is unchanged, lite logout clears both stores and warns when the
keychain will not release the entry, and ~/.litellm is created 0700
(tightened from 0755 where an older CLI left it broader).
LITELLM_CLI_DISABLE_KEYRING=1 forces the file fallback.
enterprise/ and litellm-proxy-extras/ both changed between main and staging, so each gets a PATCH bump. The 1.98.0 line already graduated with v1.98.0-rc.1, so this promotion opens the 1.99.0 line and litellm takes its MINOR bump.
uv.lock re-resolved against the three new versions; the exclude-newer timestamp moves because the lock uses a rolling P3D window
* fix(deps): ship boto3 with the base SDK so bedrock works out of the box
* keep boto3 listed in the proxy extra as well
* scrub ambient AWS env vars in the base SDK bedrock smoke check
Moves the proxy extra's cryptography floor from 48.0.1 to 49.0.0 and widens the
ceiling to <51, then holds the lock at 50.0.0 with a uv override
mlflow caps cryptography at <50 even in its newest release, so publishing a
plain >=50.0.0,<51.0 range would make `pip install "litellm[proxy,mlflow]"`
unresolvable for downstream consumers. Publishing >=49.0.0,<51.0 keeps that
combination installable (it resolves to 49.0.0), while the
override-dependencies entry, which is a uv workspace setting and never reaches
published metadata, keeps our own lock and Docker images on 50.0.0
mlflow only uses PBKDF2HMAC, AESGCM, Fernet and InvalidTag from cryptography;
none of those are affected by the 49 or 50 breaking changes, so overriding its
ceiling is safe in practice
Lock delta is cryptography 48.0.1 -> 50.0.0, the mlflow trio 3.14.0 -> 3.15.0
and msal 1.36.0 -> 1.37.0
cryptography 49 dropped its x86_64 macOS and 32-bit Windows wheels. Linux CI
and the Docker images are unaffected; developers on Intel Macs will build from
source
`import litellm` reaches litellm/integrations/otel/model/config.py via
litellm_core_utils/litellm_logging.py, so pydantic-settings is needed at import
time. It was declared only in the `proxy` extra, which left a plain
`pip install litellm` unimportable on every platform.
Adds tests/base_sdk_tests/check_base_sdk_install.py and a base_sdk_install
CircleCI job that builds the wheel, installs it into a clean venv with no extras,
and smoke-checks the import, a mock completion, a mock embedding, the bundled
pricing metadata and the token counter. The check is stdlib-only on purpose;
installing pytest into that venv would add packaging, pluggy and iniconfig and
could mask the class of undeclared dependency it exists to catch.
Previously the Windows job was the only one installing without extras, so this
class of break was caught by accident rather than by design.
aiohttp 3.14.0 and 3.14.1 re-arm the sock_read timer on a keep-alive
connection after it has already been returned to the idle pool. The stray
timer stamps a SocketTimeoutError on the pooled connection without closing
it, so the pool keeps handing it out and the next request to pick it up
fails instantly on an error left behind by an earlier, unrelated request.
Because a single pool is shared across providers, the failures appear
simultaneously across Vertex AI, Bedrock, Anthropic and OpenAI-compatible
deployments as sub-millisecond "Connection timed out" errors.
uv.lock resolved aiohttp 3.14.1 and the published images install via
`uv sync --frozen`, so every image built from that lock shipped the
regression. The wheel's own metadata declared `aiohttp>=3.10,<4.0`, which
also left pip consumers free to resolve into the same broken window, so
both the runtime floor and the uv constraint move to >=3.14.2.
Upstream fixed this in aio-libs/aiohttp#12954, released in aiohttp 3.14.2;
the lock now resolves 3.14.3. Raising the floor rather than capping below
3.14 keeps the advisories that the existing 3.14.1 floor cleared, so no
osv-scanner ignores are needed. litellm requires Python >=3.10 and aiohttp
3.14.2 requires >=3.10, so no supported interpreter loses support.
Both new tests fail on the previous pins and pass on these.
litellm already supports Google, Microsoft and generic OIDC SSO through
fastapi-sso, which has no SAML support; AuthMethod.SAML existed only as an
unused enum value. This adds real SAML 2.0 single sign-on for the admin UI.
A new SAMLAuthHandler validates signed assertions with the OneLogin
python3-saml toolkit and maps them onto a CustomOpenID, then reuses the
shared post-login path every other provider goes through, so provisioning,
role/team mapping and the UI session JWT are unchanged. Both SP-initiated
and IdP-initiated HTTP-POST flows are supported. SP-initiated logins are
bound to the browser that started them via an HttpOnly state cookie plus a
cached AuthnRequest id, and the ACS rejects any response whose InResponseTo
doesn't match; unsolicited (IdP-initiated) responses cannot be browser-bound
so they are rejected unless SAML_ALLOW_UNSOLICITED=true. Replays are rejected
by a consumed-assertion guard whose lifetime tracks each assertion's
NotOnOrAfter, and both the replay guard and the login-state binding go
through the proxy's shared in-memory + Redis cache for multi-instance
deployments. The ACS honors DISABLE_ADMIN_UI and re-applies the
free-SSO-user Enterprise gate after the assertion is validated, so an
unvalidated POST can no longer drive the billable-user count query.
SAML is configurable from the admin UI SSO settings (IdP metadata URL or
inline XML, SP entity ID, and an allow-unsolicited toggle), which persists
the SAML_* environment variables the handler reads, exactly like the Google,
Microsoft and generic OIDC providers.
python3-saml is kept as an optional saml extra; its xmlsec and lxml wheels
bundle the native libraries so no system packages are required, and the
import is guarded so the proxy still starts without the package with the
SAML routes returning a clear 501.
Resolves LIT-4016
CodSpeed benchmarks the SDK with no IO, so it can't catch regressions that
only appear under real concurrent load through the full proxy stack (auth,
routing, logging, spend, Postgres, Redis). This adds a Locust load test under
tests/e2e/load that drives concurrent POST /chat/completions traffic against a
mock deployment (litellm_params.mock_response), so the measured throughput
reflects proxy overhead rather than a provider's latency, and asserts an
aggregate RPS SLO with a failure-ratio guard. The test is marked load and the
parent conftest sorts load-marked items last so it never perturbs
latency-sensitive suites. Covers reliability.perf.throughput.under_slo.
Remove the python_version < '3.14' environment markers from redisvl,
pypdf, and openapi-core now that all three install and import cleanly
on 3.14. The relock is marker-only: no package version changed for any
Python branch, and the locked versions (redisvl 0.4.1, pypdf 6.13.3,
openapi-core 0.22.0) now serve 3.14 as well. semantic-router and
aurelio-sdk stay gated because every published release caps
python_requires below 3.14
* feat(cli): add `lite up`/`lite down` to ambiently route Claude Code through the proxy
Patches ~/.claude/settings.json in place (env.ANTHROPIC_BASE_URL + apiKeyHelper
via `lite auth print-token`) so any `claude` session started afterward, from
any terminal, routes through the local LiteLLM proxy with no wrapper command
needed, unlike the existing `lite claude` subprocess-exec approach. Backs up
the original file first and restores it on Ctrl-C/SIGTERM, or via `lite down`
after an unclean exit. Cursor is not supported: no equivalent file-based config
to patch.
* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy (#33249)
* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy
Lets a customer try litellm's complexity_router against models they already
have on their existing, unmodified production proxy, with no config.yaml
edits and no new infra. lite autoroute configure discovers accessible
models via /model_group/info and walks through tier assignment (plus
optional LLM classifier / semantic matching / adaptive selection); every
referenced model becomes its own litellm_proxy/<name> deployment forwarding
back to the real proxy with the real key, so every actual call, routed
completions, classifier calls, embedding calls, still lands on their real
proxy. lite autoroute up launches that generated config as an ephemeral
local proxy, patches ~/.claude/settings.json to point Claude Code at it, and
streams routing decisions live; Ctrl-C/SIGTERM (or lite autoroute down
after an unclean exit) restores everything.
Also adds lite model-groups list (a thin CLI wrapper over the existing
ModelGroupsManagementClient), and generalizes up.py's settings-backup/restore
helpers to take explicit paths so this feature can reuse them instead of
duplicating the logic.
Depends on litellm_lite_up_down (#33231) for that generalization.
* feat(cli): allow multiple models per autoroute tier
complexity_router already supports a pool of models per tier (randomly
picked per request; adaptive mode specifically needs a pool to choose
within), but the configure wizard only ever let you assign one. Tiers are
now a tuple of model names; the wizard prompt accepts comma-separated
indices to pick more than one per tier.
* feat(cli): fuzzy model picker and auto-route Claude Code to autorouter
Numbered-index selection didn't scale past a handful of models, so switch
the tier picker to InquirerPy's fzf-style fuzzy search. Also set
ANTHROPIC_DEFAULT_{SONNET,HAIKU,OPUS}_MODEL to "autorouter" in Claude
Code's settings, since Router resolves auto-router deployments by literal
model name with no wildcard support, so a "*" catch-all model_name would
never match real traffic.
* feat(cli): allow installing lite CLI from source via LITELLM_CLI_REF
Lets testers try an unreleased branch's CLI changes with the same
curl-piped installer, instead of waiting for a PyPI release.
* fix(ci): modernize type hints to clear ruff strict-rule budget
* fix(ci): bump httplib2 and setuptools to patched versions
Clears osv-scan findings for PYSEC-2026-3444 and PYSEC-2026-3447.
* fix(cli): write autoroute's secret-bearing files with mode 0600
commands.py wrote config.yaml (embeds the real proxy key) and Claude
Code's settings.json (embeds the ephemeral proxy's master key) with
plain open(), landing at the umask-derived default (commonly 0644)
until a later chmod call caught up. That window, and the missed case
where settings.json already exists (chmod never ran at all there),
left a credential-bearing file readable by another local account.
secure_create() fixes the mode via fchmod on the fd before any
content is written, covering both the brand-new-file and
already-exists cases, and commands.py/wizard.py now route their
sensitive writes through it.
* docs(cli): warn that a stale Claude Code session can leak to a squatted port
lite autoroute up's master key is embedded statically (unlike lite up's
apiKeyHelper, resolved per request), so a Claude Code session still
running after teardown keeps sending it, along with prompt content, to
a now-unbound loopback port that another local account can bind. This
is the same one-time-patch tradeoff lite up already accepts, just with
a static secret instead of a re-resolved one -- document it in the
README's Caveats section and surface it in the teardown message itself.
* fix(cli): address greptile review feedback on autoroute PR
- terminate the ephemeral proxy child process when its health check
fails, instead of leaking an orphaned, unrecoverable process bound
to the port
- replace bare assert isinstance checks (no-ops under python -O) with
click.ClickException in the model-groups list and configure wizard
code paths
- close launch_proxy's log file handle once the child process has
inherited its fd, instead of leaking it
- add build_generated_proxy_config to config.py's __all__
* fix(cli): close TOCTOU window in lite up's settings backup write
write_backup wrote the backup (which can embed the original
apiKeyHelper/settings content) with plain open() + a chmod call after
the fact -- the same permissive-until-corrected window already fixed
for autoroute's config.yaml and Claude settings writes, and missed
entirely when the backup file already exists with broader permissions.
Moves secure_create (atomic-enough 0600 via fchmod before any content
is written) to up.py, the module both lite up and lite autoroute
share, and has autoroute/process.py import it from there instead of
keeping its own copy.
* fix(cli): refuse autoroute up when a stale backup exists from a crash
The pid-record check only catches a still-live duplicate process; a
SIGKILL'd `up` leaves no live pid but does leave AUTOROUTE_BACKUP_PATH
behind. Without this guard, a fresh `up` overwrote that backup with
the currently-patched Claude settings instead of the true originals,
so `down`/Ctrl-C would restore the wrong content permanently. up.py's
`lite up` already guards the analogous case; mirror it here.
* fix(cli): bind the ephemeral autoroute proxy to loopback only
proxy_cli.py defaults --host to 0.0.0.0 when not passed explicitly.
launch_proxy never passed it, so the ephemeral proxy -- despite every
base_url in this module being built from 127.0.0.1 -- was actually
reachable from other hosts on the network, including its
unauthenticated-until-config-lands routes before the master key is
wired in.
* docs(cli): show curl install for the autoroute QA flow
Points readers at scripts/install-cli.sh's curl one-liner instead of
assuming uv/pip is already set up, and documents the LITELLM_CLI_REF
override for trying an unreleased branch or commit.
* fix(cli): surface a clean error on an empty or corrupt autoroute config
A configure run killed between secure_create's O_TRUNC and the write
completing leaves an empty config.yaml on disk. The next up read that
via yaml.safe_load (None) into the generated-config TypeAdapter
uncaught, surfacing a raw pydantic.ValidationError instead of pointing
the user back at `lite autoroute configure`.
* fix(cli): bind lite up's apiKeyHelper to the proxy it was started against
_ensure_fresh_login only checked token freshness, not which proxy the
cached token belonged to, and resolve_api_key_helper built a bare
`lite auth print-token` command with no --base-url. A user logged into
proxy A who ran `up --base-url proxy-b` (or LITELLM_PROXY_URL=proxy-b)
would silently get proxy A's real token wired into Claude Code's
apiKeyHelper; since apiKeyHelper is invoked bare, print-token's
existing origin check never engaged, so proxy B -- attacker-controlled
or not -- received every subsequent request's Authorization header
carrying proxy A's credential.
_ensure_fresh_login now requires the cached token's base_url to match
before treating it as usable, forcing a fresh login for the selected
proxy otherwise. resolve_api_key_helper now takes that base_url and
threads it through as an explicit --base-url, so print-token's
existing (but previously unreachable in the apiKeyHelper flow)
base_url_explicit check actually enforces the match at request time
too.
* fix(cli): surface clean errors instead of raw tracebacks in lite up/down
load_json_or_empty and read_backup both delegate to pydantic's
validate_json, which raises ValidationError on invalid JSON or a
non-object root -- neither up() nor down() caught it, so a corrupt
settings or backup file surfaced an unformatted Python traceback
instead of a clean CLI error. Both now convert to UpError, and down()
(previously uncaught entirely) and up()'s teardown path now handle it.
restore_claude_settings also gained a parent.mkdir guard before
rewriting CLAUDE_SETTINGS_PATH: if ~/.claude/ was removed while `lite
up` was running, the restore would crash before deleting the backup
file, permanently stranding it and breaking every future `lite down`.
* docs(cli): call out env-var auth for autoroute commands
* fix(cli): clean up leaked proxy and surface clean errors in autoroute
Three related gaps, all following an UpError getting raised somewhere
that wasn't catching it yet:
- up() left the just-launched ephemeral proxy running with no pid
record if load_json_or_empty/write_backup/secure_create raised after
the health check passed, mirroring the existing ProcessLaunchError
cleanup for the health-check-failure branch.
- _teardown() didn't catch restore_claude_settings raising UpError
(e.g. a corrupt backup at stop time), which would otherwise escape
to Click as an unhandled error in the normal-exit path, or print
"Error in atexit" in the atexit path. up.py's own _restore_once
handles the identical case the same way.
- read_pid_record let a corrupt PID file surface a raw
pydantic.ValidationError instead of a clean message, and did so in
down(), the command specifically meant for crash recovery. down()
now clears an unreadable pid record and continues cleanup instead of
aborting, since a corrupt pid file must never block the one command
meant to recover from exactly this kind of crash.
* docs(cli): warn against running lite up and lite autoroute up together
A single ddtrace constraint now covers every supported Python version, so this collapses the version split introduced in #33438. Also aligns the build_from_pip image pin and updates the type-only Tracer import to its current module path
* build: drop requires-python upper cap so Python 3.14 resolves to current releases
The <3.14 cap made pip on Python 3.14 fall back to litellm 1.83.7, a
pre-April release whose old auth flow fails with 400s. The cap was added
in d9a460277a because deps lacked 3.14 wheels and uv could not resolve
the 3.14 split; both are fixed now via the existing python_version
markers plus a ddtrace version split (2.x has no cp314 wheels, 3.16+
does). Verified on 3.14.5: uv sync --all-extras installs, litellm and
proxy_server import (rust bridge falls back to pure python), real
provider calls succeed sync/async/streaming, and the core-utils test
suite passes.
* build: cap requires-python at <3.15 and keep ddtrace on one major per python band
Reviewer preference to bound the supported window at the newest tested
minor rather than leaving it open-ended, and Greptile flagged the
ddtrace 3.14+ range spanning two majors; every ddtrace 4.x ships cp314
wheels so the band is now >=4.0,<5.0, matching the single-major
convention of the 2.x band.
Raise the constraint floors for two transitive dependencies so resolution moves them to their latest maintenance releases: httplib2 0.31.2 -> 0.32.0 and setuptools 82.0.1 -> 83.0.0. Both are pulled in only by optional integrations (Google API client, grpc tooling, lunary observability, the nvidia-riva extra), all lower-bound only, so the floors stay inside every requirer's allowed range and a default install is unaffected
Re-apply the maturin build backend (reverting #31470, which had temporarily
restored the pure-Python uv_build backend). maturin packages the Rust bridge
(litellm.rust_bridge._native) into the wheel; the loader already falls back
gracefully when the native module is absent, so pure-Python installs are
unaffected.
The earlier revert was needed because the release pipeline emitted a bare
cp312 linux_x86_64 wheel that PyPI rejects. That is resolved on the pipeline
side: it now branches on the build backend and, for maturin, builds proper
manylinux_2_28 wheels (x86_64 + aarch64) and validates each wheel carries the
native module.
The [tool.maturin] include for litellm/proxy/_experimental/out/** is sdist
coverage for the Admin UI bundle. maturin's include overrides .gitignore for
the sdist but not the wheel; the committed bundle stays tracked and un-ignored,
so it flows into both the wheel (maturin's source walk) and the sdist as-is.
Coordinates with the release pipeline change that builds the Admin UI from
source and gates the built wheel/sdist on the bundle being present; that should
land first so the pipeline can build and verify a maturin UI wheel.
