Commit graph

40955 commits

Author SHA1 Message Date
Tin Chi Lo
c84cb8b1e3 refactor(e2e): drop the client_credentials arm; oauth coverage stays on the authorization_code flow 2026-07-16 19:30:45 -07:00
Tin Chi Lo
59fa990c86 test(e2e): cover MCP core happy paths: upstream auth, access control, allowed_tools, aggregate routing 2026-07-16 19:30:45 -07:00
Tin Chi Lo
b92b7cc943 refactor(e2e): drop the unused conformance/prompts/resources foundation
Removes the auth_forwarder key-stamping relay (unused; raw httpx), the stub's
/conformance mount and its fixtures, MCP_STUB_CONFORMANCE_URL, the unused
prompt/resource client ops, and the unclaimed passes_official_conformance
registry cell. None of it was exercised by a test; the suite stays scoped to
the core happy paths against the deterministic stub.
2026-07-16 19:25:52 -07:00
Tin Chi Lo
644d1ec28f test(e2e): a stored per-user token lets the key in Authorization list and call tools
Positive companion to the Authorization challenge-guard: once a user has
authorized (per-user token stored via the x-litellm-api-key dance), a plain
Authorization: Bearer <key> session needs no dance and the gateway serves
tools using the stored token. Seeds the token via the dance, then lists and
calls over an Authorization-key session, and asserts the upstream received the
stored per-user token (not the key) with no key leak. Claims the previously
unclaimed mcp.{list_tools,call_tool}.bearer.succeeds cells. Green on stock.
2026-07-16 17:41:34 -07:00
Tin Chi Lo
4fba2f440f refactor(e2e): make the Authorization-header OAuth test a challenge-guard, not a full dance
The Authorization-header variant cannot complete the interactive round-trip:
once the host finishes the OAuth dance the SDK carries the minted upstream
token in Authorization, evicting the LiteLLM key the gateway needs to resolve
the per-user token. Assert the achievable, correct contract instead: a key in
Authorization on a gateway-managed oauth2 server gets the 401 challenge, not a
masked empty tool list. Full list-and-call stays on the x-litellm-api-key
variant. Red on stock main (masking); flips green with the mode-aware gateway
challenge fix.
2026-07-16 15:39:06 -07:00
Tin Chi Lo
5880135418 test(e2e): assert the server record round-trips in the ingress rejection tests 2026-07-16 13:03:24 -07:00
Tin Chi Lo
9b144f1311 test(e2e): run the list-and-call happy path through the interactive OAuth flow under both ingress headers 2026-07-16 12:49:25 -07:00
Tin Chi Lo
9fb7271ac6 refactor(e2e): name the interactive OAuth test as the list-and-call happy path it exercises 2026-07-16 12:49:25 -07:00
Tin Chi Lo
6c0d405ef4 refactor(e2e): scope the mcp suite to the core happy paths, keeping the conformance-ready foundation 2026-07-16 12:49:25 -07:00
Tin Chi Lo
c1e5ca785b test(e2e): run the official MCP conformance suite through the gateway and cover prompt/resource primitives 2026-07-16 12:49:25 -07:00
Tin Chi Lo
1b42f655b3 test(e2e): cover the interactive authorization_code MCP OAuth flow end to end 2026-07-16 12:49:25 -07:00
Tin Chi Lo
b1442d0398 refactor(e2e): unroll the auth-header matrix into explicitly named per-header tests 2026-07-16 12:49:25 -07:00
Tin Chi Lo
d32974c282 refactor(e2e): build wire headers and mint keys inline in the mcp test bodies 2026-07-16 12:49:25 -07:00
Tin Chi Lo
9ff988d7ef refactor(e2e): split the mcp gateway suite into feature-scoped specs with a parametrized auth-header matrix 2026-07-16 12:49:25 -07:00
Tin Chi Lo
ff3ef9d7e8 refactor(e2e): keep the concurrency test self-contained with local tuning values 2026-07-16 12:49:25 -07:00
Tin Chi Lo
23dffc7453 fix(e2e): use a plain union assignment instead of a PEP 695 type statement 2026-07-16 12:49:25 -07:00
Tin Chi Lo
565a87212c refactor(e2e): inline the concurrency burst in the test via a thread pool 2026-07-16 12:49:25 -07:00
Tin Chi Lo
d74cf13489 fix(e2e): restore uv.lock exclude-newer anchor and gate proxy startup on mcp-stub health 2026-07-16 12:49:25 -07:00
Tin Chi Lo
5edc486928 test(e2e): cover MCP gateway auth headers and per-server max concurrency 2026-07-16 12:49:08 -07:00
Mateo Wang
fba7ac4428
Merge pull request #33222 from BerriAI/litellm_fix_stream_reset_empty_200
fix(streaming): surface upstream connection resets instead of empty 200 streams
2026-07-16 12:43:07 -07:00
yucheng-berri
7fe3dd86a4
feat(logging): add structured budget fields to budget rejection failure logs (#33460) 2026-07-16 12:39:04 -07:00
ryan-crabbe-berri
74ff8d0ff9
fix(ui): navigate to /ui/login/ with trailing slash via hard navigation (#33561)
* fix(ui): navigate to /ui/login/ with trailing slash via hard navigation

Logged-out redirects targeted /ui/login without the trailing slash, so
Starlette's StaticFiles(html=True) mount answered with a 307 whose
absolute Location is built from the scheme the container sees. Behind a
TLS-terminating reverse proxy uvicorn does not trust X-Forwarded-Proto
by default, so the redirect downgraded https to http and stranded users
on an unreachable URL (#33454). The auth guard also used the Next client
router for this navigation, which first requests an RSC payload that the
static export cannot serve, producing 404s before falling back to a full
page load.

Centralize the login URL in getLoginUrl(), which always emits the
trailing slash so no server redirect fires, and use
window.location.replace for the login redirects so no RSC fetch is
attempted.

* test(ui): expect trailing slash in expired-token login redirect
2026-07-16 12:18:55 -07:00
mateo-berri
40aeb33d61 Merge branch 'litellm_internal_staging' into litellm_fix_stream_reset_empty_200 2026-07-16 12:09:35 -07:00
mubashir1osmani
ebdf0bbfd7
chore(e2e): establish litellm_e2e_staging integration line (#33502)
* chore(e2e): establish litellm_e2e_staging integration line

Long-lived berri branch for e2e suite recovery work (LIT-4479 through LIT-4486) before merge to litellm_internal_staging

* test(e2e): remove langfuse_otel logging e2e suite (#33558)

* test(e2e): remove langfuse_otel logging e2e suite

Removes the LIT-4483 dynamic per-team/key/org langfuse_otel logging e2e tests (tests/e2e/logging/test_langfuse_e2e.py, added in #32857). The shared logging_client harness and the langfuse coverage-registry cells are left in place; only the test module is removed. The otel and prometheus logging e2e suites are unaffected.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): drop orphaned langfuse coverage-registry cells

The three logging.langfuse.*.logs_spend P0 cells were only exercised by the deleted langfuse_otel e2e suite. Remove them so the coverage registry has no orphaned rows.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(e2e): log into the react admin ui in the management browser fixture (#33562)

The management ui_page fixture drove the old server-rendered login form: it clicked input[type="submit"] and treated wait_for_url("**/ui/**") as the done signal. /ui/ now serves the react (antd) dashboard whose submit is a <button type="submit">, so the click waited out the full 30s timeout and errored every browser test in the suite. wait_for_url also matched instantly because the login page already lives at /ui/, so on the fast path the fixture navigated before the auth cookie landed and got bounced back to login.

Click the antd submit button and wait for the token cookie loginCall sets on document.cookie, the real post-login signal.

Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(e2e): make ui login readiness robust to httpOnly token cookies (#33564)

The login readiness check waited only on document.cookie including token=, which is empty when the token cookie is httpOnly. If the server ever sets it via a Set-Cookie header, the wait would spin to the 30s timeout and silently reproduce the original hang. Also accept the login form detaching (#username gone after the post-login redirect) so readiness holds regardless of how the cookie is delivered.

Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
2026-07-16 12:09:24 -07:00
Yassin Kortam
c012373e1c
fix(router): cast model_info cost values to float in _set_model_group_info (#33556)
Cost values read from deployment model_info can be strings when the
config YAML contains scientific notation with an integer mantissa
(e.g. 1e-05), which YAML 1.2 parsers such as PyYAML 6.x treat as a
string. Comparing that string against the running float aggregate in
_set_model_group_info raised TypeError and broke /model_group/info,
the prometheus remaining-usage callback, and the
x-litellm-response-cost header. Coerce input/output cost values to
float before comparing and storing them.
2026-07-16 12:05:36 -07:00
yucheng-berri
899ddef219
feat(logging): add user and team level spend and budget to StandardLoggingPayload metadata (#33459)
* feat(logging): add user and team level spend and budget to StandardLoggingPayload metadata

* fix(logging): include user and team budget fields in dummy standard logging payload
2026-07-16 12:01:11 -07:00
tin-berri
db800152c0
Merge pull request #33450 from BerriAI/litellm_mcp_issuer_anchored_discovery
feat(mcp): issuer-anchored OAuth discovery (RFC 8414 §3.3) to close the authorization-server mix-up
2026-07-16 11:55:35 -07:00
Mateo Wang
582907d1ab
Merge pull request #32255 from BerriAI/litellm_fix_openrouter_streaming_usage_cost
fix(streaming): use provider-reported usage cost for OpenRouter streams
2026-07-16 11:18:01 -07:00
tin-berri
748ccde5fd
Merge pull request #33318 from BerriAI/litellm_semantic_filter_lazy_sync
fix(mcp): index authed request-time tools missing from the semantic filter startup index
2026-07-16 11:08:32 -07:00
mubashir1osmani
3f5ed5a9c8
fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433)
* refactor(e2e/claude_code): align proxy env names with the rest of tests/e2e

Every claude_code compat cell used to read its own `LITELLM_PROXY_BASE_URL` and `LITELLM_PROXY_API_KEY` and duplicate the same 12-line "missing env, hard fail" block. The rest of `tests/e2e/` reads `LITELLM_PROXY_URL` and `LITELLM_MASTER_KEY` from `e2e_config.py`, so anyone standing up a live proxy for one suite had to export a second spelling for claude_code, and every cell repeated the same boilerplate.

Centralize the resolution in `claude_code/_env.py`. `resolve_proxy()` prefers the suite-wide `LITELLM_PROXY_URL` / `LITELLM_MASTER_KEY` names and falls back to the legacy pair so existing CI wiring on stage keeps working during the roll-out. `require_proxy(compat_result)` is the one-liner cells call to bind `(base_url, api_key)` or hard-fail with a message that names both spellings.

55 cell files, `_basic_messaging.py`, and the driver's own unit-test fixture now go through the helper. `run_compat.sh` accepts either spelling and normalizes to the primary names before invoking pytest. `cron_vm/run_daily.sh` exports the primary names when launching pytest.

`_pr_gate_unit_tests/test_env_resolution.py` pins the resolution rules so a future edit cannot silently reintroduce the drift: primary names win on tie, legacy names still resolve when primary is unset, mixed URL-primary key-legacy still resolves, empty-string exports are treated as unset, `require_proxy` names both spellings in its error message.

Net diff: 71 files, +370/-1240.

* fix(e2e): anchor claude_code Bash pin at parents[1] so container run collects

`test_bash_tool_restrictions.py` derived `REPO_ROOT = Path(__file__).resolve().parents[4]` and then joined `tests/e2e/claude_code/<feature>`. That works locally, but the stage container mounts tests/e2e/ at /app/e2e/, so parents[4] resolves to filesystem root and the `_bash_cells()` assertion looks for `/tests/e2e/claude_code/tool_use` — a path that doesn't exist. Collection interrupts before any test runs, so the entire e2e suite appears broken.

Fix: `CLAUDE_CODE_DIR = Path(__file__).resolve().parents[1]` resolves to the sibling `claude_code/` dir in either layout, and the `relative_to(REPO_ROOT)` calls become `relative_to(CLAUDE_CODE_DIR)` so test IDs and error messages read the same.

Adds `test_claude_code_dir_anchor_is_layout_independent` as a regression pin: it checks the anchor lands on a directory named `claude_code` that contains this test file, which would fail under the old parents[4] anchor when run from /app/e2e/.

* feat(e2e/claude_code): register compat deployments via /model/new from a session fixture

Every compat cell hardcodes a virtual model name like `claude-sonnet-4-6` or `claude-sonnet-4-6-bedrock-invoke` and hits the proxy expecting it to be routable. On stage those live in the deployed model_list; locally the `docker-config.yaml` under tests/e2e/ only declares one of them, so anything past haiku 400s with `Invalid model name`.

`claude_code/test_config.yaml` is the ground-truth compat matrix config the deployment already uses. `_compat_models.py` loads it, normalizes the yaml keys pydantic would silently drop (vertex_ai_* → vertex_*), and selects the subset whose provider credentials are present in the environment. An autouse session fixture in `conftest.py` POSTs each selected deployment to `/model/new`, blocks until it is servable on the data plane, and tears them all down on session exit. Skips silently when the proxy env is unset so pure-unit runs stay hermetic.

`test_compat_models.py` pins the invariants that keep this safe. Every cell-referenced name must have a yaml entry (drift check catches a cell probing a name the fixture never registered); the yaml has no unused declarations; the fixture registers exactly 15 deployments (3 tiers × 5 provider surfaces); vertex_ai_* yaml keys populate the pydantic body's vertex_* fields (they got silently dropped historically); Azure needs both AZURE_FOUNDRY_* env vars; Bedrock lifts creds from the ambient AWS chain; Vertex needs both the yaml refs AND ambient GCP credentials.

* refactor(e2e/claude_code): inject env + runner instead of monkeypatching

`require_proxy` and `_basic_messaging.run_basic_messaging_cell` now take the env mapping (and the CLI runner) as constructor-style arguments with `os.environ` and `run_claude_models_parallel` as defaults. Tests exercise the branching by passing dicts and callables directly, so `monkeypatch.setenv` and `monkeypatch.setattr(_basic_messaging, "run_claude_models_parallel", ...)` are gone from every unit test in this refactor's blast radius.

`test_env_resolution.py` drops the `monkeypatch.setenv`/`delenv` fixtures and passes `env={...}` dicts to `require_proxy`. Added a new pinned check that a successful resolution leaves `compat_result` untouched, and split the "unset env" test into three explicit shapes (empty, primary-only, legacy-only) so a regression that swaps the precedence rule can no longer hide behind a single monkeypatched fixture.

`test_basic_messaging.py` (driver) replaces the `_install_fake_runner(monkeypatch, ...)` helper with `_make_fake_runner(...)` that returns a `(callable, captured_dict)` pair the test passes in via the helper's new `runner=` kwarg. Also drops the autouse `_proxy_env` fixture in favor of a module-level `_PROXY_ENV` dict each test wires through the helper's new `env=` kwarg. Added a regression pin that a missing-env call hard-fails without ever invoking the runner (so the guard order stays correct).

`test_run_daily_pytest_scrubs_env.py` updates its pin to assert the new suite-wide env spellings (`LITELLM_PROXY_URL` / `LITELLM_MASTER_KEY`) instead of the legacy `LITELLM_PROXY_BASE_URL` / `LITELLM_PROXY_API_KEY` that `run_daily.sh` used to export.

* handwrote rules
2026-07-16 11:05:31 -07:00
Yassin Kortam
45fb9a70b9
feat(helm): add per-component PodDisruptionBudget and topologySpreadConstraints to componentized chart (#33430)
* feat(helm): add per-component PodDisruptionBudget and topologySpreadConstraints to componentized chart

The componentized chart (helm/litellm) had no PodDisruptionBudget template
for the gateway, backend, or ui, so voluntary disruptions (node drains,
Karpenter consolidation) could evict every replica of a component at once.
The legacy chart shipped one out of the box. Deployments also had no way to
configure topologySpreadConstraints, blocking HA spread across AZs.

Adds a shared litellm.pdb helper rendered per component, gated on
<component>.pdb.enabled with minAvailable/maxUnavailable (minAvailable wins,
fallback maxUnavailable: 1), selectors matching each component's
selectorLabels. Adds <component>.topologySpreadConstraints rendered into
each Deployment pod spec. PDBs default to disabled since the default
hpa.minReplicas of 1 with minAvailable: 1 would block drains entirely.

Resolves LIT-4452

* fix(helm): honor explicit 0 in pdb minAvailable/maxUnavailable

A Go-template truthy check treated an explicit 0 (forbid all voluntary
disruptions via maxUnavailable: 0) as unset and silently replaced it with
the fallback maxUnavailable: 1, weakening the configured protection. Treat
a value as set when it is non-nil and non-empty-string instead.
2026-07-16 10:41:59 -07:00
Yassin Kortam
df51cebcd3
fix: remove dead user-cache lookup with None key in spend-update path (#33555)
With litellm_settings.enable_redis_auth_cache enabled, user_api_key_cache
is Redis-backed. _update_user_db performed a cache lookup with
key=user_id where user_id can be None; the in-memory cache tolerates a
None key but Redis raises redis.exceptions.DataError (Invalid input of
type: NoneType) on every spend update for requests without a user_id.

The looked-up value was never used by any subsequent code, so the lookup
is removed along with the user_api_key_cache parameter it existed for.
Spend updates for users, end users, and the global proxy budget are
unchanged
2026-07-16 10:39:53 -07:00
devin-ai-integration[bot]
260d1eae8e
fix(cli): make CLI output ASCII-only so it doesn't crash legacy Windows consoles (#33465)
* fix(cli): force UTF-8 output so emoji don't crash the CLI on Windows

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(cli): drop dead flush calls flagged by review

* fix(cli): replace non-ASCII CLI output with ASCII so legacy Windows consoles don't crash

---------

Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-16 10:35:06 -07:00
yucheng-berri
edc30ea515
test(e2e): datadog log delivery for successful chat, messages, and responses (LIT-4447) (#33415)
* test(e2e): datadog log delivery for successful chat, messages, and responses

Covers logging.datadog.success.exports_metric on all three routes: one
successful non-streaming call must reach the DataDog logs intake as exactly
one log event whose StandardLoggingPayload message carries the model group,
real token counts, and a response cost equal to the x-litellm-response-cost
header of the same response. Delivery is judged at the intake: the compose
stack gains a dd-sink service recording every batch the datadog callback
ships via the DD_BASE_URL testing override, and a typed reader replays it.

Writing these caught a live product bug: /v1/messages double-logs every
success (two byte-identical events per call), filed as LIT-4447; the messages
test tolerates byte-identical duplicates of the one event until it lands,
while a second differing event still fails

* test(e2e): address review findings on the datadog delivery suite

Consolidates the fresh-key first_ok helper into logging_client now that the
otel PR it mirrored has merged (both test files use the shared copy), moves
intake batch parsing into a helper so no path can leave the batch unbound,
and gives the sink's /health endpoint a truthful text/plain content type

* test(e2e): tolerate same-logical-event duplicates by call id, not byte identity

A clean LIT-4447 repro showed the duplicated payload is built twice and can
mint a fresh synthetic completion id per emission, arriving as two separate
intake POSTs with the same litellm_call_id and identical substantive fields.
Byte-identity was therefore a flaky criterion; duplicates now qualify only
when they share the call id, call type, model group, tokens, and cost, and a
second differing event still fails

* test(e2e): assert the scenario strictly; the messages test is the LIT-4447 regression pin

Per review direction the tests now assert exactly what the scenario promises:
exactly one DataDog log event per successful call, on every route. The
/v1/messages test therefore fails on current code against the known
double-log (LIT-4447) and is its regression pin; it goes green when the fix
lands. The duplicate-tolerance machinery is removed

* Simplify docstrings for DataDog log tests

Removed redundant phrasing about cost cross-checking in docstrings.

* Update test_datadog_log_e2e.py
2026-07-16 09:54:07 -07:00
yuneng-jiang
69a491e168
Merge pull request #33343 from BerriAI/litellm_/migrate-simple-tables-ac3786
refactor(ui): migrate vector stores, prompts, and skills tables onto shared DataTable
2026-07-16 07:29:59 -07:00
Yuneng Jiang
ce8ecd4fcf
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/migrate-simple-tables-ac3786
# Conflicts:
#	ui/litellm-dashboard/eslint-suppressions.json
2026-07-16 07:14:30 -07:00
yuneng-jiang
dce1beadca
Merge pull request #33357 from BerriAI/litellm_/gallant-carson-880e37
refactor(ui): migrate policies, deleted keys, deleted teams, budgets, and search tools tables onto shared DataTable
2026-07-16 07:11:15 -07:00
yuneng-jiang
ff06119aa9
Merge pull request #32853 from BerriAI/litellm_/guardrail-monitor-details-fix-8845d6
fix(guardrails): show YAML-defined guardrails in the Guardrail Monitor
2026-07-16 07:10:10 -07:00
Mateo Wang
2f03789927
Merge pull request #33489 from BerriAI/litellm_ocr_model_2512
test(ocr): use mistral-document-ai-2512 in azure_ai OCR tests
2026-07-16 00:52:51 -07:00
devin-ai-integration[bot]
ebc6fdb4c2
fix(cli/anthropic): unblock lite autoroute proxy deps, adaptive thinking, and thinking+signature streaming (#33507) 2026-07-16 00:44:00 -07:00
devin-ai-integration[bot]
5a0e1dd1dd
feat(autoroute): prompt for semantic keywords per tier in configure wizard (#33508) 2026-07-16 07:43:41 +00:00
devin-ai-integration[bot]
bbd52984b1
fix(anthropic): stop 500 on combined thinking+signature streaming chunk (#33505) 2026-07-16 00:24:02 -07:00
Tin Chi Lo
afd7917b8b fix(mcp): separate issuer identity from anchoring so carry-forward keeps endpoints
Making the in-memory issuer reflect a trust-on-first-use discovered value fixed
the registry/row token-identity drift, but it overloaded a single field: the
carry-forward gate keyed on issuer truthiness as a proxy for "endpoints are
anchored to a pinned issuer, fail-closed". A discovered issuer is truthy yet not
anchored, so a resource-rooted server that had learned its issuer would drop its
last-known-good endpoints on a transient discovery blip instead of carrying them
forward.

Anchoring is now a first-class property rather than a proxy. MCPServer carries
issuer_is_anchored, set at both build paths from the single _uses_issuer_anchor
definition (a pinned issuer on a discovery auth type). issuer stays the identity
value used by the token-identity tuple and the serializers; issuer_is_anchored is
the provenance value the carry-forward gate reads to decide fail-closed. The two
properties can no longer be conflated, so a discovered issuer keeps its
resource-rooted endpoints carrying forward while a pinned issuer still fails
closed.

Regression tests pin both directions: a discovered-but-not-anchored server
restores its endpoints on a discovery blip, an anchored server does not, and the
build sets issuer_is_anchored true only when the issuer is pinned
2026-07-15 23:11:37 -07:00
devin-ai-integration[bot]
bf3a058781
feat(complexity_router): enable session_affinity by default (#33500)
Pin a session's first-turn model for the rest of the session by default
instead of reclassifying every turn. Keeps multi-turn sessions on a single
model, preserving provider prompt caches and avoiding cross-model
conversation-history errors (e.g. Anthropic rejecting a thinking block
produced by a different model). Requests without a resolvable session_id are
unaffected. Set session_affinity: false to opt out.

Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-15 21:54:15 -07:00
Krrish Dholakia
cf90445574
feat(cli): add lite up/down to ambiently route Claude Code through the proxy (#33231)
* feat(cli): add `lite up`/`lite down` to ambiently route Claude Code through the proxy

Patches ~/.claude/settings.json in place (env.ANTHROPIC_BASE_URL + apiKeyHelper
via `lite auth print-token`) so any `claude` session started afterward, from
any terminal, routes through the local LiteLLM proxy with no wrapper command
needed, unlike the existing `lite claude` subprocess-exec approach. Backs up
the original file first and restores it on Ctrl-C/SIGTERM, or via `lite down`
after an unclean exit. Cursor is not supported: no equivalent file-based config
to patch.

* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy (#33249)

* feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy

Lets a customer try litellm's complexity_router against models they already
have on their existing, unmodified production proxy, with no config.yaml
edits and no new infra. lite autoroute configure discovers accessible
models via /model_group/info and walks through tier assignment (plus
optional LLM classifier / semantic matching / adaptive selection); every
referenced model becomes its own litellm_proxy/<name> deployment forwarding
back to the real proxy with the real key, so every actual call, routed
completions, classifier calls, embedding calls, still lands on their real
proxy. lite autoroute up launches that generated config as an ephemeral
local proxy, patches ~/.claude/settings.json to point Claude Code at it, and
streams routing decisions live; Ctrl-C/SIGTERM (or lite autoroute down
after an unclean exit) restores everything.

Also adds lite model-groups list (a thin CLI wrapper over the existing
ModelGroupsManagementClient), and generalizes up.py's settings-backup/restore
helpers to take explicit paths so this feature can reuse them instead of
duplicating the logic.

Depends on litellm_lite_up_down (#33231) for that generalization.

* feat(cli): allow multiple models per autoroute tier

complexity_router already supports a pool of models per tier (randomly
picked per request; adaptive mode specifically needs a pool to choose
within), but the configure wizard only ever let you assign one. Tiers are
now a tuple of model names; the wizard prompt accepts comma-separated
indices to pick more than one per tier.

* feat(cli): fuzzy model picker and auto-route Claude Code to autorouter

Numbered-index selection didn't scale past a handful of models, so switch
the tier picker to InquirerPy's fzf-style fuzzy search. Also set
ANTHROPIC_DEFAULT_{SONNET,HAIKU,OPUS}_MODEL to "autorouter" in Claude
Code's settings, since Router resolves auto-router deployments by literal
model name with no wildcard support, so a "*" catch-all model_name would
never match real traffic.

* feat(cli): allow installing lite CLI from source via LITELLM_CLI_REF

Lets testers try an unreleased branch's CLI changes with the same
curl-piped installer, instead of waiting for a PyPI release.

* fix(ci): modernize type hints to clear ruff strict-rule budget

* fix(ci): bump httplib2 and setuptools to patched versions

Clears osv-scan findings for PYSEC-2026-3444 and PYSEC-2026-3447.

* fix(cli): write autoroute's secret-bearing files with mode 0600

commands.py wrote config.yaml (embeds the real proxy key) and Claude
Code's settings.json (embeds the ephemeral proxy's master key) with
plain open(), landing at the umask-derived default (commonly 0644)
until a later chmod call caught up. That window, and the missed case
where settings.json already exists (chmod never ran at all there),
left a credential-bearing file readable by another local account.

secure_create() fixes the mode via fchmod on the fd before any
content is written, covering both the brand-new-file and
already-exists cases, and commands.py/wizard.py now route their
sensitive writes through it.

* docs(cli): warn that a stale Claude Code session can leak to a squatted port

lite autoroute up's master key is embedded statically (unlike lite up's
apiKeyHelper, resolved per request), so a Claude Code session still
running after teardown keeps sending it, along with prompt content, to
a now-unbound loopback port that another local account can bind. This
is the same one-time-patch tradeoff lite up already accepts, just with
a static secret instead of a re-resolved one -- document it in the
README's Caveats section and surface it in the teardown message itself.

* fix(cli): address greptile review feedback on autoroute PR

- terminate the ephemeral proxy child process when its health check
  fails, instead of leaking an orphaned, unrecoverable process bound
  to the port
- replace bare assert isinstance checks (no-ops under python -O) with
  click.ClickException in the model-groups list and configure wizard
  code paths
- close launch_proxy's log file handle once the child process has
  inherited its fd, instead of leaking it
- add build_generated_proxy_config to config.py's __all__

* fix(cli): close TOCTOU window in lite up's settings backup write

write_backup wrote the backup (which can embed the original
apiKeyHelper/settings content) with plain open() + a chmod call after
the fact -- the same permissive-until-corrected window already fixed
for autoroute's config.yaml and Claude settings writes, and missed
entirely when the backup file already exists with broader permissions.

Moves secure_create (atomic-enough 0600 via fchmod before any content
is written) to up.py, the module both lite up and lite autoroute
share, and has autoroute/process.py import it from there instead of
keeping its own copy.

* fix(cli): refuse autoroute up when a stale backup exists from a crash

The pid-record check only catches a still-live duplicate process; a
SIGKILL'd `up` leaves no live pid but does leave AUTOROUTE_BACKUP_PATH
behind. Without this guard, a fresh `up` overwrote that backup with
the currently-patched Claude settings instead of the true originals,
so `down`/Ctrl-C would restore the wrong content permanently. up.py's
`lite up` already guards the analogous case; mirror it here.

* fix(cli): bind the ephemeral autoroute proxy to loopback only

proxy_cli.py defaults --host to 0.0.0.0 when not passed explicitly.
launch_proxy never passed it, so the ephemeral proxy -- despite every
base_url in this module being built from 127.0.0.1 -- was actually
reachable from other hosts on the network, including its
unauthenticated-until-config-lands routes before the master key is
wired in.

* docs(cli): show curl install for the autoroute QA flow

Points readers at scripts/install-cli.sh's curl one-liner instead of
assuming uv/pip is already set up, and documents the LITELLM_CLI_REF
override for trying an unreleased branch or commit.

* fix(cli): surface a clean error on an empty or corrupt autoroute config

A configure run killed between secure_create's O_TRUNC and the write
completing leaves an empty config.yaml on disk. The next up read that
via yaml.safe_load (None) into the generated-config TypeAdapter
uncaught, surfacing a raw pydantic.ValidationError instead of pointing
the user back at `lite autoroute configure`.

* fix(cli): bind lite up's apiKeyHelper to the proxy it was started against

_ensure_fresh_login only checked token freshness, not which proxy the
cached token belonged to, and resolve_api_key_helper built a bare
`lite auth print-token` command with no --base-url. A user logged into
proxy A who ran `up --base-url proxy-b` (or LITELLM_PROXY_URL=proxy-b)
would silently get proxy A's real token wired into Claude Code's
apiKeyHelper; since apiKeyHelper is invoked bare, print-token's
existing origin check never engaged, so proxy B -- attacker-controlled
or not -- received every subsequent request's Authorization header
carrying proxy A's credential.

_ensure_fresh_login now requires the cached token's base_url to match
before treating it as usable, forcing a fresh login for the selected
proxy otherwise. resolve_api_key_helper now takes that base_url and
threads it through as an explicit --base-url, so print-token's
existing (but previously unreachable in the apiKeyHelper flow)
base_url_explicit check actually enforces the match at request time
too.

* fix(cli): surface clean errors instead of raw tracebacks in lite up/down

load_json_or_empty and read_backup both delegate to pydantic's
validate_json, which raises ValidationError on invalid JSON or a
non-object root -- neither up() nor down() caught it, so a corrupt
settings or backup file surfaced an unformatted Python traceback
instead of a clean CLI error. Both now convert to UpError, and down()
(previously uncaught entirely) and up()'s teardown path now handle it.

restore_claude_settings also gained a parent.mkdir guard before
rewriting CLAUDE_SETTINGS_PATH: if ~/.claude/ was removed while `lite
up` was running, the restore would crash before deleting the backup
file, permanently stranding it and breaking every future `lite down`.

* docs(cli): call out env-var auth for autoroute commands

* fix(cli): clean up leaked proxy and surface clean errors in autoroute

Three related gaps, all following an UpError getting raised somewhere
that wasn't catching it yet:

- up() left the just-launched ephemeral proxy running with no pid
  record if load_json_or_empty/write_backup/secure_create raised after
  the health check passed, mirroring the existing ProcessLaunchError
  cleanup for the health-check-failure branch.
- _teardown() didn't catch restore_claude_settings raising UpError
  (e.g. a corrupt backup at stop time), which would otherwise escape
  to Click as an unhandled error in the normal-exit path, or print
  "Error in atexit" in the atexit path. up.py's own _restore_once
  handles the identical case the same way.
- read_pid_record let a corrupt PID file surface a raw
  pydantic.ValidationError instead of a clean message, and did so in
  down(), the command specifically meant for crash recovery. down()
  now clears an unreadable pid record and continues cleanup instead of
  aborting, since a corrupt pid file must never block the one command
  meant to recover from exactly this kind of crash.

* docs(cli): warn against running lite up and lite autoroute up together
2026-07-15 21:46:02 -07:00
Krrish Dholakia
f6516e7be5
fix(ui): stop sending the complexity-router pseudo-model to /health/test_connection (#33498)
* fix(ui): stop sending the complexity-router pseudo-model to /health/test_connection

Test Connection on a saved auto-router model sent the raw
"auto_router/complexity_router" model string to the generic
health-check endpoint, which always failed with "Unmapped LLM
provider" since it's a routing-strategy config, not a real
completion endpoint.

Reuse the per-tier connection test already built for the Add Auto
Router wizard: for complexity-router models, test each configured
tier's underlying model group instead of the router pseudo-model.
Semantic-type auto routers (auto_router_config) have no equivalent
tier-based test yet, so the button is hidden for them instead of
guaranteed to fail.

* fix(ui): address review feedback on auto-router test connection fix

Type the complexity-router config parsing instead of using `any`, use
NotificationsManager.warning instead of fromBackend for the
client-generated "no tiers configured" message, remove comments added
in the previous commit, and also test the deployment's configured
complexity_router_default_model as a fallback target when it isn't
already covered by a configured tier (matches the fallback Router
itself uses for unconfigured tiers).
2026-07-15 21:41:45 -07:00
Mateo Wang
906897bebf
Merge pull request #33473 from BerriAI/litellm_claude_code_passthrough 2026-07-15 20:22:15 -07:00
yuneng-jiang
39c01fe104
Merge pull request #33482 from BerriAI/litellm_/laughing-herschel-d4f735
chore(ui): remove unmounted UsageIndicator and the Hide Usage Indicator flag
2026-07-15 18:28:26 -07:00
yucheng-berri
3cea243116
fix(key management): enforce minimum custom key length and mask short keys in key_name (#33462)
* fix(key management): enforce minimum custom key length and mask short keys in key_name

* fix(key management): validate new_key before assignment and sync generated schema docstrings

* fix(key management): lower minimum custom key length default from 20 to 16
2026-07-15 18:27:38 -07:00
Tin Chi Lo
ad73f3a7a2 fix(mcp): keep issuer provenance consistent when it changes or is discovered
Two lifecycle gaps let the issuer trust anchor drift out of sync with the
endpoints it governs. Changing or clearing a previously pinned issuer left the
authorization_url and token_url that were resolved under the old issuer in the
row, so clearing the anchor could revive stale, possibly untrusted endpoints
instead of re-discovering. And a build that discovered an issuer
trust-on-first-use persisted it to the row while the returned in-memory server
kept the issuer unset, so the registry and the row disagreed and the per-user
OAuth token identity, which includes the issuer, differed between that build and
the next rebuild and forced a spurious re-auth.

update_mcp_server now treats a change to a previously pinned issuer the same as a
url or auth_type change and clears the auth-flow-scoped endpoint fields that were
resolved under it. The trigger fires only when an issuer was already pinned and
is now changed or cleared, so establishing one for the first time, including the
trust-on-first-use discovery write-back, does not wipe the fields it just
resolved.

Both build paths, build_mcp_server_from_table and load_servers_from_config, now
construct the server with effective_issuer = manual_issuer or the discovered
issuer, skipping an origin-fallback guess exactly as the persistence does, so the
in-memory object always reflects what the row will hold.

Regression tests pin each case: clearing and re-pointing a pinned issuer clear
the stale endpoints, a first-time establish preserves the discovered fields, and
a build reflects the discovered issuer while an origin-fallback guess is not
reflected
2026-07-15 18:15:12 -07:00