MCP tool calling worked on /v1/chat/completions and /v1/responses but not on
/v1/messages. Those are the only two surfaces with an MCP gateway entry point,
so a litellm_proxy MCP reference reached Anthropic verbatim inside tools and the
API rejected the request with "Input tag 'mcp' found using 'type' does not match
any of the expected tags". The playground never surfaced this because it dropped
the reference before sending, and disabled the MCP selector for the endpoint.
Add the third entry point in anthropic_messages_handler, ahead of the provider
branch so it covers the native path and both bridges from one place. The gateway
expands the reference against the caller's own credentials and access control,
which is the whole point of routing it through litellm rather than handing the
url to the provider.
/v1/messages needs Anthropic's own tool shape, so transform_mcp_tool_to_anthropic_tool
joins the OpenAI chat and Responses transforms alongside it. The tool loop speaks
tool_use and tool_result rather than OpenAI tool_calls, and reuses the existing
FakeAnthropicMessagesStreamIterator to re-stream the result, the same pattern the
websearch interception already uses on this route. Argument extraction moves into
the shared extractor: an Anthropic tool_use block carries its arguments under
`input`, and reading only `arguments` failed silently, executing the tool with
every argument dropped.
On the frontend the request builder declared selectedMCPTools and never read it,
so no tools key was ever sent. Wire it through a shared block builder and add the
endpoint to MCP_SUPPORTED_ENDPOINTS, which is what greys the selector out.
Resolves LIT-4517
Resolves LIT-4518
_request_has_cache_control only looked at messages and system, so a client that
marks cache_control on tools alone did not suppress auto-injection. Tool
breakpoints count toward the provider's four-block limit, so three of them plus
the two injected here is five, which Anthropic rejects. Thread tools through
both entry points and treat a client-marked tool as the stand-down signal it
already is for messages and system.
* test(e2e): read datadog log delivery back from the real datadog api (#33604)
* test(e2e): read datadog log delivery back from the real datadog api
* test(e2e): compare datadog-read cost with math.isclose, not bit-equality
The response_cost now round-trips through DataDog's attribute indexing
pipeline, whose float serialization is not guaranteed to preserve the
exact bit pattern the proxy shipped. rel_tol=1e-9 (equal to 9 significant
digits) still fails on any real cost discrepancy while tolerating
representation drift. Addresses the Greptile P2 on this PR.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(e2e): widen the duplicate-settle window to 30s for real DataDog
Against the local sink one poll interval (5s) after the first hit was
enough to catch a same-call duplicate, because both events arrived in the
same flush batch. Against real DataDog, ingestion jitter can make one
call's two events searchable tens of seconds apart, so a 5s settle could
let the LIT-4447 duplicate slip past the exactly-one assertion. The reader
now keeps re-reading for DD_SETTLE_SECONDS (default 30s, env-overridable
via E2E_DD_SETTLE_SECONDS) after the first event appears, returning early
only when a duplicate is already visible - more waiting cannot clear it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(e2e): point UI tests at dashboard service; register complexity router
Stage gateway 404s /ui; the Next.js dashboard is litellm-ui:3000. Drive
playwright against E2E_UI_BASE_URL and wait on login placeholders after
client render. Register complexity-smart-router via /model/new when the
proxy does not already list it so stage matches compose config
* docs(e2e): clarify E2E_UI_BASE_URL should be ALB when ingress splits UI
* docs(e2e): prefer single path-routing host for control plane and UI
CONTROL_PLANE and UI already default to PROXY_BASE_URL; clarify that
stage should set one ALB host rather than three endpoints
* fix(e2e): always capture complexity router model_id for teardown
Split /model/new from the data-plane wait so a propagation timeout still
deletes the control-plane registration (greptile orphan-model concern)
* fix(e2e): click exact Login button so SSO control is not matched
Playwright strict mode matched both Login and Login with SSO
* fix(router): score complexity by difficulty not request length
The LLM classifier prompt treated short wording as SIMPLE, so probes like
"Is P equal to NP?" stayed on the SIMPLE backend even though the classifier
ran. Judge intellectual difficulty so short hard questions route higher
* fix(e2e): open key edit via Key ID and wait for team models
Key Alias text is not the row open control on the virtual keys table;
KeyInfoView opens from the Key ID button in that row. Also wait for a
real team model in the edit Models dropdown so we do not race the async
availableModels fetch that only has All Team Models on first paint
* fix(e2e): keep settled DD events on empty search; bump mcp for OSV
Do not let a transient empty DataDog search wipe events already seen in
the settle window (Greptile P1). Make the logs-search from window
env-overridable via E2E_DD_SEARCH_FROM (Greptile P2). Prefer the mono
Key ID button when opening key edit. Bump mcp 1.26.0 -> 1.28.1 so OSV
clears the three high GHSA findings on the staging PR
* revert: drop mcp lock bump from e2e staging PR
OSV mcp upgrade is unrelated to the e2e fixes; leave the dep pin alone
---------
Co-authored-by: yucheng-berri <yucheng@berri.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A `lite login` token 429'd with "Budget has been exceeded! Max budget:
0.25" even when no budget was configured anywhere. cli_poll_key stamped
the minted CLI session token with litellm.max_ui_session_budget ($0.25)
as a fallback whenever the user and team had no budget of their own. That
cap was designed for the Admin UI "Test Key" chat pane; the CLI reused
the same session-token machinery, so it inherited a playground-sized
budget baked into the encrypted token at login (unchangeable without
re-login), which trips fast under real CLI/agent use.
The cap is also redundant: the token already carries user_id and team_id,
so the real user/team budgets are enforced independently at request time.
Pass max_budget=None so the CLI token is governed only by those real
budgets, and drop the now-dead user/team budget lookups. The UI login
token's guard (get_experimental_ui_login_jwt_auth_token) is untouched.
Grafted from PR #33582 (closing as superseded by this PR): drives
handle_streamable_http_mcp with real MCPServer objects, parametrized over a
stamped client_credentials row and a legacy unstamped M2M-shape row; both must
reach the session manager without the per-user token store being consulted
The preemptive-401 gate for auth_type=oauth2 MCP servers keyed the challenge
on whether an Authorization header was present (not oauth2_headers). Because
the header parser classifies any Authorization bearer as an OAuth token before
the target server is resolved, a LiteLLM virtual key presented as
Authorization: Bearer sk-... suppressed the challenge on a gateway-managed
authorization_code server; the session then opened with no upstream token and
tools/list masked the failure as 200 with an empty tool list. The same gate
also wrongly challenged client_credentials (M2M) servers, which the gateway
authenticates by minting its own token at egress.
The decision is per oauth2 sub-mode, not per header. Gateway-managed modes
never receive a client-supplied upstream token: client_credentials mints at
egress so it is never challenged, and gateway-managed interactive
(authorization_code, non-delegate) is challenged whenever no stored per-user
token exists, regardless of any bearer. Only the delegate/upstream-PKCE mode,
where a present bearer genuinely is the upstream token, keeps keying on the
Authorization header. oauth2_headers itself is left untouched so the
delegate/passthrough egress paths that forward the client bearer are
unchanged.
The openai and azure_openai columns have working credentials in every suite runner, so only the bedrock_mantle column still needs an opt-in flag while the AWS account waits on the Mantle allowlist; COMPAT_GPT_CELLS becomes COMPAT_MANTLE_CELLS. The six tool_use cells now resolve the proxy through claude_code._env like the basic_messaging cells instead of hardcoding LITELLM_PROXY_BASE_URL/LITELLM_PROXY_API_KEY.
* fix(model_armor): add skip_unscannable_attachments to allow reference-only attachments through
* fix(model_armor): wire skip_unscannable_attachments through guardrail config
* fix(model_armor): make max_file_attachments configurable and scan overflow instead of dropping
* fix(model_armor): remove the per-request attachment count cap and scan all attachments
---------
Co-authored-by: yucheng <yucheng@berri.ai>
* feat(e2e): emit structured E2E_RESULT lines for package status history
Pytest progress logs only expose file basenames and collapse multi-test files
into one status-history row. Emit one logfmt E2E_RESULT per finished node with
package, file, outcome, duration_ms, node_id, and covers so Grafana can roll up
by package (stable cardinality) and drill down by node_id in Explore
* test(e2e): drop unit tests from the live e2e tree
tests/e2e is for live proxy suites only. Remove harness, coverage-registry,
and claude_code unit trees so the e2e run does not collect them
* fix(e2e): type E2E_RESULT hook for basedpyright zero-error gate
Protocol-typed covers extraction and pluggy Result typing on the
makereport hook so tests/e2e stays under the e2e basedpyright ceiling
* fix(e2e): drop unused xfailed/xpassed from E2E_RESULT Outcome
We never emit those states; xfail collapses to skipped/passed via pytest
report flags. Keep the literal honest so the dashboard only sees real outcomes
* fix(e2e): strip tests/e2e prefix when deriving E2E_RESULT package
Repo-root pytest nodeids are tests/e2e/<suite>/...; without stripping,
every line would package=tests and status history would be useless
* fix(e2e): use pytest wrapper=True instead of deprecated hookwrapper
pytest 8.1+ deprecates hookwrapper; yield returns the TestReport directly
so we return it for the outer chain and drop pluggy.Result
* fix(e2e): import e2e_result_reporter at module load
Surface a missing module as a collection-time ImportError instead of a
per-test hook failure mid-run
* test(e2e): restore coverage_registry/test_collector.py
Needed to validate registry coverage math and the checked-in cell
denominator; not a live proxy suite
With enable_redis_auth_cache and multiple replicas, /key/update and
/key/delete delete the Redis auth blob and the handling pod's in-memory
entry, but two read-path writers re-published the stale blob from any other
replica's per-pod memory back to Redis with a fresh 60s TTL on every
request: the post-auth re-cache in user_api_key_auth and the spend writeback
in update_cache. Replicas whose in-memory entries expired then re-primed
themselves from the poisoned Redis entry, so key limit and access changes
never took effect fleet-wide while traffic continued, and a deleted key kept
authenticating.
The auth object is now written only by the DB-load paths
(IdentityStore._resolve_key, get_key_object): the post-auth re-cache is
removed outright (even a local-only write could race an invalidation and
resurrect a revoked key on this worker) and spend tracking no longer writes
the auth object back at all; spend is tracked through the spend🔑*
counters. The remaining spend writebacks for user, team, end-user, and tag
objects become local-only so they cannot republish stale management objects
either, with one deliberate exception: the proxy-wide
{litellm_proxy_admin_name}:spend scalar keeps its shared Redis write because
the global max_budget check reads it between authoritative DB reloads, and
it carries no limits or permissions so sharing it cannot resurrect an
invalidated auth blob.
DualCache's redis-to-memory read backfill also ignored default_in_memory_ttl,
pinning backfilled entries for InMemoryCache's 600s default instead of the
configured 60s auth TTL; the backfill now injects the configured default like
every write path already does, so a replica primed from Redis converges
within the auth cache TTL as well.
Consolidates the sibling stale-auth-recache branch; the delete-propagation
case is the duplicate ticket LIT-4350.
Resolves LIT-4219
Team/key router_settings.enable_tag_filtering was stored and echoed by
/team/info but never applied at request time: the per-request override
whitelist in route_llm_request.py dropped it, tag filtering only read the
router-level flag, and UpdateRouterConfig silently discarded the field on
/key/generate and /config/update. Requests from teams with the toggle on
were load balanced across all deployments instead of tag-matched ones.
- add enable_tag_filtering to the router_settings_override whitelist and
strip any client-supplied copy from the request body first, so only the
key/team value reaches the router
- run tag filtering when the request carries enable_tag_filtering=True; a
request-level False cannot disable a router-level True, so per-request
settings can only scope down, never escape the global policy
- add the field to UpdateRouterConfig so key and config update paths stop
dropping it, and to all_litellm_params so it never leaks into provider
request bodies
- allow it through Router.update_settings/get_settings so the global UI
toggle persists across DB config reloads
Resolves LIT-4390
The hook returned early when the semantic filter selected no tools, which
restated a policy that SemanticMCPToolFilter.filter_tools already owns: it
returns the full tool set when nothing matches, so the selection is never
empty. The branch was unreachable, and reachable or not it changed nothing,
since the gateway reads the union of every reference's allowed_tools and
treats an empty union as unset. Its only effect was to suggest the reference
path and the plain tool path resolve a zero-match query differently.
Drop it so a single policy governs both paths, and pin that with a test
covering an unmatched query on each path. Flipping filter_tools to fail
closed now fails the test on both instead of quietly hard-limiting one
surface and not the other.
* fix(langfuse_otel): build per-request OTLP exporter from key/team dynamic Langfuse credentials
Key-scoped langfuse_otel callbacks only injected Authorization headers into the
init-time exporter, so a proxy without global LANGFUSE_* env vars kept its
fallback exporter and never exported traces to Langfuse. Dynamic params now
build a full per-request OTLP config (endpoint from the key's langfuse_host,
otlp_http, basic auth from the key's credentials).
Resolves LIT-3976
* fix(otel): log dynamic config endpoint in span processor debug output
* fix(otel): redact authorization headers in exporter debug logs
* fix(router): honor per-request routing_strategy from key/team router_settings
Key and team router_settings.routing_strategy was stored and shown in the
UI but never forwarded to the shared Router, so the global strategy always
won. Forward it through router_settings_override and resolve it in
_get_routing_context: a validated per-request strategy takes precedence
over routing groups and the top-level strategy, with lazily built cached
selectors for strategies that need one. Unknown or unsupported strategy
values are ignored with a warning instead of failing the request, and
routing_strategy is registered in all_litellm_params so it is stripped
before the provider call.
* fix(router): sweep override selectors on strategy re-init and cover coverage-gate helpers
routing_strategy_init now unregisters cached per-request override
selectors so a later update_settings strategy change cannot leave a
zombie selector receiving callback events. Adds direct tests for the
two new helpers so the router code coverage gate passes.
* docs(team): document mcp_rpm_limit in update_team docstring
The documentation CI job walks management_endpoints and requires every
UpdateTeamRequest field to appear in the update_team docstring;
mcp_rpm_limit was added to the model without a docstring line, failing
the job on unrelated PRs depending on walk order. Regenerates
schema.d.ts since the docstring feeds the OpenAPI spec.
* fix(proxy): coerce default_internal_user_params.max_budget to float on config load
* fix(proxy): log coerced default_internal_user_params and cover absent max_budget in tests
The semantic tool filter replaced each litellm_proxy MCP reference in
data["tools"] with the tools it expanded from that reference. The expansion
defaults to the Responses API tool shape, so a /chat/completions request came
out carrying flat {"type": "function", "name": ...} entries where the provider
transformations expect {"type": "function", "function": {...}}. Anthropic then
raised KeyError: 'function' and Bedrock dropped every MCP tool silently, so the
model answered as if no MCP server were connected.
Replacing the reference also removed the marker the MCP gateway matches on, so
acompletion_with_mcp never ran and tool calls were no longer auto-executed for
require_approval="never", on /responses as well as /chat/completions.
Narrow the reference through allowed_tools instead and leave it in place, so the
gateway still owns expansion and keeps both the per-endpoint tool shape and tool
auto-execution. Expansion already applies any caller-supplied allowed_tools, so
the selection can only narrow a reference further, never widen it.
* fix(proxy_cli): reap orphaned prisma query-engine processes when a worker dies
When the proxy runs multi-worker (uvicorn multiprocess supervisor or the
gunicorn arbiter), a worker that crashes or is force-killed never runs its
in-process atexit cleanup, so its prisma query-engine subprocess reparents
to PID 1 and keeps its database connection pool established forever while
the replacement worker opens a fresh pool. Active DB connections then grow
past database_connection_pool_limit with every worker death.
Run a reaper thread in the supervisor process that marks itself a child
subreaper on Linux, scans for adopted query-engine children whose worker
is gone, and terminates them with SIGTERM escalating to SIGKILL after a
bounded grace period. Engines owned by live workers are children of those
workers, never of the supervisor, so they are structurally out of reach.
Resolves LIT-4449
Fixes https://github.com/BerriAI/litellm/issues/33023
* fix(proxy_cli): address review findings on the query-engine reaper
Make start_query_engine_reaper idempotent, reap simultaneous orphans
under one shared grace period instead of serially, and log when a PID
survives SIGKILL. Also regenerate schema.d.ts for the update_team
docstring line that documents the existing mcp_rpm_limit param (fixes
the walk-order-dependent documentation CI failure) and avoid a cast in
the prctl wrapper
* test(proxy): fix reaper idempotency-test isolation and widen coverage
The daemon-thread startup test now stubs threading.enumerate so a
reaper thread left running by an earlier test in the same xdist worker
cannot satisfy the idempotency guard and skip the code under test. Add
coverage for stat-file truncation, non-numeric ppid, non-child reap,
signal-to-dead-pid, subreaper capability, and reaper-loop resilience
Both enable_anthropic_prompt_caching and anthropic_prompt_caching_ttl are
now read from LITELLM_ENABLE_ANTHROPIC_PROMPT_CACHING and
LITELLM_ANTHROPIC_PROMPT_CACHING_TTL at import, so the flag can be turned on
without a config file. An unsupported ttl falls back to the provider default
rather than reaching the provider verbatim
Anthropic only caches a prompt when the request carries explicit cache_control
breakpoints, unlike OpenAI where prompt caching is automatic and needs no
configuration. Today litellm can inject those breakpoints server-side, but only
when an admin hand-writes cache_control_injection_points into a model's
litellm_params (or router_settings.default_litellm_params). Clients such as
Claude Code and Claude Desktop never set cache_control themselves, and the
admin recipe is easy to miss, so Anthropic traffic through the proxy silently
pays full price on every repeated prefix.
This adds an opt-in litellm_settings flag, enable_anthropic_prompt_caching. When
it is on and the request has no injection points configured and no
client-supplied cache_control, litellm synthesizes a default pair of breakpoints
(the system prompt and the trailing turn) so the stable prefix is cached while
the breakpoint advances with the conversation. It is wired into both surfaces:
/chat/completions seeds the points before the existing prompt-management gate, and
/v1/messages resolves them in maybe_inject_cache_control, so the existing
AnthropicCacheControlHook applies them unchanged and keeps its four-block cap and
its refusal to overwrite client breakpoints.
The default is off, so no existing deployment changes behavior. Injection is
gated to providers that actually consume cache_control markers (anthropic and
bedrock) and to models the cost map flags as supporting prompt caching; note that
supports_prompt_caching alone is not a sufficient gate, since OpenAI, Azure and
Gemini models report it as well but never take cache_control markers. The default
ttl is Anthropic's 5 minute ephemeral cache, with an optional
anthropic_prompt_caching_ttl of "5m" or "1h"; ttl is also added to
ChatCompletionCachedContent, which the bedrock and anthropic transforms already
read at runtime but the type never declared
Resolves LIT-4478
* chore(e2e): establish litellm_e2e_staging integration line
Long-lived berri branch for e2e suite recovery work (LIT-4479 through LIT-4486) before merge to litellm_internal_staging
* test(e2e): remove langfuse_otel logging e2e suite (#33558)
* test(e2e): remove langfuse_otel logging e2e suite
Removes the LIT-4483 dynamic per-team/key/org langfuse_otel logging e2e tests (tests/e2e/logging/test_langfuse_e2e.py, added in #32857). The shared logging_client harness and the langfuse coverage-registry cells are left in place; only the test module is removed. The otel and prometheus logging e2e suites are unaffected.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): drop orphaned langfuse coverage-registry cells
The three logging.langfuse.*.logs_spend P0 cells were only exercised by the deleted langfuse_otel e2e suite. Remove them so the coverage registry has no orphaned rows.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(e2e): log into the react admin ui in the management browser fixture (#33562)
The management ui_page fixture drove the old server-rendered login form: it clicked input[type="submit"] and treated wait_for_url("**/ui/**") as the done signal. /ui/ now serves the react (antd) dashboard whose submit is a <button type="submit">, so the click waited out the full 30s timeout and errored every browser test in the suite. wait_for_url also matched instantly because the login page already lives at /ui/, so on the fast path the fixture navigated before the auth cookie landed and got bounced back to login.
Click the antd submit button and wait for the token cookie loginCall sets on document.cookie, the real post-login signal.
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(e2e): make ui login readiness robust to httpOnly token cookies (#33564)
The login readiness check waited only on document.cookie including token=, which is empty when the token cookie is httpOnly. If the server ever sets it via a Set-Cookie header, the wait would spin to the 30s timeout and silently reproduce the original hang. Also accept the login form detaching (#username gone after the post-login redirect) so readiness holds regardless of how the cookie is delivered.
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Cost values read from deployment model_info can be strings when the
config YAML contains scientific notation with an integer mantissa
(e.g. 1e-05), which YAML 1.2 parsers such as PyYAML 6.x treat as a
string. Comparing that string against the running float aggregate in
_set_model_group_info raised TypeError and broke /model_group/info,
the prometheus remaining-usage callback, and the
x-litellm-response-cost header. Coerce input/output cost values to
float before comparing and storing them.
* feat(logging): add user and team level spend and budget to StandardLoggingPayload metadata
* fix(logging): include user and team budget fields in dummy standard logging payload
* refactor(e2e/claude_code): align proxy env names with the rest of tests/e2e
Every claude_code compat cell used to read its own `LITELLM_PROXY_BASE_URL` and `LITELLM_PROXY_API_KEY` and duplicate the same 12-line "missing env, hard fail" block. The rest of `tests/e2e/` reads `LITELLM_PROXY_URL` and `LITELLM_MASTER_KEY` from `e2e_config.py`, so anyone standing up a live proxy for one suite had to export a second spelling for claude_code, and every cell repeated the same boilerplate.
Centralize the resolution in `claude_code/_env.py`. `resolve_proxy()` prefers the suite-wide `LITELLM_PROXY_URL` / `LITELLM_MASTER_KEY` names and falls back to the legacy pair so existing CI wiring on stage keeps working during the roll-out. `require_proxy(compat_result)` is the one-liner cells call to bind `(base_url, api_key)` or hard-fail with a message that names both spellings.
55 cell files, `_basic_messaging.py`, and the driver's own unit-test fixture now go through the helper. `run_compat.sh` accepts either spelling and normalizes to the primary names before invoking pytest. `cron_vm/run_daily.sh` exports the primary names when launching pytest.
`_pr_gate_unit_tests/test_env_resolution.py` pins the resolution rules so a future edit cannot silently reintroduce the drift: primary names win on tie, legacy names still resolve when primary is unset, mixed URL-primary key-legacy still resolves, empty-string exports are treated as unset, `require_proxy` names both spellings in its error message.
Net diff: 71 files, +370/-1240.
* fix(e2e): anchor claude_code Bash pin at parents[1] so container run collects
`test_bash_tool_restrictions.py` derived `REPO_ROOT = Path(__file__).resolve().parents[4]` and then joined `tests/e2e/claude_code/<feature>`. That works locally, but the stage container mounts tests/e2e/ at /app/e2e/, so parents[4] resolves to filesystem root and the `_bash_cells()` assertion looks for `/tests/e2e/claude_code/tool_use` — a path that doesn't exist. Collection interrupts before any test runs, so the entire e2e suite appears broken.
Fix: `CLAUDE_CODE_DIR = Path(__file__).resolve().parents[1]` resolves to the sibling `claude_code/` dir in either layout, and the `relative_to(REPO_ROOT)` calls become `relative_to(CLAUDE_CODE_DIR)` so test IDs and error messages read the same.
Adds `test_claude_code_dir_anchor_is_layout_independent` as a regression pin: it checks the anchor lands on a directory named `claude_code` that contains this test file, which would fail under the old parents[4] anchor when run from /app/e2e/.
* feat(e2e/claude_code): register compat deployments via /model/new from a session fixture
Every compat cell hardcodes a virtual model name like `claude-sonnet-4-6` or `claude-sonnet-4-6-bedrock-invoke` and hits the proxy expecting it to be routable. On stage those live in the deployed model_list; locally the `docker-config.yaml` under tests/e2e/ only declares one of them, so anything past haiku 400s with `Invalid model name`.
`claude_code/test_config.yaml` is the ground-truth compat matrix config the deployment already uses. `_compat_models.py` loads it, normalizes the yaml keys pydantic would silently drop (vertex_ai_* → vertex_*), and selects the subset whose provider credentials are present in the environment. An autouse session fixture in `conftest.py` POSTs each selected deployment to `/model/new`, blocks until it is servable on the data plane, and tears them all down on session exit. Skips silently when the proxy env is unset so pure-unit runs stay hermetic.
`test_compat_models.py` pins the invariants that keep this safe. Every cell-referenced name must have a yaml entry (drift check catches a cell probing a name the fixture never registered); the yaml has no unused declarations; the fixture registers exactly 15 deployments (3 tiers × 5 provider surfaces); vertex_ai_* yaml keys populate the pydantic body's vertex_* fields (they got silently dropped historically); Azure needs both AZURE_FOUNDRY_* env vars; Bedrock lifts creds from the ambient AWS chain; Vertex needs both the yaml refs AND ambient GCP credentials.
* refactor(e2e/claude_code): inject env + runner instead of monkeypatching
`require_proxy` and `_basic_messaging.run_basic_messaging_cell` now take the env mapping (and the CLI runner) as constructor-style arguments with `os.environ` and `run_claude_models_parallel` as defaults. Tests exercise the branching by passing dicts and callables directly, so `monkeypatch.setenv` and `monkeypatch.setattr(_basic_messaging, "run_claude_models_parallel", ...)` are gone from every unit test in this refactor's blast radius.
`test_env_resolution.py` drops the `monkeypatch.setenv`/`delenv` fixtures and passes `env={...}` dicts to `require_proxy`. Added a new pinned check that a successful resolution leaves `compat_result` untouched, and split the "unset env" test into three explicit shapes (empty, primary-only, legacy-only) so a regression that swaps the precedence rule can no longer hide behind a single monkeypatched fixture.
`test_basic_messaging.py` (driver) replaces the `_install_fake_runner(monkeypatch, ...)` helper with `_make_fake_runner(...)` that returns a `(callable, captured_dict)` pair the test passes in via the helper's new `runner=` kwarg. Also drops the autouse `_proxy_env` fixture in favor of a module-level `_PROXY_ENV` dict each test wires through the helper's new `env=` kwarg. Added a regression pin that a missing-env call hard-fails without ever invoking the runner (so the guard order stays correct).
`test_run_daily_pytest_scrubs_env.py` updates its pin to assert the new suite-wide env spellings (`LITELLM_PROXY_URL` / `LITELLM_MASTER_KEY`) instead of the legacy `LITELLM_PROXY_BASE_URL` / `LITELLM_PROXY_API_KEY` that `run_daily.sh` used to export.
* handwrote rules
With litellm_settings.enable_redis_auth_cache enabled, user_api_key_cache
is Redis-backed. _update_user_db performed a cache lookup with
key=user_id where user_id can be None; the in-memory cache tolerates a
None key but Redis raises redis.exceptions.DataError (Invalid input of
type: NoneType) on every spend update for requests without a user_id.
The looked-up value was never used by any subsequent code, so the lookup
is removed along with the user_api_key_cache parameter it existed for.
Spend updates for users, end users, and the global proxy budget are
unchanged
* fix(cli): force UTF-8 output so emoji don't crash the CLI on Windows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cli): drop dead flush calls flagged by review
* fix(cli): replace non-ASCII CLI output with ASCII so legacy Windows consoles don't crash
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
