* fix(anthropic): handle tool_choice type 'none' in messages API
* test(anthropic): add regression test for tool_choice type 'none'
---------
Co-authored-by: BillionClaw <267901332+BillionClaw@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
When reasoning_auto_summary is enabled (via litellm_settings or env var),
automatically set thinking.display="summarized" on native /v1/messages
requests. This ensures thinking content is returned in the response
instead of being omitted (the default on Claude 4.7+).
Only applies when thinking is enabled (type != "disabled").
The existing reasoning_auto_summary flag already handles the
/v1/responses path (summary="detailed") and the chat/completions
adapter path — this extends coverage to the native messages handler.
`get_file_ids_from_messages` and `update_messages_with_model_file_ids`
assume every content block with `type: "file"` has a nested `file` dict in
the OpenAI Chat Completions shape. That assumption is too strong: `type:
"file"` is a public content-block discriminator and several real producers
emit blocks that use it without the OpenAI `file` sub-dict. For example,
LangChain v1's `_normalize_messages` rewrites OpenAI file blocks into
`{"type":"file","id":"...","base64":"...","mime_type":"...","extras":{}}`
before they reach LiteLLM.
`AnthropicConfig.validate_environment` calls both helpers unconditionally
on every Anthropic (and Anthropic-via-Vertex) request, so any such block
raises `KeyError: 'file'` which the Vertex partner layer then wraps as a
`500 InternalServerError` before the LLM is even contacted.
This patch switches both helpers from `c["file"]` to a defensive
`c.get("file")` + dict check. When the block does not match the OpenAI
shape there is no file_id to extract or remap, so we skip it and leave
the block untouched for the downstream provider transformer to handle.
Adds 5 regression tests covering the LangChain v1 shape, the OpenAI
happy path, mixed shapes in one message, `file` set to a non-dict value,
and the remap path for non-OpenAI blocks.
Related to #24503, which proposed raising `BadRequestError` in the same
spots. For these two discovery functions specifically, the skip semantics
is strictly more permissive: well-formed OpenAI blocks still yield their
file_id, and legitimate non-OpenAI blocks stop crashing the request.
* fix(model-info): include reasoning effort support fields in get_model_info
_get_model_info_helper constructs ModelInfoBase explicitly but never
reads supports_xhigh/minimal/none_reasoning_effort from the cost map
JSON. Add the three fields so get_model_info() returns them correctly.
Also add supports_minimal_reasoning_effort to the ModelInfo TypedDict
(xhigh and none were already declared, minimal was missing).
* fix(model-registry): add missing reasoning effort fields for claude 4.6/4.7
Claude Opus 4.7 supports max reasoning effort (above xhigh).
The field was present for Opus 4.6 but missing for all Opus 4.7
entries (base, dated, Bedrock, Vertex AI, Azure AI).
All Claude 4.6/4.7 models (Opus 4.6, Sonnet 4.6, Opus 4.7) support
minimal reasoning effort via adaptive thinking. Add the field to all
provider variants.
* fix(adapter): map output_config.effort to reasoning_effort (#25079)
Anthropic's adaptive thinking (thinking.type="adaptive") and
output_config.effort were silently dropped when translating to
OpenAI format, resulting in no reasoning_effort on the outgoing
request.
Adapter changes (format translation):
- adapters/transformation.py: add "adaptive" branch to
translate_anthropic_thinking_to_reasoning_effort(); pass through
output_config.effort as-is in _translate_thinking_to_openai();
add "output_config" to translatable_anthropic_params
- adapters/handler.py: extract output_config from extra_kwargs into
request_data so it reaches the translation layer
- responses_adapters/transformation.py: add "adaptive" branch and
output_config param to translate_thinking_to_reasoning()
Handler changes (model-aware normalization):
- utils.py: add normalize_reasoning_effort_value() that uses
get_model_info() to map "max" → "xhigh"/"high" and
"minimal" → "minimal"/"low" based on model capabilities
- adapters/handler.py: call normalization before responses routing
- responses_adapters/handler.py: call normalization after translation
Relates to BerriAI/litellm#25079
* test(reasoning-effort): add tests for effort capability fields and normalize logic
Test coverage for:
- get_model_info returning supports_minimal/max_reasoning_effort fields
- JSON registry entries for claude 4.6/4.7 across all providers
- normalize_reasoning_effort_value degradation chains and exception fallback
- Adapter translation of adaptive thinking + output_config.effort
* fix: forward custom_llm_provider to normalize_reasoning_effort_value in responses adapter
* fix(mcp_semantic_tool_filter): match canonical tools that arrive with
a client-side namespace prefix.
`SemanticMCPToolFilter._get_tools_by_names` matched by exact equality
between the canonical name stored in the router
(`<server><MCP_TOOL_PREFIX_SEPARATOR><tool>`) and the name in the
incoming `tools[]` list. MCP clients such as opencode wrap every tool
name with their own additive alias prefix
(`<client_alias>_<canonical>`), so the two never matched, the filter
dropped every tool to zero, and the proxy forwarded `tools: []` with
`tool_choice: auto` — which strict upstream providers reject with a 400.
The fix adds anchored suffix matching with a separator check: the
canonical must form the complete tail of the incoming name and be
preceded by `_` or `-`. Exact matches still win over suffix matches,
incoming tools are returned at most once, and the original tool object
is passed through unchanged so the client-facing name survives for
tool-call round-trips.
Seven unit tests in a new TestGetToolsByNames class cover exact
match, underscore- and dash-prefixed variants, non-separator-anchored
suffixes (which must not match), exact-wins-over-prefixed precedence,
deduplication when two canonicals suffix-match the same incoming tool,
and ordering-follows-router-output.
Fixes#26078
* review: strengthen the suffix-fallback tie-breaker and the
deduplication regression test (Greptile comments on #26117)
- test_same_tool_not_returned_twice now passes two distinct canonicals
("read_file" and "file") that both suffix-match the same incoming
tool, rather than the same canonical twice, so the assertion
actually exercises the used_ids dedup path instead of the
duplicate-input-list path.
- The suffix fallback in _get_tools_by_names now prefers the shortest
incoming name that still qualifies under the separator-anchored
match. In the one-prefix-per-client opencode scenario this is a
no-op, but in multi-namespace configurations the shortest qualifying
name is the least-wrapped one and is the most defensible deterministic
choice, replacing the dict-insertion-order fallback.
- Adds test_suffix_fallback_prefers_shortest_candidate covering the
new tie-breaker directly.
Still 15 tests passing locally (was 14).
* review(#26117): gate suffix-matching on canonical containing MCP_TOOL_PREFIX_SEPARATOR
@krrish-berri-2 flagged a possible collision in the suffix fallback:
a local user function whose name happens to end in a bare canonical
substring (e.g. my_firecrawl_scrape vs canonical firecrawl_scrape)
would be spuriously selected.
Server-registered MCP tools are always emitted as
<server_name><MCP_TOOL_PREFIX_SEPARATOR><tool_name> via
add_server_prefix_to_name, so a canonical without the separator is
not a namespaced MCP tool and does not warrant suffix matching.
Added that guard to _name_matches_canonical with a regression test
(test_does_not_collide_with_local_function_on_unprefixed_canonical)
that reproduces the collision before the fix and is pinned after.
Pre-existing TestGetToolsByNames fixtures that relied on bare
canonicals (get_weather, search, read_file, write/delete/read) were
switched to realistic server-prefixed ones so they continue to
exercise the suffix-fallback path under the new guard. The opencode
scenario (client prefix on already-server-prefixed canonical) is
unchanged.
---------
Co-authored-by: sakenuGOD <sakenuGOD@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
Two independent bugs both masked budget_reset_at from consumers that
needed it:
1. /team/info.team_member_budget_table was typed as LiteLLM_BudgetTable
(the user-settable allowlist), which dropped server-managed fields.
Switched to LiteLLM_BudgetTableFull so budget_reset_at and created_at
are serialized.
2. _clone_team_default_budget_for_member copied the pool's numeric
fields but never set budget_reset_at on the cloned row. With
budget_duration present but no reset timestamp, the reset job never
fires on the member's budget (its query is reset_at <= now, which
never matches NULL). Now computes budget_reset_at from the cloned
budget_duration via get_budget_reset_time so each member's cycle
starts at clone time rather than inheriting the pool's stale reset.
Two changes, both test-only:
- Configure the aiohttp session with TCPConnector(force_close=True) and an
explicit ClientTimeout(total=30, connect=10). Prevents reuse of idle TCP
connections that the proxy/kernel may have closed during the long window
between setup POSTs and the later poll loop, and surfaces a blocked proxy
event loop quickly instead of hanging on aiohttp's 5-minute default.
- In poll_key_spend_until, catch aiohttp.ClientError and asyncio.TimeoutError
around the single /key/info call. A transient transport hiccup now logs and
retries on the next tick instead of failing the entire polling loop.
Addresses the ConnectionTimeoutError observed on the first /key/info call
after the 20 chat completions.
Adds back the per-cycle spend column that was replaced by Total Spend in
331e3f22. Current Cycle Spend reads membership.spend (zeroed on
budget_reset_at) — this is the value enforced against the member's
budget, so admins need it to see whether a member is approaching their
cap for the active window. Total Spend remains for lifetime analytics.
Adds a formatBudgetReset helper (dayjs-based, with validity guard) that
renders the next reset as "today" / "in N days" / "on MMM D, YYYY". The
team budget card now shows the team's reset timestamp and the member-
default reset (when a shared team_member_budget is configured), and the
Members tab gains a Budget Reset column per member.
- Narrow _team_obj_from_token to require non-None team_id so mypy
passes.
- Preserve the no-auth dev-mode contract for deployments with
master_key unset AND no JWT/OAuth2 configured — the gate
short-circuits only in that specific combination. JWT or OAuth2
deployments without master_key still run the centralized authz.
- is_database_connection_error now enumerates data-layer PrismaError
subclasses (DataError, UniqueViolationError, ForeignKeyViolationError,
MissingRequiredValueError, RawQueryError, TableNotFoundError,
RecordNotFoundError) as False, and maps everything else (bare
PrismaError, connectivity subclasses, DB_CONNECTION_ERROR_TYPES) to
True. Known-safe-to-propagate errors don't trigger HA fallback;
unknown / generic PrismaError still falls back to preserve legacy
503 behavior.
- Update test_handle_authentication_error_db_unavailable_connectivity
to include PrismaError in the fallback list.
Multiple paths through _user_api_key_auth_builder returned a
UserAPIKeyAuth without running common_checks(): OAuth2 token validation,
OAuth2 proxy header hook, JWT admin shortcut, master_key path,
pass-through custom headers, the /user/auth route, and the
allow_requests_on_db_unavailable fallback. An operator-configured key
model-access list, max_budget, team_blocked flag, or team model scope
was therefore silently skipped on those paths. The HA-fallback token
was worse: it was a full proxy-admin synthetic, so a DB outage granted
full admin to every caller.
Fix three root causes (VERIA-18):
1. Centralize common_checks in the user_api_key_auth wrapper. The
builder paths no longer call it; the wrapper runs it once after the
builder returns, for every path. Introduces _run_centralized_common_checks
which gathers team/user/project/end_user/global_spend context in
parallel via asyncio.gather. Preserves the existing
custom_auth_run_common_checks opt-out for custom-auth deployments.
2. Narrow is_database_connection_error — drop the blanket PrismaError
catch that routed data-layer errors (UniqueViolationError, etc.)
into the HA fallback. Only real connectivity failures plus the
no_db_connection marker now qualify.
3. DB-unavailable fallback issues an INTERNAL_USER token with user_id
DB_UNAVAILABLE_FALLBACK_USER_ID instead of proxy-admin. An outage
can no longer escalate an anonymous caller.
JWT admin / master_key tokens still grant admin via a synthesized
admin user_object (so non_proxy_admin_allowed_routes_check in
common_checks recognizes them); other common_checks branches
(team_blocked, team_model_access) now apply uniformly.
Brings the Snowflake, S3 Vectors, Vertex AI, and Bedrock URL construction
paths in line with the existing pattern of validating interpolated values
before use.
Consolidate 6 distinct cache-key prefixes (v2-dependencies-,
v1-router-testing-deps-, v1-router-unit-deps-, v1-llm-translation-deps-,
v1-llm-responses-deps-, v3-litellm-uv-deps-, ui-e2e-py-deps-v2-) onto a
single v1-uv-cache-<uv.lock checksum> key shared across all Python jobs.
Cache only ~/.cache/uv (the content-addressed uv download cache,
hash-verified against uv.lock at install time). Drop ./.venv,
~/.local/{bin,lib}, and /home/circleci/.{pyenv,local} from cache paths.
~/.cache/uv is the only path uv sync needs to avoid re-downloading from
PyPI; everything else is rebuilt each run from that verified cache.
Remove partial-prefix restore-keys fallbacks — cache either hits exactly
on the uv.lock hash or rebuilds cleanly.
First run after merge will cold-miss on the new key; subsequent runs
hit the unified cache.
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
The daily queue parameter types on _restore_spend_updates_to_in_memory_queues
were narrowed to specific subtypes (DailyUserSpendTransaction, etc), but
the caller passes Dict[str, BaseDailySpendTransaction] — the return type
of flush_and_get_aggregated_daily_spend_update_transactions. Widen the
parameters to the base type.
Also replace dynamic TypedDict key lookup (which returned object) with
explicit literal-keyed get() calls so mypy can type-narrow each field.
Two model-access gates run per request in `common_checks` and they're
asymmetric: `can_key_call_model` falls back to the key's
`access_group_ids`, but `can_team_access_model` only looks at
`team.models` + `team.access_group_ids`. A key granted a model via its
own access group on a model-restricted team is silently denied at the
team gate.
Wrap `can_team_access_model` in try/except in `common_checks`: on
`team_model_access_denied`, consult a new `_key_access_group_grants_model`
helper that expands `valid_token.access_group_ids` via the existing
`_get_models_from_access_groups` and checks via `_can_object_call_model`.
Re-raise if the key's access groups don't grant the model. Any other
exception propagates unchanged.
Effect: request allowed if `team allows X` OR `key's access group
grants X`, making the two gates symmetric.
Test: add three unit tests for `_key_access_group_grants_model`
covering: group covers model, key has no groups, group resolves but
does not cover model.
Address two Greptile P2 findings on the BYOK OAuth endpoints:
- GET /v1/mcp/oauth/authorize now runs _validate_redirect_uri up front
so a non-loopback redirect_uri is rejected before the HTML form is
rendered. Previously the user typed an API key, submitted, and got a
400 with no form state.
- POST /v1/mcp/oauth/token moves the master_key guard ahead of the
code-consumption and credential-store steps. Without this, a proxy
with master_key unset would burn the code and persist the credential
but return an error — leaving the user with no way to retrieve a
session token without restarting the whole flow.
Replace recursive `_walk` helper with a stack-based traversal so the
recursive_detector CI check passes without adding to the ignore list,
and avoid Python recursion limits on deeply nested payloads.
Made-with: Cursor
store_in_memory_spend_updates_in_redis drained the in-memory queues
into local variables before the rpush pipeline. If rpush raised (cloud
Redis hiccup, timeout, connection blip), those already-drained
transactions were garbage-collected with the scheduler job, silently
losing all spend aggregated during that tick.
Wrap the rpush in try/except. On failure, re-enqueue the aggregated
transactions into their respective in-memory queues so the next
scheduler tick retries.
Add a unit test that seeds real queues, simulates an rpush failure,
and asserts the transactions land back in-memory.
Replace the calibration step (one request + 10-minute poll) with an
independent ground truth computed from response usage via
litellm.cost_per_token. All N requests are made up front, so a single
dropped Redis write no longer kills the test.
Add /health/readiness checks at test start and on poll timeout so the
failure message surfaces proxy state (db, cache) instead of "calibration
timed out".
Set PROXY_BATCH_WRITE_AT=2 in the spend tracking CI job to shorten the
scheduler flush window.
- POST /v1/mcp/oauth/authorize now requires an authenticated UI session
cookie. The authenticated user_id — not the OAuth client_id form
field — is stamped onto the authorization code record (RFC 6749 §2.2:
client_id identifies the client, not the user).
- redirect_uri is restricted to loopback per RFC 8252 §7.3 (localhost
plus any ipaddress.is_loopback IP, covering 127.0.0.0/8 and IPv6
loopback forms).
- POST /v1/mcp/oauth/token enforces exact-match of the redirect_uri and
client_id submitted at /authorize (RFC 6749 §4.1.3).
- /token error responses use the RFC 6749 §5.2 format ({"error":
"<code>"}), and all /token responses set Cache-Control: no-store +
Pragma: no-cache (RFC 6749 §5.1).
Principle: GHA handles work that doesn't need external API keys; CCI
stays for integration tests that hit real API endpoints.
Four CCI jobs moved to new or extended GHA workflows:
1. check_code_and_doc_quality (was 25 runs: ruff + import-safety +
21 code_coverage_tests + 3 documentation_tests + circular-imports).
- The 21 tests/code_coverage_tests/*.py scripts and the 3
tests/documentation_tests/*.py scripts run in the new
.github/workflows/test-code-quality.yml workflow.
- ruff, import-safety, and circular-imports were already run by
.github/workflows/test-linting.yml — no new migration needed.
- The 3 documentation_tests scripts read
docs/my-website/docs/proxy/config_settings.md. Since docs have
moved to BerriAI/litellm-docs, the GHA workflow checks out that
repo and symlinks docs/my-website -> the checkout so the
existing hardcoded paths resolve without touching the scripts.
The stale local docs/my-website/ copy in this repo will be
removed in a separate PR.
2. semgrep (custom-rule SAST against .semgrep/rules).
- New .github/workflows/test-semgrep.yml.
3. installing_litellm_on_python + installing_litellm_on_python_3_13
(pip install compat checks on Python 3.12 and 3.13).
- New .github/workflows/test-install-litellm.yml as a matrix job.
- 3.12 run also verifies litellm_enterprise import; 3.13 run
skips that check (matches previous CCI behavior).
- installing_litellm_on_python_v2_migration_resolver stays in CCI
because it requires a postgres service.
CCI .circleci/config.yml: -112 lines, 4 jobs and their workflow refs
removed.
Restore guardrail spend/UI event_type wiring, request_data on streaming
OUTPUT paths, and centralized match redaction after the upstream revert.
Made-with: Cursor
