Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Waiting to run
Extract the BYOK loopback redirect_uri check into a shared
oauth_utils.validate_loopback_redirect_uri helper. Call it in
discoverable_endpoints.authorize_with_server before the client-supplied
redirect_uri is encrypted into the OAuth state.
Without this check, a non-loopback redirect_uri was encoded into the
state parameter and decoded on /callback to 302 the user back to the
attacker's URL with the authorization code attached — an open-redirect
+ code-theft primitive (VERIA-57 root cause B). The /callback handler
is already safe because state is HMAC-signed via encrypt_value_helper,
so validating at /authorize before encoding is sufficient.
Also updates existing tests to use loopback client redirect_uris and
adds regression tests for non-loopback rejection, IPv4 127.0.0.0/8
range acceptance, and full-form IPv6 loopback acceptance.
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
The original check `"gpt-5-chat" not in model` already correctly
classifies all current gpt-5 variants (including gpt-5.3-chat and
gpt-5.1-chat, which do NOT contain the substring "gpt-5-chat"). This
change replaces it with an explicit `startswith("gpt-5-chat")` prefix
test on the provider-prefix-stripped model name.
The new check is functionally equivalent for all existing model names
but makes the classification boundary unambiguous and forward-safe:
future model names that might contain "gpt-5-chat" as an interior
substring won't accidentally be excluded from the GPT-5 reasoning path.
Also moves the new regression test from tests/ root to
tests/test_litellm/llms/openai/ so it is included in `make test-unit`.
get_end_user_object raises litellm.BudgetExceededError internally
when the end-user is over budget. The previous _safe_fetch in the
centralized gate swallowed it and returned None, which caused
common_checks to see end_user_object=None and skip the budget check
entirely — silently bypassing end-user budget enforcement.
Add BudgetExceededError to the re-raise list alongside HTTPException
and ProxyException (reported by Veria AI).
The useEffect that re-fetches logs on sort/page/time changes:
useEffect(() => {
if (hasBackendFilters && accessToken) {
performSearch(filters, currentPage);
}
}, [sortBy, sortOrder, currentPage, startTime, endTime, isCustomDate]);
intentionally omits `filters` and `hasBackendFilters` from its dep array
to avoid double-fetches when a filter is applied. The side-effect is a
stale-closure bug: the effect captures `filters` and `hasBackendFilters`
from the render where its deps last changed, not from the render where
the user selected, e.g., a Key Alias.
Reproduce: set Key Alias → results appear correctly → change page or
sort → the effect fires with the OLD `filters` snapshot (no key_alias)
→ API request is sent without the filter → table shows unfiltered data.
Fix: store the latest `filters` and `hasBackendFilters` in refs that are
kept in sync on every render. The sort/page/time effect reads from the
refs instead of the closure so it always uses the current filter state
without altering the dep array.
Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
When backend filters (e.g. Key Alias) are active on the Request Logs
page, the manual Fetch button called logs.refetch() which re-runs the
main TanStack Query. That query does not carry backend-only filter
params such as key_alias, so the button had two problems:
1. It fired a redundant API request without the active filters.
2. It did not refresh the filtered result set — backendFilteredLogs
stayed frozen at the last debounce-triggered fetch.
Fix: expose refetchWithFilters() from useLogFilterLogic and route the
Fetch button through it when hasBackendFilters is true. This cancels
any in-flight debounce and calls performSearch with the current filter
state, keeping all active filters intact.
Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
* fix(anthropic): handle tool_choice type 'none' in messages API
* test(anthropic): add regression test for tool_choice type 'none'
---------
Co-authored-by: BillionClaw <267901332+BillionClaw@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
When reasoning_auto_summary is enabled (via litellm_settings or env var),
automatically set thinking.display="summarized" on native /v1/messages
requests. This ensures thinking content is returned in the response
instead of being omitted (the default on Claude 4.7+).
Only applies when thinking is enabled (type != "disabled").
The existing reasoning_auto_summary flag already handles the
/v1/responses path (summary="detailed") and the chat/completions
adapter path — this extends coverage to the native messages handler.
`get_file_ids_from_messages` and `update_messages_with_model_file_ids`
assume every content block with `type: "file"` has a nested `file` dict in
the OpenAI Chat Completions shape. That assumption is too strong: `type:
"file"` is a public content-block discriminator and several real producers
emit blocks that use it without the OpenAI `file` sub-dict. For example,
LangChain v1's `_normalize_messages` rewrites OpenAI file blocks into
`{"type":"file","id":"...","base64":"...","mime_type":"...","extras":{}}`
before they reach LiteLLM.
`AnthropicConfig.validate_environment` calls both helpers unconditionally
on every Anthropic (and Anthropic-via-Vertex) request, so any such block
raises `KeyError: 'file'` which the Vertex partner layer then wraps as a
`500 InternalServerError` before the LLM is even contacted.
This patch switches both helpers from `c["file"]` to a defensive
`c.get("file")` + dict check. When the block does not match the OpenAI
shape there is no file_id to extract or remap, so we skip it and leave
the block untouched for the downstream provider transformer to handle.
Adds 5 regression tests covering the LangChain v1 shape, the OpenAI
happy path, mixed shapes in one message, `file` set to a non-dict value,
and the remap path for non-OpenAI blocks.
Related to #24503, which proposed raising `BadRequestError` in the same
spots. For these two discovery functions specifically, the skip semantics
is strictly more permissive: well-formed OpenAI blocks still yield their
file_id, and legitimate non-OpenAI blocks stop crashing the request.
* fix(model-info): include reasoning effort support fields in get_model_info
_get_model_info_helper constructs ModelInfoBase explicitly but never
reads supports_xhigh/minimal/none_reasoning_effort from the cost map
JSON. Add the three fields so get_model_info() returns them correctly.
Also add supports_minimal_reasoning_effort to the ModelInfo TypedDict
(xhigh and none were already declared, minimal was missing).
* fix(model-registry): add missing reasoning effort fields for claude 4.6/4.7
Claude Opus 4.7 supports max reasoning effort (above xhigh).
The field was present for Opus 4.6 but missing for all Opus 4.7
entries (base, dated, Bedrock, Vertex AI, Azure AI).
All Claude 4.6/4.7 models (Opus 4.6, Sonnet 4.6, Opus 4.7) support
minimal reasoning effort via adaptive thinking. Add the field to all
provider variants.
* fix(adapter): map output_config.effort to reasoning_effort (#25079)
Anthropic's adaptive thinking (thinking.type="adaptive") and
output_config.effort were silently dropped when translating to
OpenAI format, resulting in no reasoning_effort on the outgoing
request.
Adapter changes (format translation):
- adapters/transformation.py: add "adaptive" branch to
translate_anthropic_thinking_to_reasoning_effort(); pass through
output_config.effort as-is in _translate_thinking_to_openai();
add "output_config" to translatable_anthropic_params
- adapters/handler.py: extract output_config from extra_kwargs into
request_data so it reaches the translation layer
- responses_adapters/transformation.py: add "adaptive" branch and
output_config param to translate_thinking_to_reasoning()
Handler changes (model-aware normalization):
- utils.py: add normalize_reasoning_effort_value() that uses
get_model_info() to map "max" → "xhigh"/"high" and
"minimal" → "minimal"/"low" based on model capabilities
- adapters/handler.py: call normalization before responses routing
- responses_adapters/handler.py: call normalization after translation
Relates to BerriAI/litellm#25079
* test(reasoning-effort): add tests for effort capability fields and normalize logic
Test coverage for:
- get_model_info returning supports_minimal/max_reasoning_effort fields
- JSON registry entries for claude 4.6/4.7 across all providers
- normalize_reasoning_effort_value degradation chains and exception fallback
- Adapter translation of adaptive thinking + output_config.effort
* fix: forward custom_llm_provider to normalize_reasoning_effort_value in responses adapter
* fix(mcp_semantic_tool_filter): match canonical tools that arrive with
a client-side namespace prefix.
`SemanticMCPToolFilter._get_tools_by_names` matched by exact equality
between the canonical name stored in the router
(`<server><MCP_TOOL_PREFIX_SEPARATOR><tool>`) and the name in the
incoming `tools[]` list. MCP clients such as opencode wrap every tool
name with their own additive alias prefix
(`<client_alias>_<canonical>`), so the two never matched, the filter
dropped every tool to zero, and the proxy forwarded `tools: []` with
`tool_choice: auto` — which strict upstream providers reject with a 400.
The fix adds anchored suffix matching with a separator check: the
canonical must form the complete tail of the incoming name and be
preceded by `_` or `-`. Exact matches still win over suffix matches,
incoming tools are returned at most once, and the original tool object
is passed through unchanged so the client-facing name survives for
tool-call round-trips.
Seven unit tests in a new TestGetToolsByNames class cover exact
match, underscore- and dash-prefixed variants, non-separator-anchored
suffixes (which must not match), exact-wins-over-prefixed precedence,
deduplication when two canonicals suffix-match the same incoming tool,
and ordering-follows-router-output.
Fixes#26078
* review: strengthen the suffix-fallback tie-breaker and the
deduplication regression test (Greptile comments on #26117)
- test_same_tool_not_returned_twice now passes two distinct canonicals
("read_file" and "file") that both suffix-match the same incoming
tool, rather than the same canonical twice, so the assertion
actually exercises the used_ids dedup path instead of the
duplicate-input-list path.
- The suffix fallback in _get_tools_by_names now prefers the shortest
incoming name that still qualifies under the separator-anchored
match. In the one-prefix-per-client opencode scenario this is a
no-op, but in multi-namespace configurations the shortest qualifying
name is the least-wrapped one and is the most defensible deterministic
choice, replacing the dict-insertion-order fallback.
- Adds test_suffix_fallback_prefers_shortest_candidate covering the
new tie-breaker directly.
Still 15 tests passing locally (was 14).
* review(#26117): gate suffix-matching on canonical containing MCP_TOOL_PREFIX_SEPARATOR
@krrish-berri-2 flagged a possible collision in the suffix fallback:
a local user function whose name happens to end in a bare canonical
substring (e.g. my_firecrawl_scrape vs canonical firecrawl_scrape)
would be spuriously selected.
Server-registered MCP tools are always emitted as
<server_name><MCP_TOOL_PREFIX_SEPARATOR><tool_name> via
add_server_prefix_to_name, so a canonical without the separator is
not a namespaced MCP tool and does not warrant suffix matching.
Added that guard to _name_matches_canonical with a regression test
(test_does_not_collide_with_local_function_on_unprefixed_canonical)
that reproduces the collision before the fix and is pinned after.
Pre-existing TestGetToolsByNames fixtures that relied on bare
canonicals (get_weather, search, read_file, write/delete/read) were
switched to realistic server-prefixed ones so they continue to
exercise the suffix-fallback path under the new guard. The opencode
scenario (client prefix on already-server-prefixed canonical) is
unchanged.
---------
Co-authored-by: sakenuGOD <sakenuGOD@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
Two independent bugs both masked budget_reset_at from consumers that
needed it:
1. /team/info.team_member_budget_table was typed as LiteLLM_BudgetTable
(the user-settable allowlist), which dropped server-managed fields.
Switched to LiteLLM_BudgetTableFull so budget_reset_at and created_at
are serialized.
2. _clone_team_default_budget_for_member copied the pool's numeric
fields but never set budget_reset_at on the cloned row. With
budget_duration present but no reset timestamp, the reset job never
fires on the member's budget (its query is reset_at <= now, which
never matches NULL). Now computes budget_reset_at from the cloned
budget_duration via get_budget_reset_time so each member's cycle
starts at clone time rather than inheriting the pool's stale reset.
Two changes, both test-only:
- Configure the aiohttp session with TCPConnector(force_close=True) and an
explicit ClientTimeout(total=30, connect=10). Prevents reuse of idle TCP
connections that the proxy/kernel may have closed during the long window
between setup POSTs and the later poll loop, and surfaces a blocked proxy
event loop quickly instead of hanging on aiohttp's 5-minute default.
- In poll_key_spend_until, catch aiohttp.ClientError and asyncio.TimeoutError
around the single /key/info call. A transient transport hiccup now logs and
retries on the next tick instead of failing the entire polling loop.
Addresses the ConnectionTimeoutError observed on the first /key/info call
after the 20 chat completions.
Adds back the per-cycle spend column that was replaced by Total Spend in
331e3f22. Current Cycle Spend reads membership.spend (zeroed on
budget_reset_at) — this is the value enforced against the member's
budget, so admins need it to see whether a member is approaching their
cap for the active window. Total Spend remains for lifetime analytics.
Adds a formatBudgetReset helper (dayjs-based, with validity guard) that
renders the next reset as "today" / "in N days" / "on MMM D, YYYY". The
team budget card now shows the team's reset timestamp and the member-
default reset (when a shared team_member_budget is configured), and the
Members tab gains a Budget Reset column per member.
- Narrow _team_obj_from_token to require non-None team_id so mypy
passes.
- Preserve the no-auth dev-mode contract for deployments with
master_key unset AND no JWT/OAuth2 configured — the gate
short-circuits only in that specific combination. JWT or OAuth2
deployments without master_key still run the centralized authz.
- is_database_connection_error now enumerates data-layer PrismaError
subclasses (DataError, UniqueViolationError, ForeignKeyViolationError,
MissingRequiredValueError, RawQueryError, TableNotFoundError,
RecordNotFoundError) as False, and maps everything else (bare
PrismaError, connectivity subclasses, DB_CONNECTION_ERROR_TYPES) to
True. Known-safe-to-propagate errors don't trigger HA fallback;
unknown / generic PrismaError still falls back to preserve legacy
503 behavior.
- Update test_handle_authentication_error_db_unavailable_connectivity
to include PrismaError in the fallback list.
Multiple paths through _user_api_key_auth_builder returned a
UserAPIKeyAuth without running common_checks(): OAuth2 token validation,
OAuth2 proxy header hook, JWT admin shortcut, master_key path,
pass-through custom headers, the /user/auth route, and the
allow_requests_on_db_unavailable fallback. An operator-configured key
model-access list, max_budget, team_blocked flag, or team model scope
was therefore silently skipped on those paths. The HA-fallback token
was worse: it was a full proxy-admin synthetic, so a DB outage granted
full admin to every caller.
Fix three root causes (VERIA-18):
1. Centralize common_checks in the user_api_key_auth wrapper. The
builder paths no longer call it; the wrapper runs it once after the
builder returns, for every path. Introduces _run_centralized_common_checks
which gathers team/user/project/end_user/global_spend context in
parallel via asyncio.gather. Preserves the existing
custom_auth_run_common_checks opt-out for custom-auth deployments.
2. Narrow is_database_connection_error — drop the blanket PrismaError
catch that routed data-layer errors (UniqueViolationError, etc.)
into the HA fallback. Only real connectivity failures plus the
no_db_connection marker now qualify.
3. DB-unavailable fallback issues an INTERNAL_USER token with user_id
DB_UNAVAILABLE_FALLBACK_USER_ID instead of proxy-admin. An outage
can no longer escalate an anonymous caller.
JWT admin / master_key tokens still grant admin via a synthesized
admin user_object (so non_proxy_admin_allowed_routes_check in
common_checks recognizes them); other common_checks branches
(team_blocked, team_model_access) now apply uniformly.
Brings the Snowflake, S3 Vectors, Vertex AI, and Bedrock URL construction
paths in line with the existing pattern of validating interpolated values
before use.
Consolidate 6 distinct cache-key prefixes (v2-dependencies-,
v1-router-testing-deps-, v1-router-unit-deps-, v1-llm-translation-deps-,
v1-llm-responses-deps-, v3-litellm-uv-deps-, ui-e2e-py-deps-v2-) onto a
single v1-uv-cache-<uv.lock checksum> key shared across all Python jobs.
Cache only ~/.cache/uv (the content-addressed uv download cache,
hash-verified against uv.lock at install time). Drop ./.venv,
~/.local/{bin,lib}, and /home/circleci/.{pyenv,local} from cache paths.
~/.cache/uv is the only path uv sync needs to avoid re-downloading from
PyPI; everything else is rebuilt each run from that verified cache.
Remove partial-prefix restore-keys fallbacks — cache either hits exactly
on the uv.lock hash or rebuilds cleanly.
First run after merge will cold-miss on the new key; subsequent runs
hit the unified cache.
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
The daily queue parameter types on _restore_spend_updates_to_in_memory_queues
were narrowed to specific subtypes (DailyUserSpendTransaction, etc), but
the caller passes Dict[str, BaseDailySpendTransaction] — the return type
of flush_and_get_aggregated_daily_spend_update_transactions. Widen the
parameters to the base type.
Also replace dynamic TypedDict key lookup (which returned object) with
explicit literal-keyed get() calls so mypy can type-narrow each field.
Two model-access gates run per request in `common_checks` and they're
asymmetric: `can_key_call_model` falls back to the key's
`access_group_ids`, but `can_team_access_model` only looks at
`team.models` + `team.access_group_ids`. A key granted a model via its
own access group on a model-restricted team is silently denied at the
team gate.
Wrap `can_team_access_model` in try/except in `common_checks`: on
`team_model_access_denied`, consult a new `_key_access_group_grants_model`
helper that expands `valid_token.access_group_ids` via the existing
`_get_models_from_access_groups` and checks via `_can_object_call_model`.
Re-raise if the key's access groups don't grant the model. Any other
exception propagates unchanged.
Effect: request allowed if `team allows X` OR `key's access group
grants X`, making the two gates symmetric.
Test: add three unit tests for `_key_access_group_grants_model`
covering: group covers model, key has no groups, group resolves but
does not cover model.
Address two Greptile P2 findings on the BYOK OAuth endpoints:
- GET /v1/mcp/oauth/authorize now runs _validate_redirect_uri up front
so a non-loopback redirect_uri is rejected before the HTML form is
rendered. Previously the user typed an API key, submitted, and got a
400 with no form state.
- POST /v1/mcp/oauth/token moves the master_key guard ahead of the
code-consumption and credential-store steps. Without this, a proxy
with master_key unset would burn the code and persist the credential
but return an error — leaving the user with no way to retrieve a
session token without restarting the whole flow.
