Commit graph

42355 commits

Author SHA1 Message Date
Mateo Wang
3950f5ea72
feat: add gpt-5.5 to model cost map (#26345)
* feat: add gpt-5.5 to model cost map

Add gpt-5.5 entry with pricing from OpenAI flagship page:
input $5/1M, cached input $0.50/1M, output $30/1M, 272K context.

* test: add gpt-5.5 coverage for model cost map and gpt-5 routing

- Add gpt-5.5 to GPT5_MODELS parametrized list so both OpenAIGPT5Config
  and AzureOpenAIGPT5Config routing tests cover the new model.
- Add test_generic_cost_per_token_gpt55 verifying the new entry's
  cost-map values ($5/$0.50/$30 per 1M) and that generic_cost_per_token
  returns the expected prompt/completion costs.
2026-04-23 14:05:22 -07:00
Yuneng Jiang
c41567eaa0
fix(budget_reset): use raw SQL for IS NOT NULL filter on Json? columns
The periodic budget-window reset job filtered keys/teams with
`where={"budget_limits": {"not": None}}`. The prisma-client-python
library does not support null-filtering on `Json?` columns (no
DbNull/JsonNull sentinel — upstream issue #714). The client drops the
`None` value during serialization and the engine rejects the query with
`MissingRequiredValueError: where.budget_limits.not: A value is
required but not set`, so neither the key nor team reset path runs.

Switch those two `find_many` calls to `query_raw` with
`WHERE budget_limits IS NOT NULL`, selecting only the PK and the
`budget_limits` column. Writes still go through the ORM. Add unit tests
covering the expired/unexpired paths for keys and teams, string-encoded
JSON payloads, empty payloads, error isolation between the two paths,
and a regression guard asserting the query still uses `IS NOT NULL`.
2026-04-23 12:26:25 -07:00
Michael Riad Zaky
46336b1ac3 fix linting 2026-04-23 12:08:10 -07:00
yuneng-jiang
2bfbb142b7
Merge pull request #26336 from BerriAI/litellm_yj_apr22
[IInfra] Merge dev branch
2026-04-23 12:02:47 -07:00
Yuneng Jiang
daf29d6a4a
[Infra] Add standalone create-release-branch workflow
Extracts release branch creation into a separate reusable workflow
(create-release-branch.yml) that can be triggered independently via
workflow_dispatch or called from other workflows via workflow_call.

create-release.yml now dispatches it as a dependent job after the
release publishes, keeping both workflows decoupled.
2026-04-23 12:02:25 -07:00
Yuneng Jiang
994e35135d
fix: correct image size limit enforcement and vertex_location None passthrough
token_counter.py: the previous size-limit raises were inside except Exception: pass,
so they were silently swallowed. The post-read raise was worse — img_data was already
assigned the full body before the raise, so the oversized value was used downstream.
Restructured to only assign img_data when the body is within bounds.

vertex_ai/common_utils.py and llm_passthrough_endpoints.py: the is-not-None guard
skipped validation for None, falling through to produce "https://None-aiplatform..."
Added explicit None check that raises before the regex guard.
2026-04-23 11:20:20 -07:00
Ryan Crabbe
85f9c5e83f
feat(mcp): resolve team/key MCP permissions by server name or alias
team.object_permission.mcp_servers (and the per-key equivalent) previously
only accepted server_id strings. For config-loaded MCP servers, the id is
derived from a hash that includes the server URL, so the same logical
server in two regions ends up with two different ids in a shared database.
Permission lists had to enumerate every region's id.

Add a single MCPServerManager.expand_permission_list() helper that resolves
each entry against the current region's config + DB registry union: entries
that match a server_id pass through, entries that match an alias/server_name/
name expand to every matching id, and unresolved entries drop with a debug
log so stale or typo entries are diagnosable. Wire it into the four
_get_allowed_mcp_servers_for_* helpers so direct server entries and
mcp_tool_permissions dict keys are both expanded before the intersection.

Access-check outcomes are unchanged for existing id-based permissions;
name-based entries now resolve instead of being silently denied.
2026-04-23 11:13:30 -07:00
yuneng-jiang
7ebe7cc99b
Merge pull request #26293 from BerriAI/litellm_img_edit_file_input_validation
[Fix] Image edit endpoints: enforce multipart-only file inputs
2026-04-23 09:42:03 -07:00
shin-berri
da3b715c36
Merge pull request #26286 from BerriAI/litellm_/unify_uv_cache
[Infra] CCI: cache, cleanup, anchors, install-path parity, Python 3.12, Ruby/Node pins
2026-04-23 09:39:06 -07:00
Sameer Kankute
d5449f5b1a
Merge pull request #26300 from BerriAI/litellm_oss_staging_04_22_2026
Litellm oss staging 04 22 2026
2026-04-23 18:53:58 +05:30
Sameer Kankute
2d1cc68e22
fix(dashscope): fail fast on image generation API errors
Prevent silent empty image responses by raising provider errors for non-200 HTTP statuses and DashScope API-level error payloads, with regression tests covering both paths.

Made-with: Cursor
2026-04-23 18:41:01 +05:30
Sameer Kankute
2e3a4bb27a
Fix black 2026-04-23 18:32:24 +05:30
Sameer Kankute
e3440baa0c
Merge pull request #25767 from vinhphamhuu-ct/main
feat: Expand VideoMetadata support to all Gemini Models.
2026-04-23 17:20:01 +05:30
Sameer Kankute
1385d46e99
FIx mypy issues 2026-04-23 17:17:06 +05:30
Yuneng Jiang
03a022436b
[Infra] CCI: run RVM install from its own checkout dir
The rvm/install script sources scripts/functions/installer using
paths relative to the caller's working directory (not $0), so
invoking /tmp/rvm/install from /home/circleci/project fails with
'No such file or directory'. Switch to (cd /tmp/rvm && ./install).
2026-04-22 21:50:53 -07:00
Yuneng Jiang
eb6a2d043c
[Infra] CCI: pin Ruby and Node.js installs in proxy_pass_through_endpoint_tests
Align the Ruby, Node.js, and npm install path with the rest of the
config. Three separate upstream installers were being invoked via
\`curl ... | bash\` or unlocked \`npm install\`:

- RVM's \`get.rvm.io/stable\` installer (mutable upstream script).
  Replace with a shallow git clone of the rvm/rvm repo at tag 1.29.12
  and verify HEAD matches the published commit SHA before running the
  local \`./install\` script. Same pattern already used for the
  helm-unittest plugin in .github/workflows/helm_unit_test.yml.
- NodeSource's \`deb.nodesource.com/setup_18.x\` piped into sudo bash.
  Replace with a direct download of the Node.js 18.20.8 linux-x64
  tarball from nodejs.org, verified against the published
  SHASUMS256.txt digest before extraction.
- \`npm install @google-cloud/vertexai @google/generative-ai\` and
  \`--save-dev jest\` resolved fresh from the npm registry on every
  run. Add \`tests/pass_through_tests/package.json\` with pinned
  direct-dep versions and commit the generated package-lock.json, then
  switch CI to \`npm ci\` (exact lockfile install, fails on drift).

Also scopes the Ruby+JS test runners to \`tests/pass_through_tests/\`
so they pick up the committed package.json rather than writing
node_modules at repo root.
2026-04-22 21:28:42 -07:00
Yuneng Jiang
a12a2190d7
[Infra] Flip remaining CI jobs to Python 3.12
Stragglers from the 2026-04-21 Python 3.12 standardization:
- .github/workflows/check_duplicate_issues.yml (was 3.11)
- .github/workflows/llm-translation-testing.yml (was 3.11)
- .github/workflows/scan_duplicate_issues.yml (was 3.13)
- .circleci proxy_build_from_pip_tests (was 3.13)

The only intentional non-3.12 CI job is installing_litellm_on_python_3_13,
which exists as an explicit "latest supported Python" smoke matrix.
2026-04-22 21:26:19 -07:00
Yuneng Jiang
547d60c642
[Infra] CCI: match Windows uv install path to Linux verification pattern
The Windows uv install step was piping a remote install.ps1 into
Invoke-Expression without any integrity check, while the Linux
install steps (install_uv command, line 89) download to a file,
verify SHA-256 against a hardcoded digest, and only then execute.
Bring the Windows path to the same pattern.

Also hardcode the kubectl v1.31.4 checksum in helm_chart_testing
instead of fetching kubectl.sha256 from the same origin as the
binary — if dl.k8s.io were ever to serve a tampered pair, a
co-hosted checksum provides no additional integrity.
2026-04-22 21:25:22 -07:00
Yuneng Jiang
44362cb167
[Infra] CCI: factor repeated filters and Python docker image to YAML anchors
The same branch filter block appeared 46 times in the workflow
declaration:

    filters:
      branches:
        only:
          - main
          - /litellm_.*/

And the same pinned Python docker image appeared 29 times in jobs:

    - image: cimg/python:3.12@sha256:9c796c...
      auth:
        username: ${DOCKERHUB_USERNAME}
        password: ${DOCKERHUB_PASSWORD}

Replace with YAML anchors declared at first use:

- `&main_branches` on using_litellm_on_windows's filters block;
  all other job entries reference it as `filters: *main_branches`.
- `&python312_image` on local_testing_part1's first docker image
  entry; all other jobs reference `- *python312_image`, including
  the multi-image jobs (auth_ui_unit_tests,
  installing_litellm_on_python_v2_migration_resolver) which keep
  their postgres sidecar entry inline afterwards.

Net result: one place to change when the image digest rolls or
the branch-filter convention changes. No behavior change — YAML
anchor resolution produces identical config at parse time.

Also adds Docker Hub auth block to upload-coverage (previously
pulled anonymously). No functional difference for a public
image, but avoids Docker Hub rate limits now that we reuse the
same entry.
2026-04-22 21:24:06 -07:00
Yuneng Jiang
bea872a034
[Infra] CCI: remove dead steps accumulated across jobs
Clean out copy-paste debug and workaround lines that serve no
purpose:

- `pwd && ls` echoes at the top of 30 "Run tests" steps (CCI
  already logs working_directory on every step).
- "Show git commit hash" in local_testing_part1/part2 and
  langfuse_logging_unit_tests (CCI shows the SHA in every job
  header).
- "Verify Docker is available" stubs in 6 machine-executor
  jobs (machine executors always have Docker).
- `sudo systemctl restart docker` in proxy_store_model_in_db_tests
  (one-off workaround; not used anywhere else).
- Duplicated Black formatting step in local_testing_part1 and
  local_testing_part2 — Black runs in the lint job, no reason to
  run it again here.
- Second back-to-back `helm test litellm --logs` invocation in
  helm_chart_testing (one call is enough).

No behavior change — these are all log-only or no-op steps.
2026-04-22 21:18:16 -07:00
Yuneng Jiang
8fbf0d5554
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/unify_uv_cache 2026-04-22 21:06:24 -07:00
user
f5b4564466
fix(mcp): OAuth 2.1 spec compliance + cookie exp + loopback edge cases
Address codex review P1 + P2 findings:

- BYOK /token now accepts OAuth 2.1 clients that omit redirect_uri
  (draft-15 §4.1.3 dropped the requirement). When the client does
  submit a value, equality is still enforced vs the /authorize record.
  PKCE + client_id binding cover the security role redirect_uri
  played under RFC 6749.
- _user_id_from_session_cookie requires the ``exp`` claim on the UI
  session JWT (PyJWT options={"require": ["exp"]}) so leaked cookies
  have a bounded lifetime.
- validate_loopback_redirect_uri rejects URIs with a fragment
  (RFC 6749 §3.1.2) and catches malformed-URI ValueError so
  unparseable input surfaces as 400 invalid_request instead of 500.
2026-04-23 03:39:41 +00:00
user
b5c78d7db4
fix(auth): guard _team_obj_from_token against team-less tokens
The HTTPException arm in _run_centralized_common_checks assumed the
exception came from the team-object fetch, but asyncio.gather raises
the first exception from any of the five gathered coroutines. If
get_user_object / get_project_object / get_end_user_object raises
HTTPException on a token with team_id=None, the assert inside
_team_obj_from_token fires and the outer auth-exception handler
mishandles it.

Guard on team_id before calling _team_obj_from_token and default
team_object to None otherwise. (Greptile P1.)
2026-04-23 03:17:49 +00:00
user
200a38c3af
fix(mcp): revalidate loopback at /callback + no-store on /token
Address codex review P0 + P1 findings on the discoverable OAuth proxy:

- /callback now re-validates that the decoded base_url is loopback before
  302-redirecting to it. State is encrypted but pre-existing states minted
  before the /authorize validation was added have no expiry and remain
  valid; validating at the sink closes the open-redirect + code-theft
  primitive for those stale states too. (VERIA-57 root cause B, P0.)
- /token responses now set Cache-Control: no-store + Pragma: no-cache
  per RFC 6749 §5.1 (P1).
- Move TOKEN_NO_CACHE_HEADERS constant from byok_oauth_endpoints into
  the shared oauth_utils module so both endpoints use the same value.
2026-04-23 03:16:37 +00:00
Sameer Kankute
ef1c6aeea6
Merge pull request #26306 from BerriAI/litellm_internal_staging
Some checks failed
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (proxy-utils, tests/proxy_unit_tests/test_proxy_utils.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
merge main
2026-04-23 08:39:04 +05:30
Sameer Kankute
63ba912b47
Merge pull request #26160 from BerriAI/litellm_vertex_image_edit_credentials_fix
fix(image_edit): forward litellm_params to validate_environment for Vertex AI credentials
2026-04-23 08:36:01 +05:30
Zark .
fcf917df6d
Feat(dashscope): add image generation support for qwen-image-2.0 and qwen-image-2.0-pro (#25672)
* feat: add dashscope/qwen-image-2.0 and qwen-image-2.0-pro to model cost map

* feat: implement DashScope image generation transformation class

* feat: register DashScope in ProviderConfigManager for image generation

* feat: add DashScope to image generation provider routing

* feat: auto-route qwen-image /chat/completions requests to /images/generations

* test: add unit tests for DashScope image generation (22 cases)

* refactor: remove proxy-layer qwen-image auto-routing

* feat: auto-redirect image_generation models in acompletion()

* test: add acompletion auto-redirect test for image_generation models

* fix: remove unused Union import in DashScope transformation

* fix: scope acompletion redirect to dashscope and narrow exception handler

* fix: move get_str_from_messages to module-level import and forward n param to aimage_generation

* refactor: remove acompletion image_generation auto-redirect for dashscope

* test: remove acompletion auto-redirect test for dashscope image models

---------

Co-authored-by: zark.lin <zark.lin@thinkchina.com>
2026-04-22 20:03:46 -07:00
Sameer Kankute
cc1100b4a9
Merge pull request #26305 from BerriAI/litellm_internal_staging
Some checks failed
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (proxy-utils, tests/proxy_unit_tests/test_proxy_utils.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
merge main
2026-04-23 08:32:29 +05:30
Sameer Kankute
47f46b4eab
Merge pull request #26304 from BerriAI/litellm_internal_staging
Some checks are pending
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (proxy-utils, tests/proxy_unit_tests/test_proxy_utils.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Waiting to run
Unit Tests: Security / security (push) Waiting to run
merge main
2026-04-23 08:31:32 +05:30
user
ef108e79a1
fix(mcp): validate loopback redirect_uri in discoverable OAuth proxy
Extract the BYOK loopback redirect_uri check into a shared
oauth_utils.validate_loopback_redirect_uri helper. Call it in
discoverable_endpoints.authorize_with_server before the client-supplied
redirect_uri is encrypted into the OAuth state.

Without this check, a non-loopback redirect_uri was encoded into the
state parameter and decoded on /callback to 302 the user back to the
attacker's URL with the authorization code attached — an open-redirect
+ code-theft primitive (VERIA-57 root cause B). The /callback handler
is already safe because state is HMAC-signed via encrypt_value_helper,
so validating at /authorize before encoding is sufficient.

Also updates existing tests to use loopback client redirect_uris and
adds regression tests for non-loopback rejection, IPv4 127.0.0.0/8
range acceptance, and full-form IPv6 loopback acceptance.
2026-04-23 03:01:18 +00:00
Sameer Kankute
94288d76a9
Merge pull request #26303 from BerriAI/litellm_internal_staging
Some checks failed
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (proxy-utils, tests/proxy_unit_tests/test_proxy_utils.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
merge main
2026-04-23 08:30:54 +05:30
Sameer Kankute
f3b80726a7
Merge pull request #26301 from BerriAI/litellm_internal_staging
Some checks failed
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (proxy-utils, tests/proxy_unit_tests/test_proxy_utils.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
merge main
2026-04-23 08:30:10 +05:30
Braulio Vargas López
d26bcda52a
refactor: replace substring check with startswith in is_model_gpt_5_model (#25793)
The original check `"gpt-5-chat" not in model` already correctly
classifies all current gpt-5 variants (including gpt-5.3-chat and
gpt-5.1-chat, which do NOT contain the substring "gpt-5-chat"). This
change replaces it with an explicit `startswith("gpt-5-chat")` prefix
test on the provider-prefix-stripped model name.

The new check is functionally equivalent for all existing model names
but makes the classification boundary unambiguous and forward-safe:
future model names that might contain "gpt-5-chat" as an interior
substring won't accidentally be excluded from the GPT-5 reasoning path.

Also moves the new regression test from tests/ root to
tests/test_litellm/llms/openai/ so it is included in `make test-unit`.
2026-04-22 19:55:00 -07:00
user
fb55e2f1b9
fix(auth): propagate BudgetExceededError through _safe_fetch
get_end_user_object raises litellm.BudgetExceededError internally
when the end-user is over budget. The previous _safe_fetch in the
centralized gate swallowed it and returned None, which caused
common_checks to see end_user_object=None and skip the budget check
entirely — silently bypassing end-user budget enforcement.

Add BudgetExceededError to the re-raise list alongside HTTPException
and ProxyException (reported by Veria AI).
2026-04-23 02:48:03 +00:00
Rick
c26e304abc
fix(ui): stale filters applied after sort/page/time change on Request Logs (#25789)
The useEffect that re-fetches logs on sort/page/time changes:

  useEffect(() => {
    if (hasBackendFilters && accessToken) {
      performSearch(filters, currentPage);
    }
  }, [sortBy, sortOrder, currentPage, startTime, endTime, isCustomDate]);

intentionally omits `filters` and `hasBackendFilters` from its dep array
to avoid double-fetches when a filter is applied.  The side-effect is a
stale-closure bug: the effect captures `filters` and `hasBackendFilters`
from the render where its deps last changed, not from the render where
the user selected, e.g., a Key Alias.

Reproduce: set Key Alias → results appear correctly → change page or
sort → the effect fires with the OLD `filters` snapshot (no key_alias)
→ API request is sent without the filter → table shows unfiltered data.

Fix: store the latest `filters` and `hasBackendFilters` in refs that are
kept in sync on every render.  The sort/page/time effect reads from the
refs instead of the closure so it always uses the current filter state
without altering the dep array.

Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
2026-04-22 19:41:34 -07:00
Rick
4b2fd870ca
fix(ui): Fetch button ignores active filters on Request Logs page (#25788)
When backend filters (e.g. Key Alias) are active on the Request Logs
page, the manual Fetch button called logs.refetch() which re-runs the
main TanStack Query.  That query does not carry backend-only filter
params such as key_alias, so the button had two problems:

1. It fired a redundant API request without the active filters.
2. It did not refresh the filtered result set — backendFilteredLogs
   stayed frozen at the last debounce-triggered fetch.

Fix: expose refetchWithFilters() from useLogFilterLogic and route the
Fetch button through it when hasBackendFilters is true.  This cancels
any in-flight debounce and calls performSearch with the current filter
state, keeping all active filters intact.

Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
2026-04-22 19:39:24 -07:00
BillionToken
947931858e
fix(anthropic): handle tool_choice type 'none' in messages API (#24457)
* fix(anthropic): handle tool_choice type 'none' in messages API

* test(anthropic): add regression test for tool_choice type 'none'

---------

Co-authored-by: BillionClaw <267901332+BillionClaw@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
2026-04-22 19:36:18 -07:00
Elias
bd145d18e1
fix(ovhcloud): Fix tool calling not working (#25948)
* fix(ovhcloud): fix tool calling

* fix import order
2026-04-22 19:33:58 -07:00
Vigilans
c0c7048903
feat(messages): map reasoning_auto_summary to thinking.display for native /v1/messages (#25883)
When reasoning_auto_summary is enabled (via litellm_settings or env var),
automatically set thinking.display="summarized" on native /v1/messages
requests. This ensures thinking content is returned in the response
instead of being omitted (the default on Claude 4.7+).

Only applies when thinking is enabled (type != "disabled").
The existing reasoning_auto_summary flag already handles the
/v1/responses path (summary="detailed") and the chat/completions
adapter path — this extends coverage to the native messages handler.
2026-04-22 19:29:57 -07:00
Anmol Jaiswal
0e23aa7390
fix(anthropic): tolerate non-OpenAI file content blocks in file-id discovery (#26228)
`get_file_ids_from_messages` and `update_messages_with_model_file_ids`
assume every content block with `type: "file"` has a nested `file` dict in
the OpenAI Chat Completions shape. That assumption is too strong: `type:
"file"` is a public content-block discriminator and several real producers
emit blocks that use it without the OpenAI `file` sub-dict. For example,
LangChain v1's `_normalize_messages` rewrites OpenAI file blocks into
`{"type":"file","id":"...","base64":"...","mime_type":"...","extras":{}}`
before they reach LiteLLM.

`AnthropicConfig.validate_environment` calls both helpers unconditionally
on every Anthropic (and Anthropic-via-Vertex) request, so any such block
raises `KeyError: 'file'` which the Vertex partner layer then wraps as a
`500 InternalServerError` before the LLM is even contacted.

This patch switches both helpers from `c["file"]` to a defensive
`c.get("file")` + dict check. When the block does not match the OpenAI
shape there is no file_id to extract or remap, so we skip it and leave
the block untouched for the downstream provider transformer to handle.

Adds 5 regression tests covering the LangChain v1 shape, the OpenAI
happy path, mixed shapes in one message, `file` set to a non-dict value,
and the remap path for non-OpenAI blocks.

Related to #24503, which proposed raising `BadRequestError` in the same
spots. For these two discovery functions specifically, the skip semantics
is strictly more permissive: well-formed OpenAI blocks still yield their
file_id, and legitimate non-OpenAI blocks stop crashing the request.
2026-04-22 19:22:38 -07:00
Vigilans
b42b86df7a
fix(adapter): normalize reasoning effort with graceful degradation (#26111)
* fix(model-info): include reasoning effort support fields in get_model_info

_get_model_info_helper constructs ModelInfoBase explicitly but never
reads supports_xhigh/minimal/none_reasoning_effort from the cost map
JSON. Add the three fields so get_model_info() returns them correctly.

Also add supports_minimal_reasoning_effort to the ModelInfo TypedDict
(xhigh and none were already declared, minimal was missing).

* fix(model-registry): add missing reasoning effort fields for claude 4.6/4.7

Claude Opus 4.7 supports max reasoning effort (above xhigh).
The field was present for Opus 4.6 but missing for all Opus 4.7
entries (base, dated, Bedrock, Vertex AI, Azure AI).

All Claude 4.6/4.7 models (Opus 4.6, Sonnet 4.6, Opus 4.7) support
minimal reasoning effort via adaptive thinking. Add the field to all
provider variants.

* fix(adapter): map output_config.effort to reasoning_effort (#25079)

Anthropic's adaptive thinking (thinking.type="adaptive") and
output_config.effort were silently dropped when translating to
OpenAI format, resulting in no reasoning_effort on the outgoing
request.

Adapter changes (format translation):
- adapters/transformation.py: add "adaptive" branch to
  translate_anthropic_thinking_to_reasoning_effort(); pass through
  output_config.effort as-is in _translate_thinking_to_openai();
  add "output_config" to translatable_anthropic_params
- adapters/handler.py: extract output_config from extra_kwargs into
  request_data so it reaches the translation layer
- responses_adapters/transformation.py: add "adaptive" branch and
  output_config param to translate_thinking_to_reasoning()

Handler changes (model-aware normalization):
- utils.py: add normalize_reasoning_effort_value() that uses
  get_model_info() to map "max" → "xhigh"/"high" and
  "minimal" → "minimal"/"low" based on model capabilities
- adapters/handler.py: call normalization before responses routing
- responses_adapters/handler.py: call normalization after translation

Relates to BerriAI/litellm#25079

* test(reasoning-effort): add tests for effort capability fields and normalize logic

Test coverage for:
- get_model_info returning supports_minimal/max_reasoning_effort fields
- JSON registry entries for claude 4.6/4.7 across all providers
- normalize_reasoning_effort_value degradation chains and exception fallback
- Adapter translation of adaptive thinking + output_config.effort

* fix: forward custom_llm_provider to normalize_reasoning_effort_value in responses adapter
2026-04-22 19:19:54 -07:00
yuneng-jiang
e9e86ed956
Merge pull request #26298 from BerriAI/litellm_internal_staging
[Infra] Promote interal staging to main
2026-04-22 19:16:23 -07:00
sakenuGOD
034f4fdef2
fix(mcp_semantic_tool_filter): match tools with client-side namespace prefix (#26078) (#26117)
* fix(mcp_semantic_tool_filter): match canonical tools that arrive with
a client-side namespace prefix.

`SemanticMCPToolFilter._get_tools_by_names` matched by exact equality
between the canonical name stored in the router
(`<server><MCP_TOOL_PREFIX_SEPARATOR><tool>`) and the name in the
incoming `tools[]` list. MCP clients such as opencode wrap every tool
name with their own additive alias prefix
(`<client_alias>_<canonical>`), so the two never matched, the filter
dropped every tool to zero, and the proxy forwarded `tools: []` with
`tool_choice: auto` — which strict upstream providers reject with a 400.

The fix adds anchored suffix matching with a separator check: the
canonical must form the complete tail of the incoming name and be
preceded by `_` or `-`. Exact matches still win over suffix matches,
incoming tools are returned at most once, and the original tool object
is passed through unchanged so the client-facing name survives for
tool-call round-trips.

Seven unit tests in a new TestGetToolsByNames class cover exact
match, underscore- and dash-prefixed variants, non-separator-anchored
suffixes (which must not match), exact-wins-over-prefixed precedence,
deduplication when two canonicals suffix-match the same incoming tool,
and ordering-follows-router-output.

Fixes #26078

* review: strengthen the suffix-fallback tie-breaker and the
deduplication regression test (Greptile comments on #26117)

- test_same_tool_not_returned_twice now passes two distinct canonicals
  ("read_file" and "file") that both suffix-match the same incoming
  tool, rather than the same canonical twice, so the assertion
  actually exercises the used_ids dedup path instead of the
  duplicate-input-list path.

- The suffix fallback in _get_tools_by_names now prefers the shortest
  incoming name that still qualifies under the separator-anchored
  match. In the one-prefix-per-client opencode scenario this is a
  no-op, but in multi-namespace configurations the shortest qualifying
  name is the least-wrapped one and is the most defensible deterministic
  choice, replacing the dict-insertion-order fallback.

- Adds test_suffix_fallback_prefers_shortest_candidate covering the
  new tie-breaker directly.

Still 15 tests passing locally (was 14).

* review(#26117): gate suffix-matching on canonical containing MCP_TOOL_PREFIX_SEPARATOR

@krrish-berri-2 flagged a possible collision in the suffix fallback:
a local user function whose name happens to end in a bare canonical
substring (e.g. my_firecrawl_scrape vs canonical firecrawl_scrape)
would be spuriously selected.

Server-registered MCP tools are always emitted as
<server_name><MCP_TOOL_PREFIX_SEPARATOR><tool_name> via
add_server_prefix_to_name, so a canonical without the separator is
not a namespaced MCP tool and does not warrant suffix matching.
Added that guard to _name_matches_canonical with a regression test
(test_does_not_collide_with_local_function_on_unprefixed_canonical)
that reproduces the collision before the fix and is pinned after.

Pre-existing TestGetToolsByNames fixtures that relied on bare
canonicals (get_weather, search, read_file, write/delete/read) were
switched to realistic server-prefixed ones so they continue to
exercise the suffix-fallback path under the new guard. The opencode
scenario (client prefix on already-server-prefixed canonical) is
unchanged.

---------

Co-authored-by: sakenuGOD <sakenuGOD@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
2026-04-22 19:06:34 -07:00
yuneng-jiang
6a25866f51
Merge pull request #26295 from BerriAI/yj_bump_apr22
Some checks are pending
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (proxy-utils, tests/proxy_unit_tests/test_proxy_utils.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Waiting to run
Unit Tests: Security / security (push) Waiting to run
[Infra] bump versions
2026-04-22 18:33:03 -07:00
yuneng-jiang
8bc6948a5e
Merge pull request #26290 from BerriAI/litellm_/bold-goldstine-b86b67
[Fix] Stabilize spend accuracy test transport flakes
2026-04-22 18:30:10 -07:00
Ryan Crabbe
6be8313147
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_team_member_total_spend_frontend
Some checks failed
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (proxy-utils, tests/proxy_unit_tests/test_proxy_utils.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
2026-04-22 18:29:04 -07:00
Ryan Crabbe
ac453c958e
fix(team): surface budget_reset_at on /team/info and cloned member budgets
Two independent bugs both masked budget_reset_at from consumers that
needed it:

1. /team/info.team_member_budget_table was typed as LiteLLM_BudgetTable
   (the user-settable allowlist), which dropped server-managed fields.
   Switched to LiteLLM_BudgetTableFull so budget_reset_at and created_at
   are serialized.

2. _clone_team_default_budget_for_member copied the pool's numeric
   fields but never set budget_reset_at on the cloned row. With
   budget_duration present but no reset timestamp, the reset job never
   fires on the member's budget (its query is reset_at <= now, which
   never matches NULL). Now computes budget_reset_at from the cloned
   budget_duration via get_budget_reset_time so each member's cycle
   starts at clone time rather than inheriting the pool's stale reset.
2026-04-22 18:27:36 -07:00
Yuneng Jiang
95fa7678af
uv lock 2026-04-22 18:25:37 -07:00
Yuneng Jiang
9f46d838fd
bump: version 1.83.11 → 1.83.12 2026-04-22 18:21:47 -07:00
Yuneng Jiang
3ddb3cbdf6
bump: version 0.4.67 → 0.4.68 2026-04-22 18:20:21 -07:00