Commit graph

39980 commits

Author SHA1 Message Date
harish-berri
f79d3fae2d
Merge pull request #26902 from BerriAI/litellm_pyroscope_tag_wrapper
feat(proxy): add support for Grafana Cloud Pyroscope authentication
2026-05-04 14:08:17 -07:00
mateo-berri
1ac430cfea style: make _model_supports_effort_param more concise 2026-05-04 13:54:05 -07:00
Yassin Kortam
516b741de1 feat: selectively apply routing strategy according to model name 2026-05-04 13:27:32 -07:00
user
1dcfc36393 chore(deps): align dashboard node engine 2026-05-04 13:21:03 -07:00
user
0f02a5f8f6 test: keep decode token test local 2026-05-04 13:14:59 -07:00
Cursor Agent
2cb3f0f027
refactor: remove unnecessary comments from #27074
Strip out the explanatory and historical comments that don't carry
business-logic justification. Comments that simply narrate what code
does — or that explain prior behavior, what was changed, or which PR
introduced a fix — are removed. Docstrings are reduced to a one-line
summary where the long form repeated information already evident from
the code or test data.

No code-behavior changes. All 643 affected unit tests still pass.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-05-04 19:34:56 +00:00
Cursor Agent
56070b86a3
test(model_prices): add supports_adaptive_thinking to schema
`test_aaamodel_prices_and_context_window_json_is_valid` validates the
model-map JSON against an explicit schema with `additionalProperties`,
so the new `supports_adaptive_thinking` flag added in
98ced0ae43 needs a matching schema entry.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-05-04 19:11:45 +00:00
user
e96d850b84 chore(deps): address dependency review notes 2026-05-04 12:09:04 -07:00
Cursor Agent
98ced0ae43
refactor(anthropic): drive adaptive-thinking gate via supports_adaptive_thinking flag
Three of greptile's open comments on #27074 (P2 converse:512, P1
databricks:361, and the underlying capability-flag policy rule) flagged
the same pattern: _is_claude_4_6_model(...) or _is_claude_4_7_model(...)
used inline as a runtime 'is this an adaptive-thinking model?' check.
That requires a code release each time a new adaptive Claude lands.

Consolidate the inline gating to AnthropicModelInfo._is_adaptive_thinking_model,
and switch the helper itself to read a new supports_adaptive_thinking
flag from `model_prices_and_context_window.json` via `_supports_factory`,
falling back to the family pattern only when the model-map entry doesn't
carry the flag (preserves OpenRouter / Vercel / Bedrock-prefixed variants
that route through the same code path with non-canonical ids).

Adds `supports_adaptive_thinking: true` to the four 4.6/4.7 anthropic
entries (opus-4-6 + dated, opus-4-7 + dated, sonnet-4-6). Bedrock-prefixed
and Vertex-prefixed entries don't need the flag because both fall back
through the family pattern (the helper short-circuits early on True from
either path) and the bedrock/vertex Claude IDs all match the existing
opus-4-{6,7} / sonnet-4-{6,7} pattern.

Affected call sites:

- `bedrock/chat/converse_transformation.py:_handle_reasoning_effort_parameter`
- `anthropic/chat/transformation.py:_map_reasoning_effort`
- `anthropic/chat/transformation.py:map_openai_params` (output_config branch)
- `databricks/chat/transformation.py:map_openai_params` (output_config branch)

The remaining `_is_claude_4_6_model` / `_is_claude_4_7_model` references
in `AnthropicConfig._validate_effort_for_model` and
`AnthropicConfig.get_supported_openai_params` are intentionally retained:
they're per-model gating fallbacks for variants whose model-map entries
don't yet carry the `supports_max_reasoning_effort` /
`supports_reasoning` flag. Those are documented in-place.

Tests: 537 anthropic/bedrock/databricks/vertex/messages tests pass.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-05-04 18:58:22 +00:00
user
2d36596e8b fix: preserve tokenizer decode round trips 2026-05-04 11:47:32 -07:00
user
0c3b4a06cf chore(deps): authorize pytest license 2026-05-04 11:39:46 -07:00
user
bfdd786962 chore(deps): refresh dependency locks 2026-05-04 11:36:18 -07:00
user
7c94149aeb Merge remote-tracking branch 'origin/litellm_internal_staging' into codex/cloud-storage-file-guard
# Conflicts:
#	litellm/llms/vertex_ai/files/handler.py
#	litellm/llms/vertex_ai/files/transformation.py
2026-05-04 11:25:41 -07:00
user
a05d3b5851 Fix qdrant semantic cache miss metadata 2026-05-04 11:18:41 -07:00
harish-berri
ed3b056bc8 Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior. 2026-05-04 18:17:31 +00:00
Mateo Wang
790d8bbe1a
Merge pull request #26899 from BerriAI/litellm_suppress-spend-log-tracebacks-2208
feat(spend-logs): opt-in suppression of stack traces in spend-tracking error logs
2026-05-04 11:09:03 -07:00
Cursor Agent
5d124892d2
refactor(bedrock/converse): delegate effort gating to AnthropicConfig._validate_effort_for_model
Removes the duplicated max/xhigh gating logic in
_validate_anthropic_adaptive_effort and the now-unused
_supports_effort_level_on_bedrock helper. Per-model gating now flows
through the centralized AnthropicConfig._validate_effort_for_model
(whose _supports_effort_level already strips Bedrock prefixes), so the
chat completion, /v1/messages, and Bedrock Converse paths can't drift
when a new gated effort tier is added.
2026-05-04 18:01:30 +00:00
mateo-berri
c1708ddbba fix(xai): fold reasoning_tokens into completion_tokens to satisfy OpenAI invariant
xAI's chat completions API accounts reasoning_tokens separately from
completion_tokens, but rolls them into total_tokens. This breaks the
OpenAI invariant total_tokens == prompt_tokens + completion_tokens
that downstream consumers (including litellm's own _usage_format_tests
in tests/llm_translation/base_llm_unit_tests.py:58) rely on.

Live capture (grok-3-mini-beta, 2026-05-04):
    prompt=14, completion=10, total=336, reasoning=312
    14 + 10 = 24, NOT 336.

OpenAI's o1/o3 reasoning models include reasoning_tokens in
completion_tokens, leaving the prompt+completion=total invariant
intact. xAI deviates. This patch aligns xAI to OpenAI semantics by
folding reasoning_tokens into completion_tokens after the parent
OpenAI parser runs.

The fold is idempotent and defensive:
- Only fires when total_tokens == prompt_tokens + completion_tokens
  + reasoning_tokens (the documented xAI shape). Refuses to fold if
  the gap doesn't match, guarding against silent corruption when xAI
  changes accounting.
- Skips if completion_tokens already covers the gap (already
  normalised — e.g. cost calc replays a previously-folded Usage).

xai.cost_calculator.cost_per_token already added reasoning_tokens to
the visible completion count for billing. Post-fold the Usage block
now satisfies that invariant directly, so the cost calc would
double-bill. Updated cost_per_token to detect the OpenAI-normalised
shape (total == prompt + completion) and skip the reasoning add-on
in that case, falling through to the legacy raw-shape behaviour for
callers that bypass the transformation (e.g. proxy log replay).

Tests:
- Adds TestXAIReasoningTokenFolding covering: gap-explained-fold,
  idempotent-no-double-fold, no-reasoning-skip, gap-mismatch-skip.
- Adds test_already_normalised_usage_does_not_double_count_reasoning
  to lock the cost-calc idempotency.
- Updates 7 pre-existing cost-calc tests whose total_tokens was
  internally inconsistent (used the OpenAI-normalised total but kept
  reasoning_tokens external) to use the documented xAI raw shape
  total = prompt + visible completion + reasoning. Pre-existing
  values masked the missing-fold by accident.

Verified end-to-end against the live xAI API:
    LITELLM_LOCAL_MODEL_COST_MAP=False (CI default) +
    XAI_API_KEY set +
    pytest tests/llm_translation/test_xai.py::TestXAIChat::test_prompt_caching
        -> PASSED in 18.81s (was: AssertionError on
        usage.total_tokens == usage.prompt_tokens + usage.completion_tokens)

20/20 tests in tests/test_litellm/llms/xai/test_xai_cost_calculator.py
and 8/8 in tests/test_litellm/llms/xai/test_xai_chat_transformation.py
pass.
2026-05-04 10:34:31 -07:00
harish-berri
a470309c3b Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_pyroscope_tag_wrapper
merge parent
2026-05-04 17:17:49 +00:00
yuneng-jiang
b2c270e653
Merge pull request #27123 from BerriAI/litellm_/intelligent-fermat-298a82
[Fix] Docker: Pin Wolfi And Uv To Multi-Arch Index Digests
2026-05-04 10:08:47 -07:00
Yuneng Jiang
25a5cccc7a
[Fix] Docker: Pin Uv To Multi-Arch Index Digest In Remaining Dockerfiles
Apply the same fix to the three Dockerfiles not in the release pipeline
today (alpine, dev, health_check) so they stay correct if/when they're
built for arm64 in the future.

Wolfi pins are not present in these files; the python:3.11-alpine and
python:3.13-slim digests they already use are multi-arch indexes that
include arm64/v8, so only the uv pin needed swapping.
2026-05-04 10:02:48 -07:00
Michael-RZ-Berri
675e49ed94
Merge pull request #26894 from BerriAI/litellm_langsmithRedactApiInfo
[Fix] Remove unwanted metadata info from LangSmith
2026-05-04 09:55:59 -07:00
Yuneng Jiang
08d130a8fe
[Fix] Docker: Pin Wolfi And Uv To Multi-Arch Index Digests
The previous pins resolved to single-platform amd64 manifests, so buildx
pulled the same amd64 base for both linux/amd64 and linux/arm64 targets.
The published OCI index then advertised an arm64 entry whose layers are
byte-identical to amd64 -- arm64 users got an amd64 binary.

Switch all three Dockerfiles to the multi-arch image-index digests:
  - cgr.dev/chainguard/wolfi-base   (index has linux/amd64 + linux/arm64)
  - ghcr.io/astral-sh/uv:0.11.7     (index has linux/amd64 + linux/arm64)

Resolved with `docker buildx imagetools inspect <ref>` -- that returns
the index digest. `docker pull` + `docker inspect` returns the per-host
platform digest, which is what slipped in last time.
2026-05-04 09:55:53 -07:00
Mateo Wang
196c7a0c09
Merge pull request #27077 from BerriAI/litellm_fix_responses_api_legacy_claude_4_sonnet-9574
test(responses): replace legacy `claude-4-sonnet-20250514` alias in multiturn tool-call test
2026-05-04 09:50:17 -07:00
mateo-berri
f4d6d5953d test(anthropic/chat): force PR-local model_cost map via autouse fixture
CI runs without LITELLM_LOCAL_MODEL_COST_MAP=True, so litellm.model_cost
is loaded from main-branch JSON (default model_cost_map_url) instead of
the PR's checked-out model_prices_and_context_window.json. Tests that
assert per-model flags added in this PR (supports_max_reasoning_effort,
supports_xhigh_reasoning_effort) therefore pass locally but fail in CI
with 'AssertionError: assert False is True' on 5 cases:

  - test_anthropic_model_supports_effort_param_recognizes_supporting_models
    [anthropic.claude-mythos-preview, bedrock/.../mythos-preview,
     claude-opus-4-5-20251101]
  - test_supports_effort_level_handles_provider_prefixes
    [bedrock/invoke/us.anthropic.claude-sonnet-4-6-max-True,
     claude-sonnet-4-6-max-True]

Add an autouse fixture at tests/test_litellm/llms/anthropic/chat/conftest.py
that monkey-patches litellm.model_cost to the PR-local JSON for every test
in this directory. The parent conftest already snapshots+restores
litellm.model_cost per-function, so the mutation is contained.

This is a scoped workaround. The proper fix is to set the env var
globally in the test workflow once the ~10 inline self-set test files
are audited; tracking that as a follow-up issue.
2026-05-04 09:26:57 -07:00
Sameer Kankute
c53c71ad66
test(image_gen): align Azure image gen fixture with body omitting model
Expected JSON matches deployment-scoped Azure POST (#26316).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 18:09:30 +05:30
Sameer Kankute
32a5e77adf
feat(proxy): add health_check_reasoning_effort for model health checks
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 17:13:01 +05:30
mateo-berri
2c9166c4f3 fix(databricks): narrow reasoning_effort_value to str for mypy
`non_default_params.get("reasoning_effort")` returns `Any | None`,
but `REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT.get()` expects `str`.
Mypy flagged this on the strict pass. Narrow with `isinstance` before
the lookup; non-strings fall through to the existing `BadRequestError`
below with a clean validation message, so behavior is unchanged.

Fixes a regression introduced by 1a10746e95 in this PR.
2026-05-04 00:52:09 -07:00
mateo-berri
f8f07c5cb7 refactor(anthropic): extract _validate_effort_for_model to prevent drift
The chat completion path (`_apply_output_config`) and the /v1/messages
pass-through (`AnthropicMessagesConfig._translate_reasoning_effort_to_anthropic`)
both gate `max` / `xhigh` per model. The two sites had diverged from
near-identical copies into separately maintained blocks, creating a real
drift risk when a new model tier (e.g. Claude 4.8) lands -- a contributor
could update one site and miss the other.

Centralise the gating in `AnthropicConfig._validate_effort_for_model`,
which returns an error message string or `None`. Each call site keeps
its own provider-appropriate exception type (`BadRequestError` for the
chat path, `AnthropicError` for the /v1/messages pass-through) but the
gating decision now comes from one place. Net -11 LOC.

Adds a parametrised unit test exercising the helper directly across
4.5 / 4.6 / 4.7 model families and `max` / `xhigh` / lower-effort
inputs. Existing tests at both call sites continue to pass unchanged.

Addresses Greptile finding on PR #27074.
2026-05-04 00:47:55 -07:00
Sameer Kankute
41ac026fc1
test(image_gen): expect no model in Azure image edit multipart (#26316)
Align test_azure_image_edit_litellm_sdk with deployment-scoped Azure edits.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 12:20:29 +05:30
mateo-berri
4f9a3a5c9f fix(azure_ai/anthropic): promote output_config out of extra_body so validation runs
`azure_ai` is registered in `litellm.openai_compatible_providers`, so
`add_provider_specific_params_to_optional_params` (litellm/utils.py)
auto-stuffs any non-OpenAI kwarg (e.g. `output_config={"effort": "..."}`)
into `optional_params["extra_body"]`. `AzureAnthropicConfig.transform_request`
then strips `extra_body` entirely on the way out, silently dropping the
param — and `AnthropicConfig._apply_output_config` never sees it, so
`effort="invalid"` / `effort="xhigh"` on a non-supporting model
quietly reaches the model with default behavior instead of returning a
clean 400 (as the native `anthropic` provider does).

Promote the keys back to top-level `optional_params` (using `setdefault`
so explicit top-level values win) before delegating to the parent
`AnthropicConfig`. Apply in both `validate_environment` and
`transform_request` so flag detection (`is_mcp_server_used`, etc.) and
output-config validation both run.

Surfaced by the QA matrix expansion on PR #27074: 20 cells where Azure
returned 200 while `anthropic` returned 400 — all `output_config` mode
across haiku_4_5, sonnet_4_5, opus_4_5, sonnet_4_6, opus_4_6, opus_4_7
families with `effort` in {invalid, xhigh, max, low, medium, high}.

Tests:
* `test_output_config_promoted_from_extra_body`: valid effort reaches data
* `test_invalid_output_config_effort_raises_via_extra_body`: 400 on bad effort
* `test_unsupported_effort_xhigh_raises_via_extra_body`: 400 on xhigh-on-Sonnet-4.6
* `test_extra_body_promotion_does_not_clobber_top_level`: setdefault semantics
2026-05-03 23:49:01 -07:00
Sameer Kankute
87cf5107e1
test(azure): exercise image gen JSON filter via HTTP client; dedupe image edit URL
- Image generation tests patch HTTPHandler.post / get_async_httpx_client so
  make_*_azure_httpx_request runs and wire json is asserted on call kwargs.
- Azure image edit: strip model in finalize_image_edit_multipart_data using the
  same URL string the handler passes to POST (no second get_complete_url in
  transform). BaseImageEditConfig default finalize is a no-op.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 12:09:05 +05:30
Sameer Kankute
766b67cf0d
fix(azure): omit model from image generation and image edit deployment requests
Azure OpenAI routes image gen/edit by deployment in the URL; sending the
deployment id in model breaks gpt-image-2 (invalid_value). Strip model from
JSON for deployments/.../images/generations and from multipart data for
.../images/edits. Non-deployment URLs (e.g. Azure AI FLUX) unchanged.

Fixes #26316.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 11:52:52 +05:30
Cursor Agent
1a10746e95
fix(databricks): validate reasoning_effort and set output_config on adaptive Claude
The Databricks path called `AnthropicConfig._map_reasoning_effort` for
Claude models but never validated the effort string nor set
`output_config.effort` for adaptive models (Claude 4.6/4.7). Since
`_map_reasoning_effort` returns `type=adaptive` for ANY non-None /
non-"none" string on adaptive models (including "disabled",
"invalid", ""), Databricks silently accepted garbage and emitted a
request without an `output_config.effort`, collapsing every adaptive
tier to identical behavior.

Match the Anthropic native, Bedrock Converse, Bedrock Invoke, and
/v1/messages paths: when the resolved `thinking` is non-None on a
4.6/4.7 model, look up the value in
`REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT` and either raise a clean
`BadRequestError` or set `optional_params["output_config"]`.
2026-05-03 17:30:26 +00:00
mateo-berri
108b87fb24 fix(anthropic,bedrock,databricks): four reasoning_effort follow-ups
- claude-sonnet-4-6 + reasoning_effort=max no longer 400s. Renamed
  _is_opus_4_6_model to _is_claude_4_6_model at three sites and added
  supports_max_reasoning_effort: true to 12 model entries in the JSON
  cost map (10 sonnet 4.6 ids + OpenRouter opus 4.6/4.7).
- _map_reasoning_effort now raises BadRequestError(400) directly with
  llm_provider, instead of letting Databricks (and similar callers)
  surface its raw ValueError as a 500.
- output_config.effort on Opus 4.5 over Bedrock no longer 400s for
  missing effort-2025-11-24 beta. Flipped JSON to "effort-2025-11-24"
  for bedrock + bedrock_converse and added an auto-attach branch in
  _process_tools_and_beta for non-adaptive Anthropic + output_config
  on Converse.
- reasoning_effort=xhigh / =max on legacy budget-mode models
  (Haiku 4.5, Sonnet 4.5, Opus 4.5) now map to thinking.budget_tokens
  8192 / 16384 instead of returning 400. Added two constants in
  litellm/constants.py.

Tests updated for all four flips. Validated end-to-end via 306-cell
live proxy matrix (6 model families x 3 routes x 17 effort cases),
all pass.
2026-05-03 10:03:53 -07:00
mateo-berri
36f1f13925
fix(anthropic): drive output_config.effort support from model map flags
Replace hardcoded _EFFORT_SUPPORTING_MODEL_PATTERNS with a JSON-backed
check that uses supports_*_reasoning_effort flags from the model map.
Add supports_minimal_reasoning_effort: true to opus-4-5 and mythos-preview
entries (which previously only carried supports_reasoning) so the JSON
remains the single source of truth for effort capability.
2026-05-03 11:47:19 +00:00
mateo-berri
e780a7c1dc fix(base_llm): filter all _-prefixed class attrs from get_config
The drop_params strip work added `AnthropicConfig._EFFORT_SUPPORTING_MODEL_PATTERNS`
as a private class-level lookup tuple. `BaseConfig.get_config()` only
filtered the `__`-prefixed names plus `_abc` / `_is_base_class`, so
`_EFFORT_SUPPORTING_MODEL_PATTERNS` would have leaked into the request
body the same way `REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT` did before
the previous commit.

Generalize the existing `_abc` / `_is_base_class` carve-outs to skip
every `_`-prefixed name. `AmazonConverseConfig.get_config()` overrides
the base method, so apply the same change there.

Also unblocks future internal helpers from accidentally serialising into
the wire body.
2026-05-03 04:18:50 -07:00
Claude
09d37db8d9 feat(anthropic,bedrock): strip output_config under drop_params for non-effort models
When a proxy fronts Claude Code (which always sends `output_config.effort`)
at a pre-4.5 Anthropic model — haiku-3, sonnet-3.5, opus-3, sonnet-4 — the
forwarded knob causes a forced 400 the client can't fix. Gating a strip
behind the existing `drop_params` flag lets operators opt into silent
fixup once and stop worrying about per-model param hygiene.

Default (`drop_params=False`) still forwards and surfaces the provider's
error, preserving the strict, debuggable contract from #27074.

Per https://platform.claude.com/docs/en/build-with-claude/effort the
supporting set is Opus 4.5+, Sonnet 4.6+, and Mythos Preview; everything
else is dropped (with a verbose_logger warning so the strip is visible).
Recognition uses model-name patterns plus a fallback to any
`supports_*_reasoning_effort` flag in the model map for forward
compatibility with new entries.

https://claude.ai/code/session_01WjHq31rvXT6xYNdVmSJvRp

(cherry picked from commit 1233943e78)
2026-05-03 04:18:50 -07:00
mateo-berri
82d7405c6f fix(anthropic): stop class-attr leak; gate xhigh/max on every route
The reasoning-effort mapping dict was a public class attribute on
AnthropicConfig, so BaseConfig.get_config returned it as a request
parameter and every Anthropic-backed call (Anthropic / Azure / Vertex /
Bedrock Invoke) hit a 400 'REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT:
Extra inputs are not permitted' from the provider. Move the mapping
to a module-level constant.

_supports_effort_level only looked the model up under
custom_llm_provider='anthropic', so bedrock-prefixed model ids
(e.g. bedrock/invoke/us.anthropic.claude-opus-4-7) returned False
for both 'max' and 'xhigh' even when the underlying model entry has
the flag set. Strip known provider prefixes and retry the lookup
against litellm.model_cost directly so per-model gating works on
every route.

Mirror the per-model xhigh/max gate from
AnthropicConfig._apply_output_config in
AnthropicMessagesConfig._translate_reasoning_effort_to_anthropic so
the /v1/messages route also raises a clean 400 instead of forwarding
the unsupported tier.
2026-05-03 04:18:50 -07:00
Cursor Agent
47031c08d2 style: black formatting
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-05-03 04:18:50 -07:00
Cursor Agent
039b26626f fix(anthropic,bedrock): reject unmapped reasoning_effort at mapping site
Both the chat completion path (AnthropicConfig.map_openai_params) and the
Bedrock Converse path (_handle_reasoning_effort_parameter) used
REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT.get(value, value) which falls
back to the raw input on unmapped keys. Combined with _map_reasoning_effort
returning type='adaptive' for any string on Claude 4.6/4.7, garbage values
(e.g. 'disabled') could leak into optional_params['output_config']['effort']
unvalidated if map_openai_params ran without the downstream transform_request
or _validate_anthropic_adaptive_effort check.

Mirror the /v1/messages pattern: use .get(value) (no fallback) and raise
BadRequestError immediately when the value is unmapped, co-locating
validation with the mapping for defense in depth.
2026-05-03 04:18:50 -07:00
Claude
e400de4654 fix(anthropic): translate reasoning_effort on /v1/messages route
Closes the remaining QA-sweep gap on PR #27074: Bedrock Invoke
/v1/messages was silently ignoring ``reasoning_effort`` because the
shared param filter only kept native Anthropic keys, so every effort
tier collapsed to the same behavior on the wire (27/231 cells failing
across opus-4-5 / opus-4-6 / sonnet-4-6).

Map ``reasoning_effort`` to native Anthropic ``thinking`` /
``output_config.effort`` at the ``AnthropicMessagesConfig`` layer so
all four /v1/messages routes (direct Anthropic, Azure AI, Vertex AI,
Bedrock Invoke) inherit the same translation:

- Add ``reasoning_effort`` to ``AnthropicMessagesRequestOptionalParams``
  so the param filter in
  ``AnthropicMessagesRequestUtils.get_requested_anthropic_messages_optional_param``
  no longer drops it before the transformation runs.

- Add ``_translate_reasoning_effort_to_anthropic`` and call it from
  ``transform_anthropic_messages_request``. Mirrors
  ``AnthropicConfig.map_openai_params`` on the chat completion path
  (re-uses ``_map_reasoning_effort`` and
  ``REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT``) so the two routes
  cannot drift. Pops ``reasoning_effort`` so it never reaches the wire.

- Caller-supplied native ``thinking`` / ``output_config.effort`` always
  win — same precedence as
  ``_translate_legacy_thinking_for_adaptive_model``.

- Garbage values (``""``, ``"disabled"``, ``"invalid"``) raise
  ``AnthropicError(status_code=400)`` instead of falling through and
  surfacing as 500s from the provider.

- ``"none"`` clears thinking + output_config so callers can opt out
  per request.

Also restores the non-adaptive-model test coverage on Bedrock Invoke
/v1/messages that the previous commit lost when
``test_bedrock_messages_strips_output_config`` was renamed to the
``forwards`` variant on Opus 4.7.

Adds a new test file
``test_reasoning_effort_translation.py`` covering the translation at
the shared config level (adaptive + non-adaptive models, none, garbage,
caller precedence) so all four /v1/messages routes are exercised by a
single suite.

Adds parametrized + behavioral tests on the Bedrock Invoke /v1/messages
suite covering: minimal/low/medium/high/xhigh/max mapping for adaptive
models, thinking-budget mapping for non-adaptive Opus 4.5, ``none``
clears both, garbage raises 400, explicit ``output_config`` wins.

Refs: https://github.com/BerriAI/litellm/pull/27074
2026-05-03 04:18:50 -07:00
Cursor Agent
ec022fd1bb fix(anthropic,bedrock): correct effort error message and dedupe effort_map
- Remove 'none' from the Bedrock _validate_anthropic_adaptive_effort error
  message; it was listed as a valid value but rejected by the membership
  check, leaving users in a feedback loop if they tried 'none'.
- Hoist the duplicated reasoning_effort -> output_config.effort mapping
  out of AnthropicConfig.map_openai_params and
  AmazonConverseConfig._handle_reasoning_effort_parameter into a single
  AnthropicConfig.REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT class constant
  so the two routes cannot drift.
2026-05-03 04:18:50 -07:00
Cursor Agent
13714586b6 fix(constants): make ANTHROPIC_MIN_THINKING_BUDGET_TOKENS a plain constant
The documentation CI test (tests/documentation_tests/test_env_keys.py)
asserts every os.getenv() key in the source has a matching entry in the
litellm-docs config_settings.md table. ANTHROPIC_MIN_THINKING_BUDGET_TOKENS
tracks Anthropic's published wire-protocol minimum (1024) — it's not a
user-tunable, so making it env-overridable was wrong anyway. Drop the
os.getenv() wrapper; the value is now a plain literal.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-05-03 04:18:50 -07:00
Cursor Agent
a6c673e7b9 fix(anthropic,bedrock,vertex): forward output_config.effort + 400 on garbage reasoning_effort
Follow-up bugs surfaced by the QA sweep on PR #27039
(https://github.com/BerriAI/litellm/pull/27039#issuecomment-4363363610).

1. Stop stripping output_config.effort on Bedrock + Vertex adaptive routes.
   - Vertex AI Claude 4.6/4.7 accepts output_config.effort on rawPredict
     (verified end-to-end against us-east5 / global). The strip helper now
     no-ops for effort.
   - Bedrock Converse routes output_config into additionalModelRequestFields
     for anthropic base models so the requested adaptive tier (low/medium/
     high/xhigh/max) actually reaches the wire instead of all collapsing to
     identical thinking.
   - Bedrock Invoke chat transformation (AmazonAnthropicClaudeConfig) stops
     popping output_config from the post-AnthropicConfig request body.
   - Bedrock Invoke /v1/messages allowlist (BedrockInvokeAnthropicMessagesRequest)
     now lists output_config so the runtime allowlist filter forwards it.

2. Validate effort across Bedrock Converse so 'disabled' / 'invalid' / '' /
   unsupported tiers (xhigh/max on Sonnet 4.6 or budget-mode 4.5 models)
   surface as a clean 400 BadRequestError instead of 500.

3. ValueError -> BadRequestError throughout (AnthropicConfig.map_openai_params,
   _apply_output_config, AmazonConverseConfig._handle_reasoning_effort_parameter).
   Empty-string effort is now rejected (was silently passing the
   'if effort and ...' short-circuit).

4. Floor reasoning_effort='minimal' at the Anthropic provider minimum
   (1024 budget_tokens) via new ANTHROPIC_MIN_THINKING_BUDGET_TOKENS so it's
   a usable tier on direct Anthropic / Azure AI Anthropic / Vertex AI Anthropic /
   Bedrock Invoke (all of which 400 below 1024).

5. model_prices: dedupe duplicate supports_max_reasoning_effort key on
   claude-opus-4-7 / claude-opus-4-7-20260416.

Adds regression tests across all five affected paths; existing tests asserting
the silent-strip behavior were updated to reflect the new pass-through and
clean 400 surfaces.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-05-03 04:18:50 -07:00
Mateo Wang
c011a7e3ba
Merge pull request #27041 from BerriAI/litellm_vertex-batch-error-response-null-46dd
Some checks are pending
Unit Tests: Caching (Redis) / caching-redis (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
fix(vertex-ai): set response=null on batch error entries per OpenAI spec
2026-05-03 04:08:42 -07:00
user
74b4eab364
fix(vector_store): cache use-time embedding-config resolution
Hold the resolved config in a process-memory TTL cache so the
request-handling path doesn't run litellm_proxymodeltable.find_first
on every vector-store call.
2026-05-03 10:27:53 +00:00
user
38511675b1
fix(vector_store): tighten registry-mutation comment and dedupe test helpers 2026-05-03 10:17:50 +00:00
user
b87c2f66a6
fix(vector_store): resolve embedding config at request time, never persist creds
The vector store create/update path previously called
``_resolve_embedding_config`` against the admin-configured router/DB
model and persisted the resolved ``litellm_embedding_config`` dict
(``api_key`` / ``api_base`` / ``api_version``) into the
``litellm_managedvectorstorestable.litellm_params`` column. Because the
resolver expanded ``os.environ/...`` references via ``get_secret``, the
DB row carried cleartext provider credentials, and the
``/vector_store/{new,info,update,list}`` responses returned them to any
authenticated caller who could supply a known admin model name.

Move the auto-resolve out of ``create_vector_store_in_db`` and out of
the update path. Persist only the user-supplied ``litellm_embedding_model``
reference. Resolve at request-handling time inside
``_update_request_data_with_litellm_managed_vector_store_registry`` so
the resolved config lives in the per-request ``data`` dict and is
garbage-collected after the response. Legacy rows that were created by
an earlier proxy version and already carry a resolved
``litellm_embedding_config`` skip the re-resolution and pass through
unchanged so embedding calls keep working.

The ``new_vector_store`` response now also runs the existing
``_redact_sensitive_litellm_params`` masker (already used by ``info``,
``update``, and ``list``), defending against caller-supplied cleartext
on the create path and against legacy rows whose persisted credentials
are still in the database.

Existing tests that asserted the old write-time-resolve behaviour are
updated to assert the new persistence shape (no embedding config
stored, just the model reference). Two new tests cover the use-time
path: one asserting fresh resolution happens when a row carries only
the model reference, the other asserting legacy rows with persisted
config skip re-resolution and continue to work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 10:08:28 +00:00
user
01323e8903
fix(auth): per-param allow must continue, not return early
A pre-existing logic bug in ``_check_banned_params``: when the
deployment-level ``configurable_clientside_auth_params`` permitted one
banned field, the loop ``return``-ed on the first match instead of
``continue``-ing, so any other banned param later in the same body or
metadata dict was never checked. This PR's metadata walk multiplies the
surface where that bypass matters — a body pairing an allowed
``api_base`` with an observability credential like ``langfuse_host``
would silently pass.

Proxy-wide ``allow_client_side_credentials`` keeps ``return`` (it's a
global opt-in for every banned param). The per-param branch becomes
``continue`` so only the one explicitly-permitted field is skipped.

Adds a regression test that exercises the api_base + langfuse_host pair.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 09:18:17 +00:00