Setting reasoning_effort="none" on Anthropic chat models (direct, Bedrock
Invoke, Bedrock Converse, Vertex AI Anthropic, Azure AI Anthropic) crashed
LiteLLM with:
litellm.APIConnectionError: 'NoneType' object has no attribute 'get'
Both the Anthropic chat transformation and Bedrock Converse called
``AnthropicConfig._map_reasoning_effort`` and assigned the ``None`` it returns
for ``"none"`` directly to ``optional_params["thinking"]``. Downstream
``is_thinking_enabled`` then did ``optional_params["thinking"].get("type")``
and crashed.
Pop ``thinking`` (and on Claude 4.6/4.7, ``output_config``) instead of
assigning ``None``, restoring the documented contract that
``reasoning_effort="none"`` means "do not enable thinking". This also
prevents downstream Anthropic 400s ("thinking: Input should be an object",
"output_config.effort: Input should be ...") if the bug were ever masked.
Verified end-to-end against the live Anthropic API and Bedrock Converse
on claude-opus-4-{5,6,7} and claude-sonnet-4-6, plus Bedrock Invoke for
Claude 4.5/4.6. Vertex AI Anthropic and Azure AI Anthropic inherit the
fixed ``map_openai_params`` from ``AnthropicConfig`` and need no further
changes.
Adds litellm.disable_vertex_batch_output_transformation (default False).
When True, afile_content returns raw Vertex predictions.jsonl untouched
so users that parse candidates/modelVersion directly are not broken.
The method was overwriting logging_obj.optional_params, logging_obj.model,
and logging_obj.start_time on the caller's Logging instance. When invoked
from llm_http_handler.py's generic framework path, the framework's own
logging_obj (which already went through pre_call) had its properties
clobbered, causing model and start_time to reflect the last batch line's
values rather than the original call context.
Fix: create a fresh local Logging instance for the per-line transformation
instead of mutating the incoming logging_obj. The caller's object is now
left entirely untouched regardless of whether a logging_obj was passed in
or not.
Regression tests added to verify model, start_time, and optional_params
are not mutated on the caller's logging_obj.
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
- Remove unused _encode_gcp_label_value / _decode_gcp_label_value singular
helpers; only the _chunks variants are actually called.
- Use 'is not None' check for custom_id so empty-string custom_ids are
still labeled and round-trip through batch outputs.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Newer Cloudflare Workers AI models (e.g. Nemotron) emit 'response_text'
instead of 'response' on streamed chunks. The non-streaming path was
already updated to fall back to 'response_text' (#26385), but the
streaming chunk parser still only read 'response', which caused
streaming requests against those models to silently produce empty
content.
Mirror the non-streaming fallback in CloudflareChatResponseIterator.chunk_parser
and add a streaming test for the response_text shape.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
A SCIM PUT may legally omit `active` (full-replace with the field
absent). Pydantic fills the SCIMUser.active default of True, so the PUT
handler was overwriting metadata.scim_active with True even when the
client never sent it — silently reactivating a previously SCIM-blocked
user and unblocking their keys.
Use model_fields_set to detect whether the client actually sent
`active`. If omitted, preserve the prior scim_active value and skip
the cascade to virtual keys.
Also drop comments added in this PR that just narrate what the code
does; keep only the docstrings and the SQL-NULL pitfall note that
explain non-obvious behaviour.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Container ownership and skill rows are looked up on every retrieve /
delete / list / file-content / chat-completion-with-skill call. The new
stores wrapped raw Prisma queries with no cache, putting one DB
round-trip on each request. Add an in-process TTL'd cache mirroring the
_byok_cred_cache pattern in mcp_server/server.py: per-key (value,
monotonic_timestamp), 60s TTL, 10000-entry cap with full-clear on
overflow, invalidated by every write. Negative results (`None`) are
cached too so untracked-resource checks also skip the DB.
Tests cover: cache-after-first-hit, negative caching, write
invalidation, no-caching-on-DB-error, TTL expiry, capacity eviction.
56 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SCIM DELETE /Users/{id} previously called litellm_usertable.delete without
clearing rows that FK back to the user, so Postgres rejected the delete with
LiteLLM_InvitationLink_user_id_fkey and the SCIM caller saw a 500. Add a
helper to drop invitation_link, organization_membership, and team_membership
rows before the user delete (mirrors /user/delete in internal_user_endpoints).
Also add a Status column to the Virtual Keys and Internal Users tables so
admins can see at a glance which keys are blocked and which users SCIM has
deactivated. SCIM-blocked keys carry a tooltip explaining the origin.
Pin the dashboard's Node version to 20 via .nvmrc to match CI.
base_anthropic_messages_test.test_anthropic_messages_with_thinking and
test_anthropic_streaming_with_thinking still pinned to
claude-4-sonnet-20250514 — the same legacy alias Anthropic no longer
recognizes under freshly issued keys. The other four tests in this base
class already use claude-sonnet-4-5-20250929; these two were missed.
Bump to claude-haiku-4-5-20251001 (supports_reasoning=true, no upcoming
deprecation). Subclasses including TestAnthropicPassthroughBasic
inherit these methods.
The blocklist check in _check_proxy_admin_viewer_access only fires for
routes that match LiteLLMRoutes.management_routes — the bulk-update
endpoint was missing from that list, so the test for view-only admins
on /team/permissions_bulk_update fell through to "allow."
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
If record_container_owner raises after the upstream container is created,
the user previously got a 500 with no usable container — they were billed
for an unreachable resource. Move ownership recording into the create
path's exception handling and split the two failure modes:
- HTTPException from the recorder (auth conflicts) propagates verbatim
so the client sees the real status code, not a generic LLM error.
- Unexpected exceptions are logged and swallowed; the response is
returned to the caller so they aren't billed for a container they
can't address. The DB row stays untracked until an operator reconciles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
test_anthropic_messages_streaming_cost_injection hits the proxy's
/v1/messages route, which routes via the anthropic/* wildcard to
api.anthropic.com. The 404 surfaced in the test was Anthropic's own
not_found_error propagated back through the proxy (visible from the
x-litellm-model-id hash on the response — the proxy did route).
Same root cause as the prior commit: the legacy claude-4-sonnet-20250514
alias is no longer recognized by Anthropic's main API under the new key.
Swap to claude-haiku-4-5-20251001 — same routing path, canonical model.
The previous per-provider guards in HuggingFace, Oobabooga, and Gemini
files lived in the SDK layer, breaking SDK callers who legitimately pass
URL-valued model identifiers. Move the check to the proxy boundary in
add_litellm_data_to_request so SDK users keep working while proxy users
default-deny URL-valued model and file_id, with admin opt-in via
litellm.provider_url_destination_allowed_hosts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three live-API tests pinned to claude-4-sonnet-20250514, which is a
non-canonical alias of claude-sonnet-4-20250514. Anthropic's main API
no longer resolves the legacy form under freshly issued keys, so the
tests fail with not_found_error. The token counter test pinned to
claude-sonnet-4-20250514 itself (deprecation_date 2026-05-14, two weeks
out) was on borrowed time too.
Bump all four to claude-haiku-4-5-20251001 — capability superset for what
these tests exercise (streaming, parallel tool calling, extended thinking,
token counting), no upcoming deprecation, cheaper per-token.
