PR #38593 stopped forwarding temperature to reasoning models, which left
test_extra_body_merges_with_request_data raising UnsupportedParamsError
and test_bad_request_bad_param_error no longer getting a rejection from
OpenAI because drop_params now eats the param. Both repairs are the same
hunks PR #38739 carries, so the branches merge clean in either order
A guardrail block or failed scan that fires after SSE chunks have been
flushed can no longer set an HTTP status, so raising HTTPException there
silently truncated the stream. _emit_streaming_http_error now routes
post-flush failures through the endpoint translation's
build_stream_error_items, emitting the surface-correct error frame on
chat completions (data: {error}), /v1/messages (event: error), and
/v1/responses (ErrorEvent with the next sequence number). Pre-flush
blocks still raise with a real HTTP status.
Successful flags-on scans also logged metadata.guardrail_information as
null: the chat handler planted litellm_metadata on a route whose bucket
is metadata, flipping the bucket for every later write, and responses
streams fired their spend log before the eos scan ran. The chat handler
now merges user_api_key metadata through get_or_create_metadata_bucket,
and deferred stream-complete logging is armed for aresponses like it
already was for anthropic_messages.
Adds litellm/litellm_core_utils/aws_partition.py mapping a region to its
AWS partition (aws, aws-cn, aws-us-gov, and the iso partitions), its DNS
suffix, and its ARN prefix, and uses it at every AWS host and ARN build
site: bedrock (runtime, agent, agentcore, legacy client, batches, files,
realtime), sagemaker, polly, secrets manager, s3 log uploads, bedrock
passthrough routes, and rag ingestion. ARN detection now accepts
arn:aws-cn: and arn:aws-us-gov: prefixes.
STS region resolution now falls back to the configured aws_region_name
after the aws_sts_endpoint host and the AWS_REGION/AWS_DEFAULT_REGION env
vars, so cn and gov role assumption no longer silently signs against
us-west-2.
A partition sweep test walks every endpoint builder with cn regions and
asserts no amazonaws.com host or arn:aws: prefix comes out, plus an AST
guard that fails on any new f-string hardcoding either literal.
Same-day debt cleanup on code that landed in the last 24 hours. No behavior change.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Streamed /v1/responses returned 500 whenever a Bedrock post_call guardrail
was enabled: the hook fed responses-API events into stream_chunk_builder,
which only understands chat-completions chunks, and the wrapped KeyError
surfaced as litellm.APIError before any ApplyGuardrail scan ran.
Delegate responses-API routes to UnifiedLLMGuardrails, whose translation
layer scans the assembled response at end of stream and only then releases
the buffered events, so flagged content never reaches the client.
* feat(terraform): add litellm_jwt_key_mapping resource
Adds a Terraform resource for the proxy's JWT to virtual key mappings, so a
JWT client identified by a claim such as client_id, azp or sub maps to a
virtual key and inherits its models, budgets, rate limits and spend tracking.
Covers the four mapping endpoints: /jwt/key/mapping/new, /info, /update and
/delete. is_active is applied through a follow-up update because the create
endpoint always starts a mapping active, a dropped description is sent as an
empty string because the update endpoint ignores absent fields, changing the
mapped key rotates it in place, and changing the claim name or value forces
replacement since the update endpoint cannot change them.
* fix(terraform): revert key on failed jwt_key_mapping update
Classic SDKv2 persists a failed Update's diff-applied values to state
regardless of the error, so a rejected key rotation left the new key in
state while the proxy kept the old one and the next plan falsely converged.
Revert key via GetChange and resync description/is_active/computed fields
from a post-failure Read, since Read alone can't recover key (the proxy
never returns it).
Also drop the case-insensitive "mapping not found" body match: the proxy
raises 404 for all three not-found paths (info, update, delete), so
checking the status code alone is sufficient.
Clarify the docs: referencing a litellm_key resource's write-only key is
not a null-then-400 situation, it's a static "Missing required argument"
error at plan time, in every apply ordering.
* fix(terraform): stop leaving an active mapping behind on failed cleanup
Two issues flagged by review:
- Create has no way to ask the proxy for an inactive mapping, so an
is_active=false mapping is briefly active while the follow-up
deactivation runs. If that deactivation call itself fails, the mapping
used to stay active and untracked. It's now deleted instead, closing
the exposure rather than leaving it open indefinitely.
- On a failed update, only `key` was reverted before the recovery read.
If that read also failed, description/is_active kept the rejected
values, so a later plan could report false convergence. Now all three
are reverted before the read runs.
Both come with regression tests, mutation-verified against the pre-fix
code.
* fix(deps): bump restrictedpython to 8.5 for GHSA-ffg3-p8fm-mjx2
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: retrigger ci
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(tests): stub anthropic judge credentials in funnel seeding test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* revert(deps): keep uv.lock unchanged to keep the PR terraform-only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Fabrice Pont <fabrice.pont@doctolib.com>
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(terraform): close key/team schema gaps, fix team read envelope, add import support
* feat(terraform): add fallback resource/data source and key/team block resources
* feat(terraform): add access group and unified access group resources and data sources
* feat(terraform): add guardrail and prompt resources and data sources
* feat(terraform): add agent and search tool resources and data sources
* feat(terraform): add user and budget resources and data sources
* feat(terraform): add tag and project resources and data sources
* docs(terraform): changelog and readme for parity additions
* feat(terraform): add data sources for keys, teams, models, organizations, and mcp servers
* fix(terraform): key update 400 on empty budget_duration, key info envelope, config-supplied key
* fix(terraform): hash raw keys to SHA-256 tokens in key lookup URLs and block resource IDs
RestrictedPython 8.3 extended its protected-name validation to cover
positional-only parameters, so sandboxed source can no longer bind a
local named _getattr_, _getitem_, _write_ or _print_ that takes
precedence over the hooks the custom-code guardrail sandbox installs.
8.4 and 8.5 continue in the same direction with safer_getattr and the
Python 3.15 syntax audit.
The floor moves rather than the lock alone so downstream installs of
litellm[proxy] pick up the same behaviour.
The attempt row now prices the real arm (the payload's response_cost plus its own
routing classifier when it routed) beside the shadow arm (completion plus the
classifier cost the routing decision writes back), and flags turns litellm's
response cache served. A per-leg funnel table counts the eligible requests that
produced no row (lost the sampling dice, unjudgeable shape, concurrency shed),
so results can weigh judged rows against the traffic they stand for. Job results
gain per-slice and overall arm spends plus the coverage counts, the budget gates
charge the shadow arm's classifier spend against max_budget, and the dashboard
shows the measured cost comparison beside the win rate
Resolves LIT-6358
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(proxy): reset a key's budget-window counters and broadcast the reset cross-pod
/key/{id}/reset_spend already reset the key's lifetime spend counter in
Redis, but a key with its own budget_limits (an extra time-windowed cap,
e.g. a daily budget layered on top of the lifetime max_budget) kept its
window counter untouched, so the key stayed 429'd on
"ExceededBudget: Key over <duration> budget" even after the admin action
reported spend back to $0.
Force-expire each window on reset: zero its Redis counter and restart the
window from now (window_start is derived as reset_at - budget_duration,
so reset_at must float to now + duration, not the next calendar boundary
get_budget_reset_time gives key creation - that boundary can still be
in the past relative to the spend that triggered the block).
Also close a second, narrower race: _delete_cache_key_object evicted the
cached key object only on the handling pod, so another pod could keep
serving the stale pre-reset object (and re-derive the pre-reset spend
counter via its own floor-marker cache) until its own TTL expired. It now
broadcasts the eviction, matching the pattern already used for team,
team-member, customer, and tag caches.
* fix(proxy): evict the cached key object after every reset_spend DB write
Greptile P1: eviction ran before the window-reset DB write committed, so a
request racing the reset could re-fetch and re-cache the pre-write row,
pinning that pod to the stale budget_limits for the rest of its own cache
TTL even after the write went through. Move the eviction to run last.
* test: pin real cache state and satisfy the test-quality gate
test_delete_cache_key_object_broadcasts_invalidation now asserts a real
UserApiKeyCache no longer holds the evicted entry, rather than only
inspecting a mock's call args. Suppress test-quality-ok on the
hash_token/_check_proxy_or_team_admin_for_key/_delete_cache_key_object/
publish_auth_cache_invalidation patches: none has an HTTP boundary to
fake, matching the pattern the file already uses for these same targets.
* fix(proxy): narrow budget_limits by the str branch, not the list branch
isinstance(x, list) in the else branch still leaves Sequence[object] | str
(a tuple satisfies Sequence without being a list), so json.loads() saw a
possible non-str argument. Check isinstance(x, str) instead, which narrows
each branch to exactly the type it needs.
* fix(proxy): persist advanced budget-window boundaries before zeroing counters
Greptile P1: publishing a zeroed window counter before the new reset_at
committed let a request racing the write compute window_start from the
stale boundary, re-sum the historical spend log rows the reset was
clearing, and put the counter right back above budget. Compute every
window's new boundary, persist all of them in one DB write, then zero
each window's Redis counter only once that write has landed.
An auto-router tier pin injects reasoning_effort into request_kwargs, but
provider translations give a caller-supplied thinking, output_config.effort,
or reasoning carrier precedence over the reasoning_effort alias, so the pin
never reached the wire whenever the client expressed effort natively. Drop
the client's other encodings of the setting at the tier-param merge; a
client output_config keeps its non-effort fields
