* feat(complexity_router): plan-mode tier floor for coding-agent clients
Claude Code and Copilot signal plan mode only through client-injected prompt
text, which the ask-extraction path deliberately strips, so the router could
never see it. Detect the sentinels on the raw wire body and route those
requests to at least plan_mode_min_tier.
The floor is raise-only and transient: classifier results above it still win,
it overrides a session-affinity pin only on turns carrying the sentinel
without rewriting the pin, and plan_mode decisions are not pinnable, so the
first turn after plan mode exits routes as if plan mode had never happened.
Classification is skipped when the floor is the top configured tier. On
adaptive routers the floor rides _soft_floor_pick as a hard_floor that
excludes below-floor candidates, closing the adaptive_eligible=all gap where
a request classified at or above the floor could still route below it.
Detection is staleness-aware: only leading system content and the newest-ask
tail count, so sentinels surviving in history after plan mode exits, built-in
or operator-supplied, never fire. Custom tier sets are supported with
severity from the tier_definitions list order, same as keyword_tier_rules.
Off by default; decisions are recorded with the new plan_mode cause and the
matched sentinel in matched_keyword
* fix(complexity_router): gate pin writes and the failure exit on sentinel presence, not the floor binding
A plan-mode turn classified at or above the floor keeps its ordinary cause,
but pinning it would carry a plan-mode-shaped choice past plan mode's exit
(on adaptive routers the hard floor constrained that pick), so no
sentinel-carrying turn writes the session pin. The default_model failure
exit is skipped for sentinel turns for the same reason: default_model's
placeholder tier can equal the floor while default_model itself sits in no
pool the floor can vouch for
* fix(proxy): strip callback credentials from the auth object stamped into request metadata
* style(proxy): drop the restating half of the stamp-site comment
* test(proxy): pin that the stamped auth copy carries header-derived identity
GET /guardrails/usage/overview and GET /guardrails/usage/detail/{id} 500ed on a
database that has not applied 20260817143646_add_daily_guardrail_usage_units yet
(pip installs on litellm-proxy-extras 0.4.86 with DISABLE_SCHEMA_UPDATE=true).
Both endpoints now return their metrics with empty units and log one warning
until the migration lands.
The daily guardrail metrics and usage-unit upserts are non-idempotent
increments, but the retry loop re-sent every failed row on any exception.
An ambiguous post-send failure such as a read timeout after the write had
already committed therefore stacked a second increment and inflated the
billable unit totals served by the guardrail usage endpoints.
Retry only DB_RETRY_SAFE_ERROR_TYPES (httpx.ConnectError), the same rule
the spend writer and autorouter rollup use for increment upserts, and log
any other failure once as terminal for that row while the rest of the
batch still lands.
Follows up #37225
A pass that raises (a missing Slack webhook, say) now waits the daily interval instead of logging the
same exception every 30 seconds, and the Admin UI alerting settings list the new alert type so it can be
toggled like the others
The trailing-slash normalization test used gateway.litellm-sandbox.ai as
its base URL. Swap it for gateway.example.com so the test file does not
reference a real-looking hostname. The test is fully mocked, so the host
value has no effect on what is exercised.
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
resolve_fireworks_resource_name prefixes bare names with
accounts/fireworks/models/ (or routers/ for *-fast). Azure AI Foundry
hosts Fireworks models under deployment ids like FW-Kimi-K3; rewriting
those yields 404 DeploymentNotFound.
Leave names that already start with FW- unchanged. Native Fireworks
short names still get the accounts/ path.
Co-authored-by: Cursor <cursoragent@cursor.com>
A transient DB error during the spend log flush dropped that batch's guardrail
metrics and usage unit rows for good. Retry only the rows that failed, up to 3
times with 1s/2s/4s backoff, mirroring the daily spend writer, and inject the
sleep so tests stay fast. Lowers the lint budgets the refactor freed up
An empty pass no longer holds the daily lock, a False lock claim (held or redis
error) is retried on the next 30 second poll instead of sleeping a day, and a
sent alert is stamped in the shared cache for a day so sibling pods and restarts
stay quiet
The flush and the usage endpoints summed units with a scan per distinct key,
quadratic in rows times keys; group sorted rows instead. Skip payloads without
a request_id like the metrics path, type the flush key as a NamedTuple, and drop
the (guardrail_id, date) index that the primary key already covers
* fix(shadow_eval): copy messages before router call and raise judge output cap
* fix(shadow_eval): lead failure detail with location and pin post-failure continuation
Config-driven pass_through_endpoints pointed at a comprehendmedical.*.amazonaws.com
target were being claimed by the Comprehend Medical logging handler through the
hostname arm, which overrode their operator-set cost_per_request and relabeled
their spend rows. Only the built-in /comprehendmedical routes tag the provider,
so match on that alone.
Also mirror /comprehendmedical into the helm ingress and terraform gateway
prefix lists that hand-copy gateway/routes/allowlist.py
Per-row guards in the daily metrics and usage unit flush so a single DB error no longer drops the rest of the batch, plus removal of narrating comments flagged in review
The Noma guardrail sends the conversation to the scanner in `inputs`. It
also forwarded `request_data` whole, which repeats that same conversation
under `messages` (or `input` on the responses API), and attached
`logging_obj.model_call_details`, which repeats it a third time.
For image-heavy calls that duplication is most of the request. A
production scan of a request carrying base64 images measured 100MB total,
of which 94.8MB was `request_data` against 5.1MB of `inputs` - the proxy
was uploading ~95% redundant bytes, and paying to serialize them.
Drop `messages` and `input` from `request_data` and from
`model_call_details`. This is a denylist rather than an allowlist on
purpose: every other key is still forwarded untouched, so a scanner-side
change that starts reading a new `request_data` key needs no matching
release of this hook. The removed keys are ones the scanner never reads -
it takes context only from metadata, litellm_metadata,
provider_specific_header, litellm_session_id/trace_id/call_id, stream,
response/responses ids, and litellm_logging_obj.complete_streaming_response,
all of which still pass through.
The conversation still reaches the scanner in full via `inputs`, so no
detection coverage changes.
Trimming happens before serialization, so the duplicate is never encoded.
Existing payload tests asserted the duplication; they now assert the trim
while keeping what they originally guarded - deep-copy semantics and the
unpicklable-object (uvloop.Loop) regression.
* fix(mcp): scope authorization server issuer
Generated with AI
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(mcp): keep the bare-origin issuer when no server was named
The scoped issuer must key off whether the request actually carried a server
name. _build_oauth_authorization_server_response rebinds mcp_server_name when
root discovery resolves the single configured OAuth2 server, so gating on the
rebound value also scoped /.well-known/openid-configuration, whose document is
served from the bare origin and whose issuer must stay the bare origin
Adds the named-server regression test for the reported mismatch, restores the
bare-origin assertion, and covers the OIDC document
* test(mcp): type the delegate_auth_to_upstream helper parameter
* refactor(mcp): bind the discovery issuer to a local before building the response
---------
Co-authored-by: Irosh <15094153+irosh-colombage-ZocDoc2@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
The field itself landed on staging via 0c5c9c79d7; these are the regression tests from PR #31435 for the retrieval-facing half.
(cherry picked from commit a9a322d63f6d4658b1f28d1622335775e94736a4)
Bedrock batch jobs write their results to s3_output_bucket_name when it differs
from the input bucket, but the file-content retrieval path validated the file id
only against the input bucket (s3_bucket_name). A deployment that configures a
separate output bucket therefore could not retrieve its own batch outputs: the
id validated against the input bucket and was rejected as a foreign bucket.
Resolve the trusted output bucket alongside the input bucket from the immutable
credential snapshot (or AWS_S3_OUTPUT_BUCKET_NAME), and try the file id against
each configured bucket, returning the first that validates. The SSRF guard is
preserved: only server-configured buckets are tried, never a request param, and
an id outside both is still rejected.
(cherry picked from commit 1d407c2f26)
