* fix(guardrails): add fail-open mode to CrowdStrike AIDR guardrail
Add a fail_on_error param (default True, preserving existing behaviour) to
the CrowdStrike AIDR guardrail, mirroring model_armor and generic_guardrail_api.
When fail_on_error=False the guard fails open only on server errors (5xx) and
connectivity failures, so the request proceeds unmodified. Caller-controlled
4xx responses and result.blocked policy blocks always fail closed. The
applied-guardrails header is recorded even on the fail-open path.
* fix(guardrails): fail open AIDR 4xx
* refactor(guardrails): isolate AIDR fail-open
* style(guardrails): format AIDR fail-open
* ci: satisfy unit workflow timeout invariant
* refactor(guardrails): accept AIDR mappings
* test(guardrails): inject AIDR HTTP client
* fix(guardrails): harden AIDR fail-open against delivered verdicts and record fail-open status
Reads the blocked verdict from the raw body before guard_output validation so schema drift or a changed verdict type cannot fail open past a delivered block. A transformed response that cannot be parsed fails closed so delivered redactions are never dropped. Fail-open runs record guardrail_status guardrail_failed_to_respond with timings instead of success. Restores the fail-open behavior tests dropped mid-PR and reverts the payload Mapping widening
* test(guardrails): cover fail_on_error wiring and fail-closed default for CrowdStrike AIDR
* chore(guardrails): annotate the transformed-drift detail payload for the LIT002 budget
---------
Co-authored-by: abrekhov <abrekhov@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): dry-run an auto-router config against the backend before saving it
Both auto-router forms built a payload and posted it, so anything the write gate
refused came back as a raw 400 with the backend's message buried in it. They now
POST the exact payload to /auto_router/validate_complexity_router_config first
and surface its verdict inline.
One dryRunRejection owns the gate, and it reads valid alone. The verdict's two
fields arrive independently, so gating on the error message would let a rejection
that carried none through to the write. A transport failure fails open as valid,
leaving the write gate authoritative rather than blocking a save on a flaky
network.
Applies to every auto-router, built-in tiers included.
* fix(ui): hold the auto-router create closed for the full dry-run and create sequence
A second submit while the dry-run round-trip was pending started another
create against the non-idempotent /model/new. The submit handler now
refuses re-entry and the button disables for the whole sequence, matching
the edit modal's loading guard. Also drops the explanatory comments this
PR had added.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Route records below WARNING to stdout (WARNING and above stay on stderr),
emit ANSI color codes only when both streams are a TTY (honoring NO_COLOR),
and parse JSON_LOGS strictly so JSON_LOGS=false no longer enables JSON logs.
* fix(presidio): chunk oversized text before /analyze so large content blocks do not fail
The Presidio PII guardrail sent each content block to the analyzer as a
single /analyze call with no size check. Analyzer deployments commonly cap
the request body (the reporting deployment rejects bodies over 1,000,000
bytes with HTTP 413), so large blocks failed closed, and analyzer latency
grew linearly with payload size.
analyze_text now splits texts larger than presidio_analyze_chunk_size_bytes
(default 500,000 UTF-8 bytes, configurable per guardrail) into overlapping
chunks, analyzes them concurrently, remaps each detection's start/end onto
the original text, and deduplicates detections from the overlap regions.
Anonymization, blocked-entity checks, score filtering, numbered-token
unmasking, telemetry, and the dashboard entity positions all consume the
remapped global offsets unchanged.
Resolves LIT-4785
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(presidio): review-round hardening for chunked analyze
- measure the chunk budget on the JSON-serialized text (non-ASCII escapes
expand beyond raw UTF-8, so a raw-byte budget could still exceed the
analyzer body limit)
- share the chunk fan-out semaphore per event loop and instance instead of
per call, so many oversized blocks cannot multiply concurrent analyzer
calls
- apply configured score thresholds and deny list per chunk BEFORE overlap
resolution, so a below-threshold span cannot displace a detection the
thresholds keep
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The /v1/messages bridge decided a Claude target could take `reasoning_effort` from
the model name, which says nothing about the params the provider in front of it
accepts. Snowflake serves Claude over the Anthropic dialect and declares `thinking`
alone, so `get_optional_params` raised `UnsupportedParamsError` before the request
reached the wire: every adaptive request carrying an effort tier turned a 200 into
a 400 for all seven of its Claude entries.
The tier is now offered only where the target declares the param, reading the same
`get_supported_openai_params` the sibling `_supports_prompt_cache_key` reads twelve
lines up. A target declaring neither carrier keeps its bare `thinking` block, which
is what this bridge sent before it carried a tier at all.
Without a resolved provider the tier stays behind rather than being offered blind.
Resolving one from the model's prefix instead would run an OAuth device flow for
github_copilot and chatgpt, blocking for minutes, and one of the two callers in that
position is a logging callback. The copilot case is pinned by a test.
* feat(ui): session-level cache observability in request logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: guard cache_hit filter against non-string defaults in direct calls
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): drop redundant cache_hit field comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A complexity-router setting placed beside complexity_router_config, or inside a
tier entry's litellm_params, is read by nobody: the router loads its settings only
from litellm_params.complexity_router_config. It does not stay inert. The
alias-marker forwarding and the per-tier param spread carry every unrecognized key
onto the outbound request, and all_litellm_params only knows the outer names, so
the key reaches the provider as an unknown body field and every call through that
model group fails with an error naming an internal config key.
Guard the whole set, derived from ComplexityRouterConfig.model_fields so a field
added later is covered, and scoped to complexity-router deployments because the
names only mean this there (embedding_model is a legitimate flat param on an
s3_vectors vector store). Scope is read from the same merged field view the naming
check is judged on, so a router named only by its default model is in scope and a
field added to the required-field table is covered without another edit. The write
endpoints reject with a 400 naming the keys and where they belong, config.yaml
refuses to start for the same reason max_agentic_loops does, and a tier entry is
judged by the config model itself.
An already-stored deployment keeps loading, so an upgrade cannot take a running
gateway down over a row that was written before the gate existed.
Two lazily loaded models changed without their generated artifacts being
regenerated, so check-ui-api-types has been red on every branch off staging.
The snapshot that /openapi.json serves for unloaded features was missing
ChatCompletionToolReferenceObject, and the dashboard types were missing
aws_external_id. The snapshot step runs first and short-circuits, so only the
first one was visible until it was fixed.
Both files are regenerated with `python -m litellm.proxy._lazy_openapi_snapshot`
and `npm run gen:api`, no hand edits.
* feat(alerting): add native Microsoft Teams alerting destination
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(alerting): preserve active destinations on MS Teams save and confirm health test delivery
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): read persisted alerting destinations at MS Teams save time
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
/v1/messages forwarded `thinking` verbatim for a Claude-family model and then returned,
carrying `output_config.effort` only when the model string started with a Bedrock prefix.
Every other bridged provider got a bare adaptive thinking block, so the caller's effort did
nothing: max and minimal produced byte-identical upstream bodies.
Send those targets the tier as `reasoning_effort`, which is the param they take. Bedrock keeps
taking `output_config`, since the two are not interchangeable there: an application inference
profile ARN resolves to no chat config, so `reasoning_effort` is dropped and the tier vanishes,
and a provider that rebuilds `output_config` from it overwrites a caller-set `thinking.display`
on the way. The tier stays a plain string, the summary already travelling inside the forwarded
`thinking` block. Adaptive with no tier, and budgeted thinking, both stay exactly as they were.
Kimi K3 accepts exactly low, high and max, defaults to max, and always thinks.
The map could not say that: medium and high have no supports_*_reasoning_effort
flag because every other reasoning model takes them, so the ten kimi-k3 entries
carried supports_reasoning alone and resolved to unknown. The dashboard then fell
back to a capability-blind level list that deliberately omits max, which is why a
kimi-k3 tier cannot be set to max thinking today.
Add reasoning_effort_levels, an array key in the shape the map already uses for
supported_endpoints and supported_modalities. Where present it is read first and
wins whole; every other entry keeps answering through the per-level flags,
unchanged. It is deliberately a different name from the computed
ModelGroupInfo.supported_reasoning_efforts, which stays derived from a group's
deployments and is never seeded from one deployment's model_info.
The levels are per entry rather than per model, because the deployments differ:
Moonshot, Together, Fireworks and Azure Foundry all forward the level unchanged
and get the model's own low/high/max, while Perplexity documents a six-value
enum it maps down internally and gets that. The /v1/messages degradation chain
consults the same declaration, so the level the map advertises is the level that
path forwards.
A non-ProxyException from the team, project or access-group lookup used to
escape the fallback loop and replace the provider's error. Treat it as a
denial and log it. Also drop the unrelated reformatting of test_router.py
and test_fallback_event_handlers.py so both diffs are additions only.
An MCP server behind an API gateway needs two credentials on one request: the
gateway's own token on a private header, and a separate bearer on Authorization
for the server behind it. Every arm that minted or held a token hardcoded
Authorization, and the conflict rule then dropped the operator's static
Authorization to make room, so the second credential never arrived.
ApiKeyConfig already modelled this as header_name plus value_prefix behind a
header() method. Extend that carrier to the four minted-token configs, have each
resolver arm ask its config which header to use instead of naming one, and drop
only the header the resolved credential is about to occupy.
Operators set it per server via upstream_token_header, plumbed through
config.yaml, the credentials blob, the management API and the admin form, on the
M2M, token-exchange, authorization-code and ID-JAG arms. It is non-secret so it
stays plaintext and round-trips on admin reads. Unset keeps today's behaviour.
Moving a credential off Authorization means it stops inheriting what Authorization
gets for free, so the slot now carries those protections itself. httpx drops
Authorization when a redirect crosses origin and keeps every other header, so a
custom slot is dropped by the client on the same condition, mirroring httpx's own
scheme/host/port rule with an agreement test that fails if the two ever diverge.
The v1 path also mirrors the v2 conflict rule, so an injected header cannot shadow
the credential the gateway resolved for that slot.
Which header a credential occupies, and what counts as being that header, was
answered independently in nine places by four hand-rolled comparisons. same_header,
has_header and without_header in litellm/types/mcp.py are now the one owner, shared
by both MCP stacks, and the client derives its slot once instead of three times.
The header name reaches egress verbatim, so the RFC 7230 grammar lives in one
place and is checked where servers are built: a bad value fails the config load
and the management API returns 400, rather than raising while a spec is built
and emptying the aggregate tool list for every other server. A blank means unset,
matching what the endpoint already accepts.
The two vision tests pointed at a Wikipedia-hosted cat photo, so every run
depended on upload.wikimedia.org staying up and unthrottled. It throttled,
and the 429 surfaced as a bedrock APIConnectionError, which reads as a
gateway failure rather than what it was.
The image is now a fixture in the repo, passed as a data URL. That also puts
the two providers on the same bytes: litellm downloads the image itself for
bedrock, while openai is handed the link and fetches it from its own servers,
so the hosted URL quietly meant the two tests were not testing the same thing.
The image was generated for this repo rather than borrowed, so nothing here
carries a third-party license. Also drops a stale comment about openai prompt
caching that sat above the vision helper; no caching test uses it.
Migrate the create-key user picker, add-member user search, and usage team filter onto the shared paginated selects, gate the logs error-code filter on input reasons, and add clearAllLabel, autoHighlight, and aria-required passthroughs the migrations need.
Router fallbacks configured in router_settings were attempted without
re-checking whether the calling key could use the fallback model, so a key
limited to one access group was served by any model listed as a fallback
for something it could call. Auth only validated the requested model and
fallbacks sent in the request body.
Add a fallback_access_check predicate to Router, consulted before every
cross-model-group fallback attempt; rejected targets are skipped and the
primary's own error is raised when none remain. The proxy injects a check
that runs the same key, team and project model access checks the requested
model goes through.
Select the picked label on focus and snapshot whether the pre-edit selection covered the whole input; when it did, the next input value is a full replacement, so skip the typedInsertion diff that mangles pastes sharing a prefix or suffix with the label.
Under otel_v2, a client that propagates W3C trace context in params._meta
(SEP-414) pulled the tools/call span out of the gateway's trace:
resolve_mcp_span_context parented the MCP span to the client's remote
context and demoted the gateway's own transport span to a span link. The
gateway's tracing backend only ever receives the gateway's half of such a
trace, so the span was unreachable from the trace view and the POST
transaction showed a dangling link.
Invert the anchoring: the MCP tool-call and tools/list spans now always
nest under the transport span of the request carrying the message, and the
client's propagated context is recorded as the span link instead, so the
correlation survives while every trace stays renderable. With no transport
at all the span roots its own trace and still carries the link, keeping a
single shape for the event. Both returned contexts are built on an
explicitly empty base so ambient session state can never leak in, and the
span inherits the transport's sampling decision like every other
request-level span.
The /v1/messages validator checked a tool_use block's name and id but not its
input, so a block whose location came back empty or wrong still passed, while
the chat side rejected the same damage. That gap predates this branch; it is
worth closing here because the point of the change is that every parallel call
is checked rather than counted.
AnthropicContentBlock now declares input as a typed field. It already survived
on extra="allow", but reaching it from a test needs a real field to keep the
e2e basedpyright gate at zero. Serialization is unchanged: bodies are dumped
with exclude_none, so a block without an input still replays exactly as before.
