TrustGuard reduces findings to block, ask, transform, report, or allow.
The hook only knew four of them, so a policy with an Ask gate made every
evaluation on that collector fail closed with 503 "unknown verdict",
which reads as an outage rather than a policy decision
A proxy has no approval flow to hand the question to, so ask now raises
the same 400 as block. The response carries the verdict so operators can
tell the two apart in the error body and in Activity
The config model now declares timeout (default 5 seconds), so the
Admin UI create and edit forms render a number input for it and
/guardrails/ui/provider_specific_params advertises it. The shared
LitellmParams.timeout keeps its None default for every other guardrail
because BaseLitellmParams precedes this model in the MRO, and the hook
rejects a non-positive value at startup
Every evaluate call now carries consumer_id so TrustGuard Activity and
per-consumer policies group by the LiteLLM key rather than by
conversation. The identity is resolved tier by tier across both
metadata blocks the proxy populates: key alias first, then the key's
user email, user id, and team alias. The unified guardrail path seeds
litellm_metadata with the alias under user_api_key_key_alias while the
request metadata uses user_api_key_alias, so both names are accepted
and only string values are ever sent
Tests build request_data with the proxy's own helpers for the chat
completions shape and the seeded-only shape used by MCP and pass-through,
pin the fallback order, and cover the UI field set and the timeout
defaults
The base branch dropped the soft_budget docstring from the user endpoints without regenerating the types, and the schema check runs on this PR because it touches litellm/types
When TrustGuard transformed a response into messages whose content was
empty or null, the hook dropped those messages while rebuilding texts and
then fell back to the original inputs, so a fully redacted completion
reached the client unredacted. Returning an empty list would not help
either: the chat translation handler skips the write-back when the
returned texts are empty
Texts are now rebuilt one per returned message, with "" for empty or
non-string content, and the fallback is gone. That keeps the positional
alignment the handlers rely on when they map texts back onto choices, so
a redaction that empties only the first of two choices no longer shifts
the second choice's text onto the first. When the handler sent no texts
at all (a tool-call-only completion) the hook keeps texts as sent instead
of inventing one for the placeholder message, which the Anthropic and
Responses handlers would index out of range
A transform whose message count differs from what the hook sent is now
rejected with the same 400 the tool-call count mismatch already raises.
Fewer messages used to leave trailing choices unredacted and more used to
crash the handler write-back with an IndexError
Regression tests cover the empty and null cases in both directions, the
two-choice alignment through the OpenAI chat translation handler, the
tool-call-only reply, and both count mismatches
CredentialLiteLLMParams omitted tenant_id, client_id, client_secret,
azure_scope, azure_username and azure_password, so the strict dump used
by credential reuse and Azure client init dropped them and the reused
credential ended with no auth at all
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The e2e harness exists to prove product features end to end against a live
proxy. The prior Hard Rule carved out an exception for "tests that cover the
harness itself" and pointed at coverage_registry/test_collector.py, which in
practice invited unit tests of harness helpers to be staged alongside e2e
work. That is the wrong tool: harness logic that is worth locking down does
not need a mock-driven unit test living under tests/e2e.
Drop the carve-out. The Hard Rule now reads that no unit tests of any kind
belong under tests/e2e, and the passing mention of unmarked harness coverage
in the transport section is removed so the doc no longer contradicts itself.
coverage_registry/test_collector.py still exists on disk and is left in place
for now; whether to relocate or remove it is a separate decision.
Keeps the base's rule that a non-admin id lookup matching no spend-log row answers 403, so the detail route never consults cold storage without an owner row
* fix(proxy): bound tool and guardrail index create_many by the spend-log statement budgets
One flush drains up to MAX_LOGS_PER_INTERVAL source transactions or logs, but a
transaction fans out to one LiteLLM_SpendLogToolIndex row per tool and a log
to one LiteLLM_SpendLogGuardrailIndex row per guardrail, so the index
create_many payload was unbounded. Both index writes now go through
spend_log_write_batches(SPEND_LOG_WRITE_BATCH_MAX_BYTES, SPEND_LOG_WRITE_BATCH_MAX_ROWS).
The tool index write moves out of the rollup batch_() so the split reduces
the query-engine payload; replayed index rows are no-ops under
skip_duplicates, and the daily rollup upserts stay in one transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): pin the row budget in the index fan-out tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The bridge probe asked `responses_api_bridge_check` with the summary read straight off
the Responses object, but `litellm.completion` reads it from `optional_params` via
`peek_reasoning_summary_aliases`, which the bridged request never populated. So gpt-5,
gpt-5.1 and azure/gpt-5 answered "bridging" to the probe and "not bridging" for real,
and the object still landed on Chat Completions, which only takes a string
`reasoning_effort` is now always the effort string, and `summary` rides the
`reasoning_summary` alias that main.py already reassembles into `{effort, summary}` on
the bridged path. The alias is emitted only when the probe says the model bridges, so
no chat provider ever sees it, and the probe is now asked with the exact params this
transform emits
The debug call built its message with an f-string, which
test_logging_calls_do_not_build_their_message_eagerly rejects. Pass the exception
as a %-style argument so the message is only built when the log is emitted.
The bridge probe called responses_api_bridge_check without api_base, so it
resolved the OpenAI base from globals and environment rather than from the
request, while litellm.completion runs the same check with the caller's value.
Today the two cannot disagree: this path always supplies a reasoning_effort,
which short-circuits the endpoint term in the only arm that reads it. Passing it
anyway keeps the probe a faithful mirror of the definitive check rather than one
that happens to agree.
The Responses API takes reasoning as an object, {effort, summary}. Chat
Completions takes reasoning_effort as a string enum and has no equivalent of
summary, but the completion bridge forwarded the whole object whenever summary
was set, which agentic clients set on every request.
Bedrock Converse guards its mapping with isinstance(value, str) and has no else
branch, so the object fell through, thinking was never enabled, and the caller
was billed for a non-thinking turn with nothing in the response to explain it.
The object is still forwarded for the one caller that can consume it: a model
whose cost-map mode is responses, which litellm.completion bridges back onto the
Responses API and reassembles {effort, summary} there. That decision is delegated
to responses_api_bridge_check, the same check litellm.completion runs, rather
than a second copy of the rule that could drift from it. An object carrying no
effort now yields no reasoning_effort at all.
/model/info fills a deployment's missing pricing in from the model cost map so the
Admin UI has a rate to display. Clients echo that whole model_info blob back on save,
and update_db_model merged it into the row, so editing an unrelated setting turned
that day's catalog price into a real per-deployment override. After that the
deployment ignored the cost map and Reload Price Data could no longer move it,
because the reload replays each deployment's stored pricing over the fresh catalog.
Drop the derived pricing from incoming model_info on the two write paths. The
drop-set is read off the same objects the read path uses, CustomPricingLiteLLMParams
plus the tiered *_above_N_tokens pattern that get_model_info passes through and no
model declares, so it cannot drift as new rates are added. output_vector_size is
exempt: it lives on the pricing model but is an embedding dimension, not a rate.
A deployment's own pricing still rides litellm_params, which is untouched, as is the
explicit-null clear, which reads the incoming model rather than the filtered dict.
The filter sits in the endpoint bodies rather than _add_model_to_db, which master-key
rotation reuses to re-serialize every stored deployment.
