* fix(agents): redact secret litellm_params fields from all /v1/agents responses
Secret-bearing litellm_params fields (aws_secret_access_key, api_key, and
similar) are now write-only: list, get, create, update, and patch
responses always replace them with a fixed marker, regardless of caller
role. Editing an agent no longer requires resending a real credential --
an update that omits a sensitive field, or echoes the marker back,
preserves the stored value; a real value still rotates it.
* fix(agents): redact secrets nested inside dicts/lists in litellm_params too
Greptile found that a secret nested one level down under a
non-sensitively-named key, or inside a list of per-provider configs, was
neither redacted on read nor restored symmetrically on write (the marker
string could get persisted as the real value). Recurse into lists on the
read side, and mirror that recursion on the write side so restoration
isn't limited to top-level keys. Also fixes a regression the redact
rewrite introduced (a plain string leaf like a model name was being
misinterpreted as a JSON blob and redacted), and suppresses 3 new
test-quality-gate findings on an established repo-wide mocking pattern
this PR's new tests also use.
* fix(agents): guard list-position credential restore against misassignment
Two more real gaps Greptile/veria found in the recursive redact/restore
mechanism, verified directly against the exact reported shape
(litellm_params.model_list, each entry carrying its own nested
litellm_params.api_key/aws_secret_access_key) before fixing:
- Positional restoration inside a list could attach one entry's stored
credential to a different entry if the list were reordered or resized
between GET and PUT/PATCH. Restoration by index now only fires when the
incoming and existing entries match on every non-secret field; otherwise
the caller's own value is used (never a guessed cross-entry secret).
- A subtree collapsed to the flat REDACTED_BY_LITELM marker by the
read-side recursion depth cap couldn't be recovered on write (the marker
string itself would get persisted). Restore now recognizes that shape and
recovers the whole existing subtree.
Both covered by regression tests mirroring the exact model_list shape
reported, mutation-verified.
* fix(agents): simplify list-entry credential restore to positional matching
The content-match guard from the previous commit fixed one Greptile
finding (cross-entry misassignment on reorder) but introduced a worse one:
it also rejected restoration whenever an entry's own non-secret fields
changed, which is the common case (rename a model_list entry while
leaving its own secret masked) -- silently dropping the stored credential
on an ordinary edit.
There is no stable per-element identity in a plain dict[str, object]
schema, so no rule can satisfy both 'restore whenever the entry itself
only had its secret masked' and 'never restore across a reorder' at once.
Positional correspondence is what every other part of this restore (and
the endpoints' full-replace-on-PUT semantics) already assumes, so drop
the content-match gate and rely on it here too: this fixes the common
case correctly and accepts cross-entry misassignment on a simultaneous
reorder-plus-masked-echo as a known, narrow, documented limitation (not a
leak between different agents or tenants, since it only reshuffles one
agent's own stored values). Tests updated to pin the accepted trade-off
explicitly rather than asserting it away, and to cover the previously
broken ordinary-edit case.
* feat(guardrails): add Alice by ActiveFence guardrail
Adds `guardrail: alice` — policy-based guardrails for prompts and model
responses, evaluated against ActiveFence's Alice.
What makes this different from the other providers: Alice evaluates against
policies configured per *application*, and a proxy typically fronts several of
them, so the application cannot be a static config value. It is named on the
LiteLLM virtual key instead:
curl $PROXY/key/generate -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-d '{"key_alias": "payments-bot",
"metadata": {"alice_app_id": "payments-bot"}}'
read via `CustomGuardrail._get_admin_metadata`, with `key_alias` as the
fallback. That helper is what makes it trustworthy: it reads whichever metadata
holder the proxy wrote the authenticated key's values into — which differs by
route — and the proxy strips caller-supplied `user_api_key_*` from both, so a
caller cannot point its own traffic at an application with laxer policies than
the one its key was issued for. A request whose key names no application is
refused rather than evaluated against a guess.
Implements `apply_guardrail` only, so pre_call, during_call, post_call and
streaming all come from UnifiedLLMGuardrails. Blocks with
GuardrailRaisedException; masks by substituting Alice's redacted text; a MASK
carrying no replacement blocks rather than passing the original through. A
verdict reporting `errors[]` is treated as a failure, not a pass — otherwise a
half-evaluated message would be allowed. `unreachable_fallback` (already on
LitellmParams) chooses fail-closed or fail-open on transport failure.
Config:
guardrails:
- guardrail_name: alice
litellm_params:
guardrail: alice
mode: [pre_call, post_call]
api_key: os.environ/ALICE_API_KEY
21 tests in tests/test_litellm/proxy/guardrails/guardrail_hooks/test_alice.py
cover registration, credential resolution, the app-id ladder including the
forged-metadata case, every verdict, and both unreachable policies.
No new LitellmParams field, so no schema.d.ts regeneration is needed.
* refactor(guardrails): post to Alice's LiteLLM endpoint and forward verbatim
Switches from `/v2/evaluate/message` — Alice's single-text endpoint — to
`/v2/evaluate/litellm`, which takes the hook's arguments as they arrive and
answers with a verdict.
That inverts where the work happens, and shrinks this plugin accordingly. It
now selects nothing and renames nothing: it posts `{input_type, inputs,
request_data}` and enforces `{verdict, categories, correlation_id, message,
replacements}`. Which parts of a conversation are worth evaluating, and how a
verdict is reached, are decided by Alice — so changing either is a change on
their side rather than a LiteLLM upgrade for every user.
The app-id resolution this plugin carried is gone with it. Alice reads the
application off the authenticated key's metadata itself, from the payload it is
handed, so the ladder here was duplicating a decision the far side already
makes. The security property is unchanged and still comes from the proxy
stripping caller-supplied `user_api_key_*` before a guardrail sees the request.
Masking is now positional — the far side chose which texts it was answering
for, so it says which by index. Only `texts` is written; a new
`structured_messages` object would make the chat translation layer skip the
`texts` write-back and silently drop the edits. A mask that lands nowhere
blocks rather than passing the original through.
`request_data` carries live Python objects (an OpenTelemetry span among them),
so `_json_safe` copies it into something serialisable by a mechanical rule
rather than a field list — a list drifts from what the far side needs, a rule
cannot. Serialising naively raises, and that error would read as "guardrail
unavailable" on every request.
26 tests, covering verbatim forwarding, each verdict, positional masking, the
`structured_messages` identity trap, both unreachable policies, and the
serialiser's handling of unserialisable values and cycles.
* fix(alice guardrail): satisfy lint and code-quality CI gates
- Bound _json_safe's recursion and register it in recursive_detector's
ignore list (it already caps depth and dedupes cycles by id, matching
the repo's established pattern for legitimate bounded recursion).
- Clear ruff-strict budget breaches: annotate __init__'s return type,
raise TypeError (not ValueError) for a bad response body, type
_json_safe's payload as object instead of Any, and file-scope-ignore
ANN401 for **kwargs (forwarding it as object broke the call into
CustomGuardrail.__init__, confirmed via basedpyright).
- Clear type-discipline budget breaches: suppress the construction/
annotation checks on one-shot HTTP payloads, the module-level
guardrail registries, and _json_safe's bounded accumulator; narrow
AliceVerdict's list fields to tuples and _evaluate's request_data to
Mapping[str, object] where nothing downstream mutates them.
* test(alice guardrail): assert the guardrail actually registers
The registration test called init_guardrails_v2 and asserted nothing, so it
passed whether or not the guardrail was ever registered — TQ001 in the
test-quality gate, and a fair catch: a test that cannot fail is not covering
the thing it names.
Now asserts exactly one AliceGuardrail lands in litellm.callbacks under the
configured name.
This surfaced only after the ruff-strict and type-discipline gates stopped
failing ahead of it; the lint job runs its gates in sequence, so an earlier
failure masks every later one.
* fix(alice guardrail): reach 100% patch coverage, drop the ActiveFence naming
Codecov flagged 10 uncovered lines, all of them error paths — which is where a
guardrail most needs covering, since each one decides whether traffic flows
unscreened.
Two of the ten turned out to be dead rather than untested, and are removed:
- `except GuardrailRaisedException: raise` in apply_guardrail. `_evaluate`
raises httpx errors, Timeout and TypeError, never that — so the clause could
never fire.
- the trailing `json.dumps` probe in `_json_safe`. Everything json.dumps
handles natively is caught by the isinstance branches above (a dict or list
subclass included), so anything reaching the bottom — bytes, datetime, an
OpenTelemetry span — cannot cross the wire regardless. It now says so and
returns None.
The rest are now tested: a timeout, 502/503/504 as unreachable, a 4xx as NOT
unreachable (a rejected credential is our misconfiguration, not an outage, and
must not fail open), a non-object response body, and a model whose model_dump
raises.
Also drops "by ActiveFence" throughout — the product is Alice — and points the
header at alice.io. `ui_friendly_name` is now "Alice", which is the key
guardrailLogoMap and the garden card look up, so all three moved together.
* fix(alice guardrail): strip caller credentials, widen unreachable detection, block partial MASK
Addresses PR review: request_data no longer forwards secret_fields.raw_headers or
the root api_key to Alice (the caller's Authorization token in the clear otherwise);
HTTP 500, malformed JSON, and a non-object body now route through the configured
unreachable_fallback instead of raising raw, so fail_open still fails open on those;
a MASK verdict with even one out-of-range replacement now blocks entirely instead of
silently letting the rest through unmasked. Also tightens request_data's type and
documents the known streaming-mask limitation on the class.
* fix(alice guardrail): strip credentials at any depth, stop filtering on texts
secret_fields/api_key/headers/provider_specific_header can appear nested
under proxy_server_request, metadata, litellm_metadata, and their
requester_metadata/body sub-paths in a real captured payload — a
top-level-only strip missed all of those. _json_safe now drops these keys
by name wherever they occur during serialization, so a new nesting path
can't reintroduce the leak.
apply_guardrail also stopped skipping the call whenever texts was empty,
even when tool_calls/images/structured_messages carried content — that
was the plugin making a selection decision Alice's design says belongs on
the far side. It now only skips when none of the selectable fields have
anything in them.
* fix(alice guardrail): route an undecodable response body through the fallback
`response.json()` raises UnicodeDecodeError when the body carries bytes that
are not valid UTF-8, and that escaped the except clause: UnicodeDecodeError is
a *sibling* of json.JSONDecodeError under ValueError, not a subclass of it, so
naming only JSONDecodeError left it uncaught. Both fallback modes surfaced a
raw decoding error instead of applying unreachable_fallback — which for a
fail_open deployment meant a hard failure where it had asked for an allow.
Named explicitly rather than widening to ValueError, so the clause still says
which three conditions it means. Tested under both policies.
GPT-5 and later accept a top-level anyOf natively and call tools better with it intact, so the flattening now runs only for the gpt-4, gpt-3.5, chatgpt-4o, o1, o3, and o4 families. Non-dict tool entries pass through untouched, a typeless root that carries properties counts as an object, and the bounded $ref walker is listed in the recursion detector allowlist.
A quoted routine call qualified by a schema and sitting inside a CREATE INDEX
expression, ON "Foo" (public."f"(col)), walked its qualifier read-through back
across the opening paren to the ON that introduces the indexed table, so the call
was misread as a relation and dropped from the call set, leaving a rewrite in that
routine unscanned. A word now only introduces the name when nothing but whitespace
and qualifier dots lies between them, so a paren in that gap keeps ON (and any
relation-introducing keyword) from reaching across it and the call stays a call.
The recursive_detector code-quality gate fails on litellm_internal_staging
because _flatten_form_field and _flatten_form_data_field in
llm_request_utils.py are recursive but absent from IGNORE_FUNCTIONS. Both are
bounded structural recursion over an already-parsed JSON-shaped request body
(a finite tree, no cycles possible), matching the existing ignored walkers, so
add them to the ignore list with a justification comment.
A quoted routine call with a SQL comment between its name and parenthesis,
`"backfill" /* reason */ ()`, is a real call that rewrites rows at boot, but the
call-site check read the raw SQL and stopped at the comment, so the routine was
read as uncalled and its rewrite slipped through. Read the call test from the
masked text instead, where every comment is already blanked to spaces, so a
comment between the name and its parenthesis is skipped exactly as whitespace is,
line, block and nested block comments alike, while a like-named non-call
identifier still opens no call and stays masked
A migration that defines an uncalled row-rewriting routine and elsewhere
references a quoted column, table, index, or constraint sharing the routine's
name was wrongly flagged: the guard restored every double-quoted identifier
before the name search, so a like-named identifier read as a call. Restore only
quoted names that open a call, followed by "(", so a routine invoked through a
quoted identifier is still caught while a like-named non-call identifier stays
masked and no rewrite-free migration is rejected
A migration that defines a row-rewriting routine and calls it as
"backfill"() at the top level slipped past the checker, since masking
blanks double-quoted identifiers before the routine-call search runs, so
the call could not be found by name and the body read as uncalled. mask()
now returns those identifier spans and outside_definition puts them back,
so a call written through a quoted identifier reads as the call it is and
the routine's body gets scanned the same as a bare call
The DML scan reconstructed a DO/EXECUTE'd literal with undouble, which collapses
each doubled quote to one character and shrinks the text. Every offset after a
collapsed pair then shifted, so a row-rewrite scanned out of the literal reported
an earlier file line and could miss a data-migration-ok marker placed on its real
line. Reconstruct with defuse_escapes instead, turning each doubled quote into a
quote and a space so the pair keeps its two characters and every offset holds,
while a nested -- or /* still stays inside its string.
The call-detection restore splices into a fixed-position list, so its
.ljust(end - start) holds that length invariant and a test guards it. The
DML-scan recursion instead hands the undoubled literal to a fresh scan_region
as its own region, whose length feeds nothing, so the pad only appends
trailing spaces that shift no keyword and change no reported line. Drop it and
the docstring clause that claimed it kept the offsets landing
Call detection restored a single-quoted DO or EXECUTE payload through
without_comments while its `''` escapes were still doubled. The first quote of
a pair opened an empty string and closed it on the second, leaving a `--` or
`/*` from a nested string bare, so it blanked the real call after it and the
routine read as uncalled: its rewrite body then went unscanned at boot. Undouble
each single-quoted payload before restoring it, and pad it back to the span it
fills so the later offsets still land. Dollar-quoted bodies do not escape quotes
and are left as they were.
A parenthesised VALUES list ended the search for an insert's row source
only when no group followed it, so a RETURNING or an ON CONFLICT DO
UPDATE carrying a subquery was read as the rows the insert copies. A
writable CTE bounded by its own VALUES list was handed the query the
statement ends with for the same reason: the WITH branch read the whole
statement rather than the part holding the insert.
A CREATE FUNCTION or CREATE PROCEDURE body was scanned as if it ran at
boot, but defining a routine only stores it. The body is now read when
the same migration names the routine somewhere else, so a migration that
defines a backfill and then runs it is still caught, and one whose name
needed quoting is read either way since quoting is blanked at the call
sites too.
main() had no test, so neither its exit codes nor the branch the CI gate
reads were pinned; a mutant returning 0 on a violation passed the whole
suite. Its four outcomes now have tests, along with both directions of
each fix above.
Taking the last group at the statement's outermost level assumed the row
source was written there, and an insert is allowed to carry more after it:
`(SELECT ...) ON CONFLICT ("id") DO NOTHING` ends on the conflict target
and `... RETURNING ("id")` on the returning list, so the query supplying
the rows was never reached and a full table copy passed the gate.
Each group is now read on its own terms and the first to name a row source
is the answer, since the others are the column list and the clauses an
insert may carry, none of which names one.
An `INSERT` whose `VALUES` list holds a scalar subquery was reported as a
rewrite whenever that list was not the plain top-level one: joined to
another term by `UNION`, `INTERSECT` or `EXCEPT`, or written inside
parentheses, which Postgres accepts. Both shapes insert a fixed handful of
rows, so the gate was rejecting migrations that do nothing wrong.
A set operation is now split into its terms and each is read on its own,
since the insert is a rewrite when any one term is a query. A row source
kept in parentheses is read on its own terms too. The operators are found
outside every parenthesis, so a set operation written inside a `VALUES`
list does not cut the list in half.
A `JOIN ... USING` inside a subquery that helps build an EXECUTE's command
was taken for the start of its bind values, so anything written after it
went unscanned and a rewrite there was never reported. Only a `USING` with
the parentheses closed can be the bind-values clause.
A `FOR ... LOOP` header carries no semicolon of its own, so the first
statement of the loop body is written into the same semicolon-delimited
run. Reading the pair as one statement let the header's row source stand
in as the keyword for both, which hid whatever the loop repeats: a plain
`UPDATE` in a query-driven loop went unreported, and so did an `EXECUTE`
of one. That is the shape a row-by-row backfill takes, and it is the
shape this gate exists to stop.
Reading every operator let a comparison beside an assignment look like one.
`ok := n = 1 AND stmt = '<dml>'` registered `stmt` as written, which collided
with the `EXECUTE stmt` further down and flagged a block that rewrites nothing.
A statement holds one assignment at most, so the search now stops at the first
operator that reads as one: everything after it is the expression being
assigned, where an `=` only ever compares. Nine shapes were flagged this way,
a cast, a `coalesce`, a `format`, a named-argument arrow and the rest, and all
of them are valid PL/pgSQL that leaves the table untouched.
`INTO` and `USING` no longer count as names an `EXECUTE` runs. Masking blanks a
literal in place, so `EXECUTE '<sql>' INTO n` left `INTO` looking like the name
being run, and an ordinary query reaching the same word collided with it. The
docstring claiming that collision was impossible was wrong, and both words are
now dropped instead.
A loop is a fourth way a literal reaches a variable. `FOR stmt IN SELECT
'<dml>' LOOP EXECUTE stmt` empties the table and the gate passed it, so the
target of a `FOR` or a `FOREACH` is read as assigned too.
Reading each statement once rather than once per operator also drops the cost
of a statement with thousands of them from seconds to milliseconds.
A bare = was found with partition, so a comparison earlier on the line took
the one slot and the assignment after it went unread. INTO targets and the
name an EXECUTE runs are also allowed to sit on the next line now.
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
Scanning a statement's own literals for SQL only makes sense when the name
before INTO is a variable the body later executes. INSERT INTO names a table
there, so an insert into a table sharing a variable's name was flagged for
whatever its column values happened to spell. An INSERT that really does
assign reaches INTO through RETURNING, which the preceding word separates.
Also names the scope boundary in the module docstring: the ban is on
row-rewriting DML, not on everything whose cost scales with table size.
A parenthesised query term joined to a top-level VALUES list sat behind
strip_parens, so an insert reading `VALUES (1) UNION ALL (SELECT ...)` copied a
whole table past the gate. A VALUES list now bounds an insert only while no set
operation sits beside it at that same level.
PL/pgSQL also parks dynamic SQL in a variable through a query's INTO and through
the bare `=` it takes as the assignment operator, and assigned_names read
neither, so a rewrite handed to a later EXECUTE went unseen. A bare `=` counts
only where the words ahead of it make it an assignment rather than a test.
Postgres takes the row source parenthesised, so `INSERT INTO "t" ("a") (SELECT
...)` copies a whole table at boot. Reading only the unparenthesised text let it
through: 777eb8af10 caught it, then e7dea842c3 traded it away to stop a VALUES
list joined to a query by a set operation from bounding nothing.
Read the top level first so set operations still count, then fall back to the
whole statement when no top-level VALUES bounds the insert. `TABLE t` is a row
source as much as a `SELECT` is, and it was passing too
Four more ruff rules for code the test suite runs but never checks. F601 is the
one that paid: the duplicate key it flagged in a get_form_data fixture was the
mock reproducing the production bug fixed in the previous commit.
B025 removed two unreachable handlers, one of them a pytest.skip shadowed by an
earlier `pass`, so an upstream Vertex flake reported green having asserted
nothing. F632 turned an `is ""` identity check, which passes only on CPython
interning, into the `== ""` it meant. B023 fixed three closures over loop
variables, all latent today but one iteration-order change away from checking the
last case N times.
A dollar-quoted payload is read as its own region rather than as a handed-off
string, so the marker belongs on the rewrite inside it. Pin that placement, and
pin that a marker on a DO block header never covers the block's body.
EXPLAIN ANALYZE runs the statement it wraps rather than only planning it, but
ANALYZE sits in the keyword set, so it stood in for the keyword underneath and a
rewrite left under one reached boot unflagged.
DO takes its body as a string literal, and dollar quoting is a convenience
rather than a requirement. A migration spelling the body in single quotes
got its rewrite through untouched, since nothing was reading that literal
as SQL. It is ordinary syntax rather than an attempt to hide anything, so
the miss was reachable by accident.
The module docstring now also records where concatenated dynamic SQL stops
being readable, which is a keyword split across fragments that do not hold
it. Every fragment is scanned, so the shapes people actually write are all
still caught.
A marker on an EXECUTE now covers the SQL that EXECUTE runs, so it goes
where the migration reads rather than inside the string. A literal whose
first line sat below its EXECUTE was missing the marker entirely, and the
documented placement failed CI.
A literal assigned with := counts as SQL only when an EXECUTE in the same
body runs that variable by name. An error message naming a DELETE the
application handles is text, and the only way to silence it before was a
marker claiming a bounded data migration that was not there at all.
