litellm/tests/test_litellm/proxy/utils
devin-ai-integration[bot] 26bf575f15
feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails (#43271)
* feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails

Vector store retrieval runs inside acompletion after the proxy's pre-call
guardrails have already seen the request, so a retrieved chunk carrying an
injection reached the prompt unscanned. Each retrieved context message now
goes through every pre-call guardrail the request is subject to before it
is injected: a block raises the same 400 the guardrail gives for user text,
a masking guardrail rewrites the context, and a guardrail that fails while
scanning fails the request instead of injecting the chunk unscanned

* fix(guardrails): return a guardrail block unmapped from exception_type so the Responses API surfaces the guardrail's own 400

* fix(guardrails): build the deployment hooks' identity from stamped metadata only

Top-level user_api_key_* fields in a request body are client controlled, so the pre-call, chunk scan, and post-call deployment hooks now take UserAPIKeyAuth from the metadata the proxy stamped, and the chunk scanner returns or raises on every branch.

* fix(guardrails): block route verdicts on retrieved chunks, keep guardrail verdicts out of router retries and fallbacks, and scan chunks against the client's request

* fix(guardrails): keep the merged guardrail list when scanning chunks against the client's request

The scan request laid the client's kept body over the deployment kwargs, so a client
that sent its own top-level guardrails list shadowed the merged metadata.guardrails
list and a key or team guardrail skipped the chunk scan. The kwargs now win and the
keys the proxy relocates into metadata are dropped from the body's contribution.

* fix(guardrails): strip the deployment's guardrail keys from the chunk scan request so merged team guardrails still run

* test(guardrails): type the vector store scan test doubles

* test(guardrails): type the scan double's request data

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-26 14:18:38 -07:00
..
helpers fix(agents): evict the cached agent access groups on every agent write and cap the model listing 2026-09-17 20:31:03 +00:00
prisma_and_spend Merge remote-tracking branch 'origin/main' into litellm_redis_durable_spend_log_buffer 2026-09-20 08:52:32 +00:00
proxy_logging feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails (#43271) 2026-09-26 14:18:38 -07:00
__init__.py test(proxy/utils): pin bottom-of-file helper behavior (#29509) 2026-06-02 17:45:19 -07:00