mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
15eca24fc2
* fix(guardrails): redact secrets in tool traffic and Anthropic system * fix(guardrails): bound tool payload scans and stop nested tool_result recursion Scan each tool value beside its key instead of an indented dump, read tool_result content one level deep, keep non-finite tool arguments as raw text, and give read-only guardrails one fragment per tool payload * test(guardrails): cover read-only guardrail consumers of tool and system text * fix(guardrails): keep tool argument parsing, scanning and tests within bounds Treat tool arguments that decode to a lone surrogate as raw text, decode JSON-escaped detections back to the leaf value, quote unkeyed leaves without repeating a parent key per list item, drop cycle tracking that parsed JSON cannot reach, and use the CI license instead of a fake license server * fix(guardrails): scan list items beside a capped parent key List items keep their parent key in the scanned JSON line, capped to its last 64 characters, so a credential key over a list is still detected while a long key over many items no longer multiplies the scan text * fix(guardrails): cap the scanned key after JSON escaping A key of control characters escaped to six times its length after the cap, so every list item's scan line grew with it * fix(guardrails): collapse whitespace in the scanned key context Space-heavy keys made every inherited list item scan several times slower than a short key * fix(guardrails): build each key's scan context once per payload Repeating the collapse, escape and cut for every list item made a long key over many items cost key length times item count * fix(guardrails): cover Responses tool call items and instructions Responses carries the same tool call arguments and system prompt as chat and messages under function_call.arguments, custom_tool_call.input and instructions, so secrets there reached the provider unscanned * perf(guardrails): skip rescanning text hide-secrets already found clean Agents resend their whole conversation on every turn, so each turn rescanned every earlier tool result. Each guardrail now keeps a bounded set of digests of scan inputs that held no secret and skips those; text with a secret is always rescanned and no secret material is kept * fix(guardrails): size the clean-scan cache for many sessions and replace longest secrets first 4096 digests held about 20 long agent sessions per worker, so interleaved sessions evicted each other and every resend rescanned its history; 65536 digests (about 10 MB) hold a few hundred. Tool payload secrets are now replaced longest first after JSON unescaping, so a shorter overlapping match cannot leave part of a secret behind. The key-context test checks the work it bounds instead of wall-clock time, which CI could not meet * fix(guardrails): keep unmasked tool argument text and redact keyword matches cut inside an escape * fix(guardrails): write a tool visitor's whole payload when it changes more than string values * fix(guardrails): read message text before tool and system text so a vendor error on the new text cannot skip it * fix(guardrails): moderate tool and system text in its own call so a long payload cannot hide a flagged message * fix(guardrails): skip the empty moderation call when only tool or system text is present * fix(guardrails): scan long multi-line tool strings as their own lines so the scan stays linear * fix(guardrails): keep the key beside every line of a long multi-line tool string * fix(guardrails): confine tool-traffic coverage to hide-secrets Restore iter_message_text to its main behavior, so banned keywords, Azure Content Safety, IBM detectors and the OpenAI and Google moderation hooks read exactly what they read on main. Only walk_user_text, which hide-secrets alone calls, still reaches system, instructions and tool payloads. * test(guardrails): stop the tool-traffic integration tests from flaking on a busy host Use the shared per-case timeout for the rig client, and open a new connection per call in the worker-kill burst so a pooled keep-alive socket to the killed worker is never reused. * test(guardrails): drop the wall-clock check from the deep tool arguments unit test The integration test test_deeply_nested_tool_arguments_do_not_stall_the_worker already covers the timing; the unit test keeps the redaction check. --------- Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| cloudformation_stack | ||
| dist | ||
| enterprise_hooks | ||
| enterprise_ui | ||
| litellm_enterprise | ||
| __init__.py | ||
| LICENSE.md | ||
| pyproject.toml | ||
| README.md | ||
LiteLLM Enterprise
Code in this folder is licensed under a commercial license. Please review the LICENSE file within the /enterprise folder
These features are covered under the LiteLLM Enterprise contract
👉 Using in an Enterprise / Need specific features ? Meet with us here
See all Enterprise Features here 👉 Docs