scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-22 00:31:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 51

docs(release-notes): add guardrail eval results section

Browse source
This commit is contained in:
Ishaan Jaffer 2026-02-21 15:08:06 -08:00
parent 226fc6485b
commit 99168fa736
1 changed files with 11 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
docs/my-website/release_notes/v1.81.14.md
View file

@ -59,6 +59,17 @@ Three new built-in guardrails ship alongside the Garden: a competitor name block
Guardrail tracing is also included: every request that triggers a guardrail now logs the policy name, detection method, and exact match in the logs view. When a policy fires too often in production, you can see exactly which rule triggered and on what input.
### Eval results
We benchmark every built-in guardrail against labeled datasets before shipping. Results for the two policies most relevant to topic and keyword blocking (207 investment-question cases, 299 insult cases):
| Guardrail | Precision | Recall | F1 | Latency p50 | Cost/req |
|-----------|-----------|--------|----|-------------|----------|
| Block investment questions | 100% | 100% | 100% | <0.1ms | $0 |
| Block insults / keywords | 100% | 100% | 100% | <0.1ms | $0 |
For reference, ONNX embedding approaches on the same eval set hit 95–98% precision at 2–20ms latency and require additional dependencies. The built-in content filter uses no ML model — just structured YAML rules with layered matching — so there's nothing to download, no API key needed, and latency is effectively zero.
---
---