From 99168fa736933f43fdc0031e23412e42400fad0c Mon Sep 17 00:00:00 2001 From: Ishaan Jaffer Date: Sat, 21 Feb 2026 15:08:06 -0800 Subject: [PATCH] docs(release-notes): add guardrail eval results section --- docs/my-website/release_notes/v1.81.14.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/my-website/release_notes/v1.81.14.md b/docs/my-website/release_notes/v1.81.14.md index c10254d5b28..dee4e4363b2 100644 --- a/docs/my-website/release_notes/v1.81.14.md +++ b/docs/my-website/release_notes/v1.81.14.md @@ -59,6 +59,17 @@ Three new built-in guardrails ship alongside the Garden: a competitor name block Guardrail tracing is also included: every request that triggers a guardrail now logs the policy name, detection method, and exact match in the logs view. When a policy fires too often in production, you can see exactly which rule triggered and on what input. +### Eval results + +We benchmark every built-in guardrail against labeled datasets before shipping. Results for the two policies most relevant to topic and keyword blocking (207 investment-question cases, 299 insult cases): + +| Guardrail | Precision | Recall | F1 | Latency p50 | Cost/req | +|-----------|-----------|--------|----|-------------|----------| +| Block investment questions | 100% | 100% | 100% | <0.1ms | $0 | +| Block insults / keywords | 100% | 100% | 100% | <0.1ms | $0 | + +For reference, ONNX embedding approaches on the same eval set hit 95–98% precision at 2–20ms latency and require additional dependencies. The built-in content filter uses no ML model — just structured YAML rules with layered matching — so there's nothing to download, no API key needed, and latency is effectively zero. + --- ---