This commit is contained in:
Ishaan Jaffer 2026-02-21 17:35:16 -08:00
parent 661c6faac6
commit e157f5a8f2

View file

@ -42,25 +42,39 @@ pip install litellm==1.81.14
## Key Highlights
- **Guardrail Model Garden** - [Easily browse built-in and partner guardrails by use case — competitor blockers, topic filters, keyword lists, GDPR PII, EU AI Act, prompt injection, and more](../../docs/proxy/guardrails/policy_templates)
- **3 new built-in guardrails** - [Competitor name blocker, topic blocker, and insults/keyword filter — all run at the gateway level, no external API, configurable per-team or key](../../docs/proxy/guardrails)
- **Compliance Playground** - [Run your guardrails against your own datasets or pre-built ones to measure performance before going live](../../docs/proxy/guardrails/policy_templates)
- **Claude Sonnet 4.6 — day 0** - [Full support across Anthropic and Vertex AI: reasoning, computer use, prompt caching, 200K context](../../docs/providers/anthropic)
- **Major performance batch** - 20+ targeted optimizations across router algorithms, logging overhead, cost calculator, and connection management — meaningfully lower latency and CPU overhead on every request
- **Guardrail Garden** — [Browse built-in and partner guardrails by use case — competitor blocking, topic filtering, GDPR, prompt injection, and more. Pick a template, customize it, attach it to a team or key.](../../docs/proxy/guardrails/policy_templates)
- **Compliance Playground** — [Test any guardrail policy against your own traffic before it goes live. See precision, recall, and false positive rate — so you know how it'll behave in production.](../../docs/proxy/guardrails/policy_templates)
- **3 new zero-cost built-in guardrails** — [Competitor name blocker, topic blocker, and insults filter — all gateway-level, <0.1ms latency, no external API, configurable per-team or key](../../docs/proxy/guardrails)
- **Claude Sonnet 4.6 — day 0** [Full support across Anthropic and Vertex AI: reasoning, computer use, prompt caching, 200K context](../../docs/providers/anthropic)
- **20+ performance optimizations** — Faster routing, lower logging overhead, reduced cost-calculator latency, and connection pool fixes — meaningfully less CPU and latency on every request
---
## Guardrail Model Garden
## Find, deploy, and test guardrails — all from the gateway
Browse built-in and partner guardrails organized by use case — competitor blocking, topic filtering, keyword lists, GDPR/EU AI Act compliance, prompt injection detection, and more. Pick a template, customize the parameters (keyword lists, blocked topics, score thresholds), and attach it to a team or key.
The #1 request we hear from enterprise teams shipping AI to production: *how do we operationalize our guardrails?*
They need to block investment advice, filter competitor mentions, catch insults — and they need to do it at the gateway level, not buried in application code. Once a policy goes live, the next question is always: is it over-sensitive? What's the false positive rate on real traffic? And when requirements change — can I swap from Azure Content Safety to Bedrock Guardrails without touching my app?
This release ships the full workflow: find a guardrail, deploy it, and test it before it hits production.
### 1. Browse the Guardrail Garden
Pick from built-in and partner guardrails organized by use case — competitor blocking, topic filtering, keyword lists, GDPR/EU AI Act compliance, prompt injection detection, and more. Customize the parameters (keyword lists, blocked topics, score thresholds) and attach it to a team or key.
![Guardrail Garden](../img/release_notes/guardrail_garden.png)
## 3 New Built-in Guardrails
### 2. Deploy built-in guardrails — zero latency, zero cost
A competitor name blocker, a topic blocker (keyword and embedding-based), and an insults/keyword filter. All run at the gateway level with no external API call. They're configurable per-team or per-key, and you can swap in AWS Bedrock Guardrails or Azure Content Safety on the same endpoint without changing your application code.
Three new built-in guardrails ship in this release:
### Eval results
- **Competitor name blocker** — block mentions of competitor brands in responses (handles name variations and streaming)
- **Topic blocker** — block entire topic categories like investment advice, using keyword and embedding-based detection
- **Insults / keyword filter** — catch profanity, slurs, and custom keyword lists
All three run at the gateway level with no external API call. Configure them per-team or per-key, and swap in AWS Bedrock Guardrails or Azure Content Safety on the same endpoint without changing your application code. Your governance stays central on the gateway — regardless of which guardrail provider you use underneath.
#### Eval results
We benchmark every built-in guardrail against labeled datasets before shipping. Results for the two policies most relevant to topic and keyword blocking (207 investment-question cases, 299 insult cases):
@ -71,9 +85,9 @@ We benchmark every built-in guardrail against labeled datasets before shipping.
For reference, ONNX embedding approaches on the same eval set hit 9598% precision at 220ms latency and require additional dependencies. The built-in content filter uses no ML model — just structured YAML rules with layered matching — so there's nothing to download, no API key needed, and latency is effectively zero.
## Compliance Playground
### 3. Test before you ship — Compliance Playground
Upload your own dataset or use a pre-built one to measure how a guardrail policy performs before it goes live. See precision, recall, and false positive rate on your actual traffic patterns — so you know how the policy will behave in production before you deploy it.
The hardest part isn't deploying a guardrail — it's knowing how it'll behave once it's live. "What happens when a policy rolls out and it's over-sensitive?" Upload your own dataset or use a pre-built one to see precision, recall, and false positive rate on your actual traffic patterns *before* a policy goes live. If a guardrail is blocking too aggressively, you'll know before your users do.
![Compliance Playground](../img/release_notes/compliance_playground.png)