mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
docs fix
This commit is contained in:
parent
661c6faac6
commit
e157f5a8f2
1 changed files with 26 additions and 12 deletions
|
|
@ -42,25 +42,39 @@ pip install litellm==1.81.14
|
|||
|
||||
## Key Highlights
|
||||
|
||||
- **Guardrail Model Garden** - [Easily browse built-in and partner guardrails by use case — competitor blockers, topic filters, keyword lists, GDPR PII, EU AI Act, prompt injection, and more](../../docs/proxy/guardrails/policy_templates)
|
||||
- **3 new built-in guardrails** - [Competitor name blocker, topic blocker, and insults/keyword filter — all run at the gateway level, no external API, configurable per-team or key](../../docs/proxy/guardrails)
|
||||
- **Compliance Playground** - [Run your guardrails against your own datasets or pre-built ones to measure performance before going live](../../docs/proxy/guardrails/policy_templates)
|
||||
- **Claude Sonnet 4.6 — day 0** - [Full support across Anthropic and Vertex AI: reasoning, computer use, prompt caching, 200K context](../../docs/providers/anthropic)
|
||||
- **Major performance batch** - 20+ targeted optimizations across router algorithms, logging overhead, cost calculator, and connection management — meaningfully lower latency and CPU overhead on every request
|
||||
- **Guardrail Garden** — [Browse built-in and partner guardrails by use case — competitor blocking, topic filtering, GDPR, prompt injection, and more. Pick a template, customize it, attach it to a team or key.](../../docs/proxy/guardrails/policy_templates)
|
||||
- **Compliance Playground** — [Test any guardrail policy against your own traffic before it goes live. See precision, recall, and false positive rate — so you know how it'll behave in production.](../../docs/proxy/guardrails/policy_templates)
|
||||
- **3 new zero-cost built-in guardrails** — [Competitor name blocker, topic blocker, and insults filter — all gateway-level, <0.1ms latency, no external API, configurable per-team or key](../../docs/proxy/guardrails)
|
||||
- **Claude Sonnet 4.6 — day 0** — [Full support across Anthropic and Vertex AI: reasoning, computer use, prompt caching, 200K context](../../docs/providers/anthropic)
|
||||
- **20+ performance optimizations** — Faster routing, lower logging overhead, reduced cost-calculator latency, and connection pool fixes — meaningfully less CPU and latency on every request
|
||||
|
||||
---
|
||||
|
||||
## Guardrail Model Garden
|
||||
## Find, deploy, and test guardrails — all from the gateway
|
||||
|
||||
Browse built-in and partner guardrails organized by use case — competitor blocking, topic filtering, keyword lists, GDPR/EU AI Act compliance, prompt injection detection, and more. Pick a template, customize the parameters (keyword lists, blocked topics, score thresholds), and attach it to a team or key.
|
||||
The #1 request we hear from enterprise teams shipping AI to production: *how do we operationalize our guardrails?*
|
||||
|
||||
They need to block investment advice, filter competitor mentions, catch insults — and they need to do it at the gateway level, not buried in application code. Once a policy goes live, the next question is always: is it over-sensitive? What's the false positive rate on real traffic? And when requirements change — can I swap from Azure Content Safety to Bedrock Guardrails without touching my app?
|
||||
|
||||
This release ships the full workflow: find a guardrail, deploy it, and test it before it hits production.
|
||||
|
||||
### 1. Browse the Guardrail Garden
|
||||
|
||||
Pick from built-in and partner guardrails organized by use case — competitor blocking, topic filtering, keyword lists, GDPR/EU AI Act compliance, prompt injection detection, and more. Customize the parameters (keyword lists, blocked topics, score thresholds) and attach it to a team or key.
|
||||
|
||||

|
||||
|
||||
## 3 New Built-in Guardrails
|
||||
### 2. Deploy built-in guardrails — zero latency, zero cost
|
||||
|
||||
A competitor name blocker, a topic blocker (keyword and embedding-based), and an insults/keyword filter. All run at the gateway level with no external API call. They're configurable per-team or per-key, and you can swap in AWS Bedrock Guardrails or Azure Content Safety on the same endpoint without changing your application code.
|
||||
Three new built-in guardrails ship in this release:
|
||||
|
||||
### Eval results
|
||||
- **Competitor name blocker** — block mentions of competitor brands in responses (handles name variations and streaming)
|
||||
- **Topic blocker** — block entire topic categories like investment advice, using keyword and embedding-based detection
|
||||
- **Insults / keyword filter** — catch profanity, slurs, and custom keyword lists
|
||||
|
||||
All three run at the gateway level with no external API call. Configure them per-team or per-key, and swap in AWS Bedrock Guardrails or Azure Content Safety on the same endpoint without changing your application code. Your governance stays central on the gateway — regardless of which guardrail provider you use underneath.
|
||||
|
||||
#### Eval results
|
||||
|
||||
We benchmark every built-in guardrail against labeled datasets before shipping. Results for the two policies most relevant to topic and keyword blocking (207 investment-question cases, 299 insult cases):
|
||||
|
||||
|
|
@ -71,9 +85,9 @@ We benchmark every built-in guardrail against labeled datasets before shipping.
|
|||
|
||||
For reference, ONNX embedding approaches on the same eval set hit 95–98% precision at 2–20ms latency and require additional dependencies. The built-in content filter uses no ML model — just structured YAML rules with layered matching — so there's nothing to download, no API key needed, and latency is effectively zero.
|
||||
|
||||
## Compliance Playground
|
||||
### 3. Test before you ship — Compliance Playground
|
||||
|
||||
Upload your own dataset or use a pre-built one to measure how a guardrail policy performs before it goes live. See precision, recall, and false positive rate on your actual traffic patterns — so you know how the policy will behave in production before you deploy it.
|
||||
The hardest part isn't deploying a guardrail — it's knowing how it'll behave once it's live. "What happens when a policy rolls out and it's over-sensitive?" Upload your own dataset or use a pre-built one to see precision, recall, and false positive rate on your actual traffic patterns *before* a policy goes live. If a guardrail is blocking too aggressively, you'll know before your users do.
|
||||
|
||||

|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue