diff --git a/docs/my-website/docs/realtime.md b/docs/my-website/docs/realtime.md index c853a860de1..15a838bb7d7 100644 --- a/docs/my-website/docs/realtime.md +++ b/docs/my-website/docs/realtime.md @@ -110,7 +110,88 @@ ws.on("error", function handleError(error) { }); ``` -## Logging +## Guardrails + +You can apply [LiteLLM guardrails](https://docs.litellm.ai/docs/proxy/guardrails/quick_start) to realtime sessions. + +### Set guardrails on a key or team + +The easiest production setup — attach guardrails to a virtual key or team so they always apply automatically, without any client-side changes. + +See [Virtual Keys → Guardrails](https://docs.litellm.ai/docs/proxy/virtual_keys#guardrails) and [Teams → Guardrails](https://docs.litellm.ai/docs/proxy/team_budgets). + +### Pass guardrails dynamically (easy testing) + +Pass `guardrails` as a query param when opening the WebSocket. +Useful for testing guardrails without modifying key/team config. + +```js +// node test.js +const WebSocket = require("ws"); + +const guardrails = ["your-guardrail-name"]; // comma-separated list +const url = `ws://0.0.0.0:4000/v1/realtime?model=openai-gpt-4o-realtime-audio&guardrails=${guardrails.join(",")}`; + +const ws = new WebSocket(url, { + headers: { + "Authorization": "Bearer sk-1234", + }, +}); + +ws.on("open", function open() { + console.log("Connected — guardrails active:", guardrails); +}); + +ws.on("message", function incoming(message) { + const data = JSON.parse(message); + if (data.type === "error") { + // Guardrail block is sent as an error event before the connection closes + console.error("Guardrail error:", data.error.message); + } +}); + +ws.on("close", function close(code, reason) { + console.log("Closed:", code, reason.toString()); + // code 1011 = blocked by guardrail at pre_call +}); +``` + +Or with Python: + +```python +import asyncio +import websockets + +async def main(): + url = "ws://0.0.0.0:4000/v1/realtime?model=openai-gpt-4o-realtime-audio&guardrails=your-guardrail-name" + async with websockets.connect( + url, + additional_headers={"Authorization": "Bearer sk-1234"}, + ) as ws: + print("Connected — guardrail active") + async for msg in ws: + import json + data = json.loads(msg) + if data["type"] == "error": + print("Guardrail blocked:", data["error"]["message"]) + break + +asyncio.run(main()) +``` + +When a guardrail blocks the request, the proxy sends an `error` event over the WebSocket and then closes the connection: + +```json +{ + "type": "error", + "error": { + "type": "guardrail_error", + "message": "Guardrail blocked this request: " + } +} +``` + +## Logging To prevent requests from being dropped, by default LiteLLM just logs these event types: