From 0ab74a77e3c4097e2faef8f7b965e75cdada5bfc Mon Sep 17 00:00:00 2001 From: Joseph Barker Date: Tue, 7 Apr 2026 12:33:46 -0700 Subject: [PATCH] Update Rubrik docs: config.yaml as primary, env vars as fallback Restructures the Quick Start to present config.yaml as the recommended approach with tabbed UI, and environment variables as an alternative fallback. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../docs/proxy/guardrails/rubrik.md | 94 ++++++++++++++----- 1 file changed, 71 insertions(+), 23 deletions(-) diff --git a/docs/my-website/docs/proxy/guardrails/rubrik.md b/docs/my-website/docs/proxy/guardrails/rubrik.md index 3e0d439f1ed..d75755a4faf 100644 --- a/docs/my-website/docs/proxy/guardrails/rubrik.md +++ b/docs/my-website/docs/proxy/guardrails/rubrik.md @@ -1,3 +1,6 @@ +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + # Rubrik Guardrail Use Rubrik's tool blocking and logging integration to validate LLM tool calls against an external policy service and batch-log all LLM requests/responses. @@ -11,14 +14,12 @@ Use Rubrik's tool blocking and logging integration to validate LLM tool calls ag ## Quick Start -### 1. Set Environment Variables +### 1. Configure `config.yaml` -```bash -export RUBRIK_WEBHOOK_URL="https://your-rubrik-service.example.com" -export RUBRIK_API_KEY="your-rubrik-api-key" # optional -``` +Credentials can be set directly in the YAML config or via environment variables. The config approach is recommended. -### 2. Configure `config.yaml` + + ```yaml model_list: @@ -27,6 +28,19 @@ model_list: model: openai/gpt-4 api_key: os.environ/OPENAI_API_KEY +guardrails: + - guardrail_name: "rubrik" + litellm_params: + guardrail: rubrik + mode: "post_call" + api_key: "your-rubrik-api-key" + api_base: "https://your-rubrik-service.example.com" + default_on: true +``` + +You can also reference environment variables in the config: + +```yaml guardrails: - guardrail_name: "rubrik" litellm_params: @@ -37,13 +51,43 @@ guardrails: default_on: true ``` -### 3. Launch the Proxy + + + +As an alternative, you can configure the Rubrik service URL and API key purely through environment variables. When set, these are used as fallbacks if `api_base` / `api_key` are not provided in the config. + +```bash +export RUBRIK_WEBHOOK_URL="https://your-rubrik-service.example.com" +export RUBRIK_API_KEY="your-rubrik-api-key" +``` + +With a minimal config: + +```yaml +model_list: + - model_name: gpt-4 + litellm_params: + model: openai/gpt-4 + api_key: os.environ/OPENAI_API_KEY + +guardrails: + - guardrail_name: "rubrik" + litellm_params: + guardrail: rubrik + mode: "post_call" + default_on: true +``` + + + + +### 2. Launch the Proxy ```bash litellm --config config.yaml --port 4000 ``` -### 4. Test It +### 3. Test It ```bash curl -X POST http://localhost:4000/chat/completions \ @@ -75,32 +119,36 @@ curl -X POST http://localhost:4000/chat/completions \ ## Configuration Reference +### YAML Config Parameters + +These are set under `guardrails.[].litellm_params` in your `config.yaml`: + +| Parameter | Required | Description | +|-----------|----------|-------------| +| `guardrail: rubrik` | Yes | Selects the Rubrik guardrail integration | +| `mode: "post_call"` | Yes | Run after the LLM response is received | +| `api_base` | Yes | Rubrik webhook base URL. Can use `os.environ/RUBRIK_WEBHOOK_URL`. Falls back to `RUBRIK_WEBHOOK_URL` env var if omitted. | +| `api_key` | No | Rubrik API key. Can use `os.environ/RUBRIK_API_KEY`. Falls back to `RUBRIK_API_KEY` env var if omitted. | +| `default_on` | No | When `true`, the guardrail runs on all requests without needing per-request opt-in | + ### Environment Variables +These are optional fallbacks used when `api_base` / `api_key` are not set in the YAML config. `RUBRIK_SAMPLING_RATE` and `RUBRIK_BATCH_SIZE` can only be set via environment variables. + | Variable | Required | Default | Description | |----------|----------|---------|-------------| -| `RUBRIK_WEBHOOK_URL` | Yes (or `api_base` in config) | — | Base URL of the Rubrik webhook service | +| `RUBRIK_WEBHOOK_URL` | Only if `api_base` not in config | — | Base URL of the Rubrik webhook service | | `RUBRIK_API_KEY` | No | — | Bearer token for authenticating with the Rubrik service | -| `RUBRIK_SAMPLING_RATE` | No | `1.0` | Fraction of requests to log (0.0 to 1.0). Set to `0.5` to log ~50% of requests. | +| `RUBRIK_SAMPLING_RATE` | No | `1.0` | Fraction of requests to **log** (0.0 to 1.0). Does not affect tool blocking, which always runs. Set to `0.5` to log ~50% of requests. | | `RUBRIK_BATCH_SIZE` | No | `512` | Number of log entries to buffer before flushing. Logs are also flushed on a periodic interval. | -### YAML Config Parameters - -| Parameter | Description | -|-----------|-------------| -| `guardrail: rubrik` | Selects the Rubrik guardrail integration | -| `mode: "post_call"` | Run after the LLM response is received | -| `api_key` | Rubrik API key (can use `os.environ/RUBRIK_API_KEY`) | -| `api_base` | Rubrik webhook base URL (can use `os.environ/RUBRIK_WEBHOOK_URL`) | -| `default_on` | When `true`, the guardrail runs on all requests without needing per-request opt-in | - --- ## How Tool Blocking Works 1. After the LLM returns a response with tool calls, the Rubrik guardrail sends them to the blocking service at `{api_base}/v1/after_completion/openai/v1`. 2. The service evaluates each tool call against configured policies and returns the set of **allowed** tool calls. -3. If any tool calls are blocked, the proxy returns a `200` response with the policy violation explanation instead of the original LLM response. +3. If any tool calls are blocked, the proxy returns the policy violation explanation as a response instead of the original LLM response. 4. If the blocking service is unreachable or returns an error, the guardrail **fails open** — the original response is returned unchanged. ### Request/Response format @@ -135,6 +183,6 @@ The service should return an OpenAI chat completion format response containing o All LLM requests (successes and failures) are queued and sent in batches to `{api_base}/v1/litellm/batch`. -- Logs are flushed when the queue reaches `RUBRIK_BATCH_SIZE` (default 512) or on a periodic interval (default 5 seconds). -- Use `RUBRIK_SAMPLING_RATE` to reduce logging volume in high-traffic deployments. +- Logs are flushed when the queue reaches `RUBRIK_BATCH_SIZE` (default 512) or on a periodic interval (default 5 seconds). These defaults are inherited from LiteLLM's global settings. +- Use `RUBRIK_SAMPLING_RATE` to reduce logging volume in high-traffic deployments. Sampling only affects logging — tool blocking always runs regardless of the sampling rate. - For Anthropic `/v1/messages` requests, the log ID is normalized to `litellm_call_id` for consistency across tool blocking and logging.