From 814bc67c843f3834dc23307eb6918a468d2c3623 Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Tue, 3 Sep 2024 16:58:11 -0700 Subject: [PATCH] docs control routes on proxy --- docs/my-website/docs/enterprise.md | 2 +- docs/my-website/docs/proxy/enterprise.md | 120 +++++++++++++++++++++-- 2 files changed, 114 insertions(+), 8 deletions(-) diff --git a/docs/my-website/docs/enterprise.md b/docs/my-website/docs/enterprise.md index b1dda8769de..cd83365ac3e 100644 --- a/docs/my-website/docs/enterprise.md +++ b/docs/my-website/docs/enterprise.md @@ -22,7 +22,7 @@ This covers: - ✅ [SSO for Admin UI](./proxy/ui#✨-enterprise-features) - ✅ [Audit Logs with retention policy](./proxy/enterprise#audit-logs) - ✅ [JWT-Auth](../docs/proxy/token_auth.md) - - ✅ [Control available public, private routes](./proxy/enterprise#control-available-public-private-routes) + - ✅ [Control available public, private routes (Restrict certain endpoints on proxy)](./proxy/enterprise#control-available-public-private-routes) - ✅ [[BETA] AWS Key Manager v2 - Key Decryption](./proxy/enterprise#beta-aws-key-manager---key-decryption) - ✅ IP address‑based access control lists - ✅ Track Request IP Address diff --git a/docs/my-website/docs/proxy/enterprise.md b/docs/my-website/docs/proxy/enterprise.md index a432c72ae42..ac26b9e4a8e 100644 --- a/docs/my-website/docs/proxy/enterprise.md +++ b/docs/my-website/docs/proxy/enterprise.md @@ -16,7 +16,7 @@ Features: - ✅ [SSO for Admin UI](./ui.md#✨-enterprise-features) - ✅ [Audit Logs with retention policy](#audit-logs) - ✅ [JWT-Auth](../docs/proxy/token_auth.md) - - ✅ [Control available public, private routes](#control-available-public-private-routes) + - ✅ [Control available public, private routes (Restrict certain endpoints on proxy)](#control-available-public-private-routes) - ✅ [[BETA] AWS Key Manager v2 - Key Decryption](#beta-aws-key-manager---key-decryption) - ✅ IP address‑based access control lists - ✅ Track Request IP Address @@ -609,24 +609,35 @@ Expected Response ## Control available public, private routes +**Restrict certain endpoints of proxy** + :::info -❓ Use this when you want to make an existing private route -> public - -Example - Make `/spend/calculate` a publicly available route (by default `/spend/calculate` on LiteLLM Proxy requires authentication) +❓ Use this when you want to: +- make an existing private route -> public +- set certain routes as admin_only routes ::: -#### Usage - Define public routes +#### Usage - Define public, admin only routes -**Step 1** - set allowed public routes on config.yaml +**Step 1** - Set on config.yaml + + +| Route Type | Optional | Requires Virtual Key Auth | Admin Can Access | All Roles Can Access | Description | +|------------|----------|---------------------------|-------------------|----------------------|-------------| +| `public_routes` | ✅ | ❌ | ✅ | ✅ | Routes that can be accessed without any authentication | +| `admin_only_routes` | ✅ | ✅ | ✅ | ❌ | Routes that can only be accessed by [Proxy Admin](./self_serve#available-roles) | +| `allowed_routes` | ✅ | ✅ | ✅ | ✅ | Routes are exposed on the proxy. If not set then all routes exposed. | `LiteLLMRoutes.public_routes` is an ENUM corresponding to the default public routes on LiteLLM. [You can see this here](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/_types.py) ```yaml general_settings: master_key: sk-1234 - public_routes: ["LiteLLMRoutes.public_routes", "/spend/calculate"] + public_routes: ["LiteLLMRoutes.public_routes", "/spend/calculate"] # routes that can be accessed without any auth + admin_only_routes: ["/key/generate"] # Optional - routes that can only be accessed by Proxy Admin + allowed_routes: ["/chat/completions", "/spend/calculate", "LiteLLMRoutes.public_routes"] # Optional - routes that can be accessed by anyone after Authentication ``` **Step 2** - start proxy @@ -637,6 +648,10 @@ litellm --config config.yaml **Step 3** - Test it + + + + ```shell curl --request POST \ --url 'http://localhost:4000/spend/calculate' \ @@ -649,6 +664,97 @@ curl --request POST \ 🎉 Expect this endpoint to work without an `Authorization / Bearer Token` + + + + + +**Successfull Request** + +```shell +curl --location 'http://0.0.0.0:4000/key/generate' \ +--header 'Authorization: Bearer ' \ +--header 'Content-Type: application/json' \ +--data '{}' +``` + + +**Un-successfull Request** + +```shell + curl --location 'http://0.0.0.0:4000/key/generate' \ +--header 'Authorization: Bearer ' \ +--header 'Content-Type: application/json' \ +--data '{"user_role": "internal_user"}' +``` + +**Expected Response** + +```json +{ + "error": { + "message": "user not allowed to access this route. Route=/key/generate is an admin only route", + "type": "auth_error", + "param": "None", + "code": "403" + } +} +``` + + + + + + + +**Successfull Request** + +```shell +curl http://localhost:4000/chat/completions \ +-H "Content-Type: application/json" \ +-H "Authorization: Bearer sk-1234" \ +-d '{ +"model": "fake-openai-endpoint", +"messages": [ + {"role": "user", "content": "Hello, Claude"} +] +}' +``` + + +**Un-successfull Request** + +```shell +curl --location 'http://0.0.0.0:4000/embeddings' \ +--header 'Content-Type: application/json' \ +-H "Authorization: Bearer sk-1234" \ +--data ' { +"model": "text-embedding-ada-002", +"input": ["write a litellm poem"] +}' +``` + +**Expected Response** + +```json +{ + "error": { + "message": "Route /embeddings not allowed", + "type": "auth_error", + "param": "None", + "code": "403" + } +} +``` + + + + + + + + + ## Guardrails - Secret Detection/Redaction ❓ Use this to REDACT API Keys, Secrets sent in requests to an LLM.