diff --git a/docs/my-website/docs/proxy/alerting.md b/docs/my-website/docs/proxy/alerting.md index b6296b60a7b..544c9bb905c 100644 --- a/docs/my-website/docs/proxy/alerting.md +++ b/docs/my-website/docs/proxy/alerting.md @@ -6,17 +6,13 @@ import TabItem from '@theme/TabItem'; Get alerts for: -- Hanging LLM api calls -- Slow LLM api calls -- Failed LLM api calls -- Budget Tracking per key/user -- Spend Reports - Weekly & Monthly spend per Team, Tag -- Failed db read/writes -- Model outage alerting -- Daily Reports: - - **LLM** Top 5 slowest deployments - - **LLM** Top 5 deployments with most failed requests -- **Spend** Weekly & Monthly spend per Team, Tag +| Category | Alert Type | +|----------|------------| +| **LLM Performance** | Hanging API calls, Slow API calls, Failed API calls, Model outage alerting | +| **Budget & Spend** | Budget tracking per key/user, Soft budget alerts, Weekly & Monthly spend reports per Team/Tag | +| **System Health** | Failed database read/writes | +| **Daily Reports** | Top 5 slowest LLM deployments, Top 5 LLM deployments with most failed requests, Weekly & Monthly spend per Team/Tag | + Works across: @@ -93,6 +89,51 @@ litellm_settings: redact_messages_in_exceptions: True ``` +### Soft Budget Alerts for Virtual Keys + +Use this to send an alert when a key/team is close to it's budget running out + +Step 1. Create a virtual key with a soft budget + +Set the `soft_budget` to 0.001 + +```shell +curl -X 'POST' \ + 'http://localhost:4000/key/generate' \ + -H 'accept: application/json' \ + -H 'x-goog-api-key: sk-1234' \ + -H 'Content-Type: application/json' \ + -d '{ + "key_alias": "prod-app1", + "team_id": "113c1a22-e347-4506-bfb2-b320230ea414", + "soft_budget": 0.001 +}' +``` + +Step 2. Send a request to the proxy with the virtual key + +```shell +curl http://0.0.0.0:4000/chat/completions \ +-H "Content-Type: application/json" \ +-H "Authorization: Bearer sk-Nb5eCf427iewOlbxXIH4Ow" \ +-d '{ + "model": "openai/gpt-4", + "messages": [ + { + "role": "user", + "content": "this is a test request, write a short poem" + } + ] +}' + +``` + +Step 3. Check slack for Expected Alert + + + + + ### Add Metadata to alerts @@ -123,7 +164,7 @@ response = client.chat.completions.create( -### Opting into specific alert types +### Select specific alert types Set `alert_types` if you want to Opt into only specific alert types. When alert_types is not set, all Default Alert Types are enabled. @@ -145,7 +186,7 @@ general_settings: ] ``` -### Set specific slack channels per alert type +### Map slack channels to alert type Use this if you want to set specific channels per alert type @@ -243,7 +284,7 @@ curl -i http://localhost:4000/v1/chat/completions \ ``` -### Using MS Teams Webhooks +### MS Teams Webhooks MS Teams provides a slack compatible webhook url that you can use for alerting @@ -285,7 +326,7 @@ curl --location 'http://0.0.0.0:4000/health/services?service=slack' \ -### Using Discord Webhooks +### Discord Webhooks Discord provides a slack compatible webhook url that you can use for alerting diff --git a/docs/my-website/img/soft_budget_alert.png b/docs/my-website/img/soft_budget_alert.png new file mode 100644 index 00000000000..7e1f66f0fd1 Binary files /dev/null and b/docs/my-website/img/soft_budget_alert.png differ