diff --git a/docs/features/usage-alerts.md b/docs/features/usage-alerts.md index 842709b66bc..804a8b44aec 100644 --- a/docs/features/usage-alerts.md +++ b/docs/features/usage-alerts.md @@ -1,19 +1,40 @@ -# Daily usage alerts +# Daily Performance alerts -Daily usage alerts compare a completed UTC day with earlier recorded days. They cover spend, request count, average latency, and failed-request count for the gateway, a key, a user, a team, or a model. Open an incident to inspect its saved comparison and follow its Activity link to the corresponding usage view. +Performance alerts compare a completed UTC day with earlier recorded days. They cover spend, request count, average latency, and failed-request count for the gateway, a key, a user, a team, or a model. The Performance tab groups alerts for the same resource and original UTC day. Open a resource to inspect its saved comparison and Activity. The API paths remain under `/usage`. This feature requires a database. Apply the normal LiteLLM database migrations when deploying a version that includes usage alerts. The API and dashboard use the same persisted rules and incidents; no enterprise license is required. -## Read incidents +## Read resource groups Set `LITELLM_API_BASE` to your gateway's base URL and `LITELLM_MANAGEMENT_KEY` to a management-capable credential. Keep the key in your environment. ```bash -curl "$LITELLM_API_BASE/usage/alerts?status=open&limit=50" \ +curl "$LITELLM_API_BASE/usage/alert-groups?status=open&limit=50" \ -H "Authorization: Bearer $LITELLM_MANAGEMENT_KEY" ``` -The response has `data`, `total`, `limit`, and `offset`. Each incident includes its scope, metric, direction, status, UTC day, observed value, baseline value, threshold, baseline date range, and `activity_path`. The Activity path is relative to the dashboard mount: for example, `/api-keys?...` is opened below the dashboard's `/ui` mount. +The response has `data`, `total`, `limit`, and `offset`. `total` counts resource/day groups, including when the requested offset is beyond the last page. Filters are `scope_type`, `scope_id`, `status`, `start_date`, and `end_date`. Status filters apply to individual incidents before grouping, so a group with open and resolved incidents can appear under either filter with the corresponding members. No default date cutoff hides older open incidents. + +Each group includes its stable `id`, `scope_type`, exact `scope_id`, current `scope_label` when available, original UTC `day`, `incident_count`, and `activity_path`. Its `metrics` list contains up to four entries with a `metric`, `incident_count`, and the most recently detected matching `representative` incident. These representatives are summaries, not the entire group. Missing or deleted resources keep their original IDs. Names are current display labels, not historical evidence. + +To inspect every matching rule, use `GET /usage/alerts` with the group's scope and `start_date` and `end_date` both set to its `day`. Add `metric` and `status` as needed and paginate that response. Rules can have different baseline windows and thresholds, so their comparisons must not be combined into an error rate or a cost-per-request estimate. + +The group Activity link uses a representative's saved date range and alert context. The path is relative to the dashboard mount: for example, `/api-keys?...` is opened below the dashboard's `/ui` mount. A continuing episode retains its original day and evidence; the grouped row does not describe today's traffic. + +Resolve or dismiss all matching incidents in one resource/day group with an expected current status: + +```bash +curl -X PATCH "$LITELLM_API_BASE/usage/alert-groups" \ + -H "Authorization: Bearer $LITELLM_MANAGEMENT_KEY" \ + -H "Content-Type: application/json" \ + -d '{"scope_type":"team","scope_id":"team-example","day":"2026-11-04","expected_status":"open","status":"resolved"}' +``` + +The response contains `updated_count`. Only incidents in that exact group that still have `expected_status` are updated atomically. Other statuses remain unchanged; repeating the same request returns zero after all matching incidents have changed. Use `scope_type: "deployment"` with `scope_id: null` for the gateway. + +## Read individual incidents + +`GET /usage/alerts` retains the individual-incident view. Each incident includes its scope, metric, direction, status, UTC day, observed value, baseline value, threshold, baseline date range, and `activity_path`. `GET /usage/alerts/{alert_id}` returns one incident. List filters are `scope_type`, `scope_id`, `metric`, `status`, `start_date`, and `end_date`. Dates use `YYYY-MM-DD`; date filters include both endpoints. The maximum page size is 100. @@ -55,10 +76,12 @@ The evaluator runs against daily aggregates. An hourly scheduler does not make t Use `scope_type: "deployment"` with no `scope_id` for the gateway. For `model`, use the underlying model identity recorded in Activity. For `key`, use its reporting ID, never the secret API key. Omitting `scope_id` for key, user, team, or model evaluates all entities of that type and requires a gateway administrator. +`GET /usage/alert-models?limit=20&search=example` returns paginated model identities recorded in the most recent 30 UTC days. Its `data` entries contain `scope_id` and `label`; `start_date` and `end_date` state the searched window. Pass `next_cursor` as `cursor` to read the next page. A null cursor ends the result. Search matches literal text, not SQL wildcards. This endpoint requires gateway admin or admin-viewer access. Configured aliases and deployment IDs are not reporting-model identities. An existing rule's saved model remains valid even when it falls outside the options window. + ## Manage rules and incidents -- `GET /usage/alert-rules` lists rules; `GET /usage/alert-rules/{rule_id}` reads one. -- `PUT /usage/alert-rules/{rule_id}` replaces the rule configuration. Send the configuration fields from the create example, including any notification settings you want to retain. Set `enabled` to `false` to pause it. +- `GET /usage/alert-rules` lists rules; `GET /usage/alert-rules/{rule_id}` reads one. Use `kind=custom`, `kind=default`, or `kind=all` to choose which rules to list. Each rule exposes `is_default`. The list's `default_summary` counts total and enabled defaults within the requested scope, independent of `kind`, and reports evaluation-status counts among enabled defaults. +- `PUT /usage/alert-rules/{rule_id}` replaces the rule configuration. Send the configuration fields from the create example, including any notification settings you want to retain. Set `enabled` to `false` to pause it. Built-in rules keep their scope, metric, and direction; create a custom rule to change those fields. Their names, thresholds, notification settings, and enabled state remain editable. - `DELETE /usage/alert-rules/{rule_id}` removes a rule from evaluation and preserves existing incidents. Deleted defaults stay deleted. - `PATCH /usage/alerts/{alert_id}` with `{"status":"dismissed"}` dismisses an incident. `open` and `resolved` are also accepted. diff --git a/litellm/models/usage_alerts.py b/litellm/models/usage_alerts.py index 9a10be2d409..8858835eafc 100644 --- a/litellm/models/usage_alerts.py +++ b/litellm/models/usage_alerts.py @@ -3,10 +3,15 @@ from typing import Protocol from litellm.types.proxy.usage_alerts import ( UsageAlert, + UsageAlertGroupList, + UsageAlertGroupUpdate, + UsageAlertGroupUpdateResult, UsageAlertList, UsageAlertMetric, + UsageAlertModelList, UsageAlertRule, UsageAlertRuleInput, + UsageAlertRuleKind, UsageAlertRuleList, UsageAlertScope, UsageAlertStatus, @@ -23,7 +28,12 @@ class UsageAlertStore(Protocol): async def delete_rule(self, rule_id: str) -> None: ... async def list_rules( - self, scope_type: UsageAlertScope | None = None, scope_id: str | None = None, limit: int = 50, offset: int = 0 + self, + scope_type: UsageAlertScope | None = None, + scope_id: str | None = None, + limit: int = 50, + offset: int = 0, + kind: UsageAlertRuleKind = "all", ) -> UsageAlertRuleList: ... async def get_alert(self, alert_id: str) -> UsageAlert | None: ... @@ -42,6 +52,21 @@ class UsageAlertStore(Protocol): async def set_status(self, alert_id: str, status: UsageAlertStatus) -> UsageAlert | None: ... + async def list_groups( + self, + scope_type: UsageAlertScope | None = None, + scope_id: str | None = None, + status: UsageAlertStatus | None = None, + start_date: date | None = None, + end_date: date | None = None, + limit: int = 50, + offset: int = 0, + ) -> UsageAlertGroupList: ... + + async def set_group_status(self, data: UsageAlertGroupUpdate) -> UsageAlertGroupUpdateResult: ... + + async def list_reported_models(self, search: str | None, cursor: str | None, limit: int) -> UsageAlertModelList: ... + async def key_ownership(self, key_id: str) -> tuple[str | None, str | None] | None: ... async def team_role(self, team_id: str, user_id: str) -> str | None: ... diff --git a/litellm/proxy/_types.py b/litellm/proxy/_types.py index 796e334f3cc..7254a0fb6b7 100644 --- a/litellm/proxy/_types.py +++ b/litellm/proxy/_types.py @@ -579,6 +579,8 @@ class LiteLLMRoutes(enum.Enum): ) llm_api_routes_without_mcp = sorted(set(llm_api_routes) - set(mcp_inference_routes)) info_routes = [ + "/usage/alert-groups", + "/usage/alert-models", "/usage/alert-rules", "/usage/alert-rules/{rule_id}", "/usage/alerts", @@ -650,6 +652,8 @@ class LiteLLMRoutes(enum.Enum): management_routes = ( [ + "/usage/alert-groups", + "/usage/alert-models", "/usage/alert-rules", "/usage/alert-rules/{rule_id}", "/usage/alerts", @@ -864,6 +868,8 @@ class LiteLLMRoutes(enum.Enum): ) usage_alert_routes = ( + "/usage/alert-groups", + "/usage/alert-models", "/usage/alert-rules", "/usage/alert-rules/{rule_id}", "/usage/alerts", diff --git a/litellm/proxy/alerting_endpoints/usage_alerts.py b/litellm/proxy/alerting_endpoints/usage_alerts.py index 64f06fa62ed..e6b1b22cc00 100644 --- a/litellm/proxy/alerting_endpoints/usage_alerts.py +++ b/litellm/proxy/alerting_endpoints/usage_alerts.py @@ -13,10 +13,15 @@ from litellm.repositories.daily_activity_buckets import api_key_from_bucket from litellm.types.proxy.auth.user_roles import LitellmUserRoles from litellm.types.proxy.usage_alerts import ( UsageAlert, + UsageAlertGroupList, + UsageAlertGroupUpdate, + UsageAlertGroupUpdateResult, UsageAlertList, UsageAlertMetric, + UsageAlertModelList, UsageAlertRule, UsageAlertRuleInput, + UsageAlertRuleKind, UsageAlertRuleList, UsageAlertScope, UsageAlertStatus, @@ -93,10 +98,52 @@ async def list_usage_alert_rules( scope_id: ScopeId = None, limit: Limit = 50, offset: Offset = 0, + kind: UsageAlertRuleKind = "all", ) -> UsageAlertRuleList: store: Final = require_repository(repository) await require_scope(user, store, scope_type, scope_id, write=False) - return await store.list_rules(scope_type, scope_id, limit, offset) + return await store.list_rules(scope_type, scope_id, limit, offset, kind) + + +@router.get("/alert-groups", response_model=UsageAlertGroupList) +async def list_usage_alert_groups( + user: User, + repository: Repository, + scope_type: UsageAlertScope | None = None, + scope_id: ScopeId = None, + status: UsageAlertStatus | None = None, + start_date: date | None = None, + end_date: date | None = None, + limit: Limit = 50, + offset: Offset = 0, +) -> UsageAlertGroupList: + store: Final = require_repository(repository) + await require_scope(user, store, scope_type, scope_id, write=False) + if start_date is not None and end_date is not None and start_date > end_date: + raise HTTPException(status_code=422, detail="start_date must not be after end_date") + return await store.list_groups(scope_type, scope_id, status, start_date, end_date, limit, offset) + + +@router.patch("/alert-groups", response_model=UsageAlertGroupUpdateResult) +async def update_usage_alert_group( + data: UsageAlertGroupUpdate, user: User, repository: Repository +) -> UsageAlertGroupUpdateResult: + store: Final = require_repository(repository) + await require_scope(user, store, data.scope_type, data.scope_id, write=True) + return await store.set_group_status(data) + + +@router.get("/alert-models", response_model=UsageAlertModelList) +async def list_usage_alert_models( + user: User, + repository: Repository, + search: ScopeId = None, + cursor: ScopeId = None, + limit: Limit = 20, +) -> UsageAlertModelList: + store: Final = require_repository(repository) + await require_scope(user, store, "model", None, write=False) + return await store.list_reported_models(search, cursor, limit) @router.post("/alert-rules", response_model=UsageAlertRule, status_code=201) @@ -129,6 +176,15 @@ async def update_usage_alert_rule( if rule is None: raise HTTPException(status_code=404, detail="Alert rule not found") await require_scope(user, store, rule.scope_type, rule.scope_id, write=True) + if rule.is_default and (rule.scope_type, rule.scope_id, rule.metric, rule.direction) != ( + data.scope_type, + data.scope_id, + data.metric, + data.direction, + ): + raise HTTPException( + status_code=422, detail="Create a custom rule to change a built-in rule's scope, metric, or direction" + ) await require_scope(user, store, data.scope_type, data.scope_id, write=True) saved: Final = await store.save_rule(data, user.user_id, existing=rule) if saved is None: diff --git a/litellm/repositories/usage_alerts.py b/litellm/repositories/usage_alerts.py index d2e0c38f7e3..c7efc5c77f8 100644 --- a/litellm/repositories/usage_alerts.py +++ b/litellm/repositories/usage_alerts.py @@ -3,6 +3,8 @@ import json from collections.abc import Mapping from datetime import date, datetime, timedelta, timezone +from itertools import product +from types import MappingProxyType from typing import ( TYPE_CHECKING, Final, @@ -14,16 +16,26 @@ from uuid import NAMESPACE_URL, uuid4, uuid5 from pydantic import BaseModel, TypeAdapter +from litellm.repositories.daily_activity_buckets import api_key_from_bucket from litellm.types.proxy.usage_alerts import ( UsageAlert, UsageAlertChannel, + UsageAlertDefaultSummary, UsageAlertEvaluationStatus, + UsageAlertGroup, + UsageAlertGroupList, + UsageAlertGroupMetric, + UsageAlertGroupUpdate, + UsageAlertGroupUpdateResult, UsageAlertList, UsageAlertMetric, + UsageAlertModelList, + UsageAlertModelOption, UsageAlertNotification, UsageAlertObservation, UsageAlertRule, UsageAlertRuleInput, + UsageAlertRuleKind, UsageAlertRuleList, UsageAlertScope, UsageAlertStatus, @@ -45,6 +57,37 @@ class _Count(BaseModel, frozen=True): count: int +class _RuleCounts(_Count, frozen=True): + default_summary: UsageAlertDefaultSummary + + +class _GroupRow(BaseModel, frozen=True): + scope_type: UsageAlertScope + scope_id: str | None + day: date + incident_count: int + metrics: tuple[UsageAlertGroupMetric, ...] + + +class _GroupPage(BaseModel, frozen=True): + total: int + data: tuple[_GroupRow, ...] + + +class _ResourceLabel(BaseModel, frozen=True): + scope_type: UsageAlertScope + scope_id: str | None + label: str | None + + +class _ModelIdentity(BaseModel, frozen=True): + model: str + + +class _Identifier(BaseModel, frozen=True): + id: str + + class _Pending(_Record, frozen=True): rule: Mapping[str, object] notified_channels: tuple[UsageAlertChannel, ...] @@ -55,6 +98,20 @@ class _Ownership(BaseModel, frozen=True): team_id: str | None +_DEFAULT_RULE_IDS: Final = frozenset( + str(uuid5(NAMESPACE_URL, f"litellm:usage-alert:{scope}:{metric}:{direction}")) + for scope, metric, direction in product( + ("deployment", "key", "user", "team", "model"), + ("spend", "requests", "latency", "errors"), + ("increase", "decrease"), + ) +) + + +def _rule(record: Mapping[str, object]) -> UsageAlertRule: + return UsageAlertRule.model_validate({**record, "is_default": str(record.get("id")) in _DEFAULT_RULE_IDS}) + + def activity_path( scope: UsageAlertScope, entity_id: str | None, day: date, alert_id: str, baseline_days: int = 7 ) -> str: @@ -151,7 +208,7 @@ class UsageAlertsRepository: 'SELECT record FROM "LiteLLM_UsageAlertRule" WHERE id=$1 AND NOT deleted', rule_id ) ) - return UsageAlertRule.model_validate(rows[0].record) if rows else None + return _rule(rows[0].record) if rows else None async def save_rule( self, data: UsageAlertRuleInput, created_by: str | None, existing: UsageAlertRule | None = None @@ -177,7 +234,7 @@ class UsageAlertsRepository: rule.model_dump_json(), ) saved: Final = TypeAdapter(tuple[_Record, ...]).validate_python(rows) - return UsageAlertRule.model_validate(saved[0].record) if saved else None + return _rule(saved[0].record) if saved else None async def delete_rule(self, rule_id: str) -> None: await self.database.query_raw( @@ -185,28 +242,46 @@ class UsageAlertsRepository: ) async def list_rules( - self, scope_type: UsageAlertScope | None = None, scope_id: str | None = None, limit: int = 50, offset: int = 0 + self, + scope_type: UsageAlertScope | None = None, + scope_id: str | None = None, + limit: int = 50, + offset: int = 0, + kind: UsageAlertRuleKind = "all", ) -> UsageAlertRuleList: - args: Final = (scope_type, scope_id) + args: Final = (scope_type, scope_id, json.dumps(tuple(sorted(_DEFAULT_RULE_IDS))), kind) where: Final = "NOT deleted AND ($1::text IS NULL OR scope_type=$1) AND ($2::text IS NULL OR scope_id=$2)" - count: Final = TypeAdapter(tuple[_Count, ...]).validate_python( + builtin: Final = "id IN (SELECT jsonb_array_elements_text($3::jsonb))" + count: Final = TypeAdapter(tuple[_RuleCounts, ...]).validate_python( await self.database.query_raw( - f'SELECT count(*)::int AS count FROM "LiteLLM_UsageAlertRule" WHERE {where}', *args + f"""WITH scoped AS ( + SELECT enabled,record,({builtin}) AS builtin FROM "LiteLLM_UsageAlertRule" WHERE {where} + ) SELECT count(*) FILTER (WHERE $4='all' OR builtin=($4='default'))::int AS count, + jsonb_build_object('total',count(*) FILTER (WHERE builtin), + 'enabled',count(*) FILTER (WHERE builtin AND enabled), + 'evaluation_status_counts',jsonb_build_object( + 'pending',count(*) FILTER (WHERE builtin AND enabled AND COALESCE(record->>'evaluation_status','pending')='pending'), + 'complete',count(*) FILTER (WHERE builtin AND enabled AND record->>'evaluation_status'='complete'), + 'insufficient_coverage',count(*) FILTER (WHERE builtin AND enabled AND record->>'evaluation_status'='insufficient_coverage'), + 'insufficient_history',count(*) FILTER (WHERE builtin AND enabled AND record->>'evaluation_status'='insufficient_history') + )) AS default_summary FROM scoped""", + *args, ) ) rows: Final = TypeAdapter(tuple[_Record, ...]).validate_python( await self.database.query_raw( - f'SELECT record FROM "LiteLLM_UsageAlertRule" WHERE {where} ORDER BY id LIMIT $3 OFFSET $4', + f"SELECT record FROM \"LiteLLM_UsageAlertRule\" WHERE {where} AND ($4='all' OR ({builtin})=($4='default')) ORDER BY id LIMIT $5 OFFSET $6", *args, limit, offset, ) ) return UsageAlertRuleList( - data=tuple(UsageAlertRule.model_validate(row.record) for row in rows), + data=tuple(_rule(row.record) for row in rows), total=count[0].count, limit=limit, offset=offset, + default_summary=count[0].default_summary, ) async def list_enabled_rules(self, limit: int = 100, offset: int = 0) -> tuple[UsageAlertRule, ...]: @@ -275,6 +350,165 @@ class UsageAlertsRepository: ) return UsageAlert.model_validate(rows[0].record) if rows else None + async def list_groups( + self, + scope_type: UsageAlertScope | None = None, + scope_id: str | None = None, + status: UsageAlertStatus | None = None, + start_date: date | None = None, + end_date: date | None = None, + limit: int = 50, + offset: int = 0, + ) -> UsageAlertGroupList: + rows: Final = TypeAdapter(tuple[_GroupPage, ...]).validate_python( + await self.database.query_raw( + """WITH groups AS MATERIALIZED ( + SELECT scope_type,scope_id,day,count(*)::int AS incident_count FROM "LiteLLM_UsageAlertIncident" + WHERE ($1::text IS NULL OR scope_type=$1) AND ($2::text IS NULL OR scope_id=$2) + AND ($3::text IS NULL OR status=$3) AND ($4::text IS NULL OR day >= $4) + AND ($5::text IS NULL OR day <= $5) + GROUP BY scope_type,scope_id,day + ), page AS MATERIALIZED ( + SELECT * FROM groups ORDER BY day DESC,scope_type,scope_id NULLS FIRST LIMIT $6 OFFSET $7 + ), selected AS ( + SELECT i.* FROM page p JOIN "LiteLLM_UsageAlertIncident" i + ON i.scope_type=p.scope_type AND i.scope_id=p.scope_id AND i.day=p.day + WHERE ($3::text IS NULL OR i.status=$3) + UNION ALL + SELECT i.* FROM page p JOIN "LiteLLM_UsageAlertIncident" i + ON i.scope_type=p.scope_type AND i.scope_id IS NULL AND i.day=p.day + WHERE p.scope_id IS NULL AND ($3::text IS NULL OR i.status=$3) + ), ranked AS ( + SELECT scope_type,scope_id,day,metric,record, + count(*) OVER (PARTITION BY scope_type,scope_id,day,metric)::int AS incident_count, + row_number() OVER (PARTITION BY scope_type,scope_id,day,metric + ORDER BY record->>'first_seen' DESC,id) AS rank + FROM selected + ), packed AS ( + SELECT p.scope_type,p.scope_id,p.day,p.incident_count, + jsonb_agg(jsonb_build_object('metric',r.metric,'incident_count',r.incident_count, + 'representative',r.record) ORDER BY CASE r.metric + WHEN 'requests' THEN 0 WHEN 'spend' THEN 1 WHEN 'errors' THEN 2 ELSE 3 END) AS metrics + FROM page p JOIN ranked r ON p.scope_type=r.scope_type + AND p.scope_id IS NOT DISTINCT FROM r.scope_id AND p.day=r.day AND r.rank=1 + GROUP BY p.scope_type,p.scope_id,p.day,p.incident_count + ) SELECT (SELECT count(*)::int FROM groups) AS total, + COALESCE((SELECT jsonb_agg(to_jsonb(packed) ORDER BY day DESC,scope_type,scope_id NULLS FIRST) + FROM packed),'[]'::jsonb) AS data""", + scope_type, + scope_id, + status, + start_date.isoformat() if start_date else None, + end_date.isoformat() if end_date else None, + limit, + offset, + ) + ) + page: Final = rows[0] + labels: Final[Mapping[tuple[UsageAlertScope, str | None], str | None]] = ( + await self._group_labels(page.data) if page.data else {} + ) + return UsageAlertGroupList( + data=tuple( + UsageAlertGroup( + id=str( + uuid5( + NAMESPACE_URL, + json.dumps( + ["litellm:performance-group", row.scope_type, row.scope_id, row.day.isoformat()] + ), + ) + ), + scope_type=row.scope_type, + scope_id=row.scope_id, + scope_label=labels.get((row.scope_type, row.scope_id)), + day=row.day, + incident_count=row.incident_count, + metrics=row.metrics, + activity_path=row.metrics[0].representative.activity_path, + ) + for row in page.data + ), + total=page.total, + limit=limit, + offset=offset, + ) + + async def _group_labels( + self, groups: tuple[_GroupRow, ...] + ) -> Mapping[tuple[UsageAlertScope, str | None], str | None]: + rows: Final = TypeAdapter(tuple[_ResourceLabel, ...]).validate_python( + await self.database.query_raw( + """WITH scopes AS ( + SELECT DISTINCT scope_type,scope_id,key_id FROM jsonb_to_recordset($1::jsonb) + AS x(scope_type text,scope_id text,key_id text) + ) SELECT s.scope_type,s.scope_id,CASE s.scope_type + WHEN 'key' THEN COALESCE(NULLIF(k.key_alias,''),NULLIF(k.key_name,'')) + WHEN 'team' THEN NULLIF(t.team_alias,'') + WHEN 'user' THEN COALESCE(NULLIF(u.user_alias,''),NULLIF(u.user_email,'')) + WHEN 'model' THEN s.scope_id WHEN 'deployment' THEN 'Gateway' END AS label + FROM scopes s LEFT JOIN "LiteLLM_VerificationToken" k ON s.scope_type='key' AND k.token=s.key_id + LEFT JOIN "LiteLLM_TeamTable" t ON s.scope_type='team' AND t.team_id=s.scope_id + LEFT JOIN "LiteLLM_UserTable" u ON s.scope_type='user' AND u.user_id=s.scope_id""", + json.dumps( + tuple( + { + "scope_type": row.scope_type, + "scope_id": row.scope_id, + "key_id": api_key_from_bucket(row.scope_id) + if row.scope_type == "key" and row.scope_id + else None, + } + for row in groups + ) + ), + ) + ) + return MappingProxyType({(row.scope_type, row.scope_id): row.label for row in rows}) + + async def set_group_status(self, data: UsageAlertGroupUpdate) -> UsageAlertGroupUpdateResult: + rows: Final = TypeAdapter(tuple[_Count, ...]).validate_python( + await self.database.query_raw( + """WITH changed AS ( + UPDATE "LiteLLM_UsageAlertIncident" SET status=$5, + record=jsonb_set(record,'{status}',to_jsonb($5::text)),updated_at=CURRENT_TIMESTAMP + WHERE scope_type=$1 AND scope_id IS NOT DISTINCT FROM $2::text AND day=$3 AND status=$4 + RETURNING id + ) SELECT count(*)::int AS count FROM changed""", + data.scope_type, + data.scope_id, + data.day.isoformat(), + data.expected_status, + data.status, + ) + ) + return UsageAlertGroupUpdateResult(updated_count=rows[0].count) + + async def list_reported_models(self, search: str | None, cursor: str | None, limit: int) -> UsageAlertModelList: + today: Final = datetime.now(timezone.utc).date() + rows: Final = TypeAdapter(tuple[_ModelIdentity, ...]).validate_python( + await self.database.query_raw( + """SELECT DISTINCT model FROM "LiteLLM_DailyUserSpend" + WHERE date >= $1 AND date <= $2 AND model IS NOT NULL AND model <> '' + AND ($3::text IS NULL OR strpos(lower(model),lower($3))>0) + AND ($4::text IS NULL OR model>$4) + ORDER BY model LIMIT $5""", + (today - timedelta(days=29)).isoformat(), + today.isoformat(), + search, + cursor, + limit + 1, + ) + ) + selected: Final = rows[:limit] + return UsageAlertModelList( + data=tuple(UsageAlertModelOption(scope_id=row.model, label=row.model) for row in selected), + limit=limit, + next_cursor=selected[-1].model if len(rows) > limit else None, + start_date=today - timedelta(days=29), + end_date=today, + ) + async def key_ownership(self, key_id: str) -> tuple[str | None, str | None] | None: rows: Final = TypeAdapter(tuple[_Ownership, ...]).validate_python( await self.database.query_raw( diff --git a/litellm/types/proxy/usage_alerts.py b/litellm/types/proxy/usage_alerts.py index 070586fef5b..bb719ea48a2 100644 --- a/litellm/types/proxy/usage_alerts.py +++ b/litellm/types/proxy/usage_alerts.py @@ -11,6 +11,7 @@ UsageAlertDirection: TypeAlias = Literal["increase", "decrease"] UsageAlertStatus: TypeAlias = Literal["open", "resolved", "dismissed"] UsageAlertChannel: TypeAlias = Literal["email", "slack"] UsageAlertEvaluationStatus: TypeAlias = Literal["pending", "complete", "insufficient_coverage", "insufficient_history"] +UsageAlertRuleKind: TypeAlias = Literal["all", "default", "custom"] class UsageAlertRuleInput(BaseModel): @@ -60,6 +61,7 @@ class UsageAlertRule(UsageAlertRuleInput): created_by: str | None last_evaluated_day: date | None = None evaluation_status: UsageAlertEvaluationStatus = "pending" + is_default: bool = False class UsageAlert(BaseModel): @@ -90,11 +92,25 @@ class UsageAlertUpdate(BaseModel): status: UsageAlertStatus +class UsageAlertEvaluationCounts(BaseModel, frozen=True): + pending: int = 0 + complete: int = 0 + insufficient_coverage: int = 0 + insufficient_history: int = 0 + + +class UsageAlertDefaultSummary(BaseModel, frozen=True): + total: int = 0 + enabled: int = 0 + evaluation_status_counts: UsageAlertEvaluationCounts = Field(default_factory=UsageAlertEvaluationCounts) + + class UsageAlertRuleList(BaseModel, frozen=True): data: tuple[UsageAlertRule, ...] total: int limit: int offset: int + default_summary: UsageAlertDefaultSummary = Field(default_factory=UsageAlertDefaultSummary) class UsageAlertList(BaseModel, frozen=True): @@ -104,6 +120,64 @@ class UsageAlertList(BaseModel, frozen=True): offset: int +class UsageAlertGroupMetric(BaseModel, frozen=True): + metric: UsageAlertMetric + incident_count: int + representative: UsageAlert + + +class UsageAlertGroup(BaseModel, frozen=True): + id: str + scope_type: UsageAlertScope + scope_id: str | None + scope_label: str | None + day: date + incident_count: int + metrics: tuple[UsageAlertGroupMetric, ...] = Field(max_length=4) + activity_path: str + + +class UsageAlertGroupList(BaseModel, frozen=True): + data: tuple[UsageAlertGroup, ...] + total: int + limit: int + offset: int + + +class UsageAlertGroupUpdate(BaseModel, frozen=True): + model_config = ConfigDict(extra="forbid") + scope_type: UsageAlertScope + scope_id: str | None = Field(default=None, min_length=1, max_length=1024) + day: date + expected_status: UsageAlertStatus + status: UsageAlertStatus + + @model_validator(mode="after") + def validate_group(self) -> Self: + if (self.scope_type == "deployment") != (self.scope_id is None): + raise ValueError("Specify a resource ID except for deployment alerts") + if self.expected_status == self.status: + raise ValueError("The new status must differ from the expected status") + return self + + +class UsageAlertGroupUpdateResult(BaseModel, frozen=True): + updated_count: int + + +class UsageAlertModelOption(BaseModel, frozen=True): + scope_id: str + label: str + + +class UsageAlertModelList(BaseModel, frozen=True): + data: tuple[UsageAlertModelOption, ...] + limit: int + next_cursor: str | None + start_date: date + end_date: date + + class UsageAlertNotification(BaseModel): model_config = ConfigDict(frozen=True) incident: UsageAlert diff --git a/tests/integration/database/test_usage_alerts.py b/tests/integration/database/test_usage_alerts.py index 6bf08bfd4a9..a43b5858fe0 100644 --- a/tests/integration/database/test_usage_alerts.py +++ b/tests/integration/database/test_usage_alerts.py @@ -1,13 +1,21 @@ -from datetime import date +from datetime import date, timedelta from hashlib import sha256 from typing import Final +from uuid import uuid4 import pytest from integration._support.actors import ManagementDatabase from integration._support.client import Gateway, string_value from litellm.repositories.usage_alerts import UsageAlertsRepository -from litellm.types.proxy.usage_alerts import UsageAlertObservation, UsageAlertRule, UsageAlertRuleInput +from litellm.types.proxy.usage_alerts import ( + UsageAlertGroupList, + UsageAlertMetric, + UsageAlertModelList, + UsageAlertObservation, + UsageAlertRule, + UsageAlertRuleInput, +) def test_scoped_rules_use_real_management_auth_and_preserve_read_only_roles(gateway: Gateway) -> None: @@ -252,3 +260,277 @@ async def test_rule_identity_edits_preserve_old_incident_and_filter_consistency( assert resolved is not None and resolved.status == "resolved" finally: await repository.delete_rule(rule.id) + + +@pytest.mark.asyncio(loop_scope="session") +async def test_grouped_http_pages_preserve_all_metrics_history_and_scoped_actions( + gateway: Gateway, db: ManagementDatabase +) -> None: + repository: Final = UsageAlertsRepository.from_prisma(db.prisma) + day: Final = date(2026, 11, 4) + with gateway.scenario() as scenario: + owner: Final = scenario.user(user_role="internal_user", user_alias="Checkout operator") + outsider: Final = scenario.user(user_role="internal_user") + viewer: Final = scenario.user(user_role="proxy_admin_viewer") + owner_key: Final = scenario.key(user_id=owner, key_type="management") + outsider_key: Final = scenario.key(user_id=outsider, key_type="management") + viewer_key: Final = scenario.key(user_id=viewer, key_type="read_only") + inference_key: Final = scenario.key(user_id=owner, key_type="llm_api") + metrics: Final[tuple[UsageAlertMetric, ...]] = (*(("requests",) * 27), "spend", "errors", "latency") + rules: Final = tuple( + [ + await repository.save_rule( + UsageAlertRuleInput( + name=f"Group rule {index}", + scope_type="user", + scope_id=owner, + metric=metric, + direction="increase", + baseline_days=7 if index % 2 else 14, + ), + owner, + ) + for index, metric in enumerate(metrics) + ] + ) + assert all(rule is not None for rule in rules) + incidents: Final = tuple( + [ + ( + await repository.record_observations( + rule, + ( + UsageAlertObservation( + entity_id=owner, + day=day, + observed=420, + baseline=0 if index == 0 else 120, + threshold=100 if index == 0 else 360, + breached=True, + ), + ), + ) + )[0] + for index, rule in enumerate(rules) + if rule is not None + ] + ) + old_rule: Final = await repository.save_rule( + UsageAlertRuleInput( + name="Earlier day", scope_type="user", scope_id=owner, metric="requests", direction="increase" + ), + owner, + ) + assert old_rule is not None + old: Final = ( + await repository.record_observations( + old_rule, + ( + UsageAlertObservation( + entity_id=owner, + day=day - timedelta(days=1), + observed=420, + baseline=120, + threshold=360, + breached=True, + ), + ), + ) + )[0] + other_rule: Final = await repository.save_rule( + UsageAlertRuleInput( + name="Other resource", scope_type="user", scope_id=outsider, metric="requests", direction="increase" + ), + outsider, + ) + assert other_rule is not None + other: Final = ( + await repository.record_observations( + other_rule, + ( + UsageAlertObservation( + entity_id=outsider, + day=day, + observed=420, + baseline=120, + threshold=360, + breached=True, + ), + ), + ) + )[0] + params: Final = {"scope_type": "user", "scope_id": owner, "status": "open", "limit": "1"} + try: + response: Final = gateway.request("GET", "/usage/alert-groups", key=owner_key, params=params) + assert response.status_code == 200, response.text + page: Final = UsageAlertGroupList.model_validate(response.json()) + assert page.total == 2 and len(page.data) == 1 + group: Final = page.data[0] + assert group.day == day and group.scope_label == "Checkout operator" + assert group.incident_count == 30 and len(group.metrics) == 4 + assert {metric.metric: metric.incident_count for metric in group.metrics} == { + "requests": 27, + "spend": 1, + "errors": 1, + "latency": 1, + } + assert all(metric.representative.id in {incident.id for incident in incidents} for metric in group.metrics) + assert group.activity_path in {incident.activity_path for incident in incidents} + second: Final = UsageAlertGroupList.model_validate( + gateway.request("GET", "/usage/alert-groups", key=owner_key, params={**params, "offset": "1"}).json() + ) + assert second.total == 2 and second.data[0].day == old.day + empty: Final = UsageAlertGroupList.model_validate( + gateway.request("GET", "/usage/alert-groups", key=owner_key, params={**params, "offset": "2"}).json() + ) + assert empty.total == 2 and empty.data == () + history: Final = gateway.request( + "GET", + "/usage/alerts", + key=owner_key, + params={**params, "start_date": day.isoformat(), "end_date": day.isoformat(), "metric": "requests"}, + ) + assert history.status_code == 200 and history.json()["total"] == 27 + await repository.set_status(incidents[0].id, "dismissed") + body: Final = { + "scope_type": "user", + "scope_id": owner, + "day": day.isoformat(), + "expected_status": "open", + "status": "resolved", + } + for forbidden in (outsider_key, inference_key): + assert gateway.request("GET", "/usage/alert-groups", key=forbidden, params=params).status_code == 403 + for forbidden in (outsider_key, inference_key, viewer_key): + assert gateway.request("PATCH", "/usage/alert-groups", body, key=forbidden).status_code == 403 + assert gateway.request("GET", "/usage/alert-groups", key=viewer_key, params=params).status_code == 200 + updated: Final = gateway.request("PATCH", "/usage/alert-groups", body, key=owner_key) + assert updated.status_code == 200 and updated.json()["updated_count"] == 29 + assert gateway.request("PATCH", "/usage/alert-groups", body, key=owner_key).json()["updated_count"] == 0 + assert await repository.get_alert(old.id) == old + assert await repository.get_alert(other.id) == other + dismissed: Final = await repository.get_alert(incidents[0].id) + assert dismissed is not None and dismissed.status == "dismissed" and dismissed.baseline_value == 0 + resolved: Final = await repository.get_alert(incidents[-1].id) + assert resolved == incidents[-1].model_copy(update={"status": "resolved"}) + assert ( + gateway.request( + "GET", "/usage/alert-groups", params={"scope_type": "user", "scope_id": "x' OR 1=1 --"} + ).json()["total"] + == 0 + ) + assert ( + gateway.request( + "GET", "/usage/alert-groups", params={"start_date": "2026-11-05", "end_date": "2026-11-04"} + ).status_code + == 422 + ) + finally: + for rule in (*rules, old_rule, other_rule): + if rule is not None: + await repository.delete_rule(rule.id) + deleted: Final = await repository.list_groups(scope_type="user", scope_id=owner) + assert deleted.total == 2 and all(group.scope_label is None and group.scope_id == owner for group in deleted.data) + + +@pytest.mark.asyncio(loop_scope="session") +async def test_default_rule_summary_and_kind_filters_do_not_require_loading_every_rule( + gateway: Gateway, db: ManagementDatabase +) -> None: + repository: Final = UsageAlertsRepository.from_prisma(db.prisma) + default: Final = UsageAlertRuleInput( + name="Built-in daily model requests", scope_type="model", metric="requests", direction="increase" + ) + await repository.ensure_default_rules((default,)) + custom: Final = await repository.save_rule( + default.model_copy(update={"scope_id": f"model-{uuid4().hex}"}), "qa-admin" + ) + assert custom is not None + try: + all_rules: Final = await repository.list_rules(scope_type="model", limit=1) + builtins: Final = await repository.list_rules(scope_type="model", limit=1, kind="default") + customs: Final = await repository.list_rules(scope_type="model", limit=1, kind="custom") + assert builtins.total >= 1 and all(rule.is_default for rule in builtins.data) + assert customs.total >= 1 and all(not rule.is_default for rule in customs.data) + assert all_rules.total == builtins.total + customs.total + assert all_rules.default_summary == builtins.default_summary == customs.default_summary + assert builtins.default_summary.total == builtins.total + assert ( + sum(builtins.default_summary.evaluation_status_counts.model_dump().values()) + == builtins.default_summary.enabled + ) + assert (await repository.list_rules(scope_type="model", kind="default", offset=builtins.total)).data == () + stored: Final = await repository.get_rule(builtins.data[0].id) + assert stored is not None and stored.is_default + filtered: Final = gateway.request( + "GET", "/usage/alert-rules", params={"scope_type": "model", "kind": "custom", "limit": "1"} + ) + assert filtered.status_code == 200 and filtered.json()["total"] == customs.total + assert all(not rule["is_default"] for rule in filtered.json()["data"]) + configuration: Final = stored.model_dump(mode="json", include=frozenset(UsageAlertRuleInput.model_fields)) + path: Final = f"/usage/alert-rules/{stored.id}" + for change in ( + {"scope_type": "key"}, + {"scope_id": "model-example"}, + {"metric": "requests" if stored.metric != "requests" else "spend"}, + {"direction": "increase" if stored.direction == "decrease" else "decrease"}, + ): + assert gateway.request("PUT", path, {**configuration, **change}).status_code == 422 + assert await repository.get_rule(stored.id) == stored + try: + updated: Final = gateway.request("PUT", path, {**configuration, "min_value": stored.min_value + 1}) + assert updated.status_code == 200 and updated.json()["is_default"] is True + assert updated.json()["min_value"] == stored.min_value + 1 + finally: + assert gateway.request("PUT", path, configuration).status_code == 200 + custom_input: Final = custom.model_dump(mode="json", include=frozenset(UsageAlertRuleInput.model_fields)) + assert ( + gateway.request("PUT", f"/usage/alert-rules/{custom.id}", {**custom_input, "metric": "latency"}).status_code + == 200 + ) + finally: + await repository.delete_rule(custom.id) + + +@pytest.mark.asyncio(loop_scope="session") +async def test_reported_model_search_preserves_exact_identity_and_cursor( + gateway: Gateway, db: ManagementDatabase +) -> None: + repository: Final = UsageAlertsRepository.from_prisma(db.prisma) + prefix: Final = f"reported-{uuid4().hex}" + initial: Final = await repository.list_reported_models(prefix, None, 1) + assert initial.data == () + models: Final = (f"{prefix}/first", f"{prefix}/literal%quote'", f"{prefix}/second") + try: + for index, model in enumerate((*models, models[0])): + await db.prisma.writer_db.execute_raw( + 'INSERT INTO "LiteLLM_DailyUserSpend" (id,date,api_key,model,updated_at) VALUES ($1,$2,$3,$4,CURRENT_TIMESTAMP)', + f"{prefix}-{index}", + initial.end_date.isoformat(), + prefix, + model, + ) + await db.prisma.writer_db.execute_raw( + 'INSERT INTO "LiteLLM_DailyUserSpend" (id,date,api_key,model,updated_at) VALUES ($1,$2,$3,$4,CURRENT_TIMESTAMP)', + f"{prefix}-old", + (initial.start_date - timedelta(days=1)).isoformat(), + prefix, + f"{prefix}/old", + ) + response: Final = gateway.request("GET", "/usage/alert-models", params={"search": prefix, "limit": "1"}) + assert response.status_code == 200, response.text + first: Final = UsageAlertModelList.model_validate(response.json()) + assert first.data[0].scope_id == models[0] and first.next_cursor == models[0] + second: Final = await repository.list_reported_models(prefix, first.next_cursor, 1) + third: Final = await repository.list_reported_models(prefix, second.next_cursor, 1) + assert second.data[0].scope_id == models[1] and third.data[0].scope_id == models[2] + assert third.next_cursor is None + literal: Final = await repository.list_reported_models(f"{prefix}/literal%quote'", None, 20) + assert tuple(option.scope_id for option in literal.data) == (models[1],) + assert literal.data[0].label == models[1] + with gateway.scenario() as scenario: + user: Final = scenario.user(user_role="internal_user") + key: Final = scenario.key(user_id=user, key_type="management") + assert gateway.request("GET", "/usage/alert-models", key=key).status_code == 403 + finally: + await db.prisma.writer_db.execute_raw('DELETE FROM "LiteLLM_DailyUserSpend" WHERE api_key=$1', prefix) diff --git a/tests/unit/types/proxy/test_usage_alerts.py b/tests/unit/types/proxy/test_usage_alerts.py index ad8aaaf493e..57d7e133438 100644 --- a/tests/unit/types/proxy/test_usage_alerts.py +++ b/tests/unit/types/proxy/test_usage_alerts.py @@ -3,7 +3,7 @@ from typing import Final import pytest from pydantic import ValidationError -from litellm.types.proxy.usage_alerts import UsageAlertRuleInput +from litellm.types.proxy.usage_alerts import UsageAlertGroupUpdate, UsageAlertRuleInput RULE: Final = { "name": "Daily spend", @@ -41,3 +41,21 @@ def test_zero_request_floor_and_daily_drop_multiplier_are_supported() -> None: assert rule.direction == "decrease" assert rule.multiplier == 2 assert rule.min_requests == 0 + + +@pytest.mark.parametrize( + "changes", + [{"scope_id": None}, {"scope_type": "deployment"}, {"status": "open"}, {"status": "unknown"}], +) +def test_group_updates_require_exact_resource_and_a_different_valid_status(changes: dict[str, object]) -> None: + with pytest.raises(ValidationError): + UsageAlertGroupUpdate.model_validate( + { + "scope_type": "team", + "scope_id": "team-example", + "day": "2026-11-04", + "expected_status": "open", + "status": "resolved", + **changes, + } + ) diff --git a/ui/litellm-dashboard/src/lib/http/schema.d.ts b/ui/litellm-dashboard/src/lib/http/schema.d.ts index cd55938d7c3..42ae1107a2e 100644 --- a/ui/litellm-dashboard/src/lib/http/schema.d.ts +++ b/ui/litellm-dashboard/src/lib/http/schema.d.ts @@ -18834,6 +18834,41 @@ export interface paths { patch?: never; trace?: never; }; + "/usage/alert-groups": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** List Usage Alert Groups */ + get: operations["list_usage_alert_groups_usage_alert_groups_get"]; + put?: never; + post?: never; + delete?: never; + options?: never; + head?: never; + /** Update Usage Alert Group */ + patch: operations["update_usage_alert_group_usage_alert_groups_patch"]; + trace?: never; + }; + "/usage/alert-models": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** List Usage Alert Models */ + get: operations["list_usage_alert_models_usage_alert_models_get"]; + put?: never; + post?: never; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; "/usage/alert-rules": { parameters: { query?: never; @@ -50472,6 +50507,120 @@ export interface components { /** Threshold */ threshold: number; }; + /** UsageAlertDefaultSummary */ + UsageAlertDefaultSummary: { + /** + * Enabled + * @default 0 + */ + enabled: number; + evaluation_status_counts?: components["schemas"]["UsageAlertEvaluationCounts"]; + /** + * Total + * @default 0 + */ + total: number; + }; + /** UsageAlertEvaluationCounts */ + UsageAlertEvaluationCounts: { + /** + * Complete + * @default 0 + */ + complete: number; + /** + * Insufficient Coverage + * @default 0 + */ + insufficient_coverage: number; + /** + * Insufficient History + * @default 0 + */ + insufficient_history: number; + /** + * Pending + * @default 0 + */ + pending: number; + }; + /** UsageAlertGroup */ + UsageAlertGroup: { + /** Activity Path */ + activity_path: string; + /** + * Day + * Format: date + */ + day: string; + /** Id */ + id: string; + /** Incident Count */ + incident_count: number; + /** Metrics */ + metrics: components["schemas"]["UsageAlertGroupMetric"][]; + /** Scope Id */ + scope_id: string | null; + /** Scope Label */ + scope_label: string | null; + /** + * Scope Type + * @enum {string} + */ + scope_type: "deployment" | "key" | "user" | "team" | "model"; + }; + /** UsageAlertGroupList */ + UsageAlertGroupList: { + /** Data */ + data: components["schemas"]["UsageAlertGroup"][]; + /** Limit */ + limit: number; + /** Offset */ + offset: number; + /** Total */ + total: number; + }; + /** UsageAlertGroupMetric */ + UsageAlertGroupMetric: { + /** Incident Count */ + incident_count: number; + /** + * Metric + * @enum {string} + */ + metric: "spend" | "requests" | "latency" | "errors"; + representative: components["schemas"]["UsageAlert"]; + }; + /** UsageAlertGroupUpdate */ + UsageAlertGroupUpdate: { + /** + * Day + * Format: date + */ + day: string; + /** + * Expected Status + * @enum {string} + */ + expected_status: "open" | "resolved" | "dismissed"; + /** Scope Id */ + scope_id?: string | null; + /** + * Scope Type + * @enum {string} + */ + scope_type: "deployment" | "key" | "user" | "team" | "model"; + /** + * Status + * @enum {string} + */ + status: "open" | "resolved" | "dismissed"; + }; + /** UsageAlertGroupUpdateResult */ + UsageAlertGroupUpdateResult: { + /** Updated Count */ + updated_count: number; + }; /** UsageAlertList */ UsageAlertList: { /** Data */ @@ -50483,6 +50632,32 @@ export interface components { /** Total */ total: number; }; + /** UsageAlertModelList */ + UsageAlertModelList: { + /** Data */ + data: components["schemas"]["UsageAlertModelOption"][]; + /** + * End Date + * Format: date + */ + end_date: string; + /** Limit */ + limit: number; + /** Next Cursor */ + next_cursor: string | null; + /** + * Start Date + * Format: date + */ + start_date: string; + }; + /** UsageAlertModelOption */ + UsageAlertModelOption: { + /** Label */ + label: string; + /** Scope Id */ + scope_id: string; + }; /** UsageAlertRule */ UsageAlertRule: { /** @@ -50520,6 +50695,11 @@ export interface components { evaluation_status: "pending" | "complete" | "insufficient_coverage" | "insufficient_history"; /** Id */ id: string; + /** + * Is Default + * @default false + */ + is_default: boolean; /** Last Evaluated Day */ last_evaluated_day?: string | null; /** @@ -50635,6 +50815,7 @@ export interface components { UsageAlertRuleList: { /** Data */ data: components["schemas"]["UsageAlertRule"][]; + default_summary?: components["schemas"]["UsageAlertDefaultSummary"]; /** Limit */ limit: number; /** Offset */ @@ -78561,6 +78742,109 @@ export interface operations { }; }; }; + list_usage_alert_groups_usage_alert_groups_get: { + parameters: { + query?: { + scope_type?: ("deployment" | "key" | "user" | "team" | "model") | null; + scope_id?: string | null; + status?: ("open" | "resolved" | "dismissed") | null; + start_date?: string | null; + end_date?: string | null; + limit?: number; + offset?: number; + }; + header?: never; + path?: never; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["UsageAlertGroupList"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + update_usage_alert_group_usage_alert_groups_patch: { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["UsageAlertGroupUpdate"]; + }; + }; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["UsageAlertGroupUpdateResult"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + list_usage_alert_models_usage_alert_models_get: { + parameters: { + query?: { + search?: string | null; + cursor?: string | null; + limit?: number; + }; + header?: never; + path?: never; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["UsageAlertModelList"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; list_usage_alert_rules_usage_alert_rules_get: { parameters: { query?: { @@ -78568,6 +78852,7 @@ export interface operations { scope_id?: string | null; limit?: number; offset?: number; + kind?: "all" | "default" | "custom"; }; header?: never; path?: never;