litellm/tests/unit/accounting/test_application.py
Kerry Lu d30b2b9b26
feat(proxy): enforce customer org, team and member budgets and limits on the request path (#51489)
* feat(proxy): enforce customer org, team and member budgets and limits on the request path

Resolve x-litellm-customer-member-id at auth, walk member -> team -> org, enforce max_budget,
allowed_models, tpm/rpm and per-model limits at every level in the v3 limiter, honor the hierarchy
allowlists in alternate-model steering, and settle spend to the member, team and org rows.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(auth): customer hierarchy survives virtual key auth without a legacy end user

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(auth): keys pinned to a customer organization may only act for that organization's members

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(proxy): keep the customer hierarchy spend maps within the line limit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(auth): check the key's customer org scope after the key is resolved

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): declare the hierarchy cache key family and keep type counts flat

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(auth): resolve the customer hierarchy in the shared authorization gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): drop a member's cached hierarchy through the declared cache target

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(auth): keep the hierarchy reset within the line limit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(auth): type the requested models as a sequence

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(spend): add daily customer usage and budget resets (#51493)

* feat(spend): add daily customer usage and budget resets

* chore(spend): document customer reset query bounds

* fix(spend): chart customer queue metrics and update recovery test

* fix(proxy): read stamped customer ids from either metadata bucket, alias-aware model check, broadcast member evictions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* Revert "feat(spend): add daily customer usage and budget resets (#51493)"

This reverts commit ccac47608f.

* fix(proxy): evict cached member hierarchies when team, org or template budgets change

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): declare the auth cache dependency alias as a TypeAlias

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* Reapply "feat(spend): add daily customer usage and budget resets (#51493)"

This reverts commit 7542471e17.

* fix(spend): reconcile the daily customer spend table with main

Keep main's LiteLLM_DailyCustomerMemberSpend shape, make model_group nullable and add the identity unique index the bulk upsert arbitrates on in a new migration, drop the duplicate model and migration, and feed both the member-header hierarchy and the key's customer usage attribution through the shared daily spend projection so neither path double-writes.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(pagerduty): tolerate failure metadata without customer hierarchy ids

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(accounting): key daily customer spend rows on main's hashed identity

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(pagerduty): read customer ids from failure metadata through a typed lookup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: moe-berri <moe@berri.ai>
2026-11-12 20:43:44 -08:00

70 lines
2.9 KiB
Python

from collections.abc import Mapping
from dataclasses import replace
from datetime import datetime, timezone
from typing import Final
from litellm.accounting.delivery_projection import project_deliveries
from litellm.accounting.projection import Contribution
from litellm.proxy.common_utils.customer_usage import CUSTOMER_USAGE_METADATA_KEY, CustomerUsageAttribution
from litellm.proxy.db.daily_spend_bulk_upsert import DAILY_SPEND_TABLES, merge_by_conflict_key
from litellm.repositories.accounting_delivery_repository import DeliveryProjection
_IDENTIFIER: Final = "\U0001f680" * 256
_ATTRIBUTION: Final = CustomerUsageAttribution(
customer_org_id=_IDENTIFIER, customer_team_id=_IDENTIFIER, customer_member_id=_IDENTIFIER
)
def contribution(alias: str, provider: str, spend: float) -> Contribution:
delivery: Final = DeliveryProjection(
{
"request_id": "long-customer-id-request",
"startTime": datetime(2026, 1, 1, tzinfo=timezone.utc).isoformat(),
"api_key": "hashed-key",
"model": "shared-model",
"model_group": alias,
"custom_llm_provider": provider,
"spend": spend,
"prompt_tokens": 10,
"completion_tokens": 5,
"metadata": {CUSTOMER_USAGE_METADATA_KEY: _ATTRIBUTION.model_dump_json()},
}
)
return next(item for item in project_deliveries((delivery,)) if item.table == "LiteLLM_DailyCustomerMemberSpend")
def _number(value: object) -> float:
return float(value) if isinstance(value, (int, float)) else 0.0
def _row(item: Contribution) -> Mapping[str, object]:
return {**dict(item.key), **dict(item.values), **dict(item.increments)}
def test_long_customer_ids_roll_up_without_truncation_and_aliases_and_providers_keep_distinct_rows() -> None:
first: Final = contribution("alias-a", "provider-a", 1)
second: Final = replace(contribution("alias-a", "provider-a", 2), key=tuple(reversed(first.key)))
merged: Final = merge_by_conflict_key(
DAILY_SPEND_TABLES["customer_member"],
tuple(
_row(item)
for item in (
first,
second,
contribution("alias-b", "provider-a", 4),
contribution("alias-a", "provider-b", 8),
)
),
)
rows: Final = tuple(row for _, row in merged)
assert len(rows) == 3
assert all(
row["customer_org_id"] == row["customer_team_id"] == row["customer_member_id"] == _IDENTIFIER for row in rows
)
assert {(row["model_group"], row["custom_llm_provider"]): _number(row["spend"]) for row in rows} == {
("alias-a", "provider-a"): 3,
("alias-b", "provider-a"): 4,
("alias-a", "provider-b"): 8,
}
assert sum(_number(row["api_requests"]) for row in rows) == 4
assert sum(_number(row["prompt_tokens"]) + _number(row["completion_tokens"]) for row in rows) == 60