mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-12 08:42:18 +00:00
894552205b
* feat(spend): report guardrail spend separately in daily activity Model and key spend in /user/daily/activity/aggregated include the cost of guardrails like Bedrock guardrails, and nothing in the response says how much. A customer that bills tenants per key and per model could not get model cost alone, so tenants that pay AWS for their own guardrails were charged for them twice Daily spend rows now carry a guardrail_spend column next to spend, and every breakdown in the daily activity responses exposes it, so model cost is spend - guardrail_spend. spend keeps its meaning, so budgets still count guardrail cost as before The value comes from the request's cost_breakdown.guardrail_cost, capped at the row's spend because an allowance discount or a repeat job poll can charge less than the guardrail cost the breakdown records. The delivery ledger freezes it into the accounting payload next to accounting_savings, since the stored payload does not keep cost_breakdown, and the legacy daily queue writes the same value The migration only adds the column with a 0 default, so rows written before it read 0 * fix(spend): count guardrail spend on rows billed without a cost breakdown A request blocked before the LLM call is billed for its guardrails straight from guardrail_information, so it carries no cost_breakdown and its guardrail_spend read 0. Both writers now price guardrail_spend from guardrail_information with guardrail_information_cost, the same function success and failure rows use to add guardrail cost to spend Also fills guardrail_spend in the client usage view built from request logs, qualifies the field description for rows recorded before it existed, and adds the lock timeout the migration checks require * fix(ui): avoid a nested ternary when reading guardrail entries from request logs --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| dist | ||
| litellm_proxy_extras | ||
| tests | ||
| build_and_publish.md | ||
| LICENSE | ||
| migration_runbook.md | ||
| pyproject.toml | ||
| README.md | ||
Additional files for the proxy. Reduces the size of the main litellm package.
Currently, only stores the migration.sql files for litellm-proxy.
To install, run:
uv add litellm-proxy-extras
OR
uv tool install 'litellm[proxy]' # installs litellm-proxy-extras and other proxy dependencies
To use the migrations, run:
litellm --use_prisma_migrate