mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-19 00:01:29 +00:00
e30b7f2f1f
Adds auto-router as a third savings driver alongside compression and prompt caching: a summary card, a donut segment, and a series in the savings graph. Savings are the net dollars from routing a request to the selected model instead of a counterfactual baseline. The delta prices prompt tokens at each model's input rate and completion tokens at each model's output rate, then subtracts the cache-write penalty the selected deployment incurs on a cold cache. Cache-read discounts stay attributed to the prompt-caching driver to avoid double-counting. The result is floored at zero so an escalation to a pricier model never reads as negative savings. The baseline model defaults to claude-opus-5 and is operator-configurable per deployment via the auto_router_savings_baseline_model litellm_param. It flows AutoRouter to PreRoutingHookResponse to the metadata bucket to SpendLogsMetadata to the daily spend writer, mirroring the routing_decision path, and is stripped from untrusted caller metadata so it cannot be spoofed. Savings accrue into a new autorouter_savings_spend column on the six LiteLLM_Daily*Spend rollup tables; no LiteLLM_SpendLogs queries are added. |
||
|---|---|---|
| .. | ||
| dist | ||
| litellm_proxy_extras | ||
| tests | ||
| build_and_publish.md | ||
| LICENSE | ||
| migration_runbook.md | ||
| pyproject.toml | ||
| README.md | ||
Additional files for the proxy. Reduces the size of the main litellm package.
Currently, only stores the migration.sql files for litellm-proxy.
To install, run:
uv add litellm-proxy-extras
OR
uv tool install 'litellm[proxy]' # installs litellm-proxy-extras and other proxy dependencies
To use the migrations, run:
litellm --use_prisma_migrate