mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
8b63b7045c
Background interaction billing previously lived only in the creating pod's memory: a DELETE routed to another pod or a proxy restart meant the completed provider work was never billed and the budget reservation was refunded at the poll timeout. The poll task now persists a settlement context row (virtual key attribution, model, original request id, budget reservation) in a new LiteLLM_BackgroundInteractionSettlementTable so any pod can settle. A conditional update flipping status pending to settled is the cross-pod exactly-once claim; every outcome (bill, release, timeout give-up) must win it before touching spend counters, and a losing pod finalizes its local reservation copy without touching shared counters. The delete route settles registry misses through the store by rebuilding a billable logging context, resolving provider credentials through the router by model group. A per-pod sweep job resumes pending settlements after restarts and surfaces timed-out entries as abandoned rows instead of silently reconciling to zero. Without a registered store (SDK usage) behavior is unchanged. |
||
|---|---|---|
| .. | ||
| dist | ||
| litellm_proxy_extras | ||
| tests | ||
| build_and_publish.md | ||
| LICENSE | ||
| migration_runbook.md | ||
| pyproject.toml | ||
| README.md | ||
Additional files for the proxy. Reduces the size of the main litellm package.
Currently, only stores the migration.sql files for litellm-proxy.
To install, run:
uv add litellm-proxy-extras
OR
uv tool install 'litellm[proxy]' # installs litellm-proxy-extras and other proxy dependencies
To use the migrations, run:
litellm --use_prisma_migrate