mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
b2aff8be0f
Every pod and uvicorn worker schedules its own CheckBatchCost poller against the shared managed-object table, so two of them can select the same completed batch in one polling window and both write an aretrieve_batch spend log for it, counting that batch's cost twice. Claim the row with a compare-and-swap on batch_processed, and skip the batch when another pod already holds it. The claim sits immediately before the spend log is written rather than before the results fetch, because batch_processed is also what blocks deletion of the files the fetch reads and what keeps an unbilled row selectable by later poll cycles, so claiming up front would strand the spend of any worker that died mid-fetch. A failed spend log write hands the row back. Co-authored-by: Yassin Kortam <yassin@berri.ai> |
||
|---|---|---|
| .. | ||
| cloudformation_stack | ||
| dist | ||
| enterprise_hooks | ||
| enterprise_ui | ||
| litellm_enterprise | ||
| __init__.py | ||
| LICENSE.md | ||
| pyproject.toml | ||
| README.md | ||
LiteLLM Enterprise
Code in this folder is licensed under a commercial license. Please review the LICENSE file within the /enterprise folder
These features are covered under the LiteLLM Enterprise contract
👉 Using in an Enterprise / Need specific features ? Meet with us here
See all Enterprise Features here 👉 Docs