mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-10 08:28:25 +00:00
8ab0321816
* fix(proxy): charge a video's provider-reported cost once, on the first poll that sees it completed
A custom video provider can report a video's cost on the status poll that
sees it completed. Every poll of a video shared the video id as its spend
log request_id, so the paid poll's row was dropped as a duplicate and every
later poll of the finished video charged the cost again.
Video status rows now carry their own `_video_cost` request_id, and a poll
that reports a cost goes through the same settle-once claim batches use, so
the first poll that sees the video completed charges it and writes the row,
and every later poll charges nothing.
* fix(proxy): charge a video's provider-reported cost once on model, router and session budgets too
A video status poll gets the _video_cost row suffix only when it reports a cost, so a $0 poll keeps the bare video id and adds no row of its own, and the spend log writer claims a row by that suffix.
The batch charge-once marker now covers videos, keyed on the video id under {spend_key}:video:{id}, with the batch marker key unchanged. Model and router budgets use it, and per-session budgets now charge a repolled batch or video once and ignore a cost that is not a finite number
* fix(proxy): charge a finished video's own cost once, keyed by the id the client polled
Guardrail cost on a running poll no longer takes the charge-once claim, only a completed poll charges, the session marker TTL refreshes like the others, and two deployments whose provider ids match each keep their own charge.
* fix(proxy): charge repeat poll guardrails, key video charges by canonical id and survive redaction
* fix(proxy): keep video status polls in-process for the collector and give guardrail-only job polls an unclaimable row id
* fix(proxy): keep repeat batch polls free and group guardrail-only batch rows under their batch's trace
* fix(proxy): settle a finished job whose own cost is below its poll's guardrail cost under the job's charge row
* test(proxy): count charged_poll_cost as covered through the finished job guardrail poll tests
* fix(proxy): settle a finished batch that cost nothing itself on its first priced retrieve
* fix(proxy): charge no budget for a batch retrieve before its cost is final, and read the retrieved batch from before redaction
* fix(proxy): charge sessions a poll's guardrail-inclusive cost, count a job cost row's guardrail run, and keep batch_cost_is_final importable from litellm.batches
* fix(proxy): claim a job and charge the session budget in one Redis call
The session limiter claimed the job marker and then raised spend in a second Redis call. A failure between the two left the job claimed while its charge sat only in local memory, so later polls skipped it. One script now does both, and a refused script claims and charges locally
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_background_interaction_settlement.py | ||
| test_baseline_accounting.py | ||
| test_budget_reservation.py | ||
| test_budget_reservation_redis_failure.py | ||
| test_carried_budget_state.py | ||
| test_cloudzero_endpoints.py | ||
| test_compression_savings.py | ||
| test_cost_finalization.py | ||
| test_input_tokens.py | ||
| test_key_metadata_recovery.py | ||
| test_ptu_feature_flag.py | ||
| test_ptu_flat_cost_rollup.py | ||
| test_savings.py | ||
| test_search_api_logging.py | ||
| test_session_page.py | ||
| test_spend_capture_rate.py | ||
| test_spend_counter_batch.py | ||
| test_spend_event.py | ||
| test_spend_event_producer.py | ||
| test_spend_log_error_logger.py | ||
| test_spend_management_endpoints.py | ||
| test_spend_query_optimization.py | ||
| test_spend_tracking_utils.py | ||
| test_subscription_billing_periods.py | ||