mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-06 07:48:06 +00:00
8ab0321816
* fix(proxy): charge a video's provider-reported cost once, on the first poll that sees it completed
A custom video provider can report a video's cost on the status poll that
sees it completed. Every poll of a video shared the video id as its spend
log request_id, so the paid poll's row was dropped as a duplicate and every
later poll of the finished video charged the cost again.
Video status rows now carry their own `_video_cost` request_id, and a poll
that reports a cost goes through the same settle-once claim batches use, so
the first poll that sees the video completed charges it and writes the row,
and every later poll charges nothing.
* fix(proxy): charge a video's provider-reported cost once on model, router and session budgets too
A video status poll gets the _video_cost row suffix only when it reports a cost, so a $0 poll keeps the bare video id and adds no row of its own, and the spend log writer claims a row by that suffix.
The batch charge-once marker now covers videos, keyed on the video id under {spend_key}:video:{id}, with the batch marker key unchanged. Model and router budgets use it, and per-session budgets now charge a repolled batch or video once and ignore a cost that is not a finite number
* fix(proxy): charge a finished video's own cost once, keyed by the id the client polled
Guardrail cost on a running poll no longer takes the charge-once claim, only a completed poll charges, the session marker TTL refreshes like the others, and two deployments whose provider ids match each keep their own charge.
* fix(proxy): charge repeat poll guardrails, key video charges by canonical id and survive redaction
* fix(proxy): keep video status polls in-process for the collector and give guardrail-only job polls an unclaimable row id
* fix(proxy): keep repeat batch polls free and group guardrail-only batch rows under their batch's trace
* fix(proxy): settle a finished job whose own cost is below its poll's guardrail cost under the job's charge row
* test(proxy): count charged_poll_cost as covered through the finished job guardrail poll tests
* fix(proxy): settle a finished batch that cost nothing itself on its first priced retrieve
* fix(proxy): charge no budget for a batch retrieve before its cost is final, and read the retrieved batch from before redaction
* fix(proxy): charge sessions a poll's guardrail-inclusive cost, count a job cost row's guardrail run, and keep batch_cost_is_final importable from litellm.batches
* fix(proxy): claim a job and charge the session budget in one Redis call
The session limiter claimed the job marker and then raised spend in a second Redis call. A failure between the two left the job claimed while its charge sat only in local memory, so later polls skipped it. One script now does both, and a refused script claims and charges locally
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| db_transaction_queue | ||
| mcp_server | ||
| __init__.py | ||
| conftest.py | ||
| fake_prisma_engine.py | ||
| test_autorouter_session_rollup.py | ||
| test_budget_window_spend_writer.py | ||
| test_check_migration.py | ||
| test_create_views.py | ||
| test_daily_spend_bulk_upsert.py | ||
| test_db_lookup_gate.py | ||
| test_db_span.py | ||
| test_db_spend_update_writer.py | ||
| test_db_url_settings.py | ||
| test_exception_handler.py | ||
| test_exception_handler_reconnect_retry.py | ||
| test_gateway_request_tracking.py | ||
| test_health_check_latest.py | ||
| test_log_db_metrics.py | ||
| test_log_db_metrics_service_spans.py | ||
| test_master_key_migration.py | ||
| test_model_access_group_spend.py | ||
| test_model_insights_tasks.py | ||
| test_model_usage_rollup.py | ||
| test_object_permission_repository.py | ||
| test_pgbouncer.py | ||
| test_prisma_client.py | ||
| test_prisma_planned_engine_restart.py | ||
| test_prisma_query_span.py | ||
| test_prisma_self_heal.py | ||
| test_proxy_worker_heartbeat.py | ||
| test_query_engine_reaper.py | ||
| test_rds_iam_token_expiry.py | ||
| test_replica_identity.py | ||
| test_request_error_tracking.py | ||
| test_routing_prisma_wrapper.py | ||
| test_shadow_eval_funnel.py | ||
| test_spend_counter_reseed.py | ||
| test_spend_log_batching.py | ||
| test_spend_log_tool_index.py | ||
| test_token_auth.py | ||
| test_tool_registry_writer.py | ||
| test_tool_table_bulk_upsert.py | ||
| test_update_daily_tag_spend.py | ||