mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-05 07:37:50 +00:00
8ab0321816
* fix(proxy): charge a video's provider-reported cost once, on the first poll that sees it completed
A custom video provider can report a video's cost on the status poll that
sees it completed. Every poll of a video shared the video id as its spend
log request_id, so the paid poll's row was dropped as a duplicate and every
later poll of the finished video charged the cost again.
Video status rows now carry their own `_video_cost` request_id, and a poll
that reports a cost goes through the same settle-once claim batches use, so
the first poll that sees the video completed charges it and writes the row,
and every later poll charges nothing.
* fix(proxy): charge a video's provider-reported cost once on model, router and session budgets too
A video status poll gets the _video_cost row suffix only when it reports a cost, so a $0 poll keeps the bare video id and adds no row of its own, and the spend log writer claims a row by that suffix.
The batch charge-once marker now covers videos, keyed on the video id under {spend_key}:video:{id}, with the batch marker key unchanged. Model and router budgets use it, and per-session budgets now charge a repolled batch or video once and ignore a cost that is not a finite number
* fix(proxy): charge a finished video's own cost once, keyed by the id the client polled
Guardrail cost on a running poll no longer takes the charge-once claim, only a completed poll charges, the session marker TTL refreshes like the others, and two deployments whose provider ids match each keep their own charge.
* fix(proxy): charge repeat poll guardrails, key video charges by canonical id and survive redaction
* fix(proxy): keep video status polls in-process for the collector and give guardrail-only job polls an unclaimable row id
* fix(proxy): keep repeat batch polls free and group guardrail-only batch rows under their batch's trace
* fix(proxy): settle a finished job whose own cost is below its poll's guardrail cost under the job's charge row
* test(proxy): count charged_poll_cost as covered through the finished job guardrail poll tests
* fix(proxy): settle a finished batch that cost nothing itself on its first priced retrieve
* fix(proxy): charge no budget for a batch retrieve before its cost is final, and read the retrieved batch from before redaction
* fix(proxy): charge sessions a poll's guardrail-inclusive cost, count a job cost row's guardrail run, and keep batch_cost_is_final importable from litellm.batches
* fix(proxy): claim a job and charge the session budget in one Redis call
The session limiter claimed the job marker and then raised spend in a second Redis call. A failure between the two left the job claimed while its charge sat only in local memory, so later polls skipped it. One script now does both, and a refused script claims and charges locally
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| azure_client_usage_test.py | ||
| ban_constant_numbers.py | ||
| ban_copy_deepcopy_kwargs.py | ||
| bedrock_pricing.py | ||
| callback_manager_test.py | ||
| check_data_replace_usage.py | ||
| check_e2e_no_raw_requests.py | ||
| check_endpoint_coverage.py | ||
| check_fastuuid_usage.py | ||
| check_get_model_cost_key_performance.py | ||
| check_guardrail_apply_decorator.py | ||
| check_licenses.py | ||
| check_migrations_lock_timeout.py | ||
| check_migrations_no_data_rewrites.py | ||
| check_no_none_string_error_fields.py | ||
| check_no_publicly_known_master_key.py | ||
| check_prisma_binary_cache.py | ||
| check_provider_folders_documented.py | ||
| check_py310_typing_imports.py | ||
| check_spanattributes_value_usage.py | ||
| check_telemetry_imports.py | ||
| check_unbounded_in_lists.py | ||
| check_unsafe_enterprise_import.py | ||
| check_workflow_job_name_collisions.py | ||
| check_workflow_startup_safety.py | ||
| code_qa_check_tests.py | ||
| enforce_llms_folder_style.py | ||
| ensure_async_clients_test.py | ||
| info_log_check.py | ||
| liccheck.ini | ||
| license_cache.json | ||
| litellm_logging_code_coverage.py | ||
| log.txt | ||
| memory_test.py | ||
| prevent_key_leaks_in_exceptions.py | ||
| recursive_detector.py | ||
| router_code_coverage.py | ||
| router_enforce_line_length.py | ||
| router_litellm_globals_check.py | ||
| router_surface_allowlist.json | ||
| router_surface_check.py | ||
| test_aio_http_image_conversion.py | ||
| test_ban_set_verbose.py | ||
| test_chat_completion_imports.py | ||
| test_check_no_none_string_error_fields.py | ||
| test_check_no_publicly_known_master_key.py | ||
| test_e2e_idp_stack.py | ||
| test_e2e_junit_report.py | ||
| test_e2e_metadata.py | ||
| test_generate_provider_endpoints_support.py | ||
| test_merge_smoke.py | ||
| test_no_hardcoded_secrets.py | ||
| test_provider_cache.py | ||
| test_provider_replay_harness.py | ||
| test_proxy_types_import.py | ||
| test_router_strategy_async.py | ||
| test_unit_passed_gate.py | ||
| test_workflow_job_name_collisions.py | ||
| unbounded_in_baseline.txt | ||