tessellark 8ab0321816 fix(proxy): charge a video's provider-reported cost once, on the first poll that sees it completed (#49824) ... * fix(proxy): charge a video's provider-reported cost once, on the first poll that sees it completed A custom video provider can report a video's cost on the status poll that sees it completed. Every poll of a video shared the video id as its spend log request_id, so the paid poll's row was dropped as a duplicate and every later poll of the finished video charged the cost again. Video status rows now carry their own `_video_cost` request_id, and a poll that reports a cost goes through the same settle-once claim batches use, so the first poll that sees the video completed charges it and writes the row, and every later poll charges nothing. * fix(proxy): charge a video's provider-reported cost once on model, router and session budgets too A video status poll gets the _video_cost row suffix only when it reports a cost, so a $0 poll keeps the bare video id and adds no row of its own, and the spend log writer claims a row by that suffix. The batch charge-once marker now covers videos, keyed on the video id under {spend_key}:video:{id}, with the batch marker key unchanged. Model and router budgets use it, and per-session budgets now charge a repolled batch or video once and ignore a cost that is not a finite number * fix(proxy): charge a finished video's own cost once, keyed by the id the client polled Guardrail cost on a running poll no longer takes the charge-once claim, only a completed poll charges, the session marker TTL refreshes like the others, and two deployments whose provider ids match each keep their own charge. * fix(proxy): charge repeat poll guardrails, key video charges by canonical id and survive redaction * fix(proxy): keep video status polls in-process for the collector and give guardrail-only job polls an unclaimable row id * fix(proxy): keep repeat batch polls free and group guardrail-only batch rows under their batch's trace * fix(proxy): settle a finished job whose own cost is below its poll's guardrail cost under the job's charge row * test(proxy): count charged_poll_cost as covered through the finished job guardrail poll tests * fix(proxy): settle a finished batch that cost nothing itself on its first priced retrieve * fix(proxy): charge no budget for a batch retrieve before its cost is final, and read the retrieved batch from before redaction * fix(proxy): charge sessions a poll's guardrail-inclusive cost, count a job cost row's guardrail run, and keep batch_cost_is_final importable from litellm.batches * fix(proxy): claim a job and charge the session budget in one Redis call The session limiter claimed the job marker and then raised spend in a second Redis call. A failure between the two left the job claimed while its charge sat only in local memory, so later polls skipped it. One script now does both, and a refused script claims and charges locally --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>		2026-11-04 20:22:06 +00:00
..
audio_utils	feat(realtime)!: support latest OpenAI audio models (#35600)	2026-11-03 07:27:52 +00:00
llm_cost_calc	feat: update EmpirioLabs models in model_prices_and_context_window.json (#49980)	2026-11-04 20:11:52 +00:00
llm_response_utils	feat(hosted_vllm)!: keep vLLM's stop_reason and token_ids on chat choices and stream chunks (#48393)	2026-11-02 13:26:29 -08:00
prompt_templates	fix(rate-limit): estimate embedding image and audio data URIs as media in TPM pre-call checks (#49831)	2026-11-03 22:06:59 +00:00
specialty_caches	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
__init__.py	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
attribute_forwarder.py	refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789)	2026-10-24 11:31:09 -07:00
conftest.py	test(ci): repair stale tests and flaky CI infrastructure (#43983)	2026-10-01 17:46:43 +00:00
event_loop_lag.py	test: repair stale and polluting tests red on scheduled main CI (#44229)	2026-10-02 21:32:17 +00:00
fake_secret_vault.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
messages_with_counts.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_agentic_followup_kwargs.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_allowed_tools_tool_choice.py	fix(tool_choice): accept OpenAI allowed_tools on chat completions and flatten it for the Responses bridge (#49043)	2026-11-02 18:51:58 -08:00
test_anthropic_dedup_factory.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_api_route_to_call_types.py	fix(proxy): serve /a2a/{agent_id}/message/stream routes (#49471)	2026-10-31 16:11:51 -07:00
test_app_crypto.py	test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554)	2026-10-09 09:48:10 -07:00
test_audio_utils.py	fix(proxy): account for media in combined TPM reservations (#48865)	2026-10-28 12:39:41 -07:00
test_aws_partition.py	perf(ci): isolate and shard integration suites (#44146)	2026-10-10 11:15:19 -07:00
test_batch_utils.py	fix(proxy): charge a video's provider-reported cost once, on the first poll that sees it completed (#49824)	2026-11-04 20:22:06 +00:00
test_bedrock_converse_dedup_factory.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_bug_report.py	fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344)	2026-10-16 00:53:52 +00:00
test_chat_completion_agentic_loop.py	fix(agentic)!: surface a failed agentic follow-up call instead of the pre-loop tool call (#49227)	2026-11-04 02:00:33 +00:00
test_classifier_logging.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_cli_token_utils.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_cloud_storage_security.py	fix(proxy)!: let proxy admin keys read raw cloud storage file ids on GET /files/{id}/content (#40621)	2026-10-20 10:58:42 +00:00
test_codestral_provider_routing.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_core_helpers.py	fix(cohere): emit streamed tool calls from v2 tool-call-start and tool-call-delta chunks (#47794)	2026-10-31 21:14:53 +00:00
test_coroutine_checker.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_cost_discount.py	feat(cost): support provider/model-pattern keys in cost_discount_config (#43335)	2026-10-12 14:59:18 -07:00
test_custom_logger_registry.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_dd_tracing.py	refactor: expose core private helpers under public names (#44871)	2026-10-07 00:05:27 +00:00
test_decisions_pricing.py	feat(decisions): add Parallel AI web-grounded decisions (#49188)	2026-11-03 04:47:02 +00:00
test_decode_special_tokens.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_default_encoding.py	test: move 76 live top-level tests to offline unit and integration coverage (#45352)	2026-10-08 10:33:35 -07:00
test_dot_notation_indexing.py	refactor(typing): widen delete_nested_value to Mapping and drop two unused Span aliases (#48001)	2026-10-24 19:39:27 +00:00
test_duration_parser.py	feat(proxy): report when each per-model budget window resets (#45797)	2026-10-15 10:56:45 -07:00
test_env_utils.py	fix(router): cap the default OTPM reservation and reconcile token counters after each call (#49054)	2026-10-29 20:32:04 -07:00
test_error_normalization.py	fix(anthropic): answer an upstream Anthropic 529 as 529 overloaded_error on /v1/messages (#46063)	2026-11-03 17:37:09 -08:00
test_exception_mapping_utils.py	fix(anthropic): answer an upstream Anthropic 529 as 529 overloaded_error on /v1/messages (#46063)	2026-11-03 17:37:09 -08:00
test_extract_base64_image.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_fallback_generalizations.py	refactor(fallback-generalizations)!: remove the legacy extends and mixed-rule shim (#48737)	2026-10-28 05:17:28 -07:00
test_fallback_utils.py	fix(cache_control): pick the chat-path dialect from the deployment's provider prefix (#45295)	2026-10-30 05:11:42 -07:00
test_get_litellm_params.py	feat(openai): bill gov.api.openai.com FedRAMP endpoint with 1.1x uplift (#49951)	2026-11-04 08:57:42 -08:00
test_get_llm_provider_endpoint_match.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_get_llm_provider_logic.py	fix(proxy): let typesafe/* allowlists cover /typesafe Jev calls and list Jev in the model picker (#47697)	2026-10-28 20:19:58 -07:00
test_get_model_cost_map.py	feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096)	2026-10-23 16:54:19 -07:00
test_get_supported_openai_params.py	fix(bedrock): backfill published regional service tier prices (#49113)	2026-10-29 19:46:57 -07:00
test_health_check_helpers.py	fix(health): probe only models a partial wildcard serves (#49060)	2026-10-29 23:34:57 -07:00
test_hidden_params.py	perf(streaming): stop raising caught AttributeErrors inside pydantic on every streamed chunk (#49166)	2026-10-30 10:01:44 -07:00
test_image_handling.py	fix(anthropic): send an https PDF image_url to Claude as a document (#49708)	2026-11-03 07:05:15 +00:00
test_initialize_dynamic_callback_params.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_internal_call_metadata.py	fix(responses): resume background response streams with stream and starting_after (#48819)	2026-10-29 02:31:22 -07:00
test_json_fragment_accumulator.py	test: deflake the json accumulator drain test and the armed routing prefetch test (#49505)	2026-10-31 15:14:33 -07:00
test_json_schema_validation.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_litellm_logging.py	fix(proxy): charge a video's provider-reported cost once, on the first poll that sees it completed (#49824)	2026-11-04 20:22:06 +00:00
test_llm_judge.py	fix(guardrails): resolve decision models in caller team scope (#46085)	2026-10-15 11:43:36 -07:00
test_llm_request_utils.py	fix(aiand)!: show ai&'s name and logo, and sync ai&'s model list (#49158)	2026-10-30 10:55:26 -07:00
test_logging_callback_manager.py	fix(router): unregister strategy selectors from global callbacks on discard (#45795)	2026-10-21 02:39:36 +00:00
test_logging_utils.py	perf(logging): skip the base64 regex on plain text and copy a few large strings inline (#49228)	2026-10-31 01:03:04 -07:00
test_logging_worker.py	fix(proxy): persist tag spend and drain success callbacks within the shutdown budget (#48988)	2026-10-29 09:32:18 -07:00
test_max_streaming_duration.py	fix(router): log each dropped /v1/messages attempt through its own logging copy and open the retry without its usage (#45194)	2026-10-20 21:46:45 -07:00
test_model_param_helper.py	feat(realtime)!: support latest OpenAI audio models (#35600)	2026-11-03 07:27:52 +00:00
test_model_response_utils.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_moderation_standard_logging.py	test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362)	2026-10-08 15:55:57 -07:00
test_oauth_token_exchange.py	feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158)	2026-10-09 15:19:07 -07:00
test_offline_mode.py	feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096)	2026-10-23 16:54:19 -07:00
test_optional_imports.py	refactor(sdk): separate core AWS and tokenizer dependencies (#44447)	2026-10-08 19:07:57 -07:00
test_path_utils.py	refactor(core): move path_utils from proxy common_utils to litellm_core_utils (#45775)	2026-10-10 19:07:22 +00:00
test_private_json.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_provider_affinity.py	fix(headroom): forward tenant-scoped session affinity (#49255)	2026-10-30 12:32:03 -07:00
test_provider_specific_headers.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_ptu_pricing.py	feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043)	2026-10-15 18:44:30 +00:00
test_realtime_errors.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_realtime_streaming.py	feat(realtime)!: support latest OpenAI audio models (#35600)	2026-11-03 07:27:52 +00:00
test_redact_messages.py	fix(logging): don't leak redacted reasoning via provider_specific_fields (#47398)	2026-11-04 18:31:09 +00:00
test_request_timeout_resolver.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_retry_after_headers.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_safe_divide_seconds.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_safe_json_dumps.py	fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829)	2026-10-30 08:27:35 -07:00
test_sensitive_data_masker.py	refactor: expose core private helpers under public names (#44871)	2026-10-07 00:05:27 +00:00
test_sentry_scrubbing.py	fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344)	2026-10-16 00:53:52 +00:00
test_served_output_texts.py	fix(guardrails): scan and rewrite streamed /v1/completions under post_call guardrails (#49035)	2026-10-29 08:46:49 -07:00
test_stream_usage_logging.py	test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362)	2026-10-08 15:55:57 -07:00
test_streaming_chunk_builder_cursor.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_server_tool_use.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_utils.py	feat(hosted_vllm)!: keep vLLM's stop_reason and token_ids on chat choices and stream chunks (#48393)	2026-11-02 13:26:29 -08:00
test_streaming_handler.py	feat(hosted_vllm)!: keep vLLM's stop_reason and token_ids on chat choices and stream chunks (#48393)	2026-11-02 13:26:29 -08:00
test_streaming_overhead.py	refactor: expose core private helpers under public names (#44871)	2026-10-07 00:05:27 +00:00
test_text_completion_conversion.py	refactor(types): replace Any with proven types in 14 files (#48928)	2026-10-29 07:26:07 -07:00
test_thread_pool_executor.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_token_counter.py	chore(cost-map): remove the 52 rows whose provider retired them before 2026 (#49974)	2026-11-03 16:23:06 -08:00
test_token_counter_tool.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_token_counter_tool_data.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_tokenizer.py	fix(token_counter)!: keep a loaded Hugging Face tokenizer when a slower download fails (#48916)	2026-10-29 03:50:58 -07:00
test_tool_fields.py	fix(mcp)!: answer 400 for a hosted MCP tool a translated route cannot run, drop it under drop_params (#48931)	2026-11-02 13:13:03 -08:00
test_tool_search_spend_logging.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_url_utils.py	fix(llms): join api_base and route with one shared helper (#47535)	2026-10-21 06:59:32 +00:00
test_xai_oauth_routing.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00