ryan-crabbe-berri 0235dbd7f2 fix(responses): claim a background response before the read that bills it ... Every pod and uvicorn worker schedules its own CheckResponsesCost against the shared LiteLLM_ManagedObjectTable. The poller selected eligible rows, performed the billed retrieval, and only then marked them completed in one bulk write, so two pollers could select the same terminal response and both record a charge before either completion update landed. Each row is now claimed with a compare-and-swap on batch_processed before the read, because the read is what prices the job: aget_responses stamped with the poll origin writes the spend log itself, so there is no later point at which to serialize. A row whose read raised, or whose provider status is still non-terminal, releases its claim so a later cycle retries it rather than retiring it unbilled. That is the failure #37050 fixed on the batch side. A pod that dies between winning the claim and billing would otherwise strand the row: it holds a claim nobody will release and its status never reaches terminal, so every later cycle re-selects it and loses. The updated_at arm of the claim takes such a row back after three poll cycles, and since updated_at is @updatedAt a healthy in-flight claim written moments ago is never stolen. The poller now also persists the finished response onto its managed row instead of writing status alone, so the row carries the generation's usage rather than the stale queued copy stored at create time. Reuses the existing batch_processed column, so no migration. It already sits on the shared table defaulted to false and was unused by response rows. Claude-Session: https://claude.ai/code/session_01Hn5E8Jz1LjGLFyiYxBRcBW		2026-09-15 00:37:54 +00:00
..
example_config_yaml	test: test	2026-03-28 19:17:38 -07:00
test_configs	test: test	2026-03-28 19:17:38 -07:00
test_model_response_typing	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
azure_fine_tune.jsonl	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
batch_job_results_furniture.jsonl	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
conftest.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
data_map.txt	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
eagle.wav	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
gettysburg.wav	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
large_text.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
messages_with_counts.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
model_cost.json	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
openai_batch_completions.jsonl	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
openai_batch_completions_router.jsonl	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
speech_vertex.mp3	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_aproxy_startup.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_audit_logs_proxy.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_auth_checks.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_banned_keyword_list.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_blog_posts_endpoint.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_check_batch_cost.py	Merge pull request #39626 from BerriAI/litellm_batch_ui_logs	2026-09-08 15:19:25 -07:00
test_check_responses_cost.py	fix(responses): claim a background response before the read that bills it	2026-09-15 00:37:54 +00:00
test_custom_callback_input.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_custom_logger_s3_gcs.py	chore(tests): thread config_file_path through s3/gcs custom-logger tests	2026-05-13 01:13:52 +00:00
test_custom_tokenizer_bug.py	fix(proxy): run prompt caching counts and custom tokenizer loads off the event loop	2026-09-07 22:21:45 -07:00
test_db_schema_changes.py	test: initial test to enforce all functions in user_api_key_auth.py h… (#7797)	2025-01-15 21:52:45 -08:00
test_default_end_user_budget_simple.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_deployed_proxy_keygen.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_deprecated_key_grace_period.py	Litellm key rotation bug (#27756)	2026-05-12 17:16:37 -07:00
test_e2e_pod_lock_manager.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_gemini_agents_endpoints.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_get_favicon.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_get_image.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_google_endpoint_routing.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_google_gemini_proxy_request.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_jwt.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_jwt_key_mapping.py	fix(jwt): evict mapping cache after DB write in /jwt/key/mapping update and delete	2026-09-04 14:35:42 -07:00
test_key_generate_dynamodb.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_key_generate_prisma.py	Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ruff_dead_test_code	2026-08-24 09:46:56 -07:00
test_models_fallback_endpoint.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_multipart_bypass_repro.py	fix: harden /key/update authorization checks (#27878)	2026-05-14 04:16:04 +00:00
test_prisma_client_backoff_retry.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prompt_test_endpoint.py	fix: harden /key/update authorization checks (#27878)	2026-05-14 04:16:04 +00:00
test_proxy_config_unit_test.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_custom_auth.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_custom_logger.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_encrypt_decrypt.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_exception_mapping.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_gunicorn.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_proxy_pass_user_config.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_reject_logging.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_routes.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_server.py	perf(proxy): serialize /model/info listing once with orjson	2026-09-14 19:47:51 +00:00
test_proxy_server_keys.py	fix tests	2025-10-25 10:19:24 -07:00
test_proxy_server_spend.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_proxy_setting_guardrails.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_token_counter.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_utils.py	test(newrelic): cover static default_team_settings per-team routing (#38857)	2026-08-31 16:58:35 +00:00
test_realtime_cache.py	feat: litellm oss 110626 (#30202)	2026-06-11 22:30:26 -07:00
test_reducto_ocr_route.py	fix: resolve Python 3.14 OCR annotations and remaining matrix failures	2026-09-02 14:35:38 -07:00
test_request_size_limit_middleware.py	Fix early proxy request size enforcement (#27311)	2026-05-06 12:29:11 -07:00
test_response_polling_handler.py	test(responses): annotate the processor patch for the test quality gate	2026-09-07 12:14:36 +00:00
test_response_polling_pre_call_checks.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_search_api_logging.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_server_root_path.py	fix: server rooth path (#19790)	2026-01-26 09:48:06 -08:00
test_skills_db.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_ui_path_detection.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_unit_test_max_model_budget_limiter.py	fix(proxy): share per-model budget counters across replicas through the spend counter cache (#39375)	2026-09-02 12:40:59 -07:00
test_unit_test_proxy_hooks.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_update_daily_tag_spend.py	perf(spend): write each daily spend batch in one upsert statement (#36448)	2026-08-10 17:06:00 -07:00
test_update_spend.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_user_api_key_auth.py	fix(auth): stop the team fallback from widening model access (#37962)	2026-08-22 14:25:29 -07:00
test_zero_cost_model_budget_bypass.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
vertex_key.json	test: update to new vertex ai keys	2026-03-28 20:19:05 -07:00