ryan-crabbe-berri 0235dbd7f2 fix(responses): claim a background response before the read that bills it ... Every pod and uvicorn worker schedules its own CheckResponsesCost against the shared LiteLLM_ManagedObjectTable. The poller selected eligible rows, performed the billed retrieval, and only then marked them completed in one bulk write, so two pollers could select the same terminal response and both record a charge before either completion update landed. Each row is now claimed with a compare-and-swap on batch_processed before the read, because the read is what prices the job: aget_responses stamped with the poll origin writes the spend log itself, so there is no later point at which to serialize. A row whose read raised, or whose provider status is still non-terminal, releases its claim so a later cycle retries it rather than retiring it unbilled. That is the failure #37050 fixed on the batch side. A pod that dies between winning the claim and billing would otherwise strand the row: it holds a claim nobody will release and its status never reaches terminal, so every later cycle re-selects it and loses. The updated_at arm of the claim takes such a row back after three poll cycles, and since updated_at is @updatedAt a healthy in-flight claim written moments ago is never stolen. The poller now also persists the finished response onto its managed row instead of writing status alone, so the row carries the generation's usage rather than the stale queued copy stored at create time. Reuses the existing batch_processed column, so no migration. It already sits on the shared table defaulted to false and was unused by response rows. Claude-Session: https://claude.ai/code/session_01Hn5E8Jz1LjGLFyiYxBRcBW		2026-09-15 00:37:54 +00:00
..
arize	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
azure_storage	fix(azure): restrict the storage credential chain to deployment identities (#39637)	2026-09-03 18:29:32 -07:00
bitbucket	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
cloudzero	fix(cloudzero): preserve late resource tags (#39873)	2026-09-05 12:10:05 -07:00
code_interpreter_interception	feat(sandbox): reuse e2b container across requests when metadata.session_id is set (#31688)	2026-06-30 18:58:09 -07:00
compression_interception	feat(spend): track prompt compression saved tokens in daily spend aggregates (#33810)	2026-07-18 17:47:54 -07:00
datadog	fix(datadog_llm_obs): keep tool call and result structure under redaction and emit tool output tokens (#40666)	2026-09-11 13:12:14 -07:00
dotprompt	Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_dotprompt_model_swap	2026-08-26 15:13:57 -07:00
focus	fix(spend-tracking): hand plain dict rows to polars in the CloudZero and Focus exports	2026-09-03 18:46:01 -07:00
gcs_bucket	fix(proxy): read a bucket config's include tree off the event loop	2026-09-06 02:05:18 -07:00
gcs_pubsub	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
gitlab	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
helicone	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
langfuse	fix(langfuse): warn and drop invalid LANGFUSE_TRACING_ENVIRONMENT instead of failing requests (#38582)	2026-08-27 18:03:42 -07:00
levo	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
litellm_agent	Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817)	2026-02-21 15:32:47 -08:00
mavvrik_focus	chore: litellm oss staging (#31185)	2026-06-26 09:17:44 -07:00
newrelic	feat(newrelic): export team max and remaining budget gauges to the Metric API (#40542)	2026-09-10 17:16:15 -07:00
open_telemetry	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
opik	Litellm oss staging 030626 (#29578)	2026-06-03 11:01:51 -07:00
otel	fix(responses): bill a background response once via the cost poller, not on every read	2026-09-15 00:37:54 +00:00
pointfive	feat(pointfive): add batching logger callback	2026-09-10 14:02:35 +03:00
SlackAlerting	fix(proxy): emit internal user budget alerts	2026-09-09 18:44:24 +08:00
vector_store_integrations	fix(vector-stores): report search failures on the responses API surface too	2026-09-03 01:54:09 -07:00
websearch_interception	revert: restore search tool fallback when no router is configured	2026-09-01 11:16:23 -07:00
rubrik_test_helpers.py	Litellm oss staging 04 21 2026 2 (#26569)	2026-05-20 21:25:19 -07:00
test_agentops.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_anthropic_cache_control_hook.py	fix: skip one-shot Claude Code cache injection (#40175)	2026-09-07 18:03:43 -07:00
test_athina.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_azure_sentinel.py	test(azure_sentinel): pin batch_size as a per-request bound under concurrent events (#40320)	2026-09-08 16:55:40 -07:00
test_braintrust_logging.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_braintrust_span_name.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_custom_guardrail.py	test(guardrails): type the native lifecycle logging_only test double	2026-09-13 08:47:47 +00:00
test_custom_guardrail_recursion.py	fix: recursive pydantic issue (#19531)	2026-01-22 19:56:41 -08:00
test_custom_prompt_management.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_deepeval.py	fix: use fastuuid helper (#14903)	2025-09-25 15:47:01 -07:00
test_galileo.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_guardrail_logging_sync.py	fix(guardrails): keep guardrail information in spend logs when the caller sends its own metadata	2026-07-24 16:20:44 -07:00
test_helicone.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_langfuse.py	fix(langfuse): give each call in a session header its own trace instead of upserting one trace per session (#40177)	2026-09-09 15:56:38 -07:00
test_langfuse_otel.py	feat(langfuse): support langfuse_environment as a per-key dynamic callback param (#38264)	2026-08-26 16:56:55 -07:00
test_langsmith_init.py	test: assert LangSmith periodic flush by observing a batch send	2026-09-05 10:36:15 +00:00
test_lunary.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_mlflow.py	fix(mlflow): prevent _stream_id_to_span leak and mlflow 2.x end_trace TypeError (#39049)	2026-09-08 16:18:06 -07:00
test_openmeter.py	test: unwind environment writes in tests/test_litellm with monkeypatch (#37806)	2026-08-21 20:28:37 -07:00
test_opentelemetry.py	fix(responses): bill a background response once via the cost poller, not on every read	2026-09-15 00:37:54 +00:00
test_opentelemetry_dynamic_imports.py	Make grpc dependency optional (#19447)	2026-01-20 19:03:52 -08:00
test_opik_utils.py	chore(ci): merge oss branch (#33784)	2026-07-17 23:22:13 +00:00
test_otel_guardrail_violation_spans.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_otel_team_attributes_matrix.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_api_promql_escape.py	fix(prometheus): quote api_key for PromQL string literal in spend lookup	2026-05-01 21:38:19 +00:00
test_prometheus_budget_metric_guard.py	fix(prometheus): skip budget metric DB lookups when gauges are NoOpMetric (#32834)	2026-07-10 20:25:47 -07:00
test_prometheus_budget_metrics_db_lookups.py	perf(auth): negative-cache missing user/key lookups on the request hot path (#32368)	2026-07-08 09:59:57 +03:00
test_prometheus_budget_metrics_timeout.py	feat: litellm oss staging (#31935)	2026-07-03 09:27:31 +05:30
test_prometheus_cache_metrics.py	fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803)	2026-07-27 12:28:19 -07:00
test_prometheus_caller_identity.py	test(prometheus): cover caller-identity config failure cases (#38380)	2026-08-26 11:53:49 -07:00
test_prometheus_carried_budget_state.py	perf(proxy): one MGET and one pipeline for post-call spend counters, no team/user/org refetch on the response path (#40841)	2026-09-12 16:05:50 +00:00
test_prometheus_client_ip_user_agent.py	feat(prometheus): bucket latency by input sequence length (#40059)	2026-09-10 00:29:28 -07:00
test_prometheus_custom_metadata_label_counts.py	Fix Prometheus remaining metric zero values (#27348)	2026-05-06 17:22:20 -07:00
test_prometheus_end_user_cardinality.py	perf: cap Prometheus end-user metric cardinality with TTL + LRU eviction (#27272)	2026-05-06 13:35:13 -07:00
test_prometheus_input_sequence_length_label.py	feat(prometheus): bucket latency by input sequence length (#40059)	2026-09-10 00:29:28 -07:00
test_prometheus_invalid_key_filtering.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_labels.py	fix(prometheus): label pre-call rate limit failures with the resolved api_provider	2026-09-14 19:50:21 +00:00
test_prometheus_mcp_tool_metrics.py	feat(prometheus): expose MCP tool metadata in Prometheus metrics (#31899)	2026-07-02 10:56:35 +03:00
test_prometheus_media_generation_metrics.py	feat(prometheus): expose video duration and image count consumption metrics (#33138)	2026-07-13 18:51:13 -07:00
test_prometheus_metric_name_consistency.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_prometheus_metrics_endpoint.py	perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes (#37702)	2026-08-20 16:08:22 -07:00
test_prometheus_missing_metrics.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_prometheus_none_metadata.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_overhead_with_guardrails.py	feat(prometheus): add litellm_total_overhead_latency_metric (SDK overhead + guardrails) (#31593)	2026-06-30 17:34:17 +08:00
test_prometheus_queue_guardrail_metrics.py	fix(prometheus): fold auth/pre-call time into litellm_request_total_latency_metric (#37958)	2026-08-22 14:25:55 -07:00
test_prometheus_rate_limit_labels.py	feat(prometheus): expose per-key and per-team rate limit allowed and used gauges (#39236)	2026-09-01 18:03:18 -07:00
test_prometheus_remaining_tokens_router_fallback.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_requested_model_cardinality.py	fix(prometheus): pass through router-originated labels when no proxy router exists	2026-09-01 12:14:27 -07:00
test_prometheus_service_tier_label.py	feat(otel): stamp service tier attributes on inference spans (#35679)	2026-08-03 23:10:01 -07:00
test_prometheus_services.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_spend_logs_metadata.py	fix(prometheus): expose project_alias in custom metadata labels (LIT-3741) (#31784)	2026-07-01 10:44:02 +08:00
test_prometheus_stream_label.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_prometheus_token_detail_metrics.py	fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803)	2026-07-27 12:28:19 -07:00
test_prometheus_user_team_metrics.py	fix(proxy): count only active users toward license seat limit (#31227)	2026-06-29 18:01:02 -07:00
test_prompt_manager_ssti.py	fix(security): sandbox jinja2 in gitlab/arize/bitbucket prompt managers	2026-05-02 09:14:02 +00:00
test_responses_background_cost.py	fix(responses): claim a background response before the read that bills it	2026-09-15 00:37:54 +00:00
test_rubrik.py	fix(guardrails/rubrik): attribute blocked requests to the caller that made them (#35734)	2026-08-03 19:56:24 -07:00
test_s3.py	fix(s3): bound s3 object keys and download filenames for long Responses API ids (#39164)	2026-09-01 13:30:02 -07:00
test_s3_v2.py	fix(s3_v2): freeze refreshable credentials before signing and retry 403 uploads with a fresh signature (#40187)	2026-09-09 23:14:16 -07:00
test_shadow_eval_logger.py	fix(shadow-eval): skip hosted web search samples (#40827)	2026-09-12 02:11:48 +00:00
test_weave_otel.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00