scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-13 08:52:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 174
litellm/tests/integration/spend
History
Exact
tavren27 1c68ec429a
fix(proxy): bill an OpenAI pass-through background response once across repeated reads (#50841) 
* fix(proxy): bill an OpenAI pass-through background response once across repeated reads

A finished background response now bills only on the first pass-through call
that sees it finished with usage. The claim is an insert on the new
LiteLLM_BackgroundResponseSettlement table, keyed by response id, so it holds
across workers and replicas. Every later retrieve, cancel, streamed create, or
resumed stream logs like an unbilled read. With no database or a failed insert
the call still bills.

* fix(proxy): keep pass-through background billing fail-open without prisma and skip foreground stream re-parse

* fix(proxy): claim background responses on failed streams and bound the claim

A resumed stream that fails after response.completed billed its usage through
the partial-usage path without claiming the response id, so a later read billed
it again. That path now claims the id and logs $0 when an earlier read paid.

The claim uses create_many(skip_duplicates=True) so a repeat read no longer
raises a unique violation in Postgres, and it waits at most 2 s before billing,
since the claim now runs before the client gets its response.

* fix(pass_through): keep the background response claim and its spend write together under cancellation and managed ids

* test(pass_through): cover a failed stream whose failure logging cannot be scheduled

* fix(proxy): start the background response settlement migration with a lock timeout

* fix(passthrough): leave a failed background stream's charge to its next read

A stream that failed after response.completed claimed the response id while its failure row billed $0, so no later read paid. The failure path is back to main's behavior, so the next finished read claims and bills once

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-10 07:33:25 -08:00
..
fixtures test(usage): assert per-user entity spend and metadata (#50251) 2026-11-05 20:16:40 +00:00
golden fix(usage): restore per-user spend breakdown on the user aggregated route (#49464) 2026-11-05 01:57:23 -08:00
__init__.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
_background_responses.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
_daily_activity_fixtures.py feat(usage): add top users by spend and user activity views (#45310) 2026-10-10 12:21:43 -07:00
_partial_stream.py fix(proxy): raise live budget counters for failure spend rows (#47741) 2026-10-30 13:05:12 -07:00
test_agent_spend.py fix(a2a)!: bill completion-bridge agents at their own prices on /a2a send and stream (#49150) 2026-10-30 19:43:45 -07:00
test_autorouter_session_rollup.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
test_background_interaction_settlement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_background_response_billed_once.py fix(responses): resume background response streams with stream and starting_after (#48819) 2026-10-29 02:31:22 -07:00
test_background_response_billing_proxies.py fix(proxy): report requested model in streamed Responses response.model (#46662) 2026-10-17 19:52:10 +00:00
test_background_response_poll_retirement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_baggage_tag_mappings.py chore(proxy): use neutral example names for baggage tag mappings (#50011) 2026-11-04 14:49:55 +00:00
test_baseline_accounting.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
test_batch_completion_accounting.py fix(logging): deliver completed batch success callbacks consistently (#47192) 2026-10-18 09:19:04 -07:00
test_batch_enqueued_tokens_redis_lua.py test(proxy): migrate DB and Redis backed proxy tests into tests/integration (#43996) 2026-10-01 09:23:36 -07:00
test_batch_observability.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_batch_poll_starvation.py fix(batches): stop unbillable batches from blocking file deletes and new-batch polling (#48752) 2026-10-28 12:19:36 -07:00
test_budget_limit_envelopes.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_cache_activity.py fix(proxy): scope /global/activity/cache_hits analytics to the caller's own and permitted-team logs (#46210) 2026-10-14 10:01:21 +02:00
test_cache_and_quota.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_chaos_burst_spend_once.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_chatgpt_subscription_usage.py feat(usage): show subscription-covered providers and account fees separately (#44772) 2026-10-30 03:58:36 -07:00
test_container_spend.py test(integration): cover videos_containers endpoint contracts (#44691) 2026-10-14 03:49:01 +00:00
test_customer_spend_by_team.py feat(usage): split customer spend by the key's current team (#50764) 2026-11-09 19:35:31 -08:00
test_daily_activity_aggregated_breakdowns.py fix!: retain unattributed daily activity across reports and exports (#48925) 2026-10-29 09:39:49 -07:00
test_daily_activity_key_alias_probes.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_daily_activity_key_metadata_query_timeout.py feat(ui): unify gateway and usage assistance as Aria Agent (#48082) 2026-10-23 20:24:19 +00:00
test_daily_activity_key_owner.py test(usage): assert per-user entity spend and metadata (#50251) 2026-11-05 20:16:40 +00:00
test_daily_activity_key_owner_faults.py test(usage): assert per-user entity spend and metadata (#50251) 2026-11-05 20:16:40 +00:00
test_daily_activity_key_owner_traffic.py feat(ui): unify gateway and usage assistance as Aria Agent (#48082) 2026-10-23 20:24:19 +00:00
test_daily_activity_repository.py feat(usage): add top users by spend and user activity views (#45310) 2026-10-10 12:21:43 -07:00
test_daily_activity_routes.py fix!: retain unattributed daily activity across reports and exports (#48925) 2026-10-29 09:39:49 -07:00
test_daily_rollup_retry.py fix(cost): price exact cost-map names before price-free capability rules, and fix SCIM and MCP integration tests on CircleCI (#47037) 2026-10-17 06:46:52 +00:00
test_daily_tag_spend_retention.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_disable_end_user_cost_tracking.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_disconnected_bedrock_messages_stream_billing.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_end_user_budget_enforcement.py fix(anthropic_endpoints)!: return Anthropic error envelopes on /v1/messages auth failures (#48466) 2026-11-03 01:53:29 +00:00
test_end_user_spend_without_proxy_user.py fix(anthropic_endpoints)!: return Anthropic error envelopes on /v1/messages auth failures (#48466) 2026-11-03 01:53:29 +00:00
test_expired_ui_session_failure_rows.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_fail_closed_team_member_budget.py fix(proxy): fail-closed budget enforcement returns 403 naming /team/member_add for a team user with no membership row (#50603) 2026-11-09 15:38:59 -08:00
test_failed_dispatch_tokens.py test(spend): audit #47815 (#48774) 2026-10-28 03:04:11 -07:00
test_failing_request_stored_snapshot.py test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984) 2026-11-04 09:55:53 -08:00
test_failure_spend_live_counters_chaos.py fix(proxy): raise live budget counters for failure spend rows (#47741) 2026-10-30 13:05:12 -07:00
test_filtered_ledger.py test: add extension and browser integration contracts 2026-09-16 13:15:53 -07:00
test_gemini_tts_speech_billing.py test(integration): repair seven stale integration tests on main (#47600) 2026-10-20 11:21:04 -07:00
test_global_proxy_spend_counter.py test(proxy): audit #46997 (#48918) 2026-10-28 18:05:03 -07:00
test_global_spend_report.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_global_spend_reset.py fix(proxy)!: stop answering with the literal string "None" as error type and param on the proxy-server, auth, spend, and key health routes (#39540) 2026-10-29 18:20:52 -07:00
test_google_genai_native_spend.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_image_generation_key_spend.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_key_budget_lockout.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_key_metadata_recovery_dropped_connection.py fix(spend-tracking): stop caching failed spend-log metadata lookups as confirmed misses (#43560) 2026-10-05 22:36:25 +00:00
test_key_metadata_recovery_probe_bounds.py fix(proxy-extras): build the SpendLogs indexes in the migration job instead of in migrations (#43948) 2026-10-01 14:12:22 -07:00
test_legacy_spend_logs_row_cap.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_messages_stream_usage_cost.py feat(proxy): per-request header opts a stream into usage.cost without the global include_cost_in_streaming_usage flag (#50574) 2026-11-07 16:57:20 -08:00
test_model_router_selected_model.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_non_finite_cost_spend_writes.py fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829) 2026-10-30 08:27:35 -07:00
test_normalized_error_long_message.py test(integration): give each kill-burst its own prompts so cached control replies cannot stand in for late traffic (#50569) 2026-11-07 23:08:23 +00:00
test_openrouter_upstream_provider.py feat(openrouter): record upstream provider in logging payload and spend logs (#49287) 2026-10-30 12:51:48 -07:00
test_org_budget_cli_session_token.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_partial_stream_live_budget.py test(spend): assert Anthropic envelope for Messages budget refusals (#49800) 2026-11-03 06:10:08 +00:00
test_passthrough_budget_reservation.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_passthrough_failure.py fix(proxy): write SpendLogs failure rows for config-defined pass-through endpoints (#41831) 2026-11-09 17:05:10 -08:00
test_passthrough_request_tags.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_passthrough_stream_pricing_paths_spend.py fix(proxy): bill an OpenAI pass-through background response once across repeated reads (#50841) 2026-11-10 07:33:25 -08:00
test_passthrough_upstream_model.py feat(passthrough): let upstreams report the model name via x-litellm-upstream-model (#48275) 2026-10-27 18:28:54 -07:00
test_prompt_caching_requests_pagination.py test(proxy): migrate DB and Redis backed proxy tests into tests/integration (#43996) 2026-10-01 09:23:36 -07:00
test_ptu_share_attribution.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_redis_ttl_preserving_token_increment.py test(spend): exercise TTL-preserving increments through rate limit store (#50418) 2026-11-06 17:53:25 +00:00
test_requester_ip_address_logging.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_reset_budget_leader_election.py fix: make scheduled key, user, and team spend resets idempotent (#50298) 2026-11-09 22:21:43 -08:00
test_responses_bridge_spend_once.py fix(logging): keep the caller's call_type when chat completions are bridged to the Responses API (#49710) 2026-11-03 10:02:54 +00:00
test_responses_cache_write_itemization.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_responses_stream_spend_row_id.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
test_responses_websocket_turn_rows.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_roi_branch_spend.py fix(caching): key response cache by router model group in litellm_metadata (#44542) 2026-10-05 17:06:21 +00:00
test_router_cooldown_failure_row.py test(integration): read router cooldown failure rows from the proxy that served them (#47457) 2026-10-19 18:39:16 -07:00
test_scoped_spend_reports.py test(integration): cover budgets_spend endpoint contracts (#44736) 2026-10-14 06:58:22 +00:00
test_service_tier_stream_billing.py fix(streaming): keep the served service_tier on streamed chunks and spend rows (#42870) 2026-09-29 12:54:17 -07:00
test_session_grouped_logs_pages.py test(spend): run explained reads within real read transactions (#49996) 2026-11-04 05:11:23 +00:00
test_session_total_spend.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_shutdown_flush.py test: update the spend tests that still expected the pre-ledger accounting queues (#50814) 2026-11-09 22:07:42 -08:00
test_spend_attribution_contracts.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_spend_calculate.py fix(pricing): drop the unpublished cached rate from the Gemini Live preview entries (#42651) 2026-09-22 20:35:34 -07:00
test_spend_capture_rate_captured_spend.py test(proxy): migrate DB and Redis backed proxy tests into tests/integration (#43996) 2026-10-01 09:23:36 -07:00
test_spend_log_caller_session.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_log_latest_turn.py feat(spend-logs): opt-in latest_turn mode that stores only the current turn of stateless requests (#49544) 2026-11-05 19:41:15 -08:00
test_spend_log_read_scope.py test(integration): cover budgets_spend endpoint contracts (#44736) 2026-10-14 06:58:22 +00:00
test_spend_log_tool_payload_content.py fix(streaming)!: send the include_usage chunk with an empty choices array (#47109) 2026-10-21 02:26:05 +00:00
test_spend_log_write_batching.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_logs_metadata_fields.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_rollup_accuracy.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_stream_alias_billing.py fix(proxy): stamp the client alias on a copy of each streamed chunk so pricing sees the deployment model (#44341) 2026-10-03 06:44:52 +00:00
test_streamed_usage_estimate.py test(spend): audit #47353 (#48785) 2026-10-28 03:50:50 -07:00
test_tag_budget_enforcement.py feat(proxy): map baggage values to budget tags (#47358) 2026-10-20 09:09:42 +00:00
test_tag_spend_reset.py feat(tags): reset a tag's spend through /tag/update (#50323) 2026-11-06 10:41:22 -08:00
test_team_budget_enforcement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_team_daily_activity_aggregated.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_team_member_budget_alerts.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_team_member_spend.py test: update the spend tests that still expected the pre-ledger accounting queues (#50814) 2026-11-09 22:07:42 -08:00
test_team_member_spend_flush.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_team_member_total_spend_reconciliation.py fix(proxy): reconcile team member total_spend with current cycle spend (#48260) 2026-10-24 17:57:09 -07:00
test_text_completion_stream_usage_cost.py fix(proxy)!: bill streamed /v1/completions without stream_options like chat (#45366) 2026-10-29 17:18:51 -07:00
test_unmanaged_vertex_batch_deployment_routing_wire.py fix(batches): cost an unmanaged gs:// batch through the deployment that created it (#49301) 2026-10-30 15:25:47 -07:00
test_user_budget_on_team_keys.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_vertex_image_regional_uplift.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_vertex_passthrough_regional_uplift_spend.py fix(tests): read owned proxy rows from the owned proxy's database (#47333) 2026-10-20 10:07:06 -07:00
test_vertex_regional_uplift_spend.py fix(vertex_ai): add regional endpoint uplift to gemini-3.1-flash-image (#44673) 2026-10-05 21:00:22 -07:00