scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-28 06:23:07 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 94
litellm/tests/integration/spend
History
devin-ai-integration[bot] b55024658c
fix(proxy)!: stop update_cache from resurrecting evicted auth cache entries (#46997) 
* fix(proxy): stop update_cache from resurrecting evicted auth cache entries

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): update update_cache pins for synchronous write-back

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): wrap long assert in update_cache tag test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover refill race and pin TTL with injected clock

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): count global proxy spend for every request with a user, cached on this worker or not

* fix(proxy): add global proxy spend with an atomic Redis increment that never re-creates a reset counter

The global proxy spend bump read the total, awaited, and wrote old + cost back
from a background task, so a budget reset that deleted the key in between got
the pre-reset total written back, and workers overwrote each other's totals.

RedisCache.async_increment_if_present runs INCRBYFLOAT only when the key
exists, keeping its TTL, and this worker's copy is then set from the shared
total with no await between its read and write. The bump now runs whenever a
proxy max_budget is set, matching the DB writer, and the armed post-call read
and the batch-read declaration it used are removed.

* fix(proxy): report the global spend Redis increment to service metrics and request traces

The global proxy spend increment now emits Redis service success and
failure events under the spend_counters target and passes the request
span through, so it shows as redis.incr spend_counters like the write
it replaced.

The unit tests run the real RedisCache increment against fakeredis
instead of mocks, and a new test checks the span lands under the
request span on success and on a Redis failure.

* test(proxy): keep the spend counter Redis fake off the network

* test(proxy): count increment_if_present as a shared-cache call in the key-family guard

* fix(proxy): keep the newer global spend total when Redis replies arrive out of order

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-20 17:09:53 +00:00
..
fixtures feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043) 2026-10-15 18:44:30 +00:00
golden feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043) 2026-10-15 18:44:30 +00:00
__init__.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
_background_responses.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
_daily_activity_fixtures.py feat(usage): add top users by spend and user activity views (#45310) 2026-10-10 12:21:43 -07:00
test_agent_spend.py test(integration): cover agents_a2a endpoint contracts (#44650) 2026-10-14 03:40:07 +00:00
test_autorouter_session_rollup.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
test_background_interaction_settlement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_background_response_billed_once.py fix(responses): count billed background responses toward the creator's project and per-model budgets (#47504) 2026-10-20 12:59:37 +00:00
test_background_response_billing_proxies.py fix(proxy): report requested model in streamed Responses response.model (#46662) 2026-10-17 19:52:10 +00:00
test_background_response_poll_retirement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_baggage_tag_mappings.py feat(proxy): map baggage values to budget tags (#47358) 2026-10-20 09:09:42 +00:00
test_baseline_accounting.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
test_batch_completion_accounting.py fix(logging): deliver completed batch success callbacks consistently (#47192) 2026-10-18 09:19:04 -07:00
test_batch_enqueued_tokens_redis_lua.py test(proxy): migrate DB and Redis backed proxy tests into tests/integration (#43996) 2026-10-01 09:23:36 -07:00
test_batch_observability.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_batch_poll_starvation.py fix(batch): preserve recoverable costs and prevent poll starvation (#47190) 2026-10-20 08:22:19 -07:00
test_budget_limit_envelopes.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_cache_activity.py fix(proxy): scope /global/activity/cache_hits analytics to the caller's own and permitted-team logs (#46210) 2026-10-14 10:01:21 +02:00
test_cache_and_quota.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_chaos_burst_spend_once.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_container_spend.py test(integration): cover videos_containers endpoint contracts (#44691) 2026-10-14 03:49:01 +00:00
test_daily_activity_aggregated_breakdowns.py refactor(repositories): daily activity repository with centralized bounded usage queries (#43398) 2026-10-01 16:02:01 -07:00
test_daily_activity_key_alias_probes.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_daily_activity_key_metadata_query_timeout.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_daily_activity_key_owner.py fix(proxy): recover session key owners from daily spend for usage attribution (#43642) 2026-09-29 15:36:09 -07:00
test_daily_activity_key_owner_faults.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_daily_activity_key_owner_traffic.py fix(proxy): look up hashed key names with two spend log rows per key (#43656) 2026-09-29 20:21:55 -07:00
test_daily_activity_repository.py feat(usage): add top users by spend and user activity views (#45310) 2026-10-10 12:21:43 -07:00
test_daily_activity_routes.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_daily_rollup_retry.py fix(cost): price exact cost-map names before price-free capability rules, and fix SCIM and MCP integration tests on CircleCI (#47037) 2026-10-17 06:46:52 +00:00
test_daily_tag_spend_retention.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_disable_end_user_cost_tracking.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_disconnected_bedrock_messages_stream_billing.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_end_user_budget_enforcement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_end_user_spend_without_proxy_user.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_expired_ui_session_failure_rows.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_failed_dispatch_tokens.py fix(token_counter): count DeepSeek V4 text with DeepSeek's own tokenizer (#47353) 2026-10-20 03:43:35 +00:00
test_filtered_ledger.py test: add extension and browser integration contracts 2026-09-16 13:15:53 -07:00
test_gemini_tts_speech_billing.py fix(speech)!: correctly propagate spend logging and usage for Gemini TTS bridge (#44551) 2026-10-19 23:02:11 +00:00
test_global_proxy_spend_counter.py fix(proxy)!: stop update_cache from resurrecting evicted auth cache entries (#46997) 2026-10-20 17:09:53 +00:00
test_global_spend_report.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_global_spend_reset.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_google_genai_native_spend.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_image_generation_key_spend.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_key_budget_lockout.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_key_metadata_recovery_dropped_connection.py fix(spend-tracking): stop caching failed spend-log metadata lookups as confirmed misses (#43560) 2026-10-05 22:36:25 +00:00
test_key_metadata_recovery_probe_bounds.py fix(proxy-extras): build the SpendLogs indexes in the migration job instead of in migrations (#43948) 2026-10-01 14:12:22 -07:00
test_legacy_spend_logs_row_cap.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_messages_stream_usage_cost.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_model_router_selected_model.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_normalized_error_long_message.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_org_budget_cli_session_token.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_passthrough_budget_reservation.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_passthrough_request_tags.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_passthrough_stream_pricing_paths_spend.py fix(tests): read owned proxy rows from the owned proxy's database (#47333) 2026-10-20 10:07:06 -07:00
test_prompt_caching_requests_pagination.py test(proxy): migrate DB and Redis backed proxy tests into tests/integration (#43996) 2026-10-01 09:23:36 -07:00
test_ptu_share_attribution.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_redis_ttl_preserving_token_increment.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_requester_ip_address_logging.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_reset_budget_leader_election.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_responses_bridge_spend_once.py fix(logging): log spend once for large non-streaming requests on the chat to Responses bridge (#44508) 2026-10-06 21:23:19 +00:00
test_responses_cache_write_itemization.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_responses_websocket_turn_rows.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_roi_branch_spend.py fix(caching): key response cache by router model group in litellm_metadata (#44542) 2026-10-05 17:06:21 +00:00
test_router_cooldown_failure_row.py test(integration): read router cooldown failure rows from the proxy that served them (#47457) 2026-10-19 18:39:16 -07:00
test_scoped_spend_reports.py test(integration): cover budgets_spend endpoint contracts (#44736) 2026-10-14 06:58:22 +00:00
test_service_tier_stream_billing.py fix(streaming): keep the served service_tier on streamed chunks and spend rows (#42870) 2026-09-29 12:54:17 -07:00
test_session_total_spend.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_shutdown_flush.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_spend_attribution_contracts.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_spend_calculate.py fix(pricing): drop the unpublished cached rate from the Gemini Live preview entries (#42651) 2026-09-22 20:35:34 -07:00
test_spend_capture_rate_captured_spend.py test(proxy): migrate DB and Redis backed proxy tests into tests/integration (#43996) 2026-10-01 09:23:36 -07:00
test_spend_log_caller_session.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_log_read_scope.py test(integration): cover budgets_spend endpoint contracts (#44736) 2026-10-14 06:58:22 +00:00
test_spend_log_tool_payload_content.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_log_write_batching.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_logs_metadata_fields.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_rollup_accuracy.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_stream_alias_billing.py fix(proxy): stamp the client alias on a copy of each streamed chunk so pricing sees the deployment model (#44341) 2026-10-03 06:44:52 +00:00
test_tag_budget_enforcement.py feat(proxy): map baggage values to budget tags (#47358) 2026-10-20 09:09:42 +00:00
test_team_budget_enforcement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_team_daily_activity_aggregated.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_team_member_budget_alerts.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_team_member_spend.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_team_member_spend_flush.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_user_budget_on_team_keys.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_vertex_image_regional_uplift.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_vertex_passthrough_regional_uplift_spend.py fix(tests): read owned proxy rows from the owned proxy's database (#47333) 2026-10-20 10:07:06 -07:00
test_vertex_regional_uplift_spend.py fix(vertex_ai): add regional endpoint uplift to gemini-3.1-flash-image (#44673) 2026-10-05 21:00:22 -07:00