scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-13 08:52:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 174
litellm/tests/integration/spend
History
Exact
jesus-berri f0021859a7
feat(ui): set per-threshold budget alert recipients on virtual keys (#51061) 
* feat(ui): set per-threshold budget alert recipients on virtual keys

Create Key and Edit Key get the threshold and recipients rows the team
member budget alerts already use, writing metadata.max_budget_alert_emails.
The helpers and the row editor move to key_team_helpers so the team form
and both key forms share them.

* test(ui): cover the empty metadata box path and drop a redundant payload case

* fix(ui): keep comma-separated string alert recipients stored through the API

The backend accepts a string of recipients per threshold, so Edit Key and the
team form now load them as rows instead of dropping them and deleting the
alert on the next save

* fix(ui): keep the stored key alert map on saves that do not edit its rows

Edit Key rebuilt max_budget_alert_emails from the rows on every save, so
API-written entries the rows cannot show (a null recipient list, a list with
a null, thresholds outside 1 to 100) were deleted by unrelated edits. The
map is now rebuilt only when the rows were edited and otherwise sent back
as stored

* fix(ui): load null and mixed alert recipient lists as rows and drop a typed map on Create Key

A null recipient list loads as an owner-only row and a mixed list keeps its
strings, matching what the backend sends. Create Key now ignores a
max_budget_alert_emails typed into the Metadata box, as Edit Key does, and
the hint says thresholds replace the default 80% alert

* fix(ui): keep Create Key alert rows when the Metadata box holds null

* fix(ui): move typed alert maps into rows, keep hidden alert entries, hide alert rows from team admins

A max_budget_alert_emails map typed into the Metadata box on Edit Key or Create Key now moves
into the threshold rows on blur with a toast, the same way a tags array moves to the Tags field.
Entries the rows cannot show stay in the JSON box.

When the rows are edited, the saved map keeps the stored entries the rows cannot show (thresholds
outside 1 to 100, non-numeric keys, unreadable values). Create Key keeps such typed entries too.

Team admins editing a member's key no longer see the alert rows, since that save only accepts
budget fields

* fix(ui): let the Metadata box own key alert entries the threshold rows cannot show

Edit Key now shows max_budget_alert_emails entries outside whole numbers 1 to 100 in the
Metadata JSON box and saves them from the box as the admin left it, so they can be seen,
edited and removed. Rows still own every entry they can show, and an untouched save sends
the stored map unchanged

* fix(ui): leave zero-padded alert thresholds to the Metadata box instead of duplicating a row

* fix(ui): keep loading zero-padded team member alert thresholds as rows

The canonical-key rule applies only to key alert maps, where the Metadata box
holds entries the rows cannot show. The team form has no such fallback, so it
keeps loading "080" as row 80 as it did before

* fix(ui): let the Metadata box own a non-object max_budget_alert_emails value

A string, list, number or null stored under max_budget_alert_emails stays in the
Edit Key Metadata box, is saved back as the box holds it, and is stored as typed
on Create Key. Threshold rows only own an object map, so rows the admin adds
replace a non-object value

* fix(ui): move typed alert thresholds into the rows when Edit Key saves without a blur

Submit now moves a max_budget_alert_emails map typed into the Metadata box
into the rows, as it already does for tags, and reads the edited fields at
submit time so the moved rows are saved

* refactor(ui): name the Edit Key submit handler
2026-11-12 15:39:44 -08:00
..
fixtures feat(spend): report guardrail spend separately in daily activity (#51010) 2026-11-10 13:02:26 -08:00
golden feat(spend): report guardrail spend separately in daily activity (#51010) 2026-11-10 13:02:26 -08:00
__init__.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
_background_responses.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
_daily_activity_fixtures.py feat(usage): add top users by spend and user activity views (#45310) 2026-10-10 12:21:43 -07:00
_partial_stream.py fix(proxy): raise live budget counters for failure spend rows (#47741) 2026-10-30 13:05:12 -07:00
test_agent_spend.py fix(a2a)!: bill completion-bridge agents at their own prices on /a2a send and stream (#49150) 2026-10-30 19:43:45 -07:00
test_autorouter_session_rollup.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
test_background_interaction_settlement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_background_response_billed_once.py fix(responses): resume background response streams with stream and starting_after (#48819) 2026-10-29 02:31:22 -07:00
test_background_response_billing_proxies.py fix(proxy): report requested model in streamed Responses response.model (#46662) 2026-10-17 19:52:10 +00:00
test_background_response_poll_retirement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_baggage_tag_mappings.py chore(proxy): use neutral example names for baggage tag mappings (#50011) 2026-11-04 14:49:55 +00:00
test_baseline_accounting.py test(integration): fix the baseline accounting and series cap flakes (#51096) 2026-11-11 12:36:02 -08:00
test_batch_completion_accounting.py fix(logging): deliver completed batch success callbacks consistently (#47192) 2026-10-18 09:19:04 -07:00
test_batch_enqueued_tokens_redis_lua.py test(proxy): migrate DB and Redis backed proxy tests into tests/integration (#43996) 2026-10-01 09:23:36 -07:00
test_batch_observability.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_batch_poll_starvation.py fix(batches): stop unbillable batches from blocking file deletes and new-batch polling (#48752) 2026-10-28 12:19:36 -07:00
test_budget_limit_envelopes.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_cache_activity.py fix(proxy): scope /global/activity/cache_hits analytics to the caller's own and permitted-team logs (#46210) 2026-10-14 10:01:21 +02:00
test_cache_and_quota.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_chaos_burst_spend_once.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_chatgpt_subscription_usage.py feat(usage): show subscription-covered providers and account fees separately (#44772) 2026-10-30 03:58:36 -07:00
test_container_spend.py test(integration): cover videos_containers endpoint contracts (#44691) 2026-10-14 03:49:01 +00:00
test_customer_spend_by_team.py feat(usage): split customer spend by the key's current team (#50764) 2026-11-09 19:35:31 -08:00
test_daily_activity_aggregated_breakdowns.py fix!: retain unattributed daily activity across reports and exports (#48925) 2026-10-29 09:39:49 -07:00
test_daily_activity_key_alias_probes.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_daily_activity_key_metadata_query_timeout.py feat(ui): unify gateway and usage assistance as Aria Agent (#48082) 2026-10-23 20:24:19 +00:00
test_daily_activity_key_owner.py test(usage): assert per-user entity spend and metadata (#50251) 2026-11-05 20:16:40 +00:00
test_daily_activity_key_owner_faults.py test(usage): assert per-user entity spend and metadata (#50251) 2026-11-05 20:16:40 +00:00
test_daily_activity_key_owner_traffic.py feat(ui): unify gateway and usage assistance as Aria Agent (#48082) 2026-10-23 20:24:19 +00:00
test_daily_activity_repository.py feat(usage): add top users by spend and user activity views (#45310) 2026-10-10 12:21:43 -07:00
test_daily_activity_routes.py feat(spend): report guardrail spend separately in daily activity (#51010) 2026-11-10 13:02:26 -08:00
test_daily_rollup_retry.py feat(spend): report guardrail spend separately in daily activity (#51010) 2026-11-10 13:02:26 -08:00
test_daily_tag_spend_retention.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_disable_end_user_cost_tracking.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_disconnected_bedrock_messages_stream_billing.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_end_user_budget_enforcement.py fix(anthropic_endpoints)!: return Anthropic error envelopes on /v1/messages auth failures (#48466) 2026-11-03 01:53:29 +00:00
test_end_user_spend_without_proxy_user.py fix(anthropic_endpoints)!: return Anthropic error envelopes on /v1/messages auth failures (#48466) 2026-11-03 01:53:29 +00:00
test_expired_ui_session_failure_rows.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_fail_closed_team_member_budget.py fix(proxy): fail-closed budget enforcement returns 403 naming /team/member_add for a team user with no membership row (#50603) 2026-11-09 15:38:59 -08:00
test_failed_dispatch_tokens.py test(spend): audit #47815 (#48774) 2026-10-28 03:04:11 -07:00
test_failing_request_stored_snapshot.py test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984) 2026-11-04 09:55:53 -08:00
test_failure_spend_live_counters_chaos.py fix(proxy): raise live budget counters for failure spend rows (#47741) 2026-10-30 13:05:12 -07:00
test_filtered_ledger.py test: add extension and browser integration contracts 2026-09-16 13:15:53 -07:00
test_gemini_tts_speech_billing.py test(integration): repair seven stale integration tests on main (#47600) 2026-10-20 11:21:04 -07:00
test_global_proxy_spend_counter.py test(proxy): audit #46997 (#48918) 2026-10-28 18:05:03 -07:00
test_global_spend_report.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_global_spend_reset.py fix(proxy)!: stop answering with the literal string "None" as error type and param on the proxy-server, auth, spend, and key health routes (#39540) 2026-10-29 18:20:52 -07:00
test_google_genai_native_spend.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_image_generation_key_spend.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_key_budget_alerts.py feat(ui): set per-threshold budget alert recipients on virtual keys (#51061) 2026-11-12 15:39:44 -08:00
test_key_budget_lockout.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_key_metadata_recovery_dropped_connection.py fix(spend-tracking): stop caching failed spend-log metadata lookups as confirmed misses (#43560) 2026-10-05 22:36:25 +00:00
test_key_metadata_recovery_probe_bounds.py fix(proxy-extras): build the SpendLogs indexes in the migration job instead of in migrations (#43948) 2026-10-01 14:12:22 -07:00
test_legacy_spend_logs_row_cap.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_messages_stream_usage_cost.py feat(proxy): per-request header opts a stream into usage.cost without the global include_cost_in_streaming_usage flag (#50574) 2026-11-07 16:57:20 -08:00
test_model_router_selected_model.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_non_finite_cost_spend_writes.py fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829) 2026-10-30 08:27:35 -07:00
test_normalized_error_long_message.py test(integration): give each kill-burst its own prompts so cached control replies cannot stand in for late traffic (#50569) 2026-11-07 23:08:23 +00:00
test_openrouter_upstream_provider.py feat(openrouter): record upstream provider in logging payload and spend logs (#49287) 2026-10-30 12:51:48 -07:00
test_org_budget_cli_session_token.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_partial_stream_live_budget.py test(spend): assert Anthropic envelope for Messages budget refusals (#49800) 2026-11-03 06:10:08 +00:00
test_passthrough_budget_reservation.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_passthrough_failure.py fix(proxy): write SpendLogs failure rows for config-defined pass-through endpoints (#41831) 2026-11-09 17:05:10 -08:00
test_passthrough_request_tags.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_passthrough_stream_pricing_paths_spend.py fix(proxy): bill an OpenAI pass-through background response once across repeated reads (#50841) 2026-11-10 07:33:25 -08:00
test_passthrough_upstream_model.py feat(passthrough): let upstreams report the model name via x-litellm-upstream-model (#48275) 2026-10-27 18:28:54 -07:00
test_prompt_caching_requests_pagination.py test(proxy): migrate DB and Redis backed proxy tests into tests/integration (#43996) 2026-10-01 09:23:36 -07:00
test_ptu_share_attribution.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_redis_ttl_preserving_token_increment.py test(spend): exercise TTL-preserving increments through rate limit store (#50418) 2026-11-06 17:53:25 +00:00
test_requester_ip_address_logging.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_reset_budget_leader_election.py fix: make scheduled key, user, and team spend resets idempotent (#50298) 2026-11-09 22:21:43 -08:00
test_responses_bridge_spend_once.py fix(logging): keep the caller's call_type when chat completions are bridged to the Responses API (#49710) 2026-11-03 10:02:54 +00:00
test_responses_cache_write_itemization.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_responses_stream_spend_row_id.py fix(spend): keep free response reads and cancels off the creation spend row (#51110) 2026-11-11 07:02:00 +00:00
test_responses_websocket_turn_rows.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_roi_branch_spend.py fix(caching): key response cache by router model group in litellm_metadata (#44542) 2026-10-05 17:06:21 +00:00
test_router_cooldown_failure_row.py test(integration): read router cooldown failure rows from the proxy that served them (#47457) 2026-10-19 18:39:16 -07:00
test_scoped_spend_reports.py test(integration): cover budgets_spend endpoint contracts (#44736) 2026-10-14 06:58:22 +00:00
test_service_tier_stream_billing.py fix(streaming): keep the served service_tier on streamed chunks and spend rows (#42870) 2026-09-29 12:54:17 -07:00
test_session_grouped_logs_pages.py fix(proxy): open the Logs drawer by request_id without scanning for litellm_call_id (#50789) 2026-11-10 17:03:21 +00:00
test_session_total_spend.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_shutdown_flush.py test: update the spend tests that still expected the pre-ledger accounting queues (#50814) 2026-11-09 22:07:42 -08:00
test_spend_attribution_contracts.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_spend_calculate.py fix(pricing): drop the unpublished cached rate from the Gemini Live preview entries (#42651) 2026-09-22 20:35:34 -07:00
test_spend_capture_rate_captured_spend.py test(proxy): migrate DB and Redis backed proxy tests into tests/integration (#43996) 2026-10-01 09:23:36 -07:00
test_spend_log_caller_session.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_log_latest_turn.py feat(spend-logs): opt-in latest_turn mode that stores only the current turn of stateless requests (#49544) 2026-11-05 19:41:15 -08:00
test_spend_log_read_scope.py fix(proxy): open the Logs drawer by request_id without scanning for litellm_call_id (#50789) 2026-11-10 17:03:21 +00:00
test_spend_log_tool_payload_content.py fix(spend-logs): mask token-shaped secrets in stored request and resp… (#48968) 2026-11-12 18:45:06 +00:00
test_spend_log_write_batching.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_logs_metadata_fields.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_spend_rollup_accuracy.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_stream_alias_billing.py fix(proxy): stamp the client alias on a copy of each streamed chunk so pricing sees the deployment model (#44341) 2026-10-03 06:44:52 +00:00
test_streamed_usage_estimate.py test(spend): audit #47353 (#48785) 2026-10-28 03:50:50 -07:00
test_tag_budget_enforcement.py feat(proxy): map baggage values to budget tags (#47358) 2026-10-20 09:09:42 +00:00
test_tag_spend_reset.py feat(tags): reset a tag's spend through /tag/update (#50323) 2026-11-06 10:41:22 -08:00
test_team_budget_enforcement.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_team_daily_activity_aggregated.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_team_member_budget_alerts.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_team_member_spend.py test: update the spend tests that still expected the pre-ledger accounting queues (#50814) 2026-11-09 22:07:42 -08:00
test_team_member_spend_flush.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_team_member_total_spend_reconciliation.py fix(proxy): reconcile team member total_spend with current cycle spend (#48260) 2026-10-24 17:57:09 -07:00
test_text_completion_stream_usage_cost.py fix(proxy)!: bill streamed /v1/completions without stream_options like chat (#45366) 2026-10-29 17:18:51 -07:00
test_unmanaged_vertex_batch_deployment_routing_wire.py fix(batches): cost an unmanaged gs:// batch through the deployment that created it (#49301) 2026-10-30 15:25:47 -07:00
test_user_budget_on_team_keys.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_vertex_gemini_image_token_spend.py fix(vertex_ai): bill gemini image text, thinking and priority tokens at their own rates (#51017) 2026-11-11 02:50:55 +00:00
test_vertex_image_regional_uplift.py test(spend): expect token billing for Vertex image edits with usage (#51109) 2026-11-11 03:32:19 +00:00
test_vertex_passthrough_regional_uplift_spend.py fix(tests): read owned proxy rows from the owned proxy's database (#47333) 2026-10-20 10:07:06 -07:00
test_vertex_regional_uplift_spend.py test(spend): expect token billing for Vertex image edits (#51148) 2026-11-11 06:27:58 +00:00