Tin Chi Lo c8b0530c30 fix(proxy): roll up tool spend daily instead of scanning SpendLogs ... GET /v1/tool/spend served the Cost Optimization card with two raw queries over LiteLLM_SpendLogToolIndex x LiteLLM_SpendLogs on every dashboard load; the totals query's driving scan was all of SpendLogs in the window. Both per-request tables reach 1M+ rows at customer scale, so the card cost O(traffic) per view and had to be capped at 30 days. The index writer also mined proxy_server_request.tools, i.e. tools DECLARED in the request body, attributing each request's full spend to tools that never ran; and all non-MCP mining ran against payload fields that are '{}' unless store_prompts_in_spend_logs is enabled, so non-MCP coverage silently depended on a privacy setting. Now the spend writer builds a ToolUsageTransaction at request time from invoked tools only, resolved by the shared get_tool_calls_from_response normalizer so every response surface (chat completions, Responses API, Anthropic Messages) is covered; the tool registry's response arm delegates to the same owner. Transactions queue beside the spend-log queue and the flush job writes index rows plus a new LiteLLM_DailyToolSpend rollup (date, tool_name PK) in one transaction, retrying connection errors with backoff (a failed batch commits nothing, so the retry cannot double-count) and dropping the batch with an error log on anything else. The endpoint aggregates in SQL: by_tool is the top TOOL_SPEND_TOP_TOOLS tools by spend via group_by and daily covers only those tools, so the response is bounded by days x TOOL_SPEND_TOP_TOOLS regardless of range or tool-name cardinality; the 30-day clamp is gone. total_spend is dropped from the response; it was never rendered and its deduplicated semantics are not computable from a rollup. Spend-log retention deliberately does not touch the rollup, so tool spend history outlives per-request rows.		2026-07-25 21:52:58 -07:00
..
example_config_yaml	test: test	2026-03-28 19:17:38 -07:00
test_configs	test: test	2026-03-28 19:17:38 -07:00
test_model_response_typing	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
azure_fine_tune.jsonl	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
batch_job_results_furniture.jsonl	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
conftest copy.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
conftest.py	[Fix] conftest: flush cache instances and warn on silent skips	2026-04-20 22:19:36 -07:00
data_map.txt	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
eagle.wav	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
gettysburg.wav	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
large_text.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
messages_with_counts.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
model_cost.json	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
openai_batch_completions.jsonl	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
openai_batch_completions_router.jsonl	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
speech_vertex.mp3	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_aproxy_startup.py	(Security fix) - Upgrade to fastapi==0.115.5 (#7447)	2024-12-28 17:08:19 -08:00
test_audit_logs_proxy.py	fix(proxy): require opt in for audit header fallback	2026-04-30 11:17:04 -07:00
test_auth_checks.py	fix(proxy-auth): deny provider-wildcard access inferred through an unrecognized model namespace	2026-07-11 19:00:03 -07:00
test_banned_keyword_list.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_blog_posts_endpoint.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_check_batch_cost.py	feat(batches): track cost for unmanaged Bedrock batches, generalize the flag (#32315)	2026-07-13 10:33:13 +05:30
test_check_responses_cost.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_custom_callback_input.py	fix: use fastuuid helper (#14903)	2025-09-25 15:47:01 -07:00
test_custom_logger_s3_gcs.py	chore(tests): thread config_file_path through s3/gcs custom-logger tests	2026-05-13 01:13:52 +00:00
test_custom_tokenizer_bug.py	test: make custom_tokenizer proxy tests hermetic (#29643)	2026-06-04 12:51:37 -07:00
test_db_schema_changes.py	test: initial test to enforce all functions in user_api_key_auth.py h… (#7797)	2025-01-15 21:52:45 -08:00
test_default_end_user_budget_simple.py	Litellm oss staging 030626 (#29578)	2026-06-03 11:01:51 -07:00
test_deployed_proxy_keygen.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_deprecated_key_grace_period.py	Litellm key rotation bug (#27756)	2026-05-12 17:16:37 -07:00
test_e2e_pod_lock_manager.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_gemini_agents_endpoints.py	Gemini managed agents support (#28270)	2026-05-19 16:02:03 -07:00
test_get_favicon.py	test(proxy): align favicon remote asset expectations	2026-04-30 11:46:45 -07:00
test_get_image.py	fix(static-assets): browser-load remote branding assets	2026-04-30 11:30:57 -07:00
test_google_endpoint_routing.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_google_gemini_proxy_request.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_jwt.py	test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700)	2026-06-04 14:56:02 -07:00
test_jwt_key_mapping.py	feat(auth): resolve caller identity once into a Principal at the auth seam (#30887)	2026-06-20 18:49:41 -07:00
test_key_generate_dynamodb.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_key_generate_prisma.py	fix(spend-tracking): drop orphaned imports; align tests with alias contract	2026-04-29 18:53:12 +00:00
test_models_fallback_endpoint.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_multipart_bypass_repro.py	fix: harden /key/update authorization checks (#27878)	2026-05-14 04:16:04 +00:00
test_prisma_client_backoff_retry.py	fix(tests): mock prisma.Prisma in backoff retry tests to avoid 'prisma generate'	2026-02-17 19:29:20 -03:00
test_prompt_test_endpoint.py	fix: harden /key/update authorization checks (#27878)	2026-05-14 04:16:04 +00:00
test_proxy_config_unit_test.py	fix: enable JSON logging via configuration and add regression test	2026-01-13 09:38:19 -07:00
test_proxy_custom_auth.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_proxy_custom_logger.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_proxy_encrypt_decrypt.py	test: fix test	2025-07-27 09:52:22 -07:00
test_proxy_exception_mapping.py	[Perf] Embeddings: Use router's O(1) lookup and shared sessions (#16344)	2025-11-14 09:21:45 -08:00
test_proxy_gunicorn.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_proxy_pass_user_config.py	test: test	2026-03-28 19:17:38 -07:00
test_proxy_reject_logging.py	[internal copy of #29089] fix: duplicate claude code traces (#29311)	2026-05-29 22:23:24 -07:00
test_proxy_routes.py	chore(ci): merge dev branch (#28801)	2026-05-25 13:44:49 -07:00
test_proxy_server.py	fix(team): make team member add atomic to prevent concurrent-add member loss (#34185)	2026-07-22 21:47:22 +00:00
test_proxy_server_caching.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_proxy_server_keys.py	fix tests	2025-10-25 10:19:24 -07:00
test_proxy_server_langfuse.py	(Security fix) - Upgrade to fastapi==0.115.5 (#7447)	2024-12-28 17:08:19 -08:00
test_proxy_server_spend.py	fix(pattern_match_deployments.py): default to user input if unable to… (#6632)	2024-11-08 00:55:57 +05:30
test_proxy_setting_guardrails.py	fix(lakera-guardrail): use os.environ.get() to avoid KeyError on missing LAKERA_API_KEY	2026-02-17 19:30:22 -03:00
test_proxy_token_counter.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_proxy_utils.py	feat(guardrails): add run_in_parallel opt-in for concurrent pre_call and post_call guardrails (#33770)	2026-07-24 13:25:58 -07:00
test_realtime_cache.py	feat: litellm oss 110626 (#30202)	2026-06-11 22:30:26 -07:00
test_reducto_ocr_route.py	Litellm oss staging 04 21 2026 2 (#26569)	2026-05-20 21:25:19 -07:00
test_request_size_limit_middleware.py	Fix early proxy request size enforcement (#27311)	2026-05-06 12:29:11 -07:00
test_response_polling_handler.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_response_polling_pre_call_checks.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_search_api_logging.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_server_root_path.py	fix: server rooth path (#19790)	2026-01-26 09:48:06 -08:00
test_skills_db.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_ui_path_detection.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_unit_test_max_model_budget_limiter.py	feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783)	2026-07-03 12:20:12 -07:00
test_unit_test_proxy_hooks.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_update_daily_tag_spend.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_update_spend.py	fix(proxy): roll up tool spend daily instead of scanning SpendLogs	2026-07-25 21:52:58 -07:00
test_user_api_key_auth.py	fix(proxy): populate user_email on UserAPIKeyAuth for JWT auth (#34174)	2026-07-21 16:11:47 -07:00
test_zero_cost_model_budget_bypass.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
vertex_key.json	test: update to new vertex ai keys	2026-03-28 20:19:05 -07:00