mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-12 03:48:56 +00:00
c8b0530c30
GET /v1/tool/spend served the Cost Optimization card with two raw queries
over LiteLLM_SpendLogToolIndex x LiteLLM_SpendLogs on every dashboard load;
the totals query's driving scan was all of SpendLogs in the window. Both
per-request tables reach 1M+ rows at customer scale, so the card cost
O(traffic) per view and had to be capped at 30 days.
The index writer also mined proxy_server_request.tools, i.e. tools DECLARED
in the request body, attributing each request's full spend to tools that
never ran; and all non-MCP mining ran against payload fields that are '{}'
unless store_prompts_in_spend_logs is enabled, so non-MCP coverage silently
depended on a privacy setting.
Now the spend writer builds a ToolUsageTransaction at request time from
invoked tools only, resolved by the shared get_tool_calls_from_response
normalizer so every response surface (chat completions, Responses API,
Anthropic Messages) is covered; the tool registry's response arm delegates
to the same owner. Transactions queue beside the spend-log queue and the
flush job writes index rows plus a new LiteLLM_DailyToolSpend rollup
(date, tool_name PK) in one transaction, retrying connection errors with
backoff (a failed batch commits nothing, so the retry cannot double-count)
and dropping the batch with an error log on anything else.
The endpoint aggregates in SQL: by_tool is the top TOOL_SPEND_TOP_TOOLS
tools by spend via group_by and daily covers only those tools, so the
response is bounded by days x TOOL_SPEND_TOP_TOOLS regardless of range or
tool-name cardinality; the 30-day clamp is gone. total_spend is dropped
from the response; it was never rendered and its deduplicated semantics
are not computable from a rollup. Spend-log retention deliberately does
not touch the rollup, so tool spend history outlives per-request rows.
|
||
|---|---|---|
| .. | ||
| example_config_yaml | ||
| test_configs | ||
| test_model_response_typing | ||
| azure_fine_tune.jsonl | ||
| batch_job_results_furniture.jsonl | ||
| conftest copy.py | ||
| conftest.py | ||
| data_map.txt | ||
| eagle.wav | ||
| gettysburg.wav | ||
| large_text.py | ||
| messages_with_counts.py | ||
| model_cost.json | ||
| openai_batch_completions.jsonl | ||
| openai_batch_completions_router.jsonl | ||
| speech_vertex.mp3 | ||
| test_aproxy_startup.py | ||
| test_audit_logs_proxy.py | ||
| test_auth_checks.py | ||
| test_banned_keyword_list.py | ||
| test_blog_posts_endpoint.py | ||
| test_check_batch_cost.py | ||
| test_check_responses_cost.py | ||
| test_custom_callback_input.py | ||
| test_custom_logger_s3_gcs.py | ||
| test_custom_tokenizer_bug.py | ||
| test_db_schema_changes.py | ||
| test_default_end_user_budget_simple.py | ||
| test_deployed_proxy_keygen.py | ||
| test_deprecated_key_grace_period.py | ||
| test_e2e_pod_lock_manager.py | ||
| test_gemini_agents_endpoints.py | ||
| test_get_favicon.py | ||
| test_get_image.py | ||
| test_google_endpoint_routing.py | ||
| test_google_gemini_proxy_request.py | ||
| test_jwt.py | ||
| test_jwt_key_mapping.py | ||
| test_key_generate_dynamodb.py | ||
| test_key_generate_prisma.py | ||
| test_models_fallback_endpoint.py | ||
| test_multipart_bypass_repro.py | ||
| test_prisma_client_backoff_retry.py | ||
| test_prompt_test_endpoint.py | ||
| test_proxy_config_unit_test.py | ||
| test_proxy_custom_auth.py | ||
| test_proxy_custom_logger.py | ||
| test_proxy_encrypt_decrypt.py | ||
| test_proxy_exception_mapping.py | ||
| test_proxy_gunicorn.py | ||
| test_proxy_pass_user_config.py | ||
| test_proxy_reject_logging.py | ||
| test_proxy_routes.py | ||
| test_proxy_server.py | ||
| test_proxy_server_caching.py | ||
| test_proxy_server_keys.py | ||
| test_proxy_server_langfuse.py | ||
| test_proxy_server_spend.py | ||
| test_proxy_setting_guardrails.py | ||
| test_proxy_token_counter.py | ||
| test_proxy_utils.py | ||
| test_realtime_cache.py | ||
| test_reducto_ocr_route.py | ||
| test_request_size_limit_middleware.py | ||
| test_response_polling_handler.py | ||
| test_response_polling_pre_call_checks.py | ||
| test_search_api_logging.py | ||
| test_server_root_path.py | ||
| test_skills_db.py | ||
| test_ui_path_detection.py | ||
| test_unit_test_max_model_budget_limiter.py | ||
| test_unit_test_proxy_hooks.py | ||
| test_update_daily_tag_spend.py | ||
| test_update_spend.py | ||
| test_user_api_key_auth.py | ||
| test_zero_cost_model_budget_bypass.py | ||
| vertex_key.json | ||