mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
8423efc5fa
* feat(spend): track prompt compression saved tokens in daily spend aggregates (#33810) * feat(spend): track prompt compression saved tokens in daily spend aggregates Native compression interception now records tokens_before/after/saved into the request litellm_metadata so savings land in the SpendLog metadata JSON under a typed compression_savings key. A single normalizer (extract_compression_saved_tokens) sums that key with Headroom guardrail tokens_saved; the two writers are disjoint and run at different stages, so summing never double-counts. The spend-log redactor now preserves purely numeric compression stats inside guardrail_response so Headroom savings survive the store_prompts_in_spend_logs=false default. compression_saved_tokens is threaded through BaseDailySpendTransaction, queue aggregation, the daily upsert blocks, a new BigInt column on all six daily spend tables, and the daily activity read path (SpendMetrics, DailySpendMetadata, raw-SQL rollups) * fix(spend): normalize legacy guardrail shapes and float token stats in compression savings reader * feat(spend): aggregate compression and prompt caching dollar savings in daily rollups Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(spend): update daily spend aggregation fixtures for savings columns Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(ui): add Cost Optimization dashboard page New left-nav Cost Optimization page under Observability that surfaces money saved by prompt compression and prompt caching. It reads the daily activity rollup (userDailyActivityCall / get_daily_activity) and never scans SpendLogs, so it stays fast at 1M+ rows. Renders a Total saved card, per-driver Compression and Prompt caching cards, a savings-over-time area chart, and a savings-by-driver donut, all aggregated in memory from the per-day metrics.compression_savings_spend and metrics.prompt_caching_savings_spend fields. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit |
||
|---|---|---|
| .. | ||
| arize | ||
| azure_storage | ||
| bitbucket | ||
| cloudzero | ||
| code_interpreter_interception | ||
| compression_interception | ||
| datadog | ||
| dotprompt | ||
| focus | ||
| gcs_bucket | ||
| gcs_pubsub | ||
| gitlab | ||
| langfuse | ||
| levo | ||
| litellm_agent | ||
| mavvrik_focus | ||
| newrelic | ||
| open_telemetry | ||
| opik | ||
| otel | ||
| SlackAlerting | ||
| websearch_interception | ||
| rubrik_test_helpers.py | ||
| test_agentops.py | ||
| test_anthropic_cache_control_hook.py | ||
| test_athina.py | ||
| test_azure_sentinel.py | ||
| test_braintrust_logging.py | ||
| test_braintrust_span_name.py | ||
| test_custom_guardrail.py | ||
| test_custom_guardrail_recursion.py | ||
| test_custom_prompt_management.py | ||
| test_deepeval.py | ||
| test_galileo.py | ||
| test_guardrail_logging_sync.py | ||
| test_langfuse.py | ||
| test_langfuse_otel.py | ||
| test_langsmith_init.py | ||
| test_mlflow.py | ||
| test_openmeter.py | ||
| test_opentelemetry.py | ||
| test_opentelemetry_dynamic_imports.py | ||
| test_opik_utils.py | ||
| test_otel_guardrail_violation_spans.py | ||
| test_otel_team_attributes_matrix.py | ||
| test_prometheus_api_promql_escape.py | ||
| test_prometheus_budget_metric_guard.py | ||
| test_prometheus_budget_metrics_db_lookups.py | ||
| test_prometheus_budget_metrics_timeout.py | ||
| test_prometheus_cache_metrics.py | ||
| test_prometheus_client_ip_user_agent.py | ||
| test_prometheus_custom_metadata_label_counts.py | ||
| test_prometheus_end_user_cardinality.py | ||
| test_prometheus_invalid_key_filtering.py | ||
| test_prometheus_labels.py | ||
| test_prometheus_mcp_tool_metrics.py | ||
| test_prometheus_media_generation_metrics.py | ||
| test_prometheus_metric_name_consistency.py | ||
| test_prometheus_missing_metrics.py | ||
| test_prometheus_none_metadata.py | ||
| test_prometheus_overhead_with_guardrails.py | ||
| test_prometheus_queue_guardrail_metrics.py | ||
| test_prometheus_rate_limit_labels.py | ||
| test_prometheus_remaining_tokens_router_fallback.py | ||
| test_prometheus_services.py | ||
| test_prometheus_spend_logs_metadata.py | ||
| test_prometheus_stream_label.py | ||
| test_prometheus_token_detail_metrics.py | ||
| test_prometheus_user_team_metrics.py | ||
| test_prompt_manager_ssti.py | ||
| test_responses_background_cost.py | ||
| test_rubrik.py | ||
| test_s3_v2.py | ||
| test_weave_otel.py | ||