mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-22 00:31:44 +00:00
3f3295b33f
* feat(spend): track prompt compression saved tokens in daily spend aggregates Native compression interception now records tokens_before/after/saved into the request litellm_metadata so savings land in the SpendLog metadata JSON under a typed compression_savings key. A single normalizer (extract_compression_saved_tokens) sums that key with Headroom guardrail tokens_saved; the two writers are disjoint and run at different stages, so summing never double-counts. The spend-log redactor now preserves purely numeric compression stats inside guardrail_response so Headroom savings survive the store_prompts_in_spend_logs=false default. compression_saved_tokens is threaded through BaseDailySpendTransaction, queue aggregation, the daily upsert blocks, a new BigInt column on all six daily spend tables, and the daily activity read path (SpendMetrics, DailySpendMetadata, raw-SQL rollups) * fix(spend): normalize legacy guardrail shapes and float token stats in compression savings reader * feat(spend): aggregate compression and prompt caching dollar savings in daily rollups Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(spend): update daily spend aggregation fixtures for savings columns Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(ui): add Cost Optimization dashboard page New left-nav Cost Optimization page under Observability that surfaces money saved by prompt compression and prompt caching. It reads the daily activity rollup (userDailyActivityCall / get_daily_activity) and never scans SpendLogs, so it stays fast at 1M+ rows. Renders a Total saved card, per-driver Compression and Prompt caching cards, a savings-over-time area chart, and a savings-by-driver donut, all aggregated in memory from the per-day metrics.compression_savings_spend and metrics.prompt_caching_savings_spend fields. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| policy_endpoints | ||
| scim | ||
| search_endpoints | ||
| usage_endpoints | ||
| test_access_group_endpoints.py | ||
| test_access_group_management.py | ||
| test_activity_tenant_scoping.py | ||
| test_budget_endpoints.py | ||
| test_cache_settings_endpoints.py | ||
| test_callback_management_endpoints.py | ||
| test_common_daily_activity.py | ||
| test_common_utils.py | ||
| test_compliance_endpoints.py | ||
| test_config_override_endpoints.py | ||
| test_coordination_redis_endpoints.py | ||
| test_cost_tracking_settings.py | ||
| test_credential_migration.py | ||
| test_customer_budget.py | ||
| test_customer_endpoints.py | ||
| test_delete_callbacks_endpoint.py | ||
| test_delete_verification_tokens_failed.py | ||
| test_encryption_endpoints.py | ||
| test_entraid_app_roles.py | ||
| test_internal_user_endpoints.py | ||
| test_key_management_endpoints.py | ||
| test_mcp_management_endpoints.py | ||
| test_model_management_endpoints.py | ||
| test_org_admin_team_access.py | ||
| test_organization_endpoints.py | ||
| test_policy_endpoints.py | ||
| test_project_org_authz.py | ||
| test_router_settings_endpoints.py | ||
| test_tag_management_endpoints.py | ||
| test_team_callback_endpoints.py | ||
| test_team_default_params.py | ||
| test_team_endpoints.py | ||
| test_team_model_alias_merge.py | ||
| test_tool_management_endpoints.py | ||
| test_ui_sso.py | ||
| test_workflow_management_endpoints.py | ||