scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-05 08:07:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 136
litellm/litellm-proxy-extras
History
tin-berri 3f3295b33f
feat(spend): track prompt compression saved tokens in daily spend aggregates (#33810) 
* feat(spend): track prompt compression saved tokens in daily spend aggregates

Native compression interception now records tokens_before/after/saved into the
request litellm_metadata so savings land in the SpendLog metadata JSON under a
typed compression_savings key. A single normalizer
(extract_compression_saved_tokens) sums that key with Headroom guardrail
tokens_saved; the two writers are disjoint and run at different stages, so
summing never double-counts. The spend-log redactor now preserves purely
numeric compression stats inside guardrail_response so Headroom savings
survive the store_prompts_in_spend_logs=false default. compression_saved_tokens
is threaded through BaseDailySpendTransaction, queue aggregation, the daily
upsert blocks, a new BigInt column on all six daily spend tables, and the
daily activity read path (SpendMetrics, DailySpendMetadata, raw-SQL rollups)

* fix(spend): normalize legacy guardrail shapes and float token stats in compression savings reader

* feat(spend): aggregate compression and prompt caching dollar savings in daily rollups

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(spend): update daily spend aggregation fixtures for savings columns

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(ui): add Cost Optimization dashboard page

New left-nav Cost Optimization page under Observability that surfaces money saved by prompt compression and prompt caching. It reads the daily activity rollup (userDailyActivityCall / get_daily_activity) and never scans SpendLogs, so it stays fast at 1M+ rows.

Renders a Total saved card, per-driver Compression and Prompt caching cards, a savings-over-time area chart, and a savings-by-driver donut, all aggregated in memory from the per-day metrics.compression_savings_spend and metrics.prompt_caching_savings_spend fields.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-18 17:47:54 -07:00
..
dist adding build 2026-03-21 22:55:04 -07:00
litellm_proxy_extras feat(spend): track prompt compression saved tokens in daily spend aggregates (#33810) 2026-07-18 17:47:54 -07:00
tests [Fix] v2 resolver: swallow non-connection DB errors; wrap resolve failures 2026-04-21 15:53:07 -07:00
build_and_publish.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
LICENSE install prisma migration files - connects litellm proxy to litellm's prisma migration files (#9637) 2025-03-29 15:27:09 -07:00
migration_runbook.md [Infra] Add freshness and destructive guards to migration workflow 2026-04-21 12:00:23 -07:00
pyproject.toml bump: litellm-proxy-extras 0.4.78 -> 0.4.79 (#33855) 2026-07-18 22:06:57 +00:00
README.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00

README.md

Additional files for the proxy. Reduces the size of the main litellm package.

Currently, only stores the migration.sql files for litellm-proxy.

To install, run:

uv add litellm-proxy-extras

OR

uv tool install 'litellm[proxy]' # installs litellm-proxy-extras and other proxy dependencies

To use the migrations, run:

litellm --use_prisma_migrate