mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
703fde9f1e
Adds GET /auto_router/benchmarks, an admin-only read over spend logs that reports, per configured auto-router, the four session metrics customers ask for (turns per session, average session length, tokens per session) plus a routed-vs-baseline dollar savings estimate. Session-scoped metrics can't come from the daily rollups because session_id lives only on LiteLLM_SpendLogs, so this reads spend logs directly over a window clamped to 30 days, matching the tool-spend endpoint. Spend is filtered by the auto-router alias, which keeps the turn count honest: the LLM classifier's own judge calls share the session but carry the judge model's group, not the alias. The savings figure compares the routed model mix against sending every request to a single baseline model, defaulting to the priciest model the router actually routed to in the window. A per-router benchmark_baseline_model param pins it to a fixed flagship instead. The estimate uses list prices on the tokens actually spent and does not model the caching a single-model baseline would have had; the UI says so. Surfaced as stat tiles on the Cost Optimization autorouter tab. |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||