mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
e30b7f2f1f
Adds auto-router as a third savings driver alongside compression and prompt caching: a summary card, a donut segment, and a series in the savings graph. Savings are the net dollars from routing a request to the selected model instead of a counterfactual baseline. The delta prices prompt tokens at each model's input rate and completion tokens at each model's output rate, then subtracts the cache-write penalty the selected deployment incurs on a cold cache. Cache-read discounts stay attributed to the prompt-caching driver to avoid double-counting. The result is floored at zero so an escalation to a pricier model never reads as negative savings. The baseline model defaults to claude-opus-5 and is operator-configurable per deployment via the auto_router_savings_baseline_model litellm_param. It flows AutoRouter to PreRoutingHookResponse to the metadata bucket to SpendLogsMetadata to the daily spend writer, mirroring the routing_decision path, and is stripped from untrusted caller metadata so it cannot be spoofed. Savings accrue into a new autorouter_savings_spend column on the six LiteLLM_Daily*Spend rollup tables; no LiteLLM_SpendLogs queries are added. |
||
|---|---|---|
| .. | ||
| public | ||
| scripts | ||
| src | ||
| tests | ||
| .env.development | ||
| .env.production | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc | ||
| build_release_ui.sh | ||
| build_ui.sh | ||
| build_ui_custom_path.sh | ||
| CLAUDE.md | ||
| components.json | ||
| eslint-budgets.json | ||
| eslint-suppressions.json | ||
| eslint.config.mjs | ||
| knip.json | ||
| next.config.mjs | ||
| package-lock.json | ||
| package.json | ||
| postcss.config.js | ||
| README.md | ||
| tsconfig.json | ||
| tsconfig.tsbuildinfo | ||
| vitest.config.ts | ||
This is a Next.js project bootstrapped with create-next-app.
Getting Started
First, run the development server:
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.
This project uses next/font to automatically optimize and load Inter, a custom Google Font.
Learn More
To learn more about Next.js, take a look at the following resources:
- Next.js Documentation - learn about Next.js features and API.
- Learn Next.js - an interactive Next.js tutorial.
You can check out the Next.js GitHub repository - your feedback and contributions are welcome!
Deploy on Vercel
The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.
Check out our Next.js deployment documentation for more details.