mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-08 03:08:45 +00:00
2e6d196684
Cost tells an operator the auto-router is cheaper, not that it is good. This adds the two quality signals that are already recorded on every request, so no new instrumentation is needed: escalation: a turn moved to a costlier model than the turn before it, inside one session. High precision, low recall. Only upward moves count, since a move down to a cheaper model is a deliberate price or latency trade, not evidence the router picked wrong. abandonment: the caller hung up mid-stream. Low precision, high recall, so it catches the giving-up that escalation structurally misses. They render side by side rather than blended, because they fail in opposite directions and one number would hide which of them fired. Both are computed for auto-routed sessions and for the same keys' directly addressed sessions, with identical definitions over LiteLLM_SpendLogs, so the two are comparable. The baseline is restricted to keys that actually used the router, since keys carry different model lists and pooling all non-router traffic would compare unlike populations. It is withheld entirely when the cohort is too small or when its traffic is not carrying caller-supplied session IDs, with the reason shown in place of a misleading rate. Neither signal can come from LiteLLM_AutoRouterSession: that rollup folds a session to per-model last-touch facts and discards turn order, and escalation is a question about order. Also moves avg saved per session up under the headline cost figure. |
||
|---|---|---|
| .. | ||
| public | ||
| scripts | ||
| src | ||
| tests | ||
| .env.development | ||
| .env.production | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc | ||
| build_release_ui.sh | ||
| build_ui.sh | ||
| build_ui_custom_path.sh | ||
| CLAUDE.md | ||
| components.json | ||
| eslint-budgets.json | ||
| eslint-suppressions.json | ||
| eslint.config.mjs | ||
| knip.json | ||
| next.config.mjs | ||
| package-lock.json | ||
| package.json | ||
| postcss.config.js | ||
| README.md | ||
| tsconfig.json | ||
| tsconfig.tsbuildinfo | ||
| vitest.config.ts | ||
This is a Next.js project bootstrapped with create-next-app.
Getting Started
First, run the development server:
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.
This project uses next/font to automatically optimize and load Inter, a custom Google Font.
Learn More
To learn more about Next.js, take a look at the following resources:
- Next.js Documentation - learn about Next.js features and API.
- Learn Next.js - an interactive Next.js tutorial.
You can check out the Next.js GitHub repository - your feedback and contributions are welcome!
Deploy on Vercel
The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.
Check out our Next.js deployment documentation for more details.