mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
109193f26a
* feat(router): add LLM-based classifier option to complexity router Adds classifier_type: "heuristic" | "llm" to complexity_router_config. When set to "llm", the router calls a configured model (e.g. a small model like haiku) via structured output to pick the complexity tier, falling back to the existing regex/keyword scorer on any error, empty response, or unparseable output. * feat(ui): add classifier_type option to complexity router UI, fix edit flow Adds an "Advanced: Classification Method" section to ComplexityRouterConfig with a heuristic/LLM toggle, revealing a classifier model picker and timeout when LLM is selected. Also fixes the auto router edit modal, which never rendered the complexity router UI at all (it only handled the semantic router), and the "Edit Auto Router" button visibility check, which was gated on auto_router_config and never matched complexity router deployments. * fix(router): attribute classifier calls to caller, raise default timeout Forwards the original request's litellm_metadata into the classifier's acompletion call. Without it, the proxy's cost-tracking gate sees no user_api_key/team_id/user_id and silently drops spend logging and budget accounting for every classifier call, letting an authenticated user rack up unaccounted provider spend via repeated requests. Also raises the default classifier timeout from 400ms to 3000ms (400ms undershoots real LLM latency and would silently degrade to the heuristic scorer on most requests) and corrects the module/class docstrings, which still claimed zero external API calls after the llm classifier path was added. * fix(ci): resolve ruff strict-budget and frontend-lint failures - Use PEP 585 generics (dict/tuple/list) in the new aclassify/_classify_with_llm signatures instead of typing.Dict/Tuple/List, and suppress BLE001 on the intentionally broad except in aclassify's fallback path with a reason. - Fix prettier formatting in ComplexityRouterConfig.tsx. - Regenerate eslint-metrics.json (was stale after the classifier UI changes). * fix(ci): regenerate stale eslint-metrics.json * fix(router): strip parent budget reservation from classifier metadata The classifier's internal acompletion call previously forwarded the parent request's full litellm_metadata, including its budget reservation (user_api_key_budget_reservation / user_api_key_auth). That reservation belongs to the routed completion the classifier is deciding on, not to the classifier call itself, so it's now stripped while key/team attribution fields are still forwarded for spend logging. |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||