mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
587f227b9d
* feat(complexity_router): heuristic-first classifier chaining Adds classifier_type 'heuristic_first', which scores locally on every request and only calls the LLM classifier for traffic the scorer could not place at or below heuristic_first_max_tier. A request short-circuits when the scorer landed at or below the threshold and produced at least one signal; everything else escalates. The signal requirement is load-bearing. A prompt where no dimension fires scores exactly 0.0, which is under simple_medium, so the score-to-tier mapping calls it SIMPLE by default rather than by evidence, and that is about half of general traffic. Gating on the tier alone would route it to the cheapest model without ever consulting the classifier. Introduces uses_llm_classifier as the single owner of 'does this router call the classifier model', replacing the classifier_type == 'llm' comparisons in the config validator, the prompt prebuild, the health dependency graph, the routing-test authorizer, and six dashboard sites. * fix(complexity_router): reuse the heuristic verdict on classifier failure, load the threshold on edit Three review findings, one push. The heuristic-first fallback re-scored the prompt after a classifier failure, which the README already documented as a reuse. The outcome computed before escalation is now handed to the failure path, so the scorer runs once per request. The edit modal never hydrated heuristic_first_max_tier, while save rebuilds every managed key from form state, so opening a heuristic-first router and saving it dropped a field the proxy requires. The dropdown's display fallback hid it. Both are fixed, and the hydration is extracted into a pure function so a test can pin the invariant: every managed key present in a stored config survives an untouched open-and-save. That test also covers every field added later. Classifier radio labels lost their em dashes, per the repo writing convention. |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||