mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
2d25d23566
The heuristic scorer has no way to say "I don't know". Its seven dimensions are a roughly 100-word software-vocabulary whitelist, so when a prompt matches none of them every dimension contributes 0, the weighted sum is 0.0, and 0.0 sits below simple_medium (0.15). Absence of evidence was being scored as evidence of simplicity, and unclassifiable traffic went to the cheapest tier. On a graded 809-question benchmark that no-signal mass is 50.2% of prompts; on a 257-session agent transcript corpus it is 36.6% of turns _score_and_classify now returns config.default_tier before the band mapping when nothing was recognised, under its own cause=no_signal_default with signals=['no-signal'], so a spend log row says the score did not choose the tier, the way reasoning_override already does. default_tier is new on ComplexityRouterConfig and defaults to MEDIUM; setting it to SIMPLE restores the previous behavior exactly. The LLM classifier falls back to this scorer on timeout or error, so the same setting decides where unmatched traffic lands during a classifier outage The branch tests the signals rather than the score, and rather than the individual dimension scores. A weighted score of zero does not mean nothing was recognised, since contributions cancel: "hi, quick python question" comes to zero with three dimensions firing and is real evidence of a simple request. A dimension scoring zero does not mean it stayed silent either, because _score_keyword_match takes the no-match score as a parameter, so a future dimension with a nonzero baseline would silently kill the branch. A signal is the one thing a dimension emits only when it recognised something, and a test pins that invariant across every scorer An explicitly configured default_tier with no model behind it is rejected at load rather than surfacing as a routing error on the first unmatched request. The check is skipped when the field is left implicit, so a partial tiers map keeps loading as it does today instead of turning an upgrade into a startup failure The router e2e config pins default_tier: SIMPLE. That test tells a live LLM classifier apart from a silent fallback by which backend served the request, and its prompt has no scoring signal, so leaving the new default in place would have made both arms land on the same backend and turned the test into a false green |
||
|---|---|---|
| .. | ||
| a2a | ||
| access_control | ||
| batches | ||
| claude_code | ||
| coverage_registry | ||
| guardrails | ||
| llm_translation | ||
| load | ||
| logging | ||
| management | ||
| mcp | ||
| other | ||
| quota_management | ||
| router | ||
| ui | ||
| CLAUDE.md | ||
| conftest.py | ||
| CONTRIBUTING.md | ||
| e2e_config.py | ||
| e2e_db.py | ||
| e2e_http.py | ||
| junit_properties.py | ||
| lifecycle.py | ||
| models.py | ||
| otel_client.py | ||
| proxy_client.py | ||
| pytest.ini | ||
| transport.py | ||