mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-06 08:16:43 +00:00
A coding agent names each conversation by quoting the whole session and asking for a title. The classifier rated the quoted session rather than the request, so the cheapest call the client makes routed to the most expensive tier: 11 of 17 title generations in one day of real traffic came back COMPLEX. Recognize those prompts by literal sentinel on the newest ask and route them to the cheapest configured tier without classifying them, so the call costs nothing to route. The placement is scoped to the one request that carries the sentinel: it never displaces an operator's classifier plugin, the bandit cannot reach above the tier as raised, it never becomes the session pin, and the sentinel that matched is recorded on the routing decision. Detection reads the newest ask alone, so a title request quoted into a later turn cannot cheapen the work that follows it, and a keyword rule, an escalation keyword or the plan-mode floor all still decide over it. Regenerates the lazy OpenAPI snapshot, which was already stale on the base for an unrelated Presidio guardrail field and failed the schema check on every PR. Resolves LIT-6349 |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||