scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-26 01:12:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 98

docs(complexity_router): document user_turn pin precedence over the context threshold

Browse source 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Devin AI 2026-09-25 15:42:42 +00:00
parent 4e1838e976
commit 4f3a126f06
1 changed files with 7 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
litellm/router_strategy/complexity_router/README.md
View file

@ -516,6 +516,13 @@ exceeds the limit. The estimate counts all message text at approximately four ch
so a short newest nudge in a long agentic session still reaches the classifier. Leave it unset to
keep the scorer's tier in control at any conversation size
The context veto only applies on turns that are classified, so it never overrides a held pin. With
`classification_mode: user_turn`, a continuation turn inside a session that already holds a pin
replays that pin (`x-litellm-complexity-router-cause: user_turn_continuation`) with no classifier
call, however large the conversation has grown. The threshold applies again on the next human ask,
which falls through to classification and, when the conversation exceeds the limit, goes to the LLM
classifier. With `session_affinity` on, the pin wins for new asks as well
The signal requirement is what keeps this from quietly routing everything to your cheapest model.
A prompt where no dimension fires scores exactly 0.0, which is below `simple_medium`, so the score
to tier mapping calls it SIMPLE by default rather than by evidence. Around half of general traffic