scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-04 02:31:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 71

fix(router): score complexity by difficulty not request length

Browse source 
The LLM classifier prompt treated short wording as SIMPLE, so probes like
"Is P equal to NP?" stayed on the SIMPLE backend even though the classifier
ran. Judge intellectual difficulty so short hard questions route higher
This commit is contained in:
mubashir1osmani 2026-07-16 15:50:11 -07:00
parent a0cf03cb2a
commit a224dbb777
1 changed files with 6 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
litellm/router_strategy/complexity_router/complexity_router.py
View file

@ -56,11 +56,13 @@ class TierClassification(BaseModel):
_CLASSIFICATION_PROMPT_TEMPLATE = """Classify the complexity of the following user request into exactly one tier.
Judge the intellectual difficulty of answering correctly, not how short the request is.
Tiers:
- SIMPLE: factual lookups, greetings, short direct questions with no reasoning or code involved.
- MEDIUM: everyday requests needing some explanation or minor code/technical content.
- COMPLEX: requests involving non-trivial code, architecture, or multi-step technical work.
- REASONING: requests explicitly requiring step-by-step reasoning, analysis, or weighing tradeoffs.
- SIMPLE: greetings, chitchat, or factual lookups with a short known answer. Do not use SIMPLE for unsolved problems, proofs, deep theory, multi-step analysis, or non-trivial code, even if the request is only one sentence.
- MEDIUM: everyday requests that need some explanation, light reasoning, or minor code/technical content.
- COMPLEX: non-trivial code, architecture, multi-step technical work, or specialized domain depth.
- REASONING: open-ended analysis, proofs, famous hard problems, step-by-step reasoning, tradeoffs, or anything where a correct answer requires careful thought rather than a quick lookup.
{system_context}Request:
{prompt}"""