scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102

fix(spend): resolve router kind in the order the read path resolves it

Browse source 
auto_router_kind checked quality before adaptive while
classify_strategy_router_model, which the benchmarks read path labels groups
with, checks adaptive before quality. A deployment registered under both would
have been labelled one way on the dashboard and folded the other way into the
rows behind it, which is the disagreement the function's own docstring says must
not happen.

The order now mirrors the read path's prefix resolution exactly, so the two
derivations cannot drift apart on a group that lands in more than one registry
This commit is contained in:
Tin Chi Lo 2026-08-04 09:06:23 -07:00
parent 2975645703
commit 7d902b387c
1 changed files with 8 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
litellm/proxy/spend_tracking/auto_router_sessions.py
View file

@ -302,17 +302,19 @@ def auto_router_kind(router: "Router", model_group: str) -> StrategyRouterKind |
classifier sub-calls share the session but carry the judge model's group, so
keying on it folds one turn per routed request and no classifier noise.
A complexity router with ``adaptive`` enabled owns an entry in both
registries, so complexity is answered first; that is the kind
``auto_router_group_kinds`` reports for the same deployment, and the two have
to agree or the dashboard would label the group one way and the rows another.
One deployment can own an entry in more than one registry; a complexity
router with ``adaptive`` enabled is in both. The order below is therefore the
order ``classify_strategy_router_model`` resolves its prefixes in, which is
what ``auto_router_group_kinds`` labels the same group with on the read side.
The two have to agree, or the dashboard would label a group one way and the
rows it aggregates another.
"""
if model_group in router.complexity_routers:
return "complexity"
if model_group in router.quality_routers:
return "quality"
if model_group in router.adaptive_routers:
return "adaptive"
if model_group in router.quality_routers:
return "quality"
if model_group in router.auto_routers:
return "semantic"
return None