mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
3238ce8406
* feat(auto-router): track turns per complexity tier (LIT-5302)
Stamps complexity tier at decision time (rollup never re-derives from routed
model, since tier->model mapping is mutable config). Records per-tier turn
counts in LiteLLM_AutoRouterSession.tier_turns (jsonb), rolls up per router
in benchmarks SQL via jsonb_object_agg, returns on AutoRouterBenchmarkGroup
for dashboard turns/share metrics.
Addresses Greptile/Bugbot findings:
- Missing _SessionAggRow.tier_turns field: added with field_validator to
parse jsonb text cast and handle NULL. Would 500 every benchmarks read.
- Missing ::text cast on tier parameter: Postgres fails type inference on
parameterized CASE/IS NULL without explicit cast. Added to all usages.
- Docstring false claim (only complexity routers produce tiers): quality
router stamps numeric tier '1'/'2'/'3'. Per-type grouping in SQL prevents
cross-contamination. Rewrote docstring to clarify isolation.
- Comment convention violations: stripped per CLAUDE.md rule.
- Test gaps: 8 unit tests for extraction/validation/aggregation, 7 behavior
tests for SQL semantics against real Postgres. 12 mutations killed.
Fixed fragile complexity_router test that broke on nested function calls.
No API change; extends existing GET /auto_router/benchmarks response only.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(auto-router): address review findings on tier turns tracking
- Guard router_type update so a mid-session reconfigure can't pool
foreign tier names into tier_turns
- Keep pinned turns attributed to the tier that actually serves them
- Drop stray -- AlterTable comment from hand-written migration
- Drop the now-unnecessary ::text/json.loads round-trip; prisma
already returns tier_turns as a parsed dict
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(auto-router): satisfy type-discipline lint gate
- tier_turns fields: dict[str, int] -> Mapping[str, int] (LIT001,
mutable collection in annotation); these are read-only after
construction
- _summed_agg_row: {} -> MappingProxyType({}) (LIT002, mutable dict
literal)
- default-fallback branch: replace the reassigned-without-Final
fallback_tier with a Final default_model_first flag and a single
ternary assignment (LIT010)
Verified locally: type_discipline_gate.py, ruff_strict_gate.py, and
type_check_gate.py all pass against the litellm_internal_staging
merge-base; full test_complexity_router.py (374), auto_router
management-endpoint tests (26), db-layer rollup tests (31), and the
live-Postgres proxy_behavior rollup suite (17) all pass.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| adaptive_router | ||
| test_auto_router.py | ||
| test_base_routing_strategy.py | ||
| test_budget_limiter_hotpath.py | ||
| test_complexity_router.py | ||
| test_lar1_routing.py | ||
| test_litellm_encoder.py | ||
| test_lowest_latency.py | ||
| test_quality_router.py | ||
| test_router_routing_groups.py | ||
| test_router_routing_plugins.py | ||
| test_router_tag_regex_routing.py | ||
| test_router_tag_routing.py | ||
| test_savings_baseline.py | ||