scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-19 00:01:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 87

docs(router): distinguish selected profiles from family controls

Browse source
This commit is contained in:
Tin Chi Lo 2026-09-17 07:29:10 -04:00
parent 4ea30b4316
commit 60d8fbe755
2 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
cookbook/auto_router_selective_training/PRIMARY_FINDINGS.md
View file

@ -1,6 +1,6 @@
# Primary findings: quality retained versus money saved
Training improved some comparisons, but it did not approach the ideal Luna-first policy across all 125 held-out tasks. The profiles selected on validation do not combine preservation of every Sol success with lower total cost across all four benchmarks
Training improved some comparisons, but it did not approach the ideal Luna-first policy across all 125 held-out tasks. The main profiles selected on validation do not combine preservation of every Sol success with lower total cost across all four benchmarks
Sol solved 87/125 at $20.8964. Luna solved 84/125 at at least $2.6531. There were 11 Sol-only successes, eight Luna-only successes, 76 successes shared by both, and 30 tasks both attempts failed. A perfect Luna-first decision would therefore reach 95/125 at at least $5.2653. That is an unattainable hindsight reference, with at most 74.8% savings using the recorded bills

2
cookbook/auto_router_selective_training/README.md
View file

@ -34,6 +34,6 @@ Costs use recorded gateway bills. A ≥ cost and ≤ savings mark unmetered tran
Costs include the classifier. These are task-pinned replays of fresh independent attempts, with one attempt per model per task. No guarantee of quality retention follows from a 125-task sample. The aggregate combines different benchmarks; inspect each benchmark before using a profile
The per-model control is an exploratory comparison among forty prespecified variants. No validation-selected profile combines preservation of all Sol successes and lower cost across all 125 tasks. See PRIMARY_FINDINGS.md for the interpretation
The per-model control is an exploratory comparison among forty prespecified variants. None of the main selected profiles combines preservation of all Sol successes and lower cost across all 125 tasks. See PRIMARY_FINDINGS.md for the interpretation
![Observed quality and known inference cost](quality_cost.png)