mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
2d3c3e3098
Shadow eval only answered "should this key adopt this auto-router". Once a key is on the router it is invisible to the feature, because the sampling gate skips any request the shadowed router already served, so post-adoption quality regressions go unmeasured. Reverse mode inverts the arms: sample the traffic the router did serve and duplicate it against a fixed baseline_model, judged by the same blind pairwise judge. Same job table, same attempt rows, same aggregates. real_* stays the arm the caller was served and shadow_* the duplicated one, so in reverse real_model is the router's pick and shadow_model is the baseline. The active-job slot becomes one per (key, direction) so both directions can run at once, and tier attribution in reverse reads the control request's routing decision rather than the shadow call's write-back. |
||
|---|---|---|
| .. | ||
| dist | ||
| litellm_proxy_extras | ||
| tests | ||
| build_and_publish.md | ||
| LICENSE | ||
| migration_runbook.md | ||
| pyproject.toml | ||
| README.md | ||
Additional files for the proxy. Reduces the size of the main litellm package.
Currently, only stores the migration.sql files for litellm-proxy.
To install, run:
uv add litellm-proxy-extras
OR
uv tool install 'litellm[proxy]' # installs litellm-proxy-extras and other proxy dependencies
To use the migrations, run:
litellm --use_prisma_migrate