mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
9f9ae2b718
A shadow eval samples ongoing traffic, so a job without an end date keeps billing judge calls until someone remembers to stop it — and the upfront estimate silently priced exactly one week regardless. Jobs now take a duration_days (1-30, default 7): the start endpoint stamps ends_at, the estimate scales trailing volume to the requested window, and the logger completes a job past its window through the same guarded update + cache eviction path as the spend cap (generalized into _finalize_job). The existing shadow eval migration is amended in place since it has not shipped anywhere yet. The start form no longer asks anyone to paste a key hash: the key is a type-to-search combobox backed by /key/list alias substring search that submits the token, the auto-router is a filter-as-you-type combobox fed by the configured auto-router deployments, and duration is a select. Active job cards show when the job will end. The judge model field is now labelled as such, with guidance: judging two answers blind needs solid comprehension and reliable JSON, not frontier reasoning — a mid-tier model (Claude Sonnet / GPT-4o class) is recommended, nano/mini-class judges give unreliable verdicts, and frontier reasoning models add cost without changing outcomes. Same guidance mirrored into the API field description. Co-Authored-By: Claude <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| dist | ||
| litellm_proxy_extras | ||
| tests | ||
| build_and_publish.md | ||
| LICENSE | ||
| migration_runbook.md | ||
| pyproject.toml | ||
| README.md | ||
Additional files for the proxy. Reduces the size of the main litellm package.
Currently, only stores the migration.sql files for litellm-proxy.
To install, run:
uv add litellm-proxy-extras
OR
uv tool install 'litellm[proxy]' # installs litellm-proxy-extras and other proxy dependencies
To use the migrations, run:
litellm --use_prisma_migrate