mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-15 23:31:29 +00:00
9f9ae2b718
A shadow eval samples ongoing traffic, so a job without an end date keeps billing judge calls until someone remembers to stop it — and the upfront estimate silently priced exactly one week regardless. Jobs now take a duration_days (1-30, default 7): the start endpoint stamps ends_at, the estimate scales trailing volume to the requested window, and the logger completes a job past its window through the same guarded update + cache eviction path as the spend cap (generalized into _finalize_job). The existing shadow eval migration is amended in place since it has not shipped anywhere yet. The start form no longer asks anyone to paste a key hash: the key is a type-to-search combobox backed by /key/list alias substring search that submits the token, the auto-router is a filter-as-you-type combobox fed by the configured auto-router deployments, and duration is a select. Active job cards show when the job will end. The judge model field is now labelled as such, with guidance: judging two answers blind needs solid comprehension and reliable JSON, not frontier reasoning — a mid-tier model (Claude Sonnet / GPT-4o class) is recommended, nano/mini-class judges give unreliable verdicts, and frontier reasoning models add cost without changing outcomes. Same guidance mirrored into the API field description. Co-Authored-By: Claude <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||