litellm/ui
Tin Chi Lo 9b0ffd564c fix(auto-router): advertise the context window an auto-router can serve
An auto-router deployment is a marker whose litellm_params.model is absent from
the cost map, so every source the model-info paths consult answers nothing and
the router advertises no context window at all. Clients that budget against the
advertised window then overflow and take an upstream context_length_exceeded 400
that no fallback catches, since the fallback map is keyed on model groups rather
than on the router.

Derive it from the tier model groups instead. Widest across the groups, because
context-window escalation moves a prompt off a tier that cannot hold it onto one
that can before dispatch, so the wide end is what the router serves; smallest
within each group, because the core router picks a deployment inside a group with
no such fit check. A group whose window is unresolvable is never escalated onto,
so it cannot raise the maximum and its presence makes the answer a guess: report
nothing. An operator-set model_info.max_input_tokens still wins everywhere.

Covers both model-info paths: get_model_listing_info for /v1/models, and
get_litellm_model_info for /model/info, /v1/model/info and /v2/model/info.
2026-09-10 21:30:09 -07:00
..
litellm-dashboard fix(auto-router): advertise the context window an auto-router can serve 2026-09-10 21:30:09 -07:00
Dockerfile fix(docker): bump nginx runtime to 1.31.5-alpine3.24 and pin digest to resolve critical CVEs (#39561) 2026-09-03 08:46:07 -07:00
nginx.conf fix(ui): boot the UI image as an arbitrary uid by anchoring nginx writes under /tmp (#37982) 2026-08-24 11:57:36 -07:00