mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-16 09:22:22 +00:00
An auto-router deployment is a marker whose litellm_params.model is absent from the cost map, so every source the model-info paths consult answers nothing and the router advertises no context window at all. Clients that budget against the advertised window then overflow and take an upstream context_length_exceeded 400 that no fallback catches, since the fallback map is keyed on model groups rather than on the router. Derive it from the tier model groups instead. Widest across the groups, because context-window escalation moves a prompt off a tier that cannot hold it onto one that can before dispatch, so the wide end is what the router serves; smallest within each group, because the core router picks a deployment inside a group with no such fit check. A group whose window is unresolvable is never escalated onto, so it cannot raise the maximum and its presence makes the answer a guess: report nothing. An operator-set model_info.max_input_tokens still wins everywhere. Covers both model-info paths: get_model_listing_info for /v1/models, and get_litellm_model_info for /model/info, /v1/model/info and /v2/model/info. |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||