litellm/ui
Kerry Lu 1ae27f3587 feat(router): let a deployment set a timeout per service_tier (#47406)
* feat(router): let a deployment set a timeout per service_tier

Adds timeout_by_service_tier and stream_timeout_by_service_tier to a deployment's
litellm_params. A request whose service_tier matches a key uses that value in place
of the deployment's timeout or stream_timeout. A timeout the client sends still wins
and requests without a matching tier are unchanged

Co-authored-by: adc77 <amandogra2016@gmail.com>

* style(router): sort the request_timeout_resolver import

* fix(router): read the service tier maps without the untyped params get

* fix(router): look the service tier timeout up through a typed get

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: adc77 <amandogra2016@gmail.com>

Backport of #47406 to stable/1.106.x
Cherry-picked from 62ce515d15 (main)
2026-10-20 23:54:26 +00:00
..
litellm-dashboard feat(router): let a deployment set a timeout per service_tier (#47406) 2026-10-20 23:54:26 +00:00
Dockerfile fix(docker): bump nginx runtime to 1.31.5-alpine3.24 and pin digest to resolve critical CVEs (#39561) 2026-09-03 08:46:07 -07:00
nginx.conf fix(ui): boot the UI image as an arbitrary uid by anchoring nginx writes under /tmp (#37982) 2026-08-24 11:57:36 -07:00