scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-29 06:33:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 199
litellm/tests/unit/test_router
History
Kerry Lu d64ac3998c feat(router): let a deployment set a timeout per service_tier (#47406) 
* feat(router): let a deployment set a timeout per service_tier

Adds timeout_by_service_tier and stream_timeout_by_service_tier to a deployment's
litellm_params. A request whose service_tier matches a key uses that value in place
of the deployment's timeout or stream_timeout. A timeout the client sends still wins
and requests without a matching tier are unchanged

Co-authored-by: adc77 <amandogra2016@gmail.com>

* style(router): sort the request_timeout_resolver import

* fix(router): read the service tier maps without the untyped params get

* fix(router): look the service tier timeout up through a typed get

---------

Backport of #47406 to stable/1.104.x
Cherry-picked from 62ce515d15 (main)

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: adc77 <amandogra2016@gmail.com>
2026-10-20 23:56:28 +00:00
..
__init__.py test: add __init__.py to every tests/unit directory phase 16 touches 2026-09-20 11:52:38 +00:00
test_enforce_model_rate_limits.py test: migrate phase 16 legacy tests to tests/unit 2026-09-20 10:59:02 +00:00
test_io_token_rate_limits.py test: migrate phase 16 legacy tests to tests/unit 2026-09-20 10:59:02 +00:00
test_router.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_router_timeout.py feat(router): let a deployment set a timeout per service_tier (#47406) 2026-10-20 23:56:28 +00:00