scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-29 06:33:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 200
litellm/tests/unit/test_router
History
Kerry Lu 1ae27f3587 feat(router): let a deployment set a timeout per service_tier (#47406) 
* feat(router): let a deployment set a timeout per service_tier

Adds timeout_by_service_tier and stream_timeout_by_service_tier to a deployment's
litellm_params. A request whose service_tier matches a key uses that value in place
of the deployment's timeout or stream_timeout. A timeout the client sends still wins
and requests without a matching tier are unchanged

Co-authored-by: adc77 <amandogra2016@gmail.com>

* style(router): sort the request_timeout_resolver import

* fix(router): read the service tier maps without the untyped params get

* fix(router): look the service tier timeout up through a typed get

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: adc77 <amandogra2016@gmail.com>

Backport of #47406 to stable/1.106.x
Cherry-picked from 62ce515d15 (main)
2026-10-20 23:54:26 +00:00
..
__init__.py test: add __init__.py to every tests/unit directory phase 16 touches 2026-09-20 11:52:38 +00:00
test_enforce_model_rate_limits.py test: migrate phase 16 legacy tests to tests/unit 2026-09-20 10:59:02 +00:00
test_io_token_rate_limits.py test: migrate phase 16 legacy tests to tests/unit 2026-09-20 10:59:02 +00:00
test_router.py fix(router): drop the encrypted reasoning a fallback hop's target cannot decrypt (#45393) 2026-10-08 17:30:01 -07:00
test_router_callback_hook_sequence.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_router_endpoints.py fix(vector-stores): keep proxy context out of provider vector store metadata (#47021) 2026-10-19 22:43:02 -07:00
test_router_fallbacks.py test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
test_router_get_deployments.py test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
test_router_helper_utils.py test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
test_router_provider_endpoints.py test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs (#45298) 2026-10-08 02:37:55 -07:00
test_router_retries.py test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
test_router_timeout.py feat(router): let a deployment set a timeout per service_tier (#47406) 2026-10-20 23:54:26 +00:00
test_router_vector_store_endpoints.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00