mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-26 06:03:19 +00:00
62ce515d15
* feat(router): let a deployment set a timeout per service_tier Adds timeout_by_service_tier and stream_timeout_by_service_tier to a deployment's litellm_params. A request whose service_tier matches a key uses that value in place of the deployment's timeout or stream_timeout. A timeout the client sends still wins and requests without a matching tier are unchanged Co-authored-by: adc77 <amandogra2016@gmail.com> * style(router): sort the request_timeout_resolver import * fix(router): read the service tier maps without the untyped params get * fix(router): look the service tier timeout up through a typed get --------- Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: adc77 <amandogra2016@gmail.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_enforce_model_rate_limits.py | ||
| test_io_token_rate_limits.py | ||
| test_router.py | ||
| test_router_callback_hook_sequence.py | ||
| test_router_endpoints.py | ||
| test_router_fallbacks.py | ||
| test_router_get_deployments.py | ||
| test_router_helper_utils.py | ||
| test_router_max_parallel_requests.py | ||
| test_router_provider_endpoints.py | ||
| test_router_retries.py | ||
| test_router_timeout.py | ||
| test_router_vector_store_endpoints.py | ||