scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-29 06:33:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 200
litellm/tests/unit/proxy/pass_through_endpoints
History
Kerry Lu 1ae27f3587 feat(router): let a deployment set a timeout per service_tier (#47406) 
* feat(router): let a deployment set a timeout per service_tier

Adds timeout_by_service_tier and stream_timeout_by_service_tier to a deployment's
litellm_params. A request whose service_tier matches a key uses that value in place
of the deployment's timeout or stream_timeout. A timeout the client sends still wins
and requests without a matching tier are unchanged

Co-authored-by: adc77 <amandogra2016@gmail.com>

* style(router): sort the request_timeout_resolver import

* fix(router): read the service tier maps without the untyped params get

* fix(router): look the service tier timeout up through a typed get

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: adc77 <amandogra2016@gmail.com>

Backport of #47406 to stable/1.106.x
Cherry-picked from 62ce515d15 (main)
2026-10-20 23:54:26 +00:00
..
llm_provider_handlers feat(decisions): accept decisions as a model mode alongside evaluation (#45474) 2026-10-15 06:21:36 +00:00
__init__.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_carry_guardrail_logging_info.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_deepgram_ws_passthrough_routes.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_llm_pass_through_endpoints.py feat(typesafe): support Jev 1.14 images, box questions, and long-context headers (#46596) 2026-10-15 06:32:44 +00:00
test_llm_passthrough_endpoints.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_managed_id_rewriter.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_method_specific_routing.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_pass_through_endpoints.py feat(router): let a deployment set a timeout per service_tier (#47406) 2026-10-20 23:54:26 +00:00
test_passthrough_auth_default.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_passthrough_endpoint_router.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_passthrough_endpoints_common_utils.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_passthrough_guardrail_block_otel_span.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_passthrough_guardrails.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_passthrough_guardrails_field_targeting.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_passthrough_post_call_guardrails.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_streaming_handler.py test: repair stale and polluting tests red on scheduled main CI (#44229) 2026-10-02 21:32:17 +00:00
test_streaming_handler_interrupt.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_success_handler.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_upstream_usage_headers.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_vertex_ai_batch_passthrough.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_vertex_ai_live_passthrough.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_vertex_passthrough_load_balancing.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_watsonx_proxy_route.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00