scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-29 06:33:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 267
litellm/litellm
History
Kerry Lu d64ac3998c feat(router): let a deployment set a timeout per service_tier (#47406) 
* feat(router): let a deployment set a timeout per service_tier

Adds timeout_by_service_tier and stream_timeout_by_service_tier to a deployment's
litellm_params. A request whose service_tier matches a key uses that value in place
of the deployment's timeout or stream_timeout. A timeout the client sends still wins
and requests without a matching tier are unchanged

Co-authored-by: adc77 <amandogra2016@gmail.com>

* style(router): sort the request_timeout_resolver import

* fix(router): read the service tier maps without the untyped params get

* fix(router): look the service tier timeout up through a typed get

---------

Backport of #47406 to stable/1.104.x
Cherry-picked from 62ce515d15 (main)

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: adc77 <amandogra2016@gmail.com>
2026-10-20 23:56:28 +00:00
..
a2a_protocol Merge remote-tracking branch 'origin/main' into litellm_decrease_anys_opus5_r5 2026-09-21 12:57:38 -07:00
anthropic_interface feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391) 2026-09-21 19:17:18 -07:00
assistants
batch_completion
batches feat(vertex): native batch JSONL passthrough with cost tracking (#42810) 2026-09-24 12:35:34 -07:00
caching fix(caching): stamp provider on sync cache-hit logs so responses spend logs record provider (#42830) 2026-09-23 20:49:23 -05:00
chat_completions feat(rust-bridge): add cache and secret migration foundations (#42328) 2026-09-22 03:41:04 +00:00
completion_extras feat(lint): add LIT013 flagging *-ok suppressions that suppress nothing and remove the 240 stale ones (#42793) 2026-09-23 17:50:09 -07:00
compression merge: main into litellm_headroom_protect_cached_prefix 2026-09-15 04:12:28 +00:00
containers chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
decisions fix(decisions): log every request field the caller sent on /v1/systemone and /v1/decisions (#47183) 2026-10-20 23:54:07 +00:00
embeddings feat(embeddings): add native dispatch foundation (#42799) 2026-09-23 21:11:09 +00:00
endpoints/speech/speech_to_completion_bridge
evals
experimental_mcp_client refactor: daily fresh tech debt cleanup, rolling PR (#42710) 2026-09-24 04:17:23 -07:00
files feat(vertex): native batch JSONL passthrough with cost tracking (#42810) 2026-09-24 12:35:34 -07:00
fine_tuning
google_genai fix(google_genai): drop non-object tool parameters instead of forwarding them 2026-09-19 19:19:33 -07:00
images Merge branch 'main' into litellm_add_edenai_provider 2026-09-21 19:34:49 +00:00
integrations fix(proxy)!: count endpoints return errors in their source API shape instead of silent local counts (#46815) 2026-10-19 12:35:32 -07:00
interactions feat(lint): add LIT013 flagging *-ok suppressions that suppress nothing and remove the 240 stale ones (#42793) 2026-09-23 17:50:09 -07:00
litellm_core_utils feat(router): let a deployment set a timeout per service_tier (#47406) 2026-10-20 23:56:28 +00:00
llms fix(count_tokens): narrow stable backport to the October 19 patch 2026-10-19 14:03:42 -07:00
messages chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
models fix(vector_stores): keep config-defined vector stores listed and read-only (#42574) 2026-09-23 04:02:16 +00:00
ocr refactor(ocr): remove the Python OCR execution path and require the Rust route (#43081) 2026-09-24 18:18:50 -07:00
passthrough feat(router): let a deployment set a timeout per service_tier (#47406) 2026-10-20 23:56:28 +00:00
proxy fix(proxy): restore legacy token count input precedence 2026-10-19 17:50:09 -07:00
proxy_auth refactor(types): replace Any with real types across 29 more backend files 2026-09-08 10:14:07 +00:00
rag fix(types): read upstream headers through a typed helper 2026-09-21 13:16:58 -07:00
realtime_api refactor(types): replace Any with proven types in 13 files (#42937) 2026-09-24 09:02:02 -07:00
repositories Merge remote-tracking branch 'origin/main' into litellm_decrease_anys_opus5_r5 2026-09-21 12:57:38 -07:00
rerank_api feat(lint): add LIT013 flagging *-ok suppressions that suppress nothing and remove the 240 stale ones (#42793) 2026-09-23 17:50:09 -07:00
responses refactor(types): replace Any with proven types in 13 files (#42937) 2026-09-24 09:02:02 -07:00
router_strategy feat(ui): backport deployed decision router setup to 1.104 2026-10-10 19:05:45 -07:00
router_utils feat(ui): backport deployed decision router setup to 1.104 2026-10-10 19:05:45 -07:00
rust_bridge chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
sandbox
search
secret_managers chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
skills feat(skills): semantic search over the LiteLLM-hosted skill registry (#39401) 2026-09-07 12:28:38 -07:00
systemone fix(decisions): log every request field the caller sent on /v1/systemone and /v1/decisions (#47183) 2026-10-20 23:54:07 +00:00
types feat(router): let a deployment set a timeout per service_tier (#47406) 2026-10-20 23:56:28 +00:00
vector_store_files refactor(types): replace Any with real types across 54 more backend files 2026-09-08 12:41:07 +00:00
vector_stores fix(vector_stores): keep config-defined vector stores listed and read-only (#42574) 2026-09-23 04:02:16 +00:00
videos
__init__.py feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape (#45846) 2026-10-11 01:02:56 +00:00
_internal_context.py fix(proxy): price one cost estimate at one moment 2026-09-07 18:38:12 -07:00
_lazy_imports.py feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
_lazy_imports_registry.py feat(azure_ai): support Microsoft-Decision-1 on the decisions API (#45673) 2026-10-11 01:02:00 +00:00
_logging.py feat(lint): add LIT013 flagging *-ok suppressions that suppress nothing and remove the 240 stale ones (#42793) 2026-09-23 17:50:09 -07:00
_redis.py fix(redis): authenticate sync clusters with IAM credential providers (#40204) 2026-09-23 17:23:55 -05:00
_redis_credential_provider.py fix(redis): accept every truthy flag and sign serverless ElastiCache caches 2026-09-10 10:13:03 -04:00
_service_logger.py
_uuid.py
_version.py
anthropic_beta_headers_config.json feat(router): native compact-to-fit across conversation APIs (#42074) 2026-09-21 22:52:29 -07:00
anthropic_beta_headers_manager.py fix: drop a blank anthropic-beta header before it reaches the provider 2026-09-19 20:13:59 -07:00
blog_posts.json
budget_manager.py
constants.py fix(count_tokens): omit local PDF parsing from stable backport 2026-10-19 15:20:52 -07:00
cost.json
cost_calculator.py feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape (#45846) 2026-10-11 01:02:56 +00:00
exceptions.py fix(spend): return 400 from /spend/calculate for a model with no pricing row (#42497) 2026-09-22 15:44:00 -07:00
main.py feat(providers): add Nadir intelligent-router provider (nadir/auto) (#33227) 2026-09-24 22:03:30 -07:00
model_prices_and_context_window_backup.json feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777) 2026-10-19 12:35:33 -07:00
policy_templates_backup.json
provider_endpoints_support_backup.json Merge pull request #41101 from hMED22/litellm_add_edenai_provider 2026-09-21 16:16:28 -05:00
py.typed
router.py feat(router): let a deployment set a timeout per service_tier (#47406) 2026-10-20 23:56:28 +00:00
scheduler.py
setup_wizard.py feat(anthropic): add Claude Opus 5.5 (#42489) 2026-09-22 09:52:35 -07:00
timeout.py
utils.py fix(decisions): log every request field the caller sent on /v1/systemone and /v1/decisions (#47183) 2026-10-20 23:54:07 +00:00