scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-12 03:48:56 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 57

fix(proxy): wait 5 min after health before auto-benchmark (worker init)

Browse source 
Default LITELLM_AUTO_BENCHMARK_STABILIZE_SECONDS changed from 15s to 300s (5 min) so all workers have time to initialize before benchmark starts.
This commit is contained in:
Alexsander Hamir 2026-02-02 14:25:25 -08:00
parent 69a65b4430
commit dacb1d7fde
1 changed files with 5 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
litellm/proxy/proxy_cli.py
View file

@ -212,7 +212,11 @@ class ProxyInitializationHelpers:
else:
print("LiteLLM: Auto-benchmark timed out waiting for proxy health", file=sys.stderr)
return
# Proxy is up: run benchmark (single run or RPS suite)
# Wait for workers to finish initializing (simple fixed delay after first health 200)
stabilize_seconds = float(os.getenv("LITELLM_AUTO_BENCHMARK_STABILIZE_SECONDS", "300"))
if stabilize_seconds > 0:
time.sleep(stabilize_seconds)
# Proxy and workers are up: run benchmark (single run or RPS suite)
from datetime import datetime
mock_base, _ = ProxyInitializationHelpers._get_mock_chat_endpoint()