Nate Armstrong 57ea9f5ce4 feat(router): load balance, retry and cool down across deployments of a model group (#46817) ... Deployments sharing a model name now form a model group instead of overwriting each other. router_settings drive selection (simple-shuffle weighted by weight, rpm or tpm, or least-busy), num_retries on a different deployment for connect errors, 408, 429 and 5xx received before any byte reaches the client, and allowed_fails / cooldown_time cooldown with fallback to cooling deployments when no other is left. Per-deployment health is lock-free atomics. /v1/messages (Anthropic and Bedrock deployments) and /v1/chat/completions (native OpenAI passthrough and the core chat route) route through it, and the metrics label records the deployment each attempt used. Other endpoints pick one deployment per request without retries.		2026-10-16 12:43:01 -07:00
..
adrs	docs(rust): add ADRs for the Rust core (#45205)	2026-10-08 10:21:56 -07:00
customer-proxy	feat(router): load balance, retry and cool down across deployments of a model group (#46817)	2026-10-16 12:43:01 -07:00