Nate Armstrong 16dacb02b7 feat(gateway): model capacity hook, the proxy's dynamic rate limiter (#47055) ... Adds the model_capacity hook, turned on by listing dynamic_rate_limiter_v3 (or model_capacity) in litellm_settings.callbacks. It caps each model group at the sum of its deployments' rpm and tpm across every replica, and with priority_reservation holds each priority's pool to its share once the group's saturation reaches the threshold, counting it below. It uses the proxy's model_saturation_check and priority_model keys and scripts, its refusal messages and headers, and adds a successful response's tokens to the group and the pool admission chose. The rate limit counters gain a general check-and-add over request and token pairs, and the throttling refusal shape is shared. Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5 Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>		2026-10-19 09:58:45 -07:00
..
adrs	docs(rust): add ADRs for the Rust core (#45205)	2026-10-08 10:21:56 -07:00
customer-proxy	feat(gateway): model capacity hook, the proxy's dynamic rate limiter (#47055)	2026-10-19 09:58:45 -07:00