mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
b83353906b
Adds a built-in budgets hook that runs the proxy's budget checks, in its order and with its messages, on the proxy's Redis spend:* counters. Each request reads and seeds the counters, checks them and reserves its maximum cost in one Redis script, then settles the reservation to its real cost off the request path, so replicas and Python proxies sharing the Redis enforce one budget. Without Redis the counters are kept per replica. The key store now reads user, team member and end-user budgets, and auth no longer checks budgets against stored spend. Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5 Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| bedrock_messages-flow.svg | ||
| budgets.svg | ||
| hooks-pipeline.svg | ||
| loadtest-rig.svg | ||
| openai_chat-flow.svg | ||
| ops-drain.svg | ||
| ops-metrics.svg | ||
| routing-cooldown.svg | ||
| routing-request-flow.svg | ||
| shared_state-redis.svg | ||
| spend-writer.svg | ||
| virtual_keys-auth-flow.svg | ||
