mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-11 08:38:24 +00:00
* feat(rust): add circuit-breaker crate and guard redis-client with it * fix(circuit-breaker): ignore stale probe drops --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1.8 KiB
1.8 KiB
fsmis a generic state machine runtime and knows nothing about breakers.breakeris a generic circuit breaker and knows nothing about Redis. Deciding which errors count as failures belongs to the consumerMachine::stepstays pure.&selfis config and time comes in on the event, never read inside the machine. The only clock read isAdmission's drop, at the call-site boundary- Transition rules are one
match (state, event)per machine, with no macro DSL and no_arm that would hide a new state from the compiler Sharedcounts a transition, bumps the generation and calls the observer only when the state's enum variant changes, matching Python bumpinggenerationonly when_statechanges. Data inside a variant (failure counters,since) changes without a transition- The observer runs under the lock so transitions are seen in generation order. Keep it to logging or a metric
- A disabled breaker is
Option<Breaker>::Noneat the consumer, not a flag inside the machine.Configcomes in already resolved, readingREDIS_CIRCUIT_BREAKER_*env is the consumer's job
Parity with Python RedisCircuitBreaker (litellm/caching/redis_cache.py)
- The rstest table
matches_the_python_redis_circuit_breakerwas checked case by case against the Python breaker. A behavior change here needs the same check, by driving the Python class with the same steps - Divergences, each on purpose
- monotonic
Instantinstead oftime.time(), so a wall clock jump can't open or close the breaker - a probe
Admissiondropped without an outcome records a hard failure and reopens. Python's probe that dies onCancelledError(aBaseException) records nothing and leaves the breaker HALF_OPEN for good, and a dropped Rust future is common _failure_countis dropped, it only fed a log line- no
enabledflag, see above
- monotonic