mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-13 03:59:12 +00:00
2164e5932d
Engine::route runs the loop Python's router runs for the first increment: resolve the model, drop cooling deployments, pick with simple-shuffle, retry within the group under should_retry_this_error or a retry policy, back off, fall back through generic, context-window and content-policy chains, cool deployments down and count usage. Every attempt is an invoke on a RouterHost, which keeps responses and exceptions opaque and answers with a Classified record of the error. Metadata and exception writes come back as ordered ops so the host can apply them where Python's readers look. State lives in a Store that writes the keys, Python str() encodings and TTLs Python writes, in memory and Redis, and PythonRandom reproduces CPython's Mersenne Twister so seeded runs pick identically. The gateway's Router/Deployment lookup is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Aw1xktpUnix5tWhvSzey8t |
||
|---|---|---|
| .. | ||
| .agents/skills | ||
| crates | ||
| .gitignore | ||
| AGENTS.md | ||
| Cargo.lock | ||
| Cargo.toml | ||
| clippy.toml | ||