scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-12 08:42:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 207
litellm/litellm-rust/crates
History
Exact
narmstrong-berri d554f619d8 feat(router): add the Rust router core and serve routing through it behind LITELLM_RUST 
litellm-router-core implements the router core in Rust: a versioned registry with Python's
deployment ids, selection with native simple-shuffle, the retry, fallback and cooldown loop with
Python's cache keys, and per-attempt executors with a native seam for Messages to Anthropic. The
python bridge exposes it as RouterCoreHandle, whose call loop is step driven so Rust never awaits
Python. RustCore drives that loop for completions, messages, embeddings and responses and serves
everything else from a PythonCore built from the same config. A new router rule in the route
catalog opts in with LITELLM_RUST, and any config the Rust core cannot serve gets PythonCore at
construction

The call loop follows Python's fallback walk (per-target depth, a level raising its own error,
lookup by the current and requested group, "*" after specific chains), stamps retries the way an
exhausted retry loop does, and leaves health writes to the Python attempt, which records them.
RustCore builds its handle from the constructor arguments and the applied changesets and rebuilds
it when either version moves. The core unit suite runs on RustCore too
2026-10-30 17:36:09 -07:00
..
auth refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
auth-aws refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
auth-azure refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
auth-gcp refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
auth-types refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
cache refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
cache-azure-blob refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
cache-disk refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
cache-gcs refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
cache-memory refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
cache-qdrant-semantic refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
cache-redis refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
cache-redis-semantic refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
cache-response refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
cache-s3 refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
cache-testing refactor(rust): align the cache crates with Python and wire every native backend (#42530) 2026-09-22 13:13:02 -07:00
cache-valkey-semantic refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
callbacks-legacy-python refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
circuit-breaker feat(rust): add circuit-breaker crate and guard redis-client with it (#48277) 2026-10-26 18:41:47 -07:00
config refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
core-utils fix(exceptions)!: honor status code when mapping OpenAI invalid_request_error exceptions (#32537) 2026-10-29 08:05:47 -07:00
coroutine refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python (#46299) 2026-10-14 17:50:44 -07:00
cost refactor(model-catalog): parse, lower and validate the catalog into typed prices (#48853) 2026-10-28 18:24:40 -07:00
db feat(db): add typed daily end-user spend upsert (#48237) 2026-10-26 17:05:08 -07:00
db-testing feat(rust): add redis-client crate for standalone Redis (#48269) 2026-10-26 18:41:46 -07:00
error refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
framer refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
gateway refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
gateway-auth refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
gateway-inference refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
gateway-management refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
gateway-mcp refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
gateway-ui refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
host refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
host-http refactor(rust): centralize host execution and compose callbacks (#43515) 2026-09-28 19:20:27 +00:00
host-native feat(rust): one typed decline from core up to each host (#47772) 2026-10-28 09:22:11 -07:00
host-python feat(rust): one typed decline from core up to each host (#47772) 2026-10-28 09:22:11 -07:00
http refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
inference feat(rust): one typed decline from core up to each host (#47772) 2026-10-28 09:22:11 -07:00
inference-chat refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
inference-messages refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
inference-ocr refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
inference-responses refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
inference-testing refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
inference-transcription feat(rust): one typed decline from core up to each host (#47772) 2026-10-28 09:22:11 -07:00
litellm refactor(rust): expand logging and test coverage across gateway and Anthropic messages (#43295) 2026-09-26 08:04:19 +00:00
llms refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
llms-types fix(vertex_ai,azure_ai)!: auto-add the compaction, thinking display, and tool change betas on Claude routes (#44747) 2026-10-30 09:28:19 -07:00
model-catalog ci(model-catalog): check the cost map by loading it through model-catalog (#48908) 2026-10-29 07:55:02 -07:00
model-catalog-refresh refactor(model-catalog): parse, lower and validate the catalog into typed prices (#48853) 2026-10-28 18:24:40 -07:00
python-bridge feat(router): add the Rust router core and serve routing through it behind LITELLM_RUST 2026-10-30 17:36:09 -07:00
python-compat refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
redis-client refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
redis-testing feat(gateway): connect to Redis from config.yaml (#48284) 2026-10-26 18:41:47 -07:00
router fix(vertex_ai)!: send the caller's anthropic-version in the Vertex Claude /v1/messages body, not the header (#48814) 2026-10-28 18:06:58 -07:00
router-core feat(router): add the Rust router core and serve routing through it behind LITELLM_RUST 2026-10-30 17:36:09 -07:00
router-types refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
secret-value refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
secrets refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
secrets-aws refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
secrets-azure refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
secrets-cyberark refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
secrets-google refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
secrets-hashicorp refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
secrets-types refactor(rust): extract SecretValue and use it for Redis credentials (#49186) 2026-10-30 11:31:04 -07:00
spend-clickhouse refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
storage-clickhouse refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
testkit refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
token-counter refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
token-counter-fast refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
token-counter-huggingface refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
token-counter-tiktoken refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042) 2026-10-28 09:22:11 -07:00
tracing feat(rust): add structured route lifecycle tracing (#43466) 2026-09-27 19:18:58 -07:00