- one table (litellm/router_endpoint_table.py) names each factory-built endpoint, its host op, call type and dispatch; both cores bind and dispatch from it, and an unknown call type is refused when bound
- the endpoints that resolved to None without a call now dispatch like their siblings; sync create/get/delete/cancel_interaction answer synchronously
- sync search picks among the router's search tools like asearch
- ~80 endpoints bound to litellm functions (and agenerate_content's overwritten binding, avector_store_create) now go through host.invoke
- routing decisions key on the endpoint's call type, never the host function's __name__
- arerank no longer sends the builtin input; responses get/cancel/delete/list with a plain id and no model call the handler directly
- prompt-management groups work without a litellm_call_id or logging object
Adds the litellm-router-core crate, its step-driven PyO3 bridge
(RouterCoreHandle / RouterCall) and RustCore, served behind LITELLM_RUST
through the ROUTER route rule. The host call-out is the only boundary:
every attempt goes through Host::invoke, there is no native executor path,
and RustCore never builds, wraps or forwards to PythonCore, so
router_python_core.py stays deletable. Attempt preparation lives in
litellm/router_attempt.py so it outlives router_python_core.py.
RustCore matches PythonCore's behavior, including its known bugs (fixed in
follow-up PRs). Routing decisions (strategy picks, routing groups, PTU,
pre-routing, pre-call checks, fallback walk, health routing, access groups,
weights, cooldowns, allowed fails, usage) live in the crate; work that needs
live Python objects (request shaping, endpoints, streaming, model info
facts, callbacks, alerting, prompt management) lives in host-side
litellm/router_rust_*.py modules driven by Rust steps.
The crate owns the registry. It runs the deployment build loop (secret
references, ids, organization fan-out, install order) and asks the host
for what needs Python, one deployment at a time so errors surface where
Python raised them: secrets, validate, provider and added steps. It holds
the model list, the index maps and the pattern routers with the Python
core's history rules (host_index.rs), and answers the registry lookups
(lookup.rs). The Python views of that state (model_list, index(), names(),
patterns()) read the handle, and every write to them becomes a changeset.
Model info, deployment model info, configured limits and model group info
are merged in the crate (model_info.rs, group_info.rs) from catalog rows
and facts the host answers on a model_info step asked at query time, so
price reloads and register_model stay visible. The usage cache key format
is defined once, in the crate.
tests/unit/test_router/core characterizes PythonCore's public behavior and
runs every test on both cores; the remaining Rust differences (the router's
own outcome callbacks) are listed in rust_parity_pending.txt.
* refactor(router): move litellm/router.py to litellm/router_python_core.py
A pure rename with no content change, so git and review tools track the router body as a move.
The next commit turns it into PythonCore and adds the Router facade back at litellm/router.py
* refactor(router): extract PythonCore behind RouterCore and pass the router its config
The routing logic moves unchanged from litellm/router.py into PythonCore in
litellm/router_python_core.py. Router becomes a facade that builds a RouterConfig, picks a
RouterCore with make_core and delegates every member to it. The core no longer reads or writes
litellm.* directly: process settings come from RouterDynamicConfig, and inference calls, sleeps,
callback lists and the global cache go through PythonHost. A CI check bans litellm.* in the router
modules, and the proxy's construction sites build the config outside the router
Settings are static: the constructor arguments and changeset values are copied when the router is
built or the change applied, and a write to a setting attribute becomes apply(ChangeSet(assign=...)).
A Router subclass's core class also subclasses the facade, so an override's super(Sub, self) works,
and an instance-level patch of a core method is kept on the facade so nested patches unwind.
tests/unit/test_router/core tests the core in isolation: ScriptedHost serves every host operation
(scripted inference answers, recorded sleeps, settings, callbacks, cache), and make_core runs each
test once per core in CORE_BUILDERS
Code outside the router now reaches it only through the entry points
(registry, settings, health, select, apply, observe, register_plugin, retire)
or the endpoint methods; the outside-member allowlist shrinks from 139 to 45.
litellm/router.py no longer imports litellm.proxy: it asks the installed
RouterHost (litellm/router_host.py) for vertex pass-through credentials,
member auto-router authorization and auto-router prompt compression, and the
proxy installs its host on import. Call sites where a model name can be
missing keep the old answer instead of reaching an entry point with None.
Router gains eight entry points (registry, settings, health, select, apply/aapply,
observe, register_plugin, retire) in litellm/router_api.py. The 123 public members
outside code used become DeprecatedRouterMembers (litellm/router_deprecated.py):
same signatures and results, a DeprecationWarning naming the replacement and a use
counter for callers outside the router. Router internals move to _core_ names;
subclass overrides of old members are still called by the router.
Add rustdoc overviews for a new litellm-router-core crate and for
litellm-model-catalog, a module docstring on litellm/router.py, and a
shrink-only allowlist of the Router members and router_utils modules used
outside the router, checked in CI by router_surface_check.py.
Move the router-free helpers out of litellm/router_utils: batch_utils to
litellm_core_utils, and the auto-router naming contract, tuning baseline
and baseline request next to the auto-router plugin. No behaviour change.
