Commit graph

4 commits

Author SHA1 Message Date
Nate Armstrong
c43a1addf2
refactor(router): extract PythonCore behind RouterCore and pass the router its config (#49119)
* refactor(router): move litellm/router.py to litellm/router_python_core.py

A pure rename with no content change, so git and review tools track the router body as a move.
The next commit turns it into PythonCore and adds the Router facade back at litellm/router.py

* refactor(router): extract PythonCore behind RouterCore and pass the router its config

The routing logic moves unchanged from litellm/router.py into PythonCore in
litellm/router_python_core.py. Router becomes a facade that builds a RouterConfig, picks a
RouterCore with make_core and delegates every member to it. The core no longer reads or writes
litellm.* directly: process settings come from RouterDynamicConfig, and inference calls, sleeps,
callback lists and the global cache go through PythonHost. A CI check bans litellm.* in the router
modules, and the proxy's construction sites build the config outside the router

Settings are static: the constructor arguments and changeset values are copied when the router is
built or the change applied, and a write to a setting attribute becomes apply(ChangeSet(assign=...)).
A Router subclass's core class also subclasses the facade, so an override's super(Sub, self) works,
and an instance-level patch of a core method is kept on the facade so nested patches unwind.

tests/unit/test_router/core tests the core in isolation: ScriptedHost serves every host operation
(scripted inference answers, recorded sleeps, settings, callbacks, cache), and make_core runs each
test once per core in CORE_BUILDERS
2026-10-31 12:54:23 -07:00
Nate Armstrong
ead221995e
refactor(router): move router callers onto the entry points and invert the proxy imports (#48847)
Code outside the router now reaches it only through the entry points
(registry, settings, health, select, apply, observe, register_plugin, retire)
or the endpoint methods; the outside-member allowlist shrinks from 139 to 45.
litellm/router.py no longer imports litellm.proxy: it asks the installed
RouterHost (litellm/router_host.py) for vertex pass-through credentials,
member auto-router authorization and auto-router prompt compression, and the
proxy installs its host on import. Call sites where a model name can be
missing keep the old answer instead of reaching an entry point with None.
2026-10-29 17:26:33 +00:00
Nate Armstrong
a659e8be70
refactor(router): add router entry points and keep the old members as a deprecated mixin (#48840)
Router gains eight entry points (registry, settings, health, select, apply/aapply,
observe, register_plugin, retire) in litellm/router_api.py. The 123 public members
outside code used become DeprecatedRouterMembers (litellm/router_deprecated.py):
same signatures and results, a DeprecationWarning naming the replacement and a use
counter for callers outside the router. Router internals move to _core_ names;
subclass overrides of old members are still called by the router.
2026-10-29 17:02:41 +00:00
Nate Armstrong
7650f766b0
refactor(router): fence the router surface and document the router-core boundary (#48839)
Add rustdoc overviews for a new litellm-router-core crate and for
litellm-model-catalog, a module docstring on litellm/router.py, and a
shrink-only allowlist of the Router members and router_utils modules used
outside the router, checked in CI by router_surface_check.py.

Move the router-free helpers out of litellm/router_utils: batch_utils to
litellm_core_utils, and the auto-router naming contract, tuning baseline
and baseline request next to the auto-router plugin. No behaviour change.
2026-10-29 00:01:52 +00:00