mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
24082bc07d
* fix(e2e): route model management to the control plane and restore Gateway.create_model
The split-transport routing table listed only /model/info as a control-plane
prefix, so /model/new and /model/delete were sent to the data-plane gateway,
which does not serve management routes and 404s them. Every suite that
registers deployments at runtime (llm_translation, batches, access_control)
failed on the split stage deployment because of this. Widen the prefix to
/model/ so all model-management routes reach the control plane while /models
stays on the data plane.
Separately, batch_client.py and several llm_translation tests call
gateway.create_model, but Gateway never had that method, so all 17 batch tests
errored at fixture setup with AttributeError. Add create_model/delete_model to
Gateway (with the optional mode that batches needs) and make EndpointsClient
delegate to it instead of carrying its own copy.
Regression tests cover both: the routing predicate for management vs LLM paths
and the Gateway model-management surface via a typed fake Transport. Both fail
on the previous code
* test(e2e): make the fake transport payload depend on response_type
The recording fake always answered with {"model_id": ...} even when the
caller asked for NoBody, which only validated because pydantic ignores extra
fields by default. Return an empty payload for response types that carry no
fields so a future extra="forbid" on NoBody cannot turn the delete test into
a ValidationError inside the fake
* test(e2e): probe the full spend read surface including schema-hidden routes
The curated spend-route list missed twelve read endpoints, most of them
include_in_schema=False and therefore invisible to the schema-discovery test:
/spend/logs/v2, /spend/logs/session/ui, /global/all_end_users,
/global/activity/exceptions/deployment, and the per-entity daily activity
family (user, user aggregated, team, organization, customer, end_user, tag).
Add them all, verified responsive against the live split stage deployment.
/end_user was missing from CONTROL_PLANE_PREFIXES, so /end_user/daily/activity
would have been routed to the data plane and 404ed like /model/new used to;
add the prefix and pin it plus the daily-activity routes in the transport
routing test.
/provider/budgets stays excluded with a documented reason: it returns 500
whenever router_settings.provider_budget_config is absent, so probing it on a
proxy without provider budget routing configured can never be green
52 lines
1.4 KiB
Python
52 lines
1.4 KiB
Python
"""Unit coverage for SplitTransport path routing (is_control_plane_path).
|
|
|
|
Model-management calls (/model/new, /model/delete, /model/info) must go to the
|
|
control plane: the data-plane gateway does not serve management routes, so a
|
|
misrouted /model/new 404s and takes down every suite that registers deployments
|
|
at runtime (llm_translation, batches, access_control). /models must stay on the
|
|
data plane; it is the OpenAI-compatible list-models route, not a management
|
|
route.
|
|
"""
|
|
|
|
import pytest
|
|
|
|
from transport import is_control_plane_path
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"path",
|
|
[
|
|
"/model/new",
|
|
"/model/delete",
|
|
"/model/update",
|
|
"/model/info",
|
|
"/key/generate",
|
|
"/budget/new",
|
|
"/spend/logs",
|
|
"/end_user/daily/activity",
|
|
"/user/daily/activity",
|
|
"/team/daily/activity",
|
|
"/tag/daily/activity",
|
|
],
|
|
)
|
|
def test_management_routes_go_to_the_control_plane(path: str) -> None:
|
|
assert is_control_plane_path(path), (
|
|
f"{path} is a management route; sending it to the data plane 404s"
|
|
)
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"path",
|
|
[
|
|
"/models",
|
|
"/v1/models",
|
|
"/chat/completions",
|
|
"/v1/messages",
|
|
"/embeddings",
|
|
"/anthropic/v1/messages",
|
|
],
|
|
)
|
|
def test_llm_routes_stay_on_the_data_plane(path: str) -> None:
|
|
assert not is_control_plane_path(path), (
|
|
f"{path} is an LLM route; it must go to the data plane"
|
|
)
|