mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
24082bc07d
* fix(e2e): route model management to the control plane and restore Gateway.create_model
The split-transport routing table listed only /model/info as a control-plane
prefix, so /model/new and /model/delete were sent to the data-plane gateway,
which does not serve management routes and 404s them. Every suite that
registers deployments at runtime (llm_translation, batches, access_control)
failed on the split stage deployment because of this. Widen the prefix to
/model/ so all model-management routes reach the control plane while /models
stays on the data plane.
Separately, batch_client.py and several llm_translation tests call
gateway.create_model, but Gateway never had that method, so all 17 batch tests
errored at fixture setup with AttributeError. Add create_model/delete_model to
Gateway (with the optional mode that batches needs) and make EndpointsClient
delegate to it instead of carrying its own copy.
Regression tests cover both: the routing predicate for management vs LLM paths
and the Gateway model-management surface via a typed fake Transport. Both fail
on the previous code
* test(e2e): make the fake transport payload depend on response_type
The recording fake always answered with {"model_id": ...} even when the
caller asked for NoBody, which only validated because pydantic ignores extra
fields by default. Return an empty payload for response types that carry no
fields so a future extra="forbid" on NoBody cannot turn the delete test into
a ValidationError inside the fake
* test(e2e): probe the full spend read surface including schema-hidden routes
The curated spend-route list missed twelve read endpoints, most of them
include_in_schema=False and therefore invisible to the schema-discovery test:
/spend/logs/v2, /spend/logs/session/ui, /global/all_end_users,
/global/activity/exceptions/deployment, and the per-entity daily activity
family (user, user aggregated, team, organization, customer, end_user, tag).
Add them all, verified responsive against the live split stage deployment.
/end_user was missing from CONTROL_PLANE_PREFIXES, so /end_user/daily/activity
would have been routed to the data plane and 404ed like /model/new used to;
add the prefix and pin it plus the daily-activity routes in the transport
routing test.
/provider/budgets stays excluded with a documented reason: it returns 500
whenever router_settings.provider_budget_config is absent, so probing it on a
proxy without provider budget routing configured can never be green
55 lines
2.1 KiB
Python
55 lines
2.1 KiB
Python
"""Spend-tracking suite's `client` fixture and driver-model registration.
|
|
|
|
The shared lifecycle (resources/scoped_key), proxy liveness skip, and e2e marker
|
|
live in the parent tests/e2e/conftest.py. SpendClient exposes the shared Gateway
|
|
(GatewayProvider), so the `resources` fixture cleans up keys and customers this
|
|
suite creates.
|
|
|
|
The suite drives real calls through three deployments. On the stage gateway they
|
|
are baked into the proxy config; on a local dev proxy they usually are not, so
|
|
`driver_models` registers whichever are missing via /model/new and deletes only
|
|
the ones it created, never a config-baked deployment. Each registration carries
|
|
the provider key from the test runner's env when set (so a local proxy whose
|
|
container env lacks the key still works); otherwise it falls back to an
|
|
os.environ reference resolved from the proxy's own env, the stage convention.
|
|
"""
|
|
|
|
import os
|
|
from typing import Iterator
|
|
|
|
import pytest
|
|
|
|
from models import LiteLLMParamsBody
|
|
from spend_e2e_client import SpendClient, build_client
|
|
|
|
|
|
def _driver_params(provider_model: str, env_var: str) -> LiteLLMParamsBody:
|
|
return LiteLLMParamsBody(
|
|
model=provider_model,
|
|
api_key=os.environ.get(env_var) or f"os.environ/{env_var}",
|
|
)
|
|
|
|
|
|
DRIVER_MODELS: tuple[tuple[str, str, str], ...] = (
|
|
("gemini-2.5-flash", "gemini/gemini-2.5-flash", "GEMINI_API_KEY"),
|
|
("claude-haiku-4-5", "anthropic/claude-haiku-4-5", "ANTHROPIC_API_KEY"),
|
|
("openai-text-embedding-3-small", "openai/text-embedding-3-small", "OPENAI_API_KEY"),
|
|
)
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
def client() -> SpendClient:
|
|
return build_client()
|
|
|
|
|
|
@pytest.fixture(scope="session", autouse=True)
|
|
def driver_models(client: SpendClient) -> Iterator[None]:
|
|
existing = frozenset(entry.model_name for entry in client.gateway.model_info())
|
|
created = tuple(
|
|
client.gateway.create_model(name, _driver_params(provider_model, env_var))
|
|
for name, provider_model, env_var in DRIVER_MODELS
|
|
if name not in existing
|
|
)
|
|
yield
|
|
for model_id in created:
|
|
client.gateway.delete_model(model_id)
|