mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-12 03:48:56 +00:00
fec7f5f246
Auto-routers had no home and no list. The create form was mounted in two unrelated places,
inside Models + Endpoints > Add Model and again under Cost Optimization, and neither showed
which auto routers already existed; seeing or editing one meant finding its row in the models
table and drilling in. They now get a dedicated Auto-Routers tab beside All Models, listing
every auto_router/* deployment with create, edit and delete in one place, and both former
entry points are removed.
Creating opens in a shadcn dialog rather than swapping the whole panel out, so the list stays
on screen behind it; the dialog caps its height and scrolls, since the complexity form is long.
The form's own heading goes with it, the dialog header owning that now.
An auto router is a routing construct rather than a deployment, so it also comes off the All
Models table. That table pages server-side off total_count, so a client-side filter would page
over a total including rows it never renders; /v2/model/info therefore gains
exclude_auto_routers (default false, so every existing caller is unaffected) and the filter
runs before the count. /v1/models is untouched, so clients still see auto-routers as models.
Clicking a router opens the same `?model=` drill-in the All Models table uses, so it lands in
ModelInfoView with the full Model Settings, Edit Settings, Edit Auto Router and Delete. An
earlier revision had a bespoke detail page here, which was a partial reimplementation of that
view and showed the router's type twice, once as a Type pill and again as a "Routing strategy"
field saying the same thing. Both are gone.
The auto-router list is keyed under the same `models/list` namespace as the models table
rather than a private one. It reads the same /v2/model/info data, and six call sites across
the app already invalidate ["models","list"] after a write; a separate key meant an edit made
through ModelInfoView left the tab stale until a full reload, and every future writer would
have had to remember a second key.
An auto router has no upstream credential, so its detail header drops Update API Key and
Re-use Credentials, and the destructive action names what it removes rather than saying model.
Test Connection was gated on the editor-aware predicate, which let adaptive and quality routers
through to a check that builds its targets from complexity config they do not have; it now
gates on the deployment predicate.
The edit modal also applies the semantic-matching guard the create form has. It renders those
controls now, and the backend raises on semantic_keyword_matching without an embedding model or
keyword rules, so skipping the shared validator turned an inline message into a raw 400.
Whether a row is writable has two independent axes and the dashboard needs both. STRATEGY:
there are four auto_router/* kinds and only complexity and semantic have a form here, so
adaptive and quality must not be handed an editor that would write auto_router_config onto a
deployment storing its settings elsewhere. ORIGIN: a config.yaml row reports db_model false and
the API refuses it whatever its strategy (PATCH /model/{id}/update 404s, POST /model/delete
400s). Capability is derived per capability rather than as one editable flag, because the
constraints differ: editing needs an editor, deleting removes a row by id and never reads its
config, so a DB-created adaptive router stays deletable. Both axes live in
add_model/auto_router_strategies.ts as a declarative table, one record per strategy, so a fifth
strategy is a table row rather than another branch. That also retired four copies of "is this a
complexity router", one of which was written twice in a row in model_info_view.
Creation narrows to the complexity router, which the UI calls Auto-Router v2; the semantic
option was already badged "to be deprecated" in the picker, so the picker goes away along with
the semantic submit path and its validation helper. Existing semantic routers stay editable.
The edit modal mounted ComplexityRouterConfig without the keyword, escalation and
semantic-matching handlers, so those sections never rendered and could only be set at create
time. It now hydrates them from the stored config, and the five keys become managed only when a
caller supplies that state, so a caller rendering no such control still carries them through. A
component-level round-trip test covers it: a payload-builder test cannot see a hydration bug.
A complexity tier is str | list[str] on the backend, and the UI carried three readers of that
rule, one of which dropped a pinned string. They collapse into one owner,
add_model/complexity_router_tiers.ts.
|
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm | ||
| litellm-proxy-extras | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _ws_vcr.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_passthrough_endpoints.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.