mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-07 07:58:09 +00:00
de0b389205
* feat(router): add Switchyard Stage Router strategy * feat(router): decision-model classifier and safer fallbacks for the Switchyard stage router Moves to nemo-switchyard 0.4.0a1, which adds decision-model classifiers and makes ModelCall.respond and fail async. The classifier is now a tagged union of an LLM judge and a decision model, and decision calls go through the router's adecisions so any decisions provider works. Algorithm misbehavior and library errors route to the capable model instead of failing the request, and the session id plus client session headers are always passed so capable holds apply without a classifier Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(router): keep the Switchyard router inside the type-check gate The gate's environment did not install the optional switchyard extra, so every libsy symbol counted as Unknown. nemo-switchyard joins proxy-dev like the other optional runtime packages, and imports go through switchyard_rust.libsy, which ships py.typed. Step handling moves into helpers so no Final is rebound in the loop, and the dashboard API types are regenerated for the new routing fields Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(router): route Switchyard on every endpoint and reject unsafe router saves The router now passes each request's call type to routing strategies, and Switchyard converts the request to chat messages with the existing converter for that call type before translating it. /v1/messages turns with tool or thinking blocks used to fall back to the capable model, and now route on their signals. The Anthropic converter can keep tool_result.is_error on an analysis copy, so Switchyard sees tool failures there, while requests sent to models never carry it Saves of a strategy router that lists itself now fail with a 400 instead of a 500 with a dead row, the capable and efficient models must differ, judge and decision calls are not retried before failing open, the Switchyard routing fields are classified for redaction, and the edit dialog returns the saved router instead of the stale one Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(router): catch self-routing on patch and rename, and trim Switchyard tests The self-routing check now reads the stored params through the decrypting helper and runs when a patch carries no litellm_params, so a PATCH that points a router at itself or renames it onto one of its targets returns 400 instead of a 500 with a dead row. The Switchyard tests share one transcript builder and router fixture, and reformatting noise in unrelated test files is reverted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(ui): make Switchyard a strategy in the existing auto router form Switchyard is now a card in the existing strategy picker. Picking it swaps the tier and classifier fields for the Switchyard fields, and the shared name, team and save button create it through one submit branch, the same flow as the other strategies. The separate add form, the extracted picker and the hidden-form swap are gone, and the edit dialog keeps its own Switchyard form The judge's session_affinity and message_hash_fallback settings are removed, since Switchyard rejects both for the stage router Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ui): accept partial tool semantics and show server errors on model edit The Switchyard tool semantics field rejected mappings that list only some categories, although the server accepts them. Model PATCH errors were reduced to a generic message, so it now goes through the shared API client like model create and surfaces the server text Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(ui): show the routing strategy picker up front in the auto router form Every strategy was hidden behind an Advanced configuration link, so the quick setup read as the only option and never said it creates a heuristic router. The picker now always shows with Heuristics as the labelled default, the quick setup sits under it, and picking another strategy opens that strategy prefilled with the chosen models Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ui): show the server's reason when a model settings or API key update fails Both screens replaced the rejection with a fixed message, so a 400 such as a router routing to itself read only as a generic failure Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(router): route Switchyard on the Rust router core main replaced the Python router core with the Rust one, which left Switchyard routers registered nowhere. Register them with the other pre-routing strategies, hand each strategy the endpoint the request came through, and evict Switchyard routers on reload and delete like the rest Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(router): scope Switchyard sessions per API key and fix review gaps in the Switchyard form Switchyard session state is keyed by a hash of the caller's key hash and session id, and client session headers no longer reach Switchyard unscoped. The add form carries the quick setup models into Switchyard, sends the team and access groups, clears a stale tool semantics error, and the schema reads API-saved configs with nulls and structured decision instructions Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(router): make the Switchyard test folder a package * fix(router): read the Switchyard session key hash only from the proxy's own metadata * fix(proxy): keep nested nulls as null when masking deployment params * fix(router): send Switchyard judge calls with the caller's own metadata bucket so tag routing applies * fix(router): put Switchyard judge metadata in the bucket the judge endpoint reads --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.