mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-14 09:03:16 +00:00
382588a365
* feat(proxy): publish opted-in pass-through endpoints on the Model Hub Pass-through endpoints gain display_name and show_in_model_hub. Endpoints that opt in appear on /public/v1/model_hub and /public/model_hub as rows in mode passthrough, named by their display name or their path, with the route in pass_through_path. The admin UI exposes both fields on the create and edit forms and the public hub renders passthrough rows with a route-specific curl example Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): carry pass-through methods onto hub rows and key dashboard rows by route Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): keep health, cost and shell quoting correct for pass-through hub rows Pass-through rows no longer inherit a router model's health check when they share its name, the public table shows n/a instead of Free for their token costs, and the generated curl example escapes shell characters in the route Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): quote the method in the pass-through hub curl example A configured method string is placed after curl -X inside escaped double quotes so it cannot alter the copied command Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): build pass-through hub fixtures through a helper Keeps the dashboard lint budget for inline object arguments under its limit Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): audit the pass-through Model Hub surface end to end Cover forwarding, pagination, validation, auth, updates, duplicate names, config-file precedence, health-name isolation, repeated reads and two chaos cells (create/delete churn, a killed proxy worker). Build the stored pass_through_endpoints list in one shot instead of appending to the list the config reader handed back Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): keep concurrent pass-through creates from overwriting each other Creating an endpoint appends to the general_settings list the proxy already holds instead of persisting a private copy, so two admins creating at the same time both land, as on main before this branch Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): serialize pass-through endpoint creates behind a process lock Each create copies the stored list under an asyncio lock before persisting, so simultaneous creates in one worker no longer overwrite each other and a rejected write (config-owned path) never lands in the in-memory settings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): collect paginated pass-through rows without mutating a list Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): let a pass-through with no stored timeout save from the edit form Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): type the pass-through edit fixture so a null timeout compiles Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * perf(proxy): read published pass-throughs from the in-memory settings store The public Model Hub routes read the pass-through list this worker already serves from proxy_config.settings instead of querying the config table on every anonymous hub read. Also drops a mutable-ok suppression that no longer guards a violation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): list a config file pass-through beside database rows on the hub Main merges database pass-throughs with config file ones on other paths since #43962, so a database write beside a config entry is accepted rather than rejected as config-owned Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): type the monkeypatch fixture in the pass-through hub tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(proxy): validate pass-through config entries instead of splatting unknown dicts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(proxy): list model hub rows as a tagged union of model groups and pass-throughs Health enrichment and the no-router check now match on ModelGroupRow or PassThroughRow instead of testing pass_through_path in each route. Both hub routes share published_hub_rows, the v1 row helper is sync again, and the config list route goes back to main's inline merge since the hub no longer needs configured_pass_through_endpoints * test(ui): find the pass-through hub row by role to stay under the no-node-access budget * fix(proxy): keep database pass-through endpoints in the settings store across config reloads A config reload cleared the stored database row before rebuilding the pass-through list, so without store_model_in_db the list fell back to the config file entries alone. The Admin UI triggers that reload on page load through /get/config/callbacks, which dropped UI-created endpoints from the Model Hub until the next write * refactor(proxy): make the hub row matches exhaustive with assert_never * refactor(proxy): return the exhaustive match fallthrough and build hub test inputs in one shot Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(proxy): drop the in-process create lock from pass-through endpoints The lock only covered one worker, and the read-modify-write on the general_settings blob is a separate storage problem tracked as tech debt. Create goes back to main's behavior. * fix: align pass-through model hub with current lint rules Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: share worker state for pass-through hub restart coverage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: show pass-through curl examples in model hub inspector Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: skip invalid pass-through hub entries and await peer refresh Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| .coverage_baseline | ||
| __init__.py | ||
| _coverage_check.py | ||
| _pin_check.py | ||
| conftest.py | ||
| test_background_health.py | ||
| test_exception_handlers.py | ||
| test_harness_smoke.py | ||
| test_lifecycle.py | ||
| test_openapi_customization.py | ||
| test_proxy_config.py | ||
| test_routes_anthropic_beta.py | ||
| test_routes_assistants.py | ||
| test_routes_audio.py | ||
| test_routes_chat_completions.py | ||
| test_routes_completions.py | ||
| test_routes_config.py | ||
| test_routes_embeddings.py | ||
| test_routes_invitation.py | ||
| test_routes_login_sso.py | ||
| test_routes_misc.py | ||
| test_routes_model_cost_map.py | ||
| test_routes_model_info.py | ||
| test_routes_model_metrics.py | ||
| test_routes_models.py | ||
| test_routes_moderations.py | ||
| test_routes_onboarding.py | ||
| test_routes_queue.py | ||
| test_routes_threads.py | ||
| test_routes_utils.py | ||
| test_spend_counters.py | ||
| test_streaming_helpers.py | ||
| test_team_model_name_translation.py | ||