mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-05 08:07:05 +00:00
b4ff05be8e
* fix(proxy): stop model writes 500ing on another pod's delete A model write judges the reload it triggers by diffing this pod's router before and after, and reports anything that stopped serving as damage. On a pod that has not yet polled a delete another pod made, the snapshot still lists that model; the reload then evicts it because the db no longer has it, and the guard reads its own correct reconcile as degradation. The row is written and served, but the caller gets a 500. Since propagation between pods is a 30s db poll, any delete followed by a create inside that window can land on a pod that has not caught up, so a delete-then-create pair returns 500 whenever the two requests hit different pods. _delete_deployment already computes exactly the set that settles it: the ids the db and config still want. Thread it up through _update_llm_router, add_deployment and clear_cache to the verdict, and intersect the drop set with it so an id the db no longer has stops counting as collateral. Where no reconcile ran the set is None and every drop is still reported, so a genuinely broken reload is caught as before. _delete_deployment now returns that set instead of a delete count; the count had no callers in the proxy, and the tests asserting it already assert the eviction calls. * test(proxy): fold reload-verdict test commentary into docstrings and assertions Greptile flagged the inline comments against the repo's no-new-comments rule. The case-by-case context moves into the test docstring, and the two return-contract assertions carry their reasoning as failure messages instead. * test: fix clear_cache mock return type in model block/unblock tests |
||
|---|---|---|
| .. | ||
| .coverage_baseline | ||
| __init__.py | ||
| _coverage_check.py | ||
| _pin_check.py | ||
| conftest.py | ||
| test_background_health.py | ||
| test_exception_handlers.py | ||
| test_harness_smoke.py | ||
| test_lifecycle.py | ||
| test_openapi_customization.py | ||
| test_proxy_config.py | ||
| test_routes_anthropic_beta.py | ||
| test_routes_assistants.py | ||
| test_routes_audio.py | ||
| test_routes_chat_completions.py | ||
| test_routes_completions.py | ||
| test_routes_config.py | ||
| test_routes_embeddings.py | ||
| test_routes_invitation.py | ||
| test_routes_login_sso.py | ||
| test_routes_misc.py | ||
| test_routes_model_cost_map.py | ||
| test_routes_model_info.py | ||
| test_routes_model_metrics.py | ||
| test_routes_models.py | ||
| test_routes_moderations.py | ||
| test_routes_onboarding.py | ||
| test_routes_queue.py | ||
| test_routes_threads.py | ||
| test_routes_utils.py | ||
| test_spend_counters.py | ||
| test_streaming_helpers.py | ||
| test_team_model_name_translation.py | ||