mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
abe3289398
* fix(proxy): retry model cost map fetch with Retry-After-aware backoff and stop downgrading to the packaged backup on reload failure A 429 or transient network error during a manual or scheduled model cost map reload used to silently replace litellm.model_cost with the stale backup JSON bundled in the installed wheel, stamp the reload as successful, and clear the force_reload flag, so a fleet could serve months-old pricing until the next interval. Runtime reloads now go through refetch_model_cost_map, which retries 429/5xx/transport errors up to 3 times honoring Retry-After (capped at 30s, exponential backoff with jitter otherwise) and returns a failure value instead of the backup when the fetch or integrity validation fails. On failure the pod keeps its currently loaded map, the periodic job leaves last_run and force_reload untouched so it retries on the next config poll, and the manual endpoint returns 502 with the reason instead of reporting a fake success. Startup behavior is unchanged: boot still falls back to the packaged backup since there is no previously loaded map to keep. * fix(proxy): use shared async httpx client for cost map reload and make retry tests CI-env-proof The reload fetch now goes through get_async_httpx_client with a dedicated httpxSpecialProvider.ModelCostMap pool instead of constructing a raw httpx.AsyncClient, so it inherits deployment-level TLS and transport settings and passes the ensure_async_clients gate. Tests inject a MockTransport-backed client through the same seam. An autouse fixture clears LITELLM_LOCAL_MODEL_COST_MAP, which CI exports and which short-circuited the retry tests; the two TestPriceDataReloadAPI tests and the config sync pubsub reload test that still patched get_model_cost_map now patch refetch_model_cost_map instead. |
||
|---|---|---|
| .. | ||
| html_forms | ||
| test_cache_codec.py | ||
| test_callback_utils.py | ||
| test_config_sync_pubsub.py | ||
| test_custom_openapi_spec.py | ||
| test_encrypt_decrypt_utils.py | ||
| test_expired_ui_session_key_cleanup_manager.py | ||
| test_get_routes.py | ||
| test_http_parsing_utils.py | ||
| test_json_merge_patch.py | ||
| test_key_rotation_e2e.py | ||
| test_key_rotation_integration.py | ||
| test_key_rotation_lock.py | ||
| test_key_rotation_manager.py | ||
| test_load_config_utils.py | ||
| test_openai_endpoint_utils.py | ||
| test_path_utils.py | ||
| test_reset_budget_job.py | ||
| test_static_asset_utils.py | ||
| test_timezone_utils.py | ||
| test_upsert_budget_membership.py | ||
| test_user_api_key_cache.py | ||