Yassin Kortam cbb64111fa fix(proxy): keep serving reads from the read replica when the primary DB is down at startup (#31951) ... * fix(proxy): keep serving reads from the read replica when the primary DB is down at startup RoutingPrismaWrapper.connect() connected the writer first and let a writer failure propagate, so a proxy that started during a primary outage ended up with no Prisma client at all (startup swallows the error under allow_requests_on_db_unavailable): DB-stored models never loaded and every inference request failed with 400 Invalid model name, even with a healthy DATABASE_URL_READ_REPLICA. Workers recycled via MAX_REQUESTS_BEFORE_RESTART hit this mid-outage and stayed broken for the rest of the outage. connect() now degrades on a writer-only failure: reads (key auth, DB-stored model loads) are served by the reader, writes fail at call time, and the DB health watchdog keeps retrying the writer reconnect, which clears the degraded flag once the primary recovers. A full outage (both sides down) still raises as before. Resolves LIT-4159 * fix(proxy): clear degraded-writer flag when the reconnect probe finds the writer already healthy The direct-reconnect path returns early when the writer probe succeeds (engine already reconnected by another path, e.g. an IAM token refresh), skipping recreate_prisma_client, which was the only runtime path clearing _writer_unavailable. The stale flag made the watchdog fire reconnect attempts against a healthy writer on every cooldown cycle until restart. Clear the flag in the early-return branch and cover it with a regression test that fails without the change		2026-07-04 12:45:09 -07:00
..
_experimental/mcp_server	feat(mcp): discover the OBO token endpoint via RFC 9728 -> RFC 8414 (no IdP guessing) (#31762)	2026-07-03 18:56:57 -07:00
a2a	feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950)	2026-06-29 09:32:39 +05:30
agent_endpoints	feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950)	2026-06-29 09:32:39 +05:30
anthropic_endpoints	feat: litellm oss staging (#31935)	2026-07-03 09:27:31 +05:30
auth	revert: undo teamless all-team-models denial from #32022 and #29746 (#32032)	2026-07-03 18:56:07 -07:00
batches_endpoints	test(batches): add 1:1 test file scaffold for batches component paths (#30529)	2026-06-29 09:22:58 +05:30
client	feat(proxy): add AES-256-GCM at-rest credential encryption with versioned format and re-encryption migration (#31215)	2026-06-29 20:14:22 +02:00
common_utils	fix(proxy): stop the decrypt-failure debug log from leaking the raw value	2026-07-03 13:03:45 -07:00
db	fix(proxy): keep serving reads from the read replica when the primary DB is down at startup (#31951)	2026-07-04 12:45:09 -07:00
discovery_endpoints	chore: litellm oss staging160626 (#30527)	2026-06-16 18:23:13 -07:00
experimental/mcp_server
google_endpoints	chore: litellm oss staging160626 (#30527)	2026-06-16 18:23:13 -07:00
guardrails	fix(guardrails): address review comments on headroom fail_open	2026-07-03 19:40:33 +00:00
health_endpoints	fix(proxy): authorize /health/test_connection against loaded deployment's team_id (VERIA-441) (#31767)	2026-07-01 17:05:49 -07:00
hooks	revert: undo teamless all-team-models denial from #32022 and #29746 (#32032)	2026-07-03 18:56:07 -07:00
image_endpoints	fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through (#24232)	2026-06-09 22:10:23 +02:00
logging_endpoints	feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through callbacks (#31134)	2026-06-24 15:25:10 -07:00
management_endpoints	feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783)	2026-07-03 12:20:12 -07:00
management_helpers	feat(mcp): add all-proxy-mcpservers sentinel to grant teams every MCP server (#32012)	2026-07-03 13:59:28 -07:00
memory	Litellm memory improvements v2 (#26541)	2026-04-25 19:03:43 -07:00
middleware	feat(proxy): configurable response headers and login-page hint (#30792)	2026-06-18 18:12:45 -07:00
openai_files_endpoint	fix(vertex/files): stream OpenAI->Vertex batch JSONL uploads (#31036)	2026-06-24 13:19:57 -07:00
pass_through_endpoints	test(e2e): add live batches suite across providers and routing scenarios (#30958)	2026-07-02 08:05:23 -07:00
policy_engine	fix(policies): reject non-existent team/key/model scope entries on attachment create (#32131)	2026-07-04 11:58:29 -07:00
prompts	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
proxy_server	fix: redact db environment variable debug logs	2026-07-03 13:03:45 -07:00
public_endpoints	feat(ui): add Amazon Bedrock Mantle to the Add Model provider dropdown (#31034)	2026-06-22 17:31:25 -07:00
rag_endpoints	fix: harden /key/update authorization checks (#27878)	2026-05-14 04:16:04 +00:00
realtime_endpoints	chore: litellm oss 170626 (#30637)	2026-06-17 21:11:12 -07:00
response_api_endpoints	Litellm websocket improvements (#29563)	2026-06-03 11:48:35 -07:00
shutdown	feat(proxy): native /health/drain preStop hook for graceful shutdown (#29439)	2026-06-02 16:30:44 -07:00
spend_tracking	fix(mcp): roll up MCP tool spend to user counters and usage UI (#31576)	2026-07-02 08:16:39 -07:00
test_configs
types_utils	chore(proxy): also scrub guardrail callbacks / module paths from DB overlay	2026-05-14 01:24:51 +00:00
ui_crud_endpoints	fix(mcp): byom visibility, preview UX, and admin settings gating (#31809)	2026-07-02 01:04:22 -07:00
utils	fix(proxy): isolate poison spend-log rows so one bad record can't drop the whole batch (#31705)	2026-06-30 12:21:14 -07:00
vector_store_endpoints	Title: fix(proxy): resolve vector store file list credentials from team deployments (#29739)	2026-06-06 12:36:05 -07:00
video_endpoints	test(videos): add 1:1 test file scaffold for videos component paths (#30631)	2026-06-29 09:12:51 +05:30
__init__.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
conftest.py	feat(proxy): native /health/drain preStop hook for graceful shutdown (#29439)	2026-06-02 16:30:44 -07:00
test_aiohttp_cleanup_closed.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_aiohttp_session_recovery.py	fix: use AsyncMock for concurrent test consistency	2026-03-18 00:54:23 +00:00
test_api_key_masking_in_errors.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_audio_speech_prometheus_hooks.py	fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through (#24232)	2026-06-09 22:10:23 +02:00
test_batch_expiry.py	fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata (#28425)	2026-05-26 11:35:42 -07:00
test_batch_metadata_none_fix.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_batch_retrieve_bedrock.py	fix(proxy/batches): forward model to retrieve_batch for bedrock	2026-04-29 22:48:03 +02:00
test_batch_x_litellm_model_encoding.py	test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510)	2026-06-16 10:20:59 -07:00
test_blocked_response_usage.py	feat: litellm oss staging (#31935)	2026-07-03 09:27:31 +05:30
test_budget_reservation.py	chore: litellm oss staging (#30745)	2026-06-18 13:55:35 -07:00
test_caching_routes.py	Litellm oss staging 040626 (#29671)	2026-06-04 11:07:20 -07:00
test_chat_completion_metadata.py	fix: propagate JWT auth metadata to OTEL spans (#19627)	2026-01-23 21:21:23 -08:00
test_common_request_processing.py	fix(proxy): emit x-litellm-response-cost header on /messages and /generateContent (LIT-4076) (#31675)	2026-06-29 20:29:49 -07:00
test_component_allowlists.py	chore(oss): litellm oss staging 120626 (#30292)	2026-06-12 09:49:25 -07:00
test_cors_config.py	refactor: extract _get_cors_config() for testability, fix no-op CORS tests	2026-04-11 22:24:04 +05:30
test_custom_proxy.py
test_dynamic_mcp_route.py	Litellm oss staging 040626 (#29671)	2026-06-04 11:07:20 -07:00
test_empty_model_list.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_enforce_user_param.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_fallback_management_endpoints.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_fastapi_offline_routes.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_filter_models_by_team_access_group.py	fix: resolve access group names in _filter_models_by_team_id (#25224)	2026-05-01 11:55:36 +05:30
test_health_check_functions.py	feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716)	2026-05-13 09:49:05 -07:00
test_health_check_max_tokens.py	fix: skip health check for semantic auto_router deployments (#31668)	2026-06-29 19:47:20 -07:00
test_langfuse_passthrough_security.py	chore(callbacks): guard dynamic integration hosts	2026-04-30 14:27:19 -07:00
test_lazy_openapi_snapshot.py	[Fix] Tests: Move Misplaced Import in Lazy OpenAPI Snapshot Test	2026-05-01 14:24:53 -07:00
test_litellm_pre_call_utils.py	fix(proxy): restore admin key/team callback_vars.turn_off_message_logging override (LIT-3587) (#31905)	2026-07-02 15:01:58 -07:00
test_max_budget_env_var.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_mcp_asgi_response.py	fix(mcp): surface upstream 401 for token-forwarding MCP servers (#27847)	2026-05-13 12:03:36 -07:00
test_model_based_routing_files_batches.py	test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510)	2026-06-16 10:20:59 -07:00
test_model_dump_with_preserved_fields.py	Fix_mapped tests part 2	2026-02-26 12:43:39 +05:30
test_model_id_header_propagation.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_model_info_default_limits.py	fix(proxy): align /v1/model/info with router deployments (#30025)	2026-06-10 19:38:21 -07:00
test_model_level_guardrails.py	fix(guardrails): run pre_call hook once for model-level guardrails (#30543)	2026-06-16 11:17:03 -07:00
test_model_list_healthy_only.py	feat: litellm oss 110626 (#30202)	2026-06-11 22:30:26 -07:00
test_modify_response_streaming_passthrough.py	fix(proxy): use e.request_data for logging_obj in ModifyResponseException streaming passthrough (#30800)	2026-06-18 23:29:08 -07:00
test_openapi_schema_validation.py	fix(proxy): preserve HTTP operations when injecting WebSocket stubs into OpenAPI schema	2026-05-06 00:28:42 +02:00
test_plugin_routes.py	feat: litellm plugin architecture v2 (#30688)	2026-06-20 20:37:22 -07:00
test_pricing_field_strip.py	chore: litellm oss staging160626 (#30527)	2026-06-16 18:23:13 -07:00
test_prometheus_cleanup.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_provider_url_destination_guard.py	chore(proxy): move URL-valued model/file_id guard from SDK to proxy	2026-05-02 02:17:46 +00:00
test_proxy_cli.py	chore: litellm oss staging (#30968)	2026-06-23 07:31:44 -07:00
test_proxy_logging_hook_detection.py	perf: reduce per-request and per-chunk overhead across Anthropic streaming hot paths (#28289)	2026-05-23 12:15:59 -07:00
test_proxy_server.py	fix(proxy): tighten role gating on /get/config/callbacks response (#31745)	2026-07-01 17:58:31 -07:00
test_proxy_types.py	fix(proxy): add default=None to LiteLLM_TeamMembership.litellm_budget_table (#29684)	2026-06-04 12:13:11 -07:00
test_proxy_utils.py	fix(proxy): record partial spend on the failure row for interrupted streams (#30788)	2026-06-19 12:03:15 -07:00
test_pyroscope.py	Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior.	2026-05-04 18:17:31 +00:00
test_read_model_list.py	feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135)	2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py	refactor(proxy): replace DualCache with UserApiKeyCache for user API key management	2026-04-28 19:15:03 +00:00
test_response_model_sanitization.py	perf: eliminate per-request callback scanning on proxy hot path (#27858)	2026-05-14 09:28:31 -07:00
test_route_a2a_models.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_route_llm_request.py	fix(realtime): preserve nested transcription model and session-first model priority	2026-07-03 14:21:22 -07:00
test_sensitive_route_auth.py	chore(proxy): guard sensitive public endpoints	2026-04-30 11:52:47 -07:00
test_shared_health_check.py	feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716)	2026-05-13 09:49:05 -07:00
test_spend_log_cleanup.py	feat(spend_logs): opt-in native Postgres partitioning for SpendLogs retention (#29466)	2026-06-11 11:02:42 -07:00
test_swagger_chat_completions.py	fix(proxy): avoid misleading multi-method operation ids	2026-04-30 20:44:14 -07:00
test_team_member_update.py	feat(ui): add budget duration to edit team member form (#29717)	2026-06-06 17:24:55 -07:00
test_team_org_move.py	fix(team_endpoints): auto-add SSO team members to org on move (proxy admin only) (#26377)	2026-04-24 08:36:25 -07:00
test_tools_allowlist_enforcement.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_update_llm_router_resilience.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00