mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-15 09:12:18 +00:00
b0c580d391
* feat: email owners before opted-in virtual keys expire Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: batch expiry alert queries and use branded email template Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: cast expiry alert query bounds as UTC timestamps Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: satisfy expiry alert CI contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat: single-day key expiry alerts with global default and daily check - metadata.key_expiry_alert_days is one integer (1-365) instead of a list - general_settings.key_expiry_alert_days sets a default for every key with an expiry and owner email; key metadata overrides it - LITELLM_KEY_EXPIRY_ALERT_CHECK_INTERVAL_SECONDS defaults to 24 hours * fix: isolate key expiry alert setup and check every 6 hours - build the expiry alert manager in its own startup step that logs and returns None on bad config or setup errors, so later background jobs still register - default LITELLM_KEY_EXPIRY_ALERT_CHECK_INTERVAL_SECONDS to 6 hours * refactor: import litellm_enterprise email helpers directly in key expiry alerts Replace the import_module + Protocol module validation with plain imports, matching the key rotation hook and check_batch_cost. litellm_enterprise is absent from the type-check venv, so the imported names carry the same reasoned pyright ignore used in response_api_endpoints. * fix: apply global key expiry default to keys stored with null metadata Keys created without metadata (for example via /user/new) store JSON null, which failed metadata parsing and skipped the global key_expiry_alert_days default. Read null metadata as empty. * fix: guard litellm_enterprise import in key expiry alert job check_unsafe_enterprise_import requires enterprise imports outside the enterprise package to sit in try/except ImportError. Skip the run when the package is absent, matching the key rotation email hook. * feat: run key expiry alert scan on one pod per cycle, every 12 hours Take the shared PodLockManager lock (litellm_key_expiry_alert_job) like key rotation and expired UI session cleanup, so a single pod scans LiteLLM_VerificationToken each cycle; the per-key Redis claim still guards each email. Default LITELLM_KEY_EXPIRY_ALERT_CHECK_INTERVAL_SECONDS to 12 hours. * fix: scan for key expiry alerts when the job lock cannot be read from Redis acquire_lock returns False both when another pod holds the lock and when Redis errors. Skip the cycle only when another pod's id is on the lock key; otherwise scan without the lock, keeping the per-key claim as the duplicate guard, so a Redis blip cannot drop warnings for a 12-hour cycle. Replace the startup wiring assertion with a behaviour test against the proxy's pod lock. * fix: hold the key expiry alert lock for the whole cycle Each pod's timer fires at its own boot time plus jitter, so releasing the lock after a millisecond scan let every pod run each cycle. Take it non-reentrant for nearly the full interval and never release it, like the weekly spend report, so the first pod to fire runs the cycle and the rest skip. * fix: key the key expiry alert lock by clock window Pods' interval timers fire at their own boot time plus jitter, so a fixed lock name with a TTL shorter than the interval let a late-booted pod run twice in one interval, and a TTL equal to the interval made a pod skip its own next run. Name the lock after the clock window (int(now // interval) * interval, as file_usage_caps buckets its counters) with a one-interval TTL: the first pod in each window runs and the rest skip. Release the window if the scan raises so another pod can retry, and run anyway when the Redis circuit breaker is open. * test: keep key expiry lock tests in-process with a fixed clock Simulate the Redis outage with a disconnected in-process fakeredis server (wired like the router_redis fixture) instead of a socket to a closed port, and give the startup-built job the test clock instead of the wall clock, per tests/unit/AGENTS.md. * fix: declare the key expiry alerts service family on process_alerts process_alerts reads the window lock holder from Redis directly, so test_every_function_that_reaches_redis_directly_declares_its_family requires it to carry its family, as _process_key already does. * refactor: align key expiry alerts with existing litellm surfaces - read general_settings.key_expiry_alert_days on every run via _current_general_settings(), like the performance alert and spend capture jobs, so a UI or API edit applies without a restart; an invalid value is logged and ignored - share the event name as VIRTUAL_KEY_EXPIRING_EMAIL_EVENT in litellm/types, like MODEL_DEPRECATION_EMAIL_EVENT - default the Virtual Key Expiring email event off, like the other scheduled emails - name the Redis family for the claim data (key_expiry_alert_claims) and read the window lock under POD_LOCK_TARGET, mirroring reset_budget_job._lease_is_held - give the per-key check its own validator, run with the logging check through raise_on_invalid_key_metadata * chore: drop two comments that restate the code --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| html_forms | ||
| __init__.py | ||
| test_admin_ui_utils.py | ||
| test_auth_cache_invalidation_pubsub.py | ||
| test_cache_aware_routing.py | ||
| test_cache_codec.py | ||
| test_callback_config_validation.py | ||
| test_callback_utils.py | ||
| test_check_batch_cost.py | ||
| test_check_responses_cost.py | ||
| test_codex_catalog_source.py | ||
| test_codex_model_catalog.py | ||
| test_codex_model_translation.py | ||
| test_codex_version.py | ||
| test_config_sync_pubsub.py | ||
| test_credential_hydration.py | ||
| test_custom_openapi_spec.py | ||
| test_customer_usage.py | ||
| test_debug_utils.py | ||
| test_discoverable_model_filter.py | ||
| test_encrypt_decrypt_utils.py | ||
| test_error_body_call_id.py | ||
| test_expired_ui_session_key_cleanup_manager.py | ||
| test_fips.py | ||
| test_get_routes.py | ||
| test_http_parsing_utils.py | ||
| test_json_merge_patch.py | ||
| test_key_expiry_alert_manager.py | ||
| test_key_rotation_e2e.py | ||
| test_key_rotation_integration.py | ||
| test_key_rotation_lock.py | ||
| test_key_rotation_manager.py | ||
| test_load_config_utils.py | ||
| test_model_deprecation.py | ||
| test_model_listing_utils.py | ||
| test_openai_endpoint_utils.py | ||
| test_openai_error_payload.py | ||
| test_openapi_route.py | ||
| test_openapi_schema_compat.py | ||
| test_periodic_reload_schedule.py | ||
| test_proxy_encrypt_decrypt.py | ||
| test_proxy_rate_limit_error.py | ||
| test_rbac_utils.py | ||
| test_realtime_cache.py | ||
| test_registry_read_through.py | ||
| test_request_tag_mappings.py | ||
| test_reset_budget_job.py | ||
| test_resource_ownership.py | ||
| test_scheduled_job_stagger.py | ||
| test_semantic_text_index.py | ||
| test_slow_gc_logger.py | ||
| test_sse_keepalive.py | ||
| test_static_asset_utils.py | ||
| test_swagger_utils.py | ||
| test_timezone_utils.py | ||
| test_token_count_log.py | ||
| test_token_count_settings.py | ||
| test_upsert_budget_membership.py | ||
| test_user_api_key_cache.py | ||
| test_validation_error_body.py | ||