mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-06 08:16:43 +00:00
b7b20664c1
* feat: add LITELLM_WORKER_STARTUP_HOOKS for per-worker initialization (gflags support) Add support for running user-defined startup hooks in each worker process during proxy_startup_event. This enables re-initialization of in-process state (like gflags.FLAGS) that doesn't survive uvicorn worker spawning. Usage: export LITELLM_WORKER_STARTUP_HOOKS=mymodule:init_fn,other:setup_fn Hooks run early in proxy_startup_event (before config/DB loading). Supports both sync and async callables. Errors propagate to prevent broken workers from serving traffic. No-op when env var is unset. Includes 5 tests covering sync/async hooks, multiple hooks, error propagation, and no-hooks-set scenarios. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * docs: add Worker Startup Hooks page with gflags usage example - New docs page: docs/proxy/worker_startup_hooks.md - Explains the problem (per-process state lost in multi-worker deployments) - Full gflags example with wrapper module and startup script - Covers multiple hooks, async hooks, error behavior - Architecture diagram showing master→worker flow - Added LITELLM_WORKER_STARTUP_HOOKS to config_settings.md env var table - Added to sidebar under Setup & Deployment Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * Update litellm/proxy/proxy_server.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Apply suggestion from @greptile-apps[bot] Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _experimental/mcp_server | ||
| agent_endpoints | ||
| anthropic_endpoints | ||
| auth | ||
| client | ||
| common_utils | ||
| db | ||
| discovery_endpoints | ||
| experimental/mcp_server | ||
| google_endpoints | ||
| guardrails | ||
| health_endpoints | ||
| hooks | ||
| image_endpoints | ||
| management_endpoints | ||
| management_helpers | ||
| middleware | ||
| openai_files_endpoint | ||
| pass_through_endpoints | ||
| policy_engine | ||
| prompts | ||
| public_endpoints | ||
| rag_endpoints | ||
| response_api_endpoints | ||
| spend_tracking | ||
| test_configs | ||
| ui_crud_endpoints | ||
| vector_store_endpoints | ||
| __init__.py | ||
| conftest.py | ||
| test_aiohttp_cleanup_closed.py | ||
| test_api_key_masking_in_errors.py | ||
| test_audio_speech_prometheus_hooks.py | ||
| test_batch_expiry.py | ||
| test_batch_metadata_none_fix.py | ||
| test_caching_routes.py | ||
| test_chat_completion_metadata.py | ||
| test_common_request_processing.py | ||
| test_custom_proxy.py | ||
| test_empty_model_list.py | ||
| test_enforce_user_param.py | ||
| test_fallback_management_endpoints.py | ||
| test_fastapi_offline_routes.py | ||
| test_health_check_functions.py | ||
| test_health_check_max_tokens.py | ||
| test_litellm_pre_call_utils.py | ||
| test_model_dump_with_preserved_fields.py | ||
| test_model_id_header_propagation.py | ||
| test_prometheus_cleanup.py | ||
| test_proxy_cli.py | ||
| test_proxy_server.py | ||
| test_proxy_types.py | ||
| test_proxy_utils.py | ||
| test_pyroscope.py | ||
| test_response_model_sanitization.py | ||
| test_route_a2a_models.py | ||
| test_route_llm_request.py | ||
| test_shared_health_check.py | ||
| test_spend_log_cleanup.py | ||
| test_swagger_chat_completions.py | ||
| test_team_member_update.py | ||
| test_tools_allowlist_enforcement.py | ||
| test_update_llm_router_resilience.py | ||