mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
014cb8fa9d
Split the monolithic LiteLLM proxy into independently scalable Kubernetes components to allow separate horizontal scaling of the LLM data plane and management API surfaces - Add DatabaseURLSettings pydantic-settings model that assembles DATABASE_URL (and optional DATABASE_URL_READ_REPLICA) from discrete DATABASE_* env vars before Prisma initializes, supporting both IAM token auth (minting short-lived RDS tokens) and password auth; replaces the CLI-only path that componentized entrypoints bypass - Add gateway component (port 4000) that trims the proxy route table to the LLM data-plane surface (chat, embeddings, completions, audio, realtime, provider passthroughs, health/metrics) via an allowlist applied inside the lifespan context so plugin-registered routes are captured - Add backend component (port 4001) that exposes the management/admin surface (keys, users, teams, orgs, spend analytics, model management, SSO, audit logs) with a complementary allowlist - Add ui component — Next.js static export served by nginx (port 3000) with RSC payload routing, asset prefix aliasing, and SPA fallback for dashboard routes - Add migrations component with dedicated Dockerfile that runs prisma migrate deploy via a Helm pre-install/pre-upgrade Job, eliminating per-pod schema contention on the Prisma advisory lock - Add Helm chart (helm/litellm) with separate Deployments, Services, HPAs, and ConfigMap for each component; shared _helpers.tpl emits DATABASE_*, IAM_TOKEN_DB_AUTH, REDIS_*, and DISABLE_SCHEMA_UPDATE env vars from chart values; ingress template routes traffic to the correct component by path prefix - Add comprehensive tests for DatabaseURLSettings covering IAM auth, password auth, read replica fallbacks, operator-pinned URL preservation, and percent-encoding; add coverage test asserting gateway + backend allowlist union equals the full proxy route set - Add pydantic-settings>=2.14.1 as a proxy extra dependency and update liccheck allowlist Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu> |
||
|---|---|---|
| .. | ||
| azure_client_usage_test.py | ||
| ban_constant_numbers.py | ||
| ban_copy_deepcopy_kwargs.py | ||
| bedrock_pricing.py | ||
| callback_manager_test.py | ||
| check_data_replace_usage.py | ||
| check_endpoint_coverage.py | ||
| check_fastuuid_usage.py | ||
| check_get_model_cost_key_performance.py | ||
| check_guardrail_apply_decorator.py | ||
| check_licenses.py | ||
| check_provider_folders_documented.py | ||
| check_spanattributes_value_usage.py | ||
| check_unsafe_enterprise_import.py | ||
| code_qa_check_tests.py | ||
| enforce_llms_folder_style.py | ||
| ensure_async_clients_test.py | ||
| info_log_check.py | ||
| liccheck.ini | ||
| license_cache.json | ||
| litellm_logging_code_coverage.py | ||
| log.txt | ||
| memory_test.py | ||
| pass_through_code_coverage.py | ||
| prevent_key_leaks_in_exceptions.py | ||
| recursive_detector.py | ||
| router_code_coverage.py | ||
| router_enforce_line_length.py | ||
| test_aio_http_image_conversion.py | ||
| test_ban_set_verbose.py | ||
| test_chat_completion_imports.py | ||
| test_proxy_types_import.py | ||
| test_router_strategy_async.py | ||
| user_api_key_auth_code_coverage.py | ||