The componentized chart at helm/litellm had no way to mount extra
volumes into its deployments, so custom callback or SSO handler code
could not be mounted the way the docs describe for the monolithic
chart. Adds per-component volumes and volumeMounts values for gateway,
backend, and ui, merged with the existing gateway-config volume, plus
a helm-unittest suite for the chart wired into the helm unit test
workflow
Resolves LIT-4209
* change deployment configs to include a litellm.cache for litellm-backend pod mirroring litellm-gateway pod
* omit backend annotations block when config and podAnnotations are both empty
* reuse gateway config/configmap for backend instead of separate backend config
---------
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Tin Chi Lo <tin@Tins-MBP.localdomain>
Co-authored-by: Tin Chi Lo <tin@Tins-MacBook-Pro.local>
* feat(helm): split per-component ServiceAccounts for gateway, backend, and UI
Replace the single shared serviceAccount with three separate serviceAccounts
(gateway, backend, ui) so operators can attach different IRSA / Workload
Identity annotations per component without granting data-plane credentials
to the UI pod.
Key changes:
- values.yaml: rename serviceAccount → serviceAccounts with gateway/backend/ui
sub-keys; UI defaults to automount: false
- _helpers.tpl: replace litellm.serviceAccountName with three component-scoped
helpers (litellm.gateway/backend/ui.serviceAccountName)
- serviceaccount.yaml: create up to three separate ServiceAccount objects with
component labels and per-SA automountServiceAccountToken
- gateway/backend deployments: use their respective SA helpers
- ui deployment: use litellm.ui.serviceAccountName + explicit
automountServiceAccountToken: false on the pod spec so the projected token
is absent even when the SA itself allows it
- migrations-job: share the backend SA (both need DB write access)
Resolves LIT-3171
https://claude.ai/code/session_01QPy362WnjmEpeNuJaPUqmF
* fix(helm): enforce automountServiceAccountToken on all pod specs; fix leading --- in serviceaccount.yaml
- gateway/backend deployments: add explicit automountServiceAccountToken on
the pod spec so serviceAccounts.*.automount is honoured regardless of
whether the SA is chart-created or operator-supplied (previously the flag
only took effect on the SA object when create: true, creating an asymmetry
with the UI which already enforced it at pod-spec level)
- serviceaccount.yaml: use a $prev sentinel to emit --- only between
documents, preventing a leading --- when gateway SA is skipped but
backend or ui SA is created (avoids lint/GitOps warnings from strict
YAML parsers and tools like ArgoCD)
https://claude.ai/code/session_01QPy362WnjmEpeNuJaPUqmF
---------
Co-authored-by: Claude <noreply@anthropic.com>
Split the monolithic LiteLLM proxy into independently scalable Kubernetes components to allow separate horizontal scaling of the LLM data plane and management API surfaces
- Add DatabaseURLSettings pydantic-settings model that assembles DATABASE_URL (and optional DATABASE_URL_READ_REPLICA) from discrete DATABASE_* env vars before Prisma initializes, supporting both IAM token auth (minting short-lived RDS tokens) and password auth; replaces the CLI-only path that componentized entrypoints bypass
- Add gateway component (port 4000) that trims the proxy route table to the LLM data-plane surface (chat, embeddings, completions, audio, realtime, provider passthroughs, health/metrics) via an allowlist applied inside the lifespan context so plugin-registered routes are captured
- Add backend component (port 4001) that exposes the management/admin surface (keys, users, teams, orgs, spend analytics, model management, SSO, audit logs) with a complementary allowlist
- Add ui component — Next.js static export served by nginx (port 3000) with RSC payload routing, asset prefix aliasing, and SPA fallback for dashboard routes
- Add migrations component with dedicated Dockerfile that runs prisma migrate deploy via a Helm pre-install/pre-upgrade Job, eliminating per-pod schema contention on the Prisma advisory lock
- Add Helm chart (helm/litellm) with separate Deployments, Services, HPAs, and ConfigMap for each component; shared _helpers.tpl emits DATABASE_*, IAM_TOKEN_DB_AUTH, REDIS_*, and DISABLE_SCHEMA_UPDATE env vars from chart values; ingress template routes traffic to the correct component by path prefix
- Add comprehensive tests for DatabaseURLSettings covering IAM auth, password auth, read replica fallbacks, operator-pinned URL preservation, and percent-encoding; add coverage test asserting gateway + backend allowlist union equals the full proxy route set
- Add pydantic-settings>=2.14.1 as a proxy extra dependency and update liccheck allowlist
Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu>
