The MongoDB Atlas vector store provider imports pymongo lazily from the
opt-in `mongodb` extra, but none of the shipped images installed that
extra. Any image-based deployment that configured a MongoDB vector store
failed at search time with "requires the 'pymongo' package", which the
user cannot fix without extending the image
Adds `--extra mongodb` to every uv sync in the root Dockerfile,
Dockerfile.database, Dockerfile.non_root, and the gateway component
image. The backend component does not serve /vector_stores so it is left
as is. The extra resolves from the existing uv.lock to pymongo 4.17.0
plus dnspython 2.8.0, no lock change needed
(cherry picked from commit 16fd14f537)
The pinned base (built 2026-07-02) ships busybox 1.37.0-r61 and
libcrypto3/libssl3 3.6.3-r3. Grype reports 16 fixable findings against
those revisions, 8 of them High, so the image-scan gate fails once it
gets past the migration step.
The runtime stage's `apk upgrade` cannot clear them. wolfi-base writes an
exact `=version` constraint for every package it ships into
/etc/apk/world, so `apk upgrade` is a no-op even though the fixed
revisions are in the repo. Advancing them means moving the digest.
The new digest carries busybox 1.38.0-r1, libcrypto3/libssl3 3.6.3-r5
and glibc 2.43-r15, which is at or above the fix revision Wolfi's secdb
records for every finding. Verified with cosign against
chainguard-images/images release.yaml, and grype reports no fixable
findings on the rebuilt image.
CVE-2026-14456, CVE-2026-54876, CVE-2026-38752, CVE-2026-38753,
CVE-2026-38754, CVE-2026-38755
The gateway and backend images generated the prisma client under
HOME=/home/nonroot, so the engine paths baked into the client sat inside a
directory the base image ships at mode 0700. Only uid 65532 can search it,
and prisma resolves those baked paths eagerly with an existence check that
propagates EACCES, so a container started under any other uid dies with a
PermissionError out of pathlib before the PRISMA_QUERY_ENGINE_BINARY
override is ever read. A chart that sets runAsUser, a docker run --user, or
an OpenShift namespace assigning an arbitrary uid all produce that shape,
and the gateway is the request-serving component, so the proxy does not
serve at all.
Bake to /opt/prisma instead, the fixed world-readable path the other three
images already use, and assert at build time that every baked path lands
there. chmod a+rX rather than a+r because prisma executes the engine to
check it can run on this machine. The runtime PRISMA_BINARY_CACHE_DIR pin
keeps the CLI wrapper's own resolution pointing at the bake rather than at
a /home/nonroot/.cache that no longer exists.
The componentized images exec uvicorn directly, so ddtrace-run never wraps the
interpreter. USE_DDTRACE is not inert there; the proxy lifespan still runs
patch_all and litellm's own manual spans still emit. What never gets installed
is ddtrace's ASGI TraceMiddleware: starlette builds its middleware stack lazily
on the first __call__, which is the lifespan scope, so patching from inside the
lifespan body is already too late and no root request span is ever created.
Route both entrypoints through a shared docker/component_entrypoint.sh that
mirrors the monolith's prod_entrypoint.sh contract, including the
DD_TRACE_OPENAI_ENABLED=False export that keeps ddtrace's openai integration
from double-reporting calls litellm instruments itself.
Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com>
The gateway image installed --extra proxy/proxy-runtime/extra_proxy/semantic-router
but not bedrock-realtime, so aws-sdk-bedrock-runtime was absent. Bedrock Nova Sonic
speech-to-speech uses InvokeModelWithBidirectionalStream (which boto3 cannot do) via
that package, so realtime requests failed at startup with 'Missing
aws_sdk_bedrock_runtime'. Add the extra to both uv sync stages; it is already in
uv.lock so --frozen resolves, and the package/marker (python>=3.12) matches the
python3.13 image.
Refresh the pinned cgr.dev/chainguard/wolfi-base digest from c61ac6 to
42df77a9 (current wolfi-base:latest, a multi-arch index covering amd64
and arm64). This advances the glibc family from 2.43-r8 to 2.43-r10,
with libcrypto3 and libssl3 from 3.6.3-r2 to r3 and libgcc from
16.1.0-r2 to r4; no packages are added or removed.
The image scan reports CVE-2026-6791 against glibc 2.43-r8 (fixed in
r10). The glibc subpackages are exact-version pinned, so the
in-Dockerfile apk upgrade cannot advance them past the base's baked
revision, which is why refreshing the digest is required. Same six
Dockerfiles as #31133
Re-pins LITELLM_BUILD_IMAGE and LITELLM_RUNTIME_IMAGE across all 6 Dockerfiles
from the prior digests (openssl 3.6.2-r3) to the current chainguard wolfi-base
digest c61ac691 (openssl 3.6.3-r2, >= the fixed 3.6.3-r0). The runtime stage is
the shipped image, so the runtime digest is what actually resolves the
customer-facing CVE; the build image is bumped too for hygiene. Two Dockerfiles
tracked a second equally-stale digest; both are unified onto the patched one.
* fix(docker): use system Node in componentized builders + retry apk add
Two failure modes in the componentized image builds (backend, migrations,
gateway) on project-releaser, with the same root cause:
1. The builder-stage `apk add` was missing `libatomic`. `prisma generate`
triggers prisma-client-py's `nodeenv`, which downloads the latest stable
Node.js at build time. Node 26.1.0 (last passing build on 2026-05-20) did
not dynamically link `libatomic.so.1`. Node 26.2.0 (current latest) does,
and the Wolfi builder doesn't ship libatomic — so `npm install prisma@…`
fails with `node: error while loading shared libraries: libatomic.so.1`
and exit 127. Retrying or pinning the Node version is a treadmill; the
root issue is that nodeenv decides the Node version at build time.
Fix: add `nodejs npm` to the builder-stage `apk add` so prisma-client-py
uses Wolfi's own Node via its default `PRISMA_USE_GLOBAL_NODE=true`. The
legacy `docker/Dockerfile.non_root` already does this; the componentized
Dockerfiles regressed it. Setting `PRISMA_USE_GLOBAL_NODE=true` in ENV
redundantly nails the intent so a future env override can't silently
re-enable nodeenv's download.
2. Transient `apk.cgr.dev` mirror flakes during the arm64 leg of multi-arch
builds cause individual package fetches to fail mid-install (we saw
`nss-db-2.43-r7: remote server returned error (try 'apk update')` and
similar for libzstd1, libogg, binutils in this run). None of the
componentized Dockerfiles wrap `apk add` in a retry loop.
Fix: wrap every `apk add` (builder + runtime, all three files) in the
same `for i in 1 2 3; do … && break || sleep 5; done` loop that the
legacy `docker/Dockerfile.non_root` already uses.
Affected files all have the same shape — backend, migrations, gateway —
because they're three near-identical componentizations of the original
monolithic proxy Dockerfile.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(docker): trim verbose comments on builder Node setup
Same fix, leaner comments. The apk-add note is 3 lines now (was 8), and the
PRISMA_USE_GLOBAL_NODE bullet matches the existing UV_* comment style.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(docker): make apk-add retry loop fail loudly on exhaustion
Greptile flagged that the retry pattern `apk add ... && break || sleep 5`
exits 0 when all three attempts fail, because `sleep 5` is the last
executed command. A persistent apk.cgr.dev outage would produce a silently
"successful" RUN layer with no packages installed, followed by cryptic
"command not found" errors in downstream RUN steps.
Fix: explicitly fail on the third miss before sleeping. Same pattern in
all six retry loops (3 files × builder + runtime).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Yassin Kortam <yassinkortam@Yassins-MBP.localdomain>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Split the monolithic LiteLLM proxy into independently scalable Kubernetes components to allow separate horizontal scaling of the LLM data plane and management API surfaces
- Add DatabaseURLSettings pydantic-settings model that assembles DATABASE_URL (and optional DATABASE_URL_READ_REPLICA) from discrete DATABASE_* env vars before Prisma initializes, supporting both IAM token auth (minting short-lived RDS tokens) and password auth; replaces the CLI-only path that componentized entrypoints bypass
- Add gateway component (port 4000) that trims the proxy route table to the LLM data-plane surface (chat, embeddings, completions, audio, realtime, provider passthroughs, health/metrics) via an allowlist applied inside the lifespan context so plugin-registered routes are captured
- Add backend component (port 4001) that exposes the management/admin surface (keys, users, teams, orgs, spend analytics, model management, SSO, audit logs) with a complementary allowlist
- Add ui component — Next.js static export served by nginx (port 3000) with RSC payload routing, asset prefix aliasing, and SPA fallback for dashboard routes
- Add migrations component with dedicated Dockerfile that runs prisma migrate deploy via a Helm pre-install/pre-upgrade Job, eliminating per-pod schema contention on the Prisma advisory lock
- Add Helm chart (helm/litellm) with separate Deployments, Services, HPAs, and ConfigMap for each component; shared _helpers.tpl emits DATABASE_*, IAM_TOKEN_DB_AUTH, REDIS_*, and DISABLE_SCHEMA_UPDATE env vars from chart values; ingress template routes traffic to the correct component by path prefix
- Add comprehensive tests for DatabaseURLSettings covering IAM auth, password auth, read replica fallbacks, operator-pinned URL preservation, and percent-encoding; add coverage test asserting gateway + backend allowlist union equals the full proxy route set
- Add pydantic-settings>=2.14.1 as a proxy extra dependency and update liccheck allowlist
Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu>
