Commit graph

79 commits

Author SHA1 Message Date
yuneng-jiang
490c9f9f3f
fix(docker): bump wolfi-base digest for busybox 1.38.0-r1 and openssl 3.6.3-r5 (#37950)
The pinned base (built 2026-07-02) ships busybox 1.37.0-r61 and
libcrypto3/libssl3 3.6.3-r3. Grype reports 16 fixable findings against
those revisions, 8 of them High, so the image-scan gate fails once it
gets past the migration step.

The runtime stage's `apk upgrade` cannot clear them. wolfi-base writes an
exact `=version` constraint for every package it ships into
/etc/apk/world, so `apk upgrade` is a no-op even though the fixed
revisions are in the repo. Advancing them means moving the digest.

The new digest carries busybox 1.38.0-r1, libcrypto3/libssl3 3.6.3-r5
and glibc 2.43-r15, which is at or above the fix revision Wolfi's secdb
records for every finding. Verified with cosign against
chainguard-images/images release.yaml, and grype reports no fixable
findings on the rebuilt image.

CVE-2026-14456, CVE-2026-54876, CVE-2026-38752, CVE-2026-38753,
CVE-2026-38754, CVE-2026-38755
2026-08-22 11:45:39 -07:00
Yassin Kortam
4562b539d8
fix(docker): fail the image build when the generated prisma engine paths drift off /opt/prisma (#35979)
The generated client bakes absolute query engine paths at build time, and
prisma-python scans them before it reads PRISMA_QUERY_ENGINE_BINARY. That
scan propagates EACCES rather than skipping a candidate, so a path baked
under the build user's HOME crashes client startup with a bare
PermissionError for every other uid, and the documented override cannot
recover from it.

Assert in the runtime stage that every baked query engine path sits under
the fixed, world-readable /opt/prisma bake, so a regression in the
generate step breaks the build instead of shipping an image that only
starts under the uid that built it. Adds an image-level test that runs
the same resolution as an arbitrary non-root uid.
2026-08-05 14:12:28 -07:00
yuneng-jiang
487074f602
chore(build): move the Admin UI toolchain to Node 24 (#35801)
* chore(build): move the Admin UI toolchain to Node 24

Node 18 and Node 20 both reached end of life (2025-04-30 and 2026-04-30), and
the release images along with every CI lane were still building on them. Node 24
is the current LTS through 2028-04-30, so this moves the four UI build images,
the CircleCI lanes, and the four GitHub Actions workflows onto it

Node 24 also ships npm 11.17, which is the first line that implements the
min-release-age setting this repo already carries in its .npmrc files. On npm 10
the key is parsed and discarded, so the release-age gate has had no effect
regardless of its value. Tightening the dashboard's engines range and turning on
engine-strict makes an unsupported npm fail loudly rather than skip the gate
quietly, and a new step in the UI build workflow probes an impossible cooldown
so an inert setting cannot pass unnoticed again

Node 24's bundled undici tightened its brand check on RequestInit.signal, which
rejects the AbortSignal jsdom installs and broke the two cases in
src/lib/http/api.test.ts that rebase a request onto a runtime base url. Under
jsdom the Request global comes from Node while AbortSignal comes from jsdom;
tests/jsdomFetchEnv.ts delegates to the jsdom environment and then restores
Node's native AbortController and AbortSignal so both come from one realm.
Upgrading jsdom does not address this, as jsdom still does not own Request

The workflows now read ui/litellm-dashboard/.nvmrc instead of repeating a
literal, so the Node version has a single source of truth, and ui/Dockerfile is
pinned by digest to match the other three build images. The lockfile changes are
npm 11 normalising the engines range and dropping optional peer entries it no
longer records

* fix(build): point every Admin UI build script at .nvmrc

The enterprise Docker path was left on Node 18. docker/build_admin_ui.sh runs
only when enterprise/enterprise_ui/enterprise_colors.json is present, which it
never is in the OSS tree, so neither CI nor a default image build reaches it;
it pinned nvm to v18.17.0 and then built the dashboard, which now requires Node
24, so a customized enterprise image would have failed EBADENGINE

All three UI build scripts now resolve the version from
ui/litellm-dashboard/.nvmrc rather than carrying their own pin, so the Node
version has a single home across Docker, CI, and local builds. build_ui.sh was
on v20 and build_ui_custom_path.sh on v18.17.0

Also drops the dependency-cooldown probe from the UI build workflow. The
engines floor plus engine-strict already fails an unsupported npm loudly at
install time, so the probe was redundant, and treating any nonzero exit from a
live registry call as proof of enforcement made it unsound besides
2026-08-04 12:36:07 -07:00
Yassin Kortam
35dc982692
feat(proxy): add SAML 2.0 SSO for the admin UI (#31429)
litellm already supports Google, Microsoft and generic OIDC SSO through
fastapi-sso, which has no SAML support; AuthMethod.SAML existed only as an
unused enum value. This adds real SAML 2.0 single sign-on for the admin UI.

A new SAMLAuthHandler validates signed assertions with the OneLogin
python3-saml toolkit and maps them onto a CustomOpenID, then reuses the
shared post-login path every other provider goes through, so provisioning,
role/team mapping and the UI session JWT are unchanged. Both SP-initiated
and IdP-initiated HTTP-POST flows are supported. SP-initiated logins are
bound to the browser that started them via an HttpOnly state cookie plus a
cached AuthnRequest id, and the ACS rejects any response whose InResponseTo
doesn't match; unsolicited (IdP-initiated) responses cannot be browser-bound
so they are rejected unless SAML_ALLOW_UNSOLICITED=true. Replays are rejected
by a consumed-assertion guard whose lifetime tracks each assertion's
NotOnOrAfter, and both the replay guard and the login-state binding go
through the proxy's shared in-memory + Redis cache for multi-instance
deployments. The ACS honors DISABLE_ADMIN_UI and re-applies the
free-SSO-user Enterprise gate after the assertion is validated, so an
unvalidated POST can no longer drive the billable-user count query.

SAML is configurable from the admin UI SSO settings (IdP metadata URL or
inline XML, SP entity ID, and an allow-unsolicited toggle), which persists
the SAML_* environment variables the handler reads, exactly like the Google,
Microsoft and generic OIDC providers.

python3-saml is kept as an optional saml extra; its xmlsec and lxml wheels
bundle the native libraries so no system packages are required, and the
import is guarded so the proxy still starts without the package with the
SAML routes returning a clear 501.

Resolves LIT-4016
2026-07-24 12:51:28 -07:00
yuneng-jiang
f7842cdeb7
fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid (#34325)
* fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid

The non_root image baked the prisma CLI and engines under /app/.cache and used
the CLI's default (library) engine mode. Prisma stopped baking the library
engine, so `prisma migrate deploy` fell back to downloading it at startup,
which needs network egress and a writable cache. Under an arbitrary non-root
uid (OpenShift restricted-v2), an air-gapped network, or a readOnlyRootFilesystem,
that download fails and the proxy starts on an empty schema while every DB
endpoint returns 500. The migration entrypoint exits 0 on that failure, so a
default-uid `docker run` with network never surfaced it

Bake to /opt/prisma, a fixed world-readable path no cache mount shadows, and
pin PRISMA_CLI_PATH plus PRISMA_CLI_QUERY_ENGINE_TYPE=binary so the baked binary
engine is used directly, matching Dockerfile and Dockerfile.database. A
build-time guard asserts the binary query engine is present, so a future prisma
change that stops baking it fails the image build instead of silently degrading
migrations

Adds docker/test_offline_migration.sh, run from image-scan, which migrates a
fresh Postgres with no egress as a non-root uid and asserts the schema was
created, the case a default-uid `docker run` with network cannot catch

* test(docker): move the offline migration check into a gated pytest and stop pinning XDG_CACHE_HOME at the read-only bake

The offline migration check lived in docker/ as a shell script. It now lives in
tests/proxy_migration_tests/ as a pytest gated on LITELLM_IMAGE, matching the
sibling schema-migration test gated on DATABASE_URL, and image-scan invokes it
with pytest instead of bash. It also asserts the migration entrypoint's exit
code alongside the table count, so a crash or a container-startup failure fails
loudly rather than only surfacing as a low table count

Runtime XDG_CACHE_HOME pointed at /opt/prisma/.cache, which is baked a+rX with
no write, so any XDG-aware library writing a cache at runtime would be denied
for every uid. Leave it unset so it falls back to $HOME/.cache (/app/.cache,
created here and owned by the runtime uid), matching Dockerfile and
Dockerfile.database which never pin XDG at runtime. A second test guards against
a future edit pointing a cache or home var back at the read-only bake
2026-07-22 23:03:39 -07:00
yuneng-jiang
111d447e1b
fix(docker): restore litellm-proxy-extras source dir in runtime images (#33592)
* fix(docker): restore litellm-proxy-extras source dir in runtime images

#30243 narrowed the runtime stage to an allowlist COPY, which dropped
/app/litellm-proxy-extras from the published images. Downstream
migration jobs point prisma migrate deploy at that path; with the
schema gone (or a schema with no adjacent migrations dir, where prisma
exits 0 without applying anything) those jobs went green while never
migrating the database. Restore the folder in all three runtime stages
and assert in image-scan that the schema and a non-empty migrations dir
ship at the source path

* chore(ci): drop image-scan migration-assets assertion
2026-07-16 15:04:55 -07:00
Yuneng Jiang
4cc4846ff6
fix(docker): bump wolfi-base digest for glibc 2.43-r10
Refresh the pinned cgr.dev/chainguard/wolfi-base digest from c61ac6 to
42df77a9 (current wolfi-base:latest, a multi-arch index covering amd64
and arm64). This advances the glibc family from 2.43-r8 to 2.43-r10,
with libcrypto3 and libssl3 from 3.6.3-r2 to r3 and libgcc from
16.1.0-r2 to r4; no packages are added or removed.

The image scan reports CVE-2026-6791 against glibc 2.43-r8 (fixed in
r10). The glibc subpackages are exact-version pinned, so the
in-Dockerfile apk upgrade cannot advance them past the base's baked
revision, which is why refreshing the digest is required. Same six
Dockerfiles as #31133
2026-07-06 14:15:56 -07:00
tin-berri
52e5b3ae98
build(docker): build the Admin UI from source in a build-platform-pinned stage (#31130)
The monolith images shipped whatever UI bundle was committed to
litellm/proxy/_experimental/out, so refreshing the UI for a release meant
running build_ui.sh out of band and committing the regenerated bundle. Add a
ui-builder stage to all three monolith Dockerfiles (root, database, non_root)
that compiles the Next.js static export from this exact source and replaces the
committed bundle before the final uv sync.

The stage is pinned with FROM --platform=$BUILDPLATFORM so the
architecture-independent static export compiles once on the native builder even
in a multi-arch (linux/amd64,linux/arm64) build, rather than once per target
arch under QEMU emulation. The destination is cleared before the COPY because
COPY merges directories and would otherwise leave the committed bundle's
content-hashed chunks behind alongside the fresh ones. build_admin_ui.sh still
runs afterward so the enterprise custom-color override is preserved.

The UI base image is pinned by digest to match LITELLM_BUILD_IMAGE,
LITELLM_RUNTIME_IMAGE and UV_IMAGE, and .dockerignore now excludes the local
.next/out so a developer's build artifacts never enter the context.
2026-06-25 23:41:08 -07:00
ishaan-berri
d8ef1da49d
feat: package Rust OCR bridge in LiteLLM wheel (#31267)
* feat: package rust ocr bridge in litellm wheel

* Install Rust in Windows CircleCI job

* Address Rust wheel review feedback

* Pin Windows rustup installer hash
2026-06-25 12:32:55 -07:00
yucheng-berri
fda08dd727
fix(docker): bump wolfi-base digest to patch openssl CVE-2026-34182 (#31133)
Re-pins LITELLM_BUILD_IMAGE and LITELLM_RUNTIME_IMAGE across all 6 Dockerfiles
from the prior digests (openssl 3.6.2-r3) to the current chainguard wolfi-base
digest c61ac691 (openssl 3.6.3-r2, >= the fixed 3.6.3-r0). The runtime stage is
the shipped image, so the runtime digest is what actually resolves the
customer-facing CVE; the build image is bumped too for hygiene. Two Dockerfiles
tracked a second equally-stale digest; both are unified onto the patched one.
2026-06-23 17:51:25 -07:00
yuneng-jiang
1fd8ab4c5f
fix(docker): copy only runtime artifacts into the final image (#30243)
* fix(docker): copy only runtime artifacts into the final image

The runtime stage previously copied the builder's entire /app workspace,
shipping the full source tree and dependency manifests (uv.lock, the
dashboard package-lock.json, ui/) in the published images. Tools that
read manifests found in an image attribute every pin in them to the
image, including packages that are never installed, which produces
recurring false reports against the official images.

The runtime stage now copies an explicit allowlist: the venv (where the
application is installed), the entrypoint scripts, schema.prisma,
prisma_migration.py (invoked by source path from entrypoint.sh), and
the prisma binary caches. npm and its globally installed helper
packages are dropped from the runtime stage; node stays for the prisma
CLI. The database image's /root/.cache copy is narrowed to the prisma
subdirs, matching the main Dockerfile, which removes the uv build cache
from that image (-1.1GB).

Verified on locally built images for all three variants against a
contract suite that passes 100% on the v1.88.1 baselines: byte-identical
venv vs old-Dockerfile builds from the same commit, DB-less and
Postgres-backed boots, migration entrypoint, real completion and
streaming calls, admin UI, key generation, non-root UID behavior.
Image sizes: main 1.71GB -> 1.53GB, database 2.68GB -> 1.53GB

* fix(docker): restore enterprise source dir required by runtime imports

litellm/proxy/hooks/__init__.py, callback_utils.py and
customer_endpoints.py import enterprise.* by source path, resolved via
the cwd entry proxy_cli appends to sys.path, with silent ImportError
fallbacks. Dropping /app/enterprise from the runtime image emptied
ENTERPRISE_PROXY_HOOKS and broke managed files (e2e_openai_endpoints
caught it). Restore the directory in all three runtime stages
2026-06-11 23:46:23 -07:00
Mateo Wang
48d7e15b83
chore(admin-ui): regenerate static export with trailingSlash: true (#28112)
* chore(admin-ui): regenerate static export with trailingSlash: true

Rebuilds litellm/proxy/_experimental/out/ from ui/litellm-dashboard with
`trailingSlash: true` enabled in next.config.mjs. Next.js now emits every
route as <dir>/index.html (e.g. mcp/oauth/callback/index.html) instead of
<dir>.html with a sibling metadata-only directory, which fixes the 404 on
extensionless URLs served through FastAPI's StaticFiles(html=True) mount.

This is the build artifact half of the fix; the config change, Dockerfile
cleanup, and regression test live in the follow-up source PR that stacks
on top of this branch.

* fix(admin-ui): emit nested routes as <dir>/index.html (#28106)

Linear and other OAuth providers redirect the user back to
/ui/mcp/oauth/callback?code=...&state=... after the consent step. The
packaged Next.js static export only produced /ui/mcp/oauth/callback.html,
so FastAPI's StaticFiles served a 404 on the extensionless URL and the
OAuth handshake never completed.

The Dockerfile.non_root build step tried to paper over this at image-build
time with `for html_file in *.html; do ...`, but that shell glob does not
recurse, so nested routes like mcp/oauth/callback.html were left stranded
next to an empty mcp/oauth/callback/ directory containing only Next.js
metadata. The runtime restructure step in proxy_server.py was then skipped
because the .litellm_ui_ready marker had already been dropped.

Set trailingSlash: true in the dashboard's Next.js config so the export
emits every nested route as <dir>/index.html natively. The Dockerfile loop
is now a no-op for the bundled UI and has been removed; the
.litellm_ui_ready marker is still written so the proxy keeps skipping the
redundant Python restructure step at startup. Stacks on top of the static
export regeneration in the parent branch.

* chore: restore origin/litellm_internal_staging out files
2026-05-25 21:06:50 -07:00
yuneng-jiang
f1abe03ed6
fix(docker): restore npm to non_root builder image (#28519)
The non_root builder stage installs `nodejs` but not `npm`. Without `npm`
on PATH, prisma-python falls back to downloading a Node runtime via
nodeenv from nodejs.org, and that downloaded binary fails to load
`libatomic.so.1` — breaking `prisma generate` and the image build.

`npm` was dropped from this apk list in ca52e346b0. Restoring it lets
prisma-python use the system Node + npm, matching docker/Dockerfile
which already installs `npm` for the same reason.
2026-05-21 17:02:42 -07:00
Yassin Kortam
451ce161fc fix: remove separate health app 2026-05-07 16:04:56 -07:00
Yuneng Jiang
4ee586a321
[Fix] Docker: Remove Hardcoded Prisma Binary Target For Multi-Arch Builds
PRISMA_CLI_BINARY_TARGETS="debian-openssl-3.0.x" was hardcoded in
docker/Dockerfile.non_root by #17695. On a buildx linux/arm64 leg this
forces prisma to download the amd64 schema-engine into an arm64 image,
so 'prisma migrate deploy' fails at startup with 'Could not find
schema-engine binary'.

Removing the env lets prisma auto-detect per build platform: amd64
builds still resolve to debian-openssl-3.0.x (Wolfi falls back to
debian, same binary as before), and arm64 builds now correctly fetch
linux-arm64-openssl-3.0.x. The offline-cache pre-warm goal of #17695 is
preserved — only which binaries fill the cache changes.

Fixes #19458
2026-05-04 21:37:16 -07:00
Yuneng Jiang
08d130a8fe
[Fix] Docker: Pin Wolfi And Uv To Multi-Arch Index Digests
The previous pins resolved to single-platform amd64 manifests, so buildx
pulled the same amd64 base for both linux/amd64 and linux/arm64 targets.
The published OCI index then advertised an arm64 entry whose layers are
byte-identical to amd64 -- arm64 users got an amd64 binary.

Switch all three Dockerfiles to the multi-arch image-index digests:
  - cgr.dev/chainguard/wolfi-base   (index has linux/amd64 + linux/arm64)
  - ghcr.io/astral-sh/uv:0.11.7     (index has linux/amd64 + linux/arm64)

Resolved with `docker buildx imagetools inspect <ref>` -- that returns
the index digest. `docker pull` + `docker inspect` returns the per-host
platform digest, which is what slipped in last time.
2026-05-04 09:55:53 -07:00
user
5ba6bc0784
chore(deps): bump uv to 0.11.7 + drop dead npm sed
- UV_IMAGE across all Dockerfiles: 0.10.9 -> 0.11.7.
- Loosen `required-version` in enterprise/ and litellm-proxy-extras/
  from strict `==0.10.9` to `>=0.10.9` so the new Docker image can
  build those workspace members. Matches the main pyproject range.
- Drop the `sed` block that rewrote tar/minimatch version ranges in
  npm's bundled package.json files. The override loop above already
  swaps the vendored directories on disk; npm doesn't re-resolve at
  runtime, so the sed was cosmetic.
2026-04-24 00:36:59 +00:00
user
1cc935e3e4
chore(deps): bump Wolfi base digest to latest
Chainguard rebuilds wolfi-base nightly with picked-up security patches.
The current pin is from 2026-04-01; this moves to the latest digest as
of 2026-04-24 to pick up ~3 weeks of accumulated OS package updates
(openssl, glibc, nodejs apk, etc.).
2026-04-24 00:36:59 +00:00
michelligabriele
c67d193400
fix(docker.non_root): use numeric UID 65534 for K8s runAsNonRoot (#26268) 2026-04-22 18:00:04 -07:00
Yuneng Jiang
995bff0dec
[Fix] Dockerfile.non_root: drop prisma --version sanity call
prisma --version invokes the Schema Engine, which has no binary for the
Wolfi base image (only debian). In the baseline this was silenced by a
trailing || true wrapping the whole prisma chain; removing that wrapper
uncovered the failure on arm64 builds. The main Dockerfile does not
call prisma --version at all, so drop it here to match — prisma generate
is sufficient to validate the toolchain.
2026-04-20 13:51:53 -07:00
Yuneng Jiang
db49885102
[Refactor] Dockerfile.non_root: drop UI drift guard approach
Revert the .dockerignore ui/ exclusion and remove the UI Drift Guard
workflow. _experimental/out/ refresh is already handled by the release
runbook; the global .dockerignore change also broke Dockerfile.custom_ui
(explicit COPY ./ui/litellm-dashboard) and the enterprise-colors inline
rebuild path in Dockerfile, Dockerfile.database, and Dockerfile.dev.

Dockerfile.non_root itself is unchanged functionally — still stages the
UI from the checked-in _experimental/out/. Only the companion workflow
and global dockerignore exclusion are dropped.
2026-04-20 13:47:18 -07:00
Cursor Agent
4c8cbaf0a2
[Refactor] Dockerfile.non_root: drop dead lines and shrink build context
Five small, individually-verified cleanups collected into one commit:

- Drop 'prisma migrate diff --from-empty ... > /dev/null 2>&1 || true'
  from the builder. Stdout/stderr/exit-status all discarded; nothing
  reads the output. Dead line.
- Drop 'mkdir -p /app/.cache/npm' from the same RUN. npm is gone.
- Drop the runtime's redundant 'sed -i' + 'chmod +x' on the entrypoint
  scripts. The builder already does the same three lines, and the
  runtime copies /app from the builder via COPY --from=builder, so
  the normalized files (and exec bits, which buildkit preserves) are
  already in place.
- Drop NPM_CONFIG_CACHE and NPM_CONFIG_PREFER_OFFLINE from the runtime
  ENV — nothing reads them after Task 2.2 removed npm.
- Drop '/.npm' and '/tmp/.npm' from the runtime's mkdir + chown. These
  directories only existed as npm's writable dirs for the non-root
  user; npm is gone.

.dockerignore: add 'ui/'. After Task 2.1 the non_root image sources
its UI bytes from litellm/proxy/_experimental/out/, so the whole
ui/litellm-dashboard/ source tree is dead weight when the blanket
'COPY . .' pulls it into /app. Verified (with ripgrep) that no Python
code under litellm/ opens any file under ui/. All string references to
'ui/...' are URL paths, not filesystem paths.

Final image size: 6.57GB baseline -> 1.96GB. API parity and UI visual
regression match baseline across all 12 API scenarios and 10 UI
routes. Trivy HIGH/CRITICAL: 6 -> 2, no new CVEs introduced.

Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
2026-04-19 06:50:50 +00:00
Cursor Agent
e24c02f478
[Infra] Dockerfile.non_root: add BuildKit uv cache mount
Mount /app/.cache/uv as a BuildKit type=cache on both 'uv sync' steps.
The cache persists across builds on the same builder (and, when used
with type=gha in CI, across CI runs) so repeat builds don't re-download
every wheel.

Side-effect: because the cache lives outside the image layer, the
~742MB of downloaded wheel archives that were previously baked into
/app/.cache/uv drop out of the final image. Compressed image size
goes from ~5.0GB to ~3.7GB, and the 'USER nobody' prisma-generate
layer is 1.7GB vs 2.4GB.

Warm-build timing: a uv-sync-invalidating edit now takes ~1m30s vs
~2m39s without the cache mount, on this dev VM.

API parity and UI visual regression continue to match baseline.
Trivy HIGH/CRITICAL: 6 at baseline -> 2 now, no new CVEs.

Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
2026-04-19 06:35:39 +00:00
Cursor Agent
ca52e346b0
[Infra] Dockerfile.non_root: slim C toolchain in builder stage
After Task 2.1 removed the in-image Next.js build, the builder stage no
longer needs a full C/C++ + Clang toolchain. Keep gcc + python3-dev
(required to compile ml-dtypes 0.4.1 from source — no wheel published
for Python 3.13 yet). Drop everything else.

Removed from apk: clang, llvm, lld, linux-headers, build-base,
openssl-dev, npm. Removed NVM_DIR env and /root/.nvm from PATH
(no nvm-based Node install anymore).

Kept: python3, python3-dev, gcc, bash, coreutils, curl, openssl,
libsndfile, nodejs. gcc (15.2) serves both C and C++; the separate
g++ package doesn't exist in Wolfi.

Image size unchanged (builder stage doesn't end up in the runtime);
cold builds slightly slower due to ml-dtypes source compile, but that
will be recovered in the next task via a BuildKit uv cache mount.
API parity and UI visual regression both match baseline, Trivy
HIGH/CRITICAL CVE count unchanged from opt-2 (4 CVEs, none new).

Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
2026-04-19 05:59:31 +00:00
Cursor Agent
78485f5a32
[Infra] Dockerfile.non_root: remove unused npm from runtime stage
npm was installed in the runtime only to globally install vulnerability
patched versions of tar/glob/brace-expansion/minimatch/diff and to
in-place rewrite npm's own bundled package.json. Both were to silence
CVE scanners against modules that ship with npm itself.

Since we no longer run npm anywhere in the runtime (Prisma uses the
node binary directly for migrate deploy and generate), we can just
skip installing npm in the first place. This eliminates both the
~25-line CVE-patch shuffle AND the underlying CVE surface.

Kept: nodejs (needed by prisma-python's CLI and migrate deploy).
Removed: npm apk package, all 'npm install -g', all find+sed patching,
the redundant 'apk upgrade --no-cache nodejs' (already covered by the
preceding 'apk upgrade').

Image: 4.97GB (opt-1) -> 4.97GB (opt-2); the real win is that two
CVEs (CVE-2026-33671 and GHSA-q4gf-8mx6-v5v3) drop off the Trivy
HIGH/CRITICAL list. No new CVEs introduced. API parity and UI
visual regression both match baseline.

Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
2026-04-19 05:11:06 +00:00
Cursor Agent
004c6b1b3e
[Infra] Dockerfile.non_root: stage pre-built UI from _experimental/out
The checked-in Next.js static export at litellm/proxy/_experimental/out/
is kept fresh by the UI Drift Guard CI workflow. Stage it directly
instead of re-running npm ci + npm run build inside the image.

This removes: nvm install, node 20.20.2 install, npm ci (801 pkgs),
next build, and the resulting intermediate node_modules/out tree.

Build time: ~6m25s -> ~2m (fuse-overlayfs DinD); image 6.57GB -> 5.0GB.
Behavior parity verified: API endpoints, UI screenshots (all 10 routes
pixel-perfect), and Trivy HIGH/CRITICAL CVE count (6 -> 5, one npm
GHSA removed) all match or improve over baseline.

Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
2026-04-19 04:21:46 +00:00
stuxf
a6c30b30bf
build: migrate packaging, CI, and Docker from Poetry to uv (#25007)
* build: migrate packaging metadata to uv

* ci: move automation and local tooling to uv

* docker: migrate image builds and runtime setup to uv

* docs: update install and deployment guidance for uv

* chore: align auxiliary scripts and tests with uv

* test: harden test_litellm isolation

* fix: keep release and health check images self-contained

* build: pin uv tooling and health check deps

* test: isolate bedrock image request formatting from suite state

* test: cover sandbox executor requirements flow

* ci: fix circleci no-op command steps

* ci: fix circleci publish workflow parsing

* fix: stabilize remaining uv migration CI checks

* ci: increase matrix test timeout headroom

* fix: restore published docker and license coverage

* fix: restore proxy runtime build parity

* fix: restore proxy extras parity and venv migrations

* ci: persist uv path across circleci steps

* fix: keep psycopg binary in default test env

* docker: preserve prisma cache across stages

* test: run local proxy checks through uv python

* build: restore runtime deps moved into ci

* build: refresh uv lock after upstream merge

* fix: restore module import in test_check_migration after merge

The conflict resolution imported only the function but the test body
references check_migration as a module throughout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching

- Move google-generativeai, Pillow, tenacity back to ci group (they are
  lazily imported and bloat the base SDK install needlessly)
- Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant
  in Docker where system Node.js is already installed via apk)
- Remove all nodejs-wheel node replacement and venv npm patching blocks
  from Dockerfiles since the wheel is no longer installed
- Add --no-default-groups to CodSpeed benchmark workflow so the benchmark
  environment matches the old minimal pip install footprint
- Apply standard uv two-phase Docker pattern: copy metadata first, install
  deps (cached layer), then copy source and install project
- Replace CircleCI enterprise no-op with proper uv sync command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate uv.lock after removing nodejs-wheel-binaries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): use cache/restore instead of cache to prevent cache poisoning

The old workflow used actions/cache/restore (read-only). The uv migration
changed it to actions/cache (read-write), which zizmor flags as a cache
poisoning risk. Restore the safer read-only variant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert

The setup-uv action enables caching by default, which zizmor flags as a
cache poisoning risk. Disable it since we already use a read-only
cache/restore step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv cache in publish workflow

Silences zizmor cache-poisoning alert. Publishing workflow runs
infrequently on protected branches so caching adds no real benefit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): remove duplicate verbose_logger mock in test_check_migration

The logger was patched twice — first via mocker.patch() then via
mocker.patch.object(autospec=True). The second call fails because
autospec cannot inspect an already-mocked attribute. Remove the
redundant first patch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): free disk space before Docker build in test-server-root-path

The Dockerfile.non_root build ran out of disk on the CI runner. Remove
Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 11:46:23 -07:00
joereyna
41407d0287
Fix node-gyp symlink path after npm upgrade in Dockerfile 2026-04-07 17:01:55 -07:00
Yuneng Jiang
537727f0da
[Fix] Dockerfile.non_root: handle missing .npmrc gracefully
The .npmrc file (ignore-scripts=true, min-release-age=3d) is temporarily
removed during the Docker build since lifecycle scripts are needed by
npm ci. However, the unconditional `mv` fails when the build context
doesn't include .npmrc (e.g. when LiteLLM is vendored in a subdirectory).

Make all .npmrc mv operations conditional. This is safe because npm ci
already installs from package-lock.json with pinned versions and
integrity hashes.
2026-04-07 12:44:04 -07:00
Sameer Kankute
08df8643bf
fix(docker): include enterprise bridge in non-root runtime image (#24917)
Copy the /app/enterprise bridge package into the non-root runtime image so enterprise proxy hooks register correctly (including managed_files).
2026-04-04 14:04:31 -07:00
Yuneng Jiang
fa629c307c
[Fix] Dockerfile.non_root: install node-gyp for npm ci
The wolfi-base npm@11.12.1 package does not bundle node-gyp, causing
`npm ci` to fail with `Cannot find module 'node-gyp/bin/node-gyp.js'`
when building the Admin UI in the non-root Docker image.

Install node-gyp@12.2.0 globally and symlink it into npm's internal
node_modules where @npmcli/run-script expects to find it.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 17:29:47 -07:00
Yuneng Jiang
85f72c9d24
[Fix] Remove unused aioboto3 dependency and botocore conflict workarounds
aioboto3 was listed as a dependency for async sagemaker calls but is not
imported anywhere in the codebase — async calls use httpx + botocore SigV4
instead. Removing it eliminates the unresolvable botocore version conflict
between boto3 and aiobotocore, along with all grep -v / --no-deps workarounds
across Dockerfiles and CI.

Also addresses Greptile review feedback: collapse redundant grpcio
python-version markers, bump pyproject.toml cryptography to 46.0.5 to
match Docker (GHSA-r6ph-v2qm-q3c2), and fix misleading .npmrc comment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 14:25:44 -07:00
Yuneng Jiang
821a634d25
[Fix] Handle boto3/aioboto3 botocore conflict across CI and Docker builds
boto3==1.42.80 and aioboto3==15.5.0 have incompatible botocore version
ranges. No aioboto3 release supports botocore 1.42.x yet. Both uv and
pip 26.0.1 reject the resolution.

Fix: filter aioboto3 out of requirements.txt at install time, then
install aioboto3+aiobotocore with --no-deps to bypass resolution.
Added wrapt and aioitertools to requirements.txt as pinned transitive
deps of aiobotocore (skipped by --no-deps). Fixed pip stdin handling
(/dev/stdin). Applied to all 5 Dockerfiles and all CircleCI install
paths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 12:27:21 -07:00
Yuneng Jiang
5f63873dca
[Infra] Pin all Docker build dependencies to exact versions
Pin every dependency across all Docker builds so upgrades are intentional.
Verified by building all 3 production images and diffing pip freeze against
known-good v1.83.0-nightly baselines — zero version drift.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 00:05:39 -07:00
stuxf
7066c895f6
chore: harden npm supply chain — pin overrides, enforce npm ci, add ignore-scripts (#24838)
* chore: harden npm supply chain — pin overrides, enforce npm ci, add ignore-scripts

Replace open-ended >= version overrides with exact pins matching lockfile
versions across all 6 package.json files. Remove dead overrides for packages
not present in lockfiles. Switch CI and devcontainer from npm install to
npm ci for deterministic lockfile-based installs.

Add .npmrc to all 7 JS project directories with ignore-scripts=true (blocks
postinstall RAT vectors like the axios@1.14.1 supply chain attack) and
min-release-age=3d (refuses packages published <3 days ago, requires npm
>=11.10). Remove Yarn-only resolutions field from docs/my-website.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: bump sharp to 0.33.5 in docs, add docs .npmrc

sharp 0.32.x uses postinstall to download native binaries, which breaks
with ignore-scripts=true. sharp 0.33+ distributes via optionalDependencies
instead, making it compatible with the new .npmrc hardening.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove docs .npmrc to fix Vercel deploy

Vercel's build for docs/my-website uses npm install which needs
sharp 0.32.6's postinstall script. Since we don't control Vercel's
build process, remove the .npmrc from docs rather than fight it.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: Dockerfile npm ci + nvm checksum verification

- Replace npm install with npm ci in Dockerfile.non_root,
  Dockerfile.custom_ui, and spend-logs/Dockerfile for deterministic
  lockfile-based installs
- Replace curl-pipe-bash nvm install with download-then-verify pattern
  in build_admin_ui.sh, build_ui.sh, and build_ui_custom_path.sh
- Update nvm from v0.38.0 (2021) to v0.40.4 (Jan 2026) with SHA256
  checksum verification before execution

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: macOS sha256sum compat + clarify min-release-age scope

- Use shasum -a 256 fallback on macOS where sha256sum is unavailable
- Clarify in .npmrc comments that min-release-age only protects local
  npm install, not npm ci (used in CI)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 13:41:37 -07:00
yuneng-jiang
d3587b1d8e fix: bump PyJWT to 2.12.0 in all Dockerfiles and tar to 7.5.11
All Dockerfiles were pinning PyJWT 2.9.0 (Dockerfile, Dockerfile.database,
Dockerfile.dev) or had a stale wheel build for 2.9.0 (Dockerfile.non_root).
Updated to 2.12.0 to match pyproject.toml. Also bumps tar to 7.5.11 in
Dockerfile.non_root for security.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 19:54:54 -07:00
yuneng-jiang
1f485007fb fix: update PyJWT pin in Dockerfile.non_root to 2.12.0
The wheels directory contains 2.12.0 after the pyproject.toml bump,
so the hardcoded 2.10.1 pin fails at build time.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 19:53:52 -07:00
Krish Dholakia
e7714f0ce6
Fix CVEs: bump tar/minimatch/pypdf + harden Docker SBOM patching (#23082)
* fix(docker): bump tar/minimatch/pypdf for CVE fixes + harden SBOM patching

- Bump tar 7.5.8→7.5.10, minimatch 10.2.1→10.2.4, pypdf 6.6.2→6.7.3
- Add sed-based SBOM metadata patching with properly indented find/sed
- Add npm package manager cleanup (apk del / apt-get purge) to remove
  stale SBOM entries from image scanners
- Scope || true to only apk del via brace grouping { ... || true; }
- Guard npm root -g with non-empty assertion to prevent silent failures
- Scope minimatch sed regex to ^10.x to avoid matching other major versions

Addresses: CVE-2026-27903, CVE-2026-27904, GHSA-qffp-2rhf-9h96, CVE-2026-27888

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(docker): scope find to /usr/local/lib /usr/lib, drop autoremove

- Replace `find /` with `find /usr/local/lib /usr/lib` to avoid
  traversing /proc, /sys, /dev during SBOM metadata patching
- Remove `apt-get autoremove -y` from Debian-based Dockerfiles to
  prevent nodejs from being removed as an auto-installed dependency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:31:27 -08:00
Harshit28j
3e6c10a071 security: fix critical/high CVEs in OS-level libs and NPM transitive 2026-02-24 19:40:09 +05:30
Achilleas Athanasiou Fragkoulis
cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492)
Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi
2026-02-12 19:39:04 +05:30
Harshit Jain
3b043ee8bf
fix critical CVE vulnerabliltes (#20683) 2026-02-07 22:23:01 -08:00
Ishaan Jaffer
ef66a6cb62 fix security scans 2026-02-07 11:15:02 -08:00
yuneng-jiang
7831e30666 keep package-lock.json in non-root 2026-02-04 17:58:22 -08:00
Ishaan Jaffer
a002907389 fix tar security issue with TAR 2026-01-31 11:46:53 -08:00
milan-berri
8fcdf6105f
fix: run prisma generate as nobody user in non-root container (#20000)
Fixes permission error where prisma generate fails with 'Permission denied'
when trying to write schema.prisma in non-root containers.

The issue was that prisma generate was running as root before switching
to nobody user, causing generated files to be owned by root:root.
Moving prisma generate after USER nobody ensures files are owned by
nobody:nobody and can be written to during runtime.

Fixes #19859
2026-01-29 19:04:59 -08:00
yuneng-jiang
1bf32deb6c Adding python3-dev to non root 2026-01-22 10:05:09 -08:00
Ishaan Jaff
f98814ba8a
fix include proxy/prisma_migration.py in non root (#18971) 2026-01-12 08:12:39 -08:00
Alexsander Hamir
1544e8f971
feat: Add line_profiler support for performance analysis and fix Windows CRLF issues in Docker builds (#18773) 2026-01-07 11:36:57 -08:00
yuneng-jiang
05dd247ff5 Fix UI disappearing for development instances 2025-12-23 15:24:07 -08:00
yuneng-jiang
6bb5254c9b
Revert "[Fix] UI - Disappears in Development Environments" 2025-12-23 15:08:07 -08:00