mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-24 00:52:24 +00:00
d1b160310f
The committed sk-1234 placeholder is in PUBLICLY_KNOWN_MASTER_KEYS, so once an
image ships fe480533e8 the proxy refuses to boot and the quickstart stops
working. It also meant the documented stack came up on port 4000 with a
credential anyone could guess.
Both keys now come from .env and compose refuses to render without them. The
salt key is generated alongside so it stays stable across restarts, which
keeps stored credentials readable.
Verified: no .env -> compose fails closed naming the missing variable; with a
generated .env the stack is healthy, /v1/models returns 200 for the generated
key, 401 for sk-1234 and 401 unauthenticated, /ui/ serves, and the key still
works after a restart.
103 lines
4 KiB
Markdown
103 lines
4 KiB
Markdown
# Docker Development Guide
|
|
|
|
This guide provides instructions for building and running the LiteLLM application using Docker and Docker Compose.
|
|
|
|
> **Just want to run LiteLLM?** This guide builds from source. To run the published
|
|
> image instead, use `docker-compose.quickstart.yml` in this directory — the
|
|
> two-service stack (gateway + Postgres) that the
|
|
> [Docker quickstart](https://docs.litellm.ai/docs/proxy/docker_quick_start) documents:
|
|
>
|
|
> ```bash
|
|
> curl -sSLO https://github.com/BerriAI/litellm/raw/main/docker/docker-compose.quickstart.yml
|
|
> printf 'LITELLM_MASTER_KEY=sk-%s\nLITELLM_SALT_KEY=sk-%s\n' "$(openssl rand -hex 32)" "$(openssl rand -hex 32)" > .env
|
|
> docker compose -f docker-compose.quickstart.yml up -d
|
|
> ```
|
|
|
|
## Prerequisites
|
|
|
|
- Docker
|
|
- Docker Compose
|
|
|
|
## Building and Running the Application
|
|
|
|
To build and run the application, you will use the `docker-compose.yml` file located in the root of the project. This file is configured to use the `Dockerfile.non_root` for a secure, non-root container environment.
|
|
|
|
### 1. Set the Master Key
|
|
|
|
The application requires a `LITELLM_MASTER_KEY` for signing and validating tokens. You must set this key as an environment variable before running the application.
|
|
|
|
Create a `.env` file in the root of the project and add the following line:
|
|
|
|
```
|
|
LITELLM_MASTER_KEY=your-secret-key
|
|
```
|
|
|
|
Replace `your-secret-key` with a strong, randomly generated secret.
|
|
|
|
### 2. Build and Run the Containers
|
|
|
|
Once you have set the `LITELLM_MASTER_KEY`, you can build and run the containers using the following command:
|
|
|
|
```bash
|
|
docker compose up -d --build
|
|
```
|
|
|
|
This command will:
|
|
|
|
- Build the Docker image using `Dockerfile.non_root`.
|
|
- Start the `litellm`, `litellm_db`, and `prometheus` services in detached mode (`-d`).
|
|
- The `--build` flag ensures that the image is rebuilt if there are any changes to the Dockerfile or the application code.
|
|
|
|
### 3. Verifying the Application is Running
|
|
|
|
You can check the status of the running containers with the following command:
|
|
|
|
```bash
|
|
docker compose ps
|
|
```
|
|
|
|
To view the logs of the `litellm` container, run:
|
|
|
|
```bash
|
|
docker compose logs -f litellm
|
|
```
|
|
|
|
### 4. Stopping the Application
|
|
|
|
To stop the running containers, use the following command:
|
|
|
|
```bash
|
|
docker compose down
|
|
```
|
|
|
|
## Hardened / Offline Testing
|
|
|
|
To ensure changes are safe for non-root, read-only root filesystems and restricted egress, always validate with the hardened compose file:
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml -f docker-compose.hardened.yml build --no-cache
|
|
docker compose -f docker-compose.yml -f docker-compose.hardened.yml up -d
|
|
```
|
|
|
|
This setup:
|
|
- Builds from `docker/Dockerfile.non_root` with Prisma engines and Node toolchain baked into the image.
|
|
- Runs the proxy as a non-root user with a read-only rootfs and only writable tmpfs mounts:
|
|
- `/app/cache` (Prisma/NPM cache; backing `PRISMA_BINARY_CACHE_DIR`, `NPM_CONFIG_CACHE`, `XDG_CACHE_HOME`)
|
|
- `/app/migrations` (Prisma migration workspace; backing `LITELLM_MIGRATION_DIR`)
|
|
- Pre-builds and serves the admin UI from read-only paths:
|
|
- `/var/lib/litellm/ui` (pre-restructured Next.js UI with `.litellm_ui_ready` marker)
|
|
- `/var/lib/litellm/assets` (UI logos and assets)
|
|
- Routes all outbound traffic through a local Squid proxy that denies egress, so Prisma migrations must use the cached CLI and engines.
|
|
|
|
You should also verify offline Prisma behaviour with:
|
|
|
|
```bash
|
|
docker run --rm --network none --entrypoint prisma ghcr.io/berriai/litellm:main-stable --version
|
|
```
|
|
|
|
This command should succeed (showing engine versions) even with `--network none`, confirming that Prisma binaries are available without network access.
|
|
|
|
## Troubleshooting
|
|
|
|
- **`build_admin_ui.sh: not found`**: This error can occur if the Docker build context is not set correctly. Ensure that you are running the `docker-compose` command from the root of the project.
|
|
- **`Master key is not initialized`**: This error means the `LITELLM_MASTER_KEY` environment variable is not set. Make sure you have created a `.env` file in the project root with the `LITELLM_MASTER_KEY` defined.
|