Merge remote-tracking branch 'origin/main' into litellm_autorouter_baseline_drop_cache_rate_fold
Some checks failed
Cost map pins recheck / rerun-cost-map-guard (push) Has been cancelled
LiteLLM Rust DB / db-queries (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled

# Conflicts:
#	litellm/utils.py
This commit is contained in:
mateo-berri 2026-11-14 05:05:34 -08:00
commit f87bfdcefb
10156 changed files with 927971 additions and 369928 deletions

View file

@ -1,54 +0,0 @@
---
name: database-access
description: Add database queries, extract existing Prisma calls into repositories, or explicitly port queries to Rust and SQLx without expanding migration scope
metadata:
internal: true
---
# Database access
## Adding new database access
- Put Python database access behind an operation in `litellm/repositories`
- Give callers typed inputs and outputs, not Prisma clients, models, or query dictionaries
- Keep query execution and backend selection inside the repository
- Prisma is fine for the implementation, adding a query does not require a Rust port
- Native Rust callers use `litellm-rust/crates/db` directly
## Asked to move existing access behind a repository
- Identify the calls being moved and their transaction boundary
- Move the existing Prisma implementation into the relevant repository
- Replace those direct database calls with repository operations
- Preserve return values, errors, and transaction behavior, with focused regression tests
- Keep this an extraction, not a backend migration or unrelated cleanup
## An equivalent Rust or SQLx query already exists
- Leave it and the current backend selection unchanged unless integration is explicitly requested
- Its existence alone does not justify switching a Prisma caller to Rust
- Repository extraction can keep using Prisma even when the Rust query is available
## Asked to port a Prisma query to Rust or SQLx
- Identify the existing inputs, outputs, errors, and transaction boundary
- Check `litellm-rust/crates/db` for an equivalent query and reuse it if its behavior matches
- Otherwise follow `litellm-rust/crates/db/AGENTS.md`
- Put static SQL in `queries/` and compile-check it with SQLx
- Keep typed inputs and reusable row projections in `src/`
- Refresh SQLx query metadata after SQL changes
- Keep Python conversion in the Python bridge, not in `litellm-rust/crates/db`
- When caller integration is requested, expose the Rust operation through the bridge and call it from the repository
- A query-only port can land without switching callers
- Verify behavior against real PostgreSQL before switching callers
- Check relevant null handling, conflicts, ordering, errors, and transaction semantics
- Remove the old implementation only when no callers need it
## Migration context and boundaries
- Long-term direction is to centralize Python database access in repositories, then replace their Prisma implementations incrementally with Rust APIs
- Mixed Prisma and Rust implementations are expected during the transition, unrelated existing calls can stay as they are
- Both backends may have separate pools through PgBouncer, size them together and verify pooling-mode compatibility
- PgBouncer does not share transactions between Prisma and SQLx, keep each atomic unit of work on one backend and transaction
- Never retry a failed Rust write through Prisma when it may already have committed
- Prisma owns schema migrations today, eventual SQLx migration ownership and Prisma removal are separate planned steps, not part of a query port

File diff suppressed because it is too large Load diff

View file

@ -1,68 +0,0 @@
#!/usr/bin/env bash
set -uo pipefail
category="${1:?usage: classify_changes.sh <backend|client|ui|cost-map-only|mcp-dependencies|windows-release|redis-compat>}"
has_client=false
has_backend=false
has_ci=false
has_cost_map=false
has_mcp_dependencies=false
has_windows_release=false
has_redis_compat=false
outside_cost_map_set=false
while IFS= read -r file || [ -n "$file" ]; do
[ -n "$file" ] || continue
case "$file" in
litellm/_redis.py | litellm/_redis_credential_provider.py | litellm/caching/redis_cache.py | litellm/caching/evicted_client_closer.py | tests/unit/test_redis.py | tests/integration/sdk/test_redis_cluster_iam_auth.py | tests/unit/caching/test_redis_connection_pool.py | tests/unit/caching/test_redis_cluster_cache.py | tests/unit/caching/test_evicted_client_closer.py | .github/workflows/test-unit.yml | .github/actions/detect-changes/* | .github/scripts/detect_changes.sh | .circleci/scripts/classify_changes.sh | tests/integration/run.py | pyproject.toml | uv.lock)
has_redis_compat=true ;;
esac
case "$file" in
*.md | *.mdx) : ;;
pyproject.toml | */pyproject.toml | uv.lock | uv.toml | .python-version | rust-toolchain.toml | litellm-rust/* | litellm/__init__.py | litellm/proxy/proxy_server.py | litellm/*mcp* | tests/*mcp* | litellm/integrations/arize/* | tests/base_sdk_tests/* | scripts/check_mcp_sdk_install.py | .github/workflows/test-mcp-dependency-resolution.yml | .github/actions/detect-changes/* | .github/actions/setup-uv-with-retries/* | .github/actions/cache-cargo-build/* | .github/scripts/detect_changes.sh | .github/scripts/uv_sync_with_retries.sh | .circleci/scripts/classify_changes.sh | tests/unit/test_circleci_path_filter.py | tests/unit/test_detect_changes.py)
has_mcp_dependencies=true ;;
esac
case "$file" in
litellm-rust/* | litellm/rust_bridge/* | rust-toolchain.toml | pyproject.toml | uv.lock | tests/windows_tests/* | .circleci/*)
has_windows_release=true ;;
esac
case "$file" in
ui/* | tests/e2e/ui/*) has_client=true ;;
docs/* | *.md | *.mdx) : ;;
.github/* | .circleci/*) has_ci=true; has_backend=true ;;
*) has_backend=true ;;
esac
case "$file" in
model_prices_and_context_window.json | litellm/model_prices_and_context_window_backup.json)
has_cost_map=true ;;
tests/unit/proxy/*) : ;;
*) outside_cost_map_set=true ;;
esac
done
case "$category" in
mcp-dependencies)
[ "$has_mcp_dependencies" = true ] && echo run || echo skip
;;
cost-map-only)
{ [ "$has_cost_map" = true ] && [ "$outside_cost_map_set" = false ]; } && echo run || echo skip
;;
windows-release)
[ "$has_windows_release" = true ] && echo run || echo skip
;;
redis-compat)
[ "$has_redis_compat" = true ] && echo run || echo skip
;;
backend)
[ "$has_backend" = true ] && echo run || echo skip
;;
client)
{ [ "$has_client" = true ] || [ "$has_backend" = true ]; } && echo run || echo skip
;;
ui)
{ [ "$has_client" = true ] || [ "$has_ci" = true ]; } && echo run || echo skip
;;
*)
echo run
;;
esac

View file

@ -1,116 +0,0 @@
from __future__ import annotations
import json
import os
import re
import subprocess
import sys
from pathlib import Path
from typing import Final
from xml.etree import ElementTree
SUITES: Final = {
"startup": (("test_startup.py",), 13),
"recovery": (("test_recovery.py",), 15),
"legacy": (("test_legacy.py", "test_pooling.py"), 11),
"upgrade": (("test_upgrade.py", "test_rolling_upgrade.py"), 5),
"shaped": (("test_shaped_database.py",), 1),
}
def successful_junit(path: Path, expected: int, exit_code: int) -> bool:
if exit_code != 0 or not path.is_file():
return False
try:
root: Final = ElementTree.parse(path).getroot()
except ElementTree.ParseError:
return False
cases: Final = tuple(root.iter("testcase"))
identities: Final = frozenset((case.get("classname"), case.get("name")) for case in cases)
return len(cases) == len(identities) == expected and all(
not any(case.find(tag) is not None for tag in ("failure", "error", "skipped")) for case in cases
)
def output(*command: str) -> str:
return subprocess.check_output(command, text=True, timeout=90).strip()
def record_image() -> None:
source: Final = output("git", "rev-parse", "HEAD")
image: Final = output("docker", "image", "inspect", "litellm-docker-database:ci", "--format", "{{.Id}}")
revision: Final = output(
"docker",
"image",
"inspect",
"litellm-docker-database:ci",
"--format",
'{{index .Config.Labels "org.opencontainers.image.revision"}}',
)
assert re.fullmatch(r"[0-9a-f]{40}", source), "Invalid source revision"
assert revision == source, "Candidate image revision differs from the tested source"
Path("migration-image.json").write_text(
json.dumps(
{
"source_sha": source,
"image_id": image,
"candidate_image": os.environ.get("MIGRATION_CANDIDATE_IMAGE", ""),
}
)
)
def main() -> int:
suite: Final = os.environ["MIGRATION_TEST_SUITE"]
files, expected = SUITES[suite]
metadata: Final = json.loads(Path("migration-image.json").read_text())
assert metadata["source_sha"] == output("git", "rev-parse", "HEAD"), "Image and test source revisions differ"
assert metadata["image_id"] == output(
"docker", "image", "inspect", os.environ["LITELLM_MIGRATION_TEST_IMAGE"], "--format", "{{.Id}}"
), "Loaded image differs from the build output"
assert metadata["candidate_image"] == os.environ.get("MIGRATION_CANDIDATE_IMAGE", ""), "Wrong release candidate"
destination: Final = Path(os.environ["MIGRATION_TEST_OUTPUT"])
junit: Final = destination / "junit" / "results.xml"
junit.parent.mkdir(parents=True, exist_ok=True)
result: Final = subprocess.run(
(
sys.executable,
"-m",
"pytest",
*(f"tests/e2e/migrations/{name}" for name in files),
"-vv",
"--tb=short",
"--durations=10",
f"--junitxml={junit}",
"-o",
"addopts=",
"--reruns=0",
),
check=False,
timeout=1200,
)
passed: Final = successful_junit(junit, expected, result.returncode)
(destination / "verdict.json").write_text(
json.dumps(
{
**metadata,
"suite": suite,
"baseline_image": os.environ.get("LITELLM_MIGRATION_BASELINE_IMAGE", ""),
"expected_cases": expected,
"passed": passed,
"pytest_exit_code": result.returncode,
"test_revision": metadata["source_sha"],
"workflow_id": os.environ.get("CIRCLE_WORKFLOW_ID", ""),
"job_number": os.environ.get("CIRCLE_BUILD_NUM", ""),
},
indent=2,
)
)
return 0 if passed else 1
if __name__ == "__main__":
if sys.argv[1:] == ["record-image"]:
record_image()
else:
raise SystemExit(main())

View file

@ -1,57 +0,0 @@
{
"name": "Python 3.11",
// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
"image": "mcr.microsoft.com/devcontainers/python:3.11-bookworm",
// https://github.com/devcontainers/images/tree/main/src/python
// https://mcr.microsoft.com/en-us/product/devcontainers/python/tags
// "build": {
// "dockerfile": "Dockerfile",
// "context": ".."
// },
// Features to add to the dev container. More info: https://containers.dev/features.
"features": {
"ghcr.io/devcontainers/features/node:1": {
"version": "lts"
},
"ghcr.io/devcontainers/features/docker-in-docker:2": {}
},
// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
"settings": {},
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"GitHub.copilot",
"GitHub.copilot-chat",
"ms-python.autopep8"
]
}
},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [4000],
"containerEnv": {
"LITELLM_LOG": "DEBUG"
},
// Use 'portsAttributes' to set default properties for specific forwarded ports.
// More info: https://containers.dev/implementors/json_reference/#port-attributes
"portsAttributes": {
"4000": {
"label": "LiteLLM Server",
"onAutoForward": "notify"
}
},
// More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "litellm",
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "bash ./.devcontainer/post-create.sh"
}

View file

@ -1,17 +0,0 @@
#!/usr/bin/env bash
set -e
echo "[post-create] Installing uv"
curl -LsSf https://astral.sh/uv/0.10.9/install.sh | env UV_NO_MODIFY_PATH=1 sh
export PATH="$HOME/.local/bin:$PATH"
echo "[post-create] Installing Python dependencies (uv)"
uv sync --frozen --group proxy-dev --extra proxy
echo "[post-create] Generating Prisma client"
uv run --no-sync prisma generate
echo "[post-create] Installing npm dependencies"
cd ui/litellm-dashboard && npm ci
echo "[post-create] Done"

View file

@ -1,58 +0,0 @@
docs
cookbook
.circleci
.github
tests
.git
.devcontainer
*.tgz
log.txt
docker/Dockerfile.*
# Claude Flow generated files (must be excluded from Docker build)
.claude/
.claude-flow/
.swarm/
.hive-mind/
memory/
coordination/
claude-flow
.mcp.json
hive-mind-prompt-*.txt
# Python virtual environments and version managers
.venv/
venv/
**/.venv/
**/venv/
.python-version
.pyenv/
__pycache__/
**/__pycache__/
*.pyc
.mypy_cache/
.pytest_cache/
.ruff_cache/
**/pyvenv.cfg
# Common project exclusions
.vscode
*.pyo
*.pyd
.Python
env/
.pytest_cache
.coverage
htmlcov/
dist/
build/
*.egg-info/
.DS_Store
**/node_modules
ui/litellm-dashboard/.next
ui/litellm-dashboard/out
litellm-rust/target/
litellm/rust_bridge/_native*.so
*.log
.env
.env.local

View file

@ -1,32 +0,0 @@
# OpenAI
OPENAI_API_KEY = ""
OPENAI_BASE_URL = ""
# Cohere
COHERE_API_KEY = ""
# OpenRouter
OR_SITE_URL = ""
OR_APP_NAME = "LiteLLM Example app"
OR_API_KEY = ""
# Azure API base URL
AZURE_API_BASE = ""
# Azure API version
AZURE_API_VERSION = ""
# Azure API key
AZURE_API_KEY = ""
# Replicate
REPLICATE_API_KEY = ""
REPLICATE_API_TOKEN = ""
# Anthropic
ANTHROPIC_API_KEY = ""
# Infisical
INFISICAL_TOKEN = ""
# Novita AI
NOVITA_API_KEY = ""
# INFINITY
INFINITY_API_KEY = ""
# Development Configs
# Generate one with: echo "LITELLM_MASTER_KEY=sk-$(openssl rand -hex 32)"
LITELLM_MASTER_KEY = ""
DATABASE_URL = "postgresql://llmproxy:dbpassword9090@db:5432/litellm"
STORE_MODEL_IN_DB = "True"

View file

@ -1,40 +0,0 @@
# Add the commit hash of any commit you want to ignore in `git blame` here.
# One commit hash per line.
#
# The GitHub Blame UI will use this file automatically!
#
# Run this command to always ignore formatting commits in `git blame`
# git config blame.ignoreRevsFile .git-blame-ignore-revs
# Update pydantic code to fix warnings (GH-3600)
876840e9957bc7e9f7d6a2b58c4d7c53dad16481
# style(ui): run prettier --write across the dashboard (#29622)
7edf3a9cb55548b143df1692f4ed7c4681d7fcf7
# style: reformat litellm/ with ruff format (#31317)
17bfd415aeb5a57fb646b5cc67da1c730aa7c50b
# style: unify ruff format width on 120 (#31518)
48b5a5a0cc5a694a11219416ee0b6eb6e620e74e
# refactor(imports): move collections.abc names out of typing (#35495)
397e8e4918777e4e60a7f5e88699e0a9a7dabb3d
# refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets (#35495)
b604e2b20c6db2099085a2f0e59b7e99e87eed6f
# refactor(logging): drop redundant !s conversion flags from f-strings (#35546)
7b2d3440cba3160277470f7a0180098ae9b87864
# perf: build log messages lazily so filtered-out log records cost nothing (#35703)
c9887a1f94bc1e7e4bdfe64d640f0509a0bc19dd
# feat(lint): enforce Final on locals and freeze function parameters (#35807)
2708620d6a599cc73c1950a942d26ac26a7ed3d4
# chore(lint): remove litellm/types from the ruff lint exclusion (#35926)
4e32a8bf6a1e1af1e04b67c759841ccef44b2235
# chore(lint): strip inert type: ignore comments and zero LIT009/LIT010/LIT011 headroom (#35928)
338e411103ad5d7003e97f34f04fa36bca542dbe

2
.gitattributes vendored
View file

@ -1,2 +0,0 @@
*.ipynb linguist-vendored
ui/litellm-dashboard/src/lib/http/schema.d.ts linguist-generated

View file

@ -1,106 +0,0 @@
version: 2
secret:
# Exclude files and paths by globbing
ignored_paths:
- "**/*.whl"
- "**/*.pyc"
- "**/__pycache__/**"
- "**/node_modules/**"
- "**/dist/**"
- "**/build/**"
- "**/.git/**"
- "**/venv/**"
- "**/.venv/**"
# Large data/metadata files that don't need scanning
- "**/model_prices_and_context_window*.json"
- "**/*_metadata/*.txt"
- "**/tokenizers/*.json"
- "**/tokenizers/*"
- "miniconda.sh"
# Build outputs and static assets
- "litellm/proxy/_experimental/out/**"
- "ui/litellm-dashboard/public/**"
- "**/swagger/*.js"
- "**/*.woff"
- "**/*.woff2"
- "**/*.avif"
- "**/*.webp"
# Test data files
- "**/tests/**/data_map.txt"
- "tests/**/*.txt"
# Documentation and other non-code files
- "docs/**"
- "**/*.md"
- "**/*.lock"
- "uv.lock"
- "package-lock.json"
# Ignore security incidents with the SHA256 of the occurrence (false positives)
ignored_matches:
# === Current detected false positives (SHA-based) ===
# gcs_pub_sub_body - folder name, not a password
- name: GCS pub/sub test folder name
match: 75f377c456eede69e5f6e47399ccee6016a2a93cc5dd11db09cc5b1359ae569a
# os.environ/APORIA_API_KEY_1 - environment variable reference
- name: Environment variable reference APORIA_API_KEY_1
match: e2ddeb8b88eca97a402559a2be2117764e11c074d86159ef9ad2375dea188094
# os.environ/APORIA_API_KEY_2 - environment variable reference
- name: Environment variable reference APORIA_API_KEY_2
match: 09aa39a29e050b86603aa55138af1ff08fb86a4582aa965c1bd0672e1575e052
# oidc/circleci_v2/ - test authentication path, not a secret
- name: OIDC CircleCI test path
match: feb3475e1f89a65b7b7815ac4ec597e18a9ec1847742ad445c36ca617b536e15
# text-davinci-003 - OpenAI model identifier, not a secret
- name: OpenAI model identifier text-davinci-003
match: c489000cf6c7600cee0eefb80ad0965f82921cfb47ece880930eb7e7635cf1f1
# Base64 Basic Auth in test_pass_through_endpoints.py - test fixture, not a real secret
- name: Test Base64 Basic Auth header in pass_through_endpoints test
match: 61bac0491f395040617df7ef6d06029eac4d92a4457ac784978db80d97be1ae0
# PostgreSQL password "postgres" in CI configs - standard test database password
- name: Test PostgreSQL password in CI configurations
match: 6e0d657eb1f0fbc40cf0b8f3c3873ef627cc9cb7c4108d1c07d979c04bc8a4bb
# Bearer token in locustfile.py - test/example API key for load testing
- name: Test Bearer token in locustfile load test
match: 2a0abc2b0c3c1760a51ffcdf8d6b1d384cef69af740504b1cfa82dd70cdc7ff9
# Inkeep API key in docusaurus.config.js - public documentation site key
- name: Inkeep API key in documentation config
match: c366657791bfb5fc69045ec11d49452f09a0aebbc8648f94e2469b4025e29a75
# Langfuse credentials in test_completion.py - test credentials for integration test
- name: Langfuse test credentials in test_completion
match: c39310f68cc3d3e22f7b298bb6353c4f45759adcc37080d8b7f4e535d3cfd7f4
# === Preventive patterns for test keys (pattern-based) ===
# Test API keys (124 instances across 45 files)
- name: Test API keys with sk-test prefix
match: sk-test-
# Mock API keys
- name: Mock API keys with sk-mock prefix
match: sk-mock-
# Fake API keys
- name: Fake API keys with sk-fake prefix
match: sk-fake-
# Generic test API key patterns
- name: Test API key patterns
match: test-api-key
- name: Short fake sk keys (1–9 digits only)
match: \bsk-\d{1,9}\b

5
.github/AGENTS.md vendored Normal file
View file

@ -0,0 +1,5 @@
# CI
Integration groups and replica runs remain in `workflows/test-integration.yml`. Do not duplicate them in another workflow.
CI helpers used by existing workflows live in `scripts/ci/`. Jobs that need merge bases check out full history. Removing a CI job does not count its tests as covered. List tests that no remaining job invokes explicitly in `ci-coverage-allowlist.yml`.

5
.github/CODEOWNERS vendored
View file

@ -1,5 +1,6 @@
/model_prices_and_context_window.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
/litellm/model_prices_and_context_window_backup.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
/helm/ @yassin-berriai
/terraform/ @yassin-berriai
/litellm/scripts/ci_cd/cost_map_pins.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
/litellm/deploy/helm/ @yassin-berriai
/litellm/deploy/terraform/ @yassin-berriai
/.github/workflows/zizmor.yml @yuneng-berri

388
.github/CONTRIBUTING.md vendored Normal file
View file

@ -0,0 +1,388 @@
# Contributing to LiteLLM
Thank you for your interest in contributing to LiteLLM! We welcome contributions of all kinds - from bug fixes and documentation improvements to new features and integrations.
## **Checklist before submitting a PR**
Here are the core requirements for any PR submitted to LiteLLM:
- [ ] **Sign the Contributor License Agreement (CLA)** - [see details](#contributor-license-agreement-cla)
- [ ] **Keep scope isolated** - Your changes should address 1 specific problem at a time
- [ ] **Prove it works end to end** - If your change affects what the proxy or SDK does, paste the commands you ran against a live proxy and their output in the PR - [see details](#proving-your-change-works-end-to-end)
#### Proxy (Backend) PRs
- [ ] **Add testing** - Adding at least 1 test is a hard requirement - [see details](#adding-testing)
- [ ] **Ensure your PR passes all checks**:
- [ ] [Linting / Formatting](#running-linting-and-formatting-checks) - `make lint`
- [ ] [The tests covering your change](#running-unit-tests) pass, e.g. `uv run pytest tests/unit/<your_test_file>.py -v`. CI runs the full unit test matrix, so you don't need to run the whole suite locally
#### UI PRs
- [ ] **Ensure the UI builds successfully** - `npm run build`
- [ ] **Ensure all UI unit tests pass** - `npm run test`
- [ ] **Add tests for new components or logic** - If you are adding a new component or new logic, add corresponding tests
## **Contributor License Agreement (CLA)**
Before contributing code to LiteLLM, you must sign our [Contributor License Agreement (CLA)](https://cla-assistant.io/BerriAI/litellm). This is a legal requirement for all contributions to be merged into the main repository.
**Important:** We strongly recommend reviewing and signing the CLA before starting work on your contribution to avoid any delays in the PR process.
## Quick Start
### 1. Setup Your Local Development Environment
```bash
# Fork the repository on GitHub (click the Fork button at https://github.com/BerriAI/litellm)
# Then clone your fork locally
git clone https://github.com/YOUR_USERNAME/litellm.git
cd litellm
# Create a new branch for your feature (see "Commit and Branch Conventions" below)
git checkout -b litellm_your_feature
# Install development dependencies
make install-dev
# Install git hooks that enforce commit conventions (one-time, opt-in)
make install-hooks
# Verify your setup works
make help
```
That's it! Your local development environment is ready.
## Commit and Branch Conventions
Commits follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/). Internal contributors name branches with the `litellm_` prefix and no `/`, and fork branches can use any name. Run `make install-hooks` once per clone to enable the local hook that enforces the commit convention. See the [contributor docs](https://docs.litellm.ai/docs/extras/contributing_code#commit-and-branch-conventions) for details
### 2. Development Workflow
Here's the recommended workflow for making changes:
```bash
# Make your changes to the code
# ...
# Format your code (auto-fixes formatting issues)
make format
# Run all linting checks (matches CI exactly)
make lint
# Run the tests covering your change (CI runs the full suite)
uv run pytest tests/unit/<your_test_file>.py -v
# Commit your changes (must follow Conventional Commits — see above)
git add .
git commit -m "feat(scope): your descriptive commit message"
# Push and create a PR
git push origin litellm_your_feature
```
## Adding Testing
**Adding at least 1 test is a hard requirement for all PRs.**
### Where to Add Tests
Add your tests to the [`tests/unit/` directory](https://github.com/BerriAI/litellm/tree/main/tests/unit).
- This directory mirrors the structure of the `litellm/` directory
- **Only add mocked tests** - no real LLM API calls in this directory
- For integration tests with real APIs, use the appropriate test directories
Tests that call `litellm.register_model` or change `litellm.model_cost` should use the `local_model_cost_map` fixture from `tests/unit/conftest.py`. It provides a fresh catalog and restores the previous map, so pricing and capability overrides cannot leak into later tests
### File Naming Convention
The `tests/unit/` directory follows the same structure as `litellm/`:
- `litellm/proxy/caching_routes.py` → `tests/unit/proxy/test_caching_routes.py`
- `litellm/utils.py` → `tests/unit/test_utils.py`
### Example Test
```python
import pytest
from litellm import completion
def test_your_feature():
"""Test your feature with a descriptive docstring."""
# Arrange
messages = [{"role": "user", "content": "Hello"}]
# Act
# Use mocked responses, not real API calls
# Assert
assert expected_result == actual_result
```
## Running Tests and Checks
### Running Unit Tests
Run the tests covering your change:
```bash
uv run pytest tests/unit/test_your_file.py -v
```
`tests/unit` holds thousands of tests, so running all of it locally takes a long time. CI runs it as a parallel matrix (`make test-unit-llms`, `make test-unit-proxy-core`, and the other `test-unit-*` targets) on beefier boxes, so if, for whatever reason, you must run the whole suite, it's better to rely on CI to do that.
If you're running broader test suites, proxy tests, or anything that touches PostgreSQL-backed fixtures/plugins, install the full local test environment first:
```bash
make install-test-deps
```
This syncs the locked test environment used across the repo, including `psycopg` v3 plus `psycopg-binary`, `psycopg2-binary` (used by some proxy E2E tests), and a generated Prisma client for DB-backed proxy tests, so pytest startup matches CI without manual package installs.
### Running Linting and Formatting Checks
Run all linting checks (matches CI exactly):
```bash
make lint
```
Individual linting commands:
```bash
make format-check # Check ruff format formatting
make lint-ruff # Run Ruff linting
make lint-basedpyright # Run basedpyright type checking
make check-circular-imports # Check for circular imports
make check-import-safety # Check import safety
```
Apply formatting (auto-fixes issues):
```bash
make format
```
> **Formatting is enforced in CI.** All PRs must pass the `ruff format --check` step.
>
> - **AI coding agents** (Claude Code, Copilot, Cursor, etc.): follow `AGENTS.md` and run `make format` before committing.
> - **VS Code users**: Install the [Ruff extension](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) and enable format-on-save:
> ```json
> {
> "[python]": {
> "editor.defaultFormatter": "charliermarsh.ruff",
> "editor.formatOnSave": true
> }
> }
> ```
### CI Compatibility
To ensure your changes will pass CI, run the exact same checks locally:
```bash
# This runs the same checks as the GitHub workflows
make lint
make test-unit
```
For exact CI compatibility (pins OpenAI version like CI):
```bash
make install-dev-ci # Installs exact CI dependencies
```
## Available Make Commands
Run `make help` to see all available commands:
```bash
make help # Show all available commands
make install-dev # Install development dependencies
make install-proxy-dev # Install proxy development dependencies
make install-test-deps # Install the full local test environment
make format # Apply ruff format code formatting
make format-check # Check ruff format formatting (matches CI)
make lint # Run all linting checks
make test-unit # Run unit tests
make test-integration # Run integration tests
make test-unit-helm # Run Helm unit tests
```
## Code Quality Standards
LiteLLM follows the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html).
Our automated quality checks include:
- **Ruff** for formatting, linting, and code quality
- **basedpyright** for static type checking
- **Circular import detection**
- **Import safety validation**
All checks must pass before your PR can be merged.
## Common Issues and Solutions
### 1. Linting Failures
If `make lint` fails:
1. **Formatting issues**: Run `make format` to auto-fix
2. **Ruff issues**: Check the output and fix manually
3. **basedpyright issues**: Add proper type hints
4. **Circular imports**: Refactor import dependencies
5. **Import safety**: Fix any unprotected imports
### 2. Test Failures
If `make test-unit` fails:
1. Check if you broke existing functionality
2. Add tests for your new code
3. Ensure tests use mocks, not real API calls
4. Check test file naming conventions
### 3. Common Development Tips
- **Use type hints**: basedpyright requires proper type annotations
- **Write descriptive commit messages**: Help reviewers understand your changes
- **Keep PRs focused**: One feature/fix per PR
- **Test edge cases**: Don't just test the happy path
- **Update documentation**: If you change APIs, update docs
## Proving Your Change Works End to End
Tests are required, but they do not prove that your change works for a user. A PR that changes what the proxy or SDK does shows a real run in its "Screenshots / Proof of Fix" section, meaning the exact commands you sent to a live proxy and the output that came back, against real provider APIs and not mocks. `pytest` output does not count as proof, because a passing test only shows that the code does what its mocks expect. A PR that changes only docs or CI shows the rendered page or the CI run instead
A good proof looks like this:
1. Install the proxy dependencies with `make install-proxy-dev`, then start the proxy from your branch with a config that has the model you changed, `uv run litellm --config your_config.yaml --detailed_debug`
2. Send the request a user would send with `curl`, and paste both the command and the response. When cost or routing is involved, include the response headers, since `x-litellm-response-cost` is what the user sees
3. For a bug fix, do this twice, once at the merge base with BerriAI/litellm's main (`git merge-base origin/main HEAD`, with your remote for BerriAI/litellm in place of `origin` in a fork clone) to show the bug and once at your branch's latest commit to show the fix. Name both commit hashes
4. If your change is reachable from more than one endpoint (`/v1/chat/completions`, `/v1/responses`, `/v1/messages`), show each one
5. If the request can stream, show a `"stream": true` run too, because streaming builds usage and cost on a different path. A stream sends its headers before its usage exists, so show the `usage.cost` on the last chunk, which `/v1/chat/completions` sends when the request has `"stream_options": {"include_usage": true}` and other endpoints send with `include_cost_in_streaming_usage: true` under `litellm_settings`, or, when the proxy has a database, the request's row on the Admin UI's Logs page or `GET /spend/logs`
6. For UI changes, add before and after screenshots and name the page
7. If users reach your change through a coding tool such as Claude Code or Codex, drive that tool the way a user does and add before and after screenshots of it, because a `curl` or a headless run alone does not show what the user sees
```bash
curl -sD - http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "your-model", "messages": [{"role": "user", "content": "hi"}]}'
```
When the provider reports its own usage or spend numbers, compare the proxy's numbers against them in the proof
## Contributing an LLM Provider Integration
Provider work (a new provider, a new parameter, a new usage or cost field) is the most common kind of contribution, and it is where PRs get reworked most often. Follow these rules
**Preserve zero prices.** In [`model_prices_and_context_window.json`](../model_prices_and_context_window.json), an explicit `0.0` means that the priced operation is free. A missing field or `null` means no rate is specified, so the relevant calculator may use a fallback. For example, when no batch input rate is configured, batch cache writes use `cache_creation_input_token_cost` if set, otherwise the regular input rate, then apply the batch discount. A zero ordinary input rate does not make separately priced cache reads or writes free. Test `rate is None` before substituting a fallback, since `row.get("cache_creation_input_token_cost") or fallback` would discard a valid zero. Speech keeps character pricing ahead of token pricing even when the character rate is zero. If neither rate is configured, speech cost calculation raises an error. Its model lookup preserves unset rates instead of treating compatibility defaults as explicit zero prices. Perplexity keeps a configured legacy search rate ahead of the current search rate, including zero. When registering custom prices, `None` leaves an existing rate unchanged and `0.0` overrides it
**Find the existing machinery before you add plumbing.** LiteLLM already has generic paths that move data between the endpoints, the provider transformations, and cost tracking. A second, hand-rolled path for the same data is a common reason a provider PR gets rewritten before merge. If you cannot find a mechanism for what you need, say so in the PR or ask in [#pr-review on Slack](https://join.slack.com/t/litellmossslack/shared_invite/zt-3o7nkuyfr-p_kbNJj8taRfXGgQI1~YyA) before you build one
**Bill a new provider usage field from the usage object.** Leave the field on the usage object as an extra and read it in that provider's `cost_calculator.py`. A provider without one gets a new `cost_calculator.py` and an arm for it in `cost_per_token` (`litellm/cost_calculator.py`), which picks the calculator by provider. If the provider's transform builds `Usage` from named fields, as the Anthropic, Gemini, and Bedrock Converse ones do, pass the new field there too. Streamed requests rebuild usage in `ChunkProcessor.calculate_usage` (`litellm/litellm_core_utils/streaming_chunk_builder_utils.py`), which does not copy extras yet the way the Responses-to-chat bridge does (minus a short exclusion list), so until it does, bill the field on streams by naming it there, as Anthropic's `inference_geo` is. A provider that reports its own charge for the request, as OpenRouter does in `usage.cost`, is billed at that charge from `_hidden_params` before any calculator runs, unless the deployment sets its own prices. When the field counts a built-in tool such as web search, add the provider to `get_cost_for_web_search_request` in `litellm/llms/__init__.py`, and add a gate to `litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py` only when no existing gate fires. Then show the field's cost in your proof on every endpoint it reaches, streaming included
**Do not change what a caller sees to make internals easier.** Each endpoint keeps the schema of the API it emulates. A `/v1/responses` caller gets `input_tokens` and a `/v1/chat/completions` caller gets `prompt_tokens`, whatever the provider sent and whatever cost tracking needs. A provider usage field that shows up as an extra key is fine, since it only adds a key, but renaming, removing, or reshaping a key the endpoint defines is not. Reshaping a public response to feed an internal consumer breaks every user of that endpoint
## Building and Running Locally
### LiteLLM Proxy Server
To run the proxy server locally:
```bash
# Install proxy dependencies
make install-proxy-dev
# Start the proxy server
uv run litellm --config your_config.yaml
```
### Docker Development
If you want to build the Docker image yourself:
```bash
# Build the image (runs as a non-root user by default)
docker build -f litellm/deploy/docker/Dockerfile -t litellm_dev .
# Generate a master key. Requests send it as the bearer token
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
# Run with your config
docker run \
-v $(pwd)/proxy_config.yaml:/app/config.yaml \
-e LITELLM_MASTER_KEY \
-p 4000:4000 \
litellm_dev \
--config /app/config.yaml --detailed_debug
```
## UI Development
### 1. Setup Your Local UI Development Environment
```bash
# Clone the repo (if you haven't already)
git clone https://github.com/YOUR_USERNAME/litellm.git
cd litellm
# Navigate to the UI dashboard directory
cd litellm/ui/litellm-dashboard
# Install dependencies
npm install
# Start the development server
npm run dev
```
### 2. Adding UI Tests
If you are adding a **new component** or **new logic**, you must add corresponding tests.
### 3. Running UI Unit Tests
```bash
npm run test
```
### 4. Building the UI
Ensure the UI builds successfully before submitting your PR:
```bash
npm run build
```
Local lint and budget checks follow origin's current default branch. They refresh it from the remote instead of trusting cached `origin/HEAD`. For an intentional comparison against another branch or commit, use `make check BASE_REF=<ref>` or the standalone gate's `--base <ref>` option. An explicit ref can also be used offline once it has been fetched locally. Without an override, unavailable remote metadata stops the check
## Submitting Your PR
1. **Push your branch**: `git push origin your-feature-branch`
2. **Create a PR**: Go to GitHub and open a pull request against the repository's current default branch. Run `python3 litellm/scripts/default_branch.py --branch` to check its name
3. **Fill out the PR template**: Provide clear description of changes
4. **Wait for review**: Maintainers will review and provide feedback
5. **Address feedback**: Make requested changes and push updates
6. **Merge**: Once approved, your PR will be merged!
## Getting Help
If you need help:
- 💬 [Join our Discord](https://discord.gg/wuPM9dRgDw)
- 💬 [Join our Slack](https://www.litellm.ai/support)
- 📧 Email us: ishaan@berri.ai / krrish@berri.ai
- 🐛 [Create an issue](https://github.com/BerriAI/litellm/issues/new)
## What to Contribute
Looking for ideas? Check out:
- 🐛 [Good first issues](https://github.com/BerriAI/litellm/labels/good%20first%20issue)
- 🚀 [Feature requests](https://github.com/BerriAI/litellm/labels/enhancement)
- 📚 Documentation improvements
- 🧪 Test coverage improvements
- 🔌 New LLM provider integrations
Thank you for contributing to LiteLLM! 🚀

View file

@ -4,9 +4,14 @@
Drop every section you have nothing to put in, heading included: a bare "## Relevant issues" or
"## Affected release" with nothing under it must not appear in the final description -->
<!-- Two or three plain sentences a teammate can read in 10 seconds, written the way you'd explain it in Slack
## Handwritten 1-3 sentence description
<!-- The author writes this in their own words, and it can be left empty. AI agents never write here and leave the
placeholder line below as is, and when updating an existing PR they keep whatever the author wrote. Keep this heading
Say what the PR does for users and why, then anything that keeps it working over time (a sync job, a config someone owns)
No headings, bullets, tables, or code names here. Everything else goes inside the Details block below -->
No bullets, tables, or code names here. Everything else goes inside the Details block below -->
[insert handwritten description here]
<details>
<summary>Details</summary>
@ -84,9 +89,11 @@ If you're seeing a delay in your PR being merged, ping the LiteLLM Team on [Slac
## Screenshots / Proof of Fix
<!-- Include screenshots, screen recordings, or command (e.g., curl) + output demonstrating that your changes work as expected
The proof must be completely e2e with no mocks, using actual LLM calls costing real $$$ if applicable. `pytest` commands are not enough
Show ONLY the latest run: capture Before at the merge base and After at the PR's current tip, and when new commits change behavior, replace this whole section with the fresh run instead of stacking it on top of older ones. The run must be up to date. As soon as a new commit is made and it makes this PR description's after sha stale (it's no longer tip of PR), you must re-run the QA
Structure the section exactly as below: Before and After one heading level below this section, each naming the commit hash it was captured at, one lower-level heading per case inside each, the same case names in the same order on both sides, and numbered steps (command, observed output) under every case, never loose prose; shared setup (config, payloads) goes above Before, and with a single case, drop the case headings and number the steps directly
Simple PR exception (2026-10-28, #49030). A mechanical change fully proven by focused checks may skip before/after e2e. Follow AGENTS.md's eligibility rule, trace callers, and verify bug-fix regression sensitivity. Record E2E exception at <tip SHA>, scope, rationale, commands and results here. Tests/static checks suffice only for this exception. Keep CI and bot gates, reassess changed diffs, and honor explicit e2e requests. Omit the Before/After scaffold for an exception
Otherwise proof must be fully e2e, no mocks, with real provider calls where applicable. Pytest alone is insufficient. The following format applies to live proof
Show only the latest proof. Before uses the merge base and After the tested tip. Rerun affected QA after behavior changes. Verified neutral diffs may carry proof with the tested SHA, current tip, equivalence reason and focused checks, following AGENTS.md and /qa
Use matching Before/After case headings with numbered commands and observed output. Put shared setup above them. For one case, omit case headings
A reviewer must be able to rerun every case from the body alone, so the shared setup gives the proxy start command, the config.yaml, env var names (never values) and the client code or curl used, all exactly as run. Wrap any block longer than about 20 lines in a collapsed <details> with a one-line summary
### Before (<hash>)

89
.github/SECURITY.md vendored Normal file
View file

@ -0,0 +1,89 @@
# Data Privacy and Security
## Security Announcements
LiteLLM maintains a security announcements mailing list that is open to anyone. Subscribers receive advance notice, typically one to two days, before we release a fix for a particularly severe vulnerability or for any vulnerability exploitable by an unauthenticated attacker. This notice is provided on a best-effort basis
To subscribe, visit [https://security.litellm.ai/mailinglist/](https://security.litellm.ai/mailinglist/)
## Security Vulnerability Reporting Guidelines
We value the security community's role in protecting our systems and users.
To report a security vulnerability, please submit a report through our vulnerability disclosure form:
[LiteLLM VDP](https://security.litellm.ai/vdp/)
These reports are submitted to HackerOne and are triaged there, though no account is needed.
**Please do not submit vulnerabilities as draft GHSAs. As of 2026-11-11 new GHSA submissions will be closed.**
### Vulnerability Categories
We classify vulnerabilities into the following categories:
**P0: Supply Chain Attacks**
Attacks that compromise our CI/CD pipeline, allowing a malicious actor to point our PyPI package or Docker images (GHCR or Docker Hub) to vulnerable or tampered artifacts.
**P1: Unauthenticated Proxy Access**
Application-level attacks where an unauthenticated user is able to gain access to protected data on a LiteLLM proxy instance that should be protected (e.g api keys).
**P2: Authenticated Malicious Actions**
Application-level attacks where an authenticated user is able to perform actions beyond their intended permissions, such as privilege escalation or unauthorized data access.
### Bug Bounty Program
We offer bounties for responsibly disclosed vulnerabilities based on severity:
**Note that currently only P0/P1 reports are eligible for a bounty, though submissions for P2 bugs are still encouraged**
| Severity | Bounty Range | Example |
|----------|-------------|---------|
| **Critical** | $1,500 - $3,000 | P0 supply chain compromise |
| **High** | $500 - $1,500 | P1 unauthenticated proxy access |
| **Medium** | N/A | P2 authenticated privilege escalation |
| **Low** | N/A | Minor information disclosure, low-impact misconfigurations |
To qualify for a bounty, reports must include clear reproduction steps, a reproduction video as described above, and must not involve systems or accounts you do not own. We review all submissions promptly and will follow up within 5 business days.
### Known Non-Issues
- **Misconfigurations:** attacks that require misconfiguration of the application (e.g. not setting a `master_key` on the proxy configuration).
- **Insecure configurations:** attacks that require deliberate configuration changes to disable security functionality. (e.g. IDOR on response IDs requiring configuration of `disable_responses_id_security` to `true`)
- **Social engineering:** attacks that rely on social engineering as their primary mechanism (e.g. convincing a user's agent to install a malicious package we did not publish).
## Security Measures
### LiteLLM Github
- All commits run through Github's CodeQL checking
### Self-hosted Instances LiteLLM
- **No data or telemetry is stored on LiteLLM Servers when you self host**
- For installation and configuration, see: [Self-hosting guided](https://docs.litellm.ai/docs/proxy/deploy)
- **Telemetry** We run no telemetry when you self host LiteLLM
### LiteLLM Cloud
- We encrypt all data stored using your `LITELLM_MASTER_KEY` and in transit using TLS.
- Our database and application run on GCP, AWS infrastructure, partly managed by NeonDB.
- US data region: Northern California (AWS/GCP `us-west-1`) & Virginia (AWS `us-east-1`)
- EU data region Germany/Frankfurt (AWS/GCP `eu-central-1`)
- All users have access to SSO (Single Sign-On) through OAuth 2.0 with Google, Okta, Microsoft, KeyCloak.
- Audit Logs with retention policy
- Control Allowed IP Addresses that can access your Cloud LiteLLM Instance
For security inquiries, please contact us at security@berri.ai
#### Supported data regions for LiteLLM Cloud
LiteLLM supports the following data regions:
- US, Northern California (AWS/GCP `us-west-1`)
- Europe, Frankfurt, Germany (AWS/GCP `eu-central-1`)
All data, user accounts, and infrastructure are completely separated between these two regions

View file

@ -3,7 +3,7 @@ description: >-
Cache the Cargo registry and target directory the root package's build needs,
so only the first job on a given Cargo.lock compiles the bridge from scratch.
litellm builds through maturin, which compiles litellm-rust/crates/python-bridge
litellm builds through maturin, which compiles litellm/rust/crates/python-bridge
in the dev profile for editable installs. `uv sync` therefore pays a full build
in every job that installs the workspace. Nothing caught it, because the uv cache
holds wheels uv downloads rather than wheels it builds, and a path dependency
@ -31,8 +31,8 @@ runs:
path: |
~/.cargo/registry
~/.cargo/git
litellm-rust/target
key: ${{ runner.os }}-maturin-${{ inputs.profile }}-${{ hashFiles('litellm-rust/Cargo.lock') }}
litellm/rust/target
key: ${{ runner.os }}-maturin-${{ inputs.profile }}-${{ hashFiles('litellm/rust/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-maturin-${{ inputs.profile }}-
@ -43,7 +43,7 @@ runs:
path: |
~/.cargo/registry
~/.cargo/git
litellm-rust/target
key: ${{ runner.os }}-maturin-${{ inputs.profile }}-${{ hashFiles('litellm-rust/Cargo.lock') }}
litellm/rust/target
key: ${{ runner.os }}-maturin-${{ inputs.profile }}-${{ hashFiles('litellm/rust/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-maturin-${{ inputs.profile }}-

View file

@ -0,0 +1,39 @@
name: "Cache the Go build"
description: >-
Restore the Go module and build caches on every run and save them only from
main, so pull requests reuse main's cache instead of evicting it with their
own copies.
actions/setup-go's built-in cache saves from every ref, and a cache saved by a
pull request is readable only by that pull request. Callers must pass
`cache: false` to setup-go and run this action after it.
inputs:
dependency-path:
description: "Path to the go.sum that keys the cache"
required: true
runs:
using: composite
steps:
- name: Restore and save the Go module and build caches
if: github.ref == 'refs/heads/main'
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/go/pkg/mod
~/.cache/go-build
key: ${{ runner.os }}-go-${{ hashFiles(inputs.dependency-path) }}
restore-keys: |
${{ runner.os }}-go-
- name: Restore the Go module and build caches
if: github.ref != 'refs/heads/main'
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/go/pkg/mod
~/.cache/go-build
key: ${{ runner.os }}-go-${{ hashFiles(inputs.dependency-path) }}
restore-keys: |
${{ runner.os }}-go-

View file

@ -1,7 +1,7 @@
name: "Detect relevant changes"
description: >-
Classify the pull request's changed files with .circleci/scripts/classify_changes.sh
and expose decision=run|skip for one category. backend means anything outside ui/,
Classify the pull request's changed files with .github/scripts/ci/classify_changes.sh
and expose decision=run|skip for one category. backend means anything outside litellm/ui/,
docs/ and markdown; ui means the dashboard sources alone. decision=skip lets callers
short-circuit expensive steps while the job still completes successfully and satisfies
its required status check, which a paths: filter cannot do because a workflow that

View file

@ -0,0 +1,37 @@
name: "Restore the Rust bridge"
description: >-
Restore the prebuilt Rust bridge that the unit workflow's rust-bridge job saves
on main, keyed on every input of the Rust build.
The bridge is abi3, so one build serves every supported Python version. A job
whose Rust inputs match main installs the restored wheel and native library
instead of compiling them inside `uv sync`. On a miss the caller builds from
source as before.
The restored `rust-bridge-dist` directory holds the editable litellm wheel, the
litellm-proxy gateway wheel, and `litellm/rust_bridge/_native.abi3.so`.
outputs:
key:
description: "Cache key for the bridge built from this checkout"
value: ${{ steps.key.outputs.key }}
cache-hit:
description: "'true' when the bridge for this exact key was restored"
value: ${{ steps.restore.outputs.cache-hit }}
runs:
using: composite
steps:
- name: Define the Rust bridge cache key
id: key
shell: bash
env:
RUST_BRIDGE_CACHE_KEY: ${{ runner.os }}-rust-bridge-editable-${{ hashFiles('litellm/rust/Cargo.lock', 'litellm/rust/Cargo.toml', 'litellm/rust/crates/**', 'model_prices_and_context_window.json', 'pyproject.toml', 'litellm/packaging/litellm-proxy/pyproject.toml') }}
run: echo "key=$RUST_BRIDGE_CACHE_KEY" >> "$GITHUB_OUTPUT"
- name: Restore the Rust bridge
id: restore
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: rust-bridge-dist
key: ${{ steps.key.outputs.key }}

View file

@ -11,6 +11,11 @@ inputs:
runs:
using: composite
steps:
- name: Configure the Rust build environment
shell: bash
working-directory: ${{ github.workspace }}
run: bash .github/scripts/setup_rust_environment.sh
- name: Set up uv (attempt 1)
id: attempt-1
continue-on-error: true

View file

@ -4,6 +4,16 @@ description: >-
by a job nor listed here, so every entry below is a decision on the record.
test_paths:
- reason: ran only in CircleCI, removed
paths:
- tests/e2e/migrations/test_legacy.py
- tests/e2e/migrations/test_pooling.py
- tests/e2e/migrations/test_recovery.py
- tests/e2e/migrations/test_rolling_upgrade.py
- tests/e2e/migrations/test_shaped_database.py
- tests/e2e/migrations/test_startup.py
- tests/e2e/migrations/test_upgrade.py
- tests/integration/database/test_roi_observed.py
- reason: >-
litellm.agent() end-to-end suite. It drives the real claude, codex, opencode and pi CLIs and
deepagents against a live LiteLLM AI Gateway, so it needs those binaries on PATH plus
@ -19,7 +29,7 @@ test_paths:
- tests/rust-python-harness
- reason: >-
Owned by the open pull request that moves the Rust/Python parity checks (#45135). It ran in
the CircleCI build_and_test job that globbed tests/test_*.py, which this migration deletes,
the retired build_and_test job that globbed tests/test_*.py,
and that pull request gives it a new home
paths:
- tests/test_rust_python_harness.py
@ -57,19 +67,10 @@ test_paths:
- tests/integration/test_oci_proxy_integration.py
dockerfiles:
- reason: >-
The dashboard container is a static Next.js export served by nginx, and the dashboard build
and lint workflows already exercise that output, so building the image adds no signal about it
paths:
- ui/Dockerfile
- reason: >-
An example image under cookbook/ that is documentation rather than a shipped artifact
paths:
- cookbook/litellm-ollama-docker-image/Dockerfile
- reason: >-
The Rust gateway image compiles the whole workspace in release mode, which is too slow for
a per-pull-request job while the gateway binary is still being assembled; the Rust lint,
clippy, and compile jobs already cover the code it packages. Revisit when the gateway is
published
paths:
- litellm-rust/crates/gateway/Dockerfile
- litellm/rust/crates/gateway/Dockerfile

72
.github/codecov.yml vendored Normal file
View file

@ -0,0 +1,72 @@
codecov:
require_ci_to_pass: false # post coverage status even if CI has unrelated failures
notify:
wait_for_ci: false # post as soon as expected uploads arrive, don't wait on CI
ignore:
- "litellm/rust/**"
# Flagged uploads avoid overlapping sessions on reruns.
# Only the flags below reuse parent coverage, avoiding stale line maps elsewhere.
flag_management:
default_rules:
carryforward: false
individual_flags:
- name: unit
carryforward: true
- name: core-packaging
carryforward: true
- name: mcp-integration
carryforward: true
component_management:
individual_components:
- component_id: "Router"
paths:
- "router"
- component_id: "LLMs"
paths:
- "*/llms/*"
- component_id: "Caching"
paths:
- "*/caching/*"
- ".*redis.*"
- component_id: "litellm_logging"
paths:
- "*/integrations/*"
- ".*litellm_logging.*"
- component_id: "Proxy_Authentication"
paths:
- "*/proxy/auth/**"
- component_id: "Enterprise"
paths:
- "litellm/enterprise/**"
- component_id: "Batches"
paths:
- "*/proxy/batches_endpoints/**"
- "litellm/batches/**"
- "*/llms/*/batches/**"
- component_id: "Videos"
paths:
- "litellm/videos/**"
- "*/proxy/video_endpoints/**"
- "*/llms/*/videos/**"
- component_id: "Realtime"
paths:
- "litellm/realtime_api/**"
- "*/proxy/realtime_endpoints/**"
- "*/llms/*/realtime/**"
- "litellm/litellm_core_utils/realtime_streaming.py"
comment:
layout: "header, diff, flags, components" # show component info in the PR comment
coverage:
status:
project:
default:
target: auto
threshold: 0% # do not allow project coverage to drop
patch:
default:
target: auto
threshold: 0% # patch coverage should be 100%

105
.github/gitguardian.yaml vendored Normal file
View file

@ -0,0 +1,105 @@
version: 2
secret:
# Exclude files and paths by globbing
ignored_paths:
- "**/*.whl"
- "**/*.pyc"
- "**/__pycache__/**"
- "**/node_modules/**"
- "**/dist/**"
- "**/build/**"
- "**/.git/**"
- "**/venv/**"
- "**/.venv/**"
# Large data/metadata files that don't need scanning
- "**/model_prices_and_context_window*.json"
- "**/*_metadata/*.txt"
- "**/tokenizers/*.json"
- "**/tokenizers/*"
- "miniconda.sh"
# Build outputs and static assets
- "litellm/proxy/_experimental/out/**"
- "litellm/ui/litellm-dashboard/public/**"
- "**/swagger/*.js"
- "**/*.woff"
- "**/*.woff2"
- "**/*.avif"
- "**/*.webp"
# Test data files
- "**/tests/**/data_map.txt"
- "tests/**/*.txt"
# Documentation and other non-code files
- "**/*.md"
- "**/*.lock"
- "uv.lock"
- "package-lock.json"
# Ignore security incidents with the SHA256 of the occurrence (false positives)
ignored_matches:
# === Current detected false positives (SHA-based) ===
# gcs_pub_sub_body - folder name, not a password
- name: GCS pub/sub test folder name
match: 75f377c456eede69e5f6e47399ccee6016a2a93cc5dd11db09cc5b1359ae569a
# os.environ/APORIA_API_KEY_1 - environment variable reference
- name: Environment variable reference APORIA_API_KEY_1
match: e2ddeb8b88eca97a402559a2be2117764e11c074d86159ef9ad2375dea188094
# os.environ/APORIA_API_KEY_2 - environment variable reference
- name: Environment variable reference APORIA_API_KEY_2
match: 09aa39a29e050b86603aa55138af1ff08fb86a4582aa965c1bd0672e1575e052
# oidc/circleci_v2/ - test authentication path, not a secret
- name: OIDC CircleCI test path
match: feb3475e1f89a65b7b7815ac4ec597e18a9ec1847742ad445c36ca617b536e15
# text-davinci-003 - OpenAI model identifier, not a secret
- name: OpenAI model identifier text-davinci-003
match: c489000cf6c7600cee0eefb80ad0965f82921cfb47ece880930eb7e7635cf1f1
# Base64 Basic Auth in test_pass_through_endpoints.py - test fixture, not a real secret
- name: Test Base64 Basic Auth header in pass_through_endpoints test
match: 61bac0491f395040617df7ef6d06029eac4d92a4457ac784978db80d97be1ae0
# PostgreSQL password "postgres" in CI configs - standard test database password
- name: Test PostgreSQL password in CI configurations
match: 6e0d657eb1f0fbc40cf0b8f3c3873ef627cc9cb7c4108d1c07d979c04bc8a4bb
# Bearer token in locustfile.py - test/example API key for load testing
- name: Test Bearer token in locustfile load test
match: 2a0abc2b0c3c1760a51ffcdf8d6b1d384cef69af740504b1cfa82dd70cdc7ff9
# Inkeep API key in docusaurus.config.js - public documentation site key
- name: Inkeep API key in documentation config
match: c366657791bfb5fc69045ec11d49452f09a0aebbc8648f94e2469b4025e29a75
# Langfuse credentials in test_completion.py - test credentials for integration test
- name: Langfuse test credentials in test_completion
match: c39310f68cc3d3e22f7b298bb6353c4f45759adcc37080d8b7f4e535d3cfd7f4
# === Preventive patterns for test keys (pattern-based) ===
# Test API keys (124 instances across 45 files)
- name: Test API keys with sk-test prefix
match: sk-test-
# Mock API keys
- name: Mock API keys with sk-mock prefix
match: sk-mock-
# Fake API keys
- name: Fake API keys with sk-fake prefix
match: sk-fake-
# Generic test API key patterns
- name: Test API key patterns
match: test-api-key
- name: Short fake sk keys (1–9 digits only)
match: \bsk-\d{1,9}\b

22
.github/grype.yaml vendored Normal file
View file

@ -0,0 +1,22 @@
# Wolfi's security database names zlib 1.3.3-r0 as the fix for CVE-2026-85091,
# but the newest zlib published to the Wolfi apk repo is 1.3.2-r7, so every
# wolfi-base digest reports it and no `apk upgrade` can clear it.
# Drop this once Wolfi ships zlib >= 1.3.3-r0; expected by 2026-10-15.
ignore:
- vulnerability: CVE-2026-85091
package:
name: zlib
type: apk
- vulnerability: GHSA-g5fp-32jq-cfw2
package:
name: zlib
type: apk
- vulnerability: GHSA-v4x8-3wr5-3wq8
reason: >-
filelock ReadWriteLock advisory, and no package in the image takes ReadWriteLock;
the fix (4.2.1) is blocked by litellm's own filelock<4.0 cap until #48635 or #48802 lifts it,
and the image then moves off 3.32.7, which retires this version-scoped rule
package:
name: filelock
version: 3.32.7
type: python

View file

@ -1,4 +1,4 @@
extend = "ruff.toml"
extend = "../../pyproject.toml"
[lint]
preview = true
@ -6,7 +6,7 @@ select = ["ANN", "ASYNC230", "B004", "B006", "B008", "B009", "B010", "B018", "B0
extend-select = []
# Overrides the inherited list: rules this gate enforces itself must NOT be external here,
# so this config's RUF100 flags their stale `# noqa` directives. What remains external is
# only what other tooling enforces: every base ruff.toml rule this select list doesn't
# only what other tooling enforces: every base pyproject.toml rule this select list doesn't
# re-enable (all of the default E/F families plus T20/PGH004/RUF008/RUF009, minus the
# strict-selected F401 and RUF100; F4 is split out so stale F401 noqas stay detectable),
# plus upstream litellm's ruff config.

View file

@ -1,4 +1,4 @@
# Lint config for the test tree, which ruff.toml excludes from `ruff check`.
# Lint config for the test tree, which pyproject.toml excludes from `ruff check`.
#
# Every rule here catches a test that cannot fail. Rules land one at a time, each
# with its existing violations already fixed, so this list never needs a budget

62
.github/osv-scanner.toml vendored Normal file
View file

@ -0,0 +1,62 @@
[[IgnoredVulns]]
id = "GHSA-69hj-f68j-2879"
ignoreUntil = 2027-02-01
reason = "filelock UnixFileLock truncates and chmods a file another user renames or hard-links in at the lock path, in every release before 4.2.1, which needs Python >= 3.11. Python 3.11+ now locks 4.2.4 and only the Python 3.10 resolution stays on 3.20.3, held below 3.20.4 by a [tool.uv] constraint because every newer 3.10-compatible release carries more filelock advisories. litellm's CLI takes FileLock only inside ~/.litellm, which it creates owner-only, and huggingface_hub takes it in the Hugging Face cache, which litellm reaches only to download the Kimi K3 tokenizer for Bedrock token counting, so the attack needs another local user who can rename or link files in that cache. Remove once 3.10 is dropped"
[[IgnoredVulns]]
id = "GHSA-8mx5-7m5m-r77g"
ignoreUntil = 2027-02-01
reason = "filelock WindowsFileLock ignores mode in every release from 3.10.0 before 4.2.1, which needs Python >= 3.11. Python 3.11+ now locks 4.2.4 and only the Python 3.10 resolution stays on 3.20.3, held below 3.20.4 by a [tool.uv] constraint because every newer 3.10-compatible release carries more filelock advisories. The proxy image is Linux, so only Windows installs on Python 3.10 run it, and there litellm's CLI takes FileLock only inside the owner-only ~/.litellm, whose lock file holds no data, while huggingface_hub takes it in the Hugging Face cache, which litellm reaches only to download the Kimi K3 tokenizer. Remove once 3.10 is dropped"
[[IgnoredVulns]]
id = "GHSA-pcg2-g6cj-q92c"
ignoreUntil = 2027-02-01
reason = "tornado 6.6 (published 2026-10-21) is the only fix and requires Python >= 3.11, so the Python 3.10 resolution stays on 6.5.10 while 3.11+ locks 6.6.1; litellm starts no tornado server or client, since semantic-router and aurelio-sdk only declare it and tenacity and sentry-sdk import it only to wrap tornado code when it is present, so tornado's HTTP header parsing is never reached; remove this entry once 3.10 is dropped"
[[IgnoredVulns]]
id = "GHSA-pw9p-v7r6-gx4p"
ignoreUntil = 2027-02-01
reason = "tornado 6.6 (published 2026-10-21) is the only fix and requires Python >= 3.11, so the Python 3.10 resolution stays on 6.5.10 while 3.11+ locks 6.6.1; litellm starts no tornado server or client, since semantic-router and aurelio-sdk only declare it and tenacity and sentry-sdk import it only to wrap tornado code when it is present, so tornado's RedirectHandler is never reached; remove this entry once 3.10 is dropped"
[[IgnoredVulns]]
id = "GHSA-r4j5-9xpp-3985"
ignoreUntil = 2027-02-01
reason = "tornado 6.6 (published 2026-10-21) is the only fix and requires Python >= 3.11, so the Python 3.10 resolution stays on 6.5.10 while 3.11+ locks 6.6.1; litellm starts no tornado server or client, since semantic-router and aurelio-sdk only declare it and tenacity and sentry-sdk import it only to wrap tornado code when it is present, so tornado's chunked body reading is never reached; remove this entry once 3.10 is dropped"
[[IgnoredVulns]]
id = "GHSA-45c3-73f7-pv4m"
ignoreUntil = 2027-02-01
reason = "dnspython 2.9.0 (published 2026-10-09) is the only fix and requires Python >= 3.11, so the Python 3.10 resolution stays on 2.8.0 while 3.11+ locks 2.9.2; the advisory is CPU use in dns.name.from_wire_parser and litellm only reaches dnspython through pydantic EmailStr, which never does DNS lookups; remove this entry once 3.10 is dropped or a 3.10-compatible fix ships"
[[IgnoredVulns]]
id = "GHSA-mmhr-3c5m-f64f"
ignoreUntil = 2027-02-01
reason = "soupsieve 4.0 is the first fix and requires Python >= 3.11.5, so Python 3.10 and 3.11.0-3.11.4 resolve 2.10 while newer Python versions resolve 4.1. litellm declares beautifulsoup4 in its ci dependency group and uses BeautifulSoup in tests/code_coverage_tests/bedrock_pricing.py to parse AWS pricing HTML with html.parser and find table rows and cells, not CSS selectors; no runtime litellm code imports soupsieve or beautifulsoup4. Remove once 3.10 is dropped and the minimum Python version is at least 3.11.5"
[[PackageOverrides]]
name = "pillow"
version = "12.3.0"
ecosystem = "PyPI"
vulnerability.ignore = true
effectiveUntil = 2027-02-01
reason = "pillow 13.0.0 requires Python >= 3.11, so the Python 3.10 resolution of the ci and dev groups and of the mlflow extra stays on 12.3.0; 3.11+ is on 13.0.0; pillow has never shipped an older-major release after the next major (no 10.x after 11.0.0, no 11.x after 12.0.0), so 13.x is the only fix line; extended to 2027-02-01 with maintainer approval, same date as every other Python 3.10 entry; remove once 3.10 is dropped"
[[IgnoredVulns]]
id = "GHSA-vvf4-9x63-8xvv"
ignoreUntil = 2027-02-01
reason = "dnspython 2.9.2 is the only fix and requires Python >= 3.11, so the Python 3.10 resolution stays on 2.8.0 while 3.11+ locks 2.9.2; the advisory is memory and CPU use when dns.rdata.from_text parses a WKS record from attacker-controlled zone file text, and litellm only reaches dnspython through pydantic EmailStr, which never parses zone files or does DNS lookups; remove this entry once 3.10 is dropped or a 3.10-compatible fix ships"
[[IgnoredVulns]]
id = "GHSA-5cjw-f85j-cv6r"
ignoreUntil = 2027-02-01
reason = "tornado 6.6.1 is the only fix and requires Python >= 3.11, so the Python 3.10 resolution stays on 6.5.10 while 3.11+ locks 6.6.1; litellm starts no tornado server or client, since semantic-router and aurelio-sdk only declare it and tenacity and sentry-sdk import it only to wrap tornado code when it is present, so tornado's RedirectHandler is never reached; remove this entry once 3.10 is dropped"
[[IgnoredVulns]]
id = "GHSA-6655-6qjp-jx7x"
ignoreUntil = 2027-02-01
reason = "soupsieve 3.0.2 is the first fix and requires Python >= 3.11.5, so Python 3.10 and 3.11.0-3.11.4 resolve 2.10 while newer Python versions resolve 4.1. The advisory is exponential regex backtracking when compiling an untrusted CSS selector. litellm declares beautifulsoup4 in its ci dependency group and uses BeautifulSoup only in tests/code_coverage_tests/bedrock_pricing.py, which calls find and find_all and never compiles a CSS selector; no runtime litellm code imports soupsieve or beautifulsoup4. Remove once 3.10 is dropped and the minimum Python version is at least 3.11.5"
[[IgnoredVulns]]
id = "GHSA-7g44-g9rc-x53w"
ignoreUntil = 2027-02-01
reason = "soupsieve 3.0.2 is the first fix and requires Python >= 3.11.5, so Python 3.10 and 3.11.0-3.11.4 resolve 2.10 while newer Python versions resolve 4.1. The advisory is unbounded recursion when parsing an untrusted CSS selector with deeply nested pseudo-classes. litellm declares beautifulsoup4 in its ci dependency group and uses BeautifulSoup only in tests/code_coverage_tests/bedrock_pricing.py, which calls find and find_all and never parses a CSS selector; no runtime litellm code imports soupsieve or beautifulsoup4. Remove once 3.10 is dropped and the minimum Python version is at least 3.11.5"

49
.github/packaging/build_backend.py vendored Normal file
View file

@ -0,0 +1,49 @@
import sys
from collections.abc import Mapping
from pathlib import Path
from typing import Final
import maturin
from cli_package import PACKAGE_DIR, map_wheel
from maturin import (
build_sdist as build_sdist,
)
from maturin import (
get_requires_for_build_editable as get_requires_for_build_editable,
)
from maturin import (
get_requires_for_build_sdist as get_requires_for_build_sdist,
)
from maturin import (
get_requires_for_build_wheel as get_requires_for_build_wheel,
)
from maturin import (
prepare_metadata_for_build_editable as prepare_metadata_for_build_editable,
)
from maturin import (
prepare_metadata_for_build_wheel as prepare_metadata_for_build_wheel,
)
def build_wheel(
wheel_directory: str,
config_settings: Mapping[str, str | list[str]] | None = None,
metadata_directory: str | None = None,
) -> str:
filename: Final = maturin.build_wheel(wheel_directory, config_settings, metadata_directory)
map_wheel(Path(wheel_directory) / filename)
return filename
def build_editable(
wheel_directory: str,
config_settings: Mapping[str, str | list[str]] | None = None,
metadata_directory: str | None = None,
) -> str:
filename: Final = maturin.build_editable(wheel_directory, config_settings, metadata_directory)
map_wheel(Path(wheel_directory) / filename, source=Path(__file__).resolve().parents[2] / PACKAGE_DIR)
return filename
if __name__ == "__main__":
build_editable(sys.argv[1])

93
.github/packaging/cli_package.py vendored Normal file
View file

@ -0,0 +1,93 @@
import argparse
import base64
import copy
import csv
import hashlib
import io
import os
from pathlib import Path
from tempfile import NamedTemporaryFile
from typing import Final
from zipfile import ZipFile, ZipInfo
PACKAGE_DIR: Final = "litellm/cli"
PACKAGE_NAME: Final = "litellm_cli"
def editable_support(source: Path) -> tuple[tuple[str, bytes], ...]:
module: Final = "_litellm_cli_editable"
finder: Final = (
"import importlib.util\nimport sys\n\n"
"class _Finder:\n"
" @staticmethod\n"
" def find_spec(fullname, path=None, target=None):\n"
f" if fullname == {PACKAGE_NAME!r}:\n"
" return importlib.util.spec_from_file_location(\n"
f" fullname, {str(source / '__init__.py')!r},\n"
f" submodule_search_locations=[{str(source)!r}])\n"
" return None\n\n"
"sys.meta_path.insert(0, _Finder())\n"
)
return ((f"{module}.py", finder.encode()), (f"{module}.pth", f"import {module}\n".encode()))
def _name(name: str) -> str:
prefix: Final = f"{PACKAGE_DIR}/"
return f"{PACKAGE_NAME}/{name.removeprefix(prefix)}" if name.startswith(prefix) else name
def _info(original: ZipInfo) -> ZipInfo:
mapped: Final = copy.copy(original)
mapped.filename = _name(original.filename)
return mapped
def _record(entries: tuple[tuple[ZipInfo, bytes], ...], record: str) -> bytes:
buffer: Final = io.StringIO(newline="")
writer: Final = csv.writer(buffer, lineterminator="\n")
writer.writerows(
(
info.filename,
"sha256=" + base64.urlsafe_b64encode(hashlib.sha256(data).digest()).rstrip(b"=").decode(),
str(len(data)),
)
for info, data in entries
)
writer.writerow((record, "", ""))
return buffer.getvalue().encode()
def map_wheel(wheel: Path, *, source: Path | None = None) -> None:
with ZipFile(wheel) as archive:
records: Final = tuple(info for info in archive.infolist() if info.filename.endswith(".dist-info/RECORD"))
if len(records) != 1:
raise ValueError("A wheel must contain exactly one RECORD")
entries: Final = tuple(
(_info(info), archive.read(info))
for info in archive.infolist()
if info != records[0] and (source is None or not info.filename.startswith(f"{PACKAGE_DIR}/"))
)
additions: Final = () if source is None else tuple((ZipInfo(name), data) for name, data in editable_support(source))
contents: Final = (*entries, *additions)
names: Final = tuple(info.filename for info, _ in contents)
if len(names) != len(frozenset(names)):
raise ValueError("The CLI package mapping would overwrite a wheel member")
if source is None and not any(name.startswith(f"{PACKAGE_NAME}/") for name in names):
raise ValueError("The wheel does not contain the CLI package")
with NamedTemporaryFile(dir=wheel.parent, suffix=".whl", delete=False) as temporary:
destination: Final = Path(temporary.name)
try:
with ZipFile(destination, "w") as archive:
for info, data in contents:
archive.writestr(info, data)
archive.writestr(records[0], _record(contents, records[0].filename))
os.replace(destination, wheel)
finally:
destination.unlink(missing_ok=True)
if __name__ == "__main__":
parser: Final = argparse.ArgumentParser()
parser.add_argument("wheels", nargs="+", type=Path)
for wheel in parser.parse_args().wheels:
map_wheel(wheel)

View file

@ -0,0 +1,210 @@
## Captured API proof
These are actual curl requests to a local proxy and a controlled HTTP provider, using synthetic records and keys. Addresses, header files and IDs below are placeholders. Response excerpts omit unrelated fields and credentials.
### Permission change: verified before and after
The baseline process ran clean relevant source at `6685525fa2a809e0d4bc2c13cb07aef8eec242c3`. The requests used the same bound key and org policy; the after request was rerun after the schema update: the org allowed only `gpt-4.1-mini`; the key allowed `usage-proof-model` and was bound with `metadata.customer_org_id`. No team/member was needed for this org-level comparison.
```sh
curl --silent --show-error --request POST "$PROXY_BASE/v1/chat/completions" \
--header "@$SCOPED_KEY_HEADERS" --header 'Content-Type: application/json' \
--data-binary @request.json
```
`request.json`:
```json
{
"model": "usage-proof-model",
"messages": [
{
"role": "user",
"content": "Synthetic before-after permission check"
}
]
}
```
Baseline: **HTTP 200**.
```json
{
"object": "chat.completion",
"model": "usage-proof-model",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"content": "Controlled usage proof response.",
"role": "assistant"
},
"provider_specific_fields": {}
}
]
}
```
Current implementation: **HTTP 403**, with **zero controlled-provider calls**.
```json
{
"error": {
"message": "The requested model 'usage-proof-model' is not in the allowed models for this customer. Check the models this customer can use and try again.",
"type": "customer_model_access_denied",
"param": "model",
"code": "403"
}
}
```
### Current hierarchy and key intersection
Admin requests created the records below; each returned **201**. Member `budget_id` selects its template and `budget_override_id` identifies its optional override:
`POST /management/v1/customer_organizations`
```json
{
"customer_org_id": "<org-id>",
"alias": "Permissions and deletion proof",
"budget": {
"max_budget": 500,
"allowed_models": [
"gpt-4.1-mini",
"gpt-5.5"
]
},
"default_budgets": [
{
"name": "Standard",
"is_default": true,
"max_budget": 100,
"allowed_models": [
"gpt-4.1-mini"
]
}
]
}
```
`POST /management/v1/customer_teams`
```json
{
"customer_team_id": "<team-id>",
"parent_org_id": "<org-id>",
"alias": "Proof customer team",
"budget": {
"max_budget": 250,
"allowed_models": [
"gpt-4.1-mini",
"gpt-5.5"
]
}
}
```
`POST /management/v1/customer_members`
```json
{
"customer_member_id": "<member-id>",
"parent_team_id": "<team-id>",
"alias": "Proof customer member",
"budget_override": {
"max_budget": 100,
"allowed_models": [
"gpt-4.1-mini"
]
}
}
```
A synthetic `internal_user` was created with `auto_create_key: false`. The admin then created these keys through `POST /key/generate` (each **200**; returned credentials omitted):
| Key | models | metadata / budget binding |
| --- | --- | --- |
| A | `["gpt-4.1-mini", "gpt-5.5"]` | `{"customer_org_id":"<org-id>","customer_member_id":"<member-id>"}` |
| B | `["gpt-4.1-mini"]` | Same org/member binding |
| C | `["gpt-4.1-mini"]` | `budget_id: "<shared-budget-id>"`; no customer binding |
Inference curl uses the appropriate key header file. Captured request bodies:
```json
{
"model": "gpt-4.1-mini",
"messages": [
{
"role": "user",
"content": "Synthetic permission check"
}
]
}
```
```json
{
"model": "gpt-4.1-mini",
"input": "Synthetic permission check"
}
```
| State / key | Requested model | Chat HTTP | Responses HTTP | Provider calls per request |
| --- | --- | --- | --- | --- |
| Member restriction / A | gpt-4.1-mini | 200 | 200 | 1 |
| Member restriction / A | gpt-5.5 | 403 | 403 | 0 |
| Member widened; key B remains narrow | gpt-5.5 | 403 | 403 | 0 |
| Member widened; org narrowed / A | gpt-5.5 | 403 | 403 | 0 |
The widening PATCH was `{"budget_override":{"allowed_models":["gpt-4.1-mini","gpt-5.5"]}}` on the member. The org narrowing PATCH was `{"budget":{"allowed_models":["gpt-4.1-mini"]}}`. Key B still denied the model after the member widened.
### Cascade, retained history and shared key budget
**Same-revision comparison:** deleting the populated org without `cascade=true` returned **409**:
```json
{
"type": "urn:litellm:error:customer-hierarchy-has-children",
"title": "Has children",
"status": 409,
"detail": "Delete the customer's children before deleting the parent."
}
```
The confirmed cascade request returned **204**, with an empty body:
```sh
curl --silent --show-error --request DELETE \
"$PROXY_BASE/management/v1/customer_organizations/$ORG_ID?cascade=true" \
--header "@$ADMIN_HEADERS"
```
Subsequent GETs for that org, its team and its member each returned **404**. The existing demo organization still returned **200**.
Key C referenced the org-owned budget. After PATCHing that budget to `max_budget: 0`, its inference returned **422 `budget_exceeded` before and after cascade**, with zero provider calls. Its authenticated `GET /key/info` retained the same relation and limit (selected fields):
```json
{
"info": {
"budget_id": "<shared-budget-id>",
"litellm_budget_table": {
"max_budget": 0.0
}
}
}
```
The same admin daily query returned **200** before and after cascade:
```sh
curl --silent --show-error \
"$PROXY_BASE/management/v1/customer_organizations/daily/activity?customer_org_ids=$ORG_ID&start_date=$UTC_DATE&end_date=$UTC_DATE" \
--header "@$ADMIN_HEADERS"
```
Selected captured metadata was identical before and after; API-key/entity breakdowns are omitted:
```json
{
"metadata": {
"total_spend": 0.28,
"total_prompt_tokens": 200,
"total_completion_tokens": 40,
"total_tokens": 240,
"total_api_requests": 4,
"total_successful_requests": 2,
"total_failed_requests": 2
}
}
```

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB

View file

@ -0,0 +1,69 @@
The OpenAI passthrough route now resolves customer organization, team and effective member model restrictions before forwarding the request. It uses the existing customer resolver and model matcher, then checks the model again after hooks. Gateway key permissions still apply.
Actual curl proof used a controlled HTTP upstream. Before: PR head `43d55ab2b91e4cccccf014c5f813a2b70a4e89f9` merged with target `eb811913f1`, before the fix. After: the implementation linked from the current PR description. Counts below come from the upstream request log.
Setup requests, using synthetic identifiers:
```http
POST /management/v1/customer_organizations
{"customer_org_id":"proof-org","alias":"Synthetic passthrough permission proof","budget":{"max_budget":100,"allowed_models":["gpt-4.1-mini"]}}
POST /user/new
{"user_id":"proof-user","user_role":"internal_user","auto_create_key":false}
POST /key/generate
{"user_id":"proof-user","models":["gpt-4.1-mini","gpt-5.5"],"metadata":{"customer_org_id":"proof-org"}}
POST /key/generate
{"user_id":"proof-user","models":["gpt-4.1-mini"],"metadata":{"customer_org_id":"proof-org"}}
```
The first key permits both models; the second permits only mini. Credentials from both responses are omitted.
```bash
curl "$GATEWAY_BASE_URL/openai/v1/chat/completions" \
--header "@$CUSTOMER_KEY_AB_HEADERS" \
--header 'Content-Type: application/json' \
--data '{"model":"gpt-5.5","messages":[{"role":"user","content":"Synthetic passthrough policy check"}]}'
```
Before: HTTP 200, one upstream call. Captured response excerpt:
```json
{
"model": "gpt-5.5",
"usage": {
"prompt_tokens": 100,
"completion_tokens": 20,
"total_tokens": 120
}
}
```
After: HTTP 403, zero upstream calls. Captured response:
```json
{
"error": {
"message": "The requested model 'gpt-5.5' is not in the allowed models for this customer. Check the models this customer can use and try again.",
"type": "customer_model_access_denied",
"param": "model",
"code": "403"
}
}
```
The same key still receives HTTP 200 for `gpt-4.1-mini`, with one upstream call. After widening the organization to permit both models, the mini-only key still receives HTTP 403 for `gpt-5.5`, with zero upstream calls:
```json
{
"error": {
"message": "The requested model 'gpt-5.5' is not available for this API key, or the model name is invalid. Check the models available to you and try again.",
"type": "key_model_access_denied",
"param": "model",
"code": "403"
}
}
```
The proof deletes its synthetic organization afterward. Registered-route regressions also cover team restrictions, member templates and overrides, hook model rewrites, model-less file retrieval, and the existing configured passthrough model contract.

Binary file not shown.

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 61 KiB

View file

@ -10,6 +10,7 @@ import warnings
from collections.abc import Callable, Iterable, Mapping, Sequence
from dataclasses import dataclass
from functools import lru_cache
from itertools import chain
from types import MappingProxyType
from typing import Final
@ -17,7 +18,6 @@ import yaml
REPO_ROOT = pathlib.Path(__file__).resolve().parents[2]
WORKFLOW_DIR = REPO_ROOT / ".github" / "workflows"
CIRCLECI_CONFIG = REPO_ROOT / ".circleci" / "config.yml"
ALLOWLIST_FILE = REPO_ROOT / ".github" / "ci-coverage-allowlist.yml"
TESTS_ROOT = REPO_ROOT / "tests"
@ -25,14 +25,18 @@ ALLOWLIST_KEYS = frozenset({"description", "test_paths", "dockerfiles"})
PATH_FILTER_KEYS = frozenset({"paths", "paths-ignore"})
TEST_PATH_KEYS = frozenset({"test-path", "test-paths", "test_path"})
DOCKERFILE_INPUT_KEYS = frozenset({"file", "dockerfile"})
TEST_RUNNER_RE = re.compile(r"\bpytest\b|\bcircleci tests\b|\bhelm unittest\b|\bplaywright test\b|\bpython[0-9.]*\s")
TEST_RUNNER_RE = re.compile(r"\bpytest\b|\bhelm unittest\b|\bplaywright test\b|\bpython[0-9.]*\s")
IMAGE_BUILD_RE = re.compile(r"\bdocker\s+(?:buildx\s+)?build\b")
TEST_TOKEN_RE = re.compile(r"tests/[A-Za-z0-9_./*?-]+")
IGNORE_ARG_RE: Final = re.compile(r"--ignore(?:-glob)?[= ](\S+)")
TEST_TOKEN_RE = re.compile(r"tests/[A-Za-z0-9_./*?\[\]-]+")
IGNORE_ARG_RE: Final = re.compile(r"--(?:ignore(?:-glob)?|deselect)[= ](\S+)")
DOCKERFILE_TOKEN_RE = re.compile(r"[A-Za-z0-9_./-]*Dockerfile[A-Za-z0-9_.-]*")
COMMENT_RE = re.compile(r"^\s*#.*$", re.MULTILINE)
GLOB_CHARS = frozenset("*?")
DEDICATED_INTEGRATION_WORKFLOWS: Final = MappingProxyType({"image-scan.yml": "tests/integration/database/image/"})
GLOB_CHARS = frozenset("*?[")
DEDICATED_INTEGRATION_WORKFLOWS: Final = MappingProxyType(
{
"image-scan.yml": "tests/integration/database/image/",
}
)
# Trees whose jobs are sharded with no catch-all bucket, so every child that holds
# tests has to be named by some shard or it runs nowhere. A child listed here is
@ -115,9 +119,7 @@ def _scalars(node: object, key: str) -> tuple[Scalar, ...]:
def _config_files() -> tuple[pathlib.Path, ...]:
workflows = tuple(sorted(path for path in WORKFLOW_DIR.iterdir() if path.suffix in (".yml", ".yaml")))
circleci = (CIRCLECI_CONFIG,) if CIRCLECI_CONFIG.is_file() else ()
return workflows + circleci
return tuple(sorted(path for path in WORKFLOW_DIR.iterdir() if path.suffix in (".yml", ".yaml")))
def _all_scalars() -> tuple[Scalar, ...]:
@ -243,7 +245,6 @@ def _describe(paths: tuple[str, ...]) -> str:
return f"{len(paths)} test file(s) invoked by no job: {names}{suffix}"
GLOB_CALL_RE = re.compile(r'circleci tests glob "([^"]+)"')
KEYWORD_RE = re.compile(r"-k\s+\\?[\"']([^\"'\\]+)")
@ -316,16 +317,14 @@ def _keyword_terms(
def _slices() -> tuple[Slice, ...]:
if not CIRCLECI_CONFIG.exists():
return ()
test_files: Final = frozenset(_test_files())
jobs: Final = yaml.safe_load(CIRCLECI_CONFIG.read_text()).get("jobs", {})
return tuple(
Slice(job=job, globs=globs, named=named, required=required, excluded=excluded, understood=understood)
for job, body in jobs.items()
Slice(job=f"{path.name}:{job}", globs=globs, named=named, required=required, excluded=excluded, understood=understood)
for path in _config_files()
for job, body in yaml.safe_load(path.read_text()).get("jobs", {}).items()
for text in ("\n".join(_strings(body)),)
if "pytest" in text
for globs in (tuple(GLOB_CALL_RE.findall(text)),)
for globs in (tuple(token for token in TEST_TOKEN_RE.findall(text) if GLOB_CHARS & set(token)),)
for named in (frozenset(TEST_TOKEN_RE.findall(text)) & test_files,)
for required, excluded, understood in (
_keyword_terms(tuple(KEYWORD_RE.findall(text)), attributable=len(globs) < 2),
@ -352,16 +351,11 @@ def _matchable_names(relative_path: str) -> frozenset[str]:
def _workflow_named_tokens() -> frozenset[str]:
"""Test tokens a GitHub Actions job names directly.
A CircleCI `-k` that deselects a file no longer means the file runs nowhere once a
workflow names it, so the slice check has to credit those the same way the census does.
"""
return _invoked_test_tokens(
scalar
for path in _config_files()
if path != CIRCLECI_CONFIG
for scalar in _scalars(yaml.safe_load(path.read_text(encoding="utf-8")), path.name)
if not KEYWORD_RE.search(scalar.value)
)
@ -566,13 +560,28 @@ def _owns(workflow: str, relative_path: str) -> bool:
return owned_root is not None and relative_path.startswith(owned_root)
def _expression_suites(value: str) -> tuple[str, ...]:
suites: Final = chain.from_iterable(map(json.loads, re.findall(r"'(\[[^']*\])'", value)))
return tuple(suite for suite in suites if isinstance(suite, str) and "{" not in suite)
def _matrix_suites(value: object) -> tuple[str, ...]:
if isinstance(value, list):
return tuple(suite for suite in value if isinstance(suite, str))
if isinstance(value, str):
return _expression_suites(value)
return ()
def _tokens_outside_owner(relative_path: str, workflow_tokens: Mapping[str, frozenset[str]]) -> frozenset[str]:
return frozenset[str]().union(
*(tokens for workflow, tokens in workflow_tokens.items() if not _owns(workflow, relative_path))
)
def _integration_ownership(repo_root: pathlib.Path = REPO_ROOT) -> tuple[frozenset[str], tuple[Finding, ...]]:
def _integration_ownership(
repo_root: pathlib.Path = REPO_ROOT, *, allowlist: Allowlist | None = None
) -> tuple[frozenset[str], tuple[Finding, ...]]:
runner: Final = repo_root / "tests/integration/run.py"
if not runner.exists():
return frozenset(), ()
@ -589,26 +598,19 @@ def _integration_ownership(repo_root: pathlib.Path = REPO_ROOT) -> tuple[frozens
browser_manifest: Final = repo_root / "tests/e2e/ui/tests/integrationCritical/expected.json"
browser_nodes: Final = json.loads(browser_manifest.read_text()) if browser_manifest.exists() else ()
browser_paths: Final = frozenset(node.split("::", 1)[0] for node in browser_nodes)
circle_path: Final = repo_root / ".circleci/config.yml"
circle: Final = yaml.safe_load(circle_path.read_text()) if circle_path.exists() else {}
circle_test_path_tokens: Final = _invoked_test_tokens(
scalar for scalar in _scalars(circle, "config.yml") if scalar.key == "test_path"
)
steps: Final = circle.get("jobs", {}).get("integration_contracts", {}).get("steps", ())
integration_path: Final = repo_root / ".github/workflows/test-integration.yml"
integration: Final = yaml.safe_load(integration_path.read_text()) if integration_path.exists() else {}
integration_jobs: Final = integration.get("jobs", {})
invoked: Final = any(
".circleci/scripts/run_integration.sh" in scalar.value
for scalar in _scalars(steps, "integration_contracts")
if scalar.key == "command"
".github/scripts/ci/run_integration.sh" in scalar.value
for scalar in _scalars(integration_jobs, "integration")
if scalar.key == "run"
)
scheduled: Final = frozenset(
suite
for job in circle.get("workflows", {}).get("integration", {}).get("jobs", ())
if isinstance(job, dict) and "integration_contracts" in job
for suite in job["integration_contracts"]
.get("matrix", {})
.get("parameters", {})
.get("suite", (job["integration_contracts"].get("suite"),))
if isinstance(suite, str)
for body in integration_jobs.values()
if any(".github/scripts/ci/run_integration.sh" in scalar.value for scalar in _scalars(body, "integration"))
for suite in _matrix_suites(body.get("strategy", {}).get("matrix", {}).get("suite", ()))
)
required: Final = (frozenset({"browser"}) if browser_paths else frozenset()) | frozenset(
group
@ -630,19 +632,27 @@ def _integration_ownership(repo_root: pathlib.Path = REPO_ROOT) -> tuple[frozens
}
)
gha_tokens: Final = frozenset[str]().union(*workflow_tokens.values())
circle_owned: Final = (
invoked_github_files: Final = frozenset(
path for path in github_files if any(_token_covers(token, path) for token in gha_tokens)
)
grouped: Final = (
frozenset(str(path.relative_to(repo_root)) for path in integration_root.rglob("test_*.py")) - github_files
)
findings: Final = (
tuple(
Finding(path, "integration file outside GITHUB_FILES is selected by GitHub Actions")
for path in sorted(circle_owned)
if any(_token_covers(token, path) for token in gha_tokens)
for path in sorted(grouped)
if any(
_token_covers(token, path)
for workflow, tokens in workflow_tokens.items()
if workflow != "test-integration.yml"
for token in tokens
)
)
+ tuple(
Finding(path, "GitHub-owned integration contract has no invoking job")
for path in sorted(github_files)
if not any(_token_covers(token, path) for token in gha_tokens | circle_test_path_tokens)
if path not in invoked_github_files and not (allowlist is not None and allowlist.covers_test(path))
)
+ tuple(
Finding(path, "GitHub-owned integration file is missing")
@ -658,6 +668,7 @@ def _integration_ownership(repo_root: pathlib.Path = REPO_ROOT) -> tuple[frozens
browser_commands: Final = tuple(
scalar.value
for path in (repo_root / ".github/workflows").glob("*.y*ml")
if path.name != "test-integration.yml"
for scalar in _scalars(yaml.safe_load(path.read_text()), path.name)
if scalar.key in {"run", "command"}
)
@ -690,14 +701,14 @@ def _integration_ownership(repo_root: pathlib.Path = REPO_ROOT) -> tuple[frozens
else ()
)
group_findings: Final = tuple(
Finding(group, "canonical integration group is not scheduled by CircleCI")
Finding(group, "canonical integration group is not scheduled by GitHub Actions")
for group in sorted(required - scheduled)
) + tuple(Finding(path, "canonical node must have exactly one integration group") for path in sorted(ungrouped))
if not paths or not invoked or not scheduled:
return frozenset(), findings + (
Finding(str(runner.relative_to(repo_root)), "dedicated CircleCI runner is missing"),
Finding(str(runner.relative_to(repo_root)), "dedicated GitHub Actions runner is missing"),
)
return paths | browser_paths | github_files, findings + group_findings + browser_findings + exclusion_findings
return paths | browser_paths | invoked_github_files, findings + group_findings + browser_findings + exclusion_findings
def main() -> int:
@ -709,7 +720,7 @@ def main() -> int:
allowlist = _load_allowlist()
scalars = _all_scalars()
integration_paths, ownership_findings = _integration_ownership()
integration_paths, ownership_findings = _integration_ownership(allowlist=allowlist)
test_findings = _uncovered_tests(
allowlist,
_invoked_selections(scalars) + (Selection(included=integration_paths, ignored=frozenset()),),
@ -731,8 +742,8 @@ def main() -> int:
)
if ownership_findings:
browser_remedy = (
" CircleCI's browser group owns the integrationCritical Playwright contracts, so no GitHub workflow may "
"select them and the default Playwright config must exclude them."
" The browser group in test-integration.yml owns the integrationCritical Playwright contracts. "
"Other workflows and the default Playwright config must exclude them."
if any("browser" in finding.detail or "Playwright" in finding.detail for finding in ownership_findings)
else ""
)

119
.github/scripts/assert_secret_absent.py vendored Normal file
View file

@ -0,0 +1,119 @@
import base64
import io
import os
import sys
import urllib.parse
import zipfile
from collections.abc import Callable, Iterator, Mapping
from dataclasses import dataclass
from functools import reduce
from itertools import chain
from pathlib import Path
from types import MappingProxyType
from typing import Final, TextIO
SECRET_VARIABLE: Final = "SCANNED_SECRET"
MIN_ENCODED_LENGTH: Final = 12
MAX_ARCHIVE_DEPTH: Final = 3
MASK: Final = "***"
Forms = tuple[tuple[str, str], ...]
@dataclass(frozen=True, slots=True)
class Hit:
location: str
form: str
def _aligned_encodings(raw: bytes, name: str, encode: Callable[[bytes], bytes]) -> Iterator[tuple[str, str]]:
for offset in range(3):
chunk: Final = raw[offset:]
yield f"{name} offset {offset}", encode(chunk[: len(chunk) - len(chunk) % 3]).decode()
def secret_forms(secret: str) -> Forms:
encoded: Final = (
("url-encoded", urllib.parse.quote(secret, safe="")),
*_aligned_encodings(secret.encode(), "base64", base64.b64encode),
*_aligned_encodings(secret.encode(), "base64url", base64.urlsafe_b64encode),
)
by_value: Final = MappingProxyType(
{value: name for name, value in encoded if value != secret and len(value) >= MIN_ENCODED_LENGTH}
)
return (("raw", secret), *((name, value) for value, name in by_value.items()))
def mask(text: str, forms: Forms) -> str:
return reduce(lambda masked, form: masked.replace(form[1], MASK), forms, text)
def _name_hits(location: str, name: str, forms: Forms) -> Iterator[Hit]:
for form, value in forms:
if value in name:
yield Hit(mask(location, forms), f"{form} in name")
def _content_hits(location: str, data: bytes, forms: Forms, depth: int) -> Iterator[Hit]:
for form, value in forms:
if value.encode() in data:
yield Hit(mask(location, forms), form)
if depth >= MAX_ARCHIVE_DEPTH or not zipfile.is_zipfile(io.BytesIO(data)):
return
with zipfile.ZipFile(io.BytesIO(data)) as archive:
for member in archive.infolist():
if member.is_dir():
continue
yield from _content_hits(f"{location}!{member.filename}", archive.read(member), forms, depth + 1)
def _walk(root: Path) -> Iterator[tuple[Path, bool]]:
for directory, subdirectories, files in os.walk(root):
yield from ((Path(directory, name), True) for name in subdirectories if Path(directory, name).is_symlink())
yield from ((Path(directory, name), False) for name in files)
def _path_hits(root: Path, path: Path, linked_directory: bool, forms: Forms) -> Iterator[Hit]:
location: Final = str(path.relative_to(root))
if linked_directory:
yield Hit(mask(location, forms), "directory symlink, not scanned")
return
yield from _name_hits(location, location, forms)
yield from _content_hits(location, path.read_bytes(), forms, 0)
def scan(root: Path, secret: str) -> tuple[int, tuple[Hit, ...]]:
forms: Final = secret_forms(secret)
entries: Final = tuple(_walk(root))
hits: Final = tuple(chain.from_iterable(_path_hits(root, path, linked, forms) for path, linked in entries))
return len(entries), hits
def main(argv: tuple[str, ...], environ: Mapping[str, str], out: TextIO) -> int:
if len(argv) != 1:
print(f"usage: assert_secret_absent.py <directory>, with the secret in ${SECRET_VARIABLE}", file=out)
return 2
secret: Final = environ.get(SECRET_VARIABLE, "")
root: Final = Path(argv[0])
if not secret:
print(f"{SECRET_VARIABLE} is empty, nothing to scan for", file=out)
return 0
if not root.is_dir():
print(f"{root} is not a directory, nothing to scan", file=out)
return 0
try:
scanned, hits = scan(root, secret)
except Exception as error:
print(f"scan failed with {type(error).__name__}, treating the results as unsafe", file=out)
return 1
for hit in hits:
print(f"secret found: {hit.location} ({hit.form})", file=out)
if hits:
print(f"{len(hits)} copies of the secret under {root}, refusing to upload", file=out)
return 1
print(f"no copy of the secret in {scanned} files under {root}", file=out)
return 0
if __name__ == "__main__":
sys.exit(main(tuple(sys.argv[1:]), os.environ, sys.stdout))

68
.github/scripts/ci/classify_changes.sh vendored Executable file
View file

@ -0,0 +1,68 @@
#!/usr/bin/env bash
set -uo pipefail
category="${1:?usage: classify_changes.sh <backend|client|ui|cost-map-only|mcp-dependencies|windows-release|redis-compat>}"
has_client=false
has_backend=false
has_ci=false
has_cost_map=false
has_mcp_dependencies=false
has_windows_release=false
has_redis_compat=false
outside_cost_map_set=false
while IFS= read -r file || [ -n "$file" ]; do
[ -n "$file" ] || continue
case "$file" in
litellm/_redis.py | litellm/_redis_credential_provider.py | litellm/caching/redis_cache.py | litellm/caching/evicted_client_closer.py | tests/unit/test_redis.py | tests/integration/sdk/test_redis_cluster_iam_auth.py | tests/unit/caching/test_redis_connection_pool.py | tests/unit/caching/test_redis_cluster_cache.py | tests/unit/caching/test_evicted_client_closer.py | .github/workflows/test-unit.yml | .github/actions/detect-changes/* | .github/scripts/detect_changes.sh | .github/scripts/ci/classify_changes.sh | tests/integration/run.py | pyproject.toml | uv.lock)
has_redis_compat=true ;;
esac
case "$file" in
*.md | *.mdx) : ;;
pyproject.toml | */pyproject.toml | uv.lock | uv.toml | .python-version | litellm/rust/* | litellm/__init__.py | litellm/proxy/proxy_server.py | litellm/*mcp* | tests/*mcp* | litellm/integrations/arize/* | tests/base_sdk_tests/* | litellm/scripts/check_mcp_sdk_install.py | .github/workflows/test-mcp-dependency-resolution.yml | .github/actions/detect-changes/* | .github/actions/setup-uv-with-retries/* | .github/actions/cache-cargo-build/* | .github/scripts/detect_changes.sh | .github/scripts/uv_sync_with_retries.sh | .github/scripts/ci/classify_changes.sh | tests/unit/test_ci_path_filter.py | tests/unit/test_detect_changes.py)
has_mcp_dependencies=true ;;
esac
case "$file" in
litellm/rust/* | litellm/rust_bridge/* | pyproject.toml | uv.lock | tests/windows_tests/* | .github/scripts/ci/* | .github/actions/*)
has_windows_release=true ;;
esac
case "$file" in
litellm/ui/* | tests/e2e/ui/*) has_client=true ;;
*.md | *.mdx) : ;;
.github/*) has_ci=true; has_backend=true ;;
*) has_backend=true ;;
esac
case "$file" in
model_prices_and_context_window.json | litellm/model_prices_and_context_window_backup.json)
has_cost_map=true ;;
tests/unit/proxy/*) : ;;
*) outside_cost_map_set=true ;;
esac
done
case "$category" in
mcp-dependencies)
[ "$has_mcp_dependencies" = true ] && echo run || echo skip
;;
cost-map-only)
{ [ "$has_cost_map" = true ] && [ "$outside_cost_map_set" = false ]; } && echo run || echo skip
;;
windows-release)
[ "$has_windows_release" = true ] && echo run || echo skip
;;
redis-compat)
[ "$has_redis_compat" = true ] && echo run || echo skip
;;
backend)
[ "$has_backend" = true ] && echo run || echo skip
;;
client)
{ [ "$has_client" = true ] || [ "$has_backend" = true ]; } && echo run || echo skip
;;
ui)
{ [ "$has_client" = true ] || [ "$has_ci" = true ]; } && echo run || echo skip
;;
*)
echo run
;;
esac

View file

@ -6,10 +6,12 @@ here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
run_full() {
echo "path-filter[$category]: running job ($1)"
echo "decision=run"
[ -z "${GITHUB_OUTPUT:-}" ] || echo "decision=run" >> "$GITHUB_OUTPUT"
exit 0
}
[ -n "${CIRCLE_PULL_REQUEST:-}" ] || run_full "not a pull request"
[ "${GITHUB_EVENT_NAME:-}" = pull_request ] || run_full "not a pull request"
candidate_bases="${PATH_FILTER_BASE_BRANCH:-main}"
merge_base=""
@ -36,5 +38,6 @@ if [ "$decision" = run ]; then
run_full "$category-relevant changes detected"
fi
echo "path-filter[$category]: only unrelated changes detected; halting job as successful"
circleci-agent step halt
echo "path-filter[$category]: only unrelated changes detected"
echo "decision=skip"
[ -z "${GITHUB_OUTPUT:-}" ] || echo "decision=skip" >> "$GITHUB_OUTPUT"

View file

@ -1,14 +1,14 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "${GITHUB_ACTIONS:-}" = true ]; then
echo "Integration contracts are owned by CircleCI" >&2
exit 1
fi
suite="${1:?integration suite required}"
mode="${2:-standard}"
side="${3:-}"
requested_workers="${INTEGRATION_WORKERS:-}"
license_env=()
if [ -n "${LITELLM_LICENSE:-}" ]; then
license_env=("LITELLM_LICENSE=$LITELLM_LICENSE")
fi
if [ "$mode" = replica ]; then
results="test-results/integration-${suite}-replica"
elif [ "$mode" = parity ]; then
@ -30,7 +30,7 @@ egress_cgroup=litellm-integration
cleanup() {
original_status=$?
trap - EXIT INT TERM
sudo .venv/bin/python .circleci/scripts/stop_integration_processes.py \
sudo .venv/bin/python .github/scripts/ci/stop_integration_processes.py \
"$integration_identity" "$(id -u)" "$proxy_pid" "$peer_pid" "$upstream_pid" \
> "$results/process-cleanup.txt" 2>&1 || original_status=1
for owned_pid in "$peer_pid" "$proxy_pid" "$upstream_pid"; do
@ -81,10 +81,10 @@ export INTEGRATION_UPSTREAM_URL=http://127.0.0.1:8190
export INTEGRATION_MASTER_KEY="$LITELLM_MASTER_KEY"
export LITELLM_UI_PATH="$PWD/litellm/proxy/_experimental/out"
if [ "$suite" = browser ]; then
export LITELLM_UI_PATH="$PWD/ui/litellm-dashboard/out"
export LITELLM_UI_PATH="$PWD/litellm/ui/litellm-dashboard/out"
test -f "$LITELLM_UI_PATH/index.html"
fi
export INTEGRATION_SEED="$(.venv/bin/python -c 'import hashlib,os; print(int(hashlib.sha256((os.environ.get("CIRCLE_SHA1", "local") + os.environ.get("CIRCLE_WORKFLOW_ID", "local")).encode()).hexdigest()[:8],16))')"
export INTEGRATION_SEED="$(.venv/bin/python -c 'import hashlib,os; print(int(hashlib.sha256(((os.environ.get("GITHUB_SHA") or "local") + (os.environ.get("GITHUB_RUN_ID") or "local")).encode()).hexdigest()[:8],16))')"
export INTEGRATION_ORDER_SEED="$INTEGRATION_SEED"
uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma > "$results/prisma-generate.log" 2>&1
@ -93,7 +93,7 @@ export INTEGRATION_PROXY_DATABASE_URL=""
export INTEGRATION_PROXY_READ_REPLICA_URL=""
export INTEGRATION_ROUTING=""
if [ "$mode" = replica ] || [ "$mode" = parity ]; then
.venv/bin/python .circleci/scripts/prepare_replica_roles.py > "$results/prepare-replica-roles.log" 2>&1
.venv/bin/python .github/scripts/ci/prepare_replica_roles.py > "$results/prepare-replica-roles.log" 2>&1
export INTEGRATION_PROXY_DATABASE_URL="postgresql://litellm_writer:litellm-writer@127.0.0.1:5432/circle_test"
export INTEGRATION_PROXY_READ_REPLICA_URL="postgresql://litellm_reader:litellm-reader@127.0.0.1:5432/circle_test"
fi
@ -107,7 +107,7 @@ sudo iptables -N integration_only
guard_created=true
sudo iptables -A integration_only -o lo -j ACCEPT
sudo iptables -A integration_only -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
for service in postgres-db redis-cache; do
for service in "${INTEGRATION_POSTGRES_CONTAINER:-postgres-db}" "${INTEGRATION_REDIS_CONTAINER:-redis-cache}"; do
address="$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$service")"
sudo iptables -A integration_only -d "$address" -j ACCEPT
done
@ -128,6 +128,8 @@ fi
sudo iptables -L integration_only -n -v -x > "$results/egress-guard.txt"
awk '$3 == "REJECT" && $1 > 0 { rejected=1 } END { exit !rejected }' "$results/egress-guard.txt"
export UV_OFFLINE=true
if [ "$mode" = parity ]; then
export INTEGRATION_WORKERS=1
else
@ -145,6 +147,9 @@ fi
if [ "$suite" = mcp ]; then
export INTEGRATION_WORKERS=2 INTEGRATION_COVERAGE=1
fi
if [ -n "$requested_workers" ]; then
export INTEGRATION_WORKERS="$requested_workers"
fi
isolated_workers=false
if [ "${INTEGRATION_WORKERS:-1}" -gt 1 ]; then
isolated_workers=true
@ -185,7 +190,7 @@ start_proxy() {
"${database_env[@]}" REDIS_HOST="$REDIS_HOST" REDIS_PORT="$REDIS_PORT" \
INTEGRATION_UPSTREAM_URL="$INTEGRATION_UPSTREAM_URL" \
LITELLM_MASTER_KEY="$LITELLM_MASTER_KEY" LITELLM_SALT_KEY="$LITELLM_SALT_KEY" LITELLM_UI_PATH="$LITELLM_UI_PATH" PROXY_BASE_URL="http://127.0.0.1:$port" \
LITELLM_LICENSE="${LITELLM_LICENSE:-}" \
"${license_env[@]}" \
LITELLM_MODE=PRODUCTION STORE_MODEL_IN_DB=True LITELLM_ENABLE_MCP_STDIO=true "${cost_map_env[@]}" \
AWS_EC2_METADATA_DISABLED=true DO_NOT_TRACK=1 COVERAGE_FILE="$coverage_data" \
"${proxy_command[@]}" --config tests/integration/proxy_config.yaml \
@ -197,7 +202,7 @@ start_proxy() {
if [ "$isolated_workers" = false ]; then
start_proxy 4000 proxy.log
proxy_pid="$launched_pid"
.venv/bin/python .circleci/scripts/wait_integration_services.py
.venv/bin/python .github/scripts/ci/wait_integration_services.py
curl --noproxy '*' -sSf -X POST "$INTEGRATION_PROXY_URL/config/update" \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
-d '{"router_settings": {"num_retries": 0}}' > "$results/seed-router-settings.json"
@ -205,7 +210,7 @@ if [ "$isolated_workers" = false ]; then
export INTEGRATION_PEER_URL=http://127.0.0.1:4001
start_proxy 4001 peer.log
peer_pid="$launched_pid"
.venv/bin/python .circleci/scripts/wait_integration_services.py
.venv/bin/python .github/scripts/ci/wait_integration_services.py
fi
fi
@ -228,14 +233,13 @@ if [ "$suite" = browser ]; then
E2E_UI_BASE_URL="$E2E_UI_BASE_URL" E2E_UI_ARTIFACT_DIR="$E2E_UI_ARTIFACT_DIR" \
LITELLM_MASTER_KEY="$LITELLM_MASTER_KEY" CI=true \
node tests/e2e/ui/node_modules/@playwright/test/cli.js test --config tests/e2e/ui/integration.config.ts
.venv/bin/python .circleci/scripts/verify_integration_browser.py "$results/browser-results.json"
.venv/bin/python .github/scripts/ci/verify_integration_browser.py "$results/browser-results.json"
exit 0
fi
shard_arguments=()
if [ "${CIRCLE_NODE_TOTAL:-1}" -gt 1 ]; then
.venv/bin/python tests/integration/run.py "$suite" --list \
| circleci tests split --split-by=timings --timings-type=filename > "$results/node-files.txt"
if [ -n "${INTEGRATION_SHARD_FILES:-}" ]; then
cp "$INTEGRATION_SHARD_FILES" "$results/node-files.txt"
shard_arguments=(--shard-files "$results/node-files.txt")
fi
@ -257,10 +261,10 @@ env -i PATH="$PATH" HOME="$HOME" PYTHONPATH="$PYTHONPATH" \
"${shared_services[@]}" "${worker_coverage[@]}" \
INTEGRATION_WORKERS="${INTEGRATION_WORKERS:-1}" \
LITELLM_MODE=PRODUCTION \
LITELLM_LICENSE="${LITELLM_LICENSE:-}" \
"${license_env[@]}" \
INTEGRATION_SEED="$INTEGRATION_SEED" \
INTEGRATION_ORDER_SEED="$INTEGRATION_ORDER_SEED" \
INTEGRATION_CASE_SHARDS="${CIRCLE_NODE_TOTAL:-1}" INTEGRATION_CASE_SHARD_INDEX="${CIRCLE_NODE_INDEX:-0}" \
INTEGRATION_CASE_SHARDS="${INTEGRATION_CASE_SHARDS:-1}" INTEGRATION_CASE_SHARD_INDEX="${INTEGRATION_CASE_SHARD_INDEX:-0}" \
LITELLM_LOCAL_MODEL_COST_MAP=True AWS_EC2_METADATA_DISABLED=true DO_NOT_TRACK=1 \
INTEGRATION_PROXY_DATABASE_URL="$INTEGRATION_PROXY_DATABASE_URL" \
INTEGRATION_PROXY_READ_REPLICA_URL="$INTEGRATION_PROXY_READ_REPLICA_URL" \

View file

@ -31,7 +31,7 @@ def main() -> None:
result: Final = json.loads(Path(sys.argv[1]).read_text())
assert not result.get("errors"), result.get("errors")
expected: Final = json.loads(
(Path(__file__).resolve().parents[2] / "tests/e2e/ui/tests/integrationCritical/expected.json").read_text()
(Path(__file__).resolve().parents[3] / "tests/e2e/ui/tests/integrationCritical/expected.json").read_text()
)
assert expected and result["stats"]["expected"] == len(expected)
assert all(result["stats"][name] == 0 for name in ("unexpected", "flaky", "skipped"))
@ -41,7 +41,7 @@ def main() -> None:
suites: Final = TypeAdapter(list[BrowserSuite]).validate_python(result["suites"], strict=True)
specs: Final = tuple(spec for suite in suites for spec in cases(suite))
repository: Final = Path(__file__).resolve().parents[2]
repository: Final = Path(__file__).resolve().parents[3]
report_root: Final = Path(result["config"]["rootDir"])
assert report_root.is_absolute(), "Playwright rootDir must be explicit"
observed: Final = tuple(

View file

@ -16,7 +16,7 @@ run_full() {
}
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
classify="${here}/../../.circleci/scripts/classify_changes.sh"
classify="${here}/ci/classify_changes.sh"
[ -n "${PR_NUMBER:-}" ] || run_full "not a pull_request event"
[ -n "${REPO:-}" ] || run_full "no repository in the environment"

View file

@ -0,0 +1,13 @@
#!/usr/bin/env bash
set -euo pipefail
root=$(git rev-parse --show-toplevel)
toolchain=$(sed -n 's/^[[:space:]]*channel[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p' "$root/litellm/rust/rust-toolchain.toml")
test -n "$toolchain"
rustup toolchain install "$toolchain" --profile minimal --component rustfmt --component clippy --no-self-update
{
echo "RUSTUP_TOOLCHAIN=$toolchain"
echo "CARGO_NET_RETRY=5"
echo "CARGO_HTTP_MULTIPLEXING=false"
echo "SQLX_OFFLINE=${SQLX_OFFLINE:-true}"
} >> "${GITHUB_ENV:?GITHUB_ENV is required}"

View file

@ -7,6 +7,8 @@ import zipfile
from pathlib import Path
from typing import Final
COST_MAP: Final = Path(__file__).resolve().parents[2] / "model_prices_and_context_window.json"
CHILD_SCRIPT: Final = """
from importlib.util import module_from_spec, spec_from_file_location
from pathlib import Path
@ -34,6 +36,9 @@ else:
after = module.gil_stats()
if not isinstance(after.get("releases"), int):
raise AssertionError(f"native module unusable after panic: {after!r}")
if module.bundled_model_cost_map() != Path(sys.argv[2]).read_bytes():
raise AssertionError("the wheel embeds a different model_prices_and_context_window.json than this checkout")
"""
@ -55,7 +60,9 @@ def main() -> int:
native_path: Final = Path(temporary_directory) / Path(native_members[0].filename).name
native_path.write_bytes(archive.read(native_members[0]))
result: Final = subprocess.run((sys.executable, "-c", CHILD_SCRIPT, str(native_path)), check=False)
result: Final = subprocess.run(
(sys.executable, "-c", CHILD_SCRIPT, str(native_path), str(COST_MAP)), check=False
)
if result.returncode != 0:
sys.stderr.write(f"native wheel smoke test exited with status {result.returncode}\n")

View file

@ -128,6 +128,20 @@ def main(
member.filename
for member in wheel_members
if member.filename.endswith((".pdb", ".dwp", ".rlib", ".rmeta", "Cargo.toml", "Cargo.lock"))
or member.filename.startswith(
(
"litellm/deploy/",
"litellm/ui/",
"litellm/vscode-extension/",
"litellm/enterprise/",
"litellm/proxy-extras/",
"litellm/rust/",
"litellm/packaging/",
"litellm/scripts/",
"litellm/runtime/",
"litellm/schema.prisma",
)
)
or any(part.endswith(".dSYM") for part in PurePosixPath(member.filename).parts)
)
native_member: Final = native_members[0]
@ -214,7 +228,7 @@ def main(
native_module: Final = load_native_module(native_path)
native_module_loads: Final = native_module is not None
panic_test_hook_absent: Final = native_module is not None and not hasattr(native_module, "_panic_for_test")
native_size_limit: Final = 48_000_000
native_size_limit: Final = 55_000_000
native_size_within_limit: Final = native_member.file_size <= native_size_limit
validations: Final = (
(f"Python tag is {EXPECTED_PYTHON_TAG}", python_tag == EXPECTED_PYTHON_TAG),

22
.github/semgrep/rules/README.md vendored Normal file
View file

@ -0,0 +1,22 @@
# Custom Semgrep rules for LiteLLM
Add custom rule YAML files here. Semgrep loads all `.yml`/`.yaml` files under this directory.
**Run only custom rules (CI / fail on findings):**
```bash
semgrep scan --config .github/semgrep/rules . --error
```
**Run with registry + custom rules:**
```bash
semgrep scan --config auto --config .github/semgrep/rules .
```
**Layout:**
- `python/` – Python-specific rules (security, patterns)
- Add more subdirs as needed (e.g. `generic/` for language-agnostic rules)
See [Semgrep rule syntax](https://semgrep.dev/docs/writing-rules/rule-syntax/).

View file

@ -16,8 +16,8 @@ on:
pull_request:
paths:
- .github/workflows/auto-close-duplicates.yml
- scripts/auto-close-duplicates.ts
- scripts/auto-close-duplicates.test.ts
- litellm/scripts/auto-close-duplicates.ts
- litellm/scripts/auto-close-duplicates.test.ts
permissions: {}
@ -40,7 +40,7 @@ jobs:
bun-version: "1.4.0"
- name: Test the sweep
run: bun test scripts/auto-close-duplicates.test.ts
run: bun test litellm/scripts/auto-close-duplicates.test.ts
sweep:
if: github.event_name != 'pull_request' && github.repository == 'BerriAI/litellm'
@ -62,7 +62,7 @@ jobs:
bun-version: "1.4.0"
- name: Close unanswered duplicates, reopen ones the reporter answered
run: bun run scripts/auto-close-duplicates.ts
run: bun run litellm/scripts/auto-close-duplicates.ts
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DRY_RUN: ${{ inputs.dry_run == true }}

View file

@ -3,9 +3,9 @@ name: Check Schema Sync
on:
pull_request:
paths:
- 'schema.prisma'
- 'litellm/schema.prisma'
- 'litellm/proxy/schema.prisma'
- 'litellm-proxy-extras/litellm_proxy_extras/schema.prisma'
- 'litellm/proxy-extras/litellm_proxy_extras/schema.prisma'
permissions:
contents: read
@ -16,7 +16,7 @@ concurrency:
jobs:
check-sync:
name: Verify schema.prisma copies match root
name: Verify litellm/schema.prisma copies match the canonical schema
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
@ -27,7 +27,7 @@ jobs:
- name: Reject symlinked schema files
run: |
for f in schema.prisma litellm/proxy/schema.prisma litellm-proxy-extras/litellm_proxy_extras/schema.prisma; do
for f in litellm/schema.prisma litellm/proxy/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma; do
if [ -L "$f" ]; then
echo "::error file=$f::$f is a symlink, which is not allowed"
exit 1
@ -38,24 +38,24 @@ jobs:
run: |
EXIT=0
diff schema.prisma litellm/proxy/schema.prisma || {
echo "::error file=litellm/proxy/schema.prisma::litellm/proxy/schema.prisma differs from root schema.prisma"
diff litellm/schema.prisma litellm/proxy/schema.prisma || {
echo "::error file=litellm/proxy/schema.prisma::litellm/proxy/schema.prisma differs from root litellm/schema.prisma"
EXIT=1
}
diff schema.prisma litellm-proxy-extras/litellm_proxy_extras/schema.prisma || {
echo "::error file=litellm-proxy-extras/litellm_proxy_extras/schema.prisma::litellm-proxy-extras/litellm_proxy_extras/schema.prisma differs from root schema.prisma"
diff litellm/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma || {
echo "::error file=litellm/proxy-extras/litellm_proxy_extras/schema.prisma::litellm/proxy-extras/litellm_proxy_extras/schema.prisma differs from root litellm/schema.prisma"
EXIT=1
}
if [ "$EXIT" -ne 0 ]; then
echo ""
echo "Schema files are out of sync."
echo "The root schema.prisma is the source of truth."
echo "The root litellm/schema.prisma is the source of truth."
echo ""
echo "To fix, run from the repo root:"
echo " cp schema.prisma litellm/proxy/schema.prisma"
echo " cp schema.prisma litellm-proxy-extras/litellm_proxy_extras/schema.prisma"
echo " cp litellm/schema.prisma litellm/proxy/schema.prisma"
echo " cp litellm/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma"
exit 1
fi

View file

@ -4,7 +4,7 @@ on:
pull_request:
paths:
- tests/e2e/claude_code/cron_vm/**
- tests/e2e/claude_code/pr_gate_version_resolver.py
- tests/e2e/cli_harness/npm_version_resolver.py
- .github/workflows/compat-matrix-image.yml
workflow_dispatch:
@ -26,17 +26,24 @@ jobs:
with:
persist-credentials: false
- name: Build the Render cron image
run: docker build -f tests/e2e/claude_code/cron_vm/Dockerfile -t compat-matrix:${{ github.sha }} tests/e2e
- name: Resolve and install the Claude Code CLI as the cron user
- name: Install the pinned uv the cron image ships
env:
UV_VERSION: 0.10.9
UV_SHA256: 20d79708222611fa540b5c9ed84f352bcd3937740e51aacc0f8b15b271c57594
run: |
docker run --rm compat-matrix:${{ github.sha }} bash -c '
set -euo pipefail
whoami
gh --version
uv --version
version="$(uv run --no-project --python 3.12 python /opt/litellm/tests/e2e/claude_code/pr_gate_version_resolver.py)"
/opt/litellm/tests/e2e/claude_code/cron_vm/install_claude_code.sh "${version}" /tmp/claude-cli
/tmp/claude-cli/claude --version
'
set -euo pipefail
curl -fsSLo "$RUNNER_TEMP/uv.tar.gz" "https://github.com/astral-sh/uv/releases/download/${UV_VERSION}/uv-x86_64-unknown-linux-gnu.tar.gz"
echo "${UV_SHA256} $RUNNER_TEMP/uv.tar.gz" | sha256sum -c -
mkdir -p "$RUNNER_TEMP/bin"
tar -xzf "$RUNNER_TEMP/uv.tar.gz" -C "$RUNNER_TEMP/bin" --strip-components=1 uv-x86_64-unknown-linux-gnu/uv
echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH"
- name: Resolve and install the Claude Code CLI as an unprivileged user
run: |
set -euo pipefail
whoami
gh --version
uv --version
version="$(uv run --no-project --python 3.12 python tests/e2e/cli_harness/npm_version_resolver.py @anthropic-ai/claude-code)"
tests/e2e/claude_code/cron_vm/install_claude_code.sh "${version}" "$RUNNER_TEMP/claude-cli"
"$RUNNER_TEMP/claude-cli/claude" --version

View file

@ -3,7 +3,7 @@ name: Conventional PR Title
# Squash-merge replaces the merge commit subject with the PR title, so
# enforcing Conventional Commits at the PR-title level is what actually gates
# the commits that land on the default branch. The local commit-msg hook
# (.githooks/commit-msg) is a best-effort assist; this workflow is the gate.
# (litellm/scripts/githooks/commit-msg) is a best-effort assist; this workflow is the gate.
#
# See https://www.conventionalcommits.org/en/v1.0.0/
@ -28,7 +28,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
# Must mirror the type list in .githooks/commit-msg.
# Must mirror the type list in litellm/scripts/githooks/commit-msg.
types: |
feat
fix

View file

@ -8,6 +8,7 @@ on:
permissions:
contents: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
@ -21,16 +22,21 @@ jobs:
with:
ref: ${{ github.event.pull_request.base.sha }}
persist-credentials: false
- name: Fetch the pull request head and its merge base
- name: Check out the base branch tip and fetch the pull request head and its merge base
id: revisions
env:
GH_TOKEN: ${{ github.token }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_NUMBER: ${{ github.event.pull_request.number }}
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
merge_base="$(gh api "repos/${GITHUB_REPOSITORY}/compare/${BASE_SHA}...${HEAD_SHA}" --jq '.merge_base_commit.sha')"
git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA"
base_ref="$(gh api "repos/${GITHUB_REPOSITORY}/branches/${EVENT_BASE_REF}" --jq .name)" || base_ref="$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq .base.ref)"
target="$(gh api "repos/${GITHUB_REPOSITORY}/branches/${base_ref}" --jq '.commit.sha')"
merge_base="$(gh api "repos/${GITHUB_REPOSITORY}/compare/${target}...${HEAD_SHA}" --jq '.merge_base_commit.sha')"
git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA" "$target"
git checkout -q --detach "$target"
echo "merge_base=$merge_base" >> "$GITHUB_OUTPUT"
echo "target=$target" >> "$GITHUB_OUTPUT"
- name: Set up uv
uses: ./.github/actions/setup-uv-with-retries
with:
@ -38,7 +44,38 @@ jobs:
- name: Run the guard
env:
MERGE_BASE: ${{ steps.revisions.outputs.merge_base }}
COST_MAP_GUARD_TARGET: ${{ steps.revisions.outputs.target }}
COST_MAP_GUARD_AUTHOR_TYPE: ${{ github.event.pull_request.user.type }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
HEAD_REF: ${{ github.event.pull_request.head.ref }}
run: |
uv run --no-project python ci_cd/cost_map_guard.py --base "$MERGE_BASE" --head "$HEAD_SHA" --head-ref "$HEAD_REF"
guard=litellm/scripts/ci_cd/cost_map_guard.py
# TODO: Drop the legacy path after this lands on main.
if [ ! -f "$guard" ]; then
guard=scripts/ci_cd/cost_map_guard.py
fi
if [ ! -f "$guard" ]; then
guard=ci_cd/cost_map_guard.py
fi
uv run --no-project python "$guard" --base "$MERGE_BASE" --head "$HEAD_SHA" --head-ref "$HEAD_REF"
- name: Check whether the pull request changes the cost map
id: changed
env:
MERGE_BASE: ${{ steps.revisions.outputs.merge_base }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
if git diff --quiet "$MERGE_BASE" "$HEAD_SHA" -- model_prices_and_context_window.json; then
echo "cost_map=false" >> "$GITHUB_OUTPUT"
else
echo "cost_map=true" >> "$GITHUB_OUTPUT"
fi
- if: steps.changed.outputs.cost_map == 'true'
run: rustup toolchain install --no-self-update
- name: Load the cost map through model-catalog
if: steps.changed.outputs.cost_map == 'true'
env:
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
git show "${HEAD_SHA}:model_prices_and_context_window.json" > "$RUNNER_TEMP/model_prices_and_context_window.json"
cargo run --locked --quiet --manifest-path litellm/rust/Cargo.toml -p litellm-model-catalog --bin check-registry \
-- "$RUNNER_TEMP/model_prices_and_context_window.json"

View file

@ -0,0 +1,91 @@
name: Cost map pins recheck
on:
push:
branches:
- main
- "litellm_**"
paths:
- model_prices_and_context_window.json
- litellm/model_prices_and_context_window_backup.json
- litellm/scripts/ci_cd/cost_map_pins.json
permissions: {}
jobs:
rerun-cost-map-guard:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
actions: write
contents: read
pull-requests: read
steps:
- name: Re-run the cost map guard on open pull requests the push could break
env:
GH_TOKEN: ${{ github.token }}
BASE_REF: ${{ github.ref_name }}
BEFORE_SHA: ${{ github.event.before }}
AFTER_SHA: ${{ github.event.after }}
FORCED: ${{ github.event.forced }}
COST_MAP_PATHS: '["model_prices_and_context_window.json", "litellm/model_prices_and_context_window_backup.json", "litellm/scripts/ci_cd/cost_map_pins.json"]'
PINS_PATHS: '["litellm/scripts/ci_cd/cost_map_pins.json"]'
run: |
if [[ "$BEFORE_SHA" =~ ^0+$ || "$FORCED" == true ]] \
|| ! gh api "repos/${GITHUB_REPOSITORY}/compare/${BEFORE_SHA}...${AFTER_SHA}?per_page=1" \
--jq '(env.PINS_PATHS | fromjson) as $pins | (.files | length) < 300 and all(.files[]; [.filename, .previous_filename] | any(IN($pins[])) | not)' \
| grep -qx true; then
export RECHECK_PATHS="$COST_MAP_PATHS"
else
export RECHECK_PATHS="$PINS_PATHS"
fi
rerun_cutoff="$(date -u -d '30 days ago' +%Y-%m-%dT%H:%M:%SZ)"
candidates="$(gh api graphql --paginate -F owner="${GITHUB_REPOSITORY_OWNER}" -F name="${GITHUB_REPOSITORY#*/}" -F base="$BASE_REF" \
-f query='query($owner: String!, $name: String!, $base: String!, $endCursor: String) {
repository(owner: $owner, name: $name) {
pullRequests(states: OPEN, baseRefName: $base, first: 25, after: $endCursor) {
nodes { number headRefOid changedFiles files(first: 100) { nodes { path } } }
pageInfo { hasNextPage endCursor }
}
}
}' \
--jq '(env.RECHECK_PATHS | fromjson) as $paths | .data.repository.pullRequests.nodes[]
| {number, headRefOid, listed: any(.files.nodes[].path; IN($paths[])), unlisted: (.changedFiles > 100)}
| select(.listed or .unlisted) | "\(.number) \(.headRefOid) \(.listed)"')"
failed=0
declare -A rerun_heads
while read -r number head_sha listed; do
[ -n "$number" ] || continue
if [ "$listed" = false ]; then
if ! changed="$(gh api --paginate "repos/${GITHUB_REPOSITORY}/pulls/${number}/files?per_page=100" \
--jq '(env.RECHECK_PATHS | fromjson) as $paths | .[].filename | select(IN($paths[]))')"; then
echo "::error::could not list the files of #$number"
failed=1
continue
fi
[ -n "$changed" ] || continue
fi
[ -z "${rerun_heads[$head_sha]:-}" ] || continue
rerun_heads[$head_sha]=1
if ! runs="$(gh run list --repo "$GITHUB_REPOSITORY" --workflow cost-map-guard.yml --commit "$head_sha" --limit 100 --json databaseId,createdAt --jq '.[] | "\(.databaseId) \(.createdAt)"')"; then
echo "::error::could not list the cost map guard runs of #$number"
failed=1
continue
fi
if [ -z "$runs" ]; then
echo "::warning::no cost map guard run found for #$number"
continue
fi
while read -r run_id created_at; do
if [[ "$created_at" < "$rerun_cutoff" ]]; then
echo "::warning::cost map guard run $run_id for #$number is over 30 days old, so GitHub cannot re-run it; #$number gets the pin checks on its next push"
continue
fi
gh run watch "$run_id" --repo "$GITHUB_REPOSITORY" > /dev/null || true
if ! gh run rerun "$run_id" --repo "$GITHUB_REPOSITORY"; then
echo "::error::could not re-run cost map guard run $run_id for #$number"
failed=1
fi
done <<< "$runs"
done <<< "$candidates"
exit "$failed"

View file

@ -2,7 +2,7 @@ name: Create RC Branch
on:
schedule:
- cron: "0 3 * * 5"
- cron: "0 3 * * 1"
timezone: "America/Los_Angeles"
workflow_dispatch:

View file

@ -1,30 +0,0 @@
name: Deploy main to gateway-dev
on:
push:
branches: [main]
permissions: {}
concurrency:
group: deploy-dev-dispatch
cancel-in-progress: true
jobs:
dispatch:
if: github.repository == 'BerriAI/litellm'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/create-github-app-token@fee1f7d63c2ff003460e3d139729b119787bc349 # v2.2.2
id: token
with:
app-id: ${{ secrets.LITELLM_OPS_DISPATCH_APP_ID }}
private-key: ${{ secrets.LITELLM_OPS_DISPATCH_APP_PRIVATE_KEY }}
owner: BerriAI
repositories: litellm-ops
permission-actions: write
- name: Dispatch the litellm-ops dev deploy
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
run: gh api --method POST repos/BerriAI/litellm-ops/actions/workflows/deploy-dev.yml/dispatches -f ref=main

View file

@ -13,9 +13,9 @@ on:
- .github/workflows/duplicate_issue_check.yml
- .github/prompts/duplicate-issue-check.md
- .github/prompts/duplicate-issue-check.schema.json
- scripts/flag-duplicate-issue.ts
- scripts/flag-duplicate-issue.test.ts
- scripts/auto-close-duplicates.ts
- litellm/scripts/flag-duplicate-issue.ts
- litellm/scripts/flag-duplicate-issue.test.ts
- litellm/scripts/auto-close-duplicates.ts
permissions: {}
@ -38,7 +38,7 @@ jobs:
bun-version: "1.4.0"
- name: Test the flag step
run: bun test scripts/flag-duplicate-issue.test.ts
run: bun test litellm/scripts/flag-duplicate-issue.test.ts
classify:
if: github.event_name != 'pull_request' && github.repository == 'BerriAI/litellm'
@ -125,7 +125,7 @@ jobs:
- name: Checkout scripts
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
sparse-checkout: scripts
sparse-checkout: litellm/scripts
persist-credentials: false
- name: Setup Bun
@ -134,7 +134,7 @@ jobs:
bun-version: "1.4.0"
- name: Comment and label
run: bun run scripts/flag-duplicate-issue.ts
run: bun run litellm/scripts/flag-duplicate-issue.ts
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERDICT: ${{ needs.classify.outputs.verdict }}

View file

@ -8,8 +8,8 @@ on:
paths:
- "uv.lock"
- "pyproject.toml"
- "litellm-proxy-extras/pyproject.toml"
- "enterprise/pyproject.toml"
- "litellm/proxy-extras/pyproject.toml"
- "litellm/enterprise/pyproject.toml"
permissions: {}
@ -130,8 +130,12 @@ jobs:
}
check_deps base/pyproject.toml pr/pyproject.toml "pyproject.toml" || had_error=1
check_deps base/litellm-proxy-extras/pyproject.toml pr/litellm-proxy-extras/pyproject.toml "litellm-proxy-extras/pyproject.toml" || had_error=1
check_deps base/enterprise/pyproject.toml pr/enterprise/pyproject.toml "enterprise/pyproject.toml" || had_error=1
base_extras=base/litellm/proxy-extras/pyproject.toml
[ -f "$base_extras" ] || base_extras=base/litellm-proxy-extras/pyproject.toml
base_enterprise=base/litellm/enterprise/pyproject.toml
[ -f "$base_enterprise" ] || base_enterprise=base/enterprise/pyproject.toml
check_deps "$base_extras" pr/litellm/proxy-extras/pyproject.toml "litellm/proxy-extras/pyproject.toml" || had_error=1
check_deps "$base_enterprise" pr/litellm/enterprise/pyproject.toml "litellm/enterprise/pyproject.toml" || had_error=1
if [ "$had_error" -ne 0 ]; then
echo ""

View file

@ -7,33 +7,30 @@ on:
- litellm_oss_branch
- "litellm_**"
paths:
- Dockerfile
- docker/Dockerfile.non_root
- docker/Dockerfile.database
- migrations/Dockerfile
- migrations/run.py
- gateway/Dockerfile
- gateway/main.py
- gateway/routes/allowlist.py
- backend/Dockerfile
- backend/main.py
- litellm-rust/**
- litellm/deploy/docker/Dockerfile
- litellm/deploy/docker/Dockerfile.dockerignore
- litellm/deploy/docker/docker-entrypoint.sh
- litellm/runtime/migrations/run.py
- litellm/runtime/gateway/main.py
- litellm/runtime/gateway/launch.py
- litellm/runtime/gateway/routes/allowlist.py
- litellm/cli/**
- .github/packaging/**
- litellm/runtime/backend/main.py
- litellm/runtime/backend/routes/allowlist.py
- litellm/rust/**
- litellm/proxy/lens/**
- docker/component_entrypoint.sh
- docker/entrypoint.sh
- litellm/proxy/prisma_migration.py
- litellm/proxy/admin_mcp.py
- litellm/proxy/proxy_server.py
- backend/routes/allowlist.py
- pyproject.toml
- litellm-proxy-extras/**
- litellm/proxy-extras/**
- tests/integration/database/image/**
- uv.lock
- ui/litellm-dashboard/package-lock.json
- ui/Dockerfile
- ui/nginx.conf
- litellm/ui/litellm-dashboard/package-lock.json
- litellm/ui/nginx.conf
- .github/workflows/image-scan.yml
- .grype.yaml
- .github/grype.yaml
schedule:
- cron: "41 6 * * *"
workflow_dispatch:
@ -44,14 +41,18 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env:
IMAGE: litellm-image-scan:${{ github.sha }}
ADMIN_MCP_SCHEMA_IMAGE: litellm-admin-mcp-schema:${{ github.sha }}
jobs:
image-scan:
name: image-scan
build:
name: build
runs-on: ubuntu-latest
if: >-
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
timeout-minutes: 30
timeout-minutes: 60
permissions:
contents: read
steps:
@ -59,7 +60,129 @@ jobs:
with:
persist-credentials: false
- name: Build the image
run: docker build -f litellm/deploy/docker/Dockerfile --build-arg MATURIN_PEP517_ARGS="--profile ci" --build-arg LITELLM_RELEASE_TAG=v0.0.0-lens-ci -t "$IMAGE" .
- name: Save the image for the verify matrix
run: docker save "$IMAGE" | zstd -T0 -3 -o "$RUNNER_TEMP/litellm-image.tar.zst"
- uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
with:
name: litellm-image
path: ${{ runner.temp }}/litellm-image.tar.zst
retention-days: 1
compression-level: 0
- name: Build the Admin MCP schema image
run: docker build -f litellm/deploy/docker/Dockerfile --build-arg MATURIN_PEP517_ARGS="--profile ci" --build-arg LITELLM_RELEASE_TAG=v0.0.0-lens-ci --target builder -t "$ADMIN_MCP_SCHEMA_IMAGE" .
- name: Save the Admin MCP schema image for the verify matrix
run: docker save "$ADMIN_MCP_SCHEMA_IMAGE" | zstd -T0 -3 -o "$RUNNER_TEMP/litellm-admin-mcp-schema.tar.zst"
- uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
with:
name: litellm-admin-mcp-schema
path: ${{ runner.temp }}/litellm-admin-mcp-schema.tar.zst
retention-days: 1
compression-level: 0
verify:
name: verify (${{ matrix.check }})
needs: build
runs-on: ubuntu-latest
timeout-minutes: 45
permissions:
contents: read
strategy:
fail-fast: false
matrix:
check: [scan, migrations, gateway, backend, ui, collector]
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: litellm-image
path: ${{ runner.temp }}
- name: Load the image
run: zstd -d --stdout "$RUNNER_TEMP/litellm-image.tar.zst" | docker load
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
if: matrix.check == 'migrations' || matrix.check == 'backend'
with:
name: litellm-admin-mcp-schema
path: ${{ runner.temp }}
- name: Load the Admin MCP schema image
if: matrix.check == 'migrations' || matrix.check == 'backend'
run: zstd -d --stdout "$RUNNER_TEMP/litellm-admin-mcp-schema.tar.zst" | docker load
- name: Set up Python
if: matrix.check != 'scan'
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Install pytest
if: matrix.check != 'scan'
run: python -m pip install "pytest==9.0.3"
# The prisma bake must migrate a fresh DB with no egress as an arbitrary
# non-root uid (OpenShift restricted-v2 / air-gapped / readOnlyFilesystem).
# `docker run` as the default uid with network hides a broken bake because
# the migration entrypoint exits 0 even when it applied nothing; asserting
# the schema was created is what catches it.
- name: Verify offline migration as a non-root uid
if: matrix.check == 'migrations'
env:
LITELLM_IMAGE: ${{ env.IMAGE }}
LITELLM_ADMIN_MCP_SCHEMA_IMAGE: ${{ env.ADMIN_MCP_SCHEMA_IMAGE }}
run: |
python -m pytest --noconftest -v \
tests/integration/database/image/test_offline_image_migration.py \
tests/integration/database/image/test_image_bedrock_realtime_extra.py \
tests/integration/database/image/test_image_admin_mcp.py
- name: Verify the gateway component serves offline as a non-root uid
if: matrix.check == 'gateway'
env:
LITELLM_IMAGE: ${{ env.IMAGE }}
LITELLM_COMPONENT: gateway
LITELLM_IMAGE_COMPONENT: gateway
run: |
python -m pytest --noconftest -v \
tests/integration/database/image/test_component_image_serves_offline.py \
tests/integration/database/image/test_image_admin_mcp.py
- name: Verify the backend component serves offline as a non-root uid
if: matrix.check == 'backend'
env:
LITELLM_IMAGE: ${{ env.IMAGE }}
LITELLM_COMPONENT: backend
LITELLM_IMAGE_COMPONENT: backend
LITELLM_ADMIN_MCP_SCHEMA_IMAGE: ${{ env.ADMIN_MCP_SCHEMA_IMAGE }}
run: |
python -m pytest --noconftest -v \
tests/integration/database/image/test_component_image_serves_offline.py \
tests/integration/database/image/test_image_admin_mcp.py
- name: Verify the ui component serves as an arbitrary uid with a read-only root fs
if: matrix.check == 'ui'
env:
LITELLM_IMAGE: ${{ env.IMAGE }}
run: python -m pytest --noconftest -v tests/integration/database/image/test_ui_image_serves_offline.py
- name: Verify the collector creates its socket as the image user
if: matrix.check == 'collector'
env:
LITELLM_IMAGE: ${{ env.IMAGE }}
run: python -m pytest --noconftest -v tests/integration/database/image/test_collector_image_serves_socket.py
- name: Download Grype v0.114.0
if: matrix.check == 'scan'
run: |
curl -fsSL --retry 3 -o "$RUNNER_TEMP/grype.tar.gz" \
https://github.com/anchore/grype/releases/download/v0.114.0/grype_0.114.0_linux_amd64.tar.gz
@ -67,33 +190,6 @@ jobs:
tar xzf "$RUNNER_TEMP/grype.tar.gz" -C "$RUNNER_TEMP" grype
chmod +x "$RUNNER_TEMP/grype"
# Dockerfile.non_root is the rootless variant we ship. The other
# Dockerfiles share the same wolfi base and apk set, so OS-layer coverage
# is the same; matrix-scan if those variants ever diverge.
- name: Build runtime image
run: docker build -f docker/Dockerfile.non_root -t litellm-image-scan:${{ github.sha }} .
- name: Tag the cached builder for Admin MCP schema setup
run: docker build --target builder -f docker/Dockerfile.non_root -t litellm-admin-mcp-schema:${{ github.sha }} .
# The prisma bake must migrate a fresh DB with no egress as an arbitrary
# non-root uid (OpenShift restricted-v2 / air-gapped / readOnlyRootFilesystem).
# `docker run` as the default uid with network hides a broken bake because
# the migration entrypoint exits 0 even when it applied nothing; asserting
# the schema was created is what catches it.
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Verify offline migration as a non-root uid
env:
LITELLM_IMAGE: litellm-image-scan:${{ github.sha }}
LITELLM_ADMIN_MCP_SCHEMA_IMAGE: litellm-admin-mcp-schema:${{ github.sha }}
run: |
python -m pip install "pytest==9.0.3"
python -m pytest --noconftest tests/integration/database/image/test_offline_image_migration.py tests/integration/database/image/test_image_bedrock_realtime_extra.py tests/integration/database/image/test_image_admin_mcp.py -v
# Scans the whole shipped artifact: OS/apk plus every language package
# baked into the image, including ones no lockfile declares (e.g. prisma's
# vendored node engine) that osv-scan cannot see. osv-scan stays the fast
@ -101,175 +197,12 @@ jobs:
# free OSS, run as a pinned, checksum-verified binary; no GitHub Action
# dependency and no vendor SaaS callout.
- name: Scan image for fixable HIGH/CRITICAL CVEs
if: matrix.check == 'scan'
env:
GRYPE_MATCH_PYTHON_USING_CPES: "true"
run: |
"$RUNNER_TEMP/grype" litellm-image-scan:${{ github.sha }} \
--config .grype.yaml \
"$RUNNER_TEMP/grype" "$IMAGE" \
--config .github/grype.yaml \
--only-fixed \
--fail-on high \
--output table
runtime-image:
name: runtime-image (${{ matrix.dockerfile }})
runs-on: ubuntu-latest
if: >-
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
timeout-minutes: 45
permissions:
contents: read
strategy:
fail-fast: false
matrix:
dockerfile: [Dockerfile, docker/Dockerfile.database]
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- name: Build runtime image
run: docker build --build-arg LITELLM_RELEASE_TAG=v0.0.0-lens-ci -f "${{ matrix.dockerfile }}" -t litellm-runtime-scan:${{ github.sha }} .
- name: Tag the cached builder for Admin MCP schema setup
run: docker build --target builder -f "${{ matrix.dockerfile }}" -t litellm-admin-mcp-schema:${{ github.sha }} .
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Verify offline migration as a non-root uid
env:
LITELLM_IMAGE: litellm-runtime-scan:${{ github.sha }}
LITELLM_ADMIN_MCP_SCHEMA_IMAGE: litellm-admin-mcp-schema:${{ github.sha }}
run: |
python -m pip install "pytest==9.0.3"
python -m pytest --noconftest tests/integration/database/image/test_offline_image_migration.py tests/integration/database/image/test_image_bedrock_realtime_extra.py tests/integration/database/image/test_image_admin_mcp.py -v
migrations-image:
name: migrations-image
runs-on: ubuntu-latest
if: >-
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
timeout-minutes: 30
permissions:
contents: read
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- name: Build migrations image
run: docker build -f migrations/Dockerfile -t litellm-migrations-scan:${{ github.sha }} .
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Verify offline migration as a non-root uid
env:
LITELLM_IMAGE: litellm-migrations-scan:${{ github.sha }}
LITELLM_MIGRATION_INTERPRETER: python3
LITELLM_MIGRATION_SCRIPT: /app/run.py
run: |
python -m pip install "pytest==9.0.3"
python -m pytest --noconftest tests/integration/database/image/test_offline_image_migration.py -v
gateway-image:
name: gateway-image
runs-on: ubuntu-latest
if: >-
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
timeout-minutes: 30
permissions:
contents: read
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- name: Build gateway image
run: docker build -f gateway/Dockerfile -t litellm-gateway-scan:${{ github.sha }} .
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Verify the gateway serves offline as a non-root uid
env:
LITELLM_IMAGE: litellm-gateway-scan:${{ github.sha }}
LITELLM_COMPONENT_PORT: "4000"
LITELLM_IMAGE_COMPONENT: gateway
run: |
python -m pip install "pytest==9.0.3"
python -m pytest --noconftest tests/integration/database/image/test_component_image_serves_offline.py tests/integration/database/image/test_image_bedrock_realtime_extra.py tests/integration/database/image/test_image_admin_mcp.py -v
ui-image:
name: ui-image
runs-on: ubuntu-latest
if: >-
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
timeout-minutes: 30
permissions:
contents: read
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- name: Build UI image
run: docker build -f ui/Dockerfile -t litellm-ui-scan:${{ github.sha }} .
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Verify the UI serves offline as an arbitrary uid with a read-only root fs
env:
LITELLM_IMAGE: litellm-ui-scan:${{ github.sha }}
run: |
python -m pip install "pytest==9.0.3"
python -m pytest --noconftest tests/integration/database/image/test_ui_image_serves_offline.py -v
backend-image:
name: backend-image
runs-on: ubuntu-latest
if: >-
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
timeout-minutes: 30
permissions:
contents: read
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- name: Build backend image
run: docker build -f backend/Dockerfile -t litellm-backend-scan:${{ github.sha }} .
- name: Tag the cached builder for Admin MCP schema setup
run: docker build --target builder -f backend/Dockerfile -t litellm-admin-mcp-schema:${{ github.sha }} .
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Verify the backend serves offline as a non-root uid
env:
LITELLM_IMAGE: litellm-backend-scan:${{ github.sha }}
LITELLM_ADMIN_MCP_SCHEMA_IMAGE: litellm-admin-mcp-schema:${{ github.sha }}
LITELLM_COMPONENT_PORT: "4001"
LITELLM_IMAGE_COMPONENT: backend
run: |
python -m pip install "pytest==9.0.3"
python -m pytest --noconftest tests/integration/database/image/test_component_image_serves_offline.py tests/integration/database/image/test_image_admin_mcp.py -v

View file

@ -49,9 +49,9 @@ jobs:
id: compliance
run: |
uv run --no-sync pytest tests/unit/interactions/test_openapi_compliance.py \
--interactions-spec-url=https://ai.google.dev/static/api/interactions.openapi.json \
--interactions-spec-url=https://aistudio.google.com/api-reference/interactions.openapi.json \
--tb=short -vv -p no:rerunfailures -p no:pytest-retry \
--allow-hosts=127.0.0.1,::1,localhost,ai.google.dev
--allow-hosts=127.0.0.1,::1,localhost,ai.google.dev,aistudio.google.com
tracking-issue:
name: tracking-issue

View file

@ -16,12 +16,12 @@ on:
- .github/issue-labels.json
- .github/ISSUE_TEMPLATE/bug_report.yml
- .github/ISSUE_TEMPLATE/feature_request.yml
- scripts/classify-issue.ts
- scripts/classify-issue.test.ts
- scripts/label-issue.ts
- scripts/label-issue.test.ts
- scripts/issue-labels.ts
- scripts/auto-close-duplicates.ts
- litellm/scripts/classify-issue.ts
- litellm/scripts/classify-issue.test.ts
- litellm/scripts/label-issue.ts
- litellm/scripts/label-issue.test.ts
- litellm/scripts/issue-labels.ts
- litellm/scripts/auto-close-duplicates.ts
permissions: {}
@ -49,7 +49,7 @@ jobs:
bun-version: "1.4.0"
- name: Test the gate, the validation and the label step
run: bun test scripts/classify-issue.test.ts scripts/label-issue.test.ts
run: bun test litellm/scripts/classify-issue.test.ts litellm/scripts/label-issue.test.ts
classify-issue:
# An edit to a labelled issue is dropped here; the script decides the rest against the live labels
@ -73,7 +73,7 @@ jobs:
with:
sparse-checkout: |
.github
scripts
litellm/scripts
persist-credentials: false
- name: Setup Bun
@ -108,7 +108,7 @@ jobs:
ISSUE_CLASSIFIER_MODEL: ${{ vars.ISSUE_CLASSIFIER_MODEL }}
run: |
set -euo pipefail
bun run scripts/classify-issue.ts > classification.json
bun run litellm/scripts/classify-issue.ts > classification.json
{
echo 'verdict<<CLASSIFICATION'
cat classification.json
@ -143,7 +143,7 @@ jobs:
with:
sparse-checkout: |
.github
scripts
litellm/scripts
persist-credentials: false
- name: Setup Bun
@ -153,7 +153,7 @@ jobs:
bun-version: "1.4.0"
- name: Replace the labels in each namespace
run: bun run scripts/label-issue.ts
run: bun run litellm/scripts/label-issue.ts
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERDICT: ${{ needs.classify-issue.outputs.verdict }}

View file

@ -15,9 +15,9 @@ on:
pull_request:
paths:
- .github/workflows/issue_fixed_comment.yml
- scripts/comment-fixed-issue.ts
- scripts/comment-fixed-issue.test.ts
- scripts/auto-close-duplicates.ts
- litellm/scripts/comment-fixed-issue.ts
- litellm/scripts/comment-fixed-issue.test.ts
- litellm/scripts/auto-close-duplicates.ts
permissions: {}
@ -44,7 +44,7 @@ jobs:
bun-version: "1.4.0"
- name: Test the closer lookup, the release placement, the comment and the superseded pull request close
run: bun test scripts/comment-fixed-issue.test.ts
run: bun test litellm/scripts/comment-fixed-issue.test.ts
comment-fixed-issue:
if: github.event_name != 'pull_request' && github.repository == 'BerriAI/litellm'
@ -58,7 +58,7 @@ jobs:
- name: Checkout scripts
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
sparse-checkout: scripts
sparse-checkout: litellm/scripts
persist-credentials: false
- name: Setup Bun
@ -69,7 +69,7 @@ jobs:
- name: Name the release that carries the fix and close the pull requests it supersedes
shell: bash
run: bun run scripts/comment-fixed-issue.ts | tee -a "${GITHUB_STEP_SUMMARY}"
run: bun run litellm/scripts/comment-fixed-issue.ts | tee -a "${GITHUB_STEP_SUMMARY}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ISSUE_NUMBER: ${{ github.event.issue.number || github.event.inputs.issue_number }}

View file

@ -5,7 +5,7 @@ on:
branches: [main]
paths:
- .github/issue-labels.json
- scripts/sync-issue-labels.ts
- litellm/scripts/sync-issue-labels.ts
workflow_dispatch:
inputs:
dry_run:
@ -16,9 +16,9 @@ on:
paths:
- .github/workflows/issue_label_sync.yml
- .github/issue-labels.json
- scripts/sync-issue-labels.ts
- scripts/sync-issue-labels.test.ts
- scripts/issue-labels.ts
- litellm/scripts/sync-issue-labels.ts
- litellm/scripts/sync-issue-labels.test.ts
- litellm/scripts/issue-labels.ts
permissions: {}
@ -41,7 +41,7 @@ jobs:
bun-version: "1.4.0"
- name: Test the sync
run: bun test scripts/sync-issue-labels.test.ts
run: bun test litellm/scripts/sync-issue-labels.test.ts
sync-issue-labels:
if: github.event_name != 'pull_request' && github.repository == 'BerriAI/litellm'
@ -56,7 +56,7 @@ jobs:
with:
sparse-checkout: |
.github
scripts
litellm/scripts
persist-credentials: false
- name: Setup Bun
@ -66,7 +66,7 @@ jobs:
bun-version: "1.4.0"
- name: Create or recolour every label in .github/issue-labels.json
run: bun run scripts/sync-issue-labels.ts
run: bun run litellm/scripts/sync-issue-labels.ts
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run == true }}

View file

@ -1,152 +0,0 @@
name: Lens installation smoke
on:
workflow_dispatch:
inputs:
lens_image:
description: Verified baseline Lens image at ghcr.io/berriai/lens@sha256
required: true
type: string
lens_upgrade_image:
description: Compatible Lens upgrade image at ghcr.io/berriai/lens@sha256
required: true
type: string
gateway_baseline_ref:
description: Previously qualified gateway source commit, full SHA
required: true
type: string
permissions:
contents: read
concurrency:
group: lens-install-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
build-images:
runs-on: ubuntu-latest-16-cores
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
include:
- component: gateway
dockerfile: gateway/Dockerfile
- component: backend
dockerfile: backend/Dockerfile
- component: ui
dockerfile: ui/Dockerfile
- component: migrations
dockerfile: migrations/Dockerfile
- component: monolith
dockerfile: Dockerfile
- component: gateway
dockerfile: gateway/Dockerfile
baseline: true
- component: backend
dockerfile: backend/Dockerfile
baseline: true
- component: monolith
dockerfile: Dockerfile
baseline: true
env:
COMPONENT: ${{ matrix.component }}
DOCKERFILE: ${{ matrix.dockerfile }}
SOURCE_REF: ${{ matrix.baseline && inputs.gateway_baseline_ref || github.sha }}
IMAGE_TAG: ${{ matrix.baseline && 'v0.0.0-lens-ci-baseline' || 'v0.0.0-lens-ci' }}
CURRENT_REF: ${{ github.sha }}
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
ref: ${{ env.SOURCE_REF }}
- name: Verify the selected source
shell: bash -euo pipefail {0}
run: |
[[ "$SOURCE_REF" =~ ^[0-9a-f]{40}$ ]]
test "$(git rev-parse HEAD)" = "$SOURCE_REF"
if [[ "$IMAGE_TAG" == v0.0.0-lens-ci-baseline ]]; then
test "$SOURCE_REF" != "$CURRENT_REF"
fi
- name: Build the matching release image
run: |
docker build --build-arg "LITELLM_RELEASE_TAG=$IMAGE_TAG" \
-f "$DOCKERFILE" -t "lens-ci-$COMPONENT:$IMAGE_TAG" .
- name: Save the matching release image
run: |
docker save "lens-ci-$COMPONENT:$IMAGE_TAG" \
| gzip -1 > "$RUNNER_TEMP/lens-install-$COMPONENT-$IMAGE_TAG.tar.gz"
- uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
with:
name: lens-install-${{ matrix.component }}-${{ env.IMAGE_TAG }}-${{ github.sha }}
path: ${{ runner.temp }}/lens-install-${{ matrix.component }}-${{ env.IMAGE_TAG }}.tar.gz
compression-level: 0
retention-days: 3
if-no-files-found: error
overwrite: true
helm-install:
needs: build-images
runs-on: ubuntu-latest-16-cores
timeout-minutes: 25
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4.2.1
with:
pattern: lens-install-*-${{ github.sha }}
merge-multiple: true
path: ${{ runner.temp }}/lens-install-images
- name: Load the matching release images
run: |
for archive in "$RUNNER_TEMP"/lens-install-images/lens-install-*.tar.gz; do
gzip -dc "$archive" | docker load
rm "$archive"
done
- uses: sigstore/cosign-installer@3454372f43399081ed03b604cb2d021dabca52bb # v3.8.2
with:
cosign-release: v2.4.3
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ vars.LENS_READ_USER || github.actor }}
password: ${{ secrets.LENS_READ_TOKEN }}
- name: Verify independent Lens images before loading them
env:
LENS_IMAGE: ${{ inputs.lens_image }}
LENS_UPGRADE_IMAGE: ${{ inputs.lens_upgrade_image }}
LENS_PUBLIC_KEY: ${{ vars.LENS_COSIGN_PUBLIC_KEY }}
LENS_PUBLIC_KEY_SHA256: ${{ vars.LENS_COSIGN_PUBLIC_KEY_SHA256 }}
shell: bash -euo pipefail {0}
run: |
[[ "$LENS_PUBLIC_KEY_SHA256" =~ ^[0-9a-f]{64}$ ]]
printf '%s' "$LENS_PUBLIC_KEY" > "$RUNNER_TEMP/lens.pub"
echo "$LENS_PUBLIC_KEY_SHA256 $RUNNER_TEMP/lens.pub" | sha256sum --check
for image in "$LENS_IMAGE" "$LENS_UPGRADE_IMAGE"; do
[[ "$image" =~ ^ghcr.io/berriai/lens@sha256:[0-9a-f]{64}$ ]]
cosign verify --key "$RUNNER_TEMP/lens.pub" "$image" > /dev/null
cosign verify-attestation --key "$RUNNER_TEMP/lens.pub" --type spdxjson "$image" > /dev/null
docker pull "$image"
done
docker tag "$LENS_IMAGE" lens-ci-worker:baseline
docker tag "$LENS_UPGRADE_IMAGE" lens-ci-worker:upgrade
- name: Install pinned Kubernetes test tools
run: |
curl --fail --location --output "$RUNNER_TEMP/kind" \
https://kind.sigs.k8s.io/dl/v0.27.0/kind-linux-amd64
echo "a6875aaea358acf0ac07786b1a6755d08fd640f4c79b7a2e46681cc13f49a04b $RUNNER_TEMP/kind" | sha256sum --check
chmod +x "$RUNNER_TEMP/kind"
curl --fail --location --output "$RUNNER_TEMP/kubectl" \
https://dl.k8s.io/release/v1.32.2/bin/linux/amd64/kubectl
echo "4f6a959dcc5b702135f8354cc7109b542a2933c46b808b248a214c1f69f817ea $RUNNER_TEMP/kubectl" | sha256sum --check
chmod +x "$RUNNER_TEMP/kubectl"
curl --fail --location --output "$RUNNER_TEMP/helm.tar.gz" \
https://get.helm.sh/helm-v3.19.0-linux-amd64.tar.gz
echo "a7f81ce08007091b86d8bd696eb4d86b8d0f2e1b9f6c714be62f82f96a594496 $RUNNER_TEMP/helm.tar.gz" | sha256sum --check
tar -xzf "$RUNNER_TEMP/helm.tar.gz" -C "$RUNNER_TEMP"
echo "$RUNNER_TEMP" >> "$GITHUB_PATH"
echo "$RUNNER_TEMP/linux-amd64" >> "$GITHUB_PATH"
- name: Install, ingest, upgrade, and restart both charts
run: bash tests/e2e/migrations/lens_helm_smoke.sh

View file

@ -129,7 +129,7 @@ jobs:
set +e
uv run --no-sync --with mutmut==3.5.0 mutmut export-cicd-stats > /dev/null 2>&1
uv run --no-sync --with mutmut==3.5.0 mutmut results > mutmut-results.txt 2>&1
uv run --no-sync python scripts/mutation_report.py
uv run --no-sync python litellm/scripts/mutation_report.py
# The full report can be very long for big test files; the run-page
# summary cuts off at 1 MB. Append the head of the report (summary
# + survivor list) and link out to the artifact for the full body.

View file

@ -5,6 +5,9 @@ on:
branches:
- main
- "litellm_**"
- "rc/**"
- "stable/**"
- "lts/**"
schedule:
- cron: "23 6 * * *"
workflow_dispatch:
@ -37,7 +40,7 @@ jobs:
- name: Scan lockfiles
run: |
"$RUNNER_TEMP/osv-scanner" scan source \
--config osv-scanner.toml \
--config .github/osv-scanner.toml \
-L uv.lock \
-L ui/litellm-dashboard/package-lock.json \
-L vscode-extension/package-lock.json
-L litellm/ui/litellm-dashboard/package-lock.json \
-L litellm/vscode-extension/package-lock.json

View file

@ -3,7 +3,7 @@ name: Publish lint base counts
# Every commit on main can become a future merge-base.
# Publishing its per-rule counts for each lint gate (strict ruff, type discipline,
# test quality, basedpyright) as an artifact lets the gates download them through
# scripts/lint_base_counts.py in seconds instead of scanning the merge-base tree in
# litellm/scripts/lint_base_counts.py in seconds instead of scanning the merge-base tree in
# a throwaway worktree on every fresh checkout or moved merge-base.
# No concurrency group on purpose: runs must never cancel each other, because
# every sha's artifact matters (any of them can become a merge-base).
@ -30,15 +30,15 @@ jobs:
matrix:
include:
- checker: ruff-strict
script: scripts/ruff_strict_gate.py
script: litellm/scripts/ruff_strict_gate.py
- checker: type-discipline
script: scripts/type_discipline_gate.py
script: litellm/scripts/type_discipline_gate.py
- checker: test-quality
script: scripts/test_quality_gate.py
script: litellm/scripts/test_quality_gate.py
- checker: openapi-docs
script: scripts/openapi_docs_gate.py
script: litellm/scripts/openapi_docs_gate.py
- checker: basedpyright
script: scripts/type_check_gate.py
script: litellm/scripts/type_check_gate.py
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
@ -91,7 +91,7 @@ jobs:
- name: Emit basedpyright counts for HEAD
if: matrix.checker == 'basedpyright'
run: |
python scripts/type_check_gate.py --emit-counts-dir "$RUNNER_TEMP/lint-counts"
python litellm/scripts/type_check_gate.py --emit-counts-dir "$RUNNER_TEMP/lint-counts"
- name: Name the counts artifact
run: |

27
.github/workflows/root-entry-limit.yml vendored Normal file
View file

@ -0,0 +1,27 @@
name: Root entry limit
on:
pull_request: {}
push:
branches: [main]
permissions:
contents: read
jobs:
root-entry-limit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955
with:
persist-credentials: false
- name: Check root entry count
env:
MAX_ROOT_ENTRIES: 15
run: |
count=$(find . -mindepth 1 -maxdepth 1 -printf '.\n' | wc -l)
echo "Root has $count entries. Maximum is $MAX_ROOT_ENTRIES."
if [ "$count" -gt "$MAX_ROOT_ENTRIES" ]; then
echo "::error::Repository root exceeds $MAX_ROOT_ENTRIES entries. Move new files into an existing directory."
exit 1
fi

View file

@ -19,17 +19,17 @@ jobs:
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
ref: ${{ github.event.pull_request.base.sha }}
ref: ${{ github.event.pull_request.base.ref }}
persist-credentials: false
- name: Fetch the pull request head, its merge base and its reviews
id: revisions
env:
GH_TOKEN: ${{ github.token }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
PR_NUMBER: ${{ github.event.pull_request.number }}
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: |
merge_base="$(gh api "repos/${GITHUB_REPOSITORY}/compare/${BASE_SHA}...${HEAD_SHA}" --jq '.merge_base_commit.sha')"
merge_base="$(gh api "repos/${GITHUB_REPOSITORY}/compare/${BASE_REF}...${HEAD_SHA}" --jq '.merge_base_commit.sha')"
git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA"
gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" --paginate --slurp | jq 'add // []' > "$RUNNER_TEMP/reviews.json"
echo "merge_base=$merge_base" >> "$GITHUB_OUTPUT"
@ -42,4 +42,12 @@ jobs:
MERGE_BASE: ${{ steps.revisions.outputs.merge_base }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
uv run --no-project --with pydantic==2.14.1 python ci_cd/spend_logs_guard.py --base "$MERGE_BASE" --head "$HEAD_SHA" --approver kerry-berri --reviews "$RUNNER_TEMP/reviews.json"
guard=litellm/scripts/ci_cd/spend_logs_guard.py
# TODO: Drop the legacy path after this lands on main.
if [ ! -f "$guard" ]; then
guard=scripts/ci_cd/spend_logs_guard.py
fi
if [ ! -f "$guard" ]; then
guard=ci_cd/spend_logs_guard.py
fi
uv run --no-project --with pydantic==2.14.1 python "$guard" --base "$MERGE_BASE" --head "$HEAD_SHA" --approver kerry-berri --reviews "$RUNNER_TEMP/reviews.json"

View file

@ -1,9 +1,9 @@
name: Sync schema.prisma copies
name: Sync litellm/schema.prisma copies
on:
pull_request:
paths:
- 'schema.prisma'
- 'litellm/schema.prisma'
# Scoped to ONLY the permissions needed:
# - contents:write to push the sync commit to the PR branch
@ -37,7 +37,7 @@ jobs:
- name: Reject symlinked schema files
run: |
for f in schema.prisma litellm/proxy/schema.prisma litellm-proxy-extras/litellm_proxy_extras/schema.prisma; do
for f in litellm/schema.prisma litellm/proxy/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma; do
if [ -L "$f" ]; then
echo "::error file=$f::$f is a symlink, which is not allowed"
exit 1
@ -46,13 +46,13 @@ jobs:
- name: Copy root schema to other locations
run: |
cp schema.prisma litellm/proxy/schema.prisma
cp schema.prisma litellm-proxy-extras/litellm_proxy_extras/schema.prisma
cp litellm/schema.prisma litellm/proxy/schema.prisma
cp litellm/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma
- name: Check for changes
id: diff
run: |
if git diff --quiet -- litellm/proxy/schema.prisma litellm-proxy-extras/litellm_proxy_extras/schema.prisma; then
if git diff --quiet -- litellm/proxy/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma; then
echo "changed=false" >> "$GITHUB_OUTPUT"
echo "Schemas already in sync. Nothing to do."
else
@ -68,6 +68,6 @@ jobs:
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -B "$GITHUB_HEAD_REF"
git add -- litellm/proxy/schema.prisma litellm-proxy-extras/litellm_proxy_extras/schema.prisma
git commit -m "chore: sync schema.prisma copies from root"
git add -- litellm/proxy/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma
git commit -m "chore: sync litellm/schema.prisma copies from the canonical schema"
git push origin "HEAD:$GITHUB_HEAD_REF"

220
.github/workflows/test-integration.yml vendored Normal file
View file

@ -0,0 +1,220 @@
name: integration
on:
pull_request:
types: [labeled]
schedule:
- cron: "5 8-20/3 * * 1-6"
timezone: America/Los_Angeles
workflow_dispatch:
inputs:
suite:
description: Integration group to run
type: choice
options:
- all
- providers
- extensions
- accounting
- management
- management-replica
- database
- database-replica
- mcp
- security
- cost
- sdk
- browser
default: all
workers:
description: Workers per group, blank keeps each group's default
type: string
default: ""
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}-${{ github.event.label.name }}-${{ inputs.suite }}-${{ inputs.workers }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
integration:
name: integration ${{ matrix.suite }}
if: github.event_name != 'pull_request' || github.event.label.name == 'run-ci'
environment: integration-license
runs-on: ${{ fromJSON('{"providers":"ubuntu-latest-64-cores","extensions":"ubuntu-latest-64-cores","accounting":"ubuntu-latest-32-cores","management":"ubuntu-latest-32-cores","management-replica":"ubuntu-latest-32-cores"}')[matrix.suite] || 'ubuntu-latest-16-cores' }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
suite: ${{ fromJSON((inputs.suite || 'all') == 'all' && '["providers","extensions","accounting","management","management-replica","database","database-replica","mcp","security","cost","sdk","browser"]' || format('["{0}"]', inputs.suite)) }}
env:
UV_PYTHON: "3.12"
INTEGRATION_WORKERS: ${{ inputs.workers || fromJSON('{"providers":"32","extensions":"48","accounting":"16","management":"16","management-replica":"16","cost":"16","mcp":"8","security":"8","database":"4","database-replica":"4"}')[matrix.suite] || '' }}
services:
postgres:
image: postgres:16@sha256:e17e86066e5ef83e0952a9347f5c792b7ece00972e2aa787a6986f471b3dd3d5
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: circle_test
ports:
- 5432:5432
options: >-
--shm-size 1g
--health-cmd "pg_isready -U postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 20
redis:
image: redis:7-alpine@sha256:7aec734b2bb298a1d769fd8729f13b8514a41bf90fcdd1f38ec52267fbaa8ee6
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 20
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- name: Configure PostgreSQL for integration workers
env:
POSTGRES_CONTAINER: ${{ job.services.postgres.id }}
run: |
docker exec "$POSTGRES_CONTAINER" sh -c 'printf "%s\n" "max_connections = 1000" "shared_preload_libraries = pg_stat_statements" "pg_stat_statements.track = all" "pg_stat_statements.max = 20000" >> "$PGDATA/postgresql.conf"'
docker restart "$POSTGRES_CONTAINER"
for attempt in $(seq 1 30); do
if docker exec "$POSTGRES_CONTAINER" pg_isready -U postgres; then exit 0; fi
sleep 1
done
exit 1
- name: Set up Python
timeout-minutes: 3
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.UV_PYTHON }}
- name: Set up uv
timeout-minutes: 3
uses: ./.github/actions/setup-uv-with-retries
with:
version: "0.10.9"
- name: Cache the Rust build
timeout-minutes: 5
uses: ./.github/actions/cache-cargo-build
- name: Cache Prisma binaries
timeout-minutes: 5
uses: ./.github/actions/cache-prisma-binaries
- name: Install dependencies
timeout-minutes: 20
run: .github/scripts/uv_sync_with_retries.sh --frozen --all-groups --all-extras
- name: Install the Rust gateway
if: matrix.suite == 'management' || matrix.suite == 'management-replica'
timeout-minutes: 15
working-directory: litellm/packaging/litellm-proxy
env:
CARGO_INCREMENTAL: "0"
run: |
uv run --no-project --with maturin==1.15.0 maturin build --profile dev --strip --out "$RUNNER_TEMP/gateway-dist"
uv pip install --no-deps --python ../../../.venv/bin/python "$RUNNER_TEMP"/gateway-dist/*.whl
test -x ../../../.venv/bin/litellm-gateway
- name: Install redis-server 7.2.16
timeout-minutes: 3
run: |
mkdir -p "$RUNNER_TEMP/redis-bin"
cid="$(docker create redis:7.2.16@sha256:0637954999d01b7c9ce9167db2da50656e2590d3b884f1c600c5f63bb6e6773c)"
docker cp "${cid}:/usr/local/bin/redis-server" "$RUNNER_TEMP/redis-bin/redis-server"
docker rm "$cid"
"$RUNNER_TEMP/redis-bin/redis-server" --version | grep -q 'v=7.2.16'
echo "$RUNNER_TEMP/redis-bin" >> "$GITHUB_PATH"
- name: Install MCP SDK1 peer
if: matrix.suite == 'mcp'
timeout-minutes: 5
run: |
uv venv --python 3.12 .venv-mcp-peer
uv pip install --no-config --exclude-newer "3 days" --python .venv-mcp-peer 'mcp==1.28.1' 'langchain-mcp-adapters==0.2.1'
echo "MCP_TEST_PEER_PYTHON=$PWD/.venv-mcp-peer/bin/python" >> "$GITHUB_ENV"
- name: Set up Node.js
if: matrix.suite == 'browser'
timeout-minutes: 5
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version-file: litellm/ui/litellm-dashboard/.nvmrc
cache: "npm"
cache-dependency-path: |
litellm/ui/litellm-dashboard/package-lock.json
tests/e2e/ui/package-lock.json
- name: Install browser dependencies and build the dashboard
if: matrix.suite == 'browser'
timeout-minutes: 20
run: |
(cd litellm/ui/litellm-dashboard && npm ci)
(cd tests/e2e/ui && npm ci)
sudo env PATH="$PATH" DEBIAN_FRONTEND=noninteractive \
node tests/e2e/ui/node_modules/@playwright/test/cli.js install-deps chromium
node tests/e2e/ui/node_modules/@playwright/test/cli.js install chromium
(cd litellm/ui/litellm-dashboard && NEXT_TELEMETRY_DISABLED=1 npm run build)
- name: Run integration contracts
timeout-minutes: 45
env:
SUITE: ${{ matrix.suite }}
INTEGRATION_POSTGRES_CONTAINER: ${{ job.services.postgres.id }}
INTEGRATION_REDIS_CONTAINER: ${{ job.services.redis.id }}
LITELLM_LICENSE: ${{ secrets.LITELLM_LICENSE }}
run: |
case "$SUITE" in
*-replica) bash .github/scripts/ci/run_integration.sh "${SUITE%-replica}" replica ;;
*) bash .github/scripts/ci/run_integration.sh "$SUITE" ;;
esac
- name: Verify every case ran exactly once
if: ${{ !cancelled() && matrix.suite != 'browser' }}
timeout-minutes: 2
run: |
record="$(find test-results -mindepth 2 -maxdepth 2 -name execution.json -print -quit)"
test -n "$record"
mkdir -p "$RUNNER_TEMP/shards/0"
cp "$record" "$RUNNER_TEMP/shards/0/execution.json"
.venv/bin/python -m tests.integration._support.sharding "$RUNNER_TEMP/shards" --shards 1
- name: Report MCP coverage
if: ${{ !cancelled() && matrix.suite == 'mcp' }}
timeout-minutes: 5
run: |
coverage_dir=test-results/integration-mcp/coverage
test -f "$coverage_dir/coverage.txt" && exit 0
export COVERAGE_FILE="$PWD/$coverage_dir/data"
.venv/bin/python -m coverage combine --rcfile=tests/integration/mcp_coverage.toml
.venv/bin/python -m coverage report --rcfile=tests/integration/mcp_coverage.toml > "$coverage_dir/coverage.txt"
.venv/bin/python -m coverage html --rcfile=tests/integration/mcp_coverage.toml -d "$coverage_dir/html"
tail -n 1 "$coverage_dir/coverage.txt"
- name: Check results for the license
id: secret-scan
if: ${{ !cancelled() }}
timeout-minutes: 5
env:
SCANNED_SECRET: ${{ secrets.LITELLM_LICENSE }}
run: python .github/scripts/assert_secret_absent.py test-results
- name: Upload results
if: ${{ !cancelled() && steps.secret-scan.outcome == 'success' }}
timeout-minutes: 10
uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
with:
name: integration-${{ matrix.suite }}-${{ github.run_id }}-${{ github.run_attempt }}
path: test-results/
retention-days: 14

View file

@ -21,7 +21,7 @@ jobs:
pull-requests: read
actions: read
runs-on: ubuntu-latest
timeout-minutes: 20
timeout-minutes: 30
services:
redis:
image: redis@sha256:e2debfb7956fa12c7ddc79d7e645c8cf26b30c99a6e9161ea9bf4171e1668a5f
@ -48,11 +48,13 @@ jobs:
if: steps.changes.outputs.decision != 'skip'
env:
GH_TOKEN: ${{ github.token }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_NUMBER: ${{ github.event.pull_request.number }}
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
retry() { "$@" || { sleep 15; "$@"; } || { sleep 30; "$@"; }; }
MERGE_BASE=$(retry gh api "repos/${{ github.repository }}/compare/${BASE_SHA}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
BASE_REF=$(gh api "repos/${GITHUB_REPOSITORY}/branches/${EVENT_BASE_REF}" --jq .name) || BASE_REF=$(retry gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq .base.ref)
MERGE_BASE=$(retry gh api "repos/${GITHUB_REPOSITORY}/compare/${BASE_REF}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
test -n "$MERGE_BASE"
retry git fetch --no-tags --depth=1 origin "$MERGE_BASE"
echo "GATE_BASE_SHA=$MERGE_BASE" >> "$GITHUB_ENV"
@ -71,14 +73,14 @@ jobs:
- name: Cache uv dependencies
if: steps.changes.outputs.decision != 'skip'
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.cache/uv
.venv
key: ${{ runner.os }}-uv-lint-${{ hashFiles('uv.lock') }}
key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
restore-keys: |
${{ runner.os }}-uv-lint-
${{ runner.os }}-uv-
- name: Clean Python cache
if: steps.changes.outputs.decision != 'skip'
@ -112,7 +114,7 @@ jobs:
- name: Check ruff format
if: steps.changes.outputs.decision != 'skip'
run: |
git diff --name-only --diff-filter=ACMR "$GATE_BASE_SHA" HEAD -- ':(glob)litellm/**/*.py' | grep -v '^litellm/enterprise/' > "$RUNNER_TEMP/ruff_format_files.txt" || true
git diff --name-only --diff-filter=ACMR "$GATE_BASE_SHA" HEAD -- ':(glob)litellm/**/*.py' | grep -Ev '^litellm/(enterprise|ui|vscode-extension|rust|proxy-extras|packaging|cli|scripts|runtime)/' > "$RUNNER_TEMP/ruff_format_files.txt" || true
if [ ! -s "$RUNNER_TEMP/ruff_format_files.txt" ]; then
echo "No changed litellm Python files to check with ruff format."
exit 0
@ -131,7 +133,7 @@ jobs:
- name: Check MCP operation boundary
if: steps.changes.outputs.decision != 'skip'
run: uv run --no-sync python scripts/check_mcp_operation_boundary.py
run: uv run --no-sync python litellm/scripts/check_mcp_operation_boundary.py
- name: Run Ruff linting
if: steps.changes.outputs.decision != 'skip'
@ -143,28 +145,28 @@ jobs:
- name: Run Ruff linting (test tree)
if: steps.changes.outputs.decision != 'skip'
run: |
uv run --no-sync ruff check --config ruff-tests.toml tests
uv run --no-sync ruff check --config .github/linters/ruff-tests.toml tests
- name: Check strict ruff rules (delta vs merge-base counts)
if: steps.changes.outputs.decision != 'skip'
env:
GH_TOKEN: ${{ github.token }}
run: |
uv run --no-sync python scripts/ruff_strict_gate.py --base "$GATE_BASE_SHA"
uv run --no-sync python litellm/scripts/ruff_strict_gate.py --base "$GATE_BASE_SHA"
- name: Check type discipline (mutable collections / casts / type guards / kwargs / unexplained suppressions, delta vs merge-base counts)
if: steps.changes.outputs.decision != 'skip'
env:
GH_TOKEN: ${{ github.token }}
run: |
uv run --no-sync python scripts/type_discipline_gate.py --base "$GATE_BASE_SHA"
uv run --no-sync python litellm/scripts/type_discipline_gate.py --base "$GATE_BASE_SHA"
- name: Check test quality (zero-assert / mock-echo tests, sys.path.insert, raw env writes, litellm global mutation, credential-gated skips, conftest snapshot inventory, delta vs merge-base counts)
if: steps.changes.outputs.decision != 'skip'
env:
GH_TOKEN: ${{ github.token }}
run: |
uv run --no-sync python scripts/test_quality_gate.py --base "$GATE_BASE_SHA"
uv run --no-sync python litellm/scripts/test_quality_gate.py --base "$GATE_BASE_SHA"
- name: Check OpenAPI docs (POST routes without a request schema, description or 200 schema, delta vs merge-base counts)
if: steps.changes.outputs.decision != 'skip'
@ -172,7 +174,7 @@ jobs:
GH_TOKEN: ${{ github.token }}
run: |
uv sync --frozen --group proxy-dev --group e2e-dev --extra proxy
uv run --no-sync python scripts/openapi_docs_gate.py --base "$GATE_BASE_SHA"
uv run --no-sync python litellm/scripts/openapi_docs_gate.py --base "$GATE_BASE_SHA"
- name: Print OpenAI version
if: steps.changes.outputs.decision != 'skip'
@ -184,12 +186,12 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
run: |
uv run --no-sync python scripts/type_check_gate.py --base "$GATE_BASE_SHA"
uv run --no-sync python litellm/scripts/type_check_gate.py --base "$GATE_BASE_SHA"
- name: Check tests/e2e basedpyright (zero errors)
if: steps.changes.outputs.decision != 'skip'
run: |
if git diff --name-only --diff-filter=ACMRD "$GATE_BASE_SHA" HEAD -- ':(glob)tests/e2e/**/*.py' ':(glob)tests/e2e_harness/**/*.py' pyrightconfig.json | grep -q .; then
if git diff --name-only --diff-filter=ACMRD "$GATE_BASE_SHA" HEAD -- ':(glob)tests/e2e/**/*.py' ':(glob)tests/e2e_harness/**/*.py' pyproject.toml | grep -q .; then
uv run --no-sync basedpyright tests/e2e tests/e2e_harness
else
echo "No changed tests/e2e Python files; skipping."
@ -206,7 +208,7 @@ jobs:
exit 0
fi
retry() { "$@" || { sleep 15; "$@"; } || { sleep 30; "$@"; }; }
CLAUDE_VERSION="$(retry uv run --no-sync python tests/e2e/claude_code/pr_gate_version_resolver.py)"
CLAUDE_VERSION="$(retry uv run --no-sync python tests/e2e/cli_harness/npm_version_resolver.py @anthropic-ai/claude-code)"
tests/e2e/claude_code/cron_vm/install_claude_code.sh "$CLAUDE_VERSION" "$RUNNER_TEMP/claude-cli"
PATH="$RUNNER_TEMP/claude-cli:$PATH" uv run --no-sync pytest -q tests/e2e_harness
uv run --no-sync pytest --tb=short -q --noconftest -o addopts= -o "pythonpath=tests/e2e tests/e2e_harness" -p no:rerunfailures \
@ -256,7 +258,7 @@ jobs:
if [ -n "$GITGUARDIAN_API_KEY" ]; then
retry() { "$@" || { sleep 15; "$@"; } || { sleep 30; "$@"; }; }
retry git fetch --no-tags --unshallow origin
uv tool run --from 'ggshield==1.48.0' ggshield secret scan repo .
uv tool run --from 'ggshield==1.48.0' ggshield --config-path .github/gitguardian.yaml secret scan repo .
else
echo "GITGUARDIAN_API_KEY not set, skipping ggshield scan"
fi
@ -264,11 +266,12 @@ jobs:
name: ui
permissions:
contents: read
pull-requests: read
runs-on: ubuntu-latest
timeout-minutes: 8
defaults:
run:
working-directory: ui/litellm-dashboard
working-directory: litellm/ui/litellm-dashboard
steps:
- name: Checkout repository
@ -281,16 +284,12 @@ jobs:
id: changed
env:
GH_TOKEN: ${{ github.token }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_NUMBER: ${{ github.event.pull_request.number }}
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
# base.sha is the base branch tip from when the PR was opened, while
# actions/checkout leaves HEAD on a merge of the PR into the *current*
# base tip. "$BASE_SHA"...HEAD therefore spans every base-branch commit
# landed since, so a PR that touches no UI file still gets linted
# against hundreds of other people's files. Diff the PR head against its
# own merge base instead, which is exactly what this PR changed.
merge_base=$(gh api "repos/${{ github.repository }}/compare/${BASE_SHA}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
base_ref=$(gh api "repos/${GITHUB_REPOSITORY}/branches/${EVENT_BASE_REF}" --jq .name) || base_ref=$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq .base.ref)
merge_base=$(gh api "repos/${GITHUB_REPOSITORY}/compare/${base_ref}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
test -n "$merge_base"
git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA"
: > "$RUNNER_TEMP/prettier_files.txt"
@ -316,9 +315,9 @@ jobs:
if: steps.changed.outputs.has_files == 'true'
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version-file: ui/litellm-dashboard/.nvmrc
node-version-file: litellm/ui/litellm-dashboard/.nvmrc
cache: "npm"
cache-dependency-path: ui/litellm-dashboard/package-lock.json
cache-dependency-path: litellm/ui/litellm-dashboard/package-lock.json
- name: Install dependencies
if: steps.changed.outputs.has_files == 'true'
@ -334,12 +333,12 @@ jobs:
status=0
if [ ${#prettier_files[@]} -gt 0 ]; then
echo "::group::Prettier (${#prettier_files[@]} files)"
npx prettier --check "${prettier_files[@]}" || { status=1; echo "::error::Unformatted files. Fix with: npm run format"; }
node_modules/.bin/prettier --check "${prettier_files[@]}" || { status=1; echo "::error::Unformatted files. Fix with: npm run format"; }
echo "::endgroup::"
fi
if [ ${#eslint_files[@]} -gt 0 ]; then
echo "::group::ESLint (${#eslint_files[@]} files)"
npx eslint --no-warn-ignored --pass-on-unpruned-suppressions "${eslint_files[@]}" || status=1
node_modules/.bin/eslint --no-warn-ignored --pass-on-unpruned-suppressions "${eslint_files[@]}" || status=1
echo "::endgroup::"
fi
exit $status
@ -376,10 +375,10 @@ jobs:
exit 0
fi
files="$(git diff --name-only "$base" HEAD)"
if grep -Eq '^(litellm/(proxy|types)/|ui/litellm-dashboard/(src/lib/http/schema\.d\.ts|scripts/gen-api-types\.mjs|package(-lock)?\.json)$|\.github/workflows/test-linting\.yml$)' <<< "$files"; then
if grep -Ev '^litellm/(ui|vscode-extension|rust|proxy-extras|packaging|cli|scripts|runtime)/' <<< "$files" | grep -E '^(litellm/|enterprise/|(pyproject\.toml|uv\.lock)$|\.github/workflows/test-linting\.yml$)' >/dev/null || grep -Eq '^litellm/ui/litellm-dashboard/(src/lib/http/schema\.d\.ts|scripts/gen-api-types\.mjs|package(-lock)?\.json)$' <<< "$files"; then
echo "relevant=true" >> "$GITHUB_OUTPUT"
else
echo "No proxy, types or generator changes in this pull request, nothing to verify."
echo "No litellm, enterprise, dependency or generator changes in this pull request, nothing to verify."
echo "relevant=false" >> "$GITHUB_OUTPUT"
fi
@ -397,7 +396,7 @@ jobs:
- name: Cache uv dependencies
if: steps.changes.outputs.relevant == 'true'
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.cache/uv
@ -435,7 +434,7 @@ jobs:
echo "A lazily loaded route or model changed without regenerating the snapshot that /openapi.json serves for unloaded features."
echo "To fix, run from the repo root:"
echo " uv run python -m litellm.proxy._lazy_openapi_snapshot"
echo "then run npm run gen:api from ui/litellm-dashboard and commit both files."
echo "then run npm run gen:api from litellm/ui/litellm-dashboard and commit both files."
exit 1
fi
echo "_lazy_openapi_snapshot.json is in sync with the lazily loaded routes."
@ -444,18 +443,18 @@ jobs:
if: steps.changes.outputs.relevant == 'true'
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version-file: ui/litellm-dashboard/.nvmrc
node-version-file: litellm/ui/litellm-dashboard/.nvmrc
cache: "npm"
cache-dependency-path: ui/litellm-dashboard/package-lock.json
cache-dependency-path: litellm/ui/litellm-dashboard/package-lock.json
- name: Install dashboard dependencies
if: steps.changes.outputs.relevant == 'true'
working-directory: ui/litellm-dashboard
working-directory: litellm/ui/litellm-dashboard
run: npm ci
- name: Regenerate types from the live spec
if: steps.changes.outputs.relevant == 'true'
working-directory: ui/litellm-dashboard
working-directory: litellm/ui/litellm-dashboard
env:
LITELLM_PYTHON: "uv run --no-sync python"
run: npm run gen:api
@ -463,11 +462,11 @@ jobs:
- name: Fail if types are stale
if: steps.changes.outputs.relevant == 'true'
run: |
if ! git diff --exit-code -- ui/litellm-dashboard/src/lib/http/schema.d.ts; then
echo "::error file=ui/litellm-dashboard/src/lib/http/schema.d.ts::Generated API types are out of sync with the proxy OpenAPI spec."
if ! git diff --exit-code -- litellm/ui/litellm-dashboard/src/lib/http/schema.d.ts; then
echo "::error file=litellm/ui/litellm-dashboard/src/lib/http/schema.d.ts::Generated API types are out of sync with the proxy OpenAPI spec."
echo ""
echo "A backend route or model changed without regenerating the dashboard types."
echo "To fix, run from ui/litellm-dashboard:"
echo "To fix, run from litellm/ui/litellm-dashboard:"
echo " npm run gen:api"
echo "then commit the updated src/lib/http/schema.d.ts."
exit 1
@ -525,7 +524,7 @@ jobs:
run: uv run --no-sync python ./tests/code_coverage_tests/check_provider_folders_documented.py
- name: check_provider_endpoints_support_generated
run: uv run --no-sync python scripts/generate_provider_endpoints_support.py --check
run: uv run --no-sync python litellm/scripts/generate_provider_endpoints_support.py --check
- name: test_generate_provider_endpoints_support
run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_generate_provider_endpoints_support.py
@ -630,6 +629,12 @@ jobs:
- name: check_migrations_lock_timeout
run: uv run --no-sync python ./tests/code_coverage_tests/check_migrations_lock_timeout.py
- name: check_no_none_string_error_fields
run: uv run --no-sync python ./tests/code_coverage_tests/check_no_none_string_error_fields.py
- name: test_check_no_none_string_error_fields
run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_check_no_none_string_error_fields.py
- name: check_no_publicly_known_master_key
run: uv run --no-sync python ./tests/code_coverage_tests/check_no_publicly_known_master_key.py
@ -698,7 +703,7 @@ jobs:
version: "0.10.9"
- name: Run Semgrep (custom rules)
run: uv tool run --from 'semgrep==1.157.0' semgrep scan --config .semgrep/rules . --error
run: uv tool run --from 'semgrep==1.157.0' semgrep scan --config .github/semgrep/rules . --error
lint-passed:
name: lint passed
needs: [python, secret-scan, ui, ui-api-types, code-quality, ci-coverage, semgrep]

View file

@ -10,17 +10,17 @@ on:
- "uv.lock"
- "uv.toml"
- ".python-version"
- "rust-toolchain.toml"
- "litellm-rust/**"
- "litellm/rust/rust-toolchain.toml"
- "litellm/rust/**"
- "litellm/__init__.py"
- "litellm/_version.py"
- "litellm/_lazy_imports_registry.py"
- "scripts/build_core_distribution.py"
- "litellm/scripts/build_core_distribution.py"
- "litellm/proxy/proxy_server.py"
- "litellm/**/*mcp*"
- "litellm/**/*mcp*/**"
- "litellm/integrations/arize/**"
- "scripts/check_mcp_sdk_install.py"
- "litellm/scripts/check_mcp_sdk_install.py"
- "tests/base_sdk_tests/**"
- ".github/workflows/test-mcp-dependency-resolution.yml"
- ".github/actions/detect-changes/**"
@ -28,7 +28,7 @@ on:
- ".github/actions/cache-cargo-build/**"
- ".github/scripts/detect_changes.sh"
- ".github/scripts/uv_sync_with_retries.sh"
- ".circleci/scripts/classify_changes.sh"
- ".github/scripts/ci/classify_changes.sh"
permissions:
contents: read
@ -38,6 +38,16 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
# These jobs check packaging, installation and imports, so they build the Rust
# workspace unoptimized. The release profile's fat LTO and opt-level 3 pushed
# every job to its timeout. The overrides go through Cargo's environment
# because a maturin `--profile ci` argument also reaches dependencies that
# build from source, such as jiter on Python 3.15, which have no ci profile
env:
CARGO_PROFILE_RELEASE_OPT_LEVEL: "0"
CARGO_PROFILE_RELEASE_LTO: "off"
CARGO_PROFILE_RELEASE_CODEGEN_UNITS: "256"
jobs:
resolve:
runs-on: ubuntu-latest
@ -91,7 +101,7 @@ jobs:
checker=("$GITHUB_WORKSPACE/tests/base_sdk_tests/check_base_sdk_install.py")
(cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-$extra/bin/python" -I "$GITHUB_WORKSPACE/tests/base_sdk_tests/check_sdk_http.py")
else
checker=("$GITHUB_WORKSPACE/scripts/check_mcp_sdk_install.py" --extra "$extra")
checker=("$GITHUB_WORKSPACE/litellm/scripts/check_mcp_sdk_install.py" --extra "$extra")
fi
(cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-$extra/bin/python" "${checker[@]}")
done
@ -115,7 +125,7 @@ jobs:
if [ "$extra" = core ]; then
checker=("$GITHUB_WORKSPACE/tests/base_sdk_tests/check_base_sdk_install.py")
else
checker=("$GITHUB_WORKSPACE/scripts/check_mcp_sdk_install.py" --extra "$extra")
checker=("$GITHUB_WORKSPACE/litellm/scripts/check_mcp_sdk_install.py" --extra "$extra")
fi
(cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-lowest-$extra/bin/python" "${checker[@]}")
done
@ -147,15 +157,16 @@ jobs:
- name: Build the core wheel and source distribution
run: >-
uv run --no-project --with coverage==7.14.0 python -m coverage run --rcfile=/dev/null
--branch --include="*/scripts/build_core_distribution.py"
scripts/build_core_distribution.py --out-dir dist/core
--branch --include="*/litellm/scripts/build_core_distribution.py"
litellm/scripts/build_core_distribution.py --out-dir dist/core
- name: Verify metadata, resources, and independent source rebuild
env:
CORE_DISTRIBUTION_DIR: ${{ github.workspace }}/dist/core
run: >-
uv run --no-project --with pytest==9.0.3 --with pytest-cov==5.0.0 --with coverage==7.14.0 --with 'tomli==2.4.1; python_version < "3.11"'
python -m pytest tests/base_sdk_tests/test_core_distribution.py -v
--cov-config=/dev/null --cov=scripts.build_core_distribution --cov-branch --cov-append --cov-report=
python -m coverage run --rcfile=/dev/null --append --branch
--include="*/litellm/scripts/build_core_distribution.py"
-m pytest tests/base_sdk_tests/test_core_distribution.py -v
- name: Verify isolated core installations
run: |
wheel=$(realpath dist/core/litellm_core-*.whl)

View file

@ -3,6 +3,8 @@ name: smoke
on:
pull_request:
branches: [main, litellm_internal_staging, litellm_oss_staging, "litellm_**"]
push:
branches: [main]
workflow_dispatch:
permissions:
@ -24,7 +26,7 @@ jobs:
persist-credentials: false
- name: Build the dashboard stage
run: docker build --target ui-builder -f Dockerfile .
run: docker build --target ui-builder -f litellm/deploy/docker/Dockerfile .
core-checks:
name: py${{ matrix.python-version }}
@ -52,8 +54,26 @@ jobs:
with:
version: "0.10.9"
- name: Restore the Rust bridge
id: rust-bridge
uses: ./.github/actions/restore-rust-bridge
- name: Restore the Cargo build cache
if: steps.rust-bridge.outputs.cache-hit != 'true'
uses: ./.github/actions/cache-cargo-build
- name: Install dependencies
run: .github/scripts/uv_sync_with_retries.sh --frozen --extra proxy --extra cli --group dev --group proxy-dev --python ${{ matrix.python-version }}
env:
RUST_BRIDGE_RESTORED: ${{ steps.rust-bridge.outputs.cache-hit }}
PYTHON_VERSION: ${{ matrix.python-version }}
run: |
if [ "$RUST_BRIDGE_RESTORED" != "true" ]; then
.github/scripts/uv_sync_with_retries.sh --frozen --extra proxy --extra cli --group dev --group proxy-dev --python "$PYTHON_VERSION"
exit 0
fi
.github/scripts/uv_sync_with_retries.sh --frozen --extra proxy --extra cli --group dev --group proxy-dev --python "$PYTHON_VERSION" --no-install-project
uv pip install --no-deps --python .venv/bin/python rust-bridge-dist/litellm-*.whl
cp rust-bridge-dist/litellm/rust_bridge/_native.abi3.so litellm/rust_bridge/_native.abi3.so
- name: Create the loopback-only network namespace
run: |
@ -87,6 +107,7 @@ jobs:
- name: Upload smoke diagnostics
if: always()
continue-on-error: true
uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
with:
name: merge-smoke-diagnostics-py${{ matrix.python-version }}
@ -97,6 +118,10 @@ jobs:
if: always()
run: sudo ip netns delete smoke
- name: Keep only source-built wheels in the uv cache
if: always() && github.ref == 'refs/heads/main'
run: uv cache prune --ci
all-extras:
name: all-extras-py${{ matrix.python-version }}
runs-on: ubuntu-24.04
@ -123,12 +148,34 @@ jobs:
with:
version: "0.10.9"
- name: Restore the Rust bridge
id: rust-bridge
uses: ./.github/actions/restore-rust-bridge
- name: Restore the Cargo build cache
if: steps.rust-bridge.outputs.cache-hit != 'true'
uses: ./.github/actions/cache-cargo-build
- name: Install every extra and dependency group
run: .github/scripts/uv_sync_with_retries.sh --frozen --all-groups --all-extras --python ${{ matrix.python-version }}
env:
RUST_BRIDGE_RESTORED: ${{ steps.rust-bridge.outputs.cache-hit }}
PYTHON_VERSION: ${{ matrix.python-version }}
run: |
if [ "$RUST_BRIDGE_RESTORED" != "true" ]; then
.github/scripts/uv_sync_with_retries.sh --frozen --all-groups --all-extras --python "$PYTHON_VERSION"
exit 0
fi
.github/scripts/uv_sync_with_retries.sh --frozen --all-groups --all-extras --python "$PYTHON_VERSION" --no-install-project --no-install-package litellm-proxy
uv pip install --no-deps --python .venv/bin/python rust-bridge-dist/*.whl
cp rust-bridge-dist/litellm/rust_bridge/_native.abi3.so litellm/rust_bridge/_native.abi3.so
- name: Import the SDK, the proxy server and the enterprise package
run: .venv/bin/python -c "import litellm, litellm.proxy.proxy_server, litellm_enterprise"
- name: Keep only source-built wheels in the uv cache
if: always() && github.ref == 'refs/heads/main'
run: uv cache prune --ci
windows-sdk:
name: windows-sdk
runs-on: windows-2025
@ -154,15 +201,16 @@ jobs:
with:
version: "0.10.9"
- name: Set up the Rust bridge
uses: ./.github/actions/rust-bridge
- name: Install dependencies
run: .github/scripts/uv_sync_with_retries.sh --frozen --group dev --python 3.11
- name: Run the Windows tests
run: uv run --no-sync python -m pytest --tb=short -v tests/windows_tests
- name: Keep only source-built wheels in the uv cache
if: always() && github.ref == 'refs/heads/main'
run: uv cache prune --ci
proxy-build-from-pip:
name: proxy-build-from-pip
runs-on: ubuntu-24.04
@ -200,8 +248,36 @@ jobs:
- name: Install the test client dependencies
run: .github/scripts/uv_sync_with_retries.sh --frozen --no-install-workspace --extra proxy --group dev --python 3.12
- name: Build the image from the published package
run: docker build -t litellm-build-from-pip:smoke -f docker/build_from_pip/Dockerfile.build_from_pip .
- name: Cache Prisma binaries
timeout-minutes: 3
uses: ./.github/actions/cache-prisma-binaries
- name: Install the published proxy into its own venv
run: |
uv --no-config venv --python 3.13 /tmp/litellm-pip
uv --no-config pip install --python /tmp/litellm-pip/bin/python \
"litellm[proxy,proxy-runtime]==1.83.0" \
"google-cloud-aiplatform==1.133.0" \
"google-genai==1.37.0" \
"anthropic[vertex]==0.84.0" \
"grpcio==1.78.0" \
"prometheus-client==0.20.0" \
"langfuse==2.59.7" \
"opentelemetry-api==1.28.0" \
"opentelemetry-sdk==1.28.0" \
"opentelemetry-exporter-otlp==1.28.0" \
"ddtrace==4.11.0" \
"sentry-sdk==2.21.0" \
"mangum==0.17.0" \
"azure-ai-contentsafety==1.0.0" \
"azure-storage-file-datalake==12.20.0" \
"pypdf==6.7.5" \
"llm-sandbox==0.3.31" \
"detect-secrets==1.5.0" \
"prisma==0.11.0" \
"openai==2.24.0"
export PATH="/tmp/litellm-pip/bin:$PATH"
/tmp/litellm-pip/bin/prisma generate --schema=litellm/schema.prisma
- name: Start the fake OpenAI endpoint
run: |
@ -219,23 +295,21 @@ jobs:
echo "::add-mask::${key}"
echo "LITELLM_MASTER_KEY=${key}" >> "${GITHUB_ENV}"
- name: Start the proxy container
- name: Start the published proxy
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/circle_test
LITELLM_LICENSE: bad-license
FAKE_OPENAI_API_BASE: http://localhost:8190
OTEL_EXPORTER: in_memory
USE_DDTRACE: "True"
DD_TRACE_OPENAI_ENABLED: "False"
GCS_FLUSH_INTERVAL: "1"
LITELLM_LOG: ERROR
run: |
docker run -d \
-p 4000:4000 \
-e DATABASE_URL=postgresql://postgres:postgres@host.docker.internal:5432/circle_test \
-e LITELLM_MASTER_KEY \
-e FAKE_OPENAI_API_BASE=http://host.docker.internal:8190 \
-e OTEL_EXPORTER=in_memory \
-e USE_DDTRACE=True \
-e GCS_FLUSH_INTERVAL=1 \
-e LITELLM_LOG=ERROR \
--add-host host.docker.internal:host-gateway \
--name litellm-build-from-pip \
-v "${GITHUB_WORKSPACE}/docker/build_from_pip/litellm_config.yaml:/app/config.yaml" \
litellm-build-from-pip:smoke \
--config /app/config.yaml \
--port 4000
cd /tmp/litellm-pip
nohup /tmp/litellm-pip/bin/ddtrace-run /tmp/litellm-pip/bin/litellm \
--config "${GITHUB_WORKSPACE}/tests/basic_proxy_startup_tests/build_from_pip_config.yaml" \
--port 4000 > /tmp/litellm-pip.log 2>&1 &
for _ in $(seq 1 150); do
curl -sf --max-time 5 -o /dev/null http://localhost:4000 && exit 0
sleep 2
@ -270,7 +344,11 @@ jobs:
- name: Show the proxy logs
if: always()
run: docker logs litellm-build-from-pip || true
run: cat /tmp/litellm-pip.log || true
- name: Keep only source-built wheels in the uv cache
if: always() && github.ref == 'refs/heads/main'
run: uv cache prune --ci
smoke-passed:
name: smoke passed

View file

@ -3,14 +3,14 @@ name: LiteLLM Rust DB
on:
push:
paths:
- "litellm-rust/crates/db/**"
- "litellm-rust/Cargo.toml"
- "litellm-rust/Cargo.lock"
- "litellm-rust/clippy.toml"
- "litellm-proxy-extras/litellm_proxy_extras/migrations/**"
- "litellm/rust/crates/db/**"
- "litellm/rust/Cargo.toml"
- "litellm/rust/Cargo.lock"
- "litellm/rust/clippy.toml"
- "litellm/proxy-extras/litellm_proxy_extras/migrations/**"
- "**/schema.prisma"
- ".cargo/**"
- "rust-toolchain.toml"
- "litellm/rust/.cargo/**"
- "litellm/rust/rust-toolchain.toml"
- "pyproject.toml"
- "uv.lock"
- ".github/actions/setup-uv-with-retries/**"
@ -20,14 +20,14 @@ on:
- main
- "litellm_**"
paths:
- "litellm-rust/crates/db/**"
- "litellm-rust/Cargo.toml"
- "litellm-rust/Cargo.lock"
- "litellm-rust/clippy.toml"
- "litellm-proxy-extras/litellm_proxy_extras/migrations/**"
- "litellm/rust/crates/db/**"
- "litellm/rust/Cargo.toml"
- "litellm/rust/Cargo.lock"
- "litellm/rust/clippy.toml"
- "litellm/proxy-extras/litellm_proxy_extras/migrations/**"
- "**/schema.prisma"
- ".cargo/**"
- "rust-toolchain.toml"
- "litellm/rust/.cargo/**"
- "litellm/rust/rust-toolchain.toml"
- "pyproject.toml"
- "uv.lock"
- ".github/actions/setup-uv-with-retries/**"
@ -42,6 +42,7 @@ concurrency:
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: "0"
jobs:
db-queries:
@ -66,7 +67,7 @@ jobs:
SQLX_OFFLINE: "false"
defaults:
run:
working-directory: litellm-rust
working-directory: litellm/rust
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
@ -84,8 +85,9 @@ jobs:
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: litellm-rust
workspaces: litellm/rust
cache-on-failure: true
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Install Python dependencies
working-directory: .
@ -93,13 +95,13 @@ jobs:
- name: Deploy Prisma migrations
working-directory: .
run: uv run --no-sync prisma migrate deploy --schema litellm-proxy-extras/litellm_proxy_extras/schema.prisma
run: uv run --no-sync prisma migrate deploy --schema litellm/proxy-extras/litellm_proxy_extras/schema.prisma
- name: Install SQLx CLI
run: cargo install sqlx-cli --version 0.9.0 --locked --no-default-features -F postgres
- name: Check SQLx query metadata
working-directory: litellm-rust/crates/db
working-directory: litellm/rust/crates/db
run: cargo sqlx prepare --check
- name: Test database queries

View file

@ -3,12 +3,12 @@ name: LiteLLM Rust
on:
push:
paths:
- "litellm-rust/**"
- "litellm/rust/**"
- "model_prices_and_context_window.json"
- "litellm/rust_bridge/**"
- "scripts/generate_trace_types.py"
- "scripts/trace_codegen/**"
- "scripts/lens_assets/**"
- "litellm/scripts/generate_trace_types.py"
- "litellm/scripts/trace_codegen/**"
- "litellm/scripts/lens_assets/**"
- "litellm/tracing/**"
- "tests/test_litellm_rust/**"
- "litellm/integrations/custom_logger.py"
@ -24,10 +24,10 @@ on:
- "litellm/llms/base_llm/ocr/**"
- "litellm/llms/custom_httpx/llm_http_handler.py"
- "Makefile"
- ".cargo/**"
- "litellm/rust/.cargo/**"
- "pyproject.toml"
- "uv.lock"
- "rust-toolchain.toml"
- "litellm/rust/rust-toolchain.toml"
- ".github/actions/setup-uv-with-retries/**"
- ".github/scripts/smoke_test_native_wheel.py"
- ".github/scripts/verify_linux_native_wheel.py"
@ -38,12 +38,12 @@ on:
- main
- "litellm_**"
paths:
- "litellm-rust/**"
- "litellm/rust/**"
- "model_prices_and_context_window.json"
- "litellm/rust_bridge/**"
- "scripts/generate_trace_types.py"
- "scripts/trace_codegen/**"
- "scripts/lens_assets/**"
- "litellm/scripts/generate_trace_types.py"
- "litellm/scripts/trace_codegen/**"
- "litellm/scripts/lens_assets/**"
- "litellm/tracing/**"
- "tests/test_litellm_rust/**"
- "litellm/integrations/custom_logger.py"
@ -59,10 +59,10 @@ on:
- "litellm/llms/base_llm/ocr/**"
- "litellm/llms/custom_httpx/llm_http_handler.py"
- "Makefile"
- ".cargo/**"
- "litellm/rust/.cargo/**"
- "pyproject.toml"
- "uv.lock"
- "rust-toolchain.toml"
- "litellm/rust/rust-toolchain.toml"
- ".github/actions/setup-uv-with-retries/**"
- ".github/scripts/smoke_test_native_wheel.py"
- ".github/scripts/verify_linux_native_wheel.py"
@ -78,6 +78,7 @@ concurrency:
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: "0"
jobs:
rust-lint:
@ -85,19 +86,21 @@ jobs:
timeout-minutes: 15
defaults:
run:
working-directory: litellm-rust
working-directory: litellm/rust
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- run: rustup toolchain install --no-self-update
- name: Configure the Rust build environment
working-directory: .
run: bash .github/scripts/setup_rust_environment.sh
- run: cargo fmt --all --check
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: litellm-rust
workspaces: litellm/rust
cache-on-failure: true
save-if: ${{ github.ref == 'refs/heads/main' }}
@ -111,7 +114,7 @@ jobs:
CARGO_PROFILE_DEV_DEBUG: line-tables-only
defaults:
run:
working-directory: litellm-rust
working-directory: litellm/rust
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
@ -139,13 +142,13 @@ jobs:
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: litellm-rust
workspaces: litellm/rust
cache-on-failure: true
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Check generated trace contracts
working-directory: .
run: uv run scripts/generate_trace_types.py --check
run: uv run litellm/scripts/generate_trace_types.py --check
- run: cargo nextest run --workspace --locked
@ -185,7 +188,7 @@ jobs:
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: litellm-rust
workspaces: litellm/rust
cache-on-failure: true
save-if: ${{ github.ref == 'refs/heads/main' }}

View file

@ -3,16 +3,16 @@ name: Terraform Modules
on:
push:
paths:
- "terraform/litellm/aws/**"
- "terraform/litellm/gcp/**"
- "litellm/deploy/terraform/litellm/aws/**"
- "litellm/deploy/terraform/litellm/gcp/**"
- ".github/workflows/test-terraform-modules.yml"
pull_request:
branches:
- main
- "litellm_**"
paths:
- "terraform/litellm/aws/**"
- "terraform/litellm/gcp/**"
- "litellm/deploy/terraform/litellm/aws/**"
- "litellm/deploy/terraform/litellm/gcp/**"
- ".github/workflows/test-terraform-modules.yml"
permissions:
@ -33,7 +33,7 @@ jobs:
module: [aws, gcp]
defaults:
run:
working-directory: terraform/litellm/${{ matrix.module }}
working-directory: litellm/deploy/terraform/litellm/${{ matrix.module }}
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:

View file

@ -3,15 +3,18 @@ name: Terraform Provider
on:
push:
paths:
- "terraform/provider/**"
- "litellm/deploy/terraform/provider/**"
- ".github/actions/cache-go-build/**"
- ".github/workflows/test-terraform-provider.yml"
pull_request:
branches:
- main
- "litellm_**"
paths:
- "terraform/provider/**"
- "litellm/deploy/terraform/provider/**"
- "litellm/proxy/**"
- "litellm/enterprise/**"
- ".github/actions/cache-go-build/**"
- ".github/workflows/test-terraform-provider.yml"
permissions:
@ -28,7 +31,7 @@ jobs:
timeout-minutes: 10
defaults:
run:
working-directory: terraform/provider
working-directory: litellm/deploy/terraform/provider
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
@ -36,9 +39,13 @@ jobs:
- uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version-file: terraform/provider/go.mod
cache: true
cache-dependency-path: terraform/provider/go.sum
go-version-file: litellm/deploy/terraform/provider/go.mod
cache: false
- name: Cache the Go build
uses: ./.github/actions/cache-go-build
with:
dependency-path: litellm/deploy/terraform/provider/go.sum
- name: gofmt
run: |
@ -114,14 +121,18 @@ jobs:
- name: Generate proxy OpenAPI schema
run: |
uv run --no-sync python terraform/provider/tools/dump_openapi.py "${RUNNER_TEMP}/openapi.json"
uv run --no-sync python litellm/deploy/terraform/provider/tools/dump_openapi.py "${RUNNER_TEMP}/openapi.json"
- uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version-file: terraform/provider/go.mod
cache: true
cache-dependency-path: terraform/provider/go.sum
go-version-file: litellm/deploy/terraform/provider/go.mod
cache: false
- name: Cache the Go build
uses: ./.github/actions/cache-go-build
with:
dependency-path: litellm/deploy/terraform/provider/go.sum
- name: Audit provider endpoints against the schema
working-directory: terraform/provider
working-directory: litellm/deploy/terraform/provider
run: go run ./tools/endpointaudit -provider-dir ./litellm -spec "${RUNNER_TEMP}/openapi.json" -coverage-allowlist ./tools/endpointaudit/coverage_allowlist.txt

View file

@ -5,6 +5,9 @@ on:
branches:
- main
- "litellm_**"
- "rc/**"
- "stable/**"
- "lts/**"
push:
branches:
- main
@ -38,13 +41,6 @@ jobs:
timeout-minutes: 2
uses: ./.github/actions/detect-changes
- name: Define editable Rust bridge cache key
id: rust-bridge-key
if: steps.changes.outputs.decision != 'skip'
env:
RUST_BRIDGE_CACHE_KEY: ${{ runner.os }}-rust-bridge-editable-${{ hashFiles('litellm-rust/Cargo.lock', 'litellm-rust/Cargo.toml', 'litellm-rust/crates/**', 'model_prices_and_context_window.json', 'pyproject.toml') }}
run: echo "key=$RUST_BRIDGE_CACHE_KEY" >> "$GITHUB_OUTPUT"
- name: Set up Python
if: steps.changes.outputs.decision != 'skip'
timeout-minutes: 3
@ -63,10 +59,7 @@ jobs:
id: rust-bridge-cache
if: steps.changes.outputs.decision != 'skip'
timeout-minutes: 5
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830
with:
path: rust-bridge-dist
key: ${{ steps.rust-bridge-key.outputs.key }}
uses: ./.github/actions/restore-rust-bridge
- name: Cache the Rust build
if: steps.changes.outputs.decision != 'skip' && steps.rust-bridge-cache.outputs.cache-hit != 'true'
@ -76,13 +69,23 @@ jobs:
- name: Build editable Rust bridge
if: steps.changes.outputs.decision != 'skip' && steps.rust-bridge-cache.outputs.cache-hit != 'true'
timeout-minutes: 15
env:
CARGO_INCREMENTAL: "0"
run: |
mkdir -p rust-bridge-dist
uv run --no-project --with maturin==1.15.0 python -c 'import maturin; maturin.build_editable("rust-bridge-dist")'
uv run --no-project --with maturin==1.15.0 python .github/packaging/build_backend.py rust-bridge-dist
strip --strip-debug litellm/rust_bridge/_native.abi3.so
mkdir -p rust-bridge-dist/litellm/rust_bridge
cp litellm/rust_bridge/_native.abi3.so rust-bridge-dist/litellm/rust_bridge/_native.abi3.so
- name: Build the gateway wheel once for test shards
if: steps.changes.outputs.decision != 'skip' && steps.rust-bridge-cache.outputs.cache-hit != 'true'
timeout-minutes: 15
working-directory: litellm/packaging/litellm-proxy
env:
CARGO_INCREMENTAL: "0"
run: uv run --no-project --with maturin==1.15.0 maturin build --profile dev --strip --out ../../../rust-bridge-dist
- name: Upload Rust bridge artifact
if: steps.changes.outputs.decision != 'skip'
timeout-minutes: 10
@ -98,7 +101,7 @@ jobs:
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830
with:
path: rust-bridge-dist
key: ${{ steps.rust-bridge-key.outputs.key }}
key: ${{ steps.rust-bridge-cache.outputs.key }}
- name: Expose the Rust bridge artifact
id: rust-bridge-artifact
@ -155,6 +158,7 @@ jobs:
tests/unit/google_genai
tests/unit/router_strategy
tests/unit/router_utils
tests/unit/redis
tests/unit/proxy/common_utils/test_cache_aware_routing.py
tests/unit/enterprise/enterprise_callbacks/send_emails
tests/unit/enterprise/proxy/test_afile_retrieve_returns_unified_id.py
@ -227,6 +231,7 @@ jobs:
tests/unit/llms/test_oss_decision.py
tests/unit/llms/test_polling_url_origin_match.py
tests/unit/llms/test_predibase_transformation.py
tests/unit/llms/test_subscription_accounts.py
tests/unit/llms/a2a
tests/unit/llms/aiml
tests/unit/llms/aiohttp_openai
@ -236,6 +241,7 @@ jobs:
tests/unit/llms/azure_ai
tests/unit/llms/base_llm
tests/unit/llms/baseten
tests/unit/llms/bigmodel
tests/unit/llms/black_forest_labs
tests/unit/llms/brave
tests/unit/llms/bytez
@ -244,6 +250,7 @@ jobs:
tests/unit/llms/chatgpt
tests/unit/llms/clarifai
tests/unit/llms/claude_code
tests/unit/llms/clf_ai_gateway
tests/unit/llms/cloudflare
tests/unit/llms/codestral
tests/unit/llms/codex
@ -265,6 +272,7 @@ jobs:
tests/unit/llms/e2b
tests/unit/llms/edenai
tests/unit/llms/elevenlabs
tests/unit/llms/empiriolabs
tests/unit/llms/exa_ai
tests/unit/llms/fal_ai
tests/unit/llms/fastcrw
@ -321,6 +329,7 @@ jobs:
tests/unit/llms/openai_like
tests/unit/llms/opencode
tests/unit/llms/opensandbox
tests/unit/llms/opensearch
tests/unit/llms/openrouter
tests/unit/llms/ovhcloud
tests/unit/llms/parallel_ai
@ -364,6 +373,7 @@ jobs:
tests/unit/llms/voyage
tests/unit/llms/wandb
tests/unit/llms/watsonx
tests/unit/llms/webiq
tests/unit/llms/xai
tests/unit/llms/xinference
tests/unit/llms/you_com
@ -399,6 +409,7 @@ jobs:
- shard: router
test-path: >-
tests/unit/router_rust
tests/unit/test_router
python-version: "3.12"
workers: 4
@ -516,6 +527,7 @@ jobs:
tests/unit/proxy/management_helpers
tests/unit/proxy/list_api
tests/unit/proxy/analytics_endpoints
tests/unit/proxy/alerting_endpoints
tests/unit/proxy/decisions_endpoints
tests/unit/proxy/memory
tests/unit/proxy/agent_endpoints
@ -555,12 +567,14 @@ jobs:
tests/unit/proxy/credential_endpoints
tests/unit/proxy/discovery_endpoints
tests/unit/proxy/shutdown
tests/unit/proxy/rust_forward
tests/unit/proxy/public_endpoints
tests/unit/proxy/prompts
tests/unit/proxy/rag_endpoints
tests/unit/proxy/rerank_endpoints
tests/unit/proxy/realtime_endpoints
tests/unit/proxy/ui_crud_endpoints
tests/unit/proxy/deps
tests/unit/proxy/config_resolvers
tests/unit/proxy/utils
python-version: "3.12"
@ -596,23 +610,32 @@ jobs:
job-timeout-minutes: 60
dist: loadscope
- shard: proxy-infra
- shard: proxy-infra-db-spend
test-path: |-
tests/unit/proxy/db
--ignore=tests/unit/proxy/db/db_transaction_queue/test_e2e_pod_lock_manager.py
--ignore=tests/unit/proxy/db/test_update_daily_tag_spend.py
tests/unit/proxy/middleware
--ignore=tests/unit/proxy/middleware/test_request_size_limit_middleware.py
tests/unit/proxy/spend_tracking
--ignore=tests/unit/proxy/spend_tracking/test_search_api_logging.py
tests/unit/proxy/pass_through_endpoints
tests/unit/proxy/mcp_server/test_tool_registry.py
tests/unit/proxy/common_utils
--ignore=tests/unit/proxy/common_utils/test_cache_aware_routing.py
--ignore=tests/unit/proxy/common_utils/test_check_batch_cost.py
--ignore=tests/unit/proxy/common_utils/test_check_responses_cost.py
--ignore=tests/unit/proxy/common_utils/test_proxy_encrypt_decrypt.py
--ignore=tests/unit/proxy/common_utils/test_realtime_cache.py
python-version: "3.12"
workers: 4
reruns: 0
timeout-minutes: 20
job-timeout-minutes: 60
dist: loadscope
- shard: proxy-infra-routing
test-path: |-
tests/unit/proxy/middleware
--ignore=tests/unit/proxy/middleware/test_request_size_limit_middleware.py
tests/unit/proxy/pass_through_endpoints
tests/unit/proxy/mcp_server/test_tool_registry.py
tests/unit/proxy/enterprise_billing
tests/unit/proxy/types_utils
tests/unit/proxy/logging_endpoints
@ -627,34 +650,30 @@ jobs:
job-timeout-minutes: 60
dist: loadscope
- shard: proxy-infra-root
- shard: proxy-infra-root-a-p
test-path: |-
tests/unit/proxy/test_*.py
--ignore=tests/unit/proxy/test_aproxy_startup.py
--ignore=tests/unit/proxy/test_credential_slot_registry.py
--ignore=tests/unit/proxy/test_custom_callback_input.py
--ignore=tests/unit/proxy/test_custom_logger_s3_gcs.py
--ignore=tests/unit/proxy/test_custom_tokenizer_bug.py
--ignore=tests/unit/proxy/test_db_schema_changes.py
--ignore=tests/unit/proxy/test_deprecated_key_grace_period.py
--ignore=tests/unit/proxy/test_get_favicon.py
--ignore=tests/unit/proxy/test_get_image.py
--ignore=tests/unit/proxy/test_prisma_client_backoff_retry.py
--ignore=tests/unit/proxy/test_prompt_test_endpoint.py
--ignore=tests/unit/proxy/test_proxy_config_unit_test.py
--ignore=tests/unit/proxy/test_proxy_custom_auth.py
--ignore=tests/unit/proxy/test_proxy_reject_logging.py
--ignore=tests/unit/proxy/test_proxy_server.py
--ignore=tests/unit/proxy/test_proxy_setting_guardrails.py
--ignore=tests/unit/proxy/test_proxy_token_counter.py
--ignore=tests/unit/proxy/test_proxy_utils.py
--ignore=tests/unit/proxy/test_reducto_ocr_route.py
--ignore=tests/unit/proxy/test_response_polling_pre_call_checks.py
--ignore=tests/unit/proxy/test_server_root_path.py
--ignore=tests/unit/proxy/test_ui_path_detection.py
--ignore=tests/unit/proxy/test_unit_test_proxy_hooks.py
--ignore=tests/unit/proxy/test_update_spend.py
--ignore=tests/unit/proxy/test_zero_cost_model_budget_bypass.py
tests/unit/proxy/test_[_a-p]*.py
--deselect=tests/unit/proxy/test_aproxy_startup.py
--deselect=tests/unit/proxy/test_component_allowlists.py
--deselect=tests/unit/proxy/test_credential_slot_registry.py
--deselect=tests/unit/proxy/test_custom_callback_input.py
--deselect=tests/unit/proxy/test_custom_logger_s3_gcs.py
--deselect=tests/unit/proxy/test_custom_tokenizer_bug.py
--deselect=tests/unit/proxy/test_db_schema_changes.py
--deselect=tests/unit/proxy/test_deprecated_key_grace_period.py
--deselect=tests/unit/proxy/test_get_favicon.py
--deselect=tests/unit/proxy/test_get_image.py
--deselect=tests/unit/proxy/test_prisma_client_backoff_retry.py
--deselect=tests/unit/proxy/test_prometheus_metrics_server.py
--deselect=tests/unit/proxy/test_prompt_test_endpoint.py
--deselect=tests/unit/proxy/test_proxy_cli.py
--deselect=tests/unit/proxy/test_proxy_config_unit_test.py
--deselect=tests/unit/proxy/test_proxy_custom_auth.py
--deselect=tests/unit/proxy/test_proxy_reject_logging.py
--deselect=tests/unit/proxy/test_proxy_server.py
--deselect=tests/unit/proxy/test_proxy_setting_guardrails.py
--deselect=tests/unit/proxy/test_proxy_token_counter.py
--deselect=tests/unit/proxy/test_proxy_utils.py
python-version: "3.12"
workers: 4
reruns: 0
@ -662,6 +681,35 @@ jobs:
job-timeout-minutes: 60
dist: loadscope
- shard: proxy-infra-root-q-z
test-path: |-
tests/unit/proxy/test_[q-z]*.py
--deselect=tests/unit/proxy/test_reducto_ocr_route.py
--deselect=tests/unit/proxy/test_response_polling_pre_call_checks.py
--deselect=tests/unit/proxy/test_server_root_path.py
--deselect=tests/unit/proxy/test_ui_path_detection.py
--deselect=tests/unit/proxy/test_unit_test_proxy_hooks.py
--deselect=tests/unit/proxy/test_update_spend.py
--deselect=tests/unit/proxy/test_zero_cost_model_budget_bypass.py
python-version: "3.12"
workers: 4
reruns: 0
timeout-minutes: 20
job-timeout-minutes: 60
dist: loadscope
- shard: proxy-subprocess
test-path: |-
tests/unit/proxy/test_component_allowlists.py
tests/unit/proxy/test_prometheus_metrics_server.py
tests/unit/proxy/test_proxy_cli.py
python-version: "3.12"
workers: 2
reruns: 0
timeout-minutes: 20
job-timeout-minutes: 60
dist: loadscope
- shard: caching-local
test-path: >-
tests/unit/caching
@ -955,10 +1003,10 @@ jobs:
dist: loadscope
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
timeout-minutes: ${{ matrix.shard == 'db-and-spend' && 10 || 3 }}
timeout-minutes: 3
with:
persist-credentials: false
fetch-depth: ${{ matrix.shard == 'db-and-spend' && '0' || '1' }}
fetch-depth: 1
- name: Detect relevant changes
id: changes
@ -991,27 +1039,37 @@ jobs:
with:
artifact: ${{ needs.rust-bridge.outputs.artifact }}
- name: Install dependencies and check schema
- name: Install dependencies
if: steps.changes.outputs.decision != 'skip'
timeout-minutes: 8
timeout-minutes: 5
env:
RUST_BRIDGE_ARTIFACT: ${{ needs.rust-bridge.outputs.artifact }}
UNIT_SHARD: ${{ matrix.shard }}
SCHEMA_BASE_REF: origin/${{ github.event.repository.default_branch }}
run: |
diff -u model_prices_and_context_window.json litellm/model_prices_and_context_window_backup.json
if [ -z "$RUST_BRIDGE_ARTIFACT" ]; then
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra saml --extra caching --extra extra_proxy --extra proxy-runtime --extra utils
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra switchyard --extra saml --extra extra_proxy --extra proxy-runtime --extra utils
else
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra saml --extra caching --extra extra_proxy --extra proxy-runtime --extra utils --no-install-project
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra switchyard --extra saml --extra extra_proxy --extra proxy-runtime --extra utils --no-install-project --no-install-package litellm-proxy
uv pip install --no-deps --python .venv/bin/python rust-bridge-dist/*.whl
cp rust-bridge-dist/litellm/rust_bridge/_native.abi3.so litellm/rust_bridge/_native.abi3.so
uv run --no-sync python -c "import importlib.metadata; import litellm.rust_bridge._native; print(importlib.metadata.version('litellm'))"
fi
uv run --no-sync python -c 'import os, sys; print(sys.version); assert f"{sys.version_info.major}.{sys.version_info.minor}" == os.environ["UV_PYTHON"]'
if [ "$UNIT_SHARD" = "db-and-spend" ]; then
uv run --no-sync python scripts/check_schema_compatibility.py --base-ref "$SCHEMA_BASE_REF"
fi
- name: Check litellm/schema.prisma against the merge base
if: steps.changes.outputs.decision != 'skip' && matrix.shard == 'db-and-spend' && github.event_name == 'pull_request'
timeout-minutes: 3
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
base_ref=$(gh api "repos/${GITHUB_REPOSITORY}/branches/${EVENT_BASE_REF}" --jq .name) || base_ref=$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq .base.ref)
merge_base=$(gh api "repos/${GITHUB_REPOSITORY}/compare/${base_ref}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
test -n "$merge_base"
git fetch --no-tags --depth=1 origin "$merge_base"
uv run --no-sync python litellm/scripts/check_schema_compatibility.py --base-commit "$merge_base"
- name: Cache Prisma binaries
if: steps.changes.outputs.decision != 'skip'
@ -1076,7 +1134,7 @@ jobs:
--timeout="${TEST_TIMEOUT_SECONDS}" \
--rerun-except "from pytest-timeout" \
--durations=20 \
--cov=./litellm --cov=./enterprise/litellm_enterprise \
--cov=./litellm --cov=./litellm/enterprise/litellm_enterprise \
--cov-report=xml:coverage.xml \
--cov-config=pyproject.toml
status=$?
@ -1106,7 +1164,7 @@ jobs:
timeout-minutes: 20
defaults:
run:
working-directory: ui/litellm-dashboard
working-directory: litellm/ui/litellm-dashboard
steps:
- name: Checkout repository
@ -1125,9 +1183,9 @@ jobs:
if: steps.changes.outputs.decision != 'skip'
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version-file: ui/litellm-dashboard/.nvmrc
node-version-file: litellm/ui/litellm-dashboard/.nvmrc
cache: "npm"
cache-dependency-path: ui/litellm-dashboard/package-lock.json
cache-dependency-path: litellm/ui/litellm-dashboard/package-lock.json
- name: Install dependencies
if: steps.changes.outputs.decision != 'skip'
@ -1148,18 +1206,20 @@ jobs:
env:
CI: "true"
GH_TOKEN: ${{ github.token }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_NUMBER: ${{ github.event.pull_request.number }}
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
full_suite() { npm run test -- --run --pool forks --maxWorkers=14; }
if [ -z "$BASE_SHA" ]; then
if [ -z "$PR_NUMBER" ]; then
echo "Push to $GITHUB_REF_NAME: running the full suite"
full_suite
exit 0
fi
merge_base=$(gh api "repos/${{ github.repository }}/compare/${BASE_SHA}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
base_ref=$(gh api "repos/${GITHUB_REPOSITORY}/branches/${EVENT_BASE_REF}" --jq .name) || base_ref=$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq .base.ref)
merge_base=$(gh api "repos/${GITHUB_REPOSITORY}/compare/${base_ref}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
test -n "$merge_base"
git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA"
changed_files=()
@ -1246,7 +1306,7 @@ jobs:
- name: Install dependencies
if: steps.changes.outputs.decision != 'skip'
run: |
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra switchyard
- name: Cache Prisma binaries
if: steps.changes.outputs.decision != 'skip'
@ -1295,17 +1355,16 @@ jobs:
- name: Run unit tests
run: |
for chart in helm/litellm-helm helm/litellm; do
declared="$(grep -h '^suite:' "$chart"/tests/*.yaml | wc -l | tr -d '[:space:]')"
output="$(mktemp)"
helm unittest -f 'tests/*.yaml' "$chart" | tee "$output"
executed="$(sed -n 's/^Test Suites:.*[[:space:]]\([0-9][0-9]*\) total$/\1/p' "$output")"
if [ "$declared" != "$executed" ]; then
echo "::error::$chart declares $declared test suites but helm-unittest ran $executed. Suites are being skipped silently, so their assertions never execute."
exit 1
fi
echo "$chart: all $declared declared test suites ran"
done
chart=litellm/deploy/helm/litellm
declared="$(grep -h '^suite:' "$chart"/tests/*.yaml | wc -l | tr -d '[:space:]')"
output="$(mktemp)"
helm unittest -f 'tests/*.yaml' "$chart" | tee "$output"
executed="$(sed -n 's/^Test Suites:.*[[:space:]]\([0-9][0-9]*\) total$/\1/p' "$output")"
if [ "$declared" != "$executed" ]; then
echo "::error::$chart declares $declared test suites but helm-unittest ran $executed. Suites are being skipped silently, so their assertions never execute."
exit 1
fi
echo "$chart: all $declared declared test suites ran"
openai-sdk-compat:
name: OpenAI ${{ matrix.openai-version }} compatibility
needs: rust-bridge
@ -1449,9 +1508,9 @@ jobs:
RUST_BRIDGE_ARTIFACT: ${{ needs.rust-bridge.outputs.artifact }}
run: |
if [ -z "$RUST_BRIDGE_ARTIFACT" ]; then
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra saml --extra caching --extra extra_proxy --extra proxy-runtime --extra utils
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra switchyard --extra saml --extra extra_proxy --extra proxy-runtime --extra utils
else
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra saml --extra caching --extra extra_proxy --extra proxy-runtime --extra utils --no-install-project
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra switchyard --extra saml --extra extra_proxy --extra proxy-runtime --extra utils --no-install-project --no-install-package litellm-proxy
uv pip install --no-deps --python .venv/bin/python rust-bridge-dist/*.whl
cp rust-bridge-dist/litellm/rust_bridge/_native.abi3.so litellm/rust_bridge/_native.abi3.so
fi

View file

@ -8,13 +8,13 @@ on:
- main
- "litellm_**"
paths:
- "vscode-extension/**"
- "litellm/vscode-extension/**"
- ".github/workflows/test-vscode-extension.yml"
push:
branches:
- main
paths:
- "vscode-extension/**"
- "litellm/vscode-extension/**"
- ".github/workflows/test-vscode-extension.yml"
concurrency:
@ -27,7 +27,7 @@ jobs:
timeout-minutes: 10
defaults:
run:
working-directory: vscode-extension
working-directory: litellm/vscode-extension
steps:
- name: Checkout repository
@ -41,7 +41,7 @@ jobs:
with:
node-version: "24"
cache: npm
cache-dependency-path: vscode-extension/package-lock.json
cache-dependency-path: litellm/vscode-extension/package-lock.json
- name: Install dependencies
run: npm ci
@ -59,5 +59,5 @@ jobs:
uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
with:
name: litellm-vscode
path: vscode-extension/*.vsix
path: litellm/vscode-extension/*.vsix
if-no-files-found: error

View file

@ -1,85 +0,0 @@
name: "Weekly Load Anomaly Check"
on:
schedule:
- cron: "0 12 * * 6"
workflow_dispatch:
permissions:
contents: read
jobs:
weekly-load-anomaly:
if: github.event_name != 'schedule' || github.repository == 'BerriAI/litellm'
runs-on: ubuntu-latest
timeout-minutes: 45
services:
postgres:
image: postgres:16.6
env:
POSTGRES_USER: llmproxy
POSTGRES_PASSWORD: dbpassword9090
POSTGRES_DB: litellm
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U llmproxy"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
DATABASE_URL: postgresql://llmproxy:dbpassword9090@localhost:5432/litellm
LITELLM_MASTER_KEY: sk-weekly-anomaly-check
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
AWS_BEARER_TOKEN_BEDROCK: ${{ secrets.AWS_BEARER_TOKEN_BEDROCK }}
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Set up uv
uses: ./.github/actions/setup-uv-with-retries
with:
version: "0.10.9"
- name: Cache the Rust build
uses: ./.github/actions/cache-cargo-build
- name: Install dependencies
run: |
.github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra proxy
- name: Cache Prisma binaries
uses: ./.github/actions/cache-prisma-binaries
- name: Generate Prisma client
run: |
uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma
- name: Start the proxy
run: |
nohup uv run --no-sync litellm --config tests/e2e/perf/weekly_anomaly_config.yml --port 4000 > proxy.log 2>&1 &
for _ in $(seq 1 90); do
if curl -fs http://localhost:4000/health/liveliness > /dev/null; then
exit 0
fi
sleep 2
done
echo "proxy never became live"
tail -n 100 proxy.log
exit 1
- name: Run the weekly session anomaly test
env:
E2E_WEEKLY_ANOMALY: "1"
run: |
uv run --no-sync pytest tests/e2e/perf/test_weekly_session_anomaly_e2e.py -v --tb=short -rA
- name: Show proxy log on failure
if: failure()
run: tail -n 300 proxy.log

20
.gitignore vendored
View file

@ -18,7 +18,7 @@ __pycache__/
# Rust bridge build artifacts (compiled, platform-specific; regenerated by maturin/cargo)
litellm/rust_bridge/_native*.so
litellm/rust_bridge/_native*.pyd
litellm-rust/target/
litellm/rust/target/
# Python package build output
dist/
@ -55,14 +55,14 @@ hosted_config.yaml
litellm/proxy/tests/node_modules
litellm/proxy/tests/package.json
litellm/proxy/tests/package-lock.json
ui/litellm-dashboard/.next
ui/litellm-dashboard/node_modules
ui/litellm-dashboard/next-env.d.ts
litellm/ui/litellm-dashboard/.next
litellm/ui/litellm-dashboard/node_modules
litellm/ui/litellm-dashboard/next-env.d.ts
*.tsbuildinfo
ui/litellm-dashboard/package.json
ui/litellm-dashboard/package-lock.json
helm/litellm-helm/*.tgz
helm/*.tgz
litellm/ui/litellm-dashboard/package.json
litellm/ui/litellm-dashboard/package-lock.json
litellm/deploy/helm/litellm/charts/*.tgz
litellm/deploy/helm/*.tgz
litellm/proxy/vertex_key.json
**/.vim/
**/node_modules
@ -106,7 +106,7 @@ litellm_config.yaml
litellm/proxy/to_delete_loadtest_work/*
update_model_cost_map.py
tests/unit/proxy/mcp_server/test_mcp_server_manager.py
scripts/test_vertex_ai_search.py
litellm/scripts/test_vertex_ai_search.py
LAZY_LOADING_IMPROVEMENTS.md
STABILIZATION_TODO.md
**/test-results
@ -146,7 +146,7 @@ crash.*.log
# pytest coverage data
.coverage
ui/litellm-dashboard/out/
litellm/ui/litellm-dashboard/out/
litellm.log
.coverage-rust

View file

@ -1,13 +0,0 @@
# Wolfi's security database names zlib 1.3.3-r0 as the fix for CVE-2026-85091,
# but the newest zlib published to the Wolfi apk repo is 1.3.2-r7, so every
# wolfi-base digest reports it and no `apk upgrade` can clear it.
# Drop this once Wolfi ships zlib >= 1.3.3-r0; expected by 2026-10-15.
ignore:
- vulnerability: CVE-2026-85091
package:
name: zlib
type: apk
- vulnerability: GHSA-g5fp-32jq-cfw2
package:
name: zlib
type: apk

5
.npmrc
View file

@ -1,5 +0,0 @@
# Supply-chain hardening
# Packages needing lifecycle scripts: npm rebuild <pkg>
ignore-scripts=true
# Protects local npm install only — npm ci (used in CI) ignores this
min-release-age=3

View file

@ -1,22 +0,0 @@
# Custom Semgrep rules for LiteLLM
Add custom rule YAML files here. Semgrep loads all `.yml`/`.yaml` files under this directory.
**Run only custom rules (CI / fail on findings):**
```bash
semgrep scan --config .semgrep/rules . --error
```
**Run with registry + custom rules:**
```bash
semgrep scan --config auto --config .semgrep/rules .
```
**Layout:**
- `python/` – Python-specific rules (security, patterns)
- Add more subdirs as needed (e.g. `generic/` for language-agnostic rules)
See [Semgrep rule syntax](https://semgrep.dev/docs/writing-rules/rule-syntax/).

View file

@ -1,7 +1,6 @@
Do not write comments unless they are any of:
- absolutely necessary to explain some very complex business logic (in which case, keep it concise and clear)
- used as an input for tools to read and act on. For example:
- entries in `.git-blame-ignore-revs` saying which commit is excluded from git blame
- a lint or type checker suppression like `# mutable-ok` or `# pyright: ignore[reportArgumentType] # <reason>` when introducing a truly unavoidable violation
- a TODO or FIXME
- Not great to have those, but if it's unavoidable, make sure to include a strong, concise reason for why it's there or, better yet, link to a GitHub issue for the follow-up work
@ -31,7 +30,7 @@ A test must only fail when litellm code changes. Never pin facts we don't own (a
End-to-end tests belong in `tests/e2e/`. A rule about one directory's files, like its harness or which CI runs it, goes in that directory's `AGENTS.md`, never only in a README or here. Read a directory's `AGENTS.md` before editing files in it
When creating PRs, target the repository's current default branch for both internal and external / OSS contributions. Check it with `python3 scripts/default_branch.py --branch` instead of assuming a branch name or relying on cached `origin/HEAD`
When creating PRs, target the repository's current default branch for both internal and external / OSS contributions. Check it with `python3 litellm/scripts/default_branch.py --branch` instead of assuming a branch name or relying on cached `origin/HEAD`
When writing a PR body, treat the comments and imperative instructions inside .github/PULL_REQUEST_TEMPLATE/general.md as rules to follow, not just layout. Agent harnesses may strip HTML comments from copies of that file injected into context, so read .github/PULL_REQUEST_TEMPLATE/general.md from disk before writing a PR body to make sure you see every comment rule. A section you have nothing to put in (Relevant issues, Affected release, Linear ticket, Caveats, QA runbook, and so on) is removed entirely, heading included, never left as an empty title
@ -39,14 +38,17 @@ Same applies for filing bug reports and feature requests, with .github/ISSUE_TEM
If you're resolving a linear ticket, in the "## Linear ticket" section of the PR, say "Resolves LIT-1234", replacing "LIT-1234" with the actual ticket id that you're resolving. If you don't have the ticket id, don't make one up or search for it. Just drop the section
Never use `pytest` commands or the like as "Screenshots / Proof of Fix". We prefer curl'ing a live proxy instance running on localhost:4000 (I like to run it with `python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --detailed_debug --reload --use_v2_migration_resolver 2>&1 | tee litellm.log`. The Admin UI dev server is `npm run dev` in `ui/litellm-dashboard`, served on port 3000) and showing both the command run and the output. Also, it should hit real LLM provider APIs, not mocks, and cost real $$$ because that is the most realistic test. The proof of fix should be exactly what the end user / customer would see / do. The run logs in PR #27703 is a prime example of how to do it (not a huge fan of using a python test script that future me and the team will have no visibility into. I prefer just curl commands or a short list of bash commands (e.g., using `for`)). If it's a UI thing, or the main use case runs through a headful agentic coding tool like Claude Code or Codex, drive that surface yourself and embed your own before and after screenshots of it in the PR (the Admin UI page, or what the coding tool shows), next to an ordered list of the URLs to go to (e.g., http://localhost:4000/ui/?page=logs), where to click, and what fields to fill out so a reviewer can reproduce it
Run QA proxies with `python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --num_workers 2 --detailed_debug --reload --use_v2_migration_resolver 2>&1 | tee litellm.log`, and swap its `--config` for a copy of `litellm/proxy/dev_config.yaml` that points both `router_settings` (`redis_host`, `redis_port`) and `general_settings.coordination_redis` (`host`, `port`) at one dedicated Redis instance, with credentials when required, so the workers share router state and spend, budget, and rate-limit counters. Check that two worker PIDs are serving, and use the same worker count and Redis setup for the before and after proof. When the PR is about `--run_gunicorn`, `--run_hypercorn`, or `--run_granian`, which `--reload` does not support, drop `--reload`. When it is about behavior without Redis, drop Redis. When the proxy prints `"workers" flag is ignored when reloading is enabled`, that checkout or its uvicorn reloads only one worker, so drop the two-worker check for it. Say in the proof which part you dropped and why
Simple PRs may skip before/after e2e when the full diff is mechanical and focused checks completely prove it, such as #49030 restoring an existing helper name. Trace affected callers and run relevant tests/static checks at the tip. For a bug fix, establish that the regression catches the original defect. Size alone never qualifies. New behavior, API/provider semantics, auth, billing, persistence, concurrency or uncertain integration risk need live e2e. In Screenshots / Proof of Fix, record `E2E exception at <tip SHA>`, scope, rationale, commands and results. Tests count as proof only for this exception. Reassess changed diffs, retain CI/bot gates, and honor explicit e2e requests (2026-10-28, unnecessary live QA for #49030)
For other PRs, proof must show the end user's real flow before and after, with commands and output against a live proxy and real provider calls where applicable. The body alone must let a reviewer rerun it, with the proxy command, config, env var names and client code exactly as run, and long blocks collapsed in <details>. Pytest alone is insufficient. For UI or interactive coding-tool flows, drive that surface and embed before/after screenshots, with URLs, clicks and input fields a reviewer can reproduce
If you ever write any human-facing text (pull requests, issues, commit messages, discussion posts, github comments, release notes, docs, etc.), always follow these guidelines to sound less AI-y:
- don't use emojis
- don't use "—". Don't use ";" or ":" unless it would be nonsensical not to. Instead, reach for ",", ".", conjunction words, etc. in descending order of preference: vary among them, weighted toward the front of the list, and skip "," where it would cause a comma splice or the sentence is getting long. Overusing any one of them also feels AI-y. A word cap does not penalize you for adding more sentences
- don't use the pattern "It's not X, it's Y", "You're not X, you're Y", etc.
- unless explicitly asked, don't use bulleted or numbered lists unless it would be nonsensical not to. Instead, prefer prose
- don't add a trailing "." at the end of paragraphs (just like this file). That means every paragraph, not just the last one (of the markdown file, PR description, GitHub comment, etc.). Rule of thumb: if you're adding new line(s) before the next sentence, don't add a ".". Exception is that docs (i.e., litellm-docs) should have trailing "."
- don't use →. Instead, prefer not to use arrows, and if need be, use -> instead
- use plain, simple, everyday engineering language: the common phrase engineers actually say over rare compact phrasing, in grammatically complete sentences. When explicitly asked to use bullets or ordered lists and structure legitimately helps the reader, prefer nested bullets (any depth is fine) over dense lines in a flat structure
@ -54,17 +56,17 @@ Don't hesitate to use values in .env to get needed API keys and other secrets, a
Python max line length is 120, not 88
The five lint gates (`scripts/ruff_strict_gate.py`, `scripts/type_discipline_gate.py`, `scripts/test_quality_gate.py`, `scripts/openapi_docs_gate.py`, `scripts/type_check_gate.py`) compare each rule's codebase count on your branch against the count at its merge-base with the default branch, and a rule may not grow. There are no budget files to edit or ratchet: when a gate fails, fix the violations the branch introduced or remove at least as many of that rule elsewhere in the tree. The one exception is `reportAny` / `reportExplicitAny`, which share a fixed codebase-wide cap in `ANY_CAPS` in `scripts/type_check_gate.py` because Any spreads past the lines you touch. A branch may add Anys while the total stays under that cap. Only lower `ANY_CAPS`, and only in its own PR to the default branch, never raise it on a feature branch
The five lint gates (`litellm/scripts/ruff_strict_gate.py`, `litellm/scripts/type_discipline_gate.py`, `litellm/scripts/test_quality_gate.py`, `litellm/scripts/openapi_docs_gate.py`, `litellm/scripts/type_check_gate.py`) compare each rule's codebase count on your branch against the count at its merge-base with the default branch, and a rule may not grow. There are no budget files to edit or ratchet: when a gate fails, fix the violations the branch introduced or remove at least as many of that rule elsewhere in the tree. The one exception is `reportAny` / `reportExplicitAny`, which share a fixed codebase-wide cap in `ANY_CAPS` in `litellm/scripts/type_check_gate.py` because Any spreads past the lines you touch. A branch may add Anys while the total stays under that cap. Only lower `ANY_CAPS`, and only in its own PR to the default branch, never raise it on a feature branch
`make check` (f.k.a. `make pre-commit`, which still works identically as an alias) saves its complete output to a log file in .git (overwriting previous logs) and prints that path as its first and last output lines. To inspect a run, read or grep that log instead of re-running the multi-minute checks just to see a different slice
`make check`, `make lint`, `scripts/pre_commit_lint.sh`, and the standalone lint gates (`scripts/ruff_strict_gate.py`, `scripts/type_discipline_gate.py`, `scripts/type_check_gate.py`) each hold one of 2 machine-wide slots, so when other sessions or worktrees on the same box are already running heavy work, yours prints "all N machine-wide slots are busy" followed by "queueing" and then stays quiet until a slot frees. Give the command a long timeout and let it wait rather than killing it, retrying it, or assuming it hung. Don't change the # of machine-wide slots or make it unlimited by setting `LITELLM_GATE_SLOTS=0`
`make check`, `make lint`, `litellm/scripts/pre_commit_lint.sh`, and the standalone lint gates (`litellm/scripts/ruff_strict_gate.py`, `litellm/scripts/type_discipline_gate.py`, `litellm/scripts/type_check_gate.py`) each hold one of 2 machine-wide slots, so when other sessions or worktrees on the same box are already running heavy work, yours prints "all N machine-wide slots are busy" followed by "queueing" and then stays quiet until a slot frees. Give the command a long timeout and let it wait rather than killing it, retrying it, or assuming it hung. Don't change the # of machine-wide slots or make it unlimited by setting `LITELLM_GATE_SLOTS=0`
If you're trying to create a new function that relies on untyped stuff, instead of adding more Any's and pushing `reportAny` / `reportExplicitAny` closer to their cap in `ANY_CAPS`, just validate it in the caller with Pydantic (a model or `TypeAdapter` that returns the typed thing or raises will do) and then pass the now typed variable in
If you get an LIT001 fail, refactor the code to follow functional programming best practices rather than introducing mutable data structures. For example, build values in one shot with comprehensions or generators wrapped in `tuple()` / `MappingProxyType()` / `frozenset()` instead of seeding an empty `list`/`dict`/`set` and mutating it over time. Ideally, `# mutable-ok` is never used. Reach for it only as a genuine last resort when an immutable rewrite is truly impossible, and always pair it with a real reason
Every lint or type suppression must name the exact rule inside brackets and carry a reason comment, e.g. `# pyright: ignore[reportArgumentType] # stubs lack async overload` or `# noqa: TID251 # <reason>`. `# type: ignore` is banned (LIT009): pyrightconfig.json sets `enableTypeIgnoreComments` to false, so it silently does nothing
Every lint or type suppression must name the exact rule inside brackets and carry a reason comment, e.g. `# pyright: ignore[reportArgumentType] # stubs lack async overload` or `# noqa: TID251 # <reason>`. `# type: ignore` is banned (LIT009): `[tool.pyright]` in pyproject.toml sets `enableTypeIgnoreComments` to false, so it silently does nothing
Commit and push your work when you're done without asking
@ -105,6 +107,7 @@ Follow these coding conventions for new/updated code (a three-line fix in a lega
- No file sprawl: deliberate file and folder structure
- Standard over hand-rolled: use the official SDK or a library where one exists. Where none does, follow industry standards instead of inventing local conventions
- API-fragmentation-aware: when logic must branch on which API surface produced or consumes data (e.g. chat completions vs Anthropic Messages vs Responses API shapes), proactively look for an existing shared helper (e.g. `litellm_core_utils/prompt_templates/factory.py`) before writing per-surface parsing in the new module. If none exists, add one there instead of duplicating the same format-detection logic in every new guardrail/integration
- Current default for new model names: logic that picks behavior from a model-name table (tokenizer, message framing, supported params) gives a provider's model the table does not list yet that provider's current behavior, never its oldest, since providers ship names faster than any table is updated
Follow conventional commits for commit names and PR titles

View file

@ -1,417 +0,0 @@
# LiteLLM Architecture - LiteLLM SDK + AI Gateway
This document helps contributors understand where to make changes in LiteLLM.
---
## How It Works
The LiteLLM AI Gateway (Proxy) uses the LiteLLM SDK internally for all LLM calls:
```
OpenAI SDK (client) ──▶ LiteLLM AI Gateway (proxy/) ──▶ LiteLLM SDK (litellm/) ──▶ LLM API
Anthropic SDK (client) ──▶ LiteLLMAI Gateway (proxy/) ──▶ LiteLLM SDK (litellm/) ──▶ LLM API
Any HTTP client ──▶ LiteLLMAI Gateway (proxy/) ──▶ LiteLLM SDK (litellm/) ──▶ LLM API
```
The **AI Gateway** adds authentication, rate limiting, budgets, and routing on top of the SDK.
The **SDK** handles the actual LLM provider calls, request/response transformations, and streaming.
---
## 1. AI Gateway (Proxy) Request Flow
The AI Gateway (`litellm/proxy/`) wraps the SDK with authentication, rate limiting, and management features.
```mermaid
sequenceDiagram
participant Client
participant ProxyServer as proxy/proxy_server.py
participant Auth as proxy/auth/user_api_key_auth.py
participant Redis as Redis Cache
participant Hooks as proxy/hooks/
participant Router as router.py
participant Main as main.py + utils.py
participant Handler as llms/custom_httpx/llm_http_handler.py
participant Transform as llms/{provider}/chat/transformation.py
participant Provider as LLM Provider API
participant CostCalc as cost_calculator.py
participant LoggingObj as litellm_logging.py
participant DBWriter as db/db_spend_update_writer.py
participant Postgres as PostgreSQL
%% Request Flow
Client->>ProxyServer: POST /v1/chat/completions
ProxyServer->>Auth: user_api_key_auth()
Auth->>Redis: Check API key cache
Redis-->>Auth: Key info + spend limits
ProxyServer->>Hooks: parallel_request_limiter, cache_control_check
Hooks->>Redis: Check/increment rate limit counters
ProxyServer->>Router: route_request()
Router->>Main: litellm.acompletion()
Main->>Handler: BaseLLMHTTPHandler.completion()
Handler->>Transform: ProviderConfig.transform_request()
Handler->>Provider: HTTP Request
Provider-->>Handler: Response
Handler->>Transform: ProviderConfig.transform_response()
Transform-->>Handler: ModelResponse
Handler-->>Main: ModelResponse
%% Cost Attribution (in utils.py wrapper)
Main->>LoggingObj: update_response_metadata()
LoggingObj->>CostCalc: _response_cost_calculator()
CostCalc->>CostCalc: completion_cost(tokens × price)
CostCalc-->>LoggingObj: response_cost
LoggingObj-->>Main: Set response._hidden_params["response_cost"]
Main-->>ProxyServer: ModelResponse (with cost in _hidden_params)
%% Response Headers + Async Logging
ProxyServer->>ProxyServer: Extract cost from hidden_params
ProxyServer->>LoggingObj: async_success_handler()
LoggingObj->>Hooks: async_log_success_event()
Hooks->>DBWriter: update_database(response_cost)
DBWriter->>Redis: Queue spend increment
DBWriter->>Postgres: Batch write spend logs (async)
ProxyServer-->>Client: ModelResponse + x-litellm-response-cost header
```
### Proxy Components
```mermaid
graph TD
subgraph "Incoming Request"
Client["POST /v1/chat/completions"]
end
subgraph "proxy/proxy_server.py"
Endpoint["chat_completion()"]
end
subgraph "proxy/auth/"
Auth["user_api_key_auth()"]
end
subgraph "proxy/"
PreCall["litellm_pre_call_utils.py"]
RouteRequest["route_llm_request.py"]
end
subgraph "litellm/"
Router["router.py"]
Main["main.py"]
end
subgraph "Infrastructure"
DualCache["DualCache<br/>(in-memory + Redis)"]
Postgres["PostgreSQL<br/>(keys, teams, spend logs)"]
end
Client --> Endpoint
Endpoint --> Auth
Auth --> DualCache
DualCache -.->|cache miss| Postgres
Auth --> PreCall
PreCall --> RouteRequest
RouteRequest --> Router
Router --> DualCache
Router --> Main
Main --> Client
```
**Key proxy files:**
- `proxy/proxy_server.py` - Main API endpoints
- `proxy/auth/` - Authentication (API keys, JWT, OAuth2)
- `proxy/hooks/` - Proxy-level callbacks
- `router.py` - Load balancing, fallbacks
- `router_strategy/` - Routing algorithms (`lowest_latency.py`, `simple_shuffle.py`, etc.)
**LLM-specific proxy endpoints:**
| Endpoint | Directory | Purpose |
|----------|-----------|---------|
| `/v1/messages` | `proxy/anthropic_endpoints/` | Anthropic Messages API |
| `/vertex-ai/*` | `proxy/vertex_ai_endpoints/` | Vertex AI passthrough |
| `/gemini/*` | `proxy/google_endpoints/` | Google AI Studio passthrough |
| `/v1/images/*` | `proxy/image_endpoints/` | Image generation |
| `/v1/batches` | `proxy/batches_endpoints/` | Batch processing |
| `/v1/files` | `proxy/openai_files_endpoints/` | File uploads |
| `/v1/fine_tuning` | `proxy/fine_tuning_endpoints/` | Fine-tuning jobs |
| `/v1/rerank` | `proxy/rerank_endpoints/` | Reranking |
| `/v1/responses` | `proxy/response_api_endpoints/` | OpenAI Responses API |
| `/v1/vector_stores` | `proxy/vector_store_endpoints/` | Vector stores |
| `/*` (passthrough) | `proxy/pass_through_endpoints/` | Direct provider passthrough |
**Proxy Hooks** (`proxy/hooks/__init__.py`):
| Hook | File | Purpose |
|------|------|---------|
| `parallel_request_limiter` | `proxy/hooks/parallel_request_limiter_v3.py` | Rate limiting per key/user |
| `cache_control_check` | `proxy/hooks/cache_control_check.py` | Cache validation |
| `responses_id_security` | `proxy/hooks/responses_id_security.py` | Response ID validation |
| `litellm_skills` | `proxy/hooks/litellm_skills/main.py` | Skills injection |
To add a new proxy hook, implement `CustomLogger` and register in `PROXY_HOOKS`.
### Infrastructure Components
The AI Gateway uses external infrastructure for persistence and caching:
```mermaid
graph LR
subgraph "AI Gateway (proxy/)"
Proxy["proxy_server.py"]
Auth["auth/user_api_key_auth.py"]
DBWriter["db/db_spend_update_writer.py<br/>DBSpendUpdateWriter"]
InternalCache["utils.py<br/>InternalUsageCache"]
CostCallback["hooks/proxy_track_cost_callback.py<br/>_ProxyDBLogger"]
Scheduler["APScheduler<br/>ProxyStartupEvent"]
end
subgraph "SDK (litellm/)"
Router["router.py<br/>Router.cache (DualCache)"]
LLMCache["caching/caching_handler.py<br/>LLMCachingHandler"]
CacheClass["caching/caching.py<br/>Cache"]
end
subgraph "Redis (caching/redis_cache.py)"
RateLimit["Rate Limit Counters"]
SpendQueue["Spend Increment Queue"]
KeyCache["API Key Cache"]
TPM_RPM["TPM/RPM Tracking"]
Cooldowns["Deployment Cooldowns"]
LLMResponseCache["LLM Response Cache"]
end
subgraph "PostgreSQL (proxy/schema.prisma)"
Keys["LiteLLM_VerificationToken"]
Teams["LiteLLM_TeamTable"]
SpendLogs["LiteLLM_SpendLogs"]
Users["LiteLLM_UserTable"]
end
Auth --> InternalCache
InternalCache --> KeyCache
InternalCache -.->|cache miss| Keys
InternalCache --> RateLimit
Router --> TPM_RPM
Router --> Cooldowns
LLMCache --> CacheClass
CacheClass --> LLMResponseCache
CostCallback --> DBWriter
DBWriter --> SpendQueue
DBWriter --> SpendLogs
Scheduler --> SpendLogs
Scheduler --> Keys
```
| Component | Purpose | Key Files/Classes |
|-----------|---------|-------------------|
| **Redis** | Rate limiting, API key caching, TPM/RPM tracking, cooldowns, LLM response caching, spend queuing | `caching/redis_cache.py` (`RedisCache`), `caching/dual_cache.py` (`DualCache`) |
| **PostgreSQL** | API keys, teams, users, spend logs | `proxy/utils.py` (`PrismaClient`), `proxy/schema.prisma` |
| **InternalUsageCache** | Proxy-level cache for rate limits + API keys (in-memory + Redis) | `proxy/utils.py` (`InternalUsageCache`) |
| **Router.cache** | TPM/RPM tracking, deployment cooldowns, client caching (in-memory + Redis) | `router.py` (`Router.cache: DualCache`) |
| **LLMCachingHandler** | SDK-level LLM response/embedding caching | `caching/caching_handler.py` (`LLMCachingHandler`), `caching/caching.py` (`Cache`) |
| **DBSpendUpdateWriter** | Batches spend updates to reduce DB writes | `proxy/db/db_spend_update_writer.py` (`DBSpendUpdateWriter`) |
| **Cost Tracking** | Calculates and logs response costs | `proxy/hooks/proxy_track_cost_callback.py` (`_ProxyDBLogger`) |
**Background Jobs** (APScheduler, initialized in `proxy/proxy_server.py` → `ProxyStartupEvent.initialize_scheduled_background_jobs()`):
| Job | Interval | Purpose | Key Files |
|-----|----------|---------|-----------|
| `update_spend` | 60s | Batch write spend logs to PostgreSQL | `proxy/db/db_spend_update_writer.py` |
| `reset_budget` | 10-12min | Reset budgets for keys/users/teams | `proxy/common_utils/reset_budget_job.py` |
| `add_deployment` | 10s | Sync new model deployments from DB | `proxy/proxy_server.py` (`ProxyConfig`) |
| `cleanup_old_spend_logs` | cron/interval | Delete old spend logs | `proxy/db/db_transaction_queue/spend_log_cleanup.py` |
| `check_batch_cost` | 30min | Calculate costs for batch jobs | `enterprise/litellm_enterprise/proxy/common_utils/check_batch_cost.py` |
| `check_responses_cost` | 30min | Calculate costs for responses API | `enterprise/litellm_enterprise/proxy/common_utils/check_responses_cost.py` |
| `process_rotations` | 1hr | Auto-rotate API keys | `proxy/common_utils/key_rotation_manager.py` |
| `_run_background_health_check` | continuous | Health check model deployments | `proxy/proxy_server.py` |
| `send_weekly_spend_report` | weekly | Slack spend alerts | `proxy/utils.py` (`SlackAlerting`) |
| `send_monthly_spend_report` | monthly | Slack spend alerts | `proxy/utils.py` (`SlackAlerting`) |
**Cost Attribution Flow:**
1. LLM response returns to `utils.py` wrapper after `litellm.acompletion()` completes
2. `update_response_metadata()` (`litellm_core_utils/llm_response_utils/response_metadata.py`) is called
3. `logging_obj._response_cost_calculator()` (`litellm_core_utils/litellm_logging.py`) calculates cost via `litellm.completion_cost()` (`cost_calculator.py`)
4. Cost is stored in `response._hidden_params["response_cost"]`
5. `proxy/common_request_processing.py` extracts cost from `hidden_params` and adds to response headers (`x-litellm-response-cost`)
6. `logging_obj.async_success_handler()` triggers callbacks including `_ProxyDBLogger.async_log_success_event()`
7. `DBSpendUpdateWriter.update_database()` queues spend increments to Redis
8. Background job `update_spend` flushes queued spend to PostgreSQL every 60s
Cost lookup falls back to the response model when the provider-reported name cannot be priced. Explicitly registered zero rates remain zero, and custom deployment pricing or an explicit `base_model` retain precedence
### Data Access Layer (Models & Repositories)
Database entities and the operations on them live in two packages at the root of `litellm/` so both the gateway (`proxy/`) and the SDK can use them without importing proxy internals:
- `litellm/models/` holds the canonical Pydantic definitions for every persisted entity (`LiteLLM_VerificationToken`, `LiteLLM_TeamTable`, `LiteLLM_UserTable`, etc.). `proxy/_types.py` re-exports these for backwards compatibility, so existing imports keep working.
- `litellm/repositories/` holds the data-access layer. `BaseRepository[T]` provides the generic CRUD (`find_by_id`, `find_many`, `create`, `update`, `delete`, `count`, `exists`); entity repositories such as `VerificationTokenRepository`, `TeamRepository`, and `UserRepository` add domain-specific queries and writes on top of it.
Conventions to follow when touching this layer:
| Concern | How it's handled |
|---------|------------------|
| JSON columns | Prisma `Json` columns are stored as JSON strings. Repositories `json.dumps()` on write and `json.loads()` on read (see `_to_model` and the `_build_*_data` helpers). |
| Archive-then-delete | `delete_team` / `delete_token` copy the row into the `LiteLLM_Deleted*` table and delete the original inside a single `prisma_client.db.tx()` transaction. Archive payloads are built explicitly so only columns that exist on the archive table are written. |
| Column vs. field names | Where a model field differs from its DB column (for example `org_id` maps to the `organization_id` column), the repository translates in both directions rather than relying on Pydantic to guess. |
| Array mutations | Adds use Prisma's atomic `push` (`add_member`, `add_admin`, `add_models`) to avoid read-modify-write races. Removals fall back to read-modify-write because Prisma has no atomic array remove. |
To add a new entity, define the model under `litellm/models/`, re-export it from `proxy/_types.py` if existing code imports it from there, and add a repository under `litellm/repositories/` (subclass `BaseRepository` for plain CRUD, or add bespoke methods when the entity needs encryption, archiving, or atomic array updates). Mirror the tests in `tests/unit/repositories/`.
---
## 2. SDK Request Flow
The SDK (`litellm/`) provides the core LLM calling functionality used by both direct SDK users and the AI Gateway.
```mermaid
graph TD
subgraph "SDK Entry Points"
Completion["litellm.completion()"]
Messages["litellm.messages()"]
end
subgraph "main.py"
Main["completion()<br/>acompletion()"]
end
subgraph "utils.py"
GetProvider["get_llm_provider()"]
end
subgraph "llms/custom_httpx/"
Handler["llm_http_handler.py<br/>BaseLLMHTTPHandler"]
HTTP["http_handler.py<br/>HTTPHandler / AsyncHTTPHandler"]
end
subgraph "llms/{provider}/chat/"
TransformReq["transform_request()"]
TransformResp["transform_response()"]
end
subgraph "litellm_core_utils/"
Streaming["streaming_handler.py"]
end
subgraph "integrations/ (async, off main thread)"
Callbacks["custom_logger.py<br/>Langfuse, Datadog, etc."]
end
Completion --> Main
Messages --> Main
Main --> GetProvider
GetProvider --> Handler
Handler --> TransformReq
TransformReq --> HTTP
HTTP --> Provider["LLM Provider API"]
Provider --> HTTP
HTTP --> TransformResp
TransformResp --> Streaming
Streaming --> Response["ModelResponse"]
Response -.->|async| Callbacks
```
**Key SDK files:**
- `main.py` - Entry points: `completion()`, `acompletion()`, `embedding()`
- `utils.py` - `get_llm_provider()` resolves model → provider
- `llms/custom_httpx/llm_http_handler.py` - Central HTTP orchestrator
- `llms/custom_httpx/http_handler.py` - Low-level HTTP client
- `llms/{provider}/chat/transformation.py` - Provider-specific transformations
- `litellm_core_utils/streaming_handler.py` - Streaming response handling
- `integrations/` - Async callbacks (Langfuse, Datadog, etc.)
---
## 3. Translation Layer
When a request comes in, it goes through a **translation layer** that converts between API formats.
Each translation is isolated in its own file, making it easy to test and modify independently.
### Where to find translations
| Incoming API | Provider | Translation File |
|--------------|----------|------------------|
| `/v1/chat/completions` | Anthropic | `llms/anthropic/chat/transformation.py` |
| `/v1/chat/completions` | Bedrock Converse | `llms/bedrock/chat/converse_transformation.py` |
| `/v1/chat/completions` | Bedrock Invoke | `llms/bedrock/chat/invoke_transformations/anthropic_claude_transformation.py` |
| `/v1/chat/completions` | Gemini | `llms/gemini/chat/transformation.py` |
| `/v1/chat/completions` | Vertex AI | `llms/vertex_ai/gemini/transformation.py` |
| `/v1/chat/completions` | OpenAI | `llms/openai/chat/gpt_transformation.py` |
| `/v1/messages` (passthrough) | Anthropic | `llms/anthropic/messages/native/transformation.py` |
| `/v1/messages` (passthrough) | Bedrock | `llms/bedrock/messages/invoke_transformations/anthropic_claude_transformation.py` |
| `/v1/messages` (passthrough) | Vertex AI | `llms/vertex_ai/vertex_ai_partner_models/anthropic/messages/transformation.py` |
| Passthrough endpoints | All | `proxy/pass_through_endpoints/llm_provider_handlers/` |
### Example: Debugging prompt caching
If `/v1/messages` → Bedrock Converse prompt caching isn't working but Bedrock Invoke works:
1. **Bedrock Converse translation**: `llms/bedrock/chat/converse_transformation.py`
2. **Bedrock Invoke translation**: `llms/bedrock/chat/invoke_transformations/anthropic_claude_transformation.py`
3. Compare how each handles `cache_control` in `transform_request()`
### How translations work
Each provider has a `Config` class that inherits from `BaseConfig` (`llms/base_llm/chat/transformation.py`):
```python
class ProviderConfig(BaseConfig):
def transform_request(self, model, messages, optional_params, litellm_params, headers):
# Convert OpenAI format → Provider format
return {"messages": transformed_messages, ...}
def transform_response(self, model, raw_response, model_response, logging_obj, ...):
# Convert Provider format → OpenAI format
return ModelResponse(choices=[...], usage=Usage(...))
```
The `BaseLLMHTTPHandler` (`llms/custom_httpx/llm_http_handler.py`) calls these methods - you never need to modify the handler itself.
---
## 4. Adding/Modifying Providers
### To add a new provider:
1. Create `llms/{provider}/chat/transformation.py`
2. Implement `Config` class with `transform_request()` and `transform_response()`
3. Add tests in `tests/llm_translation/test_{provider}.py`
### To add a feature (e.g., prompt caching):
1. Find the translation file from the table above
2. Modify `transform_request()` to handle the new parameter
3. Add unit tests that verify the transformation
### Testing checklist
When adding a feature, verify it works across all paths:
| Test | File Pattern |
|------|--------------|
| OpenAI passthrough | `tests/llm_translation/test_openai*.py` |
| Anthropic direct | `tests/llm_translation/test_anthropic*.py` |
| Bedrock Invoke | `tests/llm_translation/test_bedrock*.py` |
| Bedrock Converse | `tests/llm_translation/test_bedrock*converse*.py` |
| Vertex AI | `tests/llm_translation/test_vertex*.py` |
| Gemini | `tests/llm_translation/test_gemini*.py` |
### Unit testing translations
Translations are designed to be unit testable without making API calls:
```python
from litellm.llms.bedrock.chat.converse_transformation import BedrockConverseConfig
def test_prompt_caching_transform():
config = BedrockConverseConfig()
result = config.transform_request(
model="anthropic.claude-3-opus",
messages=[{"role": "user", "content": "test", "cache_control": {"type": "ephemeral"}}],
optional_params={},
litellm_params={},
headers={}
)
assert "cachePoint" in str(result) # Verify cache_control was translated
```

View file

@ -1,352 +0,0 @@
# Contributing to LiteLLM
Thank you for your interest in contributing to LiteLLM! We welcome contributions of all kinds - from bug fixes and documentation improvements to new features and integrations.
## **Checklist before submitting a PR**
Here are the core requirements for any PR submitted to LiteLLM:
- [ ] **Sign the Contributor License Agreement (CLA)** - [see details](#contributor-license-agreement-cla)
- [ ] **Keep scope isolated** - Your changes should address 1 specific problem at a time
#### Proxy (Backend) PRs
- [ ] **Add testing** - Adding at least 1 test is a hard requirement - [see details](#adding-testing)
- [ ] **Ensure your PR passes all checks**:
- [ ] [Linting / Formatting](#running-linting-and-formatting-checks) - `make lint`
- [ ] [The tests covering your change](#running-unit-tests) pass, e.g. `uv run pytest tests/unit/<your_test_file>.py -v`. CI runs the full unit test matrix, so you don't need to run the whole suite locally
#### UI PRs
- [ ] **Ensure the UI builds successfully** - `npm run build`
- [ ] **Ensure all UI unit tests pass** - `npm run test`
- [ ] **Add tests for new components or logic** - If you are adding a new component or new logic, add corresponding tests
## **Contributor License Agreement (CLA)**
Before contributing code to LiteLLM, you must sign our [Contributor License Agreement (CLA)](https://cla-assistant.io/BerriAI/litellm). This is a legal requirement for all contributions to be merged into the main repository.
**Important:** We strongly recommend reviewing and signing the CLA before starting work on your contribution to avoid any delays in the PR process.
## Quick Start
### 1. Setup Your Local Development Environment
```bash
# Fork the repository on GitHub (click the Fork button at https://github.com/BerriAI/litellm)
# Then clone your fork locally
git clone https://github.com/YOUR_USERNAME/litellm.git
cd litellm
# Create a new branch for your feature (see "Commit and Branch Conventions" below)
git checkout -b litellm_your_feature
# Install development dependencies
make install-dev
# Install git hooks that enforce commit conventions (one-time, opt-in)
make install-hooks
# Verify your setup works
make help
```
That's it! Your local development environment is ready.
## Commit and Branch Conventions
Commits follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/). Internal contributors name branches with the `litellm_` prefix and no `/`, and fork branches can use any name. Run `make install-hooks` once per clone to enable the local hook that enforces the commit convention. See the [contributor docs](https://docs.litellm.ai/docs/extras/contributing_code#commit-and-branch-conventions) for details
### 2. Development Workflow
Here's the recommended workflow for making changes:
```bash
# Make your changes to the code
# ...
# Format your code (auto-fixes formatting issues)
make format
# Run all linting checks (matches CI exactly)
make lint
# Run the tests covering your change (CI runs the full suite)
uv run pytest tests/unit/<your_test_file>.py -v
# Commit your changes (must follow Conventional Commits — see above)
git add .
git commit -m "feat(scope): your descriptive commit message"
# Push and create a PR
git push origin litellm_your_feature
```
## Adding Testing
**Adding at least 1 test is a hard requirement for all PRs.**
### Where to Add Tests
Add your tests to the [`tests/unit/` directory](https://github.com/BerriAI/litellm/tree/main/tests/unit).
- This directory mirrors the structure of the `litellm/` directory
- **Only add mocked tests** - no real LLM API calls in this directory
- For integration tests with real APIs, use the appropriate test directories
Tests that call `litellm.register_model` or change `litellm.model_cost` should use the `local_model_cost_map` fixture from `tests/unit/conftest.py`. It provides a fresh catalog and restores the previous map, so pricing and capability overrides cannot leak into later tests
### File Naming Convention
The `tests/unit/` directory follows the same structure as `litellm/`:
- `litellm/proxy/caching_routes.py` → `tests/unit/proxy/test_caching_routes.py`
- `litellm/utils.py` → `tests/unit/test_utils.py`
### Example Test
```python
import pytest
from litellm import completion
def test_your_feature():
"""Test your feature with a descriptive docstring."""
# Arrange
messages = [{"role": "user", "content": "Hello"}]
# Act
# Use mocked responses, not real API calls
# Assert
assert expected_result == actual_result
```
## Running Tests and Checks
### Running Unit Tests
Run the tests covering your change:
```bash
uv run pytest tests/unit/test_your_file.py -v
```
`tests/unit` holds thousands of tests, so running all of it locally takes a long time. CI runs it as a parallel matrix (`make test-unit-llms`, `make test-unit-proxy-core`, and the other `test-unit-*` targets) on beefier boxes, so if, for whatever reason, you must run the whole suite, it's better to rely on CI to do that.
If you're running broader test suites, proxy tests, or anything that touches PostgreSQL-backed fixtures/plugins, install the full local test environment first:
```bash
make install-test-deps
```
This syncs the locked test environment used across the repo, including `psycopg` v3 plus `psycopg-binary`, `psycopg2-binary` (used by some proxy E2E tests), and a generated Prisma client for DB-backed proxy tests, so pytest startup matches CI without manual package installs.
### Running Linting and Formatting Checks
Run all linting checks (matches CI exactly):
```bash
make lint
```
Individual linting commands:
```bash
make format-check # Check ruff format formatting
make lint-ruff # Run Ruff linting
make lint-basedpyright # Run basedpyright type checking
make check-circular-imports # Check for circular imports
make check-import-safety # Check import safety
```
Apply formatting (auto-fixes issues):
```bash
make format
```
> **Formatting is enforced in CI.** All PRs must pass the `ruff format --check` step.
>
> - **AI coding agents** (Claude Code, Copilot, Cursor, etc.): follow `AGENTS.md` and run `make format` before committing.
> - **VS Code users**: Install the [Ruff extension](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) and enable format-on-save:
> ```json
> {
> "[python]": {
> "editor.defaultFormatter": "charliermarsh.ruff",
> "editor.formatOnSave": true
> }
> }
> ```
### CI Compatibility
To ensure your changes will pass CI, run the exact same checks locally:
```bash
# This runs the same checks as the GitHub workflows
make lint
make test-unit
```
For exact CI compatibility (pins OpenAI version like CI):
```bash
make install-dev-ci # Installs exact CI dependencies
```
## Available Make Commands
Run `make help` to see all available commands:
```bash
make help # Show all available commands
make install-dev # Install development dependencies
make install-proxy-dev # Install proxy development dependencies
make install-test-deps # Install the full local test environment
make format # Apply ruff format code formatting
make format-check # Check ruff format formatting (matches CI)
make lint # Run all linting checks
make test-unit # Run unit tests
make test-integration # Run integration tests
make test-unit-helm # Run Helm unit tests
```
## Code Quality Standards
LiteLLM follows the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html).
Our automated quality checks include:
- **Ruff** for formatting, linting, and code quality
- **basedpyright** for static type checking
- **Circular import detection**
- **Import safety validation**
All checks must pass before your PR can be merged.
## Common Issues and Solutions
### 1. Linting Failures
If `make lint` fails:
1. **Formatting issues**: Run `make format` to auto-fix
2. **Ruff issues**: Check the output and fix manually
3. **basedpyright issues**: Add proper type hints
4. **Circular imports**: Refactor import dependencies
5. **Import safety**: Fix any unprotected imports
### 2. Test Failures
If `make test-unit` fails:
1. Check if you broke existing functionality
2. Add tests for your new code
3. Ensure tests use mocks, not real API calls
4. Check test file naming conventions
### 3. Common Development Tips
- **Use type hints**: basedpyright requires proper type annotations
- **Write descriptive commit messages**: Help reviewers understand your changes
- **Keep PRs focused**: One feature/fix per PR
- **Test edge cases**: Don't just test the happy path
- **Update documentation**: If you change APIs, update docs
## Building and Running Locally
### LiteLLM Proxy Server
To run the proxy server locally:
```bash
# Install proxy dependencies
make install-proxy-dev
# Start the proxy server
uv run litellm --config your_config.yaml
```
### Docker Development
If you want to build the Docker image yourself:
```bash
# Build using the non-root Dockerfile
docker build -f docker/Dockerfile.non_root -t litellm_dev .
# Generate a master key. Requests send it as the bearer token
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
# Run with your config
docker run \
-v $(pwd)/proxy_config.yaml:/app/config.yaml \
-e LITELLM_MASTER_KEY \
-p 4000:4000 \
litellm_dev \
--config /app/config.yaml --detailed_debug
```
## UI Development
### 1. Setup Your Local UI Development Environment
```bash
# Clone the repo (if you haven't already)
git clone https://github.com/YOUR_USERNAME/litellm.git
cd litellm
# Navigate to the UI dashboard directory
cd ui/litellm-dashboard
# Install dependencies
npm install
# Start the development server
npm run dev
```
### 2. Adding UI Tests
If you are adding a **new component** or **new logic**, you must add corresponding tests.
### 3. Running UI Unit Tests
```bash
npm run test
```
### 4. Building the UI
Ensure the UI builds successfully before submitting your PR:
```bash
npm run build
```
Local lint and budget checks follow origin's current default branch. They refresh it from the remote instead of trusting cached `origin/HEAD`. For an intentional comparison against another branch or commit, use `make check BASE_REF=<ref>` or the standalone gate's `--base <ref>` option. An explicit ref can also be used offline once it has been fetched locally. Without an override, unavailable remote metadata stops the check
## Submitting Your PR
1. **Push your branch**: `git push origin your-feature-branch`
2. **Create a PR**: Go to GitHub and open a pull request against the repository's current default branch. Run `python3 scripts/default_branch.py --branch` to check its name
3. **Fill out the PR template**: Provide clear description of changes
4. **Wait for review**: Maintainers will review and provide feedback
5. **Address feedback**: Make requested changes and push updates
6. **Merge**: Once approved, your PR will be merged!
## Getting Help
If you need help:
- 💬 [Join our Discord](https://discord.gg/wuPM9dRgDw)
- 💬 [Join our Slack](https://www.litellm.ai/support)
- 📧 Email us: ishaan@berri.ai / krrish@berri.ai
- 🐛 [Create an issue](https://github.com/BerriAI/litellm/issues/new)
## What to Contribute
Looking for ideas? Check out:
- 🐛 [Good first issues](https://github.com/BerriAI/litellm/labels/good%20first%20issue)
- 🚀 [Feature requests](https://github.com/BerriAI/litellm/labels/enhancement)
- 📚 Documentation improvements
- 🧪 Test coverage improvements
- 🔌 New LLM provider integrations
Thank you for contributing to LiteLLM! 🚀

View file

@ -1,186 +0,0 @@
# syntax=docker/dockerfile:1.7
# Base image for building
ARG LITELLM_BUILD_IMAGE=cgr.dev/chainguard/wolfi-base@sha256:360896efb0753a5701c48f860e17130c205eae5a264797aa3e7ea126ee2923a7
# Runtime image
ARG LITELLM_RUNTIME_IMAGE=cgr.dev/chainguard/wolfi-base@sha256:360896efb0753a5701c48f860e17130c205eae5a264797aa3e7ea126ee2923a7
ARG UV_IMAGE=ghcr.io/astral-sh/uv:0.11.7@sha256:240fb85ab0f263ef12f492d8476aa3a2e4e1e333f7d67fbdd923d00a506a516a
# Pinned by digest like the other base images; bump explicitly on Node upgrades.
ARG UI_BUILD_IMAGE=node:24.19-alpine3.24@sha256:d32cdf619f63fe0471182d08996dd516c6275bb5fd31ae06e55a570bd9e1ad43
# Checksum from https://www.pgbouncer.org/downloads/ (the Wolfi repo only carries 1.24.x)
ARG PGBOUNCER_VERSION=1.26.0
ARG PGBOUNCER_SHA256=afd25dd61ee6775d37b40629b87ce08736b3e6955f3057bb212e410fbf21c71d
FROM $UV_IMAGE AS uvbin
FROM $LITELLM_BUILD_IMAGE AS pgbouncer-builder
ARG PGBOUNCER_VERSION
ARG PGBOUNCER_SHA256
USER root
RUN apk add --no-cache build-base pkgconf libevent-dev openssl-dev curl
WORKDIR /build
RUN curl -fsSL -o pgbouncer.tar.gz "https://www.pgbouncer.org/downloads/files/${PGBOUNCER_VERSION}/pgbouncer-${PGBOUNCER_VERSION}.tar.gz" && \
echo "${PGBOUNCER_SHA256} pgbouncer.tar.gz" | sha256sum -c - && \
tar xzf pgbouncer.tar.gz --strip-components=1 && \
./configure --prefix=/usr/local --with-openssl=/usr && \
make -j"$(nproc)" pgbouncer && \
install -m 0755 pgbouncer /usr/local/bin/pgbouncer
# Admin UI builder. Pinned to the build platform so the architecture-independent
# Next.js static export compiles once natively even in a multi-arch build,
# instead of once per target arch under QEMU.
FROM --platform=$BUILDPLATFORM $UI_BUILD_IMAGE AS ui-builder
ENV NEXT_TELEMETRY_DISABLED=1 \
npm_config_fund=false \
npm_config_audit=false
WORKDIR /ui
COPY ui/litellm-dashboard/package.json ui/litellm-dashboard/package-lock.json ./
COPY ui/litellm-dashboard/vendor/ ./vendor/
RUN --mount=type=cache,target=/root/.npm npm ci --prefer-offline
COPY ui/litellm-dashboard/ ./
RUN npm run build
# Builder stage
FROM $LITELLM_BUILD_IMAGE AS builder
WORKDIR /app
USER root
COPY --from=uvbin /uv /usr/local/bin/uv
COPY --from=uvbin /uvx /usr/local/bin/uvx
RUN apk add --no-cache \
bash \
gcc \
python-3.13 \
python-3.13-dev \
rust \
openssl \
libssl3 \
openssl-dev \
nodejs \
npm \
libsndfile
ENV UV_PROJECT_ENVIRONMENT=/app/.venv \
UV_LINK_MODE=copy \
UV_PYTHON_DOWNLOADS=0 \
PATH="/app/.venv/bin:${PATH}"
# Copy dependency metadata first for layer caching
COPY pyproject.toml uv.lock ./
COPY enterprise/pyproject.toml enterprise/
COPY litellm-proxy-extras/pyproject.toml litellm-proxy-extras/
# Install third-party dependencies (cached unless pyproject.toml/uv.lock change)
RUN uv sync --frozen --no-install-project --no-install-workspace --no-default-groups --no-editable \
--extra proxy \
--extra proxy-runtime \
--group admin-mcp \
--extra extra_proxy \
--extra semantic-router \
--extra saml \
--extra bedrock-realtime \
--python python3.13
# Copy full source tree
COPY . .
# Replace the committed UI bundle with the one built from this exact source.
# Clearing first drops the committed bundle's content-hashed chunks that COPY
# would otherwise leave behind alongside the fresh ones.
RUN rm -rf litellm/proxy/_experimental/out
COPY --from=ui-builder /ui/out/. litellm/proxy/_experimental/out/
# Build Admin UI before final sync (applies the enterprise color override when present)
RUN sed -i 's/\r$//' docker/build_admin_ui.sh && chmod +x docker/build_admin_ui.sh && ./docker/build_admin_ui.sh
# Install project and workspace packages (fast - deps already cached)
RUN uv sync --frozen --no-default-groups --no-editable \
--extra proxy \
--extra proxy-runtime \
--group admin-mcp \
--extra extra_proxy \
--extra semantic-router \
--extra saml \
--extra bedrock-realtime \
--python python3.13
RUN HOME=/opt/prisma XDG_CACHE_HOME=/opt/prisma/.cache PRISMA_BINARY_CACHE_DIR=/opt/prisma/binaries \
npm_config_cache=/root/.npm \
prisma generate --schema=./schema.prisma
RUN sed -i 's/\r$//' docker/entrypoint.sh && chmod +x docker/entrypoint.sh && \
sed -i 's/\r$//' docker/prod_entrypoint.sh && chmod +x docker/prod_entrypoint.sh
FROM $LITELLM_BUILD_IMAGE AS liteadmin-builder
COPY --from=uvbin /uv /usr/local/bin/uv
RUN apk add --no-cache python-3.13
ADD --checksum=sha256:4fb1847e28e2be30ba76dac94f02c4903291326516a0f4ba772f6ffb8b25d05c \
https://github.com/BerriAI/litellm-admin-agent/archive/5661c805ce479a0cf11b842ea133cf8d22cd7e34.tar.gz /tmp/liteadmin.tar.gz
RUN mkdir /tmp/liteadmin && tar xzf /tmp/liteadmin.tar.gz --strip-components=1 -C /tmp/liteadmin && \
uv venv /opt/liteadmin --python python3.13 && \
uv pip install --python /opt/liteadmin/bin/python --require-hashes -r /tmp/liteadmin/requirements.txt && \
uv pip install --python /opt/liteadmin/bin/python --no-deps /tmp/liteadmin
# Runtime stage
FROM $LITELLM_RUNTIME_IMAGE AS runtime
ARG LITELLM_RELEASE_TAG=""
ENV LITELLM_RELEASE_TAG=${LITELLM_RELEASE_TAG}
USER root
# The base image only configures Chainguard's authenticated apk repo, which
# requires an enterprise subscription. Add the public Wolfi repo so `apk add`
# also works for anyone installing extra packages into a running container.
# https://github.com/BerriAI/litellm/issues/33518
RUN echo "https://packages.wolfi.dev/os" >> /etc/apk/repositories
# node (without npm) is required by the prisma CLI at runtime
RUN apk add --no-cache bash openssl libssl3 tzdata nodejs python-3.13 libsndfile libevent
COPY --from=pgbouncer-builder /usr/local/bin/pgbouncer /usr/local/bin/pgbouncer
WORKDIR /app
ENV PATH="/app/.venv/bin:${PATH}" \
PRISMA_BINARY_CACHE_DIR=/opt/prisma/binaries \
PRISMA_CLI_PATH=/opt/prisma/binaries/node_modules/.bin/prisma \
PRISMA_CLI_QUERY_ENGINE_TYPE=binary \
PRISMA_OFFLINE_MODE=true
# Copy only what runtime needs. The application is installed inside the venv;
# the rest of the builder's /app is source and build metadata that must not
# ship (manifest-scanning tools attribute everything in it to this image).
# entrypoint.sh invokes litellm/proxy/prisma_migration.py by source path.
COPY --from=builder /app/.venv /app/.venv
COPY --from=liteadmin-builder /opt/liteadmin /opt/liteadmin
COPY --from=builder /app/docker /app/docker
COPY --from=builder /app/schema.prisma /app/schema.prisma
COPY --from=builder /app/litellm/proxy/prisma_migration.py /app/litellm/proxy/prisma_migration.py
# enterprise/ is imported by source path at runtime (proxy_cli puts the
# working directory on sys.path; litellm/proxy/hooks resolves
# enterprise.enterprise_hooks from it)
COPY --from=builder /app/enterprise /app/enterprise
COPY --from=builder /app/litellm-proxy-extras /app/litellm-proxy-extras
# Prisma CLI + engines are baked under /opt/prisma, a fixed path every
# runtime uid can read and that no cache volume mount shadows. The paths are
# pinned via PRISMA_BINARY_CACHE_DIR / PRISMA_CLI_PATH and recorded into the
# generated client at build time, so `prisma migrate deploy` on a fresh
# database needs no npm and no network access (#33650, #24554).
COPY --from=builder /opt/prisma /opt/prisma
RUN find /app/.venv -type f -path "*/tornado/test/*" -delete && \
find /app/.venv -type d -path "*/tornado/test" -delete && \
chmod -R a+rX /opt/prisma && \
test -x /opt/prisma/binaries/node_modules/.bin/prisma && \
test -f /opt/prisma/binaries/node_modules/prisma/build/index.js && \
python -c "from prisma.client import BINARY_PATHS; paths = list(BINARY_PATHS.query_engine.values()); assert paths and all(p.startswith('/opt/prisma/') for p in paths), paths"
EXPOSE 4000/tcp
ENTRYPOINT ["docker/prod_entrypoint.sh"]
CMD ["--port", "4000"]

View file

@ -1 +0,0 @@
Read @AGENTS.md for coding guidelines

View file

@ -1,6 +1,6 @@
Portions of this software are licensed as follows:
* All content that resides under the "enterprise/" directory of this repository, if that directory exists, is licensed under the license defined in "enterprise/LICENSE".
* All content that resides under the "litellm/enterprise/" directory of this repository, if that directory exists, is licensed under the license defined in "litellm/enterprise/LICENSE".
* Content outside of the above mentioned directories or restrictions above is available under the MIT license as defined below.
---
MIT License

Some files were not shown because too many files have changed in this diff Show more