mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-13 08:52:02 +00:00
chore: move scripts and runtime sources under litellm (#51006)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Yassin Kortam
GitHub
parent
db6dfb6bb6
commit
adc4c01aa9
329 changed files with 610 additions and 1087 deletions
|
|
@ -1,73 +0,0 @@
|
|||
---
|
||||
name: database-access
|
||||
description: Organize database repositories and proxy dependency wiring, manage connection ownership, or port Prisma operations to Rust and SQLx without expanding migration scope
|
||||
metadata:
|
||||
internal: true
|
||||
---
|
||||
|
||||
# Database access
|
||||
|
||||
## Situations
|
||||
|
||||
- Adding a query or an operation to a repository, read [adding a query](references/python-repository.md#adding-a-query)
|
||||
- Moving existing Prisma calls behind a repository, read [moving existing access](references/python-repository.md#moving-existing-access-into-a-repository)
|
||||
- Porting a repository from Prisma to Rust and SQLx, read [porting to SQLx](references/python-repository.md#porting-a-repository-to-sqlx)
|
||||
- Injecting a repository into a route, read [Python and FastAPI](references/python-fastapi.md) or [Rust and Axum](references/rust-axum.md)
|
||||
|
||||
## Adding new database access
|
||||
|
||||
- Put Python database access behind an operation in `litellm/repositories`
|
||||
- Give each repository its own FastAPI provider, so a route builds only the repositories it uses
|
||||
- A provider borrows existing handles and performs no database I/O during construction
|
||||
- Define the operations a route calls as a Protocol in `litellm/models/<feature>.py`, next to its domain types
|
||||
- The Protocol takes and returns domain types, never Prisma models, raw rows or serialized payloads
|
||||
- Parsing and serialization of stored values live in the repository
|
||||
- Add a Protocol when a repository gets a provider, not up front for every repository
|
||||
- Keep query execution inside the repository, with one implementation per repository and no backend switch or fallback
|
||||
- A repository is either all Prisma or all native, never a mix of both
|
||||
- Prisma is fine for the implementation, adding a query does not require a Rust port
|
||||
- Native Rust callers use `litellm/rust/crates/db` directly
|
||||
|
||||
## Framework wiring
|
||||
|
||||
- Python routes receive a per-repository dependency such as `UserBannerRepositoryDep` and call its operations, such as `banners.get_banner()`
|
||||
- `DatabaseDep` lives in `litellm/proxy/deps/database.py`, repository providers in `litellm/proxy/deps/repositories.py`
|
||||
- A provider returns the feature's Protocol, so routes never see which backend implements it
|
||||
- Dependencies used by one feature only, like the banner audit binding, stay next to that feature's routes
|
||||
- Repositories never import FastAPI, so jobs and helpers can use them without a request
|
||||
- Use named FastAPI providers with `Annotated[..., Depends(...)]`, ordinary helpers and jobs receive explicit arguments
|
||||
- Preserve each route's no-database behavior, authorization precedence and background-task scheduling
|
||||
- Read [Python and FastAPI](references/python-fastapi.md) for provider examples, lifespan state and route verification
|
||||
- Read [Rust and Axum](references/rust-axum.md) when adding database access to Rust handlers
|
||||
|
||||
## Connection ownership
|
||||
|
||||
- Keep connection setup and shutdown separate from repositories, which own no pools
|
||||
- Reuse existing handles, never connect per request or disconnect from a repository operation
|
||||
- Give each pool one owner for setup and shutdown, borrowed handles never close it
|
||||
- Finish requests, jobs and queued database work before closing connections
|
||||
- Clean up resources opened before a startup failure, without a second disconnect fallback
|
||||
- Add a backend to the existing lifecycle owner and pass its handle only to repositories that need it
|
||||
- Preserve cancellation, timeout, error and transaction semantics across the native bridge
|
||||
|
||||
## Changing ownership or reloading resources
|
||||
|
||||
- Inventory initialization, reload, tests, background jobs and shutdown together
|
||||
- Move every writer when changing authoritative storage, copying a replaceable global into state is insufficient
|
||||
- A frozen repository does not make its clients immutable
|
||||
- Decide whether each request captures one resource generation or reads it again during processing
|
||||
- Build and validate replacements before publishing them, preserving the old generation on failure
|
||||
- Serialize reloads and coordinate publication with shutdown
|
||||
- Drain streams and background users before retiring old resources, retaining every generation grows resource use
|
||||
- Define multi-worker coordination separately from an in-process swap
|
||||
- Choose mechanisms such as `ArcSwap` only after defining publication, capture and retirement
|
||||
|
||||
## Migration context and boundaries
|
||||
|
||||
- Long-term direction is to centralize Python database access in repositories, then replace their Prisma implementations with Rust APIs one repository at a time
|
||||
- Prisma and native repositories coexist during the transition, unrelated existing calls can stay as they are
|
||||
- Prisma and SQLx keep separate client pools to PgBouncer, which pools connections to PostgreSQL
|
||||
- Size both layers together and preserve reader/writer routing and pooling-mode compatibility
|
||||
- PgBouncer does not share transactions between Prisma and SQLx, keep each atomic unit of work on one backend and transaction
|
||||
- Never retry a failed Rust write through Prisma when it may already have committed
|
||||
- Prisma owns schema migrations today, eventual SQLx migration ownership and Prisma removal are separate planned steps, not part of a query port
|
||||
|
|
@ -1,101 +0,0 @@
|
|||
# Python and FastAPI
|
||||
|
||||
## Existing pattern
|
||||
|
||||
- `UserBannerStore` in `litellm/models/user_banner.py` is the Protocol the banner routes call
|
||||
- `get_database` in `litellm/proxy/deps/database.py` reads the lifespan-owned handle from request state
|
||||
- `get_user_banner_repository` in `litellm/proxy/deps/repositories.py` builds the banner repository from that handle
|
||||
- Choose each repository's replica routing once in its provider, never at call sites
|
||||
- A handler receives `UserBannerStore | None`, never a Prisma client or a concrete repository
|
||||
- `UserBannerRepository` implements `UserBannerStore` structurally, keeping the existing Prisma table, serialization and configuration-sync behavior
|
||||
- The banner audit provider binds the legacy audit callable at request time, without performing audit I/O
|
||||
- A native repository can later implement the same Protocol without changing handler signatures, see [porting a repository to SQLx](python-repository.md#porting-a-repository-to-sqlx)
|
||||
|
||||
## Dependency injection
|
||||
|
||||
- Add one provider and one `Annotated` alias per repository in `litellm/proxy/deps/repositories.py`
|
||||
- Annotate the provider and alias with the feature's Protocol, not the concrete repository
|
||||
- A route depends only on the repositories it calls, so a request never builds repositories it does not use
|
||||
- Do not build repositories once at lifespan, a reconnect can replace the Prisma client they wrap
|
||||
|
||||
```python
|
||||
class UserBannerStore(Protocol):
|
||||
async def get_banner(self) -> UserBanner: ...
|
||||
|
||||
async def save_banner(self, banner: UserBanner) -> None: ...
|
||||
```
|
||||
|
||||
```python
|
||||
async def get_user_banner_repository(database: DatabaseDep) -> UserBannerStore | None:
|
||||
return None if database is None else UserBannerRepository(database.prisma)
|
||||
|
||||
|
||||
UserBannerRepositoryDep: TypeAlias = Annotated[UserBannerStore | None, Depends(get_user_banner_repository)]
|
||||
```
|
||||
|
||||
- FastAPI caches each provider within a request, so routes and sub-dependencies share one instance
|
||||
- Ordinary helpers and jobs receive explicit arguments, never unresolved `Depends` defaults
|
||||
- A route still reading the `prisma_client` global or building a repository inline switches to the repository's `...RepositoryDep`, keeping its no-database behavior and error precedence
|
||||
|
||||
```python
|
||||
async def get_user_banner(banners: UserBannerRepositoryDep) -> UserBanner:
|
||||
if banners is None:
|
||||
return UserBanner()
|
||||
return await banners.get_banner()
|
||||
```
|
||||
|
||||
## Lifespan state
|
||||
|
||||
- Prisma startup owns connection setup, authentication refresh, health checks and migrations
|
||||
- `Database` holds the initialized Prisma client and the `NativeDatabase` handle, without holding fields per feature
|
||||
- The native handle opens no pool yet, its `connect` and `close` are noops
|
||||
- The proxy lifespan builds the handle with `Database.open`, registers `Database.aclose` on its exit stack and publishes it in request state
|
||||
- Callers only open and close `Database`, which closes Prisma and the native handle together, and `open` closes Prisma itself when the native connect fails
|
||||
- Starlette shallow-copies lifespan state into request state, which is distinct from `app.state`
|
||||
- The database module never imports a repository, legacy globals borrow its handle without owning shutdown
|
||||
- Drain requests, jobs, logging, spend and audit work before database teardown
|
||||
- The exit stack closes the handle on exit, including startup failures after the client is created
|
||||
- When adding native resources, initialize them in the serving process, respecting runtime fork restrictions
|
||||
- Await native shutdown instead of depending on Python garbage collection
|
||||
- Apply the [connection ownership rules](../SKILL.md#connection-ownership) when changing the owner
|
||||
|
||||
## Timing and errors
|
||||
|
||||
- FastAPI resolves dependencies before entering the handler
|
||||
- Handler exception handling cannot catch failures raised during dependency resolution
|
||||
- A universal required-database dependency can override authorization and validation behavior
|
||||
- Banner GET without a database returns the default disabled banner
|
||||
- Banner PATCH checks the admin role before returning the existing missing-database error
|
||||
- Invalid stored banner data falls back to the default, but storage failures still propagate
|
||||
- Banner writes finish before the success audit is queued as a FastAPI background task
|
||||
- Queue post-response work with `BackgroundTasks`, not bare `asyncio.create_task`, so the in-flight drain waits for it before the database closes
|
||||
- Do not move background work into an awaited provider merely to remove an import
|
||||
|
||||
## Verification
|
||||
|
||||
- Exercise real FastAPI handlers using `dependency_overrides` for external boundaries
|
||||
- Keep production parsing and persistence code real
|
||||
- Restore overrides after each test
|
||||
- Verify complete response bodies, status codes and OpenAPI against the base
|
||||
- Include combined failure conditions to pin error precedence
|
||||
- Check saved values, fresh revisions, audit contents and failed-write behavior
|
||||
- Check lifespan state with a running lifespan, not only a TestClient that skips startup
|
||||
- Verify providers do not connect, query, disconnect or audit eagerly
|
||||
- Use real PostgreSQL and live proxy requests when changing persistence behavior
|
||||
|
||||
## Scope
|
||||
|
||||
- Preserve direct Python call sites or migrate them explicitly
|
||||
- Inventory imports, state writers, side effects and error ordering before migrating a feature
|
||||
- Preserve configuration sync and serialization, and state which legacy reads remain
|
||||
- Do not migrate every route in a large module just to establish the pattern
|
||||
- Do not invent providers for resources without consumers
|
||||
|
||||
## Framework references
|
||||
|
||||
- [Lifespan events](https://fastapi.tiangolo.com/advanced/events/) and [Starlette lifespan state](https://starlette.dev/lifespan/)
|
||||
- [`SessionDep`-style `Annotated` aliases](https://fastapi.tiangolo.com/tutorial/sql-databases/#create-a-session-dependency), which `DatabaseDep` and `UserBannerRepositoryDep` follow
|
||||
- [Dependency composition and caching](https://fastapi.tiangolo.com/tutorial/dependencies/sub-dependencies/)
|
||||
- [Background tasks and dependencies with `yield`](https://fastapi.tiangolo.com/advanced/advanced-dependencies/#background-tasks-and-dependencies-with-yield-technical-details)
|
||||
- [Dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
||||
- [Testing lifespan](https://fastapi.tiangolo.com/advanced/testing-events/)
|
||||
|
|
@ -1,119 +0,0 @@
|
|||
# Python repositories
|
||||
|
||||
## Adding a query
|
||||
|
||||
Use this when a feature needs a read or write that no repository offers yet, including one more operation on an existing repository
|
||||
|
||||
### Before starting
|
||||
|
||||
- Search `litellm/repositories` for a repository that already owns the table, and extend it instead of adding a second one
|
||||
- Check `litellm/models/<feature>.py` for an existing Protocol and domain types
|
||||
- Decide the route's behavior when no database is configured, and where that check sits relative to authorization and validation
|
||||
|
||||
### Steps
|
||||
|
||||
1. Domain types and Protocol in `litellm/models/<feature>.py`
|
||||
- The Protocol takes and returns domain types, never Prisma models, raw rows or serialized payloads
|
||||
- Add the Protocol together with the provider, not for repositories nothing injects
|
||||
2. Repository in `litellm/repositories/<feature>_repository.py`
|
||||
- Implement the operation with Prisma, a Rust port is not required
|
||||
- Compose the Prisma client and type the tables it touches with `TableActions` from `prisma_protocols.py`, never inherit `PrismaTableRepository`
|
||||
- Parse and serialize stored values here, so callers only see domain types
|
||||
- Never import FastAPI, connect, disconnect or choose replicas per call
|
||||
3. Provider and `Annotated` alias in `litellm/proxy/deps/repositories.py`, see [dependency injection](python-fastapi.md#dependency-injection)
|
||||
4. The route takes `<Feature>RepositoryDep`, handles `None` with the behavior decided above, and calls the operation
|
||||
5. Jobs and helpers that need the operation receive the repository as an explicit argument, built by their caller from the handle it already holds
|
||||
|
||||
### Done when
|
||||
|
||||
- Handler tests drive the real route with `dependency_overrides` and the real repository code, see [verification](python-fastapi.md#verification)
|
||||
- Tests cover the no-database path, error precedence, and what was actually stored
|
||||
- Persistence changes are checked against real PostgreSQL through a live proxy
|
||||
|
||||
### Out of scope
|
||||
|
||||
- Porting the query to Rust, see [porting a repository to SQLx](#porting-a-repository-to-sqlx)
|
||||
- Moving unrelated direct Prisma calls in the same module
|
||||
|
||||
## Moving existing access into a repository
|
||||
|
||||
Use this when code calls Prisma directly, through `prisma_client.db.<table>` or a helper, and the calls should live behind a repository. The backend stays Prisma
|
||||
|
||||
### Before starting
|
||||
|
||||
- List every call being moved, its callers, and the transaction or batch it runs in
|
||||
- List the module's imports, global state writers, side effects such as audit and config sync, and error ordering
|
||||
- Record current return values, raised errors and response bodies, these are what the change must preserve
|
||||
|
||||
### Steps
|
||||
|
||||
1. Move the existing Prisma code into the relevant repository in `litellm/repositories` without rewriting the queries
|
||||
2. Add domain types and a Protocol in `litellm/models/<feature>.py` when a route will inject the repository
|
||||
3. Replace each direct call with the repository operation, through a provider for routes and explicit arguments for jobs and helpers, see [Python and FastAPI](python-fastapi.md)
|
||||
4. Calls sharing a transaction move into the same repository or unit of work in one change
|
||||
5. State which legacy reads remain and why, when the change does not move all of them
|
||||
|
||||
### Done when
|
||||
|
||||
- Focused regression tests pin the preserved return values, errors and transaction behavior
|
||||
- Response bodies, status codes and OpenAPI match the base
|
||||
- Configuration sync and serialization behave the same, including invalid stored data
|
||||
|
||||
### Out of scope
|
||||
|
||||
- Switching the backend, even when an equivalent Rust query exists
|
||||
- Converting other routes in the module just to establish the pattern
|
||||
- Cleanups unrelated to the moved calls
|
||||
|
||||
## Porting a repository to SQLx
|
||||
|
||||
Status: TBD. No repository has been ported yet. `database.native` is a `NativeDatabase` from `crates/python-bridge/src/db.rs` whose `connect` and `close` are noops that open no pool, and the PyO3 repository surface does not exist. A real pool takes typed writer/reader settings and a token provider, never the Prisma `DATABASE_URL`, which carries Prisma-only params and rotating IAM tokens. The constraints below are settled, the steps are a sketch to revise after the first port lands
|
||||
|
||||
Use this only when a port is explicitly requested. Repository extraction does not require one
|
||||
|
||||
### When not to port
|
||||
|
||||
- An equivalent Rust or SQLx query existing is not a reason to switch a Prisma caller
|
||||
- Leave existing backend selection unchanged unless integration is explicitly requested
|
||||
- Never port part of a repository, it is either all Prisma or all native
|
||||
|
||||
### Before starting
|
||||
|
||||
- Identify the repository's inputs, outputs, errors and transaction boundary
|
||||
- List repositories that share a transaction with it, they move together
|
||||
- Check `litellm/rust/crates/db` for an equivalent query and reuse it when its behavior matches
|
||||
|
||||
### Steps
|
||||
|
||||
1. Rust queries, following `litellm/rust/crates/db/AGENTS.md`
|
||||
- Static SQL in `queries/`, compile-checked with SQLx, typed inputs and row projections in `src/`
|
||||
- Dynamic filters and sort keys become closed Rust enums that select static SQL, every value is bound
|
||||
- Refresh SQLx query metadata after SQL changes
|
||||
2. PyO3 surface exposing typed operations, never SQL text, table names or column names from Python. TBD: where it lives and how the handle reaches it
|
||||
3. Native repository implementing the same Protocol in `litellm/repositories`
|
||||
- Converts typed PyO3 results to domain types, preserving defaults, invalid-data handling and storage errors
|
||||
- Python conversion stays in the bridge, not in `crates/db`
|
||||
4. Switch the provider in `litellm/proxy/deps/repositories.py`, the Protocol and routes stay unchanged
|
||||
|
||||
```diff
|
||||
async def get_user_banner_repository(database: DatabaseDep) -> UserBannerStore | None:
|
||||
- return None if database is None else UserBannerRepository(database.prisma)
|
||||
+ return None if database is None else NativeUserBannerRepository(database.native)
|
||||
```
|
||||
|
||||
5. Delete the Prisma repository in the same change and switch every caller, including jobs and helpers
|
||||
|
||||
### Done when
|
||||
|
||||
- Behavior is verified against real PostgreSQL, covering null handling, conflicts, ordering, errors and transaction semantics
|
||||
- `DATABASE_URL=... cargo test -p litellm-db --locked -- --include-ignored` passes against a Prisma-migrated database
|
||||
- Live proxy requests show unchanged responses. TBD: a Python-side real-PostgreSQL test harness
|
||||
|
||||
### Rollback
|
||||
|
||||
- Roll back the release, never retry an uncertain native write through Prisma
|
||||
|
||||
### Out of scope
|
||||
|
||||
- SQLx owning schema migrations, and removing Prisma
|
||||
- Converting unrelated routes to dependency injection
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
# Rust and Axum
|
||||
|
||||
- This should use same `crates/db` what is exposed to Python world.
|
||||
- Details are TBD.
|
||||
|
|
@ -1,205 +0,0 @@
|
|||
---
|
||||
name: litellm-setup
|
||||
description: Set up a self-hosted LiteLLM proxy, the OpenAI-compatible LLM gateway, on this machine, from install through provider keys, config.yaml, a master key, optional Postgres and Admin UI, and a verified chat completion. Use when the user asks to set up LiteLLM, install the LiteLLM proxy, self-host an LLM gateway, or put one endpoint in front of OpenAI, Anthropic, Gemini, Azure, Bedrock, Ollama, or other providers. Not for using the LiteLLM Python SDK inside an app
|
||||
---
|
||||
|
||||
# Set up a LiteLLM proxy
|
||||
|
||||
Take the user from nothing to a running LiteLLM proxy that has answered one chat completion. Run every command yourself unless a step says the user does it
|
||||
|
||||
Never print a secret. Provider keys, the master key, the salt key, the database URL, and virtual keys live only in `.env`, and `config.yaml` reads them through `os.environ/NAME`. Check that a value is there with `grep -c '^NAME=.' .env`, never with `cat .env` or `echo "$NAME"`. Load `.env` into a shell with this line, which exports each line as it is, so a `&` or `$` in a value stays literal, and first drops any database settings the shell already had and sets `LITELLM_MODE=PRODUCTION`, which stops `litellm` from also loading a `.env` it finds above its install folder, such as `~/.env`, so only this `.env` picks the database. The steps that need it start with it:
|
||||
|
||||
```bash
|
||||
unset DATABASE_URL DIRECT_URL DATABASE_HOST; export LITELLM_MODE=PRODUCTION; while IFS= read -r line || [ -n "$line" ]; do case "$line" in [A-Za-z_]*=*) export "$line" ;; esac; done < .env
|
||||
```
|
||||
|
||||
## 1. Ask once
|
||||
|
||||
Ask everything in one message, with defaults the user can accept by saying "go":
|
||||
|
||||
- Providers and models. Default is OpenAI with its newest flagship model, using the `OPENAI_API_KEY` in your environment when `[ -n "$OPENAI_API_KEY" ] && echo set` says it is there. Run that check before asking, and say in the question that the default uses that variable, naming it but never its value, since a key in your environment can belong to another account
|
||||
- Whether they want virtual keys, teams, spend tracking, or the Admin UI. Those need Postgres, plain routing does not. Default is no
|
||||
- uv (default) or Docker. With Postgres, Docker runs the repo's quickstart, which brings its own database, and uv needs a Postgres server they already have, with a new, empty database for LiteLLM, so someone with their own Postgres server picks uv
|
||||
- Folder and port. Defaults are `./litellm-proxy` (`./litellm-gateway` for the quickstart) and 4000
|
||||
|
||||
Then follow the steps for their answers. Plain routing skips steps 3 and 6. The quickstart is steps 2, 3, 8, and 9. uv with their own Postgres skips step 3
|
||||
|
||||
In the same message, say that someone who picks uv and would rather answer prompts can run `litellm --setup` in a separate terminal window once LiteLLM is installed, not through Claude Code. Never run it yourself, since it needs a real terminal and without one prints "Setup cancelled." and writes nothing. Give them the command as `LITELLM_MODE=PRODUCTION <bin>/litellm --setup`, with `<bin>` replaced by the folder `uv tool dir --bin` prints, to run from the folder they want the config in. It writes `litellm_config.yaml` there with the provider keys and master key in plain text and offers to start the proxy, which then listens on every network interface and stops with Ctrl+C in that window. When the user is done, run this in that folder. It makes the file readable only by them, keeps it out of git (the wizard does neither), and prints only the model names. Never print that file or run `grep`, `sed`, or `cat` on it:
|
||||
|
||||
```bash
|
||||
chmod 600 litellm_config.yaml; grep -qx 'litellm_config.yaml' .gitignore 2>/dev/null || echo 'litellm_config.yaml' >> .gitignore
|
||||
"$(uv tool dir)/litellm/bin/python" -I -c 'import yaml; print(*(m["model_name"] for m in yaml.safe_load(open("litellm_config.yaml"))["model_list"]), sep="\n")'
|
||||
```
|
||||
|
||||
Then skip to step 8 and run it against their proxy, starting that shell command with this line instead of the loader line, which reads the master key without printing it:
|
||||
|
||||
```bash
|
||||
export LITELLM_MASTER_KEY="$("$(uv tool dir)/litellm/bin/python" -I -c 'import yaml; print(yaml.safe_load(open("litellm_config.yaml"))["general_settings"]["master_key"])')"
|
||||
```
|
||||
|
||||
## 2. Check prerequisites
|
||||
|
||||
```bash
|
||||
uv --version; docker info --format '{{.ServerVersion}}'; docker compose version; curl --version | head -1; openssl version; command -v python3
|
||||
```
|
||||
|
||||
On native Windows, ask the user to run Claude Code inside WSL, since every command here is POSIX shell. Every path needs curl, openssl, and python3, so ask the user to install any that is missing. The uv path needs uv, which downloads a supported Python (LiteLLM supports 3.10 to 3.14) when the `--python` range in step 4 rules out the one on the machine. If uv is missing, offer to install it, and only after the user says yes run `curl -LsSf https://astral.sh/uv/install.sh | env UV_NO_MODIFY_PATH=1 sh`, which leaves their shell profile alone (https://docs.astral.sh/uv/getting-started/installation/). It puts uv in `~/.local/bin`, so start every later command that runs `uv` or `litellm` with `export PATH="$HOME/.local/bin:$PATH";`. The Docker path needs a running Docker daemon, and the quickstart also needs Docker Compose v2. If `docker info` says `permission denied`, the user's account cannot use Docker without sudo. Offer the uv path, or ask them to add themselves to the `docker` group the way https://docs.docker.com/engine/install/linux-postinstall/ shows and start a new login session before you check again
|
||||
|
||||
Check the port is free with `curl -s -o /dev/null --max-time 2 http://127.0.0.1:4000; echo $?`, which prints `7` (could not connect) only when nothing listens there. Don't use `lsof` for this, since it misses ports that root or Docker hold. If it is taken, pick another port and use it everywhere below, because a proxy asked for a busy 4000 silently moves to a random port
|
||||
|
||||
## 3. Postgres with Docker: run the quickstart
|
||||
|
||||
The repo's quickstart starts the gateway, Postgres, and the Admin UI in Docker, listening on this machine only, and puts a `litellm-gateway` command in that folder that stops, starts, updates, or removes them. It asks nothing when it has no terminal. It writes the master key, salt key, and database password to its own `.env` and never prints them. Download it, then run it with the folder and port from step 1. It returns once the gateway is ready:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.sh -o quickstart.sh
|
||||
LITELLM_DIR="$PWD/litellm-gateway" LITELLM_PORT=4000 sh quickstart.sh --yes
|
||||
cd litellm-gateway
|
||||
```
|
||||
|
||||
Models live in the database here, so add each one through the API instead of a `config.yaml`, naming it the way step 5 says. Pipe the request body in, so the provider key never shows up in output or on a command line. Unless the user agreed to the key in your environment, as step 5 says, ask them to add the line `OPENAI_API_KEY=<their key>` to this `.env` themselves. Run the loader line first:
|
||||
|
||||
```bash
|
||||
python3 -c 'import json, os, sys; json.dump({"model_name": "gpt-6.1-sol", "litellm_params": {"model": "openai/gpt-6.1-sol", "api_key": os.environ["OPENAI_API_KEY"]}}, sys.stdout)' \
|
||||
| curl -sS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:4000/model/new \
|
||||
-H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d @-
|
||||
```
|
||||
|
||||
Run that in one shell command that starts with the loader line, since the master key is in `.env`. `200` means the model was added. Then go to step 8
|
||||
|
||||
## 4. Install
|
||||
|
||||
uv:
|
||||
|
||||
```bash
|
||||
uv tool install --python '>=3.10,<3.15' 'litellm[proxy]'
|
||||
export PATH="$(uv tool dir --bin):$PATH"; litellm --version
|
||||
```
|
||||
|
||||
With their own Postgres, install `'litellm[proxy,extra_proxy]'` instead, with the same `--python` range, which adds the Prisma client the proxy uses for the database. Running `uv tool install` again with the new extras replaces the old install. Start every later command that runs `litellm` with `export PATH="$(uv tool dir --bin):$PATH";`, so this install runs even when the shell has no `litellm` or an older one from a venv or `pip install litellm`. `uv tool update-shell` puts it on the PATH of new shells by editing the user's shell profile, so run it only after the user says yes
|
||||
|
||||
Docker:
|
||||
|
||||
```bash
|
||||
docker pull docker.litellm.ai/berriai/litellm:main-stable
|
||||
```
|
||||
|
||||
## 5. Write `.env` and `config.yaml`
|
||||
|
||||
Run every command from here on in the folder from step 1. Create `.env` readable only by the user, keep it out of git, and add a master key unless it already has one. The proxy refuses to start without one, and `sk-` is the format of every key it issues:
|
||||
|
||||
```bash
|
||||
mkdir -p litellm-proxy && cd litellm-proxy
|
||||
touch .env && chmod 600 .env; grep -qx '.env' .gitignore 2>/dev/null || echo '.env' >> .gitignore
|
||||
grep -q '^LITELLM_MASTER_KEY=.' .env || printf 'LITELLM_MASTER_KEY=sk-%s\n' "$(openssl rand -hex 32)" >> .env
|
||||
```
|
||||
|
||||
Copy a provider key that is already in your environment only after the user agreed to that variable, in their step 1 answer or when you ask now, and copy it without printing it, as in `printf 'OPENAI_API_KEY=%s\n' "$OPENAI_API_KEY" >> .env`. Otherwise ask the user to add the line `OPENAI_API_KEY=<their key>` to `.env` themselves and tell you when it is there. If they paste a key into the chat anyway, write it to `.env` and never repeat it. Keep one `NAME=value` per line with no quotes, so `docker --env-file` and the loader line read the same value
|
||||
|
||||
Confirm each model's current LiteLLM name in the cost map instead of trusting memory, since model names go stale fast. This prints the chat models of the provider named last (`openai`, `anthropic`, `gemini`, `azure`, `bedrock`, `vertex_ai`, and so on) the way `config.yaml` takes them, with the provider in front. It matches by prefix, so `bedrock` also lists the newer Bedrock models and `vertex_ai` lists every Vertex family. For Ollama, use a model from `ollama list` instead:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json | python3 -c '
|
||||
import json, sys
|
||||
for name, row in json.load(sys.stdin).items():
|
||||
provider = str(row.get("litellm_provider", "")) if isinstance(row, dict) else ""
|
||||
if provider.startswith(sys.argv[1]) and row.get("mode") == "chat":
|
||||
prefix = "bedrock" if provider == "bedrock_converse" else "vertex_ai" if provider.startswith("vertex_ai") else provider
|
||||
print(name if name.startswith(prefix + "/") else prefix + "/" + name)
|
||||
' openai
|
||||
```
|
||||
|
||||
In `config.yaml`, `model` is a name from that list, as in `openai/gpt-6.1-sol` or `bedrock/us.anthropic.<name>`, and `model_name` is what clients will send. If the folder already has a `config.yaml`, show the user its `model_name` lines and ask before replacing it. This example uses the newest GPT model in the cost map when this skill was written, so replace it with what the user picked:
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: gpt-6.1-sol
|
||||
litellm_params:
|
||||
model: openai/gpt-6.1-sol
|
||||
api_key: os.environ/OPENAI_API_KEY
|
||||
|
||||
general_settings:
|
||||
master_key: os.environ/LITELLM_MASTER_KEY
|
||||
```
|
||||
|
||||
Add one entry per model. On Azure, ask the user for each deployment's name and the model it runs. Use `azure/<deployment name>` as `model`, since Azure routes by deployment and a name from the lookup works only when the deployment has that same name, and add `model_info:` with `base_model: azure/<model it runs>` to the entry, since LiteLLM prices each call by that name. Providers that need more than a key, like Azure or Bedrock, list their `litellm_params` on their page under https://docs.litellm.ai/docs/providers. Point every secret at `.env` through `os.environ/NAME` the same way
|
||||
|
||||
## 6. Their own Postgres, on the uv path
|
||||
|
||||
Ask the user to create a new, empty database for LiteLLM on their server (`CREATE DATABASE litellm;`) and add `DATABASE_URL=postgresql://...` for it to `.env` themselves. The proxy creates its tables there on first start and stops on a database that already holds other tables. If that URL goes through a transaction pooler such as PgBouncer, they also add `DIRECT_URL` for the same database without the pooler, which the migrations use. Then add a salt key. It encrypts the provider credentials the proxy stores in the database and falls back to the master key when unset, so set it once and never change it:
|
||||
|
||||
```bash
|
||||
grep -q '^LITELLM_SALT_KEY=.' .env || printf 'LITELLM_SALT_KEY=sk-%s\n' "$(openssl rand -hex 32)" >> .env
|
||||
```
|
||||
|
||||
Generate the Prisma client once inside the tool's environment. A pip or uv install doesn't ship it generated (Docker images do):
|
||||
|
||||
```bash
|
||||
tool="$(uv tool dir)/litellm"
|
||||
schema=$("$tool/bin/python" -I -c 'import importlib.util, os; print(os.path.join(importlib.util.find_spec("litellm").submodule_search_locations[0], "proxy", "schema.prisma"))')
|
||||
PATH="$tool/bin:$PATH" prisma generate --schema "$schema"
|
||||
```
|
||||
|
||||
## 7. Start the proxy
|
||||
|
||||
Bind to `127.0.0.1` unless the user wants other machines to reach it
|
||||
|
||||
uv, in one shell command that starts with the loader line, since the installed `litellm` does not read `.env` by itself:
|
||||
|
||||
```bash
|
||||
export PATH="$HOME/.local/bin:$PATH"; export PATH="$(uv tool dir --bin):$PATH"
|
||||
nohup litellm --config config.yaml --host 127.0.0.1 --port 4000 > litellm.log 2>&1 &
|
||||
echo $! > litellm.pid
|
||||
```
|
||||
|
||||
Docker:
|
||||
|
||||
```bash
|
||||
docker run -d --name litellm --env-file .env -v "$PWD/config.yaml:/app/config.yaml:ro" -p 127.0.0.1:4000:4000 \
|
||||
docker.litellm.ai/berriai/litellm:main-stable --config /app/config.yaml --port 4000
|
||||
```
|
||||
|
||||
Wait on the health route, stopping early if the proxy exits. The first start with Postgres applies the database migrations and takes longer. Define `alive` for the path you took, `alive() { kill -0 "$(cat litellm.pid)" 2>/dev/null; }` for uv or `alive() { [ "$(docker inspect -f '{{.State.Running}}' litellm 2>/dev/null)" = true ]; }` for Docker, in the same shell command as the loop:
|
||||
|
||||
```bash
|
||||
for i in $(seq 1 180); do curl -fsS http://127.0.0.1:4000/health/liveliness >/dev/null 2>&1 && break; alive || break; sleep 1; done
|
||||
curl -sS http://127.0.0.1:4000/health/readiness
|
||||
```
|
||||
|
||||
Readiness reports `"db":"connected"` with Postgres and `"db":"Not connected"` without it. If the proxy never comes up, read the end of `litellm.log` (or `docker logs litellm`) and check the troubleshooting list
|
||||
|
||||
## 8. Verify with a chat completion
|
||||
|
||||
Use a `model_name` from `config.yaml` (or step 3), in one shell command that starts with the loader line:
|
||||
|
||||
```bash
|
||||
curl -sS http://127.0.0.1:4000/v1/chat/completions \
|
||||
-H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
|
||||
-d '{"model": "gpt-6.1-sol", "messages": [{"role": "user", "content": "Say hello in five words."}]}'
|
||||
```
|
||||
|
||||
Show the user the reply in `choices[0].message.content`. An error here means the setup is not done, so fix it before handing off
|
||||
|
||||
## 9. Hand off
|
||||
|
||||
Tell the user, without printing any secret:
|
||||
|
||||
- The base URL is `http://127.0.0.1:4000`. Any OpenAI-compatible client works with it as the base URL and the master key or a virtual key as the API key
|
||||
- The master key is `LITELLM_MASTER_KEY` in the folder's `.env`, or `general_settings.master_key` in `litellm_config.yaml` when they used `litellm --setup`
|
||||
- To stop it, `kill "$(cat litellm.pid)"` (uv), `docker stop litellm` (Docker), `<folder>/litellm-gateway stop` (quickstart), or Ctrl+C in its window (`litellm --setup`). To start it again, `docker start litellm`, `<folder>/litellm-gateway start`, on uv the whole step 7 command as you ran it, loader line and PATH export included, wrapped in `( ... )` so their shell keeps its own settings, shown as a code block and never paraphrased, since a new shell has neither, or for `litellm --setup` `LITELLM_MODE=PRODUCTION <bin>/litellm --config litellm_config.yaml --host 127.0.0.1 --port <port>` in a terminal window in that folder, with `<bin>` and `<port>` filled in, since running the wizard again writes a new config with a new master key. `update` and `uninstall` update or remove a quickstart install. Give the quickstart commands with the folder's full path, since a bare `litellm-gateway` is on PATH only when `~/.local/bin` is, and can belong to an earlier install
|
||||
- With Postgres, the Admin UI is at `http://127.0.0.1:4000/ui` (user `admin`, password the master key), and `POST /key/generate` with the master key creates virtual keys with their own budgets and model lists. Without Postgres, both need step 3 or 6 first
|
||||
- The Admin UI warns that the master-key login is a shared secret, which is fine while only this machine reaches the proxy. Before anyone else can reach it, invite an admin with their own password on the Internal Users page, then set `general_settings.disable_env_credential_login: true` in the proxy config
|
||||
- To manage the running proxy from a coding agent (keys, teams, users, models, MCP servers), install `npx skills add BerriAI/litellm-skills -g`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- `refused to start: no master key is set` means the loader line did not run in the shell command that started `litellm`, or `--env-file .env` is missing from `docker run`
|
||||
- `a database URL is set but the prisma CLI is neither on PATH nor importable` means the install lacks `extra_proxy`, so rerun step 4 with it, or, on plain routing, that the command that started `litellm` did not start with the loader line
|
||||
- `The Client hasn't been generated yet` or `Unable to find Prisma binaries` means the Prisma generate command in step 6 has not run
|
||||
- `Incorrect API key provided` or another provider `AuthenticationError` means a provider key is wrong. Fix it in `.env` and restart, since the proxy reads the environment only at startup. On uv, stop it and rerun step 7. On Docker, `docker rm -f litellm` and rerun step 7, since `docker restart` keeps the environment the container was created with. On the quickstart the key is saved with the model, so delete the model and add it again: `GET /model/info` lists each model's `model_info.id`, and `POST /model/delete` with `{"id": "<id>"}` deletes it
|
||||
- `Cannot automatically baseline this database` means `DATABASE_URL` points at a database that already holds tables LiteLLM did not create. Point it at a new, empty database
|
||||
- `LLM Provider NOT provided` means a `model` in `config.yaml` (or step 3) lacks its provider in front. Use the name as the step 5 lookup prints it
|
||||
- `Invalid model name passed in model=...` means the request's `model` is not a `model_name` in `config.yaml` or step 3. `GET /v1/models` lists the ones the key can use
|
||||
- `No connected db.` on a proxy without Postgres means the bearer token is not the master key
|
||||
- `Conflict. The container name "/litellm" is already in use` means an earlier run left a container. `docker start litellm` reuses it, or with the user's OK `docker rm -f litellm` and rerun step 7
|
||||
- Inside Docker, `localhost` in `config.yaml` (an Ollama `api_base`, say) is the container itself. Use `host.docker.internal`, and on Linux also pass `--add-host=host.docker.internal:host-gateway`
|
||||
2
.github/CODEOWNERS
vendored
2
.github/CODEOWNERS
vendored
|
|
@ -1,6 +1,6 @@
|
|||
/model_prices_and_context_window.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
|
||||
/litellm/model_prices_and_context_window_backup.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
|
||||
/ci_cd/cost_map_pins.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
|
||||
/litellm/scripts/ci_cd/cost_map_pins.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
|
||||
/litellm/deploy/helm/ @yassin-berriai
|
||||
/litellm/deploy/terraform/ @yassin-berriai
|
||||
/.github/workflows/zizmor.yml @yuneng-berri
|
||||
|
|
|
|||
2
.github/CONTRIBUTING.md
vendored
2
.github/CONTRIBUTING.md
vendored
|
|
@ -360,7 +360,7 @@ Local lint and budget checks follow origin's current default branch. They refres
|
|||
## Submitting Your PR
|
||||
|
||||
1. **Push your branch**: `git push origin your-feature-branch`
|
||||
2. **Create a PR**: Go to GitHub and open a pull request against the repository's current default branch. Run `python3 scripts/default_branch.py --branch` to check its name
|
||||
2. **Create a PR**: Go to GitHub and open a pull request against the repository's current default branch. Run `python3 litellm/scripts/default_branch.py --branch` to check its name
|
||||
3. **Fill out the PR template**: Provide clear description of changes
|
||||
4. **Wait for review**: Maintainers will review and provide feedback
|
||||
5. **Address feedback**: Make requested changes and push updates
|
||||
|
|
|
|||
|
|
@ -11,6 +11,11 @@ inputs:
|
|||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Configure the Rust build environment
|
||||
shell: bash
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: bash .github/scripts/setup_rust_environment.sh
|
||||
|
||||
- name: Set up uv (attempt 1)
|
||||
id: attempt-1
|
||||
continue-on-error: true
|
||||
|
|
|
|||
0
.gitguardian.yaml → .github/gitguardian.yaml
vendored
0
.gitguardian.yaml → .github/gitguardian.yaml
vendored
4
.github/scripts/ci/classify_changes.sh
vendored
4
.github/scripts/ci/classify_changes.sh
vendored
|
|
@ -19,11 +19,11 @@ while IFS= read -r file || [ -n "$file" ]; do
|
|||
esac
|
||||
case "$file" in
|
||||
*.md | *.mdx) : ;;
|
||||
pyproject.toml | */pyproject.toml | uv.lock | uv.toml | .python-version | rust-toolchain.toml | litellm/rust/* | litellm/__init__.py | litellm/proxy/proxy_server.py | litellm/*mcp* | tests/*mcp* | litellm/integrations/arize/* | tests/base_sdk_tests/* | scripts/check_mcp_sdk_install.py | .github/workflows/test-mcp-dependency-resolution.yml | .github/actions/detect-changes/* | .github/actions/setup-uv-with-retries/* | .github/actions/cache-cargo-build/* | .github/scripts/detect_changes.sh | .github/scripts/uv_sync_with_retries.sh | .github/scripts/ci/classify_changes.sh | tests/unit/test_ci_path_filter.py | tests/unit/test_detect_changes.py)
|
||||
pyproject.toml | */pyproject.toml | uv.lock | uv.toml | .python-version | litellm/rust/* | litellm/__init__.py | litellm/proxy/proxy_server.py | litellm/*mcp* | tests/*mcp* | litellm/integrations/arize/* | tests/base_sdk_tests/* | litellm/scripts/check_mcp_sdk_install.py | .github/workflows/test-mcp-dependency-resolution.yml | .github/actions/detect-changes/* | .github/actions/setup-uv-with-retries/* | .github/actions/cache-cargo-build/* | .github/scripts/detect_changes.sh | .github/scripts/uv_sync_with_retries.sh | .github/scripts/ci/classify_changes.sh | tests/unit/test_ci_path_filter.py | tests/unit/test_detect_changes.py)
|
||||
has_mcp_dependencies=true ;;
|
||||
esac
|
||||
case "$file" in
|
||||
litellm/rust/* | litellm/rust_bridge/* | rust-toolchain.toml | pyproject.toml | uv.lock | tests/windows_tests/* | .github/scripts/ci/* | .github/actions/*)
|
||||
litellm/rust/* | litellm/rust_bridge/* | pyproject.toml | uv.lock | tests/windows_tests/* | .github/scripts/ci/* | .github/actions/*)
|
||||
has_windows_release=true ;;
|
||||
esac
|
||||
case "$file" in
|
||||
|
|
|
|||
13
.github/scripts/setup_rust_environment.sh
vendored
Normal file
13
.github/scripts/setup_rust_environment.sh
vendored
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
root=$(git rev-parse --show-toplevel)
|
||||
toolchain=$(sed -n 's/^[[:space:]]*channel[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p' "$root/litellm/rust/rust-toolchain.toml")
|
||||
test -n "$toolchain"
|
||||
rustup toolchain install "$toolchain" --profile minimal --component rustfmt --component clippy --no-self-update
|
||||
{
|
||||
echo "RUSTUP_TOOLCHAIN=$toolchain"
|
||||
echo "CARGO_NET_RETRY=5"
|
||||
echo "CARGO_HTTP_MULTIPLEXING=false"
|
||||
echo "SQLX_OFFLINE=${SQLX_OFFLINE:-true}"
|
||||
} >> "${GITHUB_ENV:?GITHUB_ENV is required}"
|
||||
3
.github/scripts/verify_linux_native_wheel.py
vendored
3
.github/scripts/verify_linux_native_wheel.py
vendored
|
|
@ -137,6 +137,9 @@ def main(
|
|||
"litellm/proxy-extras/",
|
||||
"litellm/rust/",
|
||||
"litellm/packaging/",
|
||||
"litellm/scripts/",
|
||||
"litellm/runtime/",
|
||||
"litellm/schema.prisma",
|
||||
)
|
||||
)
|
||||
or any(part.endswith(".dSYM") for part in PurePosixPath(member.filename).parts)
|
||||
|
|
|
|||
8
.github/workflows/auto-close-duplicates.yml
vendored
8
.github/workflows/auto-close-duplicates.yml
vendored
|
|
@ -16,8 +16,8 @@ on:
|
|||
pull_request:
|
||||
paths:
|
||||
- .github/workflows/auto-close-duplicates.yml
|
||||
- scripts/auto-close-duplicates.ts
|
||||
- scripts/auto-close-duplicates.test.ts
|
||||
- litellm/scripts/auto-close-duplicates.ts
|
||||
- litellm/scripts/auto-close-duplicates.test.ts
|
||||
|
||||
permissions: {}
|
||||
|
||||
|
|
@ -40,7 +40,7 @@ jobs:
|
|||
bun-version: "1.4.0"
|
||||
|
||||
- name: Test the sweep
|
||||
run: bun test scripts/auto-close-duplicates.test.ts
|
||||
run: bun test litellm/scripts/auto-close-duplicates.test.ts
|
||||
|
||||
sweep:
|
||||
if: github.event_name != 'pull_request' && github.repository == 'BerriAI/litellm'
|
||||
|
|
@ -62,7 +62,7 @@ jobs:
|
|||
bun-version: "1.4.0"
|
||||
|
||||
- name: Close unanswered duplicates, reopen ones the reporter answered
|
||||
run: bun run scripts/auto-close-duplicates.ts
|
||||
run: bun run litellm/scripts/auto-close-duplicates.ts
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
DRY_RUN: ${{ inputs.dry_run == true }}
|
||||
|
|
|
|||
20
.github/workflows/check-schema-sync.yml
vendored
20
.github/workflows/check-schema-sync.yml
vendored
|
|
@ -3,7 +3,7 @@ name: Check Schema Sync
|
|||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- 'schema.prisma'
|
||||
- 'litellm/schema.prisma'
|
||||
- 'litellm/proxy/schema.prisma'
|
||||
- 'litellm/proxy-extras/litellm_proxy_extras/schema.prisma'
|
||||
|
||||
|
|
@ -16,7 +16,7 @@ concurrency:
|
|||
|
||||
jobs:
|
||||
check-sync:
|
||||
name: Verify schema.prisma copies match root
|
||||
name: Verify litellm/schema.prisma copies match the canonical schema
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
|
|
@ -27,7 +27,7 @@ jobs:
|
|||
|
||||
- name: Reject symlinked schema files
|
||||
run: |
|
||||
for f in schema.prisma litellm/proxy/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma; do
|
||||
for f in litellm/schema.prisma litellm/proxy/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma; do
|
||||
if [ -L "$f" ]; then
|
||||
echo "::error file=$f::$f is a symlink, which is not allowed"
|
||||
exit 1
|
||||
|
|
@ -38,24 +38,24 @@ jobs:
|
|||
run: |
|
||||
EXIT=0
|
||||
|
||||
diff schema.prisma litellm/proxy/schema.prisma || {
|
||||
echo "::error file=litellm/proxy/schema.prisma::litellm/proxy/schema.prisma differs from root schema.prisma"
|
||||
diff litellm/schema.prisma litellm/proxy/schema.prisma || {
|
||||
echo "::error file=litellm/proxy/schema.prisma::litellm/proxy/schema.prisma differs from root litellm/schema.prisma"
|
||||
EXIT=1
|
||||
}
|
||||
|
||||
diff schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma || {
|
||||
echo "::error file=litellm/proxy-extras/litellm_proxy_extras/schema.prisma::litellm/proxy-extras/litellm_proxy_extras/schema.prisma differs from root schema.prisma"
|
||||
diff litellm/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma || {
|
||||
echo "::error file=litellm/proxy-extras/litellm_proxy_extras/schema.prisma::litellm/proxy-extras/litellm_proxy_extras/schema.prisma differs from root litellm/schema.prisma"
|
||||
EXIT=1
|
||||
}
|
||||
|
||||
if [ "$EXIT" -ne 0 ]; then
|
||||
echo ""
|
||||
echo "Schema files are out of sync."
|
||||
echo "The root schema.prisma is the source of truth."
|
||||
echo "The root litellm/schema.prisma is the source of truth."
|
||||
echo ""
|
||||
echo "To fix, run from the repo root:"
|
||||
echo " cp schema.prisma litellm/proxy/schema.prisma"
|
||||
echo " cp schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma"
|
||||
echo " cp litellm/schema.prisma litellm/proxy/schema.prisma"
|
||||
echo " cp litellm/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
|
|
|||
4
.github/workflows/conventional-commits.yml
vendored
4
.github/workflows/conventional-commits.yml
vendored
|
|
@ -3,7 +3,7 @@ name: Conventional PR Title
|
|||
# Squash-merge replaces the merge commit subject with the PR title, so
|
||||
# enforcing Conventional Commits at the PR-title level is what actually gates
|
||||
# the commits that land on the default branch. The local commit-msg hook
|
||||
# (scripts/githooks/commit-msg) is a best-effort assist; this workflow is the gate.
|
||||
# (litellm/scripts/githooks/commit-msg) is a best-effort assist; this workflow is the gate.
|
||||
#
|
||||
# See https://www.conventionalcommits.org/en/v1.0.0/
|
||||
|
||||
|
|
@ -28,7 +28,7 @@ jobs:
|
|||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
with:
|
||||
# Must mirror the type list in scripts/githooks/commit-msg.
|
||||
# Must mirror the type list in litellm/scripts/githooks/commit-msg.
|
||||
types: |
|
||||
feat
|
||||
fix
|
||||
|
|
|
|||
5
.github/workflows/cost-map-guard.yml
vendored
5
.github/workflows/cost-map-guard.yml
vendored
|
|
@ -49,8 +49,11 @@ jobs:
|
|||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
HEAD_REF: ${{ github.event.pull_request.head.ref }}
|
||||
run: |
|
||||
guard=scripts/ci_cd/cost_map_guard.py
|
||||
guard=litellm/scripts/ci_cd/cost_map_guard.py
|
||||
# TODO: Drop the legacy path after this lands on main.
|
||||
if [ ! -f "$guard" ]; then
|
||||
guard=scripts/ci_cd/cost_map_guard.py
|
||||
fi
|
||||
if [ ! -f "$guard" ]; then
|
||||
guard=ci_cd/cost_map_guard.py
|
||||
fi
|
||||
|
|
|
|||
6
.github/workflows/cost-map-pins-recheck.yml
vendored
6
.github/workflows/cost-map-pins-recheck.yml
vendored
|
|
@ -8,7 +8,7 @@ on:
|
|||
paths:
|
||||
- model_prices_and_context_window.json
|
||||
- litellm/model_prices_and_context_window_backup.json
|
||||
- ci_cd/cost_map_pins.json
|
||||
- litellm/scripts/ci_cd/cost_map_pins.json
|
||||
|
||||
permissions: {}
|
||||
|
||||
|
|
@ -28,8 +28,8 @@ jobs:
|
|||
BEFORE_SHA: ${{ github.event.before }}
|
||||
AFTER_SHA: ${{ github.event.after }}
|
||||
FORCED: ${{ github.event.forced }}
|
||||
COST_MAP_PATHS: '["model_prices_and_context_window.json", "litellm/model_prices_and_context_window_backup.json", "ci_cd/cost_map_pins.json"]'
|
||||
PINS_PATHS: '["ci_cd/cost_map_pins.json"]'
|
||||
COST_MAP_PATHS: '["model_prices_and_context_window.json", "litellm/model_prices_and_context_window_backup.json", "litellm/scripts/ci_cd/cost_map_pins.json"]'
|
||||
PINS_PATHS: '["litellm/scripts/ci_cd/cost_map_pins.json"]'
|
||||
run: |
|
||||
if [[ "$BEFORE_SHA" =~ ^0+$ || "$FORCED" == true ]] \
|
||||
|| ! gh api "repos/${GITHUB_REPOSITORY}/compare/${BEFORE_SHA}...${AFTER_SHA}?per_page=1" \
|
||||
|
|
|
|||
12
.github/workflows/duplicate_issue_check.yml
vendored
12
.github/workflows/duplicate_issue_check.yml
vendored
|
|
@ -13,9 +13,9 @@ on:
|
|||
- .github/workflows/duplicate_issue_check.yml
|
||||
- .github/prompts/duplicate-issue-check.md
|
||||
- .github/prompts/duplicate-issue-check.schema.json
|
||||
- scripts/flag-duplicate-issue.ts
|
||||
- scripts/flag-duplicate-issue.test.ts
|
||||
- scripts/auto-close-duplicates.ts
|
||||
- litellm/scripts/flag-duplicate-issue.ts
|
||||
- litellm/scripts/flag-duplicate-issue.test.ts
|
||||
- litellm/scripts/auto-close-duplicates.ts
|
||||
|
||||
permissions: {}
|
||||
|
||||
|
|
@ -38,7 +38,7 @@ jobs:
|
|||
bun-version: "1.4.0"
|
||||
|
||||
- name: Test the flag step
|
||||
run: bun test scripts/flag-duplicate-issue.test.ts
|
||||
run: bun test litellm/scripts/flag-duplicate-issue.test.ts
|
||||
|
||||
classify:
|
||||
if: github.event_name != 'pull_request' && github.repository == 'BerriAI/litellm'
|
||||
|
|
@ -125,7 +125,7 @@ jobs:
|
|||
- name: Checkout scripts
|
||||
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
|
||||
with:
|
||||
sparse-checkout: scripts
|
||||
sparse-checkout: litellm/scripts
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup Bun
|
||||
|
|
@ -134,7 +134,7 @@ jobs:
|
|||
bun-version: "1.4.0"
|
||||
|
||||
- name: Comment and label
|
||||
run: bun run scripts/flag-duplicate-issue.ts
|
||||
run: bun run litellm/scripts/flag-duplicate-issue.ts
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VERDICT: ${{ needs.classify.outputs.verdict }}
|
||||
|
|
|
|||
13
.github/workflows/image-scan.yml
vendored
13
.github/workflows/image-scan.yml
vendored
|
|
@ -10,13 +10,14 @@ on:
|
|||
- litellm/deploy/docker/Dockerfile
|
||||
- litellm/deploy/docker/Dockerfile.dockerignore
|
||||
- litellm/deploy/docker/docker-entrypoint.sh
|
||||
- migrations/run.py
|
||||
- gateway/main.py
|
||||
- gateway/launch.py
|
||||
- gateway/routes/allowlist.py
|
||||
- litellm/runtime/migrations/run.py
|
||||
- litellm/runtime/gateway/main.py
|
||||
- litellm/runtime/gateway/launch.py
|
||||
- litellm/runtime/gateway/routes/allowlist.py
|
||||
- litellm/cli/**
|
||||
- backend/main.py
|
||||
- backend/routes/allowlist.py
|
||||
- .github/packaging/**
|
||||
- litellm/runtime/backend/main.py
|
||||
- litellm/runtime/backend/routes/allowlist.py
|
||||
- litellm/rust/**
|
||||
- litellm/proxy/lens/**
|
||||
- litellm/proxy/prisma_migration.py
|
||||
|
|
|
|||
22
.github/workflows/issue_classifier.yml
vendored
22
.github/workflows/issue_classifier.yml
vendored
|
|
@ -16,12 +16,12 @@ on:
|
|||
- .github/issue-labels.json
|
||||
- .github/ISSUE_TEMPLATE/bug_report.yml
|
||||
- .github/ISSUE_TEMPLATE/feature_request.yml
|
||||
- scripts/classify-issue.ts
|
||||
- scripts/classify-issue.test.ts
|
||||
- scripts/label-issue.ts
|
||||
- scripts/label-issue.test.ts
|
||||
- scripts/issue-labels.ts
|
||||
- scripts/auto-close-duplicates.ts
|
||||
- litellm/scripts/classify-issue.ts
|
||||
- litellm/scripts/classify-issue.test.ts
|
||||
- litellm/scripts/label-issue.ts
|
||||
- litellm/scripts/label-issue.test.ts
|
||||
- litellm/scripts/issue-labels.ts
|
||||
- litellm/scripts/auto-close-duplicates.ts
|
||||
|
||||
permissions: {}
|
||||
|
||||
|
|
@ -49,7 +49,7 @@ jobs:
|
|||
bun-version: "1.4.0"
|
||||
|
||||
- name: Test the gate, the validation and the label step
|
||||
run: bun test scripts/classify-issue.test.ts scripts/label-issue.test.ts
|
||||
run: bun test litellm/scripts/classify-issue.test.ts litellm/scripts/label-issue.test.ts
|
||||
|
||||
classify-issue:
|
||||
# An edit to a labelled issue is dropped here; the script decides the rest against the live labels
|
||||
|
|
@ -73,7 +73,7 @@ jobs:
|
|||
with:
|
||||
sparse-checkout: |
|
||||
.github
|
||||
scripts
|
||||
litellm/scripts
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup Bun
|
||||
|
|
@ -108,7 +108,7 @@ jobs:
|
|||
ISSUE_CLASSIFIER_MODEL: ${{ vars.ISSUE_CLASSIFIER_MODEL }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
bun run scripts/classify-issue.ts > classification.json
|
||||
bun run litellm/scripts/classify-issue.ts > classification.json
|
||||
{
|
||||
echo 'verdict<<CLASSIFICATION'
|
||||
cat classification.json
|
||||
|
|
@ -143,7 +143,7 @@ jobs:
|
|||
with:
|
||||
sparse-checkout: |
|
||||
.github
|
||||
scripts
|
||||
litellm/scripts
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup Bun
|
||||
|
|
@ -153,7 +153,7 @@ jobs:
|
|||
bun-version: "1.4.0"
|
||||
|
||||
- name: Replace the labels in each namespace
|
||||
run: bun run scripts/label-issue.ts
|
||||
run: bun run litellm/scripts/label-issue.ts
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VERDICT: ${{ needs.classify-issue.outputs.verdict }}
|
||||
|
|
|
|||
12
.github/workflows/issue_fixed_comment.yml
vendored
12
.github/workflows/issue_fixed_comment.yml
vendored
|
|
@ -15,9 +15,9 @@ on:
|
|||
pull_request:
|
||||
paths:
|
||||
- .github/workflows/issue_fixed_comment.yml
|
||||
- scripts/comment-fixed-issue.ts
|
||||
- scripts/comment-fixed-issue.test.ts
|
||||
- scripts/auto-close-duplicates.ts
|
||||
- litellm/scripts/comment-fixed-issue.ts
|
||||
- litellm/scripts/comment-fixed-issue.test.ts
|
||||
- litellm/scripts/auto-close-duplicates.ts
|
||||
|
||||
permissions: {}
|
||||
|
||||
|
|
@ -44,7 +44,7 @@ jobs:
|
|||
bun-version: "1.4.0"
|
||||
|
||||
- name: Test the closer lookup, the release placement, the comment and the superseded pull request close
|
||||
run: bun test scripts/comment-fixed-issue.test.ts
|
||||
run: bun test litellm/scripts/comment-fixed-issue.test.ts
|
||||
|
||||
comment-fixed-issue:
|
||||
if: github.event_name != 'pull_request' && github.repository == 'BerriAI/litellm'
|
||||
|
|
@ -58,7 +58,7 @@ jobs:
|
|||
- name: Checkout scripts
|
||||
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
|
||||
with:
|
||||
sparse-checkout: scripts
|
||||
sparse-checkout: litellm/scripts
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup Bun
|
||||
|
|
@ -69,7 +69,7 @@ jobs:
|
|||
|
||||
- name: Name the release that carries the fix and close the pull requests it supersedes
|
||||
shell: bash
|
||||
run: bun run scripts/comment-fixed-issue.ts | tee -a "${GITHUB_STEP_SUMMARY}"
|
||||
run: bun run litellm/scripts/comment-fixed-issue.ts | tee -a "${GITHUB_STEP_SUMMARY}"
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
ISSUE_NUMBER: ${{ github.event.issue.number || github.event.inputs.issue_number }}
|
||||
|
|
|
|||
14
.github/workflows/issue_label_sync.yml
vendored
14
.github/workflows/issue_label_sync.yml
vendored
|
|
@ -5,7 +5,7 @@ on:
|
|||
branches: [main]
|
||||
paths:
|
||||
- .github/issue-labels.json
|
||||
- scripts/sync-issue-labels.ts
|
||||
- litellm/scripts/sync-issue-labels.ts
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
dry_run:
|
||||
|
|
@ -16,9 +16,9 @@ on:
|
|||
paths:
|
||||
- .github/workflows/issue_label_sync.yml
|
||||
- .github/issue-labels.json
|
||||
- scripts/sync-issue-labels.ts
|
||||
- scripts/sync-issue-labels.test.ts
|
||||
- scripts/issue-labels.ts
|
||||
- litellm/scripts/sync-issue-labels.ts
|
||||
- litellm/scripts/sync-issue-labels.test.ts
|
||||
- litellm/scripts/issue-labels.ts
|
||||
|
||||
permissions: {}
|
||||
|
||||
|
|
@ -41,7 +41,7 @@ jobs:
|
|||
bun-version: "1.4.0"
|
||||
|
||||
- name: Test the sync
|
||||
run: bun test scripts/sync-issue-labels.test.ts
|
||||
run: bun test litellm/scripts/sync-issue-labels.test.ts
|
||||
|
||||
sync-issue-labels:
|
||||
if: github.event_name != 'pull_request' && github.repository == 'BerriAI/litellm'
|
||||
|
|
@ -56,7 +56,7 @@ jobs:
|
|||
with:
|
||||
sparse-checkout: |
|
||||
.github
|
||||
scripts
|
||||
litellm/scripts
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup Bun
|
||||
|
|
@ -66,7 +66,7 @@ jobs:
|
|||
bun-version: "1.4.0"
|
||||
|
||||
- name: Create or recolour every label in .github/issue-labels.json
|
||||
run: bun run scripts/sync-issue-labels.ts
|
||||
run: bun run litellm/scripts/sync-issue-labels.ts
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run == true }}
|
||||
|
|
|
|||
2
.github/workflows/mutation-test.yml
vendored
2
.github/workflows/mutation-test.yml
vendored
|
|
@ -129,7 +129,7 @@ jobs:
|
|||
set +e
|
||||
uv run --no-sync --with mutmut==3.5.0 mutmut export-cicd-stats > /dev/null 2>&1
|
||||
uv run --no-sync --with mutmut==3.5.0 mutmut results > mutmut-results.txt 2>&1
|
||||
uv run --no-sync python scripts/mutation_report.py
|
||||
uv run --no-sync python litellm/scripts/mutation_report.py
|
||||
# The full report can be very long for big test files; the run-page
|
||||
# summary cuts off at 1 MB. Append the head of the report (summary
|
||||
# + survivor list) and link out to the artifact for the full body.
|
||||
|
|
|
|||
14
.github/workflows/publish-lint-base-counts.yml
vendored
14
.github/workflows/publish-lint-base-counts.yml
vendored
|
|
@ -3,7 +3,7 @@ name: Publish lint base counts
|
|||
# Every commit on main can become a future merge-base.
|
||||
# Publishing its per-rule counts for each lint gate (strict ruff, type discipline,
|
||||
# test quality, basedpyright) as an artifact lets the gates download them through
|
||||
# scripts/lint_base_counts.py in seconds instead of scanning the merge-base tree in
|
||||
# litellm/scripts/lint_base_counts.py in seconds instead of scanning the merge-base tree in
|
||||
# a throwaway worktree on every fresh checkout or moved merge-base.
|
||||
# No concurrency group on purpose: runs must never cancel each other, because
|
||||
# every sha's artifact matters (any of them can become a merge-base).
|
||||
|
|
@ -30,15 +30,15 @@ jobs:
|
|||
matrix:
|
||||
include:
|
||||
- checker: ruff-strict
|
||||
script: scripts/ruff_strict_gate.py
|
||||
script: litellm/scripts/ruff_strict_gate.py
|
||||
- checker: type-discipline
|
||||
script: scripts/type_discipline_gate.py
|
||||
script: litellm/scripts/type_discipline_gate.py
|
||||
- checker: test-quality
|
||||
script: scripts/test_quality_gate.py
|
||||
script: litellm/scripts/test_quality_gate.py
|
||||
- checker: openapi-docs
|
||||
script: scripts/openapi_docs_gate.py
|
||||
script: litellm/scripts/openapi_docs_gate.py
|
||||
- checker: basedpyright
|
||||
script: scripts/type_check_gate.py
|
||||
script: litellm/scripts/type_check_gate.py
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
|
||||
|
|
@ -91,7 +91,7 @@ jobs:
|
|||
- name: Emit basedpyright counts for HEAD
|
||||
if: matrix.checker == 'basedpyright'
|
||||
run: |
|
||||
python scripts/type_check_gate.py --emit-counts-dir "$RUNNER_TEMP/lint-counts"
|
||||
python litellm/scripts/type_check_gate.py --emit-counts-dir "$RUNNER_TEMP/lint-counts"
|
||||
|
||||
- name: Name the counts artifact
|
||||
run: |
|
||||
|
|
|
|||
5
.github/workflows/spend-logs-guard.yml
vendored
5
.github/workflows/spend-logs-guard.yml
vendored
|
|
@ -42,8 +42,11 @@ jobs:
|
|||
MERGE_BASE: ${{ steps.revisions.outputs.merge_base }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
guard=scripts/ci_cd/spend_logs_guard.py
|
||||
guard=litellm/scripts/ci_cd/spend_logs_guard.py
|
||||
# TODO: Drop the legacy path after this lands on main.
|
||||
if [ ! -f "$guard" ]; then
|
||||
guard=scripts/ci_cd/spend_logs_guard.py
|
||||
fi
|
||||
if [ ! -f "$guard" ]; then
|
||||
guard=ci_cd/spend_logs_guard.py
|
||||
fi
|
||||
|
|
|
|||
12
.github/workflows/sync-schema.yml
vendored
12
.github/workflows/sync-schema.yml
vendored
|
|
@ -1,9 +1,9 @@
|
|||
name: Sync schema.prisma copies
|
||||
name: Sync litellm/schema.prisma copies
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- 'schema.prisma'
|
||||
- 'litellm/schema.prisma'
|
||||
|
||||
# Scoped to ONLY the permissions needed:
|
||||
# - contents:write to push the sync commit to the PR branch
|
||||
|
|
@ -37,7 +37,7 @@ jobs:
|
|||
|
||||
- name: Reject symlinked schema files
|
||||
run: |
|
||||
for f in schema.prisma litellm/proxy/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma; do
|
||||
for f in litellm/schema.prisma litellm/proxy/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma; do
|
||||
if [ -L "$f" ]; then
|
||||
echo "::error file=$f::$f is a symlink, which is not allowed"
|
||||
exit 1
|
||||
|
|
@ -46,8 +46,8 @@ jobs:
|
|||
|
||||
- name: Copy root schema to other locations
|
||||
run: |
|
||||
cp schema.prisma litellm/proxy/schema.prisma
|
||||
cp schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma
|
||||
cp litellm/schema.prisma litellm/proxy/schema.prisma
|
||||
cp litellm/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma
|
||||
|
||||
- name: Check for changes
|
||||
id: diff
|
||||
|
|
@ -69,5 +69,5 @@ jobs:
|
|||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git checkout -B "$GITHUB_HEAD_REF"
|
||||
git add -- litellm/proxy/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma
|
||||
git commit -m "chore: sync schema.prisma copies from root"
|
||||
git commit -m "chore: sync litellm/schema.prisma copies from the canonical schema"
|
||||
git push origin "HEAD:$GITHUB_HEAD_REF"
|
||||
|
|
|
|||
20
.github/workflows/test-linting.yml
vendored
20
.github/workflows/test-linting.yml
vendored
|
|
@ -114,7 +114,7 @@ jobs:
|
|||
- name: Check ruff format
|
||||
if: steps.changes.outputs.decision != 'skip'
|
||||
run: |
|
||||
git diff --name-only --diff-filter=ACMR "$GATE_BASE_SHA" HEAD -- ':(glob)litellm/**/*.py' | grep -Ev '^litellm/(enterprise|ui|vscode-extension|rust|proxy-extras|packaging|cli)/' > "$RUNNER_TEMP/ruff_format_files.txt" || true
|
||||
git diff --name-only --diff-filter=ACMR "$GATE_BASE_SHA" HEAD -- ':(glob)litellm/**/*.py' | grep -Ev '^litellm/(enterprise|ui|vscode-extension|rust|proxy-extras|packaging|cli|scripts|runtime)/' > "$RUNNER_TEMP/ruff_format_files.txt" || true
|
||||
if [ ! -s "$RUNNER_TEMP/ruff_format_files.txt" ]; then
|
||||
echo "No changed litellm Python files to check with ruff format."
|
||||
exit 0
|
||||
|
|
@ -133,7 +133,7 @@ jobs:
|
|||
|
||||
- name: Check MCP operation boundary
|
||||
if: steps.changes.outputs.decision != 'skip'
|
||||
run: uv run --no-sync python scripts/check_mcp_operation_boundary.py
|
||||
run: uv run --no-sync python litellm/scripts/check_mcp_operation_boundary.py
|
||||
|
||||
- name: Run Ruff linting
|
||||
if: steps.changes.outputs.decision != 'skip'
|
||||
|
|
@ -152,21 +152,21 @@ jobs:
|
|||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
uv run --no-sync python scripts/ruff_strict_gate.py --base "$GATE_BASE_SHA"
|
||||
uv run --no-sync python litellm/scripts/ruff_strict_gate.py --base "$GATE_BASE_SHA"
|
||||
|
||||
- name: Check type discipline (mutable collections / casts / type guards / kwargs / unexplained suppressions, delta vs merge-base counts)
|
||||
if: steps.changes.outputs.decision != 'skip'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
uv run --no-sync python scripts/type_discipline_gate.py --base "$GATE_BASE_SHA"
|
||||
uv run --no-sync python litellm/scripts/type_discipline_gate.py --base "$GATE_BASE_SHA"
|
||||
|
||||
- name: Check test quality (zero-assert / mock-echo tests, sys.path.insert, raw env writes, litellm global mutation, credential-gated skips, conftest snapshot inventory, delta vs merge-base counts)
|
||||
if: steps.changes.outputs.decision != 'skip'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
uv run --no-sync python scripts/test_quality_gate.py --base "$GATE_BASE_SHA"
|
||||
uv run --no-sync python litellm/scripts/test_quality_gate.py --base "$GATE_BASE_SHA"
|
||||
|
||||
- name: Check OpenAPI docs (POST routes without a request schema, description or 200 schema, delta vs merge-base counts)
|
||||
if: steps.changes.outputs.decision != 'skip'
|
||||
|
|
@ -174,7 +174,7 @@ jobs:
|
|||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
uv sync --frozen --group proxy-dev --group e2e-dev --extra proxy
|
||||
uv run --no-sync python scripts/openapi_docs_gate.py --base "$GATE_BASE_SHA"
|
||||
uv run --no-sync python litellm/scripts/openapi_docs_gate.py --base "$GATE_BASE_SHA"
|
||||
|
||||
- name: Print OpenAI version
|
||||
if: steps.changes.outputs.decision != 'skip'
|
||||
|
|
@ -186,7 +186,7 @@ jobs:
|
|||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
uv run --no-sync python scripts/type_check_gate.py --base "$GATE_BASE_SHA"
|
||||
uv run --no-sync python litellm/scripts/type_check_gate.py --base "$GATE_BASE_SHA"
|
||||
|
||||
- name: Check tests/e2e basedpyright (zero errors)
|
||||
if: steps.changes.outputs.decision != 'skip'
|
||||
|
|
@ -258,7 +258,7 @@ jobs:
|
|||
if [ -n "$GITGUARDIAN_API_KEY" ]; then
|
||||
retry() { "$@" || { sleep 15; "$@"; } || { sleep 30; "$@"; }; }
|
||||
retry git fetch --no-tags --unshallow origin
|
||||
uv tool run --from 'ggshield==1.48.0' ggshield secret scan repo .
|
||||
uv tool run --from 'ggshield==1.48.0' ggshield --config-path .github/gitguardian.yaml secret scan repo .
|
||||
else
|
||||
echo "GITGUARDIAN_API_KEY not set, skipping ggshield scan"
|
||||
fi
|
||||
|
|
@ -375,7 +375,7 @@ jobs:
|
|||
exit 0
|
||||
fi
|
||||
files="$(git diff --name-only "$base" HEAD)"
|
||||
if grep -Ev '^litellm/(ui|vscode-extension|rust|proxy-extras|packaging|cli)/' <<< "$files" | grep -E '^(litellm/|enterprise/|(pyproject\.toml|uv\.lock)$|\.github/workflows/test-linting\.yml$)' >/dev/null || grep -Eq '^litellm/ui/litellm-dashboard/(src/lib/http/schema\.d\.ts|scripts/gen-api-types\.mjs|package(-lock)?\.json)$' <<< "$files"; then
|
||||
if grep -Ev '^litellm/(ui|vscode-extension|rust|proxy-extras|packaging|cli|scripts|runtime)/' <<< "$files" | grep -E '^(litellm/|enterprise/|(pyproject\.toml|uv\.lock)$|\.github/workflows/test-linting\.yml$)' >/dev/null || grep -Eq '^litellm/ui/litellm-dashboard/(src/lib/http/schema\.d\.ts|scripts/gen-api-types\.mjs|package(-lock)?\.json)$' <<< "$files"; then
|
||||
echo "relevant=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "No litellm, enterprise, dependency or generator changes in this pull request, nothing to verify."
|
||||
|
|
@ -524,7 +524,7 @@ jobs:
|
|||
run: uv run --no-sync python ./tests/code_coverage_tests/check_provider_folders_documented.py
|
||||
|
||||
- name: check_provider_endpoints_support_generated
|
||||
run: uv run --no-sync python scripts/generate_provider_endpoints_support.py --check
|
||||
run: uv run --no-sync python litellm/scripts/generate_provider_endpoints_support.py --check
|
||||
|
||||
- name: test_generate_provider_endpoints_support
|
||||
run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_generate_provider_endpoints_support.py
|
||||
|
|
|
|||
|
|
@ -10,17 +10,17 @@ on:
|
|||
- "uv.lock"
|
||||
- "uv.toml"
|
||||
- ".python-version"
|
||||
- "rust-toolchain.toml"
|
||||
- "litellm/rust/rust-toolchain.toml"
|
||||
- "litellm/rust/**"
|
||||
- "litellm/__init__.py"
|
||||
- "litellm/_version.py"
|
||||
- "litellm/_lazy_imports_registry.py"
|
||||
- "scripts/build_core_distribution.py"
|
||||
- "litellm/scripts/build_core_distribution.py"
|
||||
- "litellm/proxy/proxy_server.py"
|
||||
- "litellm/**/*mcp*"
|
||||
- "litellm/**/*mcp*/**"
|
||||
- "litellm/integrations/arize/**"
|
||||
- "scripts/check_mcp_sdk_install.py"
|
||||
- "litellm/scripts/check_mcp_sdk_install.py"
|
||||
- "tests/base_sdk_tests/**"
|
||||
- ".github/workflows/test-mcp-dependency-resolution.yml"
|
||||
- ".github/actions/detect-changes/**"
|
||||
|
|
@ -91,7 +91,7 @@ jobs:
|
|||
checker=("$GITHUB_WORKSPACE/tests/base_sdk_tests/check_base_sdk_install.py")
|
||||
(cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-$extra/bin/python" -I "$GITHUB_WORKSPACE/tests/base_sdk_tests/check_sdk_http.py")
|
||||
else
|
||||
checker=("$GITHUB_WORKSPACE/scripts/check_mcp_sdk_install.py" --extra "$extra")
|
||||
checker=("$GITHUB_WORKSPACE/litellm/scripts/check_mcp_sdk_install.py" --extra "$extra")
|
||||
fi
|
||||
(cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-$extra/bin/python" "${checker[@]}")
|
||||
done
|
||||
|
|
@ -115,7 +115,7 @@ jobs:
|
|||
if [ "$extra" = core ]; then
|
||||
checker=("$GITHUB_WORKSPACE/tests/base_sdk_tests/check_base_sdk_install.py")
|
||||
else
|
||||
checker=("$GITHUB_WORKSPACE/scripts/check_mcp_sdk_install.py" --extra "$extra")
|
||||
checker=("$GITHUB_WORKSPACE/litellm/scripts/check_mcp_sdk_install.py" --extra "$extra")
|
||||
fi
|
||||
(cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-lowest-$extra/bin/python" "${checker[@]}")
|
||||
done
|
||||
|
|
@ -147,15 +147,16 @@ jobs:
|
|||
- name: Build the core wheel and source distribution
|
||||
run: >-
|
||||
uv run --no-project --with coverage==7.14.0 python -m coverage run --rcfile=/dev/null
|
||||
--branch --include="*/scripts/build_core_distribution.py"
|
||||
scripts/build_core_distribution.py --out-dir dist/core
|
||||
--branch --include="*/litellm/scripts/build_core_distribution.py"
|
||||
litellm/scripts/build_core_distribution.py --out-dir dist/core
|
||||
- name: Verify metadata, resources, and independent source rebuild
|
||||
env:
|
||||
CORE_DISTRIBUTION_DIR: ${{ github.workspace }}/dist/core
|
||||
run: >-
|
||||
uv run --no-project --with pytest==9.0.3 --with pytest-cov==5.0.0 --with coverage==7.14.0 --with 'tomli==2.4.1; python_version < "3.11"'
|
||||
python -m pytest tests/base_sdk_tests/test_core_distribution.py -v
|
||||
--cov-config=/dev/null --cov=scripts.build_core_distribution --cov-branch --cov-append --cov-report=
|
||||
python -m coverage run --rcfile=/dev/null --append --branch
|
||||
--include="*/litellm/scripts/build_core_distribution.py"
|
||||
-m pytest tests/base_sdk_tests/test_core_distribution.py -v
|
||||
- name: Verify isolated core installations
|
||||
run: |
|
||||
wheel=$(realpath dist/core/litellm_core-*.whl)
|
||||
|
|
|
|||
2
.github/workflows/test-merge-smoke.yml
vendored
2
.github/workflows/test-merge-smoke.yml
vendored
|
|
@ -229,7 +229,7 @@ jobs:
|
|||
"prisma==0.11.0" \
|
||||
"openai==2.24.0"
|
||||
export PATH="/tmp/litellm-pip/bin:$PATH"
|
||||
/tmp/litellm-pip/bin/prisma generate --schema=schema.prisma
|
||||
/tmp/litellm-pip/bin/prisma generate --schema=litellm/schema.prisma
|
||||
|
||||
- name: Start the fake OpenAI endpoint
|
||||
run: |
|
||||
|
|
|
|||
8
.github/workflows/test-rust-db.yml
vendored
8
.github/workflows/test-rust-db.yml
vendored
|
|
@ -9,8 +9,8 @@ on:
|
|||
- "litellm/rust/clippy.toml"
|
||||
- "litellm/proxy-extras/litellm_proxy_extras/migrations/**"
|
||||
- "**/schema.prisma"
|
||||
- ".cargo/**"
|
||||
- "rust-toolchain.toml"
|
||||
- "litellm/rust/.cargo/**"
|
||||
- "litellm/rust/rust-toolchain.toml"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- ".github/actions/setup-uv-with-retries/**"
|
||||
|
|
@ -26,8 +26,8 @@ on:
|
|||
- "litellm/rust/clippy.toml"
|
||||
- "litellm/proxy-extras/litellm_proxy_extras/migrations/**"
|
||||
- "**/schema.prisma"
|
||||
- ".cargo/**"
|
||||
- "rust-toolchain.toml"
|
||||
- "litellm/rust/.cargo/**"
|
||||
- "litellm/rust/rust-toolchain.toml"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- ".github/actions/setup-uv-with-retries/**"
|
||||
|
|
|
|||
26
.github/workflows/test-rust.yml
vendored
26
.github/workflows/test-rust.yml
vendored
|
|
@ -6,9 +6,9 @@ on:
|
|||
- "litellm/rust/**"
|
||||
- "model_prices_and_context_window.json"
|
||||
- "litellm/rust_bridge/**"
|
||||
- "scripts/generate_trace_types.py"
|
||||
- "scripts/trace_codegen/**"
|
||||
- "scripts/lens_assets/**"
|
||||
- "litellm/scripts/generate_trace_types.py"
|
||||
- "litellm/scripts/trace_codegen/**"
|
||||
- "litellm/scripts/lens_assets/**"
|
||||
- "litellm/tracing/**"
|
||||
- "tests/test_litellm_rust/**"
|
||||
- "litellm/integrations/custom_logger.py"
|
||||
|
|
@ -24,10 +24,10 @@ on:
|
|||
- "litellm/llms/base_llm/ocr/**"
|
||||
- "litellm/llms/custom_httpx/llm_http_handler.py"
|
||||
- "Makefile"
|
||||
- ".cargo/**"
|
||||
- "litellm/rust/.cargo/**"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- "rust-toolchain.toml"
|
||||
- "litellm/rust/rust-toolchain.toml"
|
||||
- ".github/actions/setup-uv-with-retries/**"
|
||||
- ".github/scripts/smoke_test_native_wheel.py"
|
||||
- ".github/scripts/verify_linux_native_wheel.py"
|
||||
|
|
@ -41,9 +41,9 @@ on:
|
|||
- "litellm/rust/**"
|
||||
- "model_prices_and_context_window.json"
|
||||
- "litellm/rust_bridge/**"
|
||||
- "scripts/generate_trace_types.py"
|
||||
- "scripts/trace_codegen/**"
|
||||
- "scripts/lens_assets/**"
|
||||
- "litellm/scripts/generate_trace_types.py"
|
||||
- "litellm/scripts/trace_codegen/**"
|
||||
- "litellm/scripts/lens_assets/**"
|
||||
- "litellm/tracing/**"
|
||||
- "tests/test_litellm_rust/**"
|
||||
- "litellm/integrations/custom_logger.py"
|
||||
|
|
@ -59,10 +59,10 @@ on:
|
|||
- "litellm/llms/base_llm/ocr/**"
|
||||
- "litellm/llms/custom_httpx/llm_http_handler.py"
|
||||
- "Makefile"
|
||||
- ".cargo/**"
|
||||
- "litellm/rust/.cargo/**"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- "rust-toolchain.toml"
|
||||
- "litellm/rust/rust-toolchain.toml"
|
||||
- ".github/actions/setup-uv-with-retries/**"
|
||||
- ".github/scripts/smoke_test_native_wheel.py"
|
||||
- ".github/scripts/verify_linux_native_wheel.py"
|
||||
|
|
@ -96,7 +96,9 @@ jobs:
|
|||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- run: rustup toolchain install --no-self-update
|
||||
- name: Configure the Rust build environment
|
||||
working-directory: .
|
||||
run: bash .github/scripts/setup_rust_environment.sh
|
||||
|
||||
- run: cargo fmt --all --check
|
||||
|
||||
|
|
@ -160,7 +162,7 @@ jobs:
|
|||
|
||||
- name: Check generated trace contracts
|
||||
working-directory: .
|
||||
run: uv run scripts/generate_trace_types.py --check
|
||||
run: uv run litellm/scripts/generate_trace_types.py --check
|
||||
|
||||
- run: cargo nextest run --workspace --locked
|
||||
|
||||
|
|
|
|||
4
.github/workflows/test-unit.yml
vendored
4
.github/workflows/test-unit.yml
vendored
|
|
@ -1080,7 +1080,7 @@ jobs:
|
|||
fi
|
||||
uv run --no-sync python -c 'import os, sys; print(sys.version); assert f"{sys.version_info.major}.{sys.version_info.minor}" == os.environ["UV_PYTHON"]'
|
||||
|
||||
- name: Check schema.prisma against the merge base
|
||||
- name: Check litellm/schema.prisma against the merge base
|
||||
if: steps.changes.outputs.decision != 'skip' && matrix.shard == 'db-and-spend' && github.event_name == 'pull_request'
|
||||
timeout-minutes: 3
|
||||
env:
|
||||
|
|
@ -1093,7 +1093,7 @@ jobs:
|
|||
merge_base=$(gh api "repos/${GITHUB_REPOSITORY}/compare/${base_ref}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
|
||||
test -n "$merge_base"
|
||||
git fetch --no-tags --depth=1 origin "$merge_base"
|
||||
uv run --no-sync python scripts/check_schema_compatibility.py --base-commit "$merge_base"
|
||||
uv run --no-sync python litellm/scripts/check_schema_compatibility.py --base-commit "$merge_base"
|
||||
|
||||
- name: Cache Prisma binaries
|
||||
if: steps.changes.outputs.decision != 'skip'
|
||||
|
|
|
|||
2
.gitignore
vendored
2
.gitignore
vendored
|
|
@ -106,7 +106,7 @@ litellm_config.yaml
|
|||
litellm/proxy/to_delete_loadtest_work/*
|
||||
update_model_cost_map.py
|
||||
tests/unit/proxy/mcp_server/test_mcp_server_manager.py
|
||||
scripts/test_vertex_ai_search.py
|
||||
litellm/scripts/test_vertex_ai_search.py
|
||||
LAZY_LOADING_IMPROVEMENTS.md
|
||||
STABILIZATION_TODO.md
|
||||
**/test-results
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ A test must only fail when litellm code changes. Never pin facts we don't own (a
|
|||
|
||||
End-to-end tests belong in `tests/e2e/`. A rule about one directory's files, like its harness or which CI runs it, goes in that directory's `AGENTS.md`, never only in a README or here. Read a directory's `AGENTS.md` before editing files in it
|
||||
|
||||
When creating PRs, target the repository's current default branch for both internal and external / OSS contributions. Check it with `python3 scripts/default_branch.py --branch` instead of assuming a branch name or relying on cached `origin/HEAD`
|
||||
When creating PRs, target the repository's current default branch for both internal and external / OSS contributions. Check it with `python3 litellm/scripts/default_branch.py --branch` instead of assuming a branch name or relying on cached `origin/HEAD`
|
||||
|
||||
When writing a PR body, treat the comments and imperative instructions inside .github/PULL_REQUEST_TEMPLATE/general.md as rules to follow, not just layout. Agent harnesses may strip HTML comments from copies of that file injected into context, so read .github/PULL_REQUEST_TEMPLATE/general.md from disk before writing a PR body to make sure you see every comment rule. A section you have nothing to put in (Relevant issues, Affected release, Linear ticket, Caveats, QA runbook, and so on) is removed entirely, heading included, never left as an empty title
|
||||
|
||||
|
|
@ -56,11 +56,11 @@ Don't hesitate to use values in .env to get needed API keys and other secrets, a
|
|||
|
||||
Python max line length is 120, not 88
|
||||
|
||||
The five lint gates (`scripts/ruff_strict_gate.py`, `scripts/type_discipline_gate.py`, `scripts/test_quality_gate.py`, `scripts/openapi_docs_gate.py`, `scripts/type_check_gate.py`) compare each rule's codebase count on your branch against the count at its merge-base with the default branch, and a rule may not grow. There are no budget files to edit or ratchet: when a gate fails, fix the violations the branch introduced or remove at least as many of that rule elsewhere in the tree. The one exception is `reportAny` / `reportExplicitAny`, which share a fixed codebase-wide cap in `ANY_CAPS` in `scripts/type_check_gate.py` because Any spreads past the lines you touch. A branch may add Anys while the total stays under that cap. Only lower `ANY_CAPS`, and only in its own PR to the default branch, never raise it on a feature branch
|
||||
The five lint gates (`litellm/scripts/ruff_strict_gate.py`, `litellm/scripts/type_discipline_gate.py`, `litellm/scripts/test_quality_gate.py`, `litellm/scripts/openapi_docs_gate.py`, `litellm/scripts/type_check_gate.py`) compare each rule's codebase count on your branch against the count at its merge-base with the default branch, and a rule may not grow. There are no budget files to edit or ratchet: when a gate fails, fix the violations the branch introduced or remove at least as many of that rule elsewhere in the tree. The one exception is `reportAny` / `reportExplicitAny`, which share a fixed codebase-wide cap in `ANY_CAPS` in `litellm/scripts/type_check_gate.py` because Any spreads past the lines you touch. A branch may add Anys while the total stays under that cap. Only lower `ANY_CAPS`, and only in its own PR to the default branch, never raise it on a feature branch
|
||||
|
||||
`make check` (f.k.a. `make pre-commit`, which still works identically as an alias) saves its complete output to a log file in .git (overwriting previous logs) and prints that path as its first and last output lines. To inspect a run, read or grep that log instead of re-running the multi-minute checks just to see a different slice
|
||||
|
||||
`make check`, `make lint`, `scripts/pre_commit_lint.sh`, and the standalone lint gates (`scripts/ruff_strict_gate.py`, `scripts/type_discipline_gate.py`, `scripts/type_check_gate.py`) each hold one of 2 machine-wide slots, so when other sessions or worktrees on the same box are already running heavy work, yours prints "all N machine-wide slots are busy" followed by "queueing" and then stays quiet until a slot frees. Give the command a long timeout and let it wait rather than killing it, retrying it, or assuming it hung. Don't change the # of machine-wide slots or make it unlimited by setting `LITELLM_GATE_SLOTS=0`
|
||||
`make check`, `make lint`, `litellm/scripts/pre_commit_lint.sh`, and the standalone lint gates (`litellm/scripts/ruff_strict_gate.py`, `litellm/scripts/type_discipline_gate.py`, `litellm/scripts/type_check_gate.py`) each hold one of 2 machine-wide slots, so when other sessions or worktrees on the same box are already running heavy work, yours prints "all N machine-wide slots are busy" followed by "queueing" and then stays quiet until a slot frees. Give the command a long timeout and let it wait rather than killing it, retrying it, or assuming it hung. Don't change the # of machine-wide slots or make it unlimited by setting `LITELLM_GATE_SLOTS=0`
|
||||
|
||||
If you're trying to create a new function that relies on untyped stuff, instead of adding more Any's and pushing `reportAny` / `reportExplicitAny` closer to their cap in `ANY_CAPS`, just validate it in the caller with Pydantic (a model or `TypeAdapter` that returns the typed thing or raises will do) and then pass the now typed variable in
|
||||
|
||||
|
|
@ -88,8 +88,6 @@ Do not put names of customers or customer company names in code, PR descriptions
|
|||
|
||||
CI supply-chain safety: Never pipe a remote script into a shell (`curl ... | bash`, `wget ... | sh`). Download the artifact to a file, verify its SHA-256 checksum, then install. Pin every external tool to a specific version with a full URL (not `latest` or `stable`). Verify checksums for all downloaded binaries, using the provider's official `.sha256` / `.sha256sum` sidecar when available. These rules apply to every download in CI
|
||||
|
||||
For database operations, connection ownership and proxy dependency wiring, follow [database-access](.agents/skills/database-access/SKILL.md)
|
||||
|
||||
Prisma migrations apply synchronously at proxy boot, before it serves traffic, so a migration must only change schema, never rewrite rows. No `UPDATE`, `DELETE` or `MERGE`, and no `INSERT ... SELECT`: on a spend-log-sized table any of those is minutes of downtime plus a doubled heap that plain autovacuum won't give back. `tests/code_coverage_tests/check_migrations_no_data_rewrites.py` enforces this. When a rewrite is genuinely bounded and has to ship inside the migration, mark the statement `-- data-migration-ok: <what bounds it>`
|
||||
|
||||
Follow these coding conventions for new/updated code (a three-line fix in a legacy file shouldn't trigger huge drive-by refactors):
|
||||
|
|
|
|||
36
Makefile
36
Makefile
|
|
@ -60,14 +60,18 @@ help:
|
|||
@echo "slots (default 2; 0 disables) so parallel sessions don't thrash one machine."
|
||||
|
||||
UV := uv
|
||||
export RUSTUP_TOOLCHAIN ?= $(shell sed -n 's/^[[:space:]]*channel[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p' litellm/rust/rust-toolchain.toml)
|
||||
export CARGO_NET_RETRY ?= 5
|
||||
export CARGO_HTTP_MULTIPLEXING ?= false
|
||||
export SQLX_OFFLINE ?= true
|
||||
UV_RUN := $(UV) run --no-sync
|
||||
BASE_REF ?=
|
||||
export BASE_REF
|
||||
RESOLVE_BASE = python3 scripts/default_branch.py --base "$(BASE_REF)"
|
||||
RESOLVE_BASE = python3 litellm/scripts/default_branch.py --base "$(BASE_REF)"
|
||||
|
||||
# Machine-wide slot queue for the heavy targets below; python3 + stdlib only, so
|
||||
# it runs before any venv exists. See scripts/gate_slot_lock.py.
|
||||
GATE_SLOT_LOCK := python3 scripts/gate_slot_lock.py
|
||||
# it runs before any venv exists. See litellm/scripts/gate_slot_lock.py.
|
||||
GATE_SLOT_LOCK := python3 litellm/scripts/gate_slot_lock.py
|
||||
|
||||
LINT_DEP_INSTALL ?= install-dev
|
||||
LINT_E2E_DEP_INSTALL ?= lint-install
|
||||
|
|
@ -90,8 +94,8 @@ install-dev:
|
|||
# machine-wide slots the CPU-bound gates below share.
|
||||
bootstrap:
|
||||
$(UV) sync --inexact --frozen --extra proxy --group proxy-dev --group e2e-dev
|
||||
$(UV_RUN) python scripts/prisma_generate_if_needed.py
|
||||
cd litellm/ui/litellm-dashboard && ../../../scripts/with_dashboard_node.sh npm install --no-audit --no-fund
|
||||
$(UV_RUN) python litellm/scripts/prisma_generate_if_needed.py
|
||||
cd litellm/ui/litellm-dashboard && ../../scripts/with_dashboard_node.sh npm install --no-audit --no-fund
|
||||
@main_root=$$(git worktree list --porcelain | head -1 | sed 's/^worktree //'); \
|
||||
if [ "$$main_root" != "$$(git rev-parse --show-toplevel)" ] && [ -f "$$main_root/.env" ] && [ ! -f .env ]; then \
|
||||
cp "$$main_root/.env" .env && echo "bootstrap: copied .env from $$main_root"; \
|
||||
|
|
@ -123,7 +127,7 @@ install-helm-unittest:
|
|||
# Install git hooks that enforce Conventional Commits.
|
||||
# Opt-in: not chained into install-dev.
|
||||
install-hooks:
|
||||
./scripts/install_git_hooks.sh
|
||||
./litellm/scripts/install_git_hooks.sh
|
||||
|
||||
# Formatting
|
||||
# Wrap width is pyproject.toml's single source of truth (line-length = 120), shared by the
|
||||
|
|
@ -139,12 +143,12 @@ lint-fetch-base:
|
|||
|
||||
# Mirror test-linting.yml's python job environment: the proxy-dev group plus a generated
|
||||
# Prisma client, so `basedpyright tests/e2e` resolves the same modules CI does. The
|
||||
# basedpyright gate itself no longer measures here (scripts/type_check_gate.py provisions its
|
||||
# basedpyright gate itself no longer measures here (litellm/scripts/type_check_gate.py provisions its
|
||||
# own .venv-typecheck). --inexact tops up the venv instead of pruning the proxy extras
|
||||
# gen:api and the running proxy need.
|
||||
lint-install:
|
||||
$(UV) sync --inexact --frozen --group proxy-dev --group e2e-dev
|
||||
$(UV_RUN) python scripts/prisma_generate_if_needed.py
|
||||
$(UV_RUN) python litellm/scripts/prisma_generate_if_needed.py
|
||||
|
||||
# Diff-scoped format check, mirroring test-linting.yml's "Check ruff format" step:
|
||||
# only the litellm Python files changed vs the base are checked, so a pre-existing
|
||||
|
|
@ -163,7 +167,7 @@ lint-format-check-changed: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
|||
|
||||
# Linting targets
|
||||
lint-ruff: $(LINT_DEP_INSTALL)
|
||||
$(UV_RUN) python scripts/check_mcp_operation_boundary.py
|
||||
$(UV_RUN) python litellm/scripts/check_mcp_operation_boundary.py
|
||||
cd litellm && $(UV_RUN) ruff check . && cd ..
|
||||
$(UV_RUN) ruff check --config .github/linters/ruff-tests.toml tests
|
||||
|
||||
|
|
@ -204,7 +208,7 @@ lint-ruff-FULL-dev: install-dev
|
|||
else echo "No changed .py files to check."; fi
|
||||
|
||||
lint-basedpyright: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
$(UV_RUN) python scripts/type_check_gate.py --base "$(BASE_REF)"
|
||||
$(UV_RUN) python litellm/scripts/type_check_gate.py --base "$(BASE_REF)"
|
||||
|
||||
lint-e2e-basedpyright: $(LINT_E2E_DEP_INSTALL)
|
||||
$(UV_RUN) basedpyright tests/e2e tests/e2e_harness
|
||||
|
|
@ -212,29 +216,29 @@ lint-e2e-basedpyright: $(LINT_E2E_DEP_INSTALL)
|
|||
# Type-discipline gate (mutable collections / casts / type guards / kwargs /
|
||||
# unexplained suppressions), the test-linting.yml step `make lint` used to omit.
|
||||
lint-type-discipline: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
$(UV_RUN) python scripts/type_discipline_gate.py --base "$(BASE_REF)"
|
||||
$(UV_RUN) python litellm/scripts/type_discipline_gate.py --base "$(BASE_REF)"
|
||||
|
||||
# Test-quality gate (zero-assert / mock-echo tests, sys.path.insert, raw env writes,
|
||||
# litellm module-global mutation, credential-gated skips, conftest snapshot
|
||||
# inventory), counted across tests/ the same delta-vs-base way.
|
||||
lint-test-quality: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
$(UV_RUN) python scripts/test_quality_gate.py --base "$(BASE_REF)"
|
||||
$(UV_RUN) python litellm/scripts/test_quality_gate.py --base "$(BASE_REF)"
|
||||
|
||||
# OpenAPI docs gate (POST routes without a request body schema, a description or a
|
||||
# 200 schema in app.openapi()), counted the same delta-vs-base way so a new endpoint
|
||||
# has to show up documented in Swagger UI.
|
||||
lint-openapi-docs: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
$(UV_RUN) python scripts/openapi_docs_gate.py --base "$(BASE_REF)"
|
||||
$(UV_RUN) python litellm/scripts/openapi_docs_gate.py --base "$(BASE_REF)"
|
||||
|
||||
lint-format: format-check
|
||||
|
||||
lint-ruff-strict: install-dev
|
||||
$(UV_RUN) python scripts/ruff_strict_gate.py --base "$(BASE_REF)"
|
||||
$(UV_RUN) python litellm/scripts/ruff_strict_gate.py --base "$(BASE_REF)"
|
||||
|
||||
# Strict gate, invoked the same way CI does in test-linting.yml so a local pass
|
||||
# means the CI check will pass too.
|
||||
lint-gate: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
$(UV_RUN) python scripts/ruff_strict_gate.py --base "$(BASE_REF)"
|
||||
$(UV_RUN) python litellm/scripts/ruff_strict_gate.py --base "$(BASE_REF)"
|
||||
|
||||
check-circular-imports: $(LINT_DEP_INSTALL)
|
||||
cd litellm && $(UV_RUN) python ../tests/documentation_tests/test_circular_imports.py && cd ..
|
||||
|
|
@ -272,7 +276,7 @@ check:
|
|||
@$(GATE_SLOT_LOCK) $(MAKE) check-inner
|
||||
|
||||
check-inner: bootstrap
|
||||
./scripts/pre_commit_lint.sh
|
||||
./litellm/scripts/pre_commit_lint.sh
|
||||
|
||||
pre-commit:
|
||||
@echo "make pre-commit is a legacy alias; use make check" >&2
|
||||
|
|
|
|||
|
|
@ -99,7 +99,7 @@ proxy, CLI, and optional extras
|
|||
Build and install core from this checkout while its release integration is pending:
|
||||
|
||||
```shell
|
||||
python scripts/build_core_distribution.py --out-dir dist/core
|
||||
python litellm/scripts/build_core_distribution.py --out-dir dist/core
|
||||
python -m pip install dist/core/litellm_core-*.whl
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -68,6 +68,9 @@ RUN apk add --no-cache \
|
|||
libsndfile
|
||||
|
||||
ENV UV_PROJECT_ENVIRONMENT=/app/.venv \
|
||||
CARGO_NET_RETRY=5 \
|
||||
CARGO_HTTP_MULTIPLEXING=false \
|
||||
SQLX_OFFLINE=true \
|
||||
UV_LINK_MODE=copy \
|
||||
UV_PYTHON_DOWNLOADS=0 \
|
||||
PATH="/app/.venv/bin:${PATH}"
|
||||
|
|
@ -116,11 +119,11 @@ RUN uv sync --frozen --no-install-package litellm-proxy --no-default-groups --no
|
|||
|
||||
RUN uv pip install --python /app/.venv/bin/python --no-deps /wheels/litellm_proxy-*.whl
|
||||
|
||||
RUN cp -r litellm/enterprise "$(.venv/bin/python -c 'import sysconfig; print(sysconfig.get_path("purelib"))')/litellm/enterprise"
|
||||
RUN cp -r litellm/enterprise litellm/runtime "$(.venv/bin/python -c 'import sysconfig; print(sysconfig.get_path("purelib"))')/litellm/"
|
||||
|
||||
RUN HOME=/opt/prisma XDG_CACHE_HOME=/opt/prisma/.cache PRISMA_BINARY_CACHE_DIR=/opt/prisma/binaries \
|
||||
npm_config_cache=/root/.npm \
|
||||
prisma generate --schema=./schema.prisma
|
||||
prisma generate --schema=litellm/schema.prisma
|
||||
|
||||
RUN sed -i 's/\r$//' litellm/deploy/docker/docker-entrypoint.sh && chmod +x litellm/deploy/docker/docker-entrypoint.sh
|
||||
|
||||
|
|
@ -181,14 +184,13 @@ ENV PATH="/app/.venv/bin:${PATH}" \
|
|||
COPY --from=builder /app/.venv /app/.venv
|
||||
COPY --from=liteadmin-builder /opt/liteadmin /opt/liteadmin
|
||||
COPY --from=builder /app/litellm/deploy/docker/docker-entrypoint.sh /app/docker-entrypoint.sh
|
||||
COPY --from=builder /app/schema.prisma /app/schema.prisma
|
||||
COPY --from=builder /app/litellm/schema.prisma /app/litellm/schema.prisma
|
||||
COPY --from=builder /app/litellm/schema.prisma /app/schema.prisma
|
||||
COPY --from=builder /app/litellm/runtime/gateway /app/gateway
|
||||
COPY --from=builder /app/litellm/proxy/prisma_migration.py /app/litellm/proxy/prisma_migration.py
|
||||
# enterprise/ is imported by source path at runtime (proxy_cli puts the
|
||||
# working directory on sys.path; litellm/proxy/hooks resolves
|
||||
# litellm.enterprise.enterprise_hooks from it)
|
||||
COPY --from=builder /app/gateway /app/gateway
|
||||
COPY --from=builder /app/backend /app/backend
|
||||
COPY --from=builder /app/migrations /app/migrations
|
||||
COPY --from=builder /opt/prisma /opt/prisma
|
||||
COPY --from=builder /var/lib/litellm /var/lib/litellm
|
||||
COPY litellm/ui/nginx.conf /etc/nginx/nginx.conf
|
||||
|
|
@ -207,7 +209,7 @@ RUN find /app/.venv -depth -type d -path "*/tornado/test" -exec rm -rf {} + && \
|
|||
ls /opt/prisma/binaries/node_modules/@prisma/engines/query-engine-* >/dev/null && \
|
||||
python -c "from prisma.client import BINARY_PATHS; paths = list(BINARY_PATHS.query_engine.values()); assert paths and all(p.startswith('/opt/prisma/') for p in paths), paths" && \
|
||||
python -c "from litellm.rust_bridge import _native" && \
|
||||
python -c "import gateway.launch, backend.main"
|
||||
python -c "import gateway.launch, litellm.runtime.backend.main"
|
||||
|
||||
USER 65532:65532
|
||||
|
||||
|
|
|
|||
|
|
@ -19,11 +19,11 @@ Saved quickstart scripts need a one-time URL override after the source layout mi
|
|||
|
||||
```bash
|
||||
LITELLM_COMPOSE_URL=https://raw.githubusercontent.com/BerriAI/litellm/main/litellm/deploy/docker/docker-compose.quickstart.yml \
|
||||
LITELLM_SCRIPT_URL=https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.sh \
|
||||
LITELLM_SCRIPT_URL=https://raw.githubusercontent.com/BerriAI/litellm/main/litellm/scripts/quickstart.sh \
|
||||
litellm-gateway update
|
||||
```
|
||||
|
||||
In PowerShell, set the same environment variables, with `LITELLM_SCRIPT_URL` ending in `scripts/quickstart.ps1`, before running `litellm-gateway update`. Remove the overrides after the saved script refreshes.
|
||||
In PowerShell, set the same environment variables, with `LITELLM_SCRIPT_URL` ending in `litellm/scripts/quickstart.ps1`, before running `litellm-gateway update`. Remove the overrides after the saved script refreshes.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -42,9 +42,9 @@ Every LiteLLM container runs the same image. The first word of the container com
|
|||
|--------------|-----------------------------------------------------|------|
|
||||
| `proxy` | `litellm ...` (everything in one process, default) | 4000 |
|
||||
| `gateway` | `python -m gateway.launch ...` (inference routes) | 4000 |
|
||||
| `backend` | `uvicorn backend.main:app ...` (management routes) | 4001 |
|
||||
| `backend` | `uvicorn litellm.runtime.backend.main:app ...` (management routes) | 4001 |
|
||||
| `ui` | nginx serving the static admin UI | 3000 |
|
||||
| `migrations` | `python migrations/run.py`, `prisma migrate deploy` then exit | |
|
||||
| `migrations` | `python -m litellm.runtime.migrations.run`, `prisma migrate deploy` then exit | |
|
||||
| `metrics` | `python -m litellm.proxy.prometheus_metrics_server ...` | `--port` |
|
||||
| `collector` | `python -m litellm.proxy.collector ...` | |
|
||||
|
||||
|
|
|
|||
|
|
@ -8,9 +8,9 @@
|
|||
#
|
||||
# proxy litellm ARGS everything in one process (default)
|
||||
# gateway python -m gateway.launch ARGS inference routes, 0.0.0.0:4000
|
||||
# backend uvicorn backend.main:app ARGS management routes, 0.0.0.0:4001
|
||||
# backend uvicorn litellm.runtime.backend.main:app ARGS management routes, 0.0.0.0:4001
|
||||
# ui nginx serving the admin UI port 3000
|
||||
# migrations python migrations/run.py prisma migrate deploy, then exit
|
||||
# migrations python litellm/runtime/migrations/run.py prisma migrate deploy, then exit
|
||||
# metrics python -m litellm.proxy.prometheus_metrics_server ARGS
|
||||
# collector python -m litellm.proxy.collector ARGS
|
||||
# --admin-agent the bundled LiteAdmin Slack agent (litellm-admin-agent --web ARGS)
|
||||
|
|
@ -97,7 +97,7 @@ os.execvp("litellm", ["litellm", *sys.argv[1:]])' "$@"
|
|||
set -- python -m gateway.launch --workers "${NUM_WORKERS:-1}" --host 0.0.0.0 --port 4000 "$@"
|
||||
;;
|
||||
backend)
|
||||
set -- uvicorn backend.main:app --host 0.0.0.0 --port 4001 "$@"
|
||||
set -- uvicorn litellm.runtime.backend.main:app --host 0.0.0.0 --port 4001 "$@"
|
||||
;;
|
||||
ui)
|
||||
ui_hook="$(dirname "$0")/ui/docker-entrypoint.d/40-litellm-runtime-config.sh"
|
||||
|
|
@ -108,7 +108,7 @@ os.execvp("litellm", ["litellm", *sys.argv[1:]])' "$@"
|
|||
exec nginx -g 'daemon off;' "$@"
|
||||
;;
|
||||
migrations)
|
||||
set -- python /app/migrations/run.py "$@"
|
||||
set -- python -m litellm.runtime.migrations.run "$@"
|
||||
;;
|
||||
metrics)
|
||||
set -- python -m litellm.proxy.prometheus_metrics_server "$@"
|
||||
|
|
|
|||
|
|
@ -296,7 +296,7 @@ by the controller rather than declared, so nothing there can collide.
|
|||
LITELLM_MASTER_KEY, and PROXY_MASTER_KEY for configs copied from litellm-helm,
|
||||
for the app containers only. The migrations Job runs as a
|
||||
pre-install hook, before the chart's generated master key Secret exists, and
|
||||
migrations/run.py never reads the key, so the Job must not reference it.
|
||||
litellm/runtime/migrations/run.py never reads the key, so the Job must not reference it.
|
||||
*/}}
|
||||
{{- define "litellm.masterKeyEnv" -}}
|
||||
{{- range list "LITELLM_MASTER_KEY" "PROXY_MASTER_KEY" }}
|
||||
|
|
@ -507,7 +507,7 @@ The migrations Job (helm.sh/hook: pre-upgrade) is the single owner of
|
|||
Prisma schema-update on startup and contends with the Job — and with each
|
||||
other — for Prisma's Postgres advisory lock on the writer, which makes the
|
||||
Job's `migrate deploy` intermittently block until its per-attempt timeout
|
||||
and retry-exhaust. The Job's entrypoint (migrations/run.py) does not import
|
||||
and retry-exhaust. The Job's entrypoint (litellm/runtime/migrations/run.py) does not import
|
||||
proxy_server and never reads DISABLE_SCHEMA_UPDATE, so emitting it here is a
|
||||
harmless no-op for the Job and authoritative for the app pods.
|
||||
*/}}
|
||||
|
|
|
|||
|
|
@ -90,7 +90,7 @@
|
|||
{{- $uiPaths = append $uiPaths (dict "path" "/*.txt" "pathType" "ImplementationSpecific") }}
|
||||
{{- end }}
|
||||
{{/*
|
||||
Gateway data-plane prefixes — must mirror gateway/routes/allowlist.py.
|
||||
Gateway data-plane prefixes — must mirror litellm/runtime/gateway/routes/allowlist.py.
|
||||
Versioned paths are listed explicitly to avoid routing management routes
|
||||
(e.g. /v1/access_group, /v2/key/info, /v1/tool/*, /v1/agents, /v1/workflows,
|
||||
/v2/user/info, /v2/team/list, /v2/model/info, /v2/login, /v2/guardrails/*,
|
||||
|
|
|
|||
|
|
@ -46,7 +46,7 @@ starts the process named by its first argument, so each workload only
|
|||
differs in the argument the module passes
|
||||
|
||||
The load balancer routes gateway path prefixes (mirrored verbatim from
|
||||
`gateway/routes/allowlist.py`) to the gateway, UI asset paths (`/`,
|
||||
`litellm/runtime/gateway/routes/allowlist.py`) to the gateway, UI asset paths (`/`,
|
||||
`/litellm-asset-prefix/*`, `/_next/*`, `/favicon.ico`) to the UI, and
|
||||
everything else to the backend.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Gateway path prefixes — mirrored verbatim from gateway/routes/allowlist.py
|
||||
# Gateway path prefixes — mirrored verbatim from litellm/runtime/gateway/routes/allowlist.py
|
||||
# and the helm ingress in litellm/deploy/helm/litellm/templates/ingress.yaml. Anything not in
|
||||
# this list and not a UI asset path falls through to the backend (management
|
||||
# API) catch-all rule on the ALB.
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Gateway path prefixes — mirrored verbatim from gateway/routes/allowlist.py
|
||||
# Gateway path prefixes — mirrored verbatim from litellm/runtime/gateway/routes/allowlist.py
|
||||
# and litellm/deploy/helm/litellm/templates/ingress.yaml. URL maps use the "path matcher"
|
||||
# rule with `paths` lists; up to 10 path globs per rule, up to 50 rules
|
||||
# per matcher. Easily fits the gateway list in one rule per chunk-of-10.
|
||||
|
|
|
|||
|
|
@ -54,8 +54,8 @@ features = ["extension-module"]
|
|||
profile = "release"
|
||||
editable-profile = "dev"
|
||||
include = [
|
||||
{ path = "rust-toolchain.toml", format = "sdist" },
|
||||
{ path = ".cargo/config.toml", format = "sdist" },
|
||||
{ path = "litellm/rust/rust-toolchain.toml", format = "sdist" },
|
||||
{ path = "litellm/rust/.cargo/config.toml", format = "sdist" },
|
||||
{ path = "model_prices_and_context_window.json", format = "sdist" },
|
||||
"litellm/router_strategy/complexity_router/artifacts/*.json",
|
||||
"litellm/router_strategy/complexity_router/fuse_presets.json",
|
||||
|
|
@ -66,6 +66,11 @@ include = [
|
|||
"litellm/proxy/common_utils/codex_bedrock_runtime_models_0.166.0.json",
|
||||
]
|
||||
exclude = [
|
||||
"litellm/scripts",
|
||||
"litellm/scripts/**",
|
||||
"litellm/runtime",
|
||||
"litellm/runtime/**",
|
||||
"litellm/schema.prisma",
|
||||
"litellm/enterprise",
|
||||
"litellm/enterprise/**",
|
||||
"litellm/proxy-extras",
|
||||
|
|
|
|||
|
|
@ -16,8 +16,8 @@ targets = [{ name = "litellm-gateway", kind = "bin" }]
|
|||
profile = "release"
|
||||
editable-profile = "dev"
|
||||
include = [
|
||||
{ path = "rust-toolchain.toml", format = "sdist" },
|
||||
{ path = ".cargo/config.toml", format = "sdist" },
|
||||
{ path = "litellm/rust/rust-toolchain.toml", format = "sdist" },
|
||||
{ path = "litellm/rust/.cargo/config.toml", format = "sdist" },
|
||||
{ path = "model_prices_and_context_window.json", format = "sdist" },
|
||||
]
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ background, once it serves).
|
|||
|
||||
A migration cannot build them: a plain `CREATE INDEX` blocks spend-log inserts for the
|
||||
whole build, and `CREATE INDEX CONCURRENTLY` is refused on a partitioned parent
|
||||
(scripts/db/partition_spend_logs.sql). `REQUEST_LOG_INDEXES` is the one list to extend;
|
||||
(litellm/scripts/db/partition_spend_logs.sql). `REQUEST_LOG_INDEXES` is the one list to extend;
|
||||
names match what Prisma derives from the `@@index` declarations in schema.prisma, so an
|
||||
index a database already has is recognized and never rebuilt.
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -147,7 +147,7 @@ _SPEND_LOGS_PK_CLAUSE_RE = re.compile(
|
|||
)
|
||||
|
||||
PARTITIONED_SPEND_LOGS_PUSH_ERROR = (
|
||||
"LiteLLM_SpendLogs is a partitioned table (see scripts/db/partition_spend_logs.sql), "
|
||||
"LiteLLM_SpendLogs is a partitioned table (see litellm/scripts/db/partition_spend_logs.sql), "
|
||||
'so its primary key must include the partition key ("startTime"). `prisma db push` '
|
||||
"reconciles the database against schema.prisma, which declares the unpartitioned "
|
||||
'primary key ("request_id"), and Postgres rejects that rewrite with: unique '
|
||||
|
|
@ -179,7 +179,7 @@ def _without_spend_logs_pk_clauses(statement: str) -> str | None:
|
|||
|
||||
def filter_partitioned_spend_logs_diff(diff_sql: str) -> str:
|
||||
"""Drop statements from a `prisma migrate diff` script that fight the
|
||||
SpendLogs partitioning runbook (scripts/db/partition_spend_logs.sql): the
|
||||
SpendLogs partitioning runbook (litellm/scripts/db/partition_spend_logs.sql): the
|
||||
primary-key rewrite on "LiteLLM_SpendLogs", which Postgres rejects on a
|
||||
partitioned table, and drops of runbook artifacts such as
|
||||
"LiteLLM_SpendLogs_legacy"."""
|
||||
|
|
@ -679,7 +679,7 @@ class ProxyExtrasDBManager:
|
|||
"""True when the connected database's LiteLLM_SpendLogs is a
|
||||
partitioned table in Prisma's target schema (the `schema` URL param,
|
||||
falling back to Prisma's default target, public), i.e. the operator
|
||||
ran scripts/db/partition_spend_logs.sql. Returns False when psycopg is
|
||||
ran litellm/scripts/db/partition_spend_logs.sql. Returns False when psycopg is
|
||||
unavailable or the database cannot be reached, preserving the
|
||||
pre-existing behavior in those cases."""
|
||||
database_url = os.getenv("DATABASE_URL")
|
||||
|
|
@ -692,7 +692,7 @@ class ProxyExtrasDBManager:
|
|||
logger.warning(
|
||||
"psycopg is not installed; skipping the LiteLLM_SpendLogs "
|
||||
"partition check. If this table is partitioned (see "
|
||||
"scripts/db/partition_spend_logs.sql), schema reconciliation "
|
||||
"litellm/scripts/db/partition_spend_logs.sql), schema reconciliation "
|
||||
"will try to rewrite its primary key and fail. Install the "
|
||||
"litellm[extra_proxy] extra, which now includes psycopg."
|
||||
)
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ Before doing anything else, make sure all `schema.prisma` files in the repo are
|
|||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `schema.prisma` (repo root) | Source of truth |
|
||||
| `litellm/schema.prisma` (canonical) | Source of truth |
|
||||
| `litellm/proxy/schema.prisma` | Used by the proxy server |
|
||||
| `litellm/proxy-extras/litellm_proxy_extras/schema.prisma` | Used for migration generation |
|
||||
|
||||
|
|
@ -18,16 +18,16 @@ Before doing anything else, make sure all `schema.prisma` files in the repo are
|
|||
|
||||
```bash
|
||||
# 1. Diff all schema files against the root source of truth
|
||||
diff schema.prisma litellm/proxy/schema.prisma
|
||||
diff schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma
|
||||
diff litellm/schema.prisma litellm/proxy/schema.prisma
|
||||
diff litellm/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma
|
||||
|
||||
# 2. If there are differences, copy the root schema to all locations
|
||||
cp schema.prisma litellm/proxy/schema.prisma
|
||||
cp schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma
|
||||
cp litellm/schema.prisma litellm/proxy/schema.prisma
|
||||
cp litellm/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma
|
||||
|
||||
# 3. Verify all files are now identical
|
||||
diff schema.prisma litellm/proxy/schema.prisma && echo "proxy schema in sync" || echo "MISMATCH"
|
||||
diff schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma && echo "extras schema in sync" || echo "MISMATCH"
|
||||
diff litellm/schema.prisma litellm/proxy/schema.prisma && echo "proxy schema in sync" || echo "MISMATCH"
|
||||
diff litellm/schema.prisma litellm/proxy-extras/litellm_proxy_extras/schema.prisma && echo "extras schema in sync" || echo "MISMATCH"
|
||||
```
|
||||
|
||||
> **Do NOT proceed to migration generation until all schema files are identical.**
|
||||
|
|
@ -43,7 +43,7 @@ brew install postgresql@14 # macOS
|
|||
export PATH="/opt/homebrew/opt/postgresql@14/bin:$PATH"
|
||||
|
||||
# Run migration
|
||||
uv run --with testing.postgresql python scripts/ci_cd/run_migration.py "your_migration_name"
|
||||
uv run --with testing.postgresql python litellm/scripts/ci_cd/run_migration.py "your_migration_name"
|
||||
```
|
||||
|
||||
## What It Does
|
||||
|
|
@ -69,7 +69,7 @@ When the guard fires:
|
|||
1. Update your branch:
|
||||
|
||||
```bash
|
||||
base_branch=$(python3 scripts/default_branch.py --branch) &&
|
||||
base_branch=$(python3 litellm/scripts/default_branch.py --branch) &&
|
||||
git fetch origin "+refs/heads/$base_branch:refs/remotes/origin/$base_branch" &&
|
||||
git rebase "origin/$base_branch"
|
||||
```
|
||||
|
|
@ -89,7 +89,7 @@ When the guard fires:
|
|||
4. Only if the drops are genuinely intentional, re-run with the flag:
|
||||
|
||||
```bash
|
||||
uv run --with testing.postgresql python scripts/ci_cd/run_migration.py "your_migration_name" --allow-destructive
|
||||
uv run --with testing.postgresql python litellm/scripts/ci_cd/run_migration.py "your_migration_name" --allow-destructive
|
||||
```
|
||||
|
||||
> **AI AGENTS / ASSISTANTS:** Do **not** automatically re-run the command with `--allow-destructive`. If the guard fires while you are driving the runbook for a human, stop, show them the error, and wait for their explicit confirmation before passing the flag. Auto-passing `--allow-destructive` is the exact failure mode this guard exists to prevent.
|
||||
|
|
@ -98,7 +98,7 @@ When the guard fires:
|
|||
|
||||
**Missing testing module:**
|
||||
```bash
|
||||
uv run --with testing.postgresql python scripts/ci_cd/run_migration.py "your_migration_name"
|
||||
uv run --with testing.postgresql python litellm/scripts/ci_cd/run_migration.py "your_migration_name"
|
||||
```
|
||||
|
||||
**initdb not found:**
|
||||
|
|
@ -115,7 +115,7 @@ rm -rf litellm/proxy-extras/litellm_proxy_extras/migrations/[empty_dir]
|
|||
## Rules
|
||||
|
||||
- Sync all `schema.prisma` files first (Step 0)
|
||||
- Update `schema.prisma` at the repo root first, then sync copies
|
||||
- Update `litellm/schema.prisma` first, then sync copies
|
||||
- Review generated SQL before committing
|
||||
- Use descriptive migration names
|
||||
- Never edit existing migration files
|
||||
|
|
|
|||
|
|
@ -3034,7 +3034,7 @@ class ConfigGeneralSettings(LiteLLMPydanticObjectBase):
|
|||
)
|
||||
use_spend_logs_partitioning: bool | None = Field(
|
||||
None,
|
||||
description="If True and LiteLLM_SpendLogs has been converted to a range-partitioned table (scripts/db/partition_spend_logs.sql), retention cleanup drops expired partitions instead of deleting rows, and pre-creates upcoming partitions. Default is False.",
|
||||
description="If True and LiteLLM_SpendLogs has been converted to a range-partitioned table (litellm/scripts/db/partition_spend_logs.sql), retention cleanup drops expired partitions instead of deleting rows, and pre-creates upcoming partitions. Default is False.",
|
||||
)
|
||||
maximum_spend_logs_cleanup_batch_size: int | None = Field(
|
||||
None,
|
||||
|
|
|
|||
|
|
@ -589,7 +589,7 @@ The Codex undo receipt is kept in a private `.litellm` directory beside the reso
|
|||
`lite configure claude` wires Claude Code up persistently with a long-lived virtual key, a pinned model and an undo, and `lite unconfigure claude` puts things back:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install.sh | sh
|
||||
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/litellm/scripts/install.sh | sh
|
||||
lite --base-url https://your-proxy.example.com configure claude --api-key sk-... --model claude-auto
|
||||
claude
|
||||
```
|
||||
|
|
@ -625,17 +625,17 @@ After upgrading the CLI, run `lite configure claude` to refresh `~/.litellm/stat
|
|||
`lite autoroute start` builds and runs a throwaway litellm proxy locally, so unlike the rest of this CLI it needs the proxy server runtime, not just the thin `litellm[cli]` client. Install `litellm[proxy]` (which ships the `lite` command too) with a single curl command -- no existing Python tooling required, `uv` is bootstrapped automatically if missing:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install.sh | sh
|
||||
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/litellm/scripts/install.sh | sh
|
||||
```
|
||||
|
||||
To QA an unreleased branch or commit instead of the latest PyPI release, set `LITELLM_CLI_REF`:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/<branch-or-commit>/scripts/install.sh | \
|
||||
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/<branch-or-commit>/litellm/scripts/install.sh | \
|
||||
LITELLM_CLI_REF=<branch-or-commit> sh
|
||||
```
|
||||
|
||||
The thin `scripts/install-cli.sh` installs only `litellm[cli]`, which is enough for `lite login`, `lite claude`, and `lite up`, but not for `lite autoroute start`; running it against a `litellm[cli]` install fails fast with a message telling you to install the proxy runtime.
|
||||
The thin `litellm/scripts/install-cli.sh` installs only `litellm[cli]`, which is enough for `lite login`, `lite claude`, and `lite up`, but not for `lite autoroute start`; running it against a `litellm[cli]` install fails fast with a message telling you to install the proxy runtime.
|
||||
|
||||
Point the CLI at your real proxy and key before running any `lite model-groups` or `lite autoroute` command -- like every other command in this CLI, they read `LITELLM_PROXY_URL`/`LITELLM_PROXY_API_KEY` (or `--base-url`/`--api-key`), no `lite login` required:
|
||||
|
||||
|
|
|
|||
|
|
@ -111,7 +111,7 @@ def start(port: int) -> None:
|
|||
"lite autoroute start launches a local litellm proxy, which needs the proxy runtime that the "
|
||||
f"thin `litellm[cli]` install does not include (missing: {', '.join(missing)}). Install the "
|
||||
"proxy runtime with `uv tool install --force 'litellm[proxy]'`, or to QA a branch, "
|
||||
"`curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/<branch>/scripts/install.sh | "
|
||||
"`curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/<branch>/litellm/scripts/install.sh | "
|
||||
"LITELLM_CLI_REF=<branch> sh`."
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -51,7 +51,7 @@ async def create_view_tolerating_race(db: SupportsExecuteRaw, view_name: str, dd
|
|||
async def create_missing_views(db: SupportsRawQueries) -> None:
|
||||
"""
|
||||
--------------------------------------------------
|
||||
NOTE: Copy of `scripts/db/create_views.py`.
|
||||
NOTE: Copy of `litellm/scripts/db/create_views.py`.
|
||||
--------------------------------------------------
|
||||
Checks if the LiteLLM_VerificationTokenView and MonthlyGlobalSpend exists in the user's db.
|
||||
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ to the OS immediately, with no tombstones and no vacuum.
|
|||
|
||||
This manager only acts when use_spend_logs_partitioning is enabled in
|
||||
general_settings AND the table is already partitioned (set up via the
|
||||
scripts/db/partition_spend_logs.sql runbook). Without both, the cleanup job
|
||||
litellm/scripts/db/partition_spend_logs.sql runbook). Without both, the cleanup job
|
||||
keeps the batched-DELETE path, so existing deployments are untouched.
|
||||
"""
|
||||
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ route table to just the management/admin surface used by the dashboard. Purely
|
|||
additive — no existing module is modified.
|
||||
|
||||
Run with:
|
||||
uvicorn backend.main:app --host 0.0.0.0 --port 4001
|
||||
uvicorn litellm.runtime.backend.main:app --host 0.0.0.0 --port 4001
|
||||
"""
|
||||
|
||||
from collections.abc import AsyncGenerator, Mapping
|
||||
|
|
@ -16,7 +16,7 @@ from starlette.applications import Starlette
|
|||
from starlette.routing import Mount
|
||||
from starlette.types import Lifespan
|
||||
|
||||
# See gateway/main.py for why we assemble DATABASE_URL(s) here before
|
||||
# See litellm/runtime/gateway/main.py for why we assemble DATABASE_URL(s) here before
|
||||
# importing proxy_server.
|
||||
from litellm.proxy.db.db_url_settings import DatabaseURLSettings
|
||||
|
||||
|
|
@ -24,7 +24,7 @@ DatabaseURLSettings.from_env().apply_to_env()
|
|||
|
||||
from litellm.proxy.proxy_server import app
|
||||
|
||||
from backend.routes.allowlist import (
|
||||
from litellm.runtime.backend.routes.allowlist import (
|
||||
BACKEND_EXACT_PATHS,
|
||||
BACKEND_MOUNT_PATHS,
|
||||
BACKEND_PATH_PREFIXES,
|
||||
|
|
@ -45,7 +45,7 @@ def _is_backend_route(route) -> bool:
|
|||
return any(path.startswith(prefix) for prefix in BACKEND_PATH_PREFIXES)
|
||||
|
||||
|
||||
# See gateway/main.py for why the trim runs inside the lifespan instead of at
|
||||
# See litellm/runtime/gateway/main.py for why the trim runs inside the lifespan instead of at
|
||||
# module scope.
|
||||
_proxy_lifespan: Final = app.router.lifespan_context
|
||||
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
"""Gateway supervisor: assemble DATABASE_URL, start the in-container PgBouncer, then run uvicorn.
|
||||
|
||||
``gateway/main.py`` assembles ``DATABASE_URL`` inside every uvicorn worker, which
|
||||
``litellm/runtime/gateway/main.py`` assembles ``DATABASE_URL`` inside every uvicorn worker, which
|
||||
is fine for a plain Postgres URL but not for the pooler: PgBouncer must be
|
||||
started exactly once per pod, before the workers fork, and the workers must be
|
||||
handed the loopback URL it listens on. A pre-existing ``DATABASE_URL`` wins in
|
||||
|
|
@ -76,7 +76,7 @@ The dashboard is a single static HTML file. Either:
|
|||
- **If your browser blocks `file://` fetches:**
|
||||
|
||||
```bash
|
||||
cd scripts/adaptive_router_demo
|
||||
cd litellm/scripts/adaptive_router_demo
|
||||
python3 -m http.server 8080
|
||||
```
|
||||
|
||||
|
|
@ -95,7 +95,7 @@ Click **Connect**. The dashboard polls `GET /adaptive_router/state` every
|
|||
In a second terminal, replacing `<your-master-key>` with the key printed in step 1:
|
||||
|
||||
```bash
|
||||
uv run python scripts/adaptive_router_demo/traffic.py \
|
||||
uv run python litellm/scripts/adaptive_router_demo/traffic.py \
|
||||
--proxy-url http://localhost:4000 \
|
||||
--api-key <your-master-key> \
|
||||
--router smart-cheap-router \
|
||||
|
|
@ -9,7 +9,7 @@ For each test case:
|
|||
4. Prints PASS or FAIL with one line of reasoning.
|
||||
|
||||
Run:
|
||||
uv run python scripts/adaptive_router_demo/eval.py \
|
||||
uv run python litellm/scripts/adaptive_router_demo/eval.py \
|
||||
--proxy-url http://localhost:4000 \
|
||||
--api-key "$LITELLM_MASTER_KEY" \
|
||||
--router smart-cheap-router \
|
||||
|
|
@ -21,7 +21,7 @@ Why this shape:
|
|||
flow through routing + post-call hooks, so no API keys / no spend.
|
||||
|
||||
Run:
|
||||
uv run python scripts/adaptive_router_demo/traffic.py \\
|
||||
uv run python litellm/scripts/adaptive_router_demo/traffic.py \\
|
||||
--proxy-url http://localhost:4000 \\
|
||||
--api-key "$LITELLM_MASTER_KEY" \\
|
||||
--router smart-cheap-router \\
|
||||
|
|
@ -13,11 +13,11 @@ streaming SSE wire format (``message_start`` -> ``content_block_delta`` ->
|
|||
can be compared without depending on real provider latency.
|
||||
|
||||
Example:
|
||||
uv run python scripts/benchmark_anthropic_messages_perf.py \
|
||||
uv run python litellm/scripts/benchmark_anthropic_messages_perf.py \
|
||||
--label baseline --proxy-command ".venv/bin/litellm"
|
||||
|
||||
Compare an already-running proxy:
|
||||
uv run python scripts/benchmark_anthropic_messages_perf.py \
|
||||
uv run python litellm/scripts/benchmark_anthropic_messages_perf.py \
|
||||
--no-start-proxy --label current
|
||||
"""
|
||||
|
||||
|
|
@ -6,11 +6,11 @@ from any checkout. That makes it useful for comparing tags/commits without
|
|||
depending on real provider latency.
|
||||
|
||||
Example:
|
||||
uv run python scripts/benchmark_chat_completions_perf.py \
|
||||
uv run python litellm/scripts/benchmark_chat_completions_perf.py \
|
||||
--label current --requests 500 --concurrency 100
|
||||
|
||||
Compare another checkout:
|
||||
uv run python scripts/benchmark_chat_completions_perf.py \
|
||||
uv run python litellm/scripts/benchmark_chat_completions_perf.py \
|
||||
--label v1.83.14-stable --litellm-dir /tmp/litellm-v1.83.14-stable
|
||||
"""
|
||||
|
||||
|
|
@ -6,8 +6,8 @@ isolate per-call cost. Driving the full wrapper adds threadpool logging,
|
|||
gc, and other noise that swamps microsecond-scale changes here.
|
||||
|
||||
Example:
|
||||
uv run python scripts/benchmark_model_response_creator.py --label baseline
|
||||
uv run python scripts/benchmark_model_response_creator.py --label optimized
|
||||
uv run python litellm/scripts/benchmark_model_response_creator.py --label baseline
|
||||
uv run python litellm/scripts/benchmark_model_response_creator.py --label optimized
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
"""Measure serial sync/async OCR latency through a loopback HTTP provider
|
||||
|
||||
Run each callback mode in a fresh process against an installed release wheel:
|
||||
python -I scripts/benchmark_ocr_callbacks.py --callbacks none --label before \
|
||||
python -I litellm/scripts/benchmark_ocr_callbacks.py --callbacks none --label before \
|
||||
--expected-transport rust --iterations 200 --warmup 20 --output before-none.json
|
||||
Repeat with --callbacks noop and with the candidate wheel in a separate venv
|
||||
"""
|
||||
|
|
@ -199,7 +199,7 @@ async def main() -> int:
|
|||
raise RuntimeError("benchmark requires initially empty callback registrations")
|
||||
callback: Final = NoopCallback()
|
||||
if args.callbacks == "noop":
|
||||
litellm.callbacks.append(callback)
|
||||
litellm.logging_callback_manager.add_litellm_callback(callback)
|
||||
|
||||
rust_toggle: Final = getattr(litellm, "rust", None)
|
||||
if callable(rust_toggle):
|
||||
|
|
@ -539,28 +539,28 @@ Examples:
|
|||
export PROVIDER_URL='https://api.openai.com/v1/chat/completions'
|
||||
export LITELLM_PROXY_API_KEY="$LITELLM_MASTER_KEY"
|
||||
export PROVIDER_API_KEY='sk-openai-key'
|
||||
python scripts/benchmark_proxy_vs_provider.py
|
||||
python litellm/scripts/benchmark_proxy_vs_provider.py
|
||||
|
||||
# 2. Multiple runs for statistical accuracy (recommended)
|
||||
python scripts/benchmark_proxy_vs_provider.py --runs 5
|
||||
python litellm/scripts/benchmark_proxy_vs_provider.py --runs 5
|
||||
|
||||
# 3. Realistic load testing with concurrency limit
|
||||
python scripts/benchmark_proxy_vs_provider.py --max-concurrent 100 --requests 2000
|
||||
python litellm/scripts/benchmark_proxy_vs_provider.py --max-concurrent 100 --requests 2000
|
||||
|
||||
# 4. Quick test with fewer requests
|
||||
python scripts/benchmark_proxy_vs_provider.py --requests 100
|
||||
python litellm/scripts/benchmark_proxy_vs_provider.py --requests 100
|
||||
|
||||
# 5. Parallel execution (not recommended - may affect accuracy)
|
||||
python scripts/benchmark_proxy_vs_provider.py --parallel
|
||||
python litellm/scripts/benchmark_proxy_vs_provider.py --parallel
|
||||
|
||||
# 6. Custom timeout for slower endpoints
|
||||
python scripts/benchmark_proxy_vs_provider.py --timeout 120
|
||||
python litellm/scripts/benchmark_proxy_vs_provider.py --timeout 120
|
||||
|
||||
# 7. Combined options for comprehensive testing
|
||||
python scripts/benchmark_proxy_vs_provider.py --runs 3 --requests 500 --max-concurrent 50
|
||||
python litellm/scripts/benchmark_proxy_vs_provider.py --runs 3 --requests 500 --max-concurrent 50
|
||||
|
||||
# 8. Skip warmup (not recommended - may affect first request accuracy)
|
||||
python scripts/benchmark_proxy_vs_provider.py --no-warmup
|
||||
python litellm/scripts/benchmark_proxy_vs_provider.py --no-warmup
|
||||
""",
|
||||
)
|
||||
parser.add_argument(
|
||||
|
|
@ -7,7 +7,7 @@ and Bedrock Converse (ModelResponseStream). A full proxy benchmark adds
|
|||
FastAPI, HTTP, and TCP latency, which dilutes the per-chunk CPU signal.
|
||||
|
||||
Example:
|
||||
uv run python scripts/benchmark_streaming_chunk_overhead.py \\
|
||||
uv run python litellm/scripts/benchmark_streaming_chunk_overhead.py \\
|
||||
--streams 500 --chunks 200 --warmup 50 --repeats 5
|
||||
"""
|
||||
|
||||
|
|
@ -8,12 +8,10 @@ import tempfile
|
|||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
ROOT: Final = Path(__file__).resolve().parents[1]
|
||||
ROOT: Final = Path(__file__).resolve().parents[2]
|
||||
SOURCES: Final = (
|
||||
"litellm",
|
||||
"litellm/rust",
|
||||
".cargo",
|
||||
"rust-toolchain.toml",
|
||||
"model_prices_and_context_window.json",
|
||||
"README.md",
|
||||
"LICENSE",
|
||||
|
|
@ -57,7 +55,10 @@ def stage_core_distribution(source: Path, destination: Path, *, package: str = "
|
|||
|
||||
def ignored_sources(directory: str, names: list[str]) -> set[str]:
|
||||
workspace_sources: Final = (
|
||||
{"enterprise", "proxy-extras", "packaging", "rust", "cli", "ui", "vscode-extension", "deploy"}
|
||||
{
|
||||
"enterprise", "proxy-extras", "packaging", "rust", "cli", "scripts", "runtime",
|
||||
"schema.prisma", "ui", "vscode-extension", "deploy",
|
||||
}
|
||||
if Path(directory) == source / "litellm"
|
||||
else set()
|
||||
)
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
#!/usr/bin/env python3
|
||||
"""List the POST operations in the proxy's OpenAPI document that Swagger UI cannot explain.
|
||||
|
||||
Every finding is one line, ``<METHOD> <path> <code>``, so scripts/openapi_docs_gate.py can
|
||||
Every finding is one line, ``<METHOD> <path> <code>``, so litellm/scripts/openapi_docs_gate.py can
|
||||
count them per rule the same delta-vs-base way as the other lint gates. The rules:
|
||||
|
||||
OA001 no request body schema (the handler reads ``request.body()`` and never told FastAPI
|
||||
|
|
@ -67,11 +67,18 @@ def main() -> int:
|
|||
options: Final = TypeAdapter(dict[str, str | None]).validate_python(vars(parser.parse_args()))
|
||||
base_commit: Final = options["base_commit"]
|
||||
base: Final = base_commit if base_commit else _merge_base(options["base_ref"] or "")
|
||||
previous: Final = subprocess.run(
|
||||
("git", "show", f"{base}:schema.prisma"), capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
canonical: Final = subprocess.run(
|
||||
("git", "show", f"{base}:litellm/schema.prisma"), capture_output=True, text=True, check=False
|
||||
)
|
||||
previous: Final = (
|
||||
canonical.stdout
|
||||
if canonical.returncode == 0
|
||||
else subprocess.run(
|
||||
("git", "show", f"{base}:schema.prisma"), capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
)
|
||||
changes: Final = check_breaking_changes(
|
||||
parse_model_fields(previous), parse_model_fields(Path("schema.prisma").read_text())
|
||||
parse_model_fields(previous), parse_model_fields(Path("litellm/schema.prisma").read_text())
|
||||
)
|
||||
if changes:
|
||||
sys.stderr.write("\n".join(changes) + "\n")
|
||||
|
|
@ -1,10 +1,10 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Test-quality checker: the test-suite smells no linter enforces.
|
||||
|
||||
Sibling of scripts/check_type_discipline.py, same output contract
|
||||
Sibling of litellm/scripts/check_type_discipline.py, same output contract
|
||||
(``path:line: CODE message``) and same stdlib-only constraint, aimed at the test
|
||||
tree instead of the package. Each rule is a shape the testing-strategy audit
|
||||
measured and named; scripts/test_quality_gate.py fails any change that grows the
|
||||
measured and named; litellm/scripts/test_quality_gate.py fails any change that grows the
|
||||
codebase total of one past its merge-base count.
|
||||
|
||||
Rules
|
||||
|
|
@ -1171,6 +1171,8 @@ def collect_paths(raw: Iterable[str]) -> Iterator[Path]:
|
|||
"litellm/proxy-extras/",
|
||||
"litellm/packaging/",
|
||||
"litellm/rust/",
|
||||
"litellm/scripts/",
|
||||
"litellm/runtime/",
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
@ -1184,6 +1186,8 @@ def collect_paths(raw: Iterable[str]) -> Iterator[Path]:
|
|||
"litellm/proxy-extras/",
|
||||
"litellm/packaging/",
|
||||
"litellm/rust/",
|
||||
"litellm/scripts/",
|
||||
"litellm/runtime/",
|
||||
)
|
||||
):
|
||||
yield p
|
||||
|
|
@ -33,7 +33,7 @@ token = "test-api-key-with-high-entropy"
|
|||
|
||||
## Configured Ignore Patterns
|
||||
|
||||
These patterns are in `.gitguardian.yaml` for high-entropy test keys:
|
||||
These patterns are in `.github/gitguardian.yaml` for high-entropy test keys:
|
||||
- `sk-test-*` - OpenAI-style test keys
|
||||
- `sk-mock-*` - Mock API keys
|
||||
- `sk-fake-*` - Fake API keys
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Every pull request whose diff against its merge base touches one of the two cost map files or the pins file gets the
|
||||
file checks: the files parse, the backup copy matches the root file, and every capability pinned in
|
||||
ci_cd/cost_map_pins.json (a value verified against a live provider call where the provider's model listing says
|
||||
litellm/scripts/ci_cd/cost_map_pins.json (a value verified against a live provider call where the provider's model listing says
|
||||
otherwise) still holds in the tree that merging the pull request into its base branch produces, so a branch cut before
|
||||
a pin was added cannot lower it, and a pull request that deletes the pins file fails too. A pull request that leaves
|
||||
all three untouched skips them, since merging it keeps the base branch's copies and its head tree only carries
|
||||
|
|
@ -26,14 +26,15 @@ from types import MappingProxyType
|
|||
from typing import Final
|
||||
|
||||
COST_MAP_PATH: Final = "model_prices_and_context_window.json"
|
||||
COST_MAP_PATHS: Final = ("litellm/data/model_prices_and_context_window.json", COST_MAP_PATH)
|
||||
BACKUP_PATH: Final = "litellm/model_prices_and_context_window_backup.json"
|
||||
PINS_PATH: Final = "ci_cd/cost_map_pins.json"
|
||||
PINS_PATH: Final = "litellm/scripts/ci_cd/cost_map_pins.json"
|
||||
PINS_PATHS: Final = (
|
||||
"litellm/scripts/ci_cd/cost_map_pins.json",
|
||||
"scripts/ci_cd/cost_map_pins.json",
|
||||
"ci_cd/cost_map_pins.json",
|
||||
)
|
||||
GUARDED_PATHS: Final = (COST_MAP_PATH, BACKUP_PATH)
|
||||
GUARDED_PATHS: Final = (*COST_MAP_PATHS, BACKUP_PATH)
|
||||
CHECKED_PATHS: Final = (*GUARDED_PATHS, *PINS_PATHS)
|
||||
BOT_BRANCH_PREFIX: Final = "litellm_cost_map_sync_"
|
||||
PRICE_SYNC_BRANCH_PREFIX: Final = "litellm-providers/price-sync"
|
||||
|
|
@ -258,7 +259,8 @@ def _git(*args: str) -> str | None:
|
|||
|
||||
def snapshot(revision: str, *, read_git: Callable[[str, str], str | None] = _git) -> Snapshot:
|
||||
return Snapshot(
|
||||
*(read_git("show", f"{revision}:{path}") or "" for path in GUARDED_PATHS),
|
||||
next((text for path in COST_MAP_PATHS if (text := read_git("show", f"{revision}:{path}")) is not None), ""),
|
||||
read_git("show", f"{revision}:{BACKUP_PATH}") or "",
|
||||
next((text for path in PINS_PATHS if (text := read_git("show", f"{revision}:{path}")) is not None), ""),
|
||||
)
|
||||
|
||||
|
|
@ -238,7 +238,7 @@ def create_migration(
|
|||
skip_freshness_check (bool): Skip the "branch is up to date" check.
|
||||
Only for intentional migrations against an older base.
|
||||
"""
|
||||
root_dir = Path(__file__).resolve().parents[2]
|
||||
root_dir = Path(__file__).resolve().parents[3]
|
||||
|
||||
if skip_freshness_check:
|
||||
print(
|
||||
|
|
@ -254,7 +254,7 @@ def create_migration(
|
|||
migrations_dir = (
|
||||
root_dir / "litellm" / "proxy-extras" / "litellm_proxy_extras" / "migrations"
|
||||
)
|
||||
schema_path = root_dir / "schema.prisma"
|
||||
schema_path = root_dir / "litellm/schema.prisma"
|
||||
|
||||
# Create temporary PostgreSQL database
|
||||
with testing.postgresql.Postgresql() as postgresql:
|
||||
|
|
@ -26,6 +26,7 @@ from pydantic import BaseModel, ConfigDict, TypeAdapter
|
|||
|
||||
WATCHED: Final = (
|
||||
"schema.prisma",
|
||||
"litellm/schema.prisma",
|
||||
"litellm/proxy/schema.prisma",
|
||||
"litellm/proxy-extras/litellm_proxy_extras/schema.prisma",
|
||||
)
|
||||
|
|
@ -24,7 +24,7 @@ import {
|
|||
type Schema,
|
||||
} from "./classify-issue";
|
||||
import { MANIFEST, NAMESPACES } from "./issue-labels";
|
||||
import schemaJson from "../.github/prompts/issue-classifier.schema.json";
|
||||
import schemaJson from "../../.github/prompts/issue-classifier.schema.json";
|
||||
|
||||
const schema = schemaJson as Schema;
|
||||
const routes = routesOf(schema);
|
||||
|
|
@ -133,7 +133,7 @@ describe("sections", () => {
|
|||
test("the known headings are exactly the field labels of the two issue forms", async () => {
|
||||
const labels = await Promise.all(
|
||||
["bug_report.yml", "feature_request.yml"].map(async (file) => {
|
||||
const form = Bun.YAML.parse(await Bun.file(`${import.meta.dir}/../.github/ISSUE_TEMPLATE/${file}`).text()) as {
|
||||
const form = Bun.YAML.parse(await Bun.file(`${import.meta.dir}/../../.github/ISSUE_TEMPLATE/${file}`).text()) as {
|
||||
readonly body: readonly { readonly attributes?: { readonly label?: string } }[];
|
||||
};
|
||||
return form.body.flatMap((field) => (field.attributes?.label === undefined ? [] : [field.attributes.label.trim()]));
|
||||
|
|
@ -376,8 +376,8 @@ export function readConfig(
|
|||
|
||||
if (import.meta.main) {
|
||||
const { token, apiBase, apiKey, ...config } = readConfig(process.env, new Date());
|
||||
const prompt = await Bun.file(`${import.meta.dir}/../.github/prompts/issue-classifier.md`).text();
|
||||
const schema = (await Bun.file(`${import.meta.dir}/../.github/prompts/issue-classifier.schema.json`).json()) as Schema;
|
||||
const prompt = await Bun.file(`${import.meta.dir}/../../.github/prompts/issue-classifier.md`).text();
|
||||
const schema = (await Bun.file(`${import.meta.dir}/../../.github/prompts/issue-classifier.schema.json`).json()) as Schema;
|
||||
const verdict = await classifyIssue(githubApi(token), litellmClient(apiBase, apiKey), config, prompt, schema);
|
||||
if (verdict === null) {
|
||||
console.error(`#${config.issueNumber}: edit ignored, the issue is already classified or older than the edit window`);
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Reference in a new issue