Merge pull request #45011 from BerriAI/litellm_pgb_bump_1103

chore(docker): bump pgbouncer to 1.26.0

.agents/skills/database-access/SKILL.md

@@ -1,73 +0,0 @@
----
-name: database-access
-description: Organize database repositories and proxy dependency wiring, manage connection ownership, or port Prisma operations to Rust and SQLx without expanding migration scope
-metadata:
-  internal: true
----
-
-# Database access
-
-## Situations
-
-- Adding a query or an operation to a repository, read [adding a query](references/python-repository.md#adding-a-query)
-- Moving existing Prisma calls behind a repository, read [moving existing access](references/python-repository.md#moving-existing-access-into-a-repository)
-- Porting a repository from Prisma to Rust and SQLx, read [porting to SQLx](references/python-repository.md#porting-a-repository-to-sqlx)
-- Injecting a repository into a route, read [Python and FastAPI](references/python-fastapi.md) or [Rust and Axum](references/rust-axum.md)
-
-## Adding new database access
-
-- Put Python database access behind an operation in `litellm/repositories`
-- Give each repository its own FastAPI provider, so a route builds only the repositories it uses
-- A provider borrows existing handles and performs no database I/O during construction
-- Define the operations a route calls as a Protocol in `litellm/models/<feature>.py`, next to its domain types
-  - The Protocol takes and returns domain types, never Prisma models, raw rows or serialized payloads
-  - Parsing and serialization of stored values live in the repository
-  - Add a Protocol when a repository gets a provider, not up front for every repository
-- Keep query execution inside the repository, with one implementation per repository and no backend switch or fallback
-- A repository is either all Prisma or all native, never a mix of both
-- Prisma is fine for the implementation, adding a query does not require a Rust port
-- Native Rust callers use `litellm-rust/crates/db` directly
-
-## Framework wiring
-
-- Python routes receive a per-repository dependency such as `UserBannerRepositoryDep` and call its operations, such as `banners.get_banner()`
-- `DatabaseDep` lives in `litellm/proxy/deps/database.py`, repository providers in `litellm/proxy/deps/repositories.py`
-  - A provider returns the feature's Protocol, so routes never see which backend implements it
-  - Dependencies used by one feature only, like the banner audit binding, stay next to that feature's routes
-- Repositories never import FastAPI, so jobs and helpers can use them without a request
-- Use named FastAPI providers with `Annotated[..., Depends(...)]`, ordinary helpers and jobs receive explicit arguments
-- Preserve each route's no-database behavior, authorization precedence and background-task scheduling
-- Read [Python and FastAPI](references/python-fastapi.md) for provider examples, lifespan state and route verification
-- Read [Rust and Axum](references/rust-axum.md) when adding database access to Rust handlers
-
-## Connection ownership
-
-- Keep connection setup and shutdown separate from repositories, which own no pools
-- Reuse existing handles, never connect per request or disconnect from a repository operation
-- Give each pool one owner for setup and shutdown, borrowed handles never close it
-- Finish requests, jobs and queued database work before closing connections
-- Clean up resources opened before a startup failure, without a second disconnect fallback
-- Add a backend to the existing lifecycle owner and pass its handle only to repositories that need it
-- Preserve cancellation, timeout, error and transaction semantics across the native bridge
-
-## Changing ownership or reloading resources
-
-- Inventory initialization, reload, tests, background jobs and shutdown together
-- Move every writer when changing authoritative storage, copying a replaceable global into state is insufficient
-- A frozen repository does not make its clients immutable
-- Decide whether each request captures one resource generation or reads it again during processing
-- Build and validate replacements before publishing them, preserving the old generation on failure
-- Serialize reloads and coordinate publication with shutdown
-- Drain streams and background users before retiring old resources, retaining every generation grows resource use
-- Define multi-worker coordination separately from an in-process swap
-- Choose mechanisms such as `ArcSwap` only after defining publication, capture and retirement
-
-## Migration context and boundaries
-
-- Long-term direction is to centralize Python database access in repositories, then replace their Prisma implementations with Rust APIs one repository at a time
-- Prisma and native repositories coexist during the transition, unrelated existing calls can stay as they are
-- Prisma and SQLx keep separate client pools to PgBouncer, which pools connections to PostgreSQL
-  - Size both layers together and preserve reader/writer routing and pooling-mode compatibility
-- PgBouncer does not share transactions between Prisma and SQLx, keep each atomic unit of work on one backend and transaction
-- Never retry a failed Rust write through Prisma when it may already have committed
-- Prisma owns schema migrations today, eventual SQLx migration ownership and Prisma removal are separate planned steps, not part of a query port

.agents/skills/database-access/references/python-fastapi.md

@@ -1,101 +0,0 @@
-# Python and FastAPI
-
-## Existing pattern
-
-- `UserBannerStore` in `litellm/models/user_banner.py` is the Protocol the banner routes call
-- `get_database` in `litellm/proxy/deps/database.py` reads the lifespan-owned handle from request state
-- `get_user_banner_repository` in `litellm/proxy/deps/repositories.py` builds the banner repository from that handle
-- Choose each repository's replica routing once in its provider, never at call sites
-- A handler receives `UserBannerStore | None`, never a Prisma client or a concrete repository
-- `UserBannerRepository` implements `UserBannerStore` structurally, keeping the existing Prisma table, serialization and configuration-sync behavior
-- The banner audit provider binds the legacy audit callable at request time, without performing audit I/O
-- A native repository can later implement the same Protocol without changing handler signatures, see [porting a repository to SQLx](python-repository.md#porting-a-repository-to-sqlx)
-
-## Dependency injection
-
-- Add one provider and one `Annotated` alias per repository in `litellm/proxy/deps/repositories.py`
-- Annotate the provider and alias with the feature's Protocol, not the concrete repository
-- A route depends only on the repositories it calls, so a request never builds repositories it does not use
-- Do not build repositories once at lifespan, a reconnect can replace the Prisma client they wrap
-
-```python
-class UserBannerStore(Protocol):
-    async def get_banner(self) -> UserBanner: ...
-
-    async def save_banner(self, banner: UserBanner) -> None: ...
-```
-
-```python
-async def get_user_banner_repository(database: DatabaseDep) -> UserBannerStore | None:
-    return None if database is None else UserBannerRepository(database.prisma)
-
-
-UserBannerRepositoryDep: TypeAlias = Annotated[UserBannerStore | None, Depends(get_user_banner_repository)]
-```
-
-- FastAPI caches each provider within a request, so routes and sub-dependencies share one instance
-- Ordinary helpers and jobs receive explicit arguments, never unresolved `Depends` defaults
-- A route still reading the `prisma_client` global or building a repository inline switches to the repository's `...RepositoryDep`, keeping its no-database behavior and error precedence
-
-```python
-async def get_user_banner(banners: UserBannerRepositoryDep) -> UserBanner:
-    if banners is None:
-        return UserBanner()
-    return await banners.get_banner()
-```
-
-## Lifespan state
-
-- Prisma startup owns connection setup, authentication refresh, health checks and migrations
-- `Database` holds the initialized Prisma client and the `NativeDatabase` handle, without holding fields per feature
-  - The native handle opens no pool yet, its `connect` and `close` are noops
-- The proxy lifespan builds the handle with `Database.open`, registers `Database.aclose` on its exit stack and publishes it in request state
-  - Callers only open and close `Database`, which closes Prisma and the native handle together, and `open` closes Prisma itself when the native connect fails
-- Starlette shallow-copies lifespan state into request state, which is distinct from `app.state`
-- The database module never imports a repository, legacy globals borrow its handle without owning shutdown
-- Drain requests, jobs, logging, spend and audit work before database teardown
-- The exit stack closes the handle on exit, including startup failures after the client is created
-- When adding native resources, initialize them in the serving process, respecting runtime fork restrictions
-- Await native shutdown instead of depending on Python garbage collection
-- Apply the [connection ownership rules](../SKILL.md#connection-ownership) when changing the owner
-
-## Timing and errors
-
-- FastAPI resolves dependencies before entering the handler
-- Handler exception handling cannot catch failures raised during dependency resolution
-- A universal required-database dependency can override authorization and validation behavior
-- Banner GET without a database returns the default disabled banner
-- Banner PATCH checks the admin role before returning the existing missing-database error
-- Invalid stored banner data falls back to the default, but storage failures still propagate
-- Banner writes finish before the success audit is queued as a FastAPI background task
-- Queue post-response work with `BackgroundTasks`, not bare `asyncio.create_task`, so the in-flight drain waits for it before the database closes
-- Do not move background work into an awaited provider merely to remove an import
-
-## Verification
-
-- Exercise real FastAPI handlers using `dependency_overrides` for external boundaries
-- Keep production parsing and persistence code real
-- Restore overrides after each test
-- Verify complete response bodies, status codes and OpenAPI against the base
-- Include combined failure conditions to pin error precedence
-- Check saved values, fresh revisions, audit contents and failed-write behavior
-- Check lifespan state with a running lifespan, not only a TestClient that skips startup
-- Verify providers do not connect, query, disconnect or audit eagerly
-- Use real PostgreSQL and live proxy requests when changing persistence behavior
-
-## Scope
-
-- Preserve direct Python call sites or migrate them explicitly
-- Inventory imports, state writers, side effects and error ordering before migrating a feature
-- Preserve configuration sync and serialization, and state which legacy reads remain
-- Do not migrate every route in a large module just to establish the pattern
-- Do not invent providers for resources without consumers
-
-## Framework references
-
-- [Lifespan events](https://fastapi.tiangolo.com/advanced/events/) and [Starlette lifespan state](https://starlette.dev/lifespan/)
-- [`SessionDep`-style `Annotated` aliases](https://fastapi.tiangolo.com/tutorial/sql-databases/#create-a-session-dependency), which `DatabaseDep` and `UserBannerRepositoryDep` follow
-- [Dependency composition and caching](https://fastapi.tiangolo.com/tutorial/dependencies/sub-dependencies/)
-- [Background tasks and dependencies with `yield`](https://fastapi.tiangolo.com/advanced/advanced-dependencies/#background-tasks-and-dependencies-with-yield-technical-details)
-- [Dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
-- [Testing lifespan](https://fastapi.tiangolo.com/advanced/testing-events/)

.agents/skills/database-access/references/python-repository.md

@@ -1,119 +0,0 @@
-# Python repositories
-
-## Adding a query
-
-Use this when a feature needs a read or write that no repository offers yet, including one more operation on an existing repository
-
-### Before starting
-
-- Search `litellm/repositories` for a repository that already owns the table, and extend it instead of adding a second one
-- Check `litellm/models/<feature>.py` for an existing Protocol and domain types
-- Decide the route's behavior when no database is configured, and where that check sits relative to authorization and validation
-
-### Steps
-
-1. Domain types and Protocol in `litellm/models/<feature>.py`
-   - The Protocol takes and returns domain types, never Prisma models, raw rows or serialized payloads
-   - Add the Protocol together with the provider, not for repositories nothing injects
-2. Repository in `litellm/repositories/<feature>_repository.py`
-   - Implement the operation with Prisma, a Rust port is not required
-   - Compose the Prisma client and type the tables it touches with `TableActions` from `prisma_protocols.py`, never inherit `PrismaTableRepository`
-   - Parse and serialize stored values here, so callers only see domain types
-   - Never import FastAPI, connect, disconnect or choose replicas per call
-3. Provider and `Annotated` alias in `litellm/proxy/deps/repositories.py`, see [dependency injection](python-fastapi.md#dependency-injection)
-4. The route takes `<Feature>RepositoryDep`, handles `None` with the behavior decided above, and calls the operation
-5. Jobs and helpers that need the operation receive the repository as an explicit argument, built by their caller from the handle it already holds
-
-### Done when
-
-- Handler tests drive the real route with `dependency_overrides` and the real repository code, see [verification](python-fastapi.md#verification)
-- Tests cover the no-database path, error precedence, and what was actually stored
-- Persistence changes are checked against real PostgreSQL through a live proxy
-
-### Out of scope
-
-- Porting the query to Rust, see [porting a repository to SQLx](#porting-a-repository-to-sqlx)
-- Moving unrelated direct Prisma calls in the same module
-
-## Moving existing access into a repository
-
-Use this when code calls Prisma directly, through `prisma_client.db.<table>` or a helper, and the calls should live behind a repository. The backend stays Prisma
-
-### Before starting
-
-- List every call being moved, its callers, and the transaction or batch it runs in
-- List the module's imports, global state writers, side effects such as audit and config sync, and error ordering
-- Record current return values, raised errors and response bodies, these are what the change must preserve
-
-### Steps
-
-1. Move the existing Prisma code into the relevant repository in `litellm/repositories` without rewriting the queries
-2. Add domain types and a Protocol in `litellm/models/<feature>.py` when a route will inject the repository
-3. Replace each direct call with the repository operation, through a provider for routes and explicit arguments for jobs and helpers, see [Python and FastAPI](python-fastapi.md)
-4. Calls sharing a transaction move into the same repository or unit of work in one change
-5. State which legacy reads remain and why, when the change does not move all of them
-
-### Done when
-
-- Focused regression tests pin the preserved return values, errors and transaction behavior
-- Response bodies, status codes and OpenAPI match the base
-- Configuration sync and serialization behave the same, including invalid stored data
-
-### Out of scope
-
-- Switching the backend, even when an equivalent Rust query exists
-- Converting other routes in the module just to establish the pattern
-- Cleanups unrelated to the moved calls
-
-## Porting a repository to SQLx
-
-Status: TBD. No repository has been ported yet. `database.native` is a `NativeDatabase` from `crates/python-bridge/src/db.rs` whose `connect` and `close` are noops that open no pool, and the PyO3 repository surface does not exist. A real pool takes typed writer/reader settings and a token provider, never the Prisma `DATABASE_URL`, which carries Prisma-only params and rotating IAM tokens. The constraints below are settled, the steps are a sketch to revise after the first port lands
-
-Use this only when a port is explicitly requested. Repository extraction does not require one
-
-### When not to port
-
-- An equivalent Rust or SQLx query existing is not a reason to switch a Prisma caller
-- Leave existing backend selection unchanged unless integration is explicitly requested
-- Never port part of a repository, it is either all Prisma or all native
-
-### Before starting
-
-- Identify the repository's inputs, outputs, errors and transaction boundary
-- List repositories that share a transaction with it, they move together
-- Check `litellm-rust/crates/db` for an equivalent query and reuse it when its behavior matches
-
-### Steps
-
-1. Rust queries, following `litellm-rust/crates/db/AGENTS.md`
-   - Static SQL in `queries/`, compile-checked with SQLx, typed inputs and row projections in `src/`
-   - Dynamic filters and sort keys become closed Rust enums that select static SQL, every value is bound
-   - Refresh SQLx query metadata after SQL changes
-2. PyO3 surface exposing typed operations, never SQL text, table names or column names from Python. TBD: where it lives and how the handle reaches it
-3. Native repository implementing the same Protocol in `litellm/repositories`
-   - Converts typed PyO3 results to domain types, preserving defaults, invalid-data handling and storage errors
-   - Python conversion stays in the bridge, not in `crates/db`
-4. Switch the provider in `litellm/proxy/deps/repositories.py`, the Protocol and routes stay unchanged
-
-```diff
- async def get_user_banner_repository(database: DatabaseDep) -> UserBannerStore | None:
--    return None if database is None else UserBannerRepository(database.prisma)
-+    return None if database is None else NativeUserBannerRepository(database.native)
-```
-
-5. Delete the Prisma repository in the same change and switch every caller, including jobs and helpers
-
-### Done when
-
-- Behavior is verified against real PostgreSQL, covering null handling, conflicts, ordering, errors and transaction semantics
-- `DATABASE_URL=... cargo test -p litellm-db --locked -- --include-ignored` passes against a Prisma-migrated database
-- Live proxy requests show unchanged responses. TBD: a Python-side real-PostgreSQL test harness
-
-### Rollback
-
-- Roll back the release, never retry an uncertain native write through Prisma
-
-### Out of scope
-
-- SQLx owning schema migrations, and removing Prisma
-- Converting unrelated routes to dependency injection

.agents/skills/database-access/references/rust-axum.md

@@ -1,4 +0,0 @@
-# Rust and Axum
-
-- This should use same `crates/db` what is exposed to Python world.
-- Details are TBD.

.cargo/config.toml

@@ -17,6 +17,3 @@ rustflags = ["-C", "link-arg=-undefined", "-C", "link-arg=dynamic_lookup"]
 
 [target.aarch64-apple-darwin]
 rustflags = ["-C", "link-arg=-undefined", "-C", "link-arg=dynamic_lookup"]
-
-[env]
-SQLX_OFFLINE = "true"

.circleci/AGENTS.md

@@ -1,7 +0,0 @@
-# .circleci
-
-The `integration` workflow on `main` runs from a scheduled pipeline in the CircleCI project settings, cron `9 0-8,14-23 * * *` (UTC) on `main` with no pipeline parameters. It has no `triggers:` block in this file because a workflow with one runs only on its schedule, so the API pipelines that set `run_migration_tests` or `routing_parity_base` would stop running it. If main stops getting integration runs, check that the schedule still exists and recreate it with these values
-
-Jobs clone with the `checkout_over_https` command, never the built-in `checkout` step, because CircleCI falls back to the project's SSH checkout key without its GitHub App access, GitHub rejects that key, and the public repo needs no key. It fetches the job's commit with full history and no file contents (`--filter=blob:none`), like the built-in step did, so `scripts/path_filter.sh` can find a merge base
-
-A step that checks out files from another commit fetches that commit with `--no-filter`, as `routing_parity` does, because the clone has no file contents and git would otherwise download each missing file in its own request, which took 210 to 896 seconds for the routing parity base

.circleci/scripts/classify_changes.sh

@@ -1,30 +1,26 @@
 #!/usr/bin/env bash
 set -uo pipefail
 
-category="${1:?usage: classify_changes.sh <backend|client|ui|cost-map-only|mcp-dependencies|windows-release|redis-compat>}"
+category="${1:?usage: classify_changes.sh <backend|client|ui|provider-harness|cost-map-only|mcp-dependencies>}"
 
 has_client=false
 has_backend=false
 has_ci=false
+has_provider_harness=false
 has_cost_map=false
 has_mcp_dependencies=false
-has_windows_release=false
-has_redis_compat=false
 outside_cost_map_set=false
 while IFS= read -r file || [ -n "$file" ]; do
   [ -n "$file" ] || continue
-  case "$file" in
-    litellm/_redis.py | litellm/_redis_credential_provider.py | litellm/caching/redis_cache.py | litellm/caching/evicted_client_closer.py | tests/unit/test_redis.py | tests/integration/sdk/test_redis_cluster_iam_auth.py | tests/unit/caching/test_redis_connection_pool.py | tests/unit/caching/test_redis_cluster_cache.py | tests/unit/caching/test_evicted_client_closer.py | .github/workflows/test-unit.yml | .github/actions/detect-changes/* | .github/scripts/detect_changes.sh | .circleci/scripts/classify_changes.sh | tests/integration/run.py | pyproject.toml | uv.lock)
-      has_redis_compat=true ;;
-  esac
   case "$file" in
     *.md | *.mdx) : ;;
-    pyproject.toml | */pyproject.toml | uv.lock | uv.toml | .python-version | rust-toolchain.toml | litellm-rust/* | litellm/__init__.py | litellm/proxy/proxy_server.py | litellm/*mcp* | tests/*mcp* | litellm/integrations/arize/* | tests/base_sdk_tests/* | scripts/check_mcp_sdk_install.py | .github/workflows/test-mcp-dependency-resolution.yml | .github/actions/detect-changes/* | .github/actions/setup-uv-with-retries/* | .github/actions/cache-cargo-build/* | .github/scripts/detect_changes.sh | .github/scripts/uv_sync_with_retries.sh | .circleci/scripts/classify_changes.sh | tests/unit/test_circleci_path_filter.py | tests/unit/test_detect_changes.py)
+    pyproject.toml | */pyproject.toml | uv.lock | uv.toml | .python-version | rust-toolchain.toml | litellm-rust/* | litellm/__init__.py | litellm/proxy/proxy_server.py | litellm/*mcp* | tests/*mcp* | litellm/integrations/arize/* | tests/base_sdk_tests/* | scripts/check_mcp_sdk_install.py | .github/workflows/test-mcp-dependency-resolution.yml | .github/actions/detect-changes/* | .github/actions/setup-uv-with-retries/* | .github/actions/cache-cargo-build/* | .github/scripts/detect_changes.sh | .github/scripts/uv_sync_with_retries.sh | .circleci/scripts/classify_changes.sh | tests/test_litellm/test_circleci_path_filter.py | tests/test_litellm/test_detect_changes.py)
       has_mcp_dependencies=true ;;
   esac
   case "$file" in
-    litellm-rust/* | litellm/rust_bridge/* | rust-toolchain.toml | pyproject.toml | uv.lock | tests/windows_tests/* | .circleci/*)
-      has_windows_release=true ;;
+    tests/e2e/*/*.py) : ;;
+    tests/e2e/*.py | tests/code_coverage_tests/test_provider_cache.py | tests/code_coverage_tests/test_provider_replay_harness.py | tests/test_litellm/test_circleci_path_filter.py | .circleci/* | pyproject.toml | uv.lock)
+      has_provider_harness=true ;;
   esac
   case "$file" in
     ui/* | tests/e2e/ui/*) has_client=true ;;
@@ -33,9 +29,9 @@ while IFS= read -r file || [ -n "$file" ]; do
     *) has_backend=true ;;
   esac
   case "$file" in
-    model_prices_and_context_window.json | litellm/model_prices_and_context_window_backup.json)
+    model_prices_and_context_window.json | litellm/model_prices_and_context_window_backup.json | model_prices_and_context_window.schema.json)
       has_cost_map=true ;;
-    tests/unit/proxy/*) : ;;
+    tests/test_litellm/* | tests/proxy_unit_tests/*) : ;;
     *) outside_cost_map_set=true ;;
   esac
 done
@@ -47,11 +43,8 @@ case "$category" in
   cost-map-only)
     { [ "$has_cost_map" = true ] && [ "$outside_cost_map_set" = false ]; } && echo run || echo skip
     ;;
-  windows-release)
-    [ "$has_windows_release" = true ] && echo run || echo skip
-    ;;
-  redis-compat)
-    [ "$has_redis_compat" = true ] && echo run || echo skip
+  provider-harness)
+    [ "$has_provider_harness" = true ] && echo run || echo skip
     ;;
   backend)
     [ "$has_backend" = true ] && echo run || echo skip

.circleci/scripts/path_filter.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 set -uo pipefail
 
-category="${1:?usage: path_filter.sh <backend|client|windows-release>}"
+category="${1:?usage: path_filter.sh <backend|client|provider-harness>}"
 here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 
 run_full() {
@@ -11,7 +11,7 @@ run_full() {
 
 [ -n "${CIRCLE_PULL_REQUEST:-}" ] || run_full "not a pull request"
 
-candidate_bases="${PATH_FILTER_BASE_BRANCH:-main}"
+candidate_bases="main litellm_internal_staging litellm_oss_staging"
 merge_base=""
 for base in $candidate_bases; do
   git fetch --quiet origin "$base" 2>/dev/null || continue

.circleci/scripts/prepare_replica_roles.py

@@ -1,52 +0,0 @@
-from __future__ import annotations
-
-import os
-from typing import Final
-from urllib.parse import urlsplit, urlunsplit
-
-import psycopg
-
-DATABASE_URL: Final = os.environ["DATABASE_URL"]
-
-
-def postgres_url() -> str:
-    parsed: Final = urlsplit(DATABASE_URL)
-    return urlunsplit(parsed._replace(path="/postgres"))
-
-
-def main() -> None:
-    with psycopg.connect(postgres_url(), autocommit=True) as admin:
-        admin.execute("CREATE EXTENSION IF NOT EXISTS pg_stat_statements")
-        admin.execute("CREATE ROLE litellm_writer LOGIN PASSWORD 'litellm-writer' NOSUPERUSER")
-        admin.execute("CREATE ROLE litellm_reader LOGIN PASSWORD 'litellm-reader' NOSUPERUSER NOINHERIT")
-        admin.execute("ALTER ROLE litellm_reader SET default_transaction_read_only = on")
-        admin.execute("ALTER DATABASE circle_test OWNER TO litellm_writer")
-        admin.execute("GRANT CONNECT ON DATABASE circle_test TO litellm_reader")
-    with psycopg.connect(DATABASE_URL, autocommit=True) as admin:
-        admin.execute("GRANT USAGE ON SCHEMA public TO litellm_reader")
-        admin.execute(
-            "ALTER DEFAULT PRIVILEGES FOR ROLE litellm_writer IN SCHEMA public GRANT SELECT ON TABLES TO litellm_reader"
-        )
-        admin.execute("GRANT SELECT ON ALL TABLES IN SCHEMA public TO litellm_reader")
-
-    parsed: Final = urlsplit(DATABASE_URL)
-    reader_url: Final = urlunsplit(
-        parsed._replace(netloc=f"litellm_reader:litellm-reader@{parsed.hostname}:{parsed.port}")
-    )
-    writer_url: Final = urlunsplit(
-        parsed._replace(netloc=f"litellm_writer:litellm-writer@{parsed.hostname}:{parsed.port}")
-    )
-    with psycopg.connect(reader_url, autocommit=True) as reader:
-        assert reader.execute("SHOW transaction_read_only").fetchone() == ("on",)
-        try:
-            reader.execute("CREATE TABLE integration_readonly_probe (id int)")
-        except psycopg.errors.ReadOnlySqlTransaction:
-            pass
-        else:
-            raise AssertionError("litellm_reader executed a write statement")
-    with psycopg.connect(writer_url, autocommit=True) as writer:
-        assert writer.execute("SELECT current_user").fetchone() == ("litellm_writer",)
-
-
-if __name__ == "__main__":
-    main()

.circleci/scripts/run_integration.sh

@@ -1,22 +1,15 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
+if [ "${GITHUB_ACTIONS:-}" = true ]; then
+  echo "Integration contracts are owned by CircleCI" >&2
+  exit 1
+fi
+
 suite="${1:?integration suite required}"
-mode="${2:-standard}"
-side="${3:-}"
-requested_workers="${INTEGRATION_WORKERS:-}"
-license_env=()
-if [ -n "${LITELLM_LICENSE:-}" ]; then
-  license_env=("LITELLM_LICENSE=$LITELLM_LICENSE")
-fi
-if [ "$mode" = replica ]; then
-  results="test-results/integration-${suite}-replica"
-elif [ "$mode" = parity ]; then
-  results="test-results/parity-${suite}/${side:?parity side required}"
-else
-  results="test-results/integration-${suite}"
-fi
+results="test-results/integration-${suite}"
 mkdir -p "$results"
+shard_timeout=11m
 integration_identity="$(.venv/bin/python -c 'import uuid; print(uuid.uuid4().hex)')"
 upstream_pid=""
 proxy_pid=""
@@ -26,7 +19,6 @@ guard_created=false
 guard_installed=false
 guard6_created=false
 guard6_installed=false
-egress_cgroup=litellm-integration
 cleanup() {
   original_status=$?
   trap - EXIT INT TERM
@@ -48,14 +40,14 @@ cleanup() {
     fi
   done
   if [ "$guard_installed" = true ]; then
-    sudo iptables -D OUTPUT -m cgroup --path "$egress_cgroup" -j integration_only || original_status=1
+    sudo iptables -D OUTPUT -m owner --uid-owner "$(id -u)" -j integration_only || original_status=1
   fi
   if [ "$guard_created" = true ]; then
     sudo iptables -F integration_only || original_status=1
     sudo iptables -X integration_only || original_status=1
   fi
   if [ "$guard6_installed" = true ]; then
-    sudo ip6tables -D OUTPUT -m cgroup --path "$egress_cgroup" -j integration_only || original_status=1
+    sudo ip6tables -D OUTPUT -m owner --uid-owner "$(id -u)" -j integration_only || original_status=1
   fi
   if [ "$guard6_created" = true ]; then
     sudo ip6tables -F integration_only || original_status=1
@@ -72,7 +64,7 @@ export PATH="$PWD/.venv/bin:$PATH"
 export PYTHONPATH="$PWD:$PWD/tests:$PWD/tests/e2e"
 export DATABASE_URL="postgresql://postgres:postgres@127.0.0.1:5432/circle_test"
 export REDIS_HOST=127.0.0.1 REDIS_PORT=6379
-export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 16)" LITELLM_SALT_KEY=sk-integration-salt
+export LITELLM_MASTER_KEY=sk-integration-master LITELLM_SALT_KEY=sk-integration-salt
 export LITELLM_MODE=PRODUCTION LITELLM_LOCAL_MODEL_COST_MAP=True
 export STORE_MODEL_IN_DB=True AWS_EC2_METADATA_DISABLED=true DO_NOT_TRACK=1
 export INTEGRATION_PROXY_URL=http://127.0.0.1:4000
@@ -84,41 +76,27 @@ if [ "$suite" = browser ]; then
   export LITELLM_UI_PATH="$PWD/ui/litellm-dashboard/out"
   test -f "$LITELLM_UI_PATH/index.html"
 fi
-export INTEGRATION_SEED="$(.venv/bin/python -c 'import hashlib,os; print(int(hashlib.sha256(((os.environ.get("CIRCLE_SHA1") or os.environ.get("GITHUB_SHA") or "local") + (os.environ.get("CIRCLE_WORKFLOW_ID") or os.environ.get("GITHUB_RUN_ID") or "local")).encode()).hexdigest()[:8],16))')"
+export INTEGRATION_SEED="$(.venv/bin/python -c 'import hashlib,os; print(int(hashlib.sha256((os.environ.get("CIRCLE_SHA1", "local") + os.environ.get("CIRCLE_WORKFLOW_ID", "local")).encode()).hexdigest()[:8],16))')"
 export INTEGRATION_ORDER_SEED="$INTEGRATION_SEED"
 
 uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma > "$results/prisma-generate.log" 2>&1
 
-export INTEGRATION_PROXY_DATABASE_URL=""
-export INTEGRATION_PROXY_READ_REPLICA_URL=""
-export INTEGRATION_ROUTING=""
-if [ "$mode" = replica ] || [ "$mode" = parity ]; then
-  .venv/bin/python .circleci/scripts/prepare_replica_roles.py > "$results/prepare-replica-roles.log" 2>&1
-  export INTEGRATION_PROXY_DATABASE_URL="postgresql://litellm_writer:litellm-writer@127.0.0.1:5432/circle_test"
-  export INTEGRATION_PROXY_READ_REPLICA_URL="postgresql://litellm_reader:litellm-reader@127.0.0.1:5432/circle_test"
-fi
-if [ "$mode" = parity ]; then
-  export INTEGRATION_ROUTING=capture
-fi
-
-sudo mkdir -p "/sys/fs/cgroup/$egress_cgroup"
-echo "$$" | sudo tee "/sys/fs/cgroup/$egress_cgroup/cgroup.procs" > /dev/null
 sudo iptables -N integration_only
 guard_created=true
 sudo iptables -A integration_only -o lo -j ACCEPT
 sudo iptables -A integration_only -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-for service in "${INTEGRATION_POSTGRES_CONTAINER:-postgres-db}" "${INTEGRATION_REDIS_CONTAINER:-redis-cache}"; do
+for service in postgres-db redis-cache; do
   address="$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$service")"
   sudo iptables -A integration_only -d "$address" -j ACCEPT
 done
 sudo iptables -A integration_only -j REJECT
-sudo iptables -I OUTPUT 1 -m cgroup --path "$egress_cgroup" -j integration_only
+sudo iptables -I OUTPUT 1 -m owner --uid-owner "$(id -u)" -j integration_only
 guard_installed=true
 sudo ip6tables -N integration_only
 guard6_created=true
 sudo ip6tables -A integration_only -o lo -j ACCEPT
 sudo ip6tables -A integration_only -j REJECT
-sudo ip6tables -I OUTPUT 1 -m cgroup --path "$egress_cgroup" -j integration_only
+sudo ip6tables -I OUTPUT 1 -m owner --uid-owner "$(id -u)" -j integration_only
 guard6_installed=true
 
 if curl --noproxy '*' --connect-timeout 2 -s http://198.51.100.1 >/dev/null 2>&1; then
@@ -128,43 +106,12 @@ fi
 sudo iptables -L integration_only -n -v -x > "$results/egress-guard.txt"
 awk '$3 == "REJECT" && $1 > 0 { rejected=1 } END { exit !rejected }' "$results/egress-guard.txt"
 
-export UV_OFFLINE=true
-
-if [ "$mode" = parity ]; then
-  export INTEGRATION_WORKERS=1
-else
-  case "$suite" in
-    management|accounting) export INTEGRATION_WORKERS=2 ;;
-    providers|extensions) export INTEGRATION_WORKERS=4 ;;
-  esac
-fi
+setsid env -i PATH="$PATH" HOME="$HOME" PYTHONPATH="$PYTHONPATH" INTEGRATION_RUN_ID="$integration_identity" \
+  .venv/bin/python -m integration._support.upstream > "$results/upstream.log" 2>&1 &
+upstream_pid=$!
 if [ "$suite" = cost ]; then
   export INTEGRATION_WORKERS=8
 fi
-if [ "$suite" = security ]; then
-  export INTEGRATION_WORKERS=4
-fi
-if [ "$suite" = mcp ]; then
-  export INTEGRATION_WORKERS=2 INTEGRATION_COVERAGE=1
-fi
-if [ -n "$requested_workers" ]; then
-  export INTEGRATION_WORKERS="$requested_workers"
-fi
-isolated_workers=false
-if [ "${INTEGRATION_WORKERS:-1}" -gt 1 ]; then
-  isolated_workers=true
-fi
-if [ "$isolated_workers" = false ]; then
-  setsid env -i PATH="$PATH" HOME="$HOME" PYTHONPATH="$PYTHONPATH" INTEGRATION_RUN_ID="$integration_identity" \
-    .venv/bin/python -m integration._support.upstream > "$results/upstream.log" 2>&1 &
-  upstream_pid=$!
-fi
-coverage_data="$PWD/$results/coverage/data"
-proxy_command=(.venv/bin/python -m integration._support.proxy)
-if [ "${INTEGRATION_COVERAGE:-0}" = 1 ]; then
-  mkdir -p "$(dirname "$coverage_data")"
-  proxy_command=(.venv/bin/python -m coverage run --rcfile=tests/integration/mcp_coverage.toml -m integration._support.proxy)
-fi
 start_proxy() {
   local port="$1"
   local log_name="$2"
@@ -174,53 +121,41 @@ start_proxy() {
       "LITELLM_MODEL_COST_MAP_URL=$INTEGRATION_UPSTREAM_URL/_cost_map"
       "MODEL_COST_MAP_MIN_MODEL_COUNT=1"
       "MODEL_COST_MAP_MAX_SHRINK_RATIO=0"
-      "GEMINI_API_BASE=$INTEGRATION_UPSTREAM_URL"
-      "ANTHROPIC_API_BASE=$INTEGRATION_UPSTREAM_URL"
-      "GEMINI_API_KEY=sk-scripted-provider"
-      "ANTHROPIC_API_KEY=sk-scripted-provider"
     )
   else
     cost_map_env=("LITELLM_LOCAL_MODEL_COST_MAP=True")
   fi
-  local -a database_env=("DATABASE_URL=${INTEGRATION_PROXY_DATABASE_URL:-$DATABASE_URL}")
-  if [ -n "$INTEGRATION_PROXY_READ_REPLICA_URL" ]; then
-    database_env+=("DATABASE_URL_READ_REPLICA=$INTEGRATION_PROXY_READ_REPLICA_URL")
-  fi
   setsid env -i PATH="$PATH" HOME="$HOME" PYTHONPATH="$PYTHONPATH" INTEGRATION_RUN_ID="$integration_identity" \
-    "${database_env[@]}" REDIS_HOST="$REDIS_HOST" REDIS_PORT="$REDIS_PORT" \
-    INTEGRATION_UPSTREAM_URL="$INTEGRATION_UPSTREAM_URL" \
+    DATABASE_URL="$DATABASE_URL" REDIS_HOST="$REDIS_HOST" REDIS_PORT="$REDIS_PORT" \
     LITELLM_MASTER_KEY="$LITELLM_MASTER_KEY" LITELLM_SALT_KEY="$LITELLM_SALT_KEY" LITELLM_UI_PATH="$LITELLM_UI_PATH" PROXY_BASE_URL="http://127.0.0.1:$port" \
-    "${license_env[@]}" \
-    LITELLM_MODE=PRODUCTION STORE_MODEL_IN_DB=True LITELLM_ENABLE_MCP_STDIO=true "${cost_map_env[@]}" \
-    AWS_EC2_METADATA_DISABLED=true DO_NOT_TRACK=1 COVERAGE_FILE="$coverage_data" \
-    "${proxy_command[@]}" --config tests/integration/proxy_config.yaml \
+    LITELLM_MODE=PRODUCTION STORE_MODEL_IN_DB=True "${cost_map_env[@]}" \
+    AWS_EC2_METADATA_DISABLED=true DO_NOT_TRACK=1 \
+    .venv/bin/python -m integration._support.proxy --config tests/integration/proxy_config.yaml \
     --host 127.0.0.1 --port "$port" --num_workers 1 --telemetry False \
-    --use_prisma_db_push \
+    --use_prisma_db_push --enforce_prisma_migration_check \
     > "$results/$log_name" 2>&1 &
   launched_pid=$!
 }
-if [ "$isolated_workers" = false ]; then
-  start_proxy 4000 proxy.log
-  proxy_pid="$launched_pid"
+start_proxy 4000 proxy.log
+proxy_pid="$launched_pid"
+.venv/bin/python .circleci/scripts/wait_integration_services.py
+curl --noproxy '*' -sSf -X POST "$INTEGRATION_PROXY_URL/config/update" \
+  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
+  -d '{"router_settings": {"num_retries": 0}}' > "$results/seed-router-settings.json"
+if [ "$suite" = management ]; then
+  export INTEGRATION_PEER_URL=http://127.0.0.1:4001
+  start_proxy 4001 peer.log
+  peer_pid="$launched_pid"
   .venv/bin/python .circleci/scripts/wait_integration_services.py
-  curl --noproxy '*' -sSf -X POST "$INTEGRATION_PROXY_URL/config/update" \
-    -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
-    -d '{"router_settings": {"num_retries": 0}}' > "$results/seed-router-settings.json"
-  if [ "$suite" = management ] || [ "$suite" = mcp ] || [ "$suite" = accounting ]; then
-    export INTEGRATION_PEER_URL=http://127.0.0.1:4001
-    start_proxy 4001 peer.log
-    peer_pid="$launched_pid"
-    .venv/bin/python .circleci/scripts/wait_integration_services.py
-  fi
 fi
 
 if [ "$suite" = providers ]; then
-  INTEGRATION_RUN_ID="$integration_identity" .venv/bin/python -m pytest --tb=short --noconftest -o addopts= \
+  INTEGRATION_RUN_ID="$integration_identity" .venv/bin/python -m pytest --noconftest -o addopts= \
     --strict-markers --strict-config -p no:pytest-retry -p no:rerunfailures --timeout=30 \
-    tests/e2e_harness/test_provider_edge.py::TestReplayMode::test_content_drift_returns_the_miss_status_naming_both_keys \
-    tests/e2e_harness/test_provider_edge.py::TestReplayMode::test_exhausted_key_returns_the_miss_status \
-    tests/e2e_harness/test_provider_edge.py::TestReplayLeftover::test_partially_consumed_recording_names_the_leftover \
-    tests/e2e_harness/test_provider_edge.py::TestStreamingFidelity::test_replay_of_a_stream_makes_no_provider_connection \
+    tests/e2e/test_provider_edge.py::TestReplayMode::test_content_drift_returns_the_miss_status_naming_both_keys \
+    tests/e2e/test_provider_edge.py::TestReplayMode::test_exhausted_key_returns_the_miss_status \
+    tests/e2e/test_provider_edge.py::TestReplayLeftover::test_partially_consumed_recording_names_the_leftover \
+    tests/e2e/test_provider_edge.py::TestStreamingFidelity::test_replay_of_a_stream_makes_no_provider_connection \
     --junitxml="$results/replay-controls.xml"
 fi
 
@@ -237,59 +172,14 @@ if [ "$suite" = browser ]; then
   exit 0
 fi
 
-shard_arguments=()
-if [ "${CIRCLE_NODE_TOTAL:-1}" -gt 1 ]; then
-  .venv/bin/python tests/integration/run.py "$suite" --list \
-    | circleci tests split --split-by=timings --timings-type=filename > "$results/node-files.txt"
-  shard_arguments=(--shard-files "$results/node-files.txt")
-fi
-
-shared_services=()
-worker_coverage=()
-if [ "$isolated_workers" = false ]; then
-  shared_services=(
-    "REDIS_HOST=$REDIS_HOST" "REDIS_PORT=$REDIS_PORT"
-    "INTEGRATION_PROXY_URL=$INTEGRATION_PROXY_URL" "INTEGRATION_PEER_URL=$INTEGRATION_PEER_URL"
-    "INTEGRATION_UPSTREAM_URL=$INTEGRATION_UPSTREAM_URL" "INTEGRATION_MASTER_KEY=$INTEGRATION_MASTER_KEY"
-  )
-elif [ "${INTEGRATION_COVERAGE:-0}" = 1 ]; then
-  worker_coverage=("INTEGRATION_COVERAGE_FILE=$coverage_data")
-fi
-
-env -i PATH="$PATH" HOME="$HOME" PYTHONPATH="$PYTHONPATH" \
+timeout --signal=TERM --kill-after=20s "$shard_timeout" env -i PATH="$PATH" HOME="$HOME" PYTHONPATH="$PYTHONPATH" \
   INTEGRATION_RUN_ID="$integration_identity" \
-  DATABASE_URL="$DATABASE_URL" \
-  "${shared_services[@]}" "${worker_coverage[@]}" \
+  DATABASE_URL="$DATABASE_URL" REDIS_HOST="$REDIS_HOST" REDIS_PORT="$REDIS_PORT" \
+  INTEGRATION_PROXY_URL="$INTEGRATION_PROXY_URL" INTEGRATION_PEER_URL="$INTEGRATION_PEER_URL" \
+  INTEGRATION_UPSTREAM_URL="$INTEGRATION_UPSTREAM_URL" \
   INTEGRATION_WORKERS="${INTEGRATION_WORKERS:-1}" \
-  LITELLM_MODE=PRODUCTION \
-  "${license_env[@]}" \
+  INTEGRATION_MASTER_KEY="$INTEGRATION_MASTER_KEY" LITELLM_MODE=PRODUCTION \
   INTEGRATION_SEED="$INTEGRATION_SEED" \
   INTEGRATION_ORDER_SEED="$INTEGRATION_ORDER_SEED" \
-  INTEGRATION_CASE_SHARDS="${CIRCLE_NODE_TOTAL:-1}" INTEGRATION_CASE_SHARD_INDEX="${CIRCLE_NODE_INDEX:-0}" \
   LITELLM_LOCAL_MODEL_COST_MAP=True AWS_EC2_METADATA_DISABLED=true DO_NOT_TRACK=1 \
-  INTEGRATION_PROXY_DATABASE_URL="$INTEGRATION_PROXY_DATABASE_URL" \
-  INTEGRATION_PROXY_READ_REPLICA_URL="$INTEGRATION_PROXY_READ_REPLICA_URL" \
-  INTEGRATION_ROUTING="$INTEGRATION_ROUTING" \
-  MCP_TEST_PEER_PYTHON="${MCP_TEST_PEER_PYTHON:-}" \
-  .venv/bin/python tests/integration/run.py "$suite" --results "$results" \
-  "${shard_arguments[@]}"
-
-if [ "${INTEGRATION_COVERAGE:-0}" = 1 ]; then
-  for covered_pid in "$proxy_pid" "$peer_pid"; do
-    [ -n "$covered_pid" ] || continue
-    kill -TERM -- "-$covered_pid"
-    for _ in {1..300}; do
-      kill -0 "$covered_pid" 2>/dev/null || break
-      sleep 0.1
-    done
-    wait "$covered_pid" 2>/dev/null || true
-  done
-  proxy_pid=""
-  peer_pid=""
-  COVERAGE_FILE="$coverage_data" .venv/bin/python -m coverage combine --rcfile=tests/integration/mcp_coverage.toml
-  COVERAGE_FILE="$coverage_data" .venv/bin/python -m coverage report --rcfile=tests/integration/mcp_coverage.toml \
-    > "$results/coverage/coverage.txt"
-  COVERAGE_FILE="$coverage_data" .venv/bin/python -m coverage html --rcfile=tests/integration/mcp_coverage.toml \
-    -d "$results/coverage/html"
-  tail -n 1 "$results/coverage/coverage.txt"
-fi
+  .venv/bin/python tests/integration/run.py "$suite" --results "$results"

.circleci/scripts/run_migration_tests.py

@@ -1,116 +0,0 @@
-from __future__ import annotations
-
-import json
-import os
-import re
-import subprocess
-import sys
-from pathlib import Path
-from typing import Final
-from xml.etree import ElementTree
-
-SUITES: Final = {
-    "startup": (("test_startup.py",), 13),
-    "recovery": (("test_recovery.py",), 15),
-    "legacy": (("test_legacy.py", "test_pooling.py"), 11),
-    "upgrade": (("test_upgrade.py", "test_rolling_upgrade.py"), 5),
-    "shaped": (("test_shaped_database.py",), 1),
-}
-
-
-def successful_junit(path: Path, expected: int, exit_code: int) -> bool:
-    if exit_code != 0 or not path.is_file():
-        return False
-    try:
-        root: Final = ElementTree.parse(path).getroot()
-    except ElementTree.ParseError:
-        return False
-    cases: Final = tuple(root.iter("testcase"))
-    identities: Final = frozenset((case.get("classname"), case.get("name")) for case in cases)
-    return len(cases) == len(identities) == expected and all(
-        not any(case.find(tag) is not None for tag in ("failure", "error", "skipped")) for case in cases
-    )
-
-
-def output(*command: str) -> str:
-    return subprocess.check_output(command, text=True, timeout=90).strip()
-
-
-def record_image() -> None:
-    source: Final = output("git", "rev-parse", "HEAD")
-    image: Final = output("docker", "image", "inspect", "litellm-docker-database:ci", "--format", "{{.Id}}")
-    revision: Final = output(
-        "docker",
-        "image",
-        "inspect",
-        "litellm-docker-database:ci",
-        "--format",
-        '{{index .Config.Labels "org.opencontainers.image.revision"}}',
-    )
-    assert re.fullmatch(r"[0-9a-f]{40}", source), "Invalid source revision"
-    assert revision == source, "Candidate image revision differs from the tested source"
-    Path("migration-image.json").write_text(
-        json.dumps(
-            {
-                "source_sha": source,
-                "image_id": image,
-                "candidate_image": os.environ.get("MIGRATION_CANDIDATE_IMAGE", ""),
-            }
-        )
-    )
-
-
-def main() -> int:
-    suite: Final = os.environ["MIGRATION_TEST_SUITE"]
-    files, expected = SUITES[suite]
-    metadata: Final = json.loads(Path("migration-image.json").read_text())
-    assert metadata["source_sha"] == output("git", "rev-parse", "HEAD"), "Image and test source revisions differ"
-    assert metadata["image_id"] == output(
-        "docker", "image", "inspect", os.environ["LITELLM_MIGRATION_TEST_IMAGE"], "--format", "{{.Id}}"
-    ), "Loaded image differs from the build output"
-    assert metadata["candidate_image"] == os.environ.get("MIGRATION_CANDIDATE_IMAGE", ""), "Wrong release candidate"
-    destination: Final = Path(os.environ["MIGRATION_TEST_OUTPUT"])
-    junit: Final = destination / "junit" / "results.xml"
-    junit.parent.mkdir(parents=True, exist_ok=True)
-    result: Final = subprocess.run(
-        (
-            sys.executable,
-            "-m",
-            "pytest",
-            *(f"tests/e2e/migrations/{name}" for name in files),
-            "-vv",
-            "--tb=short",
-            "--durations=10",
-            f"--junitxml={junit}",
-            "-o",
-            "addopts=",
-            "--reruns=0",
-        ),
-        check=False,
-        timeout=1200,
-    )
-    passed: Final = successful_junit(junit, expected, result.returncode)
-    (destination / "verdict.json").write_text(
-        json.dumps(
-            {
-                **metadata,
-                "suite": suite,
-                "baseline_image": os.environ.get("LITELLM_MIGRATION_BASELINE_IMAGE", ""),
-                "expected_cases": expected,
-                "passed": passed,
-                "pytest_exit_code": result.returncode,
-                "test_revision": metadata["source_sha"],
-                "workflow_id": os.environ.get("CIRCLE_WORKFLOW_ID", ""),
-                "job_number": os.environ.get("CIRCLE_BUILD_NUM", ""),
-            },
-            indent=2,
-        )
-    )
-    return 0 if passed else 1
-
-
-if __name__ == "__main__":
-    if sys.argv[1:] == ["record-image"]:
-        record_image()
-    else:
-        raise SystemExit(main())

.circleci/scripts/verify_integration_browser.py

@@ -31,8 +31,8 @@ def main() -> None:
     result: Final = json.loads(Path(sys.argv[1]).read_text())
     assert not result.get("errors"), result.get("errors")
     expected: Final = json.loads(
-        (Path(__file__).resolve().parents[2] / "tests/e2e/ui/tests/integrationCritical/expected.json").read_text()
-    )
+        (Path(__file__).resolve().parents[2] / "tests/integration/contracts.json").read_text()
+    )["browser"]
     assert expected and result["stats"]["expected"] == len(expected)
     assert all(result["stats"][name] == 0 for name in ("unexpected", "flaky", "skipped"))
 

.circleci/scripts/wait_integration_services.py

@@ -1,31 +1,42 @@
 import os
-from contextlib import ExitStack
+import time
 from typing import Final
 
 import httpx
-from integration._support.client import Gateway
-from integration._support.workers import wait_for_services
 from redis import Redis
 
 
 def main() -> None:
     primary: Final = os.environ["INTEGRATION_PROXY_URL"]
     peer: Final = os.environ.get("INTEGRATION_PEER_URL")
-    urls: Final = (primary, peer) if peer else (primary,)
-    upstream: Final = os.environ["INTEGRATION_UPSTREAM_URL"]
-    with ExitStack() as stack:
-        proxies: Final = tuple(
-            Gateway(
-                stack.enter_context(httpx.Client(base_url=url, trust_env=False, timeout=2)),
-                os.environ["INTEGRATION_MASTER_KEY"],
-                upstream,
-            )
-            for url in urls
-        )
-        cache: Final = stack.enter_context(
-            Redis(host=os.environ["REDIS_HOST"], port=int(os.environ["REDIS_PORT"]), socket_timeout=2)
-        )
-        wait_for_services(upstream, proxies, cache)
+    proxies: Final = (primary, peer) if peer else (primary,)
+    deadline: Final = time.monotonic() + 90
+    headers: Final = {"Authorization": f"Bearer {os.environ['INTEGRATION_MASTER_KEY']}"}
+    with httpx.Client(trust_env=False, timeout=2) as client, Redis(
+        host=os.environ["REDIS_HOST"], port=int(os.environ["REDIS_PORT"]), socket_timeout=2
+    ) as cache:
+        while True:
+            try:
+                ready: Final = (
+                    client.get(f"{os.environ['INTEGRATION_UPSTREAM_URL']}/health").status_code == 200
+                    and all(client.get(f"{url}/health/readiness").status_code == 200 for url in proxies)
+                )
+                if ready:
+                    for url in proxies:
+                        response: Final = client.get(f"{url}/cache/ping", headers=headers)
+                        response.raise_for_status()
+                        result: Final = response.json()
+                        assert result["status"] == "healthy", result
+                        assert result["cache_type"] == "redis", result
+                        assert result["ping_response"] is True, result
+                        assert result["set_cache_response"] == "success", result
+                    if cache.pubsub_numsub("litellm_proxy.auth_cache_invalidation")[0][1] >= len(proxies):
+                        return
+            except httpx.TransportError:
+                pass
+            if time.monotonic() >= deadline:
+                raise SystemExit("Integration services or auth-cache subscribers did not become ready")
+            time.sleep(0.2)
 
 
 if __name__ == "__main__":

.env.example

@@ -26,7 +26,6 @@ NOVITA_API_KEY = ""
 INFINITY_API_KEY = ""
 
 # Development Configs
-# Generate one with: echo "LITELLM_MASTER_KEY=sk-$(openssl rand -hex 32)"
-LITELLM_MASTER_KEY = ""
+LITELLM_MASTER_KEY = "sk-1234"
 DATABASE_URL = "postgresql://llmproxy:dbpassword9090@db:5432/litellm"
 STORE_MODEL_IN_DB = "True"

.gitguardian.yaml

@@ -84,6 +84,10 @@ secret:
     - name: Langfuse test credentials in test_completion
       match: c39310f68cc3d3e22f7b298bb6353c4f45759adcc37080d8b7f4e535d3cfd7f4
     
+    # Test password "sk-1234" in e2e test fixtures - test fixture, not a real secret
+    - name: Test password in e2e test fixtures
+      match: ce32b547202e209ec1dd50107b64be4cfcf2eb15c3b4f8e9dc611ef747af634f
+    
     # === Preventive patterns for test keys (pattern-based) ===
     
     # Test API keys (124 instances across 45 files)
@@ -104,3 +108,4 @@ secret:
 
     - name: Short fake sk keys (1–9 digits only)
       match: \bsk-\d{1,9}\b
+

.githooks/commit-msg

@@ -42,7 +42,7 @@ case "$subject" in
         ;;
 esac
 
-ALLOWED_TYPES="feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert|security"
+ALLOWED_TYPES="feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert"
 # Description must not start with an uppercase letter — kept in sync with the
 # subjectPattern in .github/workflows/conventional-commits.yml so the local
 # hook is the strictly tighter of the two gates. (Without this guard, a commit
@@ -61,7 +61,7 @@ cat >&2 <<EOF
   Expected: <type>(<scope>)!: <description>
             (description must start with a lowercase letter)
 
-  Allowed types: feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert, security
+  Allowed types: feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert
   Examples:
     feat(router): add weighted round-robin strategy
     fix(bedrock): decouple STS region from aws_region_name

.githooks/pre-push

@@ -0,0 +1,92 @@
+#!/usr/bin/env bash
+#
+# pre-push — enforce Conventional Branches
+#   https://conventional-branch.github.io/
+#
+# Branch format:  <type>/<description>
+#   <type> must be one of: feature, bugfix, hotfix, release, chore
+#
+# Protected branches (always allowed):
+#   - main
+#   - litellm_internal_staging
+#   - dependabot/*
+#   - gh-readonly-queue/*
+#
+# Tag pushes and branch deletions are skipped.
+# Bypass: git push --no-verify.
+
+set -eu
+
+ZERO_OID="0000000000000000000000000000000000000000"
+ZERO_OID_SHA256="0000000000000000000000000000000000000000000000000000000000000000"
+ALLOWED_TYPES="feature|bugfix|hotfix|release|chore"
+BRANCH_PATTERN="^(${ALLOWED_TYPES})/.+"
+
+PROTECTED_NAMES="main litellm_internal_staging"
+PROTECTED_PREFIXES="dependabot/ gh-readonly-queue/"
+
+is_protected() {
+    branch="$1"
+    for name in $PROTECTED_NAMES; do
+        if [ "$branch" = "$name" ]; then
+            return 0
+        fi
+    done
+    for prefix in $PROTECTED_PREFIXES; do
+        case "$branch" in "$prefix"*) return 0 ;; esac
+    done
+    return 1
+}
+
+invalid=""
+
+while read -r local_ref local_oid remote_ref remote_oid; do
+    # Branch deletion (no local commit being pushed).
+    if [ "$local_oid" = "$ZERO_OID" ] || [ "$local_oid" = "$ZERO_OID_SHA256" ]; then
+        continue
+    fi
+
+    # Only validate branch pushes; ignore tags and other ref namespaces.
+    case "$remote_ref" in
+        refs/heads/*) ;;
+        *) continue ;;
+    esac
+
+    branch="${remote_ref#refs/heads/}"
+
+    if is_protected "$branch"; then
+        continue
+    fi
+
+    if ! printf '%s' "$branch" | grep -Eq "$BRANCH_PATTERN"; then
+        invalid="$invalid $branch"
+    fi
+done
+
+if [ -n "$invalid" ]; then
+    cat >&2 <<EOF
+✗ Branch name does not follow Conventional Branches.
+
+  Invalid:$invalid
+
+  Expected: <type>/<description>
+
+  Allowed types: feature, bugfix, hotfix, release, chore
+  Examples:
+    feature/weighted-round-robin
+    bugfix/streaming-empty-chunks
+    chore/bump-deps
+    hotfix/auth-bypass
+
+  Protected (always allowed): main, litellm_internal_staging,
+  dependabot/*, gh-readonly-queue/*.
+
+See https://conventional-branch.github.io/
+
+Rename with: git branch -m <new-name>
+To bypass (use sparingly): git push --no-verify
+EOF
+    exit 1
+fi
+
+exit 0

.github/CODEOWNERS

@@ -1,6 +1,10 @@
-/model_prices_and_context_window.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
-/litellm/model_prices_and_context_window_backup.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
-/ci_cd/cost_map_pins.json @mateo-berri @ryan-crabbe-berri @kerry-berri @joshua-berri
-/helm/ @yassin-berriai
-/terraform/ @yassin-berriai
-/.github/workflows/zizmor.yml @yuneng-berri
+/ui/ @yuneng-berri @ryan-crabbe-berri
+/litellm/proxy/_experimental/out/ @yuneng-berri @ryan-crabbe-berri
+/ui/Dockerfile
+/ui/nginx.conf
+/ui/litellm-dashboard/src/lib/http/schema.d.ts
+/ui/litellm-dashboard/tsconfig.tsbuildinfo
+/model_prices_and_context_window.json @mateo-berri @ryan-crabbe-berri @kerry-berri
+/litellm/model_prices_and_context_window_backup.json @mateo-berri @ryan-crabbe-berri @kerry-berri
+/litellm-proxy-extras/litellm_proxy_extras/migrations/ @yuneng-berri @ryan-crabbe-berri
+/.github/CODEOWNERS @yuneng-berri

.github/PULL_REQUEST_TEMPLATE/general.md

@@ -1,188 +0,0 @@
-<!-- The whole description's target audience is humans, not AI agents: write it in plain, simple,
-     everyday engineering language, extremely parsable and readable at a glance. This goes double for
-     the TLDR, User Flow, and Caveats sections
-     Drop every section you have nothing to put in, heading included: a bare "## Relevant issues" or
-     "## Affected release" with nothing under it must not appear in the final description -->
-
-## Handwritten 1-3 sentence description
-
-<!-- The author writes a plain summary in their own words before requesting review.
-     Wrap newly AI-written summary text in the make-me-human-readable tags below, including backticks.
-     The author reviews or rewrites the draft and removes the tags.
-     When updating an existing PR, preserve the author's wording and removed tags.
-     Changes elsewhere in the body do not make an unchanged summary an AI draft again.
-     Add tags again only around newly AI-written replacement summary text, never merely to match this template.
-     This preserves author review across later updates (2026-11-02, #49779).
-     Say what the PR does for users and why, then anything that keeps it working over time (a sync job, a config someone owns)
-     No bullets, tables, or code names here. Everything else goes inside the Details block below -->
-
-`<make-me-human-readable>`
-<blah>
-`</make-me-human-readable>`
-
-<details>
-<summary>Details</summary>
-
-## TLDR
-
-<!-- Fill in the bullets below and keep each one short and concrete: one line per bullet, roughly 10 words max
-     If the PR intentionally changes what existing users see or how a screen behaves, add a line under the bullets that starts "Intentional product change:" describing what changes, why, and what users lose. Reviewers must never have to infer a deliberate UX change from the diff -->
-
-Problem this solves:
-
-- <blah>
-- ...
-
-How it solves it:
-
-- <blah>
-- ...
-
-## User Flow
-
-<!-- Two ordered lists, Before and After, walking the same end user through the same task, written strictly from that user's seat
-     Read the linked issue, ticket, or customer thread first so the flow reflects the real application and the routes its users actually hit; don't invent a generic scenario
-     Lead each list with one plain sentence saying where the flow fails (Before) or succeeds (After), then number the steps
-     Keep it tight: aim for 3 to 5 steps per list, one line each, roughly 20 words max, and never pad a shorter flow with filler steps to hit the count. Cover the one path the PR changes and fold variants (case, other field, second endpoint) into a clause on the step they belong to rather than their own steps. The example below is the target length
-     Every step is something the user does or observes: the HTTP method and full URL they hit, what they sent, and what visibly came back (status code, error text, the shape of an ID). UI steps name the page URL and what is on screen
-     No LiteLLM internals: never name functions, files, DB tables, config classes, hooks, callbacks, or code paths. "The upload hands back an ID that looks like OpenAI's own `file-abc123` instead of the scrambled one the gateway returned" is right, "no managed-file row was registered" is wrong
-     Keep the two lists step-for-step identical until they diverge, so the changed step is obvious
-     If the bug had a security or authorization consequence, end each list with what another user could or could no longer do
-     Regenerate this section, screenshots included, whenever new commits change the PR's behavior, so it never describes an older revision
-     If the PR changes what an Admin UI page shows, embed a before and an after screenshot of that page right after its list, taken at the same URL on the same data, with the rows, fields, or controls that changed boxed in red so a reader spots the difference without reading the steps. These are the UI screenshots for Screenshots / Proof of Fix too: embed them once here and have that section's Before and After steps point back to them instead of repeating the images
-
-Example:
-
-Before: a developer whose app streams chat completions gets no token counts back, so their cost dashboard reads zero
-
-1. They send POST https://litellm-domain/v1/chat/completions with `"stream": true` and no `stream_options`
-2. The last SSE chunk arrives with `"usage": null`, so their app records 0 prompt and 0 completion tokens
-3. They open https://litellm-domain/ui/?page=logs and see the request logged at $0 spend
-
-After: the same request comes back with real token counts, so the dashboard shows real spend
-
-1. The proxy admin sets `always_include_stream_usage: true` and restarts the proxy
-2. The developer sends the same POST https://litellm-domain/v1/chat/completions with `"stream": true` and no `stream_options`
-3. The last SSE chunk now carries a `usage` object with real prompt and completion token counts
-4. https://litellm-domain/ui/?page=logs shows that request at non-zero spend
--->
-
-## Relevant issues
-
-<!-- e.g., "Fixes #000". Drop the section if there is none -->
-
-## Affected release
-
-<!-- Only for a fix to a regression in a released or rc version (perf, memory, crash, or behavior): name the version it regressed in, e.g. "regression in v1.100.0" or "since v1.101.0-rc.1". Add the `backport-stable` label only when the regression is a P0, meaning its Linear ticket is Urgent (a security hole however narrow, data loss, or a crash or outage for every user on that version), because every labeled PR must be cherry-picked onto the baking rc line before the stable can be tagged; every other regression fix ships in the next rc unlabeled. Drop the section otherwise -->
-
-## Linear ticket
-
-<!-- if you are an internal contributor, add "Resolves " followed by the Linear ticket e.g., "Resolves LIT-1234" to link the Linear ticket to the GitHub PR. If you don't have one, drop the section rather than guessing -->
-
-## Pre-Submission checklist
-
-**Please complete all items before asking a LiteLLM maintainer to review your PR**
-
-- [ ] I have added meaningful tests
-- [ ] The handful of test files covering my change pass locally, e.g. `uv run pytest tests/unit/<your_test_file>.py -v`. Leave the suites (`make test-unit-*`, `make test-unit`) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
-- [ ] My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
-- [ ] My PR's scope is as isolated as possible; it only solves 1 specific problem
-- [ ] Every automated review on my PR's latest commit has finished, and I have addressed or replied to each of its findings before requesting a maintainer review
-
-## Delays in PR merge?
-
-If you're seeing a delay in your PR being merged, ping the LiteLLM Team on [Slack (#pr-review)](https://join.slack.com/t/litellmossslack/shared_invite/zt-3o7nkuyfr-p_kbNJj8taRfXGgQI1~YyA).
-
-## Screenshots / Proof of Fix
-
-<!-- Include screenshots, screen recordings, or command (e.g., curl) + output demonstrating that your changes work as expected
-     Simple PR exception (2026-10-28, #49030). A mechanical change fully proven by focused checks may skip before/after e2e. Follow AGENTS.md's eligibility rule, trace callers, and verify bug-fix regression sensitivity. Record E2E exception at <tip SHA>, scope, rationale, commands and results here. Tests/static checks suffice only for this exception. Keep CI and bot gates, reassess changed diffs, and honor explicit e2e requests. Omit the Before/After scaffold for an exception
-     Otherwise proof must be fully e2e, no mocks, with real provider calls where applicable. Pytest alone is insufficient. The following format applies to live proof
-     Show only the latest proof. Before uses the merge base and After the tested tip. Rerun affected QA after behavior changes. Verified neutral diffs may carry proof with the tested SHA, current tip, equivalence reason and focused checks, following AGENTS.md and /qa
-     Use matching Before/After case headings with numbered commands and observed output. Put shared setup above them. For one case, omit case headings
-
-### Before (<hash>)
-
-#### <case 1>
-
-1. ...
-2. ...
-
-#### <case 2>
-
-1. ...
-
-### After (<hash>)
-
-#### <case 1>
-
-1. ...
-2. ...
-
-#### <case 2>
-
-1. ...
-
-     For bug fixes: Before shows the reproduction, After shows the same steps passing
-     For new features: Before shows the capability missing, After shows it working end-to-end
-     If the change applies to all three LLM endpoints (/v1/responses, /v1/chat/completions, /v1/messages), make each endpoint its own case, not just one
-     For UI changes: before/after screenshots under the same headings
-     If the main use case runs through a coding tool like Claude Code or Codex, drive that tool interactively the way the user does (never `claude -p`, `codex exec`, or curl on its own) and embed before/after screenshots of its pane under the same headings; curl replays and headless runs can follow as extra cases, never as the only proof -->
-
-## Type
-
-<!-- Select the type of Pull Request -->
-<!-- Keep only the necessary ones -->
-
-🆕 New Feature
-🐛 Bug Fix
-🧹 Refactoring
-📖 Documentation
-🚄 Infrastructure
-✅ Test
-
-## Caveats (if any)
-
-<!-- Group caveats under severity subheadings (### Severe, ### High, ### Medium, ### Low), with
-     short bullet points inside each, just like the TLDR: one line per bullet, roughly 10 words max
-     Call out known limitations, follow-up work, or anything a reviewer should watch out for
-     Include only the tiers that have caveats; drop the empty ones
-     - Severe: inherent to what the PR deliberately ships, there even when the code works as intended:
-       it can degrade or take down a running deployment (e.g. a slow or table-locking boot migration),
-       rewrite data by design, break an existing workflow on purpose, or change auth behavior. An
-       operator must plan around it before rollout
-     - High: an unintended hole: a correctness, security, data-loss, or backward-compatibility bug,
-       unsafe to ship as is
-     - Medium: a real gap someone can hit, but with a workaround or a narrow blast radius
-     - Low: anything else worth noting: naming, cleanup, an edge case nobody hits
-     Nest bullets as deep as helps: hierarchy beats one long line when it makes things clearer to a
-     human reader
-     If you assumed something instead of testing it, e.g. "only reproduces with X on" or "no
-     user-observable behavior difference", list it here too with what breaks if it is wrong
-     Drop this section if there are none -->
-
-## QA runbook
-
-<!-- Only needed when your PR edits tests/e2e; delete this section otherwise
-
-For each e2e test you added or changed, list the manual steps a reviewer can follow to reproduce it by hand against a live proxy, mapping 1:1 to what the test asserts: one top-level bullet per test giving its pytest node id followed by what it proves in plain words, then a nested "- [ ]" checklist where each item is a concrete action (route, request body, expected response) and the final item is the sanity-check step shown in the examples. Note environment prerequisites (provider credentials, config flags) and any nuances a manual run will hit. See PRs #32914 and #32963 for full examples
-
-Example checklists:
-
-- tests/e2e/quota_management/ratelimit/test_rate_limit_e2e.py::TestKeyRateLimits::test_rpm_limit_blocks_over_limit - a key allowed 2 requests a minute serves exactly 2 and refuses the 3rd
-  - [ ] Generate a limited key: curl -X POST http://localhost:4000/key/generate -H "Authorization: Bearer $LITELLM_MASTER_KEY" -d '{"rpm_limit": 2}'
-  - [ ] Send three /v1/chat/completions requests with that key inside one minute
-  - [ ] Expect the first two to return 200 and the third to return 429 naming the rpm limit
-  - [ ] Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
-
-- tests/e2e/management/test_management_e2e.py::TestModelRoutes::test_model_create_appears_in_ui - a deployment created through the API shows up on the Admin UI models page
-  - [ ] POST /model/new with the master key, a bedrock model, and aws_region_name (needs STORE_MODEL_IN_DB=True and AWS credentials)
-  - [ ] Open http://localhost:4000/ui/?page=models and expect a deployment row showing the returned model id
-  - [ ] Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
--->
-
-## Final Attestation
-
-- [ ] The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
-
-</details>

.github/PULL_REQUEST_TEMPLATE/rust.md

@@ -1 +0,0 @@
-<!-- Write the final PR description as concise bullets only -->

.github/actions/cache-cargo-build/action.yml

@@ -15,35 +15,16 @@ description: >-
   cache the same directory for different workloads, and a shared key would let
   whichever ran first deny the others a save.
 
-inputs:
-  profile:
-    description: "Cargo profile the build uses (dev or release)"
-    required: false
-    default: "dev"
-
 runs:
   using: composite
   steps:
     - name: Restore the Cargo registry and target directory
-      if: github.ref == 'refs/heads/main'
       uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
       with:
         path: |
           ~/.cargo/registry
           ~/.cargo/git
           litellm-rust/target
-        key: ${{ runner.os }}-maturin-${{ inputs.profile }}-${{ hashFiles('litellm-rust/Cargo.lock') }}
+        key: ${{ runner.os }}-maturin-dev-${{ hashFiles('litellm-rust/Cargo.lock') }}
         restore-keys: |
-          ${{ runner.os }}-maturin-${{ inputs.profile }}-
-
-    - name: Restore the Cargo registry and target directory
-      if: github.ref != 'refs/heads/main'
-      uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
-      with:
-        path: |
-          ~/.cargo/registry
-          ~/.cargo/git
-          litellm-rust/target
-        key: ${{ runner.os }}-maturin-${{ inputs.profile }}-${{ hashFiles('litellm-rust/Cargo.lock') }}
-        restore-keys: |
-          ${{ runner.os }}-maturin-${{ inputs.profile }}-
+          ${{ runner.os }}-maturin-dev-

.github/actions/cache-prisma-binaries/action.yml

@@ -30,7 +30,6 @@ runs:
         echo "version=${version}" >> "$GITHUB_OUTPUT"
 
     - name: Restore Prisma binaries
-      if: github.ref == 'refs/heads/main'
       uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
       with:
         # ~/.cache/prisma-python holds the npm install tree prisma-client-py
@@ -39,12 +38,3 @@ runs:
           ~/.cache/prisma-python
           ~/.cache/prisma
         key: ${{ runner.os }}-prisma-binaries-${{ steps.version.outputs.version }}
-
-    - name: Restore Prisma binaries
-      if: github.ref != 'refs/heads/main'
-      uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
-      with:
-        path: |
-          ~/.cache/prisma-python
-          ~/.cache/prisma
-        key: ${{ runner.os }}-prisma-binaries-${{ steps.version.outputs.version }}

.github/actions/cache-uv-downloads/action.yml

@@ -1,25 +0,0 @@
-name: "Cache uv downloads"
-description: >-
-  Restore the uv download cache on every run and save it only from main, so pull
-  requests reuse main's cache instead of evicting it with their own copies.
-
-runs:
-  using: composite
-  steps:
-    - name: Restore and save the uv download cache
-      if: github.ref == 'refs/heads/main'
-      uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
-      with:
-        path: ${{ env.UV_CACHE_DIR }}
-        key: ${{ runner.os }}-uv-downloads-py${{ env.UV_PYTHON }}-${{ hashFiles('uv.lock') }}
-        restore-keys: |
-          ${{ runner.os }}-uv-downloads-py${{ env.UV_PYTHON }}-
-
-    - name: Restore the uv download cache
-      if: github.ref != 'refs/heads/main'
-      uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
-      with:
-        path: ${{ env.UV_CACHE_DIR }}
-        key: ${{ runner.os }}-uv-downloads-py${{ env.UV_PYTHON }}-${{ hashFiles('uv.lock') }}
-        restore-keys: |
-          ${{ runner.os }}-uv-downloads-py${{ env.UV_PYTHON }}-

.github/actions/rust-bridge/action.yml

@@ -1,19 +0,0 @@
-name: Set up the Rust bridge
-description: Select the shared Rust bridge artifact or the Cargo cache
-inputs:
-  artifact:
-    description: Rust bridge artifact name
-    required: false
-    default: ""
-runs:
-  using: composite
-  steps:
-    - name: Restore the Cargo build cache
-      if: inputs.artifact == ''
-      uses: ./.github/actions/cache-cargo-build
-    - name: Download the Rust bridge artifact
-      if: inputs.artifact != ''
-      uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e
-      with:
-        name: ${{ inputs.artifact }}
-        path: rust-bridge-dist

.github/actions/setup-uv-with-retries/action.yml

@@ -17,7 +17,6 @@ runs:
       uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
       with:
         version: ${{ inputs.version }}
-        save-cache: ${{ github.ref == 'refs/heads/main' }}
 
     - name: Wait before attempt 2
       if: steps.attempt-1.outcome == 'failure'
@@ -31,7 +30,6 @@ runs:
       uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
       with:
         version: ${{ inputs.version }}
-        save-cache: ${{ github.ref == 'refs/heads/main' }}
 
     - name: Wait before attempt 3
       if: steps.attempt-2.outcome == 'failure'
@@ -43,4 +41,3 @@ runs:
       uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
       with:
         version: ${{ inputs.version }}
-        save-cache: ${{ github.ref == 'refs/heads/main' }}

.github/ci-coverage-allowlist.yml

@@ -4,25 +4,27 @@ description: >-
   by a job nor listed here, so every entry below is a decision on the record.
 
 test_paths:
-  - reason: >-
-      litellm.agent() end-to-end suite. It drives the real claude, codex, opencode and pi CLIs and
-      deepagents against a live LiteLLM AI Gateway, so it needs those binaries on PATH plus
-      LITELLM_PROXY_API_BASE / LITELLM_PROXY_API_KEY, and skips without them. Run manually
-      before changing litellm/harness; the mocked coverage runs in tests/unit/harness and
-      tests/unit/llms/*/harness
-    paths:
-      - tests/harness_e2e
   - reason: >-
       The Rust/Python parity harness is run manually through its local CLI. Recorded replay,
       fixture generation, and harness checks are intentionally outside pull request CI
     paths:
       - tests/rust-python-harness
   - reason: >-
-      Owned by the open pull request that moves the Rust/Python parity checks (#45135). It ran in
-      the CircleCI build_and_test job that globbed tests/test_*.py, which this migration deletes,
-      and that pull request gives it a new home
+      What is left of the caching suite in tests/local_testing that runs nowhere. Every job that
+      globs that directory either deselects it (local_testing_part1 and part2 carry `-k "... and
+      not caching and not cache"`) or keeps only another keyword (langfuse, router, assistants),
+      and no job names these files the way redis_caching_unit_tests names test_dual_cache.py.
+      The gap was eight files and 118 tests when measured 2026-08-20; the five keyless ones now
+      run in the caching-local shard, leaving these three. Measured 2026-08-21 with no provider
+      credentials and no Redis: test_caching.py needs both (37 of 65 fail without them),
+      test_disk_cache_unit_tests.py needs OPENAI_API_KEY for 2 of its 4, and
+      test_gcs_cache_unit_tests.py needs GCS credentials for all 4. They want the keyless/live
+      split that porting tests/local_testing off CircleCI will force, not a job that is red by
+      construction
     paths:
-      - tests/test_rust_python_harness.py
+      - tests/local_testing/test_caching.py
+      - tests/local_testing/test_disk_cache_unit_tests.py
+      - tests/local_testing/test_gcs_cache_unit_tests.py
   - reason: >-
       The end-to-end suite runs against a deployed proxy from its own in-cluster rig rather than
       from a pull request; it needs a live gateway and provider credentials no PR job holds
@@ -48,6 +50,48 @@ test_paths:
       choice it informed is settled
     paths:
       - tests/code_coverage_tests/test_aio_http_image_conversion.py
+  - reason: >-
+      No job invokes this suite and its files mix pure transformation tests with ones driving live
+      vendor vector stores, so assigning them needs a per-file decision
+    paths:
+      - tests/vector_store_tests/rag/test_rag_bedrock.py
+      - tests/vector_store_tests/rag/test_rag_openai.py
+      - tests/vector_store_tests/rag/test_rag_s3_vectors.py
+      - tests/vector_store_tests/rag/test_rag_vertex_ai.py
+      - tests/vector_store_tests/test_azure_ai_vector_store.py
+      - tests/vector_store_tests/test_azure_vector_store.py
+      - tests/vector_store_tests/test_bedrock_vector_store.py
+      - tests/vector_store_tests/test_gemini_vector_store.py
+      - tests/vector_store_tests/test_milvus_vector_store.py
+      - tests/vector_store_tests/test_openai_vector_store.py
+      - tests/vector_store_tests/test_ragflow_vector_store.py
+      - tests/vector_store_tests/test_s3_vectors_vector_store.py
+      - tests/vector_store_tests/test_vertex_ai_search_api_vector_store.py
+      - tests/vector_store_tests/test_vertex_ai_vector_store.py
+  - reason: >-
+      Throughput and memory-growth measurements whose runtime and variance make them unsuitable for
+      a per-pull-request job
+    paths:
+      - tests/load_tests/test_datadog_load_test.py
+      - tests/load_tests/test_langsmith_load_test.py
+      - tests/load_tests/test_linear_memory_growth.py
+      - tests/load_tests/test_memory_usage.py
+      - tests/load_tests/test_otel_load_test.py
+      - tests/load_tests/test_vertex_embeddings_load_test.py
+      - tests/load_tests/test_vertex_load_tests.py
+  - reason: >-
+      Env-gated saturation benchmark requires a live proxy and provider credentials, so it is run
+      locally rather than in pull-request jobs
+    paths:
+      - tests/load_tests/test_granian_admission_saturation.py
+  - reason: >-
+      A local-only agent rig: test_a2a_completion_bridge.py needs a LangGraph server on
+      localhost:2024 and test_a2a.py drives a live A2A endpoint, so neither can run in a
+      pull request job. Until 2026-08-20 the CircleCI agent job hid them behind a grep -v
+      that this census could not see; the glob now excludes them structurally and this entry
+      is the decision on the record. Revisit when the A2A bridge gets a recorded-wire fixture
+    paths:
+      - tests/agent_tests/local_only_agent_tests
   - reason: >-
       Third-party integration tests that skip themselves without OCI configuration or sandbox
       credentials, neither of which a pull request job holds
@@ -66,10 +110,3 @@ dockerfiles:
       An example image under cookbook/ that is documentation rather than a shipped artifact
     paths:
       - cookbook/litellm-ollama-docker-image/Dockerfile
-  - reason: >-
-      The Rust gateway image compiles the whole workspace in release mode, which is too slow for
-      a per-pull-request job while the gateway binary is still being assembled; the Rust lint,
-      clippy, and compile jobs already cover the code it packages. Revisit when the gateway is
-      published
-    paths:
-      - litellm-rust/crates/gateway/Dockerfile

.github/codeql/codeql-config.yml

@@ -1,5 +1,8 @@
 name: "LiteLLM CodeQL config"
 
+queries:
+  - uses: security-and-quality
+
 # Known OOM queries on large Python codebases:
 # CodeQL builds a full data flow graph in memory. These two queries trace
 # sensitive data through every log call / regex pattern, causing combinatorial
@@ -11,6 +14,17 @@ query-filters:
       id: py/clear-text-logging-sensitive-data  # CWE-312
   - exclude:
       id: py/polynomial-redos                   # CWE-730
+  # Import resolution confuses stdlib types with management_endpoints/types.py.
+  # The generic cycle query also reports intentional deferred imports.
+  - exclude:
+      id: py/cyclic-import
+  - exclude:
+      id: py/unsafe-cyclic-import
+  # Known false positives on live settings and Protocol placeholders.
+  - exclude:
+      id: py/unused-global-variable
+  - exclude:
+      id: py/ineffectual-statement
 
 paths-ignore:
   - tests

.github/e2e-stack/assert_tests_ran.py

@@ -0,0 +1,53 @@
+import sys
+import xml.etree.ElementTree as ET
+from pathlib import Path
+from typing import Final
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "tests/e2e"))
+from coverage_registry.management_cases import MANAGEMENT_CASES
+
+
+def main() -> int:
+    selected: Final = tuple(sys.argv[2:])
+    try:
+        report: Final = ET.parse(Path(sys.argv[1])).getroot()
+    except (ET.ParseError, OSError):
+        _ = sys.stdout.write("::error::could not read the test execution report\n")
+        return 1
+    cases: Final = tuple(report.iter("testcase"))
+    passed: Final = frozenset(
+        case.get("file") for case in cases if all(case.find(tag) is None for tag in ("skipped", "failure", "error"))
+    )
+    missing: Final = tuple(path for path in selected if path not in passed)
+    required_nodes: Final = frozenset(case.node for case in MANAGEMENT_CASES if case.node.split("::", 1)[0] in selected)
+    passed_nodes: Final = frozenset(
+        prop.get("value")
+        for case in cases
+        if all(case.find(tag) is None for tag in ("skipped", "failure", "error"))
+        for prop in case.findall("./properties/property")
+        if prop.get("name") == "management_node"
+    )
+    missing_nodes: Final = required_nodes - passed_nodes
+    for node in sorted(missing_nodes):
+        _ = sys.stdout.write(f"::error::required management case did not pass: {node}\n")
+    for path in selected:
+        collected: Final = sum(case.get("file") == path for case in cases)
+        skipped: Final = sum(case.get("file") == path and case.find("skipped") is not None for case in cases)
+        _ = sys.stdout.write(f"{path}: {collected} collected, {skipped} skipped\n")
+        for case in cases:
+            if case.get("file") != path or all(case.find(tag) is None for tag in ("failure", "error")):
+                continue
+            _ = sys.stdout.write(f"  failed: {case.get('classname', '')}::{case.get('name', '')}\n")
+    if (
+        selected
+        and not missing
+        and not missing_nodes
+        and not any(case.find(tag) is not None for case in cases for tag in ("failure", "error"))
+    ):
+        return 0
+    _ = sys.stdout.write("::error::every selected file must execute a passing test, with no failures or errors\n")
+    return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

.github/e2e-stack/down.sh

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+set -uo pipefail
+
+STACK_DIR="${E2E_STACK_DIR:-${RUNNER_TEMP:-/tmp}/litellm-e2e-stack}"
+
+for pid_file in "${STACK_DIR}"/pids/*.pid; do
+  [[ -f "${pid_file}" ]] || continue
+  pkill -TERM -P "$(cat "${pid_file}")" 2>/dev/null
+  kill -TERM "$(cat "${pid_file}")" 2>/dev/null
+  rm -f "${pid_file}"
+done
+
+for container in e2e-nginx e2e-keycloak e2e-valkey e2e-jaeger e2e-postgres; do
+  docker rm -f "${container}" >/dev/null 2>&1
+done
+
+exit 0

.github/e2e-stack/secrets_to_env.py

@@ -0,0 +1,49 @@
+import os
+import re
+import sys
+from pathlib import Path
+from typing import Final
+
+from pydantic import TypeAdapter, ValidationError
+
+secrets_adapter: Final[TypeAdapter[dict[str, str]]] = TypeAdapter(dict[str, str])
+ENV_NAME: Final = re.compile(r"[A-Za-z_][A-Za-z0-9_]*")
+MIN_MASKED_LENGTH: Final = 8
+
+
+def main() -> int:
+    env_path: Final = Path(sys.argv[1])
+    try:
+        secrets: Final = {
+            key: value.rstrip("\r\n") for key, value in secrets_adapter.validate_json(sys.stdin.read()).items()
+        }
+    except (ValidationError, UnicodeError):
+        _ = sys.stderr.write("expected a JSON object containing string environment values\n")
+        return 1
+    unusable: Final = tuple(
+        key
+        for key, value in secrets.items()
+        if ENV_NAME.fullmatch(key) is None or any(char in value for char in "'\n\r\0")
+    )
+    if unusable:
+        _ = sys.stderr.write(
+            f"these names or values cannot be represented in both bash and dotenv: {' '.join(sorted(unusable))}\n"
+        )
+        return 1
+    for value in secrets.values():
+        if len(value) >= MIN_MASKED_LENGTH:
+            _ = sys.stdout.write(f"::add-mask::{value.replace('%', '%25')}\n")
+    sys.stdout.flush()
+    lines: Final = tuple(f"{key}='{value}'" for key, value in secrets.items() if value)
+    try:
+        with os.fdopen(os.open(env_path, os.O_WRONLY | os.O_APPEND | os.O_CREAT | os.O_NOFOLLOW, 0o600), "w") as handle:
+            os.fchmod(handle.fileno(), 0o600)
+            _ = handle.write("\n".join(lines) + "\n")
+    except OSError:
+        _ = sys.stderr.write("could not write the environment file\n")
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

.github/e2e-stack/select_tests.py

@@ -0,0 +1,47 @@
+import re
+import sys
+from typing import Final
+
+SELECTABLE: Final = re.compile(r"^tests/e2e/([A-Za-z0-9_.-]+/)*test_[A-Za-z0-9_.-]+\.py$")
+UNSUPPORTED: Final = re.compile(
+    r"^tests/e2e/(ui|claude_code|load)/"
+    r"|^tests/e2e/llm_translation/realtime/test_realtime_pipecat_audio_e2e\.py$"
+    r"|^tests/e2e/batches/test_managed_files_enforcement_e2e\.py$"
+    r"|^tests/e2e/guardrails/test_presidio_masking_e2e\.py$"
+)
+HARNESS: Final = re.compile(
+    r"^tests/e2e/[A-Za-z0-9_.-]+\.(py|ini)$"
+    r"|^tests/e2e/idp_realm\.json$"
+    r"|^tests/e2e/management/(management_client|jwt_actors|conftest)\.py$"
+    r"|^tests/e2e/coverage_registry/management_cases\.py$"
+    r"|^tests/e2e/gateway/"
+    r"|^\.github/e2e-stack/"
+    r"|^\.github/workflows/test-e2e-changed\.yml$"
+)
+UNEXPANDED: Final = re.compile(r"[*?\[]")
+
+
+def is_selectable(path: str) -> bool:
+    return SELECTABLE.match(path) is not None and UNSUPPORTED.match(path) is None
+
+
+def select(changed: tuple[str, ...], canary: tuple[str, ...]) -> tuple[str, ...]:
+    direct: Final = frozenset(path for path in changed if is_selectable(path))
+    harness_changed: Final = any(HARNESS.match(path) for path in changed)
+    canary_tests: Final = frozenset(path for path in canary if harness_changed and is_selectable(path))
+    return tuple(sorted(direct | canary_tests))
+
+
+def main() -> int:
+    canary: Final = tuple(sys.argv[1:])
+    unexpanded: Final = tuple(path for path in canary if UNEXPANDED.search(path))
+    if unexpanded:
+        _ = sys.stderr.write(f"the canary paths reached the selector unexpanded: {' '.join(unexpanded)}\n")
+        return 1
+    changed: Final = tuple(line.strip() for line in sys.stdin if line.strip())
+    _ = sys.stdout.write(" ".join(select(changed, canary)) + "\n")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

.github/e2e-stack/up.sh

@@ -0,0 +1,216 @@
+#!/usr/bin/env bash
+set -euo pipefail
+umask 077
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+STACK_DIR="${E2E_STACK_DIR:-${RUNNER_TEMP:-/tmp}/litellm-e2e-stack}"
+CERTS_DIR="${STACK_DIR}/certs"
+LOGS_DIR="${STACK_DIR}/logs"
+PIDS_DIR="${STACK_DIR}/pids"
+
+POSTGRES_IMAGE="${E2E_POSTGRES_IMAGE:-postgres:16.6}"
+VALKEY_IMAGE="${E2E_VALKEY_IMAGE:-valkey/valkey:8.1.4@sha256:81db6d39e1bba3b3ff32bd3a1b19a6d69690f94a3954ec131277b9a26b95b3aa}"
+JAEGER_IMAGE="${E2E_JAEGER_IMAGE:-jaegertracing/jaeger:2.10.0}"
+NGINX_IMAGE="${E2E_NGINX_IMAGE:-nginx:1.29.1-alpine@sha256:42a516af16b852e33b7682d5ef8acbd5d13fe08fecadc7ed98605ba5e3b26ab8}"
+
+LB_PORT="${E2E_LB_PORT:-4000}"
+GATEWAY_PORT_1="${E2E_GATEWAY_PORT_1:-4010}"
+GATEWAY_PORT_2="${E2E_GATEWAY_PORT_2:-4011}"
+BACKEND_PORT="${E2E_BACKEND_PORT:-4001}"
+REDIS_PORT="${E2E_REDIS_PORT:-6379}"
+DATABASE_HOST="${E2E_DATABASE_HOST:-127.0.0.1}"
+DATABASE_PORT="${E2E_DATABASE_PORT:-5432}"
+DATABASE_USER="${E2E_DATABASE_USER:-litellm}"
+DATABASE_PASSWORD="${E2E_DATABASE_PASSWORD:-dbpassword9090}"
+DATABASE_NAME="${E2E_DATABASE_NAME:-litellm}"
+JAEGER_OTLP_PORT="${E2E_JAEGER_OTLP_PORT:-4318}"
+JAEGER_QUERY_PORT="${E2E_JAEGER_QUERY_PORT:-16686}"
+KEYCLOAK_PORT="${E2E_KEYCLOAK_PORT:-8081}"
+
+MASTER_KEY="${LITELLM_MASTER_KEY:-sk-e2e-$(openssl rand -hex 16)}"
+
+mkdir -p "${CERTS_DIR}" "${LOGS_DIR}" "${PIDS_DIR}"
+chmod 700 "${STACK_DIR}" "${LOGS_DIR}" "${PIDS_DIR}"
+chmod 755 "${CERTS_DIR}"
+
+log() { printf 'e2e-stack: %s\n' "$*"; }
+
+port_open() { (exec 3<>"/dev/tcp/127.0.0.1/$1") 2>/dev/null; }
+
+wait_for() {
+  local label="$1" check="$2" deadline=$((SECONDS + ${3:-120}))
+  until eval "${check}"; do
+    if ((SECONDS >= deadline)); then
+      log "timed out waiting for ${label}"
+      exit 1
+    fi
+    sleep 2
+  done
+  log "${label} is up"
+}
+
+if [[ -f "${REPO_ROOT}/tests/e2e/.env" ]]; then
+  set -a
+  source "${REPO_ROOT}/tests/e2e/.env"
+  set +a
+fi
+
+if [[ -z "${DD_API_KEY:-}" ]]; then
+  log "DD_API_KEY is empty; the gateway config enables the datadog callback, so put a Datadog API key in tests/e2e/.env"
+  exit 1
+fi
+export DD_SITE="${DD_SITE:-datadoghq.com}"
+
+if ! port_open "${DATABASE_PORT}"; then
+  docker run -d --name e2e-postgres -p "${DATABASE_PORT}:5432" \
+    -e "POSTGRES_USER=${DATABASE_USER}" -e "POSTGRES_PASSWORD=${DATABASE_PASSWORD}" -e "POSTGRES_DB=${DATABASE_NAME}" \
+    "${POSTGRES_IMAGE}" >/dev/null
+fi
+wait_for "postgres" "port_open ${DATABASE_PORT}"
+
+if ! port_open "${JAEGER_QUERY_PORT}"; then
+  docker run -d --name e2e-jaeger -p "${JAEGER_OTLP_PORT}:4318" -p "${JAEGER_QUERY_PORT}:16686" \
+    "${JAEGER_IMAGE}" >/dev/null
+fi
+wait_for "jaeger" "curl -fs http://127.0.0.1:${JAEGER_QUERY_PORT}/api/services >/dev/null"
+
+openssl genrsa -out "${CERTS_DIR}/ca.key" 2048 2>/dev/null
+openssl req -x509 -new -nodes -key "${CERTS_DIR}/ca.key" -sha256 -days 7 \
+  -subj "/CN=litellm-e2e-ca" \
+  -addext "basicConstraints=critical,CA:TRUE" -addext "keyUsage=critical,keyCertSign,cRLSign" \
+  -out "${CERTS_DIR}/ca.crt" 2>/dev/null
+openssl genrsa -out "${CERTS_DIR}/server.key" 2048 2>/dev/null
+openssl req -new -key "${CERTS_DIR}/server.key" -subj "/CN=localhost" -out "${CERTS_DIR}/server.csr" 2>/dev/null
+openssl x509 -req -in "${CERTS_DIR}/server.csr" -CA "${CERTS_DIR}/ca.crt" -CAkey "${CERTS_DIR}/ca.key" \
+  -CAcreateserial -days 7 -sha256 \
+  -extfile <(printf 'basicConstraints=CA:FALSE\nkeyUsage=critical,digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth\nsubjectAltName=DNS:localhost,IP:127.0.0.1\n') \
+  -out "${CERTS_DIR}/server.crt" 2>/dev/null
+chmod 644 "${CERTS_DIR}"/*.key "${CERTS_DIR}"/*.crt
+
+CERTIFI_BUNDLE="$(cd "${REPO_ROOT}" && uv run --no-sync python -c 'import certifi; print(certifi.where())')"
+cat "${CERTIFI_BUNDLE}" "${CERTS_DIR}/ca.crt" > "${CERTS_DIR}/ca-bundle.pem"
+
+docker rm -f e2e-valkey >/dev/null 2>&1 || true
+docker run -d --name e2e-valkey -p "${REDIS_PORT}:${REDIS_PORT}" -v "${CERTS_DIR}:/certs:ro" \
+  "${VALKEY_IMAGE}" valkey-server \
+  --cluster-enabled yes --port 0 --tls-port "${REDIS_PORT}" \
+  --tls-cert-file /certs/server.crt --tls-key-file /certs/server.key --tls-ca-cert-file /certs/ca.crt \
+  --tls-auth-clients no --cluster-announce-ip 127.0.0.1 >/dev/null
+VALKEY_CLI="docker exec e2e-valkey valkey-cli --tls --cacert /certs/ca.crt -h 127.0.0.1 -p ${REDIS_PORT}"
+wait_for "valkey" "${VALKEY_CLI} ping 2>/dev/null | grep -q PONG"
+${VALKEY_CLI} cluster addslotsrange 0 16383 >/dev/null
+wait_for "valkey cluster" "${VALKEY_CLI} cluster info 2>/dev/null | grep -q cluster_state:ok"
+
+CONFIG_SOURCE="${REPO_ROOT}/tests/e2e/gateway/stage_mirror_ci_config.yml"
+CONFIG_PATH="${CONFIG_SOURCE}"
+if [[ "${REDIS_PORT}" != "6379" ]]; then
+  CONFIG_PATH="${STACK_DIR}/litellm-config.yml"
+  sed "s/port: 6379/port: ${REDIS_PORT}/" "${CONFIG_SOURCE}" > "${CONFIG_PATH}"
+fi
+
+SERVER_ENV=(
+  "LITELLM_MASTER_KEY=${MASTER_KEY}"
+  "DATABASE_HOST=${DATABASE_HOST}"
+  "DATABASE_PORT=${DATABASE_PORT}"
+  "DATABASE_USER=${DATABASE_USER}"
+  "DATABASE_PASSWORD=${DATABASE_PASSWORD}"
+  "DATABASE_NAME=${DATABASE_NAME}"
+  "DISABLE_SCHEMA_UPDATE=true"
+  "REDIS_HOST=127.0.0.1"
+  "REDIS_PORT=${REDIS_PORT}"
+  "REDIS_CLUSTER_NODES=[{\"host\":\"127.0.0.1\",\"port\":${REDIS_PORT}}]"
+  "CONFIG_FILE_PATH=${CONFIG_PATH}"
+  "STORE_MODEL_IN_DB=True"
+  "OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf"
+  "OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:${JAEGER_OTLP_PORT}"
+  "SSL_CERT_FILE=${CERTS_DIR}/ca-bundle.pem"
+  "PYTHONPATH=${REPO_ROOT}"
+  "JWT_PUBLIC_KEY_URL=http://127.0.0.1:${KEYCLOAK_PORT}/realms/litellm-e2e/protocol/openid-connect/certs"
+  "JWT_ISSUER=http://127.0.0.1:${KEYCLOAK_PORT}/realms/litellm-e2e"
+  "JWT_AUDIENCE=litellm-e2e"
+)
+if [[ -n "${VERTEXAI_CREDENTIALS:-}" ]]; then
+  printf '%s' "${VERTEXAI_CREDENTIALS}" > "${STACK_DIR}/vertex-adc.json"
+  SERVER_ENV+=("GOOGLE_APPLICATION_CREDENTIALS=${STACK_DIR}/vertex-adc.json")
+fi
+
+cd "${REPO_ROOT}"
+
+env "${SERVER_ENV[@]}" "E2E_KEYCLOAK_PORT=${KEYCLOAK_PORT}" bash .github/e2e-stack/start-idp.sh
+
+log "running migrations"
+env "${SERVER_ENV[@]}" uv run --no-sync python migrations/run.py >"${LOGS_DIR}/migrations.log" 2>&1
+
+start_server() {
+  local name="$1"; shift
+  env "${SERVER_ENV[@]}" "$@" >"${LOGS_DIR}/${name}.log" 2>&1 &
+  echo $! > "${PIDS_DIR}/${name}.pid"
+}
+
+start_server backend uv run --no-sync uvicorn backend.main:app --host 0.0.0.0 --port "${BACKEND_PORT}"
+start_server gateway-1 uv run --no-sync uvicorn gateway.main:app --workers 1 --host 0.0.0.0 --port "${GATEWAY_PORT_1}"
+start_server gateway-2 uv run --no-sync uvicorn gateway.main:app --workers 1 --host 0.0.0.0 --port "${GATEWAY_PORT_2}"
+
+if [[ "$(uname)" == "Linux" ]]; then
+  NGINX_UPSTREAM_HOST=127.0.0.1
+  NGINX_DOCKER_ARGS=(--network host)
+else
+  NGINX_UPSTREAM_HOST=host.docker.internal
+  NGINX_DOCKER_ARGS=(-p "${LB_PORT}:${LB_PORT}")
+fi
+
+cat > "${STACK_DIR}/nginx.conf" <<EOF
+events {}
+http {
+  map \$http_upgrade \$connection_upgrade {
+    default upgrade;
+    '' close;
+  }
+  upstream litellm_gateways {
+    server ${NGINX_UPSTREAM_HOST}:${GATEWAY_PORT_1};
+    server ${NGINX_UPSTREAM_HOST}:${GATEWAY_PORT_2};
+  }
+  server {
+    listen ${LB_PORT};
+    client_max_body_size 100m;
+    location / {
+      proxy_pass http://litellm_gateways;
+      proxy_http_version 1.1;
+      proxy_set_header Host \$host;
+      proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
+      proxy_set_header X-Forwarded-Proto \$scheme;
+      proxy_set_header Upgrade \$http_upgrade;
+      proxy_set_header Connection \$connection_upgrade;
+      proxy_buffering off;
+      proxy_read_timeout 600s;
+      proxy_send_timeout 600s;
+    }
+  }
+}
+EOF
+
+docker rm -f e2e-nginx >/dev/null 2>&1 || true
+docker run -d --name e2e-nginx "${NGINX_DOCKER_ARGS[@]}" \
+  -v "${STACK_DIR}/nginx.conf:/etc/nginx/nginx.conf:ro" "${NGINX_IMAGE}" >/dev/null
+
+wait_for "backend" "curl -fs http://127.0.0.1:${BACKEND_PORT}/health/liveliness >/dev/null" 300
+wait_for "gateway-1" "curl -fs http://127.0.0.1:${GATEWAY_PORT_1}/health/liveliness >/dev/null" 300
+wait_for "gateway-2" "curl -fs http://127.0.0.1:${GATEWAY_PORT_2}/health/liveliness >/dev/null" 300
+wait_for "load balancer" "curl -fs http://127.0.0.1:${LB_PORT}/health/liveliness >/dev/null" 60
+
+cat > "${STACK_DIR}/stack.env" <<EOF
+LITELLM_PROXY_URL=http://127.0.0.1:${LB_PORT}
+LITELLM_CONTROL_PLANE_URL=http://127.0.0.1:${BACKEND_PORT}
+LITELLM_PROXY_REPLICA_URLS=http://127.0.0.1:${GATEWAY_PORT_1},http://127.0.0.1:${GATEWAY_PORT_2}
+LITELLM_MASTER_KEY=${MASTER_KEY}
+REDIS_HOST=127.0.0.1
+REDIS_PORT=${REDIS_PORT}
+E2E_OTEL_QUERY_URL=http://127.0.0.1:${JAEGER_QUERY_PORT}
+E2E_KEYCLOAK_URL=http://127.0.0.1:${KEYCLOAK_PORT}
+E2E_KEYCLOAK_ADMIN_USER=admin
+E2E_KEYCLOAK_ADMIN_PASSWORD=e2e-ephemeral-idp-not-a-secret
+SSL_CERT_FILE=${CERTS_DIR}/ca-bundle.pem
+DATABASE_URL=postgresql://${DATABASE_USER}:${DATABASE_PASSWORD}@${DATABASE_HOST}:${DATABASE_PORT}/${DATABASE_NAME}
+EOF
+
+log "stack is up; pytest env written to ${STACK_DIR}/stack.env"

.github/pull_request_template.md

@@ -0,0 +1,162 @@
+<!-- The whole description's target audience is humans, not AI agents: write it in plain, simple,
+     everyday engineering language, extremely parsable and readable at a glance. This goes double for
+     the TLDR, User Flow, and Caveats sections -->
+
+## TLDR
+
+<!-- Fill in the bullets below and keep each one short and concrete: one line per bullet, roughly 10 words max -->
+
+Problem this solves:
+
+- <blah>
+- ...
+
+How it solves it:
+
+- <blah>
+- ...
+
+## User Flow
+
+<!-- Two ordered lists, Before and After, walking the same end user through the same task, written strictly from that user's seat
+     Read the linked issue, ticket, or customer thread first so the flow reflects the real application and the routes its users actually hit; don't invent a generic scenario
+     Lead each list with one plain sentence saying where the flow fails (Before) or succeeds (After), then number the steps
+     Every step is something the user does or observes: the HTTP method and full URL they hit, what they sent, and what visibly came back (status code, error text, the shape of an ID). UI steps name the page URL and what is on screen
+     No LiteLLM internals: never name functions, files, DB tables, config classes, hooks, callbacks, or code paths. "The upload hands back an ID that looks like OpenAI's own `file-abc123` instead of the scrambled one the gateway returned" is right, "no managed-file row was registered" is wrong
+     Keep the two lists step-for-step identical until they diverge, so the changed step is obvious
+     If the bug had a security or authorization consequence, end each list with what another user could or could no longer do
+     Regenerate this section whenever new commits change the PR's behavior, so it never describes an older revision
+
+Example:
+
+Before: a developer whose app streams chat completions gets no token counts back, so their cost dashboard reads zero
+
+1. They send POST https://litellm-domain/v1/chat/completions with `"stream": true` and no `stream_options`
+2. The last SSE chunk arrives with `"usage": null`, so their app records 0 prompt and 0 completion tokens
+3. They open https://litellm-domain/ui/?page=logs and see the request logged at $0 spend
+
+After: the same request comes back with real token counts, so the dashboard shows real spend
+
+1. The proxy admin sets `always_include_stream_usage: true` and restarts the proxy
+2. The developer sends the same POST https://litellm-domain/v1/chat/completions with `"stream": true` and no `stream_options`
+3. The last SSE chunk now carries a `usage` object with real prompt and completion token counts
+4. https://litellm-domain/ui/?page=logs shows that request at non-zero spend
+-->
+
+## Relevant issues
+
+<!-- e.g., "Fixes #000" -->
+
+## Affected release
+
+<!-- Only for a fix to a regression in a released or rc version (perf, memory, crash, or behavior): name the version it regressed in, e.g. "regression in v1.100.0" or "since v1.101.0-rc.1", and add the `backport-stable` label so the fix is cherry-picked onto the rc line before the stable is tagged. Leave the section blank otherwise -->
+
+## Linear ticket
+
+<!-- if you are an internal contributor, add "Resolves " followed by the Linear ticket e.g., "Resolves LIT-1234" to link the Linear ticket to the GitHub PR. If you don't have one, leave the section blank rather than guessing -->
+
+## Pre-Submission checklist
+
+**Please complete all items before asking a LiteLLM maintainer to review your PR**
+
+- [ ] I have added meaningful tests
+- [ ] The handful of test files covering my change pass locally, e.g. `uv run pytest tests/test_litellm/<your_test_file>.py -v`. Leave the suites (`make test-unit-*`, `make test-unit`) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
+- [ ] My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
+- [ ] My PR's scope is as isolated as possible; it only solves 1 specific problem
+- [ ] I have received a Greptile **Confidence Score of at least 4/5** before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment `@greptileai` to re-request a review after pushing changes)
+
+## Delays in PR merge?
+
+If you're seeing a delay in your PR being merged, ping the LiteLLM Team on [Slack (#pr-review)](https://join.slack.com/t/litellmossslack/shared_invite/zt-3o7nkuyfr-p_kbNJj8taRfXGgQI1~YyA).
+
+## Screenshots / Proof of Fix
+
+<!-- Include screenshots, screen recordings, or command (e.g., curl) + output demonstrating that your changes work as expected
+     The proof must be completely e2e with no mocks, using actual LLM calls costing real $$$ if applicable. `pytest` commands are not enough
+     Show ONLY the latest run: capture Before at the merge base and After at the PR's current tip, and when new commits change behavior, replace this whole section with the fresh run instead of stacking it on top of older ones. The run must be up to date. As soon as a new commit is made and it makes this PR description's after sha stale (it's no longer tip of PR), you must re-run the QA
+     Structure the section exactly as below: Before and After one heading level below this section, each naming the commit hash it was captured at, one lower-level heading per case inside each, the same case names in the same order on both sides, and numbered steps (command, observed output) under every case, never loose prose; shared setup (config, payloads) goes above Before, and with a single case, drop the case headings and number the steps directly
+
+### Before (<hash>)
+
+#### <case 1>
+
+1. ...
+2. ...
+
+#### <case 2>
+
+1. ...
+
+### After (<hash>)
+
+#### <case 1>
+
+1. ...
+2. ...
+
+#### <case 2>
+
+1. ...
+
+     For bug fixes: Before shows the reproduction, After shows the same steps passing
+     For new features: Before shows the capability missing, After shows it working end-to-end
+     If the change applies to all three LLM endpoints (/v1/responses, /v1/chat/completions, /v1/messages), make each endpoint its own case, not just one
+     For UI changes: before/after screenshots under the same headings
+     If the main use case runs through a coding tool like Claude Code or Codex, drive that tool interactively the way the user does (never `claude -p`, `codex exec`, or curl on its own) and embed before/after screenshots of its pane under the same headings; curl replays and headless runs can follow as extra cases, never as the only proof -->
+
+## Type
+
+<!-- Select the type of Pull Request -->
+<!-- Keep only the necessary ones -->
+
+🆕 New Feature
+🐛 Bug Fix
+🧹 Refactoring
+📖 Documentation
+🚄 Infrastructure
+✅ Test
+
+## Caveats (if any)
+
+<!-- Group caveats under severity subheadings (### Severe, ### High, ### Medium, ### Low), with
+     short bullet points inside each, just like the TLDR: one line per bullet, roughly 10 words max
+     Call out known limitations, follow-up work, or anything a reviewer should watch out for
+     Include only the tiers that have caveats; drop the empty ones
+     - Severe: inherent to what the PR deliberately ships, there even when the code works as intended:
+       it can degrade or take down a running deployment (e.g. a slow or table-locking boot migration),
+       rewrite data by design, break an existing workflow on purpose, or change auth behavior. An
+       operator must plan around it before rollout
+     - High: an unintended hole: a correctness, security, data-loss, or backward-compatibility bug,
+       unsafe to ship as is
+     - Medium: a real gap someone can hit, but with a workaround or a narrow blast radius
+     - Low: anything else worth noting: naming, cleanup, an edge case nobody hits
+     Nest bullets as deep as helps: hierarchy beats one long line when it makes things clearer to a
+     human reader
+     If you assumed something instead of testing it, e.g. "only reproduces with X on" or "no
+     user-observable behavior difference", list it here too with what breaks if it is wrong
+     Leave this section empty if there are none -->
+
+## QA runbook
+
+<!-- Only needed when your PR edits tests/e2e; delete this section otherwise
+
+For each e2e test you added or changed, list the manual steps a reviewer can follow to reproduce it by hand against a live proxy, mapping 1:1 to what the test asserts: one top-level bullet per test giving its pytest node id followed by what it proves in plain words, then a nested "- [ ]" checklist where each item is a concrete action (route, request body, expected response) and the final item is the sanity-check step shown in the examples. Note environment prerequisites (provider credentials, config flags) and any nuances a manual run will hit. See PRs #32914 and #32963 for full examples
+
+Example checklists:
+
+- tests/e2e/quota_management/ratelimit/test_rate_limit_e2e.py::TestKeyRateLimits::test_rpm_limit_blocks_over_limit - a key allowed 2 requests a minute serves exactly 2 and refuses the 3rd
+  - [ ] Generate a limited key: curl -X POST http://localhost:4000/key/generate -H "Authorization: Bearer sk-1234" -d '{"rpm_limit": 2}'
+  - [ ] Send three /v1/chat/completions requests with that key inside one minute
+  - [ ] Expect the first two to return 200 and the third to return 429 naming the rpm limit
+  - [ ] Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
+
+- tests/e2e/management/test_management_e2e.py::TestModelRoutes::test_model_create_appears_in_ui - a deployment created through the API shows up on the Admin UI models page
+  - [ ] POST /model/new with the master key, a bedrock model, and aws_region_name (needs STORE_MODEL_IN_DB=True and AWS credentials)
+  - [ ] Open http://localhost:4000/ui/?page=models and expect a deployment row showing the returned model id
+  - [ ] Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
+-->
+
+## Final Attestation
+
+- [ ] The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
+

.github/scripts/assert_ci_coverage.py

@@ -9,8 +9,6 @@ import sys
 import warnings
 from collections.abc import Callable, Iterable, Mapping, Sequence
 from dataclasses import dataclass
-from functools import lru_cache
-from types import MappingProxyType
 from typing import Final
 
 import yaml
@@ -23,30 +21,22 @@ TESTS_ROOT = REPO_ROOT / "tests"
 
 ALLOWLIST_KEYS = frozenset({"description", "test_paths", "dockerfiles"})
 PATH_FILTER_KEYS = frozenset({"paths", "paths-ignore"})
-TEST_PATH_KEYS = frozenset({"test-path", "test-paths", "test_path"})
+TEST_PATH_KEYS = frozenset({"test-path", "test-paths"})
 DOCKERFILE_INPUT_KEYS = frozenset({"file", "dockerfile"})
 TEST_RUNNER_RE = re.compile(r"\bpytest\b|\bcircleci tests\b|\bhelm unittest\b|\bplaywright test\b|\bpython[0-9.]*\s")
 IMAGE_BUILD_RE = re.compile(r"\bdocker\s+(?:buildx\s+)?build\b")
 TEST_TOKEN_RE = re.compile(r"tests/[A-Za-z0-9_./*?-]+")
-IGNORE_ARG_RE: Final = re.compile(r"--ignore(?:-glob)?[= ](\S+)")
 DOCKERFILE_TOKEN_RE = re.compile(r"[A-Za-z0-9_./-]*Dockerfile[A-Za-z0-9_.-]*")
 COMMENT_RE = re.compile(r"^\s*#.*$", re.MULTILINE)
 GLOB_CHARS = frozenset("*?")
-DEDICATED_INTEGRATION_WORKFLOWS: Final = MappingProxyType({"image-scan.yml": "tests/integration/database/image/"})
 
 # Trees whose jobs are sharded with no catch-all bucket, so every child that holds
 # tests has to be named by some shard or it runs nowhere. A child listed here is
 # itself decomposed one level deeper and is checked through its own entry.
 SHARDED_ROOTS: tuple[str, ...] = (
-    "tests/unit",
-    "tests/unit/enterprise",
-    "tests/unit/enterprise/enterprise_callbacks",
-    "tests/unit/enterprise/proxy",
-    "tests/unit/llms",
-    "tests/unit/proxy",
-    "tests/unit/proxy/_experimental",
-    "tests/unit/responses",
-    "tests/unit/skills",
+    "tests/proxy_unit_tests",
+    "tests/test_litellm",
+    "tests/test_litellm/proxy",
 )
 
 
@@ -82,17 +72,6 @@ class Scalar:
     value: str
 
 
-@dataclass(frozen=True, slots=True)
-class Selection:
-    included: frozenset[str]
-    ignored: frozenset[str]
-
-    def covers(self, relative_path: str) -> bool:
-        return any(_token_covers(token, relative_path) for token in self.included) and not any(
-            _token_covers(token, relative_path) for token in self.ignored
-        )
-
-
 @dataclass(frozen=True, slots=True)
 class Finding:
     subject: str
@@ -132,32 +111,13 @@ def _uncommented(value: str) -> str:
     return COMMENT_RE.sub("", value)
 
 
-def _selection_for_scalar(scalar: Scalar) -> Selection:
-    text: Final = _uncommented(scalar.value)
-    ignored: Final = frozenset().union(
-        *(
-            frozenset(token.rstrip("/") for token in TEST_TOKEN_RE.findall(ignored_argument))
-            for ignored_argument in IGNORE_ARG_RE.findall(text)
-        )
-    )
-    included: Final = frozenset(
-        match.group(0).rstrip("/") for match in TEST_TOKEN_RE.finditer(IGNORE_ARG_RE.sub("", text))
-    )
-    return Selection(included=included, ignored=ignored)
-
-
-def _invoked_selections(scalars: Iterable[Scalar]) -> tuple[Selection, ...]:
-    selected_scalars: Final = tuple(
-        scalar
+def _invoked_test_tokens(scalars: Iterable[Scalar]) -> frozenset[str]:
+    return frozenset(
+        match.group(0).rstrip("/")
         for scalar in scalars
         if scalar.key in TEST_PATH_KEYS or TEST_RUNNER_RE.search(scalar.value)
+        for match in TEST_TOKEN_RE.finditer(_uncommented(scalar.value))
     )
-    selections: Final = tuple(_selection_for_scalar(scalar) for scalar in selected_scalars)
-    return tuple(selection for selection in selections if selection.included)
-
-
-def _invoked_test_tokens(scalars: Iterable[Scalar]) -> frozenset[str]:
-    return frozenset().union(*(selection.included for selection in _invoked_selections(scalars)))
 
 
 def _built_dockerfile_tokens(scalars: Iterable[Scalar]) -> frozenset[str]:
@@ -220,12 +180,11 @@ def _dockerfiles() -> tuple[str, ...]:
     )
 
 
-def _uncovered_tests(allowlist: Allowlist, selections: tuple[Selection, ...]) -> tuple[Finding, ...]:
+def _uncovered_tests(allowlist: Allowlist, tokens: frozenset[str]) -> tuple[Finding, ...]:
     uncovered = tuple(
         relative_path
         for relative_path in _test_files()
-        if not any(selection.covers(relative_path) for selection in selections)
-        and not allowlist.covers_test(relative_path)
+        if not any(_token_covers(token, relative_path) for token in tokens) and not allowlist.covers_test(relative_path)
     )
     directories = tuple(dict.fromkeys(path.rsplit("/", 1)[0] for path in uncovered))
     return tuple(
@@ -276,7 +235,9 @@ class Slice:
             return True  # a `-k` this parser cannot model is assumed to claim everything
         if any(term.lower() in relative_path.lower() for term in self.excluded):
             return False
-        return not self.required or any(term.lower() in name.lower() for term in self.required for name in inner_names)
+        return not self.required or any(
+            term.lower() in name.lower() for term in self.required for name in inner_names
+        )
 
 
 def _strings(node: object) -> Iterable[str]:
@@ -318,7 +279,6 @@ def _keyword_terms(
 def _slices() -> tuple[Slice, ...]:
     if not CIRCLECI_CONFIG.exists():
         return ()
-    test_files: Final = frozenset(_test_files())
     jobs: Final = yaml.safe_load(CIRCLECI_CONFIG.read_text()).get("jobs", {})
     return tuple(
         Slice(job=job, globs=globs, named=named, required=required, excluded=excluded, understood=understood)
@@ -326,7 +286,7 @@ def _slices() -> tuple[Slice, ...]:
         for text in ("\n".join(_strings(body)),)
         if "pytest" in text
         for globs in (tuple(GLOB_CALL_RE.findall(text)),)
-        for named in (frozenset(TEST_TOKEN_RE.findall(text)) & test_files,)
+        for named in (frozenset(TEST_TOKEN_RE.findall(text)) & frozenset(_test_files()),)
         for required, excluded, understood in (
             _keyword_terms(tuple(KEYWORD_RE.findall(text)), attributable=len(globs) < 2),
         )
@@ -347,7 +307,9 @@ def _matchable_names(relative_path: str) -> frozenset[str]:
     except (OSError, SyntaxError):
         return frozenset({relative_path})
     return frozenset({relative_path}) | frozenset(
-        node.name for node in ast.walk(tree) if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef))
+        node.name
+        for node in ast.walk(tree)
+        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef))
     )
 
 
@@ -366,11 +328,12 @@ def _workflow_named_tokens() -> frozenset[str]:
 
 
 def _deselected_everywhere(allowlist: Allowlist) -> tuple[Finding, ...]:
-    matchable_names: Final = lru_cache(maxsize=None)(_matchable_names)
     slices: Final = _slices()
     named_by_workflow: Final = _workflow_named_tokens()
     globbed: Final = tuple(
-        path for path in _test_files() if any(_token_covers(glob, path) for slice_ in slices for glob in slice_.globs)
+        path
+        for path in _test_files()
+        if any(_token_covers(glob, path) for slice_ in slices for glob in slice_.globs)
     )
     return tuple(
         Finding(
@@ -380,7 +343,7 @@ def _deselected_everywhere(allowlist: Allowlist) -> tuple[Finding, ...]:
         for path in globbed
         if not allowlist.covers_test(path)
         and not any(_token_covers(token, path) for token in named_by_workflow)
-        and not any(slice_.claims(path, matchable_names(path)) for slice_ in slices)
+        and not any(slice_.claims(path, _matchable_names(path)) for slice_ in slices)
     )
 
 
@@ -400,7 +363,11 @@ def _shard_children(root: str, repo_root: pathlib.Path = REPO_ROOT) -> tuple[str
             child.relative_to(repo_root).as_posix()
             for child in (repo_root / root).iterdir()
             if not child.name.startswith(".")
-            and (_holds_tests(child) if child.is_dir() else child.name.startswith("test_") and child.suffix == ".py")
+            and (
+                _holds_tests(child)
+                if child.is_dir()
+                else child.name.startswith("test_") and child.suffix == ".py"
+            )
         )
     )
 
@@ -532,68 +499,15 @@ def _check_shards() -> int:
     return 0
 
 
-def _integration_groups(runner: pathlib.Path) -> dict[str, tuple[str, ...]]:
-    module: Final = ast.parse(runner.read_text())
-    literal: Final = next(
-        node.value
-        for node in module.body
-        if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name) and node.target.id == "GROUPS"
-    )
-    mapping: Final = literal.args[0] if isinstance(literal, ast.Call) else literal
-    return {group: tuple(folders) for group, folders in ast.literal_eval(mapping).items()}
-
-
-def _integration_github_files(runner: pathlib.Path) -> frozenset[str]:
-    module: Final = ast.parse(runner.read_text())
-    literal: Final = next(
-        (
-            node.value
-            for node in module.body
-            if isinstance(node, ast.AnnAssign)
-            and isinstance(node.target, ast.Name)
-            and node.target.id == "GITHUB_FILES"
-        ),
-        None,
-    )
-    if literal is None:
-        return frozenset()
-    values: Final = literal.args[0] if isinstance(literal, ast.Call) else literal
-    return frozenset(ast.literal_eval(values))
-
-
-def _owns(workflow: str, relative_path: str) -> bool:
-    owned_root: Final = DEDICATED_INTEGRATION_WORKFLOWS.get(workflow)
-    return owned_root is not None and relative_path.startswith(owned_root)
-
-
-def _tokens_outside_owner(relative_path: str, workflow_tokens: Mapping[str, frozenset[str]]) -> frozenset[str]:
-    return frozenset[str]().union(
-        *(tokens for workflow, tokens in workflow_tokens.items() if not _owns(workflow, relative_path))
-    )
-
-
 def _integration_ownership(repo_root: pathlib.Path = REPO_ROOT) -> tuple[frozenset[str], tuple[Finding, ...]]:
-    runner: Final = repo_root / "tests/integration/run.py"
-    if not runner.exists():
+    manifest: Final = repo_root / "tests/integration/contracts.json"
+    if not manifest.exists():
         return frozenset(), ()
-    groups: Final = _integration_groups(runner)
-    github_files: Final = _integration_github_files(runner)
-    integration_root: Final = repo_root / "tests/integration"
-    paths: Final = frozenset(
-        str(path.relative_to(repo_root))
-        for folders in groups.values()
-        for folder in folders
-        for path in (integration_root / folder).rglob("test_*.py")
-        if str(path.relative_to(repo_root)) not in github_files
-    )
-    browser_manifest: Final = repo_root / "tests/e2e/ui/tests/integrationCritical/expected.json"
-    browser_nodes: Final = json.loads(browser_manifest.read_text()) if browser_manifest.exists() else ()
-    browser_paths: Final = frozenset(node.split("::", 1)[0] for node in browser_nodes)
+    entries: Final = json.loads(manifest.read_text())
+    paths: Final = frozenset(node.split("::", 1)[0] for node in entries["tests"])
+    browser_paths: Final = frozenset(node.split("::", 1)[0] for node in entries.get("browser", {}))
     circle_path: Final = repo_root / ".circleci/config.yml"
     circle: Final = yaml.safe_load(circle_path.read_text()) if circle_path.exists() else {}
-    circle_test_path_tokens: Final = _invoked_test_tokens(
-        scalar for scalar in _scalars(circle, "config.yml") if scalar.key == "test_path"
-    )
     steps: Final = circle.get("jobs", {}).get("integration_contracts", {}).get("steps", ())
     invoked: Final = any(
         ".circleci/scripts/run_integration.sh" in scalar.value
@@ -612,48 +526,31 @@ def _integration_ownership(repo_root: pathlib.Path = REPO_ROOT) -> tuple[frozens
     )
     required: Final = (frozenset({"browser"}) if browser_paths else frozenset()) | frozenset(
         group
-        for group, folders in groups.items()
+        for group, folders in entries["groups"].items()
         if any(any(path.startswith(f"tests/integration/{folder}/") for folder in folders) for path in paths)
     )
     ungrouped: Final = frozenset(
         path
         for path in paths
         if sum(
-            any(path.startswith(f"tests/integration/{folder}/") for folder in folders) for folders in groups.values()
+            any(path.startswith(f"tests/integration/{folder}/") for folder in folders)
+            for folders in entries["groups"].values()
         )
         != 1
     )
-    workflow_tokens: Final = MappingProxyType(
-        {
-            path.name: _invoked_test_tokens(_scalars(yaml.safe_load(path.read_text()), path.name))
-            for path in (repo_root / ".github/workflows").glob("*.y*ml")
-        }
+    gha_tokens: Final = _invoked_test_tokens(
+        scalar
+        for path in (repo_root / ".github/workflows").glob("*.y*ml")
+        for scalar in _scalars(yaml.safe_load(path.read_text()), path.name)
     )
-    gha_tokens: Final = frozenset[str]().union(*workflow_tokens.values())
-    circle_owned: Final = (
-        frozenset(str(path.relative_to(repo_root)) for path in integration_root.rglob("test_*.py")) - github_files
-    )
-    findings: Final = (
-        tuple(
-            Finding(path, "integration file outside GITHUB_FILES is selected by GitHub Actions")
-            for path in sorted(circle_owned)
-            if any(_token_covers(token, path) for token in gha_tokens)
-        )
-        + tuple(
-            Finding(path, "GitHub-owned integration contract has no invoking job")
-            for path in sorted(github_files)
-            if not any(_token_covers(token, path) for token in gha_tokens | circle_test_path_tokens)
-        )
-        + tuple(
-            Finding(path, "GitHub-owned integration file is missing")
-            for path in sorted(github_files)
-            if not (repo_root / path).is_file()
-        )
-        + tuple(
-            Finding(path, "GitHub-owned integration contract runs outside a dedicated integration workflow")
-            for path in sorted(github_files)
-            if any(_token_covers(token, path) for token in _tokens_outside_owner(path, workflow_tokens))
-        )
+    findings: Final = tuple(
+        Finding(path, "integration contract is also selected by GitHub Actions")
+        for path in paths
+        if any(_token_covers(token, path) for token in gha_tokens)
+    ) + tuple(
+        Finding(path, "canonical integration test file is missing")
+        for path in paths
+        if not (repo_root / path).is_file()
     )
     browser_commands: Final = tuple(
         scalar.value
@@ -695,9 +592,9 @@ def _integration_ownership(repo_root: pathlib.Path = REPO_ROOT) -> tuple[frozens
     ) + tuple(Finding(path, "canonical node must have exactly one integration group") for path in sorted(ungrouped))
     if not paths or not invoked or not scheduled:
         return frozenset(), findings + (
-            Finding(str(runner.relative_to(repo_root)), "dedicated CircleCI runner is missing"),
+            Finding(str(manifest.relative_to(repo_root)), "dedicated CircleCI runner is missing"),
         )
-    return paths | browser_paths | github_files, findings + group_findings + browser_findings + exclusion_findings
+    return paths | browser_paths, findings + group_findings + browser_findings + exclusion_findings
 
 
 def main() -> int:
@@ -710,10 +607,7 @@ def main() -> int:
     scalars = _all_scalars()
 
     integration_paths, ownership_findings = _integration_ownership()
-    test_findings = _uncovered_tests(
-        allowlist,
-        _invoked_selections(scalars) + (Selection(included=integration_paths, ignored=frozenset()),),
-    )
+    test_findings = _uncovered_tests(allowlist, _invoked_test_tokens(scalars) | integration_paths) + ownership_findings
     dockerfile_findings = _uncovered_dockerfiles(allowlist, _built_dockerfile_tokens(scalars))
     stale_findings = _stale_allowlist_paths(allowlist, test_files=_test_files(), dockerfiles=_dockerfiles())
 
@@ -729,25 +623,13 @@ def main() -> int:
             test_findings,
             "Add each to a job's test path, or list it in .github/ci-coverage-allowlist.yml with a reason.",
         )
-    if ownership_findings:
-        browser_remedy = (
-            " CircleCI's browser group owns the integrationCritical Playwright contracts, so no GitHub workflow may "
-            "select them and the default Playwright config must exclude them."
-            if any("browser" in finding.detail or "Playwright" in finding.detail for finding in ownership_findings)
-            else ""
-        )
-        _report(
-            "integration contracts with the wrong CI owner",
-            ownership_findings,
-            "Follow tests/integration/AGENTS.md, which says which CI job owns each integration file." + browser_remedy,
-        )
     if dockerfile_findings:
         _report(
             "Dockerfiles that no CI job builds",
             dockerfile_findings,
             "Build each in a workflow, or list it in .github/ci-coverage-allowlist.yml with a reason.",
         )
-    if stale_findings or test_findings or ownership_findings or dockerfile_findings:
+    if stale_findings or test_findings or dockerfile_findings:
         return 1
 
     _write(

.github/scripts/assert_secret_absent.py

@@ -1,119 +0,0 @@
-import base64
-import io
-import os
-import sys
-import urllib.parse
-import zipfile
-from collections.abc import Callable, Iterator, Mapping
-from dataclasses import dataclass
-from functools import reduce
-from itertools import chain
-from pathlib import Path
-from types import MappingProxyType
-from typing import Final, TextIO
-
-SECRET_VARIABLE: Final = "SCANNED_SECRET"
-MIN_ENCODED_LENGTH: Final = 12
-MAX_ARCHIVE_DEPTH: Final = 3
-MASK: Final = "***"
-
-Forms = tuple[tuple[str, str], ...]
-
-
-@dataclass(frozen=True, slots=True)
-class Hit:
-    location: str
-    form: str
-
-
-def _aligned_encodings(raw: bytes, name: str, encode: Callable[[bytes], bytes]) -> Iterator[tuple[str, str]]:
-    for offset in range(3):
-        chunk: Final = raw[offset:]
-        yield f"{name} offset {offset}", encode(chunk[: len(chunk) - len(chunk) % 3]).decode()
-
-
-def secret_forms(secret: str) -> Forms:
-    encoded: Final = (
-        ("url-encoded", urllib.parse.quote(secret, safe="")),
-        *_aligned_encodings(secret.encode(), "base64", base64.b64encode),
-        *_aligned_encodings(secret.encode(), "base64url", base64.urlsafe_b64encode),
-    )
-    by_value: Final = MappingProxyType(
-        {value: name for name, value in encoded if value != secret and len(value) >= MIN_ENCODED_LENGTH}
-    )
-    return (("raw", secret), *((name, value) for value, name in by_value.items()))
-
-
-def mask(text: str, forms: Forms) -> str:
-    return reduce(lambda masked, form: masked.replace(form[1], MASK), forms, text)
-
-
-def _name_hits(location: str, name: str, forms: Forms) -> Iterator[Hit]:
-    for form, value in forms:
-        if value in name:
-            yield Hit(mask(location, forms), f"{form} in name")
-
-
-def _content_hits(location: str, data: bytes, forms: Forms, depth: int) -> Iterator[Hit]:
-    for form, value in forms:
-        if value.encode() in data:
-            yield Hit(mask(location, forms), form)
-    if depth >= MAX_ARCHIVE_DEPTH or not zipfile.is_zipfile(io.BytesIO(data)):
-        return
-    with zipfile.ZipFile(io.BytesIO(data)) as archive:
-        for member in archive.infolist():
-            if member.is_dir():
-                continue
-            yield from _content_hits(f"{location}!{member.filename}", archive.read(member), forms, depth + 1)
-
-
-def _walk(root: Path) -> Iterator[tuple[Path, bool]]:
-    for directory, subdirectories, files in os.walk(root):
-        yield from ((Path(directory, name), True) for name in subdirectories if Path(directory, name).is_symlink())
-        yield from ((Path(directory, name), False) for name in files)
-
-
-def _path_hits(root: Path, path: Path, linked_directory: bool, forms: Forms) -> Iterator[Hit]:
-    location: Final = str(path.relative_to(root))
-    if linked_directory:
-        yield Hit(mask(location, forms), "directory symlink, not scanned")
-        return
-    yield from _name_hits(location, location, forms)
-    yield from _content_hits(location, path.read_bytes(), forms, 0)
-
-
-def scan(root: Path, secret: str) -> tuple[int, tuple[Hit, ...]]:
-    forms: Final = secret_forms(secret)
-    entries: Final = tuple(_walk(root))
-    hits: Final = tuple(chain.from_iterable(_path_hits(root, path, linked, forms) for path, linked in entries))
-    return len(entries), hits
-
-
-def main(argv: tuple[str, ...], environ: Mapping[str, str], out: TextIO) -> int:
-    if len(argv) != 1:
-        print(f"usage: assert_secret_absent.py <directory>, with the secret in ${SECRET_VARIABLE}", file=out)
-        return 2
-    secret: Final = environ.get(SECRET_VARIABLE, "")
-    root: Final = Path(argv[0])
-    if not secret:
-        print(f"{SECRET_VARIABLE} is empty, nothing to scan for", file=out)
-        return 0
-    if not root.is_dir():
-        print(f"{root} is not a directory, nothing to scan", file=out)
-        return 0
-    try:
-        scanned, hits = scan(root, secret)
-    except Exception as error:
-        print(f"scan failed with {type(error).__name__}, treating the results as unsafe", file=out)
-        return 1
-    for hit in hits:
-        print(f"secret found: {hit.location} ({hit.form})", file=out)
-    if hits:
-        print(f"{len(hits)} copies of the secret under {root}, refusing to upload", file=out)
-        return 1
-    print(f"no copy of the secret in {scanned} files under {root}", file=out)
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main(tuple(sys.argv[1:]), os.environ, sys.stdout))

.github/scripts/auto_update_price_and_context_window_file.py

@@ -0,0 +1,319 @@
+import asyncio
+import aiohttp
+import json
+import math
+from typing import Any
+
+# Asynchronously fetch data from a given URL
+async def fetch_data(url):
+    try:
+        # Create an asynchronous session
+        async with aiohttp.ClientSession() as session:
+            # Send a GET request to the URL
+            async with session.get(url) as resp:
+                # Raise an error if the response status is not OK
+                resp.raise_for_status()
+                # Parse the response JSON
+                resp_json = await resp.json()
+                print("Fetch the data from URL.")
+                # Return the 'data' field from the JSON response
+                return resp_json['data']
+    except Exception as e:
+        # Print an error message if fetching data fails
+        print("Error fetching data from URL:", e)
+        return None
+
+
+FRIENDLI_API_URL = "https://api.friendli.ai/serverless/v1/models"
+FRIENDLI_PROVIDER = "friendliai"
+
+INHERITABLE_BASE_KEYS = (
+    "supports_pdf_input",
+    "supports_assistant_prefill",
+    "supports_adaptive_thinking",
+    "supports_output_config",
+)
+
+REASONING_EFFORT_LEVEL_ORDER = ("none", "minimal", "low", "medium", "high", "xhigh", "max")
+
+
+def _find_base_model_entry(base_model: str, local_data: dict) -> str | None:
+    if not base_model:
+        return None
+    bm_tail = base_model.split("/")[-1].lower()
+    if base_model in local_data:
+        return base_model
+    for key in local_data:
+        if key.startswith("sample_spec") or key == "fallback_generalizations":
+            continue
+        if key.split("/")[-1].lower() == bm_tail:
+            return key
+    return None
+
+
+def _reasoning_effort_levels(reasoning_options: list) -> list:
+    offered = {
+        val
+        for opt in reasoning_options or []
+        if opt.get("type") == "effort"
+        for val in opt.get("values", [])
+    }
+    return [level for level in REASONING_EFFORT_LEVEL_ORDER if level in offered]
+
+
+def _valid_token_price(value: object) -> bool:
+    try:
+        price = float(value)  # pyright: ignore[reportArgumentType]  # non-numeric values are rejected via the except
+    except (TypeError, ValueError):
+        return False
+    return math.isfinite(price) and price >= 0
+
+
+def _has_valid_token_prices(pricing: dict | None) -> bool:
+    prices = pricing or {}
+    return _valid_token_price(prices.get("input")) and _valid_token_price(prices.get("output"))
+
+
+def _pricing(pricing: dict) -> dict:
+    out: dict[str, Any] = {}
+    if not pricing:
+        return out
+    if "input" in pricing:
+        out["input_cost_per_token"] = float(pricing["input"])
+    if "output" in pricing:
+        out["output_cost_per_token"] = float(pricing["output"])
+    if "input_cache_read" in pricing and pricing["input_cache_read"] is not None:
+        out["cache_read_input_token_cost"] = float(pricing["input_cache_read"])
+    return out
+
+
+def _modality_flags(input_mods: list) -> dict:
+    mods = input_mods or []
+    has_image = "image" in mods
+    return {
+        "supports_vision": has_image,
+        "supports_image_input": has_image,
+        "supports_video_input": "video" in mods,
+    }
+
+
+def transform_friendli_data(data: list, local_data: dict) -> dict:
+    transformed: dict[str, dict] = {}
+    if not data:
+        return transformed
+    for model in data:
+        # An unpriced row must never wholesale-replace an already priced local entry:
+        # missing prices cost-calculate as zero, silently zeroing tracked spend
+        if not _has_valid_token_prices(model.get("pricing")):
+            continue
+        model_id = model["id"]
+        base_model = model.get("base_model") or ""
+        entry: dict[str, Any] = {
+            "litellm_provider": FRIENDLI_PROVIDER,
+        }
+
+        base_key = _find_base_model_entry(base_model, local_data)
+        if base_key:
+            base_entry = local_data[base_key]
+            for k in INHERITABLE_BASE_KEYS:
+                if k in base_entry:
+                    entry[k] = base_entry[k]
+
+        ctx = model.get("context_length")
+        if ctx is not None:
+            entry["max_input_tokens"] = int(ctx)
+        max_out = model.get("max_completion_tokens")
+        if max_out is not None:
+            entry["max_output_tokens"] = int(max_out)
+            entry["max_tokens"] = int(max_out)
+
+        pricing = _pricing(model.get("pricing", {}))
+        entry.update(pricing)
+        entry["supports_prompt_caching"] = "cache_read_input_token_cost" in pricing
+
+        reasoning = model.get("reasoning") is True
+        entry["supports_reasoning"] = reasoning
+        if reasoning:
+            entry["reasoning_effort_levels"] = _reasoning_effort_levels(
+                model.get("reasoning_options", [])
+            )
+
+        func = model.get("functionality", {})
+        entry["supports_function_calling"] = func.get("tool_call") is True
+        entry["supports_parallel_function_calling"] = func.get("parallel_tool_call") is True
+        is_struct = func.get("structured_output") is True
+        entry["supports_response_schema"] = is_struct
+        entry["supports_native_structured_output"] = is_struct
+        entry["supports_system_messages"] = func.get("system_messages") is True
+        entry["supports_tool_choice"] = func.get("tool_choice") is True
+
+        entry.update(_modality_flags(model.get("input_modalities", [])))
+
+        entry["mode"] = model.get("mode", "chat")
+
+        desc = model.get("description")
+        if desc:
+            entry["comment"] = desc
+
+        dep = model.get("deprecation_date")
+        if dep:
+            entry["deprecation_date"] = dep.split("T")[0]
+
+        entry["source"] = FRIENDLI_API_URL
+
+        transformed[f"{FRIENDLI_PROVIDER}/{model_id}"] = entry
+    return transformed
+
+# Synchronize local data with remote data
+def sync_local_data_with_remote(local_data, remote_data, replace_keys=frozenset()):
+    # Update existing keys in local_data with values from remote_data
+    # (replace_keys entries are swapped wholesale so a field the remote catalog
+    #  dropped, e.g. cache pricing, cannot survive as a stale value)
+    for key in (set(local_data) & set(remote_data)):
+        if key in replace_keys:
+            local_data[key] = remote_data[key]
+        else:
+            local_data[key].update(remote_data[key])
+
+    # Add new keys from remote_data to local_data
+    for key in (set(remote_data) - set(local_data)):
+        local_data[key] = remote_data[key]
+
+# Write data to the json file
+def write_to_file(file_path, data):
+    try:
+        # Open the file in write mode
+        with open(file_path, "w") as file:
+            # Dump the data as JSON into the file
+            json.dump(data, file, indent=4)
+        print("Values updated successfully.")
+    except Exception as e:
+        # Print an error message if writing to file fails
+        print("Error updating JSON file:", e)
+
+# Update the existing models and add the missing models for OpenRouter
+def transform_openrouter_data(data):
+    transformed = {}
+    if not data:
+        return transformed
+    for row in data:
+        # Add the fields 'max_tokens' and 'input_cost_per_token'
+        obj = {
+            "max_tokens": row["context_length"],
+            "input_cost_per_token": float(row["pricing"]["prompt"]),
+        }
+
+        # Add 'max_output_tokens' as a field if it is not None
+        if "top_provider" in row and "max_completion_tokens" in row["top_provider"] and row["top_provider"]["max_completion_tokens"] is not None:
+            obj['max_output_tokens'] = int(row["top_provider"]["max_completion_tokens"])
+
+        # Add the field 'output_cost_per_token'
+        obj.update({
+            "output_cost_per_token": float(row["pricing"]["completion"]),
+        })
+
+        # Add field 'input_cost_per_image' if it exists and is non-zero
+        if "pricing" in row and "image" in row["pricing"] and float(row["pricing"]["image"]) != 0.0:
+            obj['input_cost_per_image'] = float(row["pricing"]["image"])
+
+        # Add the fields 'litellm_provider' and 'mode'
+        obj.update({
+            "litellm_provider": "openrouter",
+            "mode": "chat"
+        })
+
+        # Add the 'supports_vision' field if the modality is 'multimodal'
+        if row.get('architecture', {}).get('modality') == 'multimodal':
+            obj['supports_vision'] = True
+
+        # Use a composite key to store the transformed object
+        transformed[f'openrouter/{row["id"]}'] = obj
+
+    return transformed
+
+# Update the existing models and add the missing models for Vercel AI Gateway
+def transform_vercel_ai_gateway_data(data):
+    transformed = {}
+    if not data:
+        return transformed
+    for row in data:
+        # Rows without token pricing or token limits (video/embedding models) previously KeyError'd the whole sync
+        if any(row.get(k) is None for k in ("context_window", "max_tokens")) or any(
+            row.get("pricing", {}).get(k) is None for k in ("input", "output")
+        ):
+            continue
+        obj = {
+            "max_tokens": row["context_window"],
+            "input_cost_per_token": float(row["pricing"]["input"]),
+            "output_cost_per_token": float(row["pricing"]["output"]),
+            'max_output_tokens': row['max_tokens'],
+            'max_input_tokens': row["context_window"],
+        }
+
+        # Handle cache pricing if available
+        if "pricing" in row:
+            if "input_cache_read" in row["pricing"] and row["pricing"]["input_cache_read"] is not None:
+                obj['cache_read_input_token_cost'] = float(f"{float(row['pricing']['input_cache_read']):e}")
+            
+            if "input_cache_write" in row["pricing"] and row["pricing"]["input_cache_write"] is not None:
+                obj['cache_creation_input_token_cost'] = float(f"{float(row['pricing']['input_cache_write']):e}")
+
+        mode = "embedding" if "embedding" in row["id"].lower() else "chat"
+        
+        obj.update({"litellm_provider": "vercel_ai_gateway", "mode": mode})
+
+        transformed[f'vercel_ai_gateway/{row["id"]}'] = obj
+
+    return transformed
+
+
+# Load local data from a specified file
+def load_local_data(file_path):
+    try:
+        # Open the file in read mode
+        with open(file_path, "r") as file:
+            # Load and return the JSON data
+            return json.load(file)
+    except FileNotFoundError:
+        # Print an error message if the file is not found
+        print("File not found:", file_path)
+        return None
+    except json.JSONDecodeError as e:
+        # Print an error message if JSON decoding fails
+        print("Error decoding JSON:", e)
+        return None
+
+def main():
+    local_file_path = "model_prices_and_context_window.json"  # Path to the local data file
+    openrouter_url = "https://openrouter.ai/api/v1/models"  # URL to fetch OpenRouter data
+    vercel_ai_gateway_url = "https://ai-gateway.vercel.sh/v1/models"  # URL to fetch Vercel AI Gateway data
+
+    # Load local data from file
+    local_data = load_local_data(local_file_path)
+    
+    # Fetch OpenRouter data
+    openrouter_data = asyncio.run(fetch_data(openrouter_url))
+    # Transform the fetched OpenRouter data
+    openrouter_data = transform_openrouter_data(openrouter_data)
+    
+    # Fetch Vercel AI Gateway data
+    vercel_data = asyncio.run(fetch_data(vercel_ai_gateway_url))
+    # Transform the fetched Vercel AI Gateway data
+    vercel_data = transform_vercel_ai_gateway_data(vercel_data)
+
+    friendli_data = asyncio.run(fetch_data(FRIENDLI_API_URL))
+    friendli_data = transform_friendli_data(friendli_data, local_data)
+    
+    # Combine both datasets
+    all_remote_data = {**openrouter_data, **vercel_data, **friendli_data}
+
+    # If both local and openrouter data are available, synchronize and save
+    if local_data and all_remote_data:
+        sync_local_data_with_remote(local_data, all_remote_data, replace_keys=frozenset(friendli_data))
+        write_to_file(local_file_path, local_data)
+    else:
+        print("Failed to fetch model data from either local file or URL.")
+
+# Entry point of the script
+if __name__ == "__main__":
+    main()

.github/scripts/read_rc_version.py

@@ -1,44 +0,0 @@
-#!/usr/bin/env python3
-"""Print `version=X.Y.0` from [project].version in pyproject.toml for $GITHUB_OUTPUT.
-
-Usage
------
-    python3 read_rc_version.py [path/to/pyproject.toml] >> "$GITHUB_OUTPUT"
-
-Exit code 1 with a `::error::` line on stderr when the version is not an X.Y.0 release.
-"""
-
-from __future__ import annotations
-
-import pathlib
-import re
-import sys
-from typing import Final
-
-if sys.version_info >= (3, 11):
-    import tomllib
-else:
-    import tomli as tomllib
-
-RELEASE_VERSION: Final = re.compile(r"[0-9]+\.[0-9]+\.0")
-
-
-def read_version(pyproject: pathlib.Path) -> str:
-    with pyproject.open("rb") as f:
-        return tomllib.load(f)["project"]["version"]
-
-
-def main(argv: list[str]) -> int:
-    pyproject: Final = pathlib.Path(argv[1]) if len(argv) > 1 else pathlib.Path("pyproject.toml")
-    version: Final = read_version(pyproject)
-    if RELEASE_VERSION.fullmatch(version) is None:
-        print(  # noqa: T201  # the ::error:: line to stderr is the workflow's failure signal
-            f"::error::pyproject.toml version {version} is not an X.Y.0 release version", file=sys.stderr
-        )
-        return 1
-    print(f"version={version}")  # noqa: T201  # stdout line is appended to $GITHUB_OUTPUT
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv))

.github/scripts/run_merge_smoke.py

@@ -1,355 +0,0 @@
-#!/usr/bin/env python3
-"""Merge smoke harness: bounded checks run inside a loopback-only Linux network namespace."""
-
-# ruff: noqa: T201  # CLI harness: stdout/stderr lines are the reported result
-
-from __future__ import annotations
-
-import argparse
-import contextlib
-import http.client
-import json
-import os
-import secrets
-import signal
-import socket
-import subprocess
-import sys
-import time
-from collections.abc import Sequence
-from dataclasses import dataclass
-from pathlib import Path
-from typing import Final, NoReturn, TextIO, cast
-
-
-@dataclass(frozen=True, slots=True)
-class CheckResult:
-    ok: bool
-    detail: str = ""
-
-
-@dataclass(slots=True)
-class _Args:
-    command: str = ""
-    no_child: bool = False
-    expect: str = ""
-    litellm_bin: str | None = None
-    lite_bin: str | None = None
-    diagnostics_dir: str = ""
-    ready_deadline: float = 120.0
-    shutdown_deadline: float = 20.0
-    poll_interval: float = 0.5
-
-
-def fail(reason: str) -> NoReturn:
-    print(f"merge-smoke: FAIL {reason}", file=sys.stderr)
-    sys.exit(1)
-
-
-def ok(step: str) -> None:
-    print(f"merge-smoke: OK {step}")
-
-
-def tail(path: Path, lines: int = 20) -> str:
-    try:
-        return "\n".join(path.read_text(errors="replace").splitlines()[-lines:])
-    except OSError as exc:
-        return f"<cannot read {path}: {exc}>"
-
-
-def cmd_verify_isolation(args: _Args) -> int:
-    if os.geteuid() == 0:
-        fail("verify-isolation must run unprivileged (geteuid()==0)")
-    try:
-        socket.create_connection(("192.0.2.1", 9), timeout=3)
-    except OSError as exc:
-        print(f"external connect blocked as expected: errno={exc.errno} {exc}")
-    else:
-        fail("external TCP connect to 192.0.2.1:9 succeeded; namespace is not isolated")
-    listener: Final = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-    listener.bind(("127.0.0.1", 0))
-    listener.listen(1)
-    port: Final = cast(int, listener.getsockname()[1])
-    client: Final = socket.create_connection(("127.0.0.1", port), timeout=5)
-    accepted: Final = listener.accept()
-    accepted[0].close()
-    client.close()
-    listener.close()
-    print(f"loopback connect ok on 127.0.0.1:{port}")
-    if not args.no_child:
-        proc: Final = subprocess.run(
-            [sys.executable, str(Path(__file__).resolve()), "verify-isolation", "--no-child"],
-            timeout=30,
-            capture_output=True,
-            text=True,
-        )
-        if proc.returncode != 0:
-            fail(f"child process did not inherit isolation: {proc.stderr.strip()}")
-        print("child process inherits isolation")
-    ok("verify-isolation")
-    return 0
-
-
-def cmd_interpreter(args: _Args) -> int:
-    print(sys.version)
-    print(sys.executable)
-    actual: Final = f"{sys.version_info.major}.{sys.version_info.minor}"
-    if actual != args.expect:
-        fail(f"interpreter is {actual}, expected {args.expect}")
-    ok(f"interpreter {actual}")
-    return 0
-
-
-def _run_cli(argv: Sequence[str], label: str) -> CheckResult:
-    try:
-        proc: Final = subprocess.run(list(argv), timeout=120, capture_output=True, text=True)
-    except subprocess.TimeoutExpired:
-        return CheckResult(ok=False, detail=f"{label} timed out after 120s")
-    sys.stdout.write(proc.stdout)
-    sys.stderr.write(proc.stderr)
-    if proc.returncode != 0:
-        return CheckResult(ok=False, detail=f"{label} exited {proc.returncode}")
-    return CheckResult(ok=True)
-
-
-def cmd_cli(args: _Args) -> int:
-    venv_bin: Final = Path(sys.executable).parent
-    litellm_bin: Final = Path(args.litellm_bin) if args.litellm_bin else venv_bin / "litellm"
-    lite_bin: Final = Path(args.lite_bin) if args.lite_bin else venv_bin / "lite"
-    commands: Final = (
-        ("import litellm", [sys.executable, "-c", "import litellm"]),
-        ("litellm --version", [str(litellm_bin), "--version"]),
-        ("lite version", [str(lite_bin), "version"]),
-    )
-    for label, argv in commands:
-        result = _run_cli(argv, label)
-        if not result.ok:
-            fail(result.detail)
-        ok(label)
-    return 0
-
-
-def _free_port() -> int:
-    sock: Final = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-    sock.bind(("127.0.0.1", 0))
-    port: Final = cast(int, sock.getsockname()[1])
-    sock.close()
-    return port
-
-
-_CONFIG_TEMPLATE: Final = """model_list:
-  - model_name: smoke-model
-    litellm_params:
-      model: openai/smoke-model
-      api_base: http://127.0.0.1:9/v1
-      api_key: synthetic-key
-general_settings:
-  master_key: os.environ/LITELLM_MASTER_KEY
-"""
-
-
-def _listen_inode(port: int) -> str | None:
-    target: Final = f"{port:04X}"
-    for table in ("/proc/net/tcp", "/proc/net/tcp6"):
-        try:
-            rows = Path(table).read_text().splitlines()[1:]
-        except OSError:
-            continue
-        for row in rows:
-            cols = row.split()
-            if len(cols) > 9 and cols[3] == "0A" and cols[1].rsplit(":", 1)[-1] == target:
-                return cols[9]
-    return None
-
-
-def _ancestors(pid: int) -> frozenset[int]:
-    chain: Final[set[int]] = set()
-    pending: Final[list[int]] = [pid]
-    while pending:
-        current = pending.pop()
-        if current <= 0 or current in chain:
-            continue
-        chain.add(current)
-        try:
-            stat = Path(f"/proc/{current}/stat").read_text()
-        except OSError:
-            continue
-        pending.append(int(stat.rpartition(")")[2].split()[1]))
-    return frozenset(chain)
-
-
-def _socket_owner_pid(inode: str) -> int | None:
-    for proc_dir in Path("/proc").iterdir():
-        if not proc_dir.name.isdigit():
-            continue
-        fd_dir = proc_dir / "fd"
-        try:
-            for fd in fd_dir.iterdir():
-                try:
-                    if os.readlink(fd) == f"socket:[{inode}]":
-                        return int(proc_dir.name)
-                except OSError:
-                    continue
-        except OSError:
-            continue
-    return None
-
-
-def _verify_port_owner(port: int, proc: subprocess.Popen[bytes]) -> CheckResult:
-    inode: Final = _listen_inode(port)
-    if inode is None:
-        return CheckResult(ok=False, detail=f"no LISTEN socket found for port {port} in /proc/net/tcp")
-    owner: Final = _socket_owner_pid(inode)
-    if owner is None:
-        return CheckResult(ok=False, detail=f"no process owns the listen socket inode {inode} for port {port}")
-    if owner != proc.pid and proc.pid not in _ancestors(owner):
-        return CheckResult(
-            ok=False, detail=f"port {port} owned by pid {owner} outside the launched process group {proc.pid}"
-        )
-    if proc.poll() is not None:
-        return CheckResult(ok=False, detail=f"proxy exited with code {proc.returncode} after readiness")
-    return CheckResult(ok=True)
-
-
-def cmd_proxy_startup(args: _Args) -> int:
-    diagnostics: Final = Path(args.diagnostics_dir)
-    diagnostics.mkdir(parents=True, exist_ok=True)
-    venv_bin: Final = Path(sys.executable).parent
-    litellm_bin: Final = Path(args.litellm_bin) if args.litellm_bin else venv_bin / "litellm"
-    port: Final = _free_port()
-    master_key: Final = "sk-smoke-" + secrets.token_hex(16)
-    config_path: Final = diagnostics / "config.yaml"
-    config_path.write_text(_CONFIG_TEMPLATE)
-    log_path: Final = diagnostics / "proxy.log"
-    result_path: Final = diagnostics / "result.json"
-    outcome: Final[dict[str, object]] = {
-        "port": port,
-        "time_to_ready_s": None,
-        "shutdown_s": None,
-        "readiness": None,
-        "outcome": "failed",
-    }
-    log_file: Final = log_path.open("w")
-    env: Final = {
-        **os.environ,
-        "LITELLM_MASTER_KEY": master_key,
-        "LITELLM_LOCAL_MODEL_COST_MAP": "True",
-    }
-    started: Final = time.monotonic()
-    proc: Final = subprocess.Popen(
-        [str(litellm_bin), "--config", str(config_path), "--host", "127.0.0.1", "--port", str(port)],
-        stdout=log_file,
-        stderr=subprocess.STDOUT,
-        start_new_session=True,
-        env=env,
-    )
-    body: str | None = None
-    last_status: int | None = None
-    while time.monotonic() - started < args.ready_deadline:
-        if proc.poll() is not None:
-            log_file.close()
-            result_path.write_text(json.dumps(outcome))
-            fail(f"proxy exited early with code {proc.returncode}\n{tail(log_path)}")
-        try:
-            conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
-            conn.request("GET", "/health/readiness")
-            resp = conn.getresponse()
-            last_status = resp.status
-            candidate = resp.read().decode()
-            conn.close()
-        except (http.client.HTTPException, ConnectionError, OSError):
-            time.sleep(args.poll_interval)
-            continue
-        if last_status == 200:
-            body = candidate
-            break
-        time.sleep(args.poll_interval)
-    outcome["time_to_ready_s"] = round(time.monotonic() - started, 3)
-    if body is None:
-        _terminate(proc, log_file)
-        result_path.write_text(json.dumps(outcome))
-        detail = f"last status {last_status}" if last_status is not None else "no response"
-        fail(f"readiness not reached within {args.ready_deadline}s ({detail})\n{tail(log_path)}")
-    outcome["readiness"] = body
-    try:
-        readiness = cast(object, json.loads(body))
-    except json.JSONDecodeError:
-        readiness = None
-    if readiness != {"status": "healthy", "db": "Not connected"}:
-        _terminate(proc, log_file)
-        result_path.write_text(json.dumps(outcome))
-        fail(f"unexpected readiness body: {body}")
-    owner_check: Final = _verify_port_owner(port, proc)
-    if not owner_check.ok:
-        _terminate(proc, log_file)
-        result_path.write_text(json.dumps(outcome))
-        fail(owner_check.detail)
-    shutdown_started: Final = time.monotonic()
-    os.killpg(proc.pid, signal.SIGTERM)
-    try:
-        proc.wait(timeout=args.shutdown_deadline)
-    except subprocess.TimeoutExpired:
-        os.killpg(proc.pid, signal.SIGKILL)
-        proc.wait(timeout=10)
-        outcome["shutdown_s"] = round(time.monotonic() - shutdown_started, 3)
-        log_file.close()
-        result_path.write_text(json.dumps(outcome))
-        fail(f"forced kill after {args.shutdown_deadline}s\n{tail(log_path)}")
-    outcome["shutdown_s"] = round(time.monotonic() - shutdown_started, 3)
-    try:
-        os.killpg(proc.pid, 0)
-    except ProcessLookupError:
-        pass
-    else:
-        os.killpg(proc.pid, signal.SIGKILL)
-        log_file.close()
-        result_path.write_text(json.dumps(outcome))
-        fail("process group survived SIGTERM")
-    log_file.close()
-    outcome["outcome"] = "ok"
-    result_path.write_text(json.dumps(outcome))
-    ok(f"proxy-startup ready={outcome['time_to_ready_s']}s shutdown={outcome['shutdown_s']}s")
-    return 0
-
-
-def _terminate(proc: subprocess.Popen[bytes], log_file: TextIO) -> None:
-    with contextlib.suppress(ProcessLookupError):
-        os.killpg(proc.pid, signal.SIGTERM)
-    try:
-        proc.wait(timeout=10)
-    except subprocess.TimeoutExpired:
-        with contextlib.suppress(ProcessLookupError):
-            os.killpg(proc.pid, signal.SIGKILL)
-        with contextlib.suppress(subprocess.TimeoutExpired):
-            proc.wait(timeout=10)
-    log_file.close()
-
-
-def main() -> int:
-    parser: Final = argparse.ArgumentParser(description=__doc__)
-    subs: Final = parser.add_subparsers(dest="command", required=True)
-    p_iso: Final = subs.add_parser("verify-isolation")
-    p_iso.add_argument("--no-child", action="store_true")
-    p_interp: Final = subs.add_parser("interpreter")
-    p_interp.add_argument("--expect", required=True)
-    p_cli: Final = subs.add_parser("cli")
-    p_cli.add_argument("--litellm-bin", default=None)
-    p_cli.add_argument("--lite-bin", default=None)
-    p_proxy: Final = subs.add_parser("proxy-startup")
-    p_proxy.add_argument("--diagnostics-dir", required=True)
-    p_proxy.add_argument("--litellm-bin", default=None)
-    p_proxy.add_argument("--ready-deadline", type=float, default=120)
-    p_proxy.add_argument("--shutdown-deadline", type=float, default=20)
-    p_proxy.add_argument("--poll-interval", type=float, default=0.5)
-    args: Final = parser.parse_args(namespace=_Args())
-    handlers: Final = {
-        "verify-isolation": cmd_verify_isolation,
-        "interpreter": cmd_interpreter,
-        "cli": cmd_cli,
-        "proxy-startup": cmd_proxy_startup,
-    }
-    return handlers[args.command](args)
-
-
-if __name__ == "__main__":
-    sys.exit(main())

.github/scripts/smoke_test_native_wheel.py

@@ -7,8 +7,6 @@ import zipfile
 from pathlib import Path
 from typing import Final
 
-COST_MAP: Final = Path(__file__).resolve().parents[2] / "model_prices_and_context_window.json"
-
 CHILD_SCRIPT: Final = """
 from importlib.util import module_from_spec, spec_from_file_location
 from pathlib import Path
@@ -36,9 +34,6 @@ else:
 after = module.gil_stats()
 if not isinstance(after.get("releases"), int):
     raise AssertionError(f"native module unusable after panic: {after!r}")
-
-if module.bundled_model_cost_map() != Path(sys.argv[2]).read_bytes():
-    raise AssertionError("the wheel embeds a different model_prices_and_context_window.json than this checkout")
 """
 
 
@@ -60,9 +55,7 @@ def main() -> int:
 
         native_path: Final = Path(temporary_directory) / Path(native_members[0].filename).name
         native_path.write_bytes(archive.read(native_members[0]))
-        result: Final = subprocess.run(
-            (sys.executable, "-c", CHILD_SCRIPT, str(native_path), str(COST_MAP)), check=False
-        )
+        result: Final = subprocess.run((sys.executable, "-c", CHILD_SCRIPT, str(native_path)), check=False)
 
     if result.returncode != 0:
         sys.stderr.write(f"native wheel smoke test exited with status {result.returncode}\n")

.github/scripts/verify_linux_native_wheel.py

@@ -134,16 +134,7 @@ def main(
         uncompressed_wheel_size: Final = sum(member.file_size for member in wheel_members)
         native_path: Final = wheel.parent / "native" / Path(native_member.filename).name
         native_path.parent.mkdir(parents=True, exist_ok=True)
-        native_bytes: Final = archive.read(native_member)
-        native_path.write_bytes(native_bytes)
-        duplicated_vocabularies: Final = tuple(
-            member.filename
-            for member in wheel_members
-            if member.filename.startswith("litellm/litellm_core_utils/tokenizers/")
-            and re.fullmatch(r"[0-9a-f]{40}", PurePosixPath(member.filename).name)
-            and member.file_size > 0
-            and archive.read(member) in native_bytes
-        )
+        native_path.write_bytes(archive.read(native_member))
 
     wheel_metadata_tags_match: Final = (
         len(wheel_metadata_tags) == len(expanded_filename_tags)
@@ -214,7 +205,7 @@ def main(
     native_module: Final = load_native_module(native_path)
     native_module_loads: Final = native_module is not None
     panic_test_hook_absent: Final = native_module is not None and not hasattr(native_module, "_panic_for_test")
-    native_size_limit: Final = 48_000_000
+    native_size_limit: Final = 25_000_000
     native_size_within_limit: Final = native_member.file_size <= native_size_limit
     validations: Final = (
         (f"Python tag is {EXPECTED_PYTHON_TAG}", python_tag == EXPECTED_PYTHON_TAG),
@@ -231,8 +222,7 @@ def main(
         ("Python extension entry point is present", extension_entry_point_present),
         ("Native module loads", native_module_loads),
         ("Production module omits the panic test hook", panic_test_hook_absent),
-        (f"Native extension does not exceed {native_size_limit / 1_000_000:.0f} MB", native_size_within_limit),
-        ("Tokenizer vocabularies are not duplicated in the native extension", not duplicated_vocabularies),
+        ("Native extension does not exceed 25 MB", native_size_within_limit),
         ("Wheel contents are valid", not unexpected_members),
     )
 
@@ -277,8 +267,7 @@ def main(
             ),
             (
                 not native_size_within_limit,
-                f"native extension exceeds {native_size_limit / 1_000_000:.0f} MB: "
-                f"{native_member.file_size / 1_000_000:.2f} MB",
+                f"native extension exceeds 20 MB: {native_member.file_size / 1_000_000:.2f} MB",
             ),
             (bool(unexpected_members), f"wheel contains unexpected build artifacts: {', '.join(unexpected_members)}"),
         )

.github/template.yaml

@@ -21,7 +21,7 @@ Parameters:
   WorkerConfigParameter:
     Type: String
     Description: Sample environment variable
-    Default: '{"model": null, "alias": null, "api_base": null, "api_version": "2023-07-01-preview", "debug": false, "temperature": null, "max_tokens": null, "request_timeout": 600, "max_budget": null, "drop_params": false, "add_function_to_prompt": false, "headers": null, "save": false, "config": null, "use_queue": false}'
+    Default: '{"model": null, "alias": null, "api_base": null, "api_version": "2023-07-01-preview", "debug": false, "temperature": null, "max_tokens": null, "request_timeout": 600, "max_budget": null, "telemetry": true, "drop_params": false, "add_function_to_prompt": false, "headers": null, "save": false, "config": null, "use_queue": false}'
 
 Resources:
   MyUrlFunctionPermissions:

.github/workflows/_test-unit-base.yml

@@ -0,0 +1,247 @@
+name: _Unit Test Base (Reusable)
+
+on:
+  workflow_call:
+    inputs:
+      test-path:
+        description: "Pytest path(s) to run"
+        required: true
+        type: string
+      workers:
+        description: "Number of pytest-xdist workers"
+        required: false
+        type: number
+        default: 2
+      reruns:
+        description: "Number of reruns for flaky tests"
+        required: false
+        type: number
+        default: 2
+      timeout-minutes:
+        description: >-
+          Timeout for the test step alone. Setup (checkout, dependency install,
+          Prisma client generation) gets its own allowance on top, so a slow
+          runner or a cold binary download can never cancel passing tests.
+        required: false
+        type: number
+        default: 20
+      job-timeout-minutes:
+        description: >-
+          Backstop for the whole job. Keep it >= `timeout-minutes` plus 40: 35 for
+          the per-step ceilings on the setup steps below, and 5 for the runner
+          overhead the job clock charges but no step owns (job init, step
+          transitions, post-job cleanup). That headroom is what makes the test
+          budget a floor rather than a hope, since setup cannot overrun into it
+          without failing its own step first. GitHub expressions have no
+          arithmetic, so the sum is passed in rather than computed.
+        required: false
+        type: number
+        default: 60
+      test-timeout-seconds:
+        description: >-
+          Per-test ceiling enforced by pytest-timeout, covering fixture setup and
+          teardown as well as the test body. A test that hangs fails with a
+          traceback of where it was stuck instead of idling the shard until
+          `timeout-minutes` cancels it. Timed-out tests are excluded from reruns
+          because pytest-timeout arms its timer once per test and
+          pytest-rerunfailures reruns inside that same window, so a rerun of a
+          timed-out test would run with no timer at all.
+        required: false
+        type: number
+        default: 120
+      max-failures:
+        description: "Stop after this many failures"
+        required: false
+        type: number
+        default: 10
+      dist:
+        description: "pytest-xdist distribution mode (loadscope|load|worksteal|loadfile|no)"
+        required: false
+        type: string
+        default: "loadscope"
+      artifact-name:
+        description: "Unique name for the coverage artifact (must be unique per run)"
+        required: true
+        type: string
+      legacy-mcp-peer:
+        description: "Install the isolated SDK1 peer for MCP compatibility tests"
+        required: false
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+
+env:
+  UV_PYTHON: "3.12"
+  LITELLM_LOCAL_MODEL_COST_MAP: "True"
+
+jobs:
+  run:
+    name: Run tests
+    runs-on: ubuntu-latest
+    timeout-minutes: ${{ inputs.job-timeout-minutes }}
+    permissions:
+      contents: read
+      pull-requests: read
+    outputs:
+      decision: ${{ steps.changes.outputs.decision }}
+
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        timeout-minutes: 3
+        with:
+          persist-credentials: false
+
+      - name: Detect relevant changes
+        id: changes
+        timeout-minutes: 2
+        uses: ./.github/actions/detect-changes
+
+      - name: Set up Python
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 3
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: ${{ env.UV_PYTHON }}
+
+      - name: Set up uv
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 3
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Cache uv dependencies
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 5
+        uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+        with:
+          path: ${{ env.UV_CACHE_DIR }}
+          key: ${{ runner.os }}-uv-downloads-py${{ env.UV_PYTHON }}-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-uv-downloads-py${{ env.UV_PYTHON }}-
+
+      - name: Cache the Rust build
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 5
+        uses: ./.github/actions/cache-cargo-build
+
+      - name: Install dependencies
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 8
+        env:
+          LEGACY_MCP_PEER: ${{ inputs.legacy-mcp-peer }}
+        run: |
+          diff -u model_prices_and_context_window.json litellm/model_prices_and_context_window_backup.json
+          .github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router --extra saml
+          uv run --no-sync python -c 'import os, sys; print(sys.version); assert f"{sys.version_info.major}.{sys.version_info.minor}" == os.environ["UV_PYTHON"]'
+          if [ "$LEGACY_MCP_PEER" = "true" ]; then
+            uv venv --python "${UV_PYTHON}" .venv-mcp-peer
+            uv pip install --python .venv-mcp-peer 'mcp==1.28.1' 'langchain-mcp-adapters==0.2.1'
+            echo "MCP_TEST_PEER_PYTHON=$GITHUB_WORKSPACE/.venv-mcp-peer/bin/python" >> "$GITHUB_ENV"
+          fi
+
+      - name: Cache Prisma binaries
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 3
+        uses: ./.github/actions/cache-prisma-binaries
+
+      - name: Generate Prisma client
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 3
+        run: |
+          uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma
+
+      - name: Run tests
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: ${{ inputs.timeout-minutes }}
+        env:
+          TEST_PATH: ${{ inputs.test-path }}
+          MAX_FAILURES: ${{ inputs.max-failures }}
+          WORKERS: ${{ inputs.workers }}
+          RERUNS: ${{ inputs.reruns }}
+          TEST_TIMEOUT_SECONDS: ${{ inputs.test-timeout-seconds }}
+          DIST: ${{ inputs.dist }}
+          COVERAGE_CORE: sysmon
+        run: |
+          if [ "${WORKERS}" = "0" ]; then
+            uv run --no-sync pytest ${TEST_PATH:?} \
+              --tb=short -vv \
+              --maxfail="${MAX_FAILURES}" \
+              --reruns "${RERUNS}" \
+              --reruns-delay 1 \
+              --timeout="${TEST_TIMEOUT_SECONDS}" \
+              --rerun-except "from pytest-timeout" \
+              --durations=20 \
+              --cov=./litellm --cov=./enterprise/litellm_enterprise \
+              --cov-report=xml:coverage.xml \
+              --cov-config=pyproject.toml
+          else
+            uv run --no-sync pytest ${TEST_PATH:?} \
+              --tb=short -vv \
+              --maxfail="${MAX_FAILURES}" \
+              -n "${WORKERS}" \
+              --reruns "${RERUNS}" \
+              --reruns-delay 1 \
+              --timeout="${TEST_TIMEOUT_SECONDS}" \
+              --rerun-except "from pytest-timeout" \
+              --dist="${DIST}" \
+              --durations=20 \
+              --cov=./litellm --cov=./enterprise/litellm_enterprise \
+              --cov-report=xml:coverage.xml \
+              --cov-config=pyproject.toml
+          fi
+
+      - name: Save coverage report
+        if: always() && steps.changes.outputs.decision != 'skip'
+        uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
+        with:
+          name: coverage-${{ inputs.artifact-name }}-${{ github.run_id }}-${{ github.run_attempt }}
+          path: coverage.xml
+          retention-days: 1
+
+  upload-coverage:
+    name: Upload coverage to Codecov
+    needs: run
+    if: always() && needs.run.outputs.decision != 'skip'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
+      pull-requests: write
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Download coverage report
+        uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4.2.1
+        with:
+          pattern: coverage-${{ inputs.artifact-name }}-${{ github.run_id }}-${{ github.run_attempt }}
+          path: coverage-reports
+          merge-multiple: true
+
+      - name: Upload to Codecov
+        id: codecov-upload
+        continue-on-error: true
+        uses: codecov/codecov-action@75cd11691c0faa626561e295848008c8a7dddffe # v5.5.4
+        with:
+          use_oidc: true
+          directory: coverage-reports
+          root_dir: ${{ github.workspace }}
+          flags: ${{ inputs.artifact-name }}
+          fail_ci_if_error: false
+
+      - name: Upload to Codecov (retry)
+        if: steps.codecov-upload.outcome == 'failure'
+        continue-on-error: true
+        uses: codecov/codecov-action@75cd11691c0faa626561e295848008c8a7dddffe # v5.5.4
+        with:
+          use_oidc: true
+          directory: coverage-reports
+          root_dir: ${{ github.workspace }}
+          flags: ${{ inputs.artifact-name }}
+          fail_ci_if_error: false

.github/workflows/auto_update_price_and_context_window.yml

@@ -0,0 +1,39 @@
+name: Updates model_prices_and_context_window.json and Create Pull Request
+
+on:
+  schedule:
+    - cron: "0 0 * * 0" # Run every Sundays at midnight
+    #- cron: "0 0 * * *" # Run daily at midnight
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  auto_update_price_and_context_window:
+    if: github.repository == 'BerriAI/litellm'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+      - name: Set up uv
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+      - name: Update JSON Data
+        run: |
+          uv run --frozen --with 'aiohttp==3.13.3' python ".github/scripts/auto_update_price_and_context_window_file.py"
+      - name: Regenerate JSON Schema
+        run: |
+          uv run --frozen python ci_cd/generate_model_prices_schema.py
+      - name: Create Pull Request
+        run: |
+          git add model_prices_and_context_window.json model_prices_and_context_window.schema.json
+          git commit -m "Update model_prices_and_context_window.json file: $(date +'%Y-%m-%d')" 
+          gh pr create --title "Update model_prices_and_context_window.json file" \
+            --body "Automated update for model_prices_and_context_window.json" \
+            --head auto-update-price-and-context-window-$(date +'%Y-%m-%d') \
+            --base main
+        env:
+          GH_TOKEN: ${{ secrets.GH_TOKEN }}

.github/workflows/check-ui-api-types.yml

@@ -0,0 +1,136 @@
+name: Check UI API Types Sync
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  check-sync:
+    name: Verify schema.d.ts matches the proxy OpenAPI spec
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+          fetch-depth: 2
+
+      - name: Detect changes that can affect the generated types
+        id: changes
+        run: |
+          set -euo pipefail
+          if ! base="$(git rev-parse --verify --quiet HEAD^2 >/dev/null && git rev-parse HEAD^1)"; then
+            echo "Not a pull request merge commit, running the full check."
+            echo "relevant=true" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+          files="$(git diff --name-only "$base" HEAD)"
+          if grep -Eq '^(litellm/(proxy|types)/|ui/litellm-dashboard/(src/lib/http/schema\.d\.ts|scripts/gen-api-types\.mjs|package(-lock)?\.json)$|\.github/workflows/check-ui-api-types\.yml$)' <<< "$files"; then
+            echo "relevant=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "No proxy, types or generator changes in this pull request, nothing to verify."
+            echo "relevant=false" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: Set up Python
+        if: steps.changes.outputs.relevant == 'true'
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        if: steps.changes.outputs.relevant == 'true'
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Cache uv dependencies
+        if: steps.changes.outputs.relevant == 'true'
+        uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+        with:
+          path: |
+            ~/.cache/uv
+            .venv
+          key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-uv-
+
+      - name: Cache the Rust build
+        if: steps.changes.outputs.relevant == 'true'
+        uses: ./.github/actions/cache-cargo-build
+
+      - name: Install backend dependencies
+        if: steps.changes.outputs.relevant == 'true'
+        run: .github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router
+
+      - name: Cache Prisma binaries
+        if: steps.changes.outputs.relevant == 'true'
+        uses: ./.github/actions/cache-prisma-binaries
+
+      - name: Generate Prisma client
+        if: steps.changes.outputs.relevant == 'true'
+        run: uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma
+
+      - name: Regenerate the lazy OpenAPI snapshot
+        if: steps.changes.outputs.relevant == 'true'
+        run: uv run --no-sync python -m litellm.proxy._lazy_openapi_snapshot
+
+      - name: Fail if the lazy OpenAPI snapshot is stale
+        if: steps.changes.outputs.relevant == 'true'
+        run: |
+          if ! git diff --exit-code -- litellm/proxy/_lazy_openapi_snapshot.json; then
+            echo "::error file=litellm/proxy/_lazy_openapi_snapshot.json::The lazy OpenAPI snapshot is out of sync with the lazily loaded routes."
+            echo ""
+            echo "A lazily loaded route or model changed without regenerating the snapshot that /openapi.json serves for unloaded features."
+            echo "To fix, run from the repo root:"
+            echo "  uv run python -m litellm.proxy._lazy_openapi_snapshot"
+            echo "then run npm run gen:api from ui/litellm-dashboard and commit both files."
+            exit 1
+          fi
+          echo "_lazy_openapi_snapshot.json is in sync with the lazily loaded routes."
+
+      - name: Set up Node.js
+        if: steps.changes.outputs.relevant == 'true'
+        uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
+        with:
+          node-version-file: ui/litellm-dashboard/.nvmrc
+          cache: "npm"
+          cache-dependency-path: ui/litellm-dashboard/package-lock.json
+
+      - name: Install dashboard dependencies
+        if: steps.changes.outputs.relevant == 'true'
+        working-directory: ui/litellm-dashboard
+        run: npm ci
+
+      - name: Regenerate types from the live spec
+        if: steps.changes.outputs.relevant == 'true'
+        working-directory: ui/litellm-dashboard
+        env:
+          LITELLM_PYTHON: "uv run --no-sync python"
+        run: npm run gen:api
+
+      - name: Fail if types are stale
+        if: steps.changes.outputs.relevant == 'true'
+        run: |
+          if ! git diff --exit-code -- ui/litellm-dashboard/src/lib/http/schema.d.ts; then
+            echo "::error file=ui/litellm-dashboard/src/lib/http/schema.d.ts::Generated API types are out of sync with the proxy OpenAPI spec."
+            echo ""
+            echo "A backend route or model changed without regenerating the dashboard types."
+            echo "To fix, run from ui/litellm-dashboard:"
+            echo "  npm run gen:api"
+            echo "then commit the updated src/lib/http/schema.d.ts."
+            exit 1
+          fi
+          echo "schema.d.ts is in sync with the proxy OpenAPI spec."

.github/workflows/ci-coverage.yml

@@ -0,0 +1,51 @@
+name: "CI Coverage"
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+  push:
+    branches:
+      - main
+      - litellm_internal_staging
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  assert-ci-coverage:
+    name: assert-ci-coverage
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Assert every test file and Dockerfile is invoked by a job
+        run: |
+          python -m pip install "pyyaml==6.0.3"
+          python .github/scripts/assert_ci_coverage.py
+
+      # The census asks whether a job names a file; this asks whether that job's -k
+      # then throws it back out. A file both globbed and deselected everywhere runs
+      # nowhere while counting as covered, which is how the caching suite went unrun.
+      - name: Assert no -k expression deselects a file from every job that globs it
+        run: python .github/scripts/assert_ci_coverage.py --slices
+
+      - name: Assert .github/workflows/ holds only workflows, correctly named
+        run: python .github/scripts/assert_workflow_dir_hygiene.py

.github/workflows/codeql.yml

@@ -1,6 +1,8 @@
 name: "CodeQL"
 
 on:
+  push:
+    branches: [main]
   pull_request:
     branches: [main]
   schedule:
@@ -41,15 +43,14 @@ jobs:
           persist-credentials: false
 
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+        uses: github/codeql-action/init@ebcb5b36ded6beda4ceefea6a8bc4cc885255bb3 # v3.34.1
         with:
           languages: ${{ matrix.language }}
           build-mode: ${{ matrix.build-mode }}
           config-file: ./.github/codeql/codeql-config.yml
-          queries: ${{ github.event_name == 'pull_request' && '+security-extended' || '' }}
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+        uses: github/codeql-action/analyze@ebcb5b36ded6beda4ceefea6a8bc4cc885255bb3 # v3.34.1
         with:
           category: "/language:${{ matrix.language }}"
           output: sarif-results
@@ -66,23 +67,17 @@ jobs:
       # further up the stack are modified. The suppression is scoped to this one
       # file/rule pair via SARIF post-filtering so every other callsite of
       # py/weak-sensitive-data-hashing in the repository continues to be analyzed.
-      # The same query fires on the HIBP k-anonymity lookup in
-      # litellm/proxy/auth/password_policy.py, where the password's SHA-1 is only
-      # a lookup key into the haveibeenpwned range API (the protocol mandates
-      # SHA-1) and the digest itself never leaves the proxy beyond its first 5
-      # characters.
-      - name: Filter SARIF (OCI sha256, HIBP sha1)
+      - name: Filter SARIF (OCI sha256)
         if: matrix.language == 'python'
         uses: advanced-security/filter-sarif@2da736ff05ef065cb2894ac6892e47b5eac2c3c0 # v1.1
         with:
           patterns: |
             -litellm/llms/oci/common_utils.py:py/weak-sensitive-data-hashing
-            -litellm/proxy/auth/password_policy.py:py/weak-sensitive-data-hashing
           input: sarif-results/python.sarif
           output: sarif-results/python.sarif
 
       - name: Upload SARIF
-        uses: github/codeql-action/upload-sarif@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+        uses: github/codeql-action/upload-sarif@ebcb5b36ded6beda4ceefea6a8bc4cc885255bb3 # v3.34.1
         with:
           sarif_file: sarif-results
           category: "/language:${{ matrix.language }}"

.github/workflows/codspeed.yml

@@ -4,6 +4,7 @@ on:
   push:
     branches:
       - main
+      - litellm_internal_staging
     paths:
       - "litellm/**"
       - "tests/benchmarks/**"
@@ -12,10 +13,10 @@ on:
       - ".github/workflows/codspeed.yml"
       - ".github/actions/setup-uv-with-retries/**"
       - ".github/actions/cache-cargo-build/**"
-      - ".github/scripts/uv_sync_with_retries.sh"
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
     paths:
       - "litellm/**"
       - "tests/benchmarks/**"
@@ -24,7 +25,6 @@ on:
       - ".github/workflows/codspeed.yml"
       - ".github/actions/setup-uv-with-retries/**"
       - ".github/actions/cache-cargo-build/**"
-      - ".github/scripts/uv_sync_with_retries.sh"
   # Allow CodSpeed to trigger backtest performance analysis
   # in order to generate initial data
   workflow_dispatch:
@@ -59,27 +59,19 @@ jobs:
 
       - name: Cache the Rust build
         uses: ./.github/actions/cache-cargo-build
-        with:
-          profile: release
 
       # Build the wheel and resolve every dependency outside the CodSpeed
       # runner: the same maturin build took 42 minutes inside `codspeed run`
       # versus under 3 minutes as a plain step (LIT-6183)
-      - name: Build the release wheel
-        run: uv build --wheel --out-dir dist
-
-      - name: Install the wheel into the benchmark environment
-        run: |
-          UV_PROJECT_ENVIRONMENT="${RUNNER_TEMP}/benchmark-venv" .github/scripts/uv_sync_with_retries.sh --frozen --no-default-groups --group benchmarks --no-install-project --python 3.12
-          uv pip install --python "${RUNNER_TEMP}/benchmark-venv/bin/python" --no-deps dist/*.whl
-
-      - name: Collect benchmarks
-        env:
-          PYTEST_DISABLE_PLUGIN_AUTOLOAD: "1"
-          LITELLM_REQUIRE_INSTALLED_WHEEL: "1"
+      - name: Build environment
         run: >
-          "${RUNNER_TEMP}/benchmark-venv/bin/python" -I -m pytest
-          --import-mode=importlib
+          env PYTEST_DISABLE_PLUGIN_AUTOLOAD=1
+          uv run --frozen --no-default-groups
+          --with pytest==8.3.5
+          --with pytest-codspeed==4.3.0
+          --with "mcp>=2.2.0,<3.0"
+          --with "a2a-sdk>=1.1.0,<2.0"
+          pytest
           -p pytest_codspeed.plugin
           tests/benchmarks/
           --codspeed
@@ -90,9 +82,13 @@ jobs:
         with:
           mode: simulation
           run: >
-            env PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 LITELLM_REQUIRE_INSTALLED_WHEEL=1
-            "${RUNNER_TEMP}/benchmark-venv/bin/python" -I -m pytest
-            --import-mode=importlib
+            env PYTEST_DISABLE_PLUGIN_AUTOLOAD=1
+            uv run --frozen --no-default-groups
+            --with pytest==8.3.5
+            --with pytest-codspeed==4.3.0
+            --with "mcp>=2.2.0,<3.0"
+            --with "a2a-sdk>=1.1.0,<2.0"
+            pytest
             -p pytest_codspeed.plugin
             tests/benchmarks/
             --codspeed

.github/workflows/compat-matrix-image.yml

@@ -1,42 +0,0 @@
-name: Compat Matrix Image
-
-on:
-  pull_request:
-    paths:
-      - tests/e2e/claude_code/cron_vm/**
-      - tests/e2e/cli_harness/npm_version_resolver.py
-      - .github/workflows/compat-matrix-image.yml
-  workflow_dispatch:
-
-permissions: {}
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
-  cancel-in-progress: true
-
-jobs:
-  compat-matrix-image:
-    name: compat-matrix-image
-    runs-on: ubuntu-latest
-    timeout-minutes: 15
-    permissions:
-      contents: read
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Build the Render cron image
-        run: docker build -f tests/e2e/claude_code/cron_vm/Dockerfile -t compat-matrix:${{ github.sha }} tests/e2e
-
-      - name: Resolve and install the Claude Code CLI as the cron user
-        run: |
-          docker run --rm compat-matrix:${{ github.sha }} bash -c '
-            set -euo pipefail
-            whoami
-            gh --version
-            uv --version
-            version="$(uv run --no-project --python 3.12 python /opt/litellm/tests/e2e/cli_harness/npm_version_resolver.py @anthropic-ai/claude-code)"
-            /opt/litellm/tests/e2e/claude_code/cron_vm/install_claude_code.sh "${version}" /tmp/claude-cli
-            /tmp/claude-cli/claude --version
-          '

.github/workflows/conventional-commits.yml

@@ -41,7 +41,6 @@ jobs:
             ci
             chore
             revert
-            security
           requireScope: false
           subjectPattern: ^(?![A-Z]).+$
           subjectPatternError: |

.github/workflows/cost-map-guard.yml

@@ -1,14 +1,15 @@
 name: Cost map guard
 
-on:
-  pull_request:
+on: # zizmor: ignore[dangerous-triggers] runs the base branch's code only; the PR's cost map files are read as data and never executed
+  pull_request_target:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
 
 permissions:
   contents: read
-  pull-requests: read
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
@@ -20,23 +21,17 @@ jobs:
     steps:
       - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
         with:
-          ref: ${{ github.event.pull_request.base.sha }}
           persist-credentials: false
-      - name: Check out the base branch tip and fetch the pull request head and its merge base
+      - name: Fetch the pull request head and its merge base
         id: revisions
         env:
           GH_TOKEN: ${{ github.token }}
-          PR_NUMBER: ${{ github.event.pull_request.number }}
-          EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
           HEAD_SHA: ${{ github.event.pull_request.head.sha }}
         run: |
-          base_ref="$(gh api "repos/${GITHUB_REPOSITORY}/branches/${EVENT_BASE_REF}" --jq .name)" || base_ref="$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq .base.ref)"
-          target="$(gh api "repos/${GITHUB_REPOSITORY}/branches/${base_ref}" --jq '.commit.sha')"
-          merge_base="$(gh api "repos/${GITHUB_REPOSITORY}/compare/${target}...${HEAD_SHA}" --jq '.merge_base_commit.sha')"
-          git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA" "$target"
-          git checkout -q --detach "$target"
+          merge_base="$(gh api "repos/${GITHUB_REPOSITORY}/compare/${BASE_SHA}...${HEAD_SHA}" --jq '.merge_base_commit.sha')"
+          git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA"
           echo "merge_base=$merge_base" >> "$GITHUB_OUTPUT"
-          echo "target=$target" >> "$GITHUB_OUTPUT"
       - name: Set up uv
         uses: ./.github/actions/setup-uv-with-retries
         with:
@@ -44,30 +39,7 @@ jobs:
       - name: Run the guard
         env:
           MERGE_BASE: ${{ steps.revisions.outputs.merge_base }}
-          COST_MAP_GUARD_TARGET: ${{ steps.revisions.outputs.target }}
-          COST_MAP_GUARD_AUTHOR_TYPE: ${{ github.event.pull_request.user.type }}
           HEAD_SHA: ${{ github.event.pull_request.head.sha }}
           HEAD_REF: ${{ github.event.pull_request.head.ref }}
         run: |
-          uv run --no-project python ci_cd/cost_map_guard.py --base "$MERGE_BASE" --head "$HEAD_SHA" --head-ref "$HEAD_REF"
-      - name: Check whether the pull request changes the cost map
-        id: changed
-        env:
-          MERGE_BASE: ${{ steps.revisions.outputs.merge_base }}
-          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
-        run: |
-          if git diff --quiet "$MERGE_BASE" "$HEAD_SHA" -- model_prices_and_context_window.json; then
-            echo "cost_map=false" >> "$GITHUB_OUTPUT"
-          else
-            echo "cost_map=true" >> "$GITHUB_OUTPUT"
-          fi
-      - if: steps.changed.outputs.cost_map == 'true'
-        run: rustup toolchain install --no-self-update
-      - name: Load the cost map through model-catalog
-        if: steps.changed.outputs.cost_map == 'true'
-        env:
-          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
-        run: |
-          git show "${HEAD_SHA}:model_prices_and_context_window.json" > "$RUNNER_TEMP/model_prices_and_context_window.json"
-          cargo run --locked --quiet --manifest-path litellm-rust/Cargo.toml -p litellm-model-catalog --bin check-registry \
-            -- "$RUNNER_TEMP/model_prices_and_context_window.json"
+          uv run --frozen python ci_cd/cost_map_guard.py --base "$MERGE_BASE" --head "$HEAD_SHA" --head-ref "$HEAD_REF"

.github/workflows/cost-map-pins-recheck.yml

@@ -1,91 +0,0 @@
-name: Cost map pins recheck
-
-on:
-  push:
-    branches:
-      - main
-      - "litellm_**"
-    paths:
-      - model_prices_and_context_window.json
-      - litellm/model_prices_and_context_window_backup.json
-      - ci_cd/cost_map_pins.json
-
-permissions: {}
-
-jobs:
-  rerun-cost-map-guard:
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    permissions:
-      actions: write
-      contents: read
-      pull-requests: read
-    steps:
-      - name: Re-run the cost map guard on open pull requests the push could break
-        env:
-          GH_TOKEN: ${{ github.token }}
-          BASE_REF: ${{ github.ref_name }}
-          BEFORE_SHA: ${{ github.event.before }}
-          AFTER_SHA: ${{ github.event.after }}
-          FORCED: ${{ github.event.forced }}
-          COST_MAP_PATHS: '["model_prices_and_context_window.json", "litellm/model_prices_and_context_window_backup.json", "ci_cd/cost_map_pins.json"]'
-          PINS_PATHS: '["ci_cd/cost_map_pins.json"]'
-        run: |
-          if [[ "$BEFORE_SHA" =~ ^0+$ || "$FORCED" == true ]] \
-            || ! gh api "repos/${GITHUB_REPOSITORY}/compare/${BEFORE_SHA}...${AFTER_SHA}?per_page=1" \
-              --jq '(env.PINS_PATHS | fromjson) as $pins | (.files | length) < 300 and all(.files[]; [.filename, .previous_filename] | any(IN($pins[])) | not)' \
-              | grep -qx true; then
-            export RECHECK_PATHS="$COST_MAP_PATHS"
-          else
-            export RECHECK_PATHS="$PINS_PATHS"
-          fi
-          rerun_cutoff="$(date -u -d '30 days ago' +%Y-%m-%dT%H:%M:%SZ)"
-          candidates="$(gh api graphql --paginate -F owner="${GITHUB_REPOSITORY_OWNER}" -F name="${GITHUB_REPOSITORY#*/}" -F base="$BASE_REF" \
-            -f query='query($owner: String!, $name: String!, $base: String!, $endCursor: String) {
-              repository(owner: $owner, name: $name) {
-                pullRequests(states: OPEN, baseRefName: $base, first: 25, after: $endCursor) {
-                  nodes { number headRefOid changedFiles files(first: 100) { nodes { path } } }
-                  pageInfo { hasNextPage endCursor }
-                }
-              }
-            }' \
-            --jq '(env.RECHECK_PATHS | fromjson) as $paths | .data.repository.pullRequests.nodes[]
-              | {number, headRefOid, listed: any(.files.nodes[].path; IN($paths[])), unlisted: (.changedFiles > 100)}
-              | select(.listed or .unlisted) | "\(.number) \(.headRefOid) \(.listed)"')"
-          failed=0
-          declare -A rerun_heads
-          while read -r number head_sha listed; do
-            [ -n "$number" ] || continue
-            if [ "$listed" = false ]; then
-              if ! changed="$(gh api --paginate "repos/${GITHUB_REPOSITORY}/pulls/${number}/files?per_page=100" \
-                --jq '(env.RECHECK_PATHS | fromjson) as $paths | .[].filename | select(IN($paths[]))')"; then
-                echo "::error::could not list the files of #$number"
-                failed=1
-                continue
-              fi
-              [ -n "$changed" ] || continue
-            fi
-            [ -z "${rerun_heads[$head_sha]:-}" ] || continue
-            rerun_heads[$head_sha]=1
-            if ! runs="$(gh run list --repo "$GITHUB_REPOSITORY" --workflow cost-map-guard.yml --commit "$head_sha" --limit 100 --json databaseId,createdAt --jq '.[] | "\(.databaseId) \(.createdAt)"')"; then
-              echo "::error::could not list the cost map guard runs of #$number"
-              failed=1
-              continue
-            fi
-            if [ -z "$runs" ]; then
-              echo "::warning::no cost map guard run found for #$number"
-              continue
-            fi
-            while read -r run_id created_at; do
-              if [[ "$created_at" < "$rerun_cutoff" ]]; then
-                echo "::warning::cost map guard run $run_id for #$number is over 30 days old, so GitHub cannot re-run it; #$number gets the pin checks on its next push"
-                continue
-              fi
-              gh run watch "$run_id" --repo "$GITHUB_REPOSITORY" > /dev/null || true
-              if ! gh run rerun "$run_id" --repo "$GITHUB_REPOSITORY"; then
-                echo "::error::could not re-run cost map guard run $run_id for #$number"
-                failed=1
-              fi
-            done <<< "$runs"
-          done <<< "$candidates"
-          exit "$failed"

.github/workflows/create-rc-branch.yml

@@ -1,79 +0,0 @@
-name: Create RC Branch
-
-on:
-  schedule:
-    - cron: "0 3 * * 1"
-      timezone: "America/Los_Angeles"
-  workflow_dispatch:
-
-permissions: {}
-
-jobs:
-  create-rc-branch:
-    name: Create RC Branch
-    if: github.event_name != 'schedule' || github.repository == 'BerriAI/litellm'
-    runs-on: ubuntu-latest
-    permissions:
-      contents: write
-    outputs:
-      version: ${{ steps.version.outputs.version }}
-    steps:
-      - name: Require main
-        env:
-          REF: ${{ github.ref }}
-        run: |
-          if [ "$REF" != "refs/heads/main" ]; then
-            echo "::error::rc branches are cut from refs/heads/main only, got $REF"
-            exit 1
-          fi
-
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Read release version
-        id: version
-        run: python3 .github/scripts/read_rc_version.py >> "$GITHUB_OUTPUT"
-
-      - name: Create rc branch
-        env:
-          VERSION: ${{ steps.version.outputs.version }}
-        uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
-        with:
-          script: |
-            const branchName = `rc/${process.env.VERSION}`;
-            const ref = `heads/${branchName}`;
-
-            const existing = await github.rest.git.getRef({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              ref,
-            }).catch((error) => {
-              if (error.status === 404) {
-                return null;
-              }
-              throw error;
-            });
-            if (existing !== null) {
-              core.setFailed(`Branch ${branchName} already exists at ${existing.data.object.sha}; leaving it untouched`);
-              return;
-            }
-
-            await github.rest.git.createRef({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              ref: `refs/${ref}`,
-              sha: context.sha,
-            });
-            core.info(`Created branch ${branchName} at ${context.sha}`);
-
-  linear-release:
-    name: Move the Linear release to rc
-    needs: create-rc-branch
-    permissions:
-      contents: read
-    uses: ./.github/workflows/linear-release.yml
-    with:
-      rc_version: ${{ needs.create-rc-branch.outputs.version }}
-    secrets:
-      LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}

.github/workflows/create-release.yml

@@ -0,0 +1,186 @@
+name: Create Release
+
+on:
+  workflow_dispatch:
+    inputs:
+      tag:
+        description: "Release tag (e.g. 1.84.0, 1.84.0rc1, 1.84.0.dev42, 1.84.0-dev.2, 1.84.0.post1; legacy v1.83.10-stable still accepted)"
+        required: true
+        type: string
+      commit_hash:
+        description: "Full 40-char commit SHA to target"
+        required: true
+        type: string
+
+permissions: {}
+
+jobs:
+  release:
+    name: Create Release
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - name: Validate inputs
+        env:
+          TAG: ${{ inputs.tag }}
+          COMMIT_HASH: ${{ inputs.commit_hash }}
+        run: |
+          if ! echo "${COMMIT_HASH}" | grep -qE '^[0-9a-f]{40}$'; then
+            echo "::error::commit_hash must be a full 40-character commit SHA"
+            exit 1
+          fi
+          if ! echo "${TAG}" | grep -qE '^v?[0-9]+\.[0-9]+\.[0-9]+'; then
+            echo "::error::tag must start with X.Y.Z (optional leading v), e.g. 1.84.0, 1.84.0rc1, 1.84.0.dev42, or v1.83.10-stable"
+            exit 1
+          fi
+
+      - name: Create release
+        env:
+          TAG: ${{ inputs.tag }}
+          COMMIT_HASH: ${{ inputs.commit_hash }}
+        uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
+        with:
+          script: |
+            const tag = process.env.TAG;
+            const commitHash = process.env.COMMIT_HASH;
+
+            // Mark RC / dev / nightly / alpha / beta tags as GitHub pre-releases.
+            // Accept both PEP 440 (`.dev`) and SemVer (`-dev`) separators so tags
+            // like `1.84.0.dev2` and `1.84.0-dev.2` are both detected.
+            // PEP 440 post-releases (e.g. `1.84.0.post1`) and legacy `-stable[.patch.N]`
+            // are stable maintenance releases, not pre-releases.
+            const isPrerelease = /(?:rc|nightly|alpha|beta|[-.]dev)/i.test(tag);
+
+            // A stable release should only claim the repo "latest" badge when its
+            // version is >= the current latest. Otherwise a backport (e.g. 1.84.6)
+            // would steal "latest" from a newer line (e.g. 1.88.1).
+            const versionKey = (rawTag) => {
+              const m = String(rawTag).match(/^v?(\d+)\.(\d+)\.(\d+)/);
+              if (!m) return null;
+              const maintenance = String(rawTag).match(/(?:\.post|\.patch\.)(\d+)/i);
+              return [Number(m[1]), Number(m[2]), Number(m[3]), maintenance ? Number(maintenance[1]) : 0];
+            };
+            const isAtLeast = (a, b) => {
+              for (let i = 0; i < a.length; i++) {
+                if (a[i] !== b[i]) return a[i] > b[i];
+              }
+              return true;
+            };
+
+            const cosignSection = [
+              `## Verify Docker Image Signature`,
+              ``,
+              `All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit \`0112e53\`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).`,
+              ``,
+              `**Verify using the pinned commit hash (recommended):**`,
+              ``,
+              `A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:`,
+              ``,
+              '```bash',
+              `cosign verify \\`,
+              `  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \\`,
+              `  ghcr.io/berriai/litellm:${tag}`,
+              '```',
+              ``,
+              `**Verify using the release tag (convenience):**`,
+              ``,
+              `Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:`,
+              ``,
+              '```bash',
+              `cosign verify \\`,
+              `  --key https://raw.githubusercontent.com/BerriAI/litellm/${tag}/cosign.pub \\`,
+              `  ghcr.io/berriai/litellm:${tag}`,
+              '```',
+              ``,
+              `Expected output:`,
+              ``,
+              '```',
+              `The following checks were performed on each of these signatures:`,
+              `  - The cosign claims were validated`,
+              `  - The signatures were verified against the specified public key`,
+              '```',
+              ``,
+              `---`,
+              ``,
+            ].join('\n');
+
+            try {
+              let makeLatest = "false";
+              const newVersion = versionKey(tag);
+              if (!isPrerelease && newVersion) {
+                let latestVersion = null;
+                try {
+                  const latest = await github.rest.repos.getLatestRelease({
+                    owner: context.repo.owner,
+                    repo: context.repo.repo,
+                  });
+                  latestVersion = versionKey(latest.data.tag_name);
+                } catch (error) {
+                  if (error.status !== 404) throw error;
+                }
+                makeLatest = (!latestVersion || isAtLeast(newVersion, latestVersion)) ? "true" : "false";
+              }
+
+              try {
+                await github.rest.git.createRef({
+                  owner: context.repo.owner,
+                  repo: context.repo.repo,
+                  ref: `refs/tags/${tag}`,
+                  sha: commitHash,
+                });
+              } catch (error) {
+                if (error.status !== 422) throw error;
+                const existing = await github.rest.git.getRef({
+                  owner: context.repo.owner,
+                  repo: context.repo.repo,
+                  ref: `tags/${tag}`,
+                });
+                if (existing.data.object.sha !== commitHash) {
+                  throw new Error(`Tag ${tag} already exists at ${existing.data.object.sha}, expected ${commitHash}`);
+                }
+              }
+
+              const response = await github.rest.repos.createRelease({
+                draft: true,
+                generate_release_notes: true,
+                name: tag,
+                owner: context.repo.owner,
+                prerelease: isPrerelease,
+                repo: context.repo.repo,
+                tag_name: tag,
+              });
+
+              const updatedBody = cosignSection + (response.data.body ?? '');
+              await github.rest.repos.updateRelease({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                release_id: response.data.id,
+                tag_name: tag,
+                body: updatedBody,
+                draft: false,
+              });
+
+              if (!isPrerelease) {
+                await github.rest.repos.updateRelease({
+                  owner: context.repo.owner,
+                  repo: context.repo.repo,
+                  release_id: response.data.id,
+                  tag_name: tag,
+                  make_latest: makeLatest,
+                });
+              }
+
+            } catch (error) {
+              core.setFailed(error.message);
+            }
+
+  create-branch:
+    name: Create Release Branch
+    needs: release
+    permissions:
+      contents: write
+    uses: ./.github/workflows/create-release-branch.yml
+    with:
+      tag: ${{ inputs.tag }}
+      commit_hash: ${{ inputs.commit_hash }}

.github/workflows/create_daily_staging_branch.yml

@@ -0,0 +1,49 @@
+name: Create Daily Staging Branch
+
+on:
+  schedule:
+    - cron: "0 0,12 * * *" # Runs every 12 hours at midnight and noon UTC
+  workflow_dispatch: # Allow manual trigger
+
+jobs:
+  create-staging-branch:
+    if: github.repository == 'BerriAI/litellm'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - name: Create daily staging branch
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          BRANCH_NAME="litellm_oss_staging_$(date +'%m_%d_%Y')"
+          echo "Creating branch: $BRANCH_NAME"
+          if gh api "repos/${{ github.repository }}/git/ref/heads/$BRANCH_NAME" --silent 2>/dev/null; then
+            echo "Branch $BRANCH_NAME already exists. Skipping creation."
+            exit 0
+          fi
+          MAIN_SHA=$(gh api "repos/${{ github.repository }}/git/ref/heads/main" --jq '.object.sha')
+          gh api "repos/${{ github.repository }}/git/refs" -f ref="refs/heads/$BRANCH_NAME" -f sha="$MAIN_SHA" --silent
+          echo "Successfully created branch: $BRANCH_NAME at $MAIN_SHA"
+
+  create-internal-dev-branch:
+    if: github.repository == 'BerriAI/litellm'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - name: Create internal dev branch
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          BRANCH_NAME="litellm_internal_dev_$(date +'%m_%d_%Y')"
+          echo "Creating branch: $BRANCH_NAME"
+          if gh api "repos/${{ github.repository }}/git/ref/heads/$BRANCH_NAME" --silent 2>/dev/null; then
+            echo "Branch $BRANCH_NAME already exists. Skipping creation."
+            exit 0
+          fi
+          MAIN_SHA=$(gh api "repos/${{ github.repository }}/git/ref/heads/main" --jq '.object.sha')
+          gh api "repos/${{ github.repository }}/git/refs" -f ref="refs/heads/$BRANCH_NAME" -f sha="$MAIN_SHA" --silent
+          echo "Successfully created branch: $BRANCH_NAME at $MAIN_SHA"

.github/workflows/guard-fork-dependencies.yml

@@ -4,6 +4,8 @@ on:
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
     paths:
       - "uv.lock"

.github/workflows/helm_unit_test.yml

@@ -0,0 +1,54 @@
+name: Helm unit test
+
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  unit-test:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Set up Helm 3.11.1
+        uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4.3.1
+        with:
+          version: "3.11.1"
+
+      - name: Download and verify Helm Unit Test Plugin
+        run: |
+          curl -fsSLo "$RUNNER_TEMP/helm-unittest.tgz" https://github.com/helm-unittest/helm-unittest/releases/download/v0.8.2/helm-unittest-linux-amd64-0.8.2.tgz
+          echo "56ab3091e6fa52a7c92ee951def9bed957f295d9ce98483aed404e748d7b3a94  $RUNNER_TEMP/helm-unittest.tgz" | sha256sum -c -
+
+      - name: Install Helm Unit Test Plugin
+        run: |
+          PLUGIN_DIR="$(helm env HELM_PLUGINS)/helm-unittest"
+          mkdir -p "$PLUGIN_DIR"
+          tar -xzf "$RUNNER_TEMP/helm-unittest.tgz" -C "$PLUGIN_DIR"
+          helm plugin list
+
+      - name: Run unit tests
+        run: |
+          for chart in helm/litellm-helm helm/litellm; do
+            declared="$(grep -h '^suite:' "$chart"/tests/*.yaml | wc -l | tr -d '[:space:]')"
+            output="$(mktemp)"
+            helm unittest -f 'tests/*.yaml' "$chart" | tee "$output"
+            executed="$(sed -n 's/^Test Suites:.*[[:space:]]\([0-9][0-9]*\) total$/\1/p' "$output")"
+            if [ "$declared" != "$executed" ]; then
+              echo "::error::$chart declares $declared test suites but helm-unittest ran $executed. Suites are being skipped silently, so their assertions never execute."
+              exit 1
+            fi
+            echo "$chart: all $declared declared test suites ran"
+          done

.github/workflows/image-scan.yml

@@ -4,32 +4,23 @@ on:
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
       - litellm_oss_branch
       - "litellm_**"
     paths:
       - Dockerfile
       - docker/Dockerfile.non_root
-      - docker/Dockerfile.database
       - migrations/Dockerfile
       - migrations/run.py
       - gateway/Dockerfile
-      - gateway/launch.py
       - gateway/main.py
-      - gateway/routes/allowlist.py
-      - litellm_cli/**
       - backend/Dockerfile
       - backend/main.py
-      - litellm-rust/**
-      - litellm/proxy/lens/**
       - docker/component_entrypoint.sh
       - docker/entrypoint.sh
       - litellm/proxy/prisma_migration.py
-      - litellm/proxy/admin_mcp.py
-      - litellm/proxy/proxy_server.py
-      - backend/routes/allowlist.py
-      - pyproject.toml
       - litellm-proxy-extras/**
-      - tests/integration/database/image/**
+      - tests/proxy_migration_tests/**
       - uv.lock
       - ui/litellm-dashboard/package-lock.json
       - ui/Dockerfile
@@ -75,9 +66,6 @@ jobs:
       - name: Build runtime image
         run: docker build -f docker/Dockerfile.non_root -t litellm-image-scan:${{ github.sha }} .
 
-      - name: Tag the cached builder for Admin MCP schema setup
-        run: docker build --target builder -f docker/Dockerfile.non_root -t litellm-admin-mcp-schema:${{ github.sha }} .
-
       # The prisma bake must migrate a fresh DB with no egress as an arbitrary
       # non-root uid (OpenShift restricted-v2 / air-gapped / readOnlyRootFilesystem).
       # `docker run` as the default uid with network hides a broken bake because
@@ -91,10 +79,9 @@ jobs:
       - name: Verify offline migration as a non-root uid
         env:
           LITELLM_IMAGE: litellm-image-scan:${{ github.sha }}
-          LITELLM_ADMIN_MCP_SCHEMA_IMAGE: litellm-admin-mcp-schema:${{ github.sha }}
         run: |
           python -m pip install "pytest==9.0.3"
-          python -m pytest --noconftest tests/integration/database/image/test_offline_image_migration.py tests/integration/database/image/test_image_bedrock_realtime_extra.py tests/integration/database/image/test_image_admin_mcp.py -v
+          python -m pytest tests/proxy_migration_tests/test_offline_image_migration.py tests/proxy_migration_tests/test_image_bedrock_realtime_extra.py -v
 
       # Scans the whole shipped artifact: OS/apk plus every language package
       # baked into the image, including ones no lockfile declares (e.g. prisma's
@@ -113,28 +100,21 @@ jobs:
             --output table
 
   runtime-image:
-    name: runtime-image (${{ matrix.dockerfile }})
+    name: runtime-image
     runs-on: ubuntu-latest
     if: >-
       github.event_name != 'pull_request' ||
       github.event.pull_request.head.repo.full_name == github.repository
-    timeout-minutes: 45
+    timeout-minutes: 30
     permissions:
       contents: read
-    strategy:
-      fail-fast: false
-      matrix:
-        dockerfile: [Dockerfile, docker/Dockerfile.database]
     steps:
       - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
         with:
           persist-credentials: false
 
       - name: Build runtime image
-        run: docker build --build-arg LITELLM_RELEASE_TAG=v0.0.0-lens-ci -f "${{ matrix.dockerfile }}" -t litellm-runtime-scan:${{ github.sha }} .
-
-      - name: Tag the cached builder for Admin MCP schema setup
-        run: docker build --target builder -f "${{ matrix.dockerfile }}" -t litellm-admin-mcp-schema:${{ github.sha }} .
+        run: docker build -f Dockerfile -t litellm-runtime-scan:${{ github.sha }} .
 
       - name: Set up Python
         uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
@@ -144,11 +124,9 @@ jobs:
       - name: Verify offline migration as a non-root uid
         env:
           LITELLM_IMAGE: litellm-runtime-scan:${{ github.sha }}
-          LITELLM_ADMIN_MCP_SCHEMA_IMAGE: litellm-admin-mcp-schema:${{ github.sha }}
         run: |
           python -m pip install "pytest==9.0.3"
-          python -m pytest --noconftest tests/integration/database/image/test_offline_image_migration.py tests/integration/database/image/test_image_bedrock_realtime_extra.py tests/integration/database/image/test_image_admin_mcp.py -v
-
+          python -m pytest tests/proxy_migration_tests/test_offline_image_migration.py tests/proxy_migration_tests/test_image_bedrock_realtime_extra.py -v
 
   migrations-image:
     name: migrations-image
@@ -179,7 +157,7 @@ jobs:
           LITELLM_MIGRATION_SCRIPT: /app/run.py
         run: |
           python -m pip install "pytest==9.0.3"
-          python -m pytest --noconftest tests/integration/database/image/test_offline_image_migration.py -v
+          python -m pytest tests/proxy_migration_tests/test_offline_image_migration.py -v
 
   gateway-image:
     name: gateway-image
@@ -207,10 +185,9 @@ jobs:
         env:
           LITELLM_IMAGE: litellm-gateway-scan:${{ github.sha }}
           LITELLM_COMPONENT_PORT: "4000"
-          LITELLM_IMAGE_COMPONENT: gateway
         run: |
           python -m pip install "pytest==9.0.3"
-          python -m pytest --noconftest tests/integration/database/image/test_component_image_serves_offline.py tests/integration/database/image/test_image_bedrock_realtime_extra.py tests/integration/database/image/test_image_admin_mcp.py -v
+          python -m pytest tests/proxy_migration_tests/test_component_image_serves_offline.py tests/proxy_migration_tests/test_image_bedrock_realtime_extra.py -v
 
   ui-image:
     name: ui-image
@@ -239,7 +216,7 @@ jobs:
           LITELLM_IMAGE: litellm-ui-scan:${{ github.sha }}
         run: |
           python -m pip install "pytest==9.0.3"
-          python -m pytest --noconftest tests/integration/database/image/test_ui_image_serves_offline.py -v
+          python -m pytest tests/proxy_migration_tests/test_ui_image_serves_offline.py -v
 
   backend-image:
     name: backend-image
@@ -258,9 +235,6 @@ jobs:
       - name: Build backend image
         run: docker build -f backend/Dockerfile -t litellm-backend-scan:${{ github.sha }} .
 
-      - name: Tag the cached builder for Admin MCP schema setup
-        run: docker build --target builder -f backend/Dockerfile -t litellm-admin-mcp-schema:${{ github.sha }} .
-
       - name: Set up Python
         uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
         with:
@@ -269,9 +243,7 @@ jobs:
       - name: Verify the backend serves offline as a non-root uid
         env:
           LITELLM_IMAGE: litellm-backend-scan:${{ github.sha }}
-          LITELLM_ADMIN_MCP_SCHEMA_IMAGE: litellm-admin-mcp-schema:${{ github.sha }}
           LITELLM_COMPONENT_PORT: "4001"
-          LITELLM_IMAGE_COMPONENT: backend
         run: |
           python -m pip install "pytest==9.0.3"
-          python -m pytest --noconftest tests/integration/database/image/test_component_image_serves_offline.py tests/integration/database/image/test_image_admin_mcp.py -v
+          python -m pytest tests/proxy_migration_tests/test_component_image_serves_offline.py -v

.github/workflows/interactions-spec-canary.yml

@@ -1,94 +0,0 @@
-name: Interactions spec canary
-
-on:
-  schedule:
-    - cron: "17 7 * * *"
-  pull_request:
-    paths:
-      - .github/workflows/interactions-spec-canary.yml
-      - tests/unit/interactions/**
-      - litellm/types/interactions/**
-  workflow_dispatch:
-
-permissions: {}
-
-concurrency: interactions-spec-canary
-
-jobs:
-  interactions-spec-canary:
-    name: interactions-spec-canary
-    if: github.repository == 'BerriAI/litellm'
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    permissions:
-      contents: read
-    outputs:
-      compliance: ${{ steps.compliance.outcome }}
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: "3.12"
-
-      - name: Set up uv
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Set up the Rust bridge
-        uses: ./.github/actions/rust-bridge
-
-      - name: Install dependencies
-        run: .github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra proxy
-
-      - name: Run the compliance tests against Google's live spec
-        id: compliance
-        run: |
-          uv run --no-sync pytest tests/unit/interactions/test_openapi_compliance.py \
-            --interactions-spec-url=https://ai.google.dev/static/api/interactions.openapi.json \
-            --tb=short -vv -p no:rerunfailures -p no:pytest-retry \
-            --allow-hosts=127.0.0.1,::1,localhost,ai.google.dev
-
-  tracking-issue:
-    name: tracking-issue
-    needs: interactions-spec-canary
-    if: always() && github.event_name != 'pull_request' && github.ref_name == github.event.repository.default_branch && github.repository == 'BerriAI/litellm'
-    runs-on: ubuntu-latest
-    timeout-minutes: 5
-    permissions:
-      issues: write
-    steps:
-      - name: File the tracking issue unless the compliance step passed, close it when it did
-        uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
-        env:
-          COMPLIANCE_OUTCOME: ${{ needs.interactions-spec-canary.outputs.compliance }}
-          RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
-        with:
-          script: |
-            const title = "Interactions spec canary is red";
-            const outcome = process.env.COMPLIANCE_OUTCOME;
-            const runUrl = process.env.RUN_URL;
-            const query = `repo:${context.repo.owner}/${context.repo.repo} is:issue is:open in:title "${title}"`;
-            const { data: found } = await github.rest.search.issuesAndPullRequests({ q: query, per_page: 20 });
-            const open = found.items.find((issue) => issue.title === title);
-            if (outcome === "success") {
-              if (open === undefined) return;
-              await github.rest.issues.createComment({ ...context.repo, issue_number: open.number, body: `Green again: ${runUrl}` });
-              await github.rest.issues.update({ ...context.repo, issue_number: open.number, state: "closed" });
-              return;
-            }
-            const state = outcome === "failure" ? "red" : `not run (compliance step outcome: ${outcome || "none"})`;
-            if (open !== undefined) {
-              await github.rest.issues.createComment({ ...context.repo, issue_number: open.number, body: `Still ${state}: ${runUrl}` });
-              return;
-            }
-            await github.rest.issues.create({
-              ...context.repo,
-              title,
-              assignees: ["mateo-berri"],
-              body: `The scheduled run of tests/unit/interactions/test_openapi_compliance.py against Google's live Interactions spec is ${state}: ${runUrl}\n\nlitellm/types/interactions/README.md says how to refresh the snapshot and regenerate the types.`,
-            });

.github/workflows/issue_fixed_comment.yml

@@ -6,12 +6,8 @@ on:
   workflow_dispatch:
     inputs:
       issue_number:
-        description: "Closed issue number to comment on and close the superseded pull requests of. Ignored by a sweep."
-        required: false
-      sweep:
-        description: "Close every open pull request whose linked issues were all fixed on the default branch. Reads every open pull request, so run it at most once an hour."
-        type: boolean
-        default: false
+        description: "Closed issue number to comment on manually."
+        required: true
   pull_request:
     paths:
       - .github/workflows/issue_fixed_comment.yml
@@ -43,17 +39,16 @@ jobs:
         with:
           bun-version: "1.4.0"
 
-      - name: Test the closer lookup, the release placement, the comment and the superseded pull request close
+      - name: Test the closer lookup, the release placement and the comment
         run: bun test scripts/comment-fixed-issue.test.ts
 
   comment-fixed-issue:
     if: github.event_name != 'pull_request' && github.repository == 'BerriAI/litellm'
     runs-on: ubuntu-latest
-    timeout-minutes: 15
+    timeout-minutes: 5
     permissions:
       contents: read
       issues: write
-      pull-requests: write
     steps:
       - name: Checkout scripts
         uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
@@ -64,16 +59,13 @@ jobs:
       - name: Setup Bun
         uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
         with:
-          # Exact version, never latest: the next step holds issues: write and pull-requests: write tokens
+          # Exact version, never latest: the next step holds an issues: write token
           bun-version: "1.4.0"
 
-      - name: Name the release that carries the fix and close the pull requests it supersedes
-        shell: bash
+      - name: Name the release that carries the fix
         run: bun run scripts/comment-fixed-issue.ts | tee -a "${GITHUB_STEP_SUMMARY}"
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           ISSUE_NUMBER: ${{ github.event.issue.number || github.event.inputs.issue_number }}
-          SWEEP: ${{ github.event.inputs.sweep }}
           DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
           DRY_RUN: ${{ vars.ISSUE_FIXED_COMMENT_ENABLED != 'true' }}
-          CLOSE_PRS_DRY_RUN: ${{ vars.ISSUE_FIXED_CLOSE_PRS_ENABLED != 'true' }}

.github/workflows/linear-release.yml

@@ -1,131 +0,0 @@
-name: Linear Release
-
-on:
-  push:
-    branches:
-      - main
-      - "rc/**"
-  release:
-    types: [published]
-  workflow_call:
-    inputs:
-      rc_version:
-        description: "X.Y.0 release whose rc branch was just cut"
-        required: true
-        type: string
-    secrets:
-      LINEAR_API_KEY:
-        required: true
-
-permissions: {}
-
-jobs:
-  linear-release:
-    name: Linear Release
-    if: github.repository == 'BerriAI/litellm'
-    runs-on: ubuntu-latest
-    permissions:
-      contents: read
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          fetch-depth: 0
-          persist-credentials: false
-
-      - name: Plan
-        id: plan
-        env:
-          EVENT: ${{ github.event_name }}
-          REF_NAME: ${{ github.ref_name }}
-          BEFORE: ${{ github.event.before }}
-          CREATED: ${{ github.event.created }}
-          RC_VERSION: ${{ inputs.rc_version }}
-          RELEASE_TAG: ${{ github.event.release.tag_name }}
-          PRERELEASE: ${{ github.event.release.prerelease }}
-        run: |
-          set -euo pipefail
-          sync_base="${BEFORE}"
-          if [ "${CREATED}" = "true" ]; then
-            sync_base=""
-          fi
-          if [ -n "${RC_VERSION}" ]; then
-            echo "version=${RC_VERSION}" >> "$GITHUB_OUTPUT"
-            echo "stage=rc" >> "$GITHUB_OUTPUT"
-          elif [ "${EVENT}" = "release" ]; then
-            if [ "${PRERELEASE}" = "true" ] || ! echo "${RELEASE_TAG}" | grep -qE '^v[0-9]+\.[0-9]+\.0$'; then
-              echo "::notice::${RELEASE_TAG} is not an X.Y.0 stable release; nothing to complete"
-              exit 0
-            fi
-            echo "version=${RELEASE_TAG#v}" >> "$GITHUB_OUTPUT"
-            echo "complete=true" >> "$GITHUB_OUTPUT"
-          elif [ "${REF_NAME}" = "main" ]; then
-            version="$(python3 .github/scripts/read_rc_version.py | cut -d= -f2)"
-            status=0
-            git ls-remote --exit-code --heads origin "rc/${version}" > /dev/null || status=$?
-            case "${status}" in
-              0)
-                IFS=. read -r major minor _ <<< "${version}"
-                version="${major}.$((minor + 1)).0"
-                ;;
-              2) ;;
-              *)
-                echo "::error::could not check whether rc/${version} exists (git ls-remote exit ${status})"
-                exit 1
-                ;;
-            esac
-            echo "version=${version}" >> "$GITHUB_OUTPUT"
-            echo "sync_base=${sync_base}" >> "$GITHUB_OUTPUT"
-            echo "main=true" >> "$GITHUB_OUTPUT"
-          else
-            echo "version=${REF_NAME#rc/}" >> "$GITHUB_OUTPUT"
-            echo "sync_base=${sync_base}" >> "$GITHUB_OUTPUT"
-            echo "stage=rc" >> "$GITHUB_OUTPUT"
-          fi
-
-      - name: Sync commits into the release
-        if: steps.plan.outputs.sync_base != ''
-        uses: linear/linear-release-action@d4af10092984f9bc6d5efa075b242bdf01333463 # v0.18.0
-        with:
-          access_key: ${{ secrets.LINEAR_API_KEY }}
-          command: sync
-          name: LiteLLM ${{ steps.plan.outputs.version }}
-          version: ${{ steps.plan.outputs.version }}
-          base_ref: ${{ steps.plan.outputs.sync_base }}
-          cli_version: v0.18.0
-
-      - name: Keep the main stage unless the rc branch was cut during this run
-        id: main_stage
-        if: steps.plan.outputs.main == 'true'
-        env:
-          VERSION: ${{ steps.plan.outputs.version }}
-        run: |
-          set -euo pipefail
-          status=0
-          git ls-remote --exit-code --heads origin "rc/${VERSION}" > /dev/null || status=$?
-          case "${status}" in
-            0) echo "::notice::rc/${VERSION} was cut during this run; leaving the release in its rc stage" ;;
-            2) echo "stage=main" >> "$GITHUB_OUTPUT" ;;
-            *)
-              echo "::error::could not check whether rc/${VERSION} exists (git ls-remote exit ${status})"
-              exit 1
-              ;;
-          esac
-
-      - name: Move the release to its stage
-        if: steps.plan.outputs.stage != '' || steps.main_stage.outputs.stage != ''
-        uses: linear/linear-release-action@d4af10092984f9bc6d5efa075b242bdf01333463 # v0.18.0
-        with:
-          access_key: ${{ secrets.LINEAR_API_KEY }}
-          command: update
-          stage: ${{ steps.plan.outputs.stage || steps.main_stage.outputs.stage }}
-          version: ${{ steps.plan.outputs.version }}
-          cli_version: v0.18.0
-
-      - name: Complete the release
-        if: steps.plan.outputs.complete == 'true'
-        uses: linear/linear-release-action@d4af10092984f9bc6d5efa075b242bdf01333463 # v0.18.0
-        with:
-          access_key: ${{ secrets.LINEAR_API_KEY }}
-          command: complete
-          version: ${{ steps.plan.outputs.version }}
-          cli_version: v0.18.0

.github/workflows/mutation-test.yml

@@ -44,7 +44,6 @@ jobs:
           version: "0.10.9"
 
       - name: Cache uv dependencies
-        if: github.ref == 'refs/heads/main'
         uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
         with:
           path: |
@@ -54,17 +53,6 @@ jobs:
           restore-keys: |
             ${{ runner.os }}-uv-
 
-      - name: Cache uv dependencies
-        if: github.ref != 'refs/heads/main'
-        uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
-        with:
-          path: |
-            ~/.cache/uv
-            .venv
-          key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
-          restore-keys: |
-            ${{ runner.os }}-uv-
-
       - name: Cache the Rust build
         uses: ./.github/actions/cache-cargo-build
 

.github/workflows/osv-scan.yml

@@ -4,6 +4,8 @@ on:
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
   schedule:
     - cron: "23 6 * * *"

.github/workflows/publish-basedpyright-base-counts.yml

@@ -0,0 +1,66 @@
+name: Publish basedpyright base counts
+
+# Every commit on main or litellm_internal_staging can become a future merge-base.
+# Publishing its per-rule basedpyright counts as an artifact lets
+# scripts/type_check_gate.py download them in seconds instead of paying a
+# 60-110s second basedpyright pass on every fresh worktree or moved merge-base.
+# No concurrency group on purpose: runs must never cancel each other, because
+# every sha's artifact matters (any of them can become a merge-base).
+
+on:
+  push:
+    branches:
+      - main
+      - litellm_internal_staging
+  workflow_dispatch:
+    inputs:
+      ref:
+        description: "Ref to compute and publish base counts for (defaults to the workflow run's commit)"
+        required: false
+
+permissions:
+  contents: read
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          ref: ${{ inputs.ref || github.sha }}
+          clean: true
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Cache the Rust build
+        uses: ./.github/actions/cache-cargo-build
+
+      - name: Cache Prisma binaries
+        uses: ./.github/actions/cache-prisma-binaries
+
+      # The gate provisions its own measurement env (.venv-typecheck: a frozen
+      # uv sync of its canonical dependency groups plus a generated Prisma
+      # client), so no install step here can drift from what local runs measure.
+      - name: Emit basedpyright counts for HEAD
+        run: |
+          python scripts/type_check_gate.py --emit-counts-dir "$RUNNER_TEMP/basedpyright-counts"
+          counts_file=$(ls "$RUNNER_TEMP"/basedpyright-counts/basedpyright-counts-*.json)
+          echo "COUNTS_ARTIFACT_NAME=$(basename "$counts_file" .json)" >> "$GITHUB_ENV"
+
+      - name: Upload counts artifact
+        uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
+        with:
+          name: ${{ env.COUNTS_ARTIFACT_NAME }}
+          path: ${{ runner.temp }}/basedpyright-counts/
+          if-no-files-found: error

.github/workflows/publish-lint-base-counts.yml

@@ -1,106 +0,0 @@
-name: Publish lint base counts
-
-# Every commit on main can become a future merge-base.
-# Publishing its per-rule counts for each lint gate (strict ruff, type discipline,
-# test quality, basedpyright) as an artifact lets the gates download them through
-# scripts/lint_base_counts.py in seconds instead of scanning the merge-base tree in
-# a throwaway worktree on every fresh checkout or moved merge-base.
-# No concurrency group on purpose: runs must never cancel each other, because
-# every sha's artifact matters (any of them can become a merge-base).
-
-on:
-  push:
-    branches:
-      - main
-  workflow_dispatch:
-    inputs:
-      ref:
-        description: "Ref to compute and publish base counts for (defaults to the workflow run's commit)"
-        required: false
-
-permissions:
-  contents: read
-
-jobs:
-  publish:
-    runs-on: ubuntu-latest
-    timeout-minutes: 20
-    strategy:
-      fail-fast: false
-      matrix:
-        include:
-          - checker: ruff-strict
-            script: scripts/ruff_strict_gate.py
-          - checker: type-discipline
-            script: scripts/type_discipline_gate.py
-          - checker: test-quality
-            script: scripts/test_quality_gate.py
-          - checker: openapi-docs
-            script: scripts/openapi_docs_gate.py
-          - checker: basedpyright
-            script: scripts/type_check_gate.py
-
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          ref: ${{ inputs.ref || github.sha }}
-          clean: true
-          persist-credentials: false
-
-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: "3.12"
-
-      - name: Set up uv
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Cache the Rust build
-        if: matrix.checker == 'basedpyright'
-        uses: ./.github/actions/cache-cargo-build
-
-      - name: Cache Prisma binaries
-        if: matrix.checker == 'basedpyright' || matrix.checker == 'openapi-docs'
-        uses: ./.github/actions/cache-prisma-binaries
-
-      # The source scanners only need the pinned dev tools (ruff and the
-      # stdlib checkers), the same versions test-linting.yml's python job runs.
-      - name: Install the dev tools
-        if: matrix.checker != 'basedpyright' && matrix.checker != 'openapi-docs'
-        run: |
-          uv sync --frozen --only-group dev --no-install-project
-
-      # The OpenAPI gate imports the proxy app, so it needs the same install the
-      # test-linting.yml python job uses, Prisma client included.
-      - name: Install the proxy
-        if: matrix.checker == 'openapi-docs'
-        run: |
-          uv sync --frozen --group proxy-dev --group e2e-dev --extra proxy
-          uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma
-
-      - name: Emit ${{ matrix.checker }} counts for HEAD
-        if: matrix.checker != 'basedpyright'
-        run: |
-          uv run --no-sync python ${{ matrix.script }} --emit-counts-dir "$RUNNER_TEMP/lint-counts"
-
-      # The basedpyright gate provisions its own measurement env (.venv-typecheck:
-      # a frozen uv sync of its canonical dependency groups plus a generated Prisma
-      # client), so no install step here can drift from what local runs measure.
-      - name: Emit basedpyright counts for HEAD
-        if: matrix.checker == 'basedpyright'
-        run: |
-          python scripts/type_check_gate.py --emit-counts-dir "$RUNNER_TEMP/lint-counts"
-
-      - name: Name the counts artifact
-        run: |
-          counts_file=$(ls "$RUNNER_TEMP"/lint-counts/${{ matrix.checker }}-counts-*.json)
-          echo "COUNTS_ARTIFACT_NAME=$(basename "$counts_file" .json)" >> "$GITHUB_ENV"
-
-      - name: Upload counts artifact
-        uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
-        with:
-          name: ${{ env.COUNTS_ARTIFACT_NAME }}
-          path: ${{ runner.temp }}/lint-counts/
-          if-no-files-found: error

.github/workflows/spend-logs-guard.yml

@@ -1,45 +0,0 @@
-name: Spend logs guard
-
-on:
-  pull_request:
-    branches:
-      - main
-
-permissions:
-  contents: read
-  pull-requests: read
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
-  cancel-in-progress: true
-
-jobs:
-  spend-logs-guard:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          ref: ${{ github.event.pull_request.base.ref }}
-          persist-credentials: false
-      - name: Fetch the pull request head, its merge base and its reviews
-        id: revisions
-        env:
-          GH_TOKEN: ${{ github.token }}
-          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
-          PR_NUMBER: ${{ github.event.pull_request.number }}
-          BASE_REF: ${{ github.event.pull_request.base.ref }}
-        run: |
-          merge_base="$(gh api "repos/${GITHUB_REPOSITORY}/compare/${BASE_REF}...${HEAD_SHA}" --jq '.merge_base_commit.sha')"
-          git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA"
-          gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" --paginate --slurp | jq 'add // []' > "$RUNNER_TEMP/reviews.json"
-          echo "merge_base=$merge_base" >> "$GITHUB_OUTPUT"
-      - name: Set up uv
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-      - name: Run the guard
-        env:
-          MERGE_BASE: ${{ steps.revisions.outputs.merge_base }}
-          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
-        run: |
-          uv run --no-project --with pydantic==2.14.1 python ci_cd/spend_logs_guard.py --base "$MERGE_BASE" --head "$HEAD_SHA" --approver kerry-berri --reviews "$RUNNER_TEMP/reviews.json"

.github/workflows/sync-together-ai-models.yml

@@ -0,0 +1,70 @@
+name: Sync Together AI model registry
+
+on:
+  schedule:
+    - cron: "30 6 * * *"
+  workflow_dispatch:
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  sync_together_ai_models:
+    if: github.repository == 'BerriAI/litellm'
+    runs-on: ubuntu-latest
+    env:
+      BASE_BRANCH: ${{ github.event.repository.default_branch }}
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          ref: ${{ env.BASE_BRANCH }}
+          persist-credentials: false
+      - name: Set up uv
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+      - name: Look for an already-open sync PR
+        id: existing
+        run: |
+          open_pr="$(gh pr list --repo "$GITHUB_REPOSITORY" --state open --limit 1000 --json headRefName \
+            --jq '[.[].headRefName | select(startswith("litellm_together_registry_sync_"))] | first // empty')"
+          echo "open_pr=$open_pr" >> "$GITHUB_OUTPUT"
+          if [ -n "$open_pr" ]; then
+            echo "An open sync PR already exists on branch $open_pr; skipping this run."
+          fi
+        env:
+          GH_TOKEN: ${{ secrets.GH_TOKEN || github.token }}
+      - name: Run the sync
+        if: steps.existing.outputs.open_pr == ''
+        run: |
+          uv run --frozen python scripts/sync_together_ai_models.py --write --pr-body-file "$RUNNER_TEMP/pr_body.md"
+        env:
+          TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
+      - name: Regenerate the JSON schema
+        if: steps.existing.outputs.open_pr == ''
+        run: |
+          uv run --frozen python ci_cd/generate_model_prices_schema.py
+      - name: Create a pull request when the registry changed
+        if: steps.existing.outputs.open_pr == ''
+        run: |
+          if git diff --quiet; then
+            echo "Registry already in sync; no PR needed."
+            exit 0
+          fi
+          branch="litellm_together_registry_sync_$(date +'%Y-%m-%d')"
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          git checkout -b "$branch"
+          git add model_prices_and_context_window.json \
+            litellm/model_prices_and_context_window_backup.json \
+            model_prices_and_context_window.schema.json
+          git commit -m "feat(models): sync together_ai model registry $(date +'%Y-%m-%d')"
+          gh auth setup-git
+          git push origin "$branch"
+          gh pr create --title "feat(models): sync together_ai model registry" \
+            --body-file "$RUNNER_TEMP/pr_body.md" \
+            --head "$branch" \
+            --base "$BASE_BRANCH"
+        env:
+          GH_TOKEN: ${{ secrets.GH_TOKEN || github.token }}

.github/workflows/test-code-quality.yml

@@ -0,0 +1,192 @@
+name: Code Quality Checks
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+  push:
+    branches:
+      - main
+      - litellm_internal_staging
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  code-quality:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Checkout litellm-docs into docs/my-website (for documentation_tests)
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          repository: BerriAI/litellm-docs
+          path: docs/my-website
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Cache uv dependencies
+        uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+        with:
+          path: |
+            ~/.cache/uv
+            .venv
+          key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-uv-
+
+      - name: Cache the Rust build
+        uses: ./.github/actions/cache-cargo-build
+
+      - name: Install dependencies
+        run: uv sync --frozen --all-groups --all-extras
+
+      - name: check_licenses
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_licenses.py
+
+      - name: check_provider_folders_documented
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_provider_folders_documented.py
+
+      - name: check_prisma_binary_cache
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_prisma_binary_cache.py
+
+      - name: check_workflow_startup_safety
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_workflow_startup_safety.py
+
+      - name: check_workflow_job_name_collisions
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_workflow_job_name_collisions.py
+
+      - name: test_workflow_job_name_collisions
+        run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_workflow_job_name_collisions.py
+
+      - name: test_e2e_changed_gate
+        run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_e2e_changed_gate.py tests/code_coverage_tests/test_e2e_idp_stack.py
+
+      - name: router_code_coverage
+        run: uv run --no-sync python ./tests/code_coverage_tests/router_code_coverage.py
+
+      - name: test_chat_completion_imports
+        run: uv run --no-sync python ./tests/code_coverage_tests/test_chat_completion_imports.py
+
+      - name: info_log_check
+        run: uv run --no-sync python ./tests/code_coverage_tests/info_log_check.py
+
+      - name: check_guardrail_apply_decorator
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_guardrail_apply_decorator.py
+
+      - name: test_ban_set_verbose
+        run: uv run --no-sync python ./tests/code_coverage_tests/test_ban_set_verbose.py
+
+      - name: code_qa_check_tests
+        run: uv run --no-sync python ./tests/code_coverage_tests/code_qa_check_tests.py
+
+      - name: check_get_model_cost_key_performance
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_get_model_cost_key_performance.py
+
+      - name: test_proxy_types_import
+        run: uv run --no-sync python ./tests/code_coverage_tests/test_proxy_types_import.py
+
+      - name: callback_manager_test
+        run: uv run --no-sync python ./tests/code_coverage_tests/callback_manager_test.py
+
+      - name: recursive_detector
+        run: uv run --no-sync python ./tests/code_coverage_tests/recursive_detector.py
+
+      - name: test_router_strategy_async
+        run: uv run --no-sync python ./tests/code_coverage_tests/test_router_strategy_async.py
+
+      - name: litellm_logging_code_coverage
+        run: uv run --no-sync python ./tests/code_coverage_tests/litellm_logging_code_coverage.py
+
+      - name: ensure_async_clients_test
+        run: uv run --no-sync python ./tests/code_coverage_tests/ensure_async_clients_test.py
+
+      - name: enforce_llms_folder_style
+        run: uv run --no-sync python ./tests/code_coverage_tests/enforce_llms_folder_style.py
+
+      - name: prevent_key_leaks_in_exceptions
+        run: uv run --no-sync python ./tests/code_coverage_tests/prevent_key_leaks_in_exceptions.py
+
+      - name: check_unsafe_enterprise_import
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_unsafe_enterprise_import.py
+
+      - name: ban_copy_deepcopy_kwargs
+        run: uv run --no-sync python ./tests/code_coverage_tests/ban_copy_deepcopy_kwargs.py
+
+      - name: check_fastuuid_usage
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_fastuuid_usage.py
+
+      - name: check_py310_typing_imports
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_py310_typing_imports.py
+
+      - name: check_e2e_no_raw_requests
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_e2e_no_raw_requests.py
+
+      - name: check_migrations_no_data_rewrites
+        run: uv run --no-sync python ./tests/code_coverage_tests/check_migrations_no_data_rewrites.py
+
+      - name: memory_test
+        run: uv run --no-sync python ./tests/code_coverage_tests/memory_test.py
+
+      - name: documentation_test_env_keys
+        run: uv run --no-sync python ./tests/documentation_tests/test_env_keys.py
+
+      - name: documentation_test_router_settings
+        run: uv run --no-sync python ./tests/documentation_tests/test_router_settings.py
+
+      - name: documentation_test_api_docs
+        run: uv run --no-sync python ./tests/documentation_tests/test_api_docs.py
+
+  python-310-import-smoke:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.10"
+
+      - name: Set up uv
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Install dependencies
+        run: uv sync --frozen --extra proxy --extra cli --python 3.10
+
+      - run: uv run --no-sync python --version
+
+      - name: Import litellm
+        run: uv run --no-sync python -c "import litellm"
+
+      - name: Check litellm CLI
+        run: uv run --no-sync litellm --version
+
+      - name: Check lite CLI
+        run: uv run --no-sync lite version

.github/workflows/test-e2e-changed.yml

@@ -0,0 +1,243 @@
+name: e2e-changed-tests
+
+on:
+  pull_request:
+
+concurrency:
+  group: e2e-changed-${{ github.event.pull_request.number }}
+  cancel-in-progress: true
+
+permissions: {}
+
+jobs:
+  detect:
+    name: Detect changed e2e tests
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    permissions:
+      contents: read
+      pull-requests: read
+    outputs:
+      tests: ${{ steps.changed.outputs.tests }}
+      any: ${{ steps.changed.outputs.any }}
+    steps:
+      - name: Checkout the selector and the canary suite
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          sparse-checkout: |
+            .github/e2e-stack
+            tests/e2e/access_control
+            tests/e2e/management/test_jwt_management_e2e.py
+            tests/e2e/other/test_jwt_auth_e2e.py
+          persist-credentials: false
+          ref: ${{ github.sha }}
+
+      - name: List the e2e test files this PR added or modified
+        id: changed
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REPO: ${{ github.repository }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+        run: |
+          gh api "repos/${REPO}/pulls/${PR_NUMBER}" \
+            --jq 'select(.head.sha == env.HEAD_SHA and .changed_files < 3000) | .head.sha' \
+            | grep -Fxq "${HEAD_SHA}"
+          files="$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/files" --paginate \
+            --jq '.[] | select(.status != "removed") | .filename')"
+          gh api "repos/${REPO}/pulls/${PR_NUMBER}" --jq '.head.sha' | grep -Fxq "${HEAD_SHA}"
+          tests="$(printf '%s\n' "${files}" \
+            | python3 .github/e2e-stack/select_tests.py tests/e2e/access_control/test_*.py \
+                tests/e2e/management/test_jwt_management_e2e.py tests/e2e/other/test_jwt_auth_e2e.py)"
+          echo "tests=${tests}" >> "${GITHUB_OUTPUT}"
+          if [ -n "${tests}" ]; then
+            echo "any=true" >> "${GITHUB_OUTPUT}"
+            echo "selected e2e tests: ${tests}"
+          else
+            echo "any=false" >> "${GITHUB_OUTPUT}"
+            echo "no changed e2e test files supported by this stack; nothing to run"
+          fi
+
+  run:
+    name: Run changed e2e tests against the stage-mirror stack
+    needs: detect
+    if: needs.detect.outputs.any == 'true' && github.event.pull_request.head.repo.full_name == github.repository
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    environment: e2e-changed
+    permissions:
+      contents: read
+      id-token: write
+    services:
+      postgres:
+        image: postgres:16.6
+        env:
+          POSTGRES_USER: litellm
+          POSTGRES_PASSWORD: dbpassword9090
+          POSTGRES_DB: litellm
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U litellm"
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 10
+      jaeger:
+        image: jaegertracing/jaeger:2.10.0
+        ports:
+          - 4318:4318
+          - 16686:16686
+    steps:
+      - name: Validate configuration
+        env:
+          ROLE: ${{ vars.E2E_AWS_ROLE_TO_ASSUME }}
+        run: test -n "${ROLE}" || { echo "::error::Set repo variable E2E_AWS_ROLE_TO_ASSUME to an OIDC role with read access to the e2e secrets"; exit 1; }
+
+      - name: Checkout
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+          ref: ${{ github.sha }}
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.13"
+
+      - name: Set up uv
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Cache the Rust build
+        uses: ./.github/actions/cache-cargo-build
+
+      - name: Install dependencies
+        run: |
+          .github/scripts/uv_sync_with_retries.sh --frozen \
+            --extra proxy --extra proxy-runtime --extra extra_proxy \
+            --extra semantic-router --extra bedrock-realtime \
+            --group ci --group proxy-dev --group e2e-dev
+          uv pip install "pipecat-ai[openai]==1.4.0"
+
+      - name: Cache Prisma binaries
+        uses: ./.github/actions/cache-prisma-binaries
+
+      - name: Generate Prisma client
+        run: uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma
+
+      - name: Install Playwright chromium
+        run: uv run --no-sync playwright install --with-deps chromium
+
+      - name: Configure AWS credentials
+        id: aws
+        uses: aws-actions/configure-aws-credentials@e7f100cf4c008499ea8adda475de1042d6975c7b # v6.2.0
+        with:
+          role-to-assume: ${{ vars.E2E_AWS_ROLE_TO_ASSUME }}
+          aws-region: us-east-1
+          role-session-name: litellm-e2e-changed-${{ github.run_id }}
+          role-duration-seconds: 900
+          output-env-credentials: false
+          output-credentials: true
+
+      - name: Fetch provider credentials from AWS Secrets Manager
+        env:
+          AWS_ACCESS_KEY_ID: ${{ steps.aws.outputs.aws-access-key-id }}
+          AWS_SECRET_ACCESS_KEY: ${{ steps.aws.outputs.aws-secret-access-key }}
+          AWS_SESSION_TOKEN: ${{ steps.aws.outputs.aws-session-token }}
+          AWS_DEFAULT_REGION: us-east-1
+        run: |
+          umask 077
+          aws secretsmanager get-secret-value --secret-id litellm-e2e-changed-provider-keys \
+            --query SecretString --output text \
+            | uv run --no-sync python .github/e2e-stack/secrets_to_env.py tests/e2e/.env
+          aws secretsmanager get-secret-value --secret-id litellm-e2e-changed-license \
+            --query SecretString --output text \
+            | jq -R -s '{"LITELLM_LICENSE": .}' \
+            | uv run --no-sync python .github/e2e-stack/secrets_to_env.py tests/e2e/.env
+
+      - name: Boot the stage-mirror stack
+        id: boot
+        run: |
+          umask 077
+          if ! bash .github/e2e-stack/up.sh > "${RUNNER_TEMP}/e2e-boot.log" 2>&1; then
+            echo "::error::stage-mirror stack failed to boot; raw logs are not published"
+            exit 1
+          fi
+
+      - name: Export stack environment
+        run: |
+          master_key="$(grep '^LITELLM_MASTER_KEY=' "${RUNNER_TEMP}/litellm-e2e-stack/stack.env" | cut -d= -f2-)"
+          echo "::add-mask::${master_key}"
+          cat "${RUNNER_TEMP}/litellm-e2e-stack/stack.env" >> "${GITHUB_ENV}"
+
+      - name: Run the selected tests three times
+        env:
+          TESTS: ${{ needs.detect.outputs.tests }}
+          E2E_FIXTURE_MODE: live
+        run: |
+          umask 077
+          read -r -a test_files <<< "${TESTS}"
+          for pass in 1 2 3; do
+            report="${RUNNER_TEMP}/e2e-pass-${pass}.xml"
+            log="${RUNNER_TEMP}/e2e-pass-${pass}.log"
+            echo "::group::pass ${pass} of 3"
+            set +e
+            uv run --no-sync pytest "${test_files[@]}" --rootdir=. -v --reruns 0 -p no:cacheprovider \
+              -o junit_family=xunit1 --junitxml="${report}" > "${log}" 2>&1
+            status=$?
+            uv run --no-sync python .github/e2e-stack/assert_tests_ran.py "${report}" "${test_files[@]}"
+            verified=$?
+            set -e
+            grep -E '^=+ .* in [0-9.]+s( \([0-9:]+\))? =+$' "${log}" | tail -n 1
+            echo "::endgroup::"
+            if [ "${status}" = "5" ]; then
+              echo "::error::the selected files collected no runnable tests, so nothing was verified"
+              exit 1
+            fi
+            if [ "${status}" != "0" ]; then
+              echo "::error::pass ${pass} of 3 failed with exit code ${status}"
+              exit "${status}"
+            fi
+            if [ "${verified}" != "0" ]; then
+              echo "::error::pass ${pass} of 3 did not verify every selected file"
+              exit 1
+            fi
+            echo "pass ${pass} of 3 passed"
+          done
+
+      - name: Stop the stack
+        if: always() && steps.boot.outcome != 'skipped'
+        run: bash .github/e2e-stack/down.sh
+
+      - name: Remove credentials and raw output
+        if: always()
+        run: |
+          rm -f tests/e2e/.env "${RUNNER_TEMP}/e2e-boot.log" "${RUNNER_TEMP}"/e2e-pass-*.log "${RUNNER_TEMP}"/e2e-pass-*.xml
+          rm -rf "${RUNNER_TEMP}/litellm-e2e-stack"
+
+  gate:
+    name: e2e-changed-tests
+    needs: [detect, run]
+    if: always()
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    steps:
+      - name: Require three successful passes when tests changed
+        env:
+          DETECT_RESULT: ${{ needs.detect.result }}
+          ANY_TESTS: ${{ needs.detect.outputs.any }}
+          RUN_RESULT: ${{ needs.run.result }}
+        run: |
+          if [ "${DETECT_RESULT}" != "success" ]; then
+            echo "::error::changed-test detection did not succeed"
+            exit 1
+          fi
+          if [ "${ANY_TESTS}" = "false" ]; then
+            echo "no changed e2e test files supported by this stack; nothing to run"
+            exit 0
+          fi
+          if [ "${ANY_TESTS}" != "true" ] || [ "${RUN_RESULT}" != "success" ]; then
+            echo "::error::selected e2e tests require an approved, successful run; fork PRs must run from a reviewed same-repository branch"
+            exit 1
+          fi

.github/workflows/test-e2e-redis-chaos.yml

@@ -91,12 +91,12 @@ jobs:
 
       - name: Run the Redis chaos load test
         env:
-          E2E_PERF_TESTS: "1"
+          E2E_REDIS_CHAOS: "1"
           LITELLM_PROXY_URL: http://localhost:4000
           REDIS_HOST: 127.0.0.1
           REDIS_PORT: "6379"
         run: |
-          uv run --no-sync pytest tests/e2e/perf/test_redis_chaos_e2e.py -v --tb=short -rA -s
+          uv run --no-sync pytest tests/e2e/load/test_redis_chaos_e2e.py -v --tb=short -rA -s
 
       - name: Show proxy log on failure
         if: failure()

.github/workflows/test-integration.yml

@@ -1,199 +0,0 @@
-name: integration
-
-on:
-  pull_request:
-    types: [labeled]
-  schedule:
-    - cron: "5 8-20/3 * * 1-6"
-      timezone: America/Los_Angeles
-  workflow_dispatch:
-    inputs:
-      suite:
-        description: Integration group to run
-        type: choice
-        options:
-          - all
-          - providers
-          - extensions
-          - accounting
-          - management
-          - management-replica
-          - database
-          - database-replica
-          - mcp
-          - security
-          - cost
-          - sdk
-          - browser
-        default: all
-      workers:
-        description: Workers per group, blank keeps each group's default
-        type: string
-        default: ""
-
-permissions:
-  contents: read
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}-${{ github.event.label.name }}-${{ inputs.suite }}-${{ inputs.workers }}
-  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
-
-jobs:
-  integration:
-    name: integration ${{ matrix.suite }}
-    if: github.event_name != 'pull_request' || github.event.label.name == 'run-ci'
-    environment: integration-license
-    runs-on: ${{ fromJSON('{"providers":"ubuntu-latest-64-cores","extensions":"ubuntu-latest-64-cores","accounting":"ubuntu-latest-32-cores","management":"ubuntu-latest-32-cores","management-replica":"ubuntu-latest-32-cores"}')[matrix.suite] || 'ubuntu-latest-16-cores' }}
-    timeout-minutes: 60
-    strategy:
-      fail-fast: false
-      matrix:
-        suite: ${{ fromJSON((inputs.suite || 'all') == 'all' && '["providers","extensions","accounting","management","management-replica","database","database-replica","mcp","security","cost","sdk","browser"]' || format('["{0}"]', inputs.suite)) }}
-    env:
-      UV_PYTHON: "3.12"
-      INTEGRATION_WORKERS: ${{ inputs.workers || fromJSON('{"providers":"32","extensions":"48","accounting":"16","management":"16","management-replica":"16","cost":"16","mcp":"8","security":"8","database":"4","database-replica":"4"}')[matrix.suite] || '' }}
-    services:
-      postgres:
-        image: postgres:16@sha256:e17e86066e5ef83e0952a9347f5c792b7ece00972e2aa787a6986f471b3dd3d5
-        command: -c max_connections=1000 -c shared_preload_libraries=pg_stat_statements -c pg_stat_statements.track=all -c pg_stat_statements.max=20000
-        env:
-          POSTGRES_USER: postgres
-          POSTGRES_PASSWORD: postgres
-          POSTGRES_DB: circle_test
-        ports:
-          - 5432:5432
-        options: >-
-          --shm-size 1g
-          --health-cmd "pg_isready -U postgres"
-          --health-interval 5s
-          --health-timeout 5s
-          --health-retries 20
-      redis:
-        image: redis:7-alpine@sha256:7aec734b2bb298a1d769fd8729f13b8514a41bf90fcdd1f38ec52267fbaa8ee6
-        ports:
-          - 6379:6379
-        options: >-
-          --health-cmd "redis-cli ping"
-          --health-interval 5s
-          --health-timeout 5s
-          --health-retries 20
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Set up Python
-        timeout-minutes: 3
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: ${{ env.UV_PYTHON }}
-
-      - name: Set up uv
-        timeout-minutes: 3
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Cache the Rust build
-        timeout-minutes: 5
-        uses: ./.github/actions/cache-cargo-build
-
-      - name: Cache Prisma binaries
-        timeout-minutes: 5
-        uses: ./.github/actions/cache-prisma-binaries
-
-      - name: Install dependencies
-        timeout-minutes: 20
-        run: .github/scripts/uv_sync_with_retries.sh --frozen --all-groups --all-extras
-
-      - name: Install redis-server 7.2.16
-        timeout-minutes: 3
-        run: |
-          mkdir -p "$RUNNER_TEMP/redis-bin"
-          cid="$(docker create redis:7.2.16@sha256:0637954999d01b7c9ce9167db2da50656e2590d3b884f1c600c5f63bb6e6773c)"
-          docker cp "${cid}:/usr/local/bin/redis-server" "$RUNNER_TEMP/redis-bin/redis-server"
-          docker rm "$cid"
-          "$RUNNER_TEMP/redis-bin/redis-server" --version | grep -q 'v=7.2.16'
-          echo "$RUNNER_TEMP/redis-bin" >> "$GITHUB_PATH"
-
-      - name: Install MCP SDK1 peer
-        if: matrix.suite == 'mcp'
-        timeout-minutes: 5
-        run: |
-          uv venv --python 3.12 .venv-mcp-peer
-          uv pip install --no-config --exclude-newer "3 days" --python .venv-mcp-peer 'mcp==1.28.1' 'langchain-mcp-adapters==0.2.1'
-          echo "MCP_TEST_PEER_PYTHON=$PWD/.venv-mcp-peer/bin/python" >> "$GITHUB_ENV"
-
-      - name: Set up Node.js
-        if: matrix.suite == 'browser'
-        timeout-minutes: 5
-        uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
-        with:
-          node-version-file: ui/litellm-dashboard/.nvmrc
-          cache: "npm"
-          cache-dependency-path: |
-            ui/litellm-dashboard/package-lock.json
-            tests/e2e/ui/package-lock.json
-
-      - name: Install browser dependencies and build the dashboard
-        if: matrix.suite == 'browser'
-        timeout-minutes: 20
-        run: |
-          (cd ui/litellm-dashboard && npm ci)
-          (cd tests/e2e/ui && npm ci)
-          sudo env PATH="$PATH" DEBIAN_FRONTEND=noninteractive \
-            node tests/e2e/ui/node_modules/@playwright/test/cli.js install-deps chromium
-          node tests/e2e/ui/node_modules/@playwright/test/cli.js install chromium
-          (cd ui/litellm-dashboard && NEXT_TELEMETRY_DISABLED=1 npm run build)
-
-      - name: Run integration contracts
-        timeout-minutes: 45
-        env:
-          SUITE: ${{ matrix.suite }}
-          INTEGRATION_POSTGRES_CONTAINER: ${{ job.services.postgres.id }}
-          INTEGRATION_REDIS_CONTAINER: ${{ job.services.redis.id }}
-          LITELLM_LICENSE: ${{ secrets.LITELLM_LICENSE }}
-        run: |
-          case "$SUITE" in
-            *-replica) bash .circleci/scripts/run_integration.sh "${SUITE%-replica}" replica ;;
-            *) bash .circleci/scripts/run_integration.sh "$SUITE" ;;
-          esac
-
-      - name: Verify every case ran exactly once
-        if: ${{ !cancelled() && matrix.suite != 'browser' }}
-        timeout-minutes: 2
-        run: |
-          record="$(find test-results -mindepth 2 -maxdepth 2 -name execution.json -print -quit)"
-          test -n "$record"
-          mkdir -p "$RUNNER_TEMP/shards/0"
-          cp "$record" "$RUNNER_TEMP/shards/0/execution.json"
-          .venv/bin/python -m tests.integration._support.sharding "$RUNNER_TEMP/shards" --shards 1
-
-      - name: Report MCP coverage
-        if: ${{ !cancelled() && matrix.suite == 'mcp' }}
-        timeout-minutes: 5
-        run: |
-          coverage_dir=test-results/integration-mcp/coverage
-          test -f "$coverage_dir/coverage.txt" && exit 0
-          export COVERAGE_FILE="$PWD/$coverage_dir/data"
-          .venv/bin/python -m coverage combine --rcfile=tests/integration/mcp_coverage.toml
-          .venv/bin/python -m coverage report --rcfile=tests/integration/mcp_coverage.toml > "$coverage_dir/coverage.txt"
-          .venv/bin/python -m coverage html --rcfile=tests/integration/mcp_coverage.toml -d "$coverage_dir/html"
-          tail -n 1 "$coverage_dir/coverage.txt"
-
-      - name: Check results for the license
-        id: secret-scan
-        if: ${{ !cancelled() }}
-        timeout-minutes: 5
-        env:
-          SCANNED_SECRET: ${{ secrets.LITELLM_LICENSE }}
-        run: python .github/scripts/assert_secret_absent.py test-results
-
-      - name: Upload results
-        if: ${{ !cancelled() && steps.secret-scan.outcome == 'success' }}
-        timeout-minutes: 10
-        uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
-        with:
-          name: integration-${{ matrix.suite }}-${{ github.run_id }}-${{ github.run_attempt }}
-          path: test-results/
-          retention-days: 14

.github/workflows/test-linting.yml

@@ -1,9 +1,11 @@
-name: lint
+name: LiteLLM Linting
 
 on:
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
 
 permissions:
@@ -14,26 +16,22 @@ concurrency:
   cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
 jobs:
-  python:
-    name: python
+  lint:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    # actions: read lets scripts/type_check_gate.py download the base-counts
+    # artifact published by publish-basedpyright-base-counts.yml instead of
+    # re-running basedpyright over the merge-base tree.
     permissions:
       contents: read
       pull-requests: read
       actions: read
-    runs-on: ubuntu-latest
-    timeout-minutes: 20
-    services:
-      redis:
-        image: redis@sha256:e2debfb7956fa12c7ddc79d7e645c8cf26b30c99a6e9161ea9bf4171e1668a5f
-        ports:
-          - 6379:6379
-        options: >-
-          --health-cmd "redis-cli ping"
-          --health-interval 10s
-          --health-timeout 5s
-          --health-retries 5
+
     steps:
       - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        # Check out the PR head, not the default refs/pull/N/merge: the merge ref
+        # folds in newer base commits, which the diff-based gates (ruff delta,
+        # Any-discipline) would otherwise blame on this branch.
         with:
           ref: ${{ github.event.pull_request.head.sha }}
           fetch-depth: 1
@@ -48,13 +46,11 @@ jobs:
         if: steps.changes.outputs.decision != 'skip'
         env:
           GH_TOKEN: ${{ github.token }}
-          PR_NUMBER: ${{ github.event.pull_request.number }}
-          EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
           HEAD_SHA: ${{ github.event.pull_request.head.sha }}
         run: |
           retry() { "$@" || { sleep 15; "$@"; } || { sleep 30; "$@"; }; }
-          BASE_REF=$(gh api "repos/${GITHUB_REPOSITORY}/branches/${EVENT_BASE_REF}" --jq .name) || BASE_REF=$(retry gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq .base.ref)
-          MERGE_BASE=$(retry gh api "repos/${GITHUB_REPOSITORY}/compare/${BASE_REF}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
+          MERGE_BASE=$(retry gh api "repos/${{ github.repository }}/compare/${BASE_SHA}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
           test -n "$MERGE_BASE"
           retry git fetch --no-tags --depth=1 origin "$MERGE_BASE"
           echo "GATE_BASE_SHA=$MERGE_BASE" >> "$GITHUB_ENV"
@@ -106,6 +102,9 @@ jobs:
         if: steps.changes.outputs.decision != 'skip'
         uses: ./.github/actions/cache-prisma-binaries
 
+      # basedpyright resolves Prisma's generated client (litellm/proxy/schema.prisma)
+      # only after `prisma generate` writes prisma/client.py et al. Without this the
+      # DB wrappers typed against the generated client would degrade to Unknown.
       - name: Generate Prisma client
         if: steps.changes.outputs.decision != 'skip'
         run: |
@@ -131,10 +130,6 @@ jobs:
           echo "File content around line 43:"
           head -50 litellm/litellm_core_utils/custom_logger_registry.py | tail -10
 
-      - name: Check MCP operation boundary
-        if: steps.changes.outputs.decision != 'skip'
-        run: uv run --no-sync python scripts/check_mcp_operation_boundary.py
-
       - name: Run Ruff linting
         if: steps.changes.outputs.decision != 'skip'
         run: |
@@ -147,41 +142,27 @@ jobs:
         run: |
           uv run --no-sync ruff check --config ruff-tests.toml tests
 
-      - name: Check strict ruff rules (delta vs merge-base counts)
+      - name: Check strict-rule budget (delta vs base)
         if: steps.changes.outputs.decision != 'skip'
-        env:
-          GH_TOKEN: ${{ github.token }}
         run: |
           uv run --no-sync python scripts/ruff_strict_gate.py --base "$GATE_BASE_SHA"
 
-      - name: Check type discipline (mutable collections / casts / type guards / kwargs / unexplained suppressions, delta vs merge-base counts)
+      - name: Check type-discipline budget (mutable collections / casts / type guards / kwargs / unexplained suppressions, delta vs base)
         if: steps.changes.outputs.decision != 'skip'
-        env:
-          GH_TOKEN: ${{ github.token }}
         run: |
           uv run --no-sync python scripts/type_discipline_gate.py --base "$GATE_BASE_SHA"
 
-      - name: Check test quality (zero-assert / mock-echo tests, sys.path.insert, raw env writes, litellm global mutation, credential-gated skips, conftest snapshot inventory, delta vs merge-base counts)
+      - name: Check test-quality budget (zero-assert / mock-echo tests, sys.path.insert, raw env writes, litellm global mutation, credential-gated skips, conftest snapshot inventory, delta vs base)
         if: steps.changes.outputs.decision != 'skip'
-        env:
-          GH_TOKEN: ${{ github.token }}
         run: |
           uv run --no-sync python scripts/test_quality_gate.py --base "$GATE_BASE_SHA"
 
-      - name: Check OpenAPI docs (POST routes without a request schema, description or 200 schema, delta vs merge-base counts)
-        if: steps.changes.outputs.decision != 'skip'
-        env:
-          GH_TOKEN: ${{ github.token }}
-        run: |
-          uv sync --frozen --group proxy-dev --group e2e-dev --extra proxy
-          uv run --no-sync python scripts/openapi_docs_gate.py --base "$GATE_BASE_SHA"
-
       - name: Print OpenAI version
         if: steps.changes.outputs.decision != 'skip'
         run: |
           uv run --no-sync python -c "import openai; print(f'OpenAI version: {openai.__version__}')"
 
-      - name: Check basedpyright (delta vs merge-base counts)
+      - name: Check basedpyright budget (delta vs base)
         if: steps.changes.outputs.decision != 'skip'
         env:
           GH_TOKEN: ${{ github.token }}
@@ -191,30 +172,12 @@ jobs:
       - name: Check tests/e2e basedpyright (zero errors)
         if: steps.changes.outputs.decision != 'skip'
         run: |
-          if git diff --name-only --diff-filter=ACMRD "$GATE_BASE_SHA" HEAD -- ':(glob)tests/e2e/**/*.py' ':(glob)tests/e2e_harness/**/*.py' pyrightconfig.json | grep -q .; then
-            uv run --no-sync basedpyright tests/e2e tests/e2e_harness
+          if git diff --name-only --diff-filter=ACMRD "$GATE_BASE_SHA" HEAD -- ':(glob)tests/e2e/**/*.py' | grep -q .; then
+            uv run --no-sync basedpyright tests/e2e
           else
             echo "No changed tests/e2e Python files; skipping."
           fi
 
-      - name: Run the e2e harness tests
-        if: steps.changes.outputs.decision != 'skip'
-        env:
-          LITELLM_MASTER_KEY: sk-e2e-harness-tests-reach-no-proxy
-          E2E_CACHE_TEST_REDIS_URL: redis://127.0.0.1:6379/0
-        run: |
-          if ! git diff --name-only --diff-filter=ACMRD "$GATE_BASE_SHA" HEAD -- tests/e2e tests/e2e_harness ':(exclude)tests/e2e/ui' tests/code_coverage_tests/test_provider_replay_harness.py tests/code_coverage_tests/test_provider_cache.py pyproject.toml uv.lock .github/workflows/test-linting.yml | grep -q .; then
-            echo "No changed e2e harness files; skipping."
-            exit 0
-          fi
-          retry() { "$@" || { sleep 15; "$@"; } || { sleep 30; "$@"; }; }
-          CLAUDE_VERSION="$(retry uv run --no-sync python tests/e2e/cli_harness/npm_version_resolver.py @anthropic-ai/claude-code)"
-          tests/e2e/claude_code/cron_vm/install_claude_code.sh "$CLAUDE_VERSION" "$RUNNER_TEMP/claude-cli"
-          PATH="$RUNNER_TEMP/claude-cli:$PATH" uv run --no-sync pytest -q tests/e2e_harness
-          uv run --no-sync pytest --tb=short -q --noconftest -o addopts= -o "pythonpath=tests/e2e tests/e2e_harness" -p no:rerunfailures \
-            tests/code_coverage_tests/test_provider_replay_harness.py \
-            tests/code_coverage_tests/test_provider_cache.py
-
       - name: Check for circular imports
         if: steps.changes.outputs.decision != 'skip'
         run: |
@@ -226,11 +189,47 @@ jobs:
         if: steps.changes.outputs.decision != 'skip'
         run: |
           uv run --no-sync python -c "from litellm import *" || (echo '🚨 import failed, this means you introduced unprotected imports! 🚨'; exit 1)
-  secret-scan:
-    permissions:
-      contents: read
+
+  # Intentionally NON-GATING. This job turns red when a *-budget.json ceiling is
+  # raised (or a rule/budget is dropped) so a loosening is obvious in review, but it
+  # must be kept OUT of the branch-protection required-checks list so a justified
+  # bump can still be merged by a human who has seen and accepted the red.
+  budget-ratchet:
     runs-on: ubuntu-latest
     timeout-minutes: 5
+    permissions:
+      contents: read
+
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          fetch-depth: 1
+          persist-credentials: false
+
+      - name: Fetch ratchet base
+        env:
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+        run: |
+          retry() { "$@" || { sleep 15; "$@"; } || { sleep 30; "$@"; }; }
+          retry git fetch --no-tags --depth=1 origin "$BASE_SHA"
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Ratchet check (budgets may only decrease; non-gating)
+        env:
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+        run: |
+          python scripts/budget_ratchet_check.py --base "$BASE_SHA"
+
+  secret-scan:
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    permissions:
+      contents: read
+
     steps:
       - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
         with:
@@ -262,459 +261,3 @@ jobs:
           else
             echo "GITGUARDIAN_API_KEY not set, skipping ggshield scan"
           fi
-  ui:
-    name: ui
-    permissions:
-      contents: read
-      pull-requests: read
-    runs-on: ubuntu-latest
-    timeout-minutes: 8
-    defaults:
-      run:
-        working-directory: ui/litellm-dashboard
-
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          fetch-depth: 1
-          persist-credentials: false
-
-      - name: Collect changed files
-        id: changed
-        env:
-          GH_TOKEN: ${{ github.token }}
-          PR_NUMBER: ${{ github.event.pull_request.number }}
-          EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
-          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
-        run: |
-          base_ref=$(gh api "repos/${GITHUB_REPOSITORY}/branches/${EVENT_BASE_REF}" --jq .name) || base_ref=$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq .base.ref)
-          merge_base=$(gh api "repos/${GITHUB_REPOSITORY}/compare/${base_ref}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
-          test -n "$merge_base"
-          git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA"
-          : > "$RUNNER_TEMP/prettier_files.txt"
-          : > "$RUNNER_TEMP/eslint_files.txt"
-          while IFS= read -r f; do
-            [ -f "$f" ] || continue
-            case "$f" in
-              *.js | *.jsx | *.ts | *.tsx | *.mjs | *.cjs)
-                printf '%s\n' "$f" >> "$RUNNER_TEMP/prettier_files.txt"
-                printf '%s\n' "$f" >> "$RUNNER_TEMP/eslint_files.txt" ;;
-              *.json | *.css | *.scss | *.md | *.mdx | *.yml | *.yaml | *.html)
-                printf '%s\n' "$f" >> "$RUNNER_TEMP/prettier_files.txt" ;;
-            esac
-          done < <(git diff --name-only --diff-filter=ACMR --relative "$merge_base" "$HEAD_SHA" -- .)
-          if [ -s "$RUNNER_TEMP/prettier_files.txt" ] || [ -s "$RUNNER_TEMP/eslint_files.txt" ]; then
-            echo "has_files=true" >> "$GITHUB_OUTPUT"
-          else
-            echo "has_files=false" >> "$GITHUB_OUTPUT"
-            echo "No lintable UI files changed in this PR; nothing to check."
-          fi
-
-      - name: Setup Node.js
-        if: steps.changed.outputs.has_files == 'true'
-        uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
-        with:
-          node-version-file: ui/litellm-dashboard/.nvmrc
-          cache: "npm"
-          cache-dependency-path: ui/litellm-dashboard/package-lock.json
-
-      - name: Install dependencies
-        if: steps.changed.outputs.has_files == 'true'
-        run: npm ci
-
-      - name: Lint changed files (prettier + eslint)
-        if: steps.changed.outputs.has_files == 'true'
-        run: |
-          prettier_files=()
-          eslint_files=()
-          while IFS= read -r f; do prettier_files+=("$f"); done < "$RUNNER_TEMP/prettier_files.txt"
-          while IFS= read -r f; do eslint_files+=("$f"); done < "$RUNNER_TEMP/eslint_files.txt"
-          status=0
-          if [ ${#prettier_files[@]} -gt 0 ]; then
-            echo "::group::Prettier (${#prettier_files[@]} files)"
-            npx prettier --check "${prettier_files[@]}" || { status=1; echo "::error::Unformatted files. Fix with: npm run format"; }
-            echo "::endgroup::"
-          fi
-          if [ ${#eslint_files[@]} -gt 0 ]; then
-            echo "::group::ESLint (${#eslint_files[@]} files)"
-            npx eslint --no-warn-ignored --pass-on-unpruned-suppressions "${eslint_files[@]}" || status=1
-            echo "::endgroup::"
-          fi
-          exit $status
-
-      - name: Check lint budgets
-        if: ${{ !cancelled() && steps.changed.outputs.has_files == 'true' }}
-        run: |
-          npx eslint . -f json -o "$RUNNER_TEMP/lint-report.json" || true
-          node scripts/check-lint-budgets.mjs "$RUNNER_TEMP/lint-report.json" eslint-budgets.json
-
-      - name: Check for dead code (knip)
-        if: ${{ !cancelled() && steps.changed.outputs.has_files == 'true' }}
-        run: npm run knip:ci
-  ui-api-types:
-    name: ui-api-types
-    permissions:
-      contents: read
-    runs-on: ubuntu-latest
-    timeout-minutes: 15
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-          fetch-depth: 2
-
-      - name: Detect changes that can affect the generated types
-        id: changes
-        run: |
-          set -euo pipefail
-          if ! base="$(git rev-parse --verify --quiet HEAD^2 >/dev/null && git rev-parse HEAD^1)"; then
-            echo "Not a pull request merge commit, running the full check."
-            echo "relevant=true" >> "$GITHUB_OUTPUT"
-            exit 0
-          fi
-          files="$(git diff --name-only "$base" HEAD)"
-          if grep -Eq '^(litellm/|enterprise/|(pyproject\.toml|uv\.lock)$|ui/litellm-dashboard/(src/lib/http/schema\.d\.ts|scripts/gen-api-types\.mjs|package(-lock)?\.json)$|\.github/workflows/test-linting\.yml$)' <<< "$files"; then
-            echo "relevant=true" >> "$GITHUB_OUTPUT"
-          else
-            echo "No litellm, enterprise, dependency or generator changes in this pull request, nothing to verify."
-            echo "relevant=false" >> "$GITHUB_OUTPUT"
-          fi
-
-      - name: Set up Python
-        if: steps.changes.outputs.relevant == 'true'
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: "3.12"
-
-      - name: Set up uv
-        if: steps.changes.outputs.relevant == 'true'
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Cache uv dependencies
-        if: steps.changes.outputs.relevant == 'true'
-        uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
-        with:
-          path: |
-            ~/.cache/uv
-            .venv
-          key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
-          restore-keys: |
-            ${{ runner.os }}-uv-
-
-      - name: Cache the Rust build
-        if: steps.changes.outputs.relevant == 'true'
-        uses: ./.github/actions/cache-cargo-build
-
-      - name: Install backend dependencies
-        if: steps.changes.outputs.relevant == 'true'
-        run: .github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router
-
-      - name: Cache Prisma binaries
-        if: steps.changes.outputs.relevant == 'true'
-        uses: ./.github/actions/cache-prisma-binaries
-
-      - name: Generate Prisma client
-        if: steps.changes.outputs.relevant == 'true'
-        run: uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma
-
-      - name: Regenerate the lazy OpenAPI snapshot
-        if: steps.changes.outputs.relevant == 'true'
-        run: uv run --no-sync python -m litellm.proxy._lazy_openapi_snapshot
-
-      - name: Fail if the lazy OpenAPI snapshot is stale
-        if: steps.changes.outputs.relevant == 'true'
-        run: |
-          if ! git diff --exit-code -- litellm/proxy/_lazy_openapi_snapshot.json; then
-            echo "::error file=litellm/proxy/_lazy_openapi_snapshot.json::The lazy OpenAPI snapshot is out of sync with the lazily loaded routes."
-            echo ""
-            echo "A lazily loaded route or model changed without regenerating the snapshot that /openapi.json serves for unloaded features."
-            echo "To fix, run from the repo root:"
-            echo "  uv run python -m litellm.proxy._lazy_openapi_snapshot"
-            echo "then run npm run gen:api from ui/litellm-dashboard and commit both files."
-            exit 1
-          fi
-          echo "_lazy_openapi_snapshot.json is in sync with the lazily loaded routes."
-
-      - name: Set up Node.js
-        if: steps.changes.outputs.relevant == 'true'
-        uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
-        with:
-          node-version-file: ui/litellm-dashboard/.nvmrc
-          cache: "npm"
-          cache-dependency-path: ui/litellm-dashboard/package-lock.json
-
-      - name: Install dashboard dependencies
-        if: steps.changes.outputs.relevant == 'true'
-        working-directory: ui/litellm-dashboard
-        run: npm ci
-
-      - name: Regenerate types from the live spec
-        if: steps.changes.outputs.relevant == 'true'
-        working-directory: ui/litellm-dashboard
-        env:
-          LITELLM_PYTHON: "uv run --no-sync python"
-        run: npm run gen:api
-
-      - name: Fail if types are stale
-        if: steps.changes.outputs.relevant == 'true'
-        run: |
-          if ! git diff --exit-code -- ui/litellm-dashboard/src/lib/http/schema.d.ts; then
-            echo "::error file=ui/litellm-dashboard/src/lib/http/schema.d.ts::Generated API types are out of sync with the proxy OpenAPI spec."
-            echo ""
-            echo "A backend route or model changed without regenerating the dashboard types."
-            echo "To fix, run from ui/litellm-dashboard:"
-            echo "  npm run gen:api"
-            echo "then commit the updated src/lib/http/schema.d.ts."
-            exit 1
-          fi
-          echo "schema.d.ts is in sync with the proxy OpenAPI spec."
-  code-quality:
-    name: code-quality
-    permissions:
-      contents: read
-    runs-on: ubuntu-latest
-    timeout-minutes: 15
-
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Checkout litellm-docs into docs/my-website (for documentation_tests)
-        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          repository: BerriAI/litellm-docs
-          path: docs/my-website
-          persist-credentials: false
-
-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: "3.12"
-
-      - name: Set up uv
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Cache uv dependencies
-        uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
-        with:
-          path: |
-            ~/.cache/uv
-            .venv
-          key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
-          restore-keys: |
-            ${{ runner.os }}-uv-
-
-      - name: Cache the Rust build
-        uses: ./.github/actions/cache-cargo-build
-
-      - name: Install dependencies
-        run: uv sync --frozen --all-groups --all-extras
-
-      - name: check_licenses
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_licenses.py
-
-      - name: check_provider_folders_documented
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_provider_folders_documented.py
-
-      - name: check_provider_endpoints_support_generated
-        run: uv run --no-sync python scripts/generate_provider_endpoints_support.py --check
-
-      - name: test_generate_provider_endpoints_support
-        run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_generate_provider_endpoints_support.py
-
-      - name: check_prisma_binary_cache
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_prisma_binary_cache.py
-
-      - name: check_workflow_startup_safety
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_workflow_startup_safety.py
-
-      - name: check_workflow_job_name_collisions
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_workflow_job_name_collisions.py
-
-      - name: test_workflow_job_name_collisions
-        run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_workflow_job_name_collisions.py
-
-      - name: test_unit_passed_gate
-        run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_unit_passed_gate.py
-
-      - name: test_e2e_idp_stack
-        run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_e2e_idp_stack.py
-
-      - name: test_e2e_metadata
-        env:
-          PYTHONPATH: tests/e2e
-        run: |
-          export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 16)"
-          uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_e2e_metadata.py tests/code_coverage_tests/test_e2e_junit_report.py
-
-      - name: Check merge smoke harness
-        run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_merge_smoke.py
-
-      - name: router_code_coverage
-        run: uv run --no-sync python ./tests/code_coverage_tests/router_code_coverage.py
-
-      - name: router_surface_check
-        run: uv run --no-sync python ./tests/code_coverage_tests/router_surface_check.py
-
-      - name: test_chat_completion_imports
-        run: uv run --no-sync python ./tests/code_coverage_tests/test_chat_completion_imports.py
-
-      - name: info_log_check
-        run: uv run --no-sync python ./tests/code_coverage_tests/info_log_check.py
-
-      - name: check_guardrail_apply_decorator
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_guardrail_apply_decorator.py
-
-      - name: test_ban_set_verbose
-        run: uv run --no-sync python ./tests/code_coverage_tests/test_ban_set_verbose.py
-
-      - name: code_qa_check_tests
-        run: uv run --no-sync python ./tests/code_coverage_tests/code_qa_check_tests.py
-
-      - name: check_get_model_cost_key_performance
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_get_model_cost_key_performance.py
-
-      - name: test_proxy_types_import
-        run: uv run --no-sync python ./tests/code_coverage_tests/test_proxy_types_import.py
-
-      - name: callback_manager_test
-        run: uv run --no-sync python ./tests/code_coverage_tests/callback_manager_test.py
-
-      - name: recursive_detector
-        run: uv run --no-sync python ./tests/code_coverage_tests/recursive_detector.py
-
-      - name: test_router_strategy_async
-        run: uv run --no-sync python ./tests/code_coverage_tests/test_router_strategy_async.py
-
-      - name: litellm_logging_code_coverage
-        run: uv run --no-sync python ./tests/code_coverage_tests/litellm_logging_code_coverage.py
-
-      - name: ensure_async_clients_test
-        run: uv run --no-sync python ./tests/code_coverage_tests/ensure_async_clients_test.py
-
-      - name: enforce_llms_folder_style
-        run: uv run --no-sync python ./tests/code_coverage_tests/enforce_llms_folder_style.py
-
-      - name: prevent_key_leaks_in_exceptions
-        run: uv run --no-sync python ./tests/code_coverage_tests/prevent_key_leaks_in_exceptions.py
-
-      - name: check_unsafe_enterprise_import
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_unsafe_enterprise_import.py
-
-      - name: check_telemetry_imports
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_telemetry_imports.py
-
-      - name: ban_copy_deepcopy_kwargs
-        run: uv run --no-sync python ./tests/code_coverage_tests/ban_copy_deepcopy_kwargs.py
-
-      - name: check_fastuuid_usage
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_fastuuid_usage.py
-
-      - name: check_py310_typing_imports
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_py310_typing_imports.py
-
-      - name: check_e2e_no_raw_requests
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_e2e_no_raw_requests.py
-
-      - name: check_migrations_no_data_rewrites
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_migrations_no_data_rewrites.py
-
-      - name: check_migrations_lock_timeout
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_migrations_lock_timeout.py
-
-      - name: check_no_none_string_error_fields
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_no_none_string_error_fields.py
-
-      - name: test_check_no_none_string_error_fields
-        run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_check_no_none_string_error_fields.py
-
-      - name: check_no_publicly_known_master_key
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_no_publicly_known_master_key.py
-
-      - name: test_check_no_publicly_known_master_key
-        run: uv run --no-sync pytest -q --noconftest -p no:cacheprovider -c /dev/null tests/code_coverage_tests/test_check_no_publicly_known_master_key.py
-
-      - name: check_unbounded_in_lists (fails on findings not in the baseline)
-        run: uv run --no-sync python ./tests/code_coverage_tests/check_unbounded_in_lists.py
-
-      - name: memory_test
-        run: uv run --no-sync python ./tests/code_coverage_tests/memory_test.py
-
-      - name: documentation_test_env_keys
-        run: uv run --no-sync python ./tests/documentation_tests/test_env_keys.py
-
-      - name: documentation_test_router_settings
-        run: uv run --no-sync python ./tests/documentation_tests/test_router_settings.py
-
-      - name: documentation_test_api_docs
-        run: uv run --no-sync python ./tests/documentation_tests/test_api_docs.py
-  ci-coverage:
-    name: ci-coverage
-    permissions:
-      contents: read
-    runs-on: ubuntu-latest
-    timeout-minutes: 5
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: "3.12"
-
-      - name: Assert every test file and Dockerfile is invoked by a job
-        run: |
-          python -m pip install "pyyaml==6.0.3"
-          python .github/scripts/assert_ci_coverage.py
-
-      - name: Assert no -k expression deselects a file from every job that globs it
-        run: python .github/scripts/assert_ci_coverage.py --slices
-
-      - name: Assert .github/workflows/ holds only workflows, correctly named
-        run: python .github/scripts/assert_workflow_dir_hygiene.py
-  semgrep:
-    name: semgrep
-    permissions:
-      contents: read
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: "3.12"
-
-      - name: Set up uv
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Run Semgrep (custom rules)
-        run: uv tool run --from 'semgrep==1.157.0' semgrep scan --config .semgrep/rules . --error
-  lint-passed:
-    name: lint passed
-    needs: [python, secret-scan, ui, ui-api-types, code-quality, ci-coverage, semgrep]
-    if: always()
-    runs-on: ubuntu-latest
-    timeout-minutes: 2
-    permissions: {}
-    steps:
-      - name: Require every lint job to succeed
-        env:
-          NEEDS: ${{ toJSON(needs) }}
-        run: |
-          jq -r 'to_entries[] | "\(.key): \(.value.result)"' <<< "$NEEDS"
-          jq -e 'all(.[]; .result == "success")' <<< "$NEEDS" > /dev/null

.github/workflows/test-litellm-ui-build.yml

@@ -0,0 +1,42 @@
+name: UI Build Check
+permissions:
+  contents: read
+  pull-requests: read
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  build-ui:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Detect relevant changes
+        id: changes
+        uses: ./.github/actions/detect-changes
+        with:
+          category: ui
+
+      # Built through the image stage rather than the checkout, because the
+      # stage copies ui/litellm-dashboard/ alone: an import reaching above the
+      # dashboard root resolves in a checkout and fails in every image we ship.
+      # Dockerfile, docker/Dockerfile.non_root and ui/Dockerfile share this
+      # stage verbatim, so building one covers all three.
+      - name: Build the dashboard as the shipped images build it
+        if: steps.changes.outputs.decision != 'skip'
+        run: docker build --target ui-builder -f Dockerfile .

.github/workflows/test-litellm-ui-lint.yml

@@ -0,0 +1,107 @@
+name: UI Lint
+permissions:
+  contents: read
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  frontend-lint:
+    runs-on: ubuntu-latest
+    timeout-minutes: 8
+    defaults:
+      run:
+        working-directory: ui/litellm-dashboard
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          fetch-depth: 1
+          persist-credentials: false
+
+      - name: Collect changed files
+        id: changed
+        env:
+          GH_TOKEN: ${{ github.token }}
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+        run: |
+          # base.sha is the base branch tip from when the PR was opened, while
+          # actions/checkout leaves HEAD on a merge of the PR into the *current*
+          # base tip. "$BASE_SHA"...HEAD therefore spans every base-branch commit
+          # landed since, so a PR that touches no UI file still gets linted
+          # against hundreds of other people's files. Diff the PR head against its
+          # own merge base instead, which is exactly what this PR changed.
+          merge_base=$(gh api "repos/${{ github.repository }}/compare/${BASE_SHA}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
+          test -n "$merge_base"
+          git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA"
+          : > "$RUNNER_TEMP/prettier_files.txt"
+          : > "$RUNNER_TEMP/eslint_files.txt"
+          while IFS= read -r f; do
+            [ -f "$f" ] || continue
+            case "$f" in
+              *.js | *.jsx | *.ts | *.tsx | *.mjs | *.cjs)
+                printf '%s\n' "$f" >> "$RUNNER_TEMP/prettier_files.txt"
+                printf '%s\n' "$f" >> "$RUNNER_TEMP/eslint_files.txt" ;;
+              *.json | *.css | *.scss | *.md | *.mdx | *.yml | *.yaml | *.html)
+                printf '%s\n' "$f" >> "$RUNNER_TEMP/prettier_files.txt" ;;
+            esac
+          done < <(git diff --name-only --diff-filter=ACMR --relative "$merge_base" "$HEAD_SHA" -- .)
+          if [ -s "$RUNNER_TEMP/prettier_files.txt" ] || [ -s "$RUNNER_TEMP/eslint_files.txt" ]; then
+            echo "has_files=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "has_files=false" >> "$GITHUB_OUTPUT"
+            echo "No lintable UI files changed in this PR; nothing to check."
+          fi
+
+      - name: Setup Node.js
+        if: steps.changed.outputs.has_files == 'true'
+        uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
+        with:
+          node-version-file: ui/litellm-dashboard/.nvmrc
+          cache: "npm"
+          cache-dependency-path: ui/litellm-dashboard/package-lock.json
+
+      - name: Install dependencies
+        if: steps.changed.outputs.has_files == 'true'
+        run: npm ci
+
+      - name: Lint changed files (prettier + eslint)
+        if: steps.changed.outputs.has_files == 'true'
+        run: |
+          prettier_files=()
+          eslint_files=()
+          while IFS= read -r f; do prettier_files+=("$f"); done < "$RUNNER_TEMP/prettier_files.txt"
+          while IFS= read -r f; do eslint_files+=("$f"); done < "$RUNNER_TEMP/eslint_files.txt"
+          status=0
+          if [ ${#prettier_files[@]} -gt 0 ]; then
+            echo "::group::Prettier (${#prettier_files[@]} files)"
+            npx prettier --check "${prettier_files[@]}" || { status=1; echo "::error::Unformatted files. Fix with: npm run format"; }
+            echo "::endgroup::"
+          fi
+          if [ ${#eslint_files[@]} -gt 0 ]; then
+            echo "::group::ESLint (${#eslint_files[@]} files)"
+            npx eslint --no-warn-ignored --pass-on-unpruned-suppressions "${eslint_files[@]}" || status=1
+            echo "::endgroup::"
+          fi
+          exit $status
+
+      - name: Check lint budgets
+        if: ${{ !cancelled() && steps.changed.outputs.has_files == 'true' }}
+        run: |
+          npx eslint . -f json -o "$RUNNER_TEMP/lint-report.json" || true
+          node scripts/check-lint-budgets.mjs "$RUNNER_TEMP/lint-report.json" eslint-budgets.json
+
+      - name: Check for dead code (knip)
+        if: ${{ !cancelled() && steps.changed.outputs.has_files == 'true' }}
+        run: npm run knip:ci

.github/workflows/test-litellm-ui-unit.yml

@@ -0,0 +1,98 @@
+name: UI Unit Tests
+permissions:
+  contents: read
+  pull-requests: read
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+  push:
+    branches:
+      - main
+      - litellm_internal_staging
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  ui-unit-tests:
+    runs-on: ubuntu-latest-16-cores
+    timeout-minutes: 20
+    defaults:
+      run:
+        working-directory: ui/litellm-dashboard
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          fetch-depth: 1
+          persist-credentials: false
+
+      - name: Detect relevant changes
+        id: changes
+        uses: ./.github/actions/detect-changes
+        with:
+          category: ui
+
+      - name: Setup Node.js
+        if: steps.changes.outputs.decision != 'skip'
+        uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
+        with:
+          node-version-file: ui/litellm-dashboard/.nvmrc
+          cache: "npm"
+          cache-dependency-path: ui/litellm-dashboard/package-lock.json
+
+      - name: Install dependencies
+        if: steps.changes.outputs.decision != 'skip'
+        run: npm ci
+
+      - name: Run UI type tests (Vitest)
+        if: steps.changes.outputs.decision != 'skip'
+        env:
+          CI: "true"
+        run: npm run test:types
+
+      - name: Run UI unit tests (Vitest)
+        if: steps.changes.outputs.decision != 'skip'
+        env:
+          CI: "true"
+          GH_TOKEN: ${{ github.token }}
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+        run: |
+          full_suite() { npm run test -- --run --pool forks --maxWorkers=14; }
+
+          if [ -z "$BASE_SHA" ]; then
+            echo "Push to $GITHUB_REF_NAME: running the full suite"
+            full_suite
+            exit 0
+          fi
+
+          merge_base=$(gh api "repos/${{ github.repository }}/compare/${BASE_SHA}...${HEAD_SHA}?per_page=1" --jq '.merge_base_commit.sha')
+          test -n "$merge_base"
+          git fetch --no-tags --depth=1 origin "$merge_base" "$HEAD_SHA"
+          changed_files=()
+          while IFS= read -r f; do
+            changed_files+=("$f")
+          done < <(git diff --name-only --relative "$merge_base" "$HEAD_SHA" -- .)
+          if [ ${#changed_files[@]} -eq 0 ]; then
+            echo "No UI files changed in this PR; skipping unit tests."
+            exit 0
+          fi
+
+          scope=$(printf '%s\n' "${changed_files[@]}" | bash "$GITHUB_WORKSPACE/.github/scripts/select_ui_test_scope.sh")
+          if [ "$scope" != related ]; then
+            echo "Pull request: ${#changed_files[@]} changed UI files reach outside src/, so related would miss their dependents; running the full suite"
+            full_suite
+            exit 0
+          fi
+
+          echo "Pull request: running tests related to ${#changed_files[@]} changed UI files"
+          npm run test -- related "${changed_files[@]}" --run --passWithNoTests \
+            --pool forks --maxWorkers=14

.github/workflows/test-mcp-dependency-resolution.yml

@@ -4,31 +4,9 @@ on:
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
-    paths:
-      - "**/pyproject.toml"
-      - "uv.lock"
-      - "uv.toml"
-      - ".python-version"
-      - "rust-toolchain.toml"
-      - "litellm-rust/**"
-      - "litellm/__init__.py"
-      - "litellm/_version.py"
-      - "litellm/_lazy_imports_registry.py"
-      - "scripts/build_core_distribution.py"
-      - "litellm/proxy/proxy_server.py"
-      - "litellm/**/*mcp*"
-      - "litellm/**/*mcp*/**"
-      - "litellm/integrations/arize/**"
-      - "scripts/check_mcp_sdk_install.py"
-      - "tests/base_sdk_tests/**"
-      - ".github/workflows/test-mcp-dependency-resolution.yml"
-      - ".github/actions/detect-changes/**"
-      - ".github/actions/setup-uv-with-retries/**"
-      - ".github/actions/cache-cargo-build/**"
-      - ".github/scripts/detect_changes.sh"
-      - ".github/scripts/uv_sync_with_retries.sh"
-      - ".circleci/scripts/classify_changes.sh"
 
 permissions:
   contents: read
@@ -41,11 +19,11 @@ concurrency:
 jobs:
   resolve:
     runs-on: ubuntu-latest
-    timeout-minutes: 25
+    timeout-minutes: 15
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14", "3.15"]
+        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
 
     steps:
       - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
@@ -89,7 +67,6 @@ jobs:
             uv pip check --python ".venv-$extra"
             if [ "$extra" = core ]; then
               checker=("$GITHUB_WORKSPACE/tests/base_sdk_tests/check_base_sdk_install.py")
-              (cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-$extra/bin/python" -I "$GITHUB_WORKSPACE/tests/base_sdk_tests/check_sdk_http.py")
             else
               checker=("$GITHUB_WORKSPACE/scripts/check_mcp_sdk_install.py" --extra "$extra")
             fi
@@ -119,63 +96,3 @@ jobs:
             fi
             (cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-lowest-$extra/bin/python" "${checker[@]}")
           done
-
-  core-distribution:
-    permissions:
-      contents: read
-      id-token: write
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    env:
-      UV_PYTHON: ${{ matrix.python-version }}
-      LITELLM_LOCAL_MODEL_COST_MAP: "True"
-    strategy:
-      fail-fast: false
-      matrix:
-        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14", "3.15"]
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: ${{ matrix.python-version }}
-      - uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-      - uses: ./.github/actions/cache-cargo-build
-      - name: Build the core wheel and source distribution
-        run: >-
-          uv run --no-project --with coverage==7.14.0 python -m coverage run --rcfile=/dev/null
-          --branch --include="*/scripts/build_core_distribution.py"
-          scripts/build_core_distribution.py --out-dir dist/core
-      - name: Verify metadata, resources, and independent source rebuild
-        env:
-          CORE_DISTRIBUTION_DIR: ${{ github.workspace }}/dist/core
-        run: >-
-          uv run --no-project --with pytest==9.0.3 --with pytest-cov==5.0.0 --with coverage==7.14.0 --with 'tomli==2.4.1; python_version < "3.11"'
-          python -m pytest tests/base_sdk_tests/test_core_distribution.py -v
-          --cov-config=/dev/null --cov=scripts.build_core_distribution --cov-branch --cov-append --cov-report=
-      - name: Verify isolated core installations
-        run: |
-          wheel=$(realpath dist/core/litellm_core-*.whl)
-          for resolution in highest lowest-direct; do
-            uv venv --python ${{ matrix.python-version }} ".venv-core-$resolution"
-            uv pip install --python ".venv-core-$resolution" --resolution "$resolution" "$wheel"
-            uv pip check --python ".venv-core-$resolution"
-            (cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-core-$resolution/bin/python" -I "$GITHUB_WORKSPACE/tests/base_sdk_tests/check_base_sdk_install.py" --profile core)
-            (cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-core-$resolution/bin/python" -I "$GITHUB_WORKSPACE/tests/base_sdk_tests/check_sdk_http.py")
-          done
-          uv pip install --python .venv-core-highest coverage==7.14.0
-          .venv-core-highest/bin/python -I -m coverage run --rcfile=/dev/null --append --branch --include="*/tests/base_sdk_tests/check_base_sdk_install.py" tests/base_sdk_tests/check_base_sdk_install.py --profile core
-          uv pip install --python .venv-core-highest boto3 tokenizers huggingface-hub
-          (cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-core-highest/bin/python" -I "$GITHUB_WORKSPACE/tests/base_sdk_tests/check_base_sdk_install.py" --profile dependencies)
-          (cd "$RUNNER_TEMP" && "$GITHUB_WORKSPACE/.venv-core-highest/bin/python" -I "$GITHUB_WORKSPACE/tests/base_sdk_tests/check_sdk_http.py")
-          uv run --no-project --with coverage==7.14.0 python -m coverage xml --rcfile=/dev/null -o coverage-core.xml
-      - name: Upload core packaging coverage
-        uses: codecov/codecov-action@0fb7174895f61a3b6b78fc075e0cd60383518dac # v5.5.5
-        with:
-          use_oidc: true
-          files: coverage-core.xml
-          flags: core-packaging
-          fail_ci_if_error: false

.github/workflows/test-merge-smoke.yml

@@ -1,288 +0,0 @@
-name: smoke
-
-on:
-  pull_request:
-    branches: [main, litellm_internal_staging, litellm_oss_staging, "litellm_**"]
-  workflow_dispatch:
-
-permissions:
-  contents: read
-
-concurrency:
-  group: merge-smoke-${{ github.event.pull_request.number || github.run_id }}
-  cancel-in-progress: true
-
-jobs:
-  dashboard-build:
-    name: dashboard-build
-    runs-on: ubuntu-24.04
-    timeout-minutes: 30
-    steps:
-      - name: Checkout
-        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Build the dashboard stage
-        run: docker build --target ui-builder -f Dockerfile .
-
-  core-checks:
-    name: py${{ matrix.python-version }}
-    runs-on: ubuntu-24.04
-    timeout-minutes: 30
-    strategy:
-      fail-fast: false
-      matrix:
-        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14", "3.15"]
-    env:
-      LITELLM_LOCAL_MODEL_COST_MAP: "True"
-    steps:
-      - name: Checkout
-        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: ${{ matrix.python-version }}
-
-      - name: Set up uv
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Install dependencies
-        run: .github/scripts/uv_sync_with_retries.sh --frozen --extra proxy --extra cli --group dev --group proxy-dev --python ${{ matrix.python-version }}
-
-      - name: Create the loopback-only network namespace
-        run: |
-          sudo ip netns add smoke
-          sudo ip netns exec smoke ip link set lo up
-          cat > "${RUNNER_TEMP}/in-netns" <<'WRAP'
-          #!/usr/bin/env bash
-          set -euo pipefail
-          exec sudo --preserve-env=LITELLM_LOCAL_MODEL_COST_MAP ip netns exec smoke setpriv --reuid "$(id -u)" --regid "$(id -g)" --init-groups -- env HOME="${HOME}" PATH="${PATH}" "$@"
-          WRAP
-          chmod +x "${RUNNER_TEMP}/in-netns"
-          echo "IN_NETNS=${RUNNER_TEMP}/in-netns" >> "${GITHUB_ENV}"
-
-      - name: Verify namespace isolation
-        run: $IN_NETNS .venv/bin/python .github/scripts/run_merge_smoke.py verify-isolation
-
-      - name: Verify interpreter version
-        run: $IN_NETNS .venv/bin/python .github/scripts/run_merge_smoke.py interpreter --expect ${{ matrix.python-version }}
-
-      - name: Import and CLI checks
-        run: $IN_NETNS .venv/bin/python .github/scripts/run_merge_smoke.py cli
-
-      - name: Routing and UUID compatibility
-        run: >-
-          $IN_NETNS .venv/bin/python -m pytest
-          tests/unit/router_strategy/test_complexity_router.py::TestCustomDimensions
-          tests/unit/test_uuid_helper.py -q
-
-      - name: Proxy startup check
-        run: $IN_NETNS .venv/bin/python .github/scripts/run_merge_smoke.py proxy-startup --diagnostics-dir "${RUNNER_TEMP}/smoke-diagnostics"
-
-      - name: Upload smoke diagnostics
-        if: always()
-        uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
-        with:
-          name: merge-smoke-diagnostics-py${{ matrix.python-version }}
-          path: ${{ runner.temp }}/smoke-diagnostics
-          if-no-files-found: ignore
-
-      - name: Remove the network namespace
-        if: always()
-        run: sudo ip netns delete smoke
-
-  all-extras:
-    name: all-extras-py${{ matrix.python-version }}
-    runs-on: ubuntu-24.04
-    timeout-minutes: 10
-    strategy:
-      fail-fast: false
-      matrix:
-        python-version: ["3.12", "3.13"]
-    env:
-      LITELLM_LOCAL_MODEL_COST_MAP: "True"
-    steps:
-      - name: Checkout
-        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: ${{ matrix.python-version }}
-
-      - name: Set up uv
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Install every extra and dependency group
-        run: .github/scripts/uv_sync_with_retries.sh --frozen --all-groups --all-extras --python ${{ matrix.python-version }}
-
-      - name: Import the SDK, the proxy server and the enterprise package
-        run: .venv/bin/python -c "import litellm, litellm.proxy.proxy_server, litellm_enterprise"
-
-  windows-sdk:
-    name: windows-sdk
-    runs-on: windows-2025
-    timeout-minutes: 30
-    env:
-      LITELLM_LOCAL_MODEL_COST_MAP: "True"
-    defaults:
-      run:
-        shell: bash
-    steps:
-      - name: Checkout
-        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: "3.11"
-
-      - name: Set up uv
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Set up the Rust bridge
-        uses: ./.github/actions/rust-bridge
-
-      - name: Install dependencies
-        run: .github/scripts/uv_sync_with_retries.sh --frozen --group dev --python 3.11
-
-      - name: Run the Windows tests
-        run: uv run --no-sync python -m pytest --tb=short -v tests/windows_tests
-
-  proxy-build-from-pip:
-    name: proxy-build-from-pip
-    runs-on: ubuntu-24.04
-    timeout-minutes: 10
-    services:
-      postgres:
-        image: postgres:14@sha256:6a70deda415ec296f977890e11aba04a0db9f632a362e3fce45e845e3db74f26
-        env:
-          POSTGRES_USER: postgres
-          POSTGRES_PASSWORD: postgres
-          POSTGRES_DB: circle_test
-        ports:
-          - 5432:5432
-        options: >-
-          --health-cmd "pg_isready -U postgres"
-          --health-interval 2s
-          --health-timeout 5s
-          --health-retries 30
-    steps:
-      - name: Checkout
-        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: "3.12"
-
-      - name: Set up uv
-        uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Install the test client dependencies
-        run: .github/scripts/uv_sync_with_retries.sh --frozen --no-install-workspace --extra proxy --group dev --python 3.12
-
-      - name: Build the image from the published package
-        run: docker build -t litellm-build-from-pip:smoke -f docker/build_from_pip/Dockerfile.build_from_pip .
-
-      - name: Start the fake OpenAI endpoint
-        run: |
-          nohup .venv/bin/python tests/_fake_openai_endpoint_server.py --host 0.0.0.0 --port 8190 > "${RUNNER_TEMP}/fake-openai.log" 2>&1 &
-          for _ in $(seq 1 30); do
-            curl -sf http://localhost:8190/health > /dev/null && exit 0
-            sleep 1
-          done
-          cat "${RUNNER_TEMP}/fake-openai.log"
-          exit 1
-
-      - name: Generate a master key
-        run: |
-          key="sk-$(openssl rand -hex 16)"
-          echo "::add-mask::${key}"
-          echo "LITELLM_MASTER_KEY=${key}" >> "${GITHUB_ENV}"
-
-      - name: Start the proxy container
-        run: |
-          docker run -d \
-            -p 4000:4000 \
-            -e DATABASE_URL=postgresql://postgres:postgres@host.docker.internal:5432/circle_test \
-            -e LITELLM_MASTER_KEY \
-            -e FAKE_OPENAI_API_BASE=http://host.docker.internal:8190 \
-            -e OTEL_EXPORTER=in_memory \
-            -e USE_DDTRACE=True \
-            -e GCS_FLUSH_INTERVAL=1 \
-            -e LITELLM_LOG=ERROR \
-            --add-host host.docker.internal:host-gateway \
-            --name litellm-build-from-pip \
-            -v "${GITHUB_WORKSPACE}/docker/build_from_pip/litellm_config.yaml:/app/config.yaml" \
-            litellm-build-from-pip:smoke \
-            --config /app/config.yaml \
-            --port 4000
-          for _ in $(seq 1 150); do
-            curl -sf --max-time 5 -o /dev/null http://localhost:4000 && exit 0
-            sleep 2
-          done
-          exit 1
-
-      - name: Run the startup tests
-        shell: bash
-        run: |
-          set -euo pipefail
-          base_url=http://localhost:4000
-          readiness_response="$(curl -sS --fail-with-body --write-out $'\n%{http_code}' "${base_url}/health/readiness")"
-          readiness_status="${readiness_response##*$'\n'}"
-          readiness_body="${readiness_response%$'\n'*}"
-          [[ "${readiness_status}" == "200" ]]
-          jq -e '.status == "healthy" or .status == "connected"' <<< "${readiness_body}" > /dev/null
-          liveness_status="$(curl -sS --fail-with-body --output /dev/null --write-out '%{http_code}' "${base_url}/health/liveness")"
-          [[ "${liveness_status}" == "200" ]]
-          completion_response="$(curl -sS --fail-with-body --write-out $'\n%{http_code}' \
-            --header "Authorization: Bearer ${LITELLM_MASTER_KEY}" \
-            --header 'Content-Type: application/json' \
-            --data '{"model":"gpt-4","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Hello!"}]}' \
-            "${base_url}/chat/completions")"
-          completion_status="${completion_response##*$'\n'}"
-          completion_body="${completion_response%$'\n'*}"
-          [[ "${completion_status}" == "200" ]]
-          jq -e '
-            (.choices | length) == 1 and
-            .choices[0].message.role == "assistant" and
-            .choices[0].message.content == "Hello! This is a mock response from the fake OpenAI endpoint."
-          ' <<< "${completion_body}" > /dev/null
-
-      - name: Show the proxy logs
-        if: always()
-        run: docker logs litellm-build-from-pip || true
-
-  smoke-passed:
-    name: smoke passed
-    needs: [dashboard-build, core-checks, all-extras, windows-sdk, proxy-build-from-pip]
-    if: always()
-    runs-on: ubuntu-latest
-    timeout-minutes: 2
-    permissions: {}
-    steps:
-      - name: Require every smoke job to succeed
-        env:
-          NEEDS: ${{ toJSON(needs) }}
-        run: |
-          jq -r 'to_entries[] | "\(.key): \(.value.result)"' <<< "$NEEDS"
-          jq -e 'all(.[]; .result == "success")' <<< "$NEEDS" > /dev/null

.github/workflows/test-postgres.yml

@@ -0,0 +1,145 @@
+name: "Postgres Tests"
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+  push:
+    branches:
+      - main
+      - litellm_internal_staging
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  postgres:
+    name: ${{ matrix.shard }}
+    runs-on: ubuntu-latest
+    timeout-minutes: ${{ matrix.job-timeout-minutes }}
+    permissions:
+      contents: read
+
+    services:
+      postgres:
+        image: postgres:16@sha256:e17e86066e5ef83e0952a9347f5c792b7ece00972e2aa787a6986f471b3dd3d5
+        env:
+          POSTGRES_USER: postgres
+          POSTGRES_PASSWORD: postgres
+          POSTGRES_DB: litellm_test
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 10
+
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - shard: proxy-behavior
+            test-path: "tests/proxy_behavior"
+            seed: db-push
+            workers: 0
+            timeout-minutes: 25
+            job-timeout-minutes: 50
+
+          - shard: proxy-security
+            test-path: "tests/proxy_security_tests"
+            seed: db-push
+            workers: 0
+            timeout-minutes: 15
+            job-timeout-minutes: 40
+
+          - shard: schema-migration
+            test-path: "tests/proxy_migration_tests"
+            seed: none
+            workers: 0
+            timeout-minutes: 20
+            job-timeout-minutes: 45
+
+    env:
+      DATABASE_URL: "postgresql://postgres:postgres@localhost:5432/litellm_test"
+
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        timeout-minutes: 3
+        with:
+          persist-credentials: false
+
+      - name: Detect relevant changes
+        id: changes
+        timeout-minutes: 2
+        uses: ./.github/actions/detect-changes
+
+      - name: Set up Python
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 3
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 3
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Cache uv dependencies
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 5
+        uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+        with:
+          path: |
+            ~/.cache/uv
+            .venv
+          key: ${{ runner.os }}-uv-postgres-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-uv-postgres-
+
+      - name: Install dependencies
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 12
+        run: |
+          .github/scripts/uv_sync_with_retries.sh --frozen --all-groups --all-extras
+
+      - name: Cache Prisma binaries
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 3
+        uses: ./.github/actions/cache-prisma-binaries
+
+      - name: Generate Prisma client
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: 5
+        run: |
+          uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma
+
+      - name: Seed database schema
+        if: steps.changes.outputs.decision != 'skip' && matrix.seed != 'none'
+        timeout-minutes: 10
+        run: |
+          uv run --no-sync prisma db push --schema litellm/proxy/schema.prisma --accept-data-loss
+
+      - name: Run tests
+        if: steps.changes.outputs.decision != 'skip'
+        timeout-minutes: ${{ matrix.timeout-minutes }}
+        env:
+          TEST_PATH: ${{ matrix.test-path }}
+          WORKERS: ${{ matrix.workers }}
+        run: |
+          if [ "${WORKERS}" = "0" ]; then
+            uv run --no-sync pytest ${TEST_PATH:?} -vv --tb=short --durations=10
+          else
+            uv run --no-sync pytest ${TEST_PATH:?} -vv --tb=short --durations=10 -n "${WORKERS}"
+          fi

.github/workflows/test-redis-compat.yml

@@ -0,0 +1,77 @@
+name: "Unit Tests: Redis Client Version Compatibility"
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+    paths:
+      - "litellm/_redis.py"
+      - "litellm/_redis_credential_provider.py"
+      - "tests/test_litellm/test_redis.py"
+      - "tests/test_litellm/caching/test_redis_connection_pool.py"
+      - ".github/workflows/test-redis-compat.yml"
+      - "pyproject.toml"
+      - "uv.lock"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  redis-compat:
+    name: "redis-py ${{ matrix.redis-version }}"
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+
+    strategy:
+      fail-fast: false
+      matrix:
+        # 5.3.1 is the version pinned in uv.lock (redisvl caps it below 6); the
+        # newer legs prove the inspect.signature introspection in litellm/_redis.py
+        # keeps extracting kwargs on the redis-py releases people actually run now.
+        # Only the exact release 6.0.0 is skipped: rq (pulled by the proxy extra)
+        # specifies `redis != 6`, which excludes 6.0.0 alone, so 6.4.0 stands in
+        # for the 6.x line.
+        redis-version: ["5.3.1", "6.4.0", "7.4.1", "8.0.1"]
+
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Install dependencies
+        run: |
+          .github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router
+
+      - name: Pin redis-py to the matrix version
+        env:
+          REDIS_VERSION: ${{ matrix.redis-version }}
+        run: |
+          uv pip install "redis==${REDIS_VERSION:?}"
+          uv run --no-sync python -c "import redis; assert redis.__version__ == '${REDIS_VERSION:?}', redis.__version__; print('redis-py', redis.__version__)"
+
+      - name: Run redis unit tests
+        run: |
+          uv run --no-sync pytest \
+            tests/test_litellm/test_redis.py \
+            tests/test_litellm/caching/test_redis_connection_pool.py \
+            --tb=short -vv \
+            --reruns 2 \
+            --reruns-delay 1 \
+            --durations=20

.github/workflows/test-rust-db.yml

@@ -1,116 +0,0 @@
-name: LiteLLM Rust DB
-
-on:
-  push:
-    paths:
-      - "litellm-rust/crates/db/**"
-      - "litellm-rust/Cargo.toml"
-      - "litellm-rust/Cargo.lock"
-      - "litellm-rust/clippy.toml"
-      - "litellm-proxy-extras/litellm_proxy_extras/migrations/**"
-      - "**/schema.prisma"
-      - ".cargo/**"
-      - "rust-toolchain.toml"
-      - "pyproject.toml"
-      - "uv.lock"
-      - ".github/actions/setup-uv-with-retries/**"
-      - ".github/workflows/test-rust-db.yml"
-  pull_request:
-    branches:
-      - main
-      - "litellm_**"
-    paths:
-      - "litellm-rust/crates/db/**"
-      - "litellm-rust/Cargo.toml"
-      - "litellm-rust/Cargo.lock"
-      - "litellm-rust/clippy.toml"
-      - "litellm-proxy-extras/litellm_proxy_extras/migrations/**"
-      - "**/schema.prisma"
-      - ".cargo/**"
-      - "rust-toolchain.toml"
-      - "pyproject.toml"
-      - "uv.lock"
-      - ".github/actions/setup-uv-with-retries/**"
-      - ".github/workflows/test-rust-db.yml"
-
-permissions:
-  contents: read
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
-  cancel-in-progress: true
-
-env:
-  CARGO_TERM_COLOR: always
-  RUSTC_WRAPPER: sccache
-  SCCACHE_GHA_ENABLED: "true"
-  SCCACHE_GHA_VERSION: litellm-ci-v1
-  SCCACHE_GHA_RW_MODE: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
-  CARGO_INCREMENTAL: "0"
-
-jobs:
-  db-queries:
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    services:
-      postgres:
-        image: postgres:16.6@sha256:557fea37a744d5f4c8faab304b0a90858b53ab119735a88c131fd19dab802f36
-        env:
-          POSTGRES_DB: litellm
-          POSTGRES_PASSWORD: postgres
-          POSTGRES_USER: postgres
-        ports:
-          - 5432:5432
-        options: >-
-          --health-cmd "pg_isready -U postgres -d litellm"
-          --health-interval 10s
-          --health-timeout 5s
-          --health-retries 5
-    env:
-      DATABASE_URL: postgresql://postgres:postgres@localhost:5432/litellm
-      SQLX_OFFLINE: "false"
-    defaults:
-      run:
-        working-directory: litellm-rust
-    steps:
-      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
-        with:
-          persist-credentials: false
-
-      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
-        with:
-          python-version: "3.12"
-
-      - uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - run: rustup toolchain install --no-self-update
-
-      - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
-        with:
-          workspaces: litellm-rust
-          cache-on-failure: true
-
-      - name: Set up sccache
-        uses: mozilla-actions/sccache-action@d651010b8da762cde178750d8eda7b5febfe147a
-        with:
-          version: "v0.18.0"
-
-      - name: Install Python dependencies
-        working-directory: .
-        run: uv sync --frozen --extra extra-proxy
-
-      - name: Deploy Prisma migrations
-        working-directory: .
-        run: uv run --no-sync prisma migrate deploy --schema litellm-proxy-extras/litellm_proxy_extras/schema.prisma
-
-      - name: Install SQLx CLI
-        run: cargo install sqlx-cli --version 0.9.0 --locked --no-default-features -F postgres
-
-      - name: Check SQLx query metadata
-        working-directory: litellm-rust/crates/db
-        run: cargo sqlx prepare --check
-
-      - name: Test database queries
-        run: cargo test -p litellm-db --locked -- --include-ignored

.github/workflows/test-rust.yml

@@ -4,12 +4,7 @@ on:
   push:
     paths:
       - "litellm-rust/**"
-      - "model_prices_and_context_window.json"
       - "litellm/rust_bridge/**"
-      - "scripts/generate_trace_types.py"
-      - "scripts/trace_codegen/**"
-      - "scripts/lens_assets/**"
-      - "litellm/tracing/**"
       - "tests/test_litellm_rust/**"
       - "litellm/integrations/custom_logger.py"
       - "litellm/litellm_core_utils/litellm_logging.py"
@@ -17,12 +12,10 @@ on:
       - "litellm/proxy/guardrails/**"
       - "litellm/utils.py"
       - "litellm/ocr/**"
-      - "litellm/chat_completions/dispatch.py"
-      - "litellm/embeddings/dispatch.py"
-      - "litellm/messages/dispatch.py"
-      - "litellm/responses/dispatch.py"
       - "litellm/llms/base_llm/ocr/**"
       - "litellm/llms/custom_httpx/llm_http_handler.py"
+      - "tests/test_litellm/ocr/**"
+      - "tests/test_litellm/conftest.py"
       - "Makefile"
       - ".cargo/**"
       - "pyproject.toml"
@@ -31,20 +24,17 @@ on:
       - ".github/actions/setup-uv-with-retries/**"
       - ".github/scripts/smoke_test_native_wheel.py"
       - ".github/scripts/verify_linux_native_wheel.py"
-      - "tests/unit/rust_bridge/native_route_wheel_test.py"
+      - "tests/test_litellm/rust_bridge/native_route_wheel_test.py"
       - ".github/workflows/test-rust.yml"
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
     paths:
       - "litellm-rust/**"
-      - "model_prices_and_context_window.json"
       - "litellm/rust_bridge/**"
-      - "scripts/generate_trace_types.py"
-      - "scripts/trace_codegen/**"
-      - "scripts/lens_assets/**"
-      - "litellm/tracing/**"
       - "tests/test_litellm_rust/**"
       - "litellm/integrations/custom_logger.py"
       - "litellm/litellm_core_utils/litellm_logging.py"
@@ -52,12 +42,10 @@ on:
       - "litellm/proxy/guardrails/**"
       - "litellm/utils.py"
       - "litellm/ocr/**"
-      - "litellm/chat_completions/dispatch.py"
-      - "litellm/embeddings/dispatch.py"
-      - "litellm/messages/dispatch.py"
-      - "litellm/responses/dispatch.py"
       - "litellm/llms/base_llm/ocr/**"
       - "litellm/llms/custom_httpx/llm_http_handler.py"
+      - "tests/test_litellm/ocr/**"
+      - "tests/test_litellm/conftest.py"
       - "Makefile"
       - ".cargo/**"
       - "pyproject.toml"
@@ -66,7 +54,7 @@ on:
       - ".github/actions/setup-uv-with-retries/**"
       - ".github/scripts/smoke_test_native_wheel.py"
       - ".github/scripts/verify_linux_native_wheel.py"
-      - "tests/unit/rust_bridge/native_route_wheel_test.py"
+      - "tests/test_litellm/rust_bridge/native_route_wheel_test.py"
       - ".github/workflows/test-rust.yml"
 
 permissions:
@@ -78,11 +66,6 @@ concurrency:
 
 env:
   CARGO_TERM_COLOR: always
-  RUSTC_WRAPPER: sccache
-  SCCACHE_GHA_ENABLED: "true"
-  SCCACHE_GHA_VERSION: litellm-ci-v1
-  SCCACHE_GHA_RW_MODE: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
-  CARGO_INCREMENTAL: "0"
 
 jobs:
   rust-lint:
@@ -104,21 +87,12 @@ jobs:
         with:
           workspaces: litellm-rust
           cache-on-failure: true
-          save-if: ${{ github.ref == 'refs/heads/main' }}
-
-      - name: Set up sccache
-        uses: mozilla-actions/sccache-action@d651010b8da762cde178750d8eda7b5febfe147a
-        with:
-          version: "v0.18.0"
 
       - run: cargo clippy --workspace --all-targets --locked -- -D warnings
 
   rust-test:
     runs-on: ubuntu-latest
-    timeout-minutes: 30
-    # Trim debuginfo to keep the job's target dir within the runner's disk
-    env:
-      CARGO_PROFILE_DEV_DEBUG: line-tables-only
+    timeout-minutes: 20
     defaults:
       run:
         working-directory: litellm-rust
@@ -131,16 +105,6 @@ jobs:
         with:
           python-version: "3.12"
 
-      - uses: ./.github/actions/setup-uv-with-retries
-        with:
-          version: "0.10.9"
-
-      - name: Install Python dependencies for the bridge tests
-        working-directory: .
-        run: |
-          uv sync --frozen --no-install-project
-          echo "PYTHONPATH=$PWD/.venv/lib/$(ls .venv/lib)/site-packages" >> "$GITHUB_ENV"
-
       - run: rustup toolchain install --no-self-update
 
       - uses: taiki-e/install-action@d438492cf8a250514fa2d34b30bc3c0dc37c65ff # v2.87.8
@@ -151,35 +115,11 @@ jobs:
         with:
           workspaces: litellm-rust
           cache-on-failure: true
-          save-if: ${{ github.ref == 'refs/heads/main' }}
-
-      - name: Set up sccache
-        uses: mozilla-actions/sccache-action@d651010b8da762cde178750d8eda7b5febfe147a
-        with:
-          version: "v0.18.0"
-
-      - name: Check generated trace contracts
-        working-directory: .
-        run: uv run scripts/generate_trace_types.py --check
 
       - run: cargo nextest run --workspace --locked
 
       - run: cargo test --workspace --doc --locked
 
-      - name: Test token counter feature combinations
-        run: |
-          for features in '' fast huggingface tiktoken fast,huggingface fast,tiktoken huggingface,tiktoken fast,huggingface,tiktoken; do
-            cargo test -p litellm-token-counter --locked --no-default-features --features "$features"
-            cargo check -p litellm-python-bridge --locked --no-default-features --features "abi3${features:+,$features}"
-          done
-
-      - name: Test secret manager feature combinations
-        run: |
-          cargo test -p litellm-auth-gcp --locked --no-default-features
-          for features in '' aws google hashicorp azure cyberark aws,google aws,azure google,azure aws,google,azure aws,google,cyberark aws,google,azure,cyberark aws,google,hashicorp,azure,cyberark; do
-            cargo test -p litellm-secrets --locked --no-default-features --features "$features"
-          done
-
   rust-wheel:
     runs-on: ubuntu-latest
     timeout-minutes: 30
@@ -202,12 +142,6 @@ jobs:
         with:
           workspaces: litellm-rust
           cache-on-failure: true
-          save-if: ${{ github.ref == 'refs/heads/main' }}
-
-      - name: Set up sccache
-        uses: mozilla-actions/sccache-action@d651010b8da762cde178750d8eda7b5febfe147a
-        with:
-          version: "v0.18.0"
 
       - run: uv build --wheel --out-dir dist
 
@@ -215,7 +149,7 @@ jobs:
         env:
           RELEASE_WHEEL_COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
 
-      - run: python tests/unit/rust_bridge/native_route_wheel_test.py dist/*.whl
+      - run: python tests/test_litellm/rust_bridge/native_route_wheel_test.py dist/*.whl
 
       - name: Run pytest tests/test_litellm_rust with the compiled extension
         run: make test-rust-extension

.github/workflows/test-semgrep.yml

@@ -0,0 +1,39 @@
+name: Semgrep
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  semgrep:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Run Semgrep (custom rules)
+        run: uv tool run --from 'semgrep==1.157.0' semgrep scan --config .semgrep/rules . --error

.github/workflows/test-terraform-modules.yml

@@ -9,6 +9,8 @@ on:
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
     paths:
       - "terraform/litellm/aws/**"

.github/workflows/test-terraform-provider.yml

@@ -8,11 +8,12 @@ on:
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
     paths:
       - "terraform/provider/**"
       - "litellm/proxy/**"
-      - "enterprise/**"
       - ".github/workflows/test-terraform-provider.yml"
 
 permissions:
@@ -78,7 +79,6 @@ jobs:
           version: "0.10.9"
 
       - name: Cache uv dependencies
-        if: github.ref == 'refs/heads/main'
         uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
         with:
           path: |
@@ -88,17 +88,6 @@ jobs:
           restore-keys: |
             ${{ runner.os }}-uv-
 
-      - name: Cache uv dependencies
-        if: github.ref != 'refs/heads/main'
-        uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
-        with:
-          path: |
-            ~/.cache/uv
-            .venv
-          key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
-          restore-keys: |
-            ${{ runner.os }}-uv-
-
       - name: Cache the Rust build
         uses: ./.github/actions/cache-cargo-build
 

.github/workflows/test-unit-documentation.yml

@@ -0,0 +1,95 @@
+name: "Unit Tests: Documentation Validation"
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+  push:
+    branches:
+      - main
+      - litellm_internal_staging
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  documentation:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    permissions:
+      contents: read
+      pull-requests: read
+
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Detect relevant changes
+        id: changes
+        uses: ./.github/actions/detect-changes
+
+      - name: Checkout litellm-docs into docs/my-website (for documentation_tests)
+        if: steps.changes.outputs.decision != 'skip'
+        uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          repository: BerriAI/litellm-docs
+          path: docs/my-website
+          persist-credentials: false
+
+      - name: Set up Python
+        if: steps.changes.outputs.decision != 'skip'
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        if: steps.changes.outputs.decision != 'skip'
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Cache uv dependencies
+        if: steps.changes.outputs.decision != 'skip'
+        uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+        with:
+          path: |
+            ~/.cache/uv
+            .venv
+          key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-uv-
+
+      - name: Cache the Rust build
+        if: steps.changes.outputs.decision != 'skip'
+        uses: ./.github/actions/cache-cargo-build
+
+      - name: Install dependencies
+        if: steps.changes.outputs.decision != 'skip'
+        run: |
+          .github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra google --extra proxy --extra semantic-router
+
+      - name: Cache Prisma binaries
+        if: steps.changes.outputs.decision != 'skip'
+        uses: ./.github/actions/cache-prisma-binaries
+
+      - name: Generate Prisma client
+        if: steps.changes.outputs.decision != 'skip'
+        run: |
+          uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma
+
+      # Run the same documentation tests that CircleCI ran (as direct Python scripts)
+      - name: Run documentation validation tests
+        if: steps.changes.outputs.decision != 'skip'
+        run: |
+          uv run --no-sync python ./tests/documentation_tests/test_env_keys.py
+          uv run --no-sync python ./tests/documentation_tests/test_router_settings.py
+          uv run --no-sync python ./tests/documentation_tests/test_api_docs.py
+          uv run --no-sync python ./tests/documentation_tests/test_circular_imports.py

.github/workflows/test-unit-proxy-db.yml

@@ -0,0 +1,207 @@
+name: "Unit Tests: Proxy DB Operations"
+
+on:
+  pull_request:
+    branches:
+      - main
+      - litellm_internal_staging
+      - litellm_oss_staging
+      - "litellm_**"
+  push:
+    branches:
+      - main
+      - litellm_internal_staging
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+# Semantic matrix: each shard groups tests by concern (auth, server, logging, …)
+# rather than alphabetical letter ranges. Adding a new test file means adding it
+# to whichever group it belongs to, not reshuffling slices.
+#
+# Design targets:
+#   * Every shard runs in <= 7 minutes of wall-clock on the default runner.
+#     Most of a shard's time is pytest plugin load + xdist worker imports +
+#     pytest-cov instrumentation, not the tests themselves. Keeping per-shard
+#     work low and matching worker count to runner cores is what controls it.
+#   * `timeout` bounds the pytest step only. Checkout, dependency install, and
+#     Prisma client generation draw on a separate allowance in the base
+#     workflow, so slow setup shows up as a slow job rather than as a
+#     cancelled shard whose tests were passing.
+#   * workers: 4 matches the 4-core ubuntu-latest runner. -n 8 on 4 cores
+#     oversubscribes 2x and workers fight for CPU during their cold-start
+#     imports (measured ~441% CPU for -n 8 locally, i.e. ~55% effective).
+#   * test_key_generate_prisma.py stays serial (workers=0) — it has event-loop
+#     conflicts with the logging worker when run in parallel.
+#   * test_proxy_utils.py runs as a single shard with --dist=worksteal so
+#     xdist balances its 188 parametrized cases across workers instead of
+#     pinning the whole file to one worker (the default --dist=loadscope
+#     behavior for single-file targets).
+jobs:
+  # Fast guard — fails the workflow when a test directory or file inside a sharded
+  # tree is claimed by no shard. The semantic-shard design has no catch-all bucket,
+  # so an unassigned child runs nowhere; assert_ci_coverage.py holds the tree list
+  # and reads the same test-path keys the coverage census does.
+  assert-shard-coverage:
+    runs-on: ubuntu-latest
+    timeout-minutes: 2
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+      - name: Assert every test directory and file is claimed by a shard
+        run: python3 .github/scripts/assert_ci_coverage.py --shards
+
+  proxy-db:
+    needs: assert-shard-coverage
+    # Display only the semantic shard name in the checks UI instead of GHA's
+    # default "proxy-db (key-generation, tests/proxy_unit_tests/…, 0, loadscope, 20)"
+    # which includes every matrix field and gets truncated past the test-path.
+    name: ${{ matrix.test-group }}
+    permissions:
+      contents: read
+      id-token: write
+      pull-requests: write
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          # Must run serially — event-loop conflict with the logging worker.
+          - test-group: key-generation
+            test-path: "tests/proxy_unit_tests/test_key_generate_prisma.py"
+            workers: 0
+            dist: loadscope
+            timeout: 20
+
+          # ---- auth: split into 2 shards ----
+          - test-group: auth-checks
+            test-path: >-
+              tests/proxy_unit_tests/test_auth_checks.py
+              tests/proxy_unit_tests/test_user_api_key_auth.py
+              tests/proxy_unit_tests/test_deprecated_key_grace_period.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+          - test-group: jwt-and-keys
+            test-path: >-
+              tests/proxy_unit_tests/test_jwt.py
+              tests/proxy_unit_tests/test_jwt_key_mapping.py
+              tests/proxy_unit_tests/test_proxy_custom_auth.py
+              tests/proxy_unit_tests/test_key_generate_dynamodb.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+
+          # ---- test_proxy_utils.py, single shard, worksteal distribution ----
+          - test-group: proxy-utils
+            test-path: "tests/proxy_unit_tests/test_proxy_utils.py"
+            workers: 4
+            dist: worksteal
+            timeout: 15
+
+          # ---- proxy server: split into 2 shards ----
+          - test-group: proxy-server-core
+            test-path: >-
+              tests/proxy_unit_tests/test_proxy_server.py
+              tests/proxy_unit_tests/test_aproxy_startup.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+          - test-group: proxy-runtime
+            test-path: >-
+              tests/proxy_unit_tests/test_proxy_config_unit_test.py
+              tests/proxy_unit_tests/test_proxy_routes.py
+              tests/proxy_unit_tests/test_server_root_path.py
+              tests/proxy_unit_tests/test_proxy_pass_user_config.py
+              tests/proxy_unit_tests/test_proxy_token_counter.py
+              tests/proxy_unit_tests/test_request_size_limit_middleware.py
+              tests/proxy_unit_tests/test_multipart_bypass_repro.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+
+          # ---- logging: split into 2 shards ----
+          - test-group: custom-logging
+            test-path: >-
+              tests/proxy_unit_tests/test_custom_callback_input.py
+              tests/proxy_unit_tests/test_custom_logger_s3_gcs.py
+              tests/proxy_unit_tests/test_proxy_custom_logger.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+          - test-group: logging-misc
+            test-path: >-
+              tests/proxy_unit_tests/test_proxy_reject_logging.py
+              tests/proxy_unit_tests/test_audit_logs_proxy.py
+              tests/proxy_unit_tests/test_search_api_logging.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+
+          - test-group: db-and-spend
+            test-path: >-
+              tests/proxy_unit_tests/test_prisma_client_backoff_retry.py
+              tests/proxy_unit_tests/test_db_schema_changes.py
+              tests/proxy_unit_tests/test_e2e_pod_lock_manager.py
+              tests/proxy_unit_tests/test_skills_db.py
+              tests/proxy_unit_tests/test_update_daily_tag_spend.py
+              tests/proxy_unit_tests/test_update_spend.py
+              tests/proxy_unit_tests/test_proxy_encrypt_decrypt.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+
+          # ---- guardrails + budget + hooks: split into 2 ----
+          - test-group: guardrails-hooks
+            test-path: >-
+              tests/proxy_unit_tests/test_proxy_setting_guardrails.py
+              tests/proxy_unit_tests/test_banned_keyword_list.py
+              tests/proxy_unit_tests/test_unit_test_proxy_hooks.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+          - test-group: budgets
+            test-path: >-
+              tests/proxy_unit_tests/test_default_end_user_budget_simple.py
+              tests/proxy_unit_tests/test_unit_test_max_model_budget_limiter.py
+              tests/proxy_unit_tests/test_zero_cost_model_budget_bypass.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+
+          - test-group: endpoints-and-responses
+            test-path: >-
+              tests/proxy_unit_tests/test_blog_posts_endpoint.py
+              tests/proxy_unit_tests/test_models_fallback_endpoint.py
+              tests/proxy_unit_tests/test_google_endpoint_routing.py
+              tests/proxy_unit_tests/test_google_gemini_proxy_request.py
+              tests/proxy_unit_tests/test_gemini_agents_endpoints.py
+              tests/proxy_unit_tests/test_get_favicon.py
+              tests/proxy_unit_tests/test_get_image.py
+              tests/proxy_unit_tests/test_reducto_ocr_route.py
+              tests/proxy_unit_tests/test_ui_path_detection.py
+              tests/proxy_unit_tests/test_prompt_test_endpoint.py
+              tests/proxy_unit_tests/test_check_batch_cost.py
+              tests/proxy_unit_tests/test_check_responses_cost.py
+              tests/proxy_unit_tests/test_response_polling_handler.py
+              tests/proxy_unit_tests/test_response_polling_pre_call_checks.py
+              tests/proxy_unit_tests/test_realtime_cache.py
+              tests/proxy_unit_tests/test_proxy_exception_mapping.py
+              tests/proxy_unit_tests/test_custom_tokenizer_bug.py
+            workers: 4
+            dist: loadscope
+            timeout: 15
+    uses: ./.github/workflows/_test-unit-base.yml
+    with:
+      test-path: ${{ matrix.test-path }}
+      workers: ${{ matrix.workers }}
+      reruns: 2
+      timeout-minutes: ${{ matrix.timeout }}
+      dist: ${{ matrix.dist }}
+      artifact-name: proxy-db-${{ matrix.test-group }}

.github/workflows/test-vscode-extension.yml

@@ -6,6 +6,8 @@ on:
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
     paths:
       - "vscode-extension/**"

.github/workflows/weekly_load_anomaly.yml

@@ -0,0 +1,85 @@
+name: "Weekly Load Anomaly Check"
+
+on:
+  schedule:
+    - cron: "0 12 * * 6"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  weekly-load-anomaly:
+    if: github.event_name != 'schedule' || github.repository == 'BerriAI/litellm'
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    services:
+      postgres:
+        image: postgres:16.6
+        env:
+          POSTGRES_USER: llmproxy
+          POSTGRES_PASSWORD: dbpassword9090
+          POSTGRES_DB: litellm
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U llmproxy"
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 10
+    env:
+      DATABASE_URL: postgresql://llmproxy:dbpassword9090@localhost:5432/litellm
+      LITELLM_MASTER_KEY: sk-weekly-anomaly-check
+      ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+      AWS_BEARER_TOKEN_BEDROCK: ${{ secrets.AWS_BEARER_TOKEN_BEDROCK }}
+    steps:
+      - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
+        with:
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: ./.github/actions/setup-uv-with-retries
+        with:
+          version: "0.10.9"
+
+      - name: Cache the Rust build
+        uses: ./.github/actions/cache-cargo-build
+
+      - name: Install dependencies
+        run: |
+          .github/scripts/uv_sync_with_retries.sh --frozen --group ci --group proxy-dev --extra proxy
+
+      - name: Cache Prisma binaries
+        uses: ./.github/actions/cache-prisma-binaries
+
+      - name: Generate Prisma client
+        run: |
+          uv run --no-sync prisma generate --schema litellm/proxy/schema.prisma
+
+      - name: Start the proxy
+        run: |
+          nohup uv run --no-sync litellm --config tests/e2e/load/weekly_anomaly_config.yml --port 4000 > proxy.log 2>&1 &
+          for _ in $(seq 1 90); do
+            if curl -fs http://localhost:4000/health/liveliness > /dev/null; then
+              exit 0
+            fi
+            sleep 2
+          done
+          echo "proxy never became live"
+          tail -n 100 proxy.log
+          exit 1
+
+      - name: Run the weekly session anomaly test
+        env:
+          E2E_WEEKLY_ANOMALY: "1"
+        run: |
+          uv run --no-sync pytest tests/e2e/load/test_weekly_session_anomaly_e2e.py -v --tb=short -rA
+
+      - name: Show proxy log on failure
+        if: failure()
+        run: tail -n 300 proxy.log

.github/workflows/zizmor.yml

@@ -2,10 +2,12 @@ name: GitHub Actions Security Analysis
 
 on:
   push:
-    branches: [main]
+    branches: [main, litellm_internal_staging]
   pull_request:
     branches:
       - main
+      - litellm_internal_staging
+      - litellm_oss_staging
       - "litellm_**"
 
 concurrency:
@@ -27,15 +29,10 @@ jobs:
         with:
           persist-credentials: false
 
-      - name: Install zizmor
-        run: |
-          curl -fsSL --retry 3 -o "$RUNNER_TEMP/zizmor.tar.gz" \
-            https://github.com/zizmorcore/zizmor/releases/download/v1.31.1/zizmor-x86_64-unknown-linux-gnu.tar.gz
-          echo "d185f726bdaf92df78273217dfdd24ebcd6b0ef48cf3ce776ff5762b6e138c99  $RUNNER_TEMP/zizmor.tar.gz" | sha256sum -c -
-          tar xzf "$RUNNER_TEMP/zizmor.tar.gz" -C "$RUNNER_TEMP" zizmor
-
       - name: Run zizmor
-        env:
-          GH_TOKEN: ${{ github.token }}
-        run: |
-          "$RUNNER_TEMP/zizmor" --no-ignores --no-config --min-severity=medium --format=github .
+        uses: zizmorcore/zizmor-action@5f14fd08f7cf1cb1609c1e344975f152c7ee938d # v0.5.6
+        with:
+          version: "1.24.1"
+          min-severity: medium
+          advanced-security: false
+          annotations: true

.gitignore

@@ -58,7 +58,6 @@ litellm/proxy/tests/package-lock.json
 ui/litellm-dashboard/.next
 ui/litellm-dashboard/node_modules
 ui/litellm-dashboard/next-env.d.ts
-*.tsbuildinfo
 ui/litellm-dashboard/package.json
 ui/litellm-dashboard/package-lock.json
 helm/litellm-helm/*.tgz
@@ -105,7 +104,7 @@ litellm_config.yaml
 .cursor
 litellm/proxy/to_delete_loadtest_work/*
 update_model_cost_map.py
-tests/unit/proxy/mcp_server/test_mcp_server_manager.py
+tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py
 scripts/test_vertex_ai_search.py
 LAZY_LOADING_IMPROVEMENTS.md
 STABILIZATION_TODO.md
@@ -151,7 +150,3 @@ litellm.log
 
 .coverage-rust
 coverage-rust.xml
-
-# make lens-dev worker token, generated config and logs
-.lens-dev/
-.mypy_cache/

.grype.yaml

@@ -11,12 +11,3 @@ ignore:
     package:
       name: zlib
       type: apk
-  - vulnerability: GHSA-v4x8-3wr5-3wq8
-    reason: >-
-      filelock ReadWriteLock advisory, and no package in the image takes ReadWriteLock;
-      the fix (4.2.1) is blocked by litellm's own filelock<4.0 cap until #48635 or #48802 lifts it,
-      and the image then moves off 3.32.7, which retires this version-scoped rule
-    package:
-      name: filelock
-      version: 3.32.7
-      type: python

AGENTS.md

@@ -23,36 +23,30 @@ When adding new features, add meaningful tests. Don't add tests that don't check
 
 Same thing for bug fixes. The tests should make it so that this specific bug can never happen again without failing tests (i.e., regression)
 
-Never test structure or implementation details of code, only function of it. A test should catch a change that breaks behavior by accident. If the only way it can fail is someone deliberately undoing the change, don't write it
+Never test structure of code only function of it
 
-A test must only fail when litellm code changes. Never pin facts we don't own (a vendor's price, a third party's field, an upstream default, today's date) as literals or as "X must be absent". Assert the invariant our code guarantees instead, e.g. two rows agree, a value is within range, a field is derived from another. If an outside fact is truly load-bearing, cite its source and date next to the assertion so a reader can tell stale from broken
+A test must only fail when litellm code changes. Never pin facts we don't own (a vendor's price, a third party's field, an upstream default, today's date) as literals or as "X must be absent"; assert the invariant our code guarantees instead, e.g. two rows agree, a value is within range, a field is derived from another. If an outside fact is truly load-bearing, cite its source and date next to the assertion so a reader can tell stale from broken
 
-`tests/unit/` mirrors `litellm/` in a parallel path (see `tests/unit/AGENTS.md`). Name tests `test_<filename>.py`, but always match the existing test file in the directory you touch — many provider dirs use longer descriptive names (e.g. `test_anthropic_chat_transformation.py`) to avoid ambiguity across sibling folders. For bug fixes, extend the existing mapped test file rather than creating a new one. Only create a new test file for a new feature (provider, endpoint, or transformation module) that has no mapped test yet, following that directory's naming convention (or `test_<filename>.py` if you're the first test there). One focused regression test beats many shallow ones
+`tests/test_litellm/` mirrors `litellm/` in a parallel path (see `tests/test_litellm/readme.md`). Name tests `test_<filename>.py`, but always match the existing test file in the directory you touch — many provider dirs use longer descriptive names (e.g. `test_anthropic_chat_transformation.py`) to avoid ambiguity across sibling folders. For bug fixes, extend the existing mapped test file rather than creating a new one. Only create a new test file for a new feature (provider, endpoint, or transformation module) that has no mapped test yet, following that directory's naming convention (or `test_<filename>.py` if you're the first test there). One focused regression test beats many shallow ones
 
-End-to-end tests belong in `tests/e2e/`. A rule about one directory's files, like its harness or which CI runs it, goes in that directory's `AGENTS.md`, never only in a README or here. Read a directory's `AGENTS.md` before editing files in it
-
-Before you build a new way to move data between the endpoints, the provider transformations, and cost tracking or logging, read the architecture decision records in `docs/adrs/`, which say why those layers are shaped the way they are
+End-to-end tests belong in `tests/e2e/` and must follow the harness conventions documented in that directory's `AGENTS.md`
 
 When creating PRs, target the repository's current default branch for both internal and external / OSS contributions. Check it with `python3 scripts/default_branch.py --branch` instead of assuming a branch name or relying on cached `origin/HEAD`
 
-When writing a PR body, treat the comments and imperative instructions inside .github/PULL_REQUEST_TEMPLATE/general.md as rules to follow, not just layout. Agent harnesses may strip HTML comments from copies of that file injected into context, so read .github/PULL_REQUEST_TEMPLATE/general.md from disk before writing a PR body to make sure you see every comment rule. A section you have nothing to put in (Relevant issues, Affected release, Linear ticket, Caveats, QA runbook, and so on) is removed entirely, heading included, never left as an empty title
+When writing a PR body, treat the comments and imperative instructions inside .github/pull_request_template.md as rules to follow, not just layout. Agent harnesses may strip HTML comments from copies of that file injected into context, so read .github/pull_request_template.md from disk before writing a PR body to make sure you see every comment rule
 
 Same applies for filing bug reports and feature requests, with .github/ISSUE_TEMPLATE/bug_report.yml and .github/ISSUE_TEMPLATE/feature_request.yml, respectively
 
-If you're resolving a linear ticket, in the "## Linear ticket" section of the PR, say "Resolves LIT-1234", replacing "LIT-1234" with the actual ticket id that you're resolving. If you don't have the ticket id, don't make one up or search for it. Just drop the section
+If you're resolving a linear ticket, in the "## Linear ticket" section of the PR, say "Resolves LIT-1234", replacing "LIT-1234" with the actual ticket id that you're resolving. If you don't have the ticket id, don't make one up or search for it. Just leave the section blank
 
-Run QA proxies with `python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --num_workers 2 --detailed_debug --reload --use_v2_migration_resolver 2>&1 | tee litellm.log`, and swap its `--config` for a copy of `litellm/proxy/dev_config.yaml` that points both `router_settings` (`redis_host`, `redis_port`) and `general_settings.coordination_redis` (`host`, `port`) at one dedicated Redis instance, with credentials when required, so the workers share router state and spend, budget, and rate-limit counters. Check that two worker PIDs are serving, and use the same worker count and Redis setup for the before and after proof. When the PR is about `--run_gunicorn`, `--run_hypercorn`, or `--run_granian`, which `--reload` does not support, drop `--reload`. When it is about behavior without Redis, drop Redis. When the proxy prints `"workers" flag is ignored when reloading is enabled`, that checkout or its uvicorn reloads only one worker, so drop the two-worker check for it. Say in the proof which part you dropped and why
-
-Simple PRs may skip before/after e2e when the full diff is mechanical and focused checks completely prove it, such as #49030 restoring an existing helper name. Trace affected callers and run relevant tests/static checks at the tip. For a bug fix, establish that the regression catches the original defect. Size alone never qualifies. New behavior, API/provider semantics, auth, billing, persistence, concurrency or uncertain integration risk need live e2e. In Screenshots / Proof of Fix, record `E2E exception at <tip SHA>`, scope, rationale, commands and results. Tests count as proof only for this exception. Reassess changed diffs, retain CI/bot gates, and honor explicit e2e requests (2026-10-28, unnecessary live QA for #49030)
-
-For other PRs, proof must show the end user's real flow before and after, with commands and output against a live proxy and real provider calls where applicable. Pytest alone is insufficient. For UI or interactive coding-tool flows, drive that surface and embed before/after screenshots, with URLs, clicks and input fields a reviewer can reproduce
+Never use `pytest` commands or the like as "Screenshots / Proof of Fix". We prefer curl'ing a live proxy instance running on localhost:4000 (I like to run it with `python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --detailed_debug --reload --use_v2_migration_resolver 2>&1 | tee litellm.log`; the Admin UI dev server is `npm run dev` in `ui/litellm-dashboard`, served on port 3000) and showing both the command run and the output. Also, it should hit real LLM provider APIs, not mocks, and cost real $$$ because that is the most realistic test. The proof of fix should be exactly what the end user / customer would see / do. The run logs in PR #27703 is a prime example of how to do it (not a huge fan of using a python test script that future me and the team will have no visibility into; I prefer just curl commands or a short list of bash commands (e.g., using `for`)). If it's a UI thing, or the main use case runs through a headful agentic coding tool like Claude Code or Codex, drive that surface yourself and embed your own before and after screenshots of it in the PR (the Admin UI page, or what the coding tool shows), next to an ordered list of the URLs to go to (e.g., http://localhost:4000/ui/?page=logs), where to click, and what fields to fill out so a reviewer can reproduce it
 
 If you ever write any human-facing text (pull requests, issues, commit messages, discussion posts, github comments, release notes, docs, etc.), always follow these guidelines to sound less AI-y:
 - don't use emojis
-- don't use "—". Don't use ";" or ":" unless it would be nonsensical not to. Instead, reach for ",", ".", conjunction words, etc. in descending order of preference: vary among them, weighted toward the front of the list, and skip "," where it would cause a comma splice or the sentence is getting long. Overusing any one of them also feels AI-y. A word cap does not penalize you for adding more sentences
+- don't use "—". Instead, reach for ",", ".", conjunction words, ":", ";", etc. in descending order of preference: vary among them, weighted toward the front of the list, and skip "," where it would cause a comma splice or the sentence is getting long. Overusing any one of them, ";" especially, also feels AI-y. A word cap does not penalize you for adding more sentences: when writing under tight word budgets, prefer a period split or a conjunction over ";", and keep to at most one ";" per message
 - don't use the pattern "It's not X, it's Y", "You're not X, you're Y", etc.
 - unless explicitly asked, don't use bulleted or numbered lists unless it would be nonsensical not to. Instead, prefer prose
-- don't add a trailing "." at the end of paragraphs (just like this file). That means every paragraph, not just the last one (of the markdown file, PR description, GitHub comment, etc.). Rule of thumb: if you're adding new line(s) before the next sentence, don't add a ".". Exception is that docs (i.e., litellm-docs) should have trailing "."
+- don't add a trailing "." at the end of paragraphs (just like this file). That means every paragraph, not just the last one (of the markdown file, PR description, GitHub comment, etc.). Rule of thumb: if you're adding new line(s) before the next sentence, don't add a "."
 - don't use →. Instead, prefer not to use arrows, and if need be, use -> instead
 - use plain, simple, everyday engineering language: the common phrase engineers actually say over rare compact phrasing, in grammatically complete sentences. When explicitly asked to use bullets or ordered lists and structure legitimately helps the reader, prefer nested bullets (any depth is fine) over dense lines in a flat structure
 
@@ -60,25 +54,25 @@ Don't hesitate to use values in .env to get needed API keys and other secrets, a
 
 Python max line length is 120, not 88
 
-The five lint gates (`scripts/ruff_strict_gate.py`, `scripts/type_discipline_gate.py`, `scripts/test_quality_gate.py`, `scripts/openapi_docs_gate.py`, `scripts/type_check_gate.py`) compare each rule's codebase count on your branch against the count at its merge-base with the default branch, and a rule may not grow. There are no budget files to edit or ratchet: when a gate fails, fix the violations the branch introduced or remove at least as many of that rule elsewhere in the tree. The one exception is `reportAny` / `reportExplicitAny`, which share a fixed codebase-wide cap in `ANY_CAPS` in `scripts/type_check_gate.py` because Any spreads past the lines you touch. A branch may add Anys while the total stays under that cap. Only lower `ANY_CAPS`, and only in its own PR to the default branch, never raise it on a feature branch
+Never edit or commit `ruff-strict-budget.json`, `type-discipline-budget.json`, `basedpyright-code-budget.json`, or `test-quality-budget.json` on a PR branch, and don't run `make lint-budget-update` there. A scheduled Devin automation lowers the limits on the default branch in its own PR by exactly what landed since the last ratchet, so concurrent PRs don't fight over the same `"limit"` lines. Keep the hosted automation's target in sync when the repository default changes. If your branch already carries a budget edit, drop it before opening the PR
 
 `make check` (f.k.a. `make pre-commit`, which still works identically as an alias) saves its complete output to a log file in .git (overwriting previous logs) and prints that path as its first and last output lines. To inspect a run, read or grep that log instead of re-running the multi-minute checks just to see a different slice
 
-`make check`, `make lint`, `scripts/pre_commit_lint.sh`, and the standalone lint gates (`scripts/ruff_strict_gate.py`, `scripts/type_discipline_gate.py`, `scripts/type_check_gate.py`) each hold one of 2 machine-wide slots, so when other sessions or worktrees on the same box are already running heavy work, yours prints "all N machine-wide slots are busy" followed by "queueing" and then stays quiet until a slot frees. Give the command a long timeout and let it wait rather than killing it, retrying it, or assuming it hung. Don't change the # of machine-wide slots or make it unlimited by setting `LITELLM_GATE_SLOTS=0`
+`make check`, `make lint`, `scripts/pre_commit_lint.sh`, and the standalone budget gates (`scripts/ruff_strict_gate.py`, `scripts/type_discipline_gate.py`, `scripts/type_check_gate.py`) each hold one of 2 machine-wide slots, so when other sessions or worktrees on the same box are already running heavy work, yours prints "all N machine-wide slots are busy; queueing" and then stays quiet until a slot frees. Give the command a long timeout and let it wait rather than killing it, retrying it, or assuming it hung. Don't change the # of machine-wide slots or make it unlimited by setting `LITELLM_GATE_SLOTS=0`
 
-If you're trying to create a new function that relies on untyped stuff, instead of adding more Any's and pushing `reportAny` / `reportExplicitAny` closer to their cap in `ANY_CAPS`, just validate it in the caller with Pydantic (a model or `TypeAdapter` that returns the typed thing or raises will do) and then pass the now typed variable in
+If you're trying to create a new function that relies on untyped stuff, instead of adding more Any's and pushing `reportAny` / `reportExplicitAny` closer to their basedpyright ceilings, just validate it in the caller with Pydantic (a model or `TypeAdapter` that returns the typed thing or raises will do) and then pass the now typed variable in
 
-If you get an LIT001 fail, refactor the code to follow functional programming best practices rather than introducing mutable data structures. For example, build values in one shot with comprehensions or generators wrapped in `tuple()` / `MappingProxyType()` / `frozenset()` instead of seeding an empty `list`/`dict`/`set` and mutating it over time. Ideally, `# mutable-ok` is never used. Reach for it only as a genuine last resort when an immutable rewrite is truly impossible, and always pair it with a real reason
+If you get an LIT001 or LIT002 fail, refactor the code to follow functional programming best practices rather than introducing mutable data structures. For example, build values in one shot with comprehensions or generators wrapped in `tuple()` / `MappingProxyType()` / `frozenset()` instead of seeding an empty `list`/`dict`/`set` and mutating it over time. Ideally, `# mutable-ok` is never used; reach for it only as a genuine last resort when an immutable rewrite is truly impossible, and always pair it with a real reason
 
 Every lint or type suppression must name the exact rule inside brackets and carry a reason comment, e.g. `# pyright: ignore[reportArgumentType]  # stubs lack async overload` or `# noqa: TID251  # <reason>`. `# type: ignore` is banned (LIT009): pyrightconfig.json sets `enableTypeIgnoreComments` to false, so it silently does nothing
 
 Commit and push your work when you're done without asking
 
-When referencing or running models (coding, QA'ing, writing docs, writing tests, etc.), use the latest model in that model family unless otherwise specified. Treat your training knowledge, memories, configs, and tests as stale, and determine the family's latest with model_prices_and_context_window.json or the web
+When referencing or running models (coding, QA'ing, writing docs, writing tests, etc.), use the latest model in that model family unless otherwise specified; treat your training knowledge, memories, configs, and tests as stale, and determine the family's latest with model_prices_and_context_window.json or the web
 
 Always pull before starting any work. The checkout or worktree may be sitting on a stale branch
 
-Internal contributors branch a new PR off the repository's current default branch, with a branch name prefixed with litellm_
+If you're an internal contributor, when creating a new PR, the typical flow is to branch off the repository's current default branch and create a branch prefixed with litellm_. Do not create a branch prefixed with claude/ and generally do not have / in your branch names
 
 Do not add `Co-Authored-By: Claude` or any Claude attribution to commit messages. Never use a `claude/` prefix or put a `/` in a branch name. Do not add "Generated with Claude Code" (or any similar attribution) to PR descriptions or comments. Do not create a new PR/branch off the existing PR to fix/add something that is related and could've just been committed directly to the existing PR's branch
 
@@ -90,9 +84,7 @@ Monkeypatching attributes of a class to do testing is an anti-pattern. Prefer de
 
 Do not put names of customers or customer company names in code, PR descriptions, issue bodies, etc. This means never mention literally any company name. Especially if you're about to say a sentence mentioning that the reason the PR exists was a feature/model/bug fix/etc. requested by a company. That's the indication that you should replace that company name with "the customer". e.g. not "Model request from Acme (Pylon #1234)" but "Model request from a customer (Pylon #1234)". This is because the codebase is public. The only exception is for publicly known providers or vendors such as OpenAI, Anthropic, AWS Bedrock, etc. only IF we're adding support for that provider/vendor in general and NOT if that PR or whatnot was a request by one of them, and they're actually one of our customers.
 
-CI supply-chain safety: Never pipe a remote script into a shell (`curl ... | bash`, `wget ... | sh`). Download the artifact to a file, verify its SHA-256 checksum, then install. Pin every external tool to a specific version with a full URL (not `latest` or `stable`). Verify checksums for all downloaded binaries, using the provider's official `.sha256` / `.sha256sum` sidecar when available. These rules apply to every download in CI
-
-For database operations, connection ownership and proxy dependency wiring, follow [database-access](.agents/skills/database-access/SKILL.md)
+CI supply-chain safety: Never pipe a remote script into a shell (`curl ... | bash`, `wget ... | sh`); download the artifact to a file, verify its SHA-256 checksum, then install. Pin every external tool to a specific version with a full URL (not `latest` or `stable`). Verify checksums for all downloaded binaries, using the provider's official `.sha256` / `.sha256sum` sidecar when available. These rules apply to every download in CI
 
 Prisma migrations apply synchronously at proxy boot, before it serves traffic, so a migration must only change schema, never rewrite rows. No `UPDATE`, `DELETE` or `MERGE`, and no `INSERT ... SELECT`: on a spend-log-sized table any of those is minutes of downtime plus a doubled heap that plain autovacuum won't give back. `tests/code_coverage_tests/check_migrations_no_data_rewrites.py` enforces this. When a rewrite is genuinely bounded and has to ship inside the migration, mark the statement `-- data-migration-ok: <what bounds it>`
 
@@ -100,20 +92,17 @@ Follow these coding conventions for new/updated code (a three-line fix in a lega
 
 - Composition over inheritance
 - Never-nester: early returns over deep nesting
-- Don't throw. Model failures as values (One function (e.g., raise_public) maps error union to existing public exception contracts via exhaustive match + assert_never)
-- No mutation. Don't reassign variables, global or local. Instead of mutable lists and dicts, prefer tuples, frozen dataclasses (with slots=True), `MappingProxyType`, etc.
-  - Annotate every variable with `: Final` (LIT010). Unpacking and walrus targets cannot carry the annotation, so they are implicitly final. Don't rebind them. Never rebind or mutate function parameters (LIT011). `self`/`cls` attribute stores are the exception. If rebinding or in-place mutation is truly unavoidable, suppress with `# rebind-ok: <reason>`
+- Don't throw; model failures as values (One function (e.g., raise_public) maps error union to existing public exception contracts via exhaustive match + assert_never)
+- No mutation; don't reassign variables, global or local. Instead of mutable lists and dicts, prefer tuples, frozen dataclasses (with slots=True), `MappingProxyType`, etc.
+  - Annotate every variable with `: Final` (LIT010). Unpacking and walrus targets cannot carry the annotation, so they are implicitly final. Don't rebind them. Never rebind or mutate function parameters (LIT011); `self`/`cls` attribute stores are the exception. If rebinding or in-place mutation is truly unavoidable, suppress with `# rebind-ok: <reason>`
   - Qualify every TypedDict field with `ReadOnly[...]` (LIT012), which nests freely with `Required` / `NotRequired` / `Annotated` in any order. If making the key writable is truly unavoidable, suppress with `# writable-ok: <reason>`
-  - Comprehensions take at most one `for` clause and one `if` clause (LIT014). Split stacked clauses into a helper generator, a named intermediate, or a plain loop. Suppress with `# comprehension-ok: <reason>` only when unavoidable
-  - Every pydantic model must be frozen (LIT015), set via `model_config = ConfigDict(frozen=True)`, a dict-literal `model_config`, an inner `class Config`, or the class keywords. Subclasses inherit it unless they override it. Replace in-place field writes with `model_copy(update=...)`. If making the model mutable is truly unavoidable, suppress with `# frozen-ok: <reason>` on the `class` line
 - Use dependency injection
-- Fully typed. No `Any` or coarse types like `dict[str, Any]` or just `dict`. Every function parameter must be strongly typed
+- Fully typed; no `Any` or coarse types like `dict[str, Any]` or just `dict`. Every function parameter must be strongly typed
 - Use tagged unions + match
 - No monster files or god objects
 - No file sprawl: deliberate file and folder structure
-- Standard over hand-rolled: use the official SDK or a library where one exists. Where none does, follow industry standards instead of inventing local conventions
-- API-fragmentation-aware: when logic must branch on which API surface produced or consumes data (e.g. chat completions vs Anthropic Messages vs Responses API shapes), proactively look for an existing shared helper (e.g. `litellm_core_utils/prompt_templates/factory.py`) before writing per-surface parsing in the new module. If none exists, add one there instead of duplicating the same format-detection logic in every new guardrail/integration
-- Current default for new model names: logic that picks behavior from a model-name table (tokenizer, message framing, supported params) gives a provider's model the table does not list yet that provider's current behavior, never its oldest, since providers ship names faster than any table is updated
+- Standard over hand-rolled: use the official SDK or a library where one exists; where none does, follow industry standards instead of inventing local conventions
+- API-fragmentation-aware: when logic must branch on which API surface produced or consumes data (e.g. chat completions vs Anthropic Messages vs Responses API shapes), proactively look for an existing shared helper (e.g. `litellm_core_utils/prompt_templates/factory.py`) before writing per-surface parsing in the new module; if none exists, add one there instead of duplicating the same format-detection logic in every new guardrail/integration
 
 Follow conventional commits for commit names and PR titles
 
@@ -139,4 +128,4 @@ Before implementing:
 
 Ask yourself: "Would a senior engineer say this is overcomplicated?" If yes, simplify
 
-Before requesting maintainer review, verify the current PR tip passes required CI and code coverage and has acceptable reviews from the review bots on the PR, such as Veria and Bugbot. Inspect warnings and findings, fix actionable issues, and rerun the affected checks and reviewers after changes. Record evidence for any false positive or unavailable review. Never treat a pending or missing bot result as a pass. Do not lower coverage thresholds or raise `ANY_CAPS` to satisfy a check
+Before requesting maintainer review, verify the current PR tip passes required CI and code coverage, meets Greptile confidence of at least 4/5, and has acceptable Veria and Bugbot reviews. Inspect warnings and findings, fix actionable issues, and rerun the affected checks and reviewers after changes. Record evidence for any false positive or unavailable review; never treat a pending or missing bot result as a pass. Do not lower coverage thresholds or lint budgets to satisfy a check

ARCHITECTURE.md

@@ -2,8 +2,6 @@
 
 This document helps contributors understand where to make changes in LiteLLM.
 
-For why a layer is shaped the way it is, read the architecture decision records in [`docs/adrs/`](docs/adrs/AGENTS.md) (Python SDK and proxy) and [`litellm-rust/docs/adrs/`](litellm-rust/docs/adrs/AGENTS.md) (Rust core)
-
 ---
 
 ## How It Works
@@ -241,8 +239,6 @@ graph LR
 7. `DBSpendUpdateWriter.update_database()` queues spend increments to Redis
 8. Background job `update_spend` flushes queued spend to PostgreSQL every 60s
 
-Cost lookup falls back to the response model when the provider-reported name cannot be priced. Explicitly registered zero rates remain zero, and custom deployment pricing or an explicit `base_model` retain precedence
-
 ### Data Access Layer (Models & Repositories)
 
 Database entities and the operations on them live in two packages at the root of `litellm/` so both the gateway (`proxy/`) and the SDK can use them without importing proxy internals:
@@ -259,7 +255,7 @@ Conventions to follow when touching this layer:
 | Column vs. field names | Where a model field differs from its DB column (for example `org_id` maps to the `organization_id` column), the repository translates in both directions rather than relying on Pydantic to guess. |
 | Array mutations | Adds use Prisma's atomic `push` (`add_member`, `add_admin`, `add_models`) to avoid read-modify-write races. Removals fall back to read-modify-write because Prisma has no atomic array remove. |
 
-To add a new entity, define the model under `litellm/models/`, re-export it from `proxy/_types.py` if existing code imports it from there, and add a repository under `litellm/repositories/` (subclass `BaseRepository` for plain CRUD, or add bespoke methods when the entity needs encryption, archiving, or atomic array updates). Mirror the tests in `tests/unit/repositories/`.
+To add a new entity, define the model under `litellm/models/`, re-export it from `proxy/_types.py` if existing code imports it from there, and add a repository under `litellm/repositories/` (subclass `BaseRepository` for plain CRUD, or add bespoke methods when the entity needs encryption, archiving, or atomic array updates). Mirror the tests in `tests/test_litellm/repositories/`.
 
 ---
 
@@ -336,13 +332,13 @@ Each translation is isolated in its own file, making it easy to test and modify
 |--------------|----------|------------------|
 | `/v1/chat/completions` | Anthropic | `llms/anthropic/chat/transformation.py` |
 | `/v1/chat/completions` | Bedrock Converse | `llms/bedrock/chat/converse_transformation.py` |
-| `/v1/chat/completions` | Bedrock Invoke | `llms/bedrock/chat/invoke_transformations/anthropic_claude_transformation.py` |
+| `/v1/chat/completions` | Bedrock Invoke | `llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.py` |
 | `/v1/chat/completions` | Gemini | `llms/gemini/chat/transformation.py` |
 | `/v1/chat/completions` | Vertex AI | `llms/vertex_ai/gemini/transformation.py` |
 | `/v1/chat/completions` | OpenAI | `llms/openai/chat/gpt_transformation.py` |
-| `/v1/messages` (passthrough) | Anthropic | `llms/anthropic/messages/native/transformation.py` |
-| `/v1/messages` (passthrough) | Bedrock | `llms/bedrock/messages/invoke_transformations/anthropic_claude_transformation.py` |
-| `/v1/messages` (passthrough) | Vertex AI | `llms/vertex_ai/vertex_ai_partner_models/anthropic/messages/transformation.py` |
+| `/v1/messages` (passthrough) | Anthropic | `llms/anthropic/experimental_pass_through/messages/transformation.py` |
+| `/v1/messages` (passthrough) | Bedrock | `llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py` |
+| `/v1/messages` (passthrough) | Vertex AI | `llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py` |
 | Passthrough endpoints | All | `proxy/pass_through_endpoints/llm_provider_handlers/` |
 
 ### Example: Debugging prompt caching
@@ -350,7 +346,7 @@ Each translation is isolated in its own file, making it easy to test and modify
 If `/v1/messages` → Bedrock Converse prompt caching isn't working but Bedrock Invoke works:
 
 1. **Bedrock Converse translation**: `llms/bedrock/chat/converse_transformation.py`
-2. **Bedrock Invoke translation**: `llms/bedrock/chat/invoke_transformations/anthropic_claude_transformation.py`
+2. **Bedrock Invoke translation**: `llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.py`
 3. Compare how each handles `cache_control` in `transform_request()`
 
 ### How translations work

CONTRIBUTING.md

@@ -8,14 +8,13 @@ Here are the core requirements for any PR submitted to LiteLLM:
 
 - [ ] **Sign the Contributor License Agreement (CLA)** - [see details](#contributor-license-agreement-cla)
 - [ ] **Keep scope isolated** - Your changes should address 1 specific problem at a time
-- [ ] **Prove it works end to end** - If your change affects what the proxy or SDK does, paste the commands you ran against a live proxy and their output in the PR - [see details](#proving-your-change-works-end-to-end)
 
 #### Proxy (Backend) PRs
 
 - [ ] **Add testing** - Adding at least 1 test is a hard requirement - [see details](#adding-testing)
 - [ ] **Ensure your PR passes all checks**:
   - [ ] [Linting / Formatting](#running-linting-and-formatting-checks) - `make lint`
-  - [ ] [The tests covering your change](#running-unit-tests) pass, e.g. `uv run pytest tests/unit/<your_test_file>.py -v`. CI runs the full unit test matrix, so you don't need to run the whole suite locally
+  - [ ] [The tests covering your change](#running-unit-tests) pass, e.g. `uv run pytest tests/test_litellm/<your_test_file>.py -v`. CI runs the full unit test matrix, so you don't need to run the whole suite locally
 
 #### UI PRs
 
@@ -40,12 +39,12 @@ git clone https://github.com/YOUR_USERNAME/litellm.git
 cd litellm
 
 # Create a new branch for your feature (see "Commit and Branch Conventions" below)
-git checkout -b litellm_your_feature
+git checkout -b feature/your-feature
 
 # Install development dependencies
 make install-dev
 
-# Install git hooks that enforce commit conventions (one-time, opt-in)
+# Install git hooks that enforce commit + branch conventions (one-time, opt-in)
 make install-hooks
 
 # Verify your setup works
@@ -56,7 +55,7 @@ That's it! Your local development environment is ready.
 
 ## Commit and Branch Conventions
 
-Commits follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/). Internal contributors name branches with the `litellm_` prefix and no `/`, and fork branches can use any name. Run `make install-hooks` once per clone to enable the local hook that enforces the commit convention. See the [contributor docs](https://docs.litellm.ai/docs/extras/contributing_code#commit-and-branch-conventions) for details
+Commits follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) and branches follow [Conventional Branches](https://conventional-branch.github.io/). Run `make install-hooks` once per clone to enable the local git hooks that enforce these — see the [contributor docs](https://docs.litellm.ai/docs/extras/contributing_code#commit-and-branch-conventions) for the full type list, examples, the protected-branch bypass list, and how to opt out.
 
 ### 2. Development Workflow
 
@@ -73,14 +72,14 @@ make format
 make lint
 
 # Run the tests covering your change (CI runs the full suite)
-uv run pytest tests/unit/<your_test_file>.py -v
+uv run pytest tests/test_litellm/<your_test_file>.py -v
 
 # Commit your changes (must follow Conventional Commits — see above)
 git add .
 git commit -m "feat(scope): your descriptive commit message"
 
-# Push and create a PR
-git push origin litellm_your_feature
+# Push and create a PR (branch must follow Conventional Branches — see above)
+git push origin feature/your-feature
 ```
 
 ## Adding Testing
@@ -89,20 +88,18 @@ git push origin litellm_your_feature
 
 ### Where to Add Tests
 
-Add your tests to the [`tests/unit/` directory](https://github.com/BerriAI/litellm/tree/main/tests/unit).
+Add your tests to the [`tests/test_litellm/` directory](https://github.com/BerriAI/litellm/tree/main/tests/test_litellm).
 
 - This directory mirrors the structure of the `litellm/` directory
 - **Only add mocked tests** - no real LLM API calls in this directory
 - For integration tests with real APIs, use the appropriate test directories
 
-Tests that call `litellm.register_model` or change `litellm.model_cost` should use the `local_model_cost_map` fixture from `tests/unit/conftest.py`. It provides a fresh catalog and restores the previous map, so pricing and capability overrides cannot leak into later tests
-
 ### File Naming Convention
 
-The `tests/unit/` directory follows the same structure as `litellm/`:
+The `tests/test_litellm/` directory follows the same structure as `litellm/`:
 
-- `litellm/proxy/caching_routes.py` → `tests/unit/proxy/test_caching_routes.py`
-- `litellm/utils.py` → `tests/unit/test_utils.py`
+- `litellm/proxy/caching_routes.py` → `tests/test_litellm/proxy/test_caching_routes.py`
+- `litellm/utils.py` → `tests/test_litellm/test_utils.py`
 
 ### Example Test
 
@@ -128,10 +125,10 @@ def test_your_feature():
 
 Run the tests covering your change:
 ```bash
-uv run pytest tests/unit/test_your_file.py -v
+uv run pytest tests/test_litellm/test_your_file.py -v
 ```
 
-`tests/unit` holds thousands of tests, so running all of it locally takes a long time. CI runs it as a parallel matrix (`make test-unit-llms`, `make test-unit-proxy-core`, and the other `test-unit-*` targets) on beefier boxes, so if, for whatever reason, you must run the whole suite, it's better to rely on CI to do that.
+`tests/test_litellm` holds thousands of tests, so running all of it locally takes a long time. CI runs it as a parallel matrix (`make test-unit-llms`, `make test-unit-proxy-core`, and the other `test-unit-*` targets) on beefier boxes, so if, for whatever reason, you must run the whole suite, it's better to rely on CI to do that.
 
 If you're running broader test suites, proxy tests, or anything that touches PostgreSQL-backed fixtures/plugins, install the full local test environment first:
 
@@ -139,7 +136,7 @@ If you're running broader test suites, proxy tests, or anything that touches Pos
 make install-test-deps
 ```
 
-This syncs the locked test environment used across the repo, including `psycopg` v3 plus `psycopg-binary`, `psycopg2-binary` (used by some proxy E2E tests), and a generated Prisma client for DB-backed proxy tests, so pytest startup matches CI without manual package installs.
+This syncs the locked test environment used across the repo, including `psycopg` v3 plus `psycopg-binary` (used by `pytest-postgresql`), `psycopg2-binary` (used by some proxy E2E tests), and a generated Prisma client for DB-backed proxy tests, so pytest startup matches CI without manual package installs.
 
 ### Running Linting and Formatting Checks
 
@@ -249,43 +246,6 @@ If `make test-unit` fails:
 - **Test edge cases**: Don't just test the happy path
 - **Update documentation**: If you change APIs, update docs
 
-## Proving Your Change Works End to End
-
-Tests are required, but they do not prove that your change works for a user. A PR that changes what the proxy or SDK does shows a real run in its "Screenshots / Proof of Fix" section, meaning the exact commands you sent to a live proxy and the output that came back, against real provider APIs and not mocks. `pytest` output does not count as proof, because a passing test only shows that the code does what its mocks expect. A PR that changes only docs or CI shows the rendered page or the CI run instead
-
-A good proof looks like this:
-
-1. Install the proxy dependencies with `make install-proxy-dev`, then start the proxy from your branch with a config that has the model you changed, `uv run litellm --config your_config.yaml --detailed_debug`
-2. Send the request a user would send with `curl`, and paste both the command and the response. When cost or routing is involved, include the response headers, since `x-litellm-response-cost` is what the user sees
-3. For a bug fix, do this twice, once at the merge base with BerriAI/litellm's main (`git merge-base origin/main HEAD`, with your remote for BerriAI/litellm in place of `origin` in a fork clone) to show the bug and once at your branch's latest commit to show the fix. Name both commit hashes
-4. If your change is reachable from more than one endpoint (`/v1/chat/completions`, `/v1/responses`, `/v1/messages`), show each one
-5. If the request can stream, show a `"stream": true` run too, because streaming builds usage and cost on a different path. A stream sends its headers before its usage exists, so show the `usage.cost` on the last chunk, which `/v1/chat/completions` sends when the request has `"stream_options": {"include_usage": true}` and other endpoints send with `include_cost_in_streaming_usage: true` under `litellm_settings`, or, when the proxy has a database, the request's row on the Admin UI's Logs page or `GET /spend/logs`
-6. For UI changes, add before and after screenshots and name the page
-7. If users reach your change through a coding tool such as Claude Code or Codex, drive that tool the way a user does and add before and after screenshots of it, because a `curl` or a headless run alone does not show what the user sees
-
-```bash
-curl -sD - http://localhost:4000/v1/chat/completions \
-  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-  -H "Content-Type: application/json" \
-  -d '{"model": "your-model", "messages": [{"role": "user", "content": "hi"}]}'
-```
-
-When the provider reports its own usage or spend numbers, compare the proxy's numbers against them in the proof
-
-## Contributing an LLM Provider Integration
-
-Provider work (a new provider, a new parameter, a new usage or cost field) is the most common kind of contribution, and it is where PRs get reworked most often. Follow these rules
-
-**Preserve zero prices.** In [`model_prices_and_context_window.json`](model_prices_and_context_window.json), an explicit `0.0` means that the priced operation is free. A missing field or `null` means no rate is specified, so the relevant calculator may use a fallback. For example, when no batch input rate is configured, batch cache writes use `cache_creation_input_token_cost` if set, otherwise the regular input rate, then apply the batch discount. A zero ordinary input rate does not make separately priced cache reads or writes free. Test `rate is None` before substituting a fallback, since `row.get("cache_creation_input_token_cost") or fallback` would discard a valid zero. Speech keeps character pricing ahead of token pricing even when the character rate is zero. If neither rate is configured, speech cost calculation raises an error. Its model lookup preserves unset rates instead of treating compatibility defaults as explicit zero prices. Perplexity keeps a configured legacy search rate ahead of the current search rate, including zero. When registering custom prices, `None` leaves an existing rate unchanged and `0.0` overrides it
-
-**Find the existing machinery before you add plumbing.** LiteLLM already has generic paths that move data between the endpoints, the provider transformations, and cost tracking. A second, hand-rolled path for the same data is a common reason a provider PR gets rewritten before merge. [`ARCHITECTURE.md`](ARCHITECTURE.md) shows where the layers live, and the decision records in [`docs/adrs/`](docs/adrs/AGENTS.md) explain why they are shaped that way. For example, [the ADR on provider usage fields](docs/adrs/adr_001_provider_usage_fields_ride_as_usage_extras.md) covers how provider-specific usage fields reach cost tracking. If you cannot find a mechanism for what you need, say so in the PR or ask in [#pr-review on Slack](https://join.slack.com/t/litellmossslack/shared_invite/zt-3o7nkuyfr-p_kbNJj8taRfXGgQI1~YyA) before you build one
-
-**Bill a new provider usage field from the usage object.** Leave the field on the usage object as an extra and read it in that provider's `cost_calculator.py`. A provider without one gets a new `cost_calculator.py` and an arm for it in `cost_per_token` (`litellm/cost_calculator.py`), which picks the calculator by provider. If the provider's transform builds `Usage` from named fields, as the Anthropic, Gemini, and Bedrock Converse ones do, pass the new field there too. Streamed requests rebuild usage in `ChunkProcessor.calculate_usage` (`litellm/litellm_core_utils/streaming_chunk_builder_utils.py`), which does not copy extras yet the way the Responses-to-chat bridge does (minus a short exclusion list), so until it does, bill the field on streams by naming it there, as Anthropic's `inference_geo` is. A provider that reports its own charge for the request, as OpenRouter does in `usage.cost`, is billed at that charge from `_hidden_params` before any calculator runs, unless the deployment sets its own prices. When the field counts a built-in tool such as web search, add the provider to `get_cost_for_web_search_request` in `litellm/llms/__init__.py`, and add a gate to `litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py` only when no existing gate fires. Then show the field's cost in your proof on every endpoint it reaches, streaming included
-
-**Do not change what a caller sees to make internals easier.** Each endpoint keeps the schema of the API it emulates. A `/v1/responses` caller gets `input_tokens` and a `/v1/chat/completions` caller gets `prompt_tokens`, whatever the provider sent and whatever cost tracking needs. A provider usage field that shows up as an extra key is fine, since it only adds a key, but renaming, removing, or reshaping a key the endpoint defines is not. Reshaping a public response to feed an internal consumer breaks every user of that endpoint
-
-If your change makes an architecture decision, such as a new cross-cutting mechanism or a deliberate difference from a provider's API shape, add an ADR to [`docs/adrs/`](docs/adrs/AGENTS.md) in the same PR
-
 ## Building and Running Locally
 
 ### LiteLLM Proxy Server
@@ -308,13 +268,10 @@ If you want to build the Docker image yourself:
 # Build using the non-root Dockerfile
 docker build -f docker/Dockerfile.non_root -t litellm_dev .
 
-# Generate a master key. Requests send it as the bearer token
-export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
-
 # Run with your config
 docker run \
     -v $(pwd)/proxy_config.yaml:/app/config.yaml \
-    -e LITELLM_MASTER_KEY \
+    -e LITELLM_MASTER_KEY="sk-1234" \
     -p 4000:4000 \
     litellm_dev \
     --config /app/config.yaml --detailed_debug

Dockerfile

@@ -1,10 +1,10 @@
 # syntax=docker/dockerfile:1.7
 
 # Base image for building
-ARG LITELLM_BUILD_IMAGE=cgr.dev/chainguard/wolfi-base@sha256:360896efb0753a5701c48f860e17130c205eae5a264797aa3e7ea126ee2923a7
+ARG LITELLM_BUILD_IMAGE=cgr.dev/chainguard/wolfi-base@sha256:1d95114038f76513a9ace6fca107d5582b08c65981f81f61cb56bf7fd2ef216d
 
 # Runtime image
-ARG LITELLM_RUNTIME_IMAGE=cgr.dev/chainguard/wolfi-base@sha256:360896efb0753a5701c48f860e17130c205eae5a264797aa3e7ea126ee2923a7
+ARG LITELLM_RUNTIME_IMAGE=cgr.dev/chainguard/wolfi-base@sha256:1d95114038f76513a9ace6fca107d5582b08c65981f81f61cb56bf7fd2ef216d
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:0.11.7@sha256:240fb85ab0f263ef12f492d8476aa3a2e4e1e333f7d67fbdd923d00a506a516a
 # Pinned by digest like the other base images; bump explicitly on Node upgrades.
 ARG UI_BUILD_IMAGE=node:24.19-alpine3.24@sha256:d32cdf619f63fe0471182d08996dd516c6275bb5fd31ae06e55a570bd9e1ad43
@@ -39,7 +39,6 @@ ENV NEXT_TELEMETRY_DISABLED=1 \
 WORKDIR /ui
 
 COPY ui/litellm-dashboard/package.json ui/litellm-dashboard/package-lock.json ./
-COPY ui/litellm-dashboard/vendor/ ./vendor/
 RUN --mount=type=cache,target=/root/.npm npm ci --prefer-offline
 
 COPY ui/litellm-dashboard/ ./
@@ -61,7 +60,6 @@ RUN apk add --no-cache \
     python-3.13-dev \
     rust \
     openssl \
-    libssl3 \
     openssl-dev \
     nodejs \
     npm \
@@ -81,7 +79,6 @@ COPY litellm-proxy-extras/pyproject.toml litellm-proxy-extras/
 RUN uv sync --frozen --no-install-project --no-install-workspace --no-default-groups --no-editable \
     --extra proxy \
     --extra proxy-runtime \
-    --group admin-mcp \
     --extra extra_proxy \
     --extra semantic-router \
     --extra saml \
@@ -104,7 +101,6 @@ RUN sed -i 's/\r$//' docker/build_admin_ui.sh && chmod +x docker/build_admin_ui.
 RUN uv sync --frozen --no-default-groups --no-editable \
     --extra proxy \
     --extra proxy-runtime \
-    --group admin-mcp \
     --extra extra_proxy \
     --extra semantic-router \
     --extra saml \
@@ -118,20 +114,8 @@ RUN HOME=/opt/prisma XDG_CACHE_HOME=/opt/prisma/.cache PRISMA_BINARY_CACHE_DIR=/
 RUN sed -i 's/\r$//' docker/entrypoint.sh && chmod +x docker/entrypoint.sh && \
     sed -i 's/\r$//' docker/prod_entrypoint.sh && chmod +x docker/prod_entrypoint.sh
 
-FROM $LITELLM_BUILD_IMAGE AS liteadmin-builder
-COPY --from=uvbin /uv /usr/local/bin/uv
-RUN apk add --no-cache python-3.13
-ADD --checksum=sha256:4fb1847e28e2be30ba76dac94f02c4903291326516a0f4ba772f6ffb8b25d05c \
-    https://github.com/BerriAI/litellm-admin-agent/archive/5661c805ce479a0cf11b842ea133cf8d22cd7e34.tar.gz /tmp/liteadmin.tar.gz
-RUN mkdir /tmp/liteadmin && tar xzf /tmp/liteadmin.tar.gz --strip-components=1 -C /tmp/liteadmin && \
-    uv venv /opt/liteadmin --python python3.13 && \
-    uv pip install --python /opt/liteadmin/bin/python --require-hashes -r /tmp/liteadmin/requirements.txt && \
-    uv pip install --python /opt/liteadmin/bin/python --no-deps /tmp/liteadmin
-
 # Runtime stage
 FROM $LITELLM_RUNTIME_IMAGE AS runtime
-ARG LITELLM_RELEASE_TAG=""
-ENV LITELLM_RELEASE_TAG=${LITELLM_RELEASE_TAG}
 
 USER root
 
@@ -142,7 +126,7 @@ USER root
 RUN echo "https://packages.wolfi.dev/os" >> /etc/apk/repositories
 
 # node (without npm) is required by the prisma CLI at runtime
-RUN apk add --no-cache bash openssl libssl3 tzdata nodejs python-3.13 libsndfile libevent
+RUN apk add --no-cache bash openssl tzdata nodejs python-3.13 libsndfile libevent
 COPY --from=pgbouncer-builder /usr/local/bin/pgbouncer /usr/local/bin/pgbouncer
 
 WORKDIR /app
@@ -157,7 +141,6 @@ ENV PATH="/app/.venv/bin:${PATH}" \
 # ship (manifest-scanning tools attribute everything in it to this image).
 # entrypoint.sh invokes litellm/proxy/prisma_migration.py by source path.
 COPY --from=builder /app/.venv /app/.venv
-COPY --from=liteadmin-builder /opt/liteadmin /opt/liteadmin
 COPY --from=builder /app/docker /app/docker
 COPY --from=builder /app/schema.prisma /app/schema.prisma
 COPY --from=builder /app/litellm/proxy/prisma_migration.py /app/litellm/proxy/prisma_migration.py

Makefile

@@ -1,13 +1,14 @@
 # LiteLLM Makefile
 # Simple Makefile for running tests and basic development tasks
 
-.PHONY: help test test-unit test-unit-llms test-unit-proxy-guardrails test-unit-proxy-core test-unit-proxy-misc test-unit-proxy-root \
+.PHONY: help test test-unit test-unit-llms test-unit-proxy-guardrails test-unit-proxy-core test-unit-proxy-misc \
 	test-unit-integrations test-unit-core-utils test-unit-other test-unit-root \
 	test-proxy-unit-a test-proxy-unit-b test-integration test-unit-helm \
-	test-rust-extension rust-sqlx-prepare \
+	test-rust-extension \
 	info lint lint-inner lint-dev lint-checks format \
-	lint-basedpyright lint-e2e-basedpyright lint-type-discipline \
-	lint-ruff-strict lint-gate lint-test-quality lint-openapi-docs \
+	lint-basedpyright lint-e2e-basedpyright lint-basedpyright-budget-update lint-type-discipline lint-type-discipline-budget-update \
+	lint-ruff-budget lint-ruff-budget-update lint-budget-update lint-gate \
+	lint-test-quality lint-test-quality-budget-update \
 	install-dev install-proxy-dev install-test-deps install-hooks \
 	install-helm-unittest check-circular-imports check-import-safety check check-inner pre-commit \
 	lint-install lint-fetch-base bootstrap
@@ -22,7 +23,7 @@ help:
 	@echo "  make install-proxy-dev-ci - Install proxy dev dependencies (CI-compatible)"
 	@echo "  make install-test-deps  - Install the full local test environment"
 	@echo "  make install-helm-unittest - Install helm unittest plugin"
-	@echo "  make install-hooks      - Install git hooks (Conventional Commits)"
+	@echo "  make install-hooks      - Install git hooks (Conventional Commits + Branches)"
 	@echo "  make check              - Run CI-equivalent lint on staged files, or on the diff vs the base branch when nothing is staged"
 	@echo "  make pre-commit         - Legacy alias for make check"
 	@echo "  make format             - Apply ruff format code formatting"
@@ -30,31 +31,31 @@ help:
 	@echo "  make lint               - Run all linting (Ruff, basedpyright, format check, circular imports, import safety)"
 	@echo "  make lint-ruff          - Run Ruff linting only"
 	@echo "  make lint-basedpyright  - Run basedpyright strict, gated by per-rule error counts"
-	@echo "  make lint-e2e-basedpyright - Run basedpyright over tests/e2e and tests/e2e_harness (zero errors allowed)"
+	@echo "  make lint-e2e-basedpyright - Run basedpyright over tests/e2e (zero errors allowed)"
+	@echo "  make lint-basedpyright-budget-update - Ratchet basedpyright limits down by what this branch fixed"
 	@echo "  make lint-format        - Check ruff format formatting (matches CI)"
-	@echo "  make lint-ruff-strict   - Gate each strict ruff rule's codebase total against its merge-base count"
+	@echo "  make lint-ruff-budget - Gate the codebase total of each strict ruff rule against its limit"
 	@echo "  make lint-gate        - Strict ruff gate in CI-parity mode (fetches the default branch, simulates the merge)"
-	@echo "  make lint-test-quality  - Gate the test suite's TQ counts against their merge-base counts"
-	@echo "  make lint-openapi-docs  - Gate undocumented POST routes in the proxy OpenAPI document against their merge-base counts"
+	@echo "  make lint-ruff-budget-update - Ratchet ruff-strict-budget.json limits down by what this branch fixed"
+	@echo "  make lint-test-quality  - Gate the test suite against test-quality-budget.json"
+	@echo "  make lint-budget-update - Ratchet all budgets down (ruff + type-discipline + test quality + basedpyright)"
 	@echo "  make check-circular-imports - Check for circular imports"
 	@echo "  make check-import-safety - Check import safety"
 	@echo "  make test               - Run all tests"
-	@echo "  make test-unit          - Run unit tests (tests/unit)"
+	@echo "  make test-unit          - Run unit tests (tests/test_litellm)"
 	@echo "  make test-unit-llms     - Run LLM provider tests (~225 files)"
 	@echo "  make test-unit-proxy-guardrails - Run proxy guardrails+mgmt tests (~51 files)"
 	@echo "  make test-unit-proxy-core - Run proxy auth+client+db+hooks tests (~52 files)"
 	@echo "  make test-unit-proxy-misc - Run proxy misc tests (~77 files)"
-	@echo "  make test-unit-proxy-root - Run proxy root-file tests (tests/unit/proxy/test_*.py)"
 	@echo "  make test-unit-integrations - Run integration tests (~60 files)"
 	@echo "  make test-unit-core-utils - Run core utils tests (~32 files)"
 	@echo "  make test-unit-other    - Run other tests (caching, responses, etc., ~69 files)"
 	@echo "  make test-unit-root     - Run root-level tests (~34 files)"
-	@echo "  make test-proxy-unit-a  - Run tests/unit/proxy (a-o)"
-	@echo "  make test-proxy-unit-b  - Run tests/unit/proxy (p-z)"
+	@echo "  make test-proxy-unit-a  - Run proxy_unit_tests (a-o, ~20 files)"
+	@echo "  make test-proxy-unit-b  - Run proxy_unit_tests (p-z, ~28 files)"
 	@echo "  make test-integration   - Run integration tests"
 	@echo "  make test-unit-helm     - Run helm unit tests"
 	@echo "  make test-rust-extension - Build the Rust extension and run its public Python tests"
-	@echo "  make rust-sqlx-prepare  - Refresh litellm-rust/crates/db/.sqlx against a migrated Postgres container"
 	@echo ""
 	@echo "Heavy targets (check, lint) queue for LITELLM_GATE_SLOTS machine-wide"
 	@echo "slots (default 2; 0 disables) so parallel sessions don't thrash one machine."
@@ -120,7 +121,7 @@ install-helm-unittest:
 		helm plugin install https://github.com/helm-unittest/helm-unittest --version v0.8.2; \
 	}
 
-# Install git hooks that enforce Conventional Commits.
+# Install git hooks that enforce Conventional Commits and Conventional Branches.
 # Opt-in: not chained into install-dev.
 install-hooks:
 	./scripts/install_git_hooks.sh
@@ -137,9 +138,9 @@ format-check: install-dev
 lint-fetch-base:
 	@$(RESOLVE_BASE)
 
-# Mirror test-linting.yml's python job environment: the proxy-dev group plus a generated
+# Mirror test-linting.yml's lint job environment: the proxy-dev group plus a generated
 # Prisma client, so `basedpyright tests/e2e` resolves the same modules CI does. The
-# basedpyright gate itself no longer measures here (scripts/type_check_gate.py provisions its
+# budget gate itself no longer measures here (scripts/type_check_gate.py provisions its
 # own .venv-typecheck). --inexact tops up the venv instead of pruning the proxy extras
 # gen:api and the running proxy need.
 lint-install:
@@ -153,7 +154,7 @@ lint-install:
 # the same set CI's ':(glob)litellm/**/*.py' selects.
 lint-format-check-changed: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
 	@base_ref=$$($(RESOLVE_BASE)) && \
-	changed=$$(git diff --name-only --diff-filter=ACMR --merge-base "$$base_ref" -- 'litellm/*.py') && \
+	changed=$$(git diff --name-only --diff-filter=ACMR "$$base_ref...HEAD" -- 'litellm/*.py') && \
 	files=$$(printf '%s\n' "$$changed" | grep -v '^litellm/enterprise/' || true) || exit $$?; \
 	if [ -z "$$files" ]; then \
 		echo "No changed litellm Python files to format-check."; \
@@ -163,7 +164,6 @@ lint-format-check-changed: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
 
 # Linting targets
 lint-ruff: $(LINT_DEP_INSTALL)
-	$(UV_RUN) python scripts/check_mcp_operation_boundary.py
 	cd litellm && $(UV_RUN) ruff check . && cd ..
 	$(UV_RUN) ruff check --config ruff-tests.toml tests
 
@@ -207,28 +207,27 @@ lint-basedpyright: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
 	$(UV_RUN) python scripts/type_check_gate.py --base "$(BASE_REF)"
 
 lint-e2e-basedpyright: $(LINT_E2E_DEP_INSTALL)
-	$(UV_RUN) basedpyright tests/e2e tests/e2e_harness
+	$(UV_RUN) basedpyright tests/e2e
 
-# Type-discipline gate (mutable collections / casts / type guards / kwargs /
+# Type-discipline budget (mutable collections / casts / type guards / kwargs /
 # unexplained suppressions), the test-linting.yml step `make lint` used to omit.
 lint-type-discipline: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
 	$(UV_RUN) python scripts/type_discipline_gate.py --base "$(BASE_REF)"
 
-# Test-quality gate (zero-assert / mock-echo tests, sys.path.insert, raw env writes,
+# Test-quality budget (zero-assert / mock-echo tests, sys.path.insert, raw env writes,
 # litellm module-global mutation, credential-gated skips, conftest snapshot
 # inventory), counted across tests/ the same delta-vs-base way.
 lint-test-quality: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
 	$(UV_RUN) python scripts/test_quality_gate.py --base "$(BASE_REF)"
 
-# OpenAPI docs gate (POST routes without a request body schema, a description or a
-# 200 schema in app.openapi()), counted the same delta-vs-base way so a new endpoint
-# has to show up documented in Swagger UI.
-lint-openapi-docs: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
-	$(UV_RUN) python scripts/openapi_docs_gate.py --base "$(BASE_REF)"
+# --update lowers each limit by what this branch fixed since its branch point, so
+# it needs the base ref fetched to resolve the merge-base.
+lint-basedpyright-budget-update: install-dev
+	$(UV_RUN) python scripts/type_check_gate.py --update --base "$(BASE_REF)"
 
 lint-format: format-check
 
-lint-ruff-strict: install-dev
+lint-ruff-budget: install-dev
 	$(UV_RUN) python scripts/ruff_strict_gate.py --base "$(BASE_REF)"
 
 # Strict gate, invoked the same way CI does in test-linting.yml so a local pass
@@ -236,16 +235,28 @@ lint-ruff-strict: install-dev
 lint-gate: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
 	$(UV_RUN) python scripts/ruff_strict_gate.py --base "$(BASE_REF)"
 
+lint-ruff-budget-update: install-dev
+	$(UV_RUN) python scripts/ruff_strict_gate.py --update --base "$(BASE_REF)"
+
+lint-type-discipline-budget-update: install-dev
+	$(UV_RUN) python scripts/type_discipline_gate.py --update --base "$(BASE_REF)"
+
+lint-test-quality-budget-update: install-dev
+	$(UV_RUN) python scripts/test_quality_gate.py --update --base "$(BASE_REF)"
+
+# Ratchet all budgets in one shot (ruff strict + type-discipline + test quality + basedpyright)
+lint-budget-update: lint-ruff-budget-update lint-type-discipline-budget-update lint-test-quality-budget-update lint-basedpyright-budget-update
+
 check-circular-imports: $(LINT_DEP_INSTALL)
 	cd litellm && $(UV_RUN) python ../tests/documentation_tests/test_circular_imports.py && cd ..
 
 check-import-safety: $(LINT_DEP_INSTALL)
 	@$(UV_RUN) python -c "from litellm import *; print('[from litellm import *] OK! no issues!');" || (echo '🚨 import failed, this means you introduced unprotected imports! 🚨'; exit 1)
 
-# Combined linting, isomorphic to test-linting.yml's python job so a local pass means a
+# Combined linting, isomorphic to test-linting.yml's lint job so a local pass means a
 # green CI lint: it installs the same env (proxy-dev + generated Prisma client) and then
 # runs the diff-scoped ruff format check, whole-tree ruff check, the strict-rule /
-# type-discipline / basedpyright gates as a delta vs the base, then the circular-import
+# type-discipline / basedpyright budgets as a delta vs the base, then the circular-import
 # and import-safety checks. Steps that compare against the base resolve it the same way CI
 # does (merge-base with origin's current default branch). Setup (env sync, Prisma client,
 # base fetch) runs once up front; the checks themselves are independent, so a sub-make
@@ -257,7 +268,7 @@ lint-inner: lint-install
 	@base_ref=$$($(RESOLVE_BASE)) && \
 	$(MAKE) BASE_REF="$$base_ref" -j $(LINT_JOBS) $(LINT_OUTPUT_SYNC) LINT_DEP_INSTALL= LINT_E2E_DEP_INSTALL= LINT_DEP_BASE= lint-checks
 
-lint-checks: lint-format-check-changed lint-ruff lint-gate lint-type-discipline lint-test-quality lint-openapi-docs lint-basedpyright lint-e2e-basedpyright check-circular-imports check-import-safety
+lint-checks: lint-format-check-changed lint-ruff lint-gate lint-type-discipline lint-test-quality lint-basedpyright lint-e2e-basedpyright check-circular-imports check-import-safety
 
 # Faster linting for local development (only checks changed code)
 lint-dev: lint-format-changed check-circular-imports check-import-safety
@@ -266,7 +277,8 @@ lint-dev: lint-format-changed check-circular-imports check-import-safety
 # is staged (warning about changed files left unstaged); with nothing staged it falls
 # back to the working tree's diff against the merge base with the base branch, so a
 # fresh merge commit or an unstaged working tree still gets checked. Mirrors
-# test-linting.yml (Python, UI, and API types), skipping any whose files aren't in scope.
+# test-linting.yml (Python), test-litellm-ui-build.yml's frontend-lint (dashboard), and
+# check-ui-api-types.yml (API-type drift), skipping any whose files aren't in scope.
 # Not auto-installed as a git hook so it never slows an unrelated human commit.
 check:
 	@$(GATE_SLOT_LOCK) $(MAKE) check-inner
@@ -288,57 +300,51 @@ test-rust-extension:
 	UV_PROJECT_ENVIRONMENT="$$temporary/venv" $(UV) sync --python 3.12 --frozen --no-install-project --all-groups --all-extras && \
 	$(UV) pip install --python "$$temporary/venv/bin/python" --no-deps "$$1" && \
 	"$$temporary/venv/bin/python" -I -m mypy.stubtest \
-		--mypy-config-file tests/unit/rust_bridge/stubtest.ini \
+		--mypy-config-file tests/test_litellm/rust_bridge/stubtest.ini \
 		litellm.rust_bridge._native && \
-	LITELLM_LOCAL_MODEL_COST_MAP=True \
+	LITELLM_RUST=1 LITELLM_LOCAL_MODEL_COST_MAP=True \
 	"$$temporary/venv/bin/python" -I -m pytest --import-mode=importlib -m requires_rust_extension tests/test_litellm_rust
 
-rust-sqlx-prepare:
-	cd litellm-rust && cargo run -p litellm-db-testing --bin sqlx-prepare
-
 test: install-test-deps
 	$(UV_RUN) pytest tests/
 
 test-unit: install-test-deps
-	$(UV_RUN) pytest tests/unit -x -vv -n 4
+	$(UV_RUN) pytest tests/test_litellm -x -vv -n 4
 
 # Matrix test targets (matching CI workflow groups)
 test-unit-llms: install-test-deps
-	$(UV_RUN) pytest tests/unit/llms --tb=short -vv -n 4 --durations=20
+	$(UV_RUN) pytest tests/test_litellm/llms --tb=short -vv -n 4 --durations=20
 
 test-unit-proxy-guardrails: install-test-deps
-	$(UV_RUN) pytest tests/unit/proxy/guardrails tests/unit/proxy/management_endpoints tests/unit/proxy/management_helpers --tb=short -vv -n 4 --durations=20
+	$(UV_RUN) pytest tests/test_litellm/proxy/guardrails tests/test_litellm/proxy/management_endpoints tests/test_litellm/proxy/management_helpers --tb=short -vv -n 4 --durations=20
 
 test-unit-proxy-core: install-test-deps
-	$(UV_RUN) pytest tests/unit/proxy/auth tests/unit/proxy/client tests/unit/proxy/db tests/unit/proxy/hooks tests/unit/proxy/policy_engine --ignore=tests/unit/proxy/db/db_transaction_queue/test_e2e_pod_lock_manager.py --ignore=tests/unit/proxy/db/test_update_daily_tag_spend.py --tb=short -vv -n 4 --durations=20
+	$(UV_RUN) pytest tests/test_litellm/proxy/auth tests/test_litellm/proxy/client tests/test_litellm/proxy/db tests/test_litellm/proxy/hooks tests/test_litellm/proxy/policy_engine --tb=short -vv -n 4 --durations=20
 
 test-unit-proxy-misc: install-test-deps
-	$(UV_RUN) pytest tests/unit/proxy/agent_endpoints tests/unit/proxy/anthropic_endpoints tests/unit/proxy/common_utils --ignore=tests/unit/proxy/common_utils/test_cache_aware_routing.py --ignore=tests/unit/proxy/common_utils/test_check_batch_cost.py --ignore=tests/unit/proxy/common_utils/test_check_responses_cost.py --ignore=tests/unit/proxy/common_utils/test_proxy_encrypt_decrypt.py --ignore=tests/unit/proxy/common_utils/test_realtime_cache.py tests/unit/proxy/discovery_endpoints tests/unit/proxy/mcp_server/test_tool_registry.py tests/unit/proxy/google_endpoints tests/unit/proxy/health_endpoints tests/unit/proxy/image_endpoints tests/unit/proxy/middleware --ignore=tests/unit/proxy/middleware/test_request_size_limit_middleware.py tests/unit/proxy/openai_files_endpoint tests/unit/proxy/pass_through_endpoints tests/unit/proxy/prompts tests/unit/proxy/public_endpoints tests/unit/proxy/response_api_endpoints tests/unit/proxy/shutdown tests/unit/proxy/spend_tracking --ignore=tests/unit/proxy/spend_tracking/test_search_api_logging.py tests/unit/proxy/ui_crud_endpoints tests/unit/proxy/vector_store_endpoints tests/unit/proxy/mcp_server/test_mcp_server_tool_calls_and_headers.py --ignore=tests/unit/proxy/google_endpoints/test_gemini_agents_endpoints.py --ignore=tests/unit/proxy/google_endpoints/test_google_endpoint_routing.py --ignore=tests/unit/proxy/google_endpoints/test_google_gemini_proxy_request.py --ignore=tests/unit/proxy/public_endpoints/test_blog_posts_endpoint.py --tb=short -vv -n 4 --durations=20
-
-test-unit-proxy-root: install-test-deps
-	$(UV_RUN) pytest tests/unit/proxy/test_*.py --ignore=tests/unit/proxy/test_aproxy_startup.py --ignore=tests/unit/proxy/test_credential_slot_registry.py --ignore=tests/unit/proxy/test_custom_callback_input.py --ignore=tests/unit/proxy/test_custom_logger_s3_gcs.py --ignore=tests/unit/proxy/test_custom_tokenizer_bug.py --ignore=tests/unit/proxy/test_db_schema_changes.py --ignore=tests/unit/proxy/test_deprecated_key_grace_period.py --ignore=tests/unit/proxy/test_get_favicon.py --ignore=tests/unit/proxy/test_get_image.py --ignore=tests/unit/proxy/test_prisma_client_backoff_retry.py --ignore=tests/unit/proxy/test_prompt_test_endpoint.py --ignore=tests/unit/proxy/test_proxy_config_unit_test.py --ignore=tests/unit/proxy/test_proxy_custom_auth.py --ignore=tests/unit/proxy/test_proxy_reject_logging.py --ignore=tests/unit/proxy/test_proxy_server.py --ignore=tests/unit/proxy/test_proxy_setting_guardrails.py --ignore=tests/unit/proxy/test_proxy_token_counter.py --ignore=tests/unit/proxy/test_proxy_utils.py --ignore=tests/unit/proxy/test_reducto_ocr_route.py --ignore=tests/unit/proxy/test_response_polling_pre_call_checks.py --ignore=tests/unit/proxy/test_server_root_path.py --ignore=tests/unit/proxy/test_ui_path_detection.py --ignore=tests/unit/proxy/test_unit_test_proxy_hooks.py --ignore=tests/unit/proxy/test_update_spend.py --ignore=tests/unit/proxy/test_zero_cost_model_budget_bypass.py --tb=short -vv -n 4 --durations=20
+	$(UV_RUN) pytest tests/test_litellm/proxy/_experimental tests/test_litellm/proxy/agent_endpoints tests/test_litellm/proxy/anthropic_endpoints tests/test_litellm/proxy/common_utils tests/test_litellm/proxy/discovery_endpoints tests/test_litellm/proxy/experimental tests/test_litellm/proxy/google_endpoints tests/test_litellm/proxy/health_endpoints tests/test_litellm/proxy/image_endpoints tests/test_litellm/proxy/middleware tests/test_litellm/proxy/openai_files_endpoint tests/test_litellm/proxy/pass_through_endpoints tests/test_litellm/proxy/prompts tests/test_litellm/proxy/public_endpoints tests/test_litellm/proxy/response_api_endpoints tests/test_litellm/proxy/shutdown tests/test_litellm/proxy/spend_tracking tests/test_litellm/proxy/ui_crud_endpoints tests/test_litellm/proxy/vector_store_endpoints tests/test_litellm/proxy/test_*.py --tb=short -vv -n 4 --durations=20
 
 test-unit-integrations: install-test-deps
-	$(UV_RUN) pytest tests/unit/integrations --tb=short -vv -n 4 --durations=20
+	$(UV_RUN) pytest tests/test_litellm/integrations --tb=short -vv -n 4 --durations=20
 
 test-unit-core-utils: install-test-deps
-	$(UV_RUN) pytest tests/unit/litellm_core_utils --tb=short -vv -n 2 --durations=20
+	$(UV_RUN) pytest tests/test_litellm/litellm_core_utils --tb=short -vv -n 2 --durations=20
 
 test-unit-other: install-test-deps
-	$(UV_RUN) pytest tests/unit/caching tests/unit/responses tests/unit/secret_managers tests/unit/vector_stores tests/unit/a2a_protocol tests/unit/completion_extras tests/unit/containers tests/unit/enterprise tests/unit/mcp_client tests/unit/google_genai tests/unit/images tests/unit/interactions tests/unit/router_strategy tests/unit/router_utils tests/unit/types --tb=short -vv -n 4 --durations=20
+	$(UV_RUN) pytest tests/test_litellm/caching tests/test_litellm/responses tests/test_litellm/secret_managers tests/test_litellm/vector_stores tests/test_litellm/a2a_protocol tests/test_litellm/anthropic_interface tests/test_litellm/completion_extras tests/test_litellm/containers tests/test_litellm/enterprise tests/test_litellm/experimental_mcp_client tests/test_litellm/google_genai tests/test_litellm/images tests/test_litellm/interactions tests/test_litellm/passthrough tests/test_litellm/router_strategy tests/test_litellm/router_utils tests/test_litellm/types --tb=short -vv -n 4 --durations=20
 
 test-unit-root: install-test-deps
-	$(UV_RUN) pytest tests/unit/test_*.py --tb=short -vv -n 4 --durations=20
+	$(UV_RUN) pytest tests/test_litellm/test_*.py --tb=short -vv -n 4 --durations=20
 
-# Proxy unit tests (tests/unit/proxy split alphabetically)
+# Proxy unit tests (tests/proxy_unit_tests split alphabetically)
 test-proxy-unit-a: install-test-deps
-	$(UV_RUN) pytest tests/unit/proxy --ignore-glob='tests/unit/proxy/test_[p-z]*.py' --tb=short -vv -n 2 --durations=20
+	$(UV_RUN) pytest tests/proxy_unit_tests/test_[a-o]*.py --tb=short -vv -n 2 --durations=20
 
 test-proxy-unit-b: install-test-deps
-	$(UV_RUN) pytest tests/unit/proxy/test_[p-z]*.py tests/unit/skills --tb=short -vv -n 2 --durations=20
+	$(UV_RUN) pytest tests/proxy_unit_tests/test_[p-z]*.py --tb=short -vv -n 2 --durations=20
 
 test-integration: install-test-deps
-	$(UV_RUN) pytest tests/
+	$(UV_RUN) pytest tests/ -k "not test_litellm"
 
 test-unit-helm: install-helm-unittest
 	helm unittest -f 'tests/*.yaml' helm/litellm-helm

README.md

@@ -9,7 +9,7 @@
         <a href="https://render.com/deploy?repo=https://github.com/BerriAI/litellm" target="_blank" rel="nofollow"><img src="https://render.com/images/deploy-to-render-button.svg" alt="Deploy to Render" height="40"></a>
         <a href="https://railway.com/deploy/RhvhdC?referralCode=7mRv9K&utm_medium=integration&utm_source=template&utm_campaign=generic"><img src="https://railway.com/button.svg" alt="Deploy on Railway" height="40"></a>
         <a href="https://console.aws.amazon.com/cloudshell/home" target="_blank" rel="nofollow"><img src="./.github/deploy-on-aws.png" alt="Deploy on AWS" height="40"></a>
-        <a href="https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FBerriAI%2Fterraform-google-litellm&cloudshell_git_branch=v1.105.0&cloudshell_workspace=examples%2Fdefault&cloudshell_tutorial=TUTORIAL.md&cloudshell_image=gcr.io/ds-artifacts-cloudshell/deploystack_custom_image&shellonly=true" target="_blank" rel="nofollow"><img src="./.github/deploy-on-gcp.png" alt="Deploy on GCP" height="40"></a>
+        <a href="https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FBerriAI%2Flitellm&cloudshell_workspace=terraform%2Flitellm%2Fgcp%2Fexamples%2Fdefault&cloudshell_tutorial=TUTORIAL.md&cloudshell_image=gcr.io/ds-artifacts-cloudshell/deploystack_custom_image&shellonly=true" target="_blank" rel="nofollow"><img src="./.github/deploy-on-gcp.png" alt="Deploy on GCP" height="40"></a>
         </p>
     </p>
 <h4 align="center"><a href="https://docs.litellm.ai/docs/simple_proxy" target="_blank">LiteLLM Proxy Server (AI Gateway)</a> | <a href="https://docs.litellm.ai/docs/enterprise#hosted-litellm-proxy" target="_blank"> Hosted Proxy</a> | <a href="https://litellm.ai/enterprise"target="_blank">Enterprise Tier</a> | <a href="https://www.litellm.ai/ai-gateway" target="_blank">Website</a></h4>
@@ -90,24 +90,6 @@ Managing LLM calls across providers gets complicated fast — different SDKs, au
 uv add litellm
 ```
 
-An independent `litellm-core` distribution provides the Python SDK with the same
-`import litellm` API and runtime dependencies. It has no optional extras, CLI entry
-points, or bundled dashboard. Install one SDK distribution per environment because
-`litellm` and `litellm-core` own overlapping Python files. Use `litellm` for the
-proxy, CLI, and optional extras
-
-Build and install core from this checkout while its release integration is pending:
-
-```shell
-python scripts/build_core_distribution.py --out-dir dist/core
-python -m pip install dist/core/litellm_core-*.whl
-```
-
-The builder requires Git, `uv` and the Rust build toolchain. It reads the release version
-from the root `pyproject.toml`, leaves source files unchanged, and produces a wheel
-and a self-contained sdist in `dist/core`. Run installation in a fresh environment
-without `litellm`
-
 ```python
 from litellm import completion
 import os
@@ -141,56 +123,12 @@ response = client.chat.completions.create(
 )
 ```
 
-</details>
-
-<details>
-<summary><b>Auto Routing</b> - Choose a model for each prompt (Python SDK + AI Gateway)</summary>
-
-### Python SDK
-
-[**Docs: Autorouter SDK**](https://docs.litellm.ai/docs/autorouter_sdk)
-
-Install with `uv add litellm` and set `OPENAI_API_KEY` for the models below.
-
-```python
-import asyncio
-from litellm import Router
-
-router = Router(model_list=[
-    {"model_name": "fast", "litellm_params": {"model": "openai/gpt-6-luna"}},
-    {"model_name": "capable", "litellm_params": {"model": "openai/gpt-6.1-sol"}},
-    {"model_name": "auto", "litellm_params": {
-        "model": "auto_router/complexity_router",
-        "complexity_router_config": {
-            "classifier_type": "heuristic",
-            "tiers": {"SIMPLE": "fast", "MEDIUM": "fast",
-                      "COMPLEX": "capable", "REASONING": "capable"},
-            "return_raw_model_name": True,
-        },
-        "complexity_router_default_model": "capable",
-    }},
-])
-response = asyncio.run(router.acompletion(
-    model="auto", messages=[{"role": "user", "content": "What is 2+2?"}],
-))
-print(response.model, response.choices[0].message.content)
-```
-
-### AI Gateway (Proxy Server)
-
-[**Create an Auto Router**](https://docs.litellm.ai/docs/auto_router/setup), then call its model name. Replace `auto` and the URL below with yours, and set `LITELLM_API_KEY` to your virtual key.
-
-```shell
-curl http://localhost:4000/v1/chat/completions \
-  -H "Authorization: Bearer $LITELLM_API_KEY" \
-  -H "Content-Type: application/json" \
-  -d '{"model":"auto","messages":[{"role":"user","content":"What is 2+2?"}]}'
-```
+[**Docs: LLM Providers**](https://docs.litellm.ai/docs/providers)
 
 </details>
 
 <details>
-<summary><b>A2A</b> - Invoke A2A Agents (Python SDK + AI Gateway)</summary>
+<summary><b>Agents</b> - Invoke A2A Agents (Python SDK + AI Gateway)</summary>
 
 [**Supported Providers**](https://docs.litellm.ai/docs/a2a#add-a2a-agents) - LangGraph, Vertex AI Agent Engine, Azure AI Foundry, Bedrock AgentCore, Pydantic AI
 
@@ -230,7 +168,7 @@ from a2a.utils.constants import TransportProtocol
 from uuid import uuid4
 
 base_url = "http://localhost:4000/a2a/my-agent"  # LiteLLM proxy + agent name
-headers = {"Authorization": "Bearer <your-master-key>"}    # LiteLLM master key or a virtual key
+headers = {"Authorization": "Bearer sk-1234"}    # LiteLLM Virtual Key
 
 async with httpx.AsyncClient(headers=headers, timeout=60.0) as http_client:
     resolver = A2ACardResolver(httpx_client=http_client, base_url=base_url)
@@ -267,7 +205,7 @@ async with httpx.AsyncClient(headers=headers, timeout=60.0) as http_client:
 ```python
 from mcp import ClientSession, StdioServerParameters
 from mcp.client.stdio import stdio_client
-from litellm import mcp_client
+from litellm import experimental_mcp_client
 import litellm
 
 server_params = StdioServerParameters(command="python", args=["mcp_server.py"])
@@ -277,7 +215,7 @@ async with stdio_client(server_params) as (read, write):
         await session.initialize()
 
         # Load MCP tools in OpenAI format
-        tools = await mcp_client.load_mcp_tools(session=session, format="openai")
+        tools = await experimental_mcp_client.load_mcp_tools(session=session, format="openai")
 
         # Use with any LiteLLM model
         response = await litellm.acompletion(
@@ -295,7 +233,7 @@ async with stdio_client(server_params) as (read, write):
 
 ```bash
 curl -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-  -H 'Authorization: Bearer <your-master-key>' \
+  -H 'Authorization: Bearer sk-1234' \
   -H 'Content-Type: application/json' \
   -d '{
     "model": "gpt-4o",
@@ -317,7 +255,7 @@ curl -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
     "LiteLLM": {
       "url": "http://localhost:4000/mcp/",
       "headers": {
-        "x-litellm-api-key": "Bearer <your-master-key>"
+        "x-litellm-api-key": "Bearer sk-1234"
       }
     }
   }
@@ -330,31 +268,6 @@ For MCP OAuth, an upstream may advertise dynamic client registration but refuse
 
 </details>
 
-<details>
-<summary><b>Agents</b> - Run Claude Code, Codex, OpenCode, pi or Deep Agents on any model (Python SDK)</summary>
-
-### Python SDK - Agents
-
-```python
-import litellm
-from litellm import Harness, sandbox
-
-result = litellm.agent(
-    Harness.CLAUDE_CODE,  # or Harness.CODEX, Harness.OPENCODE, Harness.PI, Harness.DEEPAGENTS
-    "Find why tests/test_router.py is flaky and fix it.",
-    sandbox=sandbox.local("./repo"),
-    model="litellm_proxy/claude-sonnet-4-5",  # a model group on your AI Gateway
-)
-
-print(result.text, result.cost, [f.path for f in result.files])
-```
-
-Set `LITELLM_PROXY_API_BASE` and `LITELLM_PROXY_API_KEY` and every model call the agent makes goes through your AI Gateway, tagged `harness,claude_code`. Drop the `litellm_proxy/` prefix to call a provider directly. Install `starlette uvicorn` plus the agent's CLI (`claude`, `codex`, `opencode` or `pi`), or `deepagents langchain-litellm` for Deep Agents.
-
-[**Docs: Agent Harnesses**](https://docs.litellm.ai/docs/harness)
-
-</details>
-
 ### Supported Providers ([Website Supported Models](https://models.litellm.ai/) | [Docs](https://docs.litellm.ai/docs/providers))
 
 | Provider                                                                            | `/chat/completions` | `/messages` | `/responses` | `/embeddings` | `/image/generations` | `/audio/transcriptions` | `/audio/speech` | `/moderations` | `/batches` | `/rerank` |
@@ -379,7 +292,6 @@ Set `LITELLM_PROXY_API_BASE` and `LITELLM_PROXY_API_KEY` and every model call th
 | [Bytez (`bytez`)](https://docs.litellm.ai/docs/providers/bytez) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Cerebras (`cerebras`)](https://docs.litellm.ai/docs/providers/cerebras) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Clarifai (`clarifai`)](https://docs.litellm.ai/docs/providers/clarifai) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [CLF AI Gateway (`clf_ai_gateway`)](https://docs.litellm.ai/docs/providers/clf_ai_gateway) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Cloudflare AI Workers (`cloudflare`)](https://docs.litellm.ai/docs/providers/cloudflare_workers) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Codestral (`codestral`)](https://docs.litellm.ai/docs/providers/codestral) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Cognition (`cognition`)](https://docs.litellm.ai/docs/providers/cognition) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
@@ -387,7 +299,6 @@ Set `LITELLM_PROXY_API_BASE` and `LITELLM_PROXY_API_KEY` and every model call th
 | [Cohere Chat (`cohere_chat`)](https://docs.litellm.ai/docs/providers/cohere) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [CometAPI (`cometapi`)](https://docs.litellm.ai/docs/providers/cometapi) | ✅ | ✅ | ✅ | ✅ |  |  |  |  |  |  |
 | [CompactifAI (`compactifai`)](https://docs.litellm.ai/docs/providers/compactifai) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [CoralBricks (`coralbricks`)](https://docs.litellm.ai/docs/providers/coralbricks) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Custom (`custom`)](https://docs.litellm.ai/docs/providers/custom_llm_server) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Custom OpenAI (`custom_openai`)](https://docs.litellm.ai/docs/providers/openai_compatible) | ✅ | ✅ | ✅ |  |  | ✅ | ✅ | ✅ | ✅ |  |
 | [Dashscope (`dashscope`)](https://docs.litellm.ai/docs/providers/dashscope) | ✅ | ✅ | ✅ | ✅ |  |  |  |  |  | ✅ |
@@ -396,10 +307,8 @@ Set `LITELLM_PROXY_API_BASE` and `LITELLM_PROXY_API_KEY` and every model call th
 | [Deepgram (`deepgram`)](https://docs.litellm.ai/docs/providers/deepgram) | ✅ | ✅ | ✅ |  |  | ✅ |  |  |  |  |
 | [DeepInfra (`deepinfra`)](https://docs.litellm.ai/docs/providers/deepinfra) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Deepseek (`deepseek`)](https://docs.litellm.ai/docs/providers/deepseek) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [Eden AI (`edenai`)](https://docs.litellm.ai/docs/providers/edenai) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |  |  |  |
 | [ElevenLabs (`elevenlabs`)](https://docs.litellm.ai/docs/providers/elevenlabs) | ✅ | ✅ | ✅ |  |  | ✅ | ✅ |  |  |  |
 | [Empower (`empower`)](https://docs.litellm.ai/docs/providers/empower) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [eu/jev Decisions API (`eu_jev`)](https://docs.litellm.ai/docs/pass_through/eu_jev) |  |  |  |  |  |  |  |  |  |  |
 | [Fal AI (`fal_ai`)](https://docs.litellm.ai/docs/providers/fal_ai) | ✅ | ✅ | ✅ |  | ✅ |  |  |  |  |  |
 | [Featherless AI (`featherless_ai`)](https://docs.litellm.ai/docs/providers/featherless_ai) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Fireworks AI (`fireworks_ai`)](https://docs.litellm.ai/docs/providers/fireworks_ai) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
@@ -430,7 +339,6 @@ Set `LITELLM_PROXY_API_BASE` and `LITELLM_PROXY_API_KEY` and every model call th
 | [ModelScope (`modelscope`)](https://docs.litellm.ai/docs/providers/modelscope) | ✅ | ✅ | ✅ |  | ✅ |  |  |  |  |  |
 | [Moonshot (`moonshot`)](https://docs.litellm.ai/docs/providers/moonshot) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Morph (`morph`)](https://docs.litellm.ai/docs/providers/morph) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [Nace (`nace`)](https://console.nace.ai/docs/drex) |  |  |  |  |  |  |  |  |  |  |
 | [Nebius AI Studio (`nebius`)](https://docs.litellm.ai/docs/providers/nebius) | ✅ | ✅ | ✅ | ✅ |  |  |  |  |  |  |
 | [NLP Cloud (`nlp_cloud`)](https://docs.litellm.ai/docs/providers/nlp_cloud) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Novita AI (`novita`)](https://novita.ai/models/llm?utm_source=github_litellm&utm_medium=github_readme&utm_campaign=github_link) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
@@ -444,37 +352,31 @@ Set `LITELLM_PROXY_API_BASE` and `LITELLM_PROXY_API_KEY` and every model call th
 | [OpenAI-like (`openai_like`)](https://docs.litellm.ai/docs/providers/openai_compatible) |  |  |  | ✅ |  |  |  |  |  |  |
 | [OpenRouter (`openrouter`)](https://docs.litellm.ai/docs/providers/openrouter) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [OVHCloud AI Endpoints (`ovhcloud`)](https://docs.litellm.ai/docs/providers/ovhcloud) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [Parallel AI (`parallel_ai`)](https://docs.litellm.ai/docs/search/parallel_ai) |  |  |  |  |  |  |  |  |  |  |
 | [Perplexity AI (`perplexity`)](https://docs.litellm.ai/docs/providers/perplexity) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Petals (`petals`)](https://docs.litellm.ai/docs/providers/petals) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Pinstripes (`pinstripes`)](https://docs.litellm.ai/docs/providers/pinstripes) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Predibase (`predibase`)](https://docs.litellm.ai/docs/providers/predibase) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [Qianwen AI Platform (`qwen_ai_platform`)](https://docs.litellm.ai/docs/providers/qwencloud) | ✅ | ✅ | ✅ | ✅ | ✅ |  |  |  |  | ✅ |
+| [Qwen AI Platform (`qwen_ai_platform`)](https://docs.litellm.ai/docs/providers/qwencloud) | ✅ | ✅ | ✅ | ✅ | ✅ |  |  |  |  | ✅ |
 | [QwenCloud (`qwencloud`)](https://docs.litellm.ai/docs/providers/qwencloud) | ✅ | ✅ | ✅ | ✅ | ✅ |  |  |  |  | ✅ |
 | [Recraft (`recraft`)](https://docs.litellm.ai/docs/providers/recraft) |  |  |  |  | ✅ |  |  |  |  |  |
 | [Replicate (`replicate`)](https://docs.litellm.ai/docs/providers/replicate) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Sagemaker Chat (`sagemaker_chat`)](https://docs.litellm.ai/docs/providers/aws_sagemaker) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [Sail (`sail`)](https://docs.litellm.ai/docs/providers/sail) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Sambanova (`sambanova`)](https://docs.litellm.ai/docs/providers/sambanova) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [ScaleDown (`scaledown`)](https://docs.litellm.ai/docs/providers/scaledown) | ✅ |  |  |  |  |  |  |  |  |  |
 | [Snowflake (`snowflake`)](https://docs.litellm.ai/docs/providers/snowflake) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [Strands Decider (`strands_decider`)](https://docs.litellm.ai/docs/providers) |  |  |  |  |  |  |  |  |  |  |
 | [Text Completion Codestral (`text-completion-codestral`)](https://docs.litellm.ai/docs/providers/codestral) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Text Completion OpenAI (`text-completion-openai`)](https://docs.litellm.ai/docs/providers/text_completion_openai) | ✅ | ✅ | ✅ |  |  | ✅ | ✅ | ✅ | ✅ |  |
 | [Together AI (`together_ai`)](https://docs.litellm.ai/docs/providers/togetherai) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Topaz (`topaz`)](https://docs.litellm.ai/docs/providers/topaz) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Triton (`triton`)](https://docs.litellm.ai/docs/providers/triton-inference-server) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [Typesafe Decisions API (`typesafe`)](https://docs.litellm.ai/docs/providers) |  |  |  |  |  |  |  |  |  |  |
 | [V0 (`v0`)](https://docs.litellm.ai/docs/providers/v0) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Vercel AI Gateway (`vercel_ai_gateway`)](https://docs.litellm.ai/docs/providers/vercel_ai_gateway) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [VLLM (`vllm`)](https://docs.litellm.ai/docs/providers/vllm) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Volcengine (`volcengine`)](https://docs.litellm.ai/docs/providers/volcano) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
-| [VoyageAI by MongoDB (`voyage`)](https://docs.litellm.ai/docs/providers/voyage) |  |  |  | ✅ |  |  |  |  |  |  |
+| [Voyage AI (`voyage`)](https://docs.litellm.ai/docs/providers/voyage) |  |  |  | ✅ |  |  |  |  |  |  |
 | [WandB Inference (`wandb`)](https://docs.litellm.ai/docs/providers/wandb_inference) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Watsonx Text (`watsonx_text`)](https://docs.litellm.ai/docs/providers/watsonx) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [xAI (`xai`)](https://docs.litellm.ai/docs/providers/xai) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Xinference (`xinference`)](https://docs.litellm.ai/docs/providers/xinference) |  |  |  | ✅ |  |  |  |  |  |  |
-| [ZeroGPU (`zerogpu`)](https://docs.litellm.ai/docs/providers/zerogpu) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 
 [**Read the Docs**](https://docs.litellm.ai/docs/)
 
@@ -489,7 +391,7 @@ You can use LiteLLM through either the Proxy Server or Python SDK. Both give you
 <tr>
 <th style={{width: '14%'}}></th>
 <th style={{width: '43%'}}><strong><a href="https://docs.litellm.ai/docs/simple_proxy">LiteLLM AI Gateway</a></strong></th>
-<th style={{width: '43%'}}><strong><a href="https://docs.litellm.ai/docs/python_sdk">LiteLLM Python SDK</a></strong></th>
+<th style={{width: '43%'}}><strong><a href="https://docs.litellm.ai/docs/">LiteLLM Python SDK</a></strong></th>
 </tr>
 </thead>
 <tbody>
@@ -576,9 +478,9 @@ Provider API keys live in AWS Secrets Manager; reference ARNs via `gateway_extra
 
 #### GCP — Cloud Run + Cloud SQL + Memorystore + HTTPS LB
 
-[![Open in Cloud Shell](https://gstatic.com/cloudssh/images/open-btn.png)](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FBerriAI%2Fterraform-google-litellm&cloudshell_git_branch=v1.105.0&cloudshell_workspace=examples%2Fdefault&cloudshell_tutorial=TUTORIAL.md&cloudshell_image=gcr.io/ds-artifacts-cloudshell/deploystack_custom_image&shellonly=true)
+[![Open in Cloud Shell](https://gstatic.com/cloudssh/images/open-btn.png)](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FBerriAI%2Flitellm&cloudshell_workspace=terraform%2Flitellm%2Fgcp%2Fexamples%2Fdefault&cloudshell_tutorial=TUTORIAL.md&cloudshell_image=gcr.io/ds-artifacts-cloudshell/deploystack_custom_image&shellonly=true)
 
-Real 1-click. Opens Cloud Shell, clones the [terraform-google-litellm](https://github.com/BerriAI/terraform-google-litellm) module mirror at the latest stable release (a small download instead of the full litellm repo), and walks you through `terraform apply` via a built-in [DeployStack tutorial](./terraform/litellm/gcp/examples/default/TUTORIAL.md). Pick the project, the tutorial sets up the Artifact Registry remote repo, writes `terraform.tfvars` from your answers, and runs apply
+Real 1-click. Opens Cloud Shell, clones this repo, and walks you through `terraform apply` via a built-in [DeployStack tutorial](./terraform/litellm/gcp/examples/default/TUTORIAL.md) — pick the project, the tutorial sets up the Artifact Registry remote repo, writes `terraform.tfvars` from your answers, and runs apply.
 
 [Module page →](https://registry.terraform.io/modules/BerriAI/litellm/google/latest)
 
@@ -664,15 +566,15 @@ The Terraform modules live at [`terraform/litellm/aws/`](./terraform/litellm/aws
 
 ### Verify Docker Image Signatures
 
-All LiteLLM Docker images published to GHCR are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Releases are signed with the key introduced in [commit `e82a3c6`](https://github.com/BerriAI/litellm/commit/e82a3c646908ba0a6ee0beddc4310d7a6cac4065).
+All LiteLLM Docker images published to GHCR are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).
 
 **Verify using the pinned commit hash (recommended):**
 
-A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the published signing key:
+A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:
 
 ```bash
 cosign verify \
-  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
+  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
   ghcr.io/berriai/litellm:<release-tag>
 ```
 
@@ -697,7 +599,7 @@ For companies that need better security, user management and professional suppor
 [Talk to founders](https://enterprise.litellm.ai/demo)
 
 This covers:
-- ✅ **Features under the [LiteLLM Commercial License](https://docs.litellm.ai/docs/enterprise):**
+- ✅ **Features under the [LiteLLM Commercial License](https://docs.litellm.ai/docs/proxy/enterprise):**
 - ✅ **Feature Prioritization**
 - ✅ **Custom Integrations**
 - ✅ **Professional Support - Dedicated discord + slack**
@@ -708,8 +610,6 @@ This covers:
 
 We welcome contributions to LiteLLM! Whether you're fixing bugs, adding features, or improving documentation, we appreciate your help.
 
-If you want to work closely with our team, check out [LiteLLM OSS Champions](https://www.litellm.ai/champions), our program for trusted OSS contributors with their own Slack channel and a voice in the roadmap
-
 ## Quick Start for Contributors
 
 This requires uv to be installed.