Commit graph

18 commits

Author SHA1 Message Date
yuneng-jiang
6c30b4331d
ci: measure enterprise/ coverage (#37788)
codecov.yaml has carried an `Enterprise` component scoped to `enterprise/**`
since it was written, and it has never received a line of data. Every one of
the 19 coverage invocations across the unit base, the MCP workflow and the
CircleCI config passes `--cov=./litellm` and nothing else, so 11,203 lines of
paid-customer code sat outside the measured universe while the reported number
described only the rest.

litellm-enterprise is a uv workspace member and a direct dependency, so every
job that syncs already has it installed and importable; only the measurement
was missing.

Measured on tests/test_litellm/enterprise, the shard that exercises this code:
0 enterprise files in the report before, 142 after, at `enterprise/...` paths
that match the component's existing glob. That shard alone puts enterprise at
30.8%, which nudged its own total from 24.09% to 24.20% rather than down. The
aggregate direction across every shard is not knowable until they all report,
and a drop there is the instrument working, not a regression.
2026-08-22 22:55:01 -07:00
yuneng-jiang
afec9b8ab9
perf(ci): cache the Rust build the unit shards compile from scratch (#37795)
* perf(ci): cache the Rust build the unit shards compile from scratch

Every unit shard installs the workspace, and the root package builds through
maturin, so each of the eleven jobs compiles litellm-rust/crates/python-bridge
in release mode before a single test runs. That step measured 2m40s a shard on
2026-08-21, which is more wall clock than the entire unit tier spends running
tests, and none of it was cached: the uv cache covers wheels it downloads, not
wheels it builds, and a path dependency whose source moves every commit can
never hit that cache anyway.

A composite action now exports CARGO_TARGET_DIR to a fixed workspace path and
caches it alongside the Cargo registry, keyed on Cargo.lock. Cargo rebuilds only
what changed, so a warm job pays for the bridge crate rather than its whole
dependency graph. Measured locally, that is 34s cold against 8s warm, including
after a Python-only or Rust-only edit.

The absolute path matters: uv builds the wheel from its own working directory,
so a relative target directory lands the artifacts where nothing can find them
again.

* perf(ci): cache the Rust build in the other four workflows that sync the workspace

code-quality, mcp, documentation and the schema.d.ts check each install the
workspace and so each compile the bridge from scratch, measured at 138s, 177s,
163s and 154s on 2026-08-21. The lint job pays the same and is left to #37783,
which already owns that file's setup section.

* perf(ci): cache the Rust build in the lint job too

* fix(ci): cache cargo's own target directory instead of redirecting it

uv builds the wheel in place, so cargo already writes to litellm-rust/target,
which test-rust.yml has cached all along. Redirecting CARGO_TARGET_DIR bought
nothing and cost a GITHUB_ENV write that zizmor rejects as a code-execution
path.

* chore(ci): raise the job backstop for the added setup step

The cargo cache is a fifth bounded setup step, so the base's setup ceiling goes
30m to 35m and every job budget follows: 55 to 60, and proxy-server's 95 to 100.
check_workflow_startup_safety enforces exactly this sum, and failed on the first
push without it.

* perf(ci): cache the Rust build in the four remaining workflows that sync

Six workflows were wired; ten install the workspace. The four left out still
compile the pyo3 bridge from scratch.

test-terraform-provider.yml is the one that matters per PR: its endpoint-drift
job triggers on any change under litellm/proxy/**. The other three are a
scheduled load check, a manual mutation run, and the staging-push counts
publisher, whose gate syncs the project inside scripts/type_check_gate.py
rather than in a workflow step, so nothing in the file names the build it pays
for.
2026-08-21 20:15:08 -07:00
yuneng-jiang
7b6e16cfd3
perf(ci): gate the lint, MCP and dashboard jobs on the pull request's file list (#37559)
PR #37550 taught the backend unit-test shards to read the pull request's own
file list, but four required jobs were never wired to that gate and ran in full
on every pull request regardless of what it touched. A UI-only pull request
still paid roughly 17 runner-minutes of Python work it could not have affected,
and a backend-only one still installed and built the dashboard.

Lint and the MCP suite now take the existing backend decision. The dashboard
build and unit tests take a new ui decision, which tracks ui/ rather than
reusing client: client deliberately runs whenever the backend changes, because
it gates CircleCI's end-to-end jobs that drive a real proxy, while the build and
the unit tests cannot see the backend at all. CI config counts as ui-relevant
too, so a pull request that rewrites the dashboard workflows still exercises
them instead of shipping unvalidated.

The gate stays inside the job rather than moving to on.paths or to a job-level
condition on the shard callers. A workflow filtered out by on.paths never starts
and never reports, so a required check waits forever, and a skipped caller job
publishes its own name instead of the nested "<shard> / Run tests" the ruleset
requires. Both were measured before settling on this shape.

Three setup steps in the shard base and in the documentation job also leaked
past the gate, so a skipped shard still spent about twelve seconds installing uv
and restoring its cache. They now carry the same condition, and the documentation
job stops cloning litellm-docs when it has nothing to validate.
2026-08-19 18:32:21 -07:00
mateo
ede84eee15 ci: give the remaining pull_request workflows a concurrency group
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-08 03:25:38 +00:00
mateo-berri
bb1b3dc937 fix(ci): retry setup-uv installs to survive transient manifest fetch failures 2026-07-14 14:41:36 -07:00
ishaan-berri
a2d04ccdbb
ci: harden cargo fetches during maturin builds (#31348) 2026-06-25 14:31:05 -07:00
Sameer Kankute
c712c20d0f
fix(ci): point OSS contributor workflows to litellm_oss_staging (#31270)
* fix(ci): point OSS contributor workflows to litellm_oss_staging

Workflow triggers and guard error messages incorrectly referenced litellm_oss_branch; update them to the branch we actually use for external contributions.

* fix(ci): include test-rust.yml in litellm_oss_staging rename

Missed test-rust.yml when updating OSS contributor target branch references.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-24 21:07:59 -07:00
Yuneng Jiang
538092a55f
ci: use --cov=./litellm so coverage paths resolve unambiguously in Codecov
pytest-cov treats --cov=<module-name> as a Python package and emits XML
paths relative to the package root, stripping the litellm/ prefix
(`proxy/proxy_server.py` instead of `litellm/proxy/proxy_server.py`).
Codecov's auto-prefix heuristic then drops every file whose basename is
ambiguous in the repo — `proxy_server.py` (3 copies under enterprise/),
`router.py` (2 copies), `utils.py` (20+), `main.py` (20+), `constants.py`
(2). The 11 highest-fix-rate hotspots have never appeared in Codecov.

Switching to --cov=./litellm treats the argument as a path, which makes
coverage.xml emit repo-relative paths (`litellm/proxy/proxy_server.py`).
Each path is unambiguous, so Codecov resolves all files correctly.

Verified locally: rerunning a single proxy_unit_tests test with
--cov=./litellm produced `filename="litellm/proxy/proxy_server.py"`,
`filename="litellm/router.py"`, and `filename="litellm/types/router.py"`
as distinct entries — exactly the disambiguation Codecov needs.

Touches every workflow that uploads coverage: the two reusable GHA
workflows (_test-unit-base.yml, _test-unit-services-base.yml),
test-mcp.yml, and all 14 invocations in .circleci/config.yml.
2026-05-14 14:01:05 -07:00
Yuneng Jiang
aff4717494
[Infra] Expand CI branch filters for non-main PR targets
Required test-unit-* and related workflows only triggered on PRs targeting
main, so feature PRs routed through litellm_internal_staging or
litellm_oss_branch never dispatched the full suite. Branch protection
reported BLOCKED even when CircleCI was green.

Expand pull_request and push branch filters to also match
litellm_internal_staging, litellm_oss_branch, and "litellm_**" (using **
so branch names containing "/" also match).
2026-04-15 15:39:57 -07:00
stuxf
a6c30b30bf
build: migrate packaging, CI, and Docker from Poetry to uv (#25007)
* build: migrate packaging metadata to uv

* ci: move automation and local tooling to uv

* docker: migrate image builds and runtime setup to uv

* docs: update install and deployment guidance for uv

* chore: align auxiliary scripts and tests with uv

* test: harden test_litellm isolation

* fix: keep release and health check images self-contained

* build: pin uv tooling and health check deps

* test: isolate bedrock image request formatting from suite state

* test: cover sandbox executor requirements flow

* ci: fix circleci no-op command steps

* ci: fix circleci publish workflow parsing

* fix: stabilize remaining uv migration CI checks

* ci: increase matrix test timeout headroom

* fix: restore published docker and license coverage

* fix: restore proxy runtime build parity

* fix: restore proxy extras parity and venv migrations

* ci: persist uv path across circleci steps

* fix: keep psycopg binary in default test env

* docker: preserve prisma cache across stages

* test: run local proxy checks through uv python

* build: restore runtime deps moved into ci

* build: refresh uv lock after upstream merge

* fix: restore module import in test_check_migration after merge

The conflict resolution imported only the function but the test body
references check_migration as a module throughout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching

- Move google-generativeai, Pillow, tenacity back to ci group (they are
  lazily imported and bloat the base SDK install needlessly)
- Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant
  in Docker where system Node.js is already installed via apk)
- Remove all nodejs-wheel node replacement and venv npm patching blocks
  from Dockerfiles since the wheel is no longer installed
- Add --no-default-groups to CodSpeed benchmark workflow so the benchmark
  environment matches the old minimal pip install footprint
- Apply standard uv two-phase Docker pattern: copy metadata first, install
  deps (cached layer), then copy source and install project
- Replace CircleCI enterprise no-op with proper uv sync command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate uv.lock after removing nodejs-wheel-binaries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): use cache/restore instead of cache to prevent cache poisoning

The old workflow used actions/cache/restore (read-only). The uv migration
changed it to actions/cache (read-write), which zizmor flags as a cache
poisoning risk. Restore the safer read-only variant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert

The setup-uv action enables caching by default, which zizmor flags as a
cache poisoning risk. Disable it since we already use a read-only
cache/restore step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv cache in publish workflow

Silences zizmor cache-poisoning alert. Publishing workflow runs
infrequently on protected branches so caching adds no real benefit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): remove duplicate verbose_logger mock in test_check_migration

The logger was patched twice — first via mocker.patch() then via
mocker.patch.object(autospec=True). The second call fails because
autospec cannot inspect an already-mocked attribute. Remove the
redundant first patch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): free disk space before Docker build in test-server-root-path

The Dockerfile.non_root build ran out of disk on the CI runner. Remove
Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 11:46:23 -07:00
Krrish Dholakia
dfb543369b fix: address zizmor comments 2026-03-26 21:09:01 -07:00
Yuneng Jiang
1beb687f54 pin GHA dependencies + remove unused load test files
Pin all pip install commands to exact versions and SHA-pin all GitHub
Actions to prevent supply chain attacks. Remove snok/install-poetry
in favor of direct pip install. Delete orphaned load test scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 22:32:08 -07:00
Julio Quinteros Pro
24ead9b4e1 fix(ci): install enterprise package into main project venv, not enterprise's own venv
Running `cd enterprise && poetry run pip install -e .` causes poetry to
create a separate venv in `enterprise/.venv` (since enterprise/ has its
own pyproject.toml). The main project's tests run with `.venv/bin/python`,
so the enterprise package installed in `enterprise/.venv` is never seen.

Fix: run `poetry run pip install -e enterprise/` from the repo root so
poetry uses the main project's venv. This ensures litellm_enterprise is
importable when tests run.

This explains why enterprise tests kept failing with:
  AttributeError: '_PROXY_LiteLLMManagedFiles' object has no attribute
  '_check_file_deletion_allowed'
even after --force-reinstall was added — the reinstall was going to the
wrong virtual environment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 21:24:06 -03:00
Julio Quinteros Pro
bbbac1ae0e fix(ci): apply --force-reinstall --no-deps to enterprise install in all CI configs
The same PyPI-override issue existed in test-litellm.yml, test-mcp.yml,
and .circleci/config.yml. Also adds --no-deps (enterprise has no runtime
deps) to avoid redundant dependency resolution on every forced reinstall.

Addresses greptile review comments on PR #21481.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 18:53:45 -03:00
Yuta Saito
7d7b78a53d test: Pydantic version up
mcp 1.25.0 depends on pydantic<3.0.0 and >=2.11.0
2026-01-20 10:49:33 +09:00
Yuta Saito
b229410fb5 test: ci requirements mcp version up 2026-01-20 10:42:58 +09:00
saisurya237
f3ead2c153
add a new feature fix to expose the team alias when authenticating th… (#17725)
* CLI SSO: show team aliases in team selection

* temp poetry lock fix

* add poetry.lock to gitignore

* Revert proxy_server background job refactor

* Revert proxy_server background job refactor

* rever gitignore and poetry lock file
2025-12-10 10:10:28 -08:00
Krrish Dholakia
5bae1f6540 perf(test-mcp.yml): run mcp tests on linting 2025-09-22 17:41:43 -07:00