Commit graph

11 commits

Author SHA1 Message Date
mateo-berri
cc5ff14d47 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ban_data_migrations 2026-08-22 09:55:24 -07:00
yuneng-jiang
afec9b8ab9
perf(ci): cache the Rust build the unit shards compile from scratch (#37795)
* perf(ci): cache the Rust build the unit shards compile from scratch

Every unit shard installs the workspace, and the root package builds through
maturin, so each of the eleven jobs compiles litellm-rust/crates/python-bridge
in release mode before a single test runs. That step measured 2m40s a shard on
2026-08-21, which is more wall clock than the entire unit tier spends running
tests, and none of it was cached: the uv cache covers wheels it downloads, not
wheels it builds, and a path dependency whose source moves every commit can
never hit that cache anyway.

A composite action now exports CARGO_TARGET_DIR to a fixed workspace path and
caches it alongside the Cargo registry, keyed on Cargo.lock. Cargo rebuilds only
what changed, so a warm job pays for the bridge crate rather than its whole
dependency graph. Measured locally, that is 34s cold against 8s warm, including
after a Python-only or Rust-only edit.

The absolute path matters: uv builds the wheel from its own working directory,
so a relative target directory lands the artifacts where nothing can find them
again.

* perf(ci): cache the Rust build in the other four workflows that sync the workspace

code-quality, mcp, documentation and the schema.d.ts check each install the
workspace and so each compile the bridge from scratch, measured at 138s, 177s,
163s and 154s on 2026-08-21. The lint job pays the same and is left to #37783,
which already owns that file's setup section.

* perf(ci): cache the Rust build in the lint job too

* fix(ci): cache cargo's own target directory instead of redirecting it

uv builds the wheel in place, so cargo already writes to litellm-rust/target,
which test-rust.yml has cached all along. Redirecting CARGO_TARGET_DIR bought
nothing and cost a GITHUB_ENV write that zizmor rejects as a code-execution
path.

* chore(ci): raise the job backstop for the added setup step

The cargo cache is a fifth bounded setup step, so the base's setup ceiling goes
30m to 35m and every job budget follows: 55 to 60, and proxy-server's 95 to 100.
check_workflow_startup_safety enforces exactly this sum, and failed on the first
push without it.

* perf(ci): cache the Rust build in the four remaining workflows that sync

Six workflows were wired; ten install the workspace. The four left out still
compile the pyo3 bridge from scratch.

test-terraform-provider.yml is the one that matters per PR: its endpoint-drift
job triggers on any change under litellm/proxy/**. The other three are a
scheduled load check, a manual mutation run, and the staging-push counts
publisher, whose gate syncs the project inside scripts/type_check_gate.py
rather than in a workflow step, so nothing in the file names the build it pays
for.
2026-08-21 20:15:08 -07:00
Yucheng Zhu
6c7dfbd249 ci: run the migration data-rewrite check in code quality 2026-08-21 15:27:56 -07:00
mateo-berri
c22a749f70
fix(ci): drop unsupported arithmetic from the job timeout expression
GitHub expressions have no arithmetic operators, so
`${{ inputs.timeout-minutes + inputs.setup-timeout-minutes }}` was not a value
but a startup failure. The proxy-db workflow died before creating any job on
both prior commits, which posts no check run at all: the entire suite stopped
running while the PR's checks stayed green.

Pass the job backstop in as `job-timeout-minutes` instead of computing it, and
size it as the test budget plus the 30 minutes of setup ceilings plus 5 minutes
of runner overhead the job clock charges but no step owns.

check_workflow_startup_safety.py makes this class of mistake visible before
merge, since CI cannot report it: it rejects arithmetic inside an expression
and checks every caller of the reusable workflow keeps a job budget large
enough that the deadline cannot preempt pytest inside its own budget.
2026-08-10 17:13:16 +00:00
mateo-berri
6a83a84f31
ci: cache Prisma CLI and engine binaries, split test timeout from setup
`prisma generate` runs `npm install prisma@<version>` whenever the
prisma-client-py binary cache directory has no CLI entrypoint, pulling ~85 MB
of query and schema engines over the network. Every workflow pointed
PRISMA_BINARY_CACHE_DIR at `${{ runner.temp }}/prisma-cache`, which GitHub
wipes and recreates per job, so that cache was empty on every job of every
run and the download was never avoidable.

The download is normally a few seconds and occasionally minutes. On one
proxy-db run it took 5m18s on a single shard against 3.8s on its eleven
siblings, which pushed the job past its 15 minute timeout and cancelled a
shard whose tests were at 99% and all passing.

Leave PRISMA_BINARY_CACHE_DIR unset so the binaries land in the
prisma-client-py default, which is already keyed by prisma and engine
version, and restore both that path and the @prisma/engines staging cache
through a shared composite action.

Job timeouts also counted setup against the test budget. `timeout-minutes`
now bounds the pytest step, with a separate allowance for checkout,
dependency install, and client generation, so slow setup shows up as a slow
job instead of a cancelled test run.

check_prisma_binary_cache.py guards all three invariants: no workflow
reintroduces the override, every job that generates the client restores the
cache, and the version the action greps out of uv.lock still resolves.
2026-08-10 16:10:54 +00:00
Yuneng Jiang
37ca659f84
ci: run unit tests on pushes to main and litellm_internal_staging
The unit workflows trigger on pull_request only, so a commit that
actually lands on a gated branch ends up with no unit-test check runs at
all. The commit status API reports success for those commits, which a
release gate reads as "nothing failed" rather than "never tested". PR
checks also only ever ran against the merge preview, not the commit that
landed, so two branches that are each green can still land broken
together

Add a push trigger on the two gated branches to the twelve unit
workflows and to the code-quality workflow, so every landed commit gets
check runs addressable by its SHA

test-linting.yml is deliberately left on pull_request only; six of its
steps gate on a diff against github.event.pull_request.base.sha, which
is empty outside a pull request, and a "what did this branch add" check
has no meaning on a merge commit

Also key the concurrency group on github.sha and restrict
cancel-in-progress to pull_request. The previous group was stable across
pushes to a branch, so consecutive merges would cancel the in-flight run
for the earlier commit and leave that SHA without a result, which is the
same blind spot this change is meant to close
2026-08-01 11:58:49 -07:00
mateo-berri
0bfdb37266 test(e2e): route external agent card fetch through the typed transport
Adds get_external to e2e_http.py for absolute third-party GETs (no proxy base url or auth, same Result classification) and rewires fetch_agent_card through it, dropping the urllib.request escape hatch. Creates tests/code_coverage_tests/check_e2e_no_raw_requests.py, the checker tests/e2e/CLAUDE.md already referenced, and wires it into the code-quality workflow so raw HTTP client imports outside the transport fail CI; pre-existing uses (root conftest liveness probe, claude_code version resolver) are grandfathered and exception-type-only imports stay allowed.
2026-07-21 16:35:02 -07:00
mateo-berri
bb1b3dc937 fix(ci): retry setup-uv installs to survive transient manifest fetch failures 2026-07-14 14:41:36 -07:00
Sameer Kankute
c712c20d0f
fix(ci): point OSS contributor workflows to litellm_oss_staging (#31270)
* fix(ci): point OSS contributor workflows to litellm_oss_staging

Workflow triggers and guard error messages incorrectly referenced litellm_oss_branch; update them to the branch we actually use for external contributions.

* fix(ci): include test-rust.yml in litellm_oss_staging rename

Missed test-rust.yml when updating OSS contributor target branch references.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-24 21:07:59 -07:00
Yuneng Jiang
d42281338e ci: check out litellm-docs directly into docs/my-website
Replaces the rm-and-symlink hack with a plain actions/checkout
using path: docs/my-website. The previous approach failed on this
branch because docs/my-website no longer exists in the repo (its
parent docs/ directory was also removed), so ln -s had nowhere
to create the symlink.

Also adds the same checkout step to test-unit-documentation.yml,
which was silently relying on docs/my-website existing in-tree
for test_env_keys.py and test_router_settings.py.
2026-04-24 14:21:18 -07:00
Yuneng Jiang
1b74c35b89
[Infra] Move non-API-key CCI jobs to GitHub Actions
Principle: GHA handles work that doesn't need external API keys; CCI
stays for integration tests that hit real API endpoints.

Four CCI jobs moved to new or extended GHA workflows:

1. check_code_and_doc_quality (was 25 runs: ruff + import-safety +
   21 code_coverage_tests + 3 documentation_tests + circular-imports).
   - The 21 tests/code_coverage_tests/*.py scripts and the 3
     tests/documentation_tests/*.py scripts run in the new
     .github/workflows/test-code-quality.yml workflow.
   - ruff, import-safety, and circular-imports were already run by
     .github/workflows/test-linting.yml — no new migration needed.
   - The 3 documentation_tests scripts read
     docs/my-website/docs/proxy/config_settings.md. Since docs have
     moved to BerriAI/litellm-docs, the GHA workflow checks out that
     repo and symlinks docs/my-website -> the checkout so the
     existing hardcoded paths resolve without touching the scripts.
     The stale local docs/my-website/ copy in this repo will be
     removed in a separate PR.

2. semgrep (custom-rule SAST against .semgrep/rules).
   - New .github/workflows/test-semgrep.yml.

3. installing_litellm_on_python + installing_litellm_on_python_3_13
   (pip install compat checks on Python 3.12 and 3.13).
   - New .github/workflows/test-install-litellm.yml as a matrix job.
   - 3.12 run also verifies litellm_enterprise import; 3.13 run
     skips that check (matches previous CCI behavior).
   - installing_litellm_on_python_v2_migration_resolver stays in CCI
     because it requires a postgres service.

CCI .circleci/config.yml: -112 lines, 4 jobs and their workflow refs
removed.
2026-04-22 13:38:00 -07:00