litellm/.github/workflows
yuneng-jiang da55893656
ci(unit): cut unit check time with cheaper per-test resets and fewer non-unit tests (#51864)
* test(unit): reset module state only when a test changed it

The proxy conftest deep-copied every snapshotted litellm attribute before each
test, and the copy of litellm.model_cost alone was a median 0.12 s for each of
56,031 proxy tests on the runner. The reset now copies an attribute back only
when it no longer equals the snapshot.

The enterprise and MCP server conftests reloaded litellm before every test,
3.27 s and 0.21 s per test. They now use the same reset.

* test(unit): remove six flaky tests

They failed on main without a code change behind them between 2026-11-05 and
2026-11-14: the vector store hook SDK fallback, the concurrent user config TPM
count, three Prometheus metrics server port tests and the OpenCode list cost
billing test.

* ci: run four heavy unit shards on 16-core runners and keep two smoke jobs off pull requests

proxy-feature-endpoints, proxy-endpoints, root and llms-providers ran 1.9 to 2.2
times faster with 8 workers on a 16-core runner in a benchmark run.

windows-sdk and proxy-build-from-pip still run on pushes to main and manual
runs. proxy-build-from-pip installs a published release and never runs the
pull request's code.

* test: move 109 tests that start a process or open a socket into tests/integration

tests/unit is in-process by its own rules, and these tests spawn a Python
interpreter, a shell or a local TCP or HTTP server. They took 921 s of unit
shard time per run, 396 s of it in the MCP client transport tests. Each one now
lives in a *_real_io.py file under the integration directory for its domain and
runs with that directory's group.

The tests themselves are unchanged apart from one repository-root path in the
CLI test.

* test(unit): copy back only the dict entries a test changed

A test module that registers models at import leaves litellm.model_cost four
entries larger than the snapshot before every test in its directory, so the
reset still deep-copied the whole price map for each of them. It now rebuilds a
changed dict from the live entries that still match and copies only the rest.

* ci(smoke): keep the two jobs that skip on pull requests out of the smoke gate

A skipped job passing the gate is what test_unit_passed_gate guards against,
so the gate goes back to requiring success from every job it needs and
windows-sdk and proxy-build-from-pip stop being part of it. They still run
and report on pushes to main and manual runs.
2026-11-14 15:41:30 -08:00
..
auto-close-duplicates.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
check-schema-sync.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
codeql.yml ci(codeql): run the default suite on full scans and security-extended on PRs (#45149) 2026-10-07 13:24:28 -07:00
codspeed.yml chore(ci): drop litellm_internal_staging and litellm_oss_staging references, main is the only trunk (#42745) 2026-09-23 08:14:11 -07:00
compat-matrix-image.yml feat(docker)!: one shipped image and one entrypoint beside the Dockerfile (#43332) 2026-11-10 18:32:16 -08:00
conventional-commits.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
cost-map-guard.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
cost-map-pins-recheck.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
create-rc-branch.yml ci: cut the weekly rc branch on Monday 3am PT instead of Friday (#49472) 2026-10-31 12:32:23 -07:00
create-release-branch.yml ci(release): create stable/X.Y.x line branch on X.Y.0 tags (#29457) 2026-06-01 15:56:34 -07:00
duplicate_issue_check.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
e2e_record_replay.yml ci(e2e): record the e2e suite weekly and replay it on weekdays with zero egress (#38163) 2026-08-24 23:49:03 -04:00
guard-fork-dependencies.yml chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
image-scan.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
interactions-spec-canary.yml feat(interactions): forward continuation_token and type requires_continuation so callers can resume a long decode (#50151) 2026-11-05 15:38:44 -08:00
issue-keyword-labeler.yml chore: fixes 2026-04-05 01:30:57 -07:00
issue_classifier.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
issue_fixed_comment.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
issue_label_claude_code.yml ci(issue-classifier): name every workflow, script and job after the issue it works on 2026-09-18 11:37:55 -07:00
issue_label_sync.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
linear-release.yml ci: sync the weekly release cycle with Linear releases (#43636) 2026-09-28 17:00:56 -07:00
mutation-test.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
osv-scan.yml chore: move ui and vscode-extension under litellm and delete docs (#50992) 2026-11-12 04:19:57 +00:00
publish-lint-base-counts.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
root-entry-limit.yml ci: fail when the repo root grows past 15 entries (#51007) 2026-11-12 04:19:58 +00:00
scorecard.yml chore: fixes 2026-04-05 01:30:57 -07:00
spend-logs-guard.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
stale.yml chore: fixes 2026-04-05 01:30:57 -07:00
sync-schema.yml chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test-e2e-redis-chaos.yml test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
test-integration.yml ci(integration): point the rust gateway install step at litellm/packaging (#51743) 2026-11-14 13:36:57 +09:00
test-linting.yml ci: stop PR runs from thrashing the Actions cache and stop smoke compiling from source (#51343) 2026-11-11 23:52:32 -08:00
test-mcp-dependency-resolution.yml ci(mcp-deps): build unoptimized in the MCP dependency resolution workflow (#51684) 2026-11-13 19:17:44 +00:00
test-merge-smoke.yml ci(unit): cut unit check time with cheaper per-test resets and fewer non-unit tests (#51864) 2026-11-14 15:41:30 -08:00
test-rust-db.yml feat(db): add the SQLx pool foundation with the writer pool (#49798) 2026-11-14 15:28:10 -08:00
test-rust.yml ci: stop PR runs from thrashing the Actions cache and stop smoke compiling from source (#51343) 2026-11-11 23:52:32 -08:00
test-terraform-modules.yml chore: move deployment sources under litellm/deploy (#50981) 2026-11-12 04:19:56 +00:00
test-terraform-provider.yml ci: stop PR runs from thrashing the Actions cache and stop smoke compiling from source (#51343) 2026-11-11 23:52:32 -08:00
test-unit.yml ci(unit): cut unit check time with cheaper per-test resets and fewer non-unit tests (#51864) 2026-11-14 15:41:30 -08:00
test-vscode-extension.yml chore: move ui and vscode-extension under litellm and delete docs (#50992) 2026-11-12 04:19:57 +00:00
zizmor.yml ci: run zizmor with --no-ignores and move the spend logs and cost map guards to pull_request (#48947) 2026-10-28 17:44:48 -07:00