mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
7a5d6a0548
The absence of a CI signal is indistinguishable from a passing one, and that shape has now produced several independent holes: whole test directories no job runs, and shipped images no job builds. Nothing was watching for either, so each was found by accident. assert_ci_coverage.py enumerates every test_*.py under tests/ and every Dockerfile in the repo, then credits only what the workflows and the CircleCI config actually invoke. Paths filters and lint steps that merely name a directory do not count as coverage, because crediting a mention is the same mistake one level up. Anything neither invoked nor listed in .github/ci-coverage-allowlist.yml with a written reason fails the job. The guard found 275 uncovered test files and 6 unbuilt Dockerfiles. Fixed here: the root Dockerfile, the primary published image, now gets an image-scan leg that builds it and runs the offline migration check against it, and 8 tests/test_litellm subdirectories join the shards that already enumerate their siblings. Everything else is allowlisted per file, so a new file cannot inherit an exemption, and the remaining decisions are tracked rather than invisible. The job reports but is not in branch protection, so it does not block merges; promoting it is a separate change once the allowlist has survived contact with a few pull requests.
42 lines
1 KiB
YAML
42 lines
1 KiB
YAML
name: "CI Coverage"
|
|
|
|
on:
|
|
pull_request:
|
|
branches:
|
|
- main
|
|
- litellm_internal_staging
|
|
- litellm_oss_staging
|
|
- "litellm_**"
|
|
push:
|
|
branches:
|
|
- main
|
|
- litellm_internal_staging
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
|
|
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
|
|
|
jobs:
|
|
assert-ci-coverage:
|
|
name: assert-ci-coverage
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
|
with:
|
|
python-version: "3.12"
|
|
|
|
- name: Assert every test file and Dockerfile is invoked by a job
|
|
run: |
|
|
python -m pip install "pyyaml==6.0.3"
|
|
python .github/scripts/assert_ci_coverage.py
|