mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-10 22:41:41 +00:00
ci: follow the default branch in development tooling
This commit is contained in:
Yuneng Jiang
parent
f896df1b06
commit
e238d20fbd
15 changed files with 437 additions and 94 deletions
|
|
@ -29,7 +29,7 @@ Never test structure of code only function of it
|
|||
|
||||
End-to-end tests belong in `tests/e2e/` and must follow the harness conventions documented in that directory's `CLAUDE.md`
|
||||
|
||||
When creating PRs, don't set base to `main`. `litellm_internal_staging` is the default base branch and serves that purpose for both internal and external / OSS contributions
|
||||
When creating PRs, target the repository's current default branch for both internal and external / OSS contributions. Check it with `python3 scripts/default_branch.py --branch` instead of assuming a branch name or relying on cached `origin/HEAD`
|
||||
|
||||
When writing a PR body, treat the comments and imperative instructions inside .github/pull_request_template.md as rules to follow, not just layout. Agent harnesses may strip HTML comments from copies of that file injected into context, so read .github/pull_request_template.md from disk before writing a PR body to make sure you see every comment rule
|
||||
|
||||
|
|
@ -52,7 +52,7 @@ Don't hesitate to use values in .env to get needed API keys and other secrets, a
|
|||
|
||||
Python max line length is 120, not 88
|
||||
|
||||
Never edit or commit `ruff-strict-budget.json`, `type-discipline-budget.json`, `basedpyright-code-budget.json`, or `test-quality-budget.json` on a PR branch, and don't run `make lint-budget-update` there. A scheduled Devin automation lowers the limits on `litellm_internal_staging` in its own PR by exactly what landed since the last ratchet, so concurrent PRs don't fight over the same `"limit"` lines. If your branch already carries a budget edit, drop it before opening the PR
|
||||
Never edit or commit `ruff-strict-budget.json`, `type-discipline-budget.json`, `basedpyright-code-budget.json`, or `test-quality-budget.json` on a PR branch, and don't run `make lint-budget-update` there. A scheduled Devin automation lowers the limits on the default branch in its own PR by exactly what landed since the last ratchet, so concurrent PRs don't fight over the same `"limit"` lines. Keep the hosted automation's target in sync when the repository default changes. If your branch already carries a budget edit, drop it before opening the PR
|
||||
|
||||
`make check` (f.k.a. `make pre-commit`, which still works identically as an alias) saves its complete output to a log file in .git (overwriting previous logs) and prints that path as its first and last output lines. To inspect a run, read or grep that log instead of re-running the multi-minute checks just to see a different slice
|
||||
|
||||
|
|
@ -70,7 +70,7 @@ When referencing or running models (coding, QA'ing, writing docs, writing tests,
|
|||
|
||||
Always pull before starting any work. The checkout or worktree may be sitting on a stale branch
|
||||
|
||||
If you're an internal contributor, when creating a new PR, the typical flow is to branch off litellm_internal_staging and create a branch prefixed with litellm_. Do not create a branch prefixed with claude/ and generally do not have / in your branch names
|
||||
If you're an internal contributor, when creating a new PR, the typical flow is to branch off the repository's current default branch and create a branch prefixed with litellm_. Do not create a branch prefixed with claude/ and generally do not have / in your branch names
|
||||
|
||||
Do not add `Co-Authored-By: Claude` or any Claude attribution to commit messages. Never use a `claude/` prefix or put a `/` in a branch name. Do not add "Generated with Claude Code" (or any similar attribution) to PR descriptions or comments. Do not create a new PR/branch off the existing PR to fix/add something that is related and could've just been committed directly to the existing PR's branch
|
||||
|
||||
|
|
|
|||
|
|
@ -315,10 +315,12 @@ Ensure the UI builds successfully before submitting your PR:
|
|||
npm run build
|
||||
```
|
||||
|
||||
Local lint and budget checks follow origin's current default branch. They refresh it from the remote instead of trusting cached `origin/HEAD`. For an intentional comparison against another branch or commit, use `make check BASE_REF=<ref>` or the standalone gate's `--base <ref>` option. An explicit ref can also be used offline once it has been fetched locally. Without an override, unavailable remote metadata stops the check
|
||||
|
||||
## Submitting Your PR
|
||||
|
||||
1. **Push your branch**: `git push origin your-feature-branch`
|
||||
2. **Create a PR**: Go to GitHub and open a pull request against [`litellm_internal_staging`](https://github.com/BerriAI/litellm/tree/litellm_internal_staging), which is the default base branch. Do not target `main`.
|
||||
2. **Create a PR**: Go to GitHub and open a pull request against the repository's current default branch. Run `python3 scripts/default_branch.py --branch` to check its name
|
||||
3. **Fill out the PR template**: Provide clear description of changes
|
||||
4. **Wait for review**: Maintainers will review and provide feedback
|
||||
5. **Address feedback**: Make requested changes and push updates
|
||||
|
|
|
|||
48
Makefile
48
Makefile
|
|
@ -34,7 +34,7 @@ help:
|
|||
@echo " make lint-basedpyright-budget-update - Ratchet basedpyright limits down by what this branch fixed"
|
||||
@echo " make lint-format - Check ruff format formatting (matches CI)"
|
||||
@echo " make lint-ruff-budget - Gate the codebase total of each strict ruff rule against its limit"
|
||||
@echo " make lint-gate - Strict ruff gate in CI-parity mode (fetches staging, simulates the merge)"
|
||||
@echo " make lint-gate - Strict ruff gate in CI-parity mode (fetches the default branch, simulates the merge)"
|
||||
@echo " make lint-ruff-budget-update - Ratchet ruff-strict-budget.json limits down by what this branch fixed"
|
||||
@echo " make lint-test-quality - Gate the test suite against test-quality-budget.json"
|
||||
@echo " make lint-budget-update - Ratchet all budgets down (ruff + type-discipline + test quality + basedpyright)"
|
||||
|
|
@ -60,6 +60,9 @@ help:
|
|||
|
||||
UV := uv
|
||||
UV_RUN := $(UV) run --no-sync
|
||||
BASE_REF ?=
|
||||
export BASE_REF
|
||||
RESOLVE_BASE = python3 scripts/default_branch.py --base "$(BASE_REF)"
|
||||
|
||||
# Machine-wide slot queue for the heavy targets below; python3 + stdlib only, so
|
||||
# it runs before any venv exists. See scripts/gate_slot_lock.py.
|
||||
|
|
@ -133,7 +136,7 @@ format-check: install-dev
|
|||
# Single fetch of the PR base so the delta-based gates below share one network round
|
||||
# trip instead of each re-fetching when chained from `lint`.
|
||||
lint-fetch-base:
|
||||
git fetch origin litellm_internal_staging
|
||||
@$(RESOLVE_BASE)
|
||||
|
||||
# Mirror test-linting.yml's lint job environment: the proxy-dev group plus a generated
|
||||
# Prisma client, so `basedpyright tests/e2e` resolves the same modules CI does. The
|
||||
|
|
@ -150,7 +153,9 @@ lint-install:
|
|||
# recursively, so 'litellm/*.py' covers nested modules and the top-level files that
|
||||
# CI's 'litellm/**/*.py' skips, which makes this target a superset of the CI step.
|
||||
lint-format-check-changed: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
@files=$$(git diff --name-only --diff-filter=ACMR origin/litellm_internal_staging...HEAD -- 'litellm/*.py' | grep -v '^litellm/enterprise/' || true); \
|
||||
@base_ref=$$($(RESOLVE_BASE)) && \
|
||||
changed=$$(git diff --name-only --diff-filter=ACMR "$$base_ref...HEAD" -- 'litellm/*.py') && \
|
||||
files=$$(printf '%s\n' "$$changed" | grep -v '^litellm/enterprise/' || true) || exit $$?; \
|
||||
if [ -z "$$files" ]; then \
|
||||
echo "No changed litellm Python files to format-check."; \
|
||||
else \
|
||||
|
|
@ -167,7 +172,9 @@ lint-ruff: $(LINT_DEP_INSTALL)
|
|||
# https://github.com/astral-sh/ruff/discussions/10977
|
||||
# https://github.com/astral-sh/ruff/discussions/4049
|
||||
lint-format-changed: install-dev
|
||||
@git diff origin/main --unified=0 --no-color -- '*.py' | \
|
||||
@base_ref=$$($(RESOLVE_BASE)) && \
|
||||
diff=$$(git diff "$$base_ref" --unified=0 --no-color -- '*.py') && \
|
||||
printf '%s\n' "$$diff" | \
|
||||
perl -ne '\
|
||||
if (/^diff --git a\/(.*) b\//) { $$file = $$1; } \
|
||||
if (/^@@ .* \+(\d+)(?:,(\d+))? @@/) { \
|
||||
|
|
@ -182,20 +189,22 @@ lint-format-changed: install-dev
|
|||
done
|
||||
|
||||
lint-ruff-dev: install-dev
|
||||
@tmpfile=$$(mktemp /tmp/ruff-dev.XXXXXX) && \
|
||||
@base_ref=$$($(RESOLVE_BASE)) || exit $$?; \
|
||||
tmpfile=$$(mktemp /tmp/ruff-dev.XXXXXX) && \
|
||||
cd litellm && \
|
||||
($(UV_RUN) ruff check . --output-format=pylint || true) > "$$tmpfile" && \
|
||||
$(UV_RUN) diff-quality --violations=pylint "$$tmpfile" --compare-branch=origin/main && \
|
||||
$(UV_RUN) diff-quality --violations=pylint "$$tmpfile" --compare-branch="$$base_ref" && \
|
||||
cd .. ; \
|
||||
rm -f "$$tmpfile"
|
||||
|
||||
lint-ruff-FULL-dev: install-dev
|
||||
@files=$$(git diff --name-only origin/main -- '*.py'); \
|
||||
@base_ref=$$($(RESOLVE_BASE)) && \
|
||||
files=$$(git diff --name-only "$$base_ref" -- '*.py') || exit $$?; \
|
||||
if [ -n "$$files" ]; then echo "$$files" | xargs $(UV_RUN) ruff check; \
|
||||
else echo "No changed .py files to check."; fi
|
||||
|
||||
lint-basedpyright: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
$(UV_RUN) python scripts/type_check_gate.py --base origin/litellm_internal_staging
|
||||
$(UV_RUN) python scripts/type_check_gate.py --base "$(BASE_REF)"
|
||||
|
||||
lint-e2e-basedpyright: $(LINT_E2E_DEP_INSTALL)
|
||||
$(UV_RUN) basedpyright tests/e2e
|
||||
|
|
@ -203,37 +212,37 @@ lint-e2e-basedpyright: $(LINT_E2E_DEP_INSTALL)
|
|||
# Type-discipline budget (mutable collections / casts / type guards / kwargs /
|
||||
# unexplained suppressions), the test-linting.yml step `make lint` used to omit.
|
||||
lint-type-discipline: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
$(UV_RUN) python scripts/type_discipline_gate.py --base origin/litellm_internal_staging
|
||||
$(UV_RUN) python scripts/type_discipline_gate.py --base "$(BASE_REF)"
|
||||
|
||||
# Test-quality budget (zero-assert / mock-echo tests, sys.path.insert, raw env writes,
|
||||
# litellm module-global mutation, credential-gated skips, conftest snapshot
|
||||
# inventory), counted across tests/ the same delta-vs-base way.
|
||||
lint-test-quality: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
$(UV_RUN) python scripts/test_quality_gate.py --base origin/litellm_internal_staging
|
||||
$(UV_RUN) python scripts/test_quality_gate.py --base "$(BASE_REF)"
|
||||
|
||||
# --update lowers each limit by what this branch fixed since its branch point, so
|
||||
# it needs the base ref fetched to resolve the merge-base.
|
||||
lint-basedpyright-budget-update: install-dev lint-fetch-base
|
||||
$(UV_RUN) python scripts/type_check_gate.py --update
|
||||
$(UV_RUN) python scripts/type_check_gate.py --update --base "$(BASE_REF)"
|
||||
|
||||
lint-format: format-check
|
||||
|
||||
lint-ruff-budget: install-dev
|
||||
$(UV_RUN) python scripts/ruff_strict_gate.py
|
||||
$(UV_RUN) python scripts/ruff_strict_gate.py --base "$(BASE_REF)"
|
||||
|
||||
# Strict gate, invoked the same way CI does in test-linting.yml so a local pass
|
||||
# means the CI check will pass too.
|
||||
lint-gate: $(LINT_DEP_INSTALL) $(LINT_DEP_BASE)
|
||||
$(UV_RUN) python scripts/ruff_strict_gate.py --base origin/litellm_internal_staging
|
||||
$(UV_RUN) python scripts/ruff_strict_gate.py --base "$(BASE_REF)"
|
||||
|
||||
lint-ruff-budget-update: install-dev lint-fetch-base
|
||||
$(UV_RUN) python scripts/ruff_strict_gate.py --update
|
||||
$(UV_RUN) python scripts/ruff_strict_gate.py --update --base "$(BASE_REF)"
|
||||
|
||||
lint-type-discipline-budget-update: install-dev lint-fetch-base
|
||||
$(UV_RUN) python scripts/type_discipline_gate.py --update
|
||||
$(UV_RUN) python scripts/type_discipline_gate.py --update --base "$(BASE_REF)"
|
||||
|
||||
lint-test-quality-budget-update: install-dev lint-fetch-base
|
||||
$(UV_RUN) python scripts/test_quality_gate.py --update
|
||||
$(UV_RUN) python scripts/test_quality_gate.py --update --base "$(BASE_REF)"
|
||||
|
||||
# Ratchet all budgets in one shot (ruff strict + type-discipline + test quality + basedpyright)
|
||||
lint-budget-update: lint-ruff-budget-update lint-type-discipline-budget-update lint-test-quality-budget-update lint-basedpyright-budget-update
|
||||
|
|
@ -249,14 +258,15 @@ check-import-safety: $(LINT_DEP_INSTALL)
|
|||
# runs the diff-scoped ruff format check, whole-tree ruff check, the strict-rule /
|
||||
# type-discipline / basedpyright budgets as a delta vs the base, then the circular-import
|
||||
# and import-safety checks. Steps that compare against the base resolve it the same way CI
|
||||
# does (merge-base with origin/litellm_internal_staging). Setup (env sync, Prisma client,
|
||||
# does (merge-base with origin's current default branch). Setup (env sync, Prisma client,
|
||||
# base fetch) runs once up front; the checks themselves are independent, so a sub-make
|
||||
# fans them out with -j and the fast ones finish under basedpyright's shadow.
|
||||
lint:
|
||||
@$(GATE_SLOT_LOCK) $(MAKE) lint-inner
|
||||
|
||||
lint-inner: lint-install lint-fetch-base
|
||||
$(MAKE) -j $(LINT_JOBS) $(LINT_OUTPUT_SYNC) LINT_DEP_INSTALL= LINT_E2E_DEP_INSTALL= LINT_DEP_BASE= lint-checks
|
||||
lint-inner: lint-install
|
||||
@base_ref=$$($(RESOLVE_BASE)) && \
|
||||
$(MAKE) BASE_REF="$$base_ref" -j $(LINT_JOBS) $(LINT_OUTPUT_SYNC) LINT_DEP_INSTALL= LINT_E2E_DEP_INSTALL= LINT_DEP_BASE= lint-checks
|
||||
|
||||
lint-checks: lint-format-check-changed lint-ruff lint-gate lint-type-discipline lint-test-quality lint-basedpyright lint-e2e-basedpyright check-circular-imports check-import-safety
|
||||
|
||||
|
|
|
|||
|
|
@ -6,12 +6,9 @@ import subprocess
|
|||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import testing.postgresql
|
||||
|
||||
from typing import Final
|
||||
|
||||
DESTRUCTIVE_PATTERN = re.compile(r"\bDROP\s+(COLUMN|TABLE|INDEX)\b", re.IGNORECASE)
|
||||
DEFAULT_BASE_BRANCH = "litellm_internal_staging"
|
||||
|
||||
|
||||
def _find_destructive_statements(sql: str) -> list:
|
||||
|
|
@ -94,31 +91,57 @@ def _print_stale_branch_refusal(base_branch: str, behind: int) -> None:
|
|||
print(banner, file=out)
|
||||
|
||||
|
||||
def _check_branch_freshness(root_dir: Path, base_branch: str) -> None:
|
||||
def _default_base_branch(root_dir: Path) -> str:
|
||||
try:
|
||||
result: Final = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
str(Path(__file__).resolve().parents[1] / "scripts" / "default_branch.py"),
|
||||
"--repo-root",
|
||||
str(root_dir),
|
||||
"--branch",
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=90,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError) as exc:
|
||||
_print_freshness_failure(
|
||||
"default branch",
|
||||
"Could not discover origin's default branch. Pass --base-branch <name> to choose one.",
|
||||
exc.stderr if isinstance(exc, subprocess.CalledProcessError) else str(exc),
|
||||
)
|
||||
sys.exit(3)
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def _check_branch_freshness(root_dir: Path, base_branch: str | None = None) -> None:
|
||||
"""Fetch origin/<base_branch> and exit 3 if HEAD is behind it."""
|
||||
resolved_branch: Final = base_branch or _default_base_branch(root_dir)
|
||||
cwd = str(root_dir)
|
||||
try:
|
||||
subprocess.run(
|
||||
["git", "fetch", "origin", base_branch],
|
||||
["git", "fetch", "origin", f"+refs/heads/{resolved_branch}:refs/remotes/origin/{resolved_branch}"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
cwd=cwd,
|
||||
)
|
||||
except FileNotFoundError:
|
||||
_print_freshness_failure(base_branch, "git executable not found on PATH")
|
||||
_print_freshness_failure(resolved_branch, "git executable not found on PATH")
|
||||
sys.exit(3)
|
||||
except subprocess.CalledProcessError as e:
|
||||
_print_freshness_failure(
|
||||
base_branch,
|
||||
f"`git fetch origin {base_branch}` failed",
|
||||
resolved_branch,
|
||||
f"`git fetch origin {resolved_branch}` failed",
|
||||
e.stderr or "",
|
||||
)
|
||||
sys.exit(3)
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "rev-list", "--count", f"HEAD..origin/{base_branch}"],
|
||||
["git", "rev-list", "--count", f"HEAD..origin/{resolved_branch}"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
|
|
@ -127,23 +150,23 @@ def _check_branch_freshness(root_dir: Path, base_branch: str) -> None:
|
|||
behind = int(result.stdout.strip())
|
||||
except subprocess.CalledProcessError as e:
|
||||
_print_freshness_failure(
|
||||
base_branch,
|
||||
f"`git rev-list HEAD..origin/{base_branch}` failed",
|
||||
resolved_branch,
|
||||
f"`git rev-list HEAD..origin/{resolved_branch}` failed",
|
||||
e.stderr or "",
|
||||
)
|
||||
sys.exit(3)
|
||||
except ValueError:
|
||||
_print_freshness_failure(
|
||||
base_branch,
|
||||
resolved_branch,
|
||||
"could not parse commit count from `git rev-list`",
|
||||
)
|
||||
sys.exit(3)
|
||||
|
||||
if behind > 0:
|
||||
_print_stale_branch_refusal(base_branch, behind)
|
||||
_print_stale_branch_refusal(resolved_branch, behind)
|
||||
sys.exit(3)
|
||||
|
||||
print(f"Branch freshness OK: up to date with origin/{base_branch}.")
|
||||
print(f"Branch freshness OK: up to date with origin/{resolved_branch}.")
|
||||
|
||||
|
||||
def _print_destructive_refusal(destructive_lines: list) -> None:
|
||||
|
|
@ -198,7 +221,7 @@ def _print_destructive_refusal(destructive_lines: list) -> None:
|
|||
def create_migration(
|
||||
migration_name: str = None,
|
||||
allow_destructive: bool = False,
|
||||
base_branch: str = DEFAULT_BASE_BRANCH,
|
||||
base_branch: str | None = None,
|
||||
skip_freshness_check: bool = False,
|
||||
):
|
||||
"""
|
||||
|
|
@ -211,7 +234,7 @@ def create_migration(
|
|||
DROP COLUMN, DROP TABLE, or DROP INDEX statements. Without this
|
||||
flag, the script exits non-zero and prints guidance.
|
||||
base_branch (str): Branch to check freshness against
|
||||
(default: "litellm_internal_staging").
|
||||
(default: origin's current default branch).
|
||||
skip_freshness_check (bool): Skip the "branch is up to date" check.
|
||||
Only for intentional migrations against an older base.
|
||||
"""
|
||||
|
|
@ -225,6 +248,8 @@ def create_migration(
|
|||
else:
|
||||
_check_branch_freshness(root_dir, base_branch)
|
||||
|
||||
import testing.postgresql
|
||||
|
||||
try:
|
||||
migrations_dir = (
|
||||
root_dir / "litellm-proxy-extras" / "litellm_proxy_extras" / "migrations"
|
||||
|
|
@ -342,9 +367,8 @@ if __name__ == "__main__":
|
|||
)
|
||||
parser.add_argument(
|
||||
"--base-branch",
|
||||
default=DEFAULT_BASE_BRANCH,
|
||||
help=(
|
||||
f"Branch to check freshness against (default: {DEFAULT_BASE_BRANCH}). "
|
||||
"Branch to check freshness against (default: origin's current default branch). "
|
||||
"The script fetches origin/<base-branch> and refuses to run if HEAD "
|
||||
"is behind it."
|
||||
),
|
||||
|
|
|
|||
|
|
@ -48,7 +48,7 @@ uv run --with testing.postgresql python ci_cd/run_migration.py "your_migration_n
|
|||
|
||||
## What It Does
|
||||
|
||||
1. **Verifies the current branch is up to date with `origin/litellm_internal_staging`** (see [Branch freshness](#branch-freshness-check))
|
||||
1. **Verifies the current branch is up to date with origin's current default branch** (see [Branch freshness](#branch-freshness-check))
|
||||
2. Creates temp PostgreSQL DB
|
||||
3. Applies existing migrations
|
||||
4. Compares with `schema.prisma`
|
||||
|
|
@ -57,11 +57,11 @@ uv run --with testing.postgresql python ci_cd/run_migration.py "your_migration_n
|
|||
|
||||
## Branch Freshness Check
|
||||
|
||||
Before generating anything, `run_migration.py` runs `git fetch origin <base>` and refuses to proceed if `HEAD` is behind `origin/<base>`. Default base is `litellm_internal_staging` (the branch PRs target). A previous incident saw a stale branch silently drop production columns; freshness is the first-line defense.
|
||||
Before generating anything, `run_migration.py` runs `git fetch origin <base>` and refuses to proceed if `HEAD` is behind `origin/<base>`. The default base is discovered from origin's advertised HEAD on each run, so an existing clone follows a default-branch change without trusting cached `origin/HEAD`. If discovery or fetching fails, migration generation stops. A previous incident saw a stale branch silently drop production columns; freshness is the first-line defense.
|
||||
|
||||
Flags:
|
||||
|
||||
- `--base-branch <name>` — check against a different base (e.g. `main`). Default is `litellm_internal_staging`.
|
||||
- `--base-branch <name>` — check against a different base (e.g. a release branch). Defaults to origin's current default branch
|
||||
- `--skip-freshness-check` — bypass entirely. Only for intentional migrations against an older base.
|
||||
|
||||
When the guard fires:
|
||||
|
|
@ -69,8 +69,9 @@ When the guard fires:
|
|||
1. Update your branch:
|
||||
|
||||
```bash
|
||||
git fetch origin && git rebase origin/litellm_internal_staging
|
||||
# or git merge origin/litellm_internal_staging — whichever matches your workflow
|
||||
base_branch=$(python3 scripts/default_branch.py --branch) &&
|
||||
git fetch origin "+refs/heads/$base_branch:refs/remotes/origin/$base_branch" &&
|
||||
git rebase "origin/$base_branch"
|
||||
```
|
||||
2. Re-run `run_migration.py`.
|
||||
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ import subprocess
|
|||
import sys
|
||||
from pathlib import Path
|
||||
from types import MappingProxyType
|
||||
from typing import NamedTuple
|
||||
from typing import Final, NamedTuple
|
||||
|
||||
if sys.version_info >= (3, 11):
|
||||
import tomllib
|
||||
|
|
@ -42,7 +42,6 @@ else:
|
|||
import tomli as tomllib
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
DEFAULT_BASE = "origin/litellm_internal_staging"
|
||||
DEFAULT_BUDGETS: tuple[str, ...] = (
|
||||
"ruff-strict-budget.json",
|
||||
"type-discipline-budget.json",
|
||||
|
|
@ -182,12 +181,15 @@ def regressions_for(
|
|||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--base", default=DEFAULT_BASE)
|
||||
parser.add_argument("--base", help="Comparison ref (default: origin's current default branch)")
|
||||
parser.add_argument("budgets", nargs="*", help="budget files to check")
|
||||
args = parser.parse_args()
|
||||
from default_branch import resolve_base_ref
|
||||
|
||||
base_ref: Final = resolve_base_ref(args.base, REPO_ROOT)
|
||||
budgets = args.budgets or list(DEFAULT_BUDGETS)
|
||||
|
||||
ref = _merge_base(args.base)
|
||||
ref = _merge_base(base_ref)
|
||||
if not _ref_is_commit(ref):
|
||||
print(
|
||||
f"FAIL: base ref {ref!r} does not resolve to a commit, so the ratchet has nothing "
|
||||
|
|
@ -204,14 +206,14 @@ def main() -> int:
|
|||
if base is None and head is None:
|
||||
continue
|
||||
if base is None:
|
||||
print(f"skip {rel}: new file (no base at {args.base} to ratchet against)")
|
||||
print(f"skip {rel}: new file (no base at {base_ref} to ratchet against)")
|
||||
continue
|
||||
checked.append(rel)
|
||||
regressions.extend(regressions_for(rel, base, head, graduated_selectors(rel)))
|
||||
|
||||
if regressions:
|
||||
print(
|
||||
f"FAIL: budget limit(s) loosened vs base {args.base} (merge-base {ref[:12]}):"
|
||||
f"FAIL: budget limit(s) loosened vs base {base_ref} (merge-base {ref[:12]}):"
|
||||
)
|
||||
for reg in regressions:
|
||||
print(f" {reg.budget} {reg.rule}: {reg.detail}")
|
||||
|
|
@ -223,7 +225,7 @@ def main() -> int:
|
|||
return 1
|
||||
|
||||
suffix = f" ({', '.join(checked)})" if checked else ""
|
||||
print(f"OK: no budget limit increased vs base {args.base}{suffix}")
|
||||
print(f"OK: no budget limit increased vs base {base_ref}{suffix}")
|
||||
return 0
|
||||
|
||||
|
||||
|
|
|
|||
61
scripts/default_branch.py
Normal file
61
scripts/default_branch.py
Normal file
|
|
@ -0,0 +1,61 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
|
||||
def _git(repo_root: Path, *args: str) -> str:
|
||||
try:
|
||||
result: Final = subprocess.run(
|
||||
["git", *args],
|
||||
cwd=repo_root,
|
||||
env={**os.environ, "GIT_TERMINAL_PROMPT": "0"},
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=60,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError) as exc:
|
||||
raise SystemExit(
|
||||
"Cannot verify the base branch against origin. Check remote access, "
|
||||
"or supply an explicit base ref (--base / BASE_REF). "
|
||||
f"Git operation failed: {exc}"
|
||||
) from exc
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def default_branch(repo_root: Path) -> str:
|
||||
output: Final = _git(repo_root, "ls-remote", "--symref", "origin", "HEAD")
|
||||
branches: Final = tuple(
|
||||
line.removeprefix("ref: refs/heads/").removesuffix("\tHEAD")
|
||||
for line in output.splitlines()
|
||||
if line.startswith("ref: refs/heads/") and line.endswith("\tHEAD")
|
||||
)
|
||||
if len(branches) != 1:
|
||||
raise SystemExit("Origin did not advertise a default branch. Supply an explicit base ref (--base / BASE_REF).")
|
||||
_git(repo_root, "check-ref-format", f"refs/heads/{branches[0]}")
|
||||
return branches[0]
|
||||
|
||||
|
||||
def resolve_base_ref(base_ref: str | None, repo_root: Path) -> str:
|
||||
if base_ref:
|
||||
return base_ref
|
||||
branch: Final = default_branch(repo_root)
|
||||
_git(repo_root, "fetch", "--quiet", "origin", f"+refs/heads/{branch}:refs/remotes/origin/{branch}")
|
||||
return f"origin/{branch}"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser: Final = argparse.ArgumentParser(description="Resolve the live default branch of origin.")
|
||||
parser.add_argument("--base", help="Explicit comparison ref; skips default-branch discovery")
|
||||
parser.add_argument("--repo-root", type=Path, default=Path.cwd())
|
||||
parser.add_argument("--branch", action="store_true", help="Print only the default branch name, without fetching")
|
||||
args: Final = parser.parse_args()
|
||||
print(default_branch(args.repo_root) if args.branch else resolve_base_ref(args.base, args.repo_root))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -7,7 +7,7 @@
|
|||
# - anything staged -> scope is the staged files; changed-but-unstaged files
|
||||
# whose checks were skipped are called out
|
||||
# - nothing staged -> scope is the working tree's diff against the merge base
|
||||
# with origin/litellm_internal_staging, untracked files included
|
||||
# with origin's current default branch, untracked files included
|
||||
# The per-area checks:
|
||||
# - litellm/ Python -> `make lint` (test-linting.yml's lint job)
|
||||
# - tests/e2e Python -> `make lint-e2e-basedpyright` (test-linting.yml's e2e type-check step)
|
||||
|
|
@ -33,8 +33,8 @@ set -eu
|
|||
# at a time instead of thrashing the machine. The wrapper exports
|
||||
# LITELLM_GATE_SLOT_HELD, so this re-exec happens exactly once and everything this
|
||||
# script spawns (make lint, the budget gates) skips its own acquisition.
|
||||
script_dir=$(python3 -c 'import os, sys; print(os.path.dirname(os.path.realpath(sys.argv[1])))' "$0")
|
||||
if [ -z "${LITELLM_GATE_SLOT_HELD:-}" ]; then
|
||||
script_dir=$(python3 -c 'import os, sys; print(os.path.dirname(os.path.realpath(sys.argv[1])))' "$0")
|
||||
exec python3 "$script_dir/gate_slot_lock.py" "$0" "$@"
|
||||
fi
|
||||
|
||||
|
|
@ -65,20 +65,24 @@ untracked=$(git ls-files --others --exclude-standard)
|
|||
if [ -n "$staged" ]; then
|
||||
scope=$staged
|
||||
else
|
||||
git fetch --quiet origin litellm_internal_staging 2>/dev/null || true
|
||||
merge_base=$(git merge-base origin/litellm_internal_staging HEAD 2>/dev/null) || {
|
||||
echo "check: cannot resolve the merge base with origin/litellm_internal_staging." >&2
|
||||
echo " Fix: git fetch origin litellm_internal_staging" >&2
|
||||
base_ref=$(python3 "$script_dir/default_branch.py" --base "${BASE_REF:-}") || {
|
||||
echo "check: FAIL"
|
||||
exit 1
|
||||
}
|
||||
export BASE_REF="$base_ref"
|
||||
merge_base=$(git merge-base "$base_ref" HEAD 2>/dev/null) || {
|
||||
echo "check: cannot resolve the merge base with $base_ref." >&2
|
||||
echo " Fix: fetch the base ref and provide BASE_REF=<ref>" >&2
|
||||
echo "check: FAIL"
|
||||
exit 1
|
||||
}
|
||||
scope=$(printf '%s\n' "$(git diff --name-only --diff-filter=ACMRD "$merge_base")" "$untracked" | sed '/^$/d' | sort -u)
|
||||
if [ -z "$scope" ]; then
|
||||
echo "check: nothing to check (no staged files, no working-tree changes, no branch changes vs origin/litellm_internal_staging)"
|
||||
echo "check: nothing to check (no staged files, no working-tree changes, no branch changes vs $base_ref)"
|
||||
echo "check: PASS"
|
||||
exit 0
|
||||
fi
|
||||
echo "check: nothing staged; scoping to the working tree's diff against the merge base with origin/litellm_internal_staging:"
|
||||
echo "check: nothing staged; scoping to the working tree's diff against the merge base with $base_ref:"
|
||||
printf '%s\n' "$scope" | sed 's/^/ /'
|
||||
fi
|
||||
|
||||
|
|
|
|||
|
|
@ -24,7 +24,6 @@ REPO_ROOT = Path(__file__).resolve().parent.parent
|
|||
STRICT_CONFIG = REPO_ROOT / "ruff-strict.toml"
|
||||
BUDGET_PATH = REPO_ROOT / "ruff-strict-budget.json"
|
||||
TARGET = "litellm"
|
||||
DEFAULT_BASE = "origin/litellm_internal_staging"
|
||||
|
||||
_HUNK = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@")
|
||||
|
||||
|
|
@ -193,7 +192,7 @@ def ratcheted_budget(budget: dict, current: dict, base: dict) -> dict:
|
|||
}
|
||||
|
||||
|
||||
def cmd_update(base_ref: str = DEFAULT_BASE) -> None:
|
||||
def cmd_update(base_ref: str) -> None:
|
||||
"""Ratchet each rule's limit down by the violations this branch fixed.
|
||||
|
||||
The working-tree count is compared against a ruff pass over a detached
|
||||
|
|
@ -212,13 +211,15 @@ def cmd_update(base_ref: str = DEFAULT_BASE) -> None:
|
|||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--base", default=DEFAULT_BASE)
|
||||
parser.add_argument("--base", help="Comparison ref (default: origin's current default branch)")
|
||||
parser.add_argument("--update", action="store_true")
|
||||
args = parser.parse_args()
|
||||
from default_branch import resolve_base_ref
|
||||
from gate_slot_lock import held_slot
|
||||
|
||||
base_ref: Final = resolve_base_ref(args.base, REPO_ROOT)
|
||||
with held_slot():
|
||||
cmd_update(args.base) if args.update else cmd_check(args.base)
|
||||
cmd_update(base_ref) if args.update else cmd_check(base_ref)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ immediately. ``--update`` ratchets a limit down by the violations fixed relative
|
|||
to ``--base``, so the ceilings only ever fall. Base counts are measured with the
|
||||
*current* checker, so a rule introduced on this branch is counted at the base too
|
||||
and ratchets like every other one. The ratchet runs as a scheduled automation
|
||||
against litellm_internal_staging, not on PR branches, so concurrent PRs never
|
||||
against the repository's default branch, not on PR branches, so concurrent PRs never
|
||||
race to edit the same limit.
|
||||
|
||||
The deliberate difference from its sibling: this gate has no headroom anywhere.
|
||||
|
|
@ -43,7 +43,6 @@ REPO_ROOT: Final = Path(__file__).resolve().parent.parent
|
|||
CHECKER: Final = REPO_ROOT / "scripts" / "check_test_quality.py"
|
||||
BUDGET_PATH: Final = REPO_ROOT / "test-quality-budget.json"
|
||||
TARGET: Final = "tests"
|
||||
DEFAULT_BASE: Final = "origin/litellm_internal_staging"
|
||||
TERMINATION_SIGNALS: Final = (signal.SIGTERM, signal.SIGHUP)
|
||||
|
||||
_HUNK: Final = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@", re.MULTILINE)
|
||||
|
|
@ -240,7 +239,7 @@ def ratcheted_budget(
|
|||
})
|
||||
|
||||
|
||||
def cmd_update(base_ref: str = DEFAULT_BASE) -> None:
|
||||
def cmd_update(base_ref: str) -> None:
|
||||
"""Ratchet each rule's limit down by the violations this branch fixed."""
|
||||
budget: Final = json.loads(BUDGET_PATH.read_text())
|
||||
base_point: Final = resolve_base_point(base_ref)
|
||||
|
|
@ -264,19 +263,20 @@ def cmd_seed() -> None:
|
|||
|
||||
def main() -> None:
|
||||
parser: Final = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--base", default=DEFAULT_BASE)
|
||||
parser.add_argument("--base", help="Comparison ref (default: origin's current default branch)")
|
||||
parser.add_argument("--update", action="store_true")
|
||||
parser.add_argument("--seed", action="store_true")
|
||||
args: Final = parser.parse_args()
|
||||
from default_branch import resolve_base_ref
|
||||
from gate_slot_lock import held_slot
|
||||
|
||||
with held_slot():
|
||||
if args.seed:
|
||||
cmd_seed()
|
||||
elif args.update:
|
||||
cmd_update(args.base)
|
||||
cmd_update(resolve_base_ref(args.base, REPO_ROOT))
|
||||
else:
|
||||
cmd_check(args.base)
|
||||
cmd_check(resolve_base_ref(args.base, REPO_ROOT))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
|
|
|||
|
|
@ -71,7 +71,6 @@ REPO_ROOT = Path(__file__).resolve().parent.parent
|
|||
BUDGET_PATH = REPO_ROOT / "basedpyright-code-budget.json"
|
||||
PYRIGHT_CONFIG = REPO_ROOT / "pyrightconfig.json"
|
||||
UV_LOCK = REPO_ROOT / "uv.lock"
|
||||
DEFAULT_BASE = "origin/litellm_internal_staging"
|
||||
CACHE_FILE_PREFIX = "basedpyright-base-"
|
||||
CACHE_KEEP_ENTRIES = 8
|
||||
ARTIFACT_NAME_PREFIX = "basedpyright-counts-"
|
||||
|
|
@ -578,7 +577,7 @@ def ratcheted_budget(
|
|||
}
|
||||
|
||||
|
||||
def cmd_update(current: Mapping[str, int], base_ref: str = DEFAULT_BASE) -> None:
|
||||
def cmd_update(current: Mapping[str, int], base_ref: str) -> None:
|
||||
"""Ratchet each rule's limit down by the errors this branch fixed.
|
||||
|
||||
`current` is the working-tree count; the reference count comes
|
||||
|
|
@ -666,12 +665,14 @@ def cmd_check(head: Mapping[str, int], base_ref: str) -> None:
|
|||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--base", default=DEFAULT_BASE)
|
||||
parser.add_argument("--base", help="Comparison ref (default: origin's current default branch)")
|
||||
parser.add_argument("--update", action="store_true")
|
||||
parser.add_argument("--emit-counts-dir", type=Path)
|
||||
args = parser.parse_args()
|
||||
from default_branch import resolve_base_ref
|
||||
from gate_slot_lock import held_slot
|
||||
|
||||
base_ref: Final = None if args.emit_counts_dir is not None else resolve_base_ref(args.base, REPO_ROOT)
|
||||
with held_slot():
|
||||
ensure_typecheck_env()
|
||||
head = count_basedpyright(run_basedpyright())
|
||||
|
|
@ -679,10 +680,8 @@ def main() -> None:
|
|||
cmd_emit_counts(
|
||||
head, args.emit_counts_dir, _run(["git", "rev-parse", "HEAD"]).strip()
|
||||
)
|
||||
elif args.update:
|
||||
cmd_update(head, args.base)
|
||||
else:
|
||||
cmd_check(head, args.base)
|
||||
elif base_ref is not None:
|
||||
cmd_update(head, base_ref) if args.update else cmd_check(head, base_ref)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
|
|
|||
|
|
@ -44,7 +44,6 @@ REPO_ROOT = Path(__file__).resolve().parent.parent
|
|||
CHECKER = REPO_ROOT / "scripts" / "check_type_discipline.py"
|
||||
BUDGET_PATH = REPO_ROOT / "type-discipline-budget.json"
|
||||
TARGET = "litellm"
|
||||
DEFAULT_BASE = "origin/litellm_internal_staging"
|
||||
|
||||
_HUNK = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@")
|
||||
_LINE = re.compile(r"^(?P<file>.+?):(?P<line>\d+): (?P<code>LIT\d+) ")
|
||||
|
|
@ -239,7 +238,7 @@ def _base_budget_rules(base_point: str) -> frozenset:
|
|||
return frozenset(json.loads(proc.stdout))
|
||||
|
||||
|
||||
def cmd_update(base_ref: str = DEFAULT_BASE) -> None:
|
||||
def cmd_update(base_ref: str) -> None:
|
||||
"""Ratchet each rule's limit down by the violations this branch fixed.
|
||||
|
||||
The working-tree count is compared against a checker pass over a detached
|
||||
|
|
@ -264,13 +263,15 @@ def cmd_update(base_ref: str = DEFAULT_BASE) -> None:
|
|||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--base", default=DEFAULT_BASE)
|
||||
parser.add_argument("--base", help="Comparison ref (default: origin's current default branch)")
|
||||
parser.add_argument("--update", action="store_true")
|
||||
args = parser.parse_args()
|
||||
from default_branch import resolve_base_ref
|
||||
from gate_slot_lock import held_slot
|
||||
|
||||
base_ref: Final = resolve_base_ref(args.base, REPO_ROOT)
|
||||
with held_slot():
|
||||
cmd_update(args.base) if args.update else cmd_check(args.base)
|
||||
cmd_update(base_ref) if args.update else cmd_check(base_ref)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
|
|
|||
|
|
@ -79,7 +79,7 @@ Before publishing to the Terraform Registry:
|
|||
|
||||
## What a change needs
|
||||
|
||||
1. **Land it in `BerriAI/litellm`.** Open a PR against `litellm_internal_staging` with the source change and a `CHANGELOG.md` entry under `[Unreleased]`. CI runs `gofmt`, `go vet`, build, tests and the endpoint-drift audit. A change that breaks existing configurations or state must say so in the changelog: the version number cannot signal it any more
|
||||
1. **Land it in `BerriAI/litellm`.** Open a PR against the repository's current default branch with the source change and a `CHANGELOG.md` entry under `[Unreleased]`. CI runs `gofmt`, `go vet`, build, tests and the endpoint-drift audit. A change that breaks existing configurations or state must say so in the changelog: the version number cannot signal it any more
|
||||
2. **Wait for the next LiteLLM release.** The nightly dev release carries it within a day; it reaches a stable version on the next stable cut
|
||||
3. **Verify** (optional): the version appears at https://registry.terraform.io/providers/BerriAI/litellm and https://github.com/BerriAI/terraform-provider-litellm/releases. If the tag is on the mirror but there is no release, the goreleaser run failed: https://github.com/BerriAI/terraform-provider-litellm/actions
|
||||
|
||||
|
|
|
|||
212
tests/test_litellm/test_default_branch.py
Normal file
212
tests/test_litellm/test_default_branch.py
Normal file
|
|
@ -0,0 +1,212 @@
|
|||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT: Final = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def _git(repo: Path, *args: str) -> str:
|
||||
return subprocess.run(["git", *args], cwd=repo, check=True, capture_output=True, text=True).stdout.strip()
|
||||
|
||||
|
||||
def _commit(repo: Path, message: str) -> None:
|
||||
_git(repo, "add", ".")
|
||||
_git(repo, "-c", "user.name=Test", "-c", "user.email=test@example.com", "commit", "-qm", message)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def remote_and_clone(tmp_path: Path) -> tuple[Path, Path]:
|
||||
seed: Final = tmp_path / "seed"
|
||||
seed.mkdir()
|
||||
_git(seed, "init", "-q", "-b", "litellm_internal_staging")
|
||||
(seed / "scripts").mkdir()
|
||||
for name in (
|
||||
"default_branch.py",
|
||||
"budget_ratchet_check.py",
|
||||
"ruff_strict_gate.py",
|
||||
"type_discipline_gate.py",
|
||||
"test_quality_gate.py",
|
||||
"type_check_gate.py",
|
||||
"gate_slot_lock.py",
|
||||
):
|
||||
shutil.copyfile(ROOT / "scripts" / name, seed / "scripts" / name)
|
||||
shutil.copyfile(ROOT / "Makefile", seed / "Makefile")
|
||||
(seed / "litellm").mkdir()
|
||||
(seed / "litellm" / "example.py").write_text("value = 0\n")
|
||||
(seed / "ruff-strict-budget.json").write_text('{"C901": {"limit": 1}}\n')
|
||||
_commit(seed, "staging base")
|
||||
_git(seed, "checkout", "-qb", "main")
|
||||
(seed / "litellm" / "example.py").write_text("value = 1\n")
|
||||
(seed / "ruff-strict-budget.json").write_text('{"C901": {"limit": 0}}\n')
|
||||
_commit(seed, "main base")
|
||||
remote: Final = tmp_path / "remote.git"
|
||||
_git(tmp_path, "clone", "-q", "--bare", str(seed), str(remote))
|
||||
_git(remote, "symbolic-ref", "HEAD", "refs/heads/litellm_internal_staging")
|
||||
repo: Final = tmp_path / "clone"
|
||||
_git(tmp_path, "clone", "-q", "--single-branch", str(remote), str(repo))
|
||||
return remote, repo
|
||||
|
||||
|
||||
def _resolve(repo: Path, *args: str) -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.run(
|
||||
[sys.executable, str(ROOT / "scripts" / "default_branch.py"), *args],
|
||||
cwd=repo,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
def _make(repo: Path, target: str, *args: str) -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.run(
|
||||
["make", target, "LINT_DEP_INSTALL=", "LINT_DEP_BASE=", *args],
|
||||
cwd=repo,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
env={key: value for key, value in os.environ.items() if key != "BASE_REF"},
|
||||
)
|
||||
|
||||
|
||||
def test_existing_single_branch_clone_follows_remote_switch(remote_and_clone: tuple[Path, Path]) -> None:
|
||||
remote, repo = remote_and_clone
|
||||
before: Final = _resolve(repo)
|
||||
assert before.returncode == 0, before.stderr
|
||||
assert before.stdout.strip() == "origin/litellm_internal_staging"
|
||||
_git(remote, "symbolic-ref", "HEAD", "refs/heads/main")
|
||||
after: Final = _resolve(repo)
|
||||
assert after.returncode == 0, after.stderr
|
||||
assert after.stdout.strip() == "origin/main"
|
||||
assert _git(repo, "rev-parse", "origin/main") == _git(remote, "rev-parse", "main")
|
||||
assert _git(repo, "symbolic-ref", "refs/remotes/origin/HEAD").endswith("/litellm_internal_staging")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("missing_head", [False, True])
|
||||
def test_unverifiable_default_never_uses_cached_head(
|
||||
remote_and_clone: tuple[Path, Path],
|
||||
missing_head: bool,
|
||||
) -> None:
|
||||
remote, repo = remote_and_clone
|
||||
if missing_head:
|
||||
_git(remote, "symbolic-ref", "HEAD", "refs/heads/missing")
|
||||
else:
|
||||
_git(repo, "remote", "set-url", "origin", str(remote / "missing"))
|
||||
result: Final = _resolve(repo)
|
||||
assert result.returncode != 0
|
||||
assert not result.stdout
|
||||
assert "explicit base ref" in result.stderr
|
||||
checked: Final = _make(repo, "lint-format-check-changed")
|
||||
assert checked.returncode != 0
|
||||
assert "No changed" not in checked.stdout
|
||||
|
||||
|
||||
@pytest.mark.parametrize("base_ref", ["HEAD", "origin/litellm_internal_staging"])
|
||||
def test_explicit_base_works_without_remote_access(
|
||||
remote_and_clone: tuple[Path, Path],
|
||||
base_ref: str,
|
||||
) -> None:
|
||||
remote, repo = remote_and_clone
|
||||
_git(repo, "remote", "set-url", "origin", str(remote / "missing"))
|
||||
result: Final = _resolve(repo, "--base", base_ref)
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert result.stdout.strip() == base_ref
|
||||
checked: Final = _make(repo, "lint-format-check-changed", f"BASE_REF={base_ref}")
|
||||
assert checked.returncode == 0, checked.stderr
|
||||
assert "No changed litellm Python files" in checked.stdout
|
||||
|
||||
|
||||
def test_budget_ratchet_compares_against_new_default(remote_and_clone: tuple[Path, Path]) -> None:
|
||||
remote, repo = remote_and_clone
|
||||
_git(remote, "symbolic-ref", "HEAD", "refs/heads/main")
|
||||
resolved: Final = _resolve(repo)
|
||||
assert resolved.returncode == 0, resolved.stderr
|
||||
_git(repo, "checkout", "-qb", "litellm_feature", "origin/main")
|
||||
(repo / "ruff-strict-budget.json").write_text('{"C901": {"limit": 1}}\n')
|
||||
command: Final = [sys.executable, "scripts/budget_ratchet_check.py"]
|
||||
checked: Final = subprocess.run(command, cwd=repo, capture_output=True, text=True, check=False)
|
||||
assert checked.returncode == 1
|
||||
assert "limit raised 0 -> 1" in checked.stdout
|
||||
assert "base origin/main" in checked.stdout
|
||||
overridden: Final = subprocess.run(
|
||||
[*command, "--base", "origin/litellm_internal_staging"],
|
||||
cwd=repo,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
assert overridden.returncode == 0, overridden.stdout + overridden.stderr
|
||||
|
||||
|
||||
def _freshness(repo: Path, *args: str) -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-c",
|
||||
"import sys; from pathlib import Path; "
|
||||
"from ci_cd.run_migration import _check_branch_freshness; "
|
||||
"_check_branch_freshness(Path(sys.argv[1]), sys.argv[2] if len(sys.argv) > 2 else None)",
|
||||
str(repo),
|
||||
*args,
|
||||
],
|
||||
cwd=ROOT,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
def test_migration_freshness_refuses_stale_branch_after_switch(remote_and_clone: tuple[Path, Path]) -> None:
|
||||
remote, repo = remote_and_clone
|
||||
before: Final = _freshness(repo)
|
||||
assert before.returncode == 0, before.stderr
|
||||
assert "Branch freshness OK" in before.stdout
|
||||
_git(remote, "symbolic-ref", "HEAD", "refs/heads/main")
|
||||
after: Final = _freshness(repo)
|
||||
assert after.returncode == 3
|
||||
assert "1 commit(s) behind origin/main" in after.stderr
|
||||
overridden: Final = _freshness(repo, "litellm_internal_staging")
|
||||
assert overridden.returncode == 0, overridden.stderr
|
||||
_git(repo, "merge", "--ff-only", "origin/main")
|
||||
updated: Final = _freshness(repo)
|
||||
assert updated.returncode == 0, updated.stderr
|
||||
assert "up to date with origin/main" in updated.stdout
|
||||
|
||||
|
||||
def test_migration_freshness_refuses_unavailable_remote(remote_and_clone: tuple[Path, Path]) -> None:
|
||||
remote, repo = remote_and_clone
|
||||
_git(repo, "remote", "set-url", "origin", str(remote / "missing"))
|
||||
result: Final = _freshness(repo)
|
||||
assert result.returncode == 3
|
||||
assert "Could not discover origin's default branch" in result.stderr
|
||||
explicit: Final = _freshness(repo, "litellm_internal_staging")
|
||||
assert explicit.returncode == 3
|
||||
assert "git fetch origin litellm_internal_staging" in explicit.stderr
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"gate",
|
||||
[
|
||||
"budget_ratchet_check",
|
||||
"ruff_strict_gate",
|
||||
"type_discipline_gate",
|
||||
"test_quality_gate",
|
||||
"type_check_gate",
|
||||
],
|
||||
)
|
||||
def test_each_gate_refuses_an_unverifiable_default(remote_and_clone: tuple[Path, Path], gate: str) -> None:
|
||||
remote, repo = remote_and_clone
|
||||
_git(repo, "remote", "set-url", "origin", str(remote / "missing"))
|
||||
result: Final = subprocess.run(
|
||||
[sys.executable, f"scripts/{gate}.py"],
|
||||
cwd=repo,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
assert result.returncode != 0
|
||||
assert "Cannot verify the base branch against origin" in result.stderr
|
||||
|
|
@ -159,12 +159,12 @@ def _commit_all(repo: Path, message: str) -> None:
|
|||
)
|
||||
|
||||
|
||||
def _set_base_ref(repo: Path) -> None:
|
||||
subprocess.run(
|
||||
["git", "update-ref", "refs/remotes/origin/litellm_internal_staging", "HEAD"],
|
||||
cwd=repo,
|
||||
check=True,
|
||||
)
|
||||
def _set_base_ref(repo: Path, branch: str = "litellm_internal_staging") -> None:
|
||||
remote = repo.parent / "remote.git"
|
||||
subprocess.run(["git", "clone", "-q", "--bare", str(repo), str(remote)], check=True)
|
||||
subprocess.run(["git", "update-ref", f"refs/heads/{branch}", "HEAD"], cwd=remote, check=True)
|
||||
subprocess.run(["git", "symbolic-ref", "HEAD", f"refs/heads/{branch}"], cwd=remote, check=True)
|
||||
subprocess.run(["git", "remote", "add", "origin", str(remote)], cwd=repo, check=True)
|
||||
|
||||
|
||||
def _stage_file(repo: Path, relative: str, body: str) -> None:
|
||||
|
|
@ -174,10 +174,11 @@ def _stage_file(repo: Path, relative: str, body: str) -> None:
|
|||
subprocess.run(["git", "add", relative], cwd=repo, check=True)
|
||||
|
||||
|
||||
def test_nothing_staged_scopes_to_working_tree_diff_and_runs_checks(tmp_path: Path) -> None:
|
||||
@pytest.mark.parametrize("branch", ["litellm_internal_staging", "main"])
|
||||
def test_nothing_staged_scopes_to_working_tree_diff_and_runs_checks(tmp_path: Path, branch: str) -> None:
|
||||
repo, bin_dir = _sandbox(tmp_path)
|
||||
_commit_all(repo, "base")
|
||||
_set_base_ref(repo)
|
||||
_set_base_ref(repo, branch)
|
||||
(repo / "litellm" / "foo.py").write_text("x = 2\n")
|
||||
proc = _run(repo, bin_dir, {})
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
|
|
@ -261,8 +262,8 @@ def test_nothing_staged_without_a_base_ref_fails_with_a_fetch_hint(tmp_path: Pat
|
|||
_commit_all(repo, "base")
|
||||
proc = _run(repo, bin_dir, {})
|
||||
assert proc.returncode == 1
|
||||
assert "cannot resolve the merge base" in proc.stdout
|
||||
assert "git fetch origin litellm_internal_staging" in proc.stdout
|
||||
assert "Cannot verify the base branch against origin" in proc.stdout
|
||||
assert "explicit base ref" in proc.stdout
|
||||
assert "check: FAIL" in proc.stdout
|
||||
|
||||
|
||||
|
|
@ -622,3 +623,28 @@ def test_failing_run_ends_with_a_fail_verdict(tmp_path: Path) -> None:
|
|||
assert proc.returncode == 1
|
||||
assert "check: FAIL" in proc.stdout
|
||||
assert "check: PASS" not in proc.stdout
|
||||
|
||||
|
||||
|
||||
def test_explicit_base_scopes_offline_without_a_remote(tmp_path: Path) -> None:
|
||||
repo, bin_dir = _sandbox(tmp_path)
|
||||
_commit_all(repo, "base")
|
||||
(repo / "litellm" / "foo.py").write_text("x = 2\n")
|
||||
proc = _run(repo, bin_dir, {"BASE_REF": "HEAD"})
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
assert "merge base with HEAD" in proc.stdout
|
||||
assert "linting Python" in proc.stdout
|
||||
|
||||
|
||||
def test_symlinked_hook_can_resolve_default_branch(tmp_path: Path) -> None:
|
||||
repo, bin_dir = _sandbox(tmp_path)
|
||||
_commit_all(repo, "base")
|
||||
_set_base_ref(repo, "main")
|
||||
hook = repo / ".git" / "hooks" / "pre-commit"
|
||||
hook.symlink_to(SCRIPT)
|
||||
proc = subprocess.run(
|
||||
[str(hook)], cwd=repo, capture_output=True, text=True,
|
||||
env=_env(repo, bin_dir, {}), timeout=120,
|
||||
)
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
assert "no branch changes vs origin/main" in proc.stdout
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue