litellm/tests/claude_code/_publisher_unit_tests
mateo-berri 0e65ce1495 compat-matrix: open a docs-repo PR instead of direct-pushing main
The daily Claude Code compatibility-matrix cron has been direct-pushing
`compatibility-matrix.json` to litellm-docs's main branch. Switch to
opening (or updating) a pull request so docs maintainers can review each
matrix update before it ships to readers.

Behavioural changes
-------------------

publisher.publish() now:
  * checks out a deterministic head branch
    (`compat-matrix/<litellm>-<claude>-<UTC-date>`) before staging the
    JSON, instead of committing on top of the docs branch directly;
  * `git push --force-with-lease` so a same-day rerun updates the
    existing branch (and therefore the existing PR), without
    overwriting any docs-maintainer fixup commit on the same branch;
  * shells out to `gh pr create` against `docs_repo` with a
    title/body that surfaces the resolved versions and a per-feature
    status summary, so reviewers can triage from the inbox;
  * treats 'a pull request for branch ... already exists' as success,
    so two cron runs on the same day produce one PR, not two.

Idempotency contract
--------------------

  * Same (litellm_version, claude_code_version, UTC date) -> same
    branch -> same PR. Verified by the new
    `test_pr_branch_name_is_deterministic_per_inputs` /
    `...changes_when_any_component_changes` tests.
  * Byte-identical JSON to the docs branch -> early-return before
    push, same as the previous direct-push path.
  * Empty version inputs are rejected up front so two distinct PRs
    can never silently collapse onto one branch.

Tests
-----

  * 8 new tests in `_publisher_unit_tests/test_publisher.py` cover
    `pr_branch_name`, `pr_title_for_matrix`, and `pr_body_for_matrix`
    (determinism, content, ordering, missing-provider rectangularity,
    empty-input rejection).
  * Existing 7 `commit_message_for_matrix` /
    `docker_image_for_tag` / `select_files_to_commit` tests are
    unchanged and still pass.

Workflow
--------

`.github/workflows/claude_code_compat_matrix.yml` updates only the
header doc comment to reflect that the GitHub App now needs
`pull-requests: write` in addition to `contents: write`. `gh` is
preinstalled on `ubuntu-latest` (also used by
`auto_update_price_and_context_window.yml`), so no install step is
needed.

Operator action required (one-time)
-----------------------------------

The compat-matrix GitHub App installation on `BerriAI/litellm-docs`
needs `pull-requests: write` added to its installation permissions
before the next cron run. Without it, the new `gh pr create` call
will fail with a 403; `compat-results.json` and
`compatibility-matrix.json` will still upload as workflow artifacts
for debugging.
2026-05-06 23:27:14 +00:00
..
__init__.py RALPH: compat matrix slice 4 - daily cron VM publishes matrix to docs (#26480, PRD #26476) 2026-05-06 23:27:14 +00:00
test_publisher.py compat-matrix: open a docs-repo PR instead of direct-pushing main 2026-05-06 23:27:14 +00:00
test_resolver.py RALPH: compat matrix slice 4 - daily cron VM publishes matrix to docs (#26480, PRD #26476) 2026-05-06 23:27:14 +00:00