litellm/tests/claude_code
mateo-berri bb60422e7d compat-matrix: replace publisher.py + resolver.py with a bash run_daily.sh
The previous iteration of this PR ported the populator to a Python
module (`publisher.py`) with 8 unit-tested pure helpers for branch
naming and PR-body rendering. After the docker code came out, the
GitHub App auth came out, and the worktree-vs-tempdir decision was
made, what was left was: 'git fetch + git checkout + uv sync + start
a subprocess + run pytest + clone docs + git commit + gh pr create'.
That's a bash script.

This commit replaces 800 lines of Python (publisher.py + resolver.py +
their unit tests) with a 297-line run_daily.sh and a 48-line
build_matrix.py whose only job is to be importable Python that can
call into the matrix_builder we already have. Net deletion: -822 lines.

Removed
-------

  * tests/claude_code/publisher.py — the full Python orchestrator.
    Every code path it had is now in run_daily.sh.
  * tests/claude_code/resolver.py — the GitHub Releases v*-stable
    resolver. Replaced by ~10 lines of jq inside run_daily.sh.
  * tests/claude_code/_publisher_unit_tests/ — both test files. The
    pure helpers they covered (commit message, file allowlist, branch
    name, PR title/body) only existed because publisher.py was Python.
    The bash equivalents are short heredoc strings.

Added
-----

  * tests/claude_code/cron_vm/run_daily.sh — the actual cron job,
    structured as numbered phases (resolve / worktree / proxy /
    pytest / build / publish) so journalctl output is readable.
  * tests/claude_code/cron_vm/build_matrix.py — a 48-line CLI that
    calls the existing matrix_builder.build_from_paths. Kept in
    Python because the builder itself is Python and well-tested.

Modified
--------

  * tests/claude_code/cron_vm/litellm-compat-matrix.service —
    ExecStart now invokes run_daily.sh instead of
    'python -m tests.claude_code.publisher'.
  * tests/claude_code/cron_vm/README.md — updated layout table,
    file roles, and operating commands to match.

Why this is the right shape
---------------------------

  * The failure mode at 06:00 UTC is 'read journalctl, see the literal
    failing command with its + prefix, copy-paste it into a shell to
    reproduce'. Bash makes that immediate; Python's subprocess.run
    output looks similar but the surrounding orchestration is harder
    to step through interactively.
  * Every operation the script does is already a shell command (git,
    uv, gh, jq, curl, pytest). The Python wrapper was translating
    between argv arrays and back.
  * The two pieces that genuinely benefit from being in a typed
    language are matrix_builder (already Python) and the resolver's
    semver sort (now done in jq, with the version_key tuple sort
    inline). 'Already Python' wins, 'tiny jq pipeline' wins.

What's preserved
----------------

  * Idempotency: same (litellm, claude, UTC date) -> same branch ->
    same PR. force-with-lease push, gh-pr-create no-op-on-exists.
  * Byte-identical-JSON early return (git diff --cached --quiet).
  * Per-feature status table in the PR body (jq pipeline mirroring
    the Python pr_body_for_matrix logic).
  * Persistent worktree approach so disk doesn't grow unboundedly.
  * Proxy bound to :4100 to avoid colliding with a developer's :4000.
  * SKIP_PUBLISH=1 and PYTEST_K=... operator escape hatches.
2026-05-06 23:27:14 +00:00
..
_builder_unit_tests RALPH: fix compat matrix - Azure now hosts Claude via Microsoft Foundry 2026-05-06 23:27:14 +00:00
_driver_unit_tests style: reformat test files 2026-05-06 23:27:14 +00:00
_pr_gate_unit_tests RALPH: compat matrix slice 3 - wire PR gate in CircleCI (#26479, PRD #26476) 2026-05-06 23:27:05 +00:00
basic_messaging_non_streaming style: reformat test files 2026-05-06 23:27:14 +00:00
basic_messaging_streaming style: reformat test files 2026-05-06 23:27:14 +00:00
cron_vm compat-matrix: replace publisher.py + resolver.py with a bash run_daily.sh 2026-05-06 23:27:14 +00:00
extended_thinking style: reformat test files 2026-05-06 23:27:14 +00:00
prompt_caching_5m style: reformat test files 2026-05-06 23:27:14 +00:00
tool_use style: reformat test files 2026-05-06 23:27:14 +00:00
vision style: reformat test files 2026-05-06 23:27:14 +00:00
__init__.py RALPH: tracer-bullet for Claude Code compatibility matrix (#26477, PRD #26476) 2026-05-06 23:27:05 +00:00
cli_driver.py style: reformat test files 2026-05-06 23:27:14 +00:00
conftest.py RALPH: tracer-bullet for Claude Code compatibility matrix (#26477, PRD #26476) 2026-05-06 23:27:05 +00:00
manifest.yaml RALPH: compat matrix slice 5 - add remaining 5 v0 features (full 6x5 grid) (#26481, PRD #26476) 2026-05-06 23:27:14 +00:00
matrix_builder.py RALPH: tracer-bullet for Claude Code compatibility matrix (#26477, PRD #26476) 2026-05-06 23:27:05 +00:00
pr_gate_version_resolver.py RALPH: compat matrix slice 3 - wire PR gate in CircleCI (#26479, PRD #26476) 2026-05-06 23:27:05 +00:00
sample_compatibility-matrix.json RALPH: fix compat matrix - Azure now hosts Claude via Microsoft Foundry 2026-05-06 23:27:14 +00:00
test_config.yaml style: reformat test files 2026-05-06 23:27:14 +00:00