ci(mutation): emit per-survivor diffs to run-page summary + artifact

The previous artifact only contained `mutmut results` text (which in
mutmut 3.x lists survivor names but not the actual mutations). Adds:

- `mutmut export-cicd-stats` to produce mutmut-cicd-stats.json with the
  killed/survived/total scoreboard.
- `mutmut show <name>` per surviving mutant to capture each mutation as
  a unified diff.
- A `mutmut-report.md` that combines summary + run-progress tail +
  per-survivor diffs, written to both the artifact and
  $GITHUB_STEP_SUMMARY (visible on the run page, no download needed).
- Corrected artifact paths: stats files live under mutants/, not the
  project root.
- The trampolined source file from the sandbox so survivors can be
  inspected even outside `mutmut show`.
This commit is contained in:
Ryan Crabbe 2026-05-09 17:46:18 -07:00 committed by Cursor Agent
parent 3bd003c254
commit 238126165f
No known key found for this signature in database

View file

@ -91,13 +91,58 @@ jobs:
# trampolined files are imported instead of the installed copy.
PYTHONPATH: ${{ github.workspace }}/mutants
run: |
set -o pipefail
mkdir -p mutants
uv run --no-sync --with mutmut==3.5.0 mutmut run
uv run --no-sync --with mutmut==3.5.0 mutmut run 2>&1 | tee mutmut-run.log
- name: Show mutmut results
# Build a detailed report: per-survivor diffs from `mutmut show`, plus
# the killed/survived/total summary from `mutmut export-cicd-stats`.
# Written to both an artifact and the run-page summary so results are
# visible without downloading anything.
- name: Generate detailed mutation report
if: always()
run: |
uv run --no-sync --with mutmut==3.5.0 mutmut results | tee mutmut-results.txt
set +e
uv run --no-sync --with mutmut==3.5.0 mutmut export-cicd-stats > /dev/null 2>&1
uv run --no-sync --with mutmut==3.5.0 mutmut results > mutmut-results.txt 2>&1
{
echo "# Mutation Test Report"
echo ""
echo "## Summary"
echo ""
if [ -f mutants/mutmut-cicd-stats.json ]; then
echo '```json'
cat mutants/mutmut-cicd-stats.json
echo '```'
else
echo "(mutmut-cicd-stats.json not generated)"
fi
echo ""
echo "## Run progress (tail)"
echo ""
echo '```'
tail -30 mutmut-run.log || true
echo '```'
echo ""
echo "## Surviving mutants (test gaps)"
echo ""
survivors=$(awk '/survived/ {gsub(/:$/,"",$1); print $1}' mutmut-results.txt)
if [ -z "$survivors" ]; then
echo "_None — every mutant was killed by the test suite._"
else
for m in $survivors; do
echo "### \`$m\`"
echo ""
echo '```diff'
uv run --no-sync --with mutmut==3.5.0 mutmut show "$m" 2>&1 || echo "(mutmut show failed for $m)"
echo '```'
echo ""
done
fi
} > mutmut-report.md
cat mutmut-report.md >> "$GITHUB_STEP_SUMMARY"
- name: Upload mutmut artifacts
if: always()
@ -105,8 +150,11 @@ jobs:
with:
name: mutmut-${{ github.run_id }}-${{ github.run_attempt }}
path: |
mutmut-report.md
mutmut-results.txt
mutmut-stats.json
mutmut-cache.json
mutmut-run.log
mutants/mutmut-stats.json
mutants/mutmut-cicd-stats.json
mutants/litellm/proxy/management_endpoints/router_settings_endpoints.py
if-no-files-found: warn
retention-days: 14