docs(ci): record why the runner box must not restart services mid-job

A run spans ~15h and apt-daily-upgrade.timer fires daily around 06:34, so
every scheduled run crosses it. On 2026-08-02 unattended-upgrades upgraded
openssl at 07:54:02 and needrestart restarted the Actions runner five
seconds later. The job went to Canceled 14s after that, and a cancelled
job skips even `if: always()` — so the evidence artifact died with it,
which is the one outcome the rest of this workflow's budget nesting exists
to prevent.

This is a plausible contributor to the unexplained mid-run failures in the
July dispatch cluster, none of which left an artifact behind either.

The box config itself is applied out-of-band like the rest of the instance
setup; the checklist now carries the requirement so a rebuilt box does not
silently reintroduce it.
This commit is contained in:
Gergo Magyar 2026-08-02 08:25:27 +00:00
parent 2fbd5ee515
commit a5251d6b08

View file

@ -41,6 +41,18 @@
# most weekly. Revisit if run frequency increases or the threat model
# changes; stopping already bounds the exposure window to the job's own
# runtime on 1 day out of 7.
# [ ] Stop the runner box from restarting services under a running job. A run
# spans ~15h and apt-daily-upgrade.timer fires daily (~06:34), so every
# scheduled run crosses it. On 2026-08-02 unattended-upgrades upgraded
# openssl at 07:54:02 and needrestart restarted the Actions runner five
# seconds later: the job went to Canceled, and a cancelled job skips even
# `if: always()`, so the evidence artifact died with it. Keep installing
# updates, but never let them restart services here:
# /etc/needrestart/conf.d/90-gitnexus-evolution.conf
# $nrconf{restart} = 'l';
# A drop-in, so a needrestart package upgrade cannot clobber it. Nothing
# is left unpatched in practice — the box is stopped between runs, so the
# new binaries take effect at the next boot.
# [x] Run workflow_dispatch once and confirm: containment preflight passes,
# the benchmark completes inside the job timeout, the results artifact
# uploads, and a promotion (if any) opens a well-formed PR. Run