From a5251d6b084cd835856431e98a2051659edde571 Mon Sep 17 00:00:00 2001 From: Gergo Magyar Date: Sun, 2 Aug 2026 08:25:27 +0000 Subject: [PATCH] docs(ci): record why the runner box must not restart services mid-job MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A run spans ~15h and apt-daily-upgrade.timer fires daily around 06:34, so every scheduled run crosses it. On 2026-08-02 unattended-upgrades upgraded openssl at 07:54:02 and needrestart restarted the Actions runner five seconds later. The job went to Canceled 14s after that, and a cancelled job skips even `if: always()` — so the evidence artifact died with it, which is the one outcome the rest of this workflow's budget nesting exists to prevent. This is a plausible contributor to the unexplained mid-run failures in the July dispatch cluster, none of which left an artifact behind either. The box config itself is applied out-of-band like the rest of the instance setup; the checklist now carries the requirement so a rebuilt box does not silently reintroduce it. --- .github/workflows/gitnexus-skill-evolution.yml | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/.github/workflows/gitnexus-skill-evolution.yml b/.github/workflows/gitnexus-skill-evolution.yml index 88b98e879..43e7d098b 100644 --- a/.github/workflows/gitnexus-skill-evolution.yml +++ b/.github/workflows/gitnexus-skill-evolution.yml @@ -41,6 +41,18 @@ # most weekly. Revisit if run frequency increases or the threat model # changes; stopping already bounds the exposure window to the job's own # runtime on 1 day out of 7. +# [ ] Stop the runner box from restarting services under a running job. A run +# spans ~15h and apt-daily-upgrade.timer fires daily (~06:34), so every +# scheduled run crosses it. On 2026-08-02 unattended-upgrades upgraded +# openssl at 07:54:02 and needrestart restarted the Actions runner five +# seconds later: the job went to Canceled, and a cancelled job skips even +# `if: always()`, so the evidence artifact died with it. Keep installing +# updates, but never let them restart services here: +# /etc/needrestart/conf.d/90-gitnexus-evolution.conf +# $nrconf{restart} = 'l'; +# A drop-in, so a needrestart package upgrade cannot clobber it. Nothing +# is left unpatched in practice — the box is stopped between runs, so the +# new binaries take effect at the next boot. # [x] Run workflow_dispatch once and confirm: containment preflight passes, # the benchmark completes inside the job timeout, the results artifact # uploads, and a promotion (if any) opens a well-formed PR. Run