fabro/lib/foundation
Bryan Helmkamp 1688cd5b91
Retry git pushes with a pinned token and record attempt history
Run 01M0DH033P2XSTHAGVBHG6922F completed 2.8 hours of work, then failed
terminally because four consecutive publish pushes hit GitHub's
token-replication lag (404 "Repository not found") — the push path had no
retry, the failure was misclassified as deterministic, and the same
fresh-mint-then-push pattern silently disabled metadata snapshots. This
generalizes the clone retry machinery to pushes and makes attempt detail
durable.

- clone_retry -> git_retry: the classifier's boolean becomes a
  CredentialContext derived from the token snapshot (fresh App tokens retry
  404s as replication lag, mature ones as transient infra, static
  credentials fail fast), and the attempt/backoff limits become a RetryPlan
  with layered optional bounds. Clone behavior is preserved: Docker keeps
  its absolute five-minute deadline, Daytona keeps no deadline.
- Pushes take a scoped CredentialLease before the first attempt: it owns
  the embed mutex for the whole operation, pins the single successful
  resolve, retries only failed resolves, falls back to the last embedded
  token when a mint fails, and force-re-embeds the pinned token once after
  the first auth-shaped failure (drift repair). The margin invariant
  (REFRESH_MARGIN > every push plan's max_elapsed) guarantees the pinned
  token outlives the operation; a unit test asserts it.
- Sandbox::git_push_ref now takes a RetryPlan and returns PushReport /
  PushError with per-attempt records (classification, redacted output tail,
  token generation/provenance/age, credential action, refresh errors).
  Checkpoint pushes use a 90-second budget; the terminal publish push gets
  5 attempts over at most 4 minutes.
- The single durable git.push event per push gains a nested attempts array
  (GitPushAttemptProps, token snapshot flattened to flat fields); stored
  events without it still deserialize. Publish push failures now carry an
  explicit failure category — exhausted transient retries stay
  transient_infra instead of deterministic — plus one bounded cause line
  per attempt and the last successful push time in the message.
- Metadata snapshot degradation records why it degraded: push failures with
  retryable classifications leave the writer eligible to re-probe at each
  later checkpoint, and a successful snapshot clears the degraded state and
  re-arms the warning. Permanent failures keep today's latch.

Plan: .ai/plans/git-push-token-resilience.md (PR 2: items 1, 2, 4, 7 and
the metadata re-probe).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 10:16:10 -04:00
..
build-support refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
fabro-api Add a shared RunSpec test fixture and adopt it 2026-08-19 16:50:37 -04:00
fabro-auth fix(model): prefer MOONSHOT_API_KEY 2026-07-31 09:14:58 -04:00
fabro-client Type the blob-write response hash as fabro_types::BlobHash 2026-08-17 13:56:19 -04:00
fabro-config Remove stale env-interpolation promises from docs 2026-08-17 17:36:53 -04:00
fabro-core refactor(sandbox): harden activation recovery 2026-07-28 15:38:35 -04:00
fabro-db Add SQLite blob store foundation 2026-08-18 17:41:29 -04:00
fabro-dev Merge pull request #676 from fabro-sh/remove-run-agent-permissions 2026-08-01 10:18:45 -04:00
fabro-http refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
fabro-macros refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
fabro-model Remove stale env-interpolation promises from docs 2026-08-17 17:36:53 -04:00
fabro-oauth refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
fabro-options-metadata refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
fabro-proc refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
fabro-redact refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
fabro-static Merge remote-tracking branch 'origin/main' into fix/modal-provider-catalog 2026-08-01 09:32:19 -04:00
fabro-telemetry refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
fabro-template Merge pull request #756 from fabro-sh/refactor/graph-reference-kind 2026-08-18 12:57:32 -04:00
fabro-test Generate the blob-field snapshot filters from a field list 2026-08-17 13:56:19 -04:00
fabro-types Retry git pushes with a pinned token and record attempt history 2026-08-20 10:16:10 -04:00
fabro-util Neutralize runtime for_each labels before display 2026-07-28 19:52:59 -04:00
fabro-vault refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00