claude-skills/audit/engineering-agentic-2026-07/engineering.md

Domain re-audit: engineering/ — delta vs June 2026 + agentic readiness

Audited: 2026-07-03 · 63 distinct skills under engineering/ · Method: full SKILL.md reads, June "Verify" criteria re-run, ~30 script smoke tests (all exit codes checked). Rubric: RUBRIC.md. June baseline: ../newgen-2026-06/engineering.md.

Summary stats

Delta resolution (19 non-KEEP June verdicts):

• RESOLVED: 9 — agent-designer, dependency-auditor, rag-architect, skill-tester, tech-debt-tracker (REWRITEs); command-guide (deleted); release-manager (merged into changelog-generator, hotfix/rollback tables absorbed); engineering-advanced-skills (counters now 37=37=37, paths fixed); universal-scraping-architect (all 3 scripts wired, agent/command rebuilt, layout normalized).
• PARTIALLY-RESOLVED: 6 — migration-architect, observability-designer (CLIs + gates added but textbook bodies never pruned); database-designer (wired, not merged); agent-workflow-designer, api-design-reviewer, runbook-generator.
• STILL-OPEN: 4 — database-schema-designer (no merge, zero scripts, broken seed example at L154 persists), codebase-onboarding, interview-system-designer, claude-coach (all 3 June defects untouched: dup frontmatter keys Name:+name: / 1.0.0+2.9.0, README paste L145–205, unwired classifier).
• Side-asks: 5 orphan plugins now marketplace-registered ✅ · 4 dual-published duplicates (slo/chaos/k8s/flags) still undeduped (byte-identical, diff -rq clean) ❌ · autoresearch evaluator --help-exception sentence never added ❌ · env-secrets-manager dead cross-refs persist ❌ · focused-fix superpowers:* still "REQUIRED SUB-SKILL" ❌.

Agentic-readiness distribution (63 skills): HARNESS-READY 22 · LOOP-CAPABLE 23 · TOOL-ONLY 16 · PROSE-ONLY 2. Weakest dimensions: AR5 loop discipline (caps/stop conditions rare outside v2.4+ skills) and AR1 goal intake (most tool-rich skills accept any input silently).

KEEP spot-checks (~25 contracts re-run): PASS except — ship-gate (category table 84 vs checks.md 89), write-a-skill (fails its own checklist runner: 141 lines vs its <100 rule, exit 1), workflow-builder (--sample exits 1 by design — June criterion wrong, needs one doc sentence), focused-fix (superpowers refs unresolved), env-secrets-manager (all 5 cross-ref paths fail ls). Verified anchors reproduce: slo error-budget 43.20 min, statistical-analyst +1.2pp, tc-tracker rejects planned→deployed (exit 2), commit_linter/mcp_validator --strict exit 1 correctly.

Per-skill table

Scores AR1·AR2·AR3·AR4·AR5·AR6. Class: HR=HARNESS-READY, LC=LOOP-CAPABLE, TO=TOOL-ONLY, PO=PROSE-ONLY. Delta "—" = KEEP verdict holding.

Skill	June	Delta	AR1-6	Tot	Class	Top improvement
skills/agent-designer	REWRITE	RESOLVED	1·2·2·2·1·2	10	HR	Cap the step-4 re-evaluate loop (max 3 pilot re-runs, then escalate)
skills/agent-workflow-designer	OPTIMIZE	PARTIAL	0·1·2·1·0·0	4	TO	Add June-mandated "When NOT to use → workflow-builder" block + JSON-validity gate on scaffolder output
skills/api-design-reviewer	OPTIMIZE	PARTIAL	0·1·2·2·1·1	7	LC	Cut L31-349 textbook REST to references/ (body ≤200); cap lint-fix cycles at 3
skills/api-test-suite-builder	KEEP	—	0·1·2·0·0·0	3	TO	Add gate: generated suite must pass npx vitest run/pytest -x with 0 collection errors; coverage contract per route
skills/browser-automation	KEEP	—	1·1·1·1·1·0	5	TO	Full runnable CLIs in Quick Start; require anti_detection_checker.py exit-0 pre-run; retry cap 3 on 429/403
skills/changelog-generator	KEEP	— (merge done)	1·1·2·2·1·2	9	HR	State retry cap for lint-fix cycle
skills/chaos-engineering	KEEP	dedupe open	2·2·2·2·2·2	12	HR	Deduplicate bundle/standalone copies
skills/ci-cd-pipeline-builder	KEEP	—	0·1·2·1·0·1	5	TO	Require yaml.safe_load exit-0 gate on generated pipeline; intake (platform/targets/branches)
skills/codebase-onboarding	OPTIMIZE	STILL-OPEN	0·1·2·1·0·1	5	TO	Add June-mandated gate: execute every setup command in the doc once, 0 ❌ before done
skills/database-designer	CUT-OR-MERGE	PARTIAL	1·1·2·2·1·1	8	LC	Execute the merge into sql-database-assistant (or add explicit routing); cap analyze-fix at 2 cycles
skills/database-schema-designer	CUT-OR-MERGE	STILL-OPEN	0·1·0·1·0·0	2	PO	Retire per June verdict: migrate RLS block + pitfalls table, delete broken L154 seed example
skills/dependency-auditor	REWRITE	RESOLVED	0·1·2·2·1·1	7	LC	Intake (ecosystem/policy/threshold); cap upgrade-rescan at 2 cycles
skills/engineering-advanced-skills	OPTIMIZE	RESOLVED	0·0·0·0·0·0	0	PO (index by design)	Optionally add "state which skill you loaded and why" routing rule
skills/env-secrets-manager	KEEP	cross-refs STILL-OPEN	0·1·2·1·1·1	6	LC	Fix 5 dead cross-ref paths; make env_auditor.py 0-critical a binding close gate
skills/feature-flags-architect	KEEP	dedupe open	1·2·2·2·2·2	11	HR	Deduplicate copies
skills/focused-fix	KEEP	PARTIAL	2·2·1·2·2·2	11	HR	Reword superpowers:* as optional externals; drop phantom scope skill ref (L308)
skills/full-page-screenshot	KEEP	—	1·1·2·2·1·1	8	LC	Hard gate: file out.png = PNG & height>viewport; stop after 2 --wait increases
skills/git-worktree-manager	KEEP	—	1·1·2·2·1·1	8	LC	Make Validation Checklist a required exit gate with one recovery pass then escalate
skills/interview-system-designer	OPTIMIZE	STILL-OPEN	0·1·1·1·0·0	3	TO	Wire or delete the 3 orphan root-level scripts; relocate out of engineering per June misfit note
skills/kubernetes-operator	KEEP	dedupe open	1·1·2·2·1·2	9	HR	Cap validator fix-rerun cycles at 3; deduplicate copies
skills/mcp-server-builder	KEEP	—	1·1·2·2·0·1	7	LC	Loop rule: fix + re-run mcp_validator.py --strict until exit 0, max 3 cycles; done contract (paths + JSON keys)
skills/migration-architect	REWRITE	PARTIAL	1·2·2·2·1·1	9	HR	Cut L55-429 textbook to references/ (Verify cap ≤200); stop condition: 3 failed gate revisions → escalate
skills/monorepo-navigator	KEEP	—	1·0·2·0·0·0	3	TO	Numbered workflow + gate (analyzer JSON cycles empty; affected-only CI filter); artifact = workspace map
skills/observability-designer	REWRITE	PARTIAL	1·1·2·1·1·1	7	LC	Prune L35-273 golden-signals brochure; make alert loop a hard gate (duplicate count = 0) with 1-rotation stop
skills/performance-profiler	KEEP	—	1·1·2·1·0·1	6	LC	Before/after numbers as required artifact (<10% improvement → revert); one-bottleneck-at-a-time stop rule
skills/pr-review-expert	KEEP	—	1·1·2·1·0·1	6	LC	Verdict gate (BLOCK on MUST-FIX or coverage < −5%); re-review loop max 3 rounds then human
skills/rag-architect	REWRITE	RESOLVED	1·2·2·2·2·2	11	HR	Move 3 root scripts into scripts/ (layout anomaly only)
skills/runbook-generator	OPTIMIZE	PARTIAL	1·1·2·1·0·1	6	LC	Add June-required post-generation checklist as refusal gate (rollback non-empty, every step has verify line)
skills/secrets-vault-manager	KEEP	—	1·1·1·1·0·0	4	TO	Exact CLIs for all 3 scripts; gate: audit_log_analyzer shows zero old-credential usage before rotation done
skills/self-eval	KEEP	—	1·1·0·2·1·2	7	LC	Prompt-only by design; optional tiny scores_check.py JSONL assertion to formalize AR3
skills/ship-gate	KEEP	table-drift FAIL	2·2·0·2·2·2	10	HR	Wire the fully orphaned ship_gate_scanner.py (~1230 LOC) as Step 2 with exit-code verdict; true-up table 84→89
skills/skill-security-auditor	KEEP	—	1·1·2·2·0·1	7	LC	Remediate→re-scan loop until PASS (max 3); attach JSON report to install decision
skills/skill-tester	REWRITE	RESOLVED	1·1·2·2·2·1	9	HR	Recalibrate skill_validator.py tier minimums (still scores new-style <100-line skills POOR)
skills/slo-architect	KEEP	dedupe open	2·1·2·2·1·2	10	HR	Deduplicate copies (bundle Quick Start points at standalone path — deleting standalone breaks bundle docs)
skills/spec-driven-workflow	KEEP	—	2·2·1·2·2·2	11	HR	Fix pathless CLIs (python spec_validator.py → python3 scripts/spec_validator.py, L151/177/324-333) → AR3=2
skills/sql-database-assistant	KEEP	—	0·1·2·1·0·0	4	TO	Gate every generated query through query_optimizer.py (score <70 → rewrite); fix phantom observability-platform ref
skills/tc-tracker	KEEP	—	1·1·2·2·1·2	9	HR	Already strong; add explicit iteration cap on validation-fix loop
skills/tech-debt-tracker	REWRITE	RESOLVED	1·1·2·2·1·1	8	LC	Stop condition (2 flat snapshots → re-prioritize) + sprint artifact contract → ≥9
agenthub (8 sub-skills)	KEEP	—	2·2·2·2·2·2	12	HR	Wire orphaned dry_run.py as mandatory pre-spawn gate in /hub:run; explicit max-attempt cap
autoresearch-agent (6)	KEEP	doc-ask STILL-OPEN	2·2·2·2·2·2	12	HR	Add evaluator --help-exception sentence (June ask); replace stale CronCreate/CronDelete tool names
behuman	KEEP	registered ✅	1·1·0·1·1·1	5	TO	Ship a mirror-check lint as pre-output gate; cap the rewrite loop
caveman	KEEP	—	0·0·1·1·1·1	4	TO	Inline the 3 exact caveman_lint.py invocations (now only in companion_tooling.md); require PASS/WARN before sending
claude-coach	OPTIMIZE	STILL-OPEN (all 3)	2·1·1·1·2·1	8	LC	Fix dup frontmatter + delete L145-205 README paste; wire coach_tip_classifier.py as Rule-5 gate → HR
code-tour	KEEP	—	1·1·0·1·0·2	5	TO	Add tour_validator.py (schema + file/line existence, exit 0 before save); max 2 re-verify passes
collab-proof	NEW	—	2·2·2·2·1·2	11	HR	Add retry/cap rule for token-collection fallback; translate leftover Korean rubric phrases
data-quality-auditor	KEEP	—	1·1·2·2·0·2	8	LC	Remediate→re-profile loop with DQS delta report, cap 3 cycles → HR
demo-video	KEEP	—	1·1·0·1·0·2	5	TO	Ship scenes.json validator required before build.sh; exact ffmpeg fallback commands
docker-development	KEEP	—	0·1·2·2·0·1	6	LC	Intake (Dockerfile path + size/speed/security target); analyzer-score-must-improve loop, max 3 passes
grill-me	KEEP	—	2·2·1·1·2·2	10	HR	Exact flags for extractor/generator CLIs; machine gate = session JSON all branches resolved
grill-with-docs	KEEP	registered ✅	2·2·2·2·2·2	12	HR	None blocking — exemplar
handoff (engineering)	KEEP	—	1·1·1·0·0·2	5	TO	Wire 3 scripts with exact CLIs; port sibling productivity/handoff handoff_self_check.py 6-check gate
helm-chart-builder	KEEP	—	1·1·2·2·0·1	7	LC	Fix-and-revalidate loop (chart_analyzer.py 0 CRITICAL, cap 3); intake (workload/namespace/secrets)
karpathy-coder	KEEP	—	1·2·1·1·1·0	6	LC	Exact CLIs in SKILL.md (only agent/command carry them); make pre-commit check a required gate not warn-only
llm-cost-optimizer	KEEP	registered ✅	2·1·0·2·1·1	7	LC	Add one stdlib script (savings estimator) with CLI; before/after cost-JSON gate between techniques
llm-wiki	KEEP	—	1·1·2·1·0·2	7	LC	Make lint_wiki.py exit code a required post-ingest gate (now "periodic"); fix 4 phantom related-skill refs
prompt-governance	KEEP	registered ✅	2·1·0·1·1·1	6	LC	Ship registry-YAML validator; golden-dataset minimum (20) as Mode-2 refusal gate
security-guidance	KEEP	—	0·0·1·2·0·1	5	TO (hook by design)	Optional --scan <file> manual mode for deterministic re-run-to-exit-0
statistical-analyst	KEEP	—	2·1·2·2·1·1	9	HR	Add H1 heading (currently none); cap extend/re-test loop at one extension
terraform-patterns	KEEP	—	0·1·2·1·0·1	5	TO	Gate: 0 Critical from tf_security_scanner.py --strict before apply; fix ./scripts/convert.sh path
universal-scraping-architect	OPTIMIZE	RESOLVED	1·1·2·2·1·1	8	LC	Promote agent's intake to SKILL.md forcing questions; cap re-extraction at 2 attempts → HR
workflow-builder	KEEP	— (contract nuance)	2·2·2·2·2·1	11	HR	Document that validate_workflow.py --sample exits 1 by design; add done digest
write-a-skill	KEEP	self-check FAIL	2·1·2·2·1·2	10	HR	Trim own SKILL.md to <100 lines so it passes its own checklist runner; make runner exit-0 a blocking Phase-3 gate
zero-hallucination-coder	NEW	—	2·2·0·2·2·2	10	HR	Add one stdlib plan-linter (scan for unresolved [UNKNOWN]/TODO, exit 1) — AR3=0 is the only gap

(named-persona-adversarial-review is scored in engineering-team.md — it lives at engineering-team/skills/.)

Systemic findings

Patterns

1. The REWRITE wave worked, but two were patches, not rewrites. Brochure headings ("Future Enhancements"/"Conclusion"/"Planned Features") are now zero across engineering/; 5 of 7 REWRITEs fully resolved. migration-architect (429 lines) and observability-designer (272) got a wired Quick Start + gate bolted onto an unpruned textbook body — the June Verify line caps remain unmet.
2. AR5 (loop discipline) is the domain's weakest muscle. Only v2.4+/Pocock/orchestrator- generation skills state iteration caps or stop conditions. ~20 skills have a "re-run until clean" instruction with no cap; ~25 have none at all. A single sentence pattern ("max N fix-rerun cycles, then escalate") would lift 8 skills sitting at 7–8 into HARNESS-READY.
3. AR1 (goal intake) missing from tool-rich skills. The wiring epidemic was fixed (AR3 median is now 2), but most wired skills run on whatever input arrives — no forcing questions, no refusal on vague goals. The intake patterns already exist in-house (workflow-builder, grill-me, zero-hallucination-coder) and just need porting.
4. Gates exist but aren't binding. Many skills describe a validator yet don't require its exit code before proceeding (llm-wiki "periodic" lint, karpathy-coder warn-only hook, docker/helm validate-steps without loop closure).
5. Orphan scripts persist even in KEEP skills — new cases surfaced: ship-gate's ship_gate_scanner.py (~1230 LOC, full exit-code contract, never mentioned in SKILL.md — the model is told to scan manually), agenthub's dry_run.py, interview-system-designer's 3 root scripts, secrets-vault-manager's 3 unwired tools, handoff's 3 tools.
6. Dead cross-references survived the phantom-path sweep because they're skill-name table refs, not file paths: env-secrets-manager (5 dead), sql-database-assistant (observability-platform), llm-wiki (4 phantom related skills), focused-fix (scope + superpowers:* as REQUIRED).
7. Dual-published dedupe (June finding #4) not executed. All 4 pairs remain byte-identical (no divergence yet); trap: bundle copies' Quick Starts reference the standalone paths, so naive deletion of standalone copies breaks the bundle's own docs.
8. Database-trio merge (June finding #3a) not executed — database-designer got wired instead of merged; database-schema-designer remains the domain's worst skill (PROSE-ONLY, broken seed example intact).

New defects found this audit

• ship-gate: orphaned scanner + category-table drift (SKILL.md 84 vs checks.md 89) — its KEEP contract now FAILS.
• write-a-skill: fails its own checklist runner (141 lines vs its <100 rule; exit 1) — the meta-skill doesn't dogfood.
• autoresearch loop sub-skill: instructs stale CronCreate/CronDelete tool names and a 10-min interval that conflicts with the current hourly-minimum trigger surface — broken as written.
• claude-coach: zero progress on all 3 June defects (conflicting Version: 1.0.0 / version: 2.9.0 still parses ambiguously).
• workflow-builder: validate_workflow.py --sample exits 1 by design (intentionally broken sample) — the June KEEP criterion assumed 0; needs one documenting sentence.
• database-schema-designer: broken seed example at L154 persists (explicit June Verify item).
• collab-proof: untranslated Korean rubric phrases from the upstream Vela source.
• terraform-patterns: ./scripts/convert.sh invocation resolves only from repo root; repo version 2.9.0 leaked into an Infracost policy example.
• statistical-analyst: no H1 heading; "You are an expert…" opener also in data-quality-auditor.
• skill-tester's skill_validator.py still penalizes new-style <100-line skills (scores self-eval "POOR 33.3") despite the doc-level scope note.
• No broken scripts: all ~60 --help/--sample/pipe invocations exited per contract (non-zero only where documented).

Top-10 harness-ready exemplars

agenthub (12), autoresearch-agent (12), chaos-engineering (12), grill-with-docs (12), collab-proof (11), feature-flags-architect (11), spec-driven-workflow (11), workflow-builder (11), rag-architect (11), focused-fix (11). Honorable mentions at 10: slo-architect, ship-gate, grill-me, zero-hallucination-coder, write-a-skill — each one small fix from exemplar status.

Highest-leverage next PRs

1. Wire ship-gate's scanner + fix its category table.
2. One-sentence loop-cap sweep across the eight 7–8-point LOOP-CAPABLE skills (claude-coach, data-quality-auditor, universal-scraping-architect, docker-development, helm-chart-builder, mcp-server-builder, tech-debt-tracker, skill-security-auditor) — the cheapest path to ~30 HARNESS-READY.
3. Execute the deferred structural verdicts: dedupe the 4 dual-published pairs, merge/retire the database trio, fix claude-coach.