mirror of https://github.com/alirezarezvani/claude-skills.git synced 2026-08-28 04:24:58 +00:00

feat(skills): ship chaos-engineering (Phase 3 — resilience testing discipline)

Phase 3 of the multi-skill build effort. Same 14-step pipeline. Composes
explicitly with feature-flags-architect (kill switches as abort triggers)
and kubernetes-operator (operators are common chaos targets).

## What landed

### New skill: engineering/chaos-engineering

End-to-end chaos engineering discipline. Published as BOTH:
- Standalone plugin: engineering/chaos-engineering/
- Bundled mirror:    engineering/skills/chaos-engineering/

3 stdlib-only Python tools (Karpathy complexity 95/100 — best in portfolio):
- experiment_designer.py        — generates structured plans with hypothesis,
                                   steady-state, blast radius, abort criteria,
                                   rollback. Refuses to render plans without
                                   abort criteria (exit code 1).
- blast_radius_calculator.py    — computes affected users + error budget
                                   consumption + GREEN/YELLOW/RED risk score.
                                   Validates inputs (0 ≤ traffic-share ≤ 1).
- experiment_postmortem.py      — blameless postmortems from plan + result log;
                                   detects blame-laden language ("fault of",
                                   "should have known", "stupid", etc.) and
                                   warns at write time.

4 reference docs:
- chaos_principles.md      — 4 founding principles + 5th abort principle,
                              maturity model, history, when-to-start checklist
- experiment_design.md      — 7-section plan structure, pre-flight checklist,
                              time-boxing, escalation
- attack_taxonomy.md        — 7 attack types (latency / error / resource /
                              network-partition / dependency-failure / time-skew
                              / infrastructure) with magnitudes and tooling
- tooling_landscape.md      — Chaos Toolkit / Mesh / Litmus / Gremlin / AWS FIS
                              / DIY decision tree

Templates:
- experiment_template.md    — fill-in plan with all 7 sections
- postmortem_template.md    — blameless postmortem structure

Plus: SKILL.md (213 lines), README.md, /chaos-experiment slash command.

### Audit verdict (evidence-based)

Closest existing skills:
- engineering-team/incident-response — for actual incidents, not prevention
- engineering-team/red-team — adversarial; different goal (find attack paths)
- engineering-team/threat-detection — hunting; different goal
- engineering/observability-designer — measurement, not fault injection
None cover the chaos-engineering discipline (hypothesis-driven fault injection
with bounded blast radius). Verdict: BUILD. Gap is real and tooling-shaped.

### Composition story (Phase 1+2+3 form a stack)

```
feature-flags-architect.kill_switch_audit.py
  ↓ defines kill switches that ↓
chaos-engineering.experiment_designer.py
  ↓ designs experiments against ↓
kubernetes-operator (and other targets)
```

Together: a complete progressive-delivery + resilience-testing stack.

### Marketplace / registry

- marketplace.json: chaos-engineering registered as standalone plugin
- engineering-advanced-skills bundle: 47 → 48 skills, version → 2.4.2
- engineering/.claude-plugin/plugin.json: version + skill list updated
- mkdocs.yml: nav entry under "Engineering - POWERFUL"
- docs/skills/engineering/chaos-engineering.md: docs page (manual,
  pending generate-docs.py classification fix)
- docs/commands/chaos-experiment.md: auto-generated
- .codex/, .gemini/: synced

### Karpathy-coder gates

- complexity_checker (strict): 95/100 average — BEST score in the new
  portfolio. Only 1 WARN (depth 5 in blast_radius_calculator.py validation
  branches; the other 2 scripts hit no findings whatsoever).
- All 1666 tests pass (was 1648; added 18 for the new skill).
- mkdocs build --strict: succeeded in 13.33s.

### Verifiable success criteria (all green)

✓  scripts/*.py --help     → exit 0 for all 3 scripts
✓  SKILL.md frontmatter    → name + description + tags + compatible_tools
✓  plugin.json schema      → 8 fields exact (verified by check_plugin_json.py)
✓  sync_skill_bundles      → standalone ↔ bundled mirror in sync
✓  marketplace.json        → standalone entry + bundle counts updated
✓  generate-docs.py        → command page generated (skill page manual)
✓  mkdocs build --strict   → succeeded
✓  cross-tool sync         → codex + gemini synced
✓  pytest tests/           → 1666 passed, 0 failed
✓  CHANGELOG.md            → [Unreleased] entry expanded for Phase 3
✓  Self-test (RED case)    → 50% blast radius on 99.9% baseline correctly
                             classifies as RED (17.33% of monthly budget) and
                             returns ABORT recommendation
✓  Composition test        → references named skills explicitly compose

## Phase 1+2+3 cumulative

- 3 new skills: feature-flags-architect, kubernetes-operator, chaos-engineering
- 9 new Python tools (all stdlib, all <200 LOC, average complexity 90/100)
- 12 new reference docs (~250-500 lines each)
- 3 new slash commands (/flag-cleanup, /operator-audit, /chaos-experiment)
- 2 repo-infrastructure scripts (sync_skill_bundles, check_plugin_json)
- 1 pre-existing test fix (full-page-screenshot CI red)

## Files

- engineering/chaos-engineering/                                (new standalone plugin)
- engineering/skills/chaos-engineering/                         (new bundled mirror)
- commands/chaos-experiment.md                                  (new slash command)
- docs/skills/engineering/chaos-engineering.md                  (new docs page)
- docs/commands/chaos-experiment.md                             (auto-generated)
- mkdocs.yml                                                    (nav entries)
- .claude-plugin/marketplace.json                               (registered)
- engineering/.claude-plugin/plugin.json                        (bundle bumped)
- CHANGELOG.md                                                  ([Unreleased] expanded)
- .codex/, .gemini/                                             (cross-tool sync)

https://claude.ai/code/session_01Dq12xJakFRxwaoU8Pqejdm

2026-05-09 21:24:16 +00:00

5 KiB

Raw Blame History

title	description
Chaos Engineering — Experiments That Don't Become Outages	End-to-end chaos engineering discipline for Claude Code: design experiments with hypothesis + steady-state + blast radius + abort criteria, calculate risk against error budget, and generate blameless postmortems. 3 stdlib Python tools, 4 references covering principles + design + 7-attack taxonomy + tooling. Composes with feature-flags-architect and kubernetes-operator.

Chaos Engineering

:material-rocket-launch: Engineering - POWERFUL :material-identifier: `chaos-engineering` :material-github: Source

Install: claude /plugin install chaos-engineering

Design experiments that surface real weaknesses in production systems — without becoming outages. Most "chaos engineering" attempts skip steady-state measurement, define no abort criteria, and have no blast-radius bound. This skill enforces the discipline that makes chaos experiments safe and useful.

When to use

Planning a chaos experiment (what to break, where, when, how to abort)
Calculating blast radius before running
Reviewing an experiment plan for safety
Choosing a chaos tool (Chaos Toolkit / Mesh / Litmus / Gremlin / AWS FIS)
Writing a chaos experiment postmortem
Running a Game Day exercise

When NOT to use

General incident response → incident-response
Threat hunting / red-team → red-team, threat-detection
Performance load testing (different goal — chaos is failure modes, not capacity)

Core principle: chaos without abort criteria is an outage

The 4 founding principles + 1 mandatory addition:

Build a hypothesis around steady-state behavior — measurable, falsifiable
Vary real-world events — realistic faults only
Run experiments in production — staging never has prod failure modes
Automate experiments to run continuously — single experiment = press release
Define abort criteria up front — no abort = outage

The 3 Python tools

All stdlib-only. Karpathy complexity 95/100 — best score in the portfolio.

`experiment_designer.py`

Generates a structured experiment plan. Enforces hypothesis, steady-state, blast radius, abort criteria, rollback.

python scripts/experiment_designer.py \
  --target checkout-svc \
  --hypothesis "p99 < 500ms when payment slows" \
  --attack latency --magnitude "+200ms" \
  --abort-if "p99 > 1000ms OR error_rate > +1pp"

`blast_radius_calculator.py`

Computes affected users, error budget consumed, and risk score (GREEN/YELLOW/RED).

python scripts/blast_radius_calculator.py \
  --traffic-share 0.05 --user-pop 1000000 --duration-min 15

GREEN = <1% error budget; YELLOW = 1-10%; RED = >10% (ABORT/REDUCE).

`experiment_postmortem.py`

Generates a blameless postmortem from plan + result log. Detects blame-laden language.

python scripts/experiment_postmortem.py \
  --plan plan.json --result-log results.txt

The 7 attack types

Attack	Tests
Latency	Timeouts, retries, circuit breakers
Error	Error handling, fallback paths
Resource	Saturation, autoscaling, OOM
Network partition	Consensus, leader election, failover
Dependency failure	Graceful degradation
Time skew	Clocks, TTLs, retry backoff
Infrastructure	Auto-recovery, replica maintenance

See references/attack_taxonomy.md for full magnitude examples and tooling per attack.

Tooling chooser

Tool	Stack	OSS
Chaos Toolkit	Any	Yes
Chaos Mesh	Kubernetes	Yes
Litmus	Kubernetes	Yes
Gremlin	Any (commercial)	No
AWS FIS	AWS	Paid
Custom	Any	DIY

Composition

Skill	Composition
`feature-flags-architect`	Kill switches there are abort triggers here
`kubernetes-operator`	Operators are common chaos targets
`incident-response`	Chaos that escalates becomes an incident

Slash command

/chaos-experiment — Interactive design wizard.

Reference docs

references/chaos_principles.md — 4 principles + 5th abort principle, history, when to start
references/experiment_design.md — 7 sections, pre-flight checklist, time-boxing
references/attack_taxonomy.md — 7 attacks with magnitudes and tooling
references/tooling_landscape.md — full provider comparison

Verifiable success

A team using this skill should achieve:

100% of experiments have written hypothesis, abort criteria, blast-radius calc
Blast radius for any single experiment ≤10% of monthly error budget
Mean time between experiments <14 days
Each experiment produces ≥1 follow-up action that gets shipped
No chaos experiment escalates to a customer-impacting incident in trailing 90 days

5 KiB Raw Blame History