claude-skills/engineering/memory-engineering/skills/memory-engineering/SKILL.md
Claude 5c0e5314b4
fix(memory-engineering): make all four tie-breakers reachable
Second automated review on PR #947 found a real bug, verified by execution:

`pick()` looks up `tuple(sorted([winner, runner_up]))`, but two of the four
TIE_BREAKERS keys were authored in the other order, so they could never match:

  ('flat_rag', 'structured_rag')      sorted -> matches
  ('structured_rag', 'agentic')       sorted -> ('agentic', ...)      DEAD
  ('long_context', 'flat_rag')        sorted -> ('flat_rag', ...)     DEAD
  ('long_context', 'structured_rag')  sorted -> matches

Only 2 of 4 authored questions were reachable. The two dead ones are the
plausible near-ties (structured_rag vs agentic on a high-recall/high-mutability
workload; long_context vs flat_rag under build-budget pressure), so the tool's
most distinctive behaviour — an authored, specific tie-breaking question —
silently degraded to the generic fallback with no error.

Keys are now normalized through sorted() at import, with a collision check that
raises if two entries describe the same pair. Verified by injecting a duplicate
in the reverse order: the guard fires. This repo has no test suite, so the check
runs at import rather than living in a test.

Confirmed by execution, not inspection: all 4 keys reachable, and a constraint
set that ties structured_rag against agentic now returns the authored question
("Does your memory need to correct itself without a human in the loop?")
instead of the fallback.

Also from the review: the SKILL.md workflow block labelled steps 1-4 while the
prose referenced a step 5, so a reader skimming only the code block would not
know it existed. Added a `# 5 - No command.` line and compensated elsewhere to
stay within the checklist's 100-line limit (still 6/6 PASS).

Verified: 4/4 scripts --help/--sample/--output json; all six blocking gates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jt1sqt5kQmopyfXu2Hhjnv
2026-08-09 04:57:43 +00:00

6.6 KiB
Raw Blame History

name description argument-hint license metadata
memory-engineering Use when designing, reviewing, or paying for an agent memory system — adding memory to an agent, choosing between long-context / RAG / graph / agentic memory, auditing what a CLAUDE.md or memory directory actually holds, deciding what to keep and what to expire, or when a memory store keeps growing and nobody has said what leaves it. Prices the write path, picks which cost to pay, classifies records as facts / skills / logs, and refuses a design that has no forgetting policy. [optional: path to a memory dir, design spec JSON, or a question] MIT
version build_pattern distinct_from
1.0.0 Four-lens synthesis (Stanford / Microsoft / Anthropic / Nvidia) + 4 deterministic stdlib scripts, with a blocking forgetting gate llm-wiki (maintains one specific markdown vault; this audits and prices any memory system); skillopt-sleep (runs a nightly consolidation loop; this decides whether that loop's output is worth keeping); agent-harness (bounds a task loop; this bounds a store)

Memory Engineering — engineer the forgetting, not just the remembering

Portability: 4 stdlib scripts, no APIs/LLM calls/network. They measure and gate; you decide.

What this does

Anyone can give an agent memory: vector store, pipe in the history, retrieve top-k. That works until the history outgrows the context window, the write path costs more than every query it serves, and the store fills with stale state nobody removes. Memory is not a bucket — it is a system with a metabolism.

The shift: a storer optimizes what a system remembers; a memory engineer optimizes what it forgets. The problem was never that an agent forgets — it is that it never forgets on purpose.

The four lenses

Lens Question The finding that hurts
Stanford What does remembering cost? Construction energy exceeds total query energy across 300 queries. The tuned half is the smaller half.
Microsoft What is worth keeping? More raw memory can make an agent worse. Keep facts and skills; drop the events.
Anthropic Who controls what it keeps? A wrong memory does not fail once — it persists into every future session that reads it.
Nvidia Where does it hit hardware? It is all KV cache in HBM. Construction is prefill-heavy and stalls the query a user is waiting on.

Workflow

# 1 - Price it first. Never quote a quality number without a cost number.
python scripts/memory_cost_profiler.py --print-sample-spec > workload.json
python scripts/memory_cost_profiler.py --spec workload.json
# 2 - Pick which cost to pay. No "best" verdict; on a tie it asks, exit 2.
python scripts/memory_architecture_picker.py --constraints workload.json
# 3 - Audit what the store actually holds (skip if greenfield).
python scripts/memory_density_auditor.py --dir ~/.claude/memory
# 4 - Gate on forgetting. Exit 4 is a stop, not a suggestion.
python scripts/forgetting_policy_linter.py --policy design.json
# 5 - No command. Prove each pass by hand before scheduling it.

Step 1 reports the construction/query split, cost per correct answer, and amortization — if construction dominates, cut construction tokens before touching retrieval. Step 2 names the cost the winning family makes you pay. Step 3 classifies records FACT / SKILL / LOG / PROSE (LOG-HEAVY = archiving events; PROSE-HEAVY = docs, not memory).

Step 4 is the gate: F1 (explicit forgetting rule) and F4 (contradictions surfaced, never auto-merged) are blocking. Retrofitting forgetting onto two years of records is a migration nobody does; auto-merging disagreeing memories destroys the evidence the conflict existed.

Step 5 has no script — prove each pass by hand, then automate. Run it once against real history and ask whether it changed a decision. If not, scheduling it only makes noise. Ship order: forgetting_policy_design.md §7.

Hard rules

  1. Never quote accuracy without cost per correct answer.
  2. Never return a "best" memory system — name the cost the choice makes you pay.
  3. Never auto-merge contradictions. The system surfaces; the human decides.
  4. Never call a design done without a forgetting rule. No evaluated system provides one by default.
  5. Never schedule a pass not yet run by hand.
  6. Report findings as findings. A non-zero exit is a result to surface, not an error to swallow.
  7. Attribute every number with its confidence level. Vendor customer figures are testimonials, not benchmarks.

Scripts

Script Role Exit codes
scripts/memory_cost_profiler.py Construction vs query split, cost per correct answer, amortization, co-location warning 0 · 2 finding · 3 bad input
scripts/memory_architecture_picker.py Scores 4 families, disqualifies, names the cost, refuses to pick on a tie 0 · 2 ambiguous · 3 bad input · 4 none viable
scripts/memory_density_auditor.py FACT/SKILL/LOG/PROSE, duplicates, staleness, density (--dir or --jsonl) 0 dense · 2 finding · 3 bad input
scripts/forgetting_policy_linter.py The gate: 8 checks, F1 and F4 blocking 0 PASS · 2 CONDITIONAL · 4 FAIL

All support --output json and --sample (no input file needed).

References and assets

Provenance

Framing from "How to be a Memory Engineer" by @N01ennn; every number is cited to a primary source instead, and two paraphrases are corrected — memory_cost_canon.md §2, memory_control_and_governance.md §4.