mirror of https://github.com/alirezarezvani/claude-skills.git synced 2026-10-06 02:50:08 +00:00

Merge origin/dev: reconcile docs redesign with upstream skill changes

- Resolve conflicts: keep redesigned skills index, take dev's cs-aeo link
  fix, union of DOMAIN_SEO_CONTEXT entries in generate-docs.py
- Regenerate catalog on the merged tree (dev's agent/command description
  updates, removed ai-seo/release-manager/command-guide, restructured
  universal-scraping-architect)
- Update counters to post-merge truth from scripts/derive_counters.py:
  345 skills, 78 plugins (14 bundles + 64 standalone), 570+ Python tools
- Add redirects for upstream-removed pages (ai-seo -> aeo,
  release-manager -> changelog-generator, command-guide -> engineering index)
- Add compliance-os bundle to bundle tables; rebuild 78-plugin table from
  marketplace.json
- Teach the generator to rewrite repo-root-relative source links to GitHub
  URLs — mkdocs build --strict now passes with zero warnings

https://claude.ai/code/session_015bYZ97nV4oRb3LbxCRFVcP

2026-06-11 15:52:42 +00:00

3 KiB

Raw Permalink Blame History

title	description
/hub:eval — Evaluate Agent Results — Agent Skill for Codex & OpenClaw	Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a. Agent skill for Claude Code, Codex CLI, Gemini CLI, OpenClaw.

/hub:eval — Evaluate Agent Results

:material-rocket-launch: Engineering - POWERFUL :material-identifier: `eval` :material-github: Source

Install: claude /plugin install engineering-advanced-skills

Rank all agent results for a session. Supports metric-based evaluation (run a command), LLM judge (compare diffs), or hybrid.

Usage

/hub:eval                           # Eval latest session using configured criteria
/hub:eval 20260317-143022           # Eval specific session
/hub:eval --judge                   # Force LLM judge mode (ignore metric config)

What It Does

Metric Mode (eval command configured)

Run the evaluation command in each agent's worktree:

python {skill_path}/scripts/result_ranker.py \
  --session {session-id} \
  --eval-cmd "{eval_cmd}" \
  --metric {metric} --direction {direction}

Output:

RANK  AGENT       METRIC      DELTA      FILES
1     agent-2     142ms       -38ms      2
2     agent-1     165ms       -15ms      3
3     agent-3     190ms       +10ms      1

Winner: agent-2 (142ms)

LLM Judge Mode (no eval command, or --judge flag)

For each agent:

Get the diff: git diff {base_branch}...{agent_branch}
Read the agent's result post from .agenthub/board/results/agent-{i}-result.md
Compare all diffs and rank by:
- Correctness — Does it solve the task?
- Simplicity — Fewer lines changed is better (when equal correctness)
- Quality — Clean execution, good structure, no regressions

Present rankings with justification.

Example LLM judge output for a content task:

RANK  AGENT    VERDICT                               WORD COUNT
1     agent-1  Strong narrative, clear CTA            1480
2     agent-3  Good data points, weak intro           1520
3     agent-2  Generic tone, no differentiation       1350

Winner: agent-1 (strongest narrative arc and call-to-action)

Hybrid Mode

Run metric evaluation first
If top agents are within 10% of each other, use LLM judge to break ties
Present both metric and qualitative rankings

After Eval

Update session state:

python {skill_path}/scripts/session_manager.py --update {session-id} --state evaluating

Tell the user:
- Ranked results with winner highlighted
- Next step: /hub:merge to merge the winner
- Or /hub:merge {session-id} --agent {winner} to be explicit

3 KiB Raw Permalink Blame History