fabro/evals
Bryan Helmkamp c248aaf92e
Add scoreboard system and record Haiku 4.5 baseline: 54.0% on SWE-Bench-Lite
record_results.py combines generation + eval results into a git-tracked
scoreboard. Per run: README, meta.json, instances.jsonl. Auto-generates
leaderboard.json ranked by resolve rate.

Haiku 4.5 baseline: 162/300 (54.0%), $26.13 total ($0.087/instance).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 09:56:43 -04:00
..
swe-bench Add scoreboard system and record Haiku 4.5 baseline: 54.0% on SWE-Bench-Lite 2026-03-16 09:56:43 -04:00