scriptoriumadmin/fabro
1
0
Fork 0
mirror of https://github.com/fabro-sh/fabro.git synced 2026-08-28 05:27:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
fabro/evals/swe-bench/scoreboard
History
Exact
Bryan Helmkamp f3efcc931e Record GPT-5.4 20min timeout: 65.7% on SWE-Bench-Lite ($718.62) 
Best resolve rate at 197/300 but 3.3x more expensive than Opus.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 20:55:59 -04:00
..
gpt54-20min-20260316 Record GPT-5.4 20min timeout: 65.7% on SWE-Bench-Lite ($718.62) 2026-03-16 20:55:59 -04:00
haiku-baseline-20260316 Record timeout and sandbox resources in scoreboard metadata 2026-03-16 11:24:56 -04:00
opus-20min-20260316 Record Opus 4.6 20min timeout: 58.0% on SWE-Bench-Lite ($218.65) 2026-03-16 14:43:20 -04:00
sonnet-20min-20260316 Record Sonnet 4.6 20min timeout: 57.7% on SWE-Bench-Lite ($55.22) 2026-03-16 13:36:26 -04:00
sonnet-baseline-20260316 Record timeout and sandbox resources in scoreboard metadata 2026-03-16 11:24:56 -04:00
leaderboard.json Record GPT-5.4 20min timeout: 65.7% on SWE-Bench-Lite ($718.62) 2026-03-16 20:55:59 -04:00