fabro/evals
Bryan Helmkamp 5c503565aa
Add Daytona-based SWE-bench evaluator and bump default concurrency to 100
evaluate_daytona.py runs the swebench test harness on Daytona sandboxes
instead of local Docker. Reuses the same snapshots from the generation
phase. Applies model patch + test patch, runs tests, grades with
swebench's log parsers. No local Docker needed.

Also bumps default --max-workers from 20 to 100 in run_eval.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 09:13:09 -04:00
..
swe-bench Add Daytona-based SWE-bench evaluator and bump default concurrency to 100 2026-03-16 09:13:09 -04:00