Canyon Robins
6762b579b2
[Evals] Headless evals with Docker ( #4121 )
...
* [Evals] Try to resurrect headless evals with Docker
* fixes; now it runs
* rm
* remove commented out files
* remove trailing space
* nit
* throw error on concurrency attempt
2025-05-29 15:36:11 -07:00
Shariq Riaz
ce3e4e883d
feat(evals): add UI and backend support for importing and injecting f… ( #3606 )
2025-05-19 14:58:58 -07:00
Chris Estreich
03925d25b0
Fix code cli flag in evals ( #2889 )
2025-04-23 13:26:59 -07:00
seedlord
c2dd743aeb
Fix: Preserve editor state and prevent tab unpinning during diffs ( #2857 )
...
- Maintains editor view column state when closing and reopening files during diff operations, ensuring tabs stay opened in their original position.
- Prevents closing the original editor tab when opening the diff view, preserving pinned status when applying changes via write_to_file or apply_diff.
- Updates VSCode workspace launch flag from -n to -W for compatibility.
2025-04-23 02:45:35 -04:00
Chris Estreich
61e23cccb6
Record tool use errors encountered during eval runs ( #2816 )
2025-04-21 11:04:35 -07:00
Chris Estreich
87af3b3424
Record tool usages in the Cline object, and persist them in the db for evals ( #2729 )
2025-04-17 22:21:14 -07:00
Chris Estreich
a4d2de4534
Add pass / fail events for evals ( #2656 )
2025-04-15 11:29:38 -07:00
Chris Estreich
5b48a2d708
More small evals tweaks ( #2620 )
2025-04-14 21:13:08 -07:00
nobuo kawasaki
30c3a65b7b
Fix eslint error about --ext option by remove it ( #2543 )
2025-04-14 16:25:09 -07:00
Chris Estreich
352a6b9c75
More unit test kill -9 fixes ( #2570 )
...
* More unit test kill -9 fixes
* Fix linter warning
2025-04-13 09:44:45 -07:00
Chris Estreich
ef9b339027
Evals improvements ( #2555 )
...
* Evals improvements
* Remove debugging
2025-04-13 00:57:33 -07:00
Chris Estreich
6ab9aa9f15
Control evals concurrency in web app ( #2265 )
2025-04-11 22:21:15 -07:00
Chris Estreich
2eba534dd6
Evals fixes ( #2505 )
...
* Allow Turso URLs, add support for API providers beyond OpenRouter
* Make the git branch name unique
2025-04-11 14:54:03 -07:00
Chris Estreich
6d9ebe3fcf
More sane evals default concurrency + staggered startup ( #2441 )
2025-04-09 13:04:36 -07:00
Chris Estreich
8f356711f1
Ironing out some kinks when running evals with high parallelism ( #2280 )
2025-04-04 09:08:12 -07:00
Chris Estreich
d15f813514
Evals ( #2238 )
...
* Evals
* Remove redundant line
* Fix tsc error
* Disable debug mode
* Add option to kill run
2025-04-02 16:49:41 -07:00
