Commit graph

12 commits

Author SHA1 Message Date
Hannes Rudolph
41c5ff6076
feat(web-evals): remember last Roo model selection + add evals skill (#10470)
* feat(web-evals): remember last Roo model selection

* fix(web-evals): reset model selections on provider switch and fix lint warning

- Add useEffect to reset model selections when switching between providers
  This prevents OpenRouter model IDs from persisting when switching to Roo,
  which was causing Roo's stored selection to be overwritten with wrong IDs

- Remove unused 'executionMethod' from onSubmit dependency array to fix
  react-hooks/exhaustive-deps warning

* fix(web-evals): add missing executionMethod to test cases

* fix(web-evals): harden localStorage + keep provider selections
2026-01-07 08:00:34 -07:00
Chris Estreich
424bce6078
Add an option to use our cli for evals (#10456)
Co-authored-by: Roo Code <roomote@roocode.com>
2026-01-05 11:33:19 -08:00
Hannes Rudolph
325410955c
feat(web-evals): improve run logs and formatters (#10081) 2025-12-14 17:36:00 -08:00
Hannes Rudolph
8433eafb05
feat(evals-ui): Add filtering, bulk delete, tool consolidation, and run notes (#9837) 2025-12-04 14:28:37 -07:00
Hannes Rudolph
3f0a6971ca
feat(web-evals): add task log viewing, export failed logs, and new run options (#9637)
Co-authored-by: roomote[bot] <219738659+roomote[bot]@users.noreply.github.com>
2025-11-27 21:15:31 -07:00
Hannes Rudolph
4442397507
feat(web-evals): enhance dashboard with dynamic tool columns and UX improvements (#9592)
Co-authored-by: Roo Code <roomote@roocode.com>
2025-11-26 00:15:29 -05:00
Chris Estreich
2ca9eac9e0
Enable the Roo Code Cloud provider in evals (#9492) 2025-11-22 00:06:03 -08:00
roomote[bot]
c96b399dd7
feat: add configurable timeout for evals (5-10 min) (#5865)
* feat: add configurable timeout for evals (5-10 min)

- Add timeout field to CreateRun schema with min 5, max 10, default 5
- Add timeout slider UI component to /runs/new page
- Update database schema to include timeout column in runs table
- Create migration to add timeout column with default value of 5
- Update runTask.ts to use configurable timeout from run settings
- Pass timeout parameter through the createRun action

* fix: remove unused EVALS_TIMEOUT import

* fix: add timeout field to createRun calls in copyRun test

- Added timeout: 5 to both createRun calls in copyRun.spec.ts
- This fixes the test failure caused by the new required timeout field in the runs schema
- The timeout field was added in the configurable timeout feature but the test was not updated

* fix: use configurable timeout for Redis key expiration in registerRunner

- Updated registerRunner function to accept timeoutSeconds parameter
- Modified call in runTask.ts to pass configurable timeout instead of hardcoded EVALS_TIMEOUT
- Removed unused EVALS_TIMEOUT import from redis.ts
- Ensures Redis keys remain valid for the entire duration of task execution (up to 10 minutes)

---------

Co-authored-by: Roo Code <roomote@roocode.com>
Co-authored-by: hannesrudolph <hrudolph@gmail.com>
2025-07-19 22:20:23 -04:00
Chris Estreich
ab01fb3bdb
Move to Postgres for evals on roocode.com (#4520) 2025-06-11 14:17:53 -07:00
Chris Estreich
24851b96db
Improve the evals "run status" section (#4400) 2025-06-05 22:00:15 -07:00
Chris Estreich
cb5b9c3718
Improve Docker setup for evals (#4327) 2025-06-05 12:16:36 -07:00
Chris Estreich
d87f890556
Move evals into pnpm workspace, switch from SQLite to Postgres (#4278) 2025-06-03 01:38:08 -07:00