Commit graph

20 commits

Author SHA1 Message Date
Hannes Rudolph
8de9337e63
chore: remove XML tool calling support (#10841)
Co-authored-by: daniel-lxs <ricciodaniel98@gmail.com>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
2026-01-20 20:25:08 -05:00
Hannes Rudolph
41c5ff6076
feat(web-evals): remember last Roo model selection + add evals skill (#10470)
* feat(web-evals): remember last Roo model selection

* fix(web-evals): reset model selections on provider switch and fix lint warning

- Add useEffect to reset model selections when switching between providers
  This prevents OpenRouter model IDs from persisting when switching to Roo,
  which was causing Roo's stored selection to be overwritten with wrong IDs

- Remove unused 'executionMethod' from onSubmit dependency array to fix
  react-hooks/exhaustive-deps warning

* fix(web-evals): add missing executionMethod to test cases

* fix(web-evals): harden localStorage + keep provider selections
2026-01-07 08:00:34 -07:00
Chris Estreich
424bce6078
Add an option to use our cli for evals (#10456)
Co-authored-by: Roo Code <roomote@roocode.com>
2026-01-05 11:33:19 -08:00
Hannes Rudolph
84c5d2fd61
feat(evals): improve evals UI with tool groups and duration fix (#10133)
Co-authored-by: Roo Code <roomote@roocode.com>
2025-12-16 12:43:39 -08:00
Hannes Rudolph
325410955c
feat(web-evals): improve run logs and formatters (#10081) 2025-12-14 17:36:00 -08:00
Hannes Rudolph
c103a4a639
feat: streaming tool stats + token usage throttling (#9926)
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
2025-12-08 19:40:12 -08:00
Hannes Rudolph
c10d1d9ffc
feat(web-evals): add multi-model launch and UI improvements (#9845)
Co-authored-by: Roo Code <roomote@roocode.com>
2025-12-04 20:48:38 -05:00
Chris Estreich
648e009b8d
Update the evals keygen command (#9754) 2025-12-02 19:44:11 -08:00
Hannes Rudolph
d2017c80cf
Add web-evals updates and kill run functionality (#9681)
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
Co-authored-by: Roo Code <roomote@roocode.com>
2025-11-29 13:08:56 -07:00
Hannes Rudolph
3f0a6971ca
feat(web-evals): add task log viewing, export failed logs, and new run options (#9637)
Co-authored-by: roomote[bot] <219738659+roomote[bot]@users.noreply.github.com>
2025-11-27 21:15:31 -07:00
Hannes Rudolph
4442397507
feat(web-evals): enhance dashboard with dynamic tool columns and UX improvements (#9592)
Co-authored-by: Roo Code <roomote@roocode.com>
2025-11-26 00:15:29 -05:00
Chris Estreich
2ca9eac9e0
Enable the Roo Code Cloud provider in evals (#9492) 2025-11-22 00:06:03 -08:00
Chris Estreich
ef340806c7
Evals web app fixes / tweaks (#7299) 2025-08-21 12:36:33 -07:00
Chris Estreich
984d368f7a
Fix evals; broken by #5865 (#6065) 2025-07-22 09:18:38 -07:00
roomote[bot]
c96b399dd7
feat: add configurable timeout for evals (5-10 min) (#5865)
* feat: add configurable timeout for evals (5-10 min)

- Add timeout field to CreateRun schema with min 5, max 10, default 5
- Add timeout slider UI component to /runs/new page
- Update database schema to include timeout column in runs table
- Create migration to add timeout column with default value of 5
- Update runTask.ts to use configurable timeout from run settings
- Pass timeout parameter through the createRun action

* fix: remove unused EVALS_TIMEOUT import

* fix: add timeout field to createRun calls in copyRun test

- Added timeout: 5 to both createRun calls in copyRun.spec.ts
- This fixes the test failure caused by the new required timeout field in the runs schema
- The timeout field was added in the configurable timeout feature but the test was not updated

* fix: use configurable timeout for Redis key expiration in registerRunner

- Updated registerRunner function to accept timeoutSeconds parameter
- Modified call in runTask.ts to pass configurable timeout instead of hardcoded EVALS_TIMEOUT
- Removed unused EVALS_TIMEOUT import from redis.ts
- Ensures Redis keys remain valid for the entire duration of task execution (up to 10 minutes)

---------

Co-authored-by: Roo Code <roomote@roocode.com>
Co-authored-by: hannesrudolph <hrudolph@gmail.com>
2025-07-19 22:20:23 -04:00
Chris Estreich
ab01fb3bdb
Move to Postgres for evals on roocode.com (#4520) 2025-06-11 14:17:53 -07:00
Chris Estreich
8d5dab3518
GHA evals (#4472) 2025-06-10 11:01:54 -07:00
Chris Estreich
24851b96db
Improve the evals "run status" section (#4400) 2025-06-05 22:00:15 -07:00
Chris Estreich
cb5b9c3718
Improve Docker setup for evals (#4327) 2025-06-05 12:16:36 -07:00
Chris Estreich
d87f890556
Move evals into pnpm workspace, switch from SQLite to Postgres (#4278) 2025-06-03 01:38:08 -07:00