Commit graph

39 commits

Author SHA1 Message Date
Chris Estreich
fd46e31134
Update next.js (#11108) 2026-01-30 02:08:45 -08:00
Hannes Rudolph
8de9337e63
chore: remove XML tool calling support (#10841)
Co-authored-by: daniel-lxs <ricciodaniel98@gmail.com>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
2026-01-20 20:25:08 -05:00
Hannes Rudolph
41c5ff6076
feat(web-evals): remember last Roo model selection + add evals skill (#10470)
* feat(web-evals): remember last Roo model selection

* fix(web-evals): reset model selections on provider switch and fix lint warning

- Add useEffect to reset model selections when switching between providers
  This prevents OpenRouter model IDs from persisting when switching to Roo,
  which was causing Roo's stored selection to be overwritten with wrong IDs

- Remove unused 'executionMethod' from onSubmit dependency array to fix
  react-hooks/exhaustive-deps warning

* fix(web-evals): add missing executionMethod to test cases

* fix(web-evals): harden localStorage + keep provider selections
2026-01-07 08:00:34 -07:00
Chris Estreich
424bce6078
Add an option to use our cli for evals (#10456)
Co-authored-by: Roo Code <roomote@roocode.com>
2026-01-05 11:33:19 -08:00
John Richmond
2863b1c3f1
Update next.js to ~15.2.8 (#10140) 2025-12-16 21:28:51 -08:00
Hannes Rudolph
84c5d2fd61
feat(evals): improve evals UI with tool groups and duration fix (#10133)
Co-authored-by: Roo Code <roomote@roocode.com>
2025-12-16 12:43:39 -08:00
Hannes Rudolph
325410955c
feat(web-evals): improve run logs and formatters (#10081) 2025-12-14 17:36:00 -08:00
Hannes Rudolph
c103a4a639
feat: streaming tool stats + token usage throttling (#9926)
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
2025-12-08 19:40:12 -08:00
Hannes Rudolph
c10d1d9ffc
feat(web-evals): add multi-model launch and UI improvements (#9845)
Co-authored-by: Roo Code <roomote@roocode.com>
2025-12-04 20:48:38 -05:00
Hannes Rudolph
8433eafb05
feat(evals-ui): Add filtering, bulk delete, tool consolidation, and run notes (#9837) 2025-12-04 14:28:37 -07:00
John Richmond
72a6805b4b
Update next.js (#9799) 2025-12-03 20:17:28 -08:00
Chris Estreich
648e009b8d
Update the evals keygen command (#9754) 2025-12-02 19:44:11 -08:00
Hannes Rudolph
d2017c80cf
Add web-evals updates and kill run functionality (#9681)
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
Co-authored-by: Roo Code <roomote@roocode.com>
2025-11-29 13:08:56 -07:00
Hannes Rudolph
3f0a6971ca
feat(web-evals): add task log viewing, export failed logs, and new run options (#9637)
Co-authored-by: roomote[bot] <219738659+roomote[bot]@users.noreply.github.com>
2025-11-27 21:15:31 -07:00
Hannes Rudolph
4442397507
feat(web-evals): enhance dashboard with dynamic tool columns and UX improvements (#9592)
Co-authored-by: Roo Code <roomote@roocode.com>
2025-11-26 00:15:29 -05:00
Chris Estreich
2ca9eac9e0
Enable the Roo Code Cloud provider in evals (#9492) 2025-11-22 00:06:03 -08:00
Chris Estreich
25f61943ca
Set port 3446 for web-evals in production mode (#8288) 2025-09-24 08:43:05 -07:00
Chris Estreich
0e1b23d09c
Bare metal evals fixes (#8224)
Co-authored-by: Roo Code <roomote@roocode.com>
2025-09-22 12:54:56 -07:00
Chris Estreich
f02a2bb05d
Random chat text area cleanup (#7436) 2025-08-26 20:48:24 -07:00
Chris Estreich
ef340806c7
Evals web app fixes / tweaks (#7299) 2025-08-21 12:36:33 -07:00
Chris Estreich
cb6dccab95
Miscellaneous cleanup (#6453) 2025-07-30 16:10:53 -07:00
Chris Estreich
9956cc1f48
Use SIGKILL for command execution timeouts in the "execa" variant (#6071) 2025-07-23 01:47:10 -07:00
Chris Estreich
984d368f7a
Fix evals; broken by #5865 (#6065) 2025-07-22 09:18:38 -07:00
roomote[bot]
c96b399dd7
feat: add configurable timeout for evals (5-10 min) (#5865)
* feat: add configurable timeout for evals (5-10 min)

- Add timeout field to CreateRun schema with min 5, max 10, default 5
- Add timeout slider UI component to /runs/new page
- Update database schema to include timeout column in runs table
- Create migration to add timeout column with default value of 5
- Update runTask.ts to use configurable timeout from run settings
- Pass timeout parameter through the createRun action

* fix: remove unused EVALS_TIMEOUT import

* fix: add timeout field to createRun calls in copyRun test

- Added timeout: 5 to both createRun calls in copyRun.spec.ts
- This fixes the test failure caused by the new required timeout field in the runs schema
- The timeout field was added in the configurable timeout feature but the test was not updated

* fix: use configurable timeout for Redis key expiration in registerRunner

- Updated registerRunner function to accept timeoutSeconds parameter
- Modified call in runTask.ts to pass configurable timeout instead of hardcoded EVALS_TIMEOUT
- Removed unused EVALS_TIMEOUT import from redis.ts
- Ensures Redis keys remain valid for the entire duration of task execution (up to 10 minutes)

---------

Co-authored-by: Roo Code <roomote@roocode.com>
Co-authored-by: hannesrudolph <hrudolph@gmail.com>
2025-07-19 22:20:23 -04:00
Chris Estreich
53e6419fab
Fix roocode.com Vercel preview on PR open (#4916) 2025-06-19 20:00:11 -07:00
renovate[bot]
5c7cae0c48
fix(deps): update dependency lucide-react to ^0.518.0 (#4820)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-06-19 00:52:01 -07:00
renovate[bot]
8d61bfb16f
fix(deps): update dependency lucide-react to ^0.516.0 (#4612)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-06-17 15:33:47 -07:00
Chris Estreich
e4e75648d3
Ignore logs in git (#4797) 2025-06-17 12:03:00 -07:00
Chris Estreich
62c3914034
Farewell jest (#4607)
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-16 21:39:45 -07:00
Chris Estreich
395f55b31f
Convert jest tests to vitest and disable default watch mode for vitest (#4568)
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-11 14:48:01 -07:00
Chris Estreich
ab01fb3bdb
Move to Postgres for evals on roocode.com (#4520) 2025-06-11 14:17:53 -07:00
Chris Estreich
8d5dab3518
GHA evals (#4472) 2025-06-10 11:01:54 -07:00
Chris Estreich
52673b3721
Harden evals with retry logic + centralize logs on Docker host (#4440)
* Harden evals with retry logic + centralize logs on Docker host

* Don't delete controller container on exit

* Add randomness

* More tweaks

* More tweaks
2025-06-07 08:58:09 -07:00
Chris Estreich
24851b96db
Improve the evals "run status" section (#4400) 2025-06-05 22:00:15 -07:00
Chris Estreich
cb5b9c3718
Improve Docker setup for evals (#4327) 2025-06-05 12:16:36 -07:00
renovate[bot]
fe26c05dc9
fix(deps): update dependency lucide-react to ^0.513.0 (#4345)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-06-04 16:56:35 -04:00
Chris Estreich
73f39616c3
Improve turbo task dependency declarations (#4323) 2025-06-04 09:32:27 -07:00
Chris Estreich
01eef2be99
Bring back #4128 (#4292) 2025-06-03 14:03:19 -07:00
Chris Estreich
d87f890556
Move evals into pnpm workspace, switch from SQLite to Postgres (#4278) 2025-06-03 01:38:08 -07:00