scriptoriumadmin/Roo-Code
1
0
Fork 0
mirror of https://github.com/RooVetGit/Roo-Code.git synced 2026-08-28 05:27:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update evals README.md (#4420)

Browse source 
* Update README.md

* Update README.md
This commit is contained in:
Chris Estreich 2025-06-06 09:04:11 -07:00 • committed by GitHub
parent f61b4e46a8
commit 39d7f559d2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 22 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
packages/evals/README.md
View file

@ -47,6 +47,28 @@ After clicking "Launch" you should find that a "controller" container has spawne
The web app's UI should update in realtime with the results of the eval run:
<img width="1053" alt="Screenshot 2025-06-05 at 12 14 52 PM" src="https://github.com/user-attachments/assets/6fe3b651-0898-4f14-a231-3cc8d66f0e1f" />
## Resource Usage
If you want to run evals with high parallelism by increasing the concurrency you need to be mindful of your Docker resource limits.
We've found the following formula to be helpful in practice:
```
Memory Limit = 3GB * concurrency
CPU Limit = 2 * concurrency
```
The memory and CPU limits can be set from the "Resources" section of the Docker Desktop settings:
<img width="996" alt="Screenshot 2025-06-06 at 8 54 24 AM" src="https://github.com/user-attachments/assets/a1cbb27d-b09c-450c-9fa8-b662c0537d48" />
## Stopping
To stop an evals run early you can simply stop the "controller" container using Docker Desktop. This will prevent any new task containers from being spawned. You can optionally stop any existing task containers immediately or let them finish their current tasks at which point they will exit.
<img width="1302" alt="Screenshot 2025-06-06 at 9 00 41 AM" src="https://github.com/user-attachments/assets/a9d4725b-730c-441a-ba24-ac99f9599ced" />
## Advanced Usage / Debugging
The evals system runs VS Code headlessly in Docker containers for consistent, reproducible environments. While this design ensures reliability, it can make debugging more challenging. For debugging purposes, you can run the system locally on macOS, though this approach is less reliable due to hardware and environment variability.