Roo-Code/evals
Yikai Liao b5ffaf1ba2
Fix Terminal Carriage Return Handling for Correct Progress Bar Display (#2562)
* fix(terminal): Ensure correct handling of carriage returns for progress bars

This commit refines the tests for `TerminalProcess` to ensure the correct interpretation of terminal output containing carriage returns (`\\r`), which is essential for properly handling dynamic elements like progress bars (e.g., `tqdm`).

- Validated the `processCarriageReturns` method's behavior in simulating terminal line overwrites caused by `\\r`.
- Corrected the expectation in the `handles carriage returns in mixed content` test to accurately reflect the method's output (final line content + preserved escape sequences), confirming the logic works as intended for progress-bar-like updates.
- Fixed a minor Jest `toBe` syntax error in a related test case.
- Suppressed an expected `console.warn` in the non-shell-integration test for cleaner logs.

By ensuring `processCarriageReturns` is correctly tested, we increase confidence that the component responsible for pre-processing terminal output handles progress bars appropriately before the output is potentially used elsewhere (e.g., sent to an LLM).

* fix(test): Make TerminalProcess integration test reliable

This commit fixes the flaky test case `integrates with getUnretrievedOutput to handle progress bars` in `TerminalProcess.test.ts`.

The test previously failed intermittently due to:
1.  Relying on a fixed `setTimeout` duration to wait for asynchronous stream processing, which created a race condition.
2.  Incorrectly assuming that `await terminalProcess.run(...)` would return the final output directly via its resolved value.

The fix addresses these issues by:
-   Removing the unreliable intermediate check based on `setTimeout`.
-   Modifying the test to correctly obtain the final output by listening for the `completed` event emitted by `TerminalProcess`, which is the intended way to receive the result.

This ensures the test accurately reflects the behavior of `TerminalProcess` and is no longer prone to timing-related failures.

* Add changeset for terminal carriage return fix

* Implement terminal compress progress bar feature

This commit introduces a new feature to compress terminal output by processing carriage returns. The `processCarriageReturns` function has been integrated into the `Terminal` class to handle progress bar updates effectively, ensuring only the final state is displayed.

Additionally, the `terminalCompressProgressBar` setting has been added to the global settings schema, allowing users to enable or disable this feature.

Tests have been updated to validate the new functionality and ensure correct behavior in various scenarios.

A Benchmark is also added to test the performance.

Not that there is still no i18n support for this.

* Add i18n support for compressProgressBar setting in multiple languages

* Optimize processCarriageReturns function for performance and multi-byte character handling

This commit enhances the `processCarriageReturns` function by implementing in-place string operations to improve performance, especially with large outputs. Key features include:
- Line-by-line processing to maximize chunk handling.
- Use of string indexes and substring operations instead of arrays.
- Single-pass traversal of input for efficiency.
- Special handling for multi-byte characters to prevent corruption during overwrites.

Additionally, tests have been updated to validate the new functionality, ensuring correct behavior with various character sets, including emojis and non-ASCII text.

Highly Density CR case is added to Benchmark

* slight performance improvement by caching several variable

* Optimize multi-byte character handling in processCarriageReturns

Refactor the logic within the `processCarriageReturns` function to simplify the detection of partially overwritten multi-byte characters (e.g., emojis).

Removed redundant checks and clarified the conditions for identifying potential character corruption during carriage return processing. This improves code readability and maintainability while preserving the original functionality of replacing potentially corrupted characters with a space.

Also enforced consistent use of semicolons for improved code style.

* docs: standardize carriage return (\r) and line feed (\n) terminology
Improve code clarity by consistently adding escape sequence notation to all
references of carriage returns and line feeds throughout documentation and tests.
This makes the code more readable and avoids ambiguity when discussing these special characters.

* feat: Improve terminal output processing clarity and settings UI

- Add detailed comments to `processCarriageReturns` explaining line feed handling.
- Relocate `terminalCompressProgressBar` setting below `terminalOutputLineLimit` for better context in UI.

* Fix: Compress Progress Bar Setting Checkbox

---------

Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
2025-04-24 00:22:47 -04:00
..
apps Fix Terminal Carriage Return Handling for Correct Progress Bar Display (#2562) 2025-04-24 00:22:47 -04:00
config Evals (#2238) 2025-04-02 16:49:41 -07:00
packages Feat/improve insert block content (#2510) 2025-04-21 16:49:27 -04:00
scripts Evals improvements (#2555) 2025-04-13 00:57:33 -07:00
.env.sample Evals (#2238) 2025-04-02 16:49:41 -07:00
.gitignore Evals (#2238) 2025-04-02 16:49:41 -07:00
.npmrc Evals (#2238) 2025-04-02 16:49:41 -07:00
.tool-versions Evals enhancements: delete runs, show all run instead of just completed runs (#2520) 2025-04-12 00:07:30 -07:00
package.json Control evals concurrency in web app (#2265) 2025-04-11 22:21:15 -07:00
pnpm-lock.yaml Record tool usages in the Cline object, and persist them in the db for evals (#2729) 2025-04-17 22:21:14 -07:00
pnpm-workspace.yaml Evals (#2238) 2025-04-02 16:49:41 -07:00
README.md Evals (#2238) 2025-04-02 16:49:41 -07:00
turbo.json Record tool usages in the Cline object, and persist them in the db for evals (#2729) 2025-04-17 22:21:14 -07:00

Run Roo Code Evals

Get Started

NOTE: This is MacOS only for now!

Clone the Roo Code repo:

git clone https://github.com/RooVetGit/Roo-Code.git
cd Roo-Code

Run the setup script:

cd evals
./scripts/setup.sh

Navigate to localhost:3000 in your browser.