Follow-up to #2562 adding support for backspace character compression.
Optimizes terminal output by handling backspace characters similar to
carriage returns, improving readability of progress spinners and other
terminal output that uses backspace for animation.
- Added processBackspaces function using efficient indexOf approach
- Added comprehensive test suite for backspace handling
- Integrated with terminal output compression pipeline
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix(terminal): Ensure correct handling of carriage returns for progress bars
This commit refines the tests for `TerminalProcess` to ensure the correct interpretation of terminal output containing carriage returns (`\\r`), which is essential for properly handling dynamic elements like progress bars (e.g., `tqdm`).
- Validated the `processCarriageReturns` method's behavior in simulating terminal line overwrites caused by `\\r`.
- Corrected the expectation in the `handles carriage returns in mixed content` test to accurately reflect the method's output (final line content + preserved escape sequences), confirming the logic works as intended for progress-bar-like updates.
- Fixed a minor Jest `toBe` syntax error in a related test case.
- Suppressed an expected `console.warn` in the non-shell-integration test for cleaner logs.
By ensuring `processCarriageReturns` is correctly tested, we increase confidence that the component responsible for pre-processing terminal output handles progress bars appropriately before the output is potentially used elsewhere (e.g., sent to an LLM).
* fix(test): Make TerminalProcess integration test reliable
This commit fixes the flaky test case `integrates with getUnretrievedOutput to handle progress bars` in `TerminalProcess.test.ts`.
The test previously failed intermittently due to:
1. Relying on a fixed `setTimeout` duration to wait for asynchronous stream processing, which created a race condition.
2. Incorrectly assuming that `await terminalProcess.run(...)` would return the final output directly via its resolved value.
The fix addresses these issues by:
- Removing the unreliable intermediate check based on `setTimeout`.
- Modifying the test to correctly obtain the final output by listening for the `completed` event emitted by `TerminalProcess`, which is the intended way to receive the result.
This ensures the test accurately reflects the behavior of `TerminalProcess` and is no longer prone to timing-related failures.
* Add changeset for terminal carriage return fix
* Implement terminal compress progress bar feature
This commit introduces a new feature to compress terminal output by processing carriage returns. The `processCarriageReturns` function has been integrated into the `Terminal` class to handle progress bar updates effectively, ensuring only the final state is displayed.
Additionally, the `terminalCompressProgressBar` setting has been added to the global settings schema, allowing users to enable or disable this feature.
Tests have been updated to validate the new functionality and ensure correct behavior in various scenarios.
A Benchmark is also added to test the performance.
Not that there is still no i18n support for this.
* Add i18n support for compressProgressBar setting in multiple languages
* Optimize processCarriageReturns function for performance and multi-byte character handling
This commit enhances the `processCarriageReturns` function by implementing in-place string operations to improve performance, especially with large outputs. Key features include:
- Line-by-line processing to maximize chunk handling.
- Use of string indexes and substring operations instead of arrays.
- Single-pass traversal of input for efficiency.
- Special handling for multi-byte characters to prevent corruption during overwrites.
Additionally, tests have been updated to validate the new functionality, ensuring correct behavior with various character sets, including emojis and non-ASCII text.
Highly Density CR case is added to Benchmark
* slight performance improvement by caching several variable
* Optimize multi-byte character handling in processCarriageReturns
Refactor the logic within the `processCarriageReturns` function to simplify the detection of partially overwritten multi-byte characters (e.g., emojis).
Removed redundant checks and clarified the conditions for identifying potential character corruption during carriage return processing. This improves code readability and maintainability while preserving the original functionality of replacing potentially corrupted characters with a space.
Also enforced consistent use of semicolons for improved code style.
* docs: standardize carriage return (\r) and line feed (\n) terminology
Improve code clarity by consistently adding escape sequence notation to all
references of carriage returns and line feeds throughout documentation and tests.
This makes the code more readable and avoids ambiguity when discussing these special characters.
* feat: Improve terminal output processing clarity and settings UI
- Add detailed comments to `processCarriageReturns` explaining line feed handling.
- Relocate `terminalCompressProgressBar` setting below `terminalOutputLineLimit` for better context in UI.
* Fix: Compress Progress Bar Setting Checkbox
---------
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
* fix: allow opening files without workspace root
The openFile function in open-file.ts was requiring a workspace root to be present,
which prevented opening global files (like MCP settings) when no workspace was open.
Modified the function to handle absolute paths without this requirement.
Previously, trying to open MCP settings in a new window without a workspace would
error with "Could not open file: No workspace root found". Now the function
properly handles both workspace-relative and absolute paths, allowing global
settings files to be accessed in any context.
Changes:
- Removed workspace root requirement in openFile
- Added fallback for relative paths when no workspace is present
* fix: update openFile function to use provided path without modification
---------
Co-authored-by: Roo Code <support@roo.vet>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
* Fix filename format in downloadTask function for markdown export
* Update src/integrations/misc/export-markdown.ts
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix: addLineNumbers handling of empty content
Empty files should not have line numbers, but non-empty files with empty content at a specific line offset should.
- If content is empty, return empty string for empty files
- If content is empty but startLine > 1, return line number for empty content at that offset
This ensures that the model does not think the file contains a single empty line.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: improve readFileTool XML output format
- Remove unnecessary XML indentation that could confuse the model
- Separate file content from notices and errors using dedicated tags
- Add line range information to content tags
- Handle empty files properly with self-closing tags
- Add comprehensive test coverage
Fixes#2278
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: always show line numbers in read_file XML output
- Always display line numbers in non-range reads
- Improve XML formatting with consistent newlines for better readability
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: update tests to match new XML format with line numbers
- Update test expectations to match the new XML format with newlines
- Update tests to expect line numbers attribute in content tags
- Modify test assertions to check for the correct line range values
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: consistent blank line handling in addLineNumbers
- Add newline to all output
- Handle trailing newlines and empty lines consistently
- Add test cases for blank lines:
- Multiple blank lines within content
- Multiple trailing blank lines
- Only blank lines with offset
- Trailing newlines
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: use actual addLineNumbers in read-file-xml tests
- Modified extract-text mock to preserve actual addLineNumbers implementation
- Removed mock implementation of addLineNumbers
- Updated test data to account for trailing newline
- Removed unnecessary mock verification
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: ensure actual addLineNumbers function is called in tests
- Replace direct mocking of addLineNumbers with spy on actual implementation
- Add verification to ensure the real function is called when appropriate
- Add skipAddLineNumbersCheck option for cases where function should not be called
- Update test cases to use appropriate verification options
- Fix numberedFileContent to include trailing newline for consistency
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: modify readLines to process data directly instead of line by line
- Direct data processing provides more accurate results by preserving exact content with carriage returns
- Improved performance through minimal buffering and efficient string operations
- Use string indexes to find newlines while maintaining their original format
- Handle all edge cases correctly with preserved line endings
- Add tests for various edge cases including empty files, single lines, and different line endings
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: remove unused mockInputContent variable
Remove unused variable declaration to appease ellipsis-dev linter requirements.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
---------
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
Improve handling of AI-generated line numbers:
- Clamp negative startLine values to 0 instead of throwing errors
- Convert non-integer line numbers to integers automatically
- Only reject when line numbers aren't numbers at all
These changes prevent unnecessary model retries by handling imperfect
AI-generated line numbers gracefully, making the system more robust
when interacting with AI models.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
* [clinerules] search clinerule in parent folders which make it easier to share common clinerules for a git repo
* fix indent
* Fix test
---------
Co-authored-by: Ying Liu <lying@microsoft.com>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
* feat read file range
feat: add file truncation with code structure preview
Add configurable line limit for large file handling with source code structure preview. This improves performance and memory usage while maintaining code readability.
Key changes:
Add maxReadFileLine setting (default: 4500) with UI controls
Create efficient line counter utility using streams
Add parseSourceCodeDefinitionsForFile for single file parsing
Show truncation notice with code structure for large files
Add comprehensive tests for new functionality
Move file extensions to shared constant
add line number
pr comment
Return definition ranges from tree-sitter
make treesitter always add line
add line number to rg search
enhance tree siter output & fix test
update snapshot
fix test
* Add translations, tweak range
---------
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
feat: add multiple workspaces support
- Add getWorkspacePath function to centralize workspace directory path retrieval
- Use the new workspace directory retrieval logic in Cline, Mentions, ClineProvider, and WorkspaceTracker
- Update WorkspaceFile on tab switch and prevent redundant updates by checking prevWorkSpacePath
- Fix the bug that loads the contents of the previous tab when quickly switching tabs
- Optimize getWorkspacePath return value for better reliability
Co-authored-by: xiong <yueminxiong.xym@alibaba-inc.com>
Implement applyRunLengthEncoding function to compress repeated lines in text output:
- Add line repetition compression with count message
- Focus on single line repetitions
- Only compress when beneficial
- Add tests for empty input and single line repetitions
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Use string indices to find line boundaries instead of splitting into array.
This avoids creating large arrays in memory when truncating big inputs.
- Replace split/join with indexOf/lastIndexOf for line counting
- Use slice to extract start/end sections directly from string
- Maintain same 20/80 ratio for before/after content
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
This reverts commit 7eee3e0878.
Middle-out truncation is a really great feature and it should still be
implemented, however it unnecessarily interferes with #1365 because it
hooked into the low-level chunk management that comes directly from VSCE
shell integration.
The best place to hook OutputBuilder is as follows depending on the
state of terminal interaction:
1. Foreground terminals:
Cline.ts:
executeCommandTool(...) {
process.on("line", (line) => {
lines.push(line)
...
}
}
2. For background terminals: hook in at the point that getUnretrievedOutput is consumed for active or
inactive terminals in Cline.ts:getEnvironmentDetails()
Please note:
The Terminal classes are very sensitive to change, partially because of
the complicated way that shell integration works with VSCE, and
partially because of the way that Cline interacts with the Terminal*
class abstractions that make VSCE shell integration easier to work with.
At the point that PR#1365 is merged, it is unlikely that any Terminal*
classes will need to be modified substantially. Generally speaking, we
should think of this is a stable interface and minimize changes.
Reverts: #1390
* Chore: Pretier for consistant formatting
- TODO: This PR needs to be updated by Saoud after he runs `npm install` & `npm format:fix` and commits the results of the prettier changes.
* Revert prettier config
* Run npm install
* Fix prettier config and ignore package lock
* Run format
---------
Co-authored-by: Saoud Rizwan <7799382+saoudrizwan@users.noreply.github.com>
