* feat(condense): add smart code folding with tree-sitter signatures
At context condensation time, use tree-sitter to generate folded code
signatures (function definitions, class declarations) for files read
during the conversation. Each file is included as its own <system-reminder>
block in the condensed summary, preserving structural awareness without
consuming excessive tokens.
- Add getFilesReadByRoo() method to FileContextTracker
- Create generateFoldedFileContext() using tree-sitter parsing
- Update summarizeConversation() to accept array of file sections
- Each file gets its own content block in the summary message
- Add comprehensive test coverage (12 tests)
* fix: skip tree-sitter error strings in folded file context
- Add isTreeSitterErrorString helper to detect error messages
- Skip files that return error strings instead of embedding them
- Add test for error string handling
* refactor: move generateFoldedFileContext() inside summarizeConversation()
- Update summarizeConversation() to accept filesReadByRoo, cwd, rooIgnoreController instead of pre-generated sections
- Move folded file context generation inside summarizeConversation() (lines 319-339)
- Update ContextManagementOptions type and manageContext() to pass new parameters
- Remove generateFoldedFileContext from Task.ts imports - folding now handled internally
- Update all tests to use new parameter signature
- Reduces Task.ts complexity by moving folding logic to summarization module
* fix: prioritize most recently read files in folded context
Files are now sorted by roo_read_date descending before folded context
generation, so if the character budget runs out, the most relevant
(recently read) files are included and older files are skipped.
* refactor: improve code quality in condense module
- Convert summarizeConversation to use options object instead of 11 positional params
- Extract duplicated getFilesReadByRoo error handling into helper method
- Remove unnecessary re-export of generateFoldedFileContext
- Update all test files to use new options object pattern
* fix: address roomote feedback - batch error logging and early budget exit
---------
Co-authored-by: Roo Code <roomote@roocode.com>
Co-authored-by: daniel-lxs <ricciodaniel98@gmail.com>
* feat: Add safeWriteJson utility for atomic file operations
Implements a robust JSON file writing utility that:
- Prevents concurrent writes to the same file using in-memory locks
- Ensures atomic operations with temporary file and backup strategies
- Handles error cases with proper rollback mechanisms
- Cleans up temporary files even when operations fail
- Provides comprehensive test coverage for success and failure scenarios
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: use safeWriteJson for all JSON file writes
This change refactors all direct JSON file writes to use the safeWriteJson
utility, which implements atomic file writes to prevent data corruption
during write operations.
- Modified safeWriteJson to accept optional replacer and space arguments
- Updated tests to verify correct behavior with the new implementation
Fixes: #722
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: Implement inter-process file locking for safeWriteJson
Replaces the previous in-memory lock in `safeWriteJson` with
`proper-lockfile` to provide robust, cross-process advisory file
locking. This enhances safety when multiple processes might attempt
concurrent writes to the same JSON file.
Key changes:
- Added `proper-lockfile` and `@types/proper-lockfile` dependencies.
- `safeWriteJson` now uses `proper-lockfile.lock()` with configured
retries, staleness checks (31s), and lock update intervals (10s).
- An `onCompromised` handler is included to manage scenarios where
the lock state is unexpectedly altered.
- Logging and comments within `safeWriteJson` have been refined for
clarity, ensuring error logs include backtraces.
- The test suite `safeWriteJson.test.ts` has been significantly
updated to:
- Use real timers (`jest.useRealTimers()`).
- Employ a more comprehensive mock for `fs/promises`.
- Correctly manage file pre-existence for various scenarios.
- Simulate lock contention by mocking `proper-lockfile.lock()`
using `jest.doMock` and a dynamic require for the SUT.
- Verify lock release by checking for the absence of the `.lock`
file.
All tests are passing with these changes.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: implement streaming JSON write in safeWriteJson
Refactor safeWriteJson to use stream-json for memory-efficient JSON serialization:
- Replace in-memory string creation with streaming pipeline
- Add Disassembler and Stringer from stream-json library
- Extract streaming logic to a dedicated helper function
- Add proper-lockfile and stream-json dependencies
This implementation reduces memory usage when writing large JSON objects.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: improve safeWriteJson locking mechanism
- Use file path itself for locking instead of separate lock file
- Improve error handling and clarity of code
- Enhance cleanup of temporary files
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: fix safeWriteJson test failures
- Ensure test file exists before locking
- Add proper mocking for fs.createWriteStream
- Fix test assertions to match expected behavior
- Improve test comments to follow project guidelines
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: update tests to work with safeWriteJson
Updated tests to work with safeWriteJson instead of direct fs.writeFile calls:
- Updated importExport.test.ts to expect safeWriteJson calls instead of fs.writeFile
- Fixed McpHub.test.ts by properly mocking fs/promises module:
- Moved jest.mock() to the top of the file before any imports
- Added mock implementations for all fs functions used by safeWriteJson
- Updated the test setup to work with the mocked fs module
All tests now pass successfully.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: replace JSON.stringify with safeWriteJson for file operations
Replace all non-test instances of JSON.stringify used for writing to JSON files with safeWriteJson to ensure safer file operations with proper locking, error handling, and atomic writes.
- Updated src/services/mcp/McpHub.ts
- Updated src/services/code-index/cache-manager.ts
- Updated src/api/providers/fetchers/modelEndpointCache.ts
- Updated src/api/providers/fetchers/modelCache.ts
- Updated tests to match the new implementation
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* docs: add rules for using safeWriteJson
Add concise rules for using safeWriteJson instead of JSON.stringify with file operations to ensure atomic writes and prevent data corruption.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
---------
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Daniel <57051444+daniel-lxs@users.noreply.github.com>
* feat: Add file context tracking system
This commit adds a comprehensive file context tracking system that monitors file operations (reads, edits) by both Roo and users. The system helps prevent stale context issues and improves checkpoint management.
Key features:
- Track files accessed via tools, mentions, or edits
- Monitor file changes outside of Roo using file watchers
- Store file operation metadata with timestamps
- Trigger checkpoints automatically when files are modified
- Prevent false positives by distinguishing between Roo and user edits
The implementation includes:
- New FileContextTracker class to manage file operations
- Type definitions for file metadata tracking
- Integration with all file-related tools
- File mention tracking in the mentions system
- Improved checkpoint triggering based on file modifications
* Update src/core/context-tracking/FileContextTracker.ts
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update src/core/context-tracking/FileContextTracker.ts
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* test: Add mocks for getFileContextTracker in Cline tests
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
