Commit graph

5 commits

Author SHA1 Message Date
jinliyl
37628ba524
refactor(truncation): improve file truncation logic (#184)
* refactor(file_store): simplify ChromaDB client initialization and improve file truncation logic

- Remove shutil import and _create_chroma_client method from chroma_file_store.py
- Directly initialize ChromaDB PersistentClient in start method without retry logic
- Reduce DEFAULT_MAX_BYTES from 100KB to 50KB in file_utils.py
- Update truncation notice format to provide clearer continuation instructions
- Add _truncate_fresh and _retruncate functions for better text truncation handling
- Replace inline truncation logic with dedicated function calls in file_utils.py
- Rename skills_tool_ids to md_file_tool_ids in tool_result_compactor.py
- Update file detection logic to identify any .md files instead of only skill.md
- Create comprehensive unit tests for truncation functionality in test_truncate_text_output.py

* chore(version): bump version to 0.3.1.6

- Update __version__ from 0.3.1.5 to 0.3.1.6 in __init__.py
2026-03-28 18:41:09 +08:00
jinliyl
ff49a77f18
feat(memory): improve skills tool result truncation (#182)
* fix(memory): correct line numbering and improve tool result truncation

- Changed default start_line from 0 to 1 in truncate_text_output function
- Refactored _truncate method to be a standalone method in ToolResultCompactor
- Improved tool result compaction logic to handle text blocks more efficiently
- Added detection of skill-related tool calls for special handling
- Implemented conditional byte limits based on tool type for better memory management
- Updated version number from 0.3.1.4 to 0.3.1.5

* feat(file_utils): add encoding parameter to truncate_text_output function

- Added encoding parameter with default value "utf-8" to truncate_text_output function
- Updated all encode/decode calls to use the specified encoding parameter
- Modified ToolResultCompactor to pass encoding parameter when calling truncate_text_output
- Added error handling for skill tool ID detection in message processing loop
- Fixed potential AttributeError when accessing raw_input field that might be None

* fix(file-store): handle corrupted ChromaDB initialization and improve tool result truncation

- Add shutil import for directory removal operations
- Extract ChromaDB client creation into separate _create_chroma_client method
- Implement retry mechanism with database wipe on ChromaDB initialization failure
- Add proper exception handling in tool result compaction to prevent truncation errors
- Move file writing logic outside of exception handling scope for better error management
- Add warning log when truncation fails and return original content as fallback
2026-03-27 21:14:44 +08:00
jinliyl
dc8eab56a1
refactor(core): replace text truncation utilities with new marker system (#179)
* refactor(core): replace text truncation utilities with new marker system

- Remove old truncate_text_utils module and its exports
- Replace TRUNCATION_MARKER_START with _TRUNCATION_NOTICE_MARKER constant
- Update as_msg_stat.py to split content using new marker format
- Modify FileIO tool to use TRUNCATION_NOTICE_MARKER for continuation hints
- Change is_truncated function checks to use marker presence detection
- Move transformers dependency from main deps to light extra dependencies
- Update tool result compactor tests to verify marker instead of is_truncated calls

* feat(file_io): enhance file operations with path resolution and append functionality

- Add expanduser() to resolve file paths with ~ symbol
- Implement proper file existence and type validation in update_file
- Add new append_file method to append content to files
- Update truncation notice format for better readability
- Fix typo in error message from "provide" to "provided"
- Update transformers dependency in pyproject.toml
- Remove duplicate transformers dependency from light extras

* refactor(file_io): disable pylint too-many-return-statements warning

* perf(file_watcher): increase default polling delay and optimize watcher configuration

- Increased default poll_delay_ms from 1000ms to 2000ms to reduce CPU usage
- Removed force_polling parameter as it's no longer needed with updated polling strategy
- Simplified async watch configuration by removing conditional force_polling logic
- Reduced overall system resource consumption during file watching operations

* refactor(memory): update conversation log documentation in memory summary

- Changed "Raw conversation logs" to "Earlier conversation logs" for clarity
- Added warning note about potentially large dialog file sizes
- Improved formatting with additional line break for better readability
- Maintained existing compressed summary integration unchanged

* feat(memory): add long-term memory support to file-based memory system

- Initialize _long_term_memory attribute as empty string
- Add memories section to content when long-term memory exists
- Consolidate summary and memories into single user message
- Format memories with markdown header # Memories
- Maintain existing compressed summary functionality
- Join multiple content parts with double newlines
2026-03-26 12:10:24 +08:00
jinliyl
5b801c0d3e
refactor(file_io): update file I/O operations and truncation logic (#177)
* refactor(file_io): update file I/O operations and truncation logic

* refactor(memory): update file-based memory compaction logic
2026-03-25 20:21:37 +08:00
jinli.yl
d62c6a22a5 refactor(memory): move file utility functions to shared module 2026-03-07 15:30:45 +08:00
Renamed from reme/memory/file_based/tools/utils.py (Browse further)