ReMe/reme/core/utils
jinliyl dc8eab56a1
refactor(core): replace text truncation utilities with new marker system (#179)
* refactor(core): replace text truncation utilities with new marker system

- Remove old truncate_text_utils module and its exports
- Replace TRUNCATION_MARKER_START with _TRUNCATION_NOTICE_MARKER constant
- Update as_msg_stat.py to split content using new marker format
- Modify FileIO tool to use TRUNCATION_NOTICE_MARKER for continuation hints
- Change is_truncated function checks to use marker presence detection
- Move transformers dependency from main deps to light extra dependencies
- Update tool result compactor tests to verify marker instead of is_truncated calls

* feat(file_io): enhance file operations with path resolution and append functionality

- Add expanduser() to resolve file paths with ~ symbol
- Implement proper file existence and type validation in update_file
- Add new append_file method to append content to files
- Update truncation notice format for better readability
- Fix typo in error message from "provide" to "provided"
- Update transformers dependency in pyproject.toml
- Remove duplicate transformers dependency from light extras

* refactor(file_io): disable pylint too-many-return-statements warning

* perf(file_watcher): increase default polling delay and optimize watcher configuration

- Increased default poll_delay_ms from 1000ms to 2000ms to reduce CPU usage
- Removed force_polling parameter as it's no longer needed with updated polling strategy
- Simplified async watch configuration by removing conditional force_polling logic
- Reduced overall system resource consumption during file watching operations

* refactor(memory): update conversation log documentation in memory summary

- Changed "Raw conversation logs" to "Earlier conversation logs" for clarity
- Added warning note about potentially large dialog file sizes
- Improved formatting with additional line break for better readability
- Maintained existing compressed summary integration unchanged

* feat(memory): add long-term memory support to file-based memory system

- Initialize _long_term_memory attribute as empty string
- Add memories section to content when long-term memory exists
- Consolidate summary and memories into single user message
- Format memories with markdown header # Memories
- Maintain existing compressed summary functionality
- Join multiple content parts with double newlines
2026-03-26 12:10:24 +08:00
..
__init__.py refactor(core): replace text truncation utilities with new marker system (#179) 2026-03-26 12:10:24 +08:00
agentscope_utils.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
cache_handler.py refactor(core): restructure core modules and update pre-commit configuration 2026-01-22 16:25:20 +08:00
case_converter.py refactor(core): restructure core modules and update pre-commit configuration 2026-01-22 16:25:20 +08:00
chunking_utils.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
common_utils.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
env_utils.py refactor(core): replace memory stores with file stores and update architecture 2026-02-26 14:18:12 +08:00
execute_utils.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
hf_token_counter_utils.py refactor(core): update registry registration syntax and improve code formatting 2026-03-06 01:57:29 +08:00
horse.py refactor(core): restructure application initialization and component management 2026-02-24 19:45:47 +08:00
http_client.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
llm_utils.py refactor(cli): using AgentScope components to reimplement the reme_cli logic (#153) 2026-03-12 11:23:36 +08:00
logger_utils.py fin(emb_dim) (#150) 2026-03-10 18:19:00 +08:00
logo_utils.py feat(cli): enable vector search and improve chat history management 2026-02-15 13:29:46 +08:00
mcp_client.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
pydantic_config_parser.py feat(core): upgrade version to 0.3.0.0b1 and add metadata configuration 2026-02-15 18:29:46 +08:00
pydantic_utils.py refactor(core): restructure core modules and update pre-commit configuration 2026-01-22 16:25:20 +08:00
singleton.py refactor(core): replace memory stores with file stores and update architecture 2026-02-26 14:18:12 +08:00
std_logger.py refactor(cli): using AgentScope components to reimplement the reme_cli logic (#153) 2026-03-12 11:23:36 +08:00
time.py refactor(core): simplify component registration and improve application lifecycle management 2026-01-31 01:05:31 +08:00