ReMe/reme/core/utils
jinliyl 6dd987a1d2
refactor(core): update text truncation utilities and tool result handling (#168)
* refactor(memory): remove mark filtering parameters and simplify get_memory logic

* feat(core): bump version to 0.3.1.0

* refactor(memory): update comment to clarify dialog storage persistence

* refactor(core): update text truncation utilities and tool result handling

- Add new truncate_text_head function for head-based truncation
- Introduce TRUNCATION_MARKER_START constant for truncation detection
- Replace tail-based truncation with head-based truncation in tool result compaction
- Remove tool_result_threshold and retention_days parameters from RemeLight initialization
- Update ToolResultCompactor to use configurable thresholds for recent vs old messages
- Modify compact_tool_result method to accept multiple threshold parameters
- Adjust cleanup logic to use default compactor configuration
- Simplify is_truncated function to check only start marker

* ```
feat(memory): add long line splitting in tool result compaction

- Added _split_long_lines function to break oversized lines at 10000 characters
- Implemented line splitting before saving tool results to files
- Prevents extremely long lines from breaking file-based storage
- Maintains compatibility with existing tool result format
- Preserves original content integrity through chunked processing
```
2026-03-19 19:53:32 +08:00
..
__init__.py refactor(core): update text truncation utilities and tool result handling (#168) 2026-03-19 19:53:32 +08:00
agentscope_utils.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
cache_handler.py refactor(core): restructure core modules and update pre-commit configuration 2026-01-22 16:25:20 +08:00
case_converter.py refactor(core): restructure core modules and update pre-commit configuration 2026-01-22 16:25:20 +08:00
chunking_utils.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
common_utils.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
env_utils.py refactor(core): replace memory stores with file stores and update architecture 2026-02-26 14:18:12 +08:00
execute_utils.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
hf_token_counter_utils.py refactor(core): update registry registration syntax and improve code formatting 2026-03-06 01:57:29 +08:00
horse.py refactor(core): restructure application initialization and component management 2026-02-24 19:45:47 +08:00
http_client.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
llm_utils.py refactor(cli): using AgentScope components to reimplement the reme_cli logic (#153) 2026-03-12 11:23:36 +08:00
logger_utils.py fin(emb_dim) (#150) 2026-03-10 18:19:00 +08:00
logo_utils.py feat(cli): enable vector search and improve chat history management 2026-02-15 13:29:46 +08:00
mcp_client.py refactor(core): restructure application initialization and service context 2026-02-26 17:56:59 +08:00
pydantic_config_parser.py feat(core): upgrade version to 0.3.0.0b1 and add metadata configuration 2026-02-15 18:29:46 +08:00
pydantic_utils.py refactor(core): restructure core modules and update pre-commit configuration 2026-01-22 16:25:20 +08:00
singleton.py refactor(core): replace memory stores with file stores and update architecture 2026-02-26 14:18:12 +08:00
std_logger.py refactor(cli): using AgentScope components to reimplement the reme_cli logic (#153) 2026-03-12 11:23:36 +08:00
time.py refactor(core): simplify component registration and improve application lifecycle management 2026-01-31 01:05:31 +08:00
truncate_text_utils.py refactor(core): update text truncation utilities and tool result handling (#168) 2026-03-19 19:53:32 +08:00