mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-08-28 05:25:04 +00:00
docs(readme): update documentation and examples
This commit is contained in:
jinli.yl
parent
6b742b6719
commit
dcf97dc77f
3 changed files with 102 additions and 18 deletions
27
README.md
27
README.md
|
|
@ -66,15 +66,15 @@ working_dir/
|
|||
[ReMeLight](reme/reme_light.py) is the core class of this memory system, providing complete memory management
|
||||
capabilities for AI Agents:
|
||||
|
||||
| Method | Function | Key Components |
|
||||
|------------------------|------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| `start` | 🚀 Start memory system | Initialize file store, file watcher, Embedding cache; clean up expired tool result files |
|
||||
| `close` | 📕 Close and clean up | Clean tool result files, stop file watcher, save Embedding cache |
|
||||
| `compact_memory` | 📦 Compact history to summary | [Compactor](reme/memory/file_based/compactor.py) — ReActAgent generates structured context checkpoint |
|
||||
| `summary_memory` | 📝 Write important memory to files | [Summarizer](reme/memory/file_based/summarizer.py) — ReActAgent + file tools (read / write / edit) |
|
||||
| `compact_tool_result` | ✂️ Compact oversized tool output | [ToolResultCompactor](reme/memory/file_based/tool_result_compactor.py) — Truncate and save to `tool_result/`, keep file reference in message |
|
||||
| `pre_reasoning_hook` | 🔄 Pre-reasoning hook | Auto compact tool results + generate summary + async trigger memory summarization task |
|
||||
| `memory_search` | 🔍 Semantic memory search | [MemorySearch](reme/memory/tools/chunk/memory_search.py) — Vector + BM25 hybrid retrieval |
|
||||
| Method | Function | Key Components |
|
||||
|------------------------|------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| `start` | 🚀 Start memory system | Initialize file store, file watcher, Embedding cache; clean up expired tool result files |
|
||||
| `close` | 📕 Close and clean up | Clean tool result files, stop file watcher, save Embedding cache |
|
||||
| `compact_memory` | 📦 Compact history to summary | [Compactor](reme/memory/file_based/compactor.py) — ReActAgent generates structured context checkpoint |
|
||||
| `summary_memory` | 📝 Write important memory to files | [Summarizer](reme/memory/file_based/summarizer.py) — ReActAgent + file tools (read / write / edit) |
|
||||
| `compact_tool_result` | ✂️ Compact oversized tool output | [ToolResultCompactor](reme/memory/file_based/tool_result_compactor.py) — Truncate and save to `tool_result/`, keep file reference in message |
|
||||
| `pre_reasoning_hook` | 🔄 Pre-reasoning hook | Auto compact tool results + generate summary + async trigger memory summarization task |
|
||||
| `memory_search` | 🔍 Semantic memory search | [MemorySearch](reme/memory/tools/chunk/memory_search.py) — Vector + BM25 hybrid retrieval |
|
||||
| `get_in_memory_memory` | 🗂️ Create in-memory instance | [ReMeInMemoryMemory](reme/memory/file_based/reme_in_memory_memory.py) — Token-aware memory management, supports compression summary and state serialization (static method) |
|
||||
|
||||
---
|
||||
|
|
@ -125,9 +125,9 @@ async def main():
|
|||
summary = await reme.compact_memory(
|
||||
messages=messages,
|
||||
previous_summary="",
|
||||
max_input_length=128000, # Model context window (tokens)
|
||||
compact_ratio=0.7, # Trigger compaction when reaching max_input_length * 0.7
|
||||
language="zh", # Summary language (zh / "")
|
||||
max_input_length=128000, # Model context window (tokens)
|
||||
compact_ratio=0.7, # Trigger compaction when reaching max_input_length * 0.7
|
||||
language="zh", # Summary language (zh / "")
|
||||
)
|
||||
|
||||
# 3. Submit async summary task in background (non-blocking, writes to memory/YYYY-MM-DD.md)
|
||||
|
|
@ -169,7 +169,8 @@ if __name__ == "__main__":
|
|||
```
|
||||
|
||||
> 📂 Full example code: [test_reme_light.py](tests/light/test_reme_light.py)
|
||||
> 📋 Example output: [test_reme_light.log](tests/light/test_reme_light.log) (223,838 tokens → 1,105 tokens, 99.5% compression ratio)
|
||||
> 📋 Example output: [test_reme_light.log](tests/light/test_reme_light_log.txt) (223,838 tokens → 1,105 tokens, 99.5%
|
||||
> compression ratio)
|
||||
|
||||
### File-Based ReMeLight Memory System Architecture
|
||||
|
||||
|
|
|
|||
|
|
@ -97,7 +97,6 @@ pip install -e ".[light]"
|
|||
```python
|
||||
import asyncio
|
||||
|
||||
from agentscope.message import Msg
|
||||
from reme.reme_light import ReMeLight
|
||||
|
||||
|
||||
|
|
@ -119,9 +118,9 @@ async def main():
|
|||
summary = await reme.compact_memory(
|
||||
messages=messages,
|
||||
previous_summary="",
|
||||
max_input_length=128000, # 模型上下文窗口（tokens）
|
||||
compact_ratio=0.7, # 达到 max_input_length * 0.7 时触发压缩
|
||||
language="zh", # 摘要语言（zh / ""）
|
||||
max_input_length=128000, # 模型上下文窗口（tokens）
|
||||
compact_ratio=0.7, # 达到 max_input_length * 0.7 时触发压缩
|
||||
language="zh", # 摘要语言（zh / ""）
|
||||
)
|
||||
|
||||
# 3. 后台异步提交摘要任务（不阻塞对话，摘要写入 memory/YYYY-MM-DD.md）
|
||||
|
|
@ -163,7 +162,7 @@ if __name__ == "__main__":
|
|||
```
|
||||
|
||||
> 📂 完整示例代码：[test_reme_light.py](tests/light/test_reme_light.py)
|
||||
> 📋 运行结果示例：[test_reme_light.log](tests/light/test_reme_light.log)（223,838 tokens → 1,105 tokens，压缩率 99.5%）
|
||||
> 📋 运行结果示例：[test_reme_light.log](tests/light/test_reme_light_log.txt)（223,838 tokens → 1,105 tokens，压缩率 99.5%）
|
||||
|
||||
### 基于文件的 ReMeLight 记忆系统架构
|
||||
|
||||
|
|
|
|||
84
tests/light/test_reme_light_log.txt
Normal file
84
tests/light/test_reme_light_log.txt
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
======================================================================
|
||||
ReMeLight 已启动
|
||||
======================================================================
|
||||
|
||||
[原始消息]: 18 条, 223,838 tokens
|
||||
目标阈值: 128K = 131,072 tokens
|
||||
超出阈值: True
|
||||
|
||||
======================================================================
|
||||
[步骤 1] compact_tool_result - 压缩超长工具输出
|
||||
======================================================================
|
||||
消息数量: 18 → 18
|
||||
📊 Token 统计: 223,838 → 1,107 (变化: -222,731, -99.5%)
|
||||
|
||||
======================================================================
|
||||
[步骤 2] compact_memory - 生成结构化压缩摘要
|
||||
======================================================================
|
||||
输入消息 tokens: 223,838
|
||||
压缩摘要长度: 1032 字符, 493 tokens
|
||||
压缩比: 0.2%
|
||||
|
||||
======================================================================
|
||||
[步骤 3] summary_memory - 生成完整摘要并写入文件
|
||||
======================================================================
|
||||
|
||||
reme_summarizer: [SILENT]
|
||||
输入消息 tokens: 223,838
|
||||
摘要结果长度: 8 字符
|
||||
摘要: [SILENT]
|
||||
|
||||
======================================================================
|
||||
[步骤 4] pre_reasoning_hook - 推理前预处理
|
||||
======================================================================
|
||||
消息数量: 18 → 18
|
||||
📊 Token 统计: 223,838 → 1,105 (变化: -222,733, -99.5%)
|
||||
压缩摘要: 0 字符, 0 tokens
|
||||
总上下文: 1,105 tokens
|
||||
|
||||
======================================================================
|
||||
[步骤 5] memory_search - 语义搜索记忆
|
||||
======================================================================
|
||||
搜索结果: [{'type': 'text', 'text': '[\n {\n "path": "/Users...
|
||||
|
||||
======================================================================
|
||||
[步骤 6] ReMeInMemoryMemory - 会话内存管理
|
||||
======================================================================
|
||||
已添加 18 条原始消息到内存
|
||||
|
||||
[6.1] estimate_tokens - 估算 Token 使用:
|
||||
- 总消息数: 18
|
||||
- 消息 Token 数: 223,838
|
||||
- 压缩摘要 Token 数: 0
|
||||
- 预估总 Token 数: 223,838
|
||||
- 最大输入长度: 128,000
|
||||
- 上下文使用率: 174.87%
|
||||
|
||||
[6.2] get_history_str - 格式化历史记录:
|
||||
**Conversation History**
|
||||
|
||||
- Total messages: 18
|
||||
- Estimated tokens: 223838
|
||||
- Max input length: 128000
|
||||
- Context usage: 174.9%
|
||||
- Compressed summary tokens: 0
|
||||
...
|
||||
|
||||
======================================================================
|
||||
[步骤 7] 等待后台任务完成
|
||||
======================================================================
|
||||
后台任务完成，结果长度: 0 字符
|
||||
|
||||
======================================================================
|
||||
📊 Token 变化总结
|
||||
======================================================================
|
||||
原始消息: 223,838 tokens
|
||||
Step 1 compact_tool_result 后: 1,107 tokens
|
||||
Step 2 compact_memory 摘要: 493 tokens
|
||||
Step 4 pre_reasoning_hook 后: 1,105 tokens + 摘要 0 tokens = 1,105 tokens
|
||||
最大节省: 222,733 tokens (99.5%)
|
||||
目标阈值: 131,072 tokens
|
||||
|
||||
======================================================================
|
||||
ReMeLight 已关闭
|
||||
======================================================================
|
||||
Loading…
Add table
Reference in a new issue