diff --git a/README.md b/README.md
index 7b5e4061..9d18e6ce 100644
--- a/README.md
+++ b/README.md
@@ -12,7 +12,7 @@
- 
+ 
@@ -20,66 +20,65 @@
A memory management toolkit for AI agents — Remember Me, Refine Me.
-> For legacy versions, see [0.2.x Documentation](docs/README_0_2_x.md)
+> For the older version, please refer to the [0.2.x documentation](docs/README_0_2_x_ZH.md).
---
-🧠 ReMe is a **memory management framework** built for **AI agents**, offering both **file-based** and **vector-based**
-memory systems.
+🧠 ReMe is a memory management framework designed for **AI agents**, providing both file-based and vector-based memory
+systems.
-It addresses two core problems of agent memory: **limited context windows** (early information gets truncated or lost
-during long conversations) and **stateless sessions** (new conversations cannot inherit history and always start from
-scratch).
-
-ReMe gives agents **real memory** — old conversations are automatically condensed, important information is persisted,
-and the next conversation can recall it automatically.
+It tackles two core problems of agent memory: **limited context window** (early information is truncated or lost in long
+conversations) and **stateless sessions** (new sessions cannot inherit history and always start from scratch).
+ReMe gives agents **real memory** — old conversations are automatically compacted, important information is persistently
+stored, and relevant context is automatically recalled in future interactions.
---
-## 📁 File-Based Memory System (ReMeLight)
+## 📁 File-based memory system (ReMeLight)
-> Memory as files, files as memory
+> Memory as files, files as memory.
-Treat **memory as files** — readable, editable, and portable.
-[CoPaw](https://github.com/agentscope-ai/CoPaw) implements long-term memory and context management by inheriting
+Treat **memory as files** — readable, editable, and copyable.
+[CoPaw](https://github.com/agentscope-ai/CoPaw) integrates long-term memory and context management by inheriting from
`ReMeLight`.
-| Traditional Memory Systems | File-Based ReMe |
-|----------------------------|--------------------|
-| 🗄️ Database storage | 📝 Markdown files |
-| 🔒 Opaque | 👀 Read anytime |
-| ❌ Hard to modify | ✏️ Edit directly |
-| 🚫 Hard to migrate | 📦 Copy to migrate |
+| Traditional memory system | File-based ReMe |
+|---------------------------|----------------------|
+| 🗄️ Database storage | 📝 Markdown files |
+| 🔒 Opaque | 👀 Always readable |
+| ❌ Hard to modify | ✏️ Directly editable |
+| 🚫 Hard to migrate | 📦 Copy to migrate |
```
working_dir/
-├── MEMORY.md # Long-term memory: user preferences, project config, etc.
+├── MEMORY.md # Long-term memory: persistent info such as user preferences
├── memory/
-│ └── YYYY-MM-DD.md # Daily summary logs: written automatically after conversation ends
-└── tool_result/ # Cache for oversized tool outputs (auto-managed, auto-cleaned when expired)
+│ └── YYYY-MM-DD.md # Daily journal: automatically written after each conversation
+└── tool_result/ # Cache for long tool outputs (auto-managed, expired entries auto-cleaned)
└── .txt
```
-### Core Capabilities
+### Core capabilities
-[ReMeLight](reme/reme_light.py) is the core class of this memory system, providing complete memory management
-capabilities for AI Agents:
+[ReMeLight](reme/reme_light.py) is the core class of the file-based memory system. It provides full memory management
+capabilities for AI agents:
-| Method | Function | Key Components |
-|------------------------|------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `start` | 🚀 Start memory system | Initialize file store, file watcher, Embedding cache; clean up expired tool result files |
-| `close` | 📕 Close and clean up | Clean tool result files, stop file watcher, save Embedding cache |
-| `compact_memory` | 📦 Compact history to summary | [Compactor](reme/memory/file_based/compactor.py) — ReActAgent generates structured context checkpoint |
-| `summary_memory` | 📝 Write important memory to files | [Summarizer](reme/memory/file_based/summarizer.py) — ReActAgent + file tools (read / write / edit) |
-| `compact_tool_result` | ✂️ Compact oversized tool output | [ToolResultCompactor](reme/memory/file_based/tool_result_compactor.py) — Truncate and save to `tool_result/`, keep file reference in message |
-| `pre_reasoning_hook` | 🔄 Pre-reasoning hook | Auto compact tool results + generate summary + async trigger memory summarization task |
-| `memory_search` | 🔍 Semantic memory search | [MemorySearch](reme/memory/tools/chunk/memory_search.py) — Vector + BM25 hybrid retrieval |
-| `get_in_memory_memory` | 🗂️ Create in-memory instance | [ReMeInMemoryMemory](reme/memory/file_based/reme_in_memory_memory.py) — Token-aware memory management, supports compression summary and state serialization (static method) |
+| Method | Function | Key components |
+|------------------------|--------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `check_context` | 📊 Check context size | [ContextChecker](reme/memory/file_based/component/context_checker.py) — checks whether context exceeds thresholds and splits messages |
+| `compact_memory` | 📦 Compact history into summary | [Compactor](reme/memory/file_based/component/compactor.py) — ReActAgent that generates structured context summaries |
+| `summary_memory` | 📝 Persist important memory to files | [Summarizer](reme/memory/file_based/component/summarizer.py) — ReActAgent + file tools (`read` / `write` / `edit`) |
+| `compact_tool_result` | ✂️ Compact long tool outputs | [ToolResultCompactor](reme/memory/file_based/component/tool_result_compactor.py) — truncates long tool outputs and stores them in `tool_result/` while keeping file references in messages |
+| `memory_search` | 🔍 Semantic memory search | [MemorySearch](reme/memory/file_based/tools/memory_search.py) — hybrid retrieval with vectors + BM25 |
+| `ReMeInMemoryMemory` | 🗂️ In-session memory class | [ReMeInMemoryMemory](reme/memory/file_based/reme_in_memory_memory.py) — token-aware memory management with summary compression and state serialization |
+| `pre_reasoning_hook` | 🔄 Pre-reasoning hook | `compact_tool_result` + `check_context` + `compact_memory` + `summary_memory` (async) |
+| `start` | 🚀 Start memory system | Initialize file storage, file watcher, and embedding cache; clean up expired tool result files |
+| `close` | 📕 Shutdown and cleanup | Clean up tool result files, stop file watcher, and persist embedding cache |
---
-### 🚀 Quick Start
+### 🚀 Quick start
#### Installation
@@ -87,23 +86,22 @@ capabilities for AI Agents:
pip install -e ".[light]"
```
-#### Environment Variables
+#### Environment variables
-`ReMeLight` environment variables configure Embedding and storage backend
+`ReMeLight` uses environment variables to configure the embedding model and storage backends:
-| Variable | Description | Example |
-|----------------------|--------------------------------|-----------------------------------------------------|
-| `LLM_API_KEY` | LLM API key | `sk-xxx` |
-| `LLM_BASE_URL` | LLM base URL | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
-| `EMBEDDING_API_KEY` | Embedding API key (Optional) | `sk-xxx` |
-| `EMBEDDING_BASE_URL` | Embedding base URL (Optional) | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
+| Variable | Description | Example |
+|----------------------|-------------------------------|-----------------------------------------------------|
+| `LLM_API_KEY` | LLM API key | `sk-xxx` |
+| `LLM_BASE_URL` | LLM base URL | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
+| `EMBEDDING_API_KEY` | Embedding API key (optional) | `sk-xxx` |
+| `EMBEDDING_BASE_URL` | Embedding base URL (optional) | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
-#### Python Usage
+#### Python usage
```python
import asyncio
-from agentscope.message import Msg
from reme.reme_light import ReMeLight
@@ -116,24 +114,24 @@ async def main():
)
await reme.start()
- messages = [...] # Conversation message list
+ messages = [...] # List of conversation messages
- # 1. Compact oversized tool outputs (prevent tool results from overflowing context)
+ # 1. Compact long tool outputs (prevent tool results from blowing up context)
messages = await reme.compact_tool_result(messages)
- # 2. Compact history to structured summary (can pass previous summary for incremental update)
+ # 2. Compact conversation history into a structured summary
summary = await reme.compact_memory(
messages=messages,
previous_summary="",
max_input_length=128000, # Model context window (tokens)
- compact_ratio=0.7, # Trigger compaction when reaching max_input_length * 0.7
- language="zh", # Summary language (zh / "")
+ compact_ratio=0.7, # Trigger compaction when exceeding max_input_length * 0.7
+ language="zh", # Summary language (e.g., "zh" / "")
)
- # 3. Submit async summary task in background (non-blocking, writes to memory/YYYY-MM-DD.md)
+ # 3. Submit summary task asynchronously (non-blocking, writes to memory/YYYY-MM-DD.md)
reme.add_async_summary_task(messages=messages)
- # 4. Pre-reasoning hook (auto compact tool results + generate summary)
+ # 4. Pre-reasoning hook (auto compact tool results + generate summaries)
processed_messages, compressed_summary = await reme.pre_reasoning_hook(
messages=messages,
system_prompt="You are a helpful AI assistant.",
@@ -145,22 +143,23 @@ async def main():
tool_result_compact_keep_n=3,
)
- # 5. Semantic memory search (Vector + BM25 hybrid retrieval)
+ # 5. Semantic memory search (vector + BM25 hybrid retrieval)
result = await reme.memory_search(query="Python version preference", max_results=5)
- # 6. Get in-memory instance (static method, manages single conversation context)
- memory = ReMeLight.get_in_memory_memory()
+ # 6. Create in-session memory instance (manages context for one conversation)
+ from reme.memory.file_based.reme_in_memory_memory import ReMeInMemoryMemory
+ memory = ReMeInMemoryMemory()
for msg in messages:
await memory.add(msg)
token_stats = await memory.estimate_tokens(max_input_length=128000)
print(f"Current context usage: {token_stats['context_usage_ratio']:.1f}%")
- print(f"Message tokens: {token_stats['messages_tokens']}")
+ print(f"Message token count: {token_stats['messages_tokens']}")
print(f"Estimated total tokens: {token_stats['estimated_tokens']}")
- # 7. Wait for background tasks before closing
+ # 7. Wait for background summary tasks to complete before shutdown
summary_result = await reme.await_summary_tasks()
- # Close ReMeLight
+ # Shutdown ReMeLight
await reme.close()
@@ -168,173 +167,226 @@ if __name__ == "__main__":
asyncio.run(main())
```
-> 📂 Full example code: [test_reme_light.py](tests/light/test_reme_light.py)
-> 📋 Example output: [test_reme_light.log](tests/light/test_reme_light_log.txt) (223,838 tokens → 1,105 tokens, 99.5%
-> compression ratio)
+> 📂 Full example: [test_reme_light.py](tests/light/test_reme_light.py)
+> 📋 Sample run log: [test_reme_light_log.txt](tests/light/test_reme_light_log.txt) (223,838 tokens → 1,105 tokens, 99.5%
+> compression)
-### File-Based ReMeLight Memory System Architecture
+### Architecture of the file-based ReMeLight memory system
[CoPaw MemoryManager](https://github.com/agentscope-ai/CoPaw/blob/main/src/copaw/agents/memory/memory_manager.py)
-inherits `ReMeLight` and integrates memory capabilities into the Agent reasoning flow:
-
-```mermaid
-graph TB
- CoPaw["CoPaw MemoryManager
(inherits ReMeLight)"] -->|pre_reasoning hook| Hook[MemoryCompactionHook]
- CoPaw --> ReMeLight[ReMeLight]
- Hook -->|exceeds threshold| ReMeLight
- ReMeLight --> CompactMemory[compact_memory
History compaction]
- ReMeLight --> SummaryMemory[summary_memory
Write memory to files]
- ReMeLight --> CompactToolResult[compact_tool_result
Oversized tool output compaction]
- ReMeLight --> MemSearch[memory_search
Semantic search]
- ReMeLight --> InMemory[get_in_memory_memory
ReMeInMemoryMemory]
- CompactMemory --> Compactor[Compactor
ReActAgent]
- SummaryMemory --> Summarizer[Summarizer
ReActAgent + file tools]
- CompactToolResult --> ToolResultCompactor[ToolResultCompactor
Truncate + save to file]
- Summarizer --> FileIO[FileIO
read / write / edit]
- FileIO --> MemoryFiles[memory/YYYY-MM-DD.md]
- ToolResultCompactor --> ToolResultFiles[tool_result/*.txt]
- MemoryFiles -.->|File change| FileWatcher[Async File Watcher]
- FileWatcher -->|Update index| FileStore[Local DB]
- MemSearch --> FileStore
-```
-
-### Context Compaction Mechanism
-
-#### Context Compaction
-
-[Compactor](reme/memory/file_based/compactor.py) uses ReActAgent to compact history into structured **context
-checkpoints**:
-
-| Field | Description |
-|-----------------------|-----------------------------------------------------|
-| `## Goal` | 🎯 User's objectives (can be multiple) |
-| `## Constraints` | ⚙️ Constraints and preferences mentioned by user |
-| `## Progress` | 📈 Completed / in progress / blocked tasks |
-| `## Key Decisions` | 🔑 Decisions made with brief reasons |
-| `## Next Steps` | 🗺️ Next action plan (ordered list) |
-| `## Critical Context` | 📌 File paths, function names, error messages, etc. |
-
-Supports **incremental updates**: when `previous_summary` is passed, automatically merges new conversation with old
-summary, preserving historical progress.
-
-#### Tool Result Compaction
-
-[ToolResultCompactor](reme/memory/file_based/tool_result_compactor.py) solves context overflow caused by oversized tool
-outputs (e.g., browser use):
+inherits
+`ReMeLight` and integrates its memory capabilities into the agent reasoning loop:
```mermaid
graph LR
- A[tool_result message] --> B{Content length > threshold?}
- B -->|No| C[Keep as-is]
- B -->|Yes| D[Truncate to threshold characters]
- D --> E[Write full content to tool_result/uuid.txt]
- E --> F[Append file reference path to message]
-```
-
-Expired files (exceeding `retention_days`) are automatically cleaned up during `start` / `close` /
-`compact_tool_result`.
-
-### Memory Summary: ReAct + File Tools
-
-[Summarizer](reme/memory/file_based/summarizer.py) uses the **ReAct + file tools** pattern, letting AI autonomously
-decide what to write and where:
-
-```mermaid
-graph LR
- A[Receive conversation] --> B{Think: What's worth recording?}
- B --> C[Act: read memory/YYYY-MM-DD.md]
- C --> D{Think: How to merge with existing content?}
- D --> E[Act: edit to update file]
- E --> F{Think: Anything missing?}
- F -->|Yes| B
- F -->|No| G[Done]
-```
-
-[FileIO](reme/memory/file_based/file_io.py) provides file operation tools:
-
-| Tool | Function | Use case |
-|---------|--------------------------------|-----------------------------------------|
-| `read` | Read file content (line range) | View existing memory, avoid duplicates |
-| `write` | Overwrite file | Create new memory file or major rewrite |
-| `edit` | Replace after exact match | Append or modify specific sections |
-
-### In-Memory Session Management
-
-[ReMeInMemoryMemory](reme/memory/file_based/reme_in_memory_memory.py) extends AgentScope's `InMemoryMemory`:
-
-| Feature | Description |
-|----------------------------------|---------------------------------------------------------------------|
-| `get_memory` | Filter messages by mark, auto-prepend compression summary |
-| `estimate_tokens` | Precisely estimate current context token usage and ratio |
-| `get_history_str` | Generate human-readable conversation summary (with token stats) |
-| `state_dict` / `load_state_dict` | Support state serialization / deserialization (session persistence) |
-| `mark_messages_compressed` | Mark messages as compressed state |
-| `get_compressed_summary` | Get compressed summary content |
-
-### Memory Retrieval
-
-[MemorySearch](reme/memory/tools/chunk/memory_search.py) provides **vector + BM25 hybrid retrieval**:
-
-| Retrieval | Strength | Weakness |
-|---------------------|-------------------------------------------------|----------------------------------------|
-| **Vector semantic** | Captures similar meaning with different wording | Weaker on exact token match |
-| **BM25 full-text** | Strong exact token match | No synonym or paraphrase understanding |
-
-**Fusion**: Both retrieval paths are weighted and summed (vector 0.7 + BM25 0.3), so both natural-language queries and
-exact lookups get reliable results.
-
-```mermaid
-graph LR
- Q[Search query] --> V[Vector search × 0.7]
-Q --> B[BM25 × 0.3]
-V --> M[Dedupe + weighted merge]
-B --> M
-M --> R[Top-N results]
+ Agent[Agent] -->|Before each reasoning step| Hook[pre_reasoning_hook]
+ Hook --> TC[compact_tool_result
Compact tool outputs]
+ TC --> CC[check_context
Token counting]
+ CC -->|Exceeds limit| CM[compact_memory
Generate summary]
+ CC -->|Exceeds limit| SM[summary_memory
Async persistence]
+ SM -->|ReAct + FileIO| Files[memory/*.md]
+ Agent -->|Explicit call| Search[memory_search
Vector+BM25]
+ Agent -->|In-session| InMem[ReMeInMemoryMemory
Token-aware memory]
+ Files -.->|FileWatcher| Store[(FileStore
Vector+FTS index)]
+ Search --> Store
```
---
-## 🗃️ Vector-Based Memory System
+#### 1. `check_context` — context checking
-[ReMe Vector Based](reme/reme.py) is the core class for the vector-based memory system, supporting unified management of
-three memory types:
+[ContextChecker](reme/memory/file_based/component/context_checker.py) uses token counting to determine whether the
+context exceeds thresholds and automatically splits messages into a "to compact" group and a "to keep" group.
-| Memory Type | Purpose | Usage Context |
-|------------------------------|-----------------------------------------------------|---------------|
-| **Personal memory** | User preferences, habits | `user_name` |
-| **Task / procedural memory** | Task execution experience, success/failure patterns | `task_name` |
-| **Tool memory** | Tool usage experience, parameter tuning | `tool_name` |
-
-### Core Capabilities
-
-| Method | Function | Description |
-|--------------------|---------------------|-----------------------------------------------------------|
-| `summarize_memory` | 🧠 Summarize memory | Automatically extract and store memory from conversations |
-| `retrieve_memory` | 🔍 Retrieve memory | Retrieve relevant memory by query |
-| `add_memory` | ➕ Add memory | Manually add memory to vector store |
-| `get_memory` | 📖 Get memory | Fetch a single memory by ID |
-| `update_memory` | ✏️ Update memory | Update content or metadata of existing memory |
-| `delete_memory` | 🗑️ Delete memory | Delete specified memory |
-| `list_memory` | 📋 List memory | List memories with filtering and sorting |
-
-### Installation
-
-```bash
-pip install -U reme-ai
+```mermaid
+graph LR
+ M[messages] --> H[AsMsgHandler
Token counting]
+ H --> C{total > threshold?}
+ C -->|No| K[Return all messages]
+ C -->|Yes| S[Keep from tail
reserve tokens]
+ S --> CP[messages_to_compact
Earlier messages]
+ S --> KP[messages_to_keep
Recent messages]
+ S --> V{is_valid
Tool calls aligned?}
```
-### Environment Variables
+- **Core logic**: keep `reserve` tokens from the tail; mark the rest as messages to compact.
+- **Integrity guarantee**: preserves complete user-assistant turns and tool_use/tool_result pairs without splitting
+ them.
-API keys are set via environment variables; you can put them in a `.env` file in the project root:
+---
-| Variable | Description | Example |
-|----------------------|--------------------|-----------------------------------------------------|
-| `LLM_API_KEY` | LLM API Key | `sk-xxx` |
-| `LLM_BASE_URL` | LLM Base URL | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
-| `EMBEDDING_API_KEY` | Embedding API Key | `sk-xxx` |
-| `EMBEDDING_BASE_URL` | Embedding Base URL | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
+#### 2. `compact_memory` — conversation compaction
-### Python Usage
+[Compactor](reme/memory/file_based/component/compactor.py) uses a ReActAgent to compact conversation history into a *
+*structured context summary**.
+
+```mermaid
+graph LR
+ M[messages] --> H[AsMsgHandler
format_msgs_to_str]
+ H --> A[ReActAgent
reme_compactor]
+ P[previous_summary] -->|Incremental update| A
+ A --> S[Structured summary
Goal/Progress/Decisions...]
+```
+
+**Summary structure** (context checkpoints):
+
+| Field | Description |
+|-----------------------|------------------------------------------------------------------------|
+| `## Goal` | User goals |
+| `## Constraints` | Constraints and preferences |
+| `## Progress` | Task progress |
+| `## Key Decisions` | Key decisions |
+| `## Next Steps` | Next step plans |
+| `## Critical Context` | Critical data such as file paths, function names, error messages, etc. |
+
+- **Incremental updates**: when `previous_summary` is provided, new conversations are merged into the existing summary.
+
+---
+
+#### 3. `summary_memory` — persistent memory
+
+[Summarizer](reme/memory/file_based/component/summarizer.py) uses a **ReAct + file tools** pattern so that the AI can
+decide what to write and where to write it.
+
+```mermaid
+graph LR
+ M[messages] --> A[ReActAgent
reme_summarizer]
+ A -->|read| R[Read memory/YYYY-MM-DD.md]
+ R --> T{Reason: how to merge?}
+ T -->|write| W[Overwrite]
+ T -->|edit| E[Edit in place]
+ W --> F[memory/YYYY-MM-DD.md]
+ E --> F
+```
+
+**File tools** ([FileIO](reme/memory/file_based/tools/file_io.py)):
+
+| Tool | Function |
+|---------|-----------------------|
+| `read` | Read file content |
+| `write` | Overwrite file |
+| `edit` | Find-and-replace edit |
+
+---
+
+#### 4. `compact_tool_result` — tool result compaction
+
+[ToolResultCompactor](reme/memory/file_based/component/tool_result_compactor.py) addresses the problem of long tool
+outputs bloating the context.
+
+```mermaid
+graph LR
+ M[messages] --> L{Iterate tool_result
len > threshold?}
+ L -->|No| K[Keep as-is]
+ L -->|Yes| T[truncate_text
Truncate to threshold]
+ T --> S[Write full content
tool_result/uuid.txt]
+ S --> R[Append file path reference
to message]
+ R --> C[cleanup_expired_files
Delete expired files]
+```
+
+- **Auto cleanup**: expired files (older than `retention_days`) are deleted automatically during `start` / `close` /
+ `compact_tool_result`.
+
+---
+
+#### 5. `memory_search` — memory retrieval
+
+[MemorySearch](reme/memory/file_based/tools/memory_search.py) provides **vector + BM25 hybrid retrieval**.
+
+```mermaid
+graph LR
+ Q[query] --> E[Embedding
Vectorization]
+ E --> V[vector_search
Semantic similarity]
+ Q --> B[BM25
Keyword matching]
+ V -->|" weight: 0.7 "| M[Deduplicate + weighted merge]
+ B -->|" weight: 0.3 "| M
+ M --> F[min_score filter]
+ F --> R[Top-N results]
+```
+
+- **Fusion mechanism**: vector weight 0.7 + BM25 weight 0.3 — balancing semantic similarity and exact matches.
+
+---
+
+#### 6. `ReMeInMemoryMemory` — in-session memory
+
+[ReMeInMemoryMemory](reme/memory/file_based/reme_in_memory_memory.py) extends AgentScope's `InMemoryMemory` to provide
+token-aware memory management.
+
+```mermaid
+graph LR
+ C[content] --> G[get_memory
exclude_mark=COMPRESSED]
+ G --> F[Filter out compressed messages]
+ F --> P{prepend_summary?}
+ P -->|Yes| S[Prepend previous summary]
+ S --> O[Output messages]
+ P -->|No| O
+```
+
+| Function | Description |
+|----------------------------------|---------------------------------------------------|
+| `get_memory` | Filter messages by mark and auto-append summary |
+| `estimate_tokens` | Estimate token usage of the context |
+| `state_dict` / `load_state_dict` | Serialize/deserialize state (session persistence) |
+
+---
+
+#### 7. `pre_reasoning_hook` — pre-reasoning processing
+
+This is a unified entry point that wires all the above components together and automatically manages context before each
+reasoning step.
+
+```mermaid
+graph LR
+ M[messages] --> TC[compact_tool_result
Compact long tool outputs]
+ TC --> CC[check_context
Compute remaining space]
+ CC --> D{messages_to_compact
Non-empty?}
+ D -->|No| K[Return original messages + summary]
+ D -->|Yes| V{is_valid?}
+ V -->|No| K
+ V -->|Yes| CM[compact_memory
Sync summary generation]
+ V -->|Yes| SM[add_async_summary_task
Async persistence]
+ CM --> R[Return messages_to_keep + new summary]
+```
+
+**Execution flow**:
+
+1. `compact_tool_result` — compact long tool outputs.
+2. `check_context` — check whether the context exceeds limits.
+3. `compact_memory` — generate compact summary (sync).
+4. `summary_memory` — persist memory (async in the background).
+
+---
+
+## 🗃️ Vector-based memory system
+
+[ReMe Vector Based](reme/reme.py) is the core class for the vector-based memory system. It manages three types of
+memories:
+
+| Memory type | Use case |
+|-----------------------|-------------------------------------------------------------------|
+| **Personal memory** | Records user preferences and habits |
+| **Procedural memory** | Records task execution experience and patterns of success/failure |
+| **Tool memory** | Records tool usage experience and parameter tuning |
+
+### Core capabilities
+
+| Method | Function | Description |
+|--------------------|--------------|-------------------------------------------------------------|
+| `summarize_memory` | 🧠 Summarize | Automatically extract and store memories from conversations |
+| `retrieve_memory` | 🔍 Retrieve | Retrieve related memories based on a query |
+| `add_memory` | ➕ Add | Manually add memories into the vector store |
+| `get_memory` | 📖 Get | Get a single memory by ID |
+| `update_memory` | ✏️ Update | Update existing memory content or metadata |
+| `delete_memory` | 🗑️ Delete | Delete a specific memory |
+| `list_memory` | 📋 List | List memories with filtering and sorting |
+
+### Installation and environment variables
+
+Installation and environment configuration are the same as [ReMeLight](#installation).
+API keys are configured via environment variables and can be stored in a `.env` file at the project root.
+
+### Python usage
```python
import asyncio
@@ -363,34 +415,34 @@ async def main():
messages = [
{"role": "user", "content": "Help me write a Python script", "time_created": "2026-02-28 10:00:00"},
- {"role": "assistant", "content": "Sure, I'll help you write it", "time_created": "2026-02-28 10:00:05"},
+ {"role": "assistant", "content": "Sure, I'll help you with that.", "time_created": "2026-02-28 10:00:05"},
]
- # 1. Summarize memory from conversation (auto-extract user preferences, task experience, etc.)
+ # 1. Summarize memories from conversation (automatically extract user preferences, task experience, etc.)
result = await reme.summarize_memory(
messages=messages,
user_name="alice", # Personal memory
- # task_name="code_writing", # Task memory
+ # task_name="code_writing", # Procedural memory
)
- print(f"Summarize result: {result}")
+ print(f"Summary result: {result}")
- # 2. Retrieve relevant memory
+ # 2. Retrieve related memories
memories = await reme.retrieve_memory(
query="Python programming",
user_name="alice",
# task_name="code_writing",
)
- print(f"Retrieve result: {memories}")
+ print(f"Retrieved memories: {memories}")
- # 3. Manually add memory
+ # 3. Manually add a memory
memory_node = await reme.add_memory(
- memory_content="User prefers concise code style",
+ memory_content="The user prefers concise code style.",
user_name="alice",
)
print(f"Added memory: {memory_node}")
memory_id = memory_node.memory_id
- # 4. Get single memory by ID
+ # 4. Get a single memory by ID
fetched_memory = await reme.get_memory(memory_id=memory_id)
print(f"Fetched memory: {fetched_memory}")
@@ -398,11 +450,11 @@ async def main():
updated_memory = await reme.update_memory(
memory_id=memory_id,
user_name="alice",
- memory_content="User prefers concise, well-commented code style",
+ memory_content="The user prefers concise code with comments.",
)
print(f"Updated memory: {updated_memory}")
- # 6. List all memories for user (with filtering and sorting)
+ # 6. List all memories for the user (supports filtering and sorting)
all_memories = await reme.list_memory(
user_name="alice",
limit=10,
@@ -411,11 +463,11 @@ async def main():
)
print(f"User memory list: {all_memories}")
- # 7. Delete specified memory
+ # 7. Delete a specific memory
await reme.delete_memory(memory_id=memory_id)
print(f"Deleted memory: {memory_id}")
- # 8. Delete all memories (use with caution)
+ # 8. Delete all memories (use with care)
# await reme.delete_all()
await reme.close()
@@ -425,21 +477,21 @@ if __name__ == "__main__":
asyncio.run(main())
```
-### Technical Architecture
+### Technical architecture
```mermaid
-graph TB
+graph LR
User[User / Agent] --> ReMe[Vector Based ReMe]
- ReMe --> Summarize[Memory Summarize]
- ReMe --> Retrieve[Memory Retrieve]
- ReMe --> CRUD[CRUD]
+ ReMe --> Summarize[Summarize memories]
+ ReMe --> Retrieve[Retrieve memories]
+ ReMe --> CRUD[CRUD operations]
Summarize --> PersonalSum[PersonalSummarizer]
Summarize --> ProceduralSum[ProceduralSummarizer]
Summarize --> ToolSum[ToolSummarizer]
Retrieve --> PersonalRet[PersonalRetriever]
Retrieve --> ProceduralRet[ProceduralRetriever]
Retrieve --> ToolRet[ToolRetriever]
- PersonalSum --> VectorStore[Vector DB]
+ PersonalSum --> VectorStore[Vector database]
ProceduralSum --> VectorStore
ToolSum --> VectorStore
PersonalRet --> VectorStore
@@ -447,17 +499,53 @@ graph TB
ToolRet --> VectorStore
```
-## ⭐ Community & Support
+### Experimental results
-- **Star & Watch**: Star helps more agent developers discover ReMe; Watch keeps you updated on new releases and
- features.
-- **Share your work**: In Issues or Discussions, share what ReMe unlocks for your agents — we're happy to highlight
- great community examples.
-- **Need a new feature?** Open a Feature Request; we'll iterate with the community.
-- **Code contributions**: All forms of code contribution are welcome. See
- the [Contribution Guide](docs/contribution.md).
-- **Acknowledgments**: Thanks to OpenClaw, Mem0, MemU, CoPaw, and other open-source projects for inspiration and
- support.
+Coming soon...
+
+---
+
+## 🧪 Procedural memory paper
+
+> Our procedural (task) memory paper is available on [arXiv](https://arxiv.org/abs/2512.10696).
+
+### 🌍 [Appworld benchmark](benchmark/appworld/quickstart.md)
+
+We evaluate ReMe on the Appworld environment using Qwen3-8B (non-thinking mode):
+
+| Method | Avg@4 | Pass@4 |
+|----------|---------------------|---------------------|
+| w/o ReMe | 0.1497 | 0.3285 |
+| w/ ReMe | 0.1706 **(+2.09%)** | 0.3631 **(+3.46%)** |
+
+Pass@K measures the probability that at least one of K generated candidates successfully completes the task (score=1).
+The current experiments use an internal AppWorld environment, which may differ slightly from the public version.
+
+For more details on how to reproduce the experiments, see [quickstart.md](benchmark/appworld/quickstart.md).
+
+### 🔧 [BFCL-V3 benchmark](benchmark/bfcl/quickstart.md)
+
+We evaluate ReMe on the BFCL-V3 multi-turn-base task (random split 50 train / 150 val) using Qwen3-8B (thinking mode):
+
+| Method | Avg@4 | Pass@4 |
+|----------|---------------------|---------------------|
+| w/o ReMe | 0.4033 | 0.5955 |
+| w/ ReMe | 0.4450 **(+4.17%)** | 0.6577 **(+6.22%)** |
+
+For more details on how to reproduce the experiments, see [quickstart.md](benchmark/bfcl/quickstart.md).
+
+
+## ⭐ Community & support
+
+- **Star & Watch**: Starring helps more agent developers discover ReMe; Watching keeps you up to date with new releases
+ and features.
+- **Share your results**: Share how ReMe empowers your agents in Issues or Discussions — we are happy to showcase great
+ community use cases.
+- **Need a new feature?** Open a feature request; we’ll evolve ReMe together with the community.
+- **Code contributions**: All forms of contributions are welcome. Please see
+ the [contribution guide](docs/contribution.md).
+- **Acknowledgements**: We thank excellent open-source projects such as OpenClaw, Mem0, MemU, and CoPaw for their
+ inspiration and support.
---
@@ -476,10 +564,11 @@ graph TB
## ⚖️ License
-This project is open source under the Apache License 2.0. See the [LICENSE](./LICENSE) file for details.
+This project is open-sourced under the Apache License 2.0. See [LICENSE](./LICENSE) for details.
---
-## 📈 Star History
+## 📈 Star history
[](https://www.star-history.com/#agentscope-ai/ReMe&Date)
+
diff --git a/README_ZH.md b/README_ZH.md
index 3c379a2b..8723177c 100644
--- a/README_ZH.md
+++ b/README_ZH.md
@@ -38,7 +38,7 @@ ReMe 让智能体拥有**真正的记忆力**——旧对话自动浓缩，重
> 记忆即文件，文件即记忆
将**记忆视为文件**——可读、可编辑、可复制。
-[CoPaw](https://github.com/agentscope-ai/CoPaw)通过继承 `ReMeLight` 实现了长期记忆和上下文的管理。
+[CoPaw](https://github.com/agentscope-ai/CoPaw) 通过继承 `ReMeLight` 实现了长期记忆和上下文的管理。
| 传统记忆系统 | File Based ReMe |
|-----------|-----------------|
@@ -49,9 +49,9 @@ ReMe 让智能体拥有**真正的记忆力**——旧对话自动浓缩，重
```
working_dir/
-├── MEMORY.md # 长期记忆：用户偏好、项目配置等持久信息
+├── MEMORY.md # 长期记忆：用户偏好等持久信息
├── memory/
-│ └── YYYY-MM-DD.md # 每日摘要日志：对话结束后自动写入
+│ └── YYYY-MM-DD.md # 每日日记：对话结束后自动写入
└── tool_result/ # 超长工具输出缓存（自动管理，超期自动清理）
└── .txt
```
@@ -60,17 +60,17 @@ working_dir/
[ReMeLight](reme/reme_light.py) 是该记忆系统的核心类，为 AI Agent 提供完整的记忆管理能力：
-| 方法 | 功能 | 关键组件 |
-|------------------------|--------------|----------------------------------------------------------------------------------------------------------|
-| `start` | 🚀 启动记忆系统 | 初始化文件存储、文件监控、Embedding 缓存；清理过期工具结果文件 |
-| `close` | 📕 关闭并清理 | 清理工具结果文件、停止文件监控、保存 Embedding 缓存 |
-| `compact_memory` | 📦 压缩历史对话为摘要 | [Compactor](reme/memory/file_based/compactor.py) — ReActAgent 生成结构化上下文检查点 |
-| `summary_memory` | 📝 将重要记忆写入文件 | [Summarizer](reme/memory/file_based/summarizer.py) — ReActAgent + 文件工具（read / write / edit） |
-| `compact_tool_result` | ✂️ 压缩超长工具输出 | [ToolResultCompactor](reme/memory/file_based/tool_result_compactor.py) — 截断并转存到 `tool_result/`，消息中保留文件引用 |
-| `pre_reasoning_hook` | 🔄 推理前预处理钩子 | 自动压缩工具结果 + 生成摘要 + 异步触发记忆总结任务 |
-| `memory_search` | 🔍 语义搜索记忆 | [MemorySearch](reme/memory/tools/chunk/memory_search.py) — 向量 + BM25 混合检索 |
-| `get_in_memory_memory` | 🗂️ 创建会话内存实例 | [ReMeInMemoryMemory](reme/memory/file_based/reme_in_memory_memory.py) — Token 感知的内存管理，支持压缩摘要和状态序列化（静态方法） |
-
+| 方法 | 功能 | 关键组件 |
+|------------------------|--------------|-----------------------------------------------------------------------------------------------------------------------------|
+| `check_context` | 📊 检查上下文大小 | [ContextChecker](reme/memory/file_based/component/context_checker.py) — 检查上下文是否超出阈值并拆分Message |
+| `compact_memory` | 📦 压缩历史对话为摘要 | [Compactor](reme/memory/file_based/component/compactor.py) — ReActAgent 生成结构化上下文摘要 |
+| `summary_memory` | 📝 将重要记忆写入文件 | [Summarizer](reme/memory/file_based/component/summarizer.py) — ReActAgent + 文件工具（read / write / edit） |
+| `compact_tool_result` | ✂️ 压缩超长工具输出 | [ToolResultCompactor](reme/memory/file_based/component/tool_result_compactor.py) — 截断超长的工具调用结果并转存到 `tool_result/`，消息中保留文件引用 |
+| `memory_search` | 🔍 语义搜索记忆 | [MemorySearch](reme/memory/file_based/tools/memory_search.py) — 向量 + BM25 混合检索 |
+| `ReMeInMemoryMemory` | 🗂️ 会话内存类 | [ReMeInMemoryMemory](reme/memory/file_based/reme_in_memory_memory.py) — Token 感知的内存管理，支持压缩摘要和状态序列化 |
+| `pre_reasoning_hook` | 🔄 推理前预处理钩子 | compact_tool_result + check_context + compact_memory + summary_memory(async) |
+| `start` | 🚀 启动记忆系统 | 初始化文件存储、文件监控、Embedding 缓存；清理过期工具结果文件 |
+| `close` | 📕 关闭并清理 | 清理工具结果文件、停止文件监控、保存 Embedding 缓存 |·
---
### 🚀 快速开始
@@ -85,14 +85,14 @@ pip install -e ".[light]"
`ReMeLight` 环境变量配置 Embedding 和存储后端
-| Variable | Description | Example |
-|----------------------|--------------------|-----------------------------------------------------|
-| `LLM_API_KEY` | LLM API key | `sk-xxx` |
-| `LLM_BASE_URL` | LLM base URL | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
-| `EMBEDDING_API_KEY` | Embedding API key | `sk-xxx` |
-| `EMBEDDING_BASE_URL` | Embedding base URL | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
+| Variable | Description | Example |
+|----------------------|-------------------------|-----------------------------------------------------|
+| `LLM_API_KEY` | LLM API key | `sk-xxx` |
+| `LLM_BASE_URL` | LLM base URL | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
+| `EMBEDDING_API_KEY` | Embedding API key (可选) | `sk-xxx` |
+| `EMBEDDING_BASE_URL` | Embedding base URL (可选) | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
-#### Python使用
+#### Python 使用
```python
import asyncio
@@ -141,8 +141,9 @@ async def main():
# 5. 语义搜索记忆（向量 + BM25 混合检索）
result = await reme.memory_search(query="Python 版本偏好", max_results=5)
- # 6. 获取会话内存实例（静态方法，管理单次对话的上下文）
- memory = ReMeLight.get_in_memory_memory()
+ # 6. 创建会话内存实例（管理单次对话的上下文）
+ from reme.memory.file_based.reme_in_memory_memory import ReMeInMemoryMemory
+ memory = ReMeInMemoryMemory()
for msg in messages:
await memory.add(msg)
token_stats = await memory.estimate_tokens(max_input_length=128000)
@@ -162,7 +163,7 @@ if __name__ == "__main__":
```
> 📂 完整示例代码：[test_reme_light.py](tests/light/test_reme_light.py)
-> 📋 运行结果示例：[test_reme_light.log](tests/light/test_reme_light_log.txt)（223,838 tokens → 1,105 tokens，压缩率 99.5%）
+> 📋 运行结果示例：[test_reme_light_log.txt](tests/light/test_reme_light_log.txt)（223,838 tokens → 1,105 tokens，压缩率99.5%）
### 基于文件的 ReMeLight 记忆系统架构
@@ -170,125 +171,188 @@ if __name__ == "__main__":
`ReMeLight`，将记忆能力集成到 Agent 推理流程中：
```mermaid
-graph TB
- CoPaw["CoPaw MemoryManager
(继承 ReMeLight)"] -->|pre_reasoning hook| Hook[MemoryCompactionHook]
- CoPaw --> ReMeLight[ReMeLight]
- Hook -->|超出阈值| ReMeLight
- ReMeLight --> CompactMemory[compact_memory
历史对话压缩]
- ReMeLight --> SummaryMemory[summary_memory
记忆写入文件]
- ReMeLight --> CompactToolResult[compact_tool_result
超长工具输出压缩]
- ReMeLight --> MemSearch[memory_search
语义搜索]
- ReMeLight --> InMemory[get_in_memory_memory
ReMeInMemoryMemory]
- CompactMemory --> Compactor[Compactor
ReActAgent]
- SummaryMemory --> Summarizer[Summarizer
ReActAgent + 文件工具]
- CompactToolResult --> ToolResultCompactor[ToolResultCompactor
截断 + 转存文件]
- Summarizer --> FileIO[FileIO
read / write / edit]
- FileIO --> MemoryFiles[memory/YYYY-MM-DD.md]
- ToolResultCompactor --> ToolResultFiles[tool_result/*.txt]
- MemoryFiles -.->|文件变更| FileWatcher[异步文件监控]
- FileWatcher -->|更新索引| FileStore[本地数据库]
- MemSearch --> FileStore
+graph LR
+ Agent[Agent] -->|每轮推理前| Hook[pre_reasoning_hook]
+ Hook --> TC[compact_tool_result
压缩工具输出]
+ TC --> CC[check_context
Token 计数]
+ CC -->|超限| CM[compact_memory
生成摘要]
+ CC -->|超限| SM[summary_memory
异步持久化]
+ SM -->|ReAct + FileIO| Files[memory/*.md]
+ Agent -->|主动调用| Search[memory_search
向量+BM25]
+ Agent -->|会话内存| InMem[ReMeInMemoryMemory
Token感知内存]
+ Files -.->|FileWatcher| Store[(FileStore
向量+FTS索引)]
+ Search --> Store
```
-### 上下文压缩机制
+---
-#### 上下文压缩
+#### 1. check_context — 上下文检查
-[Compactor](reme/memory/file_based/compactor.py) 使用 ReActAgent 将历史对话压缩为结构化的**上下文检查点**：
-
-| 字段 | 说明 |
-|-----------------------|-----------------------|
-| `## Goal` | 🎯 用户要完成的目标（可多项） |
-| `## Constraints` | ⚙️ 用户提到的约束和偏好 |
-| `## Progress` | 📈 已完成 / 进行中 / 阻塞的任务 |
-| `## Key Decisions` | 🔑 做出的决策及简短理由 |
-| `## Next Steps` | 🗺️ 下一步行动计划（有序列表） |
-| `## Critical Context` | 📌 文件路径、函数名、错误信息等关键数据 |
-
-支持**增量更新**：传入 `previous_summary` 时，自动将新对话与旧摘要合并，保留历史进展。
-
-#### 工具结果压缩
-
-[ToolResultCompactor](reme/memory/file_based/tool_result_compactor.py) 解决工具输出过长（比如 browser use）导致上下文膨胀的问题：
+[ContextChecker](reme/memory/file_based/component/context_checker.py) 基于 Token 计数判断上下文是否超限，自动拆分为「待压缩」和「保留」两组消息。
```mermaid
graph LR
- A[tool_result 消息] --> B{内容长度 > threshold?}
- B -->|否| C[保留原样]
- B -->|是| D[截断到 threshold 字符]
- D --> E[完整内容写入 tool_result/uuid.txt]
- E --> F[消息中追加文件引用路径]
+ M[messages] --> H[AsMsgHandler
Token 计数]
+ H --> C{total > threshold?}
+ C -->|否| K[返回全部消息]
+ C -->|是| S[从尾部向前保留
reserve tokens]
+ S --> CP[messages_to_compact
早期消息]
+ S --> KP[messages_to_keep
近期消息]
+ S --> V{is_valid
工具调用对齐?}
```
-过期文件（超过 `retention_days`）在 `start` / `close` / `compact_tool_result` 时自动清理。
+- **核心逻辑**：从尾部向前保留 `reserve` tokens，超出部分标记为待压缩
+- **完整性保证**：不拆分 user-assistant 对话对，不拆分 tool_use/tool_result 配对
-### 记忆总结：ReAct + 文件工具
+---
-[Summarizer](reme/memory/file_based/summarizer.py) 采用 **ReAct + 文件工具** 模式，让 AI 自主决定写什么、写到哪：
+#### 2. compact_memory — 对话压缩
+
+[Compactor](reme/memory/file_based/component/compactor.py) 使用 ReActAgent 将历史对话压缩为**结构化上下文摘要**。
```mermaid
graph LR
- A[接收对话] --> B{思考: 有什么值得记录?}
- B --> C[行动: read memory/YYYY-MM-DD.md]
- C --> D{思考: 如何与现有内容合并?}
- D --> E[行动: edit 更新文件]
- E --> F{思考: 还有遗漏吗?}
- F -->|是| B
- F -->|否| G[完成]
+ M[messages] --> H[AsMsgHandler
format_msgs_to_str]
+ H --> A[ReActAgent
reme_compactor]
+ P[previous_summary] -->|增量更新| A
+ A --> S[结构化摘要
Goal/Progress/Decisions...]
```
-[FileIO](reme/memory/file_based/file_io.py) 提供文件操作工具集：
+**摘要结构**（上下文检查点）：
-| 工具 | 功能 | 使用场景 |
-|---------|---------------|---------------|
-| `read` | 读取文件内容（支持行范围） | 查看现有记忆，避免重复写入 |
-| `write` | 覆盖写入文件 | 创建新记忆文件或大幅重构 |
-| `edit` | 精确匹配后替换 | 追加新内容或修改特定段落 |
+| 字段 | 说明 |
+|-----------------------|--------------------|
+| `## Goal` | 用户目标 |
+| `## Constraints` | 约束和偏好 |
+| `## Progress` | 任务进展 |
+| `## Key Decisions` | 关键决策 |
+| `## Next Steps` | 下一步计划 |
+| `## Critical Context` | 文件路径、函数名、错误信息等关键数据 |
-### 会话内存管理
+- **增量更新**：传入 `previous_summary` 时，自动将新对话与旧摘要合并
-[ReMeInMemoryMemory](reme/memory/file_based/reme_in_memory_memory.py) 扩展了 AgentScope 的 `InMemoryMemory`：
+---
-| 功能 | 说明 |
-|----------------------------------|---------------------------|
-| `get_memory` | 按标记过滤消息，自动在头部追加压缩摘要 |
-| `estimate_tokens` | 精确估算当前上下文 Token 用量及使用率 |
-| `get_history_str` | 生成人类可读的对话历史摘要（含 Token 统计） |
-| `state_dict` / `load_state_dict` | 支持状态序列化 / 反序列化（会话持久化） |
-| `mark_messages_compressed` | 标记消息为已压缩状态 |
-| `get_compressed_summary` | 获取已压缩的摘要内容 |
+#### 3. summary_memory — 记忆持久化
-### 记忆检索
-
-[MemorySearch](reme/memory/tools/chunk/memory_search.py) 提供**向量 + BM25 混合检索**能力：
-
-| 检索方式 | 优势 | 劣势 |
-|-------------|-----------------|----------------|
-| **向量语义** | 捕捉意义相近但措辞不同的内容 | 对精确 token 匹配较弱 |
-| **BM25 全文** | 精确 token 命中效果极佳 | 无法理解同义词和改写 |
-
-**融合机制**：两路召回后按权重加权求和（向量 0.7 + BM25 0.3），自然语言与精确查找均可命中。
+[Summarizer](reme/memory/file_based/component/summarizer.py) 采用 **ReAct + 文件工具** 模式，让 AI 自主决定写什么、写到哪。
```mermaid
graph LR
- Q[搜索查询] --> V[向量搜索 × 0.7]
-Q --> B[BM25 × 0.3]
-V --> M[去重 + 加权融合]
-B --> M
-M --> R[Top-N 结果]
+ M[messages] --> A[ReActAgent
reme_summarizer]
+ A -->|read| R[读取 memory/YYYY-MM-DD.md]
+ R --> T{思考: 如何合并?}
+ T -->|write| W[覆盖写入]
+ T -->|edit| E[精确替换]
+ W --> F[memory/YYYY-MM-DD.md]
+ E --> F
```
+**文件工具**（[FileIO](reme/memory/file_based/tools/file_io.py)）：
+
+| 工具 | 功能 |
+|---------|---------|
+| `read` | 读取文件内容 |
+| `write` | 覆盖写入文件 |
+| `edit` | 精确匹配后替换 |
+
+---
+
+#### 4. compact_tool_result — 工具结果压缩
+
+[ToolResultCompactor](reme/memory/file_based/component/tool_result_compactor.py) 解决工具输出过长导致上下文膨胀的问题。
+
+```mermaid
+graph LR
+ M[messages] --> L{遍历 tool_result
len > threshold?}
+ L -->|否| K[保留原样]
+ L -->|是| T[truncate_text
截断到 threshold]
+ T --> S[完整内容写入
tool_result/uuid.txt]
+ S --> R[消息追加文件路径引用]
+ R --> C[cleanup_expired_files
清理过期文件]
+```
+
+- **自动清理**：过期文件（超过 `retention_days`）在 `start`/`close`/`compact_tool_result` 时自动删除
+
+---
+
+#### 5. memory_search — 记忆检索
+
+[MemorySearch](reme/memory/file_based/tools/memory_search.py) 提供**向量 + BM25 混合检索**能力。
+
+```mermaid
+graph LR
+ Q[query] --> E[Embedding
向量化]
+ E --> V[vector_search
语义相似]
+ Q --> B[BM25
关键词匹配]
+ V -->|" weight: 0.7 "| M[去重 + 加权融合]
+ B -->|" weight: 0.3 "| M
+ M --> F[min_score 过滤]
+ F --> R[Top-N 结果]
+```
+
+- **融合机制**：向量权重 0.7 + BM25 权重 0.3，兼顾语义相似和精确匹配
+
+---
+
+#### 6. ReMeInMemoryMemory — 会话内存
+
+[ReMeInMemoryMemory](reme/memory/file_based/reme_in_memory_memory.py) 扩展 AgentScope 的 `InMemoryMemory`，提供 Token
+感知的内存管理。
+
+```mermaid
+graph LR
+ C[content] --> G[get_memory
exclude_mark=COMPRESSED]
+ G --> F[排除已压缩消息]
+ F --> P{prepend_summary?}
+ P -->|是| S[头部插入 previous-summary]
+ S --> O[输出 messages]
+ P -->|否| O
+```
+
+| 功能 | 说明 |
+|----------------------------------|-------------------|
+| `get_memory` | 按标记过滤，自动追加压缩摘要 |
+| `estimate_tokens` | 估算上下文 Token 用量 |
+| `state_dict` / `load_state_dict` | 状态序列化/反序列化（会话持久化） |
+
+---
+
+#### 7. pre_reasoning_hook — 推理前预处理
+
+整合上述组件的统一入口，在每轮推理前自动管理上下文。
+
+```mermaid
+graph LR
+ M[messages] --> TC[compact_tool_result
压缩超长工具输出]
+ TC --> CC[check_context
计算剩余空间]
+ CC --> D{messages_to_compact
非空?}
+ D -->|否| K[返回原消息 + 原摘要]
+ D -->|是| V{is_valid?}
+ V -->|否| K
+ V -->|是| CM[compact_memory
同步生成摘要]
+ V -->|是| SM[add_async_summary_task
异步持久化]
+ CM --> R[返回 messages_to_keep + 新摘要]
+```
+
+**执行流程**：
+
+1. `compact_tool_result` — 压缩超长工具输出
+2. `check_context` — 检查上下文是否超限
+3. `compact_memory` — 生成压缩摘要（同步）
+4. `summary_memory` — 持久化记忆（异步后台）
+
---
## 🗃️ 基于向量库的记忆系统
[ReMe Vector Based](reme/reme.py) 是基于向量库的记忆系统核心类，支持三种记忆类型的统一管理：
-| 记忆类型 | 用途 | 使用场景 |
-|--------------|------------------|-------------|
-| **个人记忆** | 记录用户偏好、习惯 | `user_name` |
-| **任务/程序性记忆** | 记录任务执行经验、成功/失败模式 | `task_name` |
-| **工具记忆** | 记录工具使用经验、参数优化 | `tool_name` |
+| 记忆类型 | 用途 |
+|--------------|------------------|
+| **个人记忆** | 记录用户偏好、习惯 |
+| **任务/程序性记忆** | 记录任务执行经验、成功/失败模式 |
+| **工具记忆** | 记录工具使用经验、参数优化 |
### 核心能力
@@ -302,24 +366,11 @@ M --> R[Top-N 结果]
| `delete_memory` | 🗑️ 删除记忆 | 删除指定记忆 |
| `list_memory` | 📋 列出记忆 | 列出某类记忆，支持过滤和排序 |
-### 安装
+### 安装与环境变量
-```bash
-pip install -U reme-ai
-```
+安装和环境变量配置与 [ReMeLight 一致](#安装)，通过环境变量设置 API 密钥，可写在项目根目录的 `.env` 文件中。
-### 环境变量
-
-API 密钥通过环境变量设置，可写在项目根目录的 `.env` 文件中：
-
-| 环境变量 | 说明 | 示例 |
-|----------------------|--------------------------|-----------------------------------------------------|
-| `LLM_API_KEY` | LLM 的 API Key | `sk-xxx` |
-| `LLM_BASE_URL` | LLM 的 Base URL | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
-| `EMBEDDING_API_KEY` | Embedding 的 API Key（可选） | `sk-xxx` |
-| `EMBEDDING_BASE_URL` | Embedding 的 Base URL（可选） | `https://dashscope.aliyuncs.com/compatible-mode/v1` |
-
-### Python使用
+### Python 使用
```python
import asyncio
@@ -413,7 +464,7 @@ if __name__ == "__main__":
### 技术架构
```mermaid
-graph TB
+graph LR
User[用户 / Agent] --> ReMe[Vector Based ReMe]
ReMe --> Summarize[记忆总结]
ReMe --> Retrieve[记忆检索]
@@ -432,6 +483,42 @@ graph TB
ToolRet --> VectorStore
```
+### 实验效果
+
+Coming soon...
+
+---
+
+## 🧪 程序化记忆论文
+
+> 我们的程序性（任务）记忆论文已在 [arXiv](https://arxiv.org/abs/2512.10696) 发布
+
+### 🌍 [Appworld 实验](benchmark/appworld/quickstart.md)
+
+我们在 Appworld 环境上使用 Qwen3-8B（非思考模式）进行评测：
+
+| 方法 | Avg@4 | Pass@4 |
+|---------|---------------------|---------------------|
+| 无 ReMe | 0.1497 | 0.3285 |
+| 使用 ReMe | 0.1706 **(+2.09%)** | 0.3631 **(+3.46%)** |
+
+Pass@K 衡量在生成 K 个候选中，至少一个成功完成任务（score=1）的概率。
+当前实验使用的是内部 AppWorld 环境，可能与对外版本存在轻微差异。
+
+关于如何复现实验的更多细节，见 [quickstart.md](benchmark/appworld/quickstart.md)
+
+### 🔧 [BFCL-V3 实验](benchmark/bfcl/quickstart.md)
+
+我们在 BFCL-V3 multi-turn-base 任务（随机划分 50 train / 150 val）上，使用 Qwen3-8B（思考模式）进行评测：
+
+| 方法 | Avg@4 | Pass@4 |
+|---------|---------------------|---------------------|
+| 无 ReMe | 0.4033 | 0.5955 |
+| 使用 ReMe | 0.4450 **(+4.17%)** | 0.6577 **(+6.22%)** |
+
+关于如何复现实验的更多细节，见 [quickstart.md](benchmark/bfcl/quickstart.md)
+
+
## ⭐ 社区与支持
- **Star 与 Watch**：Star 可让更多智能体开发者发现 ReMe；Watch 可助你第一时间获知新版本与特性。
diff --git a/docs/cookbook/appworld/quickstart.md b/benchmark/appworld/quickstart.md
similarity index 100%
rename from docs/cookbook/appworld/quickstart.md
rename to benchmark/appworld/quickstart.md
diff --git a/docs/cookbook/bfcl/quickstart.md b/benchmark/bfcl/quickstart.md
similarity index 100%
rename from docs/cookbook/bfcl/quickstart.md
rename to benchmark/bfcl/quickstart.md
diff --git a/benchmark/halumem/scripts.sh b/benchmark/halumem/cat_correct_scripts.sh
similarity index 100%
rename from benchmark/halumem/scripts.sh
rename to benchmark/halumem/cat_correct_scripts.sh
diff --git a/benchmark/halumem/eval_scripts.sh b/benchmark/halumem/eval_scripts.sh
new file mode 100755
index 00000000..c8a06881
--- /dev/null
+++ b/benchmark/halumem/eval_scripts.sh
@@ -0,0 +1,5 @@
+clear && python benchmark/halumem/eval_reme.py \
+ --data_path /Users/yuli/workspace/HaluMem/data/HaluMem-Medium.jsonl \
+ --reme_model_name qwen3.5-plus \
+ --batch_size 10000 \
+ --algo_version default
\ No newline at end of file
diff --git a/docs/REME2_README.md b/docs/REME2_README.md
deleted file mode 100644
index fd6174a6..00000000
--- a/docs/REME2_README.md
+++ /dev/null
@@ -1,13 +0,0 @@
-
-
-# TODO
-- [] halumem bench开发
-- [] default版本开发，for cli版本体验
-- [] cli开发
-- [] locomo bench开发
-- [] task memory迁移
-- [] mcp开发
-- [] reme外层接口完善
-- [] reme2 readme完善
-- [] 看日志，看要这个default版本怎么优化。
-- [] 学习Clawdbot记忆系统
\ No newline at end of file
diff --git a/docs/deprecated.txt b/docs/deprecated.txt
deleted file mode 100644
index 06f9648f..00000000
--- a/docs/deprecated.txt
+++ /dev/null
@@ -1,13 +0,0 @@
-from loguru import logger
-
-用英文注释，完善module/class/function docstring，要一句话简洁，不要变更代码逻辑，符合pep和pylint规范，使用list而不是typing.List/Dict，不使用typing.Union
-
-看看代码有什么问题
-用英文注释，完善module/class/function docstring，要一句话简洁，代码要简洁，符合pep和pylint规范，使用list而不是typing.List，不使用typing.Union
-C0114: Missing module docstring (missing-module-docstring)
-C0115: Missing class docstring (missing-class-docstring)
-C0116: Missing function or method docstring (missing-function-docstring)
-done: { for f in ./*.py; do [[ "$f" != "./__init__.py" ]] && grep -v '^[[:space:]]*#' "$f"; done; } | pbcopy
-
-然后是一个完整的tests，但是不要用其他的包，只是test开头的函数或者类，要求from loguru import logger
-写一个测试文件，不要使用pytest，普通的test，要求英文注释
\ No newline at end of file
diff --git a/docs/future_work.md b/docs/future_work.md
deleted file mode 100644
index 807abfed..00000000
--- a/docs/future_work.md
+++ /dev/null
@@ -1,18 +0,0 @@
-# Future Work
-
-- [ ] P0 ReMe documentation style migration: Recommend using the same doc and jupyter structure as Agentscope Runtime @jiaji
-- [ ] P0 ReMe integration with agentscope Personal/Task/Tool @jinli
-- [ ] P0 ReMe sample library examples [show case](https://github.com/agentscope-ai/agentscope-samples/tree/main/functionality/long_term_memory_mem0) @jinli
-- [ ] P0 Decouple flowllm dependencies @jinli
-- [ ] P0 ReMe support for import, improve code documentation @jinli
-- [ ] P1 ReMe integration with asio tool_memory @jinli
-- [ ] P2 ReMe integration with agentscope-Runtime tool_memory @jinli
-
-- [ ] P0 Task Memory Research Paper @zhoyin
-
-- [ ] P1 Context interface definition @jinli
-
-- [ ] P2 Database layer interface unification @jinli
-- [ ] P2 Automatic Tool Exploration Mode @wangcan
-- [ ] P2 Mem-Agent Exploration @weikang
-- [ ] P2 Desktop Pet Personal Assistant
diff --git a/docs/reme_v2_design.md b/docs/reme_v2_design.md
deleted file mode 100644
index 6fa78140..00000000
--- a/docs/reme_v2_design.md
+++ /dev/null
@@ -1,735 +0,0 @@
-# ReMeV2 深度设计文档：渐进式 Agentic Memory 方案
-
-## 一、 背景与现状分析
-
-### 1.1 当前面临的挑战
-
-* **外功修炼（接口易用性）**：现有的 `server-client` 模式对新手开发者不够友好，集成成本高，需要更直观、纯 Pythonic 的调用方式。
-* **内功修炼（架构深度）**：受 `skills` 和 `agentic memory` 启发，现有的存储检索较为机械。我们需要一种基于**渐进式检索（Progressive Retrieval）**与**渐进式总结（Progressive Summarization）**的智能体记忆方案。
-
-### 1.2 核心目标
-
-1. **极简开发体验**：开发者友好，全异步接口，支持本地直接运行与 CLI 体验。
-2. **认知架构升级**：引入 渐进式检索 & 渐进式总结 的 Agentic 模式，融合多种记忆，让记忆的存取具备“思考”过程。
-3. **生态融合**：原生支持 AgentScope、LangChain 等主流框架。
-
----
-
-## 二、 竞品调研与启示
-
-### 2.1 主流竞品深度对比
-
-| 产品 | 设计哲学 | 核心优势 | 局限性 |
-|-------------|----------|---------------------------------------------|-------------------|
-| **mem0** | 智能便签本 | 原子事实提取，极高 Token 效率。 | 缺乏对复杂逻辑链条的支持。 |
-| **Letta** | 带硬盘的 CPU | 模拟计算机三级存储（Core/Recall/Archival），Agent 自主控存。 | 状态机管理相对复杂。 |
-| **MIRIX** | 认知架构图谱 | 实体-关系双引擎，支持记忆“进化”与“固化”。 | 侧重研究，落地集成门槛较高。 |
-| **LangMem** | 用户档案系统 | 异步 Compaction（压缩），Schema 驱动，强一致性。 | 偏向 SaaS 应用，灵活性略逊。 |
-
-### 2.2 mem0
-- https://github.com/mem0ai/mem0
-- https://docs.mem0.ai/core-concepts/memory-operations/add
-- https://docs.mem0.ai/core-concepts/memory-operations/search
-- https://docs.mem0.ai/core-concepts/memory-operations/update
-- https://docs.mem0.ai/core-concepts/memory-operations/delete
-
-#### 2.2.1 API Reference
-| 接口名称 | 核心输入参数 (Inputs) | 核心输出 (Outputs) | 背后逻辑 (Internal Logic) |
-| --- | --- | --- | --- |
-| **Add** | `messages` (文本/对话), `user_id`, `metadata` | `id`, `event` (ADD/UPDATE), `data` | **提取与合并**：LLM 提取事实，自动去重并更新已有记忆，而非简单堆叠。 |
-| **Search** | `query` (自然语言), `filters`, `limit` | `id`, `memory` (事实文本), `score`, `metadata` | **语义检索**：基于向量相似度查找最相关的“原子事实”，支持多维过滤。 |
-| **Update** | `memory_id` (必填), `data` (新内容) | 操作状态 (Success/Fail) | **手动干预**：允许开发者对特定的事实进行精确修正。 |
-| **Delete** | `memory_id` 或 `user_id` (清空) | 操作状态 (Success/Fail) | **遗忘机制**：物理删除或逻辑移除不再需要的信息。 |
-
-#### 2.2.2 Tech Strategy & Benefits
-| 维度 | 技术方案 (Technical Solution) | 核心优势 (Key Advantages) |
-| --- | --- | --- |
-| **存储架构** | **混合存储**：向量数据库 (Vector) + 图数据库 (Graph) + 关系型元数据。 | **多维关联**：不仅能搜到相似内容，还能理解实体间的逻辑关系（如“父子”、“因果”）。 |
-| **数据处理** | **原子化事实提取**：利用 LLM 将长篇对话压缩为简短的 Fact。 | **极高 Token 效率**：注入 Prompt 的内容更精炼，减少 90% 以上的冗余信息，大幅降本。 |
-| **管理层级** | **多级联动**：User (长期) Agent (专业) Session (短期)。 | **个性化定制**：实现跨会话的“长效记忆”，AI 能记住用户一个月前说过的偏好。 |
-| **冲突处理** | **自适应更新算法**：新信息进入时自动比对旧记忆。 | **数据一致性**：自动处理矛盾信息（如用户更换了住址），确保记忆库始终是“最新真理”。 |
-| **兼容性** | **解耦设计**：支持多种 Embedding 模型与向量数据库后端。 | **快速集成**：几行代码即可为现有 LLM 应用增加记忆层，适配各种生产环境。 |
-
-
----
-
-### 2.3 Letta
-- https://github.com/letta-ai/letta
-- https://docs.letta.com/guides/agents/archival-memory/
-- https://docs.letta.com/guides/agents/archival-search/
-
-#### 2.3.1 存储架构层级 (Memory Tiering)
-
-Letta 将记忆分为三个物理/逻辑层，模拟计算机的存储架构：
-
-| 记忆层级 | 存储介质 | 访问方式 | 核心作用 |
-| --- | --- | --- | --- |
-| **Core Memory** | **上下文窗口 (Prompt)** | 直接读写 | **即时意识**：包含 `Persona`（AI 设定）和 `Human`（用户信息）。Agent 随时可见，响应最快。 |
-| **Recall Memory** | **关系型数据库 (SQL)** | 分页检索 | **短期/历史回顾**：存储完整的对话流（Messages）。用于回答“你刚才说了什么”。 |
-| **Archival Memory** | **向量数据库 (Vector)** | 语义搜索 | **长期知识库**：存储海量事实或文档。Agent 通过工具自主检索或存入。 |
-
-#### 2.3.2 核心操作接口 (API & Tool Reference)
-
-在 Letta 中，记忆的操作通常封装为 **Tools**，由 Agent 根据推理需求主动调用。
-
-| 接口/工具名称 | 输入参数 (Inputs) | 核心输出 (Outputs) | 背后逻辑 (Internal Logic) |
-| --- | --- | --- | --- |
-| **`core_memory_update`** | `section`, `new_content` | 更新后的段落内容 | **原子替换**：直接修改 System Prompt 中的特定块（如：更新用户的职业或 AI 的性格偏好）。 |
-| **`archival_memory_insert`** | `content` (字符串) | 写入状态/ID | **知识沉淀**：将当前对话中的重要信息或外部文件片段“持久化”到向量数据库。 |
-| **`archival_memory_search`** | `query`, `page` | 匹配的文本块列表 | **主动 RAG**：Agent 意识到知识不足时，自主发起向量检索，并将结果拉入临时上下文。 |
-| **`conversation_search`** | `query`, `start_date` | 历史消息记录 | **全文检索**：在 Recall Memory 中根据关键词或时间戳查找历史对话详情。 |
-| **`send_message`** | `message`, `agent_id` | 响应流/状态更新 | **状态循环**：这是主入口，触发 Agent 的“思考-行动-观察”循环，自动处理内存同步。 |
-
-#### 2.3.3 技术策略与核心优势 (Tech Strategy & Benefits)
-
-| 维度 | 技术方案 (Technical Solution) | 核心优势 (Key Advantages) |
-| --- | --- | --- |
-| **状态持久化** | **Agent State Snapshot**：将 Agent 的所有内存、工具定义和历史记录打包存入数据库。 | **无限存续**：Agent 不再是无状态的 API 调用。重启服务器后，Agent 依然记得所有细节。 |
-| **自主演进** | **Self-Editing Loop**：Agent 拥有修改自己 Core Memory 的权限（通过函数调用）。 | **认知闭环**：AI 能在交流中发现矛盾并自我更正，例如发现用户搬家后自动更新 `Human` 模块。 |
-| **算力调度** | **OOC (Out-of-Context) 管理**：当对话过长，系统自动将旧消息从 Core 移入 Recall。 | **突破 Context 限制**：在 8k 窗口的模型上也能处理相当于 1M 窗口的逻辑量，且成本更低。 |
-| **多代理协同** | **Letta Server 中控**：统一管理多个 Agent 的状态机与资源访问权限。 | **企业级扩展**：支持创建 Agent 团队，每个 Agent 拥有独立的记忆空间但可共享 Archival 库。 |
-| **解耦灵活性** | **Provider Agnostic**：后端支持 Postgres/Chroma，前端支持 OpenAI/Anthropic/Local LLMs。 | **无缝迁移**：不绑定特定模型，开发者可以根据成本或能力随时更换底座。 |
-
-#### 2.3.4 与 mem0 的深度对比
-
-* **设计哲学**：
-* **mem0** 像是一个**“智能记事本”**，它在后台默默地帮你总结事实。
-* **Letta** 像是一个**“带硬盘的 CPU”**，它把记忆管理完全交给了 Agent 自己的逻辑推理。
-
-
-* **交互模式**：
-* **mem0** 通常是外部干预（Add/Search）。
-* **Letta** 强调 **Agentic Control**（Agent 意识到需要搜索时才去搜索），这种模式更接近人类的思维过程。
-
----
-
-### 2.4 MIRIX
-- https://github.com/Mirix-AI/MIRIX
-- https://docs.mirix.io/
-
-#### 2.4.1 API Reference
-
-| 接口名称 | 核心输入参数 (Inputs) | 核心输出 (Outputs) | 背后逻辑 (Internal Logic) |
-| --- | --- | --- | --- |
-| **Add** | `content` (观察/对话), `agent_id`, `context_type` (如任务/闲聊) | `memory_id`, `graph_nodes`, `status` | **实体建模**：不只是提取事实，而是将信息拆解为实体（Entities）与关系（Relations），并挂载到智能体的知识图谱中。 |
-| **Query** | `query` (意图), `scope` (全局/局部), `top_k` | `retrieved_memories`, `relation_paths`, `score` | **混合检索**：结合向量（Vector）的语义相关性和图（Graph）的拓扑连接性，寻找具有逻辑深度背景的记忆。 |
-| **Evolve** | `target_memories` (可选), `agent_id` | `optimized_structure`, `merged_nodes` | **记忆固化/压缩**：模仿人类大脑的“睡眠”机制，自动合并碎片化记忆，将短期经验转化为长期的结构化知识。 |
-| **Observe** | `interaction_stream`, `feedback` | `insights`, `priority_update` | **实时学习**：根据用户反馈或环境变化，动态调整记忆的权重（Importance）和置信度。 |
-
-#### 2.4.2 Tech Strategy & Benefits
-
-| 维度 | 技术方案 (Technical Solution) | 核心优势 (Key Advantages) |
-| --- | --- | --- |
-| **存储架构** | **语义-关系双引擎**：向量索引（Vector Index）+ 属性图（Property Graph）。 | **深度上下文**：不仅知道“是什么”，还能通过图路径推理出“为什么”，有效解决 LLM 幻觉问题。 |
-| **记忆层级** | **三层架构**：感知记忆 (Perception) -> 语义记忆 (Semantic) -> 经验记忆 (Episodic)。 | **任务适应性**：不同任务自动匹配不同的记忆深度，短期任务关注细节，长期任务关注模式。 |
-| **演化机制** | **自主固化 (Self-Consolidation)**：通过 LLM 定期对冗余、矛盾信息进行清洗和逻辑抽象。 | **永久生命力**：解决随时间推移记忆库膨胀导致的检索噪声，确保记忆库“越用越聪明”。 |
-| **推理增强** | **基于记忆的 RAG+**：在检索到的事实基础上，额外提供关联的逻辑链条（Logic Chains）。 | **辅助决策**：为 Agent 提供决策支撑，使其在处理复杂流程时具备类似“长期经验值”的直觉。 |
-| **多代理协同** | **内存共享协议**：支持 Agent 之间的记忆交换与知识同步。 | **群体智能**：多个 Agent 可以共享同一套底层知识体系，同时保留各自的私有工作记忆。 |
-
-#### 2.4.3 与 mem0 的主要区别
-
-* **Mem0** 侧重于**个性化偏好存储**（Personalization），核心是记住“用户喜欢什么”。
-* **MIRIX** 侧重于**智能体认知架构**（Agent Cognition），核心是让 Agent 具备类似人类的“知识归纳”和“逻辑推理”记忆能力。
-
----
-
-### 2.5 LangMem
-- https://github.com/langchain-ai/langmem
-- https://langchain-ai.github.io/langmem/
-
-#### 2.5.1 API Reference
-
-| 接口名称 | 核心输入参数 (Inputs) | 核心输出 (Outputs) | 背后逻辑 (Internal Logic) |
-| --- | --- | --- | --- |
-| **Add Messages** | `thread_id`, `messages` (List), `user_id` | 操作确认 / 任务 ID | **流式注入**：将原始对话追加到指定的 Thread。LangMem 会自动关联用户上下文，准备进行后续的异步处理。 |
-| **Query Memory** | `user_id`, `query` (语义描述), `namespace` | 结构化记忆对象 (JSON / Text) | **多维检索**：不仅支持向量相似度搜索，还能根据定义的 Schema 返回结构化的用户画像或知识状态。 |
-| **Trigger Logic** | `thread_id`, `memory_type` | 更新后的 Memory State | **异步固化**：后台启动 LLM 任务，将长篇对话“压缩”并“提取”到长期存储中。支持自定义提取逻辑（如更新用户信息）。 |
-| **Manage State** | `user_id`, `patch_data` (增量更新) | 成功/失败 状态 | **精确受控**：开发者可以直接修改持久化的状态（State），支持类似于 Git 的状态管理。 |
-
-#### 2.5.2 Tech Strategy & Benefits
-
-| 维度 | 技术方案 (Technical Solution) | 核心优势 (Key Advantages) |
-| --- | --- | --- |
-| **存储架构** | **Stateful Persistence**：基于关系型数据库 (Postgres) + 向量索引。 | **强一致性**：利用数据库事务确保记忆更新的可靠性，支持复杂的结构化查询与过滤。 |
-| **数据处理** | **异步化 Compaction (压缩)**：在对话间隙通过后台 Worker 提取知识。 | **无感延迟**：核心对话流程不被记忆提取阻塞，通过定时或事件驱动完成“记忆固化”，优化用户体验。 |
-| **管理层级** | **Thread -> User -> Organization**：三层级联记忆。 | **上下文隔离**：完美适配 SaaS 应用场景，既能记住单次对话（Thread），也能沉淀用户习惯（User）。 |
-| **逻辑引擎** | **Schema-Driven (模式驱动)**：允许定义 JSON Schema 来规范记忆内容。 | **高度可预测**：输出不再是散乱的句子，而是结构化的字段，方便下游程序直接调用逻辑（如自动填充表单）。 |
-| **集成生态** | **LangGraph 原生集成**：作为 Checkpointer 或存储节点直接接入。 | **生态协同**：如果你已经在用 LangChain，LangMem 可以无缝接管状态流转，无需重写底层存储逻辑。 |
-
-#### 2.5.3 与 mem0 的核心差异
-
-* **mem0** 像是一个**“便签本”**：它擅长从每一句话里抠出零散的事实（如“我喜欢吃苹果”），然后把它们存成一条条语义片段。
-* **LangMem** 像是一个**“用户档案系统”**：它更擅长分析一整段对话，然后更新一个复杂的 JSON 档案（如更新用户的偏好模型、性格标签、历史任务状态）。
-
----
-
-## 三、 ReMeV2 API 接口设计
-
-### 3.1 Long-Term Memory (长期记忆)
-
-#### 3.1.1 Basic Usage (基础用法)
-
-The most straightforward way to use ReMe for long-term memory management. Supports basic summary and retrieval operations.
-
-```python
-import os
-from reme_ai import ReMe
-
-os.environ["REME_LLM_API_KEY"] = "sk-..."
-os.environ["REME_LLM_BASE_URL"] = "https://dashscope.aliyuncs.com/compatible-mode/v1"
-os.environ["REME_EMBEDDING_API_KEY"] = "sk-..."
-os.environ["REME_EMBEDDING_BASE_URL"] = "https://dashscope.aliyuncs.com/compatible-mode/v1"
-
-memory = ReMe(
- memory_space="remy", # workspace identifier
- llm={"backend": "openai", "model": "qwen-plus", "temperature": 0.6},
- embedding={"backend": "openai", "model": "text-embedding-v4", "dimension": 1024},
- vector_store={"backend": "local_file"}, # supported: local_file, chromadb, qdrant, etc.
-)
-
-# Summarize conversation into memory
-result = await memory.summary(
- messages=[
- {"role": "user", "content": "I'm travelling to SF"},
- {"role": "assistant", "content": "That's great to hear!"}
- ],
- user_id="Alice",
- # memory_type="auto" # default: auto (auto, personal, procedural, tool)
-)
-
-# Retrieve relevant memories
-memories = await memory.retrieve(
- query="what is your travel plan?",
- limit=3,
- user_id="Alice",
- # memory_type="auto" # default: auto
-)
-memories_str = "\n".join(f"- {m['memory']}" for m in memories["results"])
-print(memories_str)
-```
-
-#### 3.1.2 CLI Chat Application (命令行聊天应用)
-
-A complete example demonstrating how to build a memory-enhanced chatbot with CLI interface.
-
-```python
-import os
-from reme_ai import ReMe
-from openai import OpenAI
-
-os.environ["REME_LLM_API_KEY"] = "sk-..."
-os.environ["REME_LLM_BASE_URL"] = "https://dashscope.aliyuncs.com/compatible-mode/v1"
-os.environ["REME_EMBEDDING_API_KEY"] = "sk-..."
-os.environ["REME_EMBEDDING_BASE_URL"] = "https://dashscope.aliyuncs.com/compatible-mode/v1"
-
-memory = ReMe(
- memory_space="remy",
- llm={"backend": "openai", "model": "qwen-plus", "temperature": 0.6},
- embedding={"backend": "openai", "model": "text-embedding-v4", "dimension": 1024},
- vector_store={"backend": "local_file"},
-)
-
-os.environ["OPENAI_API_KEY"] = "sk-..."
-os.environ["OPENAI_BASE_URL"] = "https://dashscope.aliyuncs.com/compatible-mode/v1"
-openai_client = OpenAI()
-
-def chat_with_memories(
- query: str,
- history_messages: list[dict],
- user_name: str = "",
- start_summary_size: int = 2,
- keep_size: int = 0
-) -> str:
- # Retrieve relevant memories for the query
- memories = memory.retrieve(query=query, user_id=user_name, limit=3)
-
- # Build system prompt with memories
- system_prompt = (
- "You are a helpful AI named `Remy`. Use the user memories to answer the question. "
- "If you don't know the answer, just say you don't know. Don't try to make up an answer.\n"
- )
- if memories:
- memories_str = "\n".join(f"- {m['memory']}" for m in memories["results"])
- system_prompt += f"User Memories:\n{memories_str}\n"
-
- # Generate response
- system_message = {"role": "system", "content": system_prompt}
- history_messages.append({"role": "user", "content": query})
- response = openai_client.chat.completions.create(
- model="qwen-plus",
- messages=[system_message] + history_messages
- )
- history_messages.append({"role": "assistant", "content": response.choices[0].message.content})
-
- # Summarize history when it gets too long
- if len(history_messages) >= start_summary_size:
- memory.summary(history_messages[:-keep_size], user_id=user_name)
- print("Current memories: " + memory.list_memories(user_id=user_name))
- history_messages = history_messages[-keep_size:]
-
- return history_messages[-1]["content"]
-
-def main():
- user_name = input("Enter your name: ").strip()
- print("Chat with Remy (type 'exit' to quit)")
-
- messages = []
- while True:
- user_input = input(f"{user_name}: ").strip()
- if user_input.lower() == 'exit':
- print("Goodbye!")
- break
-
- print(f"Remy: {chat_with_memories(user_input, messages, user_name)}")
-
- # Cleanup
- memory.delete_all_memories(user_id=user_name)
- print("All memories deleted")
-
-if __name__ == "__main__":
- main()
-```
-
-#### 3.1.3 Advanced Usage (高级用法)
-
-For advanced users who want to customize retriever and summarizer behavior with Agentic mode.
-
-```python
-import os
-from reme_ai import ReMe
-from reme_ai.retriever import AgenticRetriever
-from reme_ai.summarizer import AgenticSummarizer
-from reme_ai.tools import ATool, BTool, CTool
-
-os.environ["REME_LLM_API_KEY"] = "sk-..."
-os.environ["REME_LLM_BASE_URL"] = "https://dashscope.aliyuncs.com/compatible-mode/v1"
-os.environ["REME_EMBEDDING_API_KEY"] = "sk-..."
-os.environ["REME_EMBEDDING_BASE_URL"] = "https://dashscope.aliyuncs.com/compatible-mode/v1"
-
-memory = ReMe(
- memory_space="remy",
- llm={"backend": "openai", "model": "qwen-plus", "temperature": 0.6},
- embedding={"backend": "openai", "model": "text-embedding-v4", "dimension": 1024},
- vector_store={"backend": "local_file"},
- use_agentic_mode=True,
-)
-
-# Customize retriever and summarizer with custom tools and prompts
-memory.set_retriever(
- AgenticRetriever(tools=[ATool(), BTool(), CTool()]),
- system_prompt="Custom retrieval instructions..."
-)
-memory.set_summarizer(
- AgenticSummarizer(tools=[ATool(), BTool(), CTool()])
-)
-
-# Use the customized memory system
-result = memory.summary(
- messages=[
- {"role": "user", "content": "I'm travelling to SF"},
- {"role": "assistant", "content": "That's great to hear!"}
- ],
- user_id="Alice",
- memory_type="auto", # auto, personal, procedural, tool
-)
-
-memories = memory.retrieve(
- query="what is your travel plan?",
- limit=3,
- user_id="Alice",
- memory_type="auto",
-)
-memories_str = "\n".join(f"- {m['memory']}" for m in memories["results"])
-print(memories_str)
-```
-
-### 3.2 Short-Term Memory (短期记忆)
-
-#### 3.2.1 Basic Usage (基础用法)
-
-Context offload/reload API for managing short-term conversational memory within a session.
-
-```python
-import os
-from reme_ai import ReMe
-
-os.environ["REME_LLM_API_KEY"] = "sk-..."
-os.environ["REME_LLM_BASE_URL"] = "https://dashscope.aliyuncs.com/compatible-mode/v1"
-os.environ["REME_EMBEDDING_API_KEY"] = "sk-..."
-os.environ["REME_EMBEDDING_BASE_URL"] = "https://dashscope.aliyuncs.com/compatible-mode/v1"
-
-memory = ReMe(
- memory_space="remy",
- llm={"backend": "openai", "model": "qwen-plus", "temperature": 0.6},
- embedding={"backend": "openai", "model": "text-embedding-v4", "dimension": 1024},
- vector_store={"backend": "local_file"},
-)
-
-# Offload context when conversation gets too long
-result = memory.offload_context(
- messages=[
- {"role": "user", "content": "I'm travelling to SF"},
- {"role": "assistant", "content": "That's great to hear!"}
- ],
-)
-
-# Reload relevant context when needed
-memories = memory.reload_context(
- query="what is your travel plan?",
- limit=3,
-)
-memories_str = "\n".join(f"- {m['memory']}" for m in memories["results"])
-print(memories_str)
-```
-
-### 3.3 Framework Integration (框架集成)
-
-#### 3.3.1 Integration with AgentScope
-
-Integration example for AgentScope ReActAgent with long-term memory support.
-
-```python
-# TODO: Provide AgentScope integration example
-```
-
-#### 3.3.2 Integration with LangChain
-
-Integration example for LangChain agents with ReMe memory layer.
-
-```python
-# TODO: Provide LangChain integration example
-```
-
-### 3.4 OpenAI Compatible Interface
-
-OpenAI-compatible API interface for seamless integration with existing OpenAI-based applications.
-
-```python
-# TODO: Research and implement OpenAI-compatible interface
-# - Support for threads and assistants API
-# - Compatible with OpenAI SDK
-# - Support for streaming responses
-```
-
-
-
----
-
-## 四、核心方案设计
-
-### 4.1 设计概述
-
-ReMeV2 采用简洁的架构设计，核心理念为：**ReMeV2 = Tool(s) + Agent(s)**
-
-- **Tool层**：提供原子化的记忆操作能力，包括增删改查、检索、元数据管理等基础操作
-- **Agent层**：基于Tool层构建的智能代理，负责复杂的记忆管理逻辑，如分类总结、渐进式检索等
-- **Runtime层**：内部调度机制，协调Tool和Agent的交互流程
-
-### 4.2 Tool层设计
-
-Tool层提供装饰器形式的记忆操作工具，每个工具类通过 `@tool` 装饰器注册，明确定义初始化参数和调用参数。
-
-#### 4.2.1 基类：BaseMemoryToolOp
-
-**初始化参数：**
-- `enable_multiple` (bool): Enable multi-item operation mode. Default: `True`
-- `enable_thinking_params` (bool): Include thinking parameter in tool schema for model reasoning. Default: `False`
-- `memory_metadata_dir` (str): Directory path for storing memory metadata. Default: `"./memory_metadata"`
-
-#### 4.2.2 Tool操作列表
-
-以下是所有Tool操作的完整定义，包括继承关系、初始化参数和调用参数：
-
-| Tool类 | 继承自 | 初始化参数（除基类外） | Tool Call参数（单项模式） | Tool Call参数（多项模式） |
-|----------------------------|------------------|------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
-| **AddMemoryOp** | BaseMemoryToolOp | `add_when_to_use` (bool, 默认: False)
`add_metadata` (bool, 默认: True) | `when_to_use` (str, 可选)
`memory_content` (str, 必需)
`metadata` (dict, 可选) | `memories` (array, 必需):
- `when_to_use` (str, 可选)
- `memory_content` (str, 必需)
- `metadata` (dict, 可选) |
-| **UpdateMemoryOp** | BaseMemoryToolOp | 无 | `memory_id` (str, 必需)
`memory_content` (str, 必需)
`metadata` (dict, 可选) | `memories` (array, 必需):
- `memory_id` (str, 必需)
- `memory_content` (str, 必需)
- `metadata` (dict, 可选) |
-| **DeleteMemoryOp** | BaseMemoryToolOp | 无 | `memory_id` (str, 必需) | `memory_ids` (array[str], 必需) |
-| **VectorRetrieveMemoryOp** | BaseMemoryToolOp | `enable_summary_memory` (bool, 默认: False)
`add_memory_type_target` (bool, 默认: False)
`top_k` (int, 默认: 20) | `query` (str, 必需)
`memory_type` (str, 可选, 枚举: [identity, personal, procedural])
`memory_target` (str, 可选) | `query_items` (array, 必需):
- `query` (str, 必需)
- `memory_type` (str, 可选)
- `memory_target` (str, 可选) |
-| **AddMetaMemoryOp** | BaseMemoryToolOp | 无 | `memory_type` (str, 必需, 枚举: [personal, procedural])
`memory_target` (str, 必需) | `meta_memories` (array, 必需):
- `memory_type` (str, 必需)
- `memory_target` (str, 必需) |
-| **ReadMetaMemoryOp** | BaseMemoryToolOp | `enable_tool_memory` (bool, 默认: False)
`enable_identity_memory` (bool, 默认: False) | 无（无输入schema） | N/A (enable_multiple=False) |
-| **AddHistoryMemoryOp** | BaseMemoryToolOp | 无 | `messages` (array[object], 必需) | N/A (enable_multiple=False) |
-| **ReadHistoryMemoryOp** | BaseMemoryToolOp | 无 | `memory_id` (str, 必需) | `memory_ids` (array[str], 必需) |
-| **AddSummaryMemoryOp** | AddMemoryOp | 无（继承自AddMemoryOp） | `summary_memory` (str, 必需)
`metadata` (dict, 可选) | N/A (enable_multiple=False) |
-| **ReadIdentityMemoryOp** | BaseMemoryToolOp | 无 | 无（无输入schema） | N/A (enable_multiple=False) |
-| **UpdateIdentityMemoryOp** | BaseMemoryToolOp | 无 | `identity_memory` (str, 必需) | N/A (enable_multiple=False) |
-| **ThinkToolOp** | BaseAsyncToolOp | `add_output_reflection` (bool, 默认: False) | `reflection` (str, 必需) | N/A |
-| **HandsOffOp** | BaseMemoryToolOp | 无 | `memory_type` (str, 必需, 枚举: [identity, personal, procedural, tool])
`memory_target` (str, 必需) | `memory_tasks` (array, 必需):
- `memory_type` (str, 必需)
- `memory_target` (str, 必需) |
-
-### 4.3 Agent层设计
-
-#### 4.3.1 基类：BaseMemoryAgentOp
-
-Agent层构建在Tool层之上，封装复杂的记忆管理逻辑。每个Agent通过组合多个Tool实现特定的记忆管理任务。
-
-#### 4.3.2 Agent操作列表
-
-以下是所有Agent操作的完整定义，包括初始化参数、调用参数和可用工具：
-
-| Agent类 | 继承自 | 初始化参数（基类外） | Tool Call参数 | 可用工具 |
-|--------------------------------|-------------------|------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------|
-| **PersonalSummaryAgentV1Op** | BaseMemoryAgentOp | None | `workspace_id` (str, required)
`memory_target` (str, required)
`query` (str, optional)
`messages` (array, optional)
`ref_memory_id` (str, required) | add_memory
update_memory
delete_memory
vector_retrieve_memory |
-| **ProceduralSummaryAgentV1Op** | BaseMemoryAgentOp | None | `workspace_id` (str, required)
`memory_target` (str, required)
`query` (str, optional)
`messages` (array, optional)
`ref_memory_id` (str, required) | add_memory
update_memory
delete_memory
vector_retrieve_memory |
-| **ToolSummaryAgentV1Op** | BaseMemoryAgentOp | None | `workspace_id` (str, required)
`memory_target` (str, required)
`query` (str, optional)
`messages` (array, optional)
`ref_memory_id` (str, required) | add_memory
update_memory
vector_retrieve_memory |
-| **IdentitySummaryAgentV1Op** | BaseMemoryAgentOp | None | `workspace_id` (str, required)
`query` (str, optional)
`messages` (array, optional) | read_identity_memory
update_identity_memory |
-| **ReMeSummaryAgentV1Op** | BaseMemoryAgentOp | `enable_tool_memory` (bool, 默认: True)
`enable_identity_memory` (bool, 默认: True) | `workspace_id` (str, required)
`query` (str, optional)
`messages` (array, optional) | add_meta_memory
add_summary_memory
hands_off
(内部调用: add_history_memory, read_identity_memory, read_meta_memory) |
-| **ReMeRetrieveAgentV1Op** | BaseMemoryAgentOp | `enable_tool_memory` (bool, 默认: True) | `workspace_id` (str, required)
`query` (str, optional)
`messages` (array, optional) | vector_retrieve_memory
read_history_memory
(内部调用: read_meta_memory) |
-| **ReMyAgentV1Op** | BaseMemoryAgentOp | `enable_tool_memory` (bool, 默认: True)
`enable_identity_memory` (bool, 默认: True) | `workspace_id` (str, required)
`query` (str, optional)
`messages` (array, optional) | vector_retrieve_memory
read_history_memory
(内部调用: read_identity_memory, read_meta_memory) |
-
-### 4.4 Runtime层设计（内部实现）
-
-Runtime层负责协调Tool和Agent的调用流程，实现记忆的渐进式处理。
-
-#### 4.4.1 渐进式总结流程（Summary）
-
-总结流程采用分层处理策略，首先保存历史对话，读取元信息，然后由主Agent协调多个专用Agent完成分类总结。
-
-**流程结构：**
-
-```python
-# Step 1: Save conversation history
-AddHistoryMemoryOp()
-
-# Step 2: Load meta information (memory types and targets)
-ReadMetaMemoryOp()
-
-# Step 3: Progressive summarization with delegation
-ReMeSummaryAgentV1Op(tools=[
- # Add meta memory entries for new memory types/targets
- AddMetaMemoryOp(list(memory_type, memory_target)),
-
- # Add general summary memory as fallback
- AddSummaryMemoryOp(summary_memory),
-
- # Delegate to specialized summary agents
- HandsOffOp(list(memory_type, memory_target), agents=[
- PersonalSummaryAgentV1Op, # Summarize personal memories
- ProceduralSummaryAgentV1Op, # Summarize procedural memories
- ToolSummaryAgentV1Op, # Summarize tool-related memories
- IdentitySummaryAgentV1Op # Update identity memory
- ]),
-])
-
-# Specialized agents and their available tools
-PersonalSummaryAgentV1Op(tools=[AddMemoryOp, UpdateMemoryOp, DeleteMemoryOp, VectorRetrieveMemoryOp])
-ProceduralSummaryAgentV1Op(tools=[AddMemoryOp, UpdateMemoryOp, DeleteMemoryOp, VectorRetrieveMemoryOp])
-ToolSummaryAgentV1Op(tools=[AddMemoryOp, UpdateMemoryOp, VectorRetrieveMemoryOp])
-IdentitySummaryAgentV1Op(tools=[ReadIdentityMemoryOp, UpdateIdentityMemoryOp])
-```
-
-#### 4.4.2 渐进式检索流程（Retrieve）
-
-检索流程采用三层检索策略，类似于技能系统的加载机制，逐层加载和过滤记忆。
-
-**流程结构：**
-
-```python
-# Progressive retrieval with three layers
-ReMeRetrieveAgentV1Op(tools=[
- # Layer 0: Load meta memory (all available memory types and targets)
- ReadMetaMemoryOp(),
- # Output format example:
- # - personal(jinli): Information about Jinli's personal life and preferences
- # - personal(jiaji): Information about Jiaji's background and interests
- # - personal(jinli&jiaji): Shared memories between Jinli and Jiaji
- # - procedural(appworld): Procedural knowledge for AppWorld tasks
- # - procedural(bfcl-v3): Procedural knowledge for BFCL-v3 benchmark
- # - tool(tool_guidelines): Guidelines for tool usage
- # - identity(self): Agent's self-identity information
-
- # Layer 1+2: Vector-based retrieval on structured memories
- VectorRetrieveMemoryOp(list(memory_type, memory_target, query)),
-
- # Layer 3: Load full conversation history for specific memory
- ReadHistoryMemoryOp(ref_memory_id),
-])
-```
-
-**与技能系统的类比：**
-
-```python
-# Skill system hierarchy (for reference)
-load_meta_skills # Load skill metadata
-load_skills # Load skill implementations
-load_reference_skills # Load detailed skill documentation
-execute_shell # Execute actual commands
-```
-
-## 五、扩展设计与实验方向
-
-### 5.1 Summary Memory机制
-
-Summary Memory作为通用维度的记忆类型，提供兜底的原始对话索引能力。
-
-**工作流程示例：**
-
-```txt
-Step 1: Progressive summarization across sessions
- session1: List[Message] -> session2: List[Message] -> session3: List[Message] -> ...
-summary ✓ (always) ✓ (always) ✓ (always)
-personal ✗ ✗ ✓ (when applicable)
-procedural ✗ ✓ (when applicable) ✗
-
-Step 2: Retrieval with fallback strategy
-vector_retrieve_memory(query, memory_type="personal", memory_target="jinli")
- -> Search in memory_type: ["personal", "summary"] # Fallback to summary if personal not found
-```
-
-**设计优势：**
-1. Provides a universal dimension for memory extraction across all memory types
-2. Ensures fallback indexing of original conversations when specific meta memory is not available
-3. Maintains conversation context even when specialized memory extraction fails
-
-### 5.2 Thinking参数实验
-
-探索不同的模型推理能力增强方案，受AgentScope和Claude启发。
-
-#### 5.2.1 Thinking参数设计
-
-```python
-async def record_to_memory(
- self,
- thinking: str,
- content: list[str],
- **kwargs: Any,
-) -> ToolResponse:
- """Use this function to record important information that you may
- need later. The target content should be specific and concise, e.g.
- who, when, where, do what, why, how, etc.
-
- Args:
- thinking (`str`):
- Your thinking and reasoning about what to record
- content (`list[str]`):
- The content to remember, which is a list of strings.
- """
-```
-
-#### 5.2.2 实验对比方案
-
-| 方案类型 | 说明 | 灵感来源 |
-|-------------------------------|--------------------------------------------|----------------|
-| Thinking Model | Native reasoning-capable models (e.g., o1) | OpenAI |
-| Instruct Model | Standard instruction-following models | Baseline |
-| Instruct Model + Thinking Params | Add thinking parameter to tool schema | AgentScope |
-| Instruct Model + Thinking Tool | Dedicated thinking tool for explicit reasoning | Claude |
-
-### 5.3 多项操作模式实验
-
-对比单次调用和批量调用的性能与准确性差异。
-
-**两种模式对比：**
-
-| 模式 | Tool调用方式 | Model调用次数 | 优势 | 劣势 |
-|--------------|----------------------------|---------------|------------------------------|--------------------------|
-| 单项模式 | Single-item per call | Multiple | Fine-grained control | Higher latency, more tokens |
-| 多项模式 | Batch multiple items | Single | Lower latency, fewer tokens | Potential batch errors |
-
-**实验目标：**
-- Evaluate accuracy: single vs. batch operations
-- Measure latency and token efficiency
-- Identify optimal use cases for each mode
-
-### 5.4 多版本与扩展性
-
-支持从基类继承实现自定义Agent，便于团队协作和功能迭代。
-
-**扩展示例：**
-
-```python
-# Version 2 implementations by different team members
-PersonalSummaryAgentV2Op / PersonalRetrieveAgentV2Op # @weikang
-ProceduralSummaryAgentV2Op / ProceduralRetrieveAgentV2Op # @zouyin
-
-# Inherit from BaseMemoryAgentOp
-class PersonalSummaryAgentV2Op(BaseMemoryAgentOp):
- """Enhanced personal memory summarization with improved algorithms"""
- pass
-```
-
-### 5.5 文件系统集成（未来方向）
-
-探索将文件操作能力集成到记忆系统中，支持基于文件的记忆管理。
-
-**挑战与考虑：**
-
-1. **操作适配性**：Current operations (retrieve/add/update/delete) need adaptation for file-based storage
-2. **工具选择**：Consider file operation tools: `grep`, `glob`, `ls`, `read_file`, `write_file`, `edit_file`
-3. **模型能力**：Base models have limited file operation capabilities; `qwen3-code` shows better performance
-
-**潜在架构：**
-
-```python
-# File-based memory operations
-FileMemoryOp(tools=[
- grep, # Search within files
- glob, # File pattern matching
- ls, # List directory contents
- read_file, # Read file contents
- write_file, # Write new memory files
- edit_file, # Update existing memory files
-])
-```
-
-### 5.6 自我修改上下文
-
-支持Agent动态修改自身的上下文状态，实现自适应记忆管理。
-
-**实现方式：**
-
-1. **Summary Agent 主动修改**：
- - `add_meta_memory` directly modifies agent context
- - Updates available memory types and targets during execution
-
-2. **ReMy Agent 被动修改**：
- - Retrieves `identity_memory` at each interaction
- - Dynamically updates self-state based on retrieved identity
- - Enables adaptive behavior based on accumulated identity knowledge
-
-## ReMe V2 开发路线图与实施计划
-
-### 技术改造阶段
-1. **代码整合与兼容**：合并flowllm中reme必要的代码，保留现在server-client的依赖，兼容现在各个仓库的依赖代码
-2. **核心接口重构**：新的ReMe接口设计，支持summary，retrieve，context_offload, context_reload 4个核心接口
-3. **Agentic算法升级**：新的agentic算法方案开发
-
-### 评估验证阶段
-4. **Benchmark测试**
- - halumem
- - locomo
- - longmemevel
- - personal-v2 ?
- - appworld/bfcl-v3
-
-### 发布推广阶段
-5. **技术报告**撰写与发布
-6. **生态更新**：更新各个仓库的依赖代码
- - agentscope
- - agentscope-runtime
- - evotraders
- - alias(tool-memory)
- - agentscope-java
- - AgentEvolver
- - cookbook: reme procedural memory paper
- - tool-memory-upgrade（将要合并）
-
-**里程碑目标**：春节前完成小版本发布
-
----
-
-## ReMe V2 核心竞争优势
-
-### 1. 渐进式 Agentic Memory 架构【核心创新】
-融合了多种记忆的渐进式agentic方案，实现从短期到长期记忆的智能化演进
-
-### 2. 全生命周期记忆管理
-同时支持长期记忆（Long-term Memory）和短期记忆（Working Memory），完整覆盖Agent认知周期
-
-### 3. 模型
-提供开源小模型
-
-### 4. 开发者友好生态
- 1. **简洁接口**：提供简洁的接口设计，全异步接口
- 2. **即开即用**：提供CLI工具，开箱即用的体验
- 3. **生态融合**：提供和AgentScope、LangChain无缝集成的方案
- 4. **高度可扩展**：支持Agentic算法的二次开发与定制
\ No newline at end of file
diff --git a/docs/todo.md b/docs/todo.md
deleted file mode 100644
index 5d73e04d..00000000
--- a/docs/todo.md
+++ /dev/null
@@ -1,3 +0,0 @@
-1. 如何更好的注册class
-2. op的返回，使用return 还是 self.output
-3. 如何把agent的东西放出来
\ No newline at end of file
diff --git a/example.env b/example.env
index 5f39a48d..d1a27415 100644
--- a/example.env
+++ b/example.env
@@ -2,6 +2,4 @@ LLM_API_KEY=sk-xxxx
LLM_BASE_URL=https://xxxx/v1
#EMBEDDING_API_KEY=sk-xxxx
#EMBEDDING_BASE_URL=https://xxxx/v1
-LLM_MODEL_NAME=qwen3.5-plus
-
#TAVILY_API_KEY=xxxx
diff --git a/reme/memory/__init__.py b/reme/memory/__init__.py
index ac825bc6..e209a3fe 100644
--- a/reme/memory/__init__.py
+++ b/reme/memory/__init__.py
@@ -1,11 +1,11 @@
"""memory"""
from . import file_based
-from . import tools
+from . import vector_tools
from . import vector_based
__all__ = [
"file_based",
- "tools",
+ "vector_tools",
"vector_based",
]
diff --git a/reme/memory/file_based/__init__.py b/reme/memory/file_based/__init__.py
index 2e01cc41..766f9b3e 100644
--- a/reme/memory/file_based/__init__.py
+++ b/reme/memory/file_based/__init__.py
@@ -9,18 +9,21 @@ Components:
- Summarizer: Generates memory summaries using LLM
- Compactor: Compacts memory content to reduce token usage
- ToolResultCompactor: Truncates large tool results and saves full content to files
+ - ContextChecker: Checks context size and splits messages for compaction
"""
from .as_msg_handler import AsMsgHandler
-from .reme_in_memory_memory import ReMeInMemoryMemory
from .component.compactor import Compactor
+from .component.context_checker import ContextChecker
from .component.summarizer import Summarizer
from .component.tool_result_compactor import ToolResultCompactor
+from .reme_in_memory_memory import ReMeInMemoryMemory
__all__ = [
"AsMsgHandler",
"ReMeInMemoryMemory",
"Summarizer",
"Compactor",
+ "ContextChecker",
"ToolResultCompactor",
]
diff --git a/reme/memory/file_based/component/context_checker.py b/reme/memory/file_based/component/context_checker.py
new file mode 100644
index 00000000..82bb381e
--- /dev/null
+++ b/reme/memory/file_based/component/context_checker.py
@@ -0,0 +1,97 @@
+"""ContextChecker module for checking context size and splitting messages."""
+
+from agentscope.message import Msg
+from agentscope.token import HuggingFaceTokenCounter
+
+from ..as_msg_handler import AsMsgHandler
+from ....core.op import BaseOp
+from ....core.utils import get_std_logger
+
+logger = get_std_logger()
+
+
+class ContextChecker(BaseOp):
+ """
+ ContextChecker class for checking context size and splitting messages.
+
+ This class analyzes conversation messages to determine if the context
+ exceeds the specified token threshold and splits messages into two groups:
+ those that should be compacted and those to keep in context.
+
+ Attributes:
+ memory_compact_threshold (int): Token count threshold for triggering compaction.
+ memory_compact_reserve (int): Token count to reserve for recent messages.
+ msg_handler (AsMsgHandler): Handler for message processing and token counting.
+ """
+
+ def __init__(
+ self,
+ memory_compact_threshold: int,
+ memory_compact_reserve: int = 10000,
+ token_counter: HuggingFaceTokenCounter | None = None,
+ **kwargs,
+ ):
+ """
+ Initialize the ContextChecker.
+
+ Args:
+ memory_compact_threshold (int): Token count threshold for triggering
+ compaction. Messages exceeding this threshold will be split.
+ memory_compact_reserve (int): Token count to reserve for recent messages
+ to keep in context. Defaults to 10000 tokens.
+ token_counter (HuggingFaceTokenCounter | None): Token counter for
+ measuring content length. If None, a default counter will be used.
+ **kwargs: Additional keyword arguments passed to BaseOp.
+ """
+ super().__init__(**kwargs)
+ self.memory_compact_threshold: int = memory_compact_threshold
+ self.memory_compact_reserve: int = memory_compact_reserve
+ assert self.memory_compact_threshold > self.memory_compact_reserve
+
+ self.msg_handler = AsMsgHandler(token_counter=token_counter)
+
+ async def execute(self) -> tuple[list[Msg], list[Msg], bool]:
+ """
+ Execute context check and split messages.
+
+ Retrieves messages from context and checks if they exceed the token
+ threshold. If so, splits them into messages to compact and messages
+ to keep.
+
+ Context Parameters:
+ messages (list[Msg]): List of conversation messages to check.
+ Retrieved from self.context.get("messages", []).
+
+ Returns:
+ tuple[list[Msg], list[Msg], bool]: A tuple containing:
+ - messages_to_compact (list[Msg]): Older messages that should
+ be compacted/summarized.
+ - messages_to_keep (list[Msg]): Recent messages to keep in context.
+ - is_valid (bool): True if the split is valid (tool calls aligned),
+ False if splitting would break conversation integrity.
+
+ Note:
+ - Returns ([], messages, True) if no compaction is needed.
+ - Ensures conversation pairs (user-assistant) are not split.
+ - is_valid=False indicates tool_use and tool_result are misaligned.
+ """
+ messages: list[Msg] = self.context.get("messages", [])
+
+ if not messages:
+ logger.info("ContextChecker: No messages to check.")
+ return [], [], True
+
+ messages_to_compact, messages_to_keep, is_valid = self.msg_handler.context_check(
+ messages=messages,
+ memory_compact_threshold=self.memory_compact_threshold,
+ memory_compact_reserve=self.memory_compact_reserve,
+ )
+
+ logger.info(
+ f"ContextChecker Result: "
+ f"to_compact={len(messages_to_compact)}, "
+ f"to_keep={len(messages_to_keep)}, "
+ f"is_valid={is_valid}",
+ )
+
+ return messages_to_compact, messages_to_keep, is_valid
diff --git a/reme/memory/file_based/tools/__init__.py b/reme/memory/file_based/tools/__init__.py
new file mode 100644
index 00000000..0fb0d814
--- /dev/null
+++ b/reme/memory/file_based/tools/__init__.py
@@ -0,0 +1,13 @@
+"""File-based memory tool implementations."""
+
+from .file_io import FileIO
+from .memory_get import MemoryGet
+from .memory_search import MemorySearch
+from .shell import Shell
+
+__all__ = [
+ "FileIO",
+ "MemoryGet",
+ "MemorySearch",
+ "Shell",
+]
diff --git a/reme/memory/tools/file/file_io.py b/reme/memory/file_based/tools/file_io.py
similarity index 77%
rename from reme/memory/tools/file/file_io.py
rename to reme/memory/file_based/tools/file_io.py
index 161c3210..58e9a5f7 100644
--- a/reme/memory/tools/file/file_io.py
+++ b/reme/memory/file_based/tools/file_io.py
@@ -7,6 +7,8 @@ from typing import Optional
from agentscope.message import TextBlock
from agentscope.tool import ToolResponse
+from .utils import DEFAULT_MAX_BYTES, read_file_safe, truncate_output
+
class FileIO:
"""File I/O operations with a configurable working directory."""
@@ -78,57 +80,64 @@ class FileIO:
)
try:
- with open(file_path, "r", encoding="utf-8") as f:
- all_lines = f.readlines()
+ content = read_file_safe(file_path)
+ all_lines = content.split("\n")
+ total = len(all_lines)
- range_requested = start_line is not None or end_line is not None
+ # Determine read range
+ s = max(1, start_line if start_line is not None else 1)
+ e = min(total, end_line if end_line is not None else total)
- if range_requested:
- total = len(all_lines)
- s = max(1, start_line if start_line is not None else 1)
- e = min(total, end_line if end_line is not None else total)
-
- if s > total:
- return ToolResponse(
- content=[
- TextBlock(
- type="text",
- text=(f"Error: start_line {s} exceeds file length " f"({total} lines) in {file_path}."),
- ),
- ],
- )
-
- if s > e:
- return ToolResponse(
- content=[
- TextBlock(
- type="text",
- text=(f"Error: start_line ({s}) is greater than " f"end_line ({e}) in {file_path}."),
- ),
- ],
- )
-
- selected = all_lines[s - 1 : e]
- content = "".join(selected)
- header = f"{file_path} (lines {s}-{e} of {total})\n"
+ if s > total:
return ToolResponse(
content=[
TextBlock(
type="text",
- text=header + content,
+ text=f"Error: start_line {s} exceeds file length ({total} lines).",
),
],
)
+
+ if s > e:
+ return ToolResponse(
+ content=[
+ TextBlock(
+ type="text",
+ text=f"Error: start_line ({s}) > end_line ({e}).",
+ ),
+ ],
+ )
+
+ # Extract selected lines
+ selected_content = "\n".join(all_lines[s - 1 : e])
+
+ # Apply smart truncation (keep head for file reading)
+ truncated, was_truncated, output_lines, reason = truncate_output(selected_content, keep="head")
+
+ # Build response with truncation hints
+ if was_truncated:
+ end_display = s + output_lines - 1
+ next_line = end_display + 1
+ if reason == "lines":
+ hint = f"\n\n[Lines {s}-{end_display} of {total}. Use start_line={next_line} to continue.]"
+ else:
+ hint = (
+ f"\n\n[Lines {s}-{end_display} of {total} ({DEFAULT_MAX_BYTES // 1024}KB limit). "
+ f"Use start_line={next_line} to continue.]"
+ )
+ text = truncated + hint
+ elif e < total:
+ remaining = total - e
+ text = (
+ f"{file_path} (lines {s}-{e} of {total})\n{truncated}\n\n[{remaining} more lines. "
+ f"Use start_line={e + 1} to continue.]"
+ )
else:
- content = "".join(all_lines)
- return ToolResponse(
- content=[
- TextBlock(
- type="text",
- text=content,
- ),
- ],
- )
+ text = truncated
+
+ return ToolResponse(
+ content=[TextBlock(type="text", text=text)],
+ )
except Exception as e:
return ToolResponse(
diff --git a/reme/memory/tools/chunk/memory_get.py b/reme/memory/file_based/tools/memory_get.py
similarity index 100%
rename from reme/memory/tools/chunk/memory_get.py
rename to reme/memory/file_based/tools/memory_get.py
diff --git a/reme/memory/tools/chunk/memory_search.py b/reme/memory/file_based/tools/memory_search.py
similarity index 100%
rename from reme/memory/tools/chunk/memory_search.py
rename to reme/memory/file_based/tools/memory_search.py
diff --git a/reme/memory/file_based/tools/shell.py b/reme/memory/file_based/tools/shell.py
new file mode 100644
index 00000000..c2714b87
--- /dev/null
+++ b/reme/memory/file_based/tools/shell.py
@@ -0,0 +1,229 @@
+# -*- coding: utf-8 -*-
+# flake8: noqa: E501
+# pylint: disable=line-too-long
+"""The shell command tool."""
+
+import asyncio
+import locale
+import subprocess
+import sys
+from pathlib import Path
+
+from agentscope.message import TextBlock
+from agentscope.tool import ToolResponse
+
+from .utils import truncate_shell_output
+
+
+def _execute_subprocess_sync(
+ cmd: str,
+ cwd: str,
+ timeout: int,
+) -> tuple[int, str, str]:
+ """Execute subprocess synchronously in a thread.
+
+ This function runs in a separate thread to avoid Windows asyncio
+ subprocess limitations.
+
+ Args:
+ cmd (`str`):
+ The shell command to execute.
+ cwd (`str`):
+ The working directory for the command execution.
+ timeout (`int`):
+ The maximum time (in seconds) allowed for the command to run.
+
+ Returns:
+ `tuple[int, str, str]`:
+ A tuple containing the return code, standard output, and
+ standard error of the executed command. If timeout occurs, the
+ return code will be -1 and stderr will contain timeout information.
+ """
+ try:
+ result = subprocess.run(
+ cmd,
+ shell=True,
+ capture_output=True,
+ text=True,
+ cwd=cwd,
+ timeout=timeout,
+ encoding=locale.getpreferredencoding(False) or "utf-8",
+ errors="replace",
+ check=True,
+ )
+ return (
+ result.returncode,
+ result.stdout.strip("\n"),
+ result.stderr.strip("\n"),
+ )
+ except subprocess.TimeoutExpired:
+ return (
+ -1,
+ "",
+ f"Command execution exceeded the timeout of {timeout} seconds.",
+ )
+ except Exception as e:
+ return -1, "", str(e)
+
+
+class Shell:
+ """Shell command execution with a configurable working directory."""
+
+ def __init__(self, working_dir: str | Path):
+ """Initialize Shell with a working directory.
+
+ Args:
+ working_dir (`str | Path`):
+ The working directory for command execution.
+ """
+ self.working_dir = Path(working_dir)
+
+ # pylint: disable=too-many-branches, too-many-statements
+ async def execute_shell_command(
+ self,
+ command: str,
+ timeout: int = 60,
+ ) -> ToolResponse:
+ """Execute given command and return the return code, standard output and
+ error within , and
+ tags.
+
+ Args:
+ command (`str`):
+ The shell command to execute.
+ timeout (`int`, defaults to `60`):
+ The maximum time (in seconds) allowed for the command to run.
+ Default is 60 seconds.
+
+ Returns:
+ `ToolResponse`:
+ The tool response containing the return code, standard output, and
+ standard error of the executed command. If timeout occurs, the
+ return code will be -1 and stderr will contain timeout information.
+ """
+
+ cmd = (command or "").strip()
+
+ # Set working directory
+ working_dir = self.working_dir
+
+ try:
+ if sys.platform == "win32":
+ # Windows: use thread pool to avoid asyncio subprocess limitations
+ returncode, stdout_str, stderr_str = await asyncio.to_thread(
+ _execute_subprocess_sync,
+ cmd,
+ str(working_dir),
+ timeout,
+ )
+ else:
+ proc = await asyncio.create_subprocess_shell(
+ cmd,
+ stdout=asyncio.subprocess.PIPE,
+ stderr=asyncio.subprocess.PIPE,
+ bufsize=0,
+ cwd=str(working_dir),
+ )
+
+ try:
+ # Apply timeout to communicate directly; wait()+communicate()
+ # can hang if descendants keep stdout/stderr pipes open.
+ stdout, stderr = await asyncio.wait_for(
+ proc.communicate(),
+ timeout=timeout,
+ )
+ encoding = locale.getpreferredencoding(False) or "utf-8"
+ stdout_str = stdout.decode(encoding, errors="replace").strip(
+ "\n",
+ )
+ stderr_str = stderr.decode(encoding, errors="replace").strip(
+ "\n",
+ )
+ returncode = proc.returncode
+
+ except asyncio.TimeoutError:
+ # Handle timeout
+ stderr_suffix = (
+ f"⚠️ TimeoutError: The command execution exceeded "
+ f"the timeout of {timeout} seconds. "
+ f"Please consider increasing the timeout value if this command "
+ f"requires more time to complete."
+ )
+ returncode = -1
+ try:
+ proc.terminate()
+ # Wait a bit for graceful termination
+ try:
+ await asyncio.wait_for(proc.wait(), timeout=1)
+ except asyncio.TimeoutError:
+ # Force kill if graceful termination fails
+ proc.kill()
+ await proc.wait()
+
+ # Avoid hanging forever while draining pipes after timeout.
+ try:
+ stdout, stderr = await asyncio.wait_for(
+ proc.communicate(),
+ timeout=1,
+ )
+ except asyncio.TimeoutError:
+ stdout, stderr = b"", b""
+ encoding = locale.getpreferredencoding(False) or "utf-8"
+ stdout_str = stdout.decode(
+ encoding,
+ errors="replace",
+ ).strip(
+ "\n",
+ )
+ stderr_str = stderr.decode(
+ encoding,
+ errors="replace",
+ ).strip(
+ "\n",
+ )
+ if stderr_str:
+ stderr_str += f"\n{stderr_suffix}"
+ else:
+ stderr_str = stderr_suffix
+ except ProcessLookupError:
+ stdout_str = ""
+ stderr_str = stderr_suffix
+
+ # Apply output truncation
+ stdout_str = truncate_shell_output(stdout_str)
+ stderr_str = truncate_shell_output(stderr_str)
+
+ # Format the response in a human-friendly way
+ if returncode == 0:
+ # Success case: just show the output
+ if stdout_str:
+ response_text = stdout_str
+ else:
+ response_text = "Command executed successfully (no output)."
+ else:
+ # Error case: show detailed information
+ response_parts = [f"Command failed with exit code {returncode}."]
+ if stdout_str:
+ response_parts.append(f"\n[stdout]\n{stdout_str}")
+ if stderr_str:
+ response_parts.append(f"\n[stderr]\n{stderr_str}")
+ response_text = "".join(response_parts)
+
+ return ToolResponse(
+ content=[
+ TextBlock(
+ type="text",
+ text=response_text,
+ ),
+ ],
+ )
+
+ except Exception as e:
+ return ToolResponse(
+ content=[
+ TextBlock(
+ type="text",
+ text=f"Error: Shell command execution failed due to \n{e}",
+ ),
+ ],
+ )
diff --git a/reme/memory/file_based/tools/utils.py b/reme/memory/file_based/tools/utils.py
new file mode 100644
index 00000000..17f58877
--- /dev/null
+++ b/reme/memory/file_based/tools/utils.py
@@ -0,0 +1,112 @@
+"""Shared utilities for file and shell tools."""
+
+# Default truncation limits
+DEFAULT_MAX_LINES = 1000
+DEFAULT_MAX_BYTES = 30 * 1024 # 30KB
+
+
+def truncate_output(
+ text: str,
+ max_lines: int = DEFAULT_MAX_LINES,
+ max_bytes: int = DEFAULT_MAX_BYTES,
+ keep: str = "head",
+) -> tuple[str, bool, int, str]:
+ """Smart truncation for large content.
+
+ Args:
+ text: Text content to truncate.
+ max_lines: Maximum number of lines.
+ max_bytes: Maximum size in bytes.
+ keep: Which part to keep - "head" (first lines) or "tail" (last lines).
+
+ Returns:
+ (truncated_content, was_truncated, output_line_count, truncate_reason)
+ """
+ if not text:
+ return text, False, 0, ""
+
+ lines = text.split("\n")
+ total_lines = len(lines)
+
+ # No truncation needed
+ if total_lines <= max_lines and len(text.encode("utf-8")) <= max_bytes:
+ return text, False, total_lines, ""
+
+ # Apply line limit
+ if total_lines > max_lines:
+ if keep == "tail":
+ lines = lines[-max_lines:]
+ else:
+ lines = lines[:max_lines]
+ reason = "lines"
+ else:
+ reason = ""
+
+ # Apply byte limit
+ if len("\n".join(lines).encode("utf-8")) > max_bytes:
+ if keep == "tail":
+ while lines and len("\n".join(lines).encode("utf-8")) > max_bytes:
+ lines.pop(0)
+ else:
+ truncated = []
+ current_bytes = 0
+ for line in lines:
+ line_bytes = len(line.encode("utf-8")) + 1
+ if current_bytes + line_bytes > max_bytes:
+ break
+ truncated.append(line)
+ current_bytes += line_bytes
+ lines = truncated
+ reason = "bytes"
+
+ return "\n".join(lines), True, len(lines), reason
+
+
+def truncate_shell_output(text: str) -> str:
+ """Truncate shell output to last N lines or M bytes, with truncation notice.
+
+ Args:
+ text: The output text to truncate.
+
+ Returns:
+ Truncated text with notice if truncated.
+ """
+ if not text:
+ return text
+
+ try:
+ total_lines = len(text.split("\n"))
+ truncated, was_truncated, output_lines, reason = truncate_output(text, keep="tail")
+
+ if not was_truncated:
+ return text
+
+ start_line = total_lines - output_lines + 1
+ if reason == "lines":
+ notice = f"\n\n[Output truncated: showing lines {start_line}-{total_lines} of {total_lines} total]"
+ else:
+ notice = (
+ f"\n\n[Output truncated: showing lines {start_line}-{total_lines} of {total_lines} "
+ f"({DEFAULT_MAX_BYTES // 1024}KB limit)]"
+ )
+
+ return truncated + notice
+ except Exception:
+ return text
+
+
+def read_file_safe(file_path: str) -> str:
+ """Read file with Unicode error handling.
+
+ Args:
+ file_path: Path to the file.
+
+ Returns:
+ File content as string.
+ """
+ try:
+ with open(file_path, "r", encoding="utf-8") as f:
+ return f.read()
+ except UnicodeDecodeError:
+ with open(file_path, "r", encoding="utf-8", errors="ignore") as f:
+ return f.read()
diff --git a/reme/memory/tools/file/__init__.py b/reme/memory/tools/file/__init__.py
deleted file mode 100644
index 8234e60d..00000000
--- a/reme/memory/tools/file/__init__.py
+++ /dev/null
@@ -1,7 +0,0 @@
-"""File-based memory tool implementations."""
-
-from .file_io import FileIO
-
-__all__ = [
- "FileIO",
-]
diff --git a/reme/memory/tools/record/__init__.py b/reme/memory/tools/record/__init__.py
deleted file mode 100644
index e69de29b..00000000
diff --git a/reme/memory/tools/__init__.py b/reme/memory/vector_tools/__init__.py
similarity index 92%
rename from reme/memory/tools/__init__.py
rename to reme/memory/vector_tools/__init__.py
index af9b851f..fd5e721f 100644
--- a/reme/memory/tools/__init__.py
+++ b/reme/memory/vector_tools/__init__.py
@@ -3,8 +3,6 @@
from .base_memory_tool import BaseMemoryTool
# chunk tools
-from .chunk.memory_get import MemoryGet
-from .chunk.memory_search import MemorySearch
from .delegate_task import DelegateTask
# history tools
@@ -36,9 +34,6 @@ __all__ = [
# base
"BaseMemoryTool",
"DelegateTask",
- # chunk tools
- "MemoryGet",
- "MemorySearch",
# history tools
"AddHistory",
"ReadHistory",
diff --git a/reme/memory/tools/base_memory_tool.py b/reme/memory/vector_tools/base_memory_tool.py
similarity index 100%
rename from reme/memory/tools/base_memory_tool.py
rename to reme/memory/vector_tools/base_memory_tool.py
diff --git a/reme/memory/tools/delegate_task.py b/reme/memory/vector_tools/delegate_task.py
similarity index 100%
rename from reme/memory/tools/delegate_task.py
rename to reme/memory/vector_tools/delegate_task.py
diff --git a/cookbook/__init__.py b/reme/memory/vector_tools/history/__init__.py
similarity index 100%
rename from cookbook/__init__.py
rename to reme/memory/vector_tools/history/__init__.py
diff --git a/reme/memory/tools/history/add_history.py b/reme/memory/vector_tools/history/add_history.py
similarity index 100%
rename from reme/memory/tools/history/add_history.py
rename to reme/memory/vector_tools/history/add_history.py
diff --git a/reme/memory/tools/history/read_history.py b/reme/memory/vector_tools/history/read_history.py
similarity index 100%
rename from reme/memory/tools/history/read_history.py
rename to reme/memory/vector_tools/history/read_history.py
diff --git a/reme/memory/tools/history/read_history_v2.py b/reme/memory/vector_tools/history/read_history_v2.py
similarity index 100%
rename from reme/memory/tools/history/read_history_v2.py
rename to reme/memory/vector_tools/history/read_history_v2.py
diff --git a/cookbook/appworld/__init__.py b/reme/memory/vector_tools/profiles/__init__.py
similarity index 100%
rename from cookbook/appworld/__init__.py
rename to reme/memory/vector_tools/profiles/__init__.py
diff --git a/reme/memory/tools/profiles/add_draft_and_read_all_profiles.py b/reme/memory/vector_tools/profiles/add_draft_and_read_all_profiles.py
similarity index 100%
rename from reme/memory/tools/profiles/add_draft_and_read_all_profiles.py
rename to reme/memory/vector_tools/profiles/add_draft_and_read_all_profiles.py
diff --git a/reme/memory/tools/profiles/add_profile.py b/reme/memory/vector_tools/profiles/add_profile.py
similarity index 100%
rename from reme/memory/tools/profiles/add_profile.py
rename to reme/memory/vector_tools/profiles/add_profile.py
diff --git a/reme/memory/tools/profiles/delete_profile.py b/reme/memory/vector_tools/profiles/delete_profile.py
similarity index 100%
rename from reme/memory/tools/profiles/delete_profile.py
rename to reme/memory/vector_tools/profiles/delete_profile.py
diff --git a/reme/memory/tools/profiles/profile_handler.py b/reme/memory/vector_tools/profiles/profile_handler.py
similarity index 100%
rename from reme/memory/tools/profiles/profile_handler.py
rename to reme/memory/vector_tools/profiles/profile_handler.py
diff --git a/reme/memory/tools/profiles/read_all_profiles.py b/reme/memory/vector_tools/profiles/read_all_profiles.py
similarity index 100%
rename from reme/memory/tools/profiles/read_all_profiles.py
rename to reme/memory/vector_tools/profiles/read_all_profiles.py
diff --git a/reme/memory/tools/profiles/update_profile.py b/reme/memory/vector_tools/profiles/update_profile.py
similarity index 100%
rename from reme/memory/tools/profiles/update_profile.py
rename to reme/memory/vector_tools/profiles/update_profile.py
diff --git a/reme/memory/tools/profiles/update_profiles_v1.py b/reme/memory/vector_tools/profiles/update_profiles_v1.py
similarity index 100%
rename from reme/memory/tools/profiles/update_profiles_v1.py
rename to reme/memory/vector_tools/profiles/update_profiles_v1.py
diff --git a/cookbook/bfcl/__init__.py b/reme/memory/vector_tools/record/__init__.py
similarity index 100%
rename from cookbook/bfcl/__init__.py
rename to reme/memory/vector_tools/record/__init__.py
diff --git a/reme/memory/tools/record/add_and_retrieve_similar_memory.py b/reme/memory/vector_tools/record/add_and_retrieve_similar_memory.py
similarity index 100%
rename from reme/memory/tools/record/add_and_retrieve_similar_memory.py
rename to reme/memory/vector_tools/record/add_and_retrieve_similar_memory.py
diff --git a/reme/memory/tools/record/add_draft_and_retrieve_similar_memory.py b/reme/memory/vector_tools/record/add_draft_and_retrieve_similar_memory.py
similarity index 100%
rename from reme/memory/tools/record/add_draft_and_retrieve_similar_memory.py
rename to reme/memory/vector_tools/record/add_draft_and_retrieve_similar_memory.py
diff --git a/reme/memory/tools/record/add_memory.py b/reme/memory/vector_tools/record/add_memory.py
similarity index 100%
rename from reme/memory/tools/record/add_memory.py
rename to reme/memory/vector_tools/record/add_memory.py
diff --git a/reme/memory/tools/record/delete_memory.py b/reme/memory/vector_tools/record/delete_memory.py
similarity index 100%
rename from reme/memory/tools/record/delete_memory.py
rename to reme/memory/vector_tools/record/delete_memory.py
diff --git a/reme/memory/tools/record/memory_handler.py b/reme/memory/vector_tools/record/memory_handler.py
similarity index 100%
rename from reme/memory/tools/record/memory_handler.py
rename to reme/memory/vector_tools/record/memory_handler.py
diff --git a/reme/memory/tools/record/retrieve_memory.py b/reme/memory/vector_tools/record/retrieve_memory.py
similarity index 100%
rename from reme/memory/tools/record/retrieve_memory.py
rename to reme/memory/vector_tools/record/retrieve_memory.py
diff --git a/reme/memory/tools/record/retrieve_recent_memory.py b/reme/memory/vector_tools/record/retrieve_recent_memory.py
similarity index 100%
rename from reme/memory/tools/record/retrieve_recent_memory.py
rename to reme/memory/vector_tools/record/retrieve_recent_memory.py
diff --git a/reme/memory/tools/record/update_memory.py b/reme/memory/vector_tools/record/update_memory.py
similarity index 100%
rename from reme/memory/tools/record/update_memory.py
rename to reme/memory/vector_tools/record/update_memory.py
diff --git a/reme/memory/tools/record/update_memory_v1.py b/reme/memory/vector_tools/record/update_memory_v1.py
similarity index 100%
rename from reme/memory/tools/record/update_memory_v1.py
rename to reme/memory/vector_tools/record/update_memory_v1.py
diff --git a/reme/memory/tools/record/update_memory_v2.py b/reme/memory/vector_tools/record/update_memory_v2.py
similarity index 100%
rename from reme/memory/tools/record/update_memory_v2.py
rename to reme/memory/vector_tools/record/update_memory_v2.py
diff --git a/reme/reme.py b/reme/reme.py
index 6c5525ee..685cdcdd 100644
--- a/reme/reme.py
+++ b/reme/reme.py
@@ -7,7 +7,7 @@ from .config import ReMeConfigParser
from .core import Application
from .core.enumeration import MemoryType, Role
from .core.schema import Message, MemoryNode
-from .memory.tools import (
+from .memory.vector_tools import (
AddDraftAndRetrieveSimilarMemory,
AddHistory,
AddMemory,
@@ -17,8 +17,8 @@ from .memory.tools import (
RetrieveMemory,
UpdateProfilesV1,
)
-from .memory.tools.profiles.profile_handler import ProfileHandler
-from .memory.tools.record.memory_handler import MemoryHandler
+from .memory.vector_tools.profiles.profile_handler import ProfileHandler
+from .memory.vector_tools.record.memory_handler import MemoryHandler
from .memory.vector_based import (
BaseMemoryAgent,
PersonalRetriever,
@@ -185,7 +185,7 @@ class ReMe(Application):
format_messages.append(message)
if version == "default":
- personal_summarizer_tools = [
+ personal_summarizer_tools: list = [
AddDraftAndRetrieveSimilarMemory(
enable_thinking_params=enable_thinking_params,
enable_memory_target=False,
diff --git a/reme/reme_light.py b/reme/reme_light.py
index 5266ae76..922b846b 100644
--- a/reme/reme_light.py
+++ b/reme/reme_light.py
@@ -26,15 +26,45 @@ from agentscope.tool import Toolkit, ToolResponse
from .config import ReMeConfigParser
from .core import Application
from .core.utils import get_hf_token_counter, get_std_logger
-from .memory.file_based import Compactor, Summarizer, ToolResultCompactor, ReMeInMemoryMemory, AsMsgHandler
-from .memory.tools import MemorySearch
-from .memory.tools.file import FileIO
+from .memory.file_based import (
+ Compactor,
+ ContextChecker,
+ Summarizer,
+ ToolResultCompactor,
+ ReMeInMemoryMemory,
+ AsMsgHandler,
+)
+from .memory.file_based import MemorySearch
+from .memory.file_based.tools import FileIO
logger = get_std_logger()
class ReMeLight(Application):
- """ReMe Light Application Class"""
+ """
+ ReMe Light Application Class.
+
+ A lightweight memory-enabled application that provides semantic search,
+ memory compaction, summarization, and tool result management capabilities.
+ Built on top of the core Application framework with integrated vector store
+ and file-based memory management.
+
+ This class is designed for applications requiring:
+ - Long conversation memory management with automatic compaction
+ - Semantic search over stored memories using hybrid vector/text search
+ - Background summarization of conversation history
+ - Automatic cleanup of expired tool results
+
+ Attributes:
+ working_path (Path): Absolute path to the working directory.
+ memory_path (Path): Path to the memory storage directory.
+ tool_result_path (Path): Path to the tool result storage directory.
+ vector_weight (float): Weight for vector search in hybrid search (0-1).
+ candidate_multiplier (float): Multiplier for candidate retrieval count.
+ tool_result_threshold (int): Character threshold for tool result compaction.
+ retention_days (int): Number of days to retain tool result files.
+ summary_tasks (list[asyncio.Task]): List of active background summary tasks.
+ """
def __init__(
self,
@@ -51,6 +81,47 @@ class ReMeLight(Application):
tool_result_threshold: int = 1000,
retention_days: int = 7,
):
+ """
+ Initialize the ReMeLight application.
+
+ Sets up the working directory structure, configures API connections,
+ and initializes memory management components.
+
+ Args:
+ working_dir (str): Base directory for all application data storage.
+ Defaults to ".reme". Will be created if it doesn't exist.
+ llm_api_key (str | None): API key for the language model service.
+ If None, will attempt to use environment variables.
+ llm_base_url (str | None): Base URL for the language model API endpoint.
+ If None, will use the default endpoint.
+ embedding_api_key (str | None): API key for the embedding model service.
+ If None, will attempt to use environment variables.
+ embedding_base_url (str | None): Base URL for the embedding API endpoint.
+ If None, will use the default endpoint.
+ default_as_llm_config (dict | None): Default configuration dictionary
+ for AgentScope language model. Overrides default settings.
+ default_embedding_model_config (dict | None): Default configuration
+ dictionary for the embedding model.
+ default_file_store_config (dict | None): Default configuration
+ dictionary for the file storage backend.
+ vector_weight (float): Weight assigned to vector similarity search
+ in hybrid search operations. Range [0.0, 1.0], default 0.7.
+ Higher values prioritize semantic similarity over keyword matching.
+ candidate_multiplier (float): Multiplier applied to max_results when
+ retrieving candidates for re-ranking. Default 3.0 means 3x more
+ candidates are retrieved than the final result count.
+ tool_result_threshold (int): Character count threshold for tool result
+ compaction. Results exceeding this length will be truncated and
+ saved to files. Default 1000 characters.
+ retention_days (int): Number of days to retain tool result files
+ before automatic cleanup. Default 7 days.
+
+ Note:
+ The following directory structure will be created:
+ - {working_dir}/ - Root working directory
+ - {working_dir}/memory/ - Memory storage files
+ - {working_dir}/tool_result/ - Compacted tool result files
+ """
# Initialize working directory structure
self.working_path = Path(working_dir).absolute()
self.working_path.mkdir(parents=True, exist_ok=True)
@@ -129,18 +200,64 @@ class ReMeLight(Application):
return 0
async def start(self):
- """Start the application lifecycle."""
+ """
+ Start the application lifecycle.
+
+ Initializes all application components by calling the parent class start
+ method, then performs initial cleanup of expired tool result files.
+
+ Returns:
+ The result from the parent Application.start() method.
+
+ Note:
+ This method should be called before using any other application
+ functionality. It ensures all services are properly initialized.
+ """
result = await super().start()
+ # Perform initial cleanup of any expired tool result files
self._cleanup_tool_results()
return result
async def close(self) -> bool:
- """Close the application and perform cleanup."""
+ """
+ Close the application and perform cleanup.
+
+ Performs final cleanup of expired tool result files and then shuts down
+ all application components by calling the parent class close method.
+
+ Returns:
+ bool: True if the application was closed successfully, False otherwise.
+
+ Note:
+ This method should be called when the application is no longer needed
+ to ensure proper resource cleanup and data persistence.
+ """
+ # Final cleanup of expired tool result files before shutdown
self._cleanup_tool_results()
return await super().close()
async def compact_tool_result(self, messages: list[Msg]) -> list[Msg]:
- """Compact tool results by truncating large outputs and saving full content to files."""
+ """
+ Compact tool results by truncating large outputs and saving full content to files.
+
+ This method processes a list of messages containing tool results and compacts
+ any that exceed the configured threshold. Large tool outputs are truncated
+ in the message while the full content is saved to separate files for later
+ retrieval if needed.
+
+ Args:
+ messages (list[Msg]): List of messages potentially containing tool results
+ that may need compaction.
+
+ Returns:
+ list[Msg]: The processed list of messages with large tool results compacted.
+ If an error occurs, returns the original unmodified messages.
+
+ Note:
+ - Tool results shorter than tool_result_threshold are left unchanged
+ - Full content of truncated results is saved to tool_result_path
+ - Expired files are automatically cleaned up during this operation
+ """
try:
# Create compactor with instance configuration
compactor = ToolResultCompactor(
@@ -162,6 +279,61 @@ class ReMeLight(Application):
logger.exception(f"Error compacting tool results: {e}")
return messages
+ async def check_context(
+ self,
+ messages: list[Msg],
+ memory_compact_threshold: int,
+ memory_compact_reserve: int = 10000,
+ token_counter: HuggingFaceTokenCounter | None = None,
+ ) -> tuple[list[Msg], list[Msg], bool]:
+ """
+ Check context size and determine if compaction is needed.
+
+ Analyzes the provided messages to determine if they exceed the configured
+ token threshold and splits them into two groups: messages that should be
+ compacted and messages to keep in context.
+
+ Args:
+ messages (list[Msg]): List of messages to check for context overflow.
+ memory_compact_threshold (int): Token count threshold for triggering
+ compaction. Messages exceeding this threshold will be split.
+ memory_compact_reserve (int): Token count to reserve for recent messages
+ to keep in context. Defaults to 10000 tokens.
+ token_counter (HuggingFaceTokenCounter | None): Token counter for
+ measuring message length. If None, uses default HuggingFace counter.
+
+ Returns:
+ tuple[list[Msg], list[Msg], bool]: A tuple containing:
+ - messages_to_compact (list[Msg]): Older messages that should
+ be compacted/summarized.
+ - messages_to_keep (list[Msg]): Recent messages to keep in context.
+ - is_valid (bool): True if the split is valid (tool calls aligned),
+ False if splitting would break conversation integrity.
+
+ Note:
+ - Returns ([], messages, True) if no compaction is needed.
+ - Ensures conversation pairs (user-assistant) are not split.
+ - is_valid=False indicates tool_use and tool_result are misaligned.
+ """
+ try:
+ if token_counter is None:
+ token_counter = get_hf_token_counter()
+
+ checker = ContextChecker(
+ memory_compact_threshold=memory_compact_threshold,
+ memory_compact_reserve=memory_compact_reserve,
+ token_counter=token_counter,
+ )
+
+ return await checker.call(
+ messages=messages,
+ service_context=self.service_context,
+ )
+
+ except Exception as e:
+ logger.exception(f"Error checking context: {e}")
+ return [], messages, False
+
async def compact_memory(
self,
messages: list[Msg],
@@ -173,7 +345,34 @@ class ReMeLight(Application):
compact_ratio: float = 0.7,
previous_summary: str = "",
) -> str:
- """Compact a list of messages into a condensed summary."""
+ """
+ Compact a list of messages into a condensed summary.
+
+ Uses the configured language model to generate a concise summary of the
+ provided messages. This is useful for reducing context window usage while
+ preserving important information from the conversation history.
+
+ Args:
+ messages (list[Msg]): List of messages to be compacted into a summary.
+ as_llm (str | ChatModelBase): Language model identifier or instance
+ to use for summarization. Defaults to "default".
+ as_llm_formatter (str | FormatterBase): Formatter for the language model.
+ Defaults to "default".
+ token_counter (HuggingFaceTokenCounter | None): Token counter for
+ measuring message length. If None, uses default HuggingFace counter.
+ language (str): Language for the summary output. "zh" for Chinese,
+ any other value for English. Defaults to "zh".
+ max_input_length (float): Maximum input length in tokens for the model.
+ Defaults to 128K tokens.
+ compact_ratio (float): Ratio used to calculate compaction threshold.
+ Defaults to 0.7.
+ previous_summary (str): Previous summary to incorporate into the new
+ summary for continuity. Defaults to empty string.
+
+ Returns:
+ str: The condensed summary of the messages, or an empty string if
+ an error occurred during compaction.
+ """
try:
if token_counter is None:
token_counter = get_hf_token_counter()
@@ -208,7 +407,37 @@ class ReMeLight(Application):
max_input_length: float = 128 * 1024,
compact_ratio: float = 0.7,
) -> str:
- """Generate a comprehensive summary of the given messages."""
+ """
+ Generate a comprehensive summary of the given messages.
+
+ Creates a detailed summary of the conversation history and persists it
+ to the memory directory as structured files. Unlike compact_memory, this
+ method produces more detailed summaries suitable for long-term storage.
+
+ Args:
+ messages (list[Msg]): List of messages to summarize.
+ as_llm (str | ChatModelBase): Language model identifier or instance
+ for summarization. Defaults to "default".
+ as_llm_formatter (str | FormatterBase): Formatter for the language model.
+ Defaults to "default".
+ token_counter (HuggingFaceTokenCounter | None): Token counter for
+ measuring message length. If None, uses default HuggingFace counter.
+ toolkit (Toolkit | None): Toolkit with file operations for persisting
+ summaries. If None, creates a default toolkit with read/write/edit.
+ language (str): Language for the summary output. "zh" for Chinese,
+ any other value for English. Defaults to "zh".
+ max_input_length (float): Maximum input length in tokens.
+ Defaults to 128K tokens.
+ compact_ratio (float): Ratio used to calculate compaction threshold.
+ Defaults to 0.7.
+
+ Returns:
+ str: The generated summary text, or an empty string if an error occurred.
+
+ Note:
+ This method may write summary files to the memory_path directory
+ using the provided or default toolkit.
+ """
try:
if token_counter is None:
token_counter = get_hf_token_counter()
@@ -238,7 +467,30 @@ class ReMeLight(Application):
return ""
def add_async_summary_task(self, messages: list[Msg], **kwargs):
- """Add an asynchronous summary task for the given messages."""
+ """
+ Add an asynchronous summary task for the given messages.
+
+ Creates a background task to generate a summary of the provided messages
+ without blocking the main execution flow. Completed tasks are automatically
+ cleaned up from the task list.
+
+ Args:
+ messages (list[Msg]): List of messages to be summarized asynchronously.
+ **kwargs: Additional keyword arguments passed to summary_memory().
+ Supported arguments include:
+ - as_llm: Language model identifier or instance
+ - as_llm_formatter: Formatter for the language model
+ - token_counter: Token counter instance
+ - toolkit: Toolkit for file operations
+ - language: Output language ("zh" or other)
+ - max_input_length: Maximum input token length
+ - compact_ratio: Compaction threshold ratio
+
+ Note:
+ - Completed/failed/cancelled tasks are cleaned up before adding new ones
+ - Task results and errors are logged automatically
+ - Use await_summary_tasks() to wait for all pending tasks to complete
+ """
remaining_tasks = []
for task in self.summary_tasks:
if task.done():
@@ -274,7 +526,49 @@ class ReMeLight(Application):
enable_tool_result_compact: bool = True,
tool_result_compact_keep_n: int = 3,
) -> tuple[list[Msg], str]:
- """Hook called before reasoning."""
+ """
+ Hook called before reasoning to manage memory and context.
+
+ This method is designed to be called before each reasoning step to ensure
+ the conversation context fits within model limits. It performs tool result
+ compaction, checks context size, and triggers memory compaction if needed.
+
+ Args:
+ messages (list[Msg]): Current conversation messages to be processed.
+ system_prompt (str): System prompt that will be included in the context.
+ Used to calculate available space. Defaults to empty string.
+ compressed_summary (str): Existing compressed summary from previous
+ compactions. Defaults to empty string.
+ as_llm (str | ChatModelBase): Language model for compaction operations.
+ Defaults to "default".
+ as_llm_formatter (str | FormatterBase): Formatter for the language model.
+ Defaults to "default".
+ token_counter (HuggingFaceTokenCounter | None): Token counter for
+ measuring content length. If None, uses default counter.
+ toolkit (Toolkit | None): Toolkit for file operations in summarization.
+ Defaults to None.
+ language (str): Language for generated summaries. Defaults to "zh".
+ max_input_length (float): Maximum context window size in tokens.
+ Defaults to 128K tokens.
+ compact_ratio (float): Ratio for calculating compaction threshold.
+ Defaults to 0.7.
+ memory_compact_reserve (int): Token count to reserve for new responses.
+ Defaults to 10000 tokens.
+ enable_tool_result_compact (bool): Whether to compact tool results.
+ Defaults to True.
+ tool_result_compact_keep_n (int): Number of recent messages to exclude
+ from tool result compaction. Defaults to 3.
+
+ Returns:
+ tuple[list[Msg], str]: A tuple containing:
+ - list[Msg]: Messages to keep in context (may be reduced)
+ - str: Updated compressed summary incorporating compacted messages
+
+ Note:
+ - Automatically triggers background summarization for compacted messages
+ - Tool results in recent messages (keep_n) are not compacted
+ - Returns original messages unchanged if no compaction is needed
+ """
if token_counter is None:
token_counter = get_hf_token_counter()
@@ -328,7 +622,25 @@ class ReMeLight(Application):
return messages_to_keep, compressed_summary
async def await_summary_tasks(self) -> str:
- """Wait for all background summary tasks to complete and collect results."""
+ """
+ Wait for all background summary tasks to complete and collect results.
+
+ Blocks until all pending summary tasks in the task list have completed,
+ cancelled, or failed. Collects status information from each task and
+ clears the task list after processing.
+
+ Returns:
+ str: A concatenated string of status messages for all tasks, including:
+ - Completion confirmations with results
+ - Cancellation notices
+ - Error messages for failed tasks
+
+ Note:
+ - This method will block if any tasks are still running
+ - All tasks are removed from summary_tasks after this call
+ - Task exceptions are logged but do not raise to the caller
+ - Use this before application shutdown to ensure all summaries complete
+ """
result = ""
for task in self.summary_tasks:
if task.done():
@@ -369,26 +681,19 @@ class ReMeLight(Application):
async def memory_search(self, query: str, max_results: int = 5, min_score: float = 0.1) -> ToolResponse:
"""
- Perform semantic memory search using vector and full-text search.
-
- This method searches the memory store for content relevant to the given query
- using a hybrid approach combining vector similarity search and full-text search.
- Results are ranked by relevance and filtered by the minimum score threshold.
+ Mandatory recall step: semantically search MEMORY.md + memory/*.md
+ (and optional session transcripts) before answering questions about
+ prior work, decisions, dates, people, preferences, or todos; returns
+ top snippets with path + lines.
Args:
- query (str): The search query string. Must not be empty.
- max_results (int): Maximum number of results to return (1-100, default: 5)
- min_score (float): Minimum relevance score threshold (0.001-0.999, default: 0.1)
+ query (str): The semantic search query to find relevant memory snippets.
+ max_results (int): Maximum number of search results to return (optional), default 5.
+ min_score (float): Minimum similarity score threshold for results (optional), default 0.1.
Returns:
ToolResponse: A ToolResponse containing the search results as text,
- or an error message if the query is empty
-
- Note:
- - Vector search weight is controlled by self.vector_weight
- - Candidate retrieval uses self.candidate_multiplier for broader search
- - Parameters are validated and clamped to valid ranges
- - Requires vector search to be enabled via embedding configuration
+ or an error message if the query is empty.
"""
# Validate query parameter
if not query:
@@ -452,7 +757,25 @@ class ReMeLight(Application):
@staticmethod
def get_in_memory_memory(token_counter: HuggingFaceTokenCounter | None = None):
- """Create and return an in-memory memory instance."""
+ """
+ Create and return an in-memory memory instance.
+
+ Factory method to create a ReMeInMemoryMemory instance configured with
+ the specified token counter. This memory instance stores data in RAM
+ without persistence, suitable for temporary or session-based storage.
+
+ Args:
+ token_counter (HuggingFaceTokenCounter | None): Token counter for
+ measuring content length in the memory. If None, creates a
+ default HuggingFace token counter.
+
+ Returns:
+ ReMeInMemoryMemory: A new in-memory memory instance ready for use.
+
+ Example:
+ >>> memory = ReMeLight.get_in_memory_memory()
+ >>> # Use memory for temporary storage during a session
+ """
if token_counter is None:
token_counter = get_hf_token_counter()
diff --git a/cookbook/frozenlake/__init__.py b/test/cookbook/__init__.py
similarity index 100%
rename from cookbook/frozenlake/__init__.py
rename to test/cookbook/__init__.py
diff --git a/cookbook/simple_demo/__init__.py b/test/cookbook/appworld/__init__.py
similarity index 100%
rename from cookbook/simple_demo/__init__.py
rename to test/cookbook/appworld/__init__.py
diff --git a/cookbook/appworld/appworld_react_agent.py b/test/cookbook/appworld/appworld_react_agent.py
similarity index 87%
rename from cookbook/appworld/appworld_react_agent.py
rename to test/cookbook/appworld/appworld_react_agent.py
index ba194462..ca8ee2f3 100644
--- a/cookbook/appworld/appworld_react_agent.py
+++ b/test/cookbook/appworld/appworld_react_agent.py
@@ -96,10 +96,7 @@ class AppworldReactAgent:
def prompt_messages(self, run_id, task_index, previous_memories: None, world: AppWorld):
app_descriptions = json.dumps(
- [
- {"name": k, "description": v}
- for (k, v) in world.task.app_descriptions.items()
- ],
+ [{"name": k, "description": v} for (k, v) in world.task.app_descriptions.items()],
indent=1,
)
dictionary = {"supervisor": world.task.supervisor, "app_descriptions": app_descriptions}
@@ -112,7 +109,12 @@ class AppworldReactAgent:
self.retrieved_memory_list[run_id][task_index] = response["metadata"]["memory_list"]
task_memory = response["answer"]
logger.info(f"loaded task_memory: {task_memory}")
- query = "Task:\n" + query + "\n\nSome Related Experience to help you to complete the task:\n" + re.sub(r'(?i)\bMemory\s*(\d+)\s*[:]', r'Experience \1:', task_memory)
+ query = (
+ "Task:\n"
+ + query
+ + "\n\nSome Related Experience to help you to complete the task:\n"
+ + re.sub(r"(?i)\bMemory\s*(\d+)\s*[:]", r"Experience \1:", task_memory)
+ )
else:
formatted_memories = []
for i, memory in enumerate(previous_memories, 1):
@@ -120,14 +122,18 @@ class AppworldReactAgent:
memory_content = memory["content"]
memory_text = f"Experience {i}:\n When to use: {condition}\n Content: {memory_content}\n"
formatted_memories.append(memory_text)
- query = "Task:\n" + query + "\n\nSome Related Experience to help you to complete the task:\n" + "\n".join(formatted_memories)
+ query = (
+ "Task:\n"
+ + query
+ + "\n\nSome Related Experience to help you to complete the task:\n"
+ + "\n".join(formatted_memories)
+ )
messages = [
{"role": "system", "content": sys_prompt},
- {"role": "user", "content": query}
+ {"role": "user", "content": query},
]
self.history[run_id][task_index] = messages
-
@staticmethod
def get_reward(world) -> float:
tracker = world.evaluate()
@@ -136,7 +142,9 @@ class AppworldReactAgent:
return num_passes / (num_passes + num_failures)
def extract_code_and_fix_content(
- self, text: str, ignore_multiple_calls=True
+ self,
+ text: str,
+ ignore_multiple_calls=True,
) -> tuple[str, str]:
full_code_regex = r"```python\n(.*?)```"
partial_code_regex = r".*```python\n(.*)"
@@ -154,7 +162,9 @@ class AppworldReactAgent:
match_end = re_match.end()
# check for partial code match at end (no terminating ```) following the last match
partial_match = re.match(
- partial_code_regex, original_text[match_end:], flags=re.DOTALL
+ partial_code_regex,
+ original_text[match_end:],
+ flags=re.DOTALL,
)
if partial_match:
output_code += partial_match.group(1).strip()
@@ -180,7 +190,12 @@ class AppworldReactAgent:
before_score = self.get_reward(world)
for i in range(self.max_interactions):
if i == 0:
- self.prompt_messages(run_id=run_id, task_index=task_index, previous_memories=previous_memories, world=world)
+ self.prompt_messages(
+ run_id=run_id,
+ task_index=task_index,
+ previous_memories=previous_memories,
+ world=world,
+ )
code_msg = self.call_llm(self.history[run_id][task_index])
code, text = self.extract_code_and_fix_content(code_msg)
self.history[run_id][task_index].append({"role": "assistant", "content": code})
@@ -189,7 +204,9 @@ class AppworldReactAgent:
# if len(output) > self.max_response_size:
# # logger.warning(f"output exceed max size={len(output)}")
# output = output[: self.max_response_size]
- self.history[run_id][task_index].append({"role": "user", "content": "Output:\n```\n" + output + "```\n\n"})
+ self.history[run_id][task_index].append(
+ {"role": "user", "content": "Output:\n```\n" + output + "```\n\n"},
+ )
if world.task_completed():
break
@@ -199,7 +216,9 @@ class AppworldReactAgent:
if self.use_memory:
if self.use_memory_addition:
- new_traj_list = [self.get_traj_from_task_history(task_id, self.history[run_id][task_index], after_score)]
+ new_traj_list = [
+ self.get_traj_from_task_history(task_id, self.history[run_id][task_index], after_score),
+ ]
previous_memories = self.add_memory(new_traj_list)
if after_score != 1:
self.delete_memory_by_ids([mem["memory_id"] for mem in previous_memories])
@@ -209,7 +228,7 @@ class AppworldReactAgent:
self.update_memory_information(self.retrieved_memory_list[run_id][task_index], update_utility)
counter += 1
- if self.use_memory_deletion: # and counter % self.delete_freq == 0:
+ if self.use_memory_deletion: # and counter % self.delete_freq == 0:
self.delete_memory()
t_result = {
@@ -261,7 +280,7 @@ class AppworldReactAgent:
return {
"task_id": task_id,
"messages": task_history,
- "score": reward
+ "score": reward,
}
def add_memory(self, trajectories):
@@ -290,8 +309,8 @@ class AppworldReactAgent:
json={
"workspace_id": self.memory_workspace_id,
"action": "delete_ids",
- "memory_ids": memory_ids
- }
+ "memory_ids": memory_ids,
+ },
)
response.raise_for_status()
@@ -318,6 +337,7 @@ class AppworldReactAgent:
)
response.raise_for_status()
+
def main():
dataset_name = "train"
task_ids = load_task_ids(dataset_name)
diff --git a/cookbook/appworld/prompt.py b/test/cookbook/appworld/prompt.py
similarity index 100%
rename from cookbook/appworld/prompt.py
rename to test/cookbook/appworld/prompt.py
diff --git a/cookbook/appworld/requirements.txt b/test/cookbook/appworld/requirements.txt
similarity index 100%
rename from cookbook/appworld/requirements.txt
rename to test/cookbook/appworld/requirements.txt
diff --git a/cookbook/appworld/run_appworld.py b/test/cookbook/appworld/run_appworld.py
similarity index 98%
rename from cookbook/appworld/run_appworld.py
rename to test/cookbook/appworld/run_appworld.py
index 3379a16a..d286c64a 100644
--- a/cookbook/appworld/run_appworld.py
+++ b/test/cookbook/appworld/run_appworld.py
@@ -90,7 +90,7 @@ def run_agent(
utility_threshold: float = 0.5,
workspace_id: str = "appworld_v1",
api_url: str = "http://0.0.0.0:8002/",
- batch_size: int = 4
+ batch_size: int = 4,
):
experiment_name = dataset_name + "_" + experiment_suffix
path: Path = Path(f"./exp_result/{model_name}")
@@ -125,7 +125,7 @@ def run_agent(
future_list: list = []
for i, task_id in enumerate(batch_task_ids):
actor = AppworldReactAgent.remote(
- index=start_idx+i,
+ index=start_idx + i,
model_name=model_name,
task_ids=[task_id],
experiment_name=experiment_name,
@@ -193,9 +193,10 @@ def run_agent(
result.append(task_results)
dump_file()
+
def main():
max_workers = 8
- num_runs = 1 # Number of runs
+ num_runs = 1 # Number of runs
batch_size = 8 # Number of concurrent tasks per batch
num_trials = 2
@@ -206,7 +207,6 @@ def main():
workspace_id = "appworld"
api_url = "http://0.0.0.0:8002/"
-
# Clean up workspace before starting
logger.info("Deleting workspace...")
delete_workspace(workspace_id=workspace_id, api_url=api_url)
@@ -216,7 +216,6 @@ def main():
logger.info("Start load experiments to build task memories")
load_memory(workspace_id=workspace_id, api_url=api_url)
-
for i in range(num_runs):
run_agent(
model_name=model_name,
@@ -232,8 +231,9 @@ def main():
utility_threshold=0.5,
workspace_id=workspace_id,
api_url=api_url,
- batch_size=batch_size
+ batch_size=batch_size,
)
+
if __name__ == "__main__":
main()
diff --git a/cookbook/appworld/run_exp_statistic.py b/test/cookbook/appworld/run_exp_statistic.py
similarity index 100%
rename from cookbook/appworld/run_exp_statistic.py
rename to test/cookbook/appworld/run_exp_statistic.py
diff --git a/cookbook/tool_memory/__init__.py b/test/cookbook/bfcl/__init__.py
similarity index 100%
rename from cookbook/tool_memory/__init__.py
rename to test/cookbook/bfcl/__init__.py
diff --git a/cookbook/bfcl/bfcl_agent.py b/test/cookbook/bfcl/bfcl_agent.py
similarity index 98%
rename from cookbook/bfcl/bfcl_agent.py
rename to test/cookbook/bfcl/bfcl_agent.py
index 64eb6e6b..2c779a6a 100644
--- a/cookbook/bfcl/bfcl_agent.py
+++ b/test/cookbook/bfcl/bfcl_agent.py
@@ -195,7 +195,7 @@ class BFCLAgent:
# Extract memory list from response
memory_list = result.get("metadata", {}).get("memory_list", [])
- logger.info(f'add new memories: {memory_list}')
+ logger.info(f"add new memories: {memory_list}")
return memory_list
def delete_memory_by_ids(self, memory_ids):
@@ -204,8 +204,8 @@ class BFCLAgent:
json={
"workspace_id": self.memory_workspace_id,
"action": "delete_ids",
- "memory_ids": memory_ids
- }
+ "memory_ids": memory_ids,
+ },
)
response.raise_for_status()
@@ -647,7 +647,9 @@ class BFCLAgent:
reward = self.get_reward(run_id, task_index)
if self.use_memory:
if self.use_memory_addition: # selectively add memories when succeed
- new_traj_list = [self.get_traj_from_task_history(task_id, self.history[run_id][task_index], reward)]
+ new_traj_list = [
+ self.get_traj_from_task_history(task_id, self.history[run_id][task_index], reward),
+ ]
previous_memories = self.add_memory(new_traj_list)
if reward != 1:
self.delete_memory_by_ids([mem["memory_id"] for mem in previous_memories])
diff --git a/cookbook/bfcl/bfcl_utils.py b/test/cookbook/bfcl/bfcl_utils.py
similarity index 100%
rename from cookbook/bfcl/bfcl_utils.py
rename to test/cookbook/bfcl/bfcl_utils.py
diff --git a/cookbook/bfcl/init_exp_pool.py b/test/cookbook/bfcl/init_exp_pool.py
similarity index 100%
rename from cookbook/bfcl/init_exp_pool.py
rename to test/cookbook/bfcl/init_exp_pool.py
diff --git a/cookbook/bfcl/init_task_memory_pool.py b/test/cookbook/bfcl/init_task_memory_pool.py
similarity index 100%
rename from cookbook/bfcl/init_task_memory_pool.py
rename to test/cookbook/bfcl/init_task_memory_pool.py
diff --git a/cookbook/bfcl/local_file_to_library.py b/test/cookbook/bfcl/local_file_to_library.py
similarity index 100%
rename from cookbook/bfcl/local_file_to_library.py
rename to test/cookbook/bfcl/local_file_to_library.py
diff --git a/cookbook/bfcl/requirements.txt b/test/cookbook/bfcl/requirements.txt
similarity index 100%
rename from cookbook/bfcl/requirements.txt
rename to test/cookbook/bfcl/requirements.txt
diff --git a/cookbook/bfcl/run_bfcl.py b/test/cookbook/bfcl/run_bfcl.py
similarity index 99%
rename from cookbook/bfcl/run_bfcl.py
rename to test/cookbook/bfcl/run_bfcl.py
index fd8cc2db..177d91e1 100644
--- a/cookbook/bfcl/run_bfcl.py
+++ b/test/cookbook/bfcl/run_bfcl.py
@@ -91,7 +91,7 @@ def main():
num_runs = 1
num_trials = 2
- model_name="qwen3-8b"
+ model_name = "qwen3-8b"
use_memory = False
use_memory_addition = False
use_memory_deletion = False
diff --git a/cookbook/bfcl/run_exp_statistic.py b/test/cookbook/bfcl/run_exp_statistic.py
similarity index 100%
rename from cookbook/bfcl/run_exp_statistic.py
rename to test/cookbook/bfcl/run_exp_statistic.py
diff --git a/cookbook/bfcl/split_into_trainval.py b/test/cookbook/bfcl/split_into_trainval.py
similarity index 100%
rename from cookbook/bfcl/split_into_trainval.py
rename to test/cookbook/bfcl/split_into_trainval.py
diff --git a/reme/memory/tools/chunk/__init__.py b/test/cookbook/frozenlake/__init__.py
similarity index 100%
rename from reme/memory/tools/chunk/__init__.py
rename to test/cookbook/frozenlake/__init__.py
diff --git a/cookbook/frozenlake/frozenlake_prompts.yaml b/test/cookbook/frozenlake/frozenlake_prompts.yaml
similarity index 100%
rename from cookbook/frozenlake/frozenlake_prompts.yaml
rename to test/cookbook/frozenlake/frozenlake_prompts.yaml
diff --git a/cookbook/frozenlake/frozenlake_react_agent.py b/test/cookbook/frozenlake/frozenlake_react_agent.py
similarity index 100%
rename from cookbook/frozenlake/frozenlake_react_agent.py
rename to test/cookbook/frozenlake/frozenlake_react_agent.py
diff --git a/cookbook/frozenlake/map_manager.py b/test/cookbook/frozenlake/map_manager.py
similarity index 100%
rename from cookbook/frozenlake/map_manager.py
rename to test/cookbook/frozenlake/map_manager.py
diff --git a/cookbook/frozenlake/run_exp_statistic.py b/test/cookbook/frozenlake/run_exp_statistic.py
similarity index 100%
rename from cookbook/frozenlake/run_exp_statistic.py
rename to test/cookbook/frozenlake/run_exp_statistic.py
diff --git a/cookbook/frozenlake/run_frozenlake.py b/test/cookbook/frozenlake/run_frozenlake.py
similarity index 100%
rename from cookbook/frozenlake/run_frozenlake.py
rename to test/cookbook/frozenlake/run_frozenlake.py
diff --git a/reme/memory/tools/history/__init__.py b/test/cookbook/simple_demo/__init__.py
similarity index 100%
rename from reme/memory/tools/history/__init__.py
rename to test/cookbook/simple_demo/__init__.py
diff --git a/cookbook/simple_demo/import_usage_demo.py b/test/cookbook/simple_demo/import_usage_demo.py
similarity index 100%
rename from cookbook/simple_demo/import_usage_demo.py
rename to test/cookbook/simple_demo/import_usage_demo.py
diff --git a/cookbook/simple_demo/mcp_task_memory.jsonl b/test/cookbook/simple_demo/mcp_task_memory.jsonl
similarity index 100%
rename from cookbook/simple_demo/mcp_task_memory.jsonl
rename to test/cookbook/simple_demo/mcp_task_memory.jsonl
diff --git a/cookbook/simple_demo/personal_memory.jsonl b/test/cookbook/simple_demo/personal_memory.jsonl
similarity index 100%
rename from cookbook/simple_demo/personal_memory.jsonl
rename to test/cookbook/simple_demo/personal_memory.jsonl
diff --git a/cookbook/simple_demo/task_memory.jsonl b/test/cookbook/simple_demo/task_memory.jsonl
similarity index 100%
rename from cookbook/simple_demo/task_memory.jsonl
rename to test/cookbook/simple_demo/task_memory.jsonl
diff --git a/cookbook/simple_demo/task_messages.jsonl b/test/cookbook/simple_demo/task_messages.jsonl
similarity index 100%
rename from cookbook/simple_demo/task_messages.jsonl
rename to test/cookbook/simple_demo/task_messages.jsonl
diff --git a/cookbook/simple_demo/use_personal_memory_demo.py b/test/cookbook/simple_demo/use_personal_memory_demo.py
similarity index 100%
rename from cookbook/simple_demo/use_personal_memory_demo.py
rename to test/cookbook/simple_demo/use_personal_memory_demo.py
diff --git a/cookbook/simple_demo/use_task_memory_demo.py b/test/cookbook/simple_demo/use_task_memory_demo.py
similarity index 100%
rename from cookbook/simple_demo/use_task_memory_demo.py
rename to test/cookbook/simple_demo/use_task_memory_demo.py
diff --git a/cookbook/simple_demo/use_task_memory_mcp_demo.py b/test/cookbook/simple_demo/use_task_memory_mcp_demo.py
similarity index 100%
rename from cookbook/simple_demo/use_task_memory_mcp_demo.py
rename to test/cookbook/simple_demo/use_task_memory_mcp_demo.py
diff --git a/cookbook/simple_demo/use_tool_memory_demo.py b/test/cookbook/simple_demo/use_tool_memory_demo.py
similarity index 100%
rename from cookbook/simple_demo/use_tool_memory_demo.py
rename to test/cookbook/simple_demo/use_tool_memory_demo.py
diff --git a/reme/memory/tools/profiles/__init__.py b/test/cookbook/tool_memory/__init__.py
similarity index 100%
rename from reme/memory/tools/profiles/__init__.py
rename to test/cookbook/tool_memory/__init__.py
diff --git a/cookbook/tool_memory/query.json b/test/cookbook/tool_memory/query.json
similarity index 100%
rename from cookbook/tool_memory/query.json
rename to test/cookbook/tool_memory/query.json
diff --git a/cookbook/tool_memory/run_reme_tool_bench.py b/test/cookbook/tool_memory/run_reme_tool_bench.py
similarity index 100%
rename from cookbook/tool_memory/run_reme_tool_bench.py
rename to test/cookbook/tool_memory/run_reme_tool_bench.py
diff --git a/cookbook/working_memory/react_agent_with_working_memory.py b/test/cookbook/working_memory/react_agent_with_working_memory.py
similarity index 70%
rename from cookbook/working_memory/react_agent_with_working_memory.py
rename to test/cookbook/working_memory/react_agent_with_working_memory.py
index 52e3fdc6..27f2275e 100644
--- a/cookbook/working_memory/react_agent_with_working_memory.py
+++ b/test/cookbook/working_memory/react_agent_with_working_memory.py
@@ -28,9 +28,11 @@ class ReactAgent:
rather than on complex agent logic.
"""
- def __init__(self,
- model_name="",
- max_steps: int = 50):
+ def __init__(
+ self,
+ model_name="",
+ max_steps: int = 50,
+ ):
# You can replace this with your own LLM wrapper if needed.
self.llm = OpenAICompatibleLLM(model_name=model_name)
@@ -65,10 +67,16 @@ class ReactAgent:
# Prepare all available tools from the MCP server.
tool_dict: Dict[str, ToolCall] = {}
- async with FastMcpClient("reme_mcp_server", {
- "type": "sse",
- "url": "http://0.0.0.0:8002/sse",
- }) as mcp_client, HttpClient(base_url="http://localhost:8003") as http_client:
+ async with (
+ FastMcpClient(
+ "reme_mcp_server",
+ {
+ "type": "sse",
+ "url": "http://0.0.0.0:8002/sse",
+ },
+ ) as mcp_client,
+ HttpClient(base_url="http://localhost:8003") as http_client,
+ ):
tool_calls = await mcp_client.list_tool_calls()
for tool_call in tool_calls:
@@ -87,25 +95,30 @@ class ReactAgent:
# - compress long histories,
# - offload detailed context into working memory storage,
# - keep the recent message(s) for short-term reasoning.
- result = await http_client.execute_flow("summary_working_memory",
- messages=[x.simple_dump() for x in messages],
- working_summary_mode="auto",
- compact_ratio_threshold=0.75,
- max_total_tokens=20000,
- max_tool_message_tokens=2000,
- group_token_threshold=None,
- keep_recent_count=1,
- store_dir="./test_working_memory")
+ result = await http_client.execute_flow(
+ "summary_working_memory",
+ messages=[x.simple_dump() for x in messages],
+ working_summary_mode="auto",
+ compact_ratio_threshold=0.75,
+ max_total_tokens=20000,
+ max_tool_message_tokens=2000,
+ group_token_threshold=None,
+ keep_recent_count=1,
+ store_dir="./test_working_memory",
+ )
# Convert the API result back into `Message` objects for the LLM.
messages = [Message(**x) for x in result.answer]
# Ask the LLM what to do next.
# You can plug in your own tool-calling strategy here.
- assistant_message: Message = await self.llm.achat(messages=messages, tools=[
- tool_dict["grep_working_memory"],
- tool_dict["read_working_memory"],
- ])
+ assistant_message: Message = await self.llm.achat(
+ messages=messages,
+ tools=[
+ tool_dict["grep_working_memory"],
+ tool_dict["read_working_memory"],
+ ],
+ )
messages.append(assistant_message)
@@ -118,20 +131,25 @@ class ReactAgent:
logger.exception(f"unknown tool_call.name={tool_call.name}")
continue
- logger.info(f"round{i + 1}.{j} submit tool_calls={tool_call.name} "
- f"argument={tool_call.argument_dict}")
+ logger.info(
+ f"round{i + 1}.{j} submit tool_calls={tool_call.name} " f"argument={tool_call.argument_dict}",
+ )
# Execute the tool via MCP and parse the result.
- result = await mcp_client.call_tool(tool_call.name,
- arguments=tool_call.argument_dict,
- parse_result=True)
+ result = await mcp_client.call_tool(
+ tool_call.name,
+ arguments=tool_call.argument_dict,
+ parse_result=True,
+ )
# Attach the tool result as a TOOL-role message so the LLM
# can see and reason about it in the next step.
- messages.append(Message(
- role=Role.TOOL,
- tool_call_id=tool_call.id,
- content=result,
- ))
+ messages.append(
+ Message(
+ role=Role.TOOL,
+ tool_call_id=tool_call.id,
+ content=result,
+ ),
+ )
return messages
diff --git a/cookbook/working_memory/work_memory_demo.py b/test/cookbook/working_memory/work_memory_demo.py
similarity index 99%
rename from cookbook/working_memory/work_memory_demo.py
rename to test/cookbook/working_memory/work_memory_demo.py
index 612a637a..826cb0f2 100644
--- a/cookbook/working_memory/work_memory_demo.py
+++ b/test/cookbook/working_memory/work_memory_demo.py
@@ -108,7 +108,7 @@ async def main():
logger.info(
f"origin_token_count: {origin_token_count} "
f"after_token_count: {after_token_count} "
- f"compress_ratio={after_token_count / origin_token_count:.2f}"
+ f"compress_ratio={after_token_count / origin_token_count:.2f}",
)
diff --git a/test/cli/__init__.py b/test/test/cli/__init__.py
similarity index 100%
rename from test/cli/__init__.py
rename to test/test/cli/__init__.py
diff --git a/test/cli/fb_cli.py b/test/test/cli/fb_cli.py
similarity index 100%
rename from test/cli/fb_cli.py
rename to test/test/cli/fb_cli.py
diff --git a/test/cli/fb_cli.yaml b/test/test/cli/fb_cli.yaml
similarity index 100%
rename from test/cli/fb_cli.yaml
rename to test/test/cli/fb_cli.yaml
diff --git a/test/cli/fb_compactor.py b/test/test/cli/fb_compactor.py
similarity index 100%
rename from test/cli/fb_compactor.py
rename to test/test/cli/fb_compactor.py
diff --git a/test/cli/fb_compactor.yaml b/test/test/cli/fb_compactor.yaml
similarity index 100%
rename from test/cli/fb_compactor.yaml
rename to test/test/cli/fb_compactor.yaml
diff --git a/test/cli/fb_context_checker.py b/test/test/cli/fb_context_checker.py
similarity index 100%
rename from test/cli/fb_context_checker.py
rename to test/test/cli/fb_context_checker.py
diff --git a/test/cli/fb_summarizer.py b/test/test/cli/fb_summarizer.py
similarity index 100%
rename from test/cli/fb_summarizer.py
rename to test/test/cli/fb_summarizer.py
diff --git a/test/cli/fb_summarizer.yaml b/test/test/cli/fb_summarizer.yaml
similarity index 100%
rename from test/cli/fb_summarizer.yaml
rename to test/test/cli/fb_summarizer.yaml
diff --git a/test/reme_cli.py b/test/test/reme_cli.py
similarity index 100%
rename from test/reme_cli.py
rename to test/test/reme_cli.py
diff --git a/test/test_fs_compactor.py b/test/test/test_fs_compactor.py
similarity index 100%
rename from test/test_fs_compactor.py
rename to test/test/test_fs_compactor.py
diff --git a/test/test_fs_context_checker.py b/test/test/test_fs_context_checker.py
similarity index 100%
rename from test/test_fs_context_checker.py
rename to test/test/test_fs_context_checker.py
diff --git a/test/test_fs_file_watch_integration.py b/test/test/test_fs_file_watch_integration.py
similarity index 100%
rename from test/test_fs_file_watch_integration.py
rename to test/test/test_fs_file_watch_integration.py
diff --git a/test/test_fs_memory_get.py b/test/test/test_fs_memory_get.py
similarity index 100%
rename from test/test_fs_memory_get.py
rename to test/test/test_fs_memory_get.py
diff --git a/test/test_fs_memory_search.py b/test/test/test_fs_memory_search.py
similarity index 100%
rename from test/test_fs_memory_search.py
rename to test/test/test_fs_memory_search.py
diff --git a/test/test_fs_summary.py b/test/test/test_fs_summary.py
similarity index 100%
rename from test/test_fs_summary.py
rename to test/test/test_fs_summary.py
diff --git a/tests/light/test_summarizer.py b/tests/light/test_summarizer.py
index a2f2d975..9718cd2a 100644
--- a/tests/light/test_summarizer.py
+++ b/tests/light/test_summarizer.py
@@ -15,7 +15,7 @@ from test_utils import (
)
from reme.core.utils import get_std_logger
from reme.memory.file_based import Summarizer
-from reme.memory.tools.file import FileIO
+from reme.memory.file_based.tools import FileIO
logger = get_std_logger()
diff --git a/tests/light/test_tools.py b/tests/light/test_tools.py
new file mode 100644
index 00000000..933d506d
--- /dev/null
+++ b/tests/light/test_tools.py
@@ -0,0 +1,321 @@
+# -*- coding: utf-8 -*-
+# pylint: disable=redefined-outer-name
+"""Unit tests for Shell and FileIO tools."""
+
+import asyncio
+import os
+import shutil
+import tempfile
+
+import pytest
+
+from reme.memory.file_based.tools.shell import Shell
+from reme.memory.file_based.tools.file_io import FileIO
+from reme.memory.file_based.tools.utils import DEFAULT_MAX_LINES, DEFAULT_MAX_BYTES
+
+
+# ============ Shell Tests ============
+
+
+@pytest.fixture(scope="module")
+def shell_env():
+ """Create temporary directory and Shell instance."""
+ test_dir = tempfile.mkdtemp(prefix="test_shell_")
+ shell = Shell(working_dir=test_dir)
+ yield {"dir": test_dir, "shell": shell}
+ shutil.rmtree(test_dir, ignore_errors=True)
+
+
+def test_shell_echo_success(shell_env):
+ """Test successful echo command execution."""
+ result = asyncio.run(shell_env["shell"].execute_shell_command("echo hello"))
+ assert result.content
+ text = result.content[0].get("text", "")
+ assert "hello" in text
+
+
+def test_shell_pwd_in_working_dir(shell_env):
+ """Test command executes in correct working directory."""
+ result = asyncio.run(shell_env["shell"].execute_shell_command("pwd"))
+ text = result.content[0].get("text", "")
+ assert shell_env["dir"] in text
+
+
+def test_shell_command_failure(shell_env):
+ """Test failed command returns error information."""
+ result = asyncio.run(shell_env["shell"].execute_shell_command("exit 1"))
+ text = result.content[0].get("text", "")
+ assert "failed" in text.lower()
+ assert "exit code" in text.lower()
+
+
+def test_shell_command_with_stderr(shell_env):
+ """Test command with stderr output."""
+ result = asyncio.run(
+ shell_env["shell"].execute_shell_command("echo error >&2 && exit 1"),
+ )
+ text = result.content[0].get("text", "")
+ assert "error" in text
+
+
+def test_shell_no_output(shell_env):
+ """Test successful command with no output."""
+ result = asyncio.run(shell_env["shell"].execute_shell_command("true"))
+ text = result.content[0].get("text", "")
+ assert "successfully" in text.lower()
+
+
+def test_shell_multiline_output(shell_env):
+ """Test command with multiline output."""
+ result = asyncio.run(
+ shell_env["shell"].execute_shell_command("echo -e 'line1\nline2\nline3'"),
+ )
+ text = result.content[0].get("text", "")
+ assert "line1" in text
+ assert "line2" in text
+ assert "line3" in text
+
+
+def test_shell_truncated_output(shell_env):
+ """Test output truncation for large output."""
+ lines_to_generate = DEFAULT_MAX_LINES + 500
+ cmd = f"seq 1 {lines_to_generate}"
+ result = asyncio.run(shell_env["shell"].execute_shell_command(cmd))
+ text = result.content[0].get("text", "")
+
+ # Should contain truncation notice
+ assert "truncated" in text.lower()
+ # Should contain the last line (tail is kept)
+ assert str(lines_to_generate) in text
+ # Verify first numeric line is > 1 (truncated from head)
+ numeric_lines = [tl for tl in text.strip().split("\n") if tl.isdigit()]
+ if numeric_lines:
+ assert int(numeric_lines[0]) > 1
+
+
+def test_shell_timeout(shell_env):
+ """Test command timeout handling."""
+ result = asyncio.run(
+ shell_env["shell"].execute_shell_command("sleep 10", timeout=1),
+ )
+ text = result.content[0].get("text", "")
+ assert "timeout" in text.lower()
+
+
+# ============ FileIO Read Tests ============
+
+
+@pytest.fixture(scope="module")
+def fileio_env():
+ """Create temporary directory with test files."""
+ test_dir = tempfile.mkdtemp(prefix="test_fileio_")
+ file_io = FileIO(working_dir=test_dir)
+
+ # Create simple test file
+ simple_file = os.path.join(test_dir, "simple.txt")
+ with open(simple_file, "w", encoding="utf-8") as f:
+ f.write("line1\nline2\nline3\nline4\nline5")
+
+ # Create large file (exceeds DEFAULT_MAX_LINES)
+ large_file = os.path.join(test_dir, "large.txt")
+ with open(large_file, "w", encoding="utf-8") as f:
+ for i in range(1, DEFAULT_MAX_LINES + 500):
+ f.write(f"line {i}\n")
+
+ # Create large bytes file (exceeds DEFAULT_MAX_BYTES)
+ large_bytes_file = os.path.join(test_dir, "large_bytes.txt")
+ with open(large_bytes_file, "w", encoding="utf-8") as f:
+ content = "x" * 100 + "\n"
+ lines_needed = (DEFAULT_MAX_BYTES // 101) + 100
+ for _ in range(lines_needed):
+ f.write(content)
+
+ yield {
+ "dir": test_dir,
+ "file_io": file_io,
+ "simple_file": simple_file,
+ "large_file": large_file,
+ "large_bytes_file": large_bytes_file,
+ }
+ shutil.rmtree(test_dir, ignore_errors=True)
+
+
+def test_read_file_success(fileio_env):
+ """Test successful file reading."""
+ result = asyncio.run(fileio_env["file_io"].read(fileio_env["simple_file"]))
+ text = result.content[0].get("text", "")
+ assert "line1" in text
+ assert "line5" in text
+
+
+def test_read_file_relative_path(fileio_env):
+ """Test reading file with relative path."""
+ result = asyncio.run(fileio_env["file_io"].read("simple.txt"))
+ text = result.content[0].get("text", "")
+ assert "line1" in text
+
+
+def test_read_file_not_exists(fileio_env):
+ """Test reading non-existent file."""
+ result = asyncio.run(fileio_env["file_io"].read("nonexistent.txt"))
+ text = result.content[0].get("text", "")
+ assert "Error" in text
+ assert "does not exist" in text
+
+
+def test_read_file_with_line_range(fileio_env):
+ """Test reading specific line range."""
+ result = asyncio.run(
+ fileio_env["file_io"].read(fileio_env["simple_file"], start_line=2, end_line=4),
+ )
+ text = result.content[0].get("text", "")
+ assert "line2" in text
+ assert "line4" in text
+ assert "lines 2-4" in text.lower()
+
+
+def test_read_file_start_line_exceeds(fileio_env):
+ """Test start_line exceeding file length."""
+ result = asyncio.run(
+ fileio_env["file_io"].read(fileio_env["simple_file"], start_line=100),
+ )
+ text = result.content[0].get("text", "")
+ assert "Error" in text
+ assert "exceeds" in text
+
+
+def test_read_file_invalid_range(fileio_env):
+ """Test invalid line range (start > end)."""
+ result = asyncio.run(
+ fileio_env["file_io"].read(fileio_env["simple_file"], start_line=4, end_line=2),
+ )
+ text = result.content[0].get("text", "")
+ assert "Error" in text
+
+
+def test_read_file_truncated_by_lines(fileio_env):
+ """Test file truncation by line limit."""
+ result = asyncio.run(fileio_env["file_io"].read(fileio_env["large_file"]))
+ text = result.content[0].get("text", "")
+ assert "line 1" in text # Head is kept
+ assert "continue" in text.lower()
+
+
+def test_read_file_truncated_by_bytes(fileio_env):
+ """Test file truncation by byte limit."""
+ result = asyncio.run(fileio_env["file_io"].read(fileio_env["large_bytes_file"]))
+ text = result.content[0].get("text", "")
+ assert "continue" in text.lower() or "KB" in text
+
+
+def test_read_directory_error(fileio_env):
+ """Test reading a directory returns error."""
+ result = asyncio.run(fileio_env["file_io"].read(fileio_env["dir"]))
+ text = result.content[0].get("text", "")
+ assert "Error" in text
+ assert "not a file" in text
+
+
+# ============ FileIO Write Tests ============
+
+
+@pytest.fixture
+def write_env():
+ """Create temporary directory for write tests."""
+ test_dir = tempfile.mkdtemp(prefix="test_fileio_write_")
+ file_io = FileIO(working_dir=test_dir)
+ yield {"dir": test_dir, "file_io": file_io}
+ shutil.rmtree(test_dir, ignore_errors=True)
+
+
+def test_write_new_file(write_env):
+ """Test writing a new file."""
+ file_path = os.path.join(write_env["dir"], "new_file.txt")
+ result = asyncio.run(write_env["file_io"].write(file_path, "test content"))
+ text = result.content[0].get("text", "")
+ assert "Wrote" in text
+
+ with open(file_path, "r", encoding="utf-8") as f:
+ assert f.read() == "test content"
+
+
+def test_write_overwrite_file(write_env):
+ """Test overwriting existing file."""
+ file_path = os.path.join(write_env["dir"], "overwrite.txt")
+ with open(file_path, "w", encoding="utf-8") as f:
+ f.write("old content")
+
+ result = asyncio.run(write_env["file_io"].write(file_path, "new content"))
+ text = result.content[0].get("text", "")
+ assert "Wrote" in text
+
+ with open(file_path, "r", encoding="utf-8") as f:
+ assert f.read() == "new content"
+
+
+def test_write_empty_path(write_env):
+ """Test writing with empty path."""
+ result = asyncio.run(write_env["file_io"].write("", "content"))
+ text = result.content[0].get("text", "")
+ assert "Error" in text
+
+
+def test_write_relative_path(write_env):
+ """Test writing file with relative path."""
+ result = asyncio.run(write_env["file_io"].write("relative.txt", "relative content"))
+ text = result.content[0].get("text", "")
+ assert "Wrote" in text
+
+ file_path = os.path.join(write_env["dir"], "relative.txt")
+ assert os.path.exists(file_path)
+
+
+# ============ FileIO Edit Tests ============
+
+
+@pytest.fixture
+def edit_env():
+ """Create temporary directory with edit test file."""
+ test_dir = tempfile.mkdtemp(prefix="test_fileio_edit_")
+ file_io = FileIO(working_dir=test_dir)
+
+ edit_file = os.path.join(test_dir, "edit_test.txt")
+ with open(edit_file, "w", encoding="utf-8") as f:
+ f.write("Hello World\nThis is a test\nHello Again")
+
+ yield {"dir": test_dir, "file_io": file_io, "edit_file": edit_file}
+ shutil.rmtree(test_dir, ignore_errors=True)
+
+
+def test_edit_replace_text(edit_env):
+ """Test replacing text in file."""
+ result = asyncio.run(
+ edit_env["file_io"].edit(edit_env["edit_file"], "Hello", "Hi"),
+ )
+ text = result.content[0].get("text", "")
+ assert "Successfully" in text
+
+ with open(edit_env["edit_file"], "r", encoding="utf-8") as f:
+ content = f.read()
+ assert "Hello" not in content
+ assert "Hi World" in content
+ assert "Hi Again" in content
+
+
+def test_edit_text_not_found(edit_env):
+ """Test editing when text not found."""
+ result = asyncio.run(
+ edit_env["file_io"].edit(edit_env["edit_file"], "NotExists", "Replacement"),
+ )
+ text = result.content[0].get("text", "")
+ assert "Error" in text
+ assert "not found" in text
+
+
+def test_edit_nonexistent_file(edit_env):
+ """Test editing non-existent file."""
+ result = asyncio.run(
+ edit_env["file_io"].edit("nonexistent.txt", "old", "new"),
+ )
+ text = result.content[0].get("text", "")
+ assert "Error" in text
diff --git a/tests/vector/test_reme_vector.py b/tests/vector/test_reme_vector.py
new file mode 100644
index 00000000..3829c841
--- /dev/null
+++ b/tests/vector/test_reme_vector.py
@@ -0,0 +1,89 @@
+"""测试 ReMe 的 vector 搜索功能"""
+
+import asyncio
+
+from reme import ReMe
+
+
+async def main():
+ """测试 ReMe 的 vector 搜索功能"""
+ # 初始化 ReMe
+ reme = ReMe(
+ working_dir=".reme",
+ default_llm_config={
+ "backend": "openai",
+ "model_name": "qwen3.5-plus",
+ },
+ default_embedding_model_config={
+ "backend": "openai",
+ "model_name": "text-embedding-v4",
+ "dimensions": 1024,
+ },
+ default_vector_store_config={
+ "backend": "local", # 支持 local/chroma/qdrant/elasticsearch
+ },
+ )
+ await reme.start()
+
+ messages = [
+ {"role": "user", "content": "帮我写一个 Python 脚本", "time_created": "2026-02-28 10:00:00"},
+ {"role": "assistant", "content": "好的，我来帮你写", "time_created": "2026-02-28 10:00:05"},
+ ]
+
+ # 1. 从对话中总结记忆（自动提取用户偏好、任务经验等）
+ result = await reme.summarize_memory(
+ messages=messages,
+ user_name="alice", # 个人记忆
+ # task_name="code_writing", # 任务记忆
+ )
+ print(f"总结结果: {result}")
+
+ # 2. 检索相关记忆
+ memories = await reme.retrieve_memory(
+ query="Python 编程",
+ user_name="alice",
+ # task_name="code_writing",
+ )
+ print(f"检索结果: {memories}")
+
+ # 3. 手动添加记忆
+ memory_node = await reme.add_memory(
+ memory_content="用户喜欢简洁的代码风格",
+ user_name="alice",
+ )
+ print(f"添加的记忆: {memory_node}")
+ memory_id = memory_node.memory_id
+
+ # 4. 通过 ID 获取单条记忆
+ fetched_memory = await reme.get_memory(memory_id=memory_id)
+ print(f"获取的记忆: {fetched_memory}")
+
+ # 5. 更新记忆内容
+ updated_memory = await reme.update_memory(
+ memory_id=memory_id,
+ user_name="alice",
+ memory_content="用户喜欢简洁且带注释的代码风格",
+ )
+ print(f"更新后的记忆: {updated_memory}")
+
+ # 6. 列出用户的所有记忆（支持过滤和排序）
+ all_memories = await reme.list_memory(
+ user_name="alice",
+ limit=10,
+ sort_key="time_created",
+ reverse=True,
+ )
+ print(f"用户记忆列表: {all_memories}")
+
+ # 7. 删除指定记忆
+ await reme.delete_memory(memory_id=memory_id)
+ print(f"已删除记忆: {memory_id}")
+
+ # 8. 删除所有记忆（谨慎使用）
+ # await reme.delete_all()
+
+ await reme.close()
+
+
+if __name__ == "__main__":
+ asyncio.run(main())