scriptoriumadmin/ReMe
1
0
Fork 0
mirror of https://github.com/agentscope-ai/ReMe.git synced 2026-08-28 05:25:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 11

Add the description document about the op in working memory.

Browse source
This commit is contained in:
方应 2025-11-27 17:25:14 +08:00
parent ed7f8e32ef
commit dfd2a35ccb
3 changed files with 390 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

171
docs/work_memory/message_offload.md Normal file
View file

@ -0,0 +1,171 @@
---
jupytext:
formats: md:myst
text_representation:
extension: .md
format_name: myst
format_version: 0.13
jupytext_version: 1.11.5
kernelspec:
display_name: Python 3
language: python
name: python3
---
# Message Offload
## 1. Background: Why Message Offload?
### The Agent Context Challenge
In modern AI agent systems, LLMs interact with tools through iterative loops, accumulating conversation history and tool results. With each iteration, a critical problem emerges:
**The Core Problem: Context Window Explosion**
When an agent executes complex tasks, it relies on maintaining conversation history to track progress and make informed decisions. However:
- **Rapid Context Growth**: Each tool call appends input parameters and output results to message history
- **Token Consumption**: A single tool call can consume hundreds or thousands of tokens, especially for data-heavy operations
- **Context Window Limits**: Most LLMs have finite context windows (e.g., 128K, 200K tokens)
- **Context Rot**: As context grows beyond optimal thresholds, model performance degrades significantly
**Example: Web Research Agent**
Imagine an agent performing research across multiple sources:
```
Iteration 1: web_search("AI context management") → 3,500 tokens
Iteration 2: read_webpage(url_1) → 8,200 tokens
Iteration 3: web_search("context compression techniques") → 4,100 tokens
Iteration 4: read_webpage(url_2) → 7,800 tokens
...
Iteration 15: summarize_findings() → Total context: 95,000 tokens
```
As context accumulates:
- **At 50K tokens**: Agent performs normally, accurate responses
- **At 100K tokens**: Responses become repetitive, slower inference
- **At 150K tokens**: Significant quality degradation, "context rot" sets in
- **At 200K tokens**: Context window exhausted, cannot continue
**Without context management, agents hit walls after just 15-20 complex tool calls.**
### The Solution: Message Offload as Context Engineering
Message Offload solves this by **intelligently moving non-essential information out of active context**, allowing agents to operate indefinitely while maintaining optimal performance:
**1. Message Compaction** (Reversible Strategy)
- **Selective Storage**: Large tool results stored in external files
- **Reference Retention**: Only file paths kept in message history
- **On-Demand Retrieval**: Full content can be retrieved when needed
**2. Message Compression** (LLM-Based Strategy)
- **Intelligent Summarization**: LLM generates concise summaries of older message groups
- **Priority Preservation**: Recent messages and system prompts remain intact
- **Information Density**: Maintains key information while reducing token count
**3. Hybrid Auto Mode** (Adaptive Strategy)
- **Compaction First**: Applies compaction to tool messages
- **Compression When Needed**: Triggers compression if compaction ratio exceeds threshold
- **Dynamic Adjustment**: Adapts strategy based on context characteristics
### Enhanced work memory management
Instead of letting context grow uncontrollably, the agent now benefits from:
```
Traditional Approach (No Context Management):
50 messages → 95,000 tokens → Context rot begins
- Response quality: Degraded
- Inference speed: Slow
- Can continue: No (approaching limit)
- Information lost: No, but unusable
+ Message Offload Approach:
50 messages → 15,000 tokens (after offload) → Optimal performance maintained
- Response quality: High
- Inference speed: Fast
- Can continue: Yes (85% headroom remaining)
- Information lost: No (stored externally, retrievable)
Offload Details:
- 20 tool messages compacted → Stored in /context_store/
- 15 older messages compressed → Summarized in system message
- 5 recent messages preserved → Full content intact
- External storage: 80,000 tokens offloaded
- Active context: 15,000 tokens (84% reduction)
```
This managed context enables the agent to:
- **Operate Indefinitely**: No hard limit on conversation length
- **Maintain Performance**: Stay within optimal token range (10-30K tokens)
- **Preserve Information**: All data accessible through file system or summaries
- **Optimize Costs**: Reduce token consumption by 70-90% in long conversations
### The Impact: From Context Explosion to Controlled Growth
**Traditional Approach (No Work Memory Management):**
```
Agent: "I've executed 20 tool calls, context is now 100K tokens"
→ Performance degradation begins
→ Slower responses, repetitive outputs
→ Cannot continue beyond 30 calls
→ Task abandoned due to context limits
```
**Message Offload Approach (Intelligent Management):**
```
Agent: "I've executed 100 tool calls, active context maintained at 18K tokens"
→ Optimal performance throughout
→ Fast, accurate responses
→ Can continue indefinitely
→ All historical data accessible when needed
```
**Real-World Impact:**
```
Before Message Offload (20 tool calls):
- Active context: 95,000 tokens
- Performance: Degraded (context rot)
- Can continue: No (near limit)
- Response quality: 6/10
- Inference time: 8-12 seconds
- Max task complexity: Low (15-20 calls)
After Message Offload (100 tool calls):
- Active context: 18,000 tokens (-81%)
- Performance: Optimal
- Can continue: Yes (90% headroom)
- Response quality: 9/10
- Inference time: 2-4 seconds (-70%)
- Max task complexity: High (100+ calls)
```
## 2. Implementation in ReMe
ReMe has fully implemented the above-mentioned message offload and reload mechanisms, inspired by [Context Engineering for AI Agents with LangChain and Manus](https://www.youtube.com/watch?v=6_BcCthVvb8). The implementation provides two core operation primitives:
### (1) Message Offload Operations
Operations for intelligently reducing context size through compaction and compression strategies.
📖 **Detailed Usage Guide**: [Message Offload Ops](message_offload_ops.md)
Key features:
- Three working summary modes: compact, compress, and auto
- Intelligent token threshold management
- Integration with file storage system
- Complete working examples in test files
### (2) Message Reload Operations
Operations for retrieving and accessing offloaded content when needed.
📖 **Detailed Usage Guide**: [Message Reload Ops](message_reload_ops.md)
Key features:
- Text search within offloaded files (GrepOp)
- Efficient file reading with pagination (ReadFileOp)
- Support for both absolute and relative paths
- Complete working examples in test files
Both operation primitives are production-ready and can be integrated into your agent workflows. Refer to the linked documentation for API specifications, parameter details, and practical usage examples.

95
docs/work_memory/message_offload_ops.md Normal file
View file

@ -0,0 +1,95 @@
---
jupytext:
formats: md:myst
text_representation:
extension: .md
format_name: myst
format_version: 0.13
jupytext_version: 1.11.5
kernelspec:
display_name: Python 3
language: python
name: python3
---
# Message Offload Ops
## MessageOffloadOp
### Purpose
Manages context window limits by intelligently offloading message content through compaction and compression strategies to reduce token usage while preserving important information.
### Functionality
- Supports three working summary modes: `compact`, `compress`, and `auto`
- **Compact mode**: Stores full content of large tool messages in external files, keeping only previews in context
- **Compress mode**: Uses LLM to generate concise summaries of older message groups
- **Auto mode** (recommended): Applies compaction first, then compression if compaction ratio exceeds `compact_ratio_threshold`
- Automatically writes offloaded content to files via `BatchWriteFileOp`
- Preserves recent messages and system messages to maintain conversation coherence
- Configurable token thresholds for both compaction and compression operations
### Parameters
- `messages` (array, **required**):
- List of conversation messages to process for working memory summarization
- Messages are analyzed for token count and processed according to management mode
- `working_summary_mode` (string, optional, default: `"auto"`):
- Working summary strategy to use
- `"compact"`: Only applies compaction to large tool messages
- `"compress"`: Only applies LLM-based compression
- `"auto"`: Applies compaction first then compression if compaction ratio exceeds threshold
- Allowed values: `["compact", "compress", "auto"]`
- `compact_ratio_threshold` (number, optional, default: `0.75`):
- Only used in `"auto"` mode
- Threshold for compaction ratio (tokens after compaction divided by original tokens)
- When the ratio is greater than this value, an additional LLM-based compression pass is triggered
- Example: If ratio is 0.76 (76%) and threshold is 0.75, compression will be applied
- `max_total_tokens` (integer, optional, default: `20000`):
- Maximum token count threshold for triggering compression/compaction
- For compaction mode: this is the total token count threshold
- For compression mode: excludes `keep_recent_count` messages and system messages
- Operation is skipped if token count is below this threshold
- `max_tool_message_tokens` (integer, optional, default: `2000`):
- Maximum token count per individual tool message before compaction is applied
- Tool messages exceeding this threshold will have full content stored in external files
- Only a preview is kept in context with a reference to the stored file
- `group_token_threshold` (integer, optional):
- Maximum token count per compression group when using LLM-based compression
- If `None` or `0`, all messages are compressed in a single group
- Messages exceeding this threshold individually will form their own group
- Only used in `"compress"` or `"auto"` mode
- `keep_recent_count` (integer, optional, default: `1` for compaction, `2` for compression):
- Number of recent messages to preserve without compression or compaction
- These messages remain unchanged to maintain conversation context
- Does not include system messages (which are always preserved)
- `store_dir` (string, optional):
- Directory path for storing summarized message content
- Full tool message content and compressed message groups are saved as files in this directory
- Required for compaction and compression operations
- `chat_id` (string, optional):
- Unique identifier for the chat session
- Used for file naming when storing compressed message groups
- If not provided, a UUID will be generated automatically
### Usage Pattern
For complete working examples of how to use MessageOffloadOp in practice, please refer to:
[test_message_offload_op.py](../../test_op/test_message_offload_op.py)
This test file demonstrates:
- **Compact mode**: How to configure and use compaction-only strategy
- **Compress mode**: How to apply LLM-based compression strategy
- **Auto mode**: How to combine compaction and compression intelligently
- Proper parameter settings for different scenarios
- Integration with `BatchWriteFileOp` for file writing
- Real-world message sequences with various token sizes

124
docs/work_memory/message_reload_ops.md Normal file
View file

@ -0,0 +1,124 @@
---
jupytext:
formats: md:myst
text_representation:
extension: .md
format_name: myst
format_version: 0.13
jupytext_version: 1.11.5
kernelspec:
display_name: Python 3
language: python
name: python3
---
# Message Reload Ops
## GrepOp
### Purpose
Provides a text search capability for locating specific content within offloaded files by pattern matching. This operation enables case-insensitive search within a single file, making it easy to find specific lines in tool messages or compressed groups.
### Functionality
- Searches for literal text patterns (case-insensitive) within a single file
- Limits result count to avoid overwhelming output (default: 50 matches)
- Returns matching lines with file path, line number, and content
- Ideal for locating specific content within known offloaded files
### Parameters
- `file_path` (string, **required**):
- The path to the file to search in
- Can be an absolute or relative path
- Must be a valid file path (not a directory)
- Examples:
- `/workspace/context_store/tool_call_123.txt`
- `./context_store/compressed_group_0.json`
- `pattern` (string, **required**):
- The text pattern to search for in the file
- Search is case-insensitive
- Searched as a literal string (special regex characters are escaped)
- Examples: `"stored in"`, `"error message"`, `"function_name"`
- `limit` (number, optional, default: `50`):
- Maximum number of matching lines to return
- Stops searching after reaching the limit
- Useful for large files to avoid token overflow
- Example: `100` returns at most 100 matching lines
### Return Value
The operation returns search results with matching lines:
- Each match is formatted as: `file_path:line_number:line_content`
- Returns up to `limit` matches
- If no matches found, returns a message indicating no matches
- Each match shows the complete line containing the pattern
Example: Searching for `"error"` in `/workspace/context_store/tool_call_123.txt` with limit 50 returns matching lines like:
```
/workspace/context_store/tool_call_123.txt:45:Error: Connection timeout
/workspace/context_store/tool_call_123.txt:78:Warning: Retrying after error
```
## ReadFileOp
### Purpose
Reads and returns the content of offloaded files, enabling on-demand access to compacted tool messages and compressed conversation history. Supports efficient pagination for handling large files.
### Functionality
- Reads file content from specified path (absolute or relative)
- Supports pagination with offset and limit for reading specific line ranges
- Uses efficient `sed` command for line-based reading
- Works with text files
- Essential for retrieving full content of compacted tool messages
- Enables access to original message groups before compression
### Parameters
- `file_path` (string, **required**):
- The path to the file to read
- Can be absolute or relative path
- Path will be expanded and resolved automatically
- Examples:
- `/workspace/context_store/tool_call_123.txt`
- `./context_store/compressed_group_0.json`
- `~/context_store/message.txt`
- `offset` (number, **required** but has default):
- The 0-based line number to start reading from
- If not provided or 0, starts from the beginning of the file
- Used in combination with `limit` for pagination
- Example: `0` starts from the first line, `100` starts from line 100
- `limit` (number, **required** but has default):
- Maximum number of lines to read from the offset
- If not provided, defaults to 1,000,000 (reads to end of file)
- Used with `offset` to implement pagination
- Example: `100` reads up to 100 lines from the offset
### Return Value
The operation returns the file content as a string:
- Content of the specified line range (from `offset` to `offset + limit`)
- Lines are returned without trailing newlines
- Empty string if the specified range is beyond the file's content
- Error message if file not found or cannot be read
Example: Reading `/workspace/context_store/tool_call_123.txt` with `offset=0` and `limit=100` returns the first 100 lines of the file.
## Usage Pattern: Combining Grep and ReadFile
For a complete working example of how to use these operations in practice, please refer to:
[test_agentic_retrieve_op.py](../../test_op/test_agentic_retrieve_op.py)
This test file demonstrates:
- How to configure the system prompt to guide AI in using Grep and ReadFile operations
- Real-world usage scenarios with message offload and reload
- Proper parameter settings for `AgenticRetrieveOp` with working memory
- Best practices for combining these operations in a retrieval workflow