mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-08-28 05:25:04 +00:00
refactor(core): restructure context management and working memory modules
This commit is contained in:
jinli.yl
parent
f478e33b2c
commit
b421515c73
32 changed files with 998 additions and 662 deletions
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -30,4 +30,6 @@ cookbook/appworld/file_vector_store/*
|
|||
/.venv/
|
||||
site/*
|
||||
docs/_build/*
|
||||
test_compact_storage/*
|
||||
test_compact_storage/*
|
||||
test_working_memory/*
|
||||
*.code-workspace
|
||||
|
|
@ -12,68 +12,128 @@ kernelspec:
|
|||
name: python3
|
||||
---
|
||||
|
||||
# Vector Store Configuration Guide
|
||||
### Vector Store User Guide
|
||||
|
||||
This guide covers how to configure vector store backends in ReMe using the `default.yaml` configuration file.
|
||||
Vector Store is a component designed for storing, managing, and retrieving vector embeddings. It supports features such as workspace management, similarity search, and metadata filtering.
|
||||
|
||||
## 📋 Overview
|
||||
## Core Concepts
|
||||
|
||||
ReMe provides multiple vector store backends for different use cases:
|
||||
**Workspace**: Each workspace is an independent vector storage unit used to organize and manage related vector nodes.
|
||||
|
||||
- **LocalVectorStore** (`backend=local`) - 📁 Simple file-based storage for development and small datasets
|
||||
- **ChromaVectorStore** (`backend=chroma`) - 🔮 Embedded vector database for moderate scale
|
||||
- **EsVectorStore** (`backend=elasticsearch`) - 🔍 Elasticsearch-based storage for production and large scale
|
||||
- **QdrantVectorStore** (`backend=qdrant`) - 🎯 High-performance vector database with advanced filtering
|
||||
- **MemoryVectorStore** (`backend=memory`) - ⚡ In-memory storage for ultra-fast access and testing
|
||||
**VectorNode**: A data unit containing text content, vector embedding, and metadata. It serves as the fundamental unit for storage and retrieval.
|
||||
|
||||
All vector stores implement the `BaseVectorStore` interface, providing a consistent API across implementations.
|
||||
**Embedding Model**: Used to convert text into vector embeddings. It supports automatic generation of both node vectors and query vectors.
|
||||
|
||||
## 📊 Comparison Table
|
||||
## Available Implementations
|
||||
|
||||
| Feature | LocalVectorStore | ChromaVectorStore | EsVectorStore | QdrantVectorStore | MemoryVectorStore |
|
||||
|----------------------|------------------|-------------------|---------------|-------------------|-------------------|
|
||||
| **Storage** | File (JSONL) | Embedded DB | Elasticsearch | Qdrant Server | In-Memory |
|
||||
| **Performance** | Medium | Good | Excellent | Excellent | Ultra-Fast |
|
||||
| **Scalability** | < 10K vectors | < 1M vectors | > 1M vectors | > 10M vectors | < 1M vectors |
|
||||
| **Persistence** | ✅ Auto | ✅ Auto | ✅ Auto | ✅ Auto | ⚠️ Manual |
|
||||
| **Setup Complexity** | 🟢 Simple | 🟡 Medium | 🔴 Complex | 🟡 Medium | 🟢 Simple |
|
||||
| **Dependencies** | None | ChromaDB | Elasticsearch | Qdrant | None |
|
||||
| **Filtering** | ❌ Basic | ✅ Metadata | ✅ Advanced | ✅ Advanced | ❌ Basic |
|
||||
| **Concurrency** | ❌ Limited | ✅ Good | ✅ Excellent | ✅ Excellent | ❌ Single Process |
|
||||
| **Async Support** | ❌ No | ❌ No | ❌ No | ✅ Native | ❌ No |
|
||||
| **Best For** | Development | Local Apps | Production | Production/Cloud | Testing |
|
||||
FlowLLM provides multiple Vector Store implementations tailored to different use cases:
|
||||
|
||||
## ⚙️ Configuration in default.yaml
|
||||
- **LocalVectorStore** ([source code](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/local_vector_store.py)): A file-based local implementation that persists data in JSONL format. Suitable for single-machine deployments and small-scale datasets.
|
||||
- **MemoryVectorStore** ([source code](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/memory_vector_store.py)): An in-memory implementation offering fast access speeds. Ideal for temporary data or testing scenarios.
|
||||
- **QdrantVectorStore** ([source code](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/qdrant_vector_store.py)): Built on the Qdrant vector database, supporting high-performance vector search. Recommended for large-scale production environments.
|
||||
- **ChromaVectorStore** ([source code](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/chroma_vector_store.py)): Based on ChromaDB, providing persistent storage and metadata filtering capabilities.
|
||||
- **EsVectorStore** ([source code](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/es_vector_store.py)): Built on Elasticsearch, enabling powerful combined full-text and vector search functionalities.
|
||||
|
||||
All vector stores are configured in the `vector_store` section of `reme_ai/config/default.yaml`. The configuration structure is:
|
||||
All Vector Store implementations inherit from **BaseVectorStore** ([source code](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/base_vector_store.py)), ensuring a consistent interface specification.
|
||||
|
||||
## Core Features
|
||||
|
||||
### Workspace Management
|
||||
|
||||
- **Create Workspace**: Create a new workspace for storing vector nodes.
|
||||
- **Delete Workspace**: Remove a workspace along with all its data.
|
||||
- **Check Workspace Existence**: Verify whether a specified workspace exists.
|
||||
- **List Workspaces**: Retrieve a list of all existing workspaces.
|
||||
- **Copy Workspace**: Duplicate data from one workspace to another.
|
||||
|
||||
### Node Operations
|
||||
|
||||
- **Insert Nodes**: Insert vector nodes into a workspace, supporting single or batch insertion with automatic vector embedding generation.
|
||||
- **Delete Nodes**: Remove specific nodes by their IDs.
|
||||
- **Iterate Nodes**: Traverse all nodes within a workspace.
|
||||
|
||||
### Vector Search
|
||||
|
||||
- **Similarity Search**: Perform vector similarity searches based on text queries, returning the top-K most similar results.
|
||||
- **Metadata Filtering**: Apply filtering conditions based on metadata, including exact matches and range queries.
|
||||
- **Similarity Scores**: Search results include similarity scores to evaluate match quality.
|
||||
|
||||
### Data Import/Export
|
||||
|
||||
- **Export Workspace**: Export workspace data to a file or specified path.
|
||||
- **Import Workspace**: Import data into a workspace from a file or a list of nodes.
|
||||
- **Callback Functions**: Support callback functions during import/export for data transformation.
|
||||
|
||||
## Synchronous and Asynchronous Interfaces
|
||||
|
||||
All Vector Store implementations provide both synchronous and asynchronous interfaces:
|
||||
|
||||
- **Synchronous Interface**: Direct method calls suitable for synchronous code environments.
|
||||
- **Asynchronous Interface**: Prefixed with `async_`, designed for asynchronous environments and offering better concurrency performance.
|
||||
|
||||
The asynchronous interface is particularly useful in the following scenarios:
|
||||
- Using asynchronous embedding models for vector generation.
|
||||
- Performing batch operations in high-concurrency environments.
|
||||
- Integrating with other asynchronous components.
|
||||
|
||||
## Configuration Options
|
||||
|
||||
### General Configuration
|
||||
|
||||
- **embedding_model**: Instance of the embedding model used to generate vector embeddings.
|
||||
- **batch_size**: Batch size for bulk operations (default: 1024).
|
||||
|
||||
### LocalVectorStore Configuration
|
||||
|
||||
- **store_dir**: Storage directory path (default: `./local_vector_store`).
|
||||
|
||||
### MemoryVectorStore Configuration
|
||||
|
||||
- **store_dir**: Persistence directory (default: `./memory_vector_store`).
|
||||
|
||||
### QdrantVectorStore Configuration
|
||||
|
||||
- **url**: Qdrant service URL (optional; used for Qdrant Cloud or custom deployments).
|
||||
- **host**: Qdrant server host (default: `localhost`).
|
||||
- **port**: Qdrant server port (default: `6333`).
|
||||
- **api_key**: API key for Qdrant Cloud authentication.
|
||||
- **distance**: Distance metric—supports COSINE, EUCLIDEAN, DOT (default: COSINE).
|
||||
|
||||
### ChromaVectorStore Configuration
|
||||
|
||||
- **store_dir**: ChromaDB data storage directory (default: `./chroma_vector_store`).
|
||||
|
||||
### EsVectorStore Configuration
|
||||
|
||||
- **hosts**: Elasticsearch host address(es), either a string or a list (default: `http://localhost:9200`).
|
||||
- **basic_auth**: Basic authentication credentials (username and password).
|
||||
|
||||
## Configuration File Examples
|
||||
|
||||
Configure Vector Store in `flowllm/config/default.yaml` under the `vector_store` section. The basic structure is as follows:
|
||||
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: <backend_name> # Required: local, chroma, elasticsearch, qdrant, or memory
|
||||
embedding_model: default # Required: Name of the embedding model configuration
|
||||
params: # Optional: Backend-specific parameters
|
||||
# Backend-specific parameters go here
|
||||
backend: <backend_name> # Required: vector store backend type
|
||||
embedding_model: default # Required: name of embedding model config
|
||||
params: # Optional: backend-specific parameters
|
||||
# Backend-specific parameters
|
||||
```
|
||||
|
||||
### Configuration Fields
|
||||
### Configuration Field Descriptions
|
||||
|
||||
- **`backend`** (required): The vector store backend to use. Valid values: `local`, `chroma`, `elasticsearch`, `qdrant`, `memory`
|
||||
- **`embedding_model`** (required): The name of the embedding model configuration from the `embedding_model` section
|
||||
- **`params`** (optional): A dictionary of backend-specific parameters that will be passed to the vector store constructor
|
||||
- **`backend`** (required): Vector store backend type. Options: `local`, `memory`, `chroma`, `qdrant`, `elasticsearch`.
|
||||
- **`embedding_model`** (required): Name of the embedding model configuration, referencing the `embedding_model` section.
|
||||
- **`params`** (optional): Dictionary of backend-specific parameters passed to the vector store constructor.
|
||||
|
||||
## 📁 Vector Store Backend Configurations
|
||||
### Configuration Examples by Type
|
||||
|
||||
### 1. LocalVectorStore (`backend=local`)
|
||||
#### 1. LocalVectorStore Configuration
|
||||
|
||||
A simple file-based vector store that saves data to local JSONL files.
|
||||
Simplest local file-based storage, ideal for development and testing.
|
||||
|
||||
#### 💡 When to Use
|
||||
- **Development and testing** - No external dependencies required 🛠️
|
||||
- **Small datasets** - Suitable for datasets with < 10,000 vectors 📊
|
||||
- **Single-user applications** - Limited concurrent access support 👤
|
||||
|
||||
#### ⚙️ Configuration
|
||||
**Implementation**: [`flowllm/core/vector_store/local_vector_store.py`](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/local_vector_store.py)
|
||||
|
||||
```yaml
|
||||
vector_store:
|
||||
|
|
@ -81,25 +141,29 @@ vector_store:
|
|||
backend: local
|
||||
embedding_model: default
|
||||
params:
|
||||
store_dir: "./local_vector_store" # Directory to store JSONL files (default: "./local_vector_store")
|
||||
batch_size: 1024 # Batch size for operations (default: 1024)
|
||||
store_dir: "./local_vector_store" # Storage directory (optional; default: "./local_vector_store")
|
||||
```
|
||||
|
||||
#### Configuration Parameters
|
||||
#### 2. MemoryVectorStore Configuration
|
||||
|
||||
- **`store_dir`** (optional): Directory path where workspace files are stored. Default: `"./local_vector_store"`
|
||||
- **`batch_size`** (optional): Batch size for bulk operations. Default: `1024`
|
||||
In-memory storage with fast access, suitable for temporary data or testing.
|
||||
|
||||
### 2. ChromaVectorStore (`backend=chroma`)
|
||||
**Implementation**: [`flowllm/core/vector_store/memory_vector_store.py`](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/memory_vector_store.py)
|
||||
|
||||
An embedded vector database that provides persistent storage with advanced features.
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: memory
|
||||
embedding_model: default
|
||||
params:
|
||||
store_dir: "./memory_vector_store" # Persistence directory (optional; default: "./memory_vector_store")
|
||||
```
|
||||
|
||||
#### 💡 When to Use
|
||||
- **Local development** with persistence requirements 🏠
|
||||
- **Medium-scale applications** (10K - 1M vectors) 📈
|
||||
- **Applications requiring metadata filtering** 🔍
|
||||
#### 3. ChromaVectorStore Configuration
|
||||
|
||||
#### ⚙️ Configuration
|
||||
Persistent storage based on ChromaDB with metadata filtering support.
|
||||
|
||||
**Implementation**: [`flowllm/core/vector_store/chroma_vector_store.py`](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/chroma_vector_store.py)
|
||||
|
||||
```yaml
|
||||
vector_store:
|
||||
|
|
@ -107,48 +171,44 @@ vector_store:
|
|||
backend: chroma
|
||||
embedding_model: default
|
||||
params:
|
||||
store_dir: "./chroma_vector_store" # Directory for Chroma database (default: "./chroma_vector_store")
|
||||
batch_size: 1024 # Batch size for operations (default: 1024)
|
||||
store_dir: "./chroma_vector_store" # ChromaDB data directory (optional; default: "./chroma_vector_store")
|
||||
```
|
||||
|
||||
#### Configuration Parameters
|
||||
#### 4. QdrantVectorStore Configuration
|
||||
|
||||
- **`store_dir`** (optional): Directory path where ChromaDB data is persisted. Default: `"./chroma_vector_store"`
|
||||
- **`batch_size`** (optional): Batch size for bulk operations. Default: `1024`
|
||||
**Implementation**: [`flowllm/core/vector_store/qdrant_vector_store.py`](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/qdrant_vector_store.py)
|
||||
|
||||
### 3. EsVectorStore (`backend=elasticsearch`)
|
||||
**Local Qdrant Instance**:
|
||||
|
||||
Production-grade vector search using Elasticsearch with advanced filtering and scaling capabilities.
|
||||
|
||||
#### 💡 When to Use
|
||||
- **Production environments** requiring high availability 🏭
|
||||
- **Large-scale applications** (1M+ vectors) 🚀
|
||||
- **Complex filtering requirements** on metadata 🎯
|
||||
|
||||
#### 🛠️ Setup Elasticsearch
|
||||
|
||||
Before using EsVectorStore, set up Elasticsearch:
|
||||
|
||||
##### Option 1: Docker Run
|
||||
```bash
|
||||
# Pull the latest Elasticsearch image
|
||||
docker pull docker.elastic.co/elasticsearch/elasticsearch-wolfi:9.0.0
|
||||
|
||||
# Run Elasticsearch container
|
||||
docker run -p 9200:9200 \
|
||||
-e "discovery.type=single-node" \
|
||||
-e "xpack.security.enabled=false" \
|
||||
-e "xpack.license.self_generated.type=trial" \
|
||||
-e "http.host=0.0.0.0" \
|
||||
docker.elastic.co/elasticsearch/elasticsearch-wolfi:9.0.0
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: qdrant
|
||||
embedding_model: default
|
||||
params:
|
||||
host: "localhost" # Qdrant server host (optional; default: localhost)
|
||||
port: 6333 # Qdrant server port (optional; default: 6333)
|
||||
distance: "COSINE" # Distance metric (optional; default: COSINE; options: COSINE, EUCLIDEAN, DOT)
|
||||
```
|
||||
|
||||
##### Environment Configuration
|
||||
```bash
|
||||
export FLOW_ES_HOSTS=http://localhost:9200
|
||||
**Qdrant Cloud Configuration**:
|
||||
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: qdrant
|
||||
embedding_model: default
|
||||
params:
|
||||
url: "https://your-cluster.qdrant.io:6333" # Qdrant Cloud URL
|
||||
api_key: "your-api-key-here" # API key
|
||||
distance: "COSINE"
|
||||
```
|
||||
|
||||
#### ⚙️ Configuration
|
||||
#### 5. EsVectorStore Configuration
|
||||
|
||||
**Implementation**: [`flowllm/core/vector_store/es_vector_store.py`](https://github.com/flowllm-ai/flowllm/blob/main/flowllm/core/vector_store/es_vector_store.py)
|
||||
|
||||
**Basic Configuration (Local Elasticsearch)**:
|
||||
|
||||
```yaml
|
||||
vector_store:
|
||||
|
|
@ -156,167 +216,11 @@ vector_store:
|
|||
backend: elasticsearch
|
||||
embedding_model: default
|
||||
params:
|
||||
hosts: "http://localhost:9200" # Elasticsearch host(s) - can be string or list (default: from FLOW_ES_HOSTS env var or "http://localhost:9200")
|
||||
basic_auth: null # Optional: ("username", "password") tuple for authentication
|
||||
batch_size: 1024 # Batch size for bulk operations (default: 1024)
|
||||
hosts: "http://localhost:9200" # Elasticsearch host(s) (optional; default: http://localhost:9200)
|
||||
```
|
||||
|
||||
#### Configuration Parameters
|
||||
**Configuration with Authentication**:
|
||||
|
||||
- **`hosts`** (optional): Elasticsearch host(s) as a string or list of strings. Defaults to the `FLOW_ES_HOSTS` environment variable or `"http://localhost:9200"` if not set
|
||||
- **`basic_auth`** (optional): Tuple of `("username", "password")` for basic authentication. Default: `null` (no authentication)
|
||||
- **`batch_size`** (optional): Batch size for bulk operations. Default: `1024`
|
||||
|
||||
### 4. QdrantVectorStore (`backend=qdrant`)
|
||||
|
||||
A high-performance vector database designed for production workloads with native async support and advanced filtering.
|
||||
|
||||
#### 💡 When to Use
|
||||
- **Production environments** requiring high performance and reliability 🏭
|
||||
- **Large-scale applications** (10M+ vectors) with excellent horizontal scaling 🚀
|
||||
- **Applications requiring native async operations** for better concurrency ⚡
|
||||
- **Complex filtering and metadata queries** on large datasets 🎯
|
||||
- **Cloud-native deployments** with Qdrant Cloud support ☁️
|
||||
|
||||
#### 🛠️ Setup Qdrant
|
||||
|
||||
Before using QdrantVectorStore, set up Qdrant:
|
||||
|
||||
##### Option 1: Docker Run (Recommended for Development)
|
||||
```bash
|
||||
# Pull the latest Qdrant image
|
||||
docker pull qdrant/qdrant
|
||||
|
||||
# Run Qdrant container
|
||||
docker run -p 6333:6333 -p 6334:6334 \
|
||||
-v $(pwd)/qdrant_storage:/qdrant/storage:z \
|
||||
qdrant/qdrant
|
||||
```
|
||||
|
||||
##### Option 2: Qdrant Cloud
|
||||
For production, you can use [Qdrant Cloud](https://cloud.qdrant.io/) for managed hosting.
|
||||
|
||||
##### Environment Configuration
|
||||
```bash
|
||||
# For local setup
|
||||
export FLOW_QDRANT_HOST=localhost
|
||||
export FLOW_QDRANT_PORT=6333
|
||||
|
||||
# For cloud setup (optional)
|
||||
export FLOW_QDRANT_API_KEY=your-api-key
|
||||
```
|
||||
|
||||
#### ⚙️ Configuration
|
||||
|
||||
##### Local Qdrant Instance
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: qdrant
|
||||
embedding_model: default
|
||||
params:
|
||||
host: "localhost" # Qdrant host (default: from FLOW_QDRANT_HOST env var or "localhost")
|
||||
port: 6333 # Qdrant port (default: from FLOW_QDRANT_PORT env var or 6333)
|
||||
batch_size: 1024 # Batch size for operations (default: 1024)
|
||||
distance: "COSINE" # Distance metric: "COSINE", "EUCLIDEAN", or "DOT" (default: "COSINE")
|
||||
```
|
||||
|
||||
##### Qdrant Cloud or Remote Server
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: qdrant
|
||||
embedding_model: default
|
||||
params:
|
||||
url: "https://your-cluster.qdrant.io:6333" # Qdrant server URL (if provided, host and port are ignored)
|
||||
api_key: "your-api-key" # API key for Qdrant Cloud authentication
|
||||
batch_size: 1024 # Batch size for operations (default: 1024)
|
||||
distance: "COSINE" # Distance metric (default: "COSINE")
|
||||
```
|
||||
|
||||
#### Configuration Parameters
|
||||
|
||||
- **`url`** (optional): Complete URL for connecting to Qdrant. If provided, `host` and `port` are ignored. Useful for Qdrant Cloud or custom deployments
|
||||
- **`host`** (optional): Host address of the Qdrant server. Defaults to the `FLOW_QDRANT_HOST` environment variable or `"localhost"` if not set
|
||||
- **`port`** (optional): Port number of the Qdrant server. Defaults to the `FLOW_QDRANT_PORT` environment variable or `6333` if not set
|
||||
- **`api_key`** (optional): API key for authentication (required for Qdrant Cloud). Can also be set via `FLOW_QDRANT_API_KEY` environment variable
|
||||
- **`distance`** (optional): Distance metric for vector similarity. Valid values: `"COSINE"`, `"EUCLIDEAN"`, `"DOT"`. Default: `"COSINE"`
|
||||
- **`batch_size`** (optional): Batch size for bulk operations. Default: `1024`
|
||||
|
||||
#### 🌟 Key Features
|
||||
|
||||
- **Native Async Support** - All operations have async equivalents for better concurrency
|
||||
- **Upsert Operations** - Insert automatically updates existing nodes with the same ID
|
||||
- **Advanced Filtering** - Support for term and range filters on metadata
|
||||
- **High Performance** - Optimized for large-scale vector similarity search
|
||||
- **Horizontal Scaling** - Supports clustering for distributed deployments
|
||||
- **Multiple Distance Metrics** - Cosine, Euclidean, and Dot Product similarity
|
||||
- **Persistent Storage** - Data is automatically persisted to disk
|
||||
- **Efficient Iteration** - Scroll through large collections with pagination
|
||||
|
||||
### 5. MemoryVectorStore (`backend=memory`)
|
||||
|
||||
An ultra-fast in-memory vector store that keeps all data in RAM for maximum performance.
|
||||
|
||||
#### 💡 When to Use
|
||||
- **Testing and development** - Fastest possible operations for unit tests 🧪
|
||||
- **Small to medium datasets** that fit in memory (< 1M vectors) 💾
|
||||
- **Applications requiring ultra-low latency** search operations ⚡
|
||||
- **Temporary workspaces** that don't need persistence 🚀
|
||||
|
||||
#### ⚙️ Configuration
|
||||
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: memory
|
||||
embedding_model: default
|
||||
params:
|
||||
store_dir: "./memory_vector_store" # Directory for backup/restore operations (default: "./memory_vector_store")
|
||||
batch_size: 1024 # Batch size for operations (default: 1024)
|
||||
```
|
||||
|
||||
#### Configuration Parameters
|
||||
|
||||
- **`store_dir`** (optional): Directory path for backup/restore operations. Default: `"./memory_vector_store"`
|
||||
- **`batch_size`** (optional): Batch size for bulk operations. Default: `1024`
|
||||
|
||||
#### ⚡ Performance Benefits
|
||||
|
||||
- **Zero I/O latency** - All operations happen in RAM
|
||||
- **Instant search results** - No disk or network overhead
|
||||
- **Perfect for testing** - Fast setup and teardown
|
||||
- **Memory efficient** - Only stores what you need
|
||||
|
||||
#### 🚨 Important Notes
|
||||
|
||||
- **Data is volatile** - Lost when process ends unless explicitly saved
|
||||
- **Memory usage** - Entire dataset must fit in available RAM
|
||||
- **No persistence** - Use `dump_workspace()` to save to disk
|
||||
- **Single process** - Not suitable for distributed applications
|
||||
|
||||
## 📝 Example Configurations
|
||||
|
||||
### Minimal Configuration (Memory Store)
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: memory
|
||||
embedding_model: default
|
||||
```
|
||||
|
||||
### Local File Storage
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: local
|
||||
embedding_model: default
|
||||
params:
|
||||
store_dir: "./my_vector_store"
|
||||
batch_size: 2048
|
||||
```
|
||||
|
||||
### Elasticsearch Production Setup
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
|
|
@ -324,40 +228,29 @@ vector_store:
|
|||
embedding_model: default
|
||||
params:
|
||||
hosts: "http://elasticsearch.example.com:9200"
|
||||
basic_auth: ["username", "password"]
|
||||
batch_size: 2048
|
||||
basic_auth: ["username", "password"] # Basic auth credentials
|
||||
```
|
||||
|
||||
### Qdrant Cloud Setup
|
||||
**Multi-Host Configuration**:
|
||||
|
||||
```yaml
|
||||
vector_store:
|
||||
default:
|
||||
backend: qdrant
|
||||
backend: elasticsearch
|
||||
embedding_model: default
|
||||
params:
|
||||
url: "https://your-cluster.qdrant.io:6333"
|
||||
api_key: "your-api-key-here"
|
||||
distance: "COSINE"
|
||||
batch_size: 1024
|
||||
hosts:
|
||||
- "http://es-node1:9200"
|
||||
- "http://es-node2:9200"
|
||||
- "http://es-node3:9200"
|
||||
```
|
||||
|
||||
## 🔄 Environment Variables
|
||||
### Complete Configuration Example
|
||||
|
||||
Some vector store backends support environment variables for configuration:
|
||||
|
||||
- **Elasticsearch**: `FLOW_ES_HOSTS` - Elasticsearch host(s)
|
||||
- **Qdrant**:
|
||||
- `FLOW_QDRANT_HOST` - Qdrant host (default: "localhost")
|
||||
- `FLOW_QDRANT_PORT` - Qdrant port (default: 6333)
|
||||
- `FLOW_QDRANT_API_KEY` - Qdrant API key for authentication
|
||||
|
||||
Environment variables are used as fallbacks when parameters are not explicitly set in the YAML configuration.
|
||||
|
||||
## 🧩 Integration with Embedding Models
|
||||
|
||||
All vector stores require an embedding model configuration. The `embedding_model` field in the vector store configuration references a model defined in the `embedding_model` section of `default.yaml`:
|
||||
Below is a complete `default.yaml` example including both embedding model and vector store configurations:
|
||||
|
||||
```yaml
|
||||
# Embedding model configuration
|
||||
embedding_model:
|
||||
default:
|
||||
backend: openai_compatible
|
||||
|
|
@ -365,10 +258,46 @@ embedding_model:
|
|||
params:
|
||||
dimensions: 1024
|
||||
|
||||
# Vector store configuration
|
||||
vector_store:
|
||||
default:
|
||||
backend: memory
|
||||
embedding_model: default # References the embedding_model.default configuration
|
||||
backend: elasticsearch
|
||||
embedding_model: default
|
||||
params:
|
||||
hosts: "http://localhost:9200"
|
||||
```
|
||||
|
||||
The embedding model configuration provides the model name, backend, and parameters needed for generating vector embeddings.
|
||||
### Environment Variable Support
|
||||
|
||||
Certain Vector Stores support environment variables as a supplement to YAML configuration:
|
||||
|
||||
- **Elasticsearch**: `FLOW_ES_HOSTS` – Elasticsearch host address.
|
||||
- **Qdrant**:
|
||||
- `FLOW_QDRANT_HOST` – Qdrant host (default: `localhost`)
|
||||
- `FLOW_QDRANT_PORT` – Qdrant port (default: `6333`)
|
||||
- `FLOW_QDRANT_API_KEY` – Qdrant API key
|
||||
|
||||
When parameters are not explicitly specified in the YAML configuration, the system falls back to environment variables.
|
||||
|
||||
## Metadata Filtering
|
||||
|
||||
Two types of metadata filtering are supported:
|
||||
|
||||
- **Exact Match**: Specify field values for exact matching.
|
||||
- **Range Queries**: Use operators `gte`, `lte`, `gt`, `lt` for numeric range queries.
|
||||
- **Nested Fields**: Access nested metadata fields using dot notation.
|
||||
|
||||
## Usage Recommendations
|
||||
|
||||
- **Development & Testing**: Use MemoryVectorStore or LocalVectorStore—no additional services required.
|
||||
- **Small-Scale Applications**: Use LocalVectorStore or ChromaVectorStore for simplicity and ease of use.
|
||||
- **Production Environments**: Use QdrantVectorStore or EsVectorStore for high performance and scalability.
|
||||
- **Hybrid Search**: Use EsVectorStore to combine vector search with full-text search capabilities.
|
||||
|
||||
## Important Notes
|
||||
|
||||
- Ensure the embedding model’s output dimension matches the Vector Store configuration.
|
||||
- For large-scale data, use professional vector databases (e.g., Qdrant, Elasticsearch).
|
||||
- Asynchronous interfaces deliver better performance in asynchronous environments.
|
||||
- Regularly back up critical data, especially when using in-memory storage.
|
||||
- Choose an appropriate batch size based on your data scale to optimize performance.
|
||||
|
|
|
|||
|
|
@ -8,7 +8,6 @@ os.environ["FLOW_APP_NAME"] = "ReMe"
|
|||
from . import agent # noqa: E402
|
||||
from . import config # noqa: E402
|
||||
from . import constants # noqa: E402
|
||||
from . import context # noqa: E402
|
||||
from . import enumeration # noqa: E402
|
||||
from . import retrieve # noqa: E402
|
||||
from . import schema # noqa: E402
|
||||
|
|
@ -22,7 +21,6 @@ __all__ = [
|
|||
"agent",
|
||||
"config",
|
||||
"constants",
|
||||
"context",
|
||||
"enumeration",
|
||||
"retrieve",
|
||||
"schema",
|
||||
|
|
|
|||
|
|
@ -11,6 +11,19 @@ language model reasoning with tool execution. The agent iteratively:
|
|||
|
||||
The agent is specifically designed for RAG (Retrieval-Augmented Generation) workflows,
|
||||
providing context management capabilities to handle long conversations efficiently.
|
||||
|
||||
Context management is controlled via ``working_summary_mode`` and
|
||||
``compact_ratio_threshold`` parameters, which are forwarded to
|
||||
``MessageOffloadOp``. ``working_summary_mode`` selects between:
|
||||
- ``compact`` – only compact verbose tool messages by storing full content externally
|
||||
and keeping short previews in the context.
|
||||
- ``compress`` – only apply LLM-based compression to generate a compact state snapshot.
|
||||
- ``auto`` – first run compaction, then optionally run compression if the
|
||||
compaction ratio is not sufficient (default).
|
||||
|
||||
``compact_ratio_threshold`` is only used in ``auto`` mode and defines the compaction
|
||||
ratio (tokens after compaction divided by original tokens) above which an additional
|
||||
LLM-based compression pass is applied. It defaults to ``0.75``.
|
||||
"""
|
||||
|
||||
from typing import Dict, List
|
||||
|
|
@ -36,6 +49,12 @@ class AgenticRetrieveOp(BaseAsyncToolOp):
|
|||
through compaction (storing large tool messages externally) and compression
|
||||
(LLM-based summarization of message history).
|
||||
|
||||
Context management behavior is configured via ``working_summary_mode`` and
|
||||
``compact_ratio_threshold`` (see module docstring for details). These options are
|
||||
passed to ``MessageOffloadOp`` to control whether the agent only compacts tool
|
||||
messages, only compresses history, or applies an automatic compaction-then-
|
||||
compression pipeline.
|
||||
|
||||
Available tools:
|
||||
- GrepOp: Search for patterns in files
|
||||
- ReadFileOp: Read file contents
|
||||
|
|
@ -49,7 +68,7 @@ class AgenticRetrieveOp(BaseAsyncToolOp):
|
|||
def __init__(
|
||||
self,
|
||||
llm: str = "qwen3_30b_instruct",
|
||||
max_steps: int = 5,
|
||||
max_steps: int = 20,
|
||||
**kwargs,
|
||||
):
|
||||
"""
|
||||
|
|
@ -87,14 +106,21 @@ class AgenticRetrieveOp(BaseAsyncToolOp):
|
|||
"description": "messages",
|
||||
"required": True,
|
||||
},
|
||||
"context_manage_mode": {
|
||||
"working_summary_mode": {
|
||||
"type": "string",
|
||||
"description": "Context management mode: 'compact' (only compacts tool messages), 'compress' "
|
||||
"(only LLM-based compression), 'auto' (compaction first then compression if "
|
||||
"needed). Defaults to 'auto'.",
|
||||
"required": True,
|
||||
"description": "summary strategy: 'compact' only compacts large tool messages, 'compress' "
|
||||
"only applies LLM-based compression, 'auto' first compacts then optionally compresses when "
|
||||
"reduction is insufficient. Defaults to 'auto'.",
|
||||
"required": False,
|
||||
"enum": ["compact", "compress", "auto"],
|
||||
},
|
||||
"compact_ratio_threshold": {
|
||||
"type": "number",
|
||||
"description": "Only used in 'auto' mode. Threshold for compaction (tokens after compaction "
|
||||
"divided by original tokens). When the ratio is greater than this value, an additional "
|
||||
"LLM-based compression pass is triggered. Defaults to 0.75.",
|
||||
"required": False,
|
||||
},
|
||||
"max_total_tokens": {
|
||||
"type": "integer",
|
||||
"description": "Maximum token threshold for triggering compression/compaction. For compaction "
|
||||
|
|
@ -156,9 +182,8 @@ class AgenticRetrieveOp(BaseAsyncToolOp):
|
|||
Each iteration is called a "round" and represents one reasoning-action cycle.
|
||||
"""
|
||||
# Import tool operators that the agent can use
|
||||
from reme_ai.context.file_tool import GrepOp, ReadFileOp
|
||||
from reme_ai.context.offload import ContextOffloadOp
|
||||
from reme_ai.context.file_tool import BatchWriteFileOp
|
||||
from reme_ai.retrieve.working import GrepOp, ReadFileOp, BatchWriteFileOp
|
||||
from reme_ai.summary.working import MessageOffloadOp
|
||||
|
||||
# Initialize available tools for the agent
|
||||
# GrepOp: Search for patterns/text in files (useful for code search)
|
||||
|
|
@ -184,9 +209,9 @@ class AgenticRetrieveOp(BaseAsyncToolOp):
|
|||
# Main ReAct loop: iterate up to max_steps times
|
||||
for i in range(self.max_steps):
|
||||
# Step 1: Context Management Phase
|
||||
# Create a pipeline: ContextOffloadOp (compacts/compresses) -> BatchWriteFileOp (saves offloaded content)
|
||||
# Create a pipeline: MessageOffloadOp (compacts/compresses) -> BatchWriteFileOp (saves offloaded content)
|
||||
# The >> operator chains these operations together
|
||||
op = ContextOffloadOp() >> BatchWriteFileOp()
|
||||
op = MessageOffloadOp() >> BatchWriteFileOp()
|
||||
|
||||
# Apply context management to current message history
|
||||
# This may compact large tool messages or compress old messages based on context_manage_mode
|
||||
|
|
@ -194,6 +219,7 @@ class AgenticRetrieveOp(BaseAsyncToolOp):
|
|||
|
||||
# Update messages with the processed/optimized version from context management
|
||||
# Large messages may now reference external files instead of containing full content
|
||||
logger.info(f"round{i + 1}.offload={op.context.response.answer}")
|
||||
messages = [Message(**x) for x in op.context.response.answer]
|
||||
|
||||
# Step 2: Reasoning Phase
|
||||
|
|
@ -265,4 +291,4 @@ class AgenticRetrieveOp(BaseAsyncToolOp):
|
|||
|
||||
# Store the complete conversation history in the context response
|
||||
# This includes all reasoning steps, tool calls, and tool results
|
||||
self.context.response.answer = messages
|
||||
self.context.response.metadata["messages"] = [x.simple_dump(add_reasoning=True) for x in messages]
|
||||
|
|
|
|||
|
|
@ -159,19 +159,23 @@ flow:
|
|||
agentic_retrieve:
|
||||
flow_content: AgenticRetrieveOp()
|
||||
|
||||
context_offload:
|
||||
flow_content: ContextOffloadOp() >> BatchWriteFileOp()
|
||||
summary_working_memory:
|
||||
flow_content: MessageOffloadOp() >> BatchWriteFileOp()
|
||||
description: "Manages context window limits by compacting tool messages and compressing conversation history. First compacts large tool messages by storing full content in external files, then applies LLM-based compression if compaction ratio exceeds threshold. This helps reduce token usage while preserving important information."
|
||||
input_schema:
|
||||
messages:
|
||||
type: array
|
||||
description: "List of conversation messages to process for context offloading"
|
||||
required: true
|
||||
context_manage_mode:
|
||||
working_summary_mode:
|
||||
type: string
|
||||
description: "Context management mode: 'compact' only applies compaction to tool messages, 'compress' only applies LLM-based compression, 'auto' applies compaction first then compression if compaction ratio exceeds threshold. Defaults to 'auto'."
|
||||
description: "Working summary strategy: 'compact' only compacts large tool messages, 'compress' only applies LLM-based compression, 'auto' first compacts then optionally compresses when reduction is insufficient. Defaults to 'auto'."
|
||||
required: false
|
||||
enum: ["compact", "compress", "auto"]
|
||||
compact_ratio_threshold:
|
||||
type: number
|
||||
description: "Only used in 'auto' mode. Threshold for compaction ratio (tokens after compaction divided by original tokens). When the ratio is greater than this value, an additional LLM-based compression pass is triggered. Defaults to 0.75."
|
||||
required: false
|
||||
max_total_tokens:
|
||||
type: integer
|
||||
description: "Maximum token count threshold for triggering compression/compaction. For compaction, this is the total token count threshold. For compression, this excludes keep_recent_count messages and system messages. Defaults to 20000."
|
||||
|
|
@ -197,19 +201,29 @@ flow:
|
|||
description: "Unique identifier for the chat session, used for file naming when storing compressed message groups. If not provided, a UUID will be generated automatically."
|
||||
required: false
|
||||
|
||||
context_offload_for_agentscope:
|
||||
flow_content: ContextOffloadOp()
|
||||
grep_working_memory:
|
||||
flow_content: GrepOp()
|
||||
|
||||
read_working_memory:
|
||||
flow_content: ReadFileOp()
|
||||
|
||||
summary_working_memory_for_agentscope:
|
||||
flow_content: MessageOffloadOp()
|
||||
description: "Context offload operation for AgentScope integration. Manages context window limits by compacting tool messages and compressing conversation history without batch file writing. Same functionality as context_offload but without the BatchWriteFileOp step."
|
||||
input_schema:
|
||||
messages:
|
||||
type: array
|
||||
description: "List of conversation messages to process for context offloading"
|
||||
required: true
|
||||
context_manage_mode:
|
||||
working_summary_mode:
|
||||
type: string
|
||||
description: "Context management mode: 'compact' only applies compaction to tool messages, 'compress' only applies LLM-based compression, 'auto' applies compaction first then compression if compaction ratio exceeds threshold. Defaults to 'auto'."
|
||||
required: true
|
||||
description: "Working summary strategy: 'compact' only compacts large tool messages, 'compress' only applies LLM-based compression, 'auto' first compacts then optionally compresses when reduction is insufficient. Defaults to 'auto'."
|
||||
required: false
|
||||
enum: ["compact", "compress", "auto"]
|
||||
compact_ratio_threshold:
|
||||
type: number
|
||||
description: "Only used in 'auto' mode. Threshold for compaction ratio (tokens after compaction divided by original tokens). When the ratio is greater than this value, an additional LLM-based compression pass is triggered. Defaults to 0.75."
|
||||
required: false
|
||||
max_total_tokens:
|
||||
type: integer
|
||||
description: "Maximum token count threshold for triggering compression/compaction. For compaction, this is the total token count threshold. For compression, this excludes keep_recent_count messages and system messages. Defaults to 20000."
|
||||
|
|
@ -245,6 +259,33 @@ llm:
|
|||
token_count: # Optional
|
||||
backend: base
|
||||
|
||||
qwen3_coder_plus:
|
||||
backend: openai_compatible
|
||||
model_name: qwen3-coder-plus
|
||||
token_count: # Optional
|
||||
model_name: Qwen/Qwen3-Coder-480B-A35B-Instruct
|
||||
backend: hf
|
||||
params:
|
||||
use_mirror: true
|
||||
|
||||
qwen3_coder_480b_instruct:
|
||||
backend: openai_compatible
|
||||
model_name: qwen3-coder-480b-a35b-instruct
|
||||
token_count: # Optional
|
||||
model_name: Qwen/Qwen3-Coder-480B-A35B-Instruct
|
||||
backend: hf
|
||||
params:
|
||||
use_mirror: true
|
||||
|
||||
qwen3_coder_30b_instruct:
|
||||
backend: openai_compatible
|
||||
model_name: qwen3-coder-30b-a3b-instruct
|
||||
token_count: # Optional
|
||||
model_name: Qwen/Qwen3-Coder-30B-A3B-Instruct
|
||||
backend: hf
|
||||
params:
|
||||
use_mirror: true
|
||||
|
||||
qwen3_30b_instruct:
|
||||
backend: openai_compatible
|
||||
model_name: qwen3-30b-a3b-instruct-2507
|
||||
|
|
|
|||
|
|
@ -1,14 +0,0 @@
|
|||
"""Context management module for ReMe framework.
|
||||
|
||||
This module provides submodules for different types of context management operations:
|
||||
- file_tool: File-related operations for reading, writing, and searching files
|
||||
- offload: Context offload operations for reducing token usage and managing context windows
|
||||
"""
|
||||
|
||||
from . import file_tool
|
||||
from . import offload
|
||||
|
||||
__all__ = [
|
||||
"file_tool",
|
||||
"offload",
|
||||
]
|
||||
|
|
@ -1,52 +0,0 @@
|
|||
"""Grep text search operation module.
|
||||
|
||||
This module provides a tool operation for searching text patterns in files.
|
||||
It enables efficient content-based search using regular expressions, with support
|
||||
for glob pattern filtering and result limiting.
|
||||
"""
|
||||
|
||||
from flowllm.core.context import C
|
||||
from flowllm.core.schema import ToolCall
|
||||
from flowllm.extensions.file_tool import GrepOp as FlowGrepOp
|
||||
|
||||
|
||||
@C.register_op()
|
||||
class GrepOp(FlowGrepOp):
|
||||
"""Grep text search operation.
|
||||
|
||||
This operation searches for text patterns in files using regular expressions.
|
||||
Supports glob pattern filtering and result limiting.
|
||||
"""
|
||||
|
||||
file_path = __file__
|
||||
|
||||
def build_tool_call(self) -> ToolCall:
|
||||
"""Build and return the tool call schema for this operator."""
|
||||
tool_params = {
|
||||
"name": "Grep",
|
||||
"description": self.get_prompt("tool_desc"),
|
||||
"input_schema": {
|
||||
"pattern": {
|
||||
"type": "string",
|
||||
"description": self.get_prompt("pattern"),
|
||||
"required": True,
|
||||
},
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": self.get_prompt("path"),
|
||||
"required": False,
|
||||
},
|
||||
"glob": {
|
||||
"type": "string",
|
||||
"description": self.get_prompt("glob"),
|
||||
"required": False,
|
||||
},
|
||||
"limit": {
|
||||
"type": "number",
|
||||
"description": self.get_prompt("limit"),
|
||||
"required": False,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
return ToolCall(**tool_params)
|
||||
|
|
@ -1,46 +0,0 @@
|
|||
"""Read file operation module.
|
||||
|
||||
This module provides a tool operation for reading file contents.
|
||||
It supports reading entire files or specific line ranges for large files.
|
||||
"""
|
||||
|
||||
from flowllm.core.context import C
|
||||
from flowllm.core.schema import ToolCall
|
||||
from flowllm.extensions.file_tool import ReadFileOp as FlowReadFileOp
|
||||
|
||||
|
||||
@C.register_op()
|
||||
class ReadFileOp(FlowReadFileOp):
|
||||
"""Read file operation.
|
||||
|
||||
This operation reads and returns the content of a specified file.
|
||||
For text files, it can read specific line ranges using offset and limit.
|
||||
"""
|
||||
|
||||
file_path = __file__
|
||||
|
||||
def build_tool_call(self) -> ToolCall:
|
||||
"""Build and return the tool call schema for this operator."""
|
||||
tool_params = {
|
||||
"name": "ReadFile",
|
||||
"description": self.get_prompt("tool_desc"),
|
||||
"input_schema": {
|
||||
"absolute_path": {
|
||||
"type": "string",
|
||||
"description": self.get_prompt("absolute_path"),
|
||||
"required": True,
|
||||
},
|
||||
"offset": {
|
||||
"type": "number",
|
||||
"description": self.get_prompt("offset"),
|
||||
"required": False,
|
||||
},
|
||||
"limit": {
|
||||
"type": "number",
|
||||
"description": self.get_prompt("limit"),
|
||||
"required": False,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
return ToolCall(**tool_params)
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
tool_desc: |
|
||||
读取指定文件内容的工具；对文本文件可通过 offset 与 limit 获取特定行区间，便于分页浏览大文件。
|
||||
tool_desc_zh: |
|
||||
Reads and returns the content of a specified file. For text files, it can read specific line ranges using the 'offset' and 'limit' parameters. Use offset and limit to paginate through large files.
|
||||
|
||||
absolute_path: |
|
||||
必填：待读取文件的绝对路径（如 "/home/user/project/file.txt"），不支持相对路径。
|
||||
absolute_path_zh: |
|
||||
The absolute path to the file to read (e.g., '/home/user/project/file.txt'). Relative paths are not supported. You must provide an absolute path.
|
||||
|
||||
offset: |
|
||||
可选：文本文件起始读取的 0 基行号；需与 limit 同时使用，适合分页查看大文件。
|
||||
offset_zh: |
|
||||
Optional: For text files, the 0-based line number to start reading from. Requires 'limit' to be set. Use for paginating through large files.
|
||||
|
||||
limit: |
|
||||
可选：文本文件最多读取的行数；与 offset 配合实现分页，若仅设 offset 则会读到文件末尾。
|
||||
limit_zh: |
|
||||
Optional: For text files, maximum number of lines to read. Use with 'offset' to paginate through large files. If omitted when offset is provided, reads from offset to the end of the file.
|
||||
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
"""Context offload package for ReMe framework.
|
||||
|
||||
This package provides context management operations that can be used in LLM-powered flows
|
||||
to reduce token usage and manage context window limits. It includes ready-to-use operations for:
|
||||
|
||||
- ContextCompactOp: Compact tool messages by storing full content in external files
|
||||
- ContextCompressOp: Compress conversation history using LLM to generate concise summaries
|
||||
- ContextOffloadOp: Orchestrate compaction and compression to reduce token usage
|
||||
"""
|
||||
|
||||
from .context_compact_op import ContextCompactOp
|
||||
from .context_compress_op import ContextCompressOp
|
||||
from .context_offload_op import ContextOffloadOp
|
||||
|
||||
__all__ = [
|
||||
"ContextCompactOp",
|
||||
"ContextCompressOp",
|
||||
"ContextOffloadOp",
|
||||
]
|
||||
|
|
@ -1,107 +0,0 @@
|
|||
"""
|
||||
Context offload module for managing context window limits through compaction and compression.
|
||||
|
||||
This module provides a high-level operation that orchestrates context compaction and compression
|
||||
to reduce token usage. It first attempts to compact tool messages, and if the compaction ratio
|
||||
is not sufficient, it applies LLM-based compression to further reduce token count.
|
||||
|
||||
The offload process:
|
||||
1. Compacts tool messages by storing full content in external files
|
||||
2. Evaluates the compaction effectiveness by comparing token counts
|
||||
3. If compaction ratio exceeds threshold, applies LLM-based compression
|
||||
"""
|
||||
|
||||
from flowllm.core.context import C
|
||||
from flowllm.core.op import BaseAsyncOp
|
||||
from flowllm.core.schema import Message
|
||||
from loguru import logger
|
||||
|
||||
from reme_ai.enumeration import ContextManageEnum
|
||||
|
||||
|
||||
@C.register_op()
|
||||
class ContextOffloadOp(BaseAsyncOp):
|
||||
"""
|
||||
Context offload operation that orchestrates compaction and compression to reduce token usage.
|
||||
|
||||
This operation combines context compaction and compression strategies to manage context
|
||||
window limits. It first applies compaction to tool messages, then evaluates the effectiveness.
|
||||
If the compaction ratio (compressed tokens / original tokens) exceeds a threshold, it
|
||||
applies additional LLM-based compression to further reduce token count.
|
||||
|
||||
Context Parameters:
|
||||
context_manage_mode (ContextManageEnum): The context management mode to use.
|
||||
- COMPACT: Only applies context compaction to tool messages.
|
||||
- COMPRESS: Only applies LLM-based compression to messages.
|
||||
- AUTO: Applies compaction first, then compression if compaction ratio exceeds threshold.
|
||||
Defaults to AUTO.
|
||||
compact_ratio_threshold (float): Threshold for compaction ratio above which compression
|
||||
is applied. Only used in AUTO mode. Defaults to 0.75. If the ratio of compressed
|
||||
tokens to original tokens exceeds this value, compression will be triggered.
|
||||
"""
|
||||
|
||||
async def async_execute(self):
|
||||
"""
|
||||
Execute the context offload operation.
|
||||
|
||||
The operation behavior depends on the context_manage_mode:
|
||||
- COMPACT: Only applies context compaction to reduce token usage in tool messages.
|
||||
- COMPRESS: Only applies LLM-based compression to generate concise summaries.
|
||||
- AUTO: Applies compaction first, then compression if compaction ratio exceeds threshold.
|
||||
|
||||
The compaction operation stores full tool message content in external files and
|
||||
keeps only previews in the context. The compression operation uses LLM to generate
|
||||
concise summaries of older messages.
|
||||
"""
|
||||
from .context_compact_op import ContextCompactOp
|
||||
from .context_compress_op import ContextCompressOp
|
||||
|
||||
# Get the context management mode from context, default to AUTO
|
||||
context_manage_mode = self.context.get("context_manage_mode", ContextManageEnum.AUTO)
|
||||
if isinstance(context_manage_mode, str):
|
||||
context_manage_mode = ContextManageEnum(context_manage_mode)
|
||||
|
||||
context_compact_op = ContextCompactOp()
|
||||
context_compress_op = ContextCompressOp()
|
||||
|
||||
if context_manage_mode == ContextManageEnum.COMPACT:
|
||||
# Only apply compaction
|
||||
logger.info("Context management mode: COMPACT")
|
||||
await context_compact_op.async_call(context=self.context)
|
||||
elif context_manage_mode == ContextManageEnum.COMPRESS:
|
||||
# Only apply compression
|
||||
logger.info("Context management mode: COMPRESS")
|
||||
await context_compress_op.async_call(context=self.context)
|
||||
elif context_manage_mode == ContextManageEnum.AUTO:
|
||||
# Apply compaction first, then compression if needed
|
||||
logger.info("Context management mode: AUTO")
|
||||
await context_compact_op.async_call(context=self.context)
|
||||
|
||||
origin_messages = [Message(**x) for x in self.context.messages]
|
||||
origin_token_cnt = self.token_count(origin_messages)
|
||||
|
||||
result_messages = [Message(**x) for x in self.context.response.answer]
|
||||
answer_token_cnt = self.token_count(result_messages)
|
||||
|
||||
compact_ratio = answer_token_cnt / origin_token_cnt
|
||||
|
||||
compact_ratio_threshold: float = self.context.get("compact_ratio_threshold", 0.75)
|
||||
if compact_ratio > compact_ratio_threshold:
|
||||
logger.info(f"Compact ratio {compact_ratio:.2f} > {compact_ratio_threshold:.2f}, compress answer")
|
||||
await context_compress_op.async_call(context=self.context)
|
||||
else:
|
||||
raise ValueError(f"Unknown context management mode: {context_manage_mode}")
|
||||
|
||||
async def async_default_execute(self, e: Exception = None, **_kwargs):
|
||||
"""Handle execution errors by returning original messages.
|
||||
|
||||
This method is called when an exception occurs during async_execute. It preserves
|
||||
the original messages and marks the operation as unsuccessful.
|
||||
|
||||
Args:
|
||||
e: The exception that occurred during execution, if any.
|
||||
**_kwargs: Additional keyword arguments (unused but required by interface).
|
||||
"""
|
||||
self.context.response.answer = self.context.messages
|
||||
self.context.response.success = False
|
||||
self.context.response.metadata["error"] = str(e)
|
||||
|
|
@ -4,10 +4,10 @@ This module provides enumerations used throughout the ReMe system,
|
|||
including language enumerations and other type definitions.
|
||||
"""
|
||||
|
||||
from reme_ai.enumeration.context_manage_enum import ContextManageEnum
|
||||
from reme_ai.enumeration.working_summary_mode import WorkingSummaryMode
|
||||
from reme_ai.enumeration.language_enum import LanguageEnum
|
||||
|
||||
__all__ = [
|
||||
"ContextManageEnum",
|
||||
"WorkingSummaryMode",
|
||||
"LanguageEnum",
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,21 +0,0 @@
|
|||
"""Context management enumeration module.
|
||||
|
||||
This module provides enumerations for context management strategies in the ReMe system.
|
||||
"""
|
||||
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class ContextManageEnum(str, Enum):
|
||||
"""
|
||||
An enumeration representing context management strategies.
|
||||
|
||||
Members:
|
||||
- COMPACT: Represents the compact context management strategy.
|
||||
- COMPRESS: Represents the compress context management strategy.
|
||||
- AUTO: Represents the automatic context management strategy.
|
||||
"""
|
||||
|
||||
COMPACT = "compact"
|
||||
COMPRESS = "compress"
|
||||
AUTO = "auto"
|
||||
22
reme_ai/enumeration/working_summary_mode.py
Normal file
22
reme_ai/enumeration/working_summary_mode.py
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
"""Working summary mode enumeration module.
|
||||
|
||||
This module defines the strategies for working-memory style summarization in the
|
||||
ReMe system.
|
||||
"""
|
||||
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class WorkingSummaryMode(str, Enum):
|
||||
"""
|
||||
Enumeration representing working summary strategies.
|
||||
|
||||
Members:
|
||||
- COMPACT: Only compact verbose tool messages into previews.
|
||||
- COMPRESS: Only apply LLM-based compression over the history.
|
||||
- AUTO: First compact messages, then optionally compress if needed.
|
||||
"""
|
||||
|
||||
COMPACT = "compact"
|
||||
COMPRESS = "compress"
|
||||
AUTO = "auto"
|
||||
|
|
@ -224,12 +224,6 @@ def main():
|
|||
Command-line arguments are passed directly to ReMeApp.__init__(), allowing
|
||||
configuration via command line:
|
||||
|
||||
Example:
|
||||
python -m reme_ai.app --llm_api_key=sk-xxx --config_path=config.yaml
|
||||
|
||||
The app runs as a context manager, ensuring proper cleanup of resources
|
||||
(database connections, API clients, etc.) on shutdown.
|
||||
|
||||
Note:
|
||||
Press Ctrl+C to gracefully shutdown the service.
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
"""File tool package for ReMe framework.
|
||||
"""Working operations package for ReMe retrieve framework.
|
||||
|
||||
This package provides file-related operations that can be used in LLM-powered flows.
|
||||
It includes ready-to-use operations for:
|
||||
This package provides working/file-related operations that can be used in LLM-powered
|
||||
flows. It currently includes ready-to-use operations for:
|
||||
|
||||
- BatchWriteFileOp: Batch write multiple files operation
|
||||
- GrepOp: Text search operation for finding patterns in files
|
||||
|
|
@ -7,9 +7,10 @@ and returning a combined result of all write operations.
|
|||
|
||||
from flowllm.core.context import C
|
||||
from flowllm.core.op import BaseAsyncOp
|
||||
from flowllm.extensions.file_tool import WriteFileOp
|
||||
from loguru import logger
|
||||
|
||||
from .write_file_op import WriteFileOp
|
||||
|
||||
|
||||
@C.register_op()
|
||||
class BatchWriteFileOp(BaseAsyncOp):
|
||||
|
|
@ -34,13 +35,10 @@ class BatchWriteFileOp(BaseAsyncOp):
|
|||
# Get write file dictionary from context
|
||||
write_file_dict: dict = self.context.get("write_file_dict", {})
|
||||
if not write_file_dict:
|
||||
self.context.response.answer = "No write file task."
|
||||
logger.info("No write file task.")
|
||||
return
|
||||
|
||||
# Process each file in the dictionary
|
||||
result = []
|
||||
for file_path, content in write_file_dict.items():
|
||||
write_op = WriteFileOp(save_answer=self.save_answer)
|
||||
await write_op.async_call(file_path=file_path, content=content)
|
||||
result.append(write_op.output)
|
||||
114
reme_ai/retrieve/working/grep_op.py
Normal file
114
reme_ai/retrieve/working/grep_op.py
Normal file
|
|
@ -0,0 +1,114 @@
|
|||
"""Grep text search operation module.
|
||||
|
||||
This module provides a tool operation for searching text patterns in files.
|
||||
It enables efficient content-based search using regular expressions, with support
|
||||
for glob pattern filtering and result limiting.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
from typing import List
|
||||
|
||||
from flowllm.core.context import C
|
||||
from flowllm.core.op import BaseAsyncToolOp
|
||||
from flowllm.core.schema import ToolCall
|
||||
from loguru import logger
|
||||
|
||||
|
||||
@C.register_op()
|
||||
class GrepOp(BaseAsyncToolOp):
|
||||
"""Grep text search operation.
|
||||
|
||||
This operation searches for text patterns in files using regular expressions.
|
||||
Supports glob pattern filtering and result limiting.
|
||||
"""
|
||||
|
||||
file_path = __file__
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
kwargs.setdefault("raise_exception", False)
|
||||
super().__init__(**kwargs)
|
||||
|
||||
def build_tool_call(self) -> ToolCall:
|
||||
"""Build and return the tool call schema for this operator."""
|
||||
tool_params = {
|
||||
"name": "Grep",
|
||||
"description": self.get_prompt("tool_desc"),
|
||||
"input_schema": {
|
||||
"file_path": {
|
||||
"type": "string",
|
||||
"description": self.get_prompt("file_path"),
|
||||
"required": True,
|
||||
},
|
||||
"pattern": {
|
||||
"type": "string",
|
||||
"description": self.get_prompt("pattern"),
|
||||
"required": True,
|
||||
},
|
||||
"limit": {
|
||||
"type": "number",
|
||||
"description": self.get_prompt("limit"),
|
||||
"required": False,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
return ToolCall(**tool_params)
|
||||
|
||||
async def async_execute(self):
|
||||
"""Execute the grep search operation."""
|
||||
pattern: str = self.input_dict.get("pattern", "").strip()
|
||||
file_path: str | None | Path = self.input_dict.get("file_path", "")
|
||||
limit: int = self.input_dict.get("limit", 50)
|
||||
|
||||
# Validate pattern
|
||||
if not pattern:
|
||||
raise ValueError("The 'pattern' parameter cannot be empty.")
|
||||
|
||||
# Determine search directory
|
||||
if file_path:
|
||||
search_dir = Path(file_path).expanduser().resolve()
|
||||
if not search_dir.exists():
|
||||
raise ValueError(f"Search file_path does not exist: {search_dir}")
|
||||
else:
|
||||
search_dir = Path.cwd()
|
||||
|
||||
# Build grep command
|
||||
cmd: List[str] = ["grep", "-RIni"]
|
||||
if limit:
|
||||
cmd.extend(["-m", str(limit)])
|
||||
cmd.extend(["--", pattern, str(search_dir)])
|
||||
|
||||
logger.info(f"Running grep command: {' '.join(cmd)}")
|
||||
|
||||
# Execute grep using an async subprocess
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
stdout, stderr = await process.communicate()
|
||||
|
||||
if process.returncode not in (0, 1):
|
||||
# grep returns 1 when no matches are found; treat other codes as errors
|
||||
err_msg = stderr.decode("utf-8", errors="ignore").strip()
|
||||
raise RuntimeError(f"grep failed with code {process.returncode}: {err_msg}")
|
||||
|
||||
output_text = stdout.decode("utf-8", errors="ignore").strip()
|
||||
|
||||
# Return raw grep output
|
||||
if not output_text:
|
||||
search_location = f'in file_path "{file_path}"' if file_path else "in the workspace directory"
|
||||
result_msg = f'No matches found for pattern "{pattern}" {search_location}.'
|
||||
else:
|
||||
result_msg = output_text
|
||||
|
||||
self.set_output(result_msg)
|
||||
|
||||
async def async_default_execute(self, e: Exception = None, **_kwargs):
|
||||
"""Fill outputs with a default failure message when execution fails."""
|
||||
pattern: str = self.input_dict.get("pattern", "").strip()
|
||||
error_msg = f'Failed to search for pattern "{pattern}"'
|
||||
if e:
|
||||
error_msg += f": {str(e)}"
|
||||
self.set_output(error_msg)
|
||||
|
|
@ -3,21 +3,16 @@ tool_desc: |
|
|||
tool_desc_zh: |
|
||||
A powerful search tool for finding patterns in files using regular expressions. Supports full regex syntax (e.g., "log.*Error", "function\\s+\\w+"), glob pattern filtering, and result limiting. Ideal for searching code or text content across multiple files.
|
||||
|
||||
file_path: |
|
||||
要执行搜索的文件路径。
|
||||
file_path_zh: |
|
||||
The file path to perform the search on.
|
||||
|
||||
pattern: |
|
||||
需要在文件内容中匹配的正则表达式模式。
|
||||
pattern_zh: |
|
||||
The regular expression pattern to search for in file contents.
|
||||
|
||||
path: |
|
||||
可选：要执行搜索的目录，默认为当前工作目录。
|
||||
path_zh: |
|
||||
Optional: The directory to search in. Defaults to current working directory.
|
||||
|
||||
glob: |
|
||||
可选：用于过滤目标文件的 glob 模式（如 "*.js"、"*.{ts,tsx}"）。
|
||||
glob_zh: |
|
||||
Optional: Glob pattern to filter files (e.g., "*.js", "*.{ts,tsx}").
|
||||
|
||||
limit: |
|
||||
可选：设置最多返回的匹配行数；未指定时会返回所有匹配。
|
||||
limit_zh: |
|
||||
120
reme_ai/retrieve/working/read_file_op.py
Normal file
120
reme_ai/retrieve/working/read_file_op.py
Normal file
|
|
@ -0,0 +1,120 @@
|
|||
"""Read file operation module.
|
||||
|
||||
This module provides a tool operation for reading file contents.
|
||||
It supports reading entire files or specific line ranges for large files.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from flowllm.core.context import C
|
||||
from flowllm.core.op import BaseAsyncToolOp
|
||||
from flowllm.core.schema import ToolCall
|
||||
|
||||
|
||||
@C.register_op()
|
||||
class ReadFileOp(BaseAsyncToolOp):
|
||||
"""Read file operation.
|
||||
|
||||
This operation reads and returns the content of a specified file.
|
||||
For text files, it can read specific line ranges using offset and limit.
|
||||
"""
|
||||
|
||||
file_path = __file__
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
kwargs.setdefault("raise_exception", False)
|
||||
super().__init__(**kwargs)
|
||||
|
||||
def build_tool_call(self) -> ToolCall:
|
||||
"""Build and return the tool call schema for this operator."""
|
||||
tool_params = {
|
||||
"name": "ReadFile",
|
||||
"description": self.get_prompt("tool_desc"),
|
||||
"input_schema": {
|
||||
"file_path": {
|
||||
"type": "string",
|
||||
"description": self.get_prompt("file_path"),
|
||||
"required": True,
|
||||
},
|
||||
"offset": {
|
||||
"type": "number",
|
||||
"description": self.get_prompt("offset"),
|
||||
"required": True,
|
||||
},
|
||||
"limit": {
|
||||
"type": "number",
|
||||
"description": self.get_prompt("limit"),
|
||||
"required": True,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
return ToolCall(**tool_params)
|
||||
|
||||
async def async_execute(self):
|
||||
"""Execute the read file operation."""
|
||||
file_path: str = self.input_dict.get("file_path", "").strip()
|
||||
offset: Optional[int] = self.input_dict.get("offset")
|
||||
limit: Optional[int] = self.input_dict.get("limit")
|
||||
|
||||
# Validate file_path
|
||||
if not file_path:
|
||||
raise ValueError("The 'file_path' parameter cannot be empty.")
|
||||
|
||||
# Resolve file path
|
||||
file_path_obj = Path(file_path).expanduser().resolve()
|
||||
|
||||
# Check if file exists
|
||||
if not file_path_obj.exists():
|
||||
raise FileNotFoundError(f"File not found: {file_path_obj}")
|
||||
|
||||
if not file_path_obj.is_file():
|
||||
raise ValueError(f"Path is not a file: {file_path_obj}")
|
||||
|
||||
# Read file content
|
||||
content = file_path_obj.read_text(encoding="utf-8")
|
||||
lines = content.split("\n")
|
||||
|
||||
# Handle line range if specified
|
||||
if offset is not None or limit is not None:
|
||||
if offset is None:
|
||||
offset = 0
|
||||
if limit is None:
|
||||
limit = len(lines)
|
||||
|
||||
# Validate offset and limit
|
||||
if offset < 0:
|
||||
raise ValueError("Offset must be a non-negative number")
|
||||
if limit <= 0:
|
||||
raise ValueError("Limit must be a positive number")
|
||||
|
||||
total_lines = len(lines)
|
||||
start = offset
|
||||
end = min(offset + limit, total_lines)
|
||||
|
||||
if start >= total_lines:
|
||||
raise ValueError(
|
||||
f"Offset {offset} is beyond file length ({total_lines} lines)",
|
||||
)
|
||||
|
||||
selected_lines = lines[start:end]
|
||||
result_content = "\n".join(selected_lines)
|
||||
|
||||
# Format output with range information
|
||||
if end < total_lines:
|
||||
result = f"Showing lines {start}-{end - 1} of {total_lines} total lines.\n\n---\n\n{result_content}"
|
||||
else:
|
||||
result = result_content
|
||||
else:
|
||||
result = content
|
||||
|
||||
self.set_output(result)
|
||||
|
||||
async def async_default_execute(self, e: Exception = None, **_kwargs):
|
||||
"""Fill outputs with a default failure message when execution fails."""
|
||||
file_path: str = self.input_dict.get("file_path", "").strip()
|
||||
error_msg = f'Failed to read file "{file_path}"'
|
||||
if e:
|
||||
error_msg += f": {str(e)}"
|
||||
self.set_output(error_msg)
|
||||
19
reme_ai/retrieve/working/read_file_prompt.yaml
Normal file
19
reme_ai/retrieve/working/read_file_prompt.yaml
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
tool_desc: |
|
||||
读取指定文件内容的工具；对文本文件可通过 offset 与 limit 获取特定行区间，便于分页浏览大文件。
|
||||
tool_desc_zh: |
|
||||
Reads and returns the content of a specified file. For text files, it can read specific line ranges using the 'offset' and 'limit' parameters. Use offset and limit to paginate through large files.
|
||||
|
||||
file_path: |
|
||||
待读取文件的路径。
|
||||
file_path_zh: |
|
||||
The path to the file to read.
|
||||
|
||||
offset: |
|
||||
文本文件起始读取的 0 基行号；需与 limit 同时使用，适合分页查看大文件。
|
||||
offset_zh: |
|
||||
For text files, the 0-based line number to start reading from. Requires 'limit' to be set. Use for paginating through large files.
|
||||
|
||||
limit: |
|
||||
文本文件最多读取的行数；与 offset 配合实现分页，若仅设 offset 则会读到文件末尾。
|
||||
limit_zh: |
|
||||
For text files, maximum number of lines to read. Use with 'offset' to paginate through large files. If omitted when offset is provided, reads from offset to the end of the file.
|
||||
89
reme_ai/retrieve/working/write_file_op.py
Normal file
89
reme_ai/retrieve/working/write_file_op.py
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
"""Write file operation module.
|
||||
|
||||
This module provides a tool operation for writing content to files.
|
||||
It supports creating new files or overwriting existing files, and automatically
|
||||
creates parent directories if they don't exist.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from flowllm.core.context import C
|
||||
from flowllm.core.op import BaseAsyncToolOp
|
||||
from flowllm.core.schema import ToolCall
|
||||
|
||||
|
||||
@C.register_op()
|
||||
class WriteFileOp(BaseAsyncToolOp):
|
||||
"""Write file operation.
|
||||
|
||||
This operation writes content to a specified file. If the file doesn't exist,
|
||||
it will be created. If parent directories don't exist, they will be created automatically.
|
||||
"""
|
||||
|
||||
file_path = __file__
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
kwargs.setdefault("raise_exception", False)
|
||||
super().__init__(**kwargs)
|
||||
|
||||
def build_tool_call(self) -> ToolCall:
|
||||
"""Build and return the tool call schema for this operator."""
|
||||
tool_params = {
|
||||
"name": "WriteFile",
|
||||
"description": self.get_prompt("tool_desc"),
|
||||
"input_schema": {
|
||||
"file_path": {
|
||||
"type": "string",
|
||||
"description": self.get_prompt("file_path"),
|
||||
"required": True,
|
||||
},
|
||||
"content": {
|
||||
"type": "string",
|
||||
"description": self.get_prompt("content"),
|
||||
"required": True,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
return ToolCall(**tool_params)
|
||||
|
||||
async def async_execute(self):
|
||||
"""Execute the write file operation."""
|
||||
file_path: str = self.input_dict.get("file_path", "").strip()
|
||||
content: str = self.input_dict.get("content", "")
|
||||
|
||||
# Validate file_path
|
||||
if not file_path:
|
||||
raise ValueError("The 'file_path' parameter cannot be empty.")
|
||||
|
||||
# Resolve file path
|
||||
file_path_obj = Path(file_path).expanduser().resolve()
|
||||
|
||||
# Check if path is a directory
|
||||
if file_path_obj.exists() and file_path_obj.is_dir():
|
||||
raise ValueError(f"Path is a directory, not a file: {file_path_obj}")
|
||||
|
||||
# Create parent directories if they don't exist
|
||||
file_path_obj.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Check if file exists
|
||||
file_exists = file_path_obj.exists() and file_path_obj.is_file()
|
||||
|
||||
# Write content to file
|
||||
file_path_obj.write_text(content, encoding="utf-8")
|
||||
|
||||
# Format success message
|
||||
if file_exists:
|
||||
result = f"Successfully overwrote file: {file_path_obj}"
|
||||
else:
|
||||
result = f"Successfully created and wrote to new file: {file_path_obj}"
|
||||
|
||||
self.set_output(result)
|
||||
|
||||
async def async_default_execute(self, e: Exception = None, **_kwargs):
|
||||
"""Fill outputs with a default failure message when execution fails."""
|
||||
file_path: str = self.input_dict.get("file_path", "").strip()
|
||||
error_msg = f'Failed to write file "{file_path}"'
|
||||
if e:
|
||||
error_msg += f": {str(e)}"
|
||||
self.set_output(error_msg)
|
||||
9
reme_ai/retrieve/working/write_file_prompt.yaml
Normal file
9
reme_ai/retrieve/working/write_file_prompt.yaml
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
tool_desc: |
|
||||
Writes content to a specified file in the local filesystem. If the file doesn't exist, it will be created. If parent directories don't exist, they will be created automatically. If the file already exists, it will be overwritten with the new content.
|
||||
|
||||
file_path: |
|
||||
The absolute path to the file to write to (e.g., '/home/user/project/file.txt'). Relative paths are not supported. You must provide an absolute path.
|
||||
|
||||
content: |
|
||||
The content to write to the file.
|
||||
|
||||
23
reme_ai/summary/working/__init__.py
Normal file
23
reme_ai/summary/working/__init__.py
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
"""Working-memory summary package for the ReMe framework.
|
||||
|
||||
This package provides *working-memory* oriented summary operations that can be used in
|
||||
LLM-powered flows to reduce token usage and keep the active message window small, while
|
||||
preserving access to detailed historical information when needed. It includes:
|
||||
|
||||
- MessageCompactOp: Compact verbose tool messages by storing full content in external files
|
||||
and keeping short previews in the working context.
|
||||
- MessageCompressOp: Compress conversation history using an LLM to generate dense summaries
|
||||
that represent the agent's state snapshot.
|
||||
- MessageOffloadOp: Orchestrate compaction and optional compression as a unified
|
||||
working-memory offload pipeline.
|
||||
"""
|
||||
|
||||
from .message_compact_op import MessageCompactOp
|
||||
from .message_compress_op import MessageCompressOp
|
||||
from .message_offload_op import MessageOffloadOp
|
||||
|
||||
__all__ = [
|
||||
"MessageCompactOp",
|
||||
"MessageCompressOp",
|
||||
"MessageOffloadOp",
|
||||
]
|
||||
|
|
@ -1,9 +1,9 @@
|
|||
"""
|
||||
Context compaction module for reducing token usage in conversation contexts.
|
||||
"""Working-memory compaction module for reducing token usage.
|
||||
|
||||
This module provides functionality to compress large tool messages by storing
|
||||
their full content in external files and keeping only previews in the context.
|
||||
This helps manage context window limits while preserving important information.
|
||||
This module provides functionality to compact large tool messages for
|
||||
*working memory summary* by storing their full content in external files and
|
||||
keeping only short previews in the active message list. This reduces token
|
||||
usage while preserving access to detailed information when needed.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
|
@ -17,13 +17,14 @@ from loguru import logger
|
|||
|
||||
|
||||
@C.register_op()
|
||||
class ContextCompactOp(BaseAsyncOp):
|
||||
class MessageCompactOp(BaseAsyncOp):
|
||||
"""
|
||||
Context compaction operation that reduces token usage by compressing tool messages.
|
||||
Working-memory compaction operation that reduces token usage by compacting tool messages.
|
||||
|
||||
When the total token count exceeds the threshold, this operation compresses large tool
|
||||
messages by truncating their content and storing the full content in external files.
|
||||
This helps manage context window limits while preserving recent tool messages.
|
||||
When the total token count exceeds the threshold, this operation truncates large tool
|
||||
messages and stores the full content in external files. This is intended for
|
||||
working-memory style summarization: the agent sees a short preview while the
|
||||
complete result remains available out-of-band.
|
||||
"""
|
||||
|
||||
async def async_execute(self):
|
||||
|
|
@ -41,7 +42,7 @@ class ContextCompactOp(BaseAsyncOp):
|
|||
# Get configuration from context
|
||||
max_total_tokens: int = self.context.get("max_total_tokens", 20000)
|
||||
max_tool_message_tokens: int = self.context.get("max_tool_message_tokens", 2000)
|
||||
preview_char_length: int = self.context.get("preview_char_length", 100)
|
||||
preview_char_length: int = self.context.get("preview_char_length", 0)
|
||||
keep_recent_count: int = self.context.get("keep_recent_count", 1)
|
||||
store_dir: Path = Path(self.context.get("store_dir", ""))
|
||||
|
||||
|
|
@ -50,12 +51,22 @@ class ContextCompactOp(BaseAsyncOp):
|
|||
assert preview_char_length >= 0, "preview_char_length must be greater than 0"
|
||||
assert keep_recent_count >= 0, "keep_recent_count must be greater than 0"
|
||||
|
||||
# Convert context messages to Message objects
|
||||
messages = [Message(**x) for x in self.context.messages]
|
||||
|
||||
# Convert context messages to Message objects
|
||||
messages_to_compress = [x for x in messages if x.role is not Role.SYSTEM]
|
||||
if keep_recent_count > 0:
|
||||
messages_to_compress = messages_to_compress[:-keep_recent_count]
|
||||
|
||||
# Extract system message (should be exactly one)
|
||||
system_message = [x for x in messages if x.role is Role.SYSTEM]
|
||||
assert len(system_message) <= 1, f"Expected at most one system message, got {len(system_message)}"
|
||||
|
||||
if len(system_message) == 0:
|
||||
system_message = Message(role=Role.SYSTEM, content="")
|
||||
else:
|
||||
system_message = system_message[0]
|
||||
|
||||
# If nothing to compress after filtering, return original messages
|
||||
if not messages_to_compress:
|
||||
self.context.response.answer = self.context.messages
|
||||
|
|
@ -103,18 +114,18 @@ class ContextCompactOp(BaseAsyncOp):
|
|||
# Store the full content for batch writing
|
||||
write_file_dict[store_path.as_posix()] = original_content
|
||||
|
||||
# Create compressed preview of the tool message content
|
||||
compact_result = original_content[:preview_char_length] + "..."
|
||||
|
||||
# Log the compaction action
|
||||
logger.info(
|
||||
f"Compacting tool message (tool_call_id={tool_message.tool_call_id}): "
|
||||
f"token count={tool_token_cnt}, saving full content to {store_path}",
|
||||
f"token count={tool_token_cnt}, saving full content to {store_path.as_posix()}",
|
||||
)
|
||||
|
||||
# Update tool message content with preview and file reference
|
||||
compact_result += f" (detailed result is stored in {store_path})"
|
||||
compact_result = f"tool call={file_name} result is stored in file path=`{store_path.as_posix()}`"
|
||||
if preview_char_length > 0:
|
||||
compact_result += f"\npreview: {original_content[:preview_char_length]}..."
|
||||
tool_message.content = compact_result
|
||||
system_message.content += f"\n\n{compact_result}"
|
||||
|
||||
# Store write_file_dict in context for potential batch writing
|
||||
if write_file_dict:
|
||||
|
|
@ -1,14 +1,14 @@
|
|||
"""
|
||||
Context compression module for reducing token usage in conversation contexts using LLM.
|
||||
"""Working-memory compression module using LLM.
|
||||
|
||||
This module provides functionality to compress conversation history by using a language
|
||||
model to generate concise summaries of older messages while preserving recent messages.
|
||||
This helps manage context window limits while maintaining conversation coherence.
|
||||
This module compresses long conversation history for *working memory summary* by
|
||||
using a language model to generate concise summaries of older messages while
|
||||
preserving more recent ones. It is designed to keep the agent's short-term
|
||||
context small, while still retaining the essential information from past turns.
|
||||
|
||||
The compression process:
|
||||
1. Identifies messages that exceed token thresholds
|
||||
2. Splits messages into groups if needed
|
||||
3. Uses LLM to generate compressed summaries of older message groups
|
||||
2. Optionally splits messages into groups based on token budget
|
||||
3. Uses an LLM to generate compressed summaries of older message groups
|
||||
4. Stores original messages to files for potential retrieval
|
||||
5. Appends compressed summaries to the system message while preserving recent messages
|
||||
"""
|
||||
|
|
@ -29,16 +29,17 @@ from reme_ai.utils.op_utils import extract_xml_tag_content
|
|||
|
||||
|
||||
@C.register_op()
|
||||
class ContextCompressOp(BaseAsyncOp):
|
||||
class MessageCompressOp(BaseAsyncOp):
|
||||
"""
|
||||
Context compression operation that uses LLM to reduce token usage.
|
||||
Working-memory compression operation that uses an LLM to reduce token usage.
|
||||
|
||||
When the total token count exceeds the threshold, this operation uses a language
|
||||
model to compress older messages into a concise summary while keeping recent
|
||||
messages intact. This preserves conversation context while reducing token usage.
|
||||
When the total token count of older messages exceeds the threshold, this operation
|
||||
calls a language model to compress them into a concise summary, while keeping
|
||||
recent messages intact. This provides a compact *state snapshot* that the agent
|
||||
can rely on for subsequent steps.
|
||||
|
||||
Attributes:
|
||||
file_path: Path to the operation file, used for configuration.
|
||||
file_path: Path to the operation file, used for configuration (e.g. prompts).
|
||||
|
||||
Context Parameters:
|
||||
max_total_tokens (int): Maximum token count threshold for compression.
|
||||
|
|
@ -205,8 +206,8 @@ class ContextCompressOp(BaseAsyncOp):
|
|||
continue
|
||||
|
||||
compress_content = (
|
||||
f"[Compressed conversation history - Part {g_idx}/{len(message_groups)}]\n{group_summary}\n\n"
|
||||
f"(Original {len(messages)} messages are stored in: {store_path.as_posix()})\n"
|
||||
f"[Compressed conversation history - Part {g_idx}/{len(message_groups)}]\n{group_summary}\n"
|
||||
f"(Original {len(messages)} messages are stored in: {store_path.as_posix()})"
|
||||
)
|
||||
compressed_tokens = self.token_count([Message(content=compress_content)])
|
||||
|
||||
112
reme_ai/summary/working/message_offload_op.py
Normal file
112
reme_ai/summary/working/message_offload_op.py
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
"""Working-memory offload module based on compaction and compression.
|
||||
|
||||
This module implements a high-level *working summary* operation that orchestrates
|
||||
message compaction and LLM-based compression to reduce token usage for long
|
||||
conversations. It first compacts verbose tool messages, and, if the reduction
|
||||
ratio is not sufficient, it further compresses the history with an LLM.
|
||||
|
||||
The offload process:
|
||||
1. Compacts tool messages by storing full content in external files and keeping previews
|
||||
2. Evaluates compaction effectiveness by comparing token counts before/after
|
||||
3. If the compaction ratio exceeds a configurable threshold, applies LLM-based compression
|
||||
"""
|
||||
|
||||
from flowllm.core.context import C
|
||||
from flowllm.core.op import BaseAsyncOp
|
||||
from flowllm.core.schema import Message
|
||||
from loguru import logger
|
||||
|
||||
from reme_ai.enumeration import WorkingSummaryMode
|
||||
|
||||
|
||||
@C.register_op()
|
||||
class MessageOffloadOp(BaseAsyncOp):
|
||||
"""
|
||||
Working-memory offload operation that orchestrates compaction and compression.
|
||||
|
||||
This operation is designed specifically for *working memory summary* in long-running
|
||||
conversations. Its behavior is controlled by ``working_summary_mode``:
|
||||
|
||||
- ``COMPACT`` – only compact verbose tool messages by storing full content externally
|
||||
and keeping short previews in the context.
|
||||
- ``COMPRESS`` – only apply LLM-based compression to generate a compact state snapshot.
|
||||
- ``AUTO`` – first run compaction, then optionally run compression if the
|
||||
compaction ratio is not sufficient.
|
||||
|
||||
Context Parameters:
|
||||
working_summary_mode (WorkingSummaryMode | str): Working summary strategy to use.
|
||||
One of ``COMPACT``, ``COMPRESS`` or ``AUTO``. Defaults to ``AUTO``.
|
||||
compact_ratio_threshold (float): Only used in ``AUTO`` mode. Threshold for
|
||||
compaction ratio (compressed tokens divided by original tokens) above which
|
||||
LLM compression is applied. Defaults to 0.75.
|
||||
"""
|
||||
|
||||
async def async_execute(self):
|
||||
"""
|
||||
Execute the working-memory offload operation.
|
||||
|
||||
The behavior is selected via ``working_summary_mode``:
|
||||
|
||||
- COMPACT: only compaction is executed.
|
||||
- COMPRESS: only compression is executed.
|
||||
- AUTO: compaction is executed first; if the reduction is insufficient, a
|
||||
compression pass is executed.
|
||||
"""
|
||||
from .message_compact_op import MessageCompactOp
|
||||
from .message_compress_op import MessageCompressOp
|
||||
|
||||
message_compact_op = MessageCompactOp()
|
||||
message_compress_op = MessageCompressOp()
|
||||
|
||||
# Resolve working summary mode (string or enum) with AUTO as default.
|
||||
working_summary_mode = self.context.get("working_summary_mode", WorkingSummaryMode.AUTO)
|
||||
if isinstance(working_summary_mode, str):
|
||||
working_summary_mode = WorkingSummaryMode(working_summary_mode)
|
||||
|
||||
if working_summary_mode == WorkingSummaryMode.COMPACT:
|
||||
logger.info("Working-memory offload mode: COMPACT (only compaction)")
|
||||
await message_compact_op.async_call(context=self.context)
|
||||
return
|
||||
|
||||
if working_summary_mode == WorkingSummaryMode.COMPRESS:
|
||||
logger.info("Working-memory offload mode: COMPRESS (only compression)")
|
||||
await message_compress_op.async_call(context=self.context)
|
||||
return
|
||||
|
||||
# AUTO mode: compaction first, then compression if reduction is not sufficient.
|
||||
logger.info("Working-memory offload mode: AUTO (compaction then optional compression)")
|
||||
await message_compact_op.async_call(context=self.context)
|
||||
|
||||
origin_messages = [Message(**x) for x in self.context.messages]
|
||||
origin_token_cnt = self.token_count(origin_messages)
|
||||
|
||||
result_messages = [Message(**x) for x in self.context.response.answer]
|
||||
answer_token_cnt = self.token_count(result_messages)
|
||||
|
||||
if origin_token_cnt <= 0:
|
||||
logger.warning("Origin token count is 0 after compaction; skip compression stage")
|
||||
return
|
||||
|
||||
compact_ratio = answer_token_cnt / origin_token_cnt
|
||||
compact_ratio_threshold: float = self.context.get("compact_ratio_threshold", 0.75)
|
||||
|
||||
if compact_ratio > compact_ratio_threshold:
|
||||
logger.info(
|
||||
f"Working-memory offload: compact ratio {compact_ratio:.2f} > "
|
||||
f"{compact_ratio_threshold:.2f}, applying compression stage",
|
||||
)
|
||||
await message_compress_op.async_call(context=self.context)
|
||||
|
||||
async def async_default_execute(self, e: Exception = None, **_kwargs):
|
||||
"""Handle execution errors by returning original messages.
|
||||
|
||||
This method is called when an exception occurs during async_execute. It preserves
|
||||
the original messages and marks the operation as unsuccessful.
|
||||
|
||||
Args:
|
||||
e: The exception that occurred during execution, if any.
|
||||
**_kwargs: Additional keyword arguments (unused but required by interface).
|
||||
"""
|
||||
self.context.response.answer = self.context.messages
|
||||
self.context.response.success = False
|
||||
self.context.response.metadata["error"] = str(e)
|
||||
112
test_op/test_agentic_retrieve_op.py
Normal file
112
test_op/test_agentic_retrieve_op.py
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
"""Test script for AgenticRetrieveOp.
|
||||
|
||||
This script provides a simple end-to-end test case for AgenticRetrieveOp.
|
||||
It can be run directly with: python test_agentic_retrieve_op.py
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
|
||||
from flowllm.core.enumeration import Role
|
||||
from flowllm.core.schema import Message, ToolCall
|
||||
from loguru import logger
|
||||
|
||||
from reme_ai.agent.react.agentic_retrieve_op import AgenticRetrieveOp
|
||||
from reme_ai.main import ReMeApp
|
||||
|
||||
|
||||
async def test_agentic_retrieve_basic():
|
||||
"""Basic test for AgenticRetrieveOp with a short conversation history."""
|
||||
logger.info("\n" + "=" * 60)
|
||||
logger.info("Test: AgenticRetrieveOp basic behavior")
|
||||
logger.info("=" * 60)
|
||||
|
||||
tool_call_id = "call_6596dafa2a6a46f7a217da"
|
||||
f = open("README.md", encoding="utf-8")
|
||||
readme_content = f.read()
|
||||
f.close()
|
||||
|
||||
messages = [
|
||||
Message(
|
||||
role=Role.SYSTEM,
|
||||
content=(
|
||||
"You are a helpful assistant. "
|
||||
"请先使用`Grep`匹配关键词或者正则表达式所在行数，然后通过`ReadFile`读取位置附近的代码。"
|
||||
"如果没有找到匹配项，永远不要放弃尝试，尝试其他的参数，比如只搜索部分关键词。"
|
||||
"`Grep`之后通过 `ReadFile` 命令，你可以从指定偏移位置`offset`+长度`limit`开始查看内容，不要超过50行。"
|
||||
"如果当前内容不足，`ReadFile` 命令也可以不断尝试不同的`offset`和`limit`参数"
|
||||
),
|
||||
),
|
||||
Message(
|
||||
role=Role.USER,
|
||||
content="搜索下reme项目的的README文件",
|
||||
),
|
||||
Message(
|
||||
role=Role.ASSISTANT,
|
||||
content="",
|
||||
tool_calls=[
|
||||
ToolCall(
|
||||
**{
|
||||
"index": 0,
|
||||
"id": tool_call_id,
|
||||
"function": {
|
||||
"arguments": '{"query": "readme"}',
|
||||
"name": "web_search",
|
||||
},
|
||||
"type": "function",
|
||||
},
|
||||
),
|
||||
],
|
||||
),
|
||||
Message(
|
||||
role=Role.TOOL,
|
||||
content=readme_content,
|
||||
tool_call_id=tool_call_id,
|
||||
),
|
||||
Message(
|
||||
role=Role.USER,
|
||||
content="根据readme回答task memory在appworld的效果是多少，需要具体的数值",
|
||||
),
|
||||
]
|
||||
|
||||
# llm = "qwen3_coder_plus"
|
||||
llm = "qwen3_30b_instruct"
|
||||
# llm = "qwen3_coder_30b_instruct"
|
||||
# llm = "qwen3_max_instruct"
|
||||
op = AgenticRetrieveOp(llm=llm)
|
||||
|
||||
await op.async_call(
|
||||
messages=[m.model_dump() for m in messages],
|
||||
working_summary_mode="auto",
|
||||
compact_ratio_threshold=0.75,
|
||||
max_total_tokens=6000,
|
||||
max_tool_message_tokens=2000,
|
||||
group_token_threshold=None,
|
||||
keep_recent_count=1,
|
||||
store_dir="./test_working_memory",
|
||||
chat_id="c123",
|
||||
)
|
||||
|
||||
answer = op.context.response.answer
|
||||
messages = op.context.response.metadata["messages"]
|
||||
logger.info(f"✓ AgenticRetrieveOp result answer: {answer}")
|
||||
logger.info(f"✓ AgenticRetrieveOp result messages: {json.dumps(messages, ensure_ascii=False, indent=2)}")
|
||||
logger.info(f" Success: {op.context.response.success}")
|
||||
|
||||
|
||||
async def async_main():
|
||||
"""Entry point for running AgenticRetrieveOp test."""
|
||||
async with ReMeApp():
|
||||
logger.info("=" * 80)
|
||||
logger.info("Testing AgenticRetrieveOp - ReAct Retrieval Workflow")
|
||||
logger.info("=" * 80)
|
||||
|
||||
await test_agentic_retrieve_basic()
|
||||
|
||||
logger.info("\n" + "=" * 80)
|
||||
logger.info("All AgenticRetrieveOp tests completed!")
|
||||
logger.info("=" * 80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(async_main())
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
"""Test script for ContextCompactOp.
|
||||
"""Test script for MessageCompactOp.
|
||||
|
||||
This script provides test cases for ContextCompactOp class.
|
||||
This script provides test cases for MessageCompactOp class.
|
||||
It can be run directly with: python test_context_compact_op.py
|
||||
"""
|
||||
|
||||
|
|
@ -9,13 +9,13 @@ import asyncio
|
|||
from flowllm.core.enumeration import Role
|
||||
from flowllm.core.schema import Message
|
||||
|
||||
from reme_ai.context.file_tool import BatchWriteFileOp
|
||||
from reme_ai.context.offload import ContextCompactOp
|
||||
from reme_ai.main import ReMeApp
|
||||
from reme_ai.retrieve.working import BatchWriteFileOp
|
||||
from reme_ai.summary.working import MessageCompactOp
|
||||
|
||||
|
||||
async def async_main():
|
||||
"""Test function for ContextCompactOp."""
|
||||
"""Test function for MessageCompactOp."""
|
||||
async with ReMeApp():
|
||||
# Create test messages with system, user, assistant, tool sequence
|
||||
messages = [
|
||||
|
|
@ -60,7 +60,7 @@ async def async_main():
|
|||
]
|
||||
|
||||
# Create op with lower thresholds for testing
|
||||
op = ContextCompactOp() >> BatchWriteFileOp()
|
||||
op = MessageCompactOp() >> BatchWriteFileOp()
|
||||
|
||||
# Execute the compaction
|
||||
await op.async_call(
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
"""
|
||||
Test script for ContextCompressOp.
|
||||
Test script for MessageCompressOp.
|
||||
|
||||
This script demonstrates how to use the context compression operation to reduce
|
||||
This script demonstrates how to use the message compression operation to reduce
|
||||
token usage in conversation histories using language models.
|
||||
"""
|
||||
|
||||
|
|
@ -9,16 +9,16 @@ import asyncio
|
|||
|
||||
from loguru import logger
|
||||
|
||||
from reme_ai.context.offload.context_compress_op import ContextCompressOp
|
||||
from reme_ai.main import ReMeApp
|
||||
from reme_ai.summary.working import MessageCompressOp
|
||||
|
||||
|
||||
async def main():
|
||||
"""Main function to test ContextCompressOp."""
|
||||
"""Main function to test MessageCompressOp."""
|
||||
|
||||
async with ReMeApp():
|
||||
logger.info("=" * 80)
|
||||
logger.info("Testing ContextCompressOp - LLM-based Context Compression")
|
||||
logger.info("Testing MessageCompressOp - LLM-based Context Compression")
|
||||
logger.info("=" * 80)
|
||||
|
||||
# Create a mock conversation with multiple messages
|
||||
|
|
@ -220,7 +220,7 @@ async def main():
|
|||
logger.info("Test 1: Messages below threshold (should skip compression)")
|
||||
logger.info("=" * 60)
|
||||
|
||||
compress_op1 = ContextCompressOp()
|
||||
compress_op1 = MessageCompressOp()
|
||||
|
||||
await compress_op1.async_call(
|
||||
messages=messages,
|
||||
|
|
@ -236,7 +236,7 @@ async def main():
|
|||
logger.info("Test 2: Messages above threshold (should compress)")
|
||||
logger.info("=" * 60)
|
||||
|
||||
compress_op2 = ContextCompressOp()
|
||||
compress_op2 = MessageCompressOp()
|
||||
|
||||
await compress_op2.async_call(
|
||||
messages=messages,
|
||||
|
|
@ -260,7 +260,7 @@ async def main():
|
|||
logger.info("Test 3: Messages above micro threshold (should compress)")
|
||||
logger.info("=!" * 30)
|
||||
|
||||
compress_op2 = ContextCompressOp()
|
||||
compress_op2 = MessageCompressOp()
|
||||
|
||||
await compress_op2.async_call(
|
||||
messages=messages,
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
"""Test script for ContextOffloadOp.
|
||||
"""Test script for MessageOffloadOp.
|
||||
|
||||
This script provides test cases for ContextOffloadOp class.
|
||||
This script provides test cases for MessageOffloadOp class.
|
||||
It can be run directly with: python test_context_offload_op.py
|
||||
"""
|
||||
|
||||
|
|
@ -10,14 +10,14 @@ from flowllm.core.enumeration import Role
|
|||
from flowllm.core.schema import Message
|
||||
from loguru import logger
|
||||
|
||||
from reme_ai.context.file_tool import BatchWriteFileOp
|
||||
from reme_ai.context.offload.context_offload_op import ContextOffloadOp
|
||||
from reme_ai.enumeration import ContextManageEnum
|
||||
from reme_ai.enumeration import WorkingSummaryMode
|
||||
from reme_ai.main import ReMeApp
|
||||
from reme_ai.retrieve.working import BatchWriteFileOp
|
||||
from reme_ai.summary.working import MessageOffloadOp
|
||||
|
||||
|
||||
async def test_compact_mode():
|
||||
"""Test COMPACT mode - Only apply compaction."""
|
||||
"""Test COMPACT mode - Only apply compaction with MessageOffloadOp."""
|
||||
logger.info("\n" + "=" * 60)
|
||||
logger.info("Test: COMPACT mode - Only apply compaction")
|
||||
logger.info("=" * 60)
|
||||
|
|
@ -64,11 +64,11 @@ async def test_compact_mode():
|
|||
),
|
||||
]
|
||||
|
||||
op = ContextOffloadOp() >> BatchWriteFileOp()
|
||||
op = MessageOffloadOp() >> BatchWriteFileOp()
|
||||
|
||||
await op.async_call(
|
||||
messages=[m.model_dump() for m in messages],
|
||||
context_manage_mode=ContextManageEnum.COMPACT,
|
||||
context_manage_mode=WorkingSummaryMode.COMPACT,
|
||||
max_total_tokens=1000, # Low threshold to trigger compaction
|
||||
max_tool_message_tokens=100, # Low threshold to compact tool messages
|
||||
preview_char_length=50, # Keep 50 chars in preview
|
||||
|
|
@ -82,7 +82,7 @@ async def test_compact_mode():
|
|||
|
||||
|
||||
async def test_compress_mode():
|
||||
"""Test COMPRESS mode - Only apply compression."""
|
||||
"""Test COMPRESS mode - Only apply compression with MessageOffloadOp."""
|
||||
logger.info("\n" + "=" * 60)
|
||||
logger.info("Test: COMPRESS mode - Only apply compression")
|
||||
logger.info("=" * 60)
|
||||
|
|
@ -129,11 +129,11 @@ async def test_compress_mode():
|
|||
),
|
||||
]
|
||||
|
||||
op = ContextOffloadOp() >> BatchWriteFileOp()
|
||||
op = MessageOffloadOp() >> BatchWriteFileOp()
|
||||
|
||||
await op.async_call(
|
||||
messages=[m.model_dump() for m in messages],
|
||||
context_manage_mode=ContextManageEnum.COMPRESS,
|
||||
context_manage_mode=WorkingSummaryMode.COMPRESS,
|
||||
max_total_tokens=2000, # Low threshold to trigger compression
|
||||
keep_recent_count=2,
|
||||
store_dir="./test_compact_storage",
|
||||
|
|
@ -145,7 +145,7 @@ async def test_compress_mode():
|
|||
|
||||
|
||||
async def test_auto_mode():
|
||||
"""Test AUTO mode - Apply compaction first, then compression if needed."""
|
||||
"""Test AUTO mode - Apply compaction first, then compression if needed using MessageOffloadOp."""
|
||||
logger.info("\n" + "=" * 60)
|
||||
logger.info("Test: AUTO mode - Apply compaction first, then compression if needed")
|
||||
logger.info("=" * 60)
|
||||
|
|
@ -209,11 +209,11 @@ async def test_auto_mode():
|
|||
),
|
||||
]
|
||||
|
||||
op = ContextOffloadOp() >> BatchWriteFileOp()
|
||||
op = MessageOffloadOp() >> BatchWriteFileOp()
|
||||
|
||||
await op.async_call(
|
||||
messages=[m.model_dump() for m in auto_messages],
|
||||
context_manage_mode=ContextManageEnum.AUTO,
|
||||
context_manage_mode=WorkingSummaryMode.AUTO,
|
||||
compact_ratio_threshold=0.2, # Low threshold, should trigger compression after compact
|
||||
max_total_tokens=1000,
|
||||
max_tool_message_tokens=100,
|
||||
|
|
@ -228,14 +228,14 @@ async def test_auto_mode():
|
|||
|
||||
|
||||
async def async_main():
|
||||
"""Test function for ContextOffloadOp."""
|
||||
"""Test function for MessageOffloadOp."""
|
||||
async with ReMeApp():
|
||||
logger.info("=" * 80)
|
||||
logger.info("Testing ContextOffloadOp - Context Management Orchestration")
|
||||
logger.info("Testing MessageOffloadOp - Context Management Orchestration")
|
||||
logger.info("=" * 80)
|
||||
|
||||
# await test_compact_mode()
|
||||
# await test_compress_mode()
|
||||
await test_compact_mode()
|
||||
await test_compress_mode()
|
||||
await test_auto_mode()
|
||||
|
||||
logger.info("\n" + "=" * 80)
|
||||
Loading…
Add table
Reference in a new issue