scriptoriumadmin/ReMe
1
0
Fork 0
mirror of https://github.com/agentscope-ai/ReMe.git synced 2026-08-28 05:25:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 11

docs: remove unused documentation files

Browse source 
- Deleted Configuration Guide, Future Roadmap, and Operations Documentation
- These files were likely outdated and no longer relevant
- Removing unused documentation improves maintainability and reduces confusion
This commit is contained in:
jinli.yl 2025-09-01 22:56:56 +08:00
parent f7dd74ea20
commit f3146aef04
7 changed files with 0 additions and 1654 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

661
doc/README.md
View file

@ -1,661 +0,0 @@
# ExperienceMaker
<p align="center">
<img src="doc/figure/logo.jpg" alt="ExperienceMaker Logo" width="100%">
</p>
<p align="center">
<a href="https://pypi.org/project/experiencemaker/"><img src="https://img.shields.io/badge/python-3.12+-blue" alt="Python Version"></a>
<a href="https://pypi.org/project/experiencemaker/"><img src="https://img.shields.io/badge/pypi-v0.1.1-blue?logo=pypi" alt="PyPI Version"></a>
<a href="./LICENSE"><img src="https://img.shields.io/badge/license-Apache--2.0-black" alt="License"></a>
<a href="https://github.com/modelscope/ExperienceMaker"><img src="https://img.shields.io/github/stars/modelscope/ExperienceMaker?style=social" alt="GitHub Stars"></a>
</p>
<p align="center">
<strong>A comprehensive framework to make & reuse & share experience for AI agent</strong><br>
<em>Empowering agents to learn from the past and excel in the future</em>
</p>
---
## 📰 What's New
- **[2025-08]** 🚀 MCP is now available! → [Quick Start Guide](./doc/mcp_quick_start.md)
- **[2025-07]** 🎉 ExperienceMaker v0.1.1 is now available on [PyPI](https://pypi.org/project/experiencemaker/)!
- **[2025-07]** 📚 Complete documentation and quick start guides released
- **[2025-06]** 🚀 Multi-backend vector store support (Elasticsearch & ChromaDB)
---
## 🚀 What's Next
- **Pre-built Experience Libraries**: Domain repositories (Finance/Coding/Education/Research) + community marketplace
- **Rich Experience Formats**: Executable code/tool configs/pipeline templates/workflows
- **Experience Validation**: Quality analysis + cross-task effectiveness + auto-refinement
- **Universal Trajectory Extraction**: Raw logs/multimodal data/execution traces → experiences
---
## 🌟 What is ExperienceMaker?
ExperienceMaker is a framework that transforms how AI agents learn and improve through **experience-driven intelligence**.
By automatically extracting, storing, and intelligently reusing experiences from agent trajectories, it enables continuous learning and progressive skill enhancement.
### ✨ Core Capabilities
#### 🔍 **Intelligent Experience Summarizer**
- **Success Pattern Recognition**: Identify what works and understand the underlying principles
- **Failure Analysis**: Learn from mistakes to avoid repeating them in future tasks
- **Comparative Insights**: Understand the critical differences between successful and failed approaches
- **Multistep Trajectory Processing**: Break down complex tasks into learnable, actionable segments
#### 🎯 **Smart Experience Retriever**
- **Semantic Search**: Find relevant experiences using advanced embedding models and semantic understanding
- **Context-Aware Ranking**: Prioritize the most applicable experiences for current task contexts
- **Dynamic Rewriting**: Intelligently adapt experiences to fit new situations and requirements
- **Multi-modal Support**: Handle various input types including query, messages
#### 🗄️ **Scalable Experience Management**
- **Multiple Storage Backends**: Choose from Elasticsearch (production-ready), ChromaDB (development), or file-based storage (testing)
- **Workspace Isolation**: Organize experiences by projects, domains, or teams with complete separation
- **Deduplication & Validation**: Ensure high-quality, unique experience storage with automated quality control
- **Batch Operations**: Efficiently handle large-scale experience processing with optimized performance
#### 🔧 **Developer-Friendly Architecture**
- **REST API Interface**: Seamless integration with existing systems through clean API design
- **Modular Pipeline Design**: Compose custom workflows from atomic operations with maximum flexibility
- **Flexible Configuration**: YAML files and command-line overrides for easy customization
- **Experience Store**: Ready-to-use out of the box — there’s no need for you to manually summarize experiences. You can directly leverage existing, comprehensive experience datasets to greatly enhance your agent’s capabilities.
<p align="center">
<img src="doc/figure/framework.png" alt="ExperienceMaker Architecture" width="70%">
</p>
---
## 🛠️ Installation
### Option 1: Install from PyPI (Recommended)
```bash
pip install experiencemaker
```
### Option 2: Install from Source
```bash
git clone https://github.com/modelscope/ExperienceMaker.git
cd ExperienceMaker
pip install .
```
## ⚙️ Environment Setup
Create a `.env` file in your project root directory:
```bash
# Required: LLM API configuration
LLM_API_KEY="sk-xxx"
LLM_BASE_URL="https://xxx.com/v1"
# Required: Embedding model configuration
EMBEDDING_MODEL_API_KEY="sk-xxx"
EMBEDDING_MODEL_BASE_URL="https://xxx.com/v1"
# Optional: Elasticsearch configuration (if using Elasticsearch backend)
```
## 🚀 Quick Start
### 🌐 HTTP Service
For testing and development, use the `local_file` backend:
```bash
experiencemaker \
http_service.port=8001 \
llm.default.model_name=qwen3-32b \
embedding_model.default.model_name=text-embedding-v4 \
vector_store.default.backend=local_file
```
💡 **Pro Tip**: Check out our [Configuration Guide](./doc/configuration_guide.md) for detailed configuration topics
including custom pipelines, operation parameters, and advanced configuration methods.
The service will start on `http://localhost:8001`
### 🔌 MCP Server
ExperienceMaker now supports Model Context Protocol (MCP) for seamless integration with MCP-compatible clients like Claude Desktop:
```bash
experiencemaker_mcp \
mcp_transport=stdio \
llm.default.model_name=qwen3-32b \
embedding_model.default.model_name=text-embedding-v4 \
vector_store.default.backend=local_file
```
For SSE transport (Server-Sent Events):
```bash
experiencemaker_mcp \
mcp_transport=sse \
http_service.port=8001 \
llm.default.model_name=qwen3-32b \
embedding_model.default.model_name=text-embedding-v4 \
vector_store.default.backend=local_file
```
🔗 **For detailed MCP setup and usage examples**, see our [MCP Quick Start Guide](./doc/mcp_quick_start.md).
### 🔍 Production Setup with Elasticsearch Backend
```bash
experiencemaker \
http_service.port=8001 \
llm.default.model_name=qwen3-32b \
embedding_model.default.model_name=text-embedding-v4 \
vector_store.default.backend=elasticsearch
```
**Setup Elasticsearch:**
```bash
export ES_HOSTS="http://localhost:9200"
# Quick setup using Elastic's official script
curl -fsSL https://elastic.co/start-local | sh
```
📖 **Need Help?** Refer to [Vector Store Setup](./doc/vector_store_setup.md) for comprehensive deployment guidance.
## 📝 Your First ExperienceMaker Script
Here's how to get started!
Note the `workspace_id` serves as your experience storage namespace. Experiences in different workspaces remain completely isolated and cannot access each other.
### 📊 Call Summarizer Examples
Transform conversation trajectories into valuable experiences using batch summarization. Each trajectory contains:
- **Message**: Complete conversation history between user and agent
- **Score**: Performance rating (0-1 scale, where 0=failure, 1=success)
The summarizer analyzes these trajectories to extract actionable insights and patterns for future interactions.
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/summarizer", json={
"workspace_id": "test_workspace",
"traj_list": [
{"messages": [{"role": "user", "content": "hello world"}], "score": 1.0}
]
})
experience_list = response.json()["experience_list"]
for experience in experience_list:
print(experience)
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/summarizer" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"traj_list": [
{
"messages": [{"role": "user", "content": "hello world"}],
"score": 1.0
}
]
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function callSummarizer() {
try {
const response = await fetch('http://0.0.0.0:8001/summarizer', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
traj_list: [
{
messages: [{ role: "user", content: "hello world" }],
score: 1.0
}
]
})
});
const data = await response.json();
const experienceList = data.experience_list;
experienceList.forEach(experience => {
console.log(experience);
});
} catch (error) {
console.error('Error:', error);
}
}
callSummarizer();
```
</details>
### 🔍 Call Retriever Examples
Intelligently search and retrieve the most relevant experiences from your workspace to enhance decision-making. The retriever:
- **Finds** the top-k most similar experiences based on semantic similarity to your query
- **Returns** pre-assembled context ready for immediate use, or raw experience data for custom processing
- **Leverages** your workspace's accumulated knowledge to provide contextually relevant insights
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/retriever", json={
"workspace_id": "test_workspace",
"query": "what is the meaning of life?",
"top_k": 1,
})
experience_merged: str = response.json()["experience_merged"]
print(f"experience_merged={experience_merged}")
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/retriever" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"query": "what is the meaning of life?",
"top_k": 1
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function callRetriever() {
try {
const response = await fetch('http://0.0.0.0:8001/retriever', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
query: "what is the meaning of life?",
top_k: 1
})
});
const data = await response.json();
const experienceMerged = data.experience_merged;
console.log(`experience_merged=${experienceMerged}`);
} catch (error) {
console.error('Error:', error);
}
}
callRetriever();
```
</details>
### 💾 Dump Experiences From Vector Store
Export and backup your valuable experience data for archival, analysis, or migration purposes. This operation:
- **Extracts** all experiences from the specified workspace in the vector store
- **Saves** them to a structured JSONL file at `{path}/{workspace_id}.jsonl`
- **Preserves** complete experience metadata and embeddings for future restoration
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/vector_store", json={
"workspace_id": "test_workspace",
"action": "dump",
"path": "./",
})
print(response.json())
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/vector_store" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"action": "dump",
"path": "./"
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function dumpExperiences() {
try {
const response = await fetch('http://0.0.0.0:8001/vector_store', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
action: "dump",
path: "./"
})
});
const data = await response.json();
console.log(data);
} catch (error) {
console.error('Error:', error);
}
}
dumpExperiences();
```
</details>
### 📥 Load Experiences To Vector Store
Import and restore previously exported experience data to populate your workspace with existing knowledge. This operation:
- **Reads** experience data from the JSONL file located at `{path}/{workspace_id}.jsonl`
- **Reconstructs** the vector embeddings and indexes them in the specified workspace
- **Enables** immediate access to imported experiences for retrieval and decision-making
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/vector_store", json={
"workspace_id": "test_workspace",
"action": "load",
"path": "./",
})
print(response.json())
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/vector_store" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"action": "load",
"path": "./"
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function loadExperiences() {
try {
const response = await fetch('http://0.0.0.0:8001/vector_store', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
action: "load",
path: "./"
})
});
const data = await response.json();
console.log(data);
} catch (error) {
console.error('Error:', error);
}
}
loadExperiences();
```
</details>
💡 **Need More Advanced Operations?** For additional workspace management features(e.g. delete_workspace,
copy_workspace), advanced configuration options, and troubleshooting guidance, check out our
comprehensive [Quick Start Guide](./cookbook/simple_demo/quick_start.md).
🎭 **Want to See It in Action?** We've prepared a [simple react agent](./cookbook/simple_demo/simple_demo.py) that demonstrates how to enhance agent capabilities by integrating summarizer and retriever components, achieving significantly better performance.
---
## 🧪 Experiments
### 🌍 Experiment on Appworld
We test ExperienceMaker on Appworld with qwen3-8b:
| Method | pass@1 | pass@2 | pass@4 |
|--------------------------------|-----------|-------------|-----------|
| w/o ExperienceMaker (baseline) | 0.083 | 0.140 | 0.228 |
| **w ExperienceMaker** | | | |
|experience(Direct Use) | **0.109** | **0.175** | **0.281** |
Pass@K measures the probability that at least one out of K generated samples successfully completes the task (achieves score=1).
The current experiments use an internal AppWorld environment which may have slight discrepancies, and we will soon update with experimental results from the standard AppWorld environment.
You may find more details to reproduce this experiment in [quickstart.md](cookbook/appworld/quickstart.md)
### 🧊 Experiment on Frozenlake
| without experience | with experience |
|:-------------------------------------------------------------------------------------------:|:-------------------------------------------:|
| <p align="center"><img src="doc/figure/frozenlake_failure.gif" alt="GIF 1" width="30%"></p> | <p align="center"><img src="doc/figure/frozenlake_success.gif" alt="GIF 2" width="30%"></p>
We test on 100 random frozenlake map with qwen3-8b:
| Method | pass rate |
|-------------------------------|------------------|
| w/o ExperienceMaker (baseline) | 0.66 |
| **w ExperienceMaker** | |
| [1] experience(Direct Use) | 0.72 **(+9.1%)** |
| [2] experience(LLM Rewritten) | 0.72 **(+9.1%)** |
We also noticed that in such simple scenarios, not using LLM rewriting may actually yield better results.
Therefore, in some simple scenarios, you can also try disabling LLM rewriting by simply changing the following in default_config.yaml:
```yaml
rewrite_experience_op:
params:
enable_llm_rewrite: false # change this to false
```
You may find more details to reproduce this experiment in [quickstart.md](cookbook/frozenlake/quickstart.md)
### 🔧 Experiment on BFCL-V3
Coming Soon! Stay tuned for comprehensive evaluation results.
---
## 🏪 Ready-made Experience Store
ExperienceMaker provides pre-built experience libraries to jumpstart your agent's capabilities.
You can directly load these curated experiences into your workspace and start benefiting from accumulated knowledge
immediately.
### 📦 Available Experience Libraries
- **`appworld_v1.jsonl`**: Comprehensive experiences from Appworld agent interactions, covering complex task planning
and execution patterns
- **`bfcl_v1.jsonl`**: Function calling experiences from Berkeley Function-Calling Leaderboard tasks
### 🚀 Quick Start with Pre-built Experiences
Here's how to load and use the Appworld experience library:
#### Step 1: Load Pre-built Experiences
<details open>
<summary><b>Python</b></summary>
```python
import requests
# Load Appworld experiences into your workspace
response = requests.post(url="http://0.0.0.0:8001/vector_store", json={
"workspace_id": "appworld_v1",
"action": "load",
"path": "./library/",
})
print(f"loading result result={response.json()}")
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/vector_store" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "appworld_v1",
"action": "load",
"path": "./library/"
}'
```
</details>
#### Step 2: Retrieve Relevant Experiences
Now you can query the loaded experiences to get contextual guidance for your tasks:
<details open>
<summary><b>Python</b></summary>
```python
import requests
# Query for app interaction experiences
response = requests.post(url="http://0.0.0.0:8001/retriever", json={
"workspace_id": "appworld_v1",
"query": "How to navigate to settings and update user profile information?",
"top_k": 1,
})
experience_merged = response.json()["experience_merged"]
print(f"Retrieved experiences: {experience_merged}")
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/retriever" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "appworld_v1",
"query": "How to navigate to settings and update user profile information?",
"top_k": 1
}'
```
</details>
---
## 📚 Additional Resources
- **[Quick Start](./cookbook/simple_demo/quick_start.md)**: This guide will help you get started with ExperienceMaker quickly using practical examples.
- **[Vector Store Setup](./doc/vector_store_setup.md)**: Complete production deployment guide
- **[Configuration Guide](./doc/configuration_guide.md)**: Describes all available command-line parameters for ExperienceMaker Service
- **[Operations Documentation](./doc/operations_documentation.md)**: Comprehensive operations configuration reference
- **[Example Collection](./cookbook)**: Practical examples and use cases
- **[Future RoadMap](./doc/future_roadmap.md)**: Our vision and upcoming features
---
## 🤝 Contributing
We warmly welcome contributions from the community! Here's how you can help make ExperienceMaker even better:
### 🐛 **Report Issues**
- Bug reports with detailed reproduction steps
- Feature requests and enhancement suggestions
- Documentation improvements and clarifications
- Performance optimization ideas
### 💻 **Code Contributions**
- New operations and tools development
- Backend implementations and optimizations
- API enhancements and new endpoints
- Test coverage improvements and quality assurance
### 📝 **Documentation**
- Usage examples and comprehensive tutorials
- Best practices guides and design patterns
- Translation and localization efforts
---
## 📄 Citation
If you use ExperienceMaker in your research or projects, please cite:
```bibtex
@software{ExperienceMaker,
title = {ExperienceMaker: A Comprehensive Framework for AI Agent Experience Generation and Reuse},
author = {The ExperienceMaker Team},
url = {https://github.com/modelscope/ExperienceMaker},
month = {08},
year = {2025},
}
```
---
## ⚖️ License
This project is licensed under the Apache License 2.0 - see the [LICENSE](./LICENSE) file for details.
---

13
doc/ROADMAP.md
View file

@ -1,13 +0,0 @@
1. library 转化 @zouyin
2. index.html @jinli
3. reme_ai两个personal的调通
4. doc
1. readme @jiaji
2. experience maker @jiaji
3. personal @jinli
5. 新增op @zouyin
6. cookbook
1. appworld @jiaji P2
2. bfcl @zouyin P1
3. frozenlake @jiaji
4. simple_demo @jinli

337
doc/configuration_guide.md
View file

@ -1,337 +0,0 @@
# Configuration Guide
This document describes all available parameters for ExperienceMaker Service.
The application uses [OmegaConf](https://omegaconf.readthedocs.io/) for configuration management, supporting both YAML
files and command-line overrides.
## Configuration Loading Priority
1. Default values from `AppConfig` dataclass
2. Pre-defined YAML configuration file (default: `demo_config.yaml`)
3. Custom YAML file (if `config_path` is specified)
4. Command-line overrides
## 🏗️ Configuration Architecture
ExperienceMaker uses a layered configuration system with the following priority order:
1. **Default Configuration** (lowest priority)
2. **YAML Configuration File**
3. **Command Line Arguments** (highest priority)
## Basic Bash Usage
```bash
experiencemaker [parameter1=value1] [parameter2=value2] ...
```
## 🧩 YAML Configuration Composition
The YAML configuration file follows a specific composition pattern that enables flexible and modular configuration:
### 1. Resource Declaration
First, you declare the three core resources that form the foundation of the system:
- **`llm`**: Language model configurations
- **`embedding_model`**: Embedding model configurations
- **`vector_store`**: Vector storage configurations
In these sections, `default` (or any custom name) represents a declared configuration object that can be referenced
later:
```yaml
llm:
default: # This is a declared LLM configuration object
backend: openai_compatible
model_name: qwen3-32b
embedding_model:
default: # This is a declared embedding model configuration object
backend: openai_compatible
model_name: text-embedding-v4
vector_store:
default: # This is a declared vector store configuration object
backend: local_file
embedding_model: default
```
### 2. Operation Backend Registration
In the `op` section, each operation declares its `backend` implementation. The backend names are registered through
`@OP_REGISTRY.register()` decorator, typically converting camel-case class names to underscore format:
```yaml
op:
recall_experience_op:
backend: recall_experience_op # Registered via @OP_REGISTRY.register()
```
### 3. Resource References
Operations reference the previously declared resources using their names:
```yaml
op:
recall_experience_op:
backend: recall_experience_op
llm: default # References the declared LLM object
embedding_model: default # References the declared embedding model object
vector_store: default # References the declared vector store object
```
### 4. Pipeline
Pipeline configurations use a special syntax to define operation flows:
- `->`: Sequential execution
- `[]`: Parallel execution group
- `|`: Alternative operations within parallel group
### Examples
```yaml
# Sequential pipeline
api:
retriever: op1->op2->op3
# Parallel execution
summarizer: op1->[op2|op3|op4]->op5
# Complex pipeline with nested parallel operations
vector_store: preprocess_op->[recall_op->rerank_op|backup_op]->merge_op
```
This compositional approach enables:
- **Modularity**: Declare resources once, reference everywhere
- **Flexibility**: Mix and match different backends and configurations
- **Complexity**: Build sophisticated processing chains through pipeline syntax
## 📁 Configuration Structure
```yaml
# Service Configuration
http_service:
host: "0.0.0.0"
port: 8001
timeout_keep_alive: 600
limit_concurrency: 64
# Pipeline Definitions
api:
retriever: recall_experience_op->rerank_experience_op->rewrite_experience_op
summarizer: trajectory_preprocess_op->[success_extraction_op|failure_extraction_op]->experience_validation_op
vector_store: vector_store_action_op
# Operation Configurations
op:
operation_name:
backend: operation_name # Register through `@OP_REGISTRY.register()`, typically by converting camel-cased types into underscored names
llm: default # Optional: reference to LLM config, Register through `@LLM_REGISTRY.register()`
embedding_model: default # Optional: reference to embedding config, Register through `@EMBEDDING_MODEL_REGISTRY.register()`
vector_store: default # Optional: reference to vector store config, Register through `@VECTOR_STORE_REGISTRY.register()`
params: # Operation-specific parameters
param1: value1
param2: value2
# Resource Configurations
llm:
default:
backend: openai_compatible
model_name: qwen3-32b
params:
temperature: 0.6
embedding_model:
default:
backend: openai_compatible
model_name: text-embedding-v4
params:
dimensions: 1024
vector_store:
default:
backend: local_file
embedding_model: default
```
## Detailed Configuration Parameters
| Parameter | Type | Default Value | Description | Example |
|----------------------|--------|-----------------|----------------------------------------------------------------------|-------------------------------------------|
| `pre_defined_config` | string | `"demo_config"` | Name of the pre-defined configuration file (without .yaml extension) | `pre_defined_config=full_pipeline_config` |
| `config_path` | string | `""` | Path to custom configuration YAML file | `config_path=/path/to/config.yaml` |
## HTTP Service Configuration
| Parameter | Type | Default Value | Description | Example |
|-----------------------------------|---------|---------------|-----------------------------------|---------------------------------------|
| `http_service.host` | string | `"0.0.0.0"` | Host address for the HTTP service | `http_service.host=127.0.0.1` |
| `http_service.port` | integer | `8001` | Port number for the HTTP service | `http_service.port=8080` |
| `http_service.timeout_keep_alive` | integer | `600` | Keep-alive timeout in seconds | `http_service.timeout_keep_alive=600` |
| `http_service.limit_concurrency` | integer | `64` | Maximum concurrent connections | `http_service.limit_concurrency=128` |
## Thread Pool Configuration
| Parameter | Type | Default Value | Description | Example |
|---------------------------|---------|---------------|----------------------------------|------------------------------|
| `thread_pool.max_workers` | integer | `10` | Maximum number of worker threads | `thread_pool.max_workers=20` |
## API Pipeline Configuration
| Parameter | Type | Default Value | Description | Example |
|--------------------|--------|---------------|------------------------------------------|--------------------------------------------------------------|
| `api.retriever` | string | `""` | Pipeline definition for retriever API | `api.retriever="build_query_op->recall_vector_store_op"` |
| `api.summarizer` | string | `""` | Pipeline definition for summarizer API | `api.summarizer="simple_summary_op->update_vector_store_op"` |
| `api.vector_store` | string | `""` | Pipeline definition for vector store API | `api.vector_store="vector_store_action_op"` |
## Operation Configuration
Operations are configured using the pattern `op.{operation_name}.{parameter}`. Each operation can have the following
parameters:
| Parameter | Type | Default Value | Description | Example |
|------------------------------|--------|---------------|--------------------------------------------|------------------------------------------------------------------------------------------|
| `op.{name}.backend` | string | `""` | Backend implementation class name | `op.build_query_op.backend=build_query_op` |
| `op.{name}.prompt_file_path` | string | `""` | Path to prompt template file | `op.react_op.prompt_file_path=/path/to/prompt.yaml` |
| `op.{name}.prompt_dict` | dict | `{}` | Direct prompt configuration dictionary | `op.react_op.prompt_dict.system="You are an AI assistant"` |
| `op.{name}.llm` | string | `""` | Reference to LLM configuration | `op.react_op.llm=default` |
| `op.{name}.embedding_model` | string | `""` | Reference to embedding model configuration | `op.recall_op.embedding_model=default` |
| `op.{name}.vector_store` | string | `""` | Reference to vector store configuration | `op.recall_op.vector_store=default` |
| `op.{name}.params.{param}` | any | `{}` | Operation-specific parameters | The parameter reference is in [operations_documentation.md](operations_documentation.md) |
## LLM Configuration
| Parameter | Type | Default Value | Description | Example |
|-----------------------------|--------|---------------|----------------------------|-----------------------------------------|
| `llm.{name}.backend` | string | `""` | LLM backend implementation | `llm.default.backend=openai_compatible` |
| `llm.{name}.model_name` | string | `""` | Model name identifier | `llm.default.model_name=qwen3-32b` |
| `llm.{name}.params.{param}` | any | `{}` | LLM-specific parameters | `llm.default.params.temperature=0.6` |
## Embedding Model Configuration
| Parameter | Type | Default Value | Description | Example |
|-----------------------------------------|--------|---------------|----------------------------------------|--------------------------------------------------------|
| `embedding_model.{name}.backend` | string | `""` | Embedding model backend implementation | `embedding_model.default.backend=openai_compatible` |
| `embedding_model.{name}.model_name` | string | `""` | Embedding model name identifier | `embedding_model.default.model_name=text-embedding-v4` |
| `embedding_model.{name}.params.{param}` | any | `{}` | Model-specific parameters | `embedding_model.default.params.dimensions=1024` |
## Vector Store Configuration
| Parameter | Type | Default Value | Description | Example |
|---------------------------------------|--------|---------------|--------------------------------------------|-----------------------------------------------------------|
| `vector_store.{name}.backend` | string | `""` | Vector store backend implementation | `vector_store.default.backend=elasticsearch` |
| `vector_store.{name}.embedding_model` | string | `""` | Reference to embedding model configuration | `vector_store.default.embedding_model=default` |
| `vector_store.{name}.params.{param}` | any | `{}` | Vector store-specific parameters | `vector_store.default.params.store_dir=file_vector_store` |
## 🎯 Practical Examples
### Example 1
```bash
experiencemaker \
http_service.port=8002 \
thread_pool.max_workers=64 \
op.recall_experience_op.params.retrieve_top_k=50 \
op.rerank_experience_op.params.top_k=10 \
llm.default.params.temperature=0.1
```
### Example 2
```yaml
# dev_config.yaml
http_service:
port: 8003
api:
retriever: recall_experience_op->rerank_experience_op
op:
recall_experience_op:
params:
retrieve_top_k: 5 # Faster for development
rerank_experience_op:
params:
top_k: 3
llm:
default:
model_name: qwen-turbo
params:
temperature: 0.8
```
```bash
experiencemaker config_path=dev_config.yaml
```
### Example 3: Multi-Backend Setup
```yaml
# multi_backend_config.yaml
llm:
fast:
backend: openai_compatible
model_name: qwen-turbo
params:
temperature: 0.9
accurate:
backend: openai_compatible
model_name: gpt-4
params:
temperature: 0.1
op:
quick_extraction_op:
backend: success_extraction_op
llm: fast
detailed_validation_op:
backend: experience_validation_op
llm: accurate
params:
validation_threshold: 0.8
```
## 📋 Configuration Tips
1. **Start Simple**: Begin with the default configuration and override specific parameters
2. **Use Environment Variables**: Set API keys and URLs in `.env` file
3. **Parameter Validation**: Invalid parameters will cause startup errors with detailed messages
4. **Performance Tuning**: Adjust `retrieve_top_k`, `top_k`, and `max_workers` based on your needs
5. **Pipeline Testing**: Use simple pipelines first, then gradually add complexity
## 🔍 Troubleshooting
### Common Issues
**Configuration Not Loading:**
```bash
# Check if config file exists and has correct YAML syntax
experiencemaker config_path=/full/path/to/config.yaml
```
**Parameter Override Not Working:**
```bash
# Use exact parameter path from configuration structure
experiencemaker op.operation_name.params.parameter_name=value
```
**Pipeline Syntax Errors:**
- Check for balanced brackets `[]`
- Ensure operation names exist in `op` section
- Use `|` only within `[]` groups
---
🎯 **Advanced Configuration Mastery!** You can now create sophisticated ExperienceMaker setups tailored to your specific
needs.

59
doc/future_roadmap.md
View file

@ -1,59 +0,0 @@
# 🗺️ ExperienceMaker Future Roadmap
## P0 - Ready-to-Use Experience Libraries
We aim to build curated experience libraries for complex scenarios, providing battle-tested best practices and lessons learned rather than simple documentation aggregation.
Just as financial analysts develop analytical frameworks, senior engineers establish coding standards, and education experts create teaching methodologies, AI agents can build professional experience repositories. Start your AI projects standing on the shoulders of giants.
**Core Features:**
- [ ] Pre-built experience libraries for key domains
- [ ] Finance
- [ ] Coding
- [ ] Education
- [ ] Research
- [ ] Experience marketplace: community-driven experience sharing and exchange
## P0 - Support for Rich Experience Formats
Expert knowledge extends beyond text to include debugged code, fine-tuned toolchains, and validated workflows. We aim to integrate diverse experience carriers:
- [ ] **Executable Code**: Functions, code files, and scripts
- [ ] **Tool Integration**: APIs, MCP configurations, and tool setups
- [ ] **Pipeline Templates**: Agent execution pipelines and multi-step tool combinations
## P0 - MCP Integration
Modernize our API architecture by migrating three core APIs to the Model Context Protocol (MCP) standard for improved interoperability and standardization.
- [ ] Summarizer API
- [ ] Retriever API
- [ ] Vector Store API
## P1 - Experience Validation & Optimization
AI-powered analysis of experience usage patterns and effectiveness, with automatic quality optimization and cross-task validation feedback loops.
## P2 - Universal Trajectory Experience Extraction
### Raw Data Processing
- [ ] Automatic extraction of valuable experiences from agent execution logs
- [ ] Multimodal support: images, videos, and other formats
### Vision
Transform valuable experience data from daily work into usable insights:
- Communication techniques from emails
- Optimization insights from code commits
- Decision-making processes from meeting recordings
Enable AI to naturally become stronger through everyday work, rather than wasting real-world experience data due to format limitations.
## P2 - Open Source Experience Libraries
Democratize AI experience sharing by making curated experience libraries publicly available on Hugging Face, enabling the broader AI community to benefit from and contribute to professional experience repositories.
- [ ] **Hugging Face Integration**: Upload and maintain experience libraries on Hugging Face Hub
- [ ] **Community Contributions**: Enable community-driven experience library improvements and additions
- [ ] **Standardized Formats**: Establish standard formats for experience sharing across different domains
- [ ] **Version Control**: Implement versioning system for experience library updates and improvements

25
doc/operations_documentation.md
View file

@ -1,25 +0,0 @@
# Operations Documentation
This document provides an overview of all operations in the ExperienceMaker framework.
## Operations Overview
| Op Class | Registered Backend | Description | Parameters |
|-----------------------------|-------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `BuildQueryOp` | `build_query_op` | Constructs retrieval queries from user requests. If request.query exists, uses it directly. If only messages are provided, can either use LLM-based query construction or create a simple summary from the last 3 messages (200 chars each). Sets search query and messages in context for downstream operations. | `op.build_query_op.params.enable_llm_build = true/false` - Enable LLM-based query construction from messages. When false, creates simple summary from last 3 messages |
| `RerankExperienceOp` | `rerank_experience_op` | Performs two-stage experience reranking: (1) LLM-based intelligent reranking using relevance evaluation, (2) Score-based filtering using confidence and validation scores. Returns top-k results after filtering. Handles parsing of LLM reranking responses in JSON format with fallback to text parsing. | `op.rerank_experience_op.params.enable_llm_rerank = true` - Enable LLM-based reranking<br>`op.rerank_experience_op.params.enable_score_filter = false` - Enable score-based filtering<br>`op.rerank_experience_op.params.min_score_threshold = 0.3` - Minimum combined score threshold for filtering<br>`op.rerank_experience_op.params.top_k = 5` - Number of top experiences to return after reranking |
| `RewriteExperienceOp` | `rewrite_experience_op` | Intelligently rewrites experience context for better task relevance. Extracts current context from recent messages (last 3), formats experiences, and optionally uses LLM to rewrite context based on current query and conversation history. Handles JSON response parsing with fallback to original content. Generates structured context messages with "When to use" and "Content" sections. | `op.rewrite_experience_op.params.enable_llm_rewrite = true` - Enable LLM-based context rewriting to make experiences more relevant and actionable for current task |
| `MergeExperienceOp` | `merge_experience_op` | Formats multiple experiences into a single structured context message. Creates "Previous Experience" header followed by bullet-pointed list of experiences with "when_to_use" and "content" fields. Adds guidance text encouraging comprehensive response using helpful parts from experiences. Simple concatenation-based approach without LLM processing. | No configurable parameters |
| `TrajectoryPreprocessOp` | `trajectory_preprocess_op` | Validates and classifies trajectories based on success threshold scoring. Separates trajectories into success/failure categories and sets up context variables (success_trajectories, failure_trajectories, all_trajectories) for downstream extraction operations. Essential preprocessing step for all summarizer operations. | `op.trajectory_preprocess_op.params.success_threshold = 1.0` - Score threshold to classify trajectories as successful. Trajectories with scores >= threshold are classified as success |
| `TrajectorySegmentationOp` | `trajectory_segmentation_op` | Uses LLM to segment trajectories into meaningful step sequences based on logical breakpoints. Supports selective segmentation of success, failure, or all trajectories. Parses LLM responses in JSON format with fallback to number extraction. Stores segmentation information in trajectory metadata for downstream operations. Formats trajectory content with step numbers and role information. | `op.trajectory_segmentation_op.params.segment_target = "all"` - Which trajectories to segment ("all", "success", "failure") |
| `ExperienceValidationOp` | `experience_validation_op` | Validates extracted experiences using LLM-based quality assessment. Evaluates experiences for actionability, accuracy, relevance, clarity, and uniqueness. Uses parallel processing for efficiency. Parses JSON validation responses with score and validity flags. Filters experiences based on validation threshold and removes invalid ones with detailed logging of rejection reasons. | `op.experience_validation_op.params.validation_threshold = 0.5` - Minimum validation score threshold for experience acceptance. Experiences with scores below this threshold are filtered out |
| `ExperienceDeduplicationOp` | `experience_deduplication_op` | Removes duplicate experiences using embedding-based similarity analysis. Compares against both existing vector store experiences and current batch experiences. Calculates cosine similarity between experience embeddings and filters duplicates above similarity threshold. Handles embedding generation failures gracefully and provides detailed logging of deduplication decisions. | `op.experience_deduplication_op.params.similarity_threshold = 0.5` - Cosine similarity threshold for duplicate detection<br>`op.experience_deduplication_op.params.max_existing_experiences = 1000` - Maximum number of existing experiences to retrieve and compare against for deduplication |
| `ComparativeExtractionOp` | `comparative_extraction_op` | Extracts insights by comparing different trajectory outcomes. Supports two comparison modes: (1) Soft comparison between highest and lowest scoring trajectories, (2) Hard comparison between similar success/failure step sequences using embedding-based similarity matching. Uses parallel processing and handles trajectory segmentation data when available. | `op.comparative_extraction_op.params.enable_soft_comparison = true` - Enable highest vs lowest score comparison<br>`op.comparative_extraction_op.params.enable_similarity_comparison = false` - Enable success vs failure similarity comparison<br>`op.comparative_extraction_op.params.max_similarity_sequences = 5` - Maximum sequences to compare for similarity<br>`op.comparative_extraction_op.params.similarity_threshold = 0.3` - Similarity threshold for step sequence matching<br>`op.comparative_extraction_op.params.max_similarity_pairs = 3` - Maximum similar pairs to extract experiences from |
| `SimpleSummaryOp` | `simple_summary_op` | Generates basic experiences from individual trajectories using LLM-based analysis. Classifies trajectories as success/failure based on score threshold and creates structured experiences with when_to_use conditions and content. Parses JSON responses with robust error handling and validation. Uses parallel processing for multiple trajectories. | `op.simple_summary_op.params.success_score_threshold = 0.9` - Score threshold to classify trajectory as successful for experience extraction |
| `SuccessExtractionOp` | `success_extraction_op` | Extracts actionable experiences from successful trajectories and their segments. Processes both segmented step sequences (when available) and entire trajectories. Uses parallel processing for efficiency. Merges message content and extracts trajectory context for rich experience generation. Creates TextExperience objects with proper metadata including workspace and author information. | No configurable parameters |
| `FailureExtractionOp` | `failure_extraction_op` | Extracts learning experiences from failed trajectories to identify failure patterns and pitfalls. Similar to SuccessExtractionOp but focuses on failure analysis. Processes segmented sequences when available or entire trajectories. Uses parallel processing and creates structured experiences with proper metadata. Helps identify common failure modes and prevention strategies. | No configurable parameters |
| `UpdateVectorStoreOp` | `update_vector_store_op` | Manages vector store updates through insert and delete operations. Handles deletion of experiences by ID list and insertion of new experience lists. Converts BaseExperience objects to VectorNode format for storage. Operates on workspace-specific vector databases with detailed logging of operation sizes and IDs. Supports batch operations for efficiency. | No configurable parameters - operations controlled by experience_list and deleted_experience_ids in response context |
| `RecallVectorStoreOp` | `recall_vector_store_op` | Retrieves relevant experiences from vector store using semantic search. Performs content-based deduplication to avoid returning identical experiences. Supports optional score-based filtering to ensure quality results. Converts VectorNode results back to BaseExperience objects. Uses search query from context set by BuildQueryOp. | `op.recall_vector_store_op.params.threshold_score = <float>` - Optional minimum similarity score threshold for filtering search results. Results below this score are excluded |
| `VectorStoreActionOp` | `vector_store_action_op` | Performs administrative operations on vector store workspaces. Supports four actions: (1) copy - duplicates workspace content, (2) delete - removes entire workspace, (3) dump - exports workspace to file with experience conversion, (4) load - imports workspace from file with node conversion. Handles callback functions for data transformation during dump/load operations. | Action-specific parameters: `request.action` ("copy"/"delete"/"dump"/"load"), `request.workspace_id` (target workspace), `request.src_workspace_id` (source workspace for copy), `request.path` (file path for dump/load) |
| `ReactV1Op` | `react_v1_op` | Implements ReAct (Reasoning and Acting) agent framework for interactive problem-solving. Manages iterative reasoning-action cycles with configurable tools and step limits. Handles tool execution with parallel processing and result collection. Supports terminate tool for early stopping. Formats conversations with role prompts, tool responses, and final prompts. Includes built-in safeguards for missing tools and infinite loops. | `op.react_v1_op.params.max_steps = 10` - Maximum number of reasoning/action steps before termination<br>`op.react_v1_op.params.tool_names = "code_tool,dashscope_search_tool,terminate_tool"` - Comma-separated list of available tools from the tool registry |

559
doc/quick_start.md
View file

@ -1,559 +0,0 @@
# ExperienceMaker Quick Start Guide
This guide will help you get started with ExperienceMaker quickly using practical examples.
## 🚀 What You'll Learn
- How to set up ExperienceMaker service
- Run an agent and generate experiences
- Retrieve and apply experiences to new tasks
- Build experience-enhanced agents
## 📋 Prerequisites
- Python 3.12+
- LLM API access (OpenAI or compatible)
- Embedding model API access
## 🛠️ Installation
### Option 1: Install from PyPI (Recommended)
```bash
pip install experiencemaker
```
### Option 2: Install from Source
```bash
git clone https://github.com/modelscope/ExperienceMaker.git
cd ExperienceMaker
pip install .
```
## ⚙️ Environment Setup
Create a `.env` file in your project directory:
```bash
# Required: LLM API configuration
LLM_API_KEY="sk-xxx"
LLM_BASE_URL="https://xxx.com/v1"
# Required: Embedding model configuration
EMBEDDING_MODEL_API_KEY="sk-xxx"
EMBEDDING_MODEL_BASE_URL="https://xxx.com/v1"
# Optional: Elasticsearch configuration (if using Elasticsearch backend)
```
## 🚀 Start the Service
For testing, use the `local_file` backend:
```bash
experiencemaker \
http_service.port=8001 \
llm.default.model_name=qwen3-32b \
embedding_model.default.model_name=text-embedding-v4 \
vector_store.default.backend=local_file
```
The service will start on `http://localhost:8001`
### Elasticsearch Backend
```bash
experiencemaker \
http_service.port=8001 \
llm.default.model_name=qwen3-32b \
embedding_model.default.model_name=text-embedding-v4 \
vector_store.default.backend=elasticsearch
```
**Setup Elasticsearch:**
```bash
export ES_HOSTS="http://localhost:9200"
# Quick setup using Elastic's official script
curl -fsSL https://elastic.co/start-local | sh
```
📖 **Need Help?** Refer to [Vector Store Setup](../../doc/vector_store_setup.md) for comprehensive deployment guidance.
## 📝 Your First ExperienceMaker Script
Here's how to get started!
Note the `workspace_id` serves as your experience storage namespace. Experiences in different workspaces remain
completely isolated and cannot access each other.
### 📊 Call Summarizer Examples
Transform conversation trajectories into valuable experiences using batch summarization. Each trajectory contains:
- **Message**: Complete conversation history between user and agent
- **Score**: Performance rating (0-1 scale, where 0=failure, 1=success)
The summarizer analyzes these trajectories to extract actionable insights and patterns for future interactions.
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/summarizer", json={
"workspace_id": "test_workspace",
"traj_list": [
{"messages": [{"role": "user", "content": "hello world"}], "score": 1.0}
]
})
experience_list = response.json()["experience_list"]
for experience in experience_list:
print(experience)
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/summarizer" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"traj_list": [
{
"messages": [{"role": "user", "content": "hello world"}],
"score": 1.0
}
]
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function callSummarizer() {
try {
const response = await fetch('http://0.0.0.0:8001/summarizer', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
traj_list: [
{
messages: [{ role: "user", content: "hello world" }],
score: 1.0
}
]
})
});
const data = await response.json();
const experienceList = data.experience_list;
experienceList.forEach(experience => {
console.log(experience);
});
} catch (error) {
console.error('Error:', error);
}
}
callSummarizer();
```
</details>
### 🔍 Call Retriever Examples
Intelligently search and retrieve the most relevant experiences from your workspace to enhance decision-making. The retriever:
- **Finds** the top-k most similar experiences based on semantic similarity to your query
- **Returns** pre-assembled context ready for immediate use, or raw experience data for custom processing
- **Leverages** your workspace's accumulated knowledge to provide contextually relevant insights
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/retriever", json={
"workspace_id": "test_workspace",
"query": "what is the meaning of life?",
"top_k": 1,
})
experience_merged: str = response.json()["experience_merged"]
print(f"experience_merged={experience_merged}")
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/retriever" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"query": "what is the meaning of life?",
"top_k": 1
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function callRetriever() {
try {
const response = await fetch('http://0.0.0.0:8001/retriever', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
query: "what is the meaning of life?",
top_k: 1
})
});
const data = await response.json();
const experienceMerged = data.experience_merged;
console.log(`experience_merged=${experienceMerged}`);
} catch (error) {
console.error('Error:', error);
}
}
callRetriever();
```
</details>
### 💾 Dump Experiences From Vector Store
Export and backup your valuable experience data for archival, analysis, or migration purposes. This operation:
- **Extracts** all experiences from the specified workspace in the vector store
- **Saves** them to a structured JSONL file at `{path}/{workspace_id}.jsonl`
- **Preserves** complete experience metadata and embeddings for future restoration
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/vector_store", json={
"workspace_id": "test_workspace",
"action": "dump",
"path": "./",
})
print(response.json())
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/vector_store" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"action": "dump",
"path": "./"
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function dumpExperiences() {
try {
const response = await fetch('http://0.0.0.0:8001/vector_store', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
action: "dump",
path: "./"
})
});
const data = await response.json();
console.log(data);
} catch (error) {
console.error('Error:', error);
}
}
dumpExperiences();
```
</details>
### 📥 Load Experiences To Vector Store
Import and restore previously exported experience data to populate your workspace with existing knowledge. This operation:
- **Reads** experience data from the JSONL file located at `{path}/{workspace_id}.jsonl`
- **Reconstructs** the vector embeddings and indexes them in the specified workspace
- **Enables** immediate access to imported experiences for retrieval and decision-making
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/vector_store", json={
"workspace_id": "test_workspace",
"action": "load",
"path": "./",
})
print(response.json())
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/vector_store" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"action": "load",
"path": "./"
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function loadExperiences() {
try {
const response = await fetch('http://0.0.0.0:8001/vector_store', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
action: "load",
path: "./"
})
});
const data = await response.json();
console.log(data);
} catch (error) {
console.error('Error:', error);
}
}
loadExperiences();
```
</details>
### 🗑️ Delete Workspace
Permanently remove a workspace and all its associated experience data when it's no longer needed. This operation:
- **Removes** all experiences, embeddings, and metadata from the specified workspace
- **Frees up** storage space and computational resources
- **Cannot be undone** - ensure you've backed up important data before deletion
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/vector_store", json={
"workspace_id": "test_workspace",
"action": "delete"
})
print(response.json())
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/vector_store" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"action": "delete"
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function deleteWorkspace() {
try {
const response = await fetch('http://0.0.0.0:8001/vector_store', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
action: "delete"
})
});
const data = await response.json();
console.log(data);
} catch (error) {
console.error('Error:', error);
}
}
deleteWorkspace();
```
</details>
### 📋 Copy Workspace
Duplicate an existing workspace to create a new one with identical experience data, perfect for experimentation or branching. This operation:
- **Clones** all experiences and embeddings from the source workspace
- **Creates** a new independent workspace with the copied data
- **Preserves** original workspace while enabling safe testing and modifications in the copy
<details open>
<summary><b>Python</b></summary>
```python
import requests
response = requests.post(url="http://0.0.0.0:8001/vector_store", json={
"workspace_id": "test_workspace",
"action": "copy",
"src_workspace_id": "src_workspace"
})
print(response.json())
```
</details>
<details>
<summary><b>curl</b></summary>
```bash
curl -X POST "http://0.0.0.0:8001/vector_store" \
-H "Content-Type: application/json" \
-d '{
"workspace_id": "test_workspace",
"action": "copy",
"src_workspace_id": "src_workspace"
}'
```
</details>
<details>
<summary><b>Node.js</b></summary>
```javascript
const fetch = require('node-fetch');
// or: import fetch from 'node-fetch';
async function copyWorkspace() {
try {
const response = await fetch('http://0.0.0.0:8001/vector_store', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
workspace_id: "test_workspace",
action: "copy",
src_workspace_id: "src_workspace"
})
});
const data = await response.json();
console.log(data);
} catch (error) {
console.error('Error:', error);
}
}
copyWorkspace();
```
</details>
🎭 **Want to See It in Action?** We've prepared a [simple react agent](../../cookbook/simple_demo/simple_demo.py) that
demonstrates how to enhance agent capabilities by integrating summarizer and retriever components, achieving
significantly better performance.
## 🐛 Common Issues
### Service Won't Start
- Check if port 8001 is available
- Verify your API keys in `.env` file
- Ensure Python version is 3.12+
### No Experiences Retrieved
- Make sure you've run the summarizer first
- Check if workspace_id matches between operations
- Verify vector store backend is properly configured
### API Connection Errors
- Confirm LLM_BASE_URL and API keys are correct
- Test API access independently
- Check network connectivity
---
🎯 **You're all set!** You now have a working ExperienceMaker setup that can learn from interactions and improve over time.

0
doc/vector_store/vector_store.md → doc/vector_store_api_guide.md
View file