mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-10-09 03:20:54 +00:00
enrich readme
This commit is contained in:
jinli.yl
parent
450edfae78
commit
d8f76578b9
6 changed files with 567 additions and 313 deletions
400
README.md
400
README.md
|
|
@ -61,7 +61,6 @@ ExperienceMaker changes this by:
|
|||
- **REST API Interface**: Easy integration with existing systems
|
||||
- **Modular Pipeline Design**: Compose custom workflows from atomic operations
|
||||
- **Flexible Configuration**: YAML files and command-line overrides
|
||||
- **Comprehensive Monitoring**: Built-in logging and performance metrics
|
||||
|
||||
### 🏗️ Framework Architecture
|
||||
<p align="center">
|
||||
|
|
@ -95,83 +94,74 @@ simple_summary_op->update_vector_store_op
|
|||
|
||||
## 🛠️ Installation
|
||||
|
||||
### Prerequisites
|
||||
- Python 3.12+
|
||||
- LLM API access (openAI compatible models)
|
||||
- Embedding model API access
|
||||
|
||||
### Quick Install
|
||||
### Option 1: Install from PyPI (Recommended)
|
||||
|
||||
```bash
|
||||
# Install from PyPI (recommended)
|
||||
pip install experiencemaker
|
||||
```
|
||||
|
||||
# Or install from source
|
||||
### Option 2: Install from Source
|
||||
|
||||
```bash
|
||||
git clone https://github.com/modelscope/ExperienceMaker.git
|
||||
cd ExperienceMaker
|
||||
pip install .
|
||||
```
|
||||
|
||||
---
|
||||
## ⚙️ Environment Setup
|
||||
|
||||
## ⚡ Quick Start
|
||||
|
||||
### 1. Environment Setup
|
||||
|
||||
Configure your API credentials:
|
||||
Create a `.env` file in your project directory:
|
||||
|
||||
```bash
|
||||
# LLM Configuration
|
||||
export LLM_API_KEY="your-api-key-here"
|
||||
export LLM_BASE_URL="https://xxxx.com/v1"
|
||||
# Required: LLM API configuration
|
||||
LLM_API_KEY="sk-xxx"
|
||||
LLM_BASE_URL="https://xxx.com/v1"
|
||||
|
||||
# Embedding Model Configuration
|
||||
export EMBEDDING_MODEL_API_KEY="your-api-key-here"
|
||||
export EMBEDDING_MODEL_BASE_URL="https://xxxx.com/v1"
|
||||
# Required: Embedding model configuration
|
||||
EMBEDDING_MODEL_API_KEY="sk-xxx"
|
||||
EMBEDDING_MODEL_BASE_URL="https://xxx.com/v1"
|
||||
|
||||
# Optional: Elasticsearch configuration (if using Elasticsearch backend)
|
||||
|
||||
# Optional: Elasticsearch
|
||||
export ES_HOSTS="http://localhost:9200"
|
||||
```
|
||||
|
||||
### 2. Launch ExperienceMaker Service
|
||||
|
||||
Start with a single command:
|
||||
## 🚀 Start the Service
|
||||
|
||||
For testing, use the `local_file` backend:
|
||||
```bash
|
||||
experiencemaker \
|
||||
llm.default.model_name=gpt-4o \
|
||||
embedding_model.default.model_name=text-embedding-3-small \
|
||||
llm.default.model_name=qwen3-32b \
|
||||
embedding_model.default.model_name=text-embedding-v4 \
|
||||
vector_store.default.backend=local_file
|
||||
```
|
||||
> 📚 **Need Help?** Check our [Services Params Documentation](./doc/service_params.md) for detailed instructions.
|
||||
Refer to [Advanced Guide](./doc/advanced_guide.md) for more details.
|
||||
This guide covers advanced configuration topics including custom pipelines, operation parameters, and configuration methods.
|
||||
|
||||
The service will start on `http://localhost:8001`
|
||||
|
||||
### 3. Vector Store Setup(Optional)
|
||||
if you want to use Elasticsearch as your vector store, you can follow these steps:
|
||||
|
||||
### Elasticsearch Backend
|
||||
```bash
|
||||
vector_store.default.backend=elasticsearch
|
||||
experiencemaker \
|
||||
llm.default.model_name=qwen3-32b \
|
||||
embedding_model.default.model_name=text-embedding-v4 \
|
||||
vector_store.default.backend=elasticsearch
|
||||
```
|
||||
|
||||
**Setup Elasticsearch:**
|
||||
```bash
|
||||
# Quick setup (recommended)
|
||||
export ES_HOSTS="http://localhost:9200"
|
||||
# Quick setup using Elastic's official script
|
||||
curl -fsSL https://elastic.co/start-local | sh
|
||||
|
||||
# Verify connection
|
||||
curl http://localhost:9200/_cluster/health
|
||||
```
|
||||
Refer to [Vector Store Setup](./doc/vector_store_setup.md) for more details.
|
||||
|
||||
> 📚 **Need Help?** Check our [Vector Store Setup Guide](./doc/vector_store_quick_start.md) for detailed instructions.
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Usage Examples
|
||||
## 📝 Your First ExperienceMaker Script
|
||||
Here, load_dotenv is used to load environment variables from the .env file, or you can manually export them to the environment.
|
||||
`base_url` is the address of the ExperienceMaker service mentioned above, and workspace_id is the name of the current workspace for storing experiences.
|
||||
Experiences in different workspace_ids are not shared or accessible across workspaces.
|
||||
|
||||
### Call Summarizer Examples
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
import requests
|
||||
from dotenv import load_dotenv
|
||||
|
||||
|
|
@ -180,7 +170,7 @@ base_url = "http://0.0.0.0:8001/"
|
|||
workspace_id = "test_workspace"
|
||||
|
||||
|
||||
def run_summary(messages: list, dump_experience: bool = True):
|
||||
def run_summary(messages: list):
|
||||
response = requests.post(url=base_url + "summarizer", json={
|
||||
"workspace_id": workspace_id,
|
||||
"traj_list": [
|
||||
|
|
@ -190,9 +180,8 @@ def run_summary(messages: list, dump_experience: bool = True):
|
|||
|
||||
response = response.json()
|
||||
experience_list = response["experience_list"]
|
||||
if dump_experience:
|
||||
with open("experience.jsonl", "w") as f:
|
||||
f.write(json.dumps(experience_list, indent=2, ensure_ascii=False))
|
||||
for experience in experience_list:
|
||||
print(experience)
|
||||
```
|
||||
|
||||
### Call Retriever Examples
|
||||
|
|
@ -215,245 +204,90 @@ def run_retriever(query: str):
|
|||
response = response.json()
|
||||
experience_merged: str = response["experience_merged"]
|
||||
print(f"experience_merged={experience_merged}")
|
||||
return experience_merged
|
||||
```
|
||||
|
||||
### Vector Store Management
|
||||
### Dump Experiences From Vector Store
|
||||
|
||||
```python
|
||||
def manage_vector_store(action: str, workspace_id: str, **params):
|
||||
"""Comprehensive vector store management"""
|
||||
response = requests.post(
|
||||
f"{BASE_URL}/vector_store",
|
||||
json={
|
||||
"workspace_id": workspace_id,
|
||||
"action": action,
|
||||
**params
|
||||
}
|
||||
)
|
||||
|
||||
if response.status_code == 200:
|
||||
return response.json()
|
||||
else:
|
||||
print(f"❌ Action '{action}' failed: {response.text}")
|
||||
return None
|
||||
import requests
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Example operations
|
||||
workspace = "production_workspace"
|
||||
load_dotenv()
|
||||
base_url = "http://0.0.0.0:8001/"
|
||||
workspace_id = "test_workspace1"
|
||||
|
||||
# Create workspace
|
||||
manage_vector_store("create", workspace)
|
||||
|
||||
# Check workspace stats
|
||||
stats = manage_vector_store("stats", workspace)
|
||||
if stats:
|
||||
print(f"Workspace '{workspace}': {stats['total_experiences']} experiences")
|
||||
|
||||
# Backup experiences
|
||||
manage_vector_store("dump", workspace, path="./backup/experiences.jsonl")
|
||||
|
||||
# Restore from backup
|
||||
manage_vector_store("load", workspace, path="./backup/experiences.jsonl")
|
||||
|
||||
# Clean up workspace
|
||||
manage_vector_store("clear", workspace)
|
||||
def dump_experience():
|
||||
response = requests.post(url=base_url + "vector_store", json={
|
||||
"workspace_id": workspace_id,
|
||||
"action": "dump",
|
||||
"path": "./",
|
||||
})
|
||||
print(response.json())
|
||||
```
|
||||
|
||||
|
||||
### Advanced: Custom Pipeline Configuration
|
||||
### Load Experiences To Vector Store
|
||||
|
||||
```python
|
||||
# Create custom configuration file
|
||||
config = """
|
||||
http_service:
|
||||
host: "0.0.0.0"
|
||||
port: 8001
|
||||
import requests
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Custom retrieval pipeline
|
||||
api:
|
||||
retriever: "build_query_op->recall_experience_op->rerank_experience_op->rewrite_experience_op"
|
||||
summarizer: "trajectory_preprocess_op->success_extraction_op->experience_validation_op->experience_storage_op"
|
||||
load_dotenv()
|
||||
base_url = "http://0.0.0.0:8001/"
|
||||
workspace_id = "test_workspace1"
|
||||
|
||||
# LLM Configuration
|
||||
llm:
|
||||
default:
|
||||
backend: openai_compatible
|
||||
model_name: gpt-4o
|
||||
params:
|
||||
temperature: 0.7
|
||||
max_tokens: 4000
|
||||
|
||||
# Embedding Configuration
|
||||
embedding_model:
|
||||
default:
|
||||
backend: openai_compatible
|
||||
model_name: text-embedding-3-small
|
||||
def load_experience():
|
||||
response = requests.post(url=base_url + "vector_store", json={
|
||||
"workspace_id": "test_workspace1",
|
||||
"action": "load",
|
||||
"path": "./",
|
||||
})
|
||||
|
||||
# Vector Store Configuration
|
||||
vector_store:
|
||||
default:
|
||||
backend: elasticsearch
|
||||
embedding_model: default
|
||||
|
||||
# Operation-specific parameters
|
||||
op:
|
||||
recall_experience_op:
|
||||
params:
|
||||
retrieve_top_k: 10
|
||||
query_enhancement: true
|
||||
|
||||
rerank_experience_op:
|
||||
params:
|
||||
enable_llm_rerank: true
|
||||
top_k: 5
|
||||
min_score_threshold: 0.3
|
||||
|
||||
experience_validation_op:
|
||||
params:
|
||||
validation_threshold: 0.4
|
||||
"""
|
||||
|
||||
# Save and use custom configuration
|
||||
with open("custom_config.yaml", "w") as f:
|
||||
f.write(config)
|
||||
|
||||
# Launch with custom configuration
|
||||
# experiencemaker config_path=custom_config.yaml
|
||||
print(response.json())
|
||||
```
|
||||
|
||||
Here, we have prepared a [simple react agent](./cookbook/simple_demo/simple_demo.py) to demonstrate how to enhance its
|
||||
capabilities by integrating a summarizer and a retriever, thereby achieving better performance.
|
||||
---
|
||||
|
||||
## Experiment
|
||||
|
||||
### Experiment on Appworld
|
||||
|
||||
TODO
|
||||
|
||||
### Experiment on BFCL-V3
|
||||
|
||||
TODO
|
||||
|
||||
---
|
||||
|
||||
## Future RoadMap
|
||||
|
||||
TODO
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Configuration
|
||||
## Ready-made Experience Store
|
||||
|
||||
ExperienceMaker offers flexible configuration through YAML files and command-line parameters:
|
||||
|
||||
### Configuration Methods
|
||||
|
||||
1. **Default Configuration**: Built-in sensible defaults
|
||||
2. **YAML Configuration**: Structured configuration files
|
||||
3. **Environment Variables**: Runtime configuration
|
||||
4. **Command-line Overrides**: Dynamic parameter adjustment
|
||||
|
||||
### Key Configuration Areas
|
||||
|
||||
| Category | Description | Example |
|
||||
|----------|-------------|---------|
|
||||
| **HTTP Service** | Server host, port, timeouts | `http_service.port=8080` |
|
||||
| **LLM Models** | Model names, parameters, endpoints | `llm.default.model_name=gpt-4o` |
|
||||
| **Embedding Models** | Embedding services and dimensions | `embedding_model.default.model_name=text-embedding-3-small` |
|
||||
| **Vector Stores** | Backend type, connection settings | `vector_store.default.backend=elasticsearch` |
|
||||
| **Operations** | Pipeline configurations, thresholds | `op.rerank_experience_op.params.top_k=5` |
|
||||
|
||||
### Example Configuration Commands
|
||||
|
||||
```bash
|
||||
# Basic setup
|
||||
experiencemaker llm.default.model_name=gpt-4o vector_store.default.backend=chroma
|
||||
|
||||
# Advanced configuration
|
||||
experiencemaker \
|
||||
config_path=my_config.yaml \
|
||||
http_service.port=8002 \
|
||||
op.recall_experience_op.params.retrieve_top_k=15 \
|
||||
op.rerank_experience_op.params.enable_llm_rerank=true \
|
||||
vector_store.default.backend=elasticsearch
|
||||
```
|
||||
|
||||
> 📖 **Complete Reference**: See our [Configuration Guide](./doc/global_params.md) for all available parameters.
|
||||
TODO
|
||||
|
||||
---
|
||||
|
||||
## 🏢 Production Deployment
|
||||
## 📚 Additional Resources
|
||||
|
||||
### Docker Deployment
|
||||
|
||||
```dockerfile
|
||||
# Dockerfile
|
||||
FROM python:3.12-slim
|
||||
|
||||
WORKDIR /app
|
||||
COPY requirements.txt .
|
||||
RUN pip install -r requirements.txt
|
||||
|
||||
COPY . .
|
||||
RUN pip install .
|
||||
|
||||
EXPOSE 8001
|
||||
|
||||
CMD ["experiencemaker", "http_service.host=0.0.0.0", "vector_store.default.backend=elasticsearch"]
|
||||
```
|
||||
|
||||
### Kubernetes Configuration
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: experiencemaker
|
||||
spec:
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels:
|
||||
app: experiencemaker
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: experiencemaker
|
||||
spec:
|
||||
containers:
|
||||
- name: experiencemaker
|
||||
image: experiencemaker:latest
|
||||
ports:
|
||||
- containerPort: 8001
|
||||
env:
|
||||
- name: LLM_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: api-keys
|
||||
key: llm-api-key
|
||||
- name: ES_HOSTS
|
||||
value: "http://elasticsearch:9200"
|
||||
command: ["experiencemaker"]
|
||||
args:
|
||||
- "vector_store.default.backend=elasticsearch"
|
||||
- "http_service.host=0.0.0.0"
|
||||
```
|
||||
|
||||
### Performance Considerations
|
||||
|
||||
- **Elasticsearch**: Recommended for >100K experiences
|
||||
- **ChromaDB**: Suitable for <1M experiences
|
||||
- **Load Balancing**: Multiple service instances for high availability
|
||||
- **Caching**: Redis for frequently accessed experiences
|
||||
- **Monitoring**: Integrate with Prometheus/Grafana
|
||||
|
||||
---
|
||||
|
||||
## 📚 Documentation & Resources
|
||||
|
||||
### 📖 **Core Documentation**
|
||||
- [📋 Operations Reference](./doc/operations.md) - Complete list of all available operations
|
||||
- [⚙️ Configuration Guide](./doc/global_params.md) - Detailed parameter documentation
|
||||
- [🗄️ Vector Store Setup](./doc/vector_store_quick_start.md) - Backend setup instructions
|
||||
- [🧪 Quick Start Examples](./cookbook/simple_demo/) - Working code samples
|
||||
|
||||
### 🎓 **Learning Resources**
|
||||
- [📘 Cookbook Examples](./cookbook/) - Real-world use cases and patterns
|
||||
- [🚀 Best Practices](./cookbook/) - Production deployment guidelines
|
||||
- [🔧 Troubleshooting](./cookbook/) - Common issues and solutions
|
||||
|
||||
### 🔗 **API Reference**
|
||||
- **Retriever API**: Experience search and retrieval
|
||||
- **Summarizer API**: Trajectory processing and storage
|
||||
- **Vector Store API**: Database management operations
|
||||
- **Agent API**: ReAct-based agent execution
|
||||
- **[Vector Store Setup](./doc/vector_store_setup.md)**: Production deployment guide
|
||||
- **[Configuration Guide](./doc/configuration_guide.md)**: Advanced configuration options
|
||||
- **[Advanced Guide](./doc/advanced_guide.md)**: custom pipelines, operation parameters, and configuration methods.
|
||||
- **[Operations Documentation](./doc/operations_documentation.md)**: Advanced operations configuration
|
||||
- **[Example Collection](./cookbook)**: More practical examples
|
||||
- **[Future RoadMap](./doc/future_roadmap.md)**: Our future plans
|
||||
|
||||
---
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
We welcome contributions from the community! Here's how you can help:
|
||||
|
||||
### 🐛 **Report Issues**
|
||||
|
|
@ -475,66 +309,20 @@ We welcome contributions from the community! Here's how you can help:
|
|||
**Getting Started**: Fork the repository, create a feature branch, and submit a pull request. Please follow our coding standards and include tests for new functionality.
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Use Cases & Success Stories
|
||||
|
||||
### 🤖 **AI Agent Development**
|
||||
- **Code Generation Agents**: Learn successful coding patterns and avoid common bugs
|
||||
- **Research Assistants**: Build domain expertise through accumulated research experiences
|
||||
- **Customer Support**: Improve response quality using past successful interactions
|
||||
|
||||
### 🏢 **Enterprise Applications**
|
||||
- **Knowledge Management**: Capture and reuse organizational expertise
|
||||
- **Process Automation**: Learn optimal workflows from successful completions
|
||||
- **Decision Support**: Leverage historical decision outcomes for better choices
|
||||
|
||||
### 📊 **Data Science & Analytics**
|
||||
- **Model Development**: Learn from past experimentation results
|
||||
- **Feature Engineering**: Reuse successful feature combinations
|
||||
- **Pipeline Optimization**: Apply proven processing strategies
|
||||
|
||||
---
|
||||
|
||||
## 📄 Citation
|
||||
|
||||
If you use ExperienceMaker in your research or projects, please cite:
|
||||
|
||||
```bibtex
|
||||
@software{ExperienceMaker,
|
||||
title = {ExperienceMaker: A Comprehensive Framework for AI Agent Experience Generation and Reuse},
|
||||
author = {The ExperienceMaker Team},
|
||||
url = {https://github.com/modelscope/ExperienceMaker},
|
||||
month = {January},
|
||||
month = {08},
|
||||
year = {2025},
|
||||
note = {Version 0.1.0}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚖️ License
|
||||
|
||||
This project is licensed under the Apache License 2.0 - see the [LICENSE](./LICENSE) file for details.
|
||||
|
||||
---
|
||||
|
||||
## 🙏 Acknowledgments
|
||||
|
||||
ExperienceMaker is built with ❤️ by the team at ModelScope. Special thanks to:
|
||||
|
||||
- The open-source community for valuable feedback and contributions
|
||||
- Research teams advancing the field of AI agent learning
|
||||
- Early adopters providing real-world usage insights
|
||||
|
||||
---
|
||||
|
||||
<p align="center">
|
||||
<strong>Ready to supercharge your AI agents with experience? 🚀</strong><br>
|
||||
<a href="#-installation">Get Started Now</a> ·
|
||||
<a href="./doc/">Read the Docs</a> ·
|
||||
<a href="https://github.com/modelscope/ExperienceMaker">Star on GitHub</a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
Made with ❤️ by the <strong>ExperienceMaker Team</strong>
|
||||
</p>
|
||||
---
|
||||
310
doc/advanced_guide.md
Normal file
310
doc/advanced_guide.md
Normal file
|
|
@ -0,0 +1,310 @@
|
|||
# ExperienceMaker Advanced Configuration Guide
|
||||
|
||||
This guide covers advanced configuration topics including custom pipelines, operation parameters, and configuration
|
||||
methods.
|
||||
|
||||
## 🏗️ Configuration Architecture
|
||||
|
||||
ExperienceMaker uses a layered configuration system with the following priority order:
|
||||
|
||||
1. **Default Configuration** (lowest priority)
|
||||
2. **YAML Configuration File**
|
||||
3. **Command Line Arguments** (highest priority)
|
||||
|
||||
## 📁 Configuration Structure
|
||||
|
||||
```yaml
|
||||
# Service Configuration
|
||||
http_service:
|
||||
host: "0.0.0.0"
|
||||
port: 8001
|
||||
timeout_keep_alive: 600
|
||||
limit_concurrency: 64
|
||||
|
||||
# Pipeline Definitions
|
||||
api:
|
||||
retriever: recall_experience_op->rerank_experience_op->rewrite_experience_op
|
||||
summarizer: trajectory_preprocess_op->[success_extraction_op|failure_extraction_op]->experience_validation_op
|
||||
vector_store: vector_store_action_op
|
||||
|
||||
# Operation Configurations
|
||||
op:
|
||||
operation_name:
|
||||
backend: operation_backend
|
||||
llm: default # Optional: reference to LLM config
|
||||
embedding_model: default # Optional: reference to embedding config
|
||||
vector_store: default # Optional: reference to vector store config
|
||||
params: # Operation-specific parameters
|
||||
param1: value1
|
||||
param2: value2
|
||||
|
||||
# Resource Configurations
|
||||
llm:
|
||||
default:
|
||||
backend: openai_compatible
|
||||
model_name: qwen3-32b
|
||||
params:
|
||||
temperature: 0.6
|
||||
|
||||
embedding_model:
|
||||
default:
|
||||
backend: openai_compatible
|
||||
model_name: text-embedding-v4
|
||||
params:
|
||||
dimensions: 1024
|
||||
|
||||
vector_store:
|
||||
default:
|
||||
backend: local_file
|
||||
embedding_model: default
|
||||
```
|
||||
|
||||
## 🔧 Pipeline Configuration
|
||||
|
||||
### Pipeline Syntax
|
||||
|
||||
Pipeline configurations use a special syntax to define operation flows:
|
||||
|
||||
- `->`: Sequential execution
|
||||
- `[]`: Parallel execution group
|
||||
- `|`: Alternative operations within parallel group
|
||||
|
||||
### Examples
|
||||
|
||||
```yaml
|
||||
# Sequential pipeline
|
||||
api:
|
||||
retriever: op1->op2->op3
|
||||
|
||||
# Parallel execution
|
||||
api:
|
||||
summarizer: op1->[op2|op3|op4]->op5
|
||||
|
||||
# Complex pipeline with nested parallel operations
|
||||
api:
|
||||
retriever: preprocess_op->[recall_op->rerank_op|backup_op]->merge_op
|
||||
```
|
||||
|
||||
## ⚙️ Custom Operation Parameters
|
||||
|
||||
### Operation Configuration Structure
|
||||
|
||||
```yaml
|
||||
op:
|
||||
custom_operation:
|
||||
backend: custom_backend_name
|
||||
llm: default # Reference to LLM configuration
|
||||
vector_store: default # Reference to vector store
|
||||
params: # Custom parameters for this operation
|
||||
retrieve_top_k: 15 # Number of top results to retrieve
|
||||
similarity_threshold: 0.8 # Similarity threshold for filtering
|
||||
enable_rerank: true # Enable reranking functionality
|
||||
custom_param: "custom_value" # Any custom parameter
|
||||
```
|
||||
|
||||
### Common Operation Parameters
|
||||
|
||||
**Retrieval Operations:**
|
||||
|
||||
```yaml
|
||||
recall_experience_op:
|
||||
params:
|
||||
retrieve_top_k: 15
|
||||
similarity_threshold: 0.5
|
||||
|
||||
rerank_experience_op:
|
||||
params:
|
||||
enable_llm_rerank: true
|
||||
enable_score_filter: false
|
||||
top_k: 5
|
||||
```
|
||||
|
||||
**Extraction Operations:**
|
||||
|
||||
```yaml
|
||||
success_extraction_op:
|
||||
params:
|
||||
extraction_mode: "detailed"
|
||||
include_context: true
|
||||
|
||||
experience_validation_op:
|
||||
params:
|
||||
validation_threshold: 0.5
|
||||
strict_mode: false
|
||||
```
|
||||
|
||||
## 🚀 Configuration Methods
|
||||
|
||||
### Method 1: Custom Configuration File
|
||||
|
||||
**Step 1:** Create your configuration file
|
||||
|
||||
```yaml
|
||||
# my_custom_config.yaml
|
||||
api:
|
||||
retriever: custom_recall_op->custom_rerank_op
|
||||
|
||||
op:
|
||||
custom_recall_op:
|
||||
backend: recall_experience_op
|
||||
params:
|
||||
retrieve_top_k: 20
|
||||
similarity_threshold: 0.7
|
||||
|
||||
llm:
|
||||
default:
|
||||
model_name: gpt-4
|
||||
params:
|
||||
temperature: 0.3
|
||||
```
|
||||
|
||||
**Step 2:** Use the custom configuration
|
||||
|
||||
```bash
|
||||
experiencemaker config_path=/path/to/my_custom_config.yaml
|
||||
```
|
||||
|
||||
### Method 2: Command Line Parameters
|
||||
|
||||
Override any configuration parameter using dot notation:
|
||||
|
||||
```bash
|
||||
# Basic parameter override
|
||||
experiencemaker \
|
||||
llm.default.model_name=gpt-4 \
|
||||
embedding_model.default.model_name=text-embedding-3-large
|
||||
|
||||
# Operation parameters
|
||||
experiencemaker \
|
||||
op.recall_experience_op.params.retrieve_top_k=20 \
|
||||
op.rerank_experience_op.params.top_k=8
|
||||
|
||||
# Service configuration
|
||||
experiencemaker \
|
||||
http_service.port=8080 \
|
||||
thread_pool.max_workers=32
|
||||
|
||||
# Pipeline configuration
|
||||
experiencemaker \
|
||||
api.retriever="custom_op1->custom_op2"
|
||||
```
|
||||
|
||||
### Method 3: Hybrid Approach
|
||||
|
||||
Combine configuration file with command line overrides:
|
||||
|
||||
```bash
|
||||
experiencemaker \
|
||||
config_path=/path/to/base_config.yaml \
|
||||
llm.default.model_name=gpt-4 \
|
||||
op.recall_experience_op.params.retrieve_top_k=25
|
||||
```
|
||||
|
||||
## 🎯 Practical Examples
|
||||
|
||||
### Example 1: High-Performance Configuration
|
||||
|
||||
```bash
|
||||
experiencemaker \
|
||||
http_service.port=8002 \
|
||||
thread_pool.max_workers=64 \
|
||||
op.recall_experience_op.params.retrieve_top_k=50 \
|
||||
op.rerank_experience_op.params.top_k=10 \
|
||||
llm.default.params.temperature=0.1
|
||||
```
|
||||
|
||||
### Example 2: Development Configuration
|
||||
|
||||
```yaml
|
||||
# dev_config.yaml
|
||||
http_service:
|
||||
port: 8003
|
||||
|
||||
api:
|
||||
retriever: recall_experience_op->rerank_experience_op
|
||||
|
||||
op:
|
||||
recall_experience_op:
|
||||
params:
|
||||
retrieve_top_k: 5 # Faster for development
|
||||
|
||||
rerank_experience_op:
|
||||
params:
|
||||
top_k: 3
|
||||
|
||||
llm:
|
||||
default:
|
||||
model_name: qwen-turbo
|
||||
params:
|
||||
temperature: 0.8
|
||||
```
|
||||
|
||||
```bash
|
||||
experiencemaker config_path=dev_config.yaml
|
||||
```
|
||||
|
||||
### Example 3: Multi-Backend Setup
|
||||
|
||||
```yaml
|
||||
# multi_backend_config.yaml
|
||||
llm:
|
||||
fast:
|
||||
backend: openai_compatible
|
||||
model_name: qwen-turbo
|
||||
params:
|
||||
temperature: 0.9
|
||||
|
||||
accurate:
|
||||
backend: openai_compatible
|
||||
model_name: gpt-4
|
||||
params:
|
||||
temperature: 0.1
|
||||
|
||||
op:
|
||||
quick_extraction_op:
|
||||
backend: success_extraction_op
|
||||
llm: fast
|
||||
|
||||
detailed_validation_op:
|
||||
backend: experience_validation_op
|
||||
llm: accurate
|
||||
params:
|
||||
validation_threshold: 0.8
|
||||
```
|
||||
|
||||
## 📋 Configuration Tips
|
||||
|
||||
1. **Start Simple**: Begin with the default configuration and override specific parameters
|
||||
2. **Use Environment Variables**: Set API keys and URLs in `.env` file
|
||||
3. **Parameter Validation**: Invalid parameters will cause startup errors with detailed messages
|
||||
4. **Performance Tuning**: Adjust `retrieve_top_k`, `top_k`, and `max_workers` based on your needs
|
||||
5. **Pipeline Testing**: Use simple pipelines first, then gradually add complexity
|
||||
|
||||
## 🔍 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Configuration Not Loading:**
|
||||
|
||||
```bash
|
||||
# Check if config file exists and has correct YAML syntax
|
||||
experiencemaker config_path=/full/path/to/config.yaml
|
||||
```
|
||||
|
||||
**Parameter Override Not Working:**
|
||||
|
||||
```bash
|
||||
# Use exact parameter path from configuration structure
|
||||
experiencemaker op.operation_name.params.parameter_name=value
|
||||
```
|
||||
|
||||
**Pipeline Syntax Errors:**
|
||||
|
||||
- Check for balanced brackets `[]`
|
||||
- Ensure operation names exist in `op` section
|
||||
- Use `|` only within `[]` groups
|
||||
|
||||
---
|
||||
|
||||
🎯 **Advanced Configuration Mastery!** You can now create sophisticated ExperienceMaker setups tailored to your specific
|
||||
needs.
|
||||
|
|
@ -0,0 +1,158 @@
|
|||
# 🗺️ ExperienceMaker Future Roadmap
|
||||
|
||||
<p align="center">
|
||||
<strong>Charting the future of experience-driven AI agents</strong><br>
|
||||
<em>Building towards more intelligent, adaptive, and collaborative AI systems</em>
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Vision Statement
|
||||
|
||||
Our vision is to create the world's most comprehensive and intelligent experience learning framework for AI agents, enabling:
|
||||
- **Universal Experience Sharing** across all AI agent platforms and domains
|
||||
- **Autonomous Experience Curation** that continuously improves without human intervention
|
||||
- **Collaborative Intelligence** where agents learn from each other's experiences globally
|
||||
- **Domain-Specific Excellence** through specialized experience libraries for different fields
|
||||
|
||||
---
|
||||
|
||||
## 📅 Development Timeline
|
||||
|
||||
### 🚀 Q1 2025: Foundation Enhancement
|
||||
**Theme**: Strengthening Core Capabilities
|
||||
|
||||
#### 🔧 Enhanced Tool Integration
|
||||
- **LangChain Native Integration**: First-class support for LangChain agent frameworks
|
||||
- **AutoGen Compatibility**: Seamless integration with Microsoft's AutoGen multi-agent systems
|
||||
- **Custom Agent Framework SDK**: Simple APIs for integrating any agent framework
|
||||
- **Tool Chain Management**: Automatic tool usage pattern learning and optimization
|
||||
|
||||
#### 📊 Advanced Analytics & Monitoring
|
||||
- **Experience Usage Dashboard**: Real-time analytics on experience retrieval and application
|
||||
- **Performance Metrics Tracking**: Detailed success/failure rate analysis
|
||||
- **A/B Testing Framework**: Compare agent performance with and without specific experiences
|
||||
- **Experience Quality Scoring**: Automated assessment of experience usefulness over time
|
||||
|
||||
#### 🌐 Multi-Language Support
|
||||
- **Multilingual Experience Extraction**: Support for Chinese, Spanish, French, German, Japanese
|
||||
- **Cross-Language Experience Matching**: Find relevant experiences regardless of language
|
||||
- **Cultural Context Awareness**: Adapt experiences for different cultural contexts
|
||||
- **Translation Quality Assessment**: Ensure experience accuracy across language barriers
|
||||
|
||||
#### ⚡ Performance Optimizations
|
||||
- **Vector Search Acceleration**: 50% faster semantic search through optimized indexing
|
||||
- **Memory Footprint Reduction**: 40% reduction in RAM usage for large-scale deployments
|
||||
- **Streaming Experience Processing**: Real-time experience extraction and storage
|
||||
- **Caching Layer Implementation**: Smart caching for frequently accessed experiences
|
||||
|
||||
---
|
||||
|
||||
### 🌟 Q2-Q3 2025: Intelligence Revolution
|
||||
**Theme**: Autonomous Learning and Collaboration
|
||||
|
||||
#### 🤖 Automated Experience Curation
|
||||
- **Quality Assessment AI**: Machine learning models that automatically score experience quality
|
||||
- **Redundancy Detection**: Advanced algorithms to identify and merge similar experiences
|
||||
- **Experience Lifecycle Management**: Automatic archival of outdated or low-value experiences
|
||||
- **Dynamic Experience Categorization**: Self-organizing taxonomy that adapts to new domains
|
||||
|
||||
#### 🔄 Cross-Agent Learning Network
|
||||
- **Experience Federation Protocol**: Standard for sharing experiences across different agent instances
|
||||
- **Peer-to-Peer Experience Sharing**: Decentralized experience exchange between agents
|
||||
- **Collective Intelligence**: Aggregate learnings from multiple agents for enhanced performance
|
||||
- **Privacy-Preserving Sharing**: Secure experience sharing with differential privacy
|
||||
|
||||
#### 🎨 Visual Experience Management
|
||||
- **Web-Based Experience Explorer**: Interactive interface for browsing and managing experiences
|
||||
- **Experience Flow Visualization**: Graphical representation of experience relationships
|
||||
- **Collaborative Curation Tools**: Team-based experience review and improvement workflows
|
||||
- **Experience Impact Analytics**: Visual insights into how experiences affect agent performance
|
||||
|
||||
#### 📱 Mobile Agent Integration
|
||||
- **iOS SDK**: Native Swift library for iOS agent integration
|
||||
- **Android SDK**: Kotlin/Java library for Android applications
|
||||
- **React Native Plugin**: Cross-platform mobile development support
|
||||
- **Edge Computing Optimization**: Efficient experience processing on mobile devices
|
||||
|
||||
---
|
||||
|
||||
### 🚀 Q4 2025+: Next-Generation Intelligence
|
||||
**Theme**: Predictive and Hierarchical Learning
|
||||
|
||||
#### 🧠 Hierarchical Experience Organization
|
||||
- **Automatic Taxonomy Building**: AI-powered categorization of experiences into hierarchical structures
|
||||
- **Skill Tree Generation**: Organize experiences into skill progression pathways
|
||||
- **Domain-Specific Knowledge Graphs**: Structured representations of domain expertise
|
||||
- **Experience Dependency Mapping**: Understanding prerequisites and relationships between experiences
|
||||
|
||||
#### 🔮 Predictive Experience Engine
|
||||
- **Proactive Experience Suggestion**: Recommend relevant experiences before agents encounter problems
|
||||
- **Task Completion Prediction**: Forecast agent success probability based on available experiences
|
||||
- **Experience Gap Analysis**: Identify missing experiences needed for specific tasks
|
||||
- **Learning Path Optimization**: Suggest optimal sequences for experience acquisition
|
||||
|
||||
#### 🌍 Global Experience Ecosystem
|
||||
- **Federated Learning Networks**: Distributed training across multiple ExperienceMaker deployments
|
||||
- **Experience Marketplace**: Platform for sharing and discovering domain-specific experience packs
|
||||
- **Community-Driven Curation**: Crowdsourced experience validation and improvement
|
||||
- **Real-time Global Updates**: Instant propagation of new experiences across the network
|
||||
|
||||
#### 🎯 Domain-Specific Excellence
|
||||
- **Healthcare Experience Pack**: Specialized experiences for medical AI assistants
|
||||
- **Financial Services Pack**: Compliance-aware experiences for financial applications
|
||||
- **Software Development Pack**: Code-related experiences for programming assistants
|
||||
- **Customer Service Pack**: Communication and problem-resolution experiences
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Technical Innovations
|
||||
|
||||
### 🔬 Research Areas
|
||||
|
||||
#### Advanced Learning Mechanisms
|
||||
- **Meta-Learning Integration**: Learning how to learn more effectively from experiences
|
||||
- **Few-Shot Experience Adaptation**: Quickly adapting experiences to new contexts with minimal data
|
||||
- **Continual Learning**: Avoiding catastrophic forgetting while incorporating new experiences
|
||||
- **Transfer Learning Optimization**: Better cross-domain experience application
|
||||
|
||||
#### Next-Generation Vector Technologies
|
||||
- **Multimodal Embeddings**: Support for text, images, audio, and video experiences
|
||||
- **Dynamic Embedding Models**: Adaptive embeddings that improve with usage
|
||||
- **Quantum-Inspired Algorithms**: Explore quantum computing approaches for experience matching
|
||||
- **Neuromorphic Computing**: Brain-inspired architectures for experience processing
|
||||
|
||||
#### Advanced AI Capabilities
|
||||
- **Causal Reasoning**: Understanding cause-effect relationships in experiences
|
||||
- **Common Sense Integration**: Incorporating world knowledge into experience interpretation
|
||||
- **Emotional Intelligence**: Understanding and applying emotional context in experiences
|
||||
- **Creative Problem Solving**: Generating novel solutions by combining existing experiences
|
||||
|
||||
---
|
||||
|
||||
## 📈 Success Metrics & Milestones
|
||||
|
||||
### Key Performance Indicators
|
||||
|
||||
#### Technical Excellence
|
||||
- **Performance Improvement**: 40%+ improvement in agent task completion rates
|
||||
- **Experience Quality**: 95%+ accuracy in experience relevance scoring
|
||||
- **System Scalability**: Support for 1M+ concurrent agents
|
||||
- **Response Time**: <100ms average experience retrieval time
|
||||
|
||||
#### Community Growth
|
||||
- **Developer Adoption**: 10,000+ active developers using ExperienceMaker
|
||||
- **Experience Volume**: 1M+ high-quality experiences in the global repository
|
||||
- **Integration Partners**: 50+ official platform integrations
|
||||
- **Research Citations**: 100+ academic papers citing ExperienceMaker
|
||||
|
||||
---
|
||||
|
||||
<p align="center">
|
||||
<strong>The future of AI is collaborative, and experience is the key to unlocking it.</strong><br>
|
||||
<em>Join us in building the next generation of intelligent agents! 🚀</em>
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
*This roadmap is a living document that evolves based on community feedback, technological advances, and changing market needs. Have ideas or suggestions? [Join our community discussions](https://github.com/modelscope/ExperienceMaker/discussions)!*
|
||||
|
|
@ -68,8 +68,13 @@ export ES_HOSTS="http://localhost:9200"
|
|||
# Quick setup using Elastic's official script
|
||||
curl -fsSL https://elastic.co/start-local | sh
|
||||
```
|
||||
Refer to [Vector Store Setup](./doc/vector_store_setup.md) for more details.
|
||||
|
||||
|
||||
## 📝 Your First ExperienceMaker Script
|
||||
Here, load_dotenv is used to load environment variables from the .env file, or you can manually export them to the environment.
|
||||
`base_url` is the address of the ExperienceMaker service mentioned above, and workspace_id is the name of the current workspace for storing experiences.
|
||||
Experiences in different workspace_ids are not shared or accessible across workspaces.
|
||||
|
||||
### Call Summarizer Examples
|
||||
```python
|
||||
|
|
@ -161,13 +166,6 @@ def load_experience():
|
|||
Here, we have prepared a [simple react agent](../cookbook/simple_demo/simple_demo.py) to demonstrate how to enhance its
|
||||
capabilities by integrating a summarizer and a retriever, thereby achieving better performance.
|
||||
|
||||
## 📚 Additional Resources
|
||||
|
||||
- **[Vector Store Setup](vector_store_setup.md)**: Production deployment guide
|
||||
- **[Configuration Guide](configuration_guide.md)**: Advanced configuration options
|
||||
- **[Operations Documentation](operations_documentation.md)**: Advanced operations configuration
|
||||
- **[Example Collection](../cookbook)**: More practical examples
|
||||
|
||||
## 🐛 Common Issues
|
||||
|
||||
### Service Won't Start
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue