update readme

This commit is contained in:
jinli.yl 2025-07-24 22:28:23 +08:00
parent 7b7af5e697
commit 482908bc14
7 changed files with 442 additions and 353 deletions

View file

@ -25,7 +25,7 @@
---
## 📰 What's Next
## 🚀 What's Next
- **Pre-built Experience Libraries**: Domain repositories (Finance/Coding/Education/Research) + community marketplace
- **Rich Experience Formats**: Executable code/tool configs/pipeline templates/workflows
- **Experience Validation**: Quality analysis + cross-task effectiveness + auto-refinement
@ -43,7 +43,7 @@ By automatically extracting, storing, and intelligently reusing experiences from
Traditional AI agents start from scratch with every new task, wasting valuable learning opportunities.
ExperienceMaker changes this paradigm by:
- **🧠 Learning from History**: Automatically extract actionable insights from both successful and failed attempts
- **🔄 Intelligent Reuse**: Apply relevant past experiences to solve new, similar challenges more effectively
- **🔄 Intelligent Reuse**: Apply relevant experiences to solve new, similar challenges more effectively
- **📈 Continuous Improvement**: Build a growing knowledge base that makes agents progressively smarter
### ✨ Core Capabilities
@ -52,12 +52,12 @@ ExperienceMaker changes this paradigm by:
- **Success Pattern Recognition**: Identify what works and understand the underlying principles
- **Failure Analysis**: Learn from mistakes to avoid repeating them in future tasks
- **Comparative Insights**: Understand the critical differences between successful and failed approaches
- **Multi-step Trajectory Processing**: Break down complex tasks into learnable, actionable segments
- **Multistep Trajectory Processing**: Break down complex tasks into learnable, actionable segments
#### 🎯 **Smart Experience Retriever**
- **Semantic Search**: Find relevant experiences using advanced embedding models and semantic understanding
- **Context-Aware Ranking**: Prioritize the most applicable experiences for current task contexts
- **Dynamic Rewriting**: Intelligently adapt past experiences to fit new situations and requirements
- **Dynamic Rewriting**: Intelligently adapt experiences to fit new situations and requirements
- **Multi-modal Support**: Handle various input types including query, messages
#### 🗄️ **Scalable Experience Management**
@ -84,7 +84,12 @@ ExperienceMaker follows a modular, production-ready architecture designed for sc
- **🗄️ Vector Store API**: Database management and workspace operations with full CRUD support
#### ⚙️ **Processing Pipeline**
Our atomic operations can be seamlessly composed into powerful processing pipelines: custom1_op->custom2_op...
Our atomic operations can be seamlessly composed into powerful processing pipelines:
```
custom1_op->custom2_op...
```
#### 🔌 **Extensible Components**
- **LLM Integration**: OpenAI-compatible APIs with flexible model switching and provider support
@ -137,7 +142,9 @@ experiencemaker \
embedding_model.default.model_name=text-embedding-v4 \
vector_store.default.backend=local_file
```
💡 **Pro Tip**: Check out our [Advanced Guide](./doc/advanced_guide.md) for detailed configuration topics including custom pipelines, operation parameters, and advanced configuration methods.
💡 **Pro Tip**: Check out our [Configuration Guide](./doc/configuration_guide.md) for detailed configuration topics
including custom pipelines, operation parameters, and advanced configuration methods.
The service will start on `http://localhost:8001`
@ -461,6 +468,10 @@ loadExperiences();
```
</details>
💡 **Need More Advanced Operations?** For additional workspace management features(e.g. delete_workspace,
copy_workspace), advanced configuration options, and troubleshooting guidance, check out our
comprehensive [Quick Start Guide](./cookbook/simple_demo/quick_start.md).
🎭 **Want to See It in Action?** We've prepared a [simple react agent](./cookbook/simple_demo/simple_demo.py) that demonstrates how to enhance agent capabilities by integrating summarizer and retriever components, achieving significantly better performance.
---
@ -488,7 +499,52 @@ Coming Soon! Stay tuned for comprehensive evaluation results.
## 🏪 Ready-made Experience Store
Pre-built experience collections for common domains and use cases are coming soon. This will include ready-to-use experiences for web automation, data processing, API interactions, and more.
ExperienceMaker provides pre-built experience libraries to jumpstart your agent's capabilities.
You can directly load these curated experiences into your workspace and start benefiting from accumulated knowledge
immediately.
### 📦 Available Experience Libraries
- **`appworld_v1.jsonl`**: Comprehensive experiences from Appworld agent interactions, covering complex task planning
and execution patterns
- **`bfcl_v1.jsonl`**: Function calling experiences from Berkeley Function-Calling Leaderboard tasks
### 🚀 Quick Start with Pre-built Experiences
Here's how to load and use the Appworld experience library:
#### Step 1: Load Pre-built Experiences
```python
import requests
# Load Appworld experiences into your workspace
response = requests.post(url="http://0.0.0.0:8001/vector_store", json={
"workspace_id": "appworld_v1",
"action": "load",
"path": "./experience_library/",
})
print(f"loading result result={response.json()}")
```
#### Step 2: Retrieve Relevant Experiences
Now you can query the loaded experiences to get contextual guidance for your tasks:
```python
import requests
# Query for app interaction experiences
response = requests.post(url="http://0.0.0.0:8001/retriever", json={
"workspace_id": "appworld_v1",
"query": "How to navigate to settings and update user profile information?",
"top_k": 1,
})
experience_merged = response.json()["experience_merged"]
print(f"Retrieved experiences: {experience_merged}")
```
---
@ -497,7 +553,6 @@ Pre-built experience collections for common domains and use cases are coming soo
- **[Quick Start](./cookbook/simple_demo/quick_start.md)**: This guide will help you get started with ExperienceMaker quickly using practical examples.
- **[Vector Store Setup](./doc/vector_store_setup.md)**: Complete production deployment guide
- **[Configuration Guide](./doc/configuration_guide.md)**: Describes all available command-line parameters for ExperienceMaker Service
- **[Advanced Guide](./doc/advanced_guide.md)**: Custom pipelines, operation parameters, and advanced configuration methods
- **[Operations Documentation](./doc/operations_documentation.md)**: Comprehensive operations configuration reference
- **[Example Collection](./cookbook)**: Practical examples and use cases
- **[Future RoadMap](./doc/future_roadmap.md)**: Our vision and upcoming features

View file

@ -64,8 +64,8 @@ def run_retriever(query: str):
def run_agent_with_experience(query_first: str, query_second: str, dump_experience: bool = True):
# messages = run_agent(query=query_second)
# run_summary(messages, dump_experience)
messages = run_agent(query=query_second)
run_summary(messages, dump_experience)
experience_merged = run_retriever(query_first)
messages = run_agent(query=f"{experience_merged}\n\nUser Question:\n{query_first}")
return messages
@ -103,7 +103,7 @@ if __name__ == "__main__":
query1 = "Analyze Xiaomi Corporation"
query2 = "Analyze the company Tesla."
# run_agent(query=query1, dump_messages=True)
# run_agent_with_experience(query_first=query1, query_second=query2)
# dump_experience()
run_agent(query=query1, dump_messages=True)
run_agent_with_experience(query_first=query1, query_second=query2)
dump_experience()
load_experience()

View file

@ -1,310 +0,0 @@
# ExperienceMaker Advanced Configuration Guide
This guide covers advanced configuration topics including custom pipelines, operation parameters, and configuration
methods.
## 🏗️ Configuration Architecture
ExperienceMaker uses a layered configuration system with the following priority order:
1. **Default Configuration** (lowest priority)
2. **YAML Configuration File**
3. **Command Line Arguments** (highest priority)
## 📁 Configuration Structure
```yaml
# Service Configuration
http_service:
host: "0.0.0.0"
port: 8001
timeout_keep_alive: 600
limit_concurrency: 64
# Pipeline Definitions
api:
retriever: recall_experience_op->rerank_experience_op->rewrite_experience_op
summarizer: trajectory_preprocess_op->[success_extraction_op|failure_extraction_op]->experience_validation_op
vector_store: vector_store_action_op
# Operation Configurations
op:
operation_name:
backend: operation_backend
llm: default # Optional: reference to LLM config
embedding_model: default # Optional: reference to embedding config
vector_store: default # Optional: reference to vector store config
params: # Operation-specific parameters
param1: value1
param2: value2
# Resource Configurations
llm:
default:
backend: openai_compatible
model_name: qwen3-32b
params:
temperature: 0.6
embedding_model:
default:
backend: openai_compatible
model_name: text-embedding-v4
params:
dimensions: 1024
vector_store:
default:
backend: local_file
embedding_model: default
```
## 🔧 Pipeline Configuration
### Pipeline Syntax
Pipeline configurations use a special syntax to define operation flows:
- `->`: Sequential execution
- `[]`: Parallel execution group
- `|`: Alternative operations within parallel group
### Examples
```yaml
# Sequential pipeline
api:
retriever: op1->op2->op3
# Parallel execution
api:
summarizer: op1->[op2|op3|op4]->op5
# Complex pipeline with nested parallel operations
api:
retriever: preprocess_op->[recall_op->rerank_op|backup_op]->merge_op
```
## ⚙️ Custom Operation Parameters
### Operation Configuration Structure
```yaml
op:
custom_operation:
backend: custom_operation #
llm: default # Reference to LLM configuration
vector_store: default # Reference to vector store
params: # Custom parameters for this operation
retrieve_top_k: 15 # Number of top results to retrieve
similarity_threshold: 0.8 # Similarity threshold for filtering
enable_rerank: true # Enable reranking functionality
custom_param: "custom_value" # Any custom parameter
```
### Common Operation Parameters
**Retrieval Operations:**
```yaml
recall_experience_op:
params:
retrieve_top_k: 15
similarity_threshold: 0.5
rerank_experience_op:
params:
enable_llm_rerank: true
enable_score_filter: false
top_k: 5
```
**Extraction Operations:**
```yaml
success_extraction_op:
params:
extraction_mode: "detailed"
include_context: true
experience_validation_op:
params:
validation_threshold: 0.5
strict_mode: false
```
## 🚀 Configuration Methods
### Method 1: Custom Configuration File
**Step 1:** Create your configuration file
```yaml
# my_custom_config.yaml
api:
retriever: custom_recall_op->custom_rerank_op
op:
custom_recall_op:
backend: recall_experience_op
params:
retrieve_top_k: 20
similarity_threshold: 0.7
llm:
default:
model_name: gpt-4
params:
temperature: 0.3
```
**Step 2:** Use the custom configuration
```bash
experiencemaker config_path=/path/to/my_custom_config.yaml
```
### Method 2: Command Line Parameters
Override any configuration parameter using dot notation:
```bash
# Basic parameter override
experiencemaker \
llm.default.model_name=gpt-4 \
embedding_model.default.model_name=text-embedding-3-large
# Operation parameters
experiencemaker \
op.recall_experience_op.params.retrieve_top_k=20 \
op.rerank_experience_op.params.top_k=8
# Service configuration
experiencemaker \
http_service.port=8080 \
thread_pool.max_workers=32
# Pipeline configuration
experiencemaker \
api.retriever="custom_op1->custom_op2"
```
### Method 3: Hybrid Approach
Combine configuration file with command line overrides:
```bash
experiencemaker \
config_path=/path/to/base_config.yaml \
llm.default.model_name=gpt-4 \
op.recall_experience_op.params.retrieve_top_k=25
```
## 🎯 Practical Examples
### Example 1: High-Performance Configuration
```bash
experiencemaker \
http_service.port=8002 \
thread_pool.max_workers=64 \
op.recall_experience_op.params.retrieve_top_k=50 \
op.rerank_experience_op.params.top_k=10 \
llm.default.params.temperature=0.1
```
### Example 2: Development Configuration
```yaml
# dev_config.yaml
http_service:
port: 8003
api:
retriever: recall_experience_op->rerank_experience_op
op:
recall_experience_op:
params:
retrieve_top_k: 5 # Faster for development
rerank_experience_op:
params:
top_k: 3
llm:
default:
model_name: qwen-turbo
params:
temperature: 0.8
```
```bash
experiencemaker config_path=dev_config.yaml
```
### Example 3: Multi-Backend Setup
```yaml
# multi_backend_config.yaml
llm:
fast:
backend: openai_compatible
model_name: qwen-turbo
params:
temperature: 0.9
accurate:
backend: openai_compatible
model_name: gpt-4
params:
temperature: 0.1
op:
quick_extraction_op:
backend: success_extraction_op
llm: fast
detailed_validation_op:
backend: experience_validation_op
llm: accurate
params:
validation_threshold: 0.8
```
## 📋 Configuration Tips
1. **Start Simple**: Begin with the default configuration and override specific parameters
2. **Use Environment Variables**: Set API keys and URLs in `.env` file
3. **Parameter Validation**: Invalid parameters will cause startup errors with detailed messages
4. **Performance Tuning**: Adjust `retrieve_top_k`, `top_k`, and `max_workers` based on your needs
5. **Pipeline Testing**: Use simple pipelines first, then gradually add complexity
## 🔍 Troubleshooting
### Common Issues
**Configuration Not Loading:**
```bash
# Check if config file exists and has correct YAML syntax
experiencemaker config_path=/full/path/to/config.yaml
```
**Parameter Override Not Working:**
```bash
# Use exact parameter path from configuration structure
experiencemaker op.operation_name.params.parameter_name=value
```
**Pipeline Syntax Errors:**
- Check for balanced brackets `[]`
- Ensure operation names exist in `op` section
- Use `|` only within `[]` groups
---
🎯 **Advanced Configuration Mastery!** You can now create sophisticated ExperienceMaker setups tailored to your specific
needs.

View file

@ -1,6 +1,6 @@
# Services Params Documentation
# Configuration Guide
This document describes all available command-line parameters for ExperienceMaker Service.
This document describes all available parameters for ExperienceMaker Service.
The application uses [OmegaConf](https://omegaconf.readthedocs.io/) for configuration management, supporting both YAML
files and command-line overrides.
@ -17,6 +17,162 @@ experiencemaker [parameter1=value1] [parameter2=value2] ...
3. Custom YAML file (if `config_path` is specified)
4. Command-line overrides
## 🏗️ Configuration Architecture
ExperienceMaker uses a layered configuration system with the following priority order:
1. **Default Configuration** (lowest priority)
2. **YAML Configuration File**
3. **Command Line Arguments** (highest priority)
## 🧩 YAML Configuration Composition
The YAML configuration file follows a specific composition pattern that enables flexible and modular configuration:
### 1. Resource Declaration
First, you declare the three core resources that form the foundation of the system:
- **`llm`**: Language model configurations
- **`embedding_model`**: Embedding model configurations
- **`vector_store`**: Vector storage configurations
In these sections, `default` (or any custom name) represents a declared configuration object that can be referenced
later:
```yaml
llm:
default: # This is a declared LLM configuration object
backend: openai_compatible
model_name: qwen3-32b
embedding_model:
default: # This is a declared embedding model configuration object
backend: openai_compatible
model_name: text-embedding-v4
vector_store:
default: # This is a declared vector store configuration object
backend: local_file
embedding_model: default
```
### 2. Operation Backend Registration
In the `op` section, each operation declares its `backend` implementation. The backend names are registered through
`@OP_REGISTRY.register()` decorator, typically converting camel-case class names to underscore format:
```yaml
op:
recall_experience_op:
backend: recall_experience_op # Registered via @OP_REGISTRY.register()
```
### 3. Resource References
Operations reference the previously declared resources using their names:
```yaml
op:
recall_experience_op:
backend: recall_experience_op
llm: default # References the declared LLM object
embedding_model: default # References the declared embedding model object
vector_store: default # References the declared vector store object
```
### 4. Pipeline Composition
Finally, using the declared operations, you can compose complex pipelines through nested structures and parallel
execution patterns:
```yaml
api:
# Complex summarizer chain with parallel operations
summarizer: trajectory_preprocess_op->[success_extraction_op|failure_extraction_op]->experience_validation_op
# Nested retriever pipeline
retriever: recall_experience_op->rerank_experience_op->rewrite_experience_op
```
This compositional approach enables:
- **Modularity**: Declare resources once, reference everywhere
- **Flexibility**: Mix and match different backends and configurations
- **Complexity**: Build sophisticated processing chains through pipeline syntax
## 📁 Configuration Structure
```yaml
# Service Configuration
http_service:
host: "0.0.0.0"
port: 8001
timeout_keep_alive: 600
limit_concurrency: 64
# Pipeline Definitions
api:
retriever: recall_experience_op->rerank_experience_op->rewrite_experience_op
summarizer: trajectory_preprocess_op->[success_extraction_op|failure_extraction_op]->experience_validation_op
vector_store: vector_store_action_op
# Operation Configurations
op:
operation_name:
backend: operation_name # Register through `@OP_REGISTRY.register()`, typically by converting camel-cased types into underscored names
llm: default # Optional: reference to LLM config, Register through `@LLM_REGISTRY.register()`
embedding_model: default # Optional: reference to embedding config, Register through `@EMBEDDING_MODEL_REGISTRY.register()`
vector_store: default # Optional: reference to vector store config, Register through `@VECTOR_STORE_REGISTRY.register()`
params: # Operation-specific parameters
param1: value1
param2: value2
# Resource Configurations
llm:
default:
backend: openai_compatible
model_name: qwen3-32b
params:
temperature: 0.6
embedding_model:
default:
backend: openai_compatible
model_name: text-embedding-v4
params:
dimensions: 1024
vector_store:
default:
backend: local_file
embedding_model: default
```
## 🔧 Pipeline Configuration
### Pipeline Syntax
Pipeline configurations use a special syntax to define operation flows:
- `->`: Sequential execution
- `[]`: Parallel execution group
- `|`: Alternative operations within parallel group
### Examples
```yaml
# Sequential pipeline
api:
retriever: op1->op2->op3
# Parallel execution
summarizer: op1->[op2|op3|op4]->op5
# Complex pipeline with nested parallel operations
vector_store: preprocess_op->[recall_op->rerank_op|backup_op]->merge_op
```
## Basic Configuration Parameters
| Parameter | Type | Default Value | Description | Example |
@ -86,38 +242,226 @@ parameters:
| `vector_store.{name}.embedding_model` | string | `""` | Reference to embedding model configuration | `vector_store.default.embedding_model=default` |
| `vector_store.{name}.params.{param}` | any | `{}` | Vector store-specific parameters | `vector_store.default.params.store_dir=file_vector_store` |
## Complete Example
## ⚙️ Custom Operation Parameters
Here's a complete example showing how to configure the entire system:
### Operation Configuration Structure
```yaml
op:
custom_operation:
backend: custom_operation # The backend names are registered through `@OP_REGISTRY.register()` decorator, typically converting camel-case class names to underscore format
llm: default # Reference to LLM configuration
vector_store: default # Reference to vector store
params: # Custom parameters for this operation
retrieve_top_k: 15 # Number of top results to retrieve
similarity_threshold: 0.8 # Similarity threshold for filtering
enable_rerank: true # Enable reranking functionality
custom_param: "custom_value" # Any custom parameter
```
### Common Operation Parameters
**Retrieval Operations:**
```yaml
recall_experience_op:
params:
retrieve_top_k: 15
similarity_threshold: 0.5
rerank_experience_op:
params:
enable_llm_rerank: true
enable_score_filter: false
top_k: 5
```
**Extraction Operations:**
```yaml
success_extraction_op:
params:
extraction_mode: "detailed"
include_context: true
experience_validation_op:
params:
validation_threshold: 0.5
strict_mode: false
```
## 🚀 Configuration Methods
### Method 1: Custom Configuration File
**Step 1:** Create your configuration file
```yaml
# my_custom_config.yaml
api:
retriever: custom_recall_op->custom_rerank_op
op:
custom_recall_op:
backend: recall_experience_op
params:
retrieve_top_k: 20
similarity_threshold: 0.7
llm:
default:
model_name: gpt-4
params:
temperature: 0.3
```
**Step 2:** Use the custom configuration
```bash
experiencemaker config_path=/path/to/my_custom_config.yaml
```
### Method 2: Command Line Parameters
Override any configuration parameter using dot notation:
```bash
# Basic parameter override
experiencemaker \
llm.default.model_name=gpt-4 \
embedding_model.default.model_name=text-embedding-3-large
# Operation parameters
experiencemaker \
op.recall_experience_op.params.retrieve_top_k=20 \
op.rerank_experience_op.params.top_k=8
# Service configuration
experiencemaker \
http_service.port=8080 \
thread_pool.max_workers=32
# Pipeline configuration
experiencemaker \
api.retriever="custom_op1->custom_op2"
```
### Method 3: Hybrid Approach
Combine configuration file with command line overrides:
```bash
experiencemaker \
http_service.port=8080 \
thread_pool.max_workers=20 \
llm.default.backend=openai_compatible \
llm.default.model_name=qwen3-32b \
llm.default.params.temperature=0.6 \
embedding_model.default.backend=openai_compatible \
embedding_model.default.model_name=text-embedding-v4 \
embedding_model.default.params.dimensions=1024 \
vector_store.default.backend=elasticsearch \
vector_store.default.embedding_model=default \
config_path=/path/to/base_config.yaml \
llm.default.model_name=gpt-4 \
op.recall_experience_op.params.retrieve_top_k=25
```
## Configuration File vs Command Line
## 🎯 Practical Examples
You can also create a YAML configuration file and override specific parameters:
1. Create a custom configuration file (`xxx/my_config.yaml`)
2. Use it with command-line overrides:
### Example 1: High-Performance Configuration
```bash
experiencemaker config_path=xxx/my_config.yaml llm.default.model_name=qwen3-32b http_service.port=8080
experiencemaker \
http_service.port=8002 \
thread_pool.max_workers=64 \
op.recall_experience_op.params.retrieve_top_k=50 \
op.rerank_experience_op.params.top_k=10 \
llm.default.params.temperature=0.1
```
## Parameter Validation
### Example 2: Development Configuration
- All parameters are validated according to their types
- Referenced configurations (like `llm`, `embedding_model`, `vector_store`) must exist
- Backend implementations must be registered in their respective registries
- Nested parameters use dot notation for access
```yaml
# dev_config.yaml
http_service:
port: 8003
api:
retriever: recall_experience_op->rerank_experience_op
op:
recall_experience_op:
params:
retrieve_top_k: 5 # Faster for development
rerank_experience_op:
params:
top_k: 3
llm:
default:
model_name: qwen-turbo
params:
temperature: 0.8
```
```bash
experiencemaker config_path=dev_config.yaml
```
### Example 3: Multi-Backend Setup
```yaml
# multi_backend_config.yaml
llm:
fast:
backend: openai_compatible
model_name: qwen-turbo
params:
temperature: 0.9
accurate:
backend: openai_compatible
model_name: gpt-4
params:
temperature: 0.1
op:
quick_extraction_op:
backend: success_extraction_op
llm: fast
detailed_validation_op:
backend: experience_validation_op
llm: accurate
params:
validation_threshold: 0.8
```
## 📋 Configuration Tips
1. **Start Simple**: Begin with the default configuration and override specific parameters
2. **Use Environment Variables**: Set API keys and URLs in `.env` file
3. **Parameter Validation**: Invalid parameters will cause startup errors with detailed messages
4. **Performance Tuning**: Adjust `retrieve_top_k`, `top_k`, and `max_workers` based on your needs
5. **Pipeline Testing**: Use simple pipelines first, then gradually add complexity
## 🔍 Troubleshooting
### Common Issues
**Configuration Not Loading:**
```bash
# Check if config file exists and has correct YAML syntax
experiencemaker config_path=/full/path/to/config.yaml
```
**Parameter Override Not Working:**
```bash
# Use exact parameter path from configuration structure
experiencemaker op.operation_name.params.parameter_name=value
```
**Pipeline Syntax Errors:**
- Check for balanced brackets `[]`
- Ensure operation names exist in `op` section
- Use `|` only within `[]` groups
---
🎯 **Advanced Configuration Mastery!** You can now create sophisticated ExperienceMaker setups tailored to your specific
needs.

View file

@ -49,10 +49,10 @@ Enable AI to naturally become stronger through everyday work, rather than wastin
- [ ] cook_book-bfcl-v3 op @zouyin delay 0730
- [x] fix multi-process bug @jinli
- [ ] logo optimize @jiaji
- [ ] Ready-made Experience Store @jinli, add appworld/bfcl-v3 default experience store @jiaji
- [x] Ready-made Experience Store @jinli, add appworld/bfcl-v3 default experience store @jiaji
- [ ] op config make up @jiaji
- [ ] config make up, easy to understand @jinli
- [ ] refine readme @jinli
- [x] config make up, easy to understand @jinli
- [x] refine readme @jinli
- [ ] integrate into beyond-agent @jinli
- [ ] rm workspace_id in code
- [x] integrate into beyond-agent @jinli
- [ ] rm workspace_id in code @jinli