scriptoriumadmin/ReMe
1
0
Fork 0
mirror of https://github.com/agentscope-ai/ReMe.git synced 2026-09-06 08:16:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

add comment on quick start

Browse source
This commit is contained in:
jinli.yl 2025-07-22 18:17:35 +08:00
parent ebe8276487
commit 450edfae78
11 changed files with 61 additions and 162 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
TODO.md
View file

@ -20,7 +20,7 @@
# 兆洋
1. bedrock代码扫一下上下文管理
2. 和亮哥合作 固话sop
2. 和亮哥合作 固话sop, 自动抽取、固化小型的SOP @唤海 @贺世奇 @刺葳 @悦鸿
3. Experience列大纲，future工作
4. 周四下午和兆洋对一下

0
cookbook/appworld/__init__.py Normal file
View file

0
cookbook/bfcl/__init__.py Normal file
View file

29
doc/service_params.md → doc/configuration_guide.md
View file

@ -1,8 +1,8 @@
# Services Params Documentation
This document describes all available command-line parameters for ExperienceMaker. The application
uses [OmegaConf](https://omegaconf.readthedocs.io/) for configuration management, supporting both YAML files and
command-line overrides.
This document describes all available command-line parameters for ExperienceMaker Service.
The application uses [OmegaConf](https://omegaconf.readthedocs.io/) for configuration management, supporting both YAML
files and command-line overrides.
## Basic Usage
@ -46,22 +46,21 @@ experiencemaker [parameter1=value1] [parameter2=value2] ...
| `api.retriever` | string | `""` | Pipeline definition for retriever API | `api.retriever="build_query_op->recall_vector_store_op"` |
| `api.summarizer` | string | `""` | Pipeline definition for summarizer API | `api.summarizer="simple_summary_op->update_vector_store_op"` |
| `api.vector_store` | string | `""` | Pipeline definition for vector store API | `api.vector_store="vector_store_action_op"` |
| `api.agent` | string | `""` | Pipeline definition for agent API | `api.agent="react_op"` |
## Operation Configuration
Operations are configured using the pattern `op.{operation_name}.{parameter}`. Each operation can have the following
parameters:
| Parameter | Type | Default Value | Description | Example |
|------------------------------|--------|---------------|--------------------------------------------|------------------------------------------------------------|
| `op.{name}.backend` | string | `""` | Backend implementation class name | `op.build_query_op.backend=build_query_op` |
| `op.{name}.prompt_file_path` | string | `""` | Path to prompt template file | `op.react_op.prompt_file_path=/path/to/prompt.yaml` |
| `op.{name}.prompt_dict` | dict | `{}` | Direct prompt configuration dictionary | `op.react_op.prompt_dict.system="You are an AI assistant"` |
| `op.{name}.llm` | string | `""` | Reference to LLM configuration | `op.react_op.llm=default` |
| `op.{name}.embedding_model` | string | `""` | Reference to embedding model configuration | `op.recall_op.embedding_model=default` |
| `op.{name}.vector_store` | string | `""` | Reference to vector store configuration | `op.recall_op.vector_store=default` |
| `op.{name}.params.{param}` | any | `{}` | Operation-specific parameters | `op.build_query_op.params.enable_llm_build=false` |
| Parameter | Type | Default Value | Description | Example |
|------------------------------|--------|---------------|--------------------------------------------|------------------------------------------------------------------------------------------|
| `op.{name}.backend` | string | `""` | Backend implementation class name | `op.build_query_op.backend=build_query_op` |
| `op.{name}.prompt_file_path` | string | `""` | Path to prompt template file | `op.react_op.prompt_file_path=/path/to/prompt.yaml` |
| `op.{name}.prompt_dict` | dict | `{}` | Direct prompt configuration dictionary | `op.react_op.prompt_dict.system="You are an AI assistant"` |
| `op.{name}.llm` | string | `""` | Reference to LLM configuration | `op.react_op.llm=default` |
| `op.{name}.embedding_model` | string | `""` | Reference to embedding model configuration | `op.recall_op.embedding_model=default` |
| `op.{name}.vector_store` | string | `""` | Reference to vector store configuration | `op.recall_op.vector_store=default` |
| `op.{name}.params.{param}` | any | `{}` | Operation-specific parameters | The parameter reference is in [operations_documentation.md](operations_documentation.md) |
## LLM Configuration
@ -109,11 +108,11 @@ experiencemaker \
You can also create a YAML configuration file and override specific parameters:
1. Create a custom configuration file (`my_config.yaml`)
1. Create a custom configuration file (`xxx/my_config.yaml`)
2. Use it with command-line overrides:
```bash
experiencemaker config_path=my_config.yaml llm.default.model_name=qwen3-32b http_service.port=8080
experiencemaker config_path=xxx/my_config.yaml llm.default.model_name=qwen3-32b http_service.port=8080
```
## Parameter Validation

BIN
doc/framework.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 252 KiB

After

Width:  |  Height:  |  Size: 417 KiB

0
doc/future_roadmap.md Normal file
View file

BIN
doc/logo_v2.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.3 MiB

After

Width:  |  Height:  |  Size: 1,009 KiB

0
doc/operations.md → doc/operations_documentation.md
View file

147
doc/quick_start.md
View file

@ -117,139 +117,56 @@ def run_retriever(query: str):
print(f"experience_merged={experience_merged}")
```
## 🎯 Step-by-Step Walkthrough
### Step 1: Run Your First Agent
```bash
python demo.py
```
This will:
1. Send a query to the agent
2. Get an analysis of Tesla's business model
3. Save the conversation messages
### Step 2: Understand the Experience Generation
The agent's conversation will be processed to extract:
- **Success patterns**: What worked well in the analysis
- **Key insights**: Important findings and methodologies
- **Failure cases**: What didn't work or could be improved
### Step 3: Experience Retrieval in Action
When you ask about Apple, the system will:
1. Search for relevant experiences (Tesla analysis)
2. Find similar business analysis patterns
3. Apply learned methodologies to the new query
## 🔧 Advanced Usage
### Custom Workspace Management
### Dump Experiences
```python
def manage_workspace(action: str):
"""Manage vector store workspace"""
response = requests.post(
url=f"{base_url}/vector_store",
json={
"workspace_id": workspace_id,
"action": action
}
)
return response.json()
import requests
from dotenv import load_dotenv
# Create a new workspace
manage_workspace("create")
load_dotenv()
base_url = "http://0.0.0.0:8001/"
workspace_id = "test_workspace1"
# Clear all experiences
manage_workspace("clear")
# Dump experiences to file
requests.post(
url=f"{base_url}/vector_store",
json={
def dump_experience():
response = requests.post(url=base_url + "vector_store", json={
"workspace_id": workspace_id,
"action": "dump",
"path": "./backup/experiences.jsonl"
}
)
"path": "./",
})
print(response.json())
```
### Batch Experience Processing
### Load Experiences
```python
def batch_process_experiences(queries: list):
"""Process multiple queries and build experience base"""
all_experiences = []
for i, query in enumerate(queries):
print(f"Processing query {i+1}/{len(queries)}: {query}")
# Run agent
messages = run_agent(query)
# Generate experiences
run_summary(messages, dump_experience=False)
print(f"Processed {len(queries)} queries and built experience base")
import requests
from dotenv import load_dotenv
# Example: Build experience base for financial analysis
financial_queries = [
"Analyze Tesla's revenue streams",
"Evaluate Apple's market position",
"Assess Microsoft's competitive advantages"
]
load_dotenv()
base_url = "http://0.0.0.0:8001/"
workspace_id = "test_workspace1"
batch_process_experiences(financial_queries)
def load_experience():
response = requests.post(url=base_url + "vector_store", json={
"workspace_id": "test_workspace2",
"action": "load",
"path": "./",
})
print(response.json())
```
## 🔍 Monitoring and Debugging
### Check Service Status
```python
def check_service_health():
"""Check if ExperienceMaker service is running"""
try:
response = requests.get(f"{base_url}/health")
return response.status_code == 200
except:
return False
if not check_service_health():
print("❌ ExperienceMaker service is not running")
print("Start it with: experiencemaker vector_store.default.backend=local_file")
else:
print("✅ ExperienceMaker service is running")
```
### View Generated Files
After running the demo, you'll have:
- `messages.jsonl`: Raw conversation data
- `experience.jsonl`: Structured experiences extracted from conversations
## 🎉 What's Next?
Now that you have ExperienceMaker running:
1. **Explore Different Domains**: Try queries in different areas (technical analysis, creative writing, problem-solving)
2. **Build Domain-Specific Experience**: Create workspaces for specific use cases
3. **Integration**: Integrate ExperienceMaker into your existing agent workflows
4. **Production Deployment**: Switch to Elasticsearch for production workloads
Here, we have prepared a [simple react agent](../cookbook/simple_demo/simple_demo.py) to demonstrate how to enhance its
capabilities by integrating a summarizer and a retriever, thereby achieving better performance.
## 📚 Additional Resources
- **[Full Documentation](./README.md)**: Complete feature reference
- **[Vector Store Setup](./doc/vector_store_quick_start.md)**: Production deployment guide
- **[Configuration Guide](./doc/global_params.md)**: Advanced configuration options
- **[Example Collection](./cookbook/)**: More practical examples
- **[Vector Store Setup](vector_store_setup.md)**: Production deployment guide
- **[Configuration Guide](configuration_guide.md)**: Advanced configuration options
- **[Operations Documentation](operations_documentation.md)**: Advanced operations configuration
- **[Example Collection](../cookbook)**: More practical examples
## 🐛 Common Issues

43
doc/vector_store_quick_start.md → doc/vector_store_setup.md
View file

@ -1,19 +1,14 @@
# 🚀 Vector Store Quick Start Guide
This comprehensive guide covers all available vector store implementations in ExperienceMaker, their differences, use cases, and setup instructions.
## 📋 Overview
ExperienceMaker supports multiple vector store backends for different use cases and deployment scenarios:
- **FileVectorStore** (`backend=local_file`) - 📁 Local file-based storage for development and small datasets
- **ChromaVectorStore** (`backend=chroma`) - 🔮 Embedded vector database for local development and moderate scale
- **EsVectorStore** (`backend=elasticsearch`) - 🔍 Elasticsearch-based storage for production and large scale
## ⚡ Vector Store Implementations
### 1. 📁 FileVectorStore (`backend=local_file`)
A simple file-based vector store that saves data to local JSONL files. Perfect for development, testing, and small datasets.
#### 💡 When to Use
@ -35,10 +30,7 @@ A simple file-based vector store that saves data to local JSONL files. Perfect f
from experiencemaker.vector_store import FileVectorStore
from experiencemaker.embedding_model.openai_compatible_embedding_model import OpenAICompatibleEmbeddingModel
embedding_model = OpenAICompatibleEmbeddingModel(
dimensions=1536,
model_name="text-embedding-3-small"
)
embedding_model = OpenAICompatibleEmbeddingModel(dimensions=1024, model_name="text-embedding-v4")
vector_store = FileVectorStore(
embedding_model=embedding_model,
@ -97,10 +89,7 @@ An embedded vector database that provides persistent storage with advanced featu
from experiencemaker.vector_store import ChromaVectorStore
from experiencemaker.embedding_model.openai_compatible_embedding_model import OpenAICompatibleEmbeddingModel
embedding_model = OpenAICompatibleEmbeddingModel(
dimensions=1536,
model_name="text-embedding-3-small"
)
embedding_model = OpenAICompatibleEmbeddingModel(dimensions=1024, model_name="text-embedding-v4")
vector_store = ChromaVectorStore(
embedding_model=embedding_model,
@ -193,10 +182,7 @@ from experiencemaker.vector_store import EsVectorStore
from experiencemaker.embedding_model.openai_compatible_embedding_model import OpenAICompatibleEmbeddingModel
import os
embedding_model = OpenAICompatibleEmbeddingModel(
dimensions=1536,
model_name="text-embedding-3-small"
)
embedding_model = OpenAICompatibleEmbeddingModel(dimensions=1024, model_name="text-embedding-v4")
vector_store = EsVectorStore(
embedding_model=embedding_model,
@ -228,7 +214,6 @@ vector_store.clear_filter()
#### 💻 Example Usage
```python
import os
from experiencemaker.schema.vector_node import VectorNode
# Configure connection
@ -271,16 +256,16 @@ for result in results:
## 📊 Comparison Matrix
| Feature | FileVectorStore | ChromaVectorStore | EsVectorStore |
|---------|----------------|------------------|---------------|
| **Setup Complexity** | ⭐ Very Easy | ⭐⭐ Easy | ⭐⭐⭐⭐ Complex |
| **Scalability** | < 10K vectors | < 1M vectors | 10M+ vectors |
| **Concurrency** | Single user | Moderate | High |
| **Persistence** | JSONL files | SQLite/DuckDB | Distributed |
| **Filtering** | Basic | Advanced | Enterprise |
| **Performance** | Good for small | Good for medium | Excellent for large |
| **Resource Usage** | Minimal | Low-Medium | High |
| **Maintenance** | None | Low | High |
| **Production Ready** | ❌ | ⚠️ Limited | ✅ Yes |
| Feature | FileVectorStore | ChromaVectorStore | EsVectorStore |
|----------------------|-----------------|-------------------|---------------------|
| **Setup Complexity** | ⭐ Very Easy | ⭐⭐ Easy | ⭐⭐⭐⭐ Complex |
| **Scalability** | < 10K vectors | < 1M vectors | 10M+ vectors |
| **Concurrency** | Single user | Moderate | High |
| **Persistence** | JSONL files | SQLite/DuckDB | Distributed |
| **Filtering** | Basic | Advanced | Enterprise |
| **Performance** | Good for small | Good for medium | Excellent for large |
| **Resource Usage** | Minimal | Low-Medium | High |
| **Maintenance** | None | Low | High |
| **Production Ready** | ❌ | ⚠️ Limited | ✅ Yes |
🎉 This guide provides everything you need to get started with vector stores in ExperienceMaker. Choose the implementation that best fits your use case and scale up as needed! ✨

2
test_workspace2.jsonl
View file

File diff suppressed because one or more lines are too long