scriptoriumadmin/ReMe
1
0
Fork 0
mirror of https://github.com/agentscope-ai/ReMe.git synced 2026-10-09 03:20:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

update readme

Browse source
This commit is contained in:
jinli.yl 2025-06-17 20:09:06 +08:00
parent bfd867f277
commit 0077050050
2 changed files with 70 additions and 923 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

102
README.md
View file

@ -1,51 +1,89 @@
# run service
# Quick Start
## Hello Experience Maker
Here is a simple user guide for ExperienceMaker.
### Step0: Preparation Work
#### Prepare LLM & EMBEDDING_MODEL
We need to prepare the API services for the LLM and the Embedding model.
Since we are using an OpenAI-compatible service, we only need to write the `OPENAI_API_KEY` and `OPENAI_BASE_URL` into the environment.
```shell
cd BeyondAgent
python beyondagent/core/service/model_service.py
export OPENAI_API_KEY="sk-xxx"
export OPENAI_BASE_URL="xxx"
```
# test service
#### Prepare Vector Store
If you want to use vector store, you need to set up a vector database. Don't forget to set up the `ES_HOSTS`.
- Elasticsearch [quick start](../vector_store/elasticsearch.md)
```shell
python beyondagent/test/test_service.py
#### Prepare Your Own Agent
Assume you have a runnable agent.
Here, we use a basic LLM combined with a simple react framework including three tools(code, web_search, terminate) as an example.
```python
class YourOwnAgent(...):
...
def think(self, **kwargs) -> bool:
...
def act(self, **kwargs):
...
def run(self, query: str, previous_experience: str):
...
```
# qingxu
Here is an [example code](./your_own_agent.py) with [prompt](./your_own_agent_prompt.yaml). To use this simple agent, you will need to set up `DASHSCOPE_API_KEY`.
```shell
# 1. edit query, port, vm etc
nano docker-compose.yml
# 2. run
docker compose down && docker compose build && docker compose up
# 3. then watch vm at http://localhost:16901 (default password is headless)
export DASHSCOPE_API_KEY="sk-xxx"
```
# vector store
If a vector database is involved, you will need an Elasticsearch environment. You can refer to the following steps:
- If you don’t have Docker installed, download and install [Docker Desktop](https://www.docker.com/products/docker-desktop) for your operating system.
- To set up [Elasticsearch](https://www.elastic.co/docs/solutions/search/run-elasticsearch-locally) and Kibana locally, run the start-local script in the command line:
### Step1: Start ExperienceMaker Http Service
- Install dependencies.
```shell
curl -fsSL https://elastic.co/start-local | sh
pip install .
```
Or manually download and load the image. Here, we take elasticsearch-wolfi:9.0.0 as an example:
- We start our context and summary services in `simple` mode. Next, you can use standard HTTP interfaces to call the services.
```shell
docker pull docker.elastic.co/elasticsearch/elasticsearch-wolfi:9.0.0
docker run -p 9200:9200 \
--memory='4GB' \
-e "discovery.type=single-node" \
-e "xpack.security.enabled=false" \
-e "xpack.license.self_generated.type=trial" \
docker.elastic.co/elasticsearch/elasticsearch-wolfi:9.0.0
pip install .
python -m experiencemaker.em_service \
--port=8001 \
--llm='{"backend": "openai_compatible", "model_name": "qwen3-32b", "temperature": 0.6}' \
--embedding_model='{"backend": "openai_compatible", "model_name": "text-embedding-v4", "dimensions": 1024}' \
--vector_store='{"backend": "elasticsearch"}' \
--context_generator='{"backend": "simple"}' \
--summarizer='{"backend": "simple"}'
```
# run module service
### Step2: Enhance Your Own Agent
[] 注册函数
Call the capabilities of ContextGenerator and Summarizer through `EMClient`.
[] 更新workspace_id + index_name
[] trajectory 加 reward
[] summary去重，context加LLM eval
[]
```python
from experiencemaker.em_client import EMClient
em_client = EMClient(base_url="http://0.0.0.0:8001")
```
Assume you have a list of messages generated by an agent; you can use the summarizer to generate experiences from them.
```python
request = SummarizerRequest(
trajectories=[Trajectory(query=query, steps=messages, answer=messages[-1].content, done=True)],
workspace_id="w_1234")
response = em_client.call_summarizer(request)
for experience in response.experiences:
print(experience.model_dump_json())
```
Assume you have a query; you can use the context generator to generate a new query from the query and the related
experiences.
```python
request = ContextGeneratorRequest(trajectory=Trajectory(query=query), retrieve_top_k=1, workspace_id="w_1234")
response = em_client.call_context_generator(request)
new_query = f"{response.context_msg.content}\n\nUser Question\n{query}"
```
The complete code can be found in the implementation of [YourOwnAgentEnhanced](./your_own_agent_enhanced.py).

891
test.py
View file

@ -1,891 +0,0 @@
msg = [
"""
end-user
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
In the paper `Formation control with collision avoidance through deep reinforcement learning using model-guided demonstration`,
what existing methods are mentioned for the problem of formation maintenance?
""",
"""
commander
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
You are the central commander, the brain of a coordinated multi-agent system. Your job is to make plans to solve a task given by
an end-user:
<task>
In the paper `Formation control with collision avoidance through deep reinforcement learning using model-guided demonstration`,
what existing methods are mentioned for the problem of formation maintenance?
</task>
However, you should not solve this task directly, instead, you should decompose the task into sub-tasks, and present them in a
markdown to-do list format. As an example, here is a to-do list for user question to "Get the GDP of the country/countries with
highest rare earth production in 2024":
<analysis>
... analysis of the task ...
</analysis>
<todo_list>
- [ ] Get the basic concept of what rare earth is, and whether it has direct relationship with GDP
- [ ] Get the rare earth production of each country in 2024, select the country with highest rare earth production
- [ ] Get the GDP of that country in 2024
</todo_list>
<next_step>
Get the GDP of that country in 2024
</next_step>
[IMPORTANT] The to-do list should be clear, short and simple. The number of to-be-completed sub-tasks should be no more than 3.
[IMPORTANT] Always use English to answer the question.
[IMPORTANT] There are 3 cases:
for this case, the output should be:
<analysis>
... 分析问题 ...
</analysis>
<todo_list>
...
</todo_list>
""",
"""
assistant
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Okay, so the user wants to know what existing methods are mentioned in the paper "Formation control with collision avoidance
through deep reinforcement learning using model-guided demonstration" for formation maintenance. Let me break this down.
First, I need to understand the question. The user is asking about existing methods for formation maintenance as mentioned in that
specific paper. My task isn't to answer it directly but to decompose it into subtasks. The example given uses a to-do list with up
to three steps, so I should follow that structure.
The first step would be to identify the key concepts. Formation maintenance is the problem area, and the paper is proposing a
method using deep reinforcement learning with model-guided demonstrations. But the user wants existing methods mentioned in the
paper, not the proposed one. So the paper probably reviews related work or existing approaches before introducing their own method
.
So subtask one: Get the basic concept of formation maintenance to understand what the problem entails. That makes sense because
knowing the problem helps in identifying the methods.
Subtask two: Access the paper and locate the sections where existing methods are discussed. Typically, in research papers, there's
a related work or literature review section that discusses existing approaches. The user needs to find those sections to extract
the mentioned methods.
Third subtask: Extract and list the existing methods from the paper. Once the relevant sections are located, the next step is to
compile the list of methods they reference.
I need to make sure the steps are clear and simple. The user might not have access to the paper, so maybe accessing it is part of
the steps. But if the assistant can't access external documents, how would it proceed? Wait, the example given in the problem
includes steps like getting GDP data which might require external sources. But in this case, the user is asking about a specific
paper, so the assumption here is that the assistant has access to the paper or can retrieve the information from it. However, in
reality, I don't have access to the paper. But since this is a hypothetical scenario for decomposition, I can outline the steps as
if the assistant can access the paper.
So the to-do list would be:
1. Understand formation maintenance basics.
2. Access the paper and find sections discussing existing methods.
3. Extract the listed existing methods.
That's three steps, which fits the requirement. The next step would be the last one, extracting the methods. But the user just
needs the decomposition, not the execution. So the answer should present this structure.
<analysis>
The task requires identifying existing methods for formation maintenance as referenced in the specified paper. This involves
understanding the problem domain, locating the relevant sections in the paper, and extracting the cited methods.
</analysis>
<todo_list>
- [ ] Understand the concept of formation maintenance to contextualize the problem
- [ ] Access the paper and locate its literature review/related work section discussing existing methods
- [ ] Extract and list the existing methods mentioned in the paper for formation maintenance
</todo_list>
""",
"""
commander
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
You're a helpful assistant named tool_exe_agent.
## What You Should Do:
Look back at the previous problem-solving steps and decide which external tool to use.
You should only focus on the very next unfinished step on the todo list. Do not look forward!
## Note:
1. Do not make any assumptions about the tool call result, keep your thought and analysis short and concise.
2. Forget everything you know, only information given by the tools are reliable.
3. Keep your thought short and concise.
4. Do not make up fake information.
5. If you need to write python code, please use the `CodeExecutionTool` to execute it. And the `code` argument should use loguru
to print the result. For example:
```python
from loguru import logger
from ... (other imports) ...
def you_function_name():
# your code
if __name__ == "__main__":
try:
logger.success(you_function_name())
except Exception as e:
logger.exception(e)
```
6. If there are tool-call histories, consider their results, do not repeat failure over and over again.
7. Respect to todo list! Respect to todo list! Respect to todo list! if there is a todo list, you must follow it.
For a example todo list:
<todo_list>
- [x] job 1 (`x` means completed)
- [ ] job 2
- [ ] job 3
</todo_list>
You must focus on job 2 only.
8. When you use chrome related tools, do not ask for any confirmation, just ask for materials.
9. <tool_and_arguments> field should be a JSON object. For example: {"name": "RealChromeBrowserUse", "arguments": {"task": "在
小红书中搜索杭州一日游攻略。"}}
## Tool names and arguments:
[{"type": "function", "function": {"name": "CodeExecutionTool", "description": "用于执行Python代码的工具，可以返回执行结果、打印
结果和错误Traceback。调用时必须使用loguru打印信息。", "parameters": {"properties": {"tool_name": {"default": "CodeExecutionTool",
"description": "用于执行Python代码的工具，可以返回执行结果、打印结果和错误Traceback。调用时必须使用loguru打印信息。", "title": "
Tool Name", "type": "string"}, "code": {"default": "", "description": "要执行的Python代码", "title": "Code", "type": "string"}}, "
title": "CodeExecutionTool", "type": "object"}}}, {"type": "function", "function": {"name": "RealChromeBrowserUse", "description":
"通过真正的Chrome浏览器搜索网页，效率略低，但更稳定，可以访问更广泛的网站。", "parameters": {"properties": {"tool_name": {"default
": "RealChromeBrowserUse", "description": "通过真正的Chrome浏览器搜索网页，效率略低，但更稳定，可以访问更广泛的网站。", "title": "
Tool Name", "type": "string"}, "task": {"default": "", "description": "一句话，描述一个明确的子任务（task）。例如：'获取2024年5月1
日阿里巴巴集团控股有限公司的收盘价'。", "title": "Task", "type": "string"}, "startup_url": {"anyOf": [{"type": "string"}, {"type":
"null"}], "default": "", "description": "起始url（非必要参数，默认留空）。可以指定一个url作为任务的起点，从而提高效率。", "title":
"Startup Url"}, "additional_tip": {"anyOf": [{"type": "string"}, {"type": "null"}], "default": "", "description": "执行浏览器任务
时的一些建议（非必要参数，默认留空）。", "title": "Additional Tip"}}, "title": "RealChromeBrowserUse", "type": "object"}}}, {"type
": "function", "function": {"name": "GetDownloadedFiles", "description": "获取下载完的文件。", "parameters": {"properties": {"tool
_name": {"default": "GetDownloadedFiles", "description": "获取下载完的文件。", "title": "Tool Name", "type": "string"}, "file_path
": {"default": "", "description": "文件的路径。", "title": "File Path", "type": "string"}}, "title": "GetDownloadedFiles", "type":
"object"}}}]
## Answer Format:
<think>
... analyze which tool should be used ...
</think>
<objective>
... next objective from the todo list ...
</objective>
<tool_and_arguments>
{"name": ..., "arguments": ...}
</tool_and_arguments>
""",
"""
tool-agent
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
<think>
The next step is to access the paper and locate sections discussing existing methods. Since the paper is not provided directly
, an external tool like a browser can be used to search for and access the paper.
</think>
<objective>
Access the paper and find sections discussing existing methods.
</objective>
<tool_and_arguments>
{"name": "RealChromeBrowserUse", "arguments": {"task": "Search for the paper 'Formation control with collision avoidance
through deep reinforcement learning using model-guided demonstration' and locate sections discussing existing methods."}}
</tool_and_arguments>[{'name': 'RealChromeBrowserUse', 'arguments': {'task': "Search for the paper 'Formation control with
collision avoidance through deep reinforcement learning using model-guided demonstration' and locate sections discussing existing
methods."}}]
""",
"""
tool
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
# The selected tools
{'name': 'RealChromeBrowserUse', 'arguments': {'task': "Search for the paper 'Formation control with collision avoidance through
deep reinforcement learning using model-guided demonstration' and locate sections discussing existing methods.", 'mcp_kwargs': {}}
}
# The toolcall result
[field 'model_output']
[field 'current_state']
[field 'evaluation_previous_goal']
Success - Successfully extracted content from the IEEE page which reveals discussion of existing methods
[field 'memory']
Found key information about existing methods in the abstract and introduction. The paper mentions that 'most of the existing
works are simple combinations of methods dealing with the two problems separately' - referring to formation maintenance and
collision avoidance. The introduction section provides context about multiagent systems and formation control challenges. However,
access to the full paper requires a subscription. Step 13/16.
[field 'next_goal']
Since we found the key discussion about existing methods but cannot access the full text due to subscription requirement, we
should summarize our findings and complete the task
[field 'action']
[field 'done']
[field 'text']
I found the paper and located discussions about existing methods. From the accessible content, the key points about
existing methods are:
1. Most existing works handle formation maintenance and collision avoidance as separate problems, simply combining
different methods for each issue.
2. The paper identifies this as a limitation, as the followers need to handle both formation maintenance and collision
avoidance simultaneously.
3. The introduction provides context that formation control is a fundamental problem in multiagent systems, where both
formation maintenance and collision avoidance are essential considerations.
4. The authors propose their new method specifically to address the limitations of existing approaches by using deep
reinforcement learning to handle both aspects in an integrated way.
While more detailed discussion of existing methods likely exists in the full paper, access requires a subscription. The
abstract and introduction sections provide the key critique that existing methods typically separate the two problems rather than
addressing them in an integrated way.
[field 'success']
True
[field 'result']
[field 'extracted_content']
I found the paper and located discussions about existing methods. From the accessible content, the key points about existing
methods are:
1. Most existing works handle formation maintenance and collision avoidance as separate problems, simply combining different
methods for each issue.
2. The paper identifies this as a limitation, as the followers need to handle both formation maintenance and collision
avoidance simultaneously.
3. The introduction provides context that formation control is a fundamental problem in multiagent systems, where both
formation maintenance and collision avoidance are essential considerations.
4. The authors propose their new method specifically to address the limitations of existing approaches by using deep
reinforcement learning to handle both aspects in an integrated way.
While more detailed discussion of existing methods likely exists in the full paper, access requires a subscription. The
abstract and introduction sections provide the key critique that existing methods typically separate the two problems rather than
addressing them in an integrated way.
[field 'state']
[field 'url']
https://ieeexplore.ieee.org/abstract/document/9142429
[field 'title']
Formation Control With Collision Avoidance Through Deep Reinforcement Learning Using Model-Guided Demonstration | IEEE
Journals & Magazine | IEEE Xplore
[field 'page_meta']
[field 'content']
Opens in a new window Opens an external website Opens an external website in a new window
Close this dialog
This website utilizes technologies such as cookies to enable essential site functionality, as well as for analytics,
personalization, and targeted advertising. To learn more, view the following link: Privacy Policy
Close Cookie Preferences
Formation Control With Collision Avoidance Through Deep Reinforcement Learning Using Model-Guided Demonstration | IEEE
Journals & Magazine | IEEE Xplore
Skip to Main Content
* IEEE.org
* IEEE *Xplore*
* IEEE SA
* IEEE Spectrum
* More Sites
Subscribe
* + Donate
+ Cart
+ Create Account
+ Personal Sign In
* Browse
* My Settings
* Help
Institutional Sign In
Institutional Sign In
AllBooksConferencesCoursesJournals & MagazinesStandardsAuthorsCitations
ADVANCED SEARCH
Journals & Magazines >IEEE Transactions on Neural N... >Volume: 32 Issue: 6
Formation Control With Collision Avoidance Through Deep Reinforcement Learning Using Model-Guided Demonstration
===============================================================================================================
Publisher: IEEE
Cite This
PDF
Zezhi Sui; Zhiqiang Pu; Jianqiang Yi; Shiguang Wu
All Authors
Sign In or Purchase
78
Cites in
Papers
4925
Full
Text Views
* Alerts
Alerts
======
Manage Content Alerts
Add to Citation Alerts
---
Abstract
Document Sections
-----------------
* I.
Introduction
* II.
Preliminaries
* III.
Problem Formulation
* IV.
Approach
* V.
Simulation and Experiment Results
Show Full Outline
Authors
Figures
References
Citations
Keywords
Metrics
More Like This
* Download PDF
* Download References
* Request Permissions
* Save to
* Alerts
Abstract:
---------
Generating collision-free, time-efficient paths in an uncertain dynamic environment poses huge challenges for the formation
control with collision avoidance (FCCA) proble...Show More
Metadata
--------
Abstract:
---------
Generating collision-free, time-efficient paths in an uncertain dynamic environment poses huge challenges for the formation
control with collision avoidance (FCCA) problem in a leader-follower structure. In particular, the followers have to take both
formation maintenance and collision avoidance into account simultaneously. Unfortunately, most of the existing works are simple
combinations of methods dealing with the two problems separately. In this article, a new method based on deep reinforcement
learning (RL) is proposed to solve the problem of FCCA. Especially, the learning-based policy is extended to the field of
formation control, which involves a two-stage training framework: an imitation learning (IL) and later an RL. In the IL stage, a
model-guided method consisting of a consensus theory-based formation controller and an optimal reciprocal collision avoidance
strategy is designed to speed up training and increase efficiency. In the RL stage, a compound reward function is presented to
guide the training. In addition, we design a formation-oriented network structure to perceive the environment. Long short-term
memory is adopted to enable the network structure to perceive the information of obstacles of an uncertain number, and a transfer
training approach is adopted to improve the generalization of the network in different scenarios. Numerous representative
simulations are conducted, and our method is further deployed to an experimental platform based on a multiomnidirectional-wheeled
car system. The effectiveness and practicability of our proposed method are validated through both the simulation and experiment
results.
**Published in:** IEEE Transactions on Neural Networks and Learning Systems ( Volume: 32, Issue: 6, June 2021)
**Page(s):** 2358 - 2372
**Date of Publication:** 16 July 2020
ISSN Information:
-----------------
**PubMed ID:** 32673195
**DOI:** 10.1109/TNNLS.2020.3004893
Publisher: IEEE
Funding Agency:
---------------
Contents
---
### I. Introduction
Multiagent systems have received increasing attention from researchers in recent years because of their great potential in a
variety of fields. Their applications can be found in collaborative explorations of monitoring and rescue, cooperative control of
satellite clusters, and formation control of unmanned aerial vehicles [1]–[3], and so on. The basic concept of multiagent systems
is to use individuals to cooperatively solve complex tasks that cannot be accomplished by a single agent even with expensive
equipment. Formation control is one fundamental problem for multiagent systems, of which the objective is to achieve and maintain
a certain formation shape so that the multiagent system could collectively accomplish a specific task. It is obvious that
formation maintenance is an essential problem in formation control. In addition, collision avoidance should also be taken into
consideration to guarantee the safety of the multiagent system. Due to the interaction among agents and the tradeoff between
collision avoidance and formation maintenance, finding collision-free, time-efficient paths in an uncertain dynamic environment
remains challenging.
Sign in to Continue Reading
Authors
-------
Figures
-------
References
----------
Citations
---------
Keywords
--------
Metrics
-------
More Like This
Collision Avoidance and Connectivity Preservation for Time-Varying Formation of Second-Order Multi-Agent Systems With a
Dynamic Leader
IEEE Access
Published: 2022
Multi-Agent System Based on Machine Learning for Early Diagnosis of Diabetes
2020 IEEE 6th International Conference on Optimization and Applications (ICOA)
Published: 2020
Show More
References
==========
**References is not available for this document.**
IEEE Personal Account
---------------------
* Change username/password
Purchase Details
----------------
* Payment Options
* View Purchased Documents
Profile Information
-------------------
* Communications Preferences
* Profession and Education
* Technical interests
Need Help?
----------
* US & Canada: +1 800 678 4333
* Worldwide: +1 732 981 0060
* Contact & Support
Follow
------
About IEEE *Xplore* | Contact Us | Help | Accessibility | Terms of Use | Nondiscrimination Policy | IEEE Ethics Reporting |
Sitemap | IEEE Privacy Policy
A public charity, IEEE is the world's largest technical professional organization dedicated to advancing technology for the
benefit of humanity.
© Copyright 2025 IEEE - All rights reserved, including rights for text and data mining and training of artificial intelligence
and similar technologies.
### IEEE Account
* Change Username/Password
* Update Address
### Purchase Details
* Payment Options
* Order History
* View Purchased Documents
### Profile Information
* Communications Preferences
* Profession and Education
* Technical Interests
### Need Help?
* **US & Canada:** +1 800 678 4333
* **Worldwide:** +1 732 981 0060
* Contact & Support
* About IEEE *Xplore*
* Contact Us
* Help
* Accessibility
* Terms of Use
* Nondiscrimination Policy
* Sitemap
* Privacy & Opting Out of Cookies
A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing
technology for the benefit of humanity.
© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.
""",
"""
commander
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Follow the plan and try to solve the task step by step according to the plan and tool execution results.
Your primary job is analyze the tool execution, do not call any external tools this time.
[IMPORTANT] Always use English to answer the question.
[IMPORTANT] Do not call any external tools this time, analyze only.
""",
"""
assistant
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Okay, let's tackle this step by step. The user wants to know the existing methods mentioned in the paper for formation maintenance
. From the tool execution results, I accessed the paper's abstract and introduction via the IEEE link.
In the abstract, it says, "most of the existing works are simple combinations of methods dealing with the two problems separately.
" The two problems here are formation maintenance and collision avoidance. So existing methods typically handle these two aspects
independently and then combine them.
The introduction further explains that formation control is a fundamental problem in multiagent systems, where both formation
maintenance and collision avoidance are essential. The authors point out that existing approaches don't integrate these two
aspects effectively. They mention that followers need to consider both simultaneously, which current methods don't address
properly.
Additionally, the paper's approach section refers to a model-guided method using consensus theory-based formation controllers and
optimal reciprocal collision avoidance strategies during the imitation learning stage. These are likely the existing techniques
they build upon.
So putting this together, the existing methods mentioned are those that separate formation maintenance and collision avoidance,
using consensus theory for formation control and reciprocal collision avoidance strategies, then combining them. The paper
critiques this approach as insufficient and proposes an integrated solution using deep reinforcement learning.
Based on the information extracted from the paper's abstract and introduction, the existing methods for formation maintenance
mentioned in the paper can be summarized as follows:
1. **Decoupled Approaches**: Most existing works treat formation maintenance and collision avoidance as separate problems,
combining solutions for each in a non-integrated manner. This means formation control and collision avoidance are addressed
individually and then merged, rather than being optimized together.
2. **Consensus Theory-Based Formation Controllers**: The paper references traditional methods that use consensus algorithms (a
common approach in multiagent systems) to achieve formation maintenance. These controllers rely on mathematical frameworks to
ensure agents reach and maintain a desired formation shape.
3. **Optimal Reciprocal Collision Avoidance (ORCA)**: The paper mentions ORCA as a strategy used in existing methods for collision
avoidance. ORCA is a well-known algorithm that ensures collision-free paths by calculating velocity obstacles for each agent to
avoid collisions with others.
4. **Model-Guided Demonstration Frameworks**: While the paper's proposed method uses a two-stage training framework (imitation
learning followed by reinforcement learning), it builds on prior work that employs model-guided demonstrations. These
demonstrations often rely on predefined controllers (like consensus or ORCA) to generate training data for learning-based systems.
### Key Critique in the Paper:
The authors highlight that these existing methods suffer from limitations such as:
- **Suboptimal Integration**: Decoupling formation maintenance and collision avoidance leads to inefficiencies, as agents must
balance both tasks simultaneously in dynamic environments.
- **Over-Reliance on Predefined Models**: Traditional approaches (e.g., consensus theory or ORCA) may struggle in uncertain or
complex environments where adaptability is required.
The paper proposes a novel deep reinforcement learning (DRL) framework to address these limitations by integrating both tasks into
a unified policy, moving beyond the decoupled and model-dependent strategies of prior work.
""",
"""
commander
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
You are the central commander, the brain of a coordinated multi-agent system. Your job is to make plans to solve a task given by
an end-user:
<task>
In the paper `Formation control with collision avoidance through deep reinforcement learning using model-guided demonstration`,
what existing methods are mentioned for the problem of formation maintenance?
</task>
However, you should not solve this task directly, instead, you should decompose the task into sub-tasks, and present them in a
markdown to-do list format. As an example, here is a to-do list for user question to "Get the GDP of the country/countries with
highest rare earth production in 2024":
<analysis>
... analysis of the task ...
</analysis>
<todo_list>
- [ ] Get the basic concept of what rare earth is, and whether it has direct relationship with GDP
- [ ] Get the rare earth production of each country in 2024, select the country with highest rare earth production
- [ ] Get the GDP of that country in 2024
</todo_list>
<next_step>
Get the GDP of that country in 2024
</next_step>
[IMPORTANT] The to-do list should be clear, short and simple. The number of to-be-completed sub-tasks should be no more than 3.
[IMPORTANT] Always use English to answer the question.
[IMPORTANT] There are 3 cases:
===
1. If there are no previous to-do list in the conversation, write a new one.
for this case, the output should be:
<analysis>
... we are dealing with the first case (as this is the first time that the problem is raised), now analyze the task ...
</analysis>
<todo_list>
...
</todo_list>
===
2. if there are already previous to-do list in the conversation, you should re-write the to-do list again, investigate what
tasks have ALREADY been done, remove tasks that is IMPOSSIBLE to be done and found alternative solutions whenever needed.
for this case, the output should be:
<analysis>
... we are dealing with the 2rd case, analyze the sub-tasks we have already completed, and write a new plan based on
current situation ...
</analysis>
<todo_list>
...
</todo_list>
<next_step>
...
</next_step>
===
3. If the task is already completed, you should return a "TASK_COMPLETED" signal, and do not write any to-do list.
for this case, the output should be:
<analysis>
... according to previous conversations, we have already obtained a possible answer, next, we verify the result ...
</analysis>
TASK_COMPLETED
""",
"""
commander
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Now, determine whether the task is completed. If the task is already completed, say `TASK_COMPLETED`, otherwise, update the to-do
list.
Note that if you need data in the analysis, you have to point out that the data is just a assumption.
""",
"""
assistant
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Okay, let me check if the task is completed. The user asked about existing methods mentioned in the paper for formation
maintenance.
From the tool execution, I accessed the paper's abstract and introduction. The abstract states that existing works combine methods
for formation maintenance and collision avoidance separately. The introduction mentions consensus theory-based controllers and
ORCA as part of existing approaches. Also, the paper's approach section refers to model-guided demonstrations using those
traditional methods.
So, the existing methods identified are:
1. Decoupled approaches handling formation and collision avoidance separately.
2. Consensus theory-based formation controllers.
3. Optimal Reciprocal Collision Avoidance (ORCA).
4. Model-guided demonstration frameworks relying on predefined controllers.
The analysis seems thorough based on the accessible parts of the paper. The full text might have more details, but the key
existing methods are covered in the abstract and introduction. Since the user's question is about what existing methods are
mentioned, and we've extracted those from the available content, the task should be completed.
<analysis>
Based on the information extracted from the paper's abstract and introduction sections (accessible without subscription), we
have identified the existing methods mentioned in the paper for formation maintenance:
1. **Decoupled Approaches**: Existing works treat formation maintenance and collision avoidance as separate problems,
combining solutions for each independently.
2. **Consensus Theory-Based Formation Controllers**: Traditional methods using consensus algorithms to achieve formation
maintenance.
3. **Optimal Reciprocal Collision Avoidance (ORCA)**: A strategy used in existing methods for collision avoidance.
4. **Model-Guided Demonstration Frameworks**: Prior work employing predefined controllers (e.g., consensus or ORCA) to
generate training data for learning-based systems.
While the full paper might contain additional details, the accessible content provides sufficient evidence to answer the user'
s question about the *mentioned* existing methods. Further access to the full text would not invalidate these findings but could
only add supplementary context.
</analysis>
TASK_COMPLETED
""",
"""
end-user
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
In the paper `Formation control with collision avoidance through deep reinforcement learning using model-guided demonstration`,
what existing methods are mentioned for the problem of formation maintenance?
""",
"""
commander
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
As you can see, the former agents have done a lot of work.
Now please make a final answer of the original task based on our conversation.
Please pay special attention to the format in which the answer is presented.
You should first analyze the answer format required by the question and then output the final answer that meets the format
requirements.
Your response should include the following content:
- `final_answer`: enclosed by <final_answer> </final_answer>, the final answer to the question.
Here are some hint about the final answer:
<hint>
Your final answer must be output exactly in the format specified by the question. It should be a number OR as few words as
possible OR a comma separated list of numbers and/or strings:
- If you are asked for a number, don't use comma to write your number neither use units such as $ or percent sign unless specified
otherwise.
- If you are asked for a string, don't use articles, neither abbreviations (e.g. for cities), and write the digits in plain text
unless specified otherwise.
- If you are asked for a comma separated list, apply the above rules depending of whether the element to be put in the list is a
number or a string.
</hint>
""",
"""
assistant
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Okay, let me try to figure out how to structure the final answer based on the user's requirements.
The original question asked what existing methods are mentioned in the paper for formation maintenance. From the previous steps,
we identified four main methods: decoupled approaches, consensus theory-based controllers, ORCA, and model-guided demonstrations.
However, the user wants the answer in a specific format—either a number, a short string, or a comma-separated list.
Looking back at the hint provided, if it's a list, each element should follow the rules (no articles, no abbreviations unless
specified). The answer should be concise. The paper's abstract and introduction mention that existing works are simple
combinations of methods dealing with the two problems (formation maintenance and collision avoidance) separately. Then, the
approach section refers to using consensus theory and ORCA in the model-guided phase.
But the key point is that the existing methods are decoupled approaches, consensus-based formation control, and ORCA. The model-
guided demonstrations are part of the proposed method, not the existing ones. Wait, in the analysis, point 4 was model-guided
demonstration frameworks as existing methods. But in the paper's approach, they use a two-stage training framework starting with
imitation learning (which uses model-guided methods). So the existing methods include consensus and ORCA, which are part of the
model-guided demonstration.
So the answer should list the existing methods as:
1. Decoupled approaches combining separate methods for formation and collision avoidance.
2. Consensus theory-based formation controllers.
3. Optimal Reciprocal Collision Avoidance (ORCA).
But the user wants the answer in a comma-separated list with as few words as possible. So perhaps abbreviate ORCA but spell out
the others. However, the hint says no abbreviations unless specified. Wait, ORCA is an abbreviation, but in the paper, it's
referred to as Optimal Reciprocal Collision Avoidance (ORCA). Since the paper uses the acronym, maybe it's acceptable.
Alternatively, use the full name. But the example answer in the hint says "don't use abbreviations (e.g. for cities)", so maybe
avoid ORCA and write out Optimal Reciprocal Collision Avoidance. However, in the context of academic papers, ORCA is a standard
term, so using the acronym might be acceptable. Need to check. Since the paper mentions ORCA in the abstract (from the tool result
), it's okay to use the acronym here.
So the final answer would be a list of three items:
Decoupled approaches,Consensus theory-based formation controllers,Optimal Reciprocal Collision Avoidance (ORCA)
Wait, but the user said "as few words as possible" or "comma separated list of numbers and/or strings". So maybe shorten them:
Decoupled approaches,Consensus-based controllers,ORCA
Yes, that's more concise. The paper refers to "consensus theory-based formation controller" which can be shortened to "consensus-
based controllers". ORCA is acceptable as an acronym. So the final answer is a comma-separated list of these three methods.
<final_answer>Decoupled approaches,Consensus-based controllers,Optimal Reciprocal Collision Avoidance</final_answer>
"""
]
from experiencemaker.module.summarizer.traj_summarizer import TrajectorySummarizer
from experiencemaker.schema.trajectory import Trajectory, Sample, SummaryMessage, Message
from experiencemaker.schema.trajectory import Message as BMessage
class Message(BMessage):
executor: str = None
ts = TrajectorySummarizer()
role_mapping = {
'system': 'system',
'end-user': 'user',
'commander': 'user',
'assistant': 'assistant',
'tool-agent': 'user',
'tool': 'tool',
}
msg_f = []
for m in msg:
role = m.split('\n')[1].strip()
content = '\n'.join(m.split('\n')[3:]).strip()
msg_f += [Message(role=role_mapping[role], content=content)]
traj = Trajectory(steps=msg_f)
ts.execute(trajectories=[traj], return_samples=True)