scriptoriumadmin/ReMe
1
0
Fork 0
mirror of https://github.com/agentscope-ai/ReMe.git synced 2026-09-06 08:16:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

refactor: Remove unused CLI scripts and update file paths

Browse source
This commit is contained in:
青轩 2024-08-21 12:07:57 +08:00
parent d201c3e621
commit 6a2b07bea9
6 changed files with 256 additions and 165 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
README.md
View file

@ -70,7 +70,7 @@ accurate relevant information when the query involves reference to time.
```
git clone https://...
nano config/demo_config_cn.yaml
vim memoryscope/core/config/demo_config.yaml
```
2. Edit `docker-compose.yml` to change environment variable.
@ -88,7 +88,7 @@ accurate relevant information when the query involves reference to time.
```
git clone https://...
nano config/demo_config_cn.yaml
vim memoryscope/core/config/demo_config.yaml
```
2. Build the `Dockerfile` with command:
@ -96,16 +96,7 @@ accurate relevant information when the query involves reference to time.
sudo docker build --network=host -t memoryscope .
```
3. Run `ElasticSearch` Container with command:
```
docker run -p 9200:9200 \
-e "discovery.type=single-node" \
-e "xpack.security.enabled=false" \
-e "xpack.license.self_generated.type=trial" \
docker.elastic.co/elasticsearch/elasticsearch:8.13.2
```
4. Launch the built image with command:
3. Launch the built image with command:
```
sudo docker run -it --rm --net=host memoryscope
```

67
docs/Installation.md Normal file
View file

@ -0,0 +1,67 @@
# Installing MemoryScope
## I. Install with docker [Recommanded]
1. Clone the repository and edit settings
```bash
# clone project
git clone https://...
cd memoryscope
# edit configuration, e.g. add api keys
vim memoryscope/core/config/demo_config.yaml
```
2. Build Docker image
```bash
sudo docker build --network=host -t memoryscope .
```
3. Launch Docker container
```bash
sudo docker run -it --rm --net=host memoryscope
```
## II. Install with docker compose [Recommanded]
1. Clone the repository and edit settings
```bash
# clone project
git clone https://...
cd memoryscope
# edit configuration, e.g. add api keys
vim memoryscope/core/config/demo_config.yaml
```
2. Edit `docker-compose.yml` to change environment variable.
```
DASHSCOPE_API_KEY: "sk-0000000000"
```
3. Run `docker-compose up` to build and launch the memory-scope cli interface.
## III. Install in native system [Linux only]
1. Clone the repository and edit settings
```bash
# clone project
git clone https://...
cd memoryscope
# edit configuration, e.g. add api keys
vim memoryscope/core/config/demo_config.yaml
```
2. Install
```bash
poetry install
```
3. Run Elasticsearch service, refer to [elasticsearch documents](https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html).
4. Launch memoryscope, also refer to [cli documents](../examples/cli/README.md)
```bash
memoryscope --config_path=memoryscope/core/config/demo_config.yaml
```

283
examples/api/simple_usages_en.ipynb
View file

@ -2,29 +2,37 @@
"cells": [
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"# Example usages of **chat** and **service** interfaces\n",
"This notebook shows simple usages of MemoryScope's **chat** and **service** interfaces, along with its main features.\n",
"\n",
"Before running this notebook, follow the **Installation** guidelines in Readme, and start the Docker image first."
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "markdown",
"source": [
"## Initiate a MemoryScope instance\n",
"First, we need to specify a configuration and initiate a MemoryScope instance."
],
"metadata": {
"collapsed": false
}
},
"source": [
"## Initiate a MemoryScope instance\n",
"First, we need to specify a configuration and initiate a MemoryScope instance.\n",
"(elasticsearch service is required, please refer to https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html)"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:34:30.387354Z",
"start_time": "2024-08-02T14:34:28.512658Z"
},
"collapsed": false
},
"outputs": [],
"source": [
"import sys\n",
@ -47,28 +55,28 @@
")\n",
"\n",
"ms = MemoryScope(arguments=arguments)\n"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:34:30.387354Z",
"start_time": "2024-08-02T14:34:28.512658Z"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"## Chat without memory\n",
"MemoryScope comes with a default **chat** interface, so it's very easy to start chatting, just as what you'll do with any LLM chatbot."
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:34:35.313590Z",
"start_time": "2024-08-02T14:34:30.388733Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -84,28 +92,28 @@
"memory_chat.run_service_operation(\"delete_all\")\n",
"response = memory_chat.chat_with_memory(query=\"My hobby is to play piano.\")\n",
"print(\"Response 1: \\n\" + response.message.content)"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:34:35.313590Z",
"start_time": "2024-08-02T14:34:30.388733Z"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"----\n",
"You can choose to chat with or without multi-round conversation contexts. However, since **Memory Consolidation** has not been called, there's no memory pieces in the system yet."
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:34:42.621918Z",
"start_time": "2024-08-02T14:34:35.315441Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -124,28 +132,28 @@
"response = memory_chat.chat_with_memory(query=\"Do you know if I play any musical instruments?\",\n",
" history_message_strategy=None)\n",
"print(\"Response 3: \\n\" + response.message.content)"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:34:42.621918Z",
"start_time": "2024-08-02T14:34:35.315441Z"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"## Memory Consolidation\n",
"Now, we do a bit more chatting and then try out **Memory Consolidation**."
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:35:07.399702Z",
"start_time": "2024-08-02T14:34:42.622804Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -171,18 +179,18 @@
"print(\"Response 6: \\n\" + response.message.content)\n",
"response = memory_chat.chat_with_memory(query=\"Help me write a short birthday wish for a friend.\")\n",
"print(\"Response 7: \\n\" + response.message.content)"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:35:07.399702Z",
"start_time": "2024-08-02T14:34:42.622804Z"
}
}
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:35:42.560772Z",
"start_time": "2024-08-02T14:35:07.401224Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -200,30 +208,30 @@
"memory_service.init_service()\n",
"result = memory_service.consolidate_memory()\n",
"print(f\"consolidate_memory result={result}\")"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:35:42.560772Z",
"start_time": "2024-08-02T14:35:07.401224Z"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"----\n",
"**Memory Consolidation** extracted 3 *observations* out of the 7 chat messages from the user, with the uninformative ones being filtered out.\n",
"\n",
"We try more cases to test its time awareness and the ability to filter out fictitious contents from the user."
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:36:30.548435Z",
"start_time": "2024-08-02T14:35:42.559854Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -283,18 +291,18 @@
"print(\"Response 11: \\n\" + response.message.content)\n",
"response = memory_chat.chat_with_memory(query=\"What kind of company is SMCI, and what do they do?\")\n",
"print(\"Response 12: \\n\" + response.message.content)"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:36:30.548435Z",
"start_time": "2024-08-02T14:35:42.559854Z"
}
}
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:37:04.220392Z",
"start_time": "2024-08-02T14:36:30.549719Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -311,30 +319,30 @@
"source": [
"result = memory_service.consolidate_memory()\n",
"print(f\"consolidate_memory result={result}\")"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:37:04.220392Z",
"start_time": "2024-08-02T14:36:30.549719Z"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"----\n",
"We can see **Memory Consolidation** successfully filtered out fictitious contents, and shows good time sensitivity.\n",
"\n",
"We try more cases to test its resolution of conflicting contents."
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:37:25.239829Z",
"start_time": "2024-08-02T14:37:04.221032Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -372,18 +380,18 @@
"print(\"Response 18: \\n\" + response.message.content)\n",
"response = memory_chat.chat_with_memory(query=\"Tomorrow is my birthday\")\n",
"print(\"Response 19：\\n\" + response.message.content)"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:37:25.239829Z",
"start_time": "2024-08-02T14:37:04.221032Z"
}
}
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:38:05.369583Z",
"start_time": "2024-08-02T14:37:25.241674Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -402,28 +410,28 @@
"source": [
"result = memory_service.consolidate_memory()\n",
"print(f\"consolidate_memory result={result}\")"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:38:05.369583Z",
"start_time": "2024-08-02T14:37:25.241674Z"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"## Reflection and Re-Consolidation\n",
"Now, we have accumulated enough new *observations* in the system, so we can call **Reflection and Re-Consolidation**, let's see what will it get."
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:38:27.126860Z",
"start_time": "2024-08-02T14:38:05.370326Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -452,28 +460,24 @@
"source": [
"result = memory_service.reflect_and_reconsolidate()\n",
"print(f\"consolidate_memory result={result}\")"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:38:27.126860Z",
"start_time": "2024-08-02T14:38:05.370326Z"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"## Low response-time (RT) for the user\n",
"Finally, we test the RT of MemoryScope system for the user. Specifically, we test the difference of RT when responding with and without retrieving memory pieces from the system."
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "code",
"execution_count": 0,
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import time\n",
@ -491,14 +495,18 @@
"end_time = time.time()\n",
"total_time = end_time - start_time\n",
"print(\"With memory retrieval\\nResponse 21: \\n\" + response.message.content + f\"\\n RT: {total_time} seconds\\n\")"
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"ExecuteTime": {
"end_time": "2024-08-02T14:38:46.158169Z",
"start_time": "2024-08-02T14:38:27.128634Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -546,46 +554,39 @@
"end_time = time.time()\n",
"total_time = end_time - start_time\n",
"print(\"Without memory retrieval\\nResponse 23: \\n\" + response.message.content + f\"\\n RT: {total_time} seconds\\n\")"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-08-02T14:38:46.158169Z",
"start_time": "2024-08-02T14:38:27.128634Z"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"----\n",
"We can see responding with retrieving memory pieces from MemoryScope does not increase RT."
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"## More Examples\n",
"We direct the reader to [Advanced Customization](./examples/api/advanced_customization_en.ipynb) for guidance on customizing the various settings of the MemoryScope system. It is also possible to create or customize your own MemoryScope **operations** by specifying a **workflow** and the corresponding **workers** that best meet your specific needs.\n",
"\n",
"Additionally, you can also try out the [CLI with a MemoryScope Chatbot](./examples/cli/dash_cli_cn1.sh). We have implemented the chatbot so that the **Memory Consolidation** and **Reflection and Re-Consolidation** operations are always run asynchronously in the backend, ensuring that they do not incur any response time for the user.\n",
"\n"
],
"metadata": {
"collapsed": false
}
]
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [],
"metadata": {
"collapsed": false
}
},
"outputs": [],
"source": []
}
],
"metadata": {

44
examples/cli/README.md
View file

@ -0,0 +1,44 @@
# The Cli Interface of MemoryScope
## Usage
MemoryScope can be launched in two different ways:
### 1. Using YAML Configuration File
If you prefer to configure your settings via a YAML file, you can do so by providing the path to the configuration file as follows:
```bash
memoryscope --config_path=memoryscope/core/config/demo_config.yaml
```
### 2. Using Command Line Arguments
Alternatively, you can specify all the parameters directly on the command line:
```bash
memoryscope --language="cn" \
--memory_chat_class="cli_memory_chat" \
--human_name="锦鲤" \
--assistant_name="AI" \
--generation_backend="dashscope_generation" \
--generation_model="qwen-max" \
--embedding_backend="dashscope_embedding" \
--embedding_model="text-embedding-v2" \
--use_dummy_ranker=False \
--rank_backend="dashscope_rank" \
--rank_model="gte-rerank"
```
Here are the available options that can be set through either method:
- `--language`: The language used for the conversation.
- `--memory_chat_class`: The class name for managing the chat history.
- `--human_name`: The name of the human user.
- `--assistant_name`: The name of the AI assistant.
- `--generation_backend`: The backend used for generating responses.
- `--generation_model`: The model used for generating responses.
- `--embedding_backend`: The backend used for text embeddings.
- `--embedding_model`: The model used for creating text embeddings.
- `--use_dummy_ranker`: A boolean indicating whether to use a dummy ranker (default is `False`).
- `--rank_backend`: The backend used for ranking responses.
- `--rank_model`: The model used for ranking responses.

1
examples/cli/dash_cli_cn1.sh
View file

@ -1 +0,0 @@
memoryscope --config_path=memoryscope/core/config/demo_config.yaml

11
examples/cli/dash_cli_cn2.sh
View file

@ -1,11 +0,0 @@
memoryscope --language="cn" \
--memory_chat_class="cli_memory_chat" \
--human_name="锦鲤" \
--assistant_name="AI" \
--generation_backend="dashscope_generation" \
--generation_model="qwen-max" \
--embedding_backend="dashscope_embedding" \
--embedding_model="text-embedding-v2" \
--use_dummy_ranker=False \
--rank_backend="dashscope_rank" \
--rank_model="gte-rerank"