mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-10-06 02:48:22 +00:00
update en readme
* update en readme * feat: Resolve conflict, auto committed by CodeFlow
This commit is contained in:
jinli.yl
fuqingxu.fqx
parent
85022a6025
commit
8157ca43f0
6 changed files with 68 additions and 41 deletions
57
README.md
57
README.md
|
|
@ -2,26 +2,37 @@ English | [**中文**](./README_ZH.md)
|
|||
|
||||
# MemoryScope
|
||||
|
||||
|
||||
|
||||
|
||||
Equip your LLM chatbot with a powerful and flexible long term memory system.
|
||||
|
||||
[](https://pypi.org/project/memoryscope/)
|
||||
[](https://pypi.org/project/memoryscope/)
|
||||
[](https://modelscope.github.io/memoryscope/#welcome-to-memoryscope-tutorial-hub)
|
||||
[](https://modelscope.github.io/memoryscope/)
|
||||
[](./LICENSE)
|
||||
[](https://modelscope.github.io/memoryscope/tutorial/contribute.html)
|
||||
|
||||
----
|
||||
## News
|
||||
|
||||
- **[2024-07-29]** We release MemoryScope v0.1.0.2 now, which is also available in [PyPI](https://pypi.org/simple)!
|
||||
- **[2024-09-06]** We release MemoryScope v0.1.1 now, which is also available in [PyPI](https://pypi.org/simple/memoryscope/)!
|
||||
----
|
||||
## What is MemoryScope？
|
||||
MemoryScope provides LLM chatbots with powerful and flexible long-term memory capabilities, offering a framework for building such abilities.
|
||||
It can be applied to scenarios like personal assistants and emotional companions, continuously learning through long-term memory to remember users' basic information as well as various habits and preferences.
|
||||
This allows users to gradually experience a sense of "understanding" when using the LLM.
|
||||
|
||||
MemoryScope is a powerful and flexible long term memory system for LLM chatbots. It consists
|
||||
of a memory database and three customizable system operations, which can be flexibly combined to provide
|
||||
robust long term memory services for your LLM chatbot.
|
||||
|
||||
|
||||
💾 Memory Database:
|
||||
- MemoryScope comes with an *ElasticSearch (ES)* vector database to store all the
|
||||
memory pieces recorded in the system.
|
||||
### Framework
|
||||
|
||||
💾 Memory Database: MemoryScope is equipped with a vector database (default is *ElasticSearch*) to store all memory fragments recorded in the system.
|
||||
|
||||
🔧 Worker Library: MemoryScope atomizes the capabilities of long-term memory into individual workers, including over 20 workers for tasks such as query information filtering, observation extraction, and insight updating.
|
||||
|
||||
🛠️ Operation Library: Based on the worker pipeline, it constructs the operations for memory services, realizing key capabilities such as memory retrieval and memory consolidation.
|
||||
|
||||
🛠️ System operations:
|
||||
- Memory Retrieval: Upon arrival of a user query, this operation returns the semantically related memory pieces
|
||||
and/or those from the corresponding time if the query involves reference to time.
|
||||
- Memory Consolidation: This operation takes in a batch of user queries and returns important user information
|
||||
|
|
@ -30,9 +41,11 @@ extracted from the queries as consolidated *observations* to be stored in the me
|
|||
to form and update *insights*. Then, memory re-consolidation is performed to ensure contradictions and repetitions
|
||||
among memory pieces are properly handled.
|
||||
|
||||
### Framework
|
||||
|
||||
|
||||
⚙️ Best Practices:
|
||||
|
||||
- Based on the core capabilities of long-term memory, MemoryScope has implemented a dialogue interface (API) with long-term memory and a command-line dialogue practice (CLI) with long-term memory.
|
||||
- MemoryScope combines currently popular agent frameworks (AutoGen, AgentScope) to provide best practices.
|
||||
|
||||
### Main Features
|
||||
|
||||
|
|
@ -52,19 +65,29 @@ from the aggregation of similarly-themed *observations*.
|
|||
- The system is time sensitive when performing both Memory Retrieval and Memory Consolidation. Therefore, it can retrieve
|
||||
accurate relevant information when the query involves reference to time.
|
||||
|
||||
----
|
||||
|
||||
## 💼 Supported Model API
|
||||
|
||||
| Backend | Task | Some Supported Models |
|
||||
|-------------------|------------|------------------------------------------------------------------------|
|
||||
| openai_backend | Generation | gpt-4o, gpt-4o-mini, gpt-4, gpt-3.5-turbo |
|
||||
| | Embedding | text-embedding-ada-002, text-embedding-3-large, text-embedding-3-small |
|
||||
| dashscope_backend | Generation | qwen-max, qwen-plus, qwen-plus, qwen2-72b-instruct |
|
||||
| | Embedding | text-embedding-v1, text-embedding-v2 |
|
||||
| | Reranker | gte-rerank |
|
||||
|
||||
In the future, we will support more model interfaces and local deployment of LLM and embedding services.
|
||||
|
||||
|
||||
## 🚀 Installation
|
||||
For installation, please refer to [Installation.md](docs/Installation.md).
|
||||
For installation, please refer to [Installation.md](docs/installation.md).
|
||||
|
||||
### One-key Demo Run
|
||||
|
||||
Run `sudo docker run -it --rm --net=host memoryscope/memoryscope` to launch memoryscope cli demo.
|
||||
|
||||
## Example Usages
|
||||
- [Simple Usages (Quick Start)](./examples/api/simple_usages_en.ipynb)
|
||||
- [CLI with a MemoryScope Chatbot](./examples/cli/dash_cli_cn1.sh)
|
||||
- [Advanced Customization](./examples/api/advanced_customization_en.ipynb)
|
||||
- [CLI with a MemoryScope Chatbot](./examples/cli/README.md)
|
||||
- [Advanced Customization](./examples/advance/custom_operator.md)
|
||||
|
||||
|
||||
## 💡 Contribute
|
||||
|
|
|
|||
16
README_ZH.md
16
README_ZH.md
|
|
@ -8,7 +8,7 @@
|
|||
|
||||
[](https://pypi.org/project/memoryscope/)
|
||||
[](https://pypi.org/project/memoryscope/)
|
||||
[](https://modelscope.github.io/memoryscope/#welcome-to-agentscope-tutorial-hub)
|
||||
[](https://modelscope.github.io/memoryscope/#welcome-to-memoryscope-tutorial-hub)
|
||||
[](https://modelscope.github.io/memoryscope/)
|
||||
[](./LICENSE)
|
||||
[](https://modelscope.github.io/memoryscope/tutorial/contribute.html)
|
||||
|
|
@ -21,7 +21,7 @@
|
|||
----
|
||||
## 新闻
|
||||
|
||||
- **[2024-09-02]** 我们现在发布了 MemoryScope v0.1.1，该版本也可以在 [PyPI](https://pypi.org/simple/memoryscope/) 上获取！
|
||||
- **[2024-09-06]** 我们现在发布了 MemoryScope v0.1.1，该版本也可以在 [PyPI](https://pypi.org/simple/memoryscope/) 上获取！
|
||||
----
|
||||
|
||||
## 什么是MemoryScope？
|
||||
|
|
@ -38,15 +38,15 @@ MemoryScope可以用于个人助理、情感陪伴等记忆场景，通过长期
|
|||
|
||||
🛠️ 核心Op库: 并基于worker的pipeline构建了memory服务的核心operation，实现了记忆检索，记忆巩固等核心能力。
|
||||
|
||||
1. 记忆检索：当用户输入对话，此操作返回语义相关的记忆片段。如果输入对话包含对时间的指涉，则同时返回相应时间中的记忆片段。
|
||||
2. 记忆巩固：此操作接收一批用户的输入对话，并从对话中提取重要的用户信息，将其作为 *observation* 形式的记忆片段存储在记忆数据库中。
|
||||
3. 反思与再巩固：每隔一段时间，此操作对新记录的 *observations* 进行反思，以形成和更新 *insight*
|
||||
- 记忆检索：当用户输入对话，此操作返回语义相关的记忆片段。如果输入对话包含对时间的指涉，则同时返回相应时间中的记忆片段。
|
||||
- 记忆巩固：此操作接收一批用户的输入对话，并从对话中提取重要的用户信息，将其作为 *observation* 形式的记忆片段存储在记忆数据库中。
|
||||
- 反思与再巩固：每隔一段时间，此操作对新记录的 *observations* 进行反思，以形成和更新 *insight*
|
||||
形式的记忆片段。然后执行记忆再巩固，以确保记忆片段之间的矛盾和重复得到妥善处理。
|
||||
|
||||
⚙️ 最佳实践:
|
||||
|
||||
1. MemoryScope在构建了长期记忆核心能力的基础上，实现了带长期记忆的对话接口(API)和带长期记忆的命令行对话实践(CLI)。
|
||||
2. MemoryScope结合了目前流行的Agent框架（AutoGen、AgentScope），给出了最佳实践。
|
||||
- MemoryScope在构建了长期记忆核心能力的基础上，实现了带长期记忆的对话接口(API)和带长期记忆的命令行对话实践(CLI)。
|
||||
- MemoryScope结合了目前流行的Agent框架（AutoGen、AgentScope），给出了最佳实践。
|
||||
|
||||
### 🤝主要特点
|
||||
|
||||
|
|
@ -89,7 +89,9 @@ MemoryScope可以用于个人助理、情感陪伴等记忆场景，通过长期
|
|||
|
||||
### Docker方式一键运行Demo
|
||||
|
||||
<!--
|
||||
运行 `sudo docker run -it --rm --net=host memoryscope/memoryscope` 一键运行memoryscope的演示。
|
||||
-->
|
||||
|
||||
完整的安装方法请参考[安装指南](docs/installation_zh.md)。
|
||||
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ memory_service:
|
|||
|
||||
retrieve_memory:
|
||||
class: core.operation.frontend_operation
|
||||
workflow: set_query_meow,[extract_time|retrieve_obs_ins,semantic_rank],fuse_rerank
|
||||
workflow: rewrite_query,[extract_time|retrieve_obs_ins,semantic_rank],fuse_rerank
|
||||
description: "retrieve long-term memory"
|
||||
|
||||
list_memory:
|
||||
|
|
@ -74,7 +74,7 @@ worker:
|
|||
class: core.worker.frontend.read_message_worker
|
||||
set_query:
|
||||
class: core.worker.frontend.set_query_worker
|
||||
set_query_meow:
|
||||
rewrite_query:
|
||||
class: contrib.example_query_worker
|
||||
generation_model: generation_model
|
||||
retrieve_obs_ins:
|
||||
|
|
|
|||
|
|
@ -26,8 +26,6 @@
|
|||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"sys.path.append(\".\")\n",
|
||||
"from memoryscope import MemoryScope, Arguments\n",
|
||||
"arguments = Arguments(\n",
|
||||
" language=\"cn\",\n",
|
||||
|
|
@ -35,14 +33,12 @@
|
|||
" assistant_name=\"AI\",\n",
|
||||
" memory_chat_class=\"api_memory_chat\",\n",
|
||||
" generation_backend=\"dashscope_generation\",\n",
|
||||
" generation_model=\"qwen2-72b-instruct\",\n",
|
||||
" generation_model=\"qwen-max\",\n",
|
||||
" embedding_backend=\"dashscope_embedding\",\n",
|
||||
" embedding_model=\"text-embedding-v2\",\n",
|
||||
" rank_backend=\"dashscope_rank\",\n",
|
||||
" rank_model=\"gte-rerank\",\n",
|
||||
" enable_ranker=True,\n",
|
||||
" worker_params={\"get_reflection_subject\": {\"reflect_num_questions\": 3}}\n",
|
||||
")\n",
|
||||
" enable_ranker=True)\n",
|
||||
"\n",
|
||||
"ms = MemoryScope(arguments=arguments)\n"
|
||||
]
|
||||
|
|
@ -465,9 +461,9 @@
|
|||
"metadata": {},
|
||||
"source": [
|
||||
"## 更多用法\n",
|
||||
"我们建议读者参考[进阶自定义用法](./examples/api/advanced_customization_cn.ipynb)来对MemoryScope系统进行各种自定义设置。您还可以通过自定义**workflow**和对应的**worker**来创建或定制满足您特定需求的**operation**。\n",
|
||||
"我们建议读者参考[进阶自定义用法](../advance/custom_operator.md)来对MemoryScope系统进行各种自定义设置。您还可以通过自定义**workflow**和对应的**worker**来创建或定制满足您特定需求的**operation**。\n",
|
||||
"\n",
|
||||
"此外，您还可以尝试使用[在命令行与MemoryScope聊天机器人交互](../cli/README.md)。我们在这里实现了始终在后台异步运行**记忆巩固**和**反思与再巩固**这两个操作，从而使得它们不会增加聊天的响应时间。"
|
||||
"此外，您还可以尝试使用[在命令行与MemoryScope聊天机器人交互](../cli/README_ZH.md)。我们在这里实现了始终在后台异步运行**记忆巩固**和**反思与再巩固**这两个操作，从而使得它们不会增加聊天的响应时间。"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
""" Version of MemoryScope."""
|
||||
__version__ = "0.1.1.1"
|
||||
__version__ = "0.1.0.9"
|
||||
import fire
|
||||
|
||||
from memoryscope.core.config.arguments import Arguments # noqa: F401
|
||||
|
|
|
|||
18
setup.py
18
setup.py
|
|
@ -7,6 +7,8 @@ python setup.py sdist bdist_wheel
|
|||
|
||||
# 3. finally, upload
|
||||
twine upload dist/*
|
||||
|
||||
rm -rf dist build && python setup.py sdist bdist_wheel && twine upload dist/*
|
||||
"""
|
||||
|
||||
import setuptools, glob, os
|
||||
|
|
@ -14,6 +16,7 @@ import setuptools, glob, os
|
|||
with open("README.md", "r", encoding="utf-8") as fh:
|
||||
long_description = fh.read()
|
||||
|
||||
|
||||
def _process_requirements():
|
||||
packages = open('requirements.txt').read().strip().split('\n')
|
||||
requires = []
|
||||
|
|
@ -25,6 +28,7 @@ def _process_requirements():
|
|||
requires.append(pkg)
|
||||
return requires
|
||||
|
||||
|
||||
def package_files(directory):
|
||||
paths = []
|
||||
for (path, directories, filenames) in os.walk(directory):
|
||||
|
|
@ -33,8 +37,8 @@ def package_files(directory):
|
|||
paths.append(os.path.join('..', path, filename))
|
||||
return paths
|
||||
|
||||
extra_files = package_files('memoryscope')
|
||||
|
||||
extra_files = package_files('memoryscope')
|
||||
|
||||
authors = [
|
||||
{"name": "Li Yu", "email": "jinli.yl@alibaba-inc.com"},
|
||||
|
|
@ -47,10 +51,12 @@ authors = [
|
|||
|
||||
setuptools.setup(
|
||||
name="memoryscope",
|
||||
version="0.1.1.1",
|
||||
version="0.1.0.9",
|
||||
author=', '.join([author['name'] for author in authors]),
|
||||
author_email=', '.join([author['email'] for author in authors]),
|
||||
description="MemoryScope is a powerful and flexible long term memory system for LLM chatbots. It consists of a memory database and three customizable system operations, which can be flexibly combined to provide robust long term memory services for your LLM chatbot.",
|
||||
description="MemoryScope is a powerful and flexible long term memory system for LLM chatbots. It consists of a "
|
||||
"memory database and three customizable system operations, which can be flexibly combined to provide "
|
||||
"robust long term memory services for your LLM chatbot.",
|
||||
long_description=long_description,
|
||||
long_description_content_type="text/markdown",
|
||||
url="https://github.com/modelscope/memoryscope",
|
||||
|
|
@ -59,16 +65,16 @@ setuptools.setup(
|
|||
},
|
||||
classifiers=[
|
||||
"Programming Language :: Python :: 3",
|
||||
"License :: Apache License",
|
||||
"License :: OSI Approved :: Apache Software License",
|
||||
"Operating System :: OS Independent",
|
||||
],
|
||||
package_dir={"": "."},
|
||||
package_data={"": extra_files},
|
||||
include_package_data=True,
|
||||
entry_points = {
|
||||
entry_points={
|
||||
'console_scripts': ['memoryscope=memoryscope:cli'],
|
||||
},
|
||||
packages=setuptools.find_packages(where="."),
|
||||
python_requires=">=3.10",
|
||||
install_requires=_process_requirements(),
|
||||
)
|
||||
)
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue