feat: Add new modules and update README

docs/README.md

@@ -9,7 +9,7 @@ Please use the following commands to build sphinx doc of MemoryScope.
 pip install sphinx sphinx-autobuild sphinx_rtd_theme myst-parser sphinxcontrib-mermaid
 
 # step 2: go into the sphinx_doc dir
-cd sphinx_doc
+cd docs/sphinx_doc
 
 # step 3: build the sphinx doc
 ./build_sphinx_doc.sh

docs/sphinx_doc/build_sphinx_doc.sh

@@ -1,7 +1,5 @@
 #!/bin/bash
 rm -rf build/html/*
-rm en/source/agentscope*.rst
-rm zh_CN/source/agentscope*.rst
 rm en/source/memory_scope*.rst
 rm zh_CN/source/memory_scope*.rst
 sphinx-apidoc -f -o en/source ../../memory_scope -t template -e

docs/sphinx_doc/en/source/index.rst

@@ -9,6 +9,20 @@ MemoryScope Documentation
 ======================================
 
 
+.. include:: tutorial/main.md
+   :parser: myst_parser.sphinx_
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+   :hidden:
+   :caption: AgentScope Tutorial
+
+   tutorial/101-agentscope.md
+   tutorial/102-installation.md
+   tutorial/103-example.md
+
+   tutorial/contribute.rst
 
 .. toctree::
    :maxdepth: 1

docs/sphinx_doc/en/source/tutorial/101-agentscope.md

@@ -0,0 +1,116 @@
+(101-agentscope-en)=
+
+# About AgentScope
+
+In this tutorial, we will provide an overview of AgentScope by answering
+several questions, including what's AgentScope, what can AgentScope provide,
+and why we should choose AgentScope. Let's get started!
+
+## What is AgentScope?
+
+AgentScope is a developer-centric multi-agent platform, which enables
+developers to build their LLM-empowered multi-agent applications with less
+effort.
+
+With the advance of large language models, developers are able to build
+diverse applications.
+In order to connect LLMs to data and services and solve complex tasks,
+AgentScope provides a series of development tools and components for ease of
+development.
+It features
+
+- **usability**,
+- **robustness**,
+- **the support of multi-modal data**,
+- **distributed deployment**.
+
+## Key Concepts
+
+### Message
+
+Message is a carrier of information (e.g. instructions, multi-modal
+data, and dialogue). In AgentScope, message is a Python dict subclass
+with `name` and `content` as necessary fields, and `url` as an optional
+field referring to additional resources.
+
+### Agent
+
+Agent is an autonomous entity capable of interacting with environment and
+agents, and taking actions to change the environment. In AgentScope, an
+agent takes message as input and generates corresponding response message.
+
+### Service
+
+Service refers to the functional APIs that enable agents to perform
+specific tasks. In AgentScope, services are categorized into model API
+services, which are channels to use the LLMs, and general API services,
+which provide a variety of tool functions.
+
+### Workflow
+
+Workflow represents ordered sequences of agent executions and message
+exchanges between agents, analogous to computational graphs in TensorFlow,
+but with the flexibility to accommodate non-DAG structures.
+
+## Why AgentScope?
+
+**Exceptional usability for developers.**
+AgentScope provides high usability for developers with flexible syntactic
+sugars, ready-to-use components, and pre-built examples.
+
+**Robust fault tolerance for diverse models and APIs.**
+AgentScope ensures robust fault tolerance for diverse models, APIs, and
+allows developers to build customized fault-tolerant strategies.
+
+**Extensive compatibility for multi-modal application.**
+AgentScope supports multi-modal data (e.g., files, images, audio and videos)
+in both dialog presentation, message transmission and data storage.
+
+**Optimized efficiency for distributed multi-agent operations.** AgentScope
+introduces an actor-based distributed mechanism that enables centralized
+programming of complex distributed workflows, and automatic parallel
+optimization.
+
+## How is AgentScope designed?
+
+The architecture of AgentScope comprises three hierarchical layers. The
+layers provide supports for multi-agent applications from different levels,
+including elementary and advanced functionalities of a single agent
+(**utility layer**), resources and runtime management (**manager and wrapper
+layer**), and agent-level to workflow-level programming interfaces (**agent
+layer**). AgentScope introduces intuitive abstractions designed to fulfill
+the diverse functionalities inherent to each layer and simplify the
+complicated interlayer dependencies when building multi-agent systems.
+Furthermore, we offer programming interfaces and default mechanisms to
+strengthen the resilience of multi-agent systems against faults within
+different layers.
+
+## AgentScope Code Structure
+
+```bash
+AgentScope
+├── src
+│   ├── agentscope
+│   |   ├── agents               # Core components and implementations pertaining to agents.
+│   |   ├── memory               # Structures for agent memory.
+│   |   ├── models               # Interfaces for integrating diverse model APIs.
+│   |   ├── pipelines            # Fundamental components and implementations for running pipelines.
+│   |   ├── rpc                  # Rpc module for agent distributed deployment.
+│   |   ├── service              # Services offering functions independent of memory and state.
+|   |   ├── web                  # WebUI used to show dialogs.
+│   |   ├── utils                # Auxiliary utilities and helper functions.
+│   |   ├── message.py           # Definitions and implementations of messaging between agents.
+│   |   ├── prompt.py            # Prompt engineering module for model input.
+│   |   ├── ... ..
+│   |   ├── ... ..
+├── scripts                      # Scripts for launching local Model API
+├── examples                     # Pre-built examples of different applications.
+├── docs                         # Documentation tool for API reference.
+├── tests                        # Unittest modules for continuous integration.
+├── LICENSE                      # The official licensing agreement for AgentScope usage.
+└── setup.py                     # Setup script for installing.
+├── ... ..
+└── ... ..
+```
+
+[[Return to the top]](#101-agentscope)

docs/sphinx_doc/en/source/tutorial/102-installation.md

@@ -0,0 +1,68 @@
+(102-installation-en)=
+
+# Installation
+
+To install AgentScope, you need to have Python 3.9 or higher installed. We recommend setting up a new virtual environment specifically for AgentScope:
+
+## Create a Virtual Environment
+
+### Using Conda
+
+If you're using Conda as your package and environment management tool, you can create a new virtual environment with Python 3.9 using the following commands:
+
+```bash
+# Create a new virtual environment named 'agentscope' with Python 3.9
+conda create -n agentscope python=3.9
+
+# Activate the virtual environment
+conda activate agentscope
+```
+
+### Using Virtualenv
+
+Alternatively, if you prefer `virtualenv`, you can install it first (if it's not already installed) and then create a new virtual environment as shown:
+
+```bash
+# Install virtualenv if it is not already installed
+pip install virtualenv
+
+# Create a new virtual environment named 'agentscope' with Python 3.9
+virtualenv agentscope --python=python3.9
+
+# Activate the virtual environment
+source agentscope/bin/activate  # On Windows use `agentscope\Scripts\activate`
+```
+
+## Installing AgentScope
+
+### Install with Pip
+
+If you prefer to install AgentScope from Pypi, you can do so easily using `pip`:
+
+```bash
+# For centralized multi-agent applications
+pip install agentscope --pre
+# For distributed multi-agent applications
+pip install agentscope[distribute] --pre # On Mac use `pip install agentscope\[distribute\] --pre`
+```
+
+### Install from Source
+
+For users who prefer to install AgentScope directly from the source code, follow these steps to clone the repository and install the platform in editable mode:
+
+**_Note: This project is under active development, it's recommended to install AgentScope from source._**
+
+```bash
+# Pull the source code from Github
+git clone https://github.com/modelscope/agentscope.git
+cd agentscope
+
+# For centralized multi-agent applications
+pip install -e .
+# For distributed multi-agent applications
+pip install -e .[distribute]  # On Mac use `pip install -e .\[distribute\]`
+```
+
+**Note**: The `[distribute]` option installs additional dependencies required for distributed applications. Remember to activate your virtual environment before running these commands.
+
+[[Return to the top]](#102-installation-en)

docs/sphinx_doc/en/source/tutorial/103-example.md

@@ -0,0 +1,108 @@
+(103-start-en)=
+
+# Quick Start
+
+AgentScope is designed with a flexible communication mechanism.
+In this tutorial, we will introduce the basic usage of AgentScope via a
+simple standalone conversation between two agents (e.g. user and assistant
+agents).
+
+## Step1: Prepare Model
+
+AgentScope decouples the deployment and invocation of models to better build multi-agent applications.
+
+In terms of model deployment, users can use third-party model services such
+as OpenAI API, Google Gemini API, HuggingFace/ModelScope Inference API, or
+quickly deploy local open-source model services through the [scripts](https://github.com/modelscope/agentscope/blob/main/scripts/README.md) in
+the repository.
+
+While for model invocation, users should prepare a model configuration to specify the model service. Taking OpenAI Chat API as an example, the model configuration is like this:
+
+```python
+model_config = {
+    "config_name": "{config_name}", # A unique name for the model config.
+    "model_type": "openai_chat",    # Choose from "openai_chat", "openai_dall_e", or "openai_embedding".
+
+    "model_name": "{model_name}",   # The model identifier used in the OpenAI API, such as "gpt-3.5-turbo", "gpt-4", or "text-embedding-ada-002".
+    "api_key": "xxx",               # Your OpenAI API key. If unset, the environment variable OPENAI_API_KEY is used.
+    "organization": "xxx",          # Your OpenAI organization ID. If unset, the environment variable OPENAI_ORGANIZATION is used.
+}
+```
+
+More details about model invocation, deployment and open-source models please refer to [Model](203-model-en) section.
+
+After preparing the model configuration, you can register your configuration by calling the `init` method of AgentScope. Additionally, you can load multiple model configurations at once.
+
+```python
+import agentscope
+
+# init once by passing a list of config dict
+openai_cfg_dict = {
+    # ...
+}
+modelscope_cfg_dict = {
+    # ...
+}
+agentscope.init(model_configs=[openai_cfg_dict, modelscope_cfg_dict])
+```
+
+## Step2: Create Agents
+
+Creating agents is straightforward in AgentScope. After initializing AgentScope with your model configurations (Step 1 above), you can then define each agent with its corresponding role and specific model.
+
+```python
+import agentscope
+from agentscope.agents import DialogAgent, UserAgent
+
+# read model configs
+agentscope.init(model_configs="./openai_model_configs.json")
+
+# Create a dialog agent and a user agent
+dialogAgent = DialogAgent(name="assistant", model_config_name="gpt-4", sys_prompt="You are a helpful ai assistant")
+userAgent = UserAgent()
+```
+
+**NOTE**: Please refer to [Customizing Your Own Agent](201-agent-en) for all available agents.
+
+## Step3: Agent Conversation
+
+"Message" is the primary means of communication between agents in AgentScope. They are Python dictionaries comprising essential fields like the actual `content` of this message and the sender's `name`. Optionally, a message can include a `url` to either a local file (image, video or audio) or website.
+
+```python
+from agentscope.message import Msg
+
+# Example of a simple text message from Alice
+message_from_alice = Msg("Alice", "Hi!")
+
+# Example of a message from Bob with an attached image
+message_from_bob = Msg("Bob", "What about this picture I took?", url="/path/to/picture.jpg")
+```
+
+To start a conversation between two agents, such as `dialog_agent` and `user_agent`, you can use the following loop. The conversation continues until the user inputs `"exit"` which terminates the interaction.
+
+```python
+x = None
+while True:
+    x = dialogAgent(x)
+    x = userAgent(x)
+
+    # Terminate the conversation if the user types "exit"
+    if x.content == "exit":
+        print("Exiting the conversation.")
+        break
+```
+
+For a more advanced approach, AgentScope offers the option of using pipelines to manage the flow of messages between agents. The `sequentialpipeline` stands for sequential speech, where each agent receive message from last agent and generate its response accordingly.
+
+```python
+from agentscope.pipelines.functional import sequentialpipeline
+
+# Execute the conversation loop within a pipeline structure
+x = None
+while x is None or x.content != "exit":
+  x = sequentialpipeline([dialog_agent, user_agent])
+```
+
+For more details about how to utilize pipelines for complex agent interactions, please refer to [Pipeline and MsgHub](202-pipeline-en).
+
+[[Return to the top]](#103-start-en)

memory_scope/__init__.py

@@ -1,3 +1,11 @@
 """ Version of MemoryScope."""
 
 __version__ = "0.1.0-alpha.1"
+
+
+from .cli import MemoryScope, CliJob
+
+__all__ = [
+    "MemoryScope",
+    "CliJob",
+]

memory_scope/chat/__init__.py

@@ -1 +1,8 @@
-""" Chat."""
+""" Chat."""
+from .base_memory_chat import BaseMemoryChat
+from .cli_memory_chat import CliMemoryChat
+
+__all__ = [
+    "BaseMemoryChat",
+    "CliMemoryChat",
+]

memory_scope/constants/__init__.py

@@ -0,0 +1,4 @@
+__all__ = [
+    "common_constants",
+    "language_constants",
+]

memory_scope/enumeration/__init__.py

@@ -0,0 +1,15 @@
+from .action_status_enum import ActionStatusEnum
+from .language_enum import LanguageEnum
+from .memory_type_enum import MemoryTypeEnum
+from .message_role_enum import MessageRoleEnum
+from .model_enum import ModelEnum
+from .store_status_enum import StoreStatusEnum
+
+__all__ = [
+    "ActionStatusEnum",
+    "LanguageEnum",
+    "MemoryTypeEnum",
+    "MessageRoleEnum",
+    "ModelEnum",
+    "StoreStatusEnum",
+]

memory_scope/memory/__init__.py

@@ -0,0 +1 @@
+from .worker import *

memory_scope/models/__init__.py

@@ -1 +1,13 @@
+from memory_scope.models.base_model import BaseModel
+from memory_scope.models.dummy_generation_model import DummyGenerationModel
+from memory_scope.models.llama_index_embedding_model import LlamaIndexEmbeddingModel
+from memory_scope.models.llama_index_generation_model import LlamaIndexGenerationModel
+from memory_scope.models.llama_index_rank_model import LlamaIndexRankModel
 
+__all__ = [
+    "BaseModel",
+    "DummyGenerationModel",
+    "LlamaIndexEmbeddingModel",
+    "LlamaIndexGenerationModel",
+    "LlamaIndexRankModel",
+]

memory_scope/scheme/__init__.py

@@ -0,0 +1,11 @@
+# -*- coding: utf-8 -*-
+
+from .memory_node import MemoryNode
+from .message import Message
+from .model_response import ModelResponse
+
+__all__ = [
+    "MemoryNode",
+    "Message",
+    "ModelResponse",
+]

memory_scope/storage/__init__.py

@@ -0,0 +1,14 @@
+# -*- coding: utf-8 -*-
+from .base_memory_store import BaseMemoryStore
+from .dummy_memory_store import DummyMemoryStore
+from .llama_index_es_memory_store import LlamaIndexEsMemoryStore
+from .llama_index_sync_elasticsearch import SyncElasticsearchStore
+from .dummy_monitor import DummyMonitor
+
+__all__ = [
+    "BaseMemoryStore",
+    "DummyMemoryStore",
+    "LlamaIndexEsMemoryStore",
+    "SyncElasticsearchStore",
+    "DummyMonitor",
+]

memory_scope/utils/__init__.py

@@ -1,14 +1,21 @@
 # -*- coding: utf-8 -*-
 """ Import modules in utils package."""
-from .datetime_handler import DatetimeHandler
-from .global_context import GlobalContext
-from .logger import Logger
-from .memory_handler import MemoryHandler
-
+from memory_scope.utils.datetime_handler import DatetimeHandler
+from memory_scope.utils.global_context import GlobalContext
+from memory_scope.utils.logger import Logger
+from memory_scope.utils.memory_handler import MemoryHandler
+from memory_scope.utils.prompt_handler import PromptHandler
+from memory_scope.utils.response_text_parser import ResponseTextParser
+from memory_scope.utils.timer import Timer
+from memory_scope.utils.registry import Registry
 
 __all__ = [
     "DatetimeHandler",
     "GlobalContext",
     "Logger",
     "MemoryHandler",
+    "PromptHandler",
+    "ResponseTextParser",
+    "Timer",
+    "Registry",
 ]