Find a file
北沐城歌 6fb402dcd1
fix(embedding): keep one bad item from dropping its whole batch (#579)
* fix(embedding): keep one bad item from dropping its whole batch

A provider-side request error rejects an entire batch, so a single over-limit
item used to cost the vectors of every item beside it. QwenPaw #8040 shows two
poison chunks turning into exactly twenty lost vectors, while the backfill loop
still reported full success.

Retry a rejected multi-item request one item at a time so only the item that
actually fails is left without a vector. Single-item requests keep their
previous behaviour, and provider-level failures (429 / quota) keep using the
existing retry paths instead of being split.

Also make the backfill loop report what it actually filled, and name the failing
chunk ids in the final reindex error. `processed` counts attempts, so it claimed
success after a partial loss.

* fix(embedding): stop quota failures and transient item errors from multiplying

Review follow-up on the two P2 findings in #579:

* quota exhaustion no longer falls into the per-item split. The store now gives
  up explicitly when the provider reports insufficient_quota, including after
  the opt-in quota retries are spent, because no single item is to blame and
  splitting only multiplied the rejected requests (3 items -> 4 requests).
* the per-item fallback goes back through `_call_with_retry`, so a 429,
  connection error or timeout on an isolated item keeps the existing backoff
  and retry policy instead of silently dropping that item's vector. A one-item
  request still cannot split again.

Regression tests cover both findings: a quota failure stays a single request
with the default policy and with exhausted quota retries, and a transient
failure on one item of a split batch still ends with every healthy sibling
embedded.

python -m pytest tests/unit/test_local_embedding_store.py tests/unit/test_file_store_consistency.py -q
-> 83 passed, 41 skipped

* style(test): hoist the sleeps collector out of the transient-failure loop

CI / Python quality failed on d5dc9b2: the pylint hook reported

    tests/unit/test_local_embedding_store.py:935:16:
    W0640: Cell variable sleeps defined in loop (cell-var-from-loop)

`fake_sleep` closes over `sleeps` while the loop rebinds it on every
iteration. The collector is now created once before the loop and cleared
at the top of each iteration, which keeps the per-iteration assertion
(`assert sleeps == [1.0]`) exactly as strict and removes the loop-scoped
rebinding.

Verification (repo hook arguments):
  pylint  -> Your code has been rated at 10.00/10, exit 0
  black --check --line-length=120 --target-version=py311 -> clean
  flake8 --extend-ignore=E203 --max-line-length=120       -> clean

* fix(embedding): keep provider-wide failures out of the per-item split

Review follow-up on the P2 raised against 4d8a3c5.

`_call_with_retry` sent every unrecognized exception to `_embed_individually`
whenever the request had more than one item. Only rate limiting and
`insufficient_quota` were excluded, so a 401, a 403 or a 5xx still split a
ten-item request into ten extra single-item requests, none of which can
succeed: credentials, permissions and server outages are properties of the
endpoint, not of any individual input.

The split path is now gated on a dedicated classifier,
`_is_provider_wide_failure()`, which recognizes 401/403 and 5xx by status
code (with the OpenAI SDK class names as a fallback for exceptions that do
not expose one). Transient server errors keep their retry policy through the
existing backoff; the remaining provider-wide failures stop the loop and
report unhealthy, like the quota branch. Input-dependent rejections such as
the per-item context-length limit are unchanged and still isolate one item
at a time.

Regression coverage: test_provider_wide_credentials_errors_are_not_split_into_per_item_requests
(401/403 -> a single request of size 10) and
test_provider_wide_server_errors_are_retried_without_splitting
(500 -> batch sizes [10, 10, 10] with sleeps [1.0, 2.0]). The existing
context-length, quota, 429 and per-item transient-retry tests are retained.

Local verification, using the repo's own hook arguments:
  pylint  -> Your code has been rated at 10.00/10, exit 0
  black   -> 2 files would be left unchanged
  flake8  -> clean
Classifier discriminator check (old vs new split decision, extracted from the
production file and executed):
  400 context-length     -> SPLIT / SPLIT
  401, 403, 500, 504     -> SPLIT / NO-SPLIT
  429 and insufficient_quota keep their own earlier branches.

* fix(embedding): retry SDK transport failures instead of splitting the batch

Third review follow-up on #579.

`_call_with_retry` only caught the built-in `TimeoutError`, `ConnectionError`
and `OSError`. `openai.APIConnectionError` and `openai.APITimeoutError` do not
inherit from any of them (they go APIError -> OpenAIError -> Exception), so
they reached the generic handler: `_is_provider_wide_failure()` did not
recognize them either, and a multi-item request fanned out into per-item
requests that failed for exactly the same reason. The same gap meant a
single-item request got no ReMe-level retry at all after a transport failure.

Transport failures are now recognized by a dedicated classifier,
`_is_transient_transport_failure()`, covering the built-in socket errors as
well as the SDK/httpx exception types (matched across the MRO, so subclasses
such as `APITimeoutError` are included) without importing the provider SDK.
They retry the original request with the existing backoff and never split; the
retry budget is unchanged.

Regression coverage:
- test_sdk_transport_errors_are_retried_without_splitting
  APIConnectionError / APITimeoutError -> batch sizes [10, 10, 10], sleeps
  [1.0, 2.0], then failure with no single-item requests.
- test_sdk_transport_error_recovers_without_losing_the_batch
  one APITimeoutError then success -> [[1.0, 0.0]] * 10, batch sizes [10, 10].
- test_isolated_item_retries_an_sdk_transport_error
  context-length rejection, then APIConnectionError on one isolated item ->
  that item's vector is retained.
The existing context-length, 401/403, 5xx, quota and rate-limit tests are
retained.

Local verification with the repo's own hook arguments:
  pylint -> Your code has been rated at 10.00/10, exit 0
  black  -> 2 files would be left unchanged
  flake8 -> clean

Branch-order discriminator check (classifiers extracted from the production
file and executed against each error type):

  case                      old     new
  400 context-length        SPLIT   SPLIT
  401 / 403                 SPLIT   STOP
  500                       SPLIT   RETRY
  SDK APIConnectionError    SPLIT   RETRY
  SDK APITimeoutError       SPLIT   RETRY
  builtin ConnectionError   RETRY   RETRY
  builtin TimeoutError      RETRY   RETRY
  429                       RETRY   RETRY
  insufficient_quota        STOP    STOP

* fix(embedding): treat 404 as a provider-wide failure too

Fourth review follow-up on #579.

The classifier added in c9448ae covered 401/403 and 5xx, but not 404. A missing
model or a wrong embedding route answers with 404 (SDK `NotFoundError`), which
no individual input can explain, so the batch still fell through to
`_embed_individually()`: a ten-item request against a bad model name produced
eleven requests, all failing for the same reason.

404 is now part of the provider-wide set (`_PROVIDER_WIDE_STATUS_CODES`) and
`NotFoundError` joins the class-name fallback. Like 401/403 it is not
transient, so the request is reported as failed once instead of being retried
or split. The context-length `400` is deliberately left out of that set: an
over-limit item is the one status a single input can explain by itself, so it
has to stay isolatable.

Regression coverage:
- test_provider_wide_statuses_exclude_input_dependent_bad_request
  401/403/404/500 -> provider-wide; an over-limit 400 -> not provider-wide.
- test_provider_wide_not_found_errors_are_not_split
  404 -> batch sizes [10], no sleeps, no single-item requests.
The existing context-length isolation, 401/403, 5xx, quota, rate-limit and SDK
transport tests are retained.

Local verification with the repo's own hook arguments:
  pylint -> Your code has been rated at 10.00/10, exit 0
  black  -> 2 files would be left unchanged
  flake8 -> clean

Branch-order discriminator check (classifiers extracted from the production
file and executed against each error type):

  case                      old     new
  400 context-length        SPLIT   SPLIT
  401 / 403                 SPLIT   STOP
  404 NotFoundError         SPLIT   STOP
  500                       SPLIT   RETRY
  SDK APIConnectionError    SPLIT   RETRY
  SDK APITimeoutError       SPLIT   RETRY
  builtin ConnectionError   RETRY   RETRY
  builtin TimeoutError      RETRY   RETRY
  429                       RETRY   RETRY
  insufficient_quota        STOP    STOP

* fix(embedding): classify every provider status shape and split only input errors

The per-item fallback still multiplied service failures into one request per item.
Both gaps come from the fallback this PR adds, so they are fixed here.

Status extraction
- `_http_status()` reads the status from `status_code`, an integer `code` (Google GenAI
  exposes the HTTP status there), `response.status_code` (httpx-style wrappers), and the
  structured `status_code` embedded in the message (AgentScope's DashScope embedding turns
  every non-200 response into a plain `RuntimeError`).
- Only integers are read from `code`. A string `code` names an error
  (`insufficient_quota`, `rate_limit_exceeded`, `InvalidApiKey`); reading it as a status
  would misclassify quota and rate-limit errors.
- `_is_provider_wide_failure`, `_is_transient_provider_failure`, `_is_rate_limited` and
  `_is_input_dependent_failure` all classify through it.

Splitting
- The generic fallback splits only when `_is_input_dependent_failure()` matches: a
  context-length rejection, the one error a single over-long item explains by itself.
  Anything unrecognized is provider-wide and is never answered with one request per item.
- This narrows the isolation this PR adds. An input-dependent error that carries neither a
  status nor an input-length signal is no longer split, so it costs its batch instead of
  multiplying the requests. Erring towards the batch keeps one provider failure from
  becoming N failures and losing the healthy vectors anyway.
- `_embed_individually()` drops the redundant `len(single) == 1` check: the
  `_call_with_retry([text])` contract only returns a result of the expected length.

Verified against the real exception classes: `google.genai.errors.APIError.code` is an
`int`, and a real DashScope response yields a `RuntimeError` whose message carries
`"status_code"`. Regression coverage uses the real classes when available and mirrors
their shape otherwise.

* fix(embedding): stop per-item isolation on a provider-wide failure and read serialized codes

- `_embed_individually` now stops asking for the remaining items as soon as one isolated
  item fails provider-wide (credentials, missing model, exhausted quota, exhausted
  transient retries). No remaining input can explain that failure, so continuing only
  multiplies the rejected requests. The vectors already recovered are kept and the items
  that were never answered stay `None`.

- `_call_with_retry` reports that outcome through an optional `terminal_failure`
  out-parameter, so its return type and every existing call site stay unchanged.

- `_error_code` reads the business error code from the attribute, the structured body, or
  the serialized message. AgentScope's DashScope embedding raises a plain `RuntimeError`
  with the response embedded in the message, so a quota error was classified as a rate
  limit and never used the configured `quota_retry_delay`.

Regression tests cover both halves of the isolation boundary (a provider-wide failure
stops the loop; an input-dependent failure does not) and the serialized DashScope quota
path with quota retries enabled.

* fix(embedding): treat an input-length 413 as a per-item rejection

* fix(embedding): keep the health state after a partial success

`_compute_batch()` set `is_healthy = True` whenever the batch produced at least one
vector. Per-item isolation can recover an early item and then meet a provider-wide
failure (exhausted quota, authentication, exhausted transient retries), which
`_call_with_retry()` correctly records as unhealthy before it stops. The recovered
vector then overwrote that state, so a request that ended in an outage reported the
provider as healthy.

The terminal-failure flag is now threaded from `_embed_individually()` back to
`_compute_batch()`. The vectors recovered before the failure are still returned and
cached; only the health state now follows the failure. This matters because
`LocalFileStore._recover_after_real_request()` schedules missing-vector backfill off
the unhealthy-to-healthy transition: a hidden outage would either trigger backfill
while the provider is still unavailable or suppress a later genuine recovery.

Adds a regression test through `get_embeddings()` that checks both the retained
partial vector and the final unhealthy state. The existing isolation test covers
`_call_with_retry()` directly, so it never saw the overwrite.

* fix(embedding): leave provider 5xx retries to the layers that own them

A store-level retry for a provider 5xx stacked on top of the retries the
embedding wrapper and the provider SDK already perform, so a batch the SDK
answered after 12 attempts became 36. Retry ownership stays with those
layers; the store answers a provider-wide 5xx once and gives up.

Transport failures keep their retry because they describe a connection that
no lower layer has reported, and the opt-in quota policy is untouched.

Tests updated to pin the new boundary: a 503 is answered once for the
OpenAI-shaped, Gemini-shaped and DashScope-shaped errors, while a 429 keeps
its retry from the rate-limit branch.

* fix(embedding): make transport retry ownership explicit

The store retried a connection failure or a timeout on top of the embedding
wrapper and the provider SDK, which already retry both, so a persistent
timeout on a batch reached the provider 36 times instead of 12: the OpenAI
model retries a batch max_retries + 1 times and the SDK retries each of those
attempts.

Transport failures stay provider-wide -- the request is never split per item
-- but the retry is now owned explicitly instead of implicitly:

* a whole-batch request leaves it to the layers below, which already retry it;
* an isolated single-item attempt keeps the store's backoff, because that
  attempt is the last chance to keep the one vector the isolation step is
  recovering.

Rate limiting is unchanged. The transport cases now pin both budgets, and
three tests drive a real OpenAIAsEmbedding wrapper and a real
openai.AsyncClient over an httpx.MockTransport, so the provider attempt count
is measured rather than assumed.

* fix(embedding): check the failing error against the provider retry policy
2026-10-21 16:32:49 +08:00
.github chore(deps): bump the codeql group with 2 updates (#638) 2026-10-19 18:05:18 +08:00
benchmark refactor: generalize decision models and add recoverable context compaction (#626) 2026-10-19 17:56:30 +08:00
deploy feat(docker): Add Docker benchmark sandbox and pooled runner (#608) 2026-10-19 16:26:38 +08:00
docs docs: add MemLens-256K results to the multimodal memory blog (#639) 2026-10-21 16:31:47 +08:00
github-pages docs: add MemLens-256K results to the multimodal memory blog (#639) 2026-10-21 16:31:47 +08:00
integrations chore: release 0.4.2.0 and simplify publishing workflows (#631) 2026-10-16 16:53:52 +08:00
plugins fix(search): accept CLI queries that look like numbers (#612) 2026-10-16 10:36:05 +08:00
reme fix(embedding): keep one bad item from dropping its whole batch (#579) 2026-10-21 16:32:49 +08:00
reme_studio feat(studio): add bilingual browser demo to documentation site (#581) 2026-10-03 22:34:11 +08:00
scripts perf(search): prepare search filters once per query (#591) 2026-10-14 18:14:21 +08:00
skills docs: recommend two-stage service usage across guides (#620) 2026-10-15 11:41:17 +08:00
tests fix(embedding): keep one bad item from dropping its whole batch (#579) 2026-10-21 16:32:49 +08:00
.dockerignore feat(docker): add container deployment and multi-platform release workflow (#582) 2026-10-03 23:54:58 +08:00
.gitignore refactor(integrations): split TypeScript host plugins (#536) 2026-09-11 13:06:02 +08:00
.pre-commit-config.yaml feat: add daily paper cookbook and DingTalk agent integration (#385) 2026-07-22 19:17:01 +08:00
AGENTS.md fix(search): accept CLI queries that look like numbers (#612) 2026-10-16 10:36:05 +08:00
CLAUDE.md docs: restructure documentation and update content organization (#339) 2026-07-13 16:50:46 +08:00
docker-compose.yml feat(docker): add container deployment and multi-platform release workflow (#582) 2026-10-03 23:54:58 +08:00
Dockerfile feat(docker): add container deployment and multi-platform release workflow (#582) 2026-10-03 23:54:58 +08:00
example.env refactor: generalize decision models and add recoverable context compaction (#626) 2026-10-19 17:56:30 +08:00
LICENSE feat(reme_ai): implement memory retrieval and merging functionality 2025-08-25 16:10:53 +08:00
pyproject.toml refactor: generalize decision models and add recoverable context compaction (#626) 2026-10-19 17:56:30 +08:00
README.md refactor: generalize decision models and add recoverable context compaction (#626) 2026-10-19 17:56:30 +08:00
README_ZH.md refactor: generalize decision models and add recoverable context compaction (#626) 2026-10-19 17:56:30 +08:00

ReMe Logo

Python Version PyPI Version PyPI Downloads GitHub commit activity License Documentation English 简体中文 GitHub Stars DeepWiki

agentscope-ai%2FReMe | Trendshift

A local-first, self-evolving personal knowledge base for AI agents.

Previous versions: 0.3.x · 0.2.x · MemoryScope

✨ Why ReMe?

🧠 ReMe turns conversations and resources into readable, editable, searchable, and interconnected Markdown memory. Agents such as QwenPaw and DeepSeek Harness can share the same workspace to retrieve, maintain, and evolve knowledge, while users retain control of the durable files.

  • Memory as File, File as Memory: ReMe stores durable memory as ordinary Markdown with frontmatter and wikilinks. Users and agents can inspect, edit, move, sync, and back it up with familiar tools, while indexes and generated metadata remain rebuildable.
  • Self-evolving knowledge base: ReMe progressively turns conversations and resources into daily notes and long-term knowledge, preserving sources while refining facts, preferences, procedures, and relationships over time.
  • Recall is precise and context-aware. BM25, optional embeddings, and wikilink expansion retrieve relevant line-level passages and their relationships without loading the entire knowledge base into the agent context.
  • One memory workspace works across agents. Personal assistants, coding agents, and other agent runtimes can share the same local workspace through native integrations, SKILL.md, CLI, HTTP, MCP, or Python APIs.

ReMe Design Philosophy

📰 Latest Updates

  • [2026.10] - Decision-model context compaction: score old tool calls and results against the next action, preserve protected history, and return a recoverable compaction proposal. Includes an SVG walkthrough, two live-model integration scenarios, and a 14-query fixed-summary Top-N benchmark with a forced Hermes summary baseline.

  • [2026.10] - ReMe Multimodal Memory update: Auto Memory reads images with their conversation context and preserves image sources; Auto Resource turns image files into searchable daily cards linked to the originals.

  • [2026.10] - ReMe Studio Playground is live: explore example memory files, edit Markdown, and browse linked memory graphs right in your browser—no installation or backend required. Everyone is welcome to try it out!

    ReMe Studio workspace preview — click to try the Playground

  • [2026.09] - ReMe Memory Tags published: an introduction to file-native entity tags, rebuildable tag indexes, and tag-filtered memory search.

  • [2026.09] - Hermes Agent memory provider available: choose HTTP or embedded mode for automatic recall before model calls and asynchronous auto_memory after completed turns. The integration supports Hermes Agent 0.21+ and includes profile-aware background work.

More updates

🚀 Quick Start

Installation

ReMe requires Python 3.11+.

Install from pip:

pip install "reme-ai[core]"

Install from source:

git clone https://github.com/agentscope-ai/ReMe.git
cd ReMe
pip install -e reme_studio -e ".[core]"
cd reme_studio
npm ci
npm run build:static
cd ..

The static build requires Node.js 22.13 or newer and makes Studio available from the source tree.

Optional decision models use pip install "reme-ai[core,decision]". Configure a TypeSafe API key and run the service demo as described in Decision model configuration.

Docker

With Docker and Compose 2.24.0+, build and start ReMe with the bundled Studio:

mkdir -p .reme
docker compose up --build -d

Open http://127.0.0.1:2333. The complete workspace persists in ./.reme. On Linux, set REME_UID and REME_GID to your user's IDs when they differ from 1000. See Docker deployment for model credentials, custom paths, published images, and upgrades.

Environment Variables

Configure environment variables when you want LLM-powered memory evolution or embedding retrieval. Embeddings are disabled by default, so the default setup does not start an embedding model or require an embedding API key.

cat > .env <<'EOF'
# Optional: used only after embedding components are explicitly enabled in the config.
# EMBEDDING_API_KEY=sk-xxx
# EMBEDDING_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1

# Required for auto_memory, auto_resource, auto_dream, and proactive refresh.
LLM_API_KEY=sk-xxx
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
EOF

Basic file operations, BM25 search, wikilink traversal, and reading proactive topics can run without LLM credentials.

Note

To enable embedding-based semantic retrieval, uncomment components.as_embedding and components.embedding_store in reme/config/default.yaml, then change components.file_store.default.embedding_store from "" to default. See the memory search guide for details.

Start the Service

Start ReMe in one terminal and leave it running:

reme start

The default service address is 127.0.0.1:2333. If the port is occupied, specify another port:

reme start service.port=8181
# reme start workspace_dir=/tmp/reme-demo service.port=8181

From another terminal, call the running service. The CLI discovers its configuration; the curl example uses the default port 2333:

reme version
reme health_check
reme help
curl -s http://127.0.0.1:2333/version -H 'Content-Type: application/json' -d '{}'

5-Minute Memory Demo

With the service running, write a memory node, let ReMe index it, then retrieve it:

reme write \
  path=digest/wiki/quick-start-demo \
  name="Quick Start Demo" \
  description="A first ReMe memory node" \
  content="# Quick Start Demo

ReMe stores agent memory as readable Markdown.

Related: [[digest/wiki/memory-as-file.md]]"

reme search query="agent memory markdown" limit=5
reme read path=digest/wiki/quick-start-demo start_line=1 end_line=20

The generated file is ordinary Markdown with frontmatter:

---
name: Quick Start Demo
description: A first ReMe memory node
---

# Quick Start Demo

ReMe stores agent memory as readable Markdown.

Related: [[digest/wiki/memory-as-file.md]]

ReMe Studio (Optional)

The core installation includes Studio. After starting ReMe, open http://127.0.0.1:2333/ to browse, edit, and search the workspace. To add Studio to a base installation, use pip install "reme-ai[web]". See the ReMe Studio guide for source builds, configuration, and development.

🤝 Use ReMe with Your Agent

ReMe can run as a local memory service accessed through the CLI, HTTP API, or MCP server, or it can be embedded in the host process through its Python API. Host integrations can add memory guidance, recall, and capture to the agent lifecycle according to the capabilities of each runtime.

Agent Recommended path Available after integration
DeepSeek Harness Install @agentscope-ai/reme-dsh-plugin with dsh plugin --profile web add @agentscope-ai/reme-dsh-plugin. Configurable memory guidance, reme_search, automatic turn capture, scheduled Auto Dream, and ReMe Status.
OpenClaw Install @agentscope-ai/reme-openclaw-plugin with openclaw plugins install clawhub:@agentscope-ai/reme-openclaw-plugin. Native memory tools, recall before user-triggered runs, and automatic turn capture.
QwenPaw Embed ReMe in-process through its Python API. Reuse the host lifecycle and model config while keeping memory local and file-based.
Claude Code Start ReMe, then install the Claude Code plugin. Automatic recall, batched completed-turn recording, and MCP tools.
Hermes Install the ReMe provider and choose HTTP or embedded mode. Recall before model calls and asynchronous auto_memory after each completed turn.
Codex Start ReMe, then install the Codex plugin. Native hooks for automatic recall and batched recording, plus MCP tools.
Other CLI agents Install or copy the ReMe Memory skill. Search, read, and write memory through the CLI.

Integration demos

Auto Memory Auto Dream
QwenPaw QwenPaw Auto Memory demo QwenPaw Auto Dream demo
Claude Code Claude Code Auto Memory demo Claude Code Auto Dream demo

🧠 How ReMe Works

Memory as File, File as Memory.

ReMe treats memory as files, progressively processing filtered conversation source records and external resources from session/ and resource/ into daily/, then digest/. The default workspace is .reme/ under the current directory; workspace_dir=... selects a different user-owned location.

Workspace Layout

<workspace_dir>/
├── metadata/       # Rebuildable indexes, graphs, catalogs, and caches
├── session/        # Conversation source records and agent sessions
│   ├── dialog/
│   │   └── <session_id>.jsonl  # Source messages saved by auto_memory
│   └── claude_code/
│       └── <session_id>.jsonl  # ReMe copy used by auto_memory_cc
├── mem_session/    # Generated agent-wrapper sessions/config, not user memory
│   ├── agentscope/
│   ├── claude_config/
│   └── codex/
├── resource/            # External raw materials
│   ├── <resource>.<ext>  # Root-level files enter today's daily layer
│   └── YYYY-MM-DD/
│       └── <resource>.<ext>
├── daily/               # Lightly processed memory: daily facts, conversation summaries, resource readings
│   ├── YYYY-MM-DD.md
│   └── YYYY-MM-DD/
│       ├── <generated_name>.md  # Topic-named conversation or resource card
│       └── interests.yaml
└── digest/              # Long-term memory: personal facts, procedural experience, knowledge nodes
    ├── personal/
    │   └── {topic/event}.md
    ├── procedure/
    │   └── {topic/event}.md
    └── wiki/
        └── {topic/event}.md

ReMe file-based memory system overview

Memory Lifecycle

ReMe follows a capture → index → consolidate → recall loop. Workspace files remain the durable source of truth; everything under metadata/ is rebuildable.

Capability Entry point What it does Output
auto_memory Agent hook or reme auto_memory Distills useful conversation facts while preserving a filtered conversation source record. session/dialog/*.jsonl, daily/<date>/<generated-name>.md
auto_resource Resource watcher or reme auto_resource Turns files under resource/ into source-linked, content-named daily cards. daily/<date>/<resource-card>.md
auto_index Background watcher or reme reindex The watcher ingests Markdown from daily/ and digest/; reindex only rebuilds BM25 and embeddings from already-ingested chunks. Searchable chunks, BM25, wikilink graph, and optional vectors
auto_dream dream_cron or reme auto_dream By default, extracts up to five reusable units from changed files in the latest two-day window, then creates, corroborates, refines, or corrects digest nodes. digest/**
proactive_read reme proactive_read before an agent decides to act Reads topics generated by the independent proactive refresh flow; the host agent decides whether and how to mention them. Structured topics from daily/<date>/interests.yaml
Memory as File Auto Memory and Resource
Auto Dream and Proactive Auto Index and Memory Search

Search returns matching chunks with line ranges and bounded wikilink neighbors. Optional vector results are fused with BM25 through reciprocal rank fusion (RRF).

Important

proactive_read only reads and exposes interest topics produced by proactive refresh. It does not independently browse the web, send notifications, or rewrite the knowledge base; the host agent decides whether and how to act on a topic.

📊 Benchmarks

ReMe evaluates multi-session and long-context memory with agentic search-and-read workflows. The figures below are the published reference runs in this repository; model, prompt, dataset, and judging details are documented with each benchmark.

Benchmark Setting Sample size Agentic score Focus
LongMemEval cleaned-s Overall 500 questions 89.4% Cross-session retrieval, knowledge updates, and temporal reasoning
BEAM 100K context 20 cases / 400 questions 66.1% Ten types of long-context memory tasks
BEAM 1M context 35 cases / 700 questions 65.0% Ultra-long conversation settings

ReMe also achieved a 0.580 PROC score across five user personas in the repository's π-Bench evaluation, 2.4% above NanoBot under the same test-model configuration. PROC measures proactive handling of hidden intent, clarification, cross-session preferences and conventions, task dependencies, and underspecified requests.

🧩 Extensions and Plugins

Plugins are optional Python distributions that contribute Component, Step, or Job backends and configuration. They are installed separately and enabled explicitly by configuration. Daily Paper and Auto Fin are independently packaged plugins; see their documentation for Daily Paper and Auto Fin.

Plugin Capability
Daily Paper Discover and rank papers, analyze PDFs with an agent, and generate file-native notes and a five-minute brief.
Auto Fin Fetch topic-related CLS news, search ReMe history, and generate wikilink-backed Markdown reports.

See Plugin Management to install, inspect, validate, enable, and uninstall ReMe plugins.

📚 Documentation

These guides cover the main user workflows and the runtime contracts implemented by the current code.

Guide What you will learn
Quick Start Install ReMe, start the service, and run the first file and memory operations.
Configuration Configure the workspace, models, Service, Jobs, Components, plugins, and CLI overrides.
Services and Deployment Use HTTP, SSE, MCP, and Studio while respecting the default security boundary.
Memory as File Understand workspace layers, frontmatter, wikilinks, chunks, and the file-as-source-of-truth model.
Auto Memory Preserve source conversations and distill reusable daily memory cards.
Auto Resource Import supported text and image resources as source-linked daily cards.
Auto Dream and Auto Link Consolidate daily notes into evolving digest nodes and readable wikilink relationships.
Memory Search Use BM25, optional vectors, RRF fusion, line-range recall, and progressive link expansion.
Proactive Read interest topics safely and integrate them into a host agent's decision flow.
Application Scenarios Follow concrete financial research, coding-memory, and personal knowledge-base examples.
Framework Understand Application, Job, Step, Component, service, configuration, and lifecycle boundaries.
Agent Integrations Choose an interface and connect DSH, Claude Code, OpenClaw, Hermes, Codex, or another agent.
DSH plugin and Claude Code plugin Configure host-native recall, automatic capture, consolidation, and diagnostics.
CLI and Job API Learn command syntax and use the generated default Job parameter reference.
Operations and Recovery Diagnose services, maintain indexes, and back up, migrate, or recover a workspace.
ReMe Blog Read the product story, design rationale, examples, and benchmark summary.

🛠️ Common Commands

Run reme help for the full job list. Common workspace and maintenance commands are:

Command Purpose
reme status Show stateful data-component memory estimates and process RSS.
reme search Retrieve memory with BM25 and wikilinks by default, plus vectors when enabled.
reme read / reme write / reme edit Inspect and maintain Markdown memory files.
reme traverse / reme graph_snapshot Explore wikilink neighborhoods or the category-rooted digest graph.
reme chat Stream a read-only, workspace-aware agent conversation. Requires LLM credentials.
reme reindex Rebuild BM25 and embedding indexes from already-ingested chunks.

🤝 Community and Contributing

  • Issues, requests, and help: Check Open Issues first. If there is no related discussion, open one with the background, expected behavior, and impact scope.
  • Code contributions: Before making changes, read the repository's contribution guide. Source, schemas, and tests are the authoritative architecture and extension guide.
  • Documentation contributions: Update the canonical files under docs/en/, docs/zh/, or the relevant package directory in this repository. The documentation site is generated from these files.
  • Commit convention: Conventional Commits are recommended, for example feat(search): add link expansion option or docs(zh): update quick start.
  • Pre-submit checks: Before submitting a PR, try to run pre-commit run --all-files and pytest. If tests that depend on LLMs, embeddings, or external services cannot run, explain that in the PR.
  • Documentation: Visit reme.agentscope.io.

Contributors

Thanks to everyone who has contributed to ReMe:

Contributors

📄 Citation

@software{ReMe2026,
  title = {Remember me, Refine me: Memory Management Kit for Agents},
  author = {ReMe Team},
  url = {https://reme.agentscope.io},
  year = {2026}
}

⚖️ License

This project is open source under the Apache License 2.0. See LICENSE for details.