docs: add comprehensive documentation for auto-dream, auto-link, and auto-resource flows (#343)

AGENTS.md

@@ -33,6 +33,8 @@ the code, schema, tests, configuration, and concise documentation together as ne
 
 - `reme/reme.py`: CLI entry point and client/server dispatch.
 - `reme/application.py`: application assembly, dependency ordering, and lifecycle.
+- `reme/components/application_context.py`: application-wide wiring and shared in-memory metadata.
+- `reme/components/runtime_context.py`: scratch state shared by steps within one execution.
 - `reme/config/default.yaml`: built-in jobs, components, and defaults.
 - `reme/schema/`: public and runtime Pydantic contracts.
 - `reme/components/`: services, stores, clients, jobs, and component registration.
@@ -79,6 +81,34 @@ Do not silently change stable CLI flags, configuration keys, workspace layouts,
 schemas, or service interfaces. When such a change is required, preserve compatibility
 where practical and make the migration explicit.
 
+## Step State Model
+
+Treat every Step as stateless. `BaseJob` stores Step specifications and builds fresh Step
+instances for each Job invocation. A Step instance must not use `self` or class variables to
+retain mutable runtime state between calls.
+
+Place state according to its lifetime:
+
+- Constructor fields on `self`: immutable Step configuration and resolved dependencies only.
+- `self.context` (`RuntimeContext`): request data and intermediate results for one Job
+  execution; sequential Steps share this context.
+- `self.app_context.metadata`: in-memory state that must be shared across Step or Job
+  invocations for the lifetime of the Application.
+- Workspace files or a dedicated Component/store: durable state that must survive an
+  Application restart.
+
+Use narrow, namespaced keys in `app_context.metadata`, following existing patterns such as
+`tool_contexts` and `channel_sink`. The ApplicationContext is shared, so account for
+concurrent access when values are mutable. New Step code must not fall back to `self.kwargs`
+or another Step field to emulate shared state when `app_context` is absent; tests of shared
+state should construct an `ApplicationContext`. If shared state grows into a stable
+service-level contract or needs its own lifecycle, locking, or persistence, promote it to a
+typed ApplicationContext field or a dedicated Component instead of expanding an ad hoc
+metadata bucket.
+
+Do not use `Response.metadata` as a state store. It is request-scoped output for callers and
+diagnostics, distinct from `ApplicationContext.metadata`.
+
 ## Validation
 
 Use the narrowest useful check while iterating, then broaden it according to risk.

docs/en/auto_dream.md

@@ -1,7 +1,209 @@
 # Auto Dream
 
-```{note}
-📖 The English version of this page is in progress.
+`auto_dream` is ReMe's long-term memory distillation flow from daily to digest. It scans daily inputs for a specified date,
+processes only files that changed since the previous dream, extracts content worth retaining as memory units, integrates those
+units into `digest/`, and generates the day's `interests.yaml` for proactive use.
 
-In the meantime, please read the <a href="../zh/auto_dream.html">中文版本</a>.
+<p align="center">
+  <img src="../figure/auto-dream-and-proactive.svg" alt="ReMe Auto Dream and Proactive flow from daily to digest to proactive" width="92%">
+</p>
+
+Its daily inputs usually come from [Auto Memory](./auto_memory.md) and [Auto Resource](./auto_resource.md). For the file
+semantics of `digest/`, `derived_from::`, and wikilinks, see [Memory as File](./memory_as_file.md). For the linking strategy
+used during Integrate, see [Auto Link](./auto_link.md). To read `interests.yaml`, use [Proactive](./proactive.md).
+
+## Configuration
+
+The default configuration is in `reme/config/default.yaml`:
+
+```yaml
+auto_dream:
+  backend: base
+  parameters:
+    date:
+      type: string
+      default: ""
+    hint:
+      type: string
+      default: ""
+    topic_count:
+      type: integer
+      default: 3
+    topic_diversity_days:
+      type: integer
+      default: 7
+  steps:
+    - backend: dream_extract_step
+      file_catalog: dream
+      topic_session_id: interests
+    - backend: dream_integrate_step
+    - backend: dream_topics_step
+      topic_count: 3
+      topic_diversity_days: 7
+    - backend: dream_finish_step
+      file_catalog: dream
 ```
+
+Parameters:
+
+| Parameter | Purpose |
+|---|---|
+| `date` | Date to process in `YYYY-MM-DD` format. When empty, use today in the application's timezone. |
+| `hint` | Additional guidance from the caller for the Extract and Integrate stages. |
+| `topic_count` | Maximum number of topics written to `interests.yaml`. Defaults to 3. |
+| `topic_diversity_days` | Number of past days of `interests.yaml` files considered when avoiding duplicate topics. Defaults to 7. |
+
+## Inputs and Outputs
+
+Inputs are daily Markdown files for the specified date:
+
+```text
+daily/<date>.md
+daily/<date>/**/*.md
+```
+
+`daily/<date>/interests.yaml` is excluded from extraction input so topics from the previous run do not feed back into the
+next extraction.
+
+The main outputs are:
+
+| Output | Description |
+|---|---|
+| `digest/procedure/*.md` | Methods, workflows, runbooks, and executable experience. |
+| `digest/personal/*.md` | User-, team-, and project-related preferences, facts, and long-term context. |
+| `digest/wiki/*.md` | General knowledge, concepts, observations, and decision precedents. |
+| `daily/<date>/interests.yaml` | Topics worth proactive attention from the host agent that day. |
+| `metadata/file_catalog/dream*` | Dream-specific catalog used to detect changes in daily inputs. |
+
+## Four Stages
+
+### 1. Extract
+
+`dream_extract_step` performs three tasks:
+
+1. Refresh the day's index page at `daily/<date>.md`.
+2. Scan `daily/<date>.md` and `daily/<date>/**/*.md` and compare their mtimes with `file_catalog: dream`.
+3. Send only changed files to the LLM and globally extract two structured result types: `units` and `topics`.
+
+`units` are long-term memory units ready to be distilled into digest. Each has `name`, `bucket`, `summary`, and `paths`.
+`bucket` may only be `procedure`, `personal`, or `wiki`; unknown values are routed to `wiki`.
+
+`topics` are proactive-interest candidates for the day. They contain `title`, `reason`, `evidence`, `keywords`, and
+`paths` and are filtered again in the Topics stage.
+
+If there are no changed files, the flow ends early with success and skips later extraction work. If files changed but no LLM
+is configured, Extract fails because extraction requires an LLM.
+
+### 2. Integrate
+
+`dream_integrate_step` invokes an agent independently for each unit and integrates that unit into one digest node. It exposes
+these tools to the agent:
+
+```text
+node_search, read, frontmatter_read, write, edit, frontmatter_update
+```
+
+This stage carries the core responsibility of `auto_link`. It first uses `node_search` to recall similar or related nodes at
+digest-node granularity, decides whether to create or update a node, and finally writes sources and related digest nodes as
+wikilinks. See [Auto Link](./auto_link.md) for the recall, deduplication, and edge-writing rules.
+
+There are four integration actions:
+
+| Action | Meaning |
+|---|---|
+| `CREATE` | No equivalent abstraction exists; create a new digest node. |
+| `CORROBORATE` | The same memory appeared again; append a source or strengthen the description. |
+| `REFINE` | New material adds boundaries, steps, prerequisites, applicability, or detail. |
+| `CORRECT` | New material corrects errors, omissions, or conflicts in the existing node. |
+
+Successfully integrated units are recorded in `integrate_results`. Failed units enter `failed_units`, and their source paths
+enter `failed_paths`. The Finish stage does not checkpoint failed paths, ensuring that they can be retried later.
+
+### 3. Topics
+
+`dream_topics_step` turns topic candidates from Extract into the final `daily/<date>/interests.yaml` for the day.
+
+It reads:
+
+```text
+daily/<date>/interests.yaml
+daily/<previous-date>/interests.yaml
+```
+
+Existing topics from the same day are preserved, while similar topics from the previous `topic_diversity_days` days are
+deduplicated. At most three topics are written by default. With an LLM configured, the LLM selects topics that are more
+specific, actionable, and non-repetitive. Without an LLM, the step falls back to local normalization and deduplication.
+
+Example output format. See [Proactive](./proactive.md) for the interface that reads this file:
+
+```yaml
+date: 2026-06-20
+topic_count: 3
+diversity_days: 7
+topics:
+  - title: Quality regression in the memory retrieval pipeline
+    reason: The user has recently made repeated changes to search, node_search, and dream integration.
+    evidence: daily/2026-06-20/session.md
+    keywords:
+      - memory search
+      - auto dream
+    paths:
+      - daily/2026-06-20/session.md
+```
+
+### 4. Finish
+
+`dream_finish_step` completes the run:
+
+1. Write successfully processed changed paths to `file_catalog: dream`.
+2. Also write `daily/<date>/interests.yaml` and `daily/<date>.md` to the catalog.
+3. Persist the dream catalog if there were upserts or deletions.
+4. Return a summary containing counts for scanned, changed, integrated, topics, checkpoints, and related values.
+
+Failed paths are not checkpointed. The next `auto_dream` run therefore continues to treat them as changed inputs until
+integration succeeds.
+
+## Running Auto Dream
+
+CLI:
+
+```bash
+reme auto_dream date=2026-06-20
+```
+
+With caller guidance:
+
+```bash
+reme auto_dream date=2026-06-20 hint="Prioritize engineering decisions and long-term preferences"
+```
+
+The same set of steps can also be placed in a `cron` Job, for example to run every morning:
+
+```yaml
+jobs:
+  daily_auto_dream:
+    backend: cron
+    cron: "30 3 * * *"
+    steps:
+      - backend: dream_extract_step
+        file_catalog: dream
+      - backend: dream_integrate_step
+      - backend: dream_topics_step
+      - backend: dream_finish_step
+        file_catalog: dream
+```
+
+## Important Boundaries
+
+`auto_dream` consumes only daily inputs and does not rewrite daily bodies. Daily preserves facts and the original situation;
+digest is the abstracted long-term memory layer.
+
+`digest` is not a copy of the source text. Its body should preserve reusable abstractions, while details point back to sources
+through `derived_from:: [[daily/<date>/...]]`. Links follow the workspace-relative wikilink semantics described in
+[Memory as File](./memory_as_file.md).
+
+`auto_dream` does not invent an overview from nothing. Only content that actually appears in daily input and is extracted as
+a unit or topic can enter digest or `interests.yaml`.
+
+The complete flow depends on an LLM for Extract and Integrate. Topics can perform local deduplication without an LLM, but that
+does not mean the full dream flow can run offline.

docs/en/auto_link.md

@@ -1,7 +1,144 @@
 # Auto Link
 
-```{note}
-📖 The English version of this page is in progress.
+In the current implementation, `auto_link` is not a separately registered Job. It is a capability of the Integrate stage in
+`auto_dream`: when `dream_integrate_step` writes a memory unit to `digest/`, it also recalls digest nodes, makes a
+deduplication decision, links sources, and weaves wikilinks to related nodes into the result.
 
-In the meantime, please read the <a href="../zh/auto_link.html">中文版本</a>.
+For the complete dream flow, see [Auto Dream](./auto_dream.md). For general wikilink, frontmatter, and workspace-relative
+path semantics, see [Memory as File](./memory_as_file.md). For question-answering retrieval, see
+[Memory Search](./memory_search.md).
+
+## Where It Runs
+
+The default `auto_dream` flow is:
+
+```yaml
+auto_dream:
+  steps:
+    - dream_extract_step
+    - dream_integrate_step   # where auto_link actually happens
+    - dream_topics_step
+    - dream_finish_step
 ```
+
+The Integrate stage processes each unit independently. A unit is written to exactly one target digest node, but that node may
+link to multiple sources and multiple related digest nodes.
+
+## Goals
+
+`auto_link` addresses graph quality at write time:
+
+| Problem | Handling |
+|---|---|
+| The same memory already exists | Recall and update the existing node instead of creating a duplicate. |
+| New and existing material are related | Write workspace-relative wikilinks into the body. |
+| A digest node is disconnected from its sources | Point back to daily/resource source material with `derived_from:: [[...]]`. |
+| A node contains only isolated prose | Add links to related digest nodes on both CREATE and UPDATE. |
+
+## Toolchain
+
+`dream_integrate_step` exposes these tools to the agent:
+
+```text
+node_search
+read
+frontmatter_read
+write
+edit
+frontmatter_update
+```
+
+`node_search` is digest-only node retrieval designed for dream integration. It returns node-level signals such as the digest
+node's `path` and the `name` and `description` from frontmatter. It does not expand the body and does not perform the link
+expansion used by ordinary search.
+
+`read` and `frontmatter_read` are used only for candidates that may be relevant, avoiding expansion of every recalled result
+into a large context.
+
+## Linking Flow
+
+### 1. Recall candidate nodes
+
+The agent first calls `node_search` with the unit's triggers, verbs, nouns, synonyms, and possible failure modes. Broad recall,
+for example `limit=20-30`, is recommended by default because this step serves both deduplication and link discovery.
+
+Recalled results are internally classified into three groups:
+
+| Classification | Meaning | Next action |
+|---|---|---|
+| `same_abstraction` | The trigger or underlying abstraction is the same, with substantial content overlap. | Use as the UPDATE target. |
+| `related` | An adjacent process, prerequisite, failure mode, concept, preference, or upstream/downstream knowledge. | Write a body wikilink. |
+| `unrelated` | Only superficially similar or unrelated. | Ignore. |
+
+### 2. Choose a write action
+
+Every unit must select one action:
+
+| Action | Linking semantics |
+|---|---|
+| `CREATE` | Write a new `digest/<bucket>/<slug>.md` and add source and related-node links to its body. |
+| `CORROBORATE` | The same abstraction appeared again; append a new `derived_from:: [[...]]` and strengthen the description when needed. |
+| `REFINE` | New material extends the existing node; insert the additional content in the appropriate section and preserve existing links. |
+| `CORRECT` | New material corrects the existing node; use source links to identify the basis for the correction. |
+
+An UPDATE should be additive whenever possible: do not delete existing wikilinks or `derived_from` entries. This prevents
+later graph indexing and retrieval from losing edges.
+
+### 3. Write source edges
+
+Source edges use Markdown wikilinks:
+
+```markdown
+derived_from:: [[daily/2026-06-20/session.md]]
+derived_from:: [[resource/2026-06-20/paper.md]]
+```
+
+These edges represent the evidence behind a digest node. Plain-text descriptions do not count as source edges because only
+wikilinks can be parsed reliably by the file graph. For the complete parsing rules, see
+[Memory as File](./memory_as_file.md#wikilink).
+
+### 4. Write relationships between digest nodes
+
+Relationships between digest nodes also use complete workspace-relative paths:
+
+```markdown
+relates_to:: [[digest/wiki/hybrid-search.md]]
+depends_on:: [[digest/procedure/rebuild-index.md]]
+blocks_on:: [[digest/personal/team-review-preference.md]]
+```
+
+Predicates are open-ended. Common forms include `relates_to::`, `depends_on::`, and `blocks_on::`. The predicate sits outside
+the brackets, while the target path goes inside `[[...]]` and should include the `.md` suffix.
+
+## Bucket Differences
+
+`auto_link` adjusts the shape of its output according to the unit bucket:
+
+| Bucket | Writing focus |
+|---|---|
+| `procedure` | Write a runbook with triggers, steps, inputs, and failure modes. Link prerequisites, substeps, and related preferences. |
+| `personal` | Write user-, team-, or project-specific facts and preferences. Link related projects, habits, and decision context. |
+| `wiki` | Write general knowledge, principles, observations, and decision precedents. Link concepts, methods, and adjacent knowledge. |
+
+Regardless of bucket, preserve source edges and weave recalled related digest nodes into the body whenever possible.
+
+## Relationship to Search
+
+`auto_link` uses `node_search`, not the question-answering `search`.
+
+| Capability | Purpose |
+|---|---|
+| `search` | External question answering; returns chunks and can expand upstream/downstream link context. |
+| `node_search` | Dream integration; recalls only digest node-level summaries for deduplication and related-link decisions. |
+
+This boundary matters. The Integrate stage needs to decide whether the same abstraction already exists and which nodes should
+be linked; it should not load large numbers of body chunks into context. [Memory Search](./memory_search.md) handles
+question-oriented chunk retrieval, RRF fusion, and link expansion.
+
+## Failure and Retry
+
+If integration of a unit fails, `dream_integrate_step` records `failed_units` and `failed_paths`.
+`dream_finish_step` does not checkpoint those source paths, so the next `auto_dream` run processes them again.
+
+This makes auto_link writes retryable: a failure does not mark the input as complete or silently discard digest edges that
+should have been created.

docs/en/auto_memory.md

@@ -1,7 +1,109 @@
 # Auto Memory
 
-```{note}
-📖 The English version of this page is in progress.
+Auto Memory is ReMe's entry point for conversational memory. Each conversation is first distilled into a daily memory card
+identified by `session_id`, and the day's `YYYY-MM-DD.md` page then indexes all of those cards. It turns "we talked about it"
+into "it was remembered" while preserving the original conversation as evidence.
 
-In the meantime, please read the <a href="../zh/auto_memory.html">中文版本</a>.
+<p align="center">
+  <img src="../figure/auto-memory-resource.svg" alt="ReMe Auto Memory and Auto Resource writing daily memory cards" width="92%">
+</p>
+
+For the general file semantics of `daily/`, `session/`, frontmatter, and wikilinks, see
+[Memory as File](./memory_as_file.md).
+
+```text
+Conversation
+  ├─ step 1: daily/YYYY-MM-DD/<session_id>.md   # one card per conversation
+  ├─ step 2: daily/YYYY-MM-DD.md                # daily index linking the cards
+  └─ source: session/dialog/<session_id>.jsonl  # original conversation
 ```
+
+## What It Records
+
+Auto Memory does not preserve a chat transcript as a running summary. It records information that may remain useful later:
+
+- User preferences: preferred style, collaboration habits, and long-term requirements.
+- Key facts: project background, important numbers, explicit conclusions, and constraints.
+- Process decisions: what happened, why a choice was made, and which alternatives were rejected.
+- Current state: what has been completed, what is blocked, and what comes next.
+- Reusable experience: commands, workflows, diagnostic methods, and solutions.
+
+## Write Location
+
+Auto Memory writes distilled memories to `daily/`. Conversations from the same day first become individual cards:
+
+Example directory:
+
+```text
+workspace/
+  daily/
+    2026-06-20.md
+    2026-06-20/
+      session-a.md
+      session-b.md
+```
+
+`daily/2026-06-20/session-a.md` and `daily/2026-06-20/session-b.md` are memory cards distilled from different
+conversations. `daily/2026-06-20.md` is the index page for that day. Resource files enter the same daily memory layer; see
+[Auto Resource](./auto_resource.md).
+
+When a call includes `session_id`, Auto Memory records that conversation separately under the given ID:
+
+```text
+daily/2026-06-20/session-a.md
+```
+
+This keeps different conversations separate. A requirements discussion, a debugging session, and a documentation update can
+each have their own memory card. To see what happened on a particular day, start with `YYYY-MM-DD.md`. To inspect what was
+distilled from one conversation, open the corresponding `<session_id>.md`.
+
+## Preserving the Original Information
+
+The distilled daily note is optimized for readability; the original conversation is retained for trust and verification.
+
+While generating memory cards, Auto Memory also saves the raw sessions:
+
+```text
+session/
+  dialog/
+    session-a.jsonl
+    session-b.jsonl
+```
+
+Each daily note points to its corresponding original conversation. When a memory needs verification, follow that link back to
+the complete context in which it was created.
+
+## Message Timestamps
+
+Auto Memory preserves each message's `created_at` in both the prompt and the raw session JSONL. When importing historical
+conversations or benchmark data, provide the actual occurrence time for every message so the model does not confuse event
+time with execution time:
+
+```bash
+reme auto_memory \
+  session_id=locomo-session \
+  messages='[
+    {"role":"user","content":"Jon lost his job today.","created_at":"2023-01-19T08:00:00"},
+    {"role":"assistant","content":"I am sorry to hear that.","created_at":"2023-01-19T08:01:00"}
+  ]'
+```
+
+For compatibility with common dataset schemas, `auto_memory` also checks `time_created`, `timestamp`, `createdAt`,
+`timeCreated`, and `created_time` when `created_at` is absent. These fields may appear either at the top level of a message
+or inside `metadata`.
+
+When a call does not explicitly provide `date`, Auto Memory uses the date of the earliest valid `created_at` value in the
+messages. If no message contains a valid timestamp, it falls back to the current date. Historical imports may also specify the
+target date directly:
+
+```bash
+reme auto_memory \
+  session_id=locomo-session \
+  date=2023-01-19 \
+  messages='[{"role":"user","content":"Jon lost his job today."}]'
+```
+
+## What Happens Next
+
+Auto Memory only creates memory in the daily layer. To distill this material further into long-term `digest/` nodes, use
+[Auto Dream](./auto_dream.md). To search daily and digest content, use [Memory Search](./memory_search.md).

docs/en/auto_resource.md

@@ -1,7 +1,101 @@
-# Auto Resource
+# Auto Resource `Beta`
 
-```{note}
-📖 The English version of this page is in progress.
+Auto Resource is ReMe's entry point for interpreting resources and is currently in **Beta**. Resource files first enter
+`resource/` by date and are then interpreted into daily resource cards. Each card's filename comes from the LLM-generated
+frontmatter `name`, and `source_resource` links the card back to its original file.
 
-In the meantime, please read the <a href="../zh/auto_resource.html">中文版本</a>.
+<p align="center">
+  <img src="../figure/auto-memory-resource.svg" alt="ReMe Auto Memory and Auto Resource writing daily memory cards" width="92%">
+</p>
+
+For the general file semantics of workspace layers, `resource/`, and `daily/`, see
+[Memory as File](./memory_as_file.md). For the flow that writes conversations to daily, see
+[Auto Memory](./auto_memory.md).
+
+```text
+resource/YYYY-MM-DD/<resource_file>
+  ├─ step 1: daily/YYYY-MM-DD/<generated_name>.md # interpreted resource card
+  ├─ step 2: source_resource points to the original resource
+  └─ step 3: daily/YYYY-MM-DD.md                  # daily index linking the cards
 ```
+
+## What It Records
+
+Auto Resource does more than copy file content. It extracts information that will make the resource easier to retrieve and
+understand later:
+
+- Core content: what the resource is mainly about.
+- Structure: its sections, tables, fields, and data organization.
+- Key details: important numbers, names, dates, and conclusions.
+- Context and purpose: why the resource exists and how it relates to current work.
+- Actionable items: tasks, deadlines, and follow-up work.
+
+In short, it turns "a file was archived" into "the resource is usable."
+
+## Original Resource Entry Point
+
+Auto Resource uses `resource/` as the entry point for source material. Resources must be placed under a date, which determines
+the day whose daily memory layer receives the interpreted card.
+
+Example directory:
+
+```text
+workspace/
+  resource/
+    2026-06-20/
+      market-report.md
+      meeting-notes.csv
+```
+
+The current Beta version is best suited to text-based resources such as `md`, `txt`, `json`, `jsonl`, `csv`, `yaml`,
+and `html`.
+
+## Resource Cards
+
+Each resource file produces one daily resource card. The system initially uses the resource file's stem as a temporary path.
+After the agent writes the card, the file is renamed according to its frontmatter `name`:
+
+```text
+resource/2026-06-20/market-report.md
+        ↓
+daily/2026-06-20/market-report-highlights.md
+```
+
+The resource card links to the original file through frontmatter:
+
+```yaml
+source_resource: "[[resource/2026-06-20/market-report.md]]"
+```
+
+When a resource changes, Auto Resource finds and updates the corresponding card through `source_resource`. When a resource is
+deleted, its daily note is also removed. The older `daily/YYYY-MM-DD/<resource_stem>.md` naming convention remains supported
+as a fallback.
+
+## Daily Index
+
+Resource cards enter the same daily memory layer as Auto Memory cards. The day's `YYYY-MM-DD.md` page acts as an index and
+organizes those resource cards:
+
+```text
+daily/
+  2026-06-20.md
+  2026-06-20/
+    market-report-highlights.md
+    meeting-notes-summary.md
+```
+
+To review which resources were processed on a day, start with `YYYY-MM-DD.md`. To inspect what was distilled from one
+resource, open its corresponding resource card.
+
+## Preserving the Original Resource
+
+The interpreted daily note is optimized for readability; the original resource is retained for trust and verification.
+
+Auto Resource does not move the original file. It remains under `resource/YYYY-MM-DD/`. Text resources can therefore enter
+the daily memory flow while their source files stay in their original location.
+
+## What Happens Next
+
+Auto Resource only creates resource interpretations in the daily layer. To distill long-term knowledge from resources into
+`digest/`, use [Auto Dream](./auto_dream.md). To search original resources, daily cards, and digest nodes, use
+[Memory Search](./memory_search.md).

docs/en/contributing.md

@@ -1,7 +1,221 @@
-# Contributing
+# Open Source and Contributing
 
-```{note}
-📖 The English version of this page is in progress.
+ReMe is open source and hosted on GitHub:
 
-In the meantime, please read the <a href="../zh/contributing.html">中文版本</a>.
+**https://github.com/agentscope-ai/ReMe**
+
+---
+
+## How to Contribute
+
+Thank you for your interest in ReMe. ReMe is a file-first, self-evolving memory system for agents. Contributions are welcome
+through issue reports, documentation improvements, additional tests, bug fixes, and new capabilities.
+
+If this is your first time running ReMe locally, start with [Quick Start](./quick_start.md). If your change affects runtime
+layers, Jobs, Steps, or components, read [ReMe Framework](./framework.md). If it affects workspace directories, frontmatter,
+wikilinks, or chunking, read [Memory as File](./memory_as_file.md).
+
+### 1. Before You Begin
+
+Before investing in an implementation:
+
+- Check [Open Issues](https://github.com/agentscope-ai/ReMe/issues) for an existing issue or discussion.
+- If a related issue is still open, comment that you would like to work on it to avoid duplicate effort.
+- If no issue exists, create one describing the context, expected behavior, possible implementation, and scope of impact.
+- For larger feature changes, align with maintainers on interfaces, configuration, compatibility, and test strategy before
+  submitting an implementation.
+
+### 2. Local Development Environment
+
+The core ReMe code is located in:
+
+- `reme/`: Python package source, including configuration, components, services, Jobs, Steps, schemas, and utilities.
+- `pyproject.toml`: project metadata, dependencies, optional dependencies, command entry points, and test configuration.
+- `tests/`: unit and integration tests.
+
+The project requires Python 3.11 or later. A virtual environment is recommended:
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev,full]"
+pre-commit install
 ```
+
+### 3. Development Model
+
+Before developing ReMe code, read [ReMe Framework](./framework.md). New or modified core capabilities should follow the
+layers and call chain described there:
+
+```text
+CLI / Client -> Service -> Application -> Job -> Step -> Component / Workspace
+```
+
+In practice:
+
+- Capabilities exposed to users or external systems should normally be orchestrated by a Job, then exposed by a Service as a
+  CLI-, HTTP-, or MCP-callable interface.
+- Reusable infrastructure belongs in `reme/components/`, with dependencies declared through `BaseComponent.bind()`.
+- Atomic business operations belong in `reme/steps/` and access the file store, agent wrapper, catalog, LLM, and other
+  components through `BaseStep.Ref`.
+- Request, response, and persistent data structures belong in `reme/schema/` or `reme/enumeration/`. Do not scatter
+  implicit structures through Step implementations.
+- Configuration-driven defaults belong in `reme/config/default.yaml`, and the default configuration must remain runnable
+  and testable.
+
+When adding a Step or Job, pay particular attention to these conventions:
+
+- Register implementations with `@R.register("<backend_name>")`. Registration names should be stable, clear, and match the
+  configured `backend`.
+- After adding a Step file, make sure its package `__init__.py` imports the module; otherwise, the registry will not load it.
+- A Step should perform one atomic business operation. Cross-step flows belong in Job configuration or a dedicated
+  orchestration Step.
+- A Job composes Steps and selects normal, streaming, background, or scheduled execution. `enable_serve` controls whether it
+  is externally exposed.
+- When a Step needs components, prefer `BaseStep.Ref`. Do not reconstruct global components inside a Step or bypass
+  `ApplicationContext`.
+- File, index, graph, frontmatter, and wikilink behavior must preserve consistent workspace-relative path semantics.
+- Add fast tests under `tests/unit/` for new capabilities. Put cross-component, LLM, embedding, or service behavior under
+  `tests/integration/` when appropriate.
+
+### 4. Code and Documentation Changes
+
+Choose the appropriate entry point for the type of change:
+
+| Change type | Primary location | Guidance |
+|---|---|---|
+| Configuration or startup behavior | `reme/config/`, `reme/application.py`, `reme/reme.py` | Keep the default configuration runnable and avoid breaking existing CLI, HTTP, and MCP entry points. |
+| Component capability | `reme/components/` | Reuse `BaseComponent`, the registry, and context objects. |
+| Job or Step | `reme/components/job/`, `reme/steps/` | Follow the Job -> Step model in [ReMe Framework](./framework.md), keep request and response schemas clear, and add corresponding tests. |
+| Data structure | `reme/schema/`, `reme/enumeration/` | Preserve serialization compatibility and existing frontmatter and wikilink semantics. |
+| Utility | `reme/utils/` | Keep function boundaries small and cover edge cases with unit tests. |
+| User documentation | `docs/en/`, `README.md` | Update documentation when user-visible behavior changes. |
+
+If a change involves an LLM, embeddings, an external service, file watching, or a background task, also describe its
+dependencies, failure behavior, and local validation method.
+
+### 5. Commit Message Format
+
+Use [Conventional Commits](https://www.conventionalcommits.org/) to keep history clear.
+
+Format:
+
+```text
+<type>(<scope>): <subject>
+```
+
+Common types:
+
+- `feat`: new feature
+- `fix`: bug fix
+- `docs`: documentation only
+- `style`: code-style change with no behavior change
+- `refactor`: refactoring that neither fixes a bug nor adds a feature
+- `perf`: performance improvement
+- `test`: add or update tests
+- `chore`: build, tooling, or maintenance work
+
+Examples:
+
+```bash
+feat(search): add link expansion option
+fix(file-graph): handle pending wikilinks after move
+docs(memory): update auto memory guide
+test(config): cover default yaml parsing
+chore(pre-commit): update lint hooks
+```
+
+### 6. Pull Request Titles
+
+PR titles should use the same format:
+
+```text
+<type>(<scope>): <description>
+```
+
+Requirements:
+
+- Use `feat`, `fix`, `docs`, `test`, `refactor`, `chore`, `perf`, `style`, `build`, or `revert` as the type.
+- Use lowercase letters, numbers, hyphens, or underscores for the scope.
+- Keep the description short and state the actual effect of the PR.
+
+Examples:
+
+```text
+feat(auto-memory): persist source conversation metadata
+fix(markdown): keep wikilink aliases during edit
+docs(en): add contribution guide
+```
+
+### 7. Pre-submit Checks
+
+Before committing or opening a PR, run at least:
+
+```bash
+pre-commit run --all-files
+pytest
+```
+
+For a localized code change, start with a narrower test set:
+
+```bash
+pytest tests/unit/test_search_step.py
+pytest tests/unit/test_reme_cli.py
+```
+
+If `pre-commit` modifies files automatically, commit those changes and rerun the checks until everything passes.
+
+The current pre-commit configuration includes YAML/TOML/JSON validation, private-key detection, trailing-whitespace checks,
+`black`, `flake8`, `pylint`, and `pyroma`. The main formatting rules are:
+
+- `black --line-length=120`
+- `flake8 --max-line-length=120`
+- `pylint --max-line-length=120`
+
+Some integration tests may require an LLM, embeddings, or external service configuration. If you cannot run them locally,
+state why they were skipped and what alternative validation you completed in the PR description.
+
+### 8. Testing Requirements
+
+Add tests according to the risk of the change:
+
+- For a bug fix, first add a regression test that reproduces the issue.
+- For a new Step, Job, or component, cover at least the main path and a failure path.
+- For changes to shared logic such as indexes, graphs, wikilinks, frontmatter, or file operations, add edge cases.
+- For changes to the CLI, services, or configuration parsing, cover the user-visible entry point.
+- Documentation-only changes usually do not require new tests, but running `pre-commit run --all-files` is still recommended.
+
+Place tests according to the existing structure:
+
+- `tests/unit/`: fast tests that require no real external service.
+- `tests/integration/`: integration tests spanning components or requiring external configuration.
+
+### 9. Documentation Contributions
+
+When a change affects how users install, configure, invoke, or understand ReMe, update the documentation as well.
+
+Documentation lives under:
+
+```text
+docs/
+```
+
+Documentation should:
+
+- Use clear titles that directly identify a capability or flow.
+- Provide commands that can be copied and run.
+- Use real repository paths such as `reme/config/default.yaml`, `reme/steps/`, and `tests/unit/`.
+- Describe default behavior according to the current code, `pyproject.toml`, and default configuration.
+
+---
+
+## Getting Help
+
+- Bugs and feature requests: [GitHub Issues](https://github.com/agentscope-ai/ReMe/issues)
+- Project home: [GitHub Repository](https://github.com/agentscope-ai/ReMe)
+- Documentation site: [https://reme.agentscope.io/](https://reme.agentscope.io/)
+
+---
+
+Thank you for contributing to ReMe. Your improvements help make long-term memory for agents more readable, controllable, and
+maintainable.

docs/en/framework.md

@@ -1,7 +1,786 @@
-# Framework
+# ReMe Framework
 
-```{note}
-📖 The English version of this page is in progress.
+## 1. Overview
 
-In the meantime, please read the <a href="../zh/framework.html">中文版本</a>.
+The ReMe runtime can be understood as follows: **a configuration-driven Application assembles components and Jobs; the
+Service exposes service-enabled Jobs to the CLI, HTTP, or MCP; and each Job executes its Steps in sequence**.
+
+<p align="center">
+  <img src="../figure/framework-structure.svg" alt="ReMe framework structure: CLI, Service, Application, Job, Step, and Component" width="92%">
+</p>
+
+To run and use ReMe first, see [Quick Start](./quick_start.md). For workspace file semantics, see
+[Memory as File](./memory_as_file.md). User-facing guides for retrieval, automatic memory, and proactive reading are
+[Memory Search](./memory_search.md), [Auto Memory](./auto_memory.md), [Auto Resource](./auto_resource.md),
+[Auto Dream](./auto_dream.md), and [Proactive](./proactive.md).
+
+### Capability Boundary
+
+ReMe v4 focuses on long-term memory: it distills conversations and resources into `daily/`, organizes them into `digest/`,
+and exposes write, retrieval, and proactive-read capabilities through the CLI, HTTP, and MCP.
+
+Single-session context-window management is outside the scope of ReMe v4. This includes compressing the current conversation,
+injecting summaries, trimming tool output, or providing an independent `/compact` interface. Those capabilities belong in
+the host agent framework. ReMe accepts conversations, resources, and file changes that have already occurred and persists the
+information with long-term value.
+
+```mermaid
+flowchart LR
+    CLI["reme CLI<br/>reme/reme.py"] --> Client["Client<br/>http / mcp"]
+    Client --> Service["Service<br/>HTTP / MCP"]
+    Service --> App["Application<br/>reme/application.py"]
+    App --> Jobs["Jobs<br/>base / stream / background / cron"]
+    Jobs --> Steps["Steps<br/>reme/steps/**"]
+    Steps --> Ctx["RuntimeContext<br/>data + Response + stream queue"]
+    Steps --> Components["Components<br/>store / graph / index / llm / agent / catalog"]
+    Components --> Workspace["Workspace<br/>daily / digest / resource / metadata"]
 ```
+
+Core layers:
+
+| Layer | Main location | Responsibility |
+|---|---|---|
+| CLI | `reme/reme.py` | Parse commands; `start` launches the service; other actions call the service through a client. |
+| Service | `reme/components/service/` | Register Jobs as HTTP endpoints or MCP tools. |
+| Application | `reme/application.py` | Assemble configured objects, start them in dependency order, close them, and invoke Jobs. |
+| Job | `reme/components/job/` | Orchestrate Steps and select normal, streaming, background, or scheduled execution. |
+| Step | `reme/steps/` | Atomic business operations such as file I/O, retrieval, indexing, and self-evolution. |
+| Component | `reme/components/` | Reusable infrastructure such as file_store, file_graph, keyword_index, and agent_wrapper. |
+| Schema | `reme/schema/` | Data structures such as `Request`, `Response`, `FileChunk`, `FileNode`, and configuration models. |
+| Config | `reme/config/` | Default YAML configuration and command-line override parsing. |
+
+## 2. Directory Structure
+
+```text
+reme/
+  reme.py                    # CLI entry point
+  application.py             # Application assembly and lifecycle
+  config/
+    default.yaml             # default service / jobs / components
+    config_parser.py         # config=, dot notation, and env placeholder parsing
+  components/
+    component_registry.py    # global registry R
+    base_component.py        # ComponentMixin / BaseComponent / bind dependency declarations
+    runtime_context.py       # context for one Job execution
+    job/                     # BaseJob / StreamJob / BackgroundJob / CronJob
+    service/                 # HTTP / MCP services
+    client/                  # HTTP / MCP clients
+    file_store/              # file-index coordination layer
+    file_graph/              # wikilink graph
+    keyword_index/           # BM25 and other keyword indexes
+    file_chunker/            # Markdown / default text chunking
+    file_catalog/            # change checkpoints
+    as_llm/, as_embedding/   # model wrappers
+    agent_wrapper/           # AgentScope / Claude Code wrappers
+  steps/
+    base_step.py             # BaseStep, Ref, dispatch_steps
+    common/                  # version, help, health_check, demo
+    file_io/                 # read/write/edit/delete/move/frontmatter/daily
+    index/                   # watch/init/update/search/traverse
+    evolve/                  # auto_memory, auto_resource, auto_dream, proactive
+    transfer/                # upload/download/ingest
+    channel/                 # MCP channel tools
+```
+
+The default workspace directories are defined by `ApplicationConfig`:
+
+```text
+<workspace_dir>/
+  metadata/     # persistent file_store, file_graph, keyword_index, file_catalog, and related state
+  session/      # agent sessions and original conversations
+  resource/     # external resources
+  daily/        # lightly processed memory
+  digest/       # long-term digest memory
+```
+
+`Application.__init__()` first ensures that these directories exist, then initializes the service, components, and Jobs.
+
+## 3. Startup and Call Chain
+
+### 3.1 CLI
+
+The entry point is `reme/reme.py::main()`:
+
+```mermaid
+flowchart LR
+    A["main()"] --> B["parse_args(*sys.argv[1:])"]
+    B --> C{action}
+    C -->|" start "| D["load_env()"]
+    D --> E["resolve_app_config(**kwargs)"]
+    E --> F["precheck_start(service)"]
+    F --> G["ReMe(**config).run_app()"]
+    C -->|" find_reme "| H["cli_find_reme()"]
+    C -->|" other actions "| I["call_server(action, **kwargs)"]
+    I --> J["R.get(ComponentEnum.CLIENT, backend)"]
+    J --> K["client(action=action, **kwargs)"]
+```
+
+Common commands:
+
+```bash
+reme start
+reme start service.port=8181
+reme version
+reme search query="memory" limit=5
+reme search query="memory" backend=mcp
+```
+
+Configuration parsing supports:
+
+| Capability | Source | Description |
+|---|---|---|
+| Default configuration | `resolve_app_config()` | Load `reme/config/default.yaml` when `config` is not specified. |
+| Explicit configuration | `config=<name-or-path>` | Accept a built-in configuration name or a YAML/JSON file path. |
+| Dot notation | `parse_dot_notation()` | For example, `service.port=8181`. |
+| Environment variables | `_expand_env_vars()` | Support `${VAR}` and `${VAR:-default}`. |
+| Value conversion | `_convert_value()` | Convert bool, int, float, JSON list/dict, and null values automatically. |
+
+### 3.2 Service
+
+`BaseService.run_app()` executes in this order:
+
+```mermaid
+flowchart LR
+    A["Service.build_service(app)"] --> B["read app.context.jobs"]
+    B --> C{"job.enable_serve == true?"}
+    C -->|yes| D["Service.add_job(job)"]
+    C -->|no| E["skip registration"]
+    D --> F["Service.start_service(app)"]
+    E --> F
+    F --> G["app.start() during lifespan"]
+    G --> H["Application starts jobs"]
+```
+
+HTTP service behavior:
+
+| Job type | HTTP exposure |
+|---|---|
+| Non-`StreamJob` with `enable_serve: true` | `POST /<job.name>` returning `Response` JSON. |
+| `StreamJob` | `POST /<job.name>` returning `text/event-stream`. |
+| `enable_serve: false` | No endpoint is registered. |
+
+MCP service behavior:
+
+| Job type | MCP exposure |
+|---|---|
+| Non-`StreamJob` with `enable_serve: true` | Registered as an MCP tool. |
+| `StreamJob` | Currently skipped and not registered. |
+| `BackgroundJob` | Forces `enable_serve=False` at construction and is never exposed. |
+
+## 4. Registry and Dependency Injection
+
+### 4.1 Global Registry R
+
+ReMe uses the process-wide singleton `R = ComponentRegistry()`. Every component, Job, and Step is registered with
+`@R.register("name")`.
+
+```python
+from ...components import R
+
+
+@R.register("version_step")
+class VersionStep(BaseStep):
+    ...
+```
+
+The registry key is:
+
+```text
+(component_type, register_name) -> class
+```
+
+`component_type` comes from a class attribute:
+
+| Type | Class attribute |
+|---|---|
+| Step | `BaseStep.component_type = ComponentEnum.STEP` |
+| Job | `BaseJob.component_type = ComponentEnum.JOB` |
+| Service | `BaseService.component_type = ComponentEnum.SERVICE` |
+| FileStore | `BaseFileStore.component_type = ComponentEnum.FILE_STORE` |
+
+The same backend name can therefore exist under different component types. For example, `http` can be both a service backend
+and a client backend.
+
+### 4.2 Registration Through Module Imports
+
+Registration happens when a module is imported. `reme/components/__init__.py` imports component packages, while
+`reme/steps/__init__.py` imports `channel/common/evolve/file_io/index/transfer`. Each package's `__init__.py` then imports
+its concrete modules, causing `@R.register(...)` to execute.
+
+After adding a Step file, make sure the package's `__init__.py` imports it. Otherwise, the backend will not appear in the
+registry.
+
+### 4.3 Component.bind
+
+Dependencies between components are declared with `BaseComponent.bind()`. At startup,
+`Application._topological_order()` reads every component's `dependencies` and starts them in topological order.
+
+```mermaid
+flowchart LR
+    A["Component.__init__<br/>self.keyword_index = self.bind(...)"] --> B["Dependency placeholder"]
+    B --> C["Application._topological_order()"]
+    C --> D["component.start()"]
+    D --> E["_resolve_bindings()"]
+    E --> F["self.keyword_index = app_context.components[type][name]"]
+    F --> G["component._start()"]
+```
+
+Rules for `BaseComponent.bind(name, BaseClass, optional=True)`:
+
+| Scenario | Behavior |
+|---|---|
+| `name` is empty | Return `None` and skip the dependency. |
+| `app_context` exists | Look up `app_context.components[ctype][name]`. |
+| Dependency missing and `optional=True` | Resolve to `None`. |
+| Dependency missing and `optional=False` | Fail at startup. |
+| Standalone mode | A private component can be created with `default_factory`. |
+
+### 4.4 Step.Ref
+
+Steps do not participate in component topological startup. They are created temporarily for each Job invocation. Steps access
+components primarily through `BaseStep.Ref`:
+
+```python
+file_store: BaseFileStore = Ref(BaseFileStore, ComponentEnum.FILE_STORE)
+agent_wrapper: BaseAgentWrapper = Ref(BaseAgentWrapper, ComponentEnum.AGENT_WRAPPER, optional=True)
+```
+
+Resolution priority:
+
+```mermaid
+flowchart LR
+    A["access self.file_store"] --> B{"same-named object in kwargs?"}
+    B -->|yes| C["use kwargs object"]
+    B -->|no| D{"same-named object in context.data?"}
+    D -->|yes| E["use context object"]
+    D -->|no| F["read name from kwargs['file_store']; default is default"]
+    F --> G["app_context.components[FILE_STORE][name]"]
+```
+
+A Step configuration can therefore specify:
+
+```yaml
+steps:
+  - backend: update_catalog_step
+    file_catalog: resource
+```
+
+Here, `file_catalog: resource` means to resolve the `file_catalog` component named `resource`.
+
+## 5. Application Lifecycle
+
+The Application converts configuration into runtime objects and starts and closes them in order.
+
+```mermaid
+flowchart LR
+    A["Application(**kwargs)"] --> B["ApplicationContext(**kwargs)<br/>parse ApplicationConfig"]
+    B --> C["_setup_workspace_directories()"]
+    C --> D["_init_service()"]
+    D --> E["_init_components()"]
+    E --> F["_init_jobs()"]
+    F --> G["run_app()"]
+    G --> H["service.run_app(app)"]
+```
+
+Startup order in `Application._start()`:
+
+```mermaid
+flowchart LR
+    A["create optional thread_pool"] --> B["topologically sort components"]
+    B --> C["start components"]
+    C --> D["start BaseJob"]
+    D --> E["start StreamJob"]
+    E --> F["start BackgroundJob"]
+    F --> G["start CronJob"]
+```
+
+During shutdown, objects in `_started_components` are closed in reverse order so dependents close before their dependencies.
+
+## 6. Job Model
+
+A Job is the orchestration unit for an externally callable capability or background task. Jobs are configured under `jobs:`
+in `reme/config/default.yaml`.
+
+### 6.1 BaseJob
+
+`BaseJob` is the most common request-oriented Job:
+
+```mermaid
+flowchart LR
+    Caller["Caller"] --> Job["BaseJob<br/>job(**kwargs)"]
+    Job --> Ctx["RuntimeContext<br/>merged_kwargs"]
+    Ctx --> S1["Step 1<br/>await step(context)"]
+    S1 --> D1["read/write context.data / response"]
+    D1 --> S2["Step 2<br/>await step(context)"]
+    S2 --> D2["read/write context.data / response"]
+    D2 --> Resp["context.response"]
+    Resp --> Caller
+```
+
+Important source behavior:
+
+| Source | Behavior |
+|---|---|
+| `_start()` | Parse each Step config from YAML into `(step_cls, params)`. |
+| `_build_steps()` | Create new Step instances for every call, avoiding state shared across requests. |
+| `__call__()` | Create a `RuntimeContext` and execute Steps sequentially. |
+| Exception handling | Catch the exception, set `response.success=False`, and set `answer=str(e)`. |
+
+### 6.2 StreamJob
+
+`StreamJob` extends `BaseJob` but returns streaming chunks:
+
+| Behavior | Description |
+|---|---|
+| Context | Includes `stream_queue`. |
+| Step output | Call `context.add_stream_string(text, ChunkEnum.CONTENT)`. |
+| Exception | Write `ChunkEnum.ERROR`. |
+| Completion | Always send a `DONE` chunk. |
+
+### 6.3 BackgroundJob
+
+`BackgroundJob` runs long-lived loops such as file watchers. Its constructor forces `enable_serve=False`.
+
+```mermaid
+flowchart LR
+    A["Application starts BackgroundJob"] --> B["_start() creates stop_event and task"]
+    B --> C["_run_with_supervisor()"]
+    C --> D["await self()"]
+    D --> E{"exception?"}
+    E -->|no, returned normally| F["finish"]
+    E -->|yes and supervisor = True| G["exponential backoff + jitter"]
+    G --> C
+    E -->|yes and supervisor = False| H["raise exception"]
+    I["close()"] --> J["stop_event.set()"]
+    J --> K["wait close_timeout; cancel on timeout"]
+```
+
+The default `BackgroundJob.__call__()` also executes configured Steps in sequence, but it does not swallow exceptions, which
+allows the supervisor to restart the task.
+
+### 6.4 CronJob
+
+`CronJob` extends `BackgroundJob` with a `cron` expression:
+
+```yaml
+jobs:
+  nightly_dream:
+    backend: cron
+    cron: "0 3 * * *"
+    steps:
+      - backend: dream_extract_step
+      - backend: dream_integrate_step
+      - backend: dream_topics_step
+      - backend: dream_finish_step
+```
+
+The current implementation uses `croniter` to calculate the next trigger time. The timezone comes from
+`app_config.timezone`.
+
+### 6.5 Default Job Types
+
+```mermaid
+flowchart LR
+    Jobs["default.yaml jobs"] --> BG["background<br/>index_update_loop<br/>resource_watch_loop<br/>digest_watch_loop"]
+    Jobs --> Base["base<br/>version / help / health_check<br/>search / node_search / traverse / reindex<br/>read / write / edit / delete / move / list / stat<br/>daily_list / daily_reindex / daily_write<br/>auto_memory / auto_resource / auto_dream / proactive"]
+```
+
+## 7. Step Model
+
+A Step is a concrete business action. Every Step extends `BaseStep` and implements `execute()`.
+
+```mermaid
+flowchart LR
+    A["Job._build_steps()"] --> B["Step.__init__()"]
+    B --> C["load prompt<br/>class-named YAML + prompt_dict override"]
+    C --> D["Step.__call__(context, **kwargs)"]
+    D --> E["clear Ref cache"]
+    E --> F["RuntimeContext.from_context()"]
+    F --> G["input_mapping"]
+    G --> H["execute()"]
+    H --> I["output_mapping"]
+    I --> J["return result"]
+```
+
+### 7.1 RuntimeContext
+
+`RuntimeContext` is shared by all Steps within one Job invocation:
+
+| Field | Description |
+|---|---|
+| `response` | Final `Response(answer, success, metadata)`. |
+| `data` | Free-form dictionary containing input parameters and intermediate results. |
+| `stream_queue` | Output queue for streaming Jobs. |
+| `stop_event` | Stop signal for background Jobs. |
+
+Common Step code:
+
+```python
+assert self.context is not None
+query = self.context.get("query", "")
+self.context["processed_query"] = query.strip().lower()
+self.context.response.answer = "..."
+self.context.response.metadata["key"] = "value"
+return self.context.response
+```
+
+### 7.2 input_mapping / output_mapping
+
+`BaseStep.__call__()` invokes `RuntimeContext.apply_mapping()` before and after execution:
+
+```yaml
+steps:
+  - backend: some_step
+    input_mapping:
+      user_query: query
+    output_mapping:
+      result: final_result
+```
+
+The semantics are to copy `context.data[source]` to `context.data[target]`.
+
+### 7.3 dispatch_steps
+
+Some Steps produce batches of events and dispatch them to other Steps. `BaseStep.dispatch_steps()` resolves and executes
+child Steps according to configuration.
+
+Example from the default configuration:
+
+```yaml
+index_update_loop:
+  backend: background
+  watch_dirs: [ daily_dir, digest_dir ]
+  watch_suffixes: [ md ]
+  steps:
+    - backend: init_changes_step
+      monitor_type: file_store
+      monitor_name: default
+      dispatch_steps: [ update_index_step ]
+    - backend: watch_changes_step
+      dispatch_steps: [ update_index_step ]
+```
+
+Flow:
+
+```mermaid
+flowchart LR
+    Init["init_changes_step"] --> Batch["changes batch"]
+    Watch["watch_changes_step"] --> Batch
+    Batch --> Dispatch["dispatch_steps(...)"]
+    Dispatch --> Update["update_index_step"]
+    Update --> Store["file_store"]
+```
+
+## 8. Components in the Default Configuration
+
+Current default components in `reme/config/default.yaml`:
+
+| ComponentEnum | Name | Backend | Description |
+|---|---|---|---|
+| `service` | singleton | `http` | Default HTTP service. |
+| `tokenizer` | `default` | `regex` | BM25 tokenizer. |
+| `as_embedding` | `default` | `${EMBEDDING_BACKEND:-openai}` | Embedding model wrapper. |
+| `embedding_store` | `default` | `local` | Embedding store depending on `as_embedding: default`. |
+| `as_llm` | `default` | `${LLM_BACKEND:-openai}` | LLM model wrapper. |
+| `agent_wrapper` | `default` | `agentscope` | AgentScope wrapper. |
+| `agent_wrapper` | `claude_code` | `claude_code` | Claude Code wrapper. |
+| `file_graph` | `default` | `local` | Wikilink graph. |
+| `file_catalog` | `default/resource/digest/dream` | `local` | File-change checkpoints. |
+| `file_chunker` | `markdown` | `markdown` | Markdown AST chunking. |
+| `file_chunker` | `default` | `default` | Default text chunking, currently supporting `jsonl`. |
+| `keyword_index` | `default` | `bm25` | BM25 keyword index. |
+| `file_store` | `default` | `local` | Combines file_graph and keyword_index; defaults to `embedding_store: ""`. |
+
+Note that the `search` Step configuration contains `vector_weight`, but `file_store.default.embedding_store` is empty by
+default. Vector retrieval is available only when the runtime configuration enables an embedding store.
+
+## 9. Adding a Step
+
+### 9.1 Minimal Step
+
+Suppose you want to add a Step that converts input text to uppercase.
+
+Create a file such as `reme/steps/common/uppercase.py`:
+
+```python
+from ..base_step import BaseStep
+from ...components import R
+
+
+@R.register("uppercase_step")
+class UppercaseStep(BaseStep):
+    async def execute(self):
+        assert self.context is not None
+        text = self.context.get("text", "")
+        result = str(text).upper()
+
+        self.context["uppercase_text"] = result
+        self.context.response.answer = result
+        self.context.response.metadata["length"] = len(result)
+        return self.context.response
+```
+
+### 9.2 Registering the Step
+
+Make sure `reme/steps/common/__init__.py` imports the new module. Add:
+
+```python
+from . import uppercase
+```
+
+The reason is that `@R.register("uppercase_step")` only executes after the module is imported.
+
+### 9.3 Accessing Components
+
+If a Step needs an existing component, prefer the Refs provided by `BaseStep`:
+
+```python
+class MySearchStep(BaseStep):
+    async def execute(self):
+        assert self.context is not None
+        results = await self.file_store.keyword_search(
+            self.context.get("query", ""),
+            limit=5,
+        )
+        ...
+```
+
+Common attributes available directly:
+
+| Attribute | Component resolved by default |
+|---|---|
+| `self.as_llm` | `.model` from `as_llm: default`. |
+| `self.agent_wrapper` | `agent_wrapper: default`; optional. |
+| `self.file_catalog` | `file_catalog: default`; optional. |
+| `self.file_store` | `file_store: default`. |
+
+To select a non-default component from Job configuration:
+
+```yaml
+steps:
+  - backend: my_step
+    file_catalog: dream
+```
+
+### 9.4 Step Design Guidance
+
+| Guidance | Reason |
+|---|---|
+| Read input from `context` and write intermediate results to `context`. | A multi-Step Job passes data through the same context. |
+| Write the final result to `context.response`. | Services and clients consume the standard `Response`. |
+| Do not store request-scoped state on a Step instance. | A Step is rebuilt for every Job call, and stateless Steps are easier to test. |
+| A background loop that supports interruption should check `context.stop_event`. | `BackgroundJob.close()` relies on the stop event for graceful shutdown. |
+| Call `add_stream_string()` only from a StreamJob. | A normal Job has no stream queue. |
+
+### 9.5 Unit Test Example
+
+A Step can be instantiated directly and passed a `RuntimeContext`:
+
+```python
+import pytest
+
+from reme.components.runtime_context import RuntimeContext
+from reme.steps.common.uppercase import UppercaseStep
+
+
+@pytest.mark.asyncio
+async def test_uppercase_step():
+    ctx = RuntimeContext(text="hello")
+    resp = await UppercaseStep()(ctx)
+    assert resp.answer == "HELLO"
+    assert ctx["uppercase_text"] == "HELLO"
+```
+
+## 10. Adding a Job
+
+A Job usually requires no new Python class; configure existing Steps instead. Add a new Job backend only when a new execution
+model is required.
+
+### 10.1 Adding a Normal Request Job
+
+Add the Job under `jobs:` in a YAML configuration:
+
+```yaml
+jobs:
+  uppercase:
+    backend: base
+    description: "Convert text to uppercase."
+    parameters:
+      type: object
+      properties:
+        text:
+          type: string
+          description: "input text"
+      required:
+        - text
+    steps:
+      - backend: uppercase_step
+```
+
+Start and call it:
+
+```bash
+reme start
+reme uppercase text="hello"
+```
+
+Call chain:
+
+```mermaid
+flowchart LR
+    CLI["CLI<br/>reme uppercase text=hello"] --> HTTP["HTTP Client"]
+    HTTP --> Req["POST /uppercase"]
+    Req --> S["HttpService"]
+    S --> J["uppercase BaseJob<br/>job(text='hello')"]
+    J --> Step["uppercase_step<br/>await step(context)"]
+    Step --> Resp["context.response.answer = HELLO"]
+    Resp --> JSON["Response JSON"]
+    JSON --> CLIOut["CLI prints answer"]
+```
+
+### 10.2 Adding a Multi-Step Job
+
+A Job can chain multiple Steps:
+
+```yaml
+jobs:
+  demo_echo:
+    backend: base
+    description: "Normalize query, then echo it."
+    parameters:
+      type: object
+      properties:
+        query:
+          type: string
+          default: ""
+        min_score:
+          type: number
+          default: 0.5
+    steps:
+      - backend: demo_echo_step1
+      - backend: demo_echo_step2
+```
+
+The first Step writes:
+
+```text
+context["processed_query"]
+context["adjusted_min_score"]
+```
+
+The second Step reads those fields and writes the final `response`.
+
+### 10.3 Adding a Stream Job
+
+Use `backend: stream` in configuration:
+
+```yaml
+jobs:
+  stream_uppercase:
+    backend: stream
+    description: "Stream uppercase text."
+    parameters:
+      type: object
+      properties:
+        text:
+          type: string
+      required:
+        - text
+    steps:
+      - backend: uppercase_prepare_step
+      - backend: uppercase_stream_step
+```
+
+Example streaming Step:
+
+```python
+from ..base_step import BaseStep
+from ...components import R
+from ...enumeration import ChunkEnum
+
+
+@R.register("uppercase_stream_step")
+class UppercaseStreamStep(BaseStep):
+    async def execute(self):
+        assert self.context is not None
+        for ch in self.context.get("uppercase_text", ""):
+            await self.context.add_stream_string(ch, ChunkEnum.CONTENT)
+        return self.context.response
+```
+
+### 10.4 Adding a Background Job
+
+Use `backend: background` in configuration:
+
+```yaml
+jobs:
+  my_watch_loop:
+    backend: background
+    watch_dirs: [ daily_dir ]
+    watch_suffixes: [ md ]
+    steps:
+      - backend: init_changes_step
+        monitor_type: file_store
+        monitor_name: default
+        dispatch_steps: [ update_index_step ]
+      - backend: watch_changes_step
+        dispatch_steps: [ update_index_step ]
+```
+
+Characteristics of a background Job:
+
+| Characteristic | Description |
+|---|---|
+| Not externally exposed | `BackgroundJob.__init__()` forces `enable_serve=False`. |
+| Has a supervisor | Restarts with exponential backoff after an exception by default. |
+| Has a stop event | Notifies the loop to exit during close. |
+| Suitable for watching/consuming | File watching, queue consumption, and periodic long-running loops. |
+
+### 10.5 Adding a Cron Job
+
+Use `backend: cron` in configuration:
+
+```yaml
+jobs:
+  daily_auto_dream:
+    backend: cron
+    cron: "30 3 * * *"
+    steps:
+      - backend: dream_extract_step
+        file_catalog: dream
+      - backend: dream_integrate_step
+      - backend: dream_topics_step
+      - backend: dream_finish_step
+        file_catalog: dream
+```
+
+An invalid `cron` expression fails at startup.
+
+### 10.6 When a New Job Backend Is Needed
+
+Most use cases require only a new Step plus a YAML Job. Consider adding `reme/components/job/*.py` only in these cases:
+
+| Requirement | New Job class? |
+|---|---|
+| Add a business command | No; use `backend: base`. |
+| Chain existing steps | No; use `steps:`. |
+| Need SSE/streaming output | No; use `backend: stream`. |
+| Need a background loop | No; use `backend: background`. |
+| Need cron scheduling | No; use `backend: cron`. |
+| Need entirely new scheduling, concurrency, or transaction semantics | Yes; add a Job backend. |
+
+Minimal shape of a new Job backend:
+
+```python
+from .base_job import BaseJob
+from ..component_registry import R
+
+
+@R.register("my_job_backend")
+class MyJob(BaseJob):
+    async def __call__(self, **kwargs):
+        # custom scheduling logic
+        return await super().__call__(**kwargs)
+```
+
+Also ensure the module is imported by `reme/components/job/__init__.py`.

docs/en/memory_as_file.md

@@ -1,7 +1,373 @@
 # Memory as File
 
-```{note}
-📖 The English version of this page is in progress.
+ReMe's core idea is **Memory as File, File as Memory**.
 
-In the meantime, please read the <a href="../zh/memory_as_file.html">中文版本</a>.
+<p align="center">
+  <img src="../figure/memory-as-file.svg" alt="ReMe Memory as File model" width="92%">
+</p>
+
+**Memory as File**: long-term memory is not hidden in a black-box database. It lives in Markdown files, resource files, and
+index snapshots under the workspace. Users and agents can directly read, write, move, and delete those files.
+
+**File as Memory**: each file is more than ordinary text. It is an indexable, linkable, and evolvable memory node. ReMe parses
+frontmatter, body chunks, and wikilink edges from files and organizes them into retrieval indexes and a graph.
+
+In other words, files are both a human-readable interface and an operational interface for agents. Directory structure
+carries the memory layers, while Markdown syntax expresses content, metadata, and relationships.
+
+## Design Goals
+
+ReMe represents memory as files not merely for convenient storage, but to give long-term memory several essential properties:
+
+| Goal | Meaning |
+|---|---|
+| Readable | Users can open the workspace directly and read daily notes, digest nodes, and source material like ordinary notes. |
+| Editable | Users and agents can correct, extend, move, or delete memory with file operations, without a specialized database client. |
+| Traceable | Long-term conclusions in digest can point back to daily, resource, or session sources through `derived_from:: [[...]]`. |
+| Portable | The workspace is an ordinary directory. Markdown, JSONL, YAML, and resource files can be backed up, synchronized, versioned, or moved to other tools. |
+| Indexable | Although the files are plain text, ReMe parses frontmatter, chunks, and wikilinks to build a retrieval index and file graph. |
+| Collaborative | Humans judge and correct; agents organize, link, and retrieve. Both operate on the same files. |
+
+ReMe memory is therefore neither a hidden database record nor a prompt fragment visible only to an LLM. It is first a file
+owned by the user and only then indexed by the system for retrieval.
+
+## Memory Layers
+
+A ReMe workspace divides memory into four layers:
+
+```text
+raw input      -> session/ + resource/
+working memory -> daily/
+long memory    -> digest/
+system state   -> metadata/
 ```
+
+Each layer solves a different problem.
+
+`session/` and `resource/` preserve raw input. Their purpose is to retain the original situation: conversations, agent
+sessions, uploaded material, web pages, and reports remain intact as evidence for later verification.
+
+`daily/` is the lightly processed layer. It organizes the day's conversations and resources into more readable daily notes:
+what happened, which conclusions were reached, which follow-up tasks remain, and where the source material lives. Daily does
+not aim for final abstraction; it is closer to a workbench for the day.
+
+`digest/` is the deeply processed layer. It stores memory nodes that can be reused over time, such as user preferences,
+project background, procedural experience, conceptual knowledge, and decision precedents. Digest should not merely copy
+daily. It should merge recurring facts, methods, and relationships into more stable descriptions.
+
+`metadata/` is the system index layer. It stores runtime state such as the file catalog, chunk index, and graph snapshots.
+Users normally do not edit this content manually. The actual editing surface is `daily/`, `digest/`, and, when necessary,
+`resource/`.
+
+These layers let ReMe preserve both the original situation and its abstraction: daily reconstructs what happened, while
+digest answers what remains reusable later.
+
+## Directory Layout
+
+ReMe uses directories to express memory organization and layers. Source material first enters `resource/` or `session/`,
+then flows into `daily/`, and is finally integrated into `digest/` by `auto_dream`.
+
+The corresponding automatic flows are [Auto Memory](./auto_memory.md), [Auto Resource](./auto_resource.md), and
+[Auto Dream](./auto_dream.md). Use [Memory Search](./memory_search.md) to retrieve these files.
+
+```text
+<workspace_dir>/
+├── metadata/                    # system index layer; persistent indexes, graph, catalogs; not a manual editing surface
+├── session/                     # raw input layer; original conversations and agent sessions
+│   ├── dialog/
+│   │   └── <session_id>.jsonl   # conversation messages saved by auto_memory
+│   ├── agentscope/
+│   │   └── <session_id>.jsonl
+│   └── claude_code/
+│       └── <session_id>.jsonl
+├── resource/                    # raw input layer; original external material
+│   └── YYYY-MM-DD/
+│       └── <resource>.<ext>
+├── daily/                       # lightly processed layer; facts, conversation summaries, and resource interpretations by date
+│   ├── YYYY-MM-DD.md            # index page for the day
+│   └── YYYY-MM-DD/
+│       ├── <session_id>.md      # daily note distilled from a conversation
+│       ├── <resource_stem>.md   # daily note distilled from a resource
+│       └── interests.yaml       # proactive interest topics generated by auto_dream
+└── digest/                      # deeply processed layer; reusable personal facts, procedures, and knowledge nodes
+    ├── personal/
+    │   └── <memory>.md          # user profile, preferences, and durable personal facts
+    ├── procedure/
+    │   └── <memory>.md          # procedures, methods, and operational experience
+    └── wiki/
+        └── <memory>.md          # general knowledge, concepts, and decision precedents
+```
+
+Typical flows:
+
+```text
+conversation
+  -> session/dialog/<session_id>.jsonl
+  -> daily/YYYY-MM-DD/<session_id>.md
+  -> digest/personal | digest/procedure | digest/wiki
+
+external resource
+  -> resource/YYYY-MM-DD/<resource>.<ext>
+  -> daily/YYYY-MM-DD/<resource_stem>.md
+  -> digest/wiki | digest/procedure
+```
+
+The first two steps focus on recording and organizing; the final step focuses on long-term distillation. `auto_memory` and
+`auto_resource` generate daily notes from raw input, and `auto_dream` extracts and integrates digest nodes from daily.
+
+## Markdown Format
+
+ReMe favors Markdown for memory because it works well for human reading, agent editing, and programmatic parsing.
+
+A typical memory file:
+
+```markdown
+---
+name: Solar Supply Chain Research
+description: An end-to-end view from polysilicon to modules
+tags: [new energy, solar]
+---
+
+# Conclusions
+
+The solar supply chain consists of [[digest/wiki/polysilicon.md]], wafers, cells, and modules.
+
+upstream:: [[digest/wiki/polysilicon.md]]
+[company:: [[digest/wiki/longi.md|LONGi]]]
+```
+
+### Frontmatter
+
+Frontmatter is a YAML block at the beginning of a file, enclosed by `---`:
+
+```markdown
+---
+name: Document name
+description: Document description
+source_conversation: [[session/dialog/abc.jsonl]]
+---
+```
+
+The current code recognizes `name` and `description` explicitly. Other fields are preserved as additional metadata. The write
+interface merges `name`, `description`, and `metadata` into frontmatter.
+
+Treat frontmatter as a node-level summary and the body as evidence, explanation, and relationships. For example:
+
+```markdown
+---
+name: "User preference: documentation style"
+description: The user prefers direct, engineering-oriented technical explanations with context but without unnecessary length.
+kind: preference
+confidence: observed
+---
+
+The user repeatedly asks documentation to explain motivation, boundaries, and examples while avoiding marketing language.
+
+derived_from:: [[daily/2026-06-20/session-a.md]]
+related:: [[digest/procedure/technical-documentation.md]]
+```
+
+This has three benefits:
+
+1. `name` and `description` serve as lightweight summaries in lists, recall results, and agent decisions.
+2. The body can carry fuller facts, conditions, counterexamples, and sources.
+3. Typed wikilinks such as `derived_from::` and `related::` can be parsed by the graph and maintained when files move.
+
+Frontmatter is best for stable, short, structured fields; the body is best for explanations meant for people. Do not put long
+body text into YAML fields.
+
+### Wikilink
+
+Wikilinks express relationships between files with `[[...]]`:
+
+```text
+[[digest/wiki/solar.md]]
+[[digest/wiki/solar.md#supply-chain]]
+[[digest/wiki/solar.md|solar]]
+![[resource/2026-06-01/report.md]]
+```
+
+ReMe wikilinks use **literal path semantics**:
+
+```text
+[[X]]  -> target_path = "X"
+```
+
+ReMe does not append `.md` automatically, search by filename, or automatically resolve folder notes. Use complete
+workspace-relative paths with their extensions.
+
+Wikilinks support these behaviors:
+
+```text
+body link          -> create a FileLink
+predicate:: link   -> create a FileLink with a relationship name
+move a file        -> rewrite [[old path]] in inbound edges by default
+delete a file      -> return remaining inbound edges so references can be cleaned up
+search match       -> expand inbound and outbound links to provide context
+```
+
+Supported relationship forms:
+
+```markdown
+industry:: [[digest/wiki/new-energy.md]]
+[competitor:: [[digest/wiki/byd.md]]]
+```
+
+Parsed result:
+
+```text
+FileLink
+  source_path = current file
+  target_path = digest/wiki/new-energy.md
+  predicate   = industry
+```
+
+### Sources and Relationships
+
+The two most important link types in ReMe are source links and conceptual relationship links.
+
+A source link explains where a long-term memory came from:
+
+```markdown
+derived_from:: [[daily/2026-06-20/session-a.md]]
+derived_from:: [[resource/2026-06-20/report.pdf]]
+```
+
+A conceptual relationship link explains which other long-term memories relate to the node:
+
+```markdown
+related:: [[digest/wiki/solar-supply-chain.md]]
+depends_on:: [[digest/procedure/research-report-analysis.md]]
+contrasts_with:: [[digest/wiki/central-inverter.md]]
+```
+
+Ordinary body wikilinks also create graph edges, but when the relationship itself has semantic value, prefer
+`predicate:: [[path]]`. This makes the meaning of links clearer to search, graph traversal, and later agent integration.
+
+## Human and Agent Editing
+
+Because memory is stored as files, users can edit the workspace directly, while agents can read and write the same files
+through ReMe's file tools. Both follow the same conventions:
+
+| Operation | Guidance |
+|---|---|
+| Add memory | Write to the appropriate directory, use frontmatter for Markdown, and prefer complete workspace-relative wikilinks. |
+| Edit a body | Preserve existing sources and important wikilinks. When correcting an old conclusion, explain how the new material changes the previous judgment. |
+| Move a file | ReMe's move tool rewrites old paths in inbound edges by default. After a manual move, inspect inbound links again. |
+| Delete a file | Check inbound links first. ReMe's delete tool returns source files that still point to the target, making dangling references easier to clean up. |
+| Edit metadata | Use frontmatter for short fields. When the body changes substantially, update `description` as well. |
+
+A practical rule is: **an agent may rewrite the wording, but it must not lose evidence edges**. In particular,
+`derived_from:: [[...]]` and existing digest-to-digest wikilinks are the basis for traceable and extensible long-term memory.
+
+## Path Semantics
+
+All file tools and wikilinks use workspace-relative paths as their basic unit:
+
+```text
+digest/wiki/solar.md
+daily/2026-06-20/session-a.md
+resource/2026-06-20/report.pdf
+```
+
+This creates a clear boundary: ReMe does not treat `[[solar]]` as a repository-wide title search and does not assume
+Obsidian-style same-name resolution. `[[digest/wiki/solar.md]]` points to that exact path.
+
+Recommended practices:
+
+1. Include `.md` when linking a Markdown file.
+2. Use the complete source path when linking from digest to daily or resource.
+3. Rename or move files through ReMe's move tool whenever possible to avoid stale paths.
+4. Put external source material under `resource/YYYY-MM-DD/...` and long-term abstractions under `digest/...`. Do not put
+   raw source material directly into digest.
+
+Explicit path semantics sacrifice a little convenience when writing by hand, but provide predictability, portability, and
+automatic maintainability.
+
+## Memory Chunking
+
+Memory chunking divides a file into retrievable fragments. ReMe does not split Markdown at fixed lengths by default; it tries
+to preserve semantic structure.
+
+This section explains how files become retrieval chunks. For index updates, BM25, vector recall, and link expansion, see
+[Memory Search](./memory_search.md).
+
+Traditional RAG often uses fixed-window splitting:
+
+```text
+Document
+  |
+  | every N tokens + overlap
+  v
+chunk 1 | chunk 2 | chunk 3 | ...
+```
+
+This is simple, but it can cut headings, tables, code blocks, lists, and `[[wikilinks]]` in the middle. After a match, the
+agent often sees only an isolated fragment without knowing its section or relationship to other memory nodes.
+
+ReMe chunking is closer to splitting memory by file structure:
+
+```text
+Markdown file
+  |
+  | frontmatter + headings + blocks + wikilinks
+  v
+semantic chunks with document skeleton
+```
+
+Comparison:
+
+```text
+traditional RAG chunk
+  = fixed-length text fragment + overlap
+
+ReMe memory chunk
+  = section structure + body fragment + line range + wikilink relationship context
+```
+
+Markdown files use `MarkdownFileChunker`:
+
+```text
+Markdown
+  |
+  | mistletoe AST
+  v
+Document
+  └─ H1 section
+      ├─ paragraph / list / table / code
+      └─ H2 section
+          └─ ...
+  |
+  v
+FileChunk[]
+```
+
+Chunking rules:
+
+```text
+1. Parse frontmatter first; send the body to the chunker separately.
+2. Build a section tree from heading levels.
+3. Prefer one complete section per chunk.
+4. When a section is too long, recursively split its subsections and body blocks.
+5. Repeat table headers when splitting tables.
+6. Repeat the fence when splitting code blocks.
+7. Pack lists by item.
+8. Only then split greedily by line and add [Part X/N].
+```
+
+By default, every chunk includes its heading skeleton:
+
+```text
+# Top-level heading
+
+## Current section
+
+Matched body fragment
+
+## Following section heading
+```
+
+This lets the agent see not only an isolated paragraph but also its structural position in the source file.
+
+Non-Markdown files use `DefaultFileChunker` by default. It splits by byte size and preserves a small overlap. For Markdown,
+the chunker also avoids cutting `[[wikilinks]]` in the middle.

docs/en/memory_search.md

@@ -1,7 +1,207 @@
 # Memory Search
 
-```{note}
-📖 The English version of this page is in progress.
+Memory Search is ReMe's memory retrieval entry point. It continuously builds files under `daily/`, `digest/`, and `resource/`
+into a searchable chunk index and wikilink graph. At query time, it first recalls the most relevant fragments and then expands
+context along the bidirectional links of the files containing those fragments.
 
-In the meantime, please read the <a href="../zh/memory_search.html">中文版本</a>.
+<p align="center">
+  <img src="../figure/auto-index-and-memory-search.svg" alt="ReMe Auto Index and Memory Search indexing, recall, fusion, and link expansion" width="92%">
+</p>
+
+For the general semantics of file layers, frontmatter, wikilinks, and chunking, see
+[Memory as File](./memory_as_file.md). This page focuses on index maintenance and query execution.
+
+```text
+workspace files
+  ├─ index_update_loop: detect added / modified / deleted
+  ├─ update_index_step: file -> FileNode + FileChunk[]
+  ├─ file_store: store chunks, BM25, optional embeddings, and the wikilink graph
+  └─ search_step: BM25 / vector recall -> RRF fusion -> link expansion
 ```
+
+## What It Searches
+
+The default `index_update_loop` watches three memory directories:
+
+- `daily_dir`: daily working memory and session memory cards generated by Auto Memory.
+- `digest_dir`: long-term distilled digest nodes.
+- `resource_dir`: external resources or imported material.
+
+The default suffixes are `md` and `jsonl`. Markdown uses the `markdown` chunker, which parses frontmatter, heading structure,
+and `[[wikilinks]]`. JSONL uses the `default` chunker and creates overlapping chunks by byte size.
+
+## How the Index Is Built
+
+The background Job `index_update_loop` maintains the index using configuration from `reme/config/default.yaml`:
+
+```yaml
+index_update_loop:
+  backend: background
+  watch_dirs: [ daily_dir, digest_dir, resource_dir ]
+  watch_suffixes: [ md, jsonl ]
+  steps:
+    - backend: init_changes_step
+      monitor_type: file_store
+      monitor_name: default
+      dispatch_steps: [ update_index_step ]
+    - backend: watch_changes_step
+      dispatch_steps: [ update_index_step ]
+```
+
+`init_changes_step` runs at startup. It scans the watched directories, compares file mtimes on disk with
+`FileNode.st_mtime` values already stored in `file_store`, calculates added, modified, and deleted changes, and passes
+`context["changes"]` to `update_index_step`.
+
+While the service is running, `watch_changes_step` takes over. It uses `watchfiles.awatch()` to watch the same directories,
+groups file events within a quiet window, and uses `coalesce_changes()` to collapse repeated events for the same path into one
+stable batch of changes.
+
+`update_index_step` performs the actual index writes:
+
+1. Select a file chunker by suffix.
+2. Parse the file into one `FileNode` and multiple `FileChunk` objects.
+3. For an added or modified file, delete its old chunks before upserting the new chunks.
+4. For a deleted file, remove its records from `file_store`, `keyword_index`, and `file_graph`.
+5. When changes exist, dump state to `metadata/` so it can be restored on the next startup.
+
+The Markdown chunker parses YAML frontmatter, heading structure, and `[[...]]` into `FileNode`, `FileChunk`, and `FileLink`
+objects. For detailed chunking rules, see [Memory as File](./memory_as_file.md#memory-chunking).
+
+## What file_store Contains
+
+The default `file_store.default` backend is `local`:
+
+```yaml
+file_store:
+  default:
+    backend: local
+    embedding_store: ""
+    keyword_index: default
+    file_graph: default
+```
+
+It combines three kinds of capability:
+
+| Part | Default state | Purpose |
+|---|---|---|
+| `file_chunks` | Enabled | Store `FileChunk` text, line numbers, scores, and optional embeddings. |
+| `keyword_index.default` | Enabled | BM25 inverted index where chunk ID is the document ID. |
+| `file_graph.default` | Enabled | Store `FileNode` objects and wikilink edges. |
+| `embedding_store` | Disabled | When enabled, generate embeddings for chunks and support vector recall. |
+
+Out of the box, search therefore uses primarily BM25 plus link expansion. After setting `embedding_store: default`,
+`SearchStep` runs vector and keyword recall together.
+
+## How to Search
+
+The `search` Job is also configured in `default.yaml`:
+
+```yaml
+search:
+  backend: base
+  description: "Hybrid workspace search (vector + BM25, RRF-fused)."
+  parameters:
+    query: string
+    limit: integer
+    min_score: number
+  steps:
+    - backend: search_step
+      vector_weight: 0.7
+      candidate_multiplier: 3.0
+      expand_links: true
+      max_links_per_direction: 10
+```
+
+Call it with:
+
+```bash
+reme search query="recent discussions about indexing" limit=5
+```
+
+`search_step` executes in this order:
+
+```mermaid
+flowchart LR
+    A["query + limit"] --> B["candidates = limit * candidate_multiplier"]
+    B --> C["file_store.vector_search(...)"]
+    B --> D["file_store.keyword_search(...)"]
+    C --> E["RRF fusion"]
+    D --> E
+    E --> F["min_score filter"]
+    F --> G["truncate to limit"]
+    G --> H["expand_links(...)"]
+    H --> I["Response.answer + metadata"]
+```
+
+If only BM25 has results, the BM25 ranking is returned directly. If only vector search has results, the vector ranking is
+returned directly. When both have results, they are fused with RRF. RRF does not compare BM25 and cosine scores directly; it
+compares ranks in the two result lists:
+
+```text
+fused_score = vector_weight / (60 + vector_rank)
+            + keyword_weight / (60 + keyword_rank)
+```
+
+The default `vector_weight=0.7` gives semantic recall more weight when embeddings are enabled, while keyword search can still
+promote chunks with exact term matches.
+
+## How BM25 Works
+
+`keyword_search()` calls `keyword_index.retrieve(query, limit)`. Each chunk is a document in the BM25 index:
+
+- `doc_id` is `FileChunk.id`.
+- `content` is `FileChunk.text`.
+- The tokenizer splits text into tokens.
+- The inverted index records which chunks contain each token and its term frequency within each chunk.
+- A query scores only the posting lists matching its tokens and returns the highest-scoring chunk IDs.
+
+When a file changes, `LocalFileStore.upsert()` first removes the BM25 documents corresponding to the file's old `chunk_ids`
+and then adds the new chunk text. Deletion is lazy; the index can later be compacted with optimize.
+
+## Progressive Expansion
+
+"Progressive" in Memory Search does not mean putting the entire repository into one result. Retrieval expands in three layers:
+
+1. Chunk recall: return only the `limit` most relevant text fragments.
+2. File location: each result includes `path:start_line-end_line`, allowing the caller to read the source precisely with `read`.
+3. Link neighbors: call `expand_links()` for each matched file and expand at most `max_links_per_direction` outlinks and
+   inlinks.
+
+Expansion data comes from `file_graph` rather than rescanning files:
+
+```text
+matched chunk
+  -> chunk.path
+  -> file_store.get_outlinks(path)
+  -> file_store.get_inlinks(path)
+  -> file_store.get_nodes(neighbor_paths)
+  -> render neighbor path, name, description, predicate, and anchor
+```
+
+This keeps search results short while still showing which long-term nodes, resources, or other daily notes a memory connects
+to. If a result is worth pursuing, use `read path=...` to open the source or
+`traverse path=... depth=2` to continue along the wikilink graph.
+
+## Return Format
+
+`SearchStep` writes results in two places:
+
+- `response.answer`: human-readable text. Each matched block contains its path, line numbers, score, and chunk content,
+  followed by outlinks and inlinks.
+- `response.metadata`: structured programmatic results containing `results`, `link_expansion`, and `counts`.
+
+Typical text structure:
+
+```text
+========== daily/2026-06-20/session-a.md:12-28 [score=0.0317 keyword=4.8120] ==========
+...matched memory fragment...
+  outlinks (2):
+    -> digest/indexing.md  name="Indexing"  description="..."
+      via predicate=related
+  inlinks (1):
+    <- daily/2026-06-19.md  name="..."
+      via plain
+```
+
+`counts` reports how many vector and keyword candidates were recalled and how many results were ultimately returned. With
+embeddings disabled by default, `vector` is usually `0` and `hybrid` is `false`.

docs/en/proactive.md

@@ -1,7 +1,138 @@
 # Proactive
 
-```{note}
-📖 The English version of this page is in progress.
+`proactive` is ReMe's interface for reading proactive memory. It does not reanalyze daily notes or call an LLM. It only reads
+the current day's interest topics written by `auto_dream`:
 
-In the meantime, please read the <a href="../zh/proactive.html">中文版本</a>.
+```text
+daily/<date>/interests.yaml
 ```
+
+A host agent can use it to learn "what is worth proactive attention today," then decide whether to remind the user, ask a
+follow-up question, recommend a next step, or produce a proactive insight.
+
+`interests.yaml` is generated by the Topics stage of [Auto Dream](./auto_dream.md). `proactive` only reads and exposes the
+result.
+
+## Configuration
+
+The default configuration is in `reme/config/default.yaml`:
+
+```yaml
+proactive:
+  backend: base
+  description: "Proactive: read daily/<date>/interests.yaml and expose the latest user-interest topics."
+  parameters:
+    date:
+      type: string
+      default: ""
+    include_content:
+      type: boolean
+      default: true
+  steps:
+    - backend: proactive_step
+```
+
+Parameters:
+
+| Parameter | Purpose |
+|---|---|
+| `date` | Date to read in `YYYY-MM-DD` format. When empty, use today in the application's timezone. |
+| `include_content` | Whether to return the raw YAML in metadata. Defaults to `true`. |
+
+## Input Contract
+
+A typical file looks like this:
+
+```yaml
+date: 2026-06-20
+topic_count: 3
+diversity_days: 7
+topics:
+  - title: Quality regression in the memory retrieval pipeline
+    reason: The user has recently made repeated changes to search, node_search, and dream integration.
+    evidence: daily/2026-06-20/session.md
+    keywords:
+      - memory search
+      - auto dream
+    paths:
+      - daily/2026-06-20/session.md
+```
+
+Only the `topics` list is parsed into structured results. Every topic requires at least `title` and `reason`;
+`evidence`, `keywords`, and `paths` are supporting fields.
+
+## Return Value
+
+When the file is read successfully, `proactive_step` writes these values to standard response metadata:
+
+| Field | Description |
+|---|---|
+| `date` | The date actually read. |
+| `path` | `daily/<date>/interests.yaml`. |
+| `topics` | Parsed topic list. |
+| `content` | Raw YAML; returned only when `include_content=true`. |
+| `skipped` | `true` when the file does not exist. |
+| `error` | Read or parse error. |
+| `summary` | Short summary. |
+
+When the file exists and parses successfully, the answer looks like:
+
+```text
+Read 3 proactive topic(s) from daily/2026-06-20/interests.yaml
+```
+
+A missing file is not an error. The call succeeds with a skipped result:
+
+```text
+Skipped: interests file not found at daily/2026-06-20/interests.yaml
+```
+
+This lets a host agent treat "there is no dream result for today yet" as a normal empty state.
+
+## Running Proactive
+
+CLI:
+
+```bash
+reme proactive date=2026-06-20
+```
+
+Omit the raw YAML content:
+
+```bash
+reme proactive date=2026-06-20 include_content=false
+```
+
+## Relationship to auto_dream
+
+`proactive` is the downstream read step for `auto_dream`:
+
+```text
+daily notes
+  -> auto_dream
+  -> daily/<date>/interests.yaml
+  -> proactive
+  -> host agent
+```
+
+The responsibilities are divided as follows. For the complete Extract, Integrate, Topics, and Finish flow, see
+[Auto Dream](./auto_dream.md):
+
+| Module | Responsibility |
+|---|---|
+| `dream_extract_step` | Extract topic candidates from changed daily inputs. |
+| `dream_topics_step` | Deduplicate, select, and write `interests.yaml`. |
+| `proactive_step` | Read `interests.yaml` and expose it to the host agent. |
+
+`proactive` does not modify files, update a catalog, or decide whether the user should be interrupted. It only provides the
+day's topic material. The caller's product policy determines whether, when, and in what tone to push it to the user.
+
+## Failure Modes
+
+| Scenario | Behavior |
+|---|---|
+| `interests.yaml` does not exist | `success=true`, `skipped=true`, `topics=[]`. |
+| YAML cannot be read or parsed | `success=false`; the answer contains an error summary. |
+| YAML exists but has no valid topics | `success=true`, `topics=[]`. |
+
+Callers should therefore check `success` first, then `skipped`, and finally whether `topics` is empty.

docs/en/quick_start.md

@@ -1,7 +1,214 @@
 # Quick Start
 
-```{note}
-📖 The English version of this page is in progress.
+## Installation
 
-In the meantime, please read the <a href="../zh/quick_start.html">中文版本</a>.
+ReMe requires Python 3.11+.
+
+Install from pip:
+
+```bash
+pip install "reme-ai[core]"
+```
+
+Install from source:
+
+```bash
+git clone https://github.com/agentscope-ai/ReMe.git
+cd ReMe
+pip install -e ".[core]"
+```
+
+Installing the `core` extra is recommended. The current code imports the AgentScope wrapper, and self-evolving memory also
+depends on it.
+
+To use agent workflows such as `auto_memory`, `auto_resource`, and `auto_dream`, configure an LLM:
+
+```bash
+cat > .env <<'EOF'
+LLM_BACKEND=openai
+LLM_MODEL_NAME=qwen3.7-plus
+LLM_API_KEY=your_api_key
+LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
+EOF
+```
+
+You can initially omit the LLM configuration if you only need basic file operations and BM25 retrieval.
+
+---
+
+## Start the Service
+
+```bash
+reme start
+```
+
+The default service address is `127.0.0.1:2333`. If the port is already in use:
+
+```bash
+reme start service.port=8181
+```
+
+```bash
+reme version
+reme health_check
+reme list
+```
+
+`reme list` lists server actions. Ordinary commands invoke server Jobs over HTTP.
+
+---
+
+## Workspace Layout
+
+The default workspace is `.reme/` under the current directory. It is created automatically at startup:
+
+```text
+.reme/
+├── metadata/   # persistent indexes, graph, catalogs, and related state
+├── session/    # agent sessions and original conversations
+├── resource/   # external resources
+├── daily/      # daily notes
+└── digest/     # long-term memory
+```
+
+For directory layers, Markdown frontmatter, and wikilink semantics, see
+[Memory as File](./memory_as_file.md).
+
+You can also specify the workspace at startup:
+
+```bash
+reme start workspace_dir=/tmp/reme-demo service.port=8181
+```
+
+---
+
+## Write, Index, and Search
+
+```bash
+reme write \
+  path=digest/wiki/quick-start-demo \
+  name="Quick Start Demo" \
+  description="Example memory for the quick start" \
+  content="# Quick Start Demo
+
+ReMe indexes Markdown under the daily, digest, and resource directories.
+
+Related link: [[digest/wiki/search-demo.md]]"
+```
+
+`path` is relative to the workspace. A missing suffix is automatically completed with `.md`. For Markdown files, `name` and
+`description` are written to frontmatter.
+
+The background watcher builds the index automatically. You can also rebuild it manually:
+
+```bash
+reme reindex
+```
+
+Search:
+
+```bash
+reme search query="quick start example memory" limit=5
+```
+
+Read:
+
+```bash
+reme read path=digest/wiki/quick-start-demo start_line=1 end_line=20
+```
+
+With the default configuration, retrieval is primarily BM25 plus wikilink graph expansion. Vector retrieval is supported by
+the code, but the embedding store is disabled by default. For the full retrieval flow, see
+[Memory Search](./memory_search.md).
+
+---
+
+## Files and Daily Notes
+
+```bash
+reme stat path=digest/wiki/quick-start-demo
+reme edit path=digest/wiki/quick-start-demo old="indexes" new="continuously indexes"
+reme frontmatter_read path=digest/wiki/quick-start-demo
+reme frontmatter_update path=digest/wiki/quick-start-demo metadata='{"tags":["demo"]}'
+```
+
+The name `list` is used by the CLI to list actions, so the file-listing Job must be called over HTTP:
+
+```bash
+curl -s http://127.0.0.1:2333/list \
+  -H 'Content-Type: application/json' \
+  -d '{"path":"digest","recursive":true,"limit":50}'
+```
+
+Daily notes:
+
+```bash
+reme write path=daily/2026-06-20/demo-session.md name=demo-session description="Demo session" content="Recorded content"
+reme daily_list
+reme daily_reindex
+```
+
+`write` can create a daily note directly. Run `daily_reindex` when the day's index needs to be refreshed.
+
+---
+
+## Automatic Memory
+
+```bash
+reme auto_memory \
+  session_id=chat-demo \
+  messages='[{"role":"user","content":"I prefer to preserve project experience as Markdown."},{"role":"assistant","content":"Recorded."}]' \
+  memory_hint="Record the user's preference"
+```
+
+After placing external material under `resource/YYYY-MM-DD/`, the default background task watches
+`md/txt/json/jsonl/csv/yaml/html`. You can also trigger processing manually:
+
+```bash
+reme auto_resource changes='[{"path":"resource/2026-06-20/report.md","change":"added"}]'
+```
+
+Distill daily notes into long-term digest memory:
+
+```bash
+reme auto_dream date=2026-06-20
+reme proactive date=2026-06-20
+```
+
+These flows require a working LLM. Without an LLM configuration, start with basic capabilities such as `write`, `read`, and
+`search`.
+
+For more detail, see [Auto Memory](./auto_memory.md), [Auto Resource](./auto_resource.md),
+[Auto Dream](./auto_dream.md), and [Proactive](./proactive.md).
+
+---
+
+## HTTP and Configuration
+
+Every service-enabled Job is exposed as `POST /<job>`:
+
+```bash
+curl -s http://127.0.0.1:2333/version \
+  -H 'Content-Type: application/json' \
+  -d '{}'
+
+curl -s http://127.0.0.1:2333/search \
+  -H 'Content-Type: application/json' \
+  -d '{"query":"quick start","limit":5}'
+```
+
+The default configuration comes from `reme/config/default.yaml`. Override it at startup with dot notation:
+
+```bash
+reme start \
+  workspace_dir=/tmp/reme-demo \
+  service.host=127.0.0.1 \
+  service.port=8181 \
+  enable_logo=false
+```
+
+You can also specify a YAML or JSON configuration file:
+
+```bash
+reme start config=/path/to/custom.yaml
 ```