mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-10-10 03:30:56 +00:00
198 lines
8.7 KiB
Markdown
198 lines
8.7 KiB
Markdown
# Auto Dream
|
|
|
|
`auto_dream` is ReMe's long-term memory distillation flow from daily to digest. By default it scans the target date and
|
|
the previous day, processes only files changed since the previous dream, extracts a small set of high-value memory units
|
|
across that window, and integrates them into `digest/`.
|
|
|
|
Its daily inputs usually come from [Auto Memory](./auto_memory.md) and [Auto Resource](./auto_resource.md). For the file
|
|
semantics of `digest/`, Sources sections, and wikilinks, see [Memory as File](./memory_as_file.md). For the linking
|
|
strategy used during Integrate, see [Auto Link](./auto_link.md). Proactive discovery is a separate flow; see
|
|
[Proactive](./proactive.md).
|
|
|
|
## Configuration
|
|
|
|
The default configuration is in `reme/config/default.yaml`:
|
|
|
|
```yaml
|
|
auto_dream:
|
|
backend: base
|
|
parameters:
|
|
date:
|
|
type: string
|
|
default: ""
|
|
hint:
|
|
type: string
|
|
default: ""
|
|
scan_days:
|
|
type: integer
|
|
default: 2
|
|
max_units:
|
|
type: integer
|
|
default: 5
|
|
steps:
|
|
- backend: dream_extract_step
|
|
file_catalog: dream
|
|
scan_days: 2
|
|
max_units: 5
|
|
- backend: dream_integrate_step
|
|
- backend: dream_finish_step
|
|
file_catalog: dream
|
|
- backend: auto_tag_step
|
|
```
|
|
|
|
Parameters:
|
|
|
|
| Parameter | Purpose |
|
|
|------------------------|---------------------------------------------------------------------------------------------------------|
|
|
| `date` | Date to process in `YYYY-MM-DD` format. When empty, use today in the application's timezone. |
|
|
| `hint` | Additional guidance from the caller for the Extract and Integrate stages. |
|
|
| `scan_days` | Recent-date window ending at `date`; defaults to 2 and has a minimum of 1. |
|
|
| `max_units` | Maximum reusable units extracted in one run; defaults to 5. |
|
|
|
|
## Inputs and Outputs
|
|
|
|
Inputs are daily Markdown files from the most recent `scan_days` ending at the specified date. For example,
|
|
`date=2026-06-20` with `scan_days=2` scans:
|
|
|
|
```text
|
|
daily/2026-06-19.md
|
|
daily/2026-06-19/**/*.md
|
|
daily/2026-06-20.md
|
|
daily/2026-06-20/**/*.md
|
|
```
|
|
|
|
Only Markdown day indexes and notes are scanned. Proactive state and `interests.yaml` are not Auto Dream inputs.
|
|
|
|
The main outputs are:
|
|
|
|
| Output | Description |
|
|
|--------------------------------|------------------------------------------------------------------------------|
|
|
| `digest/procedure/*.md` | Methods, workflows, runbooks, and executable experience. |
|
|
| `digest/personal/*.md` | User-, team-, and project-related preferences, facts, and long-term context. |
|
|
| `digest/wiki/*.md` | General knowledge, concepts, observations, and decision precedents. |
|
|
| `metadata/file_catalog/dream*` | Dream-specific catalog used to detect changes in daily inputs. |
|
|
|
|
## Four Stages
|
|
|
|
### 1. Extract
|
|
|
|
`dream_extract_step` performs three tasks:
|
|
|
|
1. Refresh each `daily/<date>.md` in the scan window.
|
|
2. Scan those day indexes and `daily/<date>/**/*.md`, comparing mtimes with `file_catalog: dream`.
|
|
3. Send all changed files together to the LLM and globally extract structured memory `units`.
|
|
|
|
`units` are long-term memory units ready to be distilled into digest. Each has `name`, `bucket`, `summary`, and `paths`.
|
|
A run returns at most `max_units`; extraction merges cross-file evidence for the same abstraction and drops passing
|
|
mentions, per-file summaries, and weak candidates without reusable value. `bucket` may only be `procedure`, `personal`,
|
|
or `wiki`; unknown values are routed to `wiki`.
|
|
|
|
If there are no changed files, Extract succeeds with no units; Integrate then has no unit work, and Finish still
|
|
performs its normal catalog summary. If files changed but no LLM is
|
|
configured, Extract fails because extraction requires an LLM.
|
|
|
|
### 2. Integrate
|
|
|
|
`dream_integrate_step` invokes an agent independently for each unit and integrates that unit into one digest node. It
|
|
exposes these tools to the agent:
|
|
|
|
```text
|
|
node_search, read, frontmatter_read, write, edit, frontmatter_update
|
|
```
|
|
|
|
This stage carries the core responsibility of `auto_link`. It first uses `node_search` to recall similar or related
|
|
nodes at digest-node granularity, decides whether to create or update a node, and finally writes sources and related
|
|
digest nodes as wikilinks. See [Auto Link](./auto_link.md) for the recall, deduplication, and edge-writing rules.
|
|
|
|
Extract is the gate for deciding whether material is worth remembering, so Integrate has no `SKIP` action: each admitted
|
|
unit must land in exactly one digest node. Creates and updates must retain provenance and weave related digest links
|
|
into contextual sentences; bare wikilinks and standalone relationship fields are not valid output.
|
|
|
|
There are four integration actions:
|
|
|
|
| Action | Meaning |
|
|
|---------------|--------------------------------------------------------------------------------|
|
|
| `CREATE` | No equivalent abstraction exists; create a new digest node. |
|
|
| `CORROBORATE` | The same memory appeared again; append a source or strengthen the description. |
|
|
| `REFINE` | New material adds boundaries, steps, prerequisites, applicability, or detail. |
|
|
| `CORRECT` | New material corrects errors, omissions, or conflicts in the existing node. |
|
|
|
|
Successfully integrated units are recorded in `integrate_results`. Failed units enter `failed_units`, and their source
|
|
paths enter `failed_paths`. The Finish stage does not checkpoint failed paths, ensuring that they can be retried later.
|
|
|
|
### 3. Finish
|
|
|
|
`dream_finish_step` completes the run:
|
|
|
|
1. Write successfully processed changed paths to `file_catalog: dream`.
|
|
2. Also write every refreshed day-index page in the scan window to the catalog.
|
|
3. Persist the dream catalog if there were upserts or deletions.
|
|
4. Return a summary containing counts for scanned, changed, integrated, checkpoints, and related values.
|
|
|
|
Auto Dream neither reads nor writes proactive state or `interests.yaml`. Those files are owned by the proactive refresh
|
|
pipeline; see [Proactive](./proactive.md).
|
|
|
|
Failed paths are not checkpointed. The next `auto_dream` run therefore continues to treat them as changed inputs until
|
|
integration succeeds.
|
|
|
|
### 4. Auto Tag
|
|
|
|
After Finish, both `auto_dream` and `dream_cron` run `auto_tag_step` on Markdown digest files actually created or modified
|
|
during integration, including writes recovered after agent errors. Repeated writes to one file are tagged once. The
|
|
Step uses the same request-scoped `changes` contract as [Auto Memory](./auto_memory.md) and writes entity tags to the
|
|
configured frontmatter key, `memory_tags` by default. Unchanged files and daily source notes are not tagged by Dream.
|
|
|
|
Tagging diagnostics appear in `metadata.auto_tag`. Per-file tagging failures preserve the dream answer, success status,
|
|
and checkpoint decisions. A later run without file changes does not automatically retry failed tagging. Tag-index
|
|
updates follow the existing asynchronous file watcher.
|
|
|
|
## Running Auto Dream
|
|
|
|
CLI:
|
|
|
|
```bash
|
|
reme auto_dream date=2026-06-20
|
|
```
|
|
|
|
With caller guidance:
|
|
|
|
```bash
|
|
reme auto_dream date=2026-06-20 hint="Prioritize engineering decisions and long-term preferences"
|
|
```
|
|
|
|
Override the default scan window and unit cap:
|
|
|
|
```bash
|
|
reme auto_dream date=2026-06-20 scan_days=3 max_units=8
|
|
```
|
|
|
|
The same set of steps can also be placed in a `cron` Job, for example to run every morning:
|
|
|
|
```yaml
|
|
jobs:
|
|
daily_auto_dream:
|
|
backend: cron
|
|
cron: "30 3 * * *"
|
|
steps:
|
|
- backend: dream_extract_step
|
|
file_catalog: dream
|
|
- backend: dream_integrate_step
|
|
- backend: dream_finish_step
|
|
file_catalog: dream
|
|
- backend: auto_tag_step
|
|
```
|
|
|
|
## Important Boundaries
|
|
|
|
`auto_dream` consumes only daily inputs and does not rewrite daily bodies. Daily preserves facts and the original
|
|
situation; digest is the abstracted long-term memory layer.
|
|
|
|
`digest` is not a copy of the source text. Its body should preserve reusable abstractions, while a Sources section
|
|
points back with contextual sentences such as `The decision was recorded in [[daily/<date>/decision.md]].` Links follow
|
|
the workspace-relative wikilink semantics described in
|
|
[Memory as File](./memory_as_file.md).
|
|
|
|
`auto_dream` does not invent an overview from nothing. Only content that actually appears in daily input and is
|
|
extracted as a memory unit can enter digest.
|
|
|
|
The complete flow depends on an LLM for Extract, Integrate, and Auto Tag.
|