ReMe/docs/en/auto_dream.md

Auto Dream

auto_dream is ReMe's long-term memory distillation flow from daily to digest. By default it scans the target date and the previous day, processes only files changed since the previous dream, extracts a small set of high-value memory units across that window, and integrates them into digest/.

Its daily inputs usually come from Auto Memory and Auto Resource. For the file semantics of digest/, Sources sections, and wikilinks, see Memory as File. For the linking strategy used during Integrate, see Auto Link. Proactive discovery is a separate flow; see Proactive.

Configuration

The default configuration is in reme/config/default.yaml:

auto_dream:
  backend: base
  parameters:
    date:
      type: string
      default: ""
    hint:
      type: string
      default: ""
    scan_days:
      type: integer
      default: 2
    max_units:
      type: integer
      default: 5
  steps:
    - backend: dream_extract_step
      file_catalog: dream
      scan_days: 2
      max_units: 5
    - backend: dream_integrate_step
    - backend: dream_finish_step
      file_catalog: dream
    - backend: auto_tag_step

Parameters:

Parameter	Purpose
date	Date to process in YYYY-MM-DD format. When empty, use today in the application's timezone.
hint	Additional guidance from the caller for the Extract and Integrate stages.
scan_days	Recent-date window ending at date; defaults to 2 and has a minimum of 1.
max_units	Maximum reusable units extracted in one run; defaults to 5.

Inputs and Outputs

Inputs are daily Markdown files from the most recent scan_days ending at the specified date. For example, date=2026-06-20 with scan_days=2 scans:

daily/2026-06-19.md
daily/2026-06-19/**/*.md
daily/2026-06-20.md
daily/2026-06-20/**/*.md

Only Markdown day indexes and notes are scanned. Proactive state and interests.yaml are not Auto Dream inputs.

The main outputs are:

Output	Description
digest/procedure/*.md	Methods, workflows, runbooks, and executable experience.
digest/personal/*.md	User-, team-, and project-related preferences, facts, and long-term context.
digest/wiki/*.md	General knowledge, concepts, observations, and decision precedents.
metadata/file_catalog/dream*	Dream-specific catalog used to detect changes in daily inputs.

Four Stages

1. Extract

dream_extract_step performs three tasks:

1. Refresh each daily/<date>.md in the scan window.
2. Scan those day indexes and daily/<date>/**/*.md, comparing mtimes with file_catalog: dream.
3. Send all changed files together to the LLM and globally extract structured memory units.

units are long-term memory units ready to be distilled into digest. Each has name, bucket, summary, and paths. A run returns at most max_units; extraction merges cross-file evidence for the same abstraction and drops passing mentions, per-file summaries, and weak candidates without reusable value. bucket may only be procedure, personal, or wiki; unknown values are routed to wiki.

If there are no changed files, Extract succeeds with no units; Integrate then has no unit work, and Finish still performs its normal catalog summary. If files changed but no LLM is configured, Extract fails because extraction requires an LLM.

2. Integrate

dream_integrate_step invokes an agent independently for each unit and integrates that unit into one digest node. It exposes these tools to the agent:

node_search, read, frontmatter_read, write, edit, frontmatter_update

This stage carries the core responsibility of auto_link. It first uses node_search to recall similar or related nodes at digest-node granularity, decides whether to create or update a node, and finally writes sources and related digest nodes as wikilinks. See Auto Link for the recall, deduplication, and edge-writing rules.

Extract is the gate for deciding whether material is worth remembering, so Integrate has no SKIP action: each admitted unit must land in exactly one digest node. Creates and updates must retain provenance and weave related digest links into contextual sentences; bare wikilinks and standalone relationship fields are not valid output.

There are four integration actions:

Action	Meaning
CREATE	No equivalent abstraction exists; create a new digest node.
CORROBORATE	The same memory appeared again; append a source or strengthen the description.
REFINE	New material adds boundaries, steps, prerequisites, applicability, or detail.
CORRECT	New material corrects errors, omissions, or conflicts in the existing node.

Successfully integrated units are recorded in integrate_results. Failed units enter failed_units, and their source paths enter failed_paths. The Finish stage does not checkpoint failed paths, ensuring that they can be retried later.

3. Finish

dream_finish_step completes the run:

1. Write successfully processed changed paths to file_catalog: dream.
2. Also write every refreshed day-index page in the scan window to the catalog.
3. Persist the dream catalog if there were upserts or deletions.
4. Return a summary containing counts for scanned, changed, integrated, checkpoints, and related values.

Auto Dream neither reads nor writes proactive state or interests.yaml. Those files are owned by the proactive refresh pipeline; see Proactive.

Failed paths are not checkpointed. The next auto_dream run therefore continues to treat them as changed inputs until integration succeeds.

4. Auto Tag

After Finish, both auto_dream and dream_cron run auto_tag_step on Markdown digest files actually created or modified during integration, including writes recovered after agent errors. Repeated writes to one file are tagged once. The Step uses the same request-scoped changes contract as Auto Memory and writes entity tags to the configured frontmatter key, memory_tags by default. Unchanged files and daily source notes are not tagged by Dream.

Tagging diagnostics appear in metadata.auto_tag. Per-file tagging failures preserve the dream answer, success status, and checkpoint decisions. A later run without file changes does not automatically retry failed tagging. Tag-index updates follow the existing asynchronous file watcher.

Running Auto Dream

CLI:

reme auto_dream date=2026-06-20

With caller guidance:

reme auto_dream date=2026-06-20 hint="Prioritize engineering decisions and long-term preferences"

Override the default scan window and unit cap:

reme auto_dream date=2026-06-20 scan_days=3 max_units=8

The same set of steps can also be placed in a cron Job, for example to run every morning:

jobs:
  daily_auto_dream:
    backend: cron
    cron: "30 3 * * *"
    steps:
      - backend: dream_extract_step
        file_catalog: dream
      - backend: dream_integrate_step
      - backend: dream_finish_step
        file_catalog: dream
      - backend: auto_tag_step

Important Boundaries

auto_dream consumes only daily inputs and does not rewrite daily bodies. Daily preserves facts and the original situation; digest is the abstracted long-term memory layer.

digest is not a copy of the source text. Its body should preserve reusable abstractions, while a Sources section points back with contextual sentences such as The decision was recorded in [[daily/<date>/decision.md]]. Links follow the workspace-relative wikilink semantics described in Memory as File.

auto_dream does not invent an overview from nothing. Only content that actually appears in daily input and is extracted as a memory unit can enter digest.

The complete flow depends on an LLM for Extract, Integrate, and Auto Tag.