ReMe/docs/en/auto_memory.md

Auto Memory

Auto Memory is ReMe's entry point for conversational memory. Within a target date, it uses session_id to find or update at most one daily memory card, whose filename is a concise topic or event name chosen by the Agent. The day's YYYY-MM-DD.md page indexes those cards. It turns "we talked about it" into "it was remembered" while retaining a source conversation record as evidence.

For the general file semantics of daily/, session/, frontmatter, and wikilinks, see Memory as File.

Conversation
  ├─ step 1: daily/YYYY-MM-DD/<generated_name>.md # one topic-named card per session
  ├─ step 2: daily/YYYY-MM-DD.md                  # daily index linking the cards
  └─ source: session/dialog/<session_id>.jsonl    # source conversation record

What It Records

Auto Memory does not preserve a chat transcript as a running summary. It records information that may remain useful later:

• User preferences: preferred style, collaboration habits, and long-term requirements.
• Key facts: project background, important numbers, explicit conclusions, and constraints.
• Process decisions: what happened, why a choice was made, and which alternatives were rejected.
• Current state: what has been completed, what is blocked, and what comes next.
• Reusable experience: commands, workflows, diagnostic methods, and solutions.

Write Location

Auto Memory writes distilled memories to daily/. Conversations from the same day first become individual cards:

Example directory:

workspace/
  daily/
    2026-06-20.md
    2026-06-20/
      login-refactor-decision.md
      retrieval-regression.md

The two files under the date directory are topic-named cards distilled from different conversations. daily/2026-06-20.md is the index page for that day. Resource files enter the same daily memory layer; see Auto Resource.

When a call includes session_id, Auto Memory uses it to find the corresponding card through frontmatter, while the Agent chooses a readable filename through name:

name: login-refactor-decision
session_id: session-a
source_conversation: "[[session/dialog/session-a.jsonl]]"

This keeps different conversations separate without forcing opaque IDs into filenames. An update locates the existing note by session_id or source_conversation; if the Agent supplies a better frontmatter name, the system can rename the note and retarget inbound wikilinks. To see what happened on a day, start with YYYY-MM-DD.md.

Preserving the Original Information

The distilled daily note is optimized for readability; a filtered source conversation record is retained for trust and verification.

While generating memory cards, Auto Memory also saves the source messages:

session/
  dialog/
    session-a.jsonl
    session-b.jsonl

Each daily note points to its corresponding conversation record. Saved messages omit tool-result blocks and base64 data blocks, preventing recalled memory and binary payloads from being mistaken for user-provided evidence later.

Images in Conversations

Auto Memory can read images together with the surrounding conversation. Images are disabled by default; enable them for a call with include_images=true.

Image input requires an agentscope wrapper with a vision-capable as_llm model and compatible formatter. Auto Memory uses that model to read the conversation, without generating captions first. When images are disabled or no image blocks are present, the existing text-only behavior is unchanged, including support for other wrappers.

Pass images as top-level AgentScope DataBlock values in messages, with an image/ media type. Text and images stay in their original order, with speaker and timestamp boundaries preserved. Base64 sources and HTTP(S) URLs pass unchanged to the formatter; Auto Memory does not download or preprocess the images. URLs must be accessible to the model provider. For local files, submit Base64 instead of a file:// URL; other URL schemes are also unsupported.

The wrapper's context_config.max_image_num limits the number of images per call; Auto Memory rejects excess images rather than increasing the limit. The AgentScope default is 5. To use a higher limit, set it when starting the service:

reme start components.agent_wrapper.default.context_config.max_image_num=20

Then call the running service from another terminal, using the same workspace:

reme auto_memory session_id=session-a include_images=true messages='[...]'

Model and formatter limits still apply. When image input is enabled and images are present, Auto Memory checks the wrapper backend, URL schemes and image count before saving the conversation. Later formatter or provider errors are returned without retrying as text-only. As with text-only calls, those errors do not roll back an already saved conversation.

Source JSONL saving follows the filtering rules above, including the omission of Base64 blocks. To process those images again, resubmit the original messages rather than the saved JSONL. No separate image files or caption cards are created, though the wrapper's internal Agent state under mem_session/agentscope can contain image inputs.

Message Timestamps

Auto Memory preserves each retained message's created_at in both the prompt and the source conversation JSONL. When importing historical conversations or benchmark data, provide the actual occurrence time for every message so the model does not confuse event time with execution time:

reme auto_memory \
  session_id=locomo-session \
  messages='[
    {"role":"user","content":"Jon lost his job today.","created_at":"2023-01-19T08:00:00"},
    {"role":"assistant","content":"I am sorry to hear that.","created_at":"2023-01-19T08:01:00"}
  ]'

For compatibility with common dataset schemas, auto_memory also checks time_created, timestamp, createdAt, timeCreated, and created_time when created_at is absent. These fields may appear either at the top level of a message or inside metadata.

When a call does not explicitly provide date, Auto Memory uses the latest valid created_at date in the messages. If no message contains a valid timestamp, it falls back to the current date. Historical imports may also specify the target date directly:

reme auto_memory \
  session_id=locomo-session \
  date=2023-01-19 \
  messages='[{"role":"user","content":"Jon lost his job today."}]'

What Happens Next

The default auto_memory and auto_memory_cc jobs run auto_tag_step after recording memory. Only a daily note that was actually created or modified is tagged, using its final path after any rename. Claude Code callers still pass only session_id; repeated Stop events with no new messages skip both memory generation and tagging.

Tags describe the document's central entities and are stored in the configured frontmatter key (memory_tags by default). Per-file tagging failures are reported in metadata.auto_tag while preserving the memory response. Calls without note changes do not automatically retry failed tagging; the existing file watcher updates the tag index asynchronously.

Auto Memory only creates memory in the daily layer. To distill this material further into long-term digest/ nodes, use Auto Dream. To search daily and digest content, use Memory Search.