ReMe/docs/en/auto_memory.md

# Auto Memory

Auto Memory is ReMe's entry point for conversational memory. Within a target date, it uses `session_id` to find or update at
most one daily memory card, whose filename is a concise topic or event name chosen by the Agent. The day's `YYYY-MM-DD.md`
page indexes those cards. It turns "we talked about it" into "it was remembered" while retaining a source conversation record
as evidence.

<p align="center">
  <img src="../figure/auto-memory-resource.svg" alt="ReMe Auto Memory and Auto Resource writing daily memory cards" width="92%">
</p>

For the general file semantics of `daily/`, `session/`, frontmatter, and wikilinks, see
[Memory as File](./memory_as_file.md).

```text
Conversation
  ├─ step 1: daily/YYYY-MM-DD/<generated_name>.md # one topic-named card per session
  ├─ step 2: daily/YYYY-MM-DD.md                  # daily index linking the cards
  └─ source: session/dialog/<session_id>.jsonl    # source conversation record
```

## What It Records

Auto Memory does not preserve a chat transcript as a running summary. It records information that may remain useful later:

- User preferences: preferred style, collaboration habits, and long-term requirements.
- Key facts: project background, important numbers, explicit conclusions, and constraints.
- Process decisions: what happened, why a choice was made, and which alternatives were rejected.
- Current state: what has been completed, what is blocked, and what comes next.
- Reusable experience: commands, workflows, diagnostic methods, and solutions.

## Write Location

Auto Memory writes distilled memories to `daily/`. Conversations from the same day first become individual cards:

Example directory:

```text
workspace/
  daily/
    2026-06-20.md
    2026-06-20/
      login-refactor-decision.md
      retrieval-regression.md
```

The two files under the date directory are topic-named cards distilled from different conversations.
`daily/2026-06-20.md` is the index page for that day. Resource files enter the same daily memory layer; see
[Auto Resource](./auto_resource.md).

When a call includes `session_id`, Auto Memory uses it to find the corresponding card through frontmatter, while the Agent
chooses a readable filename through `name`:

```yaml
name: login-refactor-decision
session_id: session-a
source_conversation: "[[session/dialog/session-a.jsonl]]"
```

This keeps different conversations separate without forcing opaque IDs into filenames. An update locates the existing note by
`session_id` or `source_conversation`; if the Agent supplies a better frontmatter `name`, the system can rename the note and
retarget inbound wikilinks. To see what happened on a day, start with `YYYY-MM-DD.md`.

## Preserving the Original Information

The distilled daily note is optimized for readability; a filtered source conversation record is retained for trust and
verification.

While generating memory cards, Auto Memory also saves the source messages:

```text
session/
  dialog/
    session-a.jsonl
    session-b.jsonl
```

Each daily note points to its corresponding conversation record. Saved messages omit tool-result blocks and base64 data
blocks, preventing recalled memory and binary payloads from being mistaken for user-provided evidence later.

## Images in Conversations

Auto Memory can read images together with the surrounding conversation. Images are disabled by default; enable them for a
call with `include_images=true`.

Image input requires an `agentscope` wrapper with a vision-capable `as_llm` model and compatible formatter.
Auto Memory uses that model to read the conversation, without generating captions first. When images are disabled or no
image blocks are present, the existing text-only behavior is unchanged, including support for other wrappers.

Pass images as top-level AgentScope `DataBlock` values in `messages`, with an `image/` media type. Text and images stay in
their original order, with speaker and timestamp boundaries preserved. Base64 sources and HTTP(S) URLs pass unchanged to
the formatter; Auto Memory does not download or preprocess the images. URLs must be accessible to the model provider. For local
files, submit Base64 instead of a `file://` URL; other URL schemes are also unsupported.

The wrapper's `context_config.max_image_num` limits the number of images per call; Auto Memory rejects excess images rather
than increasing the limit. The AgentScope default is 5. To use a higher limit, set it when starting the service:

```bash
reme start components.agent_wrapper.default.context_config.max_image_num=20
```

Then call the running service from another terminal, using the same workspace:

```bash
reme auto_memory session_id=session-a include_images=true messages='[...]'
```

Model and formatter limits still apply. When image input is enabled and images are present, Auto Memory checks the wrapper
backend, URL schemes and image count before saving the conversation. Later formatter or provider errors are returned
without retrying as text-only. As with text-only calls, those errors do not roll back an already saved conversation.

Source JSONL saving follows the filtering rules above, including the omission of Base64 blocks. To process those images
again, resubmit the original messages rather than the saved JSONL. No separate image files or caption cards are created,
though the wrapper's internal Agent state under `mem_session/agentscope` can contain image inputs.

## Message Timestamps

Auto Memory preserves each retained message's `created_at` in both the prompt and the source conversation JSONL. When importing historical
conversations or benchmark data, provide the actual occurrence time for every message so the model does not confuse event
time with execution time:

```bash
reme auto_memory \
  session_id=locomo-session \
  messages='[
    {"role":"user","content":"Jon lost his job today.","created_at":"2023-01-19T08:00:00"},
    {"role":"assistant","content":"I am sorry to hear that.","created_at":"2023-01-19T08:01:00"}
  ]'
```

For compatibility with common dataset schemas, `auto_memory` also checks `time_created`, `timestamp`, `createdAt`,
`timeCreated`, and `created_time` when `created_at` is absent. These fields may appear either at the top level of a message
or inside `metadata`.

When a call does not explicitly provide `date`, Auto Memory uses the latest valid `created_at` date in the messages. If no
message contains a valid timestamp, it falls back to the current date. Historical imports may also specify the
target date directly:

```bash
reme auto_memory \
  session_id=locomo-session \
  date=2023-01-19 \
  messages='[{"role":"user","content":"Jon lost his job today."}]'
```

## What Happens Next

The default `auto_memory` and `auto_memory_cc` jobs run `auto_tag_step` after recording memory. Only a daily note that
was actually created or modified is tagged, using its final path after any rename. Claude Code callers still pass only
`session_id`; repeated Stop events with no new messages skip both memory generation and tagging.

Tags describe the document's central entities and are stored in the configured frontmatter key (`memory_tags` by default).
Per-file tagging failures are reported in `metadata.auto_tag` while preserving the memory response. Calls without note
changes do not automatically retry failed tagging; the existing file watcher updates the tag index asynchronously.

Auto Memory only creates memory in the daily layer. To distill this material further into long-term `digest/` nodes, use
[Auto Dream](./auto_dream.md). To search daily and digest content, use [Memory Search](./memory_search.md).