ReMe/docs/en/proactive.md
imrewce 354837f9af
feat(proactive): separate proactive refresh from auto dream (#488)
* refractor(proactive): upgrade proactive feature with disentangled job and steps

* refactor(proactive): apply audit fixes

- rename read-side job 'proactive' -> 'proactive_read' (less confusing vs the refresh pipeline)
- drop dedicated agent_wrapper.proactive; extraction reuses the default wrapper
- simplify schema: remove unused ProactiveExtractOutput/TopicUpdate, drop resource_paths
- extract no longer scans resource/ directly (daily notes already carry resource content)
- update tests and docs accordingly

* feat(proactive): strict extract-output gate and prompt total budget

- parse_extract_reply now requires a contract section (follow_ups/extends/updates
  as a list); non-empty replies with misspelled section names trigger the
  existing one-shot retry instead of silently checkpointing changed files
- pack_paths gains max_total_chars; extract packs newest daily material first,
  keeps the first file on overflow, and records omitted files in a trailer
  (default budget 300000 chars, configurable via max_total_chars)
- tests: schema gate unit, schema-error retry e2e, budget unit + e2e

* feat(proactive): add scenario-card plan step and generative agenda step

* feat(proactive): digest-personal profile personalization and leaner LLM contract

- extract/plan/agenda now draw a user profile block from <digest_dir>/personal/*.md
  (frontmatter description + body excerpt, per-file budget, profile.md fallback)
- all daily access honours the configured daily_dir (prompt paths parameterized,
  config-driven fallbacks) so workspaces using e.g. memory/ work unchanged
- schema trim: drop dead fields errors/material_paths, carry_forward_all -> count
- shrink LLM output contract: new topics emit title/reason/confidence/paths only;
  keywords removed end-to-end, evidence derived from paths[0] (updates keep it)

* fix(proactive): skip checkpoint when extract reply stays unusable after retry

Two consecutive unparseable replies now short-circuit the round without
checkpointing, so the same material is retried next round instead of being
silently consumed (closes the residual audit #1 gap: the structural gate
detected schema-wrong output but a double failure still checkpointed).

* fix(proactive): replace running bool with reference-counted job activity tracker for the idle gate

* refactor(proactive): remove job activity tracking and idle gate, restore job tree to upstream

* fix(proactive): address second audit round (readonly reader, mtime checkpoint, wider fallbacks, profile containment, horizon content, expiry boundary)

* refactor(dream): strip interests.yaml ownership from dream, proactive is now the sole writer

* refactor(dream): separate proactive topic generation

* ci: update renamed auto dream smoke test

* fix(proactive): complete refresh migration and docs

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-09-07 17:23:37 +08:00

10 KiB

Proactive

proactive_read is ReMe's interface for reading proactive memory. It does not reanalyze daily notes or call an LLM. It reads interest topics written by the independent proactive refresh flow:

daily/<date>/interests.yaml

A host agent can use it to learn "what is worth proactive attention today," then decide whether to remind the user, ask a follow-up question, recommend a next step, or produce a proactive insight.

interests.yaml is generated by the proactive refresh pipeline, scheduled by proactive_refresh_cron by default. proactive_read only reads and exposes the result; Auto Dream is an independent daily-to-digest flow and does not read or write proactive state.

Configuration

The default configuration is in reme/config/default.yaml. It defines the same refresh steps twice: a local-only one-shot job for maintenance and debugging, and the scheduled job that runs every day at 18:00 in the application timezone:

proactive_refresh:
  backend: base
  enable_serve: false
  steps: &proactive_refresh_steps
    - backend: proactive_extract_step
      file_catalog: proactive
      scan_days: 2
      carry_forward_days: 14
      max_carry_forward_topics: 20
      llm_timeout_seconds: 300
      max_chars_per_file: 60000
      max_total_chars: 300000
    - backend: proactive_topics_step
      known_threshold: 0.85
      known_threshold_calibrated_for: text-embedding-v4@1024
      min_push_confidence: 0.5
      max_topics: 10
    - backend: proactive_plan_step
    - backend: proactive_agenda_step
    - backend: proactive_finish_step
      file_catalog: proactive

proactive_refresh_cron:
  backend: cron
  cron: "0 18 * * *"
  steps: *proactive_refresh_steps

The anchor above is only a compact illustration; default.yaml spells out both step lists explicitly. The read job is:

proactive_read:
  backend: base
  description: "Proactive: read daily/<date>/interests.yaml and expose the latest user-interest topics."
  parameters:
    type: object
    properties:
      date:
        type: string
        default: ""
      include_content:
        type: boolean
        default: true
      horizon_days:
        type: integer
        default: 1
      min_confidence:
        type: number
        default: 0.4
  steps:
    - backend: proactive_step
      min_confidence: 0.4

Parameters:

Parameter Purpose
date Date to read in YYYY-MM-DD format. When empty, use today in the application's timezone.
include_content Whether to return the raw YAML in the answer and metadata. Defaults to true.
horizon_days Read one day's exposure file, or use the truth source for a wider evidence horizon. Defaults to 1.
min_confidence Minimum topic confidence to return. Defaults to 0.4; legacy topics use 0.5.

Refresh cost, files, and opt-out

When no daily Markdown note changed, refresh exits before calling an LLM and does not create a new exposure file. With changed material, extraction normally makes one LLM call and may retry once after an unusable reply. If push candidates remain, planning makes one additional call; agenda generation makes one more when there are multiple candidates. A refresh therefore makes at most four LLM calls with the default chain.

The refresh pipeline maintains the rebuildable daily/_proactive.yaml truth source, writes daily/<date>/interests.yaml, and advances the independent proactive file catalog. Auto Dream does not read or write any of those proactive artifacts.

To disable automatic refresh, use an explicit application config that omits the proactive_refresh_cron job. Keep the local-only proactive_refresh job if you still want on-demand maintenance. Because it has enable_serve: false, it is not exposed through HTTP or MCP.

Input Contract

A current proactive-refresh file looks like this:

version: 2
date: 2026-06-20
generated_at: 2026-06-20T18:00:00+08:00
push: true
topics:
  - id: baa88ad49cb2
    title: Quality regression in the memory retrieval pipeline
    reason: The user has recently made repeated changes to search, node_search, and dream integration.
    kind: follow_up
    confidence: 0.86
    first_seen: 2026-06-20
    last_evidence_at: 2026-06-20
    evidence: daily/2026-06-20/session.md
    paths:
      - daily/2026-06-20/session.md
agenda:
  - topic_id: baa88ad49cb2
    title: Quality regression in the memory retrieval pipeline
    scenario_type: resume_task
    opener: Review the latest retrieval regression before the next release.
    next_action: Compare the failing query against the previous index snapshot.
    preconditions: []
    delivery: in_conversation
    linked_memory: []
    order_reason: Recent evidence and a concrete next action.
suppressed: []

Current v2 topics include stable identity, kind, confidence, evidence dates, and source paths. The reader also accepts legacy v1 files containing title, reason, evidence, keywords, and paths; missing v2 confidence falls back to 0.5.

Return Value

When the file is read successfully, proactive_step returns summary and topics in the primary answer. When include_content=true, the answer also contains content. The same result fields remain available in standard response metadata:

Field Description
date The date actually read.
path daily/<date>/interests.yaml.
topics Parsed topic list.
content Raw YAML; returned only when include_content=true.
skipped true when the file does not exist.
error Read or parse error.
summary Short summary.
agenda Today's proactive agenda (optional, v2 files only).

When today's interests.yaml was produced by the proactive refresh chain with an agenda, the answer also carries an agenda field: the ordered agenda items, each with topic_id, title, scenario_type, opener (a natural conversation opener), next_action (the minimal executable step), preconditions, delivery, linked_memory and order_reason. Agenda items whose topic is resolved or below min_confidence are filtered out on read; the field is absent when the file has no agenda.

When the file exists and parses successfully, the answer is structured data. For example:

{
  "summary": "Read 1 proactive topic(s) from daily/2026-06-20/interests.yaml",
  "topics": [
    {
      "id": "baa88ad49cb2",
      "title": "Quality regression in the memory retrieval pipeline",
      "reason": "The user has recently made repeated changes to search, node_search, and dream integration.",
      "kind": "follow_up",
      "confidence": 0.86,
      "first_seen": "2026-06-20",
      "last_evidence_at": "2026-06-20",
      "evidence": "daily/2026-06-20/session.md",
      "paths": ["daily/2026-06-20/session.md"]
    }
  ],
  "agenda": [
    {
      "topic_id": "baa88ad49cb2",
      "title": "Quality regression in the memory retrieval pipeline",
      "scenario_type": "resume_task",
      "opener": "Review the latest retrieval regression before the next release.",
      "next_action": "Compare the failing query against the previous index snapshot.",
      "preconditions": [],
      "delivery": "in_conversation",
      "linked_memory": [],
      "order_reason": "Recent evidence and a concrete next action."
    }
  ],
  "content": "date: 2026-06-20\n..."
}

With include_content=false, the content field is omitted from the answer. Missing-file and read-error answers remain explicit Skipped: ... and Error: ... messages, respectively.

A missing file is not an error. The call succeeds with a skipped result:

Skipped: interests file not found at daily/2026-06-20/interests.yaml

This lets a host agent treat "there is no proactive refresh result for today yet" as a normal empty state.

Running Proactive

Run one refresh immediately through the normal application lifecycle:

reme start job=proactive_refresh date=2026-06-20

This command may call the configured LLM and may update _proactive.yaml, interests.yaml, and the proactive catalog. It does not run Auto Dream.

Read the generated topics:

reme proactive_read date=2026-06-20

Omit the raw YAML content:

reme proactive_read date=2026-06-20 include_content=false

Relationship to auto_dream

Proactive refresh and Auto Dream consume daily notes independently:

daily notes -> auto_dream -> digest
daily notes -> proactive_refresh_cron -> daily/<date>/interests.yaml -> proactive_read -> host agent

The proactive responsibilities are divided as follows:

Module Responsibility
proactive_refresh Run the refresh pipeline once from the local CLI.
proactive_refresh_cron Run the same writer pipeline every day at 18:00.
proactive_step Read interests.yaml and expose it to the host agent.

proactive_read does not modify files, update a catalog, or decide whether the user should be interrupted. It only provides the day's topic material. The caller's product policy determines whether, when, and in what tone to push it to the user.

Failure Modes

Scenario Behavior
interests.yaml does not exist success=true, skipped=true, topics=[].
YAML cannot be read or parsed success=false; the answer contains an error summary.
YAML exists but has no valid topics success=true, topics=[].

Callers should therefore check success first, then skipped, and finally whether topics is empty.