ReMe/reme/schema/proactive.py
imrewce 354837f9af
feat(proactive): separate proactive refresh from auto dream (#488)
* refractor(proactive): upgrade proactive feature with disentangled job and steps

* refactor(proactive): apply audit fixes

- rename read-side job 'proactive' -> 'proactive_read' (less confusing vs the refresh pipeline)
- drop dedicated agent_wrapper.proactive; extraction reuses the default wrapper
- simplify schema: remove unused ProactiveExtractOutput/TopicUpdate, drop resource_paths
- extract no longer scans resource/ directly (daily notes already carry resource content)
- update tests and docs accordingly

* feat(proactive): strict extract-output gate and prompt total budget

- parse_extract_reply now requires a contract section (follow_ups/extends/updates
  as a list); non-empty replies with misspelled section names trigger the
  existing one-shot retry instead of silently checkpointing changed files
- pack_paths gains max_total_chars; extract packs newest daily material first,
  keeps the first file on overflow, and records omitted files in a trailer
  (default budget 300000 chars, configurable via max_total_chars)
- tests: schema gate unit, schema-error retry e2e, budget unit + e2e

* feat(proactive): add scenario-card plan step and generative agenda step

* feat(proactive): digest-personal profile personalization and leaner LLM contract

- extract/plan/agenda now draw a user profile block from <digest_dir>/personal/*.md
  (frontmatter description + body excerpt, per-file budget, profile.md fallback)
- all daily access honours the configured daily_dir (prompt paths parameterized,
  config-driven fallbacks) so workspaces using e.g. memory/ work unchanged
- schema trim: drop dead fields errors/material_paths, carry_forward_all -> count
- shrink LLM output contract: new topics emit title/reason/confidence/paths only;
  keywords removed end-to-end, evidence derived from paths[0] (updates keep it)

* fix(proactive): skip checkpoint when extract reply stays unusable after retry

Two consecutive unparseable replies now short-circuit the round without
checkpointing, so the same material is retried next round instead of being
silently consumed (closes the residual audit #1 gap: the structural gate
detected schema-wrong output but a double failure still checkpointed).

* fix(proactive): replace running bool with reference-counted job activity tracker for the idle gate

* refactor(proactive): remove job activity tracking and idle gate, restore job tree to upstream

* fix(proactive): address second audit round (readonly reader, mtime checkpoint, wider fallbacks, profile containment, horizon content, expiry boundary)

* refactor(dream): strip interests.yaml ownership from dream, proactive is now the sole writer

* refactor(dream): separate proactive topic generation

* ci: update renamed auto dream smoke test

* fix(proactive): complete refresh migration and docs

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-09-07 17:23:37 +08:00

134 lines
4.6 KiB
Python

"""Proactive refresh schemas.
Defines the topic model (v2), the chain-shared context state, and the
``daily/_proactive.yaml`` truth-source file model. The LLM reply contract is
validated structurally by ``parse_extract_reply`` instead of a model here.
See ``PROACTIVE_SPEC.md`` sections F1/A2 for the full contracts.
"""
from pydantic import BaseModel, ConfigDict, Field, field_validator
TOPIC_KINDS = ("follow_up", "interest_extend")
def clamp_confidence(value) -> float:
"""Coerce confidence into [0, 1]; any conversion failure falls back to 0.5."""
try:
return min(1.0, max(0.0, float(value)))
except (TypeError, ValueError):
return 0.5
class ProactiveTopic(BaseModel):
"""One proactive topic; every field has a default so v1 files parse seamlessly.
Fallback rules (A2): invalid ``kind`` -> ``interest_extend``; unparseable
``confidence`` -> 0.5. Missing ``id``, ``first_seen`` and
``last_evidence_at`` are context-dependent and therefore resolved by the
loaders, not here.
"""
id: str = ""
title: str = ""
reason: str = ""
kind: str = "interest_extend"
confidence: float = 0.5
first_seen: str = ""
last_evidence_at: str = ""
evidence: str = ""
paths: list[str] = Field(default_factory=list)
@field_validator("kind", mode="before")
@classmethod
def _fallback_kind(cls, value):
text = str(value or "").strip()
return text if text in TOPIC_KINDS else "interest_extend"
@field_validator("confidence", mode="before")
@classmethod
def _fallback_confidence(cls, value):
return clamp_confidence(value)
@field_validator("paths", mode="before")
@classmethod
def _clean_str_list(cls, value):
if not isinstance(value, list):
return []
return [str(item).strip() for item in value if str(item).strip()]
class ProactiveState(BaseModel):
"""Chain-shared proactive context state (``context['proactive']``).
Extract fills change-detection/carry-forward/LLM output fields; topics fills the
filtering fields plus ``push_candidates`` (today's pushable topics); plan
expands candidates into ``scenario_cards``; agenda selects the ordered
``agenda`` and records ``suppressed`` candidates with reasons; finish
records the catalog checkpoint. ``plan_llm_calls`` counts plan+agenda LLM
calls separately from extract's ``llm_calls``.
``file_skip_reason`` is metadata/log only and never persisted to
interests.yaml (v5 simplification R7).
"""
date: str = ""
daily_dir: str = "daily"
workspace: str = ""
scan_days: int = 2
carry_forward_days: int = 14
changed_paths: list[str] = Field(default_factory=list)
changed_mtimes: dict[str, float] = Field(default_factory=dict)
carry_forward_count: int = 0
carry_forward_prompt: list[ProactiveTopic] = Field(default_factory=list)
llm_calls: int = 0
follow_ups: list[dict] = Field(default_factory=list)
extends: list[dict] = Field(default_factory=list)
updates: list[dict] = Field(default_factory=list)
early_exit: str = ""
updates_applied: int = 0
updates_resolved: int = 0
candidates_in: int = 0
candidates: list[dict] = Field(default_factory=list)
dropped_missing: int = 0
dropped_duplicate: int = 0
dropped_known: int = 0
topics_out: list[dict] = Field(default_factory=list)
push_candidates: list[dict] = Field(default_factory=list)
scenario_cards: list[dict] = Field(default_factory=list)
agenda: list[dict] = Field(default_factory=list)
suppressed: list[dict] = Field(default_factory=list)
plan_llm_calls: int = 0
push: bool = False
file_skip_reason: str = ""
interests_path: str = ""
interests_written: bool = False
checkpoint_paths: list[str] = Field(default_factory=list)
duration_ms: int = 0
class ProactiveStateFile(BaseModel):
"""On-disk truth-source ``daily/_proactive.yaml`` (F1.3, v5: 3 sections).
``resolved`` tombstones carry ``first_seen`` so a resurrected topic can
keep its original age anchor (F2.4 reopen channel).
"""
model_config = ConfigDict(extra="ignore")
version: int = 1
open_topics: list[ProactiveTopic] = Field(default_factory=list)
resolved: list[dict] = Field(default_factory=list)
class ProactiveResult(BaseModel):
"""Result of reading daily interest topics (F5)."""
date: str = ""
path: str = ""
topics: list[dict] = Field(default_factory=list)
content: str = ""
skipped: bool = False
error: str = ""
summary: str = ""
push: bool | None = None
generated_at: str = ""
agenda: list[dict] = Field(default_factory=list)