mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-10-23 05:35:30 +00:00
Merge 480451696b into a6509ff479
This commit is contained in:
imrewce
GitHub
commit
10893ad898
21 changed files with 5711 additions and 270 deletions
|
|
@ -10,6 +10,10 @@ daily/<date>/interests.yaml
|
|||
A host agent can use it to learn "what is worth proactive attention today," then decide whether to remind the user, ask
|
||||
a follow-up question, recommend a next step, or produce a proactive insight.
|
||||
|
||||
Every topic and agenda item carries a `kind`, one of exactly four values — `reminder`, `follow_up`, `new_idea`, and
|
||||
`interest_extend` (see "Proposal Kinds" below). All four stay at the level of a proposal: the refresh pipeline never
|
||||
carries any of them out, it only records what is worth raising with the user.
|
||||
|
||||
`interests.yaml` is generated by the proactive refresh pipeline, scheduled by `proactive_refresh_cron` by default.
|
||||
`proactive_read` only reads and exposes the result; Auto Dream is an independent daily-to-digest flow and does not read
|
||||
or write proactive state.
|
||||
|
|
@ -33,13 +37,34 @@ proactive_refresh:
|
|||
llm_timeout_seconds: 300
|
||||
max_chars_per_file: 60000
|
||||
max_total_chars: 300000
|
||||
profile_max_chars: 1500
|
||||
capability_max_chars: 1200
|
||||
extends_enabled: true
|
||||
reminders_enabled: true
|
||||
new_ideas_enabled: true
|
||||
max_follow_ups: 5
|
||||
max_extends: 5
|
||||
max_new_ideas: 3
|
||||
reminder_lead_days: 3
|
||||
resources_enabled: true
|
||||
resource_max_chars: 40000
|
||||
max_notice_reminders: 3
|
||||
refine_enabled: true
|
||||
refine_timeout_seconds: 150
|
||||
- backend: proactive_topics_step
|
||||
known_threshold: 0.85
|
||||
known_threshold_calibrated_for: text-embedding-v4@1024
|
||||
min_push_confidence: 0.5
|
||||
min_reminder_confidence: 0.3
|
||||
max_topics: 10
|
||||
schedule_retention_days: 180
|
||||
max_schedules: 80
|
||||
max_schedule_history: 8
|
||||
- backend: proactive_plan_step
|
||||
max_reminder_cards: 8
|
||||
- backend: proactive_agenda_step
|
||||
min_discovery_slots: 2
|
||||
redelivery_cooldown_days: 3
|
||||
- backend: proactive_finish_step
|
||||
file_catalog: proactive
|
||||
|
||||
|
|
@ -70,9 +95,16 @@ proactive_read:
|
|||
min_confidence:
|
||||
type: number
|
||||
default: 0.4
|
||||
include_delivered:
|
||||
type: boolean
|
||||
default: false
|
||||
delivery_cooldown_days:
|
||||
type: integer
|
||||
default: 3
|
||||
steps:
|
||||
- backend: proactive_step
|
||||
min_confidence: 0.4
|
||||
delivery_cooldown_days: 3
|
||||
```
|
||||
|
||||
Parameters:
|
||||
|
|
@ -83,22 +115,50 @@ Parameters:
|
|||
| `include_content` | Whether to return the raw YAML in the answer and metadata. Defaults to `true`. |
|
||||
| `horizon_days` | Read one day's exposure file, or use the truth source for a wider evidence horizon. Defaults to `1`. |
|
||||
| `min_confidence` | Minimum topic confidence to return. Defaults to `0.4`; legacy topics use `0.5`. |
|
||||
| `include_delivered` | Also return agenda items the host already reported as delivered. Defaults to `false` (withhold). |
|
||||
| `delivery_cooldown_days` | Days an already-delivered item stays withheld; `0` withholds same-day deliveries only. Defaults to `3`. |
|
||||
|
||||
### Refresh cost, files, and opt-out
|
||||
|
||||
When no daily Markdown note changed, refresh exits before calling an LLM and does not create a new exposure file. With
|
||||
changed material, extraction normally makes one LLM call and may retry once after an unusable reply. If push candidates
|
||||
remain, planning makes one additional call; agenda generation makes one more when there are multiple candidates. A
|
||||
refresh therefore makes at most four LLM calls with the default chain.
|
||||
With no daily Markdown change, no new external notice, and no due schedule, refresh exits before calling an LLM and
|
||||
does not create a new exposure file. When only the clock produced due reminders the chain still runs to completion with
|
||||
zero LLM calls (`reminder_only=true`), so today's agenda gets its reminders. With changed material, stage-1 recall makes
|
||||
one LLM call (at most one retry after an unusable reply) and stage 2 then makes one small call per kind that actually
|
||||
produced candidates (at most four). If push candidates remain, planning makes one additional call; agenda generation
|
||||
makes one more when there are multiple candidates. A refresh therefore makes at most nine LLM calls with the default
|
||||
chain, and only stage 1 carries the material blob.
|
||||
|
||||
The refresh pipeline maintains the rebuildable `daily/_proactive.yaml` truth source, writes
|
||||
`daily/<date>/interests.yaml`, and advances the independent `proactive` file catalog. Auto Dream does not read or write
|
||||
any of those proactive artifacts.
|
||||
The refresh pipeline maintains the rebuildable `daily/_proactive.yaml` truth source (open topics, resolved tombstones,
|
||||
the long-lived `schedules` registry, and the `deliveries` ledger), writes `daily/<date>/interests.yaml`, and advances
|
||||
the independent `proactive` file catalog. Auto Dream does not read or write any of those proactive artifacts.
|
||||
|
||||
To disable automatic refresh, use an explicit application config that omits the `proactive_refresh_cron` job. Keep the
|
||||
local-only `proactive_refresh` job if you still want on-demand maintenance. Because it has `enable_serve: false`, it is
|
||||
not exposed through HTTP or MCP.
|
||||
|
||||
## Proposal Kinds
|
||||
|
||||
Extraction makes a single recall call and asks for one sectioned reply covering four kinds. Each has a distinct job and
|
||||
each is optional: a round may produce any subset of them, including none.
|
||||
|
||||
| Kind | Job | Scenario card | Per-round cap |
|
||||
|-------------------|----------------------------------------------------------------------------------|----------------------------------|--------------------------------------------------------------|
|
||||
| `reminder` | The user's own plans and appointments, plus notices and key events from external sources that the user must be told about | `prepare_upcoming` | Uncapped; undated notices are bounded by `max_notice_reminders` (3) |
|
||||
| `follow_up` | Recent progress on something the user assigned but never closed, or keeps tracking | `resume_task` / `answer_pending` | `max_follow_ups` (5) |
|
||||
| `new_idea` | Work the agent can delegate to the user, based on its own resources (data, skills, tools, APIs) | `propose_delegation` | `max_new_ideas` (3) |
|
||||
| `interest_extend` | Refinements and extensions of the topics the user cares about (growing the wiki network) | `explore_interest` | `max_extends` (5) |
|
||||
|
||||
`reminder` outranks the other three: the agenda step **pins** due schedules to the top of today's agenda (nearest due
|
||||
first), where they are never suppressed and never swapped out, while discovery items always keep their
|
||||
`min_discovery_slots` (2) slots — so a fully booked day does not collapse into a plain calendar. Reminder candidates go
|
||||
through a lower gate in the topics step (`min_reminder_confidence` 0.3) and are exempt from `min_confidence` in
|
||||
`proactive_read`.
|
||||
|
||||
Reminders are computed against the real clock (the configured application timezone) rather than from today's daily note
|
||||
alone: before any LLM decision, extraction scans the `schedules` registry and deterministically seeds the `overdue`,
|
||||
`due_today`, `conflict`, and `due_soon` (within `reminder_lead_days`) triggers. External notices under
|
||||
`resource/<date>/` travel the same clock path, so a new external source needs no change to the proactive chain.
|
||||
|
||||
## Input Contract
|
||||
|
||||
A current proactive-refresh file looks like this:
|
||||
|
|
@ -152,6 +212,7 @@ metadata:
|
|||
| `error` | Read or parse error. |
|
||||
| `summary` | Short summary. |
|
||||
| `agenda` | Today's proactive agenda (optional, v2 files only). |
|
||||
| `delivered_ids` | Agenda ids withheld this round by the delivery cooldown (when `include_delivered=false`). |
|
||||
|
||||
When today's `interests.yaml` was produced by the proactive refresh chain with an agenda,
|
||||
the answer also carries an `agenda` field: the ordered agenda items, each with `topic_id`,
|
||||
|
|
@ -206,6 +267,35 @@ Skipped: interests file not found at daily/2026-06-20/interests.yaml
|
|||
|
||||
This lets a host agent treat "there is no proactive refresh result for today yet" as a normal empty state.
|
||||
|
||||
## Delivery Receipts
|
||||
|
||||
Refresh only decides what is worth raising; it cannot know what the host actually raised. `proactive_feedback` closes
|
||||
that loop so an item the consumer already showed is not proposed again tomorrow:
|
||||
|
||||
```bash
|
||||
reme proactive_feedback date=2026-06-22 items='[{"topic_id":"<id>","status":"shown","channel":"inbox","consumer":"<host label>"}]'
|
||||
```
|
||||
|
||||
Each receipt accepts `topic_id` (the only required field), `status`, `channel`, `consumer`, `outcome`, `external_ref`,
|
||||
and `agenda_date`. `status` is one of `shown` (default), `executed`, `dismissed`, `failed`; `channel` is one of
|
||||
`in_conversation`, `inbox`, `notification`, `other` (default). Unrecognized values fall back to the default instead of
|
||||
failing the call. Receipts are stored in the `deliveries` section of the `daily/_proactive.yaml` truth source and
|
||||
trimmed by `retention_days` (180) and `max_deliveries` (500).
|
||||
|
||||
- **Identity**: `external_ref` wins when the host supplies one, otherwise `(topic_id, agenda_date, channel)`. Reporting
|
||||
the same delivery twice updates the record in place, so a host can report `shown` first and `executed` later.
|
||||
- **Status only moves forward**: a terminal record (`executed`, `dismissed`) is never downgraded by a late `shown`, and
|
||||
`failed` does not block redelivery in the next round.
|
||||
- **Two readers**: `proactive_step` withholds agenda items still inside `delivery_cooldown_days` (3), and
|
||||
`proactive_agenda_step` deterministically suppresses those candidates and writes an auditable reason into
|
||||
`suppressed`. `kind == "reminder"` is exempt from the cooldown on both sides — its `topic_id` derives from a stable
|
||||
schedule title, so normal suppression would silence a recurring reminder for the whole window after one delivery.
|
||||
- **Optional contract**: a host that never reports leaves `deliveries` empty and both readers behave exactly as before.
|
||||
The step makes no LLM call and does not touch `open_topics`, `resolved`, or `schedules`.
|
||||
|
||||
Readers return the items they withheld as `delivered_ids`, in both the `answer` and the metadata, so a consumer can
|
||||
reconcile them against its own local ledger.
|
||||
|
||||
## Running Proactive
|
||||
|
||||
### Start the Service and Read Topics
|
||||
|
|
|
|||
|
|
@ -9,6 +9,9 @@ daily/<date>/interests.yaml
|
|||
|
||||
上层 Agent 可以用它获取“今天值得主动关注什么”，再决定是否提醒、追问、推荐下一步或生成主动洞察。
|
||||
|
||||
每个 topic 和议程条目都带有 `kind`，取值固定为四类之一——`reminder`、`follow_up`、`new_idea`、`interest_extend`
|
||||
（见下文“提议类型”）。四类都停留在“提议”：refresh 链路不会执行其中任何一件，只记录“值得对用户提起什么”。
|
||||
|
||||
`interests.yaml` 由 proactive refresh writer 链路生成，默认通过 `proactive_refresh_cron` 定时执行；
|
||||
`proactive_read` 只负责读取和暴露结果。Auto Dream 是独立的 daily-to-digest 流程，不读取或写入 proactive 状态。
|
||||
|
||||
|
|
@ -30,13 +33,34 @@ proactive_refresh:
|
|||
llm_timeout_seconds: 300
|
||||
max_chars_per_file: 60000
|
||||
max_total_chars: 300000
|
||||
profile_max_chars: 1500
|
||||
capability_max_chars: 1200
|
||||
extends_enabled: true
|
||||
reminders_enabled: true
|
||||
new_ideas_enabled: true
|
||||
max_follow_ups: 5
|
||||
max_extends: 5
|
||||
max_new_ideas: 3
|
||||
reminder_lead_days: 3
|
||||
resources_enabled: true
|
||||
resource_max_chars: 40000
|
||||
max_notice_reminders: 3
|
||||
refine_enabled: true
|
||||
refine_timeout_seconds: 150
|
||||
- backend: proactive_topics_step
|
||||
known_threshold: 0.85
|
||||
known_threshold_calibrated_for: text-embedding-v4@1024
|
||||
min_push_confidence: 0.5
|
||||
min_reminder_confidence: 0.3
|
||||
max_topics: 10
|
||||
schedule_retention_days: 180
|
||||
max_schedules: 80
|
||||
max_schedule_history: 8
|
||||
- backend: proactive_plan_step
|
||||
max_reminder_cards: 8
|
||||
- backend: proactive_agenda_step
|
||||
min_discovery_slots: 2
|
||||
redelivery_cooldown_days: 3
|
||||
- backend: proactive_finish_step
|
||||
file_catalog: proactive
|
||||
|
||||
|
|
@ -67,9 +91,16 @@ proactive_read:
|
|||
min_confidence:
|
||||
type: number
|
||||
default: 0.4
|
||||
include_delivered:
|
||||
type: boolean
|
||||
default: false
|
||||
delivery_cooldown_days:
|
||||
type: integer
|
||||
default: 3
|
||||
steps:
|
||||
- backend: proactive_step
|
||||
min_confidence: 0.4
|
||||
delivery_cooldown_days: 3
|
||||
```
|
||||
|
||||
参数含义：
|
||||
|
|
@ -80,19 +111,44 @@ proactive_read:
|
|||
| `include_content` | 是否在 answer 和 metadata 中返回 YAML 原文，默认 `true`。 |
|
||||
| `horizon_days` | 读取单日曝光文件，或在更宽时间窗下读取 truth source；默认 `1`。 |
|
||||
| `min_confidence` | 返回 topic 的最低置信度，默认 `0.4`；旧版 topic 使用 `0.5`。 |
|
||||
| `include_delivered` | 是否连宿主已上报送达的议程条目一起返回；默认 `false`（扣留）。 |
|
||||
| `delivery_cooldown_days` | 已送达条目的扣留天数，`0` 表示只扣留当天的送达；默认 `3`。 |
|
||||
|
||||
### Refresh 成本、文件与关闭方式
|
||||
|
||||
没有 daily Markdown 发生变化时，refresh 会在调用 LLM 前结束，也不会生成新的曝光文件。有变化素材时，extract
|
||||
通常调用一次 LLM；回复不可用时最多重试一次。存在 push candidates 时，plan 会再调用一次；候选数大于一个时，
|
||||
agenda 会再调用一次。因此默认链路单轮最多调用四次 LLM。
|
||||
没有 daily Markdown 变化、没有新的外部通知、且没有到期日程时，refresh 会在调用 LLM 前结束，也不会生成新的曝光
|
||||
文件。只有时钟算出到期提醒时，链路仍会零 LLM 调用地走完（`reminder_only=true`），保证当日议程拿到提醒。有变化
|
||||
素材时，阶段 1 召回调用一次 LLM（回复不可用时最多重试一次），阶段 2 再对每个真正产出了候选的类型各调用一次小请求
|
||||
（至多四次）；存在 push candidates 时 plan 会再调用一次，候选数大于一个时 agenda 会再调用一次。因此默认链路单轮
|
||||
最多九次 LLM 调用，其中只有阶段 1 携带素材 blob。
|
||||
|
||||
Refresh 会维护可重建的 truth source `daily/_proactive.yaml`，写入 `daily/<date>/interests.yaml`，并推进独立的
|
||||
Refresh 会维护可重建的 truth source `daily/_proactive.yaml`（open topics、resolved 墓碑，以及长生命周期的
|
||||
`schedules` 日程注册表与 `deliveries` 投递记录），写入 `daily/<date>/interests.yaml`，并推进独立的
|
||||
`proactive` file catalog。Auto Dream 不读取或写入这些 proactive 产物。
|
||||
|
||||
如需关闭自动 refresh，请使用不包含 `proactive_refresh_cron` job 的显式应用配置。若仍需按需维护，可保留
|
||||
本地 one-shot 的 `proactive_refresh`；它设置了 `enable_serve: false`，不会暴露到 HTTP 或 MCP。
|
||||
|
||||
## 提议类型（Proposal Kinds）
|
||||
|
||||
extract 只发起一次召回调用，要求模型返回四个分节的提议。每类职责不同，且都不是必需项：单轮可以产出其中任意子集，
|
||||
也可以一条都没有。
|
||||
|
||||
| 类型 | 职责 | 对应场景卡 | 单轮上限 |
|
||||
|-------------------|-----------------------------------------------------------------|----------------------------------|------------------------------------------------------|
|
||||
| `reminder` | 用户自己的计划与日程，以及外部信源中必须告知用户的通知与关键事件 | `prepare_upcoming` | 不设上限；无日期通知受 `max_notice_reminders`（3）约束 |
|
||||
| `follow_up` | 用户指派但未闭环、或持续关心事项的最近进展 | `resume_task` / `answer_pending` | `max_follow_ups`（5） |
|
||||
| `new_idea` | 基于 agent 自身资源（数据、skill、工具与 API）可委托给用户的工作 | `propose_delegation` | `max_new_ideas`（3） |
|
||||
| `interest_extend` | 用户关心主题的细化与延伸内容（扩展 wiki 网络节点） | `explore_interest` | `max_extends`（5） |
|
||||
|
||||
`reminder` 优先级高于其余三类：agenda 步骤把到期日程**固定**在当日议程最前（按到期时间由近到远），既不会被压制也不会
|
||||
被交换，发现类条目则始终保有 `min_discovery_slots`（2）个名额，因此日程排满的一天也不会退化成一张纯日历。提醒候选在
|
||||
topics 步骤走更低门槛（`min_reminder_confidence` 0.3），并在 `proactive_read` 中豁免 `min_confidence`。
|
||||
|
||||
提醒是对照真实时钟（应用配置时区）算出来的，而不是只看当天的 daily note：extract 在任何 LLM 决策之前先扫描
|
||||
`schedules` 注册表，确定性地预置 `overdue`、`due_today`、`conflict` 与 `due_soon`（`reminder_lead_days` 之内）四类
|
||||
触发。`resource/<date>/` 下的外部通知走同一条时钟通道，因此接入新的外部信源不需要改 proactive 链路。
|
||||
|
||||
## 输入契约
|
||||
|
||||
当前 proactive refresh 生成的文件如下：
|
||||
|
|
@ -144,6 +200,7 @@ suppressed: []
|
|||
| `error` | 读取或解析异常。 |
|
||||
| `summary` | 简短摘要。 |
|
||||
| `agenda` | 当日主动议程（可选，仅 v2 文件携带时返回）。 |
|
||||
| `delivered_ids` | 本轮因送达冷却被扣留的议程 id（`include_delivered=false` 时）。 |
|
||||
|
||||
当当天的 `interests.yaml` 由 proactive refresh 链路生成并携带议程时，answer 会附带
|
||||
`agenda` 字段：按顺序排列的当日议程条目，每条包含 `topic_id`、`title`、`scenario_type`、
|
||||
|
|
@ -197,6 +254,31 @@ Skipped: interests file not found at daily/2026-06-20/interests.yaml
|
|||
|
||||
这让上层 Agent 可以把“今天还没有 proactive refresh 结果”当作正常空状态处理。
|
||||
|
||||
## 送达回执（Delivery Receipts）
|
||||
|
||||
refresh 只决定“什么值得提”，它无法知道宿主究竟提了什么。`proactive_feedback` 用来闭合这个环，避免消费方已经展示
|
||||
过的条目明天又被提议一遍：
|
||||
|
||||
```bash
|
||||
reme proactive_feedback date=2026-06-22 items='[{"topic_id":"<id>","status":"shown","channel":"inbox","consumer":"<host label>"}]'
|
||||
```
|
||||
|
||||
每条回执接受 `topic_id`（唯一必填）、`status`、`channel`、`consumer`、`outcome`、`external_ref` 与 `agenda_date`；
|
||||
`status` 取 `shown`（默认）、`executed`、`dismissed`、`failed`，`channel` 取 `in_conversation`、`inbox`、
|
||||
`notification`、`other`（默认）。无法识别的取值回落到默认值而不是让调用失败。回执写入 truth source
|
||||
`daily/_proactive.yaml` 的 `deliveries` 段，并按 `retention_days`（180）与 `max_deliveries`（500）裁剪。
|
||||
|
||||
- **身份**：宿主给了 `external_ref` 时以它为准，否则用 `(topic_id, agenda_date, channel)`；同一次送达重复上报会原地
|
||||
更新，因此可以先报 `shown`、之后再报 `executed`，不会产生第二条。
|
||||
- **状态只前进**：已终结的记录（`executed`、`dismissed`）不会被迟到的 `shown` 降级回去；`failed` 不阻止下一轮重新投递。
|
||||
- **两个读取方**：`proactive_step` 扣留仍在 `delivery_cooldown_days`（3）内的议程条目，`proactive_agenda_step` 则
|
||||
确定性压制这些候选并把可审计的理由写进 `suppressed`。`kind == "reminder"` 在两侧都豁免冷却——它的 `topic_id` 由
|
||||
稳定的日程标题派生，照常压制会让一条周期性提醒在投递一次之后就在整个窗口内静默。
|
||||
- **可选契约**：从不上报的宿主会让 `deliveries` 保持为空，两个读取方的行为与之前完全一致。该步骤不调用 LLM，也不改
|
||||
`open_topics`/`resolved`/`schedules`。
|
||||
|
||||
读取方把自己扣留的条目以 `delivered_ids` 返回（`answer` 与 `metadata` 里都有），消费方用它与自己的本地台账对账。
|
||||
|
||||
## 运行方式
|
||||
|
||||
### 启动服务并读取 topics
|
||||
|
|
|
|||
|
|
@ -100,13 +100,34 @@ jobs:
|
|||
llm_timeout_seconds: 300
|
||||
max_chars_per_file: 60000
|
||||
max_total_chars: 300000
|
||||
profile_max_chars: 1500
|
||||
capability_max_chars: 1200
|
||||
extends_enabled: true
|
||||
reminders_enabled: true
|
||||
new_ideas_enabled: true
|
||||
max_follow_ups: 5
|
||||
max_extends: 5
|
||||
max_new_ideas: 3
|
||||
reminder_lead_days: 3
|
||||
resources_enabled: true
|
||||
resource_max_chars: 40000
|
||||
max_notice_reminders: 3
|
||||
refine_enabled: true
|
||||
refine_timeout_seconds: 150
|
||||
- backend: proactive_topics_step
|
||||
known_threshold: 0.85
|
||||
known_threshold_calibrated_for: text-embedding-v4@1024
|
||||
min_push_confidence: 0.5
|
||||
min_reminder_confidence: 0.3
|
||||
max_topics: 10
|
||||
schedule_retention_days: 180
|
||||
max_schedules: 80
|
||||
max_schedule_history: 8
|
||||
- backend: proactive_plan_step
|
||||
max_reminder_cards: 8
|
||||
- backend: proactive_agenda_step
|
||||
min_discovery_slots: 2
|
||||
redelivery_cooldown_days: 3
|
||||
- backend: proactive_finish_step
|
||||
file_catalog: proactive
|
||||
|
||||
|
|
@ -122,13 +143,34 @@ jobs:
|
|||
llm_timeout_seconds: 300
|
||||
max_chars_per_file: 60000
|
||||
max_total_chars: 300000
|
||||
profile_max_chars: 1500
|
||||
capability_max_chars: 1200
|
||||
extends_enabled: true
|
||||
reminders_enabled: true
|
||||
new_ideas_enabled: true
|
||||
max_follow_ups: 5
|
||||
max_extends: 5
|
||||
max_new_ideas: 3
|
||||
reminder_lead_days: 3
|
||||
resources_enabled: true
|
||||
resource_max_chars: 40000
|
||||
max_notice_reminders: 3
|
||||
refine_enabled: true
|
||||
refine_timeout_seconds: 150
|
||||
- backend: proactive_topics_step
|
||||
known_threshold: 0.85
|
||||
known_threshold_calibrated_for: text-embedding-v4@1024
|
||||
min_push_confidence: 0.5
|
||||
min_reminder_confidence: 0.3
|
||||
max_topics: 10
|
||||
schedule_retention_days: 180
|
||||
max_schedules: 80
|
||||
max_schedule_history: 8
|
||||
- backend: proactive_plan_step
|
||||
max_reminder_cards: 8
|
||||
- backend: proactive_agenda_step
|
||||
min_discovery_slots: 2
|
||||
redelivery_cooldown_days: 3
|
||||
- backend: proactive_finish_step
|
||||
file_catalog: proactive
|
||||
|
||||
|
|
@ -267,9 +309,40 @@ jobs:
|
|||
type: number
|
||||
description: "minimum topic confidence to return (default 0.4 sits safely below the 0.5 fallback; v1 topics fall back to 0.5)"
|
||||
default: 0.4
|
||||
include_delivered:
|
||||
type: boolean
|
||||
description: "also return agenda items the host already reported as delivered (default withholds them)"
|
||||
default: false
|
||||
delivery_cooldown_days:
|
||||
type: integer
|
||||
description: "days an already-delivered agenda item stays withheld; 0 withholds only same-day deliveries"
|
||||
default: 3
|
||||
steps:
|
||||
- backend: proactive_step
|
||||
min_confidence: 0.4
|
||||
delivery_cooldown_days: 3
|
||||
|
||||
proactive_feedback:
|
||||
backend: base
|
||||
description: "Record which proactive agenda items a host agent already delivered or executed, so they are not repeated."
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
date:
|
||||
type: string
|
||||
description: "YYYY-MM-DD the report belongs to; defaults to today in the application's timezone"
|
||||
default: ""
|
||||
items:
|
||||
type: array
|
||||
description: "delivery reports: [{topic_id, status, channel, consumer, outcome, external_ref, agenda_date}]"
|
||||
items:
|
||||
type: object
|
||||
required:
|
||||
- items
|
||||
steps:
|
||||
- backend: proactive_feedback_step
|
||||
retention_days: 180
|
||||
max_deliveries: 500
|
||||
|
||||
version:
|
||||
backend: base
|
||||
|
|
|
|||
|
|
@ -8,7 +8,9 @@ from .dream import (
|
|||
IntegrateOutcome,
|
||||
)
|
||||
from .proactive import (
|
||||
ProactiveDelivery,
|
||||
ProactiveResult,
|
||||
ProactiveSchedule,
|
||||
ProactiveState,
|
||||
ProactiveStateFile,
|
||||
ProactiveTopic,
|
||||
|
|
@ -41,7 +43,9 @@ __all__ = [
|
|||
"GraphSnapshotNode",
|
||||
"IntegrateOutcome",
|
||||
"JobConfig",
|
||||
"ProactiveDelivery",
|
||||
"ProactiveResult",
|
||||
"ProactiveSchedule",
|
||||
"ProactiveState",
|
||||
"ProactiveStateFile",
|
||||
"ProactiveTopic",
|
||||
|
|
|
|||
|
|
@ -6,9 +6,44 @@ validated structurally by ``parse_extract_reply`` instead of a model here.
|
|||
See ``PROACTIVE_SPEC.md`` sections F1/A2 for the full contracts.
|
||||
"""
|
||||
|
||||
import datetime as dt
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
TOPIC_KINDS = ("follow_up", "interest_extend")
|
||||
# Topic kinds, ordered by proactive priority: ``reminder`` outranks the rest
|
||||
# and pins today's agenda, ``interest_extend`` is the lowest-priority
|
||||
# discovery kind. ``KIND_PRIORITY`` is the single ordering source used by
|
||||
# ``sort_topics`` and by the plan/agenda steps.
|
||||
TOPIC_KINDS = ("reminder", "follow_up", "new_idea", "interest_extend")
|
||||
KIND_PRIORITY = {kind: index for index, kind in enumerate(TOPIC_KINDS)}
|
||||
DEFAULT_KIND = "interest_extend"
|
||||
|
||||
# Reminder registry (``daily/_proactive.yaml: schedules``) vocabularies. The
|
||||
# registry outlives ``open_topics`` because a schedule mentioned weeks ago
|
||||
# still has to fire on its due date.
|
||||
SCHEDULE_ACTIONS = ("add", "reschedule", "confirm", "done", "cancel")
|
||||
SCHEDULE_STATUSES = ("open", "done", "cancelled")
|
||||
RECURRENCES = ("", "daily", "weekly", "monthly", "yearly")
|
||||
REMINDER_TRIGGERS = ("overdue", "due_today", "due_soon", "conflict")
|
||||
|
||||
# Delivery feedback (``daily/_proactive.yaml: deliveries``) vocabularies.
|
||||
# A host agent reports back which agenda items it already surfaced or ran, so
|
||||
# the reader and the agenda step can stop repeating them. Reporting is
|
||||
# optional: a framework that never calls ``proactive_feedback`` leaves the
|
||||
# section empty and the whole chain behaves exactly as before.
|
||||
DELIVERY_STATUSES = ("shown", "executed", "dismissed", "failed")
|
||||
DELIVERY_CHANNELS = ("in_conversation", "inbox", "notification", "other")
|
||||
DEFAULT_DELIVERY_STATUS = "shown"
|
||||
DEFAULT_DELIVERY_CHANNEL = "other"
|
||||
# Statuses that retire an agenda item for the whole cooldown window. ``failed``
|
||||
# is deliberately absent: a failed run should be retryable on the next round.
|
||||
TERMINAL_DELIVERY_STATUSES = ("executed", "dismissed")
|
||||
# ``outcome`` is a host-written summary; cap it so a chatty consumer cannot
|
||||
# inflate the truth source without bound.
|
||||
MAX_DELIVERY_OUTCOME_CHARS = 500
|
||||
|
||||
# Keys inside the untyped ``history`` audit trail that hold a date/datetime.
|
||||
HISTORY_WHEN_KEYS = ("at", "from", "to")
|
||||
|
||||
|
||||
def clamp_confidence(value) -> float:
|
||||
|
|
@ -19,6 +54,29 @@ def clamp_confidence(value) -> float:
|
|||
return 0.5
|
||||
|
||||
|
||||
def iso_text(value) -> str:
|
||||
"""Coerce a YAML-native scalar into the ISO text the string fields expect.
|
||||
|
||||
``yaml.safe_load`` reads an unquoted ``2026-10-11`` as ``datetime.date`` and
|
||||
``2026-10-11T20:00`` as ``datetime.datetime``. The truth source is a
|
||||
hand-editable file and the schedule registry is meant to outlive topics by
|
||||
months, so a bare date must never invalidate an entry: date and time
|
||||
scalars are rendered back to ISO text, anything else falls back to ``str``.
|
||||
"""
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, (dt.datetime, dt.date, dt.time)):
|
||||
return value.isoformat()
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
return str(value)
|
||||
|
||||
|
||||
def kind_priority(kind) -> int:
|
||||
"""Priority rank of a topic kind; unknown kinds sort last."""
|
||||
return KIND_PRIORITY.get(str(kind or "").strip(), len(TOPIC_KINDS))
|
||||
|
||||
|
||||
class ProactiveTopic(BaseModel):
|
||||
"""One proactive topic; every field has a default so v1 files parse seamlessly.
|
||||
|
||||
|
|
@ -31,24 +89,33 @@ class ProactiveTopic(BaseModel):
|
|||
id: str = ""
|
||||
title: str = ""
|
||||
reason: str = ""
|
||||
kind: str = "interest_extend"
|
||||
kind: str = DEFAULT_KIND
|
||||
confidence: float = 0.5
|
||||
first_seen: str = ""
|
||||
last_evidence_at: str = ""
|
||||
evidence: str = ""
|
||||
paths: list[str] = Field(default_factory=list)
|
||||
# Reminder-only links: ``schedule_id`` points at the long-lived registry
|
||||
# entry, ``due_at`` is the ISO date/datetime the reminder is about.
|
||||
schedule_id: str = ""
|
||||
due_at: str = ""
|
||||
|
||||
@field_validator("kind", mode="before")
|
||||
@classmethod
|
||||
def _fallback_kind(cls, value):
|
||||
text = str(value or "").strip()
|
||||
return text if text in TOPIC_KINDS else "interest_extend"
|
||||
return text if text in TOPIC_KINDS else DEFAULT_KIND
|
||||
|
||||
@field_validator("confidence", mode="before")
|
||||
@classmethod
|
||||
def _fallback_confidence(cls, value):
|
||||
return clamp_confidence(value)
|
||||
|
||||
@field_validator("first_seen", "last_evidence_at", "due_at", mode="before")
|
||||
@classmethod
|
||||
def _iso_when(cls, value):
|
||||
return iso_text(value)
|
||||
|
||||
@field_validator("paths", mode="before")
|
||||
@classmethod
|
||||
def _clean_str_list(cls, value):
|
||||
|
|
@ -57,6 +124,137 @@ class ProactiveTopic(BaseModel):
|
|||
return [str(item).strip() for item in value if str(item).strip()]
|
||||
|
||||
|
||||
class ProactiveSchedule(BaseModel):
|
||||
"""One personal schedule/appointment in the long-lived reminder registry.
|
||||
|
||||
The registry is a separate truth-source section from ``open_topics``: a
|
||||
schedule survives topic trimming (``carry_forward_days``) until its due
|
||||
date has passed by ``schedule_retention_days``, and every change the
|
||||
pipeline observes is appended to ``history`` so a reschedule or a
|
||||
cancellation stays auditable long after the originating daily note left
|
||||
the extract scan window.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
id: str = ""
|
||||
title: str = ""
|
||||
detail: str = ""
|
||||
due_at: str = ""
|
||||
due_end_at: str = ""
|
||||
recurrence: str = ""
|
||||
# 0 = "not specified": the reminder scan then applies the step-level
|
||||
# ``reminder_lead_days`` default instead of a per-schedule value.
|
||||
lead_days: int = 0
|
||||
status: str = "open"
|
||||
confidence: float = 0.5
|
||||
first_seen: str = ""
|
||||
last_seen_at: str = ""
|
||||
evidence: str = ""
|
||||
paths: list[str] = Field(default_factory=list)
|
||||
history: list[dict] = Field(default_factory=list)
|
||||
|
||||
@field_validator("status", mode="before")
|
||||
@classmethod
|
||||
def _fallback_status(cls, value):
|
||||
text = str(value or "").strip()
|
||||
return text if text in SCHEDULE_STATUSES else "open"
|
||||
|
||||
@field_validator("recurrence", mode="before")
|
||||
@classmethod
|
||||
def _fallback_recurrence(cls, value):
|
||||
text = str(value or "").strip().lower()
|
||||
return text if text in RECURRENCES else ""
|
||||
|
||||
@field_validator("due_at", "due_end_at", "first_seen", "last_seen_at", mode="before")
|
||||
@classmethod
|
||||
def _iso_when(cls, value):
|
||||
return iso_text(value)
|
||||
|
||||
@field_validator("confidence", mode="before")
|
||||
@classmethod
|
||||
def _fallback_confidence(cls, value):
|
||||
return clamp_confidence(value)
|
||||
|
||||
@field_validator("lead_days", mode="before")
|
||||
@classmethod
|
||||
def _fallback_lead_days(cls, value):
|
||||
try:
|
||||
return max(0, min(int(value), 365))
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
@field_validator("paths", mode="before")
|
||||
@classmethod
|
||||
def _clean_str_list(cls, value):
|
||||
if not isinstance(value, list):
|
||||
return []
|
||||
return [str(item).strip() for item in value if str(item).strip()]
|
||||
|
||||
@field_validator("history", mode="before")
|
||||
@classmethod
|
||||
def _clean_history(cls, value):
|
||||
if not isinstance(value, list):
|
||||
return []
|
||||
cleaned = []
|
||||
for item in value:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
cleaned.append({k: iso_text(v) if k in HISTORY_WHEN_KEYS else v for k, v in item.items()})
|
||||
return cleaned
|
||||
|
||||
|
||||
class ProactiveDelivery(BaseModel):
|
||||
"""One host-side report that an agenda item was surfaced or executed.
|
||||
|
||||
The identity of a record is ``(topic_id, external_ref)`` when the host
|
||||
supplies a reference (an inbox event id, a session id), and
|
||||
``(topic_id, agenda_date, channel)`` otherwise, so repeated reports of the
|
||||
same delivery update in place instead of piling up. ``status`` only moves
|
||||
forward: ``shown`` may become ``executed``/``dismissed``/``failed``, but a
|
||||
terminal record is never downgraded back to ``shown``.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
topic_id: str = ""
|
||||
agenda_date: str = ""
|
||||
delivered_at: str = ""
|
||||
channel: str = DEFAULT_DELIVERY_CHANNEL
|
||||
consumer: str = ""
|
||||
status: str = DEFAULT_DELIVERY_STATUS
|
||||
outcome: str = ""
|
||||
external_ref: str = ""
|
||||
|
||||
@field_validator("status", mode="before")
|
||||
@classmethod
|
||||
def _fallback_status(cls, value):
|
||||
text = str(value or "").strip().lower()
|
||||
return text if text in DELIVERY_STATUSES else DEFAULT_DELIVERY_STATUS
|
||||
|
||||
@field_validator("channel", mode="before")
|
||||
@classmethod
|
||||
def _fallback_channel(cls, value):
|
||||
text = str(value or "").strip().lower()
|
||||
return text if text in DELIVERY_CHANNELS else DEFAULT_DELIVERY_CHANNEL
|
||||
|
||||
@field_validator("agenda_date", "delivered_at", mode="before")
|
||||
@classmethod
|
||||
def _iso_when(cls, value):
|
||||
return iso_text(value)
|
||||
|
||||
@field_validator("outcome", mode="before")
|
||||
@classmethod
|
||||
def _truncate_outcome(cls, value):
|
||||
return str(value or "")[:MAX_DELIVERY_OUTCOME_CHARS]
|
||||
|
||||
def identity(self) -> tuple[str, str]:
|
||||
"""Idempotency key: explicit host reference wins over the natural key."""
|
||||
if self.external_ref:
|
||||
return (self.topic_id, f"ref:{self.external_ref}")
|
||||
return (self.topic_id, f"day:{self.agenda_date}|{self.channel}")
|
||||
|
||||
|
||||
class ProactiveState(BaseModel):
|
||||
"""Chain-shared proactive context state (``context['proactive']``).
|
||||
|
||||
|
|
@ -77,25 +275,56 @@ class ProactiveState(BaseModel):
|
|||
carry_forward_days: int = 14
|
||||
changed_paths: list[str] = Field(default_factory=list)
|
||||
changed_mtimes: dict[str, float] = Field(default_factory=dict)
|
||||
# External source records (``resource/``) scanned for user-facing notices.
|
||||
# They feed the reminder branch only, but share ``changed_mtimes`` and the
|
||||
# proactive catalog watermark with the daily material set, so a notice that
|
||||
# lands on a day without any conversation still triggers a round.
|
||||
resource_scanned: int = 0
|
||||
resource_skipped: int = 0
|
||||
resource_changed: list[str] = Field(default_factory=list)
|
||||
carry_forward_count: int = 0
|
||||
carry_forward_prompt: list[ProactiveTopic] = Field(default_factory=list)
|
||||
llm_calls: int = 0
|
||||
follow_ups: list[dict] = Field(default_factory=list)
|
||||
extends: list[dict] = Field(default_factory=list)
|
||||
reminders: list[dict] = Field(default_factory=list)
|
||||
new_ideas: list[dict] = Field(default_factory=list)
|
||||
updates: list[dict] = Field(default_factory=list)
|
||||
schedule_updates: list[dict] = Field(default_factory=list)
|
||||
# Stage-2 adjudication (precision pass over stage-1 candidates). Every
|
||||
# failure is fail-open, so ``refine_degraded`` counts the kinds whose
|
||||
# stage-1 candidates were kept untouched.
|
||||
refine_calls: int = 0
|
||||
refine_degraded: int = 0
|
||||
refine_dropped: list[dict] = Field(default_factory=list)
|
||||
refine_moved: list[dict] = Field(default_factory=list)
|
||||
early_exit: str = ""
|
||||
# ``reminder_only`` marks a clock-driven round with no new material: the
|
||||
# chain stays alive (no ``early_exit``) but made zero LLM calls.
|
||||
reminder_only: bool = False
|
||||
due_schedules: list[dict] = Field(default_factory=list)
|
||||
updates_applied: int = 0
|
||||
updates_resolved: int = 0
|
||||
schedules_added: int = 0
|
||||
schedules_updated: int = 0
|
||||
schedules_closed: int = 0
|
||||
# Reminders whose published due value was re-read from the registry because
|
||||
# this round's ``schedule_updates`` moved it after the clock scan.
|
||||
reminders_reanchored: int = 0
|
||||
schedules_out: list[dict] = Field(default_factory=list)
|
||||
candidates_in: int = 0
|
||||
candidates: list[dict] = Field(default_factory=list)
|
||||
dropped_missing: int = 0
|
||||
dropped_duplicate: int = 0
|
||||
dropped_known: int = 0
|
||||
dropped_over_cap: int = 0
|
||||
topics_out: list[dict] = Field(default_factory=list)
|
||||
push_candidates: list[dict] = Field(default_factory=list)
|
||||
scenario_cards: list[dict] = Field(default_factory=list)
|
||||
agenda: list[dict] = Field(default_factory=list)
|
||||
suppressed: list[dict] = Field(default_factory=list)
|
||||
# Candidates withheld because the host already reported them delivered.
|
||||
suppressed_delivered: int = 0
|
||||
plan_llm_calls: int = 0
|
||||
push: bool = False
|
||||
file_skip_reason: str = ""
|
||||
|
|
@ -103,6 +332,11 @@ class ProactiveState(BaseModel):
|
|||
interests_written: bool = False
|
||||
checkpoint_paths: list[str] = Field(default_factory=list)
|
||||
duration_ms: int = 0
|
||||
# Delivery-feedback round counters (metadata/log only).
|
||||
deliveries_in: int = 0
|
||||
deliveries_added: int = 0
|
||||
deliveries_updated: int = 0
|
||||
deliveries_dropped: int = 0
|
||||
|
||||
|
||||
class ProactiveStateFile(BaseModel):
|
||||
|
|
@ -110,6 +344,10 @@ class ProactiveStateFile(BaseModel):
|
|||
|
||||
``resolved`` tombstones carry ``first_seen`` so a resurrected topic can
|
||||
keep its original age anchor (F2.4 reopen channel).
|
||||
``schedules`` is the long-lived reminder registry; it is optional so
|
||||
files written before the reminder branch existed still load.
|
||||
``deliveries`` is the optional host-feedback log; it is likewise absent
|
||||
from files written before delivery reporting existed.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
|
@ -117,6 +355,8 @@ class ProactiveStateFile(BaseModel):
|
|||
version: int = 1
|
||||
open_topics: list[ProactiveTopic] = Field(default_factory=list)
|
||||
resolved: list[dict] = Field(default_factory=list)
|
||||
schedules: list[ProactiveSchedule] = Field(default_factory=list)
|
||||
deliveries: list[ProactiveDelivery] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ProactiveResult(BaseModel):
|
||||
|
|
@ -132,3 +372,6 @@ class ProactiveResult(BaseModel):
|
|||
push: bool | None = None
|
||||
generated_at: str = ""
|
||||
agenda: list[dict] = Field(default_factory=list)
|
||||
# Agenda topic_ids withheld because the host already reported them as
|
||||
# delivered; kept so a caller can see the suppression instead of guessing.
|
||||
delivered_ids: list[str] = Field(default_factory=list)
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
|
||||
from .agenda import ProactiveAgendaStep
|
||||
from .extract import ProactiveExtractStep
|
||||
from .feedback import ProactiveFeedbackStep
|
||||
from .finish import ProactiveFinishStep
|
||||
from .plan import ProactivePlanStep
|
||||
from .proactive import ProactiveStep
|
||||
|
|
@ -10,6 +11,7 @@ from .topics import ProactiveTopicsStep
|
|||
__all__ = [
|
||||
"ProactiveAgendaStep",
|
||||
"ProactiveExtractStep",
|
||||
"ProactiveFeedbackStep",
|
||||
"ProactiveFinishStep",
|
||||
"ProactivePlanStep",
|
||||
"ProactiveStep",
|
||||
|
|
|
|||
|
|
@ -14,6 +14,19 @@ call; LLM absent/timed out/unusable -> freshness/confidence order fallback;
|
|||
after validation the agenda is never empty while candidates exist, so the push
|
||||
semantics computed by the topics step never flip. The step also owns the final
|
||||
interests.yaml render, extending the v2 shape with ``agenda``/``suppressed``.
|
||||
|
||||
Reminder priority is enforced here, not negotiated: every reminder card is
|
||||
pinned to the front of the agenda (nearest due date first) before the editor
|
||||
LLM sees anything, and only the slots the reminders leave are up for
|
||||
selection. Pinned items are never suppressed and never counted as "silence".
|
||||
|
||||
Delivery feedback is enforced even earlier: a candidate the host already
|
||||
reported as surfaced/executed inside ``redelivery_cooldown_days`` leaves the
|
||||
candidate set before pinning and before the editor call, with an auditable
|
||||
``already delivered ...`` suppression reason. This is the write-side twin of
|
||||
the reader's filter, so a proposal cannot reappear tomorrow merely because
|
||||
the host reported late. Hosts that never report keep an empty ``deliveries``
|
||||
section and see no change.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
|
@ -29,14 +42,51 @@ from .utils import (
|
|||
current_now,
|
||||
interests_path_for,
|
||||
load_personal_profile_block,
|
||||
load_state,
|
||||
norm_path,
|
||||
parse_agenda_reply,
|
||||
render_interests,
|
||||
resolve_agent_wrapper,
|
||||
suppressed_delivery_ids,
|
||||
write_interests_if_changed,
|
||||
)
|
||||
|
||||
FALLBACK_ORDER_REASON = "deterministic fallback: freshness/confidence order"
|
||||
PINNED_REMINDER_REASONS = {
|
||||
"due": "reminder pinned: a due schedule always enters today's agenda",
|
||||
"new": "reminder pinned: a new reminder always enters today's agenda",
|
||||
"open": "reminder pinned: an open reminder always enters today's agenda",
|
||||
}
|
||||
|
||||
|
||||
def _pin_reason(candidate: dict, day: str = "") -> str:
|
||||
"""Order reason for a pinned reminder, naming the due date when known.
|
||||
|
||||
Every reminder is pinned, but not for the same reason: the audit trail
|
||||
distinguishes a schedule the clock shows as due/overdue from one this round
|
||||
merely found (whose due date may still be weeks away), so a reader can tell
|
||||
a calendar fact from a freshness rule.
|
||||
"""
|
||||
due = str(candidate.get("due_at") or "").strip()
|
||||
due_day, day = due[:10], str(day or "").strip()[:10]
|
||||
first_seen = str(candidate.get("first_seen") or "").strip()[:10]
|
||||
if due_day and day and due_day <= day:
|
||||
cause = "due"
|
||||
elif first_seen and day and first_seen >= day:
|
||||
cause = "new"
|
||||
else:
|
||||
cause = "open"
|
||||
base = PINNED_REMINDER_REASONS[cause]
|
||||
return f"{base} (due {due})" if due else base
|
||||
|
||||
|
||||
def _delivered_reason(record) -> str:
|
||||
"""Suppression reason naming when, where and how the item was already used."""
|
||||
if record is None:
|
||||
return "already delivered (within the redelivery cooldown)"
|
||||
when = str(record.agenda_date or "") or str(record.delivered_at or "")[:10]
|
||||
where = f" via {record.channel}" if record.channel else ""
|
||||
return f"already delivered on {when}{where} ({record.status})"
|
||||
|
||||
|
||||
@R.register("proactive_agenda_step")
|
||||
|
|
@ -51,17 +101,21 @@ class ProactiveAgendaStep(BaseStep):
|
|||
def __init__(
|
||||
self,
|
||||
max_agenda_items: int = 6,
|
||||
min_discovery_slots: int = 2,
|
||||
llm_timeout_seconds: float = 120,
|
||||
profile_rel_path: str = "profile.md",
|
||||
profile_max_chars: int = 2500,
|
||||
redelivery_cooldown_days: int = 3,
|
||||
skip_key: str = "proactive_skip",
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.max_agenda_items = max(int(max_agenda_items), 1)
|
||||
self.min_discovery_slots = max(int(min_discovery_slots), 0)
|
||||
self.llm_timeout_seconds = float(llm_timeout_seconds)
|
||||
self.profile_rel_path = str(profile_rel_path or "")
|
||||
self.profile_max_chars = max(int(profile_max_chars), 0)
|
||||
self.redelivery_cooldown_days = max(int(redelivery_cooldown_days), 0)
|
||||
self.skip_key = skip_key
|
||||
|
||||
async def execute(self):
|
||||
|
|
@ -91,27 +145,93 @@ class ProactiveAgendaStep(BaseStep):
|
|||
cards_by_id = {str(c.get("topic_id") or ""): c for c in state.scenario_cards if isinstance(c, dict)}
|
||||
carded = [c for c in candidates if str(c.get("id")) in cards_by_id]
|
||||
over_budget = [c for c in candidates if str(c.get("id")) not in cards_by_id]
|
||||
# Delivery feedback outranks every other selection rule: drop what the
|
||||
# host already surfaced before pinning reminders or calling the editor,
|
||||
# so the cooldown cannot be argued away by narrative coherence.
|
||||
daily = state.daily_dir or daily_dir(self)
|
||||
blocked = self._suppressed_deliveries(ws, daily, state.date)
|
||||
# Reminders are the one exemption: a due schedule has to be announced
|
||||
# every time it comes due, and reminder ids derive from a stable title,
|
||||
# so blocking them would silence a recurring reminder for the whole
|
||||
# cooldown window after a single delivery.
|
||||
reminder_ids = {str(c.get("id")) for c in candidates if str(c.get("kind")) == "reminder"}
|
||||
blocked = {
|
||||
blocked_id: record
|
||||
for blocked_id, record in blocked.items()
|
||||
if blocked_id not in reminder_ids
|
||||
}
|
||||
delivered = [c for c in candidates if str(c.get("id")) in blocked]
|
||||
if delivered:
|
||||
carded = [c for c in carded if str(c.get("id")) not in blocked]
|
||||
over_budget = [c for c in over_budget if str(c.get("id")) not in blocked]
|
||||
state.suppressed_delivered = len(delivered)
|
||||
# Reminders are pinned: every reminder - a due schedule or a freshly
|
||||
# found one - always reaches today's agenda and is never traded away for
|
||||
# narrative coherence. What the editor LLM gets
|
||||
# to choose is the remaining (discovery) part of the agenda, inside the
|
||||
# slots the pinned reminders leave - but never fewer than
|
||||
# ``min_discovery_slots``, so a reminder-heavy day still surfaces the
|
||||
# discovery kinds instead of collapsing into a plain calendar. On such a
|
||||
# day the agenda may exceed ``max_agenda_items``: pinned reminders are
|
||||
# additive by design, and dropping either side would break a guarantee.
|
||||
pinned = [c for c in carded if str(c.get("kind")) == "reminder"]
|
||||
optional = [c for c in carded if str(c.get("kind")) != "reminder"]
|
||||
pinned_ids = [str(c.get("id")) for c in pinned]
|
||||
slots = max(self.max_agenda_items - len(pinned_ids), self.min_discovery_slots)
|
||||
self.logger.info(
|
||||
f"[{self.name}] start candidates={len(candidates)} carded={len(carded)} "
|
||||
f"over_budget={len(over_budget)} max_agenda_items={self.max_agenda_items}",
|
||||
f"over_budget={len(over_budget)} already_delivered={len(delivered)} "
|
||||
f"pinned_reminders={len(pinned_ids)} slots={slots} "
|
||||
f"max_agenda_items={self.max_agenda_items}",
|
||||
)
|
||||
|
||||
if not carded:
|
||||
agenda_ids, order_reasons, suppressed_reasons = [], {}, {}
|
||||
elif len(carded) == 1:
|
||||
agenda_ids = [str(carded[0].get("id"))]
|
||||
order_reasons = {agenda_ids[0]: "single push candidate; auto-agenda without LLM"}
|
||||
if not optional:
|
||||
selected_ids, order_reasons, suppressed_reasons = [], {}, {}
|
||||
elif slots == 0:
|
||||
selected_ids, order_reasons = [], {}
|
||||
suppressed_reasons = {
|
||||
str(c.get("id")): f"agenda capacity reserved for {len(pinned_ids)} reminder(s)" for c in optional
|
||||
}
|
||||
elif len(optional) == 1:
|
||||
selected_ids = [str(optional[0].get("id"))]
|
||||
order_reasons = {selected_ids[0]: "single push candidate; auto-agenda without LLM"}
|
||||
suppressed_reasons = {}
|
||||
else:
|
||||
agenda_ids, order_reasons, suppressed_reasons = await self._llm_agenda(state, ws, carded, cards_by_id)
|
||||
selected_ids, order_reasons, suppressed_reasons = await self._llm_agenda(
|
||||
state,
|
||||
ws,
|
||||
optional,
|
||||
cards_by_id,
|
||||
slots,
|
||||
)
|
||||
for cid in pinned_ids:
|
||||
order_reasons.setdefault(
|
||||
cid,
|
||||
_pin_reason(next((c for c in pinned if str(c.get("id")) == cid), {}), state.date),
|
||||
)
|
||||
agenda_ids = pinned_ids + [cid for cid in selected_ids if cid not in pinned_ids]
|
||||
|
||||
suppressed: list[dict] = []
|
||||
for candidate in over_budget:
|
||||
for candidate in delivered:
|
||||
record = blocked.get(str(candidate.get("id")))
|
||||
suppressed.append(
|
||||
{
|
||||
"topic_id": str(candidate.get("id")),
|
||||
"title": str(candidate.get("title") or ""),
|
||||
"reason": "over plan budget: not expanded into a scenario card",
|
||||
"reason": _delivered_reason(record),
|
||||
},
|
||||
)
|
||||
for candidate in over_budget:
|
||||
is_reminder = str(candidate.get("kind")) == "reminder"
|
||||
suppressed.append(
|
||||
{
|
||||
"topic_id": str(candidate.get("id")),
|
||||
"title": str(candidate.get("title") or ""),
|
||||
"reason": (
|
||||
"over reminder card budget: not expanded into a scenario card"
|
||||
if is_reminder
|
||||
else "over plan budget: not expanded into a scenario card"
|
||||
),
|
||||
},
|
||||
)
|
||||
for candidate in carded:
|
||||
|
|
@ -136,12 +256,15 @@ class ProactiveAgendaStep(BaseStep):
|
|||
{
|
||||
"topic_id": cid,
|
||||
"title": str(candidate.get("title") or ""),
|
||||
"kind": str(candidate.get("kind") or ""),
|
||||
"scenario_type": str(card.get("scenario_type") or ""),
|
||||
"opener": str(card.get("opener") or ""),
|
||||
"next_action": str(card.get("next_action") or ""),
|
||||
"preconditions": list(card.get("preconditions") or []),
|
||||
"delivery": str(card.get("delivery") or "in_conversation"),
|
||||
"linked_memory": list(card.get("linked_memory") or []),
|
||||
**({"due_at": str(candidate.get("due_at"))} if candidate.get("due_at") else {}),
|
||||
**({"schedule_id": str(candidate.get("schedule_id"))} if candidate.get("schedule_id") else {}),
|
||||
"order_reason": order_reasons.get(cid) or "",
|
||||
},
|
||||
)
|
||||
|
|
@ -168,8 +291,8 @@ class ProactiveAgendaStep(BaseStep):
|
|||
state.duration_ms = state.duration_ms + int((time.monotonic() - started) * 1000)
|
||||
self._store(state)
|
||||
answer = (
|
||||
f"Agenda: {len(agenda)} item(s), suppressed={len(suppressed)}, push={state.push}, "
|
||||
f"written={written} to {rel_path}"
|
||||
f"Agenda: {len(agenda)} item(s), suppressed={len(suppressed)} "
|
||||
f"(already delivered={len(delivered)}), push={state.push}, written={written} to {rel_path}"
|
||||
)
|
||||
self.context.response.success = True
|
||||
self.context.response.answer = answer
|
||||
|
|
@ -182,13 +305,19 @@ class ProactiveAgendaStep(BaseStep):
|
|||
ws,
|
||||
carded: list[dict],
|
||||
cards_by_id: dict[str, dict],
|
||||
slots: int,
|
||||
) -> tuple[list[str], dict[str, str], dict[str, str]]:
|
||||
"""One list-wise LLM call; returns (agenda ids, order reasons, suppressed reasons)."""
|
||||
"""One list-wise LLM call over the non-pinned cards.
|
||||
|
||||
Returns ``(agenda ids, order reasons, suppressed reasons)``. ``slots``
|
||||
is what the pinned reminders left of ``max_agenda_items``, and it is
|
||||
both the cap told to the model and the hard truncation limit.
|
||||
"""
|
||||
carded_ids = [str(c.get("id")) for c in carded]
|
||||
wrapper = resolve_agent_wrapper(self)
|
||||
if wrapper is None:
|
||||
self.logger.warning(f"[{self.name}] no agent_wrapper available; deterministic fallback agenda")
|
||||
return self._fallback_agenda(carded_ids)
|
||||
return self._fallback_agenda(carded_ids, slots)
|
||||
|
||||
cards_json = json.dumps(
|
||||
[
|
||||
|
|
@ -207,12 +336,12 @@ class ProactiveAgendaStep(BaseStep):
|
|||
)
|
||||
user_message = self.prompt_format(
|
||||
"agenda_user_message",
|
||||
max_agenda_items=self.max_agenda_items,
|
||||
max_agenda_items=slots,
|
||||
cards_json=cards_json,
|
||||
changed_files_json=json.dumps(list(dict.fromkeys(state.changed_paths)), ensure_ascii=False),
|
||||
profile_block=self._profile_block(ws),
|
||||
)
|
||||
system_prompt = self.prompt_format("agenda_system_prompt", max_agenda_items=self.max_agenda_items)
|
||||
system_prompt = self.prompt_format("agenda_system_prompt", max_agenda_items=slots)
|
||||
state.plan_llm_calls += 1
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
|
|
@ -221,12 +350,12 @@ class ProactiveAgendaStep(BaseStep):
|
|||
)
|
||||
except asyncio.TimeoutError:
|
||||
self.logger.warning(f"[{self.name}] LLM reply timed out after {self.llm_timeout_seconds}s")
|
||||
return self._fallback_agenda(carded_ids)
|
||||
return self._fallback_agenda(carded_ids, slots)
|
||||
except Exception as e: # noqa: BLE001 - network/provider errors degrade to fallback agenda
|
||||
self.logger.warning(
|
||||
f"[{self.name}] LLM reply failed ({type(e).__name__}: {e}); using fallback agenda",
|
||||
)
|
||||
return self._fallback_agenda(carded_ids)
|
||||
return self._fallback_agenda(carded_ids, slots)
|
||||
raw = agent_reply_result_text(result)
|
||||
agenda_raw, suppressed_raw = parse_agenda_reply(raw)
|
||||
|
||||
|
|
@ -236,7 +365,7 @@ class ProactiveAgendaStep(BaseStep):
|
|||
cid = str(item.get("topic_id") or "").strip()
|
||||
if cid in allowed and cid not in agenda_ids:
|
||||
agenda_ids.append(cid)
|
||||
if len(agenda_ids) >= self.max_agenda_items:
|
||||
if len(agenda_ids) >= slots:
|
||||
break
|
||||
order_reasons = {
|
||||
str(item.get("topic_id") or "").strip(): str(item.get("order_reason") or "").strip()
|
||||
|
|
@ -250,16 +379,23 @@ class ProactiveAgendaStep(BaseStep):
|
|||
}
|
||||
if not agenda_ids:
|
||||
self.logger.warning(f"[{self.name}] agenda reply unusable; raw={raw[:200]!r}")
|
||||
return self._fallback_agenda(carded_ids)
|
||||
return self._fallback_agenda(carded_ids, slots)
|
||||
return agenda_ids, order_reasons, suppressed_reasons
|
||||
|
||||
def _fallback_agenda(self, carded_ids: list[str]) -> tuple[list[str], dict[str, str], dict[str, str]]:
|
||||
def _fallback_agenda(self, carded_ids: list[str], slots: int) -> tuple[list[str], dict[str, str], dict[str, str]]:
|
||||
"""Freshness/confidence head when generation is unavailable (cards keep sort order)."""
|
||||
agenda_ids = carded_ids[: self.max_agenda_items]
|
||||
agenda_ids = carded_ids[: max(int(slots), 0)]
|
||||
order_reasons = {cid: FALLBACK_ORDER_REASON for cid in agenda_ids}
|
||||
suppressed_reasons = {cid: "capacity (deterministic fallback)" for cid in carded_ids if cid not in agenda_ids}
|
||||
return agenda_ids, order_reasons, suppressed_reasons
|
||||
|
||||
def _suppressed_deliveries(self, ws, daily: str, day: str) -> dict:
|
||||
"""topic_id -> blocking delivery record reported by the host."""
|
||||
state_file, _needs_bootstrap = load_state(ws, daily)
|
||||
if not state_file.deliveries:
|
||||
return {}
|
||||
return suppressed_delivery_ids(state_file, day, self.redelivery_cooldown_days)
|
||||
|
||||
def _profile_block(self, ws) -> str:
|
||||
"""Digest-personal profile first, legacy single profile file as fallback."""
|
||||
digest_dir = str(self.config_value("digest_dir"))
|
||||
|
|
|
|||
|
|
@ -4,10 +4,16 @@ agenda_system_prompt: |
|
|||
candidates), the files that changed today (a proxy for what the user is
|
||||
focused on right now) and an optional user profile.
|
||||
|
||||
Reminder cards are NOT in your input: a due schedule is pinned to today's
|
||||
agenda by the pipeline and cannot be traded away. You are editing the
|
||||
remaining discovery slots.
|
||||
|
||||
Decide which cards enter today's agenda, in which order, and which are
|
||||
held back. The number of agenda items is YOUR call: judge it from content
|
||||
richness and redundancy - anything from 1 up to at most {max_agenda_items}.
|
||||
Fewer, stronger items beat many weak ones.
|
||||
When {max_agenda_items} is 0 the pinned reminders already fill today's
|
||||
agenda: suppress every card with that reason instead of forcing one in.
|
||||
|
||||
## Editorial rules
|
||||
|
||||
|
|
@ -21,6 +27,10 @@ agenda_system_prompt: |
|
|||
- Auditable silence: EVERY candidate topic_id must appear exactly once,
|
||||
either in `agenda` or in `suppressed` (with a short reason). Never drop
|
||||
a candidate without a word.
|
||||
- Quote every YAML string containing punctuation such as `:` - an unquoted
|
||||
`order_reason` with a colon in it makes the whole reply unparseable, and
|
||||
the agenda then degrades to a deterministic order. Keep both reasons to one
|
||||
short sentence.
|
||||
|
||||
## Output format
|
||||
|
||||
|
|
@ -38,9 +48,14 @@ agenda_system_prompt_zh: |
|
|||
你是个人记忆助手当日主动议程的编辑。输入是场景卡（由推送候选扩展而来）、
|
||||
今天发生变化的文件（代表用户当前关注点）以及可选的用户画像。
|
||||
|
||||
提醒类（reminder）场景卡不在你的输入里：到期日程已由流程固定进入当日议程，
|
||||
不可被替换或压制。你编辑的是剩下的发现类名额。
|
||||
|
||||
决定哪些卡片进入今日议程、按什么顺序，哪些暂缓。议程条数由你自行裁决：
|
||||
依据内容丰富度和重复程度决定，最少 1 条，至多不超过 {max_agenda_items} 条。
|
||||
少而强胜过多而弱。
|
||||
当 {max_agenda_items} 为 0 时，说明固定提醒已占满今日议程：请把每张卡都
|
||||
放入 suppressed 并写明该原因，不要硬塞。
|
||||
|
||||
## 编辑规则
|
||||
|
||||
|
|
@ -52,6 +67,9 @@ agenda_system_prompt_zh: |
|
|||
压制它并在 reason 中说明。
|
||||
- 沉默可审计：每个候选 topic_id 必须恰好出现一次，要么在 agenda，要么在
|
||||
suppressed（附简短理由）。禁止不声不响地丢弃任何候选。
|
||||
- 包含 `:` 等标点的 YAML 字符串请加引号——`order_reason` 里出现未加引号的
|
||||
冒号会让整份回复无法解析，议程随之退化为确定性排序。两类理由都请控制在一
|
||||
句话以内。
|
||||
|
||||
## 输出格式
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,25 @@
|
|||
"""Proactive refresh extract step: material scan + follow_ups/extends/updates (F2.0-F2.3)."""
|
||||
"""Proactive refresh extract step: material scan + four proposal kinds (F2.0-F2.3).
|
||||
|
||||
The step owns discovery. It reads the changed daily notes plus three cheap
|
||||
read-only views - the carry-forward topics, the long-lived schedule registry
|
||||
and the digest capability inventory - together with the current business
|
||||
clock, and asks for one sectioned reply covering four proposal kinds:
|
||||
|
||||
``follow_ups`` open loops: propose fetching the latest status and reporting;
|
||||
``extends`` knowledge-base recommendations: propose a wiki node to expand;
|
||||
``reminders`` personal schedule review: propose due/conflicting reminders,
|
||||
plus must-tell notices found in changed ``resource/`` records;
|
||||
``new_ideas`` delegatable work the user has not thought of: propose only.
|
||||
|
||||
Only the reminder branch reads the ``resource/`` layer, which is the reserved
|
||||
hook for external source interfaces: whatever lands there becomes eligible for
|
||||
the schedule review without any further change to the chain.
|
||||
|
||||
Reminders additionally have a deterministic, clock-driven path
|
||||
(``reminder_candidates_from_view``) so a due schedule still reaches today's
|
||||
agenda when there is no new material, no agent wrapper, or an unusable reply.
|
||||
Nothing here executes a proposal; the output stays a proposal end to end.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
|
|
@ -9,25 +30,55 @@ from .._evolve import agent_reply_result_text, passthrough_response
|
|||
from ....components import R
|
||||
from ....schema import ProactiveState
|
||||
from ..dream.utils import daily_dir, pack_paths, today, workspace_dir
|
||||
from .refine import apply_verdicts, candidate_payload, candidates_by_kind, parse_refine_reply
|
||||
from .schedules import (
|
||||
DUE_TRIGGERS,
|
||||
cap_notice_reminders,
|
||||
clean_reminder,
|
||||
clean_schedule_update,
|
||||
merge_reminders,
|
||||
reminder_candidates_from_view,
|
||||
schedule_prompt_entries,
|
||||
schedule_view,
|
||||
)
|
||||
from .utils import (
|
||||
clean_candidate,
|
||||
current_now,
|
||||
load_capability_block,
|
||||
load_carry_forward,
|
||||
load_personal_profile_block,
|
||||
load_state,
|
||||
parse_extract_reply,
|
||||
resolve_agent_wrapper,
|
||||
scan_material_daily,
|
||||
scan_material_resource,
|
||||
)
|
||||
|
||||
DUE_SCHEDULE_KEYS = (
|
||||
"id",
|
||||
"title",
|
||||
"status",
|
||||
"due_at",
|
||||
"next_due",
|
||||
"due_in_days",
|
||||
"trigger",
|
||||
"conflict_with",
|
||||
)
|
||||
|
||||
|
||||
@R.register("proactive_extract_step")
|
||||
class ProactiveExtractStep(BaseStep):
|
||||
"""Scan the proactive material set and extract follow-ups, extends and updates.
|
||||
"""Scan the proactive material set and extract the four proposal kinds.
|
||||
|
||||
Owns the ``proactive`` catalog watermark (never touches the dream catalog)
|
||||
and the daily LLM budget. Short-circuits via ``context[skip_key]`` on
|
||||
busy/budget/timeout per F4.3; early-exits with zero LLM calls when no new
|
||||
evidence exists (F2.0).
|
||||
budget/timeout per F4.3; early-exits with zero LLM calls when no new
|
||||
evidence exists (F2.0) - unless the clock produced due reminders, which
|
||||
are delivered without any LLM call.
|
||||
|
||||
Per-kind ceilings keep discovery proportional (``max_follow_ups``,
|
||||
``max_extends``, ``max_new_ideas``); reminders are deliberately uncapped
|
||||
because they are bounded by the registry itself and pin today's agenda.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
|
|
@ -39,7 +90,21 @@ class ProactiveExtractStep(BaseStep):
|
|||
max_chars_per_file: int = 60000,
|
||||
max_total_chars: int = 300000,
|
||||
profile_max_chars: int = 1500,
|
||||
capability_max_chars: int = 1200,
|
||||
extends_enabled: bool = True,
|
||||
reminders_enabled: bool = True,
|
||||
new_ideas_enabled: bool = True,
|
||||
max_follow_ups: int = 5,
|
||||
max_extends: int = 5,
|
||||
max_new_ideas: int = 3,
|
||||
reminder_lead_days: int = 3,
|
||||
schedule_closed_lookback_days: int = 7,
|
||||
resources_enabled: bool = True,
|
||||
resource_scan_days: int = 0,
|
||||
resource_max_chars: int = 40000,
|
||||
max_notice_reminders: int = 3,
|
||||
refine_enabled: bool = True,
|
||||
refine_timeout_seconds: float = 90,
|
||||
skip_key: str = "proactive_skip",
|
||||
**kwargs,
|
||||
):
|
||||
|
|
@ -51,9 +116,26 @@ class ProactiveExtractStep(BaseStep):
|
|||
self.max_chars_per_file = max(int(max_chars_per_file), 1000)
|
||||
self.max_total_chars = max(int(max_total_chars), 0)
|
||||
self.profile_max_chars = max(int(profile_max_chars), 0)
|
||||
self.capability_max_chars = max(int(capability_max_chars), 0)
|
||||
self.extends_enabled = bool(extends_enabled)
|
||||
self.reminders_enabled = bool(reminders_enabled)
|
||||
self.new_ideas_enabled = bool(new_ideas_enabled)
|
||||
self.max_follow_ups = max(int(max_follow_ups), 0)
|
||||
self.max_extends = max(int(max_extends), 0)
|
||||
self.max_new_ideas = max(int(max_new_ideas), 0)
|
||||
self.reminder_lead_days = max(int(reminder_lead_days), 0)
|
||||
self.schedule_closed_lookback_days = max(int(schedule_closed_lookback_days), 0)
|
||||
self.resources_enabled = bool(resources_enabled)
|
||||
# 0 inherits ``scan_days``: the notice window normally matches the
|
||||
# conversation window, but can be widened on its own.
|
||||
self.resource_scan_days = max(int(resource_scan_days), 0)
|
||||
self.resource_max_chars = max(int(resource_max_chars), 0)
|
||||
self.max_notice_reminders = max(int(max_notice_reminders), 0)
|
||||
self.refine_enabled = bool(refine_enabled)
|
||||
self.refine_timeout_seconds = float(refine_timeout_seconds)
|
||||
self.skip_key = skip_key
|
||||
|
||||
# pylint: disable=too-many-return-statements
|
||||
async def execute(self):
|
||||
assert self.context is not None
|
||||
if self.context.get(self.skip_key):
|
||||
|
|
@ -71,10 +153,27 @@ class ProactiveExtractStep(BaseStep):
|
|||
scan_days=self.scan_days,
|
||||
carry_forward_days=self.carry_forward_days,
|
||||
)
|
||||
self.logger.info(f"[{self.name}] start date={day} scan_days={self.scan_days} extends={self.extends_enabled}")
|
||||
self.logger.info(
|
||||
f"[{self.name}] start date={day} scan_days={self.scan_days} extends={self.extends_enabled} "
|
||||
f"reminders={self.reminders_enabled} new_ideas={self.new_ideas_enabled}",
|
||||
)
|
||||
|
||||
# 1) Material set M (F2.0) - all cheap, before any LLM work.
|
||||
# 1) Material set M (F2.0) - all cheap, before any LLM work. Daily
|
||||
# notes feed every branch. The ``resource/`` layer feeds the reminder
|
||||
# branch only (external notices the user must be told about), so it is
|
||||
# scanned only when reminders are active.
|
||||
m_daily = scan_material_daily(ws, day, daily, self.scan_days)
|
||||
notices = self.reminders_enabled and self.resources_enabled
|
||||
if notices:
|
||||
m_resource, resource_skipped = scan_material_resource(
|
||||
ws,
|
||||
day,
|
||||
self.resource_scan_days or self.scan_days,
|
||||
str(self.config_value("resource_dir")),
|
||||
)
|
||||
else:
|
||||
m_resource, resource_skipped = [], 0
|
||||
state.resource_scanned, state.resource_skipped = len(m_resource), resource_skipped
|
||||
|
||||
# 2) Truth source + carry-forward (F1.3/F1.4).
|
||||
state_file, needs_bootstrap = load_state(ws, daily)
|
||||
|
|
@ -90,57 +189,120 @@ class ProactiveExtractStep(BaseStep):
|
|||
state.carry_forward_count = len(carry_all)
|
||||
state.carry_forward_prompt = carry_prompt
|
||||
|
||||
# 3) Change detection against the proactive catalog watermark.
|
||||
# Only daily notes are material: fresh resource uploads already flow
|
||||
# into daily notes via auto_resource, so no separate resource scan.
|
||||
# 3) Change detection against the proactive catalog watermark. Both
|
||||
# material sets share one watermark, so an external notice arriving on
|
||||
# a day without any conversation still triggers a round.
|
||||
existing = {}
|
||||
for rel in m_daily:
|
||||
for rel in (*m_daily, *m_resource):
|
||||
try:
|
||||
existing[rel] = (ws / rel).stat().st_mtime
|
||||
except OSError as e:
|
||||
self.logger.error(f"[{self.name}] stat failed on {rel}: {e}")
|
||||
nodes = await self.file_catalog.get_nodes()
|
||||
indexed = {n.path: n.st_mtime for n in nodes}
|
||||
state.changed_paths = [rel for rel, mt in existing.items() if indexed.get(rel) != mt]
|
||||
changed = [rel for rel, mt in existing.items() if indexed.get(rel) != mt]
|
||||
daily_set = set(m_daily)
|
||||
state.changed_paths = [rel for rel in changed if rel in daily_set]
|
||||
state.resource_changed = [rel for rel in changed if rel not in daily_set]
|
||||
# mtime snapshot of what this round actually read; finish only
|
||||
# checkpoints paths whose mtime still matches, so files modified while
|
||||
# the LLM calls run stay "changed" for the next round (audit item 3).
|
||||
state.changed_mtimes = {rel: existing[rel] for rel in state.changed_paths}
|
||||
state.changed_mtimes = {rel: existing[rel] for rel in changed}
|
||||
self.logger.info(
|
||||
f"[{self.name}] material daily={len(m_daily)} "
|
||||
f"changed={len(state.changed_paths)} carry_forward={len(carry_all)}",
|
||||
f"[{self.name}] material daily={len(m_daily)} resource={len(m_resource)} "
|
||||
f"resource_skipped={resource_skipped} changed={len(state.changed_paths)} "
|
||||
f"notices_changed={len(state.resource_changed)} carry_forward={len(carry_all)}",
|
||||
)
|
||||
|
||||
# 4) Zero-consumption early exit (A4 row 1).
|
||||
if not state.changed_paths:
|
||||
# 4) Clock + schedule registry scan (branch C). Deterministic and free,
|
||||
# so it runs before any LLM decision: a due schedule must reach today's
|
||||
# agenda even when there is no new material, no agent wrapper, or an
|
||||
# unusable reply.
|
||||
now_dt = current_now(self)
|
||||
view = schedule_view(state_file.schedules, now_dt, self.reminder_lead_days) if self.reminders_enabled else []
|
||||
schedules_by_id = {str(entry.get("id") or ""): entry for entry in view if entry.get("id")}
|
||||
seeded = reminder_candidates_from_view(view, day)
|
||||
state.due_schedules = [
|
||||
{key: entry.get(key) for key in DUE_SCHEDULE_KEYS} for entry in view if entry.get("trigger") in DUE_TRIGGERS
|
||||
]
|
||||
if seeded:
|
||||
self.logger.info(
|
||||
f"[{self.name}] clock scan: registry={len(state_file.schedules)} "
|
||||
f"due={len(state.due_schedules)} seeded_reminders={len(seeded)} now={now_dt.isoformat()}",
|
||||
)
|
||||
|
||||
# 5) Zero-consumption early exit (A4 row 1) - unless the clock alone
|
||||
# already produced reminders, which need neither material nor an LLM.
|
||||
if not state.changed_paths and not state.resource_changed:
|
||||
if seeded:
|
||||
return self._clock_only(state, started, seeded, "Skipped LLM: no new evidence")
|
||||
state.early_exit = "no_new_evidence"
|
||||
return self._finish(state, started, "Skipped: no new evidence; 0 LLM calls")
|
||||
|
||||
# 5) LLM channel (F4.4): structurally one reply per round plus at most
|
||||
# 6) LLM channel (F4.4): structurally one reply per round plus at most
|
||||
# one parse-failure retry, so no persistent budget is needed (v5 R5).
|
||||
wrapper = resolve_agent_wrapper(self)
|
||||
if wrapper is None:
|
||||
if seeded:
|
||||
return self._clock_only(state, started, seeded, "Degraded: no agent_wrapper configured")
|
||||
state.early_exit = "no_agent_wrapper"
|
||||
self.logger.warning(f"[{self.name}] no agent_wrapper available; skipping round")
|
||||
return self._finish(state, started, "Skipped: no agent_wrapper configured")
|
||||
|
||||
# 6) One LLM call with sectioned output (A3); retry once on parse failure.
|
||||
# meta is None when this round must NOT checkpoint (timeout, or two
|
||||
# unusable replies): the material stays "changed" for the next round
|
||||
# instead of being silently consumed on output that can never yield
|
||||
# topics (audit #1).
|
||||
meta = await self._extract_with_retry(wrapper, state, ws, day, daily)
|
||||
# 7) One LLM call with sectioned output (A3); retry once on parse failure.
|
||||
# meta is None when this round produced no usable output (timeout, or
|
||||
# two unusable replies): the material stays "changed" for the next
|
||||
# round instead of being silently consumed on output that can never
|
||||
# yield topics (audit #1).
|
||||
meta, failure = await self._extract_with_retry(wrapper, state, ws, day, daily, now_dt, view)
|
||||
if meta is None:
|
||||
if seeded:
|
||||
return self._clock_only(state, started, seeded, f"Degraded: {failure}")
|
||||
self.context[self.skip_key] = {"reason": failure}
|
||||
self._store(state)
|
||||
return passthrough_response(self, self.skip_key)
|
||||
self._clean_output(state, meta, set(m_daily), day)
|
||||
self._clean_output(state, meta, set(m_daily), day, schedules_by_id, set(m_resource))
|
||||
if self.refine_enabled:
|
||||
await self._refine_stage(state, wrapper, ws, day, now_dt, view, set(m_resource))
|
||||
state.reminders = merge_reminders(seeded, state.reminders)
|
||||
answer = (
|
||||
f"Extracted {len(state.follow_ups)} follow_up(s), {len(state.extends)} extend(s), "
|
||||
f"{len(state.updates)} update(s) from {len(state.changed_paths)} changed file(s)"
|
||||
f"{len(state.reminders)} reminder(s), {len(state.new_ideas)} idea(s), "
|
||||
f"{len(state.updates)} update(s), {len(state.schedule_updates)} schedule update(s) "
|
||||
f"from {len(state.changed_paths)} changed file(s) "
|
||||
f"and {len(state.resource_changed)} external notice file(s)"
|
||||
)
|
||||
if state.refine_calls:
|
||||
answer += (
|
||||
f"; refine: {state.refine_calls} pass(es), dropped {len(state.refine_dropped)}, "
|
||||
f"moved {len(state.refine_moved)}, degraded {state.refine_degraded}"
|
||||
)
|
||||
return self._finish(state, started, answer)
|
||||
|
||||
def _build_messages(self, state: ProactiveState, ws, day: str, daily: str, material_blob: str) -> tuple[str, str]:
|
||||
def _clock_only(self, state: ProactiveState, started: float, seeded: list[dict], reason: str):
|
||||
"""Deliver clock-driven reminders without consuming this round's material.
|
||||
|
||||
``changed_paths`` is cleared so the finish step checkpoints nothing and
|
||||
the unread material stays "changed" for the next round; the chain still
|
||||
runs, so today's agenda gets its reminders.
|
||||
"""
|
||||
state.changed_paths, state.resource_changed, state.changed_mtimes = [], [], {}
|
||||
state.reminders = seeded
|
||||
state.reminder_only = True
|
||||
self.logger.warning(f"[{self.name}] {reason}; delivering {len(seeded)} clock-driven reminder(s)")
|
||||
return self._finish(state, started, f"{reason}; {len(seeded)} clock-driven reminder(s), 0 LLM output used")
|
||||
|
||||
def _build_messages(
|
||||
self,
|
||||
state: ProactiveState,
|
||||
ws,
|
||||
day: str,
|
||||
daily: str,
|
||||
material_blob: str,
|
||||
notice_blob: str,
|
||||
now_dt,
|
||||
view: list[dict],
|
||||
) -> tuple[str, str]:
|
||||
carry_forward_json = json.dumps(
|
||||
[
|
||||
{
|
||||
|
|
@ -157,20 +319,43 @@ class ProactiveExtractStep(BaseStep):
|
|||
ensure_ascii=False,
|
||||
)
|
||||
changed = list(dict.fromkeys(state.changed_paths))
|
||||
notices = list(dict.fromkeys(state.resource_changed))
|
||||
schedules_json = json.dumps(
|
||||
schedule_prompt_entries(view, self.schedule_closed_lookback_days, day),
|
||||
ensure_ascii=False,
|
||||
)
|
||||
user_message = self.prompt_format(
|
||||
"extract_user_message",
|
||||
date=day,
|
||||
clock=f"{now_dt.isoformat(timespec='minutes')} ({now_dt.strftime('%A')})",
|
||||
resource_dir=str(self.config_value("resource_dir")),
|
||||
changed_paths_json=json.dumps(changed, ensure_ascii=False),
|
||||
carry_forward_json=carry_forward_json,
|
||||
schedules_json=schedules_json,
|
||||
material_blob=material_blob,
|
||||
profile_block=self._profile_block(ws),
|
||||
capability_block=self._capability_block(ws),
|
||||
resource_paths_json=json.dumps(notices, ensure_ascii=False),
|
||||
resource_block=notice_blob or "(no external notices in this round)",
|
||||
extends=self.extends_enabled,
|
||||
reminders=self.reminders_enabled,
|
||||
ideas=self.new_ideas_enabled,
|
||||
resources=self.reminders_enabled and self.resources_enabled,
|
||||
)
|
||||
system_prompt = self.prompt_format(
|
||||
"extract_system_prompt",
|
||||
workspace_dir=str(ws),
|
||||
daily_dir=daily,
|
||||
resource_dir=str(self.config_value("resource_dir")),
|
||||
output_shape=self._output_shape(),
|
||||
max_follow_ups=self.max_follow_ups,
|
||||
max_extends=self.max_extends,
|
||||
max_new_ideas=self.max_new_ideas,
|
||||
max_notice_reminders=self.max_notice_reminders,
|
||||
extends=self.extends_enabled,
|
||||
reminders=self.reminders_enabled,
|
||||
ideas=self.new_ideas_enabled,
|
||||
resources=self.reminders_enabled and self.resources_enabled,
|
||||
)
|
||||
return user_message, system_prompt
|
||||
|
||||
|
|
@ -179,8 +364,41 @@ class ProactiveExtractStep(BaseStep):
|
|||
digest_dir = str(self.config_value("digest_dir"))
|
||||
return load_personal_profile_block(ws, digest_dir, self.profile_max_chars)
|
||||
|
||||
async def _extract_with_retry(self, wrapper, state, ws, day: str, daily: str) -> dict | None:
|
||||
"""Returns parsed meta; None means skip without checkpointing."""
|
||||
def _capability_block(self, ws) -> str:
|
||||
"""Digest procedure/wiki inventory; grounds the ``new_ideas`` branch."""
|
||||
digest_dir = str(self.config_value("digest_dir"))
|
||||
return load_capability_block(ws, digest_dir, self.capability_max_chars)
|
||||
|
||||
def _output_shape(self) -> str:
|
||||
"""YAML sample listing exactly the active sections.
|
||||
|
||||
Composed from per-section snippets rather than inline ``[flag]`` lines:
|
||||
the flag filter left-strips kept lines, which would flatten the
|
||||
indentation of a nested YAML example.
|
||||
"""
|
||||
names = ["follow_ups"]
|
||||
if self.extends_enabled:
|
||||
names.append("extends")
|
||||
if self.reminders_enabled:
|
||||
names.append("reminders")
|
||||
if self.new_ideas_enabled:
|
||||
names.append("new_ideas")
|
||||
names.append("updates")
|
||||
if self.reminders_enabled:
|
||||
names.append("schedule_updates")
|
||||
return "\n".join(self.get_prompt(f"extract_shape_{name}") for name in names)
|
||||
|
||||
async def _extract_with_retry(
|
||||
self,
|
||||
wrapper,
|
||||
state,
|
||||
ws,
|
||||
day: str,
|
||||
daily: str,
|
||||
now_dt,
|
||||
view,
|
||||
) -> tuple[dict | None, str]:
|
||||
"""Returns ``(meta, failure_reason)``; meta None means no usable output."""
|
||||
# Newest daily material first: when the total budget bites, the
|
||||
# freshest evidence survives and the oldest files are omitted.
|
||||
ordered = list(dict.fromkeys(state.changed_paths))[::-1]
|
||||
|
|
@ -190,48 +408,111 @@ class ProactiveExtractStep(BaseStep):
|
|||
limit_per_file=self.max_chars_per_file,
|
||||
max_total_chars=self.max_total_chars or None,
|
||||
)
|
||||
user_message, system_prompt = self._build_messages(state, ws, day, daily, material_blob)
|
||||
# External notices are packed under their own, smaller budget so a
|
||||
# large upload can neither crowd out the conversation material nor be
|
||||
# crowded out by it.
|
||||
notice_blob = pack_paths(
|
||||
ws,
|
||||
list(dict.fromkeys(state.resource_changed))[::-1],
|
||||
limit_per_file=self.max_chars_per_file,
|
||||
max_total_chars=self.resource_max_chars or None,
|
||||
)
|
||||
user_message, system_prompt = self._build_messages(
|
||||
state,
|
||||
ws,
|
||||
day,
|
||||
daily,
|
||||
material_blob,
|
||||
notice_blob,
|
||||
now_dt,
|
||||
view,
|
||||
)
|
||||
raw = ""
|
||||
for attempt in (1, 2):
|
||||
raw = await self._reply(wrapper, state, user_message, system_prompt)
|
||||
if raw is None:
|
||||
self.context[self.skip_key] = {"reason": "llm_timeout"}
|
||||
return None
|
||||
return None, "llm_timeout"
|
||||
meta = parse_extract_reply(raw)
|
||||
if meta:
|
||||
return meta
|
||||
return meta, ""
|
||||
self.logger.warning(f"[{self.name}] parse failed on attempt {attempt}; raw={raw[:200]!r}")
|
||||
self.logger.error(
|
||||
f"[{self.name}] reply unusable after 2 attempts; keeping material un-checkpointed; " f"raw={raw[:200]!r}",
|
||||
f"[{self.name}] reply unusable after 2 attempts; keeping material un-checkpointed; raw={raw[:200]!r}",
|
||||
)
|
||||
self.context[self.skip_key] = {"reason": "extract_parse_failed"}
|
||||
return None
|
||||
return None, "extract_parse_failed"
|
||||
|
||||
async def _reply(self, wrapper, state, user_message, system_prompt):
|
||||
async def _reply(self, wrapper, state, user_message, system_prompt, timeout: float | None = None):
|
||||
"""One timeout-wrapped reply; returns raw text or None on timeout."""
|
||||
state.llm_calls += 1
|
||||
limit = self.llm_timeout_seconds if timeout is None else float(timeout)
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
wrapper.reply(user_message, system_prompt=system_prompt),
|
||||
timeout=self.llm_timeout_seconds,
|
||||
timeout=limit,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
self.logger.warning(f"[{self.name}] LLM reply timed out after {self.llm_timeout_seconds}s")
|
||||
self.logger.warning(f"[{self.name}] LLM reply timed out after {limit}s")
|
||||
return None
|
||||
return agent_reply_result_text(result)
|
||||
|
||||
def _clean_output(self, state: ProactiveState, meta: dict, allowed: set[str], day: str) -> None:
|
||||
for raw in meta.get("follow_ups") or []:
|
||||
if candidate := clean_candidate(raw, allowed, "follow_up", day):
|
||||
state.follow_ups.append(candidate)
|
||||
else:
|
||||
state.dropped_missing += 1
|
||||
def _clean_output(
|
||||
self,
|
||||
state: ProactiveState,
|
||||
meta: dict,
|
||||
allowed: set[str],
|
||||
day: str,
|
||||
schedules_by_id: dict,
|
||||
notice_allowed: set[str] | None = None,
|
||||
) -> None:
|
||||
# Only the reminder branch may cite the ``resource/`` layer: the other
|
||||
# three kinds reason about the user's own conversations, and letting
|
||||
# them quote an external upload would blur that boundary.
|
||||
reminder_allowed = set(allowed) | set(notice_allowed or ())
|
||||
self._append_kind(
|
||||
state.follow_ups,
|
||||
meta.get("follow_ups"),
|
||||
allowed,
|
||||
"follow_up",
|
||||
day,
|
||||
self.max_follow_ups,
|
||||
state,
|
||||
)
|
||||
if self.extends_enabled:
|
||||
for raw in meta.get("extends") or []:
|
||||
if candidate := clean_candidate(raw, allowed, "interest_extend", day):
|
||||
state.extends.append(candidate)
|
||||
self._append_kind(
|
||||
state.extends,
|
||||
meta.get("extends"),
|
||||
allowed,
|
||||
"interest_extend",
|
||||
day,
|
||||
self.max_extends,
|
||||
state,
|
||||
)
|
||||
if self.reminders_enabled:
|
||||
cleaned: list[dict] = []
|
||||
for raw in meta.get("reminders") or []:
|
||||
if candidate := clean_reminder(raw, reminder_allowed, day, schedules_by_id, notice_allowed):
|
||||
cleaned.append(candidate)
|
||||
else:
|
||||
state.dropped_missing += 1
|
||||
# Dateless notice/key-event reminders are bounded; clock-anchored
|
||||
# ones are not (they are facts, and they pin today's agenda).
|
||||
state.reminders, over_cap = cap_notice_reminders(cleaned, self.max_notice_reminders)
|
||||
state.dropped_over_cap += over_cap
|
||||
for raw in meta.get("schedule_updates") or []:
|
||||
if update := clean_schedule_update(raw, reminder_allowed, day):
|
||||
state.schedule_updates.append(update)
|
||||
else:
|
||||
state.dropped_missing += 1
|
||||
if self.new_ideas_enabled:
|
||||
self._append_kind(
|
||||
state.new_ideas,
|
||||
meta.get("new_ideas"),
|
||||
allowed,
|
||||
"new_idea",
|
||||
day,
|
||||
self.max_new_ideas,
|
||||
state,
|
||||
)
|
||||
for raw in meta.get("updates") or []:
|
||||
if not isinstance(raw, dict):
|
||||
continue
|
||||
|
|
@ -251,6 +532,142 @@ class ProactiveExtractStep(BaseStep):
|
|||
},
|
||||
)
|
||||
|
||||
def _append_kind(
|
||||
self,
|
||||
bucket: list[dict],
|
||||
raw_items,
|
||||
allowed: set[str],
|
||||
kind: str,
|
||||
day: str,
|
||||
cap: int,
|
||||
state: ProactiveState,
|
||||
) -> None:
|
||||
"""Clean one discovery section into ``bucket``, honouring its ceiling.
|
||||
|
||||
``cap <= 0`` means unlimited. Over-cap entries are counted separately
|
||||
from invalid ones so a noisy round stays diagnosable in metadata.
|
||||
"""
|
||||
for raw in raw_items or []:
|
||||
if cap and len(bucket) >= cap:
|
||||
state.dropped_over_cap += 1
|
||||
continue
|
||||
if candidate := clean_candidate(raw, allowed, kind, day):
|
||||
bucket.append(candidate)
|
||||
else:
|
||||
state.dropped_missing += 1
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Stage 2: per-kind adjudication (precision pass)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _enabled_kinds(self) -> set[str]:
|
||||
"""Kinds this round may hold candidates in (the legal ``move`` targets)."""
|
||||
return {
|
||||
kind
|
||||
for kind, active in (
|
||||
("follow_up", True),
|
||||
("interest_extend", self.extends_enabled),
|
||||
("reminder", self.reminders_enabled),
|
||||
("new_idea", self.new_ideas_enabled),
|
||||
)
|
||||
if active
|
||||
}
|
||||
|
||||
def _kind_caps(self) -> dict[str, int]:
|
||||
"""Per-kind ceilings re-applied after stage-2 moves (0 = uncapped)."""
|
||||
return {
|
||||
"follow_up": self.max_follow_ups,
|
||||
"interest_extend": self.max_extends,
|
||||
"new_idea": self.max_new_ideas,
|
||||
"reminder": 0,
|
||||
}
|
||||
|
||||
def _cap_label(self, kind: str) -> str:
|
||||
"""Ceiling as the refine prompt should state it."""
|
||||
if kind != "reminder":
|
||||
return str(self._kind_caps()[kind])
|
||||
return f"uncapped, but at most {self.max_notice_reminders} dateless notice/key-event reminder(s)"
|
||||
|
||||
async def _refine_stage(self, state, wrapper, ws, day: str, now_dt, view, notice_paths: set[str]) -> None:
|
||||
"""One small adjudication call per kind that produced candidates.
|
||||
|
||||
Stage 2 never sees the material blob: it receives that kind's candidates
|
||||
plus the only context slice the kind needs, which keeps the pass cheap
|
||||
and its judgement focused. Each pass is independently fail-open, so a
|
||||
timeout or an unusable reply keeps the stage-1 candidates of that kind.
|
||||
"""
|
||||
verdicts: list[dict] = []
|
||||
for kind, bucket in candidates_by_kind(state):
|
||||
known = {str(candidate.get("id") or "") for candidate in bucket}
|
||||
user_message = self.prompt_format(
|
||||
"refine_user_message",
|
||||
kind=kind,
|
||||
cap=self._cap_label(kind),
|
||||
checks=self.get_prompt(f"refine_checks_{kind}"),
|
||||
candidates_json=json.dumps(candidate_payload(bucket), ensure_ascii=False),
|
||||
context_block=self._refine_context(kind, state, ws, view, day, now_dt),
|
||||
)
|
||||
system_prompt = self.prompt_format("refine_system_prompt", kind=kind)
|
||||
raw = await self._reply(wrapper, state, user_message, system_prompt, self.refine_timeout_seconds)
|
||||
state.refine_calls += 1
|
||||
if raw is None:
|
||||
state.refine_degraded += 1
|
||||
self.logger.warning(f"[{self.name}] refine({kind}) timed out; keeping stage-1 candidates")
|
||||
continue
|
||||
parsed = parse_refine_reply(raw)
|
||||
if parsed is None:
|
||||
state.refine_degraded += 1
|
||||
self.logger.warning(f"[{self.name}] refine({kind}) reply unusable; raw={raw[:160]!r}")
|
||||
continue
|
||||
# A pass may not reach into another kind's bucket: ids outside it
|
||||
# are ignored rather than trusted.
|
||||
verdicts.extend(verdict for verdict in parsed if verdict["id"] in known)
|
||||
if not verdicts:
|
||||
# Nothing to adjudicate: every pass answered "keep everything".
|
||||
# The caps only need re-applying when a move could have crossed a
|
||||
# kind boundary, but the summary is worth logging either way.
|
||||
self.logger.info(
|
||||
f"[{self.name}] refine passes={state.refine_calls} dropped=0 moved=0 "
|
||||
f"degraded={state.refine_degraded} no verdicts; kept: "
|
||||
f"reminders={len(state.reminders)} follow_ups={len(state.follow_ups)} "
|
||||
f"ideas={len(state.new_ideas)} extends={len(state.extends)}",
|
||||
)
|
||||
return
|
||||
apply_verdicts(state, verdicts, self._kind_caps(), self._enabled_kinds(), notice_paths)
|
||||
state.reminders, over_cap = cap_notice_reminders(state.reminders, self.max_notice_reminders)
|
||||
state.dropped_over_cap += over_cap
|
||||
self.logger.info(
|
||||
f"[{self.name}] refine passes={state.refine_calls} dropped={len(state.refine_dropped)} "
|
||||
f"moved={len(state.refine_moved)} degraded={state.refine_degraded} kept: "
|
||||
f"reminders={len(state.reminders)} follow_ups={len(state.follow_ups)} "
|
||||
f"ideas={len(state.new_ideas)} extends={len(state.extends)}",
|
||||
)
|
||||
|
||||
def _refine_context(self, kind: str, state, ws, view, day: str, now_dt) -> str:
|
||||
"""The single context slice this kind's adjudication needs (no material)."""
|
||||
if kind == "reminder":
|
||||
entries = schedule_prompt_entries(view, self.schedule_closed_lookback_days, day)
|
||||
return (
|
||||
f"now: {now_dt.isoformat(timespec='minutes')} ({now_dt.strftime('%A')})\n"
|
||||
"schedules (registry, triggers already computed from the clock):\n"
|
||||
f"{json.dumps(entries, ensure_ascii=False)}"
|
||||
)
|
||||
if kind == "follow_up":
|
||||
carried = [
|
||||
{"id": t.id, "title": t.title, "kind": t.kind, "last_evidence_at": t.last_evidence_at}
|
||||
for t in state.carry_forward_prompt
|
||||
]
|
||||
return (
|
||||
"carry_forward_topics (already tracked; a candidate that restates one of these "
|
||||
f"must be dropped):\n{json.dumps(carried, ensure_ascii=False)}"
|
||||
)
|
||||
if kind == "new_idea":
|
||||
return f"capability_inventory (what the agent can actually do):\n{self._capability_block(ws)}"
|
||||
return (
|
||||
"knowledge_base inventory (existing wiki nodes; a candidate that merely restates one "
|
||||
f"of these must be dropped):\n{self._capability_block(ws)}"
|
||||
)
|
||||
|
||||
def _store(self, state: ProactiveState) -> None:
|
||||
assert self.context is not None
|
||||
data = state.model_dump()
|
||||
|
|
|
|||
|
|
@ -1,27 +1,111 @@
|
|||
extract_system_prompt: |
|
||||
You are the proactive discovery agent for a personal memory workspace.
|
||||
You read recently changed conversation notes and maintain a small set of
|
||||
user-interest topics. You never browse the web,
|
||||
never write files, and never collect material yourself.
|
||||
You read recently changed conversation notes, the user's schedule registry
|
||||
and the current clock, and you propose what the assistant should bring up
|
||||
next. You never browse the web, never write files, never collect material
|
||||
yourself, and never execute anything: every entry you emit is a PROPOSAL
|
||||
that the host agent may raise with the user.
|
||||
|
||||
workspace_dir: {workspace_dir}
|
||||
|
||||
## Branch A - follow_ups (open loops)
|
||||
## The four kinds and their division of labour
|
||||
|
||||
Find unresolved matters the user may still want to close:
|
||||
- questions that were asked but never answered;
|
||||
- tasks that were started, interrupted, or postponed;
|
||||
- commitments or plans mentioned without any follow-through yet.
|
||||
Each branch below has its own section, its own job and its own ceiling.
|
||||
Every branch is optional; a quiet day produces empty lists. Never let one
|
||||
branch borrow another's job (a due schedule is a reminder, not a follow-up;
|
||||
a wiki node worth expanding is an extend, not an idea).
|
||||
|
||||
## Branch A - follow_ups (kind: follow_up)
|
||||
|
||||
Scope: matters the user assigned, or keeps caring about, that are NOT closed
|
||||
yet - tasks started/interrupted/postponed, questions asked but never
|
||||
answered, commitments mentioned without follow-through. Carry-forward topics
|
||||
already track the long-lived ones, so prefer `updates` for those and open a
|
||||
`follow_ups` entry only for a matter no carry-forward topic covers.
|
||||
Job: state the latest progress visible in the material, then propose going
|
||||
and fetching the current status so the assistant can report back to the user
|
||||
on its own initiative ("let me check where X stands and tell you"). The
|
||||
deliverable is a report, never a silent completion of the work.
|
||||
Emit each as a `follow_ups` entry.
|
||||
|
||||
[extends]## Branch B - extends (interest boundary)
|
||||
[extends]## Branch B - extends (kind: interest_extend)
|
||||
[extends]
|
||||
[extends]Infer topics the user has NOT focused on yet, but that are plausibly
|
||||
[extends]relevant to their recent work, using the conversation notes. These
|
||||
[extends]guide future knowledge-source expansion; you only describe them, you
|
||||
[extends]never collect anything.
|
||||
[extends]Scope: content recommendation for the user's personal knowledge
|
||||
[extends]base. Starting from a topic the user already cares about (see the
|
||||
[extends]wiki inventory in the user message), name one refinement or
|
||||
[extends]extension of it that is worth adding as a new wiki node: a
|
||||
[extends]sub-topic, an adjacent concept, a comparison, a missing prerequisite.
|
||||
[extends]Job: propose introducing and walking the user through that content.
|
||||
[extends]You only describe the node to expand; you never collect or write it.
|
||||
[extends]Emit each as an `extends` entry.
|
||||
|
||||
[reminders]## Branch C - reminders (kind: reminder) + schedule_updates
|
||||
[reminders]
|
||||
[reminders]Scope: the user's personal plans and calendar. You are given
|
||||
[reminders]`schedules` (a long-lived registry that may hold items mentioned
|
||||
[reminders]weeks ago) and `now` (the current clock). Your jobs:
|
||||
[reminders]1. Review - compare the registry and the material against the
|
||||
[reminders] clock, then propose a reminder for what is due soon, due today,
|
||||
[reminders] already overdue, or clashing with another entry. Warn about a
|
||||
[reminders] possible conflict explicitly and say which entries collide.
|
||||
[reminders] Also propose the upcoming todos the user would want to be
|
||||
[reminders] nudged about. Propose only: never claim a calendar was changed.
|
||||
[reminders]2. Maintain - whenever the material adds, moves, confirms,
|
||||
[reminders] completes or cancels a schedule, emit a `schedule_updates`
|
||||
[reminders] entry. The registry is the only structure that lives long
|
||||
[reminders] enough to fire on a due date mentioned long ago, so a schedule
|
||||
[reminders] you do not register will never be reminded.
|
||||
[reminders] Title an entry by the matter alone, never by its date or time:
|
||||
[reminders] the id is derived from the title, so a date in it mints a brand
|
||||
[reminders] new registry entry every time that schedule moves, and the user
|
||||
[reminders] ends up reminded twice about one appointment.
|
||||
[resources]3. Notices and key events - scan `resource_notices` (external
|
||||
[resources] source records that changed this round: whatever an interface
|
||||
[resources] dropped into `{resource_dir}/` - mail, feeds, exports,
|
||||
[resources] documents) and pick out only what the user MUST be told today.
|
||||
[resources] Two shapes count:
|
||||
[resources] a) a direct notice about the user's own affairs - a deadline or
|
||||
[resources] date that moved, a cancellation, a bill or payment falling
|
||||
[resources] due, a confirmation that needs action;
|
||||
[resources] b) a key event that materially affects the user even though
|
||||
[resources] nobody addressed it to them - a policy or rule change, an
|
||||
[resources] incident/recall/security advisory, a development in a matter
|
||||
[resources] they track, a market or supply shift they are exposed to.
|
||||
[resources] Emit each as a `reminders` entry citing that resource path. Give
|
||||
[resources] `due_at` whenever the item has a date, and register that dated
|
||||
[resources] commitment via `schedule_updates` so it keeps firing from the
|
||||
[resources] registry. A key event with no date may omit `due_at`; at most
|
||||
[resources] {max_notice_reminders} such dateless entries survive a round, so
|
||||
[resources] make them count. Merely interesting reading material is neither
|
||||
[resources] a notice nor a key event; leave it out.
|
||||
[reminders]Traceability: a reminder about a registered schedule cites it via
|
||||
[reminders]`schedule_id` and may leave `paths` empty (its evidence is the
|
||||
[reminders]clock). A reminder about a brand-new schedule must cite
|
||||
[reminders]changed_paths and carry `due_at`.
|
||||
[resources]A notice-driven reminder cites its resource_paths entry instead.
|
||||
[reminders]Reminders are the highest-priority kind: every reminder you emit
|
||||
[reminders]enters today's agenda, so they are not capped - but never invent a
|
||||
[reminders]schedule to justify one, and never repeat a schedule the registry
|
||||
[reminders]already shows as done or cancelled.
|
||||
[reminders]Emit reminders as `reminders` entries and registry maintenance as
|
||||
[reminders]`schedule_updates` entries.
|
||||
|
||||
[ideas]## Branch D - new_ideas (kind: new_idea)
|
||||
[ideas]
|
||||
[ideas]Scope: work the user probably does not realise can be delegated.
|
||||
[ideas]Combine what the agent recently executed, the data it collected and
|
||||
[ideas]the experience/skills/tools/APIs it already has (the capability
|
||||
[ideas]inventory in the user message) with what the material shows the user
|
||||
[ideas]is working on, then propose one concrete piece of work worth handing
|
||||
[ideas]over - the kind of task a user would never think to ask for.
|
||||
[ideas]Job: describe the deliverable, why it is valuable right now, and what
|
||||
[ideas]existing resource makes it feasible. Stop at the proposal: never
|
||||
[ideas]execute it and never imply it is already done.
|
||||
[ideas]Ground every idea in a capability or a dataset that actually appears
|
||||
[ideas]in the inventory or the material; an idea needing resources nobody
|
||||
[ideas]has is noise.
|
||||
[ideas]Emit each as a `new_ideas` entry.
|
||||
|
||||
## updates (carried-forward topics)
|
||||
|
||||
For every topic in carry_forward_topics, decide exactly one action:
|
||||
|
|
@ -41,6 +125,7 @@ extract_system_prompt: |
|
|||
## Confidence rubric (fixed; do not freestyle scores)
|
||||
|
||||
- 0.9: explicit unfinished action with a time or commitment;
|
||||
[reminders]- 0.8: a registered schedule the clock shows as due, overdue or clashing;
|
||||
- 0.7: the user stated intent explicitly but no concrete action yet;
|
||||
- 0.5: inferential link from several weak signals;
|
||||
- 0.3: weak association only;
|
||||
|
|
@ -50,9 +135,21 @@ extract_system_prompt: |
|
|||
|
||||
- Empty lists are a normal, expected output. Never invent topics to fill
|
||||
space, and never promote a one-off weak mention into a topic.
|
||||
- Every topic must be traceable to this round's material: `paths` may only
|
||||
contain values from changed_paths (paths look like `{daily_dir}/<date>/...`);
|
||||
entries violating this are discarded.
|
||||
- Ceilings are not targets, and entries over a ceiling are discarded
|
||||
downstream: at most {max_follow_ups} follow_ups.
|
||||
[extends]- Ceiling: at most {max_extends} extends.
|
||||
[ideas]- Ceiling: at most {max_new_ideas} new_ideas.
|
||||
[reminders]- Reminders are uncapped, but each one must be a real finding of
|
||||
[reminders]the schedule review - never invent a schedule to justify one.
|
||||
- Every topic must be traceable: `paths` may only contain values from
|
||||
changed_paths (paths look like `{daily_dir}/<date>/...`); entries violating
|
||||
this are discarded.
|
||||
[reminders]- Traceability exception: a `reminders` entry citing a registered
|
||||
[reminders]`schedule_id` may take its paths from the registry instead.
|
||||
[resources]- Traceability exception: `reminders` and `schedule_updates` may also
|
||||
[resources]cite resource_paths (`{resource_dir}/...`). No other section may.
|
||||
- user_profile and the capability inventory are background knowledge: use
|
||||
them to judge relevance, never cite them in paths/evidence.
|
||||
- Do not restate topics already listed in carry_forward_topics; use
|
||||
`updates` to change their status instead.
|
||||
- Same-matter rule: if a finding is really a carry_forward topic in
|
||||
|
|
@ -69,45 +166,93 @@ extract_system_prompt: |
|
|||
|
||||
Return only one YAML fenced block with this exact shape:
|
||||
```yaml
|
||||
follow_ups:
|
||||
- title: <specific unresolved matter>
|
||||
reason: <why it is still open>
|
||||
confidence: 0.7
|
||||
paths: [<changed path>, ...]
|
||||
[extends]extends:
|
||||
[extends] - title: <related-but-unfocused topic>
|
||||
[extends] reason: <why it matters, grounded in the material>
|
||||
[extends] confidence: 0.5
|
||||
[extends] paths: [<changed path>, ...]
|
||||
updates:
|
||||
- id: <echoed carry-forward id>
|
||||
action: keep|update|resolve
|
||||
evidence: <path or path#anchor>
|
||||
reason: <short justification>
|
||||
confidence: <re-scored per rubric; emit for update, omit otherwise>
|
||||
{output_shape}
|
||||
```
|
||||
|
||||
extract_system_prompt_zh: |
|
||||
你是个人记忆 workspace 的 proactive 发现 agent。你阅读近期变化的对话笔记
|
||||
，维护一小组用户兴趣主题。你绝不联网、
|
||||
绝不写文件、绝不自行收集资料。
|
||||
你是个人记忆 workspace 的 proactive 发现 agent。你阅读近期变化的对话笔记、
|
||||
用户日程注册表与当前时钟，产出"下一步值得主动提起什么"的提议。你绝不联网、
|
||||
绝不写文件、绝不自行收集资料、绝不执行任何动作：你输出的每一条都是提议
|
||||
（proposal），由宿主 agent 决定是否向用户提起。
|
||||
|
||||
workspace_dir: {workspace_dir}
|
||||
|
||||
## 分支 A - follow_ups（未决事项 / open loop）
|
||||
## 四类标签及其职责边界
|
||||
|
||||
找出用户可能仍想关闭的未决事项：
|
||||
- 提出过但没有得到回答的问题；
|
||||
- 开始过但被中断或搁置的任务；
|
||||
- 提到过但没有后续动作的承诺或计划。
|
||||
下面每个分支有自己的 section、职责与条数上限。每个分支都是可选的，平静的
|
||||
一天就应该输出空列表。禁止串味：到期的日程属于 reminder 而不是 follow_up；
|
||||
值得展开的知识节点属于 interest_extend 而不是 new_idea。
|
||||
|
||||
## 分支 A - follow_ups（kind: follow_up）
|
||||
|
||||
范围：用户指派过、或持续关心但尚未关闭的事项——开始过却被中断/搁置的任务、
|
||||
提出过但没有回答的问题、提到过却没有后续动作的承诺。carry_forward_topics
|
||||
已经跟踪长期未决事项，这类请优先用 `updates` 处置，只有当没有任何
|
||||
carry-forward 主题覆盖时，才新开 `follow_ups` 条目。
|
||||
职责：先说清素材里能看到的最新进展，然后提议去检索该事项的最新情况，以便
|
||||
助手主动向用户汇报（"我去确认一下 X 现在到哪一步了，再告诉你"）。交付物是
|
||||
一次汇报，绝不是悄悄替用户把事做完。
|
||||
每条作为一个 `follow_ups` 条目输出。
|
||||
|
||||
[extends]## 分支 B - extends（兴趣边界扩展）
|
||||
[extends]## 分支 B - extends（kind: interest_extend）
|
||||
[extends]
|
||||
[extends]推断用户尚未关注、但与其近期工作大概率相关的主题，依据是对话笔记
|
||||
[extends]。它们用于指引未来的知识源扩展；你只描述主题，不收集任何资料。
|
||||
[extends]范围：面向个人知识库的内容推荐。从用户已经关心的主题出发（见用户
|
||||
[extends]消息中的 wiki 节点清单），指出一个值得作为新 wiki 节点补上的细化或
|
||||
[extends]延伸方向：子话题、相邻概念、横向对比、缺失的前置知识。
|
||||
[extends]职责：提议向用户介绍并展示这部分内容。你只描述要扩展哪个节点，
|
||||
[extends]绝不自行收集或撰写内容。
|
||||
[extends]每条作为一个 `extends` 条目输出。
|
||||
|
||||
[reminders]## 分支 C - reminders（kind: reminder）+ schedule_updates
|
||||
[reminders]
|
||||
[reminders]范围：用户的个人计划与日程。输入包含 `schedules`（长生命周期
|
||||
[reminders]注册表，可能有几周前提到的条目）与 `now`（当前时钟）。职责如下：
|
||||
[reminders]1. 检查回顾——把注册表与素材对照时钟检查一遍，对即将到期、今天
|
||||
[reminders] 到期、已经逾期、或与其它日程冲突的安排提出提醒；冲突要明确
|
||||
[reminders] 预警并指出是哪几条相撞；同时提议提醒即将到来的 todos。
|
||||
[reminders] 只做提议：绝不声称已经改动了任何日历。
|
||||
[reminders]2. 维护注册表——素材中出现新增、改期、确认、完成、取消日程时，
|
||||
[reminders] 必须输出 `schedule_updates` 条目。注册表是唯一能活到"很久以
|
||||
[reminders] 前提到的到期日"那天的结构；不登记的日程永远不会被提醒。
|
||||
[reminders] 条目标题只写事项本身，绝不要带日期或时间：id 由标题推导，标题
|
||||
[reminders] 里含日期会让这条日程每次改期都在注册表里新增一条，同一个约定
|
||||
[reminders] 就会被提醒两次。
|
||||
[resources]3. 通知与关键事件——通读 `resource_notices`（本轮发生变化的外部
|
||||
[resources] 信源记录，即外部信源接口投递到 `{resource_dir}/` 的邮件、订阅
|
||||
[resources] 源、导出件、文档等），只挑出"今天必须告知用户"的内容。两种形态
|
||||
[resources] 都算：
|
||||
[resources] a) 直接通知——与用户自身事务相关的：时间/截止日被改动、被取消、
|
||||
[resources] 账单或付款到期、需要动作的确认件；
|
||||
[resources] b) 关键事件——虽不是发给用户本人、但对其有实质影响的：政策或规则
|
||||
[resources] 变化、事故/召回/安全公告、其持续跟踪事项的重要进展、其所暴露
|
||||
[resources] 的市场或供应变化。
|
||||
[resources] 每条作为 `reminders` 输出并引用该 resource 路径。带日期的一律给出
|
||||
[resources] `due_at`，并用 `schedule_updates` 把该承诺登记进注册表，使其在素材
|
||||
[resources] 离开扫描窗口后仍能按到期日触发。没有日期的关键事件可以省略
|
||||
[resources] `due_at`；这类无日期条目每轮最多保留 {max_notice_reminders} 条，
|
||||
[resources] 请挑真正重要的。仅仅是"有点意思"的阅读材料既不是通知也不是关键
|
||||
[resources] 事件，不要输出。
|
||||
[reminders]可追溯性：针对已登记日程的提醒用 `schedule_id` 引用，`paths`
|
||||
[reminders]可以为空（它的证据是时钟）；针对全新日程的提醒必须引用
|
||||
[reminders]changed_paths 并给出 `due_at`。
|
||||
[resources]来自外部通知的提醒改为引用 resource_paths 中的路径。
|
||||
[reminders]reminder 是四类中优先级最高的：你输出的每条 reminder 都会进入
|
||||
[reminders]当日 agenda，因此不设条数上限——但绝不为了凑提醒编造日程，也
|
||||
[reminders]绝不重复注册表中已完成/已取消的条目。
|
||||
[reminders]提醒输出为 `reminders` 条目，注册表维护输出为 `schedule_updates`。
|
||||
|
||||
[ideas]## 分支 D - new_ideas（kind: new_idea）
|
||||
[ideas]
|
||||
[ideas]范围：用户根本没想到"这事可以委托 AI 办"的工作事项。把 agent 最近
|
||||
[ideas]执行的结果、收集的数据、已积累的经验/skill/工具/API（见用户消息中
|
||||
[ideas]的能力清单）与素材里用户正在做的事情结合起来，提议一件具体、值得
|
||||
[ideas]交出去做的工作。
|
||||
[ideas]职责：说清交付物是什么、为什么现在有价值、依靠哪个已有资源才可行。
|
||||
[ideas]停留在提议：绝不执行，也绝不暗示已经做完。
|
||||
[ideas]每个想法必须落在能力清单或素材里真实存在的能力/数据上；需要谁都没有
|
||||
[ideas]的资源才能完成的想法属于噪音。
|
||||
[ideas]每条作为一个 `new_ideas` 条目输出。
|
||||
|
||||
## updates（carry-forward 主题处置）
|
||||
|
||||
对 carry_forward_topics 中的每个主题，恰好选择一个动作：
|
||||
|
|
@ -125,6 +270,7 @@ extract_system_prompt_zh: |
|
|||
## confidence 评分规则（固化，禁止自由打分）
|
||||
|
||||
- 0.9：有明确未完成动作且带时间/承诺；
|
||||
[reminders]- 0.8：注册表中已登记、且时钟显示即将到期/已逾期/存在冲突的日程；
|
||||
- 0.7：用户显式表达意图但无明确动作；
|
||||
- 0.5：推断性关联（多条弱信号）；
|
||||
- 0.3：弱联想；
|
||||
|
|
@ -134,8 +280,20 @@ extract_system_prompt_zh: |
|
|||
|
||||
- 空列表是正常且预期的输出。不得为凑数编造主题，不得把一次性弱提及
|
||||
拔高为主题。
|
||||
- 每个主题必须能追溯到本轮素材：`paths` 只能使用 changed_paths 中的值
|
||||
（路径形如 `{daily_dir}/<date>/...`），越界条目会被丢弃。
|
||||
- 上限是天花板而不是目标，超出上限的条目会在下游被丢弃：follow_ups 至多
|
||||
{max_follow_ups} 条。
|
||||
[extends]- 上限：extends 至多 {max_extends} 条。
|
||||
[ideas]- 上限：new_ideas 至多 {max_new_ideas} 条。
|
||||
[reminders]- reminders 不设上限，但每条都必须来自真实的日程检查结论，
|
||||
[reminders]绝不为了凑提醒编造日程。
|
||||
- 每个主题必须可追溯：`paths` 只能使用 changed_paths 中的值（路径形如
|
||||
`{daily_dir}/<date>/...`），越界条目会被丢弃。
|
||||
[reminders]- 可追溯性例外：引用了已登记 `schedule_id` 的 `reminders` 条目，
|
||||
[reminders]其 paths 可以来自注册表。
|
||||
[resources]- 可追溯性例外：`reminders` 与 `schedule_updates` 还可以引用
|
||||
[resources]resource_paths（`{resource_dir}/...`）；其他 section 不可以。
|
||||
- user_profile 与能力清单属于背景知识：只用于判断相关度，禁止引用进
|
||||
paths/evidence。
|
||||
- 不要复述 carry_forward_topics 中已 open 的主题；要改变其状态请用
|
||||
`updates`。
|
||||
- 同一件事规则：若新发现其实只是某个 carry_forward 主题的换种说法，
|
||||
|
|
@ -151,61 +309,427 @@ extract_system_prompt_zh: |
|
|||
|
||||
只返回一个 YAML fenced block，结构必须严格如下：
|
||||
```yaml
|
||||
{output_shape}
|
||||
```
|
||||
|
||||
extract_user_message: |
|
||||
date: {date}
|
||||
now (current clock in the business timezone; the reference for every due
|
||||
judgement): {clock}
|
||||
|
||||
changed_paths (the ONLY allowed values for `paths`):
|
||||
{changed_paths_json}
|
||||
|
||||
[resources]resource_paths (external source records changed this round; also
|
||||
[resources]allowed in `paths`, but ONLY for reminders/schedule_updates):
|
||||
[resources]{resource_paths_json}
|
||||
|
||||
carry_forward_topics (already open; only act on them via updates):
|
||||
{carry_forward_json}
|
||||
|
||||
[reminders]schedules (long-lived registry; `next_due`, `due_in_days`,
|
||||
[reminders]`trigger` and `conflict_with` are already computed from the clock
|
||||
[reminders]above; recently closed entries are shown so you do not re-add
|
||||
[reminders]them):
|
||||
[reminders]{schedules_json}
|
||||
|
||||
user_profile (background knowledge about the user's identity, preferences
|
||||
and constraints; use it ONLY to judge relevance and confidence - it is NOT
|
||||
material, never cite it in paths/evidence):
|
||||
{profile_block}
|
||||
|
||||
[ideas]capability_inventory (reusable experience/skills and knowledge nodes
|
||||
[ideas]the agent already has; background for new_ideas only - it is NOT
|
||||
[ideas]material, never cite it in paths/evidence):
|
||||
[ideas]{capability_block}
|
||||
|
||||
# Material
|
||||
|
||||
{material_blob}
|
||||
|
||||
[resources]# External notices ({resource_dir}/, reminder branch only)
|
||||
[resources]
|
||||
[resources]{resource_block}
|
||||
|
||||
Emit the sections your active branches define, per the contract:
|
||||
- follow_ups and updates
|
||||
[extends]- extends
|
||||
[reminders]- reminders and schedule_updates
|
||||
[ideas]- new_ideas
|
||||
Remember: every section is optional, empty lists are fine, never invent
|
||||
topics, and never execute a proposal.
|
||||
|
||||
extract_user_message_zh: |
|
||||
日期：{date}
|
||||
当前时钟（业务时区；所有"是否到期"判断的基准）：{clock}
|
||||
|
||||
changed_paths（`paths` 唯一允许的取值）：
|
||||
{changed_paths_json}
|
||||
|
||||
[resources]resource_paths（本轮变化的外部信源记录；也允许写进 `paths`，
|
||||
[resources]但仅限 reminders/schedule_updates）：
|
||||
[resources]{resource_paths_json}
|
||||
|
||||
carry_forward_topics（已 open 的主题；只能通过 updates 处置）：
|
||||
{carry_forward_json}
|
||||
|
||||
[reminders]schedules（长生命周期日程注册表；其中 `next_due`、`due_in_days`、
|
||||
[reminders]`trigger`、`conflict_with` 已按上面的时钟算好；近期已关闭的条目
|
||||
[reminders]也会列出，避免你重复登记）：
|
||||
[reminders]{schedules_json}
|
||||
|
||||
用户画像（用户身份、偏好与约束的背景知识；仅用于判断相关度与
|
||||
confidence——它不是素材，禁止引用进 paths/evidence）：
|
||||
{profile_block}
|
||||
|
||||
[ideas]能力清单（agent 已经具备的可复用经验/skill 与已有知识节点；仅供
|
||||
[ideas]new_ideas 参考——它不是素材，禁止引用进 paths/evidence）：
|
||||
[ideas]{capability_block}
|
||||
|
||||
# 本轮素材
|
||||
|
||||
{material_blob}
|
||||
|
||||
[resources]# 外部通知（{resource_dir}/，仅供 reminder 分支使用）
|
||||
[resources]
|
||||
[resources]{resource_block}
|
||||
|
||||
按契约输出当前启用分支对应的 section：
|
||||
- follow_ups 与 updates
|
||||
[extends]- extends
|
||||
[reminders]- reminders 与 schedule_updates
|
||||
[ideas]- new_ideas
|
||||
记住：每个 section 都是可选的，空列表是正常输出，绝不编造主题，也绝不
|
||||
执行任何提议。
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Output-shape snippets, composed by ``_output_shape`` for the active branches
|
||||
# only. They are separate keys (not ``[flag]`` lines) so the flag filter's
|
||||
# left-strip cannot flatten the YAML indentation of the example.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
extract_shape_follow_ups: |
|
||||
follow_ups:
|
||||
- title: <specific unresolved matter>
|
||||
reason: <latest progress seen in the material, and what to go fetch and report>
|
||||
confidence: 0.7
|
||||
paths:
|
||||
- <changed path>
|
||||
|
||||
extract_shape_extends: |
|
||||
extends:
|
||||
- title: <related-but-unfocused topic>
|
||||
reason: <which cared-about topic it refines or extends, and the wiki node it would add>
|
||||
confidence: 0.5
|
||||
paths:
|
||||
- <changed path>
|
||||
|
||||
extract_shape_reminders: |
|
||||
reminders:
|
||||
- title: <the scheduled matter>
|
||||
reason: <why now: due soon / due today / overdue / clashes with what>
|
||||
due_at: <YYYY-MM-DD or YYYY-MM-DDTHH:MM>
|
||||
schedule_id: <registry id; omit only for a brand-new schedule>
|
||||
conflict_with: [<title or schedule_id it clashes with>]
|
||||
confidence: 0.8
|
||||
paths:
|
||||
- <changed path; may be empty when schedule_id is given>
|
||||
|
||||
extract_shape_new_ideas: |
|
||||
new_ideas:
|
||||
- title: <a concrete piece of work worth delegating>
|
||||
reason: <the deliverable, why now, and which existing skill/tool/data makes it feasible>
|
||||
confidence: 0.5
|
||||
paths:
|
||||
- <changed path>
|
||||
|
||||
extract_shape_updates: |
|
||||
updates:
|
||||
- id: <echoed carry-forward id>
|
||||
action: keep|update|resolve
|
||||
evidence: <path or path#anchor>
|
||||
reason: <short justification>
|
||||
confidence: <re-scored per rubric; emit for update, omit otherwise>
|
||||
|
||||
extract_shape_schedule_updates: |
|
||||
schedule_updates:
|
||||
- id: <registry id; omit for action=add>
|
||||
action: add|reschedule|confirm|done|cancel
|
||||
title: <required for add; the matter alone, never its date or time>
|
||||
due_at: <YYYY-MM-DD or YYYY-MM-DDTHH:MM; required for add and reschedule>
|
||||
due_end_at: <optional end of a range>
|
||||
recurrence: <omit or one of daily|weekly|monthly|yearly>
|
||||
lead_days: <optional; how many days ahead to start reminding>
|
||||
detail: <optional short note>
|
||||
evidence: <path from changed_paths, or the notice path; required, it anchors the change>
|
||||
reason: <short justification>
|
||||
|
||||
extract_shape_follow_ups_zh: |
|
||||
follow_ups:
|
||||
- title: <具体的未决事项>
|
||||
reason: <为什么仍未关闭>
|
||||
reason: <素材中看到的最新进展，以及要去检索并汇报什么>
|
||||
confidence: 0.7
|
||||
paths: [<素材路径>, ...]
|
||||
[extends]extends:
|
||||
[extends] - title: <相关但未被关注的主题>
|
||||
[extends] reason: <为什么重要，基于素材>
|
||||
[extends] confidence: 0.5
|
||||
[extends] paths: [<素材路径>, ...]
|
||||
paths:
|
||||
- <素材路径>
|
||||
|
||||
extract_shape_extends_zh: |
|
||||
extends:
|
||||
- title: <相关但尚未被关注的主题>
|
||||
reason: <它细化/延伸了哪个已关心主题，会补上哪个 wiki 节点>
|
||||
confidence: 0.5
|
||||
paths:
|
||||
- <素材路径>
|
||||
|
||||
extract_shape_reminders_zh: |
|
||||
reminders:
|
||||
- title: <对应的日程事项>
|
||||
reason: <为什么是现在：即将到期 / 今天到期 / 已逾期 / 与哪条冲突>
|
||||
due_at: <YYYY-MM-DD 或 YYYY-MM-DDTHH:MM>
|
||||
schedule_id: <注册表 id；仅全新日程可省略>
|
||||
conflict_with: [<与之冲突的标题或 schedule_id>]
|
||||
confidence: 0.8
|
||||
paths:
|
||||
- <素材路径；给了 schedule_id 时可以为空>
|
||||
|
||||
extract_shape_new_ideas_zh: |
|
||||
new_ideas:
|
||||
- title: <一件具体、值得委托出去的工作>
|
||||
reason: <交付物是什么、为什么是现在、依靠哪个已有 skill/工具/数据才可行>
|
||||
confidence: 0.5
|
||||
paths:
|
||||
- <素材路径>
|
||||
|
||||
extract_shape_updates_zh: |
|
||||
updates:
|
||||
- id: <回显的 carry-forward id>
|
||||
action: keep|update|resolve
|
||||
evidence: <路径或 路径#锚点>
|
||||
reason: <简短理由>
|
||||
confidence: <按 rubric 重新打分；action=update 时给出，其余可省略>
|
||||
|
||||
extract_shape_schedule_updates_zh: |
|
||||
schedule_updates:
|
||||
- id: <注册表 id；action=add 时可省略>
|
||||
action: add|reschedule|confirm|done|cancel
|
||||
title: <add 时必填；只写事项本身，不要带日期或时间>
|
||||
due_at: <YYYY-MM-DD 或 YYYY-MM-DDTHH:MM；add 与 reschedule 必填>
|
||||
due_end_at: <可选的时间段结束点>
|
||||
recurrence: <可省略，或 daily|weekly|monthly|yearly 之一>
|
||||
lead_days: <可选；提前多少天开始提醒>
|
||||
detail: <可选的简短备注>
|
||||
evidence: <changed_paths 中的路径，或通知所来自的外部信源路径；必填，它是证据锚点>
|
||||
reason: <简短理由>
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stage 2: per-kind adjudication prompts (``_refine_stage``). One small call per
|
||||
# kind that produced candidates. The material blob is never re-sent: a pass gets
|
||||
# its own candidates plus the single context slice that kind needs.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
refine_system_prompt: |
|
||||
You are the quality gate of the proactive discovery stage of a personal
|
||||
memory system. An earlier pass already read today's material and proposed
|
||||
candidates of kind `{kind}`. You do NOT re-read the material, and you do NOT
|
||||
write new candidates: you adjudicate the list you are given.
|
||||
|
||||
For each candidate decide exactly one of:
|
||||
- `keep`: it belongs to this kind, it is grounded, and it is worth raising;
|
||||
- `drop`: reject it, and say why in one short clause;
|
||||
- `move`: it is unmistakably another kind's job - name `to_kind`.
|
||||
|
||||
You may also rescore `confidence` with the fixed rubric when the proposer
|
||||
clearly over- or under-rated it. Never rewrite `title` or `reason`: the
|
||||
wording the user will see has already been produced.
|
||||
|
||||
## Rubric (fixed; do not freestyle scores)
|
||||
|
||||
- 0.9: explicit unfinished action with a time or commitment;
|
||||
- 0.8: a registered schedule the clock shows as due, overdue or clashing;
|
||||
- 0.7: the user stated intent explicitly but no concrete action yet;
|
||||
- 0.5: inferential link from several weak signals;
|
||||
- 0.3: weak association only;
|
||||
- below 0.3: drop the candidate instead of scoring it.
|
||||
|
||||
## Hard rules
|
||||
|
||||
- Echo ids exactly; an id that is not in the candidate list is discarded.
|
||||
- A candidate you do not mention is kept unchanged, so mention only the ones
|
||||
you want to drop, move or rescore.
|
||||
- Dropping is for real defects, not for taste: when in doubt, keep.
|
||||
- The context block is background knowledge, never evidence; do not cite it.
|
||||
- These stay proposals: never word a verdict as if work had been done.
|
||||
- Quote YAML strings containing punctuation such as `:`.
|
||||
|
||||
## Output format
|
||||
|
||||
Return only one YAML fenced block:
|
||||
```yaml
|
||||
verdicts:
|
||||
- id: <echoed id>
|
||||
decision: keep|drop|move
|
||||
to_kind: <only for move: reminder|follow_up|new_idea|interest_extend>
|
||||
confidence: <optional rescore per the rubric>
|
||||
reason: <required for drop; one short clause>
|
||||
```
|
||||
|
||||
extract_user_message: |
|
||||
date: {date}
|
||||
refine_system_prompt_zh: |
|
||||
你是个人记忆系统 proactive 发现阶段的质量闸门。前一遍已经读过今天的素材，
|
||||
并产出了 kind 为 `{kind}` 的候选。你**不重读素材**，也**不新增候选**：
|
||||
你只裁决给你的这份列表。
|
||||
|
||||
changed_paths (the ONLY allowed values for `paths`):
|
||||
{changed_paths_json}
|
||||
对每个候选恰好给出一个结论：
|
||||
- `keep`：属于该 kind、有依据、值得提起；
|
||||
- `drop`：驳回，并用一小句说明原因；
|
||||
- `move`：它明显属于另一类职责——给出 `to_kind`。
|
||||
|
||||
carry_forward_topics (already open; only act on them via updates):
|
||||
{carry_forward_json}
|
||||
当 proposer 明显打高或打低时，你可以按固定评分规则重打 `confidence`。
|
||||
绝不改写 `title` 或 `reason`：给用户看的措辞已经产出了。
|
||||
|
||||
user_profile (background knowledge about the user's identity, preferences
|
||||
and constraints; use it ONLY to judge interest relevance and confidence -
|
||||
it is NOT material, never cite it in paths/evidence):
|
||||
{profile_block}
|
||||
## 评分规则（固化，禁止自由打分）
|
||||
|
||||
# Material
|
||||
- 0.9：有明确未完成动作且带时间/承诺；
|
||||
- 0.8：注册表中已登记、且时钟显示即将到期/已逾期/存在冲突的日程；
|
||||
- 0.7：用户显式表达意图但无明确动作；
|
||||
- 0.5：推断性关联（多条弱信号）；
|
||||
- 0.3：仅弱关联；
|
||||
- 低于 0.3：直接 drop，不要打分。
|
||||
|
||||
{material_blob}
|
||||
## 硬约束
|
||||
|
||||
Extract follow_ups[extends] / extends / updates per the contract. Remember:
|
||||
empty lists are fine; never invent topics.
|
||||
- id 必须原样回显；不在候选列表中的 id 会被丢弃。
|
||||
- 未被提及的候选按原样保留，所以只写你要 drop / move / 重打分的那几条。
|
||||
- drop 只用于真实缺陷，不用于口味差异：拿不准就 keep。
|
||||
- 上下文块是背景知识，不是证据，禁止引用。
|
||||
- 这些仍然是提议：任何结论都不得写成"已经做完了"。
|
||||
- 包含 `:` 等标点的 YAML 字符串请加引号。
|
||||
|
||||
extract_user_message_zh: |
|
||||
日期：{date}
|
||||
## 输出格式
|
||||
|
||||
changed_paths（`paths` 唯一允许的取值）：
|
||||
{changed_paths_json}
|
||||
只返回一个 YAML fenced block：
|
||||
```yaml
|
||||
verdicts:
|
||||
- id: <回显的 id>
|
||||
decision: keep|drop|move
|
||||
to_kind: <仅 move 时给出：reminder|follow_up|new_idea|interest_extend>
|
||||
confidence: <可选，按评分规则重打>
|
||||
reason: <drop 时必填；一小句>
|
||||
```
|
||||
|
||||
carry_forward_topics（已 open 的主题；只能通过 updates 处置）：
|
||||
{carry_forward_json}
|
||||
refine_user_message: |
|
||||
kind under review: {kind}
|
||||
ceiling for this kind: {cap}
|
||||
|
||||
用户画像（用户身份、偏好与约束的背景知识；仅用于判断兴趣相关度与
|
||||
confidence——它不是素材，禁止引用进 paths/evidence）：
|
||||
{profile_block}
|
||||
## What to reject for this kind
|
||||
|
||||
# 本轮素材
|
||||
{checks}
|
||||
|
||||
{material_blob}
|
||||
## Context for this kind only (background knowledge, never evidence)
|
||||
|
||||
按契约抽取 follow_ups[extends] / extends / updates。记住：空列表是正常
|
||||
输出，绝不编造主题。
|
||||
{context_block}
|
||||
|
||||
## Candidates to adjudicate
|
||||
|
||||
{candidates_json}
|
||||
|
||||
Return one YAML fenced block with a `verdicts` list, mentioning only the
|
||||
candidates you want to drop, move or rescore.
|
||||
|
||||
refine_user_message_zh: |
|
||||
本次裁决的 kind：{kind}
|
||||
该类上限：{cap}
|
||||
|
||||
## 该类应当驳回什么
|
||||
|
||||
{checks}
|
||||
|
||||
## 仅供该类使用的上下文（背景知识，不是证据）
|
||||
|
||||
{context_block}
|
||||
|
||||
## 待裁决候选
|
||||
|
||||
{candidates_json}
|
||||
|
||||
只返回一个 YAML fenced block，其中 `verdicts` 列表只写你要 drop / move /
|
||||
重打分的候选。
|
||||
|
||||
refine_checks_reminder: |
|
||||
- Drop a reminder whose schedule the registry already shows as done or
|
||||
cancelled, or whose due value contradicts the registry entry it cites.
|
||||
- Drop one that is really an open loop on the user's own work (follow_up) or
|
||||
a topic worth reading about (interest_extend) - but never one anchored on
|
||||
the external source layer, which no other kind is allowed to quote.
|
||||
- Keep an external notice or key event even when it also reads as interesting
|
||||
background: "the user must be told" is exactly this kind's job, and a key
|
||||
event that materially affects them is not merely reading material.
|
||||
- Keep a dated reminder even when it looks minor: a clock fact about the
|
||||
user's own calendar is exactly what this kind exists for.
|
||||
- A dateless reminder must be an external notice or a key event the user has
|
||||
to be told today; drop one that is merely interesting background, and
|
||||
remember only a few dateless entries survive the ceiling.
|
||||
- Move to follow_up only when the candidate tracks progress on a matter the
|
||||
user assigned and has no schedule attached.
|
||||
|
||||
refine_checks_reminder_zh: |
|
||||
- 若注册表已显示该日程完成或取消，或候选给出的到期值与其引用的注册表条目
|
||||
矛盾，drop。
|
||||
- 若它其实是用户自己工作的未闭环事项（follow_up），或只是值得阅读的主题
|
||||
（interest_extend），drop；但锚定在外部信源上的候选例外——其他任何类都不
|
||||
允许引用外部信源。
|
||||
- 外部通知或关键事件即使同时读起来像"有意思的背景资料"，也要 keep："必须
|
||||
告知用户"正是这一类的职责，实质影响用户的关键事件不只是阅读材料。
|
||||
- 带日期的提醒即使看起来很小也要 keep：关于用户自己日程的时钟事实正是这一
|
||||
类存在的意义。
|
||||
- 无日期的提醒必须是"今天必须告知"的外部通知或关键事件；仅仅是有意思的背景
|
||||
信息就 drop，并记住无日期条目受上限约束、只能留下少数几条。
|
||||
- 仅当候选跟踪的是用户指派事项的进展且不带任何日程时，才 move 到 follow_up。
|
||||
|
||||
refine_checks_follow_up: |
|
||||
- Drop a candidate that restates a carry_forward topic from the context:
|
||||
long-lived open loops are maintained through updates, never re-proposed.
|
||||
- Drop one with no visible open loop (nothing was assigned, asked, or left
|
||||
unfinished); a general interest is interest_extend.
|
||||
- Drop one whose "latest progress" is asserted rather than visible in its own
|
||||
reason.
|
||||
- Keep a candidate that is really a dated commitment instead of dropping it:
|
||||
stage 2 cannot attach a due date, so the calendar stays the reminder
|
||||
branch's job.
|
||||
|
||||
refine_checks_follow_up_zh: |
|
||||
- 若候选只是复述上下文中的 carry_forward 主题，drop：长期未闭环事项通过
|
||||
updates 维护，不应被重新提议。
|
||||
- 若看不到任何未闭环动作（没有被指派、没有被提问、没有被搁置的事项），drop；
|
||||
泛泛的兴趣属于 interest_extend。
|
||||
- 若所谓"最新进展"只是断言、在其自身 reason 里看不到依据，drop。
|
||||
- 若候选其实是一个带日期的承诺，keep 而不要 drop：第二阶段无法补上到期日，
|
||||
日历始终由 reminder 分支负责。
|
||||
|
||||
refine_checks_new_idea: |
|
||||
- Drop an idea that needs a capability, dataset, tool or API absent from the
|
||||
capability inventory: it would be an offer nobody can fulfil.
|
||||
- Drop one that is really a task the user already assigned (follow_up) or a
|
||||
topic to read about (interest_extend).
|
||||
- Drop one worded as a report of work already done; these stay proposals.
|
||||
- Keep only ideas naming a concrete deliverable, and rescore vague ones down.
|
||||
|
||||
refine_checks_new_idea_zh: |
|
||||
- 若想法依赖能力清单中不存在的经验/数据/工具/API，drop：那是没人能兑现的邀约。
|
||||
- 若它其实是用户已经指派的任务（follow_up）或只是可读的主题（interest_extend），
|
||||
drop。
|
||||
- 若措辞像"已经做完了的汇报"，drop；这些只能是提议。
|
||||
- 只保留说清了具体交付物的想法；含糊的把 confidence 打低。
|
||||
|
||||
refine_checks_interest_extend: |
|
||||
- Drop a candidate that merely restates a wiki node that already exists in the
|
||||
inventory: an extend must add a refinement or a new adjacent node.
|
||||
- Drop one the user has no visible interest in, so there is nothing to extend.
|
||||
- Drop one that is a to-do or a schedule rather than content worth introducing.
|
||||
- Move to new_idea only when the candidate is a piece of work to delegate
|
||||
rather than content to explain.
|
||||
|
||||
refine_checks_interest_extend_zh: |
|
||||
- 若候选只是复述清单里已存在的 wiki 节点，drop：extend 必须带来细化或新的
|
||||
相邻节点。
|
||||
- 若看不到用户的相关兴趣、无从延伸，drop。
|
||||
- 若它其实是一条待办或日程，而不是值得介绍的内容，drop。
|
||||
- 仅当候选是"可委托的一件工作"而不是"可讲解的内容"时，才 move 到 new_idea。
|
||||
|
|
|
|||
106
reme/steps/evolve/proactive/feedback.py
Normal file
106
reme/steps/evolve/proactive/feedback.py
Normal file
|
|
@ -0,0 +1,106 @@
|
|||
"""Proactive delivery feedback step: record what a host agent already surfaced.
|
||||
|
||||
The refresh chain decides what is worth raising; it cannot know what the host
|
||||
actually raised. This step closes that loop by accepting a small report from
|
||||
the consumer and appending it to the ``deliveries`` section of the truth source
|
||||
``daily/_proactive.yaml``. Two readers benefit:
|
||||
|
||||
- ``proactive_step`` withholds agenda items still inside the cooldown window,
|
||||
so the same proposal is not surfaced twice;
|
||||
- ``proactive_agenda_step`` suppresses those candidates deterministically and
|
||||
records an auditable reason instead of re-editing them into today's agenda.
|
||||
|
||||
Reporting is optional. A framework that never calls this step leaves
|
||||
``deliveries`` empty and both readers behave exactly as they did before, which
|
||||
keeps the contract additive rather than a behavioural change for every host.
|
||||
|
||||
The step is deliberately narrow: it never calls an LLM, never touches
|
||||
``open_topics``/``resolved``/``schedules``, never rewrites ``interests.yaml``
|
||||
and never advances a file catalog. It only appends feedback.
|
||||
"""
|
||||
|
||||
import time
|
||||
|
||||
from ...base_step import BaseStep
|
||||
from ....components import R
|
||||
from ..dream.utils import daily_dir, today, workspace_dir
|
||||
from .utils import (
|
||||
current_now,
|
||||
load_state,
|
||||
save_state,
|
||||
suppressed_delivery_ids,
|
||||
trim_deliveries,
|
||||
upsert_deliveries,
|
||||
)
|
||||
|
||||
|
||||
@R.register("proactive_feedback_step")
|
||||
class ProactiveFeedbackStep(BaseStep):
|
||||
"""Upsert host-reported agenda deliveries into the truth source.
|
||||
|
||||
Records are idempotent on ``(topic_id, external_ref)`` when the host
|
||||
supplies a reference and on ``(topic_id, agenda_date, channel)``
|
||||
otherwise, so a retry of the same report updates in place. A terminal
|
||||
status (``executed``/``dismissed``) is never downgraded by a later
|
||||
``shown`` report; ``failed`` never blocks re-delivery.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
retention_days: int = 180,
|
||||
max_deliveries: int = 500,
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.retention_days = max(int(retention_days), 1)
|
||||
self.max_deliveries = max(int(max_deliveries), 1)
|
||||
|
||||
async def execute(self):
|
||||
assert self.context is not None
|
||||
started = time.monotonic()
|
||||
raw_items = self.context.get("items")
|
||||
items = list(raw_items) if isinstance(raw_items, (list, tuple)) else []
|
||||
day = today(self, str(self.context.get("date", "") or ""))
|
||||
daily = daily_dir(self)
|
||||
ws = workspace_dir(self)
|
||||
self.logger.info(f"[{self.name}] start day={day} items={len(items)}")
|
||||
|
||||
if not items:
|
||||
self.context.response.success = True
|
||||
self.context.response.answer = "Feedback: no delivery items reported"
|
||||
self.context.response.metadata["proactive_feedback"] = {
|
||||
"date": day,
|
||||
"received": 0,
|
||||
"added": 0,
|
||||
"updated": 0,
|
||||
"dropped": 0,
|
||||
"trimmed": 0,
|
||||
}
|
||||
return self.context.response
|
||||
|
||||
state_file, _needs_bootstrap = load_state(ws, daily)
|
||||
counts = upsert_deliveries(state_file, items, day, current_now(self))
|
||||
trimmed = trim_deliveries(state_file, day, self.retention_days, self.max_deliveries)
|
||||
await save_state(ws, state_file, daily)
|
||||
|
||||
suppressed = suppressed_delivery_ids(state_file, day, self.retention_days)
|
||||
payload = {
|
||||
"date": day,
|
||||
"received": len(items),
|
||||
"added": counts["added"],
|
||||
"updated": counts["updated"],
|
||||
"dropped": counts["dropped"],
|
||||
"trimmed": trimmed,
|
||||
"total": len(state_file.deliveries),
|
||||
"suppressed_topics": sorted(suppressed),
|
||||
}
|
||||
self.context.response.success = True
|
||||
self.context.response.answer = (
|
||||
f"Feedback: added {counts['added']}, updated {counts['updated']}, "
|
||||
f"dropped {counts['dropped']}, trimmed {trimmed}; "
|
||||
f"{len(state_file.deliveries)} record(s) on file"
|
||||
)
|
||||
self.context.response.metadata["proactive_feedback"] = payload
|
||||
self.context.response.metadata["duration_ms"] = int((time.monotonic() - started) * 1000)
|
||||
self.logger.info(f"[{self.name}] finish {payload}")
|
||||
return self.context.response
|
||||
|
|
@ -42,7 +42,7 @@ class ProactiveFinishStep(BaseStep):
|
|||
return self.context.response
|
||||
ws = workspace_dir(self)
|
||||
|
||||
checkpoint = [rel for rel in state.changed_paths if (ws / rel).is_file()]
|
||||
checkpoint = [rel for rel in (*state.changed_paths, *state.resource_changed) if (ws / rel).is_file()]
|
||||
snapshot = state.changed_mtimes
|
||||
nodes: list[FileNode] = []
|
||||
deferred = 0
|
||||
|
|
|
|||
|
|
@ -1,22 +1,27 @@
|
|||
"""Proactive plan step: expand push candidates into scenario cards (F2.5).
|
||||
|
||||
Runs between the topics and agenda steps. Only topics that qualify for today's
|
||||
push (``first_seen == today`` and ``confidence >= min_push_confidence``,
|
||||
computed by the topics step into ``state.push_candidates``) are expanded, so
|
||||
the LLM cost stays proportional to what will actually be surfaced. One batched
|
||||
LLM call per round; on any LLM failure every selected candidate still gets a
|
||||
deterministic fallback card so the agenda step never sees an empty input.
|
||||
push (computed by the topics step into ``state.push_candidates``) are expanded,
|
||||
so the LLM cost stays proportional to what will actually be surfaced. One
|
||||
batched LLM call per round; on any LLM failure every selected candidate still
|
||||
gets a deterministic fallback card so the agenda step never sees an empty input.
|
||||
|
||||
Card contract: ``scenario_type`` (resume_task | answer_pending |
|
||||
explore_interest | prepare_upcoming), ``opener`` (a casual, friend-like
|
||||
conversation opener that already names the minimal next action - never a
|
||||
"based on your records" notification tone), ``next_action``, ``preconditions``
|
||||
and ``delivery`` (in_conversation | notification | agenda_item).
|
||||
``linked_memory`` is derived from the topic's paths/evidence by code, never by
|
||||
the LLM.
|
||||
explore_interest | prepare_upcoming | propose_delegation), ``opener`` (a casual,
|
||||
friend-like conversation opener that already names the minimal next action -
|
||||
never a "based on your records" notification tone, and never a stock lead-in
|
||||
worn into a template), ``next_action``,
|
||||
``preconditions`` and ``delivery`` (in_conversation | notification |
|
||||
agenda_item). ``linked_memory`` is derived from the topic's paths/evidence by
|
||||
code, never by the LLM.
|
||||
|
||||
Reminders are expanded first and are never squeezed out by ``max_plan_topics``
|
||||
(they are bounded by ``max_reminder_cards`` instead), because the agenda step
|
||||
pins every reminder card it receives.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import time
|
||||
|
||||
|
|
@ -24,17 +29,50 @@ from ...base_step import BaseStep
|
|||
from .._evolve import agent_reply_result_text, passthrough_response
|
||||
from ....components import R
|
||||
from ....schema import ProactiveState
|
||||
from ....schema.proactive import DEFAULT_KIND
|
||||
from ..dream.utils import workspace_dir
|
||||
from .utils import load_personal_profile_block, parse_plan_reply, resolve_agent_wrapper
|
||||
|
||||
SCENARIO_TYPES = ("resume_task", "answer_pending", "explore_interest", "prepare_upcoming")
|
||||
SCENARIO_TYPES = ("resume_task", "answer_pending", "explore_interest", "prepare_upcoming", "propose_delegation")
|
||||
DELIVERY_MODES = ("in_conversation", "notification", "agenda_item")
|
||||
DEFAULT_SCENARIO_BY_KIND = {"follow_up": "resume_task", "interest_extend": "explore_interest"}
|
||||
DEFAULT_SCENARIO_BY_KIND = {
|
||||
"follow_up": "resume_task",
|
||||
"interest_extend": "explore_interest",
|
||||
"reminder": "prepare_upcoming",
|
||||
"new_idea": "propose_delegation",
|
||||
}
|
||||
MAX_OPENER_CHARS = 300
|
||||
MAX_NEXT_ACTION_CHARS = 200
|
||||
MAX_PRECONDITIONS = 5
|
||||
MAX_PRECONDITION_CHARS = 80
|
||||
|
||||
# Deterministic openers, several per kind, so a batch of fallback cards does not
|
||||
# open every line the same way - one repeated lead-in reads as a script. ``{when}``
|
||||
# is the parenthesised due date (empty when unknown) and only reminder variants
|
||||
# carry it, because a reminder may not hide when it is due.
|
||||
FALLBACK_OPENERS = {
|
||||
"reminder": (
|
||||
"「{title}」{when}就快到了，要不要现在把准备事项先过一遍？",
|
||||
"时间快到了：「{title}」{when}，先确认下会不会和别的安排撞上？",
|
||||
"有件事排在前面了：「{title}」{when}，现在动手准备最从容。",
|
||||
),
|
||||
"new_idea": (
|
||||
"有件活儿我觉得可以替你办：「{title}」，要不要我先出个方案你看看？",
|
||||
"「{title}」这件事其实可以直接交给我，要不要我先出一版草案给你审？",
|
||||
"这活儿不必你自己来：「{title}」，我先跑个初稿你看看值不值？",
|
||||
),
|
||||
"follow_up": (
|
||||
"上次聊到「{title}」，好像还没收尾——要不要趁现在花几分钟往前推一步？",
|
||||
"「{title}」还挂着，要不要我先去查一下最新进展，再回来跟你说下一步？",
|
||||
"那件「{title}」好像只差临门一脚了，现在顺手推一下？",
|
||||
),
|
||||
"interest_extend": (
|
||||
"感觉你最近可能会想看看「{title}」，有空的话可以先从相关材料扫一眼。",
|
||||
"「{title}」这块还能再往深挖一层，要不要我挑两处给你展开讲讲？",
|
||||
"顺着「{title}」又翻到些延伸内容，值得的话我带你过一遍？",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def linked_memory(candidate: dict) -> list[str]:
|
||||
"""Workspace paths related to a topic, derived from paths + evidence."""
|
||||
|
|
@ -51,15 +89,22 @@ def linked_memory(candidate: dict) -> list[str]:
|
|||
|
||||
|
||||
def fallback_card(candidate: dict) -> dict:
|
||||
"""Deterministic card used when the LLM is absent or its output is unusable."""
|
||||
kind = str(candidate.get("kind") or "interest_extend")
|
||||
"""Deterministic card used when the LLM is absent or its output is unusable.
|
||||
|
||||
Its opener is picked per topic out of ``FALLBACK_OPENERS``, so the
|
||||
no-LLM path honours the same "no repeated lead-in" rule the prompt states.
|
||||
"""
|
||||
kind = str(candidate.get("kind") or DEFAULT_KIND)
|
||||
title = str(candidate.get("title") or "")
|
||||
scenario = DEFAULT_SCENARIO_BY_KIND.get(kind, "explore_interest")
|
||||
if kind == "follow_up":
|
||||
opener = f"上次聊到「{title}」，好像还没收尾——要不要趁现在花几分钟往前推一步？"
|
||||
scenario = DEFAULT_SCENARIO_BY_KIND.get(kind, DEFAULT_SCENARIO_BY_KIND[DEFAULT_KIND])
|
||||
opener = _fallback_opener(kind, candidate, title)
|
||||
if kind == "reminder":
|
||||
next_action = "确认时间、所需材料与是否和其它安排冲突，需要改期就现在说"
|
||||
elif kind == "new_idea":
|
||||
next_action = "确认值不值得委托；值得的话由 agent 先产出一版草案供你审阅"
|
||||
elif kind == "follow_up":
|
||||
next_action = "先回顾相关记录，确认卡点，然后给出下一步动作"
|
||||
else:
|
||||
opener = f"感觉你最近可能会想看看「{title}」，有空的话可以先从相关材料扫一眼。"
|
||||
next_action = "快速浏览相关材料，判断值不值得深入"
|
||||
memory = linked_memory(candidate)
|
||||
if memory:
|
||||
|
|
@ -73,6 +118,20 @@ def fallback_card(candidate: dict) -> dict:
|
|||
}
|
||||
|
||||
|
||||
def _fallback_opener(kind: str, candidate: dict, title: str) -> str:
|
||||
"""Pick one opener variant for this kind, seeded by the topic identity.
|
||||
|
||||
The seed is the topic id, so one topic keeps one opener across rounds
|
||||
instead of flip-flopping, while different topics in the same round land on
|
||||
different variants and the agenda does not read as one repeated sentence.
|
||||
"""
|
||||
variants = FALLBACK_OPENERS.get(kind) or FALLBACK_OPENERS[DEFAULT_KIND]
|
||||
seed = str(candidate.get("id") or title)
|
||||
picked = variants[int(hashlib.sha1(seed.encode("utf-8")).hexdigest(), 16) % len(variants)]
|
||||
due = str(candidate.get("due_at") or "").strip()
|
||||
return picked.format(title=title, when=f"（{due}）" if due else "")
|
||||
|
||||
|
||||
def _clean_card(raw: dict, candidate: dict) -> dict:
|
||||
"""Validate one LLM card; per-field fallback keeps the card always usable."""
|
||||
base = fallback_card(candidate)
|
||||
|
|
@ -105,14 +164,19 @@ def _clean_card(raw: dict, candidate: dict) -> dict:
|
|||
class ProactivePlanStep(BaseStep):
|
||||
"""Expand today's push candidates into scenario cards.
|
||||
|
||||
Zero LLM calls when there are no push candidates. At most
|
||||
``max_plan_topics`` candidates are expanded; the overflow stays card-less
|
||||
and the agenda step suppresses it with an explicit reason.
|
||||
Zero LLM calls when there are no push candidates. The two budgets are
|
||||
independent: at most ``max_reminder_cards`` reminders plus
|
||||
``max_plan_topics`` discovery candidates are expanded. A reminder-heavy day
|
||||
therefore cannot starve the discovery kinds of cards, because reminders are
|
||||
pinned into the agenda later and would otherwise take every slot with them.
|
||||
The overflow stays card-less and the agenda step suppresses it with an
|
||||
explicit reason.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
max_plan_topics: int = 6,
|
||||
max_reminder_cards: int = 8,
|
||||
llm_timeout_seconds: float = 120,
|
||||
profile_max_chars: int = 800,
|
||||
skip_key: str = "proactive_skip",
|
||||
|
|
@ -120,6 +184,7 @@ class ProactivePlanStep(BaseStep):
|
|||
):
|
||||
super().__init__(**kwargs)
|
||||
self.max_plan_topics = max(int(max_plan_topics), 1)
|
||||
self.max_reminder_cards = max(int(max_reminder_cards), 1)
|
||||
self.llm_timeout_seconds = float(llm_timeout_seconds)
|
||||
self.profile_max_chars = max(int(profile_max_chars), 0)
|
||||
self.skip_key = skip_key
|
||||
|
|
@ -149,10 +214,15 @@ class ProactivePlanStep(BaseStep):
|
|||
self.logger.info(f"[{self.name}] no push candidates; skipping")
|
||||
return self.context.response
|
||||
|
||||
selected = candidates[: self.max_plan_topics]
|
||||
# Reminders first and outside the discovery budget: they pin today's
|
||||
# agenda, so starving them here would silently break that guarantee.
|
||||
reminders = [c for c in candidates if str(c.get("kind")) == "reminder"][: self.max_reminder_cards]
|
||||
others = [c for c in candidates if str(c.get("kind")) != "reminder"]
|
||||
selected = reminders + others[: self.max_plan_topics]
|
||||
self.logger.info(
|
||||
f"[{self.name}] start candidates={len(candidates)} selected={len(selected)} "
|
||||
f"max_plan_topics={self.max_plan_topics}",
|
||||
f"reminders={len(reminders)} max_plan_topics={self.max_plan_topics} "
|
||||
f"max_reminder_cards={self.max_reminder_cards}",
|
||||
)
|
||||
cards_by_id = await self._llm_cards(state, selected)
|
||||
|
||||
|
|
@ -167,7 +237,7 @@ class ProactivePlanStep(BaseStep):
|
|||
card = {
|
||||
"topic_id": cid,
|
||||
"title": str(candidate.get("title") or ""),
|
||||
"kind": str(candidate.get("kind") or "interest_extend"),
|
||||
"kind": str(candidate.get("kind") or DEFAULT_KIND),
|
||||
**card,
|
||||
"linked_memory": linked_memory(candidate),
|
||||
}
|
||||
|
|
@ -203,6 +273,8 @@ class ProactivePlanStep(BaseStep):
|
|||
"confidence": c.get("confidence"),
|
||||
"first_seen": c.get("first_seen"),
|
||||
"evidence": c.get("evidence"),
|
||||
**({"due_at": c.get("due_at")} if c.get("due_at") else {}),
|
||||
**({"schedule_id": c.get("schedule_id")} if c.get("schedule_id") else {}),
|
||||
}
|
||||
for c in selected
|
||||
],
|
||||
|
|
|
|||
|
|
@ -12,25 +12,48 @@ plan_system_prompt: |
|
|||
- `resume_task`: a task that was started but interrupted/postponed;
|
||||
- `answer_pending`: a question that was asked but never answered;
|
||||
- `explore_interest`: an interest the user has not dived into yet;
|
||||
- `prepare_upcoming`: something with a future time anchor to prepare for.
|
||||
follow_up topics become resume_task / answer_pending / prepare_upcoming;
|
||||
interest_extend topics become explore_interest / prepare_upcoming. Pick by
|
||||
semantics; use a time anchor whenever the topic carries one.
|
||||
- `opener`: one casual spoken sentence, like a friend happening to mention
|
||||
it ("by the way..."). STRICTLY forbidden: notification tone such as
|
||||
"based on your previous records", "the system reminds you", "as you
|
||||
mentioned on <date>". The opener must already point at the minimal
|
||||
executable next action, so it invites doing, not just knowing.
|
||||
- `prepare_upcoming`: something with a future time anchor to prepare for;
|
||||
- `propose_delegation`: a piece of work worth handing over to the agent.
|
||||
Map by kind first, then refine by semantics:
|
||||
- `follow_up` -> resume_task / answer_pending; the card offers to fetch the
|
||||
latest status and report back to the user;
|
||||
- `interest_extend` -> explore_interest; the card offers to walk the user
|
||||
through a refinement or extension of a topic they care about;
|
||||
- `reminder` -> prepare_upcoming, naming the due date (and the clash to
|
||||
resolve when the topic mentions one);
|
||||
- `new_idea` -> propose_delegation, naming the deliverable the agent would
|
||||
produce - an offer, never a report of work already done.
|
||||
- `opener`: one casual spoken sentence a friend would actually say. Two
|
||||
failure modes are STRICTLY forbidden:
|
||||
1. Notification tone: "based on your previous records", "the system
|
||||
reminds you", "as you mentioned on <date>".
|
||||
2. A stock lead-in worn into a template. Never open with "by the way..."
|
||||
or any other fixed filler ("just a reminder...", "quick note..."):
|
||||
the same first words every day read as a script, not as a person.
|
||||
Start from the substance instead and change the angle card by card - the
|
||||
date or the clash, what moved since last time, the deliverable the agent
|
||||
would produce, a direct question, or whatever the user is working on right
|
||||
now. Inside one batch no two openers may share their opening words.
|
||||
The opener must already point at the minimal executable next action, so it
|
||||
invites doing, not just knowing.
|
||||
A reminder opener still sounds like a person, but it may not hide the date:
|
||||
the user has to hear when it is due.
|
||||
- `next_action`: the smallest concrete step the user can start right now.
|
||||
- `preconditions`: list of things that must hold first; empty list is fine.
|
||||
- `delivery`: `in_conversation` (weave into talk), `notification`
|
||||
(standalone nudge) or `agenda_item` (belongs on today's agenda).
|
||||
Reminders with a near or clashing due date belong on `notification` or
|
||||
`agenda_item`, not buried in conversation.
|
||||
|
||||
## Hard rules
|
||||
|
||||
- Exactly one card per input topic, no extras.
|
||||
- Ground everything in the given reason/evidence; never invent facts.
|
||||
- Proposals stay proposals: never word a card as if the work were done.
|
||||
- Openers vary: one lead-in formula per batch is a defect, not a style.
|
||||
- Quote YAML strings containing punctuation such as `:`.
|
||||
- Write `opener` and `next_action` in the language the user's own material is
|
||||
written in, so the assistant never switches tongues mid-conversation.
|
||||
|
||||
## Output format
|
||||
|
||||
|
|
@ -38,8 +61,8 @@ plan_system_prompt: |
|
|||
```yaml
|
||||
cards:
|
||||
- topic_id: <echoed id>
|
||||
scenario_type: resume_task|answer_pending|explore_interest|prepare_upcoming
|
||||
opener: <one casual sentence pointing at the next action>
|
||||
scenario_type: resume_task|answer_pending|explore_interest|prepare_upcoming|propose_delegation
|
||||
opener: <one casual sentence, substance first, pointing at the next action>
|
||||
next_action: <smallest executable step>
|
||||
preconditions: [<condition>, ...]
|
||||
delivery: in_conversation|notification|agenda_item
|
||||
|
|
@ -53,27 +76,47 @@ plan_system_prompt_zh: |
|
|||
## 每张卡的字段
|
||||
|
||||
- `topic_id`：原样回显给定 id。
|
||||
- `scenario_type`：四选一
|
||||
- `scenario_type`：五选一
|
||||
- `resume_task`：开始过但被中断/搁置的任务；
|
||||
- `answer_pending`：提出过但没有得到回答的问题；
|
||||
- `explore_interest`：用户尚未深入的兴趣方向；
|
||||
- `prepare_upcoming`：带未来时间点、需要提前准备的事项。
|
||||
follow_up 主题对应 resume_task / answer_pending / prepare_upcoming；
|
||||
interest_extend 主题对应 explore_interest / prepare_upcoming。按语义选择；
|
||||
主题带时间线索时优先 prepare_upcoming。
|
||||
- `opener`：一句口语化的开场，像朋友随口提起（"对了，上次那个……"）。
|
||||
严禁通知腔："根据您之前的记录"、"系统提醒您"、"您在某天提到过"等。
|
||||
- `prepare_upcoming`：带未来时间点、需要提前准备的事项；
|
||||
- `propose_delegation`：一件值得委托给 agent 去办的工作。
|
||||
先按 kind 映射，再按语义细化：
|
||||
- `follow_up` -> resume_task / answer_pending（卡片要表达"我去查最新进展
|
||||
再向你汇报"）；
|
||||
- `interest_extend` -> explore_interest（卡片要表达"带你看看这个主题的
|
||||
细化/延伸内容"）；
|
||||
- `reminder` -> prepare_upcoming，必须点明到期时间，若主题提到冲突还要
|
||||
点明要先协调哪一件事；
|
||||
- `new_idea` -> propose_delegation，说清 agent 会交付什么——是邀约，
|
||||
绝不能写成已经做完的汇报。
|
||||
- `opener`：一句朋友真会说出口的口语化开场。以下两种写法都严禁：
|
||||
1. 通知腔："根据您之前的记录"、"系统提醒您"、"您在某天提到过"等；
|
||||
2. 把某个口头禅当模板反复套用。不要一律以"对了……"开头，也不要换成
|
||||
"顺便说一下"、"提醒您一下"之类别的固定引子——每天都是同一句开头，
|
||||
就成了脚本，不像人。
|
||||
直接从内容本身切入，并且逐卡换角度：先说时间点或撞期、先说上次之后有
|
||||
什么变化、先说 agent 会交付什么、直接抛一个问题，或者从用户眼下正在做
|
||||
的事接过去。同一批卡片里，任意两张开场的开头用词都不得相同。
|
||||
开场必须顺势给出最小可执行的下一步，让人愿意动手，而不只是知道。
|
||||
reminder 的开场同样要像人话，但不能藏时间：用户必须听到"什么时候到期"。
|
||||
- `next_action`：用户现在就能开始的最小具体动作。
|
||||
- `preconditions`：需要先满足的条件列表，可以为空。
|
||||
- `delivery`：`in_conversation`（对话中自然带出）/ `notification`
|
||||
（单独提醒）/ `agenda_item`（列入当日议程）。
|
||||
临近到期或存在冲突的 reminder 应用 `notification` 或 `agenda_item`，
|
||||
不要埋在闲聊里。
|
||||
|
||||
## 硬约束
|
||||
|
||||
- 每个输入主题恰好一张卡，不得多给。
|
||||
- 所有内容必须基于给定的 reason/evidence，禁止编造事实。
|
||||
- 提议就是提议：任何卡片都不得写成工作已经完成。
|
||||
- 开场白必须各不相同：整批共用一个引子是缺陷，不是风格。
|
||||
- 包含 `:` 等标点的 YAML 字符串请加引号。
|
||||
- `opener` 与 `next_action` 使用用户素材本身所用的语言书写，避免助手在对话
|
||||
中途切换语言。
|
||||
|
||||
## 输出格式
|
||||
|
||||
|
|
@ -81,8 +124,8 @@ plan_system_prompt_zh: |
|
|||
```yaml
|
||||
cards:
|
||||
- topic_id: <回显的 id>
|
||||
scenario_type: resume_task|answer_pending|explore_interest|prepare_upcoming
|
||||
opener: <一句口语化开场，顺势给出下一步动作>
|
||||
scenario_type: resume_task|answer_pending|explore_interest|prepare_upcoming|propose_delegation
|
||||
opener: <一句口语化开场，直接从内容切入，顺势给出下一步动作>
|
||||
next_action: <最小可执行步骤>
|
||||
preconditions: [<条件>, ...]
|
||||
delivery: in_conversation|notification|agenda_item
|
||||
|
|
|
|||
|
|
@ -6,6 +6,11 @@ fallback) and ``horizon_days``: horizon=1 keeps the legacy single-day file
|
|||
read (v1 compatible); horizon>1 reads the truth source directly, filtering by
|
||||
``last_evidence_at`` recency (v5 R4 - the truth source already carries the
|
||||
cross-day merge, so the reader no longer re-scans exposure products).
|
||||
|
||||
Agenda items the host already reported through ``proactive_feedback`` are
|
||||
withheld while they sit inside ``delivery_cooldown_days``, so a consumer that
|
||||
reads this job daily does not resurface the same proposal. Feedback is
|
||||
optional: with an empty ``deliveries`` section nothing is withheld.
|
||||
"""
|
||||
|
||||
import datetime as dt
|
||||
|
|
@ -23,10 +28,27 @@ from .utils import (
|
|||
parse_interests_topics,
|
||||
sort_topics,
|
||||
state_file_path,
|
||||
suppressed_delivery_ids,
|
||||
topic_id,
|
||||
)
|
||||
|
||||
|
||||
def _above_floor(topic, min_confidence: float) -> bool:
|
||||
"""Confidence floor check; clock-anchored reminders are exempt.
|
||||
|
||||
A reminder carrying a due value is a clock fact about the user's own
|
||||
schedule, not a discovery guess, and the refresh chain already applied the
|
||||
(lower) ``min_reminder_confidence`` when it pinned it to the agenda.
|
||||
Filtering it again here would silently drop today's agenda item. A dateless
|
||||
reminder - an external notice or a key event - is a judgement call, so it
|
||||
still has to clear the reader's floor.
|
||||
"""
|
||||
get = topic.get if isinstance(topic, dict) else (lambda key: getattr(topic, key, ""))
|
||||
if str(get("kind") or "") == "reminder" and str(get("due_at") or "").strip():
|
||||
return True
|
||||
return float(get("confidence") or 0.0) >= min_confidence - 1e-9
|
||||
|
||||
|
||||
@R.register("proactive_step")
|
||||
class ProactiveStep(BaseStep):
|
||||
"""Read ``daily/<date>/interests.yaml`` (optionally merged over a horizon).
|
||||
|
|
@ -37,11 +59,21 @@ class ProactiveStep(BaseStep):
|
|||
- ``horizon_days>1`` reads the truth source and filters by evidence recency.
|
||||
"""
|
||||
|
||||
def __init__(self, include_content: bool = True, horizon_days: int = 1, min_confidence: float = 0.4, **kwargs):
|
||||
def __init__(
|
||||
self,
|
||||
include_content: bool = True,
|
||||
horizon_days: int = 1,
|
||||
min_confidence: float = 0.4,
|
||||
include_delivered: bool = False,
|
||||
delivery_cooldown_days: int = 3,
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.include_content = include_content
|
||||
self.horizon_days = max(int(horizon_days), 1)
|
||||
self.min_confidence = float(min_confidence)
|
||||
self.include_delivered = include_delivered
|
||||
self.delivery_cooldown_days = max(int(delivery_cooldown_days), 0)
|
||||
|
||||
async def execute(self):
|
||||
assert self.context is not None
|
||||
|
|
@ -54,27 +86,37 @@ class ProactiveStep(BaseStep):
|
|||
min_confidence = float(raw_min_confidence)
|
||||
except (TypeError, ValueError):
|
||||
min_confidence = self.min_confidence
|
||||
include_delivered = bool(self.context.get("include_delivered", self.include_delivered))
|
||||
raw_cooldown = self.context.get("delivery_cooldown_days", self.delivery_cooldown_days)
|
||||
try:
|
||||
cooldown = max(int(raw_cooldown), 0)
|
||||
except (TypeError, ValueError):
|
||||
cooldown = self.delivery_cooldown_days
|
||||
daily = self.config_value("daily_dir")
|
||||
ws = workspace_dir(self)
|
||||
blocked = {} if include_delivered else self._suppressed_deliveries(ws, daily, day, cooldown)
|
||||
default_rel = f"{daily}/{day}/interests.yaml"
|
||||
result = ProactiveResult(date=day, path=default_rel)
|
||||
self.logger.info(
|
||||
f"[{self.name}] start date={day} path={default_rel} include_content={include_content} "
|
||||
f"horizon_days={horizon} min_confidence={min_confidence}",
|
||||
f"horizon_days={horizon} min_confidence={min_confidence} "
|
||||
f"include_delivered={include_delivered} delivery_cooldown_days={cooldown} blocked={len(blocked)}",
|
||||
)
|
||||
|
||||
if horizon == 1:
|
||||
outcome = self._read_single(ws, daily, day, include_content, min_confidence, result)
|
||||
outcome = self._read_single(ws, daily, day, include_content, min_confidence, result, blocked)
|
||||
if outcome is not None:
|
||||
return outcome
|
||||
|
||||
# The horizon view reads open topics from the truth source and carries
|
||||
# no agenda, so delivery withholding does not apply to it.
|
||||
return self._read_horizon(ws, daily, day, horizon, include_content, min_confidence, result)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Single-day read (legacy-compatible path)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _read_single(self, ws, daily, day, include_content, min_confidence, result: ProactiveResult):
|
||||
def _read_single(self, ws, daily, day, include_content, min_confidence, result: ProactiveResult, blocked=None):
|
||||
rel_path = f"{daily}/{day}/interests.yaml"
|
||||
abs_path = ws / daily / day / "interests.yaml"
|
||||
if not abs_path.is_file():
|
||||
|
|
@ -119,19 +161,20 @@ class ProactiveStep(BaseStep):
|
|||
kept = [
|
||||
dump_topic(t)
|
||||
for t in sort_topics(topics)
|
||||
if t.id not in resolved_ids and t.confidence >= min_confidence - 1e-9
|
||||
if t.id not in resolved_ids and _above_floor(t, min_confidence)
|
||||
]
|
||||
result.topics = kept
|
||||
agenda_raw = data.get("agenda")
|
||||
if isinstance(agenda_raw, list):
|
||||
kept_ids = {str(t.get("id") or "") for t in kept}
|
||||
result.agenda = [
|
||||
item
|
||||
for item in agenda_raw
|
||||
if isinstance(item, dict) and str(item.get("topic_id") or "") in kept_ids
|
||||
]
|
||||
result.agenda, result.delivered_ids = self._select_agenda(agenda_raw, kept_ids, blocked)
|
||||
result.summary = f"Read {len(result.topics)} proactive topic(s) from {rel_path}"
|
||||
self.logger.info(f"[{self.name}] read done path={rel_path} topics={len(result.topics)}")
|
||||
if result.delivered_ids:
|
||||
result.summary += f"; withheld {len(result.delivered_ids)} already-delivered agenda item(s)"
|
||||
self.logger.info(
|
||||
f"[{self.name}] read done path={rel_path} topics={len(result.topics)} "
|
||||
f"agenda={len(result.agenda)} withheld={len(result.delivered_ids)}",
|
||||
)
|
||||
return self._finish(True, result, include_content=include_content)
|
||||
except Exception as e: # noqa: BLE001
|
||||
result.error, result.summary = f"{type(e).__name__}: {e}", ""
|
||||
|
|
@ -142,7 +185,16 @@ class ProactiveStep(BaseStep):
|
|||
# Multi-day merge (horizon_days > 1, or fallback target)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _read_horizon(self, ws, daily, day, horizon, include_content, min_confidence, result: ProactiveResult):
|
||||
def _read_horizon(
|
||||
self,
|
||||
ws,
|
||||
daily,
|
||||
day,
|
||||
horizon,
|
||||
include_content,
|
||||
min_confidence,
|
||||
result: ProactiveResult,
|
||||
):
|
||||
"""Truth-source view (v5 R4): open topics with evidence inside the horizon.
|
||||
|
||||
The truth source already carries the cross-day merge (carry-forward),
|
||||
|
|
@ -171,7 +223,7 @@ class ProactiveStep(BaseStep):
|
|||
for t in state_file.open_topics
|
||||
if (
|
||||
(not cutoff or cutoff <= str(t.last_evidence_at or "") <= upper_bound)
|
||||
and t.confidence >= min_confidence - 1e-9
|
||||
and _above_floor(t, min_confidence)
|
||||
)
|
||||
]
|
||||
kept = sort_topics(kept)
|
||||
|
|
@ -185,6 +237,37 @@ class ProactiveStep(BaseStep):
|
|||
state_file, _needs_bootstrap = load_state(ws, daily)
|
||||
return {str(r.get("id") or "") for r in state_file.resolved if isinstance(r, dict) and r.get("id")}
|
||||
|
||||
def _suppressed_deliveries(self, ws, daily: str, day: str, cooldown: int) -> dict:
|
||||
"""topic_id -> blocking delivery record; empty when nothing was reported."""
|
||||
state_file, _needs_bootstrap = load_state(ws, daily)
|
||||
if not state_file.deliveries:
|
||||
return {}
|
||||
return suppressed_delivery_ids(state_file, day, cooldown)
|
||||
|
||||
@staticmethod
|
||||
def _select_agenda(agenda_raw: list, kept_ids: set, blocked: dict | None) -> tuple[list, list]:
|
||||
"""Keep agenda items whose topic survived filtering and was not delivered.
|
||||
|
||||
The agenda order coming out of the refresh chain *is* the priority
|
||||
order (reminders pinned first), so this only ever removes items and
|
||||
never re-sorts them.
|
||||
"""
|
||||
blocked = blocked or {}
|
||||
agenda, withheld = [], []
|
||||
for item in agenda_raw:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
item_id = str(item.get("topic_id") or "")
|
||||
if item_id not in kept_ids:
|
||||
continue
|
||||
# Reminders stay exempt from the cooldown, matching the agenda
|
||||
# step: a due schedule is announced every time it comes due.
|
||||
if item_id in blocked and str(item.get("kind") or "") != "reminder":
|
||||
withheld.append(item_id)
|
||||
continue
|
||||
agenda.append(item)
|
||||
return agenda, withheld
|
||||
|
||||
def _finish(self, success: bool, result: ProactiveResult, *, include_content: bool):
|
||||
assert self.context is not None
|
||||
self.context.response.success = success
|
||||
|
|
@ -197,6 +280,7 @@ class ProactiveStep(BaseStep):
|
|||
"summary": result.summary,
|
||||
"topics": result.topics,
|
||||
**({"agenda": result.agenda} if result.agenda else {}),
|
||||
**({"delivered_ids": result.delivered_ids} if result.delivered_ids else {}),
|
||||
**({"content": result.content} if include_content else {}),
|
||||
}
|
||||
self.context.response.metadata.update(result.model_dump())
|
||||
|
|
|
|||
196
reme/steps/evolve/proactive/refine.py
Normal file
196
reme/steps/evolve/proactive/refine.py
Normal file
|
|
@ -0,0 +1,196 @@
|
|||
"""Stage-2 per-kind adjudication for proactive extraction (precision pass).
|
||||
|
||||
Stage 1 owns recall: one batched reply over the changed material produces the
|
||||
candidates of all four kinds. Stage 2 owns precision: every enabled kind that
|
||||
actually produced candidates gets one small, material-free call that re-checks
|
||||
those candidates against the kind's own contract plus the only context slice
|
||||
that kind needs (registry and clock for reminders, carry-forward topics for
|
||||
follow-ups, capability/wiki inventory for ideas and extends).
|
||||
|
||||
A pass may keep, drop or move a candidate and rescore its confidence. It never
|
||||
rewrites wording - stage 1 owns the user-facing text - and it never invents a
|
||||
candidate. Every failure mode is fail-open: a timeout or an unusable reply keeps
|
||||
the stage-1 candidates untouched, because losing a real finding costs more than
|
||||
keeping a borderline one.
|
||||
"""
|
||||
|
||||
from ....schema.proactive import TOPIC_KINDS, clamp_confidence
|
||||
from .utils import parse_fenced_yaml
|
||||
|
||||
# Refine order = proactive priority order, which keeps the audit log readable:
|
||||
# the highest-priority kind is adjudicated first.
|
||||
REFINE_ORDER = ("reminder", "follow_up", "new_idea", "interest_extend")
|
||||
DECISIONS = ("keep", "drop", "move")
|
||||
|
||||
# Fields only a reminder candidate carries. A move out of ``reminder`` has to
|
||||
# drop them, otherwise the receiving kind publishes a ``schedule_id`` pointing
|
||||
# at a registry entry that was never created plus a due date nothing fires on.
|
||||
REMINDER_ONLY_FIELDS = ("schedule_id", "due_at", "next_due", "trigger", "conflict_with")
|
||||
|
||||
|
||||
def _notice_anchored(item: dict, notice_paths: set[str]) -> bool:
|
||||
"""True when the candidate cites the external source (``resource/``) layer."""
|
||||
return bool({str(p) for p in (item.get("paths") or [])} & notice_paths)
|
||||
|
||||
|
||||
def _reminder_contract_met(item: dict, notice_paths: set[str]) -> bool:
|
||||
"""``clean_reminder``'s traceability rule: a due value or a notice anchor."""
|
||||
return bool(str(item.get("due_at") or "").strip()) or _notice_anchored(item, notice_paths)
|
||||
|
||||
|
||||
def candidates_by_kind(state) -> list[tuple[str, list[dict]]]:
|
||||
"""``(kind, bucket)`` pairs in refine order, skipping empty buckets."""
|
||||
buckets = kind_buckets(state)
|
||||
return [(kind, buckets[kind]) for kind in REFINE_ORDER if buckets[kind]]
|
||||
|
||||
|
||||
def kind_buckets(state) -> dict[str, list[dict]]:
|
||||
"""Map each kind to the ``ProactiveState`` bucket holding its candidates."""
|
||||
return {
|
||||
"reminder": list(state.reminders),
|
||||
"follow_up": list(state.follow_ups),
|
||||
"new_idea": list(state.new_ideas),
|
||||
"interest_extend": list(state.extends),
|
||||
}
|
||||
|
||||
|
||||
def candidate_payload(bucket: list[dict]) -> list[dict]:
|
||||
"""Candidate fields worth adjudicating; no material, paths trimmed."""
|
||||
out = []
|
||||
for item in bucket:
|
||||
payload = {
|
||||
"id": str(item.get("id") or ""),
|
||||
"title": str(item.get("title") or ""),
|
||||
"reason": str(item.get("reason") or ""),
|
||||
"confidence": clamp_confidence(item.get("confidence")),
|
||||
"paths": [str(p) for p in (item.get("paths") or [])][:3],
|
||||
}
|
||||
for key in ("due_at", "schedule_id", "trigger"):
|
||||
if str(item.get(key) or "").strip():
|
||||
payload[key] = str(item.get(key))
|
||||
conflicts = item.get("conflict_with")
|
||||
if isinstance(conflicts, list) and conflicts:
|
||||
payload["conflict_with"] = [str(c) for c in conflicts][:3]
|
||||
out.append(payload)
|
||||
return out
|
||||
|
||||
|
||||
def parse_refine_reply(text: str) -> list[dict] | None:
|
||||
"""Verdict list from a refine reply; ``None`` when the reply is unusable.
|
||||
|
||||
An empty list is a valid reply ("keep everything"); only a missing or
|
||||
non-list ``verdicts`` key counts as unusable, which lets the caller keep
|
||||
the stage-1 candidates instead of silently dropping a whole kind.
|
||||
"""
|
||||
payload = parse_fenced_yaml(text)
|
||||
if "verdicts" not in payload:
|
||||
return None
|
||||
verdicts = payload["verdicts"]
|
||||
if verdicts is None:
|
||||
# `verdicts:` with nothing beneath it is how YAML spells the empty list,
|
||||
# and an unmentioned candidate is kept - so this is the documented
|
||||
# "nothing to change" reply, not a broken one. Counting it as unusable
|
||||
# would flag the most common clean round as a degrade and hide a real
|
||||
# stage-2 failure behind the noise.
|
||||
return []
|
||||
if not isinstance(verdicts, list):
|
||||
return None
|
||||
out = []
|
||||
for raw in verdicts:
|
||||
if not isinstance(raw, dict):
|
||||
continue
|
||||
cid = str(raw.get("id") or "").strip()
|
||||
if not cid:
|
||||
continue
|
||||
decision = str(raw.get("decision") or "keep").strip().lower()
|
||||
verdict = {
|
||||
"id": cid,
|
||||
"decision": decision if decision in DECISIONS else "keep",
|
||||
"to_kind": str(raw.get("to_kind") or "").strip(),
|
||||
"reason": str(raw.get("reason") or "").strip()[:200],
|
||||
}
|
||||
if raw.get("confidence") is not None:
|
||||
verdict["confidence"] = clamp_confidence(raw.get("confidence"))
|
||||
out.append(verdict)
|
||||
return out
|
||||
|
||||
|
||||
def apply_verdicts(
|
||||
state,
|
||||
verdicts: list[dict],
|
||||
caps: dict[str, int],
|
||||
enabled: set[str],
|
||||
notice_paths: set[str] | None = None,
|
||||
) -> None:
|
||||
"""Rebuild the four buckets from the stage-2 verdicts, in place on ``state``.
|
||||
|
||||
A missing verdict means keep (fail-open), an unknown id is ignored, and a
|
||||
move to an unknown or disabled kind is ignored rather than fatal. Caps are
|
||||
re-applied afterwards in stage-1 order, so a move can never push a kind over
|
||||
its ceiling.
|
||||
|
||||
Two moves are refused rather than applied, because stage 1 owns the
|
||||
path/kind invariants and stage 2 must not break them:
|
||||
|
||||
* out of ``reminder`` for a candidate anchored on the external notice
|
||||
layer - only the reminder branch may cite ``resource/``, so demoting it
|
||||
would publish an external upload under a kind that may not quote one;
|
||||
* into ``reminder`` for a candidate with neither a due value nor a notice
|
||||
anchor - stage 2 cannot attach a date, so it could never satisfy
|
||||
``clean_reminder``'s contract for a dateless reminder.
|
||||
|
||||
Any other successful move out of ``reminder`` strips the reminder-only
|
||||
fields so no dangling registry link survives the reclassification.
|
||||
"""
|
||||
notices = {str(p) for p in (notice_paths or set())}
|
||||
buckets = kind_buckets(state)
|
||||
index: dict[str, tuple[str, dict]] = {}
|
||||
for kind, bucket in buckets.items():
|
||||
for item in bucket:
|
||||
index.setdefault(str(item.get("id") or ""), (kind, item))
|
||||
for verdict in verdicts:
|
||||
entry = index.get(verdict["id"])
|
||||
if entry is None:
|
||||
continue
|
||||
kind, item = entry
|
||||
if verdict["decision"] == "drop":
|
||||
buckets[kind] = [c for c in buckets[kind] if c is not item]
|
||||
state.refine_dropped.append(_audit(verdict, kind, item))
|
||||
elif verdict["decision"] == "move":
|
||||
target = verdict["to_kind"]
|
||||
if target not in TOPIC_KINDS or target not in enabled or target == kind:
|
||||
continue
|
||||
if kind == "reminder" and _notice_anchored(item, notices):
|
||||
continue
|
||||
if target == "reminder" and not _reminder_contract_met(item, notices):
|
||||
continue
|
||||
buckets[kind] = [c for c in buckets[kind] if c is not item]
|
||||
item["kind"] = target
|
||||
if kind == "reminder":
|
||||
for field in REMINDER_ONLY_FIELDS:
|
||||
item.pop(field, None)
|
||||
buckets[target].append(item)
|
||||
index[verdict["id"]] = (target, item)
|
||||
state.refine_moved.append({**_audit(verdict, kind, item), "to_kind": target})
|
||||
if "confidence" in verdict:
|
||||
item["confidence"] = verdict["confidence"]
|
||||
for kind, bucket in buckets.items():
|
||||
cap = int(caps.get(kind, 0))
|
||||
if cap and len(bucket) > cap:
|
||||
state.dropped_over_cap += len(bucket) - cap
|
||||
buckets[kind] = bucket[:cap]
|
||||
state.reminders = buckets["reminder"]
|
||||
state.follow_ups = buckets["follow_up"]
|
||||
state.new_ideas = buckets["new_idea"]
|
||||
state.extends = buckets["interest_extend"]
|
||||
|
||||
|
||||
def _audit(verdict: dict, kind: str, item: dict) -> dict:
|
||||
"""One audit record so a refine decision stays diagnosable in metadata."""
|
||||
return {
|
||||
"id": verdict["id"],
|
||||
"kind": kind,
|
||||
"title": str(item.get("title") or "")[:80],
|
||||
"decision": verdict["decision"],
|
||||
"reason": verdict["reason"],
|
||||
}
|
||||
1008
reme/steps/evolve/proactive/schedules.py
Normal file
1008
reme/steps/evolve/proactive/schedules.py
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -23,8 +23,18 @@ from .._evolve import passthrough_response
|
|||
from ....components import R
|
||||
from ....enumeration import ComponentEnum
|
||||
from ....schema import ProactiveState, ProactiveStateFile, ProactiveTopic
|
||||
from ....schema.proactive import clamp_confidence
|
||||
from ....schema.proactive import DEFAULT_KIND, clamp_confidence
|
||||
from ..dream.utils import daily_dir, previous_dates, today, workspace_dir
|
||||
from .schedules import (
|
||||
apply_schedule_updates,
|
||||
effective_due,
|
||||
refresh_reminder_dues,
|
||||
slim_schedule,
|
||||
sort_reminders,
|
||||
topic_from_candidate,
|
||||
trim_schedules,
|
||||
upsert_schedule_from_reminder,
|
||||
)
|
||||
from .utils import (
|
||||
current_now,
|
||||
dump_topic,
|
||||
|
|
@ -61,6 +71,10 @@ class ProactiveTopicsStep(BaseStep):
|
|||
dedup_lookback_days: int = 7,
|
||||
digest_compare_limit: int = 500,
|
||||
known_threshold_calibrated_for: str = "text-embedding-v4@1024",
|
||||
min_reminder_confidence: float = 0.3,
|
||||
schedule_retention_days: int = 180,
|
||||
max_schedules: int = 80,
|
||||
max_schedule_history: int = 8,
|
||||
skip_key: str = "proactive_skip",
|
||||
**kwargs,
|
||||
):
|
||||
|
|
@ -71,6 +85,10 @@ class ProactiveTopicsStep(BaseStep):
|
|||
self.max_topics = max(int(max_topics), 1)
|
||||
self.dedup_lookback_days = max(int(dedup_lookback_days), 0)
|
||||
self.digest_compare_limit = max(int(digest_compare_limit), 0)
|
||||
self.min_reminder_confidence = float(min_reminder_confidence)
|
||||
self.schedule_retention_days = max(int(schedule_retention_days), 1)
|
||||
self.max_schedules = max(int(max_schedules), 0)
|
||||
self.max_schedule_history = max(int(max_schedule_history), 0)
|
||||
self.skip_key = skip_key
|
||||
|
||||
# pylint: disable=too-many-statements
|
||||
|
|
@ -93,7 +111,9 @@ class ProactiveTopicsStep(BaseStep):
|
|||
daily = state.daily_dir or daily_dir(self)
|
||||
self.logger.info(
|
||||
f"[{self.name}] start date={day} follow_ups={len(state.follow_ups)} extends={len(state.extends)} "
|
||||
f"updates={len(state.updates)} carry_forward={state.carry_forward_count}",
|
||||
f"reminders={len(state.reminders)} new_ideas={len(state.new_ideas)} "
|
||||
f"updates={len(state.updates)} schedule_updates={len(state.schedule_updates)} "
|
||||
f"carry_forward={state.carry_forward_count}",
|
||||
)
|
||||
|
||||
state_file, _needs_bootstrap = load_state(ws, daily)
|
||||
|
|
@ -105,6 +125,13 @@ class ProactiveTopicsStep(BaseStep):
|
|||
# check, same shape as INV-8); otherwise it degrades to keep so stale
|
||||
# re-reads cannot rewrite evidence or refresh the freshness sort key.
|
||||
new_material = {norm_path(str(path).split("#", 1)[0]) for path in state.changed_paths}
|
||||
# The reminder branch is the only one allowed to cite the ``resource/``
|
||||
# layer and ``clean_schedule_update`` already accepts those paths, so
|
||||
# the registry anchor has to agree with it: without the notice paths a
|
||||
# dated external notice could never be registered through the explicit
|
||||
# channel and would only reach the registry via the reminder safety
|
||||
# net, losing its detail/end-time/recurrence and its own reason.
|
||||
schedule_material = new_material | {norm_path(str(path).split("#", 1)[0]) for path in state.resource_changed}
|
||||
for update in state.updates:
|
||||
topic = open_by_id.get(str(update.get("id") or ""))
|
||||
if topic is None:
|
||||
|
|
@ -138,8 +165,49 @@ class ProactiveTopicsStep(BaseStep):
|
|||
)
|
||||
state.updates_resolved += 1
|
||||
|
||||
# 1b) Schedule registry (branch C). Applied before the candidates so a
|
||||
# reminder always sees the registry it just helped maintain. The
|
||||
# registry has its own, much longer lifecycle than open_topics: a
|
||||
# schedule mentioned weeks ago must still fire on its due date.
|
||||
counts, touched_schedules = apply_schedule_updates(state_file, state.schedule_updates, day, schedule_material)
|
||||
state.schedules_added += counts["added"]
|
||||
state.schedules_updated += counts["updated"]
|
||||
state.schedules_closed += counts["closed"]
|
||||
# The clock scan read the registry before those updates landed, so a
|
||||
# seeded reminder may still advertise the previous due value; a reader
|
||||
# sorts and notifies by it, so re-anchor it on the registry.
|
||||
state.reminders_reanchored = refresh_reminder_dues(
|
||||
state.reminders,
|
||||
state_file,
|
||||
touched_schedules,
|
||||
current_now(self),
|
||||
)
|
||||
adopted_schedules = 0
|
||||
for candidate in state.reminders:
|
||||
outcome = upsert_schedule_from_reminder(state_file, candidate, day, touched_schedules)
|
||||
if outcome == "added":
|
||||
state.schedules_added += 1
|
||||
elif outcome == "updated":
|
||||
state.schedules_updated += 1
|
||||
elif outcome == "adopted":
|
||||
adopted_schedules += 1
|
||||
dropped_schedules = trim_schedules(
|
||||
state_file,
|
||||
day,
|
||||
self.schedule_retention_days,
|
||||
self.max_schedules,
|
||||
self.max_schedule_history,
|
||||
)
|
||||
state.schedules_out = [slim_schedule(s) for s in state_file.schedules if s.status == "open"]
|
||||
self.logger.info(
|
||||
f"[{self.name}] schedule registry added={state.schedules_added} updated={state.schedules_updated} "
|
||||
f"closed={state.schedules_closed} dropped={dropped_schedules} open={len(state.schedules_out)} "
|
||||
f"skipped_updates={counts['skipped']} reminders_reanchored={state.reminders_reanchored} "
|
||||
f"adopted={adopted_schedules}",
|
||||
)
|
||||
|
||||
# 2) Candidates: same-id merge, tombstone resurrect, dedup the rest.
|
||||
raw_candidates = list(state.follow_ups) + list(state.extends)
|
||||
raw_candidates = list(state.reminders) + list(state.follow_ups) + list(state.extends) + list(state.new_ideas)
|
||||
state.candidates_in = len(raw_candidates)
|
||||
merged: list[dict] = []
|
||||
resurrected: list[dict] = []
|
||||
|
|
@ -196,6 +264,13 @@ class ProactiveTopicsStep(BaseStep):
|
|||
if normalized in seen_normalized:
|
||||
dropped_duplicate += 1
|
||||
continue
|
||||
if str(candidate.get("kind") or "") == "reminder":
|
||||
# A due schedule must fire even when the same words were
|
||||
# published recently: the historical "known" gate filters
|
||||
# discovery noise, it is not a calendar.
|
||||
seen_normalized.add(normalized)
|
||||
survivors.append(candidate)
|
||||
continue
|
||||
if embedder is None:
|
||||
if normalized in comparison_norms:
|
||||
dropped_duplicate += 1
|
||||
|
|
@ -230,7 +305,7 @@ class ProactiveTopicsStep(BaseStep):
|
|||
|
||||
# 3) Truth source: add new topics, prune, single atomic write (F1.3).
|
||||
for candidate in survivors:
|
||||
state_file.open_topics.append(ProactiveTopic.model_validate(candidate))
|
||||
state_file.open_topics.append(topic_from_candidate(candidate))
|
||||
trim_state_file(state_file, day, max(state.carry_forward_days, 1))
|
||||
await save_state(ws, state_file, daily)
|
||||
|
||||
|
|
@ -239,9 +314,24 @@ class ProactiveTopicsStep(BaseStep):
|
|||
# rounds because such topics persist in the truth source once added.
|
||||
# The candidate list (sorted deterministically) feeds the plan/agenda
|
||||
# steps; ``push`` is exactly "at least one candidate".
|
||||
push_candidates = sort_topics(
|
||||
[t for t in state_file.open_topics if t.first_seen == day and t.confidence >= self.min_push_confidence],
|
||||
)
|
||||
# Reminders are the exception on both counts: one refreshed today pushes
|
||||
# regardless of ``first_seen`` (a due schedule recurs daily until it is
|
||||
# closed) and only has to clear the lower ``min_reminder_confidence``.
|
||||
reminders_today = [
|
||||
t
|
||||
for t in state_file.open_topics
|
||||
if t.kind == "reminder"
|
||||
and day in (t.first_seen, t.last_evidence_at)
|
||||
and t.confidence >= self.min_reminder_confidence
|
||||
]
|
||||
others_today = [
|
||||
t
|
||||
for t in state_file.open_topics
|
||||
if t.kind != "reminder" and t.first_seen == day and t.confidence >= self.min_push_confidence
|
||||
]
|
||||
push_candidates = sort_reminders([dump_topic(t) for t in reminders_today]) + [
|
||||
dump_topic(t) for t in sort_topics(others_today)
|
||||
]
|
||||
push = bool(push_candidates)
|
||||
if push:
|
||||
file_skip_reason = ""
|
||||
|
|
@ -286,6 +376,13 @@ class ProactiveTopicsStep(BaseStep):
|
|||
existing.confidence = clamp_confidence(candidate.get("confidence"))
|
||||
if candidate.get("paths"):
|
||||
existing.paths = list(candidate["paths"])
|
||||
if existing.kind == "reminder":
|
||||
# Keep the published due date and registry link current so a
|
||||
# recurring reminder shows its next occurrence, not the anchor.
|
||||
if due := effective_due(candidate):
|
||||
existing.due_at = due
|
||||
if candidate.get("schedule_id"):
|
||||
existing.schedule_id = str(candidate["schedule_id"])
|
||||
|
||||
@staticmethod
|
||||
def _resurrect(tombstone: dict, candidate: dict, day: str) -> ProactiveTopic:
|
||||
|
|
@ -294,16 +391,19 @@ class ProactiveTopicsStep(BaseStep):
|
|||
The over-age trim still applies to the original ``first_seen``, so a
|
||||
resurrection only extends a lifetime that has not fully elapsed.
|
||||
"""
|
||||
kind = str(candidate.get("kind") or DEFAULT_KIND)
|
||||
return ProactiveTopic(
|
||||
id=str(tombstone.get("id") or candidate.get("id") or ""),
|
||||
title=str(candidate.get("title") or tombstone.get("title") or ""),
|
||||
reason=str(candidate.get("reason") or ""),
|
||||
kind=str(candidate.get("kind") or "interest_extend"),
|
||||
kind=kind,
|
||||
confidence=clamp_confidence(candidate.get("confidence")),
|
||||
first_seen=str(tombstone.get("first_seen") or day),
|
||||
last_evidence_at=day,
|
||||
evidence=str(candidate.get("evidence") or "")[:120],
|
||||
paths=candidate.get("paths") or [],
|
||||
schedule_id=str(candidate.get("schedule_id") or ""),
|
||||
due_at=effective_due(candidate) if kind == "reminder" else "",
|
||||
)
|
||||
|
||||
async def _comparison_texts(
|
||||
|
|
|
|||
|
|
@ -17,8 +17,16 @@ from pathlib import Path
|
|||
import yaml
|
||||
|
||||
from ....enumeration import ComponentEnum
|
||||
from ....schema import ProactiveStateFile, ProactiveTopic
|
||||
from ....schema.proactive import clamp_confidence
|
||||
from ....schema import ProactiveDelivery, ProactiveSchedule, ProactiveStateFile, ProactiveTopic
|
||||
from ....schema.proactive import (
|
||||
DEFAULT_DELIVERY_CHANNEL,
|
||||
DEFAULT_DELIVERY_STATUS,
|
||||
DEFAULT_KIND,
|
||||
TERMINAL_DELIVERY_STATUSES,
|
||||
TOPIC_KINDS,
|
||||
clamp_confidence,
|
||||
kind_priority,
|
||||
)
|
||||
from ....utils import get_logger
|
||||
from ...file_io._file_io import get_path_lock
|
||||
from .._evolve import now
|
||||
|
|
@ -26,9 +34,17 @@ from ..dream.utils import clean_paths, recent_dates, scan_day_files
|
|||
|
||||
logger = get_logger(log_to_file=False)
|
||||
|
||||
|
||||
PROACTIVE_STATE_NAME = "_proactive.yaml"
|
||||
|
||||
|
||||
INTERESTS_NAME = "interests.yaml"
|
||||
EXTRACT_SECTIONS = ("follow_ups", "extends", "updates")
|
||||
|
||||
|
||||
EXTRACT_SECTIONS = ("follow_ups", "extends", "reminders", "new_ideas", "updates", "schedule_updates")
|
||||
|
||||
# Registry triggers that deserve a reminder; ``later``/``unscheduled``/``closed``
|
||||
# stay in the prompt view but never become candidates.
|
||||
|
||||
|
||||
def load_yaml_topics(path: Path, *, strict: bool = False) -> list[dict]:
|
||||
|
|
@ -101,8 +117,6 @@ def _clean_legacy_topic(raw: dict) -> dict:
|
|||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Frozen identity contract (A7 / INV-4)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def normalize_topic(title: str) -> str:
|
||||
|
|
@ -121,8 +135,6 @@ def topic_id(title: str) -> str:
|
|||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Paths and material set M (F2.0)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def state_file_path(ws: Path, daily: str = "daily") -> Path:
|
||||
|
|
@ -158,8 +170,75 @@ def scan_material_daily(ws: Path, day: str, daily: str, scan_days: int) -> list[
|
|||
return sorted(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Truth-source state file daily/_proactive.yaml (F1.3)
|
||||
# External source records the reminder branch may read directly. Binary sources
|
||||
# are deliberately absent: ``auto_resource`` interprets them into daily cards,
|
||||
# which already reach extraction through ``scan_material_daily``.
|
||||
RESOURCE_NOTICE_SUFFIXES = (
|
||||
".md",
|
||||
".markdown",
|
||||
".txt",
|
||||
".json",
|
||||
".yaml",
|
||||
".yml",
|
||||
".csv",
|
||||
".tsv",
|
||||
".html",
|
||||
".eml",
|
||||
".ics",
|
||||
)
|
||||
|
||||
|
||||
def _mtime_day(path: Path) -> str:
|
||||
"""Local date a file last changed, or ``""`` when it cannot be stat'ed."""
|
||||
try:
|
||||
return dt.date.fromtimestamp(path.stat().st_mtime).isoformat()
|
||||
except OSError:
|
||||
return ""
|
||||
|
||||
|
||||
def scan_material_resource(
|
||||
ws: Path,
|
||||
day: str,
|
||||
scan_days: int,
|
||||
resource: str = "resource",
|
||||
) -> tuple[list[str], int]:
|
||||
"""M_resource: external source records that may carry notices for the user.
|
||||
|
||||
Mirrors the ``resource/`` entry-point semantics of Auto Resource: a dated
|
||||
directory ``resource/<YYYY-MM-DD>/...`` belongs to that date, while a file
|
||||
directly under ``resource/`` (or under a non-dated folder) belongs to the
|
||||
day it landed, i.e. its mtime. Returns ``(paths, skipped_binary)``; only
|
||||
text-readable suffixes are returned, because a binary source has to be
|
||||
interpreted by ``auto_resource`` before anyone can quote it.
|
||||
|
||||
This is the reserved hook for future external source interfaces: whatever
|
||||
writes into ``resource/`` becomes eligible for the reminder review without
|
||||
any further change to the proactive chain.
|
||||
"""
|
||||
window = set(recent_dates(day, scan_days))
|
||||
root = ws / resource
|
||||
if not window or not root.is_dir():
|
||||
return [], 0
|
||||
out: list[str] = []
|
||||
skipped = 0
|
||||
for path in sorted(root.rglob("*")):
|
||||
if not path.is_file():
|
||||
continue
|
||||
rel = norm_path(path.relative_to(ws).as_posix())
|
||||
parts = rel.split("/")
|
||||
if any(part.startswith("_") or part.startswith(".") for part in parts[1:]):
|
||||
continue
|
||||
owner = parts[1] if len(parts) > 2 and safe_date(parts[1]) else _mtime_day(path)
|
||||
if owner not in window:
|
||||
continue
|
||||
if path.suffix.lower() not in RESOURCE_NOTICE_SUFFIXES:
|
||||
skipped += 1
|
||||
continue
|
||||
if rel not in out:
|
||||
out.append(rel)
|
||||
return sorted(out), skipped
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
|
|
@ -187,11 +266,56 @@ def load_state(ws: Path, daily: str = "daily") -> tuple[ProactiveStateFile, bool
|
|||
try:
|
||||
state = ProactiveStateFile.model_validate(data)
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning(f"proactive state file invalid, rebuilding empty: {path} ({e})")
|
||||
return ProactiveStateFile(), False
|
||||
# Salvage entry by entry instead of discarding the file: the schedule
|
||||
# registry is designed to outlive topics by months, so one malformed
|
||||
# hand-edited entry must not silently delete every long-lived reminder.
|
||||
# Still no bootstrap - the file exists, so this is not a first run.
|
||||
state, dropped = _salvage_state(data)
|
||||
logger.warning(
|
||||
f"proactive state file invalid, salvaged entry by entry: {path} "
|
||||
f"(dropped {dropped} unreadable entr(ies); kept open_topics={len(state.open_topics)} "
|
||||
f"schedules={len(state.schedules)} deliveries={len(state.deliveries)}; {e})",
|
||||
)
|
||||
return state, False
|
||||
return state, needs_bootstrap
|
||||
|
||||
|
||||
# Truth-source sections whose entries are validated one by one when the whole
|
||||
# file fails to parse. ``resolved`` is a plain dict list, so it only needs the
|
||||
# non-mapping items filtered out.
|
||||
_STATE_SECTIONS = (("open_topics", ProactiveTopic), ("schedules", ProactiveSchedule), ("deliveries", ProactiveDelivery))
|
||||
|
||||
|
||||
def _salvage_state(data: dict) -> tuple[ProactiveStateFile, int]:
|
||||
"""Rebuild a partly-unreadable truth source, dropping only the bad entries.
|
||||
|
||||
Returns the salvaged model and the number of entries that had to be
|
||||
dropped. Never raises: a section that is not even a list is treated as
|
||||
absent, exactly like a file written before that section existed.
|
||||
"""
|
||||
sections: dict[str, list] = {}
|
||||
dropped = 0
|
||||
for key, model in _STATE_SECTIONS:
|
||||
raw = data.get(key)
|
||||
raw = raw if isinstance(raw, list) else []
|
||||
kept = []
|
||||
for item in raw:
|
||||
try:
|
||||
kept.append(model.model_validate(item))
|
||||
except Exception: # noqa: BLE001
|
||||
dropped += 1
|
||||
sections[key] = kept
|
||||
raw_resolved = data.get("resolved")
|
||||
resolved = [item for item in (raw_resolved if isinstance(raw_resolved, list) else []) if isinstance(item, dict)]
|
||||
version = data.get("version")
|
||||
state = ProactiveStateFile(
|
||||
version=version if isinstance(version, int) else 1,
|
||||
resolved=resolved,
|
||||
**sections,
|
||||
)
|
||||
return state, dropped
|
||||
|
||||
|
||||
async def save_state(ws: Path, state_file: ProactiveStateFile, daily: str = "daily") -> None:
|
||||
"""Atomically persist the truth source (path lock + tmp file + os.replace)."""
|
||||
path = state_file_path(ws, daily)
|
||||
|
|
@ -210,7 +334,8 @@ async def save_state(ws: Path, state_file: ProactiveStateFile, daily: str = "dai
|
|||
raise
|
||||
|
||||
|
||||
def _safe_date(text: str) -> dt.date | None:
|
||||
def safe_date(text: str) -> dt.date | None:
|
||||
"""Parse an ISO ``YYYY-MM-DD`` string, returning None when it is not one."""
|
||||
try:
|
||||
return dt.date.fromisoformat(str(text or "").strip())
|
||||
except ValueError:
|
||||
|
|
@ -237,13 +362,13 @@ async def load_carry_forward(
|
|||
state_file.open_topics = _bootstrap_from_history(ws, day, days, daily)
|
||||
await save_state(ws, state_file, daily)
|
||||
resolved_ids = {str(r.get("id") or "") for r in state_file.resolved if isinstance(r, dict)}
|
||||
base = _safe_date(day)
|
||||
base = safe_date(day)
|
||||
open_topics: list[ProactiveTopic] = []
|
||||
expired = 0
|
||||
for topic in state_file.open_topics:
|
||||
if topic.id and topic.id in resolved_ids:
|
||||
continue
|
||||
first_seen = _safe_date(topic.first_seen)
|
||||
first_seen = safe_date(topic.first_seen)
|
||||
# Boundary aligned with trim_state_file/_expiry_cutoff: age >= days is
|
||||
# over-age everywhere, so a topic never enters the prompt in the same
|
||||
# round it gets pruned from the truth source (audit item 9).
|
||||
|
|
@ -259,7 +384,7 @@ async def load_carry_forward(
|
|||
|
||||
def _bootstrap_from_history(ws: Path, day: str, days: int, daily: str) -> list[ProactiveTopic]:
|
||||
"""One-time bootstrap: newest record per id wins, first_seen takes the min (F1.4)."""
|
||||
if _safe_date(day) is None:
|
||||
if safe_date(day) is None:
|
||||
return []
|
||||
records: dict[str, ProactiveTopic] = {}
|
||||
first_seen: dict[str, str] = {}
|
||||
|
|
@ -283,7 +408,7 @@ def _bootstrap_from_history(ws: Path, day: str, days: int, daily: str) -> list[P
|
|||
|
||||
def trim_state_file(state_file: ProactiveStateFile, day: str, days: int) -> None:
|
||||
"""Prune budget/exposure/resolved windows and over-age/resolved open topics."""
|
||||
base = _safe_date(day)
|
||||
base = safe_date(day)
|
||||
if base is None:
|
||||
return
|
||||
cutoff = (base - dt.timedelta(days=max(int(days), 0))).isoformat()
|
||||
|
|
@ -295,7 +420,7 @@ def trim_state_file(state_file: ProactiveStateFile, day: str, days: int) -> None
|
|||
for topic in state_file.open_topics:
|
||||
if topic.id and topic.id in resolved_ids:
|
||||
continue
|
||||
first_seen = _safe_date(topic.first_seen)
|
||||
first_seen = safe_date(topic.first_seen)
|
||||
if first_seen is not None and first_seen.isoformat() <= cutoff:
|
||||
continue
|
||||
kept.append(topic)
|
||||
|
|
@ -303,8 +428,157 @@ def trim_state_file(state_file: ProactiveStateFile, day: str, days: int) -> None
|
|||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# interests.yaml read/render (F1.2 / A2 / A4)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Delivery feedback (host -> ReMe). Optional by design: a framework that never
|
||||
# reports leaves ``deliveries`` empty and every helper below degrades to a
|
||||
# no-op, so the chain behaves exactly as it did before feedback existed.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _delivery_date(text: str) -> str:
|
||||
"""``YYYY-MM-DD`` prefix of an ISO timestamp; '' when unparseable."""
|
||||
raw = str(text or "").strip()
|
||||
parsed = safe_date(raw[:10])
|
||||
return parsed.isoformat() if parsed is not None else ""
|
||||
|
||||
|
||||
def normalize_delivery(raw, day: str, now_dt: dt.datetime) -> ProactiveDelivery | None:
|
||||
"""Coerce one host-reported item into a ``ProactiveDelivery``.
|
||||
|
||||
``topic_id`` is the only mandatory field; a record without one can never
|
||||
match an agenda item, so it is dropped instead of being stored. Missing
|
||||
``agenda_date`` defaults to the round's day and missing ``delivered_at``
|
||||
to the business clock, which keeps a minimal host report
|
||||
(``{"topic_id": ..., "status": ...}``) sufficient.
|
||||
"""
|
||||
if not isinstance(raw, dict):
|
||||
return None
|
||||
topic = str(raw.get("topic_id") or "").strip()
|
||||
if not topic:
|
||||
return None
|
||||
stamp = str(raw.get("delivered_at") or "").strip()
|
||||
if not _delivery_date(stamp):
|
||||
stamp = now_dt.isoformat(timespec="seconds")
|
||||
return ProactiveDelivery(
|
||||
topic_id=topic,
|
||||
agenda_date=str(raw.get("agenda_date") or day or "")[:10],
|
||||
delivered_at=stamp,
|
||||
channel=str(raw.get("channel") or DEFAULT_DELIVERY_CHANNEL),
|
||||
consumer=str(raw.get("consumer") or ""),
|
||||
status=str(raw.get("status") or DEFAULT_DELIVERY_STATUS),
|
||||
outcome=str(raw.get("outcome") or ""),
|
||||
external_ref=str(raw.get("external_ref") or ""),
|
||||
)
|
||||
|
||||
|
||||
def delivery_index(state_file: ProactiveStateFile) -> dict[str, list[ProactiveDelivery]]:
|
||||
"""Group delivery records by topic_id, newest first.
|
||||
|
||||
The list is kept (not collapsed to one record) because a topic can be
|
||||
surfaced through several channels; suppression only needs the most recent
|
||||
relevant record, which is the head after sorting.
|
||||
"""
|
||||
grouped: dict[str, list[ProactiveDelivery]] = {}
|
||||
for record in state_file.deliveries:
|
||||
if not record.topic_id:
|
||||
continue
|
||||
grouped.setdefault(record.topic_id, []).append(record)
|
||||
for records in grouped.values():
|
||||
records.sort(key=lambda r: (str(r.delivered_at or ""), str(r.external_ref or "")), reverse=True)
|
||||
return grouped
|
||||
|
||||
|
||||
def is_delivery_suppressed(record: ProactiveDelivery | None, day: str, cooldown_days: int) -> bool:
|
||||
"""Whether a delivery record still blocks re-proposing its topic.
|
||||
|
||||
``failed`` never blocks: a run that did not complete should be retryable
|
||||
on the next round. Every other status blocks while it is younger than
|
||||
``cooldown_days``, so a surfaced-but-ignored item can come back later
|
||||
while an executed or dismissed one does not repeat tomorrow.
|
||||
"""
|
||||
if record is None:
|
||||
return False
|
||||
if str(record.status or "") == "failed":
|
||||
return False
|
||||
base = safe_date(day)
|
||||
delivered = safe_date(_delivery_date(record.delivered_at))
|
||||
if base is None or delivered is None:
|
||||
return False
|
||||
window = max(int(cooldown_days), 0)
|
||||
return (base - delivered).days <= window
|
||||
|
||||
|
||||
def suppressed_delivery_ids(
|
||||
state_file: ProactiveStateFile,
|
||||
day: str,
|
||||
cooldown_days: int,
|
||||
) -> dict[str, ProactiveDelivery]:
|
||||
"""topic_id -> blocking record, for the reader and the agenda step."""
|
||||
out: dict[str, ProactiveDelivery] = {}
|
||||
for item_id, records in delivery_index(state_file).items():
|
||||
for record in records:
|
||||
if is_delivery_suppressed(record, day, cooldown_days):
|
||||
out[item_id] = record
|
||||
break
|
||||
return out
|
||||
|
||||
|
||||
def upsert_deliveries(
|
||||
state_file: ProactiveStateFile,
|
||||
items: list,
|
||||
day: str,
|
||||
now_dt: dt.datetime,
|
||||
) -> dict[str, int]:
|
||||
"""Merge host-reported deliveries into the truth source (idempotent).
|
||||
|
||||
Identity is ``ProactiveDelivery.identity()``: an explicit ``external_ref``
|
||||
wins, otherwise ``(topic_id, agenda_date, channel)``. Status never moves
|
||||
backwards out of a terminal value, so a late ``shown`` report cannot undo
|
||||
an ``executed`` one.
|
||||
"""
|
||||
counts = {"added": 0, "updated": 0, "dropped": 0}
|
||||
by_identity = {record.identity(): record for record in state_file.deliveries}
|
||||
for raw in items:
|
||||
record = normalize_delivery(raw, day, now_dt)
|
||||
if record is None:
|
||||
counts["dropped"] += 1
|
||||
continue
|
||||
key = record.identity()
|
||||
existing = by_identity.get(key)
|
||||
if existing is None:
|
||||
state_file.deliveries.append(record)
|
||||
by_identity[key] = record
|
||||
counts["added"] += 1
|
||||
continue
|
||||
if existing.status in TERMINAL_DELIVERY_STATUSES and record.status not in TERMINAL_DELIVERY_STATUSES:
|
||||
logger.info(f"delivery for {existing.topic_id} kept terminal status {existing.status!r}")
|
||||
counts["dropped"] += 1
|
||||
continue
|
||||
existing.agenda_date = record.agenda_date or existing.agenda_date
|
||||
existing.delivered_at = record.delivered_at
|
||||
existing.channel = record.channel
|
||||
existing.consumer = record.consumer or existing.consumer
|
||||
existing.status = record.status
|
||||
if record.outcome:
|
||||
existing.outcome = record.outcome
|
||||
counts["updated"] += 1
|
||||
return counts
|
||||
|
||||
|
||||
def trim_deliveries(state_file: ProactiveStateFile, day: str, retention_days: int, max_deliveries: int) -> int:
|
||||
"""Prune the delivery log by age then by size; returns the dropped count."""
|
||||
base = safe_date(day)
|
||||
before = len(state_file.deliveries)
|
||||
if base is not None:
|
||||
cutoff = (base - dt.timedelta(days=max(int(retention_days), 0))).isoformat()
|
||||
state_file.deliveries = [r for r in state_file.deliveries if _delivery_date(r.delivered_at) >= cutoff]
|
||||
cap = max(int(max_deliveries), 0)
|
||||
if len(state_file.deliveries) > cap:
|
||||
state_file.deliveries.sort(key=lambda r: str(r.delivered_at or ""), reverse=True)
|
||||
del state_file.deliveries[cap:]
|
||||
return before - len(state_file.deliveries)
|
||||
|
||||
|
||||
def quarantine_interests(path: Path, error: Exception) -> None:
|
||||
|
|
@ -363,23 +637,28 @@ def parse_interests_topics(data: dict, file_date: str) -> tuple[list[ProactiveTo
|
|||
id=str(raw.get("id") or "").strip() or topic_id(title),
|
||||
title=title,
|
||||
reason=reason,
|
||||
kind=raw.get("kind", "interest_extend"),
|
||||
kind=raw.get("kind", DEFAULT_KIND),
|
||||
confidence=raw.get("confidence", 0.5),
|
||||
first_seen=str(raw.get("first_seen") or "").strip() or file_date,
|
||||
last_evidence_at=str(raw.get("last_evidence_at") or "").strip() or file_date,
|
||||
evidence=str(raw.get("evidence") or "").strip()[:120],
|
||||
paths=raw.get("paths") or [],
|
||||
schedule_id=str(raw.get("schedule_id") or "").strip(),
|
||||
due_at=str(raw.get("due_at") or "").strip(),
|
||||
),
|
||||
)
|
||||
return topics, is_v1, push
|
||||
|
||||
|
||||
def sort_topics(topics: list) -> list:
|
||||
"""Order: last_evidence_at desc -> follow_up first -> confidence desc -> id asc.
|
||||
"""Order: last_evidence_at desc -> kind priority -> confidence desc -> id asc.
|
||||
|
||||
Freshness is the primary key (v5 aging fix): stale topics sink below newly
|
||||
evidenced ones of any kind, so long-lived follow_ups cannot permanently
|
||||
crowd out new discoveries.
|
||||
Kind priority is ``KIND_PRIORITY`` (reminder > follow_up > new_idea >
|
||||
interest_extend), so a reminder always outranks discovery kinds inside the
|
||||
same freshness band and survives the ``max_topics`` render cap.
|
||||
"""
|
||||
|
||||
def get(topic, key):
|
||||
|
|
@ -387,7 +666,7 @@ def sort_topics(topics: list) -> list:
|
|||
|
||||
out = sorted(topics, key=lambda t: str(get(t, "id") or ""))
|
||||
out.sort(key=lambda t: clamp_confidence(get(t, "confidence")), reverse=True)
|
||||
out.sort(key=lambda t: 0 if get(t, "kind") == "follow_up" else 1)
|
||||
out.sort(key=lambda t: kind_priority(get(t, "kind")))
|
||||
out.sort(key=lambda t: str(get(t, "last_evidence_at") or ""), reverse=True)
|
||||
return out
|
||||
|
||||
|
|
@ -395,17 +674,24 @@ def sort_topics(topics: list) -> list:
|
|||
def dump_topic(topic) -> dict:
|
||||
"""Render one topic as an A2-ordered dict for interests.yaml v2."""
|
||||
get = (lambda k: getattr(topic, k)) if not isinstance(topic, dict) else topic.get
|
||||
return {
|
||||
dumped = {
|
||||
"id": str(get("id") or ""),
|
||||
"title": str(get("title") or ""),
|
||||
"reason": str(get("reason") or ""),
|
||||
"kind": str(get("kind") or "interest_extend"),
|
||||
"kind": str(get("kind") or DEFAULT_KIND),
|
||||
"confidence": clamp_confidence(get("confidence")),
|
||||
"first_seen": str(get("first_seen") or ""),
|
||||
"last_evidence_at": str(get("last_evidence_at") or ""),
|
||||
"evidence": str(get("evidence") or "")[:120],
|
||||
"paths": [str(p) for p in (get("paths") or [])],
|
||||
}
|
||||
# Reminder links are emitted only when present so the v2 topic shape of
|
||||
# every other kind stays byte-identical.
|
||||
if schedule_id := str(get("schedule_id") or ""):
|
||||
dumped["schedule_id"] = schedule_id
|
||||
if due_at := str(get("due_at") or ""):
|
||||
dumped["due_at"] = due_at
|
||||
return dumped
|
||||
|
||||
|
||||
def render_interests(
|
||||
|
|
@ -474,8 +760,6 @@ def write_interests_if_changed(ws: Path, path: Path, rendered: dict) -> bool: #
|
|||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Candidate cleaning (extract side, A3)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def clean_candidate(raw, allowed_paths: set[str], kind: str, day: str) -> dict:
|
||||
|
|
@ -491,7 +775,7 @@ def clean_candidate(raw, allowed_paths: set[str], kind: str, day: str) -> dict:
|
|||
"id": topic_id(title),
|
||||
"title": title,
|
||||
"reason": reason,
|
||||
"kind": kind,
|
||||
"kind": kind if kind in TOPIC_KINDS else DEFAULT_KIND,
|
||||
"confidence": clamp_confidence(raw.get("confidence")),
|
||||
"first_seen": day,
|
||||
"last_evidence_at": day,
|
||||
|
|
@ -631,6 +915,57 @@ def load_personal_profile_block(
|
|||
return "(no user profile)"
|
||||
|
||||
|
||||
CAPABILITY_SECTIONS = (
|
||||
("procedure", "procedure (reusable experience, skills, tool/API know-how the agent already has)"),
|
||||
("wiki", "wiki (existing knowledge nodes of the personal knowledge base)"),
|
||||
)
|
||||
|
||||
|
||||
def load_capability_block(ws: Path, digest_dir: str, max_chars: int) -> str:
|
||||
"""Inventory of what the agent can already do, used by the ``new_ideas`` branch.
|
||||
|
||||
Reads only rebuildable digest nodes: ``<digest_dir>/procedure/*.md``
|
||||
(accumulated experience, skills, tool/API know-how) and
|
||||
``<digest_dir>/wiki/*.md`` (knowledge nodes an interest can extend).
|
||||
Each node contributes its frontmatter ``description`` so proposals stay
|
||||
grounded in resources that actually exist instead of invented capabilities.
|
||||
"""
|
||||
max_chars = int(max_chars)
|
||||
if max_chars <= 0:
|
||||
return "(no capability inventory)"
|
||||
per_section = max(max_chars // max(len(CAPABILITY_SECTIONS), 1), 200)
|
||||
sections: list[str] = []
|
||||
for subdirectory, heading in CAPABILITY_SECTIONS:
|
||||
lines = _digest_inventory_lines(ws / digest_dir / subdirectory, per_section)
|
||||
if lines:
|
||||
sections.append(f"### {heading}\n" + "\n".join(lines))
|
||||
if not sections:
|
||||
return "(no capability inventory)"
|
||||
return "\n\n".join(sections)[:max_chars]
|
||||
|
||||
|
||||
def _digest_inventory_lines(directory: Path, budget: int) -> list[str]:
|
||||
"""``- <title>: <description>`` lines for one digest subdirectory, budget-bounded."""
|
||||
if not directory.is_dir():
|
||||
return []
|
||||
lines: list[str] = []
|
||||
used = 0
|
||||
for path in sorted(p for p in directory.glob("*.md") if p.is_file()):
|
||||
try:
|
||||
raw = path.read_text(encoding="utf-8")
|
||||
except OSError:
|
||||
continue
|
||||
meta, _body = strip_frontmatter(raw)
|
||||
title = str(meta.get("name") or path.stem)
|
||||
description = " ".join(str(meta.get("description") or "").split())
|
||||
line = f"- {title}" + (f": {description[:160]}" if description else "")
|
||||
if used + len(line) > budget:
|
||||
break
|
||||
lines.append(line)
|
||||
used += len(line) + 1
|
||||
return lines
|
||||
|
||||
|
||||
def parse_fenced_yaml(text: str) -> dict:
|
||||
"""Parse the first fenced YAML/JSON block (or the bare text) into a dict.
|
||||
|
||||
|
|
|
|||
714
tests/unit/test_proactive_feedback.py
Normal file
714
tests/unit/test_proactive_feedback.py
Normal file
|
|
@ -0,0 +1,714 @@
|
|||
"""Unit tests for proactive delivery feedback (host -> ReMe reporting).
|
||||
|
||||
Covers the ``deliveries`` truth-source section, the ``proactive_feedback``
|
||||
step, the reader-side withholding in ``proactive_step`` and the write-side
|
||||
suppression in ``proactive_agenda_step``. Feedback is optional by contract, so
|
||||
every test that asserts new behaviour is paired with one asserting that an
|
||||
empty ``deliveries`` section changes nothing.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import datetime as dt
|
||||
from unittest.mock import patch
|
||||
|
||||
import yaml
|
||||
|
||||
from reme.components.application_context import ApplicationContext
|
||||
from reme.components.runtime_context import RuntimeContext
|
||||
from reme.schema import ProactiveDelivery, ProactiveStateFile
|
||||
from reme.schema.proactive import MAX_DELIVERY_OUTCOME_CHARS
|
||||
from reme.steps.evolve.proactive.feedback import ProactiveFeedbackStep
|
||||
from reme.steps.evolve.proactive.proactive import ProactiveStep
|
||||
from reme.steps.evolve.proactive.utils import (
|
||||
is_delivery_suppressed,
|
||||
load_state,
|
||||
suppressed_delivery_ids,
|
||||
topic_id,
|
||||
trim_deliveries,
|
||||
upsert_deliveries,
|
||||
)
|
||||
|
||||
from .test_proactive_refresh import (
|
||||
DAY,
|
||||
EXTRACT_CLOCK,
|
||||
_FileStore,
|
||||
_interests,
|
||||
_reply,
|
||||
_reminder,
|
||||
_run_full_chain,
|
||||
_state_topic,
|
||||
_topic,
|
||||
_touch,
|
||||
_write_state,
|
||||
)
|
||||
|
||||
NOW = dt.datetime(2026, 8, 13, 18, 0, tzinfo=dt.timezone(dt.timedelta(hours=8)))
|
||||
|
||||
|
||||
def _write_state_with_deliveries(ws, deliveries, open_topics=None):
|
||||
"""Seed the truth source with open topics plus a delivery log."""
|
||||
data = {
|
||||
"version": 1,
|
||||
"open_topics": open_topics or [],
|
||||
"resolved": [],
|
||||
"schedules": [],
|
||||
"deliveries": deliveries,
|
||||
}
|
||||
_touch(ws / "daily" / "_proactive.yaml", yaml.safe_dump(data, allow_unicode=True, sort_keys=False))
|
||||
|
||||
|
||||
def _write_interests_with_agenda(ws, topics, agenda, day=DAY, push=True):
|
||||
"""Write a v2 exposure file carrying an agenda, as the agenda step would."""
|
||||
payload = {
|
||||
"version": 2,
|
||||
"date": day,
|
||||
"generated_at": f"{day}T18:00:00+08:00",
|
||||
"push": push,
|
||||
"topics": topics,
|
||||
"agenda": agenda,
|
||||
"suppressed": [],
|
||||
}
|
||||
_touch(_interests(ws, day), yaml.safe_dump(payload, allow_unicode=True, sort_keys=False))
|
||||
|
||||
|
||||
def _agenda_item(topic_id_value, title, kind="follow_up", **extra):
|
||||
item = {
|
||||
"topic_id": topic_id_value,
|
||||
"title": title,
|
||||
"kind": kind,
|
||||
"scenario_type": "resume_task",
|
||||
"opener": f"对了，{title}还差一步。",
|
||||
"next_action": "跑一下下一步",
|
||||
"preconditions": [],
|
||||
"delivery": "in_conversation",
|
||||
"linked_memory": [],
|
||||
"order_reason": "seeded",
|
||||
}
|
||||
item.update(extra)
|
||||
return item
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schema and pure helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_delivery_model_fallbacks():
|
||||
"""Illegal status/channel fall back; outcome is capped; identity is stable."""
|
||||
record = ProactiveDelivery(topic_id="t", status="BOGUS", channel="Inbox", outcome="x" * 900, external_ref="e1")
|
||||
assert record.status == "shown"
|
||||
assert record.channel == "inbox"
|
||||
assert len(record.outcome) == MAX_DELIVERY_OUTCOME_CHARS
|
||||
assert record.identity() == ("t", "ref:e1")
|
||||
|
||||
bare = ProactiveDelivery(topic_id="t", agenda_date=DAY, channel="notification")
|
||||
assert bare.identity() == ("t", f"day:{DAY}|notification")
|
||||
assert ProactiveDelivery(topic_id="t", channel="weird").channel == "other"
|
||||
|
||||
|
||||
def test_upsert_deliveries_is_idempotent():
|
||||
"""Re-reporting the same delivery updates in place instead of appending."""
|
||||
state_file = ProactiveStateFile()
|
||||
items = [
|
||||
{"topic_id": "a", "status": "shown", "channel": "inbox", "external_ref": "ev1"},
|
||||
{"topic_id": "b", "status": "executed", "channel": "in_conversation", "outcome": "done"},
|
||||
{"topic_id": "", "status": "shown"},
|
||||
"garbage",
|
||||
]
|
||||
assert upsert_deliveries(state_file, items, DAY, NOW) == {"added": 2, "updated": 0, "dropped": 2}
|
||||
assert upsert_deliveries(state_file, items, DAY, NOW) == {"added": 0, "updated": 2, "dropped": 2}
|
||||
assert len(state_file.deliveries) == 2
|
||||
|
||||
|
||||
def test_upsert_deliveries_never_downgrades_terminal_status():
|
||||
"""A late ``shown`` report cannot undo an ``executed`` one."""
|
||||
state_file = ProactiveStateFile()
|
||||
upsert_deliveries(state_file, [{"topic_id": "a", "status": "executed", "channel": "inbox"}], DAY, NOW)
|
||||
counts = upsert_deliveries(state_file, [{"topic_id": "a", "status": "shown", "channel": "inbox"}], DAY, NOW)
|
||||
assert counts == {"added": 0, "updated": 0, "dropped": 1}
|
||||
assert state_file.deliveries[0].status == "executed"
|
||||
|
||||
|
||||
def test_upsert_deliveries_defaults_missing_clock_fields():
|
||||
"""A minimal report still gets an agenda_date and a delivered_at."""
|
||||
state_file = ProactiveStateFile()
|
||||
upsert_deliveries(state_file, [{"topic_id": "a"}], DAY, NOW)
|
||||
record = state_file.deliveries[0]
|
||||
assert record.agenda_date == DAY
|
||||
assert record.delivered_at.startswith(DAY)
|
||||
assert record.status == "shown" and record.channel == "other"
|
||||
|
||||
|
||||
def test_is_delivery_suppressed_window():
|
||||
"""``failed`` never blocks; other statuses block inside the cooldown only."""
|
||||
|
||||
def record(status, delivered_at):
|
||||
return ProactiveDelivery(topic_id="a", status=status, delivered_at=delivered_at)
|
||||
|
||||
assert is_delivery_suppressed(None, DAY, 3) is False
|
||||
assert is_delivery_suppressed(record("failed", f"{DAY}T18:00:00+08:00"), DAY, 3) is False
|
||||
assert is_delivery_suppressed(record("shown", f"{DAY}T18:00:00+08:00"), DAY, 3) is True
|
||||
assert is_delivery_suppressed(record("executed", "2026-08-10T18:00:00+08:00"), DAY, 3) is True
|
||||
assert is_delivery_suppressed(record("shown", "2026-08-09T18:00:00+08:00"), DAY, 3) is False
|
||||
assert is_delivery_suppressed(record("shown", "2026-08-09T18:00:00+08:00"), DAY, 0) is False
|
||||
assert is_delivery_suppressed(record("shown", "not-a-date"), DAY, 3) is False
|
||||
|
||||
|
||||
def test_trim_deliveries_by_age_then_size():
|
||||
"""Old records go first, then the newest ``max_deliveries`` survive."""
|
||||
state_file = ProactiveStateFile()
|
||||
for index, day in enumerate(["2026-01-01", "2026-08-11", "2026-08-12", "2026-08-13"]):
|
||||
state_file.deliveries.append(
|
||||
ProactiveDelivery(topic_id=f"t{index}", delivered_at=f"{day}T18:00:00+08:00", external_ref=f"e{index}"),
|
||||
)
|
||||
assert trim_deliveries(state_file, DAY, 180, 2) == 2
|
||||
assert [r.topic_id for r in state_file.deliveries] == ["t3", "t2"]
|
||||
# retention 1 day keeps 08-12 and 08-13; retention 0 keeps only today.
|
||||
assert trim_deliveries(state_file, DAY, 1, 10) == 0
|
||||
assert trim_deliveries(state_file, DAY, 0, 10) == 1
|
||||
assert [r.topic_id for r in state_file.deliveries] == ["t3"]
|
||||
|
||||
|
||||
def test_suppressed_delivery_ids_prefers_the_blocking_record():
|
||||
"""The newest blocking record wins; a failed run does not mask it."""
|
||||
state_file = ProactiveStateFile()
|
||||
upsert_deliveries(
|
||||
state_file,
|
||||
[
|
||||
{"topic_id": "a", "status": "failed", "channel": "inbox", "external_ref": "e1"},
|
||||
{"topic_id": "a", "status": "shown", "channel": "notification", "external_ref": "e2"},
|
||||
{"topic_id": "b", "status": "failed", "channel": "inbox", "external_ref": "e3"},
|
||||
],
|
||||
DAY,
|
||||
NOW,
|
||||
)
|
||||
blocked = suppressed_delivery_ids(state_file, DAY, 3)
|
||||
assert set(blocked) == {"a"}
|
||||
assert blocked["a"].status == "shown"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# proactive_feedback_step
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_feedback_step_records_and_persists(tmp_path):
|
||||
"""One report lands in the truth source and is summarized in the response."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
_write_state(ws, open_topics=[_state_topic("Topic A")])
|
||||
response = await _async_feedback(
|
||||
ws,
|
||||
[
|
||||
{
|
||||
"topic_id": "a1",
|
||||
"status": "executed",
|
||||
"channel": "in_conversation",
|
||||
"outcome": "已完成",
|
||||
"external_ref": "s1",
|
||||
},
|
||||
{"topic_id": "a2", "status": "shown", "channel": "inbox", "external_ref": "ev1"},
|
||||
],
|
||||
)
|
||||
assert response.success is True
|
||||
assert "added 2" in response.answer
|
||||
payload = response.metadata["proactive_feedback"]
|
||||
assert payload["added"] == 2 and payload["updated"] == 0 and payload["total"] == 2
|
||||
assert payload["suppressed_topics"] == ["a1", "a2"]
|
||||
|
||||
state_file, _ = load_state(ws)
|
||||
assert {r.topic_id: r.status for r in state_file.deliveries} == {"a1": "executed", "a2": "shown"}
|
||||
# The step must not disturb the other truth-source sections.
|
||||
assert [t.title for t in state_file.open_topics] == ["Topic A"]
|
||||
assert state_file.resolved == [] and state_file.schedules == []
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
async def _async_feedback(ws, items, day=DAY, **kwargs):
|
||||
step = ProactiveFeedbackStep(app_context=ApplicationContext(workspace_dir=str(ws)), **kwargs)
|
||||
return await step(RuntimeContext(date=day, items=items, file_store=_FileStore(ws)))
|
||||
|
||||
|
||||
def test_feedback_step_creates_missing_truth_source(tmp_path):
|
||||
"""Reporting into a fresh workspace bootstraps an otherwise empty state file."""
|
||||
|
||||
async def run():
|
||||
response = await _async_feedback(tmp_path, [{"topic_id": "a1", "status": "shown"}])
|
||||
assert response.success is True
|
||||
state_file, _ = load_state(tmp_path)
|
||||
assert [r.topic_id for r in state_file.deliveries] == ["a1"]
|
||||
assert state_file.open_topics == []
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_feedback_step_without_items_is_a_noop(tmp_path):
|
||||
"""An empty report succeeds, writes nothing and says so."""
|
||||
|
||||
async def run():
|
||||
_write_state(tmp_path, open_topics=[_state_topic("Topic A")])
|
||||
before = (tmp_path / "daily" / "_proactive.yaml").read_text(encoding="utf-8")
|
||||
response = await _async_feedback(tmp_path, [])
|
||||
assert response.success is True
|
||||
assert response.answer == "Feedback: no delivery items reported"
|
||||
assert response.metadata["proactive_feedback"]["received"] == 0
|
||||
assert (tmp_path / "daily" / "_proactive.yaml").read_text(encoding="utf-8") == before
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_feedback_step_is_idempotent_across_runs(tmp_path):
|
||||
"""Retrying the same report does not duplicate records."""
|
||||
|
||||
async def run():
|
||||
_write_state(tmp_path)
|
||||
items = [{"topic_id": "a1", "status": "shown", "channel": "inbox", "external_ref": "ev1"}]
|
||||
await _async_feedback(tmp_path, items)
|
||||
response = await _async_feedback(tmp_path, items)
|
||||
assert "updated 1" in response.answer
|
||||
state_file, _ = load_state(tmp_path)
|
||||
assert len(state_file.deliveries) == 1
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_feedback_step_trims_by_retention(tmp_path):
|
||||
"""Records older than the retention window are pruned on write."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
_write_state_with_deliveries(
|
||||
ws,
|
||||
[{"topic_id": "old", "delivered_at": "2026-01-01T18:00:00+08:00", "status": "shown", "external_ref": "e"}],
|
||||
)
|
||||
response = await _async_feedback(ws, [{"topic_id": "new", "status": "shown"}], retention_days=30)
|
||||
assert response.metadata["proactive_feedback"]["trimmed"] == 1
|
||||
state_file, _ = load_state(ws)
|
||||
assert [r.topic_id for r in state_file.deliveries] == ["new"]
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# proactive_step (reader)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _seed_reader(ws, fid, eid):
|
||||
"""Two open topics, both on today's agenda; ``eid`` already delivered."""
|
||||
follow = _state_topic("记忆检索回归", topic_id_value=fid)
|
||||
extend = _state_topic("多跳 RAG 综述", topic_id_value=eid, kind="interest_extend")
|
||||
_write_state_with_deliveries(
|
||||
ws,
|
||||
[
|
||||
{
|
||||
"topic_id": eid,
|
||||
"agenda_date": DAY,
|
||||
"delivered_at": f"{DAY}T09:00:00+08:00",
|
||||
"channel": "inbox",
|
||||
"status": "shown",
|
||||
"external_ref": "ev1",
|
||||
},
|
||||
],
|
||||
open_topics=[follow, extend],
|
||||
)
|
||||
_write_interests_with_agenda(
|
||||
ws,
|
||||
[follow, extend],
|
||||
[_agenda_item(fid, "记忆检索回归"), _agenda_item(eid, "多跳 RAG 综述", kind="interest_extend")],
|
||||
)
|
||||
|
||||
|
||||
def test_reader_withholds_delivered_agenda_items(tmp_path):
|
||||
"""A delivered agenda item disappears from the read but stays auditable."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
fid, eid = topic_id("记忆检索回归"), topic_id("多跳 RAG 综述")
|
||||
_seed_reader(ws, fid, eid)
|
||||
step = ProactiveStep(app_context=ApplicationContext(workspace_dir=str(ws)))
|
||||
response = await step(RuntimeContext(date=DAY, include_content=False, file_store=_FileStore(ws)))
|
||||
|
||||
assert response.success is True
|
||||
assert [i["topic_id"] for i in response.answer["agenda"]] == [fid]
|
||||
assert response.answer["delivered_ids"] == [eid]
|
||||
assert "withheld 1" in response.answer["summary"]
|
||||
# Topics are material, not delivery units: they stay unfiltered.
|
||||
assert {t["id"] for t in response.answer["topics"]} == {fid, eid}
|
||||
assert response.metadata["delivered_ids"] == [eid]
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_reader_include_delivered_returns_everything(tmp_path):
|
||||
"""``include_delivered`` is the escape hatch for hosts doing their own dedup."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
fid, eid = topic_id("记忆检索回归"), topic_id("多跳 RAG 综述")
|
||||
_seed_reader(ws, fid, eid)
|
||||
step = ProactiveStep(app_context=ApplicationContext(workspace_dir=str(ws)))
|
||||
response = await step(
|
||||
RuntimeContext(date=DAY, include_content=False, include_delivered=True, file_store=_FileStore(ws)),
|
||||
)
|
||||
assert [i["topic_id"] for i in response.answer["agenda"]] == [fid, eid]
|
||||
assert "delivered_ids" not in response.answer
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_reader_cooldown_expiry_lets_an_item_back(tmp_path):
|
||||
"""A merely-shown item returns once the cooldown has passed."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
fid, eid = topic_id("记忆检索回归"), topic_id("多跳 RAG 综述")
|
||||
follow = _state_topic("记忆检索回归", topic_id_value=fid)
|
||||
extend = _state_topic("多跳 RAG 综述", topic_id_value=eid, kind="interest_extend")
|
||||
# The single-day exposure is read at DAY, so the delivery is backdated
|
||||
# instead: shown a week ago, default cooldown is three days.
|
||||
_write_state_with_deliveries(
|
||||
ws,
|
||||
[
|
||||
{
|
||||
"topic_id": eid,
|
||||
"agenda_date": "2026-08-06",
|
||||
"delivered_at": "2026-08-06T09:00:00+08:00",
|
||||
"channel": "inbox",
|
||||
"status": "shown",
|
||||
},
|
||||
],
|
||||
open_topics=[follow, extend],
|
||||
)
|
||||
_write_interests_with_agenda(
|
||||
ws,
|
||||
[follow, extend],
|
||||
[_agenda_item(fid, "记忆检索回归"), _agenda_item(eid, "多跳 RAG 综述", kind="interest_extend")],
|
||||
)
|
||||
step = ProactiveStep(app_context=ApplicationContext(workspace_dir=str(ws)))
|
||||
response = await step(RuntimeContext(date=DAY, include_content=False, file_store=_FileStore(ws)))
|
||||
assert [i["topic_id"] for i in response.answer["agenda"]] == [fid, eid]
|
||||
assert "delivered_ids" not in response.answer
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_reader_without_feedback_is_unchanged(tmp_path):
|
||||
"""No ``deliveries`` section means no withholding (backwards compatible)."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
fid, eid = topic_id("记忆检索回归"), topic_id("多跳 RAG 综述")
|
||||
follow = _state_topic("记忆检索回归", topic_id_value=fid)
|
||||
extend = _state_topic("多跳 RAG 综述", topic_id_value=eid, kind="interest_extend")
|
||||
_write_state(ws, open_topics=[follow, extend])
|
||||
_write_interests_with_agenda(
|
||||
ws,
|
||||
[follow, extend],
|
||||
[_agenda_item(fid, "记忆检索回归"), _agenda_item(eid, "多跳 RAG 综述", kind="interest_extend")],
|
||||
)
|
||||
step = ProactiveStep(app_context=ApplicationContext(workspace_dir=str(ws)))
|
||||
response = await step(RuntimeContext(date=DAY, include_content=False, file_store=_FileStore(ws)))
|
||||
assert [i["topic_id"] for i in response.answer["agenda"]] == [fid, eid]
|
||||
assert "delivered_ids" not in response.answer
|
||||
assert "withheld" not in response.answer["summary"]
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_reader_reminder_still_outranks_and_survives(tmp_path):
|
||||
"""Withholding a discovery item never reorders or drops a pinned reminder."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
rid, eid = topic_id("牙医预约"), topic_id("多跳 RAG 综述")
|
||||
reminder = _state_topic("牙医预约", topic_id_value=rid, kind="reminder", confidence=0.9)
|
||||
extend = _state_topic("多跳 RAG 综述", topic_id_value=eid, kind="interest_extend")
|
||||
_write_state_with_deliveries(
|
||||
ws,
|
||||
[
|
||||
{
|
||||
"topic_id": eid,
|
||||
"agenda_date": DAY,
|
||||
"delivered_at": f"{DAY}T09:00:00+08:00",
|
||||
"channel": "inbox",
|
||||
"status": "shown",
|
||||
},
|
||||
],
|
||||
open_topics=[reminder, extend],
|
||||
)
|
||||
_write_interests_with_agenda(
|
||||
ws,
|
||||
[reminder, extend],
|
||||
[
|
||||
_agenda_item(rid, "牙医预约", kind="reminder", scenario_type="prepare_upcoming"),
|
||||
_agenda_item(eid, "多跳 RAG 综述", kind="interest_extend"),
|
||||
],
|
||||
)
|
||||
step = ProactiveStep(app_context=ApplicationContext(workspace_dir=str(ws)))
|
||||
response = await step(RuntimeContext(date=DAY, include_content=False, file_store=_FileStore(ws)))
|
||||
assert [i["topic_id"] for i in response.answer["agenda"]] == [rid]
|
||||
assert response.answer["agenda"][0]["kind"] == "reminder"
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_reader_reminder_is_exempt_from_the_cooldown(tmp_path):
|
||||
"""A reminder delivered inside the window is served again: due means due.
|
||||
|
||||
Reminder ids derive from a stable schedule title, so withholding them like
|
||||
discovery items would silence a recurring reminder for the whole cooldown
|
||||
after a single delivery.
|
||||
"""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
rid = topic_id("牙医预约")
|
||||
reminder = _state_topic("牙医预约", topic_id_value=rid, kind="reminder", confidence=0.9)
|
||||
_write_state_with_deliveries(
|
||||
ws,
|
||||
[
|
||||
{
|
||||
"topic_id": rid,
|
||||
"agenda_date": DAY,
|
||||
"delivered_at": f"{DAY}T09:00:00+08:00",
|
||||
"channel": "inbox",
|
||||
"status": "shown",
|
||||
},
|
||||
],
|
||||
open_topics=[reminder],
|
||||
)
|
||||
_write_interests_with_agenda(
|
||||
ws,
|
||||
[reminder],
|
||||
[_agenda_item(rid, "牙医预约", kind="reminder", scenario_type="prepare_upcoming")],
|
||||
)
|
||||
step = ProactiveStep(app_context=ApplicationContext(workspace_dir=str(ws)))
|
||||
response = await step(RuntimeContext(date=DAY, include_content=False, file_store=_FileStore(ws)))
|
||||
assert [i["topic_id"] for i in response.answer["agenda"]] == [rid]
|
||||
assert response.answer["agenda"][0]["kind"] == "reminder"
|
||||
assert "delivered_ids" not in response.answer
|
||||
assert "withheld" not in response.answer["summary"]
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# proactive_agenda_step (writer)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_agenda_step_suppresses_delivered_candidate(tmp_path):
|
||||
"""A reported candidate leaves the agenda with an auditable reason."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
_touch(ws / "daily" / DAY / "session.md", "we discussed the eval plan")
|
||||
_write_state_with_deliveries(
|
||||
ws,
|
||||
[
|
||||
{
|
||||
"topic_id": topic_id("任务一"),
|
||||
"agenda_date": DAY,
|
||||
"delivered_at": f"{DAY}T09:00:00+08:00",
|
||||
"channel": "in_conversation",
|
||||
"status": "executed",
|
||||
"external_ref": "s1",
|
||||
},
|
||||
],
|
||||
)
|
||||
extract_reply = _reply(
|
||||
follow_ups=[
|
||||
_topic("任务一", confidence=0.9, paths=[f"daily/{DAY}/session.md"]),
|
||||
_topic("任务二", confidence=0.8, paths=[f"daily/{DAY}/session.md"]),
|
||||
],
|
||||
)
|
||||
_, _, _, context, resps = await _run_full_chain(ws, [extract_reply])
|
||||
assert all(r.success for r in resps)
|
||||
|
||||
state = context.get("proactive")
|
||||
assert state["suppressed_delivered"] == 1
|
||||
data = yaml.safe_load(_interests(ws).read_text(encoding="utf-8"))
|
||||
assert [i["title"] for i in data["agenda"]] == ["任务二"]
|
||||
suppressed = {s["topic_id"]: s["reason"] for s in data["suppressed"]}
|
||||
reason = suppressed[topic_id("任务一")]
|
||||
assert reason.startswith("already delivered on")
|
||||
assert "in_conversation" in reason and "executed" in reason
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_agenda_step_reminder_is_exempt_from_the_cooldown(tmp_path):
|
||||
"""A reported reminder is pinned again while a reported discovery item is not."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
_touch(ws / "daily" / DAY / "session.md", "牙医复诊在即，评审材料还差一版")
|
||||
path = f"daily/{DAY}/session.md"
|
||||
rid = topic_id("牙医复诊")
|
||||
_write_state_with_deliveries(
|
||||
ws,
|
||||
[
|
||||
{
|
||||
"topic_id": rid,
|
||||
"agenda_date": DAY,
|
||||
"delivered_at": f"{DAY}T09:00:00+08:00",
|
||||
"channel": "inbox",
|
||||
"status": "shown",
|
||||
},
|
||||
{
|
||||
"topic_id": topic_id("任务一"),
|
||||
"agenda_date": DAY,
|
||||
"delivered_at": f"{DAY}T09:00:00+08:00",
|
||||
"channel": "inbox",
|
||||
"status": "shown",
|
||||
},
|
||||
],
|
||||
)
|
||||
extract_reply = _reply(
|
||||
reminders=[_reminder("牙医复诊", due_at="2026-08-14T15:00", paths=[path])],
|
||||
follow_ups=[
|
||||
_topic("任务一", confidence=0.9, paths=[path]),
|
||||
_topic("任务二", confidence=0.8, paths=[path]),
|
||||
],
|
||||
)
|
||||
with patch(EXTRACT_CLOCK, return_value=NOW):
|
||||
_, _, _, context, resps = await _run_full_chain(ws, [extract_reply])
|
||||
assert all(r.success for r in resps)
|
||||
|
||||
# only the discovery candidate counts as suppressed by the cooldown
|
||||
assert context.get("proactive")["suppressed_delivered"] == 1
|
||||
data = yaml.safe_load(_interests(ws).read_text(encoding="utf-8"))
|
||||
titles = [i["title"] for i in data["agenda"]]
|
||||
assert "牙医复诊" in titles
|
||||
assert "任务一" not in titles
|
||||
assert data["agenda"][0]["kind"] == "reminder" # still pinned first
|
||||
suppressed = {s["topic_id"] for s in data["suppressed"]}
|
||||
assert rid not in suppressed
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_agenda_step_without_feedback_is_unchanged(tmp_path):
|
||||
"""No delivery log means the deterministic fallback agenda stays intact."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
_touch(ws / "daily" / DAY / "session.md", "material")
|
||||
extract_reply = _reply(
|
||||
follow_ups=[
|
||||
_topic("任务一", confidence=0.9, paths=[f"daily/{DAY}/session.md"]),
|
||||
_topic("任务二", confidence=0.7, paths=[f"daily/{DAY}/session.md"]),
|
||||
],
|
||||
)
|
||||
_, _, _, context, resps = await _run_full_chain(ws, [extract_reply])
|
||||
assert all(r.success for r in resps)
|
||||
assert context.get("proactive")["suppressed_delivered"] == 0
|
||||
data = yaml.safe_load(_interests(ws).read_text(encoding="utf-8"))
|
||||
assert [i["title"] for i in data["agenda"]] == ["任务一", "任务二"]
|
||||
assert data["suppressed"] == []
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_agenda_step_cooldown_zero_only_blocks_same_day(tmp_path):
|
||||
"""``redelivery_cooldown_days=0`` still blocks a delivery reported today."""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
_touch(ws / "daily" / DAY / "session.md", "material")
|
||||
_write_state_with_deliveries(
|
||||
ws,
|
||||
[
|
||||
{
|
||||
"topic_id": topic_id("任务一"),
|
||||
"agenda_date": DAY,
|
||||
"delivered_at": f"{DAY}T09:00:00+08:00",
|
||||
"channel": "inbox",
|
||||
"status": "shown",
|
||||
},
|
||||
],
|
||||
)
|
||||
extract_reply = _reply(follow_ups=[_topic("任务一", confidence=0.9, paths=[f"daily/{DAY}/session.md"])])
|
||||
_, _, _, context, resps = await _run_full_chain(
|
||||
ws,
|
||||
[extract_reply],
|
||||
agenda_kwargs={"redelivery_cooldown_days": 0},
|
||||
)
|
||||
assert all(r.success for r in resps)
|
||||
assert context.get("proactive")["suppressed_delivered"] == 1
|
||||
data = yaml.safe_load(_interests(ws).read_text(encoding="utf-8"))
|
||||
assert data["agenda"] == []
|
||||
assert data["push"] is False
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Host interop: the report shape a consumer actually sends
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_consumer_report_shape_round_trips(tmp_path):
|
||||
"""A host report in the wire shape the consumer sends is stored verbatim and withholds.
|
||||
|
||||
Locks the interop contract on three points that are easy to break silently:
|
||||
every field of the report survives (an unknown ``consumer`` label must not be
|
||||
dropped, a known ``channel`` must not be coerced to ``other``), a UTC
|
||||
``delivered_at`` still suppresses on a later local business day, and the
|
||||
reader publishes ``delivered_ids`` in metadata for the host to count.
|
||||
"""
|
||||
|
||||
async def run():
|
||||
ws = tmp_path
|
||||
fid, eid = topic_id("记忆检索回归"), topic_id("多跳 RAG 综述")
|
||||
follow = _state_topic("记忆检索回归", topic_id_value=fid)
|
||||
extend = _state_topic("多跳 RAG 综述", topic_id_value=eid, kind="interest_extend")
|
||||
_write_state(ws, open_topics=[follow, extend])
|
||||
_write_interests_with_agenda(
|
||||
ws,
|
||||
[follow, extend],
|
||||
[_agenda_item(fid, "记忆检索回归"), _agenda_item(eid, "多跳 RAG 综述", kind="interest_extend")],
|
||||
)
|
||||
|
||||
# 23:30Z on the previous UTC day is 07:30 on DAY in +08:00: the consumer
|
||||
# stamps UTC, the business clock is local, so the date prefix lags.
|
||||
report = {
|
||||
"topic_id": eid,
|
||||
"agenda_date": DAY,
|
||||
"delivered_at": "2026-08-12T23:30:00.123456+00:00",
|
||||
"channel": "in_conversation",
|
||||
"consumer": "qwenpaw",
|
||||
"status": "shown",
|
||||
"outcome": "",
|
||||
"external_ref": "session-7f3a",
|
||||
}
|
||||
response = await _async_feedback(ws, [report])
|
||||
assert response.success is True and "added 1" in response.answer
|
||||
|
||||
state_file, _ = load_state(ws)
|
||||
assert len(state_file.deliveries) == 1
|
||||
stored = state_file.deliveries[0]
|
||||
assert stored.consumer == "qwenpaw" # a free-form host label is kept as sent
|
||||
assert stored.channel == "in_conversation" # in the vocabulary, so not coerced
|
||||
assert stored.external_ref == "session-7f3a"
|
||||
assert stored.agenda_date == DAY
|
||||
assert stored.delivered_at.startswith("2026-08-12T23:30:00")
|
||||
assert is_delivery_suppressed(stored, DAY, 3) is True
|
||||
|
||||
# Re-reporting the same delivery updates in place instead of piling up.
|
||||
again = await _async_feedback(ws, [dict(report, status="executed")])
|
||||
assert again.success is True
|
||||
state_file, _ = load_state(ws)
|
||||
assert len(state_file.deliveries) == 1
|
||||
assert state_file.deliveries[0].status == "executed"
|
||||
|
||||
reader = ProactiveStep(app_context=ApplicationContext(workspace_dir=str(ws)))
|
||||
read = await reader(RuntimeContext(date=DAY, include_content=False, file_store=_FileStore(ws)))
|
||||
assert read.success is True
|
||||
assert [item["topic_id"] for item in read.answer["agenda"]] == [fid]
|
||||
assert read.metadata["delivered_ids"] == [eid]
|
||||
|
||||
asyncio.run(run())
|
||||
File diff suppressed because it is too large
Load diff
Loading…
Add table
Reference in a new issue