diff --git a/.github/workflows/windows-smoke.yml b/.github/workflows/windows-smoke.yml index 522f1414..8a1f787a 100644 --- a/.github/workflows/windows-smoke.yml +++ b/.github/workflows/windows-smoke.yml @@ -32,8 +32,15 @@ jobs: - name: Install package run: | python -m pip install --upgrade pip setuptools wheel - pip install agentscope - pip install -e . + pip install -e ".[dev,core]" - name: Run version job run: reme start service.backend=cli job=version + + - name: Run Windows path tests + run: | + python -m pytest ` + tests/unit/test_auto_dream.py::test_scan_day_files_includes_nested_md_and_excludes_interests ` + tests/unit/test_auto_dream.py::test_dream_extract_matches_posix_catalog_paths ` + tests/unit/test_read_with_neighbors.py::test_read_with_neighbors_uses_posix_nested_path ` + -v diff --git a/docs/en/auto_dream.md b/docs/en/auto_dream.md index d84a24ab..749ebe46 100644 --- a/docs/en/auto_dream.md +++ b/docs/en/auto_dream.md @@ -9,7 +9,7 @@ units into `digest/`, and generates the day's `interests.yaml` for proactive use

Its daily inputs usually come from [Auto Memory](./auto_memory.md) and [Auto Resource](./auto_resource.md). For the file -semantics of `digest/`, `derived_from::`, and wikilinks, see [Memory as File](./memory_as_file.md). For the linking strategy +semantics of `digest/`, Sources sections, and wikilinks, see [Memory as File](./memory_as_file.md). For the linking strategy used during Integrate, see [Auto Link](./auto_link.md). To read `interests.yaml`, use [Proactive](./proactive.md). ## Configuration @@ -198,8 +198,8 @@ jobs: `auto_dream` consumes only daily inputs and does not rewrite daily bodies. Daily preserves facts and the original situation; digest is the abstracted long-term memory layer. -`digest` is not a copy of the source text. Its body should preserve reusable abstractions, while details point back to sources -through `derived_from:: [[daily//...]]`. Links follow the workspace-relative wikilink semantics described in +`digest` is not a copy of the source text. Its body should preserve reusable abstractions, while a Sources section points +back with entries such as `- [[daily//...]]`. Links follow the workspace-relative wikilink semantics described in [Memory as File](./memory_as_file.md). `auto_dream` does not invent an overview from nothing. Only content that actually appears in daily input and is extracted as diff --git a/docs/en/auto_link.md b/docs/en/auto_link.md index dde6afcb..0a97976a 100644 --- a/docs/en/auto_link.md +++ b/docs/en/auto_link.md @@ -32,7 +32,7 @@ link to multiple sources and multiple related digest nodes. |---|---| | The same memory already exists | Recall and update the existing node instead of creating a duplicate. | | New and existing material are related | Write workspace-relative wikilinks into the body. | -| A digest node is disconnected from its sources | Point back to daily/resource source material with `derived_from:: [[...]]`. | +| A digest node is disconnected from its sources | Add daily/resource links under a `## Sources` section. | | A node contains only isolated prose | Add links to related digest nodes on both CREATE and UPDATE. | ## Toolchain @@ -77,20 +77,22 @@ Every unit must select one action: | Action | Linking semantics | |---|---| | `CREATE` | Write a new `digest//.md` and add source and related-node links to its body. | -| `CORROBORATE` | The same abstraction appeared again; append a new `derived_from:: [[...]]` and strengthen the description when needed. | +| `CORROBORATE` | The same abstraction appeared again; append its source link and strengthen the description when needed. | | `REFINE` | New material extends the existing node; insert the additional content in the appropriate section and preserve existing links. | | `CORRECT` | New material corrects the existing node; use source links to identify the basis for the correction. | -An UPDATE should be additive whenever possible: do not delete existing wikilinks or `derived_from` entries. This prevents +An UPDATE should be additive whenever possible: do not delete existing wikilinks or source entries. This prevents later graph indexing and retrieval from losing edges. ### 3. Write source edges -Source edges use Markdown wikilinks: +Source edges are ordinary wikilinks grouped under a Markdown heading: ```markdown -derived_from:: [[daily/2026-06-20/session.md]] -derived_from:: [[resource/2026-06-20/paper.md]] +## Sources + +- [[daily/2026-06-20/session.md]] +- [[resource/2026-06-20/paper.md]] ``` These edges represent the evidence behind a digest node. Plain-text descriptions do not count as source edges because only @@ -99,17 +101,14 @@ wikilinks can be parsed reliably by the file graph. For the complete parsing rul ### 4. Write relationships between digest nodes -Relationships between digest nodes also use complete workspace-relative paths: +Relationships between digest nodes use complete workspace-relative paths woven into natural prose: ```markdown -relates_to:: [[digest/wiki/hybrid-search.md]] -depends_on:: [[digest/procedure/rebuild-index.md]] -blocks_on:: [[digest/personal/team-review-preference.md]] +This design extends [[digest/wiki/hybrid-search.md]] and uses +[[digest/procedure/rebuild-index.md]]. Follow +[[digest/personal/team-review-preference.md]] during review. ``` -Predicates are open-ended. Common forms include `relates_to::`, `depends_on::`, and `blocks_on::`. The predicate sits outside -the brackets, while the target path goes inside `[[...]]` and should include the `.md` suffix. - ## Bucket Differences `auto_link` adjusts the shape of its output according to the unit bucket: diff --git a/docs/en/memory_as_file.md b/docs/en/memory_as_file.md index 6be4d56d..da41506c 100644 --- a/docs/en/memory_as_file.md +++ b/docs/en/memory_as_file.md @@ -23,7 +23,7 @@ ReMe represents memory as files not merely for convenient storage, but to give l |---|---| | Readable | Users can open the workspace directly and read daily notes, digest nodes, and source material like ordinary notes. | | Editable | Users and agents can correct, extend, move, or delete memory with file operations, without a specialized database client. | -| Traceable | Long-term conclusions in digest can point back to daily, resource, or session sources through `derived_from:: [[...]]`. | +| Traceable | Long-term conclusions in digest can point back to daily, resource, or session files from a Sources section. | | Portable | The workspace is an ordinary directory. Markdown, JSONL, YAML, and resource files can be backed up, synchronized, versioned, or moved to other tools. | | Indexable | Although the files are plain text, ReMe parses frontmatter, chunks, and wikilinks to build a retrieval index and file graph. | | Collaborative | Humans judge and correct; agents organize, link, and retrieve. Both operate on the same files. | @@ -131,9 +131,7 @@ tags: [new energy, solar] # Conclusions The solar supply chain consists of [[digest/wiki/polysilicon.md]], wafers, cells, and modules. - -upstream:: [[digest/wiki/polysilicon.md]] -[company:: [[digest/wiki/longi.md|LONGi]]] +One major producer is [[digest/wiki/longi.md|LONGi]]. ``` ### Frontmatter @@ -163,15 +161,18 @@ confidence: observed The user repeatedly asks documentation to explain motivation, boundaries, and examples while avoiding marketing language. -derived_from:: [[daily/2026-06-20/session-a.md]] -related:: [[digest/procedure/technical-documentation.md]] +Apply this preference when following [[digest/procedure/technical-documentation.md]]. + +## Sources + +- [[daily/2026-06-20/session-a.md]] ``` This has three benefits: 1. `name` and `description` serve as lightweight summaries in lists, recall results, and agent decisions. 2. The body can carry fuller facts, conditions, counterexamples, and sources. -3. Typed wikilinks such as `derived_from::` and `related::` can be parsed by the graph and maintained when files move. +3. Ordinary wikilinks can be parsed by the graph and maintained when files move. Frontmatter is best for stable, short, structured fields; the body is best for explanations meant for people. Do not put long body text into YAML fields. @@ -181,10 +182,10 @@ body text into YAML fields. Wikilinks express relationships between files with `[[...]]`: ```text -[[digest/wiki/solar.md]] -[[digest/wiki/solar.md#supply-chain]] -[[digest/wiki/solar.md|solar]] -![[resource/2026-06-01/report.md]] +[[daily/2026-06-20/session.md]] +[[notes/example.md#L9]] +[[notes/example.md#L9-L10]] +[[notes/example.md#L9-L10,L15-L20]] ``` ReMe wikilinks use **literal path semantics**: @@ -196,54 +197,59 @@ ReMe wikilinks use **literal path semantics**: ReMe does not append `.md` automatically, search by filename, or automatically resolve folder notes. Use complete workspace-relative paths with their extensions. +Ordinary Markdown links such as `[label](../wiki/example.md)` do not create `FileLink` edges and are not rewritten by move or +retarget operations. + +Anchors such as `#L9`, `#L9-L10`, and `#L9-L10,L15-L20` remain ordinary `target_anchor` strings in the graph. The graph +parser does not validate line-anchor syntax, so values such as `#L0`, `#L10-L9`, and `#L9,` are also stored. The `read` job +does not interpret an anchor appended to `path`; use the separate 1-based, inclusive `start_line` and `end_line` arguments to +read a range, for example `read(path="digest/wiki/solar.md", start_line=9, end_line=10)`. + Wikilinks support these behaviors: ```text body link -> create a FileLink -predicate:: link -> create a FileLink with a relationship name move a file -> rewrite [[old path]] in inbound edges by default delete a file -> return remaining inbound edges so references can be cleaned up search match -> expand inbound and outbound links to provide context ``` -Supported relationship forms: - -```markdown -industry:: [[digest/wiki/new-energy.md]] -[competitor:: [[digest/wiki/byd.md]]] -``` - Parsed result: ```text FileLink source_path = current file - target_path = digest/wiki/new-energy.md - predicate = industry + target_path = notes/example.md + target_anchor = L9-L10,L15-L20 ``` +Older documents containing wrappers such as `related:: [[path]]`, +`- related:: [[path]]`, or `[related:: [[path]]]` remain readable. ReMe +ignores the surrounding text and indexes the inner `[[path]]` as an ordinary +link. After upgrading from a version that stored typed links, run `reme reindex` +once to rebuild the derived graph without the removed relationship field. + ### Sources and Relationships The two most important link types in ReMe are source links and conceptual relationship links. -A source link explains where a long-term memory came from: +A Sources section records where a long-term memory came from: ```markdown -derived_from:: [[daily/2026-06-20/session-a.md]] -derived_from:: [[resource/2026-06-20/report.pdf]] +## Sources + +- [[daily/2026-06-20/session-a.md]] +- [[resource/2026-06-20/report.pdf]] ``` -A conceptual relationship link explains which other long-term memories relate to the node: +A conceptual relationship link explains which other long-term memories relate to the node. Weave it into natural prose: ```markdown -related:: [[digest/wiki/solar-supply-chain.md]] -depends_on:: [[digest/procedure/research-report-analysis.md]] -contrasts_with:: [[digest/wiki/central-inverter.md]] +This analysis extends [[digest/wiki/solar-supply-chain.md]], follows +[[digest/procedure/research-report-analysis.md]], and contrasts with +[[digest/wiki/central-inverter.md]]. ``` -Ordinary body wikilinks also create graph edges, but when the relationship itself has semantic value, prefer -`predicate:: [[path]]`. This makes the meaning of links clearer to search, graph traversal, and later agent integration. - ## Human and Agent Editing Because memory is stored as files, users can edit the workspace directly, while agents can read and write the same files @@ -257,8 +263,8 @@ through ReMe's file tools. Both follow the same conventions: | Delete a file | Check inbound links first. ReMe's delete tool returns source files that still point to the target, making dangling references easier to clean up. | | Edit metadata | Use frontmatter for short fields. When the body changes substantially, update `description` as well. | -A practical rule is: **an agent may rewrite the wording, but it must not lose evidence edges**. In particular, -`derived_from:: [[...]]` and existing digest-to-digest wikilinks are the basis for traceable and extensible long-term memory. +A practical rule is: **an agent may rewrite the wording, but it must not lose evidence edges**. In particular, Sources entries +and existing digest-to-digest wikilinks are the basis for traceable and extensible long-term memory. ## Path Semantics diff --git a/docs/en/memory_search.md b/docs/en/memory_search.md index 9ee3605a..5f2c3784 100644 --- a/docs/en/memory_search.md +++ b/docs/en/memory_search.md @@ -66,7 +66,7 @@ stable batch of changes. 4. For a deleted file, remove its records from `file_store`, `keyword_index`, and `file_graph`. 5. When changes exist, dump state to `metadata/` so it can be restored on the next startup. -The Markdown chunker parses YAML frontmatter, heading structure, and `[[...]]` into `FileNode`, `FileChunk`, and `FileLink` +The Markdown chunker parses YAML frontmatter, heading structure, and wikilinks into `FileNode`, `FileChunk`, and `FileLink` objects. For detailed chunking rules, see [Memory as File](./memory_as_file.md#memory-chunking). ### Index Optimization @@ -182,7 +182,8 @@ and then adds the new chunk text. Deletion is lazy; the index can later be compa "Progressive" in Memory Search does not mean putting the entire repository into one result. Retrieval expands in three layers: 1. Chunk recall: return only the `limit` most relevant text fragments. -2. File location: each result includes `path:start_line-end_line`, allowing the caller to read the source precisely with `read`. +2. File location: each result includes `path:start_line-end_line`. Pass the path and line bounds separately as `path`, + `start_line`, and `end_line` when calling `read`; the range is not part of the `path` value. 3. Link neighbors: call `expand_links()` for each matched file and expand at most `max_links_per_direction` outlinks and inlinks. @@ -194,7 +195,7 @@ matched chunk -> file_store.get_outlinks(path) -> file_store.get_inlinks(path) -> file_store.get_nodes(neighbor_paths) - -> render neighbor path, name, description, predicate, and anchor + -> render neighbor path, name, description, and anchor ``` This keeps search results short while still showing which long-term nodes, resources, or other daily notes a memory connects @@ -216,10 +217,8 @@ Typical text structure: ...matched memory fragment... outlinks (2): -> digest/indexing.md name="Indexing" description="..." - via predicate=related inlinks (1): <- daily/2026-06-19.md name="..." - via plain ``` `counts` reports how many vector and keyword candidates were recalled and how many results were ultimately returned. With diff --git a/docs/en/reme_scene.md b/docs/en/reme_scene.md index 3ba652bd..d7ce86d5 100644 --- a/docs/en/reme_scene.md +++ b/docs/en/reme_scene.md @@ -110,17 +110,19 @@ name: Cobalt description: A key raw material for lithium-battery cathodes, with production concentrated in the DRC --- -downstream_product:: [[digest/wiki/ternary-cathodes.md]] -producer:: [[digest/wiki/glencore.md]] -source_event:: [[daily/2026-05-18/2026-05-18-close.md]] - # Cobalt +Used by [[digest/wiki/ternary-cathodes.md]]; a major producer is [[digest/wiki/glencore.md]]. + ## Supply Glencore's third-quarter cobalt output fell 18% year over year. Continue monitoring how tighter supply affects prices. ## Policy risk Changes to mining-rights policy in the DRC may affect KFM mine operations and should be tracked together with CMOC. + +## Sources + +- [[daily/2026-05-18/2026-05-18-close.md]] ``` Note that wikilinks use literal path semantics. Prefer complete workspace-relative paths with the `.md` extension. ReMe @@ -185,10 +187,10 @@ The result shape is: Glencore's third-quarter cobalt output fell 18% year over year... outlinks: - -> digest/wiki/ternary-cathodes.md name="Ternary Cathodes" via predicate=downstream_product - -> digest/wiki/glencore.md name="Glencore" via predicate=producer + -> digest/wiki/ternary-cathodes.md name="Ternary Cathodes" + -> digest/wiki/glencore.md name="Glencore" inlinks: - <- digest/wiki/ternary-cathodes.md name="Ternary Cathodes" via predicate=upstream_material + <- digest/wiki/ternary-cathodes.md name="Ternary Cathodes" ========== digest/wiki/ternary-cathodes.md:5-18 [score=0.0139 keyword=3.2017] ========== ... @@ -301,11 +303,10 @@ name: TypeScript project build OOM diagnostic path description: When a build stalls and memory grows, check the type-checking process first --- -source_event:: [[daily/2026-03-10/build-oom-2026-03-10.md]] -related_preference:: [[digest/personal/code-style.md]] - # TypeScript Project Build OOM Diagnostic Path +Apply [[digest/personal/code-style.md]] while following this runbook. + ## Symptoms The build stalls near the end. CPU usage is low, but memory keeps growing. @@ -317,6 +318,10 @@ The build stalls near the end. CPU usage is low, but memory keeps growing. ## Known ineffective paths - Deleting `.cache` alone did not resolve the issue on 2026-03-10. - Upgrading the terser plugin did not resolve the issue on 2026-03-10. + +## Sources + +- [[daily/2026-03-10/build-oom-2026-03-10.md]] ``` Example `digest/personal/code-style.md`: @@ -411,13 +416,14 @@ name: Alice description: A friend of the user who often recommends reading material --- -recommended_book:: [[digest/wiki/deep-work.md]] -source_event:: [[daily/2026-04-20/lunch-with-alice.md]] - # Alice ## Reading recommendations -At lunch on 2026-04-20, Alice recommended a book about attention and deep work. +At lunch on 2026-04-20, Alice recommended [[digest/wiki/deep-work.md]], a book about attention and deep work. + +## Sources + +- [[daily/2026-04-20/lunch-with-alice.md]] ``` ### An associative recall @@ -439,7 +445,7 @@ Matches: ```text digest/personal/alice.md outlinks: - -> digest/wiki/deep-work.md via predicate=recommended_book + -> digest/wiki/deep-work.md daily/2026-04-20/lunch-with-alice.md ``` diff --git a/docs/zh/auto_dream.md b/docs/zh/auto_dream.md index 315c92b9..333f58ed 100644 --- a/docs/zh/auto_dream.md +++ b/docs/zh/auto_dream.md @@ -7,7 +7,7 @@ ReMe Auto Dream and Proactive 从 daily 到 digest 再到 proactive 的流程

-它消费的 daily 输入通常来自 [Auto Memory](./auto_memory.md) 和 [Auto Resource](./auto_resource.md)。`digest/`、`derived_from::` +它消费的 daily 输入通常来自 [Auto Memory](./auto_memory.md) 和 [Auto Resource](./auto_resource.md)。`digest/`、Sources 章节 和 wikilink 的文件语义见 [Memory as File](./memory_as_file.md);Integrate 阶段的链接策略详见 [Auto Link](./auto_link.md)。 `interests.yaml` 的读取接口见 [Proactive](./proactive.md)。 @@ -189,7 +189,7 @@ jobs: `auto_dream` 只消费 daily 输入,不改写 daily 正文。daily 是事实和现场记录,digest 才是抽象后的长期记忆层。 -`digest` 不是原文复制。正文应保留可复用抽象,细节通过 `derived_from:: [[daily//...]]` 指回来源。链接写法遵循 +`digest` 不是原文复制。正文应保留可复用抽象,细节通过 Sources 章节中的 `- [[daily//...]]` 指回来源。链接写法遵循 [Memory as File](./memory_as_file.md) 中的 workspace-relative wikilink 语义。 `auto_dream` 不凭空生成总览。只有 daily 输入中确实出现、并被抽取为 unit 或 topic 的内容,才会进入 digest 或 diff --git a/docs/zh/auto_link.md b/docs/zh/auto_link.md index ae50bb0b..c34c8423 100644 --- a/docs/zh/auto_link.md +++ b/docs/zh/auto_link.md @@ -30,7 +30,7 @@ Integrate 阶段对每个 unit 独立运行。一个 unit 只落到一个目标 |--------------|----------------------------------------------------| | 已有相同记忆 | 召回后更新旧节点,而不是重复创建。 | | 新旧材料有关联 | 在正文中写入 workspace-relative wikilink。 | -| digest 与来源断开 | 用 `derived_from:: [[...]]` 指回 daily/resource 原始材料。 | +| digest 与来源断开 | 在 `## Sources` 章节加入指向 daily/resource 的链接。 | | 节点只有孤立正文 | 在 CREATE 和 UPDATE 时都补充相关 digest 节点链接。 | ## 工具链 @@ -73,19 +73,21 @@ Agent 先用 unit 的触发条件、动词、名词、同义词和可能的 fail | 动作 | 链接含义 | |---------------|-----------------------------------------------------| | `CREATE` | 写入新的 `digest//.md`,并在新正文里加入来源和相关节点链接。 | -| `CORROBORATE` | 同一抽象再次出现,追加新的 `derived_from:: [[...]]`,必要时强化描述。 | +| `CORROBORATE` | 同一抽象再次出现,追加来源链接,必要时强化描述。 | | `REFINE` | 新材料扩展了旧节点,把补充内容插入合适段落,并保留旧链接。 | | `CORRECT` | 新材料修正旧节点,用来源链接标出修正依据。 | -UPDATE 必须尽量只增不删:不要删除已有 wikilink 或 `derived_from`。这是为了让后续图谱索引和检索不会丢边。 +UPDATE 必须尽量只增不删:不要删除已有 wikilink 或来源条目。这是为了让后续图谱索引和检索不会丢边。 ### 3. 写来源边 -来源边使用 markdown wikilink: +来源边是归档在 Markdown 固定章节下的普通 Wikilink: ```markdown -derived_from:: [[daily/2026-06-20/session.md]] -derived_from:: [[resource/2026-06-20/paper.md]] +## Sources + +- [[daily/2026-06-20/session.md]] +- [[resource/2026-06-20/paper.md]] ``` 这些边表示 digest 节点的证据来源。纯文本描述不算来源边,因为只有 wikilink 能被 file graph 稳定解析。更完整的 wikilink @@ -93,17 +95,14 @@ derived_from:: [[resource/2026-06-20/paper.md]] ### 4. 写 digest 关联边 -digest 之间的关联也使用完整 workspace-relative 路径: +digest 之间的关联使用完整 workspace-relative 路径,并自然织入正文: ```markdown -relates_to:: [[digest/wiki/hybrid-search.md]] -depends_on:: [[digest/procedure/rebuild-index.md]] -blocks_on:: [[digest/personal/team-review-preference.md]] +这个设计扩展了 [[digest/wiki/hybrid-search.md]],并使用 +[[digest/procedure/rebuild-index.md]]。评审时遵循 +[[digest/personal/team-review-preference.md]]。 ``` -谓词是开放的,常见写法包括 `relates_to::`、`depends_on::`、`blocks_on::`。谓词在括号外,目标路径在 `[[...]]` 内,并且应包含 -`.md` 后缀。 - ## Bucket 差异 `auto_link` 的规则会随 unit bucket 调整写入形态: diff --git a/docs/zh/memory_as_file.md b/docs/zh/memory_as_file.md index 1be2de34..c38fb0db 100644 --- a/docs/zh/memory_as_file.md +++ b/docs/zh/memory_as_file.md @@ -22,7 +22,7 @@ ReMe 把记忆设计成文件,不只是为了“方便存储”,而是为了 |----------|----------------------------------------------------------------------| | 可读 | 用户可以直接打开 workspace,像读普通笔记一样读 daily、digest 和原始材料。 | | 可编辑 | 用户和 Agent 都能用文件操作修正、补充、移动或删除记忆,不必依赖专用数据库客户端。 | -| 可追溯 | digest 中的长期结论可以通过 `derived_from:: [[...]]` 回到 daily、resource 或 session 原文。 | +| 可追溯 | digest 中的长期结论可以通过 Sources 章节回到 daily、resource 或 session 原文。 | | 可迁移 | workspace 是普通目录,Markdown、JSONL、YAML 和资源文件可以被备份、同步、版本管理或迁移到其他工具。 | | 可索引 | 文件虽然是普通文本,但 ReMe 会解析 frontmatter、chunk、wikilink,构建检索索引和文件图谱。 | | 可协作 | 人负责判断和修正,Agent 负责整理、链接和检索;二者看到和操作的是同一套文件。 | @@ -124,9 +124,7 @@ tags: [新能源, 光伏] # 结论 光伏产业链可以拆成 [[digest/wiki/硅料.md]]、硅片、电池片和组件。 - -upstream:: [[digest/wiki/硅料.md]] -[company:: [[digest/wiki/隆基绿能.md|隆基]]] +主要生产商包括 [[digest/wiki/隆基绿能.md|隆基]]。 ``` ### Frontmatter @@ -156,15 +154,18 @@ confidence: observed 用户多次要求文档补充动机、边界和例子,但避免营销式表述。 -derived_from:: [[daily/2026-06-20/session-a.md]] -related:: [[digest/procedure/技术文档写作.md]] +执行 [[digest/procedure/技术文档写作.md]] 时应用这个偏好。 + +## Sources + +- [[daily/2026-06-20/session-a.md]] ``` 这样做有三个好处: 1. `name` 和 `description` 可以在列表、召回结果和 Agent 判断中作为轻量摘要。 2. 正文可以承载更完整的事实、条件、反例和来源。 -3. `derived_from::`、`related::` 这类 typed wikilink 可以被图谱解析,后续移动文件时也能被维护。 +3. 普通 Wikilink 可以被图谱解析,后续移动文件时也能被维护。 Frontmatter 适合放稳定、短小、结构化的字段;正文适合放需要人读的解释。不要把大段正文塞进 YAML 字段。 @@ -173,10 +174,10 @@ Frontmatter 适合放稳定、短小、结构化的字段;正文适合放需 Wikilink 用 `[[...]]` 表达文件之间的关系: ```text -[[digest/wiki/光伏.md]] -[[digest/wiki/光伏.md#产业链]] -[[digest/wiki/光伏.md|光伏]] -![[resource/2026-06-01/report.md]] +[[daily/2026-06-20/session.md]] +[[notes/example.md#L9]] +[[notes/example.md#L9-L10]] +[[notes/example.md#L9-L10,L15-L20]] ``` ReMe 的 wikilink 是**字面路径语义**: @@ -187,54 +188,57 @@ ReMe 的 wikilink 是**字面路径语义**: 它不会自动补 `.md`,不会按文件名搜索,也不会自动解析 folder note。推荐写完整的 workspace 相对路径,并带上扩展名。 +`[label](../wiki/example.md)` 这类普通 Markdown 链接不会建立 `FileLink`,move 或 retarget 操作也不会改写它们。 + +`#L9`、`#L9-L10` 和 `#L9-L10,L15-L20` 这类锚点会作为普通 `target_anchor` 字符串保存在图谱中。图谱解析器 +不会校验行号锚点,因此 `#L0`、`#L10-L9`、`#L9,` 也会被保存。`read` 不会解析追加在 `path` 后的锚点;读取指定 +范围时需要分别传入从 1 开始、首尾均包含的 `start_line` 和 `end_line`,例如 +`read(path="digest/wiki/光伏.md", start_line=9, end_line=10)`。 + Wikilink 的作用: ```text 正文链接 -> 建立 FileLink -predicate:: 链接 -> 建立带关系名的 FileLink move 文件 -> 默认改写入边中的 [[旧路径]] delete 文件 -> 返回仍存在的入边,提示清理引用 search 命中 -> 可展开出入链,帮助理解上下文 ``` -支持的关系写法: - -```markdown -industry:: [[digest/wiki/新能源.md]] -[competitor:: [[digest/wiki/比亚迪.md]]] -``` - 解析结果: ```text FileLink source_path = 当前文件 - target_path = digest/wiki/新能源.md - predicate = industry + target_path = notes/example.md + target_anchor = L9-L10,L15-L20 ``` +旧文档中的 `related:: [[path]]`、`- related:: [[path]]` 或 +`[related:: [[path]]]` 仍然可以读取。ReMe 会忽略外围文本,把内部 `[[path]]` +作为普通链接建立索引。从曾存储 typed link 的版本升级后,应执行一次 `reme reindex`, +用源文件重建不含旧关系字段的派生图索引。 + ### 来源和关系 ReMe 里最重要的两类链接是来源链接和概念关系链接。 -来源链接说明“这条长期记忆从哪里来”: +Sources 章节说明“这条长期记忆从哪里来”: ```markdown -derived_from:: [[daily/2026-06-20/session-a.md]] -derived_from:: [[resource/2026-06-20/report.pdf]] +## Sources + +- [[daily/2026-06-20/session-a.md]] +- [[resource/2026-06-20/report.pdf]] ``` -概念关系链接说明“这个节点和哪些长期记忆有关”: +概念关系链接说明“这个节点和哪些长期记忆有关”,并自然织入正文: ```markdown -related:: [[digest/wiki/光伏产业链.md]] -depends_on:: [[digest/procedure/调研报告拆解流程.md]] -contrasts_with:: [[digest/wiki/集中式逆变器.md]] +这份分析扩展了 [[digest/wiki/光伏产业链.md]],遵循 +[[digest/procedure/调研报告拆解流程.md]],并与 +[[digest/wiki/集中式逆变器.md]] 对比。 ``` -普通正文 wikilink 也会建立图边,但当关系本身有语义价值时,推荐使用 `predicate:: [[path]]`。这能让搜索、图遍历和后续 Agent -整合更容易理解链接含义。 - ## 人工编辑和 Agent 编辑 因为记忆就是文件,用户可以直接在编辑器里改 workspace;Agent 也可以通过 ReMe 的文件工具读写同一批文件。两者遵守同一套约定: @@ -247,8 +251,8 @@ contrasts_with:: [[digest/wiki/集中式逆变器.md]] | 删除文件 | 删除前检查入链;ReMe 的 delete 会返回仍然指向目标的来源文件,方便清理悬空引用。 | | 修改元数据 | 用 frontmatter 表达短字段;正文发生实质变化时同步更新 `description`。 | -一个实用规则是:**可以让 Agent 重写表达,但不要让它丢掉证据边**。尤其是 digest 节点中的 `derived_from:: [[...]]` 和已有 -digest-to-digest wikilink,是长期记忆可追溯和可扩展的基础。 +一个实用规则是:**可以让 Agent 重写表达,但不要让它丢掉证据边**。尤其是 digest 节点中的 Sources 条目和已有 +digest-to-digest Wikilink,是长期记忆可追溯和可扩展的基础。 ## 路径语义 diff --git a/docs/zh/memory_search.md b/docs/zh/memory_search.md index 483bf0d4..e97c42d4 100644 --- a/docs/zh/memory_search.md +++ b/docs/zh/memory_search.md @@ -62,7 +62,7 @@ index_update_loop: 4. 对删除的文件,从 `file_store`、`keyword_index` 和 `file_graph` 清掉对应记录。 5. 有变化时 dump 到 `metadata/`,让下次启动可以恢复。 -Markdown chunker 会解析 YAML frontmatter、标题结构和 `[[...]]`,产出 `FileNode`、`FileChunk` 和 `FileLink`。更细的分块规则见 +Markdown chunker 会解析 YAML frontmatter、标题结构和 wikilink,产出 `FileNode`、`FileChunk` 和 `FileLink`。更细的分块规则见 [Memory as File](./memory_as_file.md#memory-chunking)。 ### 索引优化 @@ -172,7 +172,8 @@ delete,后续可通过 optimize 压缩索引。 Memory Search 的“渐进式”不是一次把全库内容塞进结果,而是分三层展开: 1. 第一层是 chunk 召回:只返回最相关的 `limit` 个文本片段。 -2. 第二层是文件定位:每个结果带 `path:start_line-end_line`,可以继续用 `read` 精读原文件。 +2. 第二层是文件定位:每个结果带 `path:start_line-end_line`。调用 `read` 时需要把它们分别作为 `path`、 + `start_line` 和 `end_line` 传入,行号范围不是 `path` 的一部分。 3. 第三层是链接邻居:对命中文件调用 `expand_links()`,展开最多 `max_links_per_direction` 个 outlinks 和 inlinks。 展开的数据来自 `file_graph`,不是重新扫文件: @@ -183,7 +184,7 @@ Memory Search 的“渐进式”不是一次把全库内容塞进结果,而是 -> file_store.get_outlinks(path) -> file_store.get_inlinks(path) -> file_store.get_nodes(neighbor_paths) - -> 渲染邻居的 path、name、description、predicate、anchor + -> 渲染邻居的 path、name、description、anchor ``` 这让搜索结果既保持短,又能看到“这条记忆连接到哪些长期节点、资源或其他 daily note”。如果某条结果值得继续追,可以用 @@ -203,10 +204,8 @@ Memory Search 的“渐进式”不是一次把全库内容塞进结果,而是 ...命中的记忆片段... outlinks (2): -> digest/indexing.md name="Indexing" description="..." - via predicate=related inlinks (1): <- daily/2026-06-19.md name="..." - via plain ``` `counts` 会告诉你本次向量、关键词各召回了多少候选,以及最终返回多少条。默认 embedding 关闭时,`vector` 通常是 `0`,`hybrid` 是 diff --git a/docs/zh/reme_scene.md b/docs/zh/reme_scene.md index a658f586..a2f203f5 100644 --- a/docs/zh/reme_scene.md +++ b/docs/zh/reme_scene.md @@ -107,15 +107,17 @@ name: 钴 description: 锂电正极材料关键原料,主产区集中于刚果(金) --- -downstream_product:: [[digest/wiki/三元正极.md]] -producer:: [[digest/wiki/嘉能可.md]] -source_event:: [[daily/2026-05-18/2026-05-18-close.md]] - # 钴 +用于 [[digest/wiki/三元正极.md]];主要生产商包括 [[digest/wiki/嘉能可.md]]。 + ## 供给端 嘉能可三季度钴产量同比下滑 18%,需要继续跟踪供给收缩对价格的影响。 +## Sources + +- [[daily/2026-05-18/2026-05-18-close.md]] + ## 政策风险 刚果(金)矿权政策变化可能影响 KFM 矿运营,需联动跟踪洛阳钼业。 ``` @@ -179,10 +181,10 @@ reme search query="锂电 上下游 三元 正极 钴 宁德" limit=5 嘉能可三季度钴产量同比下滑 18%... outlinks: - -> digest/wiki/三元正极.md name="三元正极" via predicate=downstream_product - -> digest/wiki/嘉能可.md name="嘉能可" via predicate=producer + -> digest/wiki/三元正极.md name="三元正极" + -> digest/wiki/嘉能可.md name="嘉能可" inlinks: - <- digest/wiki/三元正极.md name="三元正极" via predicate=upstream_material + <- digest/wiki/三元正极.md name="三元正极" ========== digest/wiki/三元正极.md:5-18 [score=0.0139 keyword=3.2017] ========== ... @@ -291,11 +293,10 @@ name: TypeScript 项目构建 OOM 排查路径 description: build 卡住且内存上涨时,优先检查类型检查进程内存 --- -source_event:: [[daily/2026-03-10/build-oom-2026-03-10.md]] -related_preference:: [[digest/personal/code-style.md]] - # TypeScript 项目构建 OOM 排查路径 +执行这个 runbook 时遵循 [[digest/personal/code-style.md]]。 + ## 症状 构建卡在后段,CPU 不高但内存持续上涨。 @@ -307,6 +308,10 @@ related_preference:: [[digest/personal/code-style.md]] ## 已知无效路径 - 单纯删除 `.cache` 未解决 2026-03-10 的问题。 - 升级 terser 插件未解决 2026-03-10 的问题。 + +## Sources + +- [[daily/2026-03-10/build-oom-2026-03-10.md]] ``` 示例 `digest/personal/code-style.md`: @@ -398,13 +403,14 @@ name: Alice description: 用户朋友,常推荐阅读材料 --- -recommended_book:: [[digest/wiki/deep-work.md]] -source_event:: [[daily/2026-04-20/lunch-with-alice.md]] - # Alice ## 阅读推荐 -2026-04-20 午餐时推荐过一本关于注意力和深度工作的书。 +2026-04-20 午餐时推荐过 [[digest/wiki/deep-work.md]],这是一本关于注意力和深度工作的书。 + +## Sources + +- [[daily/2026-04-20/lunch-with-alice.md]] ``` ### 一次联想式回忆 @@ -426,7 +432,7 @@ reme search query="Alice 推荐 注意力 书 深度" limit=5 ```text digest/personal/alice.md outlinks: - -> digest/wiki/deep-work.md via predicate=recommended_book + -> digest/wiki/deep-work.md daily/2026-04-20/lunch-with-alice.md ``` diff --git a/reme/components/base_component.py b/reme/components/base_component.py index 2b55329a..dcb67b19 100644 --- a/reme/components/base_component.py +++ b/reme/components/base_component.py @@ -42,12 +42,12 @@ class ComponentMixin: return Path(self.app_context.app_config.workspace_dir).absolute() def to_workspace_relative(self, path: str | Path) -> str: - """Convert `path` to a workspace-relative string; return absolute path when outside.""" + """Convert ``path`` to a POSIX workspace path; keep outside paths absolute.""" abs_path = Path(path).absolute() try: - return str(abs_path.relative_to(self.workspace_path)) + return abs_path.relative_to(self.workspace_path).as_posix() except ValueError: - return str(abs_path) + return abs_path.as_posix() class Dependency: diff --git a/reme/components/file_chunker/default_file_chunker.py b/reme/components/file_chunker/default_file_chunker.py index a2c57f3c..23457f5b 100644 --- a/reme/components/file_chunker/default_file_chunker.py +++ b/reme/components/file_chunker/default_file_chunker.py @@ -73,7 +73,7 @@ class DefaultFileChunker(BaseFileChunker): ) def _link_byte_spans(self, content: str) -> list[tuple[int, int]]: - """Return [start, end) byte spans of every wikilink in content.""" + """Return byte spans of wikilinks.""" spans: list[tuple[int, int]] = [] last_char, last_byte = 0, 0 for wm in WikilinkHandler.iter_matches(content): diff --git a/reme/components/file_chunker/markdown_file_chunker.py b/reme/components/file_chunker/markdown_file_chunker.py index 1365020c..7da94ae6 100644 --- a/reme/components/file_chunker/markdown_file_chunker.py +++ b/reme/components/file_chunker/markdown_file_chunker.py @@ -12,8 +12,7 @@ and merge adjacent small subtrees at their parent. Leaf blocks (table / code / list / paragraph) split on internal boundaries and each piece is annotated ``[Part X/N]``. Wikilink extraction is delegated to :class:`reme.utils.wikilink_handler.WikilinkHandler` — -the single source of truth for ``[[...]]`` syntax (including -Dataview-style typed predicates). +the single source of truth for ``[[...]]`` syntax. """ import re diff --git a/reme/components/file_graph/neo4j_file_graph.py b/reme/components/file_graph/neo4j_file_graph.py index 4ec0cac1..332f198c 100644 --- a/reme/components/file_graph/neo4j_file_graph.py +++ b/reme/components/file_graph/neo4j_file_graph.py @@ -7,7 +7,7 @@ Property-graph mapping: Virtual node: (:File {path}) — placeholder created when something links to a path that hasn't been upserted yet. - Edge: (:File)-[:LINKS {idx, anchor, predicate}]->(:File) + Edge: (:File)-[:LINKS {idx, anchor}]->(:File) The ``links_json`` property doubles as the "is real" marker — its presence means the node was upserted with a payload; its absence @@ -43,7 +43,7 @@ from ...schema import FileLink, FileNode from ...schema.file_node import FileFrontMatter _TYPED_FRONTMATTER_FIELDS = {"name", "description"} -_LINK_FIELDS = {"source_path", "target_path", "target_anchor", "predicate"} +_LINK_FIELDS = {"source_path", "target_path", "target_anchor"} # Properties that distinguish a "real" node from a virtual placeholder. # Listed for the demote query (delete_nodes) so we can REMOVE them all. @@ -169,7 +169,6 @@ class Neo4jFileGraph(BaseFileGraph): { "idx": i, "anchor": link.target_anchor, - "predicate": link.predicate, "target": link.target_path, } for i, link in enumerate(node.links) @@ -211,7 +210,7 @@ class Neo4jFileGraph(BaseFileGraph): UNWIND n.links AS link MERGE (t:File {path: link.target}) MERGE (s)-[r:LINKS {idx: link.idx}]->(t) - SET r.anchor = link.anchor, r.predicate = link.predicate + SET r.anchor = link.anchor """, items=payload, ) @@ -313,7 +312,6 @@ class Neo4jFileGraph(BaseFileGraph): { "idx": i, "anchor": link.get("target_anchor"), - "predicate": link.get("predicate"), "target": link.get("target_path"), } for i, link in enumerate(links) @@ -340,7 +338,7 @@ class Neo4jFileGraph(BaseFileGraph): UNWIND n.links AS link MERGE (t:File {path: link.target}) MERGE (s)-[r:LINKS {idx: link.idx}]->(t) - SET r.anchor = link.anchor, r.predicate = link.predicate + SET r.anchor = link.anchor """, items=payload, ) @@ -369,8 +367,7 @@ class Neo4jFileGraph(BaseFileGraph): WHERE s.links_json IS NOT NULL MATCH (s)-[r:LINKS]->(t:File) WHERE 1=1 {target_filter} - RETURN t.path AS target, r.anchor AS anchor, - r.predicate AS predicate, r.idx AS idx + RETURN t.path AS target, r.anchor AS anchor, r.idx AS idx ORDER BY r.idx ASC """, path=path, @@ -381,7 +378,6 @@ class Neo4jFileGraph(BaseFileGraph): source_path=path, target_path=row["target"], target_anchor=row.get("anchor"), - predicate=row.get("predicate"), ) for row in rows ] @@ -405,8 +401,7 @@ class Neo4jFileGraph(BaseFileGraph): WHERE 1=1 {target_filter} MATCH (s:File)-[r:LINKS]->(t) WHERE s.links_json IS NOT NULL - RETURN r.anchor AS anchor, r.predicate AS predicate, - r.idx AS idx, s.path AS source + RETURN r.anchor AS anchor, r.idx AS idx, s.path AS source ORDER BY s.path ASC, r.idx ASC """, path=path, @@ -417,7 +412,6 @@ class Neo4jFileGraph(BaseFileGraph): source_path=row["source"], target_path=path, target_anchor=row.get("anchor"), - predicate=row.get("predicate"), ) for row in rows ] diff --git a/reme/schema/file_link.py b/reme/schema/file_link.py index b029d5a9..72324d01 100644 --- a/reme/schema/file_link.py +++ b/reme/schema/file_link.py @@ -7,12 +7,14 @@ class FileLink(BaseModel): """file link [[target_path]] [[target_path#target_anchor]] - predicate:: [[target_*]] - [predicate:: [[target_*]]] """ model_config = ConfigDict(extra="forbid") source_path: str = Field(default=..., description="source file path relative to working dir") target_path: str = Field(default=..., description="target file path relative to working dir") - target_anchor: str | None = Field(default=None, description="Heading or block anchor (text after '#')") - predicate: str | None = Field(default=None, description="Dataview-style typed-link predicate") + target_anchor: str | None = Field(default=None, description="Heading, block, or line anchor (text after '#')") + predicate: str | None = Field( + default=None, + exclude=True, + description="Deprecated compatibility field; accepted when loading legacy indexes but otherwise unused", + ) diff --git a/reme/steps/evolve/dream/integrate.yaml b/reme/steps/evolve/dream/integrate.yaml index 0189923d..6b52cb46 100644 --- a/reme/steps/evolve/dream/integrate.yaml +++ b/reme/steps/evolve/dream/integrate.yaml @@ -18,8 +18,8 @@ integrate_system_prompt_procedure: | genuinely needs more. - If the draft starts copying paragraphs or narrating the event, it belongs in the source material, not digest. - - Provenance edges carry details: cite every relevant unit_paths entry with - `derived_from:: [[]]`. + - The Sources section carries details: cite every relevant unit_paths entry + as `- [[]]`. - Digest-to-digest wikilinks carry the conceptual graph. ## Procedure Body Shape @@ -30,9 +30,9 @@ integrate_system_prompt_procedure: | - Steps: numbered or terse bullets; each step is verb-led. - Pre-conditions / inputs: short list, not prose. - Failure modes / caveats: brief. - - `derived_from:: [[]]`: at least one, and normally every - relevant path in unit_paths. Plain-prose provenance does NOT count; only - wikilinks survive future updates. + - `## Sources`: at least one `- [[]]` item, and normally one + for every relevant path in unit_paths. Plain-prose provenance does NOT + count; only wikilinks survive future updates. ## Workflow @@ -50,7 +50,7 @@ integrate_system_prompt_procedure: | - unrelated: ignore. 3. Choose exactly one action: - CREATE: no same_abstraction hit; write `{digest_dir}/procedure/.md`. - - CORROBORATE: same procedure observed again; append derived_from and + - CORROBORATE: same procedure observed again; append its source link and optionally strengthen wording. - REFINE: new pre-condition, edge case, failure mode, scope, or step; expand the relevant span or slot the step into the right position. @@ -58,21 +58,18 @@ integrate_system_prompt_procedure: | tighten or annotate inline with `> note: contradicted by [[]] - `. 4. Weave related digest nodes into the body as wikilinks on both CREATE and UPDATE. UPDATE must be additive: never remove existing wikilinks or - derived_from entries. Default to weaving more, not less; this is the only + source links. Default to weaving more, not less; this is the only chance to attach recalled related nodes. ## Wikilink Graph - - Source provenance links point from digest back to material: - `derived_from:: [[daily//.md]]` or - `derived_from:: [[resource/]]`. + - A `## Sources` section points from digest back to material with bare list + items such as `- [[daily//.md]]` and + `- [[resource/]]`. - Procedure nodes may link to any digest bucket: `[[{digest_dir}/procedure/.md]]`, `[[{digest_dir}/personal/.md]]`, or `[[{digest_dir}/wiki/.md]]`. - - Useful predicates include `derived_from::`, `relates_to::`, - `depends_on::`, and `blocks_on::`; predicates are open and live outside - the brackets. - Wikilinks must be full workspace-relative paths with `.md`. - CREATE writes inside `{digest_dir}/procedure/`. UPDATE may target any bucket if recall legitimately found the same abstraction. @@ -101,7 +98,7 @@ integrate_system_prompt_procedure_zh: | - 正文短且抽象,通常 50-200 words;只有流程本身确实需要时才更长。 - 如果草稿开始复制段落或叙述事件,说明细节放错层了。 - - Provenance edge 承载细节:用 `derived_from:: [[]]` 引用 unit_paths 中每个相关来源。 + - Sources 章节承载细节:用 `- [[]]` 引用 unit_paths 中每个相关来源。 - Digest 之间的 wikilink 承载概念图。 ## Procedure 正文形态 @@ -112,7 +109,7 @@ integrate_system_prompt_procedure_zh: | - Steps:编号或短 bullet;每步以动词开头。 - Pre-conditions / inputs:短列表,不写长 prose。 - Failure modes / caveats:简短。 - - `derived_from:: [[]]`:至少一条,通常覆盖 unit_paths 中每个相关 path。 + - `## Sources`:至少一条 `- [[]]`,通常覆盖 unit_paths 中每个相关 path。 纯文本 provenance 不算;只有 wikilink 会在未来更新中保留下来。 ## 工作流 @@ -126,22 +123,20 @@ integrate_system_prompt_procedure_zh: | - unrelated:忽略。 3. 选择且只选择一个 action: - CREATE:没有 same_abstraction 命中;写入 `{digest_dir}/procedure/.md`。 - - CORROBORATE:同一流程再次出现;追加 derived_from,可选强化措辞。 + - CORROBORATE:同一流程再次出现;追加来源链接,可选强化措辞。 - REFINE:新增前置条件、边界情况、失败模式、适用范围或步骤;扩展相关段落或把步骤插到正确位置。 - CORRECT:顺序错误、缺关键步骤、结果不好或有冲突;收紧表述,或用 `> note: contradicted by [[]] - ` 内联标注。 4. CREATE 和 UPDATE 都要把 related digest 节点织入正文 wikilink。UPDATE 必须只增不删:不要删除已有 - wikilink 或 derived_from。默认多织,而不是少织;这是挂接召回到的相关节点的唯一机会。 + wikilink 或来源链接。默认多织,而不是少织;这是挂接召回到的相关节点的唯一机会。 ## Wikilink 图 - - 来源 provenance 从 digest 指回材料: - `derived_from:: [[daily//.md]]` 或 `derived_from:: [[resource/]]`。 + - `## Sources` 章节用普通列表 Wikilink 从 digest 指回材料,例如 + `- [[daily//.md]]` 和 `- [[resource/]]`。 - Procedure 节点可以链接任意 digest bucket: `[[{digest_dir}/procedure/.md]]`、`[[{digest_dir}/personal/.md]]`、 `[[{digest_dir}/wiki/.md]]`。 - - 常用 predicate:`derived_from::`、`relates_to::`、`depends_on::`、`blocks_on::`; - predicate 词表开放,写在方括号外。 - Wikilink 必须是带 `.md` 的 workspace-relative path。 - CREATE 写入 `{digest_dir}/procedure/`。如果 recall 合法命中同一抽象,UPDATE 可以跨 bucket。 - `edit` 只改正文,并尽量只增不删:不要删除 old span 中已有的 wikilink。 @@ -171,8 +166,8 @@ integrate_system_prompt_personal: | - Body is short and operational, usually 50-200 words. - Do not narrate what the user said in detail; cite source material instead. - - Provenance edges carry details: cite every relevant unit_paths entry with - `derived_from:: [[]]`. + - The Sources section carries details: cite every relevant unit_paths entry + as `- [[]]`. - Digest-to-digest wikilinks carry the conceptual graph. ## Personal Body Shape @@ -185,9 +180,9 @@ integrate_system_prompt_personal: | - `How to apply:` contexts, tasks, boundaries, or exceptions. - Do not invent exceptions or soften a hard preference unless the source material explicitly supports that exception. - - `derived_from:: [[]]`: at least one, and normally every - relevant path in unit_paths. Plain-prose provenance does NOT count; only - wikilinks survive future updates. + - `## Sources`: at least one `- [[]]` item, and normally one + for every relevant path in unit_paths. Plain-prose provenance does NOT + count; only wikilinks survive future updates. For preferences, prefer one node per preference rather than one large person node; that is the granularity downstream search will hit. @@ -208,7 +203,7 @@ integrate_system_prompt_personal: | - unrelated: ignore. 3. Choose exactly one action: - CREATE: no same_abstraction hit; write `{digest_dir}/personal/.md`. - - CORROBORATE: the rule is reaffirmed; append derived_from and optionally + - CORROBORATE: the rule is reaffirmed; append its source link and optionally strengthen confidence. - REFINE: scope, condition, exception, or example changed; expand `How to apply:`. - CORRECT: the user/team changed their mind or evidence conflicts; tighten @@ -216,21 +211,18 @@ integrate_system_prompt_personal: | `> note: contradicted by [[]] - `. 4. Weave related digest nodes into the body as wikilinks on both CREATE and UPDATE. UPDATE must be additive: never remove existing wikilinks or - derived_from entries. Default to weaving more, not less; this is the only + source links. Default to weaving more, not less; this is the only chance to attach recalled related nodes. ## Wikilink Graph - - Source provenance links point from digest back to material: - `derived_from:: [[daily//.md]]` or - `derived_from:: [[resource/]]`. + - A `## Sources` section points from digest back to material with bare list + items such as `- [[daily//.md]]` and + `- [[resource/]]`. - Personal nodes may link to any digest bucket: `[[{digest_dir}/personal/.md]]`, `[[{digest_dir}/procedure/.md]]`, or `[[{digest_dir}/wiki/.md]]`. - - Useful predicates include `derived_from::`, `applies_to::`, - `relates_to::`, `depends_on::`, and `contradicts::`; predicates are open - and live outside the brackets. - Wikilinks must be full workspace-relative paths with `.md`. - CREATE writes inside `{digest_dir}/personal/`. UPDATE may target any bucket if recall legitimately found the same abstraction. @@ -260,7 +252,7 @@ integrate_system_prompt_personal_zh: | - 正文短且可操作,通常 50-200 words。 - 不要详细复述用户说了什么;用来源材料承载细节。 - - Provenance edge 承载细节:用 `derived_from:: [[]]` 引用 unit_paths 中每个相关来源。 + - Sources 章节承载细节:用 `- [[]]` 引用 unit_paths 中每个相关来源。 - Digest 之间的 wikilink 承载概念图。 ## Personal 正文形态 @@ -271,7 +263,7 @@ integrate_system_prompt_personal_zh: | - `Why:` 原因或上下文,帮助未来判断边界情况。 - `How to apply:` 适用上下文、任务、边界或例外。 - 不要凭空添加例外,也不要软化明确偏好;只有来源材料明确支持时才写例外。 - - `derived_from:: [[]]`:至少一条,通常覆盖 unit_paths 中每个相关 path。 + - `## Sources`:至少一条 `- [[]]`,通常覆盖 unit_paths 中每个相关 path。 纯文本 provenance 不算;只有 wikilink 会在未来更新中保留下来。 偏好类内容优先一条偏好一个 node,而不是一个人一个大 node;这是下游搜索更容易命中的粒度。 @@ -287,22 +279,20 @@ integrate_system_prompt_personal_zh: | - unrelated:忽略。 3. 选择且只选择一个 action: - CREATE:没有 same_abstraction 命中;写入 `{digest_dir}/personal/.md`。 - - CORROBORATE:规则被再次确认;追加 derived_from,可选强化置信度。 + - CORROBORATE:规则被再次确认;追加来源链接,可选强化置信度。 - REFINE:scope、条件、例外或例子变化;扩展 `How to apply:`。 - CORRECT:用户/团队改变主意或证据冲突;收紧到新旧证据都支持的表述,或用 `> note: contradicted by [[]] - ` 内联标注。 4. CREATE 和 UPDATE 都要把 related digest 节点织入正文 wikilink。UPDATE 必须只增不删:不要删除已有 - wikilink 或 derived_from。默认多织,而不是少织;这是挂接召回到的相关节点的唯一机会。 + wikilink 或来源链接。默认多织,而不是少织;这是挂接召回到的相关节点的唯一机会。 ## Wikilink 图 - - 来源 provenance 从 digest 指回材料: - `derived_from:: [[daily//.md]]` 或 `derived_from:: [[resource/]]`。 + - `## Sources` 章节用普通列表 Wikilink 从 digest 指回材料,例如 + `- [[daily//.md]]` 和 `- [[resource/]]`。 - Personal 节点可以链接任意 digest bucket: `[[{digest_dir}/personal/.md]]`、`[[{digest_dir}/procedure/.md]]`、 `[[{digest_dir}/wiki/.md]]`。 - - 常用 predicate:`derived_from::`、`applies_to::`、`relates_to::`、`depends_on::`、 - `contradicts::`;predicate 词表开放,写在方括号外。 - Wikilink 必须是带 `.md` 的 workspace-relative path。 - CREATE 写入 `{digest_dir}/personal/`。如果 recall 合法命中同一抽象,UPDATE 可以跨 bucket。 - `edit` 只改正文,并尽量只增不删:不要删除 old span 中已有的 wikilink。 @@ -335,8 +325,8 @@ integrate_system_prompt_wiki: | genuinely needs more. - If the draft starts copying paragraphs or narrating the event, it belongs in the source material, not digest. - - Provenance edges carry details: cite every relevant unit_paths entry with - `derived_from:: [[]]`. + - The Sources section carries details: cite every relevant unit_paths entry + as `- [[]]`. - Digest-to-digest wikilinks carry the conceptual graph. ## Wiki Body Shape @@ -346,11 +336,10 @@ integrate_system_prompt_wiki: | - First line: one-sentence definition or claim. - Body: short paragraphs or tight bullets with properties, sub-claims, distinctions, and one-line examples. - - Relations: typed wikilinks where the relation has semantic weight; most - cross-node links can stay bare. - - `derived_from:: [[]]`: at least one, and normally every - relevant path in unit_paths. Plain-prose provenance does NOT count; only - wikilinks survive future updates. + - Relations: weave related nodes into natural prose with ordinary wikilinks. + - `## Sources`: at least one `- [[]]` item, and normally one + for every relevant path in unit_paths. Plain-prose provenance does NOT + count; only wikilinks survive future updates. ## Workflow @@ -368,8 +357,8 @@ integrate_system_prompt_wiki: | - unrelated: ignore. 3. Choose exactly one action: - CREATE: no same_abstraction hit; write `{digest_dir}/wiki/.md`. - - CORROBORATE: the principle is reaffirmed by a new instance; append - derived_from and optionally strengthen wording. + - CORROBORATE: the principle is reaffirmed by a new instance; append its + source link and optionally strengthen wording. - REFINE: nuance, scope, edge case, or framing changes; tighten the relevant span. Body grows in precision, not detail volume. - CORRECT: factual contradiction or overstatement; tighten to the narrower @@ -377,22 +366,18 @@ integrate_system_prompt_wiki: | `> note: contradicted by [[]] - `. 4. Weave related digest nodes into the body as wikilinks on both CREATE and UPDATE. UPDATE must be additive: never remove existing wikilinks or - derived_from entries. Default to weaving more, not less; this is the only + source links. Default to weaving more, not less; this is the only chance to attach recalled related nodes. ## Wikilink Graph - - Source provenance links point from digest back to material: - `derived_from:: [[daily//.md]]` or - `derived_from:: [[resource/]]`. + - A `## Sources` section points from digest back to material with bare list + items such as `- [[daily//.md]]` and + `- [[resource/]]`. - Wiki nodes may link to any digest bucket: `[[{digest_dir}/wiki/.md]]`, `[[{digest_dir}/procedure/.md]]`, or `[[{digest_dir}/personal/.md]]`. - - Useful predicates include `derived_from::`, `is_a::`, `extends::`, - `depends_on::`, `relates_to::`, and `contradicts::`; predicates are open - and live outside the brackets. Most cross-node links can stay bare unless - the relation has clear semantic weight. - Wikilinks must be full workspace-relative paths with `.md`. - CREATE writes inside `{digest_dir}/wiki/`. UPDATE may target any bucket if recall legitimately found the same abstraction. @@ -422,7 +407,7 @@ integrate_system_prompt_wiki_zh: | - 正文短且抽象,通常 50-200 words;只有概念本身确实需要时才更长。 - 如果草稿开始复制段落或叙述事件,说明细节放错层了。 - - Provenance edge 承载细节:用 `derived_from:: [[]]` 引用 unit_paths 中每个相关来源。 + - Sources 章节承载细节:用 `- [[]]` 引用 unit_paths 中每个相关来源。 - Digest 之间的 wikilink 承载概念图。 ## Wiki 正文形态 @@ -431,8 +416,8 @@ integrate_system_prompt_wiki_zh: | - First line:一句话定义或主张。 - Body:短段落或紧凑 bullets,写属性、子主张、区分和一行例子。 - - Relations:有明确语义重量时使用 typed wikilink;大多数 cross-node link 可以裸写。 - - `derived_from:: [[]]`:至少一条,通常覆盖 unit_paths 中每个相关 path。 + - Relations:在自然语言中用普通 Wikilink 织入相关节点。 + - `## Sources`:至少一条 `- [[]]`,通常覆盖 unit_paths 中每个相关 path。 纯文本 provenance 不算;只有 wikilink 会在未来更新中保留下来。 ## 工作流 @@ -446,23 +431,20 @@ integrate_system_prompt_wiki_zh: | - unrelated:忽略。 3. 选择且只选择一个 action: - CREATE:没有 same_abstraction 命中;写入 `{digest_dir}/wiki/.md`。 - - CORROBORATE:原则被新实例再次确认;追加 derived_from,可选强化措辞。 + - CORROBORATE:原则被新实例再次确认;追加来源链接,可选强化措辞。 - REFINE:nuance、scope、edge case 或 framing 改变;收紧相关段落。正文增长的是精度,不是细节量。 - CORRECT:事实冲突或过度概括;收紧到更窄且有支持的表述,或用 `> note: contradicted by [[]] - ` 内联标注。 4. CREATE 和 UPDATE 都要把 related digest 节点织入正文 wikilink。UPDATE 必须只增不删:不要删除已有 - wikilink 或 derived_from。默认多织,而不是少织;这是挂接召回到的相关节点的唯一机会。 + wikilink 或来源链接。默认多织,而不是少织;这是挂接召回到的相关节点的唯一机会。 ## Wikilink 图 - - 来源 provenance 从 digest 指回材料: - `derived_from:: [[daily//.md]]` 或 `derived_from:: [[resource/]]`。 + - `## Sources` 章节用普通列表 Wikilink 从 digest 指回材料,例如 + `- [[daily//.md]]` 和 `- [[resource/]]`。 - Wiki 节点可以链接任意 digest bucket: `[[{digest_dir}/wiki/.md]]`、`[[{digest_dir}/procedure/.md]]`、 `[[{digest_dir}/personal/.md]]`。 - - 常用 predicate:`derived_from::`、`is_a::`、`extends::`、`depends_on::`、 - `relates_to::`、`contradicts::`;predicate 词表开放,写在方括号外。有明确语义重量时才加 - predicate,大多数 cross-node link 可以裸写。 - Wikilink 必须是带 `.md` 的 workspace-relative path。 - CREATE 写入 `{digest_dir}/wiki/`。如果 recall 合法命中同一抽象,UPDATE 可以跨 bucket。 - `edit` 只改正文,并尽量只增不删:不要删除 old span 中已有的 wikilink。 @@ -489,8 +471,8 @@ integrate_user_message: | {material_blob} Integrate this single unit into digest memory. Cite every relevant unit_paths - entry with `derived_from:: [[]]`, recall related digest nodes, and weave - useful digest wikilinks into the target node. + entry as `- [[]]` under `## Sources`, recall related digest nodes, and + weave useful digest wikilinks into the target node. integrate_user_message_zh: | 提示:{hint} @@ -506,5 +488,5 @@ integrate_user_message_zh: | {material_blob} - 将这个单独的 unit 整合进 digest memory。用 `derived_from:: [[]]` 引用 unit_paths 中每个相关来源, - 召回相关 digest 节点,并把有用的 digest wikilink 织入目标节点。 + 将这个单独的 unit 整合进 digest memory。在 `## Sources` 下用 `- [[]]` 引用 unit_paths 中每个 + 相关来源,召回相关 digest 节点,并把有用的 digest wikilink 织入目标节点。 diff --git a/reme/steps/evolve/dream/utils.py b/reme/steps/evolve/dream/utils.py index b731ff18..b6e7edc5 100644 --- a/reme/steps/evolve/dream/utils.py +++ b/reme/steps/evolve/dream/utils.py @@ -71,10 +71,10 @@ def scan_day_files(workspace: Path, day: str, daily: str, interests_name: str = out: list[str] = [] day_index = workspace / daily / f"{day}.md" if day_index.is_file(): - out.append(str(day_index.relative_to(workspace))) + out.append(day_index.relative_to(workspace).as_posix()) daily_root = workspace / daily / day if daily_root.is_dir(): - out.extend(str(p.relative_to(workspace)) for p in sorted(daily_root.rglob("*.md")) if p.is_file()) + out.extend(p.relative_to(workspace).as_posix() for p in sorted(daily_root.rglob("*.md")) if p.is_file()) return [p for p in out if p != f"{daily}/{day}/{interests_name}"] diff --git a/reme/steps/file_io/move.py b/reme/steps/file_io/move.py index c9e4ab43..e1bad6ba 100644 --- a/reme/steps/file_io/move.py +++ b/reme/steps/file_io/move.py @@ -65,6 +65,7 @@ class MoveStep(BaseStep): ) self.context.response.metadata.update(payload) + # pylint: disable=too-many-return-statements async def _move(self, src_path: str, dst_path: str, overwrite: bool, retarget: bool) -> dict: workspace_dir = Path(self.file_store.workspace_path or ".").resolve() src_abs, src_err = resolve_path(workspace_dir, src_path) if src_path else (None, "src_path is required") @@ -79,7 +80,8 @@ class MoveStep(BaseStep): assert src_abs is not None and dst_abs is not None # narrowed by precheck dst_abs.parent.mkdir(parents=True, exist_ok=True) - # Step 1 — copy. Both files exist; inbound [[src_path]] still resolves. + # Step 1 — copy. Wikilinks use workspace-relative paths, so outgoing + # targets do not change when their containing document moves. shutil.copyfile(str(src_abs), str(dst_abs)) payload: dict = {"src_path": src_path, "dst_path": dst_path, "size": dst_abs.stat().st_size} @@ -91,10 +93,13 @@ class MoveStep(BaseStep): try: report = await WikilinkHandler.retarget_links(self.file_store, src=src_path, dst=dst_path) except Exception as exc: - payload["retarget"] = {"error": f"retarget raised: {exc!r}"} + error = f"retarget raised: {exc!r}" + payload["error"] = error + payload["retarget"] = {"error": error} payload["src_removed"] = False return payload if "error" in report: + payload["error"] = report["error"] payload["retarget"] = report payload["src_removed"] = False return payload diff --git a/reme/steps/file_io/read.py b/reme/steps/file_io/read.py index 56fcecbe..ce530a87 100644 --- a/reme/steps/file_io/read.py +++ b/reme/steps/file_io/read.py @@ -168,7 +168,7 @@ class ReadStep(BaseStep): """Append the rendered neighbor block + stash raw expansion in metadata.""" assert self.context is not None try: - rel_path = str(target.relative_to(self.workspace_path)) + rel_path = target.relative_to(self.workspace_path.resolve()).as_posix() except ValueError: self.logger.info(f"[{self.name}] skip neighbors: path outside workspace_path path={target}") return diff --git a/reme/steps/index/traverse.py b/reme/steps/index/traverse.py index 98665977..93d4134b 100644 --- a/reme/steps/index/traverse.py +++ b/reme/steps/index/traverse.py @@ -1,10 +1,10 @@ """BFS over wikilink edges from one or more seed files. One record per traversed *edge* (not per node): the same target can repeat -if reached via different predicates or paths. Each record carries the -predecessor plus the link's predicate/anchor so callers can reconstruct -the path. Adjacency is built once via a single ``file_store.get_nodes()`` -call — BFS then runs purely in memory with no per-frontier round-trips. +if reached via different anchors or paths. Each record carries the +predecessor and anchor so callers can reconstruct the path. Adjacency is +built once via a single ``file_store.get_nodes()`` call — BFS then runs +purely in memory with no per-frontier round-trips. """ from collections import deque @@ -46,7 +46,7 @@ def _bfs( outbound: Adjacency, inbound: Adjacency, ) -> list[dict]: - """In-memory BFS; emits one record per unique (src, dst, predicate) edge.""" + """In-memory BFS; emits one record per unique (src, dst, anchor) edge.""" sources: list[Adjacency] = [] if direction in _OUT: sources.append(outbound) @@ -63,7 +63,7 @@ def _bfs( continue for src in sources: for next_path, link in src.get(current, ()): - key = (current, next_path, link.predicate) + key = (current, next_path, link.target_anchor) if key in visited: continue visited.add(key) @@ -72,7 +72,6 @@ def _bfs( "path": next_path, "depth": depth + 1, "via": current, - "predicate": link.predicate, "anchor": link.target_anchor, }, ) @@ -119,8 +118,7 @@ class TraverseStep(BaseStep): target = r["path"] if r["anchor"]: target = f"{target}#{r['anchor']}" - predicate = r["predicate"] or "-" - lines.append(f"[depth={r['depth']}] {r['via']} --{predicate}--> {target}") + lines.append(f"[depth={r['depth']}] {r['via']} --> {target}") answer = "\n".join(lines) self.context.response.success = True diff --git a/reme/utils/__init__.py b/reme/utils/__init__.py index a41e66e9..c9b7aaf3 100644 --- a/reme/utils/__init__.py +++ b/reme/utils/__init__.py @@ -9,6 +9,7 @@ from .common_utils import ( ) from .env_utils import load_env, parse_env_file from .link_expansion import expand_links, render_expansion_lines +from .line_anchor import format_line_anchor, parse_line_anchor from .logger_utils import get_logger from .logo_utils import print_logo from .service_utils import find_reme, locate_reme, precheck_start, cli_find_reme, running_service_config @@ -33,6 +34,8 @@ __all__ = [ "parse_env_file", "expand_links", "render_expansion_lines", + "format_line_anchor", + "parse_line_anchor", "get_logger", "print_logo", "find_reme", diff --git a/reme/utils/line_anchor.py b/reme/utils/line_anchor.py new file mode 100644 index 00000000..414eaed1 --- /dev/null +++ b/reme/utils/line_anchor.py @@ -0,0 +1,43 @@ +"""Parse and format GitHub-style 1-based line-anchor strings.""" + +import re + +_LINE_ANCHOR_RE = re.compile(r"L[0-9]+(?:-L[0-9]+)?(?:,L[0-9]+(?:-L[0-9]+)?)*") + + +def parse_line_anchor(anchor: str | None) -> list[tuple[int, int]] | None: + """Return normalized inclusive ranges, or ``None`` for a non-line anchor. + + Supported forms are ``L9``, ``L9-L10`` and + ``L9-L10,L15-L20``. Overlapping and adjacent ranges are merged. + Anchors beginning with ``L`` are treated as line anchors and raise + ``ValueError`` when malformed, zero-based, or reversed. + """ + if not anchor or not re.match(r"L[0-9]", anchor): + return None + if not _LINE_ANCHOR_RE.fullmatch(anchor): + raise ValueError(f"invalid line anchor: #{anchor}") + + ranges: list[tuple[int, int]] = [] + for item in anchor.split(","): + start_text, separator, end_text = item.partition("-L") + start = int(start_text[1:]) + end = int(end_text) if separator else start + if start < 1 or end < 1: + raise ValueError("line numbers must be at least 1") + if start > end: + raise ValueError(f"line range start ({start}) exceeds end ({end})") + ranges.append((start, end)) + + merged: list[tuple[int, int]] = [] + for start, end in sorted(ranges): + if merged and start <= merged[-1][1] + 1: + merged[-1] = (merged[-1][0], max(merged[-1][1], end)) + else: + merged.append((start, end)) + return merged + + +def format_line_anchor(ranges: list[tuple[int, int]]) -> str: + """Render normalized ranges without the leading ``#``.""" + return ",".join(f"L{start}" if start == end else f"L{start}-L{end}" for start, end in ranges) diff --git a/reme/utils/link_expansion.py b/reme/utils/link_expansion.py index f3920c67..5ec759e2 100644 --- a/reme/utils/link_expansion.py +++ b/reme/utils/link_expansion.py @@ -8,7 +8,7 @@ Two-layer split so callers can pick what they need: * :func:`expand_links` — data layer. Returns a structured dict keyed by source path, each value carrying its outlinks / inlinks with - neighbor meta and per-edge predicate/anchor. + neighbor metadata and link anchors. * :func:`render_expansion_lines` — view layer. Turns one path's expansion sub-dict into the same `` → path name=… description=…`` block ``SearchStep`` has historically printed. @@ -19,16 +19,16 @@ import asyncio from ..schema import FileLink, FileNode -def _group_by_neighbor(links: list[FileLink], key_attr: str) -> dict[str, list[dict]]: - """Group edges by neighbor path (insertion-ordered), each value a list of {predicate, anchor}.""" - out: dict[str, list[dict]] = {} +def _group_by_neighbor(links: list[FileLink], key_attr: str) -> dict[str, list[str]]: + """Group anchors by neighbor path while preserving insertion order.""" + out: dict[str, list[str]] = {} for lnk in links: neighbor = getattr(lnk, key_attr) if not neighbor: continue - out.setdefault(neighbor, []).append( - {"predicate": lnk.predicate, "anchor": lnk.target_anchor}, - ) + anchors = out.setdefault(neighbor, []) + if lnk.target_anchor: + anchors.append(lnk.target_anchor) return out @@ -55,16 +55,6 @@ def _format_meta_inline(meta: dict) -> str: return " ".join(parts) if parts else "(no meta)" -def _format_via(edge: dict) -> str: - """Render a single (predicate, anchor) edge as a 'via ...' descriptor.""" - bits = [] - if edge.get("predicate"): - bits.append(f"predicate={edge['predicate']}") - if edge.get("anchor"): - bits.append(f"anchor=#{edge['anchor']}") - return ", ".join(bits) if bits else "plain" - - async def expand_links( file_store, paths: list[str], @@ -73,7 +63,7 @@ async def expand_links( """Fetch out/in links for each path and attach neighbor meta. Returns ``{path: {"outlinks": [...], "inlinks": [...]}, ...}`` where - each list item is ``{"path": str, "meta": {...}, "edges": [{"predicate", "anchor"}, ...]}``. + each list item is ``{"path": str, "meta": {...}, "anchors": [str, ...]}``. Empty input returns ``{}``. ``max_per_direction`` caps the neighbor list per direction *before* meta lookup so we don't fetch nodes that won't be displayed. @@ -95,9 +85,10 @@ async def expand_links( nodes = await file_store.get_nodes(neighbor_paths) if neighbor_paths else [] meta_by_path = {n.path: _node_meta(n) for n in nodes} - def _attach(grouped: dict[str, list[dict]]) -> list[dict]: + def _attach(grouped: dict[str, list[str]]) -> list[dict]: return [ - {"path": npath, "meta": meta_by_path.get(npath, {}), "edges": edges} for npath, edges in grouped.items() + {"path": npath, "meta": meta_by_path.get(npath, {}), "anchors": anchors} + for npath, anchors in grouped.items() ] return {p: {"outlinks": _attach(og), "inlinks": _attach(ig)} for p, og, ig in zip(paths, out_grouped, in_grouped)} @@ -124,6 +115,6 @@ def render_expansion_lines(expansion: dict, indent: str = " ") -> list[str]: lines.append(f"{indent}{direction} ({len(items)}):") for item in items: lines.append(f"{inner}{arrow} {item['path']} {_format_meta_inline(item['meta'])}") - for edge in item["edges"]: - lines.append(f"{edge_indent}via {_format_via(edge)}") + for anchor in item["anchors"]: + lines.append(f"{edge_indent}via anchor=#{anchor}") return lines diff --git a/reme/utils/wikilink_handler.py b/reme/utils/wikilink_handler.py index 9714aa16..8c17a2e0 100644 --- a/reme/utils/wikilink_handler.py +++ b/reme/utils/wikilink_handler.py @@ -2,7 +2,7 @@ One class, :class:`WikilinkHandler`, owning every wikilink concern: -* **Pure text** — regex, Dataview predicate inference, validation: +* **Pure text** — regex, extraction, rewrite, validation: :meth:`~WikilinkHandler.extract_links` (used by :mod:`reme.components.file_chunker.markdown_file_chunker`), :meth:`~WikilinkHandler.scan_and_rewrite`, @@ -16,13 +16,12 @@ One class, :class:`WikilinkHandler`, owning every wikilink concern: post-rename to point inbound ``[[src]]`` at the new path). Source candidates come from the file_graph's reverse index — no fs scan. -Wikilink convention. Targets are taken **literally** — ``[[X]]`` → +Wikilink targets are taken **literally** — ``[[X]]`` → ``target="X"``, no implicit ``.md``, no short-form basename search, no folder-note expansion. Anchor and alias survive a rewrite -verbatim. Image marker (``!``) and Dataview predicate (``pred::`` -outside the brackets) sit outside ``[[...]]`` and are not touched by -a rewrite. Recommended form: full path relative to the workspace with -extension (``[[topics/x.md]]``). +verbatim. Text outside ``[[...]]`` is ignored. Recommended form: full +path relative to the workspace with extension (``[[topics/x.md]]``). +Ordinary Markdown links such as ``[label](path.md)`` are ignored. Stale graph entries are harmless (``scan_and_rewrite`` returns count=0 and the file is skipped), but a graph missing recent writes @@ -37,26 +36,23 @@ from ..enumeration import LinkScopeEnum from ..schema import FileLink +def _normalize_workspace_path(path: str) -> str: + """Use POSIX separators for workspace paths on every platform.""" + return path.replace("\\", "/") + + @dataclass(frozen=True) class WikilinkMatch: - """One ``[[...]]`` occurrence with parts surfaced. - - ``anchor`` / ``alias`` are stored **without** the leading ``#`` / - ``|`` so they map cleanly to :class:`FileLink.target_anchor`; the - rewrite path reads the raw regex groups (with delimiters) directly - and doesn't go through this dataclass. - """ + """The graph-relevant parts and source span of one wikilink.""" target: str anchor: str | None - alias: str | None - bang: bool start: int end: int class WikilinkHandler: - """Pure-text wikilink operations: parse, extract, rewrite, validate.""" + """Parse, extract, rewrite, and validate wikilinks.""" # Captures: optional image marker (``!``), the bare target, an # optional ``#anchor`` slice (with ``#``), and an optional ``|alias`` @@ -77,54 +73,39 @@ class WikilinkHandler: FORBIDDEN_IN_NEW = ("[", "]", "#", "|", "\n", "\r") - _DATAVIEW_LINE_RE = re.compile( - r"^[ \t]*(?:[-*+][ \t]+)?(?P[A-Za-z][A-Za-z0-9_]*)\s*::\s*(?P.+?)\s*$", - re.MULTILINE, - ) - - _INLINE_FIELD_OPEN_RE = re.compile(r"\[(?P[A-Za-z][A-Za-z0-9_]*)\s*::\s*") - # -- Low-level scan ------------------------------------------------ @classmethod def iter_matches(cls, text: str): - """Yield :class:`WikilinkMatch` for every ``[[...]]`` in ``text``. - - Skips matches whose target is empty after strip (defensive). - """ + """Yield every non-empty ``[[...]]`` occurrence in ``text``.""" for m in cls.WIKILINK_RE.finditer(text): target = m.group("target").strip() if not target: continue anchor_raw = m.group("anchor") - alias_raw = m.group("alias") yield WikilinkMatch( target=target, anchor=anchor_raw[1:].strip() if anchor_raw else None, - alias=alias_raw[1:].strip() if alias_raw else None, - bang=bool(m.group("bang")), start=m.start(), end=m.end(), ) - # -- FileLink extraction (with predicate inference) --------------- + # -- FileLink extraction ------------------------------------------ @classmethod def extract_links(cls, text: str, source_path: str) -> list[FileLink]: """Emit :class:`FileLink` edges for every wikilink in ``text``. - No resolution: ``target_path`` is the bracket contents verbatim. - Results are deduped by ``(target_path, predicate, target_anchor)`` - preserving order. + Targets remain literal. Results are deduped by + ``(target_path, target_anchor)`` while preserving order. """ if not text: return [] - inline_spans = cls._iter_inline_fields(text) + source_path = _normalize_workspace_path(source_path) out: list[FileLink] = [] seen: set[tuple] = set() for wm in cls.iter_matches(text): - predicate = cls._predicate_for(text, wm.start, inline_spans) - key = (wm.target, predicate, wm.anchor) + key = (wm.target, wm.anchor) if key in seen: continue seen.add(key) @@ -133,7 +114,6 @@ class WikilinkHandler: source_path=source_path, target_path=wm.target, target_anchor=wm.anchor, - predicate=predicate, ), ) return out @@ -147,20 +127,19 @@ class WikilinkHandler: old: str, new: str | None, ) -> tuple[str, int]: - """Find (and optionally rewrite) wikilinks whose target equals ``old``. + """Find and optionally rewrite wikilinks whose target is ``old``. Returns ``(new_text, count)``. When ``new`` is ``None`` no rewrite happens (the original text is returned), but the count is still - populated — used by ``find_inbound``. Matching is literal: - ``target == old``. No short-link, no implicit ``.md``, no - folder-note expansion. + populated — used by ``find_inbound``. Matching is literal. """ + old = _normalize_workspace_path(old) + new = _normalize_workspace_path(new) if new is not None else None count = 0 def sub(match: re.Match) -> str: nonlocal count - target = match.group("target").strip() - if target != old: + if match.group("target").strip() != old: return match.group(0) count += 1 if new is None: @@ -170,8 +149,7 @@ class WikilinkHandler: bang = match.group("bang") or "" return f"{bang}[[{new}{anchor}{alias}]]" - new_text = cls.WIKILINK_RE.sub(sub, text) - return new_text, count + return cls.WIKILINK_RE.sub(sub, text), count # -- Validation ---------------------------------------------------- @@ -201,53 +179,6 @@ class WikilinkHandler: prefix = scope.rstrip("/") + "/" return rel == scope or rel.startswith(prefix) - # -- Predicate helpers (internal) --------------------------------- - - @classmethod - def _iter_inline_fields(cls, text: str) -> list[tuple[int, int, str]]: - """Find inline-bracketed ``[predicate:: …]`` field spans by depth scan.""" - out: list[tuple[int, int, str]] = [] - for m in cls._INLINE_FIELD_OPEN_RE.finditer(text): - depth = 1 - i = m.end() - n = len(text) - while i < n: - c = text[i] - if c == "\n": - break - if c == "[": - depth += 1 - elif c == "]": - depth -= 1 - if depth == 0: - out.append((m.start(), i + 1, m.group("predicate"))) - break - i += 1 - return out - - @classmethod - def _predicate_for( - cls, - text: str, - pos: int, - inline_spans: list[tuple[int, int, str]], - ) -> str | None: - """Resolve the predicate governing a wikilink at offset ``pos``. - - Precedence: inline-bracketed > line-level Dataview > none. - """ - for field_start, field_end, predicate in inline_spans: - if field_start <= pos < field_end: - return predicate - line_start = text.rfind("\n", 0, pos) + 1 - line_end = text.find("\n", pos) - if line_end == -1: - line_end = len(text) - m = cls._DATAVIEW_LINE_RE.match(text[line_start:line_end]) - if m and line_start + m.start("value") <= pos: - return m.group("predicate") - return None - # -- Async file_graph-aware operations ----------------------------- @classmethod @@ -260,11 +191,12 @@ class WikilinkHandler: required to surface sources whose edges sit in the pending bucket. Each returned ``FileLink`` carries the linking node's ``source_path``; we dedupe to a sorted list since one source can host multiple edges - (different anchor/predicate) to the same target. Returns ``[]`` when + (different anchors) to the same target. Returns ``[]`` when there is no file_graph attached or no source references the target. """ if not file_store.file_graph: return [] + target = _normalize_workspace_path(target) inlinks = await file_store.file_graph.get_inlinks(target, scope=LinkScopeEnum.ALL) return sorted({link.source_path for link in inlinks if link.source_path}) @@ -272,12 +204,10 @@ class WikilinkHandler: async def find_inbound(cls, file_store, target: str, scope: str = "") -> dict: """Count wikilinks across the workspace that point at ``target``. - Literal matching: ``[[target]]`` only. The target file itself is - excluded — self-references don't survive a delete and aren't - actionable for the caller. Sources come from the file_graph's - reverse index; per-file counts come from reading each candidate - source (the graph dedupes by ``(target, predicate, anchor)`` so - it can't count repeated bare-wikilink occurrences directly). + The target file itself is excluded — self-references don't survive a + delete and aren't actionable for the caller. Sources come from the + file_graph's reverse index; per-file counts come from reading each + candidate source because the graph dedupes repeated edges. Result shape:: @@ -291,6 +221,8 @@ class WikilinkHandler: On bad inputs returns ``{"target": ..., "error": str}``. """ + target = _normalize_workspace_path(target) + scope = _normalize_workspace_path(scope) if not target: return {"target": target, "error": "target is required"} if Path(target).is_absolute(): @@ -340,6 +272,9 @@ class WikilinkHandler: matching only; candidate sources come from the file_graph's reverse index. """ + src = _normalize_workspace_path(src) + dst = _normalize_workspace_path(dst) + scope = _normalize_workspace_path(scope) err = cls.validate_src_dst(src, dst) if err is not None: return {"src": src, "dst": dst, "error": err} diff --git a/tests/integration/_workspace_fixture.py b/tests/integration/_workspace_fixture.py index 516610cc..b1d15641 100644 --- a/tests/integration/_workspace_fixture.py +++ b/tests/integration/_workspace_fixture.py @@ -84,7 +84,8 @@ token used to assert identity and claims between parties. ## Related Often issued by [[digest/wiki/oauth2.md]] flows. -derived_from:: [[daily/2026-05-15/auth-design/notes.md]] +## Sources +- [[daily/2026-05-15/auth-design/notes.md]] """, "digest/wiki/oauth2.md": """\ --- @@ -103,7 +104,8 @@ issued by an authorization server. - Client credentials - Refresh token -derived_from:: [[daily/2026-05-10/oauth-intro/notes.md]] +## Sources +- [[daily/2026-05-10/oauth-intro/notes.md]] """, "digest/procedure/key-rotation.md": """\ --- @@ -127,7 +129,8 @@ issuance. Default rotation cadence is **30 days**. Driven by historical practice; no formal compliance requirement has tightened this so far. -derived_from:: [[daily/2026-05-20/rotation-plan/notes.md]] +## Sources +- [[daily/2026-05-20/rotation-plan/notes.md]] """, "digest/personal/no-trailing-summary.md": """\ --- @@ -144,7 +147,8 @@ description: 不要在回复末尾加总结段落 **How to apply**: 任意编码 / 编辑任务回复结束时,直接停在最后一条 有信息量的话上,不要再补一段"以上就是本次的修改..."。 -derived_from:: [[daily/2026-05-01/style-feedback/notes.md]] +## Sources +- [[daily/2026-05-01/style-feedback/notes.md]] """, # ----- daily provenance stubs (so the digest links don't dangle) ----- "daily/2026-05-01/style-feedback/notes.md": """\ diff --git a/tests/integration/test_auto_dream.py b/tests/integration/test_auto_dream.py index 2e271c2a..e020cf18 100644 --- a/tests/integration/test_auto_dream.py +++ b/tests/integration/test_auto_dream.py @@ -182,15 +182,13 @@ def test_auto_dream_and_proactive(): if (env.workspace_dir / rel).is_file() } digest_wikilinks = [rel for rel, text in target_texts.items() if "[[digest/" in text] - provenance_links = [ - rel for rel, text in target_texts.items() if f"derived_from:: [[{DREAM_INPUT_PATH}]]" in text - ] + source_links = [rel for rel, text in target_texts.items() if f"- [[{DREAM_INPUT_PATH}]]" in text] print(f"[wikilink] integrated targets: {target_paths}") print(f"[wikilink] integrated targets with [[digest/...]] links: {digest_wikilinks}") - print(f"[wikilink] integrated targets with derived_from source links: {provenance_links}") + print(f"[wikilink] integrated targets with source links: {source_links}") assert target_texts, f"no integrated target files found: {target_paths}" - assert provenance_links, ( - "no derived_from wikilink back to the changed daily note in integrated targets\n" + assert source_links, ( + "no source wikilink back to the changed daily note in integrated targets\n" f"targets: {target_paths}" ) assert digest_wikilinks, ( diff --git a/tests/unit/test_auto_dream.py b/tests/unit/test_auto_dream.py index a6a15402..9577d999 100644 --- a/tests/unit/test_auto_dream.py +++ b/tests/unit/test_auto_dream.py @@ -11,7 +11,7 @@ from reme.components.application_context import ApplicationContext from reme.components.file_catalog import BaseFileCatalog from reme.components.file_store import BaseFileStore from reme.components.runtime_context import RuntimeContext -from reme.schema import DreamState +from reme.schema import DreamState, FileNode from reme.steps.evolve.dream.extract import DreamExtractStep from reme.steps.evolve.dream.finish import DreamFinishStep from reme.steps.evolve.dream.proactive import ProactiveStep @@ -94,6 +94,48 @@ def test_scan_day_files_includes_nested_md_and_excludes_interests(): ] +def test_dream_extract_matches_posix_catalog_paths(tmp_path): + """Unchanged nested files retain their POSIX catalog entries on every platform.""" + + class Catalog(_Catalog): + """Catalog seeded with POSIX paths and recording deletions.""" + + def __init__(self, nodes): + super().__init__() + self.nodes = nodes + self.deleted = [] + + async def delete(self, path): + self.deleted.extend(path if isinstance(path, list) else [path]) + + async def get_nodes(self, paths=None): + return self.nodes + + async def run(): + note = _touch(tmp_path / "daily" / "2026-05-28" / "nested" / "session.md") + rel_path = note.relative_to(tmp_path).as_posix() + catalog = Catalog([FileNode(path=rel_path, st_mtime=note.stat().st_mtime)]) + step = DreamExtractStep(scan_days=1, app_context=ApplicationContext(workspace_dir=str(tmp_path))) + + with patch("reme.steps.evolve.dream.extract.refresh_day_index", return_value={}): + response = await step( + RuntimeContext( + date="2026-05-28", + file_catalog=catalog, + file_store=_FileStore(tmp_path), + ), + ) + + dream = response.metadata["dream"] + assert response.success is True + assert dream["unchanged_paths"] == [rel_path] + assert dream["changed_paths"] == [] + assert dream["deleted_paths"] == [] + assert not catalog.deleted + + asyncio.run(run()) + + def test_recent_dates_includes_anchor_and_previous_days(): """Recent date window is inclusive and chronological.""" assert recent_dates("2026-05-28", 3) == ["2026-05-26", "2026-05-27", "2026-05-28"] @@ -152,7 +194,7 @@ def test_extract_without_llm_marks_changed_paths_failed(tmp_path): dream = response.metadata["dream"] assert response.success is False - assert str(note.relative_to(tmp_path)) in dream["changed_paths"] + assert note.relative_to(tmp_path).as_posix() in dream["changed_paths"] assert dream["failed_paths"] == dream["changed_paths"] asyncio.run(run()) @@ -299,9 +341,9 @@ def test_finish_does_not_checkpoint_failed_changed_paths(): dates=["2026-05-26", "2026-05-27", "2026-05-28"], workspace=str(workspace), daily_dir="daily", - changed_paths=[str(ok.relative_to(workspace)), str(failed.relative_to(workspace))], - failed_paths=[str(failed.relative_to(workspace))], - interests_paths=[str(interests.relative_to(workspace))], + changed_paths=[ok.relative_to(workspace).as_posix(), failed.relative_to(workspace).as_posix()], + failed_paths=[failed.relative_to(workspace).as_posix()], + interests_paths=[interests.relative_to(workspace).as_posix()], integrate_results=[ { "action": "CREATE", @@ -320,10 +362,10 @@ def test_finish_does_not_checkpoint_failed_changed_paths(): assert "topics:" not in resp.answer assert "Changes:" in resp.answer assert "- [digest/procedure/example.md][CREATE]: Created a concise procedure node." in resp.answer - assert str(ok.relative_to(workspace)) in upserted - assert str(failed.relative_to(workspace)) not in upserted - assert str(interests.relative_to(workspace)) in upserted - assert str(day_index.relative_to(workspace)) in upserted + assert ok.relative_to(workspace).as_posix() in upserted + assert failed.relative_to(workspace).as_posix() not in upserted + assert interests.relative_to(workspace).as_posix() in upserted + assert day_index.relative_to(workspace).as_posix() in upserted assert catalog.dumps == 1 asyncio.run(run()) diff --git a/tests/unit/test_base_component.py b/tests/unit/test_base_component.py index 6b9d994b..27ac4dd8 100644 --- a/tests/unit/test_base_component.py +++ b/tests/unit/test_base_component.py @@ -306,7 +306,7 @@ def test_to_workspace_relative_inside_workspace(): mixin = ComponentMixin() abs_path = mixin.workspace_path / "sub" / "file.md" rel = mixin.to_workspace_relative(abs_path) - assert rel == str(abs_path.relative_to(mixin.workspace_path)) + assert rel == abs_path.relative_to(mixin.workspace_path).as_posix() finally: os.chdir(old_cwd) diff --git a/tests/unit/test_common_steps.py b/tests/unit/test_common_steps.py index d6c4a13f..600ee2ca 100644 --- a/tests/unit/test_common_steps.py +++ b/tests/unit/test_common_steps.py @@ -42,12 +42,14 @@ def _run(coro): asyncio.run(coro) -def _node(path: str, links: list[tuple[str, str | None, str | None]] | None = None) -> FileNode: - """Build a FileNode with (target_path, target_anchor, predicate) outgoing edges.""" +def _node(path: str, links: list[tuple[str, str | None]] | None = None) -> FileNode: + """Build a FileNode with (target_path, target_anchor) outgoing edges.""" return FileNode( path=path, st_mtime=1.0, - links=[FileLink(source_path=path, target_path=t, target_anchor=a, predicate=p) for t, a, p in (links or [])], + links=[ + FileLink(source_path=path, target_path=target, target_anchor=anchor) for target, anchor in (links or []) + ], ) @@ -197,7 +199,7 @@ def test_traverse_forward_depth_1(): with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp): store = await _make_store( [ - _node("a.md", [("b.md", None, None), ("c.md", "intro", "ref")]), + _node("a.md", [("b.md", None), ("c.md", "intro")]), _node("b.md"), _node("c.md"), ], @@ -207,9 +209,8 @@ def test_traverse_forward_depth_1(): results = _edges(step) paths = {r["path"] for r in results} assert paths == {"b.md", "c.md"} - # The 'ref' edge should report its predicate/anchor. + # Anchors remain part of traversal metadata. c_edge = next(r for r in results if r["path"] == "c.md") - assert c_edge["predicate"] == "ref" assert c_edge["anchor"] == "intro" assert c_edge["via"] == "a.md" assert c_edge["depth"] == 1 @@ -226,8 +227,8 @@ def test_traverse_backward_returns_inlinks(): with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp): store = await _make_store( [ - _node("a.md", [("b.md", None, None)]), - _node("c.md", [("b.md", None, None)]), + _node("a.md", [("b.md", None)]), + _node("c.md", [("b.md", None)]), _node("b.md"), ], ) @@ -248,8 +249,8 @@ def test_traverse_depth_2_expands(): with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp): store = await _make_store( [ - _node("a.md", [("b.md", None, None)]), - _node("b.md", [("c.md", None, None)]), + _node("a.md", [("b.md", None)]), + _node("b.md", [("c.md", None)]), _node("c.md"), ], ) @@ -311,8 +312,8 @@ def test_traverse_both_directions(): with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp): store = await _make_store( [ - _node("upstream.md", [("center.md", None, None)]), - _node("center.md", [("downstream.md", None, None)]), + _node("upstream.md", [("center.md", None)]), + _node("center.md", [("downstream.md", None)]), _node("downstream.md"), ], ) diff --git a/tests/unit/test_default_file_chunker.py b/tests/unit/test_default_file_chunker.py index 1c300048..bf22d77e 100644 --- a/tests/unit/test_default_file_chunker.py +++ b/tests/unit/test_default_file_chunker.py @@ -105,7 +105,6 @@ def test_parse_links_bare(): assert link.source_path == "src.md" assert link.target_path == "note" assert link.target_anchor is None - assert link.predicate is None print("✓ test_parse_links_bare passed") @@ -115,7 +114,6 @@ def test_parse_links_with_anchor(): assert len(links) == 1 assert links[0].target_path == "note" assert links[0].target_anchor == "section A" - assert links[0].predicate is None print("✓ test_parse_links_with_anchor passed") @@ -137,47 +135,18 @@ def test_parse_links_anchor_and_alias(): print("✓ test_parse_links_anchor_and_alias passed") -def test_parse_links_predicate_simple(): - """Dataview inline: predicate:: [[target]].""" - links = WikilinkHandler.extract_links("author:: [[Alice]]", "src.md") - assert len(links) == 1 - assert links[0].predicate == "author" - assert links[0].target_path == "Alice" - assert links[0].target_anchor is None - print("✓ test_parse_links_predicate_simple passed") - - -def test_parse_links_predicate_bracketed(): - """Dataview inline-bracket: [predicate:: [[target]]].""" - links = WikilinkHandler.extract_links("text [author:: [[Alice]]] more", "src.md") - assert len(links) == 1 - assert links[0].predicate == "author" - assert links[0].target_path == "Alice" - print("✓ test_parse_links_predicate_bracketed passed") - - -def test_parse_links_predicate_bracketed_with_anchor(): - """[predicate:: [[target_path#target_anchor]]] — combined form.""" +def test_parse_links_ignores_legacy_relation_wrappers(): + """Legacy relation text remains compatible as ordinary wikilinks.""" links = WikilinkHandler.extract_links( - "[predicate:: [[target_path#target_anchor]]]", + "related:: [[a]]\n- related:: [[b]]\n[related:: [[c#section]]]", "src.md", ) - assert len(links) == 1 - link = links[0] - assert link.source_path == "src.md" - assert link.predicate == "predicate" - assert link.target_path == "target_path" - assert link.target_anchor == "target_anchor" - print("✓ test_parse_links_predicate_bracketed_with_anchor passed") - - -def test_parse_links_predicate_sticks_to_first(): - """Line-level predicate covers all wikilinks in its value portion.""" - links = WikilinkHandler.extract_links("pred:: [[a]] and bare [[b]]", "src.md") - assert len(links) == 2 - assert links[0].predicate == "pred" and links[0].target_path == "a" - assert links[1].predicate == "pred" and links[1].target_path == "b" - print("✓ test_parse_links_predicate_sticks_to_first passed") + assert [(link.target_path, link.target_anchor) for link in links] == [ + ("a", None), + ("b", None), + ("c", "section"), + ] + assert all(link.predicate is None for link in links) def test_parse_links_multiple_on_one_line(): @@ -190,6 +159,30 @@ def test_parse_links_multiple_on_one_line(): print("✓ test_parse_links_multiple_on_one_line passed") +def test_parse_wikilink_line_ranges(): + """Workspace paths and supported line-range forms create FileLink edges.""" + links = WikilinkHandler.extract_links( + "[[daily/2026-06-20/session.md]] [[notes/example.md#L9]] " + + "[[notes/example.md#L9-L10]] [[notes/example.md#L9-L10,L15-L20]]", + "src.md", + ) + assert [(link.target_path, link.target_anchor) for link in links] == [ + ("daily/2026-06-20/session.md", None), + ("notes/example.md", "L9"), + ("notes/example.md", "L9-L10"), + ("notes/example.md", "L9-L10,L15-L20"), + ] + + +def test_parse_markdown_links_are_ignored(): + """Ordinary Markdown links do not create FileLink edges.""" + links = WikilinkHandler.extract_links( + "[plain](../wiki/a.md) [section](../wiki/a.md#section) [ranges](../wiki/b.md#L9-L10,L15-L20)", + "daily/note.md", + ) + assert not links + + def test_parse_links_no_match(): """Strings without [[]] yield no links, even if '::' appears.""" assert len(WikilinkHandler.extract_links("no link here :: foo", "src.md")) == 0 @@ -198,15 +191,6 @@ def test_parse_links_no_match(): print("✓ test_parse_links_no_match passed") -def test_parse_links_predicate_with_underscore_and_digits(): - """Predicate identifier accepts letters, digits, underscore (no dash per Dataview spec).""" - links = WikilinkHandler.extract_links("see_also2:: [[target]]", "src.md") - assert len(links) == 1 - assert links[0].predicate == "see_also2" - assert links[0].target_path == "target" - print("✓ test_parse_links_predicate_with_underscore_and_digits passed") - - def test_parse_links_in_file(): """Integration: parse() populates FileNode.links from file content.""" @@ -227,11 +211,8 @@ def test_parse_links_in_file(): try: chunker = DefaultFileChunker() file_node, _ = await chunker.chunk(temp_path) - triples = {(link.predicate, link.target_path, link.target_anchor) for link in file_node.links} - assert (None, "alpha", None) in triples - assert (None, "beta", "h2") in triples - assert ("author", "Alice", None) in triples - assert ("ref", "paper", "chapter 1") in triples + pairs = {(link.target_path, link.target_anchor) for link in file_node.links} + assert pairs == {("alpha", None), ("beta", "h2"), ("Alice", None), ("paper", "chapter 1")} assert all(link.source_path == file_node.path for link in file_node.links) print("✓ test_parse_links_in_file passed") finally: @@ -384,13 +365,11 @@ if __name__ == "__main__": test_parse_links_with_anchor() test_parse_links_alias_dropped() test_parse_links_anchor_and_alias() - test_parse_links_predicate_simple() - test_parse_links_predicate_bracketed() - test_parse_links_predicate_bracketed_with_anchor() - test_parse_links_predicate_sticks_to_first() + test_parse_links_ignores_legacy_relation_wrappers() test_parse_links_multiple_on_one_line() + test_parse_wikilink_line_ranges() + test_parse_markdown_links_are_ignored() test_parse_links_no_match() - test_parse_links_predicate_with_underscore_and_digits() test_parse_links_in_file() test_parse_links_empty_when_no_content() test_chunk_does_not_split_wikilink_at_boundary() diff --git a/tests/unit/test_file_graph.py b/tests/unit/test_file_graph.py index 00d377c5..a125d3fe 100644 --- a/tests/unit/test_file_graph.py +++ b/tests/unit/test_file_graph.py @@ -38,6 +38,28 @@ def make_node(path: str, links: list[tuple[str, str | None]] | None = None) -> F ) +def test_file_node_loads_legacy_predicate_without_persisting_it(): + """Legacy predicates are accepted for compatibility but omitted from new snapshots.""" + node = FileNode.model_validate( + { + "path": "a.md", + "st_mtime": 1.0, + "links": [ + { + "source_path": "a.md", + "target_path": "b.md", + "target_anchor": "intro", + "predicate": "related", + }, + ], + }, + ) + assert node.links == [ + FileLink(source_path="a.md", target_path="b.md", target_anchor="intro", predicate="related"), + ] + assert "predicate" not in node.model_dump_json() + + # Both backends should satisfy the same BaseFileGraph contract. BACKENDS = [LocalFileGraph, NxFileGraph] diff --git a/tests/unit/test_line_anchor.py b/tests/unit/test_line_anchor.py new file mode 100644 index 00000000..3e678af7 --- /dev/null +++ b/tests/unit/test_line_anchor.py @@ -0,0 +1,31 @@ +"""Tests for GitHub-style line anchor parsing.""" + +import pytest + +from reme.utils import format_line_anchor, parse_line_anchor + + +def test_parse_supported_line_anchors(): + """Single lines, continuous ranges, and comma-separated ranges parse.""" + assert parse_line_anchor("L9") == [(9, 9)] + assert parse_line_anchor("L9-L10") == [(9, 10)] + assert parse_line_anchor("L9-L10,L15-L20") == [(9, 10), (15, 20)] + + +def test_parse_merges_overlapping_and_adjacent_ranges(): + """Normalization merges ranges whose covered lines touch.""" + ranges = parse_line_anchor("L9-L12,L11-L15,L16") + assert ranges == [(9, 16)] + assert format_line_anchor(ranges) == "L9-L16" + + +def test_non_line_heading_is_unchanged(): + """Ordinary heading anchors are outside this parser's contract.""" + assert parse_line_anchor("Introduction") is None + + +@pytest.mark.parametrize("anchor", ["L0", "L10-L9", "L9-Lx", "L9,"]) +def test_invalid_line_anchor_rejected(anchor): + """Line-looking anchors fail clearly when malformed or out of range.""" + with pytest.raises(ValueError): + parse_line_anchor(anchor) diff --git a/tests/unit/test_link_expansion.py b/tests/unit/test_link_expansion.py index ebbd6417..57de6aec 100644 --- a/tests/unit/test_link_expansion.py +++ b/tests/unit/test_link_expansion.py @@ -113,7 +113,7 @@ def test_expand_links_returns_outlinks_and_inlinks_with_meta(): assert len(a_out) == 1 assert a_out[0]["path"] == "B.md" assert a_out[0]["meta"] == {"name": "B Doc", "description": "beta"} - assert a_out[0]["edges"] == [{"predicate": None, "anchor": None}] + assert a_out[0]["anchors"] == [] assert result["A.md"]["inlinks"] == [] b_in = result["B.md"]["inlinks"] @@ -184,13 +184,13 @@ def test_render_expansion_lines_empty_input_yields_empty_list(): def test_render_expansion_lines_outlinks_only(): - """Single outlink with meta + plain edge renders as 3 lines.""" + """Single outlink with metadata renders without a redundant edge line.""" expansion = { "outlinks": [ { "path": "B.md", "meta": {"name": "B", "description": "beta"}, - "edges": [{"predicate": None, "anchor": None}], + "anchors": [], }, ], "inlinks": [], @@ -199,20 +199,19 @@ def test_render_expansion_lines_outlinks_only(): assert lines == [ " outlinks (1):", ' → B.md name="B" description="beta"', - " via plain", ] print("✓ test_render_expansion_lines_outlinks_only passed") -def test_render_expansion_lines_inlinks_only_with_predicate_and_anchor(): - """Inlink edge with predicate + anchor renders via descriptor.""" +def test_render_expansion_lines_inlinks_only_with_anchor(): + """Inlink anchor renders via descriptor.""" expansion = { "outlinks": [], "inlinks": [ { "path": "src.md", "meta": {}, - "edges": [{"predicate": "references", "anchor": "intro"}], + "anchors": ["intro"], }, ], } @@ -220,26 +219,26 @@ def test_render_expansion_lines_inlinks_only_with_predicate_and_anchor(): assert lines == [ " inlinks (1):", " ← src.md (no meta)", - " via predicate=references, anchor=#intro", + " via anchor=#intro", ] - print("✓ test_render_expansion_lines_inlinks_only_with_predicate_and_anchor passed") + print("✓ test_render_expansion_lines_inlinks_only_with_anchor passed") def test_render_expansion_lines_both_directions_in_order(): """outlinks block precedes inlinks block.""" expansion = { "outlinks": [ - {"path": "out.md", "meta": {"name": "Out"}, "edges": [{"predicate": None, "anchor": None}]}, + {"path": "out.md", "meta": {"name": "Out"}, "anchors": []}, ], "inlinks": [ - {"path": "in.md", "meta": {"description": "incoming"}, "edges": [{"predicate": None, "anchor": None}]}, + {"path": "in.md", "meta": {"description": "incoming"}, "anchors": []}, ], } lines = render_expansion_lines(expansion) assert lines[0] == " outlinks (1):" - assert lines[3] == " inlinks (1):" + assert lines[2] == " inlinks (1):" assert lines[1].lstrip().startswith("→") - assert lines[4].lstrip().startswith("←") + assert lines[3].lstrip().startswith("←") print("✓ test_render_expansion_lines_both_directions_in_order passed") @@ -250,5 +249,5 @@ if __name__ == "__main__": test_expand_links_node_without_meta_returns_empty_meta_dict() test_render_expansion_lines_empty_input_yields_empty_list() test_render_expansion_lines_outlinks_only() - test_render_expansion_lines_inlinks_only_with_predicate_and_anchor() + test_render_expansion_lines_inlinks_only_with_anchor() test_render_expansion_lines_both_directions_in_order() diff --git a/tests/unit/test_markdown_file_chunker.py b/tests/unit/test_markdown_file_chunker.py index 1d7bdcb6..118443ca 100644 --- a/tests/unit/test_markdown_file_chunker.py +++ b/tests/unit/test_markdown_file_chunker.py @@ -284,9 +284,9 @@ def test_parse_links_literal_targets(): path = _write_md(tmp, "note.md", body) chunker = MarkdownFileChunker() node, _ = await chunker.chunk(path) - triples = {(link.target_path, link.target_anchor, link.predicate) for link in node.links} - assert ("topics/Alice.md", None, None) in triples - assert ("topics/Bob.md", "sec", None) in triples + pairs = {(link.target_path, link.target_anchor) for link in node.links} + assert ("topics/Alice.md", None) in pairs + assert ("topics/Bob.md", "sec") in pairs # source_path always equals the node's own path for link in node.links: assert link.source_path == node.path @@ -317,8 +317,8 @@ def test_parse_links_short_and_no_ext_kept_literally(): asyncio.run(run()) -def test_parse_links_predicate_inline_and_line(): - """Both `pred:: [[X]]` (line-level) and `[pred:: [[X]]]` (inline) propagate predicate.""" +def test_parse_links_legacy_relation_text_is_ignored(): + """Old relation wrappers do not prevent their inner wikilinks from parsing.""" async def run(): with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp): @@ -326,16 +326,14 @@ def test_parse_links_predicate_inline_and_line(): path = _write_md(tmp, "note.md", body) chunker = MarkdownFileChunker() node, _ = await chunker.chunk(path) - pairs = {(link.target_path, link.predicate) for link in node.links} - assert ("A.md", "extends") in pairs - assert ("B.md", "concerns") in pairs - print("✓ test_parse_links_predicate_inline_and_line passed") + assert {link.target_path for link in node.links} == {"A.md", "B.md"} + print("✓ test_parse_links_legacy_relation_text_is_ignored passed") asyncio.run(run()) def test_parse_links_deduped(): - """Repeated wikilinks with the same (target, predicate, anchor) emit one FileLink.""" + """Repeated wikilinks with the same target and anchor emit one FileLink.""" async def run(): with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp): @@ -616,7 +614,7 @@ if __name__ == "__main__": test_parse_chunk_ids_match_node_chunk_ids() test_parse_links_literal_targets() test_parse_links_short_and_no_ext_kept_literally() - test_parse_links_predicate_inline_and_line() + test_parse_links_legacy_relation_text_is_ignored() test_parse_links_deduped() test_parse_min_chunk_byte_size_clamped() test_parse_embed_toc_prefixes_chunk_text() diff --git a/tests/unit/test_read_with_neighbors.py b/tests/unit/test_read_with_neighbors.py index 77ac5abb..56b94f0c 100644 --- a/tests/unit/test_read_with_neighbors.py +++ b/tests/unit/test_read_with_neighbors.py @@ -141,6 +141,28 @@ def test_read_with_neighbors_injects_block_and_metadata(): _run(run()) +def test_read_with_neighbors_uses_posix_nested_path(): + """Nested graph keys use POSIX separators on every platform.""" + + async def run(): + with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp): + store = await _store_with( + { + "notes/A.md": {"body": "See [[notes/B.md]].", "name": "A Doc"}, + "notes/B.md": {"body": "End node.", "name": "B Doc"}, + }, + ) + resp = await _read(store, step_kwargs={"with_neighbors": True}, path="notes/A.md") + + expansion = resp.metadata.get("link_expansion") + assert expansion is not None + assert "notes/A.md" in expansion + assert expansion["notes/A.md"]["outlinks"][0]["path"] == "notes/B.md" + await store.close() + + _run(run()) + + # -- on + zero neighbors ----------------------------------------------------- diff --git a/tests/unit/test_wikilink_utils.py b/tests/unit/test_wikilink_utils.py index 06eebbd1..7a69cfff 100644 --- a/tests/unit/test_wikilink_utils.py +++ b/tests/unit/test_wikilink_utils.py @@ -1,4 +1,4 @@ -"""Tests for the wikilink helpers in ``reme.utils.wikilink_handler``. +"""Tests for wikilink helpers in ``reme.utils.wikilink_handler``. Two pure async helpers used by file_move / file_delete: @@ -196,8 +196,8 @@ def test_retarget_image_marker_preserved(): asyncio.run(run()) -def test_retarget_dataview_predicate_preserved(): - """Line-level + inline-bracketed Dataview predicates pass through outside ``[[..]]``.""" +def test_retarget_preserves_surrounding_relation_text(): + """Retargeting changes only the inner wikilink target.""" async def run(): with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp): @@ -212,11 +212,33 @@ def test_retarget_dataview_predicate_preserved(): assert "colleague:: [[people/Alice.md]]" in body assert "[负责:: [[people/Alice.md]]]" in body await store.close() - print("✓ test_retarget_dataview_predicate_preserved passed") + print("✓ test_retarget_preserves_surrounding_relation_text passed") asyncio.run(run()) +def test_markdown_links_are_ignored(): + """Only double-bracket wikilinks create graph edges or get rewritten.""" + markdown_links = ( + "[label](../wiki/example.md) " + "[label](../wiki/example.md#section) " + "[label](../wiki/example.md#L9-L10,L15-L20)" + ) + original = f"{markdown_links} [[wiki/example.md#L9]]" + + rewritten, count = WikilinkHandler.scan_and_rewrite( + original, + old="wiki/example.md", + new="archive/example.md", + ) + + assert [(link.target_path, link.target_anchor) for link in WikilinkHandler.extract_links(original, "note.md")] == [ + ("wiki/example.md", "L9"), + ] + assert count == 1 + assert rewritten == f"{markdown_links} [[archive/example.md#L9]]" + + def test_retarget_multiple_files_aggregate_counts(): """links_changed sums across files; by_file lists per-file counts.""" @@ -387,7 +409,8 @@ if __name__ == "__main__": test_retarget_alias_preserved() test_retarget_anchor_and_alias_together() test_retarget_image_marker_preserved() - test_retarget_dataview_predicate_preserved() + test_retarget_preserves_surrounding_relation_text() + test_markdown_links_are_ignored() test_retarget_multiple_files_aggregate_counts() test_retarget_dry_run_does_not_write() test_retarget_scope_limits_sweep()