mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-08-28 05:25:04 +00:00
feat: simplify wikilink semantics and support line anchors (#412)
* feat: simplify local links and support line anchors * fix: align line anchor tests with CI lint * fix: preserve local links across file moves * fix: encode markdown paths when rewriting links * refactor(read): keep explicit line range parameters * fix: simplify legacy link predicate compatibility * docs: align local link behavior with implementation * fix: skip unsupported markdown destination escapes * fix: normalize workspace link paths across platforms * fix: bound markdown link scanning * fix: keep local link processing linear * docs: clarify permissive markdown link parsing * fix: handle local link processing failures * refactor: limit file links to wikilink syntax * docs: align wikilink contract with implementation * fix: normalize dream and neighbor paths on Windows * fix: resolve workspace path for neighbor expansion
This commit is contained in:
jinliyl
GitHub
parent
6b035c6553
commit
a9ec334adc
37 changed files with 587 additions and 491 deletions
11
.github/workflows/windows-smoke.yml
vendored
11
.github/workflows/windows-smoke.yml
vendored
|
|
@ -32,8 +32,15 @@ jobs:
|
|||
- name: Install package
|
||||
run: |
|
||||
python -m pip install --upgrade pip setuptools wheel
|
||||
pip install agentscope
|
||||
pip install -e .
|
||||
pip install -e ".[dev,core]"
|
||||
|
||||
- name: Run version job
|
||||
run: reme start service.backend=cli job=version
|
||||
|
||||
- name: Run Windows path tests
|
||||
run: |
|
||||
python -m pytest `
|
||||
tests/unit/test_auto_dream.py::test_scan_day_files_includes_nested_md_and_excludes_interests `
|
||||
tests/unit/test_auto_dream.py::test_dream_extract_matches_posix_catalog_paths `
|
||||
tests/unit/test_read_with_neighbors.py::test_read_with_neighbors_uses_posix_nested_path `
|
||||
-v
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ units into `digest/`, and generates the day's `interests.yaml` for proactive use
|
|||
</p>
|
||||
|
||||
Its daily inputs usually come from [Auto Memory](./auto_memory.md) and [Auto Resource](./auto_resource.md). For the file
|
||||
semantics of `digest/`, `derived_from::`, and wikilinks, see [Memory as File](./memory_as_file.md). For the linking strategy
|
||||
semantics of `digest/`, Sources sections, and wikilinks, see [Memory as File](./memory_as_file.md). For the linking strategy
|
||||
used during Integrate, see [Auto Link](./auto_link.md). To read `interests.yaml`, use [Proactive](./proactive.md).
|
||||
|
||||
## Configuration
|
||||
|
|
@ -198,8 +198,8 @@ jobs:
|
|||
`auto_dream` consumes only daily inputs and does not rewrite daily bodies. Daily preserves facts and the original situation;
|
||||
digest is the abstracted long-term memory layer.
|
||||
|
||||
`digest` is not a copy of the source text. Its body should preserve reusable abstractions, while details point back to sources
|
||||
through `derived_from:: [[daily/<date>/...]]`. Links follow the workspace-relative wikilink semantics described in
|
||||
`digest` is not a copy of the source text. Its body should preserve reusable abstractions, while a Sources section points
|
||||
back with entries such as `- [[daily/<date>/...]]`. Links follow the workspace-relative wikilink semantics described in
|
||||
[Memory as File](./memory_as_file.md).
|
||||
|
||||
`auto_dream` does not invent an overview from nothing. Only content that actually appears in daily input and is extracted as
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ link to multiple sources and multiple related digest nodes.
|
|||
|---|---|
|
||||
| The same memory already exists | Recall and update the existing node instead of creating a duplicate. |
|
||||
| New and existing material are related | Write workspace-relative wikilinks into the body. |
|
||||
| A digest node is disconnected from its sources | Point back to daily/resource source material with `derived_from:: [[...]]`. |
|
||||
| A digest node is disconnected from its sources | Add daily/resource links under a `## Sources` section. |
|
||||
| A node contains only isolated prose | Add links to related digest nodes on both CREATE and UPDATE. |
|
||||
|
||||
## Toolchain
|
||||
|
|
@ -77,20 +77,22 @@ Every unit must select one action:
|
|||
| Action | Linking semantics |
|
||||
|---|---|
|
||||
| `CREATE` | Write a new `digest/<bucket>/<slug>.md` and add source and related-node links to its body. |
|
||||
| `CORROBORATE` | The same abstraction appeared again; append a new `derived_from:: [[...]]` and strengthen the description when needed. |
|
||||
| `CORROBORATE` | The same abstraction appeared again; append its source link and strengthen the description when needed. |
|
||||
| `REFINE` | New material extends the existing node; insert the additional content in the appropriate section and preserve existing links. |
|
||||
| `CORRECT` | New material corrects the existing node; use source links to identify the basis for the correction. |
|
||||
|
||||
An UPDATE should be additive whenever possible: do not delete existing wikilinks or `derived_from` entries. This prevents
|
||||
An UPDATE should be additive whenever possible: do not delete existing wikilinks or source entries. This prevents
|
||||
later graph indexing and retrieval from losing edges.
|
||||
|
||||
### 3. Write source edges
|
||||
|
||||
Source edges use Markdown wikilinks:
|
||||
Source edges are ordinary wikilinks grouped under a Markdown heading:
|
||||
|
||||
```markdown
|
||||
derived_from:: [[daily/2026-06-20/session.md]]
|
||||
derived_from:: [[resource/2026-06-20/paper.md]]
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-06-20/session.md]]
|
||||
- [[resource/2026-06-20/paper.md]]
|
||||
```
|
||||
|
||||
These edges represent the evidence behind a digest node. Plain-text descriptions do not count as source edges because only
|
||||
|
|
@ -99,17 +101,14 @@ wikilinks can be parsed reliably by the file graph. For the complete parsing rul
|
|||
|
||||
### 4. Write relationships between digest nodes
|
||||
|
||||
Relationships between digest nodes also use complete workspace-relative paths:
|
||||
Relationships between digest nodes use complete workspace-relative paths woven into natural prose:
|
||||
|
||||
```markdown
|
||||
relates_to:: [[digest/wiki/hybrid-search.md]]
|
||||
depends_on:: [[digest/procedure/rebuild-index.md]]
|
||||
blocks_on:: [[digest/personal/team-review-preference.md]]
|
||||
This design extends [[digest/wiki/hybrid-search.md]] and uses
|
||||
[[digest/procedure/rebuild-index.md]]. Follow
|
||||
[[digest/personal/team-review-preference.md]] during review.
|
||||
```
|
||||
|
||||
Predicates are open-ended. Common forms include `relates_to::`, `depends_on::`, and `blocks_on::`. The predicate sits outside
|
||||
the brackets, while the target path goes inside `[[...]]` and should include the `.md` suffix.
|
||||
|
||||
## Bucket Differences
|
||||
|
||||
`auto_link` adjusts the shape of its output according to the unit bucket:
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ ReMe represents memory as files not merely for convenient storage, but to give l
|
|||
|---|---|
|
||||
| Readable | Users can open the workspace directly and read daily notes, digest nodes, and source material like ordinary notes. |
|
||||
| Editable | Users and agents can correct, extend, move, or delete memory with file operations, without a specialized database client. |
|
||||
| Traceable | Long-term conclusions in digest can point back to daily, resource, or session sources through `derived_from:: [[...]]`. |
|
||||
| Traceable | Long-term conclusions in digest can point back to daily, resource, or session files from a Sources section. |
|
||||
| Portable | The workspace is an ordinary directory. Markdown, JSONL, YAML, and resource files can be backed up, synchronized, versioned, or moved to other tools. |
|
||||
| Indexable | Although the files are plain text, ReMe parses frontmatter, chunks, and wikilinks to build a retrieval index and file graph. |
|
||||
| Collaborative | Humans judge and correct; agents organize, link, and retrieve. Both operate on the same files. |
|
||||
|
|
@ -131,9 +131,7 @@ tags: [new energy, solar]
|
|||
# Conclusions
|
||||
|
||||
The solar supply chain consists of [[digest/wiki/polysilicon.md]], wafers, cells, and modules.
|
||||
|
||||
upstream:: [[digest/wiki/polysilicon.md]]
|
||||
[company:: [[digest/wiki/longi.md|LONGi]]]
|
||||
One major producer is [[digest/wiki/longi.md|LONGi]].
|
||||
```
|
||||
|
||||
### Frontmatter
|
||||
|
|
@ -163,15 +161,18 @@ confidence: observed
|
|||
|
||||
The user repeatedly asks documentation to explain motivation, boundaries, and examples while avoiding marketing language.
|
||||
|
||||
derived_from:: [[daily/2026-06-20/session-a.md]]
|
||||
related:: [[digest/procedure/technical-documentation.md]]
|
||||
Apply this preference when following [[digest/procedure/technical-documentation.md]].
|
||||
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-06-20/session-a.md]]
|
||||
```
|
||||
|
||||
This has three benefits:
|
||||
|
||||
1. `name` and `description` serve as lightweight summaries in lists, recall results, and agent decisions.
|
||||
2. The body can carry fuller facts, conditions, counterexamples, and sources.
|
||||
3. Typed wikilinks such as `derived_from::` and `related::` can be parsed by the graph and maintained when files move.
|
||||
3. Ordinary wikilinks can be parsed by the graph and maintained when files move.
|
||||
|
||||
Frontmatter is best for stable, short, structured fields; the body is best for explanations meant for people. Do not put long
|
||||
body text into YAML fields.
|
||||
|
|
@ -181,10 +182,10 @@ body text into YAML fields.
|
|||
Wikilinks express relationships between files with `[[...]]`:
|
||||
|
||||
```text
|
||||
[[digest/wiki/solar.md]]
|
||||
[[digest/wiki/solar.md#supply-chain]]
|
||||
[[digest/wiki/solar.md|solar]]
|
||||
![[resource/2026-06-01/report.md]]
|
||||
[[daily/2026-06-20/session.md]]
|
||||
[[notes/example.md#L9]]
|
||||
[[notes/example.md#L9-L10]]
|
||||
[[notes/example.md#L9-L10,L15-L20]]
|
||||
```
|
||||
|
||||
ReMe wikilinks use **literal path semantics**:
|
||||
|
|
@ -196,54 +197,59 @@ ReMe wikilinks use **literal path semantics**:
|
|||
ReMe does not append `.md` automatically, search by filename, or automatically resolve folder notes. Use complete
|
||||
workspace-relative paths with their extensions.
|
||||
|
||||
Ordinary Markdown links such as `[label](../wiki/example.md)` do not create `FileLink` edges and are not rewritten by move or
|
||||
retarget operations.
|
||||
|
||||
Anchors such as `#L9`, `#L9-L10`, and `#L9-L10,L15-L20` remain ordinary `target_anchor` strings in the graph. The graph
|
||||
parser does not validate line-anchor syntax, so values such as `#L0`, `#L10-L9`, and `#L9,` are also stored. The `read` job
|
||||
does not interpret an anchor appended to `path`; use the separate 1-based, inclusive `start_line` and `end_line` arguments to
|
||||
read a range, for example `read(path="digest/wiki/solar.md", start_line=9, end_line=10)`.
|
||||
|
||||
Wikilinks support these behaviors:
|
||||
|
||||
```text
|
||||
body link -> create a FileLink
|
||||
predicate:: link -> create a FileLink with a relationship name
|
||||
move a file -> rewrite [[old path]] in inbound edges by default
|
||||
delete a file -> return remaining inbound edges so references can be cleaned up
|
||||
search match -> expand inbound and outbound links to provide context
|
||||
```
|
||||
|
||||
Supported relationship forms:
|
||||
|
||||
```markdown
|
||||
industry:: [[digest/wiki/new-energy.md]]
|
||||
[competitor:: [[digest/wiki/byd.md]]]
|
||||
```
|
||||
|
||||
Parsed result:
|
||||
|
||||
```text
|
||||
FileLink
|
||||
source_path = current file
|
||||
target_path = digest/wiki/new-energy.md
|
||||
predicate = industry
|
||||
target_path = notes/example.md
|
||||
target_anchor = L9-L10,L15-L20
|
||||
```
|
||||
|
||||
Older documents containing wrappers such as `related:: [[path]]`,
|
||||
`- related:: [[path]]`, or `[related:: [[path]]]` remain readable. ReMe
|
||||
ignores the surrounding text and indexes the inner `[[path]]` as an ordinary
|
||||
link. After upgrading from a version that stored typed links, run `reme reindex`
|
||||
once to rebuild the derived graph without the removed relationship field.
|
||||
|
||||
### Sources and Relationships
|
||||
|
||||
The two most important link types in ReMe are source links and conceptual relationship links.
|
||||
|
||||
A source link explains where a long-term memory came from:
|
||||
A Sources section records where a long-term memory came from:
|
||||
|
||||
```markdown
|
||||
derived_from:: [[daily/2026-06-20/session-a.md]]
|
||||
derived_from:: [[resource/2026-06-20/report.pdf]]
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-06-20/session-a.md]]
|
||||
- [[resource/2026-06-20/report.pdf]]
|
||||
```
|
||||
|
||||
A conceptual relationship link explains which other long-term memories relate to the node:
|
||||
A conceptual relationship link explains which other long-term memories relate to the node. Weave it into natural prose:
|
||||
|
||||
```markdown
|
||||
related:: [[digest/wiki/solar-supply-chain.md]]
|
||||
depends_on:: [[digest/procedure/research-report-analysis.md]]
|
||||
contrasts_with:: [[digest/wiki/central-inverter.md]]
|
||||
This analysis extends [[digest/wiki/solar-supply-chain.md]], follows
|
||||
[[digest/procedure/research-report-analysis.md]], and contrasts with
|
||||
[[digest/wiki/central-inverter.md]].
|
||||
```
|
||||
|
||||
Ordinary body wikilinks also create graph edges, but when the relationship itself has semantic value, prefer
|
||||
`predicate:: [[path]]`. This makes the meaning of links clearer to search, graph traversal, and later agent integration.
|
||||
|
||||
## Human and Agent Editing
|
||||
|
||||
Because memory is stored as files, users can edit the workspace directly, while agents can read and write the same files
|
||||
|
|
@ -257,8 +263,8 @@ through ReMe's file tools. Both follow the same conventions:
|
|||
| Delete a file | Check inbound links first. ReMe's delete tool returns source files that still point to the target, making dangling references easier to clean up. |
|
||||
| Edit metadata | Use frontmatter for short fields. When the body changes substantially, update `description` as well. |
|
||||
|
||||
A practical rule is: **an agent may rewrite the wording, but it must not lose evidence edges**. In particular,
|
||||
`derived_from:: [[...]]` and existing digest-to-digest wikilinks are the basis for traceable and extensible long-term memory.
|
||||
A practical rule is: **an agent may rewrite the wording, but it must not lose evidence edges**. In particular, Sources entries
|
||||
and existing digest-to-digest wikilinks are the basis for traceable and extensible long-term memory.
|
||||
|
||||
## Path Semantics
|
||||
|
||||
|
|
|
|||
|
|
@ -66,7 +66,7 @@ stable batch of changes.
|
|||
4. For a deleted file, remove its records from `file_store`, `keyword_index`, and `file_graph`.
|
||||
5. When changes exist, dump state to `metadata/` so it can be restored on the next startup.
|
||||
|
||||
The Markdown chunker parses YAML frontmatter, heading structure, and `[[...]]` into `FileNode`, `FileChunk`, and `FileLink`
|
||||
The Markdown chunker parses YAML frontmatter, heading structure, and wikilinks into `FileNode`, `FileChunk`, and `FileLink`
|
||||
objects. For detailed chunking rules, see [Memory as File](./memory_as_file.md#memory-chunking).
|
||||
|
||||
### Index Optimization
|
||||
|
|
@ -182,7 +182,8 @@ and then adds the new chunk text. Deletion is lazy; the index can later be compa
|
|||
"Progressive" in Memory Search does not mean putting the entire repository into one result. Retrieval expands in three layers:
|
||||
|
||||
1. Chunk recall: return only the `limit` most relevant text fragments.
|
||||
2. File location: each result includes `path:start_line-end_line`, allowing the caller to read the source precisely with `read`.
|
||||
2. File location: each result includes `path:start_line-end_line`. Pass the path and line bounds separately as `path`,
|
||||
`start_line`, and `end_line` when calling `read`; the range is not part of the `path` value.
|
||||
3. Link neighbors: call `expand_links()` for each matched file and expand at most `max_links_per_direction` outlinks and
|
||||
inlinks.
|
||||
|
||||
|
|
@ -194,7 +195,7 @@ matched chunk
|
|||
-> file_store.get_outlinks(path)
|
||||
-> file_store.get_inlinks(path)
|
||||
-> file_store.get_nodes(neighbor_paths)
|
||||
-> render neighbor path, name, description, predicate, and anchor
|
||||
-> render neighbor path, name, description, and anchor
|
||||
```
|
||||
|
||||
This keeps search results short while still showing which long-term nodes, resources, or other daily notes a memory connects
|
||||
|
|
@ -216,10 +217,8 @@ Typical text structure:
|
|||
...matched memory fragment...
|
||||
outlinks (2):
|
||||
-> digest/indexing.md name="Indexing" description="..."
|
||||
via predicate=related
|
||||
inlinks (1):
|
||||
<- daily/2026-06-19.md name="..."
|
||||
via plain
|
||||
```
|
||||
|
||||
`counts` reports how many vector and keyword candidates were recalled and how many results were ultimately returned. With
|
||||
|
|
|
|||
|
|
@ -110,17 +110,19 @@ name: Cobalt
|
|||
description: A key raw material for lithium-battery cathodes, with production concentrated in the DRC
|
||||
---
|
||||
|
||||
downstream_product:: [[digest/wiki/ternary-cathodes.md]]
|
||||
producer:: [[digest/wiki/glencore.md]]
|
||||
source_event:: [[daily/2026-05-18/2026-05-18-close.md]]
|
||||
|
||||
# Cobalt
|
||||
|
||||
Used by [[digest/wiki/ternary-cathodes.md]]; a major producer is [[digest/wiki/glencore.md]].
|
||||
|
||||
## Supply
|
||||
Glencore's third-quarter cobalt output fell 18% year over year. Continue monitoring how tighter supply affects prices.
|
||||
|
||||
## Policy risk
|
||||
Changes to mining-rights policy in the DRC may affect KFM mine operations and should be tracked together with CMOC.
|
||||
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-05-18/2026-05-18-close.md]]
|
||||
```
|
||||
|
||||
Note that wikilinks use literal path semantics. Prefer complete workspace-relative paths with the `.md` extension. ReMe
|
||||
|
|
@ -185,10 +187,10 @@ The result shape is:
|
|||
Glencore's third-quarter cobalt output fell 18% year over year...
|
||||
|
||||
outlinks:
|
||||
-> digest/wiki/ternary-cathodes.md name="Ternary Cathodes" via predicate=downstream_product
|
||||
-> digest/wiki/glencore.md name="Glencore" via predicate=producer
|
||||
-> digest/wiki/ternary-cathodes.md name="Ternary Cathodes"
|
||||
-> digest/wiki/glencore.md name="Glencore"
|
||||
inlinks:
|
||||
<- digest/wiki/ternary-cathodes.md name="Ternary Cathodes" via predicate=upstream_material
|
||||
<- digest/wiki/ternary-cathodes.md name="Ternary Cathodes"
|
||||
|
||||
========== digest/wiki/ternary-cathodes.md:5-18 [score=0.0139 keyword=3.2017] ==========
|
||||
...
|
||||
|
|
@ -301,11 +303,10 @@ name: TypeScript project build OOM diagnostic path
|
|||
description: When a build stalls and memory grows, check the type-checking process first
|
||||
---
|
||||
|
||||
source_event:: [[daily/2026-03-10/build-oom-2026-03-10.md]]
|
||||
related_preference:: [[digest/personal/code-style.md]]
|
||||
|
||||
# TypeScript Project Build OOM Diagnostic Path
|
||||
|
||||
Apply [[digest/personal/code-style.md]] while following this runbook.
|
||||
|
||||
## Symptoms
|
||||
The build stalls near the end. CPU usage is low, but memory keeps growing.
|
||||
|
||||
|
|
@ -317,6 +318,10 @@ The build stalls near the end. CPU usage is low, but memory keeps growing.
|
|||
## Known ineffective paths
|
||||
- Deleting `.cache` alone did not resolve the issue on 2026-03-10.
|
||||
- Upgrading the terser plugin did not resolve the issue on 2026-03-10.
|
||||
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-03-10/build-oom-2026-03-10.md]]
|
||||
```
|
||||
|
||||
Example `digest/personal/code-style.md`:
|
||||
|
|
@ -411,13 +416,14 @@ name: Alice
|
|||
description: A friend of the user who often recommends reading material
|
||||
---
|
||||
|
||||
recommended_book:: [[digest/wiki/deep-work.md]]
|
||||
source_event:: [[daily/2026-04-20/lunch-with-alice.md]]
|
||||
|
||||
# Alice
|
||||
|
||||
## Reading recommendations
|
||||
At lunch on 2026-04-20, Alice recommended a book about attention and deep work.
|
||||
At lunch on 2026-04-20, Alice recommended [[digest/wiki/deep-work.md]], a book about attention and deep work.
|
||||
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-04-20/lunch-with-alice.md]]
|
||||
```
|
||||
|
||||
### An associative recall
|
||||
|
|
@ -439,7 +445,7 @@ Matches:
|
|||
```text
|
||||
digest/personal/alice.md
|
||||
outlinks:
|
||||
-> digest/wiki/deep-work.md via predicate=recommended_book
|
||||
-> digest/wiki/deep-work.md
|
||||
daily/2026-04-20/lunch-with-alice.md
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
<img src="../figure/auto-dream-and-proactive.svg" alt="ReMe Auto Dream and Proactive 从 daily 到 digest 再到 proactive 的流程" width="92%">
|
||||
</p>
|
||||
|
||||
它消费的 daily 输入通常来自 [Auto Memory](./auto_memory.md) 和 [Auto Resource](./auto_resource.md)。`digest/`、`derived_from::`
|
||||
它消费的 daily 输入通常来自 [Auto Memory](./auto_memory.md) 和 [Auto Resource](./auto_resource.md)。`digest/`、Sources 章节
|
||||
和 wikilink 的文件语义见 [Memory as File](./memory_as_file.md)；Integrate 阶段的链接策略详见 [Auto Link](./auto_link.md)。
|
||||
`interests.yaml` 的读取接口见 [Proactive](./proactive.md)。
|
||||
|
||||
|
|
@ -189,7 +189,7 @@ jobs:
|
|||
|
||||
`auto_dream` 只消费 daily 输入，不改写 daily 正文。daily 是事实和现场记录，digest 才是抽象后的长期记忆层。
|
||||
|
||||
`digest` 不是原文复制。正文应保留可复用抽象，细节通过 `derived_from:: [[daily/<date>/...]]` 指回来源。链接写法遵循
|
||||
`digest` 不是原文复制。正文应保留可复用抽象，细节通过 Sources 章节中的 `- [[daily/<date>/...]]` 指回来源。链接写法遵循
|
||||
[Memory as File](./memory_as_file.md) 中的 workspace-relative wikilink 语义。
|
||||
|
||||
`auto_dream` 不凭空生成总览。只有 daily 输入中确实出现、并被抽取为 unit 或 topic 的内容，才会进入 digest 或
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ Integrate 阶段对每个 unit 独立运行。一个 unit 只落到一个目标
|
|||
|--------------|----------------------------------------------------|
|
||||
| 已有相同记忆 | 召回后更新旧节点，而不是重复创建。 |
|
||||
| 新旧材料有关联 | 在正文中写入 workspace-relative wikilink。 |
|
||||
| digest 与来源断开 | 用 `derived_from:: [[...]]` 指回 daily/resource 原始材料。 |
|
||||
| digest 与来源断开 | 在 `## Sources` 章节加入指向 daily/resource 的链接。 |
|
||||
| 节点只有孤立正文 | 在 CREATE 和 UPDATE 时都补充相关 digest 节点链接。 |
|
||||
|
||||
## 工具链
|
||||
|
|
@ -73,19 +73,21 @@ Agent 先用 unit 的触发条件、动词、名词、同义词和可能的 fail
|
|||
| 动作 | 链接含义 |
|
||||
|---------------|-----------------------------------------------------|
|
||||
| `CREATE` | 写入新的 `digest/<bucket>/<slug>.md`，并在新正文里加入来源和相关节点链接。 |
|
||||
| `CORROBORATE` | 同一抽象再次出现，追加新的 `derived_from:: [[...]]`，必要时强化描述。 |
|
||||
| `CORROBORATE` | 同一抽象再次出现，追加来源链接，必要时强化描述。 |
|
||||
| `REFINE` | 新材料扩展了旧节点，把补充内容插入合适段落，并保留旧链接。 |
|
||||
| `CORRECT` | 新材料修正旧节点，用来源链接标出修正依据。 |
|
||||
|
||||
UPDATE 必须尽量只增不删：不要删除已有 wikilink 或 `derived_from`。这是为了让后续图谱索引和检索不会丢边。
|
||||
UPDATE 必须尽量只增不删：不要删除已有 wikilink 或来源条目。这是为了让后续图谱索引和检索不会丢边。
|
||||
|
||||
### 3. 写来源边
|
||||
|
||||
来源边使用 markdown wikilink：
|
||||
来源边是归档在 Markdown 固定章节下的普通 Wikilink：
|
||||
|
||||
```markdown
|
||||
derived_from:: [[daily/2026-06-20/session.md]]
|
||||
derived_from:: [[resource/2026-06-20/paper.md]]
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-06-20/session.md]]
|
||||
- [[resource/2026-06-20/paper.md]]
|
||||
```
|
||||
|
||||
这些边表示 digest 节点的证据来源。纯文本描述不算来源边，因为只有 wikilink 能被 file graph 稳定解析。更完整的 wikilink
|
||||
|
|
@ -93,17 +95,14 @@ derived_from:: [[resource/2026-06-20/paper.md]]
|
|||
|
||||
### 4. 写 digest 关联边
|
||||
|
||||
digest 之间的关联也使用完整 workspace-relative 路径：
|
||||
digest 之间的关联使用完整 workspace-relative 路径，并自然织入正文：
|
||||
|
||||
```markdown
|
||||
relates_to:: [[digest/wiki/hybrid-search.md]]
|
||||
depends_on:: [[digest/procedure/rebuild-index.md]]
|
||||
blocks_on:: [[digest/personal/team-review-preference.md]]
|
||||
这个设计扩展了 [[digest/wiki/hybrid-search.md]]，并使用
|
||||
[[digest/procedure/rebuild-index.md]]。评审时遵循
|
||||
[[digest/personal/team-review-preference.md]]。
|
||||
```
|
||||
|
||||
谓词是开放的，常见写法包括 `relates_to::`、`depends_on::`、`blocks_on::`。谓词在括号外，目标路径在 `[[...]]` 内，并且应包含
|
||||
`.md` 后缀。
|
||||
|
||||
## Bucket 差异
|
||||
|
||||
`auto_link` 的规则会随 unit bucket 调整写入形态：
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ ReMe 把记忆设计成文件，不只是为了“方便存储”，而是为了
|
|||
|----------|----------------------------------------------------------------------|
|
||||
| 可读 | 用户可以直接打开 workspace，像读普通笔记一样读 daily、digest 和原始材料。 |
|
||||
| 可编辑 | 用户和 Agent 都能用文件操作修正、补充、移动或删除记忆，不必依赖专用数据库客户端。 |
|
||||
| 可追溯 | digest 中的长期结论可以通过 `derived_from:: [[...]]` 回到 daily、resource 或 session 原文。 |
|
||||
| 可追溯 | digest 中的长期结论可以通过 Sources 章节回到 daily、resource 或 session 原文。 |
|
||||
| 可迁移 | workspace 是普通目录，Markdown、JSONL、YAML 和资源文件可以被备份、同步、版本管理或迁移到其他工具。 |
|
||||
| 可索引 | 文件虽然是普通文本，但 ReMe 会解析 frontmatter、chunk、wikilink，构建检索索引和文件图谱。 |
|
||||
| 可协作 | 人负责判断和修正，Agent 负责整理、链接和检索；二者看到和操作的是同一套文件。 |
|
||||
|
|
@ -124,9 +124,7 @@ tags: [新能源, 光伏]
|
|||
# 结论
|
||||
|
||||
光伏产业链可以拆成 [[digest/wiki/硅料.md]]、硅片、电池片和组件。
|
||||
|
||||
upstream:: [[digest/wiki/硅料.md]]
|
||||
[company:: [[digest/wiki/隆基绿能.md|隆基]]]
|
||||
主要生产商包括 [[digest/wiki/隆基绿能.md|隆基]]。
|
||||
```
|
||||
|
||||
### Frontmatter
|
||||
|
|
@ -156,15 +154,18 @@ confidence: observed
|
|||
|
||||
用户多次要求文档补充动机、边界和例子，但避免营销式表述。
|
||||
|
||||
derived_from:: [[daily/2026-06-20/session-a.md]]
|
||||
related:: [[digest/procedure/技术文档写作.md]]
|
||||
执行 [[digest/procedure/技术文档写作.md]] 时应用这个偏好。
|
||||
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-06-20/session-a.md]]
|
||||
```
|
||||
|
||||
这样做有三个好处：
|
||||
|
||||
1. `name` 和 `description` 可以在列表、召回结果和 Agent 判断中作为轻量摘要。
|
||||
2. 正文可以承载更完整的事实、条件、反例和来源。
|
||||
3. `derived_from::`、`related::` 这类 typed wikilink 可以被图谱解析，后续移动文件时也能被维护。
|
||||
3. 普通 Wikilink 可以被图谱解析，后续移动文件时也能被维护。
|
||||
|
||||
Frontmatter 适合放稳定、短小、结构化的字段；正文适合放需要人读的解释。不要把大段正文塞进 YAML 字段。
|
||||
|
||||
|
|
@ -173,10 +174,10 @@ Frontmatter 适合放稳定、短小、结构化的字段；正文适合放需
|
|||
Wikilink 用 `[[...]]` 表达文件之间的关系：
|
||||
|
||||
```text
|
||||
[[digest/wiki/光伏.md]]
|
||||
[[digest/wiki/光伏.md#产业链]]
|
||||
[[digest/wiki/光伏.md|光伏]]
|
||||
![[resource/2026-06-01/report.md]]
|
||||
[[daily/2026-06-20/session.md]]
|
||||
[[notes/example.md#L9]]
|
||||
[[notes/example.md#L9-L10]]
|
||||
[[notes/example.md#L9-L10,L15-L20]]
|
||||
```
|
||||
|
||||
ReMe 的 wikilink 是**字面路径语义**：
|
||||
|
|
@ -187,54 +188,57 @@ ReMe 的 wikilink 是**字面路径语义**：
|
|||
|
||||
它不会自动补 `.md`，不会按文件名搜索，也不会自动解析 folder note。推荐写完整的 workspace 相对路径，并带上扩展名。
|
||||
|
||||
`[label](../wiki/example.md)` 这类普通 Markdown 链接不会建立 `FileLink`，move 或 retarget 操作也不会改写它们。
|
||||
|
||||
`#L9`、`#L9-L10` 和 `#L9-L10,L15-L20` 这类锚点会作为普通 `target_anchor` 字符串保存在图谱中。图谱解析器
|
||||
不会校验行号锚点，因此 `#L0`、`#L10-L9`、`#L9,` 也会被保存。`read` 不会解析追加在 `path` 后的锚点；读取指定
|
||||
范围时需要分别传入从 1 开始、首尾均包含的 `start_line` 和 `end_line`，例如
|
||||
`read(path="digest/wiki/光伏.md", start_line=9, end_line=10)`。
|
||||
|
||||
Wikilink 的作用：
|
||||
|
||||
```text
|
||||
正文链接 -> 建立 FileLink
|
||||
predicate:: 链接 -> 建立带关系名的 FileLink
|
||||
move 文件 -> 默认改写入边中的 [[旧路径]]
|
||||
delete 文件 -> 返回仍存在的入边，提示清理引用
|
||||
search 命中 -> 可展开出入链，帮助理解上下文
|
||||
```
|
||||
|
||||
支持的关系写法：
|
||||
|
||||
```markdown
|
||||
industry:: [[digest/wiki/新能源.md]]
|
||||
[competitor:: [[digest/wiki/比亚迪.md]]]
|
||||
```
|
||||
|
||||
解析结果：
|
||||
|
||||
```text
|
||||
FileLink
|
||||
source_path = 当前文件
|
||||
target_path = digest/wiki/新能源.md
|
||||
predicate = industry
|
||||
target_path = notes/example.md
|
||||
target_anchor = L9-L10,L15-L20
|
||||
```
|
||||
|
||||
旧文档中的 `related:: [[path]]`、`- related:: [[path]]` 或
|
||||
`[related:: [[path]]]` 仍然可以读取。ReMe 会忽略外围文本，把内部 `[[path]]`
|
||||
作为普通链接建立索引。从曾存储 typed link 的版本升级后，应执行一次 `reme reindex`，
|
||||
用源文件重建不含旧关系字段的派生图索引。
|
||||
|
||||
### 来源和关系
|
||||
|
||||
ReMe 里最重要的两类链接是来源链接和概念关系链接。
|
||||
|
||||
来源链接说明“这条长期记忆从哪里来”：
|
||||
Sources 章节说明“这条长期记忆从哪里来”：
|
||||
|
||||
```markdown
|
||||
derived_from:: [[daily/2026-06-20/session-a.md]]
|
||||
derived_from:: [[resource/2026-06-20/report.pdf]]
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-06-20/session-a.md]]
|
||||
- [[resource/2026-06-20/report.pdf]]
|
||||
```
|
||||
|
||||
概念关系链接说明“这个节点和哪些长期记忆有关”：
|
||||
概念关系链接说明“这个节点和哪些长期记忆有关”，并自然织入正文：
|
||||
|
||||
```markdown
|
||||
related:: [[digest/wiki/光伏产业链.md]]
|
||||
depends_on:: [[digest/procedure/调研报告拆解流程.md]]
|
||||
contrasts_with:: [[digest/wiki/集中式逆变器.md]]
|
||||
这份分析扩展了 [[digest/wiki/光伏产业链.md]]，遵循
|
||||
[[digest/procedure/调研报告拆解流程.md]]，并与
|
||||
[[digest/wiki/集中式逆变器.md]] 对比。
|
||||
```
|
||||
|
||||
普通正文 wikilink 也会建立图边，但当关系本身有语义价值时，推荐使用 `predicate:: [[path]]`。这能让搜索、图遍历和后续 Agent
|
||||
整合更容易理解链接含义。
|
||||
|
||||
## 人工编辑和 Agent 编辑
|
||||
|
||||
因为记忆就是文件，用户可以直接在编辑器里改 workspace；Agent 也可以通过 ReMe 的文件工具读写同一批文件。两者遵守同一套约定：
|
||||
|
|
@ -247,8 +251,8 @@ contrasts_with:: [[digest/wiki/集中式逆变器.md]]
|
|||
| 删除文件 | 删除前检查入链；ReMe 的 delete 会返回仍然指向目标的来源文件，方便清理悬空引用。 |
|
||||
| 修改元数据 | 用 frontmatter 表达短字段；正文发生实质变化时同步更新 `description`。 |
|
||||
|
||||
一个实用规则是：**可以让 Agent 重写表达，但不要让它丢掉证据边**。尤其是 digest 节点中的 `derived_from:: [[...]]` 和已有
|
||||
digest-to-digest wikilink，是长期记忆可追溯和可扩展的基础。
|
||||
一个实用规则是：**可以让 Agent 重写表达，但不要让它丢掉证据边**。尤其是 digest 节点中的 Sources 条目和已有
|
||||
digest-to-digest Wikilink，是长期记忆可追溯和可扩展的基础。
|
||||
|
||||
## 路径语义
|
||||
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ index_update_loop:
|
|||
4. 对删除的文件，从 `file_store`、`keyword_index` 和 `file_graph` 清掉对应记录。
|
||||
5. 有变化时 dump 到 `metadata/`，让下次启动可以恢复。
|
||||
|
||||
Markdown chunker 会解析 YAML frontmatter、标题结构和 `[[...]]`，产出 `FileNode`、`FileChunk` 和 `FileLink`。更细的分块规则见
|
||||
Markdown chunker 会解析 YAML frontmatter、标题结构和 wikilink，产出 `FileNode`、`FileChunk` 和 `FileLink`。更细的分块规则见
|
||||
[Memory as File](./memory_as_file.md#memory-chunking)。
|
||||
|
||||
### 索引优化
|
||||
|
|
@ -172,7 +172,8 @@ delete，后续可通过 optimize 压缩索引。
|
|||
Memory Search 的“渐进式”不是一次把全库内容塞进结果，而是分三层展开：
|
||||
|
||||
1. 第一层是 chunk 召回：只返回最相关的 `limit` 个文本片段。
|
||||
2. 第二层是文件定位：每个结果带 `path:start_line-end_line`，可以继续用 `read` 精读原文件。
|
||||
2. 第二层是文件定位：每个结果带 `path:start_line-end_line`。调用 `read` 时需要把它们分别作为 `path`、
|
||||
`start_line` 和 `end_line` 传入，行号范围不是 `path` 的一部分。
|
||||
3. 第三层是链接邻居：对命中文件调用 `expand_links()`，展开最多 `max_links_per_direction` 个 outlinks 和 inlinks。
|
||||
|
||||
展开的数据来自 `file_graph`，不是重新扫文件：
|
||||
|
|
@ -183,7 +184,7 @@ Memory Search 的“渐进式”不是一次把全库内容塞进结果，而是
|
|||
-> file_store.get_outlinks(path)
|
||||
-> file_store.get_inlinks(path)
|
||||
-> file_store.get_nodes(neighbor_paths)
|
||||
-> 渲染邻居的 path、name、description、predicate、anchor
|
||||
-> 渲染邻居的 path、name、description、anchor
|
||||
```
|
||||
|
||||
这让搜索结果既保持短，又能看到“这条记忆连接到哪些长期节点、资源或其他 daily note”。如果某条结果值得继续追，可以用
|
||||
|
|
@ -203,10 +204,8 @@ Memory Search 的“渐进式”不是一次把全库内容塞进结果，而是
|
|||
...命中的记忆片段...
|
||||
outlinks (2):
|
||||
-> digest/indexing.md name="Indexing" description="..."
|
||||
via predicate=related
|
||||
inlinks (1):
|
||||
<- daily/2026-06-19.md name="..."
|
||||
via plain
|
||||
```
|
||||
|
||||
`counts` 会告诉你本次向量、关键词各召回了多少候选，以及最终返回多少条。默认 embedding 关闭时，`vector` 通常是 `0`，`hybrid` 是
|
||||
|
|
|
|||
|
|
@ -107,15 +107,17 @@ name: 钴
|
|||
description: 锂电正极材料关键原料，主产区集中于刚果(金)
|
||||
---
|
||||
|
||||
downstream_product:: [[digest/wiki/三元正极.md]]
|
||||
producer:: [[digest/wiki/嘉能可.md]]
|
||||
source_event:: [[daily/2026-05-18/2026-05-18-close.md]]
|
||||
|
||||
# 钴
|
||||
|
||||
用于 [[digest/wiki/三元正极.md]]；主要生产商包括 [[digest/wiki/嘉能可.md]]。
|
||||
|
||||
## 供给端
|
||||
嘉能可三季度钴产量同比下滑 18%，需要继续跟踪供给收缩对价格的影响。
|
||||
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-05-18/2026-05-18-close.md]]
|
||||
|
||||
## 政策风险
|
||||
刚果(金)矿权政策变化可能影响 KFM 矿运营，需联动跟踪洛阳钼业。
|
||||
```
|
||||
|
|
@ -179,10 +181,10 @@ reme search query="锂电 上下游 三元 正极 钴 宁德" limit=5
|
|||
嘉能可三季度钴产量同比下滑 18%...
|
||||
|
||||
outlinks:
|
||||
-> digest/wiki/三元正极.md name="三元正极" via predicate=downstream_product
|
||||
-> digest/wiki/嘉能可.md name="嘉能可" via predicate=producer
|
||||
-> digest/wiki/三元正极.md name="三元正极"
|
||||
-> digest/wiki/嘉能可.md name="嘉能可"
|
||||
inlinks:
|
||||
<- digest/wiki/三元正极.md name="三元正极" via predicate=upstream_material
|
||||
<- digest/wiki/三元正极.md name="三元正极"
|
||||
|
||||
========== digest/wiki/三元正极.md:5-18 [score=0.0139 keyword=3.2017] ==========
|
||||
...
|
||||
|
|
@ -291,11 +293,10 @@ name: TypeScript 项目构建 OOM 排查路径
|
|||
description: build 卡住且内存上涨时，优先检查类型检查进程内存
|
||||
---
|
||||
|
||||
source_event:: [[daily/2026-03-10/build-oom-2026-03-10.md]]
|
||||
related_preference:: [[digest/personal/code-style.md]]
|
||||
|
||||
# TypeScript 项目构建 OOM 排查路径
|
||||
|
||||
执行这个 runbook 时遵循 [[digest/personal/code-style.md]]。
|
||||
|
||||
## 症状
|
||||
构建卡在后段，CPU 不高但内存持续上涨。
|
||||
|
||||
|
|
@ -307,6 +308,10 @@ related_preference:: [[digest/personal/code-style.md]]
|
|||
## 已知无效路径
|
||||
- 单纯删除 `.cache` 未解决 2026-03-10 的问题。
|
||||
- 升级 terser 插件未解决 2026-03-10 的问题。
|
||||
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-03-10/build-oom-2026-03-10.md]]
|
||||
```
|
||||
|
||||
示例 `digest/personal/code-style.md`：
|
||||
|
|
@ -398,13 +403,14 @@ name: Alice
|
|||
description: 用户朋友，常推荐阅读材料
|
||||
---
|
||||
|
||||
recommended_book:: [[digest/wiki/deep-work.md]]
|
||||
source_event:: [[daily/2026-04-20/lunch-with-alice.md]]
|
||||
|
||||
# Alice
|
||||
|
||||
## 阅读推荐
|
||||
2026-04-20 午餐时推荐过一本关于注意力和深度工作的书。
|
||||
2026-04-20 午餐时推荐过 [[digest/wiki/deep-work.md]]，这是一本关于注意力和深度工作的书。
|
||||
|
||||
## Sources
|
||||
|
||||
- [[daily/2026-04-20/lunch-with-alice.md]]
|
||||
```
|
||||
|
||||
### 一次联想式回忆
|
||||
|
|
@ -426,7 +432,7 @@ reme search query="Alice 推荐 注意力 书 深度" limit=5
|
|||
```text
|
||||
digest/personal/alice.md
|
||||
outlinks:
|
||||
-> digest/wiki/deep-work.md via predicate=recommended_book
|
||||
-> digest/wiki/deep-work.md
|
||||
daily/2026-04-20/lunch-with-alice.md
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -42,12 +42,12 @@ class ComponentMixin:
|
|||
return Path(self.app_context.app_config.workspace_dir).absolute()
|
||||
|
||||
def to_workspace_relative(self, path: str | Path) -> str:
|
||||
"""Convert `path` to a workspace-relative string; return absolute path when outside."""
|
||||
"""Convert ``path`` to a POSIX workspace path; keep outside paths absolute."""
|
||||
abs_path = Path(path).absolute()
|
||||
try:
|
||||
return str(abs_path.relative_to(self.workspace_path))
|
||||
return abs_path.relative_to(self.workspace_path).as_posix()
|
||||
except ValueError:
|
||||
return str(abs_path)
|
||||
return abs_path.as_posix()
|
||||
|
||||
|
||||
class Dependency:
|
||||
|
|
|
|||
|
|
@ -73,7 +73,7 @@ class DefaultFileChunker(BaseFileChunker):
|
|||
)
|
||||
|
||||
def _link_byte_spans(self, content: str) -> list[tuple[int, int]]:
|
||||
"""Return [start, end) byte spans of every wikilink in content."""
|
||||
"""Return byte spans of wikilinks."""
|
||||
spans: list[tuple[int, int]] = []
|
||||
last_char, last_byte = 0, 0
|
||||
for wm in WikilinkHandler.iter_matches(content):
|
||||
|
|
|
|||
|
|
@ -12,8 +12,7 @@ and merge adjacent small subtrees at their parent. Leaf blocks (table / code /
|
|||
list / paragraph) split on internal boundaries and each piece is annotated
|
||||
``[Part X/N]``. Wikilink extraction is
|
||||
delegated to :class:`reme.utils.wikilink_handler.WikilinkHandler` —
|
||||
the single source of truth for ``[[...]]`` syntax (including
|
||||
Dataview-style typed predicates).
|
||||
the single source of truth for ``[[...]]`` syntax.
|
||||
"""
|
||||
|
||||
import re
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ Property-graph mapping:
|
|||
Virtual node: (:File {path}) — placeholder created when something
|
||||
links to a path that hasn't been upserted yet.
|
||||
|
||||
Edge: (:File)-[:LINKS {idx, anchor, predicate}]->(:File)
|
||||
Edge: (:File)-[:LINKS {idx, anchor}]->(:File)
|
||||
|
||||
The ``links_json`` property doubles as the "is real" marker — its
|
||||
presence means the node was upserted with a payload; its absence
|
||||
|
|
@ -43,7 +43,7 @@ from ...schema import FileLink, FileNode
|
|||
from ...schema.file_node import FileFrontMatter
|
||||
|
||||
_TYPED_FRONTMATTER_FIELDS = {"name", "description"}
|
||||
_LINK_FIELDS = {"source_path", "target_path", "target_anchor", "predicate"}
|
||||
_LINK_FIELDS = {"source_path", "target_path", "target_anchor"}
|
||||
|
||||
# Properties that distinguish a "real" node from a virtual placeholder.
|
||||
# Listed for the demote query (delete_nodes) so we can REMOVE them all.
|
||||
|
|
@ -169,7 +169,6 @@ class Neo4jFileGraph(BaseFileGraph):
|
|||
{
|
||||
"idx": i,
|
||||
"anchor": link.target_anchor,
|
||||
"predicate": link.predicate,
|
||||
"target": link.target_path,
|
||||
}
|
||||
for i, link in enumerate(node.links)
|
||||
|
|
@ -211,7 +210,7 @@ class Neo4jFileGraph(BaseFileGraph):
|
|||
UNWIND n.links AS link
|
||||
MERGE (t:File {path: link.target})
|
||||
MERGE (s)-[r:LINKS {idx: link.idx}]->(t)
|
||||
SET r.anchor = link.anchor, r.predicate = link.predicate
|
||||
SET r.anchor = link.anchor
|
||||
""",
|
||||
items=payload,
|
||||
)
|
||||
|
|
@ -313,7 +312,6 @@ class Neo4jFileGraph(BaseFileGraph):
|
|||
{
|
||||
"idx": i,
|
||||
"anchor": link.get("target_anchor"),
|
||||
"predicate": link.get("predicate"),
|
||||
"target": link.get("target_path"),
|
||||
}
|
||||
for i, link in enumerate(links)
|
||||
|
|
@ -340,7 +338,7 @@ class Neo4jFileGraph(BaseFileGraph):
|
|||
UNWIND n.links AS link
|
||||
MERGE (t:File {path: link.target})
|
||||
MERGE (s)-[r:LINKS {idx: link.idx}]->(t)
|
||||
SET r.anchor = link.anchor, r.predicate = link.predicate
|
||||
SET r.anchor = link.anchor
|
||||
""",
|
||||
items=payload,
|
||||
)
|
||||
|
|
@ -369,8 +367,7 @@ class Neo4jFileGraph(BaseFileGraph):
|
|||
WHERE s.links_json IS NOT NULL
|
||||
MATCH (s)-[r:LINKS]->(t:File)
|
||||
WHERE 1=1 {target_filter}
|
||||
RETURN t.path AS target, r.anchor AS anchor,
|
||||
r.predicate AS predicate, r.idx AS idx
|
||||
RETURN t.path AS target, r.anchor AS anchor, r.idx AS idx
|
||||
ORDER BY r.idx ASC
|
||||
""",
|
||||
path=path,
|
||||
|
|
@ -381,7 +378,6 @@ class Neo4jFileGraph(BaseFileGraph):
|
|||
source_path=path,
|
||||
target_path=row["target"],
|
||||
target_anchor=row.get("anchor"),
|
||||
predicate=row.get("predicate"),
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
|
|
@ -405,8 +401,7 @@ class Neo4jFileGraph(BaseFileGraph):
|
|||
WHERE 1=1 {target_filter}
|
||||
MATCH (s:File)-[r:LINKS]->(t)
|
||||
WHERE s.links_json IS NOT NULL
|
||||
RETURN r.anchor AS anchor, r.predicate AS predicate,
|
||||
r.idx AS idx, s.path AS source
|
||||
RETURN r.anchor AS anchor, r.idx AS idx, s.path AS source
|
||||
ORDER BY s.path ASC, r.idx ASC
|
||||
""",
|
||||
path=path,
|
||||
|
|
@ -417,7 +412,6 @@ class Neo4jFileGraph(BaseFileGraph):
|
|||
source_path=row["source"],
|
||||
target_path=path,
|
||||
target_anchor=row.get("anchor"),
|
||||
predicate=row.get("predicate"),
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
|
|
|
|||
|
|
@ -7,12 +7,14 @@ class FileLink(BaseModel):
|
|||
"""file link
|
||||
[[target_path]]
|
||||
[[target_path#target_anchor]]
|
||||
predicate:: [[target_*]]
|
||||
[predicate:: [[target_*]]]
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
source_path: str = Field(default=..., description="source file path relative to working dir")
|
||||
target_path: str = Field(default=..., description="target file path relative to working dir")
|
||||
target_anchor: str | None = Field(default=None, description="Heading or block anchor (text after '#')")
|
||||
predicate: str | None = Field(default=None, description="Dataview-style typed-link predicate")
|
||||
target_anchor: str | None = Field(default=None, description="Heading, block, or line anchor (text after '#')")
|
||||
predicate: str | None = Field(
|
||||
default=None,
|
||||
exclude=True,
|
||||
description="Deprecated compatibility field; accepted when loading legacy indexes but otherwise unused",
|
||||
)
|
||||
|
|
|
|||
|
|
@ -18,8 +18,8 @@ integrate_system_prompt_procedure: |
|
|||
genuinely needs more.
|
||||
- If the draft starts copying paragraphs or narrating the event, it belongs
|
||||
in the source material, not digest.
|
||||
- Provenance edges carry details: cite every relevant unit_paths entry with
|
||||
`derived_from:: [[<path>]]`.
|
||||
- The Sources section carries details: cite every relevant unit_paths entry
|
||||
as `- [[<path>]]`.
|
||||
- Digest-to-digest wikilinks carry the conceptual graph.
|
||||
|
||||
## Procedure Body Shape
|
||||
|
|
@ -30,9 +30,9 @@ integrate_system_prompt_procedure: |
|
|||
- Steps: numbered or terse bullets; each step is verb-led.
|
||||
- Pre-conditions / inputs: short list, not prose.
|
||||
- Failure modes / caveats: brief.
|
||||
- `derived_from:: [[<material-path>]]`: at least one, and normally every
|
||||
relevant path in unit_paths. Plain-prose provenance does NOT count; only
|
||||
wikilinks survive future updates.
|
||||
- `## Sources`: at least one `- [[<material-path>]]` item, and normally one
|
||||
for every relevant path in unit_paths. Plain-prose provenance does NOT
|
||||
count; only wikilinks survive future updates.
|
||||
|
||||
## Workflow
|
||||
|
||||
|
|
@ -50,7 +50,7 @@ integrate_system_prompt_procedure: |
|
|||
- unrelated: ignore.
|
||||
3. Choose exactly one action:
|
||||
- CREATE: no same_abstraction hit; write `{digest_dir}/procedure/<slug>.md`.
|
||||
- CORROBORATE: same procedure observed again; append derived_from and
|
||||
- CORROBORATE: same procedure observed again; append its source link and
|
||||
optionally strengthen wording.
|
||||
- REFINE: new pre-condition, edge case, failure mode, scope, or step;
|
||||
expand the relevant span or slot the step into the right position.
|
||||
|
|
@ -58,21 +58,18 @@ integrate_system_prompt_procedure: |
|
|||
tighten or annotate inline with `> note: contradicted by [[<path>]] - <one-line>`.
|
||||
4. Weave related digest nodes into the body as wikilinks on both CREATE and
|
||||
UPDATE. UPDATE must be additive: never remove existing wikilinks or
|
||||
derived_from entries. Default to weaving more, not less; this is the only
|
||||
source links. Default to weaving more, not less; this is the only
|
||||
chance to attach recalled related nodes.
|
||||
|
||||
## Wikilink Graph
|
||||
|
||||
- Source provenance links point from digest back to material:
|
||||
`derived_from:: [[daily/<date>/<session>.md]]` or
|
||||
`derived_from:: [[resource/<path>]]`.
|
||||
- A `## Sources` section points from digest back to material with bare list
|
||||
items such as `- [[daily/<date>/<session>.md]]` and
|
||||
`- [[resource/<path>]]`.
|
||||
- Procedure nodes may link to any digest bucket:
|
||||
`[[{digest_dir}/procedure/<slug>.md]]`,
|
||||
`[[{digest_dir}/personal/<slug>.md]]`, or
|
||||
`[[{digest_dir}/wiki/<slug>.md]]`.
|
||||
- Useful predicates include `derived_from::`, `relates_to::`,
|
||||
`depends_on::`, and `blocks_on::`; predicates are open and live outside
|
||||
the brackets.
|
||||
- Wikilinks must be full workspace-relative paths with `.md`.
|
||||
- CREATE writes inside `{digest_dir}/procedure/`. UPDATE may target any
|
||||
bucket if recall legitimately found the same abstraction.
|
||||
|
|
@ -101,7 +98,7 @@ integrate_system_prompt_procedure_zh: |
|
|||
|
||||
- 正文短且抽象，通常 50-200 words；只有流程本身确实需要时才更长。
|
||||
- 如果草稿开始复制段落或叙述事件，说明细节放错层了。
|
||||
- Provenance edge 承载细节：用 `derived_from:: [[<path>]]` 引用 unit_paths 中每个相关来源。
|
||||
- Sources 章节承载细节：用 `- [[<path>]]` 引用 unit_paths 中每个相关来源。
|
||||
- Digest 之间的 wikilink 承载概念图。
|
||||
|
||||
## Procedure 正文形态
|
||||
|
|
@ -112,7 +109,7 @@ integrate_system_prompt_procedure_zh: |
|
|||
- Steps：编号或短 bullet；每步以动词开头。
|
||||
- Pre-conditions / inputs：短列表，不写长 prose。
|
||||
- Failure modes / caveats：简短。
|
||||
- `derived_from:: [[<material-path>]]`：至少一条，通常覆盖 unit_paths 中每个相关 path。
|
||||
- `## Sources`：至少一条 `- [[<material-path>]]`，通常覆盖 unit_paths 中每个相关 path。
|
||||
纯文本 provenance 不算；只有 wikilink 会在未来更新中保留下来。
|
||||
|
||||
## 工作流
|
||||
|
|
@ -126,22 +123,20 @@ integrate_system_prompt_procedure_zh: |
|
|||
- unrelated：忽略。
|
||||
3. 选择且只选择一个 action：
|
||||
- CREATE：没有 same_abstraction 命中；写入 `{digest_dir}/procedure/<slug>.md`。
|
||||
- CORROBORATE：同一流程再次出现；追加 derived_from，可选强化措辞。
|
||||
- CORROBORATE：同一流程再次出现；追加来源链接，可选强化措辞。
|
||||
- REFINE：新增前置条件、边界情况、失败模式、适用范围或步骤；扩展相关段落或把步骤插到正确位置。
|
||||
- CORRECT：顺序错误、缺关键步骤、结果不好或有冲突；收紧表述，或用
|
||||
`> note: contradicted by [[<path>]] - <one-line>` 内联标注。
|
||||
4. CREATE 和 UPDATE 都要把 related digest 节点织入正文 wikilink。UPDATE 必须只增不删：不要删除已有
|
||||
wikilink 或 derived_from。默认多织，而不是少织；这是挂接召回到的相关节点的唯一机会。
|
||||
wikilink 或来源链接。默认多织，而不是少织；这是挂接召回到的相关节点的唯一机会。
|
||||
|
||||
## Wikilink 图
|
||||
|
||||
- 来源 provenance 从 digest 指回材料：
|
||||
`derived_from:: [[daily/<date>/<session>.md]]` 或 `derived_from:: [[resource/<path>]]`。
|
||||
- `## Sources` 章节用普通列表 Wikilink 从 digest 指回材料，例如
|
||||
`- [[daily/<date>/<session>.md]]` 和 `- [[resource/<path>]]`。
|
||||
- Procedure 节点可以链接任意 digest bucket：
|
||||
`[[{digest_dir}/procedure/<slug>.md]]`、`[[{digest_dir}/personal/<slug>.md]]`、
|
||||
`[[{digest_dir}/wiki/<slug>.md]]`。
|
||||
- 常用 predicate：`derived_from::`、`relates_to::`、`depends_on::`、`blocks_on::`；
|
||||
predicate 词表开放，写在方括号外。
|
||||
- Wikilink 必须是带 `.md` 的 workspace-relative path。
|
||||
- CREATE 写入 `{digest_dir}/procedure/`。如果 recall 合法命中同一抽象，UPDATE 可以跨 bucket。
|
||||
- `edit` 只改正文，并尽量只增不删：不要删除 old span 中已有的 wikilink。
|
||||
|
|
@ -171,8 +166,8 @@ integrate_system_prompt_personal: |
|
|||
|
||||
- Body is short and operational, usually 50-200 words.
|
||||
- Do not narrate what the user said in detail; cite source material instead.
|
||||
- Provenance edges carry details: cite every relevant unit_paths entry with
|
||||
`derived_from:: [[<path>]]`.
|
||||
- The Sources section carries details: cite every relevant unit_paths entry
|
||||
as `- [[<path>]]`.
|
||||
- Digest-to-digest wikilinks carry the conceptual graph.
|
||||
|
||||
## Personal Body Shape
|
||||
|
|
@ -185,9 +180,9 @@ integrate_system_prompt_personal: |
|
|||
- `How to apply:` contexts, tasks, boundaries, or exceptions.
|
||||
- Do not invent exceptions or soften a hard preference unless the source
|
||||
material explicitly supports that exception.
|
||||
- `derived_from:: [[<material-path>]]`: at least one, and normally every
|
||||
relevant path in unit_paths. Plain-prose provenance does NOT count; only
|
||||
wikilinks survive future updates.
|
||||
- `## Sources`: at least one `- [[<material-path>]]` item, and normally one
|
||||
for every relevant path in unit_paths. Plain-prose provenance does NOT
|
||||
count; only wikilinks survive future updates.
|
||||
|
||||
For preferences, prefer one node per preference rather than one large person
|
||||
node; that is the granularity downstream search will hit.
|
||||
|
|
@ -208,7 +203,7 @@ integrate_system_prompt_personal: |
|
|||
- unrelated: ignore.
|
||||
3. Choose exactly one action:
|
||||
- CREATE: no same_abstraction hit; write `{digest_dir}/personal/<slug>.md`.
|
||||
- CORROBORATE: the rule is reaffirmed; append derived_from and optionally
|
||||
- CORROBORATE: the rule is reaffirmed; append its source link and optionally
|
||||
strengthen confidence.
|
||||
- REFINE: scope, condition, exception, or example changed; expand `How to apply:`.
|
||||
- CORRECT: the user/team changed their mind or evidence conflicts; tighten
|
||||
|
|
@ -216,21 +211,18 @@ integrate_system_prompt_personal: |
|
|||
`> note: contradicted by [[<path>]] - <one-line>`.
|
||||
4. Weave related digest nodes into the body as wikilinks on both CREATE and
|
||||
UPDATE. UPDATE must be additive: never remove existing wikilinks or
|
||||
derived_from entries. Default to weaving more, not less; this is the only
|
||||
source links. Default to weaving more, not less; this is the only
|
||||
chance to attach recalled related nodes.
|
||||
|
||||
## Wikilink Graph
|
||||
|
||||
- Source provenance links point from digest back to material:
|
||||
`derived_from:: [[daily/<date>/<session>.md]]` or
|
||||
`derived_from:: [[resource/<path>]]`.
|
||||
- A `## Sources` section points from digest back to material with bare list
|
||||
items such as `- [[daily/<date>/<session>.md]]` and
|
||||
`- [[resource/<path>]]`.
|
||||
- Personal nodes may link to any digest bucket:
|
||||
`[[{digest_dir}/personal/<slug>.md]]`,
|
||||
`[[{digest_dir}/procedure/<slug>.md]]`, or
|
||||
`[[{digest_dir}/wiki/<slug>.md]]`.
|
||||
- Useful predicates include `derived_from::`, `applies_to::`,
|
||||
`relates_to::`, `depends_on::`, and `contradicts::`; predicates are open
|
||||
and live outside the brackets.
|
||||
- Wikilinks must be full workspace-relative paths with `.md`.
|
||||
- CREATE writes inside `{digest_dir}/personal/`. UPDATE may target any bucket
|
||||
if recall legitimately found the same abstraction.
|
||||
|
|
@ -260,7 +252,7 @@ integrate_system_prompt_personal_zh: |
|
|||
|
||||
- 正文短且可操作，通常 50-200 words。
|
||||
- 不要详细复述用户说了什么；用来源材料承载细节。
|
||||
- Provenance edge 承载细节：用 `derived_from:: [[<path>]]` 引用 unit_paths 中每个相关来源。
|
||||
- Sources 章节承载细节：用 `- [[<path>]]` 引用 unit_paths 中每个相关来源。
|
||||
- Digest 之间的 wikilink 承载概念图。
|
||||
|
||||
## Personal 正文形态
|
||||
|
|
@ -271,7 +263,7 @@ integrate_system_prompt_personal_zh: |
|
|||
- `Why:` 原因或上下文，帮助未来判断边界情况。
|
||||
- `How to apply:` 适用上下文、任务、边界或例外。
|
||||
- 不要凭空添加例外，也不要软化明确偏好；只有来源材料明确支持时才写例外。
|
||||
- `derived_from:: [[<material-path>]]`：至少一条，通常覆盖 unit_paths 中每个相关 path。
|
||||
- `## Sources`：至少一条 `- [[<material-path>]]`，通常覆盖 unit_paths 中每个相关 path。
|
||||
纯文本 provenance 不算；只有 wikilink 会在未来更新中保留下来。
|
||||
|
||||
偏好类内容优先一条偏好一个 node，而不是一个人一个大 node；这是下游搜索更容易命中的粒度。
|
||||
|
|
@ -287,22 +279,20 @@ integrate_system_prompt_personal_zh: |
|
|||
- unrelated：忽略。
|
||||
3. 选择且只选择一个 action：
|
||||
- CREATE：没有 same_abstraction 命中；写入 `{digest_dir}/personal/<slug>.md`。
|
||||
- CORROBORATE：规则被再次确认；追加 derived_from，可选强化置信度。
|
||||
- CORROBORATE：规则被再次确认；追加来源链接，可选强化置信度。
|
||||
- REFINE：scope、条件、例外或例子变化；扩展 `How to apply:`。
|
||||
- CORRECT：用户/团队改变主意或证据冲突；收紧到新旧证据都支持的表述，或用
|
||||
`> note: contradicted by [[<path>]] - <one-line>` 内联标注。
|
||||
4. CREATE 和 UPDATE 都要把 related digest 节点织入正文 wikilink。UPDATE 必须只增不删：不要删除已有
|
||||
wikilink 或 derived_from。默认多织，而不是少织；这是挂接召回到的相关节点的唯一机会。
|
||||
wikilink 或来源链接。默认多织，而不是少织；这是挂接召回到的相关节点的唯一机会。
|
||||
|
||||
## Wikilink 图
|
||||
|
||||
- 来源 provenance 从 digest 指回材料：
|
||||
`derived_from:: [[daily/<date>/<session>.md]]` 或 `derived_from:: [[resource/<path>]]`。
|
||||
- `## Sources` 章节用普通列表 Wikilink 从 digest 指回材料，例如
|
||||
`- [[daily/<date>/<session>.md]]` 和 `- [[resource/<path>]]`。
|
||||
- Personal 节点可以链接任意 digest bucket：
|
||||
`[[{digest_dir}/personal/<slug>.md]]`、`[[{digest_dir}/procedure/<slug>.md]]`、
|
||||
`[[{digest_dir}/wiki/<slug>.md]]`。
|
||||
- 常用 predicate：`derived_from::`、`applies_to::`、`relates_to::`、`depends_on::`、
|
||||
`contradicts::`；predicate 词表开放，写在方括号外。
|
||||
- Wikilink 必须是带 `.md` 的 workspace-relative path。
|
||||
- CREATE 写入 `{digest_dir}/personal/`。如果 recall 合法命中同一抽象，UPDATE 可以跨 bucket。
|
||||
- `edit` 只改正文，并尽量只增不删：不要删除 old span 中已有的 wikilink。
|
||||
|
|
@ -335,8 +325,8 @@ integrate_system_prompt_wiki: |
|
|||
genuinely needs more.
|
||||
- If the draft starts copying paragraphs or narrating the event, it belongs
|
||||
in the source material, not digest.
|
||||
- Provenance edges carry details: cite every relevant unit_paths entry with
|
||||
`derived_from:: [[<path>]]`.
|
||||
- The Sources section carries details: cite every relevant unit_paths entry
|
||||
as `- [[<path>]]`.
|
||||
- Digest-to-digest wikilinks carry the conceptual graph.
|
||||
|
||||
## Wiki Body Shape
|
||||
|
|
@ -346,11 +336,10 @@ integrate_system_prompt_wiki: |
|
|||
- First line: one-sentence definition or claim.
|
||||
- Body: short paragraphs or tight bullets with properties, sub-claims,
|
||||
distinctions, and one-line examples.
|
||||
- Relations: typed wikilinks where the relation has semantic weight; most
|
||||
cross-node links can stay bare.
|
||||
- `derived_from:: [[<material-path>]]`: at least one, and normally every
|
||||
relevant path in unit_paths. Plain-prose provenance does NOT count; only
|
||||
wikilinks survive future updates.
|
||||
- Relations: weave related nodes into natural prose with ordinary wikilinks.
|
||||
- `## Sources`: at least one `- [[<material-path>]]` item, and normally one
|
||||
for every relevant path in unit_paths. Plain-prose provenance does NOT
|
||||
count; only wikilinks survive future updates.
|
||||
|
||||
## Workflow
|
||||
|
||||
|
|
@ -368,8 +357,8 @@ integrate_system_prompt_wiki: |
|
|||
- unrelated: ignore.
|
||||
3. Choose exactly one action:
|
||||
- CREATE: no same_abstraction hit; write `{digest_dir}/wiki/<slug>.md`.
|
||||
- CORROBORATE: the principle is reaffirmed by a new instance; append
|
||||
derived_from and optionally strengthen wording.
|
||||
- CORROBORATE: the principle is reaffirmed by a new instance; append its
|
||||
source link and optionally strengthen wording.
|
||||
- REFINE: nuance, scope, edge case, or framing changes; tighten the
|
||||
relevant span. Body grows in precision, not detail volume.
|
||||
- CORRECT: factual contradiction or overstatement; tighten to the narrower
|
||||
|
|
@ -377,22 +366,18 @@ integrate_system_prompt_wiki: |
|
|||
`> note: contradicted by [[<path>]] - <one-line>`.
|
||||
4. Weave related digest nodes into the body as wikilinks on both CREATE and
|
||||
UPDATE. UPDATE must be additive: never remove existing wikilinks or
|
||||
derived_from entries. Default to weaving more, not less; this is the only
|
||||
source links. Default to weaving more, not less; this is the only
|
||||
chance to attach recalled related nodes.
|
||||
|
||||
## Wikilink Graph
|
||||
|
||||
- Source provenance links point from digest back to material:
|
||||
`derived_from:: [[daily/<date>/<session>.md]]` or
|
||||
`derived_from:: [[resource/<path>]]`.
|
||||
- A `## Sources` section points from digest back to material with bare list
|
||||
items such as `- [[daily/<date>/<session>.md]]` and
|
||||
`- [[resource/<path>]]`.
|
||||
- Wiki nodes may link to any digest bucket:
|
||||
`[[{digest_dir}/wiki/<slug>.md]]`,
|
||||
`[[{digest_dir}/procedure/<slug>.md]]`, or
|
||||
`[[{digest_dir}/personal/<slug>.md]]`.
|
||||
- Useful predicates include `derived_from::`, `is_a::`, `extends::`,
|
||||
`depends_on::`, `relates_to::`, and `contradicts::`; predicates are open
|
||||
and live outside the brackets. Most cross-node links can stay bare unless
|
||||
the relation has clear semantic weight.
|
||||
- Wikilinks must be full workspace-relative paths with `.md`.
|
||||
- CREATE writes inside `{digest_dir}/wiki/`. UPDATE may target any bucket if
|
||||
recall legitimately found the same abstraction.
|
||||
|
|
@ -422,7 +407,7 @@ integrate_system_prompt_wiki_zh: |
|
|||
|
||||
- 正文短且抽象，通常 50-200 words；只有概念本身确实需要时才更长。
|
||||
- 如果草稿开始复制段落或叙述事件，说明细节放错层了。
|
||||
- Provenance edge 承载细节：用 `derived_from:: [[<path>]]` 引用 unit_paths 中每个相关来源。
|
||||
- Sources 章节承载细节：用 `- [[<path>]]` 引用 unit_paths 中每个相关来源。
|
||||
- Digest 之间的 wikilink 承载概念图。
|
||||
|
||||
## Wiki 正文形态
|
||||
|
|
@ -431,8 +416,8 @@ integrate_system_prompt_wiki_zh: |
|
|||
|
||||
- First line：一句话定义或主张。
|
||||
- Body：短段落或紧凑 bullets，写属性、子主张、区分和一行例子。
|
||||
- Relations：有明确语义重量时使用 typed wikilink；大多数 cross-node link 可以裸写。
|
||||
- `derived_from:: [[<material-path>]]`：至少一条，通常覆盖 unit_paths 中每个相关 path。
|
||||
- Relations：在自然语言中用普通 Wikilink 织入相关节点。
|
||||
- `## Sources`：至少一条 `- [[<material-path>]]`，通常覆盖 unit_paths 中每个相关 path。
|
||||
纯文本 provenance 不算；只有 wikilink 会在未来更新中保留下来。
|
||||
|
||||
## 工作流
|
||||
|
|
@ -446,23 +431,20 @@ integrate_system_prompt_wiki_zh: |
|
|||
- unrelated：忽略。
|
||||
3. 选择且只选择一个 action：
|
||||
- CREATE：没有 same_abstraction 命中；写入 `{digest_dir}/wiki/<slug>.md`。
|
||||
- CORROBORATE：原则被新实例再次确认；追加 derived_from，可选强化措辞。
|
||||
- CORROBORATE：原则被新实例再次确认；追加来源链接，可选强化措辞。
|
||||
- REFINE：nuance、scope、edge case 或 framing 改变；收紧相关段落。正文增长的是精度，不是细节量。
|
||||
- CORRECT：事实冲突或过度概括；收紧到更窄且有支持的表述，或用
|
||||
`> note: contradicted by [[<path>]] - <one-line>` 内联标注。
|
||||
4. CREATE 和 UPDATE 都要把 related digest 节点织入正文 wikilink。UPDATE 必须只增不删：不要删除已有
|
||||
wikilink 或 derived_from。默认多织，而不是少织；这是挂接召回到的相关节点的唯一机会。
|
||||
wikilink 或来源链接。默认多织，而不是少织；这是挂接召回到的相关节点的唯一机会。
|
||||
|
||||
## Wikilink 图
|
||||
|
||||
- 来源 provenance 从 digest 指回材料：
|
||||
`derived_from:: [[daily/<date>/<session>.md]]` 或 `derived_from:: [[resource/<path>]]`。
|
||||
- `## Sources` 章节用普通列表 Wikilink 从 digest 指回材料，例如
|
||||
`- [[daily/<date>/<session>.md]]` 和 `- [[resource/<path>]]`。
|
||||
- Wiki 节点可以链接任意 digest bucket：
|
||||
`[[{digest_dir}/wiki/<slug>.md]]`、`[[{digest_dir}/procedure/<slug>.md]]`、
|
||||
`[[{digest_dir}/personal/<slug>.md]]`。
|
||||
- 常用 predicate：`derived_from::`、`is_a::`、`extends::`、`depends_on::`、
|
||||
`relates_to::`、`contradicts::`；predicate 词表开放，写在方括号外。有明确语义重量时才加
|
||||
predicate，大多数 cross-node link 可以裸写。
|
||||
- Wikilink 必须是带 `.md` 的 workspace-relative path。
|
||||
- CREATE 写入 `{digest_dir}/wiki/`。如果 recall 合法命中同一抽象，UPDATE 可以跨 bucket。
|
||||
- `edit` 只改正文，并尽量只增不删：不要删除 old span 中已有的 wikilink。
|
||||
|
|
@ -489,8 +471,8 @@ integrate_user_message: |
|
|||
{material_blob}
|
||||
|
||||
Integrate this single unit into digest memory. Cite every relevant unit_paths
|
||||
entry with `derived_from:: [[<path>]]`, recall related digest nodes, and weave
|
||||
useful digest wikilinks into the target node.
|
||||
entry as `- [[<path>]]` under `## Sources`, recall related digest nodes, and
|
||||
weave useful digest wikilinks into the target node.
|
||||
|
||||
integrate_user_message_zh: |
|
||||
提示：{hint}
|
||||
|
|
@ -506,5 +488,5 @@ integrate_user_message_zh: |
|
|||
|
||||
{material_blob}
|
||||
|
||||
将这个单独的 unit 整合进 digest memory。用 `derived_from:: [[<path>]]` 引用 unit_paths 中每个相关来源，
|
||||
召回相关 digest 节点，并把有用的 digest wikilink 织入目标节点。
|
||||
将这个单独的 unit 整合进 digest memory。在 `## Sources` 下用 `- [[<path>]]` 引用 unit_paths 中每个
|
||||
相关来源，召回相关 digest 节点，并把有用的 digest wikilink 织入目标节点。
|
||||
|
|
|
|||
|
|
@ -71,10 +71,10 @@ def scan_day_files(workspace: Path, day: str, daily: str, interests_name: str =
|
|||
out: list[str] = []
|
||||
day_index = workspace / daily / f"{day}.md"
|
||||
if day_index.is_file():
|
||||
out.append(str(day_index.relative_to(workspace)))
|
||||
out.append(day_index.relative_to(workspace).as_posix())
|
||||
daily_root = workspace / daily / day
|
||||
if daily_root.is_dir():
|
||||
out.extend(str(p.relative_to(workspace)) for p in sorted(daily_root.rglob("*.md")) if p.is_file())
|
||||
out.extend(p.relative_to(workspace).as_posix() for p in sorted(daily_root.rglob("*.md")) if p.is_file())
|
||||
return [p for p in out if p != f"{daily}/{day}/{interests_name}"]
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -65,6 +65,7 @@ class MoveStep(BaseStep):
|
|||
)
|
||||
self.context.response.metadata.update(payload)
|
||||
|
||||
# pylint: disable=too-many-return-statements
|
||||
async def _move(self, src_path: str, dst_path: str, overwrite: bool, retarget: bool) -> dict:
|
||||
workspace_dir = Path(self.file_store.workspace_path or ".").resolve()
|
||||
src_abs, src_err = resolve_path(workspace_dir, src_path) if src_path else (None, "src_path is required")
|
||||
|
|
@ -79,7 +80,8 @@ class MoveStep(BaseStep):
|
|||
assert src_abs is not None and dst_abs is not None # narrowed by precheck
|
||||
dst_abs.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Step 1 — copy. Both files exist; inbound [[src_path]] still resolves.
|
||||
# Step 1 — copy. Wikilinks use workspace-relative paths, so outgoing
|
||||
# targets do not change when their containing document moves.
|
||||
shutil.copyfile(str(src_abs), str(dst_abs))
|
||||
payload: dict = {"src_path": src_path, "dst_path": dst_path, "size": dst_abs.stat().st_size}
|
||||
|
||||
|
|
@ -91,10 +93,13 @@ class MoveStep(BaseStep):
|
|||
try:
|
||||
report = await WikilinkHandler.retarget_links(self.file_store, src=src_path, dst=dst_path)
|
||||
except Exception as exc:
|
||||
payload["retarget"] = {"error": f"retarget raised: {exc!r}"}
|
||||
error = f"retarget raised: {exc!r}"
|
||||
payload["error"] = error
|
||||
payload["retarget"] = {"error": error}
|
||||
payload["src_removed"] = False
|
||||
return payload
|
||||
if "error" in report:
|
||||
payload["error"] = report["error"]
|
||||
payload["retarget"] = report
|
||||
payload["src_removed"] = False
|
||||
return payload
|
||||
|
|
|
|||
|
|
@ -168,7 +168,7 @@ class ReadStep(BaseStep):
|
|||
"""Append the rendered neighbor block + stash raw expansion in metadata."""
|
||||
assert self.context is not None
|
||||
try:
|
||||
rel_path = str(target.relative_to(self.workspace_path))
|
||||
rel_path = target.relative_to(self.workspace_path.resolve()).as_posix()
|
||||
except ValueError:
|
||||
self.logger.info(f"[{self.name}] skip neighbors: path outside workspace_path path={target}")
|
||||
return
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
"""BFS over wikilink edges from one or more seed files.
|
||||
|
||||
One record per traversed *edge* (not per node): the same target can repeat
|
||||
if reached via different predicates or paths. Each record carries the
|
||||
predecessor plus the link's predicate/anchor so callers can reconstruct
|
||||
the path. Adjacency is built once via a single ``file_store.get_nodes()``
|
||||
call — BFS then runs purely in memory with no per-frontier round-trips.
|
||||
if reached via different anchors or paths. Each record carries the
|
||||
predecessor and anchor so callers can reconstruct the path. Adjacency is
|
||||
built once via a single ``file_store.get_nodes()`` call — BFS then runs
|
||||
purely in memory with no per-frontier round-trips.
|
||||
"""
|
||||
|
||||
from collections import deque
|
||||
|
|
@ -46,7 +46,7 @@ def _bfs(
|
|||
outbound: Adjacency,
|
||||
inbound: Adjacency,
|
||||
) -> list[dict]:
|
||||
"""In-memory BFS; emits one record per unique (src, dst, predicate) edge."""
|
||||
"""In-memory BFS; emits one record per unique (src, dst, anchor) edge."""
|
||||
sources: list[Adjacency] = []
|
||||
if direction in _OUT:
|
||||
sources.append(outbound)
|
||||
|
|
@ -63,7 +63,7 @@ def _bfs(
|
|||
continue
|
||||
for src in sources:
|
||||
for next_path, link in src.get(current, ()):
|
||||
key = (current, next_path, link.predicate)
|
||||
key = (current, next_path, link.target_anchor)
|
||||
if key in visited:
|
||||
continue
|
||||
visited.add(key)
|
||||
|
|
@ -72,7 +72,6 @@ def _bfs(
|
|||
"path": next_path,
|
||||
"depth": depth + 1,
|
||||
"via": current,
|
||||
"predicate": link.predicate,
|
||||
"anchor": link.target_anchor,
|
||||
},
|
||||
)
|
||||
|
|
@ -119,8 +118,7 @@ class TraverseStep(BaseStep):
|
|||
target = r["path"]
|
||||
if r["anchor"]:
|
||||
target = f"{target}#{r['anchor']}"
|
||||
predicate = r["predicate"] or "-"
|
||||
lines.append(f"[depth={r['depth']}] {r['via']} --{predicate}--> {target}")
|
||||
lines.append(f"[depth={r['depth']}] {r['via']} --> {target}")
|
||||
answer = "\n".join(lines)
|
||||
|
||||
self.context.response.success = True
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ from .common_utils import (
|
|||
)
|
||||
from .env_utils import load_env, parse_env_file
|
||||
from .link_expansion import expand_links, render_expansion_lines
|
||||
from .line_anchor import format_line_anchor, parse_line_anchor
|
||||
from .logger_utils import get_logger
|
||||
from .logo_utils import print_logo
|
||||
from .service_utils import find_reme, locate_reme, precheck_start, cli_find_reme, running_service_config
|
||||
|
|
@ -33,6 +34,8 @@ __all__ = [
|
|||
"parse_env_file",
|
||||
"expand_links",
|
||||
"render_expansion_lines",
|
||||
"format_line_anchor",
|
||||
"parse_line_anchor",
|
||||
"get_logger",
|
||||
"print_logo",
|
||||
"find_reme",
|
||||
|
|
|
|||
43
reme/utils/line_anchor.py
Normal file
43
reme/utils/line_anchor.py
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
"""Parse and format GitHub-style 1-based line-anchor strings."""
|
||||
|
||||
import re
|
||||
|
||||
_LINE_ANCHOR_RE = re.compile(r"L[0-9]+(?:-L[0-9]+)?(?:,L[0-9]+(?:-L[0-9]+)?)*")
|
||||
|
||||
|
||||
def parse_line_anchor(anchor: str | None) -> list[tuple[int, int]] | None:
|
||||
"""Return normalized inclusive ranges, or ``None`` for a non-line anchor.
|
||||
|
||||
Supported forms are ``L9``, ``L9-L10`` and
|
||||
``L9-L10,L15-L20``. Overlapping and adjacent ranges are merged.
|
||||
Anchors beginning with ``L<digit>`` are treated as line anchors and raise
|
||||
``ValueError`` when malformed, zero-based, or reversed.
|
||||
"""
|
||||
if not anchor or not re.match(r"L[0-9]", anchor):
|
||||
return None
|
||||
if not _LINE_ANCHOR_RE.fullmatch(anchor):
|
||||
raise ValueError(f"invalid line anchor: #{anchor}")
|
||||
|
||||
ranges: list[tuple[int, int]] = []
|
||||
for item in anchor.split(","):
|
||||
start_text, separator, end_text = item.partition("-L")
|
||||
start = int(start_text[1:])
|
||||
end = int(end_text) if separator else start
|
||||
if start < 1 or end < 1:
|
||||
raise ValueError("line numbers must be at least 1")
|
||||
if start > end:
|
||||
raise ValueError(f"line range start ({start}) exceeds end ({end})")
|
||||
ranges.append((start, end))
|
||||
|
||||
merged: list[tuple[int, int]] = []
|
||||
for start, end in sorted(ranges):
|
||||
if merged and start <= merged[-1][1] + 1:
|
||||
merged[-1] = (merged[-1][0], max(merged[-1][1], end))
|
||||
else:
|
||||
merged.append((start, end))
|
||||
return merged
|
||||
|
||||
|
||||
def format_line_anchor(ranges: list[tuple[int, int]]) -> str:
|
||||
"""Render normalized ranges without the leading ``#``."""
|
||||
return ",".join(f"L{start}" if start == end else f"L{start}-L{end}" for start, end in ranges)
|
||||
|
|
@ -8,7 +8,7 @@ Two-layer split so callers can pick what they need:
|
|||
|
||||
* :func:`expand_links` — data layer. Returns a structured dict keyed
|
||||
by source path, each value carrying its outlinks / inlinks with
|
||||
neighbor meta and per-edge predicate/anchor.
|
||||
neighbor metadata and link anchors.
|
||||
* :func:`render_expansion_lines` — view layer. Turns one path's
|
||||
expansion sub-dict into the same `` → path name=… description=…``
|
||||
block ``SearchStep`` has historically printed.
|
||||
|
|
@ -19,16 +19,16 @@ import asyncio
|
|||
from ..schema import FileLink, FileNode
|
||||
|
||||
|
||||
def _group_by_neighbor(links: list[FileLink], key_attr: str) -> dict[str, list[dict]]:
|
||||
"""Group edges by neighbor path (insertion-ordered), each value a list of {predicate, anchor}."""
|
||||
out: dict[str, list[dict]] = {}
|
||||
def _group_by_neighbor(links: list[FileLink], key_attr: str) -> dict[str, list[str]]:
|
||||
"""Group anchors by neighbor path while preserving insertion order."""
|
||||
out: dict[str, list[str]] = {}
|
||||
for lnk in links:
|
||||
neighbor = getattr(lnk, key_attr)
|
||||
if not neighbor:
|
||||
continue
|
||||
out.setdefault(neighbor, []).append(
|
||||
{"predicate": lnk.predicate, "anchor": lnk.target_anchor},
|
||||
)
|
||||
anchors = out.setdefault(neighbor, [])
|
||||
if lnk.target_anchor:
|
||||
anchors.append(lnk.target_anchor)
|
||||
return out
|
||||
|
||||
|
||||
|
|
@ -55,16 +55,6 @@ def _format_meta_inline(meta: dict) -> str:
|
|||
return " ".join(parts) if parts else "(no meta)"
|
||||
|
||||
|
||||
def _format_via(edge: dict) -> str:
|
||||
"""Render a single (predicate, anchor) edge as a 'via ...' descriptor."""
|
||||
bits = []
|
||||
if edge.get("predicate"):
|
||||
bits.append(f"predicate={edge['predicate']}")
|
||||
if edge.get("anchor"):
|
||||
bits.append(f"anchor=#{edge['anchor']}")
|
||||
return ", ".join(bits) if bits else "plain"
|
||||
|
||||
|
||||
async def expand_links(
|
||||
file_store,
|
||||
paths: list[str],
|
||||
|
|
@ -73,7 +63,7 @@ async def expand_links(
|
|||
"""Fetch out/in links for each path and attach neighbor meta.
|
||||
|
||||
Returns ``{path: {"outlinks": [...], "inlinks": [...]}, ...}`` where
|
||||
each list item is ``{"path": str, "meta": {...}, "edges": [{"predicate", "anchor"}, ...]}``.
|
||||
each list item is ``{"path": str, "meta": {...}, "anchors": [str, ...]}``.
|
||||
Empty input returns ``{}``. ``max_per_direction`` caps the neighbor
|
||||
list per direction *before* meta lookup so we don't fetch nodes
|
||||
that won't be displayed.
|
||||
|
|
@ -95,9 +85,10 @@ async def expand_links(
|
|||
nodes = await file_store.get_nodes(neighbor_paths) if neighbor_paths else []
|
||||
meta_by_path = {n.path: _node_meta(n) for n in nodes}
|
||||
|
||||
def _attach(grouped: dict[str, list[dict]]) -> list[dict]:
|
||||
def _attach(grouped: dict[str, list[str]]) -> list[dict]:
|
||||
return [
|
||||
{"path": npath, "meta": meta_by_path.get(npath, {}), "edges": edges} for npath, edges in grouped.items()
|
||||
{"path": npath, "meta": meta_by_path.get(npath, {}), "anchors": anchors}
|
||||
for npath, anchors in grouped.items()
|
||||
]
|
||||
|
||||
return {p: {"outlinks": _attach(og), "inlinks": _attach(ig)} for p, og, ig in zip(paths, out_grouped, in_grouped)}
|
||||
|
|
@ -124,6 +115,6 @@ def render_expansion_lines(expansion: dict, indent: str = " ") -> list[str]:
|
|||
lines.append(f"{indent}{direction} ({len(items)}):")
|
||||
for item in items:
|
||||
lines.append(f"{inner}{arrow} {item['path']} {_format_meta_inline(item['meta'])}")
|
||||
for edge in item["edges"]:
|
||||
lines.append(f"{edge_indent}via {_format_via(edge)}")
|
||||
for anchor in item["anchors"]:
|
||||
lines.append(f"{edge_indent}via anchor=#{anchor}")
|
||||
return lines
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
One class, :class:`WikilinkHandler`, owning every wikilink concern:
|
||||
|
||||
* **Pure text** — regex, Dataview predicate inference, validation:
|
||||
* **Pure text** — regex, extraction, rewrite, validation:
|
||||
:meth:`~WikilinkHandler.extract_links` (used by
|
||||
:mod:`reme.components.file_chunker.markdown_file_chunker`),
|
||||
:meth:`~WikilinkHandler.scan_and_rewrite`,
|
||||
|
|
@ -16,13 +16,12 @@ One class, :class:`WikilinkHandler`, owning every wikilink concern:
|
|||
post-rename to point inbound ``[[src]]`` at the new path). Source
|
||||
candidates come from the file_graph's reverse index — no fs scan.
|
||||
|
||||
Wikilink convention. Targets are taken **literally** — ``[[X]]`` →
|
||||
Wikilink targets are taken **literally** — ``[[X]]`` →
|
||||
``target="X"``, no implicit ``.md``, no short-form basename search,
|
||||
no folder-note expansion. Anchor and alias survive a rewrite
|
||||
verbatim. Image marker (``!``) and Dataview predicate (``pred::``
|
||||
outside the brackets) sit outside ``[[...]]`` and are not touched by
|
||||
a rewrite. Recommended form: full path relative to the workspace with
|
||||
extension (``[[topics/x.md]]``).
|
||||
verbatim. Text outside ``[[...]]`` is ignored. Recommended form: full
|
||||
path relative to the workspace with extension (``[[topics/x.md]]``).
|
||||
Ordinary Markdown links such as ``[label](path.md)`` are ignored.
|
||||
|
||||
Stale graph entries are harmless (``scan_and_rewrite`` returns
|
||||
count=0 and the file is skipped), but a graph missing recent writes
|
||||
|
|
@ -37,26 +36,23 @@ from ..enumeration import LinkScopeEnum
|
|||
from ..schema import FileLink
|
||||
|
||||
|
||||
def _normalize_workspace_path(path: str) -> str:
|
||||
"""Use POSIX separators for workspace paths on every platform."""
|
||||
return path.replace("\\", "/")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WikilinkMatch:
|
||||
"""One ``[[...]]`` occurrence with parts surfaced.
|
||||
|
||||
``anchor`` / ``alias`` are stored **without** the leading ``#`` /
|
||||
``|`` so they map cleanly to :class:`FileLink.target_anchor`; the
|
||||
rewrite path reads the raw regex groups (with delimiters) directly
|
||||
and doesn't go through this dataclass.
|
||||
"""
|
||||
"""The graph-relevant parts and source span of one wikilink."""
|
||||
|
||||
target: str
|
||||
anchor: str | None
|
||||
alias: str | None
|
||||
bang: bool
|
||||
start: int
|
||||
end: int
|
||||
|
||||
|
||||
class WikilinkHandler:
|
||||
"""Pure-text wikilink operations: parse, extract, rewrite, validate."""
|
||||
"""Parse, extract, rewrite, and validate wikilinks."""
|
||||
|
||||
# Captures: optional image marker (``!``), the bare target, an
|
||||
# optional ``#anchor`` slice (with ``#``), and an optional ``|alias``
|
||||
|
|
@ -77,54 +73,39 @@ class WikilinkHandler:
|
|||
|
||||
FORBIDDEN_IN_NEW = ("[", "]", "#", "|", "\n", "\r")
|
||||
|
||||
_DATAVIEW_LINE_RE = re.compile(
|
||||
r"^[ \t]*(?:[-*+][ \t]+)?(?P<predicate>[A-Za-z][A-Za-z0-9_]*)\s*::\s*(?P<value>.+?)\s*$",
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
_INLINE_FIELD_OPEN_RE = re.compile(r"\[(?P<predicate>[A-Za-z][A-Za-z0-9_]*)\s*::\s*")
|
||||
|
||||
# -- Low-level scan ------------------------------------------------
|
||||
|
||||
@classmethod
|
||||
def iter_matches(cls, text: str):
|
||||
"""Yield :class:`WikilinkMatch` for every ``[[...]]`` in ``text``.
|
||||
|
||||
Skips matches whose target is empty after strip (defensive).
|
||||
"""
|
||||
"""Yield every non-empty ``[[...]]`` occurrence in ``text``."""
|
||||
for m in cls.WIKILINK_RE.finditer(text):
|
||||
target = m.group("target").strip()
|
||||
if not target:
|
||||
continue
|
||||
anchor_raw = m.group("anchor")
|
||||
alias_raw = m.group("alias")
|
||||
yield WikilinkMatch(
|
||||
target=target,
|
||||
anchor=anchor_raw[1:].strip() if anchor_raw else None,
|
||||
alias=alias_raw[1:].strip() if alias_raw else None,
|
||||
bang=bool(m.group("bang")),
|
||||
start=m.start(),
|
||||
end=m.end(),
|
||||
)
|
||||
|
||||
# -- FileLink extraction (with predicate inference) ---------------
|
||||
# -- FileLink extraction ------------------------------------------
|
||||
|
||||
@classmethod
|
||||
def extract_links(cls, text: str, source_path: str) -> list[FileLink]:
|
||||
"""Emit :class:`FileLink` edges for every wikilink in ``text``.
|
||||
|
||||
No resolution: ``target_path`` is the bracket contents verbatim.
|
||||
Results are deduped by ``(target_path, predicate, target_anchor)``
|
||||
preserving order.
|
||||
Targets remain literal. Results are deduped by
|
||||
``(target_path, target_anchor)`` while preserving order.
|
||||
"""
|
||||
if not text:
|
||||
return []
|
||||
inline_spans = cls._iter_inline_fields(text)
|
||||
source_path = _normalize_workspace_path(source_path)
|
||||
out: list[FileLink] = []
|
||||
seen: set[tuple] = set()
|
||||
for wm in cls.iter_matches(text):
|
||||
predicate = cls._predicate_for(text, wm.start, inline_spans)
|
||||
key = (wm.target, predicate, wm.anchor)
|
||||
key = (wm.target, wm.anchor)
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
|
|
@ -133,7 +114,6 @@ class WikilinkHandler:
|
|||
source_path=source_path,
|
||||
target_path=wm.target,
|
||||
target_anchor=wm.anchor,
|
||||
predicate=predicate,
|
||||
),
|
||||
)
|
||||
return out
|
||||
|
|
@ -147,20 +127,19 @@ class WikilinkHandler:
|
|||
old: str,
|
||||
new: str | None,
|
||||
) -> tuple[str, int]:
|
||||
"""Find (and optionally rewrite) wikilinks whose target equals ``old``.
|
||||
"""Find and optionally rewrite wikilinks whose target is ``old``.
|
||||
|
||||
Returns ``(new_text, count)``. When ``new`` is ``None`` no rewrite
|
||||
happens (the original text is returned), but the count is still
|
||||
populated — used by ``find_inbound``. Matching is literal:
|
||||
``target == old``. No short-link, no implicit ``.md``, no
|
||||
folder-note expansion.
|
||||
populated — used by ``find_inbound``. Matching is literal.
|
||||
"""
|
||||
old = _normalize_workspace_path(old)
|
||||
new = _normalize_workspace_path(new) if new is not None else None
|
||||
count = 0
|
||||
|
||||
def sub(match: re.Match) -> str:
|
||||
nonlocal count
|
||||
target = match.group("target").strip()
|
||||
if target != old:
|
||||
if match.group("target").strip() != old:
|
||||
return match.group(0)
|
||||
count += 1
|
||||
if new is None:
|
||||
|
|
@ -170,8 +149,7 @@ class WikilinkHandler:
|
|||
bang = match.group("bang") or ""
|
||||
return f"{bang}[[{new}{anchor}{alias}]]"
|
||||
|
||||
new_text = cls.WIKILINK_RE.sub(sub, text)
|
||||
return new_text, count
|
||||
return cls.WIKILINK_RE.sub(sub, text), count
|
||||
|
||||
# -- Validation ----------------------------------------------------
|
||||
|
||||
|
|
@ -201,53 +179,6 @@ class WikilinkHandler:
|
|||
prefix = scope.rstrip("/") + "/"
|
||||
return rel == scope or rel.startswith(prefix)
|
||||
|
||||
# -- Predicate helpers (internal) ---------------------------------
|
||||
|
||||
@classmethod
|
||||
def _iter_inline_fields(cls, text: str) -> list[tuple[int, int, str]]:
|
||||
"""Find inline-bracketed ``[predicate:: …]`` field spans by depth scan."""
|
||||
out: list[tuple[int, int, str]] = []
|
||||
for m in cls._INLINE_FIELD_OPEN_RE.finditer(text):
|
||||
depth = 1
|
||||
i = m.end()
|
||||
n = len(text)
|
||||
while i < n:
|
||||
c = text[i]
|
||||
if c == "\n":
|
||||
break
|
||||
if c == "[":
|
||||
depth += 1
|
||||
elif c == "]":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
out.append((m.start(), i + 1, m.group("predicate")))
|
||||
break
|
||||
i += 1
|
||||
return out
|
||||
|
||||
@classmethod
|
||||
def _predicate_for(
|
||||
cls,
|
||||
text: str,
|
||||
pos: int,
|
||||
inline_spans: list[tuple[int, int, str]],
|
||||
) -> str | None:
|
||||
"""Resolve the predicate governing a wikilink at offset ``pos``.
|
||||
|
||||
Precedence: inline-bracketed > line-level Dataview > none.
|
||||
"""
|
||||
for field_start, field_end, predicate in inline_spans:
|
||||
if field_start <= pos < field_end:
|
||||
return predicate
|
||||
line_start = text.rfind("\n", 0, pos) + 1
|
||||
line_end = text.find("\n", pos)
|
||||
if line_end == -1:
|
||||
line_end = len(text)
|
||||
m = cls._DATAVIEW_LINE_RE.match(text[line_start:line_end])
|
||||
if m and line_start + m.start("value") <= pos:
|
||||
return m.group("predicate")
|
||||
return None
|
||||
|
||||
# -- Async file_graph-aware operations -----------------------------
|
||||
|
||||
@classmethod
|
||||
|
|
@ -260,11 +191,12 @@ class WikilinkHandler:
|
|||
required to surface sources whose edges sit in the pending bucket.
|
||||
Each returned ``FileLink`` carries the linking node's ``source_path``;
|
||||
we dedupe to a sorted list since one source can host multiple edges
|
||||
(different anchor/predicate) to the same target. Returns ``[]`` when
|
||||
(different anchors) to the same target. Returns ``[]`` when
|
||||
there is no file_graph attached or no source references the target.
|
||||
"""
|
||||
if not file_store.file_graph:
|
||||
return []
|
||||
target = _normalize_workspace_path(target)
|
||||
inlinks = await file_store.file_graph.get_inlinks(target, scope=LinkScopeEnum.ALL)
|
||||
return sorted({link.source_path for link in inlinks if link.source_path})
|
||||
|
||||
|
|
@ -272,12 +204,10 @@ class WikilinkHandler:
|
|||
async def find_inbound(cls, file_store, target: str, scope: str = "") -> dict:
|
||||
"""Count wikilinks across the workspace that point at ``target``.
|
||||
|
||||
Literal matching: ``[[target]]`` only. The target file itself is
|
||||
excluded — self-references don't survive a delete and aren't
|
||||
actionable for the caller. Sources come from the file_graph's
|
||||
reverse index; per-file counts come from reading each candidate
|
||||
source (the graph dedupes by ``(target, predicate, anchor)`` so
|
||||
it can't count repeated bare-wikilink occurrences directly).
|
||||
The target file itself is excluded — self-references don't survive a
|
||||
delete and aren't actionable for the caller. Sources come from the
|
||||
file_graph's reverse index; per-file counts come from reading each
|
||||
candidate source because the graph dedupes repeated edges.
|
||||
|
||||
Result shape::
|
||||
|
||||
|
|
@ -291,6 +221,8 @@ class WikilinkHandler:
|
|||
|
||||
On bad inputs returns ``{"target": ..., "error": str}``.
|
||||
"""
|
||||
target = _normalize_workspace_path(target)
|
||||
scope = _normalize_workspace_path(scope)
|
||||
if not target:
|
||||
return {"target": target, "error": "target is required"}
|
||||
if Path(target).is_absolute():
|
||||
|
|
@ -340,6 +272,9 @@ class WikilinkHandler:
|
|||
matching only; candidate sources come from the file_graph's reverse
|
||||
index.
|
||||
"""
|
||||
src = _normalize_workspace_path(src)
|
||||
dst = _normalize_workspace_path(dst)
|
||||
scope = _normalize_workspace_path(scope)
|
||||
err = cls.validate_src_dst(src, dst)
|
||||
if err is not None:
|
||||
return {"src": src, "dst": dst, "error": err}
|
||||
|
|
|
|||
|
|
@ -84,7 +84,8 @@ token used to assert identity and claims between parties.
|
|||
## Related
|
||||
Often issued by [[digest/wiki/oauth2.md]] flows.
|
||||
|
||||
derived_from:: [[daily/2026-05-15/auth-design/notes.md]]
|
||||
## Sources
|
||||
- [[daily/2026-05-15/auth-design/notes.md]]
|
||||
""",
|
||||
"digest/wiki/oauth2.md": """\
|
||||
---
|
||||
|
|
@ -103,7 +104,8 @@ issued by an authorization server.
|
|||
- Client credentials
|
||||
- Refresh token
|
||||
|
||||
derived_from:: [[daily/2026-05-10/oauth-intro/notes.md]]
|
||||
## Sources
|
||||
- [[daily/2026-05-10/oauth-intro/notes.md]]
|
||||
""",
|
||||
"digest/procedure/key-rotation.md": """\
|
||||
---
|
||||
|
|
@ -127,7 +129,8 @@ issuance.
|
|||
Default rotation cadence is **30 days**. Driven by historical practice;
|
||||
no formal compliance requirement has tightened this so far.
|
||||
|
||||
derived_from:: [[daily/2026-05-20/rotation-plan/notes.md]]
|
||||
## Sources
|
||||
- [[daily/2026-05-20/rotation-plan/notes.md]]
|
||||
""",
|
||||
"digest/personal/no-trailing-summary.md": """\
|
||||
---
|
||||
|
|
@ -144,7 +147,8 @@ description: 不要在回复末尾加总结段落
|
|||
**How to apply**: 任意编码 / 编辑任务回复结束时,直接停在最后一条
|
||||
有信息量的话上,不要再补一段"以上就是本次的修改..."。
|
||||
|
||||
derived_from:: [[daily/2026-05-01/style-feedback/notes.md]]
|
||||
## Sources
|
||||
- [[daily/2026-05-01/style-feedback/notes.md]]
|
||||
""",
|
||||
# ----- daily provenance stubs (so the digest links don't dangle) -----
|
||||
"daily/2026-05-01/style-feedback/notes.md": """\
|
||||
|
|
|
|||
|
|
@ -182,15 +182,13 @@ def test_auto_dream_and_proactive():
|
|||
if (env.workspace_dir / rel).is_file()
|
||||
}
|
||||
digest_wikilinks = [rel for rel, text in target_texts.items() if "[[digest/" in text]
|
||||
provenance_links = [
|
||||
rel for rel, text in target_texts.items() if f"derived_from:: [[{DREAM_INPUT_PATH}]]" in text
|
||||
]
|
||||
source_links = [rel for rel, text in target_texts.items() if f"- [[{DREAM_INPUT_PATH}]]" in text]
|
||||
print(f"[wikilink] integrated targets: {target_paths}")
|
||||
print(f"[wikilink] integrated targets with [[digest/...]] links: {digest_wikilinks}")
|
||||
print(f"[wikilink] integrated targets with derived_from source links: {provenance_links}")
|
||||
print(f"[wikilink] integrated targets with source links: {source_links}")
|
||||
assert target_texts, f"no integrated target files found: {target_paths}"
|
||||
assert provenance_links, (
|
||||
"no derived_from wikilink back to the changed daily note in integrated targets\n"
|
||||
assert source_links, (
|
||||
"no source wikilink back to the changed daily note in integrated targets\n"
|
||||
f"targets: {target_paths}"
|
||||
)
|
||||
assert digest_wikilinks, (
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ from reme.components.application_context import ApplicationContext
|
|||
from reme.components.file_catalog import BaseFileCatalog
|
||||
from reme.components.file_store import BaseFileStore
|
||||
from reme.components.runtime_context import RuntimeContext
|
||||
from reme.schema import DreamState
|
||||
from reme.schema import DreamState, FileNode
|
||||
from reme.steps.evolve.dream.extract import DreamExtractStep
|
||||
from reme.steps.evolve.dream.finish import DreamFinishStep
|
||||
from reme.steps.evolve.dream.proactive import ProactiveStep
|
||||
|
|
@ -94,6 +94,48 @@ def test_scan_day_files_includes_nested_md_and_excludes_interests():
|
|||
]
|
||||
|
||||
|
||||
def test_dream_extract_matches_posix_catalog_paths(tmp_path):
|
||||
"""Unchanged nested files retain their POSIX catalog entries on every platform."""
|
||||
|
||||
class Catalog(_Catalog):
|
||||
"""Catalog seeded with POSIX paths and recording deletions."""
|
||||
|
||||
def __init__(self, nodes):
|
||||
super().__init__()
|
||||
self.nodes = nodes
|
||||
self.deleted = []
|
||||
|
||||
async def delete(self, path):
|
||||
self.deleted.extend(path if isinstance(path, list) else [path])
|
||||
|
||||
async def get_nodes(self, paths=None):
|
||||
return self.nodes
|
||||
|
||||
async def run():
|
||||
note = _touch(tmp_path / "daily" / "2026-05-28" / "nested" / "session.md")
|
||||
rel_path = note.relative_to(tmp_path).as_posix()
|
||||
catalog = Catalog([FileNode(path=rel_path, st_mtime=note.stat().st_mtime)])
|
||||
step = DreamExtractStep(scan_days=1, app_context=ApplicationContext(workspace_dir=str(tmp_path)))
|
||||
|
||||
with patch("reme.steps.evolve.dream.extract.refresh_day_index", return_value={}):
|
||||
response = await step(
|
||||
RuntimeContext(
|
||||
date="2026-05-28",
|
||||
file_catalog=catalog,
|
||||
file_store=_FileStore(tmp_path),
|
||||
),
|
||||
)
|
||||
|
||||
dream = response.metadata["dream"]
|
||||
assert response.success is True
|
||||
assert dream["unchanged_paths"] == [rel_path]
|
||||
assert dream["changed_paths"] == []
|
||||
assert dream["deleted_paths"] == []
|
||||
assert not catalog.deleted
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_recent_dates_includes_anchor_and_previous_days():
|
||||
"""Recent date window is inclusive and chronological."""
|
||||
assert recent_dates("2026-05-28", 3) == ["2026-05-26", "2026-05-27", "2026-05-28"]
|
||||
|
|
@ -152,7 +194,7 @@ def test_extract_without_llm_marks_changed_paths_failed(tmp_path):
|
|||
|
||||
dream = response.metadata["dream"]
|
||||
assert response.success is False
|
||||
assert str(note.relative_to(tmp_path)) in dream["changed_paths"]
|
||||
assert note.relative_to(tmp_path).as_posix() in dream["changed_paths"]
|
||||
assert dream["failed_paths"] == dream["changed_paths"]
|
||||
|
||||
asyncio.run(run())
|
||||
|
|
@ -299,9 +341,9 @@ def test_finish_does_not_checkpoint_failed_changed_paths():
|
|||
dates=["2026-05-26", "2026-05-27", "2026-05-28"],
|
||||
workspace=str(workspace),
|
||||
daily_dir="daily",
|
||||
changed_paths=[str(ok.relative_to(workspace)), str(failed.relative_to(workspace))],
|
||||
failed_paths=[str(failed.relative_to(workspace))],
|
||||
interests_paths=[str(interests.relative_to(workspace))],
|
||||
changed_paths=[ok.relative_to(workspace).as_posix(), failed.relative_to(workspace).as_posix()],
|
||||
failed_paths=[failed.relative_to(workspace).as_posix()],
|
||||
interests_paths=[interests.relative_to(workspace).as_posix()],
|
||||
integrate_results=[
|
||||
{
|
||||
"action": "CREATE",
|
||||
|
|
@ -320,10 +362,10 @@ def test_finish_does_not_checkpoint_failed_changed_paths():
|
|||
assert "topics:" not in resp.answer
|
||||
assert "Changes:" in resp.answer
|
||||
assert "- [digest/procedure/example.md][CREATE]: Created a concise procedure node." in resp.answer
|
||||
assert str(ok.relative_to(workspace)) in upserted
|
||||
assert str(failed.relative_to(workspace)) not in upserted
|
||||
assert str(interests.relative_to(workspace)) in upserted
|
||||
assert str(day_index.relative_to(workspace)) in upserted
|
||||
assert ok.relative_to(workspace).as_posix() in upserted
|
||||
assert failed.relative_to(workspace).as_posix() not in upserted
|
||||
assert interests.relative_to(workspace).as_posix() in upserted
|
||||
assert day_index.relative_to(workspace).as_posix() in upserted
|
||||
assert catalog.dumps == 1
|
||||
|
||||
asyncio.run(run())
|
||||
|
|
|
|||
|
|
@ -306,7 +306,7 @@ def test_to_workspace_relative_inside_workspace():
|
|||
mixin = ComponentMixin()
|
||||
abs_path = mixin.workspace_path / "sub" / "file.md"
|
||||
rel = mixin.to_workspace_relative(abs_path)
|
||||
assert rel == str(abs_path.relative_to(mixin.workspace_path))
|
||||
assert rel == abs_path.relative_to(mixin.workspace_path).as_posix()
|
||||
finally:
|
||||
os.chdir(old_cwd)
|
||||
|
||||
|
|
|
|||
|
|
@ -42,12 +42,14 @@ def _run(coro):
|
|||
asyncio.run(coro)
|
||||
|
||||
|
||||
def _node(path: str, links: list[tuple[str, str | None, str | None]] | None = None) -> FileNode:
|
||||
"""Build a FileNode with (target_path, target_anchor, predicate) outgoing edges."""
|
||||
def _node(path: str, links: list[tuple[str, str | None]] | None = None) -> FileNode:
|
||||
"""Build a FileNode with (target_path, target_anchor) outgoing edges."""
|
||||
return FileNode(
|
||||
path=path,
|
||||
st_mtime=1.0,
|
||||
links=[FileLink(source_path=path, target_path=t, target_anchor=a, predicate=p) for t, a, p in (links or [])],
|
||||
links=[
|
||||
FileLink(source_path=path, target_path=target, target_anchor=anchor) for target, anchor in (links or [])
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -197,7 +199,7 @@ def test_traverse_forward_depth_1():
|
|||
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
|
||||
store = await _make_store(
|
||||
[
|
||||
_node("a.md", [("b.md", None, None), ("c.md", "intro", "ref")]),
|
||||
_node("a.md", [("b.md", None), ("c.md", "intro")]),
|
||||
_node("b.md"),
|
||||
_node("c.md"),
|
||||
],
|
||||
|
|
@ -207,9 +209,8 @@ def test_traverse_forward_depth_1():
|
|||
results = _edges(step)
|
||||
paths = {r["path"] for r in results}
|
||||
assert paths == {"b.md", "c.md"}
|
||||
# The 'ref' edge should report its predicate/anchor.
|
||||
# Anchors remain part of traversal metadata.
|
||||
c_edge = next(r for r in results if r["path"] == "c.md")
|
||||
assert c_edge["predicate"] == "ref"
|
||||
assert c_edge["anchor"] == "intro"
|
||||
assert c_edge["via"] == "a.md"
|
||||
assert c_edge["depth"] == 1
|
||||
|
|
@ -226,8 +227,8 @@ def test_traverse_backward_returns_inlinks():
|
|||
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
|
||||
store = await _make_store(
|
||||
[
|
||||
_node("a.md", [("b.md", None, None)]),
|
||||
_node("c.md", [("b.md", None, None)]),
|
||||
_node("a.md", [("b.md", None)]),
|
||||
_node("c.md", [("b.md", None)]),
|
||||
_node("b.md"),
|
||||
],
|
||||
)
|
||||
|
|
@ -248,8 +249,8 @@ def test_traverse_depth_2_expands():
|
|||
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
|
||||
store = await _make_store(
|
||||
[
|
||||
_node("a.md", [("b.md", None, None)]),
|
||||
_node("b.md", [("c.md", None, None)]),
|
||||
_node("a.md", [("b.md", None)]),
|
||||
_node("b.md", [("c.md", None)]),
|
||||
_node("c.md"),
|
||||
],
|
||||
)
|
||||
|
|
@ -311,8 +312,8 @@ def test_traverse_both_directions():
|
|||
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
|
||||
store = await _make_store(
|
||||
[
|
||||
_node("upstream.md", [("center.md", None, None)]),
|
||||
_node("center.md", [("downstream.md", None, None)]),
|
||||
_node("upstream.md", [("center.md", None)]),
|
||||
_node("center.md", [("downstream.md", None)]),
|
||||
_node("downstream.md"),
|
||||
],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -105,7 +105,6 @@ def test_parse_links_bare():
|
|||
assert link.source_path == "src.md"
|
||||
assert link.target_path == "note"
|
||||
assert link.target_anchor is None
|
||||
assert link.predicate is None
|
||||
print("✓ test_parse_links_bare passed")
|
||||
|
||||
|
||||
|
|
@ -115,7 +114,6 @@ def test_parse_links_with_anchor():
|
|||
assert len(links) == 1
|
||||
assert links[0].target_path == "note"
|
||||
assert links[0].target_anchor == "section A"
|
||||
assert links[0].predicate is None
|
||||
print("✓ test_parse_links_with_anchor passed")
|
||||
|
||||
|
||||
|
|
@ -137,47 +135,18 @@ def test_parse_links_anchor_and_alias():
|
|||
print("✓ test_parse_links_anchor_and_alias passed")
|
||||
|
||||
|
||||
def test_parse_links_predicate_simple():
|
||||
"""Dataview inline: predicate:: [[target]]."""
|
||||
links = WikilinkHandler.extract_links("author:: [[Alice]]", "src.md")
|
||||
assert len(links) == 1
|
||||
assert links[0].predicate == "author"
|
||||
assert links[0].target_path == "Alice"
|
||||
assert links[0].target_anchor is None
|
||||
print("✓ test_parse_links_predicate_simple passed")
|
||||
|
||||
|
||||
def test_parse_links_predicate_bracketed():
|
||||
"""Dataview inline-bracket: [predicate:: [[target]]]."""
|
||||
links = WikilinkHandler.extract_links("text [author:: [[Alice]]] more", "src.md")
|
||||
assert len(links) == 1
|
||||
assert links[0].predicate == "author"
|
||||
assert links[0].target_path == "Alice"
|
||||
print("✓ test_parse_links_predicate_bracketed passed")
|
||||
|
||||
|
||||
def test_parse_links_predicate_bracketed_with_anchor():
|
||||
"""[predicate:: [[target_path#target_anchor]]] — combined form."""
|
||||
def test_parse_links_ignores_legacy_relation_wrappers():
|
||||
"""Legacy relation text remains compatible as ordinary wikilinks."""
|
||||
links = WikilinkHandler.extract_links(
|
||||
"[predicate:: [[target_path#target_anchor]]]",
|
||||
"related:: [[a]]\n- related:: [[b]]\n[related:: [[c#section]]]",
|
||||
"src.md",
|
||||
)
|
||||
assert len(links) == 1
|
||||
link = links[0]
|
||||
assert link.source_path == "src.md"
|
||||
assert link.predicate == "predicate"
|
||||
assert link.target_path == "target_path"
|
||||
assert link.target_anchor == "target_anchor"
|
||||
print("✓ test_parse_links_predicate_bracketed_with_anchor passed")
|
||||
|
||||
|
||||
def test_parse_links_predicate_sticks_to_first():
|
||||
"""Line-level predicate covers all wikilinks in its value portion."""
|
||||
links = WikilinkHandler.extract_links("pred:: [[a]] and bare [[b]]", "src.md")
|
||||
assert len(links) == 2
|
||||
assert links[0].predicate == "pred" and links[0].target_path == "a"
|
||||
assert links[1].predicate == "pred" and links[1].target_path == "b"
|
||||
print("✓ test_parse_links_predicate_sticks_to_first passed")
|
||||
assert [(link.target_path, link.target_anchor) for link in links] == [
|
||||
("a", None),
|
||||
("b", None),
|
||||
("c", "section"),
|
||||
]
|
||||
assert all(link.predicate is None for link in links)
|
||||
|
||||
|
||||
def test_parse_links_multiple_on_one_line():
|
||||
|
|
@ -190,6 +159,30 @@ def test_parse_links_multiple_on_one_line():
|
|||
print("✓ test_parse_links_multiple_on_one_line passed")
|
||||
|
||||
|
||||
def test_parse_wikilink_line_ranges():
|
||||
"""Workspace paths and supported line-range forms create FileLink edges."""
|
||||
links = WikilinkHandler.extract_links(
|
||||
"[[daily/2026-06-20/session.md]] [[notes/example.md#L9]] "
|
||||
+ "[[notes/example.md#L9-L10]] [[notes/example.md#L9-L10,L15-L20]]",
|
||||
"src.md",
|
||||
)
|
||||
assert [(link.target_path, link.target_anchor) for link in links] == [
|
||||
("daily/2026-06-20/session.md", None),
|
||||
("notes/example.md", "L9"),
|
||||
("notes/example.md", "L9-L10"),
|
||||
("notes/example.md", "L9-L10,L15-L20"),
|
||||
]
|
||||
|
||||
|
||||
def test_parse_markdown_links_are_ignored():
|
||||
"""Ordinary Markdown links do not create FileLink edges."""
|
||||
links = WikilinkHandler.extract_links(
|
||||
"[plain](../wiki/a.md) [section](../wiki/a.md#section) [ranges](../wiki/b.md#L9-L10,L15-L20)",
|
||||
"daily/note.md",
|
||||
)
|
||||
assert not links
|
||||
|
||||
|
||||
def test_parse_links_no_match():
|
||||
"""Strings without [[]] yield no links, even if '::' appears."""
|
||||
assert len(WikilinkHandler.extract_links("no link here :: foo", "src.md")) == 0
|
||||
|
|
@ -198,15 +191,6 @@ def test_parse_links_no_match():
|
|||
print("✓ test_parse_links_no_match passed")
|
||||
|
||||
|
||||
def test_parse_links_predicate_with_underscore_and_digits():
|
||||
"""Predicate identifier accepts letters, digits, underscore (no dash per Dataview spec)."""
|
||||
links = WikilinkHandler.extract_links("see_also2:: [[target]]", "src.md")
|
||||
assert len(links) == 1
|
||||
assert links[0].predicate == "see_also2"
|
||||
assert links[0].target_path == "target"
|
||||
print("✓ test_parse_links_predicate_with_underscore_and_digits passed")
|
||||
|
||||
|
||||
def test_parse_links_in_file():
|
||||
"""Integration: parse() populates FileNode.links from file content."""
|
||||
|
||||
|
|
@ -227,11 +211,8 @@ def test_parse_links_in_file():
|
|||
try:
|
||||
chunker = DefaultFileChunker()
|
||||
file_node, _ = await chunker.chunk(temp_path)
|
||||
triples = {(link.predicate, link.target_path, link.target_anchor) for link in file_node.links}
|
||||
assert (None, "alpha", None) in triples
|
||||
assert (None, "beta", "h2") in triples
|
||||
assert ("author", "Alice", None) in triples
|
||||
assert ("ref", "paper", "chapter 1") in triples
|
||||
pairs = {(link.target_path, link.target_anchor) for link in file_node.links}
|
||||
assert pairs == {("alpha", None), ("beta", "h2"), ("Alice", None), ("paper", "chapter 1")}
|
||||
assert all(link.source_path == file_node.path for link in file_node.links)
|
||||
print("✓ test_parse_links_in_file passed")
|
||||
finally:
|
||||
|
|
@ -384,13 +365,11 @@ if __name__ == "__main__":
|
|||
test_parse_links_with_anchor()
|
||||
test_parse_links_alias_dropped()
|
||||
test_parse_links_anchor_and_alias()
|
||||
test_parse_links_predicate_simple()
|
||||
test_parse_links_predicate_bracketed()
|
||||
test_parse_links_predicate_bracketed_with_anchor()
|
||||
test_parse_links_predicate_sticks_to_first()
|
||||
test_parse_links_ignores_legacy_relation_wrappers()
|
||||
test_parse_links_multiple_on_one_line()
|
||||
test_parse_wikilink_line_ranges()
|
||||
test_parse_markdown_links_are_ignored()
|
||||
test_parse_links_no_match()
|
||||
test_parse_links_predicate_with_underscore_and_digits()
|
||||
test_parse_links_in_file()
|
||||
test_parse_links_empty_when_no_content()
|
||||
test_chunk_does_not_split_wikilink_at_boundary()
|
||||
|
|
|
|||
|
|
@ -38,6 +38,28 @@ def make_node(path: str, links: list[tuple[str, str | None]] | None = None) -> F
|
|||
)
|
||||
|
||||
|
||||
def test_file_node_loads_legacy_predicate_without_persisting_it():
|
||||
"""Legacy predicates are accepted for compatibility but omitted from new snapshots."""
|
||||
node = FileNode.model_validate(
|
||||
{
|
||||
"path": "a.md",
|
||||
"st_mtime": 1.0,
|
||||
"links": [
|
||||
{
|
||||
"source_path": "a.md",
|
||||
"target_path": "b.md",
|
||||
"target_anchor": "intro",
|
||||
"predicate": "related",
|
||||
},
|
||||
],
|
||||
},
|
||||
)
|
||||
assert node.links == [
|
||||
FileLink(source_path="a.md", target_path="b.md", target_anchor="intro", predicate="related"),
|
||||
]
|
||||
assert "predicate" not in node.model_dump_json()
|
||||
|
||||
|
||||
# Both backends should satisfy the same BaseFileGraph contract.
|
||||
BACKENDS = [LocalFileGraph, NxFileGraph]
|
||||
|
||||
|
|
|
|||
31
tests/unit/test_line_anchor.py
Normal file
31
tests/unit/test_line_anchor.py
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
"""Tests for GitHub-style line anchor parsing."""
|
||||
|
||||
import pytest
|
||||
|
||||
from reme.utils import format_line_anchor, parse_line_anchor
|
||||
|
||||
|
||||
def test_parse_supported_line_anchors():
|
||||
"""Single lines, continuous ranges, and comma-separated ranges parse."""
|
||||
assert parse_line_anchor("L9") == [(9, 9)]
|
||||
assert parse_line_anchor("L9-L10") == [(9, 10)]
|
||||
assert parse_line_anchor("L9-L10,L15-L20") == [(9, 10), (15, 20)]
|
||||
|
||||
|
||||
def test_parse_merges_overlapping_and_adjacent_ranges():
|
||||
"""Normalization merges ranges whose covered lines touch."""
|
||||
ranges = parse_line_anchor("L9-L12,L11-L15,L16")
|
||||
assert ranges == [(9, 16)]
|
||||
assert format_line_anchor(ranges) == "L9-L16"
|
||||
|
||||
|
||||
def test_non_line_heading_is_unchanged():
|
||||
"""Ordinary heading anchors are outside this parser's contract."""
|
||||
assert parse_line_anchor("Introduction") is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("anchor", ["L0", "L10-L9", "L9-Lx", "L9,"])
|
||||
def test_invalid_line_anchor_rejected(anchor):
|
||||
"""Line-looking anchors fail clearly when malformed or out of range."""
|
||||
with pytest.raises(ValueError):
|
||||
parse_line_anchor(anchor)
|
||||
|
|
@ -113,7 +113,7 @@ def test_expand_links_returns_outlinks_and_inlinks_with_meta():
|
|||
assert len(a_out) == 1
|
||||
assert a_out[0]["path"] == "B.md"
|
||||
assert a_out[0]["meta"] == {"name": "B Doc", "description": "beta"}
|
||||
assert a_out[0]["edges"] == [{"predicate": None, "anchor": None}]
|
||||
assert a_out[0]["anchors"] == []
|
||||
assert result["A.md"]["inlinks"] == []
|
||||
|
||||
b_in = result["B.md"]["inlinks"]
|
||||
|
|
@ -184,13 +184,13 @@ def test_render_expansion_lines_empty_input_yields_empty_list():
|
|||
|
||||
|
||||
def test_render_expansion_lines_outlinks_only():
|
||||
"""Single outlink with meta + plain edge renders as 3 lines."""
|
||||
"""Single outlink with metadata renders without a redundant edge line."""
|
||||
expansion = {
|
||||
"outlinks": [
|
||||
{
|
||||
"path": "B.md",
|
||||
"meta": {"name": "B", "description": "beta"},
|
||||
"edges": [{"predicate": None, "anchor": None}],
|
||||
"anchors": [],
|
||||
},
|
||||
],
|
||||
"inlinks": [],
|
||||
|
|
@ -199,20 +199,19 @@ def test_render_expansion_lines_outlinks_only():
|
|||
assert lines == [
|
||||
" outlinks (1):",
|
||||
' → B.md name="B" description="beta"',
|
||||
" via plain",
|
||||
]
|
||||
print("✓ test_render_expansion_lines_outlinks_only passed")
|
||||
|
||||
|
||||
def test_render_expansion_lines_inlinks_only_with_predicate_and_anchor():
|
||||
"""Inlink edge with predicate + anchor renders via descriptor."""
|
||||
def test_render_expansion_lines_inlinks_only_with_anchor():
|
||||
"""Inlink anchor renders via descriptor."""
|
||||
expansion = {
|
||||
"outlinks": [],
|
||||
"inlinks": [
|
||||
{
|
||||
"path": "src.md",
|
||||
"meta": {},
|
||||
"edges": [{"predicate": "references", "anchor": "intro"}],
|
||||
"anchors": ["intro"],
|
||||
},
|
||||
],
|
||||
}
|
||||
|
|
@ -220,26 +219,26 @@ def test_render_expansion_lines_inlinks_only_with_predicate_and_anchor():
|
|||
assert lines == [
|
||||
" inlinks (1):",
|
||||
" ← src.md (no meta)",
|
||||
" via predicate=references, anchor=#intro",
|
||||
" via anchor=#intro",
|
||||
]
|
||||
print("✓ test_render_expansion_lines_inlinks_only_with_predicate_and_anchor passed")
|
||||
print("✓ test_render_expansion_lines_inlinks_only_with_anchor passed")
|
||||
|
||||
|
||||
def test_render_expansion_lines_both_directions_in_order():
|
||||
"""outlinks block precedes inlinks block."""
|
||||
expansion = {
|
||||
"outlinks": [
|
||||
{"path": "out.md", "meta": {"name": "Out"}, "edges": [{"predicate": None, "anchor": None}]},
|
||||
{"path": "out.md", "meta": {"name": "Out"}, "anchors": []},
|
||||
],
|
||||
"inlinks": [
|
||||
{"path": "in.md", "meta": {"description": "incoming"}, "edges": [{"predicate": None, "anchor": None}]},
|
||||
{"path": "in.md", "meta": {"description": "incoming"}, "anchors": []},
|
||||
],
|
||||
}
|
||||
lines = render_expansion_lines(expansion)
|
||||
assert lines[0] == " outlinks (1):"
|
||||
assert lines[3] == " inlinks (1):"
|
||||
assert lines[2] == " inlinks (1):"
|
||||
assert lines[1].lstrip().startswith("→")
|
||||
assert lines[4].lstrip().startswith("←")
|
||||
assert lines[3].lstrip().startswith("←")
|
||||
print("✓ test_render_expansion_lines_both_directions_in_order passed")
|
||||
|
||||
|
||||
|
|
@ -250,5 +249,5 @@ if __name__ == "__main__":
|
|||
test_expand_links_node_without_meta_returns_empty_meta_dict()
|
||||
test_render_expansion_lines_empty_input_yields_empty_list()
|
||||
test_render_expansion_lines_outlinks_only()
|
||||
test_render_expansion_lines_inlinks_only_with_predicate_and_anchor()
|
||||
test_render_expansion_lines_inlinks_only_with_anchor()
|
||||
test_render_expansion_lines_both_directions_in_order()
|
||||
|
|
|
|||
|
|
@ -284,9 +284,9 @@ def test_parse_links_literal_targets():
|
|||
path = _write_md(tmp, "note.md", body)
|
||||
chunker = MarkdownFileChunker()
|
||||
node, _ = await chunker.chunk(path)
|
||||
triples = {(link.target_path, link.target_anchor, link.predicate) for link in node.links}
|
||||
assert ("topics/Alice.md", None, None) in triples
|
||||
assert ("topics/Bob.md", "sec", None) in triples
|
||||
pairs = {(link.target_path, link.target_anchor) for link in node.links}
|
||||
assert ("topics/Alice.md", None) in pairs
|
||||
assert ("topics/Bob.md", "sec") in pairs
|
||||
# source_path always equals the node's own path
|
||||
for link in node.links:
|
||||
assert link.source_path == node.path
|
||||
|
|
@ -317,8 +317,8 @@ def test_parse_links_short_and_no_ext_kept_literally():
|
|||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_parse_links_predicate_inline_and_line():
|
||||
"""Both `pred:: [[X]]` (line-level) and `[pred:: [[X]]]` (inline) propagate predicate."""
|
||||
def test_parse_links_legacy_relation_text_is_ignored():
|
||||
"""Old relation wrappers do not prevent their inner wikilinks from parsing."""
|
||||
|
||||
async def run():
|
||||
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
||||
|
|
@ -326,16 +326,14 @@ def test_parse_links_predicate_inline_and_line():
|
|||
path = _write_md(tmp, "note.md", body)
|
||||
chunker = MarkdownFileChunker()
|
||||
node, _ = await chunker.chunk(path)
|
||||
pairs = {(link.target_path, link.predicate) for link in node.links}
|
||||
assert ("A.md", "extends") in pairs
|
||||
assert ("B.md", "concerns") in pairs
|
||||
print("✓ test_parse_links_predicate_inline_and_line passed")
|
||||
assert {link.target_path for link in node.links} == {"A.md", "B.md"}
|
||||
print("✓ test_parse_links_legacy_relation_text_is_ignored passed")
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_parse_links_deduped():
|
||||
"""Repeated wikilinks with the same (target, predicate, anchor) emit one FileLink."""
|
||||
"""Repeated wikilinks with the same target and anchor emit one FileLink."""
|
||||
|
||||
async def run():
|
||||
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
||||
|
|
@ -616,7 +614,7 @@ if __name__ == "__main__":
|
|||
test_parse_chunk_ids_match_node_chunk_ids()
|
||||
test_parse_links_literal_targets()
|
||||
test_parse_links_short_and_no_ext_kept_literally()
|
||||
test_parse_links_predicate_inline_and_line()
|
||||
test_parse_links_legacy_relation_text_is_ignored()
|
||||
test_parse_links_deduped()
|
||||
test_parse_min_chunk_byte_size_clamped()
|
||||
test_parse_embed_toc_prefixes_chunk_text()
|
||||
|
|
|
|||
|
|
@ -141,6 +141,28 @@ def test_read_with_neighbors_injects_block_and_metadata():
|
|||
_run(run())
|
||||
|
||||
|
||||
def test_read_with_neighbors_uses_posix_nested_path():
|
||||
"""Nested graph keys use POSIX separators on every platform."""
|
||||
|
||||
async def run():
|
||||
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
||||
store = await _store_with(
|
||||
{
|
||||
"notes/A.md": {"body": "See [[notes/B.md]].", "name": "A Doc"},
|
||||
"notes/B.md": {"body": "End node.", "name": "B Doc"},
|
||||
},
|
||||
)
|
||||
resp = await _read(store, step_kwargs={"with_neighbors": True}, path="notes/A.md")
|
||||
|
||||
expansion = resp.metadata.get("link_expansion")
|
||||
assert expansion is not None
|
||||
assert "notes/A.md" in expansion
|
||||
assert expansion["notes/A.md"]["outlinks"][0]["path"] == "notes/B.md"
|
||||
await store.close()
|
||||
|
||||
_run(run())
|
||||
|
||||
|
||||
# -- on + zero neighbors -----------------------------------------------------
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
"""Tests for the wikilink helpers in ``reme.utils.wikilink_handler``.
|
||||
"""Tests for wikilink helpers in ``reme.utils.wikilink_handler``.
|
||||
|
||||
Two pure async helpers used by file_move / file_delete:
|
||||
|
||||
|
|
@ -196,8 +196,8 @@ def test_retarget_image_marker_preserved():
|
|||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_retarget_dataview_predicate_preserved():
|
||||
"""Line-level + inline-bracketed Dataview predicates pass through outside ``[[..]]``."""
|
||||
def test_retarget_preserves_surrounding_relation_text():
|
||||
"""Retargeting changes only the inner wikilink target."""
|
||||
|
||||
async def run():
|
||||
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
||||
|
|
@ -212,11 +212,33 @@ def test_retarget_dataview_predicate_preserved():
|
|||
assert "colleague:: [[people/Alice.md]]" in body
|
||||
assert "[负责:: [[people/Alice.md]]]" in body
|
||||
await store.close()
|
||||
print("✓ test_retarget_dataview_predicate_preserved passed")
|
||||
print("✓ test_retarget_preserves_surrounding_relation_text passed")
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_markdown_links_are_ignored():
|
||||
"""Only double-bracket wikilinks create graph edges or get rewritten."""
|
||||
markdown_links = (
|
||||
"[label](../wiki/example.md) "
|
||||
"[label](../wiki/example.md#section) "
|
||||
"[label](../wiki/example.md#L9-L10,L15-L20)"
|
||||
)
|
||||
original = f"{markdown_links} [[wiki/example.md#L9]]"
|
||||
|
||||
rewritten, count = WikilinkHandler.scan_and_rewrite(
|
||||
original,
|
||||
old="wiki/example.md",
|
||||
new="archive/example.md",
|
||||
)
|
||||
|
||||
assert [(link.target_path, link.target_anchor) for link in WikilinkHandler.extract_links(original, "note.md")] == [
|
||||
("wiki/example.md", "L9"),
|
||||
]
|
||||
assert count == 1
|
||||
assert rewritten == f"{markdown_links} [[archive/example.md#L9]]"
|
||||
|
||||
|
||||
def test_retarget_multiple_files_aggregate_counts():
|
||||
"""links_changed sums across files; by_file lists per-file counts."""
|
||||
|
||||
|
|
@ -387,7 +409,8 @@ if __name__ == "__main__":
|
|||
test_retarget_alias_preserved()
|
||||
test_retarget_anchor_and_alias_together()
|
||||
test_retarget_image_marker_preserved()
|
||||
test_retarget_dataview_predicate_preserved()
|
||||
test_retarget_preserves_surrounding_relation_text()
|
||||
test_markdown_links_are_ignored()
|
||||
test_retarget_multiple_files_aggregate_counts()
|
||||
test_retarget_dry_run_does_not_write()
|
||||
test_retarget_scope_limits_sweep()
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue