mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-10-31 06:56:07 +00:00
Merge remote-tracking branch 'upstream/main' into organize-bench
This commit is contained in:
commit
dca1cf035b
20 changed files with 1124 additions and 111 deletions
|
|
@ -59,6 +59,8 @@ memory, then continuously indexes, links, and consolidates that memory for futur
|
|||
|
||||
## 📰 News
|
||||
|
||||
- [2026.08] - Experience-driven enhancement of agent tool-use execution built on ReMe is available on
|
||||
[arXiv:2608.03403](https://arxiv.org/abs/2608.03403).
|
||||
- [2026.07] - Introduced optional Cookbooks: [Daily Paper](cookbook/daily_paper/README.md) for paper discovery and
|
||||
analysis, and [Auto Fin](cookbook/auto-fin/README.md) for file-native ETF event research based on CLS news and
|
||||
historical market reactions.
|
||||
|
|
|
|||
|
|
@ -52,6 +52,8 @@ Agent 能够可靠召回。
|
|||
|
||||
## 📰 新闻
|
||||
|
||||
- [2026.08] - 基于 ReMe 的智能体工具使用执行增强工作见
|
||||
[arXiv:2608.03403](https://arxiv.org/abs/2608.03403)。
|
||||
- [2026.07] - 新增可选 Cookbook 工作流:[每日论文](cookbook/daily_paper/README_ZH.md)用于论文发现与解析,
|
||||
[Auto Fin](cookbook/auto-fin/README_ZH.md)用于结合财联社新闻和历史行情开展文件化 ETF 事件研究。
|
||||
- [2026.07] -
|
||||
|
|
|
|||
98
benchmark/toolmemory/README.md
Normal file
98
benchmark/toolmemory/README.md
Normal file
|
|
@ -0,0 +1,98 @@
|
|||
## Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance
|
||||
|
||||
**Language**: English (default) / [中文](./README_ZH.md)
|
||||
|
||||
> Paper: [arXiv:2608.03403](https://arxiv.org/abs/2608.03403)
|
||||
> Code: [https://github.com/WangCan1178/ExpG](https://github.com/WangCan1178/ExpG)
|
||||
|
||||
<p align="center">
|
||||
<img src="gitcha.png" alt="ExpG challenges and overview" width="85%">
|
||||
</p>
|
||||
|
||||
### Overview
|
||||
|
||||
This folder archives **ExpG**, a tool-use enhancement built on [Agentscope ReMe](https://github.com/agentscope-ai/ReMe). ExpG mines, distills, and reuses experience from historical tool calls to provide **capability boundaries** and **best-practice guidance**, which helps agents:
|
||||
|
||||
- Select and invoke tools more robustly under dynamic or noisy environments;
|
||||
- Let smaller models with guidance outperform larger, memoryless baselines;
|
||||
- Improve consistently across tool selection, tool calling, and response generation.
|
||||
|
||||
**How ReMe is used:** Start the Tool Memory service; historical tool calls are written and evaluated via `add_tool_call_result`, distilled into tool-level guidance via `summary_tool_memory`, then retrieved and injected into later reasoning via `retrieve_tool_memory`. ReMe provides the vector store and service APIs; the acquisition / distillation / reuse strategy is implemented by ExpG. Full implementation and experiments are in [WangCan1178/ExpG](https://github.com/WangCan1178/ExpG).
|
||||
|
||||
---
|
||||
|
||||
### ExpG Mechanism
|
||||
|
||||
ExpG treats tool invocations as learnable experience and runs a three-stage pipeline:
|
||||
|
||||
1. **Experience Acquisition**
|
||||
- Analyze invocation quality from historical trajectories (success/failure, cost, latency, etc.);
|
||||
- Build structured experience units per tool, recording context, parameter patterns, and outcomes.
|
||||
|
||||
2. **Experience Distillation**
|
||||
- Filter noisy or unhelpful experiences and keep representative patterns;
|
||||
- Aggregate by equivalence classes to cover common and rare failure modes;
|
||||
- Summarize with an LLM into generalizable textual guidance.
|
||||
|
||||
3. **Experience Reuse**
|
||||
- Retrieve relevant experience / guidance for future tasks;
|
||||
- Inject guidance into tool selection, argument generation, and response synthesis;
|
||||
- Improve stability under dynamic environments and imperfect feedback.
|
||||
|
||||
---
|
||||
|
||||
### Main Results
|
||||
|
||||
Performance comparison (%) across MetaTool, API-Bank, and BFCL-V3. **Bold** indicates the best results within each model.
|
||||
|
||||
| Model | Method | MetaTool Pass@1 | MetaTool Avg@3 | MetaTool Pass@3 | API-Bank Pass@1 | API-Bank Avg@3 | API-Bank Pass@3 | BFCL-V3 Pass@1 | BFCL-V3 Avg@3 | BFCL-V3 Pass@3 | Total Pass@1 | Total Avg@3 | Total Pass@3 |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| GPT-5 nano | No Method | 72.62 | 72.76 | 78.49 | 82.96 | 83.46 | 86.97 | 53.80 | 53.00 | 60.95 | 70.82 | 70.62 | 76.63 |
|
||||
| GPT-5 nano | Few-shot | 74.12 | 75.11 | 82.32 | 83.71 | 83.96 | **87.22** | 56.18 | 55.24 | 61.39 | 72.36 | 72.65 | 79.28 |
|
||||
| GPT-5 nano | DRAFT | 73.94 | 73.04 | 78.97 | 84.21 | 83.46 | **87.22** | 57.27 | 57.27 | 62.26 | 72.52 | 71.58 | 77.23 |
|
||||
| GPT-5 nano | Mem0 | 74.96 | 76.13 | 82.92 | 84.96 | 85.21 | **87.22** | 60.95 | 61.61 | 65.08 | 73.98 | 74.67 | 80.35 |
|
||||
| GPT-5 nano | **ExpG** | **81.67** | **82.07** | **84.60** | **86.72** | **86.55** | **87.22** | **64.43** | **63.99** | **66.38** | **79.32** | **79.22** | **81.69** |
|
||||
| DeepSeek-V3 | No Method | 83.10 | 82.94 | 84.66 | 84.71 | 84.38 | 85.46 | 58.79 | 59.65 | 65.94 | 78.92 | 78.66 | 81.37 |
|
||||
| DeepSeek-V3 | Few-shot | 82.74 | 83.90 | 86.28 | 85.21 | 84.63 | 86.22 | 60.52 | 60.30 | 67.90 | 79.08 | 79.45 | 82.92 |
|
||||
| DeepSeek-V3 | DRAFT | 80.23 | 80.79 | 82.44 | 84.96 | 85.63 | 86.47 | 62.26 | 61.61 | 68.55 | 77.70 | 77.80 | 80.54 |
|
||||
| DeepSeek-V3 | Mem0 | 83.88 | 84.56 | 86.40 | 85.46 | 85.55 | 86.47 | 65.08 | 65.15 | 68.33 | 80.70 | 80.91 | 83.12 |
|
||||
| DeepSeek-V3 | **ExpG** | **85.26** | **85.38** | **86.52** | **87.72** | **87.39** | **87.97** | **69.41** | **69.92** | **72.02** | **82.76** | **82.61** | **84.11** |
|
||||
| Qwen3-8B | No Method | 76.51 | 76.97 | 77.71 | 83.96 | 83.88 | 84.21 | 58.79 | 58.28 | 60.30 | 74.46 | 74.41 | 75.56 |
|
||||
| Qwen3-8B | Few-shot | 79.93 | 79.83 | 82.92 | 83.71 | 82.62 | 84.96 | 60.09 | 59.29 | 61.39 | 76.91 | 76.27 | 79.32 |
|
||||
| Qwen3-8B | DRAFT | 78.19 | 77.33 | 77.89 | 85.71 | 84.96 | 85.46 | 60.74 | 60.30 | 62.91 | 76.20 | 75.18 | 76.35 |
|
||||
| Qwen3-8B | Mem0 | 75.07 | 75.47 | 82.38 | 86.22 | 86.05 | 86.47 | 63.34 | 64.93 | 66.16 | 74.69 | 74.98 | 80.07 |
|
||||
| Qwen3-8B | **ExpG** | **83.52** | **84.88** | **85.08** | **86.47** | **87.89** | **87.97** | **67.46** | **66.96** | **68.33** | **81.06** | **81.82** | **82.48** |
|
||||
| Qwen3-32B | No Method | 80.05 | 79.43 | 80.17 | 84.71 | 84.88 | 85.21 | 65.15 | 65.08 | 66.16 | 78.05 | 77.55 | 78.41 |
|
||||
| Qwen3-32B | **ExpG** | **84.68** | **85.02** | **86.28** | **86.97** | **87.30** | **87.72** | **70.72** | **71.01** | **73.32** | **82.48** | **82.56** | **84.14** |
|
||||
| Qwen3-235B | No Method | 78.25 | 79.23 | 80.29 | 85.46 | 85.46 | 85.71 | 71.37 | 71.15 | 73.54 | 78.13 | 78.49 | 79.91 |
|
||||
| Qwen3-235B | **ExpG** | **86.34** | **86.70** | **86.94** | **87.47** | **86.97** | **88.22** | **79.61** | **78.52** | **80.04** | **85.29** | **84.98** | **85.69** |
|
||||
|
||||
---
|
||||
|
||||
### Reference Code
|
||||
|
||||
| Path | Role |
|
||||
| --- | --- |
|
||||
| [`tool_memory.py`](./tool_memory.py) | HTTP client for official ReMe Tool Memory APIs (`add_tool_call_result` / `summary_tool_memory` / `retrieve_tool_memory`) |
|
||||
| [`parse_tool_call_result_prompt.yaml`](./parse_tool_call_result_prompt.yaml) | Prompt for multi-aspect evaluation of each tool call |
|
||||
| [`summary_tool_memory_prompt.yaml`](./summary_tool_memory_prompt.yaml) | Prompt for summarizing tool call history into guidance |
|
||||
| [`tool_memory_flows.yaml`](./tool_memory_flows.yaml) | Tool Memory flow / op config excerpt |
|
||||
|
||||
These are reference snippets. For the full runnable codebase, see [WangCan1178/ExpG](https://github.com/WangCan1178/ExpG).
|
||||
|
||||
---
|
||||
|
||||
### Citation
|
||||
|
||||
```bibtex
|
||||
@misc{wang2026expg,
|
||||
title = {Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance},
|
||||
author = {Can Wang and Haoran Chen and Li Yu and Ding Hao and Bohai Zhao and Zhaoyang Liu and Zhiying Tu},
|
||||
year = {2026},
|
||||
eprint = {2608.03403},
|
||||
archivePrefix = {arXiv},
|
||||
primaryClass = {cs.AI},
|
||||
url = {https://arxiv.org/abs/2608.03403},
|
||||
howpublished = {\url{https://github.com/WangCan1178/ExpG}}
|
||||
}
|
||||
```
|
||||
98
benchmark/toolmemory/README_ZH.md
Normal file
98
benchmark/toolmemory/README_ZH.md
Normal file
|
|
@ -0,0 +1,98 @@
|
|||
## Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance
|
||||
|
||||
**语言**:中文 / [English](./README.md)
|
||||
|
||||
> 论文:[arXiv:2608.03403](https://arxiv.org/abs/2608.03403)
|
||||
> 代码:[https://github.com/WangCan1178/ExpG](https://github.com/WangCan1178/ExpG)
|
||||
|
||||
<p align="center">
|
||||
<img src="gitcha.png" alt="ExpG 挑战与概览" width="85%">
|
||||
</p>
|
||||
|
||||
### 简介
|
||||
|
||||
本目录归档基于 [Agentscope ReMe](https://github.com/agentscope-ai/ReMe) 的工具使用增强工作 **ExpG**:在 ReMe 记忆框架之上,从历史工具调用中挖掘、提炼并复用经验,为智能体提供工具的 **能力边界** 与 **最佳实践指导**,从而:
|
||||
|
||||
- 在动态或有噪环境下更鲁棒地选择和调用工具;
|
||||
- 让较小模型在带有经验指导时超越更大、但无记忆的基线;
|
||||
- 在工具选择、工具调用和响应生成等多个阶段带来一致收益。
|
||||
|
||||
**如何使用 ReMe:** 启动 Tool Memory 服务后,历史工具调用经 `add_tool_call_result` 写入并评估,经 `summary_tool_memory` 蒸馏成工具级指导,再经 `retrieve_tool_memory` 取回并注入后续推理。向量存储与服务接口由 ReMe 提供,经验获取 / 蒸馏 / 复用策略由 ExpG 实现。完整实现与实验见 [WangCan1178/ExpG](https://github.com/WangCan1178/ExpG)。
|
||||
|
||||
---
|
||||
|
||||
### ExpG 机制概览
|
||||
|
||||
ExpG 将工具调用视为可学习经验,并通过三阶段流水线完成经验的获取、提炼与复用:
|
||||
|
||||
1. **经验获取(Experience Acquisition)**
|
||||
- 从历史工具调用轨迹中分析调用质量(成功/失败、代价、时间等);
|
||||
- 针对不同工具构建结构化的经验单元,记录调用上下文、参数模式和结果。
|
||||
|
||||
2. **经验蒸馏(Experience Distillation)**
|
||||
- 过滤无效 / 噪声经验,保留具有代表性的调用模式;
|
||||
- 基于“等价类”视角对经验进行聚合,覆盖常见模式与稀有失败模式;
|
||||
- 使用 LLM 对经验进行总结,形成可泛化的文本化指导(guidance)。
|
||||
|
||||
3. **经验复用(Experience Reuse)**
|
||||
- 在未来任务中,根据当前工具调用上下文检索相关经验 / 指导;
|
||||
- 将经验引导融入到工具选择、参数生成和响应整理等环节;
|
||||
- 使得代理在面对动态环境和不完美反馈时仍能保持稳定表现。
|
||||
|
||||
---
|
||||
|
||||
### 主实验结果
|
||||
|
||||
MetaTool、API-Bank、BFCL-V3 上的性能对比(%)。**加粗**为各模型组内最优。
|
||||
|
||||
| Model | Method | MetaTool Pass@1 | MetaTool Avg@3 | MetaTool Pass@3 | API-Bank Pass@1 | API-Bank Avg@3 | API-Bank Pass@3 | BFCL-V3 Pass@1 | BFCL-V3 Avg@3 | BFCL-V3 Pass@3 | Total Pass@1 | Total Avg@3 | Total Pass@3 |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| GPT-5 nano | No Method | 72.62 | 72.76 | 78.49 | 82.96 | 83.46 | 86.97 | 53.80 | 53.00 | 60.95 | 70.82 | 70.62 | 76.63 |
|
||||
| GPT-5 nano | Few-shot | 74.12 | 75.11 | 82.32 | 83.71 | 83.96 | **87.22** | 56.18 | 55.24 | 61.39 | 72.36 | 72.65 | 79.28 |
|
||||
| GPT-5 nano | DRAFT | 73.94 | 73.04 | 78.97 | 84.21 | 83.46 | **87.22** | 57.27 | 57.27 | 62.26 | 72.52 | 71.58 | 77.23 |
|
||||
| GPT-5 nano | Mem0 | 74.96 | 76.13 | 82.92 | 84.96 | 85.21 | **87.22** | 60.95 | 61.61 | 65.08 | 73.98 | 74.67 | 80.35 |
|
||||
| GPT-5 nano | **ExpG** | **81.67** | **82.07** | **84.60** | **86.72** | **86.55** | **87.22** | **64.43** | **63.99** | **66.38** | **79.32** | **79.22** | **81.69** |
|
||||
| DeepSeek-V3 | No Method | 83.10 | 82.94 | 84.66 | 84.71 | 84.38 | 85.46 | 58.79 | 59.65 | 65.94 | 78.92 | 78.66 | 81.37 |
|
||||
| DeepSeek-V3 | Few-shot | 82.74 | 83.90 | 86.28 | 85.21 | 84.63 | 86.22 | 60.52 | 60.30 | 67.90 | 79.08 | 79.45 | 82.92 |
|
||||
| DeepSeek-V3 | DRAFT | 80.23 | 80.79 | 82.44 | 84.96 | 85.63 | 86.47 | 62.26 | 61.61 | 68.55 | 77.70 | 77.80 | 80.54 |
|
||||
| DeepSeek-V3 | Mem0 | 83.88 | 84.56 | 86.40 | 85.46 | 85.55 | 86.47 | 65.08 | 65.15 | 68.33 | 80.70 | 80.91 | 83.12 |
|
||||
| DeepSeek-V3 | **ExpG** | **85.26** | **85.38** | **86.52** | **87.72** | **87.39** | **87.97** | **69.41** | **69.92** | **72.02** | **82.76** | **82.61** | **84.11** |
|
||||
| Qwen3-8B | No Method | 76.51 | 76.97 | 77.71 | 83.96 | 83.88 | 84.21 | 58.79 | 58.28 | 60.30 | 74.46 | 74.41 | 75.56 |
|
||||
| Qwen3-8B | Few-shot | 79.93 | 79.83 | 82.92 | 83.71 | 82.62 | 84.96 | 60.09 | 59.29 | 61.39 | 76.91 | 76.27 | 79.32 |
|
||||
| Qwen3-8B | DRAFT | 78.19 | 77.33 | 77.89 | 85.71 | 84.96 | 85.46 | 60.74 | 60.30 | 62.91 | 76.20 | 75.18 | 76.35 |
|
||||
| Qwen3-8B | Mem0 | 75.07 | 75.47 | 82.38 | 86.22 | 86.05 | 86.47 | 63.34 | 64.93 | 66.16 | 74.69 | 74.98 | 80.07 |
|
||||
| Qwen3-8B | **ExpG** | **83.52** | **84.88** | **85.08** | **86.47** | **87.89** | **87.97** | **67.46** | **66.96** | **68.33** | **81.06** | **81.82** | **82.48** |
|
||||
| Qwen3-32B | No Method | 80.05 | 79.43 | 80.17 | 84.71 | 84.88 | 85.21 | 65.15 | 65.08 | 66.16 | 78.05 | 77.55 | 78.41 |
|
||||
| Qwen3-32B | **ExpG** | **84.68** | **85.02** | **86.28** | **86.97** | **87.30** | **87.72** | **70.72** | **71.01** | **73.32** | **82.48** | **82.56** | **84.14** |
|
||||
| Qwen3-235B | No Method | 78.25 | 79.23 | 80.29 | 85.46 | 85.46 | 85.71 | 71.37 | 71.15 | 73.54 | 78.13 | 78.49 | 79.91 |
|
||||
| Qwen3-235B | **ExpG** | **86.34** | **86.70** | **86.94** | **87.47** | **86.97** | **88.22** | **79.61** | **78.52** | **80.04** | **85.29** | **84.98** | **85.69** |
|
||||
|
||||
---
|
||||
|
||||
### 参考代码
|
||||
|
||||
| 路径 | 作用 |
|
||||
| --- | --- |
|
||||
| [`tool_memory.py`](./tool_memory.py) | 官方风格 ReMe Tool Memory HTTP 客户端(`add_tool_call_result` / `summary_tool_memory` / `retrieve_tool_memory`) |
|
||||
| [`parse_tool_call_result_prompt.yaml`](./parse_tool_call_result_prompt.yaml) | 单次工具调用多维评估用的 prompt |
|
||||
| [`summary_tool_memory_prompt.yaml`](./summary_tool_memory_prompt.yaml) | 将工具调用历史总结为 guidance 的 prompt |
|
||||
| [`tool_memory_flows.yaml`](./tool_memory_flows.yaml) | Tool Memory 相关的 flow / op 配置摘录 |
|
||||
|
||||
以上为参考片段。完整可运行代码见 [WangCan1178/ExpG](https://github.com/WangCan1178/ExpG)。
|
||||
|
||||
---
|
||||
|
||||
### 引用
|
||||
|
||||
```bibtex
|
||||
@misc{wang2026expg,
|
||||
title = {Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance},
|
||||
author = {Can Wang and Haoran Chen and Li Yu and Ding Hao and Bohai Zhao and Zhaoyang Liu and Zhiying Tu},
|
||||
year = {2026},
|
||||
eprint = {2608.03403},
|
||||
archivePrefix = {arXiv},
|
||||
primaryClass = {cs.AI},
|
||||
url = {https://arxiv.org/abs/2608.03403},
|
||||
howpublished = {\url{https://github.com/WangCan1178/ExpG}}
|
||||
}
|
||||
```
|
||||
BIN
benchmark/toolmemory/gitcha.png
Normal file
BIN
benchmark/toolmemory/gitcha.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.9 MiB |
49
benchmark/toolmemory/parse_tool_call_result_prompt.yaml
Normal file
49
benchmark/toolmemory/parse_tool_call_result_prompt.yaml
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
prompt: |
|
||||
You are an expert in evaluating tool invocation process. The tool is invoked by an AI agent.
|
||||
|
||||
Tool invocation Information:
|
||||
- Tool Name: {tool_name}
|
||||
- Success Flag: {success_flag}
|
||||
- Time Cost: {time_cost}s
|
||||
- Token Cost: {token_cost} tokens
|
||||
- Agent Context: {context}
|
||||
- Input Parameters: {input_params}
|
||||
- Tool Response: {response}
|
||||
- Tool Schema: {schema}
|
||||
|
||||
Evaluation Method:
|
||||
Start from a default score list of scores = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0].
|
||||
For each item below that is satisfied, assign 1 point to the corresponding index.
|
||||
The final scores should be a list of 10 integers, each being either 0 or 1.
|
||||
|
||||
1. Use Quality (total 2 points. If context is provided, use it as an aid when evaluating):
|
||||
- Index 1: Should the tool be invoked now? Consider whether all necessary information for the tool's invocation is ready, and whether the tool execution environment is correct. If it is a multi-round conversation, also consider the dependency relationships of the tool chain.
|
||||
- Index 2: If should, is the chosen tool appropriate?
|
||||
|
||||
2. Input Quality (total 4 points. When evaluating, consider both the context and the tool schema):
|
||||
- Index 3: Are all required parameters provided?
|
||||
- Index 4: Are the input parameters valid and supported by the tool?
|
||||
- Index 5: Are the input parameters in the correct format for their respective fields?
|
||||
- Index 6: Does the value (content) of input parameter correctly reflect and match the given context?
|
||||
|
||||
3. Response Quality (total 4 points):
|
||||
- Index 7: Does the response provide meaningful and useful information? Or are there any error messages or information that can be used as guidance for agent invoking tool better?
|
||||
- Index 8: Does the response match the tool's intended purpose/function?
|
||||
- Index 9: Does the response value correct (content appropriate) given the input parameters?
|
||||
- Index 10: Does the response help accomplish the task within the given context?
|
||||
|
||||
Important:
|
||||
1. Sometimes there is not enough information in the context or schema to make a complete evaluation. In such cases, make your best judgment based on the available information.
|
||||
2. Some tools (commonly system tools such as mkdir, touch, echo, etc.) modify the external environment. Since these results cannot be obtained, they return "None" as the response. At this point, all the scores in the quality of the response should be obtained and should not be seen as a problem for the tool.
|
||||
3. Evaluation independently from the success flag. The success_flag indicates whether the tool executed without technical errors. The evaluation should evaluate the quality of the tool invocation. A tool can execute successfully (Success Flag=1) but still produce low-quality or irrelevant responses, leading to a low evaluation score.
|
||||
4. Sometimes an agent will execute multiple steps and invoke multiple tools to complete a task, but you only need to evaluate the use of one tool for one of the steps, not whether the final task is completed or not.
|
||||
|
||||
Answer Format:
|
||||
Please provide your answer in the following JSON format:
|
||||
|
||||
```json
|
||||
{
|
||||
"scores": [0,0,0,0,0,0,0,0,0,0],
|
||||
"explanation": "A brief evaluation (2-3 sentences) explaining the quality of the tool invocation, based on your evaluation. Low-quality aspects need to be reified, especially the causes of tool invocation errors."
|
||||
}
|
||||
```
|
||||
32
benchmark/toolmemory/summary_tool_memory_prompt.yaml
Normal file
32
benchmark/toolmemory/summary_tool_memory_prompt.yaml
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
prompt: |
|
||||
You are an expert in analyzing tool usage patterns and generating practical usage guidance for agents.
|
||||
|
||||
Tool Information:
|
||||
- Tool Name: {tool_name}
|
||||
- Tool Schema: {tool_schema}
|
||||
|
||||
Recent Tool Invocation Experiences:
|
||||
{experiences}
|
||||
|
||||
Important:
|
||||
1. Assume the tool (tool schema) can't be changed, your task is to guide agent to use it better.
|
||||
2. Your answer must be based on the information given, don't make it up. If not enough data, state "Not enough data to determine Core Function/Success Patterns/Common Issues/Best Practices."
|
||||
3. Your answer will be used to guide the use of the tool in the future, so do not include content related to recent tool invocation experience such as "case #3" or "Call #2", but some values can be used as examples.
|
||||
4. Pay attention to information not mentioned in the tool schema, such as the response upon successful tool invocation. It's also welcome to uncover insights, such as how tools can be used more effectively, and possible dependencies between tools. But if they aren't, don't make them up.
|
||||
5. Finally, to avoid deriving incorrect guidance from individual invocation, check whether, if the agent follows the proposed guidance, it can perform better on all recent invocation histories. If not, revise the guidance until it can. Specifically:
|
||||
- Don't write guidance in an absolute tone without a very deterministic message (meaning that all invocation histories are satisfied, otherwise it will result in failure).
|
||||
- Sometimes there may be inconsistencies. Consider whether this is due to the context in which the tool is being used.
|
||||
|
||||
Your Task:
|
||||
Based on the tool invocation history, generate a concise and logical tool usage guidance following this structure:
|
||||
1. Core Function: What this tool does and when to use it.
|
||||
2. Success Patterns: Parameter patterns and usage scenarios that work well.
|
||||
3. Common Issues: Main pitfalls to avoid and why they fail.
|
||||
4. Best Practices: 2-3 actionable recommendations.
|
||||
|
||||
Answer Format:
|
||||
Provide a structured, concise guidance (max 200 words). Focus on actionable insights derived from actual usage data. Avoid generic advice and think step by step.
|
||||
|
||||
```txt
|
||||
Your concise, data-driven tool usage guidance
|
||||
```
|
||||
234
benchmark/toolmemory/tool_memory.py
Normal file
234
benchmark/toolmemory/tool_memory.py
Normal file
|
|
@ -0,0 +1,234 @@
|
|||
"""Official-style ReMe Tool Memory HTTP helpers.
|
||||
|
||||
Aligned with ReMe Tool Memory HTTP APIs (see ReMe cookbook
|
||||
``use_tool_memory_demo.py`` and docs under ``docs/tool_memory/``):
|
||||
|
||||
- ``add_tool_call_result``
|
||||
- ``summary_tool_memory``
|
||||
- ``retrieve_tool_memory``
|
||||
|
||||
Response memories are read from ``metadata.memory_list[].content``.
|
||||
This module does not use ExpG-only fields such as ``no_persist``,
|
||||
``source_task``, or ``add_to``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_BASE_URL = "http://localhost:8002"
|
||||
|
||||
|
||||
class ToolMemoryFetcher:
|
||||
"""HTTP client for ReMe Tool Memory endpoints."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
workspace_id: str,
|
||||
base_url: str = DEFAULT_BASE_URL,
|
||||
timeout: float = 60.0,
|
||||
) -> None:
|
||||
self.workspace_id = workspace_id
|
||||
self.base_url = base_url.rstrip("/")
|
||||
self.timeout = timeout
|
||||
|
||||
def _url(self, endpoint: str) -> str:
|
||||
return f"{self.base_url}/{endpoint.lstrip('/')}"
|
||||
|
||||
@staticmethod
|
||||
def _join_tool_names(tool_names: List[str] | str) -> str:
|
||||
if isinstance(tool_names, str):
|
||||
return tool_names
|
||||
return ",".join(tool_names)
|
||||
|
||||
@staticmethod
|
||||
def _memory_list(payload: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
metadata = payload.get("metadata") or {}
|
||||
if not isinstance(metadata, dict):
|
||||
return []
|
||||
memory_list = metadata.get("memory_list") or []
|
||||
return memory_list if isinstance(memory_list, list) else []
|
||||
|
||||
@classmethod
|
||||
def _content_by_tool(cls, payload: Dict[str, Any]) -> Dict[str, str]:
|
||||
result: Dict[str, str] = {}
|
||||
for memory in cls._memory_list(payload):
|
||||
if not isinstance(memory, dict):
|
||||
continue
|
||||
tool_name = str(memory.get("when_to_use") or "").strip()
|
||||
content = memory.get("content") or ""
|
||||
if tool_name:
|
||||
result[tool_name] = str(content)
|
||||
return result
|
||||
|
||||
async def add_tool_call_result_async(
|
||||
self,
|
||||
tool_call_results: List[Dict[str, Any]],
|
||||
) -> Dict[str, Any]:
|
||||
"""Call ``add_tool_call_result``."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
self._url("add_tool_call_result"),
|
||||
json={
|
||||
"workspace_id": self.workspace_id,
|
||||
"tool_call_results": tool_call_results,
|
||||
},
|
||||
timeout=self.timeout,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def summary_tool_memory_async(
|
||||
self,
|
||||
tool_names: List[str] | str,
|
||||
) -> Dict[str, Any]:
|
||||
"""Call ``summary_tool_memory``."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
self._url("summary_tool_memory"),
|
||||
json={
|
||||
"workspace_id": self.workspace_id,
|
||||
"tool_names": self._join_tool_names(tool_names),
|
||||
},
|
||||
timeout=self.timeout,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def retrieve_tool_memory_async(
|
||||
self,
|
||||
tool_names: List[str] | str,
|
||||
) -> Dict[str, Any]:
|
||||
"""Call ``retrieve_tool_memory``."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
self._url("retrieve_tool_memory"),
|
||||
json={
|
||||
"workspace_id": self.workspace_id,
|
||||
"tool_names": self._join_tool_names(tool_names),
|
||||
},
|
||||
timeout=self.timeout,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def collect_memory_async(
|
||||
self,
|
||||
tool_names: List[str],
|
||||
) -> Dict[str, str]:
|
||||
"""Summarize then retrieve guidance for tools.
|
||||
|
||||
Returns:
|
||||
Mapping from tool name to memory ``content`` string.
|
||||
"""
|
||||
if not tool_names:
|
||||
return {}
|
||||
|
||||
names = self._join_tool_names(tool_names)
|
||||
try:
|
||||
summary = await self.summary_tool_memory_async(names)
|
||||
if not summary.get("success"):
|
||||
logger.warning("summary_tool_memory failed for %s", names)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.warning("summary_tool_memory error for %s: %s", names, exc)
|
||||
|
||||
try:
|
||||
retrieved = await self.retrieve_tool_memory_async(names)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.warning("retrieve_tool_memory error for %s: %s", names, exc)
|
||||
return {}
|
||||
|
||||
if not retrieved.get("success"):
|
||||
logger.warning("retrieve_tool_memory failed for %s", names)
|
||||
return {}
|
||||
|
||||
return self._content_by_tool(retrieved)
|
||||
|
||||
def add_tool_call_result(
|
||||
self,
|
||||
tool_call_results: List[Dict[str, Any]],
|
||||
) -> Dict[str, Any]:
|
||||
"""Sync wrapper for ``add_tool_call_result``."""
|
||||
with httpx.Client() as client:
|
||||
response = client.post(
|
||||
self._url("add_tool_call_result"),
|
||||
json={
|
||||
"workspace_id": self.workspace_id,
|
||||
"tool_call_results": tool_call_results,
|
||||
},
|
||||
timeout=self.timeout,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
def summary_tool_memory(self, tool_names: List[str] | str) -> Dict[str, Any]:
|
||||
"""Sync wrapper for ``summary_tool_memory``."""
|
||||
with httpx.Client() as client:
|
||||
response = client.post(
|
||||
self._url("summary_tool_memory"),
|
||||
json={
|
||||
"workspace_id": self.workspace_id,
|
||||
"tool_names": self._join_tool_names(tool_names),
|
||||
},
|
||||
timeout=self.timeout,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
def retrieve_tool_memory(self, tool_names: List[str] | str) -> Dict[str, Any]:
|
||||
"""Sync wrapper for ``retrieve_tool_memory``."""
|
||||
with httpx.Client() as client:
|
||||
response = client.post(
|
||||
self._url("retrieve_tool_memory"),
|
||||
json={
|
||||
"workspace_id": self.workspace_id,
|
||||
"tool_names": self._join_tool_names(tool_names),
|
||||
},
|
||||
timeout=self.timeout,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
def collect_memory(self, tool_names: List[str]) -> Dict[str, str]:
|
||||
"""Sync wrapper for summarize + retrieve.
|
||||
|
||||
Prefer ``collect_memory_async`` inside an existing event loop.
|
||||
"""
|
||||
if not tool_names:
|
||||
return {}
|
||||
|
||||
names = self._join_tool_names(tool_names)
|
||||
try:
|
||||
summary = self.summary_tool_memory(names)
|
||||
if not summary.get("success"):
|
||||
logger.warning("summary_tool_memory failed for %s", names)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.warning("summary_tool_memory error for %s: %s", names, exc)
|
||||
|
||||
try:
|
||||
retrieved = self.retrieve_tool_memory(names)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.warning("retrieve_tool_memory error for %s: %s", names, exc)
|
||||
return {}
|
||||
|
||||
if not retrieved.get("success"):
|
||||
logger.warning("retrieve_tool_memory failed for %s", names)
|
||||
return {}
|
||||
|
||||
return self._content_by_tool(retrieved)
|
||||
|
||||
def get_memory_content(
|
||||
self,
|
||||
tool_names: List[str] | str,
|
||||
) -> Optional[str]:
|
||||
"""Retrieve and join memory contents for the given tools."""
|
||||
payload = self.retrieve_tool_memory(tool_names)
|
||||
if not payload.get("success"):
|
||||
return None
|
||||
contents = [content for content in self._content_by_tool(payload).values() if content]
|
||||
return "\n\n".join(contents) if contents else None
|
||||
45
benchmark/toolmemory/tool_memory_flows.yaml
Normal file
45
benchmark/toolmemory/tool_memory_flows.yaml
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
# Tool Memory flow / op config excerpt used by ExpG.
|
||||
# Full runnable code: https://github.com/WangCan1178/ExpG
|
||||
|
||||
flow:
|
||||
retrieve_tool_memory:
|
||||
flow_content: retrieve_tool_memory_op
|
||||
description: "Retrieves tool memories from the vector database based on tool names to provide tool usage patterns and best practices"
|
||||
input_schema:
|
||||
tool_names:
|
||||
type: string
|
||||
description: "Comma-separated tool names (e.g., 'tool_name1,tool_name2')"
|
||||
required: true
|
||||
|
||||
add_tool_call_result:
|
||||
flow_content: parse_tool_call_result_op >> update_vector_store_op
|
||||
description: "Evaluates and adds tool call results to the tool memory database, creating new memory or updating existing memory for the specified tool"
|
||||
input_schema:
|
||||
tool_call_results:
|
||||
type: array
|
||||
description: "List of tool call result objects, each containing: tool_name, input, output, success, time_cost, token_cost, create_time"
|
||||
required: true
|
||||
|
||||
summary_tool_memory:
|
||||
flow_content: summary_tool_memory_op >> update_vector_store_op
|
||||
description: "Analyzes tool call history and generates comprehensive usage patterns, best practices, and recommendations for the specified tools"
|
||||
input_schema:
|
||||
tool_names:
|
||||
type: string
|
||||
description: "Comma-separated tool names to summarize (e.g., 'tool_name1,tool_name2')"
|
||||
required: true
|
||||
|
||||
op:
|
||||
parse_tool_call_result_op:
|
||||
backend: parse_tool_call_result_op
|
||||
llm: default
|
||||
params:
|
||||
max_history_tool_call_cnt: 100
|
||||
evaluation_sleep_interval: 1.0
|
||||
|
||||
summary_tool_memory_op:
|
||||
backend: summary_tool_memory_op
|
||||
llm: default
|
||||
params:
|
||||
data_from: '2025-09-10 10:56:58'
|
||||
summary_sleep_interval: 1.0
|
||||
|
|
@ -258,19 +258,29 @@ jobs:
|
|||
|
||||
traverse:
|
||||
backend: base
|
||||
description: "Walk the wikilink graph from a path."
|
||||
description: >-
|
||||
Return a bounded wikilink graph with frontend-ready nodes and directed edges.
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
path:
|
||||
type: string
|
||||
description: "path"
|
||||
oneOf:
|
||||
- type: string
|
||||
- type: array
|
||||
items:
|
||||
type: string
|
||||
minItems: 1
|
||||
description: "workspace-relative path or paths used as graph seeds"
|
||||
depth:
|
||||
type: integer
|
||||
description: "hop limit"
|
||||
minimum: 0
|
||||
description: "maximum number of wikilink hops"
|
||||
default: 1
|
||||
direction:
|
||||
type: string
|
||||
description: >-
|
||||
traversal direction; returned edges always preserve the original
|
||||
wikilink source-to-target direction
|
||||
enum:
|
||||
- forward
|
||||
- backward
|
||||
|
|
@ -281,6 +291,15 @@ jobs:
|
|||
steps:
|
||||
- backend: traverse_step
|
||||
|
||||
graph_snapshot:
|
||||
backend: base
|
||||
description: "Return the category-rooted digest wikilink graph with daily-note leaves."
|
||||
parameters:
|
||||
type: object
|
||||
properties: { }
|
||||
steps:
|
||||
- backend: graph_snapshot_step
|
||||
|
||||
reindex:
|
||||
backend: base
|
||||
description: "wipe the file store and rebuild it from the existing files"
|
||||
|
|
|
|||
|
|
@ -35,12 +35,14 @@ from .dream import (
|
|||
from .emb_node import EmbNode
|
||||
from .file_chunk import FileChunk
|
||||
from .file_front_matter import FileFrontMatter
|
||||
from .graph_snapshot import GraphSnapshot, GraphSnapshotEdge, GraphSnapshotNode
|
||||
from .file_link import FileLink
|
||||
from .file_node import FileNode
|
||||
from .request import Request
|
||||
from .response import Response
|
||||
from .stream_chunk import StreamChunk
|
||||
from .token_usage import TokenUsage
|
||||
from .traverse_graph import TraverseGraph, TraverseGraphEdge, TraverseGraphNode
|
||||
|
||||
__all__ = [
|
||||
"ApplicationConfig",
|
||||
|
|
@ -74,6 +76,9 @@ __all__ = [
|
|||
"FileFrontMatter",
|
||||
"FileLink",
|
||||
"FileNode",
|
||||
"GraphSnapshot",
|
||||
"GraphSnapshotEdge",
|
||||
"GraphSnapshotNode",
|
||||
"IntegrateOutcome",
|
||||
"JobConfig",
|
||||
"PaperInfo",
|
||||
|
|
@ -86,4 +91,7 @@ __all__ = [
|
|||
"StreamChunk",
|
||||
"TokenUsage",
|
||||
"TopicSelectionOutput",
|
||||
"TraverseGraph",
|
||||
"TraverseGraphEdge",
|
||||
"TraverseGraphNode",
|
||||
]
|
||||
|
|
|
|||
32
reme/schema/graph_snapshot.py
Normal file
32
reme/schema/graph_snapshot.py
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
"""Public response schemas for a complete indexed wikilink graph snapshot."""
|
||||
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class GraphSnapshotNode(BaseModel):
|
||||
"""One virtual category or indexed Markdown file in a graph snapshot."""
|
||||
|
||||
id: str = Field(description="Stable node identifier; documents use their workspace-relative path")
|
||||
path: str = Field(description="Workspace-relative file or virtual category path")
|
||||
name: str = Field(default="", description="Document name from frontmatter")
|
||||
description: str = Field(default="", description="Document description from frontmatter")
|
||||
indexed: bool = Field(description="Whether the target has an indexed FileNode")
|
||||
virtual: bool = Field(default=False, description="Whether this is a generated category node")
|
||||
|
||||
|
||||
class GraphSnapshotEdge(BaseModel):
|
||||
"""One directed category or wikilink edge in a graph snapshot."""
|
||||
|
||||
source: str = Field(description="Source node identifier")
|
||||
target: str = Field(description="Target node identifier")
|
||||
target_anchor: str | None = Field(default=None, description="Optional heading, block, or line anchor")
|
||||
|
||||
|
||||
class GraphSnapshot(BaseModel):
|
||||
"""A category-rooted snapshot of digest wikilinks and their daily-note leaves."""
|
||||
|
||||
version: Literal[1] = 1
|
||||
nodes: list[GraphSnapshotNode]
|
||||
edges: list[GraphSnapshotEdge]
|
||||
36
reme/schema/traverse_graph.py
Normal file
36
reme/schema/traverse_graph.py
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
"""Public response schemas for wikilink graph traversal."""
|
||||
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class TraverseGraphNode(BaseModel):
|
||||
"""One indexed file or unresolved wikilink target in a traversal result."""
|
||||
|
||||
id: str = Field(description="Stable node identifier; equal to the workspace-relative path")
|
||||
path: str = Field(description="Workspace-relative file path")
|
||||
name: str = Field(default="", description="Document name from frontmatter")
|
||||
description: str = Field(default="", description="Document description from frontmatter")
|
||||
depth: int = Field(ge=0, description="Shortest hop distance from any seed")
|
||||
indexed: bool = Field(description="Whether the target has an indexed FileNode")
|
||||
|
||||
|
||||
class TraverseGraphEdge(BaseModel):
|
||||
"""One directed wikilink, preserving its original source and target."""
|
||||
|
||||
source: str = Field(description="Workspace-relative source file path")
|
||||
target: str = Field(description="Workspace-relative target file path")
|
||||
target_anchor: str | None = Field(default=None, description="Optional heading, block, or line anchor")
|
||||
depth: int = Field(ge=1, description="Traversal depth at which this edge was first reached")
|
||||
|
||||
|
||||
class TraverseGraph(BaseModel):
|
||||
"""A bounded wikilink graph rooted at one or more workspace paths."""
|
||||
|
||||
version: Literal[1] = 1
|
||||
seeds: list[str] = Field(description="Normalized workspace-relative traversal roots")
|
||||
depth: int = Field(ge=0, description="Requested hop limit")
|
||||
direction: Literal["forward", "backward", "both"]
|
||||
nodes: list[TraverseGraphNode]
|
||||
edges: list[TraverseGraphEdge]
|
||||
|
|
@ -68,7 +68,7 @@ class ListStep(BaseStep):
|
|||
out: list[str] = []
|
||||
for entry in files:
|
||||
try:
|
||||
out.append(str(entry.relative_to(workspace_dir)))
|
||||
out.append(entry.relative_to(workspace_dir).as_posix())
|
||||
except ValueError:
|
||||
out.append(str(entry))
|
||||
return out
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ from .bm25_search import Bm25SearchStep
|
|||
from .clear_paths import ClearPathsStep
|
||||
from .clear_store import ClearStoreStep
|
||||
from .draft import AddDraftStep, ReadAllDraftStep
|
||||
from .graph_snapshot import GraphSnapshotStep
|
||||
from .log_changes import LogChangesStep
|
||||
from .node_search import NodeSearchStep
|
||||
from .init_changes import InitChangesStep
|
||||
|
|
@ -30,6 +31,7 @@ __all__ = [
|
|||
"DEFAULT_LOW_POWER_POLL_MS",
|
||||
"DEFAULT_WATCH_DEBOUNCE_MS",
|
||||
"DEFAULT_WATCH_STEP_MS",
|
||||
"GraphSnapshotStep",
|
||||
"InitChangesStep",
|
||||
"LogChangesStep",
|
||||
"NodeSearchStep",
|
||||
|
|
|
|||
86
reme/steps/index/graph_snapshot.py
Normal file
86
reme/steps/index/graph_snapshot.py
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
"""Return the digest graph, rooted by its three memory categories."""
|
||||
|
||||
from ..base_step import BaseStep
|
||||
from ...components import R
|
||||
from ...enumeration import DreamBucketEnum
|
||||
from ...schema import GraphSnapshot, GraphSnapshotEdge, GraphSnapshotNode
|
||||
|
||||
_CATEGORY_BUCKETS = (
|
||||
DreamBucketEnum.WIKI,
|
||||
DreamBucketEnum.PERSONAL,
|
||||
DreamBucketEnum.PROCEDURE,
|
||||
)
|
||||
|
||||
|
||||
@R.register("graph_snapshot_step")
|
||||
class GraphSnapshotStep(BaseStep):
|
||||
"""Build the frontend digest graph through the file-store contract.
|
||||
|
||||
Category nodes connect to every indexed Markdown file in their bucket.
|
||||
Digest files retain wikilinks to other digest files and to daily notes. Daily
|
||||
notes are leaves: their own outgoing links are intentionally not returned.
|
||||
"""
|
||||
|
||||
async def execute(self):
|
||||
assert self.context is not None
|
||||
indexed_nodes = await self.file_store.get_nodes()
|
||||
node_by_path = {node.path: node for node in indexed_nodes if node.path.lower().endswith(".md")}
|
||||
|
||||
digest_dir = str(self.config_value("digest_dir")).strip("/")
|
||||
daily_dir = str(self.config_value("daily_dir")).strip("/")
|
||||
category_paths = {
|
||||
bucket: f"{digest_dir}/{bucket.value}" if digest_dir else bucket.value for bucket in _CATEGORY_BUCKETS
|
||||
}
|
||||
daily_prefix = f"{daily_dir}/" if daily_dir else ""
|
||||
|
||||
digest_paths_by_category = {
|
||||
label: sorted(path for path in node_by_path if path.startswith(f"{category_path}/"))
|
||||
for label, category_path in category_paths.items()
|
||||
}
|
||||
digest_paths = {path for paths in digest_paths_by_category.values() for path in paths}
|
||||
|
||||
edge_keys: set[tuple[str, str, str | None]] = set()
|
||||
for source in digest_paths:
|
||||
for link in node_by_path[source].links:
|
||||
target = link.target_path
|
||||
if target in digest_paths or (target in node_by_path and target.startswith(daily_prefix)):
|
||||
edge_keys.add((source, target, link.target_anchor))
|
||||
|
||||
daily_paths = {target for _source, target, _anchor in edge_keys if target.startswith(daily_prefix)}
|
||||
|
||||
nodes = [
|
||||
GraphSnapshotNode(
|
||||
id=f"virtual:{bucket.value}",
|
||||
path=category_paths[bucket],
|
||||
name=bucket.value,
|
||||
indexed=False,
|
||||
virtual=True,
|
||||
)
|
||||
for bucket in _CATEGORY_BUCKETS
|
||||
]
|
||||
for bucket in _CATEGORY_BUCKETS:
|
||||
for path in digest_paths_by_category[bucket]:
|
||||
edge_keys.add((f"virtual:{bucket.value}", path, None))
|
||||
|
||||
for path in sorted(digest_paths | daily_paths):
|
||||
node = node_by_path[path]
|
||||
nodes.append(
|
||||
GraphSnapshotNode(
|
||||
id=path,
|
||||
path=path,
|
||||
name=node.front_matter.name,
|
||||
description=node.front_matter.description,
|
||||
indexed=True,
|
||||
),
|
||||
)
|
||||
|
||||
edges = [
|
||||
GraphSnapshotEdge(source=source, target=target, target_anchor=anchor)
|
||||
for source, target, anchor in sorted(edge_keys, key=lambda edge: (edge[0], edge[1], edge[2] or ""))
|
||||
]
|
||||
graph = GraphSnapshot(nodes=nodes, edges=edges)
|
||||
|
||||
self.context.response.success = True
|
||||
self.context.response.answer = graph.model_dump()
|
||||
self.logger.info(f"[{self.name}] nodes={len(nodes)} edges={len(edges)}")
|
||||
return self.context.response
|
||||
|
|
@ -1,127 +1,181 @@
|
|||
"""BFS over wikilink edges from one or more seed files.
|
||||
|
||||
One record per traversed *edge* (not per node): the same target can repeat
|
||||
if reached via different anchors or paths. Each record carries the
|
||||
predecessor and anchor so callers can reconstruct the path. Adjacency is
|
||||
built once via a single ``file_store.get_nodes()`` call — BFS then runs
|
||||
purely in memory with no per-frontier round-trips.
|
||||
"""
|
||||
"""Build a bounded, frontend-ready graph from indexed Markdown wikilinks."""
|
||||
|
||||
from collections import deque
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from ..base_step import BaseStep
|
||||
from ...components import R
|
||||
from ...schema import FileLink
|
||||
from ...schema import FileNode, TraverseGraph, TraverseGraphEdge, TraverseGraphNode
|
||||
|
||||
_OUT = {"out", "forward", "both"}
|
||||
_IN = {"in", "backward", "both"}
|
||||
_VALID = _OUT | _IN
|
||||
|
||||
# source path -> list of (neighbor path, link)
|
||||
Adjacency = dict[str, list[tuple[str, FileLink]]]
|
||||
_DIRECTION_ALIASES = {
|
||||
"out": "forward",
|
||||
"forward": "forward",
|
||||
"in": "backward",
|
||||
"backward": "backward",
|
||||
"both": "both",
|
||||
}
|
||||
|
||||
|
||||
async def _build_adjacency(file_store) -> tuple[Adjacency, Adjacency]:
|
||||
"""Single ``get_nodes()`` pass → (outbound, inbound) adjacency maps.
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _TraversalLink:
|
||||
"""An adjacency entry carrying both traversal and original edge direction."""
|
||||
|
||||
Inbound stores the source path next to each link so BFS can attribute
|
||||
inbound edges back to their origin — ``get_inlinks`` alone returns
|
||||
target-shaped FileLinks without source attribution.
|
||||
"""
|
||||
outbound: Adjacency = {}
|
||||
inbound: Adjacency = {}
|
||||
for node in await file_store.get_nodes():
|
||||
neighbor: str
|
||||
source: str
|
||||
target: str
|
||||
target_anchor: str | None
|
||||
|
||||
|
||||
Adjacency = dict[str, list[_TraversalLink]]
|
||||
EdgeKey = tuple[str, str, str | None]
|
||||
|
||||
|
||||
def _build_adjacency(nodes: list[FileNode]) -> tuple[Adjacency, Adjacency]:
|
||||
"""Build forward/backward adjacency while preserving actual wikilink direction."""
|
||||
forward: Adjacency = {}
|
||||
backward: Adjacency = {}
|
||||
for node in nodes:
|
||||
for link in node.links:
|
||||
if link.target_path:
|
||||
outbound.setdefault(node.path, []).append((link.target_path, link))
|
||||
inbound.setdefault(link.target_path, []).append((node.path, link))
|
||||
return outbound, inbound
|
||||
if not link.target_path:
|
||||
continue
|
||||
item = _TraversalLink(
|
||||
neighbor=link.target_path,
|
||||
source=node.path,
|
||||
target=link.target_path,
|
||||
target_anchor=link.target_anchor,
|
||||
)
|
||||
forward.setdefault(node.path, []).append(item)
|
||||
backward.setdefault(link.target_path, []).append(
|
||||
_TraversalLink(
|
||||
neighbor=node.path,
|
||||
source=item.source,
|
||||
target=item.target,
|
||||
target_anchor=item.target_anchor,
|
||||
),
|
||||
)
|
||||
return forward, backward
|
||||
|
||||
|
||||
def _bfs(
|
||||
def _traverse(
|
||||
seeds: list[str],
|
||||
max_depth: int,
|
||||
direction: str,
|
||||
outbound: Adjacency,
|
||||
inbound: Adjacency,
|
||||
) -> list[dict]:
|
||||
"""In-memory BFS; emits one record per unique (src, dst, anchor) edge."""
|
||||
sources: list[Adjacency] = []
|
||||
if direction in _OUT:
|
||||
sources.append(outbound)
|
||||
if direction in _IN:
|
||||
sources.append(inbound)
|
||||
forward: Adjacency,
|
||||
backward: Adjacency,
|
||||
) -> tuple[dict[str, int], dict[EdgeKey, int]]:
|
||||
"""Return shortest node depths and directed edges reached by bounded BFS."""
|
||||
adjacency_maps = []
|
||||
if direction in {"forward", "both"}:
|
||||
adjacency_maps.append(forward)
|
||||
if direction in {"backward", "both"}:
|
||||
adjacency_maps.append(backward)
|
||||
|
||||
visited: set[tuple[str, str, str | None]] = set()
|
||||
results: list[dict] = []
|
||||
queue: deque[tuple[str, int]] = deque((s, 0) for s in seeds)
|
||||
node_depths = {seed: 0 for seed in seeds}
|
||||
edge_depths: dict[EdgeKey, int] = {}
|
||||
queue = deque(seeds)
|
||||
|
||||
while queue:
|
||||
current, depth = queue.popleft()
|
||||
if depth >= max_depth:
|
||||
current = queue.popleft()
|
||||
current_depth = node_depths[current]
|
||||
if current_depth >= max_depth:
|
||||
continue
|
||||
for src in sources:
|
||||
for next_path, link in src.get(current, ()):
|
||||
key = (current, next_path, link.target_anchor)
|
||||
if key in visited:
|
||||
continue
|
||||
visited.add(key)
|
||||
results.append(
|
||||
{
|
||||
"path": next_path,
|
||||
"depth": depth + 1,
|
||||
"via": current,
|
||||
"anchor": link.target_anchor,
|
||||
},
|
||||
)
|
||||
if depth + 1 < max_depth:
|
||||
queue.append((next_path, depth + 1))
|
||||
return results
|
||||
|
||||
next_depth = current_depth + 1
|
||||
for adjacency in adjacency_maps:
|
||||
for link in adjacency.get(current, ()):
|
||||
edge_key = (link.source, link.target, link.target_anchor)
|
||||
previous_edge_depth = edge_depths.get(edge_key)
|
||||
if previous_edge_depth is None or next_depth < previous_edge_depth:
|
||||
edge_depths[edge_key] = next_depth
|
||||
|
||||
previous_node_depth = node_depths.get(link.neighbor)
|
||||
if previous_node_depth is None or next_depth < previous_node_depth:
|
||||
node_depths[link.neighbor] = next_depth
|
||||
queue.append(link.neighbor)
|
||||
|
||||
return node_depths, edge_depths
|
||||
|
||||
|
||||
def _build_graph(
|
||||
*,
|
||||
seeds: list[str],
|
||||
max_depth: int,
|
||||
direction: str,
|
||||
indexed_nodes: list[FileNode],
|
||||
node_depths: dict[str, int],
|
||||
edge_depths: dict[EdgeKey, int],
|
||||
) -> TraverseGraph:
|
||||
"""Materialize deterministic public graph schemas from traversal state."""
|
||||
node_by_path = {node.path: node for node in indexed_nodes}
|
||||
graph_nodes = []
|
||||
for path, node_depth in sorted(node_depths.items(), key=lambda item: (item[1], item[0])):
|
||||
node = node_by_path.get(path)
|
||||
graph_nodes.append(
|
||||
TraverseGraphNode(
|
||||
id=path,
|
||||
path=path,
|
||||
name=node.front_matter.name if node is not None else "",
|
||||
description=node.front_matter.description if node is not None else "",
|
||||
depth=node_depth,
|
||||
indexed=node is not None,
|
||||
),
|
||||
)
|
||||
|
||||
graph_edges = [
|
||||
TraverseGraphEdge(source=source, target=target, target_anchor=anchor, depth=edge_depth)
|
||||
for (source, target, anchor), edge_depth in sorted(
|
||||
edge_depths.items(),
|
||||
key=lambda item: (item[1], item[0][0], item[0][1], item[0][2] or ""),
|
||||
)
|
||||
]
|
||||
return TraverseGraph(
|
||||
seeds=seeds,
|
||||
depth=max_depth,
|
||||
direction=direction,
|
||||
nodes=graph_nodes,
|
||||
edges=graph_edges,
|
||||
)
|
||||
|
||||
|
||||
@R.register("traverse_step")
|
||||
class TraverseStep(BaseStep):
|
||||
"""BFS from one or more seed files to explore wikilink relationships.
|
||||
|
||||
Parameters:
|
||||
path — single seed (str) or list of seeds (workspace-relative).
|
||||
direction — ``forward`` / ``backward`` / ``both`` (or ``out`` / ``in`` / ``both``).
|
||||
depth — hop limit (default 1 = immediate neighbors).
|
||||
"""
|
||||
"""Return a bounded wikilink graph rooted at one or more workspace paths."""
|
||||
|
||||
async def execute(self):
|
||||
assert self.context is not None
|
||||
raw = self.context.get("path")
|
||||
items = [raw] if isinstance(raw, (str, Path)) else list(raw or [])
|
||||
seeds = [str(p) for p in items if p]
|
||||
assert seeds, "path is required"
|
||||
depth = int(self.context.get("depth") or 1)
|
||||
direction = (self.context.get("direction") or "both").lower()
|
||||
assert direction in _VALID, f"direction must be one of {sorted(_VALID)}, got {direction!r}"
|
||||
raw_paths = self.context.get("path")
|
||||
items = [raw_paths] if isinstance(raw_paths, (str, Path)) else list(raw_paths or [])
|
||||
seeds = list(dict.fromkeys(str(path).replace("\\", "/") for path in items if path))
|
||||
if not seeds:
|
||||
raise ValueError("path is required")
|
||||
|
||||
outbound, inbound = await _build_adjacency(self.file_store)
|
||||
results = _bfs(seeds, depth, direction, outbound, inbound)
|
||||
raw_depth = self.context.get("depth")
|
||||
max_depth = 1 if raw_depth is None else int(raw_depth)
|
||||
if max_depth < 0:
|
||||
raise ValueError("depth must be greater than or equal to 0")
|
||||
|
||||
self.logger.info(
|
||||
f"[{self.name}] seeds={seeds!r} depth={depth} direction={direction} "
|
||||
f"nodes={len(outbound) + len(inbound)} edges={len(results)}",
|
||||
raw_direction = str(self.context.get("direction") or "both").lower()
|
||||
direction = _DIRECTION_ALIASES.get(raw_direction)
|
||||
if direction is None:
|
||||
raise ValueError(f"direction must be one of {sorted(_DIRECTION_ALIASES)}, got {raw_direction!r}")
|
||||
|
||||
indexed_nodes = await self.file_store.get_nodes()
|
||||
forward, backward = _build_adjacency(indexed_nodes)
|
||||
node_depths, edge_depths = _traverse(seeds, max_depth, direction, forward, backward)
|
||||
graph = _build_graph(
|
||||
seeds=seeds,
|
||||
max_depth=max_depth,
|
||||
direction=direction,
|
||||
indexed_nodes=indexed_nodes,
|
||||
node_depths=node_depths,
|
||||
edge_depths=edge_depths,
|
||||
)
|
||||
|
||||
label = seeds[0] if len(seeds) == 1 else f"{len(seeds)} seeds"
|
||||
if not results:
|
||||
answer = f"No edges found from {label}"
|
||||
else:
|
||||
header = f"Traversed {len(results)} edge(s) from {label}"
|
||||
lines = [header, ""]
|
||||
for r in results:
|
||||
target = r["path"]
|
||||
if r["anchor"]:
|
||||
target = f"{target}#{r['anchor']}"
|
||||
lines.append(f"[depth={r['depth']}] {r['via']} --> {target}")
|
||||
answer = "\n".join(lines)
|
||||
|
||||
self.context.response.success = True
|
||||
self.context.response.answer = answer
|
||||
self.context.response.metadata.update({"edges": results, "count": len(results)})
|
||||
self.context.response.answer = graph.model_dump()
|
||||
self.logger.info(
|
||||
f"[{self.name}] seeds={seeds!r} depth={max_depth} direction={direction} "
|
||||
f"nodes={len(graph.nodes)} edges={len(graph.edges)}",
|
||||
)
|
||||
return self.context.response
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ import warnings
|
|||
from reme.components.agent_wrapper import BaseAgentWrapper
|
||||
from reme.components.application_context import ApplicationContext
|
||||
from reme.components.file_store import LocalFileStore
|
||||
from reme.schema import FileLink, FileNode
|
||||
from reme.schema import FileFrontMatter, FileLink, FileNode, TraverseGraph
|
||||
from reme.steps.common.add import AddStep
|
||||
from reme.steps.common.health_check import _file_graph_status
|
||||
from reme.steps.common.llm_demo import LLMDemoStep
|
||||
|
|
@ -42,7 +42,13 @@ def _run(coro):
|
|||
asyncio.run(coro)
|
||||
|
||||
|
||||
def _node(path: str, links: list[tuple[str, str | None]] | None = None) -> FileNode:
|
||||
def _node(
|
||||
path: str,
|
||||
links: list[tuple[str, str | None]] | None = None,
|
||||
*,
|
||||
name: str = "",
|
||||
description: str = "",
|
||||
) -> FileNode:
|
||||
"""Build a FileNode with (target_path, target_anchor) outgoing edges."""
|
||||
return FileNode(
|
||||
path=path,
|
||||
|
|
@ -50,6 +56,7 @@ def _node(path: str, links: list[tuple[str, str | None]] | None = None) -> FileN
|
|||
links=[
|
||||
FileLink(source_path=path, target_path=target, target_anchor=anchor) for target, anchor in (links or [])
|
||||
],
|
||||
front_matter=FileFrontMatter(name=name, description=description),
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -63,7 +70,7 @@ async def _make_store(nodes: list[FileNode]) -> LocalFileStore:
|
|||
|
||||
|
||||
def _edges(step) -> list[dict]:
|
||||
return step.context.response.metadata.get("edges", [])
|
||||
return step.context.response.answer["edges"]
|
||||
|
||||
|
||||
def test_add_step_coerces_numeric_inputs():
|
||||
|
|
@ -207,12 +214,11 @@ def test_traverse_forward_depth_1():
|
|||
step = traverse_mod.TraverseStep(file_store=store)
|
||||
await step(path="a.md", direction="forward", depth=1)
|
||||
results = _edges(step)
|
||||
paths = {r["path"] for r in results}
|
||||
paths = {r["target"] for r in results}
|
||||
assert paths == {"b.md", "c.md"}
|
||||
# Anchors remain part of traversal metadata.
|
||||
c_edge = next(r for r in results if r["path"] == "c.md")
|
||||
assert c_edge["anchor"] == "intro"
|
||||
assert c_edge["via"] == "a.md"
|
||||
c_edge = next(r for r in results if r["target"] == "c.md")
|
||||
assert c_edge["target_anchor"] == "intro"
|
||||
assert c_edge["source"] == "a.md"
|
||||
assert c_edge["depth"] == 1
|
||||
await store.close()
|
||||
print("✓ test_traverse_forward_depth_1 passed")
|
||||
|
|
@ -220,6 +226,25 @@ def test_traverse_forward_depth_1():
|
|||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_traverse_normalizes_windows_seed_path():
|
||||
"""Windows-style seeds match the graph's portable POSIX path keys."""
|
||||
|
||||
async def run():
|
||||
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
|
||||
store = await _make_store(
|
||||
[
|
||||
_node("topics/a.md", [("topics/b.md", None)]),
|
||||
_node("topics/b.md"),
|
||||
],
|
||||
)
|
||||
step = traverse_mod.TraverseStep(file_store=store)
|
||||
await step(path=r"topics\a.md", direction="forward", depth=1)
|
||||
assert [edge["target"] for edge in _edges(step)] == ["topics/b.md"]
|
||||
await store.close()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_traverse_backward_returns_inlinks():
|
||||
"""direction=backward walks inbound edges."""
|
||||
|
||||
|
|
@ -235,7 +260,10 @@ def test_traverse_backward_returns_inlinks():
|
|||
step = traverse_mod.TraverseStep(file_store=store)
|
||||
await step(path="b.md", direction="backward", depth=1)
|
||||
results = _edges(step)
|
||||
assert {r["path"] for r in results} == {"a.md", "c.md"}
|
||||
assert {(r["source"], r["target"]) for r in results} == {
|
||||
("a.md", "b.md"),
|
||||
("c.md", "b.md"),
|
||||
}
|
||||
await store.close()
|
||||
print("✓ test_traverse_backward_returns_inlinks passed")
|
||||
|
||||
|
|
@ -257,7 +285,7 @@ def test_traverse_depth_2_expands():
|
|||
step = traverse_mod.TraverseStep(file_store=store)
|
||||
await step(path="a.md", direction="forward", depth=2)
|
||||
results = _edges(step)
|
||||
depth_map = {r["path"]: r["depth"] for r in results}
|
||||
depth_map = {r["target"]: r["depth"] for r in results}
|
||||
assert depth_map.get("b.md") == 1
|
||||
assert depth_map.get("c.md") == 2
|
||||
await store.close()
|
||||
|
|
@ -320,13 +348,96 @@ def test_traverse_both_directions():
|
|||
step = traverse_mod.TraverseStep(file_store=store)
|
||||
await step(path="center.md", direction="both", depth=1)
|
||||
results = _edges(step)
|
||||
assert {r["path"] for r in results} == {"upstream.md", "downstream.md"}
|
||||
assert {(r["source"], r["target"]) for r in results} == {
|
||||
("upstream.md", "center.md"),
|
||||
("center.md", "downstream.md"),
|
||||
}
|
||||
await store.close()
|
||||
print("✓ test_traverse_both_directions passed")
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_traverse_both_preserves_reciprocal_edge_directions():
|
||||
"""Opposite wikilinks remain two distinct directed graph edges."""
|
||||
|
||||
async def run():
|
||||
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
|
||||
store = await _make_store(
|
||||
[
|
||||
_node("a.md", [("b.md", None)]),
|
||||
_node("b.md", [("a.md", None)]),
|
||||
],
|
||||
)
|
||||
step = traverse_mod.TraverseStep(file_store=store)
|
||||
response = await step(path="a.md", direction="both", depth=1)
|
||||
graph = TraverseGraph.model_validate(response.answer)
|
||||
|
||||
assert {(edge.source, edge.target) for edge in graph.edges} == {
|
||||
("a.md", "b.md"),
|
||||
("b.md", "a.md"),
|
||||
}
|
||||
assert response.metadata == {}
|
||||
await store.close()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_traverse_returns_frontmatter_and_unresolved_nodes():
|
||||
"""Graph nodes expose labels and distinguish indexed files from dangling targets."""
|
||||
|
||||
async def run():
|
||||
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
|
||||
store = await _make_store(
|
||||
[
|
||||
_node(
|
||||
"a.md",
|
||||
[("missing.md", "details")],
|
||||
name="Alpha",
|
||||
description="Root note",
|
||||
),
|
||||
],
|
||||
)
|
||||
step = traverse_mod.TraverseStep(file_store=store)
|
||||
response = await step(path="a.md", direction="forward", depth=1)
|
||||
graph = TraverseGraph.model_validate(response.answer)
|
||||
nodes = {node.path: node for node in graph.nodes}
|
||||
|
||||
assert graph.version == 1
|
||||
assert graph.seeds == ["a.md"]
|
||||
assert nodes["a.md"].model_dump() == {
|
||||
"id": "a.md",
|
||||
"path": "a.md",
|
||||
"name": "Alpha",
|
||||
"description": "Root note",
|
||||
"depth": 0,
|
||||
"indexed": True,
|
||||
}
|
||||
assert nodes["missing.md"].indexed is False
|
||||
assert nodes["missing.md"].depth == 1
|
||||
assert graph.edges[0].target_anchor == "details"
|
||||
await store.close()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_traverse_depth_zero_returns_only_seed_nodes():
|
||||
"""A zero-hop traversal is valid and emits no edges."""
|
||||
|
||||
async def run():
|
||||
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
|
||||
store = await _make_store([_node("a.md", [("b.md", None)]), _node("b.md")])
|
||||
step = traverse_mod.TraverseStep(file_store=store)
|
||||
response = await step(path="a.md", direction="forward", depth=0)
|
||||
graph = TraverseGraph.model_validate(response.answer)
|
||||
|
||||
assert [node.path for node in graph.nodes] == ["a.md"]
|
||||
assert graph.edges == []
|
||||
await store.close()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print("\n=== traverse step tests ===")
|
||||
test_traverse_forward_depth_1()
|
||||
|
|
@ -335,4 +446,7 @@ if __name__ == "__main__":
|
|||
test_traverse_short_seed_yields_empty()
|
||||
test_traverse_not_found_seed()
|
||||
test_traverse_both_directions()
|
||||
test_traverse_both_preserves_reciprocal_edge_directions()
|
||||
test_traverse_returns_frontmatter_and_unresolved_nodes()
|
||||
test_traverse_depth_zero_returns_only_seed_nodes()
|
||||
print("\n所有测试通过!")
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ import asyncio
|
|||
import os
|
||||
import tempfile
|
||||
import warnings
|
||||
from pathlib import Path
|
||||
from pathlib import Path, PureWindowsPath
|
||||
|
||||
from reme.components.file_store import LocalFileStore
|
||||
from reme.schema import FileNode
|
||||
|
|
@ -177,6 +177,14 @@ def test_list_lists_files():
|
|||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_list_formats_workspace_relative_windows_paths_as_posix():
|
||||
"""Workspace-relative list results use the graph's portable path format."""
|
||||
workspace = PureWindowsPath("C:/workspace")
|
||||
files = [workspace / "topics" / "a.md"]
|
||||
|
||||
assert crud_list.ListStep._format_relative(files, workspace) == ["topics/a.md"]
|
||||
|
||||
|
||||
def test_list_empty_directory_has_explicit_answer():
|
||||
"""list tells an LLM explicitly when the target directory has no files."""
|
||||
|
||||
|
|
|
|||
94
tests/unit/test_graph_snapshot_step.py
Normal file
94
tests/unit/test_graph_snapshot_step.py
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
"""Tests for the frontend-ready category-rooted digest graph snapshot."""
|
||||
|
||||
import asyncio
|
||||
|
||||
from reme.components.file_store import LocalFileStore
|
||||
from reme.schema import FileFrontMatter, FileLink, FileNode, GraphSnapshot
|
||||
from reme.steps.index.graph_snapshot import GraphSnapshotStep
|
||||
|
||||
|
||||
def _node(
|
||||
path: str,
|
||||
links: list[tuple[str, str | None]] | None = None,
|
||||
*,
|
||||
name: str = "",
|
||||
description: str = "",
|
||||
) -> FileNode:
|
||||
return FileNode(
|
||||
path=path,
|
||||
st_mtime=1.0,
|
||||
links=[
|
||||
FileLink(source_path=path, target_path=target, target_anchor=anchor) for target, anchor in (links or [])
|
||||
],
|
||||
front_matter=FileFrontMatter(name=name, description=description),
|
||||
)
|
||||
|
||||
|
||||
def test_graph_snapshot_returns_category_roots_digest_links_and_daily_leaves(tmp_path, monkeypatch):
|
||||
"""The snapshot is rooted at digest categories and stops at indexed daily notes."""
|
||||
monkeypatch.chdir(tmp_path)
|
||||
|
||||
async def run():
|
||||
store = LocalFileStore(name="snapshot", embedding_store="")
|
||||
await store.start()
|
||||
try:
|
||||
await store.file_graph.upsert_nodes(
|
||||
[
|
||||
_node(
|
||||
"digest/wiki/alpha.md",
|
||||
[
|
||||
("digest/personal/beta.md", "intro"),
|
||||
("daily/2026-08-05/event.md", None),
|
||||
("daily/2026-08-05/missing.md", None),
|
||||
("resource/ignored.md", None),
|
||||
],
|
||||
name="Alpha",
|
||||
description="Root note",
|
||||
),
|
||||
_node("digest/personal/beta.md", [("digest/wiki/alpha.md", None)], name="Beta"),
|
||||
_node("digest/procedure/how-to.md", name="How to"),
|
||||
_node(
|
||||
"daily/2026-08-05/event.md",
|
||||
[("digest/procedure/how-to.md", None), ("daily/2026-08-05/hidden.md", None)],
|
||||
name="Event",
|
||||
),
|
||||
_node("daily/2026-08-05/isolated.md", name="Isolated daily"),
|
||||
_node("resource/ignored.md", name="Ignored resource"),
|
||||
],
|
||||
)
|
||||
|
||||
response = await GraphSnapshotStep(file_store=store)()
|
||||
graph = GraphSnapshot.model_validate(response.answer)
|
||||
|
||||
assert response.success is True
|
||||
assert response.metadata == {}
|
||||
assert [node.id for node in graph.nodes[:3]] == [
|
||||
"virtual:wiki",
|
||||
"virtual:personal",
|
||||
"virtual:procedure",
|
||||
]
|
||||
assert [node.path for node in graph.nodes[3:]] == [
|
||||
"daily/2026-08-05/event.md",
|
||||
"digest/personal/beta.md",
|
||||
"digest/procedure/how-to.md",
|
||||
"digest/wiki/alpha.md",
|
||||
]
|
||||
nodes = {node.id: node for node in graph.nodes}
|
||||
assert nodes["virtual:wiki"].path == "digest/wiki"
|
||||
assert nodes["virtual:procedure"].path == "digest/procedure"
|
||||
assert nodes["virtual:wiki"].virtual is True
|
||||
assert nodes["digest/wiki/alpha.md"].name == "Alpha"
|
||||
assert nodes["digest/wiki/alpha.md"].description == "Root note"
|
||||
assert nodes["digest/wiki/alpha.md"].virtual is False
|
||||
assert {(edge.source, edge.target, edge.target_anchor) for edge in graph.edges} == {
|
||||
("virtual:wiki", "digest/wiki/alpha.md", None),
|
||||
("virtual:personal", "digest/personal/beta.md", None),
|
||||
("virtual:procedure", "digest/procedure/how-to.md", None),
|
||||
("digest/wiki/alpha.md", "digest/personal/beta.md", "intro"),
|
||||
("digest/wiki/alpha.md", "daily/2026-08-05/event.md", None),
|
||||
("digest/personal/beta.md", "digest/wiki/alpha.md", None),
|
||||
}
|
||||
finally:
|
||||
await store.close()
|
||||
|
||||
asyncio.run(run())
|
||||
Loading…
Add table
Reference in a new issue