Remove path and line numbers from chunk ID calculation to improve
cache stability. The chunk identity now uses only the content hash
instead of including path::start::end coordinates.
This change removes the documentation about chunk identity format
and updates the hash_text call to use only the text content.
Add mistletoe as a project dependency for enhanced markdown parsing
capabilities. Refactor the LinkedFileParser to use a proper AST-based
approach with MdNode tree structure, replacing the previous flat token
processing method. The new implementation provides better handling of
markdown elements including tables, code fences, lists, and headings,
with improved chunking logic that maintains document structure in
generated content segments.
The changes include:
- Add mistletoe dependency to pyproject.toml
- Implement proper AST node representation with MdNode class
- Create recursive chunking algorithm with TOC preservation
- Add support for frontmatter extraction with FileFrontMatter schema
- Optimize leaf node splitting with proper boundary detection
- Include part numbering for split content pieces
- Replace legacy line-based chunking with AST tree approach that builds
a complete document skeleton with content inlined under relevant
sections
- Add new chunking parameters: chunk_chars (default 2000) and embed_toc
(default True) to control content size and TOC inclusion
- Implement recursive chunking algorithm that respects structural
boundaries (code lines, table rows, list items) and prevents splits
inside blocks
- Introduce part markers [Part X/N] for oversized leaf blocks that
require splitting
- Add CLI tool for inspecting parsed chunks and edges with options for
preview and configuration
- Refactor edge extraction to use FileEdge.from_text instead of
parse_wikilinks for consistency
BREAKING CHANGE: Chunk format changes significantly with full TOC
skeleton wrapping content, affecting embedding models expecting
breadcrumb prefixes.
docs(protocol): add typed edge link protocol documentation
Add comprehensive documentation for the link protocol supporting
typed edges in body text. This includes specification for three
legal inline forms (bare wikilink, line-level Dataview,
inline-bracketed Dataview), predicate syntax rules, and the
machine-managed Relations section convention for organizing
discovered edges.
fix(memory): update path reference from vault_root to working_dir
Change the memory_create operation's path anchoring from
vault_root to working_dir to maintain consistency with the
current working directory configuration.
refactor(components): remove edge_extractor module and simplify parsing
Remove the edge_extractor component module entirely and
inline edge extraction logic directly into LinkedFileParser
using parse_wikilinks utility. This simplifies the architecture
by eliminating the separate edge extraction component and
delegating edge discovery to the maintainer's enrichment operations.
feat(parser): update parse method signature and simplify edge extraction
Modify LinkedFileParser to return (FileNode, list[FileChunk])
tuple instead of ParsedFile, remove dependency on BaseEdgeExtractor,
and implement direct wikilink parsing from body text only.
```
