ReMe/reme2/memory
huangsen 51ec09d98f feat(parser): implement AST-based markdown chunking with full document TOC
- Replace legacy line-based chunking with AST tree approach that builds
  a complete document skeleton with content inlined under relevant
  sections
- Add new chunking parameters: chunk_chars (default 2000) and embed_toc
  (default True) to control content size and TOC inclusion
- Implement recursive chunking algorithm that respects structural
  boundaries (code lines, table rows, list items) and prevents splits
  inside blocks
- Introduce part markers [Part X/N] for oversized leaf blocks that
  require splitting
- Add CLI tool for inspecting parsed chunks and edges with options for
  preview and configuration
- Refactor edge extraction to use FileEdge.from_text instead of
  parse_wikilinks for consistency

BREAKING CHANGE: Chunk format changes significantly with full TOC
skeleton wrapping content, affecting embedding models expecting
breadcrumb prefixes.
2026-05-13 14:32:00 +08:00
..
schema ``` 2026-05-11 19:45:53 +08:00
__init__.py ``` 2026-05-11 10:52:25 +08:00
ingestor.py ``` 2026-05-11 19:45:53 +08:00
ingestor.yaml ``` 2026-05-11 19:45:53 +08:00
maintainer.py feat(parser): implement AST-based markdown chunking with full document TOC 2026-05-13 14:32:00 +08:00
memory_io.py feat(parser): implement AST-based markdown chunking with full document TOC 2026-05-13 14:32:00 +08:00
memory_toolkit.py ``` 2026-05-11 19:45:53 +08:00
protocol.md ``` 2026-05-11 19:45:53 +08:00
retriever.py ``` 2026-05-11 10:52:25 +08:00
summarizer.py ``` 2026-05-08 16:14:42 +08:00
summarizer.yaml ``` 2026-05-08 16:14:42 +08:00