Commit graph

7 commits

Author SHA1 Message Date
huangsen
51ec09d98f feat(parser): implement AST-based markdown chunking with full document TOC
- Replace legacy line-based chunking with AST tree approach that builds
  a complete document skeleton with content inlined under relevant
  sections
- Add new chunking parameters: chunk_chars (default 2000) and embed_toc
  (default True) to control content size and TOC inclusion
- Implement recursive chunking algorithm that respects structural
  boundaries (code lines, table rows, list items) and prevents splits
  inside blocks
- Introduce part markers [Part X/N] for oversized leaf blocks that
  require splitting
- Add CLI tool for inspecting parsed chunks and edges with options for
  preview and configuration
- Refactor edge extraction to use FileEdge.from_text instead of
  parse_wikilinks for consistency

BREAKING CHANGE: Chunk format changes significantly with full TOC
skeleton wrapping content, affecting embedding models expecting
breadcrumb prefixes.
2026-05-13 14:32:00 +08:00
jinli.yl
5c28dd8be2 up
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
2026-05-13 12:22:59 +08:00
jinli.yl
ef0b74b3e7 up 2026-05-12 20:46:40 +08:00
huangsen
8465f6d06e ```
docs(protocol): add typed edge link protocol documentation

Add comprehensive documentation for the link protocol supporting
typed edges in body text. This includes specification for three
legal inline forms (bare wikilink, line-level Dataview,
inline-bracketed Dataview), predicate syntax rules, and the
machine-managed Relations section convention for organizing
discovered edges.

fix(memory): update path reference from vault_root to working_dir

Change the memory_create operation's path anchoring from
vault_root to working_dir to maintain consistency with the
current working directory configuration.

refactor(components): remove edge_extractor module and simplify parsing

Remove the edge_extractor component module entirely and
inline edge extraction logic directly into LinkedFileParser
using parse_wikilinks utility. This simplifies the architecture
by eliminating the separate edge extraction component and
delegating edge discovery to the maintainer's enrichment operations.

feat(parser): update parse method signature and simplify edge extraction

Modify LinkedFileParser to return (FileNode, list[FileChunk])
tuple instead of ParsedFile, remove dependency on BaseEdgeExtractor,
and implement direct wikilink parsing from body text only.
```
2026-05-11 19:45:53 +08:00
jinli.yl
662e1768dc up
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
2026-05-11 17:18:07 +08:00
huangsen
dd2de16481 ```
feat(file_watcher): add directory deletion support with descendant indexing

Add support for deleting entire directories and their indexed descendants
in the file watcher. Previously only individual file deletions were
handled properly. Now when a directory is deleted, the system finds all
indexed files beneath that directory path and removes them along with
their metadata and chunks.

The implementation includes:
- New `_descendant_indexed_paths` method to find all indexed files
  under a given directory path
- Updated `_on_deleted` method to process both the target path and
  all its indexed descendants
- Proper handling of symlinks and path resolution differences
- Enhanced logging to show directory deletion with child count

Also adds necessary os import for path operations.

refactor(config): restructure configuration profiles for clarity

Rename curated.yaml to remove outdated configuration file and
rename full.yaml to expert.yaml with updated documentation.
Add new service.yaml configuration profile that provides a
service-aligned MCP surface with three main tools:
- retrieve: graph-aware hybrid retrieval
- remember: single write entry point with log/distill modes
- maintain: vault hygiene sweep

The expert configuration now excludes the ingest tool since
cold-path operations are handled by external agents, and adds
memory_lint tool for structural issue detection.

Updated documentation to clarify the different configuration
profiles and their intended usage patterns.
```
2026-05-11 10:52:25 +08:00
huangsen
514bf35050 ```
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
docs: add ReMe2 architecture design documentation

- Add comprehensive design document (reme2.md) detailing the
  three-layer architecture (L1/L2/L3) for the vault system
- Document new protocols for folder notes and memory management
- Specify interface contracts for memory_* and vault_* tools
- Outline implementation phases from current state to target

refactor: fix typo in personal retriever class

- Correct spelling error: 'retri eved_nodes' -> 'retrieved_nodes'
  in PersonalRetriever.result assignment

chore: update gitignore with vault-related patterns

- Add '/vault' to ignore vault directory
- Add '/reme-plugin' to ignore plugin files
- Add '/reme2/vault' to ignore new vault implementation
```
2026-05-08 16:14:42 +08:00