ReMe/reme2/component
huangsen 51ec09d98f feat(parser): implement AST-based markdown chunking with full document TOC
- Replace legacy line-based chunking with AST tree approach that builds
  a complete document skeleton with content inlined under relevant
  sections
- Add new chunking parameters: chunk_chars (default 2000) and embed_toc
  (default True) to control content size and TOC inclusion
- Implement recursive chunking algorithm that respects structural
  boundaries (code lines, table rows, list items) and prevents splits
  inside blocks
- Introduce part markers [Part X/N] for oversized leaf blocks that
  require splitting
- Add CLI tool for inspecting parsed chunks and edges with options for
  preview and configuration
- Refactor edge extraction to use FileEdge.from_text instead of
  parse_wikilinks for consistency

BREAKING CHANGE: Chunk format changes significantly with full TOC
skeleton wrapping content, affecting embedding models expecting
breadcrumb prefixes.
2026-05-13 14:32:00 +08:00
..
as_llm feat(components): add Anthropic LLM support and enhance component architecture 2026-04-23 14:18:57 +08:00
as_llm_formatter ``` 2026-05-08 16:14:42 +08:00
as_token_counter ``` 2026-05-08 16:14:42 +08:00
client ``` 2026-05-08 16:14:42 +08:00
embedding up 2026-05-13 14:25:52 +08:00
file_parser feat(parser): implement AST-based markdown chunking with full document TOC 2026-05-13 14:32:00 +08:00
file_store up 2026-05-13 12:22:59 +08:00
file_watcher up 2026-05-13 12:22:59 +08:00
job ``` 2026-05-08 16:14:42 +08:00
service ``` 2026-05-08 16:14:42 +08:00
tokenizer up 2026-05-12 20:32:59 +08:00
__init__.py ``` 2026-05-11 19:45:53 +08:00
application_context.py feat(components): add Anthropic LLM support and enhance component architecture 2026-04-23 14:18:57 +08:00
base_component.py up 2026-05-12 20:28:49 +08:00
base_step.py ``` 2026-05-08 16:14:42 +08:00
component_registry.py ``` 2026-05-08 16:14:42 +08:00
prompt_handler.py ``` 2026-05-08 16:14:42 +08:00
runtime_context.py feat(core): add core components and application framework 2026-04-23 16:25:31 +08:00
runtime_response.py ``` 2026-05-11 10:52:25 +08:00