mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-09-22 00:32:49 +00:00
3fc3fd65e8
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
- Introduce BaseFileParser abstract class with component registration - Add MdFileParser implementation for markdown files with YAML frontmatter - Create TextFileParser implementation with built-in chunking support - Implement file suffix enumeration for parser type safety - Add chunking logic with configurable token size and overlap - Support text file parsing with error handling for encoding issues - Include line number tracking and content hashing for file chunks
19 lines
353 B
Python
19 lines
353 B
Python
"""File suffix enumeration module.
|
|
|
|
Defines the file suffixes that can be handled by file parsers.
|
|
"""
|
|
|
|
from enum import Enum
|
|
|
|
|
|
class FileSuffixEnum(str, Enum):
|
|
"""Enumeration of supported file suffixes.
|
|
|
|
Each value represents a file extension that a file parser can handle.
|
|
"""
|
|
|
|
MD = ".md"
|
|
|
|
MARKDOWN = ".markdown"
|
|
|
|
TXT = ".txt"
|