refactor(component): rename file_store to chunk_store and update interfaces
- Rename BaseFileStore to BaseChunkStore and update component type
- Replace file_store property with chunk_store in BaseStep
- Add file_graph property to access file metadata from FileWatcher
- Update all storage backends (Chroma, Local, SQLite) to use chunk-focused APIs
- Remove file metadata handling from chunk stores (moved to FileGraph)
- Update search methods to use ChunkFilter instead of SearchFilter
- Remove file_store imports and add chunk_store imports
```
- Create MCPService class to expose jobs as MCP tools
- Implement FastMCP integration with async lifespan management
- Add environment variable setup for service information
- Implement tool registration for job execution via FunctionTool
- Support multiple transport types (sse, stdio) with configurable host/port
- Integrate with application lifecycle for proper startup/shutdown
- replace watch_paths parameter with single watch_path string
- add FileGraph integration for tracking file relationships
- implement persistent graph storage in .reme directory
- add load/build graph functionality on start
- refactor file change handling with separate methods for add/modify/delete
- update scan_existing_files to work with single path
- add file filter to exclude meta directory from watching
- remove abstract method requirement and implement concrete change handlers
BREAKING CHANGE: watch_paths parameter changed to watch_path (single path)
- Introduce BaseFileParser abstract class with component registration
- Add MdFileParser implementation for markdown files with YAML frontmatter
- Create TextFileParser implementation with built-in chunking support
- Implement file suffix enumeration for parser type safety
- Add chunking logic with configurable token size and overlap
- Support text file parsing with error handling for encoding issues
- Include line number tracking and content hashing for file chunks
- Removed hash, size, content, and chunk_count fields from FileMetadata model
- Updated chroma and local file stores to remove excluded fields from model dump
- Removed hash calculation and content storage from default file parser
- Removed chunk count tracking from file parsing logic
- Simplified FileMetadata schema to only include essential fields
- Added new SQLite file store implementation with vector and full-text search
- Added memory search step for semantic search functionality
- Remove abstract methods from base component start/close
- Update BaseJob to remove name parameter and simplify initialization
- Change file modification time field from mtime_ms to modified_time in seconds
- Add type checking imports and improve typing annotations
- Implement LocalFileStore with JSONL persistence for file chunks
- Add MdFileParser with markdown and frontmatter support
- Simplify HttpClient call method with proper kwargs handling
- Remove unused ReMe class methods and create backup version
- Update StreamJob to use step_components instead of steps attribute
- Introduce Application class for managing application lifecycle
- Add base component classes for LLM formatters and token counters
- Implement embedding model base with caching and batching support
- Create file watcher base with watchfiles integration
- Add job and step base components for workflow execution
- Update base component with async locks and improved lifecycle management
- Register new component types in component registry
- Add application context and runtime context for dependency injection
- Integrate AnthropicChatModel with new AnthropicAsLLM component
- Add component formatters for OpenAI and Anthropic chat models
- Implement token counter component with estimated token counting
- Create base client component for ReMe service communication
- Refactor BaseComponent to remove app_context parameter from _start/_close
- Update embedding model base class to remove retry logic and use npz cache
- Add job component for sequential step execution with BaseJob
- Implement step component base class for LLM workflow execution
- Enhance application context with proper type annotations
- Update component initialization to pass app_context automatically
- Remove asyncio dependency from embedding model cache operations
feat(file-store): add Chroma and SQLite file store implementations
- Add ChromaFileStore and SqliteFileStore classes to support additional
storage backends
- Export new store classes in __init__.py module
refactor(file-store): enhance BaseFileStore with hybrid search capabilities
- Add keyword scoring utility with word-match ratio and phrase bonus
- Implement hybrid search method that combines vector and keyword results
- Add merge logic for combining vector and keyword search results with
weighted scoring
- Move abstract methods to separate section for better organization
- Remove redundant search filter parameter from local implementation
refactor(file-store): simplify LocalFileStore implementation
- Remove unused delete_file_chunks and upsert_chunks methods
- Remove redundant update_file_metadata method
- Update chunk counting logic to use file_meta directly
- Simplify keyword search to use new base class scoring utility
- Remove duplicate hybrid search implementation since it's now in base class
```
- Add file_parser component with default implementation
- Introduce SearchFilter schema for path and tag filtering
- Implement filter functionality in BaseFileStore and LocalFileStore
- Update file watcher to use parser-based filtering instead of suffix filters
- Register new FILE_PARSER component enum
- Add test_data directory to gitignore
refactor: improve component imports and initialization
- Fix relative imports in application.py
- Add file_parser import to component init
- Initialize registry dict when component type doesn't exist
- Remove circular import in HttpService by using string annotation
- Update config yaml to use proper component names
refactor: enhance file watcher architecture
- Replace MdFileWatcher with more flexible FullFileWatcher and LightFileWatcher
- Remove suffix-based filtering in favor of parser-based approach
- Update BaseFileWatcher to resolve parsers from app context
- Remove unused watch_filter method
refactor: update ReMe core functionality
- Remove memory_path creation
- Simplify dream and proactive methods to return empty strings
- Update config defaults for HTTP service and component backends
docs: update component configuration in paw.yaml
- Change service backend from cmd to http
- Rename components to use correct singular forms
- Add default file parser and file watcher configurations
- Set up local file store with default settings
```
Co-authored-by: huangsen <huangsen.huang@alibaba-inc.com>
- Changed condition from divisor equals zero to divisor less than or equal to zero
- Prevents division by zero error when estimate_divisor is negative
- Maintains validation logic for invalid divisor values
- Introduce BaseAsTokenCounter and EstimatedAsTokenCounter for token estimation
- Add AsMsgStat and AsBlockStat schema for message statistics tracking
- Implement FileIO class with read/write/append/edit operations
- Create file utility functions for safe async file reading and truncation
- Add MemorySearch component for semantic search in memory files
- Register new component types in ComponentEnum and update imports
- Add constants for default host, port, and truncation limits
- Create BaseService abstract base class for service implementations
- Implement BaseStep with component accessors and lifecycle management
- Add proper __all__ exports for all new modules and components
