3 commits

Author	SHA1	Message	Date
jinliyl	8eaa96390a	refactor(file_chunker): replace file parser with file chunker component (#276) ... * refactor(file_chunker): replace file parser with file chunker component - Rename file_parser module to file_chunker across codebase - Update BaseFileParser to BaseFileChunker with corresponding component type - Rename LinkedFileParser to MarkdownFileChunker for markdown-specific chunking - Rename ChunkedFileParser to DefaultFileChunker for default byte-based chunking - Update documentation references from file_parser to file_chunker - Modify dependency injection in BaseStep to use file_chunker instead of file_parser - Update configuration and component registration to use new chunker naming - Rename all related test files and update test assertions accordingly - Add recursive option to scan_store_changes_step in default configuration * feat(database): enhance Neo4j connection with environment variable support - Add support for NEO4J_PASSWORD environment variable as fallback - Make password parameter optional in constructor with validation - Update chromadb dependency from 1.3.5 to 1.5.7 - Configure CORS credentials based on origin settings - Import os module for environment variable access * feat(config): add timezone support and remove unused dialog directory - Added timezone field to application config with IANA timezone support - Removed unused dialog_dir configuration and related directory creation - Replaced date.today() with timezone-aware now() function across daily operations - Created evolve module with timezone-aware datetime functionality - Updated daily_create, daily_list, and daily_reindex steps to use timezone-aware dates * refactor(steps): update file chunker implementation - Replace ChunkedFileParser with DefaultFileChunker in background steps - Add module docstring to evolve steps package - Update return type annotation to reflect new chunker class usage * refactor(components): rename embedding and llm components to as_embedding and as_llm - Rename reme4/components/embedding to reme4/components/as_embedding - Rename reme4/components/llm to reme4/components/as_llm - Update all imports and references from embedding to as_embedding - Update all imports and references from llm to as_llm - Change BaseEmbedding to BaseAsEmbedding and update inheritance - Change BaseLLM to BaseAsLLM and update inheritance - Update component types from LLM/EMBEDDING to AS_LLM/AS_EMBEDDING - Update configuration keys from embedding/llm to as_embedding/as_llm - Update all property references from llm to as_llm in step classes - Update test assertions to use new component enum values * refactor(embedding_store): rename embedding parameter to as_embedding - Updated configuration key from 'embedding' to 'as_embedding' - Renamed class attribute from 'embedding' to 'as_embedding' - Updated method calls to use 'as_embedding' instead of 'embedding' - Changed parameter name in constructor from 'embedding' to 'as_embedding' - Updated documentation to reflect new parameter name - Modified health check to use 'as_embedding' property * feat(agent_wrapper): add unified agent wrapper component with multiple backends - Introduce BaseAgentWrapper abstract base class for agent implementations - Add AsAgentWrapper implementation using AgentScope framework - Add CcAgentWrapper implementation using Claude Code SDK - Register agent_wrapper component type in ComponentEnum - Configure default agent_wrapper settings in default.yaml - Implement tool integration for both AgentScope and Claude Code backends - Support fluent configuration via set_system_prompt() and add_tools() methods * feat(agent-wrapper): add structured output support for agent wrappers - Import SystemMsg in AsAgentWrapper for structured output handling - Add output_schema parameter support in AsAgentWrapper with generate_structured_output - Implement set_output_schema method in BaseAgentWrapper for chaining configuration - Add output schema support in CcAgentWrapper with JSON schema format option - Return structured output when available in CcAgentWrapper response - Refactor kwargs handling to use default values consistently across wrapper classes	2026-06-05 17:27:54 +08:00
jinliyl	a4efc0f776	refactor(reme4): restructure steps packages (#258) ... * fix(bm25_index): 修正BM25索引计算中的文档长度归一化问题 修复了在计算BM25相似度时对文档长度进行不正确归一化的bug，确保所有查询都能得到准确的相关性评分。 * up * up * up * up * up * up * up * up * up * up * up * up * up * up * up * up * up * refactor(steps): Rename and adjust indexing step logic - Rename `scan_changes.py` and `reindex.py` to `clear_and_scan.py` - Update implementation details of `ScanChangesStep` and `ClearAndScanStep` - Modify the scheduling mechanism in `WatchChangesStep` - Adjust step registration and parameter configuration in config files - Update related tests to align with the new interface changes * up * feat(daily): replace daily CRUD operations with slug provisioning approach * refactor(tests): migrate CRUD step tests from HTTP server to direct LocalFileStore * up * up * up * up --------- Co-authored-by: huangsen <huangsen.huang@alibaba-inc.com>	2026-05-28 14:30:30 +08:00
Sen Huang	bb354cc580	refactor(steps): reorganize step modules and remove demo steps (#255) ... * feat(config): add comprehensive job definitions for vault operations - Add utility jobs like version, search, traverse, list, read, stat - Include file operations like move, delete, upload, download - Add daily workspace management jobs: daily_list, daily_resolve, daily_reindex - Update descriptions to reflect vault-based operations instead of working_dir - Add proper section headers and documentation for each job category refactor(steps): reorganize step modules and remove demo steps - Move steps into categorized packages: common, crud, frontmatter, daily, jobs - Remove demo steps (DemoEchoStep1, DemoEchoStep2, StreamDemoStep1, StreamDemoStep2) - Add new steps: InitStep for vault initialization, TraverseStep for graph traversal - Update __init__.py to auto-import all step modules - Organize imports by functionality (common, CRUD operations, frontmatter, daily) feat(vault): implement vault-centric file operations and configuration - Change default config to use vault_dir instead of working_dir - Add environment variable support for embedding configuration - Implement file watcher with lite backend for daily/digest directories - Update search step to use 'name' instead of 'title' from frontmatter - Create ResourceEntry schema for tracking uploaded assets docs(steps): add comprehensive documentation for all step categories - Document file-I/O split by blast radius (crud vs frontmatter packages) - Add detailed descriptions for each step category and functionality - Explain the purpose and usage patterns for different types of file operations - Provide clear parameter documentation for all new job configurations * fix(config): correct vault directory path and remove unused job configurations - Fix vault_dir from 'vaultd' to 'vault' in default configuration - Remove deprecated traverse and list job configurations - Remove unused tag tooling configurations - Remove background watch_file job configuration refactor(steps): remove unused jobs module import - Comment out jobs module import in steps/__init__.py - This removes unused synchronizer and digester step registrations refactor(tests): update import path and add pylint directive - Update ResourceEntry import from reme4.schema to reme4.schema.resource_meta - Add pylint disable directive for unused argument in test datetime mocks * efactor(steps): remove unused modules from __all__ - Remove "background" module from __all__ list - Remove "jobs" module from __all__ list - These modules were no longer being used in the steps package * feat(config): update vault directory structure and remove file watcher - Change vault_dir reference from ./vault to ./vault in CLI example - Add daily_dir, digest_dir, and resource_dir configuration options - Remove file_watcher component configuration as it's no longer needed - Update comment to reflect correct module name (reme4vault) refactor(steps): add background step and remove deprecated init step - Import and register background step module - Remove deprecated InitStep from common steps - Update __all__ export list to include background step refactor(reindex): improve reindex step to scan vault directly - Update docstring to reflect vault scanning instead of watcher sync - Replace file watcher stop/start logic with direct vault path walking - Add support for suffix filtering during reindex operation - Use index_changes job to process found files refactor(wikilink_utils): enhance inbound source lookup with link scope - Import LinkScopeEnum for proper type handling - Update get_inlinks call to use ALL scope for virtual targets - Improve documentation for reverse-index lookup behavior test(refactor): clean up test suite removing deprecated functionality - Remove test_init_job and test_demo_job unit tests - Update help job assertion to check for literal command format - Change test directory from .reme to vault in CRUD tests - Remove init and demo job calls from integration test BREAKING CHANGE: Removes file_watcher component and init step * style(steps): fix import formatting in __init__.py Add proper spacing in the background module import statement to maintain consistent code style and readability. * refactor(config): change default vault directory from vault to .reme Default dev config now points vault_dir at ./.reme so `python -m reme4 start` can be run from the repo root and exercise the full atomic-tool surface against the seeded test data. BREAKING CHANGE: The default vault directory has been changed from 'vault' to '.reme' in the configuration. * docs(reme4_report): fix markdown formatting and remove extra content * refactor(file_parser): delegate wikilink extraction to WikilinkHandler * fix(search): handle empty query case gracefully - Replace assertion with conditional check for empty query - Set response success to false when query is empty - Return error message instead of throwing assertion error - Maintain existing validation for other parameters	2026-05-25 17:52:51 +08:00