5 commits

Author	SHA1	Message	Date
jinliyl	8eaa96390a	refactor(file_chunker): replace file parser with file chunker component (#276) ... * refactor(file_chunker): replace file parser with file chunker component - Rename file_parser module to file_chunker across codebase - Update BaseFileParser to BaseFileChunker with corresponding component type - Rename LinkedFileParser to MarkdownFileChunker for markdown-specific chunking - Rename ChunkedFileParser to DefaultFileChunker for default byte-based chunking - Update documentation references from file_parser to file_chunker - Modify dependency injection in BaseStep to use file_chunker instead of file_parser - Update configuration and component registration to use new chunker naming - Rename all related test files and update test assertions accordingly - Add recursive option to scan_store_changes_step in default configuration * feat(database): enhance Neo4j connection with environment variable support - Add support for NEO4J_PASSWORD environment variable as fallback - Make password parameter optional in constructor with validation - Update chromadb dependency from 1.3.5 to 1.5.7 - Configure CORS credentials based on origin settings - Import os module for environment variable access * feat(config): add timezone support and remove unused dialog directory - Added timezone field to application config with IANA timezone support - Removed unused dialog_dir configuration and related directory creation - Replaced date.today() with timezone-aware now() function across daily operations - Created evolve module with timezone-aware datetime functionality - Updated daily_create, daily_list, and daily_reindex steps to use timezone-aware dates * refactor(steps): update file chunker implementation - Replace ChunkedFileParser with DefaultFileChunker in background steps - Add module docstring to evolve steps package - Update return type annotation to reflect new chunker class usage * refactor(components): rename embedding and llm components to as_embedding and as_llm - Rename reme4/components/embedding to reme4/components/as_embedding - Rename reme4/components/llm to reme4/components/as_llm - Update all imports and references from embedding to as_embedding - Update all imports and references from llm to as_llm - Change BaseEmbedding to BaseAsEmbedding and update inheritance - Change BaseLLM to BaseAsLLM and update inheritance - Update component types from LLM/EMBEDDING to AS_LLM/AS_EMBEDDING - Update configuration keys from embedding/llm to as_embedding/as_llm - Update all property references from llm to as_llm in step classes - Update test assertions to use new component enum values * refactor(embedding_store): rename embedding parameter to as_embedding - Updated configuration key from 'embedding' to 'as_embedding' - Renamed class attribute from 'embedding' to 'as_embedding' - Updated method calls to use 'as_embedding' instead of 'embedding' - Changed parameter name in constructor from 'embedding' to 'as_embedding' - Updated documentation to reflect new parameter name - Modified health check to use 'as_embedding' property * feat(agent_wrapper): add unified agent wrapper component with multiple backends - Introduce BaseAgentWrapper abstract base class for agent implementations - Add AsAgentWrapper implementation using AgentScope framework - Add CcAgentWrapper implementation using Claude Code SDK - Register agent_wrapper component type in ComponentEnum - Configure default agent_wrapper settings in default.yaml - Implement tool integration for both AgentScope and Claude Code backends - Support fluent configuration via set_system_prompt() and add_tools() methods * feat(agent-wrapper): add structured output support for agent wrappers - Import SystemMsg in AsAgentWrapper for structured output handling - Add output_schema parameter support in AsAgentWrapper with generate_structured_output - Implement set_output_schema method in BaseAgentWrapper for chaining configuration - Add output schema support in CcAgentWrapper with JSON schema format option - Return structured output when available in CcAgentWrapper response - Refactor kwargs handling to use default values consistently across wrapper classes	2026-06-05 17:27:54 +08:00
jinliyl	d8086039dc	refactor(Agentscope2.0): llm & embedding & agent (#271) ... * refactor(embedding): replace embedding model with embedding store architecture - Remove as_token_counter component and its estimated token counter implementation - Replace BaseEmbeddingModel with BaseEmbedding that wraps AgentScope embedding models - Add support for multiple embedding providers (OpenAI, DashScope, Gemini, Ollama) - Introduce BaseEmbeddingStore and LocalEmbeddingStore for caching and persistence - Update component registry to use new embedding and embedding_store types - Modify file stores to use embedding_store instead of embedding_model - Update health check to monitor embedding_store instead of embedding_model - Change default config to use embedding_store with local backend - Add estimate_token_count utility function to utils module * refactor(llm): replace as_llm components with unified llm implementation - Remove deprecated as_llm and as_llm_formatter modules - Add new llm module with BaseLLM and provider-specific implementations - Update component registry to use LLM instead of AS_LLM - Replace all as_llm/as_llm_formatter references with llm in steps - Update configuration schema to use llm instead of as_llm - Rename integration test file from test_as_llm to test_llm - Add proper docstrings to embedding store dimension property - Add pylint disable comment for embedding model call - Remove unused FormatterBase import in base_step - Update token_utils with function docstring * refactor(evolve): replace ReActAgent with Agent and update message handling - Removed FlexReActAgent class and direct ReActAgent imports - Updated Agent instantiation to use new constructor parameters - Changed message content to use TextBlock format instead of plain strings - Modified timestamp access from msg.timestamp to msg.created_at - Updated metadata access pattern for structured outputs - Replaced Msg.from_dict with Msg.model_validate in auto_memory.py - Updated test mocks to patch Agent instead of ReActAgent - Changed message serialization from to_dict to model_dump in tests - Moved component references to base class definition - Updated demo tools to return strings instead of ToolResponse objects * feat(step): migrate to FunctionTool and add streaming support - Replace deprecated ToolResponse with FunctionTool in base_step.py - Remove unused TextBlock import from base_step.py - Update job registration to use new FunctionTool API - Add thinking_budget parameter to llm_demo configuration - Introduce StreamLLMDemoStep with streaming output capability - Add structured output support to LLMDemoStep via generate_structured_output - Implement streaming event handling for text/thinking/tool calls - Add integration tests for embedding functionality - Add integration tests for structured output and streaming features - Update tool usage in demo steps to use new function naming convention * fix(ci): correct package installation path in unittest workflow - Updated pip install command to use proper package path "./reme4[dev,core]" - Fixed dependency installation step in CI workflow configuration * chore(workflow): update python versions in unittest workflow - Remove Python 3.10 from test matrix - Add Python 3.11 to test matrix - Add Python 3.12 to test matrix - Keep Python 3.13 in test matrix - Update matrix configuration for better version coverage * fix(health): handle missing dimensions attribute in embedding status - Wrap dimensions access in try-except to prevent AttributeError - Return None when dimensions attribute is not available - Maintain backward compatibility for components without dimensions test(component): add comprehensive tests for BaseComponent and related classes - Add tests for Dependency class including repr and attribute access - Add tests for bind method with various scenarios and edge cases - Add tests for lifecycle management and async context handling - Add tests for standalone and context-bound dependency resolution - Add tests for ComponentMixin path utilities test(common): update LocalFileStore initialization parameter - Change embedding_model parameter to embedding_store in test setup - Update all affected test files consistently test(registry): add complete test suite for ComponentRegistry - Add tests for register method with explicit names and defaults - Add tests for decorator registration pattern - Add tests for get_all method returning copies - Add tests for unregister and clear operations - Add tests for error handling of invalid registrations test(job): add comprehensive tests for BaseJob and BackgroundJob - Add tests for step resolution and exception handling - Add tests for backoff delay calculation with jitter - Add tests for supervisor loop restart behavior - Add tests for task shutdown and cancellation test(prompt): add complete test suite for PromptHandler - Add tests for prompt loading from dictionaries and files - Add tests for internationalization and language fallback - Add tests for flag filtering and variable substitution - Add tests for format validation and error handling test(runtime): add basic tests for RuntimeContext dictionary access - Add tests for item getting, setting and containment checks - Add tests for missing key error handling * feat(evolve): add permission context and agent state management - Import PermissionContext, PermissionMode and AgentState modules - Add state configuration with bypass permission mode to AutoDream agents - Add state configuration with bypass permission mode to AutoMemory agents - Implement static _to_msg method for message validation and formatting - Refactor message processing to use the new _to_msg method - Ensure proper content structure for text blocks in message conversion * style(tests): update test files with linting rules and code improvements - Add missing pylint disable directives for docstring and attribute warnings - Replace lambda expressions with proper function definitions in test cases - Import Path directly instead of using lambda with __import__ - Simplify assertion checks by using truthiness instead of equality to empty dict - Remove unused imports and reorder imports consistently - Format dictionary literals with proper indentation and line breaks	2026-06-03 11:35:43 +08:00
jinliyl	a4efc0f776	refactor(reme4): restructure steps packages (#258) ... * fix(bm25_index): 修正BM25索引计算中的文档长度归一化问题 修复了在计算BM25相似度时对文档长度进行不正确归一化的bug，确保所有查询都能得到准确的相关性评分。 * up * up * up * up * up * up * up * up * up * up * up * up * up * up * up * up * up * refactor(steps): Rename and adjust indexing step logic - Rename `scan_changes.py` and `reindex.py` to `clear_and_scan.py` - Update implementation details of `ScanChangesStep` and `ClearAndScanStep` - Modify the scheduling mechanism in `WatchChangesStep` - Adjust step registration and parameter configuration in config files - Update related tests to align with the new interface changes * up * feat(daily): replace daily CRUD operations with slug provisioning approach * refactor(tests): migrate CRUD step tests from HTTP server to direct LocalFileStore * up * up * up * up --------- Co-authored-by: huangsen <huangsen.huang@alibaba-inc.com>	2026-05-28 14:30:30 +08:00
jinliyl	71e42dbad0	refactor(reme4): replace file_watcher component with background steps pipeline (#251) ... * up * up * up * up * up * up * up * up * up * up * up * feat(config): add daily_dir configuration and background job logging - Added daily_dir setting with default value 'memory' to config - Implemented logging for background job startup events - Enhanced component start logic to handle background backend type - Updated default YAML configuration structure * refactor(file_parser): replace _get_relative_path with to_vault_relative method - Remove redundant working_dir property from base file parser - Add to_vault_relative method to base component for path resolution - Update bare_file_parser to use new to_vault_relative method - Update default_file_parser to use new to_vault_relative method - Update linked_file_parser to use new to_vault_relative method - Make working_path absolute in base_component and steps - Simplify index_changes step by removing redundant base variable - Consolidate path relative logic in single shared method * docs(reme4): update report with detailed architecture sections - Add comprehensive Markdown kernel section covering Obsidian compatibility - Include detailed explanation of YAML front matter and wikilink formats - Document smart slicing mechanism using Markdown AST instead of fixed tokens - Explain graph indexing with bidirectional links and multiple backends - Restructure sections with proper numbering from 4 to 7 - Move Markdown kernel section to appear before self-evolution features - Add detailed explanations of auto-memory, auto-dream, and auto-link processes - Document three-way hybrid search with RRF fusion and progressive expansion - Include engineering value explanations for keyword indexing in Chinese context	2026-05-22 10:26:36 +08:00
jinliyl	e411eeb4c0	dev/reme4 init merge (#236)	2026-05-17 14:14:43 +08:00