4 commits

Author	SHA1	Message	Date
jinliyl	c3fb825af0	feat(agent): refactor agent wrapper, add session persistence, auto_resource step, and watch-loop improvements (#277) ... * refactor(agent_wrapper): update agent wrapper implementations and config defaults - Set default timezone to Asia/Shanghai in application config - Add AgentScope imports and configure ReAct, context, and model configs - Simplify __all__ export formatting in agent wrapper init - Remove redundant docstring details from agent wrapper classes - Optimize tool result handling with state assignment simplification - Add permission context and state management for AgentScope backend - Update Claude Code wrapper tool creation and server registration logic - Configure default agent settings including permission mode and retry limits - Remove obsolete comments and streamline code structure * fix(agent): add output schema validation and BaseModel support - Added type assertion to ensure output_schema is a dict in as_agent_wrapper - Imported BaseModel from pydantic in base_agent_wrapper - Modified set_output_schema to accept both dict and BaseModel types - Added automatic conversion of BaseModel to JSON schema - Updated method documentation to reflect new type support * refactor(agent): replace direct agent instantiation with agent wrapper component - Removed manual Agent creation and initialization in llm_demo step - Integrated agent_wrapper component as dependency in base step - Updated llm_demo step to use agent_wrapper.reply method instead of direct agent calls - Modified structured output handling to work with new agent wrapper interface - Simplified agent configuration by using wrapper's built-in functionality - Updated documentation to reflect agent wrapper usage instead of direct as_llm access - Removed redundant imports related to manual agent management * feat(agent): add streaming support and refactor agent wrapper components - Introduce reply_stream method in base agent wrapper with fallback implementation - Add _build_agent helper method to AsAgentWrapper for agent instantiation - Implement structured output generation with proper model assertions - Update StreamLLMDemoStep to use agent_wrapper instead of direct Agent calls - Replace manual streaming logic with execute_stream_task utility function - Change default system prompt to provide detailed responses instead of concise ones - Add colored output support for different chunk types in streaming demos - Refactor test cases to use async task execution with streaming verification * refactor(agent): remove session_id parameter from reply methods - Removed session_id parameter from ASAgentWrapper.reply method signature - Removed session_id parameter from BaseAgentWrapper.reply abstract method - Removed session_id parameter from CCAgentWrapper.reply method signature - Updated reply_stream methods to remove session_id parameter across all wrappers - Modified CCAgentWrapper to use dynamic options assignment instead of hardcoded properties - Set default system_prompt in config instead of hardcoded in code - Increased default max_turns from 10 to 50 in configuration * config: update default configuration and script entry point - Change resource_dir from empty string to 'resource' - Update command line entry point from 'reme4' to 'reme' * feat(agent): add session state persistence and forking support - Implement AsStateHandler for AgentState JSONL serialization - Add session_id parameter to AsAgentWrapper.reply method - Create timestamp-based session file paths with timezone support - Load existing session state from JSONL files when session_id provided - Save updated session state after each agent interaction - Support session forking with UUID generation for new sessions - Add integration tests for session persistence and forking scenarios - Include temporary directory utilities for testing isolated sessions - Ensure parent directories are created for session files automatically * refactor(auto_memory): replace transcript parsing with direct message handling - Remove transcript loading logic and related dependencies - Add session message saving functionality with deduplication - Use agent wrapper instead of direct AgentScope agent instantiation - Simplify timezone handling using shared now utility - Update logging and response metadata structure - Remove unused imports and toolkit management methods - Change session file naming from session_{id}.jsonl to session_agent_{id}.jsonl * refactor(steps): move channel steps from index to channel module - Move ChannelNotifyStep from .index.channel_notify to .channel.channel_notify - Move ClaimChannelStep from .index.claim_channel to .channel.claim_channel - Update __init__.py imports to reflect new module structure - Reorganize steps list in __init__.py with channel section before index - Add proper file prefix handling in daily index processing - Update test imports to use new channel module location * feat(evolve): add auto_resource step for interpreting resource files - Add AutoResourceStep to interpret resource files into daily notes via an agent - Implement resource file parsing with date and filename extraction logic - Add session ID computation using MD5 hash of filename - Create delete and upsert handlers for resource file operations - Add truncation and sanitization functions for tool output in auto_memory - Register auto_resource step with proper parameter validation - Add configuration for resource watch loop with file extension filters - Update default YAML config to include resource watch and digest watch loops - Add shared watch-rule logic for scan_changes and watch_changes steps - Implement foreach_dispatch and log_changes steps for change processing - Rename update_store_index_loop to index_update_loop in configuration - Refactor file chunking interface from parse to chunk method - Remove unused imports and dependencies in auto_dream step - Fix path iteration formatting in daily_index utility function - Add comprehensive integration tests for auto_resource functionality * refactor(auto_resource): format function call with multi-line parameters - Reformatted await _handle_upsert call to use multiple lines for better readability - Removed unused imports from scan_changes.py including BaseFileCatalog and ComponentEnum - Added date parameter to RuntimeContext initialization in test cases - Updated expected file paths in test assertions to include session_agent prefix - Formatted long assertion statements across multiple lines to maintain character limit - Corrected wikilink references from generic names to session_agent prefixed names	2026-06-08 16:11:32 +08:00
jinliyl	8eaa96390a	refactor(file_chunker): replace file parser with file chunker component (#276) ... * refactor(file_chunker): replace file parser with file chunker component - Rename file_parser module to file_chunker across codebase - Update BaseFileParser to BaseFileChunker with corresponding component type - Rename LinkedFileParser to MarkdownFileChunker for markdown-specific chunking - Rename ChunkedFileParser to DefaultFileChunker for default byte-based chunking - Update documentation references from file_parser to file_chunker - Modify dependency injection in BaseStep to use file_chunker instead of file_parser - Update configuration and component registration to use new chunker naming - Rename all related test files and update test assertions accordingly - Add recursive option to scan_store_changes_step in default configuration * feat(database): enhance Neo4j connection with environment variable support - Add support for NEO4J_PASSWORD environment variable as fallback - Make password parameter optional in constructor with validation - Update chromadb dependency from 1.3.5 to 1.5.7 - Configure CORS credentials based on origin settings - Import os module for environment variable access * feat(config): add timezone support and remove unused dialog directory - Added timezone field to application config with IANA timezone support - Removed unused dialog_dir configuration and related directory creation - Replaced date.today() with timezone-aware now() function across daily operations - Created evolve module with timezone-aware datetime functionality - Updated daily_create, daily_list, and daily_reindex steps to use timezone-aware dates * refactor(steps): update file chunker implementation - Replace ChunkedFileParser with DefaultFileChunker in background steps - Add module docstring to evolve steps package - Update return type annotation to reflect new chunker class usage * refactor(components): rename embedding and llm components to as_embedding and as_llm - Rename reme4/components/embedding to reme4/components/as_embedding - Rename reme4/components/llm to reme4/components/as_llm - Update all imports and references from embedding to as_embedding - Update all imports and references from llm to as_llm - Change BaseEmbedding to BaseAsEmbedding and update inheritance - Change BaseLLM to BaseAsLLM and update inheritance - Update component types from LLM/EMBEDDING to AS_LLM/AS_EMBEDDING - Update configuration keys from embedding/llm to as_embedding/as_llm - Update all property references from llm to as_llm in step classes - Update test assertions to use new component enum values * refactor(embedding_store): rename embedding parameter to as_embedding - Updated configuration key from 'embedding' to 'as_embedding' - Renamed class attribute from 'embedding' to 'as_embedding' - Updated method calls to use 'as_embedding' instead of 'embedding' - Changed parameter name in constructor from 'embedding' to 'as_embedding' - Updated documentation to reflect new parameter name - Modified health check to use 'as_embedding' property * feat(agent_wrapper): add unified agent wrapper component with multiple backends - Introduce BaseAgentWrapper abstract base class for agent implementations - Add AsAgentWrapper implementation using AgentScope framework - Add CcAgentWrapper implementation using Claude Code SDK - Register agent_wrapper component type in ComponentEnum - Configure default agent_wrapper settings in default.yaml - Implement tool integration for both AgentScope and Claude Code backends - Support fluent configuration via set_system_prompt() and add_tools() methods * feat(agent-wrapper): add structured output support for agent wrappers - Import SystemMsg in AsAgentWrapper for structured output handling - Add output_schema parameter support in AsAgentWrapper with generate_structured_output - Implement set_output_schema method in BaseAgentWrapper for chaining configuration - Add output schema support in CcAgentWrapper with JSON schema format option - Return structured output when available in CcAgentWrapper response - Refactor kwargs handling to use default values consistently across wrapper classes	2026-06-05 17:27:54 +08:00
Sen Huang	a2d76cc034	refactor(auto_dream): improve recall workflow and documentation (#272) ... * feat(file_store): add concurrency protection to LocalFileStore.dump() * feat(file_catalog): replace file_store with file_catalog in DreamStep * feat: add ChannelSink for Claude Code channel notifications * feat(auto-memory): add transcript_path support and enhance metadata	2026-06-04 19:56:50 +08:00
jinliyl	cee2c3e338	Add JSONL support, reorganize vault structure, and update design docs (#274) ... * feat(config): add jsonl support and update LLM integration tests - Added jsonl extension to supported extensions in chunked backend - Refactored LLM integration tests to use async functions instead of nested runs - Created helper functions _run_basic_chat, _run_with_tool, _run_structured_output, _run_structured_output_enum - Implemented _run_all function to execute all test scenarios sequentially - Updated main execution block to use asyncio.run with consolidated test runner - Maintained all original test functionality while improving code structure * feat(config): add dialog directory configuration and reorganize vault structure - Add new dialog_dir field for dialog memory storage - Reorder directory initialization sequence in application setup - Simplify vault_dir description in configuration schema - Update thread_pool_max_workers description to be more concise - Move resource_dir definition earlier in the configuration schema - Remove redundant text from digest_dir description * docs(reme): update design documentation with layered memory architecture - Replace quick test section with comprehensive layered memory structure - Add detailed explanation of three-tier memory organization (resource, daily, digest) - Document Obsidian-compatible Markdown format with YAML front matter - Describe four types of wikilink syntax and semantic linking features - Explain AST-aware semantic chunking for document parsing - Detail self-evolving system with auto-resource, auto-memory, and auto-dream - Document directory structure and lifecycle characteristics - Add comprehensive table showing content nature, triggers, and examples - Include semantic link extraction and knowledge graph formation processes - Describe automated indexing and relationship building workflows * docs(reme): update design documentation with simplified structure and clearer explanations - Simplified directory structure overview with cleaner formatting - Updated memory layering explanation with more concise descriptions - Improved table layouts for better readability - Clarified auto-resource, auto-memory, and auto-dream processes - Streamlined indexing and search mechanism descriptions - Enhanced component system documentation with clearer backend options - Refined job list with more precise functional descriptions - Modernized layout diagrams and process flows - Consolidated repetitive content while maintaining comprehensive coverage * docs(reme): add application scenario documentation for financial industry use case - Document comprehensive example of ReMe usage in新能源 industry research - Detail the week-long process of automatic knowledge graph construction - Explain the auto-memory and auto-dream pipeline with concrete examples - Describe the extract and integrate phases for creating wiki nodes - Illustrate cross-file linking through relates_to and derived_from predicates - Show progressive graph growth from daily sessions to complete ecosystem - Demonstrate hybrid retrieval with vector and keyword search capabilities - Provide detailed directory structure and file organization patterns - Explain the three-phase workflow: ingestion, processing, and retrieval - Document the financial analyst persona and their information management needs * style(config): fix spacing in thread_pool_max_workers field definition - Corrected spacing around description parameter in Field definition - Simplified multi-line assertion to single line in LLM integration test	2026-06-04 16:04:02 +08:00