Commit graph

5 commits

Author SHA1 Message Date
jinliyl
d8086039dc
refactor(Agentscope2.0): llm & embedding & agent (#271)
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
* refactor(embedding): replace embedding model with embedding store architecture

- Remove as_token_counter component and its estimated token counter implementation
- Replace BaseEmbeddingModel with BaseEmbedding that wraps AgentScope embedding models
- Add support for multiple embedding providers (OpenAI, DashScope, Gemini, Ollama)
- Introduce BaseEmbeddingStore and LocalEmbeddingStore for caching and persistence
- Update component registry to use new embedding and embedding_store types
- Modify file stores to use embedding_store instead of embedding_model
- Update health check to monitor embedding_store instead of embedding_model
- Change default config to use embedding_store with local backend
- Add estimate_token_count utility function to utils module

* refactor(llm): replace as_llm components with unified llm implementation

- Remove deprecated as_llm and as_llm_formatter modules
- Add new llm module with BaseLLM and provider-specific implementations
- Update component registry to use LLM instead of AS_LLM
- Replace all as_llm/as_llm_formatter references with llm in steps
- Update configuration schema to use llm instead of as_llm
- Rename integration test file from test_as_llm to test_llm
- Add proper docstrings to embedding store dimension property
- Add pylint disable comment for embedding model call
- Remove unused FormatterBase import in base_step
- Update token_utils with function docstring

* refactor(evolve): replace ReActAgent with Agent and update message handling

- Removed FlexReActAgent class and direct ReActAgent imports
- Updated Agent instantiation to use new constructor parameters
- Changed message content to use TextBlock format instead of plain strings
- Modified timestamp access from msg.timestamp to msg.created_at
- Updated metadata access pattern for structured outputs
- Replaced Msg.from_dict with Msg.model_validate in auto_memory.py
- Updated test mocks to patch Agent instead of ReActAgent
- Changed message serialization from to_dict to model_dump in tests
- Moved component references to base class definition
- Updated demo tools to return strings instead of ToolResponse objects

* feat(step): migrate to FunctionTool and add streaming support

- Replace deprecated ToolResponse with FunctionTool in base_step.py
- Remove unused TextBlock import from base_step.py
- Update job registration to use new FunctionTool API
- Add thinking_budget parameter to llm_demo configuration
- Introduce StreamLLMDemoStep with streaming output capability
- Add structured output support to LLMDemoStep via generate_structured_output
- Implement streaming event handling for text/thinking/tool calls
- Add integration tests for embedding functionality
- Add integration tests for structured output and streaming features
- Update tool usage in demo steps to use new function naming convention

* fix(ci): correct package installation path in unittest workflow

- Updated pip install command to use proper package path "./reme4[dev,core]"
- Fixed dependency installation step in CI workflow configuration

* chore(workflow): update python versions in unittest workflow

- Remove Python 3.10 from test matrix
- Add Python 3.11 to test matrix
- Add Python 3.12 to test matrix
- Keep Python 3.13 in test matrix
- Update matrix configuration for better version coverage

* fix(health): handle missing dimensions attribute in embedding status

- Wrap dimensions access in try-except to prevent AttributeError
- Return None when dimensions attribute is not available
- Maintain backward compatibility for components without dimensions

test(component): add comprehensive tests for BaseComponent and related classes

- Add tests for Dependency class including repr and attribute access
- Add tests for bind method with various scenarios and edge cases
- Add tests for lifecycle management and async context handling
- Add tests for standalone and context-bound dependency resolution
- Add tests for ComponentMixin path utilities

test(common): update LocalFileStore initialization parameter

- Change embedding_model parameter to embedding_store in test setup
- Update all affected test files consistently

test(registry): add complete test suite for ComponentRegistry

- Add tests for register method with explicit names and defaults
- Add tests for decorator registration pattern
- Add tests for get_all method returning copies
- Add tests for unregister and clear operations
- Add tests for error handling of invalid registrations

test(job): add comprehensive tests for BaseJob and BackgroundJob

- Add tests for step resolution and exception handling
- Add tests for backoff delay calculation with jitter
- Add tests for supervisor loop restart behavior
- Add tests for task shutdown and cancellation

test(prompt): add complete test suite for PromptHandler

- Add tests for prompt loading from dictionaries and files
- Add tests for internationalization and language fallback
- Add tests for flag filtering and variable substitution
- Add tests for format validation and error handling

test(runtime): add basic tests for RuntimeContext dictionary access

- Add tests for item getting, setting and containment checks
- Add tests for missing key error handling

* feat(evolve): add permission context and agent state management

- Import PermissionContext, PermissionMode and AgentState modules
- Add state configuration with bypass permission mode to AutoDream agents
- Add state configuration with bypass permission mode to AutoMemory agents
- Implement static _to_msg method for message validation and formatting
- Refactor message processing to use the new _to_msg method
- Ensure proper content structure for text blocks in message conversion

* style(tests): update test files with linting rules and code improvements

- Add missing pylint disable directives for docstring and attribute warnings
- Replace lambda expressions with proper function definitions in test cases
- Import Path directly instead of using lambda with __import__
- Simplify assertion checks by using truthiness instead of equality to empty dict
- Remove unused imports and reorder imports consistently
- Format dictionary literals with proper indentation and line breaks
2026-06-03 11:35:43 +08:00
jinliyl
a4efc0f776
refactor(reme4): restructure steps packages (#258)
* fix(bm25_index): 修正BM25索引计算中的文档长度归一化问题

修复了在计算BM25相似度时对文档长度进行不正确归一化的bug,确保所有查询都能得到准确的相关性评分。

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* refactor(steps): Rename and adjust indexing step logic

- Rename `scan_changes.py` and `reindex.py` to `clear_and_scan.py`
- Update implementation details of `ScanChangesStep` and `ClearAndScanStep`
- Modify the scheduling mechanism in `WatchChangesStep`
- Adjust step registration and parameter configuration in config files
- Update related tests to align with the new interface changes

* up

* feat(daily): replace daily CRUD operations with slug provisioning approach

* refactor(tests): migrate CRUD step tests from HTTP server to direct LocalFileStore

* up

* up

* up

* up

---------

Co-authored-by: huangsen <huangsen.huang@alibaba-inc.com>
2026-05-28 14:30:30 +08:00
Sen Huang
7d0bec60be
feat: rename working_dir to vault_dir and update documentation (#254)
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
Tests ReMe / Unit Tests - py3.10 (push) Has been cancelled
Tests ReMe / Unit Tests - py3.13 (push) Has been cancelled
* feat: rename working_dir to vault_dir and update documentation

- Rename working_dir to vault_dir across the application
- Update documentation to reflect vault_dir instead of working_dir
- Change FileFrontMatter title field to name field
- Update .gitignore to include vault directory
- Modify file path descriptions to reference vault instead of working_dir
- Update related configuration and property names accordingly

* refactor(steps): rename working_path to vault_path in CRUD operations

- Rename parameter from `working_path` to `vault_path` in `resolve_path` function
- Update all usages in append, edit, read, and write steps to use `self.vault_path`
- Update documentation comments to reflect the new parameter name
- Update docstring in read.py to mention `vault_dir` instead of `vault`

test(chunked_file_parser): update frontmatter field from title to name

- Change frontmatter field from `title` to `name` in test cases
- Update comment in background steps test to reference `vault_path` instead of `working_path`

* refactor(schema): remove unused ResourceEntry import

* feat(file_graph): add link scope filtering to get_inlinks/get_outlinks

* feat(file-store): add scope parameter to link methods
2026-05-22 18:10:29 +08:00
jinliyl
71e42dbad0
refactor(reme4): replace file_watcher component with background steps pipeline (#251)
* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* feat(config): add daily_dir configuration and background job logging

- Added daily_dir setting with default value 'memory' to config
- Implemented logging for background job startup events
- Enhanced component start logic to handle background backend type
- Updated default YAML configuration structure

* refactor(file_parser): replace _get_relative_path with to_vault_relative method

- Remove redundant working_dir property from base file parser
- Add to_vault_relative method to base component for path resolution
- Update bare_file_parser to use new to_vault_relative method
- Update default_file_parser to use new to_vault_relative method
- Update linked_file_parser to use new to_vault_relative method
- Make working_path absolute in base_component and steps
- Simplify index_changes step by removing redundant base variable
- Consolidate path relative logic in single shared method

* docs(reme4): update report with detailed architecture sections

- Add comprehensive Markdown kernel section covering Obsidian compatibility
- Include detailed explanation of YAML front matter and wikilink formats
- Document smart slicing mechanism using Markdown AST instead of fixed tokens
- Explain graph indexing with bidirectional links and multiple backends
- Restructure sections with proper numbering from 4 to 7
- Move Markdown kernel section to appear before self-evolution features
- Add detailed explanations of auto-memory, auto-dream, and auto-link processes
- Document three-way hybrid search with RRF fusion and progressive expansion
- Include engineering value explanations for keyword indexing in Chinese context
2026-05-22 10:26:36 +08:00
jinliyl
e411eeb4c0
dev/reme4 init merge (#236) 2026-05-17 14:14:43 +08:00