* refactor(config): streamline job descriptions and parameter docs
- Simplify descriptions for search, traverse, list, read, stat,
frontmatter:read, write, edit, append, frontmatter:update,
frontmatter:delete, move, delete, upload, upload_resource, and
download jobs
- Shorten parameter descriptions to be more concise
- Maintain essential information while reducing verbosity
refactor(steps): rename daily steps and consolidate functionality
- Rename daily_resolve_step to daily_read_step
- Rename daily_create_step to daily_write_step
- Update __init__.py imports to reflect new step names
- Consolidate daily operations documentation
refactor(daily): extract helper functions and improve structure
- Rename _day_index.py to _daily_io.py
- Extract validate_slug function for Windows-safe filename validation
- Move scan_notes function to public interface
- Add comprehensive docstrings explaining slug validation and day-index
rebuild concerns
feat(daily): decouple list operation from index refresh
- Remove automatic day index refresh from daily_list_step
- Change daily_list_step to pure read operation with no side effects
- Sort notes by slug for stable output
- Update documentation to clarify read/write separation
refactor(daily): remove deprecated create step
- Remove unused daily/create.py module
- Simplify daily operations to focus on CRUD patterns
* refactor(daily): replace module imports with explicit step class imports
* feat(config): add comprehensive job definitions for vault operations
- Add utility jobs like version, search, traverse, list, read, stat
- Include file operations like move, delete, upload, download
- Add daily workspace management jobs: daily_list, daily_resolve, daily_reindex
- Update descriptions to reflect vault-based operations instead of working_dir
- Add proper section headers and documentation for each job category
refactor(steps): reorganize step modules and remove demo steps
- Move steps into categorized packages: common, crud, frontmatter, daily, jobs
- Remove demo steps (DemoEchoStep1, DemoEchoStep2, StreamDemoStep1, StreamDemoStep2)
- Add new steps: InitStep for vault initialization, TraverseStep for graph traversal
- Update __init__.py to auto-import all step modules
- Organize imports by functionality (common, CRUD operations, frontmatter, daily)
feat(vault): implement vault-centric file operations and configuration
- Change default config to use vault_dir instead of working_dir
- Add environment variable support for embedding configuration
- Implement file watcher with lite backend for daily/digest directories
- Update search step to use 'name' instead of 'title' from frontmatter
- Create ResourceEntry schema for tracking uploaded assets
docs(steps): add comprehensive documentation for all step categories
- Document file-I/O split by blast radius (crud vs frontmatter packages)
- Add detailed descriptions for each step category and functionality
- Explain the purpose and usage patterns for different types of file operations
- Provide clear parameter documentation for all new job configurations
* fix(config): correct vault directory path and remove unused job configurations
- Fix vault_dir from 'vaultd' to 'vault' in default configuration
- Remove deprecated traverse and list job configurations
- Remove unused tag tooling configurations
- Remove background watch_file job configuration
refactor(steps): remove unused jobs module import
- Comment out jobs module import in steps/__init__.py
- This removes unused synchronizer and digester step registrations
refactor(tests): update import path and add pylint directive
- Update ResourceEntry import from reme4.schema to reme4.schema.resource_meta
- Add pylint disable directive for unused argument in test datetime mocks
* efactor(steps): remove unused modules from __all__
- Remove "background" module from __all__ list
- Remove "jobs" module from __all__ list
- These modules were no longer being used in the steps package
* feat(config): update vault directory structure and remove file watcher
- Change vault_dir reference from ./vault to ./vault in CLI example
- Add daily_dir, digest_dir, and resource_dir configuration options
- Remove file_watcher component configuration as it's no longer needed
- Update comment to reflect correct module name (reme4vault)
refactor(steps): add background step and remove deprecated init step
- Import and register background step module
- Remove deprecated InitStep from common steps
- Update __all__ export list to include background step
refactor(reindex): improve reindex step to scan vault directly
- Update docstring to reflect vault scanning instead of watcher sync
- Replace file watcher stop/start logic with direct vault path walking
- Add support for suffix filtering during reindex operation
- Use index_changes job to process found files
refactor(wikilink_utils): enhance inbound source lookup with link scope
- Import LinkScopeEnum for proper type handling
- Update get_inlinks call to use ALL scope for virtual targets
- Improve documentation for reverse-index lookup behavior
test(refactor): clean up test suite removing deprecated functionality
- Remove test_init_job and test_demo_job unit tests
- Update help job assertion to check for literal command format
- Change test directory from .reme to vault in CRUD tests
- Remove init and demo job calls from integration test
BREAKING CHANGE: Removes file_watcher component and init step
* style(steps): fix import formatting in __init__.py
Add proper spacing in the background module import statement
to maintain consistent code style and readability.
* refactor(config): change default vault directory from vault to .reme
Default dev config now points vault_dir at ./.reme so `python -m
reme4 start` can be run from the repo root and exercise the full
atomic-tool surface against the seeded test data.
BREAKING CHANGE: The default vault directory has been changed from
'vault' to '.reme' in the configuration.
* docs(reme4_report): fix markdown formatting and remove extra content
* refactor(file_parser): delegate wikilink extraction to WikilinkHandler
* fix(search): handle empty query case gracefully
- Replace assertion with conditional check for empty query
- Set response success to false when query is empty
- Return error message instead of throwing assertion error
- Maintain existing validation for other parameters
* feat: rename working_dir to vault_dir and update documentation
- Rename working_dir to vault_dir across the application
- Update documentation to reflect vault_dir instead of working_dir
- Change FileFrontMatter title field to name field
- Update .gitignore to include vault directory
- Modify file path descriptions to reference vault instead of working_dir
- Update related configuration and property names accordingly
* refactor(steps): rename working_path to vault_path in CRUD operations
- Rename parameter from `working_path` to `vault_path` in `resolve_path` function
- Update all usages in append, edit, read, and write steps to use `self.vault_path`
- Update documentation comments to reflect the new parameter name
- Update docstring in read.py to mention `vault_dir` instead of `vault`
test(chunked_file_parser): update frontmatter field from title to name
- Change frontmatter field from `title` to `name` in test cases
- Update comment in background steps test to reference `vault_path` instead of `working_path`
* refactor(schema): remove unused ResourceEntry import
* feat(file_graph): add link scope filtering to get_inlinks/get_outlinks
* feat(file-store): add scope parameter to link methods
* feat(file_store): add FAISS-backed local file store implementation
- Introduce FaissLocalFileStore class with vector search capabilities using FAISS IndexFlatIP
- Implement FAISS index persistence with binary format and JSON id-map sidecar
- Add automatic index rebuilding when sidecar files are missing or corrupted
- Support tombstone mechanism for efficient deletion and compaction
- Register 'faiss' component type in the registry system
- Add faiss-cpu dependency requirement to pyproject.toml
- Update configuration schema to use simplified parameter structure
- Enhance search step to support parameter override from runtime context
- Add comprehensive unit tests for FAISS store functionality
- Implement fallback to parent methods for basic CRUD operations
* refactor(search): simplify parameter retrieval logic
- Removed _param method that checked context and kwargs
- Directly use self.kwargs.get for all parameter retrievals
- Maintained same default values for vector_weight, candidate_multiplier, expand_links, and max_links_per_direction
- Reduced code complexity by eliminating redundant context checking logic
- Add comprehensive Markdown kernel section covering Obsidian compatibility
- Include detailed explanation of YAML front matter and wikilink formats
- Document smart slicing mechanism using Markdown AST instead of fixed tokens
- Explain graph indexing with bidirectional links and multiple backends
- Restructure sections with proper numbering from 4 to 7
- Move Markdown kernel section to appear before self-evolution features
- Add detailed explanations of auto-memory, auto-dream, and auto-link processes
- Document three-way hybrid search with RRF fusion and progressive expansion
- Include engineering value explanations for keyword indexing in Chinese context
* feat(seekdb): add Seekdb file and vector stores with pyseekdb>=1.2.0
* refactor(seekdb): add pyseekdb_conn and remote-only host/port config
* refactor(embedding): remove env fallbacks from BaseEmbeddingModel; pass credentials in tests
* refactor(seekdb): drop tenant from client kwargs; default database test and empty password
* fix(deps): gate pyseekdb to Python >=3.11 for CI 3.10 compatibility
* fix(seekdb): satisfy pre-commit pylint and formatting for seekdb stores
* up
* up
* up
* up
* up
* up
* up
* up
* up
* up
* up
* feat(config): add daily_dir configuration and background job logging
- Added daily_dir setting with default value 'memory' to config
- Implemented logging for background job startup events
- Enhanced component start logic to handle background backend type
- Updated default YAML configuration structure
* refactor(file_parser): replace _get_relative_path with to_vault_relative method
- Remove redundant working_dir property from base file parser
- Add to_vault_relative method to base component for path resolution
- Update bare_file_parser to use new to_vault_relative method
- Update default_file_parser to use new to_vault_relative method
- Update linked_file_parser to use new to_vault_relative method
- Make working_path absolute in base_component and steps
- Simplify index_changes step by removing redundant base variable
- Consolidate path relative logic in single shared method
* docs(reme4): update report with detailed architecture sections
- Add comprehensive Markdown kernel section covering Obsidian compatibility
- Include detailed explanation of YAML front matter and wikilink formats
- Document smart slicing mechanism using Markdown AST instead of fixed tokens
- Explain graph indexing with bidirectional links and multiple backends
- Restructure sections with proper numbering from 4 to 7
- Move Markdown kernel section to appear before self-evolution features
- Add detailed explanations of auto-memory, auto-dream, and auto-link processes
- Document three-way hybrid search with RRF fusion and progressive expansion
- Include engineering value explanations for keyword indexing in Chinese context
Update MCPClient to accept action parameter during method calls
instead of requiring it during instantiation, making it consistent
with the new client interface.
fix(reme): update client usage patterns
Update all client usages to pass action parameter during method
calls instead of during client instantiation.
* up
* up
* up
* up
* up
* up
* up
* up
* up
* feat: implementation of the read step for reme (markdown) (#245)
* feat: implementation of the read step for reme (markdown)
* fix(step): markdown read step fixing pr comments
* fix(step): more fixes for pr comments
* further fix for better review adaptation
* accept absolute path
* fixing base job exception
* fix(core): correct tool results directory naming (#246)
- Changed directory name from 'tool_result' to 'tool_results' in documentation
- Updated path variable assignment to use correct plural form 'tool_results'
- Ensured consistent directory naming throughout initialization logic
* fix: recent unittest inconsistency (#248)
* feat: implementation of the read step for reme (markdown)
* fix(step): markdown read step fixing pr comments
* fix(step): more fixes for pr comments
* further fix for better review adaptation
* accept absolute path
* fixing base job exception
* fix: fix test inconsistency
* up
---------
Co-authored-by: imrewce <wce@pku.edu.cn>
- Changed directory name from 'tool_result' to 'tool_results' in documentation
- Updated path variable assignment to use correct plural form 'tool_results'
- Ensured consistent directory naming throughout initialization logic
* refactor(steps): Add job management methods and support registering them as tools
Added methods to the `BaseStep` class for retrieving, running, and registering jobs as tools, enhancing the functionality of the step class.
* fix doc
* chore(pyproject.toml): Update dependency versions and adjust package configuration
Bump agentscope version to 1.0.19 and reorganize the core dependency configuration structure.
* fix(reme_light): dedupe default watch paths on case-insensitive filesystems
On Windows NTFS and macOS HFS+, ``MEMORY.md`` and ``memory.md`` resolve to
the same physical file. ``ReMeLight.__init__`` hardcoded both spellings in
the default ``watch_paths`` list, so the memory markdown file was indexed
twice on those filesystems, wasting embedding calls and producing duplicate
search hits.
Dedupe the default candidate list using ``os.path.normcase`` as the
comparison key. On case-sensitive filesystems normcase is the identity
function, so both spellings continue to be watched there. The original
path strings are preserved, the caller-supplied ``watch_paths`` path is
untouched, and only the built-in fallback is affected.
Fixes#228
* refactor(reme_light): simplify watch path dedup via existence check
Replace the os.path.normcase-based dedup loop with a direct exists()
check that picks one of MEMORY.md / memory.md. On case-insensitive
filesystems both spellings resolve to the same file so exists() returns
true for both, naturally avoiding a duplicate watch — including on
macOS where os.path.normcase is the identity function and the previous
approach silently did nothing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(steps): Add job management methods and support registering them as tools
Added methods to the `BaseStep` class for retrieving, running, and registering jobs as tools, enhancing the functionality of the step class.
* fix doc
* feat: add Neo4j file graph support and markdown parser with wikilink extraction
- Add Neo4jFileGraph implementation for property-graph storage with
virtual/real node handling and link management
- Introduce LinkedFileParser for markdown files with frontmatter,
wikilink graph extraction, and full-skeleton chunking
- Update pyproject.toml to include pyyaml, mistletoe, and neo4j
dependencies
- Modify .gitignore to exclude /vault and structure.md
- Change reme CLI entry point from reme_ai.main to remecli.reme
- Register new neo4j and md components in respective registries
* refactor(file-graph): add chunk_ids support to Neo4jFileGraph
Add chunk_ids field to File node properties in Neo4jFileGraph to
enable better content chunk tracking and management.
BREAKING CHANGE: File node schema now includes chunk_ids property
which may affect existing integrations.
feat(parser): implement wikilink resolution logic
Move path resolution logic from utils/path_resolver to
linked_file_parser module and enhance wikilink resolution with
folder-note rule support and improved error handling.
fix(tests): update test assertions and variable names
Update test cases to reflect changes in data structures and
variable naming conventions across various components.
chore(config): update package entry point reference
Change reme CLI entry point from remecli.reme:main to
reme_ai.reme:main in pyproject.toml.
refactor(utils): remove deprecated path_resolver module
Remove the old path_resolver utility module as its functionality
has been moved to linked_file_parser.
docs(file-graph): update Neo4jFileGraph documentation
Update class docstrings and comments to reflect new chunk_ids
property and other structural changes.
style(formatting): adjust code formatting and line breaks
Minor formatting improvements including line length optimization
and consistent spacing adjustments throughout the codebase.
* fix(pyproject.toml): correct entry point for reme command
Change the entry point from "reme_ai.reme:main" to "reme_ai.main:main"
to fix the module reference for the reme command in project scripts.
* docs(cli): add comprehensive CLI commands documentation
- Document CLI entry point and argument parsing mechanism
- Add detailed command reference with parameters and behaviors
- Include usage examples for common operations like start, search, and reindex
- Describe backend options and service configuration overrides
- Explain local vs server-side command execution patterns
- Provide table format documentation for all available actions
* docs(reme_design): update CLI command documentation with detailed action descriptions
- Rename section from "CLI 指令" to "基础Job" and add author attribution
- Add comprehensive table documenting all available actions with parameters and behaviors
- Include detailed explanations for input/output parameters, defaults, and internal workflows
- Update example usage commands with proper parameter passing syntax
- Add metadata information for each action including health checks and component details
- Clarify the difference between local actions and server-forwarded actions
- Document the new list action that intercepts at client side without forwarding to server
* feat(vector_store): add OceanBase as a VectorStore
* refactor(obvec): make it cleaner
* docs: add obvec related info
* refactor: minor update
* refactor: clean code and pass lint
* docs: remove unrelated edit
* docs: minor update
* fix(core): handle chromadb import error gracefully
- Changed CHROMADB_AVAILABLE flag to _CHROMADB_IMPORT_ERROR exception storage
- Updated version from 0.3.1.7 to 0.3.1.8
- Modified import error handling to preserve original exception details
- Removed hardcoded ImportError message in favor of dynamic exception raising
- Added proper logger initialization using get_logger utility
* refactor(file_store): move sqlite3 imports inside initialization methods
- Moved sqlite3 import from module level to inside init methods
- Removed unused import statement at top of file
- Maintains same functionality while improving import organization
- Prevents potential issues with early sqlite3 dependency loading
* refactor(core): update import error handling with broader exception types
- Changed ImportError to Exception for ray import error handling
- Updated chromadb import error to use Exception instead of ImportError
- Modified elasticsearch import error to catch general exceptions
- Changed asyncpg import error handling from ImportError to Exception
- Updated qdrant import error to use Exception instead of ImportError
- Added explicit type hints for all import error variables as Exception | None
- Changed CHROMADB_AVAILABLE flag to _CHROMADB_IMPORT_ERROR exception storage
- Updated version from 0.3.1.7 to 0.3.1.8
- Modified import error handling to preserve original exception details
- Removed hardcoded ImportError message in favor of dynamic exception raising
- Added proper logger initialization using get_logger utility
* feat(compactor): add extra instruction support and improve error handling
- Add extra_instruction parameter to compactor for custom guidance during message compaction
- Implement try-catch blocks around AS LLM initialization with detailed error logging
- Add extra_instruction parameter to ReMe.compact method with comprehensive documentation
- Update agentscope dependency from 1.0.17 to 1.0.18 in light installation
- Bump version number from 0.3.1.6 to 0.3.1.7
- Pass extra_instruction parameter through compactor instantiation and execution flow
* fix(core): add error handling for AS LLM formatters and token counters initialization
- Wrapped AS LLM formatters initialization in try-except blocks
- Added specific error logging for failed AS LLM formatter initialization
- Wrapped AS token counters initialization in try-except blocks
- Added specific error logging for failed AS token counter initialization
- Applied same error handling pattern to both initial setup and restart operations
- Maintained existing warning logs for unsupported backends
* docs(context): add comprehensive context management design documentation
- Create detailed Chinese documentation for CoPaw context management V2
- Document memory layer and file system cache architecture
- Explain Pre-Reasoning Hook workflow with four-step process
- Detail two-stage truncation strategy for tool results
- Add examples for Browser Use and ReadFile tools
- Include Mermaid diagrams for visual flow representation
- Update README with link to new context design document
- Fix minor formatting issues in existing documentation
- Add protection thresholds for Markdown files in truncation
- Document long-term memory trigger mechanisms
* docs(README): add latest articles section and CoPaw context management design doc
- Added "Latest Articles" section to README with table format
- Included link to CoPaw Context Management Design document
- Created comprehensive documentation for CoPaw context management V2
- Documented in-memory and file system layer architecture
- Explained pre-reasoning hook and context compaction process
- Detailed two-phase truncation strategy for tool results
- Described special handling for readFile tool and markdown files
- Added long-term memory trigger logic overview
- Included mermaid diagrams for visualizing context flow
- Add context data structure diagram showing compact_summary and file system cache
- Update ToolResultCompactor section with detailed truncation strategies for recent vs old messages
- Add parameter tables for tool result compaction with recent_max_bytes and old_max_bytes settings
- Update execution flow steps with detailed descriptions of each memory operation
- Add key parameters table including tool_result_compact_keep_n and memory_compact_reserve
- Include thinking enhancement feature description for summary generation quality improvement
- Update both English and Chinese README documentation consistently
* refactor(file_store): simplify ChromaDB client initialization and improve file truncation logic
- Remove shutil import and _create_chroma_client method from chroma_file_store.py
- Directly initialize ChromaDB PersistentClient in start method without retry logic
- Reduce DEFAULT_MAX_BYTES from 100KB to 50KB in file_utils.py
- Update truncation notice format to provide clearer continuation instructions
- Add _truncate_fresh and _retruncate functions for better text truncation handling
- Replace inline truncation logic with dedicated function calls in file_utils.py
- Rename skills_tool_ids to md_file_tool_ids in tool_result_compactor.py
- Update file detection logic to identify any .md files instead of only skill.md
- Create comprehensive unit tests for truncation functionality in test_truncate_text_output.py
* chore(version): bump version to 0.3.1.6
- Update __version__ from 0.3.1.5 to 0.3.1.6 in __init__.py
* fix(memory): correct line numbering and improve tool result truncation
- Changed default start_line from 0 to 1 in truncate_text_output function
- Refactored _truncate method to be a standalone method in ToolResultCompactor
- Improved tool result compaction logic to handle text blocks more efficiently
- Added detection of skill-related tool calls for special handling
- Implemented conditional byte limits based on tool type for better memory management
- Updated version number from 0.3.1.4 to 0.3.1.5
* feat(file_utils): add encoding parameter to truncate_text_output function
- Added encoding parameter with default value "utf-8" to truncate_text_output function
- Updated all encode/decode calls to use the specified encoding parameter
- Modified ToolResultCompactor to pass encoding parameter when calling truncate_text_output
- Added error handling for skill tool ID detection in message processing loop
- Fixed potential AttributeError when accessing raw_input field that might be None
* fix(file-store): handle corrupted ChromaDB initialization and improve tool result truncation
- Add shutil import for directory removal operations
- Extract ChromaDB client creation into separate _create_chroma_client method
- Implement retry mechanism with database wipe on ChromaDB initialization failure
- Add proper exception handling in tool result compaction to prevent truncation errors
- Move file writing logic outside of exception handling scope for better error management
- Add warning log when truncation fails and return original content as fallback
