* feat(search): add tool context deduplication and improve search configuration
- Modify _make_tool methods to accept and inject tool_context_id parameter
- Add tool_context_id handling in AS and CC agent wrappers
- Increase search candidate multiplier from 3.0 to 5.0 in default config
- Extend HTTP client timeout from 30s to 3600s
- Add tool context deduplication logic to prevent duplicate search results
- Implement TTL-based expiration for seen chunks in tool contexts
- Add comprehensive unit tests for tool context deduplication behavior
- Update .gitignore to exclude longmemeval directory
- Add time import for timestamp functionality in search step
* refactor(search): replace time module with datetime for timestamp generation
- Removed unused time import
- Added static method _now_ts using datetime.timestamp
- Updated clock parameter to use _now_ts method instead of time.time
- Maintained same timestamp precision and functionality
* feat: add start_date/end_date time filter support for search job
- Add _extract_date_from_path to extract validated YYYY-MM-DD from chunk paths
- Add start_date/end_date filtering in _matches_search_filter
- Implement progressive recall in FaissLocalFileStore.vector_search
- Promote start_date/end_date from context to search_filter in SearchStep
- Add start_date/end_date parameters to search job in default.yaml
- Add unit tests for date filter functionality
* fix: validate/normalize date filters and harden _extract_date_from_path
Address three code-review comments on the time_filter search feature:
1. Validate/normalize start_date and end_date before string comparison.
_matches_search_filter does lexicographic comparison against path_date
(always canonical YYYY-MM-DD). Raw caller values like '2026-2-28' or
'abc' would produce silently wrong results. Now SearchStep normalizes
valid dates via extract_daily_date (with strptime fallback for
non-zero-padded input) and silently ignores invalid dates with a
logger.warning, removing them from the filter.
2. Clarify behavior for paths without embedded dates.
Added optional strict_date_filter parameter (default False). When True
and at least one date bound is active, chunks whose path yields no date
(e.g. digest/personal/topic.md) are excluded. When False (default),
the existing behavior is preserved — dateless paths pass through.
3. Harden _extract_date_from_path against non-standard suffixes.
Previously parts[1].split('.')[0] accepted '2026-05-18.anything' as a
valid date. Now only exact 'YYYY-MM-DD' (dir) and 'YYYY-MM-DD.md'
(day-index) forms are accepted.
* fix(auto-memory): preserve message timestamps
* fix(auto-memory): infer daily date from messages
* feat(file_io): add strict date parsing and improve daily date handling
- Add new parse_daily_date function for strict YYYY-MM-DD validation
- Replace extract_daily_date with parse_daily_date for explicit date validation
- Change _messages_day to use max date instead of min for historical imports
- Reorder imports to maintain consistent module ordering
- Move session message saving after date validation in auto_memory
- Add comprehensive tests for invalid date rejection before saving
- Add tests for strict YYYY-MM-DD date format validation
- Update test names to reflect latest date behavior
---------
Co-authored-by: Ziyang Guo <121015044+RunMarshal@users.noreply.github.com>
Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
* refactor(transfer): drop orphaned ingest step, make service discovery cross-platform
- Remove ingest step: superseded by auto_resource (drop files under
resource/ → watcher interprets them); its meta.json/<date>.md outputs
had no consumers and tripped the auto_resource watcher.
- Replace lsof/pgrep shell-outs in service_utils with psutil (per-process
enumeration, no root needed on macOS) for Windows/macOS/Linux support.
- Add cross-platform test coverage for _pid_on_port / _scan_reme_procs.
- Deps: +psutil, -filelock (only used by the removed ingest lock).
* chore(release): bump version to 0.4.0.5
* fix(mcp): resolve circular import issues and update dependencies
- Moved fastmcp imports inside functions to prevent circular dependencies
- Replaced _TRANSPORT_MAP with _VALID_TRANSPORTS set for transport validation
- Updated version number from 0.4.0.3 to 0.4.0.4
- Added claude-agent-sdk dependency to core optional dependencies
- Used TYPE_CHECKING imports for FastMCP related types
- Restructured transport mapping logic within function scope
- Fixed string annotation for CallToolResult type hints
* refactor(tests): update date handling in daily steps tests
- Replace _date.today() with timezone-aware now function
- Use Asia/Shanghai timezone for date formatting
- Change return format to use strftime instead of isoformat
- Import now function from reme.steps.evolve module
* refactor(tests): clean up unused imports in daily steps test
- Removed unused date import from datetime module
- Removed redundant pathlib Path import that was already imported later
- Kept necessary imports for asyncio, os, tempfile, warnings, and frontmatter modules
* test(daily_steps): update test to include application context for daily list step
- Add ApplicationContext initialization with temporary workspace directory
- Register file store component in application context
- Pass application context to DailyListStep constructor
- Maintain existing test assertion behavior for date metadata verification
* feat(file_io): add daily_write step for creating daily notes with conversation metadata
- Add DailyWriteStep class that delegates to write job for creating daily notes
- Register daily_write job in default configuration with proper parameters
- Include validation for name and session_id path components
- Add test coverage for daily_write functionality including metadata handling
- Preserve existing job execution method in application.py after repositioning
- Update base_step.py to use positional-only parameter syntax for job methods
- Import and expose DailyWriteStep in file_io module initialization
- Override reserved metadata keys (name, description, session_id, source_conversation) with fixed values
- Refresh daily index after successful write operation
- Generate proper source conversation links in markdown format
* feat(daily): refactor daily note system with enhanced metadata handling
- Introduce validate_filename_component function and export it
- Add _INDEX_HIDDEN_METADATA_KEYS to hide conversation metadata from index
- Update scan_notes to exclude hidden metadata keys from index rendering
- Modify auto_memory to use daily_write tool and manage session frontmatter
- Implement session note lookup and renaming based on frontmatter name
- Update daily_list to return flattened note metadata including session info
- Change daily_write to dispatch write step instead of running job
- Add test cases for updated daily note functionality and metadata handling
- Update version from 0.4.0.2 to 0.4.0.3
* fix(evolve): correct metadata update in auto memory response
- Fixed trailing comma issue in metadata dictionary update
- Ensured proper formatting of response metadata structure
- Maintained existing functionality while fixing syntax error
* refactor(auto_resource): replace daily_create with dynamic note management
- Remove DailyCreateStep and related exports from file_io module
- Replace static daily note creation with dynamic resource-linked card system
- Implement LLM-suggested naming with frontmatter-driven file management
- Add source_resource linking for tracking original files
- Introduce collision handling with hash-based suffixes
- Update documentation to reflect new resource card workflow
- Modify auto_resource prompts to use write/edit tools instead of daily_create
- Adjust test fixture comments to match new agent behavior
- Update framework diagrams and quick start examples accordingly
* feat(app): add version info to app initialization and update auto-memory logic
- Include version number in application startup logging
- Remove tool result truncation logic from auto-memory step
- Update auto-memory to exclude tool_result blocks from saved history
- Add test case to verify tool results are filtered out from message saving
- Update YAML prompts to clarify filename naming rules without dates
- Modify configuration to support new dispatch steps format with persistence control
* feat(auto_memory): add note modification tracking and optimize frontmatter updates
- Add _note_bytes and _note_modified methods to track actual file changes
- Optimize frontmatter updates by checking existing metadata before update
- Add modified flag to response metadata indicating actual note changes
- Update logging to include modified status in various operations
- Add comprehensive tests for modified/unmodified detection scenarios
- Enhance result hook logic to skip when no actual changes occur
- Refactor metadata handling to properly track creation vs modification status
* feat(evolve): enhance agent reply processing and logging capabilities
- Add agent_reply_result_text function to extract final user-visible text from agent replies
- Implement comprehensive logging throughout auto_memory, auto_resource, and dream modules
- Add max_units configuration option to limit extracted memory units
- Improve error handling and validation in auto_resource step
- Refactor dream extract step to respect max_units limit during processing
- Enhance summary rendering in dream finish step with detailed breakdown
- Add result hook functionality for embedding hosts integration
- Implement loose resource filename handling for root-level resources
- Update test cases to reflect new functionality and improved error messages
* test(background-steps): update fake upsert function to include created parameter
- Modified fake_upsert function to accept 'created' parameter instead of '_created'
- Added 'created' field to captured dictionary in fake_upsert function
- Included 'created': True in the expected response dictionary for test case
- Updated test assertion to match new parameter structure
- Implement _backfill_missing_embeddings method to handle chunks without embeddings
- Add logic to identify and process chunks that predate embedding feature
- Integrate backfill process into store loading sequence
- Add proper error handling and logging for backfill operations
- Create unit test for embedding backfill functionality
- Ensure backfilled embeddings are properly persisted to storage
* chore(logging): change info logs to debug level for data loading operations
- Changed stopwords loading log from info to debug level
- Changed file catalog nodes loading log from info to debug level
- Changed file graph nodes loading log from info to debug level
* feat(dream): add dream schema definitions and enum for auto-dream functionality
- Add DreamBucketEnum with procedure, personal, and wiki values
- Create comprehensive dream-related Pydantic models including DreamUnit,
DreamTopic, DreamExtractOutput, IntegrateOutcome, TopicSelectionOutput,
ProactiveResult, and DreamState
- Move schema definitions from local step module to shared schema package
- Update dream extraction and integration steps to use new enum-based
bucket validation
- Initialize digest directories for each dream bucket type
- Enhance embedding store health check with workspace directory logging
* refactor(tests): update DreamState import path in test_auto_dream.py
- Move DreamState import from reme.steps.evolve.dream.schema to reme.schema
- Maintain same functionality with updated module reference
- Align import with new schema location in project structure
* fix(core): update version number to 0.4.0.1
- Incremented version from 0.4.0.0 to 0.4.0.1 in __init__.py
* fix(index): remove stopwords path from tokenizer config and add keyword index repair
- Remove stopwords_path from tokenizer config to prevent index forking by install path
- Add _sync_keyword_index_from_chunks method to repair keyword index when persisted state mismatches
- Implement test for keyword index repair from persisted chunks when missing
- Add test to verify tokenizer fingerprint ignores stopwords absolute path
- Update version from 0.4.0.1 to 0.4.0.2
* feat(dream): add scan_days parameter to dream extraction process
- Add scan_days configuration option to default.yaml with default value of 2
- Implement recent_dates utility function to calculate date ranges for scanning
- Modify DreamExtractStep to scan multiple days based on scan_days parameter
- Update dream extraction to process files across multiple dates instead of single day
- Extend DreamState schema to include dates and scan_days fields
- Update DreamTopicsStep to handle multi-day topic processing
- Modify finish step to checkpoint files from all scanned dates
- Add comprehensive tests for multi-day scanning functionality
- Update prompt templates to include scan dates information
- Refactor topics writing logic to target specific date rather than current date
* docs: rename vault_dir to workspace_dir in documentation and examples
* refactor(extract): format long method call across multiple lines
* refactor(extract): format system prompt parameters for better readability
