mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-09-05 08:06:15 +00:00
22 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jinliyl
|
2e87b7a52e
|
feat(core): add shell execution and runtime memory status (#344)
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
Windows Smoke / CLI smoke - py3.11 (push) Waiting to run
* feat(core): add shell command execution and memory status reporting - Introduce ShellStep for executing shell commands with timeout support - Add StatusStep to report memory estimates for stateful data components - Register shell and status commands in default configuration - Update documentation with new reme status and shell command capabilities - Implement comprehensive unit tests for both new step types - Add support for asynchronous command execution with proper error handling * feat(config): add log_config option to suppress config loading logs - Add log_config parameter to resolve_app_config function with default True - Conditionally log config loading messages based on log_config flag - Update reme.py and service_utils.py to use log_config=False for client calls - Suppress config logging in user-facing contexts to avoid output pollution refactor(shell): rename command parameter to cmd for clarity - Change 'command' to 'cmd' in default.yaml configuration schema - Rename 'timeout' to 'shell_timeout' to avoid parameter name collisions - Update ShellStep to accept both legacy and new parameter names - Maintain backward compatibility with existing command/timeout usage test(shell): add comprehensive tests for shell step parameter handling - Add test cases for new cmd and shell_timeout parameter names - Verify legacy command and timeout parameters still work - Test blank command rejection message updated to use cmd - Create integration test for shell parameter payload passing * fix(shell): ensure proper environment loading and process timeout handling - Move load_env() call to execute before parse_args() in main function - Add proper process group killing for timeout scenarios on POSIX systems - Implement recursive child process termination on Windows for proper cleanup - Change parameter name from 'timeout' to 'shell_timeout' in shell execution - Remove support for legacy 'command' and 'timeout' parameter names - Update test cases to verify new timeout behavior and parameter requirements - Add comments explaining component size tracking implementation details |
||
|
jinliyl
|
bf7ca17705
|
feat(benchmark): add LongMemEval golden answer validation (#335)
* feat(benchmark): add golden answer validation and session review for LongMemEval - Introduce GoldenCheckStep to validate LongMemEval golden answers using structured verdicts - Add SessionReviewStep to extract query/answer-relevant evidence from all sessions - Implement concurrent session processing with configurable concurrency limits - Create check_golden job configuration with lme_review and lme_judge agent wrappers - Add Qwen3.7-plus model configuration for enhanced processing capabilities - Include python_execute tool integration for agent-based reasoning and date validation - Generate comprehensive JSON output with session summaries and validation verdicts - Add run_check_golden.py script for batch processing across all LongMemEval samples - Configure proper logging initialization with console and file output options - Update component registry and file I/O modules to support new benchmark features * feat(scripts): add script to summarize LongMemEval check_golden verdicts - Parse check_golden.json files across all LongMemEval samples - Calculate accuracy metrics for golden answers and session IDs - Provide breakdown by question type with percentage calculations - Add command line options for listing bad samples and JSON output - Include progress tracking showing completed vs pending samples - Display confidence scores and date sanity checks statistics * refactor(benchmark): move golden check scripts to longmemeval directory - Moved run_check_golden.py from scripts/ to benchmark/longmemeval/ - Moved stats_check_golden.py from scripts/ to benchmark/longmemeval/ - Updated path resolution to use parents[2] instead of parent.parent - Added new --list-run-failed option to stats script - Added logging directory constant and functions for tracking launched samples - Enhanced stats output with launched count and run failure information - Improved error reporting with run failure details and log file paths * feat(benchmark): add LongMemEval agentic answer workflow with session extraction - Add LmeAgenticAnswerStep, LmeAutoMemoryStep, and LmeExtractSessionStep to __init__.py - Create shared helper render_with_source for displaying search results with session_id - Implement agentic_answer step with vector_search, bm25_search, and extract_session_by_id tools - Add auto_memory step to convert each session into search-friendly daily notes - Create extract_session step to retrieve and analyze raw session content by session_id - Update jinli_lme.yaml with auto_memory, vector_search, bm25_search, and agentic_answer jobs - Configure lme_memory, lme_extract, and lme_agentic_answer agent wrappers - Enhance search steps with include_source option to show session_id metadata - Add proper session_id tracking and collision handling in daily note generation * feat(benchmark): add LongMemEval agentic answer evaluation pipeline - Add session_id tracking to agentic_answer.py result metadata - Introduce run_agentic_answer.py driver for complete pipeline execution - Implement auto_memory, update_index, and agentic_answer job orchestration - Add concurrent execution with configurable limits and staggering - Create aggregation script for collecting tool-call trails and results - Add stats_agentic_answer.py for comprehensive result analysis - Implement resume capability with existing output detection - Generate aggregate.json with per-sample breakdown and tool call summaries * feat(steps): add ClearPathsStep for cleaning workspace outputs before rebuild - Introduce ClearPathsStep to remove stale workspace files/directories - Add support for specifying paths and config_keys as targets to clear - Implement safety checks to prevent deletion of files outside workspace - Add logging for cleared paths and warnings for invalid paths - Configure clear_paths_step in jinli_lme.yaml to clean daily_dir - Add clear_paths_step to clean mem_answer.json before rebuilds * feat(benchmark): add resume functionality to agentic answer runner - Replace --force flag with --resume flag for controlling job execution - By default every job reruns with clean rebuild behavior using config clear steps - Add --resume option to skip samples whose output already exists and continue interrupted batches - Update documentation to reflect new default clean rebuild behavior - Modify job skipping logic to honor resume flag instead of force flag - Update dry-run output to show correct todo jobs based on resume status - Change default example command to use --resume for continuing interrupted runs * feat(benchmark): generate JSONL output for check golden records - Add write_check_golden_list function to create JSONL file - Write all readable check_golden records as JSONL format - Include check_golden_list path in stats output - Display generated JSONL file path in summary report - Maintain UTF-8 encoding with non-ASCII character support * refactor(benchmark): rename answer judge step and integrate LME LLM judge - Rename AnswerJudgeStep to LmeLlmJudgeStep and update imports - Add new llm_judge configuration in jinli_lme.yaml - Update run_agentic_answer.py to include llm_judge in pipeline - Modify LmeLlmJudgeStep to read from query.json and answer.json - Write LLM judgement results back to mem_answer.json - Add command line options for start/end sample range selection - Update aggregate.json generation to include LLM judgement data - Add resume capability for llm_judge job based on judgement presence * refactor(benchmark): rename answer judge step and integrate LME LLM judge - Rename AnswerJudgeStep to LmeLlmJudgeStep and update imports - Add new llm_judge configuration in jinli_lme.yaml - Update run_agentic_answer.py to include llm_judge in pipeline - Modify LmeLlmJudgeStep to read from query.json and answer.json - Write LLM judgement results back to mem_answer.json - Add command line options for start/end sample range selection - Update aggregate.json generation to include LLM judgement data - Add resume capability for llm_judge job based on judgement presence * feat(steps): add wait_for_paths_step to block until workspace files exist - Introduce WaitForPathsStep class that polls for required workspace-relative paths - Add step registration with 'wait_for_paths_step' backend identifier - Implement path validation to ensure targets are within workspace boundaries - Add polling mechanism with configurable intervals via poll_seconds parameter - Include logging functionality with log_every_seconds parameter for status updates - Add metadata tracking of waited paths and duration in response object - Register step in index module and expose in public API - Configure step in jinli_lme.yaml to wait for session_review.json before golden check - Add script rename from run_check_golden.py to run_golden_check.py with enhanced options * feat(benchmark): enhance longmemeval benchmarking with concurrency and progress tracking - Add benchmark extra dependency group with portalocker requirement - Introduce concurrent execution support for golden_check and session_review workflows - Add progress reporting interval option with real-time status updates - Implement global throttling mechanism for session review requests using file locks - Enhance golden check validation with current schema verification - Add active task tracking and graceful shutdown handling - Rename check_golden scripts to golden_check for consistency - Update statistics reporting with correct/incorrect terminology instead of reasonable - Add stale format detection and compatibility handling for verdict fields - Include both_correct rate calculation in accuracy metrics - Add concurrency and staggering options for better resource management * ci(workflow): add Windows smoke test workflow - Create new workflow file .github/workflows/windows-smoke.yml - Configure workflow to trigger on push and pull request events - Set up Python environment with version 3.11 - Install package dependencies using pip - Run version job as smoke test for CLI functionality - Enable concurrency control to prevent duplicate runs - Use matrix strategy for Python version testing * feat(benchmark): add retry mechanism and health check for session review - Added retry configuration options (retry_initial_seconds, retry_max_seconds, retry_max_attempts) to jinli_lme.yaml - Implemented exponential backoff retry logic with configurable parameters in session_review step - Added output_is_healthy function to verify session_review.json integrity and absence of failed reviews - Updated resume functionality to skip only healthy outputs instead of all existing files - Integrated JSON parsing and validation to check for failed reviews in output files - Enhanced error handling and logging for retry attempts and recovery scenarios * feat(benchmark): add LongMemEval session review statistics script - Create stats_session_review.py to summarize session_review.json artifacts - Add command line options for listing failed, missing, and run failed samples - Implement JSON output mode for programmatic consumption - Calculate and display health statistics including total samples, healthy outputs, failed sessions - Provide detailed failure information with session IDs and error messages - Generate re-run commands for samples with failed reviews - Add percentage calculations for better statistical overview - Include support for multiple output formats and detailed logging * feat(benchmark): add LongMemEval output cleanup script and enhance golden check retry logic - Added clean_sample_outputs.py script to remove generated LongMemEval files while preserving source inputs - Implemented configurable retry mechanism in golden_check.py with exponential backoff strategy - Added retry parameters (initial/max seconds and max attempts) to control failure recovery behavior - Integrated asyncio support for asynchronous sleep during retry intervals - Configured default retry settings in jinli_lme.yaml with 5s initial and 300s maximum intervals - Preserved core files (query.json, answer.json, session/) while cleaning generated artifacts * feat(benchmark): add AppleDouble file cleanup to sample output cleaner - Remove AppleDouble files starting with '._' recursively including under session/ - Add is_under helper function to check if path is inside parent directory - Track targets in set to avoid duplicate processing - Include AppleDouble files in cleanup targets when not already covered by existing targets - Maintain dry-run mode as default behavior with --apply flag for actual deletion * refactor(benchmark): update LongMemEval sample output cleaning script - Add time and Iterator imports for enhanced functionality - Add --progress-every argument to control progress reporting frequency - Replace is_under function with iter_sample_targets generator - Implement detailed progress tracking with timing measurements - Add sample-by-sample processing with elapsed time reporting - Include AppleDouble file detection within session directory - Update target counting and deletion statistics display - Add conditional progress updates based on progress-every setting - Improve dry-run mode with would-delete indication * chore(benchmark): increase initial interval for session review step - Changed START_INTERVAL_SECONDS from 1.0 to 3.0 seconds - Adjusted timing parameters for better benchmark stability * refactor(benchmark): implement coordinated retry mechanism for session reviews - Add retry gate condition to coordinate concurrent review attempts - Implement wait_for_healthy_start_slot to handle sequential retries - Create mark_retrying and mark_recovered functions to track retry states - Update reply_with_retry to accept index parameter for coordination - Add has_prior_retry logic to prevent race conditions during recovery - Ensure proper cleanup of retry state on success or failure - Maintain backward compatibility while adding coordination features * chore(benchmark): adjust session review start interval timeout - Changed START_INTERVAL_SECONDS from 3.0 to 5.0 seconds - Increased initial delay for session review benchmark step - Updated timeout configuration for improved stability * refactor(benchmark): update session review concurrency and throttling mechanism - Replace global throttle with per-process concurrency control - Add concurrency parameter with default value of 30 in config - Add start_interval_seconds parameter with default value of 2 seconds - Change default concurrency from 3 to 1 in command line interface - Update documentation to reflect new throttling behavior - Implement semaphore-based concurrency limiting for review tasks - Modify retry mechanism to use local locking instead of global files - Remove portalocker dependency for cross-process throttling * refactor(config): update session review configuration and concurrency settings - Removed deprecated retry configuration parameters from jinli_lme.yaml - Increased MAX_CONCURRENCY from 30 to 60 in session_review.py - Reduced START_INTERVAL_SECONDS from 2.0 to 1.0 in session_review.py - Cleaned up redundant backend specifications in configuration file - Simplified agent wrapper configurations by removing obsolete retry settings * feat(benchmark): enhance LME auto memory step with advanced scheduling and error handling - Add datetime parsing functionality for LongMemEval timestamps with regex pattern - Implement configurable concurrency limits with MAX_CONCURRENCY of 60 - Introduce retry mechanism with exponential backoff for agent interactions - Add session filtering based on date comparison with question_date validation - Create rate limiting with start interval control between requests - Implement sophisticated retry coordination using asyncio conditions - Add comprehensive error tracking for failed and filtered session extracts - Remove deprecated concurrency parameter from jinli_lme.yaml configuration - Add structured output validation in session review step - Include detailed metadata reporting with session statistics and errors * fix(benchmark): adjust default concurrency for auto_memory job - Changed default concurrency from 3 to 1 for auto_memory job to prevent API overload - Updated help text to reflect new default value of 1 for concurrency parameter - Modified documentation to clarify concurrency behavior varies by job type * refactor(search): replace hardcoded candidate multiplier with constant - Introduced _CANDIDATE_MULTIPLIER constant set to 10 - Replaced hardcoded factor of 5 with _CANDIDATE_MULTIPLIER in BM25 search - Replaced hardcoded factor of 5 with _CANDIDATE_MULTIPLIER in vector search - Updated test to verify both search steps use ten times limit for candidates - Imported VectorSearchStep and Bm25SearchStep in test module - Added comprehensive test case for candidate count calculation logic * feat(lme): add data inspection error handling with fallback mechanism - Implemented non-retryable data inspection error markers detection - Added _is_data_inspection_error method to identify inspection failures - Created fallback handling for data inspection errors in auto memory extraction - Added fallback handling for data inspection errors in session review - Extended failed extracts tracking with non-retryable and fallback flags - Separated fallback extracts from regular failed extracts in reporting - Enhanced error logging with specific data inspection failure messages - Updated metrics to track fallback extractions and reviews separately - Maintained existing retry logic for other exception types * feat(benchmark): enhance session review statistics with fallback tracking - Add support for identifying and listing non-retryable fallback reviews - Introduce --list-fallback argument to display fallback review details - Separate retryable failures from non-retryable fallbacks in reporting - Track fallback samples and sessions separately from failed ones - Update console output to show both retryable and non-retryable categories - Include fallback details in JSON output with reasons and session info - Modify failure counting logic to distinguish between retryable and fallback reviews * feat(benchmark): add question_id tracking and enhanced fallback reporting - Add question_id function to extract query.question_id from data - Initialize question_id_by_id dictionary to store question IDs by index - Store question_id for each sample during data processing - Enhance fallback output to include question IDs and session information - Format sample labels with question IDs when available - Display session IDs associated with each fallback case * feat(benchmark): add question_id support and improve bad sample reporting - Add question_id_for function to extract question_id from multiple sources - Add sample_label function to format samples as idx(question_id) when available - Store question_id in data dictionary during processing - Change bad_golden and bad_sessions to store full records instead of just indices - Update list_bad output to show formatted labels with question_id information - Improve error reporting with more detailed sample identification * feat(benchmark): enhance golden check stats with structured output - Add related_session_ids function to extract session IDs from verdict records - Create grouped_records function to group records by question type - Replace flat list output with JSON-formatted grouped records in list_bad option - Replace flat list output with JSON-formatted grouped records in list_bad_sessions option - Maintain Chinese labels while adding structured data presentation - Improve readability of bad verdict record display with hierarchical grouping * feat(benchmark): update data structure for question indexing - Replace sample_label with _idx field for index tracking - Add question_id field to store _question_id values - Maintain backward compatibility with empty string defaults - Preserve existing session_id functionality - Update data mapping to include new fields in grouped results * refactor(benchmark): streamline golden answer verification process - Replace relevance filtering with comprehensive information extraction - Remove is_relevant field and simplify session summary structure - Change relevant_info to extracted_info for clarity - Update golden check logic to work with full extractions instead of filtered summaries - Simplify prompt instructions to focus on complete information extraction - Remove redundant schema validation and structured output requirements - Adjust statistics calculation to match new extraction approach - Update metadata field names to reflect extraction rather than relevance checking * feat(benchmark): add selective file deletion option to clean_sample_outputs - Add --filename argument to delete only specific root-level files - Modify iter_sample_targets function to accept optional filenames filter - Implement validation for root-level filename constraints - Update function calls to pass filenames parameter - Add example usage for selective file deletion in documentation * feat(benchmark): add error count metrics to golden check statistics - Added golden_bad, session_bad, and both_bad calculation fields - Updated console output format to include error counts per question type - Modified table display to show both accuracy rates and error numbers - Enhanced statistical summary with additional error breakdown metrics * test(search): update search step tests with include_source parameter - Added include_source=False parameter to VectorSearchStep initialization - Added include_source=False parameter to Bm25SearchStep initialization - Maintained existing RuntimeContext parameters for both search steps - Updated test calls to match new constructor signature with include_source option |
||
|
jinliyl
|
b1c9bf67bf
|
fix(embedding): make input truncation CJK-aware (#337)
* fix(embedding): make input truncation CJK-aware * test(embedding): cover CJK-aware truncation budget |
||
|
Ziyang Guo
|
e41b1673ad
|
fix(search): honor min_score in plain search steps (#338) | ||
|
Ziyang Guo
|
c5eefe4da3
|
fix(search): expose markdown frontmatter on chunks (#314)
* fix(search): expose markdown frontmatter on chunks * style(search): apply pre-commit formatting * fix(search): make frontmatter chunk metadata opt-in * fixup! fix(search): expose markdown frontmatter on chunks * feat(markdown): add include_frontmatter_keys_in_metadata allow-list opt-in --------- Co-authored-by: RerankerGuo <1875366113@qq.com> Co-authored-by: Ziyang Guo <121015044+RunMarshal@users.noreply.github.com> |
||
|
jinliyl
|
2612d25959
|
feat(lme): add cli execution and agentic search tooling (#334)
* feat(service): add CLI service for local job execution - Introduce CliService to execute single jobs locally without serving ports - Add prepare_start_config and should_precheck_start functions for CLI job setup - Update reme start command to use CLI service when job argument is provided - Change default service backend from http to cli in jinli_lme config - Modify SearchStep to use constants and rename configuration parameters - Add unit tests for CLI service functionality and configuration handling - Update file extension support to include json format in addition to md and jsonl * feat(search): add BM25 and vector search steps with configuration updates - Add Bm25SearchStep and VectorSearchStep classes with tool context deduplication - Register new search step components in index module - Update configuration to use separate vector_search and bm25_search endpoints - Modify LLM models from qwen3.7-plus/glm-5.1 to glm-5.2 variants - Adjust search parameters and remove hybrid search implementation - Configure embedding store as default in storage settings - Remove auto-memory and file catalog configurations - Update watch directories from multiple paths to session_dir only * feat(agent): add tool result offloading and workspace management - Add tool_results_dir configuration option for offloaded tool results storage - Implement ToolResultOffloadMiddleware to persist large tool results to files - Create WorkspaceBackend to standardize file operations across tools - Add configurable builtin tools selection with sequential execution option - Integrate middleware support for agent wrapper with offloading capability - Update application initialization to create tool results directory - Add safety mechanisms for filesystem operations with sanitized filenames - Enhance agent wrapper with configurable working directory handling - Upgrade agentscope dependency to version 2.0.4 for improved features # Conflicts: # reme/application.py * feat(benchmark): add LongMemEval agentic search and result management - Introduce AgenticAnswerStep for agent-based history search - Add LmePrepareJudgeStep and LmeSaveResultStep for evaluation pipeline - Implement AddDraftStep and ReadAllDraftStep for evidence accumulation - Update configuration with new agent wrapper and search parameters - Add comparison script for analyzing agent run differences - Include documentation for LongMemEval failure analysis - Enhance tool result offloading with skip options - Modify search defaults and indexing behavior * feat(agent): implement tool result offloading with system reminders - Added tool_result_offload_message parameter to agent wrapper reply method - Implemented configurable reminder template for offloaded tool results - Created system reminder messages when tool results are offloaded to files - Added Chinese user message template for agentic answer step - Updated tool result offloading middleware to use custom reminder templates - Enhanced agentic answer instructions to handle long tool results via draft storage * feat(scripts): add LongMemEval results summarization tool - Create summarize_lme_results.py script to analyze result JSON files - Implement command line interface with answer id and dataset root options - Add support for specifying index range with start and end parameters - Include option to show failure details and non-successful completions - Calculate completion statistics and accuracy metrics - Display detailed breakdown of yes/no/other judgements - Handle missing and unreadable result files gracefully - Format output with percentages and comprehensive summary statistics * feat(summarize_lme_results): add question type breakdown to result summary - Import defaultdict from collections module - Add by_type dictionary to track statistics by question type - Count completed, yes, no, and other responses for each question type - Display detailed breakdown table showing accuracy by question type - Include question type column when processing judgements - Print comprehensive summary with question type distribution - Calculate and display accuracy percentage for each question type category * feat(lme): switch to qwen3.7-max model and add shuffle functionality - Changed default LLM model from glm-5.1 to qwen3.7-max in jinli_lme.yaml - Added random module import for shuffle functionality - Implemented --shuffle argument with BooleanOptionalAction for dataset shuffling - Added --seed argument to control random seed for reproducible shuffling - Applied random shuffle to dataset indices when shuffle is enabled - Added console output showing shuffle operation and seed information * fix(cli): set default random seed for shuffle functionality - Changed default seed value from None to 42 for consistent shuffling behavior - Ensures reproducible results when using shuffle option without explicit seed - Maintains backward compatibility while providing deterministic defaults * refactor(benchmark): update agentic answer guidelines for grounding - Updated English instruction to emphasize strict grounding in retrieved context - Modified Chinese instruction to stress evidence-based responses without inference - Removed redundant conciseness requirement in both language versions - Enhanced clarity on proper use of draft saving and retrieval mechanisms - Strengthened emphasis against hallucination of unsupported facts * refactor(benchmark): update agentic search instructions and configuration - Replace separate vector_search and bm25_search with unified search tool - Update agent instructions to use single search tool with multiple strategies - Simplify Chinese instructions for search methodology - Add comprehensive search tool configuration with hybrid vector/BM25 capabilities - Increase model retry attempts from 1 to 3 for better reliability - Remove redundant tool references from job_tools list * feat(search): add configurable search limit with environment variable support - Remove hardcoded limit and min_score parameters from config schema - Increase LLM context size from 200000 to 1000000 - Add REME_SEARCH_LIMIT environment variable support for search configuration - Implement command line argument --search-limit to override default search limit - Add input validation to ensure search limit is positive - Modify subprocess execution to pass environment variables - Update search step to use dynamic default limit from environment or fallback to 5 * refactor(benchmark): remove agentic answer step and related configurations - Removed AgenticAnswerStep class and its registration - Deleted agentic_answer.yaml prompt configuration file - Removed agentic answer related job definitions from jinli_lme.yaml - Cleaned up tool result offloading middleware implementation - Removed tool_results_dir configuration field from application config - Deleted comparison and analysis scripts for agent runs - Removed agentic answer step from LME init module exports - Updated agent wrapper to remove tool result offloading functionality - Removed unused imports and dependencies in agent wrapper module * refactor(benchmark): remove unused LME result processing components - Removed LmePrepareJudgeStep and LmeSaveResultStep classes from benchmark module - Cleaned up imports and exports in lme module initialization - Removed unused middleware configuration from agent wrapper - Deleted obsolete result.py file containing deprecated result processing logic - Simplified agent instantiation by removing middleware parameter - Updated import statements to reflect removed dependencies * refactor(index): remove unused search steps and update imports - Remove Bm25SearchStep and VectorSearchStep from index steps module - Remove unused prepare_start_config and should_precheck_start exports - Move import statements to proper location in reme.py - Update test module to use direct import path for CliService - Remove vector_search and bm25_search configurations from jinli_lme.yaml - Add workspace directory environment variable configuration - Add docstring to getcwd method in agent wrapper - Remove empty middleware list from agent wrapper initialization * feat(index): add BM25 and vector search steps with tool context deduplication - Add Bm25SearchStep for plain BM25 keyword search with tool_context deduplication - Add VectorSearchStep for plain vector search with tool_context deduplication - Implement tool context state management with TTL-based deduplication - Add support for chunk deduplication across tool contexts within TTL window - Update index steps module to include new search step classes - Add test coverage for CLI metadata output functionality - Refactor CLI service to remove unused show_status parameter - Update documentation comments to reflect internal service configuration * feat(steps): add Python code execution capability - Introduce PythonExecuteStep to run Python code in subprocess - Add configuration for python_execute step in jinli_lme.yaml - Register python_execute in available tools list - Implement timeout handling with default 60 second limit - Capture stdout/stderr output and return code metadata - Add comprehensive unit tests for execution scenarios - Support workspace directory context for code execution - Handle timeout errors and runtime exceptions gracefully * refactor(python_execute): replace subprocess with asyncio for Python code execution - Replace subprocess.run with asyncio.create_subprocess_exec for non-blocking execution - Add _PythonResult dataclass to encapsulate execution results and timeout status - Implement proper timeout handling with asyncio.wait_for and process.kill() - Update metadata to include returncode and stderr when timeout occurs - Convert synchronous _run_python method to asynchronous implementation - Maintain backward compatibility while improving execution reliability * refactor(python_execute): replace subprocess with asyncio for Python code execution - Replace subprocess.run with asyncio.create_subprocess_exec for non-blocking execution - Add _PythonResult dataclass to encapsulate execution results and timeout status - Implement proper timeout handling with asyncio.wait_for and process.kill() - Update metadata to include returncode and stderr when timeout occurs - Convert synchronous _run_python method to asynchronous implementation - Maintain backward compatibility while improving execution reliability |
||
|
xyf2020
|
82971ac5b0
|
feat(chunker): better json chunker and json chunker (#325)
* feat(file_chunker): add dedicated JSON and JSONL file chunkers - Add JsonFileChunker: structure-aware chunking preserving nested key paths, optional list-to-dict conversion, size measured by json.dumps() char count - Add JsonlFileChunker: line-aligned sliding-window chunking with configurable overlap, supports char/byte mode switching - Register both chunkers in default.yaml (json for .json, jsonl for .jsonl) - Add comprehensive unit tests (21 + 20 test cases) * chore(config): update default chunker supported_extensions to txt/log * refactor(json_chunker): optimize _build_tree O(n²) serialization and rewrite tests - Fix O(n²) redundant json.dumps in _build_tree: * Empty containers handled directly as leaves (0 serialization) * Non-empty containers recurse first, then reconstruct+dump once * Only containers that become leaves pay serialization cost - Add _reconstruct_object/_reconstruct_array helpers - Remove dead code: _merge_json method - Apply user changes: min_element_size formula 0.01->0.05, threshold < to <= - Use indent=None for compact output (consistent with _SizeNode estimation) - Remove unused _text_size from JsonlFileChunker Test rewrite: - Replace try/finally boilerplate with make_json fixture - Group tests into TestXxx classes with pytest.mark.parametrize - Add TestOutputValidation: 9 parametrized scenarios verifying: * All chunks are valid JSON * Text length <= chunk_chars (with single-leaf tolerance) * Leaf-value concatenation matches original data (dict + array roots) - Add TestSizeNode: incremental size accuracy tests - Add TestDfsAlgorithm: path wrapping, DFS order, calibration tests - Update test_min_element_size_formula for new 0.05 multiplier - Update test_build_tree_structure for larger min_element_size * chore: apply black formatting to test files |
||
|
jinliyl
|
eb471d7d94
|
fix(embedding): reject mismatched embedding dimensions (#330)
* fix(embedding): enforce strict dimension matching for embeddings - Add _embedding_dim_matches method to validate embedding dimensions - Reject embeddings with mismatched dimensions instead of padding/truncating - Drop stale embeddings with wrong dimensions during loading and upsert operations - Disable embedding store when query dimensions don't match configured dimensions - Fail health checks when embedding dimensions don't match expected values - Skip chunks with wrong dimensions during FAISS index rebuild - Add comprehensive tests for dimension validation behavior * refactor(file_store): simplify conditional checks in vector search and test assertions - Combine multiple conditionals into single check for empty FAISS index - Replace explicit empty list comparison with boolean check for node embedding calls - Maintain same functional behavior while improving code readability * fix(embedding): harden dimension validation helpers |
||
|
jinliyl
|
10da205797
|
feat(benchmark): add lme benchmark steps (#326)
* refactor(search): replace time module with datetime for timestamp generation - Removed unused time import - Added static method _now_ts using datetime.timestamp - Updated clock parameter to use _now_ts method instead of time.time - Maintained same timestamp precision and functionality * test(http): add tests for HTTP client display formatting - Add test for default metadata hiding behavior in CLI output - Add test for metadata display when show_metadata is enabled - Verify _format_for_display method correctly formats response text - Test both success case and metadata inclusion scenarios * chore(build): remove longmemeval from gitignore - Removed longmemeval directory from gitignore list - Kept evaluation and datasets directories in ignore list - Updated gitignore configuration for proper version control |
||
|
imrewce
|
1e798d3b4e
|
fix(file_io): fix risk of out-workspace paths (#322)
* fix(file_io): fix risk of out-workspace paths * chore(file_io): remove unused unittest file |
||
|
jinliyl
|
7369342115
|
feat(search): add tool context deduplication and improve search configuration (#321)
* feat(search): add tool context deduplication and improve search configuration - Modify _make_tool methods to accept and inject tool_context_id parameter - Add tool_context_id handling in AS and CC agent wrappers - Increase search candidate multiplier from 3.0 to 5.0 in default config - Extend HTTP client timeout from 30s to 3600s - Add tool context deduplication logic to prevent duplicate search results - Implement TTL-based expiration for seen chunks in tool contexts - Add comprehensive unit tests for tool context deduplication behavior - Update .gitignore to exclude longmemeval directory - Add time import for timestamp functionality in search step * refactor(search): replace time module with datetime for timestamp generation - Removed unused time import - Added static method _now_ts using datetime.timestamp - Updated clock parameter to use _now_ts method instead of time.time - Maintained same timestamp precision and functionality |
||
|
xyf2020
|
43a407bc4f
|
feat: add start_date/end_date time filter support for search job (#317)
* feat: add start_date/end_date time filter support for search job
- Add _extract_date_from_path to extract validated YYYY-MM-DD from chunk paths
- Add start_date/end_date filtering in _matches_search_filter
- Implement progressive recall in FaissLocalFileStore.vector_search
- Promote start_date/end_date from context to search_filter in SearchStep
- Add start_date/end_date parameters to search job in default.yaml
- Add unit tests for date filter functionality
* fix: validate/normalize date filters and harden _extract_date_from_path
Address three code-review comments on the time_filter search feature:
1. Validate/normalize start_date and end_date before string comparison.
_matches_search_filter does lexicographic comparison against path_date
(always canonical YYYY-MM-DD). Raw caller values like '2026-2-28' or
'abc' would produce silently wrong results. Now SearchStep normalizes
valid dates via extract_daily_date (with strptime fallback for
non-zero-padded input) and silently ignores invalid dates with a
logger.warning, removing them from the filter.
2. Clarify behavior for paths without embedded dates.
Added optional strict_date_filter parameter (default False). When True
and at least one date bound is active, chunks whose path yields no date
(e.g. digest/personal/topic.md) are excluded. When False (default),
the existing behavior is preserved — dateless paths pass through.
3. Harden _extract_date_from_path against non-standard suffixes.
Previously parts[1].split('.')[0] accepted '2026-05-18.anything' as a
valid date. Now only exact 'YYYY-MM-DD' (dir) and 'YYYY-MM-DD.md'
(day-index) forms are accepted.
|
||
|
Ziyang Guo
|
c060933e4d
|
fix(auto-memory): preserve message timestamps (#310)
* fix(auto-memory): preserve message timestamps * fix(auto-memory): infer daily date from messages * feat(file_io): add strict date parsing and improve daily date handling - Add new parse_daily_date function for strict YYYY-MM-DD validation - Replace extract_daily_date with parse_daily_date for explicit date validation - Change _messages_day to use max date instead of min for historical imports - Reorder imports to maintain consistent module ordering - Move session message saving after date validation in auto_memory - Add comprehensive tests for invalid date rejection before saving - Add tests for strict YYYY-MM-DD date format validation - Update test names to reflect latest date behavior --------- Co-authored-by: Ziyang Guo <121015044+RunMarshal@users.noreply.github.com> Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com> |
||
|
Sen Huang
|
be7d1c0cf2
|
refactor(transfer): drop orphaned ingest step, make service discovery cross-platform (#300)
* refactor(transfer): drop orphaned ingest step, make service discovery cross-platform - Remove ingest step: superseded by auto_resource (drop files under resource/ → watcher interprets them); its meta.json/<date>.md outputs had no consumers and tripped the auto_resource watcher. - Replace lsof/pgrep shell-outs in service_utils with psutil (per-process enumeration, no root needed on macOS) for Windows/macOS/Linux support. - Add cross-platform test coverage for _pid_on_port / _scan_reme_procs. - Deps: +psutil, -filelock (only used by the removed ingest lock). * chore(release): bump version to 0.4.0.5 |
||
|
jinliyl
|
ad7893e9c4
|
fix(mcp): resolve circular import issues and update dependencies (#296)
* fix(mcp): resolve circular import issues and update dependencies - Moved fastmcp imports inside functions to prevent circular dependencies - Replaced _TRANSPORT_MAP with _VALID_TRANSPORTS set for transport validation - Updated version number from 0.4.0.3 to 0.4.0.4 - Added claude-agent-sdk dependency to core optional dependencies - Used TYPE_CHECKING imports for FastMCP related types - Restructured transport mapping logic within function scope - Fixed string annotation for CallToolResult type hints * refactor(tests): update date handling in daily steps tests - Replace _date.today() with timezone-aware now function - Use Asia/Shanghai timezone for date formatting - Change return format to use strftime instead of isoformat - Import now function from reme.steps.evolve module * refactor(tests): clean up unused imports in daily steps test - Removed unused date import from datetime module - Removed redundant pathlib Path import that was already imported later - Kept necessary imports for asyncio, os, tempfile, warnings, and frontmatter modules * test(daily_steps): update test to include application context for daily list step - Add ApplicationContext initialization with temporary workspace directory - Register file store component in application context - Pass application context to DailyListStep constructor - Maintain existing test assertion behavior for date metadata verification |
||
|
jinliyl
|
ffb4d08c4f
|
feat(mem): Enhance daily note system with metadata handling and write functionality (#295)
* feat(file_io): add daily_write step for creating daily notes with conversation metadata - Add DailyWriteStep class that delegates to write job for creating daily notes - Register daily_write job in default configuration with proper parameters - Include validation for name and session_id path components - Add test coverage for daily_write functionality including metadata handling - Preserve existing job execution method in application.py after repositioning - Update base_step.py to use positional-only parameter syntax for job methods - Import and expose DailyWriteStep in file_io module initialization - Override reserved metadata keys (name, description, session_id, source_conversation) with fixed values - Refresh daily index after successful write operation - Generate proper source conversation links in markdown format * feat(daily): refactor daily note system with enhanced metadata handling - Introduce validate_filename_component function and export it - Add _INDEX_HIDDEN_METADATA_KEYS to hide conversation metadata from index - Update scan_notes to exclude hidden metadata keys from index rendering - Modify auto_memory to use daily_write tool and manage session frontmatter - Implement session note lookup and renaming based on frontmatter name - Update daily_list to return flattened note metadata including session info - Change daily_write to dispatch write step instead of running job - Add test cases for updated daily note functionality and metadata handling - Update version from 0.4.0.2 to 0.4.0.3 * fix(evolve): correct metadata update in auto memory response - Fixed trailing comma issue in metadata dictionary update - Ensured proper formatting of response metadata structure - Maintained existing functionality while fixing syntax error * refactor(auto_resource): replace daily_create with dynamic note management - Remove DailyCreateStep and related exports from file_io module - Replace static daily note creation with dynamic resource-linked card system - Implement LLM-suggested naming with frontmatter-driven file management - Add source_resource linking for tracking original files - Introduce collision handling with hash-based suffixes - Update documentation to reflect new resource card workflow - Modify auto_resource prompts to use write/edit tools instead of daily_create - Adjust test fixture comments to match new agent behavior - Update framework diagrams and quick start examples accordingly * feat(app): add version info to app initialization and update auto-memory logic - Include version number in application startup logging - Remove tool result truncation logic from auto-memory step - Update auto-memory to exclude tool_result blocks from saved history - Add test case to verify tool results are filtered out from message saving - Update YAML prompts to clarify filename naming rules without dates - Modify configuration to support new dispatch steps format with persistence control * feat(auto_memory): add note modification tracking and optimize frontmatter updates - Add _note_bytes and _note_modified methods to track actual file changes - Optimize frontmatter updates by checking existing metadata before update - Add modified flag to response metadata indicating actual note changes - Update logging to include modified status in various operations - Add comprehensive tests for modified/unmodified detection scenarios - Enhance result hook logic to skip when no actual changes occur - Refactor metadata handling to properly track creation vs modification status |
||
|
jinliyl
|
8b82ff88d0
|
feat(evolve): enhance agent reply processing and logging capabilities (#293)
* feat(evolve): enhance agent reply processing and logging capabilities - Add agent_reply_result_text function to extract final user-visible text from agent replies - Implement comprehensive logging throughout auto_memory, auto_resource, and dream modules - Add max_units configuration option to limit extracted memory units - Improve error handling and validation in auto_resource step - Refactor dream extract step to respect max_units limit during processing - Enhance summary rendering in dream finish step with detailed breakdown - Add result hook functionality for embedding hosts integration - Implement loose resource filename handling for root-level resources - Update test cases to reflect new functionality and improved error messages * test(background-steps): update fake upsert function to include created parameter - Modified fake_upsert function to accept 'created' parameter instead of '_created' - Added 'created' field to captured dictionary in fake_upsert function - Included 'created': True in the expected response dictionary for test case - Updated test assertion to match new parameter structure |
||
|
jinliyl
|
afe12b16db
|
feat(file_store): add embedding backfill for persisted chunks (#292)
- Implement _backfill_missing_embeddings method to handle chunks without embeddings - Add logic to identify and process chunks that predate embedding feature - Integrate backfill process into store loading sequence - Add proper error handling and logging for backfill operations - Create unit test for embedding backfill functionality - Ensure backfilled embeddings are properly persisted to storage |
||
|
jinliyl
|
7d86658f33
|
Refactor logging levels and add dream schema definitions (#291)
* chore(logging): change info logs to debug level for data loading operations - Changed stopwords loading log from info to debug level - Changed file catalog nodes loading log from info to debug level - Changed file graph nodes loading log from info to debug level * feat(dream): add dream schema definitions and enum for auto-dream functionality - Add DreamBucketEnum with procedure, personal, and wiki values - Create comprehensive dream-related Pydantic models including DreamUnit, DreamTopic, DreamExtractOutput, IntegrateOutcome, TopicSelectionOutput, ProactiveResult, and DreamState - Move schema definitions from local step module to shared schema package - Update dream extraction and integration steps to use new enum-based bucket validation - Initialize digest directories for each dream bucket type - Enhance embedding store health check with workspace directory logging * refactor(tests): update DreamState import path in test_auto_dream.py - Move DreamState import from reme.steps.evolve.dream.schema to reme.schema - Maintain same functionality with updated module reference - Align import with new schema location in project structure |
||
|
jinliyl
|
a3bd81bde2
|
Update version to 0.4.0.2 and improve tokenizer index handling (#290)
* fix(core): update version number to 0.4.0.1 - Incremented version from 0.4.0.0 to 0.4.0.1 in __init__.py * fix(index): remove stopwords path from tokenizer config and add keyword index repair - Remove stopwords_path from tokenizer config to prevent index forking by install path - Add _sync_keyword_index_from_chunks method to repair keyword index when persisted state mismatches - Implement test for keyword index repair from persisted chunks when missing - Add test to verify tokenizer fingerprint ignores stopwords absolute path - Update version from 0.4.0.1 to 0.4.0.2 * feat(dream): add scan_days parameter to dream extraction process - Add scan_days configuration option to default.yaml with default value of 2 - Implement recent_dates utility function to calculate date ranges for scanning - Modify DreamExtractStep to scan multiple days based on scan_days parameter - Update dream extraction to process files across multiple dates instead of single day - Extend DreamState schema to include dates and scan_days fields - Update DreamTopicsStep to handle multi-day topic processing - Modify finish step to checkpoint files from all scanned dates - Add comprehensive tests for multi-day scanning functionality - Update prompt templates to include scan dates information - Refactor topics writing logic to target specific date rather than current date |
||
|
Sen Huang
|
e31db5fe19
|
docs: rename vault_dir to workspace_dir in documentation and examples (#286)
* docs: rename vault_dir to workspace_dir in documentation and examples * refactor(extract): format long method call across multiple lines * refactor(extract): format system prompt parameters for better readability |
||
|
jinliyl
|
206a53e5ed
|
init: reme version 0.4.0 (#284) |