916 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
jinliyl	46adb5ae1e	feat: add daily paper cookbook and DingTalk agent integration (#385) ... * feat(daily-paper): add daily paper cookbook workflow with schema and tests - Introduce daily paper schema types (DailyBriefOutput, PaperInfo, PaperNoteOutput, etc.) - Create daily paper cookbook module with analyze, collect, digest, rank, and select steps - Add cookbook entry point and integrate into main steps module - Replace job config export with daily brief output in schema exports - Add comprehensive unit tests covering pipeline, filtering, and output generation - Update dependencies including openai-codex and pypdf packages - Configure standalone daily paper cron job with proper scheduling and routing * test(daily_paper): update tests to use Claude Code wrapper exclusively - Add test to verify web search is disallowed by default in Claude Code - Update imports to include DailyBriefOutput, PaperNoteOutput, and PaperSelection schemas - Change test name from standalone_config_has_backend_split to reflect Claude Code only usage - Remove default agent wrapper and configure all steps to use Claude Code wrapper - Rename select_wrapper to cc_wrapper for clarity and consistency - Remove duplicate Claude Code wrapper initialization - Update test assertions to verify output schema usage matches expected sequence - Remove unused as_llm component from standalone configuration test * refactor(agent-wrapper): simplify skill resolution logic across all wrappers - Replace duplicate skill resolution code with centralized _resolve_project_skills method - Add project_path property with configurable relative path resolution - Introduce proper validation for skill names and directory existence - Change Codex wrapper to use project_path instead of workspace_path for skills - Add SKILL.md requirement validation for project skills - Remove redundant skill processing logic from individual wrappers * feat(daily_paper): add daily paper workflow with PDF analysis and brief generation - Implement shared state management and file helpers for daily-paper steps - Add PDF download and text extraction capabilities with arXiv integration - Create paper collection step with Hugging Face weekly/monthly rankings - Build ranking system using reciprocal-rank fusion with memory keyword scoring - Add Claude Code integration for paper analysis and detailed note generation - Implement digest step to create final five-minute brief from detailed notes - Add configuration for standalone daily cookbook application with cron scheduling - Create typed schema for paper information, selection, and output formats - Add atomic file writing with temporary file safety mechanisms - Implement exclusion logic for previously recommended papers and daily filters * feat(daily_paper): add DingTalk notification integration and enhance logging - Integrate DingTalk markdown send step to notify groups about daily paper briefs - Add comprehensive logging throughout daily paper workflow including start/finish events - Update daily paper analysis prompt to include code repository context requirement - Configure DingTalk notification in daily_cookbook.yaml with app credentials - Add dingtalk-stream dependency for proactive message API integration - Enhance daily paper README with DingTalk notification section and updated flow chart - Implement detailed logging for each step including paper processing and agent calls - Add test coverage for DingTalk markdown sending functionality and configuration - Update pre-commit config to exclude skills directory from checks - Add .claude/skills to gitignore for local development environment * refactor(dingtalk): move dingtalk_stream import to local scope and improve code safety - Moved global dingtalk_stream import to local scope in send.py to avoid eager loading - Added dynamic import with error handling for optional dependency cases - Updated test suite to verify lazy loading behavior works correctly - Fixed markdown title generation by using safe variable naming in wait.py - Enhanced test coverage for arxiv PDF download caching functionality - Updated application context initialization with proper resource directory configuration - Modified paper metadata to include source PDF path reference in output files * refactor(daily_paper): remove manifest system and store selection metadata in digest files - Remove JSON manifest creation and storage functionality - Store selection data directly in digest file frontmatter instead of separate manifest files - Add load_saved_selection method to rebuild selection from digest and paper-note metadata - Update README documentation to reflect new cookbook workflow architecture - Modify test cases to verify selection metadata in digest files instead of manifest JSON - Remove unused json import from multiple daily paper modules - Integrate PaperSelection schema for proper data validation in stored metadata * docs(daily_paper): add bilingual cookbook guides	2026-07-22 19:17:01 +08:00
xyf2020	630f26b119	feat(search): scoped dedup, session-chunk merge, and unified recall formatting (#384) ... * feat(search): add tool_context-scoped chunk dedup with TTL Introduce _ToolContextDedupMixin shared by search/vector_search/bm25_search to skip already-seen chunks within one agent tool_context. Per-context state lives in app_context.metadata with configurable TTL (default 24h). * feat(search): unify chunk answer rendering with merge and explicit empty messages - Refactor SearchStep/VectorSearchStep/Bm25SearchStep to share format_chunks_answer for consistent source rendering and adjacent session-chunk merging. - Distinguish empty results: ALL_RETURNED_MESSAGE when dedup removes everything vs NO_RESULTS_MESSAGE when nothing matched. - Bump JsonlFileChunker default max_chars to 4000. - Add unit tests for source-format merge and empty-result messages. * refactor(config): reorganize file_chunker components and move jsonl max_chars into config - Register explicit markdown/json/jsonl chunkers in beam.yaml and lme.yaml with markdown options (embed_toc, max_ast_sections, frontmatter handling) and jsonl max_chars=4000. - Restrict default chunker to txt/log extensions. - Revert JsonlFileChunker code default max_chars back to 2000; the 4000 value now lives in config. * chore(benchmark): increase longmemeval num_items from 64 to 500 * refactor(search): split SearchStep into simplified and v2 variants, extract counter utility - Extract global_counter_next from ApplicationContext into reme/utils/counter.py as a standalone function operating on metadata dict with lazy initialization. - Split SearchStep into two variants: - SearchStep (simplified): inline chunk.id dedup, single-branch vector/keyword optimization based on vector_weight, inline answer formatting. - SearchV2Step (full): preserves _ToolContextDedupMixin with interval-subset-aware dedup and format_chunks_answer with session-aware chunk merging. - Update beam.yaml and lme.yaml to use search_v2_step for benchmark jobs. - Rename existing search tests to test_search_v2_step_* and add new test_search_step_* tests covering the simplified variant. * fix: normalise missing trailing newline in _build_union_chunk to prevent line collision * refactor: lazy-init counter tree in ApplicationContext metadata - Remove hardcoded _counter_tree and _counter_tree_lock initialization from ApplicationContext.metadata; rely on lazy initialization in reme.utils.counter.global_counter_next on first call - Set longmemeval num_items back to 500 - Remove obsolete trailing-newline collision tests --------- Co-authored-by: sa-buc <jiangniurou.xyf@dail-algo011164204033.ET135>	2026-07-22 17:17:23 +08:00
xyf2020	7b1da5a9ee	feat(benchmark): add BEAM & restructure LongMemEval evaluation framework (#375) ... * feat(eval): add LongMemEval evaluation framework with tool_defaults date injection - Add evaluation/longmemeval/ with run.py, config.yaml, and test scripts - Add reme/config/longmemeval.yaml for evaluation-specific model config - Add tool_defaults mechanism to as_agent_wrapper for injecting default tool kwargs (uses setdefault so LLM-provided values take priority) - Pass tool_defaults={'daily_write': {'date': day}} in auto_memory to ensure notes always use the correct historical date - Add timestamp interpolation (_interpolate_timestamps) in auto_memory for filling missing created_at fields via linear interpolation - Evaluation pipeline: ingest sessions -> dream -> search -> answer -> judge - Uses qwen3.6-flash for memory, qwen3.7-max for answer/judge * chore: gitignore logs/results/demo.py, keep empty dirs * chore: update .gitignore * feat(eval): add multiprocessing and session time filtering to longmemeval runner - Replace async execution with synchronous + multiprocessing for parallel item evaluation - Add filter_future_sessions option to only ingest sessions <= question date - Add question_types filtering in config - Add result summary with binary accuracy and avg score - Update config defaults (oracle variant, 50 items, 32 workers) - Minor code style fixes in agent_wrapper and auto_memory * feat: add bench_query_step with ReAct agent for benchmark query phase - Add BenchQueryStep using agent_wrapper with search job tool - Replace manual search+LLM answer in run.py with bench_query_job - Remove unused answer LLM config from longmemeval.yaml - Register benchmark step module in steps/__init__.py * feat: add start_date/end_date time filter support for search job - Add _extract_date_from_path to extract validated YYYY-MM-DD from chunk paths - Add start_date/end_date filtering in _matches_search_filter - Implement progressive recall in FaissLocalFileStore.vector_search - Promote start_date/end_date from context to search_filter in SearchStep - Add start_date/end_date parameters to search job in default.yaml - Add unit tests for date filter functionality * fix: validate/normalize date filters and harden _extract_date_from_path Address three code-review comments on the time_filter search feature: 1. Validate/normalize start_date and end_date before string comparison. _matches_search_filter does lexicographic comparison against path_date (always canonical YYYY-MM-DD). Raw caller values like '2026-2-28' or 'abc' would produce silently wrong results. Now SearchStep normalizes valid dates via extract_daily_date (with strptime fallback for non-zero-padded input) and silently ignores invalid dates with a logger.warning, removing them from the filter. 2. Clarify behavior for paths without embedded dates. Added optional strict_date_filter parameter (default False). When True and at least one date bound is active, chunks whose path yields no date (e.g. digest/personal/topic.md) are excluded. When False (default), the existing behavior is preserved — dateless paths pass through. 3. Harden _extract_date_from_path against non-standard suffixes. Previously parts[1].split('.')[0] accepted '2026-05-18.anything' as a valid date. Now only exact 'YYYY-MM-DD' (dir) and 'YYYY-MM-DD.md' (day-index) forms are accepted. * feat(eval): LLM-as-Judge per-type prompt routing, binary-only, progress tracking - Remove 0-5 score metric, keep only binary (yes/no) classification - Load per-question-type judge prompts from llm-as-judge.json (temporal-reasoning, knowledge-update, single-session-preference, __default__) - Replace SCORE_JUDGE_PROMPT with type-specific BINARY_JUDGE_PROMPT template - judge_response(): parameter 'metric' -> 'question_type', returns single 'judgment' - Summary output: add per-type accuracy breakdown, remove score stats - Add progress tracking: background thread prints PROGRESS every 10min - Add FINAL progress line and total elapsed time on completion - Add --log-level, --reme-log-level, -q CLI arguments - Parallel mode: pool.map -> pool.imap_unordered for real-time progress - config.yaml: full oracle (10000 items), 32 workers, all question types - Add kill.sh (process cleanup) and run_async.sh (background eval launcher) * docs: add LongMemEval oracle evaluation results (61.6% accuracy) * feat(bench): add MAX_ITERATION limit to BenchQueryStep and add _auto_memory.yaml * feat: add golden session benchmark & eval_only mode with refined prompt - Add benchmark/longmemeval/run_golden_session.py for golden session evaluation - Refine PROMPTED_SYSTEM_PROMPT: concise answer rule, remove 'Information not found' fallback - Add eval_only mode to run.py (--eval_only flag) - Add multiple eval config variants (evalonly, full, test5) - Add analyze_results.py for result parsing - Update auto_memory.yaml, longmemeval.yaml, application_config - Update result-longmemeval.md with latest evaluation results - Add benchmark results to .gitignore * update: refine answer prompts and increase max iteration to 6 - Tighten prompted-answer system prompt for more concise output - Comment out 'Information not found' fallback rule - Increase MAX_ITERATION from 5 to 6 in bench_query - Add recall_eval.py - Update evaluation results * feat(chunker): add dedicated JSON and JSONL file chunkers (cherry-pick from upstream #325) - Add JsonFileChunker: structure-aware chunking preserving nested key paths, optional list-to-dict conversion, size measured by json.dumps() char count - Add JsonlFileChunker: line-aligned sliding-window chunking with configurable overlap, supports char/byte mode switching - Register both chunkers in default.yaml (json for .json, jsonl for .jsonl) - Add comprehensive unit tests (21 + 20 test cases) * feat(service): add CLI service for local job execution (from upstream #334) - Introduce CliService to execute single jobs locally without serving ports - Add prepare_start_config and should_precheck_start functions for CLI job setup - Update reme start command to use CLI service when job argument is provided - Add show_metadata to client kwargs for optional CLI metadata output - Add unit tests for CLI service functionality and configuration handling * feat(steps): add BM25/vector search steps, Python execute step, and draft steps (from upstream #334) - Add Bm25SearchStep for plain BM25 keyword search with tool_context deduplication - Add VectorSearchStep for plain vector search with tool_context deduplication - Add PythonExecuteStep to run Python code in subprocess with timeout handling - Add AddDraftStep/ReadAllDraftStep for draft accumulation scoped by tool context - Update SearchStep with tool_context dedup, dynamic default limit via REME_SEARCH_LIMIT env, and candidate_multiplier default changed from 3.0 to 5.0 - Add comprehensive unit tests for all new steps * feat(search): add tool context deduplication and improve search configuration (#321) * feat(search): add tool context deduplication and improve search configuration - Modify _make_tool methods to accept and inject tool_context_id parameter - Add tool_context_id handling in AS and CC agent wrappers - Increase search candidate multiplier from 3.0 to 5.0 in default config - Extend HTTP client timeout from 30s to 3600s - Add tool context deduplication logic to prevent duplicate search results - Implement TTL-based expiration for seen chunks in tool contexts - Add comprehensive unit tests for tool context deduplication behavior - Update .gitignore to exclude longmemeval directory - Add time import for timestamp functionality in search step * refactor(search): replace time module with datetime for timestamp generation - Removed unused time import - Added static method _now_ts using datetime.timestamp - Updated clock parameter to use _now_ts method instead of time.time - Maintained same timestamp precision and functionality * fix(file_io): fix risk of out-workspace paths (#322) * fix(file_io): fix risk of out-workspace paths * chore(file_io): remove unused unittest file * fix(as_embedding): support both agentscope 2.0.2 and 2.0.3 (#323) 2.0.3 promoted `dimensions` to a required first-class constructor argument while keeping a backfill from `parameters.dimensions`; 2.0.2 has no such argument and reads `dimensions` from `Parameters`. Keep `dimensions` in `Parameters` for both versions and, when the model constructor accepts `dimensions`, pass `dimensions=None` so 2.0.3's backfill promotes it out of `parameters`. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * Bump version to 0.4.0.7 * refactor: delegate LLM-as-Judge to answer_judge_step and update eval config/results - run.py: replace inline judge logic with judge_response_via_job using app.run_job('answer_judge') - longmemeval.yaml: expand benchmark configuration - bench_query.py: enhance benchmark query step - result-longmemeval.md: update evaluation results - judge_all_plus_results.json: add judge all-plus results * refactor: split longmemeval.yaml into lme.yaml/beam.yaml and unify job names - Split reme/config/longmemeval.yaml into lme.yaml (LongMemEval) and beam.yaml (BEAM) - Unify job names across both configs: agentic_answer, answer_judge, context_answer - Update evaluation/longmemeval/run.py and evaluation/beam/run_beam_eval.py to use unified job names - Update all evaluation config YAMLs to reference lme.yaml - Add BEAM benchmark step implementations (agentic_answer, context_answer, llm_judge) - Remove obsolete config_test5.yaml and test_5sessions.py * eval: BEAM 100K & LongMemEval cleaned-S 评测结果记录 - BEAM 100K eval-only (32并发, 20 case): Agentic 0.631, Prompted 0.468 - LongMemEval final GT (500题): Agentic 89.0%, Prompted 83.6% - 新增 benchmark/result-beam.md, benchmark/result-longmemeval.md - benchmark/beam/config.yaml: num_workers=32 * refactor: restructure benchmark directory and clean up gitignore rules - Consolidate benchmark outputs to benchmark/results/ with .gitkeep - Remove old benchmark scripts, configs and result files from benchmark/beam/ and benchmark/longmemeval/ - Add datasets/README.md and datasets/README_EN.md with download instructions - Add datasets/longmemeval/download.py and final_groundtruth_cleaned_s.json - Add memory_workspaces .gitkeep placeholders - Restructure .gitignore: fix duplicate entries, add BEAM dataset exclusion, refine logs/results ignore patterns - Remove stale result-beam.md and result-longmemeval.md from project root * chore: clean up longmemeval benchmark scripts and update dataset docs - Remove obsolete longmemeval benchmark runner/stats scripts - Update datasets/longmemeval README and add Chinese translation - Clean up final_groundtruth_cleaned_s.json * docs(benchmark): add reproduction guide for LongMemEval and BEAM - Add bilingual README for benchmark runners (EN/ZH) - Cover prerequisites, dataset download, run commands, configs, outputs, logs, and kill.sh * refactor: migrate auto_memory steps from evolve to benchmark-specific modules - Split auto_memory into beam and lme benchmark-specific implementations - Add auto_memory.py and auto_memory.yaml under steps/benchmark/beam and steps/benchmark/lme - Slim down evolve/auto_memory.py and auto_memory.yaml to shared base only - Remove obsolete evolve/_auto_memory.yaml - Update benchmark run.py, config YAMLs, and step __init__.py registrations - Update llm_judge and context_answer minor adjustments - Remove outdated test_lme_final_answer_review.py * revert(as_agent_wrapper): sync with upstream/main Remove local-only comment to keep file identical with upstream/main. * style: add trailing commas in benchmark __init__.py __all__ lists * chore: disable vector_weight range assertion in SearchStep * chore: add tests/integration/logs/ to .gitignore * refactor: replace scipy.stats.kendalltau with pure numpy implementation scipy is not listed in project dependencies. Implement Kendall's tau-b rank correlation using only numpy to remove the undeclared dependency. * feat(benchmark): add binary score metrics, update BEAM 1M results, and improve LLM retry/prompt config - benchmark/beam/run.py: add binary score calculation per rubric item and per-type/overall binary stats - benchmark/beam/config.yaml: switch to 1M dataset, reduce workers to 18 - benchmark/result-beam.md: add 1M evaluation results with binary scores - benchmark/result-longmemeval.md: minor formatting - reme/config/beam.yaml: increase max_retries to 5 and add retry_delay 5.0 for all LLM components - reme/config/lme.yaml: increase max_retries to 5 and add retry_delay for judge/prompted/bench components - reme/steps/benchmark/lme/agentic_answer.yaml: improve search strategy and answer rules prompts * fix(benchmark): fix line-too-long and add pylint disable for main() * refactor(longmemeval): use single cleaned-S dataset with embedded ground truth - Switch to agentscope-ai/ReMe_longmemeval_clean_s_v2 HuggingFace source - Remove separate final_groundtruth_cleaned_s.json (ground truth now in data file) - Simplify download.py to fetch only longmemeval_s_reme_cleaned.json - Remove dataset.variant and dataset.ground_truth_path config options - Update benchmark and datasets READMEs to reflect new workflow - Update .gitignore for new dataset filename * fix: rename loop variable to avoid pylint redefined-outer-name warning * refactor(benchmark): restructure datasets/memory_workspaces into benchmark and simplify auto_memory steps * refactor(benchmark): extract BaseAgenticAnswerStep into base module - Add reme/steps/benchmark/base/agentic_answer.py with shared agentic answer logic - Refactor beam/lme AgenticAnswerStep to inherit from BaseAgenticAnswerStep - Simplify lme/context_answer.py and update context_answer.yaml - Update result-longmemeval.md with latest evaluation results (agentic 91.0%) * refactor(benchmark): remove context_answer steps and unused configs - Remove beam/lme context_answer job definitions and step implementations - Remove prompted LLM component from beam.yaml and lme.yaml - Delete jinli_lme.yaml (no longer needed) - Simplify benchmark run.py scripts - Clean up .gitkeep files and update .gitignore - Remove unused import in search.py * chore: remove benchmark/results/.gitkeep --------- Co-authored-by: sa-buc <jiangniurou.xyf@dail-algo011164204033.ET135> Co-authored-by: jinliyl <6469360+jinliyl@users.noreply.github.com> Co-authored-by: imrewce <wce@pku.edu.cn> Co-authored-by: Sen Huang <48879559+ployts@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-21 19:09:50 +08:00
jinliyl	e7d44f6f3b	refactor(agent): unify agent subprocess env, sessions, skills, and MCP/service jobs (#382) ... * feat(config): add environment variable configuration for agent subprocesses - Add environment field to ApplicationConfig to store variables for agent subprocesses - Remove dynamic loading of .env files in agent wrappers - Introduce subprocess_environment property in base agent wrapper - Pass application-level environment variables to Claude Code and Codex agents - Load environment variables once at startup and pass to ReMe application - Remove dependency on load_env utility in agent wrapper implementations - Update tests to use configured environment instead of dynamic loading - Remove unused environment loading utilities and related test cases * refactor(mcp): remove channel notification system and related components - Removed channel notification step implementation - Removed claim channel step implementation - Removed ChannelSink class from MCP service - Removed channel-related documentation from AGENTS.md - Removed channel instruction text from MCP service - Removed all channel-related tests - Updated application context metadata comment to remove channel sink reference - Removed channel module initialization and imports * feat(service): add job whitelisting capability to BaseService - Add optional jobs parameter to BaseService.__init__ to configure job whitelist - Store jobs as set in self.jobs attribute for efficient lookup operations - Modify add_jobs method to filter jobs based on whitelist configuration - Update documentation in both English and Chinese to describe new feature - Add comprehensive unit tests for job whitelisting behavior - Implement flowchart update showing new filtering logic - Preserve existing enable_serve flag behavior alongside new whitelisting * refactor(service): enhance service job validation and MCP tool injection - Add strict validation for service jobs whitelist with detailed error messages - Implement injected job arguments support for MCP services with conflict detection - Add tool error handling for unsuccessful responses in MCP services - Remove duplicate job names in Codex agent wrapper using dict.fromkeys - Update MCP server argument format from single JSON array to repeated --job flags - Add comprehensive test coverage for job injection and error handling scenarios - Update documentation to reflect service job validation and MCP features - Ensure application cleanup occurs even when service lifespan encounters errors * feat(agent): update skill handling to preserve existing Claude skills - Change skills parameter processing to use 'all' instead of filtered list - Add logic to select project skills without restricting Claude's existing skills - Update variable naming from 'skills' to 'selected_skills' for clarity - Modify application context metadata documentation to clarify in-memory state usage - Add test case to verify configured skills are added without filtering existing skills - Update internal skill directory handling to use renamed variable consistently * refactor(agent): restructure agent wrapper components and session storage - Move CcFileSessionStore to separate module for better organization - Add SDK package version logging in base agent wrapper - Update Claude Code agent to use new session store structure with project keys - Refactor Claude Code agent wrapper to use proper type hints and SDK integration - Add support for server tool use events in Claude Code message processing - Improve error handling and resource cleanup in streaming operations - Update Codex agent wrapper with proper type annotations and configuration - Remove deprecated system prompt mode handling from Claude Code wrapper - Fix session path construction for Claude Code transcript storage - Update dependency injection and configuration handling patterns * fix(cc_agent_wrapper): resolve Claude Code SDK integration issues - Added dataclass import and created _BlockState for content block metadata tracking - Implemented proper MCP server name constant and tool context ID validation - Fixed tool_context_id injection to prevent duplicate assignment errors - Resolved skills parameter handling in build_options method - Enhanced job tools integration with MCP servers mapping validation - Replaced deprecated block_ids/block_types/tool_call_names with block_states dict - Updated message_delta to emit USAGE chunks instead of REPLY_END - Fixed stream result handling to ensure proper REPLY_END emission - Improved error handling for session mirror failures and rate limits - Added proper cleanup for expected trailing errors in streams - Refactored Codex agent wrapper initialization and configuration management - Removed obsolete system_prompt_mode from default config - Enhanced test coverage for new block state and error handling features - Fixed async generator handling with aclosing context manager - Improved chunk type mapping for Claude Code SDK events * refactor(tests): remove demo config tests from config parser test suite - Removed test_demo_config_registers_llm_jobs function and its assertions - Eliminated verification of LLM demo job configurations - Removed checks for agent wrapper component settings - Deleted assertions for model configurations and parameters - Cleaned up deprecated test cases related to demo config parsing * refactor(evolve): simplify Claude Code session store path structure - Removed redundant project key subdirectory from session link generation - Updated CcFileSessionStore initialization to use direct session directory path - Maintained existing session layout compatibility for backward compatibility - Added unit tests to verify session persistence behavior with existing transcripts - Ensured UUID-based session files remain accessible at expected locations - Preserved existing session directory structure without additional nesting * refactor(agent): defer optional Codex SDK imports until first use - Moved openai-codex imports inside functions to avoid mandatory dependencies - Added TYPE_CHECKING guard for development time type checking only - Implemented lazy loading mechanism with _get_async_codex_class function - Updated AsyncCodex initialization to occur on demand rather than at module level - Maintained backward compatibility while improving import performance - Added test case to verify package import works without optional Codex SDK - Updated agentscope dependency to version 2.0.4.post1 in pyproject.toml * test(embedded): add compatibility tests for in-process ReMe embedding - Add test suite for QwenPaw-style embedded configurations - Verify optional defaults remain preserved in embedded configs - Ensure in-process application API stays compatible - Test model injection and lifecycle management compatibility - Remove obsolete hermes agent plugin tests - Update CLI import test to cover multiple optional SDKs - Block claude_agent_sdk and openai_codex during import testing	2026-07-20 23:52:14 +08:00
jinliyl	b4333fbef8	feat(index): add bounded memory-aware batch processing (#381) ... * test(background_steps): add comprehensive tests for batch processing and memory management - Add test for catalog upserts in batches of at most 100 files - Add test for catalog deletes in batches of at most 100 paths - Add test for index memory budget reducing batches to one file - Add test for memory target limiting cumulative batch size - Add test for invalid batch memory settings rejection - Add test for continuing after one batch fails - Add test for yielding to event loop while building batch - Add test for modified file reusing unchanged embedding - Add test for reporting memory estimation failure without aborting feat(update_changes): implement bounded batch processing with memory management - Add configurable batch parameters with default values - Implement memory budget calculation based on available system memory - Add file inspection and memory estimation before processing - Implement batch flushing when limits are reached - Add proper error handling for batch operations - Support async yielding during batch building - Add comprehensive validation for batch configuration parameters - Implement memory estimation for indexing operations - Add batch size limiting for delete operations * test(steps): add tests for memory estimation failure handling - Add test case for isolated file processing when memory estimation fails - Add test case for proper release of flushed items before building next file - Implement weak reference tracking to verify payload lifetime management - Create parametrized tests for both source and item memory estimation methods - Add assertions to verify single-item batch behavior on estimation failures - Include comprehensive error handling verification for memory budget calculations * chore(version): bump version to 0.4.1.3 - Update __version__ from 0.4.1.2 to 0.4.1.3 in __init__.py * feat(index): support batch settings from environment * refactor(index): use direct batch defaults * refactor(index): configure memory estimates through step args * ci: simplify Windows smoke dependencies	2026-07-20 17:25:00 +08:00
Sen Huang	55ef4bd6ad	fix(proactive): expose topics in primary answer (#380)	2026-07-20 16:05:47 +08:00
jinliyl	cf22ef3b1d	feat: add codex auth modes, background embedding/index repair, and qwenpaw logging (#371) ... * feat(codex): add authentication mode support with thread-safe logging - Implement _CodexAuthConfig dataclass for resolved auth settings - Add auth_mode parameter with auto/api_key/oauth options - Separate API key and OAuth authentication flows - Force specific login method based on auth mode - Add explicit API key validation requirement - Serialize concurrent logger initialization in thread lock - Close logging handlers properly during cleanup - Update default config with auth_mode presets for codex and codex_oauth - Add comprehensive tests for authentication modes and concurrent logging * feat(file_store): implement background embedding backfill and keyword index repair - Add _after_embedding_backfill hook in FAISS local file store - Schedule startup embedding repair without delaying component readiness - Cancel and collect embedding backfill task during component shutdown - Log progress at fixed percentage boundaries for long-running operations - Process embedding backfill in configurable batch sizes with progress reporting - Rebuild keyword index in bounded batches with detailed mismatch diagnostics - Format stdlib logs consistently with QwenPaw console output using relative paths - Run embedding backfill as background task that doesn't block component startup - Add comprehensive tests for background embedding and keyword index repair scenarios * fix(file-store): repair graph-chunk consistency on load - Add _repair_graph_chunk_consistency method to detect and fix mismatched graph/chunk states - Clear torn graph/chunk state when missing or orphaned chunks are detected - Ensure keyword index sync handles empty chunks properly - Add comprehensive tests for graph-chunk consistency scenarios - Update test utilities to properly seed graph/chunk snapshots - Increment version to 0.4.1.2 * feat(file_io): enhance list step response format and add comprehensive logging - Format list output with bullet points for better readability - Add explicit "No files found" message when directory is empty - Include detailed timing information for file store startup phases - Add logging for chunk loading, graph consistency checks, and keyword indexing - Provide detailed metrics for embedding backfill operations - Add comprehensive test coverage for empty directory scenarios - Include batch processing statistics for embedding operations * feat(logger): add QwenPaw logging integration with forwarding mechanism - Introduce _ForwardToLoggerHandler to forward log records to target logger - Add qwenpaw logger integration that forwards ReMe logs to QwenPaw handlers - Maintain ReMe logger stability for modules that cache it at import time - Enable QwenPaw handlers to take effect without ReMe reconfiguration - Add comprehensive tests for stdlib forwarding to QwenPaw sinks - Support explicit REME_DISABLE_LOGURU=false to keep original Loguru backend - Preserve existing logging behavior when QwenPaw is not configured * fix(file_store): serialize concurrent FAISS dump operations to prevent corruption - Add asyncio lock to ensure only one FAISS dump operation runs at a time - Generate unique temporary filenames using UUID tokens for atomic replacement - Implement proper cleanup of temporary files in finally block - Add comprehensive test to verify concurrent dumps are serialized - Ensure atomic writes by replacing both index and idmap files together - Prevent partial state writes during concurrent access scenarios	2026-07-20 14:47:34 +08:00
Sen Huang	1c08eaa559	fix: enforce markdown chunk byte limits (#370)	2026-07-17 22:03:58 +08:00
Sen Huang	987f275985	fix: bound markdown chunking for large section trees (#369)	2026-07-17 17:57:56 +08:00
Xinmin Zeng	9c9b040d42	feat(plugins): add Hermes Agent memory provider (#365) ... * feat(plugins): add Hermes Agent memory provider * fix(plugins): harden Hermes memory lifecycle * fix(plugins): keep Hermes writer recoverable	2026-07-17 14:06:12 +08:00
jinliyl	c1a25e9ff4	feat(agent): add Codex agent wrapper and ReMe MCP bridge (#358) ... * feat(agent): add Codex wrapper integration * feat(agent): enhance agent wrapper functionality and add comprehensive testing - Implement structured output schema normalization across all wrappers - Add Claude Code system prompt mode support with append/replace options - Introduce Codex agent wrapper with streaming, tool context isolation, and skill management - Enhance skill linking with validation and conflict resolution - Add approval event streaming support for Codex wrapper - Implement output schema validation and normalize function - Create dedicated test suites for Claude Code and Codex integration - Update README documentation for Codex wrapper capabilities - Refactor kwargs merging with proper schema handling - Add tool context validation when resuming sessions - Implement proper cleanup and session management for Codex wrapper * test(cc-agent): add test coverage for structured output scenarios - Add docstring for empty schema validation in build_options - Document falsy structured output preservation behavior - Add docstring for streaming wrapper schema rejection - Include lambda function reference for wrapper factory consistency - Add test documentation for live Codex wrapper contract exercise * docs: revert README changes * fix(agent): interrupt abandoned Codex turns	2026-07-17 13:39:18 +08:00
jinliyl	329fd9a6a6	refactor(config): remove max_file_bytes limit from background jobs (#367) ... - Removed max_file_bytes configuration from index_update_loop, resource_watch_loop, digest_watch_loop, and reindex jobs - Updated default.yaml to reflect simplified job configurations without file size limits - Removed corresponding test case that validated the 20 MiB limit behavior - Simplified watch directories and suffixes to basic configurations - Cleaned up unnecessary commented configurations in the YAML file	2026-07-17 11:26:39 +08:00
jinliyl	2eb05392c6	chore(benchmark): remove longmemeval final answer review file (#366) ... * feat(benchmark): add final answer review step for evaluation - Introduce FinalAnswerReviewStep to handle answer validation - Add final_answer_review.jsonl dataset with 24 evaluation cases - Include detailed reasoning and golden check results for each case - Support various question types including temporal reasoning and preferences - Implement time consistency checks for session references - Add comprehensive test coverage for different evaluation scenarios * chore(benchmark): remove longmemeval final answer review file - Removed final_answer_review.jsonl containing 23 evaluation records - Deleted question_id mappings with detailed reasoning for golden answers - Removed answer correctness assessments and session time validation checks - Cleaned up benchmark dataset used for memory evaluation testing - Eliminated JSONL format evaluation results for temporal reasoning tasks - Removed references to various session IDs and time-based validations * config(default): disable shell step configuration by commenting out - Commented out the shell step configuration in default.yaml - Disabled asynchronous shell command execution capability - Removed shell step from available backend operations - Preserved traverse backend configuration unchanged * refactor(tests): remove unused shell job test from config parser tests - Removed test_default_config_registers_shell_job function that was no longer needed - Kept existing test for frontmatter chunk metadata configuration - Cleaned up test suite by removing obsolete test case	2026-07-16 20:32:28 +08:00
jinliyl	c3b1e93918	feat(index): add file size limits and oversized file handling (#362) ... * feat(index): add file size limits and oversized file handling - Implement max_file_bytes configuration option for content processing jobs - Add default 20MB file size limit for background processing in default config - Skip oversized files during auto_resource step with appropriate metadata - Clear stale index entries when oversized files are modified - Add size-based filtering logic to update_changes step with skip reporting - Include file size validation in UpdateIndexStep with proper response handling - Add comprehensive tests for oversized file scenarios in auto_resource and update_index - Document file size limits in constants with appropriate thresholds * chore(version): bump version to 0.4.1.1 - Update __version__ from 0.4.1.0 to 0.4.1.1 in __init__.py * fix(index): isolate batch metadata and handle file races	2026-07-15 21:01:18 +08:00
jinliyl	2a85c36fa9	refactor(embedding): defer provider construction until first remote call (#361) ... - Changed dimensions property to avoid forcing provider construction - Added _ensure_model method to construct provider on demand - Modified __call__ to ensure model exists before use - Updated _start to defer provider initialization - Removed eager health check during startup - Added compact embedding serialization with base64 encoding - Implemented batch processing for vector search with heap-based ranking - Added document_ids property to keyword index interface - Updated chunk persistence to handle legacy JSON embeddings - Optimized memory usage by avoiding materialization of metadata in document_ids	2026-07-15 20:46:36 +08:00
jinliyl	2e87b7a52e	feat(core): add shell execution and runtime memory status (#344) ... * feat(core): add shell command execution and memory status reporting - Introduce ShellStep for executing shell commands with timeout support - Add StatusStep to report memory estimates for stateful data components - Register shell and status commands in default configuration - Update documentation with new reme status and shell command capabilities - Implement comprehensive unit tests for both new step types - Add support for asynchronous command execution with proper error handling * feat(config): add log_config option to suppress config loading logs - Add log_config parameter to resolve_app_config function with default True - Conditionally log config loading messages based on log_config flag - Update reme.py and service_utils.py to use log_config=False for client calls - Suppress config logging in user-facing contexts to avoid output pollution refactor(shell): rename command parameter to cmd for clarity - Change 'command' to 'cmd' in default.yaml configuration schema - Rename 'timeout' to 'shell_timeout' to avoid parameter name collisions - Update ShellStep to accept both legacy and new parameter names - Maintain backward compatibility with existing command/timeout usage test(shell): add comprehensive tests for shell step parameter handling - Add test cases for new cmd and shell_timeout parameter names - Verify legacy command and timeout parameters still work - Test blank command rejection message updated to use cmd - Create integration test for shell parameter payload passing * fix(shell): ensure proper environment loading and process timeout handling - Move load_env() call to execute before parse_args() in main function - Add proper process group killing for timeout scenarios on POSIX systems - Implement recursive child process termination on Windows for proper cleanup - Change parameter name from 'timeout' to 'shell_timeout' in shell execution - Remove support for legacy 'command' and 'timeout' parameter names - Update test cases to verify new timeout behavior and parameter requirements - Add comments explaining component size tracking implementation details	2026-07-14 16:31:41 +08:00
Sen Huang	8042f74b6f	docs: add comprehensive documentation for auto-dream, auto-link, and auto-resource flows (#343)	2026-07-14 15:34:15 +08:00
jinliyl	b5e0ec2d8d	Modify budget calculation for text limit safety margin ... Adjust budget calculation to use 92% margin for token estimation.	2026-07-14 11:05:08 +08:00
jinliyl	07d4527a0d	docs(agents): update coding conventions for state persistence (#342) ... - Add guideline that steps should be stateless - Specify storing persistent state in self.app_context.metadata - Clarify avoiding state storage on step instances	2026-07-13 23:05:35 +08:00
jinliyl	bf7ca17705	feat(benchmark): add LongMemEval golden answer validation (#335) ... * feat(benchmark): add golden answer validation and session review for LongMemEval - Introduce GoldenCheckStep to validate LongMemEval golden answers using structured verdicts - Add SessionReviewStep to extract query/answer-relevant evidence from all sessions - Implement concurrent session processing with configurable concurrency limits - Create check_golden job configuration with lme_review and lme_judge agent wrappers - Add Qwen3.7-plus model configuration for enhanced processing capabilities - Include python_execute tool integration for agent-based reasoning and date validation - Generate comprehensive JSON output with session summaries and validation verdicts - Add run_check_golden.py script for batch processing across all LongMemEval samples - Configure proper logging initialization with console and file output options - Update component registry and file I/O modules to support new benchmark features * feat(scripts): add script to summarize LongMemEval check_golden verdicts - Parse check_golden.json files across all LongMemEval samples - Calculate accuracy metrics for golden answers and session IDs - Provide breakdown by question type with percentage calculations - Add command line options for listing bad samples and JSON output - Include progress tracking showing completed vs pending samples - Display confidence scores and date sanity checks statistics * refactor(benchmark): move golden check scripts to longmemeval directory - Moved run_check_golden.py from scripts/ to benchmark/longmemeval/ - Moved stats_check_golden.py from scripts/ to benchmark/longmemeval/ - Updated path resolution to use parents[2] instead of parent.parent - Added new --list-run-failed option to stats script - Added logging directory constant and functions for tracking launched samples - Enhanced stats output with launched count and run failure information - Improved error reporting with run failure details and log file paths * feat(benchmark): add LongMemEval agentic answer workflow with session extraction - Add LmeAgenticAnswerStep, LmeAutoMemoryStep, and LmeExtractSessionStep to __init__.py - Create shared helper render_with_source for displaying search results with session_id - Implement agentic_answer step with vector_search, bm25_search, and extract_session_by_id tools - Add auto_memory step to convert each session into search-friendly daily notes - Create extract_session step to retrieve and analyze raw session content by session_id - Update jinli_lme.yaml with auto_memory, vector_search, bm25_search, and agentic_answer jobs - Configure lme_memory, lme_extract, and lme_agentic_answer agent wrappers - Enhance search steps with include_source option to show session_id metadata - Add proper session_id tracking and collision handling in daily note generation * feat(benchmark): add LongMemEval agentic answer evaluation pipeline - Add session_id tracking to agentic_answer.py result metadata - Introduce run_agentic_answer.py driver for complete pipeline execution - Implement auto_memory, update_index, and agentic_answer job orchestration - Add concurrent execution with configurable limits and staggering - Create aggregation script for collecting tool-call trails and results - Add stats_agentic_answer.py for comprehensive result analysis - Implement resume capability with existing output detection - Generate aggregate.json with per-sample breakdown and tool call summaries * feat(steps): add ClearPathsStep for cleaning workspace outputs before rebuild - Introduce ClearPathsStep to remove stale workspace files/directories - Add support for specifying paths and config_keys as targets to clear - Implement safety checks to prevent deletion of files outside workspace - Add logging for cleared paths and warnings for invalid paths - Configure clear_paths_step in jinli_lme.yaml to clean daily_dir - Add clear_paths_step to clean mem_answer.json before rebuilds * feat(benchmark): add resume functionality to agentic answer runner - Replace --force flag with --resume flag for controlling job execution - By default every job reruns with clean rebuild behavior using config clear steps - Add --resume option to skip samples whose output already exists and continue interrupted batches - Update documentation to reflect new default clean rebuild behavior - Modify job skipping logic to honor resume flag instead of force flag - Update dry-run output to show correct todo jobs based on resume status - Change default example command to use --resume for continuing interrupted runs * feat(benchmark): generate JSONL output for check golden records - Add write_check_golden_list function to create JSONL file - Write all readable check_golden records as JSONL format - Include check_golden_list path in stats output - Display generated JSONL file path in summary report - Maintain UTF-8 encoding with non-ASCII character support * refactor(benchmark): rename answer judge step and integrate LME LLM judge - Rename AnswerJudgeStep to LmeLlmJudgeStep and update imports - Add new llm_judge configuration in jinli_lme.yaml - Update run_agentic_answer.py to include llm_judge in pipeline - Modify LmeLlmJudgeStep to read from query.json and answer.json - Write LLM judgement results back to mem_answer.json - Add command line options for start/end sample range selection - Update aggregate.json generation to include LLM judgement data - Add resume capability for llm_judge job based on judgement presence * refactor(benchmark): rename answer judge step and integrate LME LLM judge - Rename AnswerJudgeStep to LmeLlmJudgeStep and update imports - Add new llm_judge configuration in jinli_lme.yaml - Update run_agentic_answer.py to include llm_judge in pipeline - Modify LmeLlmJudgeStep to read from query.json and answer.json - Write LLM judgement results back to mem_answer.json - Add command line options for start/end sample range selection - Update aggregate.json generation to include LLM judgement data - Add resume capability for llm_judge job based on judgement presence * feat(steps): add wait_for_paths_step to block until workspace files exist - Introduce WaitForPathsStep class that polls for required workspace-relative paths - Add step registration with 'wait_for_paths_step' backend identifier - Implement path validation to ensure targets are within workspace boundaries - Add polling mechanism with configurable intervals via poll_seconds parameter - Include logging functionality with log_every_seconds parameter for status updates - Add metadata tracking of waited paths and duration in response object - Register step in index module and expose in public API - Configure step in jinli_lme.yaml to wait for session_review.json before golden check - Add script rename from run_check_golden.py to run_golden_check.py with enhanced options * feat(benchmark): enhance longmemeval benchmarking with concurrency and progress tracking - Add benchmark extra dependency group with portalocker requirement - Introduce concurrent execution support for golden_check and session_review workflows - Add progress reporting interval option with real-time status updates - Implement global throttling mechanism for session review requests using file locks - Enhance golden check validation with current schema verification - Add active task tracking and graceful shutdown handling - Rename check_golden scripts to golden_check for consistency - Update statistics reporting with correct/incorrect terminology instead of reasonable - Add stale format detection and compatibility handling for verdict fields - Include both_correct rate calculation in accuracy metrics - Add concurrency and staggering options for better resource management * ci(workflow): add Windows smoke test workflow - Create new workflow file .github/workflows/windows-smoke.yml - Configure workflow to trigger on push and pull request events - Set up Python environment with version 3.11 - Install package dependencies using pip - Run version job as smoke test for CLI functionality - Enable concurrency control to prevent duplicate runs - Use matrix strategy for Python version testing * feat(benchmark): add retry mechanism and health check for session review - Added retry configuration options (retry_initial_seconds, retry_max_seconds, retry_max_attempts) to jinli_lme.yaml - Implemented exponential backoff retry logic with configurable parameters in session_review step - Added output_is_healthy function to verify session_review.json integrity and absence of failed reviews - Updated resume functionality to skip only healthy outputs instead of all existing files - Integrated JSON parsing and validation to check for failed reviews in output files - Enhanced error handling and logging for retry attempts and recovery scenarios * feat(benchmark): add LongMemEval session review statistics script - Create stats_session_review.py to summarize session_review.json artifacts - Add command line options for listing failed, missing, and run failed samples - Implement JSON output mode for programmatic consumption - Calculate and display health statistics including total samples, healthy outputs, failed sessions - Provide detailed failure information with session IDs and error messages - Generate re-run commands for samples with failed reviews - Add percentage calculations for better statistical overview - Include support for multiple output formats and detailed logging * feat(benchmark): add LongMemEval output cleanup script and enhance golden check retry logic - Added clean_sample_outputs.py script to remove generated LongMemEval files while preserving source inputs - Implemented configurable retry mechanism in golden_check.py with exponential backoff strategy - Added retry parameters (initial/max seconds and max attempts) to control failure recovery behavior - Integrated asyncio support for asynchronous sleep during retry intervals - Configured default retry settings in jinli_lme.yaml with 5s initial and 300s maximum intervals - Preserved core files (query.json, answer.json, session/) while cleaning generated artifacts * feat(benchmark): add AppleDouble file cleanup to sample output cleaner - Remove AppleDouble files starting with '._' recursively including under session/ - Add is_under helper function to check if path is inside parent directory - Track targets in set to avoid duplicate processing - Include AppleDouble files in cleanup targets when not already covered by existing targets - Maintain dry-run mode as default behavior with --apply flag for actual deletion * refactor(benchmark): update LongMemEval sample output cleaning script - Add time and Iterator imports for enhanced functionality - Add --progress-every argument to control progress reporting frequency - Replace is_under function with iter_sample_targets generator - Implement detailed progress tracking with timing measurements - Add sample-by-sample processing with elapsed time reporting - Include AppleDouble file detection within session directory - Update target counting and deletion statistics display - Add conditional progress updates based on progress-every setting - Improve dry-run mode with would-delete indication * chore(benchmark): increase initial interval for session review step - Changed START_INTERVAL_SECONDS from 1.0 to 3.0 seconds - Adjusted timing parameters for better benchmark stability * refactor(benchmark): implement coordinated retry mechanism for session reviews - Add retry gate condition to coordinate concurrent review attempts - Implement wait_for_healthy_start_slot to handle sequential retries - Create mark_retrying and mark_recovered functions to track retry states - Update reply_with_retry to accept index parameter for coordination - Add has_prior_retry logic to prevent race conditions during recovery - Ensure proper cleanup of retry state on success or failure - Maintain backward compatibility while adding coordination features * chore(benchmark): adjust session review start interval timeout - Changed START_INTERVAL_SECONDS from 3.0 to 5.0 seconds - Increased initial delay for session review benchmark step - Updated timeout configuration for improved stability * refactor(benchmark): update session review concurrency and throttling mechanism - Replace global throttle with per-process concurrency control - Add concurrency parameter with default value of 30 in config - Add start_interval_seconds parameter with default value of 2 seconds - Change default concurrency from 3 to 1 in command line interface - Update documentation to reflect new throttling behavior - Implement semaphore-based concurrency limiting for review tasks - Modify retry mechanism to use local locking instead of global files - Remove portalocker dependency for cross-process throttling * refactor(config): update session review configuration and concurrency settings - Removed deprecated retry configuration parameters from jinli_lme.yaml - Increased MAX_CONCURRENCY from 30 to 60 in session_review.py - Reduced START_INTERVAL_SECONDS from 2.0 to 1.0 in session_review.py - Cleaned up redundant backend specifications in configuration file - Simplified agent wrapper configurations by removing obsolete retry settings * feat(benchmark): enhance LME auto memory step with advanced scheduling and error handling - Add datetime parsing functionality for LongMemEval timestamps with regex pattern - Implement configurable concurrency limits with MAX_CONCURRENCY of 60 - Introduce retry mechanism with exponential backoff for agent interactions - Add session filtering based on date comparison with question_date validation - Create rate limiting with start interval control between requests - Implement sophisticated retry coordination using asyncio conditions - Add comprehensive error tracking for failed and filtered session extracts - Remove deprecated concurrency parameter from jinli_lme.yaml configuration - Add structured output validation in session review step - Include detailed metadata reporting with session statistics and errors * fix(benchmark): adjust default concurrency for auto_memory job - Changed default concurrency from 3 to 1 for auto_memory job to prevent API overload - Updated help text to reflect new default value of 1 for concurrency parameter - Modified documentation to clarify concurrency behavior varies by job type * refactor(search): replace hardcoded candidate multiplier with constant - Introduced _CANDIDATE_MULTIPLIER constant set to 10 - Replaced hardcoded factor of 5 with _CANDIDATE_MULTIPLIER in BM25 search - Replaced hardcoded factor of 5 with _CANDIDATE_MULTIPLIER in vector search - Updated test to verify both search steps use ten times limit for candidates - Imported VectorSearchStep and Bm25SearchStep in test module - Added comprehensive test case for candidate count calculation logic * feat(lme): add data inspection error handling with fallback mechanism - Implemented non-retryable data inspection error markers detection - Added _is_data_inspection_error method to identify inspection failures - Created fallback handling for data inspection errors in auto memory extraction - Added fallback handling for data inspection errors in session review - Extended failed extracts tracking with non-retryable and fallback flags - Separated fallback extracts from regular failed extracts in reporting - Enhanced error logging with specific data inspection failure messages - Updated metrics to track fallback extractions and reviews separately - Maintained existing retry logic for other exception types * feat(benchmark): enhance session review statistics with fallback tracking - Add support for identifying and listing non-retryable fallback reviews - Introduce --list-fallback argument to display fallback review details - Separate retryable failures from non-retryable fallbacks in reporting - Track fallback samples and sessions separately from failed ones - Update console output to show both retryable and non-retryable categories - Include fallback details in JSON output with reasons and session info - Modify failure counting logic to distinguish between retryable and fallback reviews * feat(benchmark): add question_id tracking and enhanced fallback reporting - Add question_id function to extract query.question_id from data - Initialize question_id_by_id dictionary to store question IDs by index - Store question_id for each sample during data processing - Enhance fallback output to include question IDs and session information - Format sample labels with question IDs when available - Display session IDs associated with each fallback case * feat(benchmark): add question_id support and improve bad sample reporting - Add question_id_for function to extract question_id from multiple sources - Add sample_label function to format samples as idx(question_id) when available - Store question_id in data dictionary during processing - Change bad_golden and bad_sessions to store full records instead of just indices - Update list_bad output to show formatted labels with question_id information - Improve error reporting with more detailed sample identification * feat(benchmark): enhance golden check stats with structured output - Add related_session_ids function to extract session IDs from verdict records - Create grouped_records function to group records by question type - Replace flat list output with JSON-formatted grouped records in list_bad option - Replace flat list output with JSON-formatted grouped records in list_bad_sessions option - Maintain Chinese labels while adding structured data presentation - Improve readability of bad verdict record display with hierarchical grouping * feat(benchmark): update data structure for question indexing - Replace sample_label with _idx field for index tracking - Add question_id field to store _question_id values - Maintain backward compatibility with empty string defaults - Preserve existing session_id functionality - Update data mapping to include new fields in grouped results * refactor(benchmark): streamline golden answer verification process - Replace relevance filtering with comprehensive information extraction - Remove is_relevant field and simplify session summary structure - Change relevant_info to extracted_info for clarity - Update golden check logic to work with full extractions instead of filtered summaries - Simplify prompt instructions to focus on complete information extraction - Remove redundant schema validation and structured output requirements - Adjust statistics calculation to match new extraction approach - Update metadata field names to reflect extraction rather than relevance checking * feat(benchmark): add selective file deletion option to clean_sample_outputs - Add --filename argument to delete only specific root-level files - Modify iter_sample_targets function to accept optional filenames filter - Implement validation for root-level filename constraints - Update function calls to pass filenames parameter - Add example usage for selective file deletion in documentation * feat(benchmark): add error count metrics to golden check statistics - Added golden_bad, session_bad, and both_bad calculation fields - Updated console output format to include error counts per question type - Modified table display to show both accuracy rates and error numbers - Enhanced statistical summary with additional error breakdown metrics * test(search): update search step tests with include_source parameter - Added include_source=False parameter to VectorSearchStep initialization - Added include_source=False parameter to Bm25SearchStep initialization - Maintained existing RuntimeContext parameters for both search steps - Updated test calls to match new constructor signature with include_source option	2026-07-13 21:26:27 +08:00
jinliyl	90e7adc2d2	chore(config): disable embeddings by default and update documentation (#341) ... - Set default version to 0.4.1.0 - Comment out embedding configuration in default.yaml - Update README and README_ZH to clarify embedding components are disabled by default - Add note explaining how to enable embedding-based semantic retrieval - Adjust table formatting and descriptions in documentation - Modify search command description to reflect vector search availability when enabled	2026-07-13 21:57:54 +09:00
Sen Huang	6a2dd02e48	docs: restructure documentation and update content organization (#339) ... * docs: restructure documentation and update content organization * docs: update documentation structure and add application scenarios	2026-07-13 16:50:46 +08:00
jinliyl	b1c9bf67bf	fix(embedding): make input truncation CJK-aware (#337) ... * fix(embedding): make input truncation CJK-aware * test(embedding): cover CJK-aware truncation budget	2026-07-13 17:43:48 +09:00
Ziyang Guo	e41b1673ad	fix(search): honor min_score in plain search steps (#338)	2026-07-13 16:32:16 +08:00
Ziyang Guo	c5eefe4da3	fix(search): expose markdown frontmatter on chunks (#314) ... * fix(search): expose markdown frontmatter on chunks * style(search): apply pre-commit formatting * fix(search): make frontmatter chunk metadata opt-in * fixup! fix(search): expose markdown frontmatter on chunks * feat(markdown): add include_frontmatter_keys_in_metadata allow-list opt-in --------- Co-authored-by: RerankerGuo <1875366113@qq.com> Co-authored-by: Ziyang Guo <121015044+RunMarshal@users.noreply.github.com>	2026-07-08 17:59:55 +09:00
jinliyl	2612d25959	feat(lme): add cli execution and agentic search tooling (#334) ... * feat(service): add CLI service for local job execution - Introduce CliService to execute single jobs locally without serving ports - Add prepare_start_config and should_precheck_start functions for CLI job setup - Update reme start command to use CLI service when job argument is provided - Change default service backend from http to cli in jinli_lme config - Modify SearchStep to use constants and rename configuration parameters - Add unit tests for CLI service functionality and configuration handling - Update file extension support to include json format in addition to md and jsonl * feat(search): add BM25 and vector search steps with configuration updates - Add Bm25SearchStep and VectorSearchStep classes with tool context deduplication - Register new search step components in index module - Update configuration to use separate vector_search and bm25_search endpoints - Modify LLM models from qwen3.7-plus/glm-5.1 to glm-5.2 variants - Adjust search parameters and remove hybrid search implementation - Configure embedding store as default in storage settings - Remove auto-memory and file catalog configurations - Update watch directories from multiple paths to session_dir only * feat(agent): add tool result offloading and workspace management - Add tool_results_dir configuration option for offloaded tool results storage - Implement ToolResultOffloadMiddleware to persist large tool results to files - Create WorkspaceBackend to standardize file operations across tools - Add configurable builtin tools selection with sequential execution option - Integrate middleware support for agent wrapper with offloading capability - Update application initialization to create tool results directory - Add safety mechanisms for filesystem operations with sanitized filenames - Enhance agent wrapper with configurable working directory handling - Upgrade agentscope dependency to version 2.0.4 for improved features # Conflicts: # reme/application.py * feat(benchmark): add LongMemEval agentic search and result management - Introduce AgenticAnswerStep for agent-based history search - Add LmePrepareJudgeStep and LmeSaveResultStep for evaluation pipeline - Implement AddDraftStep and ReadAllDraftStep for evidence accumulation - Update configuration with new agent wrapper and search parameters - Add comparison script for analyzing agent run differences - Include documentation for LongMemEval failure analysis - Enhance tool result offloading with skip options - Modify search defaults and indexing behavior * feat(agent): implement tool result offloading with system reminders - Added tool_result_offload_message parameter to agent wrapper reply method - Implemented configurable reminder template for offloaded tool results - Created system reminder messages when tool results are offloaded to files - Added Chinese user message template for agentic answer step - Updated tool result offloading middleware to use custom reminder templates - Enhanced agentic answer instructions to handle long tool results via draft storage * feat(scripts): add LongMemEval results summarization tool - Create summarize_lme_results.py script to analyze result JSON files - Implement command line interface with answer id and dataset root options - Add support for specifying index range with start and end parameters - Include option to show failure details and non-successful completions - Calculate completion statistics and accuracy metrics - Display detailed breakdown of yes/no/other judgements - Handle missing and unreadable result files gracefully - Format output with percentages and comprehensive summary statistics * feat(summarize_lme_results): add question type breakdown to result summary - Import defaultdict from collections module - Add by_type dictionary to track statistics by question type - Count completed, yes, no, and other responses for each question type - Display detailed breakdown table showing accuracy by question type - Include question type column when processing judgements - Print comprehensive summary with question type distribution - Calculate and display accuracy percentage for each question type category * feat(lme): switch to qwen3.7-max model and add shuffle functionality - Changed default LLM model from glm-5.1 to qwen3.7-max in jinli_lme.yaml - Added random module import for shuffle functionality - Implemented --shuffle argument with BooleanOptionalAction for dataset shuffling - Added --seed argument to control random seed for reproducible shuffling - Applied random shuffle to dataset indices when shuffle is enabled - Added console output showing shuffle operation and seed information * fix(cli): set default random seed for shuffle functionality - Changed default seed value from None to 42 for consistent shuffling behavior - Ensures reproducible results when using shuffle option without explicit seed - Maintains backward compatibility while providing deterministic defaults * refactor(benchmark): update agentic answer guidelines for grounding - Updated English instruction to emphasize strict grounding in retrieved context - Modified Chinese instruction to stress evidence-based responses without inference - Removed redundant conciseness requirement in both language versions - Enhanced clarity on proper use of draft saving and retrieval mechanisms - Strengthened emphasis against hallucination of unsupported facts * refactor(benchmark): update agentic search instructions and configuration - Replace separate vector_search and bm25_search with unified search tool - Update agent instructions to use single search tool with multiple strategies - Simplify Chinese instructions for search methodology - Add comprehensive search tool configuration with hybrid vector/BM25 capabilities - Increase model retry attempts from 1 to 3 for better reliability - Remove redundant tool references from job_tools list * feat(search): add configurable search limit with environment variable support - Remove hardcoded limit and min_score parameters from config schema - Increase LLM context size from 200000 to 1000000 - Add REME_SEARCH_LIMIT environment variable support for search configuration - Implement command line argument --search-limit to override default search limit - Add input validation to ensure search limit is positive - Modify subprocess execution to pass environment variables - Update search step to use dynamic default limit from environment or fallback to 5 * refactor(benchmark): remove agentic answer step and related configurations - Removed AgenticAnswerStep class and its registration - Deleted agentic_answer.yaml prompt configuration file - Removed agentic answer related job definitions from jinli_lme.yaml - Cleaned up tool result offloading middleware implementation - Removed tool_results_dir configuration field from application config - Deleted comparison and analysis scripts for agent runs - Removed agentic answer step from LME init module exports - Updated agent wrapper to remove tool result offloading functionality - Removed unused imports and dependencies in agent wrapper module * refactor(benchmark): remove unused LME result processing components - Removed LmePrepareJudgeStep and LmeSaveResultStep classes from benchmark module - Cleaned up imports and exports in lme module initialization - Removed unused middleware configuration from agent wrapper - Deleted obsolete result.py file containing deprecated result processing logic - Simplified agent instantiation by removing middleware parameter - Updated import statements to reflect removed dependencies * refactor(index): remove unused search steps and update imports - Remove Bm25SearchStep and VectorSearchStep from index steps module - Remove unused prepare_start_config and should_precheck_start exports - Move import statements to proper location in reme.py - Update test module to use direct import path for CliService - Remove vector_search and bm25_search configurations from jinli_lme.yaml - Add workspace directory environment variable configuration - Add docstring to getcwd method in agent wrapper - Remove empty middleware list from agent wrapper initialization * feat(index): add BM25 and vector search steps with tool context deduplication - Add Bm25SearchStep for plain BM25 keyword search with tool_context deduplication - Add VectorSearchStep for plain vector search with tool_context deduplication - Implement tool context state management with TTL-based deduplication - Add support for chunk deduplication across tool contexts within TTL window - Update index steps module to include new search step classes - Add test coverage for CLI metadata output functionality - Refactor CLI service to remove unused show_status parameter - Update documentation comments to reflect internal service configuration * feat(steps): add Python code execution capability - Introduce PythonExecuteStep to run Python code in subprocess - Add configuration for python_execute step in jinli_lme.yaml - Register python_execute in available tools list - Implement timeout handling with default 60 second limit - Capture stdout/stderr output and return code metadata - Add comprehensive unit tests for execution scenarios - Support workspace directory context for code execution - Handle timeout errors and runtime exceptions gracefully * refactor(python_execute): replace subprocess with asyncio for Python code execution - Replace subprocess.run with asyncio.create_subprocess_exec for non-blocking execution - Add _PythonResult dataclass to encapsulate execution results and timeout status - Implement proper timeout handling with asyncio.wait_for and process.kill() - Update metadata to include returncode and stderr when timeout occurs - Convert synchronous _run_python method to asynchronous implementation - Maintain backward compatibility while improving execution reliability * refactor(python_execute): replace subprocess with asyncio for Python code execution - Replace subprocess.run with asyncio.create_subprocess_exec for non-blocking execution - Add _PythonResult dataclass to encapsulate execution results and timeout status - Implement proper timeout handling with asyncio.wait_for and process.kill() - Update metadata to include returncode and stderr when timeout occurs - Convert synchronous _run_python method to asynchronous implementation - Maintain backward compatibility while improving execution reliability	2026-07-08 17:43:09 +09:00
xyf2020	82971ac5b0	feat(chunker): better json chunker and json chunker (#325) ... * feat(file_chunker): add dedicated JSON and JSONL file chunkers - Add JsonFileChunker: structure-aware chunking preserving nested key paths, optional list-to-dict conversion, size measured by json.dumps() char count - Add JsonlFileChunker: line-aligned sliding-window chunking with configurable overlap, supports char/byte mode switching - Register both chunkers in default.yaml (json for .json, jsonl for .jsonl) - Add comprehensive unit tests (21 + 20 test cases) * chore(config): update default chunker supported_extensions to txt/log * refactor(json_chunker): optimize _build_tree O(n²) serialization and rewrite tests - Fix O(n²) redundant json.dumps in _build_tree: * Empty containers handled directly as leaves (0 serialization) * Non-empty containers recurse first, then reconstruct+dump once * Only containers that become leaves pay serialization cost - Add _reconstruct_object/_reconstruct_array helpers - Remove dead code: _merge_json method - Apply user changes: min_element_size formula 0.01->0.05, threshold < to <= - Use indent=None for compact output (consistent with _SizeNode estimation) - Remove unused _text_size from JsonlFileChunker Test rewrite: - Replace try/finally boilerplate with make_json fixture - Group tests into TestXxx classes with pytest.mark.parametrize - Add TestOutputValidation: 9 parametrized scenarios verifying: * All chunks are valid JSON * Text length <= chunk_chars (with single-leaf tolerance) * Leaf-value concatenation matches original data (dict + array roots) - Add TestSizeNode: incremental size accuracy tests - Add TestDfsAlgorithm: path wrapping, DFS order, calibration tests - Update test_min_element_size_formula for new 0.05 multiplier - Update test_build_tree_structure for larger min_element_size * chore: apply black formatting to test files	2026-07-08 15:18:59 +08:00
jinliyl	41c6cdaff5	Bump version to 0.4.0.9	2026-07-08 14:20:54 +08:00
jinliyl	b53d3db8d0	chore(workflow): update package installation to include core extra de… (#331) ... * chore(workflow): update package installation to include core extra dependencies - Modified pre-commit workflow to install with [dev,core] extras - Updated python-publish workflow to install wheel with core extra dependency - Changed from direct dist/*.whl install to variable assignment for wheel path - Ensured core dependencies are included during test installation phase * chore(workflow): remove docs deployment workflow - Delete the entire docs.yml workflow file that was used for deploying documentation - Remove all related configuration including build and deploy jobs - Stop automatic deployment of docs on pushes to main branch - Remove GitHub Actions workflow for docs/ directory changes	2026-07-08 15:03:25 +09:00
jinliyl	eb471d7d94	fix(embedding): reject mismatched embedding dimensions (#330) ... * fix(embedding): enforce strict dimension matching for embeddings - Add _embedding_dim_matches method to validate embedding dimensions - Reject embeddings with mismatched dimensions instead of padding/truncating - Drop stale embeddings with wrong dimensions during loading and upsert operations - Disable embedding store when query dimensions don't match configured dimensions - Fail health checks when embedding dimensions don't match expected values - Skip chunks with wrong dimensions during FAISS index rebuild - Add comprehensive tests for dimension validation behavior * refactor(file_store): simplify conditional checks in vector search and test assertions - Combine multiple conditionals into single check for empty FAISS index - Replace explicit empty list comparison with boolean check for node embedding calls - Maintain same functional behavior while improving code readability * fix(embedding): harden dimension validation helpers	2026-07-08 14:23:38 +09:00
jinliyl	38cf16071b	refactor(embedding): update embedding model initialization and session storage paths (#329) ... * refactor(embedding): update embedding model initialization and session storage paths - Remove unused inspect import from as_embedding module - Pass dimensions directly to embedding model constructor instead of using parameters - Update session state file paths to use mem_session directory instead of resource - Add mem_session_dir configuration option to application config schema - Update workspace directory creation to include new mem_session directory - Change AgentScope and Claude Code session paths to use mem_session directory - Move embedding dimensions from parameters to top-level configuration - Update AgentScope dependency version from 2.0.3 to 2.0.4 - Update integration tests to reflect new session file location paths * chore(version): bump version to 0.4.0.8 - Update __version__ from 0.4.0.7 to 0.4.0.8 in __init__.py	2026-07-08 12:30:46 +09:00
jinliyl	10da205797	feat(benchmark): add lme benchmark steps (#326) ... * refactor(search): replace time module with datetime for timestamp generation - Removed unused time import - Added static method _now_ts using datetime.timestamp - Updated clock parameter to use _now_ts method instead of time.time - Maintained same timestamp precision and functionality * test(http): add tests for HTTP client display formatting - Add test for default metadata hiding behavior in CLI output - Add test for metadata display when show_metadata is enabled - Verify _format_for_display method correctly formats response text - Test both success case and metadata inclusion scenarios * chore(build): remove longmemeval from gitignore - Removed longmemeval directory from gitignore list - Kept evaluation and datasets directories in ignore list - Updated gitignore configuration for proper version control	2026-07-07 18:53:39 +09:00
jinliyl	bf902b3479	Bump version to 0.4.0.7	2026-07-06 19:04:46 +08:00
Sen Huang	0a7eea18f8	fix(as_embedding): support both agentscope 2.0.2 and 2.0.3 (#323) ... 2.0.3 promoted `dimensions` to a required first-class constructor argument while keeping a backfill from `parameters.dimensions`; 2.0.2 has no such argument and reads `dimensions` from `Parameters`. Keep `dimensions` in `Parameters` for both versions and, when the model constructor accepts `dimensions`, pass `dimensions=None` so 2.0.3's backfill promotes it out of `parameters`. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-06 19:03:32 +08:00
imrewce	1e798d3b4e	fix(file_io): fix risk of out-workspace paths (#322) ... * fix(file_io): fix risk of out-workspace paths * chore(file_io): remove unused unittest file	2026-07-06 15:25:21 +08:00
jinliyl	7369342115	feat(search): add tool context deduplication and improve search configuration (#321) ... * feat(search): add tool context deduplication and improve search configuration - Modify _make_tool methods to accept and inject tool_context_id parameter - Add tool_context_id handling in AS and CC agent wrappers - Increase search candidate multiplier from 3.0 to 5.0 in default config - Extend HTTP client timeout from 30s to 3600s - Add tool context deduplication logic to prevent duplicate search results - Implement TTL-based expiration for seen chunks in tool contexts - Add comprehensive unit tests for tool context deduplication behavior - Update .gitignore to exclude longmemeval directory - Add time import for timestamp functionality in search step * refactor(search): replace time module with datetime for timestamp generation - Removed unused time import - Added static method _now_ts using datetime.timestamp - Updated clock parameter to use _now_ts method instead of time.time - Maintained same timestamp precision and functionality	2026-07-06 16:18:39 +09:00
xyf2020	43a407bc4f	feat: add start_date/end_date time filter support for search job (#317) ... * feat: add start_date/end_date time filter support for search job - Add _extract_date_from_path to extract validated YYYY-MM-DD from chunk paths - Add start_date/end_date filtering in _matches_search_filter - Implement progressive recall in FaissLocalFileStore.vector_search - Promote start_date/end_date from context to search_filter in SearchStep - Add start_date/end_date parameters to search job in default.yaml - Add unit tests for date filter functionality * fix: validate/normalize date filters and harden _extract_date_from_path Address three code-review comments on the time_filter search feature: 1. Validate/normalize start_date and end_date before string comparison. _matches_search_filter does lexicographic comparison against path_date (always canonical YYYY-MM-DD). Raw caller values like '2026-2-28' or 'abc' would produce silently wrong results. Now SearchStep normalizes valid dates via extract_daily_date (with strptime fallback for non-zero-padded input) and silently ignores invalid dates with a logger.warning, removing them from the filter. 2. Clarify behavior for paths without embedded dates. Added optional strict_date_filter parameter (default False). When True and at least one date bound is active, chunks whose path yields no date (e.g. digest/personal/topic.md) are excluded. When False (default), the existing behavior is preserved — dateless paths pass through. 3. Harden _extract_date_from_path against non-standard suffixes. Previously parts[1].split('.')[0] accepted '2026-05-18.anything' as a valid date. Now only exact 'YYYY-MM-DD' (dir) and 'YYYY-MM-DD.md' (day-index) forms are accepted.	2026-07-03 15:58:04 +08:00
Zhaoyang Liu	f63165c66b	update the readme, reorg the content (#318)	2026-07-03 14:56:52 +08:00
jinliyl	6bf2db8ff4	Update agentscope dependency version to 2.0.3	2026-07-02 14:43:18 +08:00
Sen Huang	8877743ca9	feat(cli): route bare commands to the running server's real config (#312) ... call_server now resolves the client backend/transport/host/port from the live `reme start` process (replaying its start args through resolve_app_config) so a bare `reme <action>` reaches the server however it was actually launched, falling back to local config when none runs. Explicit backend=/transport=/host=/port= still win. Also fix as_embedding to pass `dimensions` explicitly for agentscope >=2.0.2, and add Claude Code auto-memory/auto-dream demos to the READMEs.	2026-07-01 17:15:48 +08:00
Ziyang Guo	c060933e4d	fix(auto-memory): preserve message timestamps (#310) ... * fix(auto-memory): preserve message timestamps * fix(auto-memory): infer daily date from messages * feat(file_io): add strict date parsing and improve daily date handling - Add new parse_daily_date function for strict YYYY-MM-DD validation - Replace extract_daily_date with parse_daily_date for explicit date validation - Change _messages_day to use max date instead of min for historical imports - Reorder imports to maintain consistent module ordering - Move session message saving after date validation in auto_memory - Add comprehensive tests for invalid date rejection before saving - Add tests for strict YYYY-MM-DD date format validation - Update test names to reflect latest date behavior --------- Co-authored-by: Ziyang Guo <121015044+RunMarshal@users.noreply.github.com> Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>	2026-07-01 17:15:07 +08:00
Sen Huang	5a3450ddb3	chore(release): bump version to 0.4.0.6 (#309)	2026-07-01 14:10:37 +08:00
Sen Huang	435aa713a2	fix(config): correct indentation in default.yaml (#308)	2026-07-01 12:13:21 +08:00
Ziyang Guo	9d14e988d8	docs(framework): clarify context management boundary (#306) ... Co-authored-by: Ziyang Guo <121015044+RunMarshal@users.noreply.github.com>	2026-07-01 11:17:00 +08:00
jinliyl	1c05d0359b	feat(README): Enhance documentation styling, content, and layout (#304) ... * style(docs): update visual styling and layout of documentation figures - Change background color from #f7f8fb to #fffdf8 - Update fonts to include Comic Sans MS and Bradley Hand for titles - Adjust stroke colors and widths for panels and chips - Modify arrow and line styles with new colors and dimensions - Update marker sizes and colors for better visual consistency - Add rounded corners and join styles for smoother appearance - Apply dashed borders to chip elements - Refine color palette for text and UI elements docs(readme): enhance documentation content and formatting - Improve readability with better line breaks and spacing - Update core ideas section with expanded descriptions - Add news section announcing ACL 2026 paper acceptance - Enhance agent integration section with detailed examples - Revise automatic memory flow description for clarity - Update workspace operation interface with improved categorization - Standardize table formatting and column widths - Clarify directory structure with better organization - Add minimal CLI examples for easier integration - rename session_id to session_event in directory structure - add example files under digest directory structure * style(docs): adjust image dimensions in README table - Changed table cell widths from 45% to 50% for better alignment - Reduced image width from 100% to 92% to prevent overflow - Applied consistent sizing across all four documentation images - Improved visual balance of the feature comparison table * docs(readme): update documentation with design philosophy and operation interface - Change background color in design-philosophy.svg from #f7f8fb to #ffffff - Rename 'Workspace Operation Interface' to 'ReMe Operations' in README.md - Update Chinese documentation with consistent 'ReMe Operations' title - Adjust image widths from 100% to 92% in Chinese documentation tables - Standardize summary titles in both English and Chinese documentation * docs(readme): add demo videos for auto memory and auto dream features - Added expandable details section with video demonstrations - Included side-by-side comparison of Auto Memory and Auto Dream features - Added video controls with muted loop and inline playback support - Updated both English and Chinese README files with identical content - Used table layout for proper alignment of demonstration videos - Maintained consistent styling and formatting across both language versions * docs(figure): remove qwenpaw auto memory video file - Delete the video file qwenpaw-auto-memory.mp4 from docs/figure directory - Remove all video content related to auto memory demonstration - Clean up media assets that are no longer needed in documentation * style(docs): replace details summary with centered paragraph in README files - Replaced collapsible details/summary elements with centered paragraphs - Removed unnecessary br tags in both English and Chinese documentation - Maintained the same visual presentation while simplifying HTML structure - Updated both README.md and README_ZH.md consistently * style(docs): update design philosophy diagram styling - Changed fonts to include Comic Sans MS and Bradley Hand for titles and labels - Updated color scheme with darker text colors (#1f2430 instead of #172033) - Increased stroke widths from 1.2 to 2.2 for panels and adjusted other stroke values - Added rounded line caps and joins for smoother visual appearance - Modified chip styling with dashed borders and updated stroke properties - Adjusted arrow markers to smaller sizes with updated dimensions - Refined color values for arrows, links and file lines for better contrast - Applied consistent stroke properties across all visual elements * docs(readme): update documentation and adjust svg dimensions - Updated SVG canvas dimensions from 640px to 670px height - Simplified Skill + CLI integration examples in README tables - Removed detailed command examples and collapsible sections - Streamlined automatic memory capabilities documentation - Cleaned up ReMe operations table formatting - Consolidated command usage instructions for clarity	2026-06-30 19:52:20 +08:00
jinliyl	6244e7eeaa	feat(README): Enhance documentation styling, content, and layout (#303) ... * style(docs): update visual styling and layout of documentation figures - Change background color from #f7f8fb to #fffdf8 - Update fonts to include Comic Sans MS and Bradley Hand for titles - Adjust stroke colors and widths for panels and chips - Modify arrow and line styles with new colors and dimensions - Update marker sizes and colors for better visual consistency - Add rounded corners and join styles for smoother appearance - Apply dashed borders to chip elements - Refine color palette for text and UI elements docs(readme): enhance documentation content and formatting - Improve readability with better line breaks and spacing - Update core ideas section with expanded descriptions - Add news section announcing ACL 2026 paper acceptance - Enhance agent integration section with detailed examples - Revise automatic memory flow description for clarity - Update workspace operation interface with improved categorization - Standardize table formatting and column widths - Clarify directory structure with better organization - Add minimal CLI examples for easier integration - rename session_id to session_event in directory structure - add example files under digest directory structure * style(docs): adjust image dimensions in README table - Changed table cell widths from 45% to 50% for better alignment - Reduced image width from 100% to 92% to prevent overflow - Applied consistent sizing across all four documentation images - Improved visual balance of the feature comparison table * docs(readme): update documentation with design philosophy and operation interface - Change background color in design-philosophy.svg from #f7f8fb to #ffffff - Rename 'Workspace Operation Interface' to 'ReMe Operations' in README.md - Update Chinese documentation with consistent 'ReMe Operations' title - Adjust image widths from 100% to 92% in Chinese documentation tables - Standardize summary titles in both English and Chinese documentation * docs(readme): add demo videos for auto memory and auto dream features - Added expandable details section with video demonstrations - Included side-by-side comparison of Auto Memory and Auto Dream features - Added video controls with muted loop and inline playback support - Updated both English and Chinese README files with identical content - Used table layout for proper alignment of demonstration videos - Maintained consistent styling and formatting across both language versions * docs(figure): remove qwenpaw auto memory video file - Delete the video file qwenpaw-auto-memory.mp4 from docs/figure directory - Remove all video content related to auto memory demonstration - Clean up media assets that are no longer needed in documentation * style(docs): replace details summary with centered paragraph in README files - Replaced collapsible details/summary elements with centered paragraphs - Removed unnecessary br tags in both English and Chinese documentation - Maintained the same visual presentation while simplifying HTML structure - Updated both README.md and README_ZH.md consistently * style(docs): update design philosophy diagram styling - Changed fonts to include Comic Sans MS and Bradley Hand for titles and labels - Updated color scheme with darker text colors (#1f2430 instead of #172033) - Increased stroke widths from 1.2 to 2.2 for panels and adjusted other stroke values - Added rounded line caps and joins for smoother visual appearance - Modified chip styling with dashed borders and updated stroke properties - Adjusted arrow markers to smaller sizes with updated dimensions - Refined color values for arrows, links and file lines for better contrast - Applied consistent stroke properties across all visual elements	2026-06-30 19:33:49 +08:00
Sen Huang	e7ef2c8ce6	feat(docs): add multilingual documentation with GitHub Pages deployment (#287) ... * feat(docs): add multilingual documentation with GitHub Pages deployment * docs(readme): update agent integration documentation with current status	2026-06-29 15:58:23 +08:00
Sen Huang	be7d1c0cf2	refactor(transfer): drop orphaned ingest step, make service discovery cross-platform (#300) ... * refactor(transfer): drop orphaned ingest step, make service discovery cross-platform - Remove ingest step: superseded by auto_resource (drop files under resource/ → watcher interprets them); its meta.json/<date>.md outputs had no consumers and tripped the auto_resource watcher. - Replace lsof/pgrep shell-outs in service_utils with psutil (per-process enumeration, no root needed on macOS) for Windows/macOS/Linux support. - Add cross-platform test coverage for _pid_on_port / _scan_reme_procs. - Deps: +psutil, -filelock (only used by the removed ingest lock). * chore(release): bump version to 0.4.0.5	2026-06-29 14:49:23 +08:00
Sen Huang	3dee10d4f9	feat: add Claude Code plugin with auto-memory functionality (#297) ... * feat: add Claude Code plugin with auto-memory functionality * refactor(auto_memory): fix spacing in json parsing logic	2026-06-26 14:46:42 +08:00
jinliyl	ad7893e9c4	fix(mcp): resolve circular import issues and update dependencies (#296) ... * fix(mcp): resolve circular import issues and update dependencies - Moved fastmcp imports inside functions to prevent circular dependencies - Replaced _TRANSPORT_MAP with _VALID_TRANSPORTS set for transport validation - Updated version number from 0.4.0.3 to 0.4.0.4 - Added claude-agent-sdk dependency to core optional dependencies - Used TYPE_CHECKING imports for FastMCP related types - Restructured transport mapping logic within function scope - Fixed string annotation for CallToolResult type hints * refactor(tests): update date handling in daily steps tests - Replace _date.today() with timezone-aware now function - Use Asia/Shanghai timezone for date formatting - Change return format to use strftime instead of isoformat - Import now function from reme.steps.evolve module * refactor(tests): clean up unused imports in daily steps test - Removed unused date import from datetime module - Removed redundant pathlib Path import that was already imported later - Kept necessary imports for asyncio, os, tempfile, warnings, and frontmatter modules * test(daily_steps): update test to include application context for daily list step - Add ApplicationContext initialization with temporary workspace directory - Register file store component in application context - Pass application context to DailyListStep constructor - Maintain existing test assertion behavior for date metadata verification	2026-06-26 09:37:08 +08:00