Commit graph

11 commits

Author SHA1 Message Date
xyf2020
f04eedb3ab
feat(Tag filter): Add an optional rebuildable tag index for file frontmatter (#517)
Some checks are pending
CI / Documentation / Test and build documentation (push) Waiting to run
CI / Python packages / Build and verify distributions (push) Waiting to run
CI / Python quality / Pre-commit (push) Waiting to run
CI / Python tests / Unit Tests - py3.11 (push) Waiting to run
CI / Python tests / Unit Tests - py3.12 (push) Waiting to run
CI / Python tests / Unit Tests - py3.13 (push) Waiting to run
CI / ReMe Studio / Studio checks (push) Waiting to run
CI / Windows / CLI smoke - py3.11 (push) Waiting to run
Deploy / Documentation / Build documentation (push) Waiting to run
Deploy / Documentation / deploy (push) Blocked by required conditions
Security / CodeQL / Analyze javascript-typescript (push) Waiting to run
Security / CodeQL / Analyze python (push) Waiting to run
* Add optional tag generation and normalization to auto memory

* Add tag index components and clean up temporary JSONL files

* Preserve tag index state when reconciliation fails

* Refactor and streamline application implementation

* Fix pylint C1803 warnings in tag normalization tests

* Document optional tag index configuration

* Make tag index failures non-blocking and disable auto-memory tags

* fix(tag-index): fail closed and support reindexing

* fix(tag-index): preserve complete query expressions

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-09-07 15:52:17 +08:00
jinliyl
f5ec230fef
feat: add DSH memory integration and organize extensions (#461)
* feat: add DSH memory integration and organize extensions

* fix: support newer DSH release candidates

* fix: address DSH integration review feedback

* fix: handle DSH cross-day retry edge cases
2026-08-20 15:31:51 +08:00
jinliyl
9218a2d0e3
refactor: derive dialog paths from session_dir (#421)
* refactor: derive dialog paths from session directory

* fix: normalize configured session paths

* fix: align dialog watch paths with writers

* fix: reject absolute session directories
2026-08-06 17:07:14 +08:00
xyf2020
5bc46c88b6
feat(benchmark): enhance session memory retrieval and isolate benchmark assets (#409)
* chore(benchmark): isolate dataset/workspaces/results per benchmark

- Move shared benchmark/{datasets,memory_workspaces,results} into per-benchmark subdirs benchmark/<name>/{dataset,workspaces,results}
- Update beam/longmemeval config.yaml and run.py path defaults
- Relocate longmemeval download.py to benchmark/longmemeval/ (downloads into dataset/ subdir); inline dataset download docs into README
- Update .gitignore: benchmark/*/{dataset,workspaces,results}/
- Move result-{beam,longmemeval}.md to benchmark/results_md/ and drop result- prefix; update README links
- Fix stale path refs in llm_judge.py and logs/demo_search_format.py

* feat(benchmark): add read tool to agentic answer and update BEAM results

- Add 'read' to job_tools in BaseAgenticAnswerStep for file reading capability
- Document read tool usage in lme/agentic_answer.yaml system prompt
- Update result-beam.md with latest evaluation scores (OVERALL: 0.623/0.580)

* feat(auto_memory): add source line-number markers for note traceability

- Add _format_history hook in AutoMemoryStep with line-number annotation
- Override in BeamAutoMemoryStep to prefix each turn with [Ln] for citation
- Add session_file variable to prompt templates for source marker paths
- Simplify repeated extraction rules by referencing system prompt
- Enhance agentic_answer search strategy (multi-search, read tool hint)
- Add warning log on ReadStep failure

* feat(beam): enhance auto_memory with source markers and pilot ingest tooling

* refactor(beam): rename max_chunk_words to max_segment_words, drop one-off pilot scripts

* feat: add CompressorStep and search_v2 dual-mode session compression

- Add CompressorStep (reme/steps/evolve/compressor.py) for direct LLM
  text compression with optional query-guided relevance filtering
- Extend search_v2_step to support query-aware and query-independent
  session transcript compression via _compress injected kwargs
- Refactor _source_format.py: split into render_chunk_entries +
  join_chunk_entries; session chunks now render line-aligned with
  L<n>: prefixes for verbatim/compressed parity
- Add JOB_TOOLS and INJECTED_JOB_KWARGS to BaseAgenticAnswerStep for
  per-subclass tool and parameter injection
- LmeAgenticAnswerStep injects _search._compress payload to enable
  query-aware compression during benchmark evaluation
- Record compression ablation results in result-longmemeval.md
- Add unit tests for CompressorStep and search compression paths

* refactor(compress): relax session compression to lenient format-preserving strategy and update LME results

* refactor(benchmark): make session compression config-driven via compress_session flag

Move session-transcript compression from LME hard-coded injection to a
runtime context flag set by evaluation.compress_session in each
benchmark config. Compression is off by default for both BEAM and LME,
and BaseAgenticAnswerStep now conditionally injects the _search compress
payload only when the flag is truthy.

* feat(lme/auto_memory): add source attribution markers with line numbers

Add _format_history to annotate each turn with [Ln] line numbers and
expose {session_file} in prompts so the agent can emit bare wikilink-style
source markers like [[session/dialog/s1.jsonl#L1-L2,L5-L6]] at the end
of factual entries. Consolidate the per-prompt body/format rules into
references to the system prompt to avoid drift, and add frontmatter-
protection guidance for the edit tool.

* feat: improve agentic answer prompt and update beam 100K results

- Strengthen abstention rule: prohibit extrapolation from related but
  non-direct evidence
- Add multi-angle search after preliminary answer to check for
  conflicting/supplementary/updated information
- Add max-iteration fallback to 'Information not found'
- Update beam.md with 100K results (agentscope 2.0.4.post1, from scratch)
  including per-type token consumption and memory construction stats
- config.yaml: 100K dataset, 20 workers for BEAM evaluation
- run.py: add memory construction token usage tracking (default agent)
- Overall: 0.635 → 0.654 (+0.019), contradiction_resolution: 0.338 → 0.478
  (+0.140), abstention: 0.500 → 0.525 (+0.025)

* feat(read): add session-aware formatting for read tool and update BEAM eval

- Add truncate_session_output in _file_io.py to render jsonl session
  lines as [speaker @ time] content before byte-budget truncation
- Add read_step_format_session flag to ReadStep, honoring injected
  job kwargs (precedence) and YAML fallback
- Inject read_step_format_session=True into BaseAgenticAnswerStep
  so agentic answer reads render session transcripts human-readably
- Refine BEAM agentic_answer prompt: continue multi-angle search
  after preliminary answer, forbid fabrication/extrapolation
- Update BEAM config to 1M variant and add sequential 100K-eval /
  1M-build shell script
- Refresh benchmark/results_md/beam.md with latest results

* chore(config): disable expand_links in beam and lme search_v2 configs

* refactor(beam): drop one-off sequential 100K-eval-then-1M-build script

* fix(benchmark): add compressor job to beam config and fix BEAM clone instructions

- Add compressor job and compressor as_llm component to reme/config/beam.yaml
  (aligned with lme.yaml) so that compress_session: true works for BEAM
- Add graceful degradation guard in search_v2._compress_session_entries:
  when the compressor job is missing from the active config, log a warning
  and skip compression instead of raising 'Job compressor not found'.
  Skipped when there is no app_context so unit tests mocking run_job still
  drive compression behavior.
- Fix BEAM download instructions in README.md/README_ZH.md: add mkdir -p
  before cd benchmark/beam/dataset (the directory is gitignored and absent
  in a fresh clone)

* fix(steps): guard compressor exceptions and fix ReadStep boolean override

1. search_v2: catch per-entry exceptions from run_job('compressor') inside
   compress() so asyncio.gather never propagates a compressor failure (e.g.
   temporary LLM outage). The failing entry keeps its original body while
   remaining entries are still compressed, preserving already-retrieved
   search results.

2. read: replace 'context_value or yaml_value' with an existence check so
   that a runtime-injected False can explicitly disable a YAML-true
   read_step_format_session flag.

Add focused unit tests for both paths.

* fix(search_v2): use existence check for strict_date_filter boolean override

Replace 'context_value or yaml_value' with an existence-based check so
that a runtime-injected False can explicitly disable a YAML-true
strict_date_filter flag, consistent with the read_step_format_session fix.

* refactor(search): simplify strict_date_filter fallback to truthiness-or

* style(test): rename unused param to satisfy pylint W0613

---------

Co-authored-by: sa-buc <jiangniurou.xyf@dail-algo011164204033.ET135>
2026-08-05 19:23:42 +08:00
xyf2020
f34dcdb09b
feat(Step tools): add white/black path prefix permission filtering to read, edit, write (#391)
* feat(read): add white/black path prefix permission filtering to ReadStep

* feat: add PrefixCheck mixin for path-prefix permission in file I/O steps

* feat: add injected_job_kwargs mechanism and refine path-prefix permission

* refactor(file_io): consolidate prefix_check into _path module
2026-07-27 17:20:21 +08:00
xyf2020
7b1da5a9ee
feat(benchmark): add BEAM & restructure LongMemEval evaluation framework (#375)
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
Windows Smoke / CLI smoke - py3.11 (push) Waiting to run
* feat(eval): add LongMemEval evaluation framework with tool_defaults date injection

- Add evaluation/longmemeval/ with run.py, config.yaml, and test scripts
- Add reme/config/longmemeval.yaml for evaluation-specific model config
- Add tool_defaults mechanism to as_agent_wrapper for injecting default
  tool kwargs (uses setdefault so LLM-provided values take priority)
- Pass tool_defaults={'daily_write': {'date': day}} in auto_memory to
  ensure notes always use the correct historical date
- Add timestamp interpolation (_interpolate_timestamps) in auto_memory
  for filling missing created_at fields via linear interpolation
- Evaluation pipeline: ingest sessions -> dream -> search -> answer -> judge
- Uses qwen3.6-flash for memory, qwen3.7-max for answer/judge

* chore: gitignore logs/results/demo.py, keep empty dirs

* chore: update .gitignore

* feat(eval): add multiprocessing and session time filtering to longmemeval runner

- Replace async execution with synchronous + multiprocessing for parallel item evaluation - Add filter_future_sessions option to only ingest sessions <= question date - Add question_types filtering in config - Add result summary with binary accuracy and avg score - Update config defaults (oracle variant, 50 items, 32 workers) - Minor code style fixes in agent_wrapper and auto_memory

* feat: add bench_query_step with ReAct agent for benchmark query phase

- Add BenchQueryStep using agent_wrapper with search job tool
- Replace manual search+LLM answer in run.py with bench_query_job
- Remove unused answer LLM config from longmemeval.yaml
- Register benchmark step module in steps/__init__.py

* feat: add start_date/end_date time filter support for search job

- Add _extract_date_from_path to extract validated YYYY-MM-DD from chunk paths
- Add start_date/end_date filtering in _matches_search_filter
- Implement progressive recall in FaissLocalFileStore.vector_search
- Promote start_date/end_date from context to search_filter in SearchStep
- Add start_date/end_date parameters to search job in default.yaml
- Add unit tests for date filter functionality

* fix: validate/normalize date filters and harden _extract_date_from_path

Address three code-review comments on the time_filter search feature:

1. Validate/normalize start_date and end_date before string comparison.
   _matches_search_filter does lexicographic comparison against path_date
   (always canonical YYYY-MM-DD). Raw caller values like '2026-2-28' or
   'abc' would produce silently wrong results. Now SearchStep normalizes
   valid dates via extract_daily_date (with strptime fallback for
   non-zero-padded input) and silently ignores invalid dates with a
   logger.warning, removing them from the filter.

2. Clarify behavior for paths without embedded dates.
   Added optional strict_date_filter parameter (default False). When True
   and at least one date bound is active, chunks whose path yields no date
   (e.g. digest/personal/topic.md) are excluded. When False (default),
   the existing behavior is preserved — dateless paths pass through.

3. Harden _extract_date_from_path against non-standard suffixes.
   Previously parts[1].split('.')[0] accepted '2026-05-18.anything' as a
   valid date. Now only exact 'YYYY-MM-DD' (dir) and 'YYYY-MM-DD.md'
   (day-index) forms are accepted.

* feat(eval): LLM-as-Judge per-type prompt routing, binary-only, progress tracking

- Remove 0-5 score metric, keep only binary (yes/no) classification
- Load per-question-type judge prompts from llm-as-judge.json
  (temporal-reasoning, knowledge-update, single-session-preference, __default__)
- Replace SCORE_JUDGE_PROMPT with type-specific BINARY_JUDGE_PROMPT template
- judge_response(): parameter 'metric' -> 'question_type', returns single 'judgment'
- Summary output: add per-type accuracy breakdown, remove score stats
- Add progress tracking: background thread prints PROGRESS every 10min
- Add FINAL progress line and total elapsed time on completion
- Add --log-level, --reme-log-level, -q CLI arguments
- Parallel mode: pool.map -> pool.imap_unordered for real-time progress
- config.yaml: full oracle (10000 items), 32 workers, all question types
- Add kill.sh (process cleanup) and run_async.sh (background eval launcher)

* docs: add LongMemEval oracle evaluation results (61.6% accuracy)

* feat(bench): add MAX_ITERATION limit to BenchQueryStep and add _auto_memory.yaml

* feat: add golden session benchmark & eval_only mode with refined prompt

- Add benchmark/longmemeval/run_golden_session.py for golden session evaluation
- Refine PROMPTED_SYSTEM_PROMPT: concise answer rule, remove 'Information not found' fallback
- Add eval_only mode to run.py (--eval_only flag)
- Add multiple eval config variants (evalonly, full, test5)
- Add analyze_results.py for result parsing
- Update auto_memory.yaml, longmemeval.yaml, application_config
- Update result-longmemeval.md with latest evaluation results
- Add benchmark results to .gitignore

* update: refine answer prompts and increase max iteration to 6 - Tighten prompted-answer system prompt for more concise output - Comment out 'Information not found' fallback rule - Increase MAX_ITERATION from 5 to 6 in bench_query - Add recall_eval.py - Update evaluation results

* feat(chunker): add dedicated JSON and JSONL file chunkers (cherry-pick from upstream #325)

- Add JsonFileChunker: structure-aware chunking preserving nested key paths,
  optional list-to-dict conversion, size measured by json.dumps() char count
- Add JsonlFileChunker: line-aligned sliding-window chunking with configurable
  overlap, supports char/byte mode switching
- Register both chunkers in default.yaml (json for .json, jsonl for .jsonl)
- Add comprehensive unit tests (21 + 20 test cases)

* feat(service): add CLI service for local job execution (from upstream #334)

- Introduce CliService to execute single jobs locally without serving ports
- Add prepare_start_config and should_precheck_start functions for CLI job setup
- Update reme start command to use CLI service when job argument is provided
- Add show_metadata to client kwargs for optional CLI metadata output
- Add unit tests for CLI service functionality and configuration handling

* feat(steps): add BM25/vector search steps, Python execute step, and draft steps (from upstream #334)

- Add Bm25SearchStep for plain BM25 keyword search with tool_context deduplication
- Add VectorSearchStep for plain vector search with tool_context deduplication
- Add PythonExecuteStep to run Python code in subprocess with timeout handling
- Add AddDraftStep/ReadAllDraftStep for draft accumulation scoped by tool context
- Update SearchStep with tool_context dedup, dynamic default limit via REME_SEARCH_LIMIT env,
  and candidate_multiplier default changed from 3.0 to 5.0
- Add comprehensive unit tests for all new steps

* feat(search): add tool context deduplication and improve search configuration (#321)

* feat(search): add tool context deduplication and improve search configuration

- Modify _make_tool methods to accept and inject tool_context_id parameter
- Add tool_context_id handling in AS and CC agent wrappers
- Increase search candidate multiplier from 3.0 to 5.0 in default config
- Extend HTTP client timeout from 30s to 3600s
- Add tool context deduplication logic to prevent duplicate search results
- Implement TTL-based expiration for seen chunks in tool contexts
- Add comprehensive unit tests for tool context deduplication behavior
- Update .gitignore to exclude longmemeval directory
- Add time import for timestamp functionality in search step

* refactor(search): replace time module with datetime for timestamp generation

- Removed unused time import
- Added static method _now_ts using datetime.timestamp
- Updated clock parameter to use _now_ts method instead of time.time
- Maintained same timestamp precision and functionality

* fix(file_io): fix risk of out-workspace paths (#322)

* fix(file_io): fix risk of out-workspace paths

* chore(file_io): remove unused unittest file

* fix(as_embedding): support both agentscope 2.0.2 and 2.0.3 (#323)

2.0.3 promoted `dimensions` to a required first-class constructor
argument while keeping a backfill from `parameters.dimensions`; 2.0.2
has no such argument and reads `dimensions` from `Parameters`. Keep
`dimensions` in `Parameters` for both versions and, when the model
constructor accepts `dimensions`, pass `dimensions=None` so 2.0.3's
backfill promotes it out of `parameters`.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* Bump version to 0.4.0.7

* refactor: delegate LLM-as-Judge to answer_judge_step and update eval config/results

- run.py: replace inline judge logic with judge_response_via_job using app.run_job('answer_judge')
- longmemeval.yaml: expand benchmark configuration
- bench_query.py: enhance benchmark query step
- result-longmemeval.md: update evaluation results
- judge_all_plus_results.json: add judge all-plus results

* refactor: split longmemeval.yaml into lme.yaml/beam.yaml and unify job names

- Split reme/config/longmemeval.yaml into lme.yaml (LongMemEval) and beam.yaml (BEAM)
- Unify job names across both configs: agentic_answer, answer_judge, context_answer
- Update evaluation/longmemeval/run.py and evaluation/beam/run_beam_eval.py to use unified job names
- Update all evaluation config YAMLs to reference lme.yaml
- Add BEAM benchmark step implementations (agentic_answer, context_answer, llm_judge)
- Remove obsolete config_test5.yaml and test_5sessions.py

* eval: BEAM 100K & LongMemEval cleaned-S 评测结果记录

- BEAM 100K eval-only (32并发, 20 case): Agentic 0.631, Prompted 0.468
- LongMemEval final GT (500题): Agentic 89.0%, Prompted 83.6%
- 新增 benchmark/result-beam.md, benchmark/result-longmemeval.md
- benchmark/beam/config.yaml: num_workers=32

* refactor: restructure benchmark directory and clean up gitignore rules

- Consolidate benchmark outputs to benchmark/results/ with .gitkeep
- Remove old benchmark scripts, configs and result files from benchmark/beam/ and benchmark/longmemeval/
- Add datasets/README.md and datasets/README_EN.md with download instructions
- Add datasets/longmemeval/download.py and final_groundtruth_cleaned_s.json
- Add memory_workspaces .gitkeep placeholders
- Restructure .gitignore: fix duplicate entries, add BEAM dataset exclusion, refine logs/results ignore patterns
- Remove stale result-beam.md and result-longmemeval.md from project root

* chore: clean up longmemeval benchmark scripts and update dataset docs

- Remove obsolete longmemeval benchmark runner/stats scripts

- Update datasets/longmemeval README and add Chinese translation

- Clean up final_groundtruth_cleaned_s.json

* docs(benchmark): add reproduction guide for LongMemEval and BEAM

- Add bilingual README for benchmark runners (EN/ZH)

- Cover prerequisites, dataset download, run commands, configs, outputs, logs, and kill.sh

* refactor: migrate auto_memory steps from evolve to benchmark-specific modules

- Split auto_memory into beam and lme benchmark-specific implementations
- Add auto_memory.py and auto_memory.yaml under steps/benchmark/beam and steps/benchmark/lme
- Slim down evolve/auto_memory.py and auto_memory.yaml to shared base only
- Remove obsolete evolve/_auto_memory.yaml
- Update benchmark run.py, config YAMLs, and step __init__.py registrations
- Update llm_judge and context_answer minor adjustments
- Remove outdated test_lme_final_answer_review.py

* revert(as_agent_wrapper): sync with upstream/main

Remove local-only comment to keep file identical with upstream/main.

* style: add trailing commas in benchmark __init__.py __all__ lists

* chore: disable vector_weight range assertion in SearchStep

* chore: add tests/integration/logs/ to .gitignore

* refactor: replace scipy.stats.kendalltau with pure numpy implementation

scipy is not listed in project dependencies. Implement Kendall's tau-b
rank correlation using only numpy to remove the undeclared dependency.

* feat(benchmark): add binary score metrics, update BEAM 1M results, and improve LLM retry/prompt config

- benchmark/beam/run.py: add binary score calculation per rubric item and per-type/overall binary stats
- benchmark/beam/config.yaml: switch to 1M dataset, reduce workers to 18
- benchmark/result-beam.md: add 1M evaluation results with binary scores
- benchmark/result-longmemeval.md: minor formatting
- reme/config/beam.yaml: increase max_retries to 5 and add retry_delay 5.0 for all LLM components
- reme/config/lme.yaml: increase max_retries to 5 and add retry_delay for judge/prompted/bench components
- reme/steps/benchmark/lme/agentic_answer.yaml: improve search strategy and answer rules prompts

* fix(benchmark): fix line-too-long and add pylint disable for main()

* refactor(longmemeval): use single cleaned-S dataset with embedded ground truth

- Switch to agentscope-ai/ReMe_longmemeval_clean_s_v2 HuggingFace source
- Remove separate final_groundtruth_cleaned_s.json (ground truth now in data file)
- Simplify download.py to fetch only longmemeval_s_reme_cleaned.json
- Remove dataset.variant and dataset.ground_truth_path config options
- Update benchmark and datasets READMEs to reflect new workflow
- Update .gitignore for new dataset filename

* fix: rename loop variable to avoid pylint redefined-outer-name warning

* refactor(benchmark): restructure datasets/memory_workspaces into benchmark and simplify auto_memory steps

* refactor(benchmark): extract BaseAgenticAnswerStep into base module

- Add reme/steps/benchmark/base/agentic_answer.py with shared agentic answer logic
- Refactor beam/lme AgenticAnswerStep to inherit from BaseAgenticAnswerStep
- Simplify lme/context_answer.py and update context_answer.yaml
- Update result-longmemeval.md with latest evaluation results (agentic 91.0%)

* refactor(benchmark): remove context_answer steps and unused configs

- Remove beam/lme context_answer job definitions and step implementations
- Remove prompted LLM component from beam.yaml and lme.yaml
- Delete jinli_lme.yaml (no longer needed)
- Simplify benchmark run.py scripts
- Clean up .gitkeep files and update .gitignore
- Remove unused import in search.py

* chore: remove benchmark/results/.gitkeep

---------

Co-authored-by: sa-buc <jiangniurou.xyf@dail-algo011164204033.ET135>
Co-authored-by: jinliyl <6469360+jinliyl@users.noreply.github.com>
Co-authored-by: imrewce <wce@pku.edu.cn>
Co-authored-by: Sen Huang <48879559+ployts@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 19:09:50 +08:00
Ziyang Guo
c060933e4d
fix(auto-memory): preserve message timestamps (#310)
* fix(auto-memory): preserve message timestamps

* fix(auto-memory): infer daily date from messages

* feat(file_io): add strict date parsing and improve daily date handling

- Add new parse_daily_date function for strict YYYY-MM-DD validation
- Replace extract_daily_date with parse_daily_date for explicit date validation
- Change _messages_day to use max date instead of min for historical imports
- Reorder imports to maintain consistent module ordering
- Move session message saving after date validation in auto_memory
- Add comprehensive tests for invalid date rejection before saving
- Add tests for strict YYYY-MM-DD date format validation
- Update test names to reflect latest date behavior

---------

Co-authored-by: Ziyang Guo <121015044+RunMarshal@users.noreply.github.com>
Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-07-01 17:15:07 +08:00
jinliyl
ffb4d08c4f
feat(mem): Enhance daily note system with metadata handling and write functionality (#295)
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
* feat(file_io): add daily_write step for creating daily notes with conversation metadata

- Add DailyWriteStep class that delegates to write job for creating daily notes
- Register daily_write job in default configuration with proper parameters
- Include validation for name and session_id path components
- Add test coverage for daily_write functionality including metadata handling
- Preserve existing job execution method in application.py after repositioning
- Update base_step.py to use positional-only parameter syntax for job methods
- Import and expose DailyWriteStep in file_io module initialization
- Override reserved metadata keys (name, description, session_id, source_conversation) with fixed values
- Refresh daily index after successful write operation
- Generate proper source conversation links in markdown format

* feat(daily): refactor daily note system with enhanced metadata handling

- Introduce validate_filename_component function and export it
- Add _INDEX_HIDDEN_METADATA_KEYS to hide conversation metadata from index
- Update scan_notes to exclude hidden metadata keys from index rendering
- Modify auto_memory to use daily_write tool and manage session frontmatter
- Implement session note lookup and renaming based on frontmatter name
- Update daily_list to return flattened note metadata including session info
- Change daily_write to dispatch write step instead of running job
- Add test cases for updated daily note functionality and metadata handling
- Update version from 0.4.0.2 to 0.4.0.3

* fix(evolve): correct metadata update in auto memory response

- Fixed trailing comma issue in metadata dictionary update
- Ensured proper formatting of response metadata structure
- Maintained existing functionality while fixing syntax error

* refactor(auto_resource): replace daily_create with dynamic note management

- Remove DailyCreateStep and related exports from file_io module
- Replace static daily note creation with dynamic resource-linked card system
- Implement LLM-suggested naming with frontmatter-driven file management
- Add source_resource linking for tracking original files
- Introduce collision handling with hash-based suffixes
- Update documentation to reflect new resource card workflow
- Modify auto_resource prompts to use write/edit tools instead of daily_create
- Adjust test fixture comments to match new agent behavior
- Update framework diagrams and quick start examples accordingly

* feat(app): add version info to app initialization and update auto-memory logic

- Include version number in application startup logging
- Remove tool result truncation logic from auto-memory step
- Update auto-memory to exclude tool_result blocks from saved history
- Add test case to verify tool results are filtered out from message saving
- Update YAML prompts to clarify filename naming rules without dates
- Modify configuration to support new dispatch steps format with persistence control

* feat(auto_memory): add note modification tracking and optimize frontmatter updates

- Add _note_bytes and _note_modified methods to track actual file changes
- Optimize frontmatter updates by checking existing metadata before update
- Add modified flag to response metadata indicating actual note changes
- Update logging to include modified status in various operations
- Add comprehensive tests for modified/unmodified detection scenarios
- Enhance result hook logic to skip when no actual changes occur
- Refactor metadata handling to properly track creation vs modification status
2026-06-25 21:54:56 +08:00
jinliyl
8b82ff88d0
feat(evolve): enhance agent reply processing and logging capabilities (#293)
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
* feat(evolve): enhance agent reply processing and logging capabilities

- Add agent_reply_result_text function to extract final user-visible text from agent replies
- Implement comprehensive logging throughout auto_memory, auto_resource, and dream modules
- Add max_units configuration option to limit extracted memory units
- Improve error handling and validation in auto_resource step
- Refactor dream extract step to respect max_units limit during processing
- Enhance summary rendering in dream finish step with detailed breakdown
- Add result hook functionality for embedding hosts integration
- Implement loose resource filename handling for root-level resources
- Update test cases to reflect new functionality and improved error messages

* test(background-steps): update fake upsert function to include created parameter

- Modified fake_upsert function to accept 'created' parameter instead of '_created'
- Added 'created' field to captured dictionary in fake_upsert function
- Included 'created': True in the expected response dictionary for test case
- Updated test assertion to match new parameter structure
2026-06-24 22:08:47 +08:00
Sen Huang
e31db5fe19
docs: rename vault_dir to workspace_dir in documentation and examples (#286)
* docs: rename vault_dir to workspace_dir in documentation and examples

* refactor(extract): format long method call across multiple lines

* refactor(extract): format system prompt parameters for better readability
2026-06-22 16:58:57 +08:00
jinliyl
206a53e5ed
init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
Renamed from reme4/steps/evolve/auto_memory.py (Browse further)