Commit graph

6 commits

Author SHA1 Message Date
jinliyl
215c1f72f2
feat: refine local-first research and memory workflows (#444)
Some checks are pending
NPM Format / Website checks (push) Waiting to run
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Windows Smoke / CLI smoke - py3.11 (push) Waiting to run
* feat: refine local-first research workflows

* fix: delegate structured output tool choice

* refactor(auto-fin): fetch and filter rolling CLS news

* fix(auto-fin): keep imports portable across platforms

* feat(auto-fin): expose CLS fetch controls

* fix(auto-fin): propagate configurable news window

* feat(auto_fin): normalize hybrid wikilinks in report body

- Add _normalize_hybrid_wikilinks method to remove redundant Markdown destinations
- Use regex to identify hybrid wikilinks with optional destinations
- Replace redundant destinations with simpler wikilink format for clarity
- Ensure normalization is failure-safe with exception handling and logging
- Update report body normalization process to apply hybrid wikilink fix
- Add unit tests to verify correct normalization and failure safety behavior

* fix(dream): serialize integration with application-wide asyncio lock

- Add application-wide asyncio.Lock to serialize digest writes during integration
- Update _snapshot_digest to capture metadata per bucket
- Validate bucket association when recovering from file changes
- Add tests ensuring recovery only from the correct bucket
- Add tests confirming integration lock is shared across application context
- Enhance strict topic YAML loading validation in dream utils
- Add tests for strict topic loading rejecting invalid or lossy fields

* fix(cookbook): enable configurable job_tools for digest and merge steps

- Update daily_cookbook.yaml to add job_tools: [memory_search, read] in digest steps
- Modify DailyPaperDigestStep to read job_tools from kwargs instead of fixed list
- Modify AutoFinMergeStep to similarly read job_tools from kwargs
- Update tests to pass job_tools explicitly when invoking these steps
- Remove hardcoded _TOOLS constants and replace with dynamic job_tools handling

* fix: retry incomplete dream receipts

* perf(pdf): increase max PDF pages limit from 20 to 35

- Updated configuration max_pdf_pages from 20 to 35 in daily_cookbook.yaml
- Modified code to extract up to 35 pages instead of 20 in analyze.py
- Updated README and README_ZH to document the increased max_pdf_pages
- Adjusted unit test assertions to reflect new max_pdf_pages limit of 35

* fix memory integration and daily paper links

* docs clarify cookbook tool usage
2026-08-11 23:32:34 +08:00
jinliyl
a9ec334adc
feat: simplify wikilink semantics and support line anchors (#412)
* feat: simplify local links and support line anchors

* fix: align line anchor tests with CI lint

* fix: preserve local links across file moves

* fix: encode markdown paths when rewriting links

* refactor(read): keep explicit line range parameters

* fix: simplify legacy link predicate compatibility

* docs: align local link behavior with implementation

* fix: skip unsupported markdown destination escapes

* fix: normalize workspace link paths across platforms

* fix: bound markdown link scanning

* fix: keep local link processing linear

* docs: clarify permissive markdown link parsing

* fix: handle local link processing failures

* refactor: limit file links to wikilink syntax

* docs: align wikilink contract with implementation

* fix: normalize dream and neighbor paths on Windows

* fix: resolve workspace path for neighbor expansion
2026-08-05 11:47:50 +08:00
jinliyl
7d86658f33
Refactor logging levels and add dream schema definitions (#291)
* chore(logging): change info logs to debug level for data loading operations

- Changed stopwords loading log from info to debug level
- Changed file catalog nodes loading log from info to debug level
- Changed file graph nodes loading log from info to debug level

* feat(dream): add dream schema definitions and enum for auto-dream functionality

- Add DreamBucketEnum with procedure, personal, and wiki values
- Create comprehensive dream-related Pydantic models including DreamUnit,
  DreamTopic, DreamExtractOutput, IntegrateOutcome, TopicSelectionOutput,
  ProactiveResult, and DreamState
- Move schema definitions from local step module to shared schema package
- Update dream extraction and integration steps to use new enum-based
  bucket validation
- Initialize digest directories for each dream bucket type
- Enhance embedding store health check with workspace directory logging

* refactor(tests): update DreamState import path in test_auto_dream.py

- Move DreamState import from reme.steps.evolve.dream.schema to reme.schema
- Maintain same functionality with updated module reference
- Align import with new schema location in project structure
2026-06-24 16:44:03 +08:00
jinliyl
a3bd81bde2
Update version to 0.4.0.2 and improve tokenizer index handling (#290)
* fix(core): update version number to 0.4.0.1

- Incremented version from 0.4.0.0 to 0.4.0.1 in __init__.py

* fix(index): remove stopwords path from tokenizer config and add keyword index repair

- Remove stopwords_path from tokenizer config to prevent index forking by install path
- Add _sync_keyword_index_from_chunks method to repair keyword index when persisted state mismatches
- Implement test for keyword index repair from persisted chunks when missing
- Add test to verify tokenizer fingerprint ignores stopwords absolute path
- Update version from 0.4.0.1 to 0.4.0.2

* feat(dream): add scan_days parameter to dream extraction process

- Add scan_days configuration option to default.yaml with default value of 2
- Implement recent_dates utility function to calculate date ranges for scanning
- Modify DreamExtractStep to scan multiple days based on scan_days parameter
- Update dream extraction to process files across multiple dates instead of single day
- Extend DreamState schema to include dates and scan_days fields
- Update DreamTopicsStep to handle multi-day topic processing
- Modify finish step to checkpoint files from all scanned dates
- Add comprehensive tests for multi-day scanning functionality
- Update prompt templates to include scan dates information
- Refactor topics writing logic to target specific date rather than current date
2026-06-24 15:01:07 +08:00
Sen Huang
e31db5fe19
docs: rename vault_dir to workspace_dir in documentation and examples (#286)
* docs: rename vault_dir to workspace_dir in documentation and examples

* refactor(extract): format long method call across multiple lines

* refactor(extract): format system prompt parameters for better readability
2026-06-22 16:58:57 +08:00
jinliyl
206a53e5ed
init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
Renamed from reme4/steps/evolve/dream/utils.py (Browse further)