Commit graph

4 commits

Author SHA1 Message Date
imrewce
354837f9af
feat(proactive): separate proactive refresh from auto dream (#488)
* refractor(proactive): upgrade proactive feature with disentangled job and steps

* refactor(proactive): apply audit fixes

- rename read-side job 'proactive' -> 'proactive_read' (less confusing vs the refresh pipeline)
- drop dedicated agent_wrapper.proactive; extraction reuses the default wrapper
- simplify schema: remove unused ProactiveExtractOutput/TopicUpdate, drop resource_paths
- extract no longer scans resource/ directly (daily notes already carry resource content)
- update tests and docs accordingly

* feat(proactive): strict extract-output gate and prompt total budget

- parse_extract_reply now requires a contract section (follow_ups/extends/updates
  as a list); non-empty replies with misspelled section names trigger the
  existing one-shot retry instead of silently checkpointing changed files
- pack_paths gains max_total_chars; extract packs newest daily material first,
  keeps the first file on overflow, and records omitted files in a trailer
  (default budget 300000 chars, configurable via max_total_chars)
- tests: schema gate unit, schema-error retry e2e, budget unit + e2e

* feat(proactive): add scenario-card plan step and generative agenda step

* feat(proactive): digest-personal profile personalization and leaner LLM contract

- extract/plan/agenda now draw a user profile block from <digest_dir>/personal/*.md
  (frontmatter description + body excerpt, per-file budget, profile.md fallback)
- all daily access honours the configured daily_dir (prompt paths parameterized,
  config-driven fallbacks) so workspaces using e.g. memory/ work unchanged
- schema trim: drop dead fields errors/material_paths, carry_forward_all -> count
- shrink LLM output contract: new topics emit title/reason/confidence/paths only;
  keywords removed end-to-end, evidence derived from paths[0] (updates keep it)

* fix(proactive): skip checkpoint when extract reply stays unusable after retry

Two consecutive unparseable replies now short-circuit the round without
checkpointing, so the same material is retried next round instead of being
silently consumed (closes the residual audit #1 gap: the structural gate
detected schema-wrong output but a double failure still checkpointed).

* fix(proactive): replace running bool with reference-counted job activity tracker for the idle gate

* refactor(proactive): remove job activity tracking and idle gate, restore job tree to upstream

* fix(proactive): address second audit round (readonly reader, mtime checkpoint, wider fallbacks, profile containment, horizon content, expiry boundary)

* refactor(dream): strip interests.yaml ownership from dream, proactive is now the sole writer

* refactor(dream): separate proactive topic generation

* ci: update renamed auto dream smoke test

* fix(proactive): complete refresh migration and docs

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-09-07 17:23:37 +08:00
jinliyl
d3aee1adf5
feat(evolve): report auto-dream file changes (#458)
* feat(evolve): report auto-dream content changes

* perf(evolve): use lightweight dream snapshots
2026-08-19 15:44:06 +08:00
jinliyl
215c1f72f2
feat: refine local-first research and memory workflows (#444)
Some checks are pending
NPM Format / Website checks (push) Waiting to run
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Windows Smoke / CLI smoke - py3.11 (push) Waiting to run
* feat: refine local-first research workflows

* fix: delegate structured output tool choice

* refactor(auto-fin): fetch and filter rolling CLS news

* fix(auto-fin): keep imports portable across platforms

* feat(auto-fin): expose CLS fetch controls

* fix(auto-fin): propagate configurable news window

* feat(auto_fin): normalize hybrid wikilinks in report body

- Add _normalize_hybrid_wikilinks method to remove redundant Markdown destinations
- Use regex to identify hybrid wikilinks with optional destinations
- Replace redundant destinations with simpler wikilink format for clarity
- Ensure normalization is failure-safe with exception handling and logging
- Update report body normalization process to apply hybrid wikilink fix
- Add unit tests to verify correct normalization and failure safety behavior

* fix(dream): serialize integration with application-wide asyncio lock

- Add application-wide asyncio.Lock to serialize digest writes during integration
- Update _snapshot_digest to capture metadata per bucket
- Validate bucket association when recovering from file changes
- Add tests ensuring recovery only from the correct bucket
- Add tests confirming integration lock is shared across application context
- Enhance strict topic YAML loading validation in dream utils
- Add tests for strict topic loading rejecting invalid or lossy fields

* fix(cookbook): enable configurable job_tools for digest and merge steps

- Update daily_cookbook.yaml to add job_tools: [memory_search, read] in digest steps
- Modify DailyPaperDigestStep to read job_tools from kwargs instead of fixed list
- Modify AutoFinMergeStep to similarly read job_tools from kwargs
- Update tests to pass job_tools explicitly when invoking these steps
- Remove hardcoded _TOOLS constants and replace with dynamic job_tools handling

* fix: retry incomplete dream receipts

* perf(pdf): increase max PDF pages limit from 20 to 35

- Updated configuration max_pdf_pages from 20 to 35 in daily_cookbook.yaml
- Modified code to extract up to 35 pages instead of 20 in analyze.py
- Updated README and README_ZH to document the increased max_pdf_pages
- Adjusted unit test assertions to reflect new max_pdf_pages limit of 35

* fix memory integration and daily paper links

* docs clarify cookbook tool usage
2026-08-11 23:32:34 +08:00
jinliyl
7d86658f33
Refactor logging levels and add dream schema definitions (#291)
* chore(logging): change info logs to debug level for data loading operations

- Changed stopwords loading log from info to debug level
- Changed file catalog nodes loading log from info to debug level
- Changed file graph nodes loading log from info to debug level

* feat(dream): add dream schema definitions and enum for auto-dream functionality

- Add DreamBucketEnum with procedure, personal, and wiki values
- Create comprehensive dream-related Pydantic models including DreamUnit,
  DreamTopic, DreamExtractOutput, IntegrateOutcome, TopicSelectionOutput,
  ProactiveResult, and DreamState
- Move schema definitions from local step module to shared schema package
- Update dream extraction and integration steps to use new enum-based
  bucket validation
- Initialize digest directories for each dream bucket type
- Enhance embedding store health check with workspace directory logging

* refactor(tests): update DreamState import path in test_auto_dream.py

- Move DreamState import from reme.steps.evolve.dream.schema to reme.schema
- Maintain same functionality with updated module reference
- Align import with new schema location in project structure
2026-06-24 16:44:03 +08:00
Renamed from reme/steps/evolve/dream/schema.py (Browse further)