Commit graph

14 commits

Author SHA1 Message Date
jinliyl
eb471d7d94
fix(embedding): reject mismatched embedding dimensions (#330)
* fix(embedding): enforce strict dimension matching for embeddings

- Add _embedding_dim_matches method to validate embedding dimensions
- Reject embeddings with mismatched dimensions instead of padding/truncating
- Drop stale embeddings with wrong dimensions during loading and upsert operations
- Disable embedding store when query dimensions don't match configured dimensions
- Fail health checks when embedding dimensions don't match expected values
- Skip chunks with wrong dimensions during FAISS index rebuild
- Add comprehensive tests for dimension validation behavior

* refactor(file_store): simplify conditional checks in vector search and test assertions

- Combine multiple conditionals into single check for empty FAISS index
- Replace explicit empty list comparison with boolean check for node embedding calls
- Maintain same functional behavior while improving code readability

* fix(embedding): harden dimension validation helpers
2026-07-08 14:23:38 +09:00
jinliyl
38cf16071b
refactor(embedding): update embedding model initialization and session storage paths (#329)
* refactor(embedding): update embedding model initialization and session storage paths

- Remove unused inspect import from as_embedding module
- Pass dimensions directly to embedding model constructor instead of using parameters
- Update session state file paths to use mem_session directory instead of resource
- Add mem_session_dir configuration option to application config schema
- Update workspace directory creation to include new mem_session directory
- Change AgentScope and Claude Code session paths to use mem_session directory
- Move embedding dimensions from parameters to top-level configuration
- Update AgentScope dependency version from 2.0.3 to 2.0.4
- Update integration tests to reflect new session file location paths

* chore(version): bump version to 0.4.0.8

- Update __version__ from 0.4.0.7 to 0.4.0.8 in __init__.py
2026-07-08 12:30:46 +09:00
jinliyl
10da205797
feat(benchmark): add lme benchmark steps (#326)
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
* refactor(search): replace time module with datetime for timestamp generation

- Removed unused time import
- Added static method _now_ts using datetime.timestamp
- Updated clock parameter to use _now_ts method instead of time.time
- Maintained same timestamp precision and functionality

* test(http): add tests for HTTP client display formatting

- Add test for default metadata hiding behavior in CLI output
- Add test for metadata display when show_metadata is enabled
- Verify _format_for_display method correctly formats response text
- Test both success case and metadata inclusion scenarios

* chore(build): remove longmemeval from gitignore

- Removed longmemeval directory from gitignore list
- Kept evaluation and datasets directories in ignore list
- Updated gitignore configuration for proper version control
2026-07-07 18:53:39 +09:00
Sen Huang
0a7eea18f8
fix(as_embedding): support both agentscope 2.0.2 and 2.0.3 (#323)
2.0.3 promoted `dimensions` to a required first-class constructor
argument while keeping a backfill from `parameters.dimensions`; 2.0.2
has no such argument and reads `dimensions` from `Parameters`. Keep
`dimensions` in `Parameters` for both versions and, when the model
constructor accepts `dimensions`, pass `dimensions=None` so 2.0.3's
backfill promotes it out of `parameters`.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 19:03:32 +08:00
jinliyl
7369342115
feat(search): add tool context deduplication and improve search configuration (#321)
* feat(search): add tool context deduplication and improve search configuration

- Modify _make_tool methods to accept and inject tool_context_id parameter
- Add tool_context_id handling in AS and CC agent wrappers
- Increase search candidate multiplier from 3.0 to 5.0 in default config
- Extend HTTP client timeout from 30s to 3600s
- Add tool context deduplication logic to prevent duplicate search results
- Implement TTL-based expiration for seen chunks in tool contexts
- Add comprehensive unit tests for tool context deduplication behavior
- Update .gitignore to exclude longmemeval directory
- Add time import for timestamp functionality in search step

* refactor(search): replace time module with datetime for timestamp generation

- Removed unused time import
- Added static method _now_ts using datetime.timestamp
- Updated clock parameter to use _now_ts method instead of time.time
- Maintained same timestamp precision and functionality
2026-07-06 16:18:39 +09:00
xyf2020
43a407bc4f
feat: add start_date/end_date time filter support for search job (#317)
Some checks failed
Tests ReMe / Unit Tests - py3.11 (push) Has been cancelled
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
Tests ReMe / Unit Tests - py3.12 (push) Has been cancelled
Tests ReMe / Unit Tests - py3.13 (push) Has been cancelled
* feat: add start_date/end_date time filter support for search job

- Add _extract_date_from_path to extract validated YYYY-MM-DD from chunk paths
- Add start_date/end_date filtering in _matches_search_filter
- Implement progressive recall in FaissLocalFileStore.vector_search
- Promote start_date/end_date from context to search_filter in SearchStep
- Add start_date/end_date parameters to search job in default.yaml
- Add unit tests for date filter functionality

* fix: validate/normalize date filters and harden _extract_date_from_path

Address three code-review comments on the time_filter search feature:

1. Validate/normalize start_date and end_date before string comparison.
   _matches_search_filter does lexicographic comparison against path_date
   (always canonical YYYY-MM-DD). Raw caller values like '2026-2-28' or
   'abc' would produce silently wrong results. Now SearchStep normalizes
   valid dates via extract_daily_date (with strptime fallback for
   non-zero-padded input) and silently ignores invalid dates with a
   logger.warning, removing them from the filter.

2. Clarify behavior for paths without embedded dates.
   Added optional strict_date_filter parameter (default False). When True
   and at least one date bound is active, chunks whose path yields no date
   (e.g. digest/personal/topic.md) are excluded. When False (default),
   the existing behavior is preserved — dateless paths pass through.

3. Harden _extract_date_from_path against non-standard suffixes.
   Previously parts[1].split('.')[0] accepted '2026-05-18.anything' as a
   valid date. Now only exact 'YYYY-MM-DD' (dir) and 'YYYY-MM-DD.md'
   (day-index) forms are accepted.
2026-07-03 15:58:04 +08:00
Sen Huang
8877743ca9
feat(cli): route bare commands to the running server's real config (#312)
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Deploy Docs / deploy (push) Has been cancelled
Deploy Docs / build (push) Has been cancelled
call_server now resolves the client backend/transport/host/port from the
live `reme start` process (replaying its start args through
resolve_app_config) so a bare `reme <action>` reaches the server however
it was actually launched, falling back to local config when none runs.
Explicit backend=/transport=/host=/port= still win.

Also fix as_embedding to pass `dimensions` explicitly for agentscope
>=2.0.2, and add Claude Code auto-memory/auto-dream demos to the READMEs.
2026-07-01 17:15:48 +08:00
Sen Huang
3dee10d4f9
feat: add Claude Code plugin with auto-memory functionality (#297)
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
Tests ReMe / Unit Tests - py3.11 (push) Has been cancelled
Tests ReMe / Unit Tests - py3.12 (push) Has been cancelled
Tests ReMe / Unit Tests - py3.13 (push) Has been cancelled
* feat: add Claude Code plugin with auto-memory functionality

* refactor(auto_memory): fix spacing in json parsing logic
2026-06-26 14:46:42 +08:00
jinliyl
ad7893e9c4
fix(mcp): resolve circular import issues and update dependencies (#296)
* fix(mcp): resolve circular import issues and update dependencies

- Moved fastmcp imports inside functions to prevent circular dependencies
- Replaced _TRANSPORT_MAP with _VALID_TRANSPORTS set for transport validation
- Updated version number from 0.4.0.3 to 0.4.0.4
- Added claude-agent-sdk dependency to core optional dependencies
- Used TYPE_CHECKING imports for FastMCP related types
- Restructured transport mapping logic within function scope
- Fixed string annotation for CallToolResult type hints

* refactor(tests): update date handling in daily steps tests

- Replace _date.today() with timezone-aware now function
- Use Asia/Shanghai timezone for date formatting
- Change return format to use strftime instead of isoformat
- Import now function from reme.steps.evolve module

* refactor(tests): clean up unused imports in daily steps test

- Removed unused date import from datetime module
- Removed redundant pathlib Path import that was already imported later
- Kept necessary imports for asyncio, os, tempfile, warnings, and frontmatter modules

* test(daily_steps): update test to include application context for daily list step

- Add ApplicationContext initialization with temporary workspace directory
- Register file store component in application context
- Pass application context to DailyListStep constructor
- Maintain existing test assertion behavior for date metadata verification
2026-06-26 09:37:08 +08:00
jinliyl
afe12b16db
feat(file_store): add embedding backfill for persisted chunks (#292)
- Implement _backfill_missing_embeddings method to handle chunks without embeddings
- Add logic to identify and process chunks that predate embedding feature
- Integrate backfill process into store loading sequence
- Add proper error handling and logging for backfill operations
- Create unit test for embedding backfill functionality
- Ensure backfilled embeddings are properly persisted to storage
2026-06-24 17:32:21 +08:00
jinliyl
7d86658f33
Refactor logging levels and add dream schema definitions (#291)
* chore(logging): change info logs to debug level for data loading operations

- Changed stopwords loading log from info to debug level
- Changed file catalog nodes loading log from info to debug level
- Changed file graph nodes loading log from info to debug level

* feat(dream): add dream schema definitions and enum for auto-dream functionality

- Add DreamBucketEnum with procedure, personal, and wiki values
- Create comprehensive dream-related Pydantic models including DreamUnit,
  DreamTopic, DreamExtractOutput, IntegrateOutcome, TopicSelectionOutput,
  ProactiveResult, and DreamState
- Move schema definitions from local step module to shared schema package
- Update dream extraction and integration steps to use new enum-based
  bucket validation
- Initialize digest directories for each dream bucket type
- Enhance embedding store health check with workspace directory logging

* refactor(tests): update DreamState import path in test_auto_dream.py

- Move DreamState import from reme.steps.evolve.dream.schema to reme.schema
- Maintain same functionality with updated module reference
- Align import with new schema location in project structure
2026-06-24 16:44:03 +08:00
jinliyl
a3bd81bde2
Update version to 0.4.0.2 and improve tokenizer index handling (#290)
* fix(core): update version number to 0.4.0.1

- Incremented version from 0.4.0.0 to 0.4.0.1 in __init__.py

* fix(index): remove stopwords path from tokenizer config and add keyword index repair

- Remove stopwords_path from tokenizer config to prevent index forking by install path
- Add _sync_keyword_index_from_chunks method to repair keyword index when persisted state mismatches
- Implement test for keyword index repair from persisted chunks when missing
- Add test to verify tokenizer fingerprint ignores stopwords absolute path
- Update version from 0.4.0.1 to 0.4.0.2

* feat(dream): add scan_days parameter to dream extraction process

- Add scan_days configuration option to default.yaml with default value of 2
- Implement recent_dates utility function to calculate date ranges for scanning
- Modify DreamExtractStep to scan multiple days based on scan_days parameter
- Update dream extraction to process files across multiple dates instead of single day
- Extend DreamState schema to include dates and scan_days fields
- Update DreamTopicsStep to handle multi-day topic processing
- Modify finish step to checkpoint files from all scanned dates
- Add comprehensive tests for multi-day scanning functionality
- Update prompt templates to include scan dates information
- Refactor topics writing logic to target specific date rather than current date
2026-06-24 15:01:07 +08:00
Sen Huang
e31db5fe19
docs: rename vault_dir to workspace_dir in documentation and examples (#286)
* docs: rename vault_dir to workspace_dir in documentation and examples

* refactor(extract): format long method call across multiple lines

* refactor(extract): format system prompt parameters for better readability
2026-06-22 16:58:57 +08:00
jinliyl
206a53e5ed
init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00