mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-08-28 05:25:04 +00:00
43a407bc4f
* feat: add start_date/end_date time filter support for search job
- Add _extract_date_from_path to extract validated YYYY-MM-DD from chunk paths
- Add start_date/end_date filtering in _matches_search_filter
- Implement progressive recall in FaissLocalFileStore.vector_search
- Promote start_date/end_date from context to search_filter in SearchStep
- Add start_date/end_date parameters to search job in default.yaml
- Add unit tests for date filter functionality
* fix: validate/normalize date filters and harden _extract_date_from_path
Address three code-review comments on the time_filter search feature:
1. Validate/normalize start_date and end_date before string comparison.
_matches_search_filter does lexicographic comparison against path_date
(always canonical YYYY-MM-DD). Raw caller values like '2026-2-28' or
'abc' would produce silently wrong results. Now SearchStep normalizes
valid dates via extract_daily_date (with strptime fallback for
non-zero-padded input) and silently ignores invalid dates with a
logger.warning, removing them from the filter.
2. Clarify behavior for paths without embedded dates.
Added optional strict_date_filter parameter (default False). When True
and at least one date bound is active, chunks whose path yields no date
(e.g. digest/personal/topic.md) are excluded. When False (default),
the existing behavior is preserved — dateless paths pass through.
3. Harden _extract_date_from_path against non-standard suffixes.
Previously parts[1].split('.')[0] accepted '2026-05-18.anything' as a
valid date. Now only exact 'YYYY-MM-DD' (dir) and 'YYYY-MM-DD.md'
(day-index) forms are accepted.
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_auto_dream.py | ||
| test_background_steps.py | ||
| test_base_component.py | ||
| test_bm25_index_perf.py | ||
| test_channel_notify.py | ||
| test_channel_sink.py | ||
| test_claim_channel.py | ||
| test_common_steps.py | ||
| test_component_registry.py | ||
| test_config_parser.py | ||
| test_cron_job.py | ||
| test_crud_steps.py | ||
| test_daily_steps.py | ||
| test_default_file_chunker.py | ||
| test_evolve_utils.py | ||
| test_file_catalog.py | ||
| test_file_graph.py | ||
| test_file_store_consistency.py | ||
| test_job.py | ||
| test_keyword_index.py | ||
| test_link_expansion.py | ||
| test_markdown_file_chunker.py | ||
| test_packaging.py | ||
| test_prompt_handler.py | ||
| test_read_image_steps.py | ||
| test_read_with_neighbors.py | ||
| test_reme_cli.py | ||
| test_runtime_context.py | ||
| test_search_step.py | ||
| test_service.py | ||
| test_service_utils.py | ||
| test_tokenizer.py | ||
| test_utils.py | ||
| test_wikilink_utils.py | ||
| test_write_metadata_lock.py | ||