ReMe/tests/unit
jinliyl b4333fbef8
feat(index): add bounded memory-aware batch processing (#381)
* test(background_steps): add comprehensive tests for batch processing and memory management

- Add test for catalog upserts in batches of at most 100 files
- Add test for catalog deletes in batches of at most 100 paths
- Add test for index memory budget reducing batches to one file
- Add test for memory target limiting cumulative batch size
- Add test for invalid batch memory settings rejection
- Add test for continuing after one batch fails
- Add test for yielding to event loop while building batch
- Add test for modified file reusing unchanged embedding
- Add test for reporting memory estimation failure without aborting

feat(update_changes): implement bounded batch processing with memory management

- Add configurable batch parameters with default values
- Implement memory budget calculation based on available system memory
- Add file inspection and memory estimation before processing
- Implement batch flushing when limits are reached
- Add proper error handling for batch operations
- Support async yielding during batch building
- Add comprehensive validation for batch configuration parameters
- Implement memory estimation for indexing operations
- Add batch size limiting for delete operations

* test(steps): add tests for memory estimation failure handling

- Add test case for isolated file processing when memory estimation fails
- Add test case for proper release of flushed items before building next file
- Implement weak reference tracking to verify payload lifetime management
- Create parametrized tests for both source and item memory estimation methods
- Add assertions to verify single-item batch behavior on estimation failures
- Include comprehensive error handling verification for memory budget calculations

* chore(version): bump version to 0.4.1.3

- Update __version__ from 0.4.1.2 to 0.4.1.3 in __init__.py

* feat(index): support batch settings from environment

* refactor(index): use direct batch defaults

* refactor(index): configure memory estimates through step args

* ci: simplify Windows smoke dependencies
2026-07-20 17:25:00 +08:00
..
__init__.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_as_embedding_lazy.py refactor(embedding): defer provider construction until first remote call (#361) 2026-07-15 20:46:36 +08:00
test_auto_dream.py fix(proactive): expose topics in primary answer (#380) 2026-07-20 16:05:47 +08:00
test_background_steps.py feat(index): add bounded memory-aware batch processing (#381) 2026-07-20 17:25:00 +08:00
test_base_component.py feat(benchmark): add lme benchmark steps (#326) 2026-07-07 18:53:39 +09:00
test_bm25_index_perf.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_cc_agent_wrapper.py feat(agent): add Codex agent wrapper and ReMe MCP bridge (#358) 2026-07-17 13:39:18 +08:00
test_channel_notify.py docs: rename vault_dir to workspace_dir in documentation and examples (#286) 2026-06-22 16:58:57 +08:00
test_channel_sink.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_claim_channel.py docs: rename vault_dir to workspace_dir in documentation and examples (#286) 2026-06-22 16:58:57 +08:00
test_codex_agent_wrapper.py feat: add codex auth modes, background embedding/index repair, and qwenpaw logging (#371) 2026-07-20 14:47:34 +08:00
test_common_steps.py feat(lme): add cli execution and agentic search tooling (#334) 2026-07-08 17:43:09 +09:00
test_component_registry.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_config_parser.py fix: enforce markdown chunk byte limits (#370) 2026-07-17 22:03:58 +08:00
test_cron_job.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_crud_steps.py feat: add codex auth modes, background embedding/index repair, and qwenpaw logging (#371) 2026-07-20 14:47:34 +08:00
test_daily_steps.py fix(auto-memory): preserve message timestamps (#310) 2026-07-01 17:15:07 +08:00
test_default_file_chunker.py feat(benchmark): add lme benchmark steps (#326) 2026-07-07 18:53:39 +09:00
test_evolve_utils.py fix(auto-memory): preserve message timestamps (#310) 2026-07-01 17:15:07 +08:00
test_file_catalog.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_file_graph.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_file_store_consistency.py feat: add codex auth modes, background embedding/index repair, and qwenpaw logging (#371) 2026-07-20 14:47:34 +08:00
test_hermes_agent_plugin.py feat(plugins): add Hermes Agent memory provider (#365) 2026-07-17 14:06:12 +08:00
test_http_client.py feat(benchmark): add lme benchmark steps (#326) 2026-07-07 18:53:39 +09:00
test_job.py feat(benchmark): add lme benchmark steps (#326) 2026-07-07 18:53:39 +09:00
test_json_file_chunker.py feat(chunker): better json chunker and json chunker (#325) 2026-07-08 15:18:59 +08:00
test_jsonl_file_chunker.py feat(chunker): better json chunker and json chunker (#325) 2026-07-08 15:18:59 +08:00
test_keyword_index.py refactor(embedding): defer provider construction until first remote call (#361) 2026-07-15 20:46:36 +08:00
test_link_expansion.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_lme_final_answer_review.py chore(benchmark): remove longmemeval final answer review file (#366) 2026-07-16 20:32:28 +08:00
test_local_embedding_store.py fix(embedding): make input truncation CJK-aware (#337) 2026-07-13 17:43:48 +09:00
test_logging_config.py feat: add codex auth modes, background embedding/index repair, and qwenpaw logging (#371) 2026-07-20 14:47:34 +08:00
test_markdown_file_chunker.py fix: enforce markdown chunk byte limits (#370) 2026-07-17 22:03:58 +08:00
test_packaging.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_prompt_handler.py feat(benchmark): add lme benchmark steps (#326) 2026-07-07 18:53:39 +09:00
test_read_image_steps.py docs: rename vault_dir to workspace_dir in documentation and examples (#286) 2026-06-22 16:58:57 +08:00
test_read_with_neighbors.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_reme_cli.py feat(core): add shell execution and runtime memory status (#344) 2026-07-14 16:31:41 +08:00
test_runtime_context.py feat(benchmark): add lme benchmark steps (#326) 2026-07-07 18:53:39 +09:00
test_search_step.py feat(benchmark): add LongMemEval golden answer validation (#335) 2026-07-13 21:26:27 +08:00
test_service.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_service_utils.py feat(benchmark): add lme benchmark steps (#326) 2026-07-07 18:53:39 +09:00
test_shell_step.py feat(core): add shell execution and runtime memory status (#344) 2026-07-14 16:31:41 +08:00
test_status_step.py feat(core): add shell execution and runtime memory status (#344) 2026-07-14 16:31:41 +08:00
test_tokenizer.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_utils.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
test_wikilink_utils.py docs: rename vault_dir to workspace_dir in documentation and examples (#286) 2026-06-22 16:58:57 +08:00
test_write_metadata_lock.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00