Commit graph

6 commits

Author SHA1 Message Date
jinliyl
cf22ef3b1d
feat: add codex auth modes, background embedding/index repair, and qwenpaw logging (#371)
* feat(codex): add authentication mode support with thread-safe logging

- Implement _CodexAuthConfig dataclass for resolved auth settings
- Add auth_mode parameter with auto/api_key/oauth options
- Separate API key and OAuth authentication flows
- Force specific login method based on auth mode
- Add explicit API key validation requirement
- Serialize concurrent logger initialization in thread lock
- Close logging handlers properly during cleanup
- Update default config with auth_mode presets for codex and codex_oauth
- Add comprehensive tests for authentication modes and concurrent logging

* feat(file_store): implement background embedding backfill and keyword index repair

- Add _after_embedding_backfill hook in FAISS local file store
- Schedule startup embedding repair without delaying component readiness
- Cancel and collect embedding backfill task during component shutdown
- Log progress at fixed percentage boundaries for long-running operations
- Process embedding backfill in configurable batch sizes with progress reporting
- Rebuild keyword index in bounded batches with detailed mismatch diagnostics
- Format stdlib logs consistently with QwenPaw console output using relative paths
- Run embedding backfill as background task that doesn't block component startup
- Add comprehensive tests for background embedding and keyword index repair scenarios

* fix(file-store): repair graph-chunk consistency on load

- Add _repair_graph_chunk_consistency method to detect and fix mismatched graph/chunk states
- Clear torn graph/chunk state when missing or orphaned chunks are detected
- Ensure keyword index sync handles empty chunks properly
- Add comprehensive tests for graph-chunk consistency scenarios
- Update test utilities to properly seed graph/chunk snapshots
- Increment version to 0.4.1.2

* feat(file_io): enhance list step response format and add comprehensive logging

- Format list output with bullet points for better readability
- Add explicit "No files found" message when directory is empty
- Include detailed timing information for file store startup phases
- Add logging for chunk loading, graph consistency checks, and keyword indexing
- Provide detailed metrics for embedding backfill operations
- Add comprehensive test coverage for empty directory scenarios
- Include batch processing statistics for embedding operations

* feat(logger): add QwenPaw logging integration with forwarding mechanism

- Introduce _ForwardToLoggerHandler to forward log records to target logger
- Add qwenpaw logger integration that forwards ReMe logs to QwenPaw handlers
- Maintain ReMe logger stability for modules that cache it at import time
- Enable QwenPaw handlers to take effect without ReMe reconfiguration
- Add comprehensive tests for stdlib forwarding to QwenPaw sinks
- Support explicit REME_DISABLE_LOGURU=false to keep original Loguru backend
- Preserve existing logging behavior when QwenPaw is not configured

* fix(file_store): serialize concurrent FAISS dump operations to prevent corruption

- Add asyncio lock to ensure only one FAISS dump operation runs at a time
- Generate unique temporary filenames using UUID tokens for atomic replacement
- Implement proper cleanup of temporary files in finally block
- Add comprehensive test to verify concurrent dumps are serialized
- Ensure atomic writes by replacing both index and idmap files together
- Prevent partial state writes during concurrent access scenarios
2026-07-20 14:47:34 +08:00
jinliyl
2e87b7a52e
feat(core): add shell execution and runtime memory status (#344)
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
Windows Smoke / CLI smoke - py3.11 (push) Waiting to run
* feat(core): add shell command execution and memory status reporting

- Introduce ShellStep for executing shell commands with timeout support
- Add StatusStep to report memory estimates for stateful data components
- Register shell and status commands in default configuration
- Update documentation with new reme status and shell command capabilities
- Implement comprehensive unit tests for both new step types
- Add support for asynchronous command execution with proper error handling

* feat(config): add log_config option to suppress config loading logs

- Add log_config parameter to resolve_app_config function with default True
- Conditionally log config loading messages based on log_config flag
- Update reme.py and service_utils.py to use log_config=False for client calls
- Suppress config logging in user-facing contexts to avoid output pollution

refactor(shell): rename command parameter to cmd for clarity

- Change 'command' to 'cmd' in default.yaml configuration schema
- Rename 'timeout' to 'shell_timeout' to avoid parameter name collisions
- Update ShellStep to accept both legacy and new parameter names
- Maintain backward compatibility with existing command/timeout usage

test(shell): add comprehensive tests for shell step parameter handling

- Add test cases for new cmd and shell_timeout parameter names
- Verify legacy command and timeout parameters still work
- Test blank command rejection message updated to use cmd
- Create integration test for shell parameter payload passing

* fix(shell): ensure proper environment loading and process timeout handling

- Move load_env() call to execute before parse_args() in main function
- Add proper process group killing for timeout scenarios on POSIX systems
- Implement recursive child process termination on Windows for proper cleanup
- Change parameter name from 'timeout' to 'shell_timeout' in shell execution
- Remove support for legacy 'command' and 'timeout' parameter names
- Update test cases to verify new timeout behavior and parameter requirements
- Add comments explaining component size tracking implementation details
2026-07-14 16:31:41 +08:00
Sen Huang
8877743ca9
feat(cli): route bare commands to the running server's real config (#312)
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Deploy Docs / deploy (push) Has been cancelled
Deploy Docs / build (push) Has been cancelled
call_server now resolves the client backend/transport/host/port from the
live `reme start` process (replaying its start args through
resolve_app_config) so a bare `reme <action>` reaches the server however
it was actually launched, falling back to local config when none runs.
Explicit backend=/transport=/host=/port= still win.

Also fix as_embedding to pass `dimensions` explicitly for agentscope
>=2.0.2, and add Claude Code auto-memory/auto-dream demos to the READMEs.
2026-07-01 17:15:48 +08:00
Sen Huang
be7d1c0cf2
refactor(transfer): drop orphaned ingest step, make service discovery cross-platform (#300)
* refactor(transfer): drop orphaned ingest step, make service discovery cross-platform

- Remove ingest step: superseded by auto_resource (drop files under
  resource/ → watcher interprets them); its meta.json/<date>.md outputs
  had no consumers and tripped the auto_resource watcher.
- Replace lsof/pgrep shell-outs in service_utils with psutil (per-process
  enumeration, no root needed on macOS) for Windows/macOS/Linux support.
- Add cross-platform test coverage for _pid_on_port / _scan_reme_procs.
- Deps: +psutil, -filelock (only used by the removed ingest lock).

* chore(release): bump version to 0.4.0.5
2026-06-29 14:49:23 +08:00
Sen Huang
e31db5fe19
docs: rename vault_dir to workspace_dir in documentation and examples (#286)
* docs: rename vault_dir to workspace_dir in documentation and examples

* refactor(extract): format long method call across multiple lines

* refactor(extract): format system prompt parameters for better readability
2026-06-22 16:58:57 +08:00
jinliyl
206a53e5ed
init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00