ReMe/reme4/steps/file_io/edit.py
jinliyl 041f957a7f
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.10 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
refactor(components) components and file I/O, fix method calls and validation (#268)
* refactor(components): extract shared component state into mixin

- Introduce ComponentMixin class with shared state for components and steps
- Move identity, config, and vault path functionality to ComponentMixin
- Update BaseComponent to inherit from ComponentMixin
- Update BaseStep to inherit from ComponentMixin
- Consolidate vault path helper methods in ComponentMixin
- Remove duplicate vault path implementations from BaseComponent and BaseStep
- Add ComponentMixin to components module exports

* refactor(file_io): implement path locks cache eviction mechanism

- Add _PATH_LOCKS_MAX constant set to 1024 for cache size limit
- Implement cache eviction logic when locks exceed maximum capacity
- Remove half of unlocked entries when cache limit is reached
- Use list comprehension to identify unlocked locks for removal
- Maintain existing path normalization and locking behavior

fix(edit): correct method call from public to private fail method

- Change self.fail to self._fail for internal error handling
- Maintain consistent private method usage within class

fix(mcp_client): change pop to get for optional command and args

- Replace kwargs.pop with kwargs.get to avoid removing keys
- Preserve original kwargs dictionary contents
- Maintain default empty string and list values

feat(reme): add client backend validation with error raising

- Check if client_cls is None before instantiation
- Raise ValueError with descriptive message for unknown backends
- Provide clear error feedback for invalid backend configurations

* fix(components): move directory creation to start method

- Moved component_metadata_path.mkdir call from __init__ to _start in base_keyword_index
- Moved component_metadata_path.mkdir call from __init__ to _start in local_file_graph
- Moved component_metadata_path.mkdir call from __init__ to _start in local_file_store
- Ensures directory creation happens after component initialization
- Prevents potential issues with path creation during object construction

* fix(steps): replace assertions with runtime errors for app_context validation

- Replace assert statements with explicit RuntimeError exceptions when app_context is None
- Add descriptive error messages for better debugging when resolving components
- Replace assert in resolve_component method with proper exception handling
- Replace assert in get_file_parser method with proper exception handling
- Maintain same functionality while improving error reporting clarity

* refactor(file_io): split file IO utilities into modular components

- Move daily note helpers to separate _daily_index module
- Extract path validation and resolution to new _path module
- Remove unused code and imports from _file_io module
- Update import statements across affected modules
- Introduce WikilinkHandler utility for link parsing
- Replace regex-based link extraction with WikilinkHandler
- Add integration JSONL files to gitignore
- Consolidate file locking mechanism in _file_io module

* style(formatter): fix spacing issues in file IO and chunked file parser

- Fixed whitespace around colon in slice notation in file_io.py
- Corrected spacing around colon in slice notation in chunked_file_parser.py
- Applied consistent formatting for array slicing operations
- Improved code readability by standardizing space placement in ranges

* refactor(steps): replace property-based component resolution with Ref descriptor

- Introduce Ref descriptor class for lazy component dependency resolution
- Replace _resolve method and individual properties with Ref descriptors
- Add as_llm, as_llm_formatter, as_token_counter, file_store, and embedding Ref attributes
- Remove legacy property methods and resolve logic from BaseStep
- Add cache clearing mechanism for Ref values during step calls
- Update UpdateCatalogStep to use Ref instead of property-based resolution
2026-06-01 11:35:19 +08:00

116 lines
4.6 KiB
Python

"""Find-and-replace text in a markdown file body (front matter is preserved)."""
import frontmatter
import yaml
from ._file_io import get_path_lock, read_file_safe, write_file_safe
from ._path import NON_MD_WARNING, gate_md, resolve_path
from ..base_step import BaseStep
from ...components import R
@R.register("edit_step")
class EditStep(BaseStep):
"""Replace every occurrence of ``old`` with ``new`` inside the file body.
The YAML front matter block (if any) is parsed out, kept verbatim and
re-emitted unchanged — matches that fall inside front matter are ignored,
so a typo in `old` cannot corrupt structured metadata.
Concurrency: in-process per-path ``asyncio.Lock`` serializes the
read-modify-write cycle against the same file (multi-worker / multi-
process safety is out of scope)."""
def _fail(self, message: str, **meta) -> None:
assert self.context is not None
self.context.response.success = False
self.context.response.answer = f"Error: {message}"
if meta:
self.context.response.metadata.update(meta)
async def execute(self): # pylint: disable=too-many-return-statements
assert self.context is not None
raw = str(self.context.get("path") or "")
old = self.context.get("old")
new = self.context.get("new")
if old is None or str(old) == "":
self._fail("`old` is required and must be non-empty")
return None
if new is None:
self._fail("`new` is required")
return None
old_str = str(old)
new_str = str(new)
target, err = resolve_path(self.vault_path, raw)
if err:
self._fail(err)
return None
target, is_md = gate_md(target)
lock = await get_path_lock(target)
async with lock:
if not target.exists():
self._fail(f"file {target} does not exist", path=str(target))
return None
if not target.is_file():
self._fail(f"path {target} is not a file", path=str(target))
return None
try:
raw_text, encoding = await read_file_safe(target)
except Exception as e: # pylint: disable=broad-except
self._fail(f"read failed: {e}", path=str(target))
return None
# Markdown: parse frontmatter and operate on body only. Non-markdown:
# there's no frontmatter convention, so operate on the full text.
if is_md:
try:
post = frontmatter.loads(raw_text)
except yaml.YAMLError as exc:
self._fail(f"failed to parse frontmatter in {target}: {exc}", path=str(target))
return None
body = post.content
not_found_msg = (
f"text to replace was not found in the body of {target} (front matter is excluded from edit)"
)
else:
post = None
body = raw_text
not_found_msg = f"text to replace was not found in {target}"
if old_str not in body:
self._fail(not_found_msg, path=str(target))
return None
count = body.count(old_str)
new_body = body.replace(old_str, new_str)
if is_md and post is not None:
post.content = new_body
# Re-serialize: keep front matter when present, otherwise emit body alone
# so we don't introduce an empty `---\n---\n` block.
new_text = frontmatter.dumps(post) if post.metadata else post.content
if not new_text.endswith("\n"):
new_text += "\n"
else:
new_text = new_body
# Preserve the file's original encoding (returned by read_file_safe above)
# so edits don't silently re-encode non-UTF-8 files (e.g. GBK CSV) to UTF-8.
try:
await write_file_safe(target, new_text, encoding=encoding)
except Exception as e: # pylint: disable=broad-except
self._fail(f"write failed: {e}", path=str(target))
return None
self.context.response.success = True
answer = f"Replaced {count} occurrence(s) in {target}"
if not is_md:
answer = f"{answer} [system notice: {NON_MD_WARNING}]"
self.context.response.answer = answer
self.logger.info(f"[{self.name}] edited path={target} count={count} is_md={is_md}")
return self.context.response