ReMe/reme4/components/base_component.py
jinliyl 041f957a7f
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.10 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
refactor(components) components and file I/O, fix method calls and validation (#268)
* refactor(components): extract shared component state into mixin

- Introduce ComponentMixin class with shared state for components and steps
- Move identity, config, and vault path functionality to ComponentMixin
- Update BaseComponent to inherit from ComponentMixin
- Update BaseStep to inherit from ComponentMixin
- Consolidate vault path helper methods in ComponentMixin
- Remove duplicate vault path implementations from BaseComponent and BaseStep
- Add ComponentMixin to components module exports

* refactor(file_io): implement path locks cache eviction mechanism

- Add _PATH_LOCKS_MAX constant set to 1024 for cache size limit
- Implement cache eviction logic when locks exceed maximum capacity
- Remove half of unlocked entries when cache limit is reached
- Use list comprehension to identify unlocked locks for removal
- Maintain existing path normalization and locking behavior

fix(edit): correct method call from public to private fail method

- Change self.fail to self._fail for internal error handling
- Maintain consistent private method usage within class

fix(mcp_client): change pop to get for optional command and args

- Replace kwargs.pop with kwargs.get to avoid removing keys
- Preserve original kwargs dictionary contents
- Maintain default empty string and list values

feat(reme): add client backend validation with error raising

- Check if client_cls is None before instantiation
- Raise ValueError with descriptive message for unknown backends
- Provide clear error feedback for invalid backend configurations

* fix(components): move directory creation to start method

- Moved component_metadata_path.mkdir call from __init__ to _start in base_keyword_index
- Moved component_metadata_path.mkdir call from __init__ to _start in local_file_graph
- Moved component_metadata_path.mkdir call from __init__ to _start in local_file_store
- Ensures directory creation happens after component initialization
- Prevents potential issues with path creation during object construction

* fix(steps): replace assertions with runtime errors for app_context validation

- Replace assert statements with explicit RuntimeError exceptions when app_context is None
- Add descriptive error messages for better debugging when resolving components
- Replace assert in resolve_component method with proper exception handling
- Replace assert in get_file_parser method with proper exception handling
- Maintain same functionality while improving error reporting clarity

* refactor(file_io): split file IO utilities into modular components

- Move daily note helpers to separate _daily_index module
- Extract path validation and resolution to new _path module
- Remove unused code and imports from _file_io module
- Update import statements across affected modules
- Introduce WikilinkHandler utility for link parsing
- Replace regex-based link extraction with WikilinkHandler
- Add integration JSONL files to gitignore
- Consolidate file locking mechanism in _file_io module

* style(formatter): fix spacing issues in file IO and chunked file parser

- Fixed whitespace around colon in slice notation in file_io.py
- Corrected spacing around colon in slice notation in chunked_file_parser.py
- Applied consistent formatting for array slicing operations
- Improved code readability by standardizing space placement in ranges

* refactor(steps): replace property-based component resolution with Ref descriptor

- Introduce Ref descriptor class for lazy component dependency resolution
- Replace _resolve method and individual properties with Ref descriptors
- Add as_llm, as_llm_formatter, as_token_counter, file_store, and embedding Ref attributes
- Remove legacy property methods and resolve logic from BaseStep
- Add cache clearing mechanism for Ref values during step calls
- Update UpdateCatalogStep to use Ref instead of property-based resolution
2026-06-01 11:35:19 +08:00

241 lines
8.5 KiB
Python

"""Base class for components with async lifecycle and dependency injection."""
import asyncio
from abc import ABC
from pathlib import Path
from typing import TYPE_CHECKING, Any, Callable, TypeVar, cast
from ..enumeration import ComponentEnum
from ..utils import get_logger
if TYPE_CHECKING:
from .application_context import ApplicationContext
T = TypeVar("T", bound="BaseComponent")
class ComponentMixin:
"""Shared state for components and steps: identity, config, vault paths."""
component_type = ComponentEnum.BASE
def __init__(
self,
name: str | None = None,
backend: str = "",
app_context: "ApplicationContext | None" = None,
**kwargs,
) -> None:
self.name: str = name or self.__class__.__name__
self.backend: str = backend
self.app_context: "ApplicationContext | None" = app_context
self.kwargs: dict = dict(kwargs)
logger = get_logger()
self.logger = logger.bind(component=self.name) if hasattr(logger, "bind") else logger
@property
def vault_path(self) -> Path:
"""Absolute vault root directory (cwd when no app_context is attached)."""
if self.app_context is None:
return Path.cwd()
return Path(self.app_context.app_config.vault_dir).absolute()
def to_vault_relative(self, path: str | Path) -> str:
"""Convert `path` to a vault-relative string; return absolute path when outside."""
abs_path = Path(path).absolute()
try:
return str(abs_path.relative_to(self.vault_path))
except ValueError:
return str(abs_path)
class Dependency:
"""Placeholder returned by ``BaseComponent.bind`` for an unresolved dependency.
Resolved into a real component (or None) when the owning component starts.
Accessing any attribute before resolution raises a clear error.
"""
__slots__ = ("ctype", "name", "default_factory", "optional")
def __init__(
self,
ctype: ComponentEnum,
name: str,
default_factory: Callable[[], Any] | None = None,
optional: bool = True,
) -> None:
self.ctype = ctype
self.name = name
self.default_factory = default_factory
self.optional = optional
def __repr__(self) -> str:
suffix = "?" if self.optional else ""
return f"<unresolved {self.ctype.value}:{self.name}{suffix}>"
def __getattr__(self, item: str) -> Any:
# Catches accidental use of the placeholder before start() resolves it.
raise RuntimeError(
f"Dependency {self.ctype.value}:{self.name} accessed before start() " f"(attribute '{item}')",
)
class BaseComponent(ComponentMixin, ABC):
"""Async lifecycle base class with bind-based dependency injection."""
component_type = ComponentEnum.BASE
def __init__(
self,
name: str | None = None,
backend: str = "",
app_context: "ApplicationContext | None" = None,
**kwargs,
) -> None:
super().__init__(name=name, backend=backend, app_context=app_context, **kwargs)
self._is_started: bool = False
self._lock: asyncio.Lock = asyncio.Lock()
# Components created via bind() default_factory in standalone mode;
# their lifecycle is owned by this component.
self._owned: list["BaseComponent"] = []
@property
def is_started(self) -> bool:
"""Whether the component has been started."""
return self._is_started
# ----- Dependency declaration ----------------------------------------
@staticmethod
def bind(
name: str | None,
base_cls: type[T],
*,
default_factory: Callable[[], T] | None = None,
optional: bool = True,
) -> T | None:
"""Declare a dependency on another component.
Returns a ``Dependency`` placeholder resolved into the real component
(or None / a factory-produced instance) when ``start`` runs. An empty
`name` short-circuits to None so callers can skip optional wiring.
"""
if not name:
return None
ctype = getattr(base_cls, "component_type", None)
if not isinstance(ctype, ComponentEnum) or ctype is ComponentEnum.BASE:
raise TypeError(
f"{base_cls.__name__} must declare a non-BASE ComponentEnum 'component_type'",
)
return cast(T, Dependency(ctype, name, default_factory, optional))
@property
def dependencies(self) -> list[Dependency]:
"""All unresolved dependency placeholders on this instance."""
return [v for v in self.__dict__.values() if isinstance(v, Dependency)]
async def _resolve_bindings(self) -> None:
"""Replace every ``Dependency`` attribute with its resolved target."""
for attr, dep in list(self.__dict__.items()):
if isinstance(dep, Dependency):
self._resolve_one(attr, dep)
def _resolve_one(self, attr: str, dep: Dependency) -> None:
"""Resolve a single dependency, dispatching by mode."""
if self.app_context is None:
self._resolve_standalone(attr, dep)
else:
self._resolve_from_context(attr, dep)
def _resolve_standalone(self, attr: str, dep: Dependency) -> None:
"""Standalone mode: use default_factory, or fall back to None when optional.
Required dependencies without a factory keep the placeholder so any
attribute access surfaces a clear error at the call site.
"""
if dep.default_factory is not None:
instance = dep.default_factory()
setattr(self, attr, instance)
if isinstance(instance, BaseComponent):
self._owned.append(instance)
elif dep.optional:
setattr(self, attr, None)
def _resolve_from_context(self, attr: str, dep: Dependency) -> None:
"""Context-bound mode: look up the component from ``app_context.components``."""
target = self.app_context.components.get(dep.ctype, {}).get(dep.name)
if target is not None:
setattr(self, attr, target)
elif dep.optional:
setattr(self, attr, None)
else:
raise ValueError(f"{dep.ctype.value} '{dep.name}' not found.")
# ----- Vault path helpers --------------------------------------------
@property
def vault_metadata_path(self) -> Path:
"""Vault metadata directory: ``<vault>/<metadata_dir>``."""
if self.app_context is None:
return Path.cwd() / "metadata"
return self.vault_path / self.app_context.app_config.metadata_dir
@property
def component_metadata_path(self) -> Path:
"""Per-component metadata directory under the vault."""
return self.vault_metadata_path / self.component_type.value
# ----- Lifecycle hooks (override in subclasses) ----------------------
async def _start(self) -> None:
"""Subclass hook called once after dependencies are resolved."""
async def _close(self) -> None:
"""Subclass hook called once during ``close``."""
async def dump(self) -> None:
"""Persist in-memory state to disk. Override when persistence is needed."""
async def load(self) -> None:
"""Restore in-memory state from disk. Override when persistence is needed."""
# ----- Lifecycle control --------------------------------------------
async def start(self) -> None:
"""Start the component once: resolve deps → start owned → run _start."""
async with self._lock:
if self._is_started:
return
await self._resolve_bindings()
for owned in self._owned:
await owned.start()
await self._start()
self._is_started = True
async def close(self) -> None:
"""Close the component once: run _close → close owned in reverse order."""
async with self._lock:
if not self._is_started:
return
await self._close()
for owned in reversed(self._owned):
await owned.close()
self._is_started = False
async def restart(self) -> None:
"""Close then start the component."""
await self.close()
await self.start()
async def __call__(self, **kwargs):
raise NotImplementedError
async def __aenter__(self) -> "BaseComponent":
await self.start()
return self
async def __aexit__(self, exc_type, exc_val, exc_tb) -> None:
await self.close()