ReMe/reme/components/service/base_service.py
jinliyl e7d44f6f3b
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
Windows Smoke / CLI smoke - py3.11 (push) Waiting to run
refactor(agent): unify agent subprocess env, sessions, skills, and MCP/service jobs (#382)
* feat(config): add environment variable configuration for agent subprocesses

- Add environment field to ApplicationConfig to store variables for agent subprocesses
- Remove dynamic loading of .env files in agent wrappers
- Introduce subprocess_environment property in base agent wrapper
- Pass application-level environment variables to Claude Code and Codex agents
- Load environment variables once at startup and pass to ReMe application
- Remove dependency on load_env utility in agent wrapper implementations
- Update tests to use configured environment instead of dynamic loading
- Remove unused environment loading utilities and related test cases

* refactor(mcp): remove channel notification system and related components

- Removed channel notification step implementation
- Removed claim channel step implementation
- Removed ChannelSink class from MCP service
- Removed channel-related documentation from AGENTS.md
- Removed channel instruction text from MCP service
- Removed all channel-related tests
- Updated application context metadata comment to remove channel sink reference
- Removed channel module initialization and imports

* feat(service): add job whitelisting capability to BaseService

- Add optional jobs parameter to BaseService.__init__ to configure job whitelist
- Store jobs as set in self.jobs attribute for efficient lookup operations
- Modify add_jobs method to filter jobs based on whitelist configuration
- Update documentation in both English and Chinese to describe new feature
- Add comprehensive unit tests for job whitelisting behavior
- Implement flowchart update showing new filtering logic
- Preserve existing enable_serve flag behavior alongside new whitelisting

* refactor(service): enhance service job validation and MCP tool injection

- Add strict validation for service jobs whitelist with detailed error messages
- Implement injected job arguments support for MCP services with conflict detection
- Add tool error handling for unsuccessful responses in MCP services
- Remove duplicate job names in Codex agent wrapper using dict.fromkeys
- Update MCP server argument format from single JSON array to repeated --job flags
- Add comprehensive test coverage for job injection and error handling scenarios
- Update documentation to reflect service job validation and MCP features
- Ensure application cleanup occurs even when service lifespan encounters errors

* feat(agent): update skill handling to preserve existing Claude skills

- Change skills parameter processing to use 'all' instead of filtered list
- Add logic to select project skills without restricting Claude's existing skills
- Update variable naming from 'skills' to 'selected_skills' for clarity
- Modify application context metadata documentation to clarify in-memory state usage
- Add test case to verify configured skills are added without filtering existing skills
- Update internal skill directory handling to use renamed variable consistently

* refactor(agent): restructure agent wrapper components and session storage

- Move CcFileSessionStore to separate module for better organization
- Add SDK package version logging in base agent wrapper
- Update Claude Code agent to use new session store structure with project keys
- Refactor Claude Code agent wrapper to use proper type hints and SDK integration
- Add support for server tool use events in Claude Code message processing
- Improve error handling and resource cleanup in streaming operations
- Update Codex agent wrapper with proper type annotations and configuration
- Remove deprecated system prompt mode handling from Claude Code wrapper
- Fix session path construction for Claude Code transcript storage
- Update dependency injection and configuration handling patterns

* fix(cc_agent_wrapper): resolve Claude Code SDK integration issues

- Added dataclass import and created _BlockState for content block metadata tracking
- Implemented proper MCP server name constant and tool context ID validation
- Fixed tool_context_id injection to prevent duplicate assignment errors
- Resolved skills parameter handling in build_options method
- Enhanced job tools integration with MCP servers mapping validation
- Replaced deprecated block_ids/block_types/tool_call_names with block_states dict
- Updated message_delta to emit USAGE chunks instead of REPLY_END
- Fixed stream result handling to ensure proper REPLY_END emission
- Improved error handling for session mirror failures and rate limits
- Added proper cleanup for expected trailing errors in streams
- Refactored Codex agent wrapper initialization and configuration management
- Removed obsolete system_prompt_mode from default config
- Enhanced test coverage for new block state and error handling features
- Fixed async generator handling with aclosing context manager
- Improved chunk type mapping for Claude Code SDK events

* refactor(tests): remove demo config tests from config parser test suite

- Removed test_demo_config_registers_llm_jobs function and its assertions
- Eliminated verification of LLM demo job configurations
- Removed checks for agent wrapper component settings
- Deleted assertions for model configurations and parameters
- Cleaned up deprecated test cases related to demo config parsing

* refactor(evolve): simplify Claude Code session store path structure

- Removed redundant project key subdirectory from session link generation
- Updated CcFileSessionStore initialization to use direct session directory path
- Maintained existing session layout compatibility for backward compatibility
- Added unit tests to verify session persistence behavior with existing transcripts
- Ensured UUID-based session files remain accessible at expected locations
- Preserved existing session directory structure without additional nesting

* refactor(agent): defer optional Codex SDK imports until first use

- Moved openai-codex imports inside functions to avoid mandatory dependencies
- Added TYPE_CHECKING guard for development time type checking only
- Implemented lazy loading mechanism with _get_async_codex_class function
- Updated AsyncCodex initialization to occur on demand rather than at module level
- Maintained backward compatibility while improving import performance
- Added test case to verify package import works without optional Codex SDK
- Updated agentscope dependency to version 2.0.4.post1 in pyproject.toml

* test(embedded): add compatibility tests for in-process ReMe embedding

- Add test suite for QwenPaw-style embedded configurations
- Verify optional defaults remain preserved in embedded configs
- Ensure in-process application API stays compatible
- Test model injection and lifecycle management compatibility
- Remove obsolete hermes agent plugin tests
- Update CLI import test to cover multiple optional SDKs
- Block claude_agent_sdk and openai_codex during import testing
2026-07-20 23:52:14 +08:00

100 lines
3.8 KiB
Python

"""Base class for services that expose jobs over a network protocol."""
import json
import os
from abc import abstractmethod
from contextlib import asynccontextmanager
from typing import TYPE_CHECKING
from ..base_component import BaseComponent
from ..job.base_job import BaseJob
from ...constants import REME_SERVICE_INFO
from ...enumeration import ComponentEnum
if TYPE_CHECKING:
from ...application import Application
class BaseService(BaseComponent):
"""Skeleton for services (HTTP, MCP, ...) that turn jobs into endpoints or tools."""
component_type = ComponentEnum.SERVICE
def __init__(self, jobs: list[str] | None = None, **kwargs):
super().__init__(**kwargs)
self.jobs: set[str] | None = set(jobs) if jobs is not None else None
# Underlying framework instance (FastAPI, FastMCP, ...); populated by build_service().
self.service = None
# ----- Subclass contract ---------------------------------------------
@abstractmethod
def build_service(self, app: "Application") -> None:
"""Instantiate and configure the underlying server framework."""
@abstractmethod
def add_job(self, job: BaseJob) -> bool:
"""Register a single job as a callable endpoint or tool.
Returns True when the job is exposed, False when the service intentionally
skips it (for example, unsupported job types).
"""
@abstractmethod
def start_service(self, app: "Application") -> None:
"""Block on serving requests until shutdown."""
# ----- Shared helpers ------------------------------------------------
def _lifespan(self, app: "Application", host: str, port: int):
"""Build an async-context lifespan that brackets the server with app start/close.
Publishes the bound address via the REME_SERVICE_INFO environment variable so
in-process clients can discover where this service is listening.
"""
@asynccontextmanager
async def lifespan(_):
await app.start()
try:
service_info = json.dumps({"host": host, "port": port})
os.environ[REME_SERVICE_INFO] = service_info
self.logger.info(f"{self.name} started: {REME_SERVICE_INFO}={service_info}")
yield
finally:
await app.close()
return lifespan
def add_jobs(self, app: "Application") -> None:
"""Register service-enabled jobs, optionally restricted by the configured whitelist."""
if self.jobs is not None:
missing = sorted(self.jobs.difference(app.context.jobs))
if missing:
raise KeyError(f"Service jobs not found: {', '.join(missing)}")
disabled = sorted(name for name in self.jobs if not app.context.jobs[name].enable_serve)
if disabled:
raise ValueError(f"Service jobs are not enabled for serving: {', '.join(disabled)}")
for name, job in app.context.jobs.items():
if not job.enable_serve or (self.jobs is not None and name not in self.jobs):
continue
try:
added = self.add_job(job)
except Exception as e:
if self.jobs is not None:
raise
self.logger.error(f"Failed to add job {name}: {e}")
continue
if added:
self.logger.info(f"Added job: {name}")
elif self.jobs is not None:
raise TypeError(f"Service does not support job: {name}")
else:
self.logger.warning(f"Skipped job: {name}")
def run_app(self, app: "Application") -> None:
"""Build the service, register jobs, then start serving (blocking)."""
self.build_service(app)
self.add_jobs(app)
self.start_service(app)