This commit is contained in:
celanwang 2026-11-04 14:40:26 +08:00 • committed by GitHub
commit 7410fc5690
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
50 changed files with 3004 additions and 307 deletions

View file

@ -132,8 +132,12 @@ Treat Step instances as invocation-scoped:
- `self.context.response.answer`, `success`, and `metadata` are request-scoped output. Because the same response travels
through the Step chain, later Steps may consume metadata produced earlier, but it is not application-lifetime or
durable storage.
- `self.app_context.metadata` holds in-memory state shared across Job/Step invocations for the life of one
`Application`, such as counters, tool-context state, session maps, or locks.
- `self.app_context.metadata` holds application-lifetime operational state shared across Job/Step invocations,
such as counters. `ApplicationContext.shared_components` owns genuinely shared components; its `components`
compatibility view also exposes default-workspace instances to legacy callers.
- Each `WorkspaceContext` owns its paths, stateful components, tool contexts, search budgets, agent session state,
and locks. `RuntimeContext.workspace` is the explicit binding used by workspace-bound execution; internal Steps
must not resolve a default workspace repeatedly or use the compatibility component view.
- Workspace files or a dedicated Component/store hold durable state that must survive restart.
Use narrow, namespaced keys in `app_context.metadata` and protect shared mutable values against concurrent access. The
@ -144,8 +148,9 @@ dedicated lifecycle, locking, or persistence, promote it to a typed context fiel
Additional Step contracts:
- `Ref` dependencies resolve in this order: Step kwargs, current `RuntimeContext`, then the named application component.
The value is cached only on the current Step instance and cleared before each call.
- `Ref` dependencies resolve in this order: Step kwargs, current `RuntimeContext`, then the bound workspace for scoped
components or the application shared-component registry for shared components. The value is cached only on the
current Step instance and cleared before each call.
- `input_mapping` and `output_mapping` copy keys within `RuntimeContext.data`; missing sources are ignored.
- Dispatched Steps receive the current `RuntimeContext`, so their data and response are shared.
- Base jobs convert uncaught Step errors into `Response(success=False)`; stream jobs emit an error chunk and always a

View file

@ -78,6 +78,54 @@ Keep secrets in `.env` or the process environment, never in committed configurat
`session_dir` must remain workspace-relative.
`workspace_dir` also accepts an ordered mapping of alias to path for hosts that serve several workspaces from one
configuration, for example in a YAML config file:
```yaml
workspace_dir:
main: /data/reme/main
scratch: /data/reme/scratch
```
The first entry is the default workspace; every alias gets the same subdirectory layout (`metadata_dir`,
`session_dir`, and the others above) materialized under its own root at startup. Aliases must be non-empty, and
resolved directories must be distinct and must not contain one another. A single string keeps the existing
behavior and is normalized internally to a `default` alias. Multi-workspace configurations also require a non-empty
`mem_session_dir` so agent thread ownership remains enforceable after restart.
Job calls accept an optional `workspace` field naming the alias to bind, for example
`reme read_note path=daily/note.md workspace=scratch`; omitting it binds the default workspace, and an unknown
alias fails fast. The binding is scoped to that call: workspace-aware state (file stores, indexes, graphs,
catalogs) resolves to per-alias instances, while shared components (models, tokenizers) stay single-instance.
External stateful backends that cannot isolate workspaces (currently the `neo4j` file graph) are rejected at
startup in multi-workspace configurations.
The binding propagates through nested job calls and agent tool calls (`job_tools`) as a server-owned constraint
that models cannot override. For Codex with multiple workspaces, `job_tools` requires the resident HTTP service
with MCP enabled and `tool_error_on_failure: true`; the selected jobs must be exposed by that service. Its STDIO
bridge forwards to this service without creating another Application and preserves failed Jobs as tool errors.
The multi-workspace one-shot CLI combination is not supported; `mcp_config` retains its legacy meaning only for
single-workspace Codex job tools.
Background and cron jobs bind through their job configuration, so extra workspaces are watched by declaring
explicit per-alias variants:
```yaml
jobs:
index_update_loop:
backend: background
# watches the default workspace; workspace defaults to the first alias
steps: [...]
index_update_loop_scratch:
backend: background
workspace: scratch
steps: [...]
```
Two scoping notes: the Claude Code integration reads transcripts from the global `~/.claude/projects` directory
(a Claude Code-side limitation — only ReMe's own copies are workspace-scoped), and token usage accounting remains
application-level across workspaces.
## Enabling jobs
`jobs.<name>.enabled` defaults to `true` for all Job types. Disabled jobs retain their configuration but do not

View file

@ -83,6 +83,49 @@ base_url: ${LLM_BASE_URL:-https://example.com/v1}
`session_dir` 必须保持 workspace-relative。其他 workspace 子目录也应使用清晰、稳定的相对名称。
`workspace_dir` 也接受"别名到路径"的有序映射,便于一个配置服务多个 workspace,例如在 YAML 配置文件中:
```yaml
workspace_dir:
main: /data/reme/main
scratch: /data/reme/scratch
```
第一个 entry 是默认 workspace;每个别名在启动时都会在自己的根目录下创建相同的子目录布局
(`metadata_dir`、`session_dir` 等上述目录)。别名必须非空,且解析后的目录必须互不相同、不能互相包含。
单个字符串保持原有行为,内部归一化为 `default` 别名。多 workspace 配置还要求 `mem_session_dir` 非空,
以确保重启后仍能强制执行 agent thread 的归属隔离。
Job 调用接受可选的 `workspace` 字段指定绑定的别名,例如
`reme read_note path=daily/note.md workspace=scratch`;缺省绑定默认 workspace,未知别名会直接报错。
绑定仅在单次调用内生效:workspace 相关的状态组件(file store、索引、graph、catalog)按别名解析到各自实例,
共享组件(模型、tokenizer 等)保持单实例。无法隔离 workspace 的外部有状态后端(目前是 `neo4j` file graph)
在多 workspace 配置下会在启动时被拒绝。
绑定会经嵌套 Job 调用和 agent 工具调用(`job_tools`)传播,作为模型无法覆盖的服务端约束。
多 workspace 下的 Codex `job_tools` 需要启用 MCP 且配置 `tool_error_on_failure: true` 的常驻 HTTP 服务,
且所选 Job 必须由该服务暴露;其 STDIO bridge 转发到该服务,不会创建另一个 Application,并会将失败的
Job 保留为 tool error。多 workspace 与 one-shot CLI 的组合暂不支持;`mcp_config` 仅在单 workspace 的
Codex job tools 中保留原有语义。
Background 和 cron Job 通过 Job 配置绑定别名,额外 workspace 的 watch 通过显式配置按别名的变体实现:
```yaml
jobs:
index_update_loop:
backend: background
# 监控默认 workspace;workspace 缺省为第一个别名
steps: [...]
index_update_loop_scratch:
backend: background
workspace: scratch
steps: [...]
```
两点作用域说明:Claude Code 集成从全局 `~/.claude/projects` 目录读取 transcript
(这是 Claude Code 源端的固有限制,ReMe 自己的副本按 workspace 隔离);token 用量统计保持 Application 级,
不随 workspace 拆分。
## Job 启停
`jobs.<name>.enabled` 默认为 `true`,适用于所有 Job 类型。禁用的 Job 保留配置,但不启动、不暴露接口,

View file

@ -5,7 +5,6 @@ import heapq
from collections.abc import Mapping
from concurrent.futures import ThreadPoolExecutor
from inspect import getattr_static
from pathlib import Path
from typing import Any, AsyncGenerator, TypeVar
from . import __version__
@ -18,7 +17,8 @@ from .schema import ComponentConfig, Response, StreamChunk
from .utils import execute_stream_task, print_logo, get_logger
T = TypeVar("T", bound=BaseComponent)
_NodeKey = tuple[str, str]
# (component type, component name, workspace alias); "" marks shared instances.
_NodeKey = tuple[str, str, str]
_UNSET = object()
@ -54,20 +54,11 @@ class Application(BaseComponent):
# ----- Wiring (called once during __init__) --------------------------
def _setup_workspace_directories(self) -> None:
"""Ensure the workspace root and configured subdirectories exist on disk."""
cfg = self.config
workspace_path = Path(cfg.workspace_dir).absolute()
workspace_path.mkdir(parents=True, exist_ok=True)
for subdir in [
cfg.metadata_dir,
cfg.session_dir,
cfg.mem_session_dir,
cfg.resource_dir,
cfg.daily_dir,
cfg.digest_dir,
]:
if subdir:
(workspace_path / subdir).mkdir(parents=True, exist_ok=True)
"""Ensure every configured workspace root and its configured subdirectories exist on disk."""
for workspace in self.context.workspaces.values():
workspace.workspace_path.mkdir(parents=True, exist_ok=True)
for subdir in workspace.storage_dirs():
subdir.mkdir(parents=True, exist_ok=True)
def _init_service(self) -> None:
"""Instantiate the single service backend declared in config.service."""
@ -79,17 +70,35 @@ class Application(BaseComponent):
)
def _init_components(self) -> None:
"""Instantiate every component declared under config.components."""
"""Instantiate declared components; workspace-scoped ones once per alias."""
multi_workspace = len(self.context.workspaces) > 1
for ctype, group in self.config.components.items():
self.context.components[ctype] = {}
for name, cfg in group.items():
self.context.components[ctype][name] = self._instantiate(
ctype,
cfg,
label=f"Component '{name}'",
expected_type=BaseComponent,
name=name,
)
label = f"Component '{name}'"
backend_cls = self._backend_class(ctype, cfg, label)
if multi_workspace and not backend_cls.supports_multi_workspace:
raise ValueError(
f"{label} backend '{cfg.backend}' does not support multi-workspace configurations",
)
if backend_cls.workspace_scoped:
for alias, workspace in self.context.workspaces.items():
instance = self._instantiate(
ctype,
cfg,
label=label,
expected_type=BaseComponent,
name=name,
workspace_alias=alias,
)
workspace.components.setdefault(ctype, {})[name] = instance
else:
self.context.shared_components.setdefault(ctype, {})[name] = self._instantiate(
ctype,
cfg,
label=label,
expected_type=BaseComponent,
name=name,
)
def _init_jobs(self) -> None:
"""Instantiate every job declared under config.jobs."""
@ -102,6 +111,15 @@ class Application(BaseComponent):
name=name,
)
def _backend_class(self, ctype: ComponentType, cfg: ComponentConfig, label: str) -> type[T]:
"""Resolve cfg.backend to a registered class without instantiating it."""
if not cfg.backend:
raise ValueError(f"{label} is missing the required 'backend' field")
backend_cls = self.context.registry.get(ctype, cfg.backend)
if backend_cls is None:
raise ValueError(f"Unregistered backend '{cfg.backend}' for {label}")
return backend_cls
def _instantiate(
self,
ctype: ComponentType,
@ -110,6 +128,7 @@ class Application(BaseComponent):
label: str,
expected_type: type[T],
name: str | None = None,
workspace_alias: str | None = None,
) -> T:
"""Resolve cfg.backend through the registry and construct the instance.
@ -118,17 +137,16 @@ class Application(BaseComponent):
registered under the wrong component type.
`name` is forwarded to the constructor for named components/jobs;
leave it None for the service, which is keyed solely by type.
`workspace_alias` binds workspace-scoped instances to one workspace.
"""
if not cfg.backend:
raise ValueError(f"{label} is missing the required 'backend' field")
backend_cls = self.context.registry.get(ctype, cfg.backend)
if backend_cls is None:
raise ValueError(f"Unregistered backend '{cfg.backend}' for {label}")
backend_cls = self._backend_class(ctype, cfg, label)
params = cfg.model_dump()
params["app_context"] = self.context
if name is not None:
params.setdefault("name", name)
if workspace_alias is not None:
params["workspace_alias"] = workspace_alias
instance = backend_cls(**params)
if not isinstance(instance, expected_type):
got, want = type(instance).__name__, expected_type.__name__
@ -137,16 +155,29 @@ class Application(BaseComponent):
# ----- Dependency ordering ------------------------------------------
def _component_nodes(
self,
replacement: tuple[_NodeKey, BaseComponent] | None = None,
) -> dict[_NodeKey, BaseComponent]:
"""One node per component instance; scoped components contribute one node per alias."""
nodes: dict[_NodeKey, BaseComponent] = {}
for alias, workspace in self.context.workspaces.items():
for ctype, group in workspace.components.items():
for name, instance in group.items():
nodes[(ctype, name, alias)] = instance
for ctype, group in self.context.shared_components.items():
for name, comp in group.items():
nodes[(ctype, name, "")] = comp
if replacement is not None:
nodes[replacement[0]] = replacement[1]
return nodes
def _topological_order(
self,
replacement: tuple[_NodeKey, BaseComponent] | None = None,
) -> list[BaseComponent]:
"""Return components in dependency order via Kahn's algorithm; raise on missing dep or cycle."""
nodes: dict[_NodeKey, BaseComponent] = {
(ctype, name): comp for ctype, group in self.context.components.items() for name, comp in group.items()
}
if replacement is not None:
nodes[replacement[0]] = replacement[1]
nodes = self._component_nodes(replacement)
in_degree, dependents = self._build_dependency_graph(nodes)
ready = [k for k, d in in_degree.items() if d == 0]
@ -169,13 +200,30 @@ class Application(BaseComponent):
def _build_dependency_graph(
nodes: dict[_NodeKey, BaseComponent],
) -> tuple[dict[_NodeKey, int], dict[_NodeKey, list[_NodeKey]]]:
"""Compute in-degree and adjacency lists; raise if a required dep is missing."""
"""Compute in-degree and adjacency lists; raise if a required dep is missing.
Scoped instances depend on the same-alias instance of a scoped
dependency, falling back to shared ("" alias) instances, then to the
dependency's default-alias instance for shared dependents.
"""
in_degree: dict[_NodeKey, int] = dict.fromkeys(nodes, 0)
dependents: dict[_NodeKey, list[_NodeKey]] = {k: [] for k in nodes}
for key, comp in nodes.items():
for dep in comp.dependencies:
dep_key = (dep.ctype, dep.name)
if dep_key in nodes:
dep_key = next(
(
candidate
for candidate in (
(dep.ctype, dep.name, key[2]),
(dep.ctype, dep.name, ""),
)
if candidate in nodes
),
None,
)
if dep_key is None:
dep_key = next((k for k in nodes if k[:2] == (dep.ctype, dep.name)), None)
if dep_key is not None:
dependents[dep_key].append(key)
in_degree[key] += 1
elif not dep.optional:
@ -241,11 +289,10 @@ class Application(BaseComponent):
"""Update an existing component by type/name; never creates missing components."""
async with self._component_mutation_lock:
component_type = component_type_name(component_enum)
group = self.context.components.get(component_type)
if not group or name not in group:
component = self.context.get_component(component_type, name)
if component is None:
raise KeyError(f"Component '{name}' not found in {component_type}")
component = group[name]
# Validate fields without evaluating lazy properties before injection.
for key in kwargs:
if getattr_static(component, key, _UNSET) is _UNSET and not hasattr(component, key):
@ -254,6 +301,18 @@ class Application(BaseComponent):
setattr(component, key, value)
return component
def _all_component_instances(self) -> list[BaseComponent]:
"""Every component instance: plain entries plus all alias instances of scoped components."""
instances: dict[int, BaseComponent] = {}
for workspace in self.context.workspaces.values():
for group in workspace.components.values():
for instance in group.values():
instances[id(instance)] = instance
for group in self.context.shared_components.values():
for component in group.values():
instances[id(component)] = component
return list(instances.values())
def _replacement_shutdown_order(
self,
old_component: BaseComponent,
@ -264,9 +323,7 @@ class Application(BaseComponent):
started_ids = {id(component) for component in self._started_components}
started_ids.discard(id(old_component))
started_ids.add(id(replacement))
component_ids = {
id(component) for components in self.context.components.values() for component in components.values()
}
component_ids = {id(component) for component in self._all_component_instances()}
non_components = [
component
for component in self._started_components
@ -299,8 +356,23 @@ class Application(BaseComponent):
"""
async with self._component_mutation_lock:
component_type = component_type_name(component_enum)
group = self.context.components.get(component_type)
config_group = self.config.components.get(component_type)
scoped_instances = {
alias: workspace.components[component_type][name]
for alias, workspace in self.context.workspaces.items()
if name in workspace.components.get(component_type, {})
}
if len(scoped_instances) > 1:
raise NotImplementedError(
f"replace_component does not support workspace-scoped component "
f"'{component_type}:{name}' in multi-workspace configurations",
)
alias = next(iter(scoped_instances), None)
group = (
self.context.workspaces[alias].components.get(component_type)
if alias is not None
else self.context.shared_components.get(component_type)
)
if not group or name not in group or config_group is None:
raise KeyError(f"Component '{name}' not found in {component_type}")
@ -316,6 +388,7 @@ class Application(BaseComponent):
label=f"Component '{name}'",
expected_type=BaseComponent,
name=name,
workspace_alias=alias,
)
# Validate fields without evaluating lazy properties before injection.
for key, value in (runtime_updates or {}).items():
@ -325,14 +398,12 @@ class Application(BaseComponent):
)
setattr(replacement, key, value)
node_key = (component_type, name)
node_key = (component_type, name, alias or "")
replacement_order = Application._topological_order(self, replacement=(node_key, replacement))
was_started = old_component.is_started
consumers: list[tuple[BaseComponent, str]] = []
candidates: list[BaseComponent] = [
component for components in self.context.components.values() for component in components.values()
]
candidates: list[BaseComponent] = self._all_component_instances()
candidates.extend(self.context.jobs.values())
if self.context.service is not None:
candidates.append(self.context.service)
@ -357,6 +428,8 @@ class Application(BaseComponent):
# Commit the new generation synchronously so observers cannot see
# a context with only some dependency references updated.
group[name] = replacement
if alias is not None:
self.context.workspaces[alias].components[component_type][name] = replacement
config_group[name] = replacement_config
for consumer, attr in consumers:
consumer.__dict__[attr] = replacement

View file

@ -157,7 +157,7 @@ class AsAgentWrapper(BaseAgentWrapper):
super().__init__(**kwargs)
self.as_llm = self.bind(as_llm, BaseAsLLM, optional=False)
self.session_retention_days = int(session_retention_days)
self._session_cleanup_done = False
self._session_cleanup_done: set[str] = set()
@classmethod
def _make_tool(
@ -165,9 +165,14 @@ class AsAgentWrapper(BaseAgentWrapper):
job: "BaseJob",
tool_context_id: str | None = None,
injected_job_kwargs: dict[str, Any] | None = None,
workspace: str | None = None,
) -> FunctionTool:
injected = cls._resolve_injected_job_kwargs(
{"tool_context_id": tool_context_id, "injected_job_kwargs": injected_job_kwargs},
{
"tool_context_id": tool_context_id,
"injected_job_kwargs": injected_job_kwargs,
"workspace": workspace,
},
)
async def run_job(**kwargs) -> ToolChunk:
@ -185,9 +190,10 @@ class AsAgentWrapper(BaseAgentWrapper):
names: list[str] | str | bool | None = "all",
*,
sequential_tool_calls: bool = False,
call_kwargs: dict[str, Any] | None = None,
) -> list[ToolBase]:
"""Return selected AgentScope built-in tools rooted at ``self.cwd``."""
cwd = str(self.cwd)
"""Return selected AgentScope built-in tools rooted at this call's workspace."""
cwd = str(self._call_cwd(call_kwargs or {}))
backend = WorkspaceBackend(cwd, self.bash_environment)
factories = {
"bash": lambda: BypassAnalysisBash(cwd=cwd, backend=backend),
@ -223,10 +229,15 @@ class AsAgentWrapper(BaseAgentWrapper):
@property
def session_path(self) -> Path:
"""Directory used for persisted AgentScope sessions."""
"""Directory used for persisted AgentScope sessions in the default workspace."""
return self._session_path_for({})
def _session_path_for(self, kwargs: dict[str, Any]) -> Path:
"""Session directory under the workspace this call is bound to."""
root = self._call_workspace_path(kwargs)
if self.app_context is None:
return self.workspace_path / "mem_session" / "agentscope"
return self.workspace_path / self.app_context.app_config.mem_session_dir / "agentscope"
return root / "mem_session" / "agentscope"
return root / self.app_context.app_config.mem_session_dir / "agentscope"
@staticmethod
def _validate_session_id(session_id: str, field: str = "session_id") -> str:
@ -234,15 +245,15 @@ class AsAgentWrapper(BaseAgentWrapper):
raise ValueError(f"{field} must be a valid UUID: {session_id!r}")
return session_id.lower()
def _cleanup_expired_sessions(self) -> None:
def _cleanup_expired_sessions(self, session_path: Path) -> None:
"""Delete persisted session files older than ``session_retention_days``."""
if self._session_cleanup_done or self.session_retention_days <= 0:
self._session_cleanup_done = True
key = str(session_path)
if key in self._session_cleanup_done or self.session_retention_days <= 0:
self._session_cleanup_done.add(key)
return
session_path = self.session_path
if not session_path.is_dir():
self._session_cleanup_done = True
self._session_cleanup_done.add(key)
return
cutoff = time.time() - self.session_retention_days * 24 * 60 * 60
@ -259,7 +270,7 @@ class AsAgentWrapper(BaseAgentWrapper):
self.logger.info(
f"Cleaned {removed} AgentScope session(s) older than {self.session_retention_days} day(s)",
)
self._session_cleanup_done = True
self._session_cleanup_done.add(key)
async def _load_state(self, kwargs: dict[str, Any], perm_mode: PermissionMode) -> AgentState:
resume = kwargs.get("resume") or ""
@ -274,7 +285,7 @@ class AsAgentWrapper(BaseAgentWrapper):
raise ValueError("session_id cannot be used with resume unless fork_session=True")
if resume:
handler = AsStateHandler.for_session(self.session_path, resume)
handler = AsStateHandler.for_session(self._session_path_for(kwargs), resume)
state = await handler.load_or_none()
if state is None:
raise FileNotFoundError(f"AgentScope session not found: {resume}")
@ -292,8 +303,8 @@ class AsAgentWrapper(BaseAgentWrapper):
return AgentState(session_id=session_id or str(uuid4()), permission_context=PermissionContext(mode=perm_mode))
async def _dump_state(self, state: AgentState) -> None:
await AsStateHandler.for_session(self.session_path, state.session_id).dump(state)
async def _dump_state(self, state: AgentState, kwargs: dict[str, Any]) -> None:
await AsStateHandler.for_session(self._session_path_for(kwargs), state.session_id).dump(state)
def _resolve_skills(self, skills: list[str] | str | None) -> list[str]:
"""Resolve configured skill names to AgentScope local skill directories."""
@ -305,7 +316,7 @@ class AsAgentWrapper(BaseAgentWrapper):
if model is None:
raise ValueError("AsAgentWrapper requires a bound as_llm component with a valid model.")
self._cleanup_expired_sessions()
self._cleanup_expired_sessions(self._session_path_for(kwargs))
system_prompt = kwargs.get("system_prompt", "You are a helpful assistant.")
job_tools: list[str] = kwargs.get("job_tools", [])
@ -317,8 +328,22 @@ class AsAgentWrapper(BaseAgentWrapper):
if "builtin_tools" not in kwargs and bool(kwargs.get("use_builtin_tools", False)):
builtin_tools = "all"
tools: list[ToolBase] = []
tools.extend(self._builtin_tools(builtin_tools, sequential_tool_calls=sequential_tool_calls))
tools.extend(self._make_tool(job, tool_context_id, kwargs.get("injected_job_kwargs")) for job in resolved_jobs)
tools.extend(
self._builtin_tools(
builtin_tools,
sequential_tool_calls=sequential_tool_calls,
call_kwargs=kwargs,
),
)
tools.extend(
self._make_tool(
job,
tool_context_id,
kwargs.get("injected_job_kwargs"),
workspace=kwargs.get("workspace"),
)
for job in resolved_jobs
)
toolkit = kwargs.get("toolkit") or Toolkit(
tools=tools,
skills_or_loaders=skills,
@ -350,7 +375,7 @@ class AsAgentWrapper(BaseAgentWrapper):
await agent.observe(inputs)
last_msg = await agent.reply()
usage = self._agentscope_usage(last_msg.usage) if last_msg.usage is not None else None
await self._dump_state(agent.state)
await self._dump_state(agent.state, kwargs)
result = {
"session_id": agent.state.session_id,
@ -391,7 +416,7 @@ class AsAgentWrapper(BaseAgentWrapper):
agent, _ = await self._build_agent(None, **kwargs)
config = {**(kwargs.get("context_config") or {}), "trigger_ratio": 1e-9}
await agent.compress_context(ContextConfig(**config))
await self._dump_state(agent.state)
await self._dump_state(agent.state, kwargs)
# ----- StreamChunk conversion -------------------------------------------
@ -496,4 +521,4 @@ class AsAgentWrapper(BaseAgentWrapper):
chunk.session_id = chunk.session_id or agent.state.session_id
yield chunk
await self._dump_state(agent.state)
await self._dump_state(agent.state, kwargs)

View file

@ -8,7 +8,7 @@ from typing import Any, ClassVar, TYPE_CHECKING
from pydantic import BaseModel
from ..base_component import BaseComponent
from ..base_component import BaseComponent, _context_workspaces
from ..outbound_proxy import BaseOutboundProxy
from ...enumeration import ChunkEnum, ComponentEnum
from ...schema import StreamChunk, TokenUsage
@ -58,6 +58,35 @@ class BaseAgentWrapper(BaseComponent):
cwd = Path(self._cwd)
return cwd if cwd.is_absolute() else (self.workspace_path / cwd)
def _call_workspace_path(self, kwargs: dict[str, Any]) -> Path:
"""Workspace root for this call: the forwarded binding, else the default workspace."""
alias = str(kwargs.get("workspace") or "")
if not alias:
return self.workspace_path
workspaces = _context_workspaces(self.app_context)
if workspaces is None or alias not in workspaces:
configured = sorted(workspaces) if workspaces else []
raise ValueError(f"Unknown workspace {alias!r}; configured: {configured}")
return workspaces[alias].workspace_path
def _call_cwd(self, kwargs: dict[str, Any]) -> Path:
"""Working directory for this call.
An explicit cwd wins. A relative one resolves against the bound
workspace; an absolute path keeps its explicit meaning. Otherwise an
explicitly selected workspace selects that workspace root. The default
alias injected for job tools is not an explicit selection, so a
single-workspace call keeps the project root.
"""
if self._cwd:
cwd = Path(self._cwd)
if cwd.is_absolute():
return cwd
return self._call_workspace_path(kwargs) / cwd
if kwargs.get("workspace") and kwargs.get("_workspace_explicit", True):
return self._call_workspace_path(kwargs)
return self.cwd
def set_system_prompt(self, prompt: str) -> "BaseAgentWrapper":
"""Set the agent's system prompt. Returns self for chaining."""
self.kwargs["system_prompt"] = prompt
@ -163,12 +192,15 @@ class BaseAgentWrapper(BaseComponent):
``injected_job_kwargs`` values are enforced constraints added by the
wrapper after receiving the model's tool arguments; the model can
neither see nor override them. ``tool_context_id`` keeps its dedicated
option but is carried through the same mechanism.
neither see nor override them. ``tool_context_id`` and the workspace
binding keep their dedicated options but are carried through the same
mechanism, so the model can never switch workspaces.
"""
injected = dict(kwargs.get("injected_job_kwargs") or {})
if tool_context_id := kwargs.get("tool_context_id"):
injected["tool_context_id"] = tool_context_id
if workspace := kwargs.get("workspace"):
injected["workspace"] = workspace
return injected
@staticmethod
@ -211,8 +243,18 @@ class BaseAgentWrapper(BaseComponent):
return resolved
def _merged_kwargs(self, kwargs: dict[str, Any]) -> dict[str, Any]:
"""Merge component defaults with call-time kwargs; call-time values win."""
"""Merge component defaults with call-time kwargs; call-time values win.
An omitted workspace becomes the default alias so job tools keep an
explicit binding. ``_workspace_explicit`` records whether the caller or
the component config selected that alias; cwd resolution uses it to
keep the historical project root when the alias was only filled in.
"""
merged = {**self.kwargs, **kwargs}
explicit_workspace = bool(merged.get("workspace"))
if (workspaces := _context_workspaces(self.app_context)) and not explicit_workspace:
merged["workspace"] = next(iter(workspaces))
merged["_workspace_explicit"] = explicit_workspace
if "output_schema" in merged:
merged["output_schema"] = self._normalize_output_schema(merged["output_schema"])
return merged

View file

@ -43,10 +43,15 @@ class CcAgentWrapper(BaseAgentWrapper):
@property
def session_path(self) -> Path:
"""Directory used for persisted Claude Code sessions."""
"""Directory used for persisted Claude Code sessions in the default workspace."""
return self._session_path_for({})
def _session_path_for(self, kwargs) -> Path:
"""Session directory under the workspace this call is bound to."""
root = self._call_workspace_path(kwargs)
if self.app_context is None:
return self.workspace_path / "mem_session"
return self.workspace_path / self.app_context.app_config.mem_session_dir
return root / "mem_session"
return root / self.app_context.app_config.mem_session_dir
def _ensure_claude_skill_dir(self, config_dir: Path, skills: list[str] | str) -> None:
"""Add selected project skills to Claude Code discovery locations."""
@ -83,11 +88,16 @@ class CcAgentWrapper(BaseAgentWrapper):
job: "BaseJob",
tool_context_id: str | None = None,
injected_job_kwargs: dict[str, Any] | None = None,
workspace: str | None = None,
):
from claude_agent_sdk import SdkMcpTool
injected = cls._resolve_injected_job_kwargs(
{"tool_context_id": tool_context_id, "injected_job_kwargs": injected_job_kwargs},
{
"tool_context_id": tool_context_id,
"injected_job_kwargs": injected_job_kwargs,
"workspace": workspace,
},
)
async def run_job(args):
@ -187,9 +197,10 @@ class CcAgentWrapper(BaseAgentWrapper):
)
opts.env.update(extra_env_dict)
self._add_bash_proxy_hook(opts)
self.session_path.mkdir(parents=True, exist_ok=True)
opts.cwd = opts.cwd or self.cwd
claude_config_dir = self.session_path / "claude_config"
session_path = self._session_path_for(kwargs)
session_path.mkdir(parents=True, exist_ok=True)
opts.cwd = opts.cwd or self._call_cwd(kwargs)
claude_config_dir = session_path / "claude_config"
opts.env.setdefault("CLAUDE_CONFIG_DIR", str(claude_config_dir))
if selected_skills is not None:
self._ensure_claude_skill_dir(claude_config_dir, selected_skills)
@ -203,7 +214,12 @@ class CcAgentWrapper(BaseAgentWrapper):
if self.MCP_SERVER_NAME in opts.mcp_servers:
raise ValueError(f"mcp_servers already contains reserved server name {self.MCP_SERVER_NAME!r}")
sdk_tools = [
self._make_tool(job, kwargs.get("tool_context_id"), kwargs.get("injected_job_kwargs"))
self._make_tool(
job,
kwargs.get("tool_context_id"),
kwargs.get("injected_job_kwargs"),
workspace=kwargs.get("workspace"),
)
for job in resolved_jobs
]
opts.mcp_servers[self.MCP_SERVER_NAME] = create_sdk_mcp_server(

View file

@ -16,9 +16,12 @@ import tempfile
from typing import Any, TYPE_CHECKING
from .base_agent_wrapper import BaseAgentWrapper
from ..base_component import _context_workspaces
from ..component_registry import R
from ...constants import normalize_connect_host
from ...enumeration import ChunkEnum
from ...schema import StreamChunk, TokenUsage
from ...schema.workspace import WorkspaceAgentState
if TYPE_CHECKING:
from openai_codex import AsyncCodex, AsyncThread, CodexConfig, RunInput
@ -115,8 +118,8 @@ class CodexAgentWrapper(BaseAgentWrapper):
self._codex: AsyncCodex | None = None
self._release_threads = release_threads
self._turn_lock = asyncio.Lock()
self._mcp_snapshot_path: Path | None = None
self._thread_tool_contexts: dict[str, str] = {}
self._mcp_snapshot_path: Path | None = None
@property
def session_path(self) -> Path:
@ -128,13 +131,6 @@ class CodexAgentWrapper(BaseAgentWrapper):
return self.workspace_path / "mem_session" / "codex"
return self.workspace_path / self.app_context.app_config.mem_session_dir / "codex"
@property
def wrapper_session_path(self) -> Path:
"""ReMe-owned session data, kept separate from a shared OAuth CODEX_HOME."""
if self.app_context is None:
return self.workspace_path / "mem_session" / "codex"
return self.workspace_path / self.app_context.app_config.mem_session_dir / "codex"
def _ensure_skills(self, skills: list[str] | str | None) -> None:
"""Expose selected project skills through Codex's repo-level directory."""
sources = self._resolve_project_skills(skills)
@ -154,6 +150,13 @@ class CodexAgentWrapper(BaseAgentWrapper):
relative_source = os.path.relpath(source, target.parent)
target.symlink_to(relative_source, target_is_directory=True)
@property
def wrapper_session_path(self) -> Path:
"""ReMe-owned session data, kept separate from a shared OAuth CODEX_HOME."""
if self.app_context is None:
return self.workspace_path / "mem_session" / "codex"
return self.workspace_path / self.app_context.app_config.mem_session_dir / "codex"
def _explicit_mcp_config(self, kwargs: dict[str, Any]) -> str | None:
value = kwargs.get("mcp_config") if "mcp_config" in kwargs else self.mcp_config
if value is None:
@ -240,6 +243,31 @@ class CodexAgentWrapper(BaseAgentWrapper):
names = ", ".join(invalid)
raise TypeError(f"Codex client options must be configured on the wrapper: {names}")
def _resident_mcp_endpoint(self) -> str:
from ..service import HttpService, MCPService
service = self.app_context.service if self.app_context is not None else None
if isinstance(service, HttpService) and service.mcp_enabled:
return f"http://{normalize_connect_host(service.host)}:{service.port}{service.mcp_path}"
if isinstance(service, MCPService) and service.transport == "streamable-http":
return f"http://{normalize_connect_host(service.host)}:{service.port}/mcp"
raise RuntimeError(
"Multi-workspace Codex job_tools require a resident HTTP MCP service; "
"one-shot CLI and other service backends are not supported for this combination",
)
@staticmethod
def _validate_resident_jobs(service, jobs) -> None:
if not service.tool_error_on_failure:
raise ValueError("Multi-workspace Codex job_tools require resident MCP tool_error_on_failure=true")
unavailable = [
job.name
for job in jobs
if not job.enabled or not job.enable_serve or (service.jobs is not None and job.name not in service.jobs)
]
if unavailable:
raise ValueError(f"Codex job_tools must be exposed by the resident MCP service: {', '.join(unavailable)}")
def _mcp_server_config(self, kwargs: dict[str, Any]) -> dict[str, Any] | None:
from ..job import BackgroundJob, StreamJob
@ -251,19 +279,33 @@ class CodexAgentWrapper(BaseAgentWrapper):
if unsupported:
raise TypeError(f"Codex job_tools must be non-stream request jobs: {', '.join(unsupported)}")
config_source = self._mcp_config_source(kwargs)
args = [
"-m",
"reme.components.agent_wrapper.codex_mcp_server",
"--config",
config_source,
"--workspace",
str(self.workspace_path),
]
workspaces = _context_workspaces(self.app_context)
multi_workspace = workspaces is not None and len(workspaces) > 1
if multi_workspace:
if self._explicit_mcp_config(kwargs) is not None:
raise ValueError("mcp_config is only supported for single-workspace Codex job_tools")
endpoint = self._resident_mcp_endpoint()
self._validate_resident_jobs(self.app_context.service, jobs)
args = ["-m", "reme.components.agent_wrapper.codex_mcp_server", "--endpoint", endpoint]
else:
args = [
"-m",
"reme.components.agent_wrapper.codex_mcp_server",
"--config",
self._mcp_config_source(kwargs),
"--workspace",
str(self.workspace_path),
]
if workspaces:
alias = str(kwargs.get("workspace") or next(iter(workspaces)))
args.extend(["--workspace-alias", alias])
for name in job_names:
args.extend(["--job", name])
args.extend(["--tool-context-id", str(kwargs.get("tool_context_id") or "")])
if injected_job_kwargs := dict(kwargs.get("injected_job_kwargs") or {}):
injected_job_kwargs = dict(kwargs.get("injected_job_kwargs") or {})
if workspaces and (workspace := kwargs.get("workspace")):
injected_job_kwargs["workspace"] = workspace
if injected_job_kwargs:
args.extend(["--injected-job-kwargs", json.dumps(injected_job_kwargs, sort_keys=True)])
return {
"command": sys.executable,
@ -300,6 +342,8 @@ class CodexAgentWrapper(BaseAgentWrapper):
from openai_codex import ApprovalMode, Sandbox
from openai_codex.types import Personality, ThreadSource, ThreadStartSource
alias = str(kwargs.get("workspace") or "")
tool_contexts = self._tool_contexts(alias)
resume = kwargs.get("resume") or ""
session_id = kwargs.get("session_id") or ""
if resume and session_id and resume != session_id:
@ -308,8 +352,10 @@ class CodexAgentWrapper(BaseAgentWrapper):
fork_session = bool(kwargs.get("fork_session", False))
if fork_session and not thread_id:
raise ValueError("fork_session=True requires resume or session_id")
if self._release_threads and self._thread_tool_contexts:
await self._prune_thread_tool_contexts(codex)
if thread_id:
await self._check_thread_ownership(thread_id, alias)
if self._release_threads and tool_contexts:
await self._prune_thread_tool_contexts(codex, alias)
requested_tool_context = json.dumps(
{
"tool_context_id": str(kwargs.get("tool_context_id") or ""),
@ -318,15 +364,15 @@ class CodexAgentWrapper(BaseAgentWrapper):
sort_keys=True,
default=str,
)
if not fork_session and thread_id in self._thread_tool_contexts:
if requested_tool_context != self._thread_tool_contexts[thread_id]:
if not fork_session and thread_id in tool_contexts:
if requested_tool_context != tool_contexts[thread_id]:
raise ValueError("tool_context_id and injected_job_kwargs cannot change when resuming a Codex thread")
common = {
"approval_mode": self._enum(ApprovalMode, kwargs.get("approval_mode"), ApprovalMode.auto_review),
"base_instructions": kwargs.get("base_instructions"),
"config": self._thread_config(kwargs),
"cwd": str(self.cwd),
"cwd": str(self._call_cwd(kwargs)),
"developer_instructions": kwargs.get("system_prompt"),
"model": kwargs.get("model"),
"model_provider": kwargs.get("model_provider"),
@ -353,16 +399,171 @@ class CodexAgentWrapper(BaseAgentWrapper):
thread_source=thread_source,
**common,
)
self._thread_tool_contexts[thread.id] = requested_tool_context
await self._record_thread_ownership(alias, thread.id)
tool_contexts[thread.id] = requested_tool_context
return thread
# ----- Thread ownership (per-workspace, persisted) ---------------------
def _agent_state(self, alias: str) -> WorkspaceAgentState | None:
workspaces = _context_workspaces(self.app_context)
if not workspaces:
return None
return workspaces[self._normalize_thread_alias(alias)].agent_state(self.name)
def _tool_contexts(self, alias: str) -> dict[str, str]:
state = self._agent_state(alias)
return state.thread_tool_contexts if state is not None else self._thread_tool_contexts
def _normalize_thread_alias(self, alias: str) -> str:
"""Map an omitted alias to the default while preserving explicit aliases."""
workspaces = _context_workspaces(self.app_context)
if workspaces and not alias:
return next(iter(workspaces))
return alias
def _thread_ownership_path(self, alias: str) -> Path | None:
workspaces = _context_workspaces(self.app_context)
if not workspaces:
return None
workspace = workspaces.get(alias)
if workspace is None or workspace.mem_session_path is None:
return None
return workspace.mem_session_path / "codex" / "owned_threads.json"
def _codex_thread_state(self, alias: str):
workspaces = _context_workspaces(self.app_context)
if not workspaces:
return None
return workspaces[self._normalize_thread_alias(alias)].codex_thread_state
async def _owned_thread_ids(self, alias: str) -> set[str]:
"""Load ownership through the single authoritative state of each workspace."""
state = self._codex_thread_state(alias)
if state is None:
return set()
async with state.lock:
if state.owned_thread_ids is not None:
return state.owned_thread_ids
ids: set[str] = set()
path = self._thread_ownership_path(alias)
if path is not None and path.is_file():
try:
raw = json.loads(path.read_text(encoding="utf-8"))
ids = {str(thread_id) for thread_id in raw.get("thread_ids", [])}
except (OSError, ValueError, AttributeError) as exc:
self.logger.warning(f"Ignoring unreadable Codex thread ownership file {path}: {exc}")
state.owned_thread_ids = ids
return ids
def _require_thread_alias(self, alias: str) -> str:
"""Resolve an omitted alias and reject names that are not configured."""
workspaces = _context_workspaces(self.app_context)
if not workspaces:
return alias
resolved = self._normalize_thread_alias(alias)
if resolved not in workspaces:
configured = sorted(workspaces)
raise ValueError(f"Unknown workspace {resolved!r}; configured: {configured}")
return resolved
@staticmethod
def _write_owned_thread_ids(path: Path, thread_ids: set[str]) -> None:
"""Atomically replace one workspace's ownership file."""
path.parent.mkdir(parents=True, exist_ok=True)
fd, raw_path = tempfile.mkstemp(prefix=".owned_threads-", suffix=".tmp", dir=path.parent)
try:
with os.fdopen(fd, "w", encoding="utf-8") as stream:
json.dump({"thread_ids": sorted(thread_ids)}, stream, indent=1)
os.replace(raw_path, path)
finally:
Path(raw_path).unlink(missing_ok=True)
async def _record_thread_ownership(self, alias: str, thread_id: str) -> bool:
"""Claim ``thread_id`` for ``alias``.
Returns True when this call newly records the thread. An unknown alias,
or a thread already owned by another workspace, raises ValueError before
the claim is stored.
"""
workspaces = _context_workspaces(self.app_context)
if not workspaces:
return False
alias = self._require_thread_alias(alias)
async with self.app_context.codex_thread_ownership_lock:
for owner_alias in workspaces:
if thread_id not in await self._owned_thread_ids(owner_alias):
continue
if owner_alias == alias:
return False
raise ValueError(
f"Codex thread {thread_id} belongs to workspace {owner_alias!r}; "
f"it cannot be used from workspace {alias!r}",
)
state = self._codex_thread_state(alias)
ids = await self._owned_thread_ids(alias)
path = self._thread_ownership_path(alias)
if path is None:
if thread_id in ids:
return False
ids.add(thread_id)
return True
async with state.lock:
if thread_id in ids:
return False
updated = {*ids, thread_id}
self._write_owned_thread_ids(path, updated)
ids.add(thread_id)
return True
async def _forget_thread_ownership(self, alias: str, thread_id: str) -> None:
"""Drop a claim reserved for an SDK operation that did not succeed."""
workspaces = _context_workspaces(self.app_context)
if not workspaces:
return
alias = self._normalize_thread_alias(alias)
async with self.app_context.codex_thread_ownership_lock:
ids = await self._owned_thread_ids(alias)
if thread_id not in ids:
return
path = self._thread_ownership_path(alias)
if path is None:
ids.discard(thread_id)
return
state = self._codex_thread_state(alias)
async with state.lock:
if thread_id not in ids:
return
updated = set(ids)
updated.discard(thread_id)
self._write_owned_thread_ids(path, updated)
ids.discard(thread_id)
async def _check_thread_ownership(self, thread_id: str, alias: str) -> None:
"""Reject resume/fork/compact of a thread owned by another workspace.
Threads recorded nowhere predate ownership tracking or came from other
tools; they pass through and let the SDK decide.
"""
workspaces = _context_workspaces(self.app_context)
if not workspaces:
return
alias = self._normalize_thread_alias(alias)
for owner_alias in workspaces:
if owner_alias != alias and thread_id in await self._owned_thread_ids(owner_alias):
raise ValueError(
f"Codex thread {thread_id} belongs to workspace {owner_alias!r}; "
f"it cannot be used from workspace {alias!r}",
)
def _turn_kwargs(self, kwargs: dict[str, Any]) -> dict[str, Any]:
from openai_codex import ApprovalMode, Sandbox
from openai_codex.types import Personality, ReasoningEffort, ReasoningSummary
return {
"approval_mode": self._enum(ApprovalMode, kwargs.get("approval_mode")),
"cwd": str(self.cwd),
"cwd": str(self._call_cwd(kwargs)),
"effort": self._enum(ReasoningEffort, kwargs.get("effort")),
"model": kwargs.get("model"),
"output_schema": kwargs.get("output_schema"),
@ -396,10 +597,13 @@ class CodexAgentWrapper(BaseAgentWrapper):
return codex
async def _close(self) -> None:
"""Close the persistent app-server and remove its private config snapshot."""
"""Close the persistent app-server and clear session execution guards."""
async with self._turn_lock:
codex, self._codex = self._codex, None
self._thread_tool_contexts.clear()
for workspace in (_context_workspaces(self.app_context) or {}).values():
if state := workspace.agent_states.get(self.name):
state.thread_tool_contexts.clear()
try:
if codex is not None:
await codex.close()
@ -408,8 +612,9 @@ class CodexAgentWrapper(BaseAgentWrapper):
self._mcp_snapshot_path.unlink(missing_ok=True)
self._mcp_snapshot_path = None
async def _prune_thread_tool_contexts(self, codex: AsyncCodex) -> None:
async def _prune_thread_tool_contexts(self, codex: AsyncCodex, alias: str = "") -> None:
"""Forget tool constraints only after the server confirms a thread is unloaded."""
tool_contexts = self._tool_contexts(alias)
try:
from openai_codex.generated.v2_all import ThreadLoadedListResponse
@ -428,9 +633,9 @@ class CodexAgentWrapper(BaseAgentWrapper):
except Exception as exc: # pylint: disable=broad-exception-caught
self.logger.warning(f"Failed to list loaded Codex threads: {exc}")
return
self._thread_tool_contexts = {
thread_id: context for thread_id, context in self._thread_tool_contexts.items() if thread_id in loaded_ids
}
retained = {thread_id: context for thread_id, context in tool_contexts.items() if thread_id in loaded_ids}
tool_contexts.clear()
tool_contexts.update(retained)
async def _release_thread(self, codex: AsyncCodex, thread_id: str) -> None:
"""Unsubscribe so the app-server can unload the thread once it is idle."""
@ -490,16 +695,30 @@ class CodexAgentWrapper(BaseAgentWrapper):
raise ValueError("Codex returned invalid JSON for the requested output_schema") from exc
return response
async def compact_session(self, session_id: str) -> None:
"""Start native compaction of a Codex thread."""
async def compact_session(self, session_id: str, workspace: str | None = None) -> None:
"""Compact a Codex thread after its workspace wins ownership.
The alias is validated and the cross-workspace claim is settled before
any SDK call, so a losing request cannot compact. A claim made for this
call is removed when the SDK rejects the operation; an existing claim
is left in place. Thread release still runs once a thread was resumed.
"""
await self.start()
alias = str(workspace or "")
async with self._turn_lock:
codex = await self._get_codex()
thread = await codex.thread_resume(session_id)
newly_claimed = await self._record_thread_ownership(alias, session_id)
codex = None
thread = None
try:
codex = await self._get_codex()
thread = await codex.thread_resume(session_id)
await thread.compact()
except BaseException:
if newly_claimed:
await self._forget_thread_ownership(alias, session_id)
raise
finally:
if self._release_threads:
if thread is not None and codex is not None and self._release_threads:
await self._release_thread(codex, thread.id)
@classmethod

View file

@ -1,10 +1,17 @@
"""FastMCP STDIO bridge that exposes selected ReMe jobs to Codex."""
"""Expose ReMe jobs to Codex over FastMCP STDIO.
Multi-workspace callers forward to the resident MCP service without creating
an Application. Single-workspace callers retain the legacy child-service path.
"""
import argparse
import asyncio
import json
from pathlib import Path
from typing import Any
from .base_agent_wrapper import BaseAgentWrapper
from ..client import MCPClient
from ...config import resolve_app_config
from ...reme import ReMe
@ -12,7 +19,9 @@ from ...reme import ReMe
def _parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Expose selected ReMe jobs over FastMCP STDIO")
parser.add_argument("--config", default="default", help="ReMe config name or file path")
parser.add_argument("--workspace", required=True, help="ReMe workspace directory")
parser.add_argument("--workspace", help="Single-workspace directory for the legacy child service")
parser.add_argument("--workspace-alias", help="Configured alias retained by the legacy child service")
parser.add_argument("--endpoint", help="Resident streamable-HTTP MCP endpoint for forwarding")
parser.add_argument("--job", dest="jobs", action="append", required=True, help="ReMe job name; repeat as needed")
parser.add_argument("--tool-context-id", default="", help="Context id injected into every job call")
parser.add_argument(
@ -20,6 +29,7 @@ def _parse_args() -> argparse.Namespace:
default="",
help="JSON object of server-owned kwargs injected into every job call",
)
parser.add_argument("--tool-timeout", type=float, default=300, help="Per-call timeout in seconds")
return parser.parse_args()
@ -29,7 +39,7 @@ def _prepare_config(
tool_context_id: str = "",
injected_job_kwargs: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Configure the dedicated child Application to serve selected jobs over MCP STDIO."""
"""Configure the legacy single-workspace child Application for MCP STDIO."""
selected = set(job_names)
jobs: dict[str, dict[str, Any]] = {}
for name, raw_job_config in (config.get("jobs") or {}).items():
@ -37,7 +47,6 @@ def _prepare_config(
if job_config.get("backend") in {"background", "cron"}:
continue
if name in selected:
# An explicit Codex job_tools selection has always overridden enable_serve.
job_config["enable_serve"] = True
jobs[name] = job_config
@ -63,21 +72,107 @@ def _prepare_config(
return prepared
class _ResidentMCPClient:
"""Reuse the standard ReMe MCP client for resident tool discovery and calls."""
def __init__(self, endpoint: str, timeout: float) -> None:
from fastmcp.client import StreamableHttpTransport
self._client = MCPClient(transport=StreamableHttpTransport(url=endpoint), timeout=timeout)
async def start(self) -> None:
"""Connect to the resident service."""
await self._client.start()
async def list_jobs(self) -> list[dict[str, Any]]:
"""Return the resident service's tool descriptors."""
return await self._client.list_actions()
async def call(self, job: str, kwargs: dict[str, Any], injected: dict[str, Any]) -> str:
"""Call one resident tool with server-owned arguments enforced."""
# pylint: disable=protected-access
payload = BaseAgentWrapper._merge_injected_job_kwargs(kwargs, injected)
async for text in self._client(action=job, **payload):
return text
return ""
async def close(self) -> None:
"""Close the resident connection."""
await self._client.close()
def _injected_kwargs(args: argparse.Namespace) -> dict[str, Any]:
injected = json.loads(args.injected_job_kwargs) if args.injected_job_kwargs else {}
if not isinstance(injected, dict):
raise TypeError("--injected-job-kwargs must be a JSON object")
injected = dict(injected)
if args.tool_context_id:
injected["tool_context_id"] = args.tool_context_id
return injected
async def _run_forwarder(args: argparse.Namespace, injected: dict[str, Any]) -> None:
from fastmcp import FastMCP
from fastmcp.exceptions import ToolError
from fastmcp.tools import FunctionTool
client = _ResidentMCPClient(args.endpoint, args.tool_timeout)
await client.start()
try:
metadata = {entry["name"]: entry for entry in await client.list_jobs()}
missing = sorted(set(args.jobs) - set(metadata))
if missing:
raise KeyError(f"Codex job tools not found on the resident MCP service: {', '.join(missing)}")
server = FastMCP("reme-codex-bridge")
for name in args.jobs:
entry = metadata[name]
def make_forwarder(job_name: str):
async def forward(**kwargs):
try:
return await client.call(job_name, kwargs, injected)
except Exception as exc: # noqa: BLE001 - surface every failure as a tool error
raise ToolError(str(exc)) from exc
return forward
parameters = entry.get("inputSchema") or entry.get("parameters")
# pylint: disable=protected-access
parameters = BaseAgentWrapper._strip_injected_parameters(parameters, injected)
server.add_tool(
FunctionTool(
name=name,
description=entry.get("description") or "",
parameters=parameters or {"type": "object", "properties": {}},
fn=make_forwarder(name),
),
)
await server.run_async(transport="stdio")
finally:
await client.close()
def main() -> None:
"""Load ReMe and serve the requested jobs over STDIO."""
"""Forward to a resident service or start the legacy single-workspace child service."""
args = _parse_args()
injected = _injected_kwargs(args)
if args.endpoint:
asyncio.run(_run_forwarder(args, injected))
return
if not args.workspace:
raise ValueError("--workspace is required when --endpoint is not supplied")
workspace = str(Path(args.workspace).absolute())
workspace_dir = {args.workspace_alias: workspace} if args.workspace_alias else workspace
config = resolve_app_config(
config=args.config,
workspace_dir=str(Path(args.workspace).absolute()),
workspace_dir=workspace_dir,
enable_logo=False,
log_to_console=False,
log_to_file=False,
log_config=False,
)
injected_job_kwargs = json.loads(args.injected_job_kwargs) if args.injected_job_kwargs else None
if injected_job_kwargs is not None and not isinstance(injected_job_kwargs, dict):
raise TypeError("--injected-job-kwargs must be a JSON object")
config = _prepare_config(config, args.jobs, args.tool_context_id, injected_job_kwargs)
config = _prepare_config(config, args.jobs, args.tool_context_id, injected)
ReMe(**config).run_app()

View file

@ -1,9 +1,11 @@
"""Application context: shared state container for components, jobs, and service."""
import asyncio
from collections.abc import Iterator, MutableMapping
from concurrent.futures import ThreadPoolExecutor
from typing import TYPE_CHECKING, Any
from ..schema import ApplicationConfig
from ..schema import ApplicationConfig, WorkspaceContext
from .component_registry import ComponentRegistry, create_application_registry
if TYPE_CHECKING:
@ -12,27 +14,81 @@ if TYPE_CHECKING:
from .service import BaseService
class ApplicationContext:
"""Passive state container holding parsed config and wired components.
class _LegacyComponentView(MutableMapping):
"""Compatibility mapping combining shared components with default-workspace instances."""
The Application class performs the actual wiring (registry lookups and
component instantiation); this class only stores the results so that
components, jobs, and the service can find each other at runtime.
"""
def __init__(self, context: "ApplicationContext") -> None:
self._context = context
def _default_components(self) -> dict:
return next(iter(self._context.workspaces.values())).components
def __getitem__(self, key: str) -> dict:
if key in self._context.shared_components:
return self._context.shared_components[key]
return self._default_components()[key]
def __setitem__(self, key: str, value: dict) -> None:
if value and all(getattr(component, "workspace_scoped", False) is True for component in value.values()):
self._default_components()[key] = value
self._context.shared_components.pop(key, None)
else:
self._context.shared_components[key] = value
self._default_components().pop(key, None)
def __delitem__(self, key: str) -> None:
if key in self._context.shared_components:
del self._context.shared_components[key]
else:
del self._default_components()[key]
def __iter__(self) -> Iterator[str]:
yield from self._context.shared_components
yield from (key for key in self._default_components() if key not in self._context.shared_components)
def __len__(self) -> int:
return len(set(self._context.shared_components) | set(self._default_components()))
class ApplicationContext:
"""Application-wide configuration, shared resources, and workspace registry."""
def __init__(self, *, registry: ComponentRegistry | None = None, **kwargs):
# Parse raw kwargs into a typed, validated config object.
self.app_config: ApplicationConfig = ApplicationConfig(**kwargs)
self.registry = registry or create_application_registry()
self.workspaces: dict[str, WorkspaceContext] = {
alias: WorkspaceContext.from_config(alias, path, self.app_config)
for alias, path in self.app_config.workspaces.items()
}
# Populated by Application during initialization.
self.workspace: WorkspaceContext = next(iter(self.workspaces.values()))
self.service: "BaseService | None" = None
self.components: dict[str, dict[str, "BaseComponent"]] = {}
self.shared_components: dict[str, dict[str, "BaseComponent"]] = {}
self._components = _LegacyComponentView(self)
self.jobs: dict[str, "BaseJob"] = {}
self.thread_pool: ThreadPoolExecutor | None = None
# Application-lifetime shared state. Values remain available across Job and Step
# invocations while this Application is running, so Jobs may keep cross-call state here.
# This is in-memory state, not durable storage; use workspace files or a store when state
# must survive an Application restart.
self.codex_thread_ownership_lock = asyncio.Lock()
self.metadata: dict[str, Any] = {}
@property
def components(self) -> MutableMapping:
"""Legacy component lookup; internal execution uses explicit shared or workspace ownership."""
return self._components
@components.setter
def components(self, value: dict) -> None:
self.shared_components = {}
for ctype, group in value.items():
self._components[ctype] = group
def get_component(self, ctype: str, name: str, alias: str | None = None) -> "BaseComponent | None":
"""Compatibility resolver for external callers that select a workspace by alias."""
alias = alias or next(iter(self.workspaces))
workspace = self.workspaces.get(alias)
if workspace is None:
raise KeyError(f"Unknown workspace alias {alias!r}")
if name in workspace.components.get(ctype, {}):
return workspace.get_component(ctype, name)
if any(name in ws.components.get(ctype, {}) for ws in self.workspaces.values()):
raise KeyError(f"Component '{name}' in {ctype} has no instance for workspace alias {alias!r}")
return self.shared_components.get(ctype, {}).get(name)

View file

@ -14,21 +14,44 @@ if TYPE_CHECKING:
T = TypeVar("T", bound="BaseComponent")
def _context_workspaces(app_context) -> dict | None:
"""Return the workspace mapping of a real ApplicationContext, else None.
Duck-typing is not enough here: MagicMock-style test doubles answer every
attribute access, so the workspace layer engages only for the real type.
"""
from .application_context import ApplicationContext # lazy: registry imports this module
if isinstance(app_context, ApplicationContext):
return app_context.workspaces
return None
class ComponentMixin:
"""Shared state for components and steps: identity, config, workspace paths."""
component_type = ComponentEnum.BASE
# Workspace-scoped components keep per-workspace state (indexes, graphs,
# catalogs, chunk stores) and get one instance per configured alias.
workspace_scoped: bool = False
# Stateful external backends that cannot isolate workspaces must opt out
# explicitly; Application rejects them in multi-workspace configurations.
supports_multi_workspace: bool = True
def __init__(
self,
name: str | None = None,
backend: str = "",
app_context: "ApplicationContext | None" = None,
workspace_alias: str | None = None,
**kwargs,
) -> None:
self.name: str = name or self.__class__.__name__
self.backend: str = backend
self.app_context: "ApplicationContext | None" = app_context
# Bound alias for workspace-scoped component instances; None for steps,
# jobs, and shared components.
self.workspace_alias: str | None = workspace_alias
self.kwargs: dict = dict(kwargs)
logger = get_logger()
@ -36,10 +59,25 @@ class ComponentMixin:
@property
def workspace_path(self) -> Path:
"""Absolute workspace root directory (cwd when no app_context is attached)."""
"""Workspace root: the bound alias for scoped components, the current
call's workspace binding for steps, otherwise the default workspace."""
if self.app_context is None:
return Path.cwd()
return Path(self.app_context.app_config.workspace_dir).absolute()
workspaces = _context_workspaces(self.app_context)
if workspaces is None:
# Test doubles and standalone-style contexts without the workspace layer.
return Path(self.app_context.app_config.workspace_dir).absolute()
alias = self.workspace_alias
if alias is None:
context = getattr(self, "context", None)
bound = getattr(context, "workspace", None) if context is not None else None
alias = bound.alias if bound is not None else None
if alias is None:
return next(iter(workspaces.values())).workspace_path
workspace = workspaces.get(alias)
if workspace is None:
raise KeyError(f"Unknown workspace alias {alias!r}")
return workspace.workspace_path
def to_workspace_relative(self, path: str | Path) -> str:
"""Convert ``path`` to a POSIX workspace path; keep outside paths absolute."""
@ -183,8 +221,24 @@ class BaseComponent(ComponentMixin, ABC):
setattr(self, attr, None)
def _resolve_from_context(self, attr: str, dep: Dependency) -> None:
"""Context-bound mode: look up the component from ``app_context.components``."""
target = self.app_context.components.get(dep.ctype, {}).get(dep.name)
"""Resolve scoped dependencies from their workspace and shared ones from the application."""
workspaces = getattr(self.app_context, "workspaces", None)
workspace = workspaces.get(self.workspace_alias) if workspaces and self.workspace_alias is not None else None
if workspace is not None and dep.name in workspace.components.get(dep.ctype, {}):
target = workspace.get_component(dep.ctype, dep.name)
elif workspace is not None and any(
dep.name in candidate.components.get(dep.ctype, {}) for candidate in workspaces.values()
):
raise ValueError(f"{dep.ctype} '{dep.name}' has no instance for workspace '{self.workspace_alias}'.")
else:
shared = getattr(self.app_context, "shared_components", None)
if shared is None:
shared = self.app_context.components
target = shared.get(dep.ctype, {}).get(dep.name)
if target is None and workspaces:
default_workspace = next(iter(workspaces.values()))
if dep.name in default_workspace.components.get(dep.ctype, {}):
target = default_workspace.get_component(dep.ctype, dep.name)
if target is not None:
setattr(self, attr, target)
elif dep.optional:

View file

@ -47,6 +47,8 @@ class LocalEmbeddingStore(BaseEmbeddingStore):
Delegates actual embedding computation to a bound ``as_embedding`` component.
"""
workspace_scoped = True
def __init__(
self,
as_embedding: str = "default",

View file

@ -12,6 +12,8 @@ from ...utils.jsonl_zst import read_jsonl_zst, write_jsonl_zst
class LocalFileCatalog(BaseFileCatalog):
"""Dict-backed catalog persisted as JSONL."""
workspace_scoped = True
def __init__(self, encoding: str = "utf-8", **kwargs):
super().__init__(**kwargs)
self.encoding = encoding

View file

@ -17,9 +17,14 @@ from ...utils.file_utils import check_file_version
class BaseFileChunker(BaseComponent):
"""Abstract base for file chunkers. Subclasses implement `chunk`."""
"""Abstract base for file chunkers. Subclasses implement `chunk`.
Chunkers are workspace-scoped: node paths are workspace-relative, so each
alias gets its own instance rooted at that workspace.
"""
component_type = ComponentEnum.FILE_CHUNKER
workspace_scoped = True
def __init__(self, supported_extensions: list[str] | None = None, **kwargs):
super().__init__(**kwargs)

View file

@ -13,6 +13,8 @@ from ...utils.jsonl_zst import read_jsonl_zst, write_jsonl_zst
class LocalFileGraph(BaseFileGraph):
"""Dict-backed file graph; uses FileLink.target_path for adjacency."""
workspace_scoped = True
def __init__(self, **kwargs):
super().__init__(**kwargs)
self._nodes: dict[str, FileNode] = {}

View file

@ -68,6 +68,8 @@ class Neo4jFileGraph(BaseFileGraph):
database: target db name (default ``neo4j``)
"""
supports_multi_workspace = False
def __init__(
self,
uri: str = "bolt://localhost:7687",

View file

@ -16,6 +16,8 @@ class NxFileGraph(BaseFileGraph):
Real node carries ``node`` attr; virtual (dangling target) does not.
"""
workspace_scoped = True
def __init__(self, **kwargs):
super().__init__(**kwargs)
try:

View file

@ -86,6 +86,8 @@ class LocalFileStore(BaseFileStore):
at least one of embedding / keyword must be present.
"""
workspace_scoped = True
def __init__(
self,
embedding_store: str = "default",

View file

@ -126,7 +126,8 @@ class BackgroundJob(BaseJob):
self.check_enabled()
self._record_call()
merged = {**self.kwargs, **kwargs}
context = RuntimeContext(stop_event=self._stop_event, **merged)
workspace = self._bind_workspace(merged)
context = RuntimeContext(stop_event=self._stop_event, workspace=workspace, **merged)
for step in self._build_steps():
await step(context)
return context.response

View file

@ -2,7 +2,7 @@
from typing import TYPE_CHECKING
from ..base_component import BaseComponent
from ..base_component import BaseComponent, _context_workspaces
from ..component_registry import R
from ..runtime_context import RuntimeContext
from ...enumeration import ComponentEnum
@ -90,12 +90,32 @@ class BaseJob(BaseComponent):
if not self.enabled:
raise ValueError(f"Job '{self.name}' is disabled")
def _bind_workspace(self, merged: dict):
"""Consume the optional ``workspace`` alias and resolve the bound workspace.
The alias never reaches step payloads. An unknown alias fails fast,
matching disabled-job and missing-job behavior at this boundary.
"""
alias = merged.pop("workspace", None)
workspaces = _context_workspaces(self.app_context)
if workspaces is None:
if alias is not None:
raise ValueError(f"workspace={alias!r} requires an application with configured workspaces")
return None
if alias is None:
return next(iter(workspaces.values()))
workspace = workspaces.get(alias)
if workspace is None:
raise ValueError(f"Unknown workspace {alias!r}; configured: {sorted(workspaces)}")
return workspace
async def __call__(self, **kwargs) -> Response:
"""Run all steps in order, capturing any failure into the response."""
self.check_enabled()
self._record_call()
merged = {**self.kwargs, **kwargs}
context = RuntimeContext(**merged)
workspace = self._bind_workspace(merged)
context = RuntimeContext(workspace=workspace, **merged)
try:
for step in self._build_steps():
await step(context)

View file

@ -35,7 +35,9 @@ class CronJob(BackgroundJob):
return max(0.0, (nxt - now).total_seconds())
async def _execute_steps(self) -> Response:
context = RuntimeContext(**self.kwargs)
merged = dict(self.kwargs)
workspace = self._bind_workspace(merged)
context = RuntimeContext(workspace=workspace, **merged)
for step in self._build_steps():
await step(context)
return context.response

View file

@ -15,7 +15,8 @@ class StreamJob(BaseJob):
self.check_enabled()
self._record_call()
merged = {**self.kwargs, **kwargs}
context = RuntimeContext(**merged)
workspace = self._bind_workspace(merged)
context = RuntimeContext(workspace=workspace, **merged)
try:
for step in self._build_steps():
await step(context)

View file

@ -37,6 +37,8 @@ from ...utils.async_utils import complete_in_thread
class BM25Index(BaseKeywordIndex):
"""BM25 inverted index with lazy deletion and on-disk persistence."""
workspace_scoped = True
def __init__(self, k1: float = 1.5, b: float = 0.75, index_version: str = "v1", **kwargs):
super().__init__(**kwargs)
self.k1 = k1

View file

@ -1,16 +1,22 @@
"""Per-request runtime context shared across steps and jobs."""
import asyncio
from typing import TYPE_CHECKING
from ..enumeration import ChunkEnum
from ..schema import Response, StreamChunk
if TYPE_CHECKING:
from ..schema import WorkspaceContext
class RuntimeContext:
"""Scratch space for a single execution.
Holds the response object, an optional stream queue, a stop event, and a
free-form data dict accessible via mapping-style operators (``ctx[key]``).
``workspace`` binds the whole execution to one configured workspace; job
entry points resolve the request's workspace alias into it.
"""
def __init__(
@ -18,11 +24,13 @@ class RuntimeContext:
response: Response | None = None,
stream_queue: asyncio.Queue | None = None,
stop_event: asyncio.Event | None = None,
workspace: "WorkspaceContext | None" = None,
**kwargs,
):
self.response: Response = response or Response()
self.stream_queue: asyncio.Queue | None = stream_queue
self.stop_event: asyncio.Event | None = stop_event
self.workspace: "WorkspaceContext | None" = workspace
self.data: dict = kwargs
# ----- Data dict access ----------------------------------------------

View file

@ -12,6 +12,8 @@ from ...schema import FileFrontMatter, FileNode
class LocalTagIndex(BaseTagIndex):
"""Maintain bidirectional path/tag relationships without separate source I/O."""
workspace_scoped = True
reserved_tag_keys = frozenset(FileFrontMatter.model_fields) | {
"kind",
"session_id",

View file

@ -25,6 +25,7 @@ from .response import Response
from .stream_chunk import StreamChunk
from .token_usage import TokenUsage
from .traverse_graph import TraverseGraph, TraverseGraphEdge, TraverseGraphNode
from .workspace import WorkspaceContext
__all__ = [
"CallDecision",
@ -57,4 +58,5 @@ __all__ = [
"TraverseGraph",
"TraverseGraphEdge",
"TraverseGraphNode",
"WorkspaceContext",
]

View file

@ -1,13 +1,17 @@
"""Application configuration models."""
from collections.abc import Mapping
from itertools import combinations
import os
from pathlib import Path
from pathlib import PurePosixPath, PureWindowsPath
from pydantic import BaseModel, ConfigDict, Field, field_validator
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
from ..enumeration import component_type_name
DEFAULT_WORKSPACE_ALIAS = "default"
class ComponentConfig(BaseModel):
"""Base config for a component; extra fields allowed for backend-specific options."""
@ -37,9 +41,18 @@ class ApplicationConfig(BaseModel):
)
workspace_dir: str = Field(
default=".reme",
description="Workspace root directory for runtime files",
description=(
"Workspace root directory for runtime files. Accepts a single path or an ordered mapping of "
"alias to path; the mapping form normalizes to the first entry here and exposes all entries "
"through the derived workspaces mapping."
),
validate_default=True,
)
workspaces: dict[str, str] = Field(
default_factory=dict,
exclude=True,
description="Ordered alias-to-absolute-path workspaces derived from workspace_dir",
)
metadata_dir: str = Field(default="metadata", description="Subdirectory for ReMe persistent state")
session_dir: str = Field(default="session", description="Subdirectory for persisted agent sessions")
# dialog_dir was removed; standard transcripts are always derived as ``{session_dir}/dialog``.
@ -62,6 +75,66 @@ class ApplicationConfig(BaseModel):
description="Component registry keyed by type then name",
)
@model_validator(mode="before")
@classmethod
def normalize_workspace_mapping(cls, data):
"""Resolve a mapping workspace_dir, keeping its first entry as the default workspace."""
if not isinstance(data, dict):
return data
if "workspaces" in data:
raise ValueError("workspaces is derived from workspace_dir; do not supply it")
raw = data.get("workspace_dir")
if not isinstance(raw, Mapping):
return data
if not raw:
raise ValueError("workspace_dir mapping must contain at least one workspace")
resolved: dict[str, str] = {}
for key, path in raw.items():
if not isinstance(key, str):
raise ValueError(f"workspace_dir aliases must be strings: {key!r}")
alias = key.strip()
if not alias:
raise ValueError("workspace_dir aliases must be non-empty")
if alias in resolved:
raise ValueError(f"workspace_dir aliases collide after normalization: {alias!r}")
if not isinstance(path, str) or not path.strip():
raise ValueError(f"workspace_dir[{key!r}] must be a non-empty path string: {path!r}")
resolved[alias] = str(Path(path).expanduser().resolve(strict=False))
directories = list(resolved.values())
if len(set(directories)) != len(directories):
raise ValueError(f"workspace_dir aliases must point to distinct directories: {resolved}")
for first, second in combinations(map(Path, directories), 2):
if first.is_relative_to(second) or second.is_relative_to(first):
raise ValueError(
f"workspace_dir directories must not overlap: {str(first)!r} and {str(second)!r}",
)
return {**data, "workspace_dir": directories[0], "workspaces": resolved}
@model_validator(mode="after")
def derive_default_workspace(self):
"""Normalize every workspace_dir form to an ordered mapping; the first entry is the default."""
self.workspaces = dict(self.workspaces) or {DEFAULT_WORKSPACE_ALIAS: self.workspace_dir}
if len(self.workspaces) > 1:
if not self.mem_session_dir.strip():
raise ValueError("mem_session_dir must be non-empty in multi-workspace configurations")
for alias, workspace in self.workspaces.items():
root = Path(workspace).resolve(strict=False)
for field_name in (
"metadata_dir",
"session_dir",
"mem_session_dir",
"resource_dir",
"daily_dir",
"digest_dir",
):
subdir = getattr(self, field_name)
if subdir and not (root / subdir).resolve(strict=False).is_relative_to(root):
raise ValueError(
f"{field_name} for workspace {alias!r} must stay inside the workspace "
"in multi-workspace configurations",
)
return self
@field_validator("components", mode="before")
@classmethod
def normalize_component_types(cls, value):

93
reme/schema/workspace.py Normal file
View file

@ -0,0 +1,93 @@
"""Per-workspace path views and owned state for multi-workspace applications."""
import asyncio
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from .application_config import ApplicationConfig
@dataclass
class WorkspaceAgentState:
"""Session execution state owned by one agent wrapper within one workspace."""
thread_tool_contexts: dict[str, str] = field(default_factory=dict)
@dataclass
class WorkspaceCodexThreadState:
"""Authoritative Codex thread ownership shared by every wrapper in a workspace."""
owned_thread_ids: set[str] | None = None
lock: asyncio.Lock = field(default_factory=asyncio.Lock)
@dataclass(frozen=True)
class WorkspaceContext:
"""Owns the paths, stateful components, execution state, and locks of one workspace."""
alias: str
workspace_path: Path
metadata_path: Path | None
session_path: Path | None
mem_session_path: Path | None
resource_path: Path | None
daily_path: Path | None
digest_path: Path | None
components: dict[str, dict[str, Any]] = field(default_factory=dict)
tool_contexts: dict[str, Any] = field(default_factory=dict)
search_call_budgets: dict[str, Any] = field(default_factory=dict)
agent_states: dict[str, WorkspaceAgentState] = field(default_factory=dict)
codex_thread_state: WorkspaceCodexThreadState = field(default_factory=WorkspaceCodexThreadState)
locks: dict[str, asyncio.Lock] = field(default_factory=dict)
def agent_state(self, wrapper_name: str) -> WorkspaceAgentState:
"""Return the named wrapper's state within this workspace."""
return self.agent_states.setdefault(wrapper_name, WorkspaceAgentState())
@classmethod
def from_config(cls, alias: str, workspace: str, config: "ApplicationConfig") -> "WorkspaceContext":
"""Resolve one alias of the config's workspaces mapping into absolute paths."""
root = Path(workspace).absolute()
def child(subdir: str) -> Path | None:
return root / subdir if subdir else None
return cls(
alias=alias,
workspace_path=root,
metadata_path=child(config.metadata_dir),
session_path=child(config.session_dir),
mem_session_path=child(config.mem_session_dir),
resource_path=child(config.resource_dir),
daily_path=child(config.daily_dir),
digest_path=child(config.digest_dir),
)
def get_component(self, ctype: str, name: str) -> Any:
"""Return a component owned by this workspace, never a default or shared substitute."""
try:
return self.components[ctype][name]
except KeyError as exc:
raise KeyError(f"Component '{name}' in {ctype} has no instance for workspace '{self.alias}'") from exc
def storage_dirs(self) -> list[Path]:
"""Subdirectories the application materializes at startup, in configuration order."""
return [
path
for path in (
self.metadata_path,
self.session_path,
self.mem_session_path,
self.resource_path,
self.daily_path,
self.digest_path,
)
if path is not None
]
def lock(self, name: str) -> asyncio.Lock:
"""Return the workspace-scoped lock for ``name``, created lazily."""
return self.locks.setdefault(name, asyncio.Lock())

View file

@ -7,7 +7,7 @@ from typing import Any, TYPE_CHECKING
from agentscope.model import ChatModelBase
from ..components.agent_wrapper.base_agent_wrapper import BaseAgentWrapper
from ..components.base_component import ComponentMixin
from ..components.base_component import ComponentMixin, _context_workspaces
from ..components.component_registry import R
from ..components.file_catalog import BaseFileCatalog
from ..components.file_store import BaseFileStore
@ -87,7 +87,19 @@ class Ref:
if self.optional:
return None
raise RuntimeError(f"app_context is not set when resolving '{self.key}'")
comp = obj.app_context.components.get(self.comp_enum, {}).get(name)
if (workspaces := _context_workspaces(obj.app_context)) is not None:
context = obj.context
bound = getattr(context, "workspace", None) if context is not None else None
is_scoped = any(name in workspace.components.get(self.comp_enum, {}) for workspace in workspaces.values())
if is_scoped:
if bound is None:
raise RuntimeError(f"A workspace binding is required when resolving '{self.key}'")
comp = bound.get_component(self.comp_enum, name)
else:
comp = obj.app_context.shared_components.get(self.comp_enum, {}).get(name)
else:
# Test doubles and standalone-style contexts without the workspace layer.
comp = obj.app_context.components.get(self.comp_enum, {}).get(name)
if comp is None:
if self.optional:
return None
@ -149,6 +161,8 @@ class BaseStep(ComponentMixin, ABC):
del self.__dict__[key]
self.context = RuntimeContext.from_context(context, **kwargs)
assert self.context is not None
if (workspaces := _context_workspaces(self.app_context)) is not None and self.context.workspace is None:
self.context.workspace = next(iter(workspaces.values()))
if self.input_mapping:
self.context.apply_mapping(self.input_mapping)
result = await self.execute()
@ -189,12 +203,37 @@ class BaseStep(ComponentMixin, ABC):
return self.app_context.jobs.get(name)
async def run_job(self, name: str, /, **kwargs) -> Response:
"""Execute a job by name and kwargs, return the final response."""
"""Execute a job by name and kwargs, return the final response.
Nested calls inherit the current call's workspace binding unless the
caller selects a workspace explicitly.
"""
job: "BaseJob | None" = self.get_job(name)
if job is None:
raise RuntimeError(f"Job {name} not found")
if alias := self._bound_workspace_alias():
kwargs.setdefault("workspace", alias)
return await job(**kwargs)
def _bound_workspace_alias(self) -> str:
"""Alias of the current call's workspace binding ("" only when unbound).
Execution bindings are explicit end to end: the default alias is a
real binding and is forwarded like any other, so a nested job's own
configured workspace cannot silently override the caller's binding.
"""
bound = getattr(self.context, "workspace", None) if self.context is not None else None
return bound.alias if bound is not None else ""
def _workspace_forward_kwargs(self) -> dict:
"""Forward the current call's resolved workspace binding to agent wrapper calls.
Every resolved binding is forwarded, including the default alias, so
agent tool paths execute against the caller's workspace.
"""
alias = self._bound_workspace_alias()
return {"workspace": alias} if alias else {}
def _resolve_dispatch_step(self, raw: _DispatchStep):
"""Resolve a dispatch step spec to (step class, init params)."""
if isinstance(raw, str):

View file

@ -50,6 +50,7 @@ this chat intentionally provides read-only tools. Reply in the user's language."
"system_prompt": self._system_prompt(),
"job_tools": self.READ_ONLY_TOOLS,
"builtin_tools": [],
**self._workspace_forward_kwargs(),
}
if session_id := self.context.get("session_id"):
wrapper_kwargs["resume"] = str(session_id)

View file

@ -630,6 +630,7 @@ class AutoMemoryStep(BaseStep):
system_prompt=self.prompt_format("system_prompt"),
job_tools=self.create_tools if created else self.update_tools,
**reply_kwargs,
**self._workspace_forward_kwargs(),
)
self.logger.info(f"[{self.name}] agent done path={note_path} has_result={bool(result.get('result'))}")

View file

@ -119,6 +119,7 @@ class AutoTagStep(BaseStep):
),
job_tools=self.tools,
injected_job_kwargs=tool_context,
**self._workspace_forward_kwargs(),
)
path = self.workspace_path / target.path

View file

@ -15,6 +15,7 @@ import frontmatter
from agentscope.message import DataBlock, TextBlock, UserMsg
from watchfiles import Change
from ...components.base_component import _context_workspaces
from ...components.runtime_context import RuntimeContext
from ..base_step import BaseStep
from ..file_io import refresh_day_index, validate_filename_component
@ -74,9 +75,9 @@ def _compute_note_stem(filename: str) -> str:
return PurePosixPath(filename).stem
def _compute_agent_session_id(path: str) -> str:
"""Return a stable UUID session id for agent backends."""
return str(uuid.uuid5(uuid.NAMESPACE_URL, path))
def _compute_agent_session_id(path: str, workspace_alias: str = "") -> str:
"""Return a stable UUID session id for agent backends, namespaced per workspace."""
return str(uuid.uuid5(uuid.NAMESPACE_URL, f"{workspace_alias}:{path}" if workspace_alias else path))
def _parse_resource_path(file_path: str, resource_dir: str) -> tuple[str, str]:
@ -401,9 +402,19 @@ class BaseAutoResourceStep(BaseStep):
"system_prompt": self.prompt_format("system_prompt"),
"job_tools": self.create_tools if state.created else self.update_tools,
**(reply_kwargs or {}),
**self._workspace_forward_kwargs(),
}
if "session_id" not in agent_kwargs:
agent_kwargs["session_id"] = _compute_agent_session_id(file_path)
# Storage compatibility: the default workspace keeps its historical
# (unprefixed) session UUIDs; other aliases are namespaced.
session_alias = agent_kwargs.get("workspace", "")
workspaces = _context_workspaces(self.app_context)
if workspaces and session_alias == next(iter(workspaces)):
session_alias = ""
agent_kwargs["session_id"] = _compute_agent_session_id(
file_path,
workspace_alias=session_alias,
)
# Consume the resource-only option before forwarding kwargs to the wrapper.
if agent_kwargs.pop("scope_note_tools", False):
# Bind the final allocated path last, never a path supplied by the agent.

View file

@ -302,6 +302,7 @@ class DreamExtractStep(BaseStep):
user_message,
system_prompt=system_prompt,
job_tools=list(_TOOLS),
**self._workspace_forward_kwargs(),
)
self.logger.info(f"[{self.name}] agent done files={len(batch)} has_result={bool(result.get('result'))}")
raw_result = agent_reply_result_text(result)
@ -324,6 +325,7 @@ class DreamExtractStep(BaseStep):
user_message + retry_feedback,
system_prompt=system_prompt,
job_tools=list(_TOOLS),
**self._workspace_forward_kwargs(),
)
raw_result = agent_reply_result_text(result)
state.extract_summary = "\n".join(filter(None, (state.extract_summary, raw_result)))

View file

@ -100,9 +100,16 @@ class DreamIntegrateStep(BaseStep):
return self._finish(state, not state.failed_units, answer)
def _integration_lock(self) -> asyncio.Lock:
"""Return the Application-wide lock used by AutoDream integration."""
"""Return the workspace-scoped lock used by AutoDream integration.
Bound calls lock per workspace so integrations in different workspaces
never block each other; standalone contexts keep the app-wide lock.
"""
if self.app_context is None:
raise RuntimeError("dream_integrate_step requires app_context")
bound = getattr(self.context, "workspace", None) if self.context is not None else None
if bound is not None:
return bound.lock("dream_integration")
key = "dream_integration_lock"
lock = self.app_context.metadata.get(key)
if lock is None:
@ -145,6 +152,7 @@ class DreamIntegrateStep(BaseStep):
bucket=bucket,
),
job_tools=list(_TOOLS),
**self._workspace_forward_kwargs(),
)
except Exception as e: # noqa: BLE001
after = _snapshot_digest(workspace, digest_dir)

View file

@ -21,6 +21,7 @@ import time
from ...base_step import BaseStep
from .._evolve import passthrough_response
from ....components import R
from ....components.base_component import _context_workspaces
from ....enumeration import ComponentEnum
from ....schema import ProactiveState, ProactiveStateFile, ProactiveTopic
from ....schema.proactive import clamp_confidence
@ -340,7 +341,14 @@ class ProactiveTopicsStep(BaseStep):
async def _digest_titles(self) -> list[str]:
if self.app_context is None or self.digest_compare_limit <= 0:
return []
catalog = self.app_context.components.get(ComponentEnum.FILE_CATALOG, {}).get("digest")
bound = getattr(self.context, "workspace", None) if self.context is not None else None
if _context_workspaces(self.app_context) is not None:
if bound is None:
raise RuntimeError("A workspace binding is required to resolve the digest catalog")
catalog = bound.components.get(ComponentEnum.FILE_CATALOG, {}).get("digest")
else:
# Test doubles and standalone-style contexts without the workspace layer.
catalog = self.app_context.components.get(ComponentEnum.FILE_CATALOG, {}).get("digest")
if catalog is None:
return []
try:
@ -358,7 +366,7 @@ class ProactiveTopicsStep(BaseStep):
name = self.kwargs.get("as_embedding", "default")
if self.app_context is None:
return None
return self.app_context.components.get(ComponentEnum.AS_EMBEDDING, {}).get(name)
return self.app_context.shared_components.get(ComponentEnum.AS_EMBEDDING, {}).get(name)
async def _semantic_verdict(
self,

View file

@ -681,7 +681,7 @@ def resolve_agent_wrapper(step):
if wrapper is not None:
return wrapper
if step.app_context is not None:
fallback = step.app_context.components.get(ComponentEnum.AGENT_WRAPPER, {}).get("default")
fallback = step.app_context.shared_components.get(ComponentEnum.AGENT_WRAPPER, {}).get("default")
if fallback is not None:
configured = step.kwargs.get("agent_wrapper", "default")
step.logger.warning(f"[{step.name}] agent_wrapper '{configured}' missing; using default")

View file

@ -16,6 +16,9 @@ class _ToolContextMixin:
kwargs: dict[str, Any]
def _tool_context_store(self, tool_context_id: str) -> dict:
bound = getattr(self.context, "workspace", None) if self.context is not None else None
if bound is not None:
return bound.tool_contexts.setdefault(tool_context_id, {})
if self.app_context is not None:
contexts = self.app_context.metadata.setdefault(self.TOOL_CONTEXTS_KEY, {})
else:

View file

@ -237,7 +237,11 @@ class SearchStep(_ToolContextMixin, BaseStep):
maximum = int(max_search_calls)
if maximum < 1:
raise ValueError("max_search_calls must be positive")
budgets = self.app_context.metadata.setdefault("__search_call_budgets", {})
bound = getattr(self.context, "workspace", None) if self.context is not None else None
if bound is not None:
budgets = bound.search_call_budgets
else:
budgets = self.app_context.metadata.setdefault("__search_call_budgets", {})
budget = budgets.setdefault(tool_context_id, {"count": 0, "lock": asyncio.Lock()})
async with budget["lock"]:
if budget["count"] >= maximum:

View file

@ -13,6 +13,7 @@ from watchfiles import Change
from ._change_batch import bucket_changes
from ..base_step import BaseStep
from ...components import R
from ...components.base_component import _context_workspaces
from ...components.file_chunker import BaseFileChunker
from ...enumeration import ComponentEnum
from ...schema import FileChunk, FileNode
@ -534,8 +535,14 @@ class UpdateIndexStep(ChangeApplyStep):
return await chunker.chunk(path)
def _resolve_chunker(self, path: Path) -> BaseFileChunker:
"""Resolve a file chunker for a given path."""
chunkers: dict[str, BaseFileChunker] = self.app_context.components[ComponentEnum.FILE_CHUNKER]
"""Resolve a file chunker for a given path, bound to this call's workspace."""
bound = getattr(self.context, "workspace", None) if self.context is not None else None
if bound is not None:
chunkers: dict[str, BaseFileChunker] = bound.components.get(ComponentEnum.FILE_CHUNKER, {})
elif _context_workspaces(self.app_context) is None:
chunkers = self.app_context.components[ComponentEnum.FILE_CHUNKER]
else:
raise RuntimeError("A workspace binding is required to resolve file chunkers")
suffix = path.suffix.lstrip(".").lower()
for candidate in chunkers.values():
if suffix and suffix in {ext.lower().lstrip(".") for ext in candidate.supported_extensions}:

View file

@ -563,7 +563,9 @@ async def test_configured_wrapper_and_job_image_options(
if suffix == "txt":
assert response.answer == "ok"
assert isinstance(reply.call_args.args[0], str)
assert set(reply.call_args.kwargs) == {"system_prompt", "job_tools", "session_id"}
assert set(reply.call_args.kwargs) == {"system_prompt", "job_tools", "session_id", "workspace"}
# Execution bindings are forwarded everywhere, including the default alias.
assert reply.call_args.kwargs["workspace"] == "default"
elif expected_action == "added":
note = frontmatter.load(tmp_path / result["path"])
assert note["source_resource"] == f"[[resource/2026-01-01/photo.{suffix}]]"

View file

@ -182,7 +182,7 @@ async def test_agentscope_builtin_tools_are_opt_in(tmp_path, monkeypatch, reply_
wrapper.as_llm = SimpleNamespace(model=object())
observed = {}
def builtin_tools(names, *, sequential_tool_calls=False):
def builtin_tools(names, *, sequential_tool_calls=False, **_extra):
observed["names"] = names
observed["sequential_tool_calls"] = sequential_tool_calls
return []
@ -251,7 +251,7 @@ async def test_agentscope_structured_output_uses_model_tool_choice_policy(tmp_pa
"""Return the fake AgentScope agent."""
return FakeAgent(), inputs
async def dump_state(_state):
async def dump_state(_state, _kwargs=None):
"""Skip session persistence."""
return None

View file

@ -379,7 +379,7 @@ async def test_agentscope_compact_session_forces_and_persists_compression(tmp_pa
observed["build"] = (inputs, kwargs)
return FakeAgent(), inputs
async def dump_state(state):
async def dump_state(state, _kwargs=None):
observed["state"] = state
monkeypatch.setattr(wrapper, "_build_agent", build_agent)
@ -387,7 +387,11 @@ async def test_agentscope_compact_session_forces_and_persists_compression(tmp_pa
await wrapper.compact_session("session-1")
assert observed["build"] == (None, {"resume": "session-1"})
build_inputs, build_kwargs = observed["build"]
assert build_inputs is None
# Default-alias fill-in is marked so cwd resolution can ignore it.
assert build_kwargs.pop("_workspace_explicit") is False
assert build_kwargs == {"resume": "session-1", "workspace": "default"}
assert observed["config"].trigger_ratio == 1e-9
assert observed["state"] is FakeAgent.state

View file

@ -8,15 +8,14 @@ import json
import os
from pathlib import Path
import stat
import sys
from types import SimpleNamespace
import pytest
from openai_codex.generated.v2_all import TokenUsageBreakdown
from pydantic import BaseModel
from reme.components import ApplicationContext
from reme.components.agent_wrapper.codex_agent_wrapper import CodexAgentWrapper
from reme.components.agent_wrapper.codex_mcp_server import _prepare_config
from reme.components.job import BackgroundJob
from reme.components.outbound_proxy import FixedHttpOutboundProxy
from reme.config import resolve_app_config
@ -64,10 +63,10 @@ def _wrapper(tmp_path, **kwargs):
return CodexAgentWrapper(app_context=context, **kwargs), job
def test_mcp_config_uses_stdio_bridge_and_selected_jobs(tmp_path):
def test_mcp_config_uses_legacy_child_bridge_and_selected_jobs(tmp_path):
wrapper, _job = _wrapper(tmp_path, mcp_config="custom.yaml")
config = wrapper._mcp_server_config( # pylint: disable=protected-access
config = wrapper._mcp_server_config(
{"job_tools": ["search", "search"], "tool_context_id": "ctx-1"},
)
@ -77,13 +76,16 @@ def test_mcp_config_uses_stdio_bridge_and_selected_jobs(tmp_path):
assert config["args"][config["args"].index("--job") + 1] == "search"
assert "reme.components.agent_wrapper.codex_mcp_server" in config["args"]
assert config["args"][config["args"].index("--config") + 1] == str(tmp_path / "custom.yaml")
assert config["args"][config["args"].index("--workspace") + 1] == str(tmp_path)
assert config["args"][config["args"].index("--tool-context-id") + 1] == "ctx-1"
assert "--endpoint" not in config["args"]
assert "--token" not in config["args"]
def test_mcp_config_serializes_injected_job_kwargs(tmp_path):
wrapper, _job = _wrapper(tmp_path, mcp_config="custom.yaml")
config = wrapper._mcp_server_config( # pylint: disable=protected-access
config = wrapper._mcp_server_config(
{
"job_tools": ["search"],
"tool_context_id": "ctx-1",
@ -94,27 +96,13 @@ def test_mcp_config_serializes_injected_job_kwargs(tmp_path):
raw = config["args"][config["args"].index("--injected-job-kwargs") + 1]
assert json.loads(raw) == {"_allowed_paths": ["daily/2025-06-01/note.md"]}
plain = wrapper._mcp_server_config({"job_tools": ["search"]}) # pylint: disable=protected-access
plain = wrapper._mcp_server_config({"job_tools": ["search"]})
assert "--injected-job-kwargs" not in plain["args"]
def test_prepare_config_merges_injected_job_kwargs_with_tool_context():
prepared = _prepare_config(
{"jobs": {"selected": {"backend": "base"}}},
["selected"],
"ctx-1",
{"_allowed_paths": ["daily/2025-06-01/note.md"]},
)
assert prepared["service"]["injected_job_kwargs"] == {
"_allowed_paths": ["daily/2025-06-01/note.md"],
"tool_context_id": "ctx-1",
}
def test_thread_config_preserves_other_mcp_servers(tmp_path):
wrapper, _job = _wrapper(tmp_path)
config = wrapper._thread_config( # pylint: disable=protected-access
config = wrapper._thread_config(
{
"job_tools": ["search"],
"config": {"mcp_servers": {"docs": {"url": "https://example.test/mcp"}}},
@ -171,114 +159,19 @@ def test_mcp_config_rejects_background_jobs(tmp_path):
wrapper._mcp_server_config({"job_tools": ["watch"]})
def test_prepare_config_reuses_selected_stdio_mcp_service():
prepared = _prepare_config(
{
"service": {"backend": "http"},
"jobs": {
"selected": {"backend": "base", "enable_serve": False},
"helper": {"backend": "base"},
"watch": {"backend": "background"},
},
},
["selected"],
"ctx-1",
)
assert prepared["service"] == {
"backend": "mcp",
"transport": "stdio",
"jobs": ["selected"],
"tool_error_on_failure": True,
"injected_job_kwargs": {"tool_context_id": "ctx-1"},
}
assert set(prepared["jobs"]) == {"selected", "helper"}
assert prepared["jobs"]["selected"]["enable_serve"] is True
def test_prepare_config_rejects_missing_or_background_selected_jobs():
with pytest.raises(KeyError, match="missing"):
_prepare_config({"jobs": {}}, ["missing"])
with pytest.raises(KeyError, match="watch"):
_prepare_config({"jobs": {"watch": {"backend": "background"}}}, ["watch"])
@pytest.mark.asyncio
async def test_stdio_bridge_starts_and_lists_selected_job(tmp_path):
from fastmcp import Client
from fastmcp.client import StdioTransport
config_path = tmp_path / "bridge.json"
config_path.write_text(
json.dumps(
{
"service": {"backend": "mcp"},
"workspace_dir": str(tmp_path / "workspace"),
"jobs": {
"empty": {
"backend": "base",
"description": "Return an empty response",
"parameters": {"type": "object", "properties": {}},
"steps": [],
},
},
},
),
encoding="utf-8",
)
transport = StdioTransport(
command=sys.executable,
args=[
"-m",
"reme.components.agent_wrapper.codex_mcp_server",
"--config",
str(config_path),
"--workspace",
str(tmp_path / "workspace"),
"--job",
"empty",
],
cwd=str(Path(__file__).resolve().parents[2]),
keep_alive=False,
)
async with Client(transport, timeout=None, init_timeout=STDIO_INIT_TIMEOUT) as client:
tools = await asyncio.wait_for(client.list_tools(), timeout=10)
assert [tool.name for tool in tools] == ["empty"]
@pytest.mark.asyncio
async def test_stdio_bridge_stdout_is_protocol_clean(tmp_path):
config_path = tmp_path / "bridge.json"
config_path = tmp_path / "config.json"
config_path.write_text(
json.dumps(
{
"service": {"backend": "mcp"},
"workspace_dir": str(tmp_path / "workspace"),
"jobs": {
"empty": {
"backend": "base",
"description": "Empty",
"parameters": {"type": "object", "properties": {}},
"steps": [],
},
},
},
),
json.dumps({"jobs": {"search": {"backend": "base", "steps": []}}}),
encoding="utf-8",
)
wrapper, _job = _wrapper(tmp_path, mcp_config=str(config_path))
server_config = wrapper._mcp_server_config({"job_tools": ["search"]})
proc = await asyncio.create_subprocess_exec(
sys.executable,
"-m",
"reme.components.agent_wrapper.codex_mcp_server",
"--config",
str(config_path),
"--workspace",
str(tmp_path / "workspace"),
"--job",
"empty",
cwd=str(Path(__file__).resolve().parents[2]),
server_config["command"],
*server_config["args"],
cwd=server_config["cwd"],
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
@ -309,6 +202,7 @@ async def test_stdio_bridge_stdout_is_protocol_clean(tmp_path):
except asyncio.TimeoutError:
proc.kill()
await proc.wait()
await wrapper.close()
stdout = first_line + await proc.stdout.read()
stderr = (await proc.stderr.read()).decode()
assert b"Loading config" not in stdout
@ -828,6 +722,63 @@ async def test_reply_normalizes_schema_and_reuses_persistent_client(tmp_path, mo
assert close_count == 1
@pytest.mark.asyncio
async def test_reply_keeps_project_root_cwd_without_explicit_workspace(tmp_path, monkeypatch):
"""A single-workspace reply delivers the project root to the SDK.
The default alias is still injected for job tools. Naming that alias, or
choosing another workspace, selects that workspace directory instead.
"""
project = tmp_path / "project"
workspace = project / ".reme"
context = ApplicationContext(workspace_dir=str(workspace))
wrapper = CodexAgentWrapper(
app_context=context,
project_path="..",
auth_mode="oauth",
release_threads=False,
)
observed = {"thread": [], "turn": []}
class FakeThread:
id = "thread-1"
async def run(self, _inputs, **kwargs):
observed["turn"].append(kwargs["cwd"])
return _TurnResult(final_response="ok")
class FakeCodex:
def __init__(self, _config):
pass
async def account(self):
return SimpleNamespace(account=SimpleNamespace())
async def close(self):
return None
async def thread_start(self, **kwargs):
observed["thread"].append(kwargs["cwd"])
return FakeThread()
monkeypatch.setattr("reme.components.agent_wrapper.codex_agent_wrapper.AsyncCodex", FakeCodex)
merged = wrapper._merged_kwargs({}) # pylint: disable=protected-access
assert merged["workspace"] == "default"
assert wrapper._resolve_injected_job_kwargs(merged)["workspace"] == "default" # pylint: disable=protected-access
await wrapper.reply("hello")
await wrapper.reply("again", workspace="default")
await wrapper.close()
project_cwd = str(wrapper.project_path)
workspace_cwd = str(context.workspaces["default"].workspace_path)
assert wrapper.cwd == wrapper.project_path
assert project_cwd != workspace_cwd
assert observed["thread"] == [project_cwd, workspace_cwd]
assert observed["turn"] == [project_cwd, workspace_cwd]
@pytest.mark.asyncio
@pytest.mark.parametrize("schema", [{}, _StructuredModel])
async def test_reply_stream_rejects_output_schema(tmp_path, schema):
@ -1199,11 +1150,6 @@ def test_codex_home_expands_user_directory(tmp_path):
assert wrapper.session_path == Path.home() / ".codex"
def test_named_default_mcp_config_remains_supported(tmp_path):
wrapper, _job = _wrapper(tmp_path, mcp_config="default")
assert wrapper._mcp_config_source({}) == "default" # pylint: disable=protected-access
def test_default_config_provides_codex_oauth_wrapper(monkeypatch):
monkeypatch.delenv("CODEX_HOME", raising=False)
config = resolve_app_config(log_config=False)

View file

@ -0,0 +1,160 @@
"""Compaction claims a Codex thread before any SDK side effect."""
# pylint: disable=protected-access,missing-function-docstring
import asyncio
import json
from types import SimpleNamespace
import pytest
from reme.components import ApplicationContext
from reme.components.agent_wrapper.codex_agent_wrapper import CodexAgentWrapper
def _owned_on_disk(tmp_path, alias) -> set[str]:
path = tmp_path / alias / "mem_session" / "codex" / "owned_threads.json"
if not path.is_file():
return set()
return set(json.loads(path.read_text(encoding="utf-8"))["thread_ids"])
@pytest.mark.asyncio
async def test_compact_arbitrates_before_sdk_side_effects(tmp_path, monkeypatch):
"""Compaction claims ownership before resume/compact, including under concurrency."""
context = ApplicationContext(workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")})
alpha = CodexAgentWrapper(name="codex-alpha", app_context=context, release_threads=False)
beta = CodexAgentWrapper(name="codex-beta", app_context=context, release_threads=False)
sdk_ops: list[str] = []
entered_sdk = asyncio.Event()
release_sdk = asyncio.Event()
class PausingThread:
"""Resume pauses inside the SDK so a competing resume can try to claim."""
id = "shared-thread"
async def compact(self):
sdk_ops.append("compact")
async def alpha_resume(_thread_id):
sdk_ops.append("alpha-resume")
entered_sdk.set()
await release_sdk.wait()
return PausingThread()
async def beta_resume(thread_id, **_kwargs):
sdk_ops.append("beta-resume")
return SimpleNamespace(id=thread_id)
async def alpha_codex():
return SimpleNamespace(thread_resume=alpha_resume)
monkeypatch.setattr(alpha, "_get_codex", alpha_codex)
compact_task = asyncio.create_task(alpha.compact_session("shared-thread", workspace="alpha"))
await asyncio.wait_for(entered_sdk.wait(), timeout=2)
with pytest.raises(ValueError, match="belongs to workspace 'alpha'"):
await beta._open_thread(
SimpleNamespace(thread_resume=beta_resume),
{"session_id": "shared-thread", "workspace": "beta"},
)
release_sdk.set()
await asyncio.wait_for(compact_task, timeout=2)
assert sdk_ops == ["alpha-resume", "compact"]
assert context.workspaces["alpha"].codex_thread_state.owned_thread_ids == {"shared-thread"}
assert "shared-thread" not in (context.workspaces["beta"].codex_thread_state.owned_thread_ids or set())
assert _owned_on_disk(tmp_path, "alpha") == {"shared-thread"}
assert "shared-thread" not in _owned_on_disk(tmp_path, "beta")
# An alias that was never configured fails before either SDK method.
unknown_ops: list[str] = []
async def unknown_resume(_thread_id):
unknown_ops.append("resume")
return PausingThread()
async def unknown_codex():
return SimpleNamespace(thread_resume=unknown_resume)
monkeypatch.setattr(alpha, "_get_codex", unknown_codex)
with pytest.raises(ValueError, match="Unknown workspace 'typo'"):
await alpha.compact_session("unrecorded-thread", workspace="typo")
assert not unknown_ops
assert "unrecorded-thread" not in (context.workspaces["alpha"].codex_thread_state.owned_thread_ids or set())
# Once beta owns a thread, alpha's compaction does not reach the SDK.
await beta._open_thread(
SimpleNamespace(thread_resume=beta_resume),
{"session_id": "beta-thread", "workspace": "beta"},
)
loser_ops: list[str] = []
async def loser_resume(_thread_id):
loser_ops.append("resume")
return PausingThread()
async def loser_codex():
return SimpleNamespace(thread_resume=loser_resume)
monkeypatch.setattr(alpha, "_get_codex", loser_codex)
with pytest.raises(ValueError, match="belongs to workspace 'beta'"):
await alpha.compact_session("beta-thread", workspace="alpha")
assert not loser_ops
assert context.workspaces["beta"].codex_thread_state.owned_thread_ids == {"beta-thread"}
@pytest.mark.asyncio
async def test_compact_drops_new_claim_when_sdk_rejects(tmp_path, monkeypatch):
"""A failed resume or compact releases a claim this call just made."""
context = ApplicationContext(workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")})
alpha = CodexAgentWrapper(name="codex-alpha", app_context=context)
beta = CodexAgentWrapper(name="codex-beta", app_context=context, release_threads=False)
released: list[str] = []
class RecordingClient:
"""Capture thread ids passed to the SDK unsubscribe request."""
async def request(self, method, params, *, response_model):
released.append(params["threadId"])
assert method == "thread/unsubscribe"
assert response_model is not None
return SimpleNamespace()
class RejectingThread:
"""Resume succeeds and compaction is rejected by the SDK."""
def __init__(self, thread_id):
self.id = thread_id
async def compact(self):
raise RuntimeError("sdk rejected")
async def resume(thread_id):
if thread_id == "resume-fails":
raise RuntimeError("resume rejected")
return RejectingThread(thread_id)
async def get_codex():
return SimpleNamespace(_client=RecordingClient(), thread_resume=resume)
monkeypatch.setattr(alpha, "_get_codex", get_codex)
with pytest.raises(RuntimeError, match="resume rejected"):
await alpha.compact_session("resume-fails", workspace="alpha")
with pytest.raises(RuntimeError, match="sdk rejected"):
await alpha.compact_session("new-thread", workspace="alpha")
await alpha._record_thread_ownership("alpha", "kept-thread")
with pytest.raises(RuntimeError, match="sdk rejected"):
await alpha.compact_session("kept-thread", workspace="alpha")
owned = context.workspaces["alpha"].codex_thread_state.owned_thread_ids or set()
assert "resume-fails" not in owned
assert "new-thread" not in owned
assert owned == {"kept-thread"}
assert _owned_on_disk(tmp_path, "alpha") == {"kept-thread"}
assert released == ["new-thread", "kept-thread"]
await beta._record_thread_ownership("beta", "new-thread")
assert context.workspaces["beta"].codex_thread_state.owned_thread_ids == {"new-thread"}

View file

@ -0,0 +1,271 @@
"""Multi-workspace workspace_dir configuration contracts (Phase 1)."""
# pylint: disable=protected-access,missing-function-docstring
from pathlib import Path
import pytest
from pydantic import ValidationError
import yaml
from reme.application import Application
from reme.components import ApplicationContext
from reme.config import resolve_app_config
from reme.enumeration import ComponentEnum
from reme.schema import ApplicationConfig, FileNode
def test_string_workspace_normalizes_to_default_alias(tmp_path):
config = ApplicationConfig(workspace_dir=str(tmp_path / "ws"))
assert config.workspace_dir == str(tmp_path / "ws")
assert config.workspaces == {"default": str(tmp_path / "ws")}
assert "workspaces" not in config.model_dump(mode="json")
assert "workspaces" not in config.model_dump()
def test_mapping_workspace_keeps_order_and_first_entry_as_default(tmp_path):
config = ApplicationConfig(
workspace_dir={
"beta": str(tmp_path / "b"),
"alpha": str(tmp_path / "a"),
},
)
assert list(config.workspaces) == ["beta", "alpha"]
assert config.workspace_dir == str(tmp_path / "b")
assert config.workspaces["beta"] == config.workspace_dir
def test_mapping_workspace_expands_home_and_dedupes_by_resolved_path(monkeypatch, tmp_path):
monkeypatch.setenv("HOME", str(tmp_path))
config = ApplicationConfig(workspace_dir={"home": "~/ws", "other": str(tmp_path / "ws2")})
assert config.workspaces["home"] == str(tmp_path / "ws")
with pytest.raises(ValidationError, match="distinct directories"):
ApplicationConfig(workspace_dir={"one": "~/ws", "two": str(tmp_path / "ws")})
def test_empty_mapping_rejected():
with pytest.raises(ValidationError, match="at least one workspace"):
ApplicationConfig(workspace_dir={})
def test_empty_alias_rejected(tmp_path):
with pytest.raises(ValidationError, match="aliases must be non-empty"):
ApplicationConfig(workspace_dir={" ": str(tmp_path / "ws")})
def test_overlapping_directories_rejected(tmp_path):
parent = tmp_path / "parent"
child = parent / "child"
child.mkdir(parents=True)
with pytest.raises(ValidationError, match="must not overlap"):
ApplicationConfig(workspace_dir={"parent": str(parent), "child": str(child)})
with pytest.raises(ValidationError, match="must not overlap"):
ApplicationConfig(workspace_dir={"child": str(child), "parent": str(parent)})
sibling = tmp_path / "parent-sibling"
config = ApplicationConfig(workspace_dir={"parent": str(parent), "sibling": str(sibling)})
assert list(config.workspaces) == ["parent", "sibling"]
def test_application_context_exposes_ordered_workspace_contexts(tmp_path):
context = ApplicationContext(
workspace_dir={"main": str(tmp_path / "main"), "scratch": str(tmp_path / "scratch")},
)
assert list(context.workspaces) == ["main", "scratch"]
assert context.workspace is context.workspaces["main"]
main = context.workspaces["main"]
assert main.alias == "main"
assert main.workspace_path == tmp_path / "main"
assert main.metadata_path == tmp_path / "main" / "metadata"
assert main.session_path == tmp_path / "main" / "session"
assert main.mem_session_path == tmp_path / "main" / "mem_session"
assert main.resource_path == tmp_path / "main" / "resource"
assert main.daily_path == tmp_path / "main" / "daily"
assert main.digest_path == tmp_path / "main" / "digest"
def test_workspace_context_skips_empty_subdirs(tmp_path):
context = ApplicationContext(workspace_dir=str(tmp_path / "ws"), digest_dir="")
workspace = context.workspace
assert workspace.digest_path is None
assert tmp_path / "ws" / "digest" not in workspace.storage_dirs()
assert len(workspace.storage_dirs()) == 5
def test_string_workspace_layout_unchanged(tmp_path):
app = Application(
workspace_dir=str(tmp_path / "ws"),
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
)
root = tmp_path / "ws"
assert root.is_dir()
for subdir in ("metadata", "session", "mem_session", "resource", "daily", "digest"):
assert (root / subdir).is_dir()
assert list(app.context.workspaces) == ["default"]
assert app.context.workspace.workspace_path == root
def test_application_creates_directories_for_every_alias(tmp_path):
Application(
workspace_dir={"main": str(tmp_path / "main"), "scratch": str(tmp_path / "scratch")},
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
)
for alias in ("main", "scratch"):
root = tmp_path / alias
assert root.is_dir()
for subdir in ("metadata", "session", "mem_session", "resource", "daily", "digest"):
assert (root / subdir).is_dir()
def test_yaml_config_loads_mapping_workspace(tmp_path):
config_path = tmp_path / "multi.yaml"
config_path.write_text(
yaml.safe_dump(
{
"workspace_dir": {"main": str(tmp_path / "m"), "scratch": str(tmp_path / "s")},
"enable_logo": False,
},
),
encoding="utf-8",
)
resolved = resolve_app_config(config=str(config_path), log_config=False)
config = ApplicationConfig(**resolved)
assert list(config.workspaces) == ["main", "scratch"]
assert config.workspace_dir == str(tmp_path / "m")
def test_default_yaml_string_workspace_still_loads(monkeypatch):
monkeypatch.delenv("CODEX_HOME", raising=False)
resolved = resolve_app_config(log_config=False)
config = ApplicationConfig(**resolved)
assert isinstance(config.workspace_dir, str)
assert list(config.workspaces) == ["default"]
assert config.workspaces["default"] == str(Path(".reme").resolve())
def test_alias_normalization_collision_rejected(tmp_path):
with pytest.raises(ValidationError, match="aliases collide"):
ApplicationConfig(workspace_dir={"alice": str(tmp_path / "a"), " alice ": str(tmp_path / "b")})
def test_non_string_alias_rejected(tmp_path):
with pytest.raises(ValidationError, match="aliases must be strings"):
ApplicationConfig(workspace_dir={123: str(tmp_path / "a")})
@pytest.mark.parametrize("bad_value", [None, 123, "", " "])
def test_invalid_mapping_value_rejected(bad_value):
with pytest.raises(ValidationError, match="non-empty path string"):
ApplicationConfig(workspace_dir={"alice": bad_value})
def test_supplied_workspaces_rejected_with_string_form(tmp_path):
with pytest.raises(ValidationError, match="derived from workspace_dir"):
ApplicationConfig(workspace_dir=str(tmp_path / "expected"), workspaces={"": str(tmp_path / "other")})
def test_supplied_workspaces_rejected_with_mapping_form(tmp_path):
with pytest.raises(ValidationError, match="derived from workspace_dir"):
ApplicationConfig(
workspace_dir={"main": str(tmp_path / "main")},
workspaces={"other": str(tmp_path / "other")},
)
def test_default_workspace_matches_directory_creation_source(tmp_path):
"""The default workspace_dir and the materialized directories share one derivation."""
app = Application(
workspace_dir={"main": str(tmp_path / "main"), "scratch": str(tmp_path / "scratch")},
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
)
assert app.config.workspace_dir == str(tmp_path / "main")
assert app.config.workspace_dir == app.config.workspaces["main"]
assert app.context.workspace.workspace_path == tmp_path / "main"
assert (tmp_path / "main" / "daily").is_dir()
@pytest.mark.parametrize(
"field_name",
["metadata_dir", "session_dir", "mem_session_dir", "resource_dir", "daily_dir", "digest_dir"],
)
def test_multi_workspace_rejects_derived_directory_outside_root(tmp_path, field_name):
with pytest.raises(ValidationError, match="workspace"):
ApplicationConfig(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
**{field_name: str(tmp_path / "shared")},
)
def test_multi_workspace_rejects_derived_directory_escape(tmp_path):
with pytest.raises(ValidationError, match="must stay inside the workspace"):
ApplicationConfig(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
metadata_dir="../shared",
)
def test_multi_workspace_requires_session_ownership_storage(tmp_path):
with pytest.raises(ValidationError, match="mem_session_dir must be non-empty"):
ApplicationConfig(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
mem_session_dir="",
)
config = ApplicationConfig(workspace_dir=str(tmp_path / "single"), mem_session_dir="")
assert config.mem_session_dir == ""
def test_single_workspace_keeps_absolute_derived_directory(tmp_path):
external = tmp_path / "external-metadata"
config = ApplicationConfig(workspace_dir=str(tmp_path / "single"), metadata_dir=str(external))
context = ApplicationContext(**config.model_dump())
assert context.workspace.metadata_path == external
def _graph_app(tmp_path):
return Application(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
components={"file_graph": {"default": {"backend": "local"}}},
)
@pytest.mark.asyncio
async def test_workspace_graph_contents_stay_isolated_across_restart(tmp_path):
app = _graph_app(tmp_path)
async with app:
for alias, path in (("alpha", "daily/alpha-private.md"), ("beta", "daily/beta-note.md")):
graph = app.context.workspaces[alias].get_component(ComponentEnum.FILE_GRAPH, "default")
await graph.upsert_nodes([FileNode(path=path, st_mtime=1.0)])
reopened = _graph_app(tmp_path)
async with reopened:
alpha = reopened.context.workspaces["alpha"].get_component(ComponentEnum.FILE_GRAPH, "default")
beta = reopened.context.workspaces["beta"].get_component(ComponentEnum.FILE_GRAPH, "default")
assert [node.path for node in await alpha.get_nodes()] == ["daily/alpha-private.md"]
assert [node.path for node in await beta.get_nodes()] == ["daily/beta-note.md"]

View file

@ -216,7 +216,7 @@ async def test_agentscope_stream_reply_does_not_record_token_usage(tmp_path, mon
"""Build the minimal stream double."""
return FakeAgent(), inputs
async def dump_state(_state):
async def dump_state(_state, _kwargs=None):
"""Avoid durable state writes in this accounting test."""
return None

View file

@ -0,0 +1,364 @@
"""Phase 2: requests and components bind to the selected workspace."""
# pylint: disable=protected-access,missing-function-docstring
import asyncio
import pytest
from reme.application import Application
from reme.components import ApplicationContext
from reme.components.base_component import BaseComponent
from reme.components.job import StreamJob
from reme.components.component_registry import R
from reme.enumeration import ChunkEnum, ComponentEnum
from reme.schema import FileChunk, FileNode
from reme.steps.base_step import BaseStep
def _app(tmp_path, **overrides):
config = {
"workspace_dir": {"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
"enable_logo": False,
"log_to_console": False,
"log_to_file": False,
"service": {"backend": "cli"},
"components": {
"tokenizer": {"default": {"backend": "regex"}},
"file_store": {"default": {"backend": "local", "embedding_store": "", "tag_index": ""}},
"keyword_index": {"default": {"backend": "bm25"}},
"file_graph": {"default": {"backend": "local"}},
},
"jobs": {
"write_note": {
"backend": "base",
"description": "write a note",
"parameters": {"type": "object", "properties": {}},
"steps": [{"backend": "write_step"}],
},
"read_note": {
"backend": "base",
"description": "read a note",
"parameters": {"type": "object", "properties": {}},
"steps": [{"backend": "read_step"}],
},
},
**overrides,
}
return Application(**config)
class _WorkspaceReporterStep(BaseStep):
"""Report the bound workspace alias as answer and stream chunk."""
async def execute(self):
bound = self.context.workspace
alias = bound.alias if bound is not None else ""
assert "workspace" not in self.context.data, "workspace alias must not leak into step payloads"
self.context.response.answer = alias
if self.context.stream:
await self.context.add_stream_string(alias, ChunkEnum.CONTENT)
def _reporter_context(tmp_path, job_cls):
registry = R.copy()
registry.register(_WorkspaceReporterStep, "workspace_reporter_step")
context = ApplicationContext(
registry=registry,
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
)
job = job_cls(
name="report",
app_context=context,
parameters={"type": "object", "properties": {}},
steps=[{"backend": "workspace_reporter_step"}],
)
return job
# ----- Selection semantics ---------------------------------------------------
@pytest.mark.asyncio
async def test_default_binding_uses_first_workspace(tmp_path):
app = _app(tmp_path)
async with app:
response = await app.run_job("write_note", path="daily/note.md", content="default-target")
assert response.success, response.answer
assert "default-target" in (tmp_path / "alpha" / "daily" / "note.md").read_text(encoding="utf-8")
assert not (tmp_path / "beta" / "daily" / "note.md").exists()
@pytest.mark.asyncio
async def test_unknown_workspace_alias_rejected(tmp_path):
app = _app(tmp_path)
async with app:
with pytest.raises(ValueError, match="Unknown workspace 'nope'"):
await app.run_job("write_note", path="daily/note.md", content="x", workspace="nope")
@pytest.mark.asyncio
async def test_unknown_workspace_alias_rejected_for_stream_job(tmp_path):
job = _reporter_context(tmp_path, StreamJob)
await job.start()
with pytest.raises(ValueError, match="Unknown workspace 'nope'"):
async for _chunk in _stream(job, workspace="nope"):
pass
await job.close()
async def _stream(job, **kwargs):
from reme.utils import execute_stream_task
queue = asyncio.Queue()
task = asyncio.create_task(job(stream_queue=queue, **kwargs))
async for chunk in execute_stream_task(stream_queue=queue, task=task, output_format="chunk"):
yield chunk
# ----- Concurrent and streaming isolation ------------------------------------
@pytest.mark.asyncio
async def test_concurrent_same_relative_path_stays_isolated(tmp_path):
app = _app(tmp_path)
async with app:
await asyncio.gather(
*(
app.run_job("write_note", path="daily/shared.md", content=f"{alias}-{i}", workspace=alias)
for i in range(5)
for alias in ("alpha", "beta")
),
)
alpha = (tmp_path / "alpha" / "daily" / "shared.md").read_text(encoding="utf-8")
beta = (tmp_path / "beta" / "daily" / "shared.md").read_text(encoding="utf-8")
assert "alpha-4" in alpha and "beta" not in alpha
assert "beta-4" in beta and "alpha" not in beta
@pytest.mark.asyncio
async def test_stream_chunks_carry_caller_workspace(tmp_path):
job = _reporter_context(tmp_path, StreamJob)
await job.start()
async def collect(alias):
return [chunk async for chunk in _stream(job, workspace=alias)]
chunks_alpha, chunks_beta = await asyncio.gather(collect("alpha"), collect("beta"))
content_alpha = [c.chunk for c in chunks_alpha if c.chunk_type == ChunkEnum.CONTENT]
content_beta = [c.chunk for c in chunks_beta if c.chunk_type == ChunkEnum.CONTENT]
assert content_alpha == ["alpha"]
assert content_beta == ["beta"]
assert chunks_alpha[-1].done and chunks_beta[-1].done
await job.close()
# ----- Component binding -----------------------------------------------------
@pytest.mark.asyncio
async def test_workspace_scoped_components_instantiate_per_alias(tmp_path):
app = _app(tmp_path)
async with app:
store_alpha = app.context.get_component(ComponentEnum.FILE_STORE, "default", "alpha")
store_beta = app.context.get_component(ComponentEnum.FILE_STORE, "default", "beta")
assert store_alpha is not store_beta
assert app.context.components[ComponentEnum.FILE_STORE]["default"] is store_alpha
assert ComponentEnum.FILE_STORE not in app.context.shared_components
assert app.context.workspaces["alpha"].get_component(ComponentEnum.FILE_STORE, "default") is store_alpha
assert app.context.workspaces["beta"].get_component(ComponentEnum.FILE_STORE, "default") is store_beta
with pytest.raises(KeyError, match="no instance for workspace 'alpha'"):
app.context.workspaces["alpha"].get_component(ComponentEnum.FILE_STORE, "missing")
assert store_alpha.workspace_path == tmp_path / "alpha"
assert store_beta.workspace_path == tmp_path / "beta"
assert store_alpha.chunks_path.parent == tmp_path / "alpha" / "metadata" / "file_store"
# Same-alias dependency wiring: each store binds its own alias's index/graph.
index_alpha = app.context.get_component(ComponentEnum.KEYWORD_INDEX, "default", "alpha")
graph_beta = app.context.get_component(ComponentEnum.FILE_GRAPH, "default", "beta")
assert store_alpha.keyword_index is index_alpha
assert store_beta.file_graph is graph_beta
with pytest.raises(KeyError, match="Unknown workspace alias 'nope'"):
app.context.get_component(ComponentEnum.FILE_STORE, "default", "nope")
@pytest.mark.asyncio
async def test_store_chunks_and_search_stay_in_own_workspace(tmp_path):
app = _app(tmp_path)
async with app:
for alias, text in (("alpha", "apple pie"), ("beta", "banana bread")):
store = app.context.get_component(ComponentEnum.FILE_STORE, "default", alias)
node = FileNode(path=f"daily/{alias}.md", st_mtime=0.0)
chunk = FileChunk(path=f"daily/{alias}.md", text=text, start_line=0, end_line=0)
await store.upsert([(node, [chunk])])
store_alpha = app.context.get_component(ComponentEnum.FILE_STORE, "default", "alpha")
store_beta = app.context.get_component(ComponentEnum.FILE_STORE, "default", "beta")
hits_alpha = await store_alpha.keyword_search("apple", limit=5, search_filter={})
hits_beta = await store_beta.keyword_search("apple", limit=5, search_filter={})
assert [c.path for c in hits_alpha] == ["daily/alpha.md"]
assert hits_beta == []
@pytest.mark.asyncio
async def test_store_state_persists_per_alias_across_restart(tmp_path):
app = _app(tmp_path)
async with app:
store = app.context.get_component(ComponentEnum.FILE_STORE, "default", "alpha")
await store.upsert(
[
(
FileNode(path="daily/a.md", st_mtime=0.0),
[FileChunk(path="daily/a.md", text="alpha data", start_line=0, end_line=0)],
),
],
)
rebuilt = _app(tmp_path)
async with rebuilt:
store_alpha = rebuilt.context.get_component(ComponentEnum.FILE_STORE, "default", "alpha")
store_beta = rebuilt.context.get_component(ComponentEnum.FILE_STORE, "default", "beta")
nodes_alpha = await store_alpha.get_nodes()
nodes_beta = await store_beta.get_nodes()
assert [n.path for n in nodes_alpha] == ["daily/a.md"]
assert nodes_beta == []
# ----- External backends and single-workspace parity -------------------------
@pytest.mark.asyncio
async def test_external_stateful_backend_rejected_in_multi_workspace(tmp_path):
class FakeExternalGraph(BaseComponent):
"""Stand-in for a stateful external backend that cannot isolate workspaces."""
component_type = ComponentEnum.FILE_GRAPH
supports_multi_workspace = False
with R.preserve(allow_mutation=True):
R.register(FakeExternalGraph, "fake_external")
with pytest.raises(ValueError, match="does not support multi-workspace"):
_app(
tmp_path,
components={
"file_graph": {"default": {"backend": "fake_external"}},
},
)
# Single-workspace configurations may still use the same backend.
with R.preserve(allow_mutation=True):
R.register(FakeExternalGraph, "fake_external")
app = Application(
workspace_dir=str(tmp_path / "single"),
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
components={"file_graph": {"default": {"backend": "fake_external"}}},
)
async with app:
assert app.context.components[ComponentEnum.FILE_GRAPH]["default"] is not None
@pytest.mark.asyncio
async def test_single_workspace_string_config_binds_default(tmp_path):
app = Application(
workspace_dir=str(tmp_path / "only"),
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
jobs={
"write_note": {
"backend": "base",
"parameters": {"type": "object", "properties": {}},
"steps": [{"backend": "write_step"}],
},
},
)
async with app:
response = await app.run_job("write_note", path="daily/note.md", content="plain")
assert response.success, response.answer
assert (tmp_path / "only" / "daily" / "note.md").exists()
# The explicit default alias resolves to the same workspace.
response = await app.run_job("write_note", path="daily/other.md", content="plain", workspace="default")
assert response.success
assert (tmp_path / "only" / "daily" / "other.md").exists()
# ----- Review follow-ups: binding-aware catalog reads and replace_component --
@pytest.mark.asyncio
async def test_digest_titles_follow_workspace_binding(tmp_path):
"""The digest dedup source must read the bound workspace's catalog."""
from reme.components.runtime_context import RuntimeContext
from reme.steps.evolve.proactive.topics import ProactiveTopicsStep
app = Application(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
components={"file_catalog": {"digest": {"backend": "local"}}},
)
async with app:
catalog_alpha = app.context.get_component(ComponentEnum.FILE_CATALOG, "digest", "alpha")
catalog_beta = app.context.get_component(ComponentEnum.FILE_CATALOG, "digest", "beta")
await catalog_alpha.upsert([FileNode(path="digest/2026-01-01/alpha-only.md", st_mtime=1.0)])
await catalog_beta.upsert([FileNode(path="digest/2026-01-01/beta-only.md", st_mtime=1.0)])
step = ProactiveTopicsStep(app_context=app.context, digest_compare_limit=10)
step.context = RuntimeContext(workspace=app.context.workspaces["beta"])
# Beta binding sees beta's digests only: an alpha-only title must not
# mark a beta candidate as a known duplicate.
assert await step._digest_titles() == ["beta-only"]
step.context = RuntimeContext()
with pytest.raises(RuntimeError, match="workspace binding is required"):
await step._digest_titles()
@pytest.mark.asyncio
async def test_replace_shared_component_rewires_every_alias(tmp_path):
"""Replacing a shared component rewires scoped consumers of every alias once."""
app = Application(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
components={
"tokenizer": {"default": {"backend": "regex"}},
"keyword_index": {"default": {"backend": "bm25"}},
},
)
async with app:
old = app.context.components[ComponentEnum.TOKENIZER]["default"]
close_calls = 0
original_close = old.close
async def counting_close():
nonlocal close_calls
close_calls += 1
await original_close()
old.close = counting_close
index_alpha = app.context.get_component(ComponentEnum.KEYWORD_INDEX, "default", "alpha")
index_beta = app.context.get_component(ComponentEnum.KEYWORD_INDEX, "default", "beta")
assert index_alpha.tokenizer is old
assert index_beta.tokenizer is old
replacement = await app.replace_component(ComponentEnum.TOKENIZER, "default", config={"backend": "regex"})
assert replacement is not old
assert index_alpha.tokenizer is replacement
assert index_beta.tokenizer is replacement
assert close_calls == 1
started_ids = [id(component) for component in app._started_components]
assert len(started_ids) == len(set(started_ids))

View file

@ -0,0 +1,168 @@
"""Ownership restructure: WorkspaceContext owns state; bindings stay explicit."""
# pylint: disable=protected-access,missing-function-docstring
import asyncio
import pytest
from reme.application import Application
from reme.components import ApplicationContext
from reme.components.agent_wrapper.as_agent_wrapper import AsAgentWrapper
from reme.components.agent_wrapper.cc_agent_wrapper import CcAgentWrapper
from reme.components.agent_wrapper.codex_agent_wrapper import CodexAgentWrapper
from reme.components.component_registry import R
from reme.components.job import BaseJob
from reme.components.runtime_context import RuntimeContext
from reme.enumeration import ComponentEnum
from reme.steps.base_step import BaseStep
from reme.steps.evolve.dream.integrate import DreamIntegrateStep
class _AliasReporterStep(BaseStep):
"""Answer with the bound workspace alias."""
async def execute(self):
bound = self.context.workspace
self.context.response.answer = bound.alias if bound is not None else ""
def _context(tmp_path):
registry = R.copy()
registry.register(_AliasReporterStep, "alias_reporter_step")
return ApplicationContext(
registry=registry,
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
)
def _reporter_job(context, name, **kwargs):
return BaseJob(
name=name,
app_context=context,
parameters={"type": "object", "properties": {}},
steps=[{"backend": "alias_reporter_step"}],
**kwargs,
)
@pytest.mark.asyncio
async def test_default_binding_reaches_real_agent_tool_call(tmp_path):
"""A default-bound caller keeps its binding through a real FunctionTool call."""
context = _context(tmp_path)
job = _reporter_job(context, "report", workspace="beta") # child configured for beta
job.step_specs = [job._resolve_step({"backend": "alias_reporter_step"})]
# The wrapper tool boundary injects the caller's resolved (default) binding.
tool = AsAgentWrapper._make_tool(job, None, None, workspace="alpha")
chunk = await tool._func()
text = "".join(block.text for block in chunk.content if hasattr(block, "text"))
assert text == "alpha"
# Model-supplied workspace arguments are rejected at the same boundary.
with pytest.raises(ValueError, match="cannot be provided by the model"):
await tool._func(workspace="beta")
@pytest.mark.parametrize("wrapper_cls", [AsAgentWrapper, CcAgentWrapper, CodexAgentWrapper])
def test_agent_entry_resolves_default_workspace_for_job_tools(tmp_path, wrapper_cls):
context = _context(tmp_path)
wrapper = wrapper_cls(app_context=context)
merged = wrapper._merged_kwargs({})
assert merged["workspace"] == "alpha"
assert wrapper._resolve_injected_job_kwargs(merged)["workspace"] == "alpha"
assert wrapper._merged_kwargs({"workspace": "beta"})["workspace"] == "beta"
def test_call_cwd_resolves_relative_against_bound_workspace(tmp_path):
context = _context(tmp_path)
absolute = tmp_path / "elsewhere"
wrapper = CodexAgentWrapper(app_context=context, cwd="relative/sub")
assert wrapper._call_cwd({"workspace": "beta"}) == tmp_path / "beta" / "relative" / "sub"
assert wrapper._call_cwd({}) == tmp_path / "alpha" / "relative" / "sub"
absolute_wrapper = CodexAgentWrapper(app_context=context, cwd=str(absolute))
assert absolute_wrapper._call_cwd({"workspace": "beta"}) == absolute
default_wrapper = CodexAgentWrapper(app_context=context)
assert default_wrapper._call_cwd({"workspace": "beta"}) == tmp_path / "beta"
assert default_wrapper._call_cwd({}) != tmp_path / "beta" # project path, not a workspace
# Filling in the default alias for job tools does not switch a relative cwd.
assert wrapper._call_cwd(wrapper._merged_kwargs({})) == tmp_path / "alpha" / "relative" / "sub"
def test_dream_integration_lock_is_workspace_scoped(tmp_path):
context = _context(tmp_path)
step = DreamIntegrateStep(app_context=context)
step.context = RuntimeContext(workspace=context.workspaces["alpha"])
lock_alpha = step._integration_lock()
step.context = RuntimeContext(workspace=context.workspaces["beta"])
lock_beta = step._integration_lock()
step.context = RuntimeContext(workspace=context.workspaces["alpha"])
assert step._integration_lock() is lock_alpha
assert lock_alpha is not lock_beta
# ----- Real watch loops -------------------------------------------------------
_WATCH_COMPONENTS = {
"tokenizer": {"default": {"backend": "regex"}},
"file_store": {"default": {"backend": "local", "embedding_store": "", "tag_index": ""}},
"keyword_index": {"default": {"backend": "bm25"}},
"file_graph": {"default": {"backend": "local"}},
"file_chunker": {"markdown": {"backend": "markdown", "supported_extensions": ["md"]}},
}
def _watch_job(alias):
return {
"backend": "background",
"workspace": alias,
"enable_serve": False,
"watch_dirs": ["daily_dir"],
"watch_suffixes": ["md"],
"parameters": {"type": "object", "properties": {}},
"steps": [
{
"backend": "watch_changes_step",
"debounce": 200,
"step": 100,
"poll_delay_ms": 200,
"dispatch_steps": [{"backend": "update_index_step", "persist": False}],
},
],
}
@pytest.mark.asyncio
async def test_real_watch_loops_index_own_workspace_and_clean_up(tmp_path):
app = Application(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
components=_WATCH_COMPONENTS,
jobs={"watch_alpha": _watch_job("alpha"), "watch_beta": _watch_job("beta")},
)
note = tmp_path / "beta" / "daily" / "watch-me.md"
note.parent.mkdir(parents=True, exist_ok=True)
async with app:
store_beta = app.context.get_component(ComponentEnum.FILE_STORE, "default", "beta")
store_alpha = app.context.get_component(ComponentEnum.FILE_STORE, "default", "alpha")
for i in range(30): # up to ~15s for watcher startup + debounce + dispatch
note.write_text(f"# beta watch\nreal watch event content v{i}\n", encoding="utf-8")
await asyncio.sleep(0.5)
if await store_beta.get_nodes():
break
nodes_beta = await store_beta.get_nodes()
nodes_alpha = await store_alpha.get_nodes()
assert [n.path for n in nodes_beta] == ["daily/watch-me.md"]
assert nodes_alpha == []
# Background loops stopped with the application; nothing keeps running.
for job in app.context.jobs.values():
assert job._task is None or job._task.done()

View file

@ -0,0 +1,800 @@
"""Phase 3: nested propagation, session namespacing, and background binding."""
# pylint: disable=protected-access,missing-function-docstring
import asyncio
import json
import os
from dataclasses import dataclass
from types import SimpleNamespace
import pytest
from reme.application import Application
from reme.components import ApplicationContext
from reme.components.agent_wrapper.base_agent_wrapper import BaseAgentWrapper
from reme.components.agent_wrapper.codex_agent_wrapper import CodexAgentWrapper
from reme.components.component_registry import R
from reme.components.job import BackgroundJob, BaseJob, CronJob
from reme.components.runtime_context import RuntimeContext
from reme.enumeration import ComponentEnum
from reme.steps.base_step import BaseStep
from reme.steps.evolve.base_auto_resource import _compute_agent_session_id
from reme.steps.index.draft import AddDraftStep
class _AliasReporterStep(BaseStep):
"""Answer with the bound workspace alias."""
async def execute(self):
bound = self.context.workspace
self.context.response.answer = bound.alias if bound is not None else ""
class _NestedCallerStep(BaseStep):
"""Call the report job as a nested job and relay its answer."""
async def execute(self):
response = await self.run_job("report")
self.context.response.answer = f"parent->{response.answer}"
def _registry():
registry = R.copy()
registry.register(_AliasReporterStep, "alias_reporter_step")
registry.register(_NestedCallerStep, "nested_caller_step")
return registry
def _context(tmp_path):
return ApplicationContext(
registry=_registry(),
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
)
def _job(context, name, backend="alias_reporter_step", job_cls=BaseJob, **kwargs):
return job_cls(
name=name,
app_context=context,
parameters={"type": "object", "properties": {}},
steps=[{"backend": backend}],
**kwargs,
)
# ----- Nested propagation ----------------------------------------------------
@pytest.mark.asyncio
async def test_nested_run_job_inherits_workspace_binding(tmp_path):
context = _context(tmp_path)
parent = _job(context, "parent", backend="nested_caller_step")
child = _job(context, "report")
context.jobs = {"parent": parent, "report": child}
await parent.start()
await child.start()
response = await parent(workspace="beta")
assert response.answer == "parent->beta"
# Default workspace stays the implicit binding for unscoped calls.
response = await parent()
assert response.answer == "parent->alpha"
# Explicit nested selection wins over inheritance.
caller = _NestedCallerStep(app_context=context)
caller.context = RuntimeContext(workspace=context.workspaces["alpha"])
response = await caller.run_job("report", workspace="beta")
assert response.answer == "beta"
def test_workspace_forwarded_through_injected_channel_only():
# The calling step's binding joins the server-owned injected kwargs...
injected = BaseAgentWrapper._resolve_injected_job_kwargs(
{"injected_job_kwargs": {"_allowed_paths": ["daily/x.md"]}, "workspace": "beta"},
)
assert injected == {"_allowed_paths": ["daily/x.md"], "workspace": "beta"}
# ...and model-supplied tool arguments can never override it.
with pytest.raises(ValueError, match="cannot be provided by the model"):
BaseAgentWrapper._merge_injected_job_kwargs({"workspace": "alpha"}, injected)
# ----- Session/tool-context namespacing --------------------------------------
def test_agent_session_uuid_namespaced_per_workspace():
path = "resource/2026-01-01/report.pdf"
legacy = _compute_agent_session_id(path)
alpha = _compute_agent_session_id(path, workspace_alias="alpha")
beta = _compute_agent_session_id(path, workspace_alias="beta")
# Default/empty alias keeps the historical UUID for backward compatibility.
assert _compute_agent_session_id(path, workspace_alias="") == legacy
assert len({legacy, alpha, beta}) == 3
@pytest.mark.asyncio
async def test_tool_context_state_namespaced_per_workspace(tmp_path):
context = _context(tmp_path)
step = AddDraftStep(app_context=context)
for alias in ("alpha", "beta"):
step.context = RuntimeContext(
workspace=context.workspaces[alias],
tool_context_id="ctx-1",
text=f"draft-{alias}",
)
await step.execute()
# State is owned by each WorkspaceContext; identical ids stay independent.
assert context.workspaces["alpha"].tool_contexts["ctx-1"]["drafts"] == ["draft-alpha"]
assert context.workspaces["beta"].tool_contexts["ctx-1"]["drafts"] == ["draft-beta"]
assert "tool_contexts" not in context.metadata
@pytest.mark.asyncio
async def test_codex_thread_tool_contexts_namespaced_per_workspace(tmp_path):
context = ApplicationContext(workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")})
wrapper = CodexAgentWrapper(app_context=context)
class FakeCodex:
"""Minimal Codex double echoing the requested thread id."""
def __init__(self):
self.next_id = "thread-1"
async def thread_start(self, **_kwargs):
return SimpleNamespace(id=self.next_id)
async def thread_resume(self, thread_id, **_kwargs):
return SimpleNamespace(id=thread_id)
codex = FakeCodex()
await wrapper._open_thread(codex, {"tool_context_id": "ctx-a"})
# A different thread under another workspace gets its own guard entry.
codex.next_id = "thread-2"
await wrapper._open_thread(codex, {"workspace": "beta", "tool_context_id": "ctx-b"})
# Same workspace resume with changed constraints still raises.
with pytest.raises(ValueError, match="cannot change"):
await wrapper._open_thread(codex, {"session_id": "thread-1", "tool_context_id": "ctx-b"})
# A thread owned by one workspace cannot be resumed from another.
with pytest.raises(ValueError, match="belongs to workspace 'alpha'"):
await wrapper._open_thread(codex, {"session_id": "thread-1", "workspace": "beta"})
alpha_state = context.workspaces["alpha"].agent_state(wrapper.name)
beta_state = context.workspaces["beta"].agent_state(wrapper.name)
assert set(alpha_state.thread_tool_contexts) == {"thread-1"}
assert set(beta_state.thread_tool_contexts) == {"thread-2"}
assert not wrapper._thread_tool_contexts
assert context.workspaces["alpha"].codex_thread_state.owned_thread_ids == {"thread-1"}
assert context.workspaces["beta"].codex_thread_state.owned_thread_ids == {"thread-2"}
@pytest.mark.asyncio
async def test_codex_thread_ownership_persists_across_restart(tmp_path):
context = ApplicationContext(workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")})
class FakeCodex:
"""Minimal Codex double echoing the requested thread id."""
async def thread_start(self, **_kwargs):
return SimpleNamespace(id="thread-1")
async def thread_resume(self, thread_id, **_kwargs):
return SimpleNamespace(id=thread_id)
async def thread_fork(self, thread_id, **_kwargs):
return SimpleNamespace(id=f"{thread_id}-fork")
codex = FakeCodex()
wrapper = CodexAgentWrapper(app_context=context)
await wrapper._open_thread(codex, {"workspace": "alpha"})
ownership = tmp_path / "alpha" / "mem_session" / "codex" / "owned_threads.json"
assert ownership.is_file()
# A fresh wrapper (restart) reloads the ownership records and still rejects.
restarted = CodexAgentWrapper(app_context=context)
with pytest.raises(ValueError, match="belongs to workspace 'alpha'"):
await restarted._open_thread(codex, {"session_id": "thread-1", "workspace": "beta"})
with pytest.raises(ValueError, match="belongs to workspace 'alpha'"):
await restarted._open_thread(codex, {"session_id": "thread-1", "workspace": "beta", "fork_session": True})
with pytest.raises(ValueError, match="belongs to workspace 'alpha'"):
await restarted.compact_session("thread-1", workspace="beta")
# The owning workspace keeps working normally.
thread = await restarted._open_thread(codex, {"session_id": "thread-1", "workspace": "alpha"})
assert thread.id == "thread-1"
# Threads recorded nowhere pass through and let the SDK decide.
await restarted._open_thread(codex, {"session_id": "thread-elsewhere", "workspace": "beta"})
@pytest.mark.asyncio
async def test_codex_thread_ownership_is_shared_and_atomic_across_wrappers(tmp_path):
context = ApplicationContext(workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")})
wrapper_a = CodexAgentWrapper(name="codex-a", app_context=context)
wrapper_b = CodexAgentWrapper(name="codex-b", app_context=context)
assert await wrapper_b._owned_thread_ids("alpha") == set()
await wrapper_a._record_thread_ownership("alpha", "alpha-thread")
with pytest.raises(ValueError, match="belongs to workspace 'alpha'"):
await wrapper_b._check_thread_ownership("alpha-thread", "beta")
await wrapper_b._record_thread_ownership("alpha", "second-alpha-thread")
ownership = tmp_path / "alpha" / "mem_session" / "codex" / "owned_threads.json"
assert set(json.loads(ownership.read_text(encoding="utf-8"))["thread_ids"]) == {
"alpha-thread",
"second-alpha-thread",
}
assert not list(ownership.parent.glob("*.tmp"))
restarted_context = ApplicationContext(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
)
restarted = CodexAgentWrapper(name="codex-restarted", app_context=restarted_context)
with pytest.raises(ValueError, match="belongs to workspace 'alpha'"):
await restarted._check_thread_ownership("alpha-thread", "beta")
@pytest.mark.asyncio
async def test_codex_resume_and_compact_claim_external_threads(tmp_path, monkeypatch):
context = ApplicationContext(workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")})
wrapper_a = CodexAgentWrapper(name="codex-a", app_context=context, release_threads=False)
wrapper_b = CodexAgentWrapper(name="codex-b", app_context=context, release_threads=False)
class FakeCodex:
"""Resume an existing external thread."""
async def thread_resume(self, thread_id, **_kwargs):
return SimpleNamespace(id=thread_id)
await wrapper_a._open_thread(FakeCodex(), {"session_id": "external-thread", "workspace": "alpha"})
assert context.workspaces["alpha"].codex_thread_state.owned_thread_ids == {"external-thread"}
with pytest.raises(ValueError, match="belongs to workspace 'alpha'"):
await wrapper_b._open_thread(FakeCodex(), {"session_id": "external-thread", "workspace": "beta"})
class CompactThread:
"""Minimal native compaction double."""
id = "compact-thread"
async def compact(self):
return None
compact_codex = SimpleNamespace(thread_resume=lambda _thread_id: None)
async def resume_compact(_thread_id):
return CompactThread()
compact_codex.thread_resume = resume_compact
async def get_compact_codex():
return compact_codex
monkeypatch.setattr(wrapper_a, "_get_codex", get_compact_codex)
await wrapper_a.compact_session("compact-thread", workspace="alpha")
assert "compact-thread" in context.workspaces["alpha"].codex_thread_state.owned_thread_ids
restarted_context = ApplicationContext(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
)
restarted = CodexAgentWrapper(name="codex-restarted", app_context=restarted_context)
with pytest.raises(ValueError, match="belongs to workspace 'alpha'"):
await restarted._open_thread(FakeCodex(), {"session_id": "external-thread", "workspace": "beta"})
@pytest.mark.asyncio
async def test_codex_concurrent_first_resume_has_one_owner(tmp_path):
context = ApplicationContext(workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")})
wrapper_a = CodexAgentWrapper(name="codex-a", app_context=context)
wrapper_b = CodexAgentWrapper(name="codex-b", app_context=context)
class FakeCodex:
"""Yield between resume and ownership registration to force interleaving."""
async def thread_resume(self, thread_id, **_kwargs):
await asyncio.sleep(0)
return SimpleNamespace(id=thread_id)
results = await asyncio.gather(
wrapper_a._open_thread(FakeCodex(), {"session_id": "shared-external", "workspace": "alpha"}),
wrapper_b._open_thread(FakeCodex(), {"session_id": "shared-external", "workspace": "beta"}),
return_exceptions=True,
)
assert len([result for result in results if not isinstance(result, Exception)]) == 1
assert len([result for result in results if isinstance(result, ValueError)]) == 1
owners = [
alias
for alias, workspace in context.workspaces.items()
if "shared-external" in (workspace.codex_thread_state.owned_thread_ids or set())
]
assert len(owners) == 1
other = "beta" if owners[0] == "alpha" else "alpha"
restarted_context = ApplicationContext(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
)
restarted = CodexAgentWrapper(name="codex-restarted", app_context=restarted_context)
with pytest.raises(ValueError, match="belongs to workspace"):
await restarted._open_thread(FakeCodex(), {"session_id": "shared-external", "workspace": other})
# ----- Background/cron binding -----------------------------------------------
@pytest.mark.asyncio
async def test_background_job_binds_configured_workspace(tmp_path):
context = _context(tmp_path)
job = _job(context, "bg", job_cls=BackgroundJob, workspace="beta")
job.step_specs = [job._resolve_step({"backend": "alias_reporter_step"})]
assert (await job()).answer == "beta"
default_job = _job(context, "bg-default", job_cls=BackgroundJob)
default_job.step_specs = [default_job._resolve_step({"backend": "alias_reporter_step"})]
assert (await default_job()).answer == "alpha"
with pytest.raises(ValueError, match="Unknown workspace 'nope'"):
await _job(context, "bg-bad", job_cls=BackgroundJob, workspace="nope")()
@pytest.mark.asyncio
async def test_cron_job_binds_configured_workspace(tmp_path):
context = _context(tmp_path)
job = _job(context, "cron", job_cls=CronJob, cron="0 0 * * *", workspace="beta")
job.step_specs = [job._resolve_step({"backend": "alias_reporter_step"})]
assert (await job._execute_steps()).answer == "beta"
default_job = _job(context, "cron-default", job_cls=CronJob, cron="0 0 * * *")
default_job.step_specs = [default_job._resolve_step({"backend": "alias_reporter_step"})]
assert (await default_job._execute_steps()).answer == "alpha"
@pytest.mark.asyncio
async def test_per_alias_watch_variant_jobs_write_own_workspace(tmp_path):
"""Two watch-style variants with explicit aliases act on their own workspace."""
app = Application(
workspace_dir={"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")},
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={"backend": "cli"},
components={
"tokenizer": {"default": {"backend": "regex"}},
"file_store": {"default": {"backend": "local", "embedding_store": "", "tag_index": ""}},
"keyword_index": {"default": {"backend": "bm25"}},
"file_graph": {"default": {"backend": "local"}},
},
jobs={
"watch_alpha": {
"backend": "background",
"workspace": "alpha",
"enable_serve": False,
"parameters": {"type": "object", "properties": {}},
"path": "daily/marker.md",
"content": "alpha-watch",
"steps": [{"backend": "write_step"}],
},
"watch_beta": {
"backend": "background",
"workspace": "beta",
"enable_serve": False,
"parameters": {"type": "object", "properties": {}},
"path": "daily/marker.md",
"content": "beta-watch",
"steps": [{"backend": "write_step"}],
},
},
)
async with app:
await asyncio.sleep(0.5)
alpha = (tmp_path / "alpha" / "daily" / "marker.md").read_text(encoding="utf-8")
beta = (tmp_path / "beta" / "daily" / "marker.md").read_text(encoding="utf-8")
assert "alpha-watch" in alpha and "beta-watch" not in alpha
assert "beta-watch" in beta and "alpha-watch" not in beta
# ----- MCP boundary and Codex bridge -----------------------------------------
@pytest.mark.asyncio
async def test_mcp_tool_boundary_forwards_injected_workspace(tmp_path):
from reme.components.service.mcp_tools import add_mcp_job
context = _context(tmp_path)
job = _job(context, "report")
job.step_specs = [job._resolve_step({"backend": "alias_reporter_step"})]
tools = []
server = SimpleNamespace(add_tool=tools.append)
assert add_mcp_job(server, job, injected_job_kwargs={"workspace": "beta"}, tool_error_on_failure=False)
# The server-injected binding reaches the job boundary and wins over callers.
assert await tools[0].fn() == "beta"
with pytest.raises(Exception, match="cannot be provided by the caller"):
await tools[0].fn(workspace="alpha")
@pytest.mark.asyncio
async def test_codex_bridge_uses_resident_mcp_endpoint(tmp_path):
from reme.components.service import HttpService
context = _context(tmp_path)
context.service = HttpService(app_context=context, tool_error_on_failure=True)
context.jobs = {"search_job": _job(context, "search_job")}
wrapper = CodexAgentWrapper(app_context=context)
config = wrapper._mcp_server_config({"job_tools": ["search_job"], "workspace": "beta"})
assert config["args"][config["args"].index("--endpoint") + 1] == "http://127.0.0.1:2333/mcp"
raw = config["args"][config["args"].index("--injected-job-kwargs") + 1]
assert json.loads(raw) == {"workspace": "beta"}
assert "--config" not in config["args"] and "--workspace" not in config["args"]
assert "--token" not in config["args"]
default_config = wrapper._mcp_server_config({"job_tools": ["search_job"]})
assert "--injected-job-kwargs" not in default_config["args"]
await wrapper.close()
# ----- Review follow-ups: explicit bindings, session isolation, key collisions
@pytest.mark.asyncio
async def test_nested_call_default_binding_beats_child_config(tmp_path):
"""An explicit default binding propagates; the child's configured alias must not win."""
context = _context(tmp_path)
parent = _job(context, "parent", backend="nested_caller_step")
child = _job(context, "report", workspace="beta") # job-level configured binding
context.jobs = {"parent": parent, "report": child}
await parent.start()
await child.start()
# Top-level call to the configured child still honors its configuration.
assert (await child()).answer == "beta"
# Nested under an alpha-bound parent, the parent binding wins.
response = await parent(workspace="alpha")
assert response.answer == "parent->alpha"
response = await parent(workspace="beta")
assert response.answer == "parent->beta"
@pytest.mark.asyncio
async def test_tool_context_keys_cannot_collide_across_workspaces(tmp_path):
context = _context(tmp_path)
step = AddDraftStep(app_context=context)
# Default-workspace id that spells like another alias's namespaced key.
step.context = RuntimeContext(workspace=context.workspaces["alpha"], tool_context_id="beta/ctx-1", text="d-alpha")
await step.execute()
step.context = RuntimeContext(workspace=context.workspaces["beta"], tool_context_id="ctx-1", text="d-beta")
await step.execute()
assert context.workspaces["alpha"].tool_contexts["beta/ctx-1"]["drafts"] == ["d-alpha"]
assert context.workspaces["beta"].tool_contexts["ctx-1"]["drafts"] == ["d-beta"]
@pytest.mark.asyncio
async def test_agentscope_sessions_isolated_per_workspace(tmp_path):
from uuid import uuid4
from agentscope.permission import PermissionMode
from reme.components.agent_wrapper.as_agent_wrapper import AsAgentWrapper
context = _context(tmp_path)
wrapper = AsAgentWrapper(app_context=context, as_llm="")
wrapper.as_llm = SimpleNamespace(model=object())
session_id = str(uuid4())
state = await wrapper._load_state({"session_id": session_id, "workspace": "beta"}, PermissionMode.DEFAULT)
state.summary = "beta summary"
await wrapper._dump_state(state, {"workspace": "beta"})
session_file = tmp_path / "beta" / "mem_session" / "agentscope" / f"{session_id}.jsonl"
assert session_file.is_file()
assert not (tmp_path / "alpha" / "mem_session" / "agentscope" / f"{session_id}.jsonl").exists()
# Same id under another workspace must not read beta's state.
with pytest.raises(FileNotFoundError):
await wrapper._load_state({"resume": session_id, "workspace": "alpha"}, PermissionMode.DEFAULT)
resumed = await wrapper._load_state({"resume": session_id, "workspace": "beta"}, PermissionMode.DEFAULT)
assert resumed.summary == "beta summary"
# Default workspace is untouched by the beta session.
with pytest.raises(FileNotFoundError):
await wrapper._load_state({"resume": session_id}, PermissionMode.DEFAULT)
def test_cc_options_follow_bound_workspace(tmp_path):
from reme.components.agent_wrapper.cc_agent_wrapper import CcAgentWrapper
context = _context(tmp_path)
wrapper = CcAgentWrapper(app_context=context)
opts = wrapper._build_options("hi", workspace="beta")
assert opts.env["CLAUDE_CONFIG_DIR"].startswith(str(tmp_path / "beta"))
assert opts.cwd == tmp_path / "beta"
default_opts = wrapper._build_options("hi")
assert default_opts.env["CLAUDE_CONFIG_DIR"].startswith(str(tmp_path / "alpha"))
@pytest.mark.asyncio
async def test_codex_thread_cwd_follows_bound_workspace(tmp_path):
context = _context(tmp_path)
wrapper = CodexAgentWrapper(app_context=context)
observed = {}
class FakeCodex:
"""Capture thread_start parameters without an app-server."""
next_id = 1
async def thread_start(self, **kwargs):
observed.update(kwargs)
self.next_id += 1
return SimpleNamespace(id=f"thread-{self.next_id - 1}")
codex = FakeCodex()
await wrapper._open_thread(codex, {"workspace": "beta"})
assert observed["cwd"] == str(tmp_path / "beta")
observed.clear()
await wrapper._open_thread(codex, {})
assert observed["cwd"] != str(tmp_path / "beta")
# ----- Codex bridge: real child-process execution ----------------------------
def _codex_test_app(tmp_path, workspace_dir=None):
if workspace_dir is None:
workspace_dir = {"alpha": str(tmp_path / "alpha"), "beta": str(tmp_path / "beta")}
return Application(
workspace_dir=workspace_dir,
enable_logo=False,
log_to_console=False,
log_to_file=False,
service={
"backend": "http",
"host": "127.0.0.1",
"port": 0,
"web_enabled": False,
"tool_error_on_failure": True,
},
components={
"tokenizer": {"default": {"backend": "regex"}},
"file_store": {"default": {"backend": "local", "embedding_store": "", "tag_index": ""}},
"keyword_index": {"default": {"backend": "bm25"}},
"file_graph": {"default": {"backend": "local"}},
},
jobs={
"write_note": {
"backend": "base",
"description": "write a note",
"parameters": {
"type": "object",
"properties": {"path": {"type": "string"}, "content": {"type": "string"}},
"required": ["path", "content"],
},
"steps": [{"backend": "write_step"}],
},
"read_note": {
"backend": "base",
"description": "read a note",
"parameters": {
"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"],
},
"steps": [{"backend": "read_step"}],
},
},
)
async def _start_resident_http(app):
import uvicorn
service = app.context.service
service.build_service(app)
service.add_jobs(app)
config = uvicorn.Config(service.service, host=service.host, port=0, log_level="error", lifespan="on")
server = uvicorn.Server(config)
task = asyncio.create_task(server.serve())
for _ in range(100):
if server.started:
break
await asyncio.sleep(0.05)
assert server.started
service.port = server.servers[0].sockets[0].getsockname()[1]
return server, task
async def _stop_resident_http(server, task):
server.should_exit = True
await asyncio.wait_for(task, timeout=30)
@pytest.mark.asyncio
async def test_codex_bridge_reads_resident_parent_state(tmp_path):
"""The stdio bridge reuses the resident MCP service and its live workspace state."""
from fastmcp import Client
from fastmcp.client import StdioTransport
app = _codex_test_app(tmp_path)
wrapper = CodexAgentWrapper(app_context=app.context)
async with app:
server, task = await _start_resident_http(app)
try:
await app.run_job("write_note", path="daily/resident.md", content="parent-resident-beta", workspace="beta")
from reme.components.client import MCPClient
resident_client = MCPClient(transport="streamable-http", host="127.0.0.1", port=app.context.service.port)
await resident_client.start()
assert {entry["name"] for entry in await resident_client.list_actions()} == {"read_note", "write_note"}
await resident_client.close()
server_config = wrapper._mcp_server_config(
{"job_tools": ["read_note", "write_note"], "workspace": "beta"},
)
transport = StdioTransport(
command=server_config["command"],
args=server_config["args"],
cwd=server_config["cwd"],
keep_alive=False,
env=dict(os.environ),
)
async with Client(transport, timeout=None, init_timeout=30) as client:
tools = await client.list_tools()
assert {tool.name for tool in tools} == {"read_note", "write_note"}
read = await asyncio.wait_for(client.call_tool("read_note", {"path": "daily/resident.md"}), timeout=30)
with pytest.raises(Exception):
await client.call_tool("read_note", {"path": "daily/resident.md", "workspace": "alpha"})
await asyncio.wait_for(
client.call_tool("write_note", {"path": "daily/bridge.md", "content": "via-bridge-beta"}),
timeout=30,
)
failure = await client.call_tool(
"write_note",
{"path": "../escape.md", "content": "must-not-write"},
raise_on_error=False,
)
assert failure.is_error
assert not (tmp_path / "escape.md").exists()
assert "parent-resident-beta" in str(read.content[0].text)
assert "via-bridge-beta" in (tmp_path / "beta" / "daily" / "bridge.md").read_text(encoding="utf-8")
assert not (tmp_path / "alpha" / "daily" / "bridge.md").exists()
store_alpha = app.context.get_component(ComponentEnum.FILE_STORE, "default", "alpha")
assert all("bridge" not in node.path for node in await store_alpha.get_nodes())
invalid_config = wrapper._mcp_server_config({"job_tools": ["read_note"], "workspace": "nope"})
invalid_transport = StdioTransport(
command=invalid_config["command"],
args=invalid_config["args"],
cwd=invalid_config["cwd"],
keep_alive=False,
env=dict(os.environ),
)
async with Client(invalid_transport, timeout=None, init_timeout=30) as client:
with pytest.raises(Exception, match="Unknown workspace 'nope'"):
await asyncio.wait_for(client.call_tool("read_note", {"path": "daily/resident.md"}), timeout=30)
finally:
await _stop_resident_http(server, task)
await wrapper.close()
@pytest.mark.asyncio
async def test_codex_reply_defaults_workspace_for_real_job_tools(tmp_path, monkeypatch):
from fastmcp import Client
from fastmcp.client import StdioTransport
@dataclass
class TurnResult:
"""Serializable fake turn result."""
final_response: str
usage: object = None
class FakeThread:
"""Execute the configured job tool through the real STDIO bridge."""
def __init__(self, thread_id, config):
self.id = thread_id
self.config = config
async def run(self, _inputs, **_kwargs):
server = next(iter(self.config["mcp_servers"].values()))
transport = StdioTransport(
command=server["command"],
args=server["args"],
cwd=server["cwd"],
keep_alive=False,
env=dict(os.environ),
)
arguments = {"path": f"daily/reply-{self.id}.md", "content": self.id}
if self.id == "thread-override":
arguments["workspace"] = "beta"
async with Client(transport, timeout=None, init_timeout=30) as client:
await client.call_tool("write_note", arguments)
return TurnResult(final_response="done")
class FakeCodex:
"""Create threads with deterministic IDs."""
def __init__(self):
self.next_id = 0
async def thread_start(self, **kwargs):
self.next_id += 1
thread_id = "thread-override" if self.next_id == 3 else f"thread-{self.next_id}"
return FakeThread(thread_id, kwargs["config"])
app = _codex_test_app(tmp_path)
app.context.jobs["write_note"].kwargs["workspace"] = "beta"
wrapper = CodexAgentWrapper(app_context=app.context, release_threads=False)
codex = FakeCodex()
async def get_codex():
return codex
monkeypatch.setattr(wrapper, "_get_codex", get_codex)
async with app:
server, task = await _start_resident_http(app)
try:
await wrapper.reply("default", job_tools=["write_note"])
assert (tmp_path / "alpha" / "daily" / "reply-thread-1.md").is_file()
assert not (tmp_path / "beta" / "daily" / "reply-thread-1.md").exists()
await wrapper.reply("explicit", job_tools=["write_note"], workspace="alpha")
assert (tmp_path / "alpha" / "daily" / "reply-thread-2.md").is_file()
assert not (tmp_path / "beta" / "daily" / "reply-thread-2.md").exists()
with pytest.raises(Exception):
await wrapper.reply("override", job_tools=["write_note"])
assert not (tmp_path / "beta" / "daily" / "reply-thread-override.md").exists()
finally:
await _stop_resident_http(server, task)
await wrapper.close()
@pytest.mark.asyncio
@pytest.mark.parametrize("named_alias", [False, True])
async def test_codex_legacy_bridge_preserves_single_workspace_binding(tmp_path, named_alias):
from fastmcp import Client
from fastmcp.client import StdioTransport
alias = "alice" if named_alias else "default"
root = tmp_path / alias
workspace_dir = {alias: str(root)} if named_alias else str(root)
app = _codex_test_app(tmp_path, workspace_dir=workspace_dir)
wrapper = CodexAgentWrapper(app_context=app.context)
server_config = wrapper._mcp_server_config({"job_tools": ["write_note"], "workspace": alias})
transport = StdioTransport(
command=server_config["command"],
args=server_config["args"],
cwd=server_config["cwd"],
keep_alive=False,
env=dict(os.environ),
)
async with Client(transport, timeout=None, init_timeout=30) as client:
await asyncio.wait_for(
client.call_tool("write_note", {"path": "daily/legacy.md", "content": alias}),
timeout=30,
)
assert alias in (root / "daily" / "legacy.md").read_text(encoding="utf-8")
await wrapper.close()
@pytest.mark.asyncio
async def test_codex_multi_workspace_rejects_unsupported_job_transport(tmp_path):
from reme.components.service import CliService, HttpService
context = _context(tmp_path)
context.service = CliService(app_context=context)
context.jobs = {"search_job": _job(context, "search_job")}
wrapper = CodexAgentWrapper(app_context=context)
with pytest.raises(RuntimeError, match="resident HTTP MCP service"):
wrapper._mcp_server_config({"job_tools": ["search_job"], "workspace": "beta"})
with pytest.raises(ValueError, match="only supported for single-workspace"):
wrapper._mcp_server_config(
{"job_tools": ["search_job"], "workspace": "beta", "mcp_config": "custom.yaml"},
)
context.service = HttpService(app_context=context, tool_error_on_failure=False)
with pytest.raises(ValueError, match="tool_error_on_failure=true"):
wrapper._mcp_server_config({"job_tools": ["search_job"], "workspace": "beta"})
await wrapper.close()