* refactor(codex)!: share app-server and reuse application job tools
* test(codex): use FastMCP input_schema without compatibility aliases
* feat(codex): configure native model catalog and preserve streamed metadata
* fix(codex): default to unattended tools and preserve retry streams
* fix(codex): use an explicit Responses provider for custom endpoints
* fix(codex): store runtime state in workspace .codex
* feat(config): add minimal Studio workspace config with Codex chat agent
- Introduce proactive.yaml defining a workspace with HTTP backend and web enabled
- Add background job for Markdown index update watching daily and digest directories
- Define multiple base and stream backend jobs for chat, config, version, status, health, graph, reindex, and file operations
- Configure Codex app server with API key auth and detailed model catalog including qwen3.8-max
- Set up default agent wrapper using Codex backend with high effort and read-only sandbox
- Establish default tokenizer (regex), keyword index (bm25), file graph and chunker for markdown files
- Configure file storage backend with local implementation and indexing components
* fix(codex): settle cancelled dispatch and isolate TLS callbacks
* test(codex): check cancellation barriers across Python versions
Split changed dream inputs into configurable file-size batches and bisect failed batches. Scope source references to each batch and merge same-name units across batches.
Keep one global unit budget and checkpoint only successfully processed, non-failed paths. Preserve retry eligibility for deferred inputs, including day indexes, and initialize new runs separately from legacy state.
Validation: 192 focused tests passed; isolated QwenPaw tests with a real model covered batching, restart deduplication, deferred continuation, and injected-failure recovery.
Closes#659
_resolve_service_address overwrote both fields whenever either host or
port was missing, silently discarding the explicitly supplied one in
favor of the discovered or default pair. Resolve the fallback pair
first and fill only the missing field, so explicit values always win
while discovery and defaults still cover omitted fields.
Add regression tests covering explicit-only host/port, service
discovery precedence, invalid discovery payloads, and the unaffected
MCP stdio transport.
Windows runs hit three platform assumptions in the agent wrappers:
- `CodexAgentWrapper._effective_config_snapshot()` called `os.fchmod`,
which only exists on POSIX. Building the MCP config snapshot therefore
raised `AttributeError` on Windows and the Codex wrapper could not
start. The call is now guarded by a capability check
(`hasattr(os, "fchmod")`), which also keeps the guard testable on
POSIX hosts.
- `CcFileSessionStore.list_subkeys()` rendered subpaths with `str()`,
which yields backslashes on Windows. The session-store contract - and
the SDK conformance check shipped with `claude_agent_sdk` - expects
POSIX subpaths such as `subagents/agent-2`. Render them with
`as_posix()`.
- `AutoMemoryStep` wrote session JSONL with the platform default newline
translation, so Windows stored CRLF even though the writer appends
`"\n"`. Pass `newline=""` on the append and rewrite paths so one
message stays one LF-terminated line on every platform.
Add a regression test that pins the snapshot path while `os.fchmod` is
unavailable, so the guard is covered on POSIX CI too.
* fix(embedding-store): keep shrinking an over-limit request until it fits
`LocalEmbeddingStore._call_with_retry()` shortened a rejected input once, to
half of the configured `max_input_length`. Two shapes of input-length
rejection still left the chunk unembedded, and an unembedded chunk keeps
`needs_reindex` set, so a strict reindex never completes:
- a shortened request that is still above the provider's real per-item limit
was never shortened again, even with retry budget left;
- an input already below half the configured budget produced an identical
"shorter" request, so nothing was re-sent at all.
The retry now derives a strictly smaller request from the text the provider
just rejected and keeps shrinking within an explicit round budget, still
bounded by the remaining retry budget. It stops on success, on an exhausted
budget, or when no smaller non-empty request exists. Source text, chunk
contents, configuration, vector-space checks, cache behaviour and the existing
error routing are untouched; the shrink step no longer sleeps, because an
input-length rejection is deterministic rather than rate limiting.
Refs: #683
* test(embedding-store): cover a repeated rebuild of an over-limit chunk
`reindex` is what #683 was about, and while the request-level recovery had
a repeatability test, the rebuild path did not. Add the rebuild that runs
twice over the same over-limit chunk, with the embedding cache both off and
on, so a regression on the second pass fails here.
The shrinking helper keeps the reasoning behind halving the rejected text,
because that is what makes each round a real reduction.
Chunk metadata can hold YAML lists -- frontmatter fields such as `tags`
are lists -- and the metadata filter hashed the stored value. A filter
such as `search_filter={"tags": ["python"]}` raised
`TypeError: unhashable type: 'list'`, so the whole search failed instead
of matching, and a scalar filter such as `{"tags": "python"}` never
matched a list that contained the value.
Compare metadata values by intersection: the stored value and the
accepted values are matched element-wise, so a list-valued field is
filterable and a list filter no longer raises. Scalar values keep their
previous equality semantics, and `LocalFileStore._value_matches` stays
the class-level hook so subclasses can still customize the comparison.
Add focused tests for the prepared matcher, the compatibility matcher and
the search Step covering list-valued metadata.
* fix(search): keep zero float16 cosine scores finite
Updated the handling of zero-norm rows to use a representable denominator of 1 instead of 1e-10.
* test(search): cover zero cosine vectors and batch boundaries
Added tests for batch cosine similarity with zero vectors and empty batches.
* test(search): keep zero embedding rankings finite
Add tests for zero embeddings in vector search and legacy JSON loading.
* fix(frontmatter-read): put the frontmatter in the answer, not just a key count
`frontmatter_read` answered `Read frontmatter from <path> (N key(s))` and left
the frontmatter itself in `metadata["frontmatter"]`. MCP returns only
`response.answer`, so the LLM that asked for a file's frontmatter learned how
many keys exist and nothing else — the same shape of gap #379 describes for
`list`, where the caller cannot act on the result.
Render each key on its own line in the answer. Values are flattened to one
line each so a multi-line description cannot break the line-per-key layout:
lists join with commas and nested mappings render as compact JSON. An empty
frontmatter says `(no keys)` rather than trailing off after the colon.
The answer also names the probed path instead of the raw input, so a
suffix-less request reports the `.md` file that was actually read and the
caller can follow up with `read` without consulting metadata. Structured
`metadata["frontmatter"]` is unchanged, so programmatic consumers see no
difference.
Towards #379.
* docs(config): match the frontmatter_read description to the new answer
The job description still advertised "Read a file's frontmatter as a dict",
but the answer is now rendered `key: value` lines — the dict only survives in
metadata. An LLM choosing between `frontmatter_read` and a whole-file `read`
would expect a dict it can index into and get prose instead.
This is the wording-alignment point raised on #397, applied here before
review rather than after.
* fix(frontmatter-read): never fail a valid YAML read on rendering
Review on #564 found two rendering regressions plus boundary losses:
- json.dumps(..., sort_keys=True) raised on datetime.date values (YAML
timestamps) and on mixed-type mapping keys, turning a parseable
frontmatter into a failed job with empty metadata - the base revision
read those files fine.
- Comma-joining lists lost item boundaries (["a, b", "c"] read back as
one ambiguous string), [] was indistinguishable from an empty string,
and the whitespace collapse also mangled spaces inside serialized
values.
_render_value is now a single tolerant renderer: bare strings keep their
text but have line breaks escaped; everything else goes through
json.dumps(ensure_ascii=False, default=str) (no key sort) so collections
stay structured and YAML-only types serialize; anything JSON still
cannot represent falls back to repr. Keys are rendered through the same
path so a multiline key cannot break the one-key-per-line layout either.
The structured metadata contract is untouched.
Also aligns the benchmark.yaml job description with the textual answer
(default.yaml already says 'one key: value per line') and the module
docstring now distinguishes the textual answer from the structured
payload in metadata.
Regression coverage: nested dates, mixed mapping keys, empty lists,
comma-containing items, significant whitespace, and a literal block
scalar with a real newline.
* fix(frontmatter-read): use json.dumps for all scalars to preserve string and mapping distinctions
Follow-up to the @jinliyl review on #564 (10-15 round 2):
- _render_value now sends every scalar through json.dumps rather than
the str-only branch. The str-only branch collapsed a literal '\n'
two-character sequence and an actual newline into the same rendered
form ('\n') and lost the distinction between the string 'null'
and YAML null. The JSON form preserves both.
- Mixed mapping key types (int 1 vs string '1') now round-trip with
their JSON-form distinction in the answer, matching the metadata.
Callers that re-parse the answer body as JSON still discard one
value (this is a YAML-side limitation, not a renderer one); pinned
by a regression test.
- Updated the three existing answer assertions to carry the new
quoted scalar form, and added two regression tests:
test_read_answer_distinguishes_newline_from_literal_backslash_n and
test_read_answer_mixed_mapping_key_types_render_via_json.
Note: the str-only branch's '\n' escape was the only reason the
description: |- literal-block test in test_read_answer_tolerates_rich_yaml_types
rendered as 'line one\\nline two'. With the change, the same
description renders as '"line one\nline two"' (real newline, JSON-
escaped), which is exactly the fix the review asked for.
* fix(frontmatter-read): split key/value rendering so keys are not JSON-quoted
The round-2 change sent every scalar (including string keys) through
json.dumps, which added quotes around string keys. The rendered answer
came out as '- "name": "Cobalt"' instead of '- name: "Cobalt"' on
the round-2 head; the 5 new and updated tests in test_frontmatter_steps
all failed for that reason on Unit Tests - py3.11 through py3.14.
Split the renderer:
- _render_key: pass-through str(key) so the answer reads as '- key:'.
- _render_value: still json.dumps (with default=str and ensure_ascii=False)
for the structural pinning the round-2 review asked for (literal
'\n' vs actual newline, the string 'null' vs YAML null, mixed mapping
key types). Caller formats the line as '- {key}: {value}'.
Updated the three existing answer assertions to match the key-unchanged,
value-quoted shape (e.g. '- name: "Cobalt"'). The two new tests added
in the round-2 commit (newline, mixed keys) were already pinned on the
right shape; the round-1 key-quote was the only bug.
* fix(frontmatter-read): JSON-encode keys too so every answer entry stays on one line
A multiline YAML key broke the one-key-per-line layout because keys were
interpolated raw. Route keys through the same json.dumps renderer as
values, update the expected answers to the quoted-key form, correct the
renderer docstring and mixed-key test comments (JSON collapses int 1 and
string "1" keys alike — the metadata keeps the distinction), and pin the
rich-type answer to the quoted "created": "2026-10-12" rendering.
* test(frontmatter-read): cover multiline keys, reseed YAML blocks as literal strings
Add the multiline-key regression the review asked for, restate the
mixed-mapping-key case as a documented presentation limitation (JSON
stringifies both keys, so a re-parser keeps only the last value), and
write the YAML seeds as triple-quoted literals so black stops joining
the implicit concatenations and pylint's implicit-str-concat goes quiet.
* refactor(frontmatter-read): simplify answer rendering
---------
Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
* Add sandboxed Python execution and home AutoBash plugin
* Harden read-only shell commands and filesystem sandboxes
* Group file sandbox backends in one package
* Fix macOS file sandbox isolation and runtime access
* Refine file sandbox limits and executor checks
* Harden AutoBash policy and macOS sandbox supervision
* fix(home): require ReMe release with file sandbox
* refactor(home): simplify sandbox calls and redundant tests
* fix(execution): bound output and clean up cancelled sandbox startup
* fix(tests): mock Seatbelt platform requirements on Linux
* fix(home): block Git help helpers and harden sandbox cleanup
* Fix sandbox runtime links and subprocess cleanup
* ci: cover Home plugin tests
* test: wait for pipeline descendants to exit after cancellation
* fix: align Home dependencies and preserve cleanup cancellation
Require ReMe 0.4.2.1 and verify isolated Home wheel installation in package CI. Share cancellation-safe cleanup with thread calls, retain AnyIO cancellation semantics, and record Bubblewrap mount targets during argument construction.
A per-item context-length rejection survives the per-item split: the item is resent unchanged and the provider keeps rejecting it, so the chunk never gets a vector and a scoped reindex can never finalize. Re-prepare that single item under a smaller character budget and retry it once, bounded by max_retries.
Only the request shrinks. The chunk keeps its full source text, the vector space is unchanged, and the shortening is logged so the reduced coverage is visible.
Shortening needs an attempt left to send. On the final attempt a shorter request cannot be delivered, so that branch now falls through to the existing input-dependent handling: the rejection stays specific to its item instead of being recorded as a terminal batch failure that drops the healthy items after it. The retry lives inside the input-dependent branch, so the same error is classified once, and the single-use budget helper is inlined.
Follow-up to the merged per-item isolation change (#579), rebased onto the retry-result refactor (#656).
* fix(embedding): keep one bad item from dropping its whole batch
A provider-side request error rejects an entire batch, so a single over-limit
item used to cost the vectors of every item beside it. QwenPaw #8040 shows two
poison chunks turning into exactly twenty lost vectors, while the backfill loop
still reported full success.
Retry a rejected multi-item request one item at a time so only the item that
actually fails is left without a vector. Single-item requests keep their
previous behaviour, and provider-level failures (429 / quota) keep using the
existing retry paths instead of being split.
Also make the backfill loop report what it actually filled, and name the failing
chunk ids in the final reindex error. `processed` counts attempts, so it claimed
success after a partial loss.
* fix(embedding): stop quota failures and transient item errors from multiplying
Review follow-up on the two P2 findings in #579:
* quota exhaustion no longer falls into the per-item split. The store now gives
up explicitly when the provider reports insufficient_quota, including after
the opt-in quota retries are spent, because no single item is to blame and
splitting only multiplied the rejected requests (3 items -> 4 requests).
* the per-item fallback goes back through `_call_with_retry`, so a 429,
connection error or timeout on an isolated item keeps the existing backoff
and retry policy instead of silently dropping that item's vector. A one-item
request still cannot split again.
Regression tests cover both findings: a quota failure stays a single request
with the default policy and with exhausted quota retries, and a transient
failure on one item of a split batch still ends with every healthy sibling
embedded.
python -m pytest tests/unit/test_local_embedding_store.py tests/unit/test_file_store_consistency.py -q
-> 83 passed, 41 skipped
* style(test): hoist the sleeps collector out of the transient-failure loop
CI / Python quality failed on d5dc9b2: the pylint hook reported
tests/unit/test_local_embedding_store.py:935:16:
W0640: Cell variable sleeps defined in loop (cell-var-from-loop)
`fake_sleep` closes over `sleeps` while the loop rebinds it on every
iteration. The collector is now created once before the loop and cleared
at the top of each iteration, which keeps the per-iteration assertion
(`assert sleeps == [1.0]`) exactly as strict and removes the loop-scoped
rebinding.
Verification (repo hook arguments):
pylint -> Your code has been rated at 10.00/10, exit 0
black --check --line-length=120 --target-version=py311 -> clean
flake8 --extend-ignore=E203 --max-line-length=120 -> clean
* fix(embedding): keep provider-wide failures out of the per-item split
Review follow-up on the P2 raised against 4d8a3c5.
`_call_with_retry` sent every unrecognized exception to `_embed_individually`
whenever the request had more than one item. Only rate limiting and
`insufficient_quota` were excluded, so a 401, a 403 or a 5xx still split a
ten-item request into ten extra single-item requests, none of which can
succeed: credentials, permissions and server outages are properties of the
endpoint, not of any individual input.
The split path is now gated on a dedicated classifier,
`_is_provider_wide_failure()`, which recognizes 401/403 and 5xx by status
code (with the OpenAI SDK class names as a fallback for exceptions that do
not expose one). Transient server errors keep their retry policy through the
existing backoff; the remaining provider-wide failures stop the loop and
report unhealthy, like the quota branch. Input-dependent rejections such as
the per-item context-length limit are unchanged and still isolate one item
at a time.
Regression coverage: test_provider_wide_credentials_errors_are_not_split_into_per_item_requests
(401/403 -> a single request of size 10) and
test_provider_wide_server_errors_are_retried_without_splitting
(500 -> batch sizes [10, 10, 10] with sleeps [1.0, 2.0]). The existing
context-length, quota, 429 and per-item transient-retry tests are retained.
Local verification, using the repo's own hook arguments:
pylint -> Your code has been rated at 10.00/10, exit 0
black -> 2 files would be left unchanged
flake8 -> clean
Classifier discriminator check (old vs new split decision, extracted from the
production file and executed):
400 context-length -> SPLIT / SPLIT
401, 403, 500, 504 -> SPLIT / NO-SPLIT
429 and insufficient_quota keep their own earlier branches.
* fix(embedding): retry SDK transport failures instead of splitting the batch
Third review follow-up on #579.
`_call_with_retry` only caught the built-in `TimeoutError`, `ConnectionError`
and `OSError`. `openai.APIConnectionError` and `openai.APITimeoutError` do not
inherit from any of them (they go APIError -> OpenAIError -> Exception), so
they reached the generic handler: `_is_provider_wide_failure()` did not
recognize them either, and a multi-item request fanned out into per-item
requests that failed for exactly the same reason. The same gap meant a
single-item request got no ReMe-level retry at all after a transport failure.
Transport failures are now recognized by a dedicated classifier,
`_is_transient_transport_failure()`, covering the built-in socket errors as
well as the SDK/httpx exception types (matched across the MRO, so subclasses
such as `APITimeoutError` are included) without importing the provider SDK.
They retry the original request with the existing backoff and never split; the
retry budget is unchanged.
Regression coverage:
- test_sdk_transport_errors_are_retried_without_splitting
APIConnectionError / APITimeoutError -> batch sizes [10, 10, 10], sleeps
[1.0, 2.0], then failure with no single-item requests.
- test_sdk_transport_error_recovers_without_losing_the_batch
one APITimeoutError then success -> [[1.0, 0.0]] * 10, batch sizes [10, 10].
- test_isolated_item_retries_an_sdk_transport_error
context-length rejection, then APIConnectionError on one isolated item ->
that item's vector is retained.
The existing context-length, 401/403, 5xx, quota and rate-limit tests are
retained.
Local verification with the repo's own hook arguments:
pylint -> Your code has been rated at 10.00/10, exit 0
black -> 2 files would be left unchanged
flake8 -> clean
Branch-order discriminator check (classifiers extracted from the production
file and executed against each error type):
case old new
400 context-length SPLIT SPLIT
401 / 403 SPLIT STOP
500 SPLIT RETRY
SDK APIConnectionError SPLIT RETRY
SDK APITimeoutError SPLIT RETRY
builtin ConnectionError RETRY RETRY
builtin TimeoutError RETRY RETRY
429 RETRY RETRY
insufficient_quota STOP STOP
* fix(embedding): treat 404 as a provider-wide failure too
Fourth review follow-up on #579.
The classifier added in c9448ae covered 401/403 and 5xx, but not 404. A missing
model or a wrong embedding route answers with 404 (SDK `NotFoundError`), which
no individual input can explain, so the batch still fell through to
`_embed_individually()`: a ten-item request against a bad model name produced
eleven requests, all failing for the same reason.
404 is now part of the provider-wide set (`_PROVIDER_WIDE_STATUS_CODES`) and
`NotFoundError` joins the class-name fallback. Like 401/403 it is not
transient, so the request is reported as failed once instead of being retried
or split. The context-length `400` is deliberately left out of that set: an
over-limit item is the one status a single input can explain by itself, so it
has to stay isolatable.
Regression coverage:
- test_provider_wide_statuses_exclude_input_dependent_bad_request
401/403/404/500 -> provider-wide; an over-limit 400 -> not provider-wide.
- test_provider_wide_not_found_errors_are_not_split
404 -> batch sizes [10], no sleeps, no single-item requests.
The existing context-length isolation, 401/403, 5xx, quota, rate-limit and SDK
transport tests are retained.
Local verification with the repo's own hook arguments:
pylint -> Your code has been rated at 10.00/10, exit 0
black -> 2 files would be left unchanged
flake8 -> clean
Branch-order discriminator check (classifiers extracted from the production
file and executed against each error type):
case old new
400 context-length SPLIT SPLIT
401 / 403 SPLIT STOP
404 NotFoundError SPLIT STOP
500 SPLIT RETRY
SDK APIConnectionError SPLIT RETRY
SDK APITimeoutError SPLIT RETRY
builtin ConnectionError RETRY RETRY
builtin TimeoutError RETRY RETRY
429 RETRY RETRY
insufficient_quota STOP STOP
* fix(embedding): classify every provider status shape and split only input errors
The per-item fallback still multiplied service failures into one request per item.
Both gaps come from the fallback this PR adds, so they are fixed here.
Status extraction
- `_http_status()` reads the status from `status_code`, an integer `code` (Google GenAI
exposes the HTTP status there), `response.status_code` (httpx-style wrappers), and the
structured `status_code` embedded in the message (AgentScope's DashScope embedding turns
every non-200 response into a plain `RuntimeError`).
- Only integers are read from `code`. A string `code` names an error
(`insufficient_quota`, `rate_limit_exceeded`, `InvalidApiKey`); reading it as a status
would misclassify quota and rate-limit errors.
- `_is_provider_wide_failure`, `_is_transient_provider_failure`, `_is_rate_limited` and
`_is_input_dependent_failure` all classify through it.
Splitting
- The generic fallback splits only when `_is_input_dependent_failure()` matches: a
context-length rejection, the one error a single over-long item explains by itself.
Anything unrecognized is provider-wide and is never answered with one request per item.
- This narrows the isolation this PR adds. An input-dependent error that carries neither a
status nor an input-length signal is no longer split, so it costs its batch instead of
multiplying the requests. Erring towards the batch keeps one provider failure from
becoming N failures and losing the healthy vectors anyway.
- `_embed_individually()` drops the redundant `len(single) == 1` check: the
`_call_with_retry([text])` contract only returns a result of the expected length.
Verified against the real exception classes: `google.genai.errors.APIError.code` is an
`int`, and a real DashScope response yields a `RuntimeError` whose message carries
`"status_code"`. Regression coverage uses the real classes when available and mirrors
their shape otherwise.
* fix(embedding): stop per-item isolation on a provider-wide failure and read serialized codes
- `_embed_individually` now stops asking for the remaining items as soon as one isolated
item fails provider-wide (credentials, missing model, exhausted quota, exhausted
transient retries). No remaining input can explain that failure, so continuing only
multiplies the rejected requests. The vectors already recovered are kept and the items
that were never answered stay `None`.
- `_call_with_retry` reports that outcome through an optional `terminal_failure`
out-parameter, so its return type and every existing call site stay unchanged.
- `_error_code` reads the business error code from the attribute, the structured body, or
the serialized message. AgentScope's DashScope embedding raises a plain `RuntimeError`
with the response embedded in the message, so a quota error was classified as a rate
limit and never used the configured `quota_retry_delay`.
Regression tests cover both halves of the isolation boundary (a provider-wide failure
stops the loop; an input-dependent failure does not) and the serialized DashScope quota
path with quota retries enabled.
* fix(embedding): treat an input-length 413 as a per-item rejection
* fix(embedding): keep the health state after a partial success
`_compute_batch()` set `is_healthy = True` whenever the batch produced at least one
vector. Per-item isolation can recover an early item and then meet a provider-wide
failure (exhausted quota, authentication, exhausted transient retries), which
`_call_with_retry()` correctly records as unhealthy before it stops. The recovered
vector then overwrote that state, so a request that ended in an outage reported the
provider as healthy.
The terminal-failure flag is now threaded from `_embed_individually()` back to
`_compute_batch()`. The vectors recovered before the failure are still returned and
cached; only the health state now follows the failure. This matters because
`LocalFileStore._recover_after_real_request()` schedules missing-vector backfill off
the unhealthy-to-healthy transition: a hidden outage would either trigger backfill
while the provider is still unavailable or suppress a later genuine recovery.
Adds a regression test through `get_embeddings()` that checks both the retained
partial vector and the final unhealthy state. The existing isolation test covers
`_call_with_retry()` directly, so it never saw the overwrite.
* fix(embedding): leave provider 5xx retries to the layers that own them
A store-level retry for a provider 5xx stacked on top of the retries the
embedding wrapper and the provider SDK already perform, so a batch the SDK
answered after 12 attempts became 36. Retry ownership stays with those
layers; the store answers a provider-wide 5xx once and gives up.
Transport failures keep their retry because they describe a connection that
no lower layer has reported, and the opt-in quota policy is untouched.
Tests updated to pin the new boundary: a 503 is answered once for the
OpenAI-shaped, Gemini-shaped and DashScope-shaped errors, while a 429 keeps
its retry from the rate-limit branch.
* fix(embedding): make transport retry ownership explicit
The store retried a connection failure or a timeout on top of the embedding
wrapper and the provider SDK, which already retry both, so a persistent
timeout on a batch reached the provider 36 times instead of 12: the OpenAI
model retries a batch max_retries + 1 times and the SDK retries each of those
attempts.
Transport failures stay provider-wide -- the request is never split per item
-- but the retry is now owned explicitly instead of implicitly:
* a whole-batch request leaves it to the layers below, which already retry it;
* an isolated single-item attempt keeps the store's backoff, because that
attempt is the last chance to keep the one vector the isolation step is
recovering.
Rate limiting is unchanged. The transport cases now pin both budgets, and
three tests drive a real OpenAIAsEmbedding wrapper and a real
openai.AsyncClient over an httpx.MockTransport, so the provider attempt count
is measured rather than assumed.
* fix(embedding): check the failing error against the provider retry policy
* fix(auto-memory): serialize session transcript writes per session
_save_session_messages ran its read-merge-write on the per-session
JSONL transcript without any lock. Concurrent HTTP requests saving the
same session could interleave: two writers both appending against a
stale snapshot write overlapping history twice, and an append racing
a full rewrite can be overwritten by the rewriter's stale merge,
losing messages.
Hold get_path_lock(path) across the whole read, merge, and write,
consistent with the existing session image handling in the same file.
Merge, ordering, and can_append logic are unchanged.
Add deterministic regression tests that gate one writer's read between
snapshot capture and write: concurrent appends must merge disjoint
messages in created_at order, and an appended message must survive a
concurrent full rewrite. Both tests fail without the lock.
Closes#622
* test(auto-memory): make session-lock regression tests deterministic
Replace fixed sleeps with explicit synchronization: a shared operation
log records lock/read/write progress per writer, and writer A holds its
post-read pause until writer B is queued on the transcript lock. The
tests now assert the serialization order from the log (B reads only
after A's write completed) in addition to transcript content, so they
no longer depend on IO timing.
Also fix pylint findings from CI: use from-import for the auto_memory
module and drop unused tmp_path parameters.
* fix(agent_wrapper): release Codex threads after each turn
Loaded Codex threads and their per-thread MCP server subprocesses were
never unloaded from the app-server, growing file descriptors and child
processes monotonically until the wrapper was closed (fd exhaustion,
os error 24). Reply, streaming reply, and compact_session now
unsubscribe the thread in a turn-scoped finally block by default; the
new release_threads=False option restores the previous behavior.
Unsubscribe failures are logged and never break the turn, and threads
remain resumable after being unloaded.
Closesagentscope-ai/ReMe#609
* fix(agent_wrapper): keep tool-context guard on failed release and cover stream startup
A failed thread/unsubscribe left the thread loaded in the app-server
but dropped its tool-context entry, so a later resume with changed
tool_context_id or injected_job_kwargs would silently bypass the
constraint check. Keep the guard entry unless the unsubscribe request
succeeds.
reply_stream now releases the thread from an outer try/finally scoped
right after _open_thread, so threads are also unloaded when turn/start
raises or the caller is cancelled while awaiting it.
* fix(search): accept CLI queries that look like numbers
The CLI parses key=value arguments into typed values, so query=404 reached the search steps as an int and failed on .strip(), and query=NaN or query=Infinity became non-finite floats that the HTTP client cannot encode as JSON and the MCP client sends as null.
Read the query as text in the four search steps, as ChatStep already does, and keep values that float() reads as NaN or infinity as strings.
* fix(search): treat only a missing query as empty
str(query or "") turned query=0 and query=0.0 into an empty string, so those searches still failed with "query cannot be empty". Check for None before converting instead.
The BEAM and LME plugins replace the default search job with their own SearchV2Step, which still called .strip() on the raw value, so query=404 raised AttributeError when either plugin was enabled. Read the query the same way there.
The numeric-query test is now parametrized over the six steps and passes 404, 0 and 0.0 through parse_kwargs first.
* fix(stat): put mtime in the answer where the job description promises it
The ``stat`` job is advertised to agents as "Stat path (size, mtime, exists,
is_dir, is_file)", but the answer only ever carried the path, type and size —
``mtime`` lived solely in metadata, and MCP returns only ``response.answer``.
So an agent asking how fresh a file is got no answer, even though the tool
description told it the field was there.
Render mtime in both branches of the answer:
stat: topics/n.md (file, 20 bytes, mtime 2026-09-21T15:50:52.880316)
stat: topics (dir, mtime 2026-09-21T15:50:52.880316)
The value is hoisted into a local so the answer and ``metadata["mtime"]`` are
guaranteed to come from the same ``stat()`` call rather than two reads. The
module docstring claimed the agent receives "size, mtime, mime type, and ...
frontmatter"; it now states that size and mtime are in the answer while mime
and frontmatter stay in metadata, matching reality.
Metadata is unchanged, so programmatic consumers see no difference. This is
the same answer-completeness class as #564, found while auditing which
advertised job fields actually reach the LLM.
* docs(stat): say which consumers actually see mime and frontmatter
Review feedback on #566: the module docstring told the agent it gets
"mime type and ... the parsed frontmatter alongside in metadata", which
reads as though an agent can use those fields to decide whether to
download or parse a file. It cannot: the MCP service returns only
`response.answer` (`mcp_tools.py` -> `return response.answer`), so
`metadata["mime"]` and `metadata["frontmatter"]` reach direct `Response`
consumers but never an MCP caller.
Spell the split out instead of implying one audience: the answer carries
path, type, size and mtime, and an agent that needs the frontmatter has
to read the file rather than stat it. The envelope description below is
unchanged, since it already lists the full metadata shape.
* fix(reindex): return a readable summary so Studio stops showing a placeholder
`ReindexStep` assigned the store's report dict straight to
`response.answer`. ReMe Studio's settings panel already anticipates a string
answer:
typeof result.answer === "string" && result.answer ? result.answer : t("indexRebuilt")
(`reme_studio/app/settings-center.tsx`), so with a dict answer that branch was
unreachable and every rebuild reported the generic "index rebuilt" text. The
operator never saw which scopes ran or how many entries each indexed.
Render the report as one summary line — `Reindexed scope=all: bm25=7,
embedding=5, tag=2` for the composite scope and `Reindexed scope=bm25: 3
indexed` for a single one. The report itself still lands in metadata, so
programmatic consumers are unaffected, and Studio needs no change because its
string branch now fires.
`reindex` is the only answer-carrying consumer of this step; no test asserted
the answer content, only metadata, so nothing else moved.
* chore(reindex): tighten the renderer and deduplicate the summary test
Review on #565: the helper's docstring carried the Studio background
that already lives in the PR description; the intermediate per_scope
dict collapses into the join generator; and the all-scope delegation
test now uses distinct counts (7, 5, 2) so it alone pins both the exact
summary answer and the full metadata report, replacing the overlapping
summary test.
* refactor(reindex): share summary return formatting
---------
Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
benchmark.yaml's python_execute job advertised a `timeout` parameter, but
`timeout` is one of the CLI's client-selection kwargs (reme/reme.py pops it
before the payload is sent), so `reme python_execute code=... timeout=300`
never delivered the timeout to the job. The value silently became the HTTP
client's request timeout while the subprocess kept the 60s default, so
benchmark code that needs longer was killed at 60s with no hint why. HTTP
and MCP callers were unaffected.
Rename the parameter to `python_timeout`, mirroring how the shell job
already avoids the same collision with `shell_timeout`. The step prefers
the new name and still accepts the legacy `timeout` context key so
existing benchmark configs keep working. A new guard test fails if any
job in reme/config/*.yaml or a plugin plugin.yaml declares a
client-reserved parameter name again.
* feat(docker): add container deployment and multi-platform release workflow
* fix(llm): initialize providers on model access and preserve runtime injection
* test(llm): keep rejected model updates independent of credentials
