Commit graph

141 commits

Author SHA1 Message Date
BeiMu-new
6dcc7664a3 fix(embedding): leave provider 5xx retries to the layers that own them
A store-level retry for a provider 5xx stacked on top of the retries the
embedding wrapper and the provider SDK already perform, so a batch the SDK
answered after 12 attempts became 36. Retry ownership stays with those
layers; the store answers a provider-wide 5xx once and gives up.

Transport failures keep their retry because they describe a connection that
no lower layer has reported, and the opt-in quota policy is untouched.

Tests updated to pin the new boundary: a 503 is answered once for the
OpenAI-shaped, Gemini-shaped and DashScope-shaped errors, while a 429 keeps
its retry from the rate-limit branch.
2026-10-16 21:10:39 +08:00
BeiMu-new
db1665db67 fix(embedding): keep the health state after a partial success
`_compute_batch()` set `is_healthy = True` whenever the batch produced at least one
vector. Per-item isolation can recover an early item and then meet a provider-wide
failure (exhausted quota, authentication, exhausted transient retries), which
`_call_with_retry()` correctly records as unhealthy before it stops. The recovered
vector then overwrote that state, so a request that ended in an outage reported the
provider as healthy.

The terminal-failure flag is now threaded from `_embed_individually()` back to
`_compute_batch()`. The vectors recovered before the failure are still returned and
cached; only the health state now follows the failure. This matters because
`LocalFileStore._recover_after_real_request()` schedules missing-vector backfill off
the unhealthy-to-healthy transition: a hidden outage would either trigger backfill
while the provider is still unavailable or suppress a later genuine recovery.

Adds a regression test through `get_embeddings()` that checks both the retained
partial vector and the final unhealthy state. The existing isolation test covers
`_call_with_retry()` directly, so it never saw the overwrite.
2026-10-16 21:10:39 +08:00
BeiMu-new
eca11c9e04 fix(embedding): treat an input-length 413 as a per-item rejection 2026-10-16 21:10:39 +08:00
BeiMu-new
f95ffc30ef fix(embedding): stop per-item isolation on a provider-wide failure and read serialized codes
- `_embed_individually` now stops asking for the remaining items as soon as one isolated
  item fails provider-wide (credentials, missing model, exhausted quota, exhausted
  transient retries). No remaining input can explain that failure, so continuing only
  multiplies the rejected requests. The vectors already recovered are kept and the items
  that were never answered stay `None`.

- `_call_with_retry` reports that outcome through an optional `terminal_failure`
  out-parameter, so its return type and every existing call site stay unchanged.

- `_error_code` reads the business error code from the attribute, the structured body, or
  the serialized message. AgentScope's DashScope embedding raises a plain `RuntimeError`
  with the response embedded in the message, so a quota error was classified as a rate
  limit and never used the configured `quota_retry_delay`.

Regression tests cover both halves of the isolation boundary (a provider-wide failure
stops the loop; an input-dependent failure does not) and the serialized DashScope quota
path with quota retries enabled.
2026-10-16 21:10:39 +08:00
BeiMu-new
5e70ec0476 fix(embedding): classify every provider status shape and split only input errors
The per-item fallback still multiplied service failures into one request per item.
Both gaps come from the fallback this PR adds, so they are fixed here.

Status extraction
- `_http_status()` reads the status from `status_code`, an integer `code` (Google GenAI
  exposes the HTTP status there), `response.status_code` (httpx-style wrappers), and the
  structured `status_code` embedded in the message (AgentScope's DashScope embedding turns
  every non-200 response into a plain `RuntimeError`).
- Only integers are read from `code`. A string `code` names an error
  (`insufficient_quota`, `rate_limit_exceeded`, `InvalidApiKey`); reading it as a status
  would misclassify quota and rate-limit errors.
- `_is_provider_wide_failure`, `_is_transient_provider_failure`, `_is_rate_limited` and
  `_is_input_dependent_failure` all classify through it.

Splitting
- The generic fallback splits only when `_is_input_dependent_failure()` matches: a
  context-length rejection, the one error a single over-long item explains by itself.
  Anything unrecognized is provider-wide and is never answered with one request per item.
- This narrows the isolation this PR adds. An input-dependent error that carries neither a
  status nor an input-length signal is no longer split, so it costs its batch instead of
  multiplying the requests. Erring towards the batch keeps one provider failure from
  becoming N failures and losing the healthy vectors anyway.
- `_embed_individually()` drops the redundant `len(single) == 1` check: the
  `_call_with_retry([text])` contract only returns a result of the expected length.

Verified against the real exception classes: `google.genai.errors.APIError.code` is an
`int`, and a real DashScope response yields a `RuntimeError` whose message carries
`"status_code"`. Regression coverage uses the real classes when available and mirrors
their shape otherwise.
2026-10-16 21:10:39 +08:00
BeiMu-new
eba753f783 fix(embedding): treat 404 as a provider-wide failure too
Fourth review follow-up on #579.

The classifier added in c9448ae covered 401/403 and 5xx, but not 404. A missing
model or a wrong embedding route answers with 404 (SDK `NotFoundError`), which
no individual input can explain, so the batch still fell through to
`_embed_individually()`: a ten-item request against a bad model name produced
eleven requests, all failing for the same reason.

404 is now part of the provider-wide set (`_PROVIDER_WIDE_STATUS_CODES`) and
`NotFoundError` joins the class-name fallback. Like 401/403 it is not
transient, so the request is reported as failed once instead of being retried
or split. The context-length `400` is deliberately left out of that set: an
over-limit item is the one status a single input can explain by itself, so it
has to stay isolatable.

Regression coverage:
- test_provider_wide_statuses_exclude_input_dependent_bad_request
  401/403/404/500 -> provider-wide; an over-limit 400 -> not provider-wide.
- test_provider_wide_not_found_errors_are_not_split
  404 -> batch sizes [10], no sleeps, no single-item requests.
The existing context-length isolation, 401/403, 5xx, quota, rate-limit and SDK
transport tests are retained.

Local verification with the repo's own hook arguments:
  pylint -> Your code has been rated at 10.00/10, exit 0
  black  -> 2 files would be left unchanged
  flake8 -> clean

Branch-order discriminator check (classifiers extracted from the production
file and executed against each error type):

  case                      old     new
  400 context-length        SPLIT   SPLIT
  401 / 403                 SPLIT   STOP
  404 NotFoundError         SPLIT   STOP
  500                       SPLIT   RETRY
  SDK APIConnectionError    SPLIT   RETRY
  SDK APITimeoutError       SPLIT   RETRY
  builtin ConnectionError   RETRY   RETRY
  builtin TimeoutError      RETRY   RETRY
  429                       RETRY   RETRY
  insufficient_quota        STOP    STOP
2026-10-16 21:10:39 +08:00
BeiMu-new
930fddc4d2 fix(embedding): retry SDK transport failures instead of splitting the batch
Third review follow-up on #579.

`_call_with_retry` only caught the built-in `TimeoutError`, `ConnectionError`
and `OSError`. `openai.APIConnectionError` and `openai.APITimeoutError` do not
inherit from any of them (they go APIError -> OpenAIError -> Exception), so
they reached the generic handler: `_is_provider_wide_failure()` did not
recognize them either, and a multi-item request fanned out into per-item
requests that failed for exactly the same reason. The same gap meant a
single-item request got no ReMe-level retry at all after a transport failure.

Transport failures are now recognized by a dedicated classifier,
`_is_transient_transport_failure()`, covering the built-in socket errors as
well as the SDK/httpx exception types (matched across the MRO, so subclasses
such as `APITimeoutError` are included) without importing the provider SDK.
They retry the original request with the existing backoff and never split; the
retry budget is unchanged.

Regression coverage:
- test_sdk_transport_errors_are_retried_without_splitting
  APIConnectionError / APITimeoutError -> batch sizes [10, 10, 10], sleeps
  [1.0, 2.0], then failure with no single-item requests.
- test_sdk_transport_error_recovers_without_losing_the_batch
  one APITimeoutError then success -> [[1.0, 0.0]] * 10, batch sizes [10, 10].
- test_isolated_item_retries_an_sdk_transport_error
  context-length rejection, then APIConnectionError on one isolated item ->
  that item's vector is retained.
The existing context-length, 401/403, 5xx, quota and rate-limit tests are
retained.

Local verification with the repo's own hook arguments:
  pylint -> Your code has been rated at 10.00/10, exit 0
  black  -> 2 files would be left unchanged
  flake8 -> clean

Branch-order discriminator check (classifiers extracted from the production
file and executed against each error type):

  case                      old     new
  400 context-length        SPLIT   SPLIT
  401 / 403                 SPLIT   STOP
  500                       SPLIT   RETRY
  SDK APIConnectionError    SPLIT   RETRY
  SDK APITimeoutError       SPLIT   RETRY
  builtin ConnectionError   RETRY   RETRY
  builtin TimeoutError      RETRY   RETRY
  429                       RETRY   RETRY
  insufficient_quota        STOP    STOP
2026-10-16 21:10:38 +08:00
BeiMu-new
254d895f96 fix(embedding): keep provider-wide failures out of the per-item split
Review follow-up on the P2 raised against 4d8a3c5.

`_call_with_retry` sent every unrecognized exception to `_embed_individually`
whenever the request had more than one item. Only rate limiting and
`insufficient_quota` were excluded, so a 401, a 403 or a 5xx still split a
ten-item request into ten extra single-item requests, none of which can
succeed: credentials, permissions and server outages are properties of the
endpoint, not of any individual input.

The split path is now gated on a dedicated classifier,
`_is_provider_wide_failure()`, which recognizes 401/403 and 5xx by status
code (with the OpenAI SDK class names as a fallback for exceptions that do
not expose one). Transient server errors keep their retry policy through the
existing backoff; the remaining provider-wide failures stop the loop and
report unhealthy, like the quota branch. Input-dependent rejections such as
the per-item context-length limit are unchanged and still isolate one item
at a time.

Regression coverage: test_provider_wide_credentials_errors_are_not_split_into_per_item_requests
(401/403 -> a single request of size 10) and
test_provider_wide_server_errors_are_retried_without_splitting
(500 -> batch sizes [10, 10, 10] with sleeps [1.0, 2.0]). The existing
context-length, quota, 429 and per-item transient-retry tests are retained.

Local verification, using the repo's own hook arguments:
  pylint  -> Your code has been rated at 10.00/10, exit 0
  black   -> 2 files would be left unchanged
  flake8  -> clean
Classifier discriminator check (old vs new split decision, extracted from the
production file and executed):
  400 context-length     -> SPLIT / SPLIT
  401, 403, 500, 504     -> SPLIT / NO-SPLIT
  429 and insufficient_quota keep their own earlier branches.
2026-10-16 21:10:38 +08:00
BeiMu-new
36eae4f033 style(test): hoist the sleeps collector out of the transient-failure loop
CI / Python quality failed on d5dc9b2: the pylint hook reported

    tests/unit/test_local_embedding_store.py:935:16:
    W0640: Cell variable sleeps defined in loop (cell-var-from-loop)

`fake_sleep` closes over `sleeps` while the loop rebinds it on every
iteration. The collector is now created once before the loop and cleared
at the top of each iteration, which keeps the per-iteration assertion
(`assert sleeps == [1.0]`) exactly as strict and removes the loop-scoped
rebinding.

Verification (repo hook arguments):
  pylint  -> Your code has been rated at 10.00/10, exit 0
  black --check --line-length=120 --target-version=py311 -> clean
  flake8 --extend-ignore=E203 --max-line-length=120       -> clean
2026-10-16 21:10:38 +08:00
BeiMu-new
bc47906f79 fix(embedding): stop quota failures and transient item errors from multiplying
Review follow-up on the two P2 findings in #579:

* quota exhaustion no longer falls into the per-item split. The store now gives
  up explicitly when the provider reports insufficient_quota, including after
  the opt-in quota retries are spent, because no single item is to blame and
  splitting only multiplied the rejected requests (3 items -> 4 requests).
* the per-item fallback goes back through `_call_with_retry`, so a 429,
  connection error or timeout on an isolated item keeps the existing backoff
  and retry policy instead of silently dropping that item's vector. A one-item
  request still cannot split again.

Regression tests cover both findings: a quota failure stays a single request
with the default policy and with exhausted quota retries, and a transient
failure on one item of a split batch still ends with every healthy sibling
embedded.

python -m pytest tests/unit/test_local_embedding_store.py tests/unit/test_file_store_consistency.py -q
-> 83 passed, 41 skipped
2026-10-16 21:10:38 +08:00
BeiMu-new
ab96ab680a fix(embedding): keep one bad item from dropping its whole batch
A provider-side request error rejects an entire batch, so a single over-limit
item used to cost the vectors of every item beside it. QwenPaw #8040 shows two
poison chunks turning into exactly twenty lost vectors, while the backfill loop
still reported full success.

Retry a rejected multi-item request one item at a time so only the item that
actually fails is left without a vector. Single-item requests keep their
previous behaviour, and provider-level failures (429 / quota) keep using the
existing retry paths instead of being split.

Also make the backfill loop report what it actually filled, and name the failing
chunk ids in the final reindex error. `processed` counts attempts, so it claimed
success after a partial loss.
2026-10-16 21:10:38 +08:00
jinliyl
638c81e88e
chore: release 0.4.2.0 and simplify publishing workflows (#631)
Some checks failed
CI / Python tests / Unit Tests - py3.14 (push) Has been cancelled
CI / Windows / CLI smoke - py3.11 (push) Has been cancelled
Security / CodeQL / Analyze python (push) Has been cancelled
CI / ReMe Studio / Studio checks (push) Has been cancelled
Security / CodeQL / Analyze javascript-typescript (push) Has been cancelled
CI / Python quality / Pre-commit (push) Has been cancelled
CI / DSH plugin / Validate DSH plugin (push) Has been cancelled
CI / OpenClaw plugin / Validate OpenClaw plugin (push) Has been cancelled
CI / Python packages / Build and verify distributions (push) Has been cancelled
CI / Python quality / GitHub Actions (push) Has been cancelled
CI / Python tests / Unit Tests - py3.11 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.12 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.13 (push) Has been cancelled
CI and Release / Docker / Build and test / amd64 (push) Has been cancelled
CI and Release / Docker / Build and test / arm64 (push) Has been cancelled
CI and Release / Docker / Publish multi-platform tags (push) Has been cancelled
* chore: release 0.4.2.0 and derive publish versions from source

* chore(dsh): prepare 0.1.3 for the next iteration
2026-10-16 16:53:52 +08:00
jinliyl
d3abc0b5b0
fix(codex): retain tool constraints until threads unload (#630) 2026-10-16 16:46:02 +08:00
Chen Wang
a6509ff479
fix(agent_wrapper): release Codex threads after each turn (#621)
* fix(agent_wrapper): release Codex threads after each turn

Loaded Codex threads and their per-thread MCP server subprocesses were
never unloaded from the app-server, growing file descriptors and child
processes monotonically until the wrapper was closed (fd exhaustion,
os error 24). Reply, streaming reply, and compact_session now
unsubscribe the thread in a turn-scoped finally block by default; the
new release_threads=False option restores the previous behavior.
Unsubscribe failures are logged and never break the turn, and threads
remain resumable after being unloaded.

Closes agentscope-ai/ReMe#609

* fix(agent_wrapper): keep tool-context guard on failed release and cover stream startup

A failed thread/unsubscribe left the thread loaded in the app-server
but dropped its tool-context entry, so a later resume with changed
tool_context_id or injected_job_kwargs would silently bypass the
constraint check. Keep the guard entry unless the unsubscribe request
succeeds.

reply_stream now releases the thread from an outer try/finally scoped
right after _open_thread, so threads are also unloaded when turn/start
raises or the caller is cancelled while awaiting it.
2026-10-16 13:59:30 +08:00
hwan
72e243b7d6
fix(auto-dream): checkpoint scanned source versions (#618)
Some checks failed
CI / DSH plugin / Validate DSH plugin (push) Waiting to run
CI / Python packages / Build and verify distributions (push) Waiting to run
CI / Python quality / GitHub Actions (push) Waiting to run
CI / Python quality / Pre-commit (push) Waiting to run
CI / Python tests / Unit Tests - py3.11 (push) Waiting to run
CI / Windows / CLI smoke - py3.11 (push) Waiting to run
CI / Python tests / Unit Tests - py3.12 (push) Waiting to run
CI / Python tests / Unit Tests - py3.13 (push) Waiting to run
CI / Python tests / Unit Tests - py3.14 (push) Waiting to run
CI / ReMe Studio / Studio checks (push) Waiting to run
CI and Release / Docker / Build and test / amd64 (push) Waiting to run
CI and Release / Docker / Build and test / arm64 (push) Waiting to run
CI and Release / Docker / Publish multi-platform tags (push) Blocked by required conditions
Security / CodeQL / Analyze javascript-typescript (push) Waiting to run
Security / CodeQL / Analyze python (push) Waiting to run
Deploy / Documentation / Build documentation (push) Has been cancelled
Deploy / Documentation / deploy (push) Has been cancelled
* fix(auto-dream): checkpoint scanned source versions

* test(auto-dream): exercise failed day-index checkpoint guard

* test(auto-dream): exercise valid correction receipts

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-10-16 12:13:48 +08:00
xyf2020
514ca643a1
feat(token counter): add Qwen token counting utility with fallback (#616)
* feat: add Qwen token counting utility with fallback

* fix: preserve Qwen tokenizer text prefixes and isolate loading

* fix: isolate injected Qwen tokenizers and refresh local sources

* perf: avoid unnecessary Qwen tokenizer initialization and locking

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-10-16 11:58:35 +08:00
jinliyl
e478b68085
fix(dsh): support 0.2.0 RC and current Harness settings (#629) 2026-10-16 11:32:38 +08:00
Zhengxu (Joshua) Jin
0b5bb64f26
fix(search): accept CLI queries that look like numbers (#612)
* fix(search): accept CLI queries that look like numbers

The CLI parses key=value arguments into typed values, so query=404 reached the search steps as an int and failed on .strip(), and query=NaN or query=Infinity became non-finite floats that the HTTP client cannot encode as JSON and the MCP client sends as null.

Read the query as text in the four search steps, as ChatStep already does, and keep values that float() reads as NaN or infinity as strings.

* fix(search): treat only a missing query as empty

str(query or "") turned query=0 and query=0.0 into an empty string, so those searches still failed with "query cannot be empty". Check for None before converting instead.

The BEAM and LME plugins replace the default search job with their own SearchV2Step, which still called .strip() on the raw value, so query=404 raised AttributeError when either plugin was enabled. Read the query the same way there.

The numeric-query test is now parametrized over the six steps and passes 404, 0 and 0.0 through parse_kwargs first.
2026-10-16 10:36:05 +08:00
jinliyl
32e1f7faad
refactor(stat): consolidate answer assertions and shorten docs (#627) 2026-10-15 21:04:27 +08:00
Ziyang Guo
29b83529ef
fix(stat): put mtime in the answer where the job description promises it (#566)
* fix(stat): put mtime in the answer where the job description promises it

The ``stat`` job is advertised to agents as "Stat path (size, mtime, exists,
is_dir, is_file)", but the answer only ever carried the path, type and size —
``mtime`` lived solely in metadata, and MCP returns only ``response.answer``.
So an agent asking how fresh a file is got no answer, even though the tool
description told it the field was there.

Render mtime in both branches of the answer:

    stat: topics/n.md (file, 20 bytes, mtime 2026-09-21T15:50:52.880316)
    stat: topics (dir, mtime 2026-09-21T15:50:52.880316)

The value is hoisted into a local so the answer and ``metadata["mtime"]`` are
guaranteed to come from the same ``stat()`` call rather than two reads. The
module docstring claimed the agent receives "size, mtime, mime type, and ...
frontmatter"; it now states that size and mtime are in the answer while mime
and frontmatter stay in metadata, matching reality.

Metadata is unchanged, so programmatic consumers see no difference. This is
the same answer-completeness class as #564, found while auditing which
advertised job fields actually reach the LLM.

* docs(stat): say which consumers actually see mime and frontmatter

Review feedback on #566: the module docstring told the agent it gets
"mime type and ... the parsed frontmatter alongside in metadata", which
reads as though an agent can use those fields to decide whether to
download or parse a file. It cannot: the MCP service returns only
`response.answer` (`mcp_tools.py` -> `return response.answer`), so
`metadata["mime"]` and `metadata["frontmatter"]` reach direct `Response`
consumers but never an MCP caller.

Spell the split out instead of implying one audience: the answer carries
path, type, size and mtime, and an agent that needs the frontmatter has
to read the file rather than stat it. The envelope description below is
unchanged, since it already lists the full metadata shape.
2026-10-15 20:49:44 +08:00
machaoxin0407
404aa6bf3b
perf(search): prepare search filters once per query (#591)
* perf(search): prepare search filters once per query

* test(search): bind benchmark cases without mutable defaults

* test(search): pass benchmark filters explicitly

* fix(search): preserve static subclass filter hooks

* fix(search): preserve bound instance filter overrides

---------

Co-authored-by: machaoxin0407 <221922045+machaoxin0407@users.noreply.github.com>
2026-10-14 18:14:21 +08:00
jinliyl
43ef665595
feat(classifier): add optional Jev component and service demo (#619)
* feat(classifier): add optional Jev component and CLI demo

* fix(classifier): expose Jev demo through inherited default service
2026-10-14 17:49:49 +08:00
Kyubum Hwang
c8a03dbd5a
fix(chunker): restore malformed JSON text fallback (#557)
* fix(chunker): restore malformed JSON text fallback

* refactor(chunker): apply optional review cleanups
2026-10-14 14:43:27 +08:00
Ziyang Guo
a99459ac93
fix(reindex): return a readable summary so Studio stops showing a placeholder (#565)
Some checks are pending
CI / Python packages / Build and verify distributions (push) Waiting to run
CI / Python quality / GitHub Actions (push) Waiting to run
CI / Python tests / Unit Tests - py3.11 (push) Waiting to run
CI / Python tests / Unit Tests - py3.12 (push) Waiting to run
CI / Windows / CLI smoke - py3.11 (push) Waiting to run
CI and Release / Docker / Build and test / amd64 (push) Waiting to run
CI and Release / Docker / Publish multi-platform tags (push) Blocked by required conditions
Security / CodeQL / Analyze javascript-typescript (push) Waiting to run
CI / Python quality / Pre-commit (push) Waiting to run
CI / Python tests / Unit Tests - py3.13 (push) Waiting to run
CI / Python tests / Unit Tests - py3.14 (push) Waiting to run
CI and Release / Docker / Build and test / arm64 (push) Waiting to run
Security / CodeQL / Analyze python (push) Waiting to run
* fix(reindex): return a readable summary so Studio stops showing a placeholder

`ReindexStep` assigned the store's report dict straight to
`response.answer`. ReMe Studio's settings panel already anticipates a string
answer:

    typeof result.answer === "string" && result.answer ? result.answer : t("indexRebuilt")

(`reme_studio/app/settings-center.tsx`), so with a dict answer that branch was
unreachable and every rebuild reported the generic "index rebuilt" text. The
operator never saw which scopes ran or how many entries each indexed.

Render the report as one summary line — `Reindexed scope=all: bm25=7,
embedding=5, tag=2` for the composite scope and `Reindexed scope=bm25: 3
indexed` for a single one. The report itself still lands in metadata, so
programmatic consumers are unaffected, and Studio needs no change because its
string branch now fires.

`reindex` is the only answer-carrying consumer of this step; no test asserted
the answer content, only metadata, so nothing else moved.

* chore(reindex): tighten the renderer and deduplicate the summary test

Review on #565: the helper's docstring carried the Studio background
that already lives in the PR description; the intermediate per_scope
dict collapses into the join generator; and the all-scope delegation
test now uses distinct counts (7, 5, 2) so it alone pins both the exact
summary answer and the full metadata report, replacing the overlapping
summary test.

* refactor(reindex): share summary return formatting

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-10-14 11:25:48 +08:00
Sen Huang
749c653f23
feat(integrations): add automatic memory plugins for Claude Code and Codex (#575)
* feat(integrations): add native Claude Code and Codex memory plugins

* docs(integrations): make plugin installation the coding agent entry point

* feat(integrations): configure plugin connections from user settings

* refactor(codex): use MCP tools and native plugin settings

* fix(codex): restore plugin hooks on current runtimes

* docs(codex): expand setup guide and reserve screenshot slots

* feat(codex): add configurable memory consolidation and status

* feat(codex): merge connection checks into a tabbed status app

* docs(codex): diagnose blocked status panel sandbox loading

* fix(codex): simplify status panel layout for narrow cards

* docs(codex): focus guides on connecting and using ReMe

* docs(codex): specify validated CLI and desktop version requirements

* feat(codex): add ReMe branding to plugin icons

* docs(codex): prepare manual screenshot workflow

* fix(codex): simplify ReMe status layout and activity

* docs(codex): explain Hook loading and local project context

* fix(codex): keep status refresh clear of host close control

* fix(codex): retain status resources for active MCP connections

* docs(codex): add real desktop screenshots to usage guides

* fix(integrations): address memory capture and consolidation review

* fix(codex): preserve capture endpoint during background delivery
2026-10-12 20:12:47 +08:00
jinliyl
e2da221fc4
ci: sync AgentScope pin with QwenPaw (#569) 2026-10-12 17:16:01 +08:00
WQS
c1de31ab2c
feat(auto-memory): save session images with source links (#574)
Some checks failed
CI and Release / Docker / Build and test / amd64 (push) Has been cancelled
CI and Release / Docker / Build and test / arm64 (push) Has been cancelled
CI / Python packages / Build and verify distributions (push) Has been cancelled
CI / Python quality / Pre-commit (push) Has been cancelled
CI / Python quality / GitHub Actions (push) Has been cancelled
CI / Python tests / Unit Tests - py3.11 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.12 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.13 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.14 (push) Has been cancelled
CI / Windows / CLI smoke - py3.11 (push) Has been cancelled
Deploy / Documentation / Build documentation (push) Has been cancelled
Security / CodeQL / Analyze javascript-typescript (push) Has been cancelled
Security / CodeQL / Analyze python (push) Has been cancelled
Deploy / Documentation / deploy (push) Has been cancelled
CI and Release / Docker / Publish multi-platform tags (push) Has been cancelled
* feat(auto-memory): save session images with contextual source links

* fix(auto-memory): validate image provenance before persistence

* refactor(auto-memory): keep image attachment helpers in the memory step

* fix(auto-memory): preserve main session persistence order

* refactor(auto-memory): simplify image reuse and source frontmatter
2026-10-09 17:42:13 +08:00
Ziyang Guo
e4c381c7b1
fix(python-execute): rename the timeout parameter to survive CLI client kwargs (#587)
benchmark.yaml's python_execute job advertised a `timeout` parameter, but
`timeout` is one of the CLI's client-selection kwargs (reme/reme.py pops it
before the payload is sent), so `reme python_execute code=... timeout=300`
never delivered the timeout to the job. The value silently became the HTTP
client's request timeout while the subprocess kept the 60s default, so
benchmark code that needs longer was killed at 60s with no hint why. HTTP
and MCP callers were unaffected.

Rename the parameter to `python_timeout`, mirroring how the shell job
already avoids the same collision with `shell_timeout`. The step prefers
the new name and still accepts the legacy `timeout` context key so
existing benchmark configs keep working. A new guard test fails if any
job in reme/config/*.yaml or a plugin plugin.yaml declares a
client-reserved parameter name again.
2026-10-09 16:49:05 +08:00
pei711
dc12798526
fix(agent): preserve Unicode separators in saved state (#594)
Some checks failed
CI / Python quality / GitHub Actions (push) Waiting to run
CI / Python quality / Pre-commit (push) Waiting to run
CI / Python tests / Unit Tests - py3.11 (push) Waiting to run
CI / Python tests / Unit Tests - py3.12 (push) Waiting to run
CI / Python tests / Unit Tests - py3.13 (push) Waiting to run
CI / Python tests / Unit Tests - py3.14 (push) Waiting to run
CI / Windows / CLI smoke - py3.11 (push) Waiting to run
Security / CodeQL / Analyze javascript-typescript (push) Waiting to run
Security / CodeQL / Analyze python (push) Waiting to run
CI / OpenClaw plugin / Validate OpenClaw plugin (push) Has been cancelled
CI / Python packages / Build and verify distributions (push) Has been cancelled
CI and Release / Docker / Build and test / amd64 (push) Has been cancelled
CI and Release / Docker / Build and test / arm64 (push) Has been cancelled
CI and Release / Docker / Publish multi-platform tags (push) Has been cancelled
Co-authored-by: pei711 <199601925+pei711@users.noreply.github.com>
2026-10-05 11:25:37 +08:00
jinliyl
4c54c2b650
feat(docker): add container deployment and multi-platform release workflow (#582)
Some checks failed
CI / Python packages / Build and verify distributions (push) Waiting to run
CI / Python quality / GitHub Actions (push) Waiting to run
CI / Python quality / Pre-commit (push) Waiting to run
CI / Python tests / Unit Tests - py3.11 (push) Waiting to run
CI / Python tests / Unit Tests - py3.12 (push) Waiting to run
CI / Python tests / Unit Tests - py3.13 (push) Waiting to run
CI / Python tests / Unit Tests - py3.14 (push) Waiting to run
CI / Windows / CLI smoke - py3.11 (push) Waiting to run
Deploy / Documentation / Build documentation (push) Waiting to run
Deploy / Documentation / deploy (push) Blocked by required conditions
CI and Release / Docker / Build and test / amd64 (push) Waiting to run
CI and Release / Docker / Build and test / arm64 (push) Waiting to run
CI and Release / Docker / Publish multi-platform tags (push) Blocked by required conditions
Security / CodeQL / Analyze javascript-typescript (push) Waiting to run
Security / CodeQL / Analyze python (push) Waiting to run
CI / ReMe Studio / Studio checks (push) Has been cancelled
* feat(docker): add container deployment and multi-platform release workflow

* fix(llm): initialize providers on model access and preserve runtime injection

* test(llm): keep rejected model updates independent of credentials
2026-10-03 23:54:58 +08:00
machaoxin0407
49bbbc93ff
perf(bm25): maintain live document length incrementally (#590)
Co-authored-by: machaoxin0407 <221922045+machaoxin0407@users.noreply.github.com>
2026-10-03 22:38:11 +08:00
jinliyl
81c16d3e2c
feat(jobs): add a unified enabled switch for all job types (#580)
* feat(jobs): allow disabling cron schedules independently

* refactor(jobs): unify job activation with enabled flag
2026-09-30 18:30:36 +08:00
WQS
ce386f5391
feat(auto-resource): unify text and image auto-resource using agentwrapper (#555)
* feat(auto_resource): unify text and image agent workflows

* test(auto_resource): streamline coverage and clarify image prompts

* refactor(auto_resource): simplify shared agent interpretation

* test(codex): isolate stdio startup budgets and teardown

* refactor(auto-resource): align image options and wrapper backend checks

* fix(auto-resource): finalize image notes after agent errors

* fix(auto-resource): complete image agent review fixes

* refactor(auto-resource): simplify shared reply failure finalization

* refactor(auto-resource): compose shared resource instructions

* fix(auto-resource): preserve prompt configuration compatibility

* refactor(auto-resource): remove legacy image prompt aliases
2026-09-23 10:20:35 +08:00
WQS
6125fc197d
feat(auto-memory): add session image support (#532)
* feat(auto-memory): add opt-in image input

* feat(auto-memory): caption session images into source-linked notes

* fix(auto-memory): version Pillow 10 compatible image preparation

* fix(auto-memory): harden image evidence and retry boundaries

* fix(auto-memory): preserve image evidence across replay and concurrent writes

Keep persisted image positions through disabled history backfills and transcript filtering. Merge note links atomically, map 16-bit grayscale without clipping, recheck restored caption owners, and reuse unchanged identity metadata during batch publication. Add regressions and document conservative custom-rename behavior.

* refactor(auto-memory): restore main baseline for image modes v2

* refactor(images): share resource caption preprocessing and model calls

* feat(watch): support scoped exclusions for managed session images

* feat(auto-memory): add opt-in resource and caption-only image input

* refactor(auto-memory): keep caption-only mode with text fallback

* refactor(auto-memory): make caption-only mode dispatch explicit

* docs(auto-memory): focus image guide on caption-only mode

* feat(auto-memory): add direct multimodal image extraction

* refactor(auto-memory): route direct images through the vision model

* test(auto-memory): consolidate overlapping image regressions

* feat(auto-memory): interleave direct images with conversation text

* docs(auto-memory): clarify history rendering scope for direct inputs

* feat(auto-memory): require vision declaration for direct-only images

* refactor(auto-memory): keep bound model and native image helpers

* refactor(auto-memory): keep image inputs native and opt-in

* refactor(auto-memory): simplify image docs and tests

* refactor(auto-memory): confine image adaptation to image requests

* refactor(auto-memory): remove image switch type validation
2026-09-20 15:35:00 +08:00
jinliyl
cae613d1c4
refactor(auto-fin): one note per topic, merged by a separate digest step (#558)
* fix(auto-fin): parse topic IDs from fenced JSON replies

* fix(auto-fin): limit report agent tool calls in prompt

* refactor(auto-fin): research news by topic before market open

* fix(config): update default model version for claude_code backend

- Change model version from qwen3.8-max to qwen3.7-plus
- Use environment variable LLM_MODEL_NAME to allow override
- Ensure backend configuration reflects updated model setting

* refactor(auto-fin): write one note per topic before the daily digest

The merge step did two jobs at once: it researched every topic and
combined the results into a single report. Split it the way daily-paper
separates analysis from its brief, so each topic earns a durable note of
its own.

- auto_fin_research_step writes one note per topic that had relevant
  news, tagged `kind: auto-fin-topic` and `topic` in frontmatter
- auto_fin_digest_step merges those notes into the day's brief with no
  tools of its own and appends a `## 主题详解` section linking back to
  each note
- a same-day rerun finds a topic's note by its `topic` frontmatter and
  replaces it in place, deleting the old file when the title changed
- base.py now owns the shared Markdown layer: title sanitizing, report
  normalizing, wikilink validation, note lookup, atomic frontmatter
  writes, and change tracking, so both steps share one write path
- the DingTalk step maps `auto_fin_digest_path` to `markdown_path`
  explicitly instead of relying on whichever step ran last
- drop the unused AutoFinTopicOutput schema and read `job_tools` from
  the step config rather than hardcoding `search`

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(dingtalk): surface rejection details when delivery fails

A failed group send only reported HTTPStatusError, so an operator had to
reproduce the request by hand to learn why DingTalk refused it. Include
the status code and the whitelisted error keys from the response body in
both the log line and the raised RuntimeError.

Only `code`, `message`, and `requestid` are reported: the request body
carries the message content and credentials, so an error response that
echoes it back must not reach the log. Detail is truncated to 200 chars.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(test): make the suite green on CI

- Point the cookbook claude_code model assertion at qwen3.7-plus, the
  default commit 9404e600 set, so the pre-existing red stops blocking
- Satisfy pylint on the auto-fin tests: prefer implicit booleaness for
  the recorded Agent calls and drop an unused tmp_path fixture

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor(auto-fin): name notes after the topic, not the Agent title

The research Agent returned a whole paragraph as its title; that became a
filename and blew past the filesystem's 255-byte name limit, failing with
ENAMETOOLONG inside resolve_note_path. Topics are configured values, so
they are short and predictable - use them for file names and keep the
Agent title in frontmatter.

- Name topic notes after the topic and the digest after the run date
- Fold a byte budget into normalize_title as a safety net for long topics
- Take an AutoFinReportOutput in _write_report instead of loose fields
- Ask both prompts for a short title now that it is display-only

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: isolate per-topic research failures and scope frontmatter reads

A single failing topic used to fail the whole cron job and discard the news
already gathered for the topics that had not run yet -- the 09-19 09:24 run
lost its robot notes that way. Research now logs the failure, continues with
the remaining topics, and only fails the run when no topic produced a note.

frontmatter_read was the only frontmatter step without the _allowed_paths
check that read, write, edit, and frontmatter_update already honour, so an
Agent scoped to one file could still read another file's metadata.

- Isolate per-topic research failures and report them as failed_topics
- Fail loudly when every topic fails so an empty brief is never sent
- Apply _check_path_permission in FrontmatterReadStep

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(auto-fin): keep hand-edited notes and wikilink delimiters from breaking the run

Addresses three review findings on the topic-per-note rework.

- `find_note` and `read_note` now skip a note whose YAML frontmatter does
  not parse. A hand-edited note in the day directory raised
  `yaml.parser.ParserError`, which per-topic isolation surfaced as
  "Auto Fin research failed for every topic" and took the run down with it.
- `normalize_title` also strips `[`, `]` and `#`, which `WikilinkHandler`
  treats as target delimiters. `AI[算力]` used to emit a trailer link the
  parser could not read at all, and `C#` resolved to `.../C` plus an anchor.
- `_write_report` returns the body it actually wrote, and both callers
  propagate it, so the digest answer and the note handed to the digest Agent
  no longer carry links that validation had already downgraded on disk.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-20 14:52:56 +08:00
jinliyl
5231f3970c
fix(hermes): preserve profile context and reset invalid provider state (#554)
Some checks failed
CI / Python packages / Build and verify distributions (push) Has been cancelled
CI / Python quality / GitHub Actions (push) Has been cancelled
CI / Python quality / Pre-commit (push) Has been cancelled
CI / Python tests / Unit Tests - py3.11 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.12 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.13 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.14 (push) Has been cancelled
CI / Windows / CLI smoke - py3.11 (push) Has been cancelled
Deploy / Documentation / Build documentation (push) Has been cancelled
Security / CodeQL / Analyze javascript-typescript (push) Has been cancelled
Security / CodeQL / Analyze python (push) Has been cancelled
Deploy / Documentation / deploy (push) Has been cancelled
* Fix Hermes provider profile thread compatibility and reinitialization

* docs(hermes): clarify compatibility checks were manual
2026-09-16 16:01:15 +08:00
jinliyl
9ebe17a89d
Fix cookbook embedding store config test (#553) 2026-09-16 11:09:50 +08:00
jinliyl
5f2c693ddb
feat(plugins): extract DingTalk integration (#551)
* feat(plugins): extract DingTalk integration

* feat(config): complete cookbook model setup

* docs(config): add cookbook startup guide

* fix(config): clarify DingTalk Claude tool access
2026-09-15 20:05:52 +08:00
jinliyl
dab56fc794
chore: bump ReMe and OpenClaw versions (#549) 2026-09-15 18:29:25 +08:00
jinliyl
b9caae1e50
feat(service): support explicit wildcard network binds (#547)
* feat(service): bind network services to all interfaces

* fix(service): keep network listeners local by default

* fix(service): make remote access explicit
2026-09-15 14:26:33 +08:00
jinliyl
d67f1490f5
fix(dsh): support latest host prerelease (#548) 2026-09-15 13:31:45 +08:00
WQS
6ed97f033b
feat(auto_resource): reuse parsed resource ownership per batch (#529)
* perf(auto_resource): reuse historical ownership lookup per batch

Share a lazy ownership index across routed processors and expire it at the end of each resource batch. Refresh changed days and their safe aliases after writes, deletes, and partial failures.

Add regression coverage for scan counts, ownership, lookup scope isolation, and retry behavior.

* fix(auto_resource): reconcile shared lookup across active batches

* fix(auto_resource): refresh batch ownership from filesystem metadata

* refactor(auto_resource): simplify batch ownership lookup

Keep per-day ownership caches paired with pre-read metadata and refresh changed days before subsequent resources. Remove reverse-index bookkeeping, post-read stabilization retries, and recursive scope fallback.

Handle late ELOOP errors on Python 3.13 while preserving other OS errors. Update regression tests for cross-resource freshness and retain safety and lifecycle coverage.

* refactor(auto_resource): streamline batch lookup refresh

* refactor(auto_resource): scope ownership cache to batch mutations

Replace filesystem snapshots with one lazy historical lookup shared by resource processors. Reload only days changed by this invocation and discard the cache when it ends.

Reuse canonical daily-note scans for symlink safety and Python 3.13 compatibility. Consolidate regression tests around the narrowed batch contract.

Validation: Python 3.11, 3.12 and 3.13 core/plugin suites each passed 1407 tests; pre-commit --all-files and offline lookup/deletion comparison passed.

* refactor(auto_resource): reuse initial ownership without refreshes

* test(auto_resource): streamline ownership lookup regression coverage
2026-09-14 16:55:42 +08:00
jinliyl
46eca95bb9
fix(job): preserve nested exception details (#544) 2026-09-14 16:54:54 +08:00
jinliyl
9ad3dafce5
chore(ci): harden and split workflows (#539)
Some checks failed
CI / DSH plugin / Validate DSH plugin (push) Has been cancelled
CI / OpenClaw plugin / Validate OpenClaw plugin (push) Has been cancelled
CI / Python packages / Build and verify distributions (push) Has been cancelled
CI / Python quality / GitHub Actions (push) Has been cancelled
CI / Python quality / Pre-commit (push) Has been cancelled
CI / Python tests / Unit Tests - py3.11 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.12 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.13 (push) Has been cancelled
CI / Python tests / Unit Tests - py3.14 (push) Has been cancelled
CI / ReMe Studio / Studio checks (push) Has been cancelled
CI / Windows / CLI smoke - py3.11 (push) Has been cancelled
Deploy / Documentation / Build documentation (push) Has been cancelled
Security / CodeQL / Analyze javascript-typescript (push) Has been cancelled
Security / CodeQL / Analyze python (push) Has been cancelled
Deploy / Documentation / deploy (push) Has been cancelled
* chore(ci): harden and split workflows

* fix(ci): support token-based npm publishing

* test: make disappearing resource check portable

* fix(ci): make Studio releases recoverable

* fix(ci): stop Studio publishing on cancellation
2026-09-11 18:21:49 +08:00
Sen Huang
05958d4d8b
feat(evolve): tag auto-dream outputs (#538) 2026-09-11 14:08:29 +08:00
jinliyl
6cb81e7921
refactor(integrations): split TypeScript host plugins (#536)
* refactor(integrations): split TypeScript host plugins

* feat(integrations): refresh host compatibility and status UI

* fix(openclaw): secure status diagnostics
2026-09-11 13:06:02 +08:00
jinliyl
1be61b1e4c
feat(integrations): add dual-mode Hermes memory provider (#533)
* feat(integrations): add dual-mode Hermes memory provider

* style(integrations): apply repository formatting

* fix(integrations): address Hermes provider review

* fix(integrations): bound embedded recall startup cleanup

* fix(integrations): finish embedded application cleanup

* docs(integrations): expand Hermes verification guide
2026-09-11 12:25:18 +08:00
jinliyl
7e25d4679b
feat: auto-tag plugin-generated reports (#534)
* feat(plugins): auto-tag generated reports

* fix(logging): forward host records on Python 3.13

* refactor(tags): decouple auto tagging from index updates

* fix(tags): bind auto tagging to configured index

* fix(tags): preserve standalone default index

* fix(tags): make auto tagging best effort
2026-09-11 11:52:52 +08:00
xyf2020
9975bb37b9
Separate benchmark judge plugins (#535)
Some checks failed
CI / Python tests / Unit Tests - py3.12 (push) Waiting to run
CI / Python tests / Unit Tests - py3.13 (push) Waiting to run
CI / Windows / CLI smoke - py3.11 (push) Waiting to run
Deploy / Documentation / Build documentation (push) Waiting to run
Deploy / Documentation / deploy (push) Blocked by required conditions
CI / Python packages / Build and verify distributions (push) Waiting to run
CI / Python quality / Pre-commit (push) Waiting to run
CI / Python tests / Unit Tests - py3.11 (push) Waiting to run
Security / CodeQL / Analyze javascript-typescript (push) Waiting to run
Security / CodeQL / Analyze python (push) Waiting to run
CI / Documentation / Test and build documentation (push) Has been cancelled
2026-09-10 20:16:46 +08:00
xyf2020
06fb46fa48
feat(tags): add configurable tag indexing and filtered hybrid search (#530)
* Add optional tag generation and normalization to auto memory

* Add tag index components and clean up temporary JSONL files

* Preserve tag index state when reconciliation fails

* Refactor and streamline application implementation

* Fix pylint C1803 warnings in tag normalization tests

* Document optional tag index configuration

* Make tag index failures non-blocking and disable auto-memory tags

* Add configurable tag indexing and tag listing

* Add tag-filtered hybrid search with exact candidate ranking

* Remove obsolete generated files

* Rename tag index key to tag_key and reject reserved fields

* Extract automatic tagging into a dedicated step

* Restrict frontmatter updates to authorized keys

* Refine tag filtering and automatic memory tagging

* Require underscore-separated tags in auto-tag prompts

- Forbid spaces in tags and require underscores (e.g. sam_altman) in both
  English and Chinese auto_tag prompts, with English examples switched to
  English entities (OpenAI, gold)
- Drop prompt-string assertions superseded by the new tagging rule
- Merge construction/runtime tag_key validation tests into one parametrized case

* Make max_tags_per_file configurable in auto-tag step

* Consolidate tag index tests

* Fix search test fixture lint warnings

* Align tag contracts and index health behavior

* Fall back when tag index is unavailable

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-09-10 18:01:09 +08:00