mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-10-01 02:04:21 +00:00
* fix(auto-fin): parse topic IDs from fenced JSON replies
* fix(auto-fin): limit report agent tool calls in prompt
* refactor(auto-fin): research news by topic before market open
* fix(config): update default model version for claude_code backend
- Change model version from qwen3.8-max to qwen3.7-plus
- Use environment variable LLM_MODEL_NAME to allow override
- Ensure backend configuration reflects updated model setting
* refactor(auto-fin): write one note per topic before the daily digest
The merge step did two jobs at once: it researched every topic and
combined the results into a single report. Split it the way daily-paper
separates analysis from its brief, so each topic earns a durable note of
its own.
- auto_fin_research_step writes one note per topic that had relevant
news, tagged `kind: auto-fin-topic` and `topic` in frontmatter
- auto_fin_digest_step merges those notes into the day's brief with no
tools of its own and appends a `## 主题详解` section linking back to
each note
- a same-day rerun finds a topic's note by its `topic` frontmatter and
replaces it in place, deleting the old file when the title changed
- base.py now owns the shared Markdown layer: title sanitizing, report
normalizing, wikilink validation, note lookup, atomic frontmatter
writes, and change tracking, so both steps share one write path
- the DingTalk step maps `auto_fin_digest_path` to `markdown_path`
explicitly instead of relying on whichever step ran last
- drop the unused AutoFinTopicOutput schema and read `job_tools` from
the step config rather than hardcoding `search`
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(dingtalk): surface rejection details when delivery fails
A failed group send only reported HTTPStatusError, so an operator had to
reproduce the request by hand to learn why DingTalk refused it. Include
the status code and the whitelisted error keys from the response body in
both the log line and the raised RuntimeError.
Only `code`, `message`, and `requestid` are reported: the request body
carries the message content and credentials, so an error response that
echoes it back must not reach the log. Detail is truncated to 200 chars.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(test): make the suite green on CI
- Point the cookbook claude_code model assertion at qwen3.7-plus, the
default commit 9404e600 set, so the pre-existing red stops blocking
- Satisfy pylint on the auto-fin tests: prefer implicit booleaness for
the recorded Agent calls and drop an unused tmp_path fixture
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor(auto-fin): name notes after the topic, not the Agent title
The research Agent returned a whole paragraph as its title; that became a
filename and blew past the filesystem's 255-byte name limit, failing with
ENAMETOOLONG inside resolve_note_path. Topics are configured values, so
they are short and predictable - use them for file names and keep the
Agent title in frontmatter.
- Name topic notes after the topic and the digest after the run date
- Fold a byte budget into normalize_title as a safety net for long topics
- Take an AutoFinReportOutput in _write_report instead of loose fields
- Ask both prompts for a short title now that it is display-only
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: isolate per-topic research failures and scope frontmatter reads
A single failing topic used to fail the whole cron job and discard the news
already gathered for the topics that had not run yet -- the 09-19 09:24 run
lost its robot notes that way. Research now logs the failure, continues with
the remaining topics, and only fails the run when no topic produced a note.
frontmatter_read was the only frontmatter step without the _allowed_paths
check that read, write, edit, and frontmatter_update already honour, so an
Agent scoped to one file could still read another file's metadata.
- Isolate per-topic research failures and report them as failed_topics
- Fail loudly when every topic fails so an empty brief is never sent
- Apply _check_path_permission in FrontmatterReadStep
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(auto-fin): keep hand-edited notes and wikilink delimiters from breaking the run
Addresses three review findings on the topic-per-note rework.
- `find_note` and `read_note` now skip a note whose YAML frontmatter does
not parse. A hand-edited note in the day directory raised
`yaml.parser.ParserError`, which per-topic isolation surfaced as
"Auto Fin research failed for every topic" and took the run down with it.
- `normalize_title` also strips `[`, `]` and `#`, which `WikilinkHandler`
treats as target delimiters. `AI[算力]` used to emit a trailer link the
parser could not read at all, and `C#` resolved to `.../C` plus an anchor.
- `_write_report` returns the body it actually wrote, and both callers
propagate it, so the digest answer and the note handed to the digest Agent
no longer carry links that validation had already downgraded on disk.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
264 lines
9.9 KiB
Python
264 lines
9.9 KiB
Python
"""Tests for the frontmatter-only CRUD steps (``frontmatter_read`` /
|
|
``frontmatter_update`` / ``frontmatter_delete``).
|
|
|
|
Covers the suffix-less path contract shared with ``read`` / ``write`` /
|
|
``edit``: a bare path with no suffix auto-appends ``.md``. All three
|
|
siblings must agree on path handling, disclose the substitution via
|
|
``resolved_path`` in the response metadata, and report the actually
|
|
probed path in not-found errors instead of the caller's raw input.
|
|
"""
|
|
|
|
# pylint: disable=protected-access
|
|
|
|
import os
|
|
import tempfile
|
|
from pathlib import Path
|
|
|
|
import frontmatter
|
|
import pytest
|
|
|
|
from reme.components.file_store import LocalFileStore
|
|
from reme.steps.file_io.frontmatter_delete import FrontmatterDeleteStep
|
|
from reme.steps.file_io.frontmatter_read import FrontmatterReadStep
|
|
from reme.steps.file_io.frontmatter_update import FrontmatterUpdateStep
|
|
|
|
|
|
class temp_chdir:
|
|
"""Context manager to temporarily chdir into a path and restore on exit."""
|
|
|
|
def __init__(self, path):
|
|
self.path = path
|
|
self.old = None
|
|
|
|
def __enter__(self):
|
|
self.old = os.getcwd()
|
|
os.chdir(self.path)
|
|
return self
|
|
|
|
def __exit__(self, *exc):
|
|
os.chdir(self.old)
|
|
|
|
|
|
def _seed(workspace: Path, rel: str, body: str) -> Path:
|
|
target = workspace / rel
|
|
target.parent.mkdir(parents=True, exist_ok=True)
|
|
target.write_text(body, encoding="utf-8")
|
|
return target
|
|
|
|
|
|
async def _make_store() -> LocalFileStore:
|
|
store = LocalFileStore(name="t_fm", embedding_store="")
|
|
await store.start()
|
|
return store
|
|
|
|
|
|
async def _run(step_cls, store: LocalFileStore, **kwargs):
|
|
step = step_cls(file_store=store)
|
|
await step(**kwargs)
|
|
return step.context.response
|
|
|
|
|
|
NOTE = "notes/post.md"
|
|
BODY = "---\nname: n\ntags:\n- a\n---\nbody\n"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_read_no_suffix_autoappends_md():
|
|
"""frontmatter_read on a suffix-less path resolves to the ``.md`` file."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
_seed(Path(tmp), NOTE, BODY)
|
|
store = await _make_store()
|
|
resp = await _run(FrontmatterReadStep, store, path="notes/post")
|
|
assert resp.success is True
|
|
assert resp.metadata["frontmatter"] == {"name": "n", "tags": ["a"]}
|
|
assert resp.metadata["path"] == "notes/post"
|
|
assert resp.metadata["resolved_path"] == "notes/post.md"
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_read_honors_injected_path_scope():
|
|
"""An injected allowed-path scope denies frontmatter reads outside of it."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
_seed(Path(tmp), NOTE, BODY)
|
|
store = await _make_store()
|
|
|
|
allowed = await _run(FrontmatterReadStep, store, path=NOTE, _allowed_paths=[NOTE])
|
|
assert allowed.success is True
|
|
assert allowed.metadata["frontmatter"] == {"name": "n", "tags": ["a"]}
|
|
|
|
denied = await _run(FrontmatterReadStep, store, path=NOTE, _allowed_paths=["notes/other.md"])
|
|
assert denied.success is False
|
|
assert denied.metadata["error"] == "no permission to access this file"
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_update_no_suffix_autoappends_md():
|
|
"""frontmatter_update on a suffix-less path resolves to the ``.md`` file."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
note = _seed(Path(tmp), NOTE, BODY)
|
|
store = await _make_store()
|
|
resp = await _run(FrontmatterUpdateStep, store, path="notes/post", metadata={"name": "renamed"})
|
|
assert resp.success is True
|
|
assert resp.metadata["updated"] == {"name": "renamed"}
|
|
assert resp.metadata["resolved_path"] == "notes/post.md"
|
|
assert "name: renamed" in note.read_text(encoding="utf-8")
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_update_honors_injected_frontmatter_key_scope():
|
|
"""Injected key scope rejects the entire update when any key is outside it."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
note = _seed(Path(tmp), NOTE, BODY)
|
|
store = await _make_store()
|
|
|
|
before = note.read_bytes()
|
|
resp = await _run(
|
|
FrontmatterUpdateStep,
|
|
store,
|
|
path=NOTE,
|
|
metadata={"tags": ["new"], "name": "renamed"},
|
|
_allowed_frontmatter_keys=["tags"],
|
|
)
|
|
assert resp.success is False
|
|
assert resp.metadata["error"] == "frontmatter key(s) not allowed: 'name'"
|
|
assert note.read_bytes() == before
|
|
|
|
resp = await _run(
|
|
FrontmatterUpdateStep,
|
|
store,
|
|
path=NOTE,
|
|
metadata={"tags": ["new"]},
|
|
_allowed_frontmatter_keys=["tags"],
|
|
)
|
|
assert resp.success is True
|
|
assert frontmatter.loads(note.read_text(encoding="utf-8")).metadata == {"name": "n", "tags": ["new"]}
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_update_empty_frontmatter_key_scope_denies_all_updates():
|
|
"""An empty injected list is restrictive, unlike an omitted constraint."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
note = _seed(Path(tmp), NOTE, BODY)
|
|
store = await _make_store()
|
|
before = note.read_bytes()
|
|
|
|
resp = await _run(
|
|
FrontmatterUpdateStep,
|
|
store,
|
|
path=NOTE,
|
|
metadata={"tags": ["new"]},
|
|
_allowed_frontmatter_keys=[],
|
|
)
|
|
|
|
assert resp.success is False
|
|
assert note.read_bytes() == before
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_delete_no_suffix_autoappends_md():
|
|
"""frontmatter_delete on a suffix-less path resolves to the ``.md`` file."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
note = _seed(Path(tmp), NOTE, BODY)
|
|
store = await _make_store()
|
|
resp = await _run(FrontmatterDeleteStep, store, path="notes/post", keys=["tags"])
|
|
assert resp.success is True
|
|
assert resp.metadata["deleted"] == ["tags"]
|
|
assert resp.metadata["missing"] == []
|
|
assert resp.metadata["resolved_path"] == "notes/post.md"
|
|
assert "tags" not in note.read_text(encoding="utf-8")
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_family_consistent_on_suffixless_path():
|
|
"""read / update / delete all succeed on the same suffix-less path."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
_seed(Path(tmp), NOTE, BODY)
|
|
store = await _make_store()
|
|
|
|
resp = await _run(FrontmatterReadStep, store, path="notes/post")
|
|
assert resp.success is True
|
|
|
|
resp = await _run(FrontmatterUpdateStep, store, path="notes/post", metadata={"status": "done"})
|
|
assert resp.success is True
|
|
|
|
resp = await _run(FrontmatterDeleteStep, store, path="notes/post", keys=["status"])
|
|
assert resp.success is True
|
|
assert resp.metadata["deleted"] == ["status"]
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_read_missing_reports_probed_path():
|
|
"""not-found errors surface the probed ``.md`` path, not the raw input."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
store = await _make_store()
|
|
resp = await _run(FrontmatterReadStep, store, path="digest/missing")
|
|
assert resp.success is False
|
|
assert resp.metadata["exists"] is False
|
|
assert resp.metadata["path"] == "digest/missing"
|
|
assert resp.metadata["resolved_path"] == "digest/missing.md"
|
|
assert "digest/missing.md" in str(resp.answer)
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_update_missing_reports_probed_path():
|
|
"""frontmatter_update not-found errors mention the probed ``.md`` path."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
store = await _make_store()
|
|
resp = await _run(FrontmatterUpdateStep, store, path="digest/missing", metadata={"x": 1})
|
|
assert resp.success is False
|
|
assert resp.metadata["error"] == "digest/missing.md not found"
|
|
assert resp.metadata["resolved_path"] == "digest/missing.md"
|
|
assert "digest/missing.md" in str(resp.answer)
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_delete_missing_reports_probed_path():
|
|
"""frontmatter_delete not-found errors mention the probed ``.md`` path."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
store = await _make_store()
|
|
resp = await _run(FrontmatterDeleteStep, store, path="digest/missing", keys=["x"])
|
|
assert resp.success is False
|
|
assert resp.metadata["error"] == "digest/missing.md not found"
|
|
assert resp.metadata["resolved_path"] == "digest/missing.md"
|
|
assert "digest/missing.md" in str(resp.answer)
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_non_md_rejected_without_resolved_path():
|
|
"""Non-markdown targets are rejected by all three siblings; no substitution."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
_seed(Path(tmp), "notes/data.txt", "plain\n")
|
|
store = await _make_store()
|
|
|
|
for step_cls, kwargs in (
|
|
(FrontmatterReadStep, {}),
|
|
(FrontmatterUpdateStep, {"metadata": {"x": 1}}),
|
|
(FrontmatterDeleteStep, {"keys": ["x"]}),
|
|
):
|
|
resp = await _run(step_cls, store, path="notes/data.txt", **kwargs)
|
|
assert resp.success is False
|
|
assert "not markdown" in str(resp.answer).lower()
|
|
assert "resolved_path" not in resp.metadata
|
|
await store.close()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_explicit_md_path_has_no_resolved_path():
|
|
"""Explicit ``.md`` paths need no substitution, so no ``resolved_path``."""
|
|
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
|
|
_seed(Path(tmp), NOTE, BODY)
|
|
store = await _make_store()
|
|
resp = await _run(FrontmatterReadStep, store, path=NOTE)
|
|
assert resp.success is True
|
|
assert "resolved_path" not in resp.metadata
|
|
await store.close()
|