refactor(config): streamline job descriptions and parameter docs (#257)
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
Tests ReMe / Unit Tests - py3.10 (push) Has been cancelled
Tests ReMe / Unit Tests - py3.13 (push) Has been cancelled

* refactor(config): streamline job descriptions and parameter docs

- Simplify descriptions for search, traverse, list, read, stat,
  frontmatter:read, write, edit, append, frontmatter:update,
  frontmatter:delete, move, delete, upload, upload_resource, and
  download jobs
- Shorten parameter descriptions to be more concise
- Maintain essential information while reducing verbosity

refactor(steps): rename daily steps and consolidate functionality

- Rename daily_resolve_step to daily_read_step
- Rename daily_create_step to daily_write_step
- Update __init__.py imports to reflect new step names
- Consolidate daily operations documentation

refactor(daily): extract helper functions and improve structure

- Rename _day_index.py to _daily_io.py
- Extract validate_slug function for Windows-safe filename validation
- Move scan_notes function to public interface
- Add comprehensive docstrings explaining slug validation and day-index
  rebuild concerns

feat(daily): decouple list operation from index refresh

- Remove automatic day index refresh from daily_list_step
- Change daily_list_step to pure read operation with no side effects
- Sort notes by slug for stable output
- Update documentation to clarify read/write separation

refactor(daily): remove deprecated create step

- Remove unused daily/create.py module
- Simplify daily operations to focus on CRUD patterns

* refactor(daily): replace module imports with explicit step class imports
This commit is contained in:
Sen Huang 2026-05-25 19:20:50 +08:00 committed by GitHub
parent bb354cc580
commit 83bfddb4a4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 824 additions and 551 deletions

View file

@ -84,7 +84,7 @@ jobs:
# ── Retrieve ───────────────────────────────────────────────────────
- backend: base
name: search
description: "Hybrid search over the vault (vector + BM25 fused via RRF). Returns ranked chunks with optional wikilink expansion."
description: "Hybrid vault search (vector + BM25, RRF-fused)."
parameters:
type: object
properties:
@ -93,11 +93,11 @@ jobs:
description: "search query"
limit:
type: integer
description: "max results to return"
description: "max results"
default: 5
min_score:
type: number
description: "minimum fused score threshold (RRF scores are small; default 0 disables filter)"
description: "min fused score"
default: 0.0
required:
- query
@ -110,20 +110,20 @@ jobs:
- backend: base
name: traverse
description: "Traverse the file graph from one or more seed paths up to N hops, in the chosen direction. Use to chase wikilink neighborhoods after a search hit."
description: "Walk the wikilink graph from a seed path."
parameters:
type: object
properties:
path:
type: string
description: "Seed path (relative to the vault)."
description: "seed path (vault-relative)"
depth:
type: integer
description: "Hop limit (default 1 = immediate neighbors)."
description: "hop limit"
default: 1
direction:
type: string
description: "Edge direction filter: forward / backward / both."
description: "forward / backward / both"
default: both
required:
- path
@ -133,40 +133,40 @@ jobs:
# ── Read Operations ───────────────────────────────────────────────────────────
- backend: base
name: list
description: "List files under a path; optionally recursive, with a result cap. Plain directory walker — no frontmatter parsing. Callers that need frontmatter-based filtering should iterate the result and call `frontmatter:read` per candidate."
description: "List files under a vault path."
parameters:
type: object
properties:
path:
type: string
description: "Directory path (relative to the vault). Empty = vault root."
description: "vault-relative dir; empty = root"
default: ""
recursive:
type: boolean
description: "Recurse into subdirectories."
description: "recurse"
default: false
limit:
type: integer
description: "Max results."
description: "max results"
default: 100
steps:
- backend: list_step
- backend: base
name: read
description: "read a markdown file (relative path under the vault)"
description: "Read a markdown file under the vault."
parameters:
type: object
properties:
path:
type: string
description: "relative path under the vault (no absolute paths); markdown only"
description: "vault-relative path; markdown only"
start_line:
type: integer
description: "Optional, first line to read (1-based, inclusive)"
description: "first line (1-based, inclusive)"
end_line:
type: integer
description: "Optional, last line to read (1-based, inclusive)"
description: "last line (1-based, inclusive)"
required:
- path
steps:
@ -174,13 +174,13 @@ jobs:
- backend: base
name: stat
description: "Stat a file under the vault (size, mtime, exists, is_dir, is_file)."
description: "Stat a vault file (size, mtime, exists, is_dir, is_file)."
parameters:
type: object
properties:
path:
type: string
description: "Path relative to the vault."
description: "vault-relative path"
required:
- path
steps:
@ -188,13 +188,13 @@ jobs:
- backend: base
name: frontmatter:read
description: "Read a markdown file's parsed YAML frontmatter as a dict (body excluded)."
description: "Read a file's YAML frontmatter as a dict."
parameters:
type: object
properties:
path:
type: string
description: "Path relative to the vault."
description: "vault-relative path"
required:
- path
steps:
@ -203,24 +203,22 @@ jobs:
# ── Write Operations──────────────────────────────────────────────────────────
- backend: base
name: write
description: >-
Write (create or overwrite) a markdown file. The front matter is fixed to two string
fields: `name` and `description`. Existing files are overwritten with a system notice.
description: "Write a markdown file (create or overwrite) with name/description frontmatter."
parameters:
type: object
properties:
path:
type: string
description: "relative path under the vault; markdown only."
description: "vault-relative path; markdown only"
name:
type: string
description: "front matter `name` field; short human-readable title of the file."
description: "frontmatter name"
description:
type: string
description: "front matter `description` field; one-line summary of the file."
description: "frontmatter description"
content:
type: string
description: "body content written after the front matter."
description: "body"
required:
- path
- name
@ -231,19 +229,19 @@ jobs:
- backend: base
name: edit
description: "find-and-replace text in a markdown file (replaces every occurrence)"
description: "Find-and-replace in a markdown file (all occurrences)."
parameters:
type: object
properties:
path:
type: string
description: "relative path under the vault; markdown only."
description: "vault-relative path"
old:
type: string
description: "exact text to find."
description: "text to find"
new:
type: string
description: "replacement text."
description: "replacement"
default: ""
required:
- path
@ -254,16 +252,16 @@ jobs:
- backend: base
name: append
description: "append content to the end of an existing markdown file"
description: "Append content to a markdown file."
parameters:
type: object
properties:
path:
type: string
description: "relative path under the vault; markdown only."
description: "vault-relative path"
content:
type: string
description: "content to append."
description: "content to append"
required:
- path
- content
@ -272,16 +270,16 @@ jobs:
- backend: base
name: frontmatter:update
description: "Update YAML frontmatter on an existing markdown file (merge semantics — each entry in `metadata` becomes one frontmatter key; missing keys are inserted, existing keys overwritten). Use for surgical edits to the reserved keys (`name` / `description`) or any caller-defined keys without rewriting the body."
description: "Merge keys into a file's YAML frontmatter."
parameters:
type: object
properties:
path:
type: string
description: "Path relative to the vault."
description: "vault-relative path"
metadata:
type: object
description: "Frontmatter keys to merge — every key/value becomes one frontmatter entry."
description: "keys to merge"
additionalProperties: true
required:
- path
@ -291,16 +289,16 @@ jobs:
- backend: base
name: frontmatter:delete
description: "Delete one or more YAML frontmatter keys on an existing file."
description: "Drop keys from a file's YAML frontmatter."
parameters:
type: object
properties:
path:
type: string
description: "Path relative to the vault."
description: "vault-relative path"
keys:
type: array
description: "Frontmatter keys to remove."
description: "keys to remove"
items:
type: string
required:
@ -312,23 +310,23 @@ jobs:
# ── File Operations (relocate / cross-realm) ──────────────────────────────
- backend: base
name: move
description: "Relocate / rename a file within the vault and (by default) fix every inbound wikilink. Use for promoting a draft (daily/ → digest/) or renaming a slug. retarget=true (default) rewrites [[src_path]] references across the vault to [[dst_path]] after the rename — keeping the knowledge graph consistent without a second tool call. Set retarget=false only when intentionally leaving inbound links dangling (e.g. moving a file aside to delete next). For cross-realm transfer use upload / download."
description: "Move / rename a vault file; rewrites inbound wikilinks by default."
parameters:
type: object
properties:
src_path:
type: string
description: "Source path (relative to the vault)."
description: "vault-relative source"
dst_path:
type: string
description: "Destination path (relative to the vault) with a directory component."
description: "vault-relative destination"
overwrite:
type: boolean
description: "Overwrite if dst_path exists."
description: "overwrite if dst exists"
default: false
retarget:
type: boolean
description: "Rewrite inbound wikilinks [[src_path]] → [[dst_path]] after the move (across the vault). Set to false to leave links dangling."
description: "rewrite [[src]] → [[dst]] across the vault"
default: true
required:
- src_path
@ -338,13 +336,13 @@ jobs:
- backend: base
name: delete
description: "Hard-delete a file or folder under the vault and report every inbound wikilink that pointed at the doomed targets. The delete itself is unconditional — the inbound list (path + count, literal full-path matching only; sources inside the doomed folder are filtered out) is returned so the agent can decide what to do about each surviving reference (edit the citing prose, point it at a replacement, or accept it as dangling)."
description: "Delete a vault file or folder; returns surviving inbound wikilinks."
parameters:
type: object
properties:
path:
type: string
description: "Path relative to the vault. File or directory."
description: "vault-relative path"
required:
- path
steps:
@ -352,19 +350,19 @@ jobs:
- backend: base
name: upload
description: "Copy a file INTO the vault from the local filesystem. Symmetric counterpart to `download`: source is on the local host, target is vault-relative. `dst_path` is required and must include a directory component so the caller is always explicit about where in the vault the file lands. For the passive-ingest channel-tagged resource bucket (resource/<date>/ with provenance metadata), use `upload_resource` instead."
description: "Copy a host file into the vault at an explicit destination."
parameters:
type: object
properties:
src_path:
type: string
description: "Absolute host filesystem path to the file to copy in."
description: "host absolute path"
dst_path:
type: string
description: "Vault-relative destination path (must include a directory component)."
description: "vault-relative destination"
overwrite:
type: boolean
description: "Overwrite if dst_path already exists."
description: "overwrite if dst exists"
default: false
required:
- src_path
@ -374,22 +372,22 @@ jobs:
- backend: base
name: upload_resource
description: "Land an externally-received asset into resource/<today>/ (today = local date at call time), alongside a meta.json row capturing provenance and a regenerated <date>.md index view. This is the passive-ingest entry point for external channels (wechat / email / browser / api / ...). Do NOT use it for materials the agent actively fetches or generates inside a daily task — those belong inlined inside the daily note daily/<date>/<slug>.md. The bucket file name is derived as `<channel>__<HHMMSS>__<source-basename>` (callers cannot override); source basenames with path separators, dot segments, or a leading '.' are rejected. Returns {date, name, path}. A duplicate (same channel + same second + same source basename) returns an error — the step never silently dedupes."
description: "Ingest an external-channel asset into resource/<today>/ with provenance."
parameters:
type: object
properties:
path:
type: string
description: "Source path on the local filesystem; its basename becomes the trailing component of the bucket file name."
description: "host source path"
channel:
type: string
description: "Inbound channel identifier (wechat / email / browser / api / ...). Lowercase letters / digits / dashes only; must start with a letter or digit."
description: "channel id (wechat / email / browser / api / ...)"
description:
type: string
description: "Analysis hint for downstream agents: where the asset came from, what kind of content it carries, and how it should be interpreted (skim vs. deep parse, structured extraction vs. summarization, ...). The digester reads this verbatim from meta.json to decide how to process the asset, so write enough detail to drive that decision — not just a title. Multi-line is fine; the <date>.md bullet view flattens for display while meta.json preserves the original."
description: "what the asset is and how to interpret it"
metadata:
type: object
description: "Optional extras persisted on the meta.json row. `source` (free-form origin within the channel — group name, sender, URL, ...) is conventional; any other keys pass through verbatim. Keys `name`, `channel`, `received_at`, `description` are reserved."
description: "extra provenance keys (e.g. source)"
default: {}
required:
- path
@ -400,95 +398,101 @@ jobs:
- backend: base
name: download
description: "Copy a file OUT of the vault to the local filesystem. Use to hand materials from the vault to local tools (browser, viewer). `dst_path` empty → land in a session-scoped temp file and return the realized path."
description: "Copy a vault file out to the host filesystem."
parameters:
type: object
properties:
src_path:
type: string
description: "File inside the vault (vault-relative)."
description: "vault-relative source"
dst_path:
type: string
description: "Absolute host filesystem path to write to. Empty = session-scoped temp file (the realized path is returned)."
description: "host absolute dest; empty = temp file"
default: ""
overwrite:
type: boolean
description: "Overwrite if dst_path already exists."
description: "overwrite if dst exists"
default: false
required:
- src_path
steps:
- backend: download_step
# ── Daily Operations(note genesis + day-index rollup) ───────────────────
# Note body edits use generic file + frontmatter primitives once the
# note exists.
# ── Daily Operations (note CRUD + day-index rollup) ───────────────────
- backend: base
name: daily:list
description: "List the notes under a single day AND rebuild `daily/<date>.md` as a side effect (idempotent — the freshly-rendered note inventory is exactly what callers want to read). Returns {date, notes: [{path, name, description}, ...]} — one row per `daily/<date>/<slug>.md` note file with vault-relative path / name / description (frontmatter-parsed). Read view of the same rebuild that `daily:reindex` exposes from the write side."
parameters:
type: object
properties:
date:
type: string
description: "ISO date (YYYY-MM-DD). Empty = today."
default: ""
steps:
- backend: daily_list_step
- backend: base
name: daily:resolve
description: "Ensure the day folder daily/<today>/ exists and return the vault-relative path daily/<today>/<name>.md. Pure path-shape helper — does NOT create the note file (use daily:create or file_write for that). Name must be valid as a filename on Windows (no reserved chars < > : \" / \\ | ? *, no reserved device names CON / PRN / AUX / NUL / COM1-9 / LPT1-9, no trailing '.' or whitespace). Idempotent: when the note file already exists returns {exists: true, message: ...} so the caller knows to read-modify rather than overwrite."
parameters:
type: object
properties:
name:
type: string
description: "Note slug (the .md file's stem). Must satisfy Windows filename rules."
required:
- name
steps:
- backend: daily_resolve_step
- backend: base
name: daily:create
description: "Create the note file daily/<date>/<slug>.md with a minimal `name` frontmatter, then refresh the day index daily/<date>.md. Idempotent — if the note file already exists it is left untouched (caller should read-modify rather than overwrite); the index is still refreshed because sibling notes may have changed. Returns {date, slug, path, created, index}."
name: daily:read
description: "Read daily/<date>/<slug>.md (body + frontmatter)."
parameters:
type: object
properties:
slug:
type: string
description: "Note slug (the .md file's stem under daily/<date>/)."
body:
type: string
description: "Initial body content of the note file. Empty leaves the file as frontmatter-only."
default: ""
description: "note slug"
date:
type: string
description: "ISO date (YYYY-MM-DD). Empty = today."
description: "ISO date; empty = today"
default: ""
name:
required:
- slug
steps:
- backend: daily_read_step
- backend: base
name: daily:write
description: "Write daily/<date>/<slug>.md (body + frontmatter); refreshes the day index."
parameters:
type: object
properties:
slug:
type: string
description: "Reserved-field name written to frontmatter. Empty falls back to `slug`."
description: "note slug"
body:
type: string
description: "note body"
default: ""
frontmatter:
type: object
description: "frontmatter dict; defaults to {name: <slug>}"
default: {}
date:
type: string
description: "ISO date; empty = today"
default: ""
overwrite:
type: boolean
description: "false = skip if exists; true = replace"
default: false
refresh_index:
type: boolean
description: "Refresh daily/<date>.md after the write. Set false for batch flows that will reindex at the end."
description: "refresh daily/<date>.md after write"
default: true
required:
- slug
steps:
- backend: daily_create_step
- backend: daily_write_step
- backend: base
name: daily:reindex
description: "Rebuild the day-index page daily/<date>.md from the current set of notes under that date. The day index is a derived artifact; this is the standalone writer — call it once after a batch of note mutations (resolve / write / frontmatter:update), or for historical backfill and drift recovery. Idempotent and safe to re-run. Returns {date, path, created, notes_count} — the write view (was the index page just created? how many notes were swept in?). For the per-note inventory use `daily:list` (which also triggers this rebuild)."
name: daily:list
description: "List notes under a single day."
parameters:
type: object
properties:
date:
type: string
description: "ISO date (YYYY-MM-DD). Empty = today."
description: "ISO date; empty = today"
default: ""
steps:
- backend: daily_list_step
- backend: base
name: daily:reindex
description: "Rebuild the day-index page daily/<date>.md."
parameters:
type: object
properties:
date:
type: string
description: "ISO date; empty = today"
default: ""
steps:
- backend: daily_reindex_step

View file

@ -28,7 +28,7 @@ from . import crud # noqa: F401 -- registers list/stat/upload/download/move/de
from . import frontmatter # noqa: F401 -- registers frontmatter_read_step/update/delete
from . import (
daily,
) # noqa: F401 -- registers daily_resolve_step / daily_create_step / daily_list_step / daily_reindex_step
) # noqa: F401 -- registers daily_read_step / daily_write_step / daily_list_step / daily_reindex_step
from . import background # noqa: F401
# from . import jobs # noqa: F401 -- registers synchronizer / digester

View file

@ -1,44 +1,44 @@
"""Daily-aware steps — note + day-level index, on top of generic file ops.
"""Daily-aware steps — CRUD on note md + day-level index.
A daily note is the single file ``daily/<YYYY-MM-DD>/<name>.md``.
A daily note is the single file ``daily/<YYYY-MM-DD>/<slug>.md``.
The day-level index ``daily/<YYYY-MM-DD>.md`` aggregates that day's
notes into a richer overview page: note list with name/description.
notes into a richer overview page (note list with name / description).
The index is a derived artifact its source of truth lives in each
note's frontmatter and outlinks; refreshes are idempotent and preserve
manual annotations in marker-delimited sections.
note's frontmatter; refreshes are idempotent and preserve manual
annotations in marker-delimited sections.
Tool boundary. The daily module exposes only the operations whose shape
is note- or day-specific:
Tool boundary. The daily module exposes only the operations whose
shape is note- or day-specific:
* ``daily_resolve_step`` note path resolver: ensures the day
folder ``daily/<today>/`` exists and returns the vault-relative
path ``daily/<today>/<name>.md``. Pure path-shape helper no
body, frontmatter, or index writes (those go through the generic
CRUD + reindex steps).
* ``daily_create_step`` write the note stub ``daily/<date>/<slug>.md``
with a minimal ``name`` frontmatter and refresh the day index.
Idempotent: existing file is left untouched; the index still
refreshes (cheap self-healing).
* ``daily_list_step`` list the notes under a single day
(defaults to today); returns ``{date, notes: [{path, name,
description}, ...]}``. Also rebuilds ``daily/<date>.md`` as a side
effect (idempotent the freshly-rendered inventory is what callers
want). Read view of the same operation ``daily_reindex_step`` exposes
from the write side.
* ``daily_reindex_step`` explicit, idempotent rebuild of a day's index
(historical backfill, drift recovery, batch-create reindex). Returns
the write-result fields ``{date, path, created, notes_count}``.
* ``daily_read_step`` read a note by ``slug + date``; returns the
body in ``answer`` and the parsed frontmatter as a dict in metadata,
so callers skip a separate ``frontmatter_read`` round-trip.
* ``daily_write_step`` write the full body + frontmatter for
``daily/<date>/<slug>.md`` in one shot. Validates the slug, mkdirs
the day folder, refreshes the day index. ``mode="create"`` (default)
is idempotent skip-if-exists; ``mode="overwrite"`` is unconditional.
* ``daily_list_step`` pure read of the notes under a single day
(defaults to today); returns ``{date, notes: [{path, slug, name,
description}, ...]}``. Does **not** touch the day index call
``daily_reindex`` explicitly when the rollup page needs rebuilding.
* ``daily_reindex_step`` explicit idempotent rebuild of a day's
index (historical backfill, drift recovery, batch-write reindex).
Body reads / writes / appends / overwrites all go through the generic
``file_read`` / ``file_write`` tools. Frontmatter edits go through
``property:update``. The day-index is rebuilt explicitly via
``daily_reindex`` after a batch of mutations.
Body mid-edits / appends / arbitrary-path reads go through the generic
``read`` / ``write`` / ``append`` / ``edit`` steps. Frontmatter slice
mutations go through ``frontmatter_update`` / ``frontmatter_delete``.
The day-index is rebuilt explicitly via ``daily_reindex`` after a
batch of mutations.
"""
# Module name 'list' mirrors its tool name.
# pylint: disable=redefined-builtin
from .read import DailyReadStep
from .write import DailyWriteStep
from .list import DailyListStep
from .reindex import DailyReindexStep
from . import resolve # noqa: F401 -- @R.register("daily_resolve_step")
from . import create # noqa: F401 -- @R.register("daily_create_step")
from . import list # noqa: F401 -- @R.register("daily_list_step")
from . import reindex # noqa: F401 -- @R.register("daily_reindex_step")
__all__ = [
"DailyReadStep",
"DailyWriteStep",
"DailyListStep",
"DailyReindexStep",
]

View file

@ -1,33 +1,22 @@
"""``_day_index`` — internal helper: build/refresh ``daily/<date>.md`` index page.
"""Internal helpers for daily-aware steps — slug validation + day-index rebuild.
The day index is a derived artifact whose single job is **daily-note
consolidation** its source of truth lives in each note's
frontmatter. This module rebuilds the auto-managed sections of the
index page while preserving any manual content the user has added
between markers.
Two related concerns, both private to the ``daily`` package:
Frontmatter shape only the two reserved fields::
1. **Slug naming** Windows-safe filename validation for the slug
that becomes the stem of ``daily/<YYYY-MM-DD>/<slug>.md``.
2. **Day-index** the derived rollup page ``daily/<YYYY-MM-DD>.md``
listing every note under that date with name + description. The
index is auto-managed in marker-delimited sections; user-edited
manual sections are preserved verbatim across refreshes.
name: <date>
description: <one-line note-count digest>
Public entry points:
The note inventory lives in the body's ``<!-- notes:auto -->``
wikilinks (graph edges feed off them). No bespoke status / lifecycle
/ scope / role / source / created axes those are user-defined and
intentionally absent from the auto-managed payload.
Body auto sections (rebuilt on every refresh, marker-delimited):
* ``notes`` bulleted list of ``[[link]]\\n name description`` rows
Manual sections live outside the auto markers and are preserved verbatim
across refreshes. A fresh day file gets a ``## 备忘`` section seeded as
the manual scratch area.
Entry point: ``refresh_day_index(file_store, date)`` idempotent, safe
to call after every note mutation. ``daily_reindex_step`` exposes
it as a standalone tool; orchestrators (synchronizer, batch flows) call
it explicitly after they finish writing.
* :func:`validate_slug` return an error string, or ``None`` when the
slug is safe to use as a filename.
* :func:`scan_notes` walk ``<daily_dir>/<date>/*.md`` and pull each
note's reserved frontmatter (``name`` / ``description``).
* :func:`refresh_day_index` rebuild ``<daily_dir>/<date>.md`` from
the current state of its notes. Idempotent, safe to re-run.
"""
import re
@ -35,6 +24,73 @@ from pathlib import Path
import frontmatter
# ---------------------------------------------------------------------------
# Slug validation
# ---------------------------------------------------------------------------
_INVALID_CHARS = re.compile(r'[<>:"/\\|?*\x00-\x1f]')
_RESERVED_NAMES = {
"CON",
"PRN",
"AUX",
"NUL",
*(f"COM{i}" for i in range(1, 10)),
*(f"LPT{i}" for i in range(1, 10)),
}
def validate_slug(slug: str) -> str | None:
"""Return an error message, or ``None`` when ``slug`` is a safe filename.
Rules (Windows is the strictest filesystem, so we validate to its bar):
- non-empty, no leading / trailing whitespace
- no reserved characters: ``< > : " / \\ | ? *`` or control chars (``\\x00-\\x1f``)
- no reserved device names: ``CON`` / ``PRN`` / ``AUX`` / ``NUL`` /
``COM1-9`` / ``LPT1-9`` (Windows reserves these with or without an
extension ``CON.txt`` is also forbidden)
- no trailing ``.``
"""
if not slug:
return "slug is required"
if slug != slug.strip():
return f"slug cannot have leading or trailing whitespace: {slug!r}"
if _INVALID_CHARS.search(slug):
return f'slug contains invalid characters (one of < > : " / \\ | ? * ' f"or a control char): {slug!r}"
if slug.endswith("."):
return f"slug cannot end with '.': {slug!r}"
if slug.split(".", 1)[0].upper() in _RESERVED_NAMES:
return f"slug is a Windows-reserved device name: {slug!r}"
return None
# ---------------------------------------------------------------------------
# Day-index rebuild
# ---------------------------------------------------------------------------
#
# The day index is a derived artifact whose single job is **daily-note
# consolidation** — its source of truth lives in each note's
# frontmatter. The rebuild refreshes auto-managed sections while
# preserving any manual content the user has added between markers.
#
# Frontmatter shape — only the two reserved fields::
#
# name: <date>
# description: <one-line note-count digest>
#
# The note inventory lives in the body's ``<!-- notes:auto -->``
# wikilinks (graph edges feed off them). No bespoke status / lifecycle
# / scope / role / source / created axes — those are user-defined and
# intentionally absent from the auto-managed payload.
#
# Body auto sections (rebuilt on every refresh, marker-delimited):
#
# * ``notes`` — bulleted list of ``[[link]]\n name — description`` rows
#
# Manual sections live outside the auto markers and are preserved
# verbatim across refreshes. A fresh day file gets a ``## 备忘``
# section seeded as the manual scratch area.
# Marker syntax: HTML comments so they're invisible in rendered markdown
# but trivially detectable in source. Each block has a paired open/close.
_BLOCK_NAMES = ("notes",)
@ -67,7 +123,7 @@ def _count_digest(n: int) -> str:
return f"今日 {n} 篇笔记。"
def _scan_notes(vault_dir: Path, date: str, daily_dir: str) -> list[dict]:
def scan_notes(vault_dir: Path, date: str, daily_dir: str) -> list[dict]:
"""Walk ``<daily_dir>/<date>/*.md`` and pull each note's frontmatter.
Returns one dict per note::
@ -86,7 +142,7 @@ def _scan_notes(vault_dir: Path, date: str, daily_dir: str) -> list[dict]:
slug = md_path.stem
try:
post = frontmatter.loads(md_path.read_text(encoding="utf-8"))
except Exception:
except Exception: # pylint: disable=broad-except
continue
meta = post.metadata or {}
out.append(
@ -140,14 +196,11 @@ def _replace_or_append(body: str, name: str, fresh_block: str) -> str:
pattern = _block_re(name)
if pattern.search(body):
replacement = f"{_BLOCK_OPEN.format(name=name)}\n" f"{fresh_block}\n" f"{_BLOCK_CLOSE.format(name=name)}"
# Preserve the heading the user had (if any) by only swapping
# the marker-wrapped portion.
return pattern.sub(
lambda m: (m.group("heading") or "") + replacement,
body,
count=1,
)
# Not present — append the canonical heading + block at the tail.
suffix = _wrap_block(name, fresh_block)
return f"{body.rstrip()}\n\n{suffix}\n" if body.strip() else f"{suffix}\n"
@ -218,11 +271,10 @@ async def refresh_day_index(file_store, date: str, daily_dir: str = "daily") ->
vault_dir = Path(file_store.vault_path or ".").resolve()
index_rel = f"{daily_dir}/{date}.md"
index_abs = vault_dir / index_rel
notes = _scan_notes(vault_dir, date, daily_dir)
notes = scan_notes(vault_dir, date, daily_dir)
notes_payload = [{"path": n["path"], "name": n["name"], "description": n["description"]} for n in notes]
# Nothing to index and no prior index file — quietly do nothing.
if not notes and not index_abs.is_file():
return {
"date": date,

View file

@ -1,84 +0,0 @@
"""``daily_create`` — create a daily note file + refresh the day index.
A daily note is the single file ``daily/<YYYY-MM-DD>/<slug>.md``.
The day-level index ``daily/<YYYY-MM-DD>.md`` is also refreshed so
all index views (navigation, search, distill input) reflect the new
note.
This step bakes the conventions an agent shouldn't have to memorize
on every call:
- path template (``date + slug daily/<date>/<slug>.md``)
- reserved-field default: ``name`` falls back to ``slug``
- day-index refresh so ``daily/<date>.md`` stays consistent
Frontmatter is intentionally minimal: only the reserved ``name``
field is written by default. Anything else (status, lifecycle, scope,
role, created, ...) is user-defined supply it via the generic
``property:update`` step after creation.
It does NOT do content R-M-W. Once the note exists, the agent
uses ``file_write`` for body edits and ``property:update`` for
frontmatter tweaks.
Idempotent: when the note file already exists, returns
``{created: False, ...}`` without touching it. The day index is still
refreshed because sibling notes may have changed since the last
call keeping the index in sync is cheap and self-healing. Pass
``refresh_index=False`` to skip the refresh (rare; mostly for tests /
batch-create flows where the caller will refresh once at the end).
"""
from datetime import date as _date
from pathlib import Path
import frontmatter
from ._day_index import refresh_day_index
from ..base_step import BaseStep
from ...components import R
@R.register("daily_create_step")
class DailyCreateStep(BaseStep):
"""Create the note file ``daily/<date>/<slug>.md`` (idempotent); refresh day index."""
async def execute(self):
assert self.context is not None
slug: str = self.context.get("slug", "") or ""
body: str = self.context.get("body", "") or ""
day: str = self.context.get("date") or _date.today().isoformat()
name: str = self.context.get("name", "") or ""
refresh_index: bool = bool(self.context.get("refresh_index", True))
assert slug, "slug is required"
daily_dir = self.app_context.app_config.daily_dir if self.app_context is not None else "daily"
path_rel = f"{daily_dir}/{day}/{slug}.md"
vault_dir = Path(self.file_store.vault_path or ".")
path_abs = (vault_dir / path_rel).resolve()
# Idempotent: existing note returns "already exists" — caller
# decides whether to edit (file_read + file_write) or skip.
already_existed = path_abs.is_file()
if not already_existed:
path_abs.parent.mkdir(parents=True, exist_ok=True)
post = frontmatter.Post(body, name=name or slug)
path_abs.write_text(frontmatter.dumps(post), encoding="utf-8")
payload: dict = {
"date": day,
"slug": slug,
"path": path_rel,
"created": not already_existed,
}
# Refresh even on the idempotent path: sibling notes may have
# changed since the last call and the index should track.
if refresh_index:
payload["index"] = await refresh_day_index(self.file_store, day, daily_dir)
self.context.response.success = True
verb = "Created" if not already_existed else "Reused existing"
self.context.response.answer = f"{verb} daily note {path_rel}"
self.context.response.metadata.update(payload)

View file

@ -1,25 +1,21 @@
"""``daily_list`` — list the notes under a single day.
"""``daily_list`` — list the notes under a single day (pure read, no side effects).
Always rebuilds the day index ``daily/<date>.md`` as a side effect (the
freshly-rendered note inventory is exactly what the caller is asking
to see), then returns one row per ``daily/<date>/<slug>.md`` note
file with its vault-relative ``path`` plus ``name`` / ``description``
from frontmatter.
Returns one row per ``daily/<date>/<slug>.md`` note file with its
vault-relative ``path`` plus ``slug`` / ``name`` / ``description``
from frontmatter. Sorted by slug for stable output.
Distinct from :mod:`daily_reindex` even though both call
``refresh_day_index``: this one is the read view (consumers want the
note inventory), so the index-page bookkeeping fields (``path`` of
``daily/<date>.md``, ``created``) are stripped from the response;
``daily_reindex`` is the write view (consumers want to know what was
rebuilt) and returns those fields without the per-note list.
**Does NOT refresh** ``daily/<date>.md`` call ``daily_reindex``
explicitly when the index page needs to be rebuilt. Decoupling
read from write keeps each step's effect predictable.
Input is a single optional ``date`` (ISO ``YYYY-MM-DD``); falls back to
today.
Input is a single optional ``date`` (ISO ``YYYY-MM-DD``); falls back
to today.
"""
from datetime import date as _date
from pathlib import Path
from ._day_index import refresh_day_index
from ._daily_io import scan_notes
from ..base_step import BaseStep
from ...components import R
@ -27,19 +23,25 @@ from ...components import R
@R.register("daily_list_step")
class DailyListStep(BaseStep):
"""List the notes under a single day; also refreshes ``daily/<date>.md``."""
"""List the notes under a single day. Pure read — no index refresh."""
async def execute(self):
assert self.context is not None
day: str = (self.context.get("date") or "").strip() or _date.today().isoformat()
daily_dir = self.app_context.app_config.daily_dir if self.app_context is not None else "daily"
refreshed = await refresh_day_index(self.file_store, day, daily_dir)
if "error" in refreshed:
self.context.response.success = False
self.context.response.answer = f"Error: {refreshed['error']}"
self.context.response.metadata.update(refreshed)
return
notes = refreshed["notes"]
vault_dir = Path(self.file_store.vault_path or ".").resolve()
scanned = scan_notes(vault_dir, day, daily_dir)
notes = [
{
"path": n["path"],
"slug": n["slug"],
"name": n["name"],
"description": n["description"],
}
for n in scanned
]
self.context.response.success = True
self.context.response.answer = f"Listed {len(notes)} note(s) for {refreshed['date']}"
self.context.response.metadata.update({"date": refreshed["date"], "notes": notes})
self.context.response.answer = f"Listed {len(notes)} note(s) for {day}"
self.context.response.metadata.update({"date": day, "notes": notes})

91
reme4/steps/daily/read.py Normal file
View file

@ -0,0 +1,91 @@
"""``daily_read`` — read a daily note by slug + date; return body + parsed frontmatter.
Convenience wrapper around the generic ``read`` step for the
``daily/<YYYY-MM-DD>/<slug>.md`` path shape. The value over a raw
``file_read`` is two-fold:
* slug validation (Windows-safe filename rules) up front
* frontmatter parsed into a dict in metadata, so callers don't need a
separate ``frontmatter_read`` round-trip
Inputs:
slug (required, validated)
date (default today, ISO ``YYYY-MM-DD``)
Outputs:
answer = note body (frontmatter stripped)
metadata = {date, slug, path, exists, frontmatter: dict}
For arbitrary-path reads or ranged reads use the generic ``read`` step.
"""
from datetime import date as _date
import frontmatter
from ._daily_io import validate_slug
from ..crud._file_io import read_file_safe
from ..base_step import BaseStep
from ...components import R
@R.register("daily_read_step")
class DailyReadStep(BaseStep):
"""Read ``daily/<date>/<slug>.md`` → body + parsed frontmatter."""
def _fail(self, message: str, **meta) -> None:
assert self.context is not None
self.context.response.success = False
self.context.response.answer = f"Error: {message}"
if meta:
self.context.response.metadata.update(meta)
async def execute(self):
assert self.context is not None
slug: str = self.context.get("slug", "") or ""
day: str = (self.context.get("date") or "").strip() or _date.today().isoformat()
err = validate_slug(slug)
if err:
self._fail(err)
return None
daily_dir = self.app_context.app_config.daily_dir if self.app_context is not None else "daily"
path_rel = f"{daily_dir}/{day}/{slug}.md"
path_abs = (self.vault_path / path_rel).resolve()
if not path_abs.is_file():
self._fail(
f"note {path_rel} does not exist",
date=day,
slug=slug,
path=path_rel,
exists=False,
)
return None
try:
text = await read_file_safe(path_abs)
except Exception as e: # pylint: disable=broad-except
self._fail(f"read failed: {e}", date=day, slug=slug, path=path_rel)
return None
post = frontmatter.loads(text)
body = post.content
meta = dict(post.metadata or {})
self.context.response.success = True
self.context.response.answer = body
self.context.response.metadata.update(
{
"date": day,
"slug": slug,
"path": path_rel,
"exists": True,
"frontmatter": meta,
},
)
self.logger.info(
f"[{self.name}] read path={path_rel} " f"bytes={len(body.encode('utf-8'))} fm_keys={list(meta)}",
)
return self.context.response

View file

@ -1,18 +1,17 @@
"""``daily_reindex_step`` — rebuild ``daily/<date>.md`` from its notes.
The day index ``daily/<date>.md`` is a derived artifact whose job is to
list and describe every note file under ``daily/<date>/``. It is **not**
auto-refreshed ``daily_resolve`` (path resolver), ``file_write`` and
``frontmatter_update`` all leave it stale. This step is the standalone
writer that rebuilds it for batch flows (historical backfill, drift
recovery, end-of-batch consolidation).
list and describe every note file under ``daily/<date>/``. It is auto-
refreshed by ``daily_write`` after every body write. Generic ops like
``file_write`` / ``file_append`` / ``frontmatter_update`` leave it
stale this step is the standalone writer to call after batch flows
(historical backfill, drift recovery, end-of-batch consolidation, or
a ``frontmatter_update`` that touched ``name`` / ``description``).
The same rebuild also runs as a side effect of :mod:`daily_list`. The
two steps differ in their response: this one is the **write view**
it reports the index-page path and a ``created`` flag (true when the
file was just emitted for the first time), which is what a caller
running a rebuild wants to confirm. ``daily_list`` is the **read view**
and returns the per-note inventory instead.
This is the **write view**: it reports the index-page path and a
``created`` flag (true when the file was just emitted for the first
time), which is what a caller running a rebuild wants to confirm. For
the per-note inventory use ``daily_list``.
Input is a single optional ``date`` (ISO ``YYYY-MM-DD``); falls back to
today.
@ -22,7 +21,7 @@ Always idempotent and safe to re-run.
from datetime import date as _date
from ._day_index import refresh_day_index
from ._daily_io import refresh_day_index
from ..base_step import BaseStep
from ...components import R

View file

@ -1,91 +0,0 @@
"""``daily_resolve`` — resolve a daily note path; ensure the parent day folder exists.
A daily note is a single markdown file ``daily/<YYYY-MM-DD>/<name>.md``.
This step validates ``name``, makes sure the day folder ``daily/<YYYY-MM-DD>/``
exists (so a subsequent ``file_write`` succeeds), and returns the
vault-relative path to the note file.
Input is a single ``name`` (the note slug). It must be safe to use as a
filename on all platforms Windows is the strictest, so we validate
against its rules:
- no reserved characters: ``< > : " / \\ | ? *`` or control chars (``\\x00-\\x1f``)
- no reserved device names: ``CON``, ``PRN``, ``AUX``, ``NUL``, ``COM1-9``, ``LPT1-9``
- no trailing ``.`` or whitespace
- no leading/trailing whitespace
- non-empty
Idempotent: returns ``{exists: True}`` when the note file already
exists (caller should read-modify rather than overwrite); otherwise
``{exists: False}`` the file itself is **not** created here, use
``daily_create`` or ``file_write`` for that.
"""
import re
from datetime import date as _date
from pathlib import Path
from ..base_step import BaseStep
from ...components import R
_INVALID_CHARS = re.compile(r'[<>:"/\\|?*\x00-\x1f]')
_RESERVED_NAMES = {
"CON",
"PRN",
"AUX",
"NUL",
*(f"COM{i}" for i in range(1, 10)),
*(f"LPT{i}" for i in range(1, 10)),
}
@R.register("daily_resolve_step")
class DailyResolveStep(BaseStep):
"""Ensure ``daily/<today>/`` exists; return the vault-relative path to ``<name>.md``."""
async def execute(self):
assert self.context is not None
name: str = self.context.get("name", "") or ""
err: str | None = None
if not name:
err = "name is required"
elif name != name.strip():
err = f"name cannot have leading or trailing whitespace: {name!r}"
elif _INVALID_CHARS.search(name):
err = f'name contains invalid characters (one of < > : " / \\ | ? * or a control char): {name!r}'
elif name.endswith("."):
err = f"name cannot end with '.': {name!r}"
# Windows reserves these device names with or without an extension (CON.txt also forbidden).
elif name.split(".", 1)[0].upper() in _RESERVED_NAMES:
err = f"name is a Windows-reserved device name: {name!r}"
if err:
self.context.response.success = False
self.context.response.answer = f"Error: {err}"
self.context.response.metadata.update({"error": err})
return
day = _date.today().isoformat()
daily_dir = self.app_context.app_config.daily_dir if self.app_context is not None else "daily"
path_rel = f"{daily_dir}/{day}/{name}.md"
vault_dir = Path(self.file_store.vault_path or ".")
path_abs = (vault_dir / path_rel).resolve()
path_abs.parent.mkdir(parents=True, exist_ok=True)
exists = path_abs.is_file()
payload: dict = {
"date": day,
"name": name,
"path": path_rel,
"exists": exists,
}
if exists:
payload["message"] = f"note already exists at {path_rel}"
self.context.response.success = True
verb = "Resolved" if not exists else "Resolved existing"
self.context.response.answer = f"{verb} note {path_rel}"
self.context.response.metadata.update(payload)

131
reme4/steps/daily/write.py Normal file
View file

@ -0,0 +1,131 @@
"""``daily_write`` — write a daily note's full body + frontmatter; refresh day index.
Collapses the old ``daily_resolve`` ``daily_create`` ``file_write``
chain into a single call. Validates the slug, mkdirs the day folder,
writes body + frontmatter in one shot, refreshes ``daily/<date>.md``
index.
The ``overwrite`` flag picks between two behaviours:
* ``overwrite=False`` (default) idempotent create: when the note
already exists returns ``{created: False, overwritten: False}``
without touching it (mirrors the old ``daily_resolve`` semantics).
Index still refreshes (siblings may have changed; cheap self-healing).
* ``overwrite=True`` unconditional write. Preserves existing-file
encoding via ``detect_file_encoding`` (mirrors ``write_step``).
Frontmatter input is a dict; defaults to ``{name: slug}``. Empty /
None values are dropped (mirrors ``write_step``'s lenient frontmatter
handling). For partial frontmatter mutations on an existing note use
``frontmatter_update`` instead ``daily_write`` is a full-file C/U.
"""
from datetime import date as _date
import frontmatter
from ._daily_io import refresh_day_index, validate_slug
from ..crud._file_io import detect_file_encoding, write_file_safe
from ..base_step import BaseStep
from ...components import R
@R.register("daily_write_step")
class DailyWriteStep(BaseStep):
"""Write ``daily/<date>/<slug>.md`` (create/overwrite); refresh day index."""
def _fail(self, message: str, **meta) -> None:
assert self.context is not None
self.context.response.success = False
self.context.response.answer = f"Error: {message}"
if meta:
self.context.response.metadata.update(meta)
async def execute(self): # pylint: disable=too-many-return-statements
assert self.context is not None
slug: str = self.context.get("slug", "") or ""
body: str = self.context.get("body", "") or ""
day: str = (self.context.get("date") or "").strip() or _date.today().isoformat()
overwrite: bool = bool(self.context.get("overwrite", False))
refresh_index: bool = bool(self.context.get("refresh_index", True))
fm_input = self.context.get("frontmatter")
if fm_input is None:
meta_in: dict = {"name": slug}
elif isinstance(fm_input, dict):
meta_in = dict(fm_input)
meta_in.setdefault("name", slug)
else:
self._fail(f"frontmatter must be a dict, got {type(fm_input).__name__}")
return None
err = validate_slug(slug)
if err:
self._fail(err)
return None
daily_dir = self.app_context.app_config.daily_dir if self.app_context is not None else "daily"
path_rel = f"{daily_dir}/{day}/{slug}.md"
path_abs = (self.vault_path / path_rel).resolve()
existed = path_abs.is_file()
# overwrite=False + file exists → idempotent skip (matches old daily_resolve semantics).
if existed and not overwrite:
payload: dict = {
"date": day,
"slug": slug,
"path": path_rel,
"created": False,
"overwritten": False,
}
if refresh_index:
payload["index"] = await refresh_day_index(self.file_store, day, daily_dir)
self.context.response.success = True
self.context.response.answer = f"Reused existing daily note {path_rel}"
self.context.response.metadata.update(payload)
return self.context.response
# Build the post. Drop empty / None values (write_step idiom).
clean_meta: dict = {}
for k, v in meta_in.items():
if v is None:
continue
if isinstance(v, str) and not v.strip():
continue
clean_meta[k] = v
post = frontmatter.Post(body, **clean_meta)
text = frontmatter.dumps(post)
if not text.endswith("\n"):
text += "\n"
# Preserve existing file encoding on overwrite; new files = UTF-8.
encoding = await detect_file_encoding(path_abs) if existed else "utf-8"
try:
await write_file_safe(path_abs, text, encoding=encoding)
except Exception as e: # pylint: disable=broad-except
self._fail(f"write failed: {e}", date=day, slug=slug, path=path_rel)
return None
payload = {
"date": day,
"slug": slug,
"path": path_rel,
"created": not existed,
"overwritten": existed,
}
if refresh_index:
payload["index"] = await refresh_day_index(self.file_store, day, daily_dir)
self.context.response.success = True
verb = "Wrote" if not existed else "Overwrote"
self.context.response.answer = f"{verb} daily note {path_rel}"
self.context.response.metadata.update(payload)
try:
nbytes = len(text.encode(encoding))
except (UnicodeEncodeError, LookupError):
nbytes = len(text.encode("utf-8"))
self.logger.info(
f"[{self.name}] wrote path={path_rel} bytes={nbytes} "
f"overwrite={overwrite} existed={existed} refresh_index={refresh_index}",
)
return self.context.response

View file

@ -1,22 +1,24 @@
"""Tests for daily-aware steps: daily_resolve / daily_create / daily_list / daily_reindex.
"""Tests for daily-aware steps: daily_read / daily_write / daily_list / daily_reindex.
Sets up a small ``daily/`` tree with mixed dates and exercises note
genesis + list + index-rebuild operations. Body reads / writes are
generic CRUD (covered in test_crud_steps); arbitrary frontmatter
mutation is covered in test_property_steps.
read / write / list / index-rebuild operations. Arbitrary body
mid-edits, plain appends, and frontmatter mutations are generic CRUD
(covered in test_crud_steps and test_property_steps).
A daily note is the single file ``daily/<YYYY-MM-DD>/<slug>.md``
(no folder, no sibling materials). ``daily_resolve`` ensures the day
folder ``daily/<today>/`` exists and returns the vault-relative path
to the note file, reporting whether it already ``exists`` it
does **not** create the file itself. ``daily_create`` writes the
note stub with minimal ``name`` frontmatter and refreshes the day
index.
A daily note is the single file ``daily/<YYYY-MM-DD>/<slug>.md`` (no
folder, no sibling materials). ``daily_read`` returns body in answer
and parsed frontmatter as a dict in metadata; ``daily_write`` writes
body + frontmatter in one shot ``overwrite=False`` (default) is an
idempotent skip-if-exists, ``overwrite=True`` is unconditional. Both
validate the slug up-front (Windows-safe filename rules) so the
path-shape contract is enforced at the daily boundary, not inside
generic CRUD.
``daily_list`` and ``daily_reindex`` both call ``refresh_day_index``
(daily_list as a side effect; daily_reindex as its primary act). They
differ in payload shape: daily_list returns the per-note inventory
(read view), daily_reindex returns the write-result fields (write view).
``daily_list`` is now a **pure read** it no longer triggers index
refresh. Use ``daily_reindex`` explicitly when the index page needs
to be rebuilt. ``daily_write`` auto-refreshes the index by default;
``frontmatter_update`` / ``file_append`` flows leave it stale and
require an explicit ``daily_reindex``.
Note: status / lifecycle / scope / role / source are no longer
core-reserved fields the reme schema reserves only name /
@ -36,8 +38,8 @@ import warnings
from reme4.components.file_store import LocalFileStore
from reme4.steps.daily import (
resolve as daily_resolve_step,
create as daily_create_step,
read as daily_read_step,
write as daily_write_step,
list as daily_list_step,
reindex as daily_reindex_step,
)
@ -151,8 +153,8 @@ def test_daily_list_filters_by_date():
asyncio.run(run())
def test_daily_list_returns_path_name_description():
"""Each note row exposes path / name / description (and nothing else)."""
def test_daily_list_returns_path_slug_name_description():
"""Each note row exposes path / slug / name / description (and nothing else)."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
@ -170,12 +172,13 @@ def test_daily_list_returns_path_name_description():
assert payload["notes"] == [
{
"path": "daily/2026-05-18/alpha.md",
"slug": "alpha",
"name": "Alpha Project",
"description": "JWT auth migration",
},
]
await store.close()
print("✓ test_daily_list_returns_path_name_description passed")
print("✓ test_daily_list_returns_path_slug_name_description passed")
asyncio.run(run())
@ -190,8 +193,6 @@ def test_daily_list_ignores_subdirectories():
("2026-05-18", "main", "main body"),
],
)
# Stray subdir (e.g. left over from an old folder-shaped layout)
# should not be picked up as a note.
stray = Path(tmp) / "daily" / "2026-05-18" / "old-folder"
stray.mkdir(parents=True, exist_ok=True)
(stray / "old-folder.md").write_text(
@ -227,8 +228,8 @@ def test_daily_list_empty_when_no_daily_dir():
asyncio.run(run())
def test_daily_list_triggers_index_refresh_as_side_effect():
"""Calling daily_list also rebuilds daily/<date>.md (the index page)."""
def test_daily_list_does_not_refresh_index():
"""daily_list is a pure read — it must NOT touch daily/<date>.md."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
@ -244,18 +245,15 @@ def test_daily_list_triggers_index_refresh_as_side_effect():
step = daily_list_step.DailyListStep(file_store=store)
await step(date="2026-05-18")
assert index_path.is_file()
text = index_path.read_text(encoding="utf-8")
assert "[[daily/2026-05-18/alpha.md]]" in text
assert "[[daily/2026-05-18/beta.md]]" in text
assert not index_path.exists(), "daily_list must not refresh the day index — use daily_reindex"
await store.close()
print("✓ test_daily_list_triggers_index_refresh_as_side_effect passed")
print("✓ test_daily_list_does_not_refresh_index passed")
asyncio.run(run())
def test_daily_list_response_excludes_index_page_fields():
"""daily_list is the read view — no `path` / `created` fields leak through."""
def test_daily_list_response_shape():
"""daily_list returns only {date, notes}."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
@ -269,191 +267,170 @@ def test_daily_list_response_excludes_index_page_fields():
payload = _metadata(step)
assert set(payload.keys()) == {"date", "notes"}
await store.close()
print("✓ test_daily_list_response_excludes_index_page_fields passed")
print("✓ test_daily_list_response_shape passed")
asyncio.run(run())
# -- daily_resolve_step -------------------------------------------------------
# -- daily_read_step ----------------------------------------------------------
def test_daily_resolve_ensures_day_folder_and_reports_missing_file():
"""daily_resolve on a fresh name creates the day folder, leaves the note
file unwritten, and reports ``exists=False``."""
def test_daily_read_returns_body_and_frontmatter_dict():
"""daily_read on an existing note returns body in answer and parsed frontmatter dict."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_resolve_step.DailyResolveStep(file_store=store)
await step(name="kickoff")
store = LocalFileStore(store_name="t", embedding_model="")
await store.start()
day_dir = Path(tmp) / "daily" / "2026-05-18"
day_dir.mkdir(parents=True, exist_ok=True)
(day_dir / "alpha.md").write_text(
"---\nname: Alpha Project\ndescription: JWT migration\n---\n## Objective\nfoo\n",
encoding="utf-8",
)
step = daily_read_step.DailyReadStep(file_store=store)
await step(slug="alpha", date="2026-05-18")
payload = _metadata(step)
assert payload["exists"] is False
assert payload["name"] == "kickoff"
assert payload["date"] == _today()
assert payload["path"] == f"daily/{_today()}/kickoff.md"
assert "message" not in payload
day_dir = Path(tmp) / "daily" / _today()
assert day_dir.is_dir()
# The note file itself is NOT created by resolve.
assert not (day_dir / "kickoff.md").exists()
# No index page either.
assert not (Path(tmp) / "daily" / f"{_today()}.md").exists()
assert step.context.response.success is True
assert "## Objective\nfoo" in step.context.response.answer
assert "---" not in step.context.response.answer # frontmatter stripped
assert payload["date"] == "2026-05-18"
assert payload["slug"] == "alpha"
assert payload["path"] == "daily/2026-05-18/alpha.md"
assert payload["exists"] is True
assert payload["frontmatter"] == {
"name": "Alpha Project",
"description": "JWT migration",
}
await store.close()
print("✓ test_daily_resolve_ensures_day_folder_and_reports_missing_file passed")
print("✓ test_daily_read_returns_body_and_frontmatter_dict passed")
asyncio.run(run())
def test_daily_resolve_idempotent_when_file_exists():
"""Existing note file ⇒ ``exists=True`` + message; file contents untouched."""
def test_daily_read_default_date_is_today():
"""Omitted ``date`` ⇒ today's folder."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies(
[(_today(), "ongoing", "morning thoughts")],
[(_today(), "live", "current body")],
)
file_path = Path(tmp) / "daily" / _today() / "ongoing.md"
before = file_path.read_text(encoding="utf-8")
step = daily_resolve_step.DailyResolveStep(file_store=store)
await step(name="ongoing")
step = daily_read_step.DailyReadStep(file_store=store)
await step(slug="live")
payload = _metadata(step)
assert payload["date"] == _today()
assert payload["path"] == f"daily/{_today()}/live.md"
assert payload["exists"] is True
assert payload["name"] == "ongoing"
assert payload["path"] == f"daily/{_today()}/ongoing.md"
assert "already exists" in payload["message"]
# Contents unchanged.
assert file_path.read_text(encoding="utf-8") == before
assert "current body" in step.context.response.answer
await store.close()
print("✓ test_daily_resolve_idempotent_when_file_exists passed")
print("✓ test_daily_read_default_date_is_today passed")
asyncio.run(run())
def test_daily_resolve_rejects_empty_name():
"""Empty name ⇒ error payload, success=False, no day folder created."""
def test_daily_read_missing_file_reports_exists_false():
"""Note absent ⇒ success=False, payload carries exists=False + path."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_resolve_step.DailyResolveStep(file_store=store)
await step(name="")
step = daily_read_step.DailyReadStep(file_store=store)
await step(slug="nothing-here", date="2026-05-18")
payload = _metadata(step)
assert "error" in payload
assert "required" in payload["error"]
assert step.context.response.success is False
assert not (Path(tmp) / "daily" / _today()).exists()
assert payload["exists"] is False
assert payload["date"] == "2026-05-18"
assert payload["slug"] == "nothing-here"
assert payload["path"] == "daily/2026-05-18/nothing-here.md"
await store.close()
print("✓ test_daily_resolve_rejects_empty_name passed")
print("✓ test_daily_read_missing_file_reports_exists_false passed")
asyncio.run(run())
def test_daily_resolve_rejects_windows_invalid_chars():
"""Windows-reserved characters in name ⇒ error, no day folder created."""
def test_daily_read_rejects_invalid_slug():
"""Slug validation (Windows-safe filename rules) runs up-front."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_resolve_step.DailyResolveStep(file_store=store)
for bad in (
"foo/bar",
"foo:bar",
"foo*bar",
"foo?bar",
"foo|bar",
"foo<bar",
"foo>bar",
'foo"bar',
"foo\\bar",
):
await step(name=bad)
payload = _metadata(step)
assert "error" in payload, f"expected error for {bad!r}, got {payload!r}"
assert "invalid characters" in payload["error"]
assert step.context.response.success is False
step = daily_read_step.DailyReadStep(file_store=store)
for bad in ("foo/bar", "foo:bar", "foo*bar", "CON", "lpt9", "foo.", " bar"):
await step(slug=bad)
assert step.context.response.success is False, f"expected reject for {bad!r}"
await store.close()
print("✓ test_daily_resolve_rejects_windows_invalid_chars passed")
print("✓ test_daily_read_rejects_invalid_slug passed")
asyncio.run(run())
def test_daily_resolve_rejects_windows_reserved_names():
"""Windows device-name stems (CON / PRN / AUX / NUL / COM1-9 / LPT1-9) are rejected."""
def test_daily_read_empty_frontmatter_dict():
"""No frontmatter ⇒ ``frontmatter`` key is the empty dict."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_resolve_step.DailyResolveStep(file_store=store)
for bad in ("CON", "prn", "Aux", "NUL", "COM1", "lpt9", "CON.notes", "com5.txt"):
await step(name=bad)
payload = _metadata(step)
assert "error" in payload, f"expected error for {bad!r}, got {payload!r}"
assert "reserved" in payload["error"]
store = LocalFileStore(store_name="t", embedding_model="")
await store.start()
day_dir = Path(tmp) / "daily" / "2026-05-18"
day_dir.mkdir(parents=True, exist_ok=True)
(day_dir / "plain.md").write_text("just body\n", encoding="utf-8")
step = daily_read_step.DailyReadStep(file_store=store)
await step(slug="plain", date="2026-05-18")
payload = _metadata(step)
assert payload["frontmatter"] == {}
assert step.context.response.answer.strip() == "just body"
await store.close()
print("✓ test_daily_resolve_rejects_windows_reserved_names passed")
print("✓ test_daily_read_empty_frontmatter_dict passed")
asyncio.run(run())
def test_daily_resolve_rejects_trailing_dot_or_whitespace():
"""Trailing '.' / leading-or-trailing whitespace are rejected."""
# -- daily_write_step ---------------------------------------------------------
def test_daily_write_creates_note_and_refreshes_index():
"""Fresh slug ⇒ note file written + day index refreshed by default."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_resolve_step.DailyResolveStep(file_store=store)
for bad in ("foo.", "foo ", " foo", " bar "):
await step(name=bad)
payload = _metadata(step)
assert "error" in payload, f"expected error for {bad!r}, got {payload!r}"
await store.close()
print("✓ test_daily_resolve_rejects_trailing_dot_or_whitespace passed")
asyncio.run(run())
# -- daily_create_step --------------------------------------------------------
def test_daily_create_writes_file_and_refreshes_index():
"""Fresh slug ⇒ note file created with `name` frontmatter + index refreshed."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_create_step.DailyCreateStep(file_store=store)
await step(slug="kickoff", date="2026-05-18", name="Kickoff Task", body="hello body")
step = daily_write_step.DailyWriteStep(file_store=store)
await step(
slug="kickoff",
date="2026-05-18",
body="## Plan\nfirst pass\n",
frontmatter={"name": "Kickoff Task", "description": "Day-one plan"},
)
payload = _metadata(step)
assert payload["created"] is True
assert payload["overwritten"] is False
assert payload["date"] == "2026-05-18"
assert payload["slug"] == "kickoff"
assert payload["path"] == "daily/2026-05-18/kickoff.md"
note = Path(tmp) / "daily" / "2026-05-18" / "kickoff.md"
assert note.is_file()
text = note.read_text(encoding="utf-8")
assert "name: Kickoff Task" in text
assert "hello body" in text
assert "description: Day-one plan" in text
assert "## Plan\nfirst pass" in text
# Index page refreshed.
index = Path(tmp) / "daily" / "2026-05-18.md"
assert index.is_file()
assert "[[daily/2026-05-18/kickoff.md]]" in index.read_text(encoding="utf-8")
await store.close()
print("✓ test_daily_create_writes_file_and_refreshes_index passed")
print("✓ test_daily_write_creates_note_and_refreshes_index passed")
asyncio.run(run())
def test_daily_create_is_idempotent():
"""Existing note ⇒ `created=False`, file untouched, index still refreshes."""
def test_daily_write_create_mode_is_idempotent():
"""``overwrite=False`` + file exists ⇒ created=False, file untouched, index still refreshed."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
@ -463,35 +440,228 @@ def test_daily_create_is_idempotent():
file_path = Path(tmp) / "daily" / "2026-05-18" / "ongoing.md"
before = file_path.read_text(encoding="utf-8")
step = daily_create_step.DailyCreateStep(file_store=store)
step = daily_write_step.DailyWriteStep(file_store=store)
await step(slug="ongoing", date="2026-05-18", body="ignored new body")
payload = _metadata(step)
assert payload["created"] is False
assert payload["overwritten"] is False
assert payload["path"] == "daily/2026-05-18/ongoing.md"
# File contents unchanged.
assert file_path.read_text(encoding="utf-8") == before
# But the index was still rebuilt.
assert payload["index"]["path"] == "daily/2026-05-18.md"
await store.close()
print("✓ test_daily_create_is_idempotent passed")
print("✓ test_daily_write_create_mode_is_idempotent passed")
asyncio.run(run())
def test_daily_create_name_falls_back_to_slug():
"""Omitted ``name`` arg ⇒ frontmatter ``name`` defaults to slug."""
def test_daily_write_overwrite_mode_replaces_existing():
"""``overwrite=True`` ⇒ unconditional rewrite; overwritten=True."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies(
[("2026-05-18", "live", "stale text")],
)
step = daily_write_step.DailyWriteStep(file_store=store)
await step(
slug="live",
date="2026-05-18",
body="## Updated\nfresh body\n",
frontmatter={"name": "Live", "description": "post-merge"},
overwrite=True,
)
payload = _metadata(step)
assert payload["created"] is False
assert payload["overwritten"] is True
note = Path(tmp) / "daily" / "2026-05-18" / "live.md"
text = note.read_text(encoding="utf-8")
assert "stale text" not in text
assert "## Updated\nfresh body" in text
assert "description: post-merge" in text
await store.close()
print("✓ test_daily_write_overwrite_mode_replaces_existing passed")
asyncio.run(run())
def test_daily_write_default_frontmatter_uses_slug_as_name():
"""Omitted ``frontmatter`` ⇒ defaults to ``{name: slug}``."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_create_step.DailyCreateStep(file_store=store)
step = daily_write_step.DailyWriteStep(file_store=store)
await step(slug="auth-refactor", date="2026-05-18")
note = Path(tmp) / "daily" / "2026-05-18" / "auth-refactor.md"
assert "name: auth-refactor" in note.read_text(encoding="utf-8")
await store.close()
print("✓ test_daily_create_name_falls_back_to_slug passed")
print("✓ test_daily_write_default_frontmatter_uses_slug_as_name passed")
asyncio.run(run())
def test_daily_write_default_body_is_empty():
"""Omitted ``body`` ⇒ empty body, frontmatter-only note."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_write_step.DailyWriteStep(file_store=store)
await step(slug="stub", date="2026-05-18")
note = Path(tmp) / "daily" / "2026-05-18" / "stub.md"
text = note.read_text(encoding="utf-8")
# Just frontmatter + a single newline after the closing ---.
assert text.startswith("---\n")
assert "name: stub" in text
# Body section is empty: the post body resolves to "".
assert text.rstrip().endswith("---")
await store.close()
print("✓ test_daily_write_default_body_is_empty passed")
asyncio.run(run())
def test_daily_write_drops_empty_frontmatter_values():
"""Empty / None frontmatter values are dropped (write_step idiom)."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_write_step.DailyWriteStep(file_store=store)
await step(
slug="trim",
date="2026-05-18",
frontmatter={
"name": "Trim",
"description": " ", # whitespace-only → drop
"extra": None, # None → drop
"kept": "value",
},
)
note = Path(tmp) / "daily" / "2026-05-18" / "trim.md"
text = note.read_text(encoding="utf-8")
assert "name: Trim" in text
assert "kept: value" in text
assert "description:" not in text
assert "extra:" not in text
await store.close()
print("✓ test_daily_write_drops_empty_frontmatter_values passed")
asyncio.run(run())
def test_daily_write_rejects_invalid_slug():
"""Slug validation runs before any IO."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_write_step.DailyWriteStep(file_store=store)
for bad in ("foo/bar", "foo:bar", "CON", "lpt9", "foo.", " bar"):
await step(slug=bad, date="2026-05-18", body="x")
assert step.context.response.success is False, f"expected reject for {bad!r}"
# No day folder should be created on rejection.
assert not (Path(tmp) / "daily" / "2026-05-18").exists()
await store.close()
print("✓ test_daily_write_rejects_invalid_slug passed")
asyncio.run(run())
def test_daily_write_overwrite_default_is_false_create_then_skip():
"""First call (file absent) creates; second call without overwrite skips — proves the
overwrite=False default mirrors the old daily_resolve idempotent probe."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_write_step.DailyWriteStep(file_store=store)
await step(slug="probe", date="2026-05-18", body="original")
first = _metadata(step)
assert first["created"] is True
assert first["overwritten"] is False
await step(slug="probe", date="2026-05-18", body="ignored")
second = _metadata(step)
assert second["created"] is False
assert second["overwritten"] is False
note = Path(tmp) / "daily" / "2026-05-18" / "probe.md"
assert "original" in note.read_text(encoding="utf-8")
assert "ignored" not in note.read_text(encoding="utf-8")
await store.close()
print("✓ test_daily_write_overwrite_default_is_false_create_then_skip passed")
asyncio.run(run())
def test_daily_write_rejects_non_dict_frontmatter():
"""``frontmatter`` must be a dict when supplied."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_write_step.DailyWriteStep(file_store=store)
await step(slug="ok", date="2026-05-18", frontmatter="not a dict")
assert step.context.response.success is False
assert "dict" in (step.context.response.answer or "")
await store.close()
print("✓ test_daily_write_rejects_non_dict_frontmatter passed")
asyncio.run(run())
def test_daily_write_refresh_index_can_be_disabled():
"""``refresh_index=False`` ⇒ note written but day index untouched."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
step = daily_write_step.DailyWriteStep(file_store=store)
await step(
slug="solo",
date="2026-05-18",
body="x",
refresh_index=False,
)
note = Path(tmp) / "daily" / "2026-05-18" / "solo.md"
assert note.is_file()
assert "index" not in _metadata(step)
assert not (Path(tmp) / "daily" / "2026-05-18.md").exists()
await store.close()
print("✓ test_daily_write_refresh_index_can_be_disabled passed")
asyncio.run(run())
def test_daily_write_round_trips_with_daily_read():
"""A note written via daily_write must be retrievable via daily_read with the same data."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store_with_dailies([])
body = "## Plan\nstep one\nstep two\n"
fm = {"name": "Round-trip", "description": "CRUD smoke"}
await daily_write_step.DailyWriteStep(file_store=store)(
slug="round-trip",
date="2026-05-18",
body=body,
frontmatter=fm,
)
read_step = daily_read_step.DailyReadStep(file_store=store)
await read_step(slug="round-trip", date="2026-05-18")
assert read_step.context.response.answer.strip() == body.strip()
assert _metadata(read_step)["frontmatter"] == fm
await store.close()
print("✓ test_daily_write_round_trips_with_daily_read passed")
asyncio.run(run())
@ -517,7 +687,6 @@ def test_day_index_lists_each_note():
text = _day_index_text(tmp, "2026-05-18")
assert "[[daily/2026-05-18/alpha.md]]" in text
assert "[[daily/2026-05-18/beta.md]]" in text
# Note names show on the indented sub-line.
assert "Alpha Project" in text
assert "Beta Project" in text
await store.close()
@ -537,19 +706,16 @@ def test_day_index_includes_note_descriptions():
await store.start()
cases = [
("alpha", "Alpha Project", "实现 JWT auth 中间件,迁移 session middleware"),
("beta", "beta", "调研增值税新政对 SaaS 的影响"), # name == slug
("gamma", "Gamma", ""), # no description
("beta", "beta", "调研增值税新政对 SaaS 的影响"),
("gamma", "Gamma", ""),
]
for slug, name, description in cases:
await _seed_note("2026-05-18", slug, name=name, description=description)
await daily_reindex_step.DailyReindexStep(file_store=store)(date="2026-05-18")
text = _day_index_text(tmp, "2026-05-18")
# name + description rendered together
assert "Alpha Project — 实现 JWT auth 中间件" in text
# name == slug → only description shown (no redundant "beta")
assert "调研增值税新政对 SaaS 的影响" in text
# no description → only name shown, no trailing em-dash
assert " Gamma\n" in text or text.rstrip().endswith("Gamma")
await store.close()
print("✓ test_day_index_includes_note_descriptions passed")
@ -591,7 +757,6 @@ def test_day_index_preserves_manual_segment():
reindex = daily_reindex_step.DailyReindexStep(file_store=store)
await reindex(date="2026-05-18")
# Inject a manual annotation into the index file's body.
index_path = Path(tmp) / "daily" / "2026-05-18.md"
text = index_path.read_text(encoding="utf-8")
patched = text.replace(
@ -600,13 +765,11 @@ def test_day_index_preserves_manual_segment():
)
index_path.write_text(patched, encoding="utf-8")
# Adding a sibling note + refresh — manual segment must survive.
await _seed_note("2026-05-18", "beta")
await reindex(date="2026-05-18")
after = index_path.read_text(encoding="utf-8")
assert "MY HAND-WRITTEN NOTE" in after
assert "这是我手写的备忘" in after
# Auto block was updated with the new note.
assert "[[daily/2026-05-18/beta.md]]" in after
await store.close()
print("✓ test_day_index_preserves_manual_segment passed")
@ -628,7 +791,6 @@ def test_daily_reindex_returns_write_view():
("2026-05-18", "beta", "b body"),
],
)
# Index doesn't exist yet.
assert not (Path(tmp) / "daily" / "2026-05-18.md").exists()
step = daily_reindex_step.DailyReindexStep(file_store=store)
@ -678,20 +840,27 @@ if __name__ == "__main__":
print("\n=== Daily step tests ===")
test_daily_list_default_date_is_today()
test_daily_list_filters_by_date()
test_daily_list_returns_path_name_description()
test_daily_list_returns_path_slug_name_description()
test_daily_list_ignores_subdirectories()
test_daily_list_empty_when_no_daily_dir()
test_daily_list_triggers_index_refresh_as_side_effect()
test_daily_list_response_excludes_index_page_fields()
test_daily_resolve_ensures_day_folder_and_reports_missing_file()
test_daily_resolve_idempotent_when_file_exists()
test_daily_resolve_rejects_empty_name()
test_daily_resolve_rejects_windows_invalid_chars()
test_daily_resolve_rejects_windows_reserved_names()
test_daily_resolve_rejects_trailing_dot_or_whitespace()
test_daily_create_writes_file_and_refreshes_index()
test_daily_create_is_idempotent()
test_daily_create_name_falls_back_to_slug()
test_daily_list_does_not_refresh_index()
test_daily_list_response_shape()
test_daily_read_returns_body_and_frontmatter_dict()
test_daily_read_default_date_is_today()
test_daily_read_missing_file_reports_exists_false()
test_daily_read_rejects_invalid_slug()
test_daily_read_empty_frontmatter_dict()
test_daily_write_creates_note_and_refreshes_index()
test_daily_write_create_mode_is_idempotent()
test_daily_write_overwrite_mode_replaces_existing()
test_daily_write_default_frontmatter_uses_slug_as_name()
test_daily_write_default_body_is_empty()
test_daily_write_drops_empty_frontmatter_values()
test_daily_write_rejects_invalid_slug()
test_daily_write_overwrite_default_is_false_create_then_skip()
test_daily_write_rejects_non_dict_frontmatter()
test_daily_write_refresh_index_can_be_disabled()
test_daily_write_round_trips_with_daily_read()
test_day_index_lists_each_note()
test_day_index_includes_note_descriptions()
test_day_index_description_is_note_count()