mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-09-08 22:21:15 +00:00
fix: harden image resource lifecycle
This commit is contained in:
wang-qisen
parent
4785463f4b
commit
1467c204aa
9 changed files with 614 additions and 99 deletions
|
|
@ -36,7 +36,8 @@ In short, it turns "a file was archived" into "the resource is usable."
|
|||
|
||||
Auto Resource uses `resource/` as the entry point for source material. Date directories are recommended, and their date
|
||||
determines which daily memory layer receives the interpreted card. A file directly under `resource/` is also supported
|
||||
and uses today in the application timezone.
|
||||
and uses today in the application timezone when it is first processed. On later days, an exact `source_resource` match
|
||||
keeps updates and deletion tied to that original daily card instead of creating a new card or leaving an orphan.
|
||||
|
||||
Example directory:
|
||||
|
||||
|
|
@ -65,10 +66,14 @@ The card body starts with an `![[resource/...]]` embed link and the frontmatter
|
|||
so text search reaches image content through the caption.
|
||||
|
||||
The vision model is the `vision` instance of `as_llm` when configured, and otherwise falls back to the `default`
|
||||
instance — a multimodal default model needs no extra configuration. Images larger than the request budget or in
|
||||
provider-unfriendly formats are downscaled or re-encoded in memory for the request only; the original file under
|
||||
`resource/` is never modified. When an image changes, its card is rewritten in place; when the image is deleted, the
|
||||
card is removed with it.
|
||||
instance — a multimodal default model needs no extra configuration. Images wider or taller than 2048px are downscaled,
|
||||
and provider-unfriendly formats are re-encoded, in memory for the request only; the original file under
|
||||
`resource/` is never modified. Before a full decode, image dimensions are checked against a default limit of 40,000,000
|
||||
pixels; images over the limit and Pillow decompression-bomb warnings fail only that resource. EXIF orientation is
|
||||
applied to the in-memory request copy before resizing or conversion. Oversized JPEGs first use decoder-level
|
||||
downsampling, followed by a final thumbnail pass when needed. The VLM request MIME and the card's frontmatter
|
||||
`media_type` use the format Pillow detects from the image bytes, rather than trusting the filename extension. When an
|
||||
image changes, its card is rewritten in place; when the image is deleted, the card is removed with it.
|
||||
|
||||
Image preprocessing uses Pillow from the `core` extra. HEIC resources additionally require the optional
|
||||
`image-heif` extra: `pip install "reme-ai[image-heif]"`. Other supported image formats do not load or require the HEIF
|
||||
|
|
|
|||
|
|
@ -185,8 +185,8 @@ reme auto_memory \
|
|||
```
|
||||
|
||||
After placing external material under `resource/YYYY-MM-DD/` or directly under `resource/`, the default background task
|
||||
watches
|
||||
`md/txt/json/jsonl/csv/yaml/html`. You can also trigger processing manually:
|
||||
watches text resources (`md/txt/json/jsonl/csv/yaml/html`) and image resources
|
||||
(`png/jpg/jpeg/webp/gif/bmp/tiff/heic`). You can also trigger processing manually:
|
||||
|
||||
```bash
|
||||
reme auto_resource changes='[{"path":"resource/2026-06-20/report.md","change":"added"}]'
|
||||
|
|
|
|||
|
|
@ -33,7 +33,8 @@ resource/[YYYY-MM-DD/]<resource_file>
|
|||
## 原始资料入口
|
||||
|
||||
Auto Resource 以 `resource/` 作为原始资料入口。推荐按日期放置，目录日期会决定它进入哪一天的 daily 记忆层；也支持直接放在
|
||||
`resource/` 根目录，此时使用应用时区中的今天。
|
||||
`resource/` 根目录，首次处理时使用应用时区中的今天。之后即使跨天更新或删除，也会通过精确匹配
|
||||
`source_resource` 继续操作原 daily 卡片，不会重复新建卡片或留下孤立链接。
|
||||
|
||||
示例目录：
|
||||
|
||||
|
|
@ -56,7 +57,13 @@ workspace/
|
|||
|
||||
图像文件的解读方式相同：视觉模型写入一张 caption 卡片并链接原图。卡片正文以 `![[resource/...]]` 嵌入链接开头，frontmatter 携带 `kind: image` 与 `media_type`，文本检索因此可以通过 caption 命中图像内容。
|
||||
|
||||
视觉模型优先使用配置中的 `as_llm` `vision` 实例，未配置时回退到 `default` 实例——默认模型具备视觉能力时无需额外配置。超过请求预算或格式不被模型接受的图像，仅在请求前于内存中降采样或转码；`resource/` 下的原图文件不会被修改。图像变更时卡片原地重写；图像删除时卡片随之删除。
|
||||
视觉模型优先使用配置中的 `as_llm` `vision` 实例，未配置时回退到 `default` 实例——默认模型具备视觉能力时无需额外配置。宽或高超过 2048px 的图像会降采样，格式不被模型接受的图像会转码；这些处理只发生在请求前的内存副本中，`resource/` 下的原图文件不会被修改。图像变更时卡片原地重写；图像删除时卡片随之删除。
|
||||
|
||||
在完整解码前，系统会检查图像尺寸，默认上限为 40,000,000 像素；超限图像或 Pillow
|
||||
decompression-bomb 警告只会导致当前资源失败。缩放或转码前，会按 EXIF orientation 校正仅用于请求的内存副本。
|
||||
尺寸过大的 JPEG 会先使用 decoder-level downsampling，并在需要时再完成最终缩放。
|
||||
VLM 请求的 MIME 和卡片 frontmatter 中的 `media_type` 都使用 Pillow 根据实际图像字节识别的格式，
|
||||
而不是直接信任文件扩展名。
|
||||
|
||||
图像预处理使用 `core` extra 中的 Pillow。HEIC 资源还需要可选的 `image-heif` extra：
|
||||
`pip install "reme-ai[image-heif]"`。其他受支持图像格式不会加载或依赖 HEIF 插件。
|
||||
|
|
|
|||
|
|
@ -178,7 +178,8 @@ reme auto_memory \
|
|||
memory_hint="记录用户偏好"
|
||||
```
|
||||
|
||||
外部资料放入 `resource/YYYY-MM-DD/` 或直接放在 `resource/` 下后，默认后台会监听 `md/txt/json/jsonl/csv/yaml/html`。
|
||||
外部资料放入 `resource/YYYY-MM-DD/` 或直接放在 `resource/` 下后，默认后台会监听文本资源
|
||||
(`md/txt/json/jsonl/csv/yaml/html`) 和图像资源 (`png/jpg/jpeg/webp/gif/bmp/tiff/heic`)。
|
||||
也可以手动触发：
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ import base64
|
|||
import io
|
||||
import json
|
||||
import re
|
||||
import warnings
|
||||
from pathlib import Path, PurePosixPath
|
||||
|
||||
import aiofiles
|
||||
|
|
@ -11,17 +12,22 @@ from agentscope.message import Base64Source, DataBlock, TextBlock, UserMsg
|
|||
from agentscope.model import ChatModelBase
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..file_io._path import IMAGE_MIME_BY_EXT, IMAGE_SUFFIXES
|
||||
from ..file_io._path import IMAGE_SUFFIXES
|
||||
from .base_auto_resource import _SOURCE_RESOURCE_KEY, _sanitize_note_name, BaseAutoResourceStep
|
||||
from ...components import R
|
||||
from ...enumeration import ComponentEnum
|
||||
|
||||
DEFAULT_MAX_IMAGE_INPUT_BYTES = 50 * 1024 * 1024
|
||||
DEFAULT_MAX_IMAGE_PIXELS = 40_000_000
|
||||
MAX_IMAGE_REQUEST_DIMENSION = 2048
|
||||
_JPEG_QUALITY = 85
|
||||
# Suffixes re-encoded to provider-friendly PNG/JPEG for VLM requests;
|
||||
# the stored resource file is never modified.
|
||||
_CONVERT_SUFFIXES = {".bmp", ".tiff", ".heic"}
|
||||
# Decoded formats outside this set are re-encoded to provider-friendly
|
||||
# PNG/JPEG for VLM requests; the stored resource file is never modified.
|
||||
_PASSTHROUGH_IMAGE_MIMES = frozenset({"image/png", "image/jpeg", "image/webp", "image/gif"})
|
||||
_HEIF_BRANDS = frozenset(
|
||||
{b"heic", b"heif", b"heix", b"heim", b"heis", b"hevc", b"hevx", b"hevm", b"hevs", b"mif1", b"msf1"},
|
||||
)
|
||||
_MAX_FTYP_SCAN_BYTES = 4096
|
||||
_JSON_FENCE_RE = re.compile(r"^\s*```(?:json)?\s*(.*?)\s*```\s*$", re.DOTALL)
|
||||
|
||||
|
||||
|
|
@ -37,84 +43,170 @@ class _CaptionOutput(BaseModel):
|
|||
)
|
||||
|
||||
|
||||
def _load_pillow(suffix: str):
|
||||
"""Load Pillow and optional HEIC support only when image processing runs."""
|
||||
def _load_pillow():
|
||||
"""Load the core image dependency only when image processing runs."""
|
||||
try:
|
||||
from PIL import Image # pylint: disable=import-outside-toplevel
|
||||
from PIL import Image, ImageOps # pylint: disable=import-outside-toplevel
|
||||
except ImportError as exc:
|
||||
raise RuntimeError("Image captioning requires Pillow; install reme-ai[core]") from exc
|
||||
|
||||
if suffix == ".heic":
|
||||
try:
|
||||
from pillow_heif import register_heif_opener # pylint: disable=import-outside-toplevel
|
||||
except ImportError as exc:
|
||||
raise RuntimeError("HEIC image captioning requires pillow-heif; install reme-ai[image-heif]") from exc
|
||||
try:
|
||||
register_heif_opener()
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
raise RuntimeError(f"Failed to initialize HEIC image support: {exc}") from exc
|
||||
return Image
|
||||
return Image, ImageOps
|
||||
|
||||
|
||||
def _normalize_image_bytes(data: bytes, suffix: str) -> tuple[bytes, str] | None:
|
||||
"""Downscale or re-encode image bytes in memory for a VLM request.
|
||||
def _looks_like_heif(data: bytes) -> bool:
|
||||
"""Return whether an ISO-BMFF header declares a HEIC/HEIF brand."""
|
||||
if len(data) < 12 or data[4:8] != b"ftyp":
|
||||
return False
|
||||
box_size = int.from_bytes(data[:4], "big")
|
||||
if box_size < 12:
|
||||
return False
|
||||
end = min(box_size, len(data), _MAX_FTYP_SCAN_BYTES)
|
||||
if data[8:12] in _HEIF_BRANDS:
|
||||
return True
|
||||
return any(data[index : index + 4] in _HEIF_BRANDS for index in range(16, end - 3, 4))
|
||||
|
||||
Returns ``None`` only when valid image bytes already have a suitable size
|
||||
and format. Missing dependencies and decode/convert failures are explicit.
|
||||
The stored resource file is never modified.
|
||||
"""
|
||||
image_module = _load_pillow(suffix)
|
||||
|
||||
def _register_heif_opener() -> None:
|
||||
"""Load and register HEIC support only for bytes that declare HEIF."""
|
||||
try:
|
||||
image = image_module.open(io.BytesIO(data))
|
||||
from pillow_heif import register_heif_opener # pylint: disable=import-outside-toplevel
|
||||
except ImportError as exc:
|
||||
raise RuntimeError("HEIC image captioning requires pillow-heif; install reme-ai[image-heif]") from exc
|
||||
try:
|
||||
register_heif_opener()
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
raise RuntimeError(f"Failed to initialize HEIC image support: {exc}") from exc
|
||||
|
||||
|
||||
def _normalize_image_bytes(
|
||||
data: bytes,
|
||||
suffix: str,
|
||||
*,
|
||||
max_image_pixels: int = DEFAULT_MAX_IMAGE_PIXELS,
|
||||
) -> tuple[bytes | None, str, str]:
|
||||
"""Validate and optionally normalize image bytes for a VLM request.
|
||||
|
||||
The returned tuple is ``(normalized_bytes, request_mime, source_mime)``.
|
||||
``normalized_bytes`` is ``None`` only when the original bytes can be sent
|
||||
unchanged. MIME values come from the decoded image rather than its suffix.
|
||||
Missing dependencies, unsafe pixel counts, and decode/convert failures are
|
||||
explicit. The stored resource file is never modified.
|
||||
"""
|
||||
try:
|
||||
pixel_limit = int(max_image_pixels)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(f"max_image_pixels must be a positive integer: {max_image_pixels!r}") from exc
|
||||
if pixel_limit <= 0:
|
||||
raise ValueError(f"max_image_pixels must be a positive integer: {max_image_pixels!r}")
|
||||
|
||||
image_module, image_ops = _load_pillow()
|
||||
if _looks_like_heif(data):
|
||||
_register_heif_opener()
|
||||
try:
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("error", image_module.DecompressionBombWarning)
|
||||
image = image_module.open(io.BytesIO(data))
|
||||
except (image_module.DecompressionBombWarning, image_module.DecompressionBombError) as exc:
|
||||
raise RuntimeError(
|
||||
f"Image rejected by Pillow decompression-bomb protection ({suffix or 'unknown suffix'})",
|
||||
) from exc
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
raise RuntimeError(f"Failed to decode image ({suffix or 'unknown suffix'}): {exc}") from exc
|
||||
|
||||
with image:
|
||||
width, height = image.size
|
||||
pixel_count = width * height
|
||||
if pixel_count > pixel_limit:
|
||||
raise RuntimeError(
|
||||
f"Image exceeds max_image_pixels before decode: " f"{width}x{height}={pixel_count} > {pixel_limit}",
|
||||
)
|
||||
|
||||
source_mime = str(image.get_format_mimetype() or "").strip().lower()
|
||||
if not source_mime.startswith("image/"):
|
||||
raise RuntimeError(
|
||||
f"Cannot determine decoded image MIME type ({suffix or 'unknown suffix'}, format={image.format!r})",
|
||||
)
|
||||
|
||||
needs_resize = width > MAX_IMAGE_REQUEST_DIMENSION or height > MAX_IMAGE_REQUEST_DIMENSION
|
||||
if source_mime == "image/jpeg" and needs_resize:
|
||||
max_dimension = max(width, height)
|
||||
decoder_size = (
|
||||
max(1, (width * MAX_IMAGE_REQUEST_DIMENSION + max_dimension - 1) // max_dimension),
|
||||
max(1, (height * MAX_IMAGE_REQUEST_DIMENSION + max_dimension - 1) // max_dimension),
|
||||
)
|
||||
try:
|
||||
# JPEG supports power-of-two decoder scaling. ``draft`` picks
|
||||
# the smallest decoded frame that still covers decoder_size,
|
||||
# reducing peak memory before the final LANCZOS thumbnail.
|
||||
image.draft(None, decoder_size)
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
raise RuntimeError(f"Failed to prepare JPEG decoder downsampling ({width}x{height}): {exc}") from exc
|
||||
|
||||
try:
|
||||
image.load()
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("error", image_module.DecompressionBombWarning)
|
||||
image.load()
|
||||
orientation = int(image.getexif().get(274, 1) or 1)
|
||||
image_ops.exif_transpose(image, in_place=True)
|
||||
except (image_module.DecompressionBombWarning, image_module.DecompressionBombError) as exc:
|
||||
raise RuntimeError(
|
||||
f"Image rejected by Pillow decompression-bomb protection ({suffix or 'unknown suffix'})",
|
||||
) from exc
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
raise RuntimeError(f"Failed to decode image ({suffix or 'unknown suffix'}): {exc}") from exc
|
||||
|
||||
needs_resize = image.width > MAX_IMAGE_REQUEST_DIMENSION or image.height > MAX_IMAGE_REQUEST_DIMENSION
|
||||
needs_convert = suffix in _CONVERT_SUFFIXES
|
||||
if not needs_resize and not needs_convert:
|
||||
return None
|
||||
needs_convert = source_mime not in _PASSTHROUGH_IMAGE_MIMES
|
||||
needs_orientation = orientation in range(2, 9)
|
||||
if not needs_resize and not needs_convert and not needs_orientation:
|
||||
return None, source_mime, source_mime
|
||||
try:
|
||||
if needs_resize:
|
||||
# Resize the decoded source before color conversion so large
|
||||
# non-JPEG images do not require a second full-size frame.
|
||||
# Pillow 10 cannot apply LANCZOS directly to 16-bit integer
|
||||
# modes; NEAREST keeps that path bounded without a full-size
|
||||
# RGB conversion first.
|
||||
resize_filter = image_module.Resampling.LANCZOS
|
||||
if image.mode.startswith("I;16"):
|
||||
resize_filter = image_module.Resampling.NEAREST
|
||||
image.thumbnail((MAX_IMAGE_REQUEST_DIMENSION, MAX_IMAGE_REQUEST_DIMENSION), resize_filter)
|
||||
has_alpha = image.mode in ("RGBA", "LA", "P")
|
||||
frame = image.convert("RGBA" if has_alpha else "RGB")
|
||||
if needs_resize:
|
||||
frame.thumbnail((MAX_IMAGE_REQUEST_DIMENSION, MAX_IMAGE_REQUEST_DIMENSION), image_module.LANCZOS)
|
||||
buffer = io.BytesIO()
|
||||
if frame.mode == "RGBA":
|
||||
frame.save(buffer, format="PNG")
|
||||
return buffer.getvalue(), "image/png"
|
||||
frame.save(buffer, format="JPEG", quality=_JPEG_QUALITY)
|
||||
return buffer.getvalue(), "image/jpeg"
|
||||
try:
|
||||
buffer = io.BytesIO()
|
||||
if frame.mode == "RGBA":
|
||||
frame.save(buffer, format="PNG")
|
||||
return buffer.getvalue(), "image/png", source_mime
|
||||
frame.save(buffer, format="JPEG", quality=_JPEG_QUALITY)
|
||||
return buffer.getvalue(), "image/jpeg", source_mime
|
||||
finally:
|
||||
frame.close()
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
raise RuntimeError(f"Failed to convert/resize image ({suffix or 'unknown suffix'}): {exc}") from exc
|
||||
|
||||
|
||||
def _build_image_request_payload(data: bytes, suffix: str) -> dict:
|
||||
def _build_image_request_payload(
|
||||
data: bytes,
|
||||
suffix: str,
|
||||
*,
|
||||
max_image_pixels: int = DEFAULT_MAX_IMAGE_PIXELS,
|
||||
) -> dict:
|
||||
"""Return ``{"data_b64", "mime", "source_mime", "converted"}`` for a VLM request.
|
||||
|
||||
``mime`` is the format actually sent (after in-memory downscale/re-encode);
|
||||
``source_mime`` describes the stored resource file and is what notes record.
|
||||
``source_mime`` is the decoded format of the stored resource file and is
|
||||
what notes record. Both are based on actual bytes, not the filename suffix.
|
||||
"""
|
||||
source_mime = IMAGE_MIME_BY_EXT.get(suffix, "image/png")
|
||||
normalized = _normalize_image_bytes(data, suffix)
|
||||
if normalized is None:
|
||||
return {
|
||||
"data_b64": base64.b64encode(data).decode("ascii"),
|
||||
"mime": source_mime,
|
||||
"source_mime": source_mime,
|
||||
"converted": False,
|
||||
}
|
||||
normalized_bytes, mime = normalized
|
||||
normalized_bytes, mime, source_mime = _normalize_image_bytes(
|
||||
data,
|
||||
suffix,
|
||||
max_image_pixels=max_image_pixels,
|
||||
)
|
||||
request_bytes = data if normalized_bytes is None else normalized_bytes
|
||||
return {
|
||||
"data_b64": base64.b64encode(normalized_bytes).decode("ascii"),
|
||||
"data_b64": base64.b64encode(request_bytes).decode("ascii"),
|
||||
"mime": mime,
|
||||
"source_mime": source_mime,
|
||||
"converted": normalized_bytes != data,
|
||||
"converted": normalized_bytes is not None,
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -193,7 +285,9 @@ class AutoImageResourceStep(BaseAutoResourceStep):
|
|||
"""
|
||||
|
||||
resource_suffixes = IMAGE_SUFFIXES
|
||||
router_inherit_keys = BaseAutoResourceStep.router_inherit_keys | frozenset({"as_llm", "max_image_bytes"})
|
||||
router_inherit_keys = BaseAutoResourceStep.router_inherit_keys | frozenset(
|
||||
{"as_llm", "max_image_bytes", "max_image_pixels"},
|
||||
)
|
||||
|
||||
def _max_image_bytes(self) -> int:
|
||||
"""Return the image read limit from Step or Job context."""
|
||||
|
|
@ -202,6 +296,17 @@ class AutoImageResourceStep(BaseAutoResourceStep):
|
|||
value = self.context.get("max_image_bytes")
|
||||
return int(value) if value is not None else DEFAULT_MAX_IMAGE_INPUT_BYTES
|
||||
|
||||
def _max_image_pixels(self) -> int:
|
||||
"""Return the deployment-controlled pre-decode pixel limit."""
|
||||
value = self.kwargs.get("max_image_pixels", DEFAULT_MAX_IMAGE_PIXELS)
|
||||
try:
|
||||
limit = int(value)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(f"max_image_pixels must be a positive integer: {value!r}") from exc
|
||||
if limit <= 0:
|
||||
raise ValueError(f"max_image_pixels must be a positive integer: {value!r}")
|
||||
return limit
|
||||
|
||||
def _vision_model(self) -> ChatModelBase | None:
|
||||
"""Resolve explicit ``as_llm`` through Ref, otherwise prefer vision/default."""
|
||||
context_model = self.context.get("as_llm") if self.context is not None else None
|
||||
|
|
@ -266,37 +371,49 @@ class AutoImageResourceStep(BaseAutoResourceStep):
|
|||
self.logger.warning(f"[{self.name}] resource stat failed file_path={file_path} error={exc}")
|
||||
return None
|
||||
if size_bytes > max_image_bytes:
|
||||
self.context.response.success = True
|
||||
self.context.response.answer = (
|
||||
f"Skipped oversized image resource file: {file_path} ({size_bytes} > {max_image_bytes} bytes)"
|
||||
)
|
||||
self.context.response.metadata.update(
|
||||
{
|
||||
"path": file_path,
|
||||
"action": "skipped",
|
||||
"reason": "file_too_large",
|
||||
"oversized": True,
|
||||
"size_bytes": size_bytes,
|
||||
"max_image_bytes": max_image_bytes,
|
||||
"modified": False,
|
||||
},
|
||||
)
|
||||
self.logger.warning(
|
||||
f"[{self.name}] skip oversized image resource file_path={file_path} "
|
||||
f"size_bytes={size_bytes} max_image_bytes={max_image_bytes}",
|
||||
)
|
||||
self._record_oversized_image(file_path, size_bytes, max_image_bytes)
|
||||
return None
|
||||
|
||||
self.logger.info(f"[{self.name}] read image start file_path={file_path}")
|
||||
async with aiofiles.open(source_path, "rb") as f:
|
||||
data = await f.read()
|
||||
payload = _build_image_request_payload(data, Path(file_path).suffix.lower())
|
||||
data = await f.read(max_image_bytes + 1)
|
||||
if len(data) > max_image_bytes:
|
||||
self._record_oversized_image(file_path, len(data), max_image_bytes)
|
||||
return None
|
||||
payload = _build_image_request_payload(
|
||||
data,
|
||||
Path(file_path).suffix.lower(),
|
||||
max_image_pixels=self._max_image_pixels(),
|
||||
)
|
||||
self.logger.info(
|
||||
f"[{self.name}] read image done file_path={file_path} size_bytes={size_bytes} "
|
||||
f"mime={payload['mime']} converted={payload['converted']}",
|
||||
f"mime={payload['mime']} source_mime={payload['source_mime']} converted={payload['converted']}",
|
||||
)
|
||||
return payload
|
||||
|
||||
def _record_oversized_image(self, file_path: str, size_bytes: int, max_image_bytes: int) -> None:
|
||||
"""Record a stable skip response for an image over the compressed-byte limit."""
|
||||
assert self.context is not None
|
||||
self.context.response.success = True
|
||||
self.context.response.answer = (
|
||||
f"Skipped oversized image resource file: {file_path} ({size_bytes} > {max_image_bytes} bytes)"
|
||||
)
|
||||
self.context.response.metadata.update(
|
||||
{
|
||||
"path": file_path,
|
||||
"action": "skipped",
|
||||
"reason": "file_too_large",
|
||||
"oversized": True,
|
||||
"size_bytes": size_bytes,
|
||||
"max_image_bytes": max_image_bytes,
|
||||
"modified": False,
|
||||
},
|
||||
)
|
||||
self.logger.warning(
|
||||
f"[{self.name}] skip oversized image resource file_path={file_path} "
|
||||
f"size_bytes={size_bytes} max_image_bytes={max_image_bytes}",
|
||||
)
|
||||
|
||||
async def _handle_upsert(
|
||||
self,
|
||||
file_path: str,
|
||||
|
|
|
|||
|
|
@ -236,6 +236,33 @@ class BaseAutoResourceStep(BaseStep):
|
|||
notes = list_response.metadata.get("notes") or []
|
||||
return self._find_resource_note(notes, file_path)
|
||||
|
||||
def _daily_note_days(self) -> list[str]:
|
||||
"""Return safe, deterministic daily subdirectories available for lookup."""
|
||||
workspace = self.workspace_path.resolve()
|
||||
daily_dir = str(self.config_value("daily_dir"))
|
||||
daily_root, error = resolve_path(workspace, daily_dir)
|
||||
if error or daily_root is None:
|
||||
raise ValueError(f"invalid daily_dir {daily_dir!r}: {error or 'cannot resolve path'}")
|
||||
if not daily_root.is_dir():
|
||||
return []
|
||||
return sorted(
|
||||
entry.name
|
||||
for entry in daily_root.iterdir()
|
||||
if _DATE_RE.fullmatch(entry.name) and entry.is_dir() and not entry.is_symlink()
|
||||
)
|
||||
|
||||
async def _find_loose_resource_day(self, file_path: str) -> str | None:
|
||||
"""Find the single daily-card owner for a root-level resource."""
|
||||
matches: list[tuple[str, str]] = []
|
||||
for day in self._daily_note_days():
|
||||
note = await self._list_resource_note(day, file_path)
|
||||
if note is not None:
|
||||
matches.append((day, str(note["path"])))
|
||||
if len(matches) > 1:
|
||||
paths = ", ".join(path for _, path in matches)
|
||||
raise RuntimeError(f"Multiple daily resource notes claim {file_path}: {paths}")
|
||||
return matches[0][0] if matches else None
|
||||
|
||||
async def _prepare_resource_note(self, day: str, file_path: str, note_stem: str) -> _ResourceNoteState:
|
||||
"""Find the owned note or allocate a safe path before the first write."""
|
||||
note = await self._list_resource_note(day, file_path)
|
||||
|
|
@ -492,8 +519,11 @@ class BaseAutoResourceStep(BaseStep):
|
|||
|
||||
loose_filename = _loose_resource_filename(file_path, resource_dir)
|
||||
if loose_filename:
|
||||
date_str, filename = self._today(), loose_filename
|
||||
self.logger.info(f"[{self.name}] loose resource file_path={file_path} date={date_str}")
|
||||
existing_day = await self._find_loose_resource_day(file_path)
|
||||
date_str, filename = existing_day or self._today(), loose_filename
|
||||
self.logger.info(
|
||||
f"[{self.name}] loose resource file_path={file_path} date={date_str} existing={bool(existing_day)}",
|
||||
)
|
||||
else:
|
||||
date_str, filename = _parse_resource_path(file_path, resource_dir)
|
||||
|
||||
|
|
|
|||
|
|
@ -210,13 +210,14 @@ def caption_json(name: str, description: str, caption: str) -> str:
|
|||
return json.dumps({"name": name, "description": description, "caption": caption})
|
||||
|
||||
|
||||
def image_processor(app_context, file_store, model, *, routed: bool):
|
||||
def image_processor(app_context, file_store, model, *, routed: bool, **kwargs):
|
||||
"""Build either the image processor or the public unified-router path."""
|
||||
if not routed:
|
||||
return AutoImageResourceStep(app_context=app_context, file_store=file_store, as_llm=model)
|
||||
return AutoImageResourceStep(app_context=app_context, file_store=file_store, as_llm=model, **kwargs)
|
||||
app_context.registry = R
|
||||
return AutoResourceStep(
|
||||
app_context=app_context,
|
||||
**kwargs,
|
||||
dispatch_steps=[
|
||||
{"backend": "auto_image_resource_step", "file_store": file_store, "as_llm": model},
|
||||
{
|
||||
|
|
@ -244,9 +245,9 @@ class AutoResourceTestEnv:
|
|||
"""Write a source-owned note relative to this workspace."""
|
||||
return write_note(self.workspace / relative_path, source_resource, body)
|
||||
|
||||
def processor(self, model, *, routed: bool = False):
|
||||
def processor(self, model, *, routed: bool = False, **kwargs):
|
||||
"""Build the direct processor or unified router for this workspace."""
|
||||
return image_processor(self.app_context, self.file_store, model, routed=routed)
|
||||
return image_processor(self.app_context, self.file_store, model, routed=routed, **kwargs)
|
||||
|
||||
async def run(self, step, changes, **context_kwargs):
|
||||
"""Run one processor invocation with a fresh runtime context."""
|
||||
|
|
|
|||
|
|
@ -9,9 +9,11 @@ with PIL inside a temporary workspace.
|
|||
import base64
|
||||
import hashlib
|
||||
import io
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import tomllib
|
||||
import zlib
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
|
@ -19,7 +21,7 @@ from unittest.mock import patch
|
|||
import frontmatter
|
||||
import pytest
|
||||
import yaml
|
||||
from PIL import Image
|
||||
from PIL import Image, JpegImagePlugin
|
||||
|
||||
from reme.components import R
|
||||
from reme.components.component_registry import ComponentRegistry
|
||||
|
|
@ -28,6 +30,7 @@ from reme.components.runtime_context import RuntimeContext
|
|||
from reme.enumeration import ComponentEnum
|
||||
from reme.steps.evolve.auto_image_resource import (
|
||||
AutoImageResourceStep,
|
||||
DEFAULT_MAX_IMAGE_PIXELS,
|
||||
_build_image_request_payload,
|
||||
_normalize_image_bytes,
|
||||
_parse_caption_json,
|
||||
|
|
@ -52,6 +55,14 @@ from .auto_resource_test_support import (
|
|||
pytest_plugins = ("unit.auto_resource_test_plugin",)
|
||||
|
||||
|
||||
def _png_bytes_with_header_size(width: int, height: int) -> bytes:
|
||||
"""Change only a tiny PNG's IHDR dimensions without allocating its pixels."""
|
||||
data = bytearray(_png_bytes())
|
||||
data[16:24] = struct.pack(">II", width, height)
|
||||
data[29:33] = struct.pack(">I", zlib.crc32(data[12:29]) & 0xFFFFFFFF)
|
||||
return bytes(data)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("image_format", "suffix", "source_mime", "request_mime"),
|
||||
[
|
||||
|
|
@ -178,6 +189,197 @@ async def test_auto_image_downscales_oversized_image_for_request_only(auto_resou
|
|||
assert (env.workspace / "daily/2026-01-01/huge-image.md").is_file()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_auto_image_uses_jpeg_decoder_downsampling_before_load(auto_resource_env):
|
||||
"""Large JPEGs lower their decoder allocation before full pixel decode."""
|
||||
env = auto_resource_env
|
||||
source = env.write_binary("resource/2026-01-01/large.jpg", _img_bytes("JPEG", (4096, 2048)))
|
||||
stored_bytes = source.read_bytes()
|
||||
model = _FakeVisionModel(_caption_json("large-jpeg", "Large", "A large JPEG."))
|
||||
decoded_sizes = []
|
||||
original_load = JpegImagePlugin.JpegImageFile.load
|
||||
|
||||
def record_decoder_size(image, *args, **kwargs):
|
||||
decoded_sizes.append(image.size)
|
||||
return original_load(image, *args, **kwargs)
|
||||
|
||||
with patch.object(JpegImagePlugin.JpegImageFile, "load", new=record_decoder_size):
|
||||
response = await env.run(env.processor(model), [{"change": "added", "path": str(source)}])
|
||||
|
||||
assert response.success is True
|
||||
assert decoded_sizes
|
||||
assert decoded_sizes[0] == (2048, 1024)
|
||||
data_block = model.calls[0][0].content[1]
|
||||
with Image.open(io.BytesIO(base64.b64decode(data_block.source.data))) as sent:
|
||||
assert max(sent.size) <= 2048
|
||||
assert source.read_bytes() == stored_bytes
|
||||
|
||||
|
||||
def test_image_preprocessing_resizes_16_bit_tiff_before_rgb_conversion():
|
||||
"""Pillow 10-compatible scaling keeps large I;16 TIFF images memory-bounded."""
|
||||
image = Image.new("I;16", (2049, 2), 1000)
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="TIFF")
|
||||
calls = []
|
||||
original_thumbnail = Image.Image.thumbnail
|
||||
|
||||
def record_thumbnail(frame, size, resample, *args, **kwargs):
|
||||
calls.append((frame.mode, resample))
|
||||
return original_thumbnail(frame, size, resample, *args, **kwargs)
|
||||
|
||||
with patch.object(Image.Image, "thumbnail", new=record_thumbnail):
|
||||
payload = _build_image_request_payload(buffer.getvalue(), ".tiff")
|
||||
|
||||
assert calls == [("I;16", Image.Resampling.NEAREST)]
|
||||
assert payload["mime"] == "image/jpeg"
|
||||
assert payload["source_mime"] == "image/tiff"
|
||||
with Image.open(io.BytesIO(base64.b64decode(payload["data_b64"]))) as sent:
|
||||
assert max(sent.size) <= 2048
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_auto_image_applies_exif_orientation_before_resizing(auto_resource_env):
|
||||
"""A rotated phone JPEG is normalized upright for the VLM without touching its source."""
|
||||
env = auto_resource_env
|
||||
image = Image.new("RGB", (3000, 1000), (40, 80, 120))
|
||||
exif = Image.Exif()
|
||||
exif[274] = 6
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="JPEG", exif=exif)
|
||||
source = env.write_binary("resource/2026-01-01/phone.jpg", buffer.getvalue())
|
||||
stored_bytes = source.read_bytes()
|
||||
model = _FakeVisionModel(_caption_json("upright-phone-photo", "Upright", "An upright phone photo."))
|
||||
|
||||
response = await env.run(env.processor(model), [{"change": "added", "path": str(source)}])
|
||||
|
||||
assert response.success is True
|
||||
data_block = model.calls[0][0].content[1]
|
||||
assert data_block.source.media_type == "image/jpeg"
|
||||
with Image.open(io.BytesIO(base64.b64decode(data_block.source.data))) as sent:
|
||||
assert sent.size == (683, 2048)
|
||||
assert sent.getexif().get(274) is None
|
||||
assert source.read_bytes() == stored_bytes
|
||||
assert (env.workspace / "daily/2026-01-01/upright-phone-photo.md").is_file()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("image_format", "suffix", "source_mime", "request_mime"),
|
||||
[
|
||||
("JPEG", ".png", "image/jpeg", "image/jpeg"),
|
||||
("PNG", ".jpg", "image/png", "image/png"),
|
||||
("BMP", ".png", "image/bmp", "image/jpeg"),
|
||||
],
|
||||
)
|
||||
@pytest.mark.asyncio
|
||||
async def test_auto_image_uses_decoded_format_when_suffix_is_misleading(
|
||||
image_format,
|
||||
suffix,
|
||||
source_mime,
|
||||
request_mime,
|
||||
auto_resource_env,
|
||||
):
|
||||
"""Request and note MIME values come from decoded bytes, with conversion when needed."""
|
||||
env = auto_resource_env
|
||||
source = env.write_binary(f"resource/2026-01-01/mislabeled{suffix}", _img_bytes(image_format))
|
||||
stored_bytes = source.read_bytes()
|
||||
model = _StructuredVisionModel(
|
||||
content={"name": "actual-format", "description": "Decoded", "caption": "Decoded image content."},
|
||||
)
|
||||
|
||||
response = await env.run(env.processor(model), [{"change": "added", "path": str(source)}])
|
||||
|
||||
assert response.success is True
|
||||
data_block = model.structured_calls[0][0].content[1]
|
||||
assert data_block.source.media_type == request_mime
|
||||
with Image.open(io.BytesIO(base64.b64decode(data_block.source.data))) as sent:
|
||||
assert sent.get_format_mimetype() == request_mime
|
||||
note = frontmatter.load(env.workspace / "daily/2026-01-01/actual-format.md")
|
||||
assert note.metadata["media_type"] == source_mime
|
||||
assert source.read_bytes() == stored_bytes
|
||||
|
||||
|
||||
@pytest.mark.parametrize("routed", [False, True], ids=["image", "unified-router"])
|
||||
@pytest.mark.asyncio
|
||||
async def test_auto_image_rejects_pixel_bomb_before_decode_and_isolates_batch(routed, auto_resource_env):
|
||||
"""A small compressed file with unsafe dimensions fails without stopping the next image."""
|
||||
env = auto_resource_env
|
||||
unsafe = env.write_binary(
|
||||
"resource/2026-01-01/pixel-bomb.png",
|
||||
_png_bytes_with_header_size(8000, 6000),
|
||||
)
|
||||
safe = env.write_binary("resource/2026-01-01/safe.png", _png_bytes())
|
||||
model = _FakeVisionModel(_caption_json("safe-image", "Safe", "A safe image."))
|
||||
|
||||
response = await env.run(
|
||||
env.processor(model, routed=routed),
|
||||
[
|
||||
{"change": "added", "path": str(unsafe)},
|
||||
{"change": "added", "path": str(safe)},
|
||||
],
|
||||
)
|
||||
|
||||
results = response.metadata["results"]
|
||||
assert response.success is False
|
||||
assert [item["success"] for item in results] == [False, True]
|
||||
assert results[0]["metadata"]["action"] == "failed"
|
||||
assert results[0]["metadata"]["modified"] is False
|
||||
assert f"48000000 > {DEFAULT_MAX_IMAGE_PIXELS}" in results[0]["metadata"]["error"]
|
||||
assert len(model.calls) == 1
|
||||
assert not (env.workspace / "daily/2026-01-01/pixel-bomb.md").exists()
|
||||
assert (env.workspace / "daily/2026-01-01/safe-image.md").is_file()
|
||||
|
||||
|
||||
def test_image_pixel_limit_is_checked_before_full_decode():
|
||||
"""The explicit pixel budget is enforced from image headers before ``load``."""
|
||||
with patch("PIL.PngImagePlugin.PngImageFile.load", side_effect=AssertionError("must not decode")) as image_load:
|
||||
with pytest.raises(RuntimeError, match=r"8x8=64 > 63"):
|
||||
_normalize_image_bytes(_png_bytes(), ".png", max_image_pixels=63)
|
||||
image_load.assert_not_called()
|
||||
|
||||
|
||||
def test_image_decompression_bomb_warning_becomes_an_error(monkeypatch):
|
||||
"""Pillow's warning-only bomb threshold becomes a reportable processor error."""
|
||||
monkeypatch.setattr(Image, "MAX_IMAGE_PIXELS", 32)
|
||||
|
||||
with pytest.raises(RuntimeError, match="decompression-bomb protection"):
|
||||
_build_image_request_payload(_png_bytes(), ".png")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("routed", [False, True], ids=["image", "unified-router"])
|
||||
@pytest.mark.asyncio
|
||||
async def test_decompression_bomb_warning_is_isolated_per_change(routed, auto_resource_env, monkeypatch):
|
||||
"""A Pillow bomb warning fails one resource while a safe batch peer still completes."""
|
||||
env = auto_resource_env
|
||||
warned = env.write_binary("resource/2026-01-01/warned.png", _png_bytes())
|
||||
safe = env.write_binary("resource/2026-01-01/safe.png", _png_bytes(width=4, height=4))
|
||||
model = _FakeVisionModel(_caption_json("safe-image", "Safe", "A safe image."))
|
||||
monkeypatch.setattr(Image, "MAX_IMAGE_PIXELS", 32)
|
||||
|
||||
response = await env.run(
|
||||
env.processor(model, routed=routed),
|
||||
[
|
||||
{"change": "added", "path": str(warned)},
|
||||
{"change": "added", "path": str(safe)},
|
||||
],
|
||||
)
|
||||
|
||||
results = response.metadata["results"]
|
||||
assert response.success is False
|
||||
assert [item["success"] for item in results] == [False, True]
|
||||
assert "decompression-bomb protection" in results[0]["metadata"]["error"]
|
||||
assert len(model.calls) == 1
|
||||
assert not (env.workspace / "daily/2026-01-01/warned.md").exists()
|
||||
assert (env.workspace / "daily/2026-01-01/safe-image.md").is_file()
|
||||
|
||||
|
||||
def test_auto_image_pixel_limit_cannot_be_raised_by_runtime_context():
|
||||
"""Request-scoped kwargs cannot relax the processor's deployment safety cap."""
|
||||
step = AutoImageResourceStep(max_image_pixels=63)
|
||||
step.context = RuntimeContext(max_image_pixels=DEFAULT_MAX_IMAGE_PIXELS)
|
||||
|
||||
assert step._max_image_pixels() == 63
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_auto_image_skips_oversized_file(auto_resource_env):
|
||||
"""Files beyond max_image_bytes are skipped without a VLM call."""
|
||||
|
|
@ -334,8 +536,9 @@ def test_auto_resource_router_inherits_declared_options_with_child_override():
|
|||
prompt_dict=prompt_dict,
|
||||
max_file_bytes=4,
|
||||
max_image_bytes=8,
|
||||
max_image_pixels=64,
|
||||
dispatch_steps=[
|
||||
{"backend": "auto_image_resource_step", "max_image_bytes": 32},
|
||||
{"backend": "auto_image_resource_step", "max_image_bytes": 32, "max_image_pixels": 128},
|
||||
{"backend": "auto_text_resource_step", "max_file_bytes": 16},
|
||||
],
|
||||
)
|
||||
|
|
@ -355,8 +558,16 @@ def test_auto_resource_router_inherits_declared_options_with_child_override():
|
|||
"as_llm": vision_model,
|
||||
"language": "zh",
|
||||
"max_image_bytes": 32,
|
||||
"max_image_pixels": 128,
|
||||
}
|
||||
|
||||
inherited = AutoResourceStep(
|
||||
max_image_pixels=64,
|
||||
dispatch_steps=["auto_image_resource_step", "auto_text_resource_step"],
|
||||
)
|
||||
inherited_specs = {spec["backend"]: spec for spec, _, _ in inherited._processor_routes()}
|
||||
assert inherited_specs["auto_image_resource_step"]["max_image_pixels"] == 64
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_auto_resource_router_accepts_a_registered_third_modality_without_code_changes(auto_resource_env):
|
||||
|
|
@ -511,13 +722,18 @@ def test_auto_image_named_model_uses_standard_ref_resolution():
|
|||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("blocked_module", "suffix", "error_pattern"),
|
||||
("blocked_module", "payload", "suffix", "error_pattern"),
|
||||
[
|
||||
("PIL", ".png", r"Pillow.*reme-ai\[core\]"),
|
||||
("pillow_heif", ".heic", r"pillow-heif.*reme-ai\[image-heif\]"),
|
||||
("PIL", _png_bytes(), ".png", r"Pillow.*reme-ai\[core\]"),
|
||||
(
|
||||
"pillow_heif",
|
||||
b"\x00\x00\x00\x18ftypheic\x00\x00\x00\x00mif1heic",
|
||||
".heic",
|
||||
r"pillow-heif.*reme-ai\[image-heif\]",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_image_preprocessing_reports_dependency_errors(blocked_module, suffix, error_pattern):
|
||||
def test_image_preprocessing_reports_dependency_errors(blocked_module, payload, suffix, error_pattern):
|
||||
"""Lazy image dependencies produce actionable installation errors."""
|
||||
real_import = __import__
|
||||
|
||||
|
|
@ -528,7 +744,24 @@ def test_image_preprocessing_reports_dependency_errors(blocked_module, suffix, e
|
|||
|
||||
with patch("builtins.__import__", side_effect=import_without_dependency):
|
||||
with pytest.raises(RuntimeError, match=error_pattern):
|
||||
_normalize_image_bytes(_png_bytes(), suffix)
|
||||
_normalize_image_bytes(payload, suffix)
|
||||
|
||||
|
||||
def test_misleading_heic_suffix_does_not_load_optional_dependency():
|
||||
"""A core image named ``.heic`` is decoded by content without loading pillow-heif."""
|
||||
real_import = __import__
|
||||
|
||||
def import_without_heif(name, *args, **kwargs):
|
||||
if name == "pillow_heif":
|
||||
raise AssertionError("pillow-heif must not be loaded for PNG bytes")
|
||||
return real_import(name, *args, **kwargs)
|
||||
|
||||
with patch("builtins.__import__", side_effect=import_without_heif):
|
||||
payload = _build_image_request_payload(_png_bytes(), ".heic")
|
||||
|
||||
assert payload["mime"] == "image/png"
|
||||
assert payload["source_mime"] == "image/png"
|
||||
assert payload["converted"] is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
|
|
|
|||
|
|
@ -1,8 +1,12 @@
|
|||
"""Regression tests for the safety findings from the Auto Resource PR review."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import frontmatter
|
||||
import pytest
|
||||
|
||||
from reme.steps.evolve.base_auto_resource import BaseAutoResourceStep
|
||||
|
||||
from .auto_resource_test_support import (
|
||||
FakeVisionModel,
|
||||
StructuredVisionModel,
|
||||
|
|
@ -167,3 +171,120 @@ async def test_blank_plain_caption_does_not_create_or_overwrite_note(routed, pla
|
|||
assert not (env.workspace / "daily/2026-01-01/blank-new.md").exists()
|
||||
assert old_note.read_bytes() == before
|
||||
assert len(model.structured_calls) == len(model.plain_calls) == 2
|
||||
|
||||
|
||||
@pytest.mark.parametrize("routed", [False, True], ids=["image", "unified-router"])
|
||||
async def test_loose_root_image_keeps_original_daily_card_across_days(routed, auto_resource_env):
|
||||
"""Later updates and deletion keep a loose resource's first daily-card ownership."""
|
||||
env = auto_resource_env
|
||||
source = env.write_binary("resource/photo.png", image_bytes(color=(200, 30, 30)))
|
||||
initial_model = FakeVisionModel(caption_json("original-card", "Original", "first-day caption"))
|
||||
with patch.object(BaseAutoResourceStep, "_today", return_value="2026-01-01"):
|
||||
added = await env.run(
|
||||
env.processor(initial_model, routed=routed),
|
||||
[{"change": "added", "path": str(source)}],
|
||||
)
|
||||
|
||||
owned_note = env.workspace / "daily/2026-01-01/original-card.md"
|
||||
add_result = added.metadata["results"][0]["metadata"]
|
||||
assert added.success is True
|
||||
assert add_result["path"] == "daily/2026-01-01/original-card.md"
|
||||
assert add_result["action"] == "added"
|
||||
assert add_result["index"]["date"] == "2026-01-01"
|
||||
assert "first-day caption" in owned_note.read_text(encoding="utf-8")
|
||||
|
||||
unrelated_note = env.write_note(
|
||||
"daily/2026-01-02/photo.md",
|
||||
"[[resource/other.png]]",
|
||||
body="unrelated note that must survive",
|
||||
)
|
||||
unrelated_before = unrelated_note.read_bytes()
|
||||
|
||||
first_model = FakeVisionModel(caption_json("renamed-on-day-two", "Updated", "second-day caption"))
|
||||
with patch.object(BaseAutoResourceStep, "_today", return_value="2026-01-02"):
|
||||
first_update = await env.run(
|
||||
env.processor(first_model, routed=routed),
|
||||
[{"change": "modified", "path": str(source)}],
|
||||
)
|
||||
|
||||
first_result = first_update.metadata["results"][0]["metadata"]
|
||||
assert first_update.success is True
|
||||
assert first_result["path"] == "daily/2026-01-01/original-card.md"
|
||||
assert first_result["action"] == "modified"
|
||||
assert first_result["created"] is False
|
||||
assert first_result["index"]["date"] == "2026-01-01"
|
||||
assert "second-day caption" in owned_note.read_text(encoding="utf-8")
|
||||
assert not (env.workspace / "daily/2026-01-02/renamed-on-day-two.md").exists()
|
||||
assert unrelated_note.read_bytes() == unrelated_before
|
||||
assert not (env.workspace / "daily/2026-01-02.md").exists()
|
||||
|
||||
source.write_bytes(image_bytes(color=(20, 90, 200)))
|
||||
second_model = FakeVisionModel(caption_json("renamed-on-day-three", "Updated again", "third-day caption"))
|
||||
with patch.object(BaseAutoResourceStep, "_today", return_value="2026-01-03"):
|
||||
second_update = await env.run(
|
||||
env.processor(second_model, routed=routed),
|
||||
[{"change": "modified", "path": str(source)}],
|
||||
)
|
||||
|
||||
second_result = second_update.metadata["results"][0]["metadata"]
|
||||
assert second_update.success is True
|
||||
assert second_result["path"] == "daily/2026-01-01/original-card.md"
|
||||
assert second_result["action"] == "modified"
|
||||
assert second_result["created"] is False
|
||||
assert second_result["index"]["date"] == "2026-01-01"
|
||||
assert "third-day caption" in owned_note.read_text(encoding="utf-8")
|
||||
assert not (env.workspace / "daily/2026-01-03/renamed-on-day-three.md").exists()
|
||||
assert unrelated_note.read_bytes() == unrelated_before
|
||||
assert not (env.workspace / "daily/2026-01-03.md").exists()
|
||||
|
||||
source.unlink()
|
||||
delete_model = FakeVisionModel(caption_json("unused", "Unused", "Must not be requested."))
|
||||
with patch.object(BaseAutoResourceStep, "_today", return_value="2026-01-04"):
|
||||
deleted = await env.run(
|
||||
env.processor(delete_model, routed=routed),
|
||||
[{"change": "deleted", "path": str(source)}],
|
||||
)
|
||||
|
||||
delete_result = deleted.metadata["results"][0]["metadata"]
|
||||
assert deleted.success is True
|
||||
assert delete_result["path"] == "daily/2026-01-01/original-card.md"
|
||||
assert delete_result["action"] == "deleted"
|
||||
assert delete_result["modified"] is True
|
||||
assert delete_result["index"]["date"] == "2026-01-01"
|
||||
assert delete_result["index"]["notes"] == []
|
||||
assert not owned_note.exists()
|
||||
assert unrelated_note.read_bytes() == unrelated_before
|
||||
assert "(none)" in (env.workspace / "daily/2026-01-01.md").read_text(encoding="utf-8")
|
||||
assert not (env.workspace / "daily/2026-01-04.md").exists()
|
||||
assert len(initial_model.calls) == len(first_model.calls) == len(second_model.calls) == 1
|
||||
assert not delete_model.calls
|
||||
|
||||
|
||||
@pytest.mark.parametrize("routed", [False, True], ids=["image", "unified-router"])
|
||||
async def test_loose_root_image_duplicate_daily_owners_fail_closed(routed, auto_resource_env):
|
||||
"""Ambiguous exact ownership is reported without reading the image model or changing notes."""
|
||||
env = auto_resource_env
|
||||
source = env.write_binary("resource/duplicate.png", image_bytes())
|
||||
first_note = env.write_note("daily/2026-01-01/first.md", "[[resource/duplicate.png]]", body="first owner")
|
||||
second_note = env.write_note("daily/2026-01-02/second.md", "[[resource/duplicate.png]]", body="second owner")
|
||||
before = {first_note: first_note.read_bytes(), second_note: second_note.read_bytes()}
|
||||
model = FakeVisionModel(caption_json("replacement", "Replacement", "Must not be generated."))
|
||||
|
||||
with patch.object(BaseAutoResourceStep, "_today", return_value="2026-01-03"):
|
||||
response = await env.run(
|
||||
env.processor(model, routed=routed),
|
||||
[{"change": "modified", "path": str(source)}],
|
||||
)
|
||||
|
||||
result = response.metadata["results"][0]["metadata"]
|
||||
assert response.success is False
|
||||
assert result["action"] == "failed"
|
||||
assert result["modified"] is False
|
||||
assert "Multiple daily resource notes claim resource/duplicate.png" in result["error"]
|
||||
assert "daily/2026-01-01/first.md" in result["error"]
|
||||
assert "daily/2026-01-02/second.md" in result["error"]
|
||||
assert not model.calls
|
||||
assert all(path.read_bytes() == contents for path, contents in before.items())
|
||||
assert not (env.workspace / "daily/2026-01-01.md").exists()
|
||||
assert not (env.workspace / "daily/2026-01-02.md").exists()
|
||||
assert not (env.workspace / "daily/2026-01-03.md").exists()
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue