ReMe/tests4/unit/test_read_image_steps.py
jinliyl d8086039dc
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
refactor(Agentscope2.0): llm & embedding & agent (#271)
* refactor(embedding): replace embedding model with embedding store architecture

- Remove as_token_counter component and its estimated token counter implementation
- Replace BaseEmbeddingModel with BaseEmbedding that wraps AgentScope embedding models
- Add support for multiple embedding providers (OpenAI, DashScope, Gemini, Ollama)
- Introduce BaseEmbeddingStore and LocalEmbeddingStore for caching and persistence
- Update component registry to use new embedding and embedding_store types
- Modify file stores to use embedding_store instead of embedding_model
- Update health check to monitor embedding_store instead of embedding_model
- Change default config to use embedding_store with local backend
- Add estimate_token_count utility function to utils module

* refactor(llm): replace as_llm components with unified llm implementation

- Remove deprecated as_llm and as_llm_formatter modules
- Add new llm module with BaseLLM and provider-specific implementations
- Update component registry to use LLM instead of AS_LLM
- Replace all as_llm/as_llm_formatter references with llm in steps
- Update configuration schema to use llm instead of as_llm
- Rename integration test file from test_as_llm to test_llm
- Add proper docstrings to embedding store dimension property
- Add pylint disable comment for embedding model call
- Remove unused FormatterBase import in base_step
- Update token_utils with function docstring

* refactor(evolve): replace ReActAgent with Agent and update message handling

- Removed FlexReActAgent class and direct ReActAgent imports
- Updated Agent instantiation to use new constructor parameters
- Changed message content to use TextBlock format instead of plain strings
- Modified timestamp access from msg.timestamp to msg.created_at
- Updated metadata access pattern for structured outputs
- Replaced Msg.from_dict with Msg.model_validate in auto_memory.py
- Updated test mocks to patch Agent instead of ReActAgent
- Changed message serialization from to_dict to model_dump in tests
- Moved component references to base class definition
- Updated demo tools to return strings instead of ToolResponse objects

* feat(step): migrate to FunctionTool and add streaming support

- Replace deprecated ToolResponse with FunctionTool in base_step.py
- Remove unused TextBlock import from base_step.py
- Update job registration to use new FunctionTool API
- Add thinking_budget parameter to llm_demo configuration
- Introduce StreamLLMDemoStep with streaming output capability
- Add structured output support to LLMDemoStep via generate_structured_output
- Implement streaming event handling for text/thinking/tool calls
- Add integration tests for embedding functionality
- Add integration tests for structured output and streaming features
- Update tool usage in demo steps to use new function naming convention

* fix(ci): correct package installation path in unittest workflow

- Updated pip install command to use proper package path "./reme4[dev,core]"
- Fixed dependency installation step in CI workflow configuration

* chore(workflow): update python versions in unittest workflow

- Remove Python 3.10 from test matrix
- Add Python 3.11 to test matrix
- Add Python 3.12 to test matrix
- Keep Python 3.13 in test matrix
- Update matrix configuration for better version coverage

* fix(health): handle missing dimensions attribute in embedding status

- Wrap dimensions access in try-except to prevent AttributeError
- Return None when dimensions attribute is not available
- Maintain backward compatibility for components without dimensions

test(component): add comprehensive tests for BaseComponent and related classes

- Add tests for Dependency class including repr and attribute access
- Add tests for bind method with various scenarios and edge cases
- Add tests for lifecycle management and async context handling
- Add tests for standalone and context-bound dependency resolution
- Add tests for ComponentMixin path utilities

test(common): update LocalFileStore initialization parameter

- Change embedding_model parameter to embedding_store in test setup
- Update all affected test files consistently

test(registry): add complete test suite for ComponentRegistry

- Add tests for register method with explicit names and defaults
- Add tests for decorator registration pattern
- Add tests for get_all method returning copies
- Add tests for unregister and clear operations
- Add tests for error handling of invalid registrations

test(job): add comprehensive tests for BaseJob and BackgroundJob

- Add tests for step resolution and exception handling
- Add tests for backoff delay calculation with jitter
- Add tests for supervisor loop restart behavior
- Add tests for task shutdown and cancellation

test(prompt): add complete test suite for PromptHandler

- Add tests for prompt loading from dictionaries and files
- Add tests for internationalization and language fallback
- Add tests for flag filtering and variable substitution
- Add tests for format validation and error handling

test(runtime): add basic tests for RuntimeContext dictionary access

- Add tests for item getting, setting and containment checks
- Add tests for missing key error handling

* feat(evolve): add permission context and agent state management

- Import PermissionContext, PermissionMode and AgentState modules
- Add state configuration with bypass permission mode to AutoDream agents
- Add state configuration with bypass permission mode to AutoMemory agents
- Implement static _to_msg method for message validation and formatting
- Refactor message processing to use the new _to_msg method
- Ensure proper content structure for text blocks in message conversion

* style(tests): update test files with linting rules and code improvements

- Add missing pylint disable directives for docstring and attribute warnings
- Replace lambda expressions with proper function definitions in test cases
- Import Path directly instead of using lambda with __import__
- Simplify assertion checks by using truthiness instead of equality to empty dict
- Remove unused imports and reorder imports consistently
- Format dictionary literals with proper indentation and line breaks
2026-06-03 11:35:43 +08:00

233 lines
8.4 KiB
Python

"""Tests for the ``read_image`` step.
Style mirrors ``test_crud_steps.py`` — direct step invocation against a
freshly built ``LocalFileStore`` with the step's ``vault_path`` rooted at
``cwd()`` via chdir.
Coverage:
- happy paths for known suffixes (png/jpeg)
- oversized branch (``answer`` is a notice, ``metadata.oversized=True``)
- unknown / missing suffix (compatibility mode; ``non_image_warning=True``)
- error branches: missing file, directory, empty path, bad ``max_bytes``
"""
# pylint: disable=protected-access
import asyncio
import base64
import os
import tempfile
import warnings
from pathlib import Path
from reme4.components.file_store import LocalFileStore
from reme4.steps.file_io import read_image as crud_read_image
warnings.filterwarnings("ignore", category=DeprecationWarning, module="jieba")
warnings.filterwarnings("ignore", category=DeprecationWarning, module="pkg_resources")
class temp_chdir:
"""Context manager to temporarily chdir into a path and restore on exit."""
def __init__(self, path):
self.path = path
self.old = None
def __enter__(self):
self.old = os.getcwd()
os.chdir(self.path)
return self
def __exit__(self, *exc):
os.chdir(self.old)
def _run(coro):
asyncio.run(coro)
async def _make_store() -> LocalFileStore:
store = LocalFileStore(name="t_img", embedding_store="")
await store.start()
return store
def _seed_bytes(rel: str, data: bytes) -> Path:
"""Drop raw bytes at ``cwd/rel``. Step is byte-level — no real PNG needed."""
target = Path.cwd() / rel
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(data)
return target
async def _read_image(store: LocalFileStore, *, step_kwargs: dict | None = None, **call_kwargs):
"""Run ReadImageStep; ``step_kwargs`` go to step init (kwargs/attrs)."""
step = crud_read_image.ReadImageStep(file_store=store, **(step_kwargs or {}))
await step(**call_kwargs)
return step.context.response
def test_read_image_png():
"""``read_image path=img/cat.png`` returns base64 + ``image/png`` mime."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
payload = b"\x89PNG\r\n\x1a\n" + b"fake-png-body-bytes"
_seed_bytes("img/cat.png", payload)
store = await _make_store()
resp = await _read_image(store, path="img/cat.png")
assert resp.success is True, resp
assert base64.b64decode(resp.answer) == payload, "base64 round-trip mismatch"
assert resp.metadata["mime"] == "image/png", resp.metadata
assert resp.metadata["size_bytes"] == len(payload), resp.metadata
assert "oversized" not in resp.metadata, resp.metadata
await store.close()
print("✓ test_read_image_png passed")
_run(run())
def test_read_image_jpeg():
"""``.jpg`` suffix maps to ``image/jpeg``."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
payload = b"\xff\xd8\xff\xe0" + b"jpeg-body"
_seed_bytes("dog.jpg", payload)
store = await _make_store()
resp = await _read_image(store, path="dog.jpg")
assert resp.success is True, resp
assert base64.b64decode(resp.answer) == payload
assert resp.metadata["mime"] == "image/jpeg", resp.metadata
await store.close()
print("✓ test_read_image_jpeg passed")
_run(run())
def test_read_image_oversized():
"""Above ``max_bytes`` → ``answer`` is a notice, ``metadata.oversized=True``."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
payload = b"\x89PNG\r\n\x1a\n" + b"x" * 2048
_seed_bytes("big.png", payload)
store = await _make_store()
resp = await _read_image(store, step_kwargs={"max_bytes": 1024}, path="big.png")
assert resp.success is True, resp
assert resp.metadata["oversized"] is True, resp.metadata
assert resp.metadata["max_bytes"] == 1024, resp.metadata
assert resp.metadata["size_bytes"] == len(payload), resp.metadata
assert resp.metadata["mime"] == "image/png", resp.metadata
try:
decoded = base64.b64decode(resp.answer, validate=True)
assert decoded != payload, "oversized branch must not return real base64"
except Exception:
pass # expected — answer is notice text, not base64
assert "exceeds max_bytes" in resp.answer
await store.close()
print("✓ test_read_image_oversized passed")
_run(run())
def test_read_image_unknown_suffix():
"""Unknown suffix → still returns base64, ``metadata.non_image_warning=True``."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
payload = b"any-bytes-here-for-blob"
_seed_bytes("blob.xyz", payload)
store = await _make_store()
resp = await _read_image(store, path="blob.xyz")
assert resp.success is True, resp
assert base64.b64decode(resp.answer) == payload
assert resp.metadata["non_image_warning"] is True, resp.metadata
assert resp.metadata["mime"] is None, resp.metadata
await store.close()
print("✓ test_read_image_unknown_suffix passed")
_run(run())
def test_read_image_no_suffix():
"""No suffix → compatibility mode (no auto-append), still reads as base64."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
payload = b"\x89PNG\r\n\x1a\nbody"
_seed_bytes("no_suffix_blob", payload)
store = await _make_store()
resp = await _read_image(store, path="no_suffix_blob")
assert resp.success is True, resp
assert resp.metadata["non_image_warning"] is True, resp.metadata
assert resp.metadata["mime"] is None, resp.metadata
assert base64.b64decode(resp.answer) == payload
await store.close()
print("✓ test_read_image_no_suffix passed")
_run(run())
def test_read_image_missing():
"""Non-existent path → ``success=False`` with ``does not exist`` message."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store()
resp = await _read_image(store, path="never_existed.png")
assert resp.success is False, resp
assert resp.answer.startswith("Error:"), resp
assert "does not exist" in resp.answer
await store.close()
print("✓ test_read_image_missing passed")
_run(run())
def test_read_image_is_directory():
"""Path pointing to a directory → ``success=False`` with ``is not a file``."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
(Path(tmp) / "subdir").mkdir(parents=True, exist_ok=True)
store = await _make_store()
resp = await _read_image(store, path="subdir")
assert resp.success is False, resp
assert "is not a file" in resp.answer, resp
await store.close()
print("✓ test_read_image_is_directory passed")
_run(run())
def test_read_image_path_required():
"""Empty ``path`` → ``success=False`` with ``path is required`` message."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _make_store()
resp = await _read_image(store, path="")
assert resp.success is False, resp
assert "`path` is required" in resp.answer, resp
await store.close()
print("✓ test_read_image_path_required passed")
_run(run())
def test_read_image_invalid_max_bytes():
"""``max_bytes=-1`` → ``success=False`` with positive-integer error."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
_seed_bytes("a.png", b"\x89PNG\r\n\x1a\nx")
store = await _make_store()
resp = await _read_image(store, step_kwargs={"max_bytes": -1}, path="a.png")
assert resp.success is False, resp
assert "positive integer" in resp.answer, resp
await store.close()
print("✓ test_read_image_invalid_max_bytes passed")
_run(run())