ReMe/tests4/integration/test_as_llm.py
jinliyl 3cb2579ff7
refactor(steps): update auto-memory (#263)
* refactor(steps): update naming conventions in components and configuration

Updated naming conventions across multiple files, changing colon-separated names to underscore-separated format, and added new step definitions along with documentation updates.

Key changes:
- Replaced `Synchronizer` with `AutoMemory` as the counterpart component for cold-write operations
- Updated naming conventions in all related configuration files (e.g., `frontmatter:read` → `frontmatter_read`)
- Added new step definitions such as `submit_slug_updates` and `auto_memory`
- Updated relevant documentation
- Modified log output format for improved readability

* refactor(evolve): Refactor the auto-memory module and update related configurations

- Remove the old slug update commit step file
- Add new auto-memory planner and writer steps
- Update __init__.py to export the new step classes
- Modify the auto_memory configuration structure in default.yaml
- Update the slug field description for clearer explanation of its purpose

* up

* up

* refactor(tests): Move unit test directory from `tests4/unittest` to `tests4/unit`

Additionally, the assertion logic in test files has been updated: direct comparisons of `payload["notes"]` have been replaced with checks verifying the presence of paths and metadata within the response content. Furthermore, some test expectations have been simplified—for example, using `count` instead of asserting against specific note lists.

Specific changes include:
- Updating workflow configurations to align with the new test directory structure
- Modifying assertions across multiple test methods to make them more flexible and maintainable
- Cleaning up and optimizing parts of the test code structure

This is a comprehensive test refactoring effort aimed at improving test readability and robustness.

* Refactor(steps): Update memory writing logic and optimize JSON schema structure

Improved the write strategy description in `auto_memory_writer.yaml` to emphasize using `edit` over `write`.
Adjusted the `json_schema` structure in `base_step.py` to support the new function definition format.
Also corrected grammatical issues in the related documentation.

* Fix: Improve frontend data parsing error handling and update test files

Added capture and handling logic for YAML parsing exceptions, providing more detailed error messages when frontend data format issues occur. Also corrected the description text in a test file.
2026-05-29 12:07:44 +08:00

92 lines
3 KiB
Python

"""Integration tests: drive ReActAgent through LLMDemoStep + Application wiring.
Requires LLM_API_KEY (and optionally LLM_BASE_URL / LLM_MODEL_NAME) in the
environment or a .env file at the repo root. Hits the real Anthropic API.
"""
import asyncio
import os
import tempfile
from reme4 import Application
from reme4.config import resolve_app_config
from reme4.steps.common.llm_demo import LLMDemoStep
from reme4.utils import load_env
load_env()
class _temp_chdir:
"""chdir to path for the duration of the block; restore on exit."""
def __init__(self, path):
self.path = path
self._old = None
def __enter__(self):
self._old = os.getcwd()
os.chdir(self.path)
return self
def __exit__(self, *exc):
os.chdir(self._old)
async def _make_app() -> Application:
"""Build and start an Application from the default config (LLM wired via env vars)."""
cfg = resolve_app_config(log_to_console=False, log_to_file=False, enable_logo=False)
app = Application(**cfg)
await app.start()
return app
def test_llm_demo_step_basic_chat():
"""LLMDemoStep drives ReActAgent through self.as_llm/as_llm_formatter."""
async def run():
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
app = await _make_app()
try:
step = LLMDemoStep(app_context=app.context)
response = await step(
query="What is 1 + 1? Reply with just the number.",
)
text = (response.answer or "").strip()
print(f"\n[basic_chat] response: {text!r}")
assert text, "Empty assistant response"
assert "2" in text, f"Expected '2' in response, got: {text!r}"
print("✓ test_llm_demo_step_basic_chat passed")
finally:
await app.close()
asyncio.run(run())
def test_llm_demo_step_with_tool():
"""LLMDemoStep registers the add tool and the agent invokes it."""
async def run():
with tempfile.TemporaryDirectory() as tmp, _temp_chdir(tmp):
app = await _make_app()
try:
step = LLMDemoStep(app_context=app.context)
response = await step(
query="Use the add tool to compute 21 + 21 and report the result.",
sys_prompt="Use the `add` tool whenever the user asks to add numbers.",
use_add_tool=True,
)
text = (response.answer or "").strip()
print(f"\n[with_tool] response: {text!r}")
assert "42" in text, f"Expected '42' in response, got: {text!r}"
print("✓ test_llm_demo_step_with_tool passed")
finally:
await app.close()
asyncio.run(run())
if __name__ == "__main__":
print("=== LLMDemoStep + ReActAgent integration tests ===")
test_llm_demo_step_basic_chat()
test_llm_demo_step_with_tool()
print("\nAll integration tests passed!")