ReMe/tests4/unit/test_link_expansion.py
jinliyl d8086039dc
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
refactor(Agentscope2.0): llm & embedding & agent (#271)
* refactor(embedding): replace embedding model with embedding store architecture

- Remove as_token_counter component and its estimated token counter implementation
- Replace BaseEmbeddingModel with BaseEmbedding that wraps AgentScope embedding models
- Add support for multiple embedding providers (OpenAI, DashScope, Gemini, Ollama)
- Introduce BaseEmbeddingStore and LocalEmbeddingStore for caching and persistence
- Update component registry to use new embedding and embedding_store types
- Modify file stores to use embedding_store instead of embedding_model
- Update health check to monitor embedding_store instead of embedding_model
- Change default config to use embedding_store with local backend
- Add estimate_token_count utility function to utils module

* refactor(llm): replace as_llm components with unified llm implementation

- Remove deprecated as_llm and as_llm_formatter modules
- Add new llm module with BaseLLM and provider-specific implementations
- Update component registry to use LLM instead of AS_LLM
- Replace all as_llm/as_llm_formatter references with llm in steps
- Update configuration schema to use llm instead of as_llm
- Rename integration test file from test_as_llm to test_llm
- Add proper docstrings to embedding store dimension property
- Add pylint disable comment for embedding model call
- Remove unused FormatterBase import in base_step
- Update token_utils with function docstring

* refactor(evolve): replace ReActAgent with Agent and update message handling

- Removed FlexReActAgent class and direct ReActAgent imports
- Updated Agent instantiation to use new constructor parameters
- Changed message content to use TextBlock format instead of plain strings
- Modified timestamp access from msg.timestamp to msg.created_at
- Updated metadata access pattern for structured outputs
- Replaced Msg.from_dict with Msg.model_validate in auto_memory.py
- Updated test mocks to patch Agent instead of ReActAgent
- Changed message serialization from to_dict to model_dump in tests
- Moved component references to base class definition
- Updated demo tools to return strings instead of ToolResponse objects

* feat(step): migrate to FunctionTool and add streaming support

- Replace deprecated ToolResponse with FunctionTool in base_step.py
- Remove unused TextBlock import from base_step.py
- Update job registration to use new FunctionTool API
- Add thinking_budget parameter to llm_demo configuration
- Introduce StreamLLMDemoStep with streaming output capability
- Add structured output support to LLMDemoStep via generate_structured_output
- Implement streaming event handling for text/thinking/tool calls
- Add integration tests for embedding functionality
- Add integration tests for structured output and streaming features
- Update tool usage in demo steps to use new function naming convention

* fix(ci): correct package installation path in unittest workflow

- Updated pip install command to use proper package path "./reme4[dev,core]"
- Fixed dependency installation step in CI workflow configuration

* chore(workflow): update python versions in unittest workflow

- Remove Python 3.10 from test matrix
- Add Python 3.11 to test matrix
- Add Python 3.12 to test matrix
- Keep Python 3.13 in test matrix
- Update matrix configuration for better version coverage

* fix(health): handle missing dimensions attribute in embedding status

- Wrap dimensions access in try-except to prevent AttributeError
- Return None when dimensions attribute is not available
- Maintain backward compatibility for components without dimensions

test(component): add comprehensive tests for BaseComponent and related classes

- Add tests for Dependency class including repr and attribute access
- Add tests for bind method with various scenarios and edge cases
- Add tests for lifecycle management and async context handling
- Add tests for standalone and context-bound dependency resolution
- Add tests for ComponentMixin path utilities

test(common): update LocalFileStore initialization parameter

- Change embedding_model parameter to embedding_store in test setup
- Update all affected test files consistently

test(registry): add complete test suite for ComponentRegistry

- Add tests for register method with explicit names and defaults
- Add tests for decorator registration pattern
- Add tests for get_all method returning copies
- Add tests for unregister and clear operations
- Add tests for error handling of invalid registrations

test(job): add comprehensive tests for BaseJob and BackgroundJob

- Add tests for step resolution and exception handling
- Add tests for backoff delay calculation with jitter
- Add tests for supervisor loop restart behavior
- Add tests for task shutdown and cancellation

test(prompt): add complete test suite for PromptHandler

- Add tests for prompt loading from dictionaries and files
- Add tests for internationalization and language fallback
- Add tests for flag filtering and variable substitution
- Add tests for format validation and error handling

test(runtime): add basic tests for RuntimeContext dictionary access

- Add tests for item getting, setting and containment checks
- Add tests for missing key error handling

* feat(evolve): add permission context and agent state management

- Import PermissionContext, PermissionMode and AgentState modules
- Add state configuration with bypass permission mode to AutoDream agents
- Add state configuration with bypass permission mode to AutoMemory agents
- Implement static _to_msg method for message validation and formatting
- Refactor message processing to use the new _to_msg method
- Ensure proper content structure for text blocks in message conversion

* style(tests): update test files with linting rules and code improvements

- Add missing pylint disable directives for docstring and attribute warnings
- Replace lambda expressions with proper function definitions in test cases
- Import Path directly instead of using lambda with __import__
- Simplify assertion checks by using truthiness instead of equality to empty dict
- Remove unused imports and reorder imports consistently
- Format dictionary literals with proper indentation and line breaks
2026-06-03 11:35:43 +08:00

254 lines
8.8 KiB
Python

"""Tests for ``reme4.utils.link_expansion``.
Two pure helpers:
* ``expand_links(file_store, paths, max_per_direction)`` — fetch
outlinks / inlinks for each path with neighbor meta attached.
* ``render_expansion_lines(expansion)`` — turn one path's expansion
dict into the indented ``→`` / ``←`` block used by SearchStep
answers.
"""
# pylint: disable=protected-access
import asyncio
import os
import tempfile
import warnings
from pathlib import Path
from reme4.components.file_store import LocalFileStore
from reme4.schema import FileFrontMatter, FileNode
from reme4.utils.link_expansion import expand_links, render_expansion_lines
from reme4.utils.wikilink_handler import WikilinkHandler
warnings.filterwarnings("ignore", category=DeprecationWarning, module="jieba")
warnings.filterwarnings("ignore", category=DeprecationWarning, module="pkg_resources")
class temp_chdir:
"""Test helper: chdir to ``path`` on enter, restore previous cwd on exit."""
def __init__(self, path):
self.path = path
self.old = None
def __enter__(self):
self.old = os.getcwd()
os.chdir(self.path)
return self
def __exit__(self, *exc):
os.chdir(self.old)
async def _store_with(files: dict[str, dict]) -> LocalFileStore:
"""LocalFileStore seeded with files + parsed wikilinks + optional frontmatter meta.
Each value: ``{"body": str, "name": str?, "description": str?}``. The
body is written to disk and wikilinks are extracted; ``name`` /
``description`` populate FileFrontMatter so neighbor meta lookups
have something to surface.
"""
store = LocalFileStore(name="t", embedding_store="")
await store.start()
nodes: list[FileNode] = []
root = Path.cwd()
for rel, spec in files.items():
body = spec["body"]
abs_path = root / rel
abs_path.parent.mkdir(parents=True, exist_ok=True)
abs_path.write_text(body, encoding="utf-8")
fm = FileFrontMatter(
name=spec.get("name", ""),
description=spec.get("description", ""),
)
nodes.append(
FileNode(
path=rel,
st_mtime=abs_path.stat().st_mtime,
links=WikilinkHandler.extract_links(body, rel),
front_matter=fm,
),
)
if nodes:
await store.file_graph.upsert_nodes(nodes)
return store
# -- expand_links -------------------------------------------------------------
def test_expand_links_empty_paths_short_circuits():
"""No paths ⇒ empty dict, no file_store calls needed."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = LocalFileStore(name="t", embedding_store="")
await store.start()
result = await expand_links(store, [])
assert result == {}
await store.close()
print("✓ test_expand_links_empty_paths_short_circuits passed")
asyncio.run(run())
def test_expand_links_returns_outlinks_and_inlinks_with_meta():
"""A.md links to B.md ⇒ A has B as outlink, B has A as inlink, meta surfaced."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _store_with(
{
"A.md": {"body": "See [[B.md]] for details.", "name": "A Doc", "description": "alpha"},
"B.md": {"body": "End node.", "name": "B Doc", "description": "beta"},
},
)
result = await expand_links(store, ["A.md", "B.md"])
assert set(result.keys()) == {"A.md", "B.md"}
a_out = result["A.md"]["outlinks"]
assert len(a_out) == 1
assert a_out[0]["path"] == "B.md"
assert a_out[0]["meta"] == {"name": "B Doc", "description": "beta"}
assert a_out[0]["edges"] == [{"predicate": None, "anchor": None}]
assert result["A.md"]["inlinks"] == []
b_in = result["B.md"]["inlinks"]
assert len(b_in) == 1
assert b_in[0]["path"] == "A.md"
assert b_in[0]["meta"] == {"name": "A Doc", "description": "alpha"}
assert result["B.md"]["outlinks"] == []
await store.close()
print("✓ test_expand_links_returns_outlinks_and_inlinks_with_meta passed")
asyncio.run(run())
def test_expand_links_max_per_direction_caps_neighbors():
"""max_per_direction=2 ⇒ only first two distinct neighbors per direction kept."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _store_with(
{
"hub.md": {
"body": "[[a.md]] [[b.md]] [[c.md]] [[d.md]]",
},
"a.md": {"body": "a"},
"b.md": {"body": "b"},
"c.md": {"body": "c"},
"d.md": {"body": "d"},
},
)
result = await expand_links(store, ["hub.md"], max_per_direction=2)
out = result["hub.md"]["outlinks"]
assert len(out) == 2
assert [n["path"] for n in out] == ["a.md", "b.md"]
await store.close()
print("✓ test_expand_links_max_per_direction_caps_neighbors passed")
asyncio.run(run())
def test_expand_links_node_without_meta_returns_empty_meta_dict():
"""Neighbor with no frontmatter name/description ⇒ meta = {}."""
async def run():
with tempfile.TemporaryDirectory() as tmp, temp_chdir(tmp):
store = await _store_with(
{
"src.md": {"body": "[[dst.md]]"},
"dst.md": {"body": "no meta"},
},
)
result = await expand_links(store, ["src.md"])
assert result["src.md"]["outlinks"][0]["meta"] == {}
await store.close()
print("✓ test_expand_links_node_without_meta_returns_empty_meta_dict passed")
asyncio.run(run())
# -- render_expansion_lines ---------------------------------------------------
def test_render_expansion_lines_empty_input_yields_empty_list():
"""Both directions empty ⇒ no lines."""
assert not render_expansion_lines({})
assert not render_expansion_lines({"outlinks": [], "inlinks": []})
print("✓ test_render_expansion_lines_empty_input_yields_empty_list passed")
def test_render_expansion_lines_outlinks_only():
"""Single outlink with meta + plain edge renders as 3 lines."""
expansion = {
"outlinks": [
{
"path": "B.md",
"meta": {"name": "B", "description": "beta"},
"edges": [{"predicate": None, "anchor": None}],
},
],
"inlinks": [],
}
lines = render_expansion_lines(expansion)
assert lines == [
" outlinks (1):",
' → B.md name="B" description="beta"',
" via plain",
]
print("✓ test_render_expansion_lines_outlinks_only passed")
def test_render_expansion_lines_inlinks_only_with_predicate_and_anchor():
"""Inlink edge with predicate + anchor renders via descriptor."""
expansion = {
"outlinks": [],
"inlinks": [
{
"path": "src.md",
"meta": {},
"edges": [{"predicate": "references", "anchor": "intro"}],
},
],
}
lines = render_expansion_lines(expansion)
assert lines == [
" inlinks (1):",
" ← src.md (no meta)",
" via predicate=references, anchor=#intro",
]
print("✓ test_render_expansion_lines_inlinks_only_with_predicate_and_anchor passed")
def test_render_expansion_lines_both_directions_in_order():
"""outlinks block precedes inlinks block."""
expansion = {
"outlinks": [
{"path": "out.md", "meta": {"name": "Out"}, "edges": [{"predicate": None, "anchor": None}]},
],
"inlinks": [
{"path": "in.md", "meta": {"description": "incoming"}, "edges": [{"predicate": None, "anchor": None}]},
],
}
lines = render_expansion_lines(expansion)
assert lines[0] == " outlinks (1):"
assert lines[3] == " inlinks (1):"
assert lines[1].lstrip().startswith("→")
assert lines[4].lstrip().startswith("←")
print("✓ test_render_expansion_lines_both_directions_in_order passed")
if __name__ == "__main__":
test_expand_links_empty_paths_short_circuits()
test_expand_links_returns_outlinks_and_inlinks_with_meta()
test_expand_links_max_per_direction_caps_neighbors()
test_expand_links_node_without_meta_returns_empty_meta_dict()
test_render_expansion_lines_empty_input_yields_empty_list()
test_render_expansion_lines_outlinks_only()
test_render_expansion_lines_inlinks_only_with_predicate_and_anchor()
test_render_expansion_lines_both_directions_in_order()