mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-11-02 07:13:50 +00:00
6b035c6553
Some checks are pending
Pre-commit / run (ubuntu-latest) (push) Waiting to run
Tests ReMe / Unit Tests - py3.11 (push) Waiting to run
Tests ReMe / Unit Tests - py3.12 (push) Waiting to run
Tests ReMe / Unit Tests - py3.13 (push) Waiting to run
Windows Smoke / CLI smoke - py3.11 (push) Waiting to run
* feat(counter): extend counter tree utils and record job call statistics - replace global_counter_next with fetch-and-add style global_counter_add/inc, plus read-only global_counter_get and global_counter_get_all - record per-job call counts in app_context.metadata via BaseJob._record_call, covering background/cron/stream jobs - update agentic_answer step and utils exports; add unit tests for job counting and counter utils * feat(evaluation): add check_job_count interface and report search calls in benchmarks - Extract _counter_key from BaseJob._record_call for reusable counter lookup - Add reme.utils.evaluation_interface.check_job_count read-only helper - Track and report average search calls per query in beam and longmemeval benchmarks * job counter * token消耗量统计 * benchmark输出完整token消耗统计 * benchmark统计输出改用标准差 - beam/longmemeval 的工具调用与 token 统计由方差改为标准差输出 - 修复 lint: 局部变量遮蔽 importlib.metadata、补充测试 docstring - black 格式化 * fix(evaluation): preserve complete token usage metrics * fix: exclude stream replies from token accounting * Revert "fix: exclude stream replies from token accounting" This reverts commit85bf32064d. * Reapply "fix: exclude stream replies from token accounting" This reverts commit6722c24dc5. * support agent scope 2.0.5 * feat: support injection_config to disable runtime state injection in benchmarks - Add InjectionConfig passthrough in AsAgentWrapper.reply() - Disable inject_runtime_state in BaseAgenticAnswerStep to avoid wall-clock time conflicting with benchmark query_time anchors - Disable inject_runtime_state in beam/lme llm_judge calls * feat: agentscope dual-version compat & benchmark improvements - Add version_tuple utility for semantic version comparison - AsAgentWrapper: version-aware InjectionConfig, max_iters doubling, and token usage collection (reply vs reply_stream) for AS>=2.0.5/<2.0.5 - Default inject_runtime_state=False in wrapper to avoid benchmark time-anchor conflicts; remove per-callsite injection_config overrides - longmemeval run.py: support question_ids filter in dataset config - Fix unused import in test_evaluation_interface; format fixes * chore: remove temporary flip-test benchmark config * revert: pin agentscope to 2.0.4.post1 and drop dual-version compat * fix(evaluation): clarify usage semantics and atomic counters --------- Co-authored-by: sa-buc <jiangniurou.xyf@dail-algo011164204033.ET135> Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
52 lines
1.3 KiB
Python
52 lines
1.3 KiB
Python
"""Utility modules."""
|
|
|
|
from .common_utils import (
|
|
hash_text,
|
|
execute_stream_task,
|
|
mock_reme_server,
|
|
call_action,
|
|
call_and_check,
|
|
)
|
|
from .env_utils import load_env, parse_env_file
|
|
from .link_expansion import expand_links, render_expansion_lines
|
|
from .logger_utils import get_logger
|
|
from .logo_utils import print_logo
|
|
from .service_utils import find_reme, locate_reme, precheck_start, cli_find_reme, running_service_config
|
|
from .similarity_utils import cosine_similarity, batch_cosine_similarity
|
|
from .token_utils import estimate_token_count
|
|
from .agent_state_io import AsStateHandler
|
|
from .counter import (
|
|
global_counter_add,
|
|
global_counter_add_many,
|
|
global_counter_get,
|
|
global_counter_get_all,
|
|
global_counter_inc,
|
|
)
|
|
|
|
__all__ = [
|
|
"hash_text",
|
|
"execute_stream_task",
|
|
"mock_reme_server",
|
|
"call_action",
|
|
"call_and_check",
|
|
"load_env",
|
|
"parse_env_file",
|
|
"expand_links",
|
|
"render_expansion_lines",
|
|
"get_logger",
|
|
"print_logo",
|
|
"find_reme",
|
|
"locate_reme",
|
|
"precheck_start",
|
|
"cli_find_reme",
|
|
"running_service_config",
|
|
"cosine_similarity",
|
|
"batch_cosine_similarity",
|
|
"estimate_token_count",
|
|
"AsStateHandler",
|
|
"global_counter_add",
|
|
"global_counter_add_many",
|
|
"global_counter_get",
|
|
"global_counter_get_all",
|
|
"global_counter_inc",
|
|
]
|