mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-10-03 02:24:31 +00:00
3f2eb6235f
* Simplify project implementation * Centralize benchmark agentic answer base class * Remove bundled ReMe source snapshots * Preserve local benchmark configs and plugin discovery behavior * docs: enrich job parameter descriptions in beam and lme plugin configs * Simplify project structure and remove obsolete code * Move benchmark search step configuration into BEAM and LME plugins * Rename benchmark judge packages to avoid import collisions * Remove explicit plugin package loading in favor of entry-point discovery * Export benchmark plugin Steps from public packages
38 lines
2 KiB
Markdown
38 lines
2 KiB
Markdown
# BEAM plugin
|
|
|
|
[中文说明](./README_ZH.md)
|
|
|
|
This plugin owns the BEAM memory, agentic-answer and judge Steps, their prompts,
|
|
and their Job defaults in `plugin.yaml`. ReMe's built-in `benchmark.yaml` owns the
|
|
shared evaluation Jobs and components. Dataset handling, the runner and results
|
|
remain in [`benchmark/beam`](../../benchmark/beam/README.md).
|
|
|
|
From the repository root, install ReMe and this plugin in editable mode before running the benchmark:
|
|
|
|
```bash
|
|
python -m pip install -e ".[as]"
|
|
reme plugins install ./plugins/beam --editable
|
|
reme plugins validate beam
|
|
python benchmark/beam/run.py
|
|
```
|
|
|
|
Editable installation registers the `beam` entry point while keeping source changes immediately
|
|
visible. The runner selects the built-in `benchmark` preset and explicitly enables `beam` for
|
|
each Application. Installing the plugin makes it discoverable but does not enable it globally.
|
|
|
|
`plugin.yaml` registers backends and contributes the plugin-owned `auto_memory`,
|
|
`agentic_answer` and `answer_judge` Job defaults. Start the installed plugin with
|
|
`reme start config=benchmark plugins='["beam"]'`. The shared preset does not inherit
|
|
`default`: only declared Jobs run, indexing is manual, and neither scheduled dream
|
|
nor the optional `auto_dream` Job is enabled.
|
|
The existing `auto_memory`, `agentic_answer`, `answer_judge`, `bench` and `judge`
|
|
names and model environment variables are unchanged. Explicit application/CLI overrides
|
|
still take precedence. Installing this plugin does not start an evaluation.
|
|
|
|
The shared answer base class lives in `reme.steps.benchmark.base_agentic_answer`.
|
|
The old core-owned `reme.steps.benchmark.beam` Python import path is removed.
|
|
Custom Python callers should import memory, search and answer Steps from `reme_beam`, and the
|
|
judge Step from `judge_beam`. After uninstalling,
|
|
Applications and CLI services must omit the plugin until it is installed again.
|
|
Uninstallation never removes datasets, workspaces or results.
|
|
Restart an existing service after changing plugins.
|