3f2eb6235f
* Simplify project implementation * Centralize benchmark agentic answer base class * Remove bundled ReMe source snapshots * Preserve local benchmark configs and plugin discovery behavior * docs: enrich job parameter descriptions in beam and lme plugin configs * Simplify project structure and remove obsolete code * Move benchmark search step configuration into BEAM and LME plugins * Rename benchmark judge packages to avoid import collisions * Remove explicit plugin package loading in favor of entry-point discovery * Export benchmark plugin Steps from public packages |
||
|---|---|---|
| .. | ||
| src | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| README_ZH.md | ||
BEAM plugin
This plugin owns the BEAM memory, agentic-answer and judge Steps, their prompts,
and their Job defaults in plugin.yaml. ReMe's built-in benchmark.yaml owns the
shared evaluation Jobs and components. Dataset handling, the runner and results
remain in benchmark/beam.
From the repository root, install ReMe and this plugin in editable mode before running the benchmark:
python -m pip install -e ".[as]"
reme plugins install ./plugins/beam --editable
reme plugins validate beam
python benchmark/beam/run.py
Editable installation registers the beam entry point while keeping source changes immediately
visible. The runner selects the built-in benchmark preset and explicitly enables beam for
each Application. Installing the plugin makes it discoverable but does not enable it globally.
plugin.yaml registers backends and contributes the plugin-owned auto_memory,
agentic_answer and answer_judge Job defaults. Start the installed plugin with
reme start config=benchmark plugins='["beam"]'. The shared preset does not inherit
default: only declared Jobs run, indexing is manual, and neither scheduled dream
nor the optional auto_dream Job is enabled.
The existing auto_memory, agentic_answer, answer_judge, bench and judge
names and model environment variables are unchanged. Explicit application/CLI overrides
still take precedence. Installing this plugin does not start an evaluation.
The shared answer base class lives in reme.steps.benchmark.base_agentic_answer.
The old core-owned reme.steps.benchmark.beam Python import path is removed.
Custom Python callers should import memory, search and answer Steps from reme_beam, and the
judge Step from judge_beam. After uninstalling,
Applications and CLI services must omit the plugin until it is installed again.
Uninstallation never removes datasets, workspaces or results.
Restart an existing service after changing plugins.