Commit graph

3 commits

Author SHA1 Message Date
xyf2020
67936d5a43
refactor(plugins): move BaseAgenticAnswerStep out of the core benchmark steps package into the lme/beam plugins (#576)
* refactor(plugins): move BaseAgenticAnswerStep into lme/beam plugins

Move the shared benchmark answer base class from the core
reme.steps.benchmark package into each plugin's own src tree
(reme_lme.base_agentic_answer / reme_beam.base_agentic_answer) with
absolute imports, drop the core benchmark steps package, and update
plugin READMEs accordingly.

* refactor(plugins): simplify benchmark answer steps

* docs(plugins): clarify benchmark answer step migration

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
2026-09-30 12:19:32 +08:00
xyf2020
9975bb37b9
Separate benchmark judge plugins (#535)
Some checks failed
CI / Python tests / Unit Tests - py3.12 (push) Waiting to run
CI / Python tests / Unit Tests - py3.13 (push) Waiting to run
CI / Windows / CLI smoke - py3.11 (push) Waiting to run
Deploy / Documentation / Build documentation (push) Waiting to run
Deploy / Documentation / deploy (push) Blocked by required conditions
CI / Python packages / Build and verify distributions (push) Waiting to run
CI / Python quality / Pre-commit (push) Waiting to run
CI / Python tests / Unit Tests - py3.11 (push) Waiting to run
Security / CodeQL / Analyze javascript-typescript (push) Waiting to run
Security / CodeQL / Analyze python (push) Waiting to run
CI / Documentation / Test and build documentation (push) Has been cancelled
2026-09-10 20:16:46 +08:00
xyf2020
3f2eb6235f
feat(benchmark): extract BEAM and LongMemEval into standalone plugins (#512)
* Simplify project implementation

* Centralize benchmark agentic answer base class

* Remove bundled ReMe source snapshots

* Preserve local benchmark configs and plugin discovery behavior

* docs: enrich job parameter descriptions in beam and lme plugin configs

* Simplify project structure and remove obsolete code

* Move benchmark search step configuration into BEAM and LME plugins

* Rename benchmark judge packages to avoid import collisions

* Remove explicit plugin package loading in favor of entry-point discovery

* Export benchmark plugin Steps from public packages
2026-09-03 14:06:39 +08:00