为文件型记忆(ReMeLight)新增 LoCoMo 评测支持。在此之前仅向量型记忆系统(ReMe)有评测脚本 (#283)

* feat: 新增 ReMeLight 文件版记忆的 LoCoMo 评测脚本

* feat: add LoCoMo eval for ReMeLight file-based memory

* fix: resolve pre-commit lint issues

* fix: align answer model with eval_reme.py (qwen3-30b-a3b-instruct-2507)

* fix: add --disable=E0611

.pre-commit-config.yaml

@@ -55,6 +55,7 @@ repos:
           --disable=R0913,
           --disable=R0917,
           --disable=E0401,
+          --disable=E0611,
           --disable=E1101,
           --disable=E1111,
           --disable=C0415,

benchmark/locomo/quickstart.md

@@ -0,0 +1,62 @@
+# LoCoMo — ReMeLight / ReMe 评测快速开始
+
+### 1. 安装 ReMe
+
+```bash
+pip install -e ".[light]"
+```
+
+### 2. 下载数据集
+
+```bash
+cd benchmark/locomo
+mkdir -p data
+
+# 克隆原始 LoCoMo 仓库（包含 locomo10.json）
+git clone https://github.com/luyanhexay/locomo-dynamemory.git /tmp/locomo-dynamemory
+cp /tmp/locomo-dynamemory/data/locomo10.json data/
+```
+
+数据集信息：
+- 论文: [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://arxiv.org/abs/2402.17753)
+- 项目页: https://snap-research.github.io/locomo
+- 原始仓库: https://github.com/luyanhexay/locomo-dynamemory
+
+### 3. 运行向量版评测（ReMe）
+
+```bash
+python benchmark/locomo/eval_reme.py \
+    --data_path benchmark/locomo/data/locomo10.json \
+    --reme_model_name qwen-flash \
+    --eval_model_name qwen3-max \
+    --top_k 20 \
+    --user_num 5 \
+    --max_concurrency 2
+```
+
+### 4. 运行文件版评测（ReMeLight）
+
+```bash
+python benchmark/locomo/eval_reme_light.py \
+    --data_path benchmark/locomo/data/locomo10.json \
+    --reme_model_name qwen-flash \
+    --eval_model_name qwen3-max \
+    --top_k 20 \
+    --user_num 5 \
+    --max_concurrency 2
+```
+
+首次跑建议 `--user_num 1` 验证流程，确认没问题再加。
+
+### 5. 查看结果
+
+```bash
+# 最终指标
+cat bench_results/reme_light/eval_statistics.json
+
+# 逐条 QA 详情
+cat bench_results/reme_light/eval_results.jsonl
+
+# 文件版特有的：直接看记忆写得好不好
+ls bench_results/reme_light/working_dirs/<user_name>/memory/
+```