scriptoriumadmin/ReMe
1
0
Fork 0
mirror of https://github.com/agentscope-ai/ReMe.git synced 2026-09-06 08:16:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13
ReMe/benchmark/halumem/quickstart.md
Zhouwk 625d184ca1
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
Details
添加log_to_file的开关 (#205)
2026-04-13 10:49:26 +08:00

1.6 KiB
Raw Permalink Blame History

Halumem

Experiment Quick Start Guide This guide helps you quickly set up and run Halumem experiments with ReMe integration.

1. Start ReMe Service

Install ReMe (if not already installed) If you haven't installed the ReMe environment yet, follow these steps:

# Create ReMe environment
conda create -p ./reme-env python==3.12
conda activate ./reme-env

# Install ReMe
pip install .

2. Download the Dataset

cd ./benchmark/halumem
mkdir -p data
curl -L "https://huggingface.co/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Medium.jsonl?download=true" -o data/HaluMem-Medium.jsonl
curl -L "https://huggingface.co/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Long.jsonl?download=true" -o data/HaluMem-Long.jsonl

Dataset page: https://huggingface.co/datasets/IAAR-Shanghai/HaluMem/tree/main

If the official source is slow or inaccessible in mainland China, you can use a mirror:

cd ./benchmark/halumem
mkdir -p data
curl -L "https://hf-mirror.com/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Medium.jsonl?download=true" -o data/HaluMem-Medium.jsonl
curl -L "https://hf-mirror.com/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Long.jsonl?download=true" -o data/HaluMem-Long.jsonl

3. Run Experiments

Launch the ReMe service to enable memory library functionality:

clear && python benchmark/halumem/eval_reme.py \
    --data_path benchmark/halumem/data/HaluMem-Medium.jsonl \
    --reme_model_name gpt-4o-mini-2024-07-18 \
    --eval_model_name gpt-4o-mini-2024-07-18 \
    --batch_size 40 \
    --algo_version default