scriptoriumadmin/ReMe
1
0
Fork 0
mirror of https://github.com/agentscope-ai/ReMe.git synced 2026-09-29 01:41:38 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
ReMe/benchmark/halumem/quickstart.md
方应 64adc1166c docs(benchmark): update HaluMem quickstart guide with dataset download instructions 
- Replace repository cloning with direct dataset download using curl
- Add commands to download HaluMem-Medium.jsonl and HaluMem-Long.jsonl files
- Include both official Hugging Face and mirror download sources
- Update data path reference from nested directory to local data folder
- Add dataset page link and mirror usage instructions for mainland China access
2026-04-13 10:38:35 +08:00

1.6 KiB
Raw Blame History

Halumem

Experiment Quick Start Guide This guide helps you quickly set up and run Halumem experiments with ReMe integration.

1. Start ReMe Service

Install ReMe (if not already installed) If you haven't installed the ReMe environment yet, follow these steps:

# Create ReMe environment
conda create -p ./reme-env python==3.12
conda activate ./reme-env

# Install ReMe
pip install .

2. Download the Dataset

cd ./benchmark/halumem
mkdir -p data
curl -L "https://huggingface.co/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Medium.jsonl?download=true" -o data/HaluMem-Medium.jsonl
curl -L "https://huggingface.co/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Long.jsonl?download=true" -o data/HaluMem-Long.jsonl

Dataset page: https://huggingface.co/datasets/IAAR-Shanghai/HaluMem/tree/main

If the official source is slow or inaccessible in mainland China, you can use a mirror:

cd ./benchmark/halumem
mkdir -p data
curl -L "https://hf-mirror.com/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Medium.jsonl?download=true" -o data/HaluMem-Medium.jsonl
curl -L "https://hf-mirror.com/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Long.jsonl?download=true" -o data/HaluMem-Long.jsonl

3. Run Experiments

Launch the ReMe service to enable memory library functionality:

clear && python benchmark/halumem/eval_reme.py \
    --data_path benchmark/halumem/data/HaluMem-Medium.jsonl \
    --reme_model_name gpt-4o-mini-2024-07-18 \
    --eval_model_name gpt-4o-mini-2024-07-18 \
    --batch_size 40 \
    --algo_version default