diff --git a/benchmark/halumem/quickstart.md b/benchmark/halumem/quickstart.md index 59dc68fa..4ef30846 100644 --- a/benchmark/halumem/quickstart.md +++ b/benchmark/halumem/quickstart.md @@ -14,17 +14,30 @@ conda activate ./reme-env pip install . ``` -### 2. Clone the Repository +### 2. Download the Dataset ```bash cd ./benchmark/halumem -git clone https://github.com/MemTensor/HaluMem.git +mkdir -p data +curl -L "https://huggingface.co/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Medium.jsonl?download=true" -o data/HaluMem-Medium.jsonl +curl -L "https://huggingface.co/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Long.jsonl?download=true" -o data/HaluMem-Long.jsonl +``` + +Dataset page: +https://huggingface.co/datasets/IAAR-Shanghai/HaluMem/tree/main + +If the official source is slow or inaccessible in mainland China, you can use a mirror: +```bash +cd ./benchmark/halumem +mkdir -p data +curl -L "https://hf-mirror.com/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Medium.jsonl?download=true" -o data/HaluMem-Medium.jsonl +curl -L "https://hf-mirror.com/datasets/IAAR-Shanghai/HaluMem/resolve/main/HaluMem-Long.jsonl?download=true" -o data/HaluMem-Long.jsonl ``` ### 3. Run Experiments Launch the ReMe service to enable memory library functionality: ```bash clear && python benchmark/halumem/eval_reme.py \ - --data_path benchmark/halumem/HaluMem/data/HaluMem-Medium.jsonl \ + --data_path benchmark/halumem/data/HaluMem-Medium.jsonl \ --reme_model_name gpt-4o-mini-2024-07-18 \ --eval_model_name gpt-4o-mini-2024-07-18 \ --batch_size 40 \