ReMe/bench/halumem
jinli.yl 327dc58f70 feat(bench): add HaluMem dataset statistics analyzer and update LLM concurrency control
- Add new analyze_dataset_stats.py script for comprehensive HaluMem dataset analysis
- Include statistics for user sessions, dialogues, content lengths and chunk distributions
- Replace rate limiting with concurrency control in BaseLLM using semaphore mechanism
- Update configuration to use max_concurrency instead of max_rps and rps_window
- Modify dialogue formatting to include only user messages in evaluation
- Add percentile calculations and detailed content size distribution metrics
- Implement session splitting logic based on character length thresholds
- Provide per-user statistics and summary tables for dataset analysis
- Refactor BaseLLM to use internal _chat_impl and _stream_chat_impl methods
- Remove rate limiting locks and timestamps from LLM initialization
- Add command-line interface for dataset statistics analysis tool
2026-01-14 10:14:35 +08:00
..
analyze_dataset_stats.py feat(bench): add HaluMem dataset statistics analyzer and update LLM concurrency control 2026-01-14 10:14:35 +08:00
analyze_results.py feat(benchmark): add HaluMem baseline evaluation and analysis tools 2026-01-13 23:59:55 +08:00
compute_stats_from_tmp.py feat(bench): add statistics computation script for benchmark evaluation 2026-01-11 16:17:51 +08:00
eval_baseline_simple.py feat(bench): add HaluMem dataset statistics analyzer and update LLM concurrency control 2026-01-14 10:14:35 +08:00
eval_reme.py feat(mem_agent): introduce version 2 memory agents and tools 2026-01-13 01:54:20 +08:00
eval_reme_simple.py feat(benchmark): add HaluMem baseline evaluation and analysis tools 2026-01-13 23:59:55 +08:00
eval_tools.py feat(mem_agent): introduce version 2 memory agents and tools 2026-01-13 01:54:20 +08:00
halumem.yaml feat(mem_agent): introduce version 2 memory agents and tools 2026-01-13 01:54:20 +08:00
llms.py feat(llm): add rate limiting and model override support to BaseLLM 2026-01-11 11:24:20 +08:00