Zhouwk
72eabfa858
fix(user profile): update locomo benchmark and update vector based profile code ( #225 )
...
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
* feat(reme): 添加配置选项以启用或禁用个人资料功能
- 在 ReMe 初始化方法中添加 enable_profile 参数，默认值为 True
- 根据 enable_profile 设置决定是否创建 profile 目录和设置 profile_dir
- 在 PersonalSummarizer 中根据 enable_profile 条件性地添加个人资料相关工具
- 在 PersonalRetriever 中根据 enable_profile 条件性地添加 ReadAllProfiles 工具
- 修改 profile_path 属性以在禁用个人资料时返回 None
- 修改 get_profile_handler 方法以在禁用个人资料时返回 None
- 为 enable_profile 参数添加文档说明其用于云向量存储场景
* refactor(benchmark): 重构LongMemEval基准测试中的ReMe实例管理
- 移除未使用的shutil导入
- 将固定的ReMe实例改为每个问题创建独立实例以实现隔离
- 更新LLM配置名称从qwen3-max-think到qwen-max-t
- 修改模型调用逻辑使用正确的model_name参数
- 添加qwen-flash和GPT-4o-mini等新模型配置
- 统一使用"User"作为用户名，通过集合名实现隔离
- 调整并发处理数从4降至1，批处理大小从10增至30
- 每个问题类型采样数从2增至4
- 添加异步上下文管理确保资源正确释放
* reformat 2 files
* refactor(benchmark): 重构长记忆评估中的模型配置
- 将原有的 eval_model_name 替换为专门的 retrieve_model_name 用于检索操作
- 添加对 qwen-max 模型配置的支持
- 更新参数解析器以支持新的检索模型参数
- 修改最大并发数默认值从 1 提升到 4
- 调整样本数量默认值从 4 减少到 1
- 统一模型参数命名规范，区分摘要、检索和评估模型
- 优化内存处理器初始化逻辑，支持独立的检索模型配置
* fix(benchmark): 移除数据路径默认值并设为必填参数
- 将LongMemEval评估脚本中的data_path参数改为必需参数
- 将HaluMem评估脚本中的data_path参数改为必需参数
- 删除了硬编码的默认文件路径配置
- 强制用户显式指定数据集文件路径以避免路径错误
* Update __init__.py
* Update __init__.py
* fix(benchmark): 修复ReMe评估中的模型配置和空值处理问题
- 移除了retrieve_memory调用中不需要的llm_config_name参数
- 修复了长字符串打印的换行格式问题
- 添加了eval_result为空时的初始化处理
- 在accuracy评估中加入了eval_model_name参数传递
* style(benchmark): 格式化模型名称打印输出
- 移除了多行字符串中的换行符和多余空格
- 将模型名称信息合并为单行连续显示
- 保持了原有的打印格式和信息完整性
* docs(readme): 更新文档添加实验结果表格
- 在英文版 README 中添加 🧪 Experiments 章节
- 添加 LoCoMo 和 HaluMem 两个基准测试的结果表格
- 在中文版 README_ZH 中添加 🧪 实验 章节
- 添加 LoCoMo 和 HaluMem 测试集的实验配置说明
- 添加完整的实验数据对比表格和评估协议说明
* docs(readme): 更新文档中的内存系统链接
- 为基于文件的记忆系统添加锚点链接
- 为基于向量库的记忆系统添加锚点链接
- 修复英文文档中的链接格式
- 修复中文文档中的链接格式和空行问题
* docs(readme): update experimental results section in documentation
- Remove outdated experimental data placeholder "Coming soon..."
- Add complete evaluation results for LoCoMo and HaluMem benchmarks
- Include detailed performance metrics tables for all memory methods
- Update experimental settings description with ReMe backbone details
- Align evaluation protocol information with LLM-as-a-Judge approach
- Maintain consistent formatting between English and Chinese documentation
* docs(benchmark): add quick start guides for halumem and longmemeval experiments
- Created HaluMem experiment quick start guide with ReMe integration setup
- Added detailed steps for installing ReMe environment using conda
- Included repository cloning instructions for HaluMem benchmark
- Provided complete command examples for running HaluMem experiments
- Created LongMeMEval quick start guide with data download procedures
- Added wget commands for downloading cleaned dataset files
- Included evaluation script instructions for computing experiment statistics
- Documented parameter configurations for different model types and batch sizes
* docs(longmemeval): update quickstart guide documentation
- Changed project name from Halumem to Longmemeval in title
- Updated description to reference Longmemeval experiments instead of Halumem
- Maintained existing ReMe integration instructions unchanged
* chore(logger): add test comment to logger configuration
- Added test comment in logger utility function
- Removed duplicate log handling by keeping the remove() call
* chore(logger): add test comment to logger configuration
- Added test comment in logger utility function
- Removed duplicate log handling by keeping the remove() call
* feat(core): add file logging capability to application
- Added log_to_file parameter to Application class constructor
- Integrated log_to_file option in logger initialization
- Updated ServiceContext to support file logging configuration
- Modified init_logger function to conditionally enable file logging
- Added log_to_file field to ServiceConfig schema
- Updated ReMe class to include file logging option
- Wrapped file logging setup in conditional check to prevent unnecessary operations
* docs(benchmark): update HaluMem quickstart guide with dataset download instructions
- Replace repository cloning with direct dataset download using curl
- Add commands to download HaluMem-Medium.jsonl and HaluMem-Long.jsonl files
- Include both official Hugging Face and mirror download sources
- Update data path reference from nested directory to local data folder
- Add dataset page link and mirror usage instructions for mainland China access
* feat(memory): add profile retrieval tool and refactor profile management
- Introduce RetrieveProfile tool for fetching specific user profiles
- Refactor ProfileHandler to support both filesystem and vector backends
- Add async methods to ProfileHandler with synchronous fallbacks
- Update PersonalRetriever to support two-stage profile and memory retrieval
- Enhance PersonalSummarizer with improved tool partitioning logic
- Add profile_backend, profile_store_name, and profile_max_capacity configuration options
- Replace direct ProfileHandler imports with get_profile_handler method
- Implement profile search functionality with dedicated prompts and workflows
- Add FileProfileBackend and VectorProfileBackend implementations
- Update base memory tool with new profile configuration parameters
* feat(profile): add custom profile collection name support
- Add profile_collection_name parameter to Application constructor
- Allow custom database collection name for vector profiles instead of default suffix
- Update profile vector store configuration logic to use custom collection name
- Modify _ensure_profile_vector_store_config to handle custom collection names
- Update docstring with detailed parameter descriptions for profile configuration options
* test(history): add single history id acceptance test for multiple mode
- Add test case to verify multiple-mode history lookup accepts a single history_id string
- Create FakeVectorStore stub with minimal implementation for ReadHistory tests
- Return requested history node from vector store mock
- Initialize ReadHistory tool with multiple mode enabled
- Add pylint disable comment for protected access to vector store property
* refactor(memory): update profile handler and vector tools with improved formatting and error handling
- Add module docstring to profiles/__init__.py
- Add pylint disable comments for no-name-in-module and missing-function-docstring
- Format long error message in ProfileHandler.sync_run method for better readability
- Reformat parameters in ProfileHandler.aadd method to separate lines
- Update model_copy call in reme.py to span multiple lines for better readability
- Format aadd_batch call in update_profile.py to span multiple lines
* feat(profiles): add profile management system with file and vector storage backends
- Add FileProfileBackend for filesystem-based profile persistence
- Add VectorProfileBackend for vector store-based profile management
- Create abstract BaseProfileBackend interface for profile operations
- Implement ProfileVectorHandler for vector-backed profile storage
- Add RetrieveProfile tool for semantic profile retrieval
- Update eval_reme.py to use user_message_s2 for retriever prompt
- Modify eval_reme.yaml to use {profiles} instead of {user_profile}
- Implement complete CRUD operations for profile management
- Add batch operations for efficient profile handling
- Include search functionality with semantic matching capabilities
- Add capacity limits and automatic cleanup for profile storage
* docs(profiles): add comprehensive docstrings for profile backend and handler methods
- Added documentation for get_all_sync, get_by_sync, delete_sync, delete_all_sync methods
- Documented add_sync and add_batch_sync functionality with deduping behavior
- Added docstrings for update_sync and search_sync operations
- Updated ProfileHandler.format_node method with proper documentation
- Refactored private _format_node to public format_node method
- Added comprehensive documentation for profile vector handler operations
- Documented _vector_profile_matches, _get_by_profile_id, _get_by_profile_key helper methods
- Added docstrings for retrieve_profile functionality and formatting methods
2026-04-30 10:19:36 +08:00
Zhouwk
625d184ca1
添加log_to_file的开关 ( #205 )
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
2026-04-13 10:49:26 +08:00
Zhouwk
935e886af3
更新longmemeval和halumem的quick start ( #194 )
...
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
* feat(reme): 添加配置选项以启用或禁用个人资料功能
- 在 ReMe 初始化方法中添加 enable_profile 参数，默认值为 True
- 根据 enable_profile 设置决定是否创建 profile 目录和设置 profile_dir
- 在 PersonalSummarizer 中根据 enable_profile 条件性地添加个人资料相关工具
- 在 PersonalRetriever 中根据 enable_profile 条件性地添加 ReadAllProfiles 工具
- 修改 profile_path 属性以在禁用个人资料时返回 None
- 修改 get_profile_handler 方法以在禁用个人资料时返回 None
- 为 enable_profile 参数添加文档说明其用于云向量存储场景
* refactor(benchmark): 重构LongMemEval基准测试中的ReMe实例管理
- 移除未使用的shutil导入
- 将固定的ReMe实例改为每个问题创建独立实例以实现隔离
- 更新LLM配置名称从qwen3-max-think到qwen-max-t
- 修改模型调用逻辑使用正确的model_name参数
- 添加qwen-flash和GPT-4o-mini等新模型配置
- 统一使用"User"作为用户名，通过集合名实现隔离
- 调整并发处理数从4降至1，批处理大小从10增至30
- 每个问题类型采样数从2增至4
- 添加异步上下文管理确保资源正确释放
* reformat 2 files
* refactor(benchmark): 重构长记忆评估中的模型配置
- 将原有的 eval_model_name 替换为专门的 retrieve_model_name 用于检索操作
- 添加对 qwen-max 模型配置的支持
- 更新参数解析器以支持新的检索模型参数
- 修改最大并发数默认值从 1 提升到 4
- 调整样本数量默认值从 4 减少到 1
- 统一模型参数命名规范，区分摘要、检索和评估模型
- 优化内存处理器初始化逻辑，支持独立的检索模型配置
* fix(benchmark): 移除数据路径默认值并设为必填参数
- 将LongMemEval评估脚本中的data_path参数改为必需参数
- 将HaluMem评估脚本中的data_path参数改为必需参数
- 删除了硬编码的默认文件路径配置
- 强制用户显式指定数据集文件路径以避免路径错误
* Update __init__.py
* Update __init__.py
* fix(benchmark): 修复ReMe评估中的模型配置和空值处理问题
- 移除了retrieve_memory调用中不需要的llm_config_name参数
- 修复了长字符串打印的换行格式问题
- 添加了eval_result为空时的初始化处理
- 在accuracy评估中加入了eval_model_name参数传递
* style(benchmark): 格式化模型名称打印输出
- 移除了多行字符串中的换行符和多余空格
- 将模型名称信息合并为单行连续显示
- 保持了原有的打印格式和信息完整性
* docs(readme): 更新文档添加实验结果表格
- 在英文版 README 中添加 🧪 Experiments 章节
- 添加 LoCoMo 和 HaluMem 两个基准测试的结果表格
- 在中文版 README_ZH 中添加 🧪 实验 章节
- 添加 LoCoMo 和 HaluMem 测试集的实验配置说明
- 添加完整的实验数据对比表格和评估协议说明
* docs(readme): 更新文档中的内存系统链接
- 为基于文件的记忆系统添加锚点链接
- 为基于向量库的记忆系统添加锚点链接
- 修复英文文档中的链接格式
- 修复中文文档中的链接格式和空行问题
* docs(readme): update experimental results section in documentation
- Remove outdated experimental data placeholder "Coming soon..."
- Add complete evaluation results for LoCoMo and HaluMem benchmarks
- Include detailed performance metrics tables for all memory methods
- Update experimental settings description with ReMe backbone details
- Align evaluation protocol information with LLM-as-a-Judge approach
- Maintain consistent formatting between English and Chinese documentation
* docs(benchmark): add quick start guides for halumem and longmemeval experiments
- Created HaluMem experiment quick start guide with ReMe integration setup
- Added detailed steps for installing ReMe environment using conda
- Included repository cloning instructions for HaluMem benchmark
- Provided complete command examples for running HaluMem experiments
- Created LongMeMEval quick start guide with data download procedures
- Added wget commands for downloading cleaned dataset files
- Included evaluation script instructions for computing experiment statistics
- Documented parameter configurations for different model types and batch sizes
* docs(longmemeval): update quickstart guide documentation
- Changed project name from Halumem to Longmemeval in title
- Updated description to reference Longmemeval experiments instead of Halumem
- Maintained existing ReMe integration instructions unchanged
2026-04-07 14:20:30 +08:00
hyp-001
8b45493634
增加locomo的代码 ( #148 )
...
Co-authored-by: huwen.hyp <huwen.hyp@alibaba-inc.com>
2026-03-09 16:09:06 +08:00
jinliyl
d0c9d89092
feat(memory): add ContextChecker component for context size management ( #144 )
...
* feat(memory): add ContextChecker component for context size management
* refactor(memory): restructure file-based memory tools and update imports
* docs(readme): update documentation with detailed architecture and components
* docs(readme): update Chinese documentation with enhanced memory management diagrams
* refactor(cookbook): move cookbook files to test directory and clean up docs
* docs(readme): update link path for old version documentation
* docs(readme): update documentation with improved architecture diagrams and component details
* docs(readme): update documentation with improved clarity and structure
* refactor(docs): update in-memory memory documentation
* docs(readme): add experiment reproduction link to quickstart guide
2026-03-06 23:43:42 +08:00
zouyingcao
65971bafe3
Update: check the code&docs for evaluation on bfcl&appworld ( #141 )
...
* fix: df.columns bug
* fix: await for asynchronous method
* update: docs for bfcl&appworld quickstart
* update: benchmark/bfcl for new version quickstart
* slightly revise bfcl cookbook
* update for pre-commit
* handle boolean flags in split_into_trainval.py
* fix typo in faq.md
2026-03-06 16:11:39 +08:00
Zhouwk
fd5c06bf0d
Update Halumemeval and Longmemeval ; Update Version 0.3.0.2 ( #132 )
...
* feat(reme): 添加配置选项以启用或禁用个人资料功能
2026-03-03 13:11:40 +08:00
Jiaji
44c4ce11f3
fix pre-commit bug ( #131 )
2026-03-03 12:23:58 +08:00
Zhouwk
9294e65dfb
Add bool trigger on Profile memory ; Update Longmemeval Eval and HalumemEval ( #129 )
...
* feat(reme): 添加配置选项以启用或禁用个人资料功能
2026-03-03 11:29:49 +08:00
Zhouwk
eff323105f
halumem和longmemeval的Benchmark评估代码 ( #124 )
...
* feat(benchmark): 添加 LongMemEval 评估功能和内存检索器
- 实现了 LongMemEval 数据集的评估管道
- 添加了 PersonalLongmemevalRetriever 和 PersonalLongmemevalSummarizer
- 创建了详细的统计分析工具 compute_stats.py
- 实现了完整的答案判断和准确性计算功能
- 集成了 ReMe 内存操作和查询功能
- 添加了性能指标和时间统计功能
* feat(benchmark): 添加内存准确性和完整性评估功能
- 实现了 evaluation_for_memory_accuracy 函数用于评估提取内存的准确性
- 实现了 evaluation_for_memory_integrity 函数用于评估内存完整性
- 创建了 MemoryIntegrityEvaluator 类来评估内存点覆盖情况
- 创建了 MemoryAccuracyEvaluator 类来评估提取内存的准确性
- 添加了 compute_memory_integrity_metrics 和 compute_memory_accuracy_metrics 统计函数
- 在 MetricsAggregator 中集成内存完整性和准确性指标计算
- 更新了命令行参数默认值：top_k 改为 10，batch_size 改为 16
- 重构了个人记忆汇总器中的工具循环逻辑
- 更新了评估提示词模板以支持内存质量评估
- 添加了新的检索配置和模型设置
* feat(memory): 添加个人记忆摘要器配置文件
- 新增 personal_halumem_summarizer_adddraft.yaml 配置文件
- 新增 personal_halumem_summarizer_original_backup.yaml 备份配置文件
- 实现记忆架构师系统提示和用户消息模板
- 实现个人资料代理系统提示和用户消息模板
- 支持生物特征和行为模式记忆存储
- 实现记忆去重和合并功能
- 支持用户个人资料的动态更新和删除操作
* feat(memory): 添加个人记忆摘要器配置文件
- 新增 personal_halumem_summarizer_adddraft.yaml 配置文件
- 新增 personal_halumem_summarizer_original_backup.yaml 备份配置文件
- 实现记忆架构师系统提示和用户消息模板
- 实现个人资料代理系统提示和用户消息模板
- 支持生物特征和行为模式记忆存储
- 实现记忆去重和合并功能
- 支持用户个人资料的动态更新和删除操作
* docs(readme): 添加 ReMe Memory Agent 详细介绍文档
- 创建英文版 README.md 包含核心概念、架构设计和使用指南
- 创建中文版 README_ZH.md 提供完整的本地化文档
- 介绍 Agent 驱动的记忆管理理念和层次化检索机制
- 详述项目架构包括 ReMeSummarizer 和 ReMeRetriever 组件
- 提供快速开始示例和程序化内存操作方法
- 展示 LoCoMo、LongMemEval、HaluMem 基准测试结果
- 包含完整的项目结构说明和配置要求
* chore(config): 移除配置文件中的API密钥
- 从配置文件中删除FLOW_LLM_API_KEY环境变量设置
- 移除相关的API密钥配置项
- 更新配置文档以反映新的安全实践
- 确保敏感信息不再硬编码在配置文件中
- 添加注释说明如何通过环境变量方式配置API密钥
* fix(benchmark): 修复模型调用和配置参数问题
- 修正了reme.get_llm方法的参数传递，移除冗余的name参数
- 添加了qwen3-max模型的配置支持
- 调整了默认并发数从16降至4以提高稳定性
- 修改算法版本默认值从longmemeval和v1统一为default
- 减少每类样本数量默认值从16至2以优化测试效率
* feat(benchmark): 添加记忆准确性和完整性评估功能
- 修改了 simple_request_for_json 调用以支持模型名称参数
- 新增 evaluation_for_memory_accuracy 函数用于评估记忆准确性
- 新增 evaluation_for_memory_integrity 函数用于评估记忆完整性
- 将默认模型名称从 qwen3-max 更改为 None
- 更新提取记忆逻辑以过滤 time_int 和 when_to_use 字段
- 新增 MemoryIntegrityEvaluator 类用于评估记忆完整性
- 新增 MemoryAccuracyEvaluator 类用于评估记忆准确性
- 添加 compute_memory_integrity_metrics 方法计算记忆完整性指标
- 添加 compute_memory_accuracy_metrics 方法计算记忆准确性指标
- 配置多种新 LLM 模型包括 qwen-plus-t、qwen-max-t、gpt-4o-mini 等
- 初始化完整性评估器和准确性评估器实例
- 在会话数据中添加记忆完整性和准确性评估结果
- 收集记忆完整性记录和准确性记录用于统计
- 在最终结果中包含记忆完整性和准确性指标
- 更新摘要打印方法显示记忆完整性和准确性统计信息
- 更新默认评估模型为 gpt-4o-mini-2024-07-18
* docs(readme): 删除 ReMe Memory Agent 的中英文文档
- 移除英文版 README.md 中关于 ReMe Memory Agent 的详细介绍
- 删除中文版 README_ZH.md 中关于 ReMe Memory Agent 的完整文档
- 清理了包括架构图、功能特性、快速开始和实验数据在内的所有文档内容
2026-03-02 18:58:37 +08:00
caozouying.czy
bb0d0ee248
Merge branch 'main' into dev_0227
2026-02-27 14:06:11 +08:00
caozouying.czy
3760c22d45
update for pre-commit check
2026-02-27 13:30:22 +08:00
jinli.yl
115373ab30
refactor(benchmark): remove unused llm config and update test parameters
2026-02-26 18:20:12 +08:00
caozouying.czy
dfadfed067
update: evaluation scripts on bfcl&appworld benchmarks
2026-02-26 12:48:19 +08:00
jinli.yl
55d61f1dc5
refactor(core): update registry naming and application configuration
2026-02-07 15:05:02 +08:00
jinliyl
f6ca7a733a
Merge branch 'main' into dev_0205
2026-02-06 17:47:44 +08:00
jinli.yl
1dd81f9c25
refactor(core): update config parsing and memory management system
2026-02-06 15:02:41 +08:00
方应
5ffb85a10e
fix(halumem): 修复评估模型名称变量引用错误
...
- 将reme_model_name更正为eval_model_name以匹配正确的模型配置
2026-02-03 14:39:08 +08:00
方应
d83551c2cf
feat(benchmark): 更新ReMe评估配置参数
...
- 将最大并发数从2调整为1，批量大小从20调整为40
- 算法版本从halumem更新为v1
- 添加了新的reme_model_name参数用于模型名称配置
- 在MemoryProcessor中使用reme_model_name替代eval_model_name
- 主函数中添加batch_size参数传递
- 命令行解析器中添加batch_size参数选项
- 在ReMe中将ReadHistory替换为ReadHistoryV2并配置新参数
2026-02-03 14:31:04 +08:00
jinli.yl
6a7396732b
feat(reme): add comprehensive memory management API
2026-01-31 03:40:38 +08:00
jinli.yl
c7ba031cea
feat(benchmark): add enable_thinking_params configuration option
2026-01-31 02:50:37 +08:00
jinli.yl
5a4077e68c
fix(halumem): correct async context manager implementation
2026-01-31 01:25:23 +08:00
jinli.yl
0163cb25ce
feat(cmd_service): initialize service context on command service startup
2026-01-31 01:17:00 +08:00
jinli.yl
3680571c94
refactor(core): simplify component registration and improve application lifecycle management
2026-01-31 01:05:31 +08:00
方应
61071ab654
feat(benchmark): 添加ReMe模型名称配置选项
...
- 在EvalConfig中新增reme_model_name参数，默认值为qwen-flash
- 修改ReMe实例化方式，传入配置的模型名称参数
- 更新函数签名以支持reme_model_name参数传递
- 调整命令行参数解析，区分reme_model_name和eval_model_name
- 将默认reme模型从硬编码改为可配置参数
2026-01-30 17:17:06 +08:00
方应
053c537845
feat(agent): 添加 Halumem 版本的记忆检索器和摘要器
...
- 添加 PersonalHalumemRetriever 和 PersonalHalumemSummarizer 类
- 在 memory 模块中注册新的检索器和摘要器
- 添加 UpdateProfileFilterOlder、DeleteProfile 和 AddProfile 工具
- 将 AddDraftAndRetrieveSimilarMemory 重命名为 AddAndRetrieveSimilarMemory
- 修改配置文件中的默认模型名称为 qwen-flash
- 在 benchmark 中添加 Halumem 评估支持和实时更新功能
- 降低 ProfileHandler 的最大容量限制并添加重复节点过滤逻辑
- 在 ReMe 中添加 halumem 版本的记忆代理配置
2026-01-30 16:30:18 +08:00
jinli.yl
cf62a73228
refactor(memory): restructure memory modules and add versioned personal memory agents
2026-01-29 11:21:50 +08:00
jinli.yl
12911e5148
fix(halumem): correct tmp directory path and update evaluation model default
2026-01-29 01:26:53 +08:00
jinli.yl
3989a3c1c4
refactor(memory): update memory target mappings and profile handling
2026-01-28 20:38:34 +08:00
jinli.yl
22a6321661
refactor(memory): optimize vector store operations and enhance memory management
2026-01-27 01:57:50 +08:00
jinli.yl
85a843ee2e
feat(benchmark): add configurable evaluation model for ReMe benchmark
2026-01-27 01:41:34 +08:00
jinli.yl
e63a3fa632
feat(core): add simple request methods and improve memory management
2026-01-27 01:07:39 +08:00
