Commit graph

776 commits

Author SHA1 Message Date
jinliyl
ad392738ab
feat(file-watcher): add clear-on-start option and remove redundant clears (#161) 2026-03-17 20:13:19 +08:00
jinli.yl
8cb0c11174 style(memory): update string formatting and logging messages 2026-03-17 17:17:48 +08:00
jinliyl
9a6cf2b994
Dev/token (#159)
* update

* refactor(memory): remove unnecessary type check and update error logging

* refactor(core): standardize logger import and update agentscope dependency

* fix(memory): disable console output and add logging for summarizer component

* feat(core): replace OpenAI token counter with custom ReMe token counter

- Replace OpenAITokenCounter with ReMeTokenCounter implementation
- Add support for HuggingFace mirror and configurable tokenizer
- Register ReMeTokenCounter as default token counter in registry
- Update config to use hf backend with Qwen2.5-7B-Instruct model

refactor(memory): convert token counting methods to async in message handlers

- Change count_str_token, stat_message, count_msgs_token to async methods
- Update format_msgs_to_str and context_check to use async token counting
- Modify _format_tool_result_output to support async token counting
- Adjust all dependent methods to await async token counting calls

feat(memory): add dialog persistence to in-memory storage

- Implement _append_messages_to_dialog for saving messages to JSONL files
- Add dialog_path parameter to ReMeInMemoryMemory constructor
- Persist messages to daily JSONL files based on timestamp grouping
- Update mark_messages_compressed to save and remove compressed messages
- Modify clear_content to persist all messages before clearing memory

refactor(ops): update token counter type hints and initialization

- Change BaseOp to use HuggingFaceTokenCounter instead of TokenCounterBase
- Update type annotations for as_token_counter property and parameters
- Remove direct token counter injection from Compactor and ContextChecker
- Pass as_token_counter parameter through service context mechanism

style(logging): improve error logging with exception details

- Replace logger.error with logger.exception in browser control tool
- Change logger.error to logger.exception in memory get tool error handling
- Add proper exception logging with stack trace information

chore(config): add token counter configuration to light YAML

- Add as_token_counters section with default hf backend configuration
- Configure Qwen/Qwen2.5-7B-Instruct model with mirror support enabled
- Set up pretrained_model_name_or_path and use_mirror parameters

test(context): update context check tests to async implementation

- Convert verify_context_check_invariants to async function
- Update context check test methods to use async calls
- Change stat_message calls to await async implementation
- Modify test_empty_messages and test_below_threshold_returns_all to async

* feat(core): implement context checking and memory management features

* refactor(core): replace direct loguru import with logger utility function

* refactor(reme): remove RuntimeContext dependency and simplify context checking

* feat(docs): add raw conversation persistence to ReMe framework
2026-03-17 11:07:31 +08:00
Zhouwk
67ad153a2a
添加Reme在Halumem和Locomo的实验结果 (#155)
* feat(reme): 添加配置选项以启用或禁用个人资料功能

- 在 ReMe 初始化方法中添加 enable_profile 参数,默认值为 True
- 根据 enable_profile 设置决定是否创建 profile 目录和设置 profile_dir
- 在 PersonalSummarizer 中根据 enable_profile 条件性地添加个人资料相关工具
- 在 PersonalRetriever 中根据 enable_profile 条件性地添加 ReadAllProfiles 工具
- 修改 profile_path 属性以在禁用个人资料时返回 None
- 修改 get_profile_handler 方法以在禁用个人资料时返回 None
- 为 enable_profile 参数添加文档说明其用于云向量存储场景

* refactor(benchmark): 重构LongMemEval基准测试中的ReMe实例管理

- 移除未使用的shutil导入
- 将固定的ReMe实例改为每个问题创建独立实例以实现隔离
- 更新LLM配置名称从qwen3-max-think到qwen-max-t
- 修改模型调用逻辑使用正确的model_name参数
- 添加qwen-flash和GPT-4o-mini等新模型配置
- 统一使用"User"作为用户名,通过集合名实现隔离
- 调整并发处理数从4降至1,批处理大小从10增至30
- 每个问题类型采样数从2增至4
- 添加异步上下文管理确保资源正确释放

* reformat 2 files

* refactor(benchmark): 重构长记忆评估中的模型配置

- 将原有的 eval_model_name 替换为专门的 retrieve_model_name 用于检索操作
- 添加对 qwen-max 模型配置的支持
- 更新参数解析器以支持新的检索模型参数
- 修改最大并发数默认值从 1 提升到 4
- 调整样本数量默认值从 4 减少到 1
- 统一模型参数命名规范,区分摘要、检索和评估模型
- 优化内存处理器初始化逻辑,支持独立的检索模型配置

* fix(benchmark): 移除数据路径默认值并设为必填参数

- 将LongMemEval评估脚本中的data_path参数改为必需参数
- 将HaluMem评估脚本中的data_path参数改为必需参数
- 删除了硬编码的默认文件路径配置
- 强制用户显式指定数据集文件路径以避免路径错误

* Update __init__.py

* Update __init__.py

* fix(benchmark): 修复ReMe评估中的模型配置和空值处理问题

- 移除了retrieve_memory调用中不需要的llm_config_name参数
- 修复了长字符串打印的换行格式问题
- 添加了eval_result为空时的初始化处理
- 在accuracy评估中加入了eval_model_name参数传递

* style(benchmark): 格式化模型名称打印输出

- 移除了多行字符串中的换行符和多余空格
- 将模型名称信息合并为单行连续显示
- 保持了原有的打印格式和信息完整性

* docs(readme): 更新文档添加实验结果表格

- 在英文版 README 中添加 🧪 Experiments 章节
- 添加 LoCoMo 和 HaluMem 两个基准测试的结果表格
- 在中文版 README_ZH 中添加 🧪 实验 章节
- 添加 LoCoMo 和 HaluMem 测试集的实验配置说明
- 添加完整的实验数据对比表格和评估协议说明

* docs(readme): 更新文档中的内存系统链接

- 为基于文件的记忆系统添加锚点链接
- 为基于向量库的记忆系统添加锚点链接
- 修复英文文档中的链接格式
- 修复中文文档中的链接格式和空行问题
2026-03-16 11:58:55 +08:00
zouyingcao
e5bb845196
refactor(cli): using AgentScope components to reimplement the reme_cli logic (#153)
* add: as_token_counters config for reme_cli

* add: reme_cli function

* update: format the terminal printing for reme_cli

* update: check for pre-commit

* single quotes for the inner dictionary keys

* update the usage of get_std_logger for pre-commit

* update the usage of get_std_logger for pre-commit

* add 'console_enabled' param in compactor&summarizer
2026-03-12 11:23:36 +08:00
jinliyl
8b1698451e
fin(emb_dim) (#150)
* fix(logger): handle file logging configuration errors gracefully

* fix(embedding): handle embedding dimension mismatches and improve caching

* fix(file-watcher): clear file store on changes to prevent stale data

* refactor(logger): update logger implementation and fix message translation

* refactor(logger): update logger configuration and add documentation
2026-03-10 18:19:00 +08:00
jinliyl
083ed6a137
update readme (#149) 2026-03-09 21:05:30 +08:00
hyp-001
8b45493634
增加locomo的代码 (#148)
Co-authored-by: huwen.hyp <huwen.hyp@alibaba-inc.com>
2026-03-09 16:09:06 +08:00
jinli.yl
f408d6ec4a docs(readme): update chinese badge text to simplified chinese 2026-03-07 15:57:00 +08:00
jinliyl
f4763a31da
docs(readme): update documentation with new features and installation guide (#146) 2026-03-07 15:54:08 +08:00
jinli.yl
d62c6a22a5 refactor(memory): move file utility functions to shared module 2026-03-07 15:30:45 +08:00
jinliyl
5e08aa48b8
refactor(memory): restructure file-based memory components and enhance message handling (#145) 2026-03-07 15:22:56 +08:00
jinliyl
d0c9d89092
feat(memory): add ContextChecker component for context size management (#144)
* feat(memory): add ContextChecker component for context size management

* refactor(memory): restructure file-based memory tools and update imports

* docs(readme): update documentation with detailed architecture and components

* docs(readme): update Chinese documentation with enhanced memory management diagrams

* refactor(cookbook): move cookbook files to test directory and clean up docs

* docs(readme): update link path for old version documentation

* docs(readme): update documentation with improved architecture diagrams and component details

* docs(readme): update documentation with improved clarity and structure

* refactor(docs): update in-memory memory documentation

* docs(readme): add experiment reproduction link to quickstart guide
2026-03-06 23:43:42 +08:00
jinli.yl
dcf97dc77f docs(readme): update documentation and examples 2026-03-06 16:34:04 +08:00
Sen Huang
6b742b6719
feat(memory): replace memory formatter with AsMsgHandler for enhanced message processing 2026-03-06 16:29:52 +08:00
jinli.yl
7e750a5c8e delete 2026-03-06 16:21:04 +08:00
jinli.yl
a0d3120d53 fix(tests): update context check tests to handle additional return value 2026-03-06 16:18:50 +08:00
zouyingcao
65971bafe3
Update: check the code&docs for evaluation on bfcl&appworld (#141)
* fix: df.columns bug

* fix: await for asynchronous method

* update: docs for bfcl&appworld quickstart

* update: benchmark/bfcl for new version quickstart

* slightly revise bfcl cookbook

* update for pre-commit

* handle boolean flags in split_into_trainval.py

* fix typo in faq.md
2026-03-06 16:11:39 +08:00
jinli.yl
ce53bc051a refactor(embedding): update environment variable names for API key and base URL 2026-03-06 16:04:01 +08:00
jinli.yl
46ffe42a40 feat(config): update ReMeLight initialization with default configurations 2026-03-06 15:50:47 +08:00
jinli.yl
2af9f329d2 docs(readme): update mermaid graph syntax in Chinese documentation 2026-03-06 15:43:24 +08:00
jinli.yl
c1e9faaeb2 feat(docs): update README with new pre_reasoning_hook method and enhanced examples 2026-03-06 15:36:38 +08:00
jinli.yl
32f9074235 refactor(memory): update import paths and enhance message token counting 2026-03-06 15:28:33 +08:00
jinli.yl
30278b4a4d style(tests): reorder imports in test_compactor.py 2026-03-06 02:09:55 +08:00
jinli.yl
22331ea963 refactor(tests): update test configurations and remove unused test file 2026-03-06 02:09:04 +08:00
jinli.yl
fc7b1cdba8 refactor(core): update registry registration syntax and improve code formatting 2026-03-06 01:57:29 +08:00
jinli.yl
57f8a7b42c feat(core): integrate AgentScope LLM support with enhanced memory management 2026-03-06 01:33:48 +08:00
jinli.yl
3dc3c4bf52 feat(memory): replace memory formatter with AsMsgHandler for enhanced message processing 2026-03-05 20:27:24 +08:00
jinliyl
3347506e22
feat(file-watcher): add configurable retry for file watcher (#140)
* feat(file-watcher): enhance file watcher with robust path validation and interruptible sleep

* feat(file-watcher): enhance file watcher with robust path validation and interruptible sleep
2026-03-05 14:34:00 +08:00
jinli.yl
2715c6fc90 docs: update documentation and configuration 2026-03-05 10:17:28 +08:00
jinli.yl
d7b7f6bf01 docs(readme): update file-based memory system title with ReMeLight 2026-03-04 19:22:59 +08:00
jinliyl
33aa67df4d
Improve clarity and consistency in README.md
Refine language for clarity and consistency throughout the README, including installation instructions, memory management descriptions, and community support sections.
2026-03-04 19:21:05 +08:00
jinliyl
9e2e98ef40
Dev/readme (#137)
* refactor(memory): rename copaw to reme and update module structure

* chore(release): bump version to 0.3.0.6b1

* refactor(docs): update README and ReMeLight implementation

* refactor(reme): rename ReMeCopaw to ReMeLight and remove CLI module

* docs(readme): update Chinese documentation with enhanced structure and content

* docs(readme): update Chinese documentation for context compression

* docs(readme): update context compression section header

* docs(readme): update documentation with installation and usage guide

* docs(readme): update Chinese documentation table

* chore(deps): update dependency extras configuration

* chore(test): remove deprecated test files for message operations

* docs(readme): update documentation with ReMeLight implementation changes
2026-03-04 18:43:21 +08:00
layla
3a4c1cae93
Fix docs: correct English badge link in README (#133)
Co-authored-by: jinliyl <6469360+jinliyl@users.noreply.github.com>
2026-03-04 17:40:43 +08:00
Salman Chishti
1ee7376fa4
Upgrade GitHub Actions for Node 24 compatibility (#136)
Signed-off-by: Salman Muin Kayser Chishti <13schishti@gmail.com>
2026-03-04 17:36:40 +08:00
jinli.yl
5b45392b1e refactor(reme): format compactor call parameters 2026-03-04 15:04:54 +08:00
jinli.yl
70282d4110 fix(core): update version and improve compactor message handling 2026-03-04 14:10:24 +08:00
jinli.yl
58ca1f4fac fix(core): update version and enhance compactor execution with service context 2026-03-04 13:51:55 +08:00
jinliyl
5584a5c239
feat(memory): add CoPaw file-based memory system with compaction and … (#134)
* feat(memory): add CoPaw file-based memory system with compaction and summarization

* feat(reme): add tool result cleanup and retention management

* fix(memory): resolve copaw memory processing and prompt formatting issues

* docs(reme_copaw): update documentation and initialization logic

* refactor(reme): remove override parameters from compact_tool_result

* feat(docs): update README to reflect CoPaw memory system integration

* chore(docs): update model names in documentation
2026-03-04 10:55:43 +08:00
Jiaji
2ce002fdc2
Merge pull request #128 from zouyingcao/main
Add FAQ.md about ReMe paper
2026-03-03 17:13:07 +08:00
Zhouwk
fd5c06bf0d
Update Halumemeval and Longmemeval ; Update Version 0.3.0.2 (#132)
* feat(reme): 添加配置选项以启用或禁用个人资料功能
2026-03-03 13:11:40 +08:00
zouyingcao
c5d47da0b5
Merge branch 'agentscope-ai:main' into main 2026-03-03 12:36:18 +08:00
Jiaji
44c4ce11f3
fix pre-commit bug (#131) 2026-03-03 12:23:58 +08:00
Zhouwk
9294e65dfb
Add bool trigger on Profile memory ; Update Longmemeval Eval and HalumemEval (#129)
* feat(reme): 添加配置选项以启用或禁用个人资料功能
2026-03-03 11:29:49 +08:00
caozouying.czy
dc441d86a7 check pre-commit for faq.md 2026-03-03 11:26:31 +08:00
jinli.yl
ab4dbd6220 fix(embedding): disable dimensions parameter by default 2026-03-03 10:50:56 +08:00
jinliyl
652ae29464
Comment out task_name in memory summary and retrieval 2026-03-03 10:17:28 +08:00
jinliyl
a3da3215e2
Update README.md
update readme
2026-03-03 10:16:35 +08:00
jinli.yl
11a42d02c1 feat(embedding): add conditional dimensions parameter support for OpenAI embeddings 2026-03-02 19:27:29 +08:00
Zhouwk
eff323105f
halumem和longmemeval的Benchmark评估代码 (#124)
* feat(benchmark): 添加 LongMemEval 评估功能和内存检索器

- 实现了 LongMemEval 数据集的评估管道
- 添加了 PersonalLongmemevalRetriever 和 PersonalLongmemevalSummarizer
- 创建了详细的统计分析工具 compute_stats.py
- 实现了完整的答案判断和准确性计算功能
- 集成了 ReMe 内存操作和查询功能
- 添加了性能指标和时间统计功能

* feat(benchmark): 添加内存准确性和完整性评估功能

- 实现了 evaluation_for_memory_accuracy 函数用于评估提取内存的准确性
- 实现了 evaluation_for_memory_integrity 函数用于评估内存完整性
- 创建了 MemoryIntegrityEvaluator 类来评估内存点覆盖情况
- 创建了 MemoryAccuracyEvaluator 类来评估提取内存的准确性
- 添加了 compute_memory_integrity_metrics 和 compute_memory_accuracy_metrics 统计函数
- 在 MetricsAggregator 中集成内存完整性和准确性指标计算
- 更新了命令行参数默认值:top_k 改为 10,batch_size 改为 16
- 重构了个人记忆汇总器中的工具循环逻辑
- 更新了评估提示词模板以支持内存质量评估
- 添加了新的检索配置和模型设置

* feat(memory): 添加个人记忆摘要器配置文件

- 新增 personal_halumem_summarizer_adddraft.yaml 配置文件
- 新增 personal_halumem_summarizer_original_backup.yaml 备份配置文件
- 实现记忆架构师系统提示和用户消息模板
- 实现个人资料代理系统提示和用户消息模板
- 支持生物特征和行为模式记忆存储
- 实现记忆去重和合并功能
- 支持用户个人资料的动态更新和删除操作

* feat(memory): 添加个人记忆摘要器配置文件

- 新增 personal_halumem_summarizer_adddraft.yaml 配置文件
- 新增 personal_halumem_summarizer_original_backup.yaml 备份配置文件
- 实现记忆架构师系统提示和用户消息模板
- 实现个人资料代理系统提示和用户消息模板
- 支持生物特征和行为模式记忆存储
- 实现记忆去重和合并功能
- 支持用户个人资料的动态更新和删除操作

* docs(readme): 添加 ReMe Memory Agent 详细介绍文档

- 创建英文版 README.md 包含核心概念、架构设计和使用指南
- 创建中文版 README_ZH.md 提供完整的本地化文档
- 介绍 Agent 驱动的记忆管理理念和层次化检索机制
- 详述项目架构包括 ReMeSummarizer 和 ReMeRetriever 组件
- 提供快速开始示例和程序化内存操作方法
- 展示 LoCoMo、LongMemEval、HaluMem 基准测试结果
- 包含完整的项目结构说明和配置要求

* chore(config): 移除配置文件中的API密钥

- 从配置文件中删除FLOW_LLM_API_KEY环境变量设置
- 移除相关的API密钥配置项
- 更新配置文档以反映新的安全实践
- 确保敏感信息不再硬编码在配置文件中
- 添加注释说明如何通过环境变量方式配置API密钥

* fix(benchmark): 修复模型调用和配置参数问题

- 修正了reme.get_llm方法的参数传递,移除冗余的name参数
- 添加了qwen3-max模型的配置支持
- 调整了默认并发数从16降至4以提高稳定性
- 修改算法版本默认值从longmemeval和v1统一为default
- 减少每类样本数量默认值从16至2以优化测试效率

* feat(benchmark): 添加记忆准确性和完整性评估功能

- 修改了 simple_request_for_json 调用以支持模型名称参数
- 新增 evaluation_for_memory_accuracy 函数用于评估记忆准确性
- 新增 evaluation_for_memory_integrity 函数用于评估记忆完整性
- 将默认模型名称从 qwen3-max 更改为 None
- 更新提取记忆逻辑以过滤 time_int 和 when_to_use 字段
- 新增 MemoryIntegrityEvaluator 类用于评估记忆完整性
- 新增 MemoryAccuracyEvaluator 类用于评估记忆准确性
- 添加 compute_memory_integrity_metrics 方法计算记忆完整性指标
- 添加 compute_memory_accuracy_metrics 方法计算记忆准确性指标
- 配置多种新 LLM 模型包括 qwen-plus-t、qwen-max-t、gpt-4o-mini 等
- 初始化完整性评估器和准确性评估器实例
- 在会话数据中添加记忆完整性和准确性评估结果
- 收集记忆完整性记录和准确性记录用于统计
- 在最终结果中包含记忆完整性和准确性指标
- 更新摘要打印方法显示记忆完整性和准确性统计信息
- 更新默认评估模型为 gpt-4o-mini-2024-07-18

* docs(readme): 删除 ReMe Memory Agent 的中英文文档

- 移除英文版 README.md 中关于 ReMe Memory Agent 的详细介绍
- 删除中文版 README_ZH.md 中关于 ReMe Memory Agent 的完整文档
- 清理了包括架构图、功能特性、快速开始和实验数据在内的所有文档内容
2026-03-02 18:58:37 +08:00