Commit graph

529 commits

Author SHA1 Message Date
jinli.yl
db510dd96e feat(bench): add human-in-the-loop evaluation framework for AI memory systems 2026-01-20 20:25:26 +08:00
jinli.yl
d145a67843 feat(config): update default LLM model configuration
- Changed default model from qwen3-30b-a3b-instruct-2507 to qwen-flash
- Updated evaluation tools to use EVALUATION_PROMPT_FOR_QUESTION instead of QUESTION2
- Added new PROMPT_MEMZERO_JSON2 configuration with context priority rules
- Modified llm_request_for_json to use qwen-flash as default model
- Updated halumem evaluation to specify qwen3-max model explicitly for certain requests
2026-01-19 16:52:35 +08:00
jinli.yl
e7a36067eb refactor(llm): replace concurrency control with request rate limiting 2026-01-19 01:02:55 +08:00
jinli.yl
74c1386a69 refactor(mem_agent): optimize agent execution and enhance evaluation pipeline 2026-01-19 00:46:52 +08:00
jinli.yl
08b771b6c4 feat(mem-agent): introduce version 4 memory agents and tools 2026-01-18 15:50:46 +08:00
jinli.yl
e6ad682ede feat(core): add ReMe V3 implementation with optional MCP client and enhanced filtering 2026-01-17 01:15:49 +08:00
jinli.yl
4d312ea682 feat(memory): add metadata toggle option to vector retrieval tool 2026-01-14 18:18:37 +08:00
jinliyl
12efcc9224
Merge pull request #75 from agentscope-ai/halumem_bugfix
Halumem bugfix
2026-01-14 16:53:55 +08:00
jinli.yl
681fb22170 feat(embedding): update dimensions parameter to support optional type 2026-01-14 16:21:33 +08:00
jinli.yl
ef7c4daa9a feat(memory): add memory management tools and agents for AI system 2026-01-14 14:03:52 +08:00
jinli.yl
327dc58f70 feat(bench): add HaluMem dataset statistics analyzer and update LLM concurrency control
- Add new analyze_dataset_stats.py script for comprehensive HaluMem dataset analysis
- Include statistics for user sessions, dialogues, content lengths and chunk distributions
- Replace rate limiting with concurrency control in BaseLLM using semaphore mechanism
- Update configuration to use max_concurrency instead of max_rps and rps_window
- Modify dialogue formatting to include only user messages in evaluation
- Add percentile calculations and detailed content size distribution metrics
- Implement session splitting logic based on character length thresholds
- Provide per-user statistics and summary tables for dataset analysis
- Refactor BaseLLM to use internal _chat_impl and _stream_chat_impl methods
- Remove rate limiting locks and timestamps from LLM initialization
- Add command-line interface for dataset statistics analysis tool
2026-01-14 10:14:35 +08:00
jinli.yl
9b69d96b76 refactor(llm): replace recursive rate limiting with loop-based implementation 2026-01-14 00:58:18 +08:00
jinli.yl
25fee2e67f refactor(llm): optimize rate limiting implementation with recursive checks 2026-01-14 00:54:18 +08:00
jinli.yl
e64667ba77 chore(config): reduce RPS limits for AI model requests
- Lowered max_rps from 20 to 2 to prevent rate limiting issues
- Reduced rps_window from 10 to 1 for stricter request throttling
- Updated default configuration values for better performance stability
2026-01-14 00:47:28 +08:00
jinli.yl
de6e3968a3 refactor(llm): optimize rate limiting implementation 2026-01-14 00:44:32 +08:00
jinli.yl
52a4b66d59 refactor(bench): update halumem evaluation with concurrent session processing
- Increased max_rps from 9 to 20 in default config
- Removed tqdm progress bar for session processing
- Implemented asyncio semaphore for session concurrency control
- Added parallel processing of sessions within each user
- Updated logging to show session count per user
- Changed user processing from concurrent to sequential
- Added completion tracking with progress indicators
- Modified output formatting for better readability
2026-01-14 00:13:55 +08:00
jinli.yl
f0573a09d9 feat(bench): add progress bar for session processing in halumem evaluation
- Integrated tqdm library for progress tracking
- Added progress bar display for user sessions processing
- Implemented session-by-session progress updates with custom description
- Maintained existing session processing logic while adding visual feedback
- Preserved all original functionality including logging and data handling
2026-01-14 00:02:26 +08:00
jinli.yl
b8124fe31a feat(benchmark): add HaluMem baseline evaluation and analysis tools 2026-01-13 23:59:55 +08:00
jinliyl
6125dff01e
Merge pull request #73 from agentscope-ai/halumem_eval
Halumem eval
2026-01-13 14:03:19 +08:00
jinli.yl
59c0702759 feat(halumem): update memory operations to include agent messages and success flags 2026-01-13 14:02:29 +08:00
jinli.yl
581482eeb0 feat(mem_agent): introduce version 2 memory agents and tools
- Add ReMeRetrieverV2 and ReMeSummarizerV2 components
- Implement new memory tools including AddMemoryDrafts, RetrieveMemories, UpdateMemories
- Create PersonalSummarizerV2 with three-step workflow for memory management
- Add simplified evaluation script for ReMe on HaluMem benchmark
- Update base memory agent with enhanced logging capabilities
- Introduce new prompt formats and evaluation methods for question answering
- Modify LLM utilities and message formatting with markdown header stripping option
2026-01-13 01:54:20 +08:00
jinli.yl
5a0c1309f1 feat(bench): add statistics computation script for benchmark evaluation 2026-01-11 16:17:51 +08:00
jinli.yl
e302ec72cb feat(llm): add rate limiting and model override support to BaseLLM 2026-01-11 11:24:20 +08:00
jinli.yl
2e2d96a53e feat(vector-store): add delete_all method and improve evaluation pipeline 2026-01-10 22:47:56 +08:00
jinli.yl
9f114cb9b4 feat(benchmark): add HaluMem evaluation suite with ReMe integration 2026-01-10 22:11:20 +08:00
jinli.yl
5054cf22df feat(vector-store): add sorting capability to list operations across all backends 2026-01-09 21:09:30 +08:00
jinli.yl
a172a6854b feat(vector-store): add sorting capability to list operations across all backends 2026-01-09 19:50:42 +08:00
jinli.yl
2d98169d80 style(formatting): standardize code formatting and logging statements 2026-01-09 17:55:11 +08:00
jinli.yl
0b7843f557 feat(core): implement memory node tracking and embedding text truncation 2026-01-09 17:55:11 +08:00
caozouying.czy
d91fc6a14c update docs/cookbook/appworld/quickstart.md for better reproduction 2026-01-09 17:18:53 +08:00
jinli.yl
1ada96291e refactor(core): restructure module imports and enhance embedding functionality 2026-01-08 18:04:33 +08:00
jinli.yl
cf8ee9f88e refactor(vector_store): replace async_iter_workspace_nodes with async_list_workspace_nodes 2026-01-08 16:39:28 +08:00
jinli.yl
59aa253e85 refactor(core): move get_now_time function and update imports 2026-01-08 11:39:16 +08:00
jinli.yl
b3d68dbc02 refactor(llm): update type hints and abstract method definitions 2026-01-08 11:37:02 +08:00
jinli.yl
554eec1cb9 chore(version): bump version to 0.2.0.6 2026-01-07 23:43:57 +08:00
jinli.yl
09d12c7d72 chore(deps): update flowllm dependency versions 2026-01-07 23:43:08 +08:00
jinli.yl
416206d332 feat(mem_agent): implement memory agent architecture with specialized summarizers and retrievers 2026-01-07 18:10:30 +08:00
caozouying.czy
f493ba2f3a update README.md reported results 2026-01-07 15:27:22 +08:00
jinli.yl
53aaad2a28 refactor(memory): move memory tools to dedicated module and update imports 2026-01-07 10:10:03 +08:00
jinli.yl
29a6ee1fba refactor(memory): restructure memory tools and add base memory agent 2026-01-07 00:56:22 +08:00
jinli.yl
d29aff4e6c refactor(memory): update memory tool implementations and parameters 2026-01-06 23:46:51 +08:00
jinliyl
5306ff7d40
Merge pull request #65 from agentscope-ai/dev_0105
Dev 0105
2026-01-06 15:13:33 +08:00
jinli.yl
8ffc1ccda3 chore(release): bump version to 0.2.0.5 2026-01-06 15:12:38 +08:00
jinli.yl
b3910319ce chore(deps): update flowllm dependencies to version 0.2.0.9 2026-01-06 15:11:59 +08:00
jinli.yl
245e2564e4 refactor(core): restructure tool modules and add memory operations 2026-01-06 14:31:47 +08:00
jinli.yl
ed33749cf6 feat(memory): add memory node schema and utilities 2026-01-05 22:39:10 +08:00
jinli.yl
f270e2a099 feat(core): add agent and tool modules with search and execution capabilities 2026-01-05 20:14:44 +08:00
jinliyl
67f39db57a
Merge pull request #63 from agentscope-ai/merge_fl
add flowllm v2
2026-01-05 17:17:46 +08:00
jinli.yl
3c8eca8a3b feat(core): add application lifecycle management and streaming flow execution 2026-01-05 17:15:13 +08:00
jinli.yl
36e88b26dd style(core): format code according to team style guide 2026-01-05 14:41:22 +08:00