jinliyl 9a6cf2b994 Dev/token (#159) ... * update * refactor(memory): remove unnecessary type check and update error logging * refactor(core): standardize logger import and update agentscope dependency * fix(memory): disable console output and add logging for summarizer component * feat(core): replace OpenAI token counter with custom ReMe token counter - Replace OpenAITokenCounter with ReMeTokenCounter implementation - Add support for HuggingFace mirror and configurable tokenizer - Register ReMeTokenCounter as default token counter in registry - Update config to use hf backend with Qwen2.5-7B-Instruct model refactor(memory): convert token counting methods to async in message handlers - Change count_str_token, stat_message, count_msgs_token to async methods - Update format_msgs_to_str and context_check to use async token counting - Modify _format_tool_result_output to support async token counting - Adjust all dependent methods to await async token counting calls feat(memory): add dialog persistence to in-memory storage - Implement _append_messages_to_dialog for saving messages to JSONL files - Add dialog_path parameter to ReMeInMemoryMemory constructor - Persist messages to daily JSONL files based on timestamp grouping - Update mark_messages_compressed to save and remove compressed messages - Modify clear_content to persist all messages before clearing memory refactor(ops): update token counter type hints and initialization - Change BaseOp to use HuggingFaceTokenCounter instead of TokenCounterBase - Update type annotations for as_token_counter property and parameters - Remove direct token counter injection from Compactor and ContextChecker - Pass as_token_counter parameter through service context mechanism style(logging): improve error logging with exception details - Replace logger.error with logger.exception in browser control tool - Change logger.error to logger.exception in memory get tool error handling - Add proper exception logging with stack trace information chore(config): add token counter configuration to light YAML - Add as_token_counters section with default hf backend configuration - Configure Qwen/Qwen2.5-7B-Instruct model with mirror support enabled - Set up pretrained_model_name_or_path and use_mirror parameters test(context): update context check tests to async implementation - Convert verify_context_check_invariants to async function - Update context check test methods to use async calls - Change stat_message calls to await async implementation - Modify test_empty_messages and test_below_threshold_returns_all to async * feat(core): implement context checking and memory management features * refactor(core): replace direct loguru import with logger utility function * refactor(reme): remove RuntimeContext dependency and simplify context checking * feat(docs): add raw conversation persistence to ReMe framework		2026-03-17 11:07:31 +08:00
..
light	Dev/token (#159)	2026-03-17 11:07:31 +08:00
vector	feat(memory): add ContextChecker component for context size management (#144)	2026-03-06 23:43:42 +08:00
demo_memory_search.py	refactor(core): replace memory stores with file stores and update architecture	2026-02-26 14:18:12 +08:00
mcp_servers_demo.json	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_agentscope_converter.py	feat(core): add memory storage and context checking capabilities	2026-02-08 19:24:53 +08:00
test_base_context.py	refactor(memory): rename fs components to fb and reorganize modules	2026-02-26 16:47:04 +08:00
test_base_file_watcher.py	feat(file-watcher): add configurable retry for file watcher (#140)	2026-03-05 14:34:00 +08:00
test_cache_handler.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_cache_memory_usage.py	feat(memory): implement file watcher with delta and full sync strategies	2026-02-06 01:58:28 +08:00
test_chunking_utils.py	refactor(benchmark): remove unused llm config and update test parameters	2026-02-26 18:20:12 +08:00
test_embedding.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_embedding_cache.py	feat(memory_store): add pure-python local memory store implementation	2026-02-19 23:58:54 +08:00
test_embedding_sync.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_execute_utils.py	feat(core): add async code execution and improve execution utilities	2026-02-14 21:27:04 +08:00
test_file_store.py	refactor(core): replace memory stores with file stores and update architecture	2026-02-26 14:18:12 +08:00
test_fs_tool.py	refactor(core): restructure application initialization and service context	2026-02-26 17:56:59 +08:00
test_horse.py	refactor(core): restructure application initialization and component management	2026-02-24 19:45:47 +08:00
test_keyword_search_performance.py	refactor(memory): integrate procedural memory into extension module	2026-02-27 14:55:00 +08:00
test_llm.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_llm_sync.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_logo.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_mcp_client.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_mcp_server.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_memory_vector_conversion.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_message.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_reme.py	refactor(core): update registry naming and application configuration	2026-02-07 15:05:02 +08:00
test_timer.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_token_counter.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_tool.py	refactor(core): restructure application initialization and service context	2026-02-26 17:56:59 +08:00
test_tool_call.py	refactor(core): restructure project modules and update base classes	2026-01-22 22:07:32 +08:00
test_ts_merge.py	feat(memory): add ReadHistoryV2 tool with enhanced history retrieval capabilities	2026-02-02 17:14:16 +08:00
test_vector_store.py	refactor(core): replace memory stores with file stores and update architecture	2026-02-26 14:18:12 +08:00