ReMe/docs4/old/reme4_resource.md
jinliyl c3fb825af0
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
Tests ReMe / Unit Tests - py3.11 (push) Has been cancelled
Tests ReMe / Unit Tests - py3.12 (push) Has been cancelled
Tests ReMe / Unit Tests - py3.13 (push) Has been cancelled
feat(agent): refactor agent wrapper, add session persistence, auto_resource step, and watch-loop improvements (#277)
* refactor(agent_wrapper): update agent wrapper implementations and config defaults

- Set default timezone to Asia/Shanghai in application config
- Add AgentScope imports and configure ReAct, context, and model configs
- Simplify __all__ export formatting in agent wrapper init
- Remove redundant docstring details from agent wrapper classes
- Optimize tool result handling with state assignment simplification
- Add permission context and state management for AgentScope backend
- Update Claude Code wrapper tool creation and server registration logic
- Configure default agent settings including permission mode and retry limits
- Remove obsolete comments and streamline code structure

* fix(agent): add output schema validation and BaseModel support

- Added type assertion to ensure output_schema is a dict in as_agent_wrapper
- Imported BaseModel from pydantic in base_agent_wrapper
- Modified set_output_schema to accept both dict and BaseModel types
- Added automatic conversion of BaseModel to JSON schema
- Updated method documentation to reflect new type support

* refactor(agent): replace direct agent instantiation with agent wrapper component

- Removed manual Agent creation and initialization in llm_demo step
- Integrated agent_wrapper component as dependency in base step
- Updated llm_demo step to use agent_wrapper.reply method instead of direct agent calls
- Modified structured output handling to work with new agent wrapper interface
- Simplified agent configuration by using wrapper's built-in functionality
- Updated documentation to reflect agent wrapper usage instead of direct as_llm access
- Removed redundant imports related to manual agent management

* feat(agent): add streaming support and refactor agent wrapper components

- Introduce reply_stream method in base agent wrapper with fallback implementation
- Add _build_agent helper method to AsAgentWrapper for agent instantiation
- Implement structured output generation with proper model assertions
- Update StreamLLMDemoStep to use agent_wrapper instead of direct Agent calls
- Replace manual streaming logic with execute_stream_task utility function
- Change default system prompt to provide detailed responses instead of concise ones
- Add colored output support for different chunk types in streaming demos
- Refactor test cases to use async task execution with streaming verification

* refactor(agent): remove session_id parameter from reply methods

- Removed session_id parameter from ASAgentWrapper.reply method signature
- Removed session_id parameter from BaseAgentWrapper.reply abstract method
- Removed session_id parameter from CCAgentWrapper.reply method signature
- Updated reply_stream methods to remove session_id parameter across all wrappers
- Modified CCAgentWrapper to use dynamic options assignment instead of hardcoded properties
- Set default system_prompt in config instead of hardcoded in code
- Increased default max_turns from 10 to 50 in configuration

* config: update default configuration and script entry point

- Change resource_dir from empty string to 'resource'
- Update command line entry point from 'reme4' to 'reme'

* feat(agent): add session state persistence and forking support

- Implement AsStateHandler for AgentState JSONL serialization
- Add session_id parameter to AsAgentWrapper.reply method
- Create timestamp-based session file paths with timezone support
- Load existing session state from JSONL files when session_id provided
- Save updated session state after each agent interaction
- Support session forking with UUID generation for new sessions
- Add integration tests for session persistence and forking scenarios
- Include temporary directory utilities for testing isolated sessions
- Ensure parent directories are created for session files automatically

* refactor(auto_memory): replace transcript parsing with direct message handling

- Remove transcript loading logic and related dependencies
- Add session message saving functionality with deduplication
- Use agent wrapper instead of direct AgentScope agent instantiation
- Simplify timezone handling using shared now utility
- Update logging and response metadata structure
- Remove unused imports and toolkit management methods
- Change session file naming from session_{id}.jsonl to session_agent_{id}.jsonl

* refactor(steps): move channel steps from index to channel module

- Move ChannelNotifyStep from .index.channel_notify to .channel.channel_notify
- Move ClaimChannelStep from .index.claim_channel to .channel.claim_channel
- Update __init__.py imports to reflect new module structure
- Reorganize steps list in __init__.py with channel section before index
- Add proper file prefix handling in daily index processing
- Update test imports to use new channel module location

* feat(evolve): add auto_resource step for interpreting resource files

- Add AutoResourceStep to interpret resource files into daily notes via an agent
- Implement resource file parsing with date and filename extraction logic
- Add session ID computation using MD5 hash of filename
- Create delete and upsert handlers for resource file operations
- Add truncation and sanitization functions for tool output in auto_memory
- Register auto_resource step with proper parameter validation
- Add configuration for resource watch loop with file extension filters
- Update default YAML config to include resource watch and digest watch loops
- Add shared watch-rule logic for scan_changes and watch_changes steps
- Implement foreach_dispatch and log_changes steps for change processing
- Rename update_store_index_loop to index_update_loop in configuration
- Refactor file chunking interface from parse to chunk method
- Remove unused imports and dependencies in auto_dream step
- Fix path iteration formatting in daily_index utility function
- Add comprehensive integration tests for auto_resource functionality

* refactor(auto_resource): format function call with multi-line parameters

- Reformatted await _handle_upsert call to use multiple lines for better readability
- Removed unused imports from scan_changes.py including BaseFileCatalog and ComponentEnum
- Added date parameter to RuntimeContext initialization in test cases
- Updated expected file paths in test assertions to include session_agent prefix
- Formatted long assertion statements across multiple lines to maintain character limit
- Corrected wikilink references from generic names to session_agent prefixed names
2026-06-08 16:11:32 +08:00

1.8 KiB
Raw Permalink Blame History

  • 增加agent 的component
  • 增加全局 时区time_zone,全局作用

数据结构

  • resource
    • YYYYMMDD
      • xxxx.html
      • xxxx.txt
      • xxxx.md
  • dialog
    • YYYYMMDD
    • session_{session_id}.jsonl msg->dict格式
  • daily
    • YYYYMMDD.md 索引
    • YYYYMMDD
      • session_{session_id}.md 日志本
      • resource_{resource_id}.md
  • digest
    • personal 个性化信息
      • xxx.md
    • procedural 程序化记忆
      • xxx.md
    • wiki 知识化记忆
      • xxx.md

监控任务【后台】:

  • 构建bm25+emb的index【5s】
    • 目录:daily+digest 的 所有md,使用markdown的ast解析
    • 可选:jsonl,要求保存的时候对工具结果截断,使用rag方案解析
  • daily md监控:【1min】
    • 暂无
  • digest md监控:【1min】
    • 暂无
  • resource 监控:【间隔5min】
    • 针对每一个文件,生成一个hashid,作为session_id
    • 生成一个agent,解读文件(读前1MB内容)防止上下文炸了
    • 把抽取的内容写到daily/YYYYMMDD/resource_{resource_id}.md
    • 调用推送工具:
      • qwenpaw推送收件箱/agent
      • cc可以直接推送agent @sen

hook任务:

  • auto-memory:在上下文满/每隔多少轮/session_end
    • qwenpaw:直接传message session_id date,直接append session_agent_{session_id}.jsonl @jinli
    • cc:给出新的path,对比我们保存的session_{session_id}.jsonl,看到增量msg,接着解析path的内容,保存到session_{session_id}.jsonl @sen
    • 注意保存的时候截断工具调用结构,防止太长

定时任务:

  • auto-dream:每天夜晚触发
    • 记忆整理:按照session/resource_id去做for循环整理:把YYYYMMDD.md + YYYYMMDD/* 消化更新到 digest下的personal/procedural/wiki
    • 记忆link:markdown之间link,digest内部链接,可以链接到外面。