594495de9e
- Add is_summarized boolean field to ToolCallResult schema - Implement logic to mark tool calls as summarized after processing - Skip summarization for tools with all recent calls already summarized - Update summary operation to process only tools with unsummarized calls - Enhance logging to show summarization and skip statistics - Modify response to include count of summarized vs skipped tools - Add documentation for smart skip logic and is_summarized feature - Include test scenarios for skip logic and incremental summarization
3.3 KiB
Tool Summary Operations
ParseToolCallResultOp
Purpose
Evaluates individual tool invocations and adds them to the tool memory database with comprehensive assessments.
Functionality
- Receives tool call results with input parameters, output, and metadata
- Uses LLM to evaluate each tool call based on success and parameter alignment
- Generates summary, evaluation, and score (0.0 or 1.0) for each call
- Appends evaluated results to existing tool memory or creates new memory
- Maintains a sliding window of recent tool calls (configurable limit)
Parameters
-
op.parse_tool_call_result_op.params.max_history_tool_call_cnt(integer, default:100):- Maximum number of historical tool call results to retain per tool
- When exceeded, oldest results are removed (FIFO)
-
op.parse_tool_call_result_op.params.evaluation_sleep_interval(float, default:1.0):- Delay in seconds between concurrent evaluations
- Prevents rate limiting when evaluating multiple calls
SummaryToolMemoryOp
Purpose
Analyzes accumulated tool call history and generates comprehensive usage patterns, best practices, and recommendations.
Functionality
- Retrieves existing tool memories from the vector store
- Intelligently skips tools where all recent calls have already been summarized (using
is_summarizedflag) - Analyzes the most recent N tool calls (configurable)
- Calculates statistical metrics (success rate, average scores, costs)
- Uses LLM to synthesize actionable usage guidelines
- Updates tool memory content with generated insights
- Marks processed calls as summarized to avoid redundant processing in future runs
Smart Skip Logic
To optimize costs and performance, SummaryToolMemoryOp tracks which tool call results have been included in a summary:
- Skip Condition: If all recent N calls are already summarized (
is_summarized=True), the tool is skipped entirely - Trigger Condition: If at least 1 recent call is new (
is_summarized=False), re-summarization is triggered - Automatic Marking: After successful summarization, all processed calls are marked with
is_summarized=True
Example Behavior:
Run 1: 30 new calls → Summarize all 30, mark as summarized
Run 2: Same 30 calls → Skip (all already summarized) ✓ Cost savings
Run 3: 30 old + 1 new → Re-summarize all 31, mark new call as summarized
This ensures summaries stay fresh while avoiding unnecessary LLM calls.
Parameters
-
op.summary_tool_memory_op.params.recent_call_count(integer, default:30):- Number of most recent tool calls to analyze
- Also determines the window for checking summarization status
- Focuses on recent usage patterns
-
op.summary_tool_memory_op.params.summary_sleep_interval(float, default:1.0):- Delay in seconds between concurrent summarizations
- Prevents rate limiting when summarizing multiple tools
Return Value
The operation returns a response message indicating:
- Number of tools summarized (had new unsummarized calls)
- Number of tools skipped (all recent calls already summarized)
Example: "Successfully processed 5 tool memories: 2 summarized, 3 skipped (already up-to-date)"
See Also
For detailed implementation and examples, see is_summarized Feature Documentation.