diff --git a/experiencemaker/module/context_generator/step_context_generator_prompt.yaml b/experiencemaker/module/context_generator/step_context_generator_prompt.yaml new file mode 100644 index 00000000..6c7bee33 --- /dev/null +++ b/experiencemaker/module/context_generator/step_context_generator_prompt.yaml @@ -0,0 +1,110 @@ +context_rewrite_prompt: | + You are an expert AI assistant tasked with rewriting and reorganizing context content to make it more relevant and actionable for the current task. + + Your task is to take the original context (containing multiple experiences) and rewrite it as a cohesive, task-specific guidance that directly addresses the current situation. + + REWRITING GUIDELINES: + ● TASK ALIGNMENT: Adapt the language and focus to match the current task requirements + ● CONSOLIDATION: Merge similar insights and eliminate redundancy + ● PRIORITIZATION: Emphasize the most relevant experiences for the current context + ● ACTIONABILITY: Make the guidance immediately actionable for the current situation + ● FLOW: Create a coherent narrative that builds from general principles to specific actions + + # Current Task/Query + {current_query} + + # Current Context/Situation + {current_context} + + # Original Context Content (Multiple Experiences) + {original_context} + + OUTPUT FORMAT: + Provide the rewritten context: + ```json + {{ + "rewritten_context": "A cohesive, task-specific context message that reorganizes and adapts the original experiences for the current task. This should be written as a unified guidance rather than separate experience items.", + "key_adaptations": [ + "How you adapted experience 1 for current task", + "How you consolidated experiences 2 and 3", + "What you emphasized for this specific context" + ], + "priority_focus": "The main focus area for this specific task based on the experiences" + }} + ``` + + Guidelines: + - Rewrite as a unified, flowing guidance rather than bullet points + - Adapt terminology and examples to match the current task domain + - Consolidate overlapping insights into coherent recommendations + - Prioritize experiences most relevant to the current situation + - Make the guidance feel custom-written for this specific task + +experience_rerank_prompt: | + You are an expert AI analyst tasked with reranking retrieved experiences based on their relevance to a specific query. + + Your task is to analyze the candidates and rank them by relevance, considering: + ● DIRECT RELEVANCE: How directly applicable the experience is to the current query + ● SITUATION SIMILARITY: How similar the experience context is to the current situation + ● ACTIONABILITY: How actionable and specific the experience is + ● QUALITY: The overall quality and clarity of the experience + + # Current Query + {query} + + # Candidate Experiences (Total: {num_candidates}) + {candidates} + + OUTPUT FORMAT: + Provide a ranked list of candidate indices (0-based) from most relevant to least relevant: + ```json + {{ + "ranked_indices": [2, 0, 4, 1, 3], + "reasoning": "Brief explanation of ranking rationale" + }} + ``` + + Note: Include ALL candidate indices in the ranking, even if some are less relevant. + +context_generation_prompt: | + You are an expert AI assistant tasked with synthesizing retrieved experiences into actionable context for an AI agent. + + Your task is to create a coherent, actionable context message that helps the agent leverage relevant past experiences. + + SYNTHESIS GUIDELINES: + ● RELEVANCE FOCUS: Emphasize the most relevant aspects of each experience + ● ACTIONABLE INSIGHTS: Extract specific, actionable guidance + ● COHERENT NARRATIVE: Create a flowing narrative rather than disconnected tips + ● SITUATIONAL AWARENESS: Adapt the guidance to the current situation + + # Current Query/Task + {query} + + # Current Step Context + {current_step} + + # Retrieved Experiences ({num_experiences} total) + {retrieved_experiences} + + OUTPUT FORMAT: + Create a synthesized context message: + ```json + {{ + "context": "A coherent, actionable context message that synthesizes the relevant experiences and provides specific guidance for the current task", + "key_insights": [ + "Key pattern from successful approaches", + "Common pitfall to avoid", + "Specific technique that worked well" + ], + "recommended_actions": [ + "Specific action recommendation based on experiences", + "Decision point with recommended choice" + ] + }} + ``` + + Guidelines: + - Make the context immediately actionable + - Prioritize the most relevant experiences + - Use clear, direct language + - Focus on practical guidance rather than abstract principles \ No newline at end of file diff --git a/experiencemaker/module/summarizer/step_summarizer_prompt.yaml b/experiencemaker/module/summarizer/step_summarizer_prompt.yaml new file mode 100644 index 00000000..7adf0264 --- /dev/null +++ b/experiencemaker/module/summarizer/step_summarizer_prompt.yaml @@ -0,0 +1,235 @@ +success_step_experience_prompt: | + You are an expert AI analyst reviewing successful step sequences from an AI agent execution. + + Your task is to extract reusable, actionable step-level experiences that can guide future agent executions. + Focus on identifying specific patterns, techniques, and decision points that contributed to success. + + ANALYSIS FRAMEWORK: + ● STEP PATTERN ANALYSIS: Identify the specific sequence of actions that led to success + ● DECISION POINTS: Highlight critical decisions made during these steps + ● TECHNIQUE EFFECTIVENESS: Analyze why specific approaches worked well + ● REUSABILITY: Extract patterns that can be applied to similar scenarios + + EXTRACTION PRINCIPLES: + ● Focus on TRANSFERABLE TECHNIQUES and decision frameworks + ● Frame insights as actionable guidelines and best practices + + # Original Query + {query} + + # Step Sequence Analysis + {step_sequence} + + # Context Information + {context} + + # Outcome + This step sequence was part of a {outcome} trajectory. + + OUTPUT FORMAT: + Generate 1-3 step-level success insights as JSON objects: + ```json + [ + {{ + "when_to_use": "Specific conditions when this step pattern should be applied", + "experience": "Detailed description of the successful step pattern and why it works", + "tags": ["relevant", "keywords", "for", "categorization"], + "confidence": 0.8, + "step_type": "reasoning|action|observation|decision", + "tools_used": ["list", "of", "tools"] + }} + ] + ``` + +failure_step_experience_prompt: | + You are an expert AI analyst reviewing failed step sequences from an AI agent execution. + + Your task is to extract learning experiences from failures to prevent similar mistakes in future executions. + Focus on identifying error patterns, missed opportunities, and alternative approaches. + + ANALYSIS FRAMEWORK: + ● FAILURE POINT IDENTIFICATION: Pinpoint where and why the steps went wrong + ● ERROR PATTERN ANALYSIS: Identify recurring mistakes or problematic approaches + ● ALTERNATIVE APPROACHES: Suggest what could have been done differently + ● PREVENTION STRATEGIES: Extract actionable insights to avoid similar failures + + EXTRACTION PRINCIPLES: + ● Extract GENERAL PRINCIPLES as well as SPECIFIC INSTRUCTIONS + ● Focus on PATTERNS and RULES as well as particular instances + + # Original Query + {query} + + # Step Sequence Analysis + {step_sequence} + + # Context Information + {context} + + # Outcome + This step sequence was part of a {outcome} trajectory. + + OUTPUT FORMAT: + Generate 1-3 step-level failure prevention insights as JSON objects: + ```json + [ + {{ + "when_to_use": "Specific situations where this lesson should be remembered", + "experience": "Universal principle or rule extracted from the failure pattern ", + "tags": ["error_prevention", "failure_analysis", "relevant_keywords"], + "confidence": 0.7, + "step_type": "reasoning|action|observation|decision", + "tools_used": ["list", "of", "tools"] + }} + ] + ``` + +comparative_step_experience_prompt: | + You are an expert AI analyst comparing successful and failed step sequences to extract differential insights. + + Your task is to identify the key differences between success and failure patterns at the step level. + Focus on critical decision points, technique variations, and approach differences. + + COMPARATIVE ANALYSIS FRAMEWORK: + ● DECISION CONTRAST: Compare critical decisions made in success vs failure cases + ● TECHNIQUE VARIATIONS: Identify different approaches and their outcomes + ● TIMING DIFFERENCES: Analyze when certain actions were taken and their impact + ● SUCCESS FACTORS: Extract what specifically made the difference + + EXTRACTION PRINCIPLES: + ● Frame comparisons as PRINCIPLES as well as case-specific SOLUTIONS + ● Identify PATTERNS that differentiate effective vs ineffective approaches + ● Extract RULES that can guide future similar situations + ● Focus on UNDERLYING MECHANISMS rather than surface-level differences + + # Successful Step Sequence + {success_steps} + + # Failed Step Sequence + {failure_steps} + + # Similarity Score: {similarity_score} + + OUTPUT FORMAT: + Generate 1-2 comparative insights as JSON objects: + ```json + [ + {{ + "when_to_use": "Specific scenarios where this comparative insight applies", + "experience": "Detailed comparison highlighting why success approach works better", + "tags": ["comparative_analysis", "success_factors", "relevant_keywords"], + "confidence": 0.8, + "step_type": "reasoning|action|observation|decision" + }} + ] + ``` + +general_step_experience_prompt: | + You are an expert AI analyst reviewing step sequences to extract general patterns and insights. + + Your task is to identify valuable step-level patterns without explicit success/failure labels. + Focus on effective techniques, common patterns, and general best practices. + + ANALYSIS FRAMEWORK: + ● PATTERN RECOGNITION: Identify recurring effective patterns in the steps + ● TECHNIQUE ANALYSIS: Analyze the effectiveness of different approaches + ● BEST PRACTICES: Extract general principles that appear beneficial + ● APPLICABILITY: Determine when these patterns would be most useful + + GENERALIZATION PRINCIPLES: + ● Extract UNIVERSAL PATTERNS that transcend specific contexts + ● Identify TRANSFERABLE METHODOLOGIES and approaches + ● Focus on PRINCIPLE-LEVEL insights as well as tactical details + ● Formulate insights as REUSABLE FRAMEWORKS and guidelines + + GENERALIZATION PRINCIPLES: + ● Extract UNIVERSAL PATTERNS that transcend specific contexts + ● Identify TRANSFERABLE METHODOLOGIES and approaches + ● Focus on PRINCIPLE-LEVEL insights + ● Formulate insights as REUSABLE FRAMEWORKS and guidelines + + # Original Query + {query} + + # Step Sequence Analysis + {step_sequence} + + # Context Information + {context} + + OUTPUT FORMAT: + Generate 1-2 general step insights as JSON objects: + ```json + [ + {{ + "when_to_use": "General conditions where this pattern is applicable", + "experience": "Detailed description of the effective step pattern", + "tags": ["general_pattern", "best_practice", "relevant_keywords"], + "confidence": 0.6, + "step_type": "reasoning|action|observation|decision", + "tools_used": ["list", "of", "tools"] + }} + ] + ``` + +step_segmentation_prompt: | + You are an expert AI analyst tasked with segmenting a trajectory into meaningful step sequences. + + Your task is to identify natural breakpoints in the execution where one logical unit of work ends and another begins. + Consider factors like: task completion, context switches, tool changes, reasoning phases, and logical groupings. + + SEGMENTATION CRITERIA: + ● LOGICAL COMPLETION: Steps that complete a specific sub-task or reasoning phase + ● CONTEXT SWITCHES: Points where the agent shifts focus or approach + ● TOOL BOUNDARIES: Natural breaks around tool usage patterns + ● REASONING PHASES: Distinct phases of analysis, planning, or execution + + # Original Query + {query} + + # Full Trajectory (Total steps: {total_steps}) + {trajectory_content} + + OUTPUT FORMAT: + Provide segmentation points as a JSON array of step indices where splits should occur: + ```json + {{ + "segment_points": [3, 7, 12, 18], + "reasoning": "Brief explanation of segmentation logic" + }} + ``` + + Note: Segment points indicate the END of each segment. For example, [3, 7] means: + - Segment 1: steps 0-3 + - Segment 2: steps 4-7 + - Segment 3: steps 8-end + +experience_validation_prompt: | + You are an expert AI analyst tasked with validating the quality and usefulness of extracted step-level experiences. + + Your task is to assess whether the extracted experience is actionable, accurate, and valuable for future agent executions. + + VALIDATION CRITERIA: + ● ACTIONABILITY: Is the experience specific enough to guide future actions? + ● ACCURACY: Does the experience correctly reflect the patterns observed? + ● RELEVANCE: Is the experience applicable to similar future scenarios? + ● CLARITY: Is the experience clearly articulated and understandable? + ● UNIQUENESS: Does the experience provide novel insights or common knowledge? + + # Experience to Validate + Condition: {condition} + Experience Content: {experience_content} + + OUTPUT FORMAT: + Provide validation assessment: + ```json + {{ + "is_valid": true/false, + "score": 0.8, + "feedback": "Detailed explanation of validation decision", + "recommendations": "Suggestions for improvement if applicable" + }} + ``` + + Score should be between 0.0 (poor quality) and 1.0 (excellent quality). + Mark as invalid if score is below 0.3 or if there are fundamental issues with the experience.