Same root cause as #587/#591 — Claude Code's runtime loader rejects array-form skills paths like ["./content-production", "./ai-seo", ...] even when each entry is a valid subdirectory containing SKILL.md. `claude plugin validate` accepts them but the loader does not. The proven canonical layout (used by self-improving-agent in #536): <plugin>/ ├── .claude-plugin/plugin.json skills: "./skills" └── skills/ ├── <skill-1>/SKILL.md ├── <skill-2>/SKILL.md └── ... Restructured 9 multi-skill domain plugins: - business-growth (4 skills moved) - c-level-advisor (28) - engineering (36) - engineering-team (32) - finance (2) - marketing-skill (43) - product-team (12) - project-management (8) - ra-qm-team (13) Also fixed standalone plugins that had root SKILL.md + ./skills/ subdir (agenthub, autoresearch-agent, executive-mentor, playwright-pro). The loader rejected them despite skills="./skills" because of the conflicting root SKILL.md (compare self-improving-agent which works because PR #536 moved its root SKILL.md). Moved each root SKILL.md into ./skills/<name>/. Restored standalone plugin folders to their original paths after the multi-skill restructure swept them into parent skills/ directories (marketplace.json source paths require original locations). Removed 7 orphaned marketplace entries that pointed to skill folders without their own plugin.json (content-creator, demand-gen, fullstack-engineer, aws-architect, product-manager, scrum-master, skill-security-auditor) — these were already non-functional. Bumped patch versions on every changed plugin and synced marketplace.json. Marketplace now lists 29 working plugins (down from 36). After merge: users run `/plugin marketplace update claude-code-skills` followed by `/plugin update --all` to pick up the working layout.
8 KiB
Technology Evaluation Workflows
Step-by-step workflows for common evaluation scenarios.
Table of Contents
- Framework Comparison Workflow
- TCO Analysis Workflow
- Migration Assessment Workflow
- Security Evaluation Workflow
- Cloud Provider Selection Workflow
Framework Comparison Workflow
Use this workflow when comparing frontend/backend frameworks or libraries.
Step 1: Define Requirements
-
Identify the use case:
- What type of application? (SaaS, e-commerce, real-time, etc.)
- What scale? (users, requests, data volume)
- What team size and skill level?
-
Set priorities (weights must sum to 100%):
- Performance: ____%
- Scalability: ____%
- Developer Experience: ____%
- Ecosystem: ____%
- Learning Curve: ____%
- Other: ____%
-
List constraints:
- Budget limitations
- Timeline requirements
- Compliance needs
- Existing infrastructure
Step 2: Run Comparison
python scripts/stack_comparator.py \
--technologies "React,Vue,Angular" \
--use-case "enterprise-saas" \
--weights "performance:20,ecosystem:25,scalability:20,developer_experience:35"
Step 3: Analyze Results
- Review weighted total scores
- Check confidence level (High/Medium/Low)
- Examine strengths and weaknesses for each option
- Review decision factors
Step 4: Validate Recommendation
- Match recommendation to your constraints
- Consider team skills and hiring market
- Evaluate ecosystem for your specific needs
- Check corporate backing and long-term viability
Step 5: Document Decision
Record:
- Final selection with rationale
- Trade-offs accepted
- Risks identified
- Mitigation strategies
TCO Analysis Workflow
Use this workflow for comprehensive cost analysis over multiple years.
Step 1: Gather Cost Data
Initial Costs:
- Licensing fees (if any)
- Training hours per developer
- Developer hourly rate
- Migration costs
- Setup and tooling costs
Operational Costs:
- Monthly hosting costs
- Annual support contracts
- Maintenance hours per developer per month
Scaling Parameters:
- Initial user count
- Expected annual growth rate
- Infrastructure scaling approach
Step 2: Run TCO Calculator
python scripts/tco_calculator.py \
--input assets/sample_input_tco.json \
--years 5 \
--output tco_report.json
Step 3: Analyze Cost Breakdown
- Review initial vs. operational costs ratio
- Examine year-over-year cost growth
- Check cost per user trends
- Identify scaling efficiency
Step 4: Identify Optimization Opportunities
Review:
- Can hosting costs be reduced with reserved pricing?
- Can automation reduce maintenance hours?
- Are there cheaper alternatives for specific components?
Step 5: Compare Multiple Options
Run TCO analysis for each technology option:
- Current state (baseline)
- Option A
- Option B
Compare:
- 5-year total cost
- Break-even point
- Risk-adjusted costs
Migration Assessment Workflow
Use this workflow when planning technology migrations.
Step 1: Document Current State
- Count lines of code
- List all components/modules
- Identify dependencies
- Document current architecture
- Note existing pain points
Step 2: Define Target State
- Target technology/framework
- Target architecture
- Expected benefits
- Success criteria
Step 3: Assess Team Readiness
- How many developers have target technology experience?
- What training is needed?
- What is the team's capacity during migration?
Step 4: Run Migration Analysis
python scripts/migration_analyzer.py \
--from "angular-1.x" \
--to "react" \
--codebase-size 50000 \
--components 200 \
--team-size 6
Step 5: Review Risk Assessment
For each risk category:
- Identify specific risks
- Assess probability and impact
- Define mitigation strategies
- Assign risk owners
Step 6: Plan Migration Phases
-
Phase 1: Foundation
- Setup new infrastructure
- Create migration utilities
- Train team
-
Phase 2: Incremental Migration
- Migrate by feature area
- Maintain parallel systems
- Continuous testing
-
Phase 3: Completion
- Remove legacy code
- Optimize performance
- Complete documentation
-
Phase 4: Stabilization
- Monitor production
- Address issues
- Gather metrics
Step 7: Define Rollback Plan
Document:
- Trigger conditions for rollback
- Rollback procedure
- Data recovery steps
- Communication plan
Security Evaluation Workflow
Use this workflow for security and compliance assessment.
Step 1: Identify Requirements
-
List applicable compliance standards:
- GDPR
- SOC2
- HIPAA
- PCI-DSS
- Other: _____
-
Define security priorities:
- Data encryption requirements
- Access control needs
- Audit logging requirements
- Incident response expectations
Step 2: Gather Security Data
For each technology:
- CVE count (last 12 months)
- CVE count (last 3 years)
- Severity distribution
- Average patch time
- Security features list
Step 3: Run Security Assessment
python scripts/security_assessor.py \
--technology "express-js" \
--compliance "soc2,gdpr" \
--output security_report.json
Step 4: Analyze Results
Review:
- Overall security score
- Vulnerability trends
- Patch responsiveness
- Compliance readiness per standard
Step 5: Identify Gaps
For each compliance standard:
- List missing requirements
- Estimate remediation effort
- Identify workarounds if available
- Calculate compliance cost
Step 6: Make Risk-Based Decision
Consider:
- Acceptable risk level
- Cost of remediation
- Alternative technologies
- Business impact of compliance gaps
Cloud Provider Selection Workflow
Use this workflow for AWS vs Azure vs GCP decisions.
Step 1: Define Workload Requirements
-
Workload type:
- Web application
- API services
- Data analytics
- Machine learning
- IoT
- Other: _____
-
Resource requirements:
- Compute: ____ instances, ____ cores, ____ GB RAM
- Storage: ____ TB, type (block/object/file)
- Database: ____ type, ____ size
- Network: ____ GB/month transfer
-
Special requirements:
- GPU/TPU for ML
- Edge computing
- Multi-region
- Specific compliance certifications
Step 2: Evaluate Feature Availability
For each provider, verify:
- Required services exist
- Service maturity level
- Regional availability
- SLA guarantees
Step 3: Run Cost Comparison
python scripts/tco_calculator.py \
--providers "aws,azure,gcp" \
--workload-config workload.json \
--years 3
Step 4: Assess Ecosystem Fit
Consider:
- Team's existing expertise
- Development tooling preferences
- CI/CD integration
- Monitoring and observability tools
Step 5: Evaluate Vendor Lock-in
For each provider:
- List proprietary services you'll use
- Estimate migration cost if switching
- Identify portable alternatives
- Calculate lock-in risk score
Step 6: Make Final Selection
Weight factors:
- Cost: ____%
- Features: ____%
- Team expertise: ____%
- Lock-in risk: ____%
- Support quality: ____%
Select provider with highest weighted score.
Best Practices
For All Evaluations
- Document assumptions - Make all assumptions explicit
- Validate data - Verify metrics from multiple sources
- Consider context - Generic scores may not apply to your situation
- Include stakeholders - Get input from team members who will use the technology
- Plan for change - Technology landscapes evolve; plan for flexibility
Common Pitfalls to Avoid
- Over-weighting recent popularity vs. long-term stability
- Ignoring team learning curve in timeline estimates
- Underestimating migration complexity
- Assuming vendor claims are accurate
- Not accounting for hidden costs (training, hiring, technical debt)