- Writing a test file with a single write_to_file tool call instead of implementing in parts
- Not using proper Mocha TDD structure with suite() and test() blocks
- Forgetting to implement suiteSetup() and suiteTeardown() for proper cleanup
- Creating tests that depend on each other or specific execution order
- Not cleaning up tasks and files after test completion
- Using describe/it blocks instead of the required suite/test blocks
- Not using the global api object for extension interactions
- Forgetting to set auto-approval settings (alwaysAllowWrite, alwaysAllowExecute) when testing functionality that requires user approval
- Not implementing proper async/await patterns with waitFor utilities
- Using incorrect timeout values that are too short for actual task execution
- Not properly cleaning up tasks with cancelCurrentTask or clearCurrentTask
- Assuming the AI will use specific tools instead of testing outcomes
- Assuming files will be created in the expected location without checking multiple paths
- Not accounting for the workspace directory being created by runTest.ts
- Creating test files in temporary directories instead of the VSCode workspace directory
- Not verifying files exist after creation during setup
- Forgetting that the AI may not see files in the workspace directory
- Not using flexible file location checking that searches workspace directories
- Relying on parsing AI messages to detect tool usage instead of using proper event listeners
- Expecting tool results in "tool_result" message type (which doesn't exist)
- Not listening to terminal shell execution events for command tracking
- Depending on specific message formats that may vary
- Not implementing proper event cleanup after tests
- Parsing complex AI conversation messages instead of focusing on outcomes
- Using npm test instead of npm run test:run
- Not using the correct working directory (apps/vscode-e2e)
- Running tests from the wrong directory
- Not checking available scripts with npm run when unsure
- Forgetting to use TEST_FILE environment variable for specific tests
- Not running tests incrementally during development
- Not adding sufficient logging to track test execution flow
- Not logging important events like task IDs, file paths, and AI responses
- Not using codebase_search to find similar test patterns before writing new tests
- Not checking test output carefully for error messages and stack traces
- Not validating test state at critical points
- Assuming test failures are due to code issues without checking test logic
- Using complex instructions that may confuse the AI
- Expecting the AI to use exact tools specified in prompts
- Not allowing for variations in how the AI accomplishes tasks
- Testing implementation details instead of outcomes
- Not adapting tests based on actual AI behavior
- Forgetting to tell the AI to assume files exist in the workspace directory
- Adding unnecessary waits for specific tool executions
- Using complex message parsing logic that depends on AI behavior
- Not using the simplest possible test structure
- Depending on specific AI message formats
- Not using terminal events for reliable command execution verification
- Making tests too brittle by depending on exact AI responses
- Not understanding that files may be created in /tmp/roo-test-workspace-* directories
- Assuming the AI can see files in the workspace directory
- Not checking multiple possible file locations when verifying creation
- Creating files outside the VSCode workspace during tests
- Not properly setting up the test workspace in suiteSetup()
- Forgetting to clean up workspace files in suiteTeardown()
- Expecting specific message types for tool execution results
- Not understanding that ClineMessage types have specific values
- Trying to parse tool execution from AI conversation messages
- Not checking packages/types/src/message.ts for valid message types
- Depending on message parsing instead of outcome verification
- Not using api_req_started messages to verify tool execution
- Using timeouts that are too short for actual task execution
- Not accounting for AI processing time in test timeouts
- Waiting for specific tool executions instead of task completion
- Not implementing proper retry logic for flaky operations
- Using fixed delays instead of condition-based waiting
- Not considering that some operations may take longer in CI environments
- Not creating test files in the correct workspace directory
- Using hardcoded paths that don't work across different environments
- Not storing file paths in test-scoped objects for easy reference
- Creating test data that conflicts with other tests
- Not cleaning up test data properly after tests complete
- Using test data that's too complex for the AI to handle reliably