VSCode E2E testing framework using Mocha and VSCode Test - Mocha TDD framework for test structure - VSCode Test framework for extension testing - Custom test utilities and helpers - Event-driven testing patterns - Workspace-based test execution apps/vscode-e2e/src/suite/ apps/vscode-e2e/src/utils/ apps/vscode-e2e/src/runTest.ts apps/vscode-e2e/package.json packages/types/ apps/vscode-e2e npm run test:run TEST_FILE="filename.test" npm run test:run cd apps/vscode-e2e && TEST_FILE="apply-diff.test" npm run test:run npm run - Never use npm test directly as it doesn't exist - Always use the correct working directory - Use TEST_FILE environment variable for specific tests - Check available scripts with npm run if unsure Global api object for extension interactions - api.startTask(prompt: string): Start a new task - api.cancelCurrentTask(): Cancel the current task - api.clearCurrentTask(): Clear the current task - api.abortTask(): Abort the current task - api.getTaskStatus(): Get current task status - api.onDidReceiveMessage(callback): Listen to messages - api.onTaskCompleted(callback): Listen to task completion - api.onTaskAborted(callback): Listen to task abortion - api.onTaskStarted(callback): Listen to task start - api.onDidStartTerminalShellExecution(callback): Terminal start events - api.onDidEndTerminalShellExecution(callback): Terminal end events - api.updateSettings(settings): Update extension settings - api.getSettings(): Get current settings Wait for a condition to be true await waitFor(() => condition, timeout) await waitFor(() => fs.existsSync(filePath), 5000) Wait until current task is completed await waitUntilCompleted() Default timeout for task completion Wait until current task is aborted await waitUntilAborted() Default timeout for task abortion Helper to find files in multiple possible locations Use when files might be created in different workspace directories Utility to collect and analyze events during test execution Use for comprehensive event tracking and validation Custom assertion functions for common test patterns Use for consistent validation across tests Test workspaces are created by runTest.ts /tmp/roo-test-workspace-* vscode.workspace.workspaceFolders![0].uri.fsPath Create all test files in suiteSetup() before any tests run Always create files in the VSCode workspace directory Verify files exist after creation to catch setup issues early Clean up all test files in suiteTeardown() to avoid test pollution Store file paths in a test-scoped object for easy reference The AI will not see the files in the workspace directory Tell the AI to assume files exist and proceed as if they do Always verify outcomes rather than relying on AI file visibility Understanding message types for proper event handling Check packages/types/src/message.ts for valid message types say api_req_started Indicates tool execution started JSON with tool name and execution details Most reliable way to verify tool execution Contains tool execution results Tool results appear here, not in "tool_result" type General AI conversation messages Format may vary, don't rely on parsing these for tool detection Settings to enable automatic approval of AI actions Enable for file creation/modification tests Enable for command execution tests Enable for browser-related tests ```typescript await api.updateSettings({ alwaysAllowWrite: true, alwaysAllowExecute: true }); ``` Without proper auto-approval settings, the AI won't be able to perform actions without user approval Use console.log for tracking test execution flow - Log test phase transitions - Log important events and data - Log file paths and workspace state - Log expected vs actual outcomes Helper functions to validate test state at critical points - Workspace file listing - Current working directory - Task status - Event counts Tools for analyzing test failures - Stack trace analysis - Event timeline reconstruction - File system state comparison - Message flow analysis Appropriate timeout values for different operations Use generous timeouts for task completion (30+ seconds) Shorter timeouts for file system operations (5-10 seconds) Medium timeouts for event waiting (10-15 seconds) Proper cleanup to avoid resource leaks Always clean up event listeners after tests Cancel or clear tasks in teardown Remove test files to avoid disk space issues