Roo-Code/esbuild.js
KJ7LNW 8e85a121f1
feat: standardize tree-sitter test implementations (#2960)
* refactor: standardize C# Tree-Sitter parser tests

This commit standardizes the C# Tree-Sitter parser tests to ensure:
- Variable/function names in sample code match the tests
- Each test only verifies specific identifiers relevant to that test
- Test names clearly indicate what they're testing
- No skipped tests remain
- Only one test exists per structure or combination of structures
- All debug output uses debugLog from helpers.ts

The standardization improves maintainability and clarity while preserving
all existing functionality.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: standardize C Tree-Sitter parser tests

This commit standardizes the C Tree-Sitter parser tests to ensure:
- Variable/function names in sample code match the tests (using snake_case)
- Each test only verifies specific identifiers relevant to that test
- Test names clearly indicate what they're testing
- No skipped tests remain - tests for unsupported features are removed
- Only one test exists per structure or combination of structures
- All debug output uses debugLog from helpers.ts

The standardization improves maintainability and clarity while preserving
all existing functionality. Documentation has been added for C language
constructs that are not currently supported by the parser.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: standardize Kotlin Tree-Sitter parser

Standardized Kotlin Tree-Sitter parser to ensure consistent naming and structure:
- Renamed identifiers to clearly indicate what they're testing
- Ensured all code sections are at least 4 lines long
- Organized query patterns logically with clear comments
- Removed skipped tests and unused query patterns
- Ensured only one test per structure or combination of structures

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: standardize PHP Tree-Sitter parser

Standardized PHP Tree-Sitter parser to ensure consistent naming and structure:
- Renamed identifiers to clearly indicate what they're testing
- Ensured all code sections are at least 4 lines long
- Organized query patterns logically with clear comments
- Consolidated related tests to ensure only one test per structure
- Ensured all debug output uses debugLog from helpers.ts

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: standardize Ruby Tree-Sitter parser

Standardized Ruby Tree-Sitter parser to ensure consistent naming and structure:
- Expanded query to support many more Ruby language constructs
- Added comprehensive test file with clear section headers
- Ensured all code sections are at least 4 lines long
- Added tests for each language construct
- Used consistent naming conventions for test identifiers

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: standardize Swift Tree-Sitter parser

Standardized Swift Tree-Sitter parser to ensure consistent naming and structure:
- Renamed identifiers to clearly indicate what they're testing
- Ensured all code sections are at least 4 lines long
- Organized query patterns logically with clear comments
- Removed skipped tests and unused query patterns
- Ensured only one test per structure or combination of structures

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: dynamically include all tree-sitter WASM files in build process

- Modified esbuild.js to dynamically read and copy all WASM files from tree-sitter-wasms package
- Added error handling and logging for better debugging
- Ensures all language parsers are automatically included without requiring manual updates

* refactor: move test mocks and fixtures to shared locations

Move test mocks to helpers.ts to be shared between test files
Create fixtures directory for shared test data
Move sample TSX content to fixtures/sample-tsx.ts
Update test files to use shared mocks and fixtures

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: simplify tree-sitter debug output

- Remove detailed node inspection to reduce noise
- Keep only essential tree structure output
- Remove duplicate content logging

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: standardize C# tree-sitter test structure

Move C# sample code to dedicated fixture file and create inspectCSharp test
following the same pattern as other language tests. This improves test
organization and maintainability by:

- Extracting sample C# code to fixtures/sample-c-sharp.ts
- Adding inspectCSharp.test.ts for tree structure inspection
- Updating parseSourceCodeDefinitions.c-sharp.test.ts to use fixture

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: add language-specific capture processing

- Add language parameter to processCaptures for language-specific handling
- Implement selective HTML filtering for jsx/tsx files
- Update all call sites to pass correct language parameter
- Fix type safety by properly passing language strings

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: move tree-sitter test samples to fixtures

Move sample code from parseSourceCodeDefinitions test files into dedicated
fixtures directory for better organization and reuse:

- Created fixtures directory for language samples
- Moved sample code for C#, C, JSON, Kotlin, PHP, Ruby, Rust, and Swift
- Created inspect test files for each language
- Updated original test files to import from fixtures
- Standardized test options across languages
- No behavior changes, all tests passing

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: standardize tree-sitter test code across languages

Standardize test code structure and naming across all language parsers:
- Rename identifiers to clearly indicate test purpose
- Ensure each code section has descriptive comments
- Group related code sections together
- Maintain language-specific naming conventions
- Keep test structure consistent across languages
- Remove tests for unsupported features

Changes made to:
- C, JSON, Kotlin, PHP, Ruby, Rust, and Swift test files
- Sample code fixtures
- Parser definition tests
- Inspect structure tests

No functional changes - all tests passing with improved maintainability.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: standardize tree-sitter parser tests across all languages

- Ensure all test structures span at least 4 lines for better parsing
- Create exactly one consolidated test per data structure type
- Use descriptive names that clearly indicate test purpose
- Improve query pattern organization and documentation
- Simplify inspect test files to focus on structure validation
- Implement result caching in test files for better performance
- Remove duplicate and skipped tests
- Follow consistent naming conventions across all languages

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* first pass all but elm-bash

* test: inspectTreeStructure helper now returns a string

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: establish single source of truth for MIN_COMPONENT_LINES

- Move minComponentLines parameter to a file-level constant
- Remove parameter passing to processCaptures function
- Update all references to use the global constant
- Remove redundant local constant declaration in parseFile

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: improve MIN_COMPONENT_LINES implementation

- Changed MIN_COMPONENT_LINES constant to use getter/setter functions
- Updated references to use getMinComponentLines() function
- Modified test helper to set value to 0 during tests
- This establishes a single source of truth while making testing easier

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: standardize tree-sitter test implementations

Standardized tree-sitter test implementations across all supported languages:
- Removed debugLog calls from test files
- Implemented consistent line number pattern matching
- Ensured proper test structure and execution order
- Added comprehensive test coverage for supported structures
- Maintained 1:1 mapping between queries, tests and samples
- Documented unsupported features in TODO sections

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* docs: document tree-sitter parseable but unsupported structures

Added TODO sections in test files documenting structures that can be parsed
by tree-sitter but currently lack query pattern support:

- Python: f-strings, complex type annotations, pattern matching
- C++: virtual methods, field initializers, base class clauses
- Java: import declarations, field declarations with modifiers
- TSX: React hooks, context providers, event handlers

Added examples and clarifying comments to existing TODO sections.
Enhanced Java query patterns for better structure capture including
lambda expressions, field declarations, and type parameters.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* fix: use correct distDir variable for WASM file copying

The copyWasmFiles plugin was using an undefined targetDir variable when copying
tree-sitter WASM files. Changed to use the correctly defined distDir variable,
fixing the build process.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* lint: remove unused imports and variables from tree-sitter tests

Removed unused imports and variables from tree-sitter test files to fix linting errors:
- Removed unused goQuery import from helpers.ts
- Removed unused imports and mockedFs variables from language-specific test files
- Cleaned up test files to only import what they use

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

---------

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
2025-04-28 11:27:49 -04:00

206 lines
5.4 KiB
JavaScript

const esbuild = require("esbuild")
const fs = require("fs")
const path = require("path")
const production = process.argv.includes("--production")
const watch = process.argv.includes("--watch")
/**
* @type {import('esbuild').Plugin}
*/
const esbuildProblemMatcherPlugin = {
name: "esbuild-problem-matcher",
setup(build) {
build.onStart(() => {
console.log("[watch] build started")
})
build.onEnd((result) => {
result.errors.forEach(({ text, location }) => {
console.error(`✘ [ERROR] ${text}`)
console.error(` ${location.file}:${location.line}:${location.column}:`)
})
console.log("[watch] build finished")
})
},
}
const copyWasmFiles = {
name: "copy-wasm-files",
setup(build) {
build.onEnd(() => {
const nodeModulesDir = path.join(__dirname, "node_modules")
const distDir = path.join(__dirname, "dist")
// tiktoken
fs.copyFileSync(
path.join(nodeModulesDir, "tiktoken", "tiktoken_bg.wasm"),
path.join(distDir, "tiktoken_bg.wasm"),
)
// Copy language-specific WASM files
const languageWasmDir = path.join(__dirname, "node_modules", "tree-sitter-wasms", "out")
// Dynamically read all WASM files from the directory instead of using a hardcoded list
if (fs.existsSync(languageWasmDir)) {
const wasmFiles = fs.readdirSync(languageWasmDir).filter((file) => file.endsWith(".wasm"))
console.log(`Copying ${wasmFiles.length} tree-sitter WASM files to dist directory`)
wasmFiles.forEach((filename) => {
fs.copyFileSync(path.join(languageWasmDir, filename), path.join(distDir, filename))
})
} else {
console.warn(`Tree-sitter WASM directory not found: ${languageWasmDir}`)
}
})
},
}
// Simple function to copy locale files
function copyLocaleFiles() {
const srcDir = path.join(__dirname, "src", "i18n", "locales")
const destDir = path.join(__dirname, "dist", "i18n", "locales")
const outDir = path.join(__dirname, "out", "i18n", "locales")
// Ensure source directory exists before proceeding
if (!fs.existsSync(srcDir)) {
console.warn(`Source locales directory does not exist: ${srcDir}`)
return // Exit early if source directory doesn't exist
}
// Create destination directories
fs.mkdirSync(destDir, { recursive: true })
try {
fs.mkdirSync(outDir, { recursive: true })
} catch (e) {}
// Function to copy directory recursively
function copyDir(src, dest) {
const entries = fs.readdirSync(src, { withFileTypes: true })
for (const entry of entries) {
const srcPath = path.join(src, entry.name)
const destPath = path.join(dest, entry.name)
if (entry.isDirectory()) {
// Create directory and copy contents
fs.mkdirSync(destPath, { recursive: true })
copyDir(srcPath, destPath)
} else {
// Copy the file
fs.copyFileSync(srcPath, destPath)
}
}
}
// Copy files to dist directory
copyDir(srcDir, destDir)
console.log("Copied locale files to dist/i18n/locales")
// Copy to out directory for debugging
try {
copyDir(srcDir, outDir)
console.log("Copied locale files to out/i18n/locales")
} catch (e) {
console.warn("Could not copy to out directory:", e.message)
}
}
// Set up file watcher if in watch mode
function setupLocaleWatcher() {
if (!watch) return
const localesDir = path.join(__dirname, "src", "i18n", "locales")
// Ensure the locales directory exists before setting up watcher
if (!fs.existsSync(localesDir)) {
console.warn(`Cannot set up watcher: Source locales directory does not exist: ${localesDir}`)
return
}
console.log(`Setting up watcher for locale files in ${localesDir}`)
// Use a debounce mechanism
let debounceTimer = null
const debouncedCopy = () => {
if (debounceTimer) clearTimeout(debounceTimer)
debounceTimer = setTimeout(() => {
console.log("Locale files changed, copying...")
copyLocaleFiles()
}, 300) // Wait 300ms after last change before copying
}
// Watch the locales directory
try {
fs.watch(localesDir, { recursive: true }, (eventType, filename) => {
if (filename && filename.endsWith(".json")) {
console.log(`Locale file ${filename} changed, triggering copy...`)
debouncedCopy()
}
})
console.log("Watcher for locale files is set up")
} catch (error) {
console.error(`Error setting up watcher for ${localesDir}:`, error.message)
}
}
const copyLocalesFiles = {
name: "copy-locales-files",
setup(build) {
build.onEnd(() => {
copyLocaleFiles()
})
},
}
const extensionConfig = {
bundle: true,
minify: production,
sourcemap: !production,
logLevel: "silent",
plugins: [
copyWasmFiles,
copyLocalesFiles,
/* add to the end of plugins array */
esbuildProblemMatcherPlugin,
{
name: "alias-plugin",
setup(build) {
build.onResolve({ filter: /^pkce-challenge$/ }, (args) => {
return { path: require.resolve("pkce-challenge/dist/index.browser.js") }
})
},
},
],
entryPoints: ["src/extension.ts"],
format: "cjs",
sourcesContent: false,
platform: "node",
outfile: "dist/extension.js",
external: ["vscode"],
}
async function main() {
const extensionCtx = await esbuild.context(extensionConfig)
if (watch) {
// Start the esbuild watcher
await extensionCtx.watch()
// Copy and watch locale files
console.log("Copying locale files initially...")
copyLocaleFiles()
// Set up the watcher for locale files
setupLocaleWatcher()
} else {
await extensionCtx.rebuild()
await extensionCtx.dispose()
}
}
main().catch((e) => {
console.error(e)
process.exit(1)
})