Roo-Code/.github/workflows
KJ7LNW d7ad9470e2
Tree-sitter Enhancements: TSX, TypeScript, JSON, and Markdown Support (#2169)
* feat: add Tree-sitter TSX query support

Added support for parsing TSX files with Tree-sitter:
- Created TSX query patterns for React components and functions
- Fixed field name issues by using direct node matching
- Added comprehensive documentation about TSX component structure
- Created a debug tool to inspect the actual tree structure
- Added test coverage for TSX parsing
- Embedded test fixture directly in the test file

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: combine TypeScript and TSX queries

Reduce duplication by:
- Import TypeScript queries from typescript.ts
- Keep only TSX-specific component and JSX queries
- Update documentation to reflect combined approach

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* test: update integration for parse definitions

- Adjusted logParseResult to call the actual parse definitions function using WASM from initializeWorkingParser.
- Patched TreeSitter initialization to resolve the WASM path correctly and bypass redundant init() calls.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* test: should successfully call parseSourceCodeDefinitionsForFile

Mock loadRequiredLanguageParsers to use real parser instance from initializeTreeSitter,
ensuring proper interaction between parseSourceCodeDefinitionsForFile and its dependencies.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* test: improve test structure for parseSourceCodeDefinitions

- Combine component parsing tests into a single test case
- Update test assertions to match actual parser output
- Fix interface and component definition tests to use VSCodeCheckbox as sample

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: improve React component detection in tree-sitter

- Add tests for complex TSX structures (nested components, HOCs)
- Enhance TSX queries to better detect React components
- Document current limitations in test file

* feat: improve React component detection in TSX files

Make TSX/React component detection more generic and robust by:
- Implementing structural pattern matching instead of specific React wrapper functions
- Adding configurable line threshold for React component inclusion (MIN_COMPONENT_LINES)
- Adding robust HTML element filtering with regex patterns
- Improving React component name handling for nested components
- Ensuring proper context handling for multi-line React components

Also update mock captures in tree-sitter tests to span at least 4 lines
to meet the MIN_COMPONENT_LINES threshold.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: use testParseSourceCodeDefinitions in logParseResult

- Moved testParseSourceCodeDefinitions function to top level
- Updated logParseResult to use testParseSourceCodeDefinitions
- Removed duplicate function from describe block
- Added console.log for debugging output

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: improve React component detection output format

- Remove individual line output for components that don't meet MIN_COMPONENT_LINES threshold
- Update tests to match the new behavior where lines < MIN_COMPONENT_LINES are skipped
- Maintain range output (e.g., 1--4) for component definitions
- Preserve context for larger definitions as ranges only

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: add switch/case statement support to tree-sitter TypeScript parser

- Added node patterns in typescript.ts query to capture switch statements, case clauses, and default clauses
- Modified index.ts to avoid duplicate line ranges in the output
- Added test for switch/case statements with complex case blocks
- Fixed line range tracking to prevent duplicate output

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: add support for enum declarations in tree-sitter TypeScript parser

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: add support for namespace declarations in tree-sitter TypeScript parser

Added query pattern for namespace declarations (internal_module nodes) in the TypeScript parser.
This allows the parser to identify and extract namespace declarations from TypeScript code.

- Added test case to verify namespace parsing functionality
- Added query pattern to capture namespace declarations in typescript.ts

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: add decorator pattern support to tree-sitter TypeScript parser

- Added support for parsing complex decorators with arguments
- Added test case for Component decorator pattern
- Enhanced TypeScript queries to capture decorator definitions

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: add generic type declaration support to tree-sitter TypeScript parser

- Added support for parsing generic types with constraints
- Added test case for Dictionary<K extends string | number, V> pattern
- Enhanced TypeScript queries to capture generic type definitions

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* test: add conditional type support to tree-sitter TypeScript parser

- Added test case for conditional type patterns like ReturnType<T>
- Verified that conditional types with infer keyword are already supported
- Enhanced inspectTreeStructure with detailed node inspection for debugging

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* test: add template literal type support to tree-sitter TypeScript parser

- Added test case for template literal type patterns like EventName<T>
- Verified that template literal types are already partially supported
- Confirmed support for complex template literal patterns in conditional types

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: implement tree-sitter compatible markdown processor

Adds a special case implementation for markdown files that:
- Parses markdown headers and section line ranges
- Returns captures in a format compatible with tree-sitter
- Integrates with the existing parseFile function
- Includes comprehensive tests for the implementation

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* fix: markdown parser not detecting sections with horizontal rules

The markdownParser was incorrectly interpreting horizontal rules (---) as setext headers when they appeared after non-header text. This caused some sections to be missed in the output.

This fix:
- Makes setext header detection more strict by requiring at least 3 = or - characters
- Adds validation for the text line before a potential setext header
- Ensures horizontal rules are not confused with setext headers

Added a test case to verify the fix works correctly with horizontal rules.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* refactor: move helper functions to dedicated file

Move test helper functions from parseSourceCodeDefinitions.test.ts to a new helpers.ts file.
Rename test file to parseSourceCodeDefinitions.tsx.test.ts to indicate it's for TSX tests.
This improves code organization by separating test helpers from test cases.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* feat: enable JSON structure display in list_code_definitions

This change allows list_code_definitions to show JSON structures by:
- Moving JSON query patterns into the JavaScript query file
- Using the JavaScript parser for JSON files
- Removing the separate JSON parser implementation
- Adding comprehensive tests for JSON parsing

Example output for a JSON file:
# test.json
0--90 | {
1--9 |   "server": {
4--8 |     "ssl": {
10--45 |   "database": {
11--24 |     "primary": {
14--18 |       "credentials": {
19--23 |       "pool": {
25--44 |     "replicas": [
26--43 |       {
30--42 |         "status": {
33--41 |           "metrics": {
36--40 |             "connections": {
46--73 |   "features": {
47--72 |     "auth": {
48--71 |       "providers": {
49--53 |         "local": {
54--70 |         "oauth": {
56--69 |           "providers": [
57--68 |             {

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* fix: Add compile step to CI workflow to ensure WASM files are available

The TreeSitter tests were failing in CI because the WASM files weren't being copied to the dist directory before running the tests. This adds an explicit compile step to ensure the WASM files are properly built and copied.

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

* fix: improve tree-sitter test type safety and debug logging

- Replace 'any' types with proper Parser types in helpers.ts
- Add centralized DEBUG flag and debugLog function in helpers.ts
- Update all console.log statements to use debugLog across all test files
- This change appeases @ellipsis-dev by improving type safety and
  providing a clean way to control debug logging

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

---------

Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
2025-03-31 23:34:06 -05:00
..
changeset-release.yml Require node v20 everywhere (#922) 2025-03-19 10:46:40 -07:00
code-qa.yml Tree-sitter Enhancements: TSX, TypeScript, JSON, and Markdown Support (#2169) 2025-03-31 23:34:06 -05:00
codeql.yml feat: add codeql.yml (#59) 2024-12-10 07:25:20 -08:00
discord-pr-notify.yml Different approach to escaping 2025-02-02 15:12:25 -05:00
marketplace-publish.yml ci: fix changelog extraction (#1810) 2025-03-20 07:27:41 -04:00
update-contributors.yml Add all changed files in update-contributors 2025-03-17 13:21:07 -04:00