* fix: resolve Go duplicate references in tree-sitter queries (#5367)
- Replace broad statement captures with function-scoped queries
- Eliminates overlapping captures that caused duplicate references
- Improves search quality and indexing performance for Go projects
- Add test to validate no duplicate line ranges are captured
- Maintains backward compatibility with existing functionality
Fixes#5367
* fix: resolve Go codebase indexing with language-specific thresholds
- Update Go tree-sitter queries to capture full declarations instead of just identifiers
- Implement language-specific character thresholds (50 chars for Go vs 100 default)
- Fix inspectGo.spec.ts test to match new query behavior
- Add comprehensive test coverage for Go indexing fix
This ensures Go files are properly indexed for semantic search while preventing
duplicate references. All tests now pass.
* fix: implement universal 50-character threshold for code indexing
- Changed MIN_BLOCK_CHARS from 100 to 50 in parser.ts
- Updated tests to expect single-block captures for small Go files
- Removed language-specific threshold logic
- Fixes Go files not being indexed due to high character threshold
Fixes#5367
* test: replace Go-specific test with generic MIN_BLOCK_CHARS test
- Remove go-indexing-fix.spec.ts as requested in PR feedback
- Add generic test in parser.spec.ts to verify 50-character threshold
- Test ensures content under 50 chars is filtered, 50+ chars is indexed
- Applies to all languages, not just Go
* fix: update markdown tests for MIN_BLOCK_CHARS change from 100 to 50
* feat: add markdown support to codebase indexing (#4660)
* fix: implement chunking for large markdown sections and fix Qdrant deduplication issue (#4660)
- Modified parseMarkdownContent to chunk large sections (>1150 chars)
- Added support for chunking header-less markdown files
- Fixed _chunkTextByLines to handle oversized lines properly
- Added defensive check for parseMarkdown returning undefined
- Fixed Qdrant ID generation to use segmentHash instead of file:line
- This was the root cause: chunks were being deduplicated
- Each chunk now gets a unique ID even from the same line
- Added comprehensive tests for all edge cases
- Ensures all markdown content is properly indexed in Qdrant
* fix: remove redundant supported-extensions test file
As identified in PR review, the supported-extensions.spec.ts file only tests
the contents of an array, which is already implicitly covered by the functional
tests in parser.spec.ts. Removing this reduces maintenance overhead without
sacrificing test quality.
* test: address PR review feedback
- Remove redundant test 'should handle large markdown documentation folders efficiently'
that only verified the scanner could iterate over mocked files
- Add test to verify unique point IDs are generated for each block from the same file,
ensuring the segmentHash-based ID generation prevents collisions
* fix: add segmentHash to vector point payload and refactor tests
- Add segmentHash to payload in scanner.ts to fix vector point ID generation
- Split parser.spec.ts tests into focused unit tests (mocked dependencies)
- Move integration tests to new markdownIntegration.spec.ts file
- Each test suite now has clear, distinct responsibilities
- Fixes issue #4660: vector point ID collisions for large Markdown files
* refactor: move redundant tests
* feat: enhance markdown processing with consistent chunking logic and segment hashing
---------
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
* feat: apply changes from local main
* fix: add missing types
* feat: deduplicate code blocks coming out of parser
* feat: implement a cache manager to improve cache handling
* refactor: move code index service initialization to extension and remove await from indexing process
* fix: return undefined instead of throwing if no workspace is detected
* feat: allow auto approve if it is active for read tools
* refactor: improve UI of the results and allow opening the ranges directly in the editor
* refactor: use dependency injection to improve performance
* feat: implement result filtering by directory path
* refactor: centralize path normalization logic
* refactor: remove unnecessary barrel file
* refactor: prevent restarting the service if no settings change
* fix: the indexing process should never be awaited
* refactor: cleanup unused method
* refactor: remove batch limits for ollama
* refactor(parser): simplify method signatures and improve chunking logic
- Remove redundant min/max chars parameters
- Add better handling for oversized lines
- Improve chunking logic with segment handling
- Clean up method signatures and parameter ordering
* fix(settings): make select inputs full width in CodeIndexSettings
* refactor: increase max list file limit
* feat(ui): improve codebase search result display formatting
* test: add tests for cache and config managers
* test: create unit tests for parser and scanner
* feat(parser): improve segment hash uniqueness
- Added startCharIndex to segment hash calculation in _chunkTextByLines
- Track character position when splitting oversized lines
- Ensures unique identification of segments from same line
* feat(file-watcher): add error logging and optional ignoreController injection
* fix: allow getting the state if the service is disabled
* fix: set the embedding models when cline provider is initialized
* feat: use zod to validate form
* feat(file-watcher): enhance file watcher for batched deletions and improved vector store interactions
Improve file watcher to handle file deletions in batches and optimize vector store operations.
* feat(CodeIndexSettings): move OpenAI key input to a conditional rendering block
* feat(CodeIndexSettings): update button visibility based on indexing status
* feat(file-watcher): refactor vscode mock and enhance file watcher tests
* fix(CodeIndexManager): do not await startIndexing on configuration changes
* feat(types): add codeIndexOpenAiKey and codeIndexQdrantApiKey to ProviderSettings and IpcMessage
* feat(FileWatcher): enhance file processing with batch operations and new status handling
* fix(webviewMessageHandler): handle errors during CodeIndexManager initialization
* refactor(CodeIndexManager): streamline service creation by consolidating into a single method
* feat(CodeIndex): implement minimum search score configuration and update search methods
* refactor(CodeIndexSettings): replace ApiConfiguration with ProviderSettings and update related methods
* refactor: move contants to centralized file
* refactor(constants): rename CODEBASE_INDEX_SEARCH_MIN_SCORE to SEARCH_MIN_SCORE
* feat(QdrantVectorStore): enhance search functionality with new query structure and indexing
* feat(FileWatcher): implement batch processing and retry logic for upserting points
* fix(CodeIndexSettings): rename setProviderSettingsField to setApiConfigurationField and move model label
* fix(ChatRow): remove limit from search query messages
* refactor(CodebaseSearchResult): remove unused props from component
* feat: implement batch processing for file events in FileWatcher
- Introduced a new mechanism to accumulate file events (create, change, delete) and process them in batches.
- Added debounce functionality to optimize processing frequency.
- Emitted events for batch processing start, progress updates, and completion with detailed summaries.
- Refactored existing processing logic to handle batch deletions and upserts efficiently.
- Enhanced error handling and logging for better traceability during batch operations.
* feat(CodeIndex): implement batch processing and update progress reporting
* fix: define a default url for qdrant
* feat(CodeIndexManager): add initialization check and update startIndexing logic
* feat(CodeIndexSettings): validate Qdrant URL and update settings commitment logic
* feat: refactor progress calculation and update progress bar rendering
* refactor: remove webview provider and related methods
* fix: simplify indexing status update by directly using update values
* feat: integrate .gitignore support into file processing and scanning logic
* fix: update clearCacheFile method to write an empty object instead of deleting the cache file
* Revert this
* Run prettier
* fix: add new dependencies for qdrant client and directory scanner
* feat: add codebase search functionality to localization files
* feat: add localization strings for codebase indexing settings
* feat: integrate CodeIndexSettings into ExperimentalSettings and update settings localization
* refactor: remove console logs from various components for cleaner output
* feat: enhance capabilities section and codebase search tool description
* feat: add code indexing localization for multiple languages
* fix: correct indentation for CodeIndexSettings component in ExperimentalSettings
* refactor: update unit tests to properly test current functionality
* feat: add mock implementation for p-limit and update Jest config
* feat: track file creation, change, and deletion events in accumulatedEvents
* refactor: simplify file watcher tests by removing waitForFileProcessingToFinish and using direct event accumulation
* refactor: mock ContextProxy's getValue method to return current config name in ClineProvider tests
* refactor: mock missing properties required by codebase indexing manager
---------
Co-authored-by: cte <cestreich@gmail.com>
* refactor: standardize C# Tree-Sitter parser tests
This commit standardizes the C# Tree-Sitter parser tests to ensure:
- Variable/function names in sample code match the tests
- Each test only verifies specific identifiers relevant to that test
- Test names clearly indicate what they're testing
- No skipped tests remain
- Only one test exists per structure or combination of structures
- All debug output uses debugLog from helpers.ts
The standardization improves maintainability and clarity while preserving
all existing functionality.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: standardize C Tree-Sitter parser tests
This commit standardizes the C Tree-Sitter parser tests to ensure:
- Variable/function names in sample code match the tests (using snake_case)
- Each test only verifies specific identifiers relevant to that test
- Test names clearly indicate what they're testing
- No skipped tests remain - tests for unsupported features are removed
- Only one test exists per structure or combination of structures
- All debug output uses debugLog from helpers.ts
The standardization improves maintainability and clarity while preserving
all existing functionality. Documentation has been added for C language
constructs that are not currently supported by the parser.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: standardize Kotlin Tree-Sitter parser
Standardized Kotlin Tree-Sitter parser to ensure consistent naming and structure:
- Renamed identifiers to clearly indicate what they're testing
- Ensured all code sections are at least 4 lines long
- Organized query patterns logically with clear comments
- Removed skipped tests and unused query patterns
- Ensured only one test per structure or combination of structures
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: standardize PHP Tree-Sitter parser
Standardized PHP Tree-Sitter parser to ensure consistent naming and structure:
- Renamed identifiers to clearly indicate what they're testing
- Ensured all code sections are at least 4 lines long
- Organized query patterns logically with clear comments
- Consolidated related tests to ensure only one test per structure
- Ensured all debug output uses debugLog from helpers.ts
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: standardize Ruby Tree-Sitter parser
Standardized Ruby Tree-Sitter parser to ensure consistent naming and structure:
- Expanded query to support many more Ruby language constructs
- Added comprehensive test file with clear section headers
- Ensured all code sections are at least 4 lines long
- Added tests for each language construct
- Used consistent naming conventions for test identifiers
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: standardize Swift Tree-Sitter parser
Standardized Swift Tree-Sitter parser to ensure consistent naming and structure:
- Renamed identifiers to clearly indicate what they're testing
- Ensured all code sections are at least 4 lines long
- Organized query patterns logically with clear comments
- Removed skipped tests and unused query patterns
- Ensured only one test per structure or combination of structures
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: dynamically include all tree-sitter WASM files in build process
- Modified esbuild.js to dynamically read and copy all WASM files from tree-sitter-wasms package
- Added error handling and logging for better debugging
- Ensures all language parsers are automatically included without requiring manual updates
* refactor: move test mocks and fixtures to shared locations
Move test mocks to helpers.ts to be shared between test files
Create fixtures directory for shared test data
Move sample TSX content to fixtures/sample-tsx.ts
Update test files to use shared mocks and fixtures
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: simplify tree-sitter debug output
- Remove detailed node inspection to reduce noise
- Keep only essential tree structure output
- Remove duplicate content logging
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: standardize C# tree-sitter test structure
Move C# sample code to dedicated fixture file and create inspectCSharp test
following the same pattern as other language tests. This improves test
organization and maintainability by:
- Extracting sample C# code to fixtures/sample-c-sharp.ts
- Adding inspectCSharp.test.ts for tree structure inspection
- Updating parseSourceCodeDefinitions.c-sharp.test.ts to use fixture
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: add language-specific capture processing
- Add language parameter to processCaptures for language-specific handling
- Implement selective HTML filtering for jsx/tsx files
- Update all call sites to pass correct language parameter
- Fix type safety by properly passing language strings
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: move tree-sitter test samples to fixtures
Move sample code from parseSourceCodeDefinitions test files into dedicated
fixtures directory for better organization and reuse:
- Created fixtures directory for language samples
- Moved sample code for C#, C, JSON, Kotlin, PHP, Ruby, Rust, and Swift
- Created inspect test files for each language
- Updated original test files to import from fixtures
- Standardized test options across languages
- No behavior changes, all tests passing
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: standardize tree-sitter test code across languages
Standardize test code structure and naming across all language parsers:
- Rename identifiers to clearly indicate test purpose
- Ensure each code section has descriptive comments
- Group related code sections together
- Maintain language-specific naming conventions
- Keep test structure consistent across languages
- Remove tests for unsupported features
Changes made to:
- C, JSON, Kotlin, PHP, Ruby, Rust, and Swift test files
- Sample code fixtures
- Parser definition tests
- Inspect structure tests
No functional changes - all tests passing with improved maintainability.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: standardize tree-sitter parser tests across all languages
- Ensure all test structures span at least 4 lines for better parsing
- Create exactly one consolidated test per data structure type
- Use descriptive names that clearly indicate test purpose
- Improve query pattern organization and documentation
- Simplify inspect test files to focus on structure validation
- Implement result caching in test files for better performance
- Remove duplicate and skipped tests
- Follow consistent naming conventions across all languages
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* first pass all but elm-bash
* test: inspectTreeStructure helper now returns a string
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: establish single source of truth for MIN_COMPONENT_LINES
- Move minComponentLines parameter to a file-level constant
- Remove parameter passing to processCaptures function
- Update all references to use the global constant
- Remove redundant local constant declaration in parseFile
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: improve MIN_COMPONENT_LINES implementation
- Changed MIN_COMPONENT_LINES constant to use getter/setter functions
- Updated references to use getMinComponentLines() function
- Modified test helper to set value to 0 during tests
- This establishes a single source of truth while making testing easier
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: standardize tree-sitter test implementations
Standardized tree-sitter test implementations across all supported languages:
- Removed debugLog calls from test files
- Implemented consistent line number pattern matching
- Ensured proper test structure and execution order
- Added comprehensive test coverage for supported structures
- Maintained 1:1 mapping between queries, tests and samples
- Documented unsupported features in TODO sections
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* docs: document tree-sitter parseable but unsupported structures
Added TODO sections in test files documenting structures that can be parsed
by tree-sitter but currently lack query pattern support:
- Python: f-strings, complex type annotations, pattern matching
- C++: virtual methods, field initializers, base class clauses
- Java: import declarations, field declarations with modifiers
- TSX: React hooks, context providers, event handlers
Added examples and clarifying comments to existing TODO sections.
Enhanced Java query patterns for better structure capture including
lambda expressions, field declarations, and type parameters.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: use correct distDir variable for WASM file copying
The copyWasmFiles plugin was using an undefined targetDir variable when copying
tree-sitter WASM files. Changed to use the correctly defined distDir variable,
fixing the build process.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* lint: remove unused imports and variables from tree-sitter tests
Removed unused imports and variables from tree-sitter test files to fix linting errors:
- Removed unused goQuery import from helpers.ts
- Removed unused imports and mockedFs variables from language-specific test files
- Cleaned up test files to only import what they use
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
---------
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
This commit significantly enhances the Python tree-sitter parser to support a comprehensive range of Python language constructs, enabling more accurate and detailed code analysis.
Key improvements:
- Added support for method definitions (instance, class, and static methods)
- Added support for decorators on functions and classes
- Added support for module-level variables and constants
- Added support for async functions and methods
- Added support for property getters/setters
- Added support for type annotations in various contexts
- Added support for dataclasses
- Added support for nested functions and classes
- Added support for generator functions
- Added support for list/dict/set comprehensions
- Added support for lambda functions
- Added support for abstract base classes and methods
The parser now handles Python's rich feature set more comprehensively, including special Python patterns like decorators, type annotations, and various comprehension types. This enables better code navigation, understanding, and analysis for Python codebases.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
This enhancement significantly expands the Java parser's capabilities to recognize and parse a wide range of Java language constructs:
- Added support for enum declarations and enum constants
- Added support for annotation type declarations and elements
- Added support for field declarations
- Added support for constructor declarations
- Added support for lambda expressions
- Added support for inner and anonymous classes
- Added support for type parameters (generics)
- Added support for package and import declarations
These improvements enable more comprehensive code analysis for Java projects, providing better definition extraction and navigation capabilities.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
This enhancement significantly expands the Go parser's capabilities to recognize and extract a comprehensive set of language constructs:
- Added support for struct and interface definitions with proper type identification
- Implemented parsing for constant declarations (both single and in blocks)
- Added support for variable declarations (both single and in blocks)
- Added recognition of type aliases with proper distinction from regular types
- Implemented special handling for init functions
- Added support for anonymous functions, including nested function literals
- Improved documentation and organization of query patterns
These enhancements enable more accurate code navigation, better symbol extraction, and improved code intelligence for Go codebases.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
This enhancement significantly expands the C++ parser's capabilities to recognize and extract a wide range of modern C++ language constructs, improving code navigation and analysis.
New supported language constructs include:
- Union declarations and their members
- Destructors and their implementations
- Operator overloading (including stream operators)
- Free-standing and namespace-scoped functions
- Enum declarations (both traditional and scoped enum class)
- Lambda expressions and their captures
- Attributes and annotations
- Method overrides with virtual/override specifiers
- Exception specifications (noexcept)
- Default parameters in function declarations
- Variadic templates and parameter packs
- Structured bindings (C++17)
- Inline namespaces and nested namespace declarations
- Template specializations and instantiations
- Constructor implementations
This enhancement provides more comprehensive code structure analysis for C++ codebases, particularly those using modern C++ features from C++11, C++14, and C++17 standards.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
- Enhanced the Tree-Sitter parser for JavaScript/TypeScript with support for advanced language constructs
- Modified the parser to exclude comments from the output
- Consolidated sample code in tests for better maintainability
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: add comprehensive Rust Tree-sitter tests
Add test suite for Rust language support in Tree-sitter:
- Verify parsing of struct definitions
- Verify parsing of method definitions within impl blocks
- Verify parsing of standalone function definitions
- Add tests for complex Rust structures
- Include skipped debug test for future diagnostics
These tests confirm the existing Rust support in Tree-sitter is working correctly and provide a foundation for future enhancements to the Rust parser.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: enhance Rust tree-sitter parser with advanced language structures
- Add support for additional Rust language structures:
- Enum definitions
- Trait definitions
- Impl trait for struct
- Generic structs with lifetime parameters
- Macro definitions
- Module definitions
- Union types
- Closures
- Match expressions
- Where clauses
- Async functions
- Impl blocks with generic parameters
- Complex trait bounds
- Update tests to verify capture of these structures
- Modify debug helper to use environment variables
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
---------
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: add Tree-sitter TSX query support
Added support for parsing TSX files with Tree-sitter:
- Created TSX query patterns for React components and functions
- Fixed field name issues by using direct node matching
- Added comprehensive documentation about TSX component structure
- Created a debug tool to inspect the actual tree structure
- Added test coverage for TSX parsing
- Embedded test fixture directly in the test file
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: combine TypeScript and TSX queries
Reduce duplication by:
- Import TypeScript queries from typescript.ts
- Keep only TSX-specific component and JSX queries
- Update documentation to reflect combined approach
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: update integration for parse definitions
- Adjusted logParseResult to call the actual parse definitions function using WASM from initializeWorkingParser.
- Patched TreeSitter initialization to resolve the WASM path correctly and bypass redundant init() calls.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: should successfully call parseSourceCodeDefinitionsForFile
Mock loadRequiredLanguageParsers to use real parser instance from initializeTreeSitter,
ensuring proper interaction between parseSourceCodeDefinitionsForFile and its dependencies.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: improve test structure for parseSourceCodeDefinitions
- Combine component parsing tests into a single test case
- Update test assertions to match actual parser output
- Fix interface and component definition tests to use VSCodeCheckbox as sample
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: improve React component detection in tree-sitter
- Add tests for complex TSX structures (nested components, HOCs)
- Enhance TSX queries to better detect React components
- Document current limitations in test file
* feat: improve React component detection in TSX files
Make TSX/React component detection more generic and robust by:
- Implementing structural pattern matching instead of specific React wrapper functions
- Adding configurable line threshold for React component inclusion (MIN_COMPONENT_LINES)
- Adding robust HTML element filtering with regex patterns
- Improving React component name handling for nested components
- Ensuring proper context handling for multi-line React components
Also update mock captures in tree-sitter tests to span at least 4 lines
to meet the MIN_COMPONENT_LINES threshold.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: use testParseSourceCodeDefinitions in logParseResult
- Moved testParseSourceCodeDefinitions function to top level
- Updated logParseResult to use testParseSourceCodeDefinitions
- Removed duplicate function from describe block
- Added console.log for debugging output
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: improve React component detection output format
- Remove individual line output for components that don't meet MIN_COMPONENT_LINES threshold
- Update tests to match the new behavior where lines < MIN_COMPONENT_LINES are skipped
- Maintain range output (e.g., 1--4) for component definitions
- Preserve context for larger definitions as ranges only
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: add switch/case statement support to tree-sitter TypeScript parser
- Added node patterns in typescript.ts query to capture switch statements, case clauses, and default clauses
- Modified index.ts to avoid duplicate line ranges in the output
- Added test for switch/case statements with complex case blocks
- Fixed line range tracking to prevent duplicate output
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: add support for enum declarations in tree-sitter TypeScript parser
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: add support for namespace declarations in tree-sitter TypeScript parser
Added query pattern for namespace declarations (internal_module nodes) in the TypeScript parser.
This allows the parser to identify and extract namespace declarations from TypeScript code.
- Added test case to verify namespace parsing functionality
- Added query pattern to capture namespace declarations in typescript.ts
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: add decorator pattern support to tree-sitter TypeScript parser
- Added support for parsing complex decorators with arguments
- Added test case for Component decorator pattern
- Enhanced TypeScript queries to capture decorator definitions
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: add generic type declaration support to tree-sitter TypeScript parser
- Added support for parsing generic types with constraints
- Added test case for Dictionary<K extends string | number, V> pattern
- Enhanced TypeScript queries to capture generic type definitions
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: add conditional type support to tree-sitter TypeScript parser
- Added test case for conditional type patterns like ReturnType<T>
- Verified that conditional types with infer keyword are already supported
- Enhanced inspectTreeStructure with detailed node inspection for debugging
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: add template literal type support to tree-sitter TypeScript parser
- Added test case for template literal type patterns like EventName<T>
- Verified that template literal types are already partially supported
- Confirmed support for complex template literal patterns in conditional types
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: implement tree-sitter compatible markdown processor
Adds a special case implementation for markdown files that:
- Parses markdown headers and section line ranges
- Returns captures in a format compatible with tree-sitter
- Integrates with the existing parseFile function
- Includes comprehensive tests for the implementation
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: markdown parser not detecting sections with horizontal rules
The markdownParser was incorrectly interpreting horizontal rules (---) as setext headers when they appeared after non-header text. This caused some sections to be missed in the output.
This fix:
- Makes setext header detection more strict by requiring at least 3 = or - characters
- Adds validation for the text line before a potential setext header
- Ensures horizontal rules are not confused with setext headers
Added a test case to verify the fix works correctly with horizontal rules.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: move helper functions to dedicated file
Move test helper functions from parseSourceCodeDefinitions.test.ts to a new helpers.ts file.
Rename test file to parseSourceCodeDefinitions.tsx.test.ts to indicate it's for TSX tests.
This improves code organization by separating test helpers from test cases.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: enable JSON structure display in list_code_definitions
This change allows list_code_definitions to show JSON structures by:
- Moving JSON query patterns into the JavaScript query file
- Using the JavaScript parser for JSON files
- Removing the separate JSON parser implementation
- Adding comprehensive tests for JSON parsing
Example output for a JSON file:
# test.json
0--90 | {
1--9 | "server": {
4--8 | "ssl": {
10--45 | "database": {
11--24 | "primary": {
14--18 | "credentials": {
19--23 | "pool": {
25--44 | "replicas": [
26--43 | {
30--42 | "status": {
33--41 | "metrics": {
36--40 | "connections": {
46--73 | "features": {
47--72 | "auth": {
48--71 | "providers": {
49--53 | "local": {
54--70 | "oauth": {
56--69 | "providers": [
57--68 | {
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: Add compile step to CI workflow to ensure WASM files are available
The TreeSitter tests were failing in CI because the WASM files weren't being copied to the dist directory before running the tests. This adds an explicit compile step to ensure the WASM files are properly built and copied.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: improve tree-sitter test type safety and debug logging
- Replace 'any' types with proper Parser types in helpers.ts
- Add centralized DEBUG flag and debugLog function in helpers.ts
- Update all console.log statements to use debugLog across all test files
- This change appeases @ellipsis-dev by improving type safety and
providing a clean way to control debug logging
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
---------
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
Use compact line number format matching read_file/search_files to:
- Reduce token usage in AI responses
- Simplify output parsing
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat read file range
feat: add file truncation with code structure preview
Add configurable line limit for large file handling with source code structure preview. This improves performance and memory usage while maintaining code readability.
Key changes:
Add maxReadFileLine setting (default: 4500) with UI controls
Create efficient line counter utility using streams
Add parseSourceCodeDefinitionsForFile for single file parsing
Show truncation notice with code structure for large files
Add comprehensive tests for new functionality
Move file extensions to shared constant
add line number
pr comment
Return definition ranges from tree-sitter
make treesitter always add line
add line number to rg search
enhance tree siter output & fix test
update snapshot
fix test
* Add translations, tweak range
---------
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
