GitNexus/gitnexus/test/unit/tree-sitter-queries.test.ts
Gergő Magyar 7999b6ba7b
refactor: SICP-informed LanguageProvider architecture (#488)
* refactor: SICP-informed LanguageProvider architecture for ingestion pipeline

Consolidate 16 scattered dispatch surfaces into a single LanguageProvider
Strategy interface per language. Processors are now fully language-agnostic —
zero SupportedLanguages.X enum access, zero dispatch table imports.

Architecture (5-layer DAG, zero circular dependencies):
  L0: Capability modules (dispatch tables, single source of truth)
  L1: LanguageProvider interface + createLanguageProvider factory
  L2: 13 per-language provider files (Strategy objects)
  L3: Registry with satisfies Record<SL, LP> + pre-built lookup maps
  L4: Processors (language-agnostic, all behavior via provider.*)

Key changes:
- Add LanguageProvider interface with 15 properties (6 required, 9 optional)
- Create 13 provider files in languages/ + php-helpers.ts
- Migrate all processors to getProvider(language) — cached once per scope
- Replace heritage if-checks with provider.interfaceNamePattern/heritageDefaultEdge
- Replace MRO switch(language) with switch(provider.mroStrategy)
- Replace isNodeExported with provider.exportChecker
- Move PHP description extraction behind provider.descriptionExtractor
- Move Swift implicit imports behind provider.implicitImportWirer
- Move PHP route detection behind provider.isRouteFile
- Move Kotlin wildcard append behind provider.importPathPreprocessor
- Remove deprecated TypeEnvironment.env, add fileScope()/allScopes()
- De-export TypeEnv type (module-private)
- Pre-build extensionMap, WILDCARD_LANGUAGES, SYNTHESIS_LANGUAGES at load
- Remove dead entryPointPatterns/frameworkPatterns from interface
- Derive createLanguageProvider config type via Pick/Partial/Omit
- Tighten callback types from any to SyntaxNode
- Migrate 270+ test call sites from .env to TypeEnvironment API

Adding a new language: 3 files (enum + provider + registry line).
No processor file touched. Ever.

* refactor: clean architecture for LanguageProvider with O(1) AST cache

Address all PR #488 review comments and achieve pristine SICP layer separation:

Interface redesign:
- Split LanguageProvider into Config (input) + Provider (runtime with defaults)
- Rename createLanguageProvider → defineLanguage with explicit DEFAULTS constant
- Add MroStrategy, ImportSemantics named type aliases for better IDE tooltips
- Tighten labelOverride signature: string|null → NodeLabel|null (compile-time safety)
- Tighten descriptionExtractor nodeLabel: string → NodeLabel
- Un-export LanguageProviderConfig (internal to defineLanguage)

CI fixes (all 4 failures resolved):
- isNodeExported: add null guard for unknown languages
- preprocessImportPath tests: pass getProvider() instead of raw enum
- MRO tests: update expected strings to match language-agnostic prefixes

Code deduplication:
- Extract findDescendant/extractStringContent to ast-helpers.ts (single source of truth)
- Unify Kotlin method detection: remove duplicate from extractFunctionName,
  use provider.labelOverride as single source of truth via findEnclosingFunctionId
- extractFunctionName return type: string → NodeLabel

Performance (O(1) AST node access):
- Add per-file Map-based memoization in parse-worker for parent-chain walks
- Cache enclosingClassId, enclosingFunctionId, exportStatus per SyntaxNode
- Clear caches before each file parse (not after — handles parse failures)

Architecture (pristine languages/ folder):
- Move php-helpers.ts → helpers/php.ts (L0 capability, not L2 config)
- Create helpers/swift.ts from extracted Swift provider logic
- Extract cppLabelOverride AST walk → isCppInsideClassOrStruct in ast-helpers.ts
- Extract isPhpRouteFile → helpers/php.ts
- All 13 provider files are now pure configuration — zero implementation logic
- Ruby: remove no-op namedBindingExtractor assignment (undefined from dispatch table)

* refactor: eliminate LANGUAGE_QUERIES, typeConfigs, namedBindingExtractors dispatch tables

Phase 1 of L0 dispatch table elimination. Providers now import capabilities
directly instead of indexing into redundant Record<SL, T> dispatch tables:

- LANGUAGE_QUERIES: providers import named query constants directly
  (TYPESCRIPT_QUERIES, PYTHON_QUERIES, etc.). Table kept in tree-sitter-queries.ts
  for call-processor.ts dynamic lookup + test consumers.

- typeConfigs: providers import from individual type-extractor files
  (typescriptConfig from typescript.ts, javaTypeConfig from jvm.ts, etc.).
  Dispatch table fully removed from type-extractors/index.ts.

- namedBindingExtractors: providers import extractors directly from
  named-binding-extraction.ts (extractTsNamedBindings, etc.).
  Dispatch table fully removed from import-resolution.ts.

Net: -48 LOC of dispatch table indirection. L3 satisfies Record<SL, LP>
remains the single exhaustiveness check.

* refactor: eliminate exportCheckers, callRouters, importResolvers dispatch tables

Phase 2 of L0 dispatch table elimination. All 6 dispatch tables are now gone:

- exportCheckers: individual checkers exported directly (tsExportChecker,
  pythonExportChecker, etc.). isNodeExported uses a local checkersByLanguage
  map to avoid circular dependency with languages/index.ts.

- callRouters: table removed. Providers import noRouting or routeRubyCall
  directly. noRouting now exported. Dead import removed from call-processor.ts.

- importResolvers: resolver functions exported with clean names
  (resolveTypescriptImport, resolveJavaImport, etc.). Inline lambdas
  extracted to named exports. Dispatch functions renamed from *Dispatch
  suffix to clean resolve*Import pattern.

Combined with Phase 1, all 6 L0 dispatch tables have been eliminated.
L3 satisfies Record<SL, LanguageProvider> is the single exhaustiveness check.
Providers are now fully self-contained — each imports its capabilities directly.

* perf+refactor: type-env caching, sequential fallback caching, utils.ts split

Phase 3 — performance optimizations and barrel cleanup:

Type-env parent-walk caching:
- Memoize findEnclosingClassName and findEnclosingParentClassName with
  per-file Map<SyntaxNode, string|undefined> caches
- Eliminates O(n*m) repeated child scanning in extractParentClassFromNode
- Caches cleared in buildTypeEnv before each file's walk phase

Sequential fallback caching:
- Add classIdCache + exportCache Maps to parsing-processor.ts
- Mirrors the O(1) memoization pattern from parse-worker.ts
- Both paths now have identical caching for parent-chain walks

Split utils.ts barrel into focused modules:
- noise-filter.ts: BUILT_IN_NAMES + isBuiltInOrNoise (167 LOC)
- language-detection.ts: getLanguageFromFilename (58 LOC)
- utils.ts slimmed to re-exports + yieldToEventLoop + isVerboseIngestionEnabled
- Backward compatible — existing imports from utils.ts still work

* refactor: rename resolvers/ → import-resolvers/, restructure tests per-concern

Directory renames (git mv — history preserved):
- src/core/ingestion/resolvers/ → import-resolvers/ (10 files)
- test/unit/call-routing.test.ts → call-routing/ruby.test.ts
- test/unit/named-binding-extraction.test.ts → named-bindings/csharp.test.ts
- test/unit/import-resolution.test.ts → import-resolution/preprocessing.test.ts

All 11 import paths updated to reference new import-resolvers/ location.
Test imports updated for new subdirectory depth.

Note: test/integration/resolvers/ NOT renamed — those tests cover the full
ingestion pipeline per-language, not just import resolution.

* refactor: eliminate utils.ts barrel — all 33 consumers now import directly

Migrated 65 import sites across 33 files to import from the focused source
module instead of the utils.ts barrel:

- ast-helpers.js: SyntaxNode, extractFunctionName, findEnclosingClassId, etc.
- call-analysis.js: inferCallForm, extractReceiverName, countCallArguments, etc.
- noise-filter.js: BUILT_IN_NAMES, isBuiltInOrNoise
- language-detection.js: getLanguageFromFilename

utils.ts reduced to 2 original functions only:
- yieldToEventLoop
- isVerboseIngestionEnabled

Zero re-exports remain. Every import is now direct to its source module.

* refactor: create utils/ folder, move all shared utilities, delete utils.ts barrel

Final phase of module structure migration:

- git mv ast-helpers.ts, call-analysis.ts, noise-filter.ts,
  language-detection.ts → utils/ subdirectory (history preserved)
- Extract yieldToEventLoop → utils/event-loop.ts
- Extract isVerboseIngestionEnabled → utils/verbose.ts
- Delete utils.ts (zero re-exports, zero functions remain)
- Update 38 import paths across source and test files

The ingestion/ root is now clean — only processors, capability modules,
and the pipeline orchestrator live at the top level. All shared utilities
are in utils/, all language-specific helpers in helpers/, all import
resolvers in import-resolvers/.

* refactor: move findChild from import-resolvers/utils.ts to utils/ast-helpers.ts

findChild is a generic AST helper (find first named child by type) — it
belongs with the other AST traversal utilities, not in the import resolver
module. 4 consumers updated to import from utils/ast-helpers.js.

* refactor: split named-binding-extraction.ts into per-language files

Rename named-binding-extraction.ts → named-binding-processor.ts (git mv,
history preserved), keeping only walkBindingChain for re-export chain resolution.

7 per-language extractor functions moved to named-bindings/ subdirectory:
- named-bindings/typescript.ts (extractTsNamedBindings — TS + JS)
- named-bindings/python.ts (extractPythonNamedBindings)
- named-bindings/kotlin.ts (extractKotlinNamedBindings)
- named-bindings/rust.ts (extractRustNamedBindings + collectRustBindings)
- named-bindings/php.ts (extractPhpNamedBindings)
- named-bindings/csharp.ts (extractCsharpNamedBindings)
- named-bindings/java.ts (extractJavaNamedBindings)

Each provider now imports its binding extractor from the per-language file.

* refactor: eliminate import-resolution.ts — distribute to natural homes

Split per-language resolvers into import-resolvers/ per-language files and
eliminate the import-resolution.ts catch-all module entirely:

Per-language resolvers moved to import-resolvers/:
- standard.ts: resolveStandard, resolveJavascriptImport, resolveTypescriptImport,
  resolveCImport, resolveCppImport
- jvm.ts: resolveJavaImport, resolveKotlinImport
- go.ts: resolveGoImport
- csharp.ts: resolveCSharpImport (helper renamed to Internal)
- php.ts, python.ts, ruby.ts, rust.ts: same pattern
- swift.ts: new file for resolveSwiftImport

Types distributed to their concern directories:
- import-resolvers/types.ts: ImportResult, ImportConfigs, ResolveCtx, ImportResolverFn
- named-bindings/types.ts: NamedBinding, NamedBindingExtractorFn

preprocessImportPath moved to import-processor.ts (its primary consumer).

import-resolution.ts deleted — zero catch-all modules remain.

* refactor: tighten SPR — eliminate re-exports, dead code, type holes, and redundant patterns

12 review findings resolved across the ingestion layer:

Type safety:
- CallRouter callNode: any → SyntaxNode (closes type hole)
- CaptureMap type alias replaces Record<string, any>
- providersWithImplicitWiring filter now type-narrowed (removes ! assertions)
- Ruby exportChecker: unnecessary as-cast removed, named export created

Architecture:
- Circular type dependency eliminated (ImportResolutionContext moved to types.ts)
- LANGUAGE_QUERIES residual dispatch replaced with provider.treeSitterQueries
- noRouting sentinel deleted — callRouter now properly optional on 12 providers
- All 6 re-exports from import-processor/pipeline/languages eliminated

Pattern cleanup:
- Dead checkersByLanguage table + isNodeExported removed from export-detection
- 4 duplicated config interfaces consolidated to language-config.ts
- extractCsharpNamedBindings → extractCSharpNamedBindings (casing consistency)

Simplification:
- import-resolvers/index.ts barrel deleted (dead re-exports)
- helpers/ inlined into languages/ (php.ts, swift.ts) — 1 directory removed

Verified: tsc --noEmit clean, 3837 tests pass, 0 failures.

* refactor: address review — remove LANGUAGE_QUERIES table, type-extractors barrel, fix Windows timeout

Review comment fixes (github.com/abhigyanpatwari/GitNexus/pull/488#issuecomment-4117817648):

1. LANGUAGE_QUERIES dispatch table removed from tree-sitter-queries.ts
   — 5 test files migrated to getProvider(lang).treeSitterQueries
   — eliminates last parallel dispatch surface

2. type-extractors/index.ts barrel deleted
   — type-env.ts now imports TYPED_PARAMETER_TYPES from shared.js directly

3. Windows CI timeout fix: afterAll cleanup hook in test-indexed-db.ts
   now passes explicit 120s timeout to prevent KuzuDB C++ destructor
   hang from hitting vitest's default 30s testTimeout on Windows

Verified: tsc --noEmit clean, 3835 tests pass, 0 failures.

* refactor: eliminate chained getProvider property access — assign to variable first

All getProvider(lang).property calls now follow the pattern:
  const provider = getProvider(language);
  const x = provider.property;

5 source files + 4 test files updated (~35 occurrences).
This ensures consistent provider variable usage and avoids
repeated lookups in hot paths.

* refactor: remove last 4 re-exports from import-resolvers, fix stale CaptureMap comment

- Remove `export type { TsconfigPaths }` from standard.ts
- Remove `export type { GoModuleConfig }` from go.ts
- Remove `export type { ComposerConfig }` from php.ts
- Remove `export type { CSharpProjectConfig }` from csharp.ts
  All 4 types are canonically defined in language-config.ts;
  zero consumers imported via the resolver re-exports.

- Fix stale CaptureMap JSDoc: said "Uses any" but type is SyntaxNode | undefined
2026-03-24 13:42:39 +00:00

291 lines
10 KiB
TypeScript

import { describe, it, expect } from 'vitest';
import {
TYPESCRIPT_QUERIES,
JAVASCRIPT_QUERIES,
PYTHON_QUERIES,
JAVA_QUERIES,
C_QUERIES,
GO_QUERIES,
CPP_QUERIES,
CSHARP_QUERIES,
RUST_QUERIES,
PHP_QUERIES,
SWIFT_QUERIES,
} from '../../src/core/ingestion/tree-sitter-queries.js';
describe('tree-sitter queries', () => {
describe('TypeScript queries', () => {
it('captures class declarations', () => {
expect(TYPESCRIPT_QUERIES).toContain('class_declaration');
expect(TYPESCRIPT_QUERIES).toContain('@definition.class');
});
it('captures interface declarations', () => {
expect(TYPESCRIPT_QUERIES).toContain('interface_declaration');
expect(TYPESCRIPT_QUERIES).toContain('@definition.interface');
});
it('captures function declarations', () => {
expect(TYPESCRIPT_QUERIES).toContain('function_declaration');
expect(TYPESCRIPT_QUERIES).toContain('@definition.function');
});
it('captures method definitions', () => {
expect(TYPESCRIPT_QUERIES).toContain('method_definition');
expect(TYPESCRIPT_QUERIES).toContain('@definition.method');
});
it('captures arrow functions in variable declarations', () => {
expect(TYPESCRIPT_QUERIES).toContain('arrow_function');
});
it('captures imports', () => {
expect(TYPESCRIPT_QUERIES).toContain('import_statement');
expect(TYPESCRIPT_QUERIES).toContain('@import');
});
it('captures call expressions', () => {
expect(TYPESCRIPT_QUERIES).toContain('call_expression');
expect(TYPESCRIPT_QUERIES).toContain('@call');
});
it('captures heritage (extends/implements)', () => {
expect(TYPESCRIPT_QUERIES).toContain('@heritage.extends');
expect(TYPESCRIPT_QUERIES).toContain('@heritage.implements');
});
});
describe('JavaScript queries', () => {
it('captures function and class definitions', () => {
expect(JAVASCRIPT_QUERIES).toContain('@definition.class');
expect(JAVASCRIPT_QUERIES).toContain('@definition.function');
expect(JAVASCRIPT_QUERIES).toContain('@definition.method');
});
it('captures heritage (extends)', () => {
expect(JAVASCRIPT_QUERIES).toContain('@heritage.extends');
});
it('does not have interface declarations', () => {
expect(JAVASCRIPT_QUERIES).not.toContain('interface_declaration');
});
});
describe('Python queries', () => {
it('captures class and function definitions', () => {
expect(PYTHON_QUERIES).toContain('class_definition');
expect(PYTHON_QUERIES).toContain('function_definition');
});
it('captures imports including from-imports', () => {
expect(PYTHON_QUERIES).toContain('import_statement');
expect(PYTHON_QUERIES).toContain('import_from_statement');
});
it('captures heritage (class inheritance)', () => {
expect(PYTHON_QUERIES).toContain('@heritage.extends');
});
});
describe('Java queries', () => {
it('captures all major declaration types', () => {
expect(JAVA_QUERIES).toContain('@definition.class');
expect(JAVA_QUERIES).toContain('@definition.interface');
expect(JAVA_QUERIES).toContain('@definition.enum');
expect(JAVA_QUERIES).toContain('@definition.method');
expect(JAVA_QUERIES).toContain('@definition.constructor');
expect(JAVA_QUERIES).toContain('@definition.annotation');
});
it('captures extends and implements heritage', () => {
expect(JAVA_QUERIES).toContain('@heritage.extends');
expect(JAVA_QUERIES).toContain('@heritage.implements');
});
});
describe('C queries', () => {
it('captures function definitions', () => {
expect(C_QUERIES).toContain('function_definition');
expect(C_QUERIES).toContain('@definition.function');
});
it('captures struct, union, enum, typedef', () => {
expect(C_QUERIES).toContain('@definition.struct');
expect(C_QUERIES).toContain('@definition.union');
expect(C_QUERIES).toContain('@definition.enum');
expect(C_QUERIES).toContain('@definition.typedef');
});
it('captures macros', () => {
expect(C_QUERIES).toContain('@definition.macro');
});
it('captures includes as imports', () => {
expect(C_QUERIES).toContain('preproc_include');
});
});
describe('Go queries', () => {
it('captures function and method declarations', () => {
expect(GO_QUERIES).toContain('function_declaration');
expect(GO_QUERIES).toContain('method_declaration');
});
it('captures struct and interface types', () => {
expect(GO_QUERIES).toContain('@definition.struct');
expect(GO_QUERIES).toContain('@definition.interface');
});
it('captures import declarations', () => {
expect(GO_QUERIES).toContain('import_declaration');
});
});
describe('C++ queries', () => {
it('captures class, struct, namespace', () => {
expect(CPP_QUERIES).toContain('@definition.class');
expect(CPP_QUERIES).toContain('@definition.struct');
expect(CPP_QUERIES).toContain('@definition.namespace');
});
it('captures templates', () => {
expect(CPP_QUERIES).toContain('@definition.template');
expect(CPP_QUERIES).toContain('template_declaration');
});
it('captures heritage (base class)', () => {
expect(CPP_QUERIES).toContain('@heritage.extends');
});
});
describe('C# queries', () => {
it('captures all major types', () => {
expect(CSHARP_QUERIES).toContain('@definition.class');
expect(CSHARP_QUERIES).toContain('@definition.interface');
expect(CSHARP_QUERIES).toContain('@definition.struct');
expect(CSHARP_QUERIES).toContain('@definition.enum');
expect(CSHARP_QUERIES).toContain('@definition.record');
expect(CSHARP_QUERIES).toContain('@definition.delegate');
});
it('captures namespace declarations', () => {
expect(CSHARP_QUERIES).toContain('@definition.namespace');
});
it('captures constructor and property', () => {
expect(CSHARP_QUERIES).toContain('@definition.constructor');
expect(CSHARP_QUERIES).toContain('@definition.property');
});
});
describe('Rust queries', () => {
it('captures function items', () => {
expect(RUST_QUERIES).toContain('function_item');
expect(RUST_QUERIES).toContain('@definition.function');
});
it('captures struct, enum, trait, impl', () => {
expect(RUST_QUERIES).toContain('@definition.struct');
expect(RUST_QUERIES).toContain('@definition.enum');
expect(RUST_QUERIES).toContain('@definition.trait');
expect(RUST_QUERIES).toContain('@definition.impl');
});
it('captures module, const, static, macro', () => {
expect(RUST_QUERIES).toContain('@definition.module');
expect(RUST_QUERIES).toContain('@definition.const');
expect(RUST_QUERIES).toContain('@definition.static');
expect(RUST_QUERIES).toContain('@definition.macro');
});
it('captures trait implementation heritage', () => {
expect(RUST_QUERIES).toContain('@heritage.trait');
expect(RUST_QUERIES).toContain('@heritage.class');
});
});
describe('PHP queries', () => {
it('captures class, interface, trait, enum', () => {
expect(PHP_QUERIES).toContain('@definition.class');
expect(PHP_QUERIES).toContain('@definition.interface');
expect(PHP_QUERIES).toContain('@definition.trait');
expect(PHP_QUERIES).toContain('@definition.enum');
});
it('captures top-level function definitions', () => {
expect(PHP_QUERIES).toContain('function_definition');
expect(PHP_QUERIES).toContain('@definition.function');
});
it('captures method declarations', () => {
expect(PHP_QUERIES).toContain('method_declaration');
expect(PHP_QUERIES).toContain('@definition.method');
});
it('captures class properties', () => {
expect(PHP_QUERIES).toContain('property_declaration');
expect(PHP_QUERIES).toContain('@definition.property');
});
it('captures heritage (extends, implements, use trait)', () => {
expect(PHP_QUERIES).toContain('@heritage.extends');
expect(PHP_QUERIES).toContain('@heritage.implements');
expect(PHP_QUERIES).toContain('@heritage.trait');
});
it('captures namespace definitions', () => {
expect(PHP_QUERIES).toContain('namespace_definition');
expect(PHP_QUERIES).toContain('@definition.namespace');
});
});
describe('Swift queries', () => {
it('captures class, struct, enum', () => {
expect(SWIFT_QUERIES).toContain('@definition.class');
expect(SWIFT_QUERIES).toContain('@definition.struct');
expect(SWIFT_QUERIES).toContain('@definition.enum');
});
it('captures protocols as interfaces', () => {
expect(SWIFT_QUERIES).toContain('protocol_declaration');
expect(SWIFT_QUERIES).toContain('@definition.interface');
});
it('captures init declarations as constructors', () => {
expect(SWIFT_QUERIES).toContain('init_declaration');
expect(SWIFT_QUERIES).toContain('@definition.constructor');
});
it('captures function declarations', () => {
expect(SWIFT_QUERIES).toContain('function_declaration');
expect(SWIFT_QUERIES).toContain('@definition.function');
});
it('captures protocol method declarations', () => {
expect(SWIFT_QUERIES).toContain('protocol_function_declaration');
expect(SWIFT_QUERIES).toContain('@definition.method');
});
it('captures properties', () => {
expect(SWIFT_QUERIES).toContain('property_declaration');
expect(SWIFT_QUERIES).toContain('@definition.property');
});
it('captures heritage (inheritance)', () => {
expect(SWIFT_QUERIES).toContain('@heritage.extends');
});
it('captures type aliases', () => {
expect(SWIFT_QUERIES).toContain('typealias_declaration');
expect(SWIFT_QUERIES).toContain('@definition.type');
});
it('captures extensions as classes', () => {
expect(SWIFT_QUERIES).toContain('"extension"');
});
it('captures actors as classes', () => {
expect(SWIFT_QUERIES).toContain('"actor"');
});
});
});