GitNexus/gitnexus/test/unit/import-resolution/preprocessing.test.ts
Gergő Magyar 7999b6ba7b
refactor: SICP-informed LanguageProvider architecture (#488)
* refactor: SICP-informed LanguageProvider architecture for ingestion pipeline

Consolidate 16 scattered dispatch surfaces into a single LanguageProvider
Strategy interface per language. Processors are now fully language-agnostic —
zero SupportedLanguages.X enum access, zero dispatch table imports.

Architecture (5-layer DAG, zero circular dependencies):
  L0: Capability modules (dispatch tables, single source of truth)
  L1: LanguageProvider interface + createLanguageProvider factory
  L2: 13 per-language provider files (Strategy objects)
  L3: Registry with satisfies Record<SL, LP> + pre-built lookup maps
  L4: Processors (language-agnostic, all behavior via provider.*)

Key changes:
- Add LanguageProvider interface with 15 properties (6 required, 9 optional)
- Create 13 provider files in languages/ + php-helpers.ts
- Migrate all processors to getProvider(language) — cached once per scope
- Replace heritage if-checks with provider.interfaceNamePattern/heritageDefaultEdge
- Replace MRO switch(language) with switch(provider.mroStrategy)
- Replace isNodeExported with provider.exportChecker
- Move PHP description extraction behind provider.descriptionExtractor
- Move Swift implicit imports behind provider.implicitImportWirer
- Move PHP route detection behind provider.isRouteFile
- Move Kotlin wildcard append behind provider.importPathPreprocessor
- Remove deprecated TypeEnvironment.env, add fileScope()/allScopes()
- De-export TypeEnv type (module-private)
- Pre-build extensionMap, WILDCARD_LANGUAGES, SYNTHESIS_LANGUAGES at load
- Remove dead entryPointPatterns/frameworkPatterns from interface
- Derive createLanguageProvider config type via Pick/Partial/Omit
- Tighten callback types from any to SyntaxNode
- Migrate 270+ test call sites from .env to TypeEnvironment API

Adding a new language: 3 files (enum + provider + registry line).
No processor file touched. Ever.

* refactor: clean architecture for LanguageProvider with O(1) AST cache

Address all PR #488 review comments and achieve pristine SICP layer separation:

Interface redesign:
- Split LanguageProvider into Config (input) + Provider (runtime with defaults)
- Rename createLanguageProvider → defineLanguage with explicit DEFAULTS constant
- Add MroStrategy, ImportSemantics named type aliases for better IDE tooltips
- Tighten labelOverride signature: string|null → NodeLabel|null (compile-time safety)
- Tighten descriptionExtractor nodeLabel: string → NodeLabel
- Un-export LanguageProviderConfig (internal to defineLanguage)

CI fixes (all 4 failures resolved):
- isNodeExported: add null guard for unknown languages
- preprocessImportPath tests: pass getProvider() instead of raw enum
- MRO tests: update expected strings to match language-agnostic prefixes

Code deduplication:
- Extract findDescendant/extractStringContent to ast-helpers.ts (single source of truth)
- Unify Kotlin method detection: remove duplicate from extractFunctionName,
  use provider.labelOverride as single source of truth via findEnclosingFunctionId
- extractFunctionName return type: string → NodeLabel

Performance (O(1) AST node access):
- Add per-file Map-based memoization in parse-worker for parent-chain walks
- Cache enclosingClassId, enclosingFunctionId, exportStatus per SyntaxNode
- Clear caches before each file parse (not after — handles parse failures)

Architecture (pristine languages/ folder):
- Move php-helpers.ts → helpers/php.ts (L0 capability, not L2 config)
- Create helpers/swift.ts from extracted Swift provider logic
- Extract cppLabelOverride AST walk → isCppInsideClassOrStruct in ast-helpers.ts
- Extract isPhpRouteFile → helpers/php.ts
- All 13 provider files are now pure configuration — zero implementation logic
- Ruby: remove no-op namedBindingExtractor assignment (undefined from dispatch table)

* refactor: eliminate LANGUAGE_QUERIES, typeConfigs, namedBindingExtractors dispatch tables

Phase 1 of L0 dispatch table elimination. Providers now import capabilities
directly instead of indexing into redundant Record<SL, T> dispatch tables:

- LANGUAGE_QUERIES: providers import named query constants directly
  (TYPESCRIPT_QUERIES, PYTHON_QUERIES, etc.). Table kept in tree-sitter-queries.ts
  for call-processor.ts dynamic lookup + test consumers.

- typeConfigs: providers import from individual type-extractor files
  (typescriptConfig from typescript.ts, javaTypeConfig from jvm.ts, etc.).
  Dispatch table fully removed from type-extractors/index.ts.

- namedBindingExtractors: providers import extractors directly from
  named-binding-extraction.ts (extractTsNamedBindings, etc.).
  Dispatch table fully removed from import-resolution.ts.

Net: -48 LOC of dispatch table indirection. L3 satisfies Record<SL, LP>
remains the single exhaustiveness check.

* refactor: eliminate exportCheckers, callRouters, importResolvers dispatch tables

Phase 2 of L0 dispatch table elimination. All 6 dispatch tables are now gone:

- exportCheckers: individual checkers exported directly (tsExportChecker,
  pythonExportChecker, etc.). isNodeExported uses a local checkersByLanguage
  map to avoid circular dependency with languages/index.ts.

- callRouters: table removed. Providers import noRouting or routeRubyCall
  directly. noRouting now exported. Dead import removed from call-processor.ts.

- importResolvers: resolver functions exported with clean names
  (resolveTypescriptImport, resolveJavaImport, etc.). Inline lambdas
  extracted to named exports. Dispatch functions renamed from *Dispatch
  suffix to clean resolve*Import pattern.

Combined with Phase 1, all 6 L0 dispatch tables have been eliminated.
L3 satisfies Record<SL, LanguageProvider> is the single exhaustiveness check.
Providers are now fully self-contained — each imports its capabilities directly.

* perf+refactor: type-env caching, sequential fallback caching, utils.ts split

Phase 3 — performance optimizations and barrel cleanup:

Type-env parent-walk caching:
- Memoize findEnclosingClassName and findEnclosingParentClassName with
  per-file Map<SyntaxNode, string|undefined> caches
- Eliminates O(n*m) repeated child scanning in extractParentClassFromNode
- Caches cleared in buildTypeEnv before each file's walk phase

Sequential fallback caching:
- Add classIdCache + exportCache Maps to parsing-processor.ts
- Mirrors the O(1) memoization pattern from parse-worker.ts
- Both paths now have identical caching for parent-chain walks

Split utils.ts barrel into focused modules:
- noise-filter.ts: BUILT_IN_NAMES + isBuiltInOrNoise (167 LOC)
- language-detection.ts: getLanguageFromFilename (58 LOC)
- utils.ts slimmed to re-exports + yieldToEventLoop + isVerboseIngestionEnabled
- Backward compatible — existing imports from utils.ts still work

* refactor: rename resolvers/ → import-resolvers/, restructure tests per-concern

Directory renames (git mv — history preserved):
- src/core/ingestion/resolvers/ → import-resolvers/ (10 files)
- test/unit/call-routing.test.ts → call-routing/ruby.test.ts
- test/unit/named-binding-extraction.test.ts → named-bindings/csharp.test.ts
- test/unit/import-resolution.test.ts → import-resolution/preprocessing.test.ts

All 11 import paths updated to reference new import-resolvers/ location.
Test imports updated for new subdirectory depth.

Note: test/integration/resolvers/ NOT renamed — those tests cover the full
ingestion pipeline per-language, not just import resolution.

* refactor: eliminate utils.ts barrel — all 33 consumers now import directly

Migrated 65 import sites across 33 files to import from the focused source
module instead of the utils.ts barrel:

- ast-helpers.js: SyntaxNode, extractFunctionName, findEnclosingClassId, etc.
- call-analysis.js: inferCallForm, extractReceiverName, countCallArguments, etc.
- noise-filter.js: BUILT_IN_NAMES, isBuiltInOrNoise
- language-detection.js: getLanguageFromFilename

utils.ts reduced to 2 original functions only:
- yieldToEventLoop
- isVerboseIngestionEnabled

Zero re-exports remain. Every import is now direct to its source module.

* refactor: create utils/ folder, move all shared utilities, delete utils.ts barrel

Final phase of module structure migration:

- git mv ast-helpers.ts, call-analysis.ts, noise-filter.ts,
  language-detection.ts → utils/ subdirectory (history preserved)
- Extract yieldToEventLoop → utils/event-loop.ts
- Extract isVerboseIngestionEnabled → utils/verbose.ts
- Delete utils.ts (zero re-exports, zero functions remain)
- Update 38 import paths across source and test files

The ingestion/ root is now clean — only processors, capability modules,
and the pipeline orchestrator live at the top level. All shared utilities
are in utils/, all language-specific helpers in helpers/, all import
resolvers in import-resolvers/.

* refactor: move findChild from import-resolvers/utils.ts to utils/ast-helpers.ts

findChild is a generic AST helper (find first named child by type) — it
belongs with the other AST traversal utilities, not in the import resolver
module. 4 consumers updated to import from utils/ast-helpers.js.

* refactor: split named-binding-extraction.ts into per-language files

Rename named-binding-extraction.ts → named-binding-processor.ts (git mv,
history preserved), keeping only walkBindingChain for re-export chain resolution.

7 per-language extractor functions moved to named-bindings/ subdirectory:
- named-bindings/typescript.ts (extractTsNamedBindings — TS + JS)
- named-bindings/python.ts (extractPythonNamedBindings)
- named-bindings/kotlin.ts (extractKotlinNamedBindings)
- named-bindings/rust.ts (extractRustNamedBindings + collectRustBindings)
- named-bindings/php.ts (extractPhpNamedBindings)
- named-bindings/csharp.ts (extractCsharpNamedBindings)
- named-bindings/java.ts (extractJavaNamedBindings)

Each provider now imports its binding extractor from the per-language file.

* refactor: eliminate import-resolution.ts — distribute to natural homes

Split per-language resolvers into import-resolvers/ per-language files and
eliminate the import-resolution.ts catch-all module entirely:

Per-language resolvers moved to import-resolvers/:
- standard.ts: resolveStandard, resolveJavascriptImport, resolveTypescriptImport,
  resolveCImport, resolveCppImport
- jvm.ts: resolveJavaImport, resolveKotlinImport
- go.ts: resolveGoImport
- csharp.ts: resolveCSharpImport (helper renamed to Internal)
- php.ts, python.ts, ruby.ts, rust.ts: same pattern
- swift.ts: new file for resolveSwiftImport

Types distributed to their concern directories:
- import-resolvers/types.ts: ImportResult, ImportConfigs, ResolveCtx, ImportResolverFn
- named-bindings/types.ts: NamedBinding, NamedBindingExtractorFn

preprocessImportPath moved to import-processor.ts (its primary consumer).

import-resolution.ts deleted — zero catch-all modules remain.

* refactor: tighten SPR — eliminate re-exports, dead code, type holes, and redundant patterns

12 review findings resolved across the ingestion layer:

Type safety:
- CallRouter callNode: any → SyntaxNode (closes type hole)
- CaptureMap type alias replaces Record<string, any>
- providersWithImplicitWiring filter now type-narrowed (removes ! assertions)
- Ruby exportChecker: unnecessary as-cast removed, named export created

Architecture:
- Circular type dependency eliminated (ImportResolutionContext moved to types.ts)
- LANGUAGE_QUERIES residual dispatch replaced with provider.treeSitterQueries
- noRouting sentinel deleted — callRouter now properly optional on 12 providers
- All 6 re-exports from import-processor/pipeline/languages eliminated

Pattern cleanup:
- Dead checkersByLanguage table + isNodeExported removed from export-detection
- 4 duplicated config interfaces consolidated to language-config.ts
- extractCsharpNamedBindings → extractCSharpNamedBindings (casing consistency)

Simplification:
- import-resolvers/index.ts barrel deleted (dead re-exports)
- helpers/ inlined into languages/ (php.ts, swift.ts) — 1 directory removed

Verified: tsc --noEmit clean, 3837 tests pass, 0 failures.

* refactor: address review — remove LANGUAGE_QUERIES table, type-extractors barrel, fix Windows timeout

Review comment fixes (github.com/abhigyanpatwari/GitNexus/pull/488#issuecomment-4117817648):

1. LANGUAGE_QUERIES dispatch table removed from tree-sitter-queries.ts
   — 5 test files migrated to getProvider(lang).treeSitterQueries
   — eliminates last parallel dispatch surface

2. type-extractors/index.ts barrel deleted
   — type-env.ts now imports TYPED_PARAMETER_TYPES from shared.js directly

3. Windows CI timeout fix: afterAll cleanup hook in test-indexed-db.ts
   now passes explicit 120s timeout to prevent KuzuDB C++ destructor
   hang from hitting vitest's default 30s testTimeout on Windows

Verified: tsc --noEmit clean, 3835 tests pass, 0 failures.

* refactor: eliminate chained getProvider property access — assign to variable first

All getProvider(lang).property calls now follow the pattern:
  const provider = getProvider(language);
  const x = provider.property;

5 source files + 4 test files updated (~35 occurrences).
This ensures consistent provider variable usage and avoids
repeated lookups in hot paths.

* refactor: remove last 4 re-exports from import-resolvers, fix stale CaptureMap comment

- Remove `export type { TsconfigPaths }` from standard.ts
- Remove `export type { GoModuleConfig }` from go.ts
- Remove `export type { ComposerConfig }` from php.ts
- Remove `export type { CSharpProjectConfig }` from csharp.ts
  All 4 types are canonically defined in language-config.ts;
  zero consumers imported via the resolver re-exports.

- Fix stale CaptureMap JSDoc: said "Uses any" but type is SyntaxNode | undefined
2026-03-24 13:42:39 +00:00

150 lines
7.1 KiB
TypeScript

/**
* Unit tests for import-resolution.ts
*
* Coverage notes:
* - `preprocessImportPath` is tested directly below (no tree-sitter required for most paths).
* - Rust scoped grouped import logic (`resolveRustImportDispatch`) requires a live file system
* and ResolveCtx — that path is covered by test/integration/resolvers/rust.test.ts.
* - PHP `use function` / `use const` filtering (via `extractPhpNamedBindings`) requires
* tree-sitter PHP nodes — covered by test/integration/resolvers/php.test.ts.
*/
import { describe, it, expect } from 'vitest';
import { preprocessImportPath } from '../../../src/core/ingestion/import-processor.js';
import { getProvider } from '../../../src/core/ingestion/languages/index.js';
import { SupportedLanguages } from '../../../src/config/supported-languages.js';
// ---------------------------------------------------------------------------
// Minimal SyntaxNode stub — only the fields preprocessImportPath touches.
// For non-Kotlin languages preprocessImportPath never reads the node, so an
// empty stub satisfies the type requirement without loading tree-sitter.
// ---------------------------------------------------------------------------
function makeNode(overrides: Partial<{ childCount: number; child: (i: number) => any }> = {}): any {
return {
childCount: overrides.childCount ?? 0,
child: overrides.child ?? (() => null),
};
}
// ---------------------------------------------------------------------------
// preprocessImportPath — universal cleaning behaviour
// ---------------------------------------------------------------------------
describe('preprocessImportPath', () => {
describe('quote and bracket stripping', () => {
it('strips double quotes from a bare module path', () => {
const node = makeNode();
expect(preprocessImportPath('"foo"', node, getProvider(SupportedLanguages.TypeScript))).toBe('foo');
});
it('strips single quotes from a bare module path', () => {
const node = makeNode();
expect(preprocessImportPath("'bar/baz'", node, getProvider(SupportedLanguages.JavaScript))).toBe('bar/baz');
});
it('strips angle brackets from a C-style include path', () => {
const node = makeNode();
expect(preprocessImportPath('<stdio.h>', node, getProvider(SupportedLanguages.C))).toBe('stdio.h');
});
it('strips mixed quote and angle bracket characters', () => {
const node = makeNode();
// Pathological input — all stripped characters removed
expect(preprocessImportPath('"<hello>"', node, getProvider(SupportedLanguages.TypeScript))).toBe('hello');
});
});
describe('null returns for invalid inputs', () => {
it('returns null for an empty string (after cleaning)', () => {
const node = makeNode();
// Only quote characters — cleaned result is empty string
expect(preprocessImportPath('""', node, getProvider(SupportedLanguages.TypeScript))).toBeNull();
});
it('returns null for a string containing control characters', () => {
const node = makeNode();
// \x01 is a control character that passes the length check but fails the regex guard
expect(preprocessImportPath('foo\x01bar', node, getProvider(SupportedLanguages.Rust))).toBeNull();
});
it('returns null for a string containing a null byte', () => {
const node = makeNode();
expect(preprocessImportPath('foo\x00bar', node, getProvider(SupportedLanguages.Go))).toBeNull();
});
it('returns null for a path exceeding 2048 characters', () => {
const node = makeNode();
const longPath = 'a'.repeat(2049);
expect(preprocessImportPath(longPath, node, getProvider(SupportedLanguages.Python))).toBeNull();
});
it('accepts a path of exactly 2048 characters', () => {
const node = makeNode();
const maxPath = 'a'.repeat(2048);
expect(preprocessImportPath(maxPath, node, getProvider(SupportedLanguages.Python))).toBe(maxPath);
});
});
describe('Kotlin wildcard pass-through', () => {
it('delegates to appendKotlinWildcard when language is Kotlin — no wildcard child', () => {
// Node with no children -> appendKotlinWildcard returns the path unchanged
const node = makeNode({ childCount: 0 });
const result = preprocessImportPath('com.example.models', node, getProvider(SupportedLanguages.Kotlin));
// Without a wildcard_import child the path is returned as-is
expect(result).toBe('com.example.models');
});
it('delegates to appendKotlinWildcard when language is Kotlin — wildcard_import child present', () => {
// Simulate a node that has a wildcard_import child at index 0
const wildcardChild = { type: 'wildcard_import' };
const node = makeNode({
childCount: 1,
child: (i: number) => (i === 0 ? wildcardChild : null),
});
const result = preprocessImportPath('com.example.models', node, getProvider(SupportedLanguages.Kotlin));
// appendKotlinWildcard appends .* when the wildcard_import child is found
expect(result).toBe('com.example.models.*');
});
});
describe('non-Kotlin languages are returned unchanged (after cleaning)', () => {
it('returns the cleaned path for Rust without modification', () => {
const node = makeNode();
expect(preprocessImportPath('"crate::models"', node, getProvider(SupportedLanguages.Rust))).toBe('crate::models');
});
it('returns the cleaned path for PHP without modification', () => {
const node = makeNode();
expect(preprocessImportPath('"App\\\\Models\\\\User"', node, getProvider(SupportedLanguages.PHP))).toBe('App\\\\Models\\\\User');
});
});
});
// ---------------------------------------------------------------------------
// Rust scoped grouped import logic (resolveRustImportDispatch)
// ---------------------------------------------------------------------------
// The dispatch function requires a live ResolveCtx with file lists — unit
// testing it without a file system would duplicate the integration fixtures.
// The following comment documents what the integration tests verify:
//
// test/integration/resolvers/rust.test.ts covers:
// - Top-level grouped: use {crate::a, crate::b}
// - Scoped grouped: use crate::models::{User, Repo}
// - Alias stripping: use crate::models::{User, Repo as R} -> resolves User + Repo
// - Prefix fallback: when no individual items resolve, resolves the prefix path
//
// The ::{ detection and alias-stripping logic lives in resolveRustImportDispatch()
// at import-resolution.ts lines 328-344.
// ---------------------------------------------------------------------------
// PHP use function / use const filtering (extractPhpNamedBindings)
// ---------------------------------------------------------------------------
// extractPhpNamedBindings requires live tree-sitter PHP SyntaxNode objects.
// The filtering of `use function` and `use const` declarations is covered by:
//
// test/integration/resolvers/php.test.ts
//
// which runs the full ingestion pipeline over PHP fixture repositories and
// asserts that function/const use-declarations do not produce spurious IMPORTS
// edges to non-existent class files.