diff --git a/gitnexus/src/core/ingestion/type-extractors/TYPE-RESOLUTION.md b/gitnexus/src/core/ingestion/type-extractors/TYPE-RESOLUTION.md deleted file mode 100644 index 86ae7fb2b..000000000 --- a/gitnexus/src/core/ingestion/type-extractors/TYPE-RESOLUTION.md +++ /dev/null @@ -1,303 +0,0 @@ -# Type Resolution System - -GitNexus's type resolution system maps variables to their declared types across 12 languages, enabling receiver-constrained call resolution. When code calls `user.save()`, the resolver needs to know that `user` is of type `User` to link the call to `User#save` rather than `Repo#save`. - -The system is designed to be **conservative** (no false bindings), **single-pass** (no fixpoint iteration), and **per-file** (no cross-file type inference at this layer). - -## Architecture - -``` - ┌──────────────────────┐ - │ type-env.ts │ - │ │ - │ buildTypeEnv() │ - │ - Single AST walk │ - │ - Scope tracking │ - │ - Tier orchestration │ - └──────────┬───────────┘ - │ dispatches to - ┌───────────────────────┬┴┬───────────────────────┐ - │ │ │ │ - ┌─────────▼──────────┐ ┌─────────▼─▼────────┐ ┌──────────▼─────────┐ - │ shared.ts │ │ .ts │ │ types.ts │ - │ │ │ │ │ │ - │ Container table │ │ Per-language │ │ Interface defs │ - │ Type extractors │ │ extractors │ │ for all extractor │ - │ Generic helpers │ │ (12 files) │ │ function types │ - └────────────────────┘ └──────────────────────┘ └────────────────────┘ -``` - -### Files - -| File | Purpose | -|------|---------| -| `type-env.ts` | Core engine. Single-pass AST walker that orchestrates all tiers. Exports `buildTypeEnv()` and the `TypeEnvironment` interface. | -| `types.ts` | TypeScript interfaces for all extractor function signatures (`TypeBindingExtractor`, `ForLoopExtractor`, `PatternBindingExtractor`, etc.). | -| `shared.ts` | Language-agnostic helpers: `extractSimpleTypeName`, `extractElementTypeFromString`, `resolveIterableElementType`, `CONTAINER_DESCRIPTORS`, `TYPED_PARAMETER_TYPES`. | -| `index.ts` | Dispatch map from `SupportedLanguages` to `LanguageTypeConfig` objects. | -| `typescript.ts` | TypeScript/JavaScript extractors (shared config). Includes JSDoc parsing. | -| `jvm.ts` | Java + Kotlin extractors (separate configs, shared file). | -| `csharp.ts` | C# extractors. | -| `go.ts` | Go extractors. Handles range clause semantics (channel vs slice). | -| `rust.ts` | Rust extractors. Handles `if let`, match arms, `Self` resolution. | -| `python.ts` | Python extractors. Handles `match`/`case` with `as` patterns. | -| `php.ts` | PHP extractors. Includes PHPDoc parsing. | -| `ruby.ts` | Ruby extractors. Includes YARD annotation parsing. | -| `swift.ts` | Swift extractors. Most minimal — no for-loop or pattern binding support yet. | -| `c-cpp.ts` | C/C++ extractors (shared config). Handles structured bindings and templates. | - -## Resolution Tiers - -The system resolves variable types through a priority-ordered cascade. Each tier runs during the same single AST walk; higher tiers only activate when lower tiers produce no binding. - -### Tier 0: Explicit Type Annotations - -Direct extraction from AST type nodes. This is the highest-confidence tier. - -```typescript -// TypeScript -const user: User = getUser(); // user → User - -// Java -User user = getUser(); // user → User - -// Go -var user User // user → User - -// Rust -let user: User = get_user(); // user → User - -// Python -user: User = get_user() // user → User -``` - -**How it works:** `extractDeclaration` reads the `type` field from declaration AST nodes and calls `extractSimpleTypeName` to normalize it (unwrapping generics, nullable wrappers, qualified names). - -**Parameters** are handled separately via `extractParameter`, using the same `extractSimpleTypeName` logic on function parameter type annotations. The shared `TYPED_PARAMETER_TYPES` set gates which AST node types trigger parameter extraction. - -### Tier 0b: For-Loop Element Type Resolution - -For-each loops with implicit element types (e.g., `for (var user in users)`) resolve the loop variable's type from the iterable's container type. - -```csharp -// C#: var foreach -foreach (var user in users) { user.Save(); } // user → User (from List) - -// TypeScript: for-of -for (const user of users) { user.save(); } // user → User (from User[]) - -// Rust: for-in -for user in users.iter() { user.save(); } // user → User (from Vec) -``` - -**Three-strategy cascade** (in `resolveIterableElementType`): - -1. **declarationTypeNodes** — Raw AST type annotation node. Handles container types where `extractSimpleTypeName` returned `undefined` (e.g., `User[]`, `List[User]`). Falls back to file scope when the iterable is a class field. -2. **scopeEnv string** — `extractElementTypeFromString` on the stored type string. Uses bracket-balanced parsing (no regex) for generic argument extraction. -3. **AST walk** — Language-specific upward walk to enclosing function parameters to read type annotations directly. - -**Container descriptors** (`CONTAINER_DESCRIPTORS` in `shared.ts`) map container type names to their type parameter semantics. For example, `Map` has arity 2 with `.keys()` yielding the first type arg and `.values()` yielding the last. This enables: - -```typescript -for (const key of map.keys()) { ... } // key → string (first type arg) -for (const val of map.values()) { ... } // val → User (last type arg) -``` - -### Tier 0c: Pattern Binding - -Pattern matching constructs that introduce new typed variables. - -```csharp -// C# is-pattern -if (obj is User user) { user.Save(); } // user → User - -// C# recursive_pattern -if (obj is User { Name: "Alice" } u) { u.Save(); } // u → User - -// Java instanceof -if (obj instanceof User user) { user.save(); } // user → User - -// Kotlin when/is (with position-indexed overrides) -when (obj) { - is User -> obj.save() // obj → User (within this branch only) - is Repo -> obj.archive() // obj → Repo (within this branch only) -} - -// Rust if-let -if let Some(user) = opt { user.save(); } // user → User -if let Ok(user) = result { user.save(); } // user → User - -// TypeScript instanceof -if (x instanceof User) { x.save(); } // x → User - -// Python match/case as -match obj: - case User() as user: user.save() // user → User -``` - -**Binding semantics:** -- **First-writer-wins** (default): The first pattern binding for a variable name sticks. Used by Java, C#, TypeScript, Rust, Python. -- **Position-indexed overwrite** (Kotlin only): Each branch gets its own type for the same variable, tracked by AST position ranges. Prevents cross-arm contamination. - -### Tier 1: Constructor Inference - -When no explicit type annotation exists, infer from constructor calls. - -```typescript -// TypeScript -const user = new User(); // user → User (via extractInitializer) - -// C# -var user = new User(); // user → User - -// Java -User user = new User(); // already Tier 0, but: -var user = new UserService(); // user → UserService (Tier 1) - -// Kotlin -val user = User() // user → User (needs SymbolTable to confirm User is a class) - -// Rust -let user = User::new(); // user → User - -// C++ -auto user = User(); // user → User (needs classNames lookup) - -// Ruby -user = User.new // user → User (via extractRubyConstructorAssignment) -``` - -**Cross-file verification:** Some languages (Kotlin, C++) can't distinguish `User()` from `getUser()` syntactically. The `scanConstructorBinding` scanner collects unverified `{varName, calleeName}` pairs. These are later verified against the `SymbolTable` — if the callee name matches a known class/struct, the binding is accepted. - -### Tier 2: Assignment Chain Propagation - -Single-pass propagation of type bindings through plain-identifier assignments. - -```typescript -const user: User = getUser(); // user → User (Tier 0) -const alias = user; // alias → User (Tier 2: propagated from user) -const b = alias; // b → User (Tier 2: multi-hop, if forward-declared) -``` - -**How it works:** During the AST walk, `extractPendingAssignment` collects `{lhs, rhs}` pairs for declarations where the LHS has no type and the RHS is a bare identifier. After the walk completes, a single pass resolves each pending assignment by looking up the RHS in `scopeEnv` (or file scope). - -**Limitations:** Forward-order only. `const b = a; const a: User = ...` won't resolve `b`. No fixpoint iteration — single pass covers 95%+ of real-world patterns. - -## Scope Model - -The type environment is scope-aware to prevent variable name collisions across functions. - -``` -File scope ('') -├── config → Config -├── users → Map (class field) -│ -├── processUsers@100 -│ ├── user → User (from for-loop) -│ └── alias → User (from assignment chain) -│ -└── processRepos@200 - └── repo → Repo (from for-loop) -``` - -**Scope keys:** `functionName@startIndex` for function-local scopes, `''` for file-level scope. - -**Lookup order** (in `TypeEnvironment.lookup`): -1. Position-indexed pattern overrides (Kotlin when/is) -2. Function-local scope (`processUsers@100`) -3. File-level scope (`''`) -4. Special receivers: `this`/`self`/`$this` → enclosing class name via AST walk; `super`/`base`/`parent` → parent class via heritage node - -## Language Feature Matrix - -| Feature | TS/JS | Java | Kotlin | C# | Go | Rust | Python | PHP | Ruby | Swift | C/C++ | -|---------|:-----:|:----:|:------:|:--:|:--:|:----:|:------:|:---:|:----:|:-----:|:-----:| -| Declarations | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | -| Parameters | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | -| Constructor inference | Yes | Yes | Yes | -- | -- | Yes | Yes | Yes | Yes | Yes | Yes | -| Constructor binding scan | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | -| For-loop element types | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | -- | Yes | -| Pattern binding | Yes | Yes | Yes | Yes | -- | Yes | Yes | -- | -- | -- | -- | -| Assignment chains | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | -- | Yes | -| Comment-based types | JSDoc | -- | -- | -- | -- | -- | -- | PHPDoc | YARD | -- | -- | -| Return type extraction | JSDoc | -- | -- | -- | -- | -- | -- | PHPDoc | YARD | -- | -- | - -## Container Type Descriptors - -The `CONTAINER_DESCRIPTORS` table in `shared.ts` maps container base type names to their type parameter semantics. This drives correct element type extraction from generic containers during for-loop resolution. - -**Arity 2 (key-value):** -`Map`, `WeakMap`, `HashMap`, `BTreeMap`, `LinkedHashMap`, `TreeMap`, `dict`, `Dict`, `Dictionary`, `SortedDictionary`, `Record`, `OrderedDict`, `ConcurrentHashMap`, `ConcurrentDictionary`, `MutableMap` - -**Arity 1 (single-element):** -`Array`, `List`, `ArrayList`, `LinkedList`, `Vec`, `VecDeque`, `Set`, `HashSet`, `BTreeSet`, `TreeSet`, `Queue`, `Deque`, `Stack`, `Sequence`, `Iterable`, `Iterator`, `IEnumerable`, `IList`, `ICollection`, `Collection`, `ObservableCollection`, `IEnumerator`, `SortedSet`, `Stream`, `MutableList`, `MutableSet`, `LinkedHashSet`, `ArrayDeque`, `PriorityQueue`, `list`, `set`, `tuple`, `frozenset` - -Each descriptor specifies which methods yield the key type (`.keys()`, `.keySet()`, `.Keys`) vs the value type (`.values()`, `.get()`, `.Values`). Unknown containers fall back to a method-name heuristic. - -## How It Integrates with the Pipeline - -``` - parse-worker.ts - │ - ▼ - buildTypeEnv(tree, language, symbolTable?) - │ - ├──► TypeEnvironment.lookup(varName, callNode) - │ │ - │ ▼ - │ call-processor.ts - │ - Resolves receiver type for method calls - │ - Filters candidate targets by receiver match - │ - Uses constructorBindings for cross-file inference - │ - └──► discarded after file processing -``` - -The `TypeEnvironment` is built once per file during the ingestion pipeline's call-resolution phase. The `call-processor` uses `lookup()` to determine the receiver type for each method call expression, then filters the candidate symbols from the `SymbolTable` to find the correct target. - -## Roadmap - -### Phase 7: Cross-Scope Type Propagation - -Three deferred gaps share the same root blocker — the `ForLoopExtractor` interface only receives the current method's `scopeEnv`, not the full `TypeEnvironment`. - -**7A. Go `call_expression` as range iterable** -`for _, user := range getUsers()` — the iterable is a function call, not a variable. Requires passing `returnTypeMap` to `extractForLoopBinding` so it can look up `getUsers → []User`. Touches the `ForLoopExtractor` interface which all 10 language extractors implement. - -**7B. PHP `@var` class property scope propagation** -`foreach ($this->users as $user)` only works when `$users` type is in the method's scope (via `@param`). Class property `@var` annotations are stored at file scope, but `extractForLoopBinding` only queries function scope. Same infrastructure as 7A — extractors need access to the full `TypeEnvironment`. - -**7C. Rust `struct_pattern` in match arms** -`match user { User { name, email } => ... }` — `struct_pattern` destructures named fields, but field-level type info isn't available without field resolution infrastructure. Can bind the overall variable (via `@` pattern) but not individual destructured fields. - -**Approach:** Extend the `ForLoopExtractor` type signature to accept the full `TypeEnvironment` (or at minimum, both scope-level and file-level env maps). This is a coordinated change across all 10 language extractors. - -### Phase 8: Field-Type Resolution - -Currently, the system resolves variable types but not field access chains. `user.address.city` can resolve `user → User` but cannot resolve `address → Address` without field-type information from the class definition. - -**Scope:** -- Parse class/struct field declarations to build a field type map per class -- Enable chained member access resolution: `user.address.city` → resolve each segment -- Required for: PHP chained property access (`$this->property->method()`), Rust struct field destructuring, TypeScript deep property access - -### Phase 9: Return-Type-Aware Resolution - -The `scanConstructorBinding` mechanism currently only handles `var x = CalleeName()` patterns where the callee is a class constructor. Extending this to full return-type inference would enable: - -- `var users = repo.getUsers()` → `users: List` (from `getUsers` return type) -- For-loop over function call results: `for user in getUsers()` (Phase 7A prerequisite) -- Method chain inference: `repo.getUsers().first()` → User - -**Approach:** The call-processor already extracts return types from function signatures. Feed this information back into `TypeEnvironment` as a `returnTypeMap` for cross-reference during type resolution. - -### Gaps by Language - -| Language | Missing | Phase | -|----------|---------|-------| -| Swift | For-loop binding, pattern binding, assignment chains | 7+ | -| Go | Call expression as range iterable | 7A | -| PHP | `@var` scope propagation, chained property access | 7B, 8 | -| Rust | Struct pattern destructuring | 7C | -| All | Field-type resolution | 8 | -| All | Return-type-aware variable binding | 9 | diff --git a/type-resolution-roadmap.md b/type-resolution-roadmap.md new file mode 100644 index 000000000..0398a4ce9 --- /dev/null +++ b/type-resolution-roadmap.md @@ -0,0 +1,408 @@ +# Type Resolution Roadmap + +This roadmap describes the next major capabilities needed to evolve GitNexus's type-resolution layer from a strong receiver-disambiguation aid into a broader static-analysis foundation. + +The roadmap assumes the current system already provides: + +- explicit type extraction from declarations and parameters +- initializer / constructor inference +- loop element inference for many languages +- selected pattern binding and narrowing +- comment-based fallbacks in JS/TS, PHP, and Ruby +- constrained return-type-aware receiver inference during call processing + +The remaining work is about **generalisation**, **deeper structure modelling**, and **better propagation**. + +--- + +## Principles for Future Work + +The type system should continue to preserve the qualities that make it practical today: + +- **stay conservative** +- **prefer explainable inference over clever but brittle inference** +- **limit performance overhead during ingestion** +- **keep per-language extractors explicit rather than over-generic** +- **separate "better receiver resolution" from "compiler-grade typing"** + +The goal is not to build a compiler. The goal is to support high-value static analysis for call graphs, impact analysis, context gathering, and downstream graph features. + +--- + +## Near-Term Priority: Generalise Existing Inference + +The next biggest gain is not inventing a new type system layer. It is expanding the inference the system already performs so more constructs can benefit from it. + +### Why this is the right next step + +Today, return-type-aware inference already exists in constrained form inside `call-processor.ts`, and loop element inference already handles many identifier-based iterables. + +The most valuable next move is to let those signals participate in more places, especially: + +- iterable expressions rather than only iterable identifiers +- assignment propagation from call results +- doc-comment-derived file-scope bindings where local scope is insufficient + +--- + +## Phase 7: Cross-Scope and Return-Aware Propagation + +### Goal + +Allow loop inference and assignment inference to see more than the current function-local environment. + +### Problems this phase addresses + +#### 7A. Iterable expressions in Go and similar cases + +```go +for _, user := range getUsers() { + user.Save() +} +``` + +The iterable is a call expression, not an identifier with a local binding. + +To resolve `user`, the loop extractor needs access to a return-type source for `getUsers()`. + +#### 7B. File-scope or class-scope iterable typing in PHP + +```php +foreach ($this->users as $user) { + $user->save(); +} +``` + +If `$this->users` is typed through a class property annotation or file/class-scope doc-comment information, the current local-scope-only path may not be enough. + +#### 7C. Broader use of already-known return types + +The system can already infer receiver types from uniquely resolved call results in `call-processor.ts`. That needs to be generalised so `TypeEnv` can benefit from it too. + +### Engineering direction + +- extend loop and propagation extractors so they can access more than the current local scope +- expose file-scope string bindings where needed +- introduce a shared `returnTypeMap` or equivalent lookup mechanism +- keep the interface change coordinated across extractors to avoid partial semantics by language + +### Expected impact + +This phase should unlock: + +- loop inference for iterable-producing call expressions +- broader propagation from method / function return types +- fewer missed bindings in real-world code that avoids explicit variable annotations + +### Risk level + +**Medium** + +This work touches extractor interfaces across multiple languages, so the coordination cost is real. However, the conceptual model is an extension of existing behavior rather than a new analysis paradigm. + +--- + +## Phase 8: Field and Property Type Resolution + +### Goal + +Model class / struct fields so chained member access can be resolved more accurately. + +### Problems this phase addresses + +#### 8A. Deep property chains + +```typescript +user.address.city +``` + +Today the system may resolve `user -> User`, but it cannot generally resolve: + +- `address -> Address` +- `city -> City` or scalar type + +#### 8B. Chained method targets through field access + +```typescript +user.address.save() +``` + +Without field typing, the resolver cannot reliably identify the receiver type of `address`. + +#### 8C. Pattern destructuring that depends on field knowledge + +This is especially relevant for: + +- Rust struct-pattern destructuring +- PHP chained property access +- richer TypeScript or Python object-based destructuring in future work + +### Engineering direction + +- parse field / property declarations per class or struct +- build a field-type map keyed by owning type +- teach lookup and chain-resolution logic to walk member segments +- keep this separate from the base variable-binding layer where possible + +### Expected impact + +This is the biggest unlock for richer static analysis because it allows the graph to model more than just top-level receivers. + +It would materially improve: + +- chained property resolution +- member-based call disambiguation +- deeper context extraction for downstream tooling + +### Risk level + +**High** + +This is the first phase that pushes the system from variable typing into structural object modelling. It will likely require: + +- schema expansion or new internal maps +- careful handling of inheritance / embedding / language-specific member semantics +- broader test coverage than earlier phases + +--- + +## Phase 9: Full Return-Type-Aware Variable Binding + +### Goal + +Make return-type-driven inference a first-class input to `TypeEnv`, not just a downstream verification path. + +### Problems this phase addresses + +#### 9A. Binding variables from call results + +```typescript +const users = repo.getUsers() +``` + +Desired binding: + +- `users -> List` + +#### 9B. Looping directly over call results + +```typescript +for (const user of getUsers()) { + user.save() +} +``` + +Desired binding: + +- `user -> User` + +#### 9C. Broader method-chain inference + +```typescript +repo.getUsers().first() +``` + +If return types can propagate more systematically, later chain stages become much more resolvable. + +### Engineering direction + +- expose return types as reusable inference inputs inside `TypeEnv` +- distinguish raw textual return types from normalized receiver-usable types +- make method-call return inference receiver-aware where necessary +- avoid over-eager propagation when multiple call targets remain ambiguous + +### Expected impact + +This phase would make the type system feel much closer to a static-analysis substrate rather than a set of local heuristics. + +It will especially improve codebases that rely heavily on: + +- service-returned collections +- builder APIs +- repository methods +- chain-heavy fluent interfaces + +### Risk level + +**Medium to High** + +The conceptual basis already exists, but generalising it without introducing false bindings requires careful ambiguity rules. + +--- + +## Language-Specific Gaps + +### Swift + +Current support remains relatively minimal. + +Missing or weak areas include: + +- for-loop element binding +- pattern binding +- assignment-chain propagation +- broader expression-based inference + +**Priority:** Medium +**Reason:** It matters for parity, but the biggest global analysis gains are elsewhere. + +### Go + +Key remaining gap: + +- iterable call expressions in range loops + +**Priority:** High +**Reason:** Go codebases frequently rely on return-value-based iteration patterns. + +### PHP + +Key remaining gaps: + +- file/class-scope iterable propagation +- chained property access + +**Priority:** High +**Reason:** PHP heavily benefits from doc-comment-aware field and property modelling. + +### Rust + +Key remaining gap: + +- struct-pattern field destructuring + +**Priority:** Medium +**Reason:** Important for completeness, but field-type infrastructure is the real prerequisite. + +### All languages + +Shared missing capabilities: + +- field / property type resolution +- generalised return-type-aware binding in `TypeEnv` + +**Priority:** Very High +**Reason:** These are the biggest remaining blockers to deeper static analysis. + +--- + +## Recommended Delivery Order + +### 1. Generalise existing return and loop inference + +This is the best cost-to-value step. + +Deliverables: + +- iterable call-expression support +- wider access to return-type maps +- file-scope binding visibility where needed + +### 2. Add field / property type maps + +This unlocks the next class of analysis depth. + +Deliverables: + +- per-type field metadata +- chained property resolution +- better destructuring support + +### 3. Promote return types into first-class `TypeEnv` inputs + +This converts existing downstream validation into a broader inference capability. + +Deliverables: + +- call-result variable binding +- loop inference from call results +- broader chain propagation + +### 4. Broaden branch-sensitive narrowing where low-risk + +After the structural work lands, selective branch refinement becomes more valuable and easier to reason about. + +--- + +## What “Production-Grade Static Analysis” Means Here + +For GitNexus, production-grade does **not** mean replacing a language compiler. + +A realistic target is: + +- strong receiver-constrained call resolution across common language idioms +- reliable handling of typed loops, constructor-like initializers, and common patterns +- useful return-type propagation for service/repository style code +- enough field/property knowledge to support chained-member analysis +- conservative behavior under ambiguity +- predictable performance during indexing + +That would be sufficient for: + +- better call graphs +- more accurate impact analysis +- stronger context assembly for AI workflows +- more trustworthy graph traversal features + +--- + +## Suggested Milestone Definitions + +### Milestone A — Inference Expansion + +Success looks like: + +- loop inference works for identifier iterables and common call-expression iterables +- simple call-result assignments benefit from return types more broadly +- no major regression in ambiguity handling + +### Milestone B — Structural Member Typing + +Success looks like: + +- field/property maps exist for class-like types +- chained access can resolve at least one segment beyond the base receiver +- field-aware member-call resolution works in the most important languages + +### Milestone C — Static-Analysis Foundation + +Success looks like: + +- return-type-aware variable binding is a first-class part of environment construction +- chains, loops, and assignments share a coherent propagation model +- downstream graph features can rely on more than local receiver heuristics + +--- + +## Open Questions for Future Design + +These should be resolved before or during implementation of the later phases. + +1. **Where should field-type metadata live?** + In `TypeEnv`, in `SymbolTable`, or in a dedicated side structure? + +2. **How should ambiguity be represented?** + Is `undefined` sufficient, or do later phases need a richer "known ambiguous" state? + +3. **How much receiver context should return-type inference require?** + Some methods only become meaningful once the receiver type is already partially known. + +4. **How much branch sensitivity is worth the complexity?** + Some narrowing gives clear value; full control-flow typing likely does not. + +5. **Should field typing and chain typing be one phase or two?** + Keeping them separate may reduce risk and make regressions easier to isolate. + +--- + +## Summary + +The next stage of the type system should focus on **generalising what already works** before attempting compiler-like sophistication. + +The most important path is: + +1. extend return-type and iterable inference +2. add field/property type knowledge +3. promote return-type-aware inference into `TypeEnv` + +That path preserves the current strengths of the system while moving GitNexus materially closer to a robust, production-grade static-analysis foundation. diff --git a/type-resolution-system.md b/type-resolution-system.md new file mode 100644 index 000000000..e1450e510 --- /dev/null +++ b/type-resolution-system.md @@ -0,0 +1,430 @@ +# Type Resolution System + +GitNexus's type resolution system maps variables to likely declared types across the supported languages so the ingestion pipeline can perform **receiver-constrained call resolution**. + +When the code contains a call such as `user.save()`, the resolver tries to determine that `user` is a `User`, allowing call resolution to prefer `User#save` over unrelated methods such as `Repo#save`. + +This system is designed to be: + +- **Conservative** — it prefers missing a binding over introducing a misleading one +- **Single-pass** — bindings are collected during a single AST walk, with a limited post-pass for assignment propagation +- **Scope-aware** — function-local bindings are isolated from file-level bindings +- **Per-file** — the environment is built for one file at a time, though it may consult the global `SymbolTable` for validation in specific cases + +It is **not** a full compiler type checker. Its job is to recover enough type information to improve call-edge accuracy during ingestion. + +--- + +## Purpose in the Pipeline + +Type resolution sits between parsing and call resolution. + +```text +parse-worker.ts + │ + ▼ +buildTypeEnv(tree, language, symbolTable?) + │ + ├──► TypeEnvironment.lookup(varName, callNode) + │ │ + │ ▼ + │ call-processor.ts + │ - resolves receiver type for method calls + │ - filters candidates by receiver match + │ - verifies deferred constructor / initializer bindings + │ + └──► discarded after file processing +``` + +The `TypeEnvironment` is built once per file. `call-processor.ts` then uses `lookup()` to determine receiver types and narrow candidate symbols from the `SymbolTable`. + +--- + +## Architecture + +```text + ┌──────────────────────┐ + │ type-env.ts │ + │ │ + │ buildTypeEnv() │ + │ - Single AST walk │ + │ - Scope tracking │ + │ - Tier orchestration│ + └──────────┬───────────┘ + │ dispatches to + ┌───────────────────────┬┴┬────────────────────────┐ + │ │ │ │ + ┌─────────▼──────────┐ ┌─────────▼─▼─────────┐ ┌──────────▼─────────┐ + │ shared.ts │ │ .ts │ │ types.ts │ + │ │ │ │ │ │ + │ Container table │ │ Per-language │ │ Extractor │ + │ Type helpers │ │ extractors │ │ interface defs │ + │ Generic helpers │ │ (shared + per-lang) │ │ │ + └────────────────────┘ └──────────────────────┘ └────────────────────┘ +``` + +### Main files + +| File | Purpose | +|------|---------| +| `type-env.ts` | Core engine. Walks the AST once, tracks scopes, collects bindings, and exposes `buildTypeEnv()` plus the `TypeEnvironment` interface. | +| `types.ts` | TypeScript interfaces for extractor hooks such as `TypeBindingExtractor`, `ForLoopExtractor`, and `PatternBindingExtractor`. | +| `shared.ts` | Language-agnostic helpers such as `extractSimpleTypeName`, `extractElementTypeFromString`, `resolveIterableElementType`, `CONTAINER_DESCRIPTORS`, and `TYPED_PARAMETER_TYPES`. | +| `index.ts` | Dispatch map from `SupportedLanguages` to `LanguageTypeConfig`. | +| `typescript.ts` | TypeScript and JavaScript extractors, including JSDoc support. | +| `jvm.ts` | Java and Kotlin extractors. | +| `csharp.ts` | C# extractors. | +| `go.ts` | Go extractors, including range semantics. | +| `rust.ts` | Rust extractors, including `if let`, match-related handling, and `Self` resolution. | +| `python.ts` | Python extractors, including `match` / `case` handling. | +| `php.ts` | PHP extractors, including PHPDoc support. | +| `ruby.ts` | Ruby extractors, including YARD support. | +| `swift.ts` | Swift extractors. Currently the most minimal configuration. | +| `c-cpp.ts` | Shared C / C++ extractors. | + +--- + +## Supported Languages + +The current type-resolution layer supports **13 languages**: + +- TypeScript +- JavaScript +- Python +- Java +- Kotlin +- C# +- Go +- Rust +- PHP +- Ruby +- Swift +- C +- C++ + +Not all languages have the same level of coverage. Swift remains the most minimal. C and some C++ cases naturally benefit less from receiver typing than object-oriented languages. + +--- + +## Design Constraints + +The type resolution layer is intentionally narrower than a compiler-grade type system. + +It does: + +- resolve variable types from declarations, parameters, initializers, loops, and selected pattern constructs +- normalize common wrappers such as nullable types and generic containers +- improve receiver matching during call resolution +- verify some ambiguous initializer bindings against the `SymbolTable` + +It does not: + +- perform full semantic type checking +- run fixpoint inference +- propagate inferred bindings across files as ordinary environment entries +- model deep field/property chains such as `user.address.city` +- guarantee resolution for every ambiguous construct + +--- + +## TypeEnvironment Model + +`buildTypeEnv()` returns a `TypeEnvironment` that contains: + +- scoped bindings collected from the current file +- deferred constructor / initializer binding candidates +- lookup helpers used by call resolution +- pattern override data for branch-local narrowing where supported + +### Scope model + +The environment is scope-aware so identical variable names in different functions do not collide. + +```text +File scope ('') +├── config → Config +├── users → Map +│ +├── processUsers@100 +│ ├── user → User +│ └── alias → User +│ +└── processRepos@200 + └── repo → Repo +``` + +### Scope keys + +- `''` for file scope +- `functionName@startIndex` for function-local scope + +These scope keys are also used later when verifying deferred bindings in call processing, so any future change to scope-key format must stay consistent across both layers. + +--- + +## Lookup Semantics + +`TypeEnvironment.lookup()` resolves types in this effective order: + +1. special receivers + - `this`, `self`, `$this` → enclosing class + - `super`, `base`, `parent` → parent class +2. position-indexed pattern overrides +3. function-local scope +4. file-level scope + +Special receivers are handled as a dedicated fast path rather than ordinary lexical bindings. + +--- + +## Resolution Tiers + +Bindings are collected during the same AST walk. Higher-confidence sources win over weaker inference. + +### Tier 0: Explicit Type Annotations + +Direct extraction from AST type nodes. + +```typescript +// TypeScript +const user: User = getUser() + +// Java +User user = getUser() + +// Go +var user User + +// Rust +let user: User = get_user() + +// Python +user: User = get_user() +``` + +`extractDeclaration()` reads the declaration type node and normalizes it through `extractSimpleTypeName()`. + +Parameters are handled separately by `extractParameter()` using the same normalization logic. The shared `TYPED_PARAMETER_TYPES` set controls which AST node types are treated as typed parameters. + +### Tier 0b: For-Loop Element Type Resolution + +Also referred to as **Tier 1c** in Phase 6 PR and test naming. + +For-each style loops often introduce a variable with no explicit type. In those cases, the resolver derives the loop variable type from the iterable's container type. + +```csharp +foreach (var user in users) { user.Save(); } + +// TypeScript +for (const user of users) { user.save(); } + +// Rust +for user in users { user.save(); } +``` + +This is handled by `resolveIterableElementType()` through a three-step cascade: + +1. **Declaration type nodes** + Uses raw type annotation nodes when available, including cases such as `User[]` or `List[User]`. + +2. **Scope environment string** + Uses `extractElementTypeFromString()` to parse a stored type string. + +3. **AST walk fallback** + Walks upward to enclosing declarations or parameters when needed. + +### Tier 0c: Pattern Binding + +Pattern-matching constructs may introduce a new variable or temporarily narrow an existing one. + +```csharp +if (obj is User user) { user.Save(); } + +// Java +if (obj instanceof User user) { user.save(); } + +// Rust +if let Some(user) = opt { user.save(); } + +// Python +match obj: + case User() as user: + user.save() +``` + +Binding behavior depends on the language: + +- **first-writer-wins** is used by default +- **position-indexed branch overrides** are used where branch-local narrowing must not leak between branches, most notably Kotlin + +### Tier 1: Initializer / Constructor Inference + +When there is no explicit annotation, the resolver can infer a type from the initializer. + +```typescript +const user = new User() + +// C# +var user = new User() + +// Kotlin +val user = User() + +// Go +user := User{} +ptr := &User{} +user2 := new(User) + +// Ruby +user = User.new +``` + +Some languages can identify constructor-like syntax directly. Others need validation through the `SymbolTable`, because syntax alone cannot always distinguish `User()` from `getUser()`. + +In those cases the system records an unverified binding candidate and later validates it against known class / struct symbols. + +### Tier 2: Assignment Chain Propagation + +Bindings can propagate through simple identifier assignments. + +```typescript +const user: User = getUser() +const alias = user +const other = alias +``` + +This is handled after the main walk through a single pass over pending assignments. + +This supports simple forward propagation, but there is no iterative fixpoint step. For example: + +```typescript +const b = a +const a: User = getUser() +``` + +will not resolve `b`. + +--- + +## Container Type Descriptors + +`CONTAINER_DESCRIPTORS` defines the type-parameter semantics for common containers. + +That allows the resolver to distinguish key-yielding methods from value-yielding methods instead of always assuming the last generic argument. + +```typescript +for (const key of map.keys()) { ... } // key → string +for (const val of map.values()) { ... } // val → User +``` + +Unknown containers fall back to heuristics, keeping the system conservative rather than fully semantic. + +### Examples of descriptor-driven behavior + +- `Map` / `Dictionary` / similar key-value containers +- `List` / `Array` / `Vec` / `Set` / similar single-element containers +- method-aware yield selection such as `.keys()`, `.values()`, `.keySet()`, `.Values` + +--- + +## Comment-Based Types + +For less strictly typed ecosystems, the resolver can fall back to documentation-based type information. + +Supported comment systems: + +- **JSDoc** for JavaScript / TypeScript +- **PHPDoc** for PHP +- **YARD** for Ruby + +These are used conservatively and only when AST-level type information is missing or insufficient. + +--- + +## SymbolTable Interaction + +Although the environment is built per file, it may consult the global `SymbolTable` in specific validation paths. + +This is important for languages where constructor-like syntax is ambiguous. A binding candidate such as `val user = User()` may need confirmation that `User` is a class-like symbol rather than an ordinary function. + +This means the system is still **per-file in binding construction**, but not completely isolated from project-wide symbol knowledge. + +--- + +## Deferred Binding Verification in Call Processing + +A key detail is that some initializer bindings are not fully resolved inside `TypeEnv` itself. + +`call-processor.ts` later verifies deferred bindings and may infer receiver types from: + +- validated class / struct constructor candidates +- uniquely resolved function or method calls that expose a usable return type + +So return-type-aware receiver inference already exists in a constrained downstream form today. What does **not** yet exist is feeding that information back into `TypeEnv` broadly enough to power loop inference, general assignment propagation, and wider expression typing. + +--- + +## Language Feature Matrix + +| Feature | TS/JS | Java | Kotlin | C# | Go | Rust | Python | PHP | Ruby | Swift | C/C++ | +|---------|:-----:|:----:|:------:|:--:|:--:|:----:|:------:|:---:|:----:|:-----:|:-----:| +| Declarations | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | +| Parameters | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | +| Initializer / constructor inference | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | +| Constructor binding scan | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | +| For-loop element types | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | +| Pattern binding | Yes | Yes | Yes | Yes | No | Yes | Yes | No | No | No | No | +| Assignment chains | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | +| Comment-based types | JSDoc | No | No | No | No | No | No | PHPDoc | YARD | No | No | +| Return type extraction | JSDoc | No | No | No | No | No | No | PHPDoc | YARD | No | No | + +--- + +## Current Strengths + +The current system already provides strong value for call resolution because it combines: + +- explicit annotation extraction +- generic-aware loop element typing +- initializer-based inference +- selected pattern-based narrowing +- scope-aware lookups +- comment-based fallbacks for dynamic ecosystems +- constrained return-type-aware receiver inference in call processing + +This is enough to materially improve call-edge precision even without implementing a full static type system. + +--- + +## Current Limitations + +Important gaps still remain: + +- no field / property type map for deep chains such as `user.address.city` +- no general cross-file propagation of inferred bindings +- no fixpoint inference +- limited branch-sensitive narrowing outside selected pattern constructs +- limited Swift support compared with other languages +- no complete destructuring-based field typing +- no broad expression-level return-type propagation inside `TypeEnv` + +--- + +## Contributor Notes + +When modifying this system, treat the following as load-bearing invariants: + +1. **Conservatism matters more than recall** + A missed binding is usually safer than a misleading receiver type. + +2. **Scope-key format is shared behavior** + If scope keys change, constructor-binding verification and any downstream lookup using those keys must change in sync. + +3. **Tier naming may differ across code and PR discussions** + For-loop element inference may appear as "Tier 0b" in documentation and "Tier 1c" in Phase 6 PR / test naming. + +4. **Comment-based types are fallback signals, not primary truth** + They should remain lower-trust than explicit AST-derived types. + +5. **Return-type-aware inference already exists in constrained form** + Future roadmap work should extend and generalize it rather than reintroduce it from scratch.