2 commits

Author	SHA1	Message	Date
Gergő Magyar	5fa73bafdf	feat: Phase 4 type resolution — nullable unwrapping, for-loop typing, assignment chains, code review fixes (#310) ... * feat: Phase 4 type resolution — nullable unwrapping, for-loop typing, assignment chains, Kotlin return types Phase 4.1: Nullable/optional chain unwrapping - Add stripNullable utility in shared.ts for stripping nullable wrappers - Apply in lookupInEnv to unwrap User | null → User, User? → User before receiver lookup - Handles TS union, Kotlin/C#/Swift nullable suffix, Python Union[T, None], Rust Option<T> - Enables receiver-type disambiguation through ?. optional chaining Phase 4.2: For-loop element typing (Tier 0 — Java/C#/Kotlin) - Add ForLoopExtractor type and forLoopNodeTypes to LanguageTypeConfig - Java enhanced_for_statement, C# foreach_statement, Kotlin for_statement extractors - Only explicit element types in AST (Tier 0); inference-based languages deferred Phase 4.3: Assignment chain propagation (single-pass, depth-1) - Add PendingAssignmentExtractor to LanguageTypeConfig with per-language implementations - Handles TS/JS variable_declarator, Rust let_declaration, Python assignment, Go short_var_declaration, C# equals_value_clause, Java/Kotlin variable_declarator - Single post-walk propagation pass (no fixpoint iteration per Sorbet/Pyright design) - Resolves const b = a; b.save() when a has known type from Tier 0/1/1b Phase 4.5: Kotlin return type extraction (bug fix) - Fix extractMethodSignature to handle Kotlin user_type after function_value_parameters - Remove lenient test assertions, add strict disambiguation proof Integration tests across 10+ languages with competing same-name methods and negative assertions proving disambiguation. * fix: per-language assignment chain gaps from code review - Kotlin: new extractKotlinPendingAssignment for property_declaration → variable_declaration AST (Java's variable_declarator doesn't exist in Kotlin) - Go: handle var_spec (var b = u) alongside short_var_declaration (:=) - PHP: add extractPendingAssignment for $alias = $user with $ prefix preserved Integration tests added for all three languages with competing same-name methods and negative disambiguation assertions. * fix: code review fixes — DRY nullable keywords, avoid array allocations, clarify depth comment Addresses findings from 6-agent code review on PR #310: - Move stripNullable JSDoc to correct position (was orphaned above NULLABLE_KEYWORDS) - DRY: reuse NULLABLE_KEYWORDS set in pipe-split filter instead of inline strings - Replace node.children.find() with findChildByType/manual loops in jvm.ts, go.ts, csharp.ts to avoid unnecessary array allocations per tree-sitter call - Clarify "depth-1" comment in type-env.ts: single-pass resolves multi-hop chains when forward-declared; reverse-order is depth-1 only - Annotate extractGenericTypeArgs as Phase 5 infrastructure (zero production callers) - Re-export PendingAssignmentExtractor from index.ts for API consistency - Add explicit return undefined in Go extractPendingAssignment - Remove redundant child.text === '=' check in Kotlin extractor Test coverage: - 20 new unit tests: stripNullable edge cases, per-language assignment chains, reverse-order depth limitation, nullable lookup resolution - 15 new integration tests: multi-hop chains (a→b→c), nullable+chain combined (User|null + alias), Python User|None through stripNullable path - 3 new fixtures: ts-multi-hop-chain, ts-nullable-chain, python-nullable-chain * fix: third-pass review — walrus chain, scanner allocations, Kotlin variable_declaration, C# type guard Addresses 4 new findings from third-pass CI review: 1. Python walrus operator (:=) now handled by extractPendingAssignment — named_expression nodes propagate alias chains alongside regular assignment 2. Scanner .namedChildren.find()/.some() in jvm.ts replaced with findChildByType() — consistent with 98daed4 code review fixes 3. Kotlin extractPendingAssignment extended to handle variable_declaration nodes in addition to property_declaration (function-local val/var) 4. C# extractPendingAssignment early-returns for is_pattern_expression and field_declaration nodes (never contain variable_declarator children) Integration tests: - Python: walrus chain (alias := u) with disambiguation (5 tests, 1 fixture) - Kotlin: assignment chain with typed declarations (5 tests, 1 fixture) - C#: assignment chain + is-pattern coexistence (6 tests, 1 fixture) - Unit: Python walrus propagation (1 test) * feat: nullable wrapper unwrapping + C++ assignment chains Gaps 1, 2, 4 from code review — architectural changes to type resolution: 1. extractSimpleTypeName now unwraps nullable wrapper generics: - Optional<User> → "User" (Java), Option<User> → "User" (Rust), Maybe<User> → "User" (Kotlin Arrow/Haskell-style) - Containers (List, Map) and async wrappers (Promise, Future) are NOT unwrapped — methods are called on the container, not the inner type - Uses existing extractGenericTypeArgs (now production-active, was dead code) - NULLABLE_WRAPPER_TYPES set: Optional, Option, Maybe 2. C++ extractPendingAssignment added for auto alias chains: - auto alias = user; alias.save() now propagates User type - Handles pointer/reference declarators, auto/decltype(auto) 3. Updated existing Rust test: Option<User> parameter now correctly stores "User" instead of "Option" in TypeEnv Integration tests with fixtures for Java Optional, Rust Option, C++ auto chain. Full pipeline resolution marked .todo — requires call-processor enhancement (TypeEnv stores correct types but call-processor needs additional work to produce CALLS edges for these patterns). Unit tests: 196 passed (7 new). Integration: all 9 languages green. * fix: resolve .todo tests — stale dist/ was the root cause The Rust Option<User> and C++ auto assignment chain integration tests were marked .todo because the pipeline didn't produce CALLS edges. Root cause: dist/ was compiled from pre-Phase 4 source and lacked: - NULLABLE_WRAPPER_TYPES unwrapping in extractSimpleTypeName - C++ extractPendingAssignment After npm run build, all tests pass as real assertions: - Rust: alias.save() resolves to User#save via Option<User> unwrap + chain - C++: alias.save() and rAlias.save() resolve via auto assignment chain with correct disambiguation (User vs Repo) Only remaining .todo: Rust user.unwrap().save() (Phase 5 — chained return type inference, not a TypeEnv issue).	2026-03-16 15:21:54 +00:00
Gergő Magyar	6c18ae08f7	feat: return type inference, doc-comment parsing, and per-language type extractors (#284) ... * feat: Phase 3 — return type inference, generic args extraction, Ruby YARD type extractor Three architectural improvements to the type resolution system: 1. Return type inference — wire extractMethodSignature returnType through SymbolDefinition into call-processor. When var = callee() and callee has a known return type, bind var to that type. Handles Promise<T> unwrapping, nullable stripping, pointer/reference removal. 2. Generic type argument extraction — new extractGenericTypeArgs() utility that extracts type parameters from List<User> → ['User']. Handles TS/Java/Kotlin/C#/Rust generic syntax. Building block for for-loop variable typing. 3. Ruby dedicated type extractor — replaces the stub with YARD annotation parsing (@param name [Type]), handling qualified types, nullable types, and singleton methods. Ruby now has real type resolution. Unit tests: 127 → 192+ (type-env) + 65 (symbol-table, call-processor) + 18 (generics) Integration tests: 8+ new test cases with fixtures across TS/Python/Go/Java/Ruby * fix: Phase 3 gaps — WRAPPER_GENERICS correctness, Ruby :: qualifier, namespaced constructors - Remove collection types (List, Array, Vec, Set) from WRAPPER_GENERICS to prevent false CALLS edges (e.g. List<User> no longer unwraps to User) - Add :: qualifier handling in extractReturnTypeName for Ruby/C++/Rust namespaced types - Add Ruby `constant` and `scope_resolution` node types to shared extractors - Extract shared extractRubyConstructorAssignment helper (dedup type-env.ts + ruby.ts) - Add integration tests for return type inference: Python, TypeScript, Go, Java, Ruby - Add Ruby namespaced constructor fixture (Models::UserService.new) - Add unit tests for collection reclassification and :: qualifiers * feat: Phase 4 — CONSTRUCTOR_BINDING_SCANNERS for all languages + return type inference tests Add CONSTRUCTOR_BINDING_SCANNERS for 6 missing languages, completing return type inference coverage across all 11 supported languages: - TypeScript/JS: variable_declarator with call_expression, unwraps await - Go: short_var_declaration single-assignment (skips multi-return, new/make) - Java: local_variable_declaration with `var` type + method_invocation - C#: variable_declaration with implicit_type (var) + invocation_expression - Rust: let_declaration without type annotation, handles mut_pattern - PHP: assignment_expression with function_call_expression Also adds property_identifier to extractSimpleTypeName for qualified member calls (repo.getUser → getUser), fixing namespaced constructor inference that was previously a known limitation. Integration tests added for all 11 languages with correct label assertions (Function vs Method per language's tree-sitter queries). * refactor: merge CONSTRUCTOR_BINDING_SCANNERS into per-language LanguageTypeConfig Eliminates the parallel dispatch map in type-env.ts by moving all 11 constructor binding scanners into their respective type-extractors/*.ts files as `scanConstructorBinding` on LanguageTypeConfig. - Add ConstructorBindingScanner type to types.ts - Add shared helpers: hasTypeAnnotation, unwrapAwait, extractCalleeName - Move scanners to typescript.ts, jvm.ts, python.ts, php.ts, go.ts, rust.ts, swift.ts, c-cpp.ts, csharp.ts, ruby.ts - Fix `any` types in C# scanner → SyntaxNode | null - Delete ~300 lines from type-env.ts (CONSTRUCTOR_BINDING_SCANNERS map) - Update buildTypeEnv to use config.scanConstructorBinding All 143 type-env unit tests and all 10 language integration suites pass. * fix: remove unused import, fix any type in Java scanner, update stale comment - Remove unused extractCalleeName import from jvm.ts - Fix (c: any) → (c: SyntaxNode) in Java scanner - Update stale CONSTRUCTOR_BINDING_SCANNERS reference in ruby.ts comment * fix: C# and PHP return type inference — scanner fixes, method signature extraction, and cross-file resolution Addresses code review findings on PR #284: C# scanner (csharp.ts): - Fix type node lookup: iterate children instead of childForFieldName('type') which returns undefined in tree-sitter-c-sharp - Fix initializer lookup: handle direct invocation_expression children (no equals_value_clause wrapper in tree-sitter-c-sharp) C# return type extraction (utils.ts): - Add 'returns' field check to extractMethodSignature — tree-sitter-c-sharp uses 'returns', not 'type', for method return types C# cross-file resolution (call-processor.ts + fixture): - Add constructor binding verification to sequential processCalls path (was only in the worker processCallsFromExtracted path) - Add ReturnType.csproj to csharp-return-type fixture - Update fixture namespaces to use ReturnType.Models/ReturnType.Services prefix (matches real C# project conventions) PHP scanner (php.ts): - Extend scanConstructorBinding to handle member_call_expression ($this->getUser() patterns), not just function_call_expression Shared (shared.ts): - Add member_access_expression to extractSimpleTypeName qualified-names block (C# method calls like svc.GetUser()) Tests: - Add Repo.cs/Repo.php disambiguation fixtures (two Save methods) - Strengthen C# and PHP return type tests with hard disambiguation assertions - Add C# scanner unit tests and return type extraction test * feat: per-language ReturnTypeExtractor + doc-comment @param parsing for PHP, JS, Ruby Add ReturnTypeExtractor to LanguageTypeConfig interface with implementations for Ruby (YARD @return), PHP (PHPDoc @return), and JS/TS (JSDoc @returns). The fallback is wired in both parsing-processor and parse-worker paths, activating only when extractMethodSignature finds no AST-based return type. Also add doc-comment @param type extraction for PHP and JS/TS, following Ruby's existing collectYardParams pattern. This enables parameter.method() resolution in loosely-typed codebases using PHPDoc @param or JSDoc @param. Additional fixes from PR #284 code review: - Go: add selector_expression + field_identifier to extractSimpleTypeName (enables package-qualified factory calls like models.NewUser()) - Ruby: broaden scanConstructorBinding to capture plain call assignments (user = get_user()) in addition to Class.new patterns - Ruby: harden return-type fixture with disambiguation (two save methods) Test coverage: +14 new integration tests across Go, Ruby, PHP, JS/TS * fix: JSDoc async return type, PHP attribute walkers, and $this receiver disambiguation Three fixes from fourth-pass code review on PR #284: 1. JSDoc `@returns {Promise<User>}` no longer stripped to `Promise` — extractReturnType now uses sanitizeReturnType (preserves generics) instead of normalizeJsDocType (which stripped them before extractReturnTypeName could unwrap WRAPPER_GENERICS). 2. PHP 8+ `#[Attribute]` and JS `@decorator` nodes no longer break doc-comment walkers. Both extractReturnType and collect*Params functions now skip attribute_list/decorator nodes instead of breaking on them as named siblings. 3. PHP `$this->method()` now provides receiverClassName for disambiguation. When two classes define the same method, the enclosing class narrows candidates via ownerId matching in call-processor, preventing false no-binding results. * fix: sanitizeReturnType dot corruption, JS test assertions, Ruby constant receiver - Remove redundant dot-path stripping from sanitizeReturnType that corrupted qualified names inside generics (e.g. Promise<models.User> → User>) - Split JS async fixture into separate files and add negative assertions to properly verify disambiguation (mirroring PHP test pattern) - Accept 'constant' node type in Ruby scanConstructorBinding for factory call assignments (SERVICE = build_service()) - Add 'constant' to SIMPLE_RECEIVER_TYPES so extractReceiverName handles Ruby constant receivers (SERVICE.process) * fix: nested generic arg splitting, JS/Ruby test false positives - Replace naive comma split in extractReturnTypeName with bracket-balanced extractFirstGenericArg so nested types like Future<Result<User, Error>> unwrap correctly instead of producing malformed "Result<User" - Add CompletableFuture to WRAPPER_GENERICS for Java async unwrapping - Split js-jsdoc-return-type fixture models.js into user.js/repo.js and add negative assertions to prove disambiguation (not just file match) - Split ruby-constant-factory-call fixture into separate service files and add negative assertions against AdminService resolution * fix: review findings — receiverClassName parity, Rust wrappers, Go multi-return, Kotlin/Swift qualified calls P1: Sequential path now includes receiverClassName narrowing for PHP $this->method() disambiguation (was missing vs worker path). P2: Added Rc/Arc/Weak/MutexGuard/Cow + 6 more Rust Deref types to WRAPPER_GENERICS (Box excluded — Java Swing collision). Extended Kotlin/Swift scanners to handle navigation_expression callees. Added Go multi-return support (user, err := f()) with blank/_/err/ok guard + AST-level first-return extraction in extractMethodSignature. P3: Extracted shared verifyConstructorBindings() eliminating 60 lines of duplication between sequential and worker paths. Added return-type inference integration tests for C++, Rust, Swift with competing methods and negative disambiguation assertions. * fix: Swift navigation_suffix unwrapping, Rust lifetime skipping, Kotlin disambiguation tests - Swift scanConstructorBinding: handle tree-sitter wrapping qualified identifiers in navigation_suffix nodes - Add extractFirstTypeArg to skip Rust lifetime parameters ('a, '_) when unwrapping wrapper generics like Ref<'_, User> - Kotlin tests: add Repo class fixture with competing save() methods to prove disambiguation; assert no spurious edges on known gap - Remove tree-sitter-kotlin from optionalDependencies (now regular dep) * fix: C# null-conditional calls, Ruby YARD bracket-balanced split, PHPDoc alternate order, escapeValue hardening - Add C# null-conditional call support (user?.Save()): tree-sitter query for conditional_access_expression, member_binding_expression in MEMBER_ACCESS_NODE_TYPES, receiver extraction via conditional_access_expression parent walk - Fix Ruby YARD type parsing for nested generics (Hash<Symbol, User>): replace naive split(',') with bracket-balanced splitter respecting <> depth - Add alternate YARD format (@param [Type] name) alongside standard (@param name [Type]) - Add alternate PHPDoc format (@param $name Type) alongside standard (@param Type $name) - Harden escapeValue in kuzu-adapter.ts: escape \n and \r to prevent Cypher injection - Integration tests: C# null-conditional fixture (5 tests), Ruby YARD generics fixture (6 tests) - Unit tests: PHPDoc alternate order (2 tests), C# null-conditional call-form (updated) * test: add Python static/classmethod integration tests (issue #289) Verifies that classes using only @staticmethod/@classmethod have HAS_METHOD edges connecting them to their child methods. This was the root cause of issue #289 where context() and impact() returned empty for such classes. Tests cover: HAS_METHOD edge emission, unique static method resolution (create_user, delete_user), and ambiguous same-named method handling (find_user on both UserService and AdminService — safely refused). * fix: lbug batch escapeValue newline hardening, Rust ::default() scanner exclusion - Apply \n/\r escaping to batch upsert escapeValue in lbug-adapter.ts:429 (missed instance of the CREATE-path fix from ec4dca4) - Exclude Rust ::default() from scanConstructorBinding to match extractInitializer behavior — avoids wasted cross-file lookups on the broadly-implemented Default trait - Unit tests: 2 new scanner exclusion tests (::default and ::new) - Integration tests: 6 new Rust ::default() constructor resolution tests with disambiguation fixture (User::default vs Repo::default) * fix: C#/Rust async await unwrap, PHP backslash namespace, fallback escaping - C# scanConstructorBinding: unwrap await_expression to find invocation_expression (var user = await svc.GetUserAsync() now produces constructor binding) - Rust scanConstructorBinding: unwrap .await postfix via shared unwrapAwait helper (let user = get_user().await now produces constructor binding) - extractReturnTypeName: handle PHP backslash namespace separator (\App\Models\User → User) - fallbackRelationshipInserts: match batch escapeValue hardening with \n/\r escaping Tests: 2 unit (type-env), 3 unit (call-processor), 7 integration (csharp+rust), 7 fixtures * fix: C#/Rust async-binding test false positives — add competing types and negative assertions C# fixture: add Order.cs with Order.Save(), change OrderService to return Task<Order> via GetOrderAsync, add negative assertion proving user.Save() does not resolve to Order#Save. Rust fixture: split models.rs into user.rs/repo.rs, make process_user and process_repo async fn, add bidirectional negative assertions proving no cross-contamination between User#save and Repo#save. * fix: C# async-binding broken assertion, bare wrapper type leak, JSDoc optional params - Split Program.cs Main into ProcessUser/ProcessOrder so negative assertions use strict toBeUndefined() (matching Rust pattern) - Guard bare wrapper types (Task, Promise, Option…) in extractReturnTypeName — return undefined instead of the wrapper name - Update JSDOC_PARAM_RE to capture @param {Type} [optionalName] syntax * fix: update symbol and relationship counts in documentation	2026-03-15 18:49:40 +00:00