mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-07 02:58:02 +00:00
422 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Gergő Magyar
|
997fc05b83
|
fix(resolution): resolve calls through a generic-typed field receiver in every language (#2833) (#2855)
* test(resolution): pin generic-typed field receivers across languages (#2833) A field whose declared type carries a type argument (`repo: Repo<User>`) emits zero CALLS edges — not a truncated chain, not an edge to the interface declaration, nothing. This adds the cross-language matrix that measures it, modelled on the #2807 inferred-field matrix: every language runs the same two calls, one through a generic-typed field and one through a non-generic control field, and each language is compared against its OWN control row rather than an absolute edge count. Measured state, pinned here as `known-gap` so the file is green on main and flipping a row is a visible edit: affected TypeScript, C#, C++, Python unaffected Java, Kotlin, Go, Rust, Swift, Dart The unaffected six erase type arguments at interpret time (Java's `stripGeneric`, F41 #1928; Swift likewise). TypeScript, C# and Python instead run a container ALLOW-LIST that returns the type ARGUMENT, so a user-defined `Repo<User>` survives verbatim into a lookup that binds nothing. The `ts-local-vs-field` case is the bug in one file: `viaLocal` and `viaParam` both resolve for the identical type, and only `viaField` loses every edge — a bare name reaches Case 4 and its generic-aware lookup, a dotted field receiver does not. Negative controls pin what erasure must NOT do: an unbounded type parameter denotes no declaration, and a C++ explicit specialization is a different class from its primary template. The `Box2<T>` row pins a PRE-EXISTING false edge (a workspace class named `T`) so it cannot later be mistaken for fallout from this work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * refactor(resolution): move resolveClassBindingForName to the shared walkers (#2833) Pure relocation, no behaviour change: the generic-aware class lookup moves from `passes/receiver-bound-calls.ts` to `scope/walkers.ts`, beside the bare `findClassBindingInScope` it wraps. Its two existing callers — `classifyReceiverOrigin` and Case 4 — import it from the new home and are otherwise untouched. The move is required rather than cosmetic: `receiver-bound-calls.ts` already imports from `compound-receiver.ts`, so having the compound receiver call into the pass would close an import cycle. `walkers.ts` is the shared floor both already depend on. Verified behaviour-neutral: the #2833 matrix is 44/44 identical before and after, across all fifteen fixtures. detect_changes attributes `resolveInheritanceBaseInScope`, `resolveQualifiedInheritanceBase` and `EMPTY_BINDINGS` to this commit; those are line-shift artifacts of inserting a function above them, and their bodies are byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * fix(resolution): type generic field receivers through the generic-aware lookup (#2833) A field receiver is spelled `this.repo` — dotted — so it types through the receiver-chain fold and the text cascade, both of which reach `findClassBindingInScope`. That function has no notion of type arguments, so a field declared `Repo<User>` resolved to nothing and the call site emitted NO edge at all: not the interface declaration, not the implementation fan-out, nothing. A local or parameter of the identical type is a bare name, reaches Case 4 and its generic-aware `resolveClassBindingForName`, and resolved fine. The bug was the asymmetry, not the generics. Three receiver-typing lookups now call the generic-aware helper instead: `typeOfMemberOnClass`'s primary and module-hoist branches, and the cascade's bare-identifier type-binding read. Every other one of the 38 `findClassBindingInScope` call sites is untouched — its own docstring records that widening it globally suppresses the `?? otherResolver(...)` fallbacks two dozen callers rely on, which would retarget inheritance edges, and impact rates it CRITICAL with 12 direct dependents. Order matters and is preserved: the helper tries the exact name, then an arity- and token-exact match against `def.templateArguments`, and only then falls back to the base name. Erasing first would collapse a C++ explicit specialization onto its primary template — `Vec<bool>` really is a different class. A bare type parameter carries no type arguments, so it never enters the generic branch and cannot be erased into a class that happens to share its name. Measured: TypeScript and C# generic-typed fields now emit exactly what their non-generic control rows emit, primary plus interface-dispatch fan-out. Java, Kotlin, Go, Rust, Swift and Dart are byte-identical. Both type-parameter negative controls are unchanged. C++ and Python are still open and stay pinned as known-gaps — they fail for different reasons and get their own commits. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * fix(cpp,python): bind generic-typed member fields so their calls resolve (#2833) Completes #2833 for the two languages the shared resolution change could not reach. Each failed for its own reason, and both were found by measurement rather than assumed. C++ — a CAPTURE gap, not a resolution one. All three `field_declaration` type-binding rules required `type: (type_identifier)`, so a member declared `Repo<User> repo;` is a `template_type` and matched none of them: the field got no type binding at all, and every call through it lost its edge in both the bare and `this->` spellings. A LOCAL of the identical type resolved the whole time, because the local declaration rules gained their `template_type` variant long ago. Three mirrored rules close it, one per declarator shape (plain, pointer, reference). Written as separate patterns rather than one alternation: a node-type alternation in a field position is a tree-sitter 0.21 hazard this repo has been bitten by before. Python — the bracket spelling never entered the generic branch. Its `stripGeneric` is a container allow-list over `[...]` that returns the type ARGUMENT (`list[User]` to `User`), so a user-defined `Repo[User]` matched nothing and survived verbatim, and the shared lookup's generic branch is gated on `<`. It now reduces a subscripted type neither allow-list claims to its base name — the same rule Java and Swift already apply to `<...>`. Deliberately the LAST resort: a container must reach its own rule first, or `list[User]` would type the receiver as the container and retarget every call in a for-loop chain. The as-written spelling survives on `TypeRef.declaredSpelling`, which is what the fold's index step reads. Both are parse-time and land in the cached ParsedFile, so SCHEMA_BUMP goes 45 -> 46 with its pin test. Verified free against origin/main; the ledger in that file records three prior EXACT clashes, so re-check again immediately before merge. The matrix now covers the spellings real code writes, all measured: a nullable generic, a bounded wildcard, a raw type, a nested generic and a multi-argument one. None needed work beyond the shared lookup, which is the evidence that base-name erasure is the right primitive. The C++ specialization control now asserts what it was written for: `Vec<bool>.save` and `Vec.save` are DIFFERENT target ids, so the arity/token match still wins over erasure. scope-capture is byte-identical for cpp and c, so no rebaseline — the bench corpus contains no generic-typed member field, which is worth its own coverage issue. Two pre-existing gaps were measured and are deliberately NOT fixed here, because in both cases the language's own non-generic CONTROL row fails identically: C++ `this->field.m()` emits nothing, and JavaScript/PHP docblock-declared field types bind nothing at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * fix(python): do not reduce containers or typing special forms to a base name (#2833) Review finding on this branch's own Python change, caught by probing the interpreter directly rather than by reading it. The base-name reduction was reached by FALLTHROUGH: "neither container rule matched" was treated as "not a container". It is not, and two measured shapes proved it: dict[str, list[User]] -> dict (was: the annotation, intact) Dict[str, Repo[User]] -> Dict Callable[[int], User] -> Callable Literal["a"] -> Literal Union[A, B] -> Union tuple[int, ...] -> tuple The dict rule's value group cannot span a nested `]`, so a nested value declines and falls through — and the dict rule's own comment says that shape is deliberately "left for a downstream strip pass". Collapsing it to `dict` destroyed the value type instead. The typing SPECIAL FORMS are worse: `Callable`, `Literal`, `Annotated` and `Union` are not classes, and reducing them to a bare name binds any workspace class that happens to share it — a fabricated edge, which is strictly worse than the missing edge #2833 set out to fix, and those names are ordinary enough for a real codebase to declare. Reduction is now guarded by an explicit deny set covering the containers the two allow-lists already own and the typing special forms. Everything named there keeps its as-written text and resolves exactly as it did before #2833. `arr[0]` also reduces to `arr` in isolation, but that is unreachable and is now documented as such: every Python `@type-binding.type` capture is a `(type)`, `(identifier)`, `(attribute)` or `(dotted_name)` node, so a subscripted VALUE expression never reaches the interpreter. Pinned by a new unit test that asserts all four groups — user generic reduces, container reduces to its ELEMENT, declined container shape stays intact, special form untouched. Reverting the deny set fails three of its five cases. Also corrects `resolveClassBindingForName`'s docstring, which this branch had made false: it claimed only `classifyReceiverOrigin` passes the decoration stripper, while the three receiver-typing lookups in compound-receiver.ts now pass it too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * fix(resolution): rank base-name candidates lexically and refuse arg-pinned defs (#2833) Review of #2855 found that this PR turned a MISSING C++ edge into a CONFIDENTLY WRONG one — the direction this subsystem calls unrecoverable. `resolveClassBindingForName` ended with an unguarded base-name fallback that returned the first same-named class the scope chain reached. A C++ primary template carries `templateArguments === undefined`, so it can never satisfy the exact-args branch, and every non-specialized instantiation fell through to that fallback. Measured through the real pipeline: with the primary forward-declared and the specialization defined first, `Vec<int> vi; vi.save()` emitted `Vec<bool>::save`. Declaring the primary first gave the correct target — selection was SOURCE-ORDER DEPENDENT. Two more triggers behaved the same way: a partial specialization (`Vec<int*>` against `Vec<T*>`), and lexical shadowing between a global `Box<bool>` and a namespaced `N::Box<bool>`. Two changes, neither of which is any of the three remediations the review proposed — each was rejected on measured evidence: - Exact-argument matching is now LEXICAL-FIRST. Candidates come from the scope chain, and the workspace-wide qualified-name bucket is consulted only when the chain produced no exact match, so cross-file specializations still bind. - The base-name route refuses a definition that pinned its own template arguments: if the fallback's answer carries `templateArguments`, the visible candidates are re-decided with those removed — exactly one, or decline. Why not the filed options. "If specializations exist and none matches exactly, return undefined" deletes a green committed row (`neg-cpp-specialization/runInt` legitimately resolves to the primary). "Resolve all defs for the base name, return only on exactly one" deletes a working edge for C# `partial class Repo<T>` split across files — two unspecialized defs under one name is legitimate, and `QualifiedNameIndex`'s own docstring names that case. Preferring the primary alone fixes nothing about shadowing, which is a ranking bug. The guard is expressed as `carriesOwnTemplateArguments`, not as "specialization", so shared pipeline code still names no language (AGENTS.md R6). It can only fire where a declared name carries concrete arguments — measured `undefined` for `class Repo<T>` in TypeScript and C# and for a C++ primary template — so the blast radius is bounded to C++-style specializations. Partial-specialization SELECTION is deliberately not implemented: choosing `Vec<T*>` for `Vec<int*>` needs template-argument deduction, which is a semantics expansion and cannot live in language-neutral shared code. The source-order dependence is what is fixed; the answer is now deterministically the primary. Also in this commit: dropped an unreachable `?? []` (QualifiedNameIndex returns a frozen empty array on miss by contract) whose comment was wrong on both clauses; made the docstring true about argument ERASURE being what widens what binds, rather than only the decoration stripper; and corrected a stale pointer that still placed `resolveClassBindingForName` in `receiver-bound-calls`. `findClassBindingInScope` itself is untouched — 38 call sites, CRITICAL. Verified: matrix 56/56, cpp.test.ts 334, unit scope-resolution 1505. Mutation proof: reverting this file fails the three trigger cases and passes the non-regression cases; restoring it passes all five. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * fix(python): close the deny-set drift axis by case-folding, not by vigilance (#2833) Review of #2855 found `NOT_A_USER_GENERIC` was a closed list over an open universe: four review lanes each escaped it with a DIFFERENT set of names. `Deque` was the sharpest — its lowercase twin `deque` was already listed, so the omission was an internal inconsistency rather than a judgement call, and with a workspace `class Deque` present `self.dq: Deque[User]` fabricated a `Deque.appendleft` edge. The structural cause is PEP 585: nearly every container has two spellings differing only in case (`deque`/`typing.Deque`, `frozenset`/`FrozenSet`). Exact matching forced every pair to be listed twice, so any half-pair was a silent escape. The deny lookup is now CASE-FOLDED, which closes that axis by construction — `Deque` becomes impossible rather than remembered. `SINGLE_ARG_CONTAINERS` and `MAPPING_CONTAINERS` are now the single source of truth: they build the two container regexes (verified byte-identical `.source` and `.flags`, so zero behaviour change) and feed the property test. The deny set is re-scoped to a closed, auditable universe — the documented Python stdlib type-system surface — and grew 39 -> 65 concepts: the `collections.abc` views, `contextlib` managers, `re.Pattern`/`Match`, the `IO` family, ordinary-named stdlib generics (`Queue`, `Task`, `Future`, `PathLike`), the remaining typing special forms, and the generic machinery (`Generic`, `Protocol`, `TypeVar`...). Third-party generics (`Mapped`, `QuerySet`, `Model`) are deliberately NOT added and are pinned as a decision: that universe is open, enumerating it only chases the last escape, and declining `Model` would cost real edges in the many projects that declare one. The review's suggested property test — derive the names from the `single`/`dict` regex sources — would NOT have caught `Deque`: `deque` appears in neither regex, only in the deny set. Both properties are implemented, since they catch different drift. The unit test was also TAUTOLOGICAL: it asserted members OF the deny set, so it structurally could not detect an omission. It now asserts case-fold closure and PEP 585 alias coverage, and the capture fixture drops its `as unknown as` cast for the fully-typed helper pattern the sibling `java-interpret.test.ts` already uses. Still at interpret time, so no further SCHEMA_BUMP (already 45 -> 46). Proving the base is a class the FILE can see — the real fix for the remaining exposure, since `findClassBindingInScope` binds any name with exactly one workspace def regardless of scope or imports — is a follow-up, not reachable from this file. Mutation proof: restoring HEAD's deny-set contents and exact-match lookup fails four assertions including the `Deque` pair, with the pre-existing guard rows still passing; restoring gives 125/125. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * fix(cpp): capture qualified generic member fields, and make the bench gate see them (#2833) Review of #2855 found that the three `field_declaration` rules this PR added only matched a DIRECT `template_type`, so the common real-world spelling still bound nothing: `std::vector<Item> items;`, `ns::Repo<User> r;` and `std::unique_ptr<Repo> p;` parse as a `qualified_identifier` WRAPPING a `template_type`. "C++ fixed" was overstated. Six new patterns — three declarator shapes (plain, pointer, reference) by two qualifier depths — written as separate patterns rather than one alternation, keeping the tree-sitter 0.21 field-position discipline the existing rules follow. The design choice was measured, not assumed. Codex suggested preserving the full qualified spelling and normalizing `::`; preserving resolves NOTHING, because `findClassBindingInScope`'s dotted-tail fallback splits on `.` while C++ writes `::`, and `ns::Repo` is not an index key either (C++ emits no `@declaration.qualified_name`). Measured: `ns::Repo<User>` resolves to nothing, `ns.Repo<User>` resolves to `Repo`. Since a tree-sitter capture is a NODE and not synthesized text, the only lever is which node to capture — so `@type-binding.type` goes on the INNER `template_type`, dropping the qualifier and landing on the same single-match-or-decline path the bare spelling already takes. Qualifier depth 3+ (`a:🅱️:c::Repo<User>`) remains uncaptured. Stated as a limit and pinned by a test row, not claimed as fixed. The bench blindness the review identified is also closed. The `scope-capture` C++ corpus contained ZERO template-typed member fields — confirmed a fourth way by applying six demonstrably behaviour-changing patterns and getting a byte-identical fingerprint. The corpus now carries generic and qualified-generic members, and the gate is load bearing for the first time: three states that all hashed to 856d02f3 before now differ (pre-#2833 0e7cbda7, +this PR's 3 rules de07d8b5, +these 6 rules bd47c82d). Rebaselined for cpp only; c is unchanged. Histogram diff: only 5 tags move with the fields, each by exactly +40 (20 entities x 2), and every `@reference.*` count is unchanged. Over-match is preserved: 20 shapes still produce no field capture, including the 8 original method/pointer/reference/function-pointer/ using/typedef/friend/operator forms plus their `std::`- and `a:🅱️:`-qualified variants. Not fixed here, deliberately: NON-generic qualified fields (`ns::Address addr;`, `std::string name;`) still capture nothing. Closing that needs six more patterns and would newly bind every `std::string`/`std::mutex` member repo-wide, changing edges far outside #2833. Separate issue. The template-template-parameter hazard the review filed against these rules is NOT capture-side: a tree-sitter query has no scope knowledge, so it cannot know `Map` is bound by the enclosing `template <...>` header, and the PRE-EXISTING `type: (type_identifier)` rule already captures a bare `T item;` and erases it the same way. It is handled by the lexical ranking in `walkers.ts` in this series. Mutation proof: reverting this file fails 9 of 32 assertions (all eight qualified spellings return no capture) while every over-match negative still passes; restoring gives ALL PASS. Bench `--check` passes for all 15 languages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * test(resolution): pin specialization order, shadowing and the untested spellings (#2833) Grows the generic-field matrix 56 -> 114 tests, closing every coverage gap the #2855 review named and turning the fix-agents' scratch evidence into permanent rows. The rows that discriminate against the resolver fix (they fail if `walkers.ts` is reverted): - C++ specialization must not depend on DECLARATION ORDER: the forward-declared-primary/specialization-first arrangement must land on the primary, same as the mirror arrangement. Plus a cross-case property asserting the two independently built fixtures agree. - Partial specialization is deterministic in both orders. The note says explicitly that selecting `Vec<T*>` would need argument deduction and that flipping this row later is a deliberate expansion, not a regression fix. - Lexical shadowing: the namespace-local `N::Box<bool>` wins for a field inside `N`, and the global specialization wins at global scope. The NON-REGRESSION rows are load-bearing — they are why two of the three proposed remediations were rejected: cross-file C++ specialization binding, and C# `partial class Repo<T>` split across two files with the field in a third (two legitimate unspecialized defs under one name). Coverage the review found missing: C++ pointer and reference generic fields (two of this PR's three original rules had ZERO coverage); all six qualified patterns plus the depth-3 boundary pinned as empty; TS/C# multi-arg container collision; an anti-vacuity sibling for `neg-bounded-type-parameter`; Swift/Dart rows restructured so the ANNOTATION is the only possible source (the old rows gave the field an initializer of the same generic type and could not tell which resolved); and cross-file, inheritance/MRO, import-alias, static-member and the TypeScript module-hoist branch. Six things were measured and pinned AS MEASURED rather than asserted as wishes, each flagged in its row note: a static/class-level member emits nothing for generic AND non-generic alike (a static gap, not a generics one); a cross-file C++ primary template does not bind while the cross-file specialization does; `std::unique_ptr<Payload>` types to `unique_ptr` rather than `Payload` (smart-pointer transparency is not applied on the qualified path); two same-named C++ specializations in one file collapse to one node id; and the container-name collision (`Map<string, User>` binding a workspace `class Map`) is recorded as INTENDED, since the annotation does name that class. The `new Set(...)` dedup was kept rather than narrowed: a per-case surplus-edge sweep measured ZERO duplicate edges anywhere in this file, Swift included, so the quirk that justified a blanket dedup does not reproduce. The sweep now pins zero surplus per case, so a real double-emit fails instead of being absorbed. The file is deliberately NOT split: four assertions compare cases against each other, cost is linear in cases, and the 1,800,000 ms `beforeAll` is kept because the same run measured 271-428 s depending on host load — a tighter bound converts contention into a red suite. The reasoning is recorded in the file header. Also corrects the SCHEMA_BUMP pin-test title, which still said (#2766). Mutation proof: reverting `walkers.ts` fails exactly the five order and shadowing assertions and passes the other 109; restoring gives 114/114. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * feat(resolution): capture declared type parameters so a type variable is not a class (#2833) Three review findings were blocked on one missing fact. `templateArguments` records the arguments a declaration was written AGAINST (`struct Vec<bool>`); nothing recorded the parameter list a declaration DECLARES (`template <class T>`, `class Box<T extends Repo>`). So the resolver could not tell a type variable from a class, and: - `class Box2<T> { t: T }` beside a workspace `class T` emitted a FALSE edge `run2 -> T.foo`. `T` carries no type arguments, so it never entered the generic branch — the plain lookup simply bound a same-named class. The lexical grounding added elsewhere in this series cannot help, because `export class T` IS lexically bound. - `class Box<T extends Repo> { t: T }` resolved to nothing: no recorded bound to resolve through. - A full specialization `template<> struct Vec<T*>` and a partial `template<class T> struct Vec<T*>` were byte-identical (`['T*']`). `SymbolDefinition.typeParameters` now records `{ name, bound? }` in declaration order (substitution is positional). `bound` is kept verbatim and un-split, so `Repo & Closeable` stays whole; ABSENT means UNKNOWN, never "unbounded", which is what keeps unconverted languages behaving exactly as before. Transport is the raw parameter-list node via `@declaration.type-parameters`, read by a language-neutral parser that recognizes TOKENS, not languages: `extends`/`:` introduce a bound, the name is the trailing identifier, so `class T`, `typename T`, `in T`, `out T`, `reified T` and `class... Ts` are one rule. Populated for TypeScript, C++, Java, Kotlin, C# and Rust. JavaScript, C, COBOL, PHP and Ruby have no declared type parameters to capture; Go and Python spell them with SQUARE brackets, which this parser deliberately rejects as ambiguous against subscript and array spellings (Go already has a working main-thread sidecar in this series); Dart and Swift are straightforward follow-ups. Two latent hazards found and closed on the way: - The new capture was not in `KNOWN_SUB_TAGS`, so it could out-span its own declaration and become the anchor — silently DROPPING the whole class def. - A templated C++ struct matches both the standalone and `template_declaration` patterns, minting two defs under one id, and only one twin could see the parameter list. `buildDefIndex` is first-write-wins, so MATCH ORDER decided whether `Vec` remembered `T`. A narrow duplicate-declaration backfill gives both twins the list. Also fixed by its own test: a Rust lifetime `'a` parsed as a parameter named `a`, which would have shadowed a real class. Parse-time output lands in the cached ParsedFile, so SCHEMA_BUMP goes 46 -> 47. Re-checked against origin/main at write time: main is on 45; 46 was taken by this same branch, and a warm cache stamped 46 carries ParsedFiles with no `typeParameters` at all. The csharp and rust capture goldens were regenerated with the tests' own documented `UPDATE_GOLDEN=1`; only digests moved, no captureGroups. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * fix(resolution): ground erased base names, and stop a class name from being enough (#2833) The review's central risk was that this PR converts MISSING edges into CONFIDENTLY WRONG ones. Base-name erasure (`Repo<User>` -> `Repo`, `Repo[User]` -> `Repo`) bound through a workspace-wide qualified-name fallback that consults NO scope, NO import and NO module — it bound any name with exactly one workspace def. That is why a Python `Mapped[User]` could bind an unrelated `class Mapped`, and why the language deny lists were papering over an open universe. `resolveErasedBaseName` now admits an erased base on one of four grounds, strongest first: the scope chain binds it; the declaration is in the SAME FILE; the index proves the name is a template family; or the file binds no cross-file class at all, so its silence is no evidence. The last ground fails toward permissive on purpose — every way it can be wrong costs a wrong edge that already existed, never a working one. Two measurements drove that design and refuted the simpler rule. A C++ `#include` materializes NO binding whatever, and C# resolves cross-namespace without `using` through the index — so a pure "require lexical grounding" rule would have deleted every cross-file C++ generic member. Both are now pinned. Python erases at CAPTURE time, so by resolution there is no `<` and the grounded route was never entered. `erasedTypeApplication` rebuilds the application from `TypeRef.declaredSpelling` — strictly: the raw name must be the base and the argument list the whole balanced remainder, so `User[]`, `vector<Item>` and `Repo<User>?` decline and behave exactly as before. Closing it took finding FOUR emitters, not one. Three were in Case 4; the fourth was `emitReferencesViaLookup` re-emitting the refused edge from the pre-resolved reference index, which needed the site marked handled with a recorded `receiver-unresolved`. A fifth lived in the text cascade: a declined fold falls THROUGH by design, and the cascade held its own ungrounded copy of the member-typing lookup. This file typed a receiver from a `TypeRef` in five places and the PR had wired three; all five now go through one `classOfDeclaredType`. Also here, from the same review: - Type parameters no longer bind a same-named class (uses the new `typeParameters`), and a BOUNDED parameter resolves through its bound. - A cross-file C++ PRIMARY template now binds: a ranking bug, not a capture one — the index fallback needs exactly one candidate and `Vec` held two, so removing the argument-pinned declaration leaves one. - `this->field.m()` resolved to nothing for generic AND non-generic alike. A language that declares `this` IS the enclosing class (`resolveThisViaEnclosingClass`) synthesizes no `this` typeBinding, so a chain whose BASE is `this` could never seed its head. Reading the provider flag keeps the rule language-free. - Class-level (static) member receivers emit nothing in TypeScript and Kotlin — for the non-generic control too. Case 6 types them from the DEF side (`isStatic` + `declaredType` on the field node), which needs no capture change; the target lookup stays the ordinary instance walk, so a static field HOLDING an instance still binds an instance method and a genuine static call is untouched. Partial-specialization SELECTION is deliberately not implemented: it needs argument deduction against a parameter list, and full C++ partial ordering is a real algorithm with no measured driving case. The discriminator now exists if someone wants it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * fix(cpp,js,php,go): close the remaining per-language generic-field gaps (#2833) Four language gaps the review measured, each with a different cause. **C++ qualified member fields.** `std::vector<Item> items;`, `ns::Repo<User> r;` and `ns::Address addr;` captured NOTHING: every field rule required the type node to BE a `type_identifier` or `template_type`, and a qualified member type is neither — tree-sitter wraps both in a `qualified_identifier`. Three depth-agnostic rules (one per declarator shape) now match the outer node, which also REMOVES the depth boundary rather than raising it: depths 1-4 capture, generic and non-generic alike. Preserving the qualifier resolves nothing — measured: `ns::Repo<User>` binds neither way, because the dotted-tail fallback splits on `.` while C++ writes `::`, and `ns::Repo` is not an index key. Since a capture is a NODE and not synthesized text, the qualifier is dropped in `interpret.ts` by a top-level-only `::` split, so `std::vector<std::string>` reduces to `vector<std::string>`, not `string`. Measured cost of the non-generic half, which was the reason to hesitate: field captures go 8 -> 32 across the C++ bench corpus, but the resolution-level census over those 13 repos is 32 CALLS edges before and 32 after, BYTE-IDENTICAL. It fabricates only where a workspace class shares a std name (`class string` beside `std::string name;`), which is the same accepted policy the already-landed qualified-generic rules carry, pinned in the matrix as intended. **JavaScript `@type {Repo<User>}` and PHP `@var Repo<User>`.** Neither bound a field type — and neither did the NON-generic control, so this was a docblock gap rather than a generics one. PHP needed TWO captures, not one: with only the type binding, `$this->repo->save()` resolved until a second class declared `save` and then went unresolved, because narrowing a same-named method needs the receiver's member owned. Generics do NOT come free in PHP — `normalizePhpType('Repo<User>')` returns `'User'` by the container-element convention, so passing the raw spelling through would have emitted `User::save`; type arguments are erased at capture instead. In JavaScript they DO come free, verified byte-identical to the TypeScript control. Both decline what they cannot prove: arrays, `list<User>`, unions, `Promise`/`Array` wrappers (via an exported predicate rather than a copied name list), statics, and any property that already has a native type. **Go generic interfaces.** `UserRepo` genuinely DOES implement `Repo[User]` — the spec says a generic type must be instantiated, that instantiation substitutes type arguments and yields a new non-generic type, and that a type implements an interface when it is in its type set. So the old behaviour was a FALSE NEGATIVE and the matrix note calling it "already correct" was wrong. Satisfaction is now checked against POSITIONALLY SUBSTITUTED method sets, so `Repo[Order]` does not match a `Save(x User)` implementor — substitution, not erasure. #2829's exact method-set model is untouched: pointer receivers still follow MS(*T), unexported names stay package-scoped, the declaration's own method set is still checked first, and the harvest is gated so a repo with no generic interface never runs it. `go.test.ts` is unchanged at 296 passing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * test(resolution): pin every fix from the review, 114 -> 155 rows (#2833) Eight rows in this matrix pinned gaps that the fixes in this series close, so each asserted the opposite of the new truth. All eight are flipped, and the prose describing them as open gaps is corrected. Nine new cases cover the fixes that would otherwise have shipped unpinned. Flipped, each measured: the type-parameter FALSE edge (`run2`) is gone; a bounded parameter now resolves through its bound with fan-out; the cross-file C++ primary binds; the C++ qualifier depth boundary is removed rather than raised; Go gains its two structural implementors and JOINS the paired sweep, which had quietly excluded it — that exclusion was the taxonomy admitting a bug; and both static-member rows resolve. Added: JS `@type` and PHP `@var` docblock fields with three PHP declines; a Kotlin `companion object` receiver (given an INTERFACE control so the paired sweep can check it, which `ts-reach-shapes` cannot — its two sides are not count-comparable); the Python third-party grounding refusal plus the ground that still ADMITS, so an empty row can never be read as "erased names never resolve"; the four mirrors that would break if grounding were tightened (same-file and imported Python, a C++ `#include`, C# cross-namespace without `using`); C++ qualified non-generic fields including the fabrication policy and its absence case; `this->field.m()` for generic and non-generic with bare controls; and a Go negative proving substitution is positional, not erasure. Three shapes are pinned AS MEASURED with notes saying they are deliberate limits so nobody "fixes" them by accident: C++ partial-specialization selection is deterministically the primary (real selection needs argument deduction); `std::unique_ptr<T>` types to the pointer, not the pointee (`.` and `->` are indistinguishable to the resolver, so transparency would trade a recoverable miss for a confident wrong edge); and two same-named C++ specializations in one file collapse to one node id, which is why the shadowing fixture uses two files. One row pins a REMAINING wrong edge rather than hiding it: `m.inner.ping()` on a `Mapped[User]` head still binds the unrelated workspace class, while the one-segment-shallower `m.save(u)` correctly declines. The obvious one-line guard was written and MEASURED not to close it, so the surviving route is elsewhere and wants its own diagnosis — a broader refusal would change chain-head resolution for every language without pinning the shape it is meant to fix. `bench/scope-capture` is rebaselined for the six languages whose captures moved, regenerated from a fresh measurement rather than pasted; `--check` passes for all 15. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * perf(resolution): remove three measured hot-path regressions this series added (#2833) A quality pass over the #2833 series found three performance defects it had introduced, all measured, plus dead code and stale docs from six agents having appended to the same files across four rounds. No behaviour change: the resolver suite is identical before and after, and every scope-capture fingerprint is byte-identical. **An accidental quadratic in Go instantiation harvesting.** `collectGoInstantiations` calls `record()` for every type binding and every declared, return and parameter type in every Go file, and the `includes('[')` gate does not filter Go's most common types — `map[string]string`, `[]map[string]*v1.Pod` and `map[string]map[string]int` all produce a `map` candidate. Each false base then failed a full scope-chain walk and fell through to a LINEAR SCAN OF EVERY INTERFACE IN THE PROGRAM, with no dedupe on the spelling, so the same `map[string]string` written 10,000 times paid 10,000 scans. Now a qualified-name index built in `buildDetectionIndexes` (one probe, ambiguity semantics preserved exactly) plus a per-scope base memo: 8,000 interfaces / 80,000 spellings: 6,662 ms -> 104 ms (64x) `resolveEmbeddedInterface` held a byte-identical copy of that scan and now shares the helper. `GoInstantiation` was a single-field wrapper and collapses to the array it wrapped; its two parallel maps fold into one whose inner key IS the dedupe. `candidateStructIdsFor` was rebuilt per instantiation although every substituted method set has the same key set — hoisted, and materialized, because one branch returned a live iterator that would have yielded nothing on a second pass. **`scanForCrossFileClass` asked a name-keyed question that needs no name key.** It answered "does this file bind any cross-file class" by probing every accessible namespace once PER NAME. It now iterates the channels directly, taking whichever side is smaller so a large namespace table cannot reintroduce the product. Predicate and early exit preserved: 5,000 module names x 1,000 namespaces: 159.0 ms -> 1.2 ms (132x) **A duplicated scope walk on every generic receiver.** `resolveClassBindingForName` computed the lexical candidate list, then `resolveErasedBaseName` recomputed the identical `findAllBindingsInScope`. Computed once and passed: receiver at depth 8: 5,617 ns -> 3,091 ns (-45%) **A whole extra AST traversal per JavaScript and PHP file.** The docblock synthesis passes each added a full tree walk to find one node kind — the ninth in the JS emitter, the third in PHP. `node.namedChildren` materializes a wrapper array across the N-API boundary for every node, so one added pass cost 1.9x what parsing the entire file costs. Folded into the existing walks as one more node kind; capture output is byte-identical and every fingerprint is unchanged. Total emit time per file drops 4-7%. Hygiene, all verified stale rather than assumed: - `receiverOriginOpts` passed `resolveThisViaEnclosingClass`, which `classifyReceiverOrigin` never reads — the "both hooks" comment above it is true again. - The `stripDecoration` docstring's caller roll-call claimed the only edge-emitting caller "emits no edge and can only change a diagnostic label". Case 6 passes it and does emit edges. Replaced the roll-call with the rule; six rounds each appending a name to a list is how it went wrong. - A Python comment described the resolution-time grounding as a follow-up that "this parse-time pass cannot do" — it landed in this same branch and is pinned by `py-erased-grounding`. - `classOfDeclaredType` took a `scopeId` all five callers derived from the `TypeRef` they also passed. Dropped, so "these five are the same call" is enforced rather than asserted. - Three exports with no consumer outside their own file. - PHP had three copies of one preceding-comment sibling walk and two regexes for one tag, so a fix to either reader of `@var` would land on one and not the other — the symptom being a field typed differently from its own foreach element type. One walk, one regex. Tests: the new matrix leaked a fixture repo per case; it now carries the sibling suite's `cleanupTempDirSync` and the Windows EBUSY reasoning that goes with it. `PAIRED` was a second hand-maintained list and 19 of 41 cases had silently fallen out of it — it is derived from the cases now, with a new assertion that each case is either swept as a pair or carries a written reason it is not. That recovered one genuine omission (`php-typed-property`). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KtNfG6EPn738Y51AYs7wDp * test(bench): rebaseline receiver-resolution for the #2833 this-> fix The `Receiver-resolution drop guards` CI step failed on this branch: shapeArm.cpp.fieldReceiverCall: "INVISIBLE-GAP" -> "RESOLVES" shapeArm.cpp.decoratedFieldType: "INVISIBLE-GAP" -> "RESOLVES" Both are the intended improvement. The guard is exact-match by design — the drop count cannot move without a deliberate rebaseline, and the rebaseline path demands the movement be explained — so this records the two shape flips and leaves the call-drop count arm untouched. BASELINE.md still claimed `this->repo.save()` and `this->repo->save()` were INVISIBLE-GAP. That is now false: the `resolveThisViaEnclosingClass` head seed added in this PR resolves both. Also notes what the control established — this was never a generics gap, since the non-generic control failed identically before the fix. * docs(parse-cache): narrow the SCHEMA_BUMP ledger to what the bump delivers The ledger claimed a warm cache would make "the whole fix ... a silent no-op on every incremental analyze". That overstates the constant. The bump invalidates the PARSE half; whether the re-parsed captures reach the graph is gated separately and does not move: - `isIncremental` (core/run-analyze.ts) tests `!options.force`, an existing meta, `!schemaFingerprintMismatch(...)`, feature parity, non-empty `fileHashes` and a git repo. SCHEMA_BUMP is in none of them. - the incremental branch writes back only `hashDiff.toWrite` and logs the rest as "unchanged file rows preserved". - SCHEMA_FINGERPRINT hashes node/relation DDL, untouched here, so it is byte-identical and moves nothing either. So an incremental analyze re-parses an unchanged file correctly but keeps its existing rows; the new edges land on the next full rebuild. That is the pre-existing contract for every capture change, not a regression in this PR — but the comment should not promise more than it delivers. Comment only; no behavior change. SCHEMA_BUMP stays 48. --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
azizur100389
|
1fa751d76d
|
fix(spring): extract method-level RequestMapping routes (#2857)
* fix(spring): extract RequestMapping route methods Co-authored-by: Cursor <cursoragent@cursor.com> * fix(spring): address RequestMapping review findings * fix(spring): accept trivia in request methods --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
e69d3c49c4
|
fix(analyze): gate FTS-indexed DML before the incremental writeback (#2841) (#2854)
* fix(lbug): never report a drop that could not happen, and gate FTS-indexed DML `CALL DROP_FTS_INDEX` is itself an FTS-extension function, so with the extension unloaded it fails with `Catalog exception: function DROP_FTS_INDEX is not defined`. `isBenignDropFtsIndexError` classifies that as "nothing to drop" — correct when the index does not exist, wrong when it does: the drop silently no-ops and the next write to that table dies at bind time with an engine message that never mentions FTS (#2841). The classifier stays pure (a message cannot tell you whether an index is live). Instead `dropFTSIndex` settles liveness with a catalog read on the ERROR path only and raises an FTS-named, remedy-bearing error when the index is present but undroppable. Adds `ensureFtsRowDmlSafe`, the FTS twin of `ensureEmbeddingRowDmlSafe` (#2623): catalog first, load FTS with the analyze policy only when an index actually gates DML. LadybugDB refuses that DML at BIND time — a DETACH DELETE matching zero rows fails exactly as hard as one matching thousands — and the indexes cannot be cleared in place, so a verdict is the only useful answer. Both gates now share one `SHOW_INDEXES` read via `readIndexCatalogRows`, so adding the FTS check costs no extra catalog round-trip. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): escalate instead of crashing when FTS blocks incremental DML The incremental writeback decided its write plan without ever asking whether row-level DML was legal. On a DB carrying FTS indexes with an unloadable FTS extension, `deleteNodesForFiles` then died mid-writeback: Binder exception: Trying to delete from an index on table File but its extension is not loaded. with no mention of FTS anywhere in the run — the only install-capable load happened in Phase 3, long after the writes (#2841). The incremental branch now reads the index catalog once and derives both extension verdicts before any DML. When FTS (or VECTOR) blocks in-place writes, the run falls through to the existing wipe-and-bulk-COPY escalation — the same answer #2623 gave for VECTOR, and the only one available, since the indexes cannot be dropped without the extension. Every blocked extension is named in the reason log, not just the first one checked: a DB can carry both a vector index and FTS indexes, and reporting half the cause is how this failure stayed mis-diagnosed. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): cover the FTS DML gate, both-blocked escalation, and the drop guard New `incremental-index-extension-dml-gate.test.ts` drives the real `runFullAnalysis` against a real mini-repo and a real LadybugDB: - a DB carrying FTS indexes with FTS made unloadable escalates to a full DB write, names FTS in the log, ends with zero FTS indexes, and still has the newly committed content in the graph (pre-fix: Binder exception, exit 1); - FTS available keeps the surgical plan and the indexes; - a DB that never carried FTS indexes is not escalated (the catalog-first check must not tax FTS-less machines); - FTS and VECTOR both blocked produce ONE escalation naming both. `drop-fts-index-error-classification.test.ts` gains the two `dropFTSIndex` cases the #2841 guard turns on: live index + unloaded extension rejects with an FTS-named error, absent index still resolves. The existing classifier assertions are unchanged — it stays pure. The CLI e2e reproduces the reporter's exact journey (analyze with the extension, remove it, touch a file, analyze again) and asserts exit 0 plus an FTS-named reason. It skips visibly when the seeded extension cannot load on the host, so it can never report a false red about the fix. Mutation-verified: reverting the run-analyze gate fails the first scenario; reverting the dropFTSIndex guard fails the live-index case. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(lbug): make every catalog-gated path fail closed, and classify the drop remedy Review findings on #2854 (two-engine, 17 lanes). H3 — `ftsIndexExistsInCatalog` returned `false` when the catalog could not be read, i.e. "index absent", so `dropFTSIndex` swallowed the error and the caller proceeded as if the index were gone. That is the #2841 symptom the guard exists to make loud, and it contradicted the contract `readIndexCatalogRows` states two functions above. It now fails closed. §6.A — `ensureFtsRowDmlSafe` keyed on `index_type === 'FTS'`, which answers `undefined === 'FTS'` → false → *no gate* for a row whose shape cannot be read: fail-open, in the gate whose only job is preventing an unsafe write, while the VECTOR twin fails closed on the same input. Now only a positively-identified non-FTS index is waved through. Deliberately NOT the twin's `!== 'HASH'`: that is safe there only because it is scoped to the embedding table first, and this gate is table-agnostic — `!== 'HASH'` would let the HNSW index gate FTS DML. §5.A — `undefined` was overloaded: "caller passed nothing" and "caller tried and could not prove anything" shared one value, so a failed shared read silently became three reads and the two gates could decide from different snapshots. The failed snapshot is now representable (`INDEX_CATALOG_UNREADABLE`), leaving one unambiguous `??` in `resolveGateRows`. §5.B — both gates regained the unconditional null-connection precondition the refactor moved into the reader. §5.G — the throw's remedy now routes through `diagnoseExtensionLoad`, like `--repair-fts` and `ftsDegradedWarning`, so a missing runtime dependency is not told to reinstall. The message stays path-free (#2374/#2375). The dead positional row fallbacks are kept and marked `LADYBUGDB-CONTRACT`: removing them would turn a proven-inert hedge into a fail-open gate if a future engine returns unnamed tuples. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): never undo an explicit wipe, stage extension-forced rebuilds, report honestly Review findings on #2854 (two-engine, 17 lanes). H1 (P1, both engines) — `analyze --drop-embeddings` was silently reverted. The `--drop-embeddings` → `force` conversion sits inside the `embeddingCheckpoint` branch, so without a checkpoint the run stays incremental and reaches the gate; the flag then *deliberately* leaves `cachedEmbeddings` empty, which is exactly the rescue's trigger, so every row the operator asked to destroy was read back and restored, exit 0. Widening the rescue from `!embeddingRowDmlSafe` to `extensionForcedRebuild` moved that latent bug onto the dominant path, because every analyzed DB carries FTS indexes. Guarded on the flag itself — NOT on `shouldLoadCache`, which is false in the meta-under-reports case the rescue exists for and would have deleted the safeguard while fixing the wipe. The `--drop-embeddings --embeddings` variant is covered by the same guard. H2 — an extension-forced escalation wiped the LIVE index in place: `buildPath` was frozen ~440 lines earlier while the run was still classified incremental, so an interrupt or ENOSPC left no complete index, where main failed at bind time with it intact. Extension-forced rebuilds now build into a staging file and publish via the existing atomic swap; size-forced ones stay in place, since that trigger is the repo's own churn rather than a machine condition. H5 — the escalation log asserted a vector index "exists" and that the store "carries FTS indexes" in exactly the case the catalog read proved nothing, while the only truthful signal went to stderr rather than the IPC log. It now emits a distinct unreadable-catalog cause, and "this index carries" (which pointed at the vector index just named) reads "the graph store carries". §5.D — the write-set cause was dropped whenever an extension cause co-occurred; causes are appended now, not selected between. §5.C — after an FTS-forced rebuild stamped lastCommit, a plain rerun on the same commit hit the alreadyUpToDate fast path before Phase 3, so the CLI's "install … then rerun" advice could never restore FTS. The fast path is now bypassed when meta records FTS unavailable and the extension can load again, keyed on the persisted capabilities stamp rather than new state. §5.F (skip the escalation for a zero-change commit) is deliberately NOT implemented: `deleteSpringAutoConfigurationSyntheticClasses` and `deleteSpringAopEvidenceNodes` run unconditionally on the surgical branch and bind against FTS-indexed `Class`/`CodeElement`, and a zero-row DETACH DELETE fails at bind time exactly as hard as a large one — so the skip would restore the original crash. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * perf(search): read the index catalog once per drop sweep, and state the real contract Review findings on #2854. H4 — on a machine where FTS cannot load and the DB carries no FTS index, the gate correctly returned early without loading the extension, but the surgical path still ran the full 20-entry drop sweep: every `CALL DROP_FTS_INDEX` raised "function DROP_FTS_INDEX is not defined", and the new liveness guard then fired a fresh catalog read per table — 20 reads every run, forever, for exactly the offline/load-only population, contradicting the "healthy path costs nothing" claim shipped with the guard. The sweep now reads the catalog once and skips entirely when no FTS-typed index exists. An unreadable catalog runs the sweep, so an unprovable catalog never skips real work. H8 — the docstring still promised `dropFTSIndex` "tolerates" an unloadable extension. Post-#2854 a live index plus an unloadable extension throws, and safety rests on caller ordering discipline rather than the type system — which is what would have talked the next caller out of that ordering. GUARDRAILS — the "switching to a full DB write" sign described exactly one trigger (write set >~50%). Since #2623 and #2841 an unloadable extension escalates regardless of write-set size; documented with its recovery steps. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): cover the wipe guard, the staged rebuild, and the fail-closed branches Review findings on #2854. H1/H2 mutation-verified: removing `!options.dropEmbeddings` fails the new drop-embeddings case ("expected true to be false"); disabling the staging upgrade fails the staging case ("expected 0 to be greater than 0"), so both assert behaviour rather than describe it. Gate suite (7 cases): `--drop-embeddings` under an FTS-forced escalation ends at zero embedding rows and logs no "Preserving"; the escalation is one-shot — a third run on a healthy host returns to surgery and rebuilds every FTS index; an extension-forced rebuild is observed building into `lbug.staging.*` and leaves none behind; the rescue complement still preserves un-stamped rows when no wipe was requested; the never-built case now asserts the commit reached the graph. H6 — the both-blocked case hard-asserted `createVectorIndex()` while the suite probed FTS only, so it went red on any FTS-yes/VECTOR-no host. VECTOR is probed now and gates only that case, with a GITNEXUS_REQUIRE_VECTOR hard-fail. H7 — the fail-closed branches had no coverage although the VECTOR twin's test and interception technique were ready to copy: `ensureFtsRowDmlSafe` under an unreadable catalog now proves it routes to the load, and `dropFTSIndex` proves it rejects rather than silently tolerating. Plus a redaction case that forces a real path-bearing load failure — under policy `never` the assertion would have been vacuous, since that reason carries no path. §5.E/§6.B — the suite is registered in the cross-platform matrix (its sibling was; it wasn't, and GITNEXUS_REQUIRE_VECTOR is set only on that job) and moved into the sequential lbug-db project per TESTING.md:68, verified not to drop it from the sharded ubuntu job. A Windows shard weight is added as a labelled estimate — the 8s floor would skew the split it exists to protect. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * refactor(analyze): make the FTS gate's fast path cheap, its claims provable, and its remedies classified Cleanup review of the #2841 work (four parallel angles: reuse, simplification, efficiency, altitude). Behaviour-preserving except where the previous behaviour was wrong. Correctness the review caught: - The fast-path probe keyed on `capabilities.fts.status === 'unavailable'`, which collapses "extension unavailable" and "index build failed". A deterministic build failure (an un-tokenizable row, #2544) therefore bypassed `alreadyUpToDate` on EVERY subsequent run, re-analyzed the whole repo, failed the same way, and restamped — a permanent loop where the run used to be one `stat`. Phase 3 already computes the discriminator; it is now persisted as `fts.skipReason` and the probe only runs for `extension-unavailable`. Metas written before this carry no field and keep today's behaviour. - `dropSearchFTSIndexes` skipped its sweep when no row read `index_type === 'FTS'`, while `ensureFtsRowDmlSafe` treats an unreadable type as "might be FTS". Opposite polarity, under a comment claiming they matched: a row-shape change would let the gate wave the surgical plan through while the sweep dropped nothing, putting DELETEs back on tables carrying live FTS indexes — #2589 again. The sweep now decides per configured index on identity, which is also strictly more precise. Its old justification (leftover indexes under other names) was unreachable — the loop only ever drops configured entries. - `dropFTSIndex` threw "FTS index X on table Y exists" on the one path where the catalog could not be read — a fabricated claim, on a DB the same run had just shown carries no FTS index. Presence is now `present | absent | unverifiable` and the message says which. - The remedy was hand-written for three of the four load-failure classes, discarding `missingFileRemedy`/`corruptFileRemedy`, so a corrupt extension file was told to retry an install — the misdirection #2383 fixed. Both the drop error and the escalation log now use the classified remedy. Cost, measured on a 391 MB index (cold open ~1 s, SHOW_INDEXES ~4 ms): - The probe opened the live index WRITABLE on the millisecond fast path, dragging in schema DDL, the cross-process write lock, sidecar reclaim and a CHECKPOINT on close. It is read-only now. That also closes an install trap: `doInitLbug`'s pre-load resolves the env policy on the writable branch, so an operator following our own `GITNEXUS_LBUG_EXTENSION_INSTALL=auto` advice paid a forked 15 s installer on every up-to-date run (memoized per process; the CLI is a fresh process each time). The read-only branch pins `load-only`. - A failed staged rebuild orphaned a full index-sized copy until the next lock sweep; the failure path now reclaims it. - The sweep re-read a catalog the run already held, defeating the invariant the snapshot type exists to enforce. Structure: row-shape accessors have one home, so the LADYBUGDB-CONTRACT grep claim is true by construction; staging now applies to both escalation causes, since recoverability is a property of the wipe-then-COPY plan, not of the trigger; `getExtensionCapability`/`getFtsCapability` replace hand-spelled lookups where the seam allows. Two lookups in run-analyze.ts deliberately keep the exported `getExtensionCapabilities()` form: the #2383 tests stub that export, and an ESM module mock does not intercept a helper's internal call — routing through it silently degraded the classified remedy to generic text. Recorded in-comment. Not taken, deliberately: extracting the escalation message and replacing the snapshot protocol with a connection-scoped catalog memo (both sound, both restructure code this PR just stabilised — they belong in their own change); an extension registry (premature at two instances, and the FTS/VECTOR polarity difference is exactly what it would have to parameterize back out). Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): pin both sides of the degraded-FTS fast-path bypass `healDegradedFts` (§5.C) had zero coverage — three separate review angles flagged it, and the cleanup pass then found it sat one conjunct away from a permanent full-re-analyze loop. Both sides are pinned now: - it re-analyzes past `alreadyUpToDate` when the stored meta says FTS is degraded and the extension loads again: run 1 analyzes with loads blocked (asserting the precondition — `status: 'unavailable'`, `skipReason: 'extension-unavailable'` — rather than assuming it), then a same-commit clean-tree rerun rebuilds every FTS index without a file changing; - it stands down when the degradation was a BUILD failure: the stored `skipReason` is rewritten to 'build-failed' and the rerun must take the fast path, because that rebuild would fail identically on every run forever. The build-failed state is reached by rewriting the stamped discriminator, not by provoking a real tokenizer failure: a genuine one needs a stored row the native tokenizer rejects (#2544/#2546), which is neither portable across the CI matrix nor deterministic, and §5.C reads only that field. Also folds the first escalation case into the one-shot case. The claim that it was fully subsumed did not hold on audit: `logs` containing 'FTS' was unique as expected, but so was the duplicate-File-node row count — every other reader goes through a Map keyed by path, which collapses a stale twin an appending rebuild would leave. Both assertions moved rather than one being dropped. Net suite runtime goes UP (two cycles removed, four added), against the cross-platform-matrix argument that motivated the dedup — recorded here because the shard weight is an estimate pending a real Windows measurement. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(search): keep the whole-module adapter mock in step with the row accessors The cleanup pass moved the LadybugDB row-shape reads behind named accessors so the column contract has one home. `fts-indexes.test.ts` mocks the entire adapter module with a hand-written factory, which still exposed only the three exports the file imported before — so `verifySearchFTSIndexes` failed with "No `indexRowName` export is defined on the mock" while production was fine. The added accessors mirror the real implementations rather than returning stubs. A stub would have read `undefined` out of every catalog row and let the suite pass for the wrong reason — the failure mode a whole-module mock invites whenever the module under test grows an import. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * revert(analyze): drop the degraded-FTS auto-heal, fix the advice it existed to justify §5.C's complaint was that the CLI tells users to "install the extension … then rerun" when a rerun lands on the up-to-date fast path and rebuilds nothing. The answer shipped for it was a probe that bypasses that fast path. Four independent problems later, the sentence is cheaper to fix than to make true: - it could not tell "extension was missing" from "index build failed" without a stamped discriminator, so a deterministic build failure (#2544/#2546) re-analyzed the entire repo on every invocation, forever, where the run used to be one `stat`; - it opened the live index on the millisecond fast path — writable at first, dragging in DDL, the cross-process lock and a CHECKPOINT (~1 s on a 391 MB index), and even read-only it is a full open; - `doInitLbug`'s pre-load resolves the env policy, so an operator following our own `GITNEXUS_LBUG_EXTENSION_INSTALL=auto` advice paid a forked 15 s installer per up-to-date run; - and it turns the fast path into a full re-analysis whenever an index authored where FTS was unavailable is later read where it loads — a legitimate, common state, and the invariant `analyzer-identity-cli.test.ts` pins. So: no probe. The degraded-search warning now points at `gitnexus analyze --repair-fts`, which rebuilds the search indexes without re-parsing the repo, instead of "then rerun". One line, no new failure modes, and it is what the issue actually asked for. `capabilities.fts.skipReason` stays in the meta stamp: it costs three lines, makes the two degradation causes distinguishable for support, and is what any future correct answer here would key on. Also gates the H2 staging assertion on the production predicate. It asserted staging unconditionally while the upgrade requires `posixSwap || windowsSwapOk`, and `windowsSwapOk` is opt-in (#2614) — so it failed on the Windows matrix for a reason unrelated to #2841. Registering this suite cross-platform is what exposed it; the assertion now mirrors the condition it is testing. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): never stage around a damaged index — escalate in place when the catalog is unreadable CI caught this on ubuntu and macOS: `analyze-wal-checkpoint-failure` stopped failing, which is worse than it sounds. That test plants a directory at `.gitnexus/lbug.wal.checkpoint` so the auto-checkpoint's rename target is blocked, and asserts analyze exits non-zero with the `--wal-checkpoint-threshold` hint. But LadybugDB cannot open that path at all, so `CALL SHOW_INDEXES()` now fails with `IO exception: … Is a directory`. The catalog read returns UNREADABLE, both DML gates correctly fail closed, both extension loads fail with the same IO error, and the run escalates — and since the escalation stages, it built a fresh index at `lbug.staging.<uuid>`, swapped it in, and exited 0. The blocked path was never touched. The run "succeeded" while the damage sat untouched on disk, waiting to break the next in-place writeback. So the staging upgrade is now conditional on the catalog having been READ. Staging exists to protect a healthy live index from a machine-level cause (an extension that will not load); it must not be used to route around a damaged one. When we are escalating out of ignorance, build in place so the underlying IO fault lands on the failure path where the operator gets a diagnosis. Verified against the real CLI, not just the suite: with a directory planted at the checkpoint path, analyze now exits 1 and prints `gitnexus analyze --wal-checkpoint-threshold 67108864`. The healthy extension-forced case still stages (gate suite 6/6). Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
a033b04c46
|
fix(go): scope and define each type_spec, not the type_declaration (#2837) (#2843) | ||
|
Gergő Magyar
|
aaa78f9590
|
fix(scope-resolution): fan out interface dispatch from Case 3b receivers (#2832) (#2842)
* fix(scope-resolution): fan out interface dispatch from Case 3b receivers (#2832)
Case 3b (chain-typebinding) folds a receiver through the same
`resolveCompoundReceiverClass` call and the same `[owner, ...mroFor(owner)]`
walk Case 0 uses, but emitted its edge without calling
`emitInterfaceDispatchFor`. When that fold landed on an Interface the site got
one edge to the interface's bodiless declaration and none to any
implementation — the defect #2813 reported for field receivers, in the half
#2829 did not cover.
The gap was a property of how a receiver was SPELLED rather than of what it
resolved to. `d.repo.save()` contains a dot, so it took Case 0 and fanned out;
binding the identical field to a local first — `const r = d.repo; r.save()` —
made the receiver a bare name with a dotted typeBinding, which is Case 3b, and
lost every implementation edge.
`ownerDef` is the receiver's own folded type, matching Case 0's `currentClass`
and Case 4's `ownerDef`, not the owner of the member the MRO walk settled on:
a receiver folding to a concrete class that merely inherits an interface method
must not fan out, because its runtime type is that class. The closure
self-gates on `ownerDef.type !== 'Interface'`, so the call is inert for every
concrete receiver and needs no language check. Confidence is the 0.85 literal
this case's own primary emits, so dispatch edges never outrank the edge they
hang off; Case 4's site.kind-dependent value has no counterpart here because
Case 3b's primary does not vary that way.
The new fixture pins the route as well as the fix. `const r = d.repo` reaches
Case 3b and nothing else can take the site: Case 0 needs a `.`/`(` in the
receiver name or a minted receiver chain, and `encodeReceiverChain` returns
undefined for the empty step list a bare identifier produces; Case 4 excludes
itself on the dot. Before the fix the primary assertion passed while the
fan-out came back empty — the exact "reached Case 3b and stopped at the
declaration" signature.
Resolution-side only: this changes what the resolver produces, not how it is
stored, so no SCHEMA_BUMP applies. An existing index must be re-analyzed to
show the new edges.
Follow-up from #2829.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(scope-resolution): record Case 3b's interface-dispatch fan-out in I4 (#2832)
Invariant I4 documented the fan-out as something "Cases 0 and 4 both perform"
and spelled out Case 0.5's exclusion, while saying nothing about Case 3b —
which is what made 3b's missing fan-out an undocumented asymmetry rather than
a deliberate exclusion someone could defend or point at.
With the fan-out added, Case 0.5 is the only case that folds or walks to a
receiver type without dispatching to implementations, and its exclusion is
gated behind `resolveThisViaEnclosingClass`. Saying so explicitly keeps the
next reader from having to re-derive which cases fan out by reading the pass.
Comment-only; `detect-changes --scope staged` reports no graph change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(scope-resolution): add the concrete-implementor control for Case 3b (#2832)
The Case 3b fan-out shipped with one negative control — a chain folding to
PlainCache, a class that implements nothing. That proves only the weak claim:
no interface anywhere near the site, no fan-out.
Add the stronger negative. SqlRepo implements Repo, so an interface IS in
scope and `save` is a name Repo declares, yet the receiver's folded type is
the concrete class and nothing may fan out. This is the control that fails if
a later change fans out from the interface a member is DECLARED in rather than
from the receiver's own folded type.
The comment says what the control cannot do, too: it cannot catch "member
owner passed instead of folded type" in TypeScript, because an implementing
class always declares the member itself, so the MRO walk never settles on the
interface's bodiless declaration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NdZtWXQJUGB1ZGLH2YNw3o
* docs(scope-resolution): correct four overclaims the review found (#2832)
A multi-lane review of this PR reproduced, against the real pipeline, that
several claims in the comments and one test name assert more than the code
delivers. No behavior changes here — only the text, and one test rename.
1. The test comment gave the WRONG REASON why the concrete-implementor control
cannot catch "member owner passed instead of folded type". It said an
implementing class always declares the member itself; `class C extends Base
implements I {}` is valid TypeScript and inherits it. The real reason is that
TypeScript's MRO chain never contains an implemented interface, so the walk
cannot settle on an interface declaration for a concrete receiver. The
mutation IS expressible where a concrete class inherits a `default` interface
method (Java, Kotlin) — reproduced during review — so this is language-scoped,
not inherent, and a follow-up fixture is tracked.
2. "fans out to every implementation of the folded interface" certified a
completeness that does not exist. TypeScript emits heritage edges for
`class_declaration` only (languages/typescript/captures.ts:749, stated in its
own docstring at :732-733), so `abstract class X implements I` and `interface
B extends A` produce no heritage edge and still dead-end on the bodiless
declaration. Renamed to name the shape actually covered, with a KNOWN GAP
note. The gap is in the capture layer and predates this fan-out.
3. Invariant I4 said Case 0.5 is the ONLY case that resolves a receiver type
without fanning out. Cases 3 and 5 do too, by direct lookup rather than a fold
or MRO walk. The sentence now says which distinction it means and states the
reachability argument (no known language reaches Case 3 with an Interface —
every one that could strips the namespace qualifier first, sending it to
Case 4) instead of implying a completed audit.
4. The gate's rationale claimed the `ownerDef.type !== 'Interface'` test is right
for every non-Interface receiver. An abstract-class receiver also dead-ends on
a declaration-only member and does not fan out. Noted, with why widening the
gate belongs to Cases 0 and 4 across all languages rather than to #2832.
Also completes the module-level case ladder, which still credited the fan-out to
Case 0 alone and omitted it from the Case 3b entry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NdZtWXQJUGB1ZGLH2YNw3o
* docs(scope-resolution): name Case 2 in the I4 exclusion list too (#2832)
The first pass at this correction listed Cases 3 and 5 as the other cases that
resolve a receiver type without fanning out, and was itself incomplete: Case 2
also walks an MRO and its binding admits `Interface`. It is excluded for a
different reason than 3 and 5 — its receiver IS the type name, so the site is
static dispatch and a fan-out would be wrong, whereas 3 and 5 resolve by direct
lookup rather than a fold or MRO walk. Say both rather than enumerate one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NdZtWXQJUGB1ZGLH2YNw3o
* fix(scope-resolution): close the two gaps the #2842 review left open
Both were pre-existing and reached by Cases 0 and 4 as well; the Case 3b
fan-out only widened the population of sites that hit them. Researched against
the real TypeScript compiler and the language service before choosing
semantics, plus how comparable tools draw the same lines.
1. THE FAN-OUT COULD TARGET A STATIC MEMBER
`class C implements I { static save() {} }` does not satisfy `I` — TypeScript
rejects it as TS2420, "Property 'save' is missing" — so an edge from an
`I`-typed receiver to a static member names a target dispatch can never
produce. The closure picked targets with `pickOverload`, which applies no
static filter, while the surrounding cases pick their own primary with
`pickFirstNonStaticOnly`: the speculative edges were picked with weaker rules
than the certain edge they hang off. A same-name static+instance pair also made
`pickOverload` return OVERLOAD_AMBIGUOUS, suppressing the CORRECT edge too, so
this was a false negative as well as a false positive.
Every comparable tool draws this line: tsserver partitions static from instance
results, clangd gates on `isVirtual()` (C++ forbids virtual statics), jdtls
filters abstract-or-static, and class-hierarchy analysis expands only VIRTUAL
call sites.
The guard prefers `provider.isStaticOnly` where a language declares it and
falls back to the graph node's `isStatic`. That order is load-bearing, not
stylistic: the method extractor derives `isStatic` from the OWNER type as well
as the member (`staticOwnerTypes`), and the JVM config lists
`object_declaration` — so reading the flag first would delete Kotlin `object`
implementations, which are singleton INSTANCES and genuinely reachable. Kotlin
is the only hook implementor and marks exactly the companion-promoted set;
Ruby's `singleton_class` (`def self.foo`) is correctly filtered by the
fallback.
2. TYPESCRIPT HERITAGE WAS CLASS-ONLY
`interface B extends A` and `abstract class X implements I` emitted no heritage
edge at all, so the subtype closure had nothing to descend and both shapes
dead-ended on a bodiless declaration — including the very example the closure's
own docstring cites as the reason it exists. Since Case 3b's dotted-alias
binding survives qualifier-stripping only in TS/JS, this was the language that
actually reaches the new path.
The two shapes reach their bases differently: an abstract class carries the
same `class_heritage` child a concrete one does, while an interface's bases
hang off `extends_type_clause` directly. That clause's `type` field is
`multiple: true`, so `childForFieldName('type')` would silently drop `C` from
`interface B extends A, C` — hence iterating named children.
Deliberately NOT structural matching. TypeScript is structurally typed, so a
class satisfies an interface without `implements`, but tsc's own navigation is
declaration-only and says why: "users are typically only interested in explicit
implementations... The type checker doesn't let us make the distinction between
structurally compatible implementations and explicit implementations, so we
must use the AST." scip-typescript reached the same design independently. gopls
does match structurally, but only because Go has no `implements` keyword to
prefer.
Abstract declarations are still walked THROUGH rather than targeted — the rule
everywhere is "does it have a body?", which is what `isDeclarationOnly`
already tests.
VERSIONING. The capture change is parse-time, so a v43 warm cache would serve
entries missing the new matches: SCHEMA_BUMP 43 -> 44 with its pin test moved
in the same commit, verified against origin/main at
|
||
|
Gergő Magyar
|
9372b17049
|
fix(python): resolve calls through an unaliased dotted namespace import (#2826) (#2828)
* fix(python): resolve calls through an unaliased dotted namespace import (#2826) `import pkg.db` followed by `pkg.db.session_scope()` emitted no CALLS edge, while all three sibling spellings resolved. In a codebase whose style guide mandates absolute imports this is close to the only cross-module call form used, so `impact()` reported `impactedCount: 0, risk: LOW, epistemic: exact` for functions with dozens of real callers — a dropped caller reading as a verified all-clear. The resolution path was never missing; one map was keyed on the wrong half of the import. `interpretPythonImport`'s plain arm splits `import pkg.db` into `localName: 'pkg'` (the name Python actually binds) and `importedName: 'pkg.db'`, and finalize carries both onto the edge as `localName` / `targetExportedName`. `collectNamespaceTargets` keyed only on `localName`, but the receiver text captured at the call site is the whole dotted path — Python's query binds the attribute's `object` field with a wildcard, so `pkg.db.session_scope()` yields the receiver `pkg.db`. Case 0 declines it (a module is not a class) and falls through, Case 1 looks up `pkg.db` and misses, and Case 1.5 needs `resolveQualifiedReceiverMember`, which only the C++ provider implements. The site drops silently. Key the map on the dotted import path as well — gated on a provider opt-in, not on the edge shape. The shape alone cannot decide it: Swift's `import Foo.Bar` produces the identical pair (`localName: 'Foo'`, `targetExportedName: 'Foo.Bar'`), but there the FIRST segment is the resolved target and `Foo.Bar` names a nested type. Minting a key for it would hand `resolveConstructionExpressionClass` an authoritative namespace — that branch deliberately does not fall through on a miss — and break `Foo.Bar(x)` construction that resolves correctly today. Hence `ScopeResolver.namespaceReceiverIncludesImportPath`, which only Python sets. The root-segment check on the added key does real work: `import pkg.db as pdb` binds only `pdb`, so writing `pkg.db.f()` there is a NameError, and its edge (localName `pdb`, path `pkg.db`) is correctly rejected. Two same-package imports stay separate — `import pkg.db` + `import pkg.cache` key `pkg.db` and `pkg.cache` independently, so neither call can land in the other's module; the shared `pkg` bucket keeps its existing ambiguity rather than gaining any. Tests: five integration rows (the issue's own repro, the three sibling spellings as controls, non-crossing two-package imports, a three-segment receiver, and dotted construction) plus a unit pin on the keying rule that asserts a Swift-shaped edge mints nothing. All five integration rows fail on the pre-fix tree; the controls pass on both, which is what makes them controls. Resolver integration suite 3024 passed / 1 skipped / 0 failed; scope-resolution unit suite 1446 passed. This changes what the resolver produces, not how it is stored — no schema or version constant applies, and an existing index needs a re-analyze to show the new edges. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(resolution): shadow-test a dotted namespace key by its root segment (#2826) `isNamespaceNameShadowed` walks the scope chain looking for a binding, type binding, lexical name, or owned def named exactly `namespaceName`. Once a namespace key can be a dotted import path, that string never matches anything: `import pkg.db` binds `pkg`, so a local `pkg = Decoy()` shadows the import, but the guard was asked about `pkg.db` and answered "not shadowed". The consequence is not a missed edge but a wrong one. The caller treats a verified namespace as authoritative and deliberately does not fall through to the workspace-wide simple-name heuristics, so an unguarded shadowed receiver resolves construction against the imported module instead of the local value. Test the first dot-separated segment instead. Single-segment names are unaffected — their root is themselves — so every pre-existing row keeps its behaviour. This ships with the key that first routes a dotted name into the guard rather than after it: the previous commit is what makes the defect reachable. The new pin fails on the pre-fix guard (verified by reverting the four comparisons and re-running: 1 failed / 5 passed), so it discriminates rather than merely passing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(python): pin the callee name on the dotted-construction row (#2826) The row asserted only that `builds` reached `pkg/db.py`. That module also exports `session_scope`, so a regression that resolved the construction to the wrong member of the right module would have kept the test green — it pinned the file, not the answer. Assert the exact edge set for the caller instead. Verified against the current tree with a scratch probe: `builds -> Model@pkg/db.py` is the only edge the file produces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(plans): include the #2826 engineering plan in the PR `.gitignore` keeps `docs/*` local because planning output is normally throwaway. Force-added here at the reviewer's request so the plan travels with the work it drove: it records the evidence chain behind the fix, the two places the plan turned out to be wrong, and the follow-ups deliberately left out of scope. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(resolution): make the namespace shadow guard shared (#2826) `isNamespaceNameShadowed` lived module-private in `compound-receiver.ts` with a single caller. The namespace map it guards has three consumers, and the next commit adds the guard to a second one, so it moves to `scope/walkers.ts` alongside the other scope-chain primitives rather than being duplicated. Behaviour is unchanged — this is a move plus documentation. Two notes were added because both are easy to get wrong later: - Fails closed on a missing scope or a parent cycle. For every caller, suppressing costs a missing edge while trusting a corrupt scope chain costs a wrong one, so the bias is deliberate. - It reads `scope.bindings` DIRECTLY rather than through `lookupBindingsAt`, which is the opposite of the fix #2745 applied to Rust's `headBoundLocally`. There the question was "is this name bound at all?", so missing finalize's import channels lost real bindings. Here the question is "does something LOCAL shadow the import?", and the import's own finalized binding is exactly what must not count — routing this through `lookupBindingsAt` would find every namespace import shadowing itself and suppress the lot. Verified against a target module carrying a self-named def, which still resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(python): close the three remaining namespace-receiver gaps (#2826) Three defects the first fix left behind. All three were confirmed by probe before being touched, and a fourth suspected gap was disproved the same way. ## 1. Case 1 resolved through an import a local had shadowed `namespaceTargets` is collected per FILE, but Case 1 in `receiver-bound-calls` consulted it with no lexical guard at all, so import pkg.db def f(pkg): # parameter shadows the package return pkg.db.session_scope() emitted an edge to pkg/db.py. That is a WRONG edge, and it predates the dotted key: the single-segment spelling (`import single` + `def f(single)`) failed identically. The compound-receiver construction path has applied this guard since #2770; Case 1 simply never did. Now both use the shared guard. ## 2 + 3. The root key named the leaf module, not the package These read as two gaps and are one. `import a.b.c` binds ONE name — `a` — but makes three attribute paths callable, naming three different files: a → a/__init__.py a.b → a/b/__init__.py a.b.c → a/b/c.py The map keyed only `a`, pointed at the LEAF. So `a.helper()` resolved into a/b/c.py whenever that module happened to export `helper` — silently preferring a decoy over the real definition in the package — and `a.b.mid()` resolved to nothing at all. One wrong edge and one missing edge from a single mis-keying. Fixing it needs per-language knowledge the shared collector cannot have: which prefixes are reachable, and which file each names. The `__init__.py` convention is Python's alone, and the edge shape is ambiguous across languages — Swift's `import Foo.Bar` produces an identical `localName`/`targetExportedName` pair that means the opposite thing. So the previous commit's boolean opt-in is replaced by `ScopeResolver.namespaceReceiverPaths`, which returns every spelling with the file it names; absent or declining, the shared default (bound name → own target) is unchanged for every other language. Prefix files are proposed, not asserted — `moduleFileExists` drops any the workspace never parsed, so a PEP-420 namespace package contributes no key rather than one pointing at a missing file. ## Disproved: C# was not a fourth gap The plan listed C# `using System.Collections.Generic` + `System.Collections.Generic.List` as the same class of bug. It is not: a probe shows `My.Deep.Space.Helpers.Work()` already resolves through the FQN namespace bindings in `walkers.ts`. No change made, and the claim is withdrawn rather than carried forward as a known gap. ## Testing Integration: the shadow block asserts the exact surviving edge set (an absence-only assertion would also pass if the guard over-suppressed and killed the clean rows); the prefix block asserts all three spellings land on their own file, with `helper` defined in BOTH package and leaf so a wrong edge is visible rather than merely possible. Unit: 16 rows on the keying contract, including that a Swift-shaped edge mints nothing and an alias import keys neither the path nor the root. Resolver integration 3024 passed / 1 skipped / 0 failed; scope-resolution unit 1452 passed; tsc clean in both packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(python): probe both path separators when resolving a prefix package (#2826) Workspace file paths are not normalized to POSIX at ingestion — `import-target` already re-normalizes at five other comparison points, and `moduleScopeByFile` is keyed by the raw `ParsedFile.filePath`. The prefix probe built only the `/` spelling, so on Windows it would compare `a/b/__init__.py` against an `a\b\__init__.py` key, find nothing, and mint no prefix keys at all. That fails quietly, which is the worst shape for it: `a.b.mid()` simply goes back to unresolved on one platform, with no drop recorded and every test on POSIX still green. Probe both spellings and key whichever the workspace actually holds. The new row is mutation-tested — reverting to the `/`-only probe turns it red (1 failed / 10 passed), so it pins the behaviour rather than passing alongside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(python): correct three defects a multi-lane review found in this PR (#2826) All three were introduced by this PR's own earlier commits, and none was found by re-reading the diff — each came from a lane attacking an angle the author had not. ## 1. The shadow guard ran BEFORE the map lookup it gates Case 1 evaluated `isNamespaceNameShadowed` unconditionally, then consulted `namespaceTargets`. So every call/read/write site with an explicit receiver, in every language, paid a scope-chain walk (a Set allocation, three Map lookups and a linear `ownedDefs` scan per level) ahead of an O(1) hash miss that was going to decline it anyway. The proof it was an oversight rather than a decision sits in this same PR: the sibling guard in `compound-receiver.ts` reads the map first and only guards on a hit. Two call sites of one shared function, opposite order. Semantics are identical either way — a miss yields `undefined` regardless — which is exactly why it survived several readings. ## 2. Prefix packages were anchored on the import spelling, not the resolved leaf `pythonNamespaceReceiverPaths` built `a/__init__.py` from the dotted path joined at the workspace root, never consulting the file the import actually resolved to. But `resolvePythonImportTarget` resolves off-root in two of its three tiers, so `import utils.db` can land on `libs/common/utils/db.py`. That produced a wrong edge where a same-named `utils/` package exists at the root, and produced NOTHING in a `src/` layout — the prefix feature was inert for the most common Python project shape, silently. Now the prefix directories are derived by walking back from the resolved leaf, which is exact for root, `src/` and off-root layouts alike. It also inherits the leaf's own separator, which subsumes the previous dual-separator probe: that probe was dead code anyway, because `filesystem-walker.ts` normalizes `\` to `/` before a path ever becomes a `ParsedFile.filePath`. Its test row is removed rather than left asserting an unreachable state. ## 3. Keying the root at `__init__.py` INSTEAD of the leaf lost re-exports `findExportedDef` accepts only a binding whose `origin === 'local'`. The canonical Python package re-exports from its submodules — `from .b.c import helper` in `__init__.py` — which is an IMPORT binding, so it is rejected. Keying the prefix solely at the package therefore turned `a.helper()` from a correct edge into no edge at all for the most common package shape. Every fixture in this PR defined its members locally in `__init__.py`, which is precisely the one layout where that mistake is invisible. The prefix now keys the package FIRST and the leaf behind it. A real definition in `__init__.py` still wins over a same-named decoy deeper in the package, and a name merely re-exported there still resolves through the leaf. Ordering is the contract, so the unit rows assert the exact arrays rather than membership. ## Testing New rows: off-root layout with a decoy `utils/` at the root, and a `src/` layout. Both mutation-tested — reverting to the spelling-anchored build turns them red. The re-export case was verified end-to-end with a scratch fixture whose `__init__.py` only re-exports (`uses -> helper@a/b/c.py`). Resolver integration 3131 passed / 1 skipped / 0 failed — unchanged from before these fixes, so they regress nothing. Scope-resolution unit 1459 passed. tsc clean in both packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(resolution): stop the namespace shadow guard AT the module scope (#2826) CI caught a regression this PR introduced: `cjs-exports-assignment.test.ts` lost both of its cross-file rows — cross-file require() member call resolves expected [] to deeply equal [ 'handle' ] an `exports` parameter does not hijack the module (UMD factory) expected [] to deeply equal [ 'publicApi' ] — i.e. `const svc = require('./svc'); svc.handle()` stopped resolving in JavaScript. Cause: in CommonJS the namespace import IS a variable declaration. One statement produces both the ImportEdge and a module-scope `const` binding, so the guard, by inspecting the module scope, found the import's own name there and read it as a shadow of itself — suppressing exactly the receivers it exists to enable. The guard's own contract sentence already said the right thing: "a declaration BETWEEN the call site and its module scope". The module scope is the floor of that walk, not a rung on it. It now returns at Module without inspecting it. Nothing is lost on the suppression side: a genuine shadow is a parameter, a local, or a nested declaration, and all of those live in scopes strictly inside the module. The Python rows that pin suppression (`def f(pkg): pkg.db.f()` and its single-segment `import single` twin) still pass, because a parameter is an inner scope. Worth recording for the next reader: two independent review lanes examined this exact scenario and both REFUTED it, reasoning that `require()` yields an ImportEdge in `scope.imports` rather than a local binding. That is true for Python's `import x` and false for CommonJS, where one statement is both. My own probe used a Python fixture and so could not surface it either. Agreement between reviewers was not evidence; the test corpus was. Verified: cjs-exports-assignment 36/36, the #2826 integration rows 7/7, scope-resolution unit 126/126. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Shifra Williams
|
a6a8aa788c
|
feat(serve): validate and port-scope the origin/proxy configuration surface (#2820) | ||
|
Gergő Magyar
|
cabd5b82f9
|
fix(go): model Go method sets exactly so interface satisfaction is decidable (#2813) (#2829)
* test(go): pin calls through an interface-typed struct field (#2813) A call through an interface-typed struct field never reaches the implementation: the CALLS edge stops at the interface DECLARATION, so `impact()` on the implementing method reports 0 callers. This commit adds the executable statement of that defect; the fixes follow. Two stacked defects produce it, and either alone is enough to reproduce — which is why no existing fixture could observe it: D1 `buildDetectionIndexes` skips every POINTER-receiver method, so a struct whose methods are all `func (r *T)` has an empty method set, structurally satisfies nothing, and gets no IMPLEMENTS edge. Go's rule is that the method set of *T includes pointer-receiver methods, and idiomatic Go stores *T in an interface-typed field. D2 Case 0 (compound receiver) emits its primary edge and short-circuits without the interface-dispatch fan-out Case 4 performs. A struct field receiver `s.orderRepo` contains a dot and so always takes Case 0; a local or parameter receiver is a bare name and reaches Case 4. Every implementor in both pre-existing structural-dispatch fixtures uses a VALUE receiver, and the one pointer-receiver type is pinned as a negative (`not.toContain('PointerOnlyThing -> PointerOnly')`), so the corpus could not see D1 by construction. The new fixture is pointer-receiver throughout, cross-package, and carries concrete-field controls in the same structs. Failing-first, verified against this tree: 7 of the 11 new assertions fail and 4 pass. The 4 that pass are exactly the controls that must not regress — the primary edge to the interface declaration, the concrete-field call, the absence of fan-out on a concrete field, and the partial-signature negative — so the suite discriminates rather than merely failing. Two recorded artifacts move here because the FIXTURE was added, not because capture output changed: - test/fixtures/go-captures-golden/expected-captures.json — regenerated additively (32 insertions, 0 deletions). - bench/scope-capture/baselines.json — go fingerprint, fixture_count 102 -> 110. Both are regenerated in this commit rather than deferred to the end of the series: the fixture is their only cause, no later commit touches capture emission, so they cannot re-drift and every commit stays green. The check that this is corpus growth and not a capture regression is that go was the only one of 15 language fingerprints to move on the same run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(go): count pointer-receiver methods toward structural interface satisfaction (#2813) D1 of two stacked defects. `buildDetectionIndexes` skipped every method whose receiver is a pointer, so a struct declaring `func (r *OrderRepo) DeleteItem(...)` had an EMPTY method set, structurally satisfied nothing, and produced no IMPLEMENTS edge at all. Go's method-set rule is per-type, and there are two types involved: the method set of `T` holds only value-receiver methods, while the method set of `*T` holds both. #1966 implemented the `T` reading, which is exactly right for `T` — and leaves `*T` permanently empty. GitNexus models one Struct node per type with no separate `*T` node, so only one of the two can be represented, and the `T` reading is the one idiomatic Go almost never uses: methods take pointer receivers so they can mutate, and `*T` is what gets stored in an interface-typed field. The cost was silence rather than caution. With no IMPLEMENTS edge, a call through an interface-typed field resolved to the interface DECLARATION and `impact()` on the implementing method returned 0 callers — byte-identical to a symbol that genuinely has none, which is what made the reporter's blast-radius check unusable rather than merely incomplete. This picks the `*T` reading: the graph now answers "which types provide this interface's behaviour", and no longer proves `var x I = T{}` invalid. The trade is deliberate and was checked against every consumer of IMPLEMENTS before being made — MRO/METHOD_IMPLEMENTS derivation, community clustering, the receiver-dispatch fan-out index, and the epistemic heritage probe. None performs value-assignability checking. Two negative pins encoded the #1966 decision and are REVERSED here rather than deleted, each keeping a comment that explains why the polarity moved: - go.test.ts: `PointerOnlyThing -> PointerOnly` now expected to be emitted. - go-hooks.test.ts: the pointer-receiver-only unit case now expects the implementor instead of `undefined`. `goReceiverKind` is still stamped in method-owners.ts — it is the hook a future value/pointer-aware model would read — but is deliberately no longer a filter. Its now-dead local predicate and type alias are removed so the file no longer carries a helper asserting the reverted rule. Measured on the #2813 fixture, this commit alone: the two IMPLEMENTS assertions flip to passing (6 pass, up from 4) while the five interface-dispatch fan-out assertions still fail — those are D2, fixed in the next commit. Keeping the two commits separate is what makes that attribution visible. Go unit suite: 91 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(resolution): fan out interface dispatch from a compound receiver (#2813) D2 of two stacked defects, and the one that closes the issue. Case 0 (compound receiver) emitted its primary edge and short-circuited without the interface-dispatch fan-out that Case 4 performs, so a call whose receiver is a struct FIELD stopped at the interface's method DECLARATION and never reached any implementation. The gap was a property of receiver SYNTAX rather than of types. Case 0 is selected by `receiverName.includes('.')`, so a field receiver (`s.orderRepo`) always lands there, while the very same interface reached through a local or a parameter is a bare name and falls through to Case 4 — which fans out correctly. Field-held interfaces, i.e. dependency injection, were the half that silently lost every implementation edge; the pre-existing fixtures exercise the local and parameter forms only, which is why the suite was green. The fix is the call Case 4 already makes, placed after Case 0's primary `tryEmitEdge` and before its `handledSites.add`. It stays language-agnostic (AGENTS.md section 42): `emitInterfaceDispatchFor` self-gates on `ownerDef.type !== 'Interface'`, so a receiver that folds to a Struct emits nothing extra and no language check is needed. Confidence is Case 0's own 0.85 literal, not Case 4's site.kind-dependent value — Case 0 has no read/write arm to mirror. The case ladder itself is untouched: invariant I4 in contract/scope-resolver.ts makes the ordering a contract, so the fan-out is added INSIDE Case 0 rather than by reordering or merging cases. Also flips a second, previously unnoticed encoding of the #1966 value-only reading that the full sweep surfaced: the exact-set assertion at go.test.ts:361 enumerates every structural IMPLEMENTS edge, and D1 correctly adds `PointerOnlyThing -> PointerOnly` to it. It is D1 fallout rather than D2's, but D1 had already landed; recording it here with its reason beats amending a commit whose separate measurability is the point. Measured: - #2813 suite: 11 of 11 pass (was 7 failing after D1 alone, which fixed only the two IMPLEMENTS rows). - go.test.ts: 160 passed. - Full cross-language sweep, test/integration/resolvers: 3027 passed, 1 skipped, across 52 files. The single failure in that run was the exact-set assertion above, fixed here; no other language regressed. `detect_changes` rates this HIGH (6 affected flows, all EmitReceiverBoundCalls at step 1) — inherent to editing a hub symbol in the resolution pipeline. The sweep above is the empirical answer to that label. An existing index must be re-analyzed to show the new edges; this changes what the resolver produces, not how it is stored, so no SCHEMA_BUMP applies. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(go): pin the heritage edges that make impact() hedge an interface-bound count (#2813) The epistemic half of the issue, resolved by MEASUREMENT rather than by new code, and pinned at its mechanism. The reporter's disqualifying complaint was that `impact()` reported `impactedCount 0, epistemic "exact", risk LOW` for a method reachable only through an interface-typed field — byte-identical to what it reports for a symbol that genuinely has no callers. A zero therefore could not be used defensively, which was the entire use case. That verdict comes from `computeEpistemicBoundary`, which has two producers and neither fired: the call sites were not DROPPED (they resolved, just to the interface declaration, so the #2744 receiver-typing producer saw nothing), and its heritage probe walks IMPLEMENTS/METHOD_IMPLEMENTS edges out of the queried symbol — of which there were none, because the pointer-receiver exclusion (D1) meant no such edge was ever emitted. Restoring those edges fixes the epistemics as a side effect, so the planned conditional change to local-backend.ts is NOT needed. Measured on this fixture against the fixed tree: impact(OrderRepo.DeleteItem, upstream) before: impactedCount 0, epistemic "exact" after: impactedCount 3, epistemic "lower-bound", with an interface boundary note; the three callers are OrderHandlers.Delete, PickService.StartSession and WaveService.Release — all correct. impact(CartRepo.Get, upstream) [concrete receiver, no interface] after: impactedCount 1, epistemic "exact" The second row is the one that matters for trust: the hedge discriminates instead of firing on everything, so "exact" still means exact. This test asserts the METHOD_IMPLEMENTS edges the probe walks. Pinning the mechanism keeps the resolver suite from reaching into the MCP layer while still failing loudly if the edges regress; the impact() numbers above are recorded in the commit message and PR body rather than re-asserted here. #2813 suite: 12 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(go): model Go method sets exactly so interface satisfaction is decidable (#2813) Replaces the approximate structural-interface model with the rules the Go spec actually defines, so the graph answers what the compiler answers instead of a useful-but-wrong summary of it. Three answers were provably wrong before; all three are now exact and covered. Method sets (go.dev/ref/spec#Method_sets): MS(T) = methods declared with receiver T MS(*T) = methods declared with receiver *T OR T Promotion (#Struct_types): S embeds T -> MS(S) and MS(*S) get promoted methods with receiver T; MS(*S) ALSO gets those with receiver *T S embeds *T -> MS(S) AND MS(*S) both get receiver T or *T Identifier identity (#Uniqueness_of_identifiers): "Two identifiers are different if they are spelled differently, OR IF THEY APPEAR IN DIFFERENT PACKAGES AND ARE NOT EXPORTED." func (b *Base) Ping() // pointer receiver type ByValue struct{ Base } type ByPointer struct{ *Base } type before exact answer Base IMPLEMENTS only *Base implements ByValue IMPLEMENTS only *ByValue implements ByPointer IMPLEMENTS the VALUE type implements All three were the same edge. Two of the three were wrong, and nothing in the graph could tell them apart. Worse, in a different direction: package sealed; type Sealed interface { seal() } package foreign; func (t *T) seal() {} `foreign.T` cannot implement `sealed.Sealed` in Go — `seal` is unexported, so the two identifiers are DIFFERENT. Matching on the bare name emitted a FALSE IMPLEMENTS edge, and the interface-dispatch fan-out then turned it into an impossible CALLS edge. That is the entire basis of the sealed-interface idiom. - `methodSetKey` qualifies UNEXPORTED method names with their declaring package, leaving exported names unqualified (which is what makes cross-package satisfaction work at all). Exactness, not a heuristic: the sealed case now emits no edge, while the legitimate same-package implementor is retained. - `collectStructMethodEntries` builds MS(T) and MS(*T) together and applies the promotion table above. The embed FORM is load-bearing, so it is now captured: `@reference.embedded-pointer` records `*T` versus `T`, which the parser previously discarded (the `*` is an unnamed token). - Detection returns `{ structDefId, receiverForm }`. `receiverForm: 'pointer'` means the value type does NOT implement and only `*T` does — the fact `var x I = T{}` turns on. - The form rides in the edge `reason` (`-structural-implements-pointer`). Relationships carry no arbitrary properties, so a new field would change the relation DDL, move SCHEMA_FINGERPRINT and force a rebuild for a fact a string already expresses. Value-form implementors keep the ORIGINAL unsuffixed reason, so a consumer matching the old string now sees exactly the assignable set — which is what that string always claimed to mean. - `emitInterfaceDispatchFor` walks the SUBTYPE CLOSURE (IMPLEMENTS + EXTENDS) and skips bodiless declarations, instead of stopping at depth 1. Two reproduced Java shapes emitted an edge to a second abstract declaration while the only class with a body got nothing: a sub-interface that re-declares the method, and an abstract base between interface and implementation. Both now reach the implementation and neither emits the declaration edge. - The fan-out is bounded by `MAX_INTERFACE_DISPATCH_FANOUT` (32, `GITNEXUS_MAX_INTERFACE_DISPATCH_FANOUT`) and reports what it dropped, mirroring `MAX_PROPERTY_DISPATCH_FANOUT`. A bare cap would silently discard valid dispatch targets, which is the same false-safe silence this issue is about. - Corrects a rationale comment that was factually wrong about the code 70 lines above it (Case 0 DOES branch on `site.kind`, at :713-716; what it lacks is a read/write branch in its reason/confidence computation). - Updates both copies of the case-ladder contract, which still described the fan-out as Case-4-exclusive. The embed-pointer marker is PARSE-TIME capture emission, so a warm cache would replay the pre-marker capture set and the distinction would never appear — silently, the v27/v30 failure mode. 43 and not 40 because origin/main allocated 40, 41 and 42 while this branch was in review, which is exactly the window this file's history records both prior EXACT clashes landing in. Pin moved with it. RE-CHECK AGAINST origin/main IMMEDIATELY BEFORE MERGE. - Go unit: 93 passed, including new rows pinning that `populateGoOwners` stamps `goReceiverKind` (previously the field had no reader and could rot silently) and that a pointer-receiver-only type implements in POINTER form only. - Cross-language sweep, test/integration/resolvers: 3034 passed, 1 skipped, 52 files, zero regressions. - scope-capture bench: PASS (15 languages). Go is the ONLY fingerprint that moved, which is the check that this is a Go capture change and not a cross-language regression; rebaselined with rationale. - Also closes review gaps in this PR's own tests: the concrete-field control was vacuous with respect to the type gate (repointed at a struct that IS an implementor), the two-service-file row could not distinguish the two files it is named for (both ends now file-qualified), plus new rows for signature mismatch, emitted confidence, and an exact N-by-M fan-out bound. An existing index must be re-analyzed; the schema bump forces it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
c1103f38f2
|
fix: type an inference-typed class field so it can act as a call receiver (#2807) (#2810)
* test(helpers): add the shared temp-repo lifecycle helper `createTempDirPool` gives a suite one owner for its temp fixture repos — create on demand, remove them all in one `afterAll` — instead of a hand-rolled mkdtemp/rmSync pair per file. The PDG receiver pin added in the next commit uses it. Cherry-picked verbatim from |
||
|
Gergő Magyar
|
9eaf2e6c4e
|
perf(mcp): cut the analyze-only language-provider closure out of MCP server startup (#2802) (#2806)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Gitleaks / gitleaks (push) Has been cancelled
Publish / Classify release event (push) Has been cancelled
Scorecard / Scorecard analysis (push) Has been cancelled
Skill copy sync / shipped skills drift guard (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-cli) (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-web) (push) Has been cancelled
Publish / RC guard (marker + release-PR skip) (push) Has been cancelled
Publish / ci (push) Has been cancelled
Publish / Publish to npm (push) Has been cancelled
Publish / Build & Push RC Docker images (push) Has been cancelled
* fix(mcp): key the empty-ascent note on CALL_SUMMARY data, not language (#2802) `pdg-impact.ts` decided whether to append a "return-value ascent is TypeScript/JavaScript-only" caveat to the `impact(mode:'pdg')` note by looking up the criterion file's language. That put language-specific logic in a layer that must be language-agnostic, and it was a lossy proxy for a fact the graph already holds. Whether the ascent can fire is a property of the persisted CALL_SUMMARY edges. The descent already computes it, so thread the resolved-callee and return-flowing counts out of `interproceduralDescent` and key the note on those instead. Three defects the language proxy carried, all gone: - Wrong for `.mjs`/`.cjs`/`.mts`/`.cts`: the provider registry's extension arrays omit them while the ingestion pipeline parses them as TS/JS, so those files were harvested but the note claimed their ascent was empty. - Silently stale: any language whose harvester started recording formal indices would keep getting the caveat until someone edited the list. - Wrong in reverse: a TS/JS callee with no return-flow got no caveat, so an ascent that found nothing read like one that covered the slice. `pdg-impact.ts` now names no language and imports nothing from the language layer, which also drops the analyze-only provider closure from MCP server startup. Measured on overlayfs against a full build: import mcp/local/local-backend.js before 565-648 ms / 548 modules import mcp/local/local-backend.js after 458-463 ms / 170 modules Tests hold CALL_SUMMARY content fixed while varying the file extension across nine languages and assert the note text is identical, then hold the extension fixed and vary the summary to show the note tracks the data. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): guard MCP startup against the language-provider closure returning The eager `pdg-impact.ts -> core/ingestion/languages` edge was found and lost once already during #2793 before #2802 re-derived it, so it gets a test rather than a comment. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(lbug): record why csv-generator is not lazy-imported #2802 proposed cutting `csv-generator.js` out of the adapter chain to shorten MCP server startup. Measured on a native filesystem, the marginal cost is small relative to the siblings this module already imports, and `core/search/bm25-index.ts` statically imports `normalizeFtsText` from the same module on a path `local-backend.ts` reaches dynamically for FTS — so deferring would relocate the cost to first query, not remove it. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(pdg): pin chained receiver calls reaching BasicBlock.calleeIds The PDG inter-procedural descent hops through `BasicBlock.calleeIds`, so it can only cross a call boundary the resolver resolved. Chained receiver calls reach `calleeIds` through the receiver-typing pass's own `calleeIdSink` — a separate path from plain calls. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(analyze): drop the stale per-language cross-reference (#2802 review P3-4) `pdgModeMismatch`'s comment told readers to keep "the diagnostic per-language refinement in the impact CONSUMER (see pdg-impact.ts assemblePdgImpactResult)". That refinement is no longer per-language — removing it is the point of #2802, which now keys the empty-ascent note on the persisted CALL_SUMMARY data instead. The comment's real invariant is untouched and still correct: the values in `resolvePdgConfig` must stay scalar, because the comparison below is a shallow `!==` and an object would compare by reference. Only the cross-reference was stale. Comment-only; no executable line changes. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): probe the real module loader for the startup language closure (#2802 review P1-2) The previous guard hand-rolled a regex walk over TypeScript source to assert `core/ingestion/languages` was not statically reachable from MCP startup. Four bypasses were reproduced against it, any one of which let the exact 226-module regression return while the test stayed green: a. Wrong entry root. It walked from `mcp/local/local-backend.ts`, but the server module is `mcp/server.ts` — which imports LocalBackend as `import type`, so the guard's anchor was not even on server.ts's runtime closure. Ten real startup modules sat outside it. b. A top-level `await import(...)` executes during module evaluation, so it is eager at startup — but the walker skipped every `import(...)` by construction. c. The `import type` strip deleted a 16,445-character window of `pdg-impact.ts`: an `export type X =` matched lazily to the next `from "…"`, which lives inside a string literal. Any import in that window was invisible. d. The comment strip treated a `/*` inside a string literal as a comment opener. Replace the approximation with a real module-load probe: spawn a child node process per entry, import the built `dist/` entry, and report what the loader actually pulled in. Rooted at `dist/mcp/server.js` and `dist/cli/mcp.js` (the real startup entries) plus `dist/mcp/local/local-backend.js`. Syntax cannot fool it. One deviation from the two existing sibling probes is load-bearing: `dist/` is ESM, so a `require.cache` diff alone cannot see the first-party `dist/**` graph — it only catches CJS and native modules, which is why `import-closure.test.ts` gets away with it (it asserts on `@ladybugdb/core`). A pure cache diff here would have reported zero language modules unconditionally, i.e. a new vacuous guard. This probe unions `module.registerHooks({ load })` with the cache diff, and each entry carries a non-vacuity anchor and a module floor so an empty result fails loudly. Verified load-bearing: adding a top-level `await import('../core/ingestion/languages/index.js')` to `src/mcp/resources.ts` and rebuilding turns `dist/mcp/server.js` red with 70+ named offenders, while the `local-backend` and `cli/mcp` cases stay green — which is bypass (a) demonstrated directly. The old guard passed that poisoned tree entirely. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(lbug): drop the unreproducible 9p multiplier from the csv-generator note (#2802 review P3-2) The comment justifying why `csv-generator.js` is NOT lazy-imported carried a hard "~40x" figure for how much a 9p mount inflates per-file ESM resolve. Three independent measurements during review produced ~40x, ~7.3x and ~30x, so the multiplier is not a reproducible quantity and had no business being stated as one in a durable comment. Reworked so the STRUCTURAL argument leads and the numbers only support it. That argument is what actually settles the question and it does not rot: `core/search/bm25-index.ts` statically imports `normalizeFtsText` from `csv-generator.js`, and `local-backend.ts` reaches bm25-index through a dynamic import on the FTS query path — so deferring here relocates the cost to first query rather than removing it. Both verified again at `bm25-index.ts:15` and `local-backend.ts:2756`. Remaining figures are re-measured, attributed to a date and issue, and labelled by filesystem: ~1.6 ms marginal (median of 45 cold imports on local disk) versus ~50 ms for the same import on a network mount, stated as environment-bound rather than as a property of the module. The provider-registry cost is given as "several hundred modules" — the static walk, the runtime hook, and the reviewer's probe each counted it differently (375 / 439 / 407), so no single number was picked to go stale. The old "226 modules" was real but counted only the `languages/` subtree and undercounted the win. Also repoints the trailing reference to the guard's new home at `test/integration/mcp/startup-language-closure.test.ts` (same comment block, inseparable from this rewrite). Comment-only; no executable line changes. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): stop the empty-ascent note asserting a fact an undecodable summary contradicts (#2802 review P2-2) The note claimed "this is a property of the persisted summaries" whenever the descent resolved callees and none carried a return-flow. But `decodeCallSummary` never throws by design: a version-skewed (`2|r:1`), corrupt (`1|r:zz`), or NULL `reason` yields no entry, which was indistinguishable from a cleanly-decoded empty summary. So the note could assert "no formal parameter is recorded as flowing to its return value" about a callee whose CALL_SUMMARY actually records `p0 -> return`. `meta.pdg.hasCallSummary` is a plain boolean and stores no codec version, so nothing else caught it. `calleesWithReturnFlow` now reports three outcomes instead of two — flowing, decoded-empty, and undecodable — and the undecodable count is threaded through the descent to the note. When it is non-zero the note says so and points at a re-index; when every summary decoded, the persisted-summaries claim is kept and now explicitly conditioned on that. Soundness is unchanged: an undecodable summary still licenses no ascent and never enters the return-flowing set, so the ascent path is byte-identical. Only the note's wording moves. Tests drive all three undecodable forms through the mock and assert the false claim is gone, the remedy is reported, and the ascent is still withheld. A companion assertion pins that the all-decoded case KEEPS the persisted-summaries claim, so the fix cannot degenerate into deleting the sentence. Verified load-bearing: reverting the source alone fails 6 of 34. Impact analysis: `calleesWithReturnFlow` upstream LOW (2 callers, both in this file); `assemblePdgImpactResult` upstream LOW (1 caller). Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(pdg): cover every chained-receiver shape and pin the inference gap (#2802 review P2-1, P3-1) The fixture proved chained receiver calls reach `BasicBlock.calleeIds` using exactly one receiver form — a local `const`. That is the shape that works, so a single-shape fixture implied general support the resolver does not have. This repo has been burned by that before: a drop-count gate blind to fixed shapes. Measuring nine forms against the real pipeline also corrects how the gap was originally characterised. It is NOT local-versus-field. An annotated field resolves fine, including the constructor-assigned variant: private p: Outer = new Outer(); -> both links private p: Outer; this.p = new Outer(); -> both links private p = new Outer(); -> EMPTY CELL private p; this.p = new Outer(); -> EMPTY CELL The discriminator is the type ANNOTATION. When a field's type must be inferred from its initializer the whole `calleeIds` cell empties — so even `Outer.inner`, an ordinary named-receiver call, is lost, and the inter-procedural descent cannot cross the boundary at all. Pre-existing; independent of #2802, which does not touch receiver resolution. The fixture is now table-driven over seven working forms (local const, local in a method, annotated field, ctor-assigned annotated, ctor-param assigned, call-result receiver, three-link chain) plus the two inference-typed forms, each row carrying its expected chain-link ids. Assertions moved from substring to exact id membership, split with the production `splitCalleeIds` reader — so `Inner.compute` can no longer be satisfied by `Inner.computeExtra` or `OtherInner.compute`, which matters because the descent keys on exact ids for span and CALL_SUMMARY lookup. The two known-gap rows are pinned with `it.fails` plus a hard assertion on the exact gap-row set, so a resolver fix turns them red instead of passing silently, and an anti-vacuity guard requires every shape to match exactly one block — without it a drifted fixture matching zero blocks would let `it.fails` pass for the wrong reason. Proven by mutation: relabelling a working row as a known gap fails both pins. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): qualify the empty-ascent note when the examined callee set is incomplete (#2802 review P2-4) The note asserted "none of the N resolved callees carry a CALL_SUMMARY return-flow", and on the all-decoded path that this is "a property of the persisted summaries". Both are universal claims over the callees the descent actually examined, and two mechanisms can leave that set incomplete without the note saying so: 1. Budget truncation. The descent stops on depth/limit/node-cap, so a callee that DOES carry a return-flow can sit in a hop never reached. A 4-deep chain reported "none of the 3 resolved callees" while link 4 held the only summary. 2. Emit-time capping. When a block's `calleeIds` cell was capped, `splitCalleeIds` strips CALLEES_TRUNCATED_SENTINEL, so the dropped callees are invisible to both the scan and the counters — even though the callgraph bridge in this same file already treats such a block as callee-incomplete. Add `calleeIdsWereTruncated`, the counterpart to the sentinel strip, read from the raw cell before splitting so a block whose entire list was capped away still raises the flag. Thread it through the descent to the note. Case 1 needs no new plumbing — the aggregate `truncated` is already on the input object. Using the aggregate rather than a descent-only flag is deliberate: seed truncation and intra-BFS depth truncation also shrink the initial slice, so their callees are never gathered either. It is a sound superset that never under-hedges. When either mechanism fired, one clause naming the reasons is appended and the whole-slice assertion softens to "every summary examined decoded … a property of those summaries". When the set is complete both branches stay byte-identical to before, so this does not become a blanket hedge. Tests pin truncated, untruncated, emit-capped-alone, both-mechanisms, and undecodable+truncated, asserting the truncation premise rather than assuming it. Verified load-bearing: reverting the source alone fails 6 of 42, and the HEAD note printed in those failures is the bug verbatim. Impact analysis: `assemblePdgImpactResult`, `calleeIdsByBlock`, `interproceduralDescent` all upstream LOW; every caller is in this file and `runImpactPDG`'s exported signature is unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): stop the empty-ascent note calling call-site references "resolved callees" (#2802 review P3-7) The note printed "none of the N resolved callees carry a CALL_SUMMARY return-flow (no formal parameter is recorded as flowing to its return value)". N counted the raw `BasicBlock.calleeIds` cell, which carries ids `resolveCalleeSpans` never enters — out-of-repo targets, interface methods, and the `Class:` id a `new X()` emits. On the chained-receiver fixture that inflated N from 1 to 3. Two defects, both in the wording rather than the arithmetic: "resolved" implies a symbol-table lookup that did not happen for those ids, and the parenthetical asserted a FORMALS-level property about symbols never resolved to a body. Reworded rather than re-seeded, deliberately. `calleesWithReturnFlow` scans the RAW id set, so the claim "none of these carries a return-flow" is exactly established for all N — the scan really did check the `Class:` id. Re-seeding N from the resolved spans would make the sentence quantify over a strict SUBSET of what was checked, silently dropping the un-enterable references from a claim that genuinely covers them, and would desync N from `calleesUndecodable`, which is derived from the same scan population. none of the N resolved callees carry ... none of the N call-site callee references carry ... and the formals parenthetical is dropped. The note gets shorter, not longer. `calleesResolved` is renamed `calleeReferences` end-to-end (file-local; nothing outside referenced it), and the descent's return-type doc — which called them "callee symbols the descent resolved" and reinforced the wrong reading — now states that un-enterable ids ride the same cell, are scanned, and are never entered. The `> 0` gate is unchanged, so no slice that previously produced the note stops producing one. A test pins that explicitly: an all-un-enterable cell resolves no span, takes no hop, and emits no ascent sentence despite a non-zero count — so a future re-seeding cannot silently move when the note fires. Tests also pin the quoted number and singular/plural against a mixed cell, with a discriminator asserting `reachableBlocks` is byte-identical while the count moves 1 -> 3. Verified load-bearing: reverting the source alone fails 6 of 7 new tests, printing the finding verbatim. Impact analysis: `assemblePdgImpactResult` and `interproceduralDescent` upstream LOW, sole caller `runImpactPDG` in the same file; exported signature unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): pin cross-hop callee accumulation and the mixed return-flow contract (#2802 review P2-5) Every case in this file drove a single hop, so the Set union the descent performs across hops (`calleeReferencesSeen` / `calleesReturnFlowingSeen`) was never proven to accumulate rather than overwrite — a one-hop descent cannot tell the two apart. And although a sibling commit added a three-id cell, none of those ids return-flowed, so the "some callees flow, some do not" boundary was entirely unpinned. Extends the mock with a `secondSummary` knob that drives a genuine second hop: `helper2` is named only in `helper`'s own body block, so the descent must cross a second boundary to reach it. Three mock handlers are made faithful to the parameters they already bind — `calleeIdsByBlock` now routes on the asked `$ids`, and the CALL_SUMMARY scan and span resolve answer per asked id — which is what makes a second callee answerable at all. Existing cases are behavior-identical. Five tests: the union count across two hops; a return-flow on hop 0 surviving a later empty hop; a return-flow found only on hop 1; mixed callees in one examined set going silent rather than partial; and a flowing callee alongside an undecodable sibling staying silent including the decode remedy. The mixed case pins a deliberate contract rather than proposing one. The production condition is `calleesReturnFlowing === 0`, so partial coverage is reported as silence. A reviewer considered and dropped "report partial coverage" as a product change; this makes flipping it a conscious edit instead of an accident. Verified load-bearing against three separate source mutations: accumulating only on hop 0 (2 fail), each hop overwriting instead of unioning (3 fail), and flipping the gate to partial-coverage reporting (4 fail). In all three every PRE-EXISTING test still passed — which is the finding restated as evidence. Test-only; `pdg-impact.ts` is byte-identical to HEAD. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(mcp): consolidate the empty-ascent rationale to one canonical site (#2802 review P3-6) The "keyed on observed CALL_SUMMARY data, never on the criterion's language" rationale was restated in full at four comment sites. It exists because a reviewer asked "why not just look up the language?", so it has to stay findable — but not four times. The canonical explanation now lives in `interproceduralDescent`'s return-type doc, where the counters are actually computed, organised as POPULATION (why the raw `calleeIds` tally is the right set to quantify over) and OBSERVED DATA, NEVER THE CRITERION'S LANGUAGE (the full answer, including the producer-change argument and the no-language-naming rule). The other three sites keep only what is locally load-bearing and point here. Deliberately preserved, because each carries a non-obvious fact: why an undecodable summary licenses no ascent, why the aggregate `truncated` is used rather than a descent-only flag, and the raw-id-tally population argument. Net comment delta -11 lines. The reviewer also flagged the local/field naming asymmetry (`calleeReferencesSeen` vs `calleeReferences`). Keeping the suffix, with a comment recording why so it is not re-raised: the premise that every other local matches its field is true, but those locals are identity-returned, whereas these are `Set<string>` accumulators returned as `.size`. Dropping the suffix would give one identifier two types in one file — a `Set` at the accumulation site and a `number` where the note does arithmetic and pluralisation on it ~900 lines away. The Set-ness is also load-bearing: the dedup is why a callee invoked from two hops is not double-counted, which is what makes the note's count correct. Comment-only. Verified mechanically: every added and removed line in `git diff -U0` matches a comment pattern, so the note's template literals are untouched and its rendered text is byte-identical. 89 tests unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(mcp): collapse the ascent plumbing accreted across 13 fix commits Quality cleanup, no behavior change. Four independent review passes converged on the same root cause: thirteen commits each fixed one review finding in isolation, and the ascent facts grew one loose field at a time until 62% of the changed region was comments explaining plumbing. Five changes: - `calleeIdsFromBlocks` deleted. Zero call sites anywhere in src/ or test/ — already dead on main, and this branch had edited it to keep it compiling. Its only reference was a stale `{@link}` in a neighbour's doc, now rewritten to stand alone. - `parseCalleeIdsCell` replaces the two-pass read. `calleeIdsWereTruncated` and `splitCalleeIds` were splitting the same cell on adjacent lines, which measured ~2x the parse cost (0.82 -> 1.59 ms at a realistic hop, 57.7 -> 92.7 ms at the per-statement site cap) and was a second independent encoding of the sentinel format — exactly what `splitCalleeIds` was extracted to prevent. One pass classifies as it walks; `splitCalleeIds` stays as a wrapper so its two external callers are untouched. The single-use `export` is gone. - `AscentCoverage` replaces four fields threaded through three signatures. ~12 declaration sites become 3, and the canonical rationale now lives on the type by construction — which is why the earlier doc-consolidation commit was needed at all. - `calleesReturnFlowing` becomes a boolean. Its only reads were `=== 0`, twice; it cost a Set sized to every callee in the slice plus a per-hop union loop. The flag is set inside the existing `returnFlowing.size > 0` branch — equivalent, since the cross-hop union is non-empty iff some hop's was. - The duplicated empty-ascent note head is collapsed to one gate and one head with per-arm tails. Both arms had been edited in lockstep twice in this branch's own history. The rendered note text is byte-identical. Verified structurally and then empirically: both expressions reconstructed standalone and diffed across the full cross product of references x returnFlowing x undecodable x truncated x listTruncated — 288 combinations, 0 mismatches. Net -53 lines. 102 tests pass unedited; the unused-symbol lint warning is gone. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): parallelise the startup probes, drop a redundant pin, name the mock knobs Quality cleanup from the same review passes. The set of verified behaviors is unchanged except where noted. **Startup probes run concurrently.** `spawnSync` blocks the event loop and vitest runs a file's tests in order, so the three probes strictly serialised. Launching all three with async `spawn` in `beforeAll` and asserting over the collected outcomes cuts the file from ~12.7 s to ~3.9 s wall (-69%). Every promise is caught before `Promise.all`, so all three children are reaped and failures report per entry rather than surfacing only the first rejection. Preserved and each proven by mutation: the missing-dist error names its entry, a raised module floor fails only its own row, and a bogus anchor still reports the loaded-module count. **The two `it.fails` rows are removed.** They pinned the inference-typed receiver gap that the strict `toEqual` pin beside them already covers — and they were the weaker of the two, because `it.fails` passes when the body throws for ANY reason, including `idsFor`'s own non-vacuity guard. A renamed fixture marker would have kept them green on a rotted premise. The strict pin is self-diffing and was verified load-bearing on its own: pointing a known-gap marker at a resolving shape fails it with the two newly-present ids listed. The file header now carries the gap's durable description. **The ascent-note mock takes options objects.** `descentExec` and `run` had grown to five and seven positional parameters in the order five agents added them, so call sites read `run(FILE, true, null, 3, false, undefined, null)` — several carrying `undefined` purely to reach a later argument. All 34 call sites are converted; nine that used only defaults are now bare `run(file)`. No knob renamed — they are orthogonal and correctly named. Code lines are exactly neutral (353 -> 353); the win is at the call sites. Also refreshes five comments that still described `calleesReturnFlowingSeen` and the two-branch note, both of which the preceding commit replaced. 102 unit and 10 integration tests pass; test count moves 9 -> 7 in the chained-receiver file, exactly the two redundant rows. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(mcp): publish return-value-ascent coverage on the PDG impact result `impact(mode:'pdg')` computed four facts about ascent coverage and used them exactly once — to interpolate an English sentence. They never reached the result object, so an agent consuming this MCP output could only ask "was the ascent complete, and if not why" by regexing prose. The cost was already demonstrated: a pure rewording commit earlier in this branch broke ~30 assertions and would have silently broken any consumer keying on the old phrase. Adds `pdgEvidence.ascent`: referencesScanned how many call-site callee references were scanned returnFlowFound did the ascent fire anywhere in this slice undecodableSummaryCount summaries the codec could not decode examinedComplete was the examined set the whole callee list incompleteReasons 'traversal-truncated' | 'callee-list-capped' callSummaryLayerPresent false => pre-FU-C (v3) index Nested under `pdgEvidence` because that is the established counts-and- classification namespace, and `composeUnifiedPdgImpactResult` already spreads it, so the member survives the unified compose untouched. `incompleteReasons` carries CODES, following the existing `truncatedByReasons: ('depth'|'limit')[]` precedent. The prose clause and the structured field now render from one array computed once, so an agent branching on codes and a human reading the note cannot disagree, and a third reason becomes a rendering decision rather than a contract change. Two shape decisions worth recording. `callSummaryLayerPresent` exists because without it a v3 index publishes `referencesScanned: N, returnFlowFound: false`, which reads as "these callees record no return-flow" when the truth is "the layer that records it is absent" — the note already distinguishes those, and the structured surface must not be less honest than the prose. And the field is ABSENT rather than zeroed when the descent never ran (upstream slices): "nothing was scanned" is a different fact from "we scanned and found nothing". `pdgResultVersion` stays 2. The documented trigger is a BREAKING change to the result shape; this removes nothing, renames nothing, and changes no existing field's meaning. Confirmed mechanically: zero top-level key drift across 2304 cases. The historical v2 bump was for changing an existing field's semantics (startLine 0- to 1-based). The note prose is byte-identical, proven across the same 2304 cases with a negative control — perturbing one character of the phrase table produces 60 drifts, so the harness demonstrably detects what it asserts. 14 new tests cover the structured surface and all 14 fail when the source is reverted, while the 54 prose tests pass unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(helpers): share one module-load probe, and fix two guards that passed on broken builds Three tests independently spawned a child node process to inspect what a built `dist/` entry loads, duplicating the REPO_ROOT derivation, the probe source, the missing-dist guard, the spawn with NODE_OPTIONS cleared, the status-vs-signal rendering, and the payload parse. The newest copy was also the only correct one, so the next author had 2-in-3 odds of copying a weaker probe. The two older probes diff `require.cache` only, which is structurally blind to the first-party ESM `dist/**` graph. That is not theoretical — both were demonstrated passing on genuinely broken builds: - Severing `dist/cli/mcp.js -> stdio-context.js` (a pure ESM change) leaves the require.cache diff EMPTY, so `import-closure.test.ts`'s two assertions reduce to `[].filter(...) === []`. It reported 2 passed on a severed graph. - Severing `registry -> swift/query.js` leaves 76 unrelated CJS entries, which satisfied `registry-import-closure.test.ts`'s indirect guard. The Swift half of its headline had gone vacuous and it reported 1 passed. Both now fail on those same builds, naming the missing anchor. `test/helpers/module-load-probe.ts` unions the ESM `registerHooks({ load })` channel with the cache diff, probes entries concurrently, and makes non-vacuity STRUCTURAL: `anchor` and `minModules` are required fields and the helper throws when either fails. A vacuous probe is a harness failure, not a silently green test, so it cannot be forgotten. Forbidden patterns and remedy text stay per-test — the harness is the shared part, the policy is not. Also fixes `toRepoRelativePosix` resolving non-absolute specifiers against `process.cwd()`, and dedupes modules a CJS-from-ESM import reported once per channel. Faster despite doing more: the registry file goes 12.4s -> 6.75s, because `spawnSync` burned the parent thread polling while the child loaded native grammars. `import-closure` drops to one spawn from two. The `local-backend.js` entry is kept although its closure is currently a strict subset of `server.js`'s: that is an observation, not an invariant. If `server.js` ever stops eagerly reaching the local backend, the server probe stays green while the module #2802 actually changed goes unobserved — and now that anchors are mandatory, that entry is what pins `pdg-impact.js`. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(lbug): trim the csv-generator note and fix the claim it got wrong Two reviewers split on this comment: one wanted it cut to the structural argument, the other said a comment is the right depth for documenting a rejected change since there is no invariant to guard. Both are right, so it stays a comment and gets shorter — 13 lines to 6. Trimmed because it had already taken two corrections (an unreproducible "~40x" figure, and a pointer to a test file that no longer exists), and its tail had drifted from its own guard: the comment said "several hundred modules, ~150 ms" where `startup-language-closure.test.ts` says "~226 extra modules and ~130 ms". Two numbers for one fact. That tail is documented better in the guard's own header, so deleting it loses nothing. It also stated the load-bearing claim inaccurately. The old text said bm25-index imports `normalizeFtsText` "from here" — but `lbug-adapter.ts` neither exports nor re-exports it; the only occurrence of the identifier in this file WAS the comment. Anyone verifying would have grepped, found nothing, and concluded the note was stale. Now names `csv-generator.js` explicitly, re-verified at `bm25-index.ts:15` (static) and `local-backend.ts:2756` (dynamic, on the FTS query path). Comment-only, proven two ways: every changed line matches a comment pattern, and stripping all `//` lines from HEAD and from the working tree yields byte-identical text. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(helpers): extract the temp-repo lifecycle, collapsing five hand-rolled cleanups into one Four cfg integration tests each hand-rolled a `tmpDirs` array, a mkdtemp-and-register step, and an `afterAll` rmSync. It is actually five registrations across six creation sites — `pipeline-pdg.test.ts` keeps a second pool for its C-family fixtures. Seeding genuinely varies four ways (recursive cpSync, single copyFileSync, inline mkdir+writeFile, and nothing at all), so a fixture-copier helper would have fitted about half the sites and made things worse. Extracted the LIFECYCLE instead — mkdtemp, register, afterAll cleanup — which is byte-identical at all five registrations and is the correctness-critical part. `dir()` returns an empty registered directory for callers that seed themselves; `fromFixture()` covers the common case. That fits 6/6. The duplication had already produced a latent defect: `cFamilyTmpDirs` was cleaned by TWO `afterAll` blocks, harmless only because `rmSync` was called with `force: true`. Now one hook. `createTempDirPool` is a function called from each test file's module scope rather than a top-level hook in the helper, because under ESM caching a module-level `afterAll` would register once, against whichever file imported it first. That hazard is documented in the helper. Raw line count is roughly neutral (-44 across the tests, +62 for the helper, 29 of which are the rationale). The win is that a cleanup invariant went from five copies to one. Cleanup verified empirically, including the failure path: a throwaway suite whose `beforeAll` throws still has its directory removed, and every temp directory created by the four migrated files is gone after a run. 46 tests pass across the four files. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(resolvers): pin the inference-typed field receiver gap at the resolver level The gap was pinned only in a PDG test, asserting on `BasicBlock.calleeIds` behind the full `--pdg` pipeline. But it is a resolver fact: when a class field's type must be inferred from its initializer, chained receiver calls resolve to nothing. Whoever closes it will be working in the resolver suite and would have got a red CFG/PDG test with no resolver-side signal. Asserts CALLS edges directly, alongside `python-constructor-field-receiver.test.ts`. Nine receiver shapes run the identical statement; seven resolve, two do not: const o = new Outer() resolves private p: Outer = new Outer() resolves private p: Outer; this.p = new Outer() resolves private p: Outer; this.p = p (ctor arg) resolves constructor(private p: Outer) {} resolves makeOuter().inner().compute() resolves o.inner().mid().compute() (three links) resolves private p = new Outer() NO EDGES private p; this.p = new Outer() NO EDGES Two things the fixture establishes that the PDG-side pin could not. The discriminator is the type ANNOTATION, not local-versus-field — the parameter-property form resolves fine. And the initializer is NOT invisible to the resolver: `new Outer()` still emits its own constructor CALLS edge, byte-identical to the annotated twin. Only the initializer-to-field-type binding is missing, which narrows where a fix belongs. Assertions key on exact node ids rather than names, because `compute` is ambiguous across two classes and keying on the source name collides with `Object.prototype.constructor`. No `describe.skip` and no `it.fails` — the latter passes when the body throws for ANY reason, so it can go green on a rotted premise. The gap is pinned as its explicit current value, which self-diffs: simulating the fix fails one test showing the two newly-resolved ids, and renaming a fixture symbol fails the non-vacuity guard. Runtime is comparable to the PDG-side pin (~9-11s, both dominated by worker startup), so this is an altitude and scope win, not a speed one. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): replace the extension sweeps with a stronger language-agnosticism pin Two `it.each` sweeps over nine file extensions asserted that the empty-ascent caveat was present (or absent) for each. They looked like the pin for the property the whole change exists for — `pdg-impact.ts` must name no language and its output must not vary by extension — but they were the weakest available form of it. They asserted substring presence/absence, so a language dependence that ADDS text while leaving the caveat intact passes them. Demonstrated, not assumed: injecting a `.py`-only hedge inside the caveat sentence and replaying the two sweeps verbatim against that source gives 18 passed. The byte-identity test beside them caught it. So the sweeps are deleted and the identity test carries the property alone, hardened in two ways: - Two rows instead of one, covering BOTH sides of the caveat gate. The silent (return-flow present) branch previously had no identity counterpart at all — nine runs proving one fact, with nothing checking that its rendering was extension-invariant. - The fingerprint spans the note AND the reachable blocks, not just the note. Strictly more than the sweeps verified. Entailment is exact: identity across the extension set, plus the two existing single-extension content assertions, gives "every extension gets the caveat" and "no extension gets it". Reducing a sweep to one extension was rejected because it reproduces an assertion already present verbatim. Also converts the incompleteness block from six near-identical bodies to a 3-row premise table crossed with two assertions. Each row now names the exact phrase set its clause must contain, so presence and absence are asserted together — which adds three checks the longhand version lacked (the budget row now also proves the emit-cap phrase is absent). And three tests that re-rendered one fixture to make one assertion each are hoisted to a single render. 97 tests, down from 116: -18 sweep cases, -2 from the hoist, +1 identity row. No assertion was lost; several were added. Verified by injection: a `.py`-only note change fails the identity pin, and a dependence in the shared hop sentence fails BOTH rows, confirming the second row is load-bearing rather than decorative. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(mcp): lazy-import syncGroup so MCP startup skips the group extractor closure `core/group/service.ts` statically imported `./sync.js`, which pulls all six contract extractors, five of which statically import the native `tree-sitter` binding. That put the whole parser stack on every MCP server start, for a server that never syncs. Only `groupSync` needs it. The other seven group tools — `group_list`, `group_impact`, `group_query`, `group_contracts`, `group_status`, `group_trace`, `group_context` — do not, and now never load it. `syncGroup` has a single call site, already inside an `async` method, so this is a lazy `await import(...)` at that call site and nothing else: no signature change, no async ripple, no change to `local-backend.ts`. The pattern is already established on this exact module — `cli/group.ts`'s sync command lazy-imports `sync.js` the same way. `service.ts` was the outlier. Measured on a native filesystem (overlayfs; /workspace is a 9p mount that inflates ESM resolve, so it is not a valid measurement surface), 5 cold runs, medians: dist/mcp/server.js 521 ms -> 133 ms (-75%) dist/mcp/local/local-backend.js 453 ms -> 66 ms (-85%) tree-sitter modules at both entries: 11 -> 0 Same defect class as #2802, which cut the language-provider registry from the same startup path; this is what remained. The cost is moved rather than deleted: the first `group_sync` call now pays the module load. That is the right trade — `group_sync` is already a long-running operation, and sessions that never sync pay nothing. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): guard MCP startup against the group extractor closure returning Sibling forbidden-pattern case in the #2802 startup guard, reusing the concurrent probes it already collects — no new spawn, no new harness. Asserts that none of `dist/mcp/server.js`, `dist/cli/mcp.js`, or `dist/mcp/local/local-backend.js` loads a `core/group/extractors/` module or the native `tree-sitter` package. The parser is matched by package prefix rather than a bare substring, so a source file that merely mentions the word can neither satisfy nor trip it. Verified load-bearing rather than assumed: restoring the static `import { syncGroup }` in `core/group/service.ts` and rebuilding turns `dist/mcp/server.js` red and names all seven offenders — http-route, grpc, thrift, topic, include, manifest and workspace extractors. Reverted and re-confirmed green. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(mcp): keep the analyze-only CFG closure off MCP server startup (#2802 review) `mcp/local/pdg-impact.ts` imported `CALLEES_TRUNCATED_SENTINEL` and `CALLEE_ID_SEP` from `core/ingestion/cfg/emit.ts`. ESM evaluates a module to import any binding from it, so those two strings dragged the whole analyze-only CFG closure into every MCP server start. Measured against a clean build, per entry point: 8 modules — `emit`, `reaching-defs`, `reaching-defs-graph`, `control-dependence`, `post-dominators`, `synthetic-escape`, `call-site-harvest`, `reaching-def-reason-codec` — present at `dist/mcp/server.js`, `dist/mcp/local/local-backend.js` and `dist/mcp/http-transport.js`. Same defect class as the language-provider closure this branch already removed, and the guard could not see it: `FORBIDDEN_RE` covers `core/ingestion/languages/` and `FORBIDDEN_GROUP_RE` covers `core/group/extractors/|node_modules/tree-sitter`, neither of which matches `core/ingestion/cfg/`. The format constants move to a new LEAF module `cfg/callee-cell-format.ts` that imports nothing; `emit.ts` re-exports both names so every existing importer is untouched, and producer and consumer still resolve to one definition — the drift the shared constant exists to prevent stays impossible. Deleted, not deferred — the same bar #2802 held its own csv-generator proposal to. After: cfg modules at startup 8 -> 2, and both survivors (`callee-cell-format`, `reaching-def-reason-codec`) are leaves that import nothing. Totals: `server.js` 387 -> 380, `local-backend.js` 163 -> 156, `http-transport.js` 523 -> 516. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): stop pdgEvidence.ascent claiming a completeness it cannot have (#2802 review) `examinedComplete` is the field a consumer reads to decide whether `returnFlowFound: false` is a whole-slice claim. It could be published `true` over a callee set the descent never finished examining — the exact false all-clear the field was added to prevent. Root cause: `bfsReachableBlocks` sets `truncatedByDepth` when its frontier is still non-empty at the budget, but both call sites inside `interproceduralDescent` folded only the row-limit flag and dropped the depth flag. The top-level intra BFS's copy of that same flag was already propagated, so the asymmetry was unintended — one `if`-pair folding limit-but-not-depth, within a merge that already folds the node cap too. Reproduced at `maxDepth: 3`, the shipped default: a criterion calling a helper whose body is a 5-block dependence chain, with the return-flowing callee on the block past the clamp. Result reported `truncated: undefined`, `examinedComplete: true`, `incompleteReasons: []` and an unqualified universal note sentence. Fixed by propagating the dropped flags rather than inventing a parallel channel: `intraDepthBudget` is documented in-file as the SAME clamp the top-level intra BFS applies, and that one's depth truncation is already result-level. So the result's own `truncated`/`truncatedBy` were under-reporting for the same reason, and both surfaces are corrected together. Four further honesty fixes to the same published record: - Blocks reached only by the U-C4 ascent went into `reachable` but never `hopReached`, so their `calleeIds` cells were never scanned, never counted, and could not raise `callee-list-capped`. They are slice blocks; they now enter the hop set and get the same treatment as every other one. - `pdgEvidence.ascent` was absent on the empty-slice early return even though the descent had already run and scanned, contradicting the "present iff the descent ran" contract this branch itself added to `tools.ts`. Both exits now classify through one shared helper so they cannot disagree. - A block carrying call sites in `callees` but no resolved ids in `calleeIds` (the whole-file case where `emit.ts` has no fileMap) silently shrank the population while `examinedComplete` still reported `true`. That now raises a third reason, `callee-ids-unrecorded`. - `referencesScanned` is a distinct-callee tally and both surfaces described it as a call-site count. Field name kept — a rename is breaking at `pdgResultVersion: 2` — and the prose corrected instead. `PdgAscentIncompleteReason` gains a member, which is additive, so `pdgResultVersion` stays 2. Visible output change worth knowing: slices whose callee chain outruns `maxDepth` now report `truncatedBy: 'depth'` where they previously reported none, and a repo with id-less call sites now reports `examinedComplete: false`. Both are strictly more honest. Every behavioural change carries a mutation proof — revert the source, watch the new test go red, restore. One exception is documented inline rather than faked: the ascent-side fold cannot be observed independently, because the re-seed shares the caller's `visited` set and so can only reach past the budget when the traversal that covered that closure was already cut and had already raised a flag. Suite: 49 -> 59 tests. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): anchor each import-closure policy on the edge it polices (#2802 review) `module-load-probe.ts` makes non-vacuity structural via a required `anchor` — but the anchor was one per ENTRY while `startup-language-closure.test.ts` now runs TWO independent policies. The group-extractor policy added in |
||
|
Gergő Magyar
|
7468cc915b
|
fix(analyze): replace the hand-incremented schema version with a derived DDL fingerprint (#2798) (#2808)
* feat(schema): derive a fingerprint from the DDL this build creates `SCHEMA_FINGERPRINT` is a sha256 digest of the node and relation DDL that `runSchemaCreationQueries` actually executes, in the same shape as the existing `taintModelVersion` stamp (hex, sliced to 12). It exists because `INCREMENTAL_SCHEMA_VERSION` is hand-picked and has to *predict* whether an on-disk database matches this build's DDL. That number has collided with `main` eight times, twice exactly — and an exact clash is the quiet one, because the reuse gate is a strict `===`. `EMBEDDING_SCHEMA` is deliberately excluded: its `FLOAT[N]` width comes from `GITNEXUS_EMBEDDING_DIMS` at module load, so folding it in would make the digest a function of the environment rather than of code, and two runs of the same build under different env would thrash full rebuilds. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(storage): record the DDL fingerprint in RepoMeta `RepoMeta.schemaFingerprint` stores the digest of the DDL an index's tables were actually created from. It is the derived companion to `schemaVersion`, not its replacement: both are compared, and both must match. Absent means mismatch, deliberately. Grandfathering a missing fingerprint would let an incremental top-up stamp a fresh one onto a database whose DDL was never verified, permanently certifying exactly the wrong-shaped index the field exists to catch. The cost is one full rebuild per pre-existing index. The version ladder gains a note that its "re-check against origin/main before merge" ritual now only guards *semantic* bumps. v25, v26, v30, v31 and v34 all changed emitted ids, edges or wire formats while leaving the DDL byte-identical, and the fingerprint cannot see any of them — but DDL collisions no longer need renumbering. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(analyze): gate index reuse on the DDL fingerprint, not just the version (#2798) `INCREMENTAL_SCHEMA_VERSION` is a hand-incremented integer that has to predict a derived fact: whether the on-disk DDL matches the code's DDL. It has collided with `main` eight times, and twice the collision was *exact*. An exact clash is the silent one. Two builds stamp the same number over different DDL, the `===` reuse gate reads the index as current, every `CREATE ... TABLE` is then skipped as "already exists" (suppressed in `runSchemaCreationQueries`), and the edges whose endpoint pair the live database cannot hold are dropped by `fallbackRelationshipInserts`' bare `catch`. The result is a wrong graph, with no error anywhere. Reuse now requires the version AND the DDL fingerprint to match, in both the pre-pipeline force-rebuild guard and the `isIncremental` predicate, and the fingerprint is stamped alongside the version at the end of a run. Both conditions are necessary. The fingerprint does not replace the integer: most entries in the version ladder change emitted ids, edges or wire formats while the DDL stays byte-identical, and a fingerprint-only gate would stop forcing rebuilds for all of them. What it does buy is that two branches picking the same number no longer need renumbering. The new branch sits above the `alreadyUpToDate` fast path for the same reason the version guard does — a clean tree at an unchanged commit would otherwise early-return before either check ran. Closes #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(analyze): pin the DDL fingerprint gate and its two failure cases `schema-fingerprint.test.ts` pins the properties the gate rests on: the digest covers exactly the node and relation DDL that gets executed (recomputed from the exported lists, so adding a table or a FROM/TO pair without the fingerprint moving is impossible), it excludes the environment-derived embedding DDL, and it moves when any covered string moves. The two `incremental-orchestration` cases exercise the production path rather than modelling it: an index carrying the *current* version with a foreign fingerprint, and one with no fingerprint at all. Both were run against the pre-fix tree first and both failed there with `alreadyUpToDate === true` — the fast path swallowing the mismatch, which is the #2798 symptom exactly. `call-summary-schema-version.test.ts` widens its gate model to two equalities. The second argument defaults to the current fingerprint so all 33 existing version cases read unchanged, and a new case covers the collision, the legacy absence, and the semantic bump the fingerprint cannot see. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(review-skill): point the schema-constant check at the fingerprint, not the deleted integer All four `gitnexus-review` SKILL.md mirrors told reviewers to verify `INCREMENTAL_SCHEMA_VERSION` "was bumped or regenerated". That constant no longer exists, so the instruction sent every future reviewer looking for something they could not find — and, worse, past its replacement. The check for graph DDL is now derived: `SCHEMA_FINGERPRINT` moves on its own, so the question is whether the diff changed a string in `NODE_SCHEMA_QUERIES` / `REL_SCHEMA_QUERIES`, and whether a newly added DDL array was folded into the fingerprint at all — the one way the derived gate can still be bypassed. What did NOT change is called out explicitly: the parse-store `SCHEMA_BUMP` and the bench fingerprint sets are still hand-maintained and still need the re-check-against-base ritual, and semantic changes that leave the DDL untouched fall outside the fingerprint entirely — those rely on the analyzer runner-identity receipt. Found by the review swarm's docs lane. The original plan for #2798 claimed no documentation mentioned the constant; that sweep covered five root docs and never looked at `.claude/skills/**` or the three mirrors. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(migration): record the one-time rebuild the fingerprint switch costs Replacing `schemaVersion` with `schemaFingerprint` means every index written by an earlier GitNexus carries no fingerprint, reads as a mismatch, and is rebuilt once. That is deliberate — grandfathering absence would stamp a fresh fingerprint onto a database whose DDL was never verified — but until now it was undocumented, so a user's first post-upgrade analyze would announce a full re-analyze with nothing to explain it. MIGRATION.md already sets the precedent: PR #2363's meta.json → gitnexus.json rename was equally automatic and equally in need of an entry. This follows that shape, and is explicit about the parts that are easy to undersell: - the cost is per INDEX, and branch-scoped slots (#2106) each pay separately; on a large repository a full re-analyze is substantial, not a blip; - rollback is safe — an older binary sees no `schemaVersion` and forces its own rebuild, which is a cost, never a stale graph; - alternating between an old and a new binary rebuilds on every switch, because the end-of-run meta is written as a fresh literal so neither field survives the other's run. The retired ladder's per-version rationale is pointed at in git history rather than reproduced: `git show 561f913a3:.../repo-manager.ts`. That commit is an ancestor of origin/main, so the pointer survives this branch being squash-merged. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(identity): cover workspace-linked packages in the analyzer dependency digest `dependencyNames` enumerated `dependencies`, `optionalDependencies` and `peerDependencies` only. `gitnexus-shared` is declared as a devDependency (`file:../gitnexus-shared`), and in a source-mode run the build root is the gitnexus package tree, which does not contain that sibling. So a change to gitnexus-shared moved neither `build.digest` nor `dependencyRuntime.digest`. That gap matters more since #2798 deleted `INCREMENTAL_SCHEMA_VERSION`. A DDL-affecting edit there is still caught by `SCHEMA_FINGERPRINT`, but a SEMANTIC-only edit — a new `REL_TYPES` member, say, where the relation table carries a bare `type STRING` column so no CREATE statement moves — was covered by nothing at all. Roughly thirty of the retired ladder's entries were exactly that change class, and the runner-identity receipt is what now carries them. Only checkout-local specifiers are added: `file:`, `link:`, `workspace:`, `portal:` and npm's bare local-path shorthands. Pulling in every devDependency was rejected — vitest, eslint and typescript would enter the digest and force a full re-analyze on unrelated tool bumps, which is worse than the hole. Scanning the linked sibling for the first time exposed a latent throw: `collectArtifacts` honoured `PRUNED_RUNTIME_DIRECTORIES` only for a real directory, so a SYMLINKED `node_modules` fell through to the payload branch and died with "Analyzer identity input is not a file". Worktree-style dev layouts and pnpm shared stores hit this immediately — verified in this worktree, where `gitnexus-shared/node_modules` is such a symlink. Pruning it loses nothing: packages beneath are still reached through `resolveDependencyPackageRoot`. Verified: a real `analyze` in this worktree succeeds with `packageCount` 259; editing the linked package's source moves the digest, bumping an installed registry devDependency does not, and removing the link moves it. `DEPENDENCY_RUNTIME_CANONICALIZATION` is deliberately not bumped — freshness compares digests, not the label, and the input-set change already moves them. Follow-up worth having: no fixture in the suite declares `devDependencies`, so this has no regression test yet. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(analyze)!: delete INCREMENTAL_SCHEMA_VERSION, gate reuse on the DDL fingerprint alone The integer and its ~180-line version ladder are gone, along with `RepoMeta.schemaVersion`. Index reuse is now decided solely by `SCHEMA_FINGERPRINT`; a mismatch — including the absent stamp every pre-existing index carries — warns and forces a full re-analyze, which wipes and recreates the database so the tables are built from the current DDL. Deleting the integer is safe because it was already redundant: the runner-identity guard deep-compares the whole schema-v4 receipt, including a digest over the build tree, and forces a rebuild on ANY analyzer delta. Verified empirically — a comment-only edit to logger.ts, with the fingerprint byte identical, produced "runner identity changed ... forcing a full rebuild". The fingerprint is not thereby redundant. It fires where that guard cannot: a DDL-affecting change in `gitnexus-shared`, which is a workspace-linked devDependency and so sat outside both digests until the companion commit closed that gap. Review findings folded in, each correcting a line this rewrite itself introduced and never published: - B1: two assertions matched a log string the rewrite had renamed; both tests failed. They now assert what production emits. - B2: the pre-existing downgrade test perturbed `schemaVersion: 7`, a field this change deletes, so the spread carried a valid fingerprint, every guard passed, and the run legitimately took the fast path. It perturbs the fingerprint now, restoring the only integration coverage of the gate-above-the-fast-path ordering invariant. - N5: duplicate `schemaFingerprint` keys silently collapsed two assertions into one (TS1117). - N6: the absent-stamp message told non-git repositories their index was "built by an older GitNexus version" — on every run, about an index this exact build had just written. Non-git repos never record a fingerprint, and now the message says so. - N9: the on-disk stamp is shape-checked before being echoed, so a crafted gitnexus.json cannot push ANSI escapes through the CLI log. - N7: a test case that re-computed the same digest expression with its operands swapped, mislabelled as a randomness check on a module-level const. - N10: comments claiming the digest "cannot collide" (it is 48 bits), pointing at a vector-column gate that does not exist, and asserting storage/ is free of a core/ dependency two lines below a core/ value import. None of these were caught by `tsc -p tsconfig.json`, which covers src only, nor by eslint, where no-dupe-keys is off. `tsconfig.test.json` reports all three test defects and is not currently wired into CI. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(schema): pin that the fingerprint covers every DDL statement init executes `SCHEMA_QUERIES` is the list `runSchemaCreationQueries` iterates — the DDL that actually runs. The fingerprint hashes only two of its three members, and until now no test imported `SCHEMA_QUERIES` at all, so nothing tied the two together. A fourth member appended to that array — the one literally named for what init executes — would have been invisible to the gate. Every existing test would still pass, because they all recompute the digest from the same two arrays the fingerprint already uses. An index whose gate passed would then run `initLbug` over the old database, where `runSchemaCreationQueries` suppresses "already exists", so the new table would never be created and its edges would be dropped by `fallbackRelationshipInserts`' bare catch. A wrong graph, no error — exactly the failure #2798 exists to end. The check is a pure predicate over (executed, fingerprinted, documented exclusions) rather than a positional `toEqual`, so `EMBEDDING_SCHEMA` is named as an exclusion with its reason — its FLOAT[N] width is environment-derived — rather than sitting in a list where a future reader might "fix" it by folding it in. It asserts both directions and is order-insensitive, leaving ordering to the digest assertion that already pins it. The negative case is pinned in CI rather than checked by hand once: the same predicate over a synthetic fourth member must report it. If a refactor ever makes the predicate vacuous, that case fails even though the positive one would not. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(analyze): name the invariant the version deletion now rests on Deleting `INCREMENTAL_SCHEMA_VERSION` moved a load-bearing guarantee into an implicit one. Roughly thirty of the retired ladder's entries changed no DDL at all — node ids, wire formats, resolution tiers — and the fingerprint is structurally incapable of firing on any of them. Their only remaining cover is the analyzer runner-identity receipt, and nothing in the suite said so. This adds a table over the real `analyzerRunnerIdentitiesEqual` with a well-formed schema-v4 receipt: byte-identical reuses; an entrypoint-only difference reuses (CLI vs analyze worker); a moved build digest with unchanged DDL forces — that case IS the invariant, commented as such; and a dependency change, an ABI change, undefined, null, a schema-v3 legacy receipt, a missing build section and a non-sha256 digest all fail closed. The deleted `expect(INCREMENTAL_SCHEMA_VERSION).toBe(35)` pin is also worth naming: it failed CI on every bump by design, which is what made an author stop and think. Nothing replaced it. This does not restore that — a digest has no literal to pin — but it does make the mechanism that took over the job visible to the next person who reads the file. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(spring): pin CLASS_SCHEMA's membership in the fingerprinted DDL set When `INCREMENTAL_SCHEMA_VERSION` went away, its sibling in basicblock-callee-ids-schema.test.ts got a replacement assertion tying BASICBLOCK_SCHEMA to the fingerprint's input set. This file's `>= 23` floor was deleted with nothing put in its place. The file still asserts CLASS_SCHEMA's CONTENT — that the `frameworkAnnotations` column exists — but not that CLASS_SCHEMA is part of what the digest covers, and the second is what makes an index built before that column carry a different fingerprint and get rebuilt. Mirrors the sibling so the two read the same way. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(identity): stop a symlinked directory from aborting the whole analyze `collectArtifacts` fused two orthogonal facts into one condition: that four directory names never carry runtime payload, and that a symlink where a real directory was assumed falls through to the payload branch, where `snapshotReadableFile` stats the target, sees a directory, and throws "Analyzer identity input is not a file". The second was only fixed for those four names. Every other symlinked directory in a scanned package root still aborted the run — `dist -> build`, a vendored grammar link, anything inside a linked sibling checkout. Newly reachable, because making workspace-linked packages scannable pointed the scanner at a live checkout instead of an immutable registry tarball for the first time. Split along the actual seam: prune on the NAME alone, and give symlinks their own branch in the type dispatch, ahead of the payload branch. Link text is recorded rather than followed. Following was rejected on three grounds, each checked in source: the traversal is a stack with no visited set, so a self-referential link would recurse to `runtimeDepth` — which throws, trading one hard abort for another; `snapshotDirectory` rejects a symlink outright, so the directory guard could not accept one without a realpath rewrite of its canonical-path identity; and a link into an already-scanned tree double-counts against `runtimeEntries`/`runtimeBytes`, which also throw. The cost is stated in code: a link out of the package contributes its text, not its target's content. Links resolving to a regular file keep the existing content digest. The new `'unfollowed-symlink'` kind is threaded through every consumer, including the cache validator — which re-probes with `mode: 'link'`, since the readable-file probe resolves the target and would return null for exactly this kind, silently failing every warm validation. No canonicalization or cache-schema bump. Digest content changes only for trees that previously crashed: a delta scan over all 258 scanned roots of this install found no regular file bearing a pruned name and no symlink failing to resolve to a file, so `dependencyRuntime.digest` is byte-identical here. Six of the eight new tests fail against the unfixed tree with the exact production error; all eight pass after. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(analyze): give the reuse gate a real seam and sanitize logs at the funnel Cleanup pass over the #2798 branch. Net -183 lines. The gate had no extracted predicate, so its own test asserted it by regex-matching run-analyze.ts SOURCE TEXT. That pinned production formatting: one pattern froze three back-to-back single-name imports from './lbug/schema.js', so merging them — the obvious tidy-up — failed a test named "still imports the DDL digest itself". `schemaFingerprintMismatch` and `isSchemaFingerprintShaped` now live in core/lbug/schema.ts beside the constant. Not in run-analyze.ts next to `pdgModeMismatch`, because storage/ must stay off the analyze pipeline and mcp/resources.ts is a plausible second consumer — the same reasoning that puts `cjkSegmentationModeMismatch` in core/search/. The regex block is gone; the test calls the predicate. The three imports are merged. ANSI sanitation moved from one field to the funnel. The per-field guard's own comment stated the general hazard — gitnexus.json is parsed with no runtime shape validation and the notice reaches console.log — while two sibling guards twelve lines away echoed `runnerIdentity.schemaVersion` and `cjkSegmentation` from that same file raw into the same log. `log()` now strips C0/C1 controls, covering all seven guard messages and any written later. Also: - Deleted a duplicate integration test. After the downgrade test was repointed at `schemaFingerprint` it became the same scenario as the new one, differing only by an extra log assertion — which is now folded into the survivor. Saves a fixture and two full pipeline runs per CI pass. - Replaced a 3-parameter set-difference helper with one set equality. Its doc was false at one call site (arguments semantically swapped) and it needed a fourth test purely to prove itself non-vacuous; set equality cannot go vacuous. - Removed ~115 lines of runner-identity table that duplicated analyzer-identity.test.ts. The three genuinely uncovered cases moved there, and the #2798 invariant — build digest moved while the DDL did not — now asserts against a REAL analyzer-build-tree edit rather than a hand-built literal, which is strictly stronger than what it replaces. - MIGRATION.md quoted a log line the code cannot emit; it was written before the placeholder changed. - Restored the rationale on the `capabilities` docstring, which a previous pass replaced with its consequence — leaving a maintainer reading "duplicated by hand" as a wart to fix by importing, which is what the original forbade. - Marked the `isIncremental` conjunct as belt-and-braces: `!options.force` short-circuits before it, so it cannot decide anything. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(analyze): force a rebuild when the vector column width changes `CodeEmbedding.embedding` is declared `FLOAT[EMBEDDING_DIMS]`, resolved from `GITNEXUS_EMBEDDING_DIMS` at module load. Nothing gated it. Flip the variable on a same-commit clean tree and no guard fired at all: `alreadyUpToDate` returned over a `FLOAT[384]` table while the process embedded at 768. The only reaction anywhere discards the embedding CACHE and re-embeds — into a column whose type it never revisits. This predates #2798; `INCREMENTAL_SCHEMA_VERSION` never covered dims either. It surfaced because the fingerprint work had to reason about why `EMBEDDING_SCHEMA` must stay OUT of the digest: its width is environment-derived, so folding it in would make the same build disagree with itself and thrash rebuilds. That exclusion is correct, and it leaves the width needing its own guard. Modelled on `cjkSegmentation`, the closest sibling: an env-resolved scalar stamped at write time and compared by a small exported predicate that forces on mismatch. `embeddingDimsMismatch` sits in core/lbug/schema.ts beside `EMBEDDING_DIMS`, so the query side can adopt it without importing the analyze pipeline — mcp/local/local-backend.ts already warns on a cjkSegmentation disagreement and has the identical claim here, since the query path embeds at the live width against a table of unknown width with no validation at all today. ABSENCE IS NOT A MISMATCH, deliberately. Forcing on it would be dead code: `embeddingDims` and `schemaFingerprint` ship together, and a missing fingerprint already forces exactly one rebuild — which is where this stamp lands. Absence also carries no signal here, unlike the fingerprint: a missing fingerprint means "DDL this build cannot vouch for" and ships WITH a DDL change, whereas a missing dims stamp means only "written before the field existed", and that run's table agreed with that run's width. Drift requires the env to change, which absence says nothing about. The `cjkSegmentation` trick of folding absence into the default was unavailable — there is no width that is safe to assume for an existing table — so the stamp is instead written unconditionally, giving absence exactly one meaning. Malformed values are not grandfathered: null, '384', NaN and objects all read as a mismatch and err toward a rebuild. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(mcp): warn when the served index's vector width differs from the query embedder's The analyze side now forces a rebuild when the vector column width changes. The query side had no equivalent: a serving process embeds a query at its own width and searches a table whose width was fixed when the index was built. Disagree and the user gets wrong or missing semantic results with nothing explaining why. Mirrors the cjkSegmentation drift warning immediately above it — same warnings[] array, same per-query recomputation, agent-visible in the tool response, and it warns rather than refuses. A width mismatch degrades the semantic lane only; keyword results are unaffected, so `partial` is deliberately not set. Compares against `getEmbeddingDims()` — the width the query embedder actually produces — NOT schema.ts's `EMBEDDING_DIMS`. The two diverge exactly when GITNEXUS_EMBEDDING_DIMS is set on a server that embeds LOCALLY: the query path ignores that variable and embeds at 384, so comparing against the env-derived constant would report drift on a lane that works fine. The recorded width is what the vector CAST actually binds. `embeddingDimsMismatch` is imported from core/lbug/schema.js rather than restated, so "absent is not a mismatch" cannot drift between the analyze and query sides. That predicate was placed in schema.ts precisely so this consumer could reach it without importing the analyze pipeline. Two gates keep it quiet when it would be noise: it fires only for a repo where this process actually produced a query vector, so an index analyzed without --embeddings (or a server whose embedder is unavailable) never carries it. An untrusted recorded value — meta.json is schema-less JSON — is reported as "an unrecognized width" rather than echoed. `loadMeta` is hoisted out of the neighbouring try so both diagnostics share one read and an invalid GITNEXUS_FTS_CJK_SEGMENTATION cannot take this one down with it. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(identity): detect an npm-linked dev dependency the specifier cannot see `isLocallyLinkedSpecifier` admits a devDependency whose SPECIFIER is checkout-local. `npm link <pkg>` leaves the specifier a registry range while the node_modules entry symlinks to a checkout — locally linked, invisible to a specifier check, so a semantic-only edit there still moves neither digest. The obvious placement is unaffordable, measured rather than assumed: probing every dev-only name inside collectRuntimePackages costs 1998 resolutions, not the ~8 it looks like, because dependencyNames runs for every package in the BFS and published tarballs retain their devDependencies. Persisted path guards go 2221 -> 11050 (+398%), and every guard is re-probed on each warm validation — the path `status` takes. Scoped to the root package instead. The declared-intent half is untouched and still enumerated everywhere: it alone can emit the `<missing>` edge for a declared link whose checkout is absent, where resolution returns null and cannot distinguish that from an uninstalled dev tool. The new resolved-location half runs only when `parent.root === packageRoot`, resolves through the existing resolver so its path guards are recorded, and admits a name iff the realpath'd root carries no node_modules segment. Bounded against mis-fire by EXPANSION. "Not under node_modules" is a proxy for "checkout-local"; under a relocated pnpm virtual store every dev dep passes it and the whole dev tree folds into the receipt — against limits that THROW, so a legitimate install would abort. Measured here: uncapped, that shape takes 259 -> 347 packages and 2250 -> 3786 guards. The cap admits at most four and DROPS THE WHOLE CHANNEL on overflow rather than an arbitrary prefix, because the abort comes from the transitive payload of whichever trees get folded in — four of a mis-fired thirteen is still unbounded, and a sorted-prefix receipt would be arbitrary. Overflow falls back to the specifier-only receipt that ships today. Cost on this install: 259 packages unchanged, 13 dev names resolved, guards 2221 -> 2250 (+29, +1.3%). Verified against the real implementation, not just a replay: validation guards 16295 -> 16324, packageCount and artifactCount unchanged, and `dependencyRuntime.digest` byte-identical — so this forces no re-analysis for anyone. Each test fails on the defect it targets: disabling the channel kills the npm-link and cap cases; dropping the root-only scope makes the differential guard-count case fail at 2.8x guards; removing the specifier half kills the `<missing>` case. Refs #2798 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
990d79ba8c
|
fix(mcp): make impact/context reproducible — deterministic ordering on every capped query (#2787) (#2796) | ||
|
Gergő Magyar
|
010a7d806a
|
fix(schema): declare the full scope-resolution relation cross product (#2792) (#2793)
* fix(schema): declare the full scope-resolution relation cross product (#2792) `RELATION_SCHEMA` was hand-listed, and every prior fix added only the FROM/TO pair named in a crash report — `Const→Method` in #2769, the Swift/Rust member pairs before it. So `analyze` kept aborting at `assertDeclaredPair` on the next codebase whose edges happened to land on a different pair; #2792 reports `Class→Variable` on Java. Audit the surface instead of the symptom. `buildGraphNodeLookup` skips any node whose label is not in `isLinkableLabel`, so the lookup holds only linkable-labelled nodes — and both endpoints of every graph-bridge edge resolve through that lookup. The emittable surface is therefore exactly: FROM LINKABLE_LABELS + File (the module-level caller fallback) TO LINKABLE_LABELS + CALL_TARGET_TYPES `isCallerAnchorLabel` is a strict subset of linkable and contributes nothing on top. `CALL_TARGET_TYPES` contributes `Delegate`, which `tryEmitEdgeWithExplicitTargetId` can emit without going through the lookup at all. Generate that 14x14 block into the DDL rather than listing it: 223 -> 322 declared pairs, and no future pair from these sets can be missing by construction. The containment/inheritance/DI/route/cluster/PDG pairs stay hand-declared — no single predicate describes them. Both label sets live in the ingestion layer, which `core/lbug` must not import, so schema.ts carries twin lists. test/unit/schema-pair-coverage.ts derives the requirement from the originals and fails CI when either set grows without the pairs landing here — the piecemeal loop this fix ends. Measured before widening: at 322 pairs the cost is inside noise (1.09s vs 1.12s per 300 anchored queries on a 32-table DB), but the full 32x32 cross product is ~1.8x on untyped-endpoint anchored queries. The audited subset is the right scope, not "declare everything". INCREMENTAL_SCHEMA_VERSION 34 -> 35: LadybugDB fixes endpoint pairs when the rel table is created, so a pre-v35 database physically cannot store these edges. Closes #2792 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(schema): declare the non-bridge structural pairs COBOL and Vue emit The generated scope-resolution block closed the half of RELATION_SCHEMA a label predicate can describe. The hand-declared half was still stale: with #2791's Function->Variable fix applied, `analyze` continued to abort on this repo's own test/fixtures/lang-resolution with Relationship label pair Module→Property is not declared A full sweep (assertDeclaredPair patched to log-and-skip, run over the whole fixture corpus) found 13 undeclared pairs over 106 edges. This branch already covered 3 of them via the cross product; the remaining 10 come from emitters outside the graph bridge: - cobol-processor.ts mints Module / Namespace / Record / Property / CodeElement and wires them with CONTAINS, CALLS and ACCESSES (9 pairs) - vue-sfc-extractor.ts emits BINDS_EVENT_HANDLER from a handler Function to the child component's File, the only edge whose target is a File (1 pair) CodeElement, Namespace, Record and File are in neither scope-bridge label set, so neither the generated block nor schema-pair-coverage.test.ts can reach them. Adds test/integration/structural-pair-coverage.test.ts, which derives the requirement from a corpus instead of a predicate: it runs the real pipeline over the non-bridge fixtures and requires every FROM/TO pair they produce to be declared. Mutation-checked — dropping `FROM Function TO File` fails it with exactly Function|File. Verified: cobol-app, vue-basic and php-transitive-traits now index instead of aborting; the full lang-resolution corpus completes at 10,876 nodes / 18,517 edges; scrypster/muninndb at 0b7a4272 (the #2789 repro) completes at 20,069 nodes / 71,580 edges, matching #2791 exactly, so this supersedes that PR. * refactor(test): simplify the structural pair coverage guard Cleanup pass over the previous commit. No behaviour change to the schema. - reuse `FIXTURES` and `runPipelineFromRepo` from resolvers/helpers.ts instead of re-deriving the fixture root and importing pipeline.js directly - gate on `distWorkerExists()` like every other integration test that passes `workerUrlForTest`, so a missing dist skips rather than fails - run the three fixtures with `it.concurrent.each`; they share nothing and the cost is almost all worker spawn plus grammar load, which overlaps well (tests phase 21-24s -> 5.6s measured) - replace the sentinel-in-a-Set filter with a plain `.filter()` chain, matching the sibling unit test, and move the declared/table lookups off the per-edge path onto the deduped set - move the pure string pin out of the integration tier into schema-pair-coverage.test.ts, where the identical construct already lives, so it needs no build and survives fixture deletion - trim the schema and test prose that restated the code, and correct the BINDS_EVENT_HANDLER attribution: it is emitted by languages/vue/scope-resolver.ts, not vue-sfc-extractor.ts - amend the v35 comment to mention the 10 structural pairs it now also stamps Still mutation-checked: dropping `FROM Function TO File` now fails both the integration sweep and the unit pin with exactly Function|File. 89 tests green. * fix(schema): generate the attachment pair surface and close four analyze aborts Review of the generated scope-bridge cross product found four `analyze` hard-aborts still live at head, each reproduced end-to-end on the default user path (`analyze --index-only --skip-git`): Method→Annotation Spring `@Bean` + `@ConditionalOnMissingBean` (Java + Kotlin) Method→File Vue Options-API `methods:` handler bound to a child event Namespace→Record COBOL `DECLARATIVES` / `USE AFTER STANDARD ERROR ON <file>` Class→Tool `@mcp.tool()` applied to a class All four are pre-existing on main, and both existing guards were structurally blind to them: the unit guard derives from LINKABLE_LABELS ∪ CALL_TARGET_TYPES (none of Annotation/Tool/Record/File-as-target is a member) and the corpus guard ran three fixtures that exercise none of these emitters. All 16 tests passed while all four crashes were live. The PR's model — "bridge endpoint × structural endpoint" — does not fit: Namespace→Record is structural on both sides. The property that does hold is that the ANCHOR is a lookup result, not a literal at the emit site, so the emitter cannot constrain its label. That gives a second closed-form rule: DEFINITION_ANCHOR_LABELS × ATTACHMENT_TARGET_LABELS DEFINITION_ANCHOR_LABELS is derived from NODE_TABLES by subtraction, so a new node table joins automatically. 332 → 450 declared pairs. Sized against a committed harness (gitnexus/bench/schema-pairs), real @ladybugdb/core, identical data: 450 costs 0.93–1.05× of 332 on untyped-endpoint anchored queries — inside noise — versus 1.22–1.43× at 641 and 2.03–2.34× at 1024. The harness reproduces the known #2792 cliff, which is what makes the 450 figure trustworthy. Also in this change: - Delete the 161 hand-declared pairs the rules already generate (233 → 72). The declared set is byte-identical at 450; those lines were load-bearing shadow, because the generator suppresses anything already declared structurally, so narrowing a rule later would silently keep pairs alive. A new guard fails CI if a hand-declared pair is ever re-added inside a rule. - Import LINKABLE_LABELS / CALL_TARGET_TYPES instead of hand-copying them. The twins' stated justification ("the ingestion layer must not be imported here") is false: csv-generator.ts and lbug-adapter.ts, siblings in the same directory, already do, and no rule in AGENTS.md / ARCHITECTURE.md / CONTRIBUTING.md / GUARDRAILS.md states otherwise. - Resolve `resolveStreamGraphEmit` after the guards that rebind `options.force`, not at function entry. It gates on `force`, and every freshness guard runs ~360 lines later, so the v34→v35 bump would have pushed every existing index down the non-streamed emit path — losing the #2680 memory streaming added for the #2649 kernel-scale OOM, for exactly the population most likely to be memory-constrained. - `UndeclaredRelationPairError` now carries the relationship type, both node ids and the source file, with a matching CLI branch. The old message named only the abstract label pair, which a user could not act on. Found through the cause chain, since pipeline-phases/runner.ts rewraps every phase failure. - Share one classifier (`relPairKeyFor`) across the router, both emit sinks and the corpus guard, which previously hand-mirrored the router's skip rule; one cause-chain walker in lib/utils.ts; one exported pair-matching regex. - Corpus guard: four new fixtures reproducing the aborts, per-fixture sentinel pairs so a fixture that stops emitting fails loudly instead of passing vacuously on an empty graph. The per-edge path stays allocation-free: the failure context is passed positionally and the message is built only inside the throw. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182jkjQqzACkJKYw4MLDnhX * test(bench): re-baseline the COBOL capture fingerprint for the new fixture `bench/scope-capture` globs `lang-resolution/cobol-*`, so the `cobol-declaratives` fixture added in |
||
|
Gergő Magyar
|
911151e230
|
fix(resolution): resolve Go pointer-receiver calls, and report the program boundary instead of hedging (#2766) (#2782)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
Karl Lehenbauer
|
639eb04b31
|
fix(swift): preprocess indented conditional directives so class bodies survive parsing (#2771)
* fix(swift): preprocess indented conditional directives so class bodies survive parsing * fix(swift): make conditional-directive blanking comment-, string- and brace-aware (#2771) Addresses the review findings on PR #2771. The transform fired unconditionally, which turned valid Swift into parse errors while missing the most common shape it was written for. - The blank/keep decision now consults `blockCommentDepth`, so ` #endif */` — the result of commenting out a conditional block — keeps its comment terminator. Previously `hasError` went raw=false -> preprocessed=true and the rest of the file was swallowed. - The decision keys on the scanner's brace depth instead of indentation. A column-0 `#if` inside a class body is blanked (6 of 7 body shapes previously still lost the enclosing declaration) and an indented file-scope directive is not — matching what the doc comment already claimed. Bare-CR line endings, NBSP/ideographic indentation and a leading BOM are recognized too. - A group is blanked only when every branch is brace-balanced. An `#if`/`#else` that splits a declaration header leaves one unmatched `{` once both branches survive, which collapsed five top-level nodes into one and gave unrelated types fabricated `NetworkClient.` qualified names. Such a group now degrades to the pre-fix behavior. - Multiline strings honour `\"""` escapes, and a plain `"""` closes even when a `#` follows it, so the scanner no longer wedges in string state and silently stops blanking for the rest of the file. - The pound run is counted once per position and skipped. It was quadratic: 10.6s for one 64k-`#` line, well inside the 512 KB walker limit. - Extended regex literals (`#/.../#`) no longer open a phantom block comment. - Directive-free files return early, matching `stripUeMacros`. Worker parity: `emitSwiftScopeCaptures` and `emitCppScopeCaptures` re-apply their provider's `preprocessSource` on the parse-cache-miss path — Dart already did this — and the embedding parse in `ensureAndParse` applies the hook as well. Before this the worker and the scope-capture/embedding halves analyzed different programs, turning a consistent degradation into cold-run/warm-run non-determinism. A new parity test pins the equivalence for every provider that defines the hook. SCHEMA_BUMP 37 -> 38: this changes parse semantics, the chunk key hashes raw on-disk bytes, and `preprocessSource` runs after the key is computed — so a same-package-version warm cache would replay pre-fix Swift results verbatim, including across `--force`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(ingestion): apply preprocessSource once in the scope bridge (#2771) Follow-up cleanup on the review fixes. The previous commit re-applied each provider's `preprocessSource` inside `emitSwiftScopeCaptures` and `emitCppScopeCaptures`, mirroring what Dart already did — three copies of the same rule, and a contract that asked every future emitter to remember it. `extractParsedFile` is the single funnel every `emitScopeCaptures` caller passes through (parse worker, scope-resolution run, Vue script extraction), and it already receives the provider. Applying the hook there on the cache-miss path covers all three languages and every future one, names no language in shared code, and drops Dart's unconditional transform on the cache-hit path. Verified the three emitters use `sourceText` for nothing but the parse, so the substitution is output-identical — which the parity test asserts directly. Also from the cleanup pass: - the parity test derives its language list from the provider registry, so a new provider adopting the hook fails until it adds a fixture - `ensureAndParse` resolves the provider from the language it already computed, instead of a second extension table (`getProviderForFile`) - the preprocessor returns `sourceText` unchanged when no group was blanked, which is the common case for files whose only directives are top-level - `split(/(\r\n|\n|\r)/)` replaces the hand-rolled line splitter, and the per-group brace bookkeeping is two scalars instead of an array - the hint regex is derived from the line regex so the two cannot drift - unit assertions compare the WHOLE preprocessed file against the expected blanking, replacing per-line spot checks; the pipeline tests share one `runFixture` helper and `getNodesForFile` in the resolver test helpers - `LanguageProvider.preprocessSource` documents the real call sites and says plainly that the set is not closed — `populateRangeBindings` still hands language helpers raw text Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
azizur100389
|
d268f351d3
|
fix(group): preserve manifest-only impact crossings (#2784)
* fix(group): preserve manifest-only impact crossings Keep proven manifest cross-repo hits when the far endpoint has no concrete graph symbol, avoiding a guaranteed failed UID fan-out. * fix(group): verify manifest-only neighbor repos Keep manifest-only crossings from bypassing neighbor repository resolution so unavailable repos still surface as truncated fan-out. * fix(group): distinguish boundary-only impact crossings Keep manifest-only boundaries visible without treating unattempted fan-out as completed impact or escalating risk, and cover service scope, deduplication, and real bridge persistence. --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
MyShining
|
1147646518
|
feat(spring): model AOP transactions, caching, and security (#2783)
* feat(spring): model AOP advice and proxy behavior * fix(spring): address AOP review findings --------- Co-authored-by: Shining <xuenning@qiyi.com> |
||
|
Gergő Magyar
|
064832f50c
|
fix(mcp): resolve false FTS-missing warnings in the query tool (#2773)
* fix(mcp): surface resolved repo/branch/indexed-at in the FTS-degraded warning Turns the generic "FTS indexes missing" message into a diagnostic that reveals what this MCP session actually resolved, so a CLI/MCP mismatch or stale-connection theory is visible in the warning text itself instead of requiring a separate debugging round-trip (#2767). * fix(mcp): stop swallowing real FTS query errors behind the missing-index message queryFTSViaExecutor previously collapsed 'index genuinely missing' and 'a real query/connection error occurred' into the same silent null, so a real failure could masquerade as the generic FTS-degraded message with no diagnostic trail — even when it happened on only some of the per-table queries while others succeeded. Classifies the failure (mirroring queryFTS's own check for this exact cypher call), always logs a non-benign error server-side regardless of overall outcome, and surfaces it (redacted) in the client warning only when every table failed (#2767). * fix(mcp): give --repair-fts a dedicated freshness signal for warm readers --repair-fts intentionally never restamps indexedAt (it doesn't regenerate the graph), so a long-lived MCP session's pool staleness check had no explicit signal that a repair happened, only the incidental file-identity delta. Reuses the existing (forensic-only) capabilities.fts.status field: repair-fts now stamps just that sub-field (everything else byte-identical), and ensureInitialized compares it as a third, independent reinit trigger alongside the existing stamp/identity checks, seeded at cold init too so a fresh process's first warm check doesn't false-trigger (#2767). * test(mcp): warm session picks up an out-of-band --repair-fts rebuild (#2767) New end-to-end integration test: a real writable LadybugDB session builds an index WITHOUT FTS, a real LocalBackend observes 'FTS indexes missing' through the real pool, a separate writable session performs the exact repair-fts writes (real createSearchFTSIndexes + the #2767 capability-only meta stamp), and the SAME still-warm backend re-queries successfully without a restart — closing the one end-to-end gap no existing test covered. Running this against the real engine surfaced a second real message shape for a missing FTS index ("doesn't have an index with name X", not just "does not exist") that the U2 classifier didn't recognize — fixed classifyFtsQueryError to match both, with a regression test pinning the exact observed string. * fix(review): address code-review findings on the #2767 FTS fix - Anchor classifyFtsQueryError to the exception class (mirroring isBenignDropFtsIndexError) instead of a bare substring search, so a real, differently-classed error that happens to echo the benign phrase in its body (e.g. an echoed user query) can't be misclassified as a benign missing-index (adversarial review). - Re-read the on-disk meta immediately before the --repair-fts capability stamp write instead of reusing the pre-rebuild snapshot, so a concurrent writer (e.g. the HTTP server's background embedding checkpoint job) landing mid-repair isn't silently reverted. - Surface a client-facing partial-result warning (mirroring the existing enrichmentDegraded convention) when some FTS tables succeed but at least one hits a real error, instead of only logging it server-side. - Update RepoMeta.capabilities' stale 'no programmatic readers' docstring now that ensureInitialized reads capabilities.fts.status. - Widen the warm-session integration test's polling deadline for more margin over the production 5s staleness-check throttle. * fix(ci): drop the cold-init loadMeta call ensureInitialized never needed It stole the mocked loadMeta call an unrelated upstream PDG test depends on (test/integration/impact-pdg-statement-precise.test.ts queues a single mockResolvedValueOnce for its own PDG-config read; the extra call consumed that slot before the PDG code ran, so it fell through to the mock's null default and epistemic came back undefined instead of 'pdg-intra-procedural'). Cold init now leaves lastObservedFtsStatus unseeded — the cost is at most one redundant initLbug call on the first warm check, which no-ops via a single fs.stat when nothing actually changed, not a real reopen. * fix(review): address tri-review findings on the #2767 FTS fix Fixes two P1s (misleading repair-fts advice on real query errors; embedding-checkpoint job silently reverting the capabilities.fts stamp for up to its 30-minute lifetime), five P2/P3s (stale indexedAt in warnings, extension-unavailable noise, mismatched log severity, a table-missing vs index-missing conflation confirmed against a live LadybugDB, and a reinit-watermark latching bug), and the four residual items already self-disclosed in this PR's description (shared FTS error classifier, consolidated per-pool observed-state map, a redactPaths whitespace gap, and an isolated ftsCapsChanged test). A /simplify pass afterward caught one more real bug: the extension-unavailable short-circuit only guarded the MCP pool path, so the CLI-path fix above it started surfacing spurious non-benign errors for the same expected degraded state the pool path stays silent on — now both paths agree. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
azizur100389
|
84f584449d
|
fix(python): resolve classes through module imports (#2770) | ||
|
azizur100389
|
454d383416
|
fix(ingestion): join multi-line closure bindings on initializer startLine (#2735) (#2762)
* fix(ingestion): join multi-line closure bindings on initializer startLine Graph-node captures sit on the outer binding wrapper while scope-resolution anchors on the inner callable; the line-only position join missed when those split across lines and fail-closed dropped the real CALLS edge (#2735). * fix(ingestion): unwrap Ruby call+block for multi-line lambda joins Cover Kotlin/Ruby/Dart multi-line closure CALLS in integration tests, and dig Ruby's call/block field so do-end bindings join on the block start line. * style(ingestion): format closure join changes * fix(ingestion): make closure position join language agnostic |
||
|
MyShining
|
de84ad6297
|
feat(spring): index @Bean factories and @Resource injection (#2740)
* feat(spring): index Bean factories and Resource injection * fix(spring): address Bean and Resource review findings * refactor(lbug): keep relation pair parsing in router * test(lbug): preserve schema exports in WAL mocks * test(cache): align schema bump pin --------- Co-authored-by: Shining <xuenning@qiyi.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
27ab37c432
|
feat(resolution): type receiver chains from AST structure across all 14 languages (#2708) + epistemic lower-bound (#2744) (#2747) | ||
|
Gergő Magyar
|
9c24e3459e
|
fix(rust): qualify items by their enclosing mod chain so same-named ones stay distinct (#2742) (#2745)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(rust): let the qualified-call filter see inline modules The negative filter added in #2741 builds its set of known module names from FILE PATHS, so an inline `mod x { … }` — which appears in no path — was absent from it. Every module-qualified call into an inline module was therefore rejected before any candidate channel ran, which is a hole in that optimisation rather than in the resolution logic it guards. The per-pass index now unions the file-derived names with inline module names taken from the scope model: a `mod` declaration binds a `Namespace` def locally in the declaring scope, and that binding is the only place an inline module's name exists. Collected in the same walk that already builds the module → scope map, so it costs no extra pass. Found while fixing #2742, where a correctly resolved call into `mod inner { … }` still could not reach its target. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(scope-resolution): try the namespace-prefixed node key before the bare one `resolveDefGraphId` looked up the plain `qualifiedName` key first and only retried with the `namespacePrefix`-qualified key afterwards. For the defs that carry a prefix the qualified name is a bare TAIL, so the plain key happily matched a same-named item at a different namespace depth in the same file and returned it before the more specific retry was ever reached. The namespace-prefixed key is strictly the more specific of the two, so it is now tried first. Where no such node exists the lookup falls through to exactly the previous order, which keeps the #1982 behaviour this retry was added for. Without this, a call into `mod inner { fn dispatch }` resolved to the correct definition and then mapped it onto the crate-root `fn dispatch` node — the self-loop #2742 describes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): qualify items by their enclosing mod chain so same-named ones stay distinct (#2742) Node identity is `<label>:<file>:<qualifiedName>` and carried no module path, so an inline `mod inner { fn dispatch }` and a crate-root `fn dispatch` in the same file collapsed onto `Function:<file>:dispatch`, first-wins. Resolution already picked the right definition — the target simply was not representable, so a correct resolution still rendered as a self-loop and `impact` reported the real callee as unreached. The mechanism already existed: `qualifyRustImplTargetByModScope` has walked `mod_item` ancestors for impl targets since #1982. Generalised to `qualifyByEnclosingModScope` and applied to free items, so `mod inner { fn dispatch }` becomes `Function:<file>:inner.dispatch`. Keyed purely on the `mod_item` node type, exactly as the impl qualifier already was, so it is a no-op for every language whose grammar has no such node. Two constraints found by tests rather than by reading, both now encoded: - The helper normalised `::` to `.` unconditionally. With no enclosing `mod` that rewrote a top-level `impl a::Inner` from `a::Inner` to `a.Inner` and moved its node id away from the one the HAS_METHOD owner edge emits, breaking the #1975 scoped-impl ownership. It now returns raw text untouched when there are no mod segments, which also makes the change strictly additive for every id that has no enclosing module. - Qualification is scoped to items with no enclosing class/impl. A method already carries its owner's name, and that owner's id is mod-scoped by the impl qualifier, so qualifying the method again breaks the same byte-for-byte agreement. Same-named methods on same-named types in sibling modules therefore still collapse — a narrower residual than the free-item case fixed here, and one belonging to the owner edge rather than to this path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(storage): bump schema versions for the mod-qualified Rust node ids (#2742) `INCREMENTAL_SCHEMA_VERSION` 24 -> 25 and `SCHEMA_BUMP` 31 -> 32. Node ids change for every Rust item inside any `mod` block, and `#[cfg(test)] mod tests` makes that close to every Rust repository. A pre-v25 index therefore holds ids an incremental top-up cannot reconcile — the old nodes would simply be stranded — so the reuse gate has to force a full re-analyze. The qualified name is computed in the parse worker, so a warm parse cache would likewise replay the old unqualified ids and keep the collapse. This branch originally claimed v24; #2708 took that number and merged first, so it is renumbered to v25 here. That is exactly the collision the v29 note in parse-cache.ts warns about, and re-checking against origin/main at rebase time rather than at branch time is what caught it. #2708 did not touch `SCHEMA_BUMP`, so 32 is free. The version-pin test moves with the bump by design, including the new pre-v25 row in the reuse-gate table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): stop mod-qualifying container ids while their owner edges stay bare (#2745 review) #2742 re-keyed Rust node ids by the enclosing `mod` chain. The mint moved; the owner-edge anchor did not. `findEnclosingClassInfo` mints a member's owner id from the container's BARE `nameNode.text` and only follows a qualified shape when the provider sets `classExtractor.qualifiedNodeId`, which Rust does not. So every `struct` / `trait` / `enum` / `impl` declared directly inside a `mod` got a node id that none of its member edges pointed at. Five lines of idiomatic Rust were enough: pub mod engine { pub struct Config { pub retries: usize } } NODE Struct:src/lib.rs:engine.Config DANGLING HAS_PROPERTY Struct:src/lib.rs:Config -> Property:src/lib.rs:Config.retries The rows are discarded by the IGNORE_ERRORS COPY retry, so the struct silently lost every field. A trait impl inside a `mod` additionally dropped its METHOD_IMPLEMENTS edge outright. The same gap put `impl a::Inner` inside a `mod` back on the #1975 rake that `qualifyByEnclosingModScope`'s own docblock warns about. The impl-target branch deliberately fires only for an UNSCOPED `type_identifier`; the new gate had no such restriction and picked up the scoped targets that branch had just excluded, minting `Impl:<file>:outer.a.Inner` against an anchor still reading `Impl:<file>🅰️:Inner`. The member side was already excluded via `!enclosingClassInfo`. This adds the owner side, gated on `MEMBER_OWNER_NODE_TYPES` — derived from `CLASS_CONTAINER_TYPES`, which is already the single source of "this node type owns member edges" and already carries an INVARIANT note binding it to `CONTAINER_TYPE_TO_LABEL`. A language adding a container therefore cannot gain a mismatched id shape here without also failing that invariant. Keyed purely on tree-sitter node types, so no language name enters shared ingestion. `union_item` is listed too: its fields are captured as Property but it is not a recognized owner, so they carry no HAS_PROPERTY edge and cannot dangle — it is here so a union's id keeps the same shape as the struct beside it. Containers still collapse across sibling modules, exactly as before this fix. That residual belongs to the owner edge, and is not worked around here. Regression tests use the UNFILTERED `findDanglingEdges(result)`. Every other dangling assertion in `rust.test.ts` passes `['HAS_METHOD']`, which is precisely why the HAS_PROPERTY breakage shipped with a green suite. They assert the NODE id rather than only the edge's anchor, because the anchor was already bare while the bug was live — an edge-only assertion passes in both builds. All four fail when the new gate clause alone is reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * fix(rust): resolve modules nested inside an inline mod (#2745 review) The #2730 self-loop survived one `mod` deeper: pub mod outer { pub mod tools { pub fn dispatch() {} } pub fn dispatch() { tools::dispatch(); } } CALLS outer.dispatch -> outer.dispatch <- the #2730 symptom NODE outer.tools.dispatch <- correct target, unlinked Two gates were blind to a nested inline module, so the hook refused and the shared lexical tier bound the call to the enclosing same-name `dispatch`: `knownModuleNames` was collected by walking `moduleScopeByFile`, which maps a file to its ROOT `Module` scope only. A `mod` nested inside an inline `mod` binds in the parent module's scope, so the walk saw depth-1 inline modules and missed every nested one — `tools` never entered the set and the negative filter rejected the qualifier before any candidate ran. `declaresSubmodule` had the same root-only assumption, so even with the name known the candidate `outer::tools` was never yielded. Both now read the def index. Names come from every `Namespace` def; inline module PATHS are derived from the members' `namespacePrefix` rather than from the `mod` defs, because a `mod` def carries no nesting information of its own — inside `mod outer { mod tools { … } }` the inner def is `qualifiedName: 'tools'` with NO `namespacePrefix`, while every def within it is stamped `outer.tools`. A `Namespace` scope also owns its OWN def rather than its children's, so the scope tree cannot answer this either: the `mod outer` scope lists `outer`, never `tools`. Restricted to non-empty prefixes, so this stays a DECLARATION check. Including file-derived modules would let an undeclared or `cfg`-gated file on disk outrank a real `use` binding — the regression #2741's review already fixed once. File-backed submodules therefore keep going through the binding check. A module with no defs at all is absent from the set, which is harmless: it has no member for a qualified call to resolve to. Cost is one pass over an already-resident def index, memoized per resolution pass on the existing WeakMap — the same order of work as the binding walk it replaces, and it subsumes it. `isLocalNamespaceBinding` was going to single-source the duplicated "locally declared submodule" predicate the review flagged; deriving paths from members removed the second copy outright instead. Regression fixture covers depth 2 and depth 3, so the fix is depth-agnostic rather than depth-2 special-cased. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * fix(rust): let an imported type outrank a same-named module (#2745 review) Widening the negative filter to inline `mod` names let a type-qualified call through whenever a module happened to share the type's name. The crate-root-relative candidate then captured it: // src/lib.rs pub mod Buffer { pub fn with_capacity() -> usize { 111 } } // src/b.rs use crate::c::Buffer; // the real target lives in c.rs pub fn call() -> usize { Buffer::with_capacity() } base: (no CALLS edge — unresolved) PR: CALLS b::call -> Function:src/lib.rs:Buffer.with_capacity <- fabricated `ids.ts` states the doctrine this broke: a missing edge is the correct failure direction for a graph whose consumers include `impact`; a fabricated caller is not. The base produced the missing edge and the PR produced the fabricated one. That third candidate is the loosest of the three — a guess at a crate-root-relative path the caller never wrote, kept for 2015-edition style. In Rust 2018 a bare first segment resolves in the CALLER's module, so a local binding for that segment settles the question: it is now skipped when the head names anything non-module in the caller's own module. Candidates 1 and 2 are untouched, and they run first, so the legitimate `use crate::tools;` path is unaffected. The binding lookup goes through `lookupBindingsAt`. A first attempt read `Scope.bindings` directly and the guard never fired: a `use` binding is finalize OUTPUT and absent from the scope's own local table, which is exactly the imported-type case being guarded. Contract I8 in `contract/scope-resolver.ts` requires that channel anyway. The regression test asserts the forbidden TARGET rather than an empty edge set, and separately asserts the module member still exists as a node — otherwise the test would pass just as well if the call went unresolved for some unrelated reason, or if the module node disappeared entirely. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * fix(scope-resolution): try the namespace-prefixed name on the TAGGED keys too (#2745 review) `resolveDefGraphId` gained a namespace-prefixed retry for the plain qualified key in this PR, but the five tagged keys above it — template constraints, parameter types, parameter shape, arity, template arguments — kept composing from the bare `qualifiedName`. For a namespace- or `mod`-qualified def those keys are simply dead: `node-lookup.ts` registers them under the QUALIFIED name (`inner.dispatch#0`) while this side built `dispatch#0`. The keys exist to separate overloads, so a mod-scoped overload set was relying on whichever later key happened to catch it. Verified as a miss rather than a mis-hit before changing anything — an end-to-end run with a crate-root decoy of the same name and arity binds correctly — so this is hygiene, not a live bug. Worth doing while the code is open rather than leaving five keys dead and the behaviour dependent on fallback order. Both name forms now go through one `lookupTagged` helper, most specific first, so a sixth tagged key cannot be added with the bare form only. That also removes the five hand-repeated `qualifiedKey(...)` / `nodeLookup.get(...)` pairs. Also pins the C++ `EXTENDS` retarget this PR's reorder produces. `cpp-two-phase-dependent-base-cross-ns-deep` declares a global `Inner` decoy alongside `ns:🅰️🅱️:Inner`; the base's `qualifiedName` is a bare `Inner` with the path on `namespacePrefix`, so only the prefixed key separates them, and only if it runs first. The improvement was riding unasserted in a Rust-scoped PR. The captures golden covers every `rust-*` fixture, so the three fixtures added by this review series drift it; regenerated with UPDATE_GOLDEN=1. Verified: 785 tests across cpp / csharp / rust resolvers and the callable-id-lockstep unit test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * fix(rust): keep a mod declared inside a fn from hoisting above the callable (#2745 review) `fn wrapper() { mod helper { fn dispatch } }` minted `Function:<file>:helper.wrapper.dispatch@2:8` — the mod segment composed OUTSIDE the enclosing-callable prefix, inverting the real nesting. Nothing dangled: the `@line:col` suffix already makes a function-local callable's id unique, which is also why the mod segment adds no identity in this position. The path simply read as a lie about the source. It is now skipped rather than reordered — interleaving two qualifier passes to fix the order would be real machinery for a shape whose ids are already unique. Also folds in the three documentation and structure findings from the same review: - The 4-clause gate is extracted to a named `qualifiesByEnclosingModScope`, matching the two conditions directly above it in the same function, which were already named consts. - `qualifyByEnclosingModScope`'s docblock documented only the impl-target contract even though the generalized name has had a second, looser caller since #2742. It now states both, and says which gate belongs to which — that gap is what let the #1975 scoped-impl regression through in the first place. - The "cheap rejection BEFORE any index work" comment was no longer true: `passIndexFor` walks the def index on its first call in a pass. Corrected rather than left to mislead the next reader into thinking the filter is free. What it still buys — skipping the per-site candidate search, the part that scales with the workspace — is stated instead. Verified: 279 tests across the Rust resolver suite and the Rust scope-resolution unit tests. Captures golden regenerated for the extended fixture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * test(storage): move main's SCHEMA_BUMP pin to 32 for the mod-qualified ids (#2745 review) `#2736` added a pin asserting `SCHEMA_BUMP === 31` on main, which arrived on this branch through the merge of main while `0062a5c2` had already bumped the constant to 32. Neither side conflicted textually — the pin and the constant live in different files — so the merge was clean and the test failed instead. That is the pin working as designed: it exists so a bump cannot ride along unnoticed, and this is the fifth time a SCHEMA_BUMP collision has been caught by a guard rather than by review. Updated to 32 with the reason recorded inline. `INCREMENTAL_SCHEMA_VERSION` needs no second bump: 25 was introduced by this unmerged branch, so no released index carries it, and its own pin in `call-summary-schema-version.test.ts` is already consistent. Verified: 119 tests across the parse-cache, schema-version, incremental-orchestration and the two identity suites that arrived with the merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * test(bench): rebaseline the Rust capture fingerprint for the three new fixtures (#2745 review) CI caught what I missed: [scope-capture --check] FAIL: rust: capture fingerprint drift (got 05acbaca..., expected 90fda086...) fixture_count 202 `bench/scope-capture` fingerprints the whole `rust-*` fixture corpus, so the three fixtures added by this review series drift it. I rebaselined the `rust-captures-golden` snapshot and stopped there — a new fixture is a call site of BOTH, and updating only one is how this reached CI red. This is the same class as PR #2743's headline finding, from the other direction: an id-shape change makes every synthetic corpus a call site, and the author fixed the unit-test fixture and missed the bench. Here it is a fixture-count change rather than an id-shape change, and the review that flagged the #2743 lead as "REFUTED, bench/ has no Rust node-id corpus" was right about node ids and wrong about the corpus fingerprint. Noted for the next author in the baseline entry itself. Verified as pure corpus growth rather than a capture-logic shift: removing ONLY the three new fixture directories and re-running reproduces the prior fingerprint exactly (196 fixtures, capture_groups_fp 3432), and restoring them gives the new one (202, 3556). `emitRustScopeCaptures` is untouched by this series. Scaling 1.022 local / 1.057 CI, well inside the 1.5 budget. `bench/python-scope` globs `python-*` only and is unaffected; no other bench walks the Rust corpus. `--check` now PASSes for all 15 languages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
azizur100389
|
e723f3c2ee
|
fix(scope-resolution): parse def coordinates after file paths (#2743)
* fix(scope-resolution): parse def coordinates after file paths Anchor coordinate parsing to the known file path so coordinate-like path fragments and private symbol names cannot corrupt closure attribution. * fix(bench): use production definition ids |
||
|
azizur100389
|
bee3e82ab2
|
test(scope-resolution): guard closure identity invariants (#2748) | ||
|
Gergő Magyar
|
bc76ba2f25
|
fix(resolution): type inline constructor receivers in every spelling (#2708) (#2737)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(resolution): resolve constructor-expression receivers (#2708) `Service(db).do_work()` emitted no CALLS edge, so the caller was missing from `impact(direction: "upstream")` and `context()` while the two-step spelling of the same call (`s = Service(db)` then `s.do_work()`) resolved. The receiver reaches `resolveCompoundReceiverClass` intact — Case 0 in `receiver-bound-calls` routes it there because the text contains `(`. The free-call branch then only knew one shape: a function whose return-type binding names a class. A class has no return-type binding, so `Service` resolved to nothing and the member call was dropped. Handle the constructor shape: in languages that construct without a `new` keyword (Python, Kotlin, Swift, Scala) a free call naming a class IS a constructor call, so the expression's type is that class. The existing return-type path still runs first and wins, keeping this strictly additive — `new`-keyword languages never reach the new line because their receiver text keeps the keyword (`new Service(db)`), which matches no class binding. Verified on the issue's 4-file repro: `route_inline` now emits `CALLS → Service.do_work` and `impactedCount` goes 1 → 2. Note the issue's second ask — degrading `epistemic` to `lower-bound` when a receiver goes unresolved — is NOT addressed here. `computeEpistemicBoundary` keys only on the target's own heritage edges and runs at query time against the index, while unresolved references live in an in-memory `resolutionOutcomes[]` that is never persisted. That needs unresolved-receiver counts in the index first, so it is left for a follow-up. Tests: new `python-inline-constructor-receiver` fixture plus three integration cases (inline resolves, two-step still resolves, no cross-class fan-out). Two of the three fail without the source change. Full `test/integration/resolvers` suite passes (2928 tests) — the fix is shared across every language, so no-regression coverage matters more than the new cases. Python captures golden regenerated: additions only, no existing digest changed, confirming capture output is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * refactor(resolution): state the construction rule once, cover every spelling (#2708) The first commit fixed `Service(db).do_work()` by special-casing a bare class-name callee inside the free-call branch of the compound receiver resolver. That was the right rule in the wrong place: it covered one surface syntax out of three, and asserted rather than declared which languages it applied to. Probing the same shape across languages showed the bug is wider: | spelling | languages | dropped before? | |------------------------|--------------------|-----------------| | `Service(db).m()` | Python | yes | | `new Service(db).m()` | JS/TS, Java, C# | yes | | `Service.new.m()` | Ruby | yes | | both forms | PHP, Swift, Dart, | no — already | | | Kotlin | resolved | So the rule is stated once — "constructing a class yields an instance of that class" — and the per-language surface syntax is declared through a new `ScopeResolver.constructionSyntax` hook, matching how this file already gates language-varying behaviour (`stripReceiverCastExpressions`, `hoistTypeBindingsToModule`). Shared pipeline code names no language. - `bare: true` — Python - `keyword: 'new'` — JS/TS, Java, C# - `selector: 'new'` — Ruby, including the parenthesis-less `Service.new` spelling that reaches the chain walker rather than the call branch Opt-in is per-language for two reasons. Correctness: `bare` would mistype `stat(&st).field` in C, where a struct and a function may share a name. Evidence: PHP, Swift, Dart and Kotlin resolve this shape already, so they stay unwired instead of carrying a declaration that changes nothing — each verified by diffing analyzer output between builds with and without the change, not assumed. The keyword gate also keeps a bare factory call honest: in a `new` language, `makeOther(db).doWork()` still resolves through the factory's return type and is never read as constructing a same-named class. Tests: TypeScript fixture (inline `new`, a plain `.js` file for the javascript provider, two-step, and the factory guard) and a Ruby fixture (`Service.new` with and without an argument list, plus two-step). With the source change stashed, the inline cases fail and the factory/two-step cases still pass. The Python cases from the first commit are unchanged. No Kotlin fixture: its cases passed without the change, so they would document coverage this commit does not provide. Full `test/integration/resolvers` + `test/unit/scope-resolution`: 4234 passed, 1 skipped. Ruby captures golden regenerated — additions only, no existing digest changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * fix(resolution): only treat a construction selector as construction on the class itself (#2708) The `selector: 'new'` rule fired on any receiver whose type was class-like, which is true both when the receiver IS the class constant (`Factory.new`) and when it is a value of that class (`factory.new`). `isClassLike(...)` cannot tell those apart, so an instance receiver took the construction path too and skipped the member lookup that should have run. That replaced a CORRECT edge with a wrong one. Measured against the base build on a class defining an instance method `new` returning a `Product`: factory = Factory.new; factory.new.run before this PR: Product#run (correct) after this PR: Factory#run (wrong) Track whether resolution currently sits on the class constant or on a value of that class, and apply the selector rule only to the former. The head of a chain is a class constant only when it resolved straight to a class binding rather than through a typeBinding; every hop past it yields a value, so the flag clears. The `obj.method()` branch derives the same fact from whether `objExpr` is a bare name resolving to that class. `Factory.new.run` keeps the behaviour this PR introduced (Factory#run), which is itself a fix over the base build's Product#run. KNOWN LIMITATION, now documented on the contract field and asserted by a test so a future change to it is deliberate: a class-level override (`def self.new` returning another type) is still read as construction. The scope model records no staticness per member, so `def new` and `def self.new` are indistinguishable at this layer; separating them needs the language provider to record staticness first. An earlier attempt to use `TypeRef.source` as a proxy was abandoned after tracing showed Ruby records body-inferred return types as `return-annotation` too, so it does not discriminate. Tests: `ruby-construction-selector` fixture pins all three shapes — class constant, instance receiver, and the documented class-level-override limitation. Ruby resolver suites: 185 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * fix(resolution): resolve generic construction receivers (#2708) `new Box<string>().unwrap()` reached the class lookup as `Box<string>`, which names no class binding, so the member edge was still dropped while the non-generic spelling resolved. `new Foo<T>()` is ordinary in all three keyword-wired languages, so the fix covered a materially narrower slice of real code than intended. Retry the lookup on the base name via `stripTemplateArguments` — the same normalization `resolveClassBindingForName` already applies to typed receivers in the sibling `receiver-bound-calls` pass. The exact-name lookup still runs first, so a class whose name legitimately contains `<` is unaffected. Measured on the probe that first showed the gap: before: | viaGeneric | Class:src/box.ts:Box | (construction edge only) after: | viaGeneric | Method:src/box.ts:Box.get#0 | (member edge resolved) Tests: `viaGenericCtor` added to the typescript-inline-constructor-receiver fixture, asserting both the target file and that the resolved id is `Box`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * fix(resolution): resolve construction in the chain-head position (#2708) `new Service(db).inner.deep()` emitted only the construction edge. The chain walker seeds its starting class from the head segment, which arrives as `new Service(db)` and reduces via `stripCallParens` to `new Service` — no binding and no class of that name, so the walk was never seeded and every segment after it resolved to nothing. Seed the head through the same construction rule the call branch already uses. A constructed value is an instance, so the class-constant flag from the previous commit correctly stays false — `new Factory().new` does not get the selector treatment. The gap was asymmetric across the languages this PR wires: Python's bare form strips to a plain `Service` and was already seeded, so only the keyword languages were affected. Tests: `viaChainHead` added to the typescript-inline-constructor-receiver fixture. Note the fixture annotates `readonly inner: Inner` explicitly — with an unannotated initializer the walk stops at the field, which is field-type inference and a separate concern from head seeding. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * fix(resolution): match the construction keyword by token, not by one space (#2708) The keyword form was matched with `startsWith(`${keyword} `)`, so only a single space separated `new` from the type. Any other trivia the source used — a tab, a line break — failed the match and the member-call edge was lost. Match the keyword as a whole token followed by one or more whitespace characters instead. `newService()` still fails the match, which is the point: it is an ordinary call, not a construction, and must keep resolving through its own return type. The keyword is escaped before it enters the pattern. It comes from a language provider rather than from user input, but a keyword containing a regex metacharacter would otherwise build a silently wrong pattern. Tests: tab-separated and newline-separated `new` added to the typescript-inline-constructor-receiver fixture. Note these cases only survive because `gitnexus/test/fixtures/` is listed in the repo-root `.prettierignore` — running prettier from inside `gitnexus/` does not pick that file up and normalizes the tab away. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * fix(resolution): resolve qualified construction callees (#2708) `new ns.Service().doWork()` emitted only the construction edge. The call branch splits the callee at its last `.` before construction is considered, so a qualified type name was routed into `obj.method()` resolution as if `ns` were a receiver and `Service` a member. A keyword-marked expression is never a member call, so resolve it as construction before the split. The callee lookup now also handles a dotted name: an unambiguous `qualifiedNames` match first, then the trailing simple name, mirroring how receiver resolution elsewhere in this pass degrades. Measured: before: | viaQualified | Class:src/svc.ts:Service | (construction only) after: | viaQualified | Method:src/svc.ts:Service.doWork#0 | Bare-form qualified construction (Python `models.User(db).save()`) is NOT addressed here: that shape currently emits no edges at all, including no construction edge, so it is a namespace-import resolution gap upstream of this pass rather than a construction-typing one. Tests: `viaQualifiedCtor` added to the typescript-inline-constructor-receiver fixture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * fix(java): drop the unreachable constructionSyntax declaration (#2708) Java was wired `{ keyword: 'new' }`, and the PR described it as one of the languages that needed the fix. Measuring both ways shows it never did: Java resolves `new Svc().doWork()` identically with and without the change, because `java/captures.ts` (#2564) already rewrites an `object_creation_expression` receiver to the constructed type's simple name, so the raw `new Svc()` text never reaches this resolver. The decisive evidence is generics: Java resolves `new Box<User>().doWork()`, which the keyword path could not do before the template-argument fix earlier in this series — the resolution demonstrably comes from the capture rewrite, not from here. Removing the declaration rather than leaving it as defensive configuration: an unreachable per-language opt-in reads as coverage that does not exist, and the contract now records why Java is excluded so the omission is not mistaken for an oversight. Verified after removal: the Java probe still resolves both the inline and two-step spellings, and the Java resolver suites pass (252 passed, 1 skipped). An earlier coordinator measurement in this review claimed Java WAS broken on base; that comparison was invalid (the "without fix" build had not been rebuilt). Corrected here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * refactor(resolution): state the selector rule once and derive its option type (#2708) Two follow-ups from review, no behaviour change (643 resolver tests pass unchanged before and after): The `Class.new` selector rule was written out twice — in the `obj.method()` branch and again in the chain walker — against differently named locals, while the construction helper's own doc comment claimed the rule was stated in exactly one place. Both sites ask the identical question, so they now call one `isConstructionSelectorHop` predicate, and the doc comment says what is actually true. `ResolveCompoundReceiverOptions.constructionSyntax` re-declared the contract's object shape by hand. It was the file's first object-shaped duplicate, and because the value arrives as a non-literal variable, TypeScript's excess property check would not fire: a sub-field added to the contract later would type-check and then be silently ignored here. It is now derived with `ScopeResolver['constructionSyntax']`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * test(resolution): cover the C# construction path and pin the wiring inventory (#2708) Three coverage gaps from review, no behaviour change. C# had no fixture despite being the only keyword-wired language whose behaviour genuinely depends on the construction rule — measured absent on base and present on head. `csharp-inline-constructor-receiver` covers the inline spelling, the two-step spelling, and a static factory that must keep resolving through its return type rather than being read as construction. The TypeScript two-step assertion checked only `toContain('Service')`, and the same fixture defines `LegacyService` — `'LegacyService'.includes('Service')` is true, so the assertion could not distinguish the two targets. It now pins `targetFilePath` the way its sibling assertions already do. Nothing guarded the deliberate opt-in set, so an accidental wiring of a language that already resolves the shape, or a silent loss of one that needs it, would pass the whole suite. `construction-syntax-wiring.test.ts` pins the inventory in both directions: exactly which languages declare `constructionSyntax` and with which spelling, and that java/php/swift/dart/ kotlin stay unwired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * chore(storage): bump INCREMENTAL_SCHEMA_VERSION to 23 for the #2708 edge changes This series changes which CALLS edges are emitted for source whose CONTENT has not changed — inline constructor receivers that previously emitted nothing now resolve, and the Ruby selector fix moves one edge back to the member it always belonged to. That is precisely the class of change the version-history block in this file requires a bump for, and the reuse gate is a strict equality on the persisted stamp. Without it, every existing v22 index passes the gate on the next `analyze` — or is served by the same-commit "already up to date" fast path — and keeps returning the pre-fix graph for unchanged files. `impact(direction: "upstream")` and `context()` would go on omitting the very callers #2708 is about, with no warning, until something unrelated forced a full re-analyze. The fix would have shipped without reaching anyone who already had an index. Precedent is unbroken across the recent resolution PRs: #2723 → v22, #2699 → v21, #2695 → v20, #2563 → v14, each with its own rationale paragraph. This adds v23 in the same form. The pinned assertion in call-summary-schema-version.test.ts moves with it, as that test documents it is designed to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * chore(bench): re-baseline the fixture-corpus fingerprints for #2708 Both bench harnesses fingerprint an entire fixture corpus by directory prefix (`bench/python-scope/measure.mjs:38`, `bench/scope-capture/measure.mjs:76`), so every fixture directory this series adds moves a committed baseline. Neither script writes the baseline itself — running without `--check` only prints, and the file is edited deliberately, which is what its own comment asks for. Regenerated, last in the series so the fixture set was final: bench/python-scope/baseline-fingerprint.txt 36e29abc… -> f120df92… bench/scope-capture/baselines.json ruby 070e4e11… -> fea3edf8… typescript 281e9548… -> cad25be9… csharp e05dc274… -> 05a85bae… CI only ever reported the python drift, because the benchmarks job runs the python step first and aborts there; the cross-language step never ran. Both were verified locally after the update: [measure --check] PASS (capture fingerprint + scaling) [import-target-fingerprint --check] PASS (resolver fingerprint) [scope-capture --check] PASS (15 languages) The `csharp` and `ruby` entries moved because of the fixtures added earlier in this series, not the original ones — a reminder that this baseline moves with any fixture addition, not just the one that first triggered it. Captures goldens regenerated alongside (csharp, ruby); both additive only, no existing digest changed. The python golden did not move: no `python-*` fixture was added after its last regeneration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqnsJK3Cnbu3bjdZbzgMMP * Update tests for passesReuseGate function --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
df06529950
|
fix(rust): resolve module-qualified calls against the module tree (#2730) (#2741)
* fix(rust): resolve module-qualified calls against the module tree (#2730) A Rust call written with a path (`tools::dispatch(..)`) was captured with only its tail identifier, making it indistinguishable from a bare `dispatch(..)`. The scope-chain walk then resolved the bare name lexically and bound it to whatever `dispatch` was nearest — which, for the common wrapper idiom fn dispatch(..) -> ToolOutcome { tools::dispatch(..) } is the wrapper itself. The graph gained a self-loop, the real cross-module edge never existed, and `impact` reported the callee as unreached: the issue's repository showed its central tool dispatcher as `risk: LOW` with 0 affected processes and both "callers" being `#[cfg(test)]` functions, while still labelling the result `epistemic: "exact"`. Resolve paths the way rustc does, over the module tree rather than the filesystem: - `mod_item` now emits `@declaration.namespace`, so a Rust module is a named definition rather than an anonymous scope region. This mirrors the existing C++ `namespace_definition` capture and lets the shared `tagNamespacePrefixes` pass stamp members with their enclosing module path — that pass needed no changes to start working for Rust. - `module-path.ts` reconstructs the other half of the tree: crate roots are directories holding `main.rs`/`lib.rs`, and a file's module path is its location below that root. A definition's module is its file's module plus any enclosing `mod` blocks. - `crate::`, `self::` and `super::` are prefix transforms on the calling module, not reasons to stop resolving. - The final path segment is looked up as a member of the resolved module, including members it only re-exports. A `pub use` creates no binding on the re-exporting module's own scope, so re-exports are followed through that module's import edges. Resolution runs ahead of the implicit-`this` and scope-chain tiers, so an explicit path outranks a lexical shadow, and returns undefined on an unknown module, a missing member or a tie — leaving the existing chain untouched. The new `ScopeResolver.resolveQualifiedFreeCall` hook is optional and unset for every other language, so this is additive. Fixes the reported case (direct callers 2 -> 3, impacted 2 -> 6, the Agent module now visible) plus multi-segment paths, `super::` paths and `pub use` facades, each of which previously produced a wrong edge. Known limitation, pre-existing and unchanged by this commit: an inline `mod inner { fn dispatch }` and a crate-root `fn dispatch` in the same file collapse to one graph node, because node identity is `<file>:<qualifiedName>` and does not carry the module path. That is a separate defect requiring module-path-qualified node ids and an incremental-schema migration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * test(rust): rebaseline the scope-capture fingerprint for the module-tree captures `mod_item` now emits `@declaration.namespace` and scoped call sites carry `@reference.qualified-name`. Both are additive, so every bench fixture holding a `mod` block or a `Foo::bar()` call gains capture groups, and the corpus grew by the three `rust-2730-*` fixtures. Only the Rust fingerprint moves. The other 14 languages are byte-identical, which is the intended blast radius for a language-local capture change. Scaling stays linear at 1.043, well inside the 1.5 budget. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): carry crate identity in qualified module paths (#2741 review H1) A module was identified by its path segments below a crate root, so `crates/alpha/src/tools.rs` and `crates/beta/src/tools.rs` were the same module. A cargo workspace routinely gives several members the same internal module name — `util`, `error`, `config`, `types` are near-universal — and that made qualified resolution do one of two wrong things: - where only one member defined the called name, the call bound ACROSS crates; - where both defined it, the lookup saw two candidates, refused, and handed the site back to the lexical walk that emits the same-name self-loop. The fix for #2730 therefore switched itself off in exactly the workspace layouts it was written for, and #2730's own reported reproduction repository is multi-crate. A module is now `{ crateRoot, segments }` and `sameModule` compares both. Rust has no implicit cross-crate paths — reaching another crate requires naming it — so two modules in different crates are never the same module. Anchored paths (`crate::`, `self::`, `super::`) resolve inside the caller's own crate and inherit its root. Covered by a two-member workspace fixture where both crates define `tools::dispatch` behind a same-name wrapper, plus unit tests for the path arithmetic itself, including the branches no fixture reaches (a file under no crate root, a `super::` chain walking above the crate root). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): count only module members when resolving a qualified call (#2741 review H3) Module membership was inferred from the file path alone, so any callable in the right file counted as a member of the module. A `fn` nested inside another `fn` has the same `filePath`, the same bare `qualifiedName` and no owner, making it indistinguishable from a module-level item: pub fn dispatch() -> usize { 3 } // the real member pub fn wrapper() -> usize { fn dispatch() -> usize { 99 } // counted as a second member dispatch() } Two candidates tie, the lookup refuses, and the call falls back to the lexical walk that emits the same-name self-loop — so an unrelated local helper anywhere in a module silently reinstated #2730 for every qualified call into it. The scope model already draws the line exactly: a module-level item is bound with `origin: 'local'` in its module's own scope, a function-local item binds in the enclosing Block, and an `impl`/trait method binds in the Class scope. Membership is now that binding lookup rather than a path comparison. Inline-`mod` members bind in their Namespace scope rather than the file's Module scope, and reaching it would mean walking every child scope — faulting them back in from disk on the out-of-core path. They keep being identified by the `namespacePrefix` the shared tagging pass stamps on them, which a file-module member never carries. The documented residual is a `fn` nested inside a `fn` inside an inline `mod`, which inherits that prefix; that is strictly smaller than before and costs a refusal, never a wrong edge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): require a use-binding to name a module, not a type (#2741 review H2) Import resolution deliberately strips a trailing symbol segment when probing for a file — "the last segment might be a symbol (function, struct, etc.), not a module. Strip it and try again" (import-resolvers/rust.ts). So `use crate::client::ClientBuilder;` also resolves to `client/mod.rs`. The qualified-call resolver took that at face value and treated the imported TYPE as the module `client`. Rust impl methods carry a bare `qualifiedName`, so `ClientBuilder::new()` was then looked up among `client`'s module members and bound to an unrelated module-level `new` — turning an unresolved site into a false edge, which the module's own contract calls the worse outcome. A binding now has to name the module it resolved to. The edge's `targetExportedName` is the tail of the written path, so comparing it against the resolved module's own tail separates the cases exactly: use crate::tools; tail `tools` module ['tools'] accept use crate:🅰️:b as tools; tail `b` module ['a','b'] accept use crate::tools::{self, Ctx}; tail `tools` module ['tools'] accept use crate::client::ClientBuilder; tail `ClientBuilder` module ['client'] reject Covered by a fixture where `client/mod.rs` deliberately holds both `impl ClientBuilder { fn new }` and a module-level `fn new`, so a regression re-binds to the wrong one, plus a control asserting a genuine `client::new()` module qualifier still resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): give src/bin targets their own crate root (#2741 review) Cargo auto-discovers a binary target for every `src/bin/<name>.rs`. Each is a separate crate with its own `crate::` root, and its submodules live under `src/bin/<name>/`. Only `main.rs` and `lib.rs` established a crate root, so those entry files were folded into the surrounding library and given the invented module path `bin::<name>`. That made `crate::helper()` inside a binary resolve into the LIBRARY's `helper` — and unlike the other findings in this review, this one downgraded an edge the lexical walk had previously resolved correctly, so it made existing output worse rather than merely failing to improve it. `src/bin/<name>.rs` is now its own crate root (as is the `src/bin/<name>/main.rs` directory form), so a binary's modules and the library's modules of the same name are no longer the same module. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): only try a submodule candidate the caller actually declares (#2741 review) The first candidate module was `callerModule ++ qualifier`, yielded before the `use` channel and never checked against anything. That let file layout outrank a real import: with `use crate::b;` in `src/a/mod.rs` and an undeclared — or `cfg`-gated — `src/a/b.rs` present on disk, `b::f()` bound to the sibling file, where rustc resolves it to `crate::b`. A `mod` declaration, inline or file-backed, emits a `Namespace` def bound locally in the declaring scope, so the candidate is now gated on that binding rather than assumed. When the caller does not declare the submodule the candidate is skipped and the `use` and crate-root channels still run, so this only removes guesses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): follow only real re-exports, and refuse on an ambiguous one (#2741 review) Two problems in the re-export channel. A private `use` was followed as though it re-exported. `use crate::tools::helper;` makes `helper` visible INSIDE the module; it does not put it on the module's public surface, so `facade::helper()` does not compile. Only `pub use` does, and finalize already distinguishes them — `reexport` for `pub use`, `named` for a private one. The `alias` kind is now accepted alongside `reexport`, because `pub use x::y as name` is a re-export that was previously ignored entirely. The lookup also took the first matching edge in file-iteration order, which is parse-pool order. Two `cfg`-exclusive facades re-exporting the same name are indistinguishable at this layer, so picking one baked a coin flip into the graph. It now refuses on a genuine tie, consistent with how member lookup already behaves. The pre-existing limitation that only FILE modules are reachable — a `pub use` inside an inline `mod facade { … }` has no `moduleScopeByFile` entry — is now stated in the code. Reaching those would mean walking every child scope and faulting the scope tree back in from disk, which is the cost that index exists to avoid; a miss falls through to the unchanged chain rather than guessing. The regression test deliberately makes the re-exported name globally ambiguous. Without that, the pre-existing unique-global free-call fallback resolves the call on its own and the assertion passes whatever this channel does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * perf(rust): stop type-qualified calls paying for module resolution (#2741 review) The capture carrying `rawQualifiedName` matches every `scoped_identifier` callee, so this hook was reached by `Vec::new()`, `String::from()`, `Self::method()` and every other type-qualified call — the overwhelming majority of `::` calls in real Rust, none of which name a module. Each one ran the full candidate search before returning undefined, and every candidate that missed then walked all of `workspaceIndex.moduleScopeByFile`. Total cost grew as `qualified-call-sites x files`; two independent measurements put per-site cost at 0.117 -> 0.428 ms across 301 -> 1201 files, i.e. linear in workspace size. Two changes: - The module index now carries a flat set of every module segment name in the workspace, and a qualifier whose head matches none of them is rejected before any candidate work. Measured at 0.02 us per rejected call and flat in file count (500 -> 8000 files), against a previously linear per-site cost. - Module scopes are indexed by module identity once per pass rather than rediscovered by scanning every file per candidate. On the out-of-core scope index that scan was worse than CPU: `moduleScopeByFile` fetches through `scopeTree.getScope`, so a full sweep could fault every module scope back in from disk — the pattern `workspace-index.ts` added `exportedCallableByName` to avoid. Given the #2649 and #1871 history this mattered before merge. The captures golden is regenerated for the fixture files added earlier in this series; `emitRustScopeCaptures` itself is unchanged by this commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(storage): bump schema versions so the #2730 fix reaches existing indexes Neither invalidation constant was bumped, so the fix did not reach the users who reported the bug. `INCREMENTAL_SCHEMA_VERSION` 22 -> 23. The incremental write set only covers CHANGED files, so a top-up against a pre-v23 index keeps the wrong self-loop — and keeps reporting the callee as unreached — for every unchanged Rust file. The constant's own doc block states this rule, and the precedent is exact: v11 is the same file (`rust/query.ts`) gaining a capture that changes CALLS edges, with the same "force a full re-analyze" contract, and v12 is a second Rust instance. `SCHEMA_BUMP` 30 -> 31. `@declaration.namespace` and `@reference.qualified-name` are parse-time captures, so a warm parse cache replays the old capture set verbatim: `rawQualifiedName` comes back undefined and no Namespace def exists to hang a module prefix on, turning the entire resolution tier into a no-op on unchanged files. `PARSE_CACHE_VERSION` folds in the package version, so a tagged release would have invalidated eventually — but source, dev and CI builds at the same version would not, and the v29 note already warns that relying on someone else's bump is how a change ships with no invalidation at all. Re-checked against origin/main at commit time, as that note instructs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(scope-resolution): let a language opt out of the already-namespaced guard (#2741 review) `tagNamespacePrefixes` skips a def whose `qualifiedName` already equals, or is prefixed by, its enclosing namespace path. That is right for C++ and C#, where the qualified name genuinely carries the namespace. Rust qualified names never do, so the guard fired on a coincidence: in `mod a { pub fn a() }` the member's name equals its module's name, the prefix was skipped, and `moduleOfDef` then reported the member as belonging to the PARENT module. `crate:🅰️:a()` refused, and the def became indistinguishable from a crate-root `fn a` for the module matcher. The guard is now conditional on a `qualifiedNamesCarryNamespace` option that defaults to the existing behaviour, and Rust opts out. The shared pass stays language-neutral — the decision lives with the provider that knows what its own qualified names contain. C++ and C# resolver suites pass unchanged alongside the Rust ones (600 tests). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): refuse a leading :: path instead of reading it as relative (#2741 review) A leading `::` anchors at the extern prelude: `::tools::dispatch()` names the CRATE `tools`, not a module of the current one. The path split filtered the empty leading segment away, which silently reinterpreted the path as relative and let it resolve against a local module that happens to share the name. Extern crates are outside the workspace module tree, so the qualified tier now refuses and leaves the site to the unchanged chain. The regression test asserts the tier does not bind into the local `tools` module, rather than asserting no edge at all: the lexical tier still resolves the bare tail on its own, and that behaviour is not what this change governs. Asserting an empty edge list would have been testing a different tier. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * refactor(rust): reuse the canonical callable predicate and drop dead re-exports (#2741 review) `CALLABLE_TYPES` was a local copy of the set behind `isOverloadableCallable` in `utils/callable-labels.ts`. Two copies of the same set drift: extending the canonical one with a new callable kind would silently leave qualified calls of that kind unresolved here, with nothing to catch it. Use the shared predicate. The trailing `export { moduleOfFile, moduleOfDef }` and `export type { ScopeResolutionIndexes }` were commented as being "for the resolver's unit tests". No test imports them: the only importer of this module anywhere in src or test is `rust/scope-resolver.ts`, which takes just `resolveRustQualifiedFreeCall`. Both functions are already exported from `module-path.ts` (where the new unit tests take them from), and `ScopeResolutionIndexes` is canonically exported from `model/scope-resolution-indexes.ts`. Removed rather than left as surface that implies a contract it does not have. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * test(rust): rebaseline the scope-capture fingerprint with the correct prior hash The rebaseline note added with the original fix cited `Prior 655aed01…`, which was two rebaselines stale — it predates both #2604 and #2714. The true pre-PR value on the base commit is `7f1240b3…`. CI could not catch it: the gate compares the live fingerprint against the stored one and never reads the prose, so the audit chain these notes exist to provide was broken with nothing to flag it. The note now carries the correct prior value, and the fingerprint is regenerated for the fixtures this review series added. Scaling 1.061, well inside the 1.5 budget; fixture_count 196; the other 14 languages remain byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * test: move the schema-version pin to 23 `call-summary-schema-version.test.ts` asserts the exact value of `INCREMENTAL_SCHEMA_VERSION` and enumerates which stamped versions the incremental reuse gate accepts. It moves with every bump by design — that pin is what stops an id- or edge-changing commit shipping without invalidation. Updated for the bump to 23, with the pre-v23 case added to the reuse-gate table: a v22 index predates Rust module-qualified call resolution, so every unchanged Rust file would keep the same-name self-loop and keep reporting the real callee as unreached. Caught by CI rather than locally, because the earlier sweeps in this series covered `test/integration/resolvers/` and `test/unit/scope-resolution/` only — the pin lives outside both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
azizur100389
|
79ff44dfa9
|
fix(config): honor parts negation on Windows (#2720)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Skill copy sync / shipped skills drift guard (push) Has been cancelled
Normalize repository-relative paths before applying ignore-package rules so `.gitnexusignore` negation can override hardcoded `parts` exclusions during Windows traversal. Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
89ea233e10
|
fix(js): index CommonJS exports.foo = function () {} exports (#2723) (#2729)
* fix(js): index CommonJS `exports.foo = function () {}` exports (#2723)
Functions assigned to an `exports` / `module.exports` property were not
indexed at all. On a CommonJS codebase — the dominant pre-ESM Node style
(Express, Firebase Functions) — the graph held every internal helper and
missed the entire public API: `impact({target: 'areVariablesValid'})`
answered `Target not found` for the one symbol whose blast radius mattered.
The gap had two halves, and fixing either alone leaves the feature broken:
1. `tree-sitter-queries.ts` carried `@definition.function` rules for every
declaration form and every variable-binding closure form, but none for
`assignment_expression` — so no `Function` node was created.
2. The scope-resolution queries (`languages/{javascript,typescript}/query.ts`)
likewise had no `@declaration.function` for the shape. Adding only (1)
moves `impact` from "not found" to "found, zero callers", because call
resolution reaches a definition through the scope declaration, not
through the graph node.
Both layers now carry the rule, for `function` / `async function` / arrow /
async arrow / generator right-hand sides, in JavaScript and TypeScript. The
receiver is pinned to `exports` / `module.exports` with `#eq?` predicates:
the general `X.foo = function () {}` shape also covers `Foo.prototype.bar`
and `this.handler`, which are member constructs with their own ownership
questions, and a broader rule would emit ownerless top-level Functions for
them. The declaration binds the bare property name into the module scope,
which is what importers see, so `const { foo } = require('./m')` matches by
name and a namespace `m.foo()` walks the module's defs.
Verified end to end: node emission for every listed form plus TS parity, and
CALLS edges for same-file `exports.foo()`, cross-file namespace `m.foo()`,
and cross-file destructured `require()`. The generator call-resolution case
was confirmed to fail against the pre-fix build before the rule landed.
Fixes #2723
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(js): stop the CJS export rule from shadowing a declared function (#2723)
Review of the previous commit caught a regression it introduced. The CJS
`@declaration.function` rules bind the exported property name into the module
scope — which is the point, since that is what importers resolve against. But
when the file ALSO declares that name lexically:
function dup(v) { return v; }
exports.dup = function (v) { return !v; };
function callIt(v) { return dup(v); }
the module scope ends up holding two declarations named `dup`, the name is
ambiguous, and the resolver drops `callIt -> dup` entirely — an edge that
resolved fine before #2723. Confirmed by rebuilding both states: present at
|
||
|
Gergő Magyar
|
0ce7880290
|
fix(scope-resolution): a closure binding is a call SOURCE in every language, and function-local values carry their own identity (closes #2699) (#2718)
* test(scope-resolution): audit the consumers of file-scoped node ids (#2699 part A)
#2699 item 4 — "audit consumers that assume file-scoped ids" — after #2695/#2714
gave function-local CALLABLES position-bearing ids. Tests and findings only; no
production change. That split is deliberate: `impact` reports
`resolveDefGraphId` at CRITICAL with 23 DIRECT dependents across 7 modules
(every language MRO builder, both Spring attachers, C++ member lookup,
tryEmitEdge, emitReferencesViaLookup, buildGraphTargetIndex, emitFreeCallFallback,
emitReceiverBoundCalls, preEmitInheritanceEdges, emitDetectedInterfaceImplementations,
phpEmitUnresolvedReceiverEdges, emitRubyMixinEdges, emitRustTraitImplEdges,
emitDartHeritageEdges), so changing that key chain is its own change, not a
rider on an audit.
A2 — detect_changes: CONCERN RESOLVED, now pinned. The worry was that an id
containing `@row:col` re-keys whenever a declaration MOVES, making every edit
look like symbol churn. It cannot: `local-backend.ts` maps diff hunks to
symbols by LINE-RANGE OVERLAP (`n.startLine`/`n.endLine`) and merely REPORTS
`n.id`. Node identity never participates in the match. New structural test
asserts the WHERE clause never gains `n.id =` or `n.id IN`, keeps the one
legitimate id-shaped predicate (the `BasicBlock:` prefix exclusion, #2082 U7),
and confirms the id is returned rather than matched. Structural in the same
idiom as `detect-changes-worktree.test.ts`, and labelled as not proving runtime
behaviour.
A1 — ANSWERED, and the answer is that #2699 is NOT fully closed by items 1-3.
The fail-closed guard is gated on `isOverloadableCallable`
(Function | Method | Constructor), so a function-local VALUE never reaches it.
Measured on a fixture: a top-level `const handler` and a function-local
`const handler` still produce ONE node, `Const:v.ts:handler`. That is the
residual half of the issue's original complaint. Pinned as a KNOWN LIMIT with
its reason (widening identity to values re-keys ~14,700 build-time nodes to
change ~800 persisted ones — the decision recorded in `parse-worker.ts`), and
deliberately NOT fixed here.
A3 — id-persisting consumers, classified:
- detect_changes ................ SAFE (position-keyed; pinned by A2)
- MCP impact/context/trace ...... SAFE (resolve by name/uid at query time)
- bench fingerprints ............ SAFE (digest capture shape, not node ids)
- rust-captures golden .......... SAFE (digests captures, not ids)
- cfg pipeline-pdg snapshot ..... AT RISK by design — pins exact edge ids, so
it trips whenever attribution changes. That is the gate working; #2714
already exercised it.
- wiki / group-contract links ... NOT id-keyed on locals (locals are never
cross-file addressable, per the document-scoped contract of item 2).
Verified: tsc clean; 14/14 across the two touched files; `detect_changes`
reports 0 changed symbols (tests only).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(php): a closure binding is a call SOURCE, not only a TARGET (#2699 part B, S1)
A call made inside a closure binding was attributed to the ENCLOSING scope, so
the closure was a call TARGET but never a call SOURCE: impact(handler,
direction:"downstream") reported nothing even though the closure calls out.
Root cause, probe-measured rather than inferred. Instrumenting
pickCallerCallableDef (graph-bridge/ids.ts) to log every rejection reason shows
the closure's own scope EXISTS and its range DOES contain the call site, but its
ownedDefs is EMPTY, so the ":94" owned-callable filter drops it and attribution
falls through to the ":97" enclosing-scope fallback.
The reason is one missing query rule. javascript/query.ts pairs the binding name
with the closure via @declaration.function anchored on the INNER arrow node, so
anchor.range equals the @scope.function range and pass2AttachDeclarations
attaches the declaration to the CLOSURE's scope. No other language had that
rule — PHP, Rust, Kotlin, Ruby and Dart all captured named function
declarations only. That single omission is the entire empty-ownedDefs cause.
This ports the rule to PHP with the same anchor discipline (@declaration.function
on the inner anonymous_function / arrow_function, NOT on the
assignment_expression wrapper). PHP needs nothing else: it already declares
(anonymous_function) and (arrow_function) as @scope.function, so the rule alone
completes it.
Measured on a fixture: `$handler = function ($x) { return target($x); }` inside
outer() now emits
Function:src/a.php:outer.$handler@3:2 -> Function:src/a.php:target
where it previously emitted `outer -> target`.
The pinned test in closure-binding-labels.test.ts asserted the OLD, wrong
behaviour by design ("to catch that asymmetry changing in EITHER direction"), so
it is INVERTED here rather than deleted, per its own instruction. Its block
comment is corrected to record the measured root cause, including that Kotlin
and Ruby will need BOTH this rule AND a relaxed kind gate (their lambda_literal
/ do_block is @scope.block deliberately, #1757), and that Dart has no closure
scope at all.
Verification: closure-binding-labels 50/50; PHP resolver suites 221/221
(php, php-coverage, php-response-shapes). detect_changes {staged}: 1 changed
symbol (PHP_SCOPE_QUERY), 0 affected processes, risk LOW.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(rust): emit a node for a closure binding and make it a call SOURCE (#2699 part B, S3)
Rust was the one exception to #2687's "a closure bound to a name is a Function
node in every language": `let handler = || target(1);` produced NO graph node at
all, so the closure could be neither a call target nor a call source.
Needed BOTH query channels, which is the finding worth recording. Porting only
the scope-resolution rule (as S1 did for PHP) changed nothing measurable here,
because there was no node to attribute anything to:
- languages/rust/query.ts — closure-binding declaration, @declaration.function
on the INNER closure_expression so anchor.range aligns with the existing
(closure_expression) @scope.function. This is what gives the closure's own
scope a callable in ownedDefs, which is what stops pickCallerCallableDef
falling through to the enclosing fn.
- tree-sitter-queries.ts — @definition.function on the OUTER let_declaration.
This emits the Function NODE that Rust never had.
Note the deliberate anchor asymmetry between the two channels: the graph-node
channel anchors the WRAPPER (matching the existing
(lexical_declaration (variable_declarator ... (arrow_function))) rule), while
the scope-resolution channel anchors the INNER closure (to align with
@scope.function). Getting these backwards silently produces either no node or
an unattributable one, so both sites carry a comment saying so.
Measured on a fixture — `let handler = || target(1);` inside outer():
Function:src/a.rs:outer CALLS Function:src/a.rs:outer.handler@2:4
Function:src/a.rs:outer.handler@2:4 CALLS Function:src/a.rs:target
Previously the whole binding was absent and the call read as `outer -> target`.
The rule also covers `move` closures: the closure_expression node spans the
`move` keyword.
Verification: closure-binding-labels 50/50; rust.test.ts 192/192;
rust-coverage, rust-f70, rust-scope all pass; rust-captures-golden passes
UNCHANGED, so no golden regeneration was required. detect_changes {staged}:
2 changed symbols (RUST_SCOPE_QUERY, RUST_QUERIES), 0 affected processes,
risk LOW.
One caveat on the suite runs: this host times out `beforeAll` hooks at the
default 60s under load — rust.test.ts needed --hookTimeout=600000 to complete,
and a concurrent second vitest run starves worker startup entirely (every test
fails at ~5001ms). Both are host artifacts, not signal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(kotlin,ruby): a closure binding is a call SOURCE, via a Block-scope callable boundary (#2699 part B, S2)
Kotlin and Ruby anchor a closure on a Block-kind scope — Kotlin lambda_literal
and Ruby do_block/block are @scope.block DELIBERATELY (#1757 smart casts), so
they must not be re-kinded. pickCallerCallableDef gated its child-scope walk on
kind === 'Function', so a closure there could never become a call SOURCE.
Both halves are required; neither alone changes anything:
1. kotlin/query.ts and ruby/query.ts gain the closure-binding declaration rule,
with @declaration.function on the INNER lambda_literal / block so its range
aligns with the @scope.block range (the anchor discipline documented in
javascript/query.ts). Without this the closure scope owns no callable def.
2. pickCallerCallableDef accepts a Block-kind child as a callable boundary when
the scope IS that callable's body. Without this the kind gate still rejects.
The alignment test in (2) is the part worth scrutiny. Relaxing the kind gate to
accept ANY Block owning a callable would be a real regression: a nested
`fun foo()` declared inside a block is owned by that block, so a call made at
BLOCK level — outside foo — would be misattributed to foo. Comparing the def's
declaration position against the scope's start position discriminates them: for
a closure the declaration and the scope sit on the SAME node, so the positions
match; for a nested function the block starts at `{` while the def starts at the
declaration, so they do not. Existing Function-kind behaviour is untouched, so
every already-working language is unaffected by construction.
The comparison is base-safe: scope-extractor.ts builds a def id as
`def:<filePath>#<startLine>:<startCol>:<type>:<name>` from the same Range a
scope carries, so both sides share one coordinate base. This is called out in
the helper's docblock because `defStartLine` nearby documents its own output as
1-based, which invites a wrong "fix" (#2377 is exactly this class of hazard).
Ruby's call forms are restricted to lambda/proc by name: an unrestricted
(call block: (block)) would match ANY method call taking a block, so
`mapped = items.map { |i| ... }` would wrongly declare `mapped` a callable.
Verified against the parser: 3 matches (->, lambda, proc), map excluded.
Separate #eq? patterns rather than one #match? alternation, which is a known
hazard on this tree-sitter line.
Measured on fixtures:
Kotlin Function:src/A.kt:outer.handler@2:4 CALLS Function:src/A.kt:target
Ruby Function:src/a.rb:outer.handler@4:2 CALLS Method:src/a.rb:target#1
previously `outer -> target` and `outer#0 -> target#1`.
The pinned Kotlin test asserted the old behaviour by design and is INVERTED, not
deleted. Ruby had NO pinned case, so a new one is added rather than inverted.
The describe title no longer claimed something false ("not yet a call SOURCE"
now holds only for Dart) and was retitled.
Verification: closure-binding-labels 51/51; kotlin.test.ts, kotlin-coverage,
ruby.test.ts, ruby-scope, ruby-namespaced all pass (478 passed / 1 expected
inversion before the test was flipped). impact on pickCallerCallableDef:
CRITICAL, 191 impacted, ONE d=1 (resolveCallerGraphId) — the return contract is
unchanged, so that dependent is unaffected. detect_changes {staged}: 5 changed
symbols, 2 affected processes (both EmitReferencesViaLookup, one of them the new
ScopeIsCallableBody step), risk medium.
Dart remains the last failing language: dart/query.ts declares no
@scope.function at all, and dart/captures.ts synthesizes one only from a
declaration WITH a body node, which an expression-bodied closure lacks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(dart): give a closure binding a scope and a distinct identity (#2699 part B, S4)
Dart was the last language where a closure binding could not be a call SOURCE,
and fixing only that would have made the graph WORSE, not better. This lands
both halves together for that reason.
## The attribution half
A Dart closure had no scope at all. `dart/query.ts` declares no
@scope.function anywhere — Dart's function scopes are SYNTHESIZED in
`dart/captures.ts` from `declNode` + `findFunctionBody(declNode)`, and
`findFunctionBody` looked only at the next named SIBLING for a `function_body`.
A closure literal carries its body as a CHILD (`function_expression_body`), so
it matched nothing and no scope was produced.
`query.ts` gains the closure-binding declaration rule and `findFunctionBody`
understands the child form. Deliberately NO @scope.function is added to the
query: it would collide at identical range with the synthesized one, and
duplicate scope ids make `buildScopeTree` throw, which DROPS THE WHOLE FILE.
## The identity half, and why it is not optional
With attribution alone, two same-named closures in one file both keyed to the
bare `Function:a.dart:handler`. One node then appeared to call BOTH targets —
a CALLS edge present nowhere in the source. That is worse than the missing edge
it replaced, so S4 could not ship without this.
Root cause is not Dart-specific. `enclosingCallablePrefix` derives a SEMANTIC
relation — what encloses this callable — by SYNTACTIC ancestor walk. Dart parses
`int outer() { … }` as `function_signature` followed by `function_body` as
SIBLINGS, so the enclosing callable is never an ancestor of code inside it and
no membership set can fix that; the walk looks in the wrong direction.
This is what SCIP and real compilers avoid by construction. SCIP keeps a local
symbol opaque (`local <id>` — no name, no position, no chain) and models
containment as a SEPARATE `enclosing_symbol` field; its spec says the local/global
choice should follow ACCESSIBILITY, not the ability to name an enclosure. Dart's
own analyzer answers this from `Element.enclosingElement` in the element model,
never from AST ancestry. clang uses `name@offset` for a function-local; Kythe
uses a document-scoped VName plus a `childof` edge. Identity is positional and
opaque; enclosure is a relation.
`findSplitBodyCallableAncestor` is the narrow fix at that seam: a fallback used
ONLY when the ancestor walk finds nothing, recovering the callable from the
body's preceding sibling.
The sibling must be a BARE SIGNATURE, and that restriction is load-bearing —
"any preceding callable sibling" is WRONG and was caught regressing PHP during
this work. In `<?php function target($x) {…} $handler = function ($x) {…};` the
closure is at FILE level, so the ancestor walk correctly finds nothing, the
fallback runs, and an unrestricted version mis-qualified the file-level
`$handler` as `target.$handler`. A preceding sibling is only an ENCLOSING
callable when it cannot hold its own body.
`SPLIT_SIGNATURE_NODE_TYPES` is exactly that set and is DERIVED, not listed:
`LOCAL_SCOPE_BODY_NODE_TYPES` is already `FUNCTION_NODE_TYPES` minus the bare
signature types, so the difference between them IS the split-signature set
(`function_signature`, `method_signature` — verified at runtime). PHP's
`function_definition` carries a body and is in both, so it is excluded. No
language is named in shared code, and any future split-grammar language is
covered for free.
## Verification
Full resolver sweep — the gate that caught #2714's Rust regression — 2926
passed / 1 skipped / 0 failed across 51 files. closure-binding-labels 52/52;
dart.test.ts, dart-coverage, callable-id-lockstep, function-local-identity,
caller-identity-regression all pass (156/156 across 6 files).
impact on `enclosingCallablePrefix`: LOW, 5 impacted, 3 d=1 all inside
parse-worker. detect_changes {staged}: 5 changed symbols, 0 affected processes,
risk LOW.
Three existing Dart expectations FLIPPED rather than being deleted: Dart locals
now carry the same enclosing-callable + position identity every other language
got in #2695, so `local.dart:handler` became `local.dart:caller.handler@1:2`.
A new test pins the actual defect — two same-named closures staying DISTINCT
nodes — because the qualification assertions alone would not fail if the
fabricated edge returned.
One note for future work: an id-shape assertion here carries a call-site suffix
on indirect invocations (`…handler@3:2:5:9`) but not on direct calls. That is
the callable-value-flow pass keying its edge by invocation position, not part of
the node id.
Part B is now complete: PHP (S1), Rust (S3), Kotlin + Ruby (S2), Dart (S4).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(scope-resolution): close every deferred item on #2699 (A1 values, twin-list guard, schema bumps)
Clears the limitations this PR had been carrying rather than leaving them as
follow-ups.
## A1 — function-local VALUES now carry their own identity
This was #2699's ORIGINAL complaint and the one a callable-only gate could never
reach: a top-level `const handler` and a function-local `const handler`
collapsed onto ONE `Const:v.ts:handler`. #2695 restricted position-qualified
identity to Function|Method|Constructor because the collision that produced
wrong CALLS edges was between callables, and widening churned ids for symbols
the pruner mostly deletes. The churn is real and is accepted here deliberately.
Widening needed THREE gates aligned, not one:
- id-building — `parse-worker.ts` nestedCallablePrefix
- resolution — `ids.ts` position key
- registration — `node-lookup.ts` position-key registration
Missing the third would register no position key for values, so every lookup
misses and falls through silently. That is the #2714 failure mode: the caller
attaches to a node that does not exist and the edge is DROPPED, which looks like
"zero dangling edges" from outside. All three now route through ONE predicate,
`isPositionQualifiedLocalLabel`, rather than repeating the label set a third
time.
Only LOCALS move. The prefix comes from `enclosingCallablePrefix`, which returns
undefined when nothing encloses the declaration, so top-level and class-member
ids are untouched — verified by the full resolver sweep, where a leak onto class
members would have broken assertions in every language. `Property` is included
on purpose: a class field stays unqualified because the prefix walk boundaries
on class-likes, while an object-literal property inside a function is genuinely
local and would otherwise keep the old collision.
Measured: `Const:v.ts:handler` + `Const:v.ts:run.handler@3:2`, two distinct
nodes. The KNOWN LIMIT test is FLIPPED per its own former instruction ("this
test should be updated as part of it rather than deleted").
## Schema bumps — required by Part B, not just by A1
INCREMENTAL_SCHEMA_VERSION 20 -> 21, parse-cache SCHEMA_BUMP 27 -> 29.
SCHEMA_BUMP is 29, not 28, and that is the point of re-checking it against
origin/main at MERGE time rather than branch time. This branch cut at 27 and
bumped to 28; #2415 also bumped 27 -> 28 and merged first. The automated
main-merge onto this branch surfaced the collision — leaving it at 28 would have
shipped this whole change with NO parse-cache invalidation, so every warm cache
keeps replaying the pre-fix captures and ids. This is the third instance of that
collision recorded in parse-cache.ts (#2632/#2653 hit it at v21, and
#2653/#2654 hit INCREMENTAL_SCHEMA_VERSION the same way).
Part B already changed emitted node ids AND edges on files that did not
themselves change (Dart locals re-keyed, Rust gained a node it never emitted,
five languages gained closure-source attribution). A v20 index topped up
incrementally keeps serving the old attribution, and a warm parse cache replays
the old captures and ids verbatim. Shipping S1-S4 without these would have let
every existing index silently keep the pre-fix graph.
## Twin-list drift guard — the sixth instance in this family
`IMPLICIT_RECEIVERS` (gitnexus-shared lookup-core.ts) and `THIS_RECEIVERS`
(type-env.ts) spell the same concept in two packages, and nothing enforced
agreement — `$this` was added to the shared list in #2714 only because it was
already in the other. New structural test asserts set equality plus the ONE
deliberate asymmetry (`Me`, Visual Basic spelling, absent from the shared list
because no SupportedLanguages entry uses it) in BOTH directions, so re-adding it
there or dropping it here each fail loudly.
Structural rather than value-imported: both constants are module-private, and
exporting them purely to be testable would widen two public surfaces to satisfy
a test.
## Two false comments corrected
- `lookup-core.ts` said "see the drift guard noted in #2714", implying a guard
existed when it was only a deferred follow-up. It exists now, and the
comment points at it.
- `callable-id-lockstep.test.ts` claimed its regex "fails if any site
reconstructs the id". It matches ONE template spelling; a hand-rolled
concatenation still slips past. Now stated as a tripwire for the known
shape, not a proof.
## Skill learnings
Four entries appended to eval/workflow_bench/learnings.jsonl from this run: the
v9fs safe-writer failure, backticks silently terminating a query template
literal (hit three times), a module-level TDZ const that passes tsc and then
presents as N file failures with ZERO failing assertions, and concurrent vitest
runs starving worker startup so a whole suite fails at ~5001ms.
## Verification
Full resolver sweep 2926 passed / 1 skipped / 0 failed (51 files) — identical to
pre-A1, which is the evidence that only locals moved. All EIGHT bench gates PASS
with fingerprints UNCHANGED, so no regeneration was needed. function-local-identity,
callable-id-lockstep, receiver-twin-list-drift and closure-binding-labels 71/71.
tsc --noEmit clean.
detect_changes {staged}: 9 changed symbols, 14 affected processes, risk HIGH —
expected, and the reason the sweep above is the gate rather than a targeted list.
Every affected process routes through `resolveDefGraphId`, the key chain Part A
measured at CRITICAL with 23 direct dependents.
Deliberately NOT done: the SCIP end state (opaque `local <id>` plus an explicit
enclosure EDGE instead of containment encoded in the id string). It is a design
direction, not a limitation of this work, and it is INCOMPATIBLE with A1 — A1
widens chain-encoded identity, that removes chain encoding entirely. Bundling
both would re-key every local twice. Written up in the research notes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* test: update two assertions the #2699 changes correctly invalidated
Both failed on CI at
|
||
|
MyShining
|
ff86ccf1e7
|
feat(spring): model profiles, conditions, and auto-configuration (#2678)
* feat(spring): model conditions and auto-configuration * fix(spring): align auto-configuration declarations * perf(spring): streamline auto-configuration indexing * test(spring): move timing benchmark out of vitest --------- Co-authored-by: Shining <xuenning@qiyi.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
e307286d52
|
fix(scope-resolution): a named receiver's member never resolves lexically, + two #2695 follow-ups (#2714)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(scope-resolution): a named receiver's member never resolves lexically (#2699) `lookupCore` Step 1 walked the lexical scope chain for every lookup, including explicit-receiver property reads. So `options.baseUrl` could bind to an unrelated function-local `const baseUrl` in the same file, and `config.extractVisibility(node)` to the enclosing class's own method. This is the residual half of the defect JS/TS block scopes narrowed in #2695. Blocks moved nested-block locals off the chain of a reference outside the block, which removed 114 false edges; a local declared directly in the function body stayed on it, and no amount of extra scopes reaches that case. Fixed at the cause instead: `recv.name` names a member of whatever `recv` denotes, so a binding of the bare tail name in an enclosing scope is never the right answer. Steps 2 and 3 (receiver type / owner members) are the legitimate routes. `this` and `self` are EXEMPT, and that exemption was measured, not assumed. Skipping Step 1 for every explicit receiver removed 711 edges on a 762-file corpus — but 2 of those were genuine: `self.srcIx` and `self.streamedAt(...)` after `const self = this`, reaching their own class's members through the class-body scope. For a self-receiver the members and the lexical chain legitimately overlap; for a named receiver they never do. Exempting the self names keeps both true edges and still removes 709 false ones, adding none. The removals were classified by reading source at the site, not by pattern- matching ids — an "is the target a member of the source's owner?" heuristic labelled 43 of them plausible and every one I then read was false: language = config.language; -> the class's own `language` dirMap.get(...) / exactMap.get(...) -> a sibling object-literal `get` return config.extractVisibility(n); -> the class's own method (self-edge) writer.close(); -> GraphEmitSink.close Residual, deliberately kept: a `this.x` read can still bind lexically to a same-named local. That is the price of the two true self-alias edges above. `INCREMENTAL_SCHEMA_VERSION` 19 -> 20: a v19 index holds these false CALLS/ACCESSES on every unchanged file and would keep serving them through the reuse gate. Test confirmed discriminating: it fails with the guard reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR * fix(typescript,javascript): a generator expression binding is a Function node (#2693) `const g = function* () {}` matched none of the closure-binding definition rules — they covered `arrow_function` and `function_expression` only — so the binding emitted a `Const` node. `buildGraphTargetIndex` admits callable nodes only, so `g()` resolved to nothing. Same defect shape as the `var` case #2693 already fixed: a different grammar node for the same construct, and the resulting graph node was not callable. Adds the four variable-binding shapes in both languages: `const`/`let` and `var`, each plain and exported. Purely additive — no existing pattern is reordered or rewritten, because the #2687 pre-scan dedup is order-dependent and collapsing the value/callable pair depends on which match wins. Deliberately NOT covered, and the query comment says so: a generator in an object-literal pair or a HOC wrapper still falls through anonymous. Those are rarer, and each additional pattern is another chance to disturb the dedup. `SCHEMA_BUMP` 26 -> 27: definition captures are parse-time, so a warm parse cache would replay the old ones verbatim — `--force` does not clear it. Two tests confirmed discriminating (they fail with the patterns reverted), plus a guard that the already-working generator DECLARATION form is unaffected, since it shares the emit path these were inserted beside. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR * fix(ingestion): keep caller attribution in lockstep with definition ids (#2699) The definition phase appends `localIdentity` to a nested callable's own name segment (`run.save@3:2`); `findEnclosingFunctionId` did not, so the two phases derived different ids for the same callable. The failure mode is silent — the caller id names a node that does not exist, so the edge is dropped rather than reported — which is why the parse-worker docblock calls this pair a lockstep guarantee and asks that both phases derive the prefix from one place. The condition is now byte-identical to the definition phase's (`nestedPrefix !== undefined`), so the two cannot diverge again. Scope of the claim, stated plainly: no reproducing case was found, and this changes nothing measurable on a 762-file TypeScript corpus. TS/JS resolve callers through `resolveCallerGraphId` in the graph bridge, not this path; `findEnclosingFunctionId` serves the `callExtractor` languages, and the corpus does not exercise a nested callable there. The review that raised it (P3) observed zero dangling edges, and "zero dangling" is also what silently dropped edges look like — so this closes a documented contract rather than a demonstrated bug, and carries no test of its own. Rides the `SCHEMA_BUMP` 26 -> 27 in the preceding commit: caller attribution runs in the worker, so a warm parse cache would replay the old ids. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR * docs(test): correct the block-scope header that this PR made false (#2699) Review finding (MEDIUM). The file header still described `lookupCore` Step 1 as walking the lexical chain for EVERY lookup, and called the function-body-local case "unchanged and still mis-resolves ... pre-existing and tracked separately". Commit |
||
|
Gergő Magyar
|
4906daf27b
|
fix(scope-resolution): resolve calls through a closure-valued binding across languages (#2693) (#2695)
* fix(scope-resolution): resolve calls through a closure-valued binding (#2693)
`val f = { }; f()` emitted no CALLS edge in Kotlin or Swift, so `impact` on
such a symbol under-reported to zero — the same false all-clear as #2687.
The cause was not, as first suspected, that these languages fail to feed
`callable-value-flow`. They do: `synthesizeCallableFlowCaptures` is called
from 15 language capture modules, and Kotlin already resolves reassignment
through the pass (`var f = ::a; if (c) f = ::b; f(1)` reaches both targets).
Their captures are already exactly right — the seed names the binding as its
own callable, per the anonymous-callable convention in
callable-flow-captures.ts.
They died one layer later, at the `buildGraphTargetIndex` gate:
if (!isCallable(def) && providerTarget?.(def) !== true) continue;
`isCallable` is Function/Method/Constructor, but the scope-resolution layer
declares a closure binding with its VALUE label (Kotlin/Swift `Property`),
and `isCallableValueTarget` is implemented by exactly one provider — COBOL.
So the binding never entered `graphTargets`; `lexicalCallableLookup` then
returned `shadowed: true` with no targets, which also suppressed the
workspace-wide fallback, and the seed resolved to nothing.
Only the graph knows a value binding holds a callable — since #2687 it emits
a single `Function` node for one. So value bindings now resolve their graph
id first and are admitted on the label of the node they actually reach.
This is self-limiting: a genuine constant keeps its own Const/Property node,
so `resolveDefGraphId`'s qualified key hits before the label-agnostic
`simpleKey` fallback can reach a same-named callable. Only a binding whose
own value node was replaced by a callable one gets through.
No scope kind changes — Kotlin's `lambda_literal` stays `@scope.block`, so
#1757 smart-cast semantics are untouched by construction. The fix is
language-neutral: it discriminates on the graph node label, never on a
language name.
Dart is fixed separately; its root cause is independent.
* fix(dart): resolve calls through a closure-valued binding (#2693)
Dart needed more than the shared gate fix: neither of its closure-binding
forms could resolve, for two different reasons, and the plan's one-line
diagnosis turned out to be incomplete.
TOP-LEVEL `var f = (x) => x;`
A graph Function node already existed (#2687), but no `@declaration.*`
matched the binding, so scope resolution had no SymbolDefinition to attach
a flow seed to. Adding the declaration exposed a second problem: Dart's
`initialized_identifier` is FIELDLESS, so the shared field-based assignment
fallback (`left`/`name`/`value`/…) decomposed nothing and the binding still
emitted no flow captures at all. Kotlin's fieldless `assignment` node hit
exactly this and took the same remedy — a provider `extractAssignment`.
FUNCTION-LOCAL `void m() { var f = (x) => x; }`
Locals parse as `initialized_variable_definition`, which the top-level
graph-node rules are deliberately anchored under (program) to avoid, so a
local closure had no graph node at all — nothing for the widened
`buildGraphTargetIndex` gate to admit.
Both new rules are restricted to a `function_expression` value. Declaring
every Dart variable would mint defs and nodes repo-wide for no resolution
benefit; ordinary locals stay unindexed exactly as before. The top-level
declaration reuses the (program) anchor the graph-node query already relies
on, so class-body fields — which share `initialized_identifier_list` and are
already `@declaration.property` — are never matched twice.
Also drops the now-false note in tree-sitter-queries.ts claiming `f()` does
not resolve for Dart. That node is now the evidence that makes it resolve.
* docs(scope-resolution): document the callable-flow capture contract (#2693)
The module is 1200+ lines behind a nine-line docblock, and the only worked
example was C. Both root causes fixed in this series were "the contract was
discoverable only by reading the emitter":
- the anonymous-callable convention (a seed whose source is a closure takes
its DESTINATION's name) is what makes closure bindings resolvable at all,
and is the reason the widened target gate is correct;
- a fieldless binding node silently decomposes to nothing under the shared
assignment fallback, which cost Kotlin one debugging cycle in #2522 and
Dart another here;
- captures alone are never enough — the bound name also needs a
`@declaration.*` or there is no cell to key the seed on.
Records the cell/site model, both traps, and points at the fullest and
smallest worked examples.
Bumps INCREMENTAL_SCHEMA_VERSION 15 → 16 and the parse-cache SCHEMA_BUMP
22 → 23: this series emits NEW CALLS edges and new Dart Function nodes, and
the incremental write set only covers changed files, so an existing index
would keep reporting a zero blast radius for exactly the symbols the fix is
about.
* perf(scope-resolution): pre-filter value bindings in the callable target index (#2693)
Widening the `buildGraphTargetIndex` gate to consider VALUE bindings put the
hot loop on a much larger def population — value bindings outnumber callables
in real source — and the naive version paid full price per binding. Measured
on a synthetic 800-file corpus (8 value bindings per file, 1 of them a closure
binding), the widening cost 2.50-2.82x the pre-#2693 callable-only build.
Two wastes, both provable rather than guessed:
1. `definitionAnchorKey` ran for every def, including value bindings. The
anchor index is keyed by callable LABEL and the key is built from
`def.type`, so a value def can never hit it — and the key costs a regex
per def.
2. Every value binding paid the whole `resolveDefGraphId` key chain only to be
rejected. It need not: every qualified key that function tries embeds
`def.type`, so for a VALUE def those can only ever reach a value-labelled
node. Its one route to a callable is the label-agnostic
`simpleKey(filePath, simpleName)` fallback, which by construction requires
a callable node with the SAME file and simple name. So a value binding with
no such node cannot resolve to a callable, and one Set lookup decides it.
That set is derived in the graph walk the anchor index already performs, so it
costs no extra pass.
large_ms 7.79-8.37 -> 4.90-5.02 (1.61x faster)
widening_overhead 2.50-2.82 -> 1.45-1.50
The resolved target-set fingerprint is byte-identical across both, which is
the point: this is a cost change, not a behaviour change.
Adds bench/callable-value-flow/ (fingerprint + scaling + widening-overhead
gates) and wires it into ci-tests.yml beside the other build-free benches. The
overhead budget of 1.9 sits between the measured with-filter and without-filter
bands, so it cannot be met if the pre-filter is removed. Timings use the MIN of
15 warmed reps, not the median: the same build reported 1.65 idle and 2.03
under load, and a median-based gate would have to be loosened past the point of
detecting the regression it exists to catch.
`buildGraphTargetIndex` is exported for the bench; it is pure and not part of
the pass's public contract.
* test(scope-resolution): assert the declaration route does not double-emit (#2693)
Go, Python, C++ and TS/JS already resolved a closure-binding call through
their `@declaration.function` capture. The widened `buildGraphTargetIndex`
gate gives the same call a SECOND possible route, so each must still produce
exactly one edge.
`tryEmitEdge` dedups by key, but a collapsed key and a site-anchored key are
DIFFERENT keys — a real double-emit would show up as two ids for one call
site, not be silently collapsed. Asserting on edge ids rather than target ids
is what makes that visible.
* fix(scope-resolution): join value bindings to their callable node by POSITION (#2693)
Review found the first cut of this series minted FALSE CALLS edges. Admitting a
value binding whose *resolved* graph node is callable let `resolveDefGraphId`
fall through to its label-agnostic, first-write-wins
`simpleKey(filePath, simpleName)` and bind the name to ANY same-named callable
in the file.
The safety argument in the previous commit — "a genuine constant keeps its own
Const/Property node, so the qualified key hits first" — silently assumed
`def.type === node.label`. It does not hold:
- TypeScript declares `const` as `Variable` but emits a `Const` NODE, so the
qualified key misses even though the value node exists;
- Rust `let` bindings get no graph node at all, so the fallback is the only
route.
Reproduced, all previously emitting a fabricated caller:
const save = (x: number) => x * 2; // next to an unrelated Svc.save
-> Method:svc.ts:Svc.save#1 // Svc never instantiated
const handler = other; // shadowing a top-level handler
-> Function:app.ts:handler // unreachable from here
let handler = cb; // Rust
-> Function:main.rs:handler
Worse in Dart, where the same collision INVERTED the feature: the only edge went
to the class method and the closure's own node got none. The result was also
declaration-order dependent — two files differing only in declaration order got
different CALLS sets — and it propagated through argument-to-formal binding into
functions whose source never mentions the name.
A closure binding IS its callable node: same file, same line, same name. An
aliasing local is not. So the join is positional now — a file/line/name index
built in the graph walk `byAnchor` already performs — and value bindings never
run the key chain at all. That is both correct and cheaper:
large_ms 4.90-5.02 -> 4.37-4.63
widening_overhead 1.45-1.50 -> 1.43-1.58 (name-match design: 2.50-2.82)
with a byte-identical target-set fingerprint on the bench corpus.
Also from review:
- `Static` dropped from VALUE_BINDING_DEF_TYPES: `normalizeNodeLabel` has no
`static` case, so no def can carry that type — it was an entry no fixture
could ever exercise. The remaining set now documents why it deliberately
does NOT reuse `isOwnableValueLabel`, which is contracted to a different
consumer.
- Dart `final`/`const` top-level closures (static_final_declaration_list) and
every declarator after the first in a multi-name local now resolve; both
parse into shapes the earlier rules never reached.
- The bench source carried a literal NUL byte, so git recorded it as BINARY
and the only artifact pinning the target set was unreviewable in the PR
diff. It is written as an escape now. Its corpus also modelled `startLine`
as 1-based where graph nodes are 0-based, which would have stopped it
exercising the value-binding path at all.
- `call-summary-schema-version.test.ts` asserted `passesReuseGate(15)` is
true; the 15 to 16 bump made that false and the test RED. It now pins 16 as
current and 15 as rejected, matching the pattern every prior bump followed.
- The v23 parse-cache comment is at the top of the list, not mid-list.
Tests: the five collision cases above are new regression tests, each confirmed
failing against the previous commit. Also added Kotlin class-body closures (the
only case exercising the Method arm), Dart top-level `final`, Dart multi-name
locals, and a warm-parse-cache replay for Kotlin and Dart — the #2693 captures
are replayed verbatim, so a serialization change would surface only on a SECOND
analyze and every other test here runs cold. The previous negative tests were
vacuous: they paired names that did not collide (`maxSize` vs `size`), so the
pre-filter rejected them before the guard they were named after could run.
* docs(storage): fix the schema-version changelog blocks (#2693)
Two problems, one mine and one not.
MINE: the `INCREMENTAL_SCHEMA_VERSION` block is ASCENDING (v2 … v15), and I
inserted v16 above v15 rather than at the end — I had just moved the parse-cache
entry to the top of ITS block, which is descending, and applied the same habit
to a list ordered the other way. Moved to the end; both blocks are now
internally consistent.
NOT MINE: the parse-cache block carries TWO v21 entries, with v20 wedged between
them. Tracing it: #2632 (Spring DI facts) bumped 20 -> 21 and merged first;
#2653 (Java JLS local-class identities) had branched at 20, also bumped to 21,
and merged second — so it shipped with NO invalidation of its own. An index
already stamped 21 by the first change was treated as current by the second and
kept serving stale local-class identities from the warm cache.
Numbers left alone: both genuinely shipped as 21, and renumbering them now would
misstate what users' indexes actually contain. Instead the entry says so
explicitly, and points at the process fix — re-check the constant against
origin/main immediately before merging, not just when the branch is cut. The
identical collision hit INCREMENTAL_SCHEMA_VERSION in #2653/#2654, so this is a
recurring failure mode of concurrent PRs, not a one-off typo.
Comment-only; no constant changes value.
* feat(scope-resolution): resolve closure bindings in Ruby, Java, C#, PHP and JS/TS var (#2693)
Ruby, Java, C# and PHP already emitted correct callable-flow seeds and invokes.
What they lacked was the #2687 piece — a CALLABLE graph node at the binding,
which is what buildGraphTargetIndex joins to by position. PHP additionally had
no scope declaration for the bound name, so the flow pass had nothing to attach
its seed to.
ruby handler = ->(x) { x } handler.call(1) -> Function:a.rb:handler
java Function<..> handler = x->x handler.apply(1) -> Function:A.java:A.handler
csharp Func<int,int> handler = ... handler(1) -> Function:A.cs:A.handler
php $handler = fn($x) => $x $handler(1) -> Function:a.php:handler
Ruby and Java invoke through the callable-object protocol; C# and PHP call the
binding directly. Locals work in all four, and a binding whose name collides
with a same-named method resolves to the CLOSURE, not the method.
Two things the sweep caught:
JAVA TWIN. Anchoring the rule on the inner variable_declarator produced BOTH a
Function and a Property node — the exact double-indexing #2687 removed. The
parse-worker dedup keys on (definition node, name), and Java's value rule
anchors on field_declaration, so the keys never matched. Re-anchored on
field_declaration / local_variable_declaration.
JS/TS `var`. `var f = (x) => x` kept a Variable label while const/let got
Function, because `var` is a different grammar node (variable_declaration vs
lexical_declaration) that no closure rule covered. A call through the binding
still resolved via the declaration route, so the CALLS edge pointed at a
NON-callable node. Now consistent across const/let/var.
That last one flipped an existing assertion in const-function-twin.test.ts,
which expected `Variable` for a var-bound function-expression. Its comment
explained why — "var has no matching @definition.function pattern, so nothing
claims the name" — i.e. it documented the gap rather than defending it. The
property it was really protecting (an UNCLAIMED value node survives) now has
its own case with a non-function initializer, and the var-closure case asserts
the collapse to one node, which is also the twin guard for the new rule.
Known limits, both pre-existing and both failing safe:
- A PHP local closure whose name collides with a top-level function gets no
edge: both want id Function:<file>:<name>, so the closure never gets its own
node. This is the file-scoped node-identity convention — TypeScript, Python
and Dart collapse identically at base.
- TS/JS class-field arrows stay Property (Kotlin's equivalent emits Method).
They already resolve; changing the label risks the HAS_PROPERTY ownership
regression #2687 hit once.
The invalidation constants already bumped in this PR (INCREMENTAL_SCHEMA_VERSION
16, SCHEMA_BUMP 23) cover these additional languages; their notes now say so.
Tests: one case per newly-resolving language plus the PHP anonymous-function
form and the JS var form, in closure-binding-labels.test.ts. The file now spins
a worker pool per test across a dozen languages, so its timeout is raised
file-wide — a case that takes ~7s alone was exceeding the 30s default under
that contention.
* fix(ingestion): class-field closures are callable members in TS/JS (#2693)
A CALLS edge must target a callable node. `class A { handler = (x) => x }` emitted
a Property, so calling it produced `CALLS -> Property:A.ts:A.handler` — an edge
pointing at something the graph says is not callable. Same defect class as the
JS/TS `var` binding fixed in the previous commit, and the last place a closure
binding still carried a value label.
Kotlin already models its class-body closure as Method + HAS_METHOD; TS/JS now
match, so all three agree:
class-field closure -> Method + HAS_METHOD (CALLS target is callable)
plain class field -> Property + HAS_PROPERTY (unchanged, no CALLS)
Anchored on public_field_definition / field_definition — the same nodes the
property rules use — so the parse-worker dedup collapses the pair rather than
leaving a Method/Property twin, the failure the Java rule hit in the previous
commit.
ON MATCHING THE COMPILERS. This deliberately diverges from tsc and SCIP. The
TypeScript compiler classes `handler = () => {}` as a PropertyDeclaration
("a property declaration independently from what it's assigned to"), and SCIP
gives it a `.` term descriptor, the same suffix as any field — both call it a
property, and Kotlin's compiler likewise treats `val f = { }` as a property with
a function type. The divergence is intentional: GitNexus's Function/Method label
does not mean "tsc SymbolFlags", it means "this node can be the target of a
CALLS edge", which is the convention #2687 set for closure bindings in every
language. Modelling it the compiler's way would mean either dropping call
resolution for these members or emitting a separate node for the lambda and
flowing the property to it — the two-node shape #2687 removed. Recorded here so
the next reader does not "fix" it back.
Tests: TS and JS class-field arrows resolve to their Method node, plus a guard
that a NON-closure class field stays a Property — the closure rule must key on
the initializer, not on the field syntax.
* fix(php): keep the $ sigil on closure-binding nodes so locals stop colliding (#2693)
A PHP local closure whose name matched a file-level function got NO edge at all:
function save($x) { return $x; }
function run() {
$save = fn($x) => $x * 2;
return $save(1); // no CALLS edge
}
Both minted the id Function:<file>:save, so the closure's node was swallowed by
the function's and the positional join found nothing at the binding's line.
The fix is PHP's own semantics rather than a change to node identity across the
graph. PHP holds variables and functions in SEPARATE namespaces — $save and
save() cannot collide in the language — and the sigil is what separates them.
Dropping it was the bug. The node rule now captures the whole variable_name, so
the closure is Function:<file>:$save and the function stays Function:<file>:save.
languages/php/query.ts already keeps the sigil on property declarations for the
same reason, so this makes the two consistent.
The positional join normalises a leading $/@ on both sides, matching what the
scope layer and the callable-flow synthesizer already do, so the binding still
matches its own declaration while its NODE stays distinct.
local closure + same-named function -> Function:c.php:$save (the closure)
calling the real function -> Function:f.php:save (unchanged)
plain $max = 10 -> no node, no edge (unchanged)
WHAT THIS DOES NOT FIX. The general problem is wider than PHP: GitNexus node ids
are file-scoped, so a function-local symbol and a file-level one with the same
name collapse in TypeScript, Python and Dart too, and Java/C# only escape by
qualifying on the enclosing CLASS (so two same-named locals in different methods
still collide). SCIP solves it with a separate `local <id>` keyspace that is
document-scoped and never globally addressable. That is issue #2699 — it changes
persisted ids for every function-local symbol and needs its own invalidation, so
it is not bundled here. PHP is fixed on its own merits: the sigil belongs in the
identity regardless of how locals are eventually scoped.
* test(scope-resolution): pin the closure-binding caller-attribution limit (#2693)
Review of this PR found the new callable nodes are call TARGETS but never call
SOURCES: a call made INSIDE a closure binding is attributed to the enclosing
scope, so `impact(handler, direction:"downstream")` reports nothing even though
the closure calls out. Consistent across Kotlin, Dart, Ruby and PHP; TS/JS free
bindings are the exception because their arrow carries a @scope.function whose
range matches.
Not fixed here — pinned, so the boundary is visible instead of surprising, and
so a change in EITHER direction fails a test.
The cause is precise: `pickCallerCallableDef` (graph-bridge/ids.ts) finds the
caller by walking CHILD scopes whose range contains the call site, gated on
`child.kind === 'Function'`. A closure literal is a BLOCK scope in these
languages (Kotlin deliberately, #1757 smart casts), AND the binding's def is
owned by the enclosing scope rather than by the closure's scope — so neither
half of the link exists. Fixing it needs "callable boundary" decoupled from
scope `kind` plus an association between the closure scope and its binding.
That is a change to the caller anchor used by every call in the repo, which is
not something to land at the tail of this PR.
Also adds a unit suite for `buildGraphTargetIndex` itself, covering what the
integration tier cannot isolate: a binding is admitted only on POSITIONAL
evidence, a name-only match is rejected, a non-callable node at that position is
rejected, an ambiguous position claimed by two callables is rejected, and the
PHP dollar sigil normalises across the join while still not matching a
same-named function on another line. That last one closes the review's LOW —
the node/declaration name asymmetry now has an executable contract rather than
resting on a comment.
* docs(test): correct the per-language cause of the attribution limit (#2693)
The comment on the pinned attribution tests claimed "a closure literal is a
BLOCK scope in these languages". That is true for Kotlin (lambda_literal
@scope.block, #1757) and Ruby (do_block/block @scope.block) and FALSE for PHP:
anonymous_function and arrow_function are already @scope.function
(php/query.ts:61-62). Dart is a third case again — it has no scope over a
closure literal at all.
So the four languages fail at three different points, not one:
Kotlin, Ruby fail the `child.kind === 'Function'` gate
PHP passes that gate; its closure scope owns no callable def,
because the binding's def belongs to the enclosing scope
Dart has no child scope for the walk to consider
Worth correcting carefully rather than tidying: a follow-up plan re-stated this
comment instead of re-deriving it, and inherited the misdiagnosis — it proposed
"relax the kind gate" as required for all four, which is a no-op for PHP and
unreachable for Dart. A review caught it. The comment now states each language's
actual blocker and says why the distinction matters.
Comment-only; the three pinned tests are unchanged and still pass.
* fix(scope-resolution): an ordinary JS/TS `function` binds its own `this` (#2701)
`this.m()` inside a nested `function` resolved to the lexically enclosing
class, so it emitted a CALLS edge that does not exist at runtime — including
the exact `forEach(function () { this.m(); })` shape arrow functions were
introduced to avoid:
class D {
m() {}
build() { const h = function () { this.m(); }; return h; }
}
// CALLS: Function:D.ts:D.h -> Method:D.ts:D.m#0 FALSE
ECMA-262 gives an arrow `[[ThisMode]] = lexical`: it has no `this` binding in
its environment record, so the lookup passes through to the enclosing
environment. Every other function form binds `this` at call time. `tsc` draws
the same line by resolving `this` through `getThisContainer` with
`includeArrowFunctions = false`. That one rule is the whole fix.
Languages declare it; shared code never learns a language. The query files —
the one place that already names grammar nodes — tag every non-arrow function
form with `@receiver-owner.this`, which becomes `Scope.ownsReceivers`. A
receiver walk that reaches such a scope without finding the name stops there
instead of borrowing an enclosing scope's binding. Every other language leaves
the field unset and is bit-for-bit unchanged; a Kotlin lambda, which DOES
capture the enclosing `this`, still resolves (pinned as a test).
THREE GATES, ALL LOAD-BEARING. The false edge survived each one alone, which
is why the tests assert on the emitted edge rather than any single walk:
1. `Scope.ownsReceivers` stops BOTH receiver-type walks — `findReceiver
TypeBinding` here and its twin `lookupReceiverType` in gitnexus-shared's
`lookup-core`, which was resolving the receiver independently.
2. `LanguageTypeConfig.thisBoundaryNodeTypes` stops the type-env AST walk
that infers a receiver's type during capture.
3. `isReceiverOwnedButUnbound` makes `receiver-bound-calls` SUPPRESS the
site. Without it the member still resolved by NAME through `lookupCore`'s
lexical chain — the class-body scope binds `m` two scopes up — merely at
lower confidence. An owned-but-unbound receiver is a definitive negative,
not a miss, so it must not reach a receiver-blind fallback.
Also fixed: `function*(){}` as an expression was not a `@scope.function` at
all, so `this` inside one read as the enclosing method's.
WHAT THIS GIVES UP. The fix REMOVES edges, and some were correct:
`.bind(this)`, `.call(this)` and `forEach(fn, thisArg)` do make `this` the
instance at runtime. Their correctness is fixed at the CALL SITE, which no
scope-level rule can see, so the choice is between losing them and keeping
every detached-callback false positive. All three are pinned as tests
asserting the empty result, so changing the trade later is deliberate.
`this` in a static method also stops resolving to the INSTANCE member — that
edge was wrong in the other direction.
INVALIDATION. Both constants move, and the parse-cache one is not optional:
`ownsReceivers` lives on the cached `Scope`, and a warm cache replays scopes
without it — verified by probe that `--force` alone does NOT re-derive it, so
the fix silently did nothing until SCHEMA_BUMP moved. INCREMENTAL_SCHEMA_
VERSION 16 -> 17 (the incremental write set covers only changed files, so
unchanged TS/JS files would keep their fabricated `this` edges);
SCHEMA_BUMP 23 -> 24.
Verified against a built index, not by reading: all three false edges from the
issue gone, every correct edge kept, same result in JavaScript through its
separate grammar. 64 tests green across the new suite plus the closure-binding
and schema-version suites. The full suite's 36 failures are pre-existing
load-flakes — confirmed by A/B: `skip-git-cli` fails FOUR tests on a clean
HEAD versus three with this change, and `pipeline-pdg-streaming` passes in
isolation either way.
Refs #2701
* fix(ingestion): give function-local callables their own identity (#2699)
Graph node ids were file-scoped, so a local callable and a same-named
file-level one collapsed onto ONE node. That is a wrong answer, not a missing
one — the local call was attributed to the file-level symbol:
export function save(x) { return x; }
export function run() { const save = x => x * 2; return save(1); }
export function other() { const save = x => x * 3; return save(2); }
// ONE node Function:a.ts:save, and BOTH run and other pointed at it, so
// `impact` on the top-level save reported two callers that never call it.
A local's identity is now its enclosing-callable chain plus its own position —
`run.save@2:2`. The chain is for humans reading `impact`; the position is what
makes it correct. Names alone cannot express what ECMAScript actually
specifies, and the gap is the language's, not the grammar's: an environment
record is created per function AND per block, so an anonymous function has no
name to contribute and sibling blocks hold distinct bindings under the same
name. One positional rule settles both, with no conditionals and no
"disambiguate only when it looks ambiguous" heuristic — the ambiguity-flag
class of bug that bit #2514. SCIP reaches the same place with its
document-scoped `local <id>` keyspace.
Top-level functions and class methods are NOT locals and keep their ids
byte-for-byte. That is the bound on the churn: this touches only symbols that
are unreachable from outside their own document anyway.
RESOLUTION JOINS BY POSITION, NOT BY NAME. `resolveDefGraphId` matches a def
to its node on (file, label, line, simple name). A def and its node are the
same construct, so this needs no scope chain at all — which is the point:
re-deriving the chain in the resolver would be a second implementation that
could silently disagree with the first. A genuine tie (two callables on one
line) stores an AMBIGUOUS_POSITION tombstone and falls through to the existing
name keys rather than picking by source order. Without this the node ids were
already correct and calls STILL resolved to the file-level symbol — the fix is
only half a fix without it.
JS/TS GAIN BLOCK SCOPES. They emitted no `@scope.block` at all, so the
resolver could not tell two `const pick` in sibling branches apart. Giving
them distinct ids made that visible as DUPLICATE edges — each call resolving
to BOTH — which is worse than the collapse it replaced. `(statement_block)
@scope.block` supplies the missing environment record. The other half of the
ECMAScript rule was already implemented and waiting: `tsBindingScopeFor`
hoists `var` past blocks to the enclosing Function/Module while `let`/`const`
bind innermost, and its docblock already claimed "the innermost default covers
these" for block scopes that did not exist. All 82 scope-resolution test files
pass with blocks on.
Verified by probe, per case: two locals in different functions, a local inside
an ANONYMOUS function (`outer.fn@1:9.save@2:4`), sibling blocks resolving to
their own binding, `var` still hoisting out of its block, a nested named
`function` vs a file-level one, PHP composing with the `$` sigil from #2693,
and Python. Top-level/method ids unchanged, asserted directly.
Every assertion is on the EDGE, not on node existence. Ids are built twice and
independently — definition phase and caller attribution — and a one-character
disagreement makes the caller attach to a node that does not exist and the
edge vanish, with nothing thrown and no test failing. An edge assertion can
only pass if both phases agree.
INVALIDATION. INCREMENTAL_SCHEMA_VERSION 17 -> 18 and SCHEMA_BUMP 24 -> 25:
persisted node ids change for every function-local callable, and the cached
scope tree lacks block scopes. A top-up would leave unchanged files on the old
ids while changed files emit the new ones, splitting each symbol in two.
Bench fingerprint unchanged and both timing budgets pass. The one full-suite
failure (incremental-orchestration) passes in isolation — its log shows stale
init locks and WAL reclaim, i.e. LadybugDB contention under the parallel run.
Refs #2699
* perf(ingestion): emit block scopes only where they bind something (#2699)
Block scopes make `let`/`const` in sibling blocks distinct bindings, which is
what stopped a call in one branch resolving to both. Emitted naively — one
scope per `statement_block` — they also cost ~10% of analyze wall time, because
every scope-chain walk in every function then steps through levels that bind
nothing.
Two emit-side filters keep the semantics and drop the waste:
1. A block that IS a function body duplicates the enclosing Function scope.
Nothing can be declared between a function and its own body, so a binding
in either resolves identically — the inner scope is pure depth.
2. A block that declares no `let`/`const`/`class`/`function` binds nothing,
so it is transparent: a lookup finds nothing in it and walks to the
parent. `var` is deliberately excluded from that list — it hoists past the
block to the function, so a block containing only `var` still binds
nothing.
MEASURED, on a 762-file / 228k-line TypeScript corpus (gitnexus/src), min of 6
warmed reps with the cold first rep discarded:
block scopes emitted 19,389 -> 5,331 (-72%)
total scopes 35,942 -> 21,884 (-39%)
analyze wall time +9.8% -> +1.6-2.5% vs pre-#2699
peak RSS (whole tree) 2398MB -> 2434MB (+1.5%, inside run-to-run noise)
The filters themselves are free: scope emission over the same corpus measured
12.6s naive vs 12.5s filtered.
Wall-clock on a shared runner has a ±10% spread run to run, which is wider than
the effect being optimised, so the durable gate added here counts scopes
instead. `bench/scope-emission/measure.mjs --check` asserts an EXACT scope set
over a synthetic corpus that mixes the shapes the filters discriminate between
— function/method/arrow bodies, non-declaring if/else/for/while/try, blocks
that declare `const`, and a `var`-only block. Baseline is 2 block scopes per
module: only the two `if`/`else` branches that declare `const chosen`. If the
filters regress that number jumps immediately, in a way wall-clock CI could
never resolve from noise. Wired into the existing benchmarks job.
Behaviour is unchanged: 86 scope-resolution and identity test files, 1371
tests, all green — including the sibling-block case this could plausibly have
broken — and the callable-value-flow fingerprint is untouched.
Refs #2699
* test(bench): re-baseline the TS/JS scope-capture fingerprints for #2701
`bench/scope-capture` fingerprints the full capture set per language, and
#2701 added a `@receiver-owner.this` marker to every non-arrow function form
so a scope that BINDS its own `this` can terminate the receiver walk. That is
a capture-set change, so the TypeScript and JavaScript fingerprints moved and
the benchmarks job has been failing since that commit — I pushed it without
checking CI.
A fingerprint is a correctness gate, so this does not simply adopt the new
value. Verified first by diffing the capture-name HISTOGRAM over the same
fixture corpus against
|
||
|
Gergő Magyar
|
89bbdcf566
|
fix(ingestion): stop double-indexing const X = () => {} as Function + edgeless Const twin (#2687) (#2691)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Devcontainer Smoke / Config-transform unit tests (push) Has been cancelled
Devcontainer Smoke / Build devcontainer image (push) Has been cancelled
Skill copy sync / shipped skills drift guard (push) Has been cancelled
|
||
|
Gergő Magyar
|
ad1b9227c4
|
fix: large-repo analyze OOM and false worker-timeout cascade (#2649) (#2679) | ||
|
Gergő Magyar
|
7316503ebc
|
perf(analyze): hold structural relationships out of the JS heap, on by default (#2680) (#2685)
* refactor(lbug): extract SyncCsvWriter into a shared module
`PdgEmitSink` (#2202) declared `SyncCsvWriter` as a private, non-exported
class. The structural streaming sink for #2680 needs the same buffered
sync-write + poison/openFailure IO discipline, and importing it is not
possible while it is module-private — so the alternative was copying ~90
lines of it.
Extract the class (and the chunk-rows default it uses) into
`sync-csv-writer.ts` and have `PdgEmitSink` import it.
`DEFAULT_PDG_EMIT_CHUNK_ROWS` stays exported as an alias so no existing
caller changes.
Pure refactor: no behaviour change. pdg-emit-sink.ts 396 -> 302 lines;
tsc clean; the 23 existing #2202 tests pass unchanged.
Refs #2680
* feat(lbug): add GraphEmitSink for streaming structural relationship emit
Structural sibling of PdgEmitSink (#2202): a KnowledgeGraph façade that
routes relationships no mid-pipeline phase reads back to bounded
CSV-on-disk and never stores them. Nothing constructs it yet.
Measurement drove the design. On a kernel-shaped synthetic graph (400k
nodes, 2.7 edges/node):
nodes only ...... 367 B/node
nodes + edges ... 2075 B/node <- reproduces the #2649 ~2.1 KB/node
=> the relationship layer is 83% of graph heap, ~646 B/edge
so streaming *relationships* is where the memory is; nodes stay resident
(they are 17%, and two scope-resolution index builders scan them).
Dropping just the redundant relationshipsByType/edgeIdsByNode indexes was
also measured — 174 of 648 B/edge, ~1.3x — and is not a substitute.
RETAINED_REL_TYPES is derived from an exhaustive audit of every
relationship read site under src/, and each entry names its reader. An
earlier draft carried 14 types, 5 of which no reachable phase reads.
Two deliberate departures from PdgEmitSink, both because its invariants
do not hold here:
- dedup by relationship id, since no upstream per-file uniqueness
guarantee exists for structural edges and COPY would violate the PK;
- removeRelationship on an already-streamed id throws instead of
no-oping, so a mutating consumer cannot corrupt the graph undetected.
Also exposes hasStreamedSemanticEdge for the local-symbol pruner: without
it a block-local symbol referenced only by a streamed edge looks
unreferenced and gets pruned, leaving a CSV row pointing at a node with
no row.
Refs #2680
* feat(analyze): stream structural relationships to CSV under GITNEXUS_STREAM_GRAPH_EMIT
Wires GraphEmitSink into the pipeline behind a full-rebuild-only flag, so
relationships that no mid-pipeline phase reads back never enter the JS
heap. Measured ~2.9x reduction of graph heap:
0.17 (nodes) + 0.83 * 0.21 (retained edges) = 0.344 retained. This is a
constant factor, NOT O(chunk) — node identity and the resolution
registries stay O(repo).
The sink is armed at the PARSE boundary, not at graph construction. An
exhaustive audit of every relationship read site under src/ found four
mid-pipeline CALLS consumers, not the two an earlier draft assumed:
- local-symbol-pruner (full iterRelationships scan, then removeNode)
- communities / processes (whole-graph forEachRelationship)
- mapCobolToGraph, which scans CALLS and REMOVES the unresolved ones —
and runs BEFORE parse, so streaming from construction would have
silently stopped COBOL cross-program call resolution
- taintSummaries, gated on `pdg` and NOT on `skipGraphPhases`, so it
needs its own gate or --pdg + this flag yields an empty taint layer
Accordingly communities, processes, taintSummaries and callSummaries are
all disabled under the flag, and the run logs what it is giving up.
Two fixes that are correct independently of the flag:
- runPipelineFromRepo keyed its community/process extraction off
`!skipGraphPhases` while getPhaseOutput THROWS on a phase filtered out
by any enabledWhen predicate — now a presence check, so filtered
combinations return undefined instead of crashing.
- loadGraphToLbug COPYs one job per CSV FILE rather than per label pair.
#2202's throw-on-collision merge is only sound because BasicBlock pairs
are disjoint; a streamed CALLS edge is Function|Function and always
collides with the whole-graph CSV for that pair, so the structural
manifest appends instead.
The buffer-pool hint adds the streamed row count back in: the hint only
ever shrinks the pool, so sizing it from the post-streaming
relationshipCount would starve the COPY at exactly the scale this
targets.
detect_changes: 18 symbols / 10 files / 9 processes, all within the
planned scope. Full suite green with the flag off.
Refs #2680
* fix(mcp): stop impact() under-reporting risk on a streamed index
An index built with streamed structural emit has no Process or Community
rows, and impact()'s risk scorer uses processCount >= 5 and
moduleCount >= 5 as two of its four CRITICAL escalation criteria. The
missing-table errors are swallowed as benign without raising `partial`,
so nothing distinguished 'this repo has no processes' from 'this index
was built without them' — the same change would report LOW off a streamed
index and CRITICAL off a complete one, with no signal either way.
That is the false-clean shape #2283 ruled out for detect_changes, and it
matters more here because the repo's own workflow mandates impact()
before every symbol edit.
Stamp `graphPhases: 'complete' | 'skipped'` into RepoMeta and have
impact() attach riskUnderstated + an explanatory riskNote when the index
is stamped skipped, so the reported level is explicitly a lower bound.
Unlike the rest of RepoMeta.capabilities this stamp has a real
programmatic reader.
Also documents GITNEXUS_STREAM_GRAPH_EMIT in the README env table,
including everything the flag disables.
Refs #2680
* test(lbug): differential set-identity gate for streamed structural emit
The acceptance property for #2680: for the same node/edge set, the rows
reaching the bulk COPY must be identical whether streaming is on or off.
With streaming on they arrive from two places — the residual in-memory
graph via streamAllCSVsToDisk, plus the sink's per-pair CSVs — so the
test asserts their UNION equals the single whole-graph emit.
Also asserts the split is real (retained + streamed == total, streamed >
0), so a sink that silently streamed nothing cannot pass the equality
vacuously. Verified discriminating: with sink.arm() commented out the
test fails ('expected 0 to be greater than 0'); restored, it passes.
Fixture spans both sides of RETAINED_REL_TYPES and includes a self-edge
and a duplicate relationship id — the cases where a naive sink diverges
from the whole-graph emit.
Drives the sink directly rather than running analyze, matching
pdg-emit-streaming-roundtrip.test.ts: the guarantee is about emitted
rows, and the worker pool would add unrelated machinery without
strengthening the assertion.
Refs #2680
* fix(test): remove literal NUL byte and cover streamGraphEmit phase gating
Two review findings, both verified before accepting.
1. The round-trip test contained a literal NUL byte as a key separator,
which made Git treat the whole .ts file as BINARY —
`git show --numstat` reported `-\t-` for it, so the file would not
diff or blame and CI text tooling would skip it. Replaced with the
escaped \\u0000 sequence; behaviour is identical, the file is text
again. (Found by the Codex swarm lane.)
2. buildPhaseList's four new streamGraphEmit gating predicates and the
flag-off default path had no test that would fail on revert — two
review lanes flagged this independently. Reversing any enabledWhen
condition would have passed the suite silently, which matters because
an ungated taintSummaries yields an empty taint layer rather than an
error.
Added four cases: the streamed run drops communities/processes/
taintSummaries/callSummaries; it keeps mro/di (their reads are all in
RETAINED_REL_TYPES); the flag-off list is untouched; and skipGraphPhases
still works independently.
Refs #2680
* fix(analyze): don't leak a temp dir when streaming is off; correct two overclaims
Three review findings, all verified before accepting.
1. `graphEmitCsvDir: resolveNativeSafeStorageDir(...)` was evaluated
unconditionally inside the pipeline-options literal. On a Windows
non-ASCII storage path that helper mkdtempSyncs a REAL directory, so
every analyze leaked one temp dir even with the flag off. Now resolved
only when streaming is active, matching how the PDG sibling resolves
inside its own guard. This was the only finding affecting flag-off
users.
2. The retain-set comment claimed 'the differential round-trip test is
what catches drift'. It cannot. addRelationship PARTITIONS edges
between the graph and the CSVs, and the union of a partition is
invariant under where the partition line falls — so that test stays
green no matter how RETAINED_REL_TYPES is drawn. Only the read-site
audit protects the invariant, and the comment now says so and names
the grep to re-run.
3. The ~2.9x figure assigned streamed edges a retained cost of zero,
ignoring the sink's own streamedIds/streamedEndpoints Sets — and
relationship ids are plain concatenations of both endpoint ids, not
hashes. Review measured those Sets at ~35% of full per-edge retention,
not the '~a tenth' assumed, putting the real figure nearer ~1.7-2.2x;
a member-dense Java/C# repo lands lower still, since the retained
structural spine is a larger share there than in the TypeScript census
the 0.21 came from. Code comment and README now give a range and say
plainly that no end-to-end measurement on a real repository exists yet.
Refs #2680
* fix(mcp): disclose degraded risk in detect_changes; stop pinning the sink
Two more review findings, both cross-lane corroborated.
1. detect_changes derives risk_level SOLELY from affected-process count,
and a graphPhases:'skipped' index has zero Process rows by
construction. The STEP_IN_PROCESS query then succeeds with zero rows,
so queryDegraded stays false and the tool returns risk_level 'low',
affected_count 0, with no partial marker — for every change, forever.
That is a false-clean on the gate this repo mandates before every
commit, and it is the same #2283 shape the previous commit fixed in
impact() while leaving its sibling untouched. Now carries the same
riskUnderstated + riskNote disclosure.
2. PipelineResult.graphEmitSink had zero readers — the pruner predicate
and the manifest are both threaded elsewhere — but returning it kept
the sink, and therefore its O(streamed-edges) id and endpoint Sets,
reachable through the entire COPY/FTS/embedding phase. That is
precisely the phase this feature exists to fit inside RAM, so the
field actively worked against the change's purpose. Dropped.
Refs #2680
* refactor(2680): one named capability, one risk helper, a shorter header
Pure cleanup pass — no behaviour change, 66 tests across the six affected
suites still green, and the round-trip test still fails when the sink is
left un-started.
Three things were untidy:
1. The phase layer reached the sink through TWO loose callbacks bolted
onto PipelineContext (`armStreaming`, `hasStreamedSemanticEdge`) —
two fields, two wiring lines, no name for the thing they belonged to.
Replaced by one `graphEmit?: GraphEmitControl`, a two-method interface
declared beside the sink. Phases now say what they mean:
`ctx.graphEmit?.beginStreaming()`. Also renames `arm()` to
`beginStreaming()`, which needs no comment to explain.
2. The degraded-index risk disclosure was copy-pasted into impact() and
detect_changes() — two meta probes, two near-identical prose blocks,
and two long comments restating the same reasoning. Now one
`streamedIndexRiskDisclosure()` helper carrying the explanation once;
each caller passes only the clause naming which count is structurally
zero for it. Same file, 45 lines in / 45 out, with the duplication gone.
3. The sink's file header had grown into a changelog of my own review
corrections ('this once assumed', 'review measured'). A reader does not
care what an earlier draft believed. Rewritten to state the design
argument once — relationships are ~83% of graph heap, so they are what
streams; nodes are the other 17% and are scanned, so they stay — under
headings, with the honest 'this is an estimate, ~1.7-2.2x, no real-repo
measurement yet' caveat kept in full.
Refs #2680
* feat(analyze): make streamed graph emit the default, with nothing traded away
Streaming was opt-in because it disabled the four phases that consume the
whole CALLS graph — communities, processes, taintSummaries, callSummaries.
That made it unshippable as a default: query() is process-grouped and
clusters/skill-gen are community-backed, so every index would have silently
lost them.
The sink now answers a COMPLETE relationship read. It keeps streamed edges
as four parallel columns over an interned node table — sourceId, targetId,
type, confidence — and iterRelationships/iterRelationshipsByType/
forEachRelationship/relationshipCount return the retained edges
concatenated with those. Every consumer therefore sees the whole graph and
no phase knows streaming happened.
Four fields, not six, because an audit showed community-processor,
process-processor, taint-summaries and the pruner read only those — none
keys on rel.id. That matters: relationship ids are unique long strings, and
retaining them is precisely what made a fully-columnar attempt LOSE to the
object graph (measured 838 MB vs 822 MB). Ids stay out of the columns; a
read synthesizes one, which is safe because buildRelRow never persists it.
Consequently deleted, not merely disabled:
- the four enabledWhen gates and the 'what you give up' warning;
- the pruner's hasStreamedSemanticEdge predicate and its plumbing — a
complete scan sees streamed edges, so the dangling-edge hazard is gone by
construction rather than by compensation;
- the whole degraded-index apparatus: the graphPhases RepoMeta stamp,
streamedIndexRiskDisclosure, and the riskUnderstated markers on impact()
and detect_changes(). Nothing degrades, so nothing needs disclosing.
Default is ON for full rebuilds; GITNEXUS_STREAM_GRAPH_EMIT=0 (or an
explicit option) is the escape hatch, for bisecting a suspected
streaming fault rather than routine use. Incremental runs still refuse it —
the writeback reads relationships back out of the in-memory graph.
Measured A/B, 400k nodes / 1.08M edges, all edges streamable (worst case
for this design): 823 MB -> 626 MB, ~1.3x, all 1.08M edges still visible.
That is deliberately less than the ~2.9x the retained-share formula
implies — losslessness costs the dedup Set and the columns. The earlier,
bigger number was bought by disabling phases. README and the file header
both state 1.3x measured; neither claims O(chunk).
New coverage: reads are complete (proven discriminating — 3 tests fail when
the streamed leg is removed), endpoints/confidence survive the round trip,
per-type lookup finds streamed types, and every CALLS-consuming phase stays
registered under the flag.
Refs #2680
* docs(2680): pin the invariants the default-on change relies on
Review follow-ups. No behaviour change except the id-uniqueness fix.
- pipeline.ts returns the RAW graph, not the sink, and that is load-bearing:
phases read the sink so their scans are complete, but loadGraphToLbug feeds
this value to streamAllCSVsToDisk, whose iterator would then emit every
streamed edge a SECOND time on top of the per-pair CSVs the sink already
wrote. Returning the sink there silently doubles every streamed
relationship in the persisted graph, so the reason is now written down at
the return site.
- Synthesized ids now carry the column index, making them unique even when
two streamed edges share (type, source, target) and differ only in
reason/step. Harmless today because no consumer keys on relationship id,
but real ids are unique and the synthesized ones should match, so a future
id-keyed consumer cannot silently collapse two edges.
- Recorded WHY dropping reason/step is safe, which is not the same argument
as for id: the persisted row keeps their true values because buildRelRow
receives the original relationship on the way through, so only in-memory
reads see the 'streamed' placeholder. The ACCESSES reason:'read'|'write'
distinction that MCP queries depend on therefore survives in the database.
A future in-pipeline consumer needing either field must add a column rather
than trust the placeholder.
Also verified while chasing a review lead: removeNodesByFile has no
production callers and removeNode has exactly one (the pruner), which reads
through the sink and so sees streamed edges. The dangling-edge hazard the
deleted hasStreamedSemanticEdge predicate used to compensate for is closed
by construction, not by luck.
Refs #2680
* fix(2680): fail loudly on a missing CSV dir, and guard the retain set
Resolves both findings from the review of this branch.
MEDIUM — pipeline.ts silently skipped streaming when `streamGraphEmit` was
true but `graphEmitCsvDir` was absent. The CLI always supplies the dir, but
streaming is on by DEFAULT now, and the callers that build PipelineOptions
themselves (eval-server, MCP daemon, tests) are exactly the ones that would
omit it — so they would ask for streaming, not get it, and still see a
successful run. That is the silent-degraded-outcome shape the rest of this
work exists to prevent, so it now throws with the resolution hint. Covered by
a test asserting the rejection.
LOW — RETAINED_REL_TYPES had no automated guard, and the round-trip test
structurally cannot be one: addRelationship PARTITIONS edges between the
graph and the CSVs, and a partition's union is invariant under where the line
falls, so that test stays green for any partitioning including a wrong one.
Drift there yields a silently incomplete mid-pipeline edge set, not a crash.
Added a test that derives the required set by grepping every literal
iterRelationshipsByType('X') under src/ and asserts the constant covers it,
with CALLS as the documented exemption (taintSummaries reads it, which is why
the sink answers a complete read rather than retaining it). Proven
discriminating: removing EXTENDS from the constant fails with
"expected [ 'EXTENDS' ] to deeply equal []".
128 tests green across the eight affected suites, including the index-lock
suite that arrived with the #2677 merge.
Refs #2680
* docs(2680): record the measured CPU cost, not just the memory win
I measured memory before shipping and never measured time, which was a gap:
reads now allocate, rebuilding objects instead of returning stored ones, and
a real analyze does SIX full relationship scans (pruner, communities x2,
processes x2, the taint fixpoint's CALLS pass).
Same 400k-node / 1.08M-edge graph:
heap 820 MB -> 623 MB (1.32x better)
scans 96 ms -> 651 ms (6.8x WORSE)
6.8x on iteration is worth knowing, but the absolute number decides it:
~0.5 s here, ~2 s extrapolated to kernel scale, against an analyze measured
in minutes — under 1% of wall-clock. The ~26M short-lived objects at kernel
scale are young-generation churn (the cheap case), and being ~800 MB further
from the heap ceiling matters more than the churn costs: #2649's cascade came
from GC thrash NEAR the limit, not from allocation volume as such.
Also names the first lever if these scans ever go hot — a per-type index over
the columns, so iterRelationshipsByType stops scanning all streamed edges —
and notes that it trades memory back, so it needs a measurement first.
Refs #2680
* perf(2680): cut the iteration regression from 6.8x to 1.8x
The memory win came with an unmeasured CPU cost. Iteration went from
returning stored objects to rebuilding them, across the SIX full relationship
scans an analyze performs (pruner, communities x2, processes x2, taint's CALLS
pass). First measurement: 90 ms -> 651 ms, 6.8x worse. Fixed properly rather
than documented away.
Two causes, each measured before and after:
1. The ~150-character synthesized `id` was built eagerly on every read — 6.5M
concatenations per analyze, for a field NO in-pipeline consumer reads.
Isolating it (constant id) showed 436 ms of the 555 ms regression. Now a
lazy prototype getter on a fixed-shape `StreamedRelationship` class: the
string is built only if someone asks, and V8 keeps one hidden class across
millions of instances.
2. Generator and iterator-protocol overhead on million-edge walks.
`forEachRelationship` (community detection's form, called twice) now loops
the columns directly, skipping both. `iterRelationships` keeps an iterator
but reuses one result record — a hand-rolled version allocating a fresh
{value, done} per edge measured WORSE than the generator (252 ms), which is
why the obvious rewrite is not the one that shipped.
heap 821 MB -> 623 MB (1.32x better)
scans 90 ms -> 180 ms (was 651 ms)
The residual ~90 ms is object allocation, 6.5M instances across six scans, and
it is irreducible while the read API returns objects at all. The remaining fix
for true parity is a field-wise callback passing sourceId/targetId/type/
confidence as primitives — all four hot consumers read only those — but that
changes the KnowledgeGraph interface and its consumers, so it belongs in its
own measured change rather than bolted on here.
Refs #2680
* perf(2680): zero-allocation field scan brings iteration back to parity
Third and final step on the iteration cost. The memory win had come with a
6.8x iteration regression; the previous commit cut that to 1.8x by making the
synthesized id lazy and removing generator overhead. The residual was object
allocation itself — 6.5M instances across the six full relationship scans an
analyze performs — which no amount of tuning removes while the read API hands
back objects.
So the hot consumers stop asking for objects. Adds
`KnowledgeGraph.forEachRelationshipFields`, which passes
(sourceId, targetId, type, confidence) as primitives — exactly and only what
every whole-graph scan reads. On the sink those come straight out of the
columns, allocating nothing; on the object-based graph they are read off the
stored relationship, so the flag-off path is unaffected.
Converted the five whole-graph scans: community detection (x2), process
extraction (x2), and the local-symbol pruner. `isFileDefinesEdge` now takes
(type, sourceId) rather than a relationship. The taint fixpoint's by-type pass
is left alone — one scan of six, and converting it would turn an indexed
bucket lookup into a full scan on the object-based graph.
heap 820 MB -> 623 MB (1.32x better)
scans ~82 ms -> ~90 ms (was 651 ms; now parity within noise)
Also deletes the pruner's `hasStreamedSemanticEdge` option, which has had no
caller since the sink's reads became complete — a dead knob is worse than no
knob.
Verified: 104 tests across the eight affected suites, including the pruner's
pipeline integration test (which needs the raised worker-ready timeout on this
host; it passes cleanly with it and its failures are the known 5s handshake).
Refs #2680
* perf(2680): compact dedup keys — 1.32x -> 1.59x, speed unchanged
An audit of where duplicate relationship ids actually come from, then the
saving it unlocked.
The audit (instrumented analyze of this repo): 25 duplicate-id hits across
63,412 streamed edges — 0.04%, all CALLS, every one the SAME call site
re-emitted when a file is resolved in more than one language pass. Three
things follow, and they rule out the cheap options:
- dedup cannot be dropped (25 != 0, and a duplicate reaching COPY is a wrong
graph);
- it cannot move to row contents, because emit-references builds ids as
`...->target:line:col`, so two calls between the same pair at different sites
have byte-identical CSV rows that the whole-graph emit keeps;
- it cannot move to a per-file source guard like `pdgEmittedFiles`, because a
later language pass can resolve genuinely NEW edges for the same file.
What was left was the key itself. An id embeds both node ids in full (~200
chars here) while the endpoints are ALREADY interned for the columns, so the
Set was storing them twice. Keys are now built from the interner indices plus
the id's trailing disambiguator parsed into NUMBERS.
Numbers, not substrings, and that is load-bearing: a key built by slicing
inside a long string is a V8 sliced/cons string that keeps its parent alive, so
the id would never be freed and the saving would silently fail to appear. An
earlier attempt at this measured no improvement for exactly that reason.
Unrecognized id shapes (`rel:contains:` has no tail) fall back to storing the
id verbatim — correctness first, saving second.
heap 821 MB -> 518 MB (1.59x, was 1.32x)
scans ~83 ms -> ~88 ms (parity, unchanged)
Speed is untouched by construction: dedup is on the WRITE path, and none of
the six full scans reads it.
Also fixes removeRelationship, which the test suite caught: it looked up the
raw id in a Set that now holds compact keys, so it silently stopped throwing on
an already-streamed edge. It cannot recompute a key from a bare id, so it is
now conservative — anything the real graph does not hold is treated as
possibly-streamed once streaming has begun and fails loudly. A genuinely-absent
id throws where main returns false; acceptable because the only production
caller (the COBOL resolver) runs before the sink is armed.
89 tests green across the six affected suites.
Refs #2680
* fix(2680): dedup key dropped edges when tail segment counts differed
Both findings from the review of this branch, and the coverage gap named
alongside them.
HIGH — the compact dedup key packed the id's trailing numeric segments as
`|${a}|${b}`, with `b` defaulting to 0 when only one segment was present and
the segment COUNT absent from the key. So `:7` and `:7:0` produced the same
key and the second edge was silently discarded as a duplicate: a lost
relationship, no error, no warning. Found by probe, not by reading — two
distinct ids for one (source, target, type) went in and one edge came out.
The key now carries `seen`.
Nothing existing caught it. The round-trip test compares the UNION of graph
and CSV rows, and a dropped edge is missing from both, so it stayed green;
the duplicate test only feeds a genuinely identical id, which is the case
that SHOULD collapse. Four new cases pin the boundary instead: differing
segment counts stay distinct, two call sites between one pair stay distinct
(the `:line:col` shape from emit-references), a truly repeated id still
collapses, and a non-numeric tail falls back to the full id. Proven
discriminating — reverting the fix fails with "expected 1 to be 2".
This costs ~66 MB at 400k nodes / 1.08M edges (584 MB, was 518 MB), so the
heap win is 1.40x rather than 1.59x. Not a trade worth making the other way:
a silently missing relationship is the exact failure class the rest of this
work exists to prevent. I am not asserting a mechanism for why two extra
characters per key cost that much — it is stable and reproducible across
runs, and inventing a cause is how I got the earlier cons-string diagnosis
wrong.
LOW — removeRelationship throws for an absent id once streaming has begun,
where KnowledgeGraph.removeRelationship returns false. The behaviour is
deliberate (a bare id cannot be turned back into a compact key, and answering
"false" for an edge already on disk is the worse failure) but it was
undocumented and untested. Now stated on the interface itself and pinned by
two cases: absent-id-while-streaming throws, absent-id-before-streaming
returns false.
Coverage gap — added a test asserting forEachRelationshipFields yields the
same (source, target, type, confidence) tuples as iterRelationships. That
guards the five whole-graph scans converted in
|
||
|
Gergő Magyar
|
1e764cd475
|
fix(analyze): single-writer lock for the index write path (#2658) (#2677) | ||
|
Copilot
|
d3d4fa31bb
|
fix(scope-resolution): gate C#/Kotlin free calls by instance ownership (#2563) (#2654)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* Initial plan * fix(scope-resolution): gate C# and Kotlin free calls * fix(scope-resolution): keep Kotlin ownership gate safe * Apply remaining changes * perf(scope-resolution): benchmark and cache ownership gates * test(scope-resolution): simplify benchmark scaling loop * refactor(scope-resolution): encapsulate ownership cache * test(scope-resolution): enforce subquadratic ownership scaling * fix(scope-resolution): address ownership review findings * test(csharp): regenerate capture golden for #2563 fixtures The committed expected-captures.json was missing the new NamespaceOwnerCollision.cs entry and carried a stale SameFileCases.cs digest/count (56 → 67), so csharp-captures-golden.test.ts was the sole red check on the PR. Regenerate with UPDATE_GOLDEN=1 to match the fixtures the bench fingerprint already reflects. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Copilot
|
450cebc268
|
fix(java): JLS binary-name identities for local classes, enums, records & interfaces (#2562) (#2653)
* Initial plan * docs(plans): add Java local class naming plan * fix(java): model local class binary names * docs(java): clarify local class naming guards * fix(java): recognize local classes in compact constructors * chore: remove Java naming plan * fix(java): harden local type identities and scope * perf(java): linearize local type ordinal allocation * fix(java): harden ordinal benchmark follow-up * docs(java): clarify ordinal benchmark invariants * test(java): cover local type ownership paths --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
MyShining
|
4af6fe8587
|
feat(spring): resolve constructor and standard injection (#2632) | ||
|
Gergő Magyar
|
170805647c
|
fix(rust): keep duplicate type names ambiguous in range binding (#2514) (#2652)
* fix(rust): latch duplicate type-name ambiguity in range binding (#2514) The range-binding prepass tracked cross-file return and field types in two maps and used map presence itself as the ambiguity flag: the second definition of a name deleted it, but a third definition found it absent and re-inserted the last-scanned file's type. Odd duplicate counts (3, 5, ...) therefore resolved a genuinely ambiguous name to whichever file was scanned last, while even counts stayed ambiguous. Latch ambiguity in a dedicated Set per registry (ambiguousReturnTypes, ambiguousFieldTypes): once a name has two or more workspace definitions it never resolves again, regardless of duplicate count or file order. Adds integration coverage for two/three-duplicate functions and structs, permuted file order, and a unique-name over-suppression guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rust): bump INCREMENTAL_SCHEMA_VERSION to 12 for the #2514 range-binding fix The duplicate-name ambiguity latch changes which cross-file Rust CALLS edges the range-binding prepass emits. The incremental writeback persists only changed-file nodes, so an incremental top-up against a pre-v12 index would keep the old spurious edges on every unchanged Rust file. Bump the schema version to force a one-time full re-analyze, matching the v7/v11 contract for edge-affecting resolver changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(rust): resolve import-disambiguated duplicate types in for-loops & destructuring Follow-up to the #2514 ambiguity latch. When several modules define the same function/struct name and a call site disambiguates it with a `use` import (including aliases and `use x::*` globs), range-binding now resolves the for-loop element type and the destructured field type to that specific imported definition, instead of leaving it unresolved. The bare-name return/field maps are (correctly) ambiguous for duplicates, but the call site's import pins a definition. range-binding records the full, untruncated return/field type per defining file, and resolveImportedDef() resolves a name to the single in-scope definition, mirroring Rust name resolution: - tier 1: explicit `use`/re-export imports and local defs (lookupBindingsAt); these shadow globs, so if any exist we decide within them alone; - tier 2: glob imports, consulted only when tier 1 is empty; a `wildcard-expanded` ImportEdge names the target module, so we resolve only when exactly one glob-target file actually defines the name. Two or more visible definitions stay unresolved, preserving the #2514 latch. normalizeRustReturnType is untouched (its Vec<T> -> Vec truncation is load-bearing for receiver resolution), so the full generic is read from the per-file map instead. Covered by integration tests: explicit / aliased / single-glob imports resolve to the imported definition; two globs that both export the name stay ambiguous; a local definition shadows a glob; no-import duplicates stay unresolved (#2514). INCREMENTAL_SCHEMA_VERSION stays at 12 (bumped by the #2514 commit in this PR); its note now also covers these added resolution edges. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(rust): parse each file once in range-binding when the workspace fits a budget populateRustRangeBindings makes two passes over every file and, because the shared treeCache is empty in the analyze flow, re-parsed each file in both — a workspace of N files paid 2N parses. It now parses each file once and reuses the tree across both passes via an in-function store, gated by a source-byte budget: workspaces up to 16 MiB of Rust source (essentially every real repo) reuse trees; larger ones fall back to per-pass re-parsing so peak RSS stays bounded on huge repos (the memory-sensitive case keeps its current profile). Also collapses the parse+timeout boilerplate that was copy-pasted in both loops into one getOrParseTree helper, and adds a PROF-gated `rangeBind=` segment to the scope-resolution profiler for phase-level observability. Measured on a 500-file synthetic Rust workspace (PROF_SCOPE_RESOLUTION=1): the range-binding phase drops ~370ms -> ~320ms (~14%), parses 1000 -> 500. Behavior is unchanged (199 rust + range-binding-order + parse-timeout tests green); repos above the budget are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(rust): update schema-version gate to v12; regenerate golden + bench baseline for new fixtures CI surfaced three deterministic-artifact failures, all from this PR's own additions: - call-summary-schema-version.test.ts hardcoded INCREMENTAL_SCHEMA_VERSION === 11 (the #2604 window); #2514 bumped it to 12. Update the gate and extend the reuse-gate version history so a v11 stamp now forces a full re-analyze. - rust-captures-golden expected-captures.json drifted (130 -> 174 entries) because the new rust-import-* / rust-dup-* fixtures joined the rust-* corpus. Regenerated (UPDATE_GOLDEN=1): additions only, no existing captures changed — emitRustScopeCaptures is untouched. - bench/scope-capture/baselines.json rust fingerprint drifted for the same reason. Rebaselined with a provenance note; scaling 1.06 < 1.5 budget. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude <claude@anthropic.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
cdbdf219dc
|
fix(lbug): reclaim missing-shadow WAL quarantine files on write-path init (#2638)
Some checks are pending
Gitleaks / gitleaks (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
Abhigyan Patwari
|
0eeecb37f3
|
fix(python): resolve calls through constructor-injected fields (#2628)
* fix(python): resolve calls through injected fields * fix(ci): update python capture benchmark fingerprint * fix(python): make constructor field inference conservative --------- Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
e814e28f1f
|
fix(deps): bump @ladybugdb/core to ^0.18.3 — rel-property IN-predicate fix (#2508) (#2634)
LadybugDB ≤0.18.2 mis-evaluated `r.type IN [...]` on relationship table groups: the boolean-filter fallback skipped writing selection buffers for single-row unflat chunks, dropping/duplicating callers in context() and impact() (upstream LadybugDB#692, fixed by LadybugDB#699, shipped in 0.18.3). Floor the dependency at ^0.18.3 and lock core + all five platform packages. Resurrect the caller-identity regression test from PR #2553 (closed as superseded by the upstream fix): it pins context()/impact() to exact caller IDs across CodeRelation sub-table pairs so any future predicate regression fails loudly. Note: with CREATE-seeded data the test also passes on 0.18.2 (the upstream repro needs COPY-written chunk layouts) — it is a behavioural pin, not a bug reproduction. Fixes #2508 Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Gergő Magyar
|
7f7255aef8
|
fix(analyze): load VECTOR before the incremental writeback touches embedding rows (#2623) (#2624)
* feat(lbug): add ensureEmbeddingRowDmlSafe VECTOR gate for embedding-row DML LadybugDB refuses every mutation of a table carrying an HNSW index while the VECTOR extension is not loaded on that connection: DELETE and CREATE raise a Binder exception, DROP TABLE is refused while the index references it, and SET segfaults the process. Dropping the index is not an available recovery either — CALL DROP_VECTOR_INDEX is itself a VECTOR-extension function and is undefined in exactly that state. Add a single primitive that loads VECTOR under the analyze install policy and, only when that fails, reads CALL SHOW_INDEXES (which works without the extension) to decide whether an index actually exists to trip over. No call sites yet. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): pin the #2623 VECTOR gate for embedding-row DML Three cases: no index + VECTOR unavailable stays safe (no needless escalation); index present + VECTOR unavailable is reported blocked AND the raw deleteNodesForFiles genuinely throws 'extension is not loaded' (proving the hazard is real, not theoretical); index present + VECTOR loadable is safe, the delete works, and the HNSW index survives — the invariant run-analyze relies on when it keeps the index across a surgical incremental run. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): load VECTOR before the incremental writeback touches embedding rows Incremental analyze died on every content change once a repo had built code_embedding_idx: Analysis failed: Binder exception: Trying to delete from an index on table CodeEmbedding but its extension is not loaded. The surgical writeback's first statement is deleteNodesForFiles' CodeEmbedding join-delete, but nothing on that path loaded VECTOR until Phase 4 — so the engine refused the delete. This is an ordering defect, not an environment one: it reproduces on machines where VECTOR loads fine. The dirty-flag recovery then forced a full rebuild on the next run, which is why it read as 'just slow'. Call ensureEmbeddingRowDmlSafe() once, before the escalation gate and before any row is touched — the same 'index lifecycle before row DML' seam dropSearchFTSIndexes occupies for FTS (#2589). Unconditional, because a DB carrying the index from an earlier --embeddings run hits the same wall on a plain incremental run. When VECTOR truly cannot load the table is immutable (the index cannot be dropped without the extension either), so the run falls through to the existing wipe-and-COPY escalation with a message naming cause, consequence and remedy. Fixes #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(analyze): pin the #2623 VECTOR-before-embedding-DML ordering end-to-end Sibling of the #2589 FTS drop-before-delete suite, same shape: drive the real runFullAnalysis incremental path over a real git repo and a real LadybugDB, seed real embedding rows, build the HNSW index, then assert the index state at the exact moment deleteNodesForFiles is invoked. Both cases were confirmed to discriminate — with the run-analyze change reverted they fail with the reported 'Trying to delete from an index on table CodeEmbedding but its extension is not loaded', and pass with it: - surgical path: the run completes, the index is still present AND extension_loaded at delete time, exactly one row per nodeId survives, and the untouched file's rows are preserved - blocked path: with GITNEXUS_LBUG_EXTENSION_INSTALL=never the run escalates to a full DB write and says so, instead of crashing Also applies prettier's reindent to the run-analyze log ternary. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(lbug): cite the pinned LadybugDB version in the #2623 probe note The probe matrix behind ensureEmbeddingRowDmlSafe was first recorded on 0.18.0, but gitnexus/package-lock.json pins 0.18.2 (#2587). Re-ran every case on 0.18.2: refused DELETE, refused CREATE, SIGSEGV on SET, DROP_VECTOR_INDEX undefined, DROP TABLE refused, SHOW_INDEXES readable with extension_loaded intact. Identical on both, so the design is unchanged — only the citation was wrong. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): preserve embeddings across the VECTOR-blocked rebuild, and check the catalog before loading Three follow-ups from reviewing the fix itself. 1. Data loss on the blocked path. Escalating wipes the DB files, and Phase 3.5 restores embedding rows from cachedEmbeddings — which deriveEmbeddingMode only populates when meta.stats.embeddings > 0. A DB holding embedding rows that its meta does not account for therefore had every vector destroyed silently by a rebuild it never asked for. Probe on a 3-file repo: 3 rows before, 0 after, no warning. Read the rows before escalating (a plain MATCH, no extension needed) so the existing restore has something to restore, and say so in the log. The blocked-path test now asserts the seeded rows survive exactly once, and that assertion fails without this rescue. 2. Catalog before extension. ensureEmbeddingRowDmlSafe loaded VECTOR first and only read SHOW_INDEXES on failure, so every incremental analyze on a machine without VECTOR paid a bounded out-of-process INSTALL attempt plus an 'extension unavailable' warning — including repos that never built an embedding index and can never hit this bug. One local catalog read settles that case first; the load is attempted only when an index actually gates DML, or when the catalog cannot be read. 3. Dead branch. targetConn is always the module singleton there, so the isSharedSingletonConn ternary could never take its second arm. Collapsed to withConnLock. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(doctor): live-probe the VECTOR extension instead of printing the static platform capability Review finding on #2624 (MEDIUM), and exactly what #2623's reporter hit: doctor printed 'VECTOR index: available' — derived from a static platform check — while every incremental analyze on the same machine was dying on an unloaded VECTOR extension. The FTS line was switched to a live LOAD probe for the identical contradiction under #2374; VECTOR now gets the same treatment. probeVectorExtensionLoad shares the FTS probe's implementation (bounded, offline-safe, never runs the installer) and doctor's semantic-mode line now follows the probe, not the platform: without a loadable extension the vector index can be neither built nor queried, so search really is on exact scan. The load-error classifier's remedies are label-parameterized so the VECTOR row stops dispensing FTS-specific advice — 'run analyze --repair-fts' repairs FTS indexes only and was actively wrong for a missing vector extension. Default label stays 'FTS'; every existing caller and pinned remedy string is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lbug): remove the stale Windows VECTOR gate — the extension ships for win_amd64 The codebase categorically refused VECTOR on Windows (platform !== 'win32' in isVectorExtensionSupportedByPlatform, plus a hard early-return in loadVectorExtension) on the strength of an early-era report that in-process INSTALL VECTOR could SIGSEGV (#1365). That belief is stale, verified directly: - the extension server hosts win_amd64 VECTOR artifacts for every 0.18.x extension version — v0.18.0 and v0.18.1 both serve a real 14 MB PE32+ DLL (curl-probed; 'file' confirms PE32+ x86-64) - the pinned 0.18.2 core resolves its extension directory to 0.18.1 (strace-verified LOAD open()), so the pinned version's Windows artifact exists too - INSTALL now runs in a spawned child (installDuckDbExtensionOutOfProcess), so even a crashing installer kills only the child and degrades to unavailable — the original hazard cannot reach the parent process any more Windows now takes the same runtime path as every other OS: try LOAD, install out-of-process when policy allows, degrade to exact scan when it truly fails. The MCP semantic-search lane loses its static platform gate too — it always attempts the vector index and falls back to the exact scan on runtime failure, with a once-per-backend diagnostic naming the real error instead of a platform-policy message. isVectorExtensionSupportedByPlatform is deleted; getRuntimeCapabilities reports the platform capability as available everywhere and defers machine truth to the live probe. Windows CI is the enforcement: the vector suites skip visibly only when the extension genuinely cannot load, so green Windows lanes now actually exercise VECTOR instead of silently skipping by policy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): pin the catalog-read-failure fallback in ensureEmbeddingRowDmlSafe Review finding on #2624 (LOW): the one branch where the gate cannot cheaply prove safety — SHOW_INDEXES itself erroring — was exercised only by inference. Force it with a Connection.prototype.query spy over the real DB: the catalog read fails, and the gate must fall through to actually attempting the extension load (asserted via the recorded statement stream) rather than guessing, returning true here because the extension is loadable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): load VECTOR on the pool's shared Database so the semantic vector lane actually works Review finding on #2624 (MEDIUM): extension load scope is per-Database (probe-verified — LOAD on one connection enables QUERY_VECTOR_INDEX on every connection of the same Database), and the pool pre-warm loaded only FTS. So LocalBackend's vector lane has ALWAYS raised 'Catalog exception: function QUERY_VECTOR_INDEX is not defined' through the pool and silently fallen back to the exact scan — repos above the 10k exact-scan cap got empty semantic results. The serve path was unaffected (the embedding pipeline loads the extension itself). Mirror the FTS line at BOTH load sites — doInitLbug's pre-warm and initLbugWithDb's external-Database adoption — under the same load-only contract (the read pool never triggers a network install), tracked by a new SharedDB.vectorLoaded flag reset where ftsLoaded resets. The new pool test is discriminating and deliberately closes the writable core adapter before the pool opens: a shared/injected Database would inherit the VECTOR load from test seeding and pass either way, so the case forces the pool onto its OWN fresh read-only Database where only the pre-warm can make the lane legal. Verified: fails at the pre-fix tree with the exact Catalog exception, passes with the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: run the #2623 ordering suite on Windows/macOS and pre-install VECTOR alongside FTS Two review findings on #2624, both landing in existing seams: - scripts/cross-platform-tests.ts gains incremental-vector-extension-ordering .test.ts: the win32 VECTOR gate is gone in this PR, so the #2623 drop-ordering + blocked-path escalation must be proven on the windows-latest native addon, not just Ubuntu. (The review's claim that lbug-delete-nodes-for-files.test.ts was also missing was wrong — it has been on the roster since #2409.) - scripts/ensure-fts.ts now pre-installs VECTOR under the same best-effort auto-policy contract, so every sharded CI process LOADs from ~/.lbdb instead of racing its own bounded out-of-process INSTALL; the workflow's extension cache already covers it (path is the whole extension dir — key kept for cache continuity). The cross-platform job sets GITNEXUS_REQUIRE_VECTOR=1 beside GITNEXUS_REQUIRE_FTS so a genuinely unavailable VECTOR is a loud failure, never a silent skip. Windows/macOS cannot be executed locally; the PR's CI lanes are the proof for this commit. Linux smoke: ensure-fts.ts reports both extensions ready; all 79 roster entries resolve. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(pool): register loadVectorExtension in the pool unit-suite mocks The pool adapter's new loadVectorExtension import surfaced in four suites that mock lbug-adapter.js with explicit factories (vitest fails loudly on a missing mocked export). Register the export in each — resolving false where the suite's world assumes no vector, true where it mirrors FTS — and extend lbug-pool-fts-load.test.ts, the suite that owns pre-warm extension loading, with the vector pair: successful load cached per shared Database, failed load retried on the next open, both pinned to policy load-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(analyze): use POSIX literals for graph paths in the #2623 ordering suite First Windows CI run of this suite (it joined the cross-platform roster this PR) failed with 'Parser exception: Invalid input <MATCH (n:Function) WHERE n.filePath = '>' — path.join produces backslashes on Windows, and a backslash inside the seed helper's single-quoted Cypher literal breaks the parser. The graph stores repo-relative filePaths with forward slashes on every OS, so graph-side paths are POSIX literals now (the incremental-orchestration convention); path.join stays only for real filesystem access. The same Windows lane also proved the substance this suite exists for: lbug-vector-extension passed 7/7 on windows-latest — the extension installed, loaded, and built a real HNSW index there — and the pool vector-lane and DML gate suites passed too. This commit fixes the harness, not the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <abhigyan1.patwari@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Abhigyan Patwari
|
9efc6bfcad
|
fix(lbug/analyze): atomic index swap + read-pool staleness invalidation (#2614)
* fix(lbug): re-open the read pool when analyze rebuilds the index under it
The MCP read pool's initLbug early-returned on an existing pool entry with no
freshness check, so after analyze rebuilt or mutated the on-disk index the
pool kept serving the old (POSIX: unlinked-but-open) inode until LRU/idle
eviction — a silent stale-read window of up to IDLE_TIMEOUT_MS (5 min).
Record the file identity {ino, mtimeMs, size} on each PoolEntry at open, and
re-stat in initLbug: unchanged → reuse; changed & idle → closeOne + reopen the
new file; changed while a query is in flight → serve the current handle (a
later idle initLbug reopens, since closing an in-use connection is a native
use-after-free). A stat failure (ENOENT during a full rebuild's unlink window)
is treated as unchanged so the reader keeps its valid open inode until the new
file appears. Mirrors the bridge cache's mtime-invalidation pattern.
Step 1 of docs/plans/2026-07-21-...-analyze-atomic-swap-invalidation. The
end-to-end reopen-on-swap path is exercised by the reader-during-rebuild
integration test in a later step.
* fix(analyze): publish a full rebuild via an atomic swap (POSIX)
The full-rebuild path wiped the live index (wipeLbugDbFiles(lbugPath)) and
rebuilt it in place, so a concurrent MCP reader that opened mid-build could
see an empty/half-loaded DB, and a crash between the wipe and the end-of-run
left the index destroyed (recoverable only by --force).
Build the fresh index at <lbugPath>.new and swap it over the live index in one
atomic rename at the end. All DB work flows through the singleton connection,
so only initLbug/wipeLbugDbFiles take the temp target; the close already
checkpoint-consolidates the build to a single file (verified: no residual
.wal/.shadow), so the rename publishes a complete index in one step. A reader
opening mid-build only ever sees the previous complete index; a reader holding
the old inode keeps a consistent stale snapshot until the pool re-opens onto
the new one (the pool staleness invalidation from the prior commit). On
failure the swap is skipped, leaving the previous index byte-for-byte intact.
POSIX only: the common CLI/serve-worker analyze paths skip the native close
(closeLbugBeforeExit, #2264) and leave the build handle open at swap time.
POSIX renames an open file cleanly; a same-process open handle blocks the
rename on Windows, so Windows keeps the current in-place behavior
(buildPath === lbugPath) until that is resolved. The Windows atomic swap and a
deterministic concurrent reader-during-rebuild test are deferred follow-ups.
Steps 2b + partial 3 of docs/plans/2026-07-21-...-analyze-atomic-swap-invalidation.
Integration test asserts the no-temp-leak + inode-swap invariants and the
crash-safety guarantee (a load failure leaves the live index untouched).
* test(analyze): end-to-end read-pool reopen after an atomic swap
Adds the deferred reader-during-rebuild / pool-reopen integration test:
analyze v1 -> read pool serves it -> rebuild with a renamed function (atomic
swap) -> the same repoId's initLbug detects the swapped inode and re-opens the
pool onto the new index. Asserts the pool sees the renamed function and NOT the
stale v1 name, exercising #1 (invalidation) and #2 (swap) together end to end.
* fix(lbug): bound pooled read queries with setQueryTimeout
The read pool relied only on a JS-side Promise.race (QUERY_TIMEOUT_MS) that
frees the waiter but leaves the native call running. Set the engine-level
setQueryTimeout on every pooled connection so a pathological query is bounded
at the source too.
* fix(lbug): name the held-open cause for WAL checkpoint failures (#2599)
A WAL-checkpoint IO error that also carries a busy/lock signal means another
handle (a gitnexus mcp server, or this process's own reader) holds the store
open, not a disk fault. Add isLbugCheckpointBusyError (reusing the tested
isDbBusyError keyword set) and, when the checkpoint driver exhausts its retry
budget on such an error, annotate the surfaced error with the actionable
held-open cause instead of a raw IO string.
Note: overlaps in-flight work on repro/issue-2599-windows-wal-checkpoint;
bundled here at the maintainer's request.
* feat(analyze): opt-in atomic incremental + best-effort Windows swap
Extends the atomic-swap publish (POSIX full rebuild) to two more cases:
- Windows: the swap now applies when a real close is safe to release the build
handle before the rename — i.e. non-pdg runs (windowsSwapOk excludes --pdg,
the #2264 destructor-crash case), forcing a real close on the swap path.
UNVERIFIED on Windows (no Windows runner here); --pdg and any failure fall
back to today's in-place behavior, so it can never corrupt.
- Incremental (opt-in, GITNEXUS_ATOMIC_INCREMENTAL=1): copies the live index
into the temp, applies the incremental delete/writeback to the copy, and
swaps at the end. Off by default because the whole-file copy negates
incremental's speed premise — kept behind a flag pending a benchmark. The
escalation valve also targets the temp so an escalated write stays atomic.
Integration test covers the opt-in incremental path end to end (no temp leak,
the incremental change is reflected after the swap).
* refactor(lbug): centralize the read-pool + bridge open-retry budgets
The lbug-config retry registry documented the open/handle-release/query-time
budgets but the read pool's LOCK_RETRY_* (pool-adapter) and the bridge's
LBUG_OPEN_RETRY_* (group/bridge-db) kept private copies that could drift. Move
both into the registry as exported constants (POOL_OPEN_LOCK_RETRY_*,
BRIDGE_OPEN_RETRY_*) and alias the local names to them — one tuning surface,
no behavior change.
* fix: address CI regressions from the bundled follow-ups
- setQueryTimeout: guard the call so test doubles that don't model the engine
method don't break connection creation.
- atomic swap: skip the rename when the build produced no DB at buildPath (an
empty repo / mocked pipeline) instead of throwing ENOENT.
- #2599: don't wrap the checkpoint error in the driver (it hid the IO signature
the CLI's --wal-checkpoint-threshold hint keys on); name the held-open cause
at the CLI instead, beside that hint, keeping the original error intact.
- retry consolidation: revert to documentation-only — moving the pool/bridge
budgets into lbug-config broke every explicit lbug-config test mock. The
registry now catalogues all budgets with their in-file locations.
- analyze-wal-checkpoint-failure test: block both lbug.wal.checkpoint and
lbug.new.wal.checkpoint, since a full rebuild now checkpoints the temp.
* fix(analyze): publish the swap before stamping meta; identity-gate the reader (#2614 F1)
Review found a HIGH regression: the full-rebuild wrote the freshness stamp
(saveMeta, indexedAt=T_new) BEFORE the atomic swap, so a concurrent MCP reader
that reinited in the saveMeta->swap window opened the OLD inode, recorded
observed=T_new, and then never reinited again (ensureInitialized returns early
on 'current') — serving the pre-rebuild graph indefinitely. The build-into-temp
change inverted the pre-PR invariant that 'meta shows T_new' implied 'lbugPath
holds T_new data'.
Two coordinated fixes:
- run-analyze: move the final saveMeta AFTER the swap, so meta.indexedAt only
becomes visible once lbugPath resolves to the new inode. Verified nothing in
the span reads on-disk meta and registerRepo writes only the registry.
Leaving the dirty flag set across the swap also improves crash-safety.
- local-backend: the reader staleness gate now also compares the lbug file
IDENTITY (ino/mtime/size), reiniting on an inode change even when
meta.indexedAt is unchanged. This closes the swap-window latch and covers the
in-place incremental case — and is what actually makes the pool's dbIdentity
net reachable for the MCP reader (the indexedAt gate otherwise bypassed it).
* fix(lbug/analyze): WAL-aware incremental, residual-sidecar reconcile, Windows opt-in, #2599 anchor (#2614 F2-F4)
Review remediations:
- F3: gate atomic incremental on a CLEAN live index (inspectLbugSidecars) — the
main-file-only copy would drop an orphan .wal's delta; fall back to in-place.
- F4: on the swap, MOVE a residual <buildPath>.wal/.shadow beside the published
index (not orphan it) so a swallowed final checkpoint's delta is replayed.
- F2: record identity on the shared read-only Database and warn when a cached
handle is reused after its on-disk index was rebuilt while another consumer
holds it (unreachable via MCP — one consumer per lbugPath; a complete fix
needs per-inode handles, documented).
- Windows swap: opt-in (GITNEXUS_ATOMIC_WINDOWS_SWAP=1), default off — the
forced real close re-bets an unproven #2264 assumption and can't be verified
without a Windows runner, so the default Windows path stays in-place.
- #2599: anchor isLbugCheckpointBusyError to real held-open wording instead of
isDbBusyError's bare .includes('lock') over a message that embeds the DB path
(a repo under blockchain-app misclassified a disk fault as held-open).
- Docs: corrected retry-catalogue budgets (linear, not exp) and the
checkedOut>0 bound comment (load-bounded, not IDLE_TIMEOUT_MS).
* test(analyze): cover the production close path in the atomic swap (#2614 F5)
Adds a full-rebuild swap test with skipNativeCloseOnExit:true — the close path
the CLI and serve-worker actually ship (build handle left open at swap time),
distinct from the default real-close the other swap tests exercise. Asserts the
POSIX swap still publishes a single consolidated lbug with no .new temp and no
orphan sidecar.
* test(analyze): give the follow-up git commits an inline identity (CI fix)
The end-to-end reopen and atomic-incremental tests' second commits used a bare
`git commit`, which fails on CI runners with no global git identity (empty
ident name). makeRepo's initial commit already passes -c user.name/-c
user.email inline; apply the same to the rename/change commits. No code change.
* fix(mcp): route reader reinit through initLbug's active-query guard (#2614 review)
Review found an active-query retirement race: LocalBackend.ensureInitialized
detected an identity/stamp change and called closeLbug(poolKey) DIRECTLY, but
closeOne closes the shared Database at refCount 0 regardless of checked-out
connections. So a reader detecting the new generation could close the Database
a concurrent query is still executing on — a native use-after-free. This
bypassed the checkedOut>0 guard that initLbug itself has.
Fix (delegate, not close directly): initLbug now returns whether it actually
rolled the pool over; ensureInitialized calls initLbug (which serves the
current handle while a query is in flight and reopens only when idle) instead
of closeLbug. The observed IDENTITY is advanced only when the pool actually
reopened — if a query was in flight, the identity stays divergent and the
reopen retries on a later idle check rather than latching on the old handle.
The observed STAMP advances regardless so a same-file stamp change can't loop.
Old generation now stays alive until its in-flight queries drain (lazy
rollover); new requests during the busy window share the old handle until the
pool goes idle, then reopen. No parallel open-both-generations, but no UAF and
no stale latch.
---------
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
|
||
|
Gergo Magyar
|
66f11badaf | style: wrap long filter predicate per prettier (PR autofix) | ||
|
Gergő Magyar
|
aaefbda226
|
Merge branch 'main' into fix/2604-rust-trait-object-dispatch | ||
|
Gergo Magyar
|
052319c9cc |
test(rust): add regression coverage for trait-object dispatch (#2604)
New minimal fixture (single trait + impl + &dyn Trait call site, no other same-named callers) proves the dyn-dispatch CALLS edge discriminates: fails against the pre-fix source (0 edges) and passes against the two preceding commits' fix (exactly 1 edge, verified via the CLI analyze pipeline against a standalone repo). The existing rust-abstract-dispatch fixture was NOT extended for this, deliberately: it already has other callers referencing the same method names (process()'s repo.find()/save()/count()), and an existing resolution fallback picks those up via simple-name matching regardless of receiver type — masking this specific defect in the in-process test-pipeline path. A dedicated, single-caller fixture keeps the regression test load-bearing. |
||
|
Claude
|
70e0a7766c |
fix(java): address #2561 review — inherited-dispatch test + bodied fail-safe
Two gitnexus-review-agent findings on PR #2602: - MEDIUM: the bodied-constant MRO-to-host-enum path (a qualified call to an inherited, non-overridden enum method) was claimed in a comment but never tested. Add EnumConst.A.log() -> EnumConst.log#0, exercising E$N's @reference.inherits MRO arm end to end. - LOW: `bodiedName ?? hostEnum` conflated "body-less" with "name synthesis failed on a bodied constant" (reachable only on malformed/error-recovery trees), silently binding an overriding constant's receiver to the host enum — a wrong edge instead of no edge. Switch to `isBodied ? bodiedName : hostEnum` so a bodied constant binds ONLY to its E$N class, mirroring the object_creation_expression branch's skip-on-synthesis-failure. Verified output-neutral on the well-formed bench corpus. Rebaseline the java scope-capture fingerprint (a822cef9 -> d04298a9): the bench corpus IS test/fixtures/lang-resolution, so the new dispatchInherited fixture method shifts it (+6 capture groups); the logic change contributes nothing (confirmed by isolating the fixture-only fingerprint). java.test.ts 242 passed; measure.mjs --check PASS (14 languages); tsc/prettier/eslint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |