mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-16 23:43:12 +00:00
34 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
MyShining
|
1147646518
|
feat(spring): model AOP transactions, caching, and security (#2783)
* feat(spring): model AOP advice and proxy behavior * fix(spring): address AOP review findings --------- Co-authored-by: Shining <xuenning@qiyi.com> |
||
|
azizur100389
|
454d383416
|
fix(ingestion): join multi-line closure bindings on initializer startLine (#2735) (#2762)
* fix(ingestion): join multi-line closure bindings on initializer startLine Graph-node captures sit on the outer binding wrapper while scope-resolution anchors on the inner callable; the line-only position join missed when those split across lines and fail-closed dropped the real CALLS edge (#2735). * fix(ingestion): unwrap Ruby call+block for multi-line lambda joins Cover Kotlin/Ruby/Dart multi-line closure CALLS in integration tests, and dig Ruby's call/block field so do-end bindings join on the block start line. * style(ingestion): format closure join changes * fix(ingestion): make closure position join language agnostic |
||
|
MyShining
|
de84ad6297
|
feat(spring): index @Bean factories and @Resource injection (#2740)
* feat(spring): index Bean factories and Resource injection * fix(spring): address Bean and Resource review findings * refactor(lbug): keep relation pair parsing in router * test(lbug): preserve schema exports in WAL mocks * test(cache): align schema bump pin --------- Co-authored-by: Shining <xuenning@qiyi.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
27ab37c432
|
feat(resolution): type receiver chains from AST structure across all 14 languages (#2708) + epistemic lower-bound (#2744) (#2747) | ||
|
Gergő Magyar
|
9c24e3459e
|
fix(rust): qualify items by their enclosing mod chain so same-named ones stay distinct (#2742) (#2745)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(rust): let the qualified-call filter see inline modules The negative filter added in #2741 builds its set of known module names from FILE PATHS, so an inline `mod x { … }` — which appears in no path — was absent from it. Every module-qualified call into an inline module was therefore rejected before any candidate channel ran, which is a hole in that optimisation rather than in the resolution logic it guards. The per-pass index now unions the file-derived names with inline module names taken from the scope model: a `mod` declaration binds a `Namespace` def locally in the declaring scope, and that binding is the only place an inline module's name exists. Collected in the same walk that already builds the module → scope map, so it costs no extra pass. Found while fixing #2742, where a correctly resolved call into `mod inner { … }` still could not reach its target. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(scope-resolution): try the namespace-prefixed node key before the bare one `resolveDefGraphId` looked up the plain `qualifiedName` key first and only retried with the `namespacePrefix`-qualified key afterwards. For the defs that carry a prefix the qualified name is a bare TAIL, so the plain key happily matched a same-named item at a different namespace depth in the same file and returned it before the more specific retry was ever reached. The namespace-prefixed key is strictly the more specific of the two, so it is now tried first. Where no such node exists the lookup falls through to exactly the previous order, which keeps the #1982 behaviour this retry was added for. Without this, a call into `mod inner { fn dispatch }` resolved to the correct definition and then mapped it onto the crate-root `fn dispatch` node — the self-loop #2742 describes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): qualify items by their enclosing mod chain so same-named ones stay distinct (#2742) Node identity is `<label>:<file>:<qualifiedName>` and carried no module path, so an inline `mod inner { fn dispatch }` and a crate-root `fn dispatch` in the same file collapsed onto `Function:<file>:dispatch`, first-wins. Resolution already picked the right definition — the target simply was not representable, so a correct resolution still rendered as a self-loop and `impact` reported the real callee as unreached. The mechanism already existed: `qualifyRustImplTargetByModScope` has walked `mod_item` ancestors for impl targets since #1982. Generalised to `qualifyByEnclosingModScope` and applied to free items, so `mod inner { fn dispatch }` becomes `Function:<file>:inner.dispatch`. Keyed purely on the `mod_item` node type, exactly as the impl qualifier already was, so it is a no-op for every language whose grammar has no such node. Two constraints found by tests rather than by reading, both now encoded: - The helper normalised `::` to `.` unconditionally. With no enclosing `mod` that rewrote a top-level `impl a::Inner` from `a::Inner` to `a.Inner` and moved its node id away from the one the HAS_METHOD owner edge emits, breaking the #1975 scoped-impl ownership. It now returns raw text untouched when there are no mod segments, which also makes the change strictly additive for every id that has no enclosing module. - Qualification is scoped to items with no enclosing class/impl. A method already carries its owner's name, and that owner's id is mod-scoped by the impl qualifier, so qualifying the method again breaks the same byte-for-byte agreement. Same-named methods on same-named types in sibling modules therefore still collapse — a narrower residual than the free-item case fixed here, and one belonging to the owner edge rather than to this path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(storage): bump schema versions for the mod-qualified Rust node ids (#2742) `INCREMENTAL_SCHEMA_VERSION` 24 -> 25 and `SCHEMA_BUMP` 31 -> 32. Node ids change for every Rust item inside any `mod` block, and `#[cfg(test)] mod tests` makes that close to every Rust repository. A pre-v25 index therefore holds ids an incremental top-up cannot reconcile — the old nodes would simply be stranded — so the reuse gate has to force a full re-analyze. The qualified name is computed in the parse worker, so a warm parse cache would likewise replay the old unqualified ids and keep the collapse. This branch originally claimed v24; #2708 took that number and merged first, so it is renumbered to v25 here. That is exactly the collision the v29 note in parse-cache.ts warns about, and re-checking against origin/main at rebase time rather than at branch time is what caught it. #2708 did not touch `SCHEMA_BUMP`, so 32 is free. The version-pin test moves with the bump by design, including the new pre-v25 row in the reuse-gate table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): stop mod-qualifying container ids while their owner edges stay bare (#2745 review) #2742 re-keyed Rust node ids by the enclosing `mod` chain. The mint moved; the owner-edge anchor did not. `findEnclosingClassInfo` mints a member's owner id from the container's BARE `nameNode.text` and only follows a qualified shape when the provider sets `classExtractor.qualifiedNodeId`, which Rust does not. So every `struct` / `trait` / `enum` / `impl` declared directly inside a `mod` got a node id that none of its member edges pointed at. Five lines of idiomatic Rust were enough: pub mod engine { pub struct Config { pub retries: usize } } NODE Struct:src/lib.rs:engine.Config DANGLING HAS_PROPERTY Struct:src/lib.rs:Config -> Property:src/lib.rs:Config.retries The rows are discarded by the IGNORE_ERRORS COPY retry, so the struct silently lost every field. A trait impl inside a `mod` additionally dropped its METHOD_IMPLEMENTS edge outright. The same gap put `impl a::Inner` inside a `mod` back on the #1975 rake that `qualifyByEnclosingModScope`'s own docblock warns about. The impl-target branch deliberately fires only for an UNSCOPED `type_identifier`; the new gate had no such restriction and picked up the scoped targets that branch had just excluded, minting `Impl:<file>:outer.a.Inner` against an anchor still reading `Impl:<file>🅰️:Inner`. The member side was already excluded via `!enclosingClassInfo`. This adds the owner side, gated on `MEMBER_OWNER_NODE_TYPES` — derived from `CLASS_CONTAINER_TYPES`, which is already the single source of "this node type owns member edges" and already carries an INVARIANT note binding it to `CONTAINER_TYPE_TO_LABEL`. A language adding a container therefore cannot gain a mismatched id shape here without also failing that invariant. Keyed purely on tree-sitter node types, so no language name enters shared ingestion. `union_item` is listed too: its fields are captured as Property but it is not a recognized owner, so they carry no HAS_PROPERTY edge and cannot dangle — it is here so a union's id keeps the same shape as the struct beside it. Containers still collapse across sibling modules, exactly as before this fix. That residual belongs to the owner edge, and is not worked around here. Regression tests use the UNFILTERED `findDanglingEdges(result)`. Every other dangling assertion in `rust.test.ts` passes `['HAS_METHOD']`, which is precisely why the HAS_PROPERTY breakage shipped with a green suite. They assert the NODE id rather than only the edge's anchor, because the anchor was already bare while the bug was live — an edge-only assertion passes in both builds. All four fail when the new gate clause alone is reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * fix(rust): resolve modules nested inside an inline mod (#2745 review) The #2730 self-loop survived one `mod` deeper: pub mod outer { pub mod tools { pub fn dispatch() {} } pub fn dispatch() { tools::dispatch(); } } CALLS outer.dispatch -> outer.dispatch <- the #2730 symptom NODE outer.tools.dispatch <- correct target, unlinked Two gates were blind to a nested inline module, so the hook refused and the shared lexical tier bound the call to the enclosing same-name `dispatch`: `knownModuleNames` was collected by walking `moduleScopeByFile`, which maps a file to its ROOT `Module` scope only. A `mod` nested inside an inline `mod` binds in the parent module's scope, so the walk saw depth-1 inline modules and missed every nested one — `tools` never entered the set and the negative filter rejected the qualifier before any candidate ran. `declaresSubmodule` had the same root-only assumption, so even with the name known the candidate `outer::tools` was never yielded. Both now read the def index. Names come from every `Namespace` def; inline module PATHS are derived from the members' `namespacePrefix` rather than from the `mod` defs, because a `mod` def carries no nesting information of its own — inside `mod outer { mod tools { … } }` the inner def is `qualifiedName: 'tools'` with NO `namespacePrefix`, while every def within it is stamped `outer.tools`. A `Namespace` scope also owns its OWN def rather than its children's, so the scope tree cannot answer this either: the `mod outer` scope lists `outer`, never `tools`. Restricted to non-empty prefixes, so this stays a DECLARATION check. Including file-derived modules would let an undeclared or `cfg`-gated file on disk outrank a real `use` binding — the regression #2741's review already fixed once. File-backed submodules therefore keep going through the binding check. A module with no defs at all is absent from the set, which is harmless: it has no member for a qualified call to resolve to. Cost is one pass over an already-resident def index, memoized per resolution pass on the existing WeakMap — the same order of work as the binding walk it replaces, and it subsumes it. `isLocalNamespaceBinding` was going to single-source the duplicated "locally declared submodule" predicate the review flagged; deriving paths from members removed the second copy outright instead. Regression fixture covers depth 2 and depth 3, so the fix is depth-agnostic rather than depth-2 special-cased. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * fix(rust): let an imported type outrank a same-named module (#2745 review) Widening the negative filter to inline `mod` names let a type-qualified call through whenever a module happened to share the type's name. The crate-root-relative candidate then captured it: // src/lib.rs pub mod Buffer { pub fn with_capacity() -> usize { 111 } } // src/b.rs use crate::c::Buffer; // the real target lives in c.rs pub fn call() -> usize { Buffer::with_capacity() } base: (no CALLS edge — unresolved) PR: CALLS b::call -> Function:src/lib.rs:Buffer.with_capacity <- fabricated `ids.ts` states the doctrine this broke: a missing edge is the correct failure direction for a graph whose consumers include `impact`; a fabricated caller is not. The base produced the missing edge and the PR produced the fabricated one. That third candidate is the loosest of the three — a guess at a crate-root-relative path the caller never wrote, kept for 2015-edition style. In Rust 2018 a bare first segment resolves in the CALLER's module, so a local binding for that segment settles the question: it is now skipped when the head names anything non-module in the caller's own module. Candidates 1 and 2 are untouched, and they run first, so the legitimate `use crate::tools;` path is unaffected. The binding lookup goes through `lookupBindingsAt`. A first attempt read `Scope.bindings` directly and the guard never fired: a `use` binding is finalize OUTPUT and absent from the scope's own local table, which is exactly the imported-type case being guarded. Contract I8 in `contract/scope-resolver.ts` requires that channel anyway. The regression test asserts the forbidden TARGET rather than an empty edge set, and separately asserts the module member still exists as a node — otherwise the test would pass just as well if the call went unresolved for some unrelated reason, or if the module node disappeared entirely. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * fix(scope-resolution): try the namespace-prefixed name on the TAGGED keys too (#2745 review) `resolveDefGraphId` gained a namespace-prefixed retry for the plain qualified key in this PR, but the five tagged keys above it — template constraints, parameter types, parameter shape, arity, template arguments — kept composing from the bare `qualifiedName`. For a namespace- or `mod`-qualified def those keys are simply dead: `node-lookup.ts` registers them under the QUALIFIED name (`inner.dispatch#0`) while this side built `dispatch#0`. The keys exist to separate overloads, so a mod-scoped overload set was relying on whichever later key happened to catch it. Verified as a miss rather than a mis-hit before changing anything — an end-to-end run with a crate-root decoy of the same name and arity binds correctly — so this is hygiene, not a live bug. Worth doing while the code is open rather than leaving five keys dead and the behaviour dependent on fallback order. Both name forms now go through one `lookupTagged` helper, most specific first, so a sixth tagged key cannot be added with the bare form only. That also removes the five hand-repeated `qualifiedKey(...)` / `nodeLookup.get(...)` pairs. Also pins the C++ `EXTENDS` retarget this PR's reorder produces. `cpp-two-phase-dependent-base-cross-ns-deep` declares a global `Inner` decoy alongside `ns:🅰️🅱️:Inner`; the base's `qualifiedName` is a bare `Inner` with the path on `namespacePrefix`, so only the prefixed key separates them, and only if it runs first. The improvement was riding unasserted in a Rust-scoped PR. The captures golden covers every `rust-*` fixture, so the three fixtures added by this review series drift it; regenerated with UPDATE_GOLDEN=1. Verified: 785 tests across cpp / csharp / rust resolvers and the callable-id-lockstep unit test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * fix(rust): keep a mod declared inside a fn from hoisting above the callable (#2745 review) `fn wrapper() { mod helper { fn dispatch } }` minted `Function:<file>:helper.wrapper.dispatch@2:8` — the mod segment composed OUTSIDE the enclosing-callable prefix, inverting the real nesting. Nothing dangled: the `@line:col` suffix already makes a function-local callable's id unique, which is also why the mod segment adds no identity in this position. The path simply read as a lie about the source. It is now skipped rather than reordered — interleaving two qualifier passes to fix the order would be real machinery for a shape whose ids are already unique. Also folds in the three documentation and structure findings from the same review: - The 4-clause gate is extracted to a named `qualifiesByEnclosingModScope`, matching the two conditions directly above it in the same function, which were already named consts. - `qualifyByEnclosingModScope`'s docblock documented only the impl-target contract even though the generalized name has had a second, looser caller since #2742. It now states both, and says which gate belongs to which — that gap is what let the #1975 scoped-impl regression through in the first place. - The "cheap rejection BEFORE any index work" comment was no longer true: `passIndexFor` walks the def index on its first call in a pass. Corrected rather than left to mislead the next reader into thinking the filter is free. What it still buys — skipping the per-site candidate search, the part that scales with the workspace — is stated instead. Verified: 279 tests across the Rust resolver suite and the Rust scope-resolution unit tests. Captures golden regenerated for the extended fixture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * test(storage): move main's SCHEMA_BUMP pin to 32 for the mod-qualified ids (#2745 review) `#2736` added a pin asserting `SCHEMA_BUMP === 31` on main, which arrived on this branch through the merge of main while `0062a5c2` had already bumped the constant to 32. Neither side conflicted textually — the pin and the constant live in different files — so the merge was clean and the test failed instead. That is the pin working as designed: it exists so a bump cannot ride along unnoticed, and this is the fifth time a SCHEMA_BUMP collision has been caught by a guard rather than by review. Updated to 32 with the reason recorded inline. `INCREMENTAL_SCHEMA_VERSION` needs no second bump: 25 was introduced by this unmerged branch, so no released index carries it, and its own pin in `call-summary-schema-version.test.ts` is already consistent. Verified: 119 tests across the parse-cache, schema-version, incremental-orchestration and the two identity suites that arrived with the merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z * test(bench): rebaseline the Rust capture fingerprint for the three new fixtures (#2745 review) CI caught what I missed: [scope-capture --check] FAIL: rust: capture fingerprint drift (got 05acbaca..., expected 90fda086...) fixture_count 202 `bench/scope-capture` fingerprints the whole `rust-*` fixture corpus, so the three fixtures added by this review series drift it. I rebaselined the `rust-captures-golden` snapshot and stopped there — a new fixture is a call site of BOTH, and updating only one is how this reached CI red. This is the same class as PR #2743's headline finding, from the other direction: an id-shape change makes every synthetic corpus a call site, and the author fixed the unit-test fixture and missed the bench. Here it is a fixture-count change rather than an id-shape change, and the review that flagged the #2743 lead as "REFUTED, bench/ has no Rust node-id corpus" was right about node ids and wrong about the corpus fingerprint. Noted for the next author in the baseline entry itself. Verified as pure corpus growth rather than a capture-logic shift: removing ONLY the three new fixture directories and re-running reproduces the prior fingerprint exactly (196 fixtures, capture_groups_fp 3432), and restoring them gives the new one (202, 3556). `emitRustScopeCaptures` is untouched by this series. Scaling 1.022 local / 1.057 CI, well inside the 1.5 budget. `bench/python-scope` globs `python-*` only and is unaffected; no other bench walks the Rust corpus. `--check` now PASSes for all 15 languages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
df06529950
|
fix(rust): resolve module-qualified calls against the module tree (#2730) (#2741)
* fix(rust): resolve module-qualified calls against the module tree (#2730) A Rust call written with a path (`tools::dispatch(..)`) was captured with only its tail identifier, making it indistinguishable from a bare `dispatch(..)`. The scope-chain walk then resolved the bare name lexically and bound it to whatever `dispatch` was nearest — which, for the common wrapper idiom fn dispatch(..) -> ToolOutcome { tools::dispatch(..) } is the wrapper itself. The graph gained a self-loop, the real cross-module edge never existed, and `impact` reported the callee as unreached: the issue's repository showed its central tool dispatcher as `risk: LOW` with 0 affected processes and both "callers" being `#[cfg(test)]` functions, while still labelling the result `epistemic: "exact"`. Resolve paths the way rustc does, over the module tree rather than the filesystem: - `mod_item` now emits `@declaration.namespace`, so a Rust module is a named definition rather than an anonymous scope region. This mirrors the existing C++ `namespace_definition` capture and lets the shared `tagNamespacePrefixes` pass stamp members with their enclosing module path — that pass needed no changes to start working for Rust. - `module-path.ts` reconstructs the other half of the tree: crate roots are directories holding `main.rs`/`lib.rs`, and a file's module path is its location below that root. A definition's module is its file's module plus any enclosing `mod` blocks. - `crate::`, `self::` and `super::` are prefix transforms on the calling module, not reasons to stop resolving. - The final path segment is looked up as a member of the resolved module, including members it only re-exports. A `pub use` creates no binding on the re-exporting module's own scope, so re-exports are followed through that module's import edges. Resolution runs ahead of the implicit-`this` and scope-chain tiers, so an explicit path outranks a lexical shadow, and returns undefined on an unknown module, a missing member or a tie — leaving the existing chain untouched. The new `ScopeResolver.resolveQualifiedFreeCall` hook is optional and unset for every other language, so this is additive. Fixes the reported case (direct callers 2 -> 3, impacted 2 -> 6, the Agent module now visible) plus multi-segment paths, `super::` paths and `pub use` facades, each of which previously produced a wrong edge. Known limitation, pre-existing and unchanged by this commit: an inline `mod inner { fn dispatch }` and a crate-root `fn dispatch` in the same file collapse to one graph node, because node identity is `<file>:<qualifiedName>` and does not carry the module path. That is a separate defect requiring module-path-qualified node ids and an incremental-schema migration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * test(rust): rebaseline the scope-capture fingerprint for the module-tree captures `mod_item` now emits `@declaration.namespace` and scoped call sites carry `@reference.qualified-name`. Both are additive, so every bench fixture holding a `mod` block or a `Foo::bar()` call gains capture groups, and the corpus grew by the three `rust-2730-*` fixtures. Only the Rust fingerprint moves. The other 14 languages are byte-identical, which is the intended blast radius for a language-local capture change. Scaling stays linear at 1.043, well inside the 1.5 budget. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): carry crate identity in qualified module paths (#2741 review H1) A module was identified by its path segments below a crate root, so `crates/alpha/src/tools.rs` and `crates/beta/src/tools.rs` were the same module. A cargo workspace routinely gives several members the same internal module name — `util`, `error`, `config`, `types` are near-universal — and that made qualified resolution do one of two wrong things: - where only one member defined the called name, the call bound ACROSS crates; - where both defined it, the lookup saw two candidates, refused, and handed the site back to the lexical walk that emits the same-name self-loop. The fix for #2730 therefore switched itself off in exactly the workspace layouts it was written for, and #2730's own reported reproduction repository is multi-crate. A module is now `{ crateRoot, segments }` and `sameModule` compares both. Rust has no implicit cross-crate paths — reaching another crate requires naming it — so two modules in different crates are never the same module. Anchored paths (`crate::`, `self::`, `super::`) resolve inside the caller's own crate and inherit its root. Covered by a two-member workspace fixture where both crates define `tools::dispatch` behind a same-name wrapper, plus unit tests for the path arithmetic itself, including the branches no fixture reaches (a file under no crate root, a `super::` chain walking above the crate root). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): count only module members when resolving a qualified call (#2741 review H3) Module membership was inferred from the file path alone, so any callable in the right file counted as a member of the module. A `fn` nested inside another `fn` has the same `filePath`, the same bare `qualifiedName` and no owner, making it indistinguishable from a module-level item: pub fn dispatch() -> usize { 3 } // the real member pub fn wrapper() -> usize { fn dispatch() -> usize { 99 } // counted as a second member dispatch() } Two candidates tie, the lookup refuses, and the call falls back to the lexical walk that emits the same-name self-loop — so an unrelated local helper anywhere in a module silently reinstated #2730 for every qualified call into it. The scope model already draws the line exactly: a module-level item is bound with `origin: 'local'` in its module's own scope, a function-local item binds in the enclosing Block, and an `impl`/trait method binds in the Class scope. Membership is now that binding lookup rather than a path comparison. Inline-`mod` members bind in their Namespace scope rather than the file's Module scope, and reaching it would mean walking every child scope — faulting them back in from disk on the out-of-core path. They keep being identified by the `namespacePrefix` the shared tagging pass stamps on them, which a file-module member never carries. The documented residual is a `fn` nested inside a `fn` inside an inline `mod`, which inherits that prefix; that is strictly smaller than before and costs a refusal, never a wrong edge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): require a use-binding to name a module, not a type (#2741 review H2) Import resolution deliberately strips a trailing symbol segment when probing for a file — "the last segment might be a symbol (function, struct, etc.), not a module. Strip it and try again" (import-resolvers/rust.ts). So `use crate::client::ClientBuilder;` also resolves to `client/mod.rs`. The qualified-call resolver took that at face value and treated the imported TYPE as the module `client`. Rust impl methods carry a bare `qualifiedName`, so `ClientBuilder::new()` was then looked up among `client`'s module members and bound to an unrelated module-level `new` — turning an unresolved site into a false edge, which the module's own contract calls the worse outcome. A binding now has to name the module it resolved to. The edge's `targetExportedName` is the tail of the written path, so comparing it against the resolved module's own tail separates the cases exactly: use crate::tools; tail `tools` module ['tools'] accept use crate:🅰️:b as tools; tail `b` module ['a','b'] accept use crate::tools::{self, Ctx}; tail `tools` module ['tools'] accept use crate::client::ClientBuilder; tail `ClientBuilder` module ['client'] reject Covered by a fixture where `client/mod.rs` deliberately holds both `impl ClientBuilder { fn new }` and a module-level `fn new`, so a regression re-binds to the wrong one, plus a control asserting a genuine `client::new()` module qualifier still resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): give src/bin targets their own crate root (#2741 review) Cargo auto-discovers a binary target for every `src/bin/<name>.rs`. Each is a separate crate with its own `crate::` root, and its submodules live under `src/bin/<name>/`. Only `main.rs` and `lib.rs` established a crate root, so those entry files were folded into the surrounding library and given the invented module path `bin::<name>`. That made `crate::helper()` inside a binary resolve into the LIBRARY's `helper` — and unlike the other findings in this review, this one downgraded an edge the lexical walk had previously resolved correctly, so it made existing output worse rather than merely failing to improve it. `src/bin/<name>.rs` is now its own crate root (as is the `src/bin/<name>/main.rs` directory form), so a binary's modules and the library's modules of the same name are no longer the same module. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): only try a submodule candidate the caller actually declares (#2741 review) The first candidate module was `callerModule ++ qualifier`, yielded before the `use` channel and never checked against anything. That let file layout outrank a real import: with `use crate::b;` in `src/a/mod.rs` and an undeclared — or `cfg`-gated — `src/a/b.rs` present on disk, `b::f()` bound to the sibling file, where rustc resolves it to `crate::b`. A `mod` declaration, inline or file-backed, emits a `Namespace` def bound locally in the declaring scope, so the candidate is now gated on that binding rather than assumed. When the caller does not declare the submodule the candidate is skipped and the `use` and crate-root channels still run, so this only removes guesses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): follow only real re-exports, and refuse on an ambiguous one (#2741 review) Two problems in the re-export channel. A private `use` was followed as though it re-exported. `use crate::tools::helper;` makes `helper` visible INSIDE the module; it does not put it on the module's public surface, so `facade::helper()` does not compile. Only `pub use` does, and finalize already distinguishes them — `reexport` for `pub use`, `named` for a private one. The `alias` kind is now accepted alongside `reexport`, because `pub use x::y as name` is a re-export that was previously ignored entirely. The lookup also took the first matching edge in file-iteration order, which is parse-pool order. Two `cfg`-exclusive facades re-exporting the same name are indistinguishable at this layer, so picking one baked a coin flip into the graph. It now refuses on a genuine tie, consistent with how member lookup already behaves. The pre-existing limitation that only FILE modules are reachable — a `pub use` inside an inline `mod facade { … }` has no `moduleScopeByFile` entry — is now stated in the code. Reaching those would mean walking every child scope and faulting the scope tree back in from disk, which is the cost that index exists to avoid; a miss falls through to the unchanged chain rather than guessing. The regression test deliberately makes the re-exported name globally ambiguous. Without that, the pre-existing unique-global free-call fallback resolves the call on its own and the assertion passes whatever this channel does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * perf(rust): stop type-qualified calls paying for module resolution (#2741 review) The capture carrying `rawQualifiedName` matches every `scoped_identifier` callee, so this hook was reached by `Vec::new()`, `String::from()`, `Self::method()` and every other type-qualified call — the overwhelming majority of `::` calls in real Rust, none of which name a module. Each one ran the full candidate search before returning undefined, and every candidate that missed then walked all of `workspaceIndex.moduleScopeByFile`. Total cost grew as `qualified-call-sites x files`; two independent measurements put per-site cost at 0.117 -> 0.428 ms across 301 -> 1201 files, i.e. linear in workspace size. Two changes: - The module index now carries a flat set of every module segment name in the workspace, and a qualifier whose head matches none of them is rejected before any candidate work. Measured at 0.02 us per rejected call and flat in file count (500 -> 8000 files), against a previously linear per-site cost. - Module scopes are indexed by module identity once per pass rather than rediscovered by scanning every file per candidate. On the out-of-core scope index that scan was worse than CPU: `moduleScopeByFile` fetches through `scopeTree.getScope`, so a full sweep could fault every module scope back in from disk — the pattern `workspace-index.ts` added `exportedCallableByName` to avoid. Given the #2649 and #1871 history this mattered before merge. The captures golden is regenerated for the fixture files added earlier in this series; `emitRustScopeCaptures` itself is unchanged by this commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(storage): bump schema versions so the #2730 fix reaches existing indexes Neither invalidation constant was bumped, so the fix did not reach the users who reported the bug. `INCREMENTAL_SCHEMA_VERSION` 22 -> 23. The incremental write set only covers CHANGED files, so a top-up against a pre-v23 index keeps the wrong self-loop — and keeps reporting the callee as unreached — for every unchanged Rust file. The constant's own doc block states this rule, and the precedent is exact: v11 is the same file (`rust/query.ts`) gaining a capture that changes CALLS edges, with the same "force a full re-analyze" contract, and v12 is a second Rust instance. `SCHEMA_BUMP` 30 -> 31. `@declaration.namespace` and `@reference.qualified-name` are parse-time captures, so a warm parse cache replays the old capture set verbatim: `rawQualifiedName` comes back undefined and no Namespace def exists to hang a module prefix on, turning the entire resolution tier into a no-op on unchanged files. `PARSE_CACHE_VERSION` folds in the package version, so a tagged release would have invalidated eventually — but source, dev and CI builds at the same version would not, and the v29 note already warns that relying on someone else's bump is how a change ships with no invalidation at all. Re-checked against origin/main at commit time, as that note instructs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(scope-resolution): let a language opt out of the already-namespaced guard (#2741 review) `tagNamespacePrefixes` skips a def whose `qualifiedName` already equals, or is prefixed by, its enclosing namespace path. That is right for C++ and C#, where the qualified name genuinely carries the namespace. Rust qualified names never do, so the guard fired on a coincidence: in `mod a { pub fn a() }` the member's name equals its module's name, the prefix was skipped, and `moduleOfDef` then reported the member as belonging to the PARENT module. `crate:🅰️:a()` refused, and the def became indistinguishable from a crate-root `fn a` for the module matcher. The guard is now conditional on a `qualifiedNamesCarryNamespace` option that defaults to the existing behaviour, and Rust opts out. The shared pass stays language-neutral — the decision lives with the provider that knows what its own qualified names contain. C++ and C# resolver suites pass unchanged alongside the Rust ones (600 tests). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * fix(rust): refuse a leading :: path instead of reading it as relative (#2741 review) A leading `::` anchors at the extern prelude: `::tools::dispatch()` names the CRATE `tools`, not a module of the current one. The path split filtered the empty leading segment away, which silently reinterpreted the path as relative and let it resolve against a local module that happens to share the name. Extern crates are outside the workspace module tree, so the qualified tier now refuses and leaves the site to the unchanged chain. The regression test asserts the tier does not bind into the local `tools` module, rather than asserting no edge at all: the lexical tier still resolves the bare tail on its own, and that behaviour is not what this change governs. Asserting an empty edge list would have been testing a different tier. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * refactor(rust): reuse the canonical callable predicate and drop dead re-exports (#2741 review) `CALLABLE_TYPES` was a local copy of the set behind `isOverloadableCallable` in `utils/callable-labels.ts`. Two copies of the same set drift: extending the canonical one with a new callable kind would silently leave qualified calls of that kind unresolved here, with nothing to catch it. Use the shared predicate. The trailing `export { moduleOfFile, moduleOfDef }` and `export type { ScopeResolutionIndexes }` were commented as being "for the resolver's unit tests". No test imports them: the only importer of this module anywhere in src or test is `rust/scope-resolver.ts`, which takes just `resolveRustQualifiedFreeCall`. Both functions are already exported from `module-path.ts` (where the new unit tests take them from), and `ScopeResolutionIndexes` is canonically exported from `model/scope-resolution-indexes.ts`. Removed rather than left as surface that implies a contract it does not have. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * test(rust): rebaseline the scope-capture fingerprint with the correct prior hash The rebaseline note added with the original fix cited `Prior 655aed01…`, which was two rebaselines stale — it predates both #2604 and #2714. The true pre-PR value on the base commit is `7f1240b3…`. CI could not catch it: the gate compares the live fingerprint against the stored one and never reads the prose, so the audit chain these notes exist to provide was broken with nothing to flag it. The note now carries the correct prior value, and the fingerprint is regenerated for the fixtures this review series added. Scaling 1.061, well inside the 1.5 budget; fixture_count 196; the other 14 languages remain byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV * test: move the schema-version pin to 23 `call-summary-schema-version.test.ts` asserts the exact value of `INCREMENTAL_SCHEMA_VERSION` and enumerates which stamped versions the incremental reuse gate accepts. It moves with every bump by design — that pin is what stops an id- or edge-changing commit shipping without invalidation. Updated for the bump to 23, with the pre-v23 case added to the reuse-gate table: a v22 index predates Rust module-qualified call resolution, so every unchanged Rust file would keep the same-name self-loop and keep reporting the real callee as unreached. Caught by CI rather than locally, because the earlier sweeps in this series covered `test/integration/resolvers/` and `test/unit/scope-resolution/` only — the pin lives outside both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
89ea233e10
|
fix(js): index CommonJS exports.foo = function () {} exports (#2723) (#2729)
* fix(js): index CommonJS `exports.foo = function () {}` exports (#2723)
Functions assigned to an `exports` / `module.exports` property were not
indexed at all. On a CommonJS codebase — the dominant pre-ESM Node style
(Express, Firebase Functions) — the graph held every internal helper and
missed the entire public API: `impact({target: 'areVariablesValid'})`
answered `Target not found` for the one symbol whose blast radius mattered.
The gap had two halves, and fixing either alone leaves the feature broken:
1. `tree-sitter-queries.ts` carried `@definition.function` rules for every
declaration form and every variable-binding closure form, but none for
`assignment_expression` — so no `Function` node was created.
2. The scope-resolution queries (`languages/{javascript,typescript}/query.ts`)
likewise had no `@declaration.function` for the shape. Adding only (1)
moves `impact` from "not found" to "found, zero callers", because call
resolution reaches a definition through the scope declaration, not
through the graph node.
Both layers now carry the rule, for `function` / `async function` / arrow /
async arrow / generator right-hand sides, in JavaScript and TypeScript. The
receiver is pinned to `exports` / `module.exports` with `#eq?` predicates:
the general `X.foo = function () {}` shape also covers `Foo.prototype.bar`
and `this.handler`, which are member constructs with their own ownership
questions, and a broader rule would emit ownerless top-level Functions for
them. The declaration binds the bare property name into the module scope,
which is what importers see, so `const { foo } = require('./m')` matches by
name and a namespace `m.foo()` walks the module's defs.
Verified end to end: node emission for every listed form plus TS parity, and
CALLS edges for same-file `exports.foo()`, cross-file namespace `m.foo()`,
and cross-file destructured `require()`. The generator call-resolution case
was confirmed to fail against the pre-fix build before the rule landed.
Fixes #2723
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(js): stop the CJS export rule from shadowing a declared function (#2723)
Review of the previous commit caught a regression it introduced. The CJS
`@declaration.function` rules bind the exported property name into the module
scope — which is the point, since that is what importers resolve against. But
when the file ALSO declares that name lexically:
function dup(v) { return v; }
exports.dup = function (v) { return !v; };
function callIt(v) { return dup(v); }
the module scope ends up holding two declarations named `dup`, the name is
ambiguous, and the resolver drops `callIt -> dup` entirely — an edge that
resolved fine before #2723. Confirmed by rebuilding both states: present at
|
||
|
Gergő Magyar
|
0ce7880290
|
fix(scope-resolution): a closure binding is a call SOURCE in every language, and function-local values carry their own identity (closes #2699) (#2718)
* test(scope-resolution): audit the consumers of file-scoped node ids (#2699 part A)
#2699 item 4 — "audit consumers that assume file-scoped ids" — after #2695/#2714
gave function-local CALLABLES position-bearing ids. Tests and findings only; no
production change. That split is deliberate: `impact` reports
`resolveDefGraphId` at CRITICAL with 23 DIRECT dependents across 7 modules
(every language MRO builder, both Spring attachers, C++ member lookup,
tryEmitEdge, emitReferencesViaLookup, buildGraphTargetIndex, emitFreeCallFallback,
emitReceiverBoundCalls, preEmitInheritanceEdges, emitDetectedInterfaceImplementations,
phpEmitUnresolvedReceiverEdges, emitRubyMixinEdges, emitRustTraitImplEdges,
emitDartHeritageEdges), so changing that key chain is its own change, not a
rider on an audit.
A2 — detect_changes: CONCERN RESOLVED, now pinned. The worry was that an id
containing `@row:col` re-keys whenever a declaration MOVES, making every edit
look like symbol churn. It cannot: `local-backend.ts` maps diff hunks to
symbols by LINE-RANGE OVERLAP (`n.startLine`/`n.endLine`) and merely REPORTS
`n.id`. Node identity never participates in the match. New structural test
asserts the WHERE clause never gains `n.id =` or `n.id IN`, keeps the one
legitimate id-shaped predicate (the `BasicBlock:` prefix exclusion, #2082 U7),
and confirms the id is returned rather than matched. Structural in the same
idiom as `detect-changes-worktree.test.ts`, and labelled as not proving runtime
behaviour.
A1 — ANSWERED, and the answer is that #2699 is NOT fully closed by items 1-3.
The fail-closed guard is gated on `isOverloadableCallable`
(Function | Method | Constructor), so a function-local VALUE never reaches it.
Measured on a fixture: a top-level `const handler` and a function-local
`const handler` still produce ONE node, `Const:v.ts:handler`. That is the
residual half of the issue's original complaint. Pinned as a KNOWN LIMIT with
its reason (widening identity to values re-keys ~14,700 build-time nodes to
change ~800 persisted ones — the decision recorded in `parse-worker.ts`), and
deliberately NOT fixed here.
A3 — id-persisting consumers, classified:
- detect_changes ................ SAFE (position-keyed; pinned by A2)
- MCP impact/context/trace ...... SAFE (resolve by name/uid at query time)
- bench fingerprints ............ SAFE (digest capture shape, not node ids)
- rust-captures golden .......... SAFE (digests captures, not ids)
- cfg pipeline-pdg snapshot ..... AT RISK by design — pins exact edge ids, so
it trips whenever attribution changes. That is the gate working; #2714
already exercised it.
- wiki / group-contract links ... NOT id-keyed on locals (locals are never
cross-file addressable, per the document-scoped contract of item 2).
Verified: tsc clean; 14/14 across the two touched files; `detect_changes`
reports 0 changed symbols (tests only).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(php): a closure binding is a call SOURCE, not only a TARGET (#2699 part B, S1)
A call made inside a closure binding was attributed to the ENCLOSING scope, so
the closure was a call TARGET but never a call SOURCE: impact(handler,
direction:"downstream") reported nothing even though the closure calls out.
Root cause, probe-measured rather than inferred. Instrumenting
pickCallerCallableDef (graph-bridge/ids.ts) to log every rejection reason shows
the closure's own scope EXISTS and its range DOES contain the call site, but its
ownedDefs is EMPTY, so the ":94" owned-callable filter drops it and attribution
falls through to the ":97" enclosing-scope fallback.
The reason is one missing query rule. javascript/query.ts pairs the binding name
with the closure via @declaration.function anchored on the INNER arrow node, so
anchor.range equals the @scope.function range and pass2AttachDeclarations
attaches the declaration to the CLOSURE's scope. No other language had that
rule — PHP, Rust, Kotlin, Ruby and Dart all captured named function
declarations only. That single omission is the entire empty-ownedDefs cause.
This ports the rule to PHP with the same anchor discipline (@declaration.function
on the inner anonymous_function / arrow_function, NOT on the
assignment_expression wrapper). PHP needs nothing else: it already declares
(anonymous_function) and (arrow_function) as @scope.function, so the rule alone
completes it.
Measured on a fixture: `$handler = function ($x) { return target($x); }` inside
outer() now emits
Function:src/a.php:outer.$handler@3:2 -> Function:src/a.php:target
where it previously emitted `outer -> target`.
The pinned test in closure-binding-labels.test.ts asserted the OLD, wrong
behaviour by design ("to catch that asymmetry changing in EITHER direction"), so
it is INVERTED here rather than deleted, per its own instruction. Its block
comment is corrected to record the measured root cause, including that Kotlin
and Ruby will need BOTH this rule AND a relaxed kind gate (their lambda_literal
/ do_block is @scope.block deliberately, #1757), and that Dart has no closure
scope at all.
Verification: closure-binding-labels 50/50; PHP resolver suites 221/221
(php, php-coverage, php-response-shapes). detect_changes {staged}: 1 changed
symbol (PHP_SCOPE_QUERY), 0 affected processes, risk LOW.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(rust): emit a node for a closure binding and make it a call SOURCE (#2699 part B, S3)
Rust was the one exception to #2687's "a closure bound to a name is a Function
node in every language": `let handler = || target(1);` produced NO graph node at
all, so the closure could be neither a call target nor a call source.
Needed BOTH query channels, which is the finding worth recording. Porting only
the scope-resolution rule (as S1 did for PHP) changed nothing measurable here,
because there was no node to attribute anything to:
- languages/rust/query.ts — closure-binding declaration, @declaration.function
on the INNER closure_expression so anchor.range aligns with the existing
(closure_expression) @scope.function. This is what gives the closure's own
scope a callable in ownedDefs, which is what stops pickCallerCallableDef
falling through to the enclosing fn.
- tree-sitter-queries.ts — @definition.function on the OUTER let_declaration.
This emits the Function NODE that Rust never had.
Note the deliberate anchor asymmetry between the two channels: the graph-node
channel anchors the WRAPPER (matching the existing
(lexical_declaration (variable_declarator ... (arrow_function))) rule), while
the scope-resolution channel anchors the INNER closure (to align with
@scope.function). Getting these backwards silently produces either no node or
an unattributable one, so both sites carry a comment saying so.
Measured on a fixture — `let handler = || target(1);` inside outer():
Function:src/a.rs:outer CALLS Function:src/a.rs:outer.handler@2:4
Function:src/a.rs:outer.handler@2:4 CALLS Function:src/a.rs:target
Previously the whole binding was absent and the call read as `outer -> target`.
The rule also covers `move` closures: the closure_expression node spans the
`move` keyword.
Verification: closure-binding-labels 50/50; rust.test.ts 192/192;
rust-coverage, rust-f70, rust-scope all pass; rust-captures-golden passes
UNCHANGED, so no golden regeneration was required. detect_changes {staged}:
2 changed symbols (RUST_SCOPE_QUERY, RUST_QUERIES), 0 affected processes,
risk LOW.
One caveat on the suite runs: this host times out `beforeAll` hooks at the
default 60s under load — rust.test.ts needed --hookTimeout=600000 to complete,
and a concurrent second vitest run starves worker startup entirely (every test
fails at ~5001ms). Both are host artifacts, not signal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(kotlin,ruby): a closure binding is a call SOURCE, via a Block-scope callable boundary (#2699 part B, S2)
Kotlin and Ruby anchor a closure on a Block-kind scope — Kotlin lambda_literal
and Ruby do_block/block are @scope.block DELIBERATELY (#1757 smart casts), so
they must not be re-kinded. pickCallerCallableDef gated its child-scope walk on
kind === 'Function', so a closure there could never become a call SOURCE.
Both halves are required; neither alone changes anything:
1. kotlin/query.ts and ruby/query.ts gain the closure-binding declaration rule,
with @declaration.function on the INNER lambda_literal / block so its range
aligns with the @scope.block range (the anchor discipline documented in
javascript/query.ts). Without this the closure scope owns no callable def.
2. pickCallerCallableDef accepts a Block-kind child as a callable boundary when
the scope IS that callable's body. Without this the kind gate still rejects.
The alignment test in (2) is the part worth scrutiny. Relaxing the kind gate to
accept ANY Block owning a callable would be a real regression: a nested
`fun foo()` declared inside a block is owned by that block, so a call made at
BLOCK level — outside foo — would be misattributed to foo. Comparing the def's
declaration position against the scope's start position discriminates them: for
a closure the declaration and the scope sit on the SAME node, so the positions
match; for a nested function the block starts at `{` while the def starts at the
declaration, so they do not. Existing Function-kind behaviour is untouched, so
every already-working language is unaffected by construction.
The comparison is base-safe: scope-extractor.ts builds a def id as
`def:<filePath>#<startLine>:<startCol>:<type>:<name>` from the same Range a
scope carries, so both sides share one coordinate base. This is called out in
the helper's docblock because `defStartLine` nearby documents its own output as
1-based, which invites a wrong "fix" (#2377 is exactly this class of hazard).
Ruby's call forms are restricted to lambda/proc by name: an unrestricted
(call block: (block)) would match ANY method call taking a block, so
`mapped = items.map { |i| ... }` would wrongly declare `mapped` a callable.
Verified against the parser: 3 matches (->, lambda, proc), map excluded.
Separate #eq? patterns rather than one #match? alternation, which is a known
hazard on this tree-sitter line.
Measured on fixtures:
Kotlin Function:src/A.kt:outer.handler@2:4 CALLS Function:src/A.kt:target
Ruby Function:src/a.rb:outer.handler@4:2 CALLS Method:src/a.rb:target#1
previously `outer -> target` and `outer#0 -> target#1`.
The pinned Kotlin test asserted the old behaviour by design and is INVERTED, not
deleted. Ruby had NO pinned case, so a new one is added rather than inverted.
The describe title no longer claimed something false ("not yet a call SOURCE"
now holds only for Dart) and was retitled.
Verification: closure-binding-labels 51/51; kotlin.test.ts, kotlin-coverage,
ruby.test.ts, ruby-scope, ruby-namespaced all pass (478 passed / 1 expected
inversion before the test was flipped). impact on pickCallerCallableDef:
CRITICAL, 191 impacted, ONE d=1 (resolveCallerGraphId) — the return contract is
unchanged, so that dependent is unaffected. detect_changes {staged}: 5 changed
symbols, 2 affected processes (both EmitReferencesViaLookup, one of them the new
ScopeIsCallableBody step), risk medium.
Dart remains the last failing language: dart/query.ts declares no
@scope.function at all, and dart/captures.ts synthesizes one only from a
declaration WITH a body node, which an expression-bodied closure lacks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(dart): give a closure binding a scope and a distinct identity (#2699 part B, S4)
Dart was the last language where a closure binding could not be a call SOURCE,
and fixing only that would have made the graph WORSE, not better. This lands
both halves together for that reason.
## The attribution half
A Dart closure had no scope at all. `dart/query.ts` declares no
@scope.function anywhere — Dart's function scopes are SYNTHESIZED in
`dart/captures.ts` from `declNode` + `findFunctionBody(declNode)`, and
`findFunctionBody` looked only at the next named SIBLING for a `function_body`.
A closure literal carries its body as a CHILD (`function_expression_body`), so
it matched nothing and no scope was produced.
`query.ts` gains the closure-binding declaration rule and `findFunctionBody`
understands the child form. Deliberately NO @scope.function is added to the
query: it would collide at identical range with the synthesized one, and
duplicate scope ids make `buildScopeTree` throw, which DROPS THE WHOLE FILE.
## The identity half, and why it is not optional
With attribution alone, two same-named closures in one file both keyed to the
bare `Function:a.dart:handler`. One node then appeared to call BOTH targets —
a CALLS edge present nowhere in the source. That is worse than the missing edge
it replaced, so S4 could not ship without this.
Root cause is not Dart-specific. `enclosingCallablePrefix` derives a SEMANTIC
relation — what encloses this callable — by SYNTACTIC ancestor walk. Dart parses
`int outer() { … }` as `function_signature` followed by `function_body` as
SIBLINGS, so the enclosing callable is never an ancestor of code inside it and
no membership set can fix that; the walk looks in the wrong direction.
This is what SCIP and real compilers avoid by construction. SCIP keeps a local
symbol opaque (`local <id>` — no name, no position, no chain) and models
containment as a SEPARATE `enclosing_symbol` field; its spec says the local/global
choice should follow ACCESSIBILITY, not the ability to name an enclosure. Dart's
own analyzer answers this from `Element.enclosingElement` in the element model,
never from AST ancestry. clang uses `name@offset` for a function-local; Kythe
uses a document-scoped VName plus a `childof` edge. Identity is positional and
opaque; enclosure is a relation.
`findSplitBodyCallableAncestor` is the narrow fix at that seam: a fallback used
ONLY when the ancestor walk finds nothing, recovering the callable from the
body's preceding sibling.
The sibling must be a BARE SIGNATURE, and that restriction is load-bearing —
"any preceding callable sibling" is WRONG and was caught regressing PHP during
this work. In `<?php function target($x) {…} $handler = function ($x) {…};` the
closure is at FILE level, so the ancestor walk correctly finds nothing, the
fallback runs, and an unrestricted version mis-qualified the file-level
`$handler` as `target.$handler`. A preceding sibling is only an ENCLOSING
callable when it cannot hold its own body.
`SPLIT_SIGNATURE_NODE_TYPES` is exactly that set and is DERIVED, not listed:
`LOCAL_SCOPE_BODY_NODE_TYPES` is already `FUNCTION_NODE_TYPES` minus the bare
signature types, so the difference between them IS the split-signature set
(`function_signature`, `method_signature` — verified at runtime). PHP's
`function_definition` carries a body and is in both, so it is excluded. No
language is named in shared code, and any future split-grammar language is
covered for free.
## Verification
Full resolver sweep — the gate that caught #2714's Rust regression — 2926
passed / 1 skipped / 0 failed across 51 files. closure-binding-labels 52/52;
dart.test.ts, dart-coverage, callable-id-lockstep, function-local-identity,
caller-identity-regression all pass (156/156 across 6 files).
impact on `enclosingCallablePrefix`: LOW, 5 impacted, 3 d=1 all inside
parse-worker. detect_changes {staged}: 5 changed symbols, 0 affected processes,
risk LOW.
Three existing Dart expectations FLIPPED rather than being deleted: Dart locals
now carry the same enclosing-callable + position identity every other language
got in #2695, so `local.dart:handler` became `local.dart:caller.handler@1:2`.
A new test pins the actual defect — two same-named closures staying DISTINCT
nodes — because the qualification assertions alone would not fail if the
fabricated edge returned.
One note for future work: an id-shape assertion here carries a call-site suffix
on indirect invocations (`…handler@3:2:5:9`) but not on direct calls. That is
the callable-value-flow pass keying its edge by invocation position, not part of
the node id.
Part B is now complete: PHP (S1), Rust (S3), Kotlin + Ruby (S2), Dart (S4).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* fix(scope-resolution): close every deferred item on #2699 (A1 values, twin-list guard, schema bumps)
Clears the limitations this PR had been carrying rather than leaving them as
follow-ups.
## A1 — function-local VALUES now carry their own identity
This was #2699's ORIGINAL complaint and the one a callable-only gate could never
reach: a top-level `const handler` and a function-local `const handler`
collapsed onto ONE `Const:v.ts:handler`. #2695 restricted position-qualified
identity to Function|Method|Constructor because the collision that produced
wrong CALLS edges was between callables, and widening churned ids for symbols
the pruner mostly deletes. The churn is real and is accepted here deliberately.
Widening needed THREE gates aligned, not one:
- id-building — `parse-worker.ts` nestedCallablePrefix
- resolution — `ids.ts` position key
- registration — `node-lookup.ts` position-key registration
Missing the third would register no position key for values, so every lookup
misses and falls through silently. That is the #2714 failure mode: the caller
attaches to a node that does not exist and the edge is DROPPED, which looks like
"zero dangling edges" from outside. All three now route through ONE predicate,
`isPositionQualifiedLocalLabel`, rather than repeating the label set a third
time.
Only LOCALS move. The prefix comes from `enclosingCallablePrefix`, which returns
undefined when nothing encloses the declaration, so top-level and class-member
ids are untouched — verified by the full resolver sweep, where a leak onto class
members would have broken assertions in every language. `Property` is included
on purpose: a class field stays unqualified because the prefix walk boundaries
on class-likes, while an object-literal property inside a function is genuinely
local and would otherwise keep the old collision.
Measured: `Const:v.ts:handler` + `Const:v.ts:run.handler@3:2`, two distinct
nodes. The KNOWN LIMIT test is FLIPPED per its own former instruction ("this
test should be updated as part of it rather than deleted").
## Schema bumps — required by Part B, not just by A1
INCREMENTAL_SCHEMA_VERSION 20 -> 21, parse-cache SCHEMA_BUMP 27 -> 29.
SCHEMA_BUMP is 29, not 28, and that is the point of re-checking it against
origin/main at MERGE time rather than branch time. This branch cut at 27 and
bumped to 28; #2415 also bumped 27 -> 28 and merged first. The automated
main-merge onto this branch surfaced the collision — leaving it at 28 would have
shipped this whole change with NO parse-cache invalidation, so every warm cache
keeps replaying the pre-fix captures and ids. This is the third instance of that
collision recorded in parse-cache.ts (#2632/#2653 hit it at v21, and
#2653/#2654 hit INCREMENTAL_SCHEMA_VERSION the same way).
Part B already changed emitted node ids AND edges on files that did not
themselves change (Dart locals re-keyed, Rust gained a node it never emitted,
five languages gained closure-source attribution). A v20 index topped up
incrementally keeps serving the old attribution, and a warm parse cache replays
the old captures and ids verbatim. Shipping S1-S4 without these would have let
every existing index silently keep the pre-fix graph.
## Twin-list drift guard — the sixth instance in this family
`IMPLICIT_RECEIVERS` (gitnexus-shared lookup-core.ts) and `THIS_RECEIVERS`
(type-env.ts) spell the same concept in two packages, and nothing enforced
agreement — `$this` was added to the shared list in #2714 only because it was
already in the other. New structural test asserts set equality plus the ONE
deliberate asymmetry (`Me`, Visual Basic spelling, absent from the shared list
because no SupportedLanguages entry uses it) in BOTH directions, so re-adding it
there or dropping it here each fail loudly.
Structural rather than value-imported: both constants are module-private, and
exporting them purely to be testable would widen two public surfaces to satisfy
a test.
## Two false comments corrected
- `lookup-core.ts` said "see the drift guard noted in #2714", implying a guard
existed when it was only a deferred follow-up. It exists now, and the
comment points at it.
- `callable-id-lockstep.test.ts` claimed its regex "fails if any site
reconstructs the id". It matches ONE template spelling; a hand-rolled
concatenation still slips past. Now stated as a tripwire for the known
shape, not a proof.
## Skill learnings
Four entries appended to eval/workflow_bench/learnings.jsonl from this run: the
v9fs safe-writer failure, backticks silently terminating a query template
literal (hit three times), a module-level TDZ const that passes tsc and then
presents as N file failures with ZERO failing assertions, and concurrent vitest
runs starving worker startup so a whole suite fails at ~5001ms.
## Verification
Full resolver sweep 2926 passed / 1 skipped / 0 failed (51 files) — identical to
pre-A1, which is the evidence that only locals moved. All EIGHT bench gates PASS
with fingerprints UNCHANGED, so no regeneration was needed. function-local-identity,
callable-id-lockstep, receiver-twin-list-drift and closure-binding-labels 71/71.
tsc --noEmit clean.
detect_changes {staged}: 9 changed symbols, 14 affected processes, risk HIGH —
expected, and the reason the sweep above is the gate rather than a targeted list.
Every affected process routes through `resolveDefGraphId`, the key chain Part A
measured at CRITICAL with 23 direct dependents.
Deliberately NOT done: the SCIP end state (opaque `local <id>` plus an explicit
enclosure EDGE instead of containment encoded in the id string). It is a design
direction, not a limitation of this work, and it is INCOMPATIBLE with A1 — A1
widens chain-encoded identity, that removes chain encoding entirely. Bundling
both would re-key every local twice. Written up in the research notes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR
* test: update two assertions the #2699 changes correctly invalidated
Both failed on CI at
|
||
|
MyShining
|
ff86ccf1e7
|
feat(spring): model profiles, conditions, and auto-configuration (#2678)
* feat(spring): model conditions and auto-configuration * fix(spring): align auto-configuration declarations * perf(spring): streamline auto-configuration indexing * test(spring): move timing benchmark out of vitest --------- Co-authored-by: Shining <xuenning@qiyi.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
e307286d52
|
fix(scope-resolution): a named receiver's member never resolves lexically, + two #2695 follow-ups (#2714)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(scope-resolution): a named receiver's member never resolves lexically (#2699) `lookupCore` Step 1 walked the lexical scope chain for every lookup, including explicit-receiver property reads. So `options.baseUrl` could bind to an unrelated function-local `const baseUrl` in the same file, and `config.extractVisibility(node)` to the enclosing class's own method. This is the residual half of the defect JS/TS block scopes narrowed in #2695. Blocks moved nested-block locals off the chain of a reference outside the block, which removed 114 false edges; a local declared directly in the function body stayed on it, and no amount of extra scopes reaches that case. Fixed at the cause instead: `recv.name` names a member of whatever `recv` denotes, so a binding of the bare tail name in an enclosing scope is never the right answer. Steps 2 and 3 (receiver type / owner members) are the legitimate routes. `this` and `self` are EXEMPT, and that exemption was measured, not assumed. Skipping Step 1 for every explicit receiver removed 711 edges on a 762-file corpus — but 2 of those were genuine: `self.srcIx` and `self.streamedAt(...)` after `const self = this`, reaching their own class's members through the class-body scope. For a self-receiver the members and the lexical chain legitimately overlap; for a named receiver they never do. Exempting the self names keeps both true edges and still removes 709 false ones, adding none. The removals were classified by reading source at the site, not by pattern- matching ids — an "is the target a member of the source's owner?" heuristic labelled 43 of them plausible and every one I then read was false: language = config.language; -> the class's own `language` dirMap.get(...) / exactMap.get(...) -> a sibling object-literal `get` return config.extractVisibility(n); -> the class's own method (self-edge) writer.close(); -> GraphEmitSink.close Residual, deliberately kept: a `this.x` read can still bind lexically to a same-named local. That is the price of the two true self-alias edges above. `INCREMENTAL_SCHEMA_VERSION` 19 -> 20: a v19 index holds these false CALLS/ACCESSES on every unchanged file and would keep serving them through the reuse gate. Test confirmed discriminating: it fails with the guard reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR * fix(typescript,javascript): a generator expression binding is a Function node (#2693) `const g = function* () {}` matched none of the closure-binding definition rules — they covered `arrow_function` and `function_expression` only — so the binding emitted a `Const` node. `buildGraphTargetIndex` admits callable nodes only, so `g()` resolved to nothing. Same defect shape as the `var` case #2693 already fixed: a different grammar node for the same construct, and the resulting graph node was not callable. Adds the four variable-binding shapes in both languages: `const`/`let` and `var`, each plain and exported. Purely additive — no existing pattern is reordered or rewritten, because the #2687 pre-scan dedup is order-dependent and collapsing the value/callable pair depends on which match wins. Deliberately NOT covered, and the query comment says so: a generator in an object-literal pair or a HOC wrapper still falls through anonymous. Those are rarer, and each additional pattern is another chance to disturb the dedup. `SCHEMA_BUMP` 26 -> 27: definition captures are parse-time, so a warm parse cache would replay the old ones verbatim — `--force` does not clear it. Two tests confirmed discriminating (they fail with the patterns reverted), plus a guard that the already-working generator DECLARATION form is unaffected, since it shares the emit path these were inserted beside. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR * fix(ingestion): keep caller attribution in lockstep with definition ids (#2699) The definition phase appends `localIdentity` to a nested callable's own name segment (`run.save@3:2`); `findEnclosingFunctionId` did not, so the two phases derived different ids for the same callable. The failure mode is silent — the caller id names a node that does not exist, so the edge is dropped rather than reported — which is why the parse-worker docblock calls this pair a lockstep guarantee and asks that both phases derive the prefix from one place. The condition is now byte-identical to the definition phase's (`nestedPrefix !== undefined`), so the two cannot diverge again. Scope of the claim, stated plainly: no reproducing case was found, and this changes nothing measurable on a 762-file TypeScript corpus. TS/JS resolve callers through `resolveCallerGraphId` in the graph bridge, not this path; `findEnclosingFunctionId` serves the `callExtractor` languages, and the corpus does not exercise a nested callable there. The review that raised it (P3) observed zero dangling edges, and "zero dangling" is also what silently dropped edges look like — so this closes a documented contract rather than a demonstrated bug, and carries no test of its own. Rides the `SCHEMA_BUMP` 26 -> 27 in the preceding commit: caller attribution runs in the worker, so a warm parse cache would replay the old ids. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184RmD24KFJidYqpM7v3XjR * docs(test): correct the block-scope header that this PR made false (#2699) Review finding (MEDIUM). The file header still described `lookupCore` Step 1 as walking the lexical chain for EVERY lookup, and called the function-body-local case "unchanged and still mis-resolves ... pre-existing and tracked separately". Commit |
||
|
Gergő Magyar
|
4906daf27b
|
fix(scope-resolution): resolve calls through a closure-valued binding across languages (#2693) (#2695)
* fix(scope-resolution): resolve calls through a closure-valued binding (#2693)
`val f = { }; f()` emitted no CALLS edge in Kotlin or Swift, so `impact` on
such a symbol under-reported to zero — the same false all-clear as #2687.
The cause was not, as first suspected, that these languages fail to feed
`callable-value-flow`. They do: `synthesizeCallableFlowCaptures` is called
from 15 language capture modules, and Kotlin already resolves reassignment
through the pass (`var f = ::a; if (c) f = ::b; f(1)` reaches both targets).
Their captures are already exactly right — the seed names the binding as its
own callable, per the anonymous-callable convention in
callable-flow-captures.ts.
They died one layer later, at the `buildGraphTargetIndex` gate:
if (!isCallable(def) && providerTarget?.(def) !== true) continue;
`isCallable` is Function/Method/Constructor, but the scope-resolution layer
declares a closure binding with its VALUE label (Kotlin/Swift `Property`),
and `isCallableValueTarget` is implemented by exactly one provider — COBOL.
So the binding never entered `graphTargets`; `lexicalCallableLookup` then
returned `shadowed: true` with no targets, which also suppressed the
workspace-wide fallback, and the seed resolved to nothing.
Only the graph knows a value binding holds a callable — since #2687 it emits
a single `Function` node for one. So value bindings now resolve their graph
id first and are admitted on the label of the node they actually reach.
This is self-limiting: a genuine constant keeps its own Const/Property node,
so `resolveDefGraphId`'s qualified key hits before the label-agnostic
`simpleKey` fallback can reach a same-named callable. Only a binding whose
own value node was replaced by a callable one gets through.
No scope kind changes — Kotlin's `lambda_literal` stays `@scope.block`, so
#1757 smart-cast semantics are untouched by construction. The fix is
language-neutral: it discriminates on the graph node label, never on a
language name.
Dart is fixed separately; its root cause is independent.
* fix(dart): resolve calls through a closure-valued binding (#2693)
Dart needed more than the shared gate fix: neither of its closure-binding
forms could resolve, for two different reasons, and the plan's one-line
diagnosis turned out to be incomplete.
TOP-LEVEL `var f = (x) => x;`
A graph Function node already existed (#2687), but no `@declaration.*`
matched the binding, so scope resolution had no SymbolDefinition to attach
a flow seed to. Adding the declaration exposed a second problem: Dart's
`initialized_identifier` is FIELDLESS, so the shared field-based assignment
fallback (`left`/`name`/`value`/…) decomposed nothing and the binding still
emitted no flow captures at all. Kotlin's fieldless `assignment` node hit
exactly this and took the same remedy — a provider `extractAssignment`.
FUNCTION-LOCAL `void m() { var f = (x) => x; }`
Locals parse as `initialized_variable_definition`, which the top-level
graph-node rules are deliberately anchored under (program) to avoid, so a
local closure had no graph node at all — nothing for the widened
`buildGraphTargetIndex` gate to admit.
Both new rules are restricted to a `function_expression` value. Declaring
every Dart variable would mint defs and nodes repo-wide for no resolution
benefit; ordinary locals stay unindexed exactly as before. The top-level
declaration reuses the (program) anchor the graph-node query already relies
on, so class-body fields — which share `initialized_identifier_list` and are
already `@declaration.property` — are never matched twice.
Also drops the now-false note in tree-sitter-queries.ts claiming `f()` does
not resolve for Dart. That node is now the evidence that makes it resolve.
* docs(scope-resolution): document the callable-flow capture contract (#2693)
The module is 1200+ lines behind a nine-line docblock, and the only worked
example was C. Both root causes fixed in this series were "the contract was
discoverable only by reading the emitter":
- the anonymous-callable convention (a seed whose source is a closure takes
its DESTINATION's name) is what makes closure bindings resolvable at all,
and is the reason the widened target gate is correct;
- a fieldless binding node silently decomposes to nothing under the shared
assignment fallback, which cost Kotlin one debugging cycle in #2522 and
Dart another here;
- captures alone are never enough — the bound name also needs a
`@declaration.*` or there is no cell to key the seed on.
Records the cell/site model, both traps, and points at the fullest and
smallest worked examples.
Bumps INCREMENTAL_SCHEMA_VERSION 15 → 16 and the parse-cache SCHEMA_BUMP
22 → 23: this series emits NEW CALLS edges and new Dart Function nodes, and
the incremental write set only covers changed files, so an existing index
would keep reporting a zero blast radius for exactly the symbols the fix is
about.
* perf(scope-resolution): pre-filter value bindings in the callable target index (#2693)
Widening the `buildGraphTargetIndex` gate to consider VALUE bindings put the
hot loop on a much larger def population — value bindings outnumber callables
in real source — and the naive version paid full price per binding. Measured
on a synthetic 800-file corpus (8 value bindings per file, 1 of them a closure
binding), the widening cost 2.50-2.82x the pre-#2693 callable-only build.
Two wastes, both provable rather than guessed:
1. `definitionAnchorKey` ran for every def, including value bindings. The
anchor index is keyed by callable LABEL and the key is built from
`def.type`, so a value def can never hit it — and the key costs a regex
per def.
2. Every value binding paid the whole `resolveDefGraphId` key chain only to be
rejected. It need not: every qualified key that function tries embeds
`def.type`, so for a VALUE def those can only ever reach a value-labelled
node. Its one route to a callable is the label-agnostic
`simpleKey(filePath, simpleName)` fallback, which by construction requires
a callable node with the SAME file and simple name. So a value binding with
no such node cannot resolve to a callable, and one Set lookup decides it.
That set is derived in the graph walk the anchor index already performs, so it
costs no extra pass.
large_ms 7.79-8.37 -> 4.90-5.02 (1.61x faster)
widening_overhead 2.50-2.82 -> 1.45-1.50
The resolved target-set fingerprint is byte-identical across both, which is
the point: this is a cost change, not a behaviour change.
Adds bench/callable-value-flow/ (fingerprint + scaling + widening-overhead
gates) and wires it into ci-tests.yml beside the other build-free benches. The
overhead budget of 1.9 sits between the measured with-filter and without-filter
bands, so it cannot be met if the pre-filter is removed. Timings use the MIN of
15 warmed reps, not the median: the same build reported 1.65 idle and 2.03
under load, and a median-based gate would have to be loosened past the point of
detecting the regression it exists to catch.
`buildGraphTargetIndex` is exported for the bench; it is pure and not part of
the pass's public contract.
* test(scope-resolution): assert the declaration route does not double-emit (#2693)
Go, Python, C++ and TS/JS already resolved a closure-binding call through
their `@declaration.function` capture. The widened `buildGraphTargetIndex`
gate gives the same call a SECOND possible route, so each must still produce
exactly one edge.
`tryEmitEdge` dedups by key, but a collapsed key and a site-anchored key are
DIFFERENT keys — a real double-emit would show up as two ids for one call
site, not be silently collapsed. Asserting on edge ids rather than target ids
is what makes that visible.
* fix(scope-resolution): join value bindings to their callable node by POSITION (#2693)
Review found the first cut of this series minted FALSE CALLS edges. Admitting a
value binding whose *resolved* graph node is callable let `resolveDefGraphId`
fall through to its label-agnostic, first-write-wins
`simpleKey(filePath, simpleName)` and bind the name to ANY same-named callable
in the file.
The safety argument in the previous commit — "a genuine constant keeps its own
Const/Property node, so the qualified key hits first" — silently assumed
`def.type === node.label`. It does not hold:
- TypeScript declares `const` as `Variable` but emits a `Const` NODE, so the
qualified key misses even though the value node exists;
- Rust `let` bindings get no graph node at all, so the fallback is the only
route.
Reproduced, all previously emitting a fabricated caller:
const save = (x: number) => x * 2; // next to an unrelated Svc.save
-> Method:svc.ts:Svc.save#1 // Svc never instantiated
const handler = other; // shadowing a top-level handler
-> Function:app.ts:handler // unreachable from here
let handler = cb; // Rust
-> Function:main.rs:handler
Worse in Dart, where the same collision INVERTED the feature: the only edge went
to the class method and the closure's own node got none. The result was also
declaration-order dependent — two files differing only in declaration order got
different CALLS sets — and it propagated through argument-to-formal binding into
functions whose source never mentions the name.
A closure binding IS its callable node: same file, same line, same name. An
aliasing local is not. So the join is positional now — a file/line/name index
built in the graph walk `byAnchor` already performs — and value bindings never
run the key chain at all. That is both correct and cheaper:
large_ms 4.90-5.02 -> 4.37-4.63
widening_overhead 1.45-1.50 -> 1.43-1.58 (name-match design: 2.50-2.82)
with a byte-identical target-set fingerprint on the bench corpus.
Also from review:
- `Static` dropped from VALUE_BINDING_DEF_TYPES: `normalizeNodeLabel` has no
`static` case, so no def can carry that type — it was an entry no fixture
could ever exercise. The remaining set now documents why it deliberately
does NOT reuse `isOwnableValueLabel`, which is contracted to a different
consumer.
- Dart `final`/`const` top-level closures (static_final_declaration_list) and
every declarator after the first in a multi-name local now resolve; both
parse into shapes the earlier rules never reached.
- The bench source carried a literal NUL byte, so git recorded it as BINARY
and the only artifact pinning the target set was unreviewable in the PR
diff. It is written as an escape now. Its corpus also modelled `startLine`
as 1-based where graph nodes are 0-based, which would have stopped it
exercising the value-binding path at all.
- `call-summary-schema-version.test.ts` asserted `passesReuseGate(15)` is
true; the 15 to 16 bump made that false and the test RED. It now pins 16 as
current and 15 as rejected, matching the pattern every prior bump followed.
- The v23 parse-cache comment is at the top of the list, not mid-list.
Tests: the five collision cases above are new regression tests, each confirmed
failing against the previous commit. Also added Kotlin class-body closures (the
only case exercising the Method arm), Dart top-level `final`, Dart multi-name
locals, and a warm-parse-cache replay for Kotlin and Dart — the #2693 captures
are replayed verbatim, so a serialization change would surface only on a SECOND
analyze and every other test here runs cold. The previous negative tests were
vacuous: they paired names that did not collide (`maxSize` vs `size`), so the
pre-filter rejected them before the guard they were named after could run.
* docs(storage): fix the schema-version changelog blocks (#2693)
Two problems, one mine and one not.
MINE: the `INCREMENTAL_SCHEMA_VERSION` block is ASCENDING (v2 … v15), and I
inserted v16 above v15 rather than at the end — I had just moved the parse-cache
entry to the top of ITS block, which is descending, and applied the same habit
to a list ordered the other way. Moved to the end; both blocks are now
internally consistent.
NOT MINE: the parse-cache block carries TWO v21 entries, with v20 wedged between
them. Tracing it: #2632 (Spring DI facts) bumped 20 -> 21 and merged first;
#2653 (Java JLS local-class identities) had branched at 20, also bumped to 21,
and merged second — so it shipped with NO invalidation of its own. An index
already stamped 21 by the first change was treated as current by the second and
kept serving stale local-class identities from the warm cache.
Numbers left alone: both genuinely shipped as 21, and renumbering them now would
misstate what users' indexes actually contain. Instead the entry says so
explicitly, and points at the process fix — re-check the constant against
origin/main immediately before merging, not just when the branch is cut. The
identical collision hit INCREMENTAL_SCHEMA_VERSION in #2653/#2654, so this is a
recurring failure mode of concurrent PRs, not a one-off typo.
Comment-only; no constant changes value.
* feat(scope-resolution): resolve closure bindings in Ruby, Java, C#, PHP and JS/TS var (#2693)
Ruby, Java, C# and PHP already emitted correct callable-flow seeds and invokes.
What they lacked was the #2687 piece — a CALLABLE graph node at the binding,
which is what buildGraphTargetIndex joins to by position. PHP additionally had
no scope declaration for the bound name, so the flow pass had nothing to attach
its seed to.
ruby handler = ->(x) { x } handler.call(1) -> Function:a.rb:handler
java Function<..> handler = x->x handler.apply(1) -> Function:A.java:A.handler
csharp Func<int,int> handler = ... handler(1) -> Function:A.cs:A.handler
php $handler = fn($x) => $x $handler(1) -> Function:a.php:handler
Ruby and Java invoke through the callable-object protocol; C# and PHP call the
binding directly. Locals work in all four, and a binding whose name collides
with a same-named method resolves to the CLOSURE, not the method.
Two things the sweep caught:
JAVA TWIN. Anchoring the rule on the inner variable_declarator produced BOTH a
Function and a Property node — the exact double-indexing #2687 removed. The
parse-worker dedup keys on (definition node, name), and Java's value rule
anchors on field_declaration, so the keys never matched. Re-anchored on
field_declaration / local_variable_declaration.
JS/TS `var`. `var f = (x) => x` kept a Variable label while const/let got
Function, because `var` is a different grammar node (variable_declaration vs
lexical_declaration) that no closure rule covered. A call through the binding
still resolved via the declaration route, so the CALLS edge pointed at a
NON-callable node. Now consistent across const/let/var.
That last one flipped an existing assertion in const-function-twin.test.ts,
which expected `Variable` for a var-bound function-expression. Its comment
explained why — "var has no matching @definition.function pattern, so nothing
claims the name" — i.e. it documented the gap rather than defending it. The
property it was really protecting (an UNCLAIMED value node survives) now has
its own case with a non-function initializer, and the var-closure case asserts
the collapse to one node, which is also the twin guard for the new rule.
Known limits, both pre-existing and both failing safe:
- A PHP local closure whose name collides with a top-level function gets no
edge: both want id Function:<file>:<name>, so the closure never gets its own
node. This is the file-scoped node-identity convention — TypeScript, Python
and Dart collapse identically at base.
- TS/JS class-field arrows stay Property (Kotlin's equivalent emits Method).
They already resolve; changing the label risks the HAS_PROPERTY ownership
regression #2687 hit once.
The invalidation constants already bumped in this PR (INCREMENTAL_SCHEMA_VERSION
16, SCHEMA_BUMP 23) cover these additional languages; their notes now say so.
Tests: one case per newly-resolving language plus the PHP anonymous-function
form and the JS var form, in closure-binding-labels.test.ts. The file now spins
a worker pool per test across a dozen languages, so its timeout is raised
file-wide — a case that takes ~7s alone was exceeding the 30s default under
that contention.
* fix(ingestion): class-field closures are callable members in TS/JS (#2693)
A CALLS edge must target a callable node. `class A { handler = (x) => x }` emitted
a Property, so calling it produced `CALLS -> Property:A.ts:A.handler` — an edge
pointing at something the graph says is not callable. Same defect class as the
JS/TS `var` binding fixed in the previous commit, and the last place a closure
binding still carried a value label.
Kotlin already models its class-body closure as Method + HAS_METHOD; TS/JS now
match, so all three agree:
class-field closure -> Method + HAS_METHOD (CALLS target is callable)
plain class field -> Property + HAS_PROPERTY (unchanged, no CALLS)
Anchored on public_field_definition / field_definition — the same nodes the
property rules use — so the parse-worker dedup collapses the pair rather than
leaving a Method/Property twin, the failure the Java rule hit in the previous
commit.
ON MATCHING THE COMPILERS. This deliberately diverges from tsc and SCIP. The
TypeScript compiler classes `handler = () => {}` as a PropertyDeclaration
("a property declaration independently from what it's assigned to"), and SCIP
gives it a `.` term descriptor, the same suffix as any field — both call it a
property, and Kotlin's compiler likewise treats `val f = { }` as a property with
a function type. The divergence is intentional: GitNexus's Function/Method label
does not mean "tsc SymbolFlags", it means "this node can be the target of a
CALLS edge", which is the convention #2687 set for closure bindings in every
language. Modelling it the compiler's way would mean either dropping call
resolution for these members or emitting a separate node for the lambda and
flowing the property to it — the two-node shape #2687 removed. Recorded here so
the next reader does not "fix" it back.
Tests: TS and JS class-field arrows resolve to their Method node, plus a guard
that a NON-closure class field stays a Property — the closure rule must key on
the initializer, not on the field syntax.
* fix(php): keep the $ sigil on closure-binding nodes so locals stop colliding (#2693)
A PHP local closure whose name matched a file-level function got NO edge at all:
function save($x) { return $x; }
function run() {
$save = fn($x) => $x * 2;
return $save(1); // no CALLS edge
}
Both minted the id Function:<file>:save, so the closure's node was swallowed by
the function's and the positional join found nothing at the binding's line.
The fix is PHP's own semantics rather than a change to node identity across the
graph. PHP holds variables and functions in SEPARATE namespaces — $save and
save() cannot collide in the language — and the sigil is what separates them.
Dropping it was the bug. The node rule now captures the whole variable_name, so
the closure is Function:<file>:$save and the function stays Function:<file>:save.
languages/php/query.ts already keeps the sigil on property declarations for the
same reason, so this makes the two consistent.
The positional join normalises a leading $/@ on both sides, matching what the
scope layer and the callable-flow synthesizer already do, so the binding still
matches its own declaration while its NODE stays distinct.
local closure + same-named function -> Function:c.php:$save (the closure)
calling the real function -> Function:f.php:save (unchanged)
plain $max = 10 -> no node, no edge (unchanged)
WHAT THIS DOES NOT FIX. The general problem is wider than PHP: GitNexus node ids
are file-scoped, so a function-local symbol and a file-level one with the same
name collapse in TypeScript, Python and Dart too, and Java/C# only escape by
qualifying on the enclosing CLASS (so two same-named locals in different methods
still collide). SCIP solves it with a separate `local <id>` keyspace that is
document-scoped and never globally addressable. That is issue #2699 — it changes
persisted ids for every function-local symbol and needs its own invalidation, so
it is not bundled here. PHP is fixed on its own merits: the sigil belongs in the
identity regardless of how locals are eventually scoped.
* test(scope-resolution): pin the closure-binding caller-attribution limit (#2693)
Review of this PR found the new callable nodes are call TARGETS but never call
SOURCES: a call made INSIDE a closure binding is attributed to the enclosing
scope, so `impact(handler, direction:"downstream")` reports nothing even though
the closure calls out. Consistent across Kotlin, Dart, Ruby and PHP; TS/JS free
bindings are the exception because their arrow carries a @scope.function whose
range matches.
Not fixed here — pinned, so the boundary is visible instead of surprising, and
so a change in EITHER direction fails a test.
The cause is precise: `pickCallerCallableDef` (graph-bridge/ids.ts) finds the
caller by walking CHILD scopes whose range contains the call site, gated on
`child.kind === 'Function'`. A closure literal is a BLOCK scope in these
languages (Kotlin deliberately, #1757 smart casts), AND the binding's def is
owned by the enclosing scope rather than by the closure's scope — so neither
half of the link exists. Fixing it needs "callable boundary" decoupled from
scope `kind` plus an association between the closure scope and its binding.
That is a change to the caller anchor used by every call in the repo, which is
not something to land at the tail of this PR.
Also adds a unit suite for `buildGraphTargetIndex` itself, covering what the
integration tier cannot isolate: a binding is admitted only on POSITIONAL
evidence, a name-only match is rejected, a non-callable node at that position is
rejected, an ambiguous position claimed by two callables is rejected, and the
PHP dollar sigil normalises across the join while still not matching a
same-named function on another line. That last one closes the review's LOW —
the node/declaration name asymmetry now has an executable contract rather than
resting on a comment.
* docs(test): correct the per-language cause of the attribution limit (#2693)
The comment on the pinned attribution tests claimed "a closure literal is a
BLOCK scope in these languages". That is true for Kotlin (lambda_literal
@scope.block, #1757) and Ruby (do_block/block @scope.block) and FALSE for PHP:
anonymous_function and arrow_function are already @scope.function
(php/query.ts:61-62). Dart is a third case again — it has no scope over a
closure literal at all.
So the four languages fail at three different points, not one:
Kotlin, Ruby fail the `child.kind === 'Function'` gate
PHP passes that gate; its closure scope owns no callable def,
because the binding's def belongs to the enclosing scope
Dart has no child scope for the walk to consider
Worth correcting carefully rather than tidying: a follow-up plan re-stated this
comment instead of re-deriving it, and inherited the misdiagnosis — it proposed
"relax the kind gate" as required for all four, which is a no-op for PHP and
unreachable for Dart. A review caught it. The comment now states each language's
actual blocker and says why the distinction matters.
Comment-only; the three pinned tests are unchanged and still pass.
* fix(scope-resolution): an ordinary JS/TS `function` binds its own `this` (#2701)
`this.m()` inside a nested `function` resolved to the lexically enclosing
class, so it emitted a CALLS edge that does not exist at runtime — including
the exact `forEach(function () { this.m(); })` shape arrow functions were
introduced to avoid:
class D {
m() {}
build() { const h = function () { this.m(); }; return h; }
}
// CALLS: Function:D.ts:D.h -> Method:D.ts:D.m#0 FALSE
ECMA-262 gives an arrow `[[ThisMode]] = lexical`: it has no `this` binding in
its environment record, so the lookup passes through to the enclosing
environment. Every other function form binds `this` at call time. `tsc` draws
the same line by resolving `this` through `getThisContainer` with
`includeArrowFunctions = false`. That one rule is the whole fix.
Languages declare it; shared code never learns a language. The query files —
the one place that already names grammar nodes — tag every non-arrow function
form with `@receiver-owner.this`, which becomes `Scope.ownsReceivers`. A
receiver walk that reaches such a scope without finding the name stops there
instead of borrowing an enclosing scope's binding. Every other language leaves
the field unset and is bit-for-bit unchanged; a Kotlin lambda, which DOES
capture the enclosing `this`, still resolves (pinned as a test).
THREE GATES, ALL LOAD-BEARING. The false edge survived each one alone, which
is why the tests assert on the emitted edge rather than any single walk:
1. `Scope.ownsReceivers` stops BOTH receiver-type walks — `findReceiver
TypeBinding` here and its twin `lookupReceiverType` in gitnexus-shared's
`lookup-core`, which was resolving the receiver independently.
2. `LanguageTypeConfig.thisBoundaryNodeTypes` stops the type-env AST walk
that infers a receiver's type during capture.
3. `isReceiverOwnedButUnbound` makes `receiver-bound-calls` SUPPRESS the
site. Without it the member still resolved by NAME through `lookupCore`'s
lexical chain — the class-body scope binds `m` two scopes up — merely at
lower confidence. An owned-but-unbound receiver is a definitive negative,
not a miss, so it must not reach a receiver-blind fallback.
Also fixed: `function*(){}` as an expression was not a `@scope.function` at
all, so `this` inside one read as the enclosing method's.
WHAT THIS GIVES UP. The fix REMOVES edges, and some were correct:
`.bind(this)`, `.call(this)` and `forEach(fn, thisArg)` do make `this` the
instance at runtime. Their correctness is fixed at the CALL SITE, which no
scope-level rule can see, so the choice is between losing them and keeping
every detached-callback false positive. All three are pinned as tests
asserting the empty result, so changing the trade later is deliberate.
`this` in a static method also stops resolving to the INSTANCE member — that
edge was wrong in the other direction.
INVALIDATION. Both constants move, and the parse-cache one is not optional:
`ownsReceivers` lives on the cached `Scope`, and a warm cache replays scopes
without it — verified by probe that `--force` alone does NOT re-derive it, so
the fix silently did nothing until SCHEMA_BUMP moved. INCREMENTAL_SCHEMA_
VERSION 16 -> 17 (the incremental write set covers only changed files, so
unchanged TS/JS files would keep their fabricated `this` edges);
SCHEMA_BUMP 23 -> 24.
Verified against a built index, not by reading: all three false edges from the
issue gone, every correct edge kept, same result in JavaScript through its
separate grammar. 64 tests green across the new suite plus the closure-binding
and schema-version suites. The full suite's 36 failures are pre-existing
load-flakes — confirmed by A/B: `skip-git-cli` fails FOUR tests on a clean
HEAD versus three with this change, and `pipeline-pdg-streaming` passes in
isolation either way.
Refs #2701
* fix(ingestion): give function-local callables their own identity (#2699)
Graph node ids were file-scoped, so a local callable and a same-named
file-level one collapsed onto ONE node. That is a wrong answer, not a missing
one — the local call was attributed to the file-level symbol:
export function save(x) { return x; }
export function run() { const save = x => x * 2; return save(1); }
export function other() { const save = x => x * 3; return save(2); }
// ONE node Function:a.ts:save, and BOTH run and other pointed at it, so
// `impact` on the top-level save reported two callers that never call it.
A local's identity is now its enclosing-callable chain plus its own position —
`run.save@2:2`. The chain is for humans reading `impact`; the position is what
makes it correct. Names alone cannot express what ECMAScript actually
specifies, and the gap is the language's, not the grammar's: an environment
record is created per function AND per block, so an anonymous function has no
name to contribute and sibling blocks hold distinct bindings under the same
name. One positional rule settles both, with no conditionals and no
"disambiguate only when it looks ambiguous" heuristic — the ambiguity-flag
class of bug that bit #2514. SCIP reaches the same place with its
document-scoped `local <id>` keyspace.
Top-level functions and class methods are NOT locals and keep their ids
byte-for-byte. That is the bound on the churn: this touches only symbols that
are unreachable from outside their own document anyway.
RESOLUTION JOINS BY POSITION, NOT BY NAME. `resolveDefGraphId` matches a def
to its node on (file, label, line, simple name). A def and its node are the
same construct, so this needs no scope chain at all — which is the point:
re-deriving the chain in the resolver would be a second implementation that
could silently disagree with the first. A genuine tie (two callables on one
line) stores an AMBIGUOUS_POSITION tombstone and falls through to the existing
name keys rather than picking by source order. Without this the node ids were
already correct and calls STILL resolved to the file-level symbol — the fix is
only half a fix without it.
JS/TS GAIN BLOCK SCOPES. They emitted no `@scope.block` at all, so the
resolver could not tell two `const pick` in sibling branches apart. Giving
them distinct ids made that visible as DUPLICATE edges — each call resolving
to BOTH — which is worse than the collapse it replaced. `(statement_block)
@scope.block` supplies the missing environment record. The other half of the
ECMAScript rule was already implemented and waiting: `tsBindingScopeFor`
hoists `var` past blocks to the enclosing Function/Module while `let`/`const`
bind innermost, and its docblock already claimed "the innermost default covers
these" for block scopes that did not exist. All 82 scope-resolution test files
pass with blocks on.
Verified by probe, per case: two locals in different functions, a local inside
an ANONYMOUS function (`outer.fn@1:9.save@2:4`), sibling blocks resolving to
their own binding, `var` still hoisting out of its block, a nested named
`function` vs a file-level one, PHP composing with the `$` sigil from #2693,
and Python. Top-level/method ids unchanged, asserted directly.
Every assertion is on the EDGE, not on node existence. Ids are built twice and
independently — definition phase and caller attribution — and a one-character
disagreement makes the caller attach to a node that does not exist and the
edge vanish, with nothing thrown and no test failing. An edge assertion can
only pass if both phases agree.
INVALIDATION. INCREMENTAL_SCHEMA_VERSION 17 -> 18 and SCHEMA_BUMP 24 -> 25:
persisted node ids change for every function-local callable, and the cached
scope tree lacks block scopes. A top-up would leave unchanged files on the old
ids while changed files emit the new ones, splitting each symbol in two.
Bench fingerprint unchanged and both timing budgets pass. The one full-suite
failure (incremental-orchestration) passes in isolation — its log shows stale
init locks and WAL reclaim, i.e. LadybugDB contention under the parallel run.
Refs #2699
* perf(ingestion): emit block scopes only where they bind something (#2699)
Block scopes make `let`/`const` in sibling blocks distinct bindings, which is
what stopped a call in one branch resolving to both. Emitted naively — one
scope per `statement_block` — they also cost ~10% of analyze wall time, because
every scope-chain walk in every function then steps through levels that bind
nothing.
Two emit-side filters keep the semantics and drop the waste:
1. A block that IS a function body duplicates the enclosing Function scope.
Nothing can be declared between a function and its own body, so a binding
in either resolves identically — the inner scope is pure depth.
2. A block that declares no `let`/`const`/`class`/`function` binds nothing,
so it is transparent: a lookup finds nothing in it and walks to the
parent. `var` is deliberately excluded from that list — it hoists past the
block to the function, so a block containing only `var` still binds
nothing.
MEASURED, on a 762-file / 228k-line TypeScript corpus (gitnexus/src), min of 6
warmed reps with the cold first rep discarded:
block scopes emitted 19,389 -> 5,331 (-72%)
total scopes 35,942 -> 21,884 (-39%)
analyze wall time +9.8% -> +1.6-2.5% vs pre-#2699
peak RSS (whole tree) 2398MB -> 2434MB (+1.5%, inside run-to-run noise)
The filters themselves are free: scope emission over the same corpus measured
12.6s naive vs 12.5s filtered.
Wall-clock on a shared runner has a ±10% spread run to run, which is wider than
the effect being optimised, so the durable gate added here counts scopes
instead. `bench/scope-emission/measure.mjs --check` asserts an EXACT scope set
over a synthetic corpus that mixes the shapes the filters discriminate between
— function/method/arrow bodies, non-declaring if/else/for/while/try, blocks
that declare `const`, and a `var`-only block. Baseline is 2 block scopes per
module: only the two `if`/`else` branches that declare `const chosen`. If the
filters regress that number jumps immediately, in a way wall-clock CI could
never resolve from noise. Wired into the existing benchmarks job.
Behaviour is unchanged: 86 scope-resolution and identity test files, 1371
tests, all green — including the sibling-block case this could plausibly have
broken — and the callable-value-flow fingerprint is untouched.
Refs #2699
* test(bench): re-baseline the TS/JS scope-capture fingerprints for #2701
`bench/scope-capture` fingerprints the full capture set per language, and
#2701 added a `@receiver-owner.this` marker to every non-arrow function form
so a scope that BINDS its own `this` can terminate the receiver walk. That is
a capture-set change, so the TypeScript and JavaScript fingerprints moved and
the benchmarks job has been failing since that commit — I pushed it without
checking CI.
A fingerprint is a correctness gate, so this does not simply adopt the new
value. Verified first by diffing the capture-name HISTOGRAM over the same
fixture corpus against
|
||
|
Gergő Magyar
|
89bbdcf566
|
fix(ingestion): stop double-indexing const X = () => {} as Function + edgeless Const twin (#2687) (#2691)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Devcontainer Smoke / Config-transform unit tests (push) Has been cancelled
Devcontainer Smoke / Build devcontainer image (push) Has been cancelled
Skill copy sync / shipped skills drift guard (push) Has been cancelled
|
||
|
Copilot
|
450cebc268
|
fix(java): JLS binary-name identities for local classes, enums, records & interfaces (#2562) (#2653)
* Initial plan * docs(plans): add Java local class naming plan * fix(java): model local class binary names * docs(java): clarify local class naming guards * fix(java): recognize local classes in compact constructors * chore: remove Java naming plan * fix(java): harden local type identities and scope * perf(java): linearize local type ordinal allocation * fix(java): harden ordinal benchmark follow-up * docs(java): clarify ordinal benchmark invariants * test(java): cover local type ownership paths --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
MyShining
|
4af6fe8587
|
feat(spring): resolve constructor and standard injection (#2632) | ||
|
FAll
|
2cfbc4a259
|
feat(spring): build bean candidate inventory (#2494)
* feat(java): inventory Spring bean candidates * fix(java): fail closed on Spring annotation shadowing * fix(java): resolve Spring beans after imports * fix(java): remove stale bean extraction path * style: satisfy locked Prettier version * fix(spring): address PR review findings * feat(spring): share bean inventory across Java and Kotlin * fix(spring): gate bean inventory analysis completeness * fix(kotlin): avoid reloading cached scope source * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
12600000e3
|
feat(java): model enum constant bodies as first-class instances; JLS 13.1 anonymous naming (#2558)
Some checks are pending
Scorecard / Scorecard analysis (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(java): JLS 13.1 immediate-host naming for anonymous bodies + v9 schema window (#2555, step 1) `synthesizeJavaAnonymousClassName` generalizes to both anonymous-body shapes (`object_creation_expression` with a `class_body`; `enum_constant` with a `body:` field) and switches from topmost-host naming to JLS 13.1 binary names: the `$`-joined chain of enclosing host types (`EnumWrap$Mode$1`), numbered per IMMEDIATE host in source order across both shapes (javac's shared counter). Every existing fixture's immediate host is its top-level type, so existing names are unchanged — proven by the 11 #2550 tests passing untouched, not assumed. The owner walk's anonymous branch also fires on `enum_constant` now (the synthesis returns undefined for body-less constants, so the walk continues to `enum_declaration` as before). Identity window: INCREMENTAL_SCHEMA_VERSION 8→9, parse-cache SCHEMA_BUMP 18→19, U-C5 pin extended with the v8-stamp rejection (enum-constant methods re-key `E.hook`→`E$1.hook`; nested-host anons re-key `EnumWrap$1`→`EnumWrap$Mode$1`). Enum-constant Class-node emission and scope-side ownership land in the next commits per docs/plans/2026-07-18-gitnexus-plan-enum-constant-bodies.md (plan is local — docs/ gitignored). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(java): model enum constant bodies as first-class instances (#2555, steps 2-4) `enum E { A { void hook(){} } }` — javac's other anonymous-class shape — joins the #2550 instance model: - Structure: `(enum_constant body: (class_body)) @definition.class` in JAVA_QUERIES; `enum_constant` in javaClassConfig.typeDeclarationNodes with extractName synthesis. The shouldSkipClassCapture guard now also covers enum_constant — without it, extract()'s name fallback would fabricate a Class node from the constant's own identifier (`A`). - Scope: `(enum_constant body: (class_body) @scope.class)` + synthesized `@declaration.class`/`@declaration.name` anchored on the body, so the constant's methods are owned (`ownerId`) and re-keyed (`Method:...:EnumConst$1.hook#0`). - Inheritance: a body-anchored `@reference.inherits` naming the HOST ENUM (javac semantics: E$N extends E) — `mroFor(E$N) ∋ E`, so bare calls from the body to enum helpers pass the ownership gate's MRO arm while the same-file bare-call leak for constant-body method names is closed (discrimination evidence: the #2549 review's archived S1b probe showed the identical shape resolving `local-call` pre-fix). - Nested-host JLS naming verified end-to-end: `EnumWrap$Mode$1` (not `EnumWrap$1`). - Bench: java scope-capture fingerprint rebaselined (new captures + two fixtures), `measure.mjs --check` PASS across all 14 languages. Verified: full java.test.ts 230/230 twice sequentially; TS 254 + JS/ Kotlin 289 (shared-file spot set); schema/scope/owner unit suites 90. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(java): exempt $-chain anonymous class defs from nested-class qualification (#2555 review) Review lens probe caught a HIGH collapse: same-named methods across sibling enum constant bodies attributed to the FIRST body's Method node (`M3$1.hook -> M3.log` where the log() call lives in C's body; the same-target sibling edge vanished entirely under dedup). Root cause: `populateClassOwnedMembers`'s qualifier chains a constant-body class def to `M3.M3$2` — its Class scope's parent is the enum's Class scope, unlike OCE anons whose parent is a Function scope — and its methods to `M3.M3$2.hook`. The structure-phase node id encodes `M3$2.hook`, so the graph-bridge's qualified key misses and falls to the file-wide simple-name lookup: first-write-wins. Fix: `qualify()` now skips CLASS-LIKE defs whose name already carries a `$` chain — a synthesized anonymous binary name is complete by construction (JLS 13.1). Narrowly scoped: `$`-named MEMBERS (legal and real in JS/TS) still qualify against their class, and named nested classes (`Outer.Inner`, #1978) are untouched. Discriminating regression test: same-name/distinct-target sibling bodies must each own their edge, and the misattributed cross-edge must not exist. Verified: full java.test.ts 231/231; Python+Kotlin 459 (heaviest populateClassOwnedMembers consumers) — zero assertion failures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(ci): prettier formatting + java bench rebaseline at the final corpus (#2555) Two CI reds from the review-fix commit landing AFTER the bench rebaseline: (1) prettier reformat of the new java.test.ts describe; (2) the java scope-capture fingerprint drifted again because the review fix added the java-enum-constant-same-name fixture to the corpus — rebaselined at the true final corpus (196 fixtures, ce104a76…, scaling 1.05 < 1.5), local `measure.mjs --check` PASS across all 14 languages. Lesson honored going forward: the bench rebaseline is the LAST artifact step — any post-review fixture addition reopens it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(java): strict JLS 13.1 chaining through anonymous enclosing types (#2555) Per review discussion: anonymous enclosing types now chain into the binary name instead of flattening to the nearest NAMED host — the immediately enclosing type per JLS 13.1 may itself be anonymous: - anon inside an anon: NestHost$1$1 (was NestHost$2) - anon inside an enum constant: N$1$1 (was N$2) - named nested hosts (unchanged): EnumWrap$Mode$1 `nearestJavaAnonHost` becomes `nearestJavaEnclosingType` (named hosts OR anonymous bodies); an anonymous enclosing type's prefix is its own synthesized name (memo-bounded recursion); numbering is per immediately enclosing type in source order. Top-level-hosted names are untouched — the full existing suite passes unchanged. New coverage: anon-in-anon chain, anon-in-constant-body chain (with ownership), and a bodied constant in a NESTED enum (EnumWrap2$Mode$1 — the one host combination previously untested). Rides the unreleased v9 identity window (doc wording tightened); java bench fingerprint rebaselined at the final corpus, `--check` PASS across 14 languages; prettier clean. Verified: full java.test.ts 234/234 (one worker-crash flake rerun green in isolation). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Gergő Magyar
|
1abcac9c16
|
fix(scope-resolution): stop platform builtins resolving to unrelated same-file symbols (#2549)
* fix(scope-resolution): stop platform builtins resolving to unrelated same-file symbols (#2545) An unqualified call to a platform/language builtin (e.g. TypeScript's global fetch()) could resolve to an unrelated same-file declaration sharing that name, most visibly a Cloudflare Worker's `export default { async fetch(req) {...} }` handler. Two contributing gaps, both fixed: - Object literals had no scope boundary in the TS/JS grammar queries, so a method's/property-arrow's name auto-hoisted past the literal into whatever lexically enclosed it (scope-extractor.ts's auto-hoist logic had nowhere to stop). Give object literals a Block scope, like 6 other languages already do for lexical blocks. - Independently, finalize's per-file bindings bucket (materializeBindings in gitnexus-shared) flattens every local declaration in a file onto its module scope for cross-file import resolution, regardless of true nesting -- so free-call-fallback's scope-chain walk could still hit the leaked binding at module scope. Guard free-call resolution: when a match for a known builtin name (LanguageProvider.isBuiltInName, already populated for TS/JS but never consulted by this pass) has no binding reachable via the true lexical scope chain, leave the call unresolved instead of emitting a false CALLS edge. Verified against the full TS/JS resolver suites plus every other language populating builtInNames (Python, Go, C/C++, C#, Dart, Kotlin, PHP, Ruby, Rust, Swift, Vue) -- 2333 tests, no regressions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(scope-resolution): extend the #2545 scope-leak fix to Kotlin and Java Anonymous object-expressions (Kotlin `object { ... }`) and anonymous class bodies (Java `new Runnable() { ... }`) have the same missing scope-boundary gap that caused #2545 in TypeScript/JavaScript: a method declared inside has no scope of its own to stop the auto-hoist at, so its name leaks past the container into the enclosing scope. - Kotlin: `(object_literal) @scope.class` (distinct from the already- scoped named `object_declaration`/`companion_object`). Kotlin already populates `builtInNames`, so free-call-fallback's isBuiltInName guard (added for #2545) fully closes the equivalent leak here too -- verified with a `println`-shadowing regression test. - Java: `(object_creation_expression (class_body) @scope.class)`, matching PHP's existing `anonymous_class` handling. Java has no `builtInNames` list, so the isBuiltInName guard doesn't engage -- the scope-tree fix is still correct and necessary (the anonymous class's own methods are now owned by the right scope), but an unqualified call to an unrelated same-file method sharing the anonymous class's method name can still resolve via finalize's per-file module-scope bucket (materializeBindings, shared/ language-agnostic, intentionally not touched by this PR). Documented in the test as a known residual gap, same as TS/JS/Kotlin's own non-builtin-name collisions. Audited every other language for the same shape (a value/container node with no @scope.* capture hosting a would-be-auto-hoisted named declaration): PHP and Vue already handle it correctly (PHP scopes anonymous_class; Vue's <script> delegates to the now-fixed TS/JS query). Ruby, Python, Dart, C#, Swift, Go, Rust, and C/C++ have no query pattern that treats a literal/container value position as a named declaration in the first place, so the bug shape can't occur there. Verified: full Kotlin + Java resolver suites, 468 tests, no regressions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(scope-resolution): dedicated Object scope kind for object literals (#2545, #2551) Review of the #2545 fix surfaced two defects, both fixed here: 1. The isBuiltInName guard suppressed genuine cross-file imports whose name matches a builtin (`import { fetch } from './fetch-polyfill'` silently stopped resolving -- verified regression vs. main). The leak the guard targets is inherently same-file (finalize's flat bucket is per-file), so the guard now also requires `fnDef.filePath === parsed.filePath`. New regression test covers the polyfill-import shape. 2. The sibling-property case of the reported bug was still broken and masked by a tautological assertion (`c.reason` -- a property that doesn't exist; the real path is `c.rel.reason` -- so the test passed regardless of behavior). In `export default { fetch() {...}, handler: () => fetch(...) }`, `handler`'s bare `fetch()` still resolved to its sibling. Reusing the `Block` scope kind was the root cause: correct for a real lexical block (a nested closure legitimately sees a sibling `let`/`const` from an enclosing `if`/`for`), wrong for object literals, whose members are reachable only via property access -- never as bare identifiers, not even by sibling property bodies. Fix: a dedicated `Object` ScopeKind (gitnexus-shared) -- a hoist boundary whose own bindings scope-chain walkers never consult while still traversing past it to the parent. TS/JS object literals now emit `@scope.object`; the four chain walkers in scope-resolution/scope/walkers.ts (walkScopeChain, findAllCallableBindingsInScope, findCallableBindingsAndAdlBlocker, findExportedDefByName) and free-call-fallback's hasGenuineLexicalBinding skip Object scopes' bindings. Kotlin's anonymous `object {}` keeps `@scope.class` -- unlike JS object literals it has real implicit-this sibling dispatch. Verified with the full resolver matrix run sequentially (TS 254, JS/ Kotlin/Java/Python/Go + TS variants 960, C/C++/C#/Dart/PHP/Ruby 1049, Rust/Swift/Vue/Cobol + route/flow/unit suites 828, scope-extractor/ scope-tree units 51). Worker-pool crashes under parallel suite load reproduced on unrelated files and pass in isolation (known flake, not caused by this change). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * feat(java): model anonymous class bodies as first-class Class nodes (#2550, step 1) `new Runnable() { public void run() {} }` now emits a synthesized javac-style `Class` node (`Worker$1`, `$N` = source order within the top-level class) and owns its methods: the enclosing-owner walk attributes `run` to `Worker$1` (re-keyed `Method:...:Worker$1.run#0`, HAS_METHOD from the anonymous class) instead of the lexically enclosing named class. - `synthesizeJavaAnonymousClassName` (ast-helpers): single naming authority for every layer that keys the anonymous class; returns undefined for `object_creation_expression` without a `class_body` child, which also keeps it a no-op for C#'s same-named node type. - `findEnclosingClassInfo`: anonymous-body branch before the generic container walk. - JAVA_QUERIES: `(object_creation_expression (class_body)) @definition.class` (no @name); `getLabelFromCaptures` now lets a nameless `definition.class` through — the parse-worker's existing `!nameNode && !extractedClassSymbol` gate still drops any nameless class the extractor cannot name, so other languages are unaffected. - `javaClassConfig.extractName` synthesizes the name on the extractor path (worker node emission). - Node identities move on unchanged files: INCREMENTAL_SCHEMA_VERSION 7→8 and parse-cache SCHEMA_BUMP 17→18 (the v5 Route-identity precedent) force full re-analyze / cache invalidation. Verified: new #2550 identity tests + resolve-enclosing-owner and has-method suites (53 tests) green. Prep for step 2/3 (scope-side ownership + receiver typeBinding) and the free-call instance-ownership gate per docs/plans/2026-07-18-gitnexus-plan-java-instance-scoped-freecalls.md (plan file is local — docs/ is gitignored by repo policy). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(java): instance-scoped free-call resolution for anonymous-class methods (#2550, steps 2-4) Completes the #2550 instance model on top of the Worker$N identity commit: - Scope-side ownership (java/captures.ts): synthesize `@declaration.class` + `@declaration.name` (`Worker$N`) anchored on the anonymous `class_body` — same range as its `@scope.class`, so the def lands in that Class scope's ownedDefs, `populateClassOwnedMembers` stamps `ownerId` on the anonymous class's methods, and the name auto-hoists exactly like a named class declaration. - Receiver typeBinding (java/captures.ts + type-extractors/jvm.ts): `Runnable handler = new Runnable() { ... }` binds `handler` to the ANONYMOUS class (`Worker$1`), not the declared JDK interface — in both the scope-side TypeRef channel (receiver-bound Case 4) and the worker typeEnv. `handler.run()` now resolves through the receiver path (reason 'global', target `Worker$1.run#0`) instead of depending on the free-call finalize-bucket leak — which is why the prior gate attempt broke it (the #2550 landmine, now explained and structurally removed). - Instance-ownership gate (free-call-fallback.ts + contract + run.ts + java opt-in): with `ScopeResolver.freeCallsRequireInstanceOwnership`, a free call may resolve to a `Method` only when the caller's enclosing class chain (self + MRO via `scopes.methodDispatch.mroFor`) contains the method's owner. Same-file matches only — the `materializeBindings` leak is per-file; cross-file Method matches come through genuine import channels (suppressing them broke the arity-narrowing parity suite, verified). Suppressions recorded as `'free-call-instance-ownership'` outcomes. Java opts in; every other language is byte-identical (flag off). Result on the #2545 fixture: `process()`'s bare `run()` emits NO edge to the unrelated anonymous method (the #2550 bug, closed), while `handler.run()`, same-class implicit-this dispatch, and bare inherited calls (MRO arm) all keep resolving. Verified: full java.test.ts 223/223 twice sequentially (landmine gate); cross-language matrix (TS/JS/Kotlin/Python/Go/C/C++/C#/Dart/PHP/Ruby/ Rust/Swift/Vue/Cobol + callable-value-flow + java-class-impact + core units) — zero assertion failures; worker-crash flakes re-verified green in single-file isolation. Known deferral (documented): EXTENDS/IMPLEMENTS edges from the anonymous class to its constructed type are not yet emitted, so a same-file inherited-but-not-overridden member called ON the anonymous instance does not resolve through the anon MRO; tracked as the follow-up in #2550. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(java): anonymous-class inheritance, host coverage, and phantom-node guard (#2550 review) Self-review of the instance model (gitnexus-review with empirical lens probes) surfaced three defects, all fixed: 1. HIGH — the ownership gate suppressed TRUE bare calls to inherited methods inside an anonymous body extending a same-file class (`new Base() { void extra() { work(); } }` lost `extra -> work`): the anon class had no inheritance edge, so `mroFor(Worker$N)` was empty and the MRO arm could never pass. The synthesis now emits an `@reference.inherits` for the constructed type, anchored on the `class_body` so the reference's enclosing class resolves to the SYNTHESIZED def (anchoring on the type node would sit outside the anonymous scope and attribute the edge to the wrong class). Anon classes now get real EXTENDS/IMPLEMENTS edges and inherited bare calls pass the gate. 2. MEDIUM — hostless anonymous bodies materialized a phantom Class node named after the CONSTRUCTED type (`Class:...:Runnable`) via extract()'s extractTypeNameFromNode fallback. New `shouldSkipClassCapture` in javaClassConfig drops the capture when no name can be synthesized. 3. MEDIUM — enum/interface/record-hosted anonymous bodies silently fell back to the pre-#2550 model (mis-attribution + open leak). The topmost-host walk now accepts all four host type declarations (JAVA_ANON_HOST_TYPES), so `EnumHost$1` etc. are modeled; the phantom-node shape disappears for those hosts as a side effect. Also: per-parse-tree WeakMap memo for the `$N` numbering — the helper is called from four independent layers per anonymous body and each call re-scanned the host subtree (`descendantsOfType`), quadratic on anon-heavy files (old-style listener-per-widget Java); and the scope-capture bench fingerprints rebaselined for java/typescript/ javascript/kotlin (`measure.mjs --check` now passes all 14 languages — it failed for every scope query this PR touched; drift notes added per the file's convention). Verified: full java.test.ts 225/225; all 11 #2550 tests including the new anon-extends-base and enum-host scenarios; bench --check PASS. Known remaining (documented, unchanged-old behavior): enum CONSTANT bodies (`A { ... }`) stay unmodeled; nested-host naming is top-level- anchored (`EnumWrap$1`, not javac's `EnumWrap$Mode$1`). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * test(storage): update the INCREMENTAL_SCHEMA_VERSION pin to v8 (#2550) The U-C5 reuse-gate test deliberately pins the exact schema version so a bump cannot land without consciously extending the gate expectations. Extend for v8 (Java anonymous-class node identities, #2550): a v7 stamp now fails the strict-equality reuse gate — a pre-v8 index would strand old `Worker.run`-keyed Method nodes alongside the re-keyed `Worker$N.run` ones on unchanged files — and v8 passes. Caught by CI (tests/ubuntu coverage shard 2/3 on PR #2549); the local matrix had not included this unit file. All 7 schema-referencing unit suites verified green (109 tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
ed8ab1c246
|
fix(scope-resolution): resolve callable reference flows (#2437) (#2522)
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* docs(plans): add provider-hook value-refs plan (#2437) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(plans): deepen #2437 plan to USES + property-dispatch design Design revised after prior-art research (Kythe ref vs ref/call, Joern METHOD_REF, Feldthaus field-based call graphs, CodeQL impliedReceiverStep): registration sites emit reference-class USES, invocation is recovered by a field-based property-dispatch pass synthesizing CALLS at member-call sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scope-resolution): model provider-hook value references (#2437) Functions referenced as object-literal property values (provider hooks like emitScopeCaptures: emitCppScopeCaptures) previously produced no edge at all, so impact/context reported a false-safe 0 upstream dependents. Two coordinated halves, per prior art (Kythe ref vs ref/call, Joern METHOD_REF, Feldthaus ICSE'13 field-based call graphs, CodeQL impliedReceiverStep): - Registration -> USES: new ReferenceKind 'value-ref'; TS/JS queries capture pair values and shorthand properties (with @reference.property-key); emitted as a reference-class USES edge, reason 'scope-resolution: value-ref'. Resolution is callable-gated so plain values emit nothing. - Dispatch -> CALLS: new shared pass emitPropertyDispatchCalls synthesizes CALLS (reason 'property-dispatch', confidence 0.7, per-key fan-out cap 32 calibrated on this repo's 16-provider hook tables) from member-call sites to every function registered under the same property key. Deviation from plan: the pass owns value-ref resolution entirely via the post-finalize findCallableBindingInScope walker — the shared registries only see pre-finalize local bindings, so imported hooks (the c-cpp.ts case) were unresolvable through lookupForSite; Reference.propertyKey passthrough dropped as unnecessary. SCHEMA_BUMP 13 -> 14: ParsedFile gains value-ref sites + propertyKey. Verified end-to-end: impact(emitCppScopeCaptures, upstream) now reports 8 impacted / HIGH with extractParsedFile (true dispatch caller) at d=1 via property-dispatch and the c-cpp.ts registration via USES. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(scope-resolution): cover value-ref registration and property dispatch (#2437) Integration: same-file/cross-file/aliased/shorthand registrations emit USES; non-callable and destructuring values emit nothing; dispatch sites gain property-dispatch CALLS (incl. JS twins and per-language partitioning); fan-out-capped keys are dropped entirely; factory-call values unchanged. Unit: capture-shape pins for @reference.value-ref + @reference.property-key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope-resolution): surface dropped property-dispatch keys in stats (#2437) Review finding: skippedKeys was returned but discarded — a hook table larger than the fan-out cap silently reopened the #2437 gap for those keys. Log dropped keys and fold value-ref USES + dispatch CALLS into referenceEdgesEmitted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(plans): add callable reference-flow implementation plan * fix(scope-resolution): close property-dispatch review gaps * feat(scope-resolution): add callable flow facts * feat(scope-resolution): resolve callable value flow * feat(scope-resolution): resolve callable references across providers * fix: harden callable reference flow resolution * fix(scope-resolution): preserve callable binding semantics * docs(plans): add pr-2522-review-fixes plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(storage): bump INCREMENTAL_SCHEMA_VERSION for callable-value-flow edges Callable-value-flow CALLS/USES edges (#2437) can connect two files whose content did not change, but the incremental write set only covers changed files — a top-up against a pre-v7 index would silently omit the new edges for every unchanged file pair, indefinitely. Force the one-time full re-analyze (review finding 1, #2522). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(storage): sanitize callable-flow sites per-site at load, log drops The load-time validator rejected the WHOLE ParsedFile when one site was malformed or over-bound, with no logging — and C++ legitimately emits empty-string parameterTypes entries ('' = unknown, the ReferenceSite.argumentTypes convention) for cv-only/ERROR-recovered types, so real repos fell into a permanent, silent warm-cache-miss reparse loop through the #1983-sensitive main-thread path (review finding 7, #2522). Now: '' entries are valid in type arrays; a malformed/over-bound site drops only itself (counted, warned once per load); only non-array garbage — evidence the serialization itself is untrustworthy — rejects the file. Deviation from plan §6 wording: validator-side tolerance replaces emit-side clamps — smaller diff, same asymmetry closed at the single chokepoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope-resolution): keep declarations in the union for reassigned callable cells The binding-lookup suppression for fact-constrained cells was wholesale: reassigning a declared function through its own name (greet = other; greet()) deferred the call to the solver, which then refused the lexical lookup that resolves the declaration — an unresolvable RHS yielded zero CALLS for a call that resolved pre-flow (review finding 8, #2522). Suppression now applies only to cells bound by FORMAL facts — its actual purpose (a parameter whose grammar emits no declaration binding must not adopt a same-named outer function). Copy/alias/store/load destinations keep their declaration as an inclusion seed (Andersen-style union). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope-resolution): count forfeited deferred sites in the budget-bailout warning On work-budget exhaustion the deferred invoke sites end the run with zero CALLS — free-call fallback and reference emission already skipped them — but the warning said 'ordinary graph emission remains untouched', which is false for exactly those sites. The warning context now carries the unresolved deferred-site count and the comment states the real cost (review finding: budget-bailout honesty, #2522). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scope-resolution): surface dropped property-dispatch keys in stats and warn payload The over-cap warning carried only a count; the dropped key NAMES were discarded and RunScopeResolutionStats had no field, so the PR-body claim 'includes them in resolver statistics' was unimplemented (review finding, #2522; reviewer ask on the fan-out cap). The warn payload now names up to 20 dropped keys and the stats carry propertyDispatchSkippedKeys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(scope-resolution): drop producer-less ownerQualifiedName from formal sites No capture emitter anywhere produces @callable-flow.owner-qualified-name — the solver branch consuming it was unreachable in production, yet the field was typed, parsed, validated, and unit-tested with hand-built input (review finding 16, #2522; YAGNI). Re-add with a real producer if C++ qualified member declarators ever need it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(scope-resolution): drop dead callable-flow knobs CallableFlowPassingMode 'callable-object' had no producer and no consumer distinguishing it, and CallableFlowCaptureOptions.extractCallArguments had no language providing it (unlike its live sibling extractCallCallee) — review finding 17, #2522 (YAGNI). The invocation-kind 'callable-object' is a different, live concept and stays. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): bind subscripted callable cells to the container, not the index terminalIdentifier iterates children in reverse, so tbl[i] = handler seeded the INDEX variable's cell (polluting a same-named formal) and tbl[i](7) looked up the callee under i in a different scope — no join, no CALLS edge for the classic function-pointer-array dispatch (review finding 12, #2522). Subscript nodes now recurse into their container field only, in both bindingIdentifier and terminalIdentifier, across the fielded grammars (C/C++/JS/TS/Python/Go/Java). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): make cross-function file-scope callable bindings resolvable Two stacked gaps killed the canonical C callback-registration pattern (fp assigned in init(), called in run()) — the exact #2437 false-safe this PR exists to fix (review finding H1, #2522): 1. isVisibleValueBinding only consulted assignment regions and formals, so a call in a function OTHER than the assigning one emitted no invoke fact. A declared callable-typed binding is now a value binding wherever its declaration is visible (visibleCallableSignature). 2. The C scope query had no @declaration.variable pattern for function- pointer declarators — void (*fp)(int); created no scope-tree binding, so the seed (init) and invoke (run) cells canonicalized to different keys and never joined. Both bare and initialized forms now bind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(c): detect variadic parameters via the named variadic_parameter node tree-sitter-c materializes '...' as a named variadic_parameter node; the anonymous-token checks never matched, so variadic function-pointer signatures were emitted with a wrong fixed arity and no '...' sentinel (review finding, #2522). C++ is unaffected ('...' stays an anonymous token there); the token checks remain for such grammars. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): emit invoke facts for field-stored callable member calls The C ops-vtable pattern (o->run = handler; o->run(1)) captured the store but never the call — the member path in emitCallFacts bailed for languages without protocol methods, and the value-binding index recorded the member store under the OBJECT's name ('o'), not the member's ('run') (review finding 11/M3, #2522). Member destinations now also record their terminal member name, and a member call whose name-cell has a visible store emits an indirect invoke — gated on the store so plain accessor calls (map.get) stay inert. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): disambiguate (obj->*ptr)() ERROR recovery by token order tree-sitter-cpp groups the recovered '->*' two ways depending on error-recovery cost (identifier lengths): [identifier, ERROR '->*m'] or [ERROR 'obj->*', identifier]. The recovery assumed the first shape, so the second silently swapped receiver/member and dropped the call site — the committed test passed only by name luck (review finding H2, #2522). The identifier's position relative to '->*' inside the ERROR now decides roles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): class members are never file-local in hasFileLocalCallableLinkage The name-keyed file-local set is populated from every static declaration, so an in-class 'static void make();' (external linkage — in-class static means no-instance) and any member sharing a name with a static free function were over-marked, refusing legitimate cross-file declaration/definition joins (review finding 13/M2, #2522). Method and Constructor defs now bypass the name-set, per the hook's own linkage-only contract. Deviation from plan step 13: the regression is a unit-level contract pin rather than an end-to-end join test — C++ merges out-of-line member definitions onto the member node by qualified identity, so the graph shape cannot discriminate the join refusal for members. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): classify parameter passing mode from the declarator chain only A whole-subtree scan for reference_declarator inverted copy vs alias: void reg(void (*cb)(int& out)) marked the by-value pointer cb as 'reference' because of the NESTED parameter's int&, making the solver back-propagate formal targets into every caller's argument cell — alias semantics for a copy (review finding 14/M5, #2522). The chain walk never descends into nested parameter lists; a reference anywhere ON the chain (int& x, void (*&cb)(int)) still aliases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ruby): bare identifiers are calls, not callable references Ruby parses a receiver-less zero-arg method call identically to a variable read, so 'action = process' — which CALLS process and stores its return — seeded action with the callable and minted a wrong CALLS edge from any dispatch through it, confirmed end-to-end (review finding 15/HIGH, #2522). New provider knob bareNamesAreCalls: a bare name that is not a provably local value binding and not an explicit reference form (method(:x), lambda/proc) emits no flow fact, on both the assignment and argument paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(go): pair multi-value := positionally instead of cross-wiring The shared field fallback took the FIRST LHS identifier and the LAST RHS identifier of Go's expression_list pair, cross-wiring 'a, b := f, g' and synthesizing a garbage comma-joined qualified name — the real relationships were silently dropped (review finding 16, #2522). extractAssignment may now return multiple pairs; Go pairs list entries positionally and emits nothing for a length mismatch (multi-return call RHS). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(java): drop get/test from callableProtocolMethods 'get' and 'test' collide with ubiquitous non-functional-interface APIs (Map/List/Optional/Future.get), so every ordinary container access emitted a spurious callable-object invoke fact — high-volume misleading graph facts with a cross-wiring risk on receiver-name reuse (review finding 17, #2522). Supplier.get/Predicate.test dispatch is deliberately traded away until the check can gate on the receiver's declared type. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(rust): pin the qualified-name no-degrade guard as a hard invariant Rust's scoped_identifier callable-reference capture over-includes unit enum variants and associated constants (Shape::Square seeds as if callable); they stay edge-free only because resolveSeedCandidates refuses to degrade an unresolved qualified name to a simple-name lookup (review finding 18, #2522). Capture-side type filtering would false-negative on tuple-variant constructors, so the guard IS the contract: documented as a hard invariant (Go's mis-shaped multi-value forms also rely on it) and pinned end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(php): remove nonexistent optional_parameter node type tree-sitter-php has no 'optional_parameter' — defaults ride on simple_parameter — so the entry was dead weight the #1920 literal gate does not cover for capture-option Sets (review finding 19, #2522). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cobol): detect procedure pointers on fixed-format sources Two stacked defects made the feature a no-op on classic sequence-numbered fixed format (review finding 20/H3, #2522): 1. parseDataItemClauses' USAGE alternation knew POINTER but not PROCEDURE-POINTER/FUNCTION-POINTER, so the dataItems filter was dead. 2. The raw-line fallback scanned UNCLEANED text, where the sequence number satisfied the leading digits and the LEVEL NUMBER got captured as the pointer name. It now scans preprocessed lines and requires a letter- initial name (COBOL data names must contain a letter). 161 COBOL preprocessor/copy-expander tests stay green; free-format matrix case unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cobol): skip comment lines in SET seed/copy scans A commented-out SET (indicator-column '*'/'/' or free-format '*>') produced a live seed and a false CALLS edge from dead code (review finding 21/M1, #2522). The scan now skips indicator-column comment lines and strips inline '*>' tails before matching. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(architecture): document callable-flow-only mode and skipped-key reporting The Callable-value flow section omitted scopeResolutionEdgeMode: 'callable-flow-only' — a real emit-pipeline branch that suppresses all ordinary emission for standalone providers (review finding 22, #2522) — and predated the skipped-key names/stats surfacing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(scope-resolution): correct value-ref resolution attribution and stale pdg-gating comments The value-ref contract comment claimed MethodRegistry resolution — the mechanism is the post-finalize findCallableBindingInScope walker owned by emitPropertyDispatchCalls (resolveReferenceSites skips these sites). Three 'only under --pdg' calleeIdSink comments were falsified by the #2437 gating change (callee-id-sink.ts's header was updated; these copies were missed). Review finding 23, #2522. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(ingestion): direct unit coverage for synthesizeCallableFlowCaptures The 1,100-line shared synthesizer had no test naming it — only downstream consumers were covered (review finding 24, #2522). Pins seed/invoke/ formal/argument emission, subscript container binding, store-gated member invokes, produced-value guards, and the bareNamesAreCalls knob over a minimal options object so assertions target the synthesizer's own semantics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(resolvers): deepen shallow-language coverage; fix Kotlin/Swift reassignment gaps it exposed Adds the COBOL SET x TO y copy-branch scenario and conditional-assignment scenarios for Kotlin, C#, Swift, and Dart (10 languages previously had one generic case each — review finding 25, #2522). The new scenarios exposed two real capture gaps, fixed here: - tree-sitter-kotlin's 'assignment' node is fieldless, so nested reassignments (chosen = ::target inside a block) produced no flow facts; Kotlin's extractAssignment now decomposes it positionally. - tree-sitter-swift fields its assignment as target:/result:, neither in the shared fallback's field lists; both added. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(infra): literal-validation gate for callable-capture option Sets The #1920 gate validates query literals and exported configs but not the module-private *_CALLABLE_CAPTURE_OPTIONS Sets consumed by the shared synthesizer — a typo'd node type silently captures nothing (PHP shipped a dead 'optional_parameter'; review finding 26, #2522). Every <key>NodeTypes Set literal is now validated against its language's grammar; name-carrying sets (callableProtocolMethods, memberPointerOperators) are deliberately outside the contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(storage): centralize corrupt-fixture casts into makeStoreEntry The callable-flow store tests scattered 'as unknown as' double-casts per fixture (review finding 27, #2522; standing no-as-any rule). One typed helper now owns the single controlled escape hatch for building malformed serialization-boundary payloads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(bench): refresh capture fingerprints after review fixes python-scope: the committed baseline (8d5c3699) never matched this branch's code — CI's benchmarks arm was red on the PR head (review finding 2/HIGH, #2522); regenerated (a99e69ab), scaling 1.04 in budget. scope-capture: ruby/cpp/swift/java/kotlin drifted from the review-fix commits (bare-name suppression, passing modes + ->* recovery, assignment fields, protocol narrowing, positional assignment); all 14 languages re-verified PASS with ratios <= 1.18 against the 1.5 budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(docs): untrack docs/plans working documents docs/ is gitignored (local working docs); the plan files were force-added past the ignore. Untracked from the index only — they stay on disk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(golden): regenerate captures goldens after callable-flow review fixes The per-language digest guards (csharp/go/php/python/ruby/rust/swift) locked the pre-fix capture output; the review-fix series intentionally changed it — store-gated member invokes, subscript container binding, Ruby bare-name suppression, Swift assignment fields, positional pairing. Regenerated with UPDATE_GOLDEN=1; clean verification run 59/59; all other parity/golden guards (pipeline-graph, spring-route, python parity) pass untouched at 33/33. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): prototypes are callees, not callable value cells The cross-function visibility fix indexed EVERY signature-bearing declaration as a value binding — including plain function/method prototypes (void f(int);). Every call to a declared function then became an indirect invoke, and with emitCanonicalInvokeReference (C/C++) minted a free-call reference that resolved through the registry, bypassing the precise passes' two-phase/ambiguity/subobject suppression — eight phantom CALLS edges in the cpp resolver suite on CI. Only declarations whose binding identifier sits under a pointer/ parenthesized declarator (callable-typed variables like void (*fp)(int);) create value cells now. cpp resolver suite 331/331; callable-value-flow + C/C++ suites 181/181 (the cross-function fp regression still passes); cpp fingerprint rebaselined, both bench gates PASS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Eva
|
3d6908ba4b | fix(cache): bound parsedfile generations | ||
|
Gergő Magyar
|
5f4964b4e6
|
fix: resolve imported/composed FastAPI route path constants (#2391) (#2393)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(routes): add pure Python string-constant resolver (#2391 U1) * feat(routes): extract Python module constants from tree (#2391 U2) * feat(routes): capture non-literal FastAPI decorator args + per-file constants, bump parse-cache schema (#2391 U3) * feat(routes): resolve composed decorator route constants in parse-impl + skip floor (#2391 U4) * feat(routes): resolve composed FastAPI route constants in group HTTP-contract layer (#2391 U5) * test(routes): multi-hop, ingestion↔group parity, and warm-cache regression locks (#2391 U6) * docs(routes): mark the language-agnostic seam for cross-language const resolution (#2391) * refactor(routes): extract language-agnostic constant-fold core; Python becomes a binding (#2391) The fold, cycle guard, and depth cap now live in constant-resolver.ts and take a pluggable ImportResolver. python-const-resolver.ts supplies the Python import semantics + tree extractor and re-exports the same surface, so no call site changes. A Spring/Kotlin/C# binding can now reuse the core with its own resolver (proven by constant-resolver.test.ts driving it with a Java-style resolver). * fix(routes): treat the constant-fold cycle guard as a recursion stack (#2391) The `visited` set in `foldName` was added-to but never removed on unwind, so a constant referenced more than once in a single fold — `A + A`, a reused separator (`SLASH + PATH + SLASH`), or a diamond `X = P + Q` where P and Q share a base — tripped the cycle guard on its second occurrence and the whole route was silently dropped by the skip floor. Pop the guard in `finally` so it tracks the ACTIVE resolution stack, not every name ever seen: a true cycle (a name still on the stack) is still caught, but a name that already resolved and popped folds again. Re-computation stays bounded by MAX_RESOLVE_DEPTH, so no blowup is reintroduced. Locked in constant-resolver.test.ts (A+A, reused separator, shared-base diamond); the pre-existing real-cycle and depth-cap cases still return null. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): make module-constant binding writes mutually exclusive (#2391) `extractPythonModuleConstants` kept `literals`, `exprs`, and `imports` as three independent maps: `setName` cleared literals+exprs but never `imports`, and an import never cleared a prior literal/expr. Since `foldName` checks literals > exprs > imports regardless of source order, a name that was both imported and locally (re)assigned kept both bindings and the wrong one won — `from .c import ROUTE; ROUTE = os.getenv(...)` resolved the STALE import instead of dropping, a confidently wrong route path (the exact skip-floor invariant this feature is meant to uphold). Treat the three maps as one logical namespace: any write to one clears the other two for that name (via `imports.delete` in `setName` and a `bindImport` helper), so last-binding-in-source-order wins, matching Python. An import both imported and dynamically rebound now drops. Folding `+=`/`+` onto an imported base remains deferred (it drops safely, never a stale value). Locked in python-const-resolver.test.ts: dynamic-rebind drops, literal-shadows- import, import-shadows-literal, and `+=`-on-import drops. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): widen the group cost-gate to catch literal-leading concats (#2391) `NONLITERAL_ROUTE_DECORATOR_RE` required the first decorator argument to START with an identifier, so a string-literal-leading concat like `@router.get("/api" + SUFFIX)` never tripped `hasComposedRoute`. When such a route was the ONLY composed shape in a repo, the group layer left `constantsByFile` empty and dropped the route, while the ingestion side (which has no gate) resolved `/api/users` and emitted a Route node — an R4 provider/graph parity break. Widen the gate to also fire on a string-literal-leading `+`-concat, detected by a `+` before the closing paren on the decorator line. Gating on the `+` (not merely a leading quote) keeps a plain literal route `@router.get("/x")` OFF the gate, so a literal-only repo still pays no parse pass. Locked in fastapi-composed-provider.test.ts: a sole literal-leading concat now resolves (parseCalls>0 + provider emitted), plus previously-uncovered `@app.<verb>(CONST)` EXPR-branch resolution; the literal-only no-parse gate case still passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): correct package-init and over-deep relative import resolution (#2391) Two edges in `resolvePythonImport`: - `from . import X` (empty module after the dots) resolved to a sibling `<dir>.py` instead of the package `<dir>/__init__.py`. Resolve the bare-package case to `__init__.py`. - An over-deep relative import (more extra dots than the importing file has directory levels) silently clamped `dirOf('')` to `''` and could match an unrelated root-level `<name>.py` — a wrong file. Guard with `walk > depth → null` so an import that escapes above the repo root drops (skip floor). Both preserve the exact-match / ambiguity→null behavior for ordinary relative and absolute imports. Locked in python-const-resolver.test.ts: `from . import` → `__init__.py` (and null when absent), and an over-deep import returns null even when the clamped target file exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): bound parseConstOperands recursion depth (#2391) `parseConstOperands` recursed on `binary_operator` children with no depth bound. A stack overflow is not currently reachable (tree-sitter caps expression nesting below the JS stack limit, so it throws on a deep `+`-chain before this runs), but add a depth guard (cap 64, mirroring the fold engine's MAX_RESOLVE_DEPTH) as defense-in-depth: a pathological chain now floors to null (skip) rather than relying on tree-sitter's limit. The `depth` parameter defaults to 0, so all existing callers are unaffected. Locked in python-const-resolver.test.ts: a 100-term `+` chain yields no binding (null) instead of throwing; ordinary short chains still fold. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(routes): read each .py once in buildPythonRepoContext (#2391) The group repo-context builder read every `.py` file from disk twice: once in the `include_router` cross-file pre-pass and again in the #2391 constant cost-gate loop — an unconditional 2x read on every Python repo, on every group extraction. Hoist a single read pass that populates one `pyContents` map (and computes the composed-route cost gate); both the include_router pre-pass and the constant-map pass now consume the cached content. Behavior-preserving — a literal-only repo still does one read and zero parses. Covered by the existing group unit + integration suites (R4 parity and include_router prefix joins unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(routes): tidy constant-resolver docs and declaration order (#2391) Three no-behavior nits from the PR #2393 review: - Name `conditional_expression` (`x if c else y`) in the `parseConstOperands` jsdoc list of shapes that deferred to null. - Move `NONLITERAL_ROUTE_DECORATOR_RE` above `buildPythonRepoContext`, which references it — it read as a forward reference before (runtime-safe, but confusing). - Correct the integration-test comment that called `/v2/api/v1/widgets/get` "ingestion-only garnish": the group side emits it too (asserted separately); the four paths in that block are the shared-parity set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(routes): fold `X += "…"` onto an imported base constant (#2391) Previously `from .c import BASE; BASE += "/v1"` dropped (the extractor could not represent "the imported prior value" as an operand without self-referencing X and tripping the cycle guard). Preserve the imported prior under a synthetic `$imp$N` key — `$` can never appear in a Python identifier, so it cannot collide with a real name — and reference it, so the augmented assignment folds to `<imported BASE>/v1`. Extractor-only: no change to the `Operand` type, the fold core, or the cache shape, so no SCHEMA_BUMP. An imported base that is itself unresolvable still drops (skip floor preserved — never a wrong path). Locked in python-const-resolver.test.ts: single and chained `+=` fold onto an imported base; an unresolvable base still drops. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(routes): resolve bare decorator constants via the by-name entry (#2391) The group `resolveExprArg` hand-built `[{ kind: 'ref', name }]` and called `resolveOperands` for a bare-constant decorator argument — exactly what the language-agnostic core's `resolveConstant(file, name, repo)` seam does. Call it directly for the identifier case. This gives the previously test-only by-name entry point a real production caller (it is the documented reuse seam for future JVM/other bindings), drops the synthetic operand construction, and lets the now- unused `Operand` type import go. Behavior-identical — the `+`-concat path still parses to an operand list and folds via `resolveOperands`. Guarded by the existing group provider suite (bare-constant and concat cases). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(routes): parse each .py once in buildPythonRepoContext (#2391) The repo-context builder ran two parse loops — the include_router prefix pre-pass and the #2391 constant-map pass — so an include_router file in a composed repo was tree-sitter-parsed twice. Merge them into a single pass that parses each `.py` at most once and feeds both extractions from the same tree; a file that needs neither pass is still not parsed at all (cost gates unchanged). Complements the earlier single-read-pass change (this is the single-parse counterpart). Behavior-preserving (prefixes, R4 parity, and cost gates verified by the group + integration suites). Locked with a parseCalls assertion: a file needing both passes is parsed once, not twice. Note: a cross-run (cross-process) constant-map cache — the other deferred perf idea — remains out of scope; it needs disk persistence + invalidation and would add hashing/IO cost on the common path, so it fails the minimal-change bar this single-parse dedup meets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): bound constant-fold work and output to prevent OOM (#2391) The `finally`-popped cycle guard (recursion-stack semantics) correctly folds diamonds/repeated refs, but popping the guard removed the accidental work cap the old seen-ever set provided: a wide shared-descendant DAG re-folds each child once per reference, and a self-multiplying concat (`X = A + A; A = B + B; …`) builds a genuinely exponential string. Reviewers reproduced ~16.8M folds escalating to `RangeError: Invalid string length` and heap OOM — and neither fold call site is wrapped in try/catch, so it crashed the whole phase rather than dropping the route. Two complementary bounds, both flooring to null (skip), never a wrong value: - a never-popped `memo` in `foldName` caps recomputation at O(nodes) (successes only — a null may be transient on a cyclic branch); - a `MAX_FOLD_LENGTH` (8192) cap in `foldExpr` drops a fold whose output grows past any real route path, bounding the string size the depth cap does not. Corrects the prior "≤ 2^8 folds" comment (output grows multiplicatively, not additively). Locked with a 64^4-fanout construction that now drops in ~ms instead of OOMing; diamonds/cycles/depth-cap behavior unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): snapshot assignment RHS refs at the assignment line (#2391) `ROUTE = BASE` was stored as a lazy `ref(BASE)`, resolved against BASE's FINAL binding. So `ROUTE = BASE; BASE += "/v1"` (or `ROUTE = API; API = "/other"`) resolved ROUTE to the MUTATED value — a confidently wrong path, since Python assigns by value at the `ROUTE =` line. This was latent for local constants at the base of this feature and the `+=`-on-import work extended it to imports. Snapshot each assignment/`+=` RHS reference to a bound name into that name's current frozen value at the assignment line (`freeze`/`snapshot`): a literal value, a copy of the current expr (whose refs are already frozen), or an import preserved under a `$imp$N` alias. Unbound refs (forward references) stay lazy. A later rebind of the aliased name can no longer change the earlier binding. `freeze` also unifies the previous `currentOps` + inline import-alias logic. Locked in python-const-resolver.test.ts: aliased-import-then-`+=`, aliased-local-then-`+=`, aliased-local-then-rebind all resolve to the pre-mutation value; normal reference chains still fold. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): fold group identifier args via resolveOperands for parity (#2391) Resolving a bare-constant decorator arg through `resolveConstant` entered `foldName` at depth 0, whereas the ingestion side folds `routePathOperands` through `resolveOperands([{ref}])`, entering at depth 1. At the MAX_RESOLVE_DEPTH boundary the group tolerated one more hop than ingestion, so a deep alias/re-export chain resolved in the group provider set but dropped from the graph Route nodes — an R4 parity break. Restore the operand-list path in the group so both subsystems share identical fold-entry depth. (`resolveConstant` reverts to the documented agnostic-core seam.) Locked in constant-resolver.test.ts: a 4-hop chain that `resolveOperands([ref])` drops but `resolveConstant` resolves, documenting why the group must use the operand-list entry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): match multiline literal-leading concats in the cost gate (#2391) `NONLITERAL_ROUTE_DECORATOR_RE` used `[^)\n]*` so it only saw a literal-leading `+`-concat when the `+` was on the same line as the opening quote. A Black-formatted `@router.get(\n "/api"\n + SUFFIX\n)` therefore failed the gate, and when it was the only composed route in a repo the group dropped it while ingestion (which parses the tree, not the raw line) resolved it — an R4 parity break. Drop the `\n` exclusion: `[^)]*` spans the wrapped argument but stays bounded by the decorator's own closing paren, so a plain literal route still never trips the gate. Locked in fastapi-composed-provider.test.ts with a multiline concat fixture. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): bump SCHEMA_BUMP for changed extractor output + E2E snapshot lock (#2391) `extractPythonModuleConstants` now emits DIFFERENT `moduleConstants` for the same source (binding mutual-exclusivity clears stale imports; RHS refs are snapshotted; `$imp$N` aliases). That output is cached verbatim in the parse cache, so a warm shard built at the pre-fix version would replay stale — in one case actively wrong — folded values, and the correctness fixes would silently no-op on upgrade. Bump SCHEMA_BUMP 11→12 to force re-extraction (same warm-cache-replay class the original 10→11 bump addressed for the field addition). Also adds the first end-to-end coverage for the new behavior through the real ingestion pipeline: app/snapshot.py aliases a constant (`SNAP = API_V1`) then mutates the source (`API_V1 += "/mutated"`), and the test asserts the Route node is `/api/v1`, never `/api/v1/mutated` — a case the pure-function unit tests covered but the pipeline did not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
ChunxueLi
|
1029a8ddd7
|
feat: add Spring DI resolver for @Autowired List<T> injection (#2200)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
* feat: add Spring DI resolver for @Autowired List<T> injection Addresses all P0/P1 findings from tri-review (#2200): - P0: Register INJECTS in RelationshipType union (compiles) - P0: Rewrite execute() to emit consumer→implementation edges from graph data only - P1: Register in VALID_RELATION_TYPES, single-pass O(N) indexes - P1: Java-only gate with early exit on non-Java repos - P1: Update FULL_ORDER golden test - 8 unit tests covering all edge cases * test: make VALID_RELATION_TYPES size assertion array-driven (no hardcoded count) The security test hardcoded toBe(16) for the relation type count, but PR #2200 added INJECTS, bumping it to 17. Replace the magic number with an EXPECTED_RELATION_TYPES array whose .length drives the size assertion, so future additions only need to append to the list. Fixes CI failure on PR #2200. * fix(ingestion): thread raw generic field types onto Property nodes so Spring DI matching works (review 4616076037 P0) Production declaredType is generics-stripped by design (extractSimpleTypeName: List<Shape> -> "List"), so the spring-di phase's anchored regexes could never match real extraction output — the phase was a silent no-op on every real Java repository, while its unit tests passed against hand-built node shapes. Add FieldInfo.rawDeclaredType captured verbatim from the field's type node (.text, generics and qualifiers preserved — same precedent as the JVM method extractor), thread it through both parse-worker Property sites, add it to the shared NodeProperties contract, and match on rawDeclaredType ONLY (no declaredType fallback: it can never match real data and would mask future plumbing regressions as quiet no-ops). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): gate Spring DI on real injection annotations, honest edge reason (review 4616076037 P1) Extract Java field annotations (shared extractAnnotations helper, moved verbatim from the method extractor) onto Property nodes and require @Autowired or @Inject before a collection field becomes an INJECTS candidate. Previously every edge's reason string fabricated "@Autowired" without any annotation ever being checked, and any plain collection field would have fanned out false edges once matching worked. @Resource is deliberately excluded: JSR-250 resolves by bean name first (defaulting to the field name), injecting a single named collection bean — the opposite of the collect-all-implementers fan-out INJECTS models. Pinned by a test. An annotated candidate missing rawDeclaredType now logs an isDev warning (plumbing-contract breach signal) instead of vanishing silently. SCHEMA_BUMP 9 -> 10: Property nodes gained rawDeclaredType + annotations; warm parse caches must invalidate or the DI phase silently no-ops on replayed pre-upgrade nodes (the #2038 trap). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ingestion): framework-neutral di phase + language-scoped Spring matcher registry (review 4616076037 P1) spring-di was the only pipeline phase naming a language in shared core/ingestion code (DoD.md language rule; the maintainer's direction is a generic DI solution). Split it: - di-extractors/spring.ts: the Spring matcher (annotation gate, collection type parse, @Resource exclusion rationale, framework-specific reason payload) — language-scoped home, mirroring route-extractors/. - di-extractors/index.ts: DI_MATCHERS, a single-valued ReadonlyMap<SupportedLanguages, DiFieldMatcher> mirroring the SCOPE_RESOLVERS registry shape sanctioned by AGENTS.md. Constructor injection deliberately out of scope; widen to arrays only when a second same-language framework lands. - pipeline-phases/di.ts (renamed from spring-di.ts): framework-neutral — routes Property nodes to registered matchers by node language via a typed guard, then runs the unchanged reverse-index fan-out. Zero language or framework names remain (grep-verified). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): language- and qualified-name-scoped interface resolution for DI fan-out (review 4616076037 P2) The interface index was built from ALL Interface nodes regardless of language, keyed by bare simple name with last-writer-wins overwrite — a polyglot repo with a TS and a Java 'Shape' could fan Java INJECTS edges into TypeScript classes, and two same-named Java interfaces in different packages silently collapsed to whichever parsed last (documented GitNexus bug class: #2054, PR #1956). Resolution is now per-language with qualifiedName as the primary key (Interface nodes already carry package-qualified qualifiedName); dotted element types resolve via qualifiedName, bare names via a per-language simple-name index that records ambiguity and fails CLOSED. Ambiguity skips are observable: DIOutput.ambiguousSkipped + an aggregated isDev debug log, so 'no DI fields' is distinguishable from 'all candidates ambiguous'. Same-package tiebreaking is a pinned, documented follow-up. Order-independence pinned by running collision tests in both insertion orders. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): depth-aware Spring collection-type parser for idiomatic generics (review 4616076037 P3) The two anchored regexes silently skipped idiomatic Spring shapes: Map<Pair<A,B>, IFoo> (nested-generic key broke the [^,]+ split), List<? extends IFoo> / List<? super IFoo> (bounded wildcards), java.util.List<IFoo> (qualified wrapper), and whitespace/multi-line declarations. Replace them with a small scanner: whitespace normalization, wrapper matched by last dotted segment, depth-aware top-level-comma split, wildcard bound stripping, and a final plain-dotted-type-name gate so anything else (nested-generic elements, arrays, unbounded wildcards, embedded comments, unbalanced brackets) fails closed. Every accept and reject is documented in the module docstring and pinned by 27 table-driven cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(integration): prove Spring DI end-to-end through the real pipeline (review 4616076037 P1) Both no-op incarnations of this feature shipped with a green unit suite because every test hand-built the exact graph shape the phase expected — no test ever ran real Java source through the actual extraction pipeline. Add test/integration/spring-di-pipeline.test.ts: real .java fixtures via runPipelineFromRepo, pinning (a) the extraction contract on the annotated field's Property node (declaredType 'List', rawDeclaredType 'List<IFoo>', annotations ['@Autowired']), (b) set-equality on ALL INJECTS edges (exactly Consumer->FooA and Consumer->FooB; the non-annotated 'plain' field of the same type contributes nothing; no self-edges), and (c) a negative-control fixture with no injection annotations producing zero INJECTS edges. Either historical regression fails at least one of these. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(incremental): register INJECTS across product surfaces + delete-before-writeback (review 4616076037 P2) INJECTS was allowlisted in VALID_RELATION_TYPES but invisible or unhandled everywhere else. Register it deliberately: - REL_TYPES (gitnexus-shared schema-constants): web-side validRelType() otherwise silently rejects INJECTS filters (CLI/web single source of truth). - mcp/tools.ts cypher edge list (agent-facing schema discovery). - isGraphWideRelType: INJECTS validity is a whole-program property — a change to a THIRD file (the interface, or a new/removed implementer) creates/invalidates edges between two untouched files (the TAINT_PATH / #2084 M4 U6 class), so incremental extraction must always re-include the full fresh set. - deleteAllInjects (lbug-adapter): mirrors deleteAllInterprocTaintPaths — COUNT-then-DELETE under withConnLock, benign missing-table carve-out, re-throw otherwise (CodeRelation has no PK and there is no read-side dedup; a fail-soft delete + re-add would silently duplicate rows). - run-analyze.ts: the delete is UNCONDITIONAL, next to the Communities delete — deliberately NOT inside the options.pdg block: the di phase runs on every persisting analyze while the graph-wide re-include is unconditional, so a pdg-gated delete would append without deleting on every non-pdg incremental run (N runs = N copies). - local-backend.ts comment: opt-in traversal by design (not in default impact()/context() lists; no IMPACT_RELATION_CONFIDENCE entry per the WRAPS/FETCHES precedent — edges carry their own 0.8). - ARCHITECTURE.md: 14 -> 15 phases, DAG diagram, phase table, skip-list. Note: the tools.ts edge list also predates WRAPS/QUERIES/USES — that drift is pre-existing and left for a follow-up. Idempotency pinned end-to-end: two successive incremental runs (real runFullAnalysis + real LadybugDB, unrelated-file touches) leave the INJECTS row count stable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: describe INJECTS' actual precondition; drop stale fixed-at-16 comments (review 4616076037 P3) The shared-schema doc for INJECTS claimed an @Autowired precondition the code (pre-fix) never checked, and hardwired Spring semantics into what is now a framework-neutral edge type. Reword: precondition is an injection annotation recognized by a per-language matcher in di-extractors/; framework specifics live in the reason payload, not the type contract. security.test.ts comments still said the allow-list size 'stays fixed at 16' (it is 17 and the assertion derives from EXPECTED_RELATION_TYPES). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: simplify DI surfaces — narrow matcher contract, dedup delete-alls, derive tools edge list Post-implementation simplification pass (4 review angles): - DiFieldMatch/CandidateField carried collectionType + matchedAnnotation that no consumer read (the matcher bakes both into reason) — narrowed to {elementTypeName, reason}. - parseElementTypeName had two guard branches fully subsumed by the final plain-dotted-type-name gate — deleted, rationale folded into the regex comment. - The three byte-identical delete-all-by-rel-type functions in lbug-adapter (TAINT_PATH / CALL_SUMMARY / INJECTS) are now one parameterized helper + thin wrappers with identical names, signatures, and message text (character-diff verified) — the missing-table regex and abort policy now live in exactly one place. - The cypher tool's hand-maintained edge-type list (already missing WRAPS/QUERIES/USES) is now derived from the canonical REL_TYPES — the drift class is gone rather than patched. - di phase: interface indexes are built only for languages that actually have candidates; test builder gained a rawDeclaredType opt-out replacing a hand-rolled node. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: apply Tier-2 review findings — qualified-name fail-closed, honest cypher docs, pinned delete contract, hook isolation - byQualifiedName was last-writer-wins on duplicate qualified names (reproduced: order-dependent INJECTS edges with ambiguousSkipped 0 — same package+interface duplicated across monorepo modules/source roots; Java qualifiedName has no file-path component). Both indexes now share the AMBIGUOUS fail-closed sentinel; order-flip test added. - The REL_TYPES-derived cypher edge list advertised pdg-gated types with no caveat (LLM queries on them silently return zero rows on default indexes) — caveat appended, INJECTS example added, impact relationTypes description now names the DI fan-out opt-in. - The delete-all re-throw contract (only defense against duplicate CodeRelation rows) was untested — error classification extracted to a pure classifyDeleteAllError and pinned exhaustively. - extractRawType/extractAnnotations hooks lacked the per-hook try/catch the pipeline applies elsewhere (#2286 pattern): a throwing hook would silently drop every remaining file in the language group. Hardened, degradation tested. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Parafee41
|
7ca7166b8e
|
fix(fastapi): apply APIRouter constructor prefixes (#2312)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
henry201605
|
d7ff76e6e9
|
fix(ingestion/routes): resolve Spring interface-inherited routes (#2288) (#2290) | ||
|
henry201605
|
b16ec344f7
|
perf(group/http): skip source parse for graph-covered route files (#2138 Part 2) (#2265)
* feat(routes): resolve + persist handler symbol on Route nodes (#2138 part 2, WIP) Part 2 groundwork for #2138: give the graph-assisted HTTP provider path the handler symbol directly, so it no longer re-parses source to recover the handler name. (The remaining parse-skip in extract() + a call-count benchmark land in a follow-up commit.) - ExtractedDecoratorRoute gains `handlerName`; the Spring extractor captures the decorated method's name (the method_declaration node is in hand). - New `resolveRouteHandlerSymbols` (call-processor) resolves each route's handler to a real symbol UID, keyed by normalized route URL — Laravel framework routes (controller + method) and decorator routes (Spring/FastAPI) both reduce to `(filePath, name) -> nodeId`. Threaded through the parse phase onto `ParseOutput.routeHandlerSymbols`. - routes phase stamps `Route.handlerSymbolId`; persisted end-to-end (schema + Route CSV row + getCopyQuery COPY columns), mirroring Part 1's `method`. - HttpRouteExtractor: `HANDLES_ROUTE_QUERY` returns `handlerSymbolId`; `extractProvidersGraph` uses it as the authoritative symbol and SKIPS `getDetections()` for resolved rows (CONTAINS is a cheap graph lookup for the display name only — no tree-sitter parse). Fully backward compatible: an unresolved/old-index route with no `handlerSymbolId` keeps the source-scan fallback. - Extracted `normalizeExtractedRoutePath` to `route-extractors/route-path.ts` (shared by routes phase + resolver without an import cycle). - SCHEMA_BUMP 6->7 (ParseWorkerResult gained `handlerName`); regenerated the emit-persistence byte-identity baseline (route.csv header gained two columns). - Tests: Spring pipeline asserts the Route node carries a handlerSymbolId resolving to the handler method; extractor fast-path test proves the handler resolves with zero source detections. Refs #2138 * perf(group/http): skip source parse for graph-covered route files (#2138 Part 2) Builds on the persisted Route.handlerSymbolId (U0–U3a). When a file's HANDLES_ROUTE rows all resolve a handler symbol AND its language plugin declares routeCoverage: 'complete' (Java/Python/PHP), the graph is authoritative for that file's providers, so the source scan + tree-sitter parse can be skipped — the scan would only re-discover routes the graph already has. This is the measurable parse reduction #2167 could not show. Consumer safety: routeCoverage: 'complete' asserts *provider* Route-node completeness only. The scan() of those same languages also emits consumer detections (RestTemplate/WebClient/OkHttp/Feign, Guzzle/Http::, requests/httpx), and ingestion's FETCHES edges are JS/TS-only — so the graph cannot back up server-side consumers. A provider-covered controller that also calls out would otherwise lose its consumer contract. Guarded by a cheap, parse-free text gate. - types: HttpLanguagePlugin gains - routeCoverage?: 'complete' | 'partial' (default 'partial') - hasConsumerSignals?(content): false only when the raw source provably has no outbound-HTTP call this plugin detects (conservative). - java/python/php: mark routeCoverage 'complete' + implement hasConsumerSignals with a token regex over their consumer idioms. - http-route-extractor: run the graph provider pass first to build a coveredFiles set; then keep a file covered only when hasConsumerSignals(content) === false (read via readSafe, no parse). scanFiles = files not covered → drives collectProjectDetections + both source scans. Fail-open per file: any unresolved row, a 'partial' language, a positive consumer signal, a missing hook, or an unreadable file leaves the file in the scan set. The orchestrator names no languages — token knowledge stays in the plugins. Net: pure-provider controllers skip the parse (the win); controllers that also call out are still parsed (no consumer loss); partial-coverage languages and graph-less runs are unchanged. - test: route-parse-skip integration test spies the real parseSourceSafe to COUNT parses over a temp repo of Spring controllers with a mock DB — baseline (every file parsed), fully-covered (0 parses), mixed (unresolved file falls back, resolved stays skipped), and provider+consumer (a covered controller that also calls restTemplate is parsed; its consumer contract survives). * fix(group/http): cover Spring HTTP Interface @*Exchange in Java consumer-signal gate #2254 (merged) added Spring 6 HTTP Interface `@(Get|...)Exchange` / `@HttpExchange` as a new Java *consumer* idiom. The #2138 parse-skip consumer-safety gate must recognize it, or a provider-covered file carrying an `@GetExchange` could be parse-skipped and lose that consumer contract. Add `Exchange` to JAVA_HTTP_PLUGIN.hasConsumerSignals (conservative; also matches `restTemplate.exchange(`). * style(group/http): prettier formatting for #2138 Part 2 files * style(ingestion): prettier formatting for call-processor.ts (#2138 Part 2) * fix(group/http): P1 (Java over-claim) + P2 (handler mis-attribution) on top of #2268 (#2138 Part 2) Re-applied on the maintainer's #2268 (expanded Java/Kotlin consumer extraction) base. P1 — `routeCoverage: 'complete'` over-claimed for Java: the graph provider set is a strict subset of the group scan (array-form `@GetMapping({...})`, interface-inherited routes, same-URL multi-verb have no graph Route node), so parse-skip could drop those group-only providers. - java/python → default 'partial' (always source-scanned). Java flips to 'complete' only once ingestion provider extraction matches the group scan (a separate follow-up). Python was a no-op anyway (no handlerName resolved); 'complete' was a latent trap. PHP stays 'complete' (Laravel ingestion ⊇ the group scan, the one language the skip engages for). - python hasConsumerSignals widened to a true superset of scan() (uri=/url= wrapper, aiohttp, urllib). Java's gate already covers #2268's consumer set (same receivers; the @*Exchange token is present). P2 — resolveRouteHandlerSymbols: reserve the URL slot on first encounter even when unresolved (mirrors addRoute first-writer-wins, so a later same-URL route can't stamp the node-winner's slot); refuse to guess on an ambiguous same-name lookup (exactly one match → use it; zero/many → fail-open, never a wrong handler). The cross-source case (filesystem route winning a URL a framework route also normalizes to) is unchanged — the resolver never receives filesystem routes — and stays fail-open. Tests: - route-parse-skip rewritten: the parse-skip win is proven on PHP (fully covered → 0 parses; mixed fallback; consumer-covered file still parsed), plus three Java P1 regression guards (array-form / interface-inherited / multi-verb) asserting the group-only routes survive — verified they go red if Java is flipped back to 'complete'. - resolve-route-handler-symbols: direct unit tests (the fn had none) — unique resolve, ambiguous/unknown fail-open, same-URL reservation, first-writer-wins. - http-consumer-signals: each plugin's hasConsumerSignals is a superset of its scan() consumer idioms; pure providers return false. - route-handler-symbol-roundtrip: real-LadybugDB CSV→COPY→query for Route.handlerSymbolId. --------- Co-authored-by: henry <zhangwei2017@unipus.cn> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
78b4077d8a
|
feat(impact): opt-in PDG-backed impact mode - statement + inter-procedural slicing, resolved-callee-id soundness, mutation-oracle validated (#2227)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
Gergő Magyar
|
14397dd4aa
|
feat(taint): intra-procedural taint analysis (#2083) (#2164)
* feat(taint): harvest occurrence-tagged call/member sites on StatementFacts (#2083 U1) Worker-side site harvest in TsHarvester: call/new/member-read records with dotted callee paths, receiver slots, per-argument occurrence tagging with nested-site links, per-declarator resultDefs, spread/template/require-literal markers. hasTaintSafeSites validation seam. The pdg parse-cache chunk-key namespace is versioned (pdg:1 -> pdg:2) instead of a global SCHEMA_BUMP so flag-off users keep warm caches; bench fingerprints re-baselined for the three call-bearing scenarios (straight-line/dense-bindings byte-unchanged). * feat(taint): built-in TS/JS source/sink/sanitizer model + site matcher (#2083 U2) Typed spec (kind taxonomy; sanitizers carry neutralizes-kinds), the canonical Express/Node model, and matchFunctionSites: ESM alias/namespace + require- literal callee resolution, bare-name fallback restricted to true globals, sanitizers module-or-global only (never user-shadowable by name), spread/ template arg-position rules, deterministic taintModelVersion. * feat(taint): pure intra-procedural taint propagation engine (#2083 U3) Two-rule model (statement-local + du-fact worklist) with per-taint neutralized-kind exclusion sets: sanitizers exclude only the sink kinds they neutralize (escape(req.body) suppresses res.send but still fires db.query; exec(path.basename(t)) fires), intersection-over-paths so a bypass occurrence keeps the taint live, kill locality on resultDefs, propagate-through args+receiver with viaCall hops, one path per finding, deterministic caps, coverage-gap statuses. Test-first: 38 scenarios on real harvested CFGs. * feat(taint): thread taint caps + model version through pdg config/meta (#2083 U5) resolvePdgConfig gains maxTaintFindingsPerFunction (200), maxTaintHops (32), and the taintModelVersion digest; RepoMeta.pdg + RunScopeResolutionInput surfaces added. The key-union comparator trips full writeback on M2->M3 upgrade and on model-version change without --force (mode-flip tested). No CLI flags or rc keys (programmatic parity with the other caps). * feat(taint): in-phase taint emit with sparse TAINTED/SANITIZES edges (#2083 U4) run.ts pdg window: match-first fast path (solver only when a function has both a matched source and sink) -> computeReachingDefs with the shared RD fact derivation -> computeTaintFlows -> per-finding TAINTED (versioned hop-encoded reason via the shared path codec, statement-level occurrence identity) + per-kill SANITIZES, dedup-before-budget, truncate-and-warn. All emit counters surfaced (aggregate warn for gaps/drops, debug for volume); PROF gains taint=. Flag-off golden untouched. * feat(mcp): explain tool for persisted taint findings (#2083 U6) Anchorless calls enumerate the sparse TAINTED table (bounded, deterministic, limit-clamped); anchored calls (file or symbol via resolveSymbolCandidates) return full decoded hop detail. sinkKind rides a version-1 codec header (1;<kind>|hops — no other persisted channel exists; U4/U6 ship together). RepoMeta.pdg probe yields a no-taint-layer note instead of an error. TAINTED/SANITIZES pinned OUT of VALID_RELATION_TYPES (KTD9a negative- membership tests); generators + canonical skill docs + mirrors updated. * test(taint): acceptance fixture battery, snapshots, and bench gates (#2083 U7) pdg-repo taint-cases fixtures complete the six plan shapes; committed findings/kills snapshot via a shared pure-path harness that also feeds the AE2 exact-equality assertion (stored TAINTED == pure-path findings, the no-explosion gate). New taint-dense bench scenario with four --check gates: per-function findings pinned AT the cap, absolute reason-byte + site-bytes disk ceilings (the load-bearing R10 gate), zero-match pass < 0.5x match- dense, N-linearity. Pre-existing scenario baselines untouched. * refactor(taint): share one pointKey helper across propagate + emit (#2083 review) Extract pointKey(ProgramPoint) to cfg/reaching-defs.ts (colon-separated, matching the codebase block:stmt id convention) and import it in both propagate.ts and emit.ts, replacing the two divergent locals (':' vs '.'). Edge-id material now uses the colon form; ids are in-memory only and no test asserts the pointKey segment shape. * fix(taint): discriminate taint state by source occurrence (#2083 review) Two distinct sources flowing into one variable at one def point no longer collapse to a single TAINTED edge: the taint-state key gains a root source-occurrence discriminator ({point, siteIndex} — the same fields recordFinding's identity uses, excluding kind). Def->use fact lookup keys on the source-independent (binding, def-point) portion. Same-source multi-path flows still share one state so their exclusion sets intersect (the raw arm soundly wins); termination holds (finite keys, monotone shrink, no cross-source ping-pong). Restores the KTD6 identity contract. * fix(mcp): route dotted symbol names in explain to symbol resolution (#2083 review) The fileish classifier matched any dotted name (UserController.create) as a file via its extension-like suffix, so symbol resolution never ran and the tool returned a silent empty file-anchored result. Tighten the classifier to require a path separator or a real source extension (derived from the resolver's EXTENSIONS list, multi-language), so dotted/bare names route to resolveSymbolCandidates (found / ambiguous / not-found). * fix(mcp): gate explain no-taint-layer note on taintModelVersion (#2083 review) An M1/M2-era --pdg index has meta.pdg defined (BasicBlock/REACHING_DEF recorded) but no taintModelVersion and zero TAINTED rows. The probe keyed on generic meta.pdg presence, so explain returned the generic empty note instead of the actionable 'no taint layer — run analyze' hint. Gate on meta.pdg?.taintModelVersion (the field M3 stamps) so an M2-era index gets the layer hint; a taint-stamped index with no findings still gets the generic note. * fix(taint): sequence-expression value flows only the final operand (#2083 review) A comma expression in value position (exec((log(x), 'safe'))) default- descended, fanning every operand's occurrences into the enclosing sink argument — over-tainting exec's arg 0 with x. Add an explicit walkValue case that records earlier operands' uses with occurrence fan-out suppressed (new FactAccumulator.suppressOccurrences) and routes only the last operand through the value path. Sites-layer only; defs/uses/mayDefs byte-identical (cfg + reaching-defs snapshots unchanged). * perf(taint): FIFO head-cursor worklist + dedup before chainHops (#2083 review) Replace queue.shift() (O(N) dequeue) with a strict-FIFO head cursor plus order-preserving prefix reclamation; FIFO is load-bearing because chainHops reads the live taints map whose parent/source/viaCall are rewritten order-sensitively on monotone shrink, so hop determinism is dequeue-order contingent. Extract findingKey() and dedup-check before chainHops in the justify branch — already-recorded identities discard their hop chain (first write wins), so the ancestry walk was pure waste. The else kill branch is untouched. Findings + hops byte-identical (snapshot unchanged). * perf(taint): O(1) member-read dedup via composite-key set (#2083 review) addMemberRead rescanned the whole per-statement sites array per call to dedup by (object, property, parent) — O(n^2) on member-read-dense statements. Track a composite-key Set alongside sites for O(1) dedup. (The require-literal join is already O(sites) with a no-op body on non-require sites, so no early-exit is needed there.) Behavior identical: harvest + model-match + taint snapshots unchanged. * refactor(taint): drop test-only export; source taint caps via emit.ts (#2083 review) Remove the sanitizerNeutralizes export (its only consumers were two test assertions — inlined to entry.neutralizes membership). Re-export the DEFAULT_PDG_MAX_TAINT_* caps from emit.ts and point run.ts at emit.ts, so the pipeline's taint dependency surface is the single orchestration module rather than reaching into propagate.ts. * test(taint): extract the shared TS CFG/taint test harness (#2083 review) The parse/collectFunctions/cfgOf/cfgsOf/importsFor harness was copied byte-for-byte across four suites (harvest, model-match, propagate, taint-emit). Promote it to test/helpers/ts-cfg-harness.ts and import it. site-safety/reaching-defs carry a structurally different inlined builder and are left as-is. Pure extraction, no assertion changes. * test(mcp): harden explain limit-rejection battery (#2083 review) Add NaN, Infinity, -Infinity, and a numeric string to the out-of-bounds limit cases — a regression fence over the interpolated LIMIT, confirming the Number.isInteger guard rejects every non-integer/non-finite/string input before it reaches the query. |
||
|
Gergő Magyar
|
bde340a5b4
|
feat(cfg): intra-procedural REACHING_DEF data-dependence layer (#2082) (#2160)
Some checks failed
Devcontainer Smoke / Config-transform unit tests (push) Has been cancelled
Devcontainer Smoke / Build devcontainer image (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Gitleaks / gitleaks (push) Has been cancelled
Publish / Classify release event (push) Has been cancelled
Scorecard / Scorecard analysis (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-cli) (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-web) (push) Has been cancelled
Publish / RC guard (marker + release-PR skip) (push) Has been cancelled
Publish / ci (push) Has been cancelled
Publish / Publish to npm (push) Has been cancelled
Publish / Build & Push RC Docker images (push) Has been cancelled
* fix(cfg): route early exits through finally with target-relative threading (#2082 U2) * feat(cfg): harvest per-statement def/use facts into the side channel (#2082 U1) * feat(cfg): add reaching-definitions solver with GEN/KILL fixpoint + statement sweep (#2082 U3) * feat(cfg): persist budgeted REACHING_DEF projection with RepoMeta coherence (#2082 U4) * test(cfg): REACHING_DEF snapshot, pipeline both-sinks, and cache-seam coverage (#2082 U5) * bench(cfg): reaching-defs scaling gates — dense-bindings + fact-fanout scenarios (#2082 U6) * fix(mcp): exclude BasicBlock pseudo-symbols from detect_changes on pdg indexes (#2082 U7) * style: prettier pass over M2 files * fix(cfg): review-pass fixes — defKey overflow guard, catch-param block, class defs, intra-statement reads, graceful fact degradation (#2082) - reaching-defs: STMT_STRIDE 2^16→2^21 + upfront aliasing bail-out; a use that shares its statement with a def now also sees the same-statement def (assign-and-test idiom was a taint false negative); drop dead posInOrder - visitor: catch-param def gets its own once-executed block (prepending into a loop-header entry re-genned per iteration and killed loop-carried redefs); unresolved-label jumps now thread all active finallys; the finalizer-threading protocol moved to control-flow-context as shared helpers for future language visitors - harvest: class declarations def their name (was a bogus use in JS, silent skip in TS); class-expression names stay internal - emit: isEmitSafeCfg adds index==position contiguity; fact validation split into hasEmitSafeFacts so malformed facts degrade to CFG-only instead of dropping the function's whole CFG layer; facts-per-edge multiplier single source; lazy top-binding tally; dead solveMs removed - run-analyze: pdgModeMismatch compares the key union structurally — new resolved knobs join the comparison automatically - mcp: BasicBlock exclusion via id prefix (NULL-name rows of real symbols are no longer dropped) + same filter on the BM25 filePath fallback - bench: rd ratio denominator clamped (gate no longer self-disables at fast small-N); PROF-gated pdg timing in run.ts * test(run-analyze): model the M2 RepoMeta.pdg stamp in resolvePdgConfig defaults The DEFAULTS constant lacked the maxReachingDefEdgesPerFunction field that resolvePdgConfig resolves since the M2 stamp landed, failing two strict toEqual expectations (the CI 'tests' job failures). Models M2 steady-state equality; the M1-era-stamp upgrade path stays pinned in pdg-mode-flip.test.ts. Finding P1-4 of review 4471987625 (#2160). * test(cfg): reassign the shadowing fixture's bindings — fixes prefer-const CI errors Both withShadowing let bindings now genuinely reassign (s = s + 1 per scope), clearing the two prefer-const errors that failed quality/lint. Plain const would change the binding kind the harvest test exercises; reassignment keeps the let semantics and enriches the reaching-defs facts the snapshot pins (snapshot + per-binding assertion updated accordingly). Finding P2-6 of review 4471987625 (#2160). * fix(cfg): validate entry/exit indices in the emit-safety guard A corrupted side-channel element with an out-of-range entryIndex passed isEmitSafeCfg and threw inside the reaching-defs RPO walk — caught by the per-FILE try/catch, costing every sibling function's REACHING_DEF projection instead of the one element (and logging a misleading message). entry/exit join the guard's id-anchor checks. Finding P3 (entryIndex) of review 4471987625 (#2160). * fix(cfg): report the def-key stride bail-out as a distinct 'overflow' status The STMT_STRIDE aliasing guard reused status 'truncated', so the emit warn misnamed it as the fact-materialization limit (printing an unrelated maxFacts value, including '(0)' when unlimited) and telemetry conflated the two. A distinct 'overflow' status gets its own warn naming the actual cause; the function's CFG layer is explicitly unaffected. Finding P3 (stride-bail diagnosis) of review 4471987625 (#2160). * perf(cfg): cache the nearest enclosing scope per node during the prescan resolve() walked the AST parent chain per identifier — O(expression nesting depth), quadratic on deeply-chained single-statement expressions in generated code (not caught by any bench scenario, which scale blocks/bindings, not expression depth). The prescan already visits every node once, so caching its innermost scope makes phase-2 resolution O(scope-chain). Behavior-identical; the parent-chain walk survives as fallback for prescan-unvisited nodes. Finding P2 (resolve depth walk) of review 4471987625 (#2160). * fix(cfg): stop harvesting initializer-less var declarators as defs A bare `var x;` mid-function is hoisted and writes nothing at runtime, but the harvester recorded a def — fabricating a kill of the live def in the same block: `x = source(); var x; sink(x)` lost the source→sink fact (a reaching-defs false negative). Defs now require an initializer for variable_declaration declarators; let/const genuinely initialize and keep their def. Finding P2-5 of review 4471987625 (#2160). * fix(cfg): unwrap parenthesized/non-null lvalue wrappers before def detection `(x) += 1` and `(x)++` gated the def on the node type being exactly 'identifier', so the parenthesized form fell to the uses-only branch — the def (and its kill) silently vanished. Wrappers that don't change the lvalue (parenthesized_expression, TS non_null_expression) now unwrap at all three lvalue sites. Finding P3 (parenthesized lvalues) of review 4471987625 (#2160). * fix(cfg): conditionally-evaluated defs are MAY-defs — gen without kill A def inside a short-circuit right operand, ternary arm, logical assignment, or switch case test was harvested as a must-def; the solver's total kill then erased the prior def on the not-taken path — a taint false negative on core idioms (`if (a && (x = clean())) {} sink(x)` lost source→sink; `cached ?? (cached = load())` likewise). StatementFacts gains an optional mayDefs field (conditional-context tracking in the harvester); the solver's per-block GEN carries {set, kills} so a may-def UNIONS into the binding's set instead of replacing it, in both the transfer and the statement sweep; the emit fact-guard validates mayDefs indices; switch case tests harvest via the conditional path. Finding P1-1 of review 4471987625 (#2160). * fix(cfg): model labeled statements generically — break keeps its real continuation A break to a label the visitor didn't model (labeled non-loop block, the OUTER label of a doubly-labeled construct) routed to EXIT, REMOVING the only path that kept the pre-jump def live — a reaching-defs false kill the in-code comment wrongly called sound. Loop/switch frames now carry their full label LIST (`outer: inner: for` resolves both); a labeled non-loop statement gets a break-target frame whose target is a synthesized join after the body; an unlabeled break never matches a block frame; labels compose with finalizer threading (a labeled break crossing a finally still threads it). Finding P1-2 of review 4471987625 (#2160). * fix(cfg): throw edges deliver ALL of a block's defs to the handler The throw contribution was IN ∪ OUT — entry and final states only. The intermediate defs of a multi-def coalesced block were invisible to the handler, though they are exactly what the catch observes when a later statement throws: `try { x = parse(a); x = normalize(x); } catch { sink(x) }` lost the parse→sink fact (normalize throwing delivers parse's value). Throw predecessors now contribute IN(from) ∪ allDefs(from) — a static per-block all-def-sites map — which subsumes OUT; monotone and deterministic. Finding P1-3 of review 4471987625 (#2160). |
||
|
Gergő Magyar
|
5bf8a17cd5
|
feat(ingestion): add control-flow-graph layer for TS/JS (#2081) (#2099)
* feat(cfg): language-agnostic CFG construction core (#2081) U1 of M1 (CFG layer). Plain JSON-serializable CFG data model (BasicBlockData/ CfgEdgeData/FunctionCfg — must survive the worker→main boundary + ParsedFile store), a CfgBuilder accumulator (leaders→blocks→edges, synthetic ENTRY/EXIT, idempotent edges), a ControlFlowContext (break/continue/switch + labeled-jump target stacks), and a TraversalResult ({entry, dangling exits}). AST-agnostic and unit-tested on the classic control-flow topologies (if/else, while back-edge, mid-block return, labeled break/continue) the S2 spike validated; reachability helper backs the R9 property test. * feat(ingestion): U2 — TS/JS CFG visitor over tree-sitter AST (#2081) Add the TS/JS CfgVisitor that walks a function's tree-sitter AST and drives the U1 CfgBuilder to produce a serializable FunctionCfg. One visitor covers both languages (shared grammar family). Handles the classic CFG hazards explicitly (R2, R10): - loops allocate a dedicated loop-exit block so `break` has a concrete target before the loop's successor is known; `continue`/back-edge close the loop (while, do-while, C-for with init-once + increment-as-continue-target, for-in, for-of) - switch fallthrough falls out naturally: a non-breaking case yields exits we wire to the next case as `fallthrough`; a breaking case wires to the switch exit via ControlFlowContext - try/catch/finally: normal completion AND exceptional flow both route through finally (post-domination); a conservative exceptional edge models that the protected region may raise to its handler (not just explicit `throw`) - labeled break/continue resolve against the labeled loop's frame - early return/throw wire to EXIT/handler and terminate their block 19 hazard tests (one per construct) + AC1 10-function fixture; all green. No change to the committed U1 core or ControlFlowContext. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ingestion): U3 — worker CFG build + cfgSideChannel + cache coherence (#2081) Run the CFG visitor in the parse worker (where the AST lives), serialize the per-function CFG onto a new ParsedFile.cfgSideChannel, and keep it coherent across the disk-backed store and the warm/durable parse cache (R3, R4). - gitnexus-shared parsed-file.ts: add `cfgSideChannel?: unknown` as a DISTINCT field from captureSideChannel (different producer/consumer/lifecycle; plain JSON data — blocks/edges deliberately lack the `nodeId` the store's interning reviver keys on, so no mis-interning). - cfg/types.ts + visitors/typescript.ts: add CfgVisitor.isFunction so the worker enumerates functions (and applies the line budget) by a cheap node-type test. - cfg/collect.ts (new): collectFunctionCfgs walks the tree, builds one CFG per function (nested included), applies maxFunctionLines (over-cap = skipped). - language-provider.ts: add `cfgVisitor?: CfgVisitor<SyntaxNode>` hook; typescript.ts attaches it to both the TS and JS providers (shared grammar). - parse-worker.ts: read pdg + pdgMaxFunctionLines from workerData (read once at init — the worker never sees PipelineOptions), gate the build, attach cfgSideChannel alongside captureSideChannel. - parse-cache.ts: bump SCHEMA_BUMP 4→5 (ParsedFile shape changed) and fold the pdg flag into computeChunkHash so a pdg-off cached chunk is NOT reused on a --pdg run (the #2038-class warm-cache trap). Default path keeps its keys. - worker-pool.ts + parse-impl.ts + pipeline.ts: thread pdg/pdgMaxFunctionLines PipelineOptions → WorkerPoolOptions → workerData, and into the chunk-hash key. 9 boundary tests: collect contract, JSON round-trip identity (no AST leakage), the pdg cache-key guard, the line-cap skip, and the no-visitor gate. Full CFG suite (U1+U2+U3) green; build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ingestion): U4 — emit BasicBlock + CFG within scope-resolution (#2081) Emit persisted BasicBlock nodes + CFG edges from each ParsedFile's worker-built cfgSideChannel, INSIDE scope-resolution's Phase-4 graph emission — the last point where the worker-built CFGs are loaded (emitParsedFiles carries the channel; the disk store is cleared right after the orchestrator returns). This is the architecture the doc-review corrected to: a standalone post-`mro` phase (the issue's literal subtask) provably reads empty data (KTD1). - cfg/emit.ts (new): pure emitFileCfgs(graph, cfgs, maxEdgesPerFunction, onWarn). BasicBlock id = `BasicBlock:<filePath>:<functionStartLine>:<blockIndex>` (KTD3 — funcStart disambiguates blocks across functions in one file; no `name` column). CFG edge = CodeRelation type 'CFG' with the edge KIND (seq/cond-true/…) in `reason` (kinds can't be their own edge type). Per- function edge cap stops at the cap and warns with the dropped count — no silent truncation (R6/KTD6). - run.ts: pdg-gated emit pass over emitParsedFiles after emitPostResolutionEdges (store still live); RunScopeResolutionInput gains pdg + pdgMaxEdgesPerFunction. - phase.ts: thread ctx.options.pdg / pdgMaxEdgesPerFunction into the call. - pipeline.ts: PipelineOptions.pdgMaxEdgesPerFunction. 6 tests: node/edge shape (KTD3 id, no name, type='CFG', kind in reason), cross-function id uniqueness, AC2 reachability-from-ENTRY property, the edge cap's no-silent-truncation contract, and empty-input no-op. Flag-off byte-identity + full runPipelineFromRepo round-trip land in U7. Build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): U5 — `--pdg` opt-in plumbing (CLI + .gitnexusrc → both sinks) (#2081) Expose the CFG/PDG substrate as an opt-in and thread it from CLI/.gitnexusrc to the single source of truth (PipelineOptions.pdg), which fans out to BOTH sinks already wired in U3/U4: the worker build gate (workerData.pdg) and the scope-resolution emit gate. Off by default (R7). - cli/index.ts: `--pdg` commander flag. - cli/analyze.ts: AnalyzeOptions.pdg + pass `pdg` into runFullAnalysis options. - cli/analyze-config.ts: KEY_SPECS `pdg` (boolean) so `.gitnexusrc { "pdg": true }` normalizes and a non-boolean value fails closed with GitNexusRcError. - core/run-analyze.ts: AnalyzeOptions.pdg → runPipelineFromRepo({ pdg }). (The internal PipelineOptions/WorkerPoolOptions/workerData fields + the parse-cache key fold landed in U3/U4; this unit adds the user-facing surface. The budget knobs stay at internal defaults for M1.) Tests: analyze-config pdg normalization + non-boolean rejection; opt-in.test.ts covers the CLI/file merge precedence and that pdg perturbs the chunk-dispatch key. The full worker-build + main-emit round-trip is the U7 integration test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ingestion): U7 — CFG acceptance fixtures, parity, end-to-end + docs (#2081) Acceptance criteria for the M1 CFG layer: - AC1: a 10-function TS fixture's CFG node/edge set matches a committed snapshot (cfg-snapshot.test.ts). - AC2: every BasicBlock is reachable from its function ENTRY (property test over the emitted graph; the fixture has no dead code). - AC3: hazard fixtures lock the classic-bug coverage — try/throw/finally post-domination + labeled break/continue resolution. - AC4: the existing pipeline-graph-golden test stays byte-identical with --pdg off (verified; no UPDATE_GOLDEN), proving the opt-in adds zero default-run drift. - End-to-end (pipeline-pdg.test.ts): runPipelineFromRepo({ pdg: true }) on a tiny repo emits BasicBlock nodes + CFG edges with both endpoints present — the true both-sinks proof (worker builds → store → scope-resolution emits); the default run emits zero. Docs: CHANGELOG M1 entry, ARCHITECTURE "Optional CFG/PDG emission" subsection (why emit is in-phase, not post-mro), README CFG language-support note. Full CFG suite (U1–U7): 56 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ingestion): drop unused helper in cfg-snapshot test (#2081) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): apply ce-code-review autofix feedback (#2081) Review (10 reviewers) confirmed OFF-path byte-identity (adversarial + golden) and found defects all within the --pdg path. Fixes: - P1 same-line BasicBlock id collision: add a start-column disambiguator to FunctionCfg + the id (`BasicBlock:<file>:<line>:<col>:<idx>`) so two functions sharing a start line no longer collide under first-writer-wins addNode. - P1 worker crash-cascade: per-file try/catch around collectFunctionCfgs so a CFG-build throw cannot escape to the language-group catch and silently drop every remaining file in the group. - P2 edge-cap drop now logs unconditionally (input.onWarn is validator-gated/ silent in prod) — upholds the no-silent-truncation guarantee. - P2 Array.isArray guard before the cfgSideChannel cast in run.ts. - P2 maxFunctionLines default: worker applies DEFAULT_PDG_MAX_FUNCTION_LINES=2000 when unset; caps forwarded through run-analyze AnalyzeOptions (closes the server-path drop). - P3 README duplicate paragraph removed; `0`-vs-default docstrings corrected; CLI --pdg flag made language-neutral; reachableBlocks JSDoc corrected. - Documented the break-through-finally + stacked-label CFG limitations. - Tests: same-line id-collision regression, standalone throw→EXIT, dead-code- after-return, async/generator/method coverage, strengthened labeled-continue. Refuted: the HTTP-500 getNodeQuery finding — M0 already shipped the BasicBlock branch + name-floor (R12/web-safety handled). CFG + analyze-config suites: 95 tests green; golden parity (AC4) byte-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(ingestion): benchmark CFG construction + O(n) block-text accumulation (#2081) Closes the M1 review's requires_verification perf gap ("no benchmark for collectFunctionCfgs; a wall-time + cfgSideChannel byte-size regression gate would catch the extendBlock concatenation before kernel scale"). - bench/cfg/measure.mjs (new): build-free tsx harness timing collectFunctionCfgs (parse once, reuse the tree) across three scaling scenarios — straight-line (extendBlock path), many-functions (collect walk), branchy (block/edge growth) — at 500→2000. Reports a wall-time scaling ratio AND a cfgSideChannel byte-size ratio, plus an order-independent sha256 over the emitted blocks/edges as the behavior gate. `--check` compares both ratios + the fingerprint against bench/cfg/baselines.json; mirrors the scope-capture / python-scope harnesses. - .github/workflows/ci-tests.yml: run the gate on every test job (build-free, alongside the existing scope-capture guards) so an O(n^2) re-regression fails CI. - cfg-builder.ts: structural fix for the one real hotspot the bench surfaced — accumulate basic-block text as fragments joined once in finish(), instead of concatenating onto a growing string per coalesced statement (O(n^2) → O(n)). Behavior-identical (the CFG fingerprint + the AC1 snapshot are unchanged). Measured (post-fix): time ratios straight-line ~1.3, many-functions ~1.0, branchy ~1.1 (all sub-quadratic; a true O(n^2) would be ~4.0). cfgSideChannel bytes scale linearly (~1.0-1.04). 60 CFG tests green; build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(ingestion): add memory + disk growth gates to the CFG benchmark (#2081) Extend bench/cfg/measure.mjs beyond wall-time to the two other scalability dimensions that matter at kernel scale: - DISK growth: utf8 byte size of the serialized cfgSideChannel — exactly what a --pdg run writes onto every ParsedFile shard (durable store + parse cache). - MEMORY growth: retained JS heap of the cfgSideChannel payload, measured by the release-delta method (heap held minus heap after dropping it) — robust to pre-existing garbage and dead-stable run-to-run. Needs `node --expose-gc`; without it the heap metric is null and its gate is skipped (local runs still work). ci-tests.yml now passes --expose-gc so the heap gate runs in CI. Both gated on linear scaling in baselines.json (disk_bytes_budget / heap_budget 1.2-1.3). Measured: disk ~1.0-1.04, retained heap ~0.87-1.0 — both linear (~1KB/function each; ~2MB heap / 1.6MB disk at 2000 functions, --pdg only). Bumped REPS 7->15 to stabilize the noisier time signal and widened the coarse time tripwire budgets (the disk/heap gates carry the tight regression detection). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ingestion): address tri-review + CFG-expert findings (#2081) Corroborated findings from the tri-review (Codex + CE personas + GitNexus swarm + a CFG/program-analysis domain-expert lane). The OFF-path stays byte-identical; all fixes are within the --pdg path or the benchmark. - [Codex+CFG-expert] Exceptional `throw` edges now wire EVERY block in a try's protected region to the handler, not just the body ENTRY. A branched try body (`try { if (x) { use(t); } } catch`) previously left interior blocks with no path to `catch` — a taint false-negative into the handler for the M2 PDG pass. - [Codex+CFG-expert] An unresolved labeled jump (a stacked outer label or a labeled non-loop block) now routes to the function EXIT instead of leaving a dangling sink — restores the single-exit invariant post-dominator/PDG computation needs. - [Codex] computeChunkHash now folds pdgMaxFunctionLines/pdgMaxEdgesPerFunction into the chunk key (not just the pdg boolean), so a warm cache built under one cap is never served to a run with a different cap (#2038 class, extended to the budgets). Adds PdgCacheKey; boolean form kept for back-compat. - [perf] visitTry resolves catch/finally in a single namedChild pass (the double `namedChildren.find` allocated two throwaway arrays). - [adversarial] The bench `straight-line` scenario now runs at 2000->8000: output is a constant 4 blocks so disk/heap can't see the concat path, and at the old N a genuine O(n²) was masked by V8 cons-strings. Verified at the new N: the array-join impl ~1.0, a rope-optimized `+=` ~1.0 (correctly not flagged), a real O(n²) (re-join-every-append) ~3.8 — budget tightened 2.0->1.5. - [adversarial+Codex] The bench `--check` now FAILS LOUDLY when run without `--expose-gc` instead of silently skipping the retained-heap gate. - Doc: re-labeled the finally-bypass as a SOUNDNESS (false-negative) limitation tracked for M2, not mere "precision." 3 new regression tests (branched-try interior→handler, stacked-label→EXIT, cap-fold key). 99 CFG tests pass; build clean; bench gate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(parse-cache): clarify that SCHEMA_BUMP still invalidates caches once (#2099 F6) The computeChunkHash comment claimed pdg-off warm caches "survive this change untouched" — true for the key FORMAT, but misleading as an upgrade-behavior promise: SCHEMA_BUMP 4→5 changes PARSE_CACHE_VERSION and both stores hard-invalidate on it. Separate the two facts so the next cache change isn't reasoned about from a false premise. Review finding F6 (P3) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): correct for-loop back-edge kinds when no increment clause (#2099 F5) A for with a body but no increment emitted an unconditional header→header 'loop-back' self-edge (a path that never executes the body) while the real back-edge body→header was labeled 'seq'. Any consumer identifying loops via reason='loop-back' picked the phantom edge and excluded the body from the natural loop. Gate the self-edge on the body being absent (the one case where the header genuinely re-tests itself) and carry 'loop-back' on the body's exits when they ARE the back-edge, matching visitWhile/visitForIn. Review finding F5 (P3) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): treat an empty catch clause as a real handler (#2099 F2) visitTry keyed handler semantics off the traversal result — null for an empty body, since visitSeq([]) returns null — instead of the syntactic clause. An empty `catch {}` was therefore treated as NO catch: the swallowed exception escaped to the outer handler/EXIT, the no-catch re-propagation misfired past finally, and code after a try whose body always throws became unreachable from ENTRY — a hard false-negative source for the M2 taint pass, on an extremely common pattern. Synthesize one empty block spanning the clause (entry == sole exit) when the catch body traverses to null, before the protected region is walked. Exception flow lands in it and rejoins the normal continuation; all downstream wiring (handler selection, finally routing, the !catchRes re-propagation gate) operates on the syntactically-correct shape. Review finding F2 (P2, reproduced) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): guard CFG emission per element, not just per outer array (#2099 F4) The cfgSideChannel guard checked only Array.isArray before casting to FunctionCfg[] — its own comment promised a wrong-shape value would 'skip emission, not throw a TypeError mid-graph-build', but a malformed ELEMENT sailed through. Worse, the obvious-looking failure shape never throws at all: emitFileCfgs string-templates any edge endpoint into the BasicBlock id and graph inserts are no-throw, so a non-integer endpoint silently became a dangling 'BasicBlock:…:undefined' edge that degrades the DB rel-pair COPY to row-by-row fallback inserts much later. Layered fix matching house precedents (parsedfile-store reviver, worker-side per-file catch): a per-element shape+content predicate (arrays + integer edge endpoints) that warns and skips malformed elements while valid siblings still emit, plus a per-file try/catch backstop for shapes that genuinely throw (e.g. a null inside blocks). Review finding F4 (P3) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse-cache): drop emit-time edge cap from the pdg chunk key (#2099 F3) pdgMaxEdgesPerFunction is applied exclusively in emitFileCfgs during scope-resolution on the main thread — the worker never receives it (workerData carries only pdg + pdgMaxFunctionLines), so the cached worker output is byte-identical across cap values. Folding it into the chunk key (added by a prior review round) only converted a free knob into a repo-sized cost: every cap change forced a full re-parse and a durable-store rewrite of unchanged data. Keep pdg + maxFunctionLines (genuinely worker-visible, shape the cached cfgSideChannel) and document the classification test in the PdgCacheKey doc comment so the next option gets sorted deliberately: worker-shard inputs go in this key; persisted-graph-only inputs belong in the RepoMeta pdg stamp (F1). Chunks written under the old ns string miss once and prune — no migration needed. Review finding F3 (P2) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): record pdg config in RepoMeta; force full writeback on mode flip (#2099 F1) Running --pdg against an already-indexed repo silently persisted ~zero CFG: incremental eligibility had no pdg term, RepoMeta recorded no mode, and extractChangedSubgraph keeps only changed-file nodes — on a no-change --pdg re-run every freshly built BasicBlock was dropped from the written subgraph ('Incremental: changed=0', run succeeds, zero rows). The converse flip left zombie mixed-coverage blocks only --force could clean. Worse, a clean-tree flip hit the alreadyUpToDate fast path and never ran the pipeline at all. - RepoMeta gains an additive-optional pdg stamp ({maxFunctionLines, maxEdgesPerFunction}, resolved values; absent ≡ pdg-off, which covers every legacy meta). No INCREMENTAL_SCHEMA_VERSION bump — that would force a one-time full rebuild for everyone. The end-of-run meta is a fresh literal, so omitting the field on a pdg-off run is what clears the stamp after an on→off flip. - pdgModeMismatch (pure, exported) compares the resolved triple; the flip check sits before the fast path and always logs its notice (not gated on options.force — --skills implies force with no message of its own), naming the .gitnexusrc pdg key that pins the mode. - The full-rebuild branch now writes the incrementalInProgress dirty flag (toWriteCount: 0 sentinel) before the wipe whenever a prior meta exists, mirroring the incremental branch. This closes the crash window where a rebuild dying between the bulk load and saveMeta left meta/DB inconsistent and the fast path certified zombie (or missing) CFG rows indefinitely — and incidentally closes the same pre-existing hole for user --force runs. Recovery log reworded accordingly. Tests: pdg-mode-flip.test.ts (real git + LadybugDB; primary assertion is a direct BasicBlock table count — meta.stats aggregates nondeterministic Community/Process rows) covering off→on, steady-state fast path, on→off zombie cleanup, cap-change rebuild, and dirty-flag + flip composition; pure-helper tests for default resolution and the 0=unlimited carve-out. Review finding F1 (P1) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
3d30b94c46
|
fix(parse): survive non-cloneable worker results so large-repo analyze doesn't crash (#2112) (#2135)
* fix(parse): survive non-cloneable worker results so large-repo analyze doesn't crash (#2112) A parse worker delivers its accumulated result to the main thread via postMessage, which structured-clones the payload synchronously on the worker thread and throws a DataCloneError on the first value it can't serialize. The reporter's case was a node `properties` value pointing at a native `toString`. The worker re-posted the throw as {type:'error'}, the pool counted it as a worker death, and under GITNEXUS_WORKER_POOL_SIZE=1 the same graph re-threw on every respawn until the slot's budget was exhausted and the whole parse phase aborted -- defeating even the conservative single-worker workaround. Add a clone-safety net at the worker result boundary. On a clone failure the worker isolates the offending file, strips the non-cloneable value from a plain extraction record (keeping the record -- strictly-missing data, never wrong) or drops a whole ParsedFile so scope-resolution re-derives it on the main thread with intact edge data, records the affected paths on the result, warns naming the field + file so the leak is diagnosable, and re-posts. Healthy runs are byte-identical: the net runs only after a real DataCloneError, so there is zero overhead on the fast path. Skipped paths surface via the parsing processor alongside the skipped-language telemetry. The strip drops the same values the store path's JSON.stringify already silently removes, so store/no-store runs converge. Scope: PR-1 -- failure mode C, the deterministic POOL_SIZE=1 killer. The timeout/native-abort graceful-degradation cascade (failure modes A & B) is coupled to downstream-exclusion + a hard worker watchdog and is tracked as follow-up work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): fail-closed clone-safety recovery + bound recursion depth (#2135 review) The clone-safety recovery path could re-arm the #2112 worker-death cascade it was built to prevent: in postResultCloneSafe the sanitizer call and the re-post sat outside the try/catch, and containsNonCloneable/stripNonCloneable recursed with a cycle guard but no depth bound. A throw inside the sanitizer (a RangeError from a deeply-nested record, reproduced at depth >=3000) escaped to the message handler's {type:'error'}, which under GITNEXUS_WORKER_POOL_SIZE=1 is the respawn-budget-exhaustion abort. Wrap the sanitizer + re-post in their own try/catch so any throw fails closed to a primitive-only {type:'error'} deliberately, and thread a MAX_CLONE_DEPTH bound through both scan/strip functions so an over-deep subtree is treated as non-cloneable (dropped/undefined) instead of overflowing the stack. The isStructuredCloneable catch-all is left broad on purpose — it bounds structuredClone's own internal recursion in the non-plain-object probe. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): harden clone-safety against throwing getters and detached buffers (#2135 review) Two sanitizer-defeat vectors let the re-post throw a DataCloneError again: - A throwing getter on a record: containsNonCloneable/stripNonCloneable read obj[key], so a getter that throws escaped the scan/strip pass. Read defensively — a throwing property read is treated as non-cloneable (scan returns true, strip drops the property). - A detached ArrayBuffer/TypedArray: both passed buffers/views through unconditionally, but structuredClone rejects a detached one, so the re-post threw. Route buffers/views through the authoritative isStructuredCloneable probe instead. No byteLength heuristic — a legitimately empty new Uint8Array(0) also has byteLength 0 yet clones fine, so a length check would false-positive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): memoize stripped copies so DAG-aliased records aren't over-dropped (#2135 review) stripNonCloneable carried a shared `seen` WeakSet and returned the ORIGINAL (un-stripped) value on revisit. When a non-cloneable was reachable via two paths (a DAG), the second path spliced the original function-bearing object back into the output, so the rebuilt element failed the last-resort isStructuredCloneable guard and the whole record was dropped as "unsalvageable" — contradicting the "record kept, value stripped" contract. Replace the WeakSet with a Map<object, stripped-copy>: allocate the empty copy, memoize it before recursing into children (so cycles return the in-progress copy), and return the memoized copy on revisit. DAG-aliased subtrees now collapse to one shared stripped copy and are kept-and-stripped, not dropped. The array branch moves from .map() to allocate-then-push so its identity can be pre-inserted. Object Map/Set keys aren't identity-preserved across stripping — acceptable because parse-result Maps are primitive-keyed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(parse): single-pass clone-safety scan preserving array identity (#2135 review) makeWorkerResultCloneSafe scanned each dirty array twice — a field-level whole-array containsNonCloneable probe, then a per-element pass — and always reassigned the field. Fold into one per-element pass that builds the output array lazily (copying the clean prefix only once the first dirty element appears) and reassigns the field only when something changed. A fully-clean array is now scanned once and keeps its referential identity; the clean prefix of a dirty array is copied by reference. Behavior is otherwise identical (failure-path-only code). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(parse): drop unused generic + pin clone-safe field names to keyof (#2135 review) makeWorkerResultCloneSafe carried a generic `<T extends Record<string,unknown>>` that was never load-bearing (it mutates in place and returns {skipped}), and the call site passed untyped string-literal option sets — so renaming `parsedFiles` or `skippedPaths` would silently disable the drop-whole / skip protection. Drop the generic (plain `Record<string,unknown>` param) and type the option sets at the call site as `Set<keyof ParseWorkerResult>`, so a field rename is now a compile error. The `as unknown as Record<string,unknown>` widening stays — it's the standard cast for a no-index-signature interface (TS rejects a single-step `as`); the function genuinely operates structurally on the result's arrays. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): keep the per-file reason in the clone-safety skip log (#2135 review) The processor's skipped-file warning logged only the paths, dropping the per-file reason the worker already attached — losing the distinction between a recoverable "stripped N value(s)" and a whole-record "dropped" entry. Format each entry as `path (reason)` so the aggregate line carries the diagnostic detail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): deterministic findFilePath attribution for ParsedNode (#2135 review) findFilePath swept all child objects one level deep in Object.keys order, so a ParsedNode could be attributed to a sibling child's path-like key instead of its real path at properties.filePath. Check the known `properties` child first, then fall back to the generic sweep, so node attribution is deterministic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): zero skippedPaths in the slim cache result (#2135 review) slimParseWorkerResultsForCache spread the worker result without clearing the clone-safety skippedPaths telemetry, so a sanitized result persisted its skip list into the on-disk parse-cache shard. Replay already ignores the field; zero it (like calls/assignments/parsedFiles) to keep shards lean and the intent explicit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(parse): exercise real postResultCloneSafe wiring + tighten RED control (#2135 review) The integration GREEN worker re-implemented postResultCloneSafe inline, so the production wiring (the {type:'warning'} post + the skippedPaths append) had no coverage, and the RED control asserted a bare .rejects.toThrow() that any failure would satisfy. Extract postResultCloneSafe into a side-effect-free module (post-result.ts) — importing it from the parse-worker entry module would construct the parser, post ready, and attach the real handler — and have the GREEN test worker import and call the real one. Tighten the RED matcher to the actual abort contract (/circuit breaker|consecutive failures|respawn budget|could not be cloned/), which also documents that the raw poison result aborts via the pool's consecutive-failure circuit breaker under POOL_SIZE=1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): recover the clone-safety net from any post failure, not only DataCloneError (#2135 review) The V8 structured-clone research surfaced the net's one real correctness hole: structuredClone invokes getters, and a getter that THROWS surfaces its own error (a RangeError, etc.) — NOT a DataCloneError (confirmed against a real MessageChannel). postResultCloneSafe gated recovery on isDataCloneError, so such a throw re-threw past the sanitizer and re-armed, under POOL_SIZE=1, the worker-death cascade the net exists to prevent. Attempt the sanitize + re-post recovery for ANY first-post failure (the sanitizer already reads properties defensively, so a throwing getter is dropped), falling closed to a primitive-only {type:'error'} only if the re-post still fails. Adds an integration case: a node with a throwing getter is recovered and delivered, not re-thrown. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(parse): name the exact offending key path in the clone-skip diagnostic (#2135 review) The clone-safety net's skip reason named only the array field + file ("stripped 1 value from nodes"), not the offending property key — which is precisely why the original #2112 leak stayed unpinned. Thread a dotted key path through stripNonCloneable (recording each stripped value's path: properties.toString, meta.data[3], …) and surface the first few in the reason ("from nodes: properties.toString"). Now a single log line — or the contract/strict checks — names the leaking property, so a residual runtime escape can be fixed at source. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(parse): clone contract — a representative ParseWorkerResult is structured-cloneable (#2135 review) Shape-regression guard: builds a representative ParseWorkerResult (typed as the real interface) and asserts isStructuredCloneable. Typing it as ParseWorkerResult makes adding a new boundary field a compile error here until the test is updated, and the runtime assert catches a field whose type regresses to a non-cloneable shape — independent of language input. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(parse): strict-mode clone gate (GITNEXUS_STRICT_CLONE) — fail loudly instead of silent sanitize (#2135 review) The runtime net's silent recovery in production is exactly what let the original #2112 leak stay unpinned. Add an opt-in strict mode (GITNEXUS_STRICT_CLONE=1, inherited by workers): on a clone failure, postResultCloneSafe THROWS with the exact offending key path instead of sanitizing + delivering, so a leak introduced by a future provider/extractor change fails loudly at its origin (CI/dev) rather than being quietly stripped. Off in production, where the net keeps the run alive. Adds a self-contained integration case (sets the flag, asserts the poison run rejects with the key path) and skips the synthetic-poison suite under a global strict run (its value there is running the REAL-extractor integration tests under strict). Wiring a strict CI lane (GITNEXUS_STRICT_CLONE=1 on a vitest integration step) is left to the maintainer — it needs a green full-suite verification and touches the protected workflow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(server): don't ship pipelineResult across the analyze-worker IPC boundary (#2112) The forked analyze worker reports completion to the parent over child_process IPC, which uses Node's DEFAULT 'json' serialization (api.ts forks with no `serialization:` option). `AnalyzeResult.pipelineResult` is populated on every successful analysis and carries `pipelineResult.graph` — the live KnowledgeGraph closure object. Sending the raw result is wrong three ways: (1) the graph's nodes/relationships getters force-materialize the entire graph into two arrays and JSON-stringify them on every analyze, discarded immediately (a multi-hundred-MB no-op on a large repo — the #2112 scenario); (2) the graph's methods are own function properties that JSON drops silently, so a surviving graph is a husk whose forEachNode() throws far from the cause; (3) a BigInt/circular value anywhere in the payload makes process.send throw TypeError synchronously — caught and re-sent as {type:'error'}, mis-reporting a SUCCESSFUL analysis (DB already written) as a FAILURE. This is the #2112 failure family on the server path, and unlike the parse-worker result boundary it has no clone-safety net. The parent (api.ts) reads only result.repoName; pipelineResult's real consumers (CLI skill generation, cli/analyze.ts) call runFullAnalysis in-process and never cross this fork. So project the result down to an explicit JSON-safe allowlist of scalar fields. Typed as Omit<AnalyzeResult,'pipelineResult'> so a future non-serializable field added to AnalyzeResult fails to compile until handled here deliberately. Found by the #2112 cross-process serialization-boundary audit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ingestion): Cloneable<T> + assertCloneable() compile-time clone-boundary guard (#2143) The runtime clone-safety net is the production backstop; this is its compile-time complement. The worker result is plain data except a few `unknown`-typed sinks (a node's `properties` bag, the provider `extractTemplateConstraints` / `collectCaptureSideChannel` hook returns) — `unknown` lets a non-serializable value (a function, a leaked tree-sitter SyntaxNode, …) cross the structured-clone boundary with no compile-time guard. That is the structural hole #2112 leaked through. `Cloneable<T>` is a homomorphic recursive mapped type that maps a function or symbol member to `never`, so a struct carrying one is no longer assignable to its own `Cloneable<T>`. `assertCloneable(value)` is a runtime identity (zero cost) whose parameter is `T extends Cloneable<T> ? T : Cloneable<T>`, so a clone-unsafe argument fails to compile, naming the offending key. Because it is a homomorphic mapped type it preserves `interface` shapes and `readonly` modifiers and needs NO index signature on the payload types — this sidesteps the "closed interface is not assignable to a recursive index-signature type" wall that blocked the original value-typed-`Cloneable` attempt (the reason #2143 was deferred from PR #2135). The conditional parameter type avoids the `T extends Cloneable<T>` circular-constraint error. Tests: runtime identity contract, plus type-level @ts-expect-error assertions (enforced by tsconfig.test.json) that a function/symbol member is rejected and clean interface payloads are accepted. Applied to the real provider hooks in the next commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ingestion): guard provider clone-boundary hooks with assertCloneable (#2143) Apply the compile-time guard to the provider hooks that feed the `unknown`-typed worker-result sinks, so a future non-serializable value in their payloads is a compile error at the source site rather than a runtime DataCloneError at the worker post: - C++ extractTemplateConstraints (CppConstraintPayload) - C++ collectCaptureSideChannel (CppCaptureSideChannel) - C collectCaptureSideChannel (CCaptureSideChannel) - Kotlin collectCaptureSideChannel (KotlinCaptureSideChannel) The C++ template-constraint adapter previously returned `unknown`; it now returns the concrete `CppConstraintPayload | undefined` and routes its payload through `assertCloneable`. The side-channel hooks are wrapped at their provider wiring sites. `assertCloneable` is a runtime identity, so behavior is unchanged (C static-linkage + C++ constraint suites stay green); the guarantee is the type-check — src tsc now proves every nested member of those real payload trees is structured-clone safe. Test: type-level assertions (enforced by tsconfig.test.json) that each concrete payload type is `Cloneable<T>`, INDEPENDENT of the provider wiring — so the regression is caught even if the assertCloneable wrapper is later removed. Proven non-vacuous (a function-bearing type fails the same assertion). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): scan an array's non-index own properties in the clone sanitizer (#2135 review) structuredClone serializes an array's NON-index own-enumerable properties (e.g. `arr.meta = fn`) and throws DataCloneError on a non-cloneable one. The clone sanitizer's array branches iterated numeric indices only, so such an array was waved through (containsNonCloneable returned false, makeWorkerResultCloneSafe left the field unrewritten with skipped:[]) — the re-post then threw, fell through to the fail-closed {type:'error'}, and re-armed the POOL_SIZE=1 cascade the net exists to prevent. Add isArrayIndexKey() and, in BOTH containsNonCloneable and stripNonCloneable array branches (kept in lockstep), scan/strip the non-index own-enumerable keys after the index loop. A cloneable non-index prop is carried onto the stripped copy; a non-cloneable one is stripped and recorded. Not reachable from current parse output (no extractor attaches non-index array props) — a defense-in-depth hole closed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): contain a throw inside the clone sanitizer instead of escaping to fail-closed (#2135 review) findFilePath was documented "never throws" but read element properties unguarded in its generic sweep — a throwing getter at a non-path key (or a Proxy with a throwing ownKeys trap) threw out of makeWorkerResultCloneSafe, past postResultCloneSafe's recovery, to the fail-closed {type:'error'} that under POOL_SIZE=1 re-arms the cascade the net prevents. Likewise a Proxy with a throwing getPrototypeOf trap throws inside containsNonCloneable's instanceof checks. - findFilePath/pathFromChild now read via safeGet (try/catch) and guard Object.keys, honoring the "never throws" contract. - Each element's sanitize in makeWorkerResultCloneSafe is wrapped: a throw during scan/strip drops that one element (recorded as "sanitizer error") rather than sinking the whole result — so one pathological element can't fail-close the run. - Corrected the makeWorkerResultCloneSafe JSDoc ("ONLY after a DataCloneError" → after ANY post failure, matching the caller) and documented the deliberate failure-path double-traversal (the non-allocating pre-scan is what preserves clean-element referential identity). Tests: a throwing getter on a path-less element is stripped & delivered (not escaped); a Proxy structural-trap element is dropped, clean siblings survive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse): add a final cloneable postcondition gate to the clone sanitizer (#2135 review) makeWorkerResultCloneSafe rewrote only ARRAY result fields, so a future non-array sink (a nested object / Map result field) carrying a non-cloneable value — or an array field whose own non-index property the element loop didn't reach — would survive the sanitizer and throw on the re-post. Add a final `if (!isStructuredCloneable(result))` gate that strips any remaining offending field in place, making "the returned result is structured-cloneable" a hard postcondition independent of future ParseWorkerResult shape. Failure-path-only and a no-op once the array loop already made the result clean (the per-field probe short-circuits every clean field, so it adds no work or skip entries then). Tests: a function on a non-array result field is stripped & the result becomes cloneable; the gate adds no skip entry when the array loop already cleaned up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(parse): reject an `any`-typed member in the Cloneable<T> compile-time guard (#2135 review) `Cloneable<any>` previously resolved to `any` (not `never`), so a payload with an `any`-typed member — the most likely escape hatch, since `unknown` is already blocked — passed `assertCloneable` with no compile error. Add an `IsAny<T>` branch (the canonical `0 extends 1 & T` probe) as the FIRST arm so `any` resolves to `never`, matching how `unknown` is already rejected. It must precede the primitive arm: `any extends CloneablePrimitive` would otherwise resolve to `any` and re-admit it. The IsAny-first arm perturbs inference for a bare `undefined` literal argument (T infers as `unknown` → never); real consumers pass `X | undefined` unions (the provider hooks), which are unaffected (src tsc clean), so the runtime identity test now uses a `string | undefined` value — the realistic shape. Tests: an `any` member fails `assertCloneable` (@ts-expect-error, enforced by tsconfig.test.json) and `Cloneable<any>` resolves to `never` at the type level. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(server): type the analyze-worker IPC projection as a Pick allowlist, not Omit (#2135 review) `AnalyzeResultIpc = Omit<AnalyzeResult,'pipelineResult'>` kept every other field in the type — including optional ones like `isPrimaryBranch?` — so the type advertised a field the runtime allowlist never sends, and the doc-comment's "a future field fails to compile until handled here" only held for REQUIRED fields. Switch to `Pick<AnalyzeResult, …the six scalar fields…>`: the allowlist IS the type, so the projection return literal is exhaustive by construction (omitting a key is a compile error) and a new `AnalyzeResult` field is simply absent from the wire until deliberately added here. `isPrimaryBranch` is intentionally excluded (nothing consumes it server-side over this fork; the parent reads only `repoName`). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(parse): remove the now-dead isDataCloneError export (#2135 review) postResultCloneSafe recovers on ANY fast-path post failure and never inspects the error type (a throwing getter surfaces a RangeError, not a DataCloneError — gating on the type was the original net-gap bug). isDataCloneError has no production caller; it was only exercised by its own unit test. Remove the function and that test block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(parse): use the exported SkippedPath type in parsing-processor (#2135 review) The clone-safety telemetry accumulator inlined `Array<{path,reason}>` — a structural duplicate of the exported `SkippedPath`. Import and use the canonical type so a future rename of its fields is a compile error here instead of a silent structural drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(parse): document the cloneable-return contract on the worker-boundary hooks (#2135 review) extractTemplateConstraints and collectCaptureSideChannel return `unknown` and feed values across the worker structured-clone boundary, but the hook contracts didn't state the cloneability requirement — a future language implementing them without care could leak a non-serializable value. Document that the return MUST be structured-clone-safe and should be wrapped with assertCloneable, so the guarantee is a compile error at the source (#2143). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(parse): assert the clone-skip telemetry surfaces in the GREEN integration case (#2135 review) The GREEN clone-safety integration test asserted only graph content (all files present), not that the skippedPaths / {type:'warning'} wiring its docstring claims to cover actually fired. Capture the production logger via _captureLogger and assert the sanitize telemetry names the offending file (poison.ts) AND the exact stripped key path (properties.toString) — proving the worker's skippedPaths append + the parsing-processor warning surfaced end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(server): cover the IPC projection against a real KnowledgeGraph (#2135 review) The IPC projection tests used a hand-built hostile object. Add a case that puts a real createKnowledgeGraph (whose nodes/relationships getters would materialize the whole graph under JSON.stringify) in pipelineResult and asserts the projection drops it entirely — the serialized payload stays under 300 bytes (a materialized 50-node graph would be thousands), with the scalar fields intact. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(parse): cover the unsalvageable-drop branch and the skippedPaths merge union (#2135 review) Two untested clone-safety branches from the tri-review: - "dropped unsalvageable": a dirty element whose stripped copy is STILL not structured-cloneable must be dropped, not delivered (else the re-post throws). Add a deterministic test (a non-plain member with a stateful getter that the strip-time probe sees clean but that turns into a function on the post-strip verification) asserting the element is dropped and the run survives. - mergeResult skippedPaths union across sub-batches. mergeResult (and its appendAll helper) was module-private in the parse-worker ENTRY module, which a main-thread test can't import (it runs MessagePort setup). Extract it to a side-effect-free result-merge.ts (mirroring post-result.ts) and unit-test the union (including the `??=` target-init path), the skippedLanguages sum, and array append. parse-worker imports it back; verified the built worker still parses + merges via the real-worker integration path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(parse): root-prettier format the clone-safety review-fix files (#2135 review) Clears the failing `quality / format` CI gate (root prettier, not the gitnexus-local config). Reformats the pre-existing #2143 wrapping lines in c-cpp.ts + kotlin.ts plus the clone-safety review-fix files touched in this PR-update (clone-safety.ts and the new/updated tests). Formatting-only — no behavior change; tsc, the type-level assertions (tsconfig.test.json), and the unit + integration suites stay green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(parse): avoid js/trivial-conditional in the type-level clone assertions (#2135 review) CodeQL flagged the `expect(a && b && c).toBe(true)` lines in the type-level test assertions as js/trivial-conditional: after type erasure the operands are constant `true`, so the `&&` chain always evaluates the same. Replace the `&&` chain with array equality (`expect([...]).toEqual([true, ...])`) — no conditional, and the real assertions remain the `const x: …IsNever = true` / `: IsCloneable<…> = true` annotations (enforced by tsconfig.test.json, which fail to compile if a guard regresses). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
95f87fc12a
|
perf(ingestion): Linux-kernel-scale analysis — worker-pool parse + finalize O(n²) + scope-resolution memory wall (#1983) (#2038)
* fix(ingestion): reduce parse-phase memory for huge repos (#1983)
Stop retaining full parse-cache chunks in RAM alongside the merged graph,
slim on-disk shards, defer worker ParsedFile emission for scope-resolver
languages, and add GITNEXUS_DEBUG_HEAP probes for OOM diagnosis.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ingestion): address #2038 tri-review findings (parse-phase memory)
Resolves the confirmed review findings on PR #2038:
- P1: thread exportedTypeMap through the sequential parse path
(processParsingSequential) so a no-worker run over a partially-warm
cache no longer silently drops the sequential-miss files' exported
types. Cache hits made exportedTypeMap.size > 0, suppressing the
end-of-loop buildExportedTypeMapFromGraph rebuild, but the sequential
path never populated the map. Regression test added (fails on the
pre-fix tree, passes after) plus a fully-sequential differential oracle.
- P2: saveParseCache builds its on-disk index from hashes actually
written/copied (writtenKeys), never a usedKeys hash whose shard write
or copy was skipped — no more phantom index entries.
- P2: add a unit test asserting SCOPE_RESOLUTION_LANGUAGES stays in sync
with SCOPE_RESOLVERS (asymmetric drift would lose a language's ParsedFile).
- Backfill cache coverage: loadParseCacheChunk missing/corrupt -> undefined,
pruneCache onDiskKeys branch, slim preserves nodes, saveParseCache
copy-evicted-shard round-trip.
- Cleanups: single-source heap-probe gating via isDebugHeapEnabled();
hoist the per-chunk mkdir in persistParseCacheChunk behind a
process-scoped Set; gate COBOL's unused worker-side ParsedFile
extraction (graph nodes still come from cobolPhase) while keeping
fileCount/progress unconditional.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ingestion): remove dead worker-side ParsedFile extraction
After #2038 gated worker `ParsedFile` emission behind `!isScopeResolutionLanguage(language)`, and with all 16 SupportedLanguages registered in SCOPE_RESOLVERS, that gate was structurally always true — the worker already produced no ParsedFiles and scope-resolution re-extracts each file from source on the main thread (run.ts). Remove the now-dead machinery:
- Drop both worker `extractParsedFile` call-sites (tree-sitter processFileGroup + the standalone-provider branch) and the `result.parsedFiles.push`. The standalone branch keeps fileCount/onFileProcessed per file. `result.parsedFiles` stays declared but empty (field removal deferred).
- Remove the now-orphaned `scopeSourceKind` var + `ScopeCaptureSourceKind`/`extractParsedFile`/`isScopeResolutionLanguage` imports.
- Delete the consumerless `migrated-languages.ts` (isScopeResolutionLanguage + SCOPE_RESOLUTION_LANGUAGES) and its drift-guard test — parse-worker was their only importer. Also improves AGENTS.md "shared ingestion code must not name languages" compliance.
`extractParsedFile` and the scope-extractor-bridge stay (scope-resolution/run.ts + Vue resolver use them). Behavior-preserving: worker-sequential-parity passes before and after; tsc/eslint clean; no baseline/golden drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ingestion): worker-pool-only parsing; remove sequential parser (#1983)
Completes the #1983 huge-repo parse-OOM effort by making the worker pool
GitNexus's sole parse path.
Parallel serialization (the perf core): workers serialize their ParsedFiles to
a disk store in parallel and stream them back to scope-resolution, so the main
thread no longer re-parses every file (the tree-sitter native-memory leak that
caused the OOM). Adds chunk merge-pipelining + work-proportional chunk sizing so
the pool stays saturated.
Remove the sequential parser: `--workers 0`, `GITNEXUS_WORKER_POOL_SIZE=0`, and
`skipWorkers` now hard-error (no silent degrade — #1741); the small-repo
threshold no longer selects an in-process path; pool creation stays lazy /
cache-miss-gated so warm all-hit runs never spawn workers.
Worker-path parity fixes — removing sequential surfaced two pre-existing gaps
that tiny-fixture tests had masked by running below the worker threshold, both
fixed by carrying per-file metadata as DATA across the worker boundary (never
re-parsing on the main thread, preserving the OOM fix):
- C++: templateConstraints wired into worker node identity (SFINAE overload
disambiguation) + ADL / inline-namespace capture side-channel serialized
onto the ParsedFile.
- Kotlin: companion-scope side-channel serialized the same way (companion /
static dispatch).
Validation: tsc + build clean; full suite green (10,190 pass — the only
deterministic failures were the now-fixed C++/Kotlin worker-path gaps; the 2
remaining full-run failures are pre-existing load flakiness, green in
isolation); cpp-pipeline benchmark stays linear on a 1-worker pool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): wire C static-linkage side-channel + ADL O(1) collect + tri-review cleanups (#1983)
Follow-up to the worker-pool-only refactor, from a tri-review of the parse path.
- C static-linkage side-channel (P1): cProvider had no collect/applyCaptureSideChannel,
so on the now-sole worker path C `static` file-local marks were lost across the worker
boundary -> false cross-file CALLS edges + over-broad #include wildcard visibility on
every C analysis (the Linux kernel is C). Mirror the C++/Kotlin wiring: serialize
`staticNames` per file onto ParsedFile.captureSideChannel and restore it on the main
thread (no re-parse). + a worker-path regression test (the existing c-static-isolation
fixture passed vacuously — its collision resolves via #include before the global
free-call fallback ever consults static-linkage).
- captureSideChannel `kind` discriminant: add `kind:'cpp'`/`kind:'c'` tags + guards
(Kotlin already had one) now that C/C++/Kotlin share the single generic field.
- Perf: collectCppAdlSideChannel scanned the whole argInfoBySite/noAdlSites maps per file
(O(F^2) per sub-batch, ~100M parseSiteKey calls at kernel scale). Add per-filePath
lockstep indexes -> O(1) collect; serialized snapshot byte-identical.
- Cleanups: inline the one-line processParsingWithWorkers wrapper into processParsing;
drop the always-empty WorkerExtractedData.calls/assignments/constructorBindings fields;
remove the voided astCache param from processParsing; refresh stale "sequential
fallback" JSDoc.
Validation: tsc + build clean; cpp 297/297, c 8/8 (incl. the new worker-path
static-linkage guard), typescript + parsedfile-store green; cpp ADL benchmark stays linear.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): index C/C++ #include resolution in finalize (O(n²)→O(n))
Kernel-scale C/C++ analysis ground in finalizeScopeModel because three
per-#include operations each did a full O(F) scan with no index — the
finalize O(n²) that surfaced once the #1983 parse-phase OOM was fixed:
- expand{C,Cpp}WildcardNames: parsedFiles.find() per wildcard edge → O(R·F)
- resolveImportTarget: new Set(allFilePaths) rebuilt per #include
- resolveCImportTarget: suffix-match scanned all workspace paths
Each is replaced with a WeakMap-per-pass index keyed on the stable
parsedFiles/allFilePaths references that scope-resolution run.ts passes
once per pass:
- Map<ScopeId,ParsedFile> for wildcard expansion (c/static-linkage.ts +
cpp/file-local-linkage.ts)
- memoized augmented header set (c/scope-resolver.ts + cpp/scope-resolver.ts)
- basename-bucketed suffix index in resolveCImportTarget (c/import-target.ts),
shared by C and C++ since resolveCppImportTarget delegates to it
Collapses the C/C++ finalize from O(R·F) to O(R+F). Pure-perf, byte-identical
edge output: 962 targeted tests green (490 C + 472 C/C++ scope-resolution);
the basename index preserves the exact endsWith('/'+target) match and the
fewest-path-components-then-lexicographic tie-break.
The kernel's ~25-30k .h headers are classified C++, so both providers must
be fixed. Proven on the Linux kernel: the C finalize completed
(sr-post-finalize lang=c → sr-end lang=c), which the pre-fix run never
reached in 16+ min of grinding.
Build-independent follow-ups (separate from this finalize fix), documented
for later: emitFreeCallFallback same-name buckets (emit phase),
buildGraphNodeLookup + precount global setup, the ParsedFile store-load,
the dart/go/ruby expand-wildcards .find siblings, and the ~26GB
scope-resolution memory floor (full kernel completion needs >~40GB RAM).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(bench): regenerate C scope-capture baseline for the #1983 c-static-linkage-worker fixture
bench/scope-capture/measure.mjs fingerprints emitCScopeCaptures over the
lang-resolution/c-* fixture corpus. The #1983 PR added the
c-static-linkage-worker fixture (caller.c/lib.c/lib.h/local.c — the
worker-path static-linkage side-channel test) but did not regenerate the C
baseline, so `--check` has been red on this branch (main, lacking the
fixture, still matches 0de009b).
Pure fixture-corpus drift — no c/captures.ts or query change branch-vs-main,
existing fixtures' captures byte-identical (c-captures.test.ts 45/45),
scaling stays linear (~0.97). Regenerated: 0de009b -> 39f3a83. Bench now
PASS (14 languages). Unrelated to the finalize O(n²) fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): lower kernel-scale resident memory floor + setup cost
Reduce the scope-resolution resident-memory floor and setup throughput on
huge repos (Linux kernel), the wall that remains after #1983 (parse OOM) and
the finalize O(n^2) fix (
|
||
|
Gergő Magyar
|
083aedbc41
|
refactor(ingestion): delete legacy call-resolution DAG + heritage processor (RING4-1, #942) (#2023)
* refactor(ingestion): delete legacy call-resolution DAG + heritage processor (#942) RING4-1: all 16 production languages (incl. Vue #940) are registry-primary, so the legacy resolution legs only ran under the now-removed CI parity gate. Calls and inheritance now resolve exclusively through scope-resolution (Registry.lookup, preEmitInheritanceEdges, emitHeritageEdges, buildMro → MethodDispatchIndex). Removed: - Call-resolution DAG: call-processor.ts legacy body (processCalls, processCallsFromExtracted, resolveCallTarget + all resolver/dispatch/chain helpers), model/resolve.ts MRO-via-HeritageMap, model/heritage-map.ts, type-env DAG types; inferImplicitReceiver/selectDispatch LanguageProvider hooks + Ruby impls; DispatchDecision/ImplicitReceiverOverride/ReceiverEnriched. - Legacy heritage path: heritage-processor.ts, heritage-types.ts, heritage-extractors/, @heritage.* tree-sitter queries, heritageExtractor/ heritageDefaultEdge/interfaceNamePattern wiring, worker + parse-impl heritage passes (parse-worker/parsing-processor lockstep), cross-file-impl DAG pass. - Scope-parity infrastructure entirely (no legacy↔registry parity left to run): scripts/run-parity.ts, scripts/ci-list-migrated-languages.ts, ci-scope-parity.yml, test:parity, and the scope-parity ci.yml gate. Resolver integration tests still run via the normal tests job. Kept (shared infra, NOT call-DAG-only): type-env.ts buildTypeEnv (field extraction / structure phase / embeddings), model/resolve.ts c3Linearize + gatherAncestors (mro-processor mroPhase), route/fetch/exported-type-map helpers in call-processor.ts, preEmitInheritanceEdges (legacy-edge dedup simplified). Acceptance: grep for resolveCallTarget/inferImplicitReceiver/selectDispatch/ buildHeritageMap/HeritageMap/processHeritage/heritageExtractor/@heritage. is zero across src + test. tsc clean (both packages); resolver integration suite green (bit-compatible EXTENDS/IMPLEMENTS/CALLS); scope-capture fingerprints unchanged (python re-baselined: removed redundant ignored captures). ARCHITECTURE.md updated to scope-resolution-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): apply autofix feedback (#942) ce-code-review autofix pass on the RING4-1 deletion: - parse-cache.ts: bump SCHEMA_BUMP 2→3 — ParseWorkerResult lost its `heritage` field, so stale on-disk caches must invalidate (prevents a rollback replaying a heritage-less cache into legacy code) [api-contract P2]. - parse-impl.ts: drop 3 now-unused type imports (ExtractedCall, ExtractedAssignment, FileConstructorBindings) left by the deferred-block removal — would fail the eslint CI gate [correctness+maintainability P1]. - AGENTS.md / CLAUDE.md / scope-resolver.ts contract doc: fix stale pointers to the deleted "§ Call-Resolution DAG" section + removed hooks; preserve the language-neutrality rule [project-standards P1]. - registry-primary-flag.ts / cross-file.ts / parse-impl.ts: refresh stale comments referencing deleted symbols (legacy DAG, runCrossFileBindingPropagation). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ingestion): remove the vestigial isRegistryPrimary flag (#942) With the legacy call-resolution DAG deleted, the per-language `REGISTRY_PRIMARY_<LANG>` / `isRegistryPrimary` / `MIGRATED_LANGUAGES` flag had only one meaningful state — every production language resolves via scope-resolution — and an explicit `=0` override could only *disable* resolution with no fallback (a footgun the review flagged). Removing it. - Delete `registry-primary-flag.ts` and the now-dead `shadow-harness.ts` (legacy↔registry shadow-parity tool) + its test. - Collapse the three flag gates to their behavior-preserving outcome (`SCOPE_RESOLVERS == MIGRATED_LANGUAGES`, so this is a no-op): - scope-resolution phase now runs for every registered `SCOPE_RESOLVERS` entry (was `∩ MIGRATED_LANGUAGES`). - import-processor `addImportGraphEdge` + parse-impl `shouldAccumulate`: the legacy emit/accumulate paths were already inert for migrated languages (scope-resolution owns IMPORTS via the imports-to-edges bridge); drop the flag term. - Collapse flag-branching tests to the scope-resolution path and delete the csharp legacy-`=0`-leg describe blocks; remove the ruby/rust-scope env-forcing hooks (no-ops now). - Refresh docs/comments (ARCHITECTURE.md "one registration", scope-resolver cookbook, phase deps) — adding a language is now a single `SCOPE_RESOLVERS` registration. Verified: tsc clean (both packages); resolver integration tests green (747 assertions across cobol/csharp/ruby/rust/typescript/go, IMPORTS edges intact); grep for the flag symbols is zero across src + test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(format): prettier formatting on #942 changes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): drop legacy heritage-capture tests + re-baseline scope-capture fingerprints (#942) Two CI failures from the #942 cleanup, surfaced by the tri-review + CI: - tree-sitter-languages.test.ts: two tests asserted `@heritage.*` captures (Rust trait-impl, Dart extends/implements/with) that this PR removed. The acceptance grep used `@heritage\.` (with `@`); these reference the runtime capture name `heritage.trait` (no `@`), so they slipped the earlier sweep. Inheritance is now covered by the resolver integration suite. (fixed macos-latest) - Re-baselined the scope-capture bench fingerprints for csharp/rust/ruby/java/ javascript/kotlin (baselines.json) + python (python-scope/baseline-fingerprint.txt). The earlier test-cleanup reworded comments inside the lang-resolution fixture files (Shapes.cs, child.rs, derived.rb, IA.java/Plain.java, Service.js, F.kt, app.py) to scrub deleted-symbol references for the acceptance grep; those are the bench corpus, so capture node positions shifted. Capture LOGIC is unchanged — verified `--check` passes for all 14 langs + python. (fixed benchmarks) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs/chore: scrub remaining REGISTRY_PRIMARY + deleted-symbol references (#942) Tri-review P3 follow-ups (verified): - TESTING.md: rewrite the "Scope-resolution parity" section — the legacy dual-leg (REGISTRY_PRIMARY_<LANG>=0/1) and `npm run test:parity` no longer exist; resolver tests run once on the sole scope-resolution path in the normal tests job. - scripts/bench-scope-resolution.ts: drop the inert `REGISTRY_PRIMARY_PYTHON=1` env set + usage hint (the flag is gone). - ruby/scope-resolver.ts, php/captures.ts: re-point doc-comments off the deleted heritage-map.ts / heritage-processor.ts to the current behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): prettier format + regenerate scope-capture goldens (#942) Two more CI failures, same root cause as the bench re-baseline (the test-cleanup reworded comments in lang-resolution bench/golden-corpus fixtures): - quality/format: prettier on tree-sitter-languages.test.ts (blank line left by the deleted heritage-capture tests) + TESTING.md (the rewritten section). - tests/ubuntu/coverage: `csharp-captures-golden` (and python/ruby/rust) drifted because the edited fixtures feed the per-language capture-golden snapshots too (not just the bench). Regenerated via UPDATE_GOLDEN=1. Verified safe: only the edited-fixture entries changed; csharp `captureGroups` unchanged (38) — digest shifted from comment-position only; capture LOGIC untouched. 1168 scope- resolution tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(resolvers): drop createResolverParityIt wrapper, use vitest it directly The parity-aware `it` wrapper became a no-op when #942 removed the legacy call-resolution DAG (it just returned vitest's `it`). Remove it entirely so the resolver tests call vitest's `it` directly instead of shadowing it with a local `const it` (or `pit`/`rustParityIt`): - helpers.ts: delete createResolverParityIt + its now-unused vitestIt import and VitestIt type. - 16 files: drop `const it = createResolverParityIt('x')` and import `it` from vitest instead. - ruby.test.ts (pit) + rust.test.ts (rustParityIt): rename calls to `it`. - Scrub every comment that described the removed wrapper / dual-mode parity skip / legacy_skip gate (vue-scope, js/ts/dart/php/python headers, rust x2, cpp, swift x4, rust-coverage). Genuine test rationale is kept; only the vestigial two-leg framing is dropped. Accurate "legacy DAG (removed in #942)" historical notes are retained. No fixtures touched (no bench/golden re-baseline). tsc clean; rust+ruby resolver suites green (323 tests, incl. #1992 worker-path parity after a local dist build). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
99168be773
|
feat(ingestion): trace indirect call patterns — FastAPI Depends() and frontend HTTP consumers (#1852) | ||
|
BlackOvOoo
|
263ca353a6
|
fix: shard parse cache persistence on large repos (#1580)
* fix: shard parse cache persistence on large repos * fix(parse-cache): validate shard keys, docs, and sharded-cache tests - Reject non-sha256-hex keys from index.json before path.join (path traversal). - saveParseCache: skip invalid keys defensively; try/catch per-shard JSON.stringify. - Clarify save comment (tmp dir + rename vs atomic). - Tests: hex keys throughout, traversal keys, multi-shard, version-mismatch+legacy, second save, legacy removal. - AGENTS.md / GUARDRAILS.md: document .gitnexus/parse-cache/ vs legacy parse-cache.json. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Abhigyan Patwari
|
4fa40e9881
|
feat(analyze): incremental indexing (parse cache + DB writeback + scope-res short-circuit) (#1479)
* docs: incremental indexing design spec Captures the design agreed in brainstorming on 2026-05-10: - Transitive importer closure with public-surface-change optimization - Git-only change detection (non-git repos: full rebuild as today) - New default behavior; --force opts out - New hydratePhase + loadGraphFromLbug primitive - Iterative closure expansion with parseCache reuse - incrementalInProgress dirty flag for crash recovery Prior art: PR #592 (zenprocess), PR #533 (davidbeesley), PR #1146 (azeemshaik025) — referenced and credited. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(communities): seed Leiden RNG for deterministic community detection The vendored Leiden algorithm defaults to Math.random for tie-breaking and randomized walks, which produces non-deterministic community assignments and modularity values across runs on the same graph. Pass a seeded mulberry32 RNG (LEIDEN_SEED=0xC0DE) so: - The same graph always produces the same partition - Modularity values are reproducible - Equivalence tests for incremental indexing can compare community assignments byte-for-byte This is foundational for the upcoming incremental-indexing feature (see docs/superpowers/specs/2026-05-10-incremental-indexing-design.md) where the correctness contract is incremental output ≡ full rebuild output. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(incremental): change-detection, surface signatures, closure expansion Three new modules supporting the incremental-indexing pipeline: * core/incremental/git-diff.ts — getChangedFilesSinceCommit() unions 'git diff lastCommit HEAD' (committed) with 'git status --porcelain' (dirty tree). Renames flattened to delete(orig) + add(new). Throws LastCommitMissingError when lastCommit is gone (caller falls back to full rebuild). * core/incremental/surface.ts — extractSurfaceSignature() produces a stable hash of a file's publicly-visible symbols (functions, classes, methods, interfaces, types, heritage). Body-only edits → same hash. Signature/heritage changes → different hash. Drives the closure scoping optimization. * core/incremental/closure.ts — computeImporterClosure() iterative fixpoint: parse each closure file, extract surface, query DB importers, expand. Uses a parseCache so each file is parsed once. Generic over TParseResult so closure logic is decoupled from the pipeline's parse representation. 32 unit tests across the three modules. Tests cover edge cases: clean tree, dirty-only, mixed, renames, deletes, multi-hop cascade, cycle termination, surface invariance, etc. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lbug): loadGraphFromLbug, queryImporters, deleteAllCommunitiesAndProcesses Three new primitives in lbug-adapter.ts to support incremental indexing: * loadGraphFromLbug(graph, unchangedFilePaths) — streams all nodes for files in the set across every hydratable node table (excludes Community/Process — graph-wide, regenerated downstream). Then loads edges where both endpoints belong to loaded nodes, excluding MEMBER_OF / STEP_IN_PROCESS edges (also graph-wide). FilePaths chunked at 200 per query to keep statement size bounded on huge repos. Endpoint-level join filters by source-side filePath in the query, target-side checked JS-side via the loadedNodeIds set. * queryImporters(targetFilePath) — returns DISTINCT a.filePath where a -[IMPORTS]-> b and b.filePath = target. Powers closure expansion: when a changed file's surface signature changes, all its importers must be re-parsed. * deleteAllCommunitiesAndProcesses() — drops Community/Process nodes (and their edges via DETACH DELETE) at the start of each incremental run so the communities/processes phases regenerate them from the fully-merged graph. Required for the 'Leiden runs on full graph' correctness invariant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(pipeline): hydrate phase + parse-filter for incremental indexing Wires the incremental-indexing infrastructure into the phase-based pipeline. Three coordinated changes: * New hydratePhase (deps: structure) — loads node/edge state for files OUTSIDE ctx.options.filesToParse from the existing LadybugDB index. Runs before parse so the parse phase can produce a partial graph while downstream phases (mro, communities, processes) still see the full graph. No-op in full-rebuild mode (filesToParse unset). * PipelineOptions.filesToParse: optional ReadonlySet<string>. When set, parse phase filters scanned files to this set; hydrate fills the complement. Set by runFullAnalysis when it detects an eligible incremental run; never set by callers directly. * gitnexus-shared PipelinePhase enum: 'hydrate' added so progress callbacks can report the new phase distinctly from 'structure'. Phase order: scan → structure → hydrate → markdown,cobol → parse → routes,tools,orm → crossFile → scopeResolution → mro → communities → processes. Communities (Leiden) still runs on the full graph, satisfying the correctness invariant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(analyze): incremental orchestrator branch + meta schema Wires incremental indexing into runFullAnalysis. Highlights: * RepoMeta schema extended: schemaVersion, surfaceSignatures, and incrementalInProgress fields. INCREMENTAL_SCHEMA_VERSION = 1. * core/incremental/file-hash.ts — v1 surface signature: SHA-256 of file content. v2 will switch to a true surface-only signature (defined in surface.ts) so body-only edits don't expand the closure. The plumbing is signature-agnostic so the swap is local. * core/incremental/orchestrator.ts — eligibility check, closure computation (uses file-hash as the surface signal), dirty-flag management, subgraph extraction, signature merge. * run-analyze.ts adds: - hasDirtyTree() check on the existing 'lastCommit==HEAD' early-exit so an uncommitted edit triggers re-index (was a coarse equality check before). - incremental branch: try incremental first; fall through to full rebuild on any setup failure or eligibility miss. - runIncrementalBranch() — opens existing DB, deletes closure-file rows + Community/Process, runs pipeline with filesToParse, writes only the changed-subgraph back, refreshes FTS, updates meta with new surfaceSignatures and clears the dirty flag. - Full-rebuild path now populates surfaceSignatures + schemaVersion in meta.json so the next run is eligible for incremental. Crash recovery: incrementalInProgress is set BEFORE any DB mutation and cleared on success by overwriting meta.json. A crash anywhere in between leaves the flag set, and the next analyze run forces a full rebuild (cheapest path back to a known-good index). v1 limitation documented: body-only edits trigger 1-hop closure expansion (content-hash signal). True surface-only optimization is deferred to v2 — see design doc for the integration path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(incremental): drop invalid --no-renames=false from git diff The flag --no-renames=false isn't valid git syntax (it's parsed as a file path). Git's default rename detection is on; removing the flag keeps that behavior. Caught while running an end-to-end smoke test against a small fixture repo: incremental setup failed with 'Command failed: git diff --name-status -z --no-renames=false ...'. After the fix, the incremental path runs cleanly: closure is computed, hydrate phase loads unchanged-file state from DB, parse phase only re-parses files in closure, and the writeback updates only changed nodes/edges. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Revert v1 incremental indexing (5 commits) Reverts the v1 design that parsed only closure files into a fresh graph and tried to hydrate the rest from DB. Real-repo equivalence test failed: cross-file resolution operates on partial parse data (closure files only), so CALLS edges that resolve through unchanged files silently fall off. Diff against full rebuild on the same edited state: -50 nodes, -425 edges, -5 communities, -48 processes. Architecture pivot: switch to PR #533-style content-addressed parse cache. Pipeline parses every file (cache-served when possible), giving cross-file resolution full data, with DB writeback then restricted to changed-file rows. Reverts: |