mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
74409a37f6
* perf(cpp): index qualified namespace members once per pipeline run (#2788)
`resolveCppQualifiedNamespaceMember` walked every parsed file — rebuilding a
per-file `scopesById` map each time — once per qualified `ns::member()` call
site, so the scope-resolution emit phase cost O(callsites x scopes). On a
1,473-file C++ repo that was 25.3 min of a 33-min analyze, with 75% of total
self-time in this one function. Its inner `findMemberInNamespaceTransitive`
compounded it: each recursion step filtered `scopesById.values()` by parent,
O(scopes^2) per file on its own.
This is the same bug #1990 fixed in the sibling ADL path (`pickCppAdlCandidates`
-> `AdlCandidateIndex`), so it gets the same fix: a `QualifiedNsMemberIndex`
(receiver simple name -> member simple name -> callable defs) built lazily once
per `parsedFiles` identity and reset by `clearCppInlineNamespaces`, which runs
from `cppScopeResolver.loadResolutionConfig` at the start of every pass. Per
call site the work drops to two Map lookups.
Ordering is preserved exactly — file-major, `parsed.scopes` declaration order,
a namespace's own `ownedDefs` before its inline-namespace children, depth-first
— because the caller takes `allHits[0]` for the single-hit case and
`narrowOverloadCandidates` is first-wins. Non-inline nested namespaces are
still not descended into, and same-name hits across inline children still
report `'ambiguous'` (#1564).
Measured with `PROF_SCOPE_RESOLUTION=1 analyze --force --index-only` on a
synthetic corpus (`namespace ns_i { inline namespace v1 { ... } }` plus 20
`ns_j::fn()` call sites per file):
| files | emit before | emit after |
|-------|-------------|------------|
| 100 | 153ms | 16ms |
| 200 | 704ms | 24ms |
| 400 | 3,293ms | 42ms |
| 800 | 16,898ms | 78ms |
Before, doubling the file count quadrupled emit; now it doubles. At 800 files
total scope resolution goes 17.2s -> 394ms.
Output is unchanged, verified rather than assumed: a full graph dump (sorted
nodes + relationships) from a baseline build at the parent commit and from this
one are byte-identical on all 134 `cpp-*` fixtures merged into a single repo
(1573 nodes / 1997 relationships) and on the 400-file synthetic corpus.
`test/integration/resolvers/cpp.test.ts` passes 334/334.
#1990 shipped its ADL fix without a scaling gate, which is how the bug class
came straight back here, so this adds one: `bench/cpp-qualified-ns` measures
`(t_large/t_small)/(1600/400)` — 0.93-1.21 indexed versus 3.45 for the old
per-call-site scan — alongside a fingerprint over every
`receiver::member -> outcome` the corpus resolves, and CI runs it with
`--check`. `test/unit/cpp-qualified-ns-index.test.ts` covers the cache
invalidation the index introduces.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(cpp): address tri-review findings on the qualified-namespace index (#2788)
Multi-engine review of this PR (Claude swarm + ce-code-review, Codex
gpt-5.6-sol swarm + ce + adversarial) returned two P1s and five smaller
findings. All are fixed here.
P1 — the index defeated the pipeline's post-language memory release.
`scope-resolution/pipeline/phase.ts` evicts each language's files and then
calls `forceGc()`, on a stated premise that "This language's ParsedFiles are
now unreachable", sizing C/C++ at ~17-20GB on the Linux kernel. The
module-level `let qualifiedNsIndexSource` falsified that: it pinned the whole
`parsedFiles` array, and the index held defs reaching into those files'
scopes, until the *next* C++ pass cleared it — which in a single analyze never
comes. C++ is 7th of 16 in SCOPE_RESOLVERS, so the set survived nine later
language passes plus emit. Replaced with a
`WeakMap<readonly ParsedFile[], QualifiedNsMemberIndex>`, the pattern already
used by `moduleScopeIndexByPass` in `cpp/file-local-linkage.ts`.
`clearCppInlineNamespaces` still swaps in a fresh WeakMap, because the index
has a second input (`inlineNamespaceScopeIds`) the key cannot observe.
Measured with `--expose-gc`: 61.2MB retained after the caller drops the array
before, 0.1MB after.
The ADL twin (`adl.ts`) has the same pattern, so the hazard predates this PR —
but `pickCppAdlCandidates` returns early before `ensureAdlIndex` on
`noAdlSites`/empty `argInfoBySite`, so it rarely arms, whereas a qualified
`ns::member()` index arms on almost every C++ workspace. Moving the ADL twin
to a WeakMap is left as a follow-up.
P1 — the new bench could not see the regression class it exists to gate.
`callSites()` drew every receiver from `ns_${...}`, so the receiver lookup
never missed; production is the opposite, since Case 1.5 in
`receiver-bound-calls.ts` is reached by every plain-identifier receiver call
and misses on most. A rescan reintroduced only on the receiver-bucket-absent
path scored 1.279 and PASSED the old bench. The corpus now mirrors production
(~1 in 5 receivers name a declared namespace) and adds a namespace reopened
across files, a same-name inline nest, a member declared at both namespace and
inline-child level, and call sites carrying a real `Callsite` so
`narrowOverloadCandidates`/`cppConversionRank`/
`isOverloadAmbiguousAfterNormalization` are inside the fingerprinted surface at
all. That same rescan now measures 4.538 and FAILS; defeating the dedup now
fails the fingerprint arm where it previously passed byte-identical. The
fingerprint moved once, deliberately, for the corpus expansion — recorded in
`_rebaseline_2788_review`, explicitly not precedent.
Also fixed:
- Unbounded recursion aborted analyze. `collectNamespaceMembers` recursed per
inline child with no bound and threw an uncontained `RangeError` at inline
depth 8000 (`phase.ts`'s try has a `finally`, no `catch`), and a receiver
*miss* paid full recursion where the deleted walker skipped on a name
mismatch. An explicit work-stack alone would only have converted that into
an OOM at depth 6000, because the eager table was quadratic in memory too:
for a depth-D chain it legitimately holds D(D+1)/2 entries, since `v2::foo()`
is a valid receiver at every level. Replaced with a lazily-queried node graph
(per-scope own-member buckets plus direct child links, resolved on demand and
memoized per receiver+member). Build is now linear; depth 100000 costs 133ms
where 8000 previously threw.
- "#1990 shipped without a scaling gate" was false. #1990 did ship
`test/integration/cpp-adl-benchmark.test.ts` (f1b843838). Corrected in the
bench header and the CI step comment. The accurate point is narrower and
stronger: that bench asserts `callsResolved === 0`, so it never drives the
qualified-receiver path, and it is `skipIf(!GITNEXUS_BENCH)` while the only
step setting that variable lists neither C++ bench — so it has never run in
CI. Wiring it in is a follow-up.
- "Ordering is load-bearing" was not a live property: `allHits[0]` is only
reached at length 1, and both tail branches return `'ambiguous'`. Order is
still preserved for byte-identity with the pre-#2788 walker; the comment now
says that instead, and the test named for ordering is renamed to the
parent/inline-child visibility it actually asserts.
- "Same contract as `ensureAdlIndex`" overstated parity — the sibling ships a
`validateAdlSeqCoverage` guard because it reads
`seqByNodeId.get(...) ?? 0`, which can silently collapse candidates. This
index has no analogous defaulting read, so no guard is added; the comment now
says why.
- Two coverage gaps closed, both mutation-verified: cross-file merge of one
namespace reopened in two files (no existing test covered it — confirmed by
making each file clobber the previous and watching only the new test fail),
and the same-name inline nest whose dedup, when defeated, flips a resolved
def to `'ambiguous'`.
- `resolveCppQualifiedNamespaceMember`'s JSDoc now names both production call
sites, including the callsite-less `resolveAdlCandidates` path.
- Memoized candidate buckets are frozen, so a future in-place sort in
`overload-narrowing.ts` throws instead of silently corrupting later
resolutions now that the array is shared across call sites.
- `_scaling_note`'s "measured 0.93-1.21" band did not reproduce; it is now the
honestly observed 1.28-1.45, with the small arm widened to ~14ms (halves the
spread) and a triage line saying a scaling failure is a timing signal to
re-run, unlike the deterministic fingerprint arm.
Verification: 840,000 differential probes against the pre-#2788 walker
extracted from base, 0 mismatches, plus 48,000 candidate-order comparisons,
0 mismatches. cpp resolver integration suite 334/334. Unit suite 10/10.
tsc, eslint, prettier clean. Bench --check PASS.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(bench): escape the NUL separator so measure.mjs stays a text file
The fingerprint key separator was written as a literal NUL byte instead of the
`\u0000` escape. Git classifies any file containing a NUL as binary, so the
whole bench showed as `Bin` with no diff on GitHub and could not be reviewed —
the same defect this branch already fixed once before the tri-review.
Escaping it is byte-for-byte equivalent at runtime (both produce U+0000), so
the committed fingerprint is unchanged and `--check` still passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(cpp): quality cleanups from a four-angle review of the #2788 series
Reuse, simplification, efficiency and altitude passes over the diff. No
resolution behaviour changes: the bench fingerprint is unchanged and the C++
resolver integration suite still passes in full.
Efficiency
- The `Object.freeze` added in the review-fix commit to close an aliasing
residual costs 4.6x on the narrowing path — V8 moves frozen arrays to
PACKED_FROZEN_ELEMENTS, off the fast path for the `.filter`/`.map`/`.some`
runs `narrowOverloadCandidates` does at every multi-candidate call site.
The hazard it guards is already a compile error (both the memo and the
parameter are `readonly SymbolDefinition[]`), so it is now a dev-only
tripwire. Gated on `isSemanticModelValidatorEnabled()` — the repo's opt-IN
form used by `phase.ts` and `validate-bindings-immutability.ts` — not
`adl.ts`'s opt-out `NODE_ENV !== 'production'`, which would keep paying the
cost in CLI runs where `NODE_ENV` is unset. Large bench arm 100.3 -> 70.6ms.
- The index build scanned every scope in every file twice; pass 1 now collects
`[scope, node]` pairs for pass 2 to iterate. 800k scopes 14.40 -> 8.21ms.
- `simpleNameOf` uses `lastIndexOf('.')` + `slice` instead of
`split('.').pop()` (83 -> 24 ns/call), semantics verified byte-equivalent
over 12 edge cases including `undefined`, `''`, `'a.'`, `'.b'` and `'a..b'`.
- The per-call `hookCtx` literal is hoisted to a module const.
- The bench's fingerprint pass resolved 960k call sites to produce 5,800
distinct outcomes; it now dedups on the key it already builds. Bench wall
time 2.4 -> 1.9s, fingerprint byte-identical.
Reuse
- `bucketOwnMembers` used an inlined `Function | Method | Constructor` compare;
it now calls the canonical `isOverloadableCallable`. `graph-bridge/ids.ts`
already carries a note that inlining this list recreated twin-list drift once.
Altitude
- `adl.ts` had the same retention defect this series just fixed next door: a
module-level `let adlIndex` + `let adlIndexSource` strongly pinning the whole
`parsedFiles` array until the next C++ pass, which in a single analyze never
comes. Converted to the same `WeakMap` shape. Measured with `--expose-gc`:
89.11MB retained after the caller drops the array before, 0.17MB after. Six
file-local helpers now take the index as a parameter; no exported signature
changed.
- The index's freshness depended on `clearCppInlineNamespaces()` being called
from another file, guarded only by a warning paragraph. An epoch bumped in
both `populateCppInlineNamespaceScopes` and the clear is now stored with the
memo, so a missed clear degrades to a rebuild instead of a stale answer —
confirmed by driving inline state mid-pass without the clear. Roughly line
neutral, since it replaces most of the paragraph.
- `test/integration/cpp-adl-benchmark.test.ts` is wired into the
`GITNEXUS_BENCH` step. It is `skipIf`-gated and was absent from that step's
explicit file list, so #1990's ADL emit-scaling guard had never executed in
CI. It passes; ~50s added to a 25-minute job. `cpp-pipeline-benchmark.test.ts`
is deliberately NOT wired: it costs 115s for guards covering generic
per-language pipeline scaling that nothing here touches.
Simplification
- Deleted a comment referencing a `inlineChildrenByParent` map that only ever
existed inside this branch's own first commit, so "the legacy map" pointed a
reader at code that never shipped.
- Dropped three unreachable `undefined` guards (`strict: false`, no
`noUncheckedIndexedAccess`), keeping the load-bearing `visited` check.
- Compressed the `rootsByReceiver` doc from 17 lines to 7 — it was the longest
comment in the file and guarded the least consequential property — the
`validateAdlSeqCoverage` paragraph from 7 lines to 3, and turned three
restatements of the uncaught-throw and dedup arguments into pointers.
- Test fixtures: dropped the dead `'Module'` union arm, added a one-line `ns()`
builder for the nine hand-written scope literals, and moved the file to
`test/unit/scope-resolution/cpp/` where every other C++ scope-resolution unit
test lives. 403 -> 337 lines, same 10 tests, and the cross-file mutation check
still fails exactly one test.
Not done, and why: merging this index into `AdlCandidateIndex` (they key on
different names with different inline-transparency depth — a refactor with
correctness risk, not a cleanup); `ScopeTree.getChildren` (trades in-memory
`parsed.scopes` for store hits on a hot path); a shared `cpp/` util for the five
pre-existing `simpleName` copies; sharing fixtures across the bench/test
boundary (no precedent in this repo); and an exact-count arm for the bench,
which is a gate redesign worth its own change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
7 lines
4.1 KiB
JSON
7 lines
4.1 KiB
JSON
{
|
|
"_comment": "Baselines for bench/cpp-qualified-ns/measure.mjs --check (#2788). `fingerprint` is a sha256 over every `receiver::member(arity|argumentTypes) -> outcome` the synthetic corpus resolves at the LARGE scale (hit nodeId, `<ambiguous>` per #1564, or `<none>`); it is a CORRECTNESS gate, so drift means C++ qualified `ns::member()` lookup started resolving a different symbol set and must be explained, never re-baselined to make CI green. THAT RULE IS UNCHANGED and applies to every future edit of inline-namespaces.ts. `scaling_budget` is a timing gate and carries deliberate headroom for shared CI runners.",
|
|
"_rebaseline_2788_review": "This fingerprint was moved ONCE, deliberately, during review of #2788 — because the bench CORPUS was expanded, not because a check failed. Do not read it as precedent. What changed: (1) receivers now mirror production — ~1 in 5 name a declared namespace, ~4 in 5 are plain identifiers naming none (`obj0`, `Widget3`, `buf12`). The previous corpus drew every receiver from `ns_${…}`, so the receiver lookup NEVER missed, while Case 1.5 in scope-resolution/passes/receiver-bound-calls.ts is reached by every plain-identifier receiver call and misses on the overwhelming majority. (2) A namespace reopened across two files (C++ namespaces are open — the cross-file merge property). (3) A same-name inline nest `namespace ns { inline namespace ns { … } }`, which is the only shape that observes `gatherQualifiedNsMember`'s `visited` dedup. (4) A member declared at BOTH the namespace level and in an inline child, selected apart by argument type, pinning both collection sources by nodeId. (5) Call sites carrying a real `Callsite`, without which narrowOverloadCandidates / cppConversionRank / isOverloadAmbiguousAfterNormalization were outside the fingerprinted surface entirely. Measured effect, same patched resolver, old bench vs new: removing the `visited` dedup — old PASS with a byte-identical fingerprint, new FAIL (fingerprint 1e6c51b9… != aba39c34…); resetting `visited` per root instead of across roots — old PASS, new FAIL on the same fingerprint. A PURE reorder of a candidate list still passes both, and correctly so: the resolver's return contract is order-blind by construction (see QualifiedNsMemberIndex's doc comment), so there is no behaviour there to gate.",
|
|
"fingerprint": "aba39c342ce536006bebded8b32260dc7807487be91f5c7ee9548f9e9283f9c9",
|
|
"scaling_budget": 1.8,
|
|
"_scaling_note": "(t_large/t_small)/(1600/400). ~1.0 is linear. OBSERVED BAND: 1.28-1.45 over ten unloaded runs on a 24-core dev box. The band this file previously claimed — 0.93-1.21 — did not reproduce and was an artifact: the small arm then measured ~1.7 ms, small enough that timer granularity and JIT warm-up, not scaling, set the number (the same ten-run sweep of that bench spanned 1.11-1.40). CALLS_PER_FILE is now sized so the small arm lands at ~14 ms; that halves the unloaded spread (0.30 -> 0.16) and costs ~2.0 s of wall time for the whole bench. The residual above 1.0 is real and not a defect: at LARGE the index and corpus are 4x the working set, so per-call-site locality is worse (~85 ns/site vs ~63 ns) while the algorithm stays linear. TRIAGE: a scaling failure is a TIMING signal — RE-RUN IT on an idle machine before investigating. Runner contention dominates everything above: pinned to 2 CPUs against 2 spinners the identical binary produced 1.16-2.18, i.e. a spurious FAIL, and the sibling bench/callable-value-flow drifts out of its own documented band the same way. The fingerprint arm is the opposite — it is deterministic; a re-run never changes it and must never be used to wish it away. Floor check: a per-call-site workspace rescan reintroduced ONLY on the receiver-bucket-absent path (the most plausible way #2788 returns) measures 4.538 at these same 400/1600 file scales — 812x slower on the small arm, 2850x on the large — while leaving the fingerprint byte-identical. The old always-hits corpus scored that same patch 1.279 and printed PASS. Resolution is timed alone; the fingerprint's outcome strings are built in a separate untimed pass because their allocation cost grows with the corpus and would otherwise show up as scaling."
|
|
}
|