GitNexus/gitnexus/test/integration/structural-pair-coverage.test.ts
Gergő Magyar 010a7d806a
fix(schema): declare the full scope-resolution relation cross product (#2792) (#2793)
* fix(schema): declare the full scope-resolution relation cross product (#2792)

`RELATION_SCHEMA` was hand-listed, and every prior fix added only the
FROM/TO pair named in a crash report — `Const→Method` in #2769, the
Swift/Rust member pairs before it. So `analyze` kept aborting at
`assertDeclaredPair` on the next codebase whose edges happened to land on
a different pair; #2792 reports `Class→Variable` on Java.

Audit the surface instead of the symptom. `buildGraphNodeLookup` skips
any node whose label is not in `isLinkableLabel`, so the lookup holds
only linkable-labelled nodes — and both endpoints of every graph-bridge
edge resolve through that lookup. The emittable surface is therefore
exactly:

  FROM  LINKABLE_LABELS + File   (the module-level caller fallback)
  TO    LINKABLE_LABELS + CALL_TARGET_TYPES

`isCallerAnchorLabel` is a strict subset of linkable and contributes
nothing on top. `CALL_TARGET_TYPES` contributes `Delegate`, which
`tryEmitEdgeWithExplicitTargetId` can emit without going through the
lookup at all.

Generate that 14x14 block into the DDL rather than listing it: 223 -> 322
declared pairs, and no future pair from these sets can be missing by
construction. The containment/inheritance/DI/route/cluster/PDG pairs stay
hand-declared — no single predicate describes them.

Both label sets live in the ingestion layer, which `core/lbug` must not
import, so schema.ts carries twin lists. test/unit/schema-pair-coverage.ts
derives the requirement from the originals and fails CI when either set
grows without the pairs landing here — the piecemeal loop this fix ends.

Measured before widening: at 322 pairs the cost is inside noise
(1.09s vs 1.12s per 300 anchored queries on a 32-table DB), but the full
32x32 cross product is ~1.8x on untyped-endpoint anchored queries. The
audited subset is the right scope, not "declare everything".

INCREMENTAL_SCHEMA_VERSION 34 -> 35: LadybugDB fixes endpoint pairs when
the rel table is created, so a pre-v35 database physically cannot store
these edges.

Closes #2792

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(schema): declare the non-bridge structural pairs COBOL and Vue emit

The generated scope-resolution block closed the half of RELATION_SCHEMA a
label predicate can describe. The hand-declared half was still stale: with
#2791's Function->Variable fix applied, `analyze` continued to abort on this
repo's own test/fixtures/lang-resolution with

  Relationship label pair Module→Property is not declared

A full sweep (assertDeclaredPair patched to log-and-skip, run over the whole
fixture corpus) found 13 undeclared pairs over 106 edges. This branch already
covered 3 of them via the cross product; the remaining 10 come from emitters
outside the graph bridge:

  - cobol-processor.ts mints Module / Namespace / Record / Property /
    CodeElement and wires them with CONTAINS, CALLS and ACCESSES (9 pairs)
  - vue-sfc-extractor.ts emits BINDS_EVENT_HANDLER from a handler Function to
    the child component's File, the only edge whose target is a File (1 pair)

CodeElement, Namespace, Record and File are in neither scope-bridge label set,
so neither the generated block nor schema-pair-coverage.test.ts can reach them.

Adds test/integration/structural-pair-coverage.test.ts, which derives the
requirement from a corpus instead of a predicate: it runs the real pipeline
over the non-bridge fixtures and requires every FROM/TO pair they produce to
be declared. Mutation-checked — dropping `FROM Function TO File` fails it with
exactly Function|File.

Verified: cobol-app, vue-basic and php-transitive-traits now index instead of
aborting; the full lang-resolution corpus completes at 10,876 nodes / 18,517
edges; scrypster/muninndb at 0b7a4272 (the #2789 repro) completes at 20,069
nodes / 71,580 edges, matching #2791 exactly, so this supersedes that PR.

* refactor(test): simplify the structural pair coverage guard

Cleanup pass over the previous commit. No behaviour change to the schema.

- reuse `FIXTURES` and `runPipelineFromRepo` from resolvers/helpers.ts instead
  of re-deriving the fixture root and importing pipeline.js directly
- gate on `distWorkerExists()` like every other integration test that passes
  `workerUrlForTest`, so a missing dist skips rather than fails
- run the three fixtures with `it.concurrent.each`; they share nothing and the
  cost is almost all worker spawn plus grammar load, which overlaps well
  (tests phase 21-24s -> 5.6s measured)
- replace the sentinel-in-a-Set filter with a plain `.filter()` chain, matching
  the sibling unit test, and move the declared/table lookups off the per-edge
  path onto the deduped set
- move the pure string pin out of the integration tier into
  schema-pair-coverage.test.ts, where the identical construct already lives, so
  it needs no build and survives fixture deletion
- trim the schema and test prose that restated the code, and correct the
  BINDS_EVENT_HANDLER attribution: it is emitted by
  languages/vue/scope-resolver.ts, not vue-sfc-extractor.ts
- amend the v35 comment to mention the 10 structural pairs it now also stamps

Still mutation-checked: dropping `FROM Function TO File` now fails both the
integration sweep and the unit pin with exactly Function|File. 89 tests green.

* fix(schema): generate the attachment pair surface and close four analyze aborts

Review of the generated scope-bridge cross product found four `analyze`
hard-aborts still live at head, each reproduced end-to-end on the default
user path (`analyze --index-only --skip-git`):

  Method→Annotation   Spring `@Bean` + `@ConditionalOnMissingBean` (Java + Kotlin)
  Method→File         Vue Options-API `methods:` handler bound to a child event
  Namespace→Record    COBOL `DECLARATIVES` / `USE AFTER STANDARD ERROR ON <file>`
  Class→Tool          `@mcp.tool()` applied to a class

All four are pre-existing on main, and both existing guards were structurally
blind to them: the unit guard derives from LINKABLE_LABELS ∪ CALL_TARGET_TYPES
(none of Annotation/Tool/Record/File-as-target is a member) and the corpus
guard ran three fixtures that exercise none of these emitters. All 16 tests
passed while all four crashes were live.

The PR's model — "bridge endpoint × structural endpoint" — does not fit:
Namespace→Record is structural on both sides. The property that does hold is
that the ANCHOR is a lookup result, not a literal at the emit site, so the
emitter cannot constrain its label. That gives a second closed-form rule:

  DEFINITION_ANCHOR_LABELS × ATTACHMENT_TARGET_LABELS

DEFINITION_ANCHOR_LABELS is derived from NODE_TABLES by subtraction, so a new
node table joins automatically. 332 → 450 declared pairs.

Sized against a committed harness (gitnexus/bench/schema-pairs), real
@ladybugdb/core, identical data: 450 costs 0.93–1.05× of 332 on untyped-endpoint
anchored queries — inside noise — versus 1.22–1.43× at 641 and 2.03–2.34× at
1024. The harness reproduces the known #2792 cliff, which is what makes the 450
figure trustworthy.

Also in this change:

- Delete the 161 hand-declared pairs the rules already generate (233 → 72).
  The declared set is byte-identical at 450; those lines were load-bearing
  shadow, because the generator suppresses anything already declared
  structurally, so narrowing a rule later would silently keep pairs alive.
  A new guard fails CI if a hand-declared pair is ever re-added inside a rule.
- Import LINKABLE_LABELS / CALL_TARGET_TYPES instead of hand-copying them.
  The twins' stated justification ("the ingestion layer must not be imported
  here") is false: csv-generator.ts and lbug-adapter.ts, siblings in the same
  directory, already do, and no rule in AGENTS.md / ARCHITECTURE.md /
  CONTRIBUTING.md / GUARDRAILS.md states otherwise.
- Resolve `resolveStreamGraphEmit` after the guards that rebind `options.force`,
  not at function entry. It gates on `force`, and every freshness guard runs
  ~360 lines later, so the v34→v35 bump would have pushed every existing index
  down the non-streamed emit path — losing the #2680 memory streaming added for
  the #2649 kernel-scale OOM, for exactly the population most likely to be
  memory-constrained.
- `UndeclaredRelationPairError` now carries the relationship type, both node ids
  and the source file, with a matching CLI branch. The old message named only
  the abstract label pair, which a user could not act on. Found through the
  cause chain, since pipeline-phases/runner.ts rewraps every phase failure.
- Share one classifier (`relPairKeyFor`) across the router, both emit sinks and
  the corpus guard, which previously hand-mirrored the router's skip rule; one
  cause-chain walker in lib/utils.ts; one exported pair-matching regex.
- Corpus guard: four new fixtures reproducing the aborts, per-fixture sentinel
  pairs so a fixture that stops emitting fails loudly instead of passing
  vacuously on an empty graph.

The per-edge path stays allocation-free: the failure context is passed
positionally and the message is built only inside the throw.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182jkjQqzACkJKYw4MLDnhX

* test(bench): re-baseline the COBOL capture fingerprint for the new fixture

`bench/scope-capture` globs `lang-resolution/cobol-*`, so the
`cobol-declaratives` fixture added in 81daf370e (to reproduce the
`Namespace→Record` analyze abort) joined that corpus and shifted the
fingerprint — 14 → 15 files.

Verified corpus-only, not a capture change: with that one fixture moved
aside the fingerprint is byte-identical to the prior baseline
(d45bb091…), and 81daf370e touches no COBOL capture code. The new value
reproduces CI's reported hash exactly. Scaling 0.677 < 1.5 budget.

`bench/scope-capture/measure.mjs --check` → PASS (15 languages).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182jkjQqzACkJKYw4MLDnhX

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 16:26:25 +01:00

180 lines
8.2 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

/**
* Corpus-derived coverage for the HAND-DECLARED half of `RELATION_SCHEMA`.
*
* `test/unit/schema-pair-coverage.test.ts` derives its requirement from
* schema.ts's two rules — the scope-resolution bridge cross product, and
* `DEFINITION_ANCHOR_LABELS × ATTACHMENT_TARGET_LABELS` for the framework and
* pipeline-phase overlays — so both generated halves are covered there. What
* neither rule can reach is `STRUCTURAL_PAIR_DDL`: the containment, inheritance
* and import pairs BETWEEN TWO DEFINITION LABELS. Eleven node tables are absent
* from every rule's target side (`CodeElement`, `Impl`, `Namespace`,
* `Template`, `TypeAlias`, `Typedef`, `Union`, `Static`, `Section`, `Folder`,
* and the PDG-only `BasicBlock`), so a pair pointing at one is hand-declared or
* it does not exist. No predicate describes that surface — any container can
* hold any definition — so this asks the emitters directly: run the real
* pipeline and require every FROM/TO pair it produces to be declared.
*
* Each entry also pins the pair it exists to guard. Without that the suite is
* vacuous: `undeclared` is derived from what the pipeline emitted, so a fixture
* that stopped emitting — renamed directory, grammar that failed to load,
* swallowed parse error — yields an empty set and passes green while guarding
* nothing. `sentinels` turns each case from "nothing undeclared" into "this
* emitter still fires, and everything it emits is declared".
*
* Coverage is bounded by `NON_BRIDGE_CORPUS`: a sample, not a proof. A
* language whose fixture is absent is unguarded, so a new structural emitter
* should land with an entry here.
*
* Deliberately isolated from the resolver suites that already build three of
* these graphs (`resolvers/cobol.test.ts`, `resolvers/vue.test.ts`,
* `resolvers/php.test.ts`) — see NOTE below the corpus before re-raising that.
*/
import { describe, it, expect, vi, beforeAll, afterAll } from 'vitest';
import path from 'path';
import { NODE_TABLES } from 'gitnexus-shared';
import { FIXTURES, runPipelineFromRepo } from './resolvers/helpers.js';
import { RELATION_SCHEMA } from '../../src/core/lbug/schema.js';
import { parseRelationSchemaPairs, relPairKeyFor } from '../../src/core/lbug/rel-pair-routing.js';
import { DIST_WORKER_URL, distWorkerExists } from '../helpers/worker-parse.js';
vi.setConfig({ testTimeout: 180_000 });
const describeIfWorkerBuilt = distWorkerExists() ? describe : describe.skip;
type CorpusEntry = {
/** Fixture directory under `test/fixtures/lang-resolution`. */
readonly fixture: string;
/**
* The emitter this fixture exists to exercise, short enough that vitest does
* not truncate it out of the case title (~36 chars).
*/
readonly emitter: string;
/**
* FROM|TO pairs the fixture MUST still emit. These are the anti-vacuity
* check: they fail loudly when the fixture stops reaching the emitter,
* which is the failure mode "no undeclared pairs" cannot see.
*/
readonly sentinels: readonly string[];
};
/**
* Fixtures chosen to reach a structural emitter that no other suite drives.
*
* The sentinels are the anti-vacuity anchor: each names an emitter that must
* still fire. Two of them (`CodeElement|Property`, `Module|Namespace`) are also
* the only pairs here that no generated rule can reach; the other nine are
* rule-derived, and stay because a rule DECLARING a pair says nothing about
* whether any emitter still PRODUCES it — which is the failure this corpus
* exists to catch.
*/
const NON_BRIDGE_CORPUS = [
{
// `cobol-processor.ts`: CONTAINS/CALLS/ACCESSES over Module / Namespace /
// Record / Property / CodeElement.
fixture: 'cobol-app',
emitter: 'cobol-processor containment',
sentinels: ['CodeElement|Property', 'Module|Namespace', 'Module|Record', 'Record|Record'],
},
{
// Same processor, DECLARATIVES section: the USE-procedure Namespace
// ACCESSES a file Record.
fixture: 'cobol-declaratives',
emitter: 'cobol-processor DECLARATIVES',
sentinels: ['Namespace|Record'],
},
{
// `languages/vue/scope-resolver.ts`: the only edge whose target is a
// `File`. Function→File from `<script setup>` (App.vue), Method→File from
// the Options-API `methods:` host (OptionsHost.vue).
fixture: 'vue-basic',
emitter: 'vue BINDS_EVENT_HANDLER',
sentinels: ['Function|File', 'Method|File'],
},
{
// The inheritance pass, including trait-to-trait IMPLEMENTS.
fixture: 'php-transitive-traits',
emitter: 'inheritance-pass IMPLEMENTS',
sentinels: ['Class|Trait', 'Trait|Trait'],
},
{
// `frameworks/spring/conditionals.ts`: @ConditionalOn* on a @Bean method,
// in both Java and Kotlin.
fixture: 'spring-conditional-app',
emitter: 'spring CONDITIONAL_ON',
sentinels: ['Method|Annotation'],
},
{
// `pipeline-phases/tools.ts`: a class-based MCP tool handler.
fixture: 'mcp-tool-class',
emitter: 'tools-phase HANDLES_TOOL',
sentinels: ['Class|Tool'],
},
] as const satisfies readonly CorpusEntry[];
/*
* NOTE — why this suite runs its own pipelines instead of reusing the resolver
* suites' graphs (measured, not assumed):
*
* 1. Vitest runs with `pool: 'forks'` and default isolation, so every test
* FILE gets its own child process. A fixture-keyed result cache in
* `resolvers/helpers.ts` would be per-file module state and would share
* nothing across files — zero saving.
* 2. The graphs are not interchangeable. `resolvers/cobol.test.ts` builds
* cobol-app with `{ skipGraphPhases: true }`, which drops the phase-emitted
* edges (`Function|Process`, `Function|Community`, and the whole class of
* pair `mcp-tool-class` exists to guard: HANDLES_TOOL is a pipeline phase).
* Asserting there would silently cover LESS surface than here.
* 3. Half the corpus has no existing home anyway, and the cases run
* concurrently, so they overlap into roughly one fixture's wall time: all
* 6 measured ~6s of test time against the ~16s this file spends on
* transform+import before the first case starts. Hosting only the 3
* homeless ones measured ~5s — a ~1s saving that still cannot remove a
* file, so it cannot remove that ~16s.
*/
const DECLARED = parseRelationSchemaPairs(RELATION_SCHEMA);
const VALID_TABLES = new Set<string>(NODE_TABLES);
// Cold worker-pool startups otherwise flake against the 5s default ready budget
// on a loaded runner, failing for reasons unrelated to the schema (#1741).
// Safe under `it.concurrent`: env stubs are process-global, but `pool: 'forks'`
// gives this file its own process and every case wants the same value.
beforeAll(() => vi.stubEnv('GITNEXUS_WORKER_READY_TIMEOUT_MS', '60000'));
afterAll(() => vi.unstubAllEnvs());
/**
* The FROM/TO pairs a fixture emits, deduped.
*
* Classifies through `relPairKeyFor` — the same call the emit path routes with
* — so this guard and the router cannot disagree about which edges are skipped.
* It keys off the node IDS, which is what `assertDeclaredPair` actually sees,
* not the node's `label` field.
*/
const pairsEmittedBy = async (fixture: string): Promise<Set<string>> => {
const result = await runPipelineFromRepo(path.join(FIXTURES, fixture), () => {}, {
workerPoolSize: 1,
workerUrlForTest: DIST_WORKER_URL,
});
const emitted = new Set<string>();
for (const rel of result.graph.iterRelationships()) {
const pair = relPairKeyFor(rel.sourceId, rel.targetId, VALID_TABLES);
if (pair !== undefined) emitted.add(pair);
}
return emitted;
};
describeIfWorkerBuilt('RELATION_SCHEMA covers the non-bridge emitters', () => {
// Concurrent because the cases share nothing but cost ~5s each serially,
// almost all of it worker spawn and grammar load, which overlaps well.
it.concurrent.each(NON_BRIDGE_CORPUS)(
'$fixture emits only declared FROM/TO pairs, and still reaches $emitter',
async ({ fixture, sentinels }) => {
const emitted = await pairsEmittedBy(fixture);
// Sorted so a failure is stable and names the pair to declare.
expect({
undeclaredPairs: [...emitted].filter((pair) => !DECLARED.has(pair)).sort(),
missingSentinelPairs: sentinels.filter((pair) => !emitted.has(pair)),
}).toEqual({ undeclaredPairs: [], missingSentinelPairs: [] });
},
);
});