GitNexus/gitnexus/test/unit/scope-resolution/cobol-import-target-parity.test.ts
Kevin Rajan 3236e2fbcd
fix(cobol): prefer copybook dirs so COPY EXTERNAL does not hit vendor decoys (#3240)
* fix(cobol): prefer copybook dirs so COPY EXTERNAL does not hit vendor decoys

COPY of an out-of-repo member first-won any same-named .cpy, so
vendor/EXTERNAL.cpy became a live cobol-copy IMPORTS edge. Share one
resolver between census and the regex processor: prefer copybooks/cpy/copy
plus the importer dir when present, else fail-open. Drop COBOL KNOWN_GAPS.

Fixes #2967

* bench(cobol): update depth budget for copybook-dir preference (#2967)

COBOL resolver now prefers well-known copybook directories (copybooks/,
cpy/, copy/, plus importer dir) over vendor paths when resolving COPY
statements. This intentional behavior change moves the resolver from
depth-free (prior measured ~0.885) to depth-sensitive (measured 1.751
on CI run 34394116972), because the new preferredCopybookDirs check
walks path components.

- Raise depth_budget from 1.6 to 2.4 (~1.37x the measured ratio)
- Update _measured.depth_ratio from 0.885 to 1.751
- Add _cobol_copybook_dir_preference_2967 note documenting the change

The COBOL fingerprints already reflect the new target set behavior
(vendor/EXTERNAL.cpy correctly returns null when a copybook dir is
present) per commit fc9e8270.

Co-authored-by: Kevin Rajan <kvnloo@users.noreply.github.com>

* bench(cobol): update baselines for copybook-dir preference (#2967)

COBOL preferred-dir filtering now affects resolution outcomes. The
unique-arm layouts (mixed copybooks/src dirs) drop from 1153 to 442
resolved as files outside preferred directories are correctly filtered.
The collide arm (all files in svc${d}/copybooks) keeps 1153 resolved
because ALL files remain in the preferred class.

- Update small/deep resolved: 1153 → 442
- Update small/large/deep fingerprints for new target set
- Raise heap_bound_bytes.cobol: 3500000 → 6200000 (1.5x measured 4112464 B)
- Add measure.mjs exception: collide legitimately differs from small
- Update _heap_bound_note with new cobol measurement context
- Expand _cobol_copybook_dir_preference_2967 note to explain collide delta

The collide arm now measures collision behavior within the preferred
class rather than across mixed layouts — an intentional outcome of the
preferred-dir semantics rather than a corpus defect.

Refs #2967

Co-authored-by: Kevin Rajan <kvnloo@users.noreply.github.com>

* fix(cobol): P1-A stem key with uppercase extensions, P1-B polyglot preferred-class latch

P1-A: Use raw extension for path.basename so CUSTREC.CPY keys as CUSTREC
  - Before: path.basename('CUSTREC.CPY', '.cpy') -> 'CUSTREC.CPY' (no strip)
  - After: path.basename('CUSTREC.CPY', '.CPY') -> 'CUSTREC' (stripped)
  - Processor used raw extension; census now matches

P1-B: Latch preferred-class only from copybook-tier paths
  - Before: docs/copy/README.md triggered hasPreferredDir = true
  - After: check preferred-dir only after extension filter
  - Processor receives polyglot allPathSet; test added

Test coverage:
  - cobol-copy-external-imports.test.ts: processor pins for both fixes
  - cobol-import-target-parity.test.ts: resolver parity updated to new behavior
  - All existing COBOL tests pass

Co-authored-by: Kevin Rajan <kvnloo@users.noreply.github.com>

* test(cobol): fix Mixed.CPY integration test for P1-A behavior

The integration test in cobol-import-index-reuse.test.ts had outdated
expectations from before P1-A. With P1-A, path.basename uses the raw
extension, so Mixed.CPY is now keyed as MIXED (extension stripped).

Before (pre-P1-A):
- Mixed.CPY → keyed as MIXED.CPY (uppercase ext not stripped)
- COPY MIXED.CPY → found, COPY MIXED → null

After (P1-A):
- Mixed.CPY → keyed as MIXED (raw ext stripped)
- COPY MIXED → found, COPY MIXED.CPY → null

Updated test expectations to match P1-A behavior. All three validation
tests now pass:
- cobol-import-index-reuse.test.ts (integration, index reuse)
- cobol-copy-external-imports.test.ts (processor P1-A/P1-B pins)
- cobol-import-target-parity.test.ts (resolver parity)

Fixes PER-994 CI failure on abhigyanpatwari/GitNexus#3240.

Co-authored-by: Kevin Rajan <kvnloo@users.noreply.github.com>

---------

Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Kevin Rajan <kvnloo@users.noreply.github.com>
2026-09-10 15:42:59 +01:00

389 lines
15 KiB
TypeScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

/**
* Differential harness for the COBOL `COPY`-target index hoist (#2908).
*
* `cobolScopeResolver.resolveImportTarget` used to answer every `COPY` with TWO
* full `allFilePaths` scans — copybooks first, then COBOL sources — each calling
* `path.extname` + `path.basename` + `toUpperCase` on every entry, so resolution
* cost O(copies × files) and a `COPY` of a member that is not in the repo (the
* common case) ran both scans to completion. Replacing them with a per-run
* two-tier index is a pure performance change ONLY if every implicit tie-break
* survives, and none of them is visible to the type system:
*
* - TIER ORDER: a `.cpy`/`.copybook` hit beats a `.cbl`/`.cob`/`.cobol` hit
* even when the source file comes FIRST in Set-iteration order. Collapsing
* the two tiers into one first-wins map is the "obvious" rewrite and it
* silently inverts this;
* - WITHIN A TIER: the first file in Set-iteration order wins, because the
* scans returned on first match;
* - CASE: the extension is compared LOWER-cased while the basename is
* compared UPPER-cased, and `path.basename(fp, ext)` strips the suffix only
* on an exact, case-sensitive match — so `Foo.CPY` is indexed under
* `FOO.CPY`, not `FOO`, and is unreachable by a `COPY FOO`;
* - `path` SEMANTICS: Node's `path.extname`/`path.basename` are what decide
* where the stem starts, and on POSIX they do not treat `\` as a separator.
* Hand-rolled slicing on `/` would start resolving backslash paths that
* previously returned null.
*
* So this file keeps a VERBATIM copy of the pre-change resolver body
* (`git show HEAD~:…/languages/cobol/scope-resolver.ts`) and asserts the new
* implementation agrees with it on a deterministic generated corpus plus a
* hand-built layout per tie-break. The copy is the specification; if an arm here
* fails, the resolver's OUTPUT moved and COBOL's IMPORTS edges move with it.
*
* Mutation-tested against the new implementation — each of these was inserted,
* confirmed RED here, and reverted: tiers collapsed into one map; within-tier
* first-wins flipped to last-wins; `targetRaw.toUpperCase()` dropped;
* `path.extname(fp).toLowerCase()` left un-lowercased.
*
* This file calls the resolver directly, which for COBOL is also the
* orchestrator adapter — but the arms below say nothing about the Set being
* passed THROUGH, and a defensive `new Set(allFilePaths)` would leave them all
* green while restoring the per-import rebuild. That failure is guarded by
* `test/integration/cobol-import-index-reuse.test.ts`.
*/
import path from 'node:path';
import { describe, expect, it } from 'vitest';
import { cobolScopeResolver } from '../../../src/core/ingestion/languages/cobol/scope-resolver.js';
const { resolveImportTarget } = cobolScopeResolver;
/** COBOL takes no `resolutionConfig` and ignores `fromFile`; both are pinned. */
const FROM_FILE = 'src/PROG.cbl';
function resolve(targetRaw: string, files: ReadonlySet<string>): string | readonly string[] | null {
return resolveImportTarget(targetRaw, FROM_FILE, files, undefined);
}
// ─── verbatim pre-change implementation ──────────────────────────────────────
const LEGACY_COPYBOOK_EXTENSIONS = new Set(['.cpy', '.copybook']);
function legacyResolveCobolImportTarget(
targetRaw: string,
allFilePaths: ReadonlySet<string>,
): string | null {
const upper = targetRaw.toUpperCase();
// Check copybook files first
for (const fp of allFilePaths) {
const extRaw = path.extname(fp);
const extLower = extRaw.toLowerCase();
if (!LEGACY_COPYBOOK_EXTENSIONS.has(extLower)) continue;
const basename = path.basename(fp, extRaw).toUpperCase();
if (basename === upper) return fp;
}
// Also search COBOL source files (.cbl, .cob, .cobol)
const COBOL_SOURCE_EXTS = new Set(['.cbl', '.cob', '.cobol']);
for (const fp of allFilePaths) {
const extRaw = path.extname(fp);
const extLower = extRaw.toLowerCase();
if (!COBOL_SOURCE_EXTS.has(extLower)) continue;
const basename = path.basename(fp, extRaw).toUpperCase();
if (basename === upper) return fp;
}
return null;
}
// ─── deterministic corpus ────────────────────────────────────────────────────
/** Murmur3 finalizer — a reproducible stand-in for `Math.random()`. */
function mix(n: number): number {
let x = n >>> 0;
x = Math.imul(x ^ (x >>> 16), 0x85ebca6b) >>> 0;
x = Math.imul(x ^ (x >>> 13), 0xc2b2ae35) >>> 0;
return (x ^ (x >>> 16)) >>> 0;
}
/**
* Directory shapes of a typical mainframe checkout, including one whose
* segments are separated by BACKSLASHES — on POSIX that is one long filename,
* which is precisely the `path` semantic the index must not "simplify" away.
*/
const DIRS = [
'',
'copybooks',
'COPYBOOKS',
'src',
'src/copy',
'legacy/cpy',
'jcl/proclib',
'win\\dir',
];
/** Member names in the case mixture a real repo has. */
const STEMS = ['CUSTREC', 'custrec', 'AcctRec', 'PAYROLL', 'BOOK', 'COMMON', 'TAXCALC', 'ERRDEMO'];
/**
* Both tiers, both cases, plus two extensions in NEITHER tier: `.txt` (a
* non-COBOL file that must never answer a `COPY`) and `''` (a file with no
* extension at all, which `path.extname` reports as the empty string and which
* therefore falls out of both extension sets).
*/
const EXTS = ['.cpy', '.copybook', '.CPY', '.cbl', '.cob', '.cobol', '.CBL', '.txt', ''];
/** Directories with no well-known copybook segment — fail-open parity only. */
const FAIL_OPEN_DIRS = ['', 'src', 'jcl/proclib', 'win\\dir'];
function corpusFromDirs(seed: number, fileCount: number, dirs: readonly string[]): Set<string> {
const files = new Set<string>();
for (let i = 0; i < fileCount; i++) {
const a = mix(seed * 7919 + i);
const b = mix(a ^ 0x9e3779b9);
const c = mix(b ^ 0x85ebca6b);
const dir = dirs[a % dirs.length];
const stem = STEMS[b % STEMS.length];
const rel = `${stem}${EXTS[c % EXTS.length]}`;
files.add(dir === '' ? rel : `${dir}/${rel}`);
}
// Backslash-separated paths, which `path` reads differently per platform and
// a `/`-slicing rewrite would read differently from `path` on POSIX.
files.add('win\\dir\\BOOK.cpy');
files.add('win\\dir\\PAYROLL.cbl');
return files;
}
function corpus(seed: number, fileCount: number): Set<string> {
return corpusFromDirs(seed, fileCount, DIRS);
}
/**
* `COPY` operands as they appear in source, plus the spellings that reach the
* corpus's awkward files. Lower-case entries are what breaks if the target
* stops being upper-cased; the `.CPY`/`.CBL` suffixed entries are what reaches
* a file whose mixed-case extension `path.basename` refused to strip.
*/
const TARGETS = [
'',
'CUSTREC',
'custrec',
'CustRec',
'ACCTREC',
'PAYROLL',
'payroll',
'BOOK',
'COMMON',
'TAXCALC',
'ERRDEMO',
'MISSING',
'BOOK.CPY',
'CUSTREC.CPY',
'PAYROLL.CBL',
'win\\dir\\BOOK',
'WIN\\DIR\\BOOK',
'win/dir/BOOK',
];
const REPOS = 40;
function hasPreferredCopyDir(files: ReadonlySet<string>): boolean {
for (const fp of files) {
const parts = fp.split('/');
for (let i = 0; i < parts.length - 1; i++) {
if (
parts[i] === 'copybooks' ||
parts[i] === 'COPYBOOKS' ||
parts[i] === 'cpy' ||
parts[i] === 'copy'
) {
return true;
}
}
}
return false;
}
describe('COBOL COPY-target index hoist — output parity with the pre-change scans (#2908)', () => {
it('agrees with the verbatim pre-change resolver over fail-open corpora (no well-known copybook dir)', () => {
let checked = 0;
for (let repo = 0; repo < REPOS; repo++) {
const files = corpusFromDirs(repo, 6 + (repo % 25), FAIL_OPEN_DIRS);
expect(hasPreferredCopyDir(files), `repo=${repo} accidentally grew a copybook dir`).toBe(
false,
);
for (const target of TARGETS) {
expect(resolve(target, files), `cobol "${target}" repo=${repo}`).toEqual(
legacyResolveCobolImportTarget(target, files),
);
checked++;
}
}
expect(checked).toBe(REPOS * TARGETS.length);
});
it('COPY EXTERNAL misses vendor/EXTERNAL.cpy when a copybooks/ dir is present (#2967)', () => {
const files = new Set(['copybooks/CUSTREC.cpy', 'vendor/EXTERNAL.cpy', 'src/PROG.cbl']);
expect(resolve('EXTERNAL', files)).toBeNull();
expect(resolve('CUSTREC', files)).toBe('copybooks/CUSTREC.cpy');
});
it('fail-open: without a copybook dir, COPY EXTERNAL still basename-matches (#2967)', () => {
const files = new Set(['vendor/EXTERNAL.cpy', 'src/PROG.cbl']);
expect(resolve('EXTERNAL', files)).toBe('vendor/EXTERNAL.cpy');
});
it('P1-A: uppercase .CPY extension does not break stem extraction', () => {
const files = new Set(['copybooks/CUSTREC.CPY', 'src/PROG.cbl']);
expect(resolve('CUSTREC', files)).toBe('copybooks/CUSTREC.CPY');
});
it('P1-B: polyglot file set with copy/cpy segments does not latch preferred-class', () => {
const files = new Set([
'docs/copy/README.md',
'src/copy/clipboard.ts',
'copybooks/CUSTREC.cpy',
'vendor/EXTERNAL.cpy',
'src/PROG.cbl',
]);
expect(resolve('CUSTREC', files)).toBe('copybooks/CUSTREC.cpy');
expect(resolve('EXTERNAL', files)).toBeNull();
});
it('the corpus actually resolves things (the parity arm is not vacuous)', () => {
// A corpus that resolved nothing would make the arm above pass on
// `null === null` forever. Measured on this corpus: 390 hits.
let hits = 0;
for (let repo = 0; repo < REPOS; repo++) {
const files = corpus(repo, 6 + (repo % 25));
for (const target of TARGETS) {
hits += legacyResolveCobolImportTarget(target, files) === null ? 0 : 1;
}
}
expect(hits).toBeGreaterThan(300);
});
});
// ─── hand-built tie-breaks ───────────────────────────────────────────────────
/**
* `path` decides where the stem of a backslash path starts: on POSIX the whole
* `dir\sub\BOOK` is the stem, on Windows only `BOOK`. Deriving the target
* through the SAME call keeps the two arms below a hit and a miss respectively
* on both platforms, so what they pin is the `path` semantics rather than the
* host — and a rewrite that replaces `path` with slicing on `/` changes what
* they resolve to on Windows.
*/
const BACKSLASH_FILE = 'dir\\sub\\BOOK.cpy';
const BACKSLASH_STEM = path.basename(BACKSLASH_FILE, '.cpy').toUpperCase();
/** The stem's last backslash-delimited segment — a HIT only where `path` splits on `\`. */
const BACKSLASH_LEAF = 'BOOK';
const BACKSLASH_LEAF_EXPECTED = BACKSLASH_STEM === BACKSLASH_LEAF ? BACKSLASH_FILE : null;
interface HandBuilt {
readonly why: string;
/** Insertion order IS the Set-iteration order, and for most arms it IS the tie-break. */
readonly files: readonly string[];
readonly target: string;
/** The one path (or `null`) both implementations must return. */
readonly expected: string | null;
}
const HANDBUILT: readonly HandBuilt[] = [
{
why: 'a copybook beats a COBOL source that comes FIRST in Set order (tier order)',
files: ['src/BOOK.cbl', 'copybooks/BOOK.cpy'],
target: 'BOOK',
expected: 'copybooks/BOOK.cpy',
},
{
why: '.copybook is tier 1 too, and beats an earlier .cob',
files: ['src/BOOK.cob', 'copybooks/BOOK.copybook'],
target: 'BOOK',
expected: 'copybooks/BOOK.copybook',
},
{
why: 'the source tier answers only when every copybook has missed',
files: ['copybooks/OTHER.cpy', 'src/BOOK.cbl'],
target: 'BOOK',
expected: 'src/BOOK.cbl',
},
{
why: 'within the copybook tier, first in Set order wins',
files: ['a/BOOK.cpy', 'b/BOOK.cpy'],
target: 'BOOK',
expected: 'a/BOOK.cpy',
},
{
why: 'within the source tier, first in Set order wins',
files: ['b/BOOK.cbl', 'a/BOOK.cob', 'c/BOOK.cobol'],
target: 'BOOK',
expected: 'b/BOOK.cbl',
},
{
why: 'the basename is compared UPPER-cased, so a lower-case file answers an upper-case COPY',
files: ['copybooks/custrec.cpy'],
target: 'CUSTREC',
expected: 'copybooks/custrec.cpy',
},
{
why: 'the TARGET is upper-cased too, so a lower-case COPY reaches an upper-case file',
files: ['copybooks/CUSTREC.cpy'],
target: 'custrec',
expected: 'copybooks/CUSTREC.cpy',
},
{
why: 'the extension is matched LOWER-cased for tier, stem stripped with lowercase ext, so `Foo.CPY` is reachable as FOO',
files: ['copybooks/Foo.CPY'],
target: 'FOO',
expected: 'copybooks/Foo.CPY',
},
{
why: 'a `.CPY` file (uppercase ext) is keyed without suffix, hits in tier 1',
files: ['x/BOOK.cbl', 'y/BOOK.CPY'],
target: 'BOOK',
expected: 'y/BOOK.CPY',
},
{
why: 'an uppercase source extension is a source file (`.CBL` → tier 2, keyed without suffix)',
files: ['src/Pay.CBL'],
target: 'PAY',
expected: 'src/Pay.CBL',
},
{
why: 'a file with NO extension never answers a COPY',
files: ['copybooks/BOOK'],
target: 'BOOK',
expected: null,
},
{
why: 'a non-COBOL extension never answers a COPY',
files: ['copybooks/BOOK.txt', 'docs/BOOK.md'],
target: 'BOOK',
expected: null,
},
{
why: 'a target matching nothing resolves to null',
files: ['copybooks/BOOK.cpy', 'src/PROG.cbl'],
target: 'NOSUCHBOOK',
expected: null,
},
{
why: 'an empty target matches nothing (no file has an empty stem)',
files: ['copybooks/BOOK.cpy', 'src/PROG.cbl'],
target: '',
expected: null,
},
{
why: 'a backslash path is addressed by the stem `path` reports for it',
files: [BACKSLASH_FILE],
target: BACKSLASH_STEM,
expected: BACKSLASH_FILE,
},
{
why: 'its trailing segment is a hit only where `path` treats `\\` as a separator',
files: [BACKSLASH_FILE],
target: BACKSLASH_LEAF,
expected: BACKSLASH_LEAF_EXPECTED,
},
];
describe('COBOL COPY-target index hoist — hand-built tie-breaks (#2908)', () => {
it.each(HANDBUILT)('$why', ({ files, target, expected }) => {
const set = new Set(files);
// Two assertions, not one: agreeing with the legacy copy proves the hoist
// preserved the behaviour, and pinning the literal proves the behaviour
// being preserved is the one the case is named for — `toEqual(null)` on
// both sides would otherwise satisfy an arm that stopped resolving.
expect(legacyResolveCobolImportTarget(target, set), `legacy: ${target}`).toBe(expected);
expect(resolve(target, set), `new: ${target}`).toBe(expected);
});
});