From 239967116f688f8a3f941788467b65e72f204517 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gerg=C5=91=20Magyar?= Date: Sat, 20 Jun 2026 20:40:28 +0100 Subject: [PATCH 1/4] fix(impact-pdg): make the Impact PDG Mutation Report workflow pass (3 latent oracle bugs) (#2258) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(impact-pdg): run mutation oracle's analyze child from built dist, not tsx-over-src The nightly Impact PDG Mutation Report workflow failed at the first fixture with ERR_MODULE_NOT_FOUND for src/cli/lazy-action.js. The harness shelled the real CLI out as `node --import tsx src/cli/index.ts analyze …`; on the CI runner's Node 22.22.3, native TypeScript type-stripping is enabled by default and handles the .ts entry instead of tsx, and native stripping does NOT remap the `./lazy-action.js` import specifier to lazy-action.ts the way tsx does — so CLI startup crashes before analyze even runs. The workflow already builds dist/ (build: 'true'). Prefer the shipped dist/cli/index.js (plain compiled JS — no tsx, no strip-types, and the parse workers it spawns also resolve from dist/) for the analyze child, falling back to tsx's own CLI over src only for build-free local runs. Production-faithful and version-agnostic across the engines range (node >=22.0). Verified on a real Node 22.22.3: the dist child starts cleanly with no lazy-action resolution error; the full `--mutation --only=inter-dispatcher-thin` run scores realized recall 1.0 and gate-mutation-recall passes. Workers are independently confirmed green on 22.22.3 in CI (run 27874383902). Co-Authored-By: Claude Opus 4.8 (1M context) * fix(impact-pdg): declare the mutation oracle's @babel/* deps `bench/impact-pdg/mutation-oracle.mjs` imports @babel/parser, @babel/traverse, @babel/generator and @babel/types to instrument + value-diff the fixture AST, but none were declared in package.json. @babel/parser and @babel/types happen to be hoisted into gitnexus/node_modules transitively, but @babel/traverse and @babel/generator are only present at the monorepo root — so a fresh `npm ci` in gitnexus/ (CI) can't resolve them and the oracle dies at module load with `Cannot find package '@babel/traverse'` right after analyze succeeds. Declare all four as devDependencies (they're already lazily imported only on the --mutation path, so they stay out of the unit-test module graph). Verified the oracle resolves them from gitnexus/node_modules and scores recall 1.0. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(impact-pdg): gate only recall-gated mutation checks (honor recallGated) The recall gate filtered checks by `typeof c.recall === 'number'`, which includes the UPSTREAM fixtures. The mutation oracle is a FORWARD value-diff: it mutates the criterion line and observes which downstream lines' values change, so its behavioral AIS can never intersect a reverse (upstream) PDG slice — recall is 0 by construction. measure.mjs already marks these `recallGated: false` (alongside id-discrimination corroboration cases) and excludes them from its own internal gate; the standalone gate just didn't honor that flag, so `intra-control-loop` (direction: upstream, recall 0) tripped the floor even though the oracle ran the full suite cleanly (mean recall 0.923). Filter on `c.recallGated === true` so the floor applies only to the downstream cases the forward oracle can fairly validate. Verified locally: an upstream+downstream report now scores 1 of 2 and the gate passes. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(impact-pdg): fail the mutation gate when it has no recall signal + fix README drift Tri-review hardening of this PR's own changes: - gate-mutation-recall.mjs: the floor check passed vacuously when `scored` was empty (`min === null` short-circuits `min !== null && min < floor`). Narrowing the filter to `recallGated === true` made an empty `scored` set reachable in more inputs (a degenerate corpus, or a harvest that silently emptied every behavioral AIS). Now fail loudly when checks exist but none are recall-gated, so a hollow gate is red rather than a green "scored cases: 0 of N". A genuinely empty report (0 checks) still passes — it's not a degenerate-corpus signal. - README.md: the harness substrate section still documented the old `node --import tsx src/cli/index.ts …` child invocation this PR replaced; update it to the dist-preferred form to match `cliChildArgs`. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- gitnexus/bench/impact-pdg/README.md | 8 +- .../bench/impact-pdg/gate-mutation-recall.mjs | 20 +- gitnexus/bench/impact-pdg/measure.mjs | 45 ++++- gitnexus/package-lock.json | 173 +++++++++++++----- gitnexus/package.json | 4 + 5 files changed, 195 insertions(+), 55 deletions(-) diff --git a/gitnexus/bench/impact-pdg/README.md b/gitnexus/bench/impact-pdg/README.md index 845debfa2..a7bbac3ce 100644 --- a/gitnexus/bench/impact-pdg/README.md +++ b/gitnexus/bench/impact-pdg/README.md @@ -227,10 +227,14 @@ analyze via a temp `GITNEXUS_HOME`, mock-free**. Per fixture: first, keeping the source tree clean). 2. **Shell out** to the real CLI as a child process — child-process isolation sidesteps `process.exit`; real `saveMeta` + `registerRepo` land in the temp - home; parse workers spawn from `dist/` (so the harness needs a built `dist/`): + home. The harness prefers the built `dist/` CLI (plain JS, no tsx; the parse + workers it spawns also load from `dist/`), so it needs a built `dist/`; it + falls back to tsx's own CLI over `src/` for build-free local runs. (`node + --import tsx src/cli/index.ts` is avoided: Node ≥22.18 native type-stripping + breaks the `.ts` entry's `./lazy-action.js`→`.ts` import resolution.) ``` - node --import tsx src/cli/index.ts analyze --pdg --skip-git --index-only + node dist/cli/index.js analyze --pdg --skip-git --index-only ``` 3. `new LocalBackend(); await init()` resolves the fixture via the **real** registry (the parent process sets `GITNEXUS_HOME` too, so `init()` reads the diff --git a/gitnexus/bench/impact-pdg/gate-mutation-recall.mjs b/gitnexus/bench/impact-pdg/gate-mutation-recall.mjs index 5d3e44a0d..2ab46072a 100644 --- a/gitnexus/bench/impact-pdg/gate-mutation-recall.mjs +++ b/gitnexus/bench/impact-pdg/gate-mutation-recall.mjs @@ -17,7 +17,14 @@ const floor = Number(process.env.MUTATION_RECALL_FLOOR ?? '0.5'); const report = JSON.parse(fs.readFileSync(reportPath, 'utf8')); const checks = Array.isArray(report?.mutation?.checks) ? report.mutation.checks : []; -const scored = checks.filter((c) => typeof c.recall === 'number'); +// Gate only the checks the oracle marked recall-gated. measure.mjs sets +// `recallGated: false` for cases a forward value-diff oracle cannot fairly +// score against the PDG slice: UPSTREAM fixtures (the oracle runs in its native +// downstream sense, so its behavioral AIS can never intersect a reverse slice — +// recall is 0 by construction) and id-discrimination corroboration fixtures. +// Those still carry a numeric `recall` for the report, so the legacy +// `typeof c.recall === 'number'` filter wrongly tripped the floor on them. +const scored = checks.filter((c) => c.recallGated === true && typeof c.recall === 'number'); const recalls = scored.map((c) => c.recall); const min = recalls.length ? Math.min(...recalls) : null; const mean = recalls.length ? recalls.reduce((a, b) => a + b, 0) / recalls.length : null; @@ -39,6 +46,17 @@ if (process.env.GITHUB_STEP_SUMMARY) { } process.stdout.write(summary + '\n'); +// A report that produced checks but gated NONE of them has no recall signal: +// the floor check below would pass vacuously (`min === null`). Fail loudly so a +// degenerate corpus, or a harvest that silently emptied every behavioral AIS, +// surfaces as a red run instead of a green "scored cases: 0 of N". +if (checks.length > 0 && scored.length === 0) { + console.error( + `Mutation gate has no signal: 0 of ${checks.length} checks were recall-gated — refusing to pass.`, + ); + process.exit(1); +} + if (min !== null && min < floor) { console.error(`Mutation recall regression: min realized recall ${fmt(min)} < floor ${floor}`); process.exit(1); diff --git a/gitnexus/bench/impact-pdg/measure.mjs b/gitnexus/bench/impact-pdg/measure.mjs index 65d3a7675..f6158f1da 100644 --- a/gitnexus/bench/impact-pdg/measure.mjs +++ b/gitnexus/bench/impact-pdg/measure.mjs @@ -25,11 +25,16 @@ * `repo-manager.getGlobalDir()` — it roots the registry; the per-repo DB * lands in `/.gitnexus/`, so fixtures are copied to a temp * working dir to keep the source tree clean); - * 2. SHELL OUT to the real CLI as a child process: - * node --import tsx src/cli/index.ts analyze --pdg --skip-git --index-only - * (child-process isolation sidesteps `process.exit`; real `saveMeta` + - * `registerRepo` land in the temp home; workers spawn from `dist/`, so the - * harness builds `dist/` first — run `node scripts/build.js`); + * 2. SHELL OUT to the real CLI as a child process (see `cliChildArgs`): + * node dist/cli/index.js analyze --pdg --skip-git --index-only + * preferring the BUILT `dist/` CLI when present — plain JS, no tsx, and the + * parse workers it spawns also load from `dist/`. The mutation workflow + * builds `dist/` first (`node scripts/build.js`); `node --import tsx + * src/cli/index.ts` is NOT used because Node >=22.18 native type-stripping + * breaks the `.js`->`.ts` entry resolution (ERR_MODULE_NOT_FOUND on + * `lazy-action.js`). Build-free runs fall back to tsx's own CLI over src. + * (Child-process isolation sidesteps `process.exit`; real `saveMeta` + + * `registerRepo` land in the temp home); * 3. `new LocalBackend(); await init()` resolves the fixture via the REAL * registry (the parent process ALSO sets `GITNEXUS_HOME` so init reads the * temp registry, not the user's ~/.gitnexus); @@ -53,6 +58,7 @@ import os from 'node:os'; import path from 'node:path'; import crypto from 'node:crypto'; import { spawnSync } from 'node:child_process'; +import { createRequire } from 'node:module'; import { fileURLToPath, pathToFileURL } from 'node:url'; import { @@ -95,6 +101,33 @@ const REPO_ROOT = path.resolve(__dirname, '..', '..'); // gitnexus/ const FIXTURES_DIR = path.join(__dirname, 'fixtures'); const BASELINE_PATH = path.join(__dirname, 'baselines.json'); const CLI_ENTRY = path.join(REPO_ROOT, 'src', 'cli', 'index.ts'); +// Shipped CLI entry (package.json `bin`). PREFERRED for the child analyze: it's +// plain compiled JS, so the analyze process — AND the parse workers it spawns, +// which resolve relative to the running entry — load from `dist/` with no tsx in +// the loop. The build-free path below stays as a fallback. +const DIST_CLI = path.join(REPO_ROOT, 'dist', 'cli', 'index.js'); +// Build-free fallback: tsx's OWN cli entry (resolved from this package), NOT +// `node --import tsx .ts`. On Node >=22.18 native TypeScript type- +// stripping is enabled by default and intercepts the `.ts` entry before tsx's +// `--import` resolve hook applies; native stripping does NOT remap `./foo.js` +// specifiers to `foo.ts` (tsx does), so `node --import tsx src/cli/index.ts` +// crashes resolving `./lazy-action.js` (ERR_MODULE_NOT_FOUND) on newer Node. +// The tsx CLI takes over module loading and is version-agnostic across the +// declared engines range (node >=22.0, where `--no-experimental-strip-types` +// is not a universally-recognized flag). Workers still spawn from src via tsx on +// this path, so it is only robust on the older Node devs run locally. +const TSX_CLI = createRequire(import.meta.url).resolve('tsx/cli'); + +/** + * Build the argv that runs the real CLI as a child of `process.execPath`. + * Prefers the built `dist/` CLI (production-faithful, no tsx, dist workers) when + * present — this is what the mutation workflow uses (it builds dist first). Falls + * back to the tsx CLI over src for build-free local runs. Returns the args AFTER + * the node binary, i.e. ready for `spawnSync(process.execPath, [...args])`. + */ +function cliChildArgs(rest) { + return fs.existsSync(DIST_CLI) ? [DIST_CLI, ...rest] : [TSX_CLI, CLI_ENTRY, ...rest]; +} const SCOPES = ['intra', 'inter', 'mixed']; const MODES = ['callgraph', 'pdg']; @@ -138,7 +171,7 @@ async function analyzeAndImpact(fx, home, { pdgOn = true } = {}) { fs.cpSync(path.join(fx.dir, 'src'), path.join(work, 'src'), { recursive: true }); const env = { ...process.env, GITNEXUS_HOME: home }; - const args = ['--import', 'tsx', CLI_ENTRY, 'analyze', work, '--skip-git', '--index-only']; + const args = cliChildArgs(['analyze', work, '--skip-git', '--index-only']); if (pdgOn) args.push('--pdg'); const an = spawnSync(process.execPath, args, { env, diff --git a/gitnexus/package-lock.json b/gitnexus/package-lock.json index b28f005d7..927fe93c3 100644 --- a/gitnexus/package-lock.json +++ b/gitnexus/package-lock.json @@ -52,6 +52,10 @@ "gitnexus": "dist/cli/index.js" }, "devDependencies": { + "@babel/generator": "^7.29.7", + "@babel/parser": "^7.29.7", + "@babel/traverse": "^7.29.7", + "@babel/types": "^7.29.7", "@types/busboy": "^1.5.4", "@types/cli-progress": "^3.11.6", "@types/cors": "^2.8.17", @@ -76,10 +80,59 @@ "typescript": "^6.0.3" } }, + "node_modules/@babel/code-frame": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.29.7.tgz", + "integrity": "sha512-Aup7aUOfpbAUg2ROOJN6Iw5f9DMBlzu0mIkm/malLQFN/YQgO48wCj0Kxa3sEHJvPVFg7siR+qRInwXd2qhQKw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/helper-validator-identifier": "^7.29.7", + "js-tokens": "^4.0.0", + "picocolors": "^1.1.1" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/code-frame/node_modules/js-tokens": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz", + "integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/@babel/generator": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/generator/-/generator-7.29.7.tgz", + "integrity": "sha512-DkXD5OJQaAQIdZ1bt3UZdEnHAn9Imd3IVBdX03UFe+ony9Ojw5pzr9YVKGDY1jt+Gcn/FnGkNf8r+Vj5NOJWtQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/parser": "^7.29.7", + "@babel/types": "^7.29.7", + "@jridgewell/gen-mapping": "^0.3.12", + "@jridgewell/trace-mapping": "^0.3.28", + "jsesc": "^3.0.2" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-globals": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-globals/-/helper-globals-7.29.7.tgz", + "integrity": "sha512-3nQVUAtvkKH9zahfWgw96Jc/uFOmjACE1kQz82E2lqWmHBgjzbNlsC22nuQTfahmWeQtTq5nQ/4Nnd2A1wj4zA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, "node_modules/@babel/helper-string-parser": { - "version": "7.27.1", - "resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.27.1.tgz", - "integrity": "sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA==", + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.29.7.tgz", + "integrity": "sha512-Pb5ijPrZ89GDH8223L4UP8i6QApWxs04RbPQJTeWDV0/keR2E36MeKnyr6LYmUUvqRRI+Iv87SuF1W6ErINzYw==", "dev": true, "license": "MIT", "engines": { @@ -87,9 +140,9 @@ } }, "node_modules/@babel/helper-validator-identifier": { - "version": "7.28.5", - "resolved": "https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-7.28.5.tgz", - "integrity": "sha512-qSs4ifwzKJSV39ucNjsvc6WVHs6b7S03sOh2OcHF9UHfVPqWWALUsNUVzhSBiItjRZoLHx7nIarVjqKVusUZ1Q==", + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-7.29.7.tgz", + "integrity": "sha512-qehxGkRj55h/ff8EMaJ+cYhyaKlHIxqYDn682wQD7RNp9UujOQsHog2uS0r2vzr4pW+sXf90NeeayjcNaX3fFg==", "dev": true, "license": "MIT", "engines": { @@ -97,13 +150,13 @@ } }, "node_modules/@babel/parser": { - "version": "7.29.2", - "resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.29.2.tgz", - "integrity": "sha512-4GgRzy/+fsBa72/RZVJmGKPmZu9Byn8o4MoLpmNe1m8ZfYnz5emHLQz3U4gLud6Zwl0RZIcgiLD7Uq7ySFuDLA==", + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.29.7.tgz", + "integrity": "sha512-hnORnjP/1P/zFEndoeX+n+t1RwWRJiJpM/jO7FW32Kn9r5+sJB2JWOdYo4L6k78j15eCwY3Gm/7364B1EMwtNg==", "dev": true, "license": "MIT", "dependencies": { - "@babel/types": "^7.29.0" + "@babel/types": "^7.29.7" }, "bin": { "parser": "bin/babel-parser.js" @@ -112,15 +165,49 @@ "node": ">=6.0.0" } }, - "node_modules/@babel/types": { - "version": "7.29.0", - "resolved": "https://registry.npmjs.org/@babel/types/-/types-7.29.0.tgz", - "integrity": "sha512-LwdZHpScM4Qz8Xw2iKSzS+cfglZzJGvofQICy7W7v4caru4EaAmyUuO6BGrbyQ2mYV11W0U8j5mBhd14dd3B0A==", + "node_modules/@babel/template": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/template/-/template-7.29.7.tgz", + "integrity": "sha512-puq+Gf35oI24FeN11LkoUQFqv9uwNeWpxXZi/Ji3rRIoKAzKnxRaZ+Gkj0vKS9ZCiTESfng1N9LyOyXvo+m+Gg==", "dev": true, "license": "MIT", "dependencies": { - "@babel/helper-string-parser": "^7.27.1", - "@babel/helper-validator-identifier": "^7.28.5" + "@babel/code-frame": "^7.29.7", + "@babel/parser": "^7.29.7", + "@babel/types": "^7.29.7" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/traverse": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/traverse/-/traverse-7.29.7.tgz", + "integrity": "sha512-EhlfNQtZ+NK22w5BM61ciuiq1m58ed33Wr1Xan//ZRTy6hgjnwyCffRYwzsGXdASJSUJ1guZILsErh1eQcl+zw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/code-frame": "^7.29.7", + "@babel/generator": "^7.29.7", + "@babel/helper-globals": "^7.29.7", + "@babel/parser": "^7.29.7", + "@babel/template": "^7.29.7", + "@babel/types": "^7.29.7", + "debug": "^4.3.1" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/types": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/types/-/types-7.29.7.tgz", + "integrity": "sha512-4zBIxpPzowiZpusoFkyGVwakdRJUyuH5PxQ/PrqghfdFWWasvnCdPfQXHrenDai+gyLARulZjZowCOj6fjT4pA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/helper-string-parser": "^7.29.7", + "@babel/helper-validator-identifier": "^7.29.7" }, "engines": { "node": ">=6.9.0" @@ -1128,6 +1215,17 @@ "node": ">=18.0.0" } }, + "node_modules/@jridgewell/gen-mapping": { + "version": "0.3.13", + "resolved": "https://registry.npmjs.org/@jridgewell/gen-mapping/-/gen-mapping-0.3.13.tgz", + "integrity": "sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/sourcemap-codec": "^1.5.0", + "@jridgewell/trace-mapping": "^0.3.24" + } + }, "node_modules/@jridgewell/resolve-uri": { "version": "3.1.2", "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz", @@ -1471,9 +1569,6 @@ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1491,9 +1586,6 @@ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -1511,9 +1603,6 @@ "ppc64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1531,9 +1620,6 @@ "s390x" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1551,9 +1637,6 @@ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1571,9 +1654,6 @@ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -3487,6 +3567,19 @@ "js-yaml": "bin/js-yaml.js" } }, + "node_modules/jsesc": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/jsesc/-/jsesc-3.1.0.tgz", + "integrity": "sha512-/sM3dO2FOzXjKQhJuo0Q173wf2KOo8t4I8vHy6lF9poUp7bKT0/NHE8fPX23PwfhnykfqnC2xRxOnVw5XuGIaA==", + "dev": true, + "license": "MIT", + "bin": { + "jsesc": "bin/jsesc" + }, + "engines": { + "node": ">=6" + } + }, "node_modules/json-bignum": { "version": "0.0.3", "resolved": "https://registry.npmjs.org/json-bignum/-/json-bignum-0.0.3.tgz", @@ -3668,9 +3761,6 @@ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -3692,9 +3782,6 @@ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -3716,9 +3803,6 @@ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -3740,9 +3824,6 @@ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MPL-2.0", "optional": true, "os": [ diff --git a/gitnexus/package.json b/gitnexus/package.json index 6fa0c9007..82f2cabfd 100644 --- a/gitnexus/package.json +++ b/gitnexus/package.json @@ -94,6 +94,10 @@ "uuid": "^14.0.0" }, "devDependencies": { + "@babel/generator": "^7.29.7", + "@babel/parser": "^7.29.7", + "@babel/traverse": "^7.29.7", + "@babel/types": "^7.29.7", "@types/busboy": "^1.5.4", "@types/cli-progress": "^3.11.6", "@types/cors": "^2.8.17", From 7916c315f0ca8f672794b7b41ee03446a70f787a Mon Sep 17 00:00:00 2001 From: Parafee41 Date: Sun, 21 Jun 2026 03:45:09 +0800 Subject: [PATCH 2/4] ci: fail tree-sitter summary parse drift (#2246) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * ci: fail tree-sitter summary parse drift * docs(ci): cross-reference readiness regexes to their test mirror The two report.match() literals in the upsert-issue github-script step are duplicated as _ISSUE_READY_RE / _ISSUE_BLOCKER_RE in test_check_tree_sitter_upgrade_readiness.py, and only the Python copy is asserted against the rendered report. Since a stale regex now throws via requireMatch (instead of the old silent '?' fallback), add a reciprocal keep-in-sync note at the workflow site so a future prose edit can't desync the JS literal from the asserted mirror undetected. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(ci): route read-phase network failures to fetch_failed, not a crash npm_view_json and fetch_text caught only (URLError, HTTPError[, JSONDecodeError]). urllib wraps connect-phase OSErrors into URLError, but a failure during resp.read() AFTER urlopen returns (ConnectionResetError, ssl.SSLError, socket.timeout, http.client.IncompleteRead) is not a URLError subclass — it escaped the helper, crashed main(), and left stdout empty. main() is unguarded (the only top-level except wraps just stdout.reconfigure), and the report print is its last statement, so an empty report then makes the workflow's requireMatch throw on a non-drift scheduled run. Broaden both except tuples with OSError + http.client.IncompleteRead so a transient mid-body network blip yields None, routing the grammar to the existing fetch_failed blocker bucket (a complete report) — preserving the fail-loud intent for real drift while removing the crash-to-empty-stdout path. JSONDecodeError stays explicit (it is a ValueError, not an OSError). Adds read-phase regression tests that fail on the old narrow tuple. Co-Authored-By: Claude Opus 4.8 (1M context) * test(ci): document the regex-contract assertion counts test_issue_update_summary_regex_matches_current_report asserts hardcoded capture groups ("9","10") and "2" with no explanation. Document the derivation from _render_report()'s mock corpus — 9 of 10 npm grammars Ready (tree-sitter-cpp is the intentional pin), 2 blockers (pinned tree-sitter-cpp + held vendored tree-sitter-c) — so a future grammar or pin change is an obvious two-step update (mock + counts) rather than a mystery failure. Assertions unchanged; comment only. Co-Authored-By: Claude Opus 4.8 (1M context) * ci: name _Resp method receivers 'self' to clear py/not-named-self The _Resp stub inside _patch_urlopen named its method receivers `self_inner`, which CodeQL flags as py/not-named-self (PEP 8) — three alerts on this PR's merge ref (lines 230/233/236). _patch_urlopen is a @staticmethod, so there is no outer `self` to collide with; rename the receivers to the conventional `self`. Pure rename, no behavior change. All 25 tests in test_check_tree_sitter_upgrade_readiness still pass. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Gergő Magyar Co-authored-by: Claude Opus 4.8 (1M context) --- .../check-tree-sitter-upgrade-readiness.py | 27 +++++++- ...est_check_tree_sitter_upgrade_readiness.py | 65 +++++++++++++++++++ .../tree-sitter-upgrade-readiness.yml | 31 +++++++-- 3 files changed, 116 insertions(+), 7 deletions(-) diff --git a/.github/scripts/check-tree-sitter-upgrade-readiness.py b/.github/scripts/check-tree-sitter-upgrade-readiness.py index 269afa7b2..c499f48e9 100644 --- a/.github/scripts/check-tree-sitter-upgrade-readiness.py +++ b/.github/scripts/check-tree-sitter-upgrade-readiness.py @@ -13,6 +13,7 @@ issue). stdlib-only — runs on any vanilla runner. from __future__ import annotations +import http.client import json import os import pathlib @@ -179,7 +180,20 @@ def npm_view_json(pkg: str) -> dict | None: req = urllib.request.Request(url, headers={"Accept": "application/json"}) with urllib.request.urlopen(req, timeout=8) as resp: return json.loads(resp.read().decode("utf-8")) - except (urllib.error.URLError, urllib.error.HTTPError, json.JSONDecodeError): + # OSError covers read-phase transport failures (ConnectionResetError, + # ssl.SSLError, socket.timeout) that escape resp.read() AFTER urlopen + # returns — urllib only wraps connect-phase OSErrors into URLError, so these + # are not URLError subclasses. http.client.IncompleteRead is an HTTPException, + # not an OSError, so it must be named explicitly. Returning None routes the + # grammar to the fetch_failed blocker bucket (a complete report) instead of + # crashing main() to empty stdout. + except ( + urllib.error.URLError, + urllib.error.HTTPError, + OSError, + http.client.IncompleteRead, + json.JSONDecodeError, + ): return None @@ -249,7 +263,16 @@ def fetch_text(url: str, timeout: int = 8) -> str | None: req = urllib.request.Request(url, headers=headers) with urllib.request.urlopen(req, timeout=timeout) as resp: return resp.read().decode("utf-8", errors="ignore") - except (urllib.error.URLError, urllib.error.HTTPError): + # See npm_view_json: OSError + http.client.IncompleteRead catch read-phase + # transport failures that escape resp.read() and are not URLError subclasses, + # so a transient network blip yields None (→ fetch_failed) rather than + # crashing the report to empty stdout. + except ( + urllib.error.URLError, + urllib.error.HTTPError, + OSError, + http.client.IncompleteRead, + ): return None diff --git a/.github/scripts/test_check_tree_sitter_upgrade_readiness.py b/.github/scripts/test_check_tree_sitter_upgrade_readiness.py index b3526fd9f..399a4ebc5 100644 --- a/.github/scripts/test_check_tree_sitter_upgrade_readiness.py +++ b/.github/scripts/test_check_tree_sitter_upgrade_readiness.py @@ -17,6 +17,7 @@ and the report never renders a bare ``?`` placeholder. All network is mocked. from __future__ import annotations import contextlib +import http.client import importlib.util import io import json @@ -213,6 +214,62 @@ class AssertCurrent(TestCase): self.assertIn("outside current runtime range", buf.getvalue()) +class FetchHelperReadPhaseErrors(TestCase): + """Read-phase transport failures — raised by resp.read() AFTER urlopen has + returned (ConnectionResetError, ssl.SSLError, socket.timeout, + http.client.IncompleteRead) — are NOT urllib.error.URLError subclasses + (urllib only wraps connect-phase OSErrors). A prior revision's narrow except + tuple let them escape npm_view_json / fetch_text, crash main(), and empty + stdout — which makes the workflow's requireMatch throw on a non-drift + scheduled run. The helpers must swallow them to None so the grammar routes to + the fetch_failed blocker bucket and the report still renders completely.""" + + @staticmethod + def _patch_urlopen(*, read_returns=None, read_raises=None): + class _Resp: + def __enter__(self): + return self + + def __exit__(self, *exc): + return False + + def read(self, *a, **k): + if read_raises is not None: + raise read_raises + return read_returns + + def _fake_urlopen(*a, **k): + return _Resp() + + import urllib.request + + return mock.patch.object(urllib.request, "urlopen", side_effect=_fake_urlopen) + + def test_npm_view_json_swallows_read_phase_connection_reset(self): + # ConnectionResetError is an OSError but NOT a URLError — the broadened + # OSError clause must catch it so the helper returns None, not raises. + with mock.patch.object(readiness, "OFFLINE", False), self._patch_urlopen( + read_raises=ConnectionResetError("peer reset mid-body") + ): + self.assertIsNone(readiness.npm_view_json("tree-sitter-anything")) + + def test_fetch_text_swallows_read_phase_incomplete_read(self): + # http.client.IncompleteRead is an HTTPException (not OSError), so it must + # be named explicitly in the except tuple. + with mock.patch.object(readiness, "OFFLINE", False), self._patch_urlopen( + read_raises=http.client.IncompleteRead(partial=b"half") + ): + self.assertIsNone(readiness.fetch_text("https://example.com/parser.c")) + + def test_npm_view_json_still_swallows_bad_json(self): + # JSONDecodeError is a ValueError, not an OSError — broadening the tuple + # must not drop it. Non-JSON body still yields None. + with mock.patch.object(readiness, "OFFLINE", False), self._patch_urlopen( + read_returns=b"<>" + ): + self.assertIsNone(readiness.npm_view_json("tree-sitter-anything")) + + class ReportRendering(TestCase): @classmethod def setUpClass(cls): @@ -303,6 +360,14 @@ class ReportRendering(TestCase): blockers = _ISSUE_BLOCKER_RE.search(self.report) self.assertIsNotNone(ready) self.assertIsNotNone(blockers) + # Counts are derived from _render_report()'s mock corpus (all npm peer + # deps mocked permissive): of the 10 npm-installed grammars, 9 render + # Ready and 1 — tree-sitter-cpp — is the intentional pin (#1242), so it is + # not counted ready. The 2 blockers are that same pinned tree-sitter-cpp + # plus the vendored, ABI-held tree-sitter-c (the only out-of-range + # vendored grammar). If a grammar is added/removed or a pin/hold changes, + # update _render_report()'s mock AND these expected counts together; a + # mismatch here means the report prose drifted, not the regex. self.assertEqual(ready.groups(), ("9", "10")) self.assertEqual(blockers.group(1), "2") diff --git a/.github/workflows/tree-sitter-upgrade-readiness.yml b/.github/workflows/tree-sitter-upgrade-readiness.yml index 72fa54b55..d4a774633 100644 --- a/.github/workflows/tree-sitter-upgrade-readiness.yml +++ b/.github/workflows/tree-sitter-upgrade-readiness.yml @@ -133,11 +133,32 @@ jobs: const existing = open.find(i => i.title === title); if (existing) { // Extract ready/total count for the changelog comment. - const readyMatch = report.match(/- (\d+)\/(\d+) npm-installed grammars already accept tree-sitter@/); - const blockerMatch = report.match(/\*\*Blocked\*\* — (\d+) grammars? /); - const ready = readyMatch ? readyMatch[1] : '?'; - const total = readyMatch ? readyMatch[2] : '?'; - const blockers = blockerMatch ? blockerMatch[1] : '?'; + // The two report.match() regexes below are mirrored as + // _ISSUE_READY_RE / _ISSUE_BLOCKER_RE in + // .github/scripts/test_check_tree_sitter_upgrade_readiness.py, which is + // the ONLY place the contract is asserted against the rendered report. + // Keep all three in sync: changing the report prose means updating both + // these literals AND the test mirror, or requireMatch throws on the next + // scheduled run (the silent "?" fallback that used to hide drift is gone). + const requireMatch = (name, match) => { + if (!match) { + throw new Error( + `Could not extract ${name} from tree-sitter readiness report`, + ); + } + return match; + }; + const readyMatch = requireMatch( + 'ready npm grammar count', + report.match(/- (\d+)\/(\d+) npm-installed grammars already accept tree-sitter@/), + ); + const blockerMatch = requireMatch( + 'blocker count', + report.match(/\*\*Blocked\*\* — (\d+) grammars? /), + ); + const ready = readyMatch[1]; + const total = readyMatch[2]; + const blockers = blockerMatch[1]; // Find grammars whose status changed by diffing the old and // new table rows. Each row looks like: From 221069785bea9b5a3bd6bd823cd28e61554e01be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gerg=C5=91=20Magyar?= Date: Sat, 20 Jun 2026 21:09:54 +0100 Subject: [PATCH 3/4] chore: release v1.6.8 (#2260) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bump gitnexus, plugin.json, and marketplace.json to 1.6.8 and add the CHANGELOG section. Headline: opt-in PDG-backed impact analysis plus the full PDG/taint substrate (CFG → reaching-defs → intra/inter-procedural taint → control dependence) across the language matrix, multi-branch indexing, private GitHub PAT + Azure DevOps support, and MCP trace/HTTP. Co-authored-by: Claude Opus 4.8 (1M context) --- .claude-plugin/marketplace.json | 2 +- .../.claude-plugin/plugin.json | 2 +- gitnexus/CHANGELOG.md | 56 +++++++++++++++++++ gitnexus/package-lock.json | 4 +- gitnexus/package.json | 2 +- 5 files changed, 61 insertions(+), 5 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 9d95ede9f..62a46d586 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -11,7 +11,7 @@ "plugins": [ { "name": "gitnexus", - "version": "1.6.7", + "version": "1.6.8", "source": "./gitnexus-claude-plugin", "description": "Code intelligence powered by a knowledge graph. Provides execution flow tracing, blast radius analysis, and augmented search across your codebase." } diff --git a/gitnexus-claude-plugin/.claude-plugin/plugin.json b/gitnexus-claude-plugin/.claude-plugin/plugin.json index fe3e641ac..875e339b6 100644 --- a/gitnexus-claude-plugin/.claude-plugin/plugin.json +++ b/gitnexus-claude-plugin/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "gitnexus", "description": "Code intelligence powered by a knowledge graph. Provides execution flow tracing, blast radius analysis, and augmented search across your codebase.", - "version": "1.6.7", + "version": "1.6.8", "author": { "name": "GitNexus" }, diff --git a/gitnexus/CHANGELOG.md b/gitnexus/CHANGELOG.md index e860e71cd..f716d45c7 100644 --- a/gitnexus/CHANGELOG.md +++ b/gitnexus/CHANGELOG.md @@ -4,6 +4,62 @@ All notable changes to GitNexus will be documented in this file. ## [Unreleased] +## [1.6.8] - 2026-06-20 + +### Added + +- **PDG-backed impact analysis (opt-in)** — `impact` gains a `mode: 'pdg'` that runs statement-level and inter-procedural program slicing for far more precise blast radius, with resolved-callee-id soundness and validation by a mutation oracle; the default call-graph mode is unchanged (#2227) +- **Program Dependence Graph substrate across the language matrix** — a control-flow-graph layer landed for TS/JS (#2099) and was extended to PDG/CFG visitors for every supported language (#2197); on top of it an intra-procedural `REACHING_DEF` data-dependence layer (#2160), value-position branches (`if`/`when`/`switch`/`match`/`try` used as expressions) modeled as control dependence (#2211), and full control dependence via post-dominators + a Ferrante-style CDG (#2188). All layers are opt-in behind `--pdg`; a default `analyze` run stays byte-identical (#2082, #2085, #2205, #2207, #2195) +- **Taint analysis** — intra-procedural taint tracking over the PDG (#2164) plus inter-procedural taint via function summaries propagated over resolved `CALLS` edges (#2179) (#2083, #2084) +- **Multi-branch indexing and branch-scoped querying** — analyze and query a repository per branch, with each branch stored under its own subdirectory and the primary branch layout left unchanged (#2137, #2106) +- **Private GitHub repos via PAT + Azure DevOps Server support** — `gitnexus analyze` can clone private GitHub repositories with a personal access token and supports Azure DevOps Server remotes (#2223, #2076, #2210) +- **MCP `trace` tool** — returns the shortest call path between two symbols (#2173) +- **MCP HTTP server** — `gitnexus mcp --http` exposes the server over Streamable HTTP with legacy SSE transport support (#2141) +- **HTTP route extraction** — Java Spring route annotations are now extracted into `Route` nodes (#2078), and the HTTP method is persisted on each `Route` node (#2234, #2138) +- **`gitnexus analyze` circular import cycle check** (#2166) +- **`gitnexus analyze` embeddings flags** — `--embeddings-baseurl`, `--embeddings-model`, `--embeddings-auth-token`, and `--embeddings-dims` to point analyze at a custom embeddings provider (#2140) +- **`gitnexus setup` coding-agent integration selection** — choose which coding-agent integrations to install during setup (#2168) +- **C++ CUDA source extensions parsed** — `.cu`/`.cuh` files are now ingested (#2213) + +### Fixed + +- **`impact()` / `route_map` under-reporting blast radius** — name-resolution gaps that caused callers and routes to be dropped are fixed, with ambiguous symbols reported per-candidate (#2136, #2129, #1858, #1852) +- **Single-ancestor method override detection in the MRO processor** (#2199) +- **MCP `query` / `cypher` parameter names** — renamed so Claude Code can invoke them, while still accepting the legacy parameter (#2186) +- **C++ overload resolution** — homogeneous braced-init overloads are now ranked (#2214), deleted overload winners are suppressed (#2094), and the C++ hook layer handles pack-base comments and missing hook overrides (#2247) +- **Large-repo `analyze` crash** — the pipeline now survives non-cloneable worker results instead of aborting (#2135, #2112) +- **Embeddings** — `onnxruntime-common` resolves under pnpm-strict / `pnpm dlx` installs (#2139, #307), and the `VECTOR` index is created via `conn.query` rather than the prepared-statement path that silently skipped it (#2114) +- **Vendored tree-sitter grammars** — loaded from `vendor/` by absolute path so analyze finds them regardless of CWD (#2144, #2111) +- **Registry wipe on transient I/O errors prevented** — a failed read no longer clears the repository registry (#2124) +- **Server roots resolve from `GITNEXUS_HOME`** — clone, upload, and mapping roots honor the configured home directory (#2229) +- **Wiki generation keeps the graph DB pinned** so it is not evicted mid-generation (#2232) +- **Group sync pins repositories** so large groups resolve their cross-repo links (#2191) +- **Web viewer** — a chat-only mode for large projects prevents the WebUI from hanging (#2185, #2178), and the broken Browse-for-folder control was replaced with an upload directory picker (#1850) +- **Hooks** — the augment CLI child is wrapped in the orphan guard (#2169), db-lock probe subprocesses are bounded and gated behind a hook slot (#2165), and the MCP-owned-DB augment-skip diagnostic is silenced for strict hook runners (#2134, #2163, #1913) +- **Docker image ships runtime-needed published assets** — `hooks/` and `skills/` are copied into the image so `gitnexus analyze` no longer crashes with `MODULE_NOT_FOUND` (#2132, #2130) +- **`gitnexus analyze` preserves trailing spaces in git roots** (#2192) +- **Write-route origin guard scoped to the server's own bound host** (#2172) +- **Impact PDG Mutation Report workflow** — fixed three latent oracle bugs (dist-CLI invocation under Node ≥ 22.18 type-stripping, undeclared `@babel/*` deps, and a recall-gated check filter) so the mutation oracle CI runs green (#2258) + +### Changed + +- **tree-sitter readiness/summary CI hardened** — readiness and grammar-update workflows aligned on a shared manifest (#2187, #858), readiness summary counts kept current (#2196), and the summary now fails on parse drift (#2246) +- **Devcontainer simplified** — Dockerfile and `devcontainer.json` no longer pin version args for the AI CLIs (#2174) + +### Performance + +- **Graph-DB emit/persistence** — cut overall emit/persistence wall time (#2215) and overlap node `COPY` with relationship emit (#2226) (#2203) +- **PDG/CFG emit** — streaming/chunked PDG graph emit for full-kernel-scale repos (#2216, #2202) and an SSA-sparse reaching-defs solver replacing the dense-set worklist (#2212, #2201) +- **Hook db-lock scan** — cmdline-first on Linux, dropping the `lsof` fallback (#2183, #2180) + +### Chore / Dependencies + +- **gitnexus runtime** — bump `hono` 4.12.23 → 4.12.26 (#2244), `tar` 7.5.13 → 7.5.16 (#2218), `protobufjs` 7.5.8 → 7.6.4 (#2219), `js-yaml` 4.1.1 → 4.2.0 (#2097, #2217), and an `npm_and_yarn` security group (3 updates) (#2220) +- **gitnexus dev** — bump `vitest` 4.1.8 → 4.1.9 (#2249), `@vitest/coverage-v8` (#2250), `esbuild` 0.28.0 → 0.28.1 (#2182), and `@types/node` (#2128, #2222) +- **gitnexus-web** — bump `react-dom` 19.2.6 → 19.2.7 (#2240), `langchain` 1.4.2 → 1.4.4 (#2149), `@langchain/langgraph` (#2235), `@langchain/ollama` (#2236), `mnemonist` 0.39.8 → 0.40.4 (#2237), `lucide-react` (#2238), `sigma` 3.0.2 → 3.0.3 (#2151), `dompurify` 3.4.7 → 3.4.8 (#2150, #2245), `@vercel/node` (#2156), and `@vitest/coverage-v8` (#2153) +- **eval** — bump `aiohttp` (#2224) +- **CI actions** — bump `gitleaks/gitleaks-action` 2.3.9 → 3.0.0 (#2241), `github/codeql-action` 4.36.0 → 4.36.2 (#2242), `actions/checkout` 6.0.2 → 6.0.3 (#2152), `actions/attest-build-provenance` 2.4.0 → 4.1.0 (#2158), `docker/setup-qemu-action` 4.0.0 → 4.1.0 (#2159), `release-drafter/release-drafter` 7.3.0 → 7.3.1 (#2157), and `actions/setup-python` 5.6.0 → 6.2.0 (#2155) + ## [1.6.7] - 2026-06-09 ### Added diff --git a/gitnexus/package-lock.json b/gitnexus/package-lock.json index 927fe93c3..97bbbc1a1 100644 --- a/gitnexus/package-lock.json +++ b/gitnexus/package-lock.json @@ -1,12 +1,12 @@ { "name": "gitnexus", - "version": "1.6.7", + "version": "1.6.8", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "gitnexus", - "version": "1.6.7", + "version": "1.6.8", "hasInstallScript": true, "license": "PolyForm-Noncommercial-1.0.0", "dependencies": { diff --git a/gitnexus/package.json b/gitnexus/package.json index 82f2cabfd..f4adcb9da 100644 --- a/gitnexus/package.json +++ b/gitnexus/package.json @@ -1,6 +1,6 @@ { "name": "gitnexus", - "version": "1.6.7", + "version": "1.6.8", "description": "Graph-powered code intelligence for AI agents. Index any codebase, query via MCP or CLI.", "author": "Abhigyan Patwari", "license": "PolyForm-Noncommercial-1.0.0", From f44c0714cea2801894655daf442c0f1df1792c47 Mon Sep 17 00:00:00 2001 From: azizur100389 Date: Sat, 20 Jun 2026 21:47:06 +0100 Subject: [PATCH 4/4] feat(taint): add Python source/sink model (#2253) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add Python taint source sink model * Fix Python taint argument and class shadowing * Address Python taint review cleanup * test(cfg): align Python keyword argument harvest --------- Co-authored-by: Gergő Magyar --- .../ingestion/cfg/visitors/python-harvest.ts | 119 ++++++++----- .../scope-resolution/pipeline/run.ts | 10 +- .../src/core/ingestion/taint/python-model.ts | 46 +++++ .../core/ingestion/taint/typescript-model.ts | 29 +++- gitnexus/test/unit/cfg/harvest.test.ts | 8 +- gitnexus/test/unit/cfg/python-visitor.test.ts | 34 +++- gitnexus/test/unit/taint/model-match.test.ts | 20 ++- .../unit/taint/python-model-match.test.ts | 162 ++++++++++++++++++ 8 files changed, 351 insertions(+), 77 deletions(-) create mode 100644 gitnexus/src/core/ingestion/taint/python-model.ts create mode 100644 gitnexus/test/unit/taint/python-model-match.test.ts diff --git a/gitnexus/src/core/ingestion/cfg/visitors/python-harvest.ts b/gitnexus/src/core/ingestion/cfg/visitors/python-harvest.ts index c272544b8..acecf42c1 100644 --- a/gitnexus/src/core/ingestion/cfg/visitors/python-harvest.ts +++ b/gitnexus/src/core/ingestion/cfg/visitors/python-harvest.ts @@ -116,10 +116,10 @@ export class PythonHarvester { /** >0 while walking a conditionally-evaluated subexpression — defs become may-defs. */ private conditionalDepth = 0; /** - * `call` node id → binding indices its single-target result is assigned to - * (`x = f()` ⇒ `[x]`). Populated just before the value walk reaches the call - * (see {@link registerResultDefs}) and consumed by {@link visitCall}. Mirrors - * the Go harvester's `resultDefTargets`. + * `call` node id → binding indices its result is assigned to (`x = f()` ⇒ + * `[x]`, `a, b = f()` ⇒ `[a, b]`). Populated just before the value walk reaches + * the call (see {@link registerResultDefs}) and consumed by {@link visitCall}. + * Mirrors the Go harvester's `resultDefTargets`. */ private readonly resultDefTargets = new Map(); @@ -227,7 +227,18 @@ export class PythonHarvester { */ private prescan(node: SyntaxNode): void { const t = node.type; - if (NESTED_FUNCTION_TYPES.has(t) && node.id !== this.fnId) return; + if (NESTED_FUNCTION_TYPES.has(t) && node.id !== this.fnId) { + if (t === 'function_definition') { + const name = node.childForFieldName('name'); + if (name?.type === 'identifier') this.declare(name, 'function'); + } + return; + } + if (t === 'class_definition') { + const name = node.childForFieldName('name'); + if (name?.type === 'identifier') this.declare(name, 'class'); + return; + } switch (t) { case 'global_statement': @@ -555,9 +566,9 @@ export class PythonHarvester { const left = node.childForFieldName('left'); const right = node.childForFieldName('right'); // Register result-defs BEFORE walking the value so the nested call site - // (reached during the value walk) carries them — single identifier - // target only (`x = f()`); a tuple/attribute LHS attaches nothing. - if (left?.type === 'identifier' && right) this.registerResultDefs(right, [left]); + // (reached during the value walk) carries them. Plain and unpack targets + // are preserved; attribute/subscript LHS attaches nothing. + if (left && right) this.registerResultDefs(right, this.targetDefIndices(left)); if (right) this.walkValue(right, acc); if (left) this.defTargets(left, acc); return; @@ -581,7 +592,9 @@ export class PythonHarvester { // walrus `(n := v)` — `n` is a def, `v` a use. const name = node.childForFieldName('name'); const value = node.childForFieldName('value'); - if (name?.type === 'identifier' && value) this.registerResultDefs(value, [name]); + if (name?.type === 'identifier' && value) { + this.registerResultDefs(value, this.targetDefIndices(name)); + } if (value) this.walkValue(value, acc); if (name?.type === 'identifier') this.def(name, acc); return; @@ -673,26 +686,33 @@ export class PythonHarvester { if (body) this.walkValue(body, acc); } - // ── taint-site harvest (#2227 follow-up) ───────────────────────────────── - - /** - * When `value`'s root (after stripping parens) is a `call`, remember that - * call site should carry `resultDefs` — the binding indices of `targets` - * (def-position identifiers). Consumed by {@link visitCall} once the value - * walk reaches the node. Single-target only (the caller restricts to a plain - * identifier LHS); the blank identifier (`_`) binds nothing and is skipped. - */ - private registerResultDefs(value: SyntaxNode, targets: readonly SyntaxNode[]): void { - const root = this.unwrapValue(value); - if (root.type !== 'call') return; - const defs: number[] = []; - for (const target of targets) { - if (target.type !== 'identifier' || target.text === '_') continue; - defs.push(this.resolve(target)); + /** Binding indices assigned by a target pattern, without mutating statement facts. */ + private targetDefIndices(target: SyntaxNode): number[] { + if (target.type === 'identifier') return target.text === '_' ? [] : [this.resolve(target)]; + if (PATTERN_LIST_TYPES.has(target.type)) { + const out: number[] = []; + for (let i = 0; i < target.namedChildCount; i++) { + const c = target.namedChild(i); + if (c) out.push(...this.targetDefIndices(c)); + } + return out; } - if (defs.length > 0) this.resultDefTargets.set(root.id, defs); + if (target.type === 'list_splat_pattern') { + const id = target.namedChild(0); + return id ? this.targetDefIndices(id) : []; + } + return []; } + /** Record result defs for a call-valued assignment. */ + private registerResultDefs(value: SyntaxNode, defs: readonly number[]): void { + if (defs.length === 0) return; + const root = this.unwrapValue(value); + if (root.type === 'call') this.resultDefTargets.set(root.id, [...defs]); + } + + // ── taint-site harvest (#2227 follow-up) ───────────────────────────────── + /** Strip `parenthesized_expression` wrappers around a value (`(f())`). */ private unwrapValue(node: SyntaxNode): SyntaxNode { let n = node; @@ -740,28 +760,37 @@ export class PythonHarvester { } const resultDefs = this.resultDefTargets.get(node.id); if (resultDefs !== undefined) acc.setSiteResultDefs(siteIdx, resultDefs); - if (argsNode) { - let pos = 0; - for (let i = 0; i < argsNode.namedChildCount; i++) { - const arg = argsNode.namedChild(i); - if (!arg || arg.type === 'comment') continue; - if (arg.type === 'list_splat' || arg.type === 'dictionary_splat') { - // `f(*args)` / `f(**kw)` — a spread. Mark the first spread position so - // the matcher degrades soundly; the inner value still walks. - acc.setFrameArg(pos); - acc.setSiteSpread(siteIdx, pos); - const inner = arg.namedChild(0); - if (inner) this.walkValue(inner, acc); - } else { - // Positional or `keyword_argument` — the `keyword_argument` case in - // `walkValue` walks only its `value`, so the key name is not a use. - acc.setFrameArg(pos); - this.walkValue(arg, acc); - } + this.walkArgs(argsNode, acc, siteIdx); + acc.popFrame(); + } + + /** Walk arguments, assigning only positional values to positional sink slots. */ + private walkArgs(args: SyntaxNode | null, acc: FactAccumulator, siteIdx: number): void { + if (!args) return; + let pos = 0; + for (let i = 0; i < args.namedChildCount; i++) { + const arg = args.namedChild(i); + if (!arg || arg.type === 'comment') continue; + if (arg.type === 'keyword_argument') { + const value = arg.childForFieldName('value'); + // SiteRecord has no keyword-name metadata. Keep the value's ordinary + // uses/sources, but do not guess a positional sink slot from source order. + if (value) acc.suppressOccurrences(() => this.walkValue(value, acc)); + } else if (arg.type === 'list_splat') { + acc.setFrameArg(pos); + acc.setSiteSpread(siteIdx, pos); + const inner = arg.namedChild(0); + if (inner) this.walkValue(inner, acc); + pos++; + } else if (arg.type === 'dictionary_splat') { + const inner = arg.namedChild(0); + if (inner) acc.suppressOccurrences(() => this.walkValue(inner, acc)); + } else { + acc.setFrameArg(pos); + this.walkValue(arg, acc); pos++; } } - acc.popFrame(); } /** diff --git a/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts b/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts index e8d74d194..62cb0083e 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts @@ -844,11 +844,11 @@ export function runScopeResolution( // ── M3 taint setup (#2083 U4) ──────────────────────────────────────── // Explicit model-registration seam (idempotent, cheap) — the registry // stays empty on non-pdg runs, preserving default-run parity. The - // registry is keyed by `SupportedLanguages` enum VALUES ('typescript' / - // 'javascript'), and `ScopeResolver.language` IS a `SupportedLanguages` - // member registered under those same constants — the join is direct - // equality, no mapping table. A language without a registered spec - // (python, go, …) skips taint entirely: no work, no warn spam (KTD8). + // registry is keyed by SupportedLanguages enum values, and + // ScopeResolver.language is registered under those same constants - + // the join is direct equality, with no mapping table. A language without a + // registered spec (go, ruby, ...) skips taint entirely: no work, no warn spam + // (KTD8). registerBuiltinTaintModels(); const taintSpec = getSourceSinkConfig(provider.language); // Taint-side solver fact cap: the SAME derivation emitFileReachingDefs diff --git a/gitnexus/src/core/ingestion/taint/python-model.ts b/gitnexus/src/core/ingestion/taint/python-model.ts new file mode 100644 index 000000000..443fb5b7b --- /dev/null +++ b/gitnexus/src/core/ingestion/taint/python-model.ts @@ -0,0 +1,46 @@ +/** + * Built-in Python taint model (#2204 first slice). + * + * Keep the model intentionally conservative: import-aware sinks for standard + * library command execution, receiver-conventional database execution calls, + * and Flask/FastAPI-style request-object member reads. No sanitizers are + * registered yet because a false sanitizer kill can hide a real finding. + */ + +import type { SourceSinkSanitizerSpec } from './source-sink-config.js'; + +export const PYTHON_TAINT_MODEL: SourceSinkSanitizerSpec = { + sources: [ + { + kind: 'remote-input', + objects: ['request', 'req'], + properties: [ + 'args', + 'form', + 'json', + 'data', + 'headers', + 'cookies', + 'path_params', + 'query_params', + ], + }, + ], + sinks: [ + { name: 'system', kind: 'command-injection', args: [0], module: 'os' }, + { name: 'popen', kind: 'command-injection', args: [0], module: 'os' }, + { name: 'call', kind: 'command-injection', args: [0], module: 'subprocess' }, + { name: 'run', kind: 'command-injection', args: [0], module: 'subprocess' }, + { name: 'Popen', kind: 'command-injection', args: [0], module: 'subprocess' }, + { name: 'check_call', kind: 'command-injection', args: [0], module: 'subprocess' }, + { name: 'check_output', kind: 'command-injection', args: [0], module: 'subprocess' }, + { name: 'eval', kind: 'code-injection', args: [0], global: true }, + { name: 'exec', kind: 'code-injection', args: [0], global: true }, + { name: 'open', kind: 'path-traversal', args: [0], global: true }, + { name: 'query', kind: 'sql-injection', args: [0], anyReceiver: true }, + { name: 'execute', kind: 'sql-injection', args: [0], anyReceiver: true }, + { name: 'executemany', kind: 'sql-injection', args: [0], anyReceiver: true }, + { name: 'executescript', kind: 'sql-injection', args: [0], anyReceiver: true }, + ], + sanitizers: [], +}; diff --git a/gitnexus/src/core/ingestion/taint/typescript-model.ts b/gitnexus/src/core/ingestion/taint/typescript-model.ts index 9e08f5ce7..ebe7aa9c9 100644 --- a/gitnexus/src/core/ingestion/taint/typescript-model.ts +++ b/gitnexus/src/core/ingestion/taint/typescript-model.ts @@ -18,6 +18,7 @@ import { createHash } from 'node:crypto'; import { SupportedLanguages } from 'gitnexus-shared'; import type { SourceSinkSanitizerSpec } from './source-sink-config.js'; +import { PYTHON_TAINT_MODEL } from './python-model.js'; import { registerSourceSinkConfig } from './source-sink-registry.js'; /** @@ -78,7 +79,11 @@ export const TS_JS_TAINT_MODEL: SourceSinkSanitizerSpec = { * array order is semantic (entry identity) and intentionally preserved. */ export function computeTaintModelVersion(spec: SourceSinkSanitizerSpec): string { - return createHash('sha256').update(canonicalJson(spec)).digest('hex').slice(0, 12); + return computeModelDigest(spec); +} + +function computeModelDigest(value: unknown): string { + return createHash('sha256').update(canonicalJson(value)).digest('hex').slice(0, 12); } function canonicalJson(value: unknown): string { @@ -93,16 +98,26 @@ function canonicalJson(value: unknown): string { return JSON.stringify(value); } -/** Version stamp of the built-in TS/JS model (joins the RepoMeta pdg key in U5). */ -export const taintModelVersion: string = computeTaintModelVersion(TS_JS_TAINT_MODEL); +export const BUILTIN_TAINT_MODELS = { + [SupportedLanguages.JavaScript]: TS_JS_TAINT_MODEL, + [SupportedLanguages.Python]: PYTHON_TAINT_MODEL, + [SupportedLanguages.TypeScript]: TS_JS_TAINT_MODEL, +} as const satisfies Record; /** - * Register the built-in model for TypeScript and JavaScript. Explicit init - * seam for the U4 emit path (call before the pdg window consumes the - * registry); idempotent. Vue and other TS-adjacent language ids are - * deliberately NOT registered — the M3 scope is TS/JS only. + * Version stamp of every built-in model (joins the RepoMeta pdg key in U5). + * Adding a language model must invalidate existing persisted taint findings. + */ +export const taintModelVersion: string = computeModelDigest(BUILTIN_TAINT_MODELS); + +/** + * Register the built-in models for TypeScript, JavaScript, and Python. + * Explicit init seam for the U4 emit path (call before the pdg window + * consumes the registry); idempotent. Other language ids remain unregistered + * until they have a dedicated model. */ export function registerBuiltinTaintModels(): void { registerSourceSinkConfig(SupportedLanguages.TypeScript, TS_JS_TAINT_MODEL); registerSourceSinkConfig(SupportedLanguages.JavaScript, TS_JS_TAINT_MODEL); + registerSourceSinkConfig(SupportedLanguages.Python, PYTHON_TAINT_MODEL); } diff --git a/gitnexus/test/unit/cfg/harvest.test.ts b/gitnexus/test/unit/cfg/harvest.test.ts index 6d42ea206..79bfcbbb7 100644 --- a/gitnexus/test/unit/cfg/harvest.test.ts +++ b/gitnexus/test/unit/cfg/harvest.test.ts @@ -878,13 +878,15 @@ describe('Python call-site harvest', () => { expect(sites[execIdx].args).toEqual([[[x, escapeIdx]]]); }); - it('keyword argument f(k=v) → only the value v is an occurrence (key is not a use)', () => { + it('keyword argument f(k=v) → value remains a use without minting a positional slot', () => { const cfg = py.cfgOf(`def f(v):\n foo(k=v)\n`); const s = siteFact(cfg, 2).sites![0]; expect(s.callee).toBe('foo'); - // `k` mints no binding; `v` occurs at position 0. + // `k` mints no binding; `v` remains a statement use, but keyword names are + // unavailable in SiteRecord so mapping it to positional slot 0 is unsound. expect(bindingIdxs(cfg, 'k')).toHaveLength(0); - expect(s.args).toEqual([[bindingIdx(cfg, 'v')]]); + expect(usesOf(cfg)).toContain(bindingIdx(cfg, 'v')); + expect(s.args).toBeUndefined(); }); it('def/use facts stay intact alongside the new sites (regression guard)', () => { diff --git a/gitnexus/test/unit/cfg/python-visitor.test.ts b/gitnexus/test/unit/cfg/python-visitor.test.ts index 76411af39..fde196571 100644 --- a/gitnexus/test/unit/cfg/python-visitor.test.ts +++ b/gitnexus/test/unit/cfg/python-visitor.test.ts @@ -1,7 +1,7 @@ import { describe, it, expect } from 'vitest'; import { createRequire } from 'node:module'; import { createPythonCfgVisitor } from '../../../src/core/ingestion/cfg/visitors/python.js'; -import type { FunctionCfg } from '../../../src/core/ingestion/cfg/types.js'; +import type { FunctionCfg, SiteRecord } from '../../../src/core/ingestion/cfg/types.js'; import { makeCfgHarness, type CfgHarness, @@ -380,13 +380,29 @@ describe('Python CfgVisitor — production CDG probe (plan-required)', () => { }); }); -describe('Python CfgVisitor — taint call sites harvested (this unit)', () => { - it('call statements now carry harvested call sites (callee path recorded)', () => { - const cfg = py.cfgOf(`def f(cmd):\n exec(cmd)\n x = escape(cmd)\n use(x)\n`); - const callees = cfg.blocks - .flatMap((b) => b.statements ?? []) - .flatMap((s) => s.sites ?? []) - .map((site) => site.callee); - expect(callees).toEqual(expect.arrayContaining(['exec', 'escape', 'use'])); +describe('Python CfgVisitor — taint-site substrate', () => { + it('harvests call, member-read, argument, receiver, and result-def sites', () => { + const cfg = py.cfgOf( + `def f(request, db):\n db.query(request.args)\n value = sanitize(request.form)\n`, + ); + const request = bindingIdx(cfg, 'request'); + const db = bindingIdx(cfg, 'db'); + const value = bindingIdx(cfg, 'value'); + const sites: SiteRecord[] = cfg.blocks.flatMap((b) => + (b.statements ?? []).flatMap((s) => [...(s.sites ?? [])]), + ); + + const query = sites.find((s) => s.kind === 'call' && s.callee === 'db.query'); + expect(query?.receiver).toBe(db); + expect(query?.args?.[0]).toContain(request); + expect(query?.at).toEqual([2, 4]); + + expect( + sites.some((s) => s.kind === 'member-read' && s.object === request && s.property === 'args'), + ).toBe(true); + + const sanitize = sites.find((s) => s.kind === 'call' && s.callee === 'sanitize'); + expect(sanitize?.resultDefs).toEqual([value]); + expect(sanitize?.at).toEqual([3, 12]); }); }); diff --git a/gitnexus/test/unit/taint/model-match.test.ts b/gitnexus/test/unit/taint/model-match.test.ts index 374961d27..32c7cf49a 100644 --- a/gitnexus/test/unit/taint/model-match.test.ts +++ b/gitnexus/test/unit/taint/model-match.test.ts @@ -13,6 +13,7 @@ import { cfgOf, importsFor } from '../../helpers/ts-cfg-harness.js'; import { hasTaintSafeSites } from '../../../src/core/ingestion/taint/site-safety.js'; import type { SourceSinkSanitizerSpec } from '../../../src/core/ingestion/taint/source-sink-config.js'; import { + BUILTIN_TAINT_MODELS, TS_JS_TAINT_MODEL, computeTaintModelVersion, registerBuiltinTaintModels, @@ -272,18 +273,19 @@ function f(x) { exec(escape(x)); }`); describe('registry + model identity', () => { beforeEach(() => clearSourceSinkRegistry()); - it('registerBuiltinTaintModels registers typescript AND javascript (idempotent); others stay undefined', () => { + it('registerBuiltinTaintModels registers TS, JS, and Python (idempotent); others stay undefined', () => { registerBuiltinTaintModels(); registerBuiltinTaintModels(); // idempotent — last-write-wins on the same ids - expect(registeredTaintLanguages().sort()).toEqual(['javascript', 'typescript']); + expect(registeredTaintLanguages().sort()).toEqual(['javascript', 'python', 'typescript']); expect(getSourceSinkConfig('typescript')).toBe(TS_JS_TAINT_MODEL); expect(getSourceSinkConfig('javascript')).toBe(TS_JS_TAINT_MODEL); - expect(getSourceSinkConfig('python')).toBeUndefined(); + expect(getSourceSinkConfig('python')).toBe(BUILTIN_TAINT_MODELS.python); + expect(getSourceSinkConfig('ruby')).toBeUndefined(); }); - it('taintModelVersion is the digest of the full built-in model', () => { - expect(taintModelVersion).toBe(computeTaintModelVersion(TS_JS_TAINT_MODEL)); + it('taintModelVersion covers the full built-in model registry', () => { expect(taintModelVersion).toMatch(/^[0-9a-f]{12}$/); + expect(taintModelVersion).not.toBe(computeTaintModelVersion(TS_JS_TAINT_MODEL)); }); it('adding an entry changes the version', () => { @@ -291,7 +293,7 @@ describe('registry + model identity', () => { ...TS_JS_TAINT_MODEL, sinks: [...TS_JS_TAINT_MODEL.sinks, { name: 'load', kind: 'code-injection', module: 'vm' }], }; - expect(computeTaintModelVersion(added)).not.toBe(taintModelVersion); + expect(computeTaintModelVersion(added)).not.toBe(computeTaintModelVersion(TS_JS_TAINT_MODEL)); }); it('changing only a kind label changes the version', () => { @@ -301,7 +303,9 @@ describe('registry + model identity', () => { s.name === 'exec' ? { ...s, kind: 'xss' as const } : s, ), }; - expect(computeTaintModelVersion(relabeled)).not.toBe(taintModelVersion); + expect(computeTaintModelVersion(relabeled)).not.toBe( + computeTaintModelVersion(TS_JS_TAINT_MODEL), + ); }); it('the version is content-derived: key order does not matter, entry order does', () => { @@ -310,6 +314,6 @@ describe('registry + model identity', () => { sinks: TS_JS_TAINT_MODEL.sinks, sources: TS_JS_TAINT_MODEL.sources, }; - expect(computeTaintModelVersion(reordered)).toBe(taintModelVersion); + expect(computeTaintModelVersion(reordered)).toBe(computeTaintModelVersion(TS_JS_TAINT_MODEL)); }); }); diff --git a/gitnexus/test/unit/taint/python-model-match.test.ts b/gitnexus/test/unit/taint/python-model-match.test.ts new file mode 100644 index 000000000..478d6d16d --- /dev/null +++ b/gitnexus/test/unit/taint/python-model-match.test.ts @@ -0,0 +1,162 @@ +/** + * Python taint model (#2204) over real Python CFG and import capture output. + */ + +import Python from 'tree-sitter-python'; +import type { ParsedImport } from 'gitnexus-shared'; +import { describe, expect, it } from 'vitest'; +import { createPythonCfgVisitor } from '../../../src/core/ingestion/cfg/visitors/python.js'; +import { emitPythonScopeCaptures } from '../../../src/core/ingestion/languages/python/captures.js'; +import { interpretPythonImport } from '../../../src/core/ingestion/languages/python/interpret.js'; +import { PYTHON_TAINT_MODEL } from '../../../src/core/ingestion/taint/python-model.js'; +import { hasTaintSafeSites } from '../../../src/core/ingestion/taint/site-safety.js'; +import { + buildTaintImportIndex, + matchFunctionSites, + type FunctionSiteMatches, + type MatchedSinkCall, + type MatchedSourceRead, +} from '../../../src/core/ingestion/taint/match.js'; +import { makeCfgHarness } from '../../helpers/cfg-harness.js'; + +const harness = makeCfgHarness(Python, createPythonCfgVisitor(), 'fixture.py'); + +function importsFor(src: string): ParsedImport[] { + return emitPythonScopeCaptures(src, 'fixture.py') + .filter((m) => m['@import.statement'] !== undefined) + .map((m) => interpretPythonImport(m)) + .filter((p): p is ParsedImport => p !== null); +} + +function matchesOf(code: string, fnIndex = 0): FunctionSiteMatches { + const cfg = harness.cfgOf(code, fnIndex); + expect(hasTaintSafeSites(cfg)).toBe(true); + return matchFunctionSites(cfg, PYTHON_TAINT_MODEL, buildTaintImportIndex(importsFor(code))); +} + +const allSinks = (m: FunctionSiteMatches): MatchedSinkCall[] => + m.statements.flatMap((s) => [...s.sinks]); +const allSources = (m: FunctionSiteMatches): MatchedSourceRead[] => + m.statements.flatMap((s) => [...s.sources]); + +describe('Python taint model (#2204)', () => { + it('matches Flask/FastAPI-style request member reads as remote-input sources', () => { + const m = matchesOf(` +def f(request, req, other): + a = request.args + b = req.json + c = other.args +`); + expect(allSources(m).map((s) => s.entry.kind)).toEqual(['remote-input', 'remote-input']); + expect(m.hasSource).toBe(true); + }); + + it('resolves named stdlib imports for command sinks', () => { + const m = matchesOf(` +from os import system + +def f(request): + system(request.args) +`); + const sinks = allSinks(m); + expect(sinks).toHaveLength(1); + expect(sinks[0].entry.kind).toBe('command-injection'); + expect(sinks[0].entry.name).toBe('system'); + expect(allSources(m)).toHaveLength(1); + }); + + it('resolves namespace stdlib imports for subprocess sinks', () => { + const m = matchesOf(` +import subprocess as sp + +def f(request): + sp.run(request.args) +`); + expect(allSinks(m).map((s) => s.entry.name)).toEqual(['run']); + }); + + it('does not guess positional sink slots for keyword arguments', () => { + const m = matchesOf(` +import subprocess as sp + +def f(request): + sp.run(shell=request.args, args="safe") +`); + expect(allSinks(m)).toHaveLength(0); + expect(allSources(m)).toHaveLength(1); + }); + + it('matches global code and path sinks at argument position zero', () => { + const m = matchesOf(` +def f(request): + eval(request.args) + exec(request.form) + open(request.path_params) +`); + const sinks = allSinks(m); + expect(sinks.map((s) => s.entry.kind)).toEqual([ + 'code-injection', + 'code-injection', + 'path-traversal', + ]); + expect(sinks.map((s) => [...s.argPositions])).toEqual([[0], [0], [0]]); + expect(allSources(m)).toHaveLength(3); + }); + + it('matches conventional database calls at argument position zero', () => { + const m = matchesOf(` +def f(request, cursor, db): + cursor.execute(request.args) + db.query(request.form) + cursor.executemany(request.json) + cursor.executescript(request.data) +`); + const sinks = allSinks(m); + expect(sinks.map((s) => s.entry.name)).toEqual([ + 'execute', + 'query', + 'executemany', + 'executescript', + ]); + expect(sinks.map((s) => [...s.argPositions])).toEqual([[0], [0], [0], [0]]); + expect(allSources(m)).toHaveLength(4); + }); + + it('does not match a locally shadowed stdlib sink name', () => { + const m = matchesOf(` +from os import system + +def f(request): + def system(value): + return value + system(request.args) +`); + expect(allSinks(m)).toHaveLength(0); + expect(allSources(m)).toHaveLength(1); + }); + + it('does not let methods in a nested class shadow an enclosing import', () => { + const m = matchesOf(` +from os import system + +def f(request): + class Helpers: + def system(self, value): + return value + system(request.args) +`); + expect(allSinks(m)).toHaveLength(1); + expect(allSources(m)).toHaveLength(1); + }); + + it('treats a nested class declaration as a local sink-name binding', () => { + const m = matchesOf(` +def f(request): + class open: + pass + open(request.args) +`); + expect(allSinks(m)).toHaveLength(0); + expect(allSources(m)).toHaveLength(1); + }); +});