mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
* feat(taint): harvest occurrence-tagged call/member sites on StatementFacts (#2083 U1) Worker-side site harvest in TsHarvester: call/new/member-read records with dotted callee paths, receiver slots, per-argument occurrence tagging with nested-site links, per-declarator resultDefs, spread/template/require-literal markers. hasTaintSafeSites validation seam. The pdg parse-cache chunk-key namespace is versioned (pdg:1 -> pdg:2) instead of a global SCHEMA_BUMP so flag-off users keep warm caches; bench fingerprints re-baselined for the three call-bearing scenarios (straight-line/dense-bindings byte-unchanged). * feat(taint): built-in TS/JS source/sink/sanitizer model + site matcher (#2083 U2) Typed spec (kind taxonomy; sanitizers carry neutralizes-kinds), the canonical Express/Node model, and matchFunctionSites: ESM alias/namespace + require- literal callee resolution, bare-name fallback restricted to true globals, sanitizers module-or-global only (never user-shadowable by name), spread/ template arg-position rules, deterministic taintModelVersion. * feat(taint): pure intra-procedural taint propagation engine (#2083 U3) Two-rule model (statement-local + du-fact worklist) with per-taint neutralized-kind exclusion sets: sanitizers exclude only the sink kinds they neutralize (escape(req.body) suppresses res.send but still fires db.query; exec(path.basename(t)) fires), intersection-over-paths so a bypass occurrence keeps the taint live, kill locality on resultDefs, propagate-through args+receiver with viaCall hops, one path per finding, deterministic caps, coverage-gap statuses. Test-first: 38 scenarios on real harvested CFGs. * feat(taint): thread taint caps + model version through pdg config/meta (#2083 U5) resolvePdgConfig gains maxTaintFindingsPerFunction (200), maxTaintHops (32), and the taintModelVersion digest; RepoMeta.pdg + RunScopeResolutionInput surfaces added. The key-union comparator trips full writeback on M2->M3 upgrade and on model-version change without --force (mode-flip tested). No CLI flags or rc keys (programmatic parity with the other caps). * feat(taint): in-phase taint emit with sparse TAINTED/SANITIZES edges (#2083 U4) run.ts pdg window: match-first fast path (solver only when a function has both a matched source and sink) -> computeReachingDefs with the shared RD fact derivation -> computeTaintFlows -> per-finding TAINTED (versioned hop-encoded reason via the shared path codec, statement-level occurrence identity) + per-kill SANITIZES, dedup-before-budget, truncate-and-warn. All emit counters surfaced (aggregate warn for gaps/drops, debug for volume); PROF gains taint=. Flag-off golden untouched. * feat(mcp): explain tool for persisted taint findings (#2083 U6) Anchorless calls enumerate the sparse TAINTED table (bounded, deterministic, limit-clamped); anchored calls (file or symbol via resolveSymbolCandidates) return full decoded hop detail. sinkKind rides a version-1 codec header (1;<kind>|hops — no other persisted channel exists; U4/U6 ship together). RepoMeta.pdg probe yields a no-taint-layer note instead of an error. TAINTED/SANITIZES pinned OUT of VALID_RELATION_TYPES (KTD9a negative- membership tests); generators + canonical skill docs + mirrors updated. * test(taint): acceptance fixture battery, snapshots, and bench gates (#2083 U7) pdg-repo taint-cases fixtures complete the six plan shapes; committed findings/kills snapshot via a shared pure-path harness that also feeds the AE2 exact-equality assertion (stored TAINTED == pure-path findings, the no-explosion gate). New taint-dense bench scenario with four --check gates: per-function findings pinned AT the cap, absolute reason-byte + site-bytes disk ceilings (the load-bearing R10 gate), zero-match pass < 0.5x match- dense, N-linearity. Pre-existing scenario baselines untouched. * refactor(taint): share one pointKey helper across propagate + emit (#2083 review) Extract pointKey(ProgramPoint) to cfg/reaching-defs.ts (colon-separated, matching the codebase block:stmt id convention) and import it in both propagate.ts and emit.ts, replacing the two divergent locals (':' vs '.'). Edge-id material now uses the colon form; ids are in-memory only and no test asserts the pointKey segment shape. * fix(taint): discriminate taint state by source occurrence (#2083 review) Two distinct sources flowing into one variable at one def point no longer collapse to a single TAINTED edge: the taint-state key gains a root source-occurrence discriminator ({point, siteIndex} — the same fields recordFinding's identity uses, excluding kind). Def->use fact lookup keys on the source-independent (binding, def-point) portion. Same-source multi-path flows still share one state so their exclusion sets intersect (the raw arm soundly wins); termination holds (finite keys, monotone shrink, no cross-source ping-pong). Restores the KTD6 identity contract. * fix(mcp): route dotted symbol names in explain to symbol resolution (#2083 review) The fileish classifier matched any dotted name (UserController.create) as a file via its extension-like suffix, so symbol resolution never ran and the tool returned a silent empty file-anchored result. Tighten the classifier to require a path separator or a real source extension (derived from the resolver's EXTENSIONS list, multi-language), so dotted/bare names route to resolveSymbolCandidates (found / ambiguous / not-found). * fix(mcp): gate explain no-taint-layer note on taintModelVersion (#2083 review) An M1/M2-era --pdg index has meta.pdg defined (BasicBlock/REACHING_DEF recorded) but no taintModelVersion and zero TAINTED rows. The probe keyed on generic meta.pdg presence, so explain returned the generic empty note instead of the actionable 'no taint layer — run analyze' hint. Gate on meta.pdg?.taintModelVersion (the field M3 stamps) so an M2-era index gets the layer hint; a taint-stamped index with no findings still gets the generic note. * fix(taint): sequence-expression value flows only the final operand (#2083 review) A comma expression in value position (exec((log(x), 'safe'))) default- descended, fanning every operand's occurrences into the enclosing sink argument — over-tainting exec's arg 0 with x. Add an explicit walkValue case that records earlier operands' uses with occurrence fan-out suppressed (new FactAccumulator.suppressOccurrences) and routes only the last operand through the value path. Sites-layer only; defs/uses/mayDefs byte-identical (cfg + reaching-defs snapshots unchanged). * perf(taint): FIFO head-cursor worklist + dedup before chainHops (#2083 review) Replace queue.shift() (O(N) dequeue) with a strict-FIFO head cursor plus order-preserving prefix reclamation; FIFO is load-bearing because chainHops reads the live taints map whose parent/source/viaCall are rewritten order-sensitively on monotone shrink, so hop determinism is dequeue-order contingent. Extract findingKey() and dedup-check before chainHops in the justify branch — already-recorded identities discard their hop chain (first write wins), so the ancestry walk was pure waste. The else kill branch is untouched. Findings + hops byte-identical (snapshot unchanged). * perf(taint): O(1) member-read dedup via composite-key set (#2083 review) addMemberRead rescanned the whole per-statement sites array per call to dedup by (object, property, parent) — O(n^2) on member-read-dense statements. Track a composite-key Set alongside sites for O(1) dedup. (The require-literal join is already O(sites) with a no-op body on non-require sites, so no early-exit is needed there.) Behavior identical: harvest + model-match + taint snapshots unchanged. * refactor(taint): drop test-only export; source taint caps via emit.ts (#2083 review) Remove the sanitizerNeutralizes export (its only consumers were two test assertions — inlined to entry.neutralizes membership). Re-export the DEFAULT_PDG_MAX_TAINT_* caps from emit.ts and point run.ts at emit.ts, so the pipeline's taint dependency surface is the single orchestration module rather than reaching into propagate.ts. * test(taint): extract the shared TS CFG/taint test harness (#2083 review) The parse/collectFunctions/cfgOf/cfgsOf/importsFor harness was copied byte-for-byte across four suites (harvest, model-match, propagate, taint-emit). Promote it to test/helpers/ts-cfg-harness.ts and import it. site-safety/reaching-defs carry a structurally different inlined builder and are left as-is. Pure extraction, no assertion changes. * test(mcp): harden explain limit-rejection battery (#2083 review) Add NaN, Infinity, -Infinity, and a numeric string to the out-of-bounds limit cases — a regression fence over the interpolated LIMIT, confirming the Number.isInteger guard rejects every non-integer/non-finite/string input before it reaches the query.
305 lines
12 KiB
TypeScript
305 lines
12 KiB
TypeScript
/**
|
|
* U4 (#2083 M3) — the shared taint-path reason codec (plan KTD6).
|
|
*
|
|
* The wire format must round-trip BYTE-EXACT through the CSV persistence
|
|
* layer (`escapeCSVField ∘ sanitizeUTF8`, csv-generator.ts) — that
|
|
* composition is exercised here verbatim, including the un-escape a DB load
|
|
* performs. Truncation (hop cap, byte cap, unencodable hop) must decode as
|
|
* "path incomplete", never as an error; malformed input must produce a typed
|
|
* failure, never a throw.
|
|
*/
|
|
|
|
import { describe, it, expect } from 'vitest';
|
|
import {
|
|
TAINT_PATH_CODEC_VERSION,
|
|
TAINT_REASON_MAX_BYTES,
|
|
TAINT_PATH_TRUNCATION_MARKER,
|
|
encodeTaintPath,
|
|
decodeTaintPath,
|
|
type TaintPathHopInput,
|
|
} from '../../../src/core/ingestion/taint/path-codec.js';
|
|
import { escapeCSVField, sanitizeUTF8 } from '../../../src/core/lbug/csv-generator.js';
|
|
|
|
/** Inverse of escapeCSVField — what a CSV/DB load applies to the stored cell. */
|
|
function unescapeCSVField(cell: string): string {
|
|
expect(cell.startsWith('"') && cell.endsWith('"')).toBe(true);
|
|
return cell.slice(1, -1).replace(/""/g, '"');
|
|
}
|
|
|
|
const roundTrip = (hops: readonly TaintPathHopInput[]) => {
|
|
const { reason } = encodeTaintPath(hops);
|
|
const decoded = decodeTaintPath(reason);
|
|
if (!decoded.ok) throw new Error(`decode failed: ${decoded.error}`);
|
|
return { reason, decoded };
|
|
};
|
|
|
|
describe('encodeTaintPath / decodeTaintPath round trip', () => {
|
|
it('round-trips an ordered multi-hop path with variables, lines, and viaCall', () => {
|
|
const hops: TaintPathHopInput[] = [
|
|
{ name: 'req', line: 3 },
|
|
{ name: 'cmd', line: 4, viaCall: true },
|
|
{ name: 'cmd', line: 7 },
|
|
];
|
|
const { reason, decoded } = roundTrip(hops);
|
|
expect(reason).toBe('1|req:3|cmd:4:c|cmd:7');
|
|
expect(decoded.version).toBe(TAINT_PATH_CODEC_VERSION);
|
|
expect(decoded.truncated).toBe(false);
|
|
expect(decoded.hops).toEqual([
|
|
{ variable: 'req', line: 3, viaCall: false },
|
|
{ variable: 'cmd', line: 4, viaCall: true },
|
|
{ variable: 'cmd', line: 7, viaCall: false },
|
|
]);
|
|
});
|
|
|
|
it('round-trips the empty path (version prefix only)', () => {
|
|
const { reason, decoded } = roundTrip([]);
|
|
expect(reason).toBe('1');
|
|
expect(decoded.hops).toEqual([]);
|
|
expect(decoded.truncated).toBe(false);
|
|
});
|
|
|
|
it('round-trips identifier-charset names: $, _, #, digits, case', () => {
|
|
const names = ['$jq', '_private', '#3', 'CONST_99', 'aB$_#z', 'x'];
|
|
const hops = names.map((name, i) => ({ name, line: i + 1, viaCall: i % 2 === 0 }));
|
|
const { decoded } = roundTrip(hops);
|
|
expect(decoded.hops.map((h) => h.variable)).toEqual(names);
|
|
});
|
|
|
|
it('fuzz-ish sweep: random identifier-charset names of varied length survive', () => {
|
|
const CHARSET = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_$#';
|
|
// Deterministic LCG so a failure is reproducible.
|
|
let seed = 0xc0ffee;
|
|
const next = (): number => {
|
|
seed = (seed * 1103515245 + 12345) & 0x7fffffff;
|
|
return seed;
|
|
};
|
|
for (let trial = 0; trial < 200; trial++) {
|
|
const hops: TaintPathHopInput[] = [];
|
|
const count = (next() % 8) + 1;
|
|
for (let i = 0; i < count; i++) {
|
|
const len = (next() % 24) + 1;
|
|
let name = '';
|
|
for (let j = 0; j < len; j++) name += CHARSET[next() % CHARSET.length];
|
|
hops.push({ name, line: next() % 100000, viaCall: next() % 2 === 0 });
|
|
}
|
|
const { reason, decoded } = roundTrip(hops);
|
|
expect(decoded.truncated).toBe(false);
|
|
expect(decoded.hops).toEqual(
|
|
hops.map((h) => ({ variable: h.name, line: h.line, viaCall: h.viaCall === true })),
|
|
);
|
|
// The whole wire string is printable ASCII (the CSV-survival invariant).
|
|
expect(/^[\x20-\x7e]+$/.test(reason)).toBe(true);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('kind header (;<kind> — U6, the only persisted channel for sinkKind)', () => {
|
|
it('round-trips a kind header with hops', () => {
|
|
const { reason } = encodeTaintPath(
|
|
[
|
|
{ name: 'req', line: 3 },
|
|
{ name: 'cmd', line: 4 },
|
|
],
|
|
{ kind: 'command-injection' },
|
|
);
|
|
expect(reason).toBe('1;command-injection|req:3|cmd:4');
|
|
const decoded = decodeTaintPath(reason);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) {
|
|
expect(decoded.kind).toBe('command-injection');
|
|
expect(decoded.hops.map((h) => h.variable)).toEqual(['req', 'cmd']);
|
|
}
|
|
});
|
|
|
|
it('round-trips a kind header on the hop-less path', () => {
|
|
const { reason } = encodeTaintPath([], { kind: 'xss' });
|
|
expect(reason).toBe('1;xss');
|
|
const decoded = decodeTaintPath(reason);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) {
|
|
expect(decoded.kind).toBe('xss');
|
|
expect(decoded.hops).toEqual([]);
|
|
}
|
|
});
|
|
|
|
it('kind + truncation marker coexist; the header is never sacrificed to the byte cap', () => {
|
|
const { reason, truncated } = encodeTaintPath([{ name: 'longVariableName', line: 12345 }], {
|
|
kind: 'sql-injection',
|
|
maxBytes: 8, // far below the header size — floor lifts it, hops drop
|
|
});
|
|
expect(truncated).toBe(true);
|
|
expect(reason).toBe(`1;sql-injection|${TAINT_PATH_TRUNCATION_MARKER}`);
|
|
const decoded = decodeTaintPath(reason);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) {
|
|
expect(decoded.kind).toBe('sql-injection');
|
|
expect(decoded.truncated).toBe(true);
|
|
expect(decoded.hops).toEqual([]);
|
|
}
|
|
});
|
|
|
|
it('a kind outside [a-z0-9-] is dropped (header omitted), never corrupted into the wire', () => {
|
|
for (const bad of ['Command-Injection', 'a;b', 'k|x', 'café', '']) {
|
|
const { reason } = encodeTaintPath([{ name: 'x', line: 1 }], { kind: bad });
|
|
expect(reason).toBe('1|x:1');
|
|
const decoded = decodeTaintPath(reason);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) expect(decoded.kind).toBeUndefined();
|
|
}
|
|
});
|
|
|
|
it('a header-less version-1 string still decodes (kind undefined)', () => {
|
|
const decoded = decodeTaintPath('1|a:1');
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) expect(decoded.kind).toBeUndefined();
|
|
});
|
|
|
|
it('the kind header survives the CSV persistence transform byte-exact', () => {
|
|
const { reason } = encodeTaintPath([{ name: 'req', line: 2 }], { kind: 'path-traversal' });
|
|
const loaded = unescapeCSVField(escapeCSVField(sanitizeUTF8(reason)));
|
|
expect(loaded).toBe(reason);
|
|
const decoded = decodeTaintPath(loaded);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) expect(decoded.kind).toBe('path-traversal');
|
|
});
|
|
|
|
it('fails typed on a malformed kind header', () => {
|
|
for (const bad of ['1;|a:1', '1;BAD|a:1', '1;', '1;a;b|x:1']) {
|
|
const decoded = decodeTaintPath(bad);
|
|
expect(decoded.ok, bad).toBe(false);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('CSV persistence composition (escapeCSVField ∘ sanitizeUTF8)', () => {
|
|
it('the encoding survives the exact persistence transform byte-exact', () => {
|
|
const hops: TaintPathHopInput[] = [
|
|
{ name: 'req', line: 12 },
|
|
{ name: '$tmp_2', line: 13, viaCall: true },
|
|
{ name: '#7', line: 99 },
|
|
];
|
|
const { reason } = encodeTaintPath(hops);
|
|
// csv-generator applies sanitizeUTF8 INSIDE escapeCSVField; compose both
|
|
// explicitly anyway so the test pins each layer.
|
|
const stored = escapeCSVField(sanitizeUTF8(reason));
|
|
const loaded = unescapeCSVField(stored);
|
|
expect(loaded).toBe(reason); // byte-exact
|
|
const decoded = decodeTaintPath(loaded);
|
|
expect(decoded.ok).toBe(true);
|
|
});
|
|
|
|
it('a truncated encoding also survives the persistence transform', () => {
|
|
const { reason, truncated } = encodeTaintPath([{ name: 'x', line: 1 }], { truncated: true });
|
|
expect(truncated).toBe(true);
|
|
const loaded = unescapeCSVField(escapeCSVField(sanitizeUTF8(reason)));
|
|
expect(loaded).toBe(reason);
|
|
});
|
|
});
|
|
|
|
describe('truncation', () => {
|
|
it('caller-flagged truncation (hop cap upstream) emits the marker; decode reports path-incomplete', () => {
|
|
const { reason, truncated } = encodeTaintPath([{ name: 'a', line: 1 }], { truncated: true });
|
|
expect(truncated).toBe(true);
|
|
expect(reason).toBe(`1|a:1|${TAINT_PATH_TRUNCATION_MARKER}`);
|
|
const decoded = decodeTaintPath(reason);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) {
|
|
expect(decoded.truncated).toBe(true); // informational — NOT an error
|
|
expect(decoded.hops).toEqual([{ variable: 'a', line: 1, viaCall: false }]);
|
|
}
|
|
});
|
|
|
|
it('byte-cap overflow drops TRAILING hops (source-side prefix kept) and sets the marker', () => {
|
|
const hops: TaintPathHopInput[] = [];
|
|
for (let i = 0; i < 1000; i++) hops.push({ name: `variable_${i}`, line: i });
|
|
const { reason, truncated } = encodeTaintPath(hops);
|
|
expect(truncated).toBe(true);
|
|
expect(reason.length).toBeLessThanOrEqual(TAINT_REASON_MAX_BYTES);
|
|
const decoded = decodeTaintPath(reason);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) {
|
|
expect(decoded.truncated).toBe(true);
|
|
expect(decoded.hops.length).toBeGreaterThan(0);
|
|
expect(decoded.hops.length).toBeLessThan(hops.length);
|
|
// Prefix discipline: hop k decodes hop k of the input, in order.
|
|
decoded.hops.forEach((h, i) => {
|
|
expect(h.variable).toBe(`variable_${i}`);
|
|
expect(h.line).toBe(i);
|
|
});
|
|
}
|
|
});
|
|
|
|
it('a tiny maxBytes still yields a well-formed (possibly hop-less) truncated path', () => {
|
|
const { reason, truncated } = encodeTaintPath([{ name: 'longVariableName', line: 123 }], {
|
|
maxBytes: 8,
|
|
});
|
|
expect(truncated).toBe(true);
|
|
expect(reason).toBe(`1|${TAINT_PATH_TRUNCATION_MARKER}`);
|
|
const decoded = decodeTaintPath(reason);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) {
|
|
expect(decoded.hops).toEqual([]);
|
|
expect(decoded.truncated).toBe(true);
|
|
}
|
|
});
|
|
|
|
it('an unencodable hop name stops encoding at that hop and marks truncation (defend, never corrupt)', () => {
|
|
const cases = ['a|b', 'a:b', 'café', 'name~x', '', 'a b'];
|
|
for (const bad of cases) {
|
|
const { reason, truncated } = encodeTaintPath([
|
|
{ name: 'ok1', line: 1 },
|
|
{ name: bad, line: 2 },
|
|
{ name: 'ok2', line: 3 }, // dropped too — prefix discipline
|
|
]);
|
|
expect(truncated).toBe(true);
|
|
const decoded = decodeTaintPath(reason);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) {
|
|
expect(decoded.truncated).toBe(true);
|
|
expect(decoded.hops).toEqual([{ variable: 'ok1', line: 1, viaCall: false }]);
|
|
}
|
|
}
|
|
});
|
|
|
|
it('a non-integer or negative line is unencodable the same way', () => {
|
|
for (const line of [1.5, -1, NaN, Infinity]) {
|
|
const { truncated, reason } = encodeTaintPath([{ name: 'x', line }]);
|
|
expect(truncated).toBe(true);
|
|
const decoded = decodeTaintPath(reason);
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) expect(decoded.hops).toEqual([]);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('typed parse failures (never a throw)', () => {
|
|
const failing: Array<[string, unknown]> = [
|
|
['empty string', ''],
|
|
['non-string', 42],
|
|
['undefined', undefined],
|
|
['unknown version', '2|a:1'],
|
|
['missing separator after version', '1a:1'],
|
|
['hop with no line', '1|a'],
|
|
['hop with too many fields', '1|a:1:c:d'],
|
|
['non-numeric line', '1|a:x'],
|
|
['negative line', '1|a:-1'],
|
|
['invalid variable charset', '1|a b:1'],
|
|
['empty variable', '1|:1'],
|
|
['uppercase flag (reserved charset is lowercase)', '1|a:1:C'],
|
|
['marker not trailing', `1|${TAINT_PATH_TRUNCATION_MARKER}|a:1`],
|
|
['empty hop segment', '1|a:1||b:2'],
|
|
];
|
|
for (const [label, input] of failing) {
|
|
it(`fails typed on ${label}`, () => {
|
|
const decoded = decodeTaintPath(input);
|
|
expect(decoded.ok).toBe(false);
|
|
if (!decoded.ok) expect(decoded.error.length).toBeGreaterThan(0);
|
|
});
|
|
}
|
|
|
|
it('accepts unknown RESERVED lowercase flag letters (forward compatibility)', () => {
|
|
const decoded = decodeTaintPath('1|a:1:cz');
|
|
expect(decoded.ok).toBe(true);
|
|
if (decoded.ok) expect(decoded.hops[0]).toEqual({ variable: 'a', line: 1, viaCall: true });
|
|
});
|
|
});
|