GitNexus/gitnexus/test/unit/taint/path-codec.test.ts
Gergő Magyar 14397dd4aa
feat(taint): intra-procedural taint analysis (#2083) (#2164)
* feat(taint): harvest occurrence-tagged call/member sites on StatementFacts (#2083 U1)

Worker-side site harvest in TsHarvester: call/new/member-read records with
dotted callee paths, receiver slots, per-argument occurrence tagging with
nested-site links, per-declarator resultDefs, spread/template/require-literal
markers. hasTaintSafeSites validation seam. The pdg parse-cache chunk-key
namespace is versioned (pdg:1 -> pdg:2) instead of a global SCHEMA_BUMP so
flag-off users keep warm caches; bench fingerprints re-baselined for the
three call-bearing scenarios (straight-line/dense-bindings byte-unchanged).

* feat(taint): built-in TS/JS source/sink/sanitizer model + site matcher (#2083 U2)

Typed spec (kind taxonomy; sanitizers carry neutralizes-kinds), the canonical
Express/Node model, and matchFunctionSites: ESM alias/namespace + require-
literal callee resolution, bare-name fallback restricted to true globals,
sanitizers module-or-global only (never user-shadowable by name), spread/
template arg-position rules, deterministic taintModelVersion.

* feat(taint): pure intra-procedural taint propagation engine (#2083 U3)

Two-rule model (statement-local + du-fact worklist) with per-taint
neutralized-kind exclusion sets: sanitizers exclude only the sink kinds
they neutralize (escape(req.body) suppresses res.send but still fires
db.query; exec(path.basename(t)) fires), intersection-over-paths so a
bypass occurrence keeps the taint live, kill locality on resultDefs,
propagate-through args+receiver with viaCall hops, one path per finding,
deterministic caps, coverage-gap statuses. Test-first: 38 scenarios on
real harvested CFGs.

* feat(taint): thread taint caps + model version through pdg config/meta (#2083 U5)

resolvePdgConfig gains maxTaintFindingsPerFunction (200), maxTaintHops (32),
and the taintModelVersion digest; RepoMeta.pdg + RunScopeResolutionInput
surfaces added. The key-union comparator trips full writeback on M2->M3
upgrade and on model-version change without --force (mode-flip tested).
No CLI flags or rc keys (programmatic parity with the other caps).

* feat(taint): in-phase taint emit with sparse TAINTED/SANITIZES edges (#2083 U4)

run.ts pdg window: match-first fast path (solver only when a function has
both a matched source and sink) -> computeReachingDefs with the shared RD
fact derivation -> computeTaintFlows -> per-finding TAINTED (versioned
hop-encoded reason via the shared path codec, statement-level occurrence
identity) + per-kill SANITIZES, dedup-before-budget, truncate-and-warn.
All emit counters surfaced (aggregate warn for gaps/drops, debug for
volume); PROF gains taint=. Flag-off golden untouched.

* feat(mcp): explain tool for persisted taint findings (#2083 U6)

Anchorless calls enumerate the sparse TAINTED table (bounded, deterministic,
limit-clamped); anchored calls (file or symbol via resolveSymbolCandidates)
return full decoded hop detail. sinkKind rides a version-1 codec header
(1;<kind>|hops — no other persisted channel exists; U4/U6 ship together).
RepoMeta.pdg probe yields a no-taint-layer note instead of an error.
TAINTED/SANITIZES pinned OUT of VALID_RELATION_TYPES (KTD9a negative-
membership tests); generators + canonical skill docs + mirrors updated.

* test(taint): acceptance fixture battery, snapshots, and bench gates (#2083 U7)

pdg-repo taint-cases fixtures complete the six plan shapes; committed
findings/kills snapshot via a shared pure-path harness that also feeds the
AE2 exact-equality assertion (stored TAINTED == pure-path findings, the
no-explosion gate). New taint-dense bench scenario with four --check gates:
per-function findings pinned AT the cap, absolute reason-byte + site-bytes
disk ceilings (the load-bearing R10 gate), zero-match pass < 0.5x match-
dense, N-linearity. Pre-existing scenario baselines untouched.

* refactor(taint): share one pointKey helper across propagate + emit (#2083 review)

Extract pointKey(ProgramPoint) to cfg/reaching-defs.ts (colon-separated,
matching the codebase block:stmt id convention) and import it in both
propagate.ts and emit.ts, replacing the two divergent locals (':' vs '.').
Edge-id material now uses the colon form; ids are in-memory only and no
test asserts the pointKey segment shape.

* fix(taint): discriminate taint state by source occurrence (#2083 review)

Two distinct sources flowing into one variable at one def point no longer
collapse to a single TAINTED edge: the taint-state key gains a root
source-occurrence discriminator ({point, siteIndex} — the same fields
recordFinding's identity uses, excluding kind). Def->use fact lookup keys
on the source-independent (binding, def-point) portion. Same-source
multi-path flows still share one state so their exclusion sets intersect
(the raw arm soundly wins); termination holds (finite keys, monotone
shrink, no cross-source ping-pong). Restores the KTD6 identity contract.

* fix(mcp): route dotted symbol names in explain to symbol resolution (#2083 review)

The fileish classifier matched any dotted name (UserController.create)
as a file via its extension-like suffix, so symbol resolution never ran
and the tool returned a silent empty file-anchored result. Tighten the
classifier to require a path separator or a real source extension (derived
from the resolver's EXTENSIONS list, multi-language), so dotted/bare names
route to resolveSymbolCandidates (found / ambiguous / not-found).

* fix(mcp): gate explain no-taint-layer note on taintModelVersion (#2083 review)

An M1/M2-era --pdg index has meta.pdg defined (BasicBlock/REACHING_DEF
recorded) but no taintModelVersion and zero TAINTED rows. The probe keyed
on generic meta.pdg presence, so explain returned the generic empty note
instead of the actionable 'no taint layer — run analyze' hint. Gate on
meta.pdg?.taintModelVersion (the field M3 stamps) so an M2-era index gets
the layer hint; a taint-stamped index with no findings still gets the
generic note.

* fix(taint): sequence-expression value flows only the final operand (#2083 review)

A comma expression in value position (exec((log(x), 'safe'))) default-
descended, fanning every operand's occurrences into the enclosing sink
argument — over-tainting exec's arg 0 with x. Add an explicit walkValue
case that records earlier operands' uses with occurrence fan-out suppressed
(new FactAccumulator.suppressOccurrences) and routes only the last operand
through the value path. Sites-layer only; defs/uses/mayDefs byte-identical
(cfg + reaching-defs snapshots unchanged).

* perf(taint): FIFO head-cursor worklist + dedup before chainHops (#2083 review)

Replace queue.shift() (O(N) dequeue) with a strict-FIFO head cursor plus
order-preserving prefix reclamation; FIFO is load-bearing because chainHops
reads the live taints map whose parent/source/viaCall are rewritten
order-sensitively on monotone shrink, so hop determinism is dequeue-order
contingent. Extract findingKey() and dedup-check before chainHops in the
justify branch — already-recorded identities discard their hop chain
(first write wins), so the ancestry walk was pure waste. The else kill
branch is untouched. Findings + hops byte-identical (snapshot unchanged).

* perf(taint): O(1) member-read dedup via composite-key set (#2083 review)

addMemberRead rescanned the whole per-statement sites array per call to
dedup by (object, property, parent) — O(n^2) on member-read-dense
statements. Track a composite-key Set alongside sites for O(1) dedup.
(The require-literal join is already O(sites) with a no-op body on
non-require sites, so no early-exit is needed there.) Behavior identical:
harvest + model-match + taint snapshots unchanged.

* refactor(taint): drop test-only export; source taint caps via emit.ts (#2083 review)

Remove the sanitizerNeutralizes export (its only consumers were two test
assertions — inlined to entry.neutralizes membership). Re-export the
DEFAULT_PDG_MAX_TAINT_* caps from emit.ts and point run.ts at emit.ts, so
the pipeline's taint dependency surface is the single orchestration module
rather than reaching into propagate.ts.

* test(taint): extract the shared TS CFG/taint test harness (#2083 review)

The parse/collectFunctions/cfgOf/cfgsOf/importsFor harness was copied
byte-for-byte across four suites (harvest, model-match, propagate,
taint-emit). Promote it to test/helpers/ts-cfg-harness.ts and import it.
site-safety/reaching-defs carry a structurally different inlined builder
and are left as-is. Pure extraction, no assertion changes.

* test(mcp): harden explain limit-rejection battery (#2083 review)

Add NaN, Infinity, -Infinity, and a numeric string to the out-of-bounds
limit cases — a regression fence over the interpolated LIMIT, confirming
the Number.isInteger guard rejects every non-integer/non-finite/string
input before it reaches the query.
2026-06-12 07:35:09 +01:00

305 lines
12 KiB
TypeScript

/**
* U4 (#2083 M3) — the shared taint-path reason codec (plan KTD6).
*
* The wire format must round-trip BYTE-EXACT through the CSV persistence
* layer (`escapeCSVField ∘ sanitizeUTF8`, csv-generator.ts) — that
* composition is exercised here verbatim, including the un-escape a DB load
* performs. Truncation (hop cap, byte cap, unencodable hop) must decode as
* "path incomplete", never as an error; malformed input must produce a typed
* failure, never a throw.
*/
import { describe, it, expect } from 'vitest';
import {
TAINT_PATH_CODEC_VERSION,
TAINT_REASON_MAX_BYTES,
TAINT_PATH_TRUNCATION_MARKER,
encodeTaintPath,
decodeTaintPath,
type TaintPathHopInput,
} from '../../../src/core/ingestion/taint/path-codec.js';
import { escapeCSVField, sanitizeUTF8 } from '../../../src/core/lbug/csv-generator.js';
/** Inverse of escapeCSVField — what a CSV/DB load applies to the stored cell. */
function unescapeCSVField(cell: string): string {
expect(cell.startsWith('"') && cell.endsWith('"')).toBe(true);
return cell.slice(1, -1).replace(/""/g, '"');
}
const roundTrip = (hops: readonly TaintPathHopInput[]) => {
const { reason } = encodeTaintPath(hops);
const decoded = decodeTaintPath(reason);
if (!decoded.ok) throw new Error(`decode failed: ${decoded.error}`);
return { reason, decoded };
};
describe('encodeTaintPath / decodeTaintPath round trip', () => {
it('round-trips an ordered multi-hop path with variables, lines, and viaCall', () => {
const hops: TaintPathHopInput[] = [
{ name: 'req', line: 3 },
{ name: 'cmd', line: 4, viaCall: true },
{ name: 'cmd', line: 7 },
];
const { reason, decoded } = roundTrip(hops);
expect(reason).toBe('1|req:3|cmd:4:c|cmd:7');
expect(decoded.version).toBe(TAINT_PATH_CODEC_VERSION);
expect(decoded.truncated).toBe(false);
expect(decoded.hops).toEqual([
{ variable: 'req', line: 3, viaCall: false },
{ variable: 'cmd', line: 4, viaCall: true },
{ variable: 'cmd', line: 7, viaCall: false },
]);
});
it('round-trips the empty path (version prefix only)', () => {
const { reason, decoded } = roundTrip([]);
expect(reason).toBe('1');
expect(decoded.hops).toEqual([]);
expect(decoded.truncated).toBe(false);
});
it('round-trips identifier-charset names: $, _, #, digits, case', () => {
const names = ['$jq', '_private', '#3', 'CONST_99', 'aB$_#z', 'x'];
const hops = names.map((name, i) => ({ name, line: i + 1, viaCall: i % 2 === 0 }));
const { decoded } = roundTrip(hops);
expect(decoded.hops.map((h) => h.variable)).toEqual(names);
});
it('fuzz-ish sweep: random identifier-charset names of varied length survive', () => {
const CHARSET = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_$#';
// Deterministic LCG so a failure is reproducible.
let seed = 0xc0ffee;
const next = (): number => {
seed = (seed * 1103515245 + 12345) & 0x7fffffff;
return seed;
};
for (let trial = 0; trial < 200; trial++) {
const hops: TaintPathHopInput[] = [];
const count = (next() % 8) + 1;
for (let i = 0; i < count; i++) {
const len = (next() % 24) + 1;
let name = '';
for (let j = 0; j < len; j++) name += CHARSET[next() % CHARSET.length];
hops.push({ name, line: next() % 100000, viaCall: next() % 2 === 0 });
}
const { reason, decoded } = roundTrip(hops);
expect(decoded.truncated).toBe(false);
expect(decoded.hops).toEqual(
hops.map((h) => ({ variable: h.name, line: h.line, viaCall: h.viaCall === true })),
);
// The whole wire string is printable ASCII (the CSV-survival invariant).
expect(/^[\x20-\x7e]+$/.test(reason)).toBe(true);
}
});
});
describe('kind header (;<kind> — U6, the only persisted channel for sinkKind)', () => {
it('round-trips a kind header with hops', () => {
const { reason } = encodeTaintPath(
[
{ name: 'req', line: 3 },
{ name: 'cmd', line: 4 },
],
{ kind: 'command-injection' },
);
expect(reason).toBe('1;command-injection|req:3|cmd:4');
const decoded = decodeTaintPath(reason);
expect(decoded.ok).toBe(true);
if (decoded.ok) {
expect(decoded.kind).toBe('command-injection');
expect(decoded.hops.map((h) => h.variable)).toEqual(['req', 'cmd']);
}
});
it('round-trips a kind header on the hop-less path', () => {
const { reason } = encodeTaintPath([], { kind: 'xss' });
expect(reason).toBe('1;xss');
const decoded = decodeTaintPath(reason);
expect(decoded.ok).toBe(true);
if (decoded.ok) {
expect(decoded.kind).toBe('xss');
expect(decoded.hops).toEqual([]);
}
});
it('kind + truncation marker coexist; the header is never sacrificed to the byte cap', () => {
const { reason, truncated } = encodeTaintPath([{ name: 'longVariableName', line: 12345 }], {
kind: 'sql-injection',
maxBytes: 8, // far below the header size — floor lifts it, hops drop
});
expect(truncated).toBe(true);
expect(reason).toBe(`1;sql-injection|${TAINT_PATH_TRUNCATION_MARKER}`);
const decoded = decodeTaintPath(reason);
expect(decoded.ok).toBe(true);
if (decoded.ok) {
expect(decoded.kind).toBe('sql-injection');
expect(decoded.truncated).toBe(true);
expect(decoded.hops).toEqual([]);
}
});
it('a kind outside [a-z0-9-] is dropped (header omitted), never corrupted into the wire', () => {
for (const bad of ['Command-Injection', 'a;b', 'k|x', 'café', '']) {
const { reason } = encodeTaintPath([{ name: 'x', line: 1 }], { kind: bad });
expect(reason).toBe('1|x:1');
const decoded = decodeTaintPath(reason);
expect(decoded.ok).toBe(true);
if (decoded.ok) expect(decoded.kind).toBeUndefined();
}
});
it('a header-less version-1 string still decodes (kind undefined)', () => {
const decoded = decodeTaintPath('1|a:1');
expect(decoded.ok).toBe(true);
if (decoded.ok) expect(decoded.kind).toBeUndefined();
});
it('the kind header survives the CSV persistence transform byte-exact', () => {
const { reason } = encodeTaintPath([{ name: 'req', line: 2 }], { kind: 'path-traversal' });
const loaded = unescapeCSVField(escapeCSVField(sanitizeUTF8(reason)));
expect(loaded).toBe(reason);
const decoded = decodeTaintPath(loaded);
expect(decoded.ok).toBe(true);
if (decoded.ok) expect(decoded.kind).toBe('path-traversal');
});
it('fails typed on a malformed kind header', () => {
for (const bad of ['1;|a:1', '1;BAD|a:1', '1;', '1;a;b|x:1']) {
const decoded = decodeTaintPath(bad);
expect(decoded.ok, bad).toBe(false);
}
});
});
describe('CSV persistence composition (escapeCSVField ∘ sanitizeUTF8)', () => {
it('the encoding survives the exact persistence transform byte-exact', () => {
const hops: TaintPathHopInput[] = [
{ name: 'req', line: 12 },
{ name: '$tmp_2', line: 13, viaCall: true },
{ name: '#7', line: 99 },
];
const { reason } = encodeTaintPath(hops);
// csv-generator applies sanitizeUTF8 INSIDE escapeCSVField; compose both
// explicitly anyway so the test pins each layer.
const stored = escapeCSVField(sanitizeUTF8(reason));
const loaded = unescapeCSVField(stored);
expect(loaded).toBe(reason); // byte-exact
const decoded = decodeTaintPath(loaded);
expect(decoded.ok).toBe(true);
});
it('a truncated encoding also survives the persistence transform', () => {
const { reason, truncated } = encodeTaintPath([{ name: 'x', line: 1 }], { truncated: true });
expect(truncated).toBe(true);
const loaded = unescapeCSVField(escapeCSVField(sanitizeUTF8(reason)));
expect(loaded).toBe(reason);
});
});
describe('truncation', () => {
it('caller-flagged truncation (hop cap upstream) emits the marker; decode reports path-incomplete', () => {
const { reason, truncated } = encodeTaintPath([{ name: 'a', line: 1 }], { truncated: true });
expect(truncated).toBe(true);
expect(reason).toBe(`1|a:1|${TAINT_PATH_TRUNCATION_MARKER}`);
const decoded = decodeTaintPath(reason);
expect(decoded.ok).toBe(true);
if (decoded.ok) {
expect(decoded.truncated).toBe(true); // informational — NOT an error
expect(decoded.hops).toEqual([{ variable: 'a', line: 1, viaCall: false }]);
}
});
it('byte-cap overflow drops TRAILING hops (source-side prefix kept) and sets the marker', () => {
const hops: TaintPathHopInput[] = [];
for (let i = 0; i < 1000; i++) hops.push({ name: `variable_${i}`, line: i });
const { reason, truncated } = encodeTaintPath(hops);
expect(truncated).toBe(true);
expect(reason.length).toBeLessThanOrEqual(TAINT_REASON_MAX_BYTES);
const decoded = decodeTaintPath(reason);
expect(decoded.ok).toBe(true);
if (decoded.ok) {
expect(decoded.truncated).toBe(true);
expect(decoded.hops.length).toBeGreaterThan(0);
expect(decoded.hops.length).toBeLessThan(hops.length);
// Prefix discipline: hop k decodes hop k of the input, in order.
decoded.hops.forEach((h, i) => {
expect(h.variable).toBe(`variable_${i}`);
expect(h.line).toBe(i);
});
}
});
it('a tiny maxBytes still yields a well-formed (possibly hop-less) truncated path', () => {
const { reason, truncated } = encodeTaintPath([{ name: 'longVariableName', line: 123 }], {
maxBytes: 8,
});
expect(truncated).toBe(true);
expect(reason).toBe(`1|${TAINT_PATH_TRUNCATION_MARKER}`);
const decoded = decodeTaintPath(reason);
expect(decoded.ok).toBe(true);
if (decoded.ok) {
expect(decoded.hops).toEqual([]);
expect(decoded.truncated).toBe(true);
}
});
it('an unencodable hop name stops encoding at that hop and marks truncation (defend, never corrupt)', () => {
const cases = ['a|b', 'a:b', 'café', 'name~x', '', 'a b'];
for (const bad of cases) {
const { reason, truncated } = encodeTaintPath([
{ name: 'ok1', line: 1 },
{ name: bad, line: 2 },
{ name: 'ok2', line: 3 }, // dropped too — prefix discipline
]);
expect(truncated).toBe(true);
const decoded = decodeTaintPath(reason);
expect(decoded.ok).toBe(true);
if (decoded.ok) {
expect(decoded.truncated).toBe(true);
expect(decoded.hops).toEqual([{ variable: 'ok1', line: 1, viaCall: false }]);
}
}
});
it('a non-integer or negative line is unencodable the same way', () => {
for (const line of [1.5, -1, NaN, Infinity]) {
const { truncated, reason } = encodeTaintPath([{ name: 'x', line }]);
expect(truncated).toBe(true);
const decoded = decodeTaintPath(reason);
expect(decoded.ok).toBe(true);
if (decoded.ok) expect(decoded.hops).toEqual([]);
}
});
});
describe('typed parse failures (never a throw)', () => {
const failing: Array<[string, unknown]> = [
['empty string', ''],
['non-string', 42],
['undefined', undefined],
['unknown version', '2|a:1'],
['missing separator after version', '1a:1'],
['hop with no line', '1|a'],
['hop with too many fields', '1|a:1:c:d'],
['non-numeric line', '1|a:x'],
['negative line', '1|a:-1'],
['invalid variable charset', '1|a b:1'],
['empty variable', '1|:1'],
['uppercase flag (reserved charset is lowercase)', '1|a:1:C'],
['marker not trailing', `1|${TAINT_PATH_TRUNCATION_MARKER}|a:1`],
['empty hop segment', '1|a:1||b:2'],
];
for (const [label, input] of failing) {
it(`fails typed on ${label}`, () => {
const decoded = decodeTaintPath(input);
expect(decoded.ok).toBe(false);
if (!decoded.ok) expect(decoded.error.length).toBeGreaterThan(0);
});
}
it('accepts unknown RESERVED lowercase flag letters (forward compatibility)', () => {
const decoded = decodeTaintPath('1|a:1:cz');
expect(decoded.ok).toBe(true);
if (decoded.ok) expect(decoded.hops[0]).toEqual({ variable: 'a', line: 1, viaCall: true });
});
});