mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-11-04 07:27:20 +00:00
19 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Gergő Magyar
|
bad2422aa1
|
feat: add the test scenario map and a Vitest 5 mutation runner (#3686)
* feat(ci): add advisory duplicate-test scan to the PR report Adds `npm run test:dupes` (gitnexus/scripts/test-dupes.ts) and a comment-only CI job that reports, for the lines a PR changes: - same-test: same body and same setup as an existing test - assertions-pinned: every assertion is a literal fn(...) -> matcher that another test already makes - same-fixture-setup: a before* hook builds a fixture another describe already builds - copied-helper: a top-level test helper identical to one elsewhere - orphan-bench-gate: a bench script with --check that nothing runs A case's identity includes its hooks, describe-level statements, wrapper arguments, loop heads, each tables, referenced file-local declarations and resolved imports, with fixture paths compared by value. Extra context can only split a group, so identical bodies that run against different fixtures, mocks or env are not reported. The ci.yml job is continue-on-error and outside the CI Gate. ci-report.yml renders its JSON into the sticky CI comment, keeping only whitelisted kinds and strict path/line shapes because the artifact comes from PR code. `// dupe-ok: <reason>` silences an intentional repeat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(mutation): record Vitest 5 and Stryker 10 probe findings Go/no-go probe for the Stryker Vitest 5 runner: name-filter separator, per-spec task ids, static mutant activation, TypeScript plugin loading in child processes, setup-file meta and hook ordering. * feat(mutation): prepare a mutable copy and a Vitest mutation config Prepared copy blanks TypeScript as-assertions (Stryker's instrumenter skips every TSAsExpression) with equal-length spaces, checked by AST equality, and relocates gitnexus-shared/src inside the package so it can be mutated. The mutation Vitest config resolves gitnexus-shared to the relocated sources and takes its test list from a scope file. * feat(mutation): add a Stryker test runner for Vitest 5 Stryker's vitest-runner 10.0.0 runs zero tests per mutant under Vitest 5 because Vitest 5 matches testNamePattern against ' > '-joined names. This runner selects tests by Vitest task id per file, keeps the configured fork pool, shares the dry-run manifest across Stryker worker processes, attributes hook-time hits to the suite's tests, and records a per-test outcome class in sidecar files. Adds exact devDependencies @stryker-mutator/core and @stryker-mutator/api 10.0.0. * feat(mutation): add global mutant identities and a span-map check A global mutant identity (original repo path, original content hash, span, mutator, replacement) makes mutants comparable across Stryker runs. The span-map check proves every mutant of a prepared copy sits on the same span and text in the original source, apart from blanked as-assertions. * feat(mutation): add a scope driver for Stryker runs run-scope turns a scope file into a prepared copy, a generated Stryker config (vitest5 or command runner), a Stryker run and normalised results: global mutant ids, runner test ids, sidecar outcome classes, consistency flags and the span-map check. * fix(mutation): count assertion kills only for killed runs A run classified timeout, inconclusive, invalid or infrastructure can still carry assertion rows (a test failing after a swallowed hit-limit error, or under a retry). Those kills are not trustworthy, so assertionKilledBy is now empty unless the run's status is killed. Found by the conformance suite. * feat(mutation): add the runner conformance suite (gate 2) A fixture project under test/fixtures/mutation-conformance pins 23 runner behaviours (static and collection-time mutants, beforeAll attribution, disableBail, it.each row ids, outcome classes for runtime, hook, collection, timeout, infrastructure, retry and zero-test runs, NoCoverage, environment isolation, runner reuse and manifest sharing) in two scheduling profiles. The path map and scope driver accept an explicit gitnexus/test/fixtures/mutation-* root for it. * feat(mutation): classify pure-JS unit tests for mutation runs Walks each unit test's import closure (static imports plus statically resolvable dynamic imports, require and createRequire) and rejects tests that reach native packages, worker threads or child processes, read source or fixture files a prepared copy would break, or contain unresolvable dynamic imports. 247 of 911 unit test files are pure JS. * feat(mutation): add gate 1 execution-path fidelity and the envelope record Gate 1 compares a normal Vitest 5 run of the pure-JS category with the runner's unmutated Stryker run: test identities with locations, outcomes, effective config against an allowed-difference list, resolved module paths (everything inside the active sandbox) and a runtime load check for native bindings, worker construction and child processes. It passes on 246 files / 4541 tests and writes test/fixtures/mutation-oracle/envelope.json. The runtime check found commander loading child_process, so the classifier now rejects tests importing commander. * feat(mutation): add the oracle's command-runner run and classifier The Vitest 5 cross-check oracle runs its own Stryker run with the command runner (cleanTempDir false). A wrapper runs the scope's tests per mutant with an oracle reporter and writes a per-mutant record, and the oracle classifies every mutant into the KTD17 outcome classes with its own classifier. Oracle code imports no runner code. * feat(mutation): add the oracle single-test launcher and cost benchmark launch-single runs one test with one mutant in a fresh process inside the oracle's kept sandbox, selected by file:line plus an anchored name pattern, with the mutant installed before any import, an oracle setup file that proves the installed id and buckets hits (load and collection, hook, per-test), a wall-clock limit and a process-group kill. A launch is valid only when exactly the intended test ran. The benchmark projects the three-scope oracle sample's cost on 4 vCPU. A boundary test keeps oracle code from importing runner code. * feat(mutation): add the oracle freeze and gate 3 compare freeze samples mutants per scope with a recorded seed, tops up strata, counts stratum populations from the oracle's own coverage probe, launches every sampled mutant against every scope test (resumable), and writes a manifest keyed by content hashes of the scope's import closure, Vitest configs and prepare tool. compare checks freshness, inventories (tests and global mutant ids) before outcomes, every mutant's class and sampled per-test outcomes, allowing only the four KTD13 normalisations. * feat(mutation): add gate 4 positive controls and determinism Nine planted controls, selected from the scopes' own Stryker mutants by global id (three in relocated gitnexus-shared as-const tables), must be killed by exactly their expected killers. Two scheduling profiles (concurrency 1 with unlimited runner reuse, concurrency 3 with one mutant per runner process) must give identical outcome classes and kill sets. * ci(mutation): add an advisory mutation-runner validation workflow Runs the runner conformance suite (gate 2) on pull requests that touch the mutation tooling, its fixtures, the gitnexus manifest, lockfile or Vitest config. workflow_dispatch also runs gate 1, the gate 3 oracle compare (or a per-scope re-freeze whose manifests are uploaded, never committed), gate 4 and the oracle cost benchmark. All jobs are advisory. Documents the tooling in TESTING.md and the residual deltas against upstream StrykerJS. * test(mutation): freeze the gate 3 oracle manifests and record Phase 0 results Frozen Vitest 5 cross-check oracle manifests for the three pure-JS scopes (seed u7-gate3-1): shared.ts, schema-constants.ts and middleware.ts. Gate 3 passes: all 1,606 mutant classes and all 20,455 sampled per-test outcomes agree with the runner, 0 differences. Adds the approved Test Scenario Map design (updated with Phase 0 findings and gate results) and the Phase 0 implementation plan. * refactor(mutation): simplify the Phase 0 tooling Shared CLI/fs helpers for the non-oracle tools, an oracle-local util for isObject and atomic writes, formatMutateEntry and formatDuration reused, a typed isUnusableClass, and hot-path fixes: per-sidecar index in the gate 3 compare, lazy suite-chain in the runner's dry-run hit attribution, one Classifier per freeze, memoised display paths and Vitest lookups. Behaviour unchanged: unit tests, gate 3 compare (byte-identical report) and the conformance suite (23/23) pass. * fix(review): apply review findings - Move the duplicate-test scanner out of the published scripts/ directory to gitnexus/tools/test-dupes/ so it no longer ships in the npm package. - Scope the orphan bench gate to touched lines, like every other scanner finding. - Let the conformance environment case reach Stryker through an explicit childEnv override, so it exercises the runner's own __STRYKER_ACTIVE_MUTANT__ deletion (a negative check now fails it). - Do not raise the zero-tests flag when a timeout outside-test row explains the zero. - Run Stryker in its own process group and signal the group on cancellation. - Fingerprint conformance inputs and refuse stale artefacts in --compare-only. - Path-gate the mutation workflow on gitnexus-shared/package.json and the setup action; anchor the PR-report path filter with \A...\z. * chore(security): mark the runner's provide/inject key as a gitleaks false positive gitleaks' generic-api-key rule matched PROTOCOL_KEY = 'gitnexusStrykerVitest5' (a Vitest provide/inject key name). Allow the line inline and ignore the fingerprint in the commit that introduced it. * test(mutation): re-freeze the schema-constants oracle manifest Merging main changed the scope's import closure (template-arguments.ts), so gate 3 reported re-freeze required. Re-frozen with seed u7-gate3-2: 72 sampled mutants, 3024 valid observations, 0 invalid. Gate 3 now passes on all three scopes (shared 1232, schema-constants 72, middleware 302 mutants) with 0 differences. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(plans): add the Test Scenario Map Phase 1 plan * chore(security): stop the gitleaks allowlist comment from quoting the key The previous comment quoted the provide/inject key assignment and gitleaks flagged the comment line itself (generic-api-key). Reword it and allowlist that historical finding by fingerprint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mutation): select single tests in the vitest5 runner and keep per-test classes Phase 1 U1 (KTD2, KTD9, KTD16): - driver scope accepts optional testIds, passed as the vitest5.testIds runner option; the dry run collects, maps ids to task ids, runs only those tests and fails before any mutant run on unknown ids - the dry-run sidecar (schema 2) carries the hook marker: per mutant, the tests a suite hook attributed it to - results.json (schema 2) keeps every sidecar attempt's per-test rows (perTest), attemptsDisagree and hookAttributedTests - gate 2 gains a selection scope and hook-marker cases (26 cases hold) Gate 3 hashed inputs are untouched. * refactor(test-dupes): export Layer 0's case extraction for the scenario map Phase 1 U2 (KTD6): export extract, extractCases, buildFrame, closure, staticValue, suppressed and dupeOkReason, plus a per-case record with the describe path, title, context hash, body, each-table node and enclosing loop head. Layer 0's --all scan output is byte-identical before and after (1464 files, 114 findings). * test(mutation): refresh the gate 1 envelope after the runner extensions Gate 1 re-ran over the current pure-JS category after U1 and passed with 0 differences: 251 files, 4667 tests. The classifier now rejects conformance-compare.test.ts (imports the child_process-spawning conformance runner) and php-namespace-extraction.test.ts (now reaches tree-sitter through main), and accepts the three oracle unit tests. * feat(scenario-map): add the eligible-test inventory, identity join and anchors Phase 1 U3 (R1, R2, R15; KTD3, KTD7): - eligibility = gate 1 envelope intersect a fresh classifier run, with the delta; controls mode enrols a fixture's *.case.ts files - static expansion of it.each/test.each/test.for tables and literal for-of loops into per-row instances; anything not statically evaluable is unresolved (fail closed) - a pure join of static instances to runtime records through Phase 0 test ids; any count or name mismatch makes the tests unsupported - continuity anchors (no line; row and body fingerprints) and the one production-file predicate shared by every later stage - test-dupes: additive exports (shapeOf, roleOf, eachTableOf, ASTs, an isTest option); Layer 0 output byte-identical Over the repo: 4667 runtime tests, 4597 joined one-to-one, 70 unsupported (5 tables built at runtime), 0 mismatches. * test(scenario-map): add the controls fixture project with expected verdicts Phase 1 U17: one pair per design §8 pure-JS adversarial case (Math.round, parseInt radix, localeCompare, normalize, Date boundary, regex alternatives, numeric boundary with a constants-only module, an expected-only difference, a derived value reaching toFixed, an enum-member-only difference), an order-dependent pair, an object-key row, and positive controls (verbatim duplicate, repo arithmetic, Set table reads). expected.json was written from the design's stated outcomes. The fixture has its own Vitest project and tsconfig; no repo project collects it. * feat(scenario-map): extract per-instance checks and context Phase 1 U5 (R5, KTD6): - checks: the shape of each expect(...).matcher(...) with literals abstracted and selector literals in the subject kept exact; snapshots hashed from inline text or the .snap entry; a documented supported matcher list, anything else unsupported; per-row for it.each/test.for - context: Layer 0's context hash plus body-level mocks/env writes and the test call's own modifiers and options - checksContextKey for the second candidate source Over the 251 eligible files: 4600 instances in 2.9 s, 183 with an unsupported form, 37 with an unresolved selector. * feat(scenario-map): add Layer 1 per-test V8 coverage and candidate retrieval Phase 1 U4 (R3, R4; KTD4, KTD5): - a Vitest setup file records per-test V8 block coverage in a prepared copy under the mutation config (gitnexus-shared runs from source), with design §5.2 attribution: module load discarded, a suite's beforeAll delta taken at its first beforeEach, concurrent tests flagged unattributable - provenance per test: executed files, native-package entries and worker or child-process spawn call sites - candidates: exact Jaccard >= 0.99 (size-ratio and prefix filters) plus identical checks-and-context buckets; connected components schedule Layer 3 only Over the 251 eligible files: 4667 tests in 1:55 (82.8 s Vitest), 6281 candidate pairs in 611 groups, largest group 36 tests. Controls: 16/16 pairs are candidates, 0 cross-pair candidates. The directory is path-coverage/ because coverage/ is ignored by git, prettier and eslint. * feat(scenario-map): compute the per-instance input facet Phase 1 U6 (R5, R9; KTD6), porting the input-facet spike without its hard-coded inputs: - rules 1-8: tree-sitter shapes of inline source, the COBOL profile, best-fit production tables (executed path, 2-hop import closure, property columns; never pooled), selector literals (reusing the checks predicate), object keys, source order, shared fixtures, fixture files - numeric literals with 0/1/-1/negatives exact, and the +-1 rule against production numeric constants from executed files and the static import closure (folded initialisers included) - discarded values per position with source locations for value flow - the residual-difference guard: a body difference the facet misses makes the pair facet-unequal and both instances unresolved Controls: all 16 pairs match expected.json. The spike's untouched holdout-4 still scores 23/23 distinct. Eligible run: 4667 instances in about 33 s, 245 unresolved. * feat(scenario-map): run Layer 3 per candidate group and merge raw observations Phase 1 U7 (R6, R7, R11; KTD7, KTD8, KTD14, KTD17): - whole-file mutate scope per group from Layer 1 provenance and the static closure; executed-but-unmutated repo files mark tests excluded - a run manifest of the prepared inputs that every shard verifies - two plain Vitest 5 baselines (fresh instance per run) plus the Stryker dry run must agree; the generated config must carry perTest coverage, disableBail and static mutants before mutating - raw observations from every sidecar attempt, keyed by group, run kind, lineage, attempt, test and mutant; class disagreement is inconclusive and a missing result is unknown, never a non-kill - deterministic shard planning that never splits a group, with group-over-budget and start-only-if-it-fits rules Controls: all 16 pairs ran as real groups; every kill set matches expected.json. * feat(scenario-map): add the full isolation pass for repeat-stage pairs Phase 1 U8 (R7; KTD8, KTD9): - select repeat-stage pairs (same coveredBy path, checks, context, facet without a residual difference, and the same non-empty kill set) - per pair one together run and one alone run per test (shared across pairs), mutate restricted to the covered universe, a fresh runner process per mutant, all behind the same KTD8 baseline and config checks (runGroup's steps moved into runMutationRun) - coverage and outcome class per test and mutant must agree across main, together and alone; a missing rerun result is unknown; hook-attributed mutants cap the pair at weak Controls: order-dependent ends disagrees; verbatim-duplicate, repo-arithmetic and set-non-member agree, matching expected.json. Measured fresh-process cost: about 5.5 s per mutant. * feat(scenario-map): add the row-deletion operator Phase 1 U9 (R8, KTD10): - find the one supported production table (object literal with plain keys, literal Set/Map) holding both differing values; spreads, computed keys, Object.assign, writes, getters, proxies, duplicate keys or an ambiguous table make the outcome inconclusive - splice one key's text in a throwaway prepared copy, run the pair with a fresh Vitest instance per run after a control run that proves both pass and the edited file loaded - symmetric / asymmetric (Row) / not applicable / inconclusive, with RowDeletion global mutant ids - vitest-run: an opt-in load probe (probeFiles, loadedFiles) Controls: all 16 pairs match expected.json (object-key-row and set-membership asymmetric). * feat(scenario-map): add the static value-flow check Phase 1 U10 (R9, KTD11): for facet-equal pairs whose discarded values differ, all sources must come back clean or the pair is vetoed to Review: - Babel binds each differing value to its entry (call argument, fixture field, table key); an expected-only difference is unknown - the TypeScript 7.0.2 unstable checker resolves entries and callees through aliases; a version mismatch degrades the whole run - a walk stitched across functions over GitNexus PDG REACHING_DEF and CALL_SUMMARY edges (pdg_query flows are intra-procedural): opaque sinks (lib/@types/node_modules, any regex) veto, truncation, dynamic dispatch, a depth cap or a missing PDG layer are unknown or degraded; repo code is clean only inside the pair's mutated files - row-deletion exports scanSupportedTables so table reads mean exactly the tables row deletion supports Controls: 0 mismatches against expected.json. Whole-repo analyze --pdg: 6:49 wall, 8.4 GB peak RSS. * feat(scenario-map): add the value-flow trace run Phase 1 U11 (R9, KTD11): one traced Vitest 5 run over the tests of every facet-equal candidate pair, with a setup file that wraps value-sensitive built-ins (Math, string transforms and comparisons, RegExp, number parsing and formatting, JSON, Intl, Date, sort, plus U10's opaque callees) and records, per test and production call site, the receiver, arguments and count. Storage and table reads stay unwrapped. A shared site with a different receiver or argument, or asymmetric sites or counts, vetoes the pair; a test whose outcome changes under tracing makes it inconclusive. Controls: all 13 pairs with a trace expectation match expected.json, including receivers 24 and 25 reaching toFixed. Overhead on 40 eligible files: 1.19x pooled, no outcome changes. * feat(scenario-map): add the verdict engine Phase 1 U12 (R10, R11, R15; KTD5, KTD8, KTD9, KTD12): a pure function over merged observations and every stage's evidence. Design §2 precedence as an ordered rule list (Inconclusive, Out of scope, Weak, Repeat strong, Repeat weak, Row, Review, Distinct), with the value-flow and trace vetoes as a gate before any repeat category and the residual difference ending in Review. One verdict per pair with its deciding rule and every weak disqualifier; per-test Weak, Inconclusive and Subsumed; per-file gaps recomputed from all observations with the §5.4 wording; dupe-ok pairs kept with their evidence. Output is deterministic. Controls: all 16 verdicts and rules match expected.json. * feat(scenario-map): build the report, issue body and week-over-week diff Phase 1 U13 (R12, R13; KTD13, KTD15): - the JSON report: run provenance (the design §7 cache-key inputs), completeness and degraded flags, the eligibility delta, Layer 0 findings, every verdict with its evidence, per-test properties, gaps, a seeded spot audit of 10 strong candidates, N of M evaluated and the precision-not-measured statement - continuity: mutant fingerprints without revision or span, universe ids from the pair's common universe, evidence hashes per design §5.5 - a markdown rendering and the tracking-issue body (strong repeats, weak tests, gaps; truncated at 60,000 characters; a run-id marker so a stale run never overwrites a newer body) - baseline comparison: new, changed, unchanged, resolved, with first-run and incompatible-baseline banners * feat(scenario-map): add the CLI and the end-to-end controls run Phase 1 U14 (R14, R16; KTD1, KTD14, KTD17): npm run test:scenario-map with plan, shard, value-flow, merge, all and controls stages exchanging JSON in a work directory, so CI jobs and a local run produce the same report. Shards batch groups into one mutation run when their combined estimate fits, and persist row-deletion and isolation results as they finish. Exit codes are set explicitly. controls: all 16 fixture pairs match expected.json (exit 0, 76 min). * feat(scenario-map): fit the scheduled run into the CI budget Phase 1 U14 follow-up (KTD14, KTD17), measured on a seeded slice of the real category; no evidence rule is relaxed: - value flow runs before the shards and re-plans them, so isolation is spent only on pairs that can still be strong (a value-flow veto or unknown ends Review, a trace inconclusive ends Inconclusive); a missing, stale or degraded value-flow result falls back to the provisional plan - a shards stage prints the final shard count for the CI matrix - --concurrency and --speed-factor feed the estimate; the main-run cost form is fixed + mutated bytes (measured; it does not grow with tests) - a group whose isolation does not fit still runs its main run and row deletion; its pairs get isolation not-run (weak cap), and a shard isolates deferred pairs only when their measured universe fits - groups that execute no production file are not run; their pairs are out of scope Predicted CI run: 9 shards at concurrency 4, 1658-1829 min in total, largest shard 196-253 min (budget 300), 0 groups over budget. controls: 16/16 pairs match, 28 min. * ci(scenario-map): add the weekly Test Scenario Map workflow and a PR controls job Phase 1 U15 (R13, R14; KTD13, KTD14, KTD17): - test-scenario-map.yml (weekly + dispatch, advisory): plan, value-flow, shards (sizes the matrix from the value-flow re-plan), a shard matrix (fail-fast off, max-parallel 10, 300-minute deadline), merge (always, against the newest scenario-map-baseline artifact) and upsert-issue (default branch only; the only job with issues: write; replaces the tracking issue's body only for a complete run that is not older than the body's run marker, comments on partial runs, never closes it) - mutation-runner-validation.yml: a path-gated scenario-map-controls job that runs the controls fixture end to end on PRs touching the tooling actionlint (with shellcheck), zizmor 1.24.1 and the concurrency check pass. Neither workflow can run before the merge. * docs(scenario-map): record Phase 1 in the design and the testing guide Phase 1 U16: design corrections (the stitched PDG walk, the trace's receivers and scope, row deletion and baselines on plain Vitest 5, the isolation pass on testIds after value flow), the demote-only additions, the refreshed gates, the measured Phase 1 costs and the CI topology change; TESTING.md gains a Test Scenario Map section (stages, controls fixture, supported matchers, wrapped built-ins, weekly workflow, dupe-ok). * fix(scenario-map): store observations compactly so large batches fit in memory The first full run crashed a shard with "Invalid string length": observations were materialised for every collected test x every mutant, including a valid not-covered survival for each pair (1.6M objects for a batch of 73 selected tests over 21,965 mutants; the batch file needed about 700 MB of JSON). - only a run's selected tests produce observations - each run records its universe (selected tests, mutants with their status, reasons and hook marker); only rows, covered or static mutants without a row, and hook-named tests are stored explicitly; a not-covered survival is derived when read - every consumer (pair comparison, isolation, verdict engine, report, batch split) reads through one index; group observation and isolation run files move to schema 2, and old files are rejected with a message - merge lists an unreadable group result as incomplete instead of dropping it; repeat-stage decisions keep comparison summaries only On the crashing batch: 141,308 explicit observations instead of 1.6M, 12.1 MB instead of an overflow, and a byte-identical verdict report. controls: 16/16 pairs match. * docs(scenario-map): record the Phase 1 measured slice; give the real-repo classify test headroom - design §9: the full-category report comes from the first weekly workflow run; the local full run was stopped because it saturated the shared machine. Records the measured slice and the whole-category plan and value-flow figures. - classify.test.ts: classify the real repo once in beforeAll with a 120 s timeout; it takes about 30 s on a busy host and timed out at the default 30 s. * fix(review): apply review findings and simplify the scenario-map tool Review (ce-code-review, validated findings): - trace: a baseline or traced run that is invalid, inconclusive, retried or has outside-test failures, unhandled errors, unwrapped entries or recorder errors makes every pair inconclusive (trace-run-invalid), never clean; the value-flow stage reports it and merge marks the run partial - verdict: tests pin that a not-required static check for differing values reads as unknown, plus the worker-spawn, missing-provenance, unresolved-input and unknown-hook-marker weak caps - workflow: the baseline comes only from this workflow's own scheduled or dispatched default-branch runs of this repository; an unreadable baseline shows the incompatible-baseline banner instead of failing merge - controls: the oracle fails on an empty expected list, on any unlisted repeat verdict and on a case file without an expected pair - value flow: the PDG analyze budget (25 min) sits below the 50-minute CI step; the TypeScript checker server is closed when opening fails - --concurrency now also bounds every in-process Vitest run (maxWorkers), so local runs stay within the requested cores - merge reads isolation runs tolerantly (an unreadable file makes the run partial); a duplicate test title is renamed; a stale TESTING.md heading reference is updated Simplification (behavior-preserving; merge and plan outputs byte-identical on fixed inputs): one util.ts for shared helpers, a run-level row-deletion cache, one classify() per plan, memoised universe ids and single-pass pair/subsumption/report loops. * fix(test-dupes): keep beforeAll and beforeEach fixture builds apart A same-fixture-setup key now includes the hook name. A beforeAll fixture is built once per suite and a beforeEach one per test, so sharing one result between them would drop per-test isolation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(scenario-map): temporary label-gated pull_request run for PR #3686 schedule and workflow_dispatch only run from the default branch, so the Test Scenario Map workflow cannot run before this PR merges. Adding the run-ci label to the PR starts one full run. A pull_request run writes no tracking issue and no baseline. Remove the trigger before merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(scenario-map): remove the temporary pull_request trigger The one full run on PR #3686 finished (run 40183982111, complete, 9 of 9 shards). This reverts |
||
|
Gergő Magyar
|
d8a256a03a
|
fix(taint): avoid false positives for guarded Git arguments (#3703)
* fix(taint): refine values after accepted guards Preserve bounded value restrictions through local flows and function summaries. Check the complete command against the modeled sink use before suppressing a finding. Unknown paths and exhausted budgets retain conservative findings. Refs #3490 * fix(taint): preserve guard proof identities Reject destructured/rest formals before they inherit a whole-argument proof, and include transitive alias uses in bounded guard certificates. Add worker regressions, argument-heavy scaling coverage and global refinement fallback diagnostics. Refs #3703 * perf(cfg): reuse scope and argument facts Reduce duplicate native AST reads and allocation in TypeScript CFG collection while preserving argument forwarding and malformed-input recovery. Add regression cases and a portable paired benchmark with recorded results. * test(cfg): distinguish validator cases --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
samuel mbabhazi
|
0f3bad23b9
|
feat(web): resolve a root-relative GITNEXUS_BACKEND_URL against the page origin (#3640)
* feat(web): resolve a root-relative GITNEXUS_BACKEND_URL against the page origin A GITNEXUS_BACKEND_URL of `/` (or `/prefix`) is injected as-is, and the web client resolves it against the origin that served the page, so one gitnexus-web deployment reaches its API on every hostname a reverse proxy gives it. Protocol-relative values are left to the URL check and rejected. Documents the two ways to route /api/* behind a proxy. Closes #3593 * fix(web): reject a backslash-prefixed same-origin path The URL parser reads a backslash as a slash, so `/\host` resolved against the page origin became `http://host/`, an origin change. Both the server check and the client resolution now treat it like `//host`. * fix(web): keep a configured base ending in /api, and drop the unsupported direct route normalizeServerUrl stripped a trailing /api from every URL, so a deployment configured with GITNEXUS_BACKEND_URL=/proxy/api passed health and discovery and then connected to /proxy/repo. The deployment's own base is now kept as configured; the /api cleanup stays for hand-entered URLs. A regression covers health, discovery and repository selection under /proxy/api. The README no longer offers routing /api/* straight to `gitnexus serve` on a public hostname: serve refuses to start with GITNEXUS_PUBLIC_ORIGIN set until it has authentication of its own. * fix(web): keep only a same-origin base as configured The bypass in normalizeServerUrl matched any injected default, so an absolute GITNEXUS_BACKEND_URL ending in /api would have produced a doubled /api segment, and one carrying credentials would have kept them. Only a root-relative deployment base is kept now; it is resolved from the page origin, so it has neither. Absolute injected URLs are normalized as before. |
||
|
Gergő Magyar
|
48f108a58a
|
docs: simplify guides and verify receiver gaps (#3694)
* docs: align resolution architecture and roadmap * docs: add language ingestion gap inventories * test: audit receiver gap targets and coverage Check exact declaration IDs, expose the Dart await wrong-owner edge, and execute previously excluded C++, Rust, PHP, Dart and Vue shapes. Refresh the baseline and all language inventories from fresh measurements; call-drop counts remain unchanged. * docs: focus roadmap and shorten project README Keep the resolution roadmap limited to future work and acceptance criteria. Move detailed editor, CLI, MCP, deployment and wiki instructions into linked guides, preserve their command examples, and correct stale onboarding claims. * docs: correct setup and audit descriptions --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
01af833875
|
fix(scope): preserve lexical bindings in HTTP wrapper detection (#3632)
* docs(scope): clarify lexical binding ownership and wrapper provenance Checkpoint for #3630. Publishing the implemented source, regression tests, and benchmark harness as a draft PR; full-suite and final scaling validation remain in progress. * fix(scope): publish #3630 docs/plans * fix(scope): publish #3630 gitnexus/bench/lexical-binding-provenance * fix(scope): publish #3630 gitnexus/bench/lexical-binding-provenance/fixtures * fix(scope): publish #3630 gitnexus/bench/lexical-binding-provenance/results * fix(scope): publish #3630 gitnexus/src/core/ingestion * fix(scope): publish #3630 gitnexus/src/core/ingestion/languages/typescript * fix(scope): publish #3630 gitnexus/src/core/ingestion/languages * fix(scope): publish #3630 gitnexus/src/core/ingestion/pipeline-phases * fix(scope): publish #3630 gitnexus/src/core/ingestion/route-extractors * fix(scope): publish #3630 gitnexus/src/core/ingestion/workers * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/lib * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/packages/http * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/packages/http/src * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/server * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/web * fix(scope): publish #3630 gitnexus/test/integration/resolvers * fix(scope): publish #3630 gitnexus/test/unit * fix(scope): publish #3630 gitnexus/test/unit/group * fix(scope): publish #3630 gitnexus/test/unit/scope-resolution * fix(scope): publish #3630 gitnexus/test/unit/scope-resolution/typescript * fix(scope): publish #3630 gitnexus-shared/src/scope-resolution * fix(scope): treat wrapped assignment targets as writes `(app) = new Map()`, `(app as T) = x` and `app! = x` assign the same binding as `app = x`, but route-fact invalidation and the shared pattern walker stopped at the wrapper, so Express provenance survived the write and produced a false route. Recurse through parenthesized, as/satisfies and non-null targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(routes): resolve wrappers through overloads, declarator lists and barrels Three wrapper shapes lost their identity: - Bodyless overload signatures were indexed as separate definitions sharing the implementation's binding, so the binding looked ambiguous and both local and imported calls were rejected. Signatures now declare no callable. - Facts for `const base = 0, request = (url) => fetch(url)` followed the first declarator. The captured name now selects its own declarator. - An import resolved to a workspace barrel stopped at index.ts. Workers now record named, star and imported-then-exported re-exports, and resolution follows them to the declaring module: local and named exports shadow `export *`, star exports never carry `default`, and conflicting star exports block the name. No repository-wide name fallback is restored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(scope): assert closure ownership against the declaration The closure-write assertion compared one lookup with itself, which also passes when both are undefined. Compare it with the outer declaration and the shadowing parameter, and keep the memoization check explicit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(routes): share lexical helpers and release per-file HTTP contexts - Share binding keys, parameter patterns and transparent expression wrappers from lexical-bindings instead of four local copies; drop the unused moduleStem export; name the inferred-source predicates in lookup. - Key the per-file HTTP context by its tree rather than its ParsedFile. Workers retain ParsedFiles for the whole chunk, which kept every file's AST and traversal maps alive until the chunk flushed. - Index re-exports by name, memoize barrel walks per starting export, and keep facts without import targets as-is instead of copying them. - Move TS-only wrapper cases into the TS describe.each instead of branching inside the per-language loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(routes): keep wrapper identity for HOCs, CommonJS, object keys and fetch packages Four definition shapes that main linked by name lost their wrapper identity: - `const load = useCallback((url) => fetch(url))` and curried HOCs produced no facts because definition lookup stopped at the call; it now follows the callable argument, as the structural definition capture does. - `module.exports = { request }`, `exports.x = …` and `module.exports.x = …` were not exports, so destructured `require` callers missed the wrapper. CommonJS export assignments are recorded when `module`/`exports` are the unshadowed globals, including functions defined inline in the export object. - An object-literal key or method name was resolved as a lexical binding and collided with a same-named module wrapper. Only binding identifiers now name a definition's binding. - `import fetch from 'node-fetch'` no longer counted as fetch. Imports of known fetch implementations (node-fetch, cross-fetch, isomorphic-fetch, the fetch member of undici and whatwg-fetch) are fetch evidence again; parameters and other local names still are not. Reference identifiers are also resolved only when their name is bound to a callable or class region; other identifiers cannot add a reachability edge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(routes): drive the forwarding fallback case from data and cover re-export merges Replace the boolean-switched forwarding fallback case with table rows instead of branching inside the test, and assert that barrel re-export facts survive worker sub-batch merges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(bench): reject stale or dirty call-graph exports The committed comparison named a candidate revision that is not on this branch, and nothing stopped it. Comparison now requires both exports to come from clean checkouts and the candidate export to come from the checkout running the comparison. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(bench): regenerate the call-graph comparison on the reviewed head Clean baseline |
||
|
Gergő Magyar
|
82347f0f07
|
fix(scope): resolve local imports by lexical ownership (#3532) | ||
|
Gergő Magyar
|
274ec3df6e
|
feat: index Jupyter notebooks as Python (#3381) | ||
|
mengkaka
|
4154b63131
|
feat(indexing): add Objective-C semantic indexing support (#3179)
* docs: add Objective-C fork provider notes * feat(objective-c): add deterministic provider and grammar * feat(objective-c): finalize provider MVP * fix(objective-c): harden provider integration * fix(objective-c): normalize bare macro markers * docs(objective-c): integrate provider documentation * fix(objective-c): harden resolution and header classification * fix(objective-c): complete provider follow-ups * fix: address Objective-C review follow-ups * chore: format Objective-C grammar sources * fix(objective-c): harden review follow-ups * Address PR review feedback (#3179) Keep Objective-C chunking and macro recovery aligned with the grammar, and stop Community MEMBER_OF edges from leaking into symbol context. Co-authored-by: Cursor <cursoragent@cursor.com> * Address follow-up review on ObjC chunking and language fallback. Keep preprocessor directive text from changing file-scope brace depth, group real ivar nodes, skip header modifiers, and restore Rakefile/Gemfile detection through getLanguageFromFilename. Co-authored-by: Cursor <cursoragent@cursor.com> * Parse Objective-C headers with the objc grammar in embeddings. ensureAndParse and structural extraction now use the same content classifier as ingest, including method snippets from .h files, so Protocol/Category/Class chunks are not re-parsed as C++. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3179) Keep file-scope macro elision off C line splices and @interface/@protocol/@implementation bodies, and attach ivar attributes to the following instance variable when chunking. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(bench): rebaseline Objective-C CSV emit * feat(objective-c): add workspace resolution and linear emit benches Plain .h files are classified as C++, so the ObjC pass could not resolve #import of those headers. Load a C/C#-style workspace once per pass, and keep protocol-candidate USES linear. Refs #3179 Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3179) - Compare LadybugDB labels() as a scalar when excluding Community MEMBER_OF edges. - Walk superclass members, skip file-static C sibling defs, and ignore comments in ObjC header/macro scans. Note: pre-existing failure in objective-c-provider integration (worker-pool ready timeout) not addressed by this PR. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3179) Emit Objective-C declaration captures so compilation-unit siblings can share header/implementation bindings, and keep class vs protocol visibility groups distinct. Note: pre-existing failure in worker-pool startup (GITNEXUS_WORKER_READY_TIMEOUT_MS) not addressed by this PR. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3179) Emit every comma-separated property/ivar declarator, and count @interface after a multiline block comment closes so in-declaration macros stay intact. Note: pre-existing failure in worker-pool startup (GITNEXUS_WORKER_READY_TIMEOUT_MS) not addressed by this PR. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: ximengkai <ximengkai@soyoung.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Gergő Magyar
|
54f97c86c7
|
fix(impact): make File risk comparable via shared axes (#3075) (#3082)
* docs(plans): add impact file risk plan Capture the evidence, constraints, and verification path for fixing incomparable File and symbol impact risk. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(impact): centralize risk scoring Keep the existing thresholds in one shared scorer and expose a common-axis comparison for targets with unavailable enrichment axes. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(impact): expose incomparable file risk scale Mark File impact results when process and module axes are unavailable, and provide a common-axis score for honest cross-kind comparisons. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(impact): explain cross-kind risk comparisons Surface the common-axis score in CLI and agent guidance while reusing the shared threshold ladder in the web impact tool. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(impact): fail closed when enrichment is incomplete Preserve proved HIGH/CRITICAL process counts, treat failed queries as UNKNOWN, and surface riskScale metadata on MCP, group, CLI, and Graph-RAG File walks. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Gergő Magyar
|
9372b17049
|
fix(python): resolve calls through an unaliased dotted namespace import (#2826) (#2828)
* fix(python): resolve calls through an unaliased dotted namespace import (#2826) `import pkg.db` followed by `pkg.db.session_scope()` emitted no CALLS edge, while all three sibling spellings resolved. In a codebase whose style guide mandates absolute imports this is close to the only cross-module call form used, so `impact()` reported `impactedCount: 0, risk: LOW, epistemic: exact` for functions with dozens of real callers — a dropped caller reading as a verified all-clear. The resolution path was never missing; one map was keyed on the wrong half of the import. `interpretPythonImport`'s plain arm splits `import pkg.db` into `localName: 'pkg'` (the name Python actually binds) and `importedName: 'pkg.db'`, and finalize carries both onto the edge as `localName` / `targetExportedName`. `collectNamespaceTargets` keyed only on `localName`, but the receiver text captured at the call site is the whole dotted path — Python's query binds the attribute's `object` field with a wildcard, so `pkg.db.session_scope()` yields the receiver `pkg.db`. Case 0 declines it (a module is not a class) and falls through, Case 1 looks up `pkg.db` and misses, and Case 1.5 needs `resolveQualifiedReceiverMember`, which only the C++ provider implements. The site drops silently. Key the map on the dotted import path as well — gated on a provider opt-in, not on the edge shape. The shape alone cannot decide it: Swift's `import Foo.Bar` produces the identical pair (`localName: 'Foo'`, `targetExportedName: 'Foo.Bar'`), but there the FIRST segment is the resolved target and `Foo.Bar` names a nested type. Minting a key for it would hand `resolveConstructionExpressionClass` an authoritative namespace — that branch deliberately does not fall through on a miss — and break `Foo.Bar(x)` construction that resolves correctly today. Hence `ScopeResolver.namespaceReceiverIncludesImportPath`, which only Python sets. The root-segment check on the added key does real work: `import pkg.db as pdb` binds only `pdb`, so writing `pkg.db.f()` there is a NameError, and its edge (localName `pdb`, path `pkg.db`) is correctly rejected. Two same-package imports stay separate — `import pkg.db` + `import pkg.cache` key `pkg.db` and `pkg.cache` independently, so neither call can land in the other's module; the shared `pkg` bucket keeps its existing ambiguity rather than gaining any. Tests: five integration rows (the issue's own repro, the three sibling spellings as controls, non-crossing two-package imports, a three-segment receiver, and dotted construction) plus a unit pin on the keying rule that asserts a Swift-shaped edge mints nothing. All five integration rows fail on the pre-fix tree; the controls pass on both, which is what makes them controls. Resolver integration suite 3024 passed / 1 skipped / 0 failed; scope-resolution unit suite 1446 passed. This changes what the resolver produces, not how it is stored — no schema or version constant applies, and an existing index needs a re-analyze to show the new edges. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(resolution): shadow-test a dotted namespace key by its root segment (#2826) `isNamespaceNameShadowed` walks the scope chain looking for a binding, type binding, lexical name, or owned def named exactly `namespaceName`. Once a namespace key can be a dotted import path, that string never matches anything: `import pkg.db` binds `pkg`, so a local `pkg = Decoy()` shadows the import, but the guard was asked about `pkg.db` and answered "not shadowed". The consequence is not a missed edge but a wrong one. The caller treats a verified namespace as authoritative and deliberately does not fall through to the workspace-wide simple-name heuristics, so an unguarded shadowed receiver resolves construction against the imported module instead of the local value. Test the first dot-separated segment instead. Single-segment names are unaffected — their root is themselves — so every pre-existing row keeps its behaviour. This ships with the key that first routes a dotted name into the guard rather than after it: the previous commit is what makes the defect reachable. The new pin fails on the pre-fix guard (verified by reverting the four comparisons and re-running: 1 failed / 5 passed), so it discriminates rather than merely passing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(python): pin the callee name on the dotted-construction row (#2826) The row asserted only that `builds` reached `pkg/db.py`. That module also exports `session_scope`, so a regression that resolved the construction to the wrong member of the right module would have kept the test green — it pinned the file, not the answer. Assert the exact edge set for the caller instead. Verified against the current tree with a scratch probe: `builds -> Model@pkg/db.py` is the only edge the file produces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(plans): include the #2826 engineering plan in the PR `.gitignore` keeps `docs/*` local because planning output is normally throwaway. Force-added here at the reviewer's request so the plan travels with the work it drove: it records the evidence chain behind the fix, the two places the plan turned out to be wrong, and the follow-ups deliberately left out of scope. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(resolution): make the namespace shadow guard shared (#2826) `isNamespaceNameShadowed` lived module-private in `compound-receiver.ts` with a single caller. The namespace map it guards has three consumers, and the next commit adds the guard to a second one, so it moves to `scope/walkers.ts` alongside the other scope-chain primitives rather than being duplicated. Behaviour is unchanged — this is a move plus documentation. Two notes were added because both are easy to get wrong later: - Fails closed on a missing scope or a parent cycle. For every caller, suppressing costs a missing edge while trusting a corrupt scope chain costs a wrong one, so the bias is deliberate. - It reads `scope.bindings` DIRECTLY rather than through `lookupBindingsAt`, which is the opposite of the fix #2745 applied to Rust's `headBoundLocally`. There the question was "is this name bound at all?", so missing finalize's import channels lost real bindings. Here the question is "does something LOCAL shadow the import?", and the import's own finalized binding is exactly what must not count — routing this through `lookupBindingsAt` would find every namespace import shadowing itself and suppress the lot. Verified against a target module carrying a self-named def, which still resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(python): close the three remaining namespace-receiver gaps (#2826) Three defects the first fix left behind. All three were confirmed by probe before being touched, and a fourth suspected gap was disproved the same way. ## 1. Case 1 resolved through an import a local had shadowed `namespaceTargets` is collected per FILE, but Case 1 in `receiver-bound-calls` consulted it with no lexical guard at all, so import pkg.db def f(pkg): # parameter shadows the package return pkg.db.session_scope() emitted an edge to pkg/db.py. That is a WRONG edge, and it predates the dotted key: the single-segment spelling (`import single` + `def f(single)`) failed identically. The compound-receiver construction path has applied this guard since #2770; Case 1 simply never did. Now both use the shared guard. ## 2 + 3. The root key named the leaf module, not the package These read as two gaps and are one. `import a.b.c` binds ONE name — `a` — but makes three attribute paths callable, naming three different files: a → a/__init__.py a.b → a/b/__init__.py a.b.c → a/b/c.py The map keyed only `a`, pointed at the LEAF. So `a.helper()` resolved into a/b/c.py whenever that module happened to export `helper` — silently preferring a decoy over the real definition in the package — and `a.b.mid()` resolved to nothing at all. One wrong edge and one missing edge from a single mis-keying. Fixing it needs per-language knowledge the shared collector cannot have: which prefixes are reachable, and which file each names. The `__init__.py` convention is Python's alone, and the edge shape is ambiguous across languages — Swift's `import Foo.Bar` produces an identical `localName`/`targetExportedName` pair that means the opposite thing. So the previous commit's boolean opt-in is replaced by `ScopeResolver.namespaceReceiverPaths`, which returns every spelling with the file it names; absent or declining, the shared default (bound name → own target) is unchanged for every other language. Prefix files are proposed, not asserted — `moduleFileExists` drops any the workspace never parsed, so a PEP-420 namespace package contributes no key rather than one pointing at a missing file. ## Disproved: C# was not a fourth gap The plan listed C# `using System.Collections.Generic` + `System.Collections.Generic.List` as the same class of bug. It is not: a probe shows `My.Deep.Space.Helpers.Work()` already resolves through the FQN namespace bindings in `walkers.ts`. No change made, and the claim is withdrawn rather than carried forward as a known gap. ## Testing Integration: the shadow block asserts the exact surviving edge set (an absence-only assertion would also pass if the guard over-suppressed and killed the clean rows); the prefix block asserts all three spellings land on their own file, with `helper` defined in BOTH package and leaf so a wrong edge is visible rather than merely possible. Unit: 16 rows on the keying contract, including that a Swift-shaped edge mints nothing and an alias import keys neither the path nor the root. Resolver integration 3024 passed / 1 skipped / 0 failed; scope-resolution unit 1452 passed; tsc clean in both packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(python): probe both path separators when resolving a prefix package (#2826) Workspace file paths are not normalized to POSIX at ingestion — `import-target` already re-normalizes at five other comparison points, and `moduleScopeByFile` is keyed by the raw `ParsedFile.filePath`. The prefix probe built only the `/` spelling, so on Windows it would compare `a/b/__init__.py` against an `a\b\__init__.py` key, find nothing, and mint no prefix keys at all. That fails quietly, which is the worst shape for it: `a.b.mid()` simply goes back to unresolved on one platform, with no drop recorded and every test on POSIX still green. Probe both spellings and key whichever the workspace actually holds. The new row is mutation-tested — reverting to the `/`-only probe turns it red (1 failed / 10 passed), so it pins the behaviour rather than passing alongside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(python): correct three defects a multi-lane review found in this PR (#2826) All three were introduced by this PR's own earlier commits, and none was found by re-reading the diff — each came from a lane attacking an angle the author had not. ## 1. The shadow guard ran BEFORE the map lookup it gates Case 1 evaluated `isNamespaceNameShadowed` unconditionally, then consulted `namespaceTargets`. So every call/read/write site with an explicit receiver, in every language, paid a scope-chain walk (a Set allocation, three Map lookups and a linear `ownedDefs` scan per level) ahead of an O(1) hash miss that was going to decline it anyway. The proof it was an oversight rather than a decision sits in this same PR: the sibling guard in `compound-receiver.ts` reads the map first and only guards on a hit. Two call sites of one shared function, opposite order. Semantics are identical either way — a miss yields `undefined` regardless — which is exactly why it survived several readings. ## 2. Prefix packages were anchored on the import spelling, not the resolved leaf `pythonNamespaceReceiverPaths` built `a/__init__.py` from the dotted path joined at the workspace root, never consulting the file the import actually resolved to. But `resolvePythonImportTarget` resolves off-root in two of its three tiers, so `import utils.db` can land on `libs/common/utils/db.py`. That produced a wrong edge where a same-named `utils/` package exists at the root, and produced NOTHING in a `src/` layout — the prefix feature was inert for the most common Python project shape, silently. Now the prefix directories are derived by walking back from the resolved leaf, which is exact for root, `src/` and off-root layouts alike. It also inherits the leaf's own separator, which subsumes the previous dual-separator probe: that probe was dead code anyway, because `filesystem-walker.ts` normalizes `\` to `/` before a path ever becomes a `ParsedFile.filePath`. Its test row is removed rather than left asserting an unreachable state. ## 3. Keying the root at `__init__.py` INSTEAD of the leaf lost re-exports `findExportedDef` accepts only a binding whose `origin === 'local'`. The canonical Python package re-exports from its submodules — `from .b.c import helper` in `__init__.py` — which is an IMPORT binding, so it is rejected. Keying the prefix solely at the package therefore turned `a.helper()` from a correct edge into no edge at all for the most common package shape. Every fixture in this PR defined its members locally in `__init__.py`, which is precisely the one layout where that mistake is invisible. The prefix now keys the package FIRST and the leaf behind it. A real definition in `__init__.py` still wins over a same-named decoy deeper in the package, and a name merely re-exported there still resolves through the leaf. Ordering is the contract, so the unit rows assert the exact arrays rather than membership. ## Testing New rows: off-root layout with a decoy `utils/` at the root, and a `src/` layout. Both mutation-tested — reverting to the spelling-anchored build turns them red. The re-export case was verified end-to-end with a scratch fixture whose `__init__.py` only re-exports (`uses -> helper@a/b/c.py`). Resolver integration 3131 passed / 1 skipped / 0 failed — unchanged from before these fixes, so they regress nothing. Scope-resolution unit 1459 passed. tsc clean in both packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(resolution): stop the namespace shadow guard AT the module scope (#2826) CI caught a regression this PR introduced: `cjs-exports-assignment.test.ts` lost both of its cross-file rows — cross-file require() member call resolves expected [] to deeply equal [ 'handle' ] an `exports` parameter does not hijack the module (UMD factory) expected [] to deeply equal [ 'publicApi' ] — i.e. `const svc = require('./svc'); svc.handle()` stopped resolving in JavaScript. Cause: in CommonJS the namespace import IS a variable declaration. One statement produces both the ImportEdge and a module-scope `const` binding, so the guard, by inspecting the module scope, found the import's own name there and read it as a shadow of itself — suppressing exactly the receivers it exists to enable. The guard's own contract sentence already said the right thing: "a declaration BETWEEN the call site and its module scope". The module scope is the floor of that walk, not a rung on it. It now returns at Module without inspecting it. Nothing is lost on the suppression side: a genuine shadow is a parameter, a local, or a nested declaration, and all of those live in scopes strictly inside the module. The Python rows that pin suppression (`def f(pkg): pkg.db.f()` and its single-segment `import single` twin) still pass, because a parameter is an inner scope. Worth recording for the next reader: two independent review lanes examined this exact scenario and both REFUTED it, reasoning that `require()` yields an ImportEdge in `scope.imports` rather than a local binding. That is true for Python's `import x` and false for CommonJS, where one statement is both. My own probe used a Python fixture and so could not surface it either. Agreement between reviewers was not evidence; the test corpus was. Verified: cjs-exports-assignment 36/36, the #2826 integration rows 7/7, scope-resolution unit 126/126. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
ed8ab1c246
|
fix(scope-resolution): resolve callable reference flows (#2437) (#2522)
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* docs(plans): add provider-hook value-refs plan (#2437) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(plans): deepen #2437 plan to USES + property-dispatch design Design revised after prior-art research (Kythe ref vs ref/call, Joern METHOD_REF, Feldthaus field-based call graphs, CodeQL impliedReceiverStep): registration sites emit reference-class USES, invocation is recovered by a field-based property-dispatch pass synthesizing CALLS at member-call sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scope-resolution): model provider-hook value references (#2437) Functions referenced as object-literal property values (provider hooks like emitScopeCaptures: emitCppScopeCaptures) previously produced no edge at all, so impact/context reported a false-safe 0 upstream dependents. Two coordinated halves, per prior art (Kythe ref vs ref/call, Joern METHOD_REF, Feldthaus ICSE'13 field-based call graphs, CodeQL impliedReceiverStep): - Registration -> USES: new ReferenceKind 'value-ref'; TS/JS queries capture pair values and shorthand properties (with @reference.property-key); emitted as a reference-class USES edge, reason 'scope-resolution: value-ref'. Resolution is callable-gated so plain values emit nothing. - Dispatch -> CALLS: new shared pass emitPropertyDispatchCalls synthesizes CALLS (reason 'property-dispatch', confidence 0.7, per-key fan-out cap 32 calibrated on this repo's 16-provider hook tables) from member-call sites to every function registered under the same property key. Deviation from plan: the pass owns value-ref resolution entirely via the post-finalize findCallableBindingInScope walker — the shared registries only see pre-finalize local bindings, so imported hooks (the c-cpp.ts case) were unresolvable through lookupForSite; Reference.propertyKey passthrough dropped as unnecessary. SCHEMA_BUMP 13 -> 14: ParsedFile gains value-ref sites + propertyKey. Verified end-to-end: impact(emitCppScopeCaptures, upstream) now reports 8 impacted / HIGH with extractParsedFile (true dispatch caller) at d=1 via property-dispatch and the c-cpp.ts registration via USES. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(scope-resolution): cover value-ref registration and property dispatch (#2437) Integration: same-file/cross-file/aliased/shorthand registrations emit USES; non-callable and destructuring values emit nothing; dispatch sites gain property-dispatch CALLS (incl. JS twins and per-language partitioning); fan-out-capped keys are dropped entirely; factory-call values unchanged. Unit: capture-shape pins for @reference.value-ref + @reference.property-key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope-resolution): surface dropped property-dispatch keys in stats (#2437) Review finding: skippedKeys was returned but discarded — a hook table larger than the fan-out cap silently reopened the #2437 gap for those keys. Log dropped keys and fold value-ref USES + dispatch CALLS into referenceEdgesEmitted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(plans): add callable reference-flow implementation plan * fix(scope-resolution): close property-dispatch review gaps * feat(scope-resolution): add callable flow facts * feat(scope-resolution): resolve callable value flow * feat(scope-resolution): resolve callable references across providers * fix: harden callable reference flow resolution * fix(scope-resolution): preserve callable binding semantics * docs(plans): add pr-2522-review-fixes plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(storage): bump INCREMENTAL_SCHEMA_VERSION for callable-value-flow edges Callable-value-flow CALLS/USES edges (#2437) can connect two files whose content did not change, but the incremental write set only covers changed files — a top-up against a pre-v7 index would silently omit the new edges for every unchanged file pair, indefinitely. Force the one-time full re-analyze (review finding 1, #2522). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(storage): sanitize callable-flow sites per-site at load, log drops The load-time validator rejected the WHOLE ParsedFile when one site was malformed or over-bound, with no logging — and C++ legitimately emits empty-string parameterTypes entries ('' = unknown, the ReferenceSite.argumentTypes convention) for cv-only/ERROR-recovered types, so real repos fell into a permanent, silent warm-cache-miss reparse loop through the #1983-sensitive main-thread path (review finding 7, #2522). Now: '' entries are valid in type arrays; a malformed/over-bound site drops only itself (counted, warned once per load); only non-array garbage — evidence the serialization itself is untrustworthy — rejects the file. Deviation from plan §6 wording: validator-side tolerance replaces emit-side clamps — smaller diff, same asymmetry closed at the single chokepoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope-resolution): keep declarations in the union for reassigned callable cells The binding-lookup suppression for fact-constrained cells was wholesale: reassigning a declared function through its own name (greet = other; greet()) deferred the call to the solver, which then refused the lexical lookup that resolves the declaration — an unresolvable RHS yielded zero CALLS for a call that resolved pre-flow (review finding 8, #2522). Suppression now applies only to cells bound by FORMAL facts — its actual purpose (a parameter whose grammar emits no declaration binding must not adopt a same-named outer function). Copy/alias/store/load destinations keep their declaration as an inclusion seed (Andersen-style union). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope-resolution): count forfeited deferred sites in the budget-bailout warning On work-budget exhaustion the deferred invoke sites end the run with zero CALLS — free-call fallback and reference emission already skipped them — but the warning said 'ordinary graph emission remains untouched', which is false for exactly those sites. The warning context now carries the unresolved deferred-site count and the comment states the real cost (review finding: budget-bailout honesty, #2522). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scope-resolution): surface dropped property-dispatch keys in stats and warn payload The over-cap warning carried only a count; the dropped key NAMES were discarded and RunScopeResolutionStats had no field, so the PR-body claim 'includes them in resolver statistics' was unimplemented (review finding, #2522; reviewer ask on the fan-out cap). The warn payload now names up to 20 dropped keys and the stats carry propertyDispatchSkippedKeys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(scope-resolution): drop producer-less ownerQualifiedName from formal sites No capture emitter anywhere produces @callable-flow.owner-qualified-name — the solver branch consuming it was unreachable in production, yet the field was typed, parsed, validated, and unit-tested with hand-built input (review finding 16, #2522; YAGNI). Re-add with a real producer if C++ qualified member declarators ever need it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(scope-resolution): drop dead callable-flow knobs CallableFlowPassingMode 'callable-object' had no producer and no consumer distinguishing it, and CallableFlowCaptureOptions.extractCallArguments had no language providing it (unlike its live sibling extractCallCallee) — review finding 17, #2522 (YAGNI). The invocation-kind 'callable-object' is a different, live concept and stays. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): bind subscripted callable cells to the container, not the index terminalIdentifier iterates children in reverse, so tbl[i] = handler seeded the INDEX variable's cell (polluting a same-named formal) and tbl[i](7) looked up the callee under i in a different scope — no join, no CALLS edge for the classic function-pointer-array dispatch (review finding 12, #2522). Subscript nodes now recurse into their container field only, in both bindingIdentifier and terminalIdentifier, across the fielded grammars (C/C++/JS/TS/Python/Go/Java). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): make cross-function file-scope callable bindings resolvable Two stacked gaps killed the canonical C callback-registration pattern (fp assigned in init(), called in run()) — the exact #2437 false-safe this PR exists to fix (review finding H1, #2522): 1. isVisibleValueBinding only consulted assignment regions and formals, so a call in a function OTHER than the assigning one emitted no invoke fact. A declared callable-typed binding is now a value binding wherever its declaration is visible (visibleCallableSignature). 2. The C scope query had no @declaration.variable pattern for function- pointer declarators — void (*fp)(int); created no scope-tree binding, so the seed (init) and invoke (run) cells canonicalized to different keys and never joined. Both bare and initialized forms now bind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(c): detect variadic parameters via the named variadic_parameter node tree-sitter-c materializes '...' as a named variadic_parameter node; the anonymous-token checks never matched, so variadic function-pointer signatures were emitted with a wrong fixed arity and no '...' sentinel (review finding, #2522). C++ is unaffected ('...' stays an anonymous token there); the token checks remain for such grammars. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): emit invoke facts for field-stored callable member calls The C ops-vtable pattern (o->run = handler; o->run(1)) captured the store but never the call — the member path in emitCallFacts bailed for languages without protocol methods, and the value-binding index recorded the member store under the OBJECT's name ('o'), not the member's ('run') (review finding 11/M3, #2522). Member destinations now also record their terminal member name, and a member call whose name-cell has a visible store emits an indirect invoke — gated on the store so plain accessor calls (map.get) stay inert. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): disambiguate (obj->*ptr)() ERROR recovery by token order tree-sitter-cpp groups the recovered '->*' two ways depending on error-recovery cost (identifier lengths): [identifier, ERROR '->*m'] or [ERROR 'obj->*', identifier]. The recovery assumed the first shape, so the second silently swapped receiver/member and dropped the call site — the committed test passed only by name luck (review finding H2, #2522). The identifier's position relative to '->*' inside the ERROR now decides roles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): class members are never file-local in hasFileLocalCallableLinkage The name-keyed file-local set is populated from every static declaration, so an in-class 'static void make();' (external linkage — in-class static means no-instance) and any member sharing a name with a static free function were over-marked, refusing legitimate cross-file declaration/definition joins (review finding 13/M2, #2522). Method and Constructor defs now bypass the name-set, per the hook's own linkage-only contract. Deviation from plan step 13: the regression is a unit-level contract pin rather than an end-to-end join test — C++ merges out-of-line member definitions onto the member node by qualified identity, so the graph shape cannot discriminate the join refusal for members. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): classify parameter passing mode from the declarator chain only A whole-subtree scan for reference_declarator inverted copy vs alias: void reg(void (*cb)(int& out)) marked the by-value pointer cb as 'reference' because of the NESTED parameter's int&, making the solver back-propagate formal targets into every caller's argument cell — alias semantics for a copy (review finding 14/M5, #2522). The chain walk never descends into nested parameter lists; a reference anywhere ON the chain (int& x, void (*&cb)(int)) still aliases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ruby): bare identifiers are calls, not callable references Ruby parses a receiver-less zero-arg method call identically to a variable read, so 'action = process' — which CALLS process and stores its return — seeded action with the callable and minted a wrong CALLS edge from any dispatch through it, confirmed end-to-end (review finding 15/HIGH, #2522). New provider knob bareNamesAreCalls: a bare name that is not a provably local value binding and not an explicit reference form (method(:x), lambda/proc) emits no flow fact, on both the assignment and argument paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(go): pair multi-value := positionally instead of cross-wiring The shared field fallback took the FIRST LHS identifier and the LAST RHS identifier of Go's expression_list pair, cross-wiring 'a, b := f, g' and synthesizing a garbage comma-joined qualified name — the real relationships were silently dropped (review finding 16, #2522). extractAssignment may now return multiple pairs; Go pairs list entries positionally and emits nothing for a length mismatch (multi-return call RHS). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(java): drop get/test from callableProtocolMethods 'get' and 'test' collide with ubiquitous non-functional-interface APIs (Map/List/Optional/Future.get), so every ordinary container access emitted a spurious callable-object invoke fact — high-volume misleading graph facts with a cross-wiring risk on receiver-name reuse (review finding 17, #2522). Supplier.get/Predicate.test dispatch is deliberately traded away until the check can gate on the receiver's declared type. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(rust): pin the qualified-name no-degrade guard as a hard invariant Rust's scoped_identifier callable-reference capture over-includes unit enum variants and associated constants (Shape::Square seeds as if callable); they stay edge-free only because resolveSeedCandidates refuses to degrade an unresolved qualified name to a simple-name lookup (review finding 18, #2522). Capture-side type filtering would false-negative on tuple-variant constructors, so the guard IS the contract: documented as a hard invariant (Go's mis-shaped multi-value forms also rely on it) and pinned end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(php): remove nonexistent optional_parameter node type tree-sitter-php has no 'optional_parameter' — defaults ride on simple_parameter — so the entry was dead weight the #1920 literal gate does not cover for capture-option Sets (review finding 19, #2522). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cobol): detect procedure pointers on fixed-format sources Two stacked defects made the feature a no-op on classic sequence-numbered fixed format (review finding 20/H3, #2522): 1. parseDataItemClauses' USAGE alternation knew POINTER but not PROCEDURE-POINTER/FUNCTION-POINTER, so the dataItems filter was dead. 2. The raw-line fallback scanned UNCLEANED text, where the sequence number satisfied the leading digits and the LEVEL NUMBER got captured as the pointer name. It now scans preprocessed lines and requires a letter- initial name (COBOL data names must contain a letter). 161 COBOL preprocessor/copy-expander tests stay green; free-format matrix case unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cobol): skip comment lines in SET seed/copy scans A commented-out SET (indicator-column '*'/'/' or free-format '*>') produced a live seed and a false CALLS edge from dead code (review finding 21/M1, #2522). The scan now skips indicator-column comment lines and strips inline '*>' tails before matching. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(architecture): document callable-flow-only mode and skipped-key reporting The Callable-value flow section omitted scopeResolutionEdgeMode: 'callable-flow-only' — a real emit-pipeline branch that suppresses all ordinary emission for standalone providers (review finding 22, #2522) — and predated the skipped-key names/stats surfacing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(scope-resolution): correct value-ref resolution attribution and stale pdg-gating comments The value-ref contract comment claimed MethodRegistry resolution — the mechanism is the post-finalize findCallableBindingInScope walker owned by emitPropertyDispatchCalls (resolveReferenceSites skips these sites). Three 'only under --pdg' calleeIdSink comments were falsified by the #2437 gating change (callee-id-sink.ts's header was updated; these copies were missed). Review finding 23, #2522. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(ingestion): direct unit coverage for synthesizeCallableFlowCaptures The 1,100-line shared synthesizer had no test naming it — only downstream consumers were covered (review finding 24, #2522). Pins seed/invoke/ formal/argument emission, subscript container binding, store-gated member invokes, produced-value guards, and the bareNamesAreCalls knob over a minimal options object so assertions target the synthesizer's own semantics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(resolvers): deepen shallow-language coverage; fix Kotlin/Swift reassignment gaps it exposed Adds the COBOL SET x TO y copy-branch scenario and conditional-assignment scenarios for Kotlin, C#, Swift, and Dart (10 languages previously had one generic case each — review finding 25, #2522). The new scenarios exposed two real capture gaps, fixed here: - tree-sitter-kotlin's 'assignment' node is fieldless, so nested reassignments (chosen = ::target inside a block) produced no flow facts; Kotlin's extractAssignment now decomposes it positionally. - tree-sitter-swift fields its assignment as target:/result:, neither in the shared fallback's field lists; both added. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(infra): literal-validation gate for callable-capture option Sets The #1920 gate validates query literals and exported configs but not the module-private *_CALLABLE_CAPTURE_OPTIONS Sets consumed by the shared synthesizer — a typo'd node type silently captures nothing (PHP shipped a dead 'optional_parameter'; review finding 26, #2522). Every <key>NodeTypes Set literal is now validated against its language's grammar; name-carrying sets (callableProtocolMethods, memberPointerOperators) are deliberately outside the contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(storage): centralize corrupt-fixture casts into makeStoreEntry The callable-flow store tests scattered 'as unknown as' double-casts per fixture (review finding 27, #2522; standing no-as-any rule). One typed helper now owns the single controlled escape hatch for building malformed serialization-boundary payloads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(bench): refresh capture fingerprints after review fixes python-scope: the committed baseline (8d5c3699) never matched this branch's code — CI's benchmarks arm was red on the PR head (review finding 2/HIGH, #2522); regenerated (a99e69ab), scaling 1.04 in budget. scope-capture: ruby/cpp/swift/java/kotlin drifted from the review-fix commits (bare-name suppression, passing modes + ->* recovery, assignment fields, protocol narrowing, positional assignment); all 14 languages re-verified PASS with ratios <= 1.18 against the 1.5 budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(docs): untrack docs/plans working documents docs/ is gitignored (local working docs); the plan files were force-added past the ignore. Untracked from the index only — they stay on disk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(golden): regenerate captures goldens after callable-flow review fixes The per-language digest guards (csharp/go/php/python/ruby/rust/swift) locked the pre-fix capture output; the review-fix series intentionally changed it — store-gated member invokes, subscript container binding, Ruby bare-name suppression, Swift assignment fields, positional pairing. Regenerated with UPDATE_GOLDEN=1; clean verification run 59/59; all other parity/golden guards (pipeline-graph, spring-route, python parity) pass untouched at 33/33. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): prototypes are callees, not callable value cells The cross-function visibility fix indexed EVERY signature-bearing declaration as a value binding — including plain function/method prototypes (void f(int);). Every call to a declared function then became an indirect invoke, and with emitCanonicalInvokeReference (C/C++) minted a free-call reference that resolved through the registry, bypassing the precise passes' two-phase/ambiguity/subobject suppression — eight phantom CALLS edges in the cpp resolver suite on CI. Only declarations whose binding identifier sits under a pointer/ parenthesized declarator (callable-typed variables like void (*fp)(int);) create value cells now. cpp resolver suite 331/331; callable-value-flow + C/C++ suites 181/181 (the cross-function fp regression still passes); cpp fingerprint rebaselined, both bench gates PASS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Gergő Magyar
|
a05b501102
|
fix(cli): make Claude skills discoverable (#2434)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
Gergő Magyar
|
c6445096eb
|
fix: stop Napi::Error SIGABRT on analyze — index C++ type lookups, terminate workers only at JS-safe points (#2432) (#2436)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Gitleaks / gitleaks (push) Has been cancelled
Publish / Classify release event (push) Has been cancelled
Scorecard / Scorecard analysis (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-cli) (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-web) (push) Has been cancelled
Publish / RC guard (marker + release-PR skip) (push) Has been cancelled
Publish / ci (push) Has been cancelled
Publish / Publish to npm (push) Has been cancelled
Publish / Build & Push RC Docker images (push) Has been cancelled
|
||
|
ChamHerry
|
fc6007e70b
|
feat(i18n): make web and CLI language-aware (#1748) | ||
|
1PLee
|
05ca80ea30
|
feat(ingestion): add thrift contracts impl (#1234) | ||
|
Ivan Uzun
|
46586a8319
|
fix(group): add configurable cross-link path exclusions to reduce false positives (#1093)
* fix(group): add configurable cross-link path exclusions to reduce false positives Add matching.exclude_links_paths and matching.exclude_links_param_only_paths to group.yaml config. These filter out noisy HTTP contracts (health checks, param-only catch-all routes) from cross-link matching while preserving them in the contract registry for documentation purposes. Defaults are empty/false for backward compatibility — no behavior change unless the operator explicitly configures exclusions. * fix(group): address review findings — filter unmatched, normalize trailing slash, add tests - Excluded contracts no longer inflate SyncResult.unmatched (isNoisy guard) - pathPart in buildNoisyContractFilter strips trailing slashes before comparison - 8 new unit tests for buildNoisyContractFilter covering all code paths - Config-parser test asserts defaults for new matching fields * fix(group): normalize configured exclusion paths and add root-path test - Strip trailing slashes from configured exclude_links_paths at Set-build time so root path '/' (which normalizes to '') matches correctly - Add test: exclude_links_paths: ['/'] suppresses http::GET::/ contracts - Add new matching fields as commented examples in fixture group.yaml (DoD §2.4) * docs(group): document exclude_links_paths and exclude_links_param_only_paths config fields Add JSDoc to MatchingConfig interface, update the microservices guide YAML example and field notes, and scaffold the new fields (commented out) in the group create template. |
||
|
ivkond
|
fb3bc7829e
|
docs(group): add gRPC microservices group guide (#906) (#994)
* docs(group): add gRPC microservices group guide (#906) Adds `docs/guides/microservices-grpc.md`, a walkthrough for using GitNexus across multiple repositories whose services communicate over gRPC. Covers the group mental model, per-repo `gitnexus analyze`, the `group.yaml` schema, `group sync`, inspecting `contracts.json`, running cross-repo `impact` with `@<group>` routing, the gRPC extractor's provider/consumer signals per language, the `config.links` manifest escape hatch, and a short troubleshooting list. Wires the new page from the group-mode note in AGENTS.md. Closes #906. Made-with: Cursor * docs(grpc-guide): drop hard line wraps, rely on editor soft wrap Made-with: Cursor |
||
|
ivkond
|
be4458b650 |
docs: add design spec and implementation plan for PR #626 HIGH fixes
Spec covers 4 HIGH-priority issues from review: path traversal via group name, gRPC proto regex nested braces, service boundary detector directory exclusions, double-close of LadybugDB pools. Plan: 6 tasks with TDD, ordered by complexity. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
d2cd0b676f
|
feat: add COBOL language support with regex extraction pipeline (#498)
* feat: add COBOL language support with regex extraction pipeline Standalone COBOL processor following the markdown-processor.ts pattern: - No LanguageProvider modification — COBOL uses regex, not tree-sitter - No SupportedLanguages enum change — standalone processor pattern New files: - cobol-processor.ts — orchestrator (processCobol, isCobolFile, isJclFile) - cobol/cobol-preprocessor.ts — regex state machine extraction (~888 LOC) - cobol/cobol-copy-expander.ts — COPY statement expansion with circular detection - cobol/jcl-parser.ts — JCL job/step/DD extraction - cobol/jcl-processor.ts — JCL graph node creation Extraction produces: - Module nodes (PROGRAM-ID) - Function nodes (paragraphs) - Namespace nodes (sections) - Property nodes (data items) - CALLS edges (PERFORM intra-file, CALL cross-program) - IMPORTS edges (COPY statements) - CONTAINS edges (section → paragraph hierarchy) Pipeline integration: single processCobol() call in Phase 2.6 54 new tests (33 COBOL + 21 JCL), all 3889 tests pass. * docs: document custom processor pattern in pipeline.ts Add comment block at the custom processor integration point documenting the pattern for future non-tree-sitter language additions. * feat(cobol): enrich graph with EXEC SQL/CICS, ENTRY points, MOVE data flow, PERFORM THRU Maps the remaining 60% of CobolRegexResults to the graph: - EXEC SQL blocks → CodeElement nodes + ACCESSES edges to DB tables - EXEC CICS LINK/XCTL → CodeElement nodes + cross-program CALLS edges - ENTRY points → Constructor nodes (registered for cross-program resolution) - MOVE statements → ACCESSES edges (read/write data flow tracking) - PERFORM THRU → expanded CALLS edges for range targets - File declarations → Record nodes with assignment metadata - Cross-program CALL 2nd pass: resolves unresolved targets after all programs processed * test(cobol): add 26 integration tests with exact assertions + fix CICS resolution bug Integration tests (test/integration/resolvers/cobol.test.ts): - 26 tests covering full COBOL system extraction - ALL assertions use exact toBe(N) — zero fuzzy assertions - Fixtures: CUSTUPDT.cbl, AUDITLOG.cbl, CUSTDAT.cpy, RPTGEN.cbl, RUNJOBS.jcl Bug fix (cobol-processor.ts): - CICS LINK/XCTL cross-program resolution was broken — edges were created with "resolved" reason but pointing to <unresolved> targets - Fix: use cics-link-unresolved / cics-xctl-unresolved suffix pattern matching the existing cobol-call-unresolved pattern - Second-pass resolver now patches both CALL and CICS unresolved edges All 3915 tests pass, 0 failures. * test(cobol): exhaustive 57-test suite with strict exact assertions Complete rewrite of COBOL integration tests using ground-truth approach: dump the full graph, then assert EVERY node and EVERY edge. 57 tests across 9 sections: - Node completeness: Module(3), Function(13), Namespace(2), Property(21), Record(1), CodeElement(8), Constructor(1) — exact sorted arrays - Edge completeness: 22 tests covering every type+reason combination with exact source→target pairs - Cross-program resolution: 6 tests verifying CALL, CICS LINK/XCTL, JCL - COPY expansion: copybook data items in RPTGEN - Section hierarchy: exact paragraph membership per section - Data item ownership: exact per-module breakdown - MOVE data flow: exact read/write pairs - JCL integration: job/step/dataset containment - Grand totals: CALLS(22), CONTAINS(48), IMPORTS(1), ACCESSES(7) Fixture enhancements: - CUSTUPDT.cbl: added INIT-SECTION + PROCESSING-SECTION, PERFORM THRU - AUDITLOG.cbl: added ENTRY "AUDITLOG-BATCH" - RPTGEN.cbl: added EXEC CICS XCTL Zero fuzzy assertions — every expect uses toBe(N) or toEqual([...sorted]). * fix(cobol): add removeRelationship API + single-quote CALL/COPY/ENTRY, PERFORM keyword skip Phase 0A: Add removeRelationship(id) to KnowledgeGraph interface and implementation (trivial Map.delete wrapper). Required for orphan edge cleanup in next commit. Phase 1A (from PR #500 review, modified): - RE_CALL and RE_COPY_QUOTED now match both "double" and 'single' quotes - parseSingleCopyStatement in copy-expander updated for single quotes - PERFORM_KEYWORD_SKIP set prevents UNTIL/VARYING/WITH/TEST/FOREVER from being stored as false-positive perform targets - Sequence number stripping uses /[^0-9 ]/ (preserves numeric seq numbers unlike PR #500's /\S/ which stripped them) - Normalized || to ?? for regex group extraction in copy-expander 5 new graph unit tests, all 57 COBOL integration tests pass. * fix(cobol): RE_ENTRY single-quote + remove orphan unresolved CALLS edges Phase 1B: RE_ENTRY regex now supports both "double" and 'single' quoted ENTRY targets. Uses named intermediates (entryName, usingClause) with ?? operator. USING capture group shifted from [2] to [3]. Phase 1C: Second-pass resolution now collects resolved orphan edge IDs during iteration and removes them after the loop completes, using the new graph.removeRelationship() API. Graph no longer contains phantom <unresolved>: edges alongside their resolved replacements. CALLS count drops from 22 to 18 (4 orphan edges removed). * fix(cobol): Property ID collisions + O(1) Map lookup for MOVE edges Phase 1D+3C (atomic): Property node IDs now use composite key filePath:section:level:name instead of filePath:name. This prevents duplicate data item names in different sections (e.g., STATUS in both WORKING-STORAGE and LINKAGE) from silently colliding. New generatePropertyId() helper ensures both node creation and MOVE edge lookup use the identical key formula. buildDataItemMap() replaces the O(n) findDataItemNode linear scan with O(1) Map lookup, built once per file before MOVE processing. * feat(cobol): MOVE multi-target extraction with OF/IN qualifier filtering MOVE X TO A B C now produces write edges for all targets, not just the first. extractMoveTargets() helper handles OF/IN qualified names (WS-NAME OF WS-RECORD -> target is WS-NAME), subscript stripping (WS-TABLE(I) -> WS-TABLE), and MOVE_SKIP filtering on targets. Data model: CobolRegexResults.moves.to:string -> targets:string[] MOVE CORRESPONDING stays single-target per COBOL standard. Processor MOVE loop now iterates move.targets. * feat(cobol): COPY IN/OF library, pseudotext REPLACING, dynamic CALL, PERFORM TIMES, CICS MAP unquoted Phase 2B: COPY ... IN/OF library-name now captured as metadata in CopyResolution (IN and OF are synonyms per COBOL-85 standard). Phase 2C: COPY REPLACING ==pseudotext== support. Tokenizer handles ==...== delimiters alongside "quoted" strings. Pseudotext forces EXACT type. Two-pass applyReplacing: first pass handles space-containing/ non-identifier pseudotext via global string replace; second pass handles identifier-level LEADING/TRAILING/EXACT. New test file cobol-copy-expander.test.ts with 10 tests. Phase 2E: PERFORM WS-COUNT TIMES no longer produces a false-positive perform target (checks for TIMES keyword after captured identifier). Phase 2F: Dynamic CALL via data item (CALL WS-PROG-NAME without quotes) now emits a CodeElement annotation node with description 'dynamic-call' instead of silently ignoring. Adds isQuoted:boolean to call results. Phase 3A: CICS MAP(WS-MAP-NAME) unquoted identifiers now captured. Phase 3B: Normalized || to ?? in copy-expander (done in Phase 1A). * feat(cobol): nested program support — capture multiple PROGRAM-IDs per file Phase 2D: The state machine now captures all PROGRAM-IDs, not just the first. The primary program name stays in programName; additional nested programs go into nestedPrograms[]. The processor creates separate Module nodes for each nested program, contained by the outer module, and registers them in moduleNodeIds for cross-program CALL resolution. Paragraphs/data items are not yet scoped per-program (attributed to the outer module) — full per-program scoping is a future enhancement that requires END PROGRAM boundary tracking in the state machine. * test(cobol): expand integration tests for all new language features New fixtures: - NESTED.cbl — two PROGRAM-IDs (OUTER-PROG, INNER-PROG) for nested program support testing - COPYLIB.cpy — copybook for pseudotext REPLACING test target Modified fixtures: - CUSTUPDT.cbl — single-quoted ENTRY 'ALTENTRY', multi-target MOVE (WS-AMT TO FIELD-A FIELD-B), dynamic CALL WS-PROG-NAME, COPY COPYLIB with pseudotext REPLACING, LINKAGE SECTION with LS-PARAM - RPTGEN.cbl — PERFORM WS-COUNT TIMES (false-positive guard), unquoted MAP(WS-MAP-NAME), additional data items WS-COUNT WS-MAP-NAME Integration test rewritten with 62 exact assertions covering: - 5 Module, 17 Function, 33 Property, 9 CodeElement, 2 Constructor nodes - Nested program containment (OUTER-PROG -> INNER-PROG) - Dynamic CALL annotation (CodeElement with cobol-dynamic-call) - Multi-target MOVE (UPDATE-BALANCE: 2 reads, 3 writes) - Single-quoted ENTRY (ALTENTRY under CUSTUPDT) - PERFORM TIMES guard (WS-COUNT not in CALLS) - Orphan unresolved edge removal (zero -unresolved edges) - Grand totals: 21 CALLS, 68 CONTAINS, 2 IMPORTS, 10 ACCESSES * fix(cobol): pseudotext REPLACING now applies correctly via isPseudotext flag Root cause: ==PREFIX-== matched /^[A-Z][A-Z0-9-]*$/i (trailing hyphens allowed), routing it to the second-pass EXACT identifier match where PREFIX-RECORD !== PREFIX- failed silently. Fix: Propagate isPseudotext from parseReplacingClause to CopyReplacing interface, then use it in applyReplacing first-pass condition to force global string replacement for all pseudotext entries regardless of whether the content looks like an identifier. Result: COPY COPYLIB REPLACING ==PREFIX-== BY ==WS-==. now correctly transforms PREFIX-RECORD → WS-RECORD, PREFIX-CODE → WS-CODE, etc. * refactor(cobol): per-program scoping via boundary tracking + line-range grouping State machine changes (minimal, ~30 lines): - Add RE_END_PROGRAM regex for END PROGRAM program-name. detection - Replace nestedPrograms[] with programs[] containing startLine/endLine/ nestingDepth metadata for each PROGRAM-ID in the file - Reset division/section/paragraph state on new PROGRAM-ID boundary - EOF finalization flushes remaining stack entries (single-program files) - Programs sorted by startLine (outer before inner) Processor changes: - Uses programs[] with line-range containment to find enclosing parent Module for nested programs (replaces hardcoded nestedParent logic) - programModuleIds Map tracks Module node IDs per program name Fixture: NESTED.cbl now includes END PROGRAM lines for both programs. Integration test: PREFIX-* Property nodes now correctly appear as WS-* after the pseudotext REPLACING fix from the previous commit. * feat(cobol): free-format COBOL support (>>source free) Auto-detects >>SOURCE FREE directive in the first 500 chars and switches to free-format line processing: - No column-position rules (cols 1-6 are program text, not sequence area) - Comments use *> prefix instead of col 7 indicator - No continuation line indicator - Strip inline *> comments - Skip >>SOURCE directive lines preprocessCobolSource() skips col-1-6 stripping for free-format files. Paragraph/section regexes relaxed from fixed 7-space prefix to flexible whitespace with case-insensitivity (/^\s*([A-Z][A-Z0-9-]+)\.\s*$/i). EXCLUDED_PARA_NAMES expanded with COBOL verbs (GOBACK, END-READ, etc.) to prevent false-positive paragraph detection in free-format. Also fixes: entry-point-scoring.ts crash when language is 'cobol' (MERGED_ENTRY_POINT_PATTERNS[language] was undefined → optional chaining). Benchmark on ACAS 3.01 (268 GnuCOBOL free-format programs, 10MB): - Before: 407 nodes, 393 edges (near-empty, only file nodes) - After: 4,297 nodes, 3,612 edges, 542 clusters, 11 flows * fix(cobol): relax data item regexes for free-format (^\s+ to ^\s*) RE_FD, RE_DATA_ITEM, RE_ANONYMOUS_REDEFINES, and RE_88_LEVEL all used ^\s+ which requires at least 1 leading space. In free-format mode, lines are trimmed before processing, so data items like "01 WS-FIELD PIC X." have no leading whitespace after trimming. Changed to ^\s* (zero or more spaces) which works for both fixed-format (indented lines still have spaces) and free-format (trimmed lines). ACAS benchmark (268 GnuCOBOL programs): - Before: 4,297 nodes, 3,612 edges (paragraphs only) - After: 13,832 nodes, 8,615 edges (+ data items, FDs, 88-levels) * feat(cobol): 100% structural feature coverage — GO TO, SCREEN, SD/RD, SORT, SEARCH, CANCEL, Level 66 New extractions: GO TO (CALLS edges), SCREEN SECTION data items, SD/RD alongside FD (Record nodes), SORT/MERGE USING/GIVING (ACCESSES), SEARCH (ACCESSES), CANCEL (CALLS), Level 66 RENAMES (Property), IS EXTERNAL/IS GLOBAL (Property description enrichment). ACAS: 13,951 nodes | 13,193 edges | 685 clusters | 150 flows (+53% edges from new GO TO/SORT/SEARCH/CANCEL extractions) * feat(cobol): enriched CICS extraction — file I/O, dynamic PROGRAM, queues, HANDLE ABEND EXEC CICS blocks now extract: - FILE/DATASET clause: captures VSAM file name (literal or data item ref) for READ/WRITE/REWRITE/DELETE/STARTBR/READNEXT/READPREV → ACCESSES edges - PROGRAM clause: now handles unquoted variable references (dynamic CICS program transfer) → CodeElement annotation with cics-dynamic-program reason - QUEUE clause: captures TS/TD queue names from WRITEQ/READQ → ACCESSES edges - LABEL clause: captures HANDLE ABEND error handler targets → CALLS edges - TRANSID: now handles unquoted variable references CodeElement descriptions enriched with all captured fields (map, program, transid, file, queue, label). CardDemo benchmark: +49 nodes, +33 edges from enriched CICS extraction. * feat(cobol): complete CICS command extraction — all 7 expert recommendations From COBOL expert agent analysis: 1. ENDBR added to isRead file command list 2. LOAD added to PROGRAM edge commands (alongside LINK/XCTL) 3. Two-word commands expanded: WRITEQ/READQ/DELETEQ TS/TD, HANDLE ABEND/AID/CONDITION, START TRANSID 4. Queue reason differentiated: cics-queue-read/-write/-delete 5. RETURN/START TRANSID → CALLS edges to synthetic <transid> target 6. MAP → ACCESSES edges for screen traceability 7. INTO/FROM data fields extracted → ACCESSES edges to data items Also: dataItemMap built before CICS block processing (was declared after), CodeElement descriptions enriched with all captured CICS fields. * test(cobol): strict exhaustive integration tests with exact edgeSet assertions Every edge reason has exact sorted pair assertions via edgeSet(), not just counts. Any change to extraction that adds, removes, or reorders edges will produce a precise, descriptive failure. Updated RPTGEN.cbl fixture with: - GO TO EXIT-PARAGRAPH, SORT USING/GIVING, SEARCH table - EXEC CICS READ FILE INTO, WRITEQ TS QUEUE FROM, SEND MAP FROM - EXEC CICS HANDLE ABEND LABEL, RETURN TRANSID, XCTL PROGRAM(variable) - ABEND-HANDLER and EXIT-PARAGRAPH paragraphs 46 tests covering 24 CALLS + 79 CONTAINS + 18 ACCESSES + 2 IMPORTS edges across 15 distinct edge reason codes, all with exact sorted pair lists. * fix(cobol): address 5 findings from second Claude review (compiler front-end perspective) Finding #2: Numeric sequence numbers now stripped (changed /[^0-9 ]/ to /\S/ in preprocessCobolSource). Lines like "000100 MAIN-PARAGRAPH." now have cols 1-6 blanked so paragraph regex matches correctly. Finding #11: JCL in-stream PROC ordering fixed — pre-register all PROCs into moduleNames before step processing. Steps that EXEC a PROC defined later in the same file now get CALLS edges. Finding #A: PROCEDURE DIVISION USING no longer captures calling-convention keywords (BY, VALUE, REFERENCE, CONTENT, ADDRESS, OF) as parameter names. Finding #C: SORT/MERGE USING/GIVING now captures ALL file references (multi-file), not just the first. Changed from single-match to section extraction with split. Finding #D: Section headers no longer set currentParagraph, preventing PERFORM caller misattribution to Namespace instead of Function nodes. * fix(cobol): address code review findings — ReDoS fix, perf, cleanup P1 CRITICAL — ReDoS in SORT USING/GIVING: Replaced nested-quantifier regex with safe indexOf+substring+split approach. No backtracking possible on crafted input. P2 — readCopy O(M) linear scan: Added copybookByPath reverse Map for O(1) path-to-content lookup. P3 — Dead code removal: Deleted unused RE_SORT_USING and RE_SORT_GIVING constants. P3 — EXCLUDED_PARA_NAMES simplification: Replaced 20 END-* entries with startsWith('END-') prefix check. Auto-covers future END-* verbs. P3 — Misplaced JSDoc on removeRelationship: Fixed comment that described removeNodesByFile instead. Added missing JSDoc to removeNodesByFile. Review agents: architecture-strategist, performance-oracle, security-sentinel, code-simplicity-reviewer. * refactor: add Cobol to SupportedLanguages with parseStrategy: standalone New languages/cobol.ts — standalone regex processor provider with no-op tree-sitter fields. Declares parseStrategy: 'standalone' to distinguish from tree-sitter-based languages. Added parseStrategy: 'tree-sitter' | 'standalone' to LanguageProviderConfig for languages that use their own processor instead of tree-sitter. Removed all 11 'cobol' as any casts — now uses SupportedLanguages.Cobol. Added empty Cobol entries to entry-point-scoring and framework-detection. * fix(cobol): 5 fixes from third Claude review + 3 regression tests Fixes: - Line numbers now 1-indexed in fixed-format (was 0-indexed, off-by-one in jump-to-definition links) - Copybook content preprocessed before COPY expansion (sequence numbers and patch markers in copybooks no longer survive into expanded source) - ENTRY USING filters calling-convention keywords (BY, VALUE, REFERENCE, CONTENT, ADDRESS, OF) — same fix as PROCEDURE DIVISION USING - SORT/MERGE trailing period stripped from USING/GIVING file tokens - Paragraph exclusion uses exact match for SECTION/DIVISION (was substring match that excluded valid names like CROSS-SECTION-ANALYSIS) USING_KEYWORDS moved to module scope for reuse by both PROCEDURE DIVISION USING and ENTRY USING handlers. New unit tests: - ENTRY USING BY VALUE filtering - Paragraph names containing SECTION not excluded - Numeric sequence numbers stripped enabling paragraph detection * fix(cobol): address 6 findings from fourth Claude review + tests Fourth review findings fixed: - New #IV: PERFORM TIMES guard uses perfMatch.index instead of line.indexOf (prevents wrong match when target appears earlier in line) - New #V: 88-level condition values now handle single-quoted literals ('Y' no longer stored with embedded quotes) - New #I: CANCEL edges use two-pass resolution like CALL (no longer silently dropped when target indexed after source) - New #3: Multi-line SORT/MERGE accumulation — sortAccum state variable accumulates lines until period, then extracts USING/GIVING from full statement (95% of production SORT statements span multiple lines) - New #II: PROCEDURE DIVISION USING on split lines — pendingProcUsing flag defers parameter capture to next line if USING not on same line - New #6 (prior): EXCLUDED_PARA_NAMES exact match for SECTION/DIVISION Updated fixture: RPTGEN.cbl SORT now uses multi-line format with GIVING on separate line (period-terminated). New sort-giving integration test. ACCESSES total: 18 → 19 (new sort-giving edge from multi-line capture). * fix(cobol): address 4 findings from fifth Claude review Finding #B (5 reviews old): Section/paragraph node IDs now include enclosing program name to prevent collision when nested programs share section/paragraph names. New findOwningProgramName() helper uses programs[] line ranges to find the innermost enclosing program. Finding #α: pendingProcUsing now reset in the if(procUsingMatch) branch (was only set in else branch, could leak across nested programs). Finding #β: RE_CALL_DYNAMIC uses negative lookbehind (?<![A-Z0-9-]) to prevent false-positive on compound identifiers like WS-CALL OCCURS. Finding #γ: sortAccum flushed at EOF (parallel to flushSelect and pendingFdName EOF cleanup). Prevents silent loss of SORT USING/GIVING relationships in truncated files. * fix(cobol): address findings from reviews 5+6 with full test coverage Review 5 fixes: - #α: pendingProcUsing reset in if(procUsingMatch) branch - #β: RE_CALL_DYNAMIC negative lookbehind prevents WS-CALL false positive - #γ: sortAccum flushed at EOF for truncated files - #B: Section/paragraph IDs include owning program name Review 6 fixes: - #P: sectionNodeIds/paraNodeIds maps use program-scoped keys (PROGNAME:NAME). New scopedParaLookup/scopedCallerLookup helpers. findContainingSection updated with programs parameter. - #Q: RETURNING added to USING_KEYWORDS for COBOL 2002+ - #R: RE_PERFORM matches both THRU and THROUGH via alternation New unit tests (6): - PERFORM THROUGH captures thruTarget - PROCEDURE DIVISION USING RETURNING filters keyword - RE_CALL_DYNAMIC no false-match on WS-CALL compound identifier - Multi-line SORT captures USING/GIVING from continuation lines - PROCEDURE DIVISION USING on split line via pendingProcUsing - Copybook preprocessing strips sequence numbers * fix(cobol): address findings from seventh Claude review + 3 tests Review 7 fixes: - #i: findContainingSection only updates best when lookup succeeds (prevents undefined overwriting valid parent section) - #ii: RE_PROC_SECTION handles segment numbers (SECTION 30.) - #III: procedureUsing now stored per-program on boundary stack entries, propagated to programs[] output. Inner programs no longer overwrite outer program's parameters. - #δ: Dynamic CANCEL (CANCEL variable) now creates CodeElement annotation node, matching dynamic CALL behavior. RE_CANCEL_DYNAMIC with negative lookbehind. cancels[] gains isQuoted field. - #Q: RETURNING added to USING_KEYWORDS (already in prev commit) - #R: PERFORM THROUGH already fixed (THRU|THROUGH alternation) New unit tests: - Nested programs carry per-program procedureUsing - SECTION with segment number detected - Dynamic CANCEL via data item captured with isQuoted=false * feat(cobol): link PROCEDURE DIVISION USING to LINKAGE data items + close 4 findings Finding #10 FIXED: procedureUsing parameters now create ACCESSES edges with reason 'cobol-procedure-using' from Module to matching LINKAGE SECTION Property nodes. This exposes the program's parameter contract in the graph (e.g., AUDITLOG → LS-CUST-ID, AUDITLOG → LS-AMOUNT). Findings closed by expert agent consensus: - #6 COPY IN library: WONTFIX — captured metadata, no universal library-to-directory mapping exists. Field costs nothing and is useful for library queries. - #14 SQL DELETE: WONTFIX — DB2 requires FROM; existing FROM pattern handles it. Bare DELETE would risk false positives. - #E OCCURS DEPENDING ON: WONTFIX — runtime sizing concern, not structural. The static occurs count is sufficient for indexing. All 39 findings from 7 Claude reviews now resolved or closed. * fix(cobol): resolve 48 review findings across 9 review cycles Ninth deep review resolved all remaining COBOL parser gaps identified by 5 specialist agents (COBOL expert, architecture strategist, TypeScript reviewer, security sentinel, code simplicity reviewer). Fixes (P1 — critical): - SELECT OPTIONAL now correctly skips OPTIONAL keyword (C1) - RETURNING params excluded from PROCEDURE DIVISION USING list (C7) - SORT GIVING no longer captures clause keywords as file names (C5) - Extract flushSort() helper eliminating 40-line duplication (S2) - Flush unclosed EXEC blocks at EOF matching SORT/SELECT pattern (S3) - Guard undefined map key in jcl-processor moduleNames (S1) - Add MAX_TOTAL_EXPANSIONS=500 to prevent exponential COPY breadth (S4) Fixes (P2 — important): - Quote-aware stripInlineComment for | and *> in string literals (C2+C3) - Fixed-format literal continuation now handles quoted strings (C6) - PROGRAM-ID detected regardless of division state for siblings (C9) Fixes (P3 — cleanup): - EXEC SQL INTO restricted to INSERT INTO to avoid FETCH false-pos (C8) - Copy expander line numbers fixed from 0-based to 1-based (C11) - Remove dead code: inInStreamProc, fileIsLiteral, expansionDepth (S7-S10) Also fixes 8th-review findings: nested program CONTAINS attribution, multi-PERFORM on same line, INPUT/OUTPUT PROCEDURE IS in SORT, GO TO DEPENDING ON multi-target, MOVE CORR abbreviation, per-program procedureUsing ACCESSES edges. Tests: 145 COBOL tests passing (59 integration + 86 unit) Benchmarks: CardDemo 12,323 nodes/8,893 edges (7.4s) ACAS 14,016 nodes/15,452 edges (9.3s, -9% faster) * docs(cobol): update documentation for ninth review cycle fixes Update all 4 COBOL documentation files to reflect the 16 fixes from the ninth review cycle: - regex-extraction.md: quote-aware comment stripping, SELECT OPTIONAL, RETURNING exclusion, SORT_CLAUSE_NOISE filter, flushSort() helper, GO TO multi-target, PROGRAM-ID division-independent detection - copy-expansion.md: MAX_TOTAL_EXPANSIONS=500 breadth guard, 1-based line numbers, removed expansionDepth/warnedCircular param - deep-indexing.md: GO TO DEPENDING ON, INPUT/OUTPUT PROCEDURE IS, MOVE CORR edge reasons, INSERT INTO restriction, literal continuation - performance.md: updated benchmarks (CardDemo 12,323n/8,893e/7.4s, ACAS 14,016n/15,452e/9.3s), COPY breadth guard * fix(cobol): resolve 10th review findings — nested program edge attribution Fix 6 findings from the 10th review (PR #498 comment #4132201110): #A+#F: All CALL/CANCEL/CICS/ENTRY/SQL/SEARCH/file-declaration edges now use owningModuleId() for nested program attribution instead of the outer program's parentId. Added helper function owningModuleId() to centralize the pattern. #B: Added USING and GIVING to SORT_CLAUSE_NOISE set to prevent MERGE USING + OUTPUT PROCEDURE from capturing clause keywords as file names. #C: INPUT/OUTPUT PROCEDURE regex now captures optional THRU/THROUGH range end paragraph, mirroring RE_PERFORM's THRU support. #D: scopedCallerLookup fallback now uses programModuleIds.get(pgm) instead of parentId, so PERFORM/MOVE/GOTO in nested programs with unresolvable paragraphs fall back to the correct inner module. #E: pendingProcUsing only set when PROCEDURE DIVISION line is NOT period-terminated, preventing false USING expectation. Tests: 145 passing | TypeScript clean * fix(cobol): resolve 10th review findings — nested program edge attribution Fix 6 findings from the 10th review (PR #498 comment #4132201110): #A+#F: All CALL/CANCEL/CICS/ENTRY/SQL/SEARCH/file-declaration edges now use owningModuleId() for nested program attribution instead of the outer program's parentId. Added helper function owningModuleId() to centralize the pattern. #B: Added USING and GIVING to SORT_CLAUSE_NOISE set to prevent MERGE USING + OUTPUT PROCEDURE from capturing clause keywords as file names. #C: INPUT/OUTPUT PROCEDURE regex now captures optional THRU/THROUGH range end paragraph, mirroring RE_PERFORM's THRU support. #D: scopedCallerLookup fallback now uses programModuleIds.get(pgm) instead of parentId, so PERFORM/MOVE/GOTO in nested programs with unresolvable paragraphs fall back to the correct inner module. #E: pendingProcUsing only set when PROCEDURE DIVISION line is NOT period-terminated, preventing false USING expectation. Tests: 145 passing | TypeScript clean * fix(cobol): resolve 11th review findings — final nested program + multi-CALL gaps #1: scopedCallerLookup(null) now uses owningModuleId(lineNum) instead of parentId, fixing PERFORM/MOVE/GOTO before first paragraph in nested programs. #2+#3: CALL and CANCEL extraction now uses matchAll (global flag) to capture multiple occurrences on the same line. Dynamic CALL/CANCEL checked independently instead of in else branch. #4: SORT/MERGE ACCESSES edge IDs now use owningModuleId(sort.line) instead of parentId for nested program correctness. #5: preprocessCobolSource free-format detection now uses first 10 lines (consistent with extractCobolSymbolsWithRegex threshold). #6: EXCLUDED_PARA_NAMES expanded with DISPLAY, ACCEPT, WRITE, READ, REWRITE, DELETE, OPEN, CLOSE, RETURN, RELEASE, SORT, MERGE to prevent false-positive paragraph detection on isolated verbs. Also removed unused GraphNode import from cobol-processor.ts. Tests: 145 passing | TypeScript clean * docs(cobol): deepened full language coverage plan with research findings 3 research agents analyzed Phase 1-2 features and graph value ranking. Key findings: cobol-call-using is #1 edge type (9.2/10); multi-line accumulation is dominant challenge; DECLARATIVES is lowest-risk Phase 2 item; SET TO TRUE covers 80-90% of SET usage. * feat(cobol): implement Phase 1 — high-value data flow edges 4 new extraction features that create new ACCESSES and IMPORTS edges: 1.1: EXEC SQL INCLUDE -> IMPORTS edges with reason 'sql-include' Handles unquoted (SQLCA), quoted ('DBRMLIB.MEMBER'), and underscored (CUST_TBL_DCL) member names. 1.2: CALL USING parameter extraction -> ACCESSES edges Extracts parameters from CALL USING clause, filtering BY/REFERENCE/ CONTENT/VALUE/ADDRESS/OF/LENGTH/OMITTED keywords. Creates 'cobol-call-using' ACCESSES edges (graph value: 9.2/10). 1.4: OCCURS DEPENDING ON -> ACCESSES edges with reason 'cobol-depends-on' Extended OCCURS regex captures DEPENDING ON field with subscript stripping. Creates dependency edge from table to controlling field. 1.5: VALUE clause for standard data items Extracts VALUE from data item clauses: quoted strings with type prefix (X/N/G/B), ALL literals, numerics (incl negative/decimal), and figurative constants. Populates Property node values. Tests: 145 passing (+2 ACCESSES from CALL USING) | TypeScript clean * feat(cobol): implement Phase 2 — DECLARATIVES, SET, INSPECT, EXEC DLI 4 new extraction features for error handling, data flow, and IMS/DB: 2.1: EXEC DLI (IMS/DB) -> CodeElement + ACCESSES edges Accumulates EXEC DLI blocks like EXEC SQL. Parses DLI verbs (GU, GN, ISRT, REPL, DLET, CHKP, SCHD, TERM). Extracts SEGMENT, PCB, INTO/FROM, PSB. Creates dli-{verb} ACCESSES edges to <ims>:segment Record nodes. 2.2: DECLARATIVES / USE AFTER EXCEPTION -> ACCESSES edges Tracks inDeclaratives state. Detects USE AFTER STANDARD EXCEPTION ON file-name. Creates cobol-error-handler ACCESSES edge from handler section to file Record. 2.3: SET statement -> ACCESSES edges Detects SET TO TRUE (80-90% of SET usage) and SET index TO/UP BY/DOWN BY. Creates cobol-set-condition / cobol-set-index write edges + cobol-set-read for identifier values. 2.4: INSPECT -> ACCESSES edges with multi-line accumulator Accumulates INSPECT until period (like SORT). Extracts inspected field + tally counters. Creates cobol-inspect-read/write/tally edges. Form detection: tallying/replacing/converting/combined. Preprocessor: 1398 -> 1597 LOC (+199). Tests: 145 passing. * feat(cobol): implement Phase 3 — completeness fixes 6 partial features fixed to first-class support: 3.1: CALL RETURNING -> ACCESSES write edge (cobol-call-returning) 3.2: SELECT OPTIONAL flag preserved in FileDeclaration + Record node 3.3: ALTERNATE RECORD KEY extraction (matchAll for multiple keys) 3.4: COMMON attribute on nested programs (RE_PROGRAM_ID extended) 3.5: IS EXTERNAL / IS GLOBAL as first-class boolean properties (removed usage string hack) 3.6: AUTHOR / DATE-WRITTEN mapped to Module node description Tests: 145 passing | TypeScript clean * feat(cobol): implement Phase 4 — INITIALIZE + metadata completeness 4.1: INITIALIZE statement -> ACCESSES write edge (cobol-initialize) 4.2: DATE-COMPILED and INSTALLATION paragraphs extracted and mapped to Module node description alongside existing AUTHOR/DATE-WRITTEN All 4 plan phases complete. Coverage: ~95% (up from 71.9%). Tests: 145 passing | TypeScript clean * test(cobol): add 24 unit tests for Phase 1-4 features Coverage for all new extraction features: Phase 1 (8 tests): - EXEC SQL INCLUDE (unquoted, quoted, underscored) - CALL USING (simple, mixed modes, ADDRESS OF, OMITTED) - CALL RETURNING - OCCURS DEPENDING ON - VALUE clause (string, numeric, figurative constant) Phase 2 (10 tests): - EXEC DLI GU/ISRT/SCHD (verb, segment, PCB, INTO, FROM, PSB) - DECLARATIVES USE AFTER EXCEPTION (single + multiple sections) - SET TO TRUE, SET index UP BY - INSPECT TALLYING, INSPECT REPLACING Phase 3-4 (6 tests): - SELECT OPTIONAL flag - ALTERNATE RECORD KEY - PROGRAM-ID IS COMMON - IS EXTERNAL / IS GLOBAL booleans - INITIALIZE extraction - Full programMetadata (AUTHOR, DATE-WRITTEN, DATE-COMPILED, INSTALLATION) Total: 168 tests passing (145 + 24 - 1 removed duplicate) * fix(cobol): use /\r?\n/ split for Windows CRLF compatibility All 4 COBOL source files now split on /\r?\n/ instead of '\n' to handle CRLF line endings on Windows. Previously, trailing \r in lines caused RE_GOTO's $ anchor to fail on multi-line GO TO DEPENDING ON statements, producing only 1 goto edge instead of 4. Files fixed: cobol-preprocessor.ts (2 sites), cobol-processor.ts, jcl-parser.ts, cobol-copy-expander.ts Tests: 168 passing | TypeScript clean * fix(cobol): resolve 12th review — dynamic CALL/CANCEL dedup + trailing anchors #1+#2: Removed incorrect hasQuotedCall/hasQuotedCancel deduplication guards. RE_CALL_DYNAMIC and RE_CANCEL_DYNAMIC require [A-Z] after CALL/CANCEL, so they CANNOT match quoted targets — the guards were both unnecessary and actively harmful, suppressing dynamic CALL/CANCEL in ON EXCEPTION patterns. #3+#5: Changed RE_CALL_DYNAMIC and RE_CANCEL_DYNAMIC trailing anchor from (?:\s|\.) to (?=\s|\.|$) (lookahead). The consuming anchor failed when the identifier was the last token on a physical line. Tests: 168 passing | TypeScript clean * feat(cobol): add CALL accumulator + fix SORT double-statement (#4, #6) Finding #4: Multi-line CALL USING accumulator Added callAccum state variable that accumulates CALL statements spanning multiple physical lines until period or END-CALL is found. Uses flushCallAccum() to re-extract CALL target + USING parameters from the full accumulated statement. This fixes the silent loss of ACCESSES parameter edges when USING appears on lines after CALL. Finding #6: SORT double-statement on same line After flushSort(), the code now falls through to re-check the current line for a new SORT/MERGE start (was previously blocked by the sortAccum === null check evaluating before flushSort ran). Also fixed: used non-global regex for CALL detection test to avoid the classic global regex .test() lastIndex bug. Tests: 168 passing (+1 ACCESSES from multi-line CALL USING) * fix(cobol): resolve 13th review — CICS LOAD, USING extraction, file scoping #1: CICS LOAD unresolved edge no longer silently deleted in second pass. Changed narrow cics-link/cics-xctl check to catch-all pattern: rel.reason?.startsWith('cics-') && rel.reason.endsWith('-unresolved') #2: flushCallAccum USING extraction now stops before COBOL statement verbs (INSPECT, SEARCH, SORT, MERGE, DISPLAY, ACCEPT, MOVE, PERFORM, GO TO, CALL, IF, EVALUATE). Prevents absorbing adjacent statements as false USING parameters in legacy pre-COBOL-85 code without END-CALL. #3: CICS FILE Record nodes now globally-scoped (<cics-file>:FILENAME) instead of per-file-scoped. Enables cross-program CICS file access analysis, consistent with SQL table scoping (<db>:TABLE). #4: callAccum pre-check regex now has (?<![A-Z0-9-]) lookbehind to prevent false activation on compound identifiers like WS-CALL-FLAG. Tests: 168 passing | TypeScript clean * fix(cobol): resolve 14th review — callAccum false paragraph + Area A guard #1: callAccum continuation lines now check for COBOL statement verb starts (GO TO, PERFORM, MOVE, etc.) and paragraph/section headers. If detected, the CALL is flushed as-is and the line processed normally — prevents false paragraph detection and currentParagraph corruption from lines like "WS-ADDR." being treated as paragraphs. #4: callAccum pre-check now guarded by currentDivision === 'procedure' to prevent unnecessary activations in DATA DIVISION. #5: Fixed-format paragraph detection now rejects lines with >7 leading spaces (Area B indentation) as paragraph candidates. Paragraph names in fixed-format must start in Area A (col 8-11, max 7 spaces). Free-format mode is unaffected. Tests: 168 passing | TypeScript clean * fix(cobol): resolve 15th review — callAccum Area A + verb boundary fixes #A: Column-position-aware paragraph detection in callAccum flush. #B: inspectAccum early-flush on paragraph/section/verb headers. #C: Verb boundary \b → (?:\s|$) prevents MOVE-COUNT false flush. * test(cobol): add 17 edge-case regression tests + fix USING verb boundary 17 new tests covering all recurring review patterns: Multi-line CALL USING (7 tests): - Parameters on separate continuation lines (IBM mainframe style) - No absorption of INSPECT/GO TO/paragraphs following CALL - END-CALL scope terminator - Hyphenated identifiers (MOVE-COUNT) not triggering false flush - Dual quoted+dynamic CALL on same line (ON EXCEPTION) Nested program attribution (2 tests): - CALL in inner program within inner line range - PERFORM before first paragraph has null caller CRLF compatibility (1 test): - GO TO DEPENDING ON with \r\n line endings Area A paragraph detection (2 tests): - Area B (>7 spaces) rejected; Area A (7 spaces) accepted SORT/MERGE (1 test): COLLATING SEQUENCE keywords not captured PROCEDURE USING (2 tests): RETURNING excluded, period-terminated Comment stripping (1 test): pipe in quoted string preserved SELECT OPTIONAL (1 test): correct file name, not OPTIONAL keyword Bug fix: USING extraction regex verb terminators changed from \bVERB\b to \bVERB(?=\s|$) in flushCallAccum — prevents truncation on hyphenated identifiers like MOVE-COUNT, PERFORM-LIMIT. Total: 185 tests passing * test(cobol): add 32 comprehensive edge-case regression tests 13 new describe blocks covering all extraction features: - EXEC DLI: no-SEGMENT, multi-line accumulation (2 tests) - SET: multiple targets, DOWN BY, TO numeric (3 tests) - INSPECT: CONVERTING, multiple counters, tallying-replacing, paragraph flush during accumulation (4 tests) - DECLARATIVES: no-STANDARD keyword, I-O mode, post-END paragraphs (3) - COPY REPLACING: pseudotext deletion ==OLD== BY ==== (1 test) - VALUE: hex literal, negative numeric, ALL literal (3 tests) - OCCURS: TO range, fixed-size without DEPENDING ON (2 tests) - Dynamic CALL/CANCEL: end-of-line, multiple CANCELs (3 tests) - EXEC SQL: INCLUDE skips tables, SELECT INTO host vars, host variable extraction (3 tests) - INITIALIZE: target and caller context (1 test) - Nested programs: sibling scoping, PROGRAM-ID without ID DIV (2) - EXEC EOF flush: unclosed EXEC SQL flushed (1 test) - Multi-PERFORM: IF/ELSE dual PERFORM on single line (1 test) - IS EXTERNAL: USAGE not polluted by external flag (1 test) Total: 215 tests passing * fix(cobol): resolve 16th review — CANCEL in CALL block + USING boundary #1: flushCallAccum now extracts CANCEL statements from within CALL ON EXCEPTION blocks. Adds RE_CANCEL + RE_CANCEL_DYNAMIC matchAll passes alongside existing CALL extraction. #2: Added \bCANCEL(?=\s|$) to USING lookahead regex to prevent CANCEL keyword being captured as false USING parameter. #3: Multi-line CALL start now returns immediately to prevent the CALL start line from simultaneously feeding sortAccum/inspectAccum. #6: Division transitions now flush all active accumulators (callAccum, sortAccum, inspectAccum) to prevent state leakage across programs. Also added CANCEL to callAccum flush trigger verb list. Tests: 215 passing | TypeScript clean * refactor(cobol): extract shared verb constants + resolve 17th review Extract COBOL_STATEMENT_VERBS, RE_STATEMENT_VERB_START, and RE_USING_PARAMS as shared constants — eliminates 4 duplicated 25-verb regex patterns. 17th review: #1 flushCallAccum before EXEC entry, #2 inspectAccum verb parity via shared constant. Tests: 215 passing | TypeScript clean * test(cobol): replace all fuzzy assertions with exact toBe checks Replaced 7 toBeGreaterThan/toBeLessThan/toBeGreaterThanOrEqual assertions with exact toBe values: - dataItems.length: >= 3 → toBe(3) - calls.length: >= 1 → toBe(1) - calls[0].line: range check → toBe(10) - programs[].startLine/endLine: comparison → exact values - innerA.endLine/innerB.startLine: comparison → exact values Also added 11 new edge-case tests (accumulator flush on EXEC/division transitions, free-format, CANCEL in CALL block, SORT THRU, verb flush, integration). 226 tests passing — zero fuzzy assertions remain. * fix(cobol): resolve 19th review + 15 accumulator flush tests Fixes: #1: END PROGRAM flushes callAccum/sortAccum/inspectAccum #2: PROGRAM-ID sibling path flushes all accumulators #3: Added COMPUTE/ADD/SUBTRACT/MULTIPLY/DIVIDE/STRING/UNSTRING to COBOL_STATEMENT_VERBS (now 32 verbs) Tests (15 new): - END PROGRAM flush: single + nested programs (2) - PROGRAM-ID sibling flush (1) - Arithmetic verb flush: COMPUTE/ADD/SUBTRACT/MULTIPLY/DIVIDE (5) - String verb flush: STRING/UNSTRING (2) - Arithmetic not captured as false USING params (1) - SORT flushed at END PROGRAM (1) - INSPECT flushed at END PROGRAM (1) - All with exact toBe assertions (2) Total: 239 tests passing | Zero fuzzy assertions * fix(cobol): resolve 20th review — INITIALIZE multi-target + 2 tests Finding 1: INITIALIZE now captures multiple targets with REPLACING clause keyword filtering. Regex changed to lazy match stopping at REPLACING/WITH/period boundary. Targets split on whitespace and filtered against INITIALIZE_CLAUSE_KEYWORDS set. Tests (2 new): - INITIALIZE multi-target: WS-CUSTOMER WS-ORDER WS-LINE-ITEM → 3 - INITIALIZE with REPLACING: only WS-RECORD captured, not keywords Total: 241 tests passing | TypeScript clean |