5 commits

Author	SHA1	Message	Date
FAll	2cfbc4a259	feat(spring): build bean candidate inventory (#2494) ... * feat(java): inventory Spring bean candidates * fix(java): fail closed on Spring annotation shadowing * fix(java): resolve Spring beans after imports * fix(java): remove stale bean extraction path * style: satisfy locked Prettier version * fix(spring): address PR review findings * feat(spring): share bean inventory across Java and Kotlin * fix(spring): gate bean inventory analysis completeness * fix(kotlin): avoid reloading cached scope source * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-07-20 09:28:23 +01:00
henry201605	b16ec344f7	perf(group/http): skip source parse for graph-covered route files (#2138 Part 2) (#2265) ... * feat(routes): resolve + persist handler symbol on Route nodes (#2138 part 2, WIP) Part 2 groundwork for #2138: give the graph-assisted HTTP provider path the handler symbol directly, so it no longer re-parses source to recover the handler name. (The remaining parse-skip in extract() + a call-count benchmark land in a follow-up commit.) - ExtractedDecoratorRoute gains `handlerName`; the Spring extractor captures the decorated method's name (the method_declaration node is in hand). - New `resolveRouteHandlerSymbols` (call-processor) resolves each route's handler to a real symbol UID, keyed by normalized route URL — Laravel framework routes (controller + method) and decorator routes (Spring/FastAPI) both reduce to `(filePath, name) -> nodeId`. Threaded through the parse phase onto `ParseOutput.routeHandlerSymbols`. - routes phase stamps `Route.handlerSymbolId`; persisted end-to-end (schema + Route CSV row + getCopyQuery COPY columns), mirroring Part 1's `method`. - HttpRouteExtractor: `HANDLES_ROUTE_QUERY` returns `handlerSymbolId`; `extractProvidersGraph` uses it as the authoritative symbol and SKIPS `getDetections()` for resolved rows (CONTAINS is a cheap graph lookup for the display name only — no tree-sitter parse). Fully backward compatible: an unresolved/old-index route with no `handlerSymbolId` keeps the source-scan fallback. - Extracted `normalizeExtractedRoutePath` to `route-extractors/route-path.ts` (shared by routes phase + resolver without an import cycle). - SCHEMA_BUMP 6->7 (ParseWorkerResult gained `handlerName`); regenerated the emit-persistence byte-identity baseline (route.csv header gained two columns). - Tests: Spring pipeline asserts the Route node carries a handlerSymbolId resolving to the handler method; extractor fast-path test proves the handler resolves with zero source detections. Refs #2138 * perf(group/http): skip source parse for graph-covered route files (#2138 Part 2) Builds on the persisted Route.handlerSymbolId (U0–U3a). When a file's HANDLES_ROUTE rows all resolve a handler symbol AND its language plugin declares routeCoverage: 'complete' (Java/Python/PHP), the graph is authoritative for that file's providers, so the source scan + tree-sitter parse can be skipped — the scan would only re-discover routes the graph already has. This is the measurable parse reduction #2167 could not show. Consumer safety: routeCoverage: 'complete' asserts *provider* Route-node completeness only. The scan() of those same languages also emits consumer detections (RestTemplate/WebClient/OkHttp/Feign, Guzzle/Http::, requests/httpx), and ingestion's FETCHES edges are JS/TS-only — so the graph cannot back up server-side consumers. A provider-covered controller that also calls out would otherwise lose its consumer contract. Guarded by a cheap, parse-free text gate. - types: HttpLanguagePlugin gains - routeCoverage?: 'complete' | 'partial' (default 'partial') - hasConsumerSignals?(content): false only when the raw source provably has no outbound-HTTP call this plugin detects (conservative). - java/python/php: mark routeCoverage 'complete' + implement hasConsumerSignals with a token regex over their consumer idioms. - http-route-extractor: run the graph provider pass first to build a coveredFiles set; then keep a file covered only when hasConsumerSignals(content) === false (read via readSafe, no parse). scanFiles = files not covered → drives collectProjectDetections + both source scans. Fail-open per file: any unresolved row, a 'partial' language, a positive consumer signal, a missing hook, or an unreadable file leaves the file in the scan set. The orchestrator names no languages — token knowledge stays in the plugins. Net: pure-provider controllers skip the parse (the win); controllers that also call out are still parsed (no consumer loss); partial-coverage languages and graph-less runs are unchanged. - test: route-parse-skip integration test spies the real parseSourceSafe to COUNT parses over a temp repo of Spring controllers with a mock DB — baseline (every file parsed), fully-covered (0 parses), mixed (unresolved file falls back, resolved stays skipped), and provider+consumer (a covered controller that also calls restTemplate is parsed; its consumer contract survives). * fix(group/http): cover Spring HTTP Interface @*Exchange in Java consumer-signal gate #2254 (merged) added Spring 6 HTTP Interface `@(Get|...)Exchange` / `@HttpExchange` as a new Java *consumer* idiom. The #2138 parse-skip consumer-safety gate must recognize it, or a provider-covered file carrying an `@GetExchange` could be parse-skipped and lose that consumer contract. Add `Exchange` to JAVA_HTTP_PLUGIN.hasConsumerSignals (conservative; also matches `restTemplate.exchange(`). * style(group/http): prettier formatting for #2138 Part 2 files * style(ingestion): prettier formatting for call-processor.ts (#2138 Part 2) * fix(group/http): P1 (Java over-claim) + P2 (handler mis-attribution) on top of #2268 (#2138 Part 2) Re-applied on the maintainer's #2268 (expanded Java/Kotlin consumer extraction) base. P1 — `routeCoverage: 'complete'` over-claimed for Java: the graph provider set is a strict subset of the group scan (array-form `@GetMapping({...})`, interface-inherited routes, same-URL multi-verb have no graph Route node), so parse-skip could drop those group-only providers. - java/python → default 'partial' (always source-scanned). Java flips to 'complete' only once ingestion provider extraction matches the group scan (a separate follow-up). Python was a no-op anyway (no handlerName resolved); 'complete' was a latent trap. PHP stays 'complete' (Laravel ingestion ⊇ the group scan, the one language the skip engages for). - python hasConsumerSignals widened to a true superset of scan() (uri=/url= wrapper, aiohttp, urllib). Java's gate already covers #2268's consumer set (same receivers; the @*Exchange token is present). P2 — resolveRouteHandlerSymbols: reserve the URL slot on first encounter even when unresolved (mirrors addRoute first-writer-wins, so a later same-URL route can't stamp the node-winner's slot); refuse to guess on an ambiguous same-name lookup (exactly one match → use it; zero/many → fail-open, never a wrong handler). The cross-source case (filesystem route winning a URL a framework route also normalizes to) is unchanged — the resolver never receives filesystem routes — and stays fail-open. Tests: - route-parse-skip rewritten: the parse-skip win is proven on PHP (fully covered → 0 parses; mixed fallback; consumer-covered file still parsed), plus three Java P1 regression guards (array-form / interface-inherited / multi-verb) asserting the group-only routes survive — verified they go red if Java is flipped back to 'complete'. - resolve-route-handler-symbols: direct unit tests (the fn had none) — unique resolve, ambiguous/unknown fail-open, same-URL reservation, first-writer-wins. - http-consumer-signals: each plugin's hasConsumerSignals is a superset of its scan() consumer idioms; pure providers return false. - route-handler-symbol-roundtrip: real-LadybugDB CSV→COPY→query for Route.handlerSymbolId. --------- Co-authored-by: henry <zhangwei2017@unipus.cn> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>	2026-06-23 07:22:43 +01:00
Gergő Magyar	78b4077d8a	feat(impact): opt-in PDG-backed impact mode - statement + inter-procedural slicing, resolved-callee-id soundness, mutation-oracle validated (#2227)	2026-06-20 12:04:32 +01:00
henry201605	a691dcb320	feat(routes): persist HTTP method on Route nodes (#2138 part 1/2) (#2234) ... * feat(routes): persist HTTP method on Route nodes Part 1 of 2 for issue #2138 (skip redundant HTTP provider source-scan). The ingestion routes phase already knows each route's HTTP verb — `ExtractedRoute.httpMethod` (Spring/Laravel framework routes) and `ExtractedDecoratorRoute.httpMethod` (decorator routes) — but dropped it when creating the Route graph node. As a result `HttpRouteExtractor`'s graph-assisted path could not recover the verb for `framework-route` sources (whose edge `reason` is undecodable by `methodFromRouteReason`) and had to fall back to re-scanning the handler source. Changes: - routes phase: carry `httpMethod` into `RouteEntry` and persist it as `Route.method` (filesystem-derived Next.js/Expo/PHP routes have no structural verb, so they stay method-less). - HttpRouteExtractor: HANDLES_ROUTE query now returns `route.method`; `extractProvidersGraph` prefers it and falls back to the edge reason for older indexes / method-less routes (fail-open, fully backward compatible). - tests: graph-method precedence, multi-verb handler disambiguation via the persisted verb, case normalization, and old-index fallback. This change is intentionally NOT a performance optimization on its own: the graph path still parses handler files to recover the handler *name*. Eliminating that parse (and thus the redundant source-scan #2138 targets) requires linking HANDLES_ROUTE to the handler symbol, which lands in Part 2. This PR is the data-completeness groundwork for that. Refs #2138 * test: account for new Route.method in blade route-registry assertion The routes phase now persists httpMethod onto RouteEntry/Route nodes, so the strict toEqual on the framework-route registry entry must include the new method field. * fix(routes): persist Route.method end-to-end + real-lbug round-trip test Addresses review on #2234 (magyargergo + tri-review): the prior commit read `route.method` in HANDLES_ROUTE_QUERY but never added the column to the schema/persistence path, so against a real LadybugDB the query failed to bind (`Cannot find property method for r.`) and the `catch { return [] }` silently swallowed it — regressing the graph-assisted HTTP provider path. - schema: add `method STRING` to ROUTE_SCHEMA. - csv-generator: write `method` in the Route CSV row (header + row, column order aligned with the COPY statement). - lbug-adapter: add `method` to getCopyQuery('Route'). - routes phase: normalizeRouteMethod() canonicalizes the verb to upper-case and skips non-verbs — Laravel resource/apiResource carry httpMethod values like `resource`/`apiResource`, which must not land a junk method. - http-route-extractor: log at debug when the HANDLES_ROUTE / FETCHES graph query throws, so a total graph-provider outage is observable instead of silently swallowed. Export HANDLES_ROUTE_QUERY for the round-trip test. - tests: add a real-lbug round-trip (graph -> CSV -> COPY -> HANDLES_ROUTE_QUERY) asserting the verb persists and reads back; update the blade registry assertion for the normalized (upper-case) method. Refs #2138 * fix(csv): coerce Route.method to string for escapeCSVField typecheck node.properties.method is typed unknown (not a declared property), so `x || ''` stayed unknown and failed tsc against escapeCSVField's string|number param. Coerce explicitly with String(... ?? ''). * test(bench): regenerate emit-persistence fingerprint for Route.method column Adding the method column to route.csv changes the byte-identity fingerprint of the emit-persistence benchmark (the synthetic graph's route.csv header now includes 'method'). scaling_ratio unchanged (~0.9, linear); this is the documented regenerate-on-legitimate-emit-change path. Streaming baseline (BasicBlock/PDG) is unaffected. --------- Co-authored-by: henry <zhangwei2017@unipus.cn> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>	2026-06-20 06:30:16 +01:00
Gergő Magyar	df08ecc397	perf(lbug): cut graph-DB emit/persistence wall time (#2203) (#2215) ... * perf(lbug): add PROF_LBUG_LOAD persistence-path timing breakdown (#2203 U1) loadGraphToLbug is un-timed today; the analyze 'emit' number is the scope-resolution emit bucket, not the CSV->COPY persistence path. Add a zero-cost-when-off per-stage breakdown (csv-emit/copy-nodes/rel-split/ copy-rels/fallback/total + node/rel counts) gated by PROF_LBUG_LOAD=1, mirroring the PROF_SCOPE_RESOLUTION pattern. Document the flag in README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(lbug): route relationships to per-pair CSVs in the emit pass (#2203 U2) Relationships were written once to a monolithic relations.csv, then re-read line-by-line (regex per edge) and re-split into per-FROM->TO-label-pair files before COPY — writing and reading the entire ~1M-edge set twice. Route each edge to its pair file directly during the single emit pass via a shared RelPairRouter, eliminating the monolithic write + re-read + per-edge regex. The router applies the SAME getNodeLabel + validTables filter as the legacy splitRelCsvByLabelPair, which is retained as a differential oracle. A new differential test asserts the direct-emit per-pair files are byte-for-byte identical to the oracle's, with identical skip/total accounting. The prof line (U1) drops its rel-split stage (routing now folds into csv-emit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(lbug): skip per-row microtask tick in BufferedCSVWriter (#2203 U3) addRow awaited an already-resolved promise on every buffered row, scheduling a microtask per node even when nothing flushed (millions at scale). It now returns a promise ONLY when it flushes; the node-emit loop awaits once per iteration after the switch. Flush/drain semantics are unchanged, so backpressure on the rows that actually write is preserved and the emitted CSV bytes are byte-identical (covered by the determinism + differential tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * bench(lbug): emit throughput + byte-identity gate for the persistence path (#2203 U4) Build-free bench (bench/emit-persistence/measure.mjs) times streamAllCSVsToDisk on a synthetic graph at two scales and gates: (1) an order-independent sha256 fingerprint over every emitted CSV line — the byte-identity guard for the U2/U3 emit optimisations — and (2) a scaling-ratio budget catching an O(n^2) emit re-regression. Wired into ci-tests.yml alongside the cfg/scope-capture benches. The LadybugDB COPY half needs a real DB, so its timing stays in PROF_LBUG_LOAD + the integration round-trip tests (documented in the bench README, with the deferred COPY-parallelism follow-up). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): apply autofix feedback (#2203) - P1: router backpressure drain-await rejected with a generic AbortError, masking the real EMFILE/disk-full error. Expose RelPairRouter.lastError and rethrow it in the emit catch — mirrors the oracle's throw streamError ?? err. - P1: cover RelPairRouter error + backpressure + teardown paths with a new unit test (test/unit/rel-pair-routing.test.ts) using an injected mock stream. - P2: wrap streamAllCSVsToDisk body in try/finally so the setMaxListeners bump is always restored (the U2 rel-routing throw path could leak it). - P2: dedup WriteStreamFactory — re-export the canonical type from rel-pair-routing instead of a second identical declaration. - P2: annotate splitRelCsvByLabelPair @internal as the retained differential oracle so a future dead-code sweep doesn't delete the byte-identity guard. - P3: differential test now covers the proc_ prefix + clears GITNEXUS_SORT_GRAPH_OUTPUT to prevent env-leak desync. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(lbug): scope byte-identity to quote-free ids + lock the quote-in-id divergence (#2215 review) The 'byte-identical' claim was unconditional, but the router derives labels from the raw id while the retained splitRelCsvByLabelPair oracle re-derives them via a regex over the escaped row — so for an id containing a double-quote they diverge (the router is the more-correct path). Soften the wording in rel-pair-routing.ts, the bench README, and the differential-test comment to document the exception, and add a differential test asserting the intended divergence (router routes the quote-in-id edge; oracle drops it) so a future change can't silently revert to the buggy regex semantics. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * bench(lbug): per-file fingerprint so the gate catches pair-file mis-routing (#2215 review) fingerprintEmit flattened every line of every per-pair file into one array, sorted globally, and hashed — losing file boundaries, so a row routed to the WRONG pair file produced an identical fingerprint. Hash a per-file digest (filename + sha256(file bytes)) and combine the sorted entry list, so mis-routing (and within-file row reordering) now changes the fingerprint. Baseline regenerated; the new scheme yields a different hash on byte-identical emit, confirming it is sensitive to file structure the old flatten ignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * bench(lbug): add absolute large-scale wall-time backstop to the emit gate (#2215 review) The scaling-ratio gate only compares large/small, so a uniform Nx slowdown at both scales passes with ratio ~1.0. Add an opt-in max_ms_large ceiling (1000ms vs observed ~200ms — generous, host-noise-tolerant) that --check enforces alongside the ratio, catching a gross absolute regression the ratio misses. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): cover the sorted-output path in the byte-identity differential (#2215 review) The differential test only exercised the default insertion-order emit path. Add a case under GITNEXUS_SORT_GRAPH_OUTPUT=1 that feeds the oracle the same id-sorted order orderedRelationships() uses and asserts per-pair byte-identity, so within-pair row reordering on the sorted path can't slip past the gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): cover the invalid-TO-label skip branch (#2215 review) Only an invalid-FROM label was exercised; the validTables skip is an OR over both endpoints, so the invalid-TO branch was untested (an inverted && would have slipped through). Add a valid-FROM/invalid-TO edge to the differential test and the router unit test, asserting it's skipped identically by router and oracle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): exercise the BufferedCSVWriter FLUSH_EVERY boundary in vitest (#2215 review) The U3 addRow change (returns a flush promise only on flush; undefined when buffered) and the loop's `if (pending) await pending` were only crossed by the bench, never vitest (all fixtures are <500 nodes). Add a 600-node graph through streamAllCSVsToDisk asserting all rows land exactly once across the 500-row flush boundary — no drops, dups, or corruption. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(lbug): drop redundant step cast in buildRelRow (#2215 review) GraphRelationship.step is already typed number?, so (rel as { step?: number }).step was a no-op structural cast that obscured the shared-type coupling. Use rel.step directly. Byte-identical — bench fingerprint unchanged, differential test green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(lbug): make the unknown-label node drop explicit (#2215 review) With the U3 `let pending` switch idiom, a node whose label matches neither codeWriterMap nor multiLangWriters left `pending` undefined and was silently dropped — a footgun for a future node type. Add an explicit else with a comment documenting that unknown labels are intentionally not persisted and that a new type must be wired into a writer map. No behavior change (byte-identity + tests unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(lbug): drop the unused WriteStreamFactory re-export (#2215 review) The type was re-exported from lbug-adapter 'to preserve this module's surface,' but no external code imports it by name from here (the only test reference is a comment). Keep the import from rel-pair-routing.ts (its canonical home, still used by splitRelCsvByLabelPair's signature) and drop the dead re-export. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 19:40:59 +01:00