Commit graph

7 commits

Author SHA1 Message Date
Gergő Magyar
7e993ab897
fix(group): fail ambiguous sync names and honor analyze --name (#3094)
* fix(group): fail sync when a member name is ambiguous

Silent first-match bound the wrong clone when --allow-duplicate-name
registered two paths under one alias. Refs #3028.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(analyze): apply --name on the already-up-to-date path

A rename should not require --force when the index is already current.
Register before the same-commit branch restamp. Refs #3028.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(group): hint member path when impact --repo is an alias

$localRepo stays the yaml key; joining on the registry alias is a
non-join. List matching keys so operators can retry. Refs #3028.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(group): keep injected sync and alias hints consistent

Workspace-deps path maps reuse the resolved handle so duplicate names
cannot throw after an injected resolver. Alias hints match case-insensitively.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-29 21:48:46 +00:00
azizur100389
f64cc8b7a8
feat(group): add GraphQL cross-repo contracts (#3070)
* feat(group): add GraphQL contract extraction

* fix(group): tighten GraphQL contract guards

* fix(group): complete GraphQL review hardening

* fix(group): isolate bounded GraphQL reads

* fix(group): harden GraphQL contract extraction
2026-08-29 08:39:09 +01:00
DuduPhudu
0f793558ad
fix(group)!: stop group sync claiming matching it never did (#3020)
* fix(group)!: remove the matching cascade that was advertised but never built

`gitnexus group create` wrote `matching.bm25_threshold` and
`matching.embedding_threshold` into every generated group.yaml, and no matcher
ever read either one. That was not the whole of it — an entire feature surface
described a BM25/embedding cascade that does not exist:

- `matching.bm25_threshold` / `matching.embedding_threshold` — parsed, persisted,
  unread
- `detect.embedding_fallback` — defaulted and templated, unread
- `MatchType` declared `'bm25' | 'embedding'`; both variants unreachable
- `SyncOptions.skipEmbeddings` — declared in sync.ts and never read
- `gitnexus group sync --skip-embeddings` — accepted, threaded through
  GroupService, ignored
- CLI help in en and zh-CN promised "Exact + BM25 only (no embedding fallback)"
- the MCP `group_sync` schema exposed `skipEmbeddings`, described as
  "Exact + BM25 only (Demo PR: same as default exact path)"

`sync.ts` imports exactly `buildProviderIndex`, `runExactMatch` and
`runWildcardMatch`, and the printed cascade has one stage. An operator whose
links do not match reaches for those thresholds first, and turning either knob
changes nothing — config that silently does nothing is how people conclude a
feature is broken.

Evidence that the cascade should be deleted rather than implemented, from a real
backend/frontend pair: of 165 consumer contracts, 149 link exactly and 16 do not.
Nine of the sixteen are third-party APIs (Google OAuth, Apple public keys,
PostHog, image annotation) with no in-group provider by construction — similarity
matching cannot recover them, it can only invent false links. Two are verb
mismatches: the frontend calls `POST /links` and `GET /links/check-exists` while
the backend declares `GET /links` and eleven other `/links/*` routes but neither
of those, so a fuzzy path match would link a POST consumer to a GET provider. The
rest are path-extraction artifacts. Roughly none of the sixteen would be
correctly recovered, and several would be actively mis-linked.

BREAKING CHANGE: `gitnexus group sync --skip-embeddings` and the MCP `group_sync`
`skipEmbeddings` parameter are removed. Both were accepted and ignored, so no
behavior changes — but a script passing the flag now fails with `unknown option`
instead of being silently misled. Existing group.yaml files keep loading: the
removed keys are simply no longer part of the schema, and a regression test pins
that a legacy config carrying all three still parses.

Closes #3006

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(group)!: honour --exact-only, drop inert --allow-stale, report every matching stage

Addresses the review findings on #3020, all of which are the same defect the PR
itself is about: group-sync surface that describes behaviour the pipeline does
not have.

`exactOnly` was inert in exactly the way `skipEmbeddings` was — declared on
`SyncOptions`, threaded through the CLI and the MCP tool, and read by nothing —
and strictly worse, because the stage it promised to suppress DOES run and DOES
write `matchType:'wildcard'` links into contracts.json and the bridge, which
`group impact` and cross-repo `trace` then traverse. It is now honoured rather
than deleted: unlike the never-built BM25/embedding stages, the stage it names
exists, so the flag describes a real choice. The substituted result is
`{ matched: [], remaining: unmatched }`, not an empty result — `wildcard.remaining`
IS `SyncResult.unmatched`, so skipping the stage has to leave its input unmatched
rather than dropping it from the count an operator reads.

`allowStale` had no such stage to gate: `syncGroup` emits no stale warning at any
point (the `checkStaleness` call lives in `groupStatus`, a different path), so it
is removed under the same rationale as `skipEmbeddings`.

`group sync` now prints every matching stage instead of `exact` alone. The old
block printed a `Matching cascade:` header and counted only exact links while the
next line reported `result.crossLinks.length` — which also includes `manifest` and
`wildcard` — so for any group with those the two numbers disagreed with nothing on
screen explaining why. Counting is an exhaustive `Record<MatchType, number>`, so a
new MatchType fails the build here instead of going silently uncounted, and reads
through `?? 0` so a legacy registry carrying a removed matchType prints an honest
count rather than `NaN`.

Also: the MCP `group_sync` description no longer omits the wildcard stage that
always runs, `exactOnly`'s description no longer refers to a "cascade", and
bench/cross-repo-trace/verify.mjs no longer generates the removed threshold keys
into a fresh group.yaml.

Tests: `sync-exact-only.test.ts` pins both directions of the gate (mutation-verified:
removing the gate, or returning `remaining: []`, both go red). `group-tools.test.ts`
pins that the MCP schema dropped `skipEmbeddings` and kept `exactOnly`.
`group-cli.test.ts` pins that both removed flags are rejected, with `--exact-only`
as an accepted-flag control. `config-parser.test.ts` now pins that legacy keys are
PRESERVED (measured, not assumed) rather than only that parsing does not throw.

The type narrowing's fallout in test files is cleared: `tsc -p tsconfig.test.json`
is 987 errors at head against 987 measured on origin/main, with the two error sets
identical — zero net, zero new, zero masked.

Verification: `tsc --noEmit` exit 0; prettier clean; eslint 0 errors (2 warnings,
both pre-existing on base); 69 test files / 1169 tests green across
test/unit/group, test/integration/group, tools, cli-i18n and cli-index-help.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(group): reject malformed and retired group_sync parameters (U1)

`GroupService.groupSync` read `exactOnly` off an untyped MCP payload with
`Boolean(params.exactOnly)`. While the flag was inert that coercion was
harmless; now that it gates the wildcard matching stage, the string "false" --
a routine shape for an LLM caller emitting JSON -- is truthy, so a caller that
asked to KEEP wildcard matching got it suppressed and a registry with fewer
cross-links persisted to disk. The opposite of the request, written down.

Validate instead of coercing, at the service boundary: the MCP SDK does not
enforce a tool's advertised inputSchema and `callTool` is reachable directly,
so this method is the real gate. The validator mirrors `validateImpactMode`'s
`{ value } | { error }` shape -- the established idiom for this boundary, and
the one groupSync's other guards already return through.

Also refuse `skipEmbeddings` and `allowStale` by name. The CLI rejects them
outright because commander errors on an unknown option; the MCP path accepted
and silently dropped them, so an agent working from a cached tool schema was
never told. Removing them took away discoverability, not acceptance.

Both guards run before the group is read off disk, so a rejected call performs
no work. Every test asserts the sync did NOT run -- an error string alone
cannot distinguish "refused" from "refused but synced anyway".

The tool description gains the validation note AFTER the registryOutcome
paragraph: `tools.test.ts` slices that description by ordinal position of the
'preserved' / 'superseded' / 'no-prior-registry' literals, so appending past
all three leaves those slices intact (verified, 44/44).

tsc clean; 1039/1039 group unit tests pass.

* fix(group): record the matching stages a sync was told to skip (U2)

An `--exact-only` sync wrote a contracts.json and bridge with fewer cross-links
and nothing recorded that the wildcard stage had been suppressed by request.
`group_impact` and cross-repo `trace` read that registry as authoritative, so a
narrowed graph was indistinguishable from a complete one -- and because
`group_sync` is MCP-exposed, one agent call durably narrowed the shared answer
for every later reader with no signal at all.

Add `suppressedMatchStages` to ContractRegistry and SyncResult, following the
`unreadableRepos` tri-state end to end: absent means a registry written before
the field existed, `[]` is the measurement "this run suppressed nothing", and a
populated list names the stages. The writer always emits it, because omitting
the empty case is what made "measured, none" unreachable for `unreadableRepos`.

Two properties that are easy to get backwards, and are why the split matters:

- SyncResult carries the marker on EVERY outcome. The sync genuinely did skip
  the stage whatever happened to the file afterwards, and the CLI summary (U3)
  renders from this rather than re-deriving it from the caller's options.
- The PERSISTED registry stamps it only on the `written` outcome. The preserve
  path re-writes `{ ...prior }`, so a carried-forward registry keeps the marker
  of the sync that actually produced its contracts instead of being relabelled
  with this run's request. That holds by construction: the registry literal
  carrying the field is only reachable on the written path.

`loadContractRegistryResilient` gains an explicit line, because it rebuilds the
envelope field by field with no spread of the parsed root -- a new on-disk field
is silently dropped unless named there.

Its reader is `recordedMatchStages`, not the existing `recordedRepoList`: that
one validates `string[]`, which is right for repo names and one notch too weak
here. This repo has already retired MatchType members ('bm25', 'embedding'), so
a stale value on disk is a real shape, and dropping non-members keeps an unknown
stage name from reaching a caller typed as a live one.

Surfaced on `group_contracts` and on `group_sync`'s own return -- deliberately
kept separate from the truncated/truncationReason/riskEpistemic triple. That
triple reports limits a run hit by accident, whose remedy is to fix the repo; a
suppressed stage was asked for, and its remedy is to re-sync without the flag.
Conflating them would tell an agent to retry something that returns identically.

tsc clean; 1043/1043 group unit tests pass.

* fix(group): name a skipped matching stage as skipped, and pin it (U3, U4)

Two facts were printing as the same line. `wildcard: 0 cross-links` meant both
"the stage ran and matched nothing" and "the stage never ran because you passed
--exact-only" -- the same conflation this summary block was introduced to remove
one line up, reintroduced by the flag that made the block necessary.

Render a suppressed stage as `skipped (--exact-only)`, driven by the sync's own
`suppressedMatchStages` rather than by `opts.exactOnly`. The renderer reports
what the sync did, not what the caller asked for, so it stays correct on the
outcomes where the run ended without writing a registry -- which is where the
summary is least legible and a re-derivation from the options would have been
wrong.

Also drops the `?? 0` fallback and the comment justifying it. The comment
claimed a legacy registry could carry a retired matchType into this loop. It
cannot: `syncGroup` returns a freshly computed `crossLinks` array on every
outcome, and even on the preserve path the prior links go to disk while the
fresh array is returned. The code was harmless; the stated reason was false, and
a comment that explains an unreachable path is worse than no comment.

U4 pins both halves through the CLI. A manifest fixture is sufficient: the stage
counts must sum to the total on the `Wrote contracts.json (…)` line, and the
skipped rendering does not need a stage to have matched anything, because
--exact-only records the suppression whatever the fixture holds. That is why
this coverage did not need indexed gRPC/Thrift fixture repos.

Verified by mutation, not assertion: removing the skipped-rendering branch turns
`names a stage it was told to skip as skipped` red and leaves the other 22
green. A control case pins the opposite direction -- the same group without the
flag still reports the stage as zero -- so `skipped` cannot be printed
unconditionally and pass.

U3 and U4 land together: the test has no value without the renderer, so one
commit keeps a revert clean. Both depend on U2, which introduced the field they
read.

tsc clean; 1066/1066 across the group unit and CLI integration suites.

* fix(group): make every description of --exact-only match what it does (U5)

Two descriptions this branch wrote or touched still misstated behavior.

The MCP `exactOnly` description carries "Manifest links still apply." The CLI
help and both locale strings, rewritten in the same commit, omit it -- so the
surface most operators read understated what still runs. Manifest cross-links
are computed before the gate and are genuinely unaffected by the flag, so the
caveat is the accurate half and the CLI now says it too.

The `group_sync` tool description opened with "extract HTTP contracts". That
clause was carried forward byte-identical while only the trailing cross-linking
half was rewritten, and it is wrong: the detect config has six non-HTTP
extraction toggles, and this branch's own new test fixture is Thrift.

`help-i18n.ts` is deliberately untouched. It maps an option to its translation
key and that key already exists; only the commander string and the two locale
values carry text, so a text-only change does not reach it.

The tool-description edit sits ahead of the registryOutcome paragraph, leaving
the relative order of the 'preserved' / 'superseded' / 'no-prior-registry'
literals intact -- `tools.test.ts` slices that description by their positions.

tsc clean; 64/64 across the locale-parity, help-registration, tool-schema and
group-tool suites.

* fix(group)!: remove max_candidates_per_step and shared_libs (U6)

Both keys were declared, defaulted, written into every generated group.yaml,
and read by nothing -- the same three-station dead surface this PR removed for
bm25_threshold, embedding_threshold and detect.embedding_fallback. Every other
DetectConfig field gates a real extractor in sync.ts; shared_libs gates nothing,
because 'lib' contracts come only from the operator-declared manifest extractor.
MatchingConfig reaches matching.ts solely through buildNoisyContractFilter,
which reads exclude_links_paths and exclude_links_param_only_paths and nothing
else.

Existing group.yaml files keep loading and keep their keys. parseGroupConfig
spreads the raw block over its defaults, so a key the schema no longer knows
about survives into the returned config -- which matters because `group add` and
`group remove` round-trip the operator's file through loadGroupConfig ->
yaml.dump -> write, so anything the parser dropped would be deleted from their
checked-in file. The legacy-config test now pins both keys in the same cast form
as its three siblings, and the fixture carries shared_libs so that assertion is
not vacuous.

Two stations that are easy to miss and are swept here:

- gitnexus/bench/cross-repo-trace/verify.mjs GENERATES a fresh group.yaml. It is
  not a preserve-path fixture, so "leave YAML fixtures alone" does not cover it;
  the repo has two generators and both are updated. It is a .mjs file outside
  tsconfig's include, so no type gate would have caught it.
- config-parser.test.ts asserted the removed default at runtime, which vitest
  DOES run. That assertion is gone from the defaults case (the key no longer has
  a default) and re-formed as a preserve assertion in the legacy case.

Verification gate, corrected: "zero net new errors against origin/main" would
have measured the whole branch delta and been red through no fault of this
commit. Measured instead against the branch tip immediately before it --
tsc -p tsconfig.test.json --noEmit reports 989 before and 989 after. Twenty-four
typed-literal sites across ten test files, none of them CI-gated, plus the two
runtime sites above which are.

Note the deliberate side effect: removing a key from the defaults also stops the
group add round-trip from re-adding it to a file that never carried it. Nothing
in src reads either key, so no behavior changes.

BREAKING CHANGE: `matching.max_candidates_per_step` and `detect.shared_libs` are
no longer part of the group.yaml schema and are no longer written into generated
templates. Existing files carrying them continue to parse and retain them.

src tsc clean; 1189/1189 across the group unit, group integration, locale-parity,
help-registration and tool-schema suites.

* docs(group): map PR #3020 review findings to the commits that close them

Retitles the ledger to hold one section per reviewed PR and adds #3020's ten
findings. Two things are stated rather than claimed away: `abda0d041` closes
three findings because they are one code block plus the test that pins it, and
the suppressed-stage marker is a coupled set because the renderer consumes the
field the earlier commit introduces.

Also records what is NOT closed here -- the PR description's false claim about
`max_candidates_per_step` lives outside this branch.

* refactor(group): apply simplify-pass findings

Four cleanup agents (reuse, simplification, efficiency, altitude) over this
run's diff. Efficiency was clean. The rest found five things worth fixing, two
of which were real gaps rather than style.

`recordedMatchStages` filtered unknown values instead of rejecting the list.
That inverted the tri-state on the one field built to prevent exactly this
conflation: a stale `['bm25']` -- the scenario its own comment cites as the
motivation -- survived as `[]`, which on this field MEANS "measured, nothing was
suppressed". A confident clean answer manufactured from a value we could not
read. Now all-or-nothing, matching `recordedRepoList`.

`gitnexus group contracts` showed nothing after an exact-only sync. The human
renderer destructures a fixed field list and gates its incompleteness warning on
`truncated`, so the marker reached the MCP payload and the JSON output but not
the listing an operator actually reads. It now warns, separately from the
`truncated` warning, because the remedies differ: one says fix the repo, this
one says re-run without the flag.

`verbose` was still coerced with `Boolean()` in the same call whose tool
description this branch changed to promise "PARAMETERS ARE VALIDATED". Validated
now, and added to the tool schema -- it was read by the backend and advertised
nowhere.

Reuse: the thrift wildcard-matchable pair existed twice, near-verbatim, in
`sync-exact-only` and `registry-suppressed-stages`. Both now call a shared
`makeWildcardPair` fixture, so the shape `runWildcardMatch` fires on is defined
once.

Simplification: dropped a `Set` built per sync over a list that only ever holds
zero or one entries; iterating `Object.keys(STAGE_COUNTS) as MatchType[]` also
keeps the exhaustiveness the `Record` was built for, which `Object.entries` had
discarded.

Deliberately not done, with reasons: a schema-driven unknown-parameter layer at
the MCP chokepoint (five parameters are read by backends and declared in no
schema, so a strict layer rejects working calls today, and it cannot produce the
"was removed" message finding 3 is about); folding the marker into
`GROUP_IMPACT_TRUNCATION_REASONS` (reverses a recorded plan decision and the
bridge scope is an open question for the maintainer); a per-stage suppression
cause `Record` (no second suppressor exists -- speculative); collapsing the six
`detect` extractor branches into a table (a real generalization, but a refactor
outside this diff); and converging an untouched pre-existing CLI test onto the
new manifest helper (it captures a value the helper does not return, so the
change risks more than the duplication costs).

tsc clean; eslint 0 errors (1 pre-existing warning); 1085/1085.

* docs(group): remove REVIEW-FINDINGS-MAP.md

Removes the findings-to-commits ledger from the source tree.

Note for anyone reading this in history: the file was introduced on main by
#3012 and carried that PR's findings map; this branch had appended a #3020
section. Deleting it drops both. #3012's content is recoverable with
`git show 2c0fb7753:gitnexus/src/core/group/REVIEW-FINDINGS-MAP.md`.

* fix(group): stop cross-repo impact and trace claiming a narrowed graph is complete

Closes the half of the suppressed-stage finding that was deferred. The reviewers
were right that deferring it was the weak point: the motivating harm was named
as `group_impact` and cross-repo `trace` traversing a graph missing real edges,
and those were exactly the surfaces left uncovered.

The deferral rested on an assumption that does not hold. "It is already blind to
this, so we do not make it worse" is false: `--exact-only` was inert before this
PR, so the number of narrowed registries in the world goes from zero to nonzero
exactly when this lands. The blindness was harmless only while narrowing was
impossible. And silence there is not neutral -- `cross-impact.ts` documents
`truncated: false` as an affirmative completeness claim, so those tools were
about to start asserting a complete answer over a knowingly short graph.

`suppressedMatchStages` now rides the bridge the same way `unreadableRepos`
does: persisted in meta.json (no BRIDGE_SCHEMA_VERSION bump -- meta fields have
this precedent), read back all-or-nothing, and carried across the preserve path
through `refreshPreservedBridgeMeta`'s diagnostics so a preserved bridge keeps
the marker of the sync that actually built it.

`crossRepoCompleteness` folds it in, which is what makes this one change reach
all three surfaces -- that function is by design the ONE computation behind the
truncation triple. Precedence is explicit: an unreadable or unaccounted repo
outranks a suppressed stage, because it is the more serious structural gap and
its remedy has to be the one reported.

`'suppressed-stage'` is a new member of the truncation-reason union rather than
a reuse of `'incomplete-sync'`. The earlier decision not to touch that union was
about not conflating remedies -- telling an agent to repair a repo that read
fine, for a narrowing it requested. A distinct member preserves that reasoning
while letting the answer stop claiming completeness, which is what reusing the
existing member would have destroyed.

The union's guard test did its job: adding a member failed the check that every
reason is explained on the agent-facing surface, so the impact tool description
now names this one and its distinct remedy (re-run WITHOUT the flag; nothing
failed to read).

`group status` and its CLI renderer surface it too, on the populated case only
-- absent is a registry predating the field and empty is the ordinary clean
sync; neither earns a line.

Deliberately still not done, and why: a repo-wide unknown-parameter layer for
every MCP tool. Five parameters are read by backends and declared in no schema
(`subgroupExact`, `unmatchedOnly`, `showClusters`, `showProcesses`, and
`verbose` until this branch declared it), and three tools dispatch with no
schema entry at all, so a strict layer rejects working calls until each is
reconciled. That reconciliation is the work; the layer is the cheap part. It
also cannot produce the "was removed and is no longer accepted" message the
retired-parameter guard exists to give.

tsc clean; eslint 0 errors (2 pre-existing warnings); 1159/1159 across the group
unit, group integration and tool-schema suites.

* fix(group): make the suppressed-stage signal actually reach its readers

Applies the mechanical findings from the code review of the previous commit.
That commit claimed cross-repo impact and trace stop reporting a narrowed graph
as complete. Trace did; impact did not, and two operator-facing messages said
something false. Four reviewers plus the cross-model pass converged on the same
two defects, and the untested seams were exactly where they were.

`runGroupImpact` recomputed the truncation reason and hardcoded its fallback, so
it could never emit 'suppressed-stage' -- the value the previous commit added to
the union and documented in the tool description. Every narrowed-but-readable
bridge was reported as 'incomplete-sync', telling the caller to repair a repo
that read fine. It now propagates the bridge's own reason, as cross-trace.ts
already did.

The preserve path stamped this run's request onto an older bridge. When no repo
can be read the database and registry are kept from an earlier sync, so
meta.json has to keep describing that sync; instead `{ ...existing,
...diagnostics }` overwrote its marker, leaving contracts.json, meta.json and
bridge.lbug describing three different runs. Currently masked by unreadable-repo
precedence, one loosened condition from a live wrong verdict.

`group contracts` printed "the last sync did not record which repos it could
read" after any exact-only sync: truncated was set with both repo lists empty,
so the message fell through to the wrong branch. It is now gated on the reason,
not the flag. `group impact` likewise blamed the local walk for a floor the flag
caused.

The tri-state reader is now defined once, in the leaf module whose own comment
says it exists so this exact duplication cannot recur -- it had been copied into
bridge-db.ts within one commit of that comment being true.

Both agent-facing descriptions now name the field. The previous commit added it
to three payloads and documented it on none.

Tests cover what shipped green: the preserve path for both artifacts (verified
by mutation -- reintroducing the stamp turns exactly one test red), and the
reason's REACHABILITY. The existing guard only asserted each reason is
described, which is why a documented-but-unemittable value passed it.

Also corrects a comment that said the marker is deliberately not folded into the
truncation triple. True when written; false one commit later.

tsc clean; eslint 0 errors; 1163/1163 across the group unit, group integration
and tool-schema suites.

* fix(group): drop verbose from the MCP surface, fail a superseded bridge closed

Two maintainer-directed findings from the review.

verbose is removed from the group_sync MCP schema and from GroupService, and
kept on the CLI. The parameter never did what either description claimed: the
gates emit workspace-dependency discovery stats and one aggregate manifest line,
not "each cross-link". Worse, they emit them through the server's logger, which
an MCP caller cannot read at all -- so advertising it introduced precisely the
kind of knob this PR exists to delete, in the PR that deletes them. SyncOptions
keeps the field and the CLI keeps --verbose, because a CLI user really can see
that output; its help now says "Show additional sync diagnostics", which is
what it shows. It was added to the MCP schema earlier in this same PR, so there
is no published compatibility burden in taking it back out. A caller that still
sends it is ignored rather than refused: it was never a documented parameter,
and the retired-name guard is reserved for ones this tool actually withdrew.

The second fixes a split-brain the completeness work made materially worse. When
contracts.json commits and the bridge write then fails, the previous database
stays in place describing an EARLIER sync. Until now it kept vouching for
itself, so group_impact could traverse the superseded graph and call its answer
complete while group_contracts reported the advanced registry -- two public
surfaces making contradictory epistemic claims out of one sync. That was
tolerable when the disagreement was about counts. It is not, now that
suppressed-stage makes completeness a correctness property.

markBridgeProvenanceUnknown withdraws the claim without touching the database:
bridgeMetaMatchesFile already gives provenanceUnknown highest precedence and
refuses to vouch for the pair, so cross-repo answers downgrade to a floor until
a sync succeeds. Deliberately not a re-stamp -- the metadata still describes the
database it was written for, and saying otherwise recreates the mis-pairing the
preserve path avoids. Deliberately not a delete -- the old graph is still worth
having as a floor, it just stops being called complete. Best-effort, because it
runs inside a failure handler and must not replace a reported bridge failure
with an unrelated one; the warning now states which of the two happened.

Shared registry+bridge generation identity is the architectural fix and is
deliberately NOT attempted here. This is the PR-sized containment.

Verified by mutation, both directions: neutering the withdrawal turns the new
test red, and a control pins that a healthy sync does not withdraw provenance --
otherwise every successful run would report its own answers as a floor.

tsc clean; eslint 0 errors; 1185/1185.

* refactor(group): apply simplify-pass findings

Four cleanup agents over the last five commits. Efficiency was clean and traced
why: the containment helper is failure-path only, the reason ternary sits after
the fan-out loop, and the tri-state readers run once per artifact read.

The strongest finding was one the diff itself proved. `refreshPreservedBridgeMeta`
enforced the never-persisted rule for `repoListsUnreadable` and
`pairedWithDatabase` with two deletes in its own body, under a comment noting it
was the only code that read metadata and wrote it back. That held exactly as
long as there was one such caller. `markBridgeProvenanceUnknown` made it two,
and inherited nothing. The strip now lives in `writeBridgeMeta`, so every writer
gets it and no future one can forget; `pairedWithDatabase` is the dangerous one,
because persisted it tells every later reader the pair was verified when nothing
verified it.

`group impact` still printed "fan-out stopped early" whenever `truncatedRepos`
was non-empty — but the bridge's incomplete repos are unioned into that list
even when zero crossings were attempted, so a structural gap was reported as a
runtime one, with the only working remedy omitted. That is the same false-cause
shape the contract listing was re-gated for one commit ago, left live one
command over because the new reason was bolted in front of the old branch rather
than replacing the thing it branched on. Now keyed on the reason.

The `?? 'incomplete-sync'` arm in cross-impact was unreachable: reaching it
needs `truncated` true with all three of its inputs false, which
`truncated = runtimeTruncated || bridge.truncated` forbids. Flattened.

Also: a `recordedMatchStages` insert had split `crossRepoCompleteness` from its
own JSDoc; one new test was a strict subset of another; and the bridge-failure
warning interleaved concatenation with a mid-chain ternary.

The new invariant assertion was caught being VACUOUS by mutation before it
shipped — seeded with a valid repo list, `readBridgeMeta` never sets the
reader-only field, so it passed with or without the strip. The fixture now seeds
an unreadable list, and both it and the pre-existing assertion go red when the
strip is removed.

Deliberately skipped, with reasons: a shared `firstTruncated` fold over
`TruncationFields` (the right altitude, but it changes cross-trace's return
assembly and that surface separately documents a 'timeout' rung it cannot emit —
a behavior change, not a cleanup); a reason-keyed `explainFloor` helper across
all four CLI renderers (real, but a four-site refactor); narrowing the persisted
stage vocabulary to a `SuppressibleStage` alias (would be undone by the very
extension the field was modelled as a list to allow); moving `verbose` to
`logger.debug` and deleting `SyncOptions.verbose` (the maintainer explicitly
directed keeping both); and merging the two tri-state readers behind a predicate
(they are adjacent in one file now, so a tightening applies to both by
inspection — the duplication the comment warned about was cross-FILE).

tsc clean; eslint 0 errors; 1164/1164.

* fix(group): address gitnexus-check findings

Seven bot comments across two review rounds; five distinct after dedup. Four
were valid and are fixed, two were already resolved by later commits the bot
had not seen.

The validator could throw from its own error path. `JSON.stringify` is the right
renderer there — it is what distinguishes the string "false" from the boolean,
which is the entire point of the message — but it throws on a BigInt and on a
cyclic object. So a validator promising a structured `{ error }` instead
rejected, and `callTool` is reachable directly, so neither input is
hypothetical. Guarded, keeping the distinction and falling back for the shapes
that cannot serialize.

An unreadable suppression record read as "nothing was suppressed".
`recordedMatchStages` is all-or-nothing by design, so garbage collapses to
`undefined` — and the consumer treated `undefined` as an empty measurement,
throwing that safety away and reporting a registry it could not parse as
complete. Present-but-unreadable now forces the floor, while absent stays
legitimate: a registry written before the field existed has no opinion and
should not be dragged to a floor for it.

Two test-side findings, both real and both invisible to CI because
`tsconfig.json` is src-only. Three `mock.calls[0][1]` accesses did not
type-check against a zero-arg mock, and four assertions read `truncationReason`
/ `riskEpistemic` straight off `CrossRepoCompleteness`, which is a discriminated
union carrying them on one arm. Also removed a `StoredContract` import that went
dead when those fixtures moved to `makeWildcardPair`.

Worth recording: U6 set a test-config gate at 989 errors and later commits
walked it to 994 without anyone re-measuring — the bot caught three of the five.
Now 987, below the original baseline.

Already fixed, not by this commit: the preserve-path stamp the bot flagged
against 6ceac8b1f (fixed in 1fbe0dc6b) and the displaced completeness JSDoc
(fixed in 2d2ef8c47).

Both behavior fixes are mutation-verified: restoring the unguarded stringify
turns the new unserializable-value test red, and a control pins that an absent
record still reads as complete so the fails-closed change cannot pass by forcing
every registry to a floor.

src tsc clean; eslint clean; 1167/1167.

* chore(autofix): apply prettier + eslint fixes via /autofix command

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-27 18:27:32 +01:00
WENJIE HUANG
32b5c0e3fc
feat: add IncludeExtractor for C++ cross-repo include tracking (group) (#1156)
* feat: add IncludeExtractor for C++ cross-repo include tracking (group)

* fix: address CodeQL warnings on include-extractor

- Remove unused HEADER_GLOB constant in include-extractor.ts
- Use fs.mkdtempSync for secure temp dir creation in tests
  (CodeQL: 'Insecure temporary file')

* fix(group): close missing ); in manifest-extractor include branch

The 'include' branch in ManifestExtractor.resolveSymbol was missing
the closing ); for the executor() call, causing a syntax error that
broke ESLint, Prettier, and the full test CI on all platforms.

Reported by Claude PR review on #1156.

* chore: drop test/global-setup.ts + test/vitest.d.ts

Upstream removed these in commit 3f0c74fe (ladybugdb 0.16.0 upgrade).
Commit 3f5d21c5 accidentally restored them during a rebase dance.

* style(group): reformat VALID_CONTRACT_TYPES array to satisfy prettier

Adding 'include' pushed the array over prettier's 100-char limit,
so prettier prefers multi-line. Apply the reformat to unbreak
ci-quality/format job.

* fix(include-extractor): address PR #1156 Claude review findings #3-#7

Claude Deep Review raised 7 findings on the IncludeExtractor. #1/#2
(BLOCKERs) were fixed earlier. This commit closes the remaining five.

#3 HIGH  case-sensitive FS -> provider contract-id collision
  Document the deliberate case-folding trade-off on normalizeIncludePath
  (matches C/C++ convention on Windows/macOS; collapses Foo.h & foo.h on
  Linux). Add a unit test pinning the behavior.

#4 HIGH  suffixResolve short-suffix match silently drops cross-repo include
  When a local file ends with the same basename as an external include
  (e.g. local internal/api.h vs. #include "ext/api.h"), suffixResolve
  returned a bogus local hit and suppressed the cross-repo consumer.
  Replace the suffixResolve lookup inside include-extractor with a
  strict isLocalInclude() that only accepts full-path hits via
  SuffixIndex.get / getInsensitive. Callers of suffixResolve elsewhere
  are unaffected. Add 3 unit tests covering the regression.

#5 MEDIUM regex fallback matched #include inside /* ... */
  Strip block comments before running the fallback regex scan.
  Add a unit test.

#6 MEDIUM meta.source was hard-coded to 'tree_sitter'
  Track the actual extraction path with an extractionSource local and
  write it into meta.source so downstream audits can distinguish
  tree-sitter parses from regex fallbacks. Add 2 unit tests.

#7 MEDIUM missing end-to-end coverage
  Add test/integration/group/include-extractor-sync.test.ts with 3
  cases exercising extractor -> syncGroup -> CrossLink (mocked
  contracts, mixed-case/backslash normalization, real temp repos).

Tests: 21 unit + 3 integration, all green.

* fix(lbug): robust Windows lock acquisition for CI integration tests

LadybugDB's `new Database()` raises `Could not set lock on file` from
local_file_system.cpp synchronously inside the constructor — before any
query is issued, so `withLbugDb`'s query-time retry never sees it. On
Windows CI this surfaces as flaky integration tests due to AV-scanner
holds, libuv handle-release lag, and stale `.wal` sidecars from aborted
prior runs.

This change closes the gap at *open time*:

- `openLbugConnection` now wraps `new lbug.Database()` in a bounded
  busy-retry (5x100ms back-off) inside `lbug-config.ts`. Errors that
  exhaust the budget are tagged via `LBUG_OPEN_RETRY_EXHAUSTED` so
  `withLbugDb`'s outer 3x retry skips re-retrying a freshly-exhausted
  path (eliminates the 3x5=15-attempt / ~6s tail latency).
- For recognized test fixtures only (immediate-parent dir matches a
  known prefix AND resolves under `os.tmpdir()`), one final stale-
  sidecar sweep removes `.wal`/`.lock` and retries once. Production
  paths never enter this branch.
- `safeClose` on Windows runs a bounded `fs.open` probe to absorb
  native handle-release lag; logs a warning if the probe exhausts so
  operators can spot AV interference.
- `isDbBusyError` is now defined in `lbug-config.ts` as the single
  source of truth, re-exported from `lbug-adapter.ts` for compatibility.
- New tests cover open-time retry (happy/retry/exhaust/non-busy/tag),
  stale-sidecar sweep (test-fixture-only, production-rejection,
  preserves-original-error), `isTestFixturePath` direct unit suite
  (accept/reject/traversal/nested/trailing-sep), and
  `waitForWindowsHandleRelease` (openable/ENOENT/no-leak).
- The two new test files are added to vitest's existing serialized
  `lbug-db` project (already `fileParallelism: false`).

Closes the chronic Windows CI flake on lbug-touching integration tests
while preserving the existing single-writable-Database-per-process
LadybugDB contract. No public API surface changed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lbug): drop isDbBusyError re-export, import from lbug-config directly

The re-export from lbug-adapter.ts was a transitional convenience — with
the matcher now living in lbug-config.ts, having two import paths for the
same symbol invites future drift. Updated the two real consumers
(lbug-lock-retry.test.ts, lbug-open-retry.test.ts) to import from
lbug-config directly, removed the re-export equality test (now vacuous),
and refreshed the explanatory comment so it no longer references a
re-export pattern that doesn't exist.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(lbug): silence benign LadybugDB v0.16.1 schema-init lock warnings on Windows

doInitLbug logs "⚠️ Schema creation warning: ... Could not set lock on
file" on every CREATE NODE TABLE call after the first init on a given
dbPath, on Windows. The lock is internal to LadybugDB v0.16.1 and is
resolved before the table is created — same tolerance pattern as the
existing "already exists" filter. Genuine cross-process lock contention
still surfaces on the next operation through withLbugDb's retry, so
filtering at the schema-init catch only suppresses noise, not signal.

Also extend the safeClose Windows handle-release probe to cover the
.wal sidecar (the previous Database's WAL handle was the slowest to
release, surfacing as the schema-query lock contention) and switch the
probe back to 'r+' so it actually detects exclusive locks.

Test loop in lbug-close-handle-release.test.ts simplified to 10 plain
iterations now that the underlying noise is filtered upstream.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(lbug): isDbBusyError review fixes

- Drop redundant `could not set lock` term — already subsumed by `lock`.
- Document the intentionally-broad matcher: graph-DB lock-shaped errors
  ("deadlock", "unlock failed", "lock contention", "could not open lock
  file") are all treated as transient. If a non-transient surfaces,
  tighten the matcher rather than raise the retry budget.
- Add positive test cases covering those lock-shaped strings so the
  intent is visible and a future tightening would deliberately break
  these.
- Fix the open-retry back-off comment: max sleep is 100+200+300+400 =
  1000ms (no sleep after the final attempt), not 1.5s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(group): address PR #1156 follow-up review findings

Addresses two blockers and two mediums from the deep review.

BLOCKER 1: Windows CI ENOTEMPTY in sync.test.ts
  After this PR added writeBridge() to syncGroup, the existing test
  "writes registry to groupDir when skipWrite is false" fails on
  windows-latest. LadybugDB's checkpoint thread briefly outlives
  closeBridgeDb, holding a Win32 lock on bridge.lbug; the test's
  fs.rmSync then fails with ENOTEMPTY. Switched the test cleanup to
  cleanupTempDir from test/helpers/test-db.ts which already tolerates
  EBUSY/EPERM/EACCES/ENOTEMPTY with bounded retries — same pattern
  used elsewhere for LadybugDB-touching tests.

BLOCKER 2: Graph provider absolute-path bug
  extractProvidersGraph queried File.filePath from the LadybugDB graph
  but never stripped the repo root, so provider contract IDs ended up
  as include::/abs/path/foo.h while consumers emitted include::foo.h.
  These never matched through runExactMatch — silently producing 0
  cross-links for any indexed C++ repo (the primary use case).
  Now passes repoPath into extractProvidersGraph and applies
  path.relative(); rows that resolve outside repoPath (stale absolute
  paths from another machine, system headers somehow indexed) are
  dropped instead of polluting the registry.

MEDIUM: `../` relative includes produce spurious noise
  `#include "../foo.h"` is almost always intra-repo, but the suffix
  index can never match a `..`-prefixed path so it became a consumer
  contract no provider could satisfy. Now skipped before matching;
  covers both forward-slash and backslash forms.

MEDIUM: writeBridge error in sync.ts propagates uncaught
  contracts.json is the canonical source of truth and was just written
  successfully when writeBridge runs. A bridge-only failure (disk full,
  schema error, permission denied) shouldn't mask the registry. Wrapped
  writeBridge in try/catch with a logger.warn surfacing the path and
  recovery instructions.

Tests added:
  - extractProvidersGraph repo-relative ID generation (stub Cypher
    executor returns absolute paths)
  - extractProvidersGraph drops rows whose path resolves outside repo
  - `../foo.h` forward-slash skip
  - `..\foo.h` backslash-form skip

Skipped findings:
  - canExtract() removal (#5, low): canExtract is part of the
    ContractExtractor interface; every other extractor implements the
    same `return true` shape. Removing it from IncludeExtractor would
    break the interface contract — keeping for consistency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(group): close PR #1156 Codex adversarial findings

Two HIGH findings from the Codex adversarial review on
feat/group-include-extractor:

1. Default-on extraction silently changes existing groups (BLOCKER)
   DEFAULT_DETECT.includes was true, so any pre-existing group.yaml
   that omits the new field would gain a wave of include::* contracts
   on the next sync after upgrade. Flipped to false (opt-in). The
   integration test already declares includes: true explicitly so it
   survives unchanged; the unit extractor tests bypass parseGroupConfig
   entirely; the sync test uses extractorOverride. Only config-parser
   needed regression tests covering omitted/explicit/false variants.

2. IncludeExtractor scans outside the indexed file universe (BLOCKER)
   The extractor was running glob('**/*', { ignore: STANDARD_IGNORES })
   twice with a hand-rolled 9-pattern list, no .gitignore/.gitnexusignore
   honoring, and no max-file-size cap. That meant File:<path> contracts
   could appear for files ingestion would never index, producing
   cross-links group impact cannot fan out to (silent false-negatives).
   Refactored to a single discoverIndexableFiles() helper that mirrors
   walkRepositoryPaths exactly: createIgnoreFilter + getMaxFileSizeBytes,
   one discovery pass shared by provider and consumer paths. Dropped
   STANDARD_IGNORES and SOURCE_GLOB entirely.

   third_party and 3rdparty (the C/C++ vendored-deps conventions) were
   in the local ignore list but not in the canonical DEFAULT_IGNORE_LIST
   used by ingestion. Folded both into the canonical set rather than
   keep a parallel list — the whole point of the Codex finding is that
   two file-discovery implementations drift. Single source of truth.

Tests: 5 new regression tests for the discovery alignment (.gitignore,
.gitnexusignore, max-file-size on both provider and consumer paths)
plus 4 for the opt-in default. All 30 include-extractor tests + the
494-test group suite + ignore-service tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): apply autofix feedback

ce-code-review surfaced 6 safe_auto findings on commit a9936a9b:

- T1 (testing, P2): the sync.ts:174 gate was untested with includes:false.
  Added a sync-level test mirroring the existing thrift-off pattern at
  sync.test.ts:545, asserting zero include contracts when the gate is
  disabled in a real syncGroup call.

- T3 (testing, P3): third_party and 3rdparty entries in DEFAULT_IGNORE_LIST
  had no regression test. Added both to ignore-service.test.ts's
  dependency-directories it.each block.

- M1 (maintainability, P3): discoverIndexableFiles JSDoc lacked a
  fork-warning relative to walkRepositoryPaths. Added a MAINTENANCE
  note explaining why the duplication is tolerated and the contract
  the two implementations must keep.

- M2 (maintainability, P3): thrift-extractor still hand-rolls its
  ignore array with no signal that DEFAULT_IGNORE_LIST additions
  silently do not apply there. Added TODO(#1156-followup) comments
  above both call sites.

- M3 (maintainability, P3): SOURCE_EXTENSIONS duplicated the four
  HEADER_EXTENSIONS entries with no expressed subset relationship.
  Spread HEADER_EXTENSIONS into SOURCE_EXTENSIONS so future header-
  extension additions propagate.

- C1+T4 (correctness+testing, P3, cross-reviewer corroborated):
  discoverIndexableFiles swallowed all fs.stat errors silently,
  including EACCES/EMFILE/EIO. Narrowed the catch to ENOENT (the
  documented benign glob/stat race) and added a logger.warn for
  any other code so operators can spot permission/resource issues.

All 629 tests pass; typecheck + prettier clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(group): use retryRename in writeContractRegistry to absorb Windows EPERM

`storage.ts:62` used raw `fsp.rename` for the contracts.json atomic swap.
On Windows, AV scanners and concurrent renames briefly hold the
destination handle between rename calls, surfacing as EPERM/EBUSY.
The `insecure-tempfile.test.ts > concurrent writes do not collide`
test was flaking with `EPERM: operation not permitted, rename` on
windows-latest CI.

`bridge-db.ts` already has a battle-tested `retryRename(src, dst, 3)`
helper used at six call sites for exactly this pattern. Reusing it
here keeps the Windows-rename policy single-source-of-truth across
the group package.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(group): drop macro-style #include from consumer contracts

Tree-sitter's `(_) @import.source` wildcard matches the identifier node
of `#include PLATFORM_HEADER`, so the cleaned value `PLATFORM_HEADER`
slipped past the system-header / `..` filters and was emitted as a
permanently orphaned consumer contract (no file is named after a macro
identifier, so no provider can ever match). Add a shape guard that
skips cleaned values lacking both a path separator and an extension
dot, plus regression tests for single and multi-macro files.

Also document `IncludeExtractor.canExtract()` as unused by sync.ts
(gated via `config.detect.includes` instead) and kept solely for
ContractExtractor interface uniformity.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: HuangWenjie <zhoudeng.hwj@alibaba-inc.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 09:31:59 +01:00
1PLee
05ca80ea30
feat(ingestion): add thrift contracts impl (#1234) 2026-05-06 09:19:10 +01:00
Ivan Uzun
46586a8319
fix(group): add configurable cross-link path exclusions to reduce false positives (#1093)
* fix(group): add configurable cross-link path exclusions to reduce false positives

Add matching.exclude_links_paths and matching.exclude_links_param_only_paths
to group.yaml config. These filter out noisy HTTP contracts (health checks,
param-only catch-all routes) from cross-link matching while preserving them
in the contract registry for documentation purposes.

Defaults are empty/false for backward compatibility — no behavior change
unless the operator explicitly configures exclusions.

* fix(group): address review findings — filter unmatched, normalize trailing slash, add tests

- Excluded contracts no longer inflate SyncResult.unmatched (isNoisy guard)
- pathPart in buildNoisyContractFilter strips trailing slashes before comparison
- 8 new unit tests for buildNoisyContractFilter covering all code paths
- Config-parser test asserts defaults for new matching fields

* fix(group): normalize configured exclusion paths and add root-path test

- Strip trailing slashes from configured exclude_links_paths at Set-build
  time so root path '/' (which normalizes to '') matches correctly
- Add test: exclude_links_paths: ['/'] suppresses http::GET::/ contracts
- Add new matching fields as commented examples in fixture group.yaml (DoD §2.4)

* docs(group): document exclude_links_paths and exclude_links_param_only_paths config fields

Add JSDoc to MatchingConfig interface, update the microservices guide
YAML example and field notes, and scaffold the new fields (commented out)
in the group create template.
2026-04-28 08:22:14 +01:00
ivkond
52277247fe feat(group): add group infrastructure and contract matching
Core foundation for repository group analysis:
- Type system: ContractType, ExtractedContract, StoredContract, CrossLink
  with optional `service` field for intra-repo matching
- Config parser for group.yaml (repos, detection flags, matching thresholds)
- Contract registry storage with atomic writes
- Exact matching engine with per-type normalization (HTTP, gRPC, topic)
  and intra-repo support (different services within same repo can match)
- Extract LadybugDB pool-adapter from MCP backend for reuse by sync pipeline
- Git staleness checker for group status reporting

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 00:39:43 +03:00