Commit graph

6 commits

Author SHA1 Message Date
1PLee
05ca80ea30
feat(ingestion): add thrift contracts impl (#1234) 2026-05-06 09:19:10 +01:00
Christian C. Berclaz
7cce07b419
feat(group): workspace extractors for Node, Python, Go, Java, Elixir (#1260)
* feat(group): auto-discover Node/TS workspace cross-package contracts

Scan package.json dependencies and ES/CJS imports to find PascalCase
type exports crossing workspace package boundaries. Same pipeline as
Rust workspace extractor — emits GroupManifestLink[] with type:custom.

Supports: ES named imports, default imports, CommonJS destructured
require, scoped packages (@org/pkg), subpath imports, aliased imports.
Filters to PascalCase names only (types/classes, not functions).

* feat(group): auto-discover Python workspace cross-package contracts

Scan pyproject.toml/setup.py dependencies and `from <pkg> import`
statements to find PascalCase type exports crossing workspace package
boundaries. Handles hyphenated names (PEP 503 normalization),
submodule imports, aliased imports, and optional-dependencies.

* feat(group): auto-discover Go workspace cross-module contracts

Scan go.mod require/replace directives and Go source files for
exported PascalCase type usage (pkg.TypeName) crossing module
boundaries within a group. Handles block syntax, subpackage
imports, and local replace directives.

* refactor(group): extract workspace discovery orchestrator from sync

Move per-ecosystem workspace extractor calls into a single
discoverWorkspaceLinks() orchestrator. Reduces sync.ts from 295
to 264 lines and gives a clean extension point for adding
more ecosystem extractors.

* feat(group): auto-discover Java/Kotlin workspace cross-project contracts

Scan Maven pom.xml and Gradle build files for inter-project deps,
then match Java/Kotlin import statements against known group-internal
base packages. Supports Maven dependency blocks, Gradle coordinate
and project() dependencies, static imports, and Kotlin files.

* feat(group): auto-discover Elixir workspace cross-app contracts

Scan mix.exs deps and Elixir source files for alias directives and
direct module references crossing OTP app boundaries. Handles
umbrella deps (in_umbrella), git/path deps, grouped aliases
(alias MyApp.{ModA, ModB}), underscore-to-PascalCase app name
mapping, and collapses nested submodules to top-level contracts.

* fix(group): apply PR review fixes to all workspace extractors

Address review findings from PR #1256 across Node, Python, Go, Java,
and Elixir extractors:
- Replace hardcoded IGNORE sets with shared IgnoreService
  (shouldIgnorePath + loadIgnoreRules) to honor .gitnexusignore
- Qualify contract names with provider identifier to prevent
  contractId collisions across providers
- Warn and skip duplicate project/module/app names
- Update all test assertions for qualified contract format

* fix(workspace): address review findings and fix CI

- Fix prettier formatting on Rust workspace extractor files
- Fix double readRegistry() call in syncGroup (hoist to function scope)
- Fix console.warn spy leak in duplicate crate test (try/finally)
- Add sync-level integration tests: workspace_deps true/false gating,
  Rust and Node link discovery through syncGroup orchestrator (3 tests)

* style(workspace): fix Prettier formatting on all workspace extractors

* fix(workspace): strip qualified prefix in custom contract resolution, default workspace_deps to false

resolveSymbol for custom contracts now strips the "provider::" prefix
before querying graph nodes, so workspace-generated contracts like
"mathlex::Expression" correctly resolve to the "Expression" symbol.

Change workspace_deps default from true to false for safe rollout —
existing groups won't silently gain 6-ecosystem scans on upgrade.

* fix(workspace): address medium review findings from PR #1260

- Elixir: strip comment lines before direct module reference scan to
  prevent false positives from commented-out module references
- Go: use full module path for contract naming to avoid basename
  collisions between repos with identical last path segments
- Sync tests: replace toBeGreaterThanOrEqual with exact toHaveLength
  assertions per DoD §2.7
- Add workspace_deps: false to makeConfig helper for type correctness
- Add Elixir test proving comment-only references do not emit links

* fix(workspace): address second-round medium review findings

- Go: add test asserting aliased imports produce 0 links, guarding the
  V1 false-negative boundary at the assertion level
- Elixir: add code comment documenting that contracts use full module
  names without appName:: prefix and that resolveSymbol resolution
  depends on Elixir indexer storing fully-qualified names

* fix(workspace): eliminate regex backtracking in pyproject.toml parser

CodeQL flagged exponential backtracking in the [project] name regex.
Replace [^\[]*?\n (ambiguous lazy quantifier) with [^\n\[]*\n (atomic
per-line match that still stops at section boundaries).

* fix(test): use mkdtempSync for secure temp dir creation

CodeQL flagged insecure temporary file creation (High) in sync.test.ts.
Replace path.join(os.tmpdir(), predictable-name) + mkdirSync with
fs.mkdtempSync which creates temp dirs atomically with random suffix,
preventing symlink race conditions.
2026-05-04 09:43:21 +01:00
Christian C. Berclaz
b9a17f553d
feat(group): auto-discover Rust workspace cross-crate contracts (#1256)
Some checks are pending
Release Candidate / Check if release candidate should run (push) Waiting to run
Release Candidate / ci (push) Blocked by required conditions
Release Candidate / Publish release candidate to npm (push) Blocked by required conditions
Release Candidate / Build & Push RC Docker images (push) Blocked by required conditions
2026-05-03 01:43:22 +01:00
Jonas Vanderhaegen
385ee037bd
[group/sync] Fix ManifestExtractor never called — config.links always produced 0 cross-links (#827)
* fix(group/sync): wire ManifestExtractor into syncGroup pipeline

ManifestExtractor was fully implemented in extractors/manifest-extractor.ts
but never imported or called in sync.ts. As a result, any links declared in
group.yaml were parsed and validated by config-parser.ts but silently dropped
— config.links was always an empty dead-end as far as syncGroup was concerned.

Changes:
- Import ManifestExtractor in sync.ts
- Call extractFromManifest(config.links, dbExecutors) inside the outer try
  block, after all repos are processed but before the finally closes the DB
  pools (symbol resolution via resolveSymbol requires open executors)
- Collect the resulting contracts into autoContracts and the cross-links into
  a separate manifestCrossLinks array
- Merge manifestCrossLinks into the final crossLinks alongside runExactMatch
  results

Without this fix, users who declare explicit service dependencies in
group.yaml links (the documented workaround for HTTP clients that use absolute
URLs and are invisible to the auto-extractors) get 0 cross-links regardless of
what they configure.

* test(group/sync): cover manifest links producing cross-links

Add a unit test that asserts config.links entries produce contract pairs
and a manifest cross-link (matchType: 'manifest') via syncGroup.

Also refactors the manifest extraction call to sit outside the else/try
block so it runs regardless of extractorOverride arity — makes the code
testable without mocked DB pools and ensures links work when callers supply
a zero-arity override (e.g. in tests or programmatic usage).

* style: prettier format sync.ts and sync.test.ts

Also removes the stray empty line in the finally block (noted in review).

* fix(group/sync): dedupe cross-links and warn on dangling manifest repos

Addresses review feedback on PR #827:

1. Dedupe cross-links. Manifest contracts participate in runExactMatch, so a
   manifest-declared link also emitted a duplicate matchType:'exact' CrossLink
   for the same endpoint pair. Dedupe by (from, to, type, contractId) and
   prefer manifest (operator-declared intent).

2. Warn on dangling repos. When a manifest link references a repo not in
   config.repos, log a warning. Synthetic UIDs keep the cross-link
   deterministic, but the operator probably meant something else.

3. Tests:
   - Assert no duplicate 'exact' CrossLink is emitted alongside the manifest one.
   - Assert synthetic UID format when no DB executors are available.
   - New test: dangling manifest repo still produces a cross-link + logs a warning.

* perf(group/manifest): parallelize and memoize symbol resolution

Previous implementation ran 2N sequential Cypher round-trips per
manifest (one for provider side, one for consumer, awaited in-order
per link). For manifests with tens of links this dominated syncGroup
latency in groups with many declared cross-repo contracts.

Changes:
- Resolve provider + consumer in parallel per link (Promise.all).
- Resolve all links in parallel (outer Promise.all over links.map).
  Each repo's executor pool is independent, so cross-repo fan-out
  scales with the number of distinct repos in the manifest.
- Memoize by (repo, type, contract). Manifests frequently declare
  the same contract from both directions or across sibling groups,
  so duplicate triples now hit the DB once instead of 2× per link.

Correctness:
- resolveSymbol is a pure LIMIT 1 read, so caching + concurrent
  invocation is safe.
- Iteration order over links is preserved in the final
  contracts / crossLinks arrays — result shape is identical.

Test:
- New test asserts that two links sharing (repo, type, contract)
  produce exactly one DB call per distinct repo-tuple.

---------

Co-authored-by: jonasvanderhaegen-xve <>
Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
2026-04-15 09:14:09 +01:00
ivkond
255e3e79eb fix(group): address 4 HIGH-priority issues from PR #626 review
1. Path traversal via group name — add validateGroupName() with regex
   [a-zA-Z0-9][a-zA-Z0-9_-]*, called in getGroupDir (defense in depth)

2. gRPC proto regex can't handle nested braces — replace serviceRe with
   extractServiceBlocks() brace-depth counter (init depth=1, skip
   malformed protos)

3. Service boundary detector directory exclusions — add EXCLUDED_DIRS
   set (vendor, target, build, dist, __pycache__, .venv, venv, .tox,
   .mypy_cache, .gradle, .mvn, out, bin) replacing inline node_modules

4. Double-close of LadybugDB pools — remove blanket closeLbug() from
   cli/group.ts; sync.ts per-id cleanup is sufficient

Tests: 22 new tests across 5 files. Full suite: 4706 passed, 0 failed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 12:55:33 +03:00
ivkond
4fed097abb feat(group): add sync pipeline, CLI, MCP tools, and monorepo fixture
Wire extractors into the sync pipeline with service boundary detection.
GroupService provides high-level API for all group operations.

- Sync pipeline: orchestrates extraction (HTTP, gRPC, topics) with
  service boundary assignment and exact matching
- GroupService: groupList, groupSync, groupContracts, groupQuery,
  groupStatus (groupImpact deferred to cross-repo follow-up PR)
- CLI: group create/add/remove/list/sync/contracts/query/status
- MCP tools: group_list, group_sync, group_contracts, group_query,
  group_status
- Monorepo fixture: 3 services (auth/orders/gateway) connected via
  gRPC + Kafka + HTTP — all intra-repo cross-links discovered
- Documentation: CLI commands and MCP tools added to both READMEs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 00:40:31 +03:00