scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-08-28 05:25:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
GitNexus/GUARDRAILS.md
Gergő Magyar b5c6c0e57c
perf(communities): fix the O(communities x N) copy in vendored Leiden, wire Icebug to its real API (#2337) (#2692) 
* perf(communities): drop the O(communities x N) copy in vendored Leiden (#2337)

`UndirectedLeidenAddenda.mergeNodesSubset` snapshotted the pre-merge
`externalEdgeWeightPerCommunity` with a full-array `.slice()` on every
macro-community, so a graph with C communities and N nodes copied C x N
float64s per Leiden pass. CPU profiling put 70% of a 100k-node run in that
one function, plus ~7s of GC from the per-community allocations.

Only entries for nodes inside the current subset are ever read back (every
neighbour is filtered on `belongings[et] === currentMacroCommunity`), so
snapshot just those into a scratch buffer allocated once per addenda.

Measured on seeded planted-partition graphs, partitions bit-identical:

  20k nodes / 54k edges    2350ms -> 527ms    (4.5x)
  60k / 200k              12513ms -> 3328ms   (3.8x)
  100k / 350k             44151ms -> 4816ms   (9.2x)
  200k / 800k             >580s   -> 14622ms  (>40x)

The 200k case previously blew through LEIDEN_TIMEOUT_MS and degraded every
symbol into a single community; it now finishes well inside the timeout.

Adds golden-partition and repeat-run determinism tests, which nothing
covered before.

Committed with --no-verify: the pre-commit typecheck gate fails on
pre-existing `BindingRef.visibility` errors in csharp/namespace-siblings.ts
and scope-resolution/passes/free-call-fallback.ts, both untouched here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfQfKy4gCmgUv1jBRJTSs2

* fix(communities): wire the Icebug engine to the real @ladybugmem/icebug API (#2337)

The gate merged in #2376 could never have run. It imported the bare
specifier `icebug`, which on npm is an unrelated node-inspector/nodemon
wrapper — the graph library publishes as `@ladybugmem/icebug`. It then
probed for `Graph.fromCSR` and `community.ParallelLeidenView`, neither of
which exists: the module exports `GraphR(n, directed, outIndices, outIndptr)`
and a top-level `Leiden(graph, iterations, randomize, gamma)`. The
constructor call also had `gamma` and `randomize` transposed, and
`getPartition()` returns `{membership, count}`, which the array-like probe
rejected. Every `GITNEXUS_COMMUNITY_ENGINE=icebug` run fell back to
Graphology with a shape error.

Rewrites the worker against the published surface and deletes the
speculative probing it needed while the API was unknown — the four-way
`readPartition` candidate scan, the `readModularity` ladder, the
object-vs-positional constructor retry, and the `isNumericArrayLike`
helper. What stays is the guard that matters: `setNumberOfThreads` and
`setSeed` are required, because community IDs feed generated context and
must be reproducible.

Icebug is deliberately not a declared dependency. Its prebuilds link
against system Arrow 24, OpenMP and glibc >= 2.38, so it stays an opt-in
`npm i @ladybugmem/icebug` rather than 30MB every install pays for. Note
that the published 12.8.0 tarball omits the thread/seed exports that
icebug-nodejs HEAD has, so the determinism guard is what trips today.

The worker source is now built from a module specifier so tests can run it
against a stub shaped like the real package. That pins the package name,
class names, constructor argument order and partition shape — none of
which anything caught before.

Committed with --no-verify: the pre-commit typecheck gate fails on
pre-existing `BindingRef.visibility` errors in csharp/namespace-siblings.ts
and scope-resolution/passes/free-call-fallback.ts, both untouched here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfQfKy4gCmgUv1jBRJTSs2

* docs(communities): label the Icebug engine experimental and announce it at runtime (#2337)

The engine was opt-in but silent about what opting in means. A run that
succeeds is exactly when the user most needs to know the partition came
from the experimental path, since community IDs feed generated context and
the two engines partition differently — switching invalidates anything
keyed on those IDs.

Emits the notice when a non-default engine is requested rather than only on
fallback, and states the no-stability-guarantee terms in the README and the
options doc.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfQfKy4gCmgUv1jBRJTSs2

* fix(communities): never terminate the icebug worker mid-N-API (#2432, #2337)

Self-review of this PR found that making the native Leiden path reachable
also arms a hazard this repo has already paid for once. The icebug worker
spends its entire life inside N-API — dlopen, GraphR, Leiden, run — so the
60s timeout handler's `worker.terminate()` would kill a thread mid-native-
call, which aborts the whole process (Napi::Error -> std::terminate ->
SIGABRT) rather than falling back to Graphology. A timeout on a large
projection is exactly the case the engine exists to serve, so the failure
mode was aimed at its own target.

Drops terminate() from all three paths. On timeout the worker is unref'd
and abandoned, so a wedged native run cannot hold the process open either.
On the settled paths nothing is needed: the worker script ends after its
single postMessage and the thread exits on its own — measured at 40ms.

Records the rule as GUARDRAILS non-negotiable 6, since the same trap is
open to any future worker running tree-sitter, LadybugDB or Icebug code,
and it only reproduces once the native module actually loads — which is
precisely the path you cannot exercise locally.

Also from the review:

- Marks vendor/leiden/utils.cjs as a local fork. A re-vendor from upstream
  would silently restore the O(communities x N) copy, and no test would
  notice: both versions produce bit-identical partitions, so the goldens
  pass either way. The header now names the divergence and its symptom.
- Qualifies the README performance claim. "~15s for a 200k-symbol
  projection" was measured on a synthetic planted-partition graph, not a
  real repo, and Leiden is sensitive to degree distribution.

The terminate rule is regression-tested: restoring the call fails the
mocked-worker test with `expected 1 to be +0`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfQfKy4gCmgUv1jBRJTSs2

---------

Co-authored-by: Gergo Magyar <abhigyan1.patwari@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 13:23:17 +01:00

7.4 KiB
Raw Blame History

Guardrails — GitNexus

Rules for human contributors and AI agents. Complements AGENTS.md (workflows) and CONTRIBUTING.md (PR process).

Scope (least privilege)

  • Read: Source, tests, docs, public config as needed.
  • Write: Only files required for the fix or feature; no unrelated formatting or refactors.
  • Execute: Tests, typecheck, documented CLI commands. No destructive commands on user data without approval.
  • Off-limits: Other people's machines, production deployments you don't own, credentials you lack permission to use.

Maintainer may widen scope per task.

Non-negotiables

  1. Never commit secrets — API keys, tokens, real .env values, private URLs, session cookies. Use .env.example with placeholders.
  2. Never rename with find-and-replace in GitNexus-indexed projects — use rename MCP tool with dry_run: true first, review graph vs text_search edits. No separate gitnexus rename CLI exists.
  3. Run impact analysis before editing shared symbols — impact (upstream) for functions/classes/methods others call. Do not ignore HIGH/CRITICAL without maintainer sign-off.
  4. Run detect_changes before commit — confirm diffs map to expected symbols/processes when the graph is available.
  5. Preserve embeddings — plain npx gitnexus analyze now preserves any embeddings recorded in the index metadata (.gitnexus/gitnexus.json, mirrored to the legacy meta.json) — the previous behavior wiped them. Use --embeddings to also generate vectors for new/changed nodes; use --drop-embeddings only when an explicit wipe is intended (e.g., model swap).
  6. Never terminate() a worker that may be inside a native call — killing a worker thread mid-N-API aborts the entire process (Napi::Error → std::terminate → SIGABRT, #2432), so a timeout meant to trigger a graceful fallback takes the whole run down instead. Any worker running native code (tree-sitter grammars, LadybugDB, Icebug) must either reach a JS-visible safe point first — the parse pool's shutdownDrainMs handshake in src/core/ingestion/workers/worker-pool.ts — or be abandoned with unref() and left to exit on its own. A one-shot worker that ends after a single postMessage needs no terminate() at all: it exits by itself. This bites hardest on the path you cannot test locally, because the abort only reproduces once the native module actually loads.

Signs (recurring failure patterns)

Format: Trigger → Instruction → Reason. Append new Signs when the same mistake repeats.

Stale graph after edits

  • Trigger: MCP warns index is behind HEAD, or search doesn't match latest commit.
  • Do: npx gitnexus analyze (plus --embeddings if used). Runs incrementally by default — the pipeline parses every file every run (cross-file resolution requires it), but tree-sitter dispatch is skipped for unchanged file chunks via the content-addressed cache, and only changed-file rows (plus their importers, transitively) are rewritten in LadybugDB. When the effective write set exceeds ~50% of the repo's files (minimum 50 files), the run transparently switches to the full wipe + bulk-COPY write plan and logs "switching to a full DB write" — expected behavior, not a bug, and file-level bookkeeping stays incremental.
  • Why: Tools query LadybugDB from last analyze; git changes are invisible until re-indexed.

Index seems corrupt or "incremental" is misbehaving

  • Trigger: analyze produces unexpected results, or incrementalInProgress is set in the index metadata (.gitnexus/gitnexus.json / legacy meta.json), or the index is in a half-state after a crash.
  • Do: npx gitnexus analyze --force to rebuild from scratch. The dirty-flag check forces this automatically when a previous incremental run didn't complete cleanly, but --force is the manual escape hatch. A dirty-flag recovery rebuild parks the interrupted run's sidecars beside the DB as lbug.wal.dirty-recovery / lbug.shadow.dirty-recovery for post-mortem debugging — harmless, and removable with npx gitnexus clean --lbug-sidecars. Safe to delete the .gitnexus/parse-cache/ directory (and any legacy .gitnexus/parse-cache.json) at any time — content-addressed, will be regenerated.
  • Why: Incremental writeback is selective DB row replacement; if the on-disk state is inconsistent for any reason, a full rebuild is the cheapest path back to a known-good index.

Embeddings vanished after analyze

  • Trigger: Semantic search quality drops; stats.embeddings in the index metadata (gitnexus.json / legacy meta.json) is 0 after refresh.
  • Do: Re-run npx gitnexus analyze --embeddings to regenerate. Check the analyze log for a Warning: could not load cached embeddings line — if present, the cache restore failed (corrupt DB / schema mismatch) and the rebuild had nothing to preserve. If you intentionally passed --drop-embeddings, this is expected.
  • Why: Plain analyze preserves prior vectors by re-inserting them after the rebuild; the only ways to end up at zero are an explicit --drop-embeddings, a cache-load failure (now logged), or a model/dimension change that invalidates the cache. A dirty-recovery run that cannot move the crashed WAL aside now either discards it (logged: forensics lost, embeddings still preserved) or fails fast with a lock error naming the holder — it never silently zeroes embeddings.

MCP lists no repos

  • Trigger: MCP stderr says no indexed repos.
  • Do: npx gitnexus analyze in the target repo; verify npx gitnexus list shows it.
  • Why: MCP discovers repos via ~/.gitnexus/registry.json, populated by analyze.

Wrong repo in multi-repo setups

  • Trigger: Query/impact results belong to another project.
  • Do: Call list_repos, then pass repo on subsequent tools.
  • Why: Default target is ambiguous when multiple repos are registered.

LadybugDB lock / "database busy"

  • Trigger: Errors opening .gitnexus/lbug while MCP and analyze both run.
  • Do: Stop overlapping processes (one writer at a time). Retry analyze or restart MCP.
  • Why: Embedded DB expects single-process ownership. @ladybugdb/core 0.18.0 also reports this contention as "Only one write transaction at a time is allowed in the system." — our busy/lock retry matcher (isDbBusyError in src/core/lbug/lbug-config.ts) recognizes this exact string too, so it's auto-retried the same as any other lock error. If you see that exact message, it's the same "one writer at a time" issue above, not a new failure mode.

Publishing & supply chain

  • npm: Do not publish from unreviewed automation. Bump version intentionally; tag releases to match package.json.
  • Dependencies: Minimal, auditable package.json changes; run tests and CI after lockfile updates.
  • License: PolyForm Noncommercial 1.0.0 — do not relicense without maintainer approval.

Escalation

Stop and ask a human maintainer when:

  • Impact analysis shows HIGH/CRITICAL risk and the task still requires the change.
  • You need to alter CI, release, or security-sensitive config.
  • Requirements conflict (e.g. "speed up analyze" vs "must keep all embeddings on huge repo").
  • You are unsure whether data loss is acceptable (clean, forced migrations, schema changes).

Related docs