Commit graph

10 commits

Author SHA1 Message Date
Gergő Magyar
cc6225d2cb
feat(analyze): make indexing resource-bounded and resumable (#3829)
* docs(plans): add resumable analyze plan

* perf(bench): measure resumable storage primitives

* feat(embeddings): bound and persist work selection

* feat(embeddings): persist resumable vector jobs

Keep graph readers independent of vector writes. Acknowledge units only after complete chunk verification and a native checkpoint.

* feat(embeddings): stream resumable job units

* feat(storage): add generation publication

Publish durable graph generations independently of vector jobs and gate readers on matching identities. Retain the legacy graph inode under a private name so older binaries cannot serve a stale canonical graph. Coordinator integration follows separately.

Refs #3598

* feat(embeddings): coordinate durable jobs

Join published graph selection, durable work, bounded native handles and vector publication behind one inactive controller. Limit optional HTTP response decoding within the job reservation and admit supported large inputs under larger budgets.

Refs #3598

* feat(analyze): pass phase resource allocations

Carry per-run worker heaps and native pools without changing legacy defaults. Hold parser reservations through safe native exit; refuse database reallocation after an unverified close.

* feat(embeddings): reserve sidecar resources

Pin each durable job to an admitted model heap and CPU config. Retain native allocations after caller cancellation until response or process exit, and reject phase handoff when release is unverified.

* feat(status): separate graph and vector readiness

Report embedding debt independently from graph content freshness. Read bounded durable progress without loading native runtimes, gate stale vector capability stamps, and allowlist additive phase counters on public SSE and ops payloads.

* fix(embeddings): recover interrupted job setup

Recover an owned empty native catalog after interrupted bootstrap, and rebuild damaged failure accounting without repeating completed provider work. Preserve foreign artifacts and propagate filesystem errors.

* fix(resources): verify native pool release

Keep failed and concurrent native closes tracked until release succeeds. Refuse another reserved allocation while cleanup is unverified, including failed connection construction. Let graph readers tear down after result-close errors.

* fix(resources): drain graph before phase admission

Wait for session work and verify the prior main graph pool has closed before parser or embedding admission. Recheck cancellation after the drain so cancelled jobs do not read or mutate job metadata.

* docs(plans): revise resumable analyze seams

* perf(bench): measure paired analyze runs

* feat(index): publish logical graph generations

Retain private database files across incremental generations and fence prepared publication against superseding writers. Exit-only producers must exit before publication; interrupted in-place writes remain explicitly dirty.

* feat(index): guard graph process handoff

Wait for actual process exit and reacquire the ordinary slot lock before
publishing the prepared attempt. Recheck placement and the registered
name, and retain graph discoverability after cancellation.

Add a bounded nonterminal worker receipt without serializing the live
graph. Production phase activation remains separate.

* feat(cli): defer success across graph handoff

Finish live-graph skill generation before the child hands off. Transfer
only bounded counters, identity and the pre-run self-commit safety flags;
wait for IPC delivery before returning for native process exit.

Reject missing supervisors and mismatched prepared results before skill
writes. Finalization, self-commit and success remain deferred until the
production supervisor completes embedding; core activation follows the
pending-artifact and resource contracts.

Refs #3598

* feat(embeddings): rebase pending paid artifacts

Separate graph job identity from mutable artifact ownership. Keep paid
compatible groups across source edits using bounded revalidation and
selection pruning; retain one sealed fallback for the active space.

* feat(embeddings): preserve historical coverage

* feat(embeddings): reuse verified legacy vectors

Retain a directly referenced legacy source across pending-artifact
rebases. Stream checkpoint exclusions and build a native metadata index
without copying vectors or retaining corpus-sized JavaScript maps.

Require recorded provider identity and provable complete input groups.
Keep unknown provenance explicit, source artifacts immutable, compatible
relocations free of provider calls, and new inputs in normal batches.

The controller remains inactive until coordinated writer activation.

Refs #3598

* feat(analyze): account for supervised phases

Charge parent memory once per phase. Carry allowlisted progress through
worker IPC and preserve counters across logs without completing the job.

Refs #3598

* fix(storage): retain owned analysis generations

Keep private generation files in place through shared-store preparation
and publication. Retain prepared publications and pending embedding work
across the unlocked process handoff, including work in branch slots.

Refs #3598

* fix(embeddings): share pinned vector readers

Share immutable vector file identities so sibling hard links retain one
native reservation. Keep the handle alive until its final reader closes,
including failed initialization and replacement of another reader's path.

Refs #3598

* feat(storage): share completed vector artifacts

Pin immutable vectors independently in checkout slots and shared graph snapshots. Keep mutable jobs with their owning checkout; resume interrupted adoption without overwriting unverified files.

Refs #3598

* feat(analyze): retain caches across phase handoff

Keep prepared staging during lock reacquisition and share cache
publication between the graph writer and exit continuation. Bound the
filesystem handoff to chunk keys; never copy live cache contents.

Refs #3598

* fix(analyze): retain owned generation routing

Keep prepared and pending work under its original slot lock during shared-store adoption. Preserve private generation filenames during incremental writes without copying the graph.

Drain model and native resources only when graph work actually starts, including FTS repair. Leave the warm no-op path unchanged.

Validated 158 tests, source build, test typecheck, formatting and lint. Complete graph gates cover 29 symbols in two files and the expected coordinator flow. End-to-end embedding activation and performance acceptance remain pending.

* feat(analyze): wire resumable phase execution

Publish verified graphs before embedding and hand off resource ownership
across CLI, watch, auto-sync, and HTTP phases. Keep completed vectors
readable while durable embedding units resume in a separate generation.

Preserve legacy vector reuse, avoid graph rebuilds for dimension changes,
and cover interruption boundaries with fresh-process restart tests.

Refs #3598

* fix(analyze): close recovery and ownership gaps

Retain interrupted private work, clear stale query widths, and keep
native reservations until confirmed worker exit. Keep shared-member
admission gates outside garbage-collected directories for flat and
branch writers. Reject invalid
routes and overlaps, and distinguish validation from uncertain writes.

Make test imports work before build, isolate host-capacity assertions,
and give legacy vector fixtures the production native pool allocation.

Focused regressions, clean-build types, formatting and lint pass.
Graph analysis maps 51 symbols across 21 paths. Two import-only test
paths remain unmapped; direct review and integration tests cover them.
The parent full suite passed 32,084 tests with five failures: two fixed
legacy-vector fixture cases and three recovery cases awaiting isolation.
Final full-suite and performance acceptance remain pending on the draft.

Refs #3598

* perf(embeddings): batch reused vector writes

Avoid a native transaction per reused node while retaining the byte
budget and durable unit boundary. Cover mixed inputs, cancellation,
and replay after read or committed-write failures.

Native regression fails before the fix (65 commits for 64 nodes).
79 focused tests and all three former recovery timeouts pass.
Fresh graph gate: 18 symbols, 2 files, 36 affected flows; complete
mapping. Final whole-command performance acceptance remains pending.

Refs #3598

* fix(analyze): honor explicit worker overrides

Preserve the existing CPU oversubscription override while reserving memory for every requested worker. Automatic concurrency stays CPU-bounded.

Validate on one- and four-CPU hosts: 81 tests pass; the override regressions fail before the fix.

* fix(storage): validate opened artifact files

Validate descriptors before reading bounded receipts and manifests. Use nonblocking opens to reject FIFO substitutions and verify directory entries when O_NOFOLLOW is unavailable. Preserve the metadata cache fast path and guard generation paths explicitly.

Remove the redundant pending-node fallback. Add race, symlink, cleanup, and cache regressions to platform coverage.

* fix(storage): guard the graph path directly

Apply the containment check to the same path passed to lstat. The nested normalization guarded a different expression in CodeQL. path.relative already resolves its inputs, so remove the redundant normalization and retain relative-path behavior.

* fix(embeddings): preserve AST inputs on resume

Carry indexed source policy into bounded resumable jobs and preserve exact-input reuse across sibling edits. Include strategy and retention in job identity; admit source memory before reading.

* chore: run checks for the rebased stack

* refactor(embeddings): simplify reuse plumbing

Name reuse-validation options and share the owned-directory loop.
Preserve callback order, directory sync, and ownership checks.

* test: fix recovery and Windows CI fixtures

Distinguish legacy checkpoints without input provenance from verified source-aware chunks. Check the embed API at its shared-phase boundary and model pathname races without rename-over-open on Windows.

* fix: close resource admission and CI gaps

Keep live sidecars charged after a reap timeout. Let sync and HTTP jobs use automatic host budgets instead of overcommitting to default V8 limits. Explain the incompatible native zero-pool override, clean up failed HTTP fixtures, and run the native vector suites across platforms.

* fix: preserve analysis resource ownership

Pin admitted model settings and retain worker and supervisor reservations.
Recheck canonical directory identity before receipt reads or writes.
Canonicalize the native publication fixture for macOS and Windows.

* test: distinguish parameterized CI cases

* docs: remove implementation plan from PR

* ci: gate resumable analyze performance

* fix(analyze): close recovery and CI gaps

Retain ownership receipts and paid vectors through interrupted work
and cleanup. Restore automatic heap and parser-worker sizing, and
cover publication, cancellation and capacity failures.

Refs #3829, #3598

Note: pre-existing setup.test.ts and setup-skills.test.ts failures
with inherited CODEX_HOME; all 62 tests pass with it unset for the
test process.

* fix(embeddings): normalize graph parent paths

Bind the canonical job root to the same graph path on macOS temp aliases
and Windows short paths. Keep graph identity and owned-directory checks.
Canonicalize the retention fixture and show subprocess stderr on failure.

Validation: 65 focused tests; 26 tests through an aliased TMPDIR; build,
test typecheck, formatting and lint; graph change analysis (4 files).

* fix(embeddings): batch writes and guard recovery

Batch bounded vector and receipt writes to remove per-node native calls
without changing validation, transactions, or durable checkpoints.

Pin publication descriptors before entry validation. Preserve excluded
vector coverage, reject malformed status paths, retry orphan sidecars,
and complete shutdown after child spawn failures.

Validation: 159 focused tests and 89 native/restart tests; build, test
 typecheck, lint and formatting. Refreshed graph analysis maps production
changes; three test files remain unmapped.

* ci: profile fresh analyze overhead

Capture process and parser-worker CPU profiles in a separate CI-only run. Reject profiling with --check so diagnostic overhead never affects the acceptance gate.

* perf(analyze): trim continuation startup work

* fix(analyze): preserve recovery and storage bounds

Keep automatic writer pools page-scaled and restore frontend heap caps.
Adopt a generation recovered during handoff, reclaim sealed artifacts,
and retire compatible legacy sources after migration completes.

Report semantic degradation and omit provider causes from API logs.
Enforce no paid replay in the CI resume benchmark.

* perf(analyze): defer unused parser imports

* fix(workers): retain failed shutdown reservations

* perf: skip unused budget supervisor receipt

* fix: bound rejection history and watch budgets

* perf: batch provider vector writes within units

* fix: harden cleanup and semantic availability

* perf: avoid repeated analysis startup probes

Skip unused automatic-heap parent provenance and reuse one npm version
probe for runner selection and its warning. Keep child/watch identity
capture and manual/test heap overrides. Refs #3829.

* fix: normalize legacy export paths on Windows

Cover Unicode export directories and restore the heap parser comment. Refs #3829.

* test: restore watch resource mock contract

Provide the supervisor reservation used by watch refresh admission. Refs #3829.

* fix: retain watch budgets and bound lookahead

Keep explicit heap reservations across refreshes and limit brace scans to the existing three-line window. Refs #3829.

* fix: stop retries on shutdown and allow empty parse

* refactor: simplify embedding setup and test lists

* perf: batch vector completion hash updates

Keep hash finalization inside the existing transaction and row bounds. Run embedding input-isolation tests in the serial native project.

* fix: address PR #3829 recovery feedback

Preserve terminal worker results, reclaim orphan attempt metadata, retain artifacts with unrecognized space ownership, and correct the cancellation comment.

* perf: retain simple vector completion query

* perf: skip redundant alias identity probes

* perf: avoid Git probes for ordinary checkout roots

* perf: avoid redundant vector transactions

Standalone writes already execute in automatic transactions. Keep multi-query transactions and durable unit checkpoints unchanged.

* docs: clarify cancellation grace period

* fix: finish vector recovery and explicit drops

Preserve paid peers through source-admission aborts and complete legacy retirement after interrupted publication. Explicit drops remove validated vector payloads before their ownership receipts so later runs regenerate them.

Refs #3829

* fix: preserve storage errors during aborts

Only unwrap aborts raised by source admission. A simultaneous native write failure must propagate instead of entering the provider-abort completion path.

Refs #3829

* fix: validate retained analysis artifacts

Reject malformed cache keys before shard writes and retain legacy work
before shared-store adoption. Explicit drops validate legacy ownership
and keep receipts until payload cleanup succeeds. Damaged registered
graphs block stale local hook fallback.

* fix: preserve legacy and preflight contracts

Keep the native-default pool override, retain incomplete legacy seeds
with their owner, and include resource preflight in the worker timeout.

* fix: honor resource and preflight boundaries

Release optional cache space before rejecting an admissible source read.
Start server deadlines before preflight and reject null progress counts.
Pass the native-default pool sentinel through the database opener.

* test: align native and drop contracts

Accept the documented native-default zero sentinel. Verify the rebuilt graph and empty vectors instead of an obsolete escalation log.

Refs #3829

* perf: avoid redundant vector checkpoints

Acknowledge verified WAL commits without rewriting the database per unit.
Keep the final publication checkpoint and prove crash recovery from WAL.
Give publication test cases distinct titles for the CI execution audit.

Refs #3829

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-11-07 11:27:47 +00:00
Gergő Magyar
a0586d967f
feat(embeddings): preserve AST chunk coverage (#3817)
* feat(embeddings): preserve AST chunk coverage

Index long callable tails without duplicating class bodies. Preserve unchanged input reuse and include local model identity in cache checks.

Refs #702

* fix(embeddings): clean up benchmark failures

Protect fixture setup and model initialization with cleanup. Remove
fixtures even when sidecar disposal fails, and cover failure paths.
Clarify the one-time refresh for legacy non-git indexes.

Refs #3817

* fix(embeddings): address projection regressions

Preserve Kotlin/C++ class declarations, strip property bodies, and
keep fallback File nodes out of source hydration. Bump the class
projection revision so stale cached inputs regenerate.

Keep clone fixtures byte-identical under Windows Git settings.
Run paired model benchmarks directly instead of adding skipped CI tests.

Refs #3817

Note: pre-existing failures in setup.test.ts, setup-skills.test.ts,
and auto-sync.test.ts are not addressed by this PR.

* fix(embeddings): preserve empty declarations

Empty direct-member declarations have no member start offset. Keep
the whole declaration and invalidate v2 class projections so corrected
inputs regenerate. Clarify named lookup and single-candidate fallback.

Refs #3817

* chore(bench): standardize chunking artifacts

Move the paired benchmark and record measured baselines under bench/.
Remove the implementation plan from the PR and update its references.

* ci: run embedding chunk benchmarks directly

Build the comparison revision and candidate in ci-tests, then retain
paired source-aware measurements. Remove the benchmark integration test.

* fix: close embedding review and CI gaps

Filter constructor-default and heritage lambdas plus C# expression
getters from class inputs. Bump the class fingerprint to invalidate
cached v3 projections and cover the cases with real parsers.

Install the optional embedding runtime before CI benchmarking and
serialize the native lifecycle test under lbug-db.

Refs #702

* fix(ci): isolate native benchmark revisions

Independent LadybugDB addons segfault at process exit when loaded together. Run each revision in its own child process, preserve paired sampling, and require clean teardown before writing results.

Give Kotlin regression fixtures unique names for the execution audit.

* feat(embeddings): adapt cAST splitting

Reuse existing parsers without adding a chunking dependency. Refresh
source-backed layout hashes even when retained snippets fit the budget.

Refs #702

* perf(embeddings): speed up AST chunking

* docs: remove chunking optimization report

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-11-05 14:08:43 +00:00
samuel mbabhazi
e3e50be954
feat(embeddings): select the local model with GITNEXUS_EMBEDDING_MODEL_ID (#3802)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(embeddings): select the local model with GITNEXUS_EMBEDDING_MODEL_ID

The in-process embedder always loaded Snowflake/snowflake-arctic-embed-xs.
GITNEXUS_EMBEDDING_MODEL_ID now names another transformers.js model, and a
custom model takes its vector width from GITNEXUS_EMBEDDING_DIMS, the value
the vector column is already created with.

- A custom model joins the embedding content hash, so switching models
  re-embeds every node instead of reusing vectors from the previous model.
  Default-model hashes are unchanged.
- A custom model whose vectors have another width stops the run on the first
  sub-batch with the value to set, printed without a stack dump.
- Unset, every path behaves as before: the default model keeps 384
  dimensions and no width check, whatever GITNEXUS_EMBEDDING_DIMS says.

Refs #628

* fix(embeddings): document safe model switches

Require forced embedding generation when switching local models:
unchanged source can otherwise retain vectors from the old model.
Verify replacement and reuse through real analysis and LadybugDB.

Refs #3802

* test(embeddings): isolate model config cases

Preserve inherited model IDs and verify one error message with no
stack output. Correct the MCP comment for custom local dimensions.

Refs #3802

---------

Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-11-03 12:14:49 +00:00
breezefaith
54db34d5cb
feat(go): index Gin and Echo routes through configured aliases (#3750)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
2026-11-01 21:20:00 +00:00
Abhigyan Patwari
872e9e8c79
fix(impact): smaller agent output, token-budget trimming, old-index walks (#3793)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Skill copy sync / shipped skills drift guard (push) Has been cancelled
2026-11-01 06:43:47 +00:00
samuel mbabhazi
02101fdfed
feat(mcp): add an opt-in, index-only refresh_index tool (#3769)
* feat(mcp): add an opt-in refresh_index tool

A local `gitnexus mcp` server started with GITNEXUS_MCP_ENABLE_REFRESH=1
lists refresh_index, which rebuilds one registered index from its working
tree through the forked analyze worker auto-sync uses, with the same
index-only options: no AGENTS.md, CLAUDE.md, skill, hook or repository
file is written.

The tool lives outside GITNEXUS_TOOLS, so the default catalog and its byte
budget are unchanged. `gitnexus serve` never passes the option, so the
web/Docker /api/mcp surface cannot list it; read-only mode hides and
rejects it through the existing allowlist.

A repository whose last index recorded more than
GITNEXUS_MCP_REFRESH_MAX_FILES files (default 5000), or none, is refused.
After GITNEXUS_MCP_REFRESH_TIMEOUT_MS the call stops waiting; the worker
never interrupts a started analysis, so it finishes in the background and
keeps the index lock. One refresh runs per repository at a time, and the
slot is held until the worker is done. A collapsed graph write is reported
as an error, as analyze and serve report it. dry_run reports staleness,
the file count and whether .gitnexusrc asks for embeddings. Worker
progress is relayed as MCP progress notifications when the client sends a
progress token. Malformed limits fail startup.

Refs #3524.

* fix(mcp): keep worker log lines out of refresh progress

The analyze worker sends its log lines as progress messages with percent
-1. The launcher now drops them before onProgress, so the listener only
sees real progress (the server relay already discarded them).

The refresh tests now unset GITNEXUS_MCP_REFRESH_MAX_FILES and
GITNEXUS_MCP_REFRESH_TIMEOUT_MS, so an inherited value cannot change the
defaults they assert. The tool-arguments comment names the schema lookup
it actually uses.

* test(mcp): check ignored paths after a refresh, not only git status

analyze excludes its own storage through .git/info/exclude, so a clean
git status says nothing about ignored paths. The e2e now also lists them
and allows none outside .gitnexus/.

* fix(mcp): count the working tree against the refresh file limit

refresh_index compared GITNEXUS_MCP_REFRESH_MAX_FILES with the file
count of the last index. A stale index is exactly the case where that
count can be far below the working tree, so a repository that grew from
100 to 50,000 files since its last index was admitted under the default
limit of 5,000 and dry_run reported 100.

countScanCandidates (filesystem-walker.ts) walks the same glob patterns
and ignore rules as walkRepositoryPaths, without stat-ing, and stops one
past the limit, so admission costs the same on a huge tree as on a small
one. Large files, non-regular entries and declaration siblings are not
subtracted, so the count is an upper bound on what analyze indexes.

refreshIndex admits the rebuild from that count. dry_run reports it as
estimatedFiles (null past the limit, where only "more than maxFiles" is
known, and with no limit, where nothing is counted) and the registry
count as lastIndexedFiles. An index without a recorded count is no
longer refused, since the tree is counted. A tree that cannot be read
is refused before the worker, like an unreadable .gitnexusrc.

Tests: the walker count matches the scan's paths, stops at limit + 1,
and counts a file the scan drops for its size. The backend refuses a
tree over the limit whose last index was under it, without starting the
worker, and does not count with no limit. The e2e test commits two files
past a limit of 4 after an index of 3 and checks that dry_run and the
refresh refuse, with the real enumeration, and that the index stays at
its commit. Reading the registry count again fails the backend and e2e
tests; dropping the early stop fails the walker test.

* test(ingestion): ignore a directory only .gitignore excludes in the count test

The fixture ignored generated/ through .gitignore, but the ignore service
already skips a root generated/ directory on its own, so the test passed
with .gitignore disabled. scratch/ is excluded by .gitignore alone.

* test(mcp): cover refresh progress and readers

* fix(mcp): retain timed-out refresh slots (#3769)

Keep the repository busy until an abandoned worker exits, so retries cannot start another worker while the original refresh is still running. Cover late success and failure, and update the timeout documentation.

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
2026-10-31 13:57:02 +00:00
Sir Lord Brian Frank Burkett
b75e56a9cf
feat(ingestion): opt-in indexing of named dot directories (GITNEXUS_INCLUDE_DOT_DIRS) (#3745)
* feat(ingestion): opt-in indexing of named dot directories (GITNEXUS_INCLUDE_DOT_DIRS)

* fix(walker): a listed dot directory absent from the repo is empty, not a scan failure

The root-level pattern makes glob read the listed directory itself; in strict enumeration (onPathsDiscovered) that ENOENT was recorded as fatal. Skip ENOENT only for the listed roots; every other read error stays fatal.

* fix(ingestion): protect dot-directory indexing

* fix(ingestion): use glob public path type

Keep the parse-cache dependency ownership audit intact and strengthen the unlisted-directory control with a non-ignored path.

* fix(ingestion): verify symlinks after globbing

Glob can emit descendants before its cached parent type identifies a symlink. Verify filesystem ancestry before metadata callbacks and content readers, sharing parent checks across candidates.

Regression coverage includes already-emitted linked paths and fail-closed inspection errors. A 1000-walk stress probe changed from 36 leaks to zero.

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-10-29 07:02:45 +00:00
Neutron
c2d9012b58
fix(lbug): guard read queries against runaway native memory (#3696) (#3749)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(lbug): guard read queries against runaway native memory (#3696)

A single impact query was reported to allocate ~184 GB of native heap on a
66k-symbol index. That memory is outside the buffer pool and V8, so neither
bufferManagerSize nor --max-old-space-size bounds it, and the Node binding
cannot interrupt a running query.

- Add a process-wide memory guard around every pooled read query. While a
  query runs it samples RSS every 200 ms and, on macOS, the physical
  footprint for queries running over 1 s (RSS excludes compressed and
  swapped pages there). Past the budget it names the running queries on
  stderr and exits with code 70 instead of letting the host run out of
  memory. GITNEXUS_QUERY_MAX_RSS_MB sets the limit (0 disables); the
  default is max(8 GiB, 25% of RAM).
- Add GITNEXUS_QUERY_TRACE=1: one stderr line per query with duration,
  rows and memory, so a reporter can identify the query that blows up.
- Cap the impact BFS frontier query at 100k rows per depth level. A capped
  level stops the walk, marks the result partial, and cannot report a
  LOW/MEDIUM risk.

* fix(ci): initialize shards after fork approval

GitHub approval increments the attempt number without running any jobs. Allow initial timing snapshot creation on attempt 2 only when the first attempt required approval and had zero jobs. Keep real retries fail-closed on missing frozen input.

* fix(impact): qualify both capped risk scores

Apply the same conservative cap policy to risk and riskSharedAxes, retain proven high warnings, and strengthen frontier-limit regression assertions. Correct the memory-budget test description.

* fix: skip query phases after deadline

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-10-28 14:08:17 +00:00
Gergő Magyar
13d06d41c4
feat(ci): rebalance test shards from recorded timings (#3748)
* feat(ci): rebalance test shards from recorded timings

* fix(ci): preserve artifact retries for shard timings

Find shard-plan generation by its stable step ID and clarify the native runner cohort count (#3748).

Note: pre-existing reference-oracle parser type failures in two test_oracle_assets cases remain outside this change.

* fix(ci): keep retry timing snapshots immutable

Require the existing snapshot on reruns so a failed download cannot refresh history and change shard assignments. Cover the workflow contract and document recovery when the artifact is unavailable.

* docs(ci): document shard timing profile selection

* fix(ci): balance cold shard estimates consistently

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-10-28 11:58:29 +00:00
Gergő Magyar
48f108a58a
docs: simplify guides and verify receiver gaps (#3694)
* docs: align resolution architecture and roadmap

* docs: add language ingestion gap inventories

* test: audit receiver gap targets and coverage

Check exact declaration IDs, expose the Dart await wrong-owner edge, and execute previously excluded C++, Rust, PHP, Dart and Vue shapes. Refresh the baseline and all language inventories from fresh measurements; call-drop counts remain unchanged.

* docs: focus roadmap and shorten project README

Keep the resolution roadmap limited to future work and acceptance criteria. Move detailed editor, CLI, MCP, deployment and wiki instructions into linked guides, preserve their command examples, and correct stale onboarding claims.

* docs: correct setup and audit descriptions

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-10-23 09:06:03 +01:00