* feat(mcp): add an opt-in refresh_index tool
A local `gitnexus mcp` server started with GITNEXUS_MCP_ENABLE_REFRESH=1
lists refresh_index, which rebuilds one registered index from its working
tree through the forked analyze worker auto-sync uses, with the same
index-only options: no AGENTS.md, CLAUDE.md, skill, hook or repository
file is written.
The tool lives outside GITNEXUS_TOOLS, so the default catalog and its byte
budget are unchanged. `gitnexus serve` never passes the option, so the
web/Docker /api/mcp surface cannot list it; read-only mode hides and
rejects it through the existing allowlist.
A repository whose last index recorded more than
GITNEXUS_MCP_REFRESH_MAX_FILES files (default 5000), or none, is refused.
After GITNEXUS_MCP_REFRESH_TIMEOUT_MS the call stops waiting; the worker
never interrupts a started analysis, so it finishes in the background and
keeps the index lock. One refresh runs per repository at a time, and the
slot is held until the worker is done. A collapsed graph write is reported
as an error, as analyze and serve report it. dry_run reports staleness,
the file count and whether .gitnexusrc asks for embeddings. Worker
progress is relayed as MCP progress notifications when the client sends a
progress token. Malformed limits fail startup.
Refs #3524.
* fix(mcp): keep worker log lines out of refresh progress
The analyze worker sends its log lines as progress messages with percent
-1. The launcher now drops them before onProgress, so the listener only
sees real progress (the server relay already discarded them).
The refresh tests now unset GITNEXUS_MCP_REFRESH_MAX_FILES and
GITNEXUS_MCP_REFRESH_TIMEOUT_MS, so an inherited value cannot change the
defaults they assert. The tool-arguments comment names the schema lookup
it actually uses.
* test(mcp): check ignored paths after a refresh, not only git status
analyze excludes its own storage through .git/info/exclude, so a clean
git status says nothing about ignored paths. The e2e now also lists them
and allows none outside .gitnexus/.
* fix(mcp): count the working tree against the refresh file limit
refresh_index compared GITNEXUS_MCP_REFRESH_MAX_FILES with the file
count of the last index. A stale index is exactly the case where that
count can be far below the working tree, so a repository that grew from
100 to 50,000 files since its last index was admitted under the default
limit of 5,000 and dry_run reported 100.
countScanCandidates (filesystem-walker.ts) walks the same glob patterns
and ignore rules as walkRepositoryPaths, without stat-ing, and stops one
past the limit, so admission costs the same on a huge tree as on a small
one. Large files, non-regular entries and declaration siblings are not
subtracted, so the count is an upper bound on what analyze indexes.
refreshIndex admits the rebuild from that count. dry_run reports it as
estimatedFiles (null past the limit, where only "more than maxFiles" is
known, and with no limit, where nothing is counted) and the registry
count as lastIndexedFiles. An index without a recorded count is no
longer refused, since the tree is counted. A tree that cannot be read
is refused before the worker, like an unreadable .gitnexusrc.
Tests: the walker count matches the scan's paths, stops at limit + 1,
and counts a file the scan drops for its size. The backend refuses a
tree over the limit whose last index was under it, without starting the
worker, and does not count with no limit. The e2e test commits two files
past a limit of 4 after an index of 3 and checks that dry_run and the
refresh refuse, with the real enumeration, and that the index stays at
its commit. Reading the registry count again fails the backend and e2e
tests; dropping the early stop fails the walker test.
* test(ingestion): ignore a directory only .gitignore excludes in the count test
The fixture ignored generated/ through .gitignore, but the ignore service
already skips a root generated/ directory on its own, so the test passed
with .gitignore disabled. scratch/ is excluded by .gitignore alone.
* test(mcp): cover refresh progress and readers
* fix(mcp): retain timed-out refresh slots (#3769)
Keep the repository busy until an abandoned worker exits, so retries cannot start another worker while the original refresh is still running. Cover late success and failure, and update the timeout documentation.
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
* feat(ingestion): opt-in indexing of named dot directories (GITNEXUS_INCLUDE_DOT_DIRS)
* fix(walker): a listed dot directory absent from the repo is empty, not a scan failure
The root-level pattern makes glob read the listed directory itself; in strict enumeration (onPathsDiscovered) that ENOENT was recorded as fatal. Skip ENOENT only for the listed roots; every other read error stays fatal.
* fix(ingestion): protect dot-directory indexing
* fix(ingestion): use glob public path type
Keep the parse-cache dependency ownership audit intact and strengthen the unlisted-directory control with a non-ignored path.
* fix(ingestion): verify symlinks after globbing
Glob can emit descendants before its cached parent type identifies a symlink. Verify filesystem ancestry before metadata callbacks and content readers, sharing parent checks across candidates.
Regression coverage includes already-emitted linked paths and fail-closed inspection errors. A 1000-walk stress probe changed from 36 leaks to zero.
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
* fix(lbug): guard read queries against runaway native memory (#3696)
A single impact query was reported to allocate ~184 GB of native heap on a
66k-symbol index. That memory is outside the buffer pool and V8, so neither
bufferManagerSize nor --max-old-space-size bounds it, and the Node binding
cannot interrupt a running query.
- Add a process-wide memory guard around every pooled read query. While a
query runs it samples RSS every 200 ms and, on macOS, the physical
footprint for queries running over 1 s (RSS excludes compressed and
swapped pages there). Past the budget it names the running queries on
stderr and exits with code 70 instead of letting the host run out of
memory. GITNEXUS_QUERY_MAX_RSS_MB sets the limit (0 disables); the
default is max(8 GiB, 25% of RAM).
- Add GITNEXUS_QUERY_TRACE=1: one stderr line per query with duration,
rows and memory, so a reporter can identify the query that blows up.
- Cap the impact BFS frontier query at 100k rows per depth level. A capped
level stops the walk, marks the result partial, and cannot report a
LOW/MEDIUM risk.
* fix(ci): initialize shards after fork approval
GitHub approval increments the attempt number without running any jobs. Allow initial timing snapshot creation on attempt 2 only when the first attempt required approval and had zero jobs. Keep real retries fail-closed on missing frozen input.
* fix(impact): qualify both capped risk scores
Apply the same conservative cap policy to risk and riskSharedAxes, retain proven high warnings, and strengthen frontier-limit regression assertions. Correct the memory-budget test description.
* fix: skip query phases after deadline
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
* feat(ci): rebalance test shards from recorded timings
* fix(ci): preserve artifact retries for shard timings
Find shard-plan generation by its stable step ID and clarify the native runner cohort count (#3748).
Note: pre-existing reference-oracle parser type failures in two test_oracle_assets cases remain outside this change.
* fix(ci): keep retry timing snapshots immutable
Require the existing snapshot on reruns so a failed download cannot refresh history and change shard assignments. Cover the workflow contract and document recovery when the artifact is unavailable.
* docs(ci): document shard timing profile selection
* fix(ci): balance cold shard estimates consistently
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
* docs: align resolution architecture and roadmap
* docs: add language ingestion gap inventories
* test: audit receiver gap targets and coverage
Check exact declaration IDs, expose the Dart await wrong-owner edge, and execute previously excluded C++, Rust, PHP, Dart and Vue shapes. Refresh the baseline and all language inventories from fresh measurements; call-drop counts remain unchanged.
* docs: focus roadmap and shorten project README
Keep the resolution roadmap limited to future work and acceptance criteria. Move detailed editor, CLI, MCP, deployment and wiki instructions into linked guides, preserve their command examples, and correct stale onboarding claims.
* docs: correct setup and audit descriptions
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
