4 commits

Author	SHA1	Message	Date
dependabot[bot]	8ed4623352	chore(deps)(deps-dev): bump typescript in /gitnexus-shared (#2941) ... Bumps [typescript](https://github.com/microsoft/TypeScript) from 6.0.3 to 7.0.2. - [Release notes](https://github.com/microsoft/TypeScript/releases) - [Commits](https://github.com/microsoft/TypeScript/commits) --- updated-dependencies: - dependency-name: typescript dependency-version: 7.0.2 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-08-13 06:23:16 +01:00
Gergő Magyar	152a0506c9	feat: shared resilient-fetch (retries + circuit breaker) (#1448) ... * feat: shared resilient-fetch (retries + circuit breaker) Add a small, runtime-agnostic resilience layer in gitnexus-shared and migrate every backend HTTP outbound call (CLI, MCP, wiki LLM, web → backend) through it. Helpers (gitnexus-shared/src/integrations/): - retry.ts — withRetry(fn, opts) with caller-supplied retryability classification and full-jitter exponential backoff. - circuit-breaker.ts — closed/open/half-open per-process breaker with injectable clock, plus a keyed registry so callers targeting the same endpoint share state. - resilient-fetch.ts — composed wrapper: retries 5xx + 429 + retryable network throws, treats AbortSignal.timeout() and 4xx (other than 429) as terminal, honors Retry-After (capped at 30s), throws CircuitOpenError when the breaker opens. Migrations (no behaviour regression — all existing tests pass): - gitnexus/src/core/embeddings/http-client.ts (covers analyze + MCP query path) — replaces inline linear-backoff retry. - gitnexus/src/core/wiki/llm-client.ts — preserves Azure content-filter branch; resilientFetch handles 5xx/429. - gitnexus-web/src/services/backend-client.ts (fetchWithTimeout helper) — small retry budget (2 attempts, 250–1500 ms) so a dead local backend still fails fast for the user. - gitnexus-web/src/core/llm/settings-service.ts (OpenRouter model list). Deliberately not migrated: - gitnexus-web/src/services/backend-client.ts streamJob() — Server-Sent Events stream; the existing reconnect-with-Last-Event-ID logic is not unary-fetch shaped. - gitnexus-web/src/components/SettingsPanel.tsx checkOllamaStatus() — one-shot health probe; retrying delays the "Ollama not running" error rather than improving UX. 41 new helper tests cover backoff math, breaker state transitions, Retry-After parsing (delta-seconds + HTTP-date), 401/422 terminal classification, and breaker fail-fast on three exhausted retry batches. * fix(review): apply autofix feedback Address Claude's two MEDIUM blocking findings on PR #1448 plus the CodeQL SSRF false-positive flag. - backend-client `fetchWithTimeout` now uses `AbortSignal.timeout()` merged with the caller's signal via `AbortSignal.any()`. Timer-fired aborts surface as `DOMException(name='TimeoutError')` so resilientFetch routes them through the terminal-network branch (no retry, no breaker hit), instead of incrementing the breaker for user-side network slowness. - Method-aware retry budget in `fetchWithTimeout`: idempotent verbs (GET/HEAD/OPTIONS) keep the 2-attempt budget; POST/PATCH/PUT/DELETE default to single-attempt so a 5xx on `startAnalyze` cannot start a duplicate job. New `forceRetry` parameter for callers that know-idempotent mutations (e.g. DELETE of a known-deleted resource). - `resilient-fetch.ts` carries a documented suppression for CodeQL js/server-side-request-forgery on the inner fetch call. Every concrete caller passes a hardcoded URL constant or a value from configuration (env vars, saved settings); user request input never flows into the URL parameter. - New test file `backend-client-retry.test.ts` covers all three paths: GET retries on 503, POST does not retry, timeout does not increment the breaker. * fix(resilient-fetch): address Codex adversarial findings Closes the three blocking issues from Codex's review on PR #1448. U1 — Add `recordNeutral()` to CircuitBreaker. Third outcome path that's an explicit no-op for state and the consecutive-failure counter. Distinct from `recordSuccess` (closes the breaker) and `recordFailure` (may open it). Used for outcomes that are neither evidence of backend health nor evidence of backend failure. U2 — Route terminal-client / terminal-network through `recordNeutral`. Previously a 401 or local timeout called `recordSuccess`, which reset `consecutiveFailures` to 0. A 5xx → 401 → 5xx → 401 → 5xx sequence would NEVER trip the breaker because each 4xx in between erased the running count. Also classify external `AbortError` as terminal-network (was retryable-network), so caller-driven cancellation no longer retries against an already-aborted signal or counts toward breaker failures on exhaustion. U3 — Per-origin breaker key in web `fetchWithTimeout`. Was hardcoded to `'web-backend'` even though `_backendUrl` is mutable via `setBackendUrl`. Switching backend URLs after a circuit tripped on host-A would strand the user during the full cooldown. Key is now `web-backend:<origin>`, so each backend URL gets its own breaker state. Tests: +5 recordNeutral, +4 resilient-fetch (interleaved 4xx/5xx, external AbortError, prior-state preservation), +1 web switch-backend regression. All 70 gitnexus integration tests + 15 web tests green. * fix(resilient-fetch): tolerate header-less fetch mocks on 429 `classifyOutcome` called `resp.headers.get('Retry-After')` directly, which crashed when a test stubs `fetch` with a plain object like `{ ok: false, status: 429 }` (no `headers` field). Real `Response` always has Headers, so this surfaces only in test setups, but the helper has no business assuming caller-side correctness on this — the defensive guard is cheap and a missing `Retry-After` falls through to exponential-backoff retry like any 429 without the header. Surfaced by `gitnexus/test/unit/http-embedder.test.ts > retries on rate limit`, which the embeddings migration exercises against a plain-object 429 stub. Locked in with a new `classifies 429 from a header-less fetch mock without throwing` case. * fix(review): apply autofix feedback Closes findings from the third multi-agent review pass on PR #1448. #1 (P1) callLLM had no per-attempt timeout Wiki LLM calls passed no `signal` to resilientFetch; each of three retry attempts could hang indefinitely on a frozen TCP connection. Add `signal: AbortSignal.timeout(60_000)` so the per-attempt budget matches what http-client.ts and backend-client.ts already provide. #2 (P2) drop dead `lastRetryableResp` post-loop fallback Variable was set in one switch arm but only read in unreachable code after the loop. The retry loop always returns/throws on every iteration. Keep only the defensive `throw` so TypeScript's control-flow analysis still sees `Promise<Response>` as the return. #5 (P2) gate test-only exports behind a subpath `__resetBreakerRegistry__` and `classifyOutcome` were reachable from the main `gitnexus-shared` barrel — production code calling `__resetBreakerRegistry__` from a tool implementation would silently nuke every circuit breaker process-wide. Move to a new `gitnexus-shared/test-helpers` subpath export. Production callers see the cleaner public API; tests import via the explicit `gitnexus-shared/test-helpers` path. #6 (P2) exhaustiveness guard on Outcome switch Add a `default: const _: never = outcome` arm so a future sixth `Outcome.kind` won't compile silently — it'll surface at the switch site rather than fall through to a retry/no-retry default. #9 (P3) document cumulative wall-clock budget Add a "Cumulative wall-clock budget" paragraph to resilientFetch's JSDoc explaining the worst-case total wait (`maxAttempts × (per-attempt timeout + capDelayMs)` ≈ 60s with defaults) and pointing callers at outer `AbortSignal.timeout()` when they want a tighter bound. Deferred to follow-up PRs (per review's Auto-resolve recommendation): - #3 idempotency knob to shared API (forceRetry into ResilientFetchOptions) - #4 publish.ts migration to resilientFetch - #7 parseRetryAfter past-HTTP-date / negative-seconds asymmetry - #8 recordNeutral counter time-decay (documented breaker semantic) * fix(circuit-breaker): gate half-open to a single in-flight probe Closes the Codex adversarial-review finding on PR #1448 that flagged a recovery-time thundering herd: when cooldown expired, every concurrent caller transitioned the breaker to half-open and probed the still- recovering dependency in lockstep, defeating the breaker's "fail fast" promise. U1 — probe-permit gate in CircuitBreaker.check() Added a `probeInFlight: boolean` field. After cooldown expires, the first `check()` admits the probe and consumes the permit; subsequent callers throw `CircuitOpenError` with a configurable `halfOpenRetryAfterMs` (default 1000ms) until the probe resolves. Critical design point: `recordNeutral` now RELEASES the permit but does NOT transition state. Without that split, a single `TimeoutError` from per-attempt `AbortSignal.timeout` (which routes through neutral classification) would permanently park the breaker in half-open. By separating permit-release from state-resolution, we keep the "neutral doesn't claim health" semantic without creating that wedge. Other changes: - `halfOpenRetryAfterMs` is now a constructor option for consumers with long-running protected ops (LLM streaming, large uploads). - `getState()` is documented as a pure read; the implicit Open -> Half-Open transition lives in `check()` only, so tests that inspect state never inadvertently consume a probe permit. - `isProbeInFlight()` test-only accessor for assertion clarity. - JSDoc on `check()` records the JS event-loop atomicity dependency and the load-bearing `try/finally` pairing invariant. U2 — End-to-end concurrency regression through resilientFetch Three new scenarios in resilient-fetch.test.ts (26 -> 29): - 3 concurrent calls + probe gets 200 -> 1 hits fetch, 2 throw CircuitOpenError, breaker closes. - 3 concurrent calls + probe gets 503 -> ResilientFetchExhaustedError on probe; concurrent callers see halfOpenRetryAfterMs (1000ms); fresh caller after probe resolves sees the FULL new cooldown (10000ms), not the probe-in-flight default. - Probe cancelled mid-flight via AbortError -> permit released, state stays half-open, next caller becomes the new probe and succeeds. Plus 9 new circuit-breaker unit tests (16 -> 25) covering the permit gate, recordNeutral-releases-permit semantic, fresh-cooldown distinction, default vs configurable halfOpenRetryAfterMs, getState() purity, and the three-probes-via-neutrals chain. Total integration test count: 70 -> 82. All 106 gitnexus + 15 web tests pass; both packages typecheck. Maintainer decisions (deferred per plan 003 Open Questions): - Plan 002's deferral judgement was reversed on Codex's argument without new measurement / incident data. The reversal is defensible on principle (Hystrix / Resilience4j alignment) but lacks workload- driven evidence. - Probe-blocked callers throw silently (no log / event hook). R4's "no new public API" prevents adding observability; loosen if a debug log on probe-blocked is wanted. * refactor(embeddings): replace bespoke HF breaker with shared CircuitBreaker Deleted the local `HfDownloadCircuitBreaker` class and the manual retry loop in `withHfDownloadRetry`. Both are now backed by the shared `gitnexus-shared` primitives: - `hfDownloadCircuit` is `new CircuitBreaker({ failureThreshold, cooldownMs, key: 'hf-download' })` — same state machine as before PLUS the single-permit half-open gate that prevents recovery-time stampedes when CLI + MCP embedders concurrently re-load the model. - `withHfDownloadRetry` delegates the loop to `withRetry` from the shared package. Per-attempt timeout (`withDownloadTimeout`), network-vs-non-network classification, circuit recording, and the `onRetry` callback wire through `withRetry`'s `isRetryable` callback. Behaviour preserved: - Pre-flight `CIRCUIT_OPEN_TAG` rejection when the breaker is open. - Mid-loop `CIRCUIT_OPEN_TAG` "opened after N consecutive failures" when a network error trips the threshold. - Non-network errors (e.g. CUDA unavailable) bypass retry and go through `recordNeutral` instead of resetting the breaker's failure-count progress. - `onRetry(attempt+1, max, err)` fires only when there's a next attempt, matching the prior semantic. Generic CircuitBreaker gained two inspection accessors: - `getOpenedAt(): number | null` - `getCooldownMs(): number` Used by `withHfDownloadRetry` to compute `secsUntilReset` without consuming a probe permit (which `check()` would do). Test consolidation: the 7 bespoke `HfDownloadCircuitBreaker` state-machine tests in hf-env.test.ts were 1:1 duplicates of existing tests in `circuit-breaker.test.ts` and were deleted. Remaining 42 hf-env tests all pass; full integration sweep (148 gitnexus + 15 web) green.	2026-05-09 15:18:09 +01:00
dependabot[bot]	42d276bc4a	chore(deps)(deps-dev): bump typescript in /gitnexus-shared (#1034)	2026-04-23 05:06:42 +01:00
Gergő Magyar	fd7fb5bf1f	feat: unify web and cli ingestion pipeline (#536) ... * feat: add server-side ingestion API (POST /api/analyze, SSE progress) Extract core analysis orchestration from CLI into shared run-analyze.ts module. Add server-side analyze endpoints so the web app can trigger ingestion via HTTP instead of running the full pipeline in-browser. New files: - src/core/run-analyze.ts — shared runFullAnalysis() orchestrator - src/server/analyze-job.ts — job manager (single-slot, dedup, SSE events) - src/server/analyze-worker.ts — forked child process (8GB heap, IPC) - src/server/git-clone.ts — shallow clone/pull with SSRF protection API endpoints: - POST /api/analyze — start analysis (returns 202 + jobId) - GET /api/analyze/:jobId — poll job status - GET /api/analyze/:jobId/progress — SSE progress stream Security: URL validation blocks private IPs and non-HTTP schemes. Path validation requires absolute paths. Git stderr not leaked to API. * feat(web): add server-side analyze UI (Phase 2) Add "Analyze on Server" flow to the web app's Server tab so users can trigger server-side ingestion from the browser. On completion, the graph is automatically loaded via the existing connectToServer flow. New files: - AnalyzeProgress.tsx — progress bar with phase label, elapsed time, cancel Modified files: - backend.ts — startAnalyze(), streamAnalyzeProgress() SSE client - DropZone.tsx — analyze URL input + button below Connect section - App.tsx — onServerAnalyze handler wires analyze -> connect flow * feat: add job cancellation, timeout, and child process tracking (Phase 3) - DELETE /api/analyze/:jobId — cancel running analysis (SIGTERM to worker) - 30-minute timeout kills long-running workers automatically - Child process refs tracked in JobManager for cleanup on shutdown - dispose() kills all active children on SIGINT/SIGTERM - Web cancel button now calls server DELETE endpoint - cancelAnalyze() added to web backend client * refactor(web): remove browser ingestion pipeline (Phase 4) Delete 16 duplicated ingestion files, 2 unused service files (git-clone, zip), and tree-sitter parser-loader from gitnexus-web. All ingestion now runs server-side via POST /api/analyze. Deleted (18 files, ~5,000 lines): - core/ingestion/*.ts (16 pipeline processors) - core/tree-sitter/parser-loader.ts (WASM tree-sitter loader) - services/git-clone.ts (isomorphic-git client-side clone) - services/zip.ts (JSZip extraction) Simplified: - DropZone.tsx — server-only (removed ZIP/GitHub tabs) - ingestion.worker.ts — removed runPipeline/runPipelineFromFiles - useAppState.tsx — removed pipeline callbacks - App.tsx — removed handleFileSelect/handleGitClone - main.tsx — removed Buffer polyfill for isomorphic-git - types/pipeline.ts — removed PipelineResult/serialize helpers Kept: cluster-enricher.ts (LLM enrichment, still used by worker) Dependencies now removable: web-tree-sitter, isomorphic-git, @isomorphic-git/lightning-fs, jszip (estimated 3-4MB bundle savings) * refactor(web): sync graph schema from CLI + delete WASM grammars Sync graph/types.ts and lbug/schema.ts from the CLI (source of truth) to the web module so the browser LadybugDB can handle all node and relationship types the server pipeline produces. Synced types: Route, Tool, Section node labels; HANDLES_ROUTE, FETCHES, HANDLES_TOOL, ENTRY_POINT_OF, WRAPS, QUERIES relationship types; description fields on Function/Class/Interface/Method/CodeElement. Deleted: public/wasm/ directory (14 tree-sitter WASM grammars + core). Removed deps: web-tree-sitter, isomorphic-git, @isomorphic-git/lightning-fs, jszip, buffer, @types/jszip (~3-4MB bundle savings). * feat: create gitnexus-shared package for unified type definitions Create a new gitnexus-shared package that is the single source of truth for types shared between the CLI and web modules: - SupportedLanguages enum (15 languages) - Graph types: NodeLabel, NodeProperties, RelationshipType, GraphNode, GraphRelationship - Schema constants: NODE_TABLES, REL_TYPES, REL_TABLE_NAME, EMBEDDING_TABLE_NAME - Pipeline types: PipelinePhase, PipelineProgress Both gitnexus (CLI) and gitnexus-web import from gitnexus-shared via file: dependency. Each package re-exports and extends with platform-specific additions (CLI: KnowledgeGraph with mutation methods; Web: simpler KnowledgeGraph). This ensures types can never drift between packages — adding a new language, node type, or relationship type in gitnexus-shared automatically propagates to both consumers. * refactor: import shared types directly from gitnexus-shared at call sites Replace all re-export patterns with direct imports from gitnexus-shared. 72 files updated across CLI and web: - SupportedLanguages: 49 CLI files now import from 'gitnexus-shared' instead of '../config/supported-languages.js' - GraphNode, GraphRelationship, NodeLabel: 22 CLI + 10 web files now import from 'gitnexus-shared' instead of local re-export wrappers - NODE_TABLES: api.ts imports from 'gitnexus-shared' - PipelineProgress: useAppState.tsx imports from 'gitnexus-shared' Local types.ts files now only define platform-specific KnowledgeGraph (CLI has mutation methods, web has add-only). No more re-exports. * fix: update lock files for gitnexus-shared, remove stale vite polyfills Add gitnexus-shared@1.0.0 to lock files so npm ci succeeds in CI. Remove buffer polyfill and global define from vite.config.ts (isomorphic-git was removed). * fix(security): add write guard to HTTP /api/query, fix CORS proxy bypass - Add isWriteQuery() check to POST /api/query handler — blocks CREATE, DELETE, SET, MERGE, DROP, etc. via HTTP API (guard was only in MCP pool adapter and browser-side, not the HTTP server path) - Extend CYPHER_WRITE_RE with CALL, INSTALL, LOAD keywords - Fix CORS proxy subdomain bypass: endsWith('github.com') allowed 'evil-github.com'. Now requires exact match or '.github.com' suffix * feat(server): enhance /api/search with enrichment, add /api/grep, strip graph content - POST /api/search: add mode param (hybrid|semantic|bm25), server-side enrichment returns connections/cluster/processes per result in one call (collapses 31 sequential HTTP calls to 1 for the agent search tool) - GET /api/grep: regex search across indexed file contents, eliminates need to transfer all file contents to browser - GET /api/graph: strip content field by default (80-95% payload reduction). Use ?includeContent=true for backward compat - Add LRU cache invalidation hook point for future caching * feat(server): add /api/embed endpoint for server-side embedding generation - POST /api/embed: triggers embedding pipeline via onnxruntime-node with JobManager for single-slot concurrency, timeout, and dedup - GET /api/embed/:jobId: poll job status - GET /api/embed/:jobId/progress: SSE stream with heartbeat, event IDs, and X-Accel-Buffering:no header for proxy compatibility - DELETE /api/embed/:jobId: cancel running embedding job - Maps embedding pipeline phases (ready→complete, error→failed) to JobManager status conventions * feat(web): create consolidated BackendClient module Single HTTP client replacing backend.ts, server-connection.ts, and worker HTTP helpers. Includes: - Typed methods: runQuery, search (enriched), grep, readFile, connect - Generic streamSSE<T> utility extracted from analyze progress pattern - BackendError with discriminated code field (network/server/client/timeout) - Embed API: startEmbeddings, streamEmbeddingProgress, cancelEmbeddings - Search with mode param (hybrid|semantic|bm25) and enrichment * refactor(web): rewrite Graph RAG tools for backend-only HTTP queries - Search tool: uses enriched /api/search (1 call replaces 31 sequential queries) - Cypher tool: removes browser-side embedding; {{QUERY_VECTOR}} routes to /api/search with mode:'semantic' instead of local transformers.js - Grep tool: uses /api/grep instead of in-memory fileContents map - Read tool: uses /api/file instead of fileContents map lookup - Impact tool: getCallSiteSnippet now async via /api/file - createGraphRAGTools now accepts GraphRAGBackend interface instead of 7 separate function params + fileContents map - createGraphRAGAgent simplified to (config, backend, context?) - Removed imports: embedder, lbug/schema (replaced with gitnexus-shared) - Net: -205 lines * refactor(web): delete WASM infrastructure, remove 7 packages (-5242 lines) Delete browser-side LadybugDB, embeddings, search, and worker: - gitnexus-web/src/core/lbug/ (adapter, csv-generator, schema, query-result) - gitnexus-web/src/core/embeddings/ (embedder, pipeline, text-gen, types) - gitnexus-web/src/core/search/ (bm25-index, hybrid-search) - gitnexus-web/src/workers/ingestion.worker.ts (828 lines) - gitnexus-web/src/services/server-connection.ts (merged into backend-client) - gitnexus-web/src/types/lbug-wasm.d.ts Remove packages: @ladybugdb/wasm-core, @huggingface/transformers, comlink, minisearch, vite-plugin-wasm, vite-plugin-top-level-await, vite-plugin-static-copy Update vite.config.ts: remove WASM plugins, COOP/COEP headers, worker config, optimizeDeps exclude Update imports: App.tsx, DropZone, Header, AnalyzeProgress, BackendRepoSelector, useBackend → backend-client * refactor(web): replace Worker/Comlink with direct BackendClient calls - useAppState: remove Worker instantiation, Comlink.wrap, apiRef. All queries now go through BackendClient HTTP functions directly. - Agent runs on main thread (I/O-bound LLM streaming, not CPU-bound) - initializeAgent: creates GraphRAGAgent with GraphRAGBackend interface bound to BackendClient methods (runQuery, search, grep, readFile) - startEmbeddings: calls POST /api/embed + SSE progress instead of running browser-side transformers.js pipeline - switchRepo: no longer loads graph into WASM DB or extracts fileContents - App.tsx: handleServerConnect simplified (no fileContents, no loadServerGraph) - Delete old backend.ts (replaced by backend-client.ts) - Net: -396 lines * fix(web): fix await-in-map build error in agent streaming Move dynamic import of AIMessage outside .map() callback to avoid "await can only be used inside an async function" build error. * fix(web): remove stale apiRef references that broke chat functionality sendChatMessage referenced apiRef.current (deleted Worker ref) which would throw TypeError. Replaced with agentRef.current guard since agent now runs on main thread. * fix(server): dispose embedJobManager on shutdown, fix job mutation - Add embedJobManager.dispose() to shutdown handler (was missing, causing cleanup timer to keep Node process alive) - Replace direct job.repoName/status mutation with updateJob() to ensure SSE event emission for initial status change * fix(server): parameterize Cypher, harden grep, unify SSE endpoints - Search enrichment: replace string interpolation with executePrepared() using $nid parameter binding to prevent Cypher injection - Add executePrepared() to core lbug-adapter (prepare/execute pattern) - /api/grep: add 200-char pattern length limit (ReDoS protection), search files on disk instead of loading entire corpus into memory (constant memory usage regardless of repo size) - Extract mountSSEProgress() shared helper for SSE streaming — both analyze and embed endpoints now have consistent heartbeat (30s), event IDs (reconnection support), and X-Accel-Buffering header * refactor(web): remove dead code from Worker-era architecture - Remove loadServerGraph no-op function, interface member, and all consumers - Remove testArrayParams stub and interface member - Remove fileContents state from GraphStateProvider (never populated in server-side architecture) - Remove forceDevice parameter from startEmbeddings (server-side, no device choice) - Replace phantom EmbeddingProgress type with inline { phase, percent } - Replace resolvePathFromContents (needed fileContents Map) with graph-based file path resolution using filePathIndex built from graph nodes - Fix: AI citation grounding ([[file.ts:10]]) now works via graph node lookup instead of broken fileContents-based resolution * fix(web): use streamAgentResponse for full tool_call/reasoning streaming Replace naive agent.stream() loop that only handled content chunks with streamAgentResponse() generator from agent.ts. This properly routes: - reasoning tokens (before/between tool calls) - tool_call events (name, args, status) - tool_result events (completed tool output) - content tokens (final answer after all tools done) Previously the onChunk handler for tool_call/tool_result/reasoning was dead code since the streaming loop only emitted content events. * fix(web): resolve CI type errors from dead code removal - Import GraphNode/GraphRelationship from gitnexus-shared in graph.ts (not re-exported from local types.ts) - Add Route, Tool entries to NODE_COLORS and NODE_SIZES constants - Add PipelineResult type to web types/pipeline.ts - Remove fileContents from CodeReferencesPanel and RightPanel - Remove testArrayParams and forceDevice from EmbeddingStatus - Remove forceDevice args from startEmbeddings() calls in App.tsx - Fix embeddingProgress property accesses for simplified type * fix(ci): add setup-gitnexus-web action, build shared once per job - Remove prepare script from gitnexus-shared (tsc not available during npm ci of consuming packages) - Create .github/actions/setup-gitnexus-web composite action: builds gitnexus-shared then runs npm ci for gitnexus-web - setup-gitnexus action: already builds gitnexus-shared for CLI jobs - ci-quality typecheck-web: uses setup-gitnexus-web (DRY) - ci-e2e: uses setup-gitnexus-web (DRY) - ci-tests: gitnexus-shared already built by setup-gitnexus, just install web deps without rebuilding * fix(ci): use prepare script so gitnexus-shared builds during npm ci Move typescript from devDependencies to dependencies in gitnexus-shared so the prepare script (tsc) works when npm resolves file: deps during npm ci. No GHA modifications needed — npm handles the build lifecycle automatically. Remove manual gitnexus-shared build steps from setup-gitnexus and setup-gitnexus-web actions. * fix(ci): build gitnexus-shared explicitly in setup actions The file: dependency protocol doesn't reliably run prepare scripts because devDependencies aren't installed first. Instead of fragile lifecycle hacks, build gitnexus-shared explicitly in both setup actions: - setup-gitnexus: npm install && npm run build in gitnexus-shared/ - setup-gitnexus-web: same, before npm ci in gitnexus-web/ - ci-tests: shared already built by setup-gitnexus, web just npm ci No prepare script, no dist in git, no typescript as a prod dependency. * fix: remove CALL from CYPHER_WRITE_RE — breaks FTS and vector search CALL is used by read-only procedures: CALL QUERY_FTS_INDEX(...) and CALL QUERY_VECTOR_INDEX(...). Adding it to the write guard blocked all FTS search, causing 3 test failures. The database is opened in read-only mode as defense-in-depth against write procedures via CALL. Keep INSTALL and LOAD in the blocklist (genuinely dangerous). * fix(web): update vercel.json for gitnexus-shared, remove COOP/COEP - Add installCommand that builds gitnexus-shared before installing web deps (Vercel doesn't know about the monorepo file: dependency) - Remove Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy headers (no longer needed — WASM LadybugDB removed) * fix(web): update tests for deleted modules - Delete csv-generator.test.ts (tests deleted WASM-only csv-generator) - Update security-guards.test.ts: import NODE_TABLES/REL_TYPES from gitnexus-shared instead of deleted src/core/lbug/schema - Update server-connection.test.ts: import normalizeServerUrl from backend-client, remove extractFileContents tests (function deleted) * fix(e2e): remove Server tab click — UI is now server-only The DropZone no longer has ZIP/GitHub/Server tabs (browser ingestion was removed). The server URL input is directly visible on the landing page. Update e2e test to skip the tab click and go straight to input. All 5 e2e tests pass locally. * refactor: use gitnexus-shared for PipelinePhase/PipelineProgress types CLI was duplicating PipelinePhase and PipelineProgress locally instead of importing from gitnexus-shared. Updated all consumers to import directly. Also removed dead code: SerializablePipelineResult, serializePipelineResult(), deserializePipelineResult(). * fix(server): address PR #536 review — security, race conditions, dead code - Fix path traversal in POST /api/analyze: split into isAbsolute + normalize check - Add shared repo lock (activeRepoPaths) preventing concurrent analyze+embed on same repo - Fix 202 response returning actual job.status instead of hardcoded 'queued' - Add 30-minute timeout for embedding jobs (was missing unlike analyze jobs) - Fix DropZone calling startAnalyze without setting backend URL first - Add SSE reconnect with exponential backoff (3 retries) and Last-Event-ID - Fix normalizeServerUrl to return base URL (no /api suffix) — clear contract - Delete dead code: proxy.ts, server-graph-hydration.ts, pipeline.ts re-export barrel - Update LoadingOverlay to import PipelineProgress directly from gitnexus-shared * fix(server): fix repo lock key mismatch and embed cancel race - Use getStoragePath(targetPath) as lock key in analyze handler to match embed handler's entry.storagePath — keys now always align - Guard embed completion: don't overwrite 'failed' with 'complete' when job was cancelled while pipeline was still running - Remove unused jobType parameter from acquireRepoLock - Log backend.init() errors instead of silently swallowing * fix: add gitnexus-shared as a local dependency in package-lock.json * refactor: move language detection to gitnexus-shared, add syntax highlighting for all 15 languages Move getLanguageFromFilename() from CLI to gitnexus-shared with COBOL support added. Add getSyntaxLanguageFromFilename() for Prism-compatible syntax highlighting covering all 15 code languages plus auxiliary formats (json, yaml, markdown, html, css, bash, sql, xml). Refactor CodeReferencesPanel to use shared function instead of a local 30-line switch. Delete dead gitnexus-web/src/config/supported-languages.ts (web already imports SupportedLanguages from gitnexus-shared). * feat(web): add first-time user onboarding with auto server detection Replace the manual "Connect to Server" panel with an automatic onboarding flow that guides first-time users through starting the GitNexus server. Server detection: - useBackend hook polls via setTimeout chain (3s, no overlap) - Page Visibility API pauses polling when tab is hidden - SSE heartbeat (/api/heartbeat) for instant disconnect detection Onboarding UI (OnboardingGuide.tsx): - Step-by-step flow: copy command → run → auto-connect - Smart command: shows `gitnexus serve` in dev, `npx gitnexus@latest serve` in prod - Node.js version auto-detected from package.json via Vite define - Faux terminal windows with copy-to-clipboard, platform tabs, polling indicator Transitions (DropZone.tsx): - Crossfade wrapper with snapshot pattern for smooth phase transitions - Three phases: onboarding → success (1.2s hold) → loading → graph - Auto-recovery: falls back to onboarding if server dies or connect fails Server changes: - GET /api/heartbeat: SSE endpoint for liveness detection - GET /api/info: version, launch context, Node.js version - npm run serve script for local development - app.disable('x-powered-by') hardening * feat(web): add repo analysis UI, SSE heartbeat, and review fixes Repo analysis: - AnalyzeOnboarding: empty-state card when server has zero repos - RepoAnalyzer: GitHub URL + Local Folder tabs with browse button - Header repo dropdown: click project badge to switch repos or analyze new - DropZone 'analyze' phase integrated into Crossfade transitions Reliability fixes from 5-agent review: - Polling: stop scheduling timers when tab hidden, restart on visibility return - Heartbeat: exponential backoff (1s/2s/4s, 3 retries) prevents graph loss on blip - RepoAnalyzer: completion timer tracked in ref, cleaned up on unmount - DropZone: standardized card padding (p-7), heading sizes (text-lg) Accessibility: - prefers-reduced-motion global CSS rule (WCAG 2.3.3) - focus-visible rings on CopyButton - cursor-pointer on all Header buttons - Consistent rounded-xl on all dropdowns Cleanup: - Deleted dead AnalyzeSheet.tsx (219 LOC) and BackendRepoSelector.tsx (89 LOC) - Fixed AnalyzeProgress lucide import (lucide-react → @/lib/lucide-icons) * fix(server): resolve analyze worker fork crash in dev mode The forked analyze worker was crashing immediately with exit code 1 when running via `npm run serve` (tsx). Two issues: 1. Worker path resolved to `analyze-worker.js` but only `.ts` exists in the source directory — the `.js` file is only in `dist/`. 2. On Windows, bare `--import tsx` in execArgv fails because Node's ESM resolver for --import uses the child's CWD, not the parent's node_modules. Windows also rejects raw paths as `d:` is not a valid URL scheme. Fix: detect dev vs prod via `import.meta.url` extension. In dev mode, resolve `tsx/esm` to an absolute `file://` URL via `pathToFileURL()` anchored to the parent's `createRequire` context. This works on all platforms and doesn't depend on the child's CWD or PATH. Also captures child stderr for better crash diagnostics. Verified: `POST /api/analyze` with GitHub URL completes successfully in dev mode (tsx) — status goes from cloning → analyzing → complete. * fix(server): add worker auto-retry, error handling, and crash diagnostics Worker resilience: - Auto-retry up to 2 times with exponential backoff (1s, 2s) on crash - SSE progress shows "Retrying after crash (1/2)..." during retry - Captures child stderr for crash diagnostics in failure message - AnalyzeJob tracks retryCount per job Server error handling: - app.listen wrapped in Promise so EADDRINUSE/EACCES propagate cleanly - serve.ts catches startup errors with friendly messages and exit code 1 - EADDRINUSE gets actionable guidance (stop other process or --port flag) - Global uncaughtException/unhandledRejection handlers prevent silent exits - DEBUG=1 env var shows full stack traces * feat: add e2e tests for onboarding flows, worker retry, and error handling E2E tests (onboarding.spec.ts — 11 tests): - Flow 1: OnboardingGuide shown when server unreachable (6 tests) - Flow 2: Auto-connect with success card, analyze phase for zero repos - Flow 3: Analyze form — GitHub URL validation, Local Folder tab, tab switching - Flow 4: Repo dropdown in exploring view (skipped without live server) Updated server-connect.spec.ts: - Replaced manual Connect button flow with auto-connect waitForGraphLoaded Server resilience: - Worker auto-retry (2 attempts with exponential backoff) on crash - Friendly error messages for serve startup failures (EADDRINUSE etc.) - Global uncaughtException/unhandledRejection handlers prevent silent exits - app.listen wrapped in Promise for proper error propagation * refactor(shared): enforce exhaustive language coverage via Record types Replace the if/else chain in getLanguageFromFilename with two exhaustive Record<SupportedLanguages, ...> maps: - EXTENSION_MAP: every language → its file extensions - SYNTAX_MAP: every language → its Prism syntax identifier Adding a new member to the SupportedLanguages enum without adding it to both maps now produces a TypeScript compile error: Property '[SupportedLanguages.NewLang]' is missing in type... This matches the existing pattern in languages/index.ts (providers table) which already uses `satisfies Record<SupportedLanguages, LanguageProvider>`. Three compile-time enforcement points now exist: 1. EXTENSION_MAP in language-detection.ts (file extensions) 2. SYNTAX_MAP in language-detection.ts (Prism syntax identifiers) 3. providers in languages/index.ts (LanguageProvider instances) * feat(web): load source code from server and scroll to selected line CodeReferencesPanel now fetches file content via GET /api/file when a node is selected, instead of showing "Code not available in memory". - Fetches via readFile() from backend-client when selectedFilePath changes - Shows loading spinner while fetching - After content loads, auto-scrolls to the selected node's startLine - Highlights the selected line range with a cyan left border - Cancels in-flight fetch if selection changes before it completes Also: refactored language-detection.ts to use exhaustive Record types (EXTENSION_MAP and SYNTAX_MAP) so adding a new SupportedLanguages enum member without implementing extensions/syntax is a compile error. * feat: buffered file reading for Code Inspector Server: GET /api/file now supports ?startLine=N&endLine=M for reading a line range instead of the entire file. Returns { content, startLine, endLine, totalLines }. Client: readFile() returns ReadFileResult with metadata. When selecting a symbol (function, class, method), fetches only ±50 lines around the symbol's startLine/endLine instead of the full file. File nodes still fetch the entire file. SyntaxHighlighter startingLineNumber set from the buffer offset so line numbers are correct even for partial reads. * fix: adapt readFile callers to new ReadFileResult return type tools.ts: readFile comes from GraphRAGBackend interface which returns Promise<string> (the adapter in useAppState extracts .content), so revert the { content } destructuring back to plain string assignment. useAppState.tsx: wrap backendReadFile with { repo } options object and extract .content to satisfy the GraphRAGBackend interface. * fix(web): ensure new repos appear in list immediately after analysis Two fixes: 1. DropZone: handleAnalyzeComplete now passes the repoName through to connectToServer so the specific newly-analyzed repo loads — not the server's default first repo. 2. App.tsx: fetchRepos() is now awaited BEFORE handleServerConnect in both the DropZone and Header flows. This ensures the repo list is populated before the exploring view renders, so the new repo appears in the header dropdown immediately without a page reload. * feat: delete repos, re-analyze with force, select after analysis Server — DELETE /api/repo: - Acquires repo lock first (409 if analyze/embed in flight) - Closes LadybugDB, deletes index + clone dir, unregisters, re-inits - Lock released in finally block Server — analyze complete: - backend.init() must succeed before SSE complete fires - If backend.init() fails, job is marked failed (not complete) Web — Header repo dropdown: - Re-analyze: calls POST /api/analyze with force=true, shows spinning icon + inline progress bar via SSE - Delete: acquires lock, aborts any running re-analysis SSE for same repo, refreshes list, switches to next repo - After analysis completes: refreshes repo list, connects to the specific repo by name, loads graph, shows in explorer - Retry with 1.5s backoff on 404 (server may still be reinitializing) Type safety: - err: any → err: unknown + instanceof BackendError in retry loop - Added missing BackendRepo + BackendError imports in App.tsx	2026-03-28 14:07:11 +00:00