scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-08-28 05:25:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

Merge remote-tracking branch 'origin' into feat/phase14-cross-file-binding-propagation

Browse source
This commit is contained in:
Gergo Magyar 2026-03-21 06:09:57 +00:00
commit f1fbe643df
28 changed files with 1065 additions and 89 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
README.md
View file

@ -34,7 +34,7 @@ https://github.com/user-attachments/assets/172685ba-8e54-4ea7-9ad1-e31a3398da72
> *Like DeepWiki, but deeper.* DeepWiki helps you *understand* code. GitNexus lets you *analyze* it — because a knowledge graph tracks every relationship, not just descriptions. > *Like DeepWiki, but deeper.* DeepWiki helps you *understand* code. GitNexus lets you *analyze* it — because a knowledge graph tracks every relationship, not just descriptions.
**TL;DR:** The **Web UI** is a quick way to chat with any repo. The **CLI + MCP** is how you make your AI agent actually reliable — it gives Cursor, Claude Code, and friends a deep architectural view of your codebase so they stop missing dependencies, breaking call chains, and shipping blind edits. Even smaller models get full architectural clarity, making it compete with goliath models. **TL;DR:** The **Web UI** is a quick way to chat with any repo. The **CLI + MCP** is how you make your AI agent actually reliable — it gives Cursor, Claude Code, Codex, and friends a deep architectural view of your codebase so they stop missing dependencies, breaking call chains, and shipping blind edits. Even smaller models get full architectural clarity, making it compete with goliath models.
--- ---
@ -48,7 +48,7 @@ https://github.com/user-attachments/assets/172685ba-8e54-4ea7-9ad1-e31a3398da72
| | **CLI + MCP** | **Web UI** | | | **CLI + MCP** | **Web UI** |
| ----------------- | -------------------------------------------------------------- | ------------------------------------------------------------ | | ----------------- | -------------------------------------------------------------- | ------------------------------------------------------------ |
| **What** | Index repos locally, connect AI agents via MCP | Visual graph explorer + AI chat in browser | | **What** | Index repos locally, connect AI agents via MCP | Visual graph explorer + AI chat in browser |
| **For** | Daily development with Cursor, Claude Code, Windsurf, OpenCode, Codex | Quick exploration, demos, one-off analysis | | **For** | Daily development with Cursor, Claude Code, Codex, Windsurf, OpenCode | Quick exploration, demos, one-off analysis |
| **Scale** | Full repos, any size | Limited by browser memory (~5k files), or unlimited via backend mode | | **Scale** | Full repos, any size | Limited by browser memory (~5k files), or unlimited via backend mode |
| **Install** | `npm install -g gitnexus` | No install —[gitnexus.vercel.app](https://gitnexus.vercel.app) | | **Install** | `npm install -g gitnexus` | No install —[gitnexus.vercel.app](https://gitnexus.vercel.app) |
| **Storage** | LadybugDB native (fast, persistent) | LadybugDB WASM (in-memory, per session) | | **Storage** | LadybugDB native (fast, persistent) | LadybugDB WASM (in-memory, per session) |
@ -84,17 +84,23 @@ To configure MCP for your editor, run `npx gitnexus setup` once — or set it up
| --------------------- | --- | ------ | -------------------- | -------------- | | --------------------- | --- | ------ | -------------------- | -------------- |
| **Claude Code** | Yes | Yes | Yes (PreToolUse + PostToolUse) | **Full** | | **Claude Code** | Yes | Yes | Yes (PreToolUse + PostToolUse) | **Full** |
| **Cursor** | Yes | Yes | — | MCP + Skills | | **Cursor** | Yes | Yes | — | MCP + Skills |
| **Codex** | Yes | Yes | — | MCP + Skills |
| **Windsurf** | Yes | — | — | MCP | | **Windsurf** | Yes | — | — | MCP |
| **OpenCode** | Yes | Yes | — | MCP + Skills | | **OpenCode** | Yes | Yes | — | MCP + Skills |
| **Codex** | Yes | — | — | MCP | | **Codex** | Yes | — | — | MCP |
> **Claude Code** gets the deepest integration: MCP tools + agent skills + PreToolUse hooks that enrich searches with graph context + PostToolUse hooks that auto-reindex after commits. > **Claude Code** gets the deepest integration: MCP tools + agent skills + PreToolUse hooks that enrich searches with graph context + PostToolUse hooks that auto-reindex after commits.
### Community Integrations ## Community Integrations
| Agent | Install | Source | Built by the community — not officially maintained, but worth checking out.
|-------|---------|--------|
| [pi](https://pi.dev) | `pi install npm:pi-gitnexus` | [pi-gitnexus](https://github.com/tintinweb/pi-gitnexus) | | Project | Author | Description |
|---------|--------|-------------|
| [pi-gitnexus](https://github.com/tintinweb/pi-gitnexus) | [@tintinweb](https://github.com/tintinweb) | GitNexus plugin for [pi](https://pi.dev) — `pi install npm:pi-gitnexus` |
| [gitnexus-stable-ops](https://github.com/ShunsukeHayashi/gitnexus-stable-ops) | [@ShunsukeHayashi](https://github.com/ShunsukeHayashi) | Stable ops & deployment workflows (Miyabi ecosystem) |
> Have a project built on GitNexus? Open a PR to add it here!
If you prefer manual configuration: If you prefer manual configuration:
@ -104,6 +110,12 @@ If you prefer manual configuration:
claude mcp add gitnexus -- npx -y gitnexus@latest mcp claude mcp add gitnexus -- npx -y gitnexus@latest mcp
``` ```
**Codex** (full support — MCP + skills):
```bash
codex mcp add gitnexus -- npx -y gitnexus@latest mcp
```
**Cursor** (`~/.cursor/mcp.json` — global, works for all projects): **Cursor** (`~/.cursor/mcp.json` — global, works for all projects):
```json ```json
@ -280,7 +292,7 @@ The web UI uses the same indexing pipeline as the CLI but runs entirely in WebAs
## The Problem GitNexus Solves ## The Problem GitNexus Solves
Tools like **Cursor**, **Claude Code**, **Cline**, **Roo Code**, and **Windsurf** are powerful — but they don't truly know your codebase structure. Tools like **Cursor**, **Claude Code**, **Codex**, **Cline**, **Roo Code**, and **Windsurf** are powerful — but they don't truly know your codebase structure.
**What happens:** **What happens:**

33
gitnexus-web/src/App.tsx
View file

@ -40,6 +40,7 @@ const AppContent = () => {
availableRepos, availableRepos,
setAvailableRepos, setAvailableRepos,
switchRepo, switchRepo,
hydrateWorkerFromServer,
} = useAppState(); } = useAppState();
const graphCanvasRef = useRef<GraphCanvasHandle>(null); const graphCanvasRef = useRef<GraphCanvasHandle>(null);
@ -157,21 +158,31 @@ const AppContent = () => {
// Transition directly to exploring view // Transition directly to exploring view
setViewMode('exploring'); setViewMode('exploring');
setProgress(null);
// Initialize agent if LLM is configured // Hydrate the worker-side DB (LadybugDB + BM25) so Query/Processes/embeddings work
if (getActiveProviderConfig()) { hydrateWorkerFromServer(result.nodes, result.relationships, result.fileContents).then(() => {
initializeAgent(projectName); // Initialize agent if LLM is configured
} if (getActiveProviderConfig()) {
initializeAgent(projectName);
}
// Auto-start embeddings // Auto-start embeddings (now that LadybugDB is ready)
startEmbeddings().catch((err) => { startEmbeddings().catch((err) => {
if (err?.name === 'WebGPUNotAvailableError' || err?.message?.includes('WebGPU')) { if (err?.name === 'WebGPUNotAvailableError' || err?.message?.includes('WebGPU')) {
startEmbeddings('wasm').catch(console.warn); startEmbeddings('wasm').catch(console.warn);
} else { } else {
console.warn('Embeddings auto-start failed:', err); console.warn('Embeddings auto-start failed:', err);
}
});
}).catch((err) => {
console.warn('Worker hydration failed (non-fatal):', err);
// Still initialize agent even if hydration fails
if (getActiveProviderConfig()) {
initializeAgent(projectName);
} }
}); });
}, [setViewMode, setGraph, setFileContents, setProjectName, initializeAgent, startEmbeddings]); }, [setViewMode, setGraph, setFileContents, setProjectName, setProgress, initializeAgent, startEmbeddings, hydrateWorkerFromServer]);
// Auto-connect when ?server query param is present (bookmarkable shortcut) // Auto-connect when ?server query param is present (bookmarkable shortcut)
const autoConnectRan = useRef(false); const autoConnectRan = useRef(false);

61
gitnexus-web/src/components/SettingsPanel.tsx
View file

@ -281,7 +281,7 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
if (!isOpen) return null; if (!isOpen) return null;
const providers: LLMProvider[] = ['openai', 'gemini', 'anthropic', 'azure-openai', 'ollama', 'openrouter']; const providers: LLMProvider[] = ['openai', 'gemini', 'anthropic', 'azure-openai', 'ollama', 'openrouter', 'minimax'];
return ( return (
@ -366,7 +366,7 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
w-8 h-8 rounded-lg flex items-center justify-center text-lg w-8 h-8 rounded-lg flex items-center justify-center text-lg
${settings.activeProvider === provider ? 'bg-accent/20' : 'bg-surface'} ${settings.activeProvider === provider ? 'bg-accent/20' : 'bg-surface'}
`}> `}>
{provider === 'openai' ? '🤖' : provider === 'gemini' ? '💎' : provider === 'anthropic' ? '🧠' : provider === 'ollama' ? '🦙' : provider === 'openrouter' ? '🌐' : '☁️'} {provider === 'openai' ? '🤖' : provider === 'gemini' ? '💎' : provider === 'anthropic' ? '🧠' : provider === 'ollama' ? '🦙' : provider === 'openrouter' ? '🌐' : provider === 'minimax' ? '⚡' : '☁️'}
</div> </div>
<span className="font-medium">{getProviderDisplayName(provider)}</span> <span className="font-medium">{getProviderDisplayName(provider)}</span>
</button> </button>
@ -814,7 +814,64 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
</div> </div>
)} )}
{/* MiniMax Settings */}
{settings.activeProvider === 'minimax' && (
<div className="space-y-4 animate-fade-in">
<div className="space-y-2">
<label className="flex items-center gap-2 text-sm font-medium text-text-secondary">
<Key className="w-4 h-4" />
API Key
</label>
<div className="relative">
<input
type={showApiKey['minimax'] ? 'text' : 'password'}
value={settings.minimax?.apiKey ?? ''}
onChange={e => setSettings(prev => ({
...prev,
minimax: { ...prev.minimax!, apiKey: e.target.value }
}))}
placeholder="Enter your MiniMax API key"
className="w-full px-4 py-3 pr-12 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all"
/>
<button
type="button"
onClick={() => toggleApiKeyVisibility('minimax')}
className="absolute right-3 top-1/2 -translate-y-1/2 p-1 text-text-muted hover:text-text-primary transition-colors"
>
{showApiKey['minimax'] ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
</button>
</div>
<p className="text-xs text-text-muted">
Get your API key from{' '}
<a
href="https://platform.minimax.io"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
MiniMax Platform
</a>
</p>
</div>
<div className="space-y-2">
<label className="text-sm font-medium text-text-secondary">Model</label>
<input
type="text"
value={settings.minimax?.model ?? 'MiniMax-M2.5'}
onChange={e => setSettings(prev => ({
...prev,
minimax: { ...prev.minimax!, model: e.target.value }
}))}
placeholder="e.g., MiniMax-M2.5, MiniMax-M2.5-highspeed"
className="w-full px-4 py-3 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all font-mono text-sm"
/>
<p className="text-xs text-text-muted">
Available models: MiniMax-M2.5 (default), MiniMax-M2.5-highspeed (faster)
</p>
</div>
</div>
)}
{/* Privacy Note */} {/* Privacy Note */}
<div className="p-4 bg-elevated/50 border border-border-subtle rounded-xl"> <div className="p-4 bg-elevated/50 border border-border-subtle rounded-xl">

16
gitnexus-web/src/core/lbug/lbug-adapter.ts
View file

@ -190,7 +190,7 @@ export const loadGraphToLbug = async (
for (const tableName of NODE_TABLES) { for (const tableName of NODE_TABLES) {
try { try {
const countRes = await conn.query(`MATCH (n:${tableName}) RETURN count(n) AS cnt`); const countRes = await conn.query(`MATCH (n:${tableName}) RETURN count(n) AS cnt`);
const countRows = await countRes.getAll(); const countRows = await (countRes.getAll?.() ?? countRes.getAllObjects?.() ?? countRes.getAllRows?.() ?? []);
const countRow = countRows[0]; const countRow = countRows[0];
const count = countRow ? (countRow.cnt ?? countRow[0] ?? 0) : 0; const count = countRow ? (countRow.cnt ?? countRow[0] ?? 0) : 0;
totalNodes += Number(count); totalNodes += Number(count);
@ -293,8 +293,8 @@ export const executeQuery = async (cypher: string): Promise<any[]> => {
}); });
} }
// Collect all rows // Collect all rows (handle API differences across LadybugDB versions)
const allRows = await result.getAll(); const allRows = await (result.getAll?.() ?? result.getAllObjects?.() ?? result.getAllRows?.() ?? []);
const rows: any[] = []; const rows: any[] = [];
for (const row of allRows) { for (const row of allRows) {
// Convert tuple to named object if we have column names and row is array // Convert tuple to named object if we have column names and row is array
@ -331,7 +331,7 @@ export const getLbugStats = async (): Promise<{ nodes: number; edges: number }>
for (const tableName of NODE_TABLES) { for (const tableName of NODE_TABLES) {
try { try {
const nodeResult = await conn.query(`MATCH (n:${tableName}) RETURN count(n) AS cnt`); const nodeResult = await conn.query(`MATCH (n:${tableName}) RETURN count(n) AS cnt`);
const nodeRows = await nodeResult.getAll(); const nodeRows = await (nodeResult.getAll?.() ?? nodeResult.getAllObjects?.() ?? nodeResult.getAllRows?.() ?? []);
const nodeRow = nodeRows[0]; const nodeRow = nodeRows[0];
totalNodes += Number(nodeRow?.cnt ?? nodeRow?.[0] ?? 0); totalNodes += Number(nodeRow?.cnt ?? nodeRow?.[0] ?? 0);
} catch { } catch {
@ -343,7 +343,7 @@ export const getLbugStats = async (): Promise<{ nodes: number; edges: number }>
let totalEdges = 0; let totalEdges = 0;
try { try {
const edgeResult = await conn.query(`MATCH ()-[r:${REL_TABLE_NAME}]->() RETURN count(r) AS cnt`); const edgeResult = await conn.query(`MATCH ()-[r:${REL_TABLE_NAME}]->() RETURN count(r) AS cnt`);
const edgeRows = await edgeResult.getAll(); const edgeRows = await (edgeResult.getAll?.() ?? edgeResult.getAllObjects?.() ?? edgeResult.getAllRows?.() ?? []);
const edgeRow = edgeRows[0]; const edgeRow = edgeRows[0];
totalEdges = Number(edgeRow?.cnt ?? edgeRow?.[0] ?? 0); totalEdges = Number(edgeRow?.cnt ?? edgeRow?.[0] ?? 0);
} catch { } catch {
@ -408,7 +408,7 @@ export const executePrepared = async (
const result = await conn.execute(stmt, params); const result = await conn.execute(stmt, params);
const rows = await result.getAll(); const rows = await (result.getAll?.() ?? result.getAllObjects?.() ?? result.getAllRows?.() ?? []);
await stmt.close(); await stmt.close();
return rows; return rows;
@ -472,7 +472,7 @@ export const testArrayParams = async (): Promise<{ success: boolean; error?: str
for (const tableName of NODE_TABLES) { for (const tableName of NODE_TABLES) {
try { try {
const nodeResult = await conn.query(`MATCH (n:${tableName}) RETURN n.id AS id LIMIT 1`); const nodeResult = await conn.query(`MATCH (n:${tableName}) RETURN n.id AS id LIMIT 1`);
const nodeRows = await nodeResult.getAll(); const nodeRows = await (nodeResult.getAll?.() ?? nodeResult.getAllObjects?.() ?? nodeResult.getAllRows?.() ?? []);
const nodeRow = nodeRows[0]; const nodeRow = nodeRows[0];
if (nodeRow) { if (nodeRow) {
testNodeId = nodeRow.id ?? nodeRow[0]; testNodeId = nodeRow.id ?? nodeRow[0];
@ -509,7 +509,7 @@ export const testArrayParams = async (): Promise<{ success: boolean; error?: str
const verifyResult = await conn.query( const verifyResult = await conn.query(
`MATCH (e:${EMBEDDING_TABLE_NAME} {nodeId: '${testNodeId}'}) RETURN e.embedding AS emb` `MATCH (e:${EMBEDDING_TABLE_NAME} {nodeId: '${testNodeId}'}) RETURN e.embedding AS emb`
); );
const verifyRows = await verifyResult.getAll(); const verifyRows = await (verifyResult.getAll?.() ?? verifyResult.getAllObjects?.() ?? verifyResult.getAllRows?.() ?? []);
const verifyRow = verifyRows[0]; const verifyRow = verifyRows[0];
const storedEmb = verifyRow?.emb ?? verifyRow?.[0]; const storedEmb = verifyRow?.emb ?? verifyRow?.[0];

34
gitnexus-web/src/core/llm/agent.ts
View file

@ -13,14 +13,15 @@ import { ChatAnthropic } from '@langchain/anthropic';
import { ChatOllama } from '@langchain/ollama'; import { ChatOllama } from '@langchain/ollama';
import type { BaseChatModel } from '@langchain/core/language_models/chat_models'; import type { BaseChatModel } from '@langchain/core/language_models/chat_models';
import { createGraphRAGTools } from './tools'; import { createGraphRAGTools } from './tools';
import type { import type {
ProviderConfig, ProviderConfig,
OpenAIConfig, OpenAIConfig,
AzureOpenAIConfig, AzureOpenAIConfig,
GeminiConfig, GeminiConfig,
AnthropicConfig, AnthropicConfig,
OllamaConfig, OllamaConfig,
OpenRouterConfig, OpenRouterConfig,
MiniMaxConfig,
AgentStreamChunk, AgentStreamChunk,
} from './types'; } from './types';
import { import {
@ -197,7 +198,7 @@ export const createChatModel = (config: ProviderConfig): BaseChatModel => {
case 'openrouter': { case 'openrouter': {
const openRouterConfig = config as OpenRouterConfig; const openRouterConfig = config as OpenRouterConfig;
// Debug logging // Debug logging
if (import.meta.env.DEV) { if (import.meta.env.DEV) {
console.log('🌐 OpenRouter config:', { console.log('🌐 OpenRouter config:', {
@ -207,11 +208,11 @@ export const createChatModel = (config: ProviderConfig): BaseChatModel => {
baseUrl: openRouterConfig.baseUrl, baseUrl: openRouterConfig.baseUrl,
}); });
} }
if (!openRouterConfig.apiKey || openRouterConfig.apiKey.trim() === '') { if (!openRouterConfig.apiKey || openRouterConfig.apiKey.trim() === '') {
throw new Error('OpenRouter API key is required but was not provided'); throw new Error('OpenRouter API key is required but was not provided');
} }
return new ChatOpenAI({ return new ChatOpenAI({
openAIApiKey: openRouterConfig.apiKey, openAIApiKey: openRouterConfig.apiKey,
apiKey: openRouterConfig.apiKey, // Fallback for some versions apiKey: openRouterConfig.apiKey, // Fallback for some versions
@ -225,7 +226,26 @@ export const createChatModel = (config: ProviderConfig): BaseChatModel => {
streaming: true, streaming: true,
}); });
} }
case 'minimax': {
const minimaxConfig = config as MiniMaxConfig;
if (!minimaxConfig.apiKey || minimaxConfig.apiKey.trim() === '') {
throw new Error('MiniMax API key is required but was not provided');
}
return new ChatAnthropic({
anthropicApiKey: minimaxConfig.apiKey,
model: minimaxConfig.model,
temperature: minimaxConfig.temperature ?? 0.1,
maxTokens: minimaxConfig.maxTokens ?? 8192,
streaming: true,
clientOptions: {
baseURL: 'https://api.minimax.io/anthropic',
},
});
}
default: default:
throw new Error(`Unsupported provider: ${(config as any).provider}`); throw new Error(`Unsupported provider: ${(config as any).provider}`);
} }

38
gitnexus-web/src/core/llm/settings-service.ts
View file

@ -5,9 +5,9 @@
* All API keys are stored locally - never sent to any server except the LLM provider. * All API keys are stored locally - never sent to any server except the LLM provider.
*/ */
import { import {
LLMSettings, LLMSettings,
DEFAULT_LLM_SETTINGS, DEFAULT_LLM_SETTINGS,
LLMProvider, LLMProvider,
OpenAIConfig, OpenAIConfig,
AzureOpenAIConfig, AzureOpenAIConfig,
@ -15,6 +15,7 @@ import {
AnthropicConfig, AnthropicConfig,
OllamaConfig, OllamaConfig,
OpenRouterConfig, OpenRouterConfig,
MiniMaxConfig,
ProviderConfig, ProviderConfig,
} from './types'; } from './types';
@ -60,6 +61,10 @@ export const loadSettings = (): LLMSettings => {
...DEFAULT_LLM_SETTINGS.openrouter, ...DEFAULT_LLM_SETTINGS.openrouter,
...parsed.openrouter, ...parsed.openrouter,
}, },
minimax: {
...DEFAULT_LLM_SETTINGS.minimax,
...parsed.minimax,
},
}; };
} catch (error) { } catch (error) {
console.warn('Failed to load LLM settings:', error); console.warn('Failed to load LLM settings:', error);
@ -89,6 +94,7 @@ export const updateProviderSettings = <T extends LLMProvider>(
T extends 'gemini' ? Partial<Omit<GeminiConfig, 'provider'>> : T extends 'gemini' ? Partial<Omit<GeminiConfig, 'provider'>> :
T extends 'anthropic' ? Partial<Omit<AnthropicConfig, 'provider'>> : T extends 'anthropic' ? Partial<Omit<AnthropicConfig, 'provider'>> :
T extends 'ollama' ? Partial<Omit<OllamaConfig, 'provider'>> : T extends 'ollama' ? Partial<Omit<OllamaConfig, 'provider'>> :
T extends 'minimax' ? Partial<Omit<MiniMaxConfig, 'provider'>> :
never never
> >
): LLMSettings => { ): LLMSettings => {
@ -162,6 +168,17 @@ export const updateProviderSettings = <T extends LLMProvider>(
saveSettings(updated); saveSettings(updated);
return updated; return updated;
} }
case 'minimax': {
const updated: LLMSettings = {
...current,
minimax: {
...(current.minimax ?? {}),
...(updates as Partial<Omit<MiniMaxConfig, 'provider'>>),
},
};
saveSettings(updated);
return updated;
}
default: { default: {
// Should be unreachable due to T extends LLMProvider, but keep a safe fallback // Should be unreachable due to T extends LLMProvider, but keep a safe fallback
const updated: LLMSettings = { ...current }; const updated: LLMSettings = { ...current };
@ -245,7 +262,16 @@ export const getActiveProviderConfig = (): ProviderConfig | null => {
temperature: settings.openrouter.temperature, temperature: settings.openrouter.temperature,
maxTokens: settings.openrouter.maxTokens, maxTokens: settings.openrouter.maxTokens,
} as OpenRouterConfig; } as OpenRouterConfig;
case 'minimax':
if (!settings.minimax?.apiKey) {
return null;
}
return {
provider: 'minimax',
...settings.minimax,
} as MiniMaxConfig;
default: default:
return null; return null;
} }
@ -282,6 +308,8 @@ export const getProviderDisplayName = (provider: LLMProvider): string => {
return 'Ollama (Local)'; return 'Ollama (Local)';
case 'openrouter': case 'openrouter':
return 'OpenRouter'; return 'OpenRouter';
case 'minimax':
return 'MiniMax';
default: default:
return provider; return provider;
} }
@ -303,6 +331,8 @@ export const getAvailableModels = (provider: LLMProvider): string[] => {
return ['claude-sonnet-4-20250514', 'claude-3-5-sonnet-20241022', 'claude-3-5-haiku-20241022', 'claude-3-opus-20240229']; return ['claude-sonnet-4-20250514', 'claude-3-5-sonnet-20241022', 'claude-3-5-haiku-20241022', 'claude-3-opus-20240229'];
case 'ollama': case 'ollama':
return ['llama3.2', 'llama3.1', 'mistral', 'codellama', 'deepseek-coder']; return ['llama3.2', 'llama3.1', 'mistral', 'codellama', 'deepseek-coder'];
case 'minimax':
return ['MiniMax-M2.5', 'MiniMax-M2.5-highspeed'];
default: default:
return []; return [];
} }

19
gitnexus-web/src/core/llm/types.ts
View file

@ -8,7 +8,7 @@
/** /**
* Supported LLM providers * Supported LLM providers
*/ */
export type LLMProvider = 'openai' | 'azure-openai' | 'gemini' | 'anthropic' | 'ollama' | 'openrouter'; export type LLMProvider = 'openai' | 'azure-openai' | 'gemini' | 'anthropic' | 'ollama' | 'openrouter' | 'minimax';
/** /**
* Base configuration shared by all providers * Base configuration shared by all providers
@ -78,10 +78,19 @@ export interface OpenRouterConfig extends BaseProviderConfig {
baseUrl?: string; // defaults to https://openrouter.ai/api/v1 baseUrl?: string; // defaults to https://openrouter.ai/api/v1
} }
/**
* MiniMax configuration (Anthropic-compatible API)
*/
export interface MiniMaxConfig extends BaseProviderConfig {
provider: 'minimax';
apiKey: string;
model: string; // e.g., 'MiniMax-M2.5', 'MiniMax-M2.5-highspeed'
}
/** /**
* Union type for all provider configurations * Union type for all provider configurations
*/ */
export type ProviderConfig = OpenAIConfig | AzureOpenAIConfig | GeminiConfig | AnthropicConfig | OllamaConfig | OpenRouterConfig; export type ProviderConfig = OpenAIConfig | AzureOpenAIConfig | GeminiConfig | AnthropicConfig | OllamaConfig | OpenRouterConfig | MiniMaxConfig;
/** /**
* Stored settings (what goes to localStorage) * Stored settings (what goes to localStorage)
@ -98,6 +107,7 @@ export interface LLMSettings {
anthropic?: Partial<Omit<AnthropicConfig, 'provider'>>; anthropic?: Partial<Omit<AnthropicConfig, 'provider'>>;
ollama?: Partial<Omit<OllamaConfig, 'provider'>>; ollama?: Partial<Omit<OllamaConfig, 'provider'>>;
openrouter?: Partial<Omit<OpenRouterConfig, 'provider'>>; openrouter?: Partial<Omit<OpenRouterConfig, 'provider'>>;
minimax?: Partial<Omit<MiniMaxConfig, 'provider'>>;
// Intelligent Clustering Settings // Intelligent Clustering Settings
intelligentClustering: boolean; intelligentClustering: boolean;
@ -148,6 +158,11 @@ export const DEFAULT_LLM_SETTINGS: LLMSettings = {
baseUrl: 'https://openrouter.ai/api/v1', baseUrl: 'https://openrouter.ai/api/v1',
temperature: 0.1, temperature: 0.1,
}, },
minimax: {
apiKey: '',
model: 'MiniMax-M2.5',
temperature: 0.1,
},
}; };
/** /**

36
gitnexus-web/src/hooks/useAppState.tsx
View file

@ -125,6 +125,7 @@ interface AppState {
runPipelineFromFiles: (files: FileEntry[], onProgress: (p: PipelineProgress) => void, clusteringConfig?: ProviderConfig) => Promise<PipelineResult>; runPipelineFromFiles: (files: FileEntry[], onProgress: (p: PipelineProgress) => void, clusteringConfig?: ProviderConfig) => Promise<PipelineResult>;
runQuery: (cypher: string) => Promise<any[]>; runQuery: (cypher: string) => Promise<any[]>;
isDatabaseReady: () => Promise<boolean>; isDatabaseReady: () => Promise<boolean>;
hydrateWorkerFromServer: (nodes: any[], relationships: any[], fileContents: Record<string, string>) => Promise<void>;
// Embedding state // Embedding state
embeddingStatus: EmbeddingStatus; embeddingStatus: EmbeddingStatus;
@ -482,6 +483,16 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
} }
}, []); }, []);
const hydrateWorkerFromServer = useCallback(async (
nodes: any[],
relationships: any[],
fileContents: Record<string, string>
): Promise<void> => {
const api = apiRef.current;
if (!api) throw new Error('Worker not initialized');
await api.hydrateFromServerData(nodes, relationships, fileContents);
}, []);
// Embedding methods // Embedding methods
const startEmbeddings = useCallback(async (forceDevice?: 'webgpu' | 'wasm'): Promise<void> => { const startEmbeddings = useCallback(async (forceDevice?: 'webgpu' | 'wasm'): Promise<void> => {
const api = apiRef.current; const api = apiRef.current;
@ -1018,15 +1029,23 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
setFileContents(fileMap); setFileContents(fileMap);
setViewMode('exploring'); setViewMode('exploring');
setProgress(null);
if (getActiveProviderConfig()) initializeAgent(pName); // Hydrate the worker-side DB (LadybugDB + BM25) so Query/Processes/embeddings work
hydrateWorkerFromServer(result.nodes, result.relationships, result.fileContents).then(() => {
if (getActiveProviderConfig()) initializeAgent(pName);
startEmbeddings().catch((err) => { startEmbeddings().catch((err) => {
if (err?.name === 'WebGPUNotAvailableError' || err?.message?.includes('WebGPU')) { if (err?.name === 'WebGPUNotAvailableError' || err?.message?.includes('WebGPU')) {
startEmbeddings('wasm').catch(console.warn); startEmbeddings('wasm').catch(console.warn);
} else { } else {
console.warn('Embeddings auto-start failed:', err); console.warn('Embeddings auto-start failed:', err);
} }
});
}).catch((err) => {
console.warn('Worker hydration failed (non-fatal):', err);
// Still initialize agent even if hydration fails
if (getActiveProviderConfig()) initializeAgent(pName);
}); });
} catch (err) { } catch (err) {
console.error('Repo switch failed:', err); console.error('Repo switch failed:', err);
@ -1037,7 +1056,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
}); });
setTimeout(() => { setViewMode('exploring'); setProgress(null); }, 3000); setTimeout(() => { setViewMode('exploring'); setProgress(null); }, 3000);
} }
}, [serverBaseUrl, setProgress, setViewMode, setProjectName, setGraph, setFileContents, initializeAgent, startEmbeddings, setHighlightedNodeIds, clearAIToolHighlights, clearBlastRadius, setSelectedNode, setQueryResult, setCodeReferences, setCodePanelOpen, setCodeReferenceFocus]); }, [serverBaseUrl, setProgress, setViewMode, setProjectName, setGraph, setFileContents, initializeAgent, startEmbeddings, hydrateWorkerFromServer, setHighlightedNodeIds, clearAIToolHighlights, clearBlastRadius, setSelectedNode, setQueryResult, setCodeReferences, setCodePanelOpen, setCodeReferenceFocus]);
const removeCodeReference = useCallback((id: string) => { const removeCodeReference = useCallback((id: string) => {
setCodeReferences(prev => { setCodeReferences(prev => {
@ -1142,6 +1161,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
runPipelineFromFiles, runPipelineFromFiles,
runQuery, runQuery,
isDatabaseReady, isDatabaseReady,
hydrateWorkerFromServer,
// Embedding state and methods // Embedding state and methods
embeddingStatus, embeddingStatus,
embeddingProgress, embeddingProgress,

46
gitnexus-web/src/workers/ingestion.worker.ts
View file

@ -1,5 +1,7 @@
import * as Comlink from 'comlink'; import * as Comlink from 'comlink';
import { runIngestionPipeline, runPipelineFromFiles } from '../core/ingestion/pipeline'; import { runIngestionPipeline, runPipelineFromFiles } from '../core/ingestion/pipeline';
import { createKnowledgeGraph } from '../core/graph/graph';
import type { GraphNode, GraphRelationship } from '../core/graph/types';
import { PipelineProgress, SerializablePipelineResult, serializePipelineResult } from '../types/pipeline'; import { PipelineProgress, SerializablePipelineResult, serializePipelineResult } from '../types/pipeline';
import { FileEntry } from '../services/zip'; import { FileEntry } from '../services/zip';
import { import {
@ -207,6 +209,50 @@ const workerApi = {
return serializePipelineResult(result); return serializePipelineResult(result);
}, },
/**
* Hydrate the worker-side database and indexes from server-loaded data.
* This is the missing step when using server/bridge mode — the main thread
* builds the React graph, but the worker's LadybugDB + BM25 stay empty.
*/
async hydrateFromServerData(
nodes: GraphNode[],
relationships: GraphRelationship[],
fileContents: Record<string, string>
): Promise<void> {
// 1. Build a KnowledgeGraph the same way the pipeline does
const graph = createKnowledgeGraph();
for (const node of nodes) graph.addNode(node);
for (const rel of relationships) graph.addRelationship(rel);
// 2. Store file contents for grep/read tools
storedFileContents = new Map(Object.entries(fileContents));
// 3. Build BM25 keyword index
const bm25DocCount = buildBM25Index(storedFileContents);
if (import.meta.env.DEV) {
console.log(`🔍 BM25 index built (server mode): ${bm25DocCount} documents`);
}
// 4. Set currentGraphResult so the agent context builder works
currentGraphResult = { graph, fileContents: storedFileContents };
// 5. Load graph into LadybugDB for Cypher queries (optional — gracefully degrades)
try {
const lbug = await getLbugAdapter();
await lbug.loadGraphToLbug(graph, storedFileContents);
if (import.meta.env.DEV) {
const stats = await lbug.getLbugStats();
console.log('✅ LadybugDB hydrated (server mode):', stats);
}
} catch (err) {
// LadybugDB is optional — silently continue without it
if (import.meta.env.DEV) {
console.warn('⚠️ LadybugDB hydration failed (non-fatal):', err);
}
}
},
/** /**
* Execute a Cypher query against the LadybugDB database * Execute a Cypher query against the LadybugDB database
* @param cypher - The Cypher query string * @param cypher - The Cypher query string

9
gitnexus/README.md
View file

@ -2,7 +2,7 @@
**Graph-powered code intelligence for AI agents.** Index any codebase into a knowledge graph, then query it via MCP or CLI. **Graph-powered code intelligence for AI agents.** Index any codebase into a knowledge graph, then query it via MCP or CLI.
Works with **Cursor**, **Claude Code**, **Windsurf**, **Cline**, **OpenCode**, and any MCP-compatible tool. Works with **Cursor**, **Claude Code**, **Codex**, **Windsurf**, **Cline**, **OpenCode**, and any MCP-compatible tool.
[![npm version](https://img.shields.io/npm/v/gitnexus.svg)](https://www.npmjs.com/package/gitnexus) [![npm version](https://img.shields.io/npm/v/gitnexus.svg)](https://www.npmjs.com/package/gitnexus)
[![License: PolyForm Noncommercial](https://img.shields.io/badge/License-PolyForm%20Noncommercial-blue.svg)](https://polyformproject.org/licenses/noncommercial/1.0.0/) [![License: PolyForm Noncommercial](https://img.shields.io/badge/License-PolyForm%20Noncommercial-blue.svg)](https://polyformproject.org/licenses/noncommercial/1.0.0/)
@ -34,6 +34,7 @@ To configure MCP for your editor, run `npx gitnexus setup` once — or set it up
|--------|-----|--------|---------------------|---------| |--------|-----|--------|---------------------|---------|
| **Claude Code** | Yes | Yes | Yes (PreToolUse) | **Full** | | **Claude Code** | Yes | Yes | Yes (PreToolUse) | **Full** |
| **Cursor** | Yes | Yes | — | MCP + Skills | | **Cursor** | Yes | Yes | — | MCP + Skills |
| **Codex** | Yes | Yes | — | MCP + Skills |
| **Windsurf** | Yes | — | — | MCP | | **Windsurf** | Yes | — | — | MCP |
| **OpenCode** | Yes | Yes | — | MCP + Skills | | **OpenCode** | Yes | Yes | — | MCP + Skills |
@ -55,6 +56,12 @@ If you prefer to configure manually instead of using `gitnexus setup`:
claude mcp add gitnexus -- npx -y gitnexus@latest mcp claude mcp add gitnexus -- npx -y gitnexus@latest mcp
``` ```
### Codex (full support — MCP + skills)
```bash
codex mcp add gitnexus -- npx -y gitnexus@latest mcp
```
### Cursor / Windsurf ### Cursor / Windsurf
Add to `~/.cursor/mcp.json` (global — works for all projects): Add to `~/.cursor/mcp.json` (global — works for all projects):

1
gitnexus/package.json
View file

@ -20,6 +20,7 @@
"knowledge-graph", "knowledge-graph",
"cursor", "cursor",
"claude", "claude",
"codex",
"ai-agent", "ai-agent",
"gitnexus", "gitnexus",
"static-analysis", "static-analysis",

3
gitnexus/src/cli/ai-context.ts
View file

@ -2,7 +2,7 @@
* AI Context Generator * AI Context Generator
* *
* Creates AGENTS.md and CLAUDE.md with full inline GitNexus context. * Creates AGENTS.md and CLAUDE.md with full inline GitNexus context.
* AGENTS.md is the standard read by Cursor, Windsurf, OpenCode, Cline, etc. * AGENTS.md is the standard read by Cursor, Windsurf, OpenCode, Codex, Cline, etc.
* CLAUDE.md is for Claude Code which only reads that file. * CLAUDE.md is for Claude Code which only reads that file.
*/ */
@ -308,4 +308,3 @@ export async function generateAIContextFiles(
return { files: createdFiles }; return { files: createdFiles };
} }

3
gitnexus/src/cli/index.ts
View file

@ -18,9 +18,10 @@ program
program program
.command('setup') .command('setup')
.description('One-time setup: configure MCP for Cursor, Claude Code, OpenCode') .description('One-time setup: configure MCP for Cursor, Claude Code, OpenCode, Codex')
.action(createLazyAction(() => import('./setup.js'), 'setupCommand')); .action(createLazyAction(() => import('./setup.js'), 'setupCommand'));
program program
.command('analyze [path]') .command('analyze [path]')
.description('Index a repository (full analysis)') .description('Index a repository (full analysis)')

88
gitnexus/src/cli/setup.ts
View file

@ -9,12 +9,15 @@
import fs from 'fs/promises'; import fs from 'fs/promises';
import path from 'path'; import path from 'path';
import os from 'os'; import os from 'os';
import { execFile } from 'child_process';
import { promisify } from 'util';
import { fileURLToPath } from 'url'; import { fileURLToPath } from 'url';
import { glob } from 'glob'; import { glob } from 'glob';
import { getGlobalDir } from '../storage/repo-manager.js'; import { getGlobalDir } from '../storage/repo-manager.js';
const __filename = fileURLToPath(import.meta.url); const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename); const __dirname = path.dirname(__filename);
const execFileAsync = promisify(execFile);
interface SetupResult { interface SetupResult {
configured: string[]; configured: string[];
@ -239,12 +242,75 @@ async function setupOpenCode(result: SetupResult): Promise<void> {
} }
} }
/**
* Build a TOML section for Codex MCP config (~/.codex/config.toml).
*/
function getCodexMcpTomlSection(): string {
const entry = getMcpEntry();
const command = JSON.stringify(entry.command);
const args = `[${entry.args.map(arg => JSON.stringify(arg)).join(', ')}]`;
return `[mcp_servers.gitnexus]\ncommand = ${command}\nargs = ${args}\n`;
}
/**
* Append GitNexus MCP server config to Codex's config.toml if missing.
*/
async function upsertCodexConfigToml(configPath: string): Promise<void> {
let existing = '';
try {
existing = await fs.readFile(configPath, 'utf-8');
} catch {
existing = '';
}
if (existing.includes('[mcp_servers.gitnexus]')) {
return;
}
const section = getCodexMcpTomlSection();
const nextContent = existing.trim().length > 0
? `${existing.trimEnd()}\n\n${section}`
: section;
await fs.mkdir(path.dirname(configPath), { recursive: true });
await fs.writeFile(configPath, `${nextContent.trimEnd()}\n`, 'utf-8');
}
async function setupCodex(result: SetupResult): Promise<void> {
const codexDir = path.join(os.homedir(), '.codex');
if (!(await dirExists(codexDir))) {
result.skipped.push('Codex (not installed)');
return;
}
try {
const entry = getMcpEntry();
await execFileAsync(
'codex',
['mcp', 'add', 'gitnexus', '--', entry.command, ...entry.args],
{ shell: process.platform === 'win32' }
);
result.configured.push('Codex');
return;
} catch {
// Fallback for environments where `codex` binary isn't on PATH.
}
try {
const configPath = path.join(codexDir, 'config.toml');
await upsertCodexConfigToml(configPath);
result.configured.push('Codex (MCP added to ~/.codex/config.toml)');
} catch (err: any) {
result.errors.push(`Codex: ${err.message}`);
}
}
// ─── Skill Installation ─────────────────────────────────────────── // ─── Skill Installation ───────────────────────────────────────────
/** /**
* Install GitNexus skills to a target directory. * Install GitNexus skills to a target directory.
* Each skill is installed as {targetDir}/gitnexus-{skillName}/SKILL.md * Each skill is installed as {targetDir}/gitnexus-{skillName}/SKILL.md
* following the Agent Skills standard (both Cursor and Claude Code). * following the Agent Skills standard (Cursor, Claude Code, and Codex).
* *
* Supports two source layouts: * Supports two source layouts:
* - Flat file: skills/{name}.md → copied as SKILL.md * - Flat file: skills/{name}.md → copied as SKILL.md
@ -353,6 +419,24 @@ async function installOpenCodeSkills(result: SetupResult): Promise<void> {
} }
} }
/**
* Install global Codex skills to ~/.agents/skills/gitnexus/
*/
async function installCodexSkills(result: SetupResult): Promise<void> {
const codexDir = path.join(os.homedir(), '.codex');
if (!(await dirExists(codexDir))) return;
const skillsDir = path.join(os.homedir(), '.agents', 'skills');
try {
const installed = await installSkillsTo(skillsDir);
if (installed.length > 0) {
result.configured.push(`Codex skills (${installed.length} skills → ~/.agents/skills/)`);
}
} catch (err: any) {
result.errors.push(`Codex skills: ${err.message}`);
}
}
// ─── Main command ────────────────────────────────────────────────── // ─── Main command ──────────────────────────────────────────────────
export const setupCommand = async () => { export const setupCommand = async () => {
@ -375,12 +459,14 @@ export const setupCommand = async () => {
await setupCursor(result); await setupCursor(result);
await setupClaudeCode(result); await setupClaudeCode(result);
await setupOpenCode(result); await setupOpenCode(result);
await setupCodex(result);
// Install global skills for platforms that support them // Install global skills for platforms that support them
await installClaudeCodeSkills(result); await installClaudeCodeSkills(result);
await installClaudeCodeHooks(result); await installClaudeCodeHooks(result);
await installCursorSkills(result); await installCursorSkills(result);
await installOpenCodeSkills(result); await installOpenCodeSkills(result);
await installCodexSkills(result);
// Print results // Print results
if (result.configured.length > 0) { if (result.configured.length > 0) {

5
gitnexus/src/core/graph/types.ts
View file

@ -32,7 +32,8 @@ export type NodeLabel =
| 'Delegate' | 'Delegate'
| 'Annotation' | 'Annotation'
| 'Constructor' | 'Constructor'
| 'Template'; | 'Template'
| 'Section';
import { SupportedLanguages } from '../../config/supported-languages.js'; import { SupportedLanguages } from '../../config/supported-languages.js';
@ -65,6 +66,8 @@ export type NodeProperties = {
entryPointReason?: string, entryPointReason?: string,
// Method signature (for MRO disambiguation) // Method signature (for MRO disambiguation)
parameterCount?: number, parameterCount?: number,
// Section-specific (markdown heading level, 1-6)
level?: number,
returnType?: string, returnType?: string,
} }

157
gitnexus/src/core/ingestion/markdown-processor.ts Normal file
View file

@ -0,0 +1,157 @@
/**
* Markdown Processor
*
* Extracts structure from .md files using regex (no tree-sitter dependency).
* Creates Section nodes for headings with hierarchy, and IMPORTS edges for
* cross-file links.
*/
import path from 'node:path';
import { generateId } from '../../lib/utils.js';
import { KnowledgeGraph, GraphNode, GraphRelationship } from '../graph/types.js';
const HEADING_RE = /^(#{1,6})\s+(.+)$/;
const LINK_RE = /\[([^\]]*)\]\(([^)]+)\)/g;
const MD_EXTENSIONS = new Set(['.md', '.mdx']);
interface MdFile {
path: string;
content: string;
}
export const processMarkdown = (
graph: KnowledgeGraph,
files: MdFile[],
allPathSet: Set<string>,
): { sections: number; links: number } => {
let totalSections = 0;
let totalLinks = 0;
for (const file of files) {
const ext = path.extname(file.path).toLowerCase();
if (!MD_EXTENSIONS.has(ext)) continue;
const fileNodeId = generateId('File', file.path);
// Skip if file node doesn't exist (shouldn't happen, structure-processor creates it)
if (!graph.getNode(fileNodeId)) continue;
const lines = file.content.split('\n');
// --- Extract headings and build hierarchy ---
// First pass: collect all heading positions so we can compute endLine spans
const headings: { level: number; heading: string; lineNum: number }[] = [];
for (let i = 0; i < lines.length; i++) {
const match = lines[i].match(HEADING_RE);
if (!match) continue;
headings.push({
level: match[1].length,
heading: match[2].trim(),
lineNum: i + 1, // 1-indexed
});
}
// Second pass: create nodes with proper endLine spans
const sectionStack: { level: number; id: string }[] = [];
for (let h = 0; h < headings.length; h++) {
const { level, heading, lineNum } = headings[h];
// endLine = line before next heading at same or higher level, or EOF
let endLine = lines.length;
for (let j = h + 1; j < headings.length; j++) {
if (headings[j].level <= level) {
endLine = headings[j].lineNum - 1;
break;
}
}
const sectionId = generateId('Section', `${file.path}:L${lineNum}:${heading}`);
const node: GraphNode = {
id: sectionId,
label: 'Section',
properties: {
name: heading,
filePath: file.path,
startLine: lineNum,
endLine,
level,
description: `h${level}`,
},
};
graph.addNode(node);
totalSections++;
// Find parent: pop stack until we find a level strictly less than current
while (sectionStack.length > 0 && sectionStack[sectionStack.length - 1].level >= level) {
sectionStack.pop();
}
const parentId = sectionStack.length > 0
? sectionStack[sectionStack.length - 1].id
: fileNodeId;
graph.addRelationship({
id: generateId('CONTAINS', `${parentId}->${sectionId}`),
type: 'CONTAINS',
sourceId: parentId,
targetId: sectionId,
confidence: 1.0,
reason: 'markdown-heading',
});
sectionStack.push({ level, id: sectionId });
}
// --- Extract links to other files in the repo ---
const fileDir = path.dirname(file.path);
const seenLinks = new Set<string>();
let linkMatch: RegExpExecArray | null;
LINK_RE.lastIndex = 0;
while ((linkMatch = LINK_RE.exec(file.content)) !== null) {
const href = linkMatch[2];
// Skip external URLs, anchors, and mailto
if (href.startsWith('http://') || href.startsWith('https://') ||
href.startsWith('#') || href.startsWith('mailto:')) {
continue;
}
// Strip anchor fragments from local links
const cleanHref = href.split('#')[0];
if (!cleanHref) continue;
// Resolve relative to the file's directory, then normalize
const resolved = path.posix.normalize(path.posix.join(fileDir, cleanHref));
if (allPathSet.has(resolved)) {
const targetFileId = generateId('File', resolved);
// Skip if target file node doesn't exist
if (!graph.getNode(targetFileId)) continue;
// Dedup: skip if we've already linked this file pair
const linkKey = `${fileNodeId}->${targetFileId}`;
if (seenLinks.has(linkKey)) continue;
seenLinks.add(linkKey);
const relId = generateId('IMPORTS', linkKey);
graph.addRelationship({
id: relId,
type: 'IMPORTS',
sourceId: fileNodeId,
targetId: targetFileId,
confidence: 0.8,
reason: 'markdown-link',
});
totalLinks++;
}
}
}
return { sections: totalSections, links: totalLinks };
};

16
gitnexus/src/core/ingestion/pipeline.ts
View file

@ -1,5 +1,6 @@
import { createKnowledgeGraph } from '../graph/graph.js'; import { createKnowledgeGraph } from '../graph/graph.js';
import { processStructure } from './structure-processor.js'; import { processStructure } from './structure-processor.js';
import { processMarkdown } from './markdown-processor.js';
import { processParsing } from './parsing-processor.js'; import { processParsing } from './parsing-processor.js';
import { import {
processImports, processImports,
@ -298,6 +299,21 @@ export const runPipelineFromRepo = async (
stats: { filesProcessed: totalFiles, totalFiles, nodesCreated: graph.nodeCount }, stats: { filesProcessed: totalFiles, totalFiles, nodesCreated: graph.nodeCount },
}); });
// ── Phase 2.5: Markdown processing (headings + cross-links) ────────
const mdScanned = scannedFiles.filter(f => f.path.endsWith('.md') || f.path.endsWith('.mdx'));
if (mdScanned.length > 0) {
const mdContents = await readFileContents(repoPath, mdScanned.map(f => f.path));
const mdFiles = mdScanned
.filter(f => mdContents.has(f.path))
.map(f => ({ path: f.path, content: mdContents.get(f.path)! }));
const allPathSet = new Set(allPaths);
const mdResult = processMarkdown(graph, mdFiles, allPathSet);
if (isDev) {
console.log(` Markdown: ${mdResult.sections} sections, ${mdResult.links} cross-links from ${mdFiles.length} files`);
}
}
// ── Phase 3+4: Chunked read + parse ──────────────────────────────── // ── Phase 3+4: Chunked read + parse ────────────────────────────────
// Group parseable files into byte-budget chunks so only ~20MB of source // Group parseable files into byte-budget chunks so only ~20MB of source
// is in memory at a time. Each chunk is: read → parse → extract → free. // is in memory at a time. Each chunk is: read → parse → extract → free.

20
gitnexus/src/core/lbug/csv-generator.ts
View file

@ -238,6 +238,9 @@ export const streamAllCSVsToDisk = async (
const communityWriter = new BufferedCSVWriter(path.join(csvDir, 'community.csv'), 'id,label,heuristicLabel,keywords,description,enrichedBy,cohesion,symbolCount'); const communityWriter = new BufferedCSVWriter(path.join(csvDir, 'community.csv'), 'id,label,heuristicLabel,keywords,description,enrichedBy,cohesion,symbolCount');
const processWriter = new BufferedCSVWriter(path.join(csvDir, 'process.csv'), 'id,label,heuristicLabel,processType,stepCount,communities,entryPointId,terminalId'); const processWriter = new BufferedCSVWriter(path.join(csvDir, 'process.csv'), 'id,label,heuristicLabel,processType,stepCount,communities,entryPointId,terminalId');
// Section nodes have an extra 'level' column
const sectionWriter = new BufferedCSVWriter(path.join(csvDir, 'section.csv'), 'id,name,filePath,startLine,endLine,level,content,description');
// Multi-language node types share the same CSV shape (no isExported column) // Multi-language node types share the same CSV shape (no isExported column)
const multiLangHeader = 'id,name,filePath,startLine,endLine,content,description'; const multiLangHeader = 'id,name,filePath,startLine,endLine,content,description';
const MULTI_LANG_TYPES = ['Struct', 'Enum', 'Macro', 'Typedef', 'Union', 'Namespace', 'Trait', 'Impl', const MULTI_LANG_TYPES = ['Struct', 'Enum', 'Macro', 'Typedef', 'Union', 'Namespace', 'Trait', 'Impl',
@ -324,6 +327,20 @@ export const streamAllCSVsToDisk = async (
].join(',')); ].join(','));
break; break;
} }
case 'Section': {
const content = await extractContent(node, contentCache);
await sectionWriter.addRow([
escapeCSVField(node.id),
escapeCSVField(node.properties.name || ''),
escapeCSVField(node.properties.filePath || ''),
escapeCSVNumber(node.properties.startLine, -1),
escapeCSVNumber(node.properties.endLine, -1),
escapeCSVNumber((node.properties as any).level, 1),
escapeCSVField(content),
escapeCSVField((node.properties as any).description || ''),
].join(','));
break;
}
default: { default: {
// Code element nodes (Function, Class, Interface, CodeElement) // Code element nodes (Function, Class, Interface, CodeElement)
const writer = codeWriterMap[node.label]; const writer = codeWriterMap[node.label];
@ -361,7 +378,7 @@ export const streamAllCSVsToDisk = async (
} }
// Finish all node writers // Finish all node writers
const allWriters = [fileWriter, folderWriter, functionWriter, classWriter, interfaceWriter, methodWriter, codeElemWriter, communityWriter, processWriter, ...multiLangWriters.values()]; const allWriters = [fileWriter, folderWriter, functionWriter, classWriter, interfaceWriter, methodWriter, codeElemWriter, communityWriter, processWriter, sectionWriter, ...multiLangWriters.values()];
await Promise.all(allWriters.map(w => w.finish())); await Promise.all(allWriters.map(w => w.finish()));
// --- Stream relationship CSV --- // --- Stream relationship CSV ---
@ -387,6 +404,7 @@ export const streamAllCSVsToDisk = async (
['Interface', interfaceWriter], ['Method', methodWriter], ['Interface', interfaceWriter], ['Method', methodWriter],
['CodeElement', codeElemWriter], ['CodeElement', codeElemWriter],
['Community', communityWriter], ['Process', processWriter], ['Community', communityWriter], ['Process', processWriter],
['Section' as NodeTableName, sectionWriter],
...Array.from(multiLangWriters.entries()).map(([name, w]) => [name as NodeTableName, w] as [NodeTableName, BufferedCSVWriter]), ...Array.from(multiLangWriters.entries()).map(([name, w]) => [name as NodeTableName, w] as [NodeTableName, BufferedCSVWriter]),
]; ];
for (const [name, writer] of tableMap) { for (const [name, writer] of tableMap) {

9
gitnexus/src/core/lbug/lbug-adapter.ts
View file

@ -336,6 +336,9 @@ const getCopyQuery = (table: NodeTableName, filePath: string): string => {
if (table === 'Process') { if (table === 'Process') {
return `COPY ${t}(id, label, heuristicLabel, processType, stepCount, communities, entryPointId, terminalId) FROM "${filePath}" ${COPY_CSV_OPTS}`; return `COPY ${t}(id, label, heuristicLabel, processType, stepCount, communities, entryPointId, terminalId) FROM "${filePath}" ${COPY_CSV_OPTS}`;
} }
if (table === 'Section') {
return `COPY ${t}(id, name, filePath, startLine, endLine, level, content, description) FROM "${filePath}" ${COPY_CSV_OPTS}`;
}
if (table === 'Method') { if (table === 'Method') {
return `COPY ${t}(id, name, filePath, startLine, endLine, isExported, content, description, parameterCount, returnType) FROM "${filePath}" ${COPY_CSV_OPTS}`; return `COPY ${t}(id, name, filePath, startLine, endLine, isExported, content, description, parameterCount, returnType) FROM "${filePath}" ${COPY_CSV_OPTS}`;
} }
@ -380,6 +383,9 @@ export const insertNodeToLbug = async (
query = `CREATE (n:File {id: ${escapeValue(properties.id)}, name: ${escapeValue(properties.name)}, filePath: ${escapeValue(properties.filePath)}, content: ${escapeValue(properties.content || '')}})`; query = `CREATE (n:File {id: ${escapeValue(properties.id)}, name: ${escapeValue(properties.name)}, filePath: ${escapeValue(properties.filePath)}, content: ${escapeValue(properties.content || '')}})`;
} else if (label === 'Folder') { } else if (label === 'Folder') {
query = `CREATE (n:Folder {id: ${escapeValue(properties.id)}, name: ${escapeValue(properties.name)}, filePath: ${escapeValue(properties.filePath)}})`; query = `CREATE (n:Folder {id: ${escapeValue(properties.id)}, name: ${escapeValue(properties.name)}, filePath: ${escapeValue(properties.filePath)}})`;
} else if (label === 'Section') {
const descPart = properties.description ? `, description: ${escapeValue(properties.description)}` : '';
query = `CREATE (n:Section {id: ${escapeValue(properties.id)}, name: ${escapeValue(properties.name)}, filePath: ${escapeValue(properties.filePath)}, startLine: ${properties.startLine || 0}, endLine: ${properties.endLine || 0}, level: ${properties.level || 1}, content: ${escapeValue(properties.content || '')}${descPart}})`;
} else if (TABLES_WITH_EXPORTED.has(label)) { } else if (TABLES_WITH_EXPORTED.has(label)) {
const descPart = properties.description ? `, description: ${escapeValue(properties.description)}` : ''; const descPart = properties.description ? `, description: ${escapeValue(properties.description)}` : '';
query = `CREATE (n:${t} {id: ${escapeValue(properties.id)}, name: ${escapeValue(properties.name)}, filePath: ${escapeValue(properties.filePath)}, startLine: ${properties.startLine || 0}, endLine: ${properties.endLine || 0}, isExported: ${!!properties.isExported}, content: ${escapeValue(properties.content || '')}${descPart}})`; query = `CREATE (n:${t} {id: ${escapeValue(properties.id)}, name: ${escapeValue(properties.name)}, filePath: ${escapeValue(properties.filePath)}, startLine: ${properties.startLine || 0}, endLine: ${properties.endLine || 0}, isExported: ${!!properties.isExported}, content: ${escapeValue(properties.content || '')}${descPart}})`;
@ -451,6 +457,9 @@ export const batchInsertNodesToLbug = async (
query = `MERGE (n:File {id: ${escapeValue(properties.id)}}) SET n.name = ${escapeValue(properties.name)}, n.filePath = ${escapeValue(properties.filePath)}, n.content = ${escapeValue(properties.content || '')}`; query = `MERGE (n:File {id: ${escapeValue(properties.id)}}) SET n.name = ${escapeValue(properties.name)}, n.filePath = ${escapeValue(properties.filePath)}, n.content = ${escapeValue(properties.content || '')}`;
} else if (label === 'Folder') { } else if (label === 'Folder') {
query = `MERGE (n:Folder {id: ${escapeValue(properties.id)}}) SET n.name = ${escapeValue(properties.name)}, n.filePath = ${escapeValue(properties.filePath)}`; query = `MERGE (n:Folder {id: ${escapeValue(properties.id)}}) SET n.name = ${escapeValue(properties.name)}, n.filePath = ${escapeValue(properties.filePath)}`;
} else if (label === 'Section') {
const descPart = properties.description ? `, n.description = ${escapeValue(properties.description)}` : '';
query = `MERGE (n:Section {id: ${escapeValue(properties.id)}}) SET n.name = ${escapeValue(properties.name)}, n.filePath = ${escapeValue(properties.filePath)}, n.startLine = ${properties.startLine || 0}, n.endLine = ${properties.endLine || 0}, n.level = ${properties.level || 1}, n.content = ${escapeValue(properties.content || '')}${descPart}`;
} else if (TABLES_WITH_EXPORTED.has(label)) { } else if (TABLES_WITH_EXPORTED.has(label)) {
const descPart = properties.description ? `, n.description = ${escapeValue(properties.description)}` : ''; const descPart = properties.description ? `, n.description = ${escapeValue(properties.description)}` : '';
query = `MERGE (n:${t} {id: ${escapeValue(properties.id)}}) SET n.name = ${escapeValue(properties.name)}, n.filePath = ${escapeValue(properties.filePath)}, n.startLine = ${properties.startLine || 0}, n.endLine = ${properties.endLine || 0}, n.isExported = ${!!properties.isExported}, n.content = ${escapeValue(properties.content || '')}${descPart}`; query = `MERGE (n:${t} {id: ${escapeValue(properties.id)}}) SET n.name = ${escapeValue(properties.name)}, n.filePath = ${escapeValue(properties.filePath)}, n.startLine = ${properties.startLine || 0}, n.endLine = ${properties.endLine || 0}, n.isExported = ${!!properties.isExported}, n.content = ${escapeValue(properties.content || '')}${descPart}`;

20
gitnexus/src/core/lbug/schema.ts
View file

@ -13,7 +13,7 @@
// NODE TABLE NAMES // NODE TABLE NAMES
// ============================================================================ // ============================================================================
export const NODE_TABLES = [ export const NODE_TABLES = [
'File', 'Folder', 'Function', 'Class', 'Interface', 'Method', 'CodeElement', 'Community', 'Process', 'File', 'Folder', 'Function', 'Class', 'Interface', 'Method', 'CodeElement', 'Community', 'Process', 'Section',
// Multi-language support // Multi-language support
'Struct', 'Enum', 'Macro', 'Typedef', 'Union', 'Namespace', 'Trait', 'Impl', 'Struct', 'Enum', 'Macro', 'Typedef', 'Union', 'Namespace', 'Trait', 'Impl',
'TypeAlias', 'Const', 'Static', 'Property', 'Record', 'Delegate', 'Annotation', 'Constructor', 'Template', 'Module' 'TypeAlias', 'Const', 'Static', 'Property', 'Record', 'Delegate', 'Annotation', 'Constructor', 'Template', 'Module'
@ -192,6 +192,19 @@ export const ANNOTATION_SCHEMA = CODE_ELEMENT_BASE('Annotation');
export const CONSTRUCTOR_SCHEMA = CODE_ELEMENT_BASE('Constructor'); export const CONSTRUCTOR_SCHEMA = CODE_ELEMENT_BASE('Constructor');
export const TEMPLATE_SCHEMA = CODE_ELEMENT_BASE('Template'); export const TEMPLATE_SCHEMA = CODE_ELEMENT_BASE('Template');
export const MODULE_SCHEMA = CODE_ELEMENT_BASE('Module'); export const MODULE_SCHEMA = CODE_ELEMENT_BASE('Module');
// Markdown heading sections
export const SECTION_SCHEMA = `
CREATE NODE TABLE Section (
id STRING,
name STRING,
filePath STRING,
startLine INT64,
endLine INT64,
level INT64,
content STRING,
description STRING,
PRIMARY KEY (id)
)`;
// ============================================================================ // ============================================================================
// RELATION TABLE SCHEMA // RELATION TABLE SCHEMA
@ -225,6 +238,7 @@ CREATE REL TABLE ${REL_TABLE_NAME} (
FROM File TO \`Constructor\`, FROM File TO \`Constructor\`,
FROM File TO \`Template\`, FROM File TO \`Template\`,
FROM File TO \`Module\`, FROM File TO \`Module\`,
FROM File TO Section,
FROM Folder TO Folder, FROM Folder TO Folder,
FROM Folder TO File, FROM Folder TO File,
FROM Function TO Function, FROM Function TO Function,
@ -289,6 +303,8 @@ CREATE REL TABLE ${REL_TABLE_NAME} (
FROM \`Template\` TO Interface, FROM \`Template\` TO Interface,
FROM \`Template\` TO \`Constructor\`, FROM \`Template\` TO \`Constructor\`,
FROM \`Module\` TO \`Module\`, FROM \`Module\` TO \`Module\`,
FROM Section TO Section,
FROM Section TO File,
FROM CodeElement TO Community, FROM CodeElement TO Community,
FROM Interface TO Community, FROM Interface TO Community,
FROM Interface TO Function, FROM Interface TO Function,
@ -447,6 +463,8 @@ export const NODE_SCHEMA_QUERIES = [
CONSTRUCTOR_SCHEMA, CONSTRUCTOR_SCHEMA,
TEMPLATE_SCHEMA, TEMPLATE_SCHEMA,
MODULE_SCHEMA, MODULE_SCHEMA,
// Markdown support
SECTION_SCHEMA,
]; ];
export const REL_SCHEMA_QUERIES = [ export const REL_SCHEMA_QUERIES = [

14
gitnexus/src/mcp/core/lbug-adapter.ts
View file

@ -148,6 +148,8 @@ function closeOne(repoId: string): void {
* Create a new Connection from a repo's Database. * Create a new Connection from a repo's Database.
* Silences stdout to prevent native module output from corrupting MCP stdio. * Silences stdout to prevent native module output from corrupting MCP stdio.
*/ */
let activeQueryCount = 0;
function silenceStdout(): void { function silenceStdout(): void {
if (stdoutSilenceCount++ === 0) { if (stdoutSilenceCount++ === 0) {
process.stdout.write = (() => true) as any; process.stdout.write = (() => true) as any;
@ -163,8 +165,10 @@ function restoreStdout(): void {
// Safety watchdog: restore stdout if it gets stuck silenced (e.g. native crash // Safety watchdog: restore stdout if it gets stuck silenced (e.g. native crash
// inside createConnection before restoreStdout runs). // inside createConnection before restoreStdout runs).
// Exempts active queries and pre-warm — these legitimately hold silence for
// longer than 1 second (queries can take up to QUERY_TIMEOUT_MS = 30s).
setInterval(() => { setInterval(() => {
if (stdoutSilenceCount > 0 && !preWarmActive) { if (stdoutSilenceCount > 0 && !preWarmActive && activeQueryCount === 0) {
stdoutSilenceCount = 0; stdoutSilenceCount = 0;
process.stdout.write = realStdoutWrite; process.stdout.write = realStdoutWrite;
} }
@ -458,12 +462,16 @@ export const executeQuery = async (repoId: string, cypher: string): Promise<any[
entry.lastUsed = Date.now(); entry.lastUsed = Date.now();
const conn = await checkout(entry); const conn = await checkout(entry);
silenceStdout();
activeQueryCount++;
try { try {
const queryResult = await withTimeout(conn.query(cypher), QUERY_TIMEOUT_MS, 'Query'); const queryResult = await withTimeout(conn.query(cypher), QUERY_TIMEOUT_MS, 'Query');
const result = Array.isArray(queryResult) ? queryResult[0] : queryResult; const result = Array.isArray(queryResult) ? queryResult[0] : queryResult;
const rows = await result.getAll(); const rows = await result.getAll();
return rows; return rows;
} finally { } finally {
activeQueryCount--;
restoreStdout();
checkin(entry, conn); checkin(entry, conn);
} }
}; };
@ -485,6 +493,8 @@ export const executeParameterized = async (
entry.lastUsed = Date.now(); entry.lastUsed = Date.now();
const conn = await checkout(entry); const conn = await checkout(entry);
silenceStdout();
activeQueryCount++;
try { try {
const stmt = await withTimeout(conn.prepare(cypher), QUERY_TIMEOUT_MS, 'Prepare'); const stmt = await withTimeout(conn.prepare(cypher), QUERY_TIMEOUT_MS, 'Prepare');
if (!stmt.isSuccess()) { if (!stmt.isSuccess()) {
@ -496,6 +506,8 @@ export const executeParameterized = async (
const rows = await result.getAll(); const rows = await result.getAll();
return rows; return rows;
} finally { } finally {
activeQueryCount--;
restoreStdout();
checkin(entry, conn); checkin(entry, conn);
} }
}; };

113
gitnexus/src/mcp/local/local-backend.ts
View file

@ -88,6 +88,8 @@ export class LocalBackend {
private repos: Map<string, RepoHandle> = new Map(); private repos: Map<string, RepoHandle> = new Map();
private contextCache: Map<string, CodebaseContext> = new Map(); private contextCache: Map<string, CodebaseContext> = new Map();
private initializedRepos: Set<string> = new Set(); private initializedRepos: Set<string> = new Set();
private reinitPromises: Map<string, Promise<void>> = new Map();
private lastStalenessCheck: Map<string, number> = new Map();
// ─── Initialization ────────────────────────────────────────────── // ─── Initialization ──────────────────────────────────────────────
@ -246,12 +248,51 @@ export class LocalBackend {
// ─── Lazy LadybugDB Init ──────────────────────────────────────────── // ─── Lazy LadybugDB Init ────────────────────────────────────────────
private async ensureInitialized(repoId: string): Promise<void> { private async ensureInitialized(repoId: string): Promise<void> {
// Always check the actual pool — the idle timer may have evicted the connection // If a reinit is already in progress for this repo, wait for it
if (this.initializedRepos.has(repoId) && isLbugReady(repoId)) return; const pending = this.reinitPromises.get(repoId);
if (pending) return pending;
const handle = this.repos.get(repoId); const handle = this.repos.get(repoId);
if (!handle) throw new Error(`Unknown repo: ${repoId}`); if (!handle) throw new Error(`Unknown repo: ${repoId}`);
// Check if the index was rebuilt since we opened the connection (#297).
// Throttle staleness checks to at most once per 5 seconds per repo to
// avoid an fs.readFile round-trip on every tool invocation.
if (this.initializedRepos.has(repoId) && isLbugReady(repoId)) {
const now = Date.now();
const lastCheck = this.lastStalenessCheck.get(repoId) ?? 0;
if (now - lastCheck < 5000) return; // Checked recently — skip
this.lastStalenessCheck.set(repoId, now);
try {
const metaPath = path.join(handle.storagePath, 'meta.json');
const metaRaw = await fs.readFile(metaPath, 'utf-8');
const meta = JSON.parse(metaRaw);
if (meta.indexedAt && meta.indexedAt !== handle.indexedAt) {
// Index was rebuilt — close stale connection and re-init.
// Wrap in reinitPromises to prevent TOCTOU race where concurrent
// callers both detect staleness and double-close the pool.
const reinit = (async () => {
try {
await closeLbug(repoId);
this.initializedRepos.delete(repoId);
handle.indexedAt = meta.indexedAt;
await initLbug(repoId, handle.lbugPath);
this.initializedRepos.add(repoId);
} finally {
this.reinitPromises.delete(repoId);
}
})();
this.reinitPromises.set(repoId, reinit);
return reinit;
} else {
return; // Pool is current
}
} catch {
return; // Can't read meta — assume pool is fine
}
}
try { try {
await initLbug(repoId, handle.lbugPath); await initLbug(repoId, handle.lbugPath);
this.initializedRepos.add(repoId); this.initializedRepos.add(repoId);
@ -1438,31 +1479,51 @@ export class LocalBackend {
let affectedModules: any[] = []; let affectedModules: any[] = [];
if (impacted.length > 0) { if (impacted.length > 0) {
const allIds = impacted.map(i => `'${i.id.replace(/'/g, "''")}'`).join(', '); // Cap IN-clause to 100 IDs to prevent oversized queries that crash
const d1Ids = (grouped[1] || []).map((i: any) => `'${i.id.replace(/'/g, "''")}'`).join(', '); // the native DB engine on arm64 macOS (#292)
const cappedImpacted = impacted.slice(0, 100);
const allIds = cappedImpacted.map(i => `'${String(i.id ?? '').replace(/'/g, "''")}'`).join(', ');
const d1Items = (grouped[1] || []).slice(0, 100);
const d1Ids = d1Items.map((i: any) => `'${String(i.id ?? '').replace(/'/g, "''")}'`).join(', ');
// Affected processes: which execution flows are broken and at which step // Enrichment queries: sequential on arm64 macOS to avoid SIGSEGV from
const [processRows, moduleRows, directModuleRows] = await Promise.all([ // concurrent native DB access (#285, #290, #292); parallel elsewhere
executeQuery(repo.id, ` // to preserve performance on unaffected platforms.
MATCH (s)-[r:CodeRelation {type: 'STEP_IN_PROCESS'}]->(p:Process) const isArm64Mac = process.platform === 'darwin' && process.arch === 'arm64';
WHERE s.id IN [${allIds}]
RETURN p.heuristicLabel AS name, COUNT(DISTINCT s.id) AS hits, MIN(r.step) AS minStep, p.stepCount AS stepCount const processQuery = executeQuery(repo.id, `
ORDER BY hits DESC MATCH (s)-[r:CodeRelation {type: 'STEP_IN_PROCESS'}]->(p:Process)
LIMIT 20 WHERE s.id IN [${allIds}]
`).catch(() => []), RETURN p.heuristicLabel AS name, COUNT(DISTINCT s.id) AS hits, MIN(r.step) AS minStep, p.stepCount AS stepCount
executeQuery(repo.id, ` ORDER BY hits DESC
MATCH (s)-[:CodeRelation {type: 'MEMBER_OF'}]->(c:Community) LIMIT 20
WHERE s.id IN [${allIds}] `).catch(() => []);
RETURN c.heuristicLabel AS name, COUNT(DISTINCT s.id) AS hits const moduleQuery = () => executeQuery(repo.id, `
ORDER BY hits DESC MATCH (s)-[:CodeRelation {type: 'MEMBER_OF'}]->(c:Community)
LIMIT 20 WHERE s.id IN [${allIds}]
`).catch(() => []), RETURN c.heuristicLabel AS name, COUNT(DISTINCT s.id) AS hits
d1Ids ? executeQuery(repo.id, ` ORDER BY hits DESC
MATCH (s)-[:CodeRelation {type: 'MEMBER_OF'}]->(c:Community) LIMIT 20
WHERE s.id IN [${d1Ids}] `).catch(() => []);
RETURN DISTINCT c.heuristicLabel AS name const directModuleQuery = () => d1Ids
`).catch(() => []) : Promise.resolve([]), ? executeQuery(repo.id, `
]); MATCH (s)-[:CodeRelation {type: 'MEMBER_OF'}]->(c:Community)
WHERE s.id IN [${d1Ids}]
RETURN DISTINCT c.heuristicLabel AS name
`).catch(() => [])
: Promise.resolve([]);
let processRows: any[], moduleRows: any[], directModuleRows: any[];
if (isArm64Mac) {
// Sequential: avoid concurrent native DB access
processRows = await processQuery;
moduleRows = await moduleQuery();
directModuleRows = await directModuleQuery();
} else {
// Parallel: safe on non-arm64 platforms
processRows = await processQuery;
[moduleRows, directModuleRows] = await Promise.all([moduleQuery(), directModuleQuery()]);
}
affectedProcesses = processRows.map((r: any) => ({ affectedProcesses = processRows.map((r: any) => ({
name: r.name || r[0], name: r.name || r[0],

38
gitnexus/test/integration/setup-skills.test.ts
View file

@ -9,6 +9,7 @@ describe('setupCommand skills integration', () => {
let tempHome: string; let tempHome: string;
const originalHome = process.env.HOME; const originalHome = process.env.HOME;
const originalUserProfile = process.env.USERPROFILE; const originalUserProfile = process.env.USERPROFILE;
const originalPath = process.env.PATH;
const testId = `${Date.now()}-${process.pid}`; const testId = `${Date.now()}-${process.pid}`;
const flatSkillName = `test-flat-skill-${testId}`; const flatSkillName = `test-flat-skill-${testId}`;
const dirSkillName = `test-dir-skill-${testId}`; const dirSkillName = `test-dir-skill-${testId}`;
@ -47,6 +48,7 @@ describe('setupCommand skills integration', () => {
await fs.rm(path.join(packageSkillsRoot, dirSkillName), { recursive: true, force: true }); await fs.rm(path.join(packageSkillsRoot, dirSkillName), { recursive: true, force: true });
process.env.HOME = originalHome; process.env.HOME = originalHome;
process.env.USERPROFILE = originalUserProfile; process.env.USERPROFILE = originalUserProfile;
process.env.PATH = originalPath;
await fs.rm(tempHome, { recursive: true, force: true }); await fs.rm(tempHome, { recursive: true, force: true });
}); });
@ -85,4 +87,40 @@ describe('setupCommand skills integration', () => {
); );
expect(nestedInstalled).toContain('Directory Nested File'); expect(nestedInstalled).toContain('Directory Nested File');
}); });
it('falls back to Codex config.toml and installs skills into ~/.agents/skills when codex CLI is unavailable', async () => {
await fs.mkdir(path.join(tempHome, '.codex'), { recursive: true });
process.env.PATH = '';
await setupCommand();
const codexConfig = await fs.readFile(
path.join(tempHome, '.codex', 'config.toml'),
'utf-8',
);
expect(codexConfig).toContain('[mcp_servers.gitnexus]');
expect(codexConfig).toContain('gitnexus@latest');
const codexSkill = await fs.readFile(
path.join(tempHome, '.agents', 'skills', 'gitnexus-cli', 'SKILL.md'),
'utf-8',
);
expect(codexSkill).toContain('GitNexus CLI Commands');
});
it('does not duplicate the Codex MCP section on repeated fallback setup runs', async () => {
await fs.mkdir(path.join(tempHome, '.codex'), { recursive: true });
process.env.PATH = '';
await setupCommand();
await setupCommand();
const codexConfig = await fs.readFile(
path.join(tempHome, '.codex', 'config.toml'),
'utf-8',
);
const sectionMatches = codexConfig.match(/\[mcp_servers\.gitnexus\]/g) ?? [];
expect(sectionMatches).toHaveLength(1);
});
}); });

12
gitnexus/test/integration/skills-e2e.test.ts
View file

@ -2389,8 +2389,16 @@ export function createEntry(level: string, msg: string) {
/* CI timeout tolerance */ /* CI timeout tolerance */
if (result1.status === null || result2.status === null) return; if (result1.status === null || result2.status === null) return;
expect(result1.status).toBe(0); expect(result1.status, [
expect(result2.status).toBe(0); `first analyze --skills exited with code ${result1.status}`,
`stdout: ${result1.stdout?.slice(0, 500)}`,
`stderr: ${result1.stderr?.slice(0, 500)}`,
].join('\n')).toBe(0);
expect(result2.status, [
`second analyze --skills exited with code ${result2.status}`,
`stdout: ${result2.stdout?.slice(0, 500)}`,
`stderr: ${result2.stderr?.slice(0, 500)}`,
].join('\n')).toBe(0);
const generatedDir = path.join(tmpDir, '.claude', 'skills', 'generated'); const generatedDir = path.join(tmpDir, '.claude', 'skills', 'generated');
expect(fs.existsSync(generatedDir)).toBe(true); expect(fs.existsSync(generatedDir)).toBe(true);

226
gitnexus/test/integration/staleness-and-stability.test.ts Normal file
View file

@ -0,0 +1,226 @@
/**
* E2E Tests: Stale Data Detection + Sequential Enrichment Stability
*
* Validates the fixes in PR #396:
* 1. Sequential enrichment: impact() enrichment queries run without
* SIGSEGV on arm64 macOS (sequential on arm64, parallel elsewhere)
* 2. Consecutive tool stability: MCP server stays alive after 10+
* consecutive tool calls (no stdout corruption)
* 3. Watchdog guard: activeQueryCount prevents premature stdout restore
* 4. Stale data detection: ensureInitialized() detects meta.json changes
*
* All tests share one withTestLbugDB lifecycle to avoid cross-block
* DB closure issues (LadybugDB's shared global DB in a single fork).
*
* Issues: #285, #290, #292, #297
*/
import { describe, it, expect, afterAll } from 'vitest';
import fs from 'fs/promises';
import path from 'path';
import {
initLbug,
executeQuery,
closeLbug,
} from '../../src/mcp/core/lbug-adapter.js';
import { withTestLbugDB } from '../helpers/test-indexed-db.js';
import { LOCAL_BACKEND_SEED_DATA, LOCAL_BACKEND_FTS_INDEXES } from '../fixtures/local-backend-seed.js';
import { LocalBackend } from '../../src/mcp/local/local-backend.js';
import { listRegisteredRepos } from '../../src/storage/repo-manager.js';
import { vi } from 'vitest';
vi.mock('../../src/storage/repo-manager.js', () => ({
listRegisteredRepos: vi.fn().mockResolvedValue([]),
cleanupOldKuzuFiles: vi.fn().mockResolvedValue({ found: false, needsReindex: false }),
}));
withTestLbugDB('staleness-and-stability', (handle) => {
let backend: LocalBackend;
let storagePath: string;
// ─── Setup ─────────────────────────────────────────────────────────
describe('setup', () => {
it('initialize backend', async () => {
const ext = handle as typeof handle & { _backend?: LocalBackend };
if (!ext._backend) throw new Error('LocalBackend not initialized');
backend = ext._backend;
storagePath = handle.tmpHandle.dbPath;
});
});
// ─── Block 1: Sequential enrichment queries (#285, #290, #292) ─────
describe('impact enrichment queries run without crashes', () => {
it('impact with enrichment completes without SIGSEGV', async () => {
const result = await backend.callTool('impact', {
target: 'validate',
direction: 'upstream',
});
expect(result).not.toHaveProperty('error');
expect(result.impactedCount).toBeGreaterThanOrEqual(1);
expect(result).toHaveProperty('affected_processes');
expect(result).toHaveProperty('affected_modules');
});
it('impact with large maxDepth completes without crash', async () => {
const result = await backend.callTool('impact', {
target: 'login',
direction: 'downstream',
maxDepth: 5,
});
expect(result).toBeDefined();
expect(result).not.toHaveProperty('error');
});
});
// ─── Block 2: Consecutive tool call stability ──────────────────────
describe('MCP server stays alive after 10+ consecutive tool calls', () => {
it('10 consecutive cypher calls complete without stdout corruption', async () => {
for (let i = 0; i < 10; i++) {
const result = await backend.callTool('cypher', {
query: `MATCH (n:Function) RETURN n.name AS name LIMIT ${i + 1}`,
});
expect(result).toHaveProperty('row_count');
expect(result.row_count).toBeGreaterThanOrEqual(1);
}
});
it('mixed tool calls: context → impact → query → cypher cycle', async () => {
for (let i = 0; i < 3; i++) {
const ctx = await backend.callTool('context', { name: 'login' });
expect(ctx.status).toBe('found');
const imp = await backend.callTool('impact', {
target: 'validate',
direction: 'upstream',
});
expect(imp).not.toHaveProperty('error');
const qry = await backend.callTool('query', { query: 'login' });
expect(qry).not.toHaveProperty('error');
const cyp = await backend.callTool('cypher', {
query: 'MATCH (n:Function) RETURN COUNT(n) AS cnt',
});
expect(cyp).toHaveProperty('row_count');
}
});
it('stdout.write is still a function after all calls', () => {
expect(typeof process.stdout.write).toBe('function');
});
});
// ─── Block 3: Watchdog / activeQueryCount ──────────────────────────
describe('watchdog does not restore stdout during active queries', () => {
const REPO = 'watchdog-test';
let poolInited = false;
const ensurePool = async () => {
if (!poolInited) {
await initLbug(REPO, handle.dbPath);
poolInited = true;
}
};
afterAll(async () => {
try { await closeLbug(REPO); } catch { /* best-effort */ }
});
it('parallel queries complete and stdout is restored', async () => {
await ensurePool();
const queries = Array.from({ length: 4 }, (_, i) =>
executeQuery(REPO, `MATCH (n:Function) RETURN n.name AS name LIMIT ${i + 1}`)
);
const results = await Promise.all(queries);
expect(results).toHaveLength(4);
for (const r of results) {
expect(r.length).toBeGreaterThanOrEqual(1);
}
});
it('sequential queries still work', async () => {
await ensurePool();
for (let i = 0; i < 5; i++) {
const rows = await executeQuery(REPO, 'MATCH (n:Function) RETURN n.name');
expect(rows.length).toBeGreaterThanOrEqual(1);
}
});
});
// ─── Block 4: Stale data detection (#297) ──────────────────────────
// LAST: triggers closeLbug internally which may affect shared state
describe('stale data detection via meta.json', () => {
it('initial query works', async () => {
const result = await backend.callTool('cypher', {
query: 'MATCH (n:Function) RETURN n.name AS name ORDER BY n.name',
});
expect(result).toHaveProperty('row_count');
expect(result.row_count).toBeGreaterThanOrEqual(3);
});
it('detects stale index when meta.json indexedAt changes', async () => {
const metaPath = path.join(storagePath, 'meta.json');
await fs.writeFile(metaPath, JSON.stringify({
indexedAt: new Date(Date.now() + 60000).toISOString(),
lastCommit: 'new-commit-hash',
stats: { files: 2, nodes: 3, communities: 1, processes: 1 },
}));
// Next call triggers re-init. May fail but must NOT crash.
try {
const result = await backend.callTool('cypher', {
query: 'MATCH (n:Function) RETURN COUNT(n) AS cnt',
});
expect(result).toBeDefined();
} catch (err: any) {
expect(err.message).not.toMatch(/SIGSEGV/i);
}
});
it('throttle: no re-read within 5s window', async () => {
const metaPath = path.join(storagePath, 'meta.json');
await fs.writeFile(metaPath, JSON.stringify({
indexedAt: new Date(Date.now() + 120000).toISOString(),
lastCommit: 'another-commit',
stats: { files: 2, nodes: 3, communities: 1, processes: 1 },
}));
try {
const result = await backend.callTool('cypher', {
query: 'MATCH (n:Function) RETURN COUNT(n) AS cnt',
});
expect(result).toBeDefined();
} catch {
// No crash = success
}
});
});
}, {
seed: LOCAL_BACKEND_SEED_DATA,
ftsIndexes: LOCAL_BACKEND_FTS_INDEXES,
poolAdapter: true,
afterSetup: async (handle) => {
// Write initial meta.json for staleness tests
const metaPath = path.join(handle.tmpHandle.dbPath, 'meta.json');
const initialMeta = {
indexedAt: new Date().toISOString(),
lastCommit: 'abc123',
stats: { files: 2, nodes: 3, communities: 1, processes: 1 },
};
await fs.writeFile(metaPath, JSON.stringify(initialMeta));
vi.mocked(listRegisteredRepos).mockResolvedValue([
{
name: 'test-repo',
path: '/test/repo',
storagePath: handle.tmpHandle.dbPath,
indexedAt: initialMeta.indexedAt,
lastCommit: 'abc123',
stats: { files: 2, nodes: 3, communities: 1, processes: 1 },
},
]);
const backend = new LocalBackend();
await backend.init();
(handle as any)._backend = backend;
},
});

6
gitnexus/test/unit/schema.test.ts
View file

@ -40,7 +40,7 @@ describe('LadybugDB Schema', () => {
it('has expected total count', () => { it('has expected total count', () => {
// 9 core + 18 multi-language = 27 // 9 core + 18 multi-language = 27
expect(NODE_TABLES).toHaveLength(27); expect(NODE_TABLES).toHaveLength(28);
}); });
}); });
@ -164,7 +164,7 @@ describe('LadybugDB Schema', () => {
describe('schema query ordering', () => { describe('schema query ordering', () => {
it('NODE_SCHEMA_QUERIES has correct count', () => { it('NODE_SCHEMA_QUERIES has correct count', () => {
expect(NODE_SCHEMA_QUERIES).toHaveLength(27); expect(NODE_SCHEMA_QUERIES).toHaveLength(28);
}); });
it('REL_SCHEMA_QUERIES has one relation table', () => { it('REL_SCHEMA_QUERIES has one relation table', () => {
@ -173,7 +173,7 @@ describe('LadybugDB Schema', () => {
it('SCHEMA_QUERIES includes all node + rel + embedding schemas', () => { it('SCHEMA_QUERIES includes all node + rel + embedding schemas', () => {
// 27 node + 1 rel + 1 embedding = 29 // 27 node + 1 rel + 1 embedding = 29
expect(SCHEMA_QUERIES).toHaveLength(29); expect(SCHEMA_QUERIES).toHaveLength(30);
}); });
it('node schemas come before relation schemas in SCHEMA_QUERIES', () => { it('node schemas come before relation schemas in SCHEMA_QUERIES', () => {

103
gitnexus/test/unit/setup-codex.test.ts Normal file
View file

@ -0,0 +1,103 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import fs from 'fs/promises';
import os from 'os';
import path from 'path';
const execFileMock = vi.fn((...args: any[]) => {
const callback = args.at(-1);
if (typeof callback === 'function') {
callback(null, '', '');
}
});
vi.mock('child_process', () => ({
execFile: execFileMock,
}));
describe('setupCommand codex execution', () => {
let tempHome: string;
let originalHome: string | undefined;
let originalUserProfile: string | undefined;
let platformDescriptor: PropertyDescriptor | undefined;
const setPlatform = (value: NodeJS.Platform) => {
Object.defineProperty(process, 'platform', {
value,
configurable: true,
});
};
beforeEach(async () => {
vi.resetModules();
vi.clearAllMocks();
originalHome = process.env.HOME;
originalUserProfile = process.env.USERPROFILE;
tempHome = await fs.mkdtemp(path.join(os.tmpdir(), 'gn-codex-setup-'));
process.env.HOME = tempHome;
process.env.USERPROFILE = tempHome;
await fs.mkdir(path.join(tempHome, '.codex'), { recursive: true });
platformDescriptor = Object.getOwnPropertyDescriptor(process, 'platform');
setPlatform('win32');
vi.spyOn(console, 'log').mockImplementation(() => {});
});
afterEach(async () => {
vi.restoreAllMocks();
if (platformDescriptor) {
Object.defineProperty(process, 'platform', platformDescriptor);
}
process.env.HOME = originalHome;
process.env.USERPROFILE = originalUserProfile;
await fs.rm(tempHome, { recursive: true, force: true });
});
it('invokes codex mcp add with shell enabled on Windows', async () => {
const { setupCommand } = await import('../../src/cli/setup.js');
await setupCommand();
expect(execFileMock).toHaveBeenCalledWith(
'codex',
['mcp', 'add', 'gitnexus', '--', 'cmd', '/c', 'npx', '-y', 'gitnexus@latest', 'mcp'],
{ shell: true },
expect.any(Function),
);
});
it('invokes codex mcp add without shell on non-Windows and does not write fallback config', async () => {
setPlatform('darwin');
const { setupCommand } = await import('../../src/cli/setup.js');
await setupCommand();
expect(execFileMock).toHaveBeenCalledWith(
'codex',
['mcp', 'add', 'gitnexus', '--', 'npx', '-y', 'gitnexus@latest', 'mcp'],
{ shell: false },
expect.any(Function),
);
await expect(
fs.access(path.join(tempHome, '.codex', 'config.toml')),
).rejects.toThrow();
});
it('skips Codex setup entirely when ~/.codex is missing', async () => {
await fs.rm(path.join(tempHome, '.codex'), { recursive: true, force: true });
const { setupCommand } = await import('../../src/cli/setup.js');
await setupCommand();
expect(execFileMock).not.toHaveBeenCalled();
await expect(
fs.access(path.join(tempHome, '.agents', 'skills')),
).rejects.toThrow();
});
});

2
gitnexus/vitest.config.ts
View file

@ -59,6 +59,7 @@ export default defineConfig({
'test/integration/search-core.test.ts', 'test/integration/search-core.test.ts',
'test/integration/search-pool.test.ts', 'test/integration/search-pool.test.ts',
'test/integration/augmentation.test.ts', 'test/integration/augmentation.test.ts',
'test/integration/staleness-and-stability.test.ts',
], ],
fileParallelism: false, fileParallelism: false,
sequence: { groupOrder: 1 }, sequence: { groupOrder: 1 },
@ -79,6 +80,7 @@ export default defineConfig({
'test/integration/search-core.test.ts', 'test/integration/search-core.test.ts',
'test/integration/search-pool.test.ts', 'test/integration/search-pool.test.ts',
'test/integration/augmentation.test.ts', 'test/integration/augmentation.test.ts',
'test/integration/staleness-and-stability.test.ts',
], ],
}, },
}, },