Roo-Code/src/core/prompts/tools/index.ts
Daniel 61e122dc34
Codebase indexing (Clean history) (#3137)
* feat: apply changes from local main

* fix: add missing types

* feat: deduplicate code blocks coming out of parser

* feat: implement a cache manager to improve cache handling

* refactor: move code index service initialization to extension and remove await from indexing process

* fix: return undefined instead of throwing if no workspace is detected

* feat: allow auto approve if it is active for read tools

* refactor: improve UI of the results and allow opening the ranges directly in the editor

* refactor: use dependency injection to improve performance

* feat: implement result filtering by directory path

* refactor: centralize path normalization logic

* refactor: remove unnecessary barrel file

* refactor: prevent restarting the service if no settings change

* fix: the indexing process should never be awaited

* refactor: cleanup unused method

* refactor: remove batch limits for ollama

* refactor(parser): simplify method signatures and improve chunking logic

- Remove redundant min/max chars parameters
- Add better handling for oversized lines
- Improve chunking logic with segment handling
- Clean up method signatures and parameter ordering

* fix(settings): make select inputs full width in CodeIndexSettings

* refactor: increase max list file limit

* feat(ui): improve codebase search result display formatting

* test: add tests for cache and config managers

* test: create unit tests for parser and scanner

* feat(parser): improve segment hash uniqueness

- Added startCharIndex to segment hash calculation in _chunkTextByLines
- Track character position when splitting oversized lines
- Ensures unique identification of segments from same line

* feat(file-watcher): add error logging and optional ignoreController injection

* fix: allow getting the state if the service is disabled

* fix: set the embedding models when cline provider is initialized

* feat: use zod to validate form

* feat(file-watcher): enhance file watcher for batched deletions and improved vector store interactions

Improve file watcher to handle file deletions in batches and optimize vector store operations.

* feat(CodeIndexSettings): move OpenAI key input to a conditional rendering block

* feat(CodeIndexSettings): update button visibility based on indexing status

* feat(file-watcher): refactor vscode mock and enhance file watcher tests

* fix(CodeIndexManager): do not await startIndexing on configuration changes

* feat(types): add codeIndexOpenAiKey and codeIndexQdrantApiKey to ProviderSettings and IpcMessage

* feat(FileWatcher): enhance file processing with batch operations and new status handling

* fix(webviewMessageHandler): handle errors during CodeIndexManager initialization

* refactor(CodeIndexManager): streamline service creation by consolidating into a single method

* feat(CodeIndex): implement minimum search score configuration and update search methods

* refactor(CodeIndexSettings): replace ApiConfiguration with ProviderSettings and update related methods

* refactor: move contants to centralized file

* refactor(constants): rename CODEBASE_INDEX_SEARCH_MIN_SCORE to SEARCH_MIN_SCORE

* feat(QdrantVectorStore): enhance search functionality with new query structure and indexing

* feat(FileWatcher): implement batch processing and retry logic for upserting points

* fix(CodeIndexSettings): rename setProviderSettingsField to setApiConfigurationField and move model label

* fix(ChatRow): remove limit from search query messages

* refactor(CodebaseSearchResult): remove unused props from component

* feat: implement batch processing for file events in FileWatcher

- Introduced a new mechanism to accumulate file events (create, change, delete) and process them in batches.
- Added debounce functionality to optimize processing frequency.
- Emitted events for batch processing start, progress updates, and completion with detailed summaries.
- Refactored existing processing logic to handle batch deletions and upserts efficiently.
- Enhanced error handling and logging for better traceability during batch operations.

* feat(CodeIndex): implement batch processing and update progress reporting

* fix: define a default url for qdrant

* feat(CodeIndexManager): add initialization check and update startIndexing logic

* feat(CodeIndexSettings): validate Qdrant URL and update settings commitment logic

* feat: refactor progress calculation and update progress bar rendering

* refactor: remove webview provider and related methods

* fix: simplify indexing status update by directly using update values

* feat: integrate .gitignore support into file processing and scanning logic

* fix: update clearCacheFile method to write an empty object instead of deleting the cache file

* Revert this

* Run prettier

* fix: add new dependencies for qdrant client and directory scanner

* feat: add codebase search functionality to localization files

* feat: add localization strings for codebase indexing settings

* feat: integrate CodeIndexSettings into ExperimentalSettings and update settings localization

* refactor: remove console logs from various components for cleaner output

* feat: enhance capabilities section and codebase search tool description

* feat: add code indexing localization for multiple languages

* fix: correct indentation for CodeIndexSettings component in ExperimentalSettings

* refactor: update unit tests to properly test current functionality

* feat: add mock implementation for p-limit and update Jest config

* feat: track file creation, change, and deletion events in accumulatedEvents

* refactor: simplify file watcher tests by removing waitForFileProcessingToFinish and using direct event accumulation

* refactor: mock ContextProxy's getValue method to return current config name in ClineProvider tests

* refactor: mock missing properties required by codebase indexing manager

---------

Co-authored-by: cte <cestreich@gmail.com>
2025-05-23 19:07:19 -06:00

138 lines
4.9 KiB
TypeScript

import { ToolName } from "../../../schemas"
import { TOOL_GROUPS, ALWAYS_AVAILABLE_TOOLS, DiffStrategy } from "../../../shared/tools"
import { McpHub } from "../../../services/mcp/McpHub"
import { Mode, ModeConfig, getModeConfig, isToolAllowedForMode, getGroupName } from "../../../shared/modes"
import { ToolArgs } from "./types"
import { getExecuteCommandDescription } from "./execute-command"
import { getReadFileDescription } from "./read-file"
import { getFetchInstructionsDescription } from "./fetch-instructions"
import { getWriteToFileDescription } from "./write-to-file"
import { getSearchFilesDescription } from "./search-files"
import { getListFilesDescription } from "./list-files"
import { getInsertContentDescription } from "./insert-content"
import { getSearchAndReplaceDescription } from "./search-and-replace"
import { getListCodeDefinitionNamesDescription } from "./list-code-definition-names"
import { getBrowserActionDescription } from "./browser-action"
import { getAskFollowupQuestionDescription } from "./ask-followup-question"
import { getAttemptCompletionDescription } from "./attempt-completion"
import { getUseMcpToolDescription } from "./use-mcp-tool"
import { getAccessMcpResourceDescription } from "./access-mcp-resource"
import { getSwitchModeDescription } from "./switch-mode"
import { getNewTaskDescription } from "./new-task"
import { getCodebaseSearchDescription } from "./codebase-search"
import { CodeIndexManager } from "../../../services/code-index/manager"
// Map of tool names to their description functions
const toolDescriptionMap: Record<string, (args: ToolArgs) => string | undefined> = {
execute_command: (args) => getExecuteCommandDescription(args),
read_file: (args) => getReadFileDescription(args),
fetch_instructions: () => getFetchInstructionsDescription(),
write_to_file: (args) => getWriteToFileDescription(args),
search_files: (args) => getSearchFilesDescription(args),
list_files: (args) => getListFilesDescription(args),
list_code_definition_names: (args) => getListCodeDefinitionNamesDescription(args),
browser_action: (args) => getBrowserActionDescription(args),
ask_followup_question: () => getAskFollowupQuestionDescription(),
attempt_completion: () => getAttemptCompletionDescription(),
use_mcp_tool: (args) => getUseMcpToolDescription(args),
access_mcp_resource: (args) => getAccessMcpResourceDescription(args),
codebase_search: () => getCodebaseSearchDescription(),
switch_mode: () => getSwitchModeDescription(),
new_task: (args) => getNewTaskDescription(args),
insert_content: (args) => getInsertContentDescription(args),
search_and_replace: (args) => getSearchAndReplaceDescription(args),
apply_diff: (args) =>
args.diffStrategy ? args.diffStrategy.getToolDescription({ cwd: args.cwd, toolOptions: args.toolOptions }) : "",
}
export function getToolDescriptionsForMode(
mode: Mode,
cwd: string,
supportsComputerUse: boolean,
codeIndexManager?: CodeIndexManager,
diffStrategy?: DiffStrategy,
browserViewportSize?: string,
mcpHub?: McpHub,
customModes?: ModeConfig[],
experiments?: Record<string, boolean>,
): string {
const config = getModeConfig(mode, customModes)
const args: ToolArgs = {
cwd,
supportsComputerUse,
diffStrategy,
browserViewportSize,
mcpHub,
}
const tools = new Set<string>()
// Add tools from mode's groups
config.groups.forEach((groupEntry) => {
const groupName = getGroupName(groupEntry)
const toolGroup = TOOL_GROUPS[groupName]
if (toolGroup) {
toolGroup.tools.forEach((tool) => {
if (
isToolAllowedForMode(
tool as ToolName,
mode,
customModes ?? [],
undefined,
undefined,
experiments ?? {},
)
) {
tools.add(tool)
}
})
}
})
// Add always available tools
ALWAYS_AVAILABLE_TOOLS.forEach((tool) => tools.add(tool))
// Conditionally exclude codebase_search if feature is disabled or not configured
if (
!codeIndexManager ||
!(codeIndexManager.isFeatureEnabled && codeIndexManager.isFeatureConfigured && codeIndexManager.isInitialized)
) {
tools.delete("codebase_search")
}
// Map tool descriptions for allowed tools
const descriptions = Array.from(tools).map((toolName) => {
const descriptionFn = toolDescriptionMap[toolName]
if (!descriptionFn) {
return undefined
}
return descriptionFn({
...args,
toolOptions: undefined, // No tool options in group-based approach
})
})
return `# Tools\n\n${descriptions.filter(Boolean).join("\n\n")}`
}
// Export individual description functions for backward compatibility
export {
getExecuteCommandDescription,
getReadFileDescription,
getFetchInstructionsDescription,
getWriteToFileDescription,
getSearchFilesDescription,
getListFilesDescription,
getListCodeDefinitionNamesDescription,
getBrowserActionDescription,
getAskFollowupQuestionDescription,
getAttemptCompletionDescription,
getUseMcpToolDescription,
getAccessMcpResourceDescription,
getSwitchModeDescription,
getInsertContentDescription,
getSearchAndReplaceDescription,
getCodebaseSearchDescription,
}