feat: add batch limiting to code indexer to control memory usage
- Add MAX_PENDING_BATCHES constant (20) to limit concurrent batches
- Implement backpressure mechanism to pause file parsing when limit reached
- Prevent memory overflow during large codebase indexing
* feat: add configurable max search results for codebase indexing (#5149)
- Add codebaseIndexSearchMaxResults to configuration schema with validation (10-1000)
- Update Qdrant client to accept maxResults parameter in search method
- Add UI slider in Experimental Settings to configure max search results
- Rename constants to DEFAULT_MAX_SEARCH_RESULTS and DEFAULT_SEARCH_MIN_SCORE for clarity
- Add translations for new setting across all 17 supported languages
- Add comprehensive test coverage for config manager, Qdrant client, and UI components
fix: settings persistence for codebase index configuration
- Add new updateCodebaseIndexConfig message type to properly merge config updates
- Update SettingsView to send entire codebaseIndexConfig object instead of just enabled flag
- Add backend handler to merge configuration updates instead of overwriting
- Add tests for the new message handler functionality
This ensures the max search results setting persists correctly when saved.
* fix: correct property name in updateCodebaseIndexConfig message
The frontend was sending 'config' but the backend expects 'codebaseIndexConfig'.
This mismatch was preventing the max search results setting from persisting.
* feat: refactor codebase index constants and update search result defaults
* feat(chat): add advanced settings for maximum search results configuration
* refactor: remove updateCodebaseIndexConfig and integrate max search results into saveCodeIndexSettingsAtomic
- Removed updateCodebaseIndexConfig message type and handler as per PR feedback
- Added codebaseIndexSearchMaxResults to codeIndexSettings type in WebviewMessage.ts
- Updated saveCodeIndexSettingsAtomic to save codebaseIndexSearchMaxResults
- Fixed SettingsView.tsx to use codebaseIndexEnabled message instead of updateCodebaseIndexConfig
* Delete webview-ui/src/components/settings/__tests__/ExperimentalSettings.spec.tsx
* refactor: remove updateCodebaseIndexConfig tests to streamline codebase indexing logic
* revert this
---------
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
Co-authored-by: Daniel <57051444+daniel-lxs@users.noreply.github.com>
* fix: resolve Go duplicate references in tree-sitter queries (#5367)
- Replace broad statement captures with function-scoped queries
- Eliminates overlapping captures that caused duplicate references
- Improves search quality and indexing performance for Go projects
- Add test to validate no duplicate line ranges are captured
- Maintains backward compatibility with existing functionality
Fixes#5367
* fix: resolve Go codebase indexing with language-specific thresholds
- Update Go tree-sitter queries to capture full declarations instead of just identifiers
- Implement language-specific character thresholds (50 chars for Go vs 100 default)
- Fix inspectGo.spec.ts test to match new query behavior
- Add comprehensive test coverage for Go indexing fix
This ensures Go files are properly indexed for semantic search while preventing
duplicate references. All tests now pass.
* fix: implement universal 50-character threshold for code indexing
- Changed MIN_BLOCK_CHARS from 100 to 50 in parser.ts
- Updated tests to expect single-block captures for small Go files
- Removed language-specific threshold logic
- Fixes Go files not being indexed due to high character threshold
Fixes#5367
* test: replace Go-specific test with generic MIN_BLOCK_CHARS test
- Remove go-indexing-fix.spec.ts as requested in PR feedback
- Add generic test in parser.spec.ts to verify 50-character threshold
- Test ensures content under 50 chars is filtered, 50+ chars is indexed
- Applies to all languages, not just Go
* fix: update markdown tests for MIN_BLOCK_CHARS change from 100 to 50
* feat: apply changes from local main
* fix: add missing types
* feat: deduplicate code blocks coming out of parser
* feat: implement a cache manager to improve cache handling
* refactor: move code index service initialization to extension and remove await from indexing process
* fix: return undefined instead of throwing if no workspace is detected
* feat: allow auto approve if it is active for read tools
* refactor: improve UI of the results and allow opening the ranges directly in the editor
* refactor: use dependency injection to improve performance
* feat: implement result filtering by directory path
* refactor: centralize path normalization logic
* refactor: remove unnecessary barrel file
* refactor: prevent restarting the service if no settings change
* fix: the indexing process should never be awaited
* refactor: cleanup unused method
* refactor: remove batch limits for ollama
* refactor(parser): simplify method signatures and improve chunking logic
- Remove redundant min/max chars parameters
- Add better handling for oversized lines
- Improve chunking logic with segment handling
- Clean up method signatures and parameter ordering
* fix(settings): make select inputs full width in CodeIndexSettings
* refactor: increase max list file limit
* feat(ui): improve codebase search result display formatting
* test: add tests for cache and config managers
* test: create unit tests for parser and scanner
* feat(parser): improve segment hash uniqueness
- Added startCharIndex to segment hash calculation in _chunkTextByLines
- Track character position when splitting oversized lines
- Ensures unique identification of segments from same line
* feat(file-watcher): add error logging and optional ignoreController injection
* fix: allow getting the state if the service is disabled
* fix: set the embedding models when cline provider is initialized
* feat: use zod to validate form
* feat(file-watcher): enhance file watcher for batched deletions and improved vector store interactions
Improve file watcher to handle file deletions in batches and optimize vector store operations.
* feat(CodeIndexSettings): move OpenAI key input to a conditional rendering block
* feat(CodeIndexSettings): update button visibility based on indexing status
* feat(file-watcher): refactor vscode mock and enhance file watcher tests
* fix(CodeIndexManager): do not await startIndexing on configuration changes
* feat(types): add codeIndexOpenAiKey and codeIndexQdrantApiKey to ProviderSettings and IpcMessage
* feat(FileWatcher): enhance file processing with batch operations and new status handling
* fix(webviewMessageHandler): handle errors during CodeIndexManager initialization
* refactor(CodeIndexManager): streamline service creation by consolidating into a single method
* feat(CodeIndex): implement minimum search score configuration and update search methods
* refactor(CodeIndexSettings): replace ApiConfiguration with ProviderSettings and update related methods
* refactor: move contants to centralized file
* refactor(constants): rename CODEBASE_INDEX_SEARCH_MIN_SCORE to SEARCH_MIN_SCORE
* feat(QdrantVectorStore): enhance search functionality with new query structure and indexing
* feat(FileWatcher): implement batch processing and retry logic for upserting points
* fix(CodeIndexSettings): rename setProviderSettingsField to setApiConfigurationField and move model label
* fix(ChatRow): remove limit from search query messages
* refactor(CodebaseSearchResult): remove unused props from component
* feat: implement batch processing for file events in FileWatcher
- Introduced a new mechanism to accumulate file events (create, change, delete) and process them in batches.
- Added debounce functionality to optimize processing frequency.
- Emitted events for batch processing start, progress updates, and completion with detailed summaries.
- Refactored existing processing logic to handle batch deletions and upserts efficiently.
- Enhanced error handling and logging for better traceability during batch operations.
* feat(CodeIndex): implement batch processing and update progress reporting
* fix: define a default url for qdrant
* feat(CodeIndexManager): add initialization check and update startIndexing logic
* feat(CodeIndexSettings): validate Qdrant URL and update settings commitment logic
* feat: refactor progress calculation and update progress bar rendering
* refactor: remove webview provider and related methods
* fix: simplify indexing status update by directly using update values
* feat: integrate .gitignore support into file processing and scanning logic
* fix: update clearCacheFile method to write an empty object instead of deleting the cache file
* Revert this
* Run prettier
* fix: add new dependencies for qdrant client and directory scanner
* feat: add codebase search functionality to localization files
* feat: add localization strings for codebase indexing settings
* feat: integrate CodeIndexSettings into ExperimentalSettings and update settings localization
* refactor: remove console logs from various components for cleaner output
* feat: enhance capabilities section and codebase search tool description
* feat: add code indexing localization for multiple languages
* fix: correct indentation for CodeIndexSettings component in ExperimentalSettings
* refactor: update unit tests to properly test current functionality
* feat: add mock implementation for p-limit and update Jest config
* feat: track file creation, change, and deletion events in accumulatedEvents
* refactor: simplify file watcher tests by removing waitForFileProcessingToFinish and using direct event accumulation
* refactor: mock ContextProxy's getValue method to return current config name in ClineProvider tests
* refactor: mock missing properties required by codebase indexing manager
---------
Co-authored-by: cte <cestreich@gmail.com>
