* feat: add OpenRouter embedding provider support
Implement comprehensive OpenRouter embedding provider support for codebase indexing with the following features:
- New OpenRouterEmbedder class with full API compatibility
- Support for OpenRouter's OpenAI-compatible embedding endpoint
- Rate limiting and retry logic with exponential backoff
- Base64 embedding handling to bypass OpenAI package limitations
- Global rate limit state management across embedder instances
- Configuration updates for API key storage and provider selection
- UI integration for OpenRouter provider settings
- Comprehensive test suite with mocking
- Model dimension support for OpenRouter's embedding models
This adds OpenRouter as the 7th supported embedding provider alongside OpenAI, Ollama, OpenAI-compatible, Gemini, Mistral, and Vercel AI Gateway.
* Add translation key
* Fix mutex double release bug
* Add translations
* Add more translations
* Fix failing tests
* code-index(openrouter): fix HTTP-Referer header to RooCodeInc/Roo-Code; i18n: add and wire OpenRouter Code Index strings; test: assert default headers in embedder
---------
Co-authored-by: daniel-lxs <ricciodaniel98@gmail.com>
* feat: add Mistral embedding provider with OpenAI Compatible Wrapper
- Implement MistralEmbedder class using OpenAI-compatible API
- Add comprehensive unit tests with 100% coverage
- Update type definitions for Mistral provider support
- Integrate Mistral option in UI components and configuration
- Add internationalization support for Mistral provider
- Fix API key storage and retrieval for embedding providers
- Update service factory to support Mistral embeddings
- Add proper error handling and validation
This implementation allows users to use Mistral's embedding models
through the existing OpenAI-compatible wrapper approach, providing
a seamless integration experience.
* feat: add Mistral embedding provider support
- Implement MistralEmbedder class with API integration
- Add Mistral models to embedding model configurations
- Update UI to include Mistral provider option
- Add comprehensive unit tests for Mistral embedder
- Update type definitions and interfaces
- Add internationalization support for Mistral provider
* fix: add missing translations for Mistral embedding provider
* fix: address PR review feedback - improve translations and add clarifying comment
* fix: add embedder validation to prevent misleading status indicators (#4398)
* fix: address PR feedback and fix critical issues
- Fixed settings-save flow to save before validation
- Fixed Error constructor usage in scanner.ts
- Fixed segment identification in file-watcher.ts
- Added missing translation keys for embedder validation errors
* fix: add missing Ollama translation keys
- Added missing ollama.title, description, and settings keys
- Fixed translation check failure in CI/CD pipeline
- Synchronized all 17 non-English locale files
* feat: add proactive embedder validation on provider switch
- Validate embedder connection when switching providers
- Prevent misleading 'Indexed' status when embedder is unavailable
- Show immediate error feedback for invalid configurations
- Add comprehensive test coverage for validation flow
This ensures users get immediate feedback when configuring embedders,
preventing confusion when providers like Ollama are not accessible.
* fix: improve error handling and validation in code indexing process
* refactor: extract common embedder validation and error handling logic
- Created shared/validation-helpers.ts with centralized error handling utilities
- Refactored OpenAI, OpenAI-Compatible, and Ollama embedders to use shared helpers
- Eliminated duplicate error handling code across embedders
- Improved maintainability and consistency of error handling
- Fixed test compatibility in manager.spec.ts
- All 2721 tests passing
* refactor: simplify validation helpers by removing unnecessary wrapper functions
- Removed getErrorMessageForConnectionError and inlined logic into handleValidationError
- Removed isRateLimitError, logRateLimitRetry, and logEmbeddingError wrapper functions
- Updated openai.ts and openai-compatible.ts to inline rate limit checking and logging
- Reduced code complexity while maintaining all functionality
- All 311 tests continue to pass
* fix: add missing invalidResponse i18n key and fix French translation
- Added missing 'invalidResponse' key to all locale files
- Fixed French translation: changed 'and accessible' to 'et accessible'
- Ensures proper error messages are displayed when embedder returns invalid responses
* fix: restore removed score settings in webviewMessageHandler
- Restored codebaseIndexSearchMaxResults and codebaseIndexSearchMinScore settings that were unintentionally removed
- Keep embedder validation related changes
* fix: revert unintended changes to file-watcher and scanner
- Reverted point ID generation back to using line numbers instead of segmentHash
- Restored { cause: deleteError } parameter in scanner error handling
- These changes were unrelated to the embedder validation feature
---------
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
* feat: add configurable max search results for codebase indexing (#5149)
- Add codebaseIndexSearchMaxResults to configuration schema with validation (10-1000)
- Update Qdrant client to accept maxResults parameter in search method
- Add UI slider in Experimental Settings to configure max search results
- Rename constants to DEFAULT_MAX_SEARCH_RESULTS and DEFAULT_SEARCH_MIN_SCORE for clarity
- Add translations for new setting across all 17 supported languages
- Add comprehensive test coverage for config manager, Qdrant client, and UI components
fix: settings persistence for codebase index configuration
- Add new updateCodebaseIndexConfig message type to properly merge config updates
- Update SettingsView to send entire codebaseIndexConfig object instead of just enabled flag
- Add backend handler to merge configuration updates instead of overwriting
- Add tests for the new message handler functionality
This ensures the max search results setting persists correctly when saved.
* fix: correct property name in updateCodebaseIndexConfig message
The frontend was sending 'config' but the backend expects 'codebaseIndexConfig'.
This mismatch was preventing the max search results setting from persisting.
* feat: refactor codebase index constants and update search result defaults
* feat(chat): add advanced settings for maximum search results configuration
* refactor: remove updateCodebaseIndexConfig and integrate max search results into saveCodeIndexSettingsAtomic
- Removed updateCodebaseIndexConfig message type and handler as per PR feedback
- Added codebaseIndexSearchMaxResults to codeIndexSettings type in WebviewMessage.ts
- Updated saveCodeIndexSettingsAtomic to save codebaseIndexSearchMaxResults
- Fixed SettingsView.tsx to use codebaseIndexEnabled message instead of updateCodebaseIndexConfig
* Delete webview-ui/src/components/settings/__tests__/ExperimentalSettings.spec.tsx
* refactor: remove updateCodebaseIndexConfig tests to streamline codebase indexing logic
* revert this
---------
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
Co-authored-by: Daniel <57051444+daniel-lxs@users.noreply.github.com>
* feat: Add OpenAI Compatible embedder for codebase indexing
- Implement OpenAiCompatibleEmbedder with batching and retry logic
- Add configuration support for base URL and API key
- Update UI with provider selection and input fields
- Add comprehensive test coverage
- Support for all OpenAI-compatible endpoints (LiteLLM, LMStudio, Ollama, etc.)
- Add internationalization for 17 languages
* fix: Update CodeIndexSettings tests for OpenAI Compatible provider
- Fix field count expectations (4 fields including Qdrant)
- Use specific test IDs for button selection
- Fix input handling with clear() before type()
- Use toHaveBeenLastCalledWith for better assertions
- Fix status text matching with regex pattern
* fix: resolve UI test failures and ESLint errors
- Remove unused waitFor import to fix ESLint error
- Fix test expectations to match actual component behavior for input fields
- Simplify provider selection test by removing complex mock interactions
- All CodeIndexSettings tests now pass (20/20)
* feat: add custom model infrastructure for OpenAI-compatible embedder
- Add manual model ID and embedding dimension configuration
- Enable custom model input via text field in settings UI
- Add modelDimension parameter to OpenAiCompatibleEmbedder
- Update configuration management to persist dimension setting
- Prioritize manual dimension over hardcoded model profiles
- Add comprehensive test coverage for new functionality
This allows users to specify any custom embedding model and its
dimension for OpenAI-compatible providers, removing dependency
on hardcoded model profiles.
* Add missing translations for OpenAI-compatible model dimension settings in all locales
* refactor: remove unused modelDimension parameter from OpenAiCompatibleEmbedder
- Remove modelDimension property and constructor parameter from OpenAiCompatibleEmbedder class
- Update ServiceFactory to not pass dimension to embedder constructor
- Update tests to match new constructor signature
- The dimension is still used for QdrantVectorStore configuration
* chore: bot suggestion
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* chore: bot suggestion
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* refactor: rename OpenAiCompatibleEmbedder to OpenAICompatibleEmbedder for consistency
* feat: add model dimension validation for OpenAI-compatible settings
* refactor: improve default model ID retrieval logic for embedding providers
* feat: add default model ID retrieval for openai-compatible provider
* refactor: update default model ID retrieval to use shared utility function
* fix: Remove unnecessary type assertion in OpenAICompatibleEmbedder
* feat: add model dimension input for openai-compatible provider
---------
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
Co-authored-by: Daniel <57051444+daniel-lxs@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* feat: apply changes from local main
* fix: add missing types
* feat: deduplicate code blocks coming out of parser
* feat: implement a cache manager to improve cache handling
* refactor: move code index service initialization to extension and remove await from indexing process
* fix: return undefined instead of throwing if no workspace is detected
* feat: allow auto approve if it is active for read tools
* refactor: improve UI of the results and allow opening the ranges directly in the editor
* refactor: use dependency injection to improve performance
* feat: implement result filtering by directory path
* refactor: centralize path normalization logic
* refactor: remove unnecessary barrel file
* refactor: prevent restarting the service if no settings change
* fix: the indexing process should never be awaited
* refactor: cleanup unused method
* refactor: remove batch limits for ollama
* refactor(parser): simplify method signatures and improve chunking logic
- Remove redundant min/max chars parameters
- Add better handling for oversized lines
- Improve chunking logic with segment handling
- Clean up method signatures and parameter ordering
* fix(settings): make select inputs full width in CodeIndexSettings
* refactor: increase max list file limit
* feat(ui): improve codebase search result display formatting
* test: add tests for cache and config managers
* test: create unit tests for parser and scanner
* feat(parser): improve segment hash uniqueness
- Added startCharIndex to segment hash calculation in _chunkTextByLines
- Track character position when splitting oversized lines
- Ensures unique identification of segments from same line
* feat(file-watcher): add error logging and optional ignoreController injection
* fix: allow getting the state if the service is disabled
* fix: set the embedding models when cline provider is initialized
* feat: use zod to validate form
* feat(file-watcher): enhance file watcher for batched deletions and improved vector store interactions
Improve file watcher to handle file deletions in batches and optimize vector store operations.
* feat(CodeIndexSettings): move OpenAI key input to a conditional rendering block
* feat(CodeIndexSettings): update button visibility based on indexing status
* feat(file-watcher): refactor vscode mock and enhance file watcher tests
* fix(CodeIndexManager): do not await startIndexing on configuration changes
* feat(types): add codeIndexOpenAiKey and codeIndexQdrantApiKey to ProviderSettings and IpcMessage
* feat(FileWatcher): enhance file processing with batch operations and new status handling
* fix(webviewMessageHandler): handle errors during CodeIndexManager initialization
* refactor(CodeIndexManager): streamline service creation by consolidating into a single method
* feat(CodeIndex): implement minimum search score configuration and update search methods
* refactor(CodeIndexSettings): replace ApiConfiguration with ProviderSettings and update related methods
* refactor: move contants to centralized file
* refactor(constants): rename CODEBASE_INDEX_SEARCH_MIN_SCORE to SEARCH_MIN_SCORE
* feat(QdrantVectorStore): enhance search functionality with new query structure and indexing
* feat(FileWatcher): implement batch processing and retry logic for upserting points
* fix(CodeIndexSettings): rename setProviderSettingsField to setApiConfigurationField and move model label
* fix(ChatRow): remove limit from search query messages
* refactor(CodebaseSearchResult): remove unused props from component
* feat: implement batch processing for file events in FileWatcher
- Introduced a new mechanism to accumulate file events (create, change, delete) and process them in batches.
- Added debounce functionality to optimize processing frequency.
- Emitted events for batch processing start, progress updates, and completion with detailed summaries.
- Refactored existing processing logic to handle batch deletions and upserts efficiently.
- Enhanced error handling and logging for better traceability during batch operations.
* feat(CodeIndex): implement batch processing and update progress reporting
* fix: define a default url for qdrant
* feat(CodeIndexManager): add initialization check and update startIndexing logic
* feat(CodeIndexSettings): validate Qdrant URL and update settings commitment logic
* feat: refactor progress calculation and update progress bar rendering
* refactor: remove webview provider and related methods
* fix: simplify indexing status update by directly using update values
* feat: integrate .gitignore support into file processing and scanning logic
* fix: update clearCacheFile method to write an empty object instead of deleting the cache file
* Revert this
* Run prettier
* fix: add new dependencies for qdrant client and directory scanner
* feat: add codebase search functionality to localization files
* feat: add localization strings for codebase indexing settings
* feat: integrate CodeIndexSettings into ExperimentalSettings and update settings localization
* refactor: remove console logs from various components for cleaner output
* feat: enhance capabilities section and codebase search tool description
* feat: add code indexing localization for multiple languages
* fix: correct indentation for CodeIndexSettings component in ExperimentalSettings
* refactor: update unit tests to properly test current functionality
* feat: add mock implementation for p-limit and update Jest config
* feat: track file creation, change, and deletion events in accumulatedEvents
* refactor: simplify file watcher tests by removing waitForFileProcessingToFinish and using direct event accumulation
* refactor: mock ContextProxy's getValue method to return current config name in ClineProvider tests
* refactor: mock missing properties required by codebase indexing manager
---------
Co-authored-by: cte <cestreich@gmail.com>
