* feat: add prompt caching support for LiteLLM (#5791)
- Add litellmUsePromptCache configuration option to provider settings
- Implement cache control headers in LiteLLM handler when enabled
- Add UI checkbox for enabling prompt caching (only shown for supported models)
- Track cache read/write tokens in usage data
- Add comprehensive test for prompt caching functionality
- Reuse existing translation keys for consistency across languages
This allows LiteLLM users to benefit from prompt caching with supported models
like Claude 3.7, reducing costs and improving response times.
* fix: improve LiteLLM prompt caching to work for multi-turn conversations
- Convert system message to structured format with cache_control
- Handle both string and array content types for user messages
- Apply cache_control to content items, not just message level
- Update tests to match new message structure
This ensures prompt caching works correctly for all messages in a conversation,
not just the initial system prompt and first user message.
* fix: resolve TypeScript linter error for cache_control property
Use type assertion to handle cache_control property that's not in OpenAI types
* feat: Adding more settings and control over Gemini
- with topP, topK, maxOutputTokens
- allow users to enable URL context and Grounding Research
* feat: Adding parameter titles and descriptions + translation to all languages
* feat: adding more translations
* feat: adding `contextLimit` implementation from `maxContextWindow` PR + working with profile-specific thresholding
* feat: max value for context limit to model's limit + converting description and titles to settings for translation purposes
* feat: all languages translated
* feat: changing profile-specific threshold in context management setting will also change in Gemini context management
- sync between Context Management Settting <-> Gemini Context Management with regards to thresholding
* feat: max value of maxOutputTokens is model's maxTokens + adding more tests
* feat: improve unit tests and adding `data-testid` to slider and checkbox components
* fix: small changes in geminiContextManagement descriptions + minor fix
* fix: Switching from "Gemini Context Management" to "Token Management
- better naming and correct purpose
* fix: input field showed NaN -> annoying UX
* fix: Removing redundant "tokens" after the "set context limit"'s checkbox + removing the lengthy description
* fix: Changing the translation to be consistent with the english one
* fix: more translations
* fix: translations
* fix: removing contextLimit and token management related code
- due to the decision in: https://github.com/RooCodeInc/Roo-Code/issues/3717
* fix: removing `contextLimit` test and removing token management in translations
* fix: changing from `Advanced Features` to `Tools` to be consistent with Gemini docs/AI studio
* fix: adding `try-catch` block for `generateContentStream`
* feat: Include citations + improved type safety
* feat: adding citation for streams (generateContextStream)
* fix: set default values for `topP`, `topK` and `maxOutputTokens`
* fix: changing UI/UX according to the review/feedback from `daniel-lxs`
* fix: updating the `Gemini.spec.tsx` unit test
- testing when it is hidden
- testing when users click on the collapsible trigger and model configuration appears
* fix: more changes from the feedback/review from `daniel-lxs`
* fix: adding sources at the end of the stream to preserve
* fix: change the description for grounding with google search and url context
* fix: adding translations
* fix: removing redundant extra translations - a mistake made by the agent
* fix: remove duplicate translation keys in geminiSections and geminiParameters
- Fixed duplicate keys in 13 localization files (es, fr, hi, id, it, ja, ko, nl, pl, pt-BR, ru, tr, vi)
- Removed second occurrence of geminiSections and geminiParameters keys
- Kept first occurrence which contains more comprehensive descriptions
- All JSON files validated for syntax correctness
- Translation completeness verified with missing translations script
Resolves duplicate key issue identified in PR #4895
* fix: delete topK, topP and maxOutputTokens from Gemini
* fix: deleting topK, topP and maxOutputTokens from translations/locales
* fix: adjust spacing between labels and descriptions + sentence casing
* fix: adding maxOutputTokens back and removing unknown type
* fix: internalizing error Gemini error message
* fix: updating tests in Gemini and Vertex to adjust to the new error logging
* fix: address PR review feedback for Gemini tools feature
- Fix Hindi translation grammatical error in settings.json
- Internationalize 'Sources:' string and error messages in gemini.ts
- Add comprehensive error scenario tests to gemini-handler.spec.ts
- Remove unused currentModelId prop from Gemini component
- Update all locale files with new translation keys
---------
Co-authored-by: Roo Code <roomote@roocode.com>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
refactor: move HuggingFace models API to providers/fetchers
- Moved getHuggingFaceModels functionality from src/api/huggingface-models.ts to src/api/providers/fetchers/huggingface.ts
- Added getHuggingFaceModelsWithMetadata function to maintain the same API interface
- Updated import in webviewMessageHandler.ts to use the new location
- Deleted the now redundant src/api/huggingface-models.ts file
This consolidates all HuggingFace-related API logic into a single location within the providers/fetchers directory structure.
* add more details
* format details better
* fix tests
* fix: address PR #6190 review feedback
- Move huggingface-models.ts to src/api/providers/fetchers/huggingface.ts
- Remove 'any' types and add proper TypeScript interfaces
- Add missing i18n keys and translations for all languages
- Replace magic numbers with named constants
- Add JSDoc documentation for HuggingFaceModel interface
- Improve error handling in API endpoint
- Update model capabilities display to match other providers
- Remove tool calling display (not used)
- Add comprehensive test coverage for new UI features
* fix: preserve HuggingFace provider details in model response
- Store raw HuggingFace models in cache to preserve provider information
- Export getCachedRawHuggingFaceModels to retrieve full model data
- Update huggingface-models.ts to return cached raw models when available
- Include provider name in model descriptions
- Always add provider-specific variants to show all available providers
- Remove console.log statements from fetcher
---------
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
* basic hugging face provider
* fetch hf models and providers
* save provider to config
* Update translations
---------
Co-authored-by: Thomas G. Lopes <26071571+TGlide@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Daniel <57051444+daniel-lxs@users.noreply.github.com>
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
- Update parseOpenRouterModel to always use actual max_completion_tokens from OpenRouter API
- Remove artificial restriction that only reasoning budget and Anthropic models get their actual max tokens
- Fall back to 20% of context window when max_completion_tokens is null
- Update getModelMaxOutputTokens to use same fallback logic for consistency
- Update tests to reflect new behavior
- Fixes issue where reserved tokens showed ~209k instead of actual model limits (e.g. GPT-4o: 16,384)
* chore: adding x-title header and testing for litellm
* chore: indentation fi and headers order fix
* chore: spacing fix
* chore: removed white space
* fix: allow user headers to override default headers and clean up formatting
- Reorder header spread in router-provider.ts so user-provided openAiHeaders can override DEFAULT_HEADERS
- Remove unnecessary blank lines after imports for consistency
- This matches the pattern used in openai.ts where DEFAULT_HEADERS come first
---------
Co-authored-by: Brendan-Z <brendanzhou.99@gmail.com>
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
Co-authored-by: Daniel <57051444+daniel-lxs@users.noreply.github.com>
* fix: resolve Claude Code token counting inefficiency and enable caching (#5104)
- Remove 1.5x fudge factor from Claude Code token counting
- Enable prompt caching support for all Claude Code models
- Add comprehensive tests for token counting and caching
- Update existing tests to reflect accurate token counting
This fixes the extreme token inefficiency where simple messages would
jump from ~40k to over 60k tokens, causing API hangs when approaching
the artificial 120k limit. Claude Code now properly utilizes its full
200k context window with accurate token counting.
* fix: address PR review comments
- Extract IMAGE_TOKEN_ESTIMATE as a named constant for clarity
- Update token counting tests to use exact counts instead of ranges for deterministic testing
- Fix test expectations to match actual tokenizer output
* Remove token counting changes, keep only cache support
- Removed custom countTokens override from claude-code.ts
- Deleted claude-code-token-counting.spec.ts test file
- Kept cache token collection and reporting functionality
- Kept supportsPromptCache: true for all Claude Code models
- Kept claude-code-caching.spec.ts tests
This focuses the PR on enabling cache support without modifying token counting behavior.
* fix: update webview test to expect supportsPromptCache=true for Claude Code models
---------
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
* Fix temperature parameter error for Azure OpenAI reasoning models
* Fix tests: Update O3 family model tests to expect temperature: undefined
- Updated failing tests in openai.spec.ts to expect temperature: undefined for O3 models
- This aligns with the PR changes that remove temperature parameter for Azure OpenAI o1, o3, and o4 models
- All 4 previously failing tests now pass
---------
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Daniel <57051444+daniel-lxs@users.noreply.github.com>
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
Co-authored-by: Matt Rubens <mrubens@users.noreply.github.com>
fix: handle null families field in Ollama model details schema
- Updated OllamaModelDetailsSchema to make families field nullable and optional
- Made all unused properties optional in Ollama schemas to prevent validation errors
- Added test cases to verify handling of null families field
- Only required properties that are actually used in the code are now mandatory
- Fixes Zod validation error when Ollama returns null for families array
* Add reasoning budget support to Bedrock models and update related components
- Introduced `supportsReasoningBudget` property in Bedrock models.
- Enhanced `AwsBedrockHandler` to handle reasoning budget in payloads.
- Updated `ThinkingBudget` component to dynamically set max tokens based on reasoning support.
- Modified `ApiOptions` and `Bedrock` components to conditionally render `ThinkingBudget`.
- Added tests for extended thinking functionality in `bedrock-reasoning.test.ts`.
* Add BedrockThinkingConfig interface and update payload structure
* fix: address PR review feedback (#4481)
- Simplify ThinkingBudget ternary logic since component only renders when reasoning budget supported
- Break down complex thinking enabled condition with clear documentation
- Replace 'as any' usage with proper TypeScript interfaces for AWS SDK events
- Add comprehensive documentation for multiple stream structures explaining AWS SDK compatibility
* feat: show ThinkingBudget component unconditionally
Remove selectedProviderModels.length check to display ThinkingBudget
for all providers, not just those with available models
---------
Co-authored-by: hannesrudolph <hrudolph@gmail.com>
* Add reasoning budget support to Bedrock models and update related components
- Introduced `supportsReasoningBudget` property in Bedrock models.
- Enhanced `AwsBedrockHandler` to handle reasoning budget in payloads.
- Updated `ThinkingBudget` component to dynamically set max tokens based on reasoning support.
- Modified `ApiOptions` and `Bedrock` components to conditionally render `ThinkingBudget`.
- Added tests for extended thinking functionality in `bedrock-reasoning.test.ts`.
* Add BedrockThinkingConfig interface and update payload structure
* fix: address PR review feedback (#4481)
- Simplify ThinkingBudget ternary logic since component only renders when reasoning budget supported
- Break down complex thinking enabled condition with clear documentation
- Replace 'as any' usage with proper TypeScript interfaces for AWS SDK events
- Add comprehensive documentation for multiple stream structures explaining AWS SDK compatibility
* feat: Add DeepSeek R1 support to Chutes provider (#4523)
- Modified BaseOpenAiCompatibleProvider to expose client as protected
- Enhanced ChutesHandler to detect DeepSeek R1 models and parse reasoning chunks
- Applied R1 format conversion for message formatting
- Set appropriate temperature (0.6) for DeepSeek models
- Migrated tests from Jest to Vitest format
- Added comprehensive tests for DeepSeek R1 functionality
This ensures reasoning chunks are properly separated from regular content
when using DeepSeek R1 models via Chutes provider.
* feat: Enhance DeepSeek R1 support with <think> tag handling in Chutes provider
* fix: Correct temperature retrieval in ChutesHandler to use model's info
* fix: Update condition for DeepSeek-R1 model identification in createMessage method
---------
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
Currently, when you use OpenRouter with your own key for the underlying service, the costs shown by Roo Code are way off what it actually costs.
With bring your own key, OpenRouter charges 5% of what it normally would (see https://openrouter.ai/docs/use-cases/byok)
so we have to multiply the reported cost by 20 to get an estimate of what it actually costs.
Co-authored-by: Johan Otten <drakonen@drakonen.com>
Co-authored-by: Eamon Nerbonne <eamon@nerbonne.org>
* fix: remove explicit cache_control for Google models in OpenRouter (#4487)
- Remove all Google models from OPEN_ROUTER_PROMPT_CACHING_MODELS set
- This resolves 3+ minute lag when using google/gemini-2.5-pro-preview
- OpenRouter still provides automatic implicit ephemeral caching for these models
- Updated tests to handle intentional exclusion of Google models from explicit caching
Fixes#4487
* refactor: simplify OpenRouter caching test logic
- Replace hardcoded exclusion list with simple Google model filter
- Keep original validation logic but make it more maintainable
- Still ensures all our caching models are supported by OpenRouter
- Still verifies we exclude all Google models from explicit caching
* cleanup: remove unused excludedModels variable
- Variable was defined but never used
- Keeps the test logic clean and focused
* refactor: only exclude google/gemini-2.5-pro-preview from caching
- More surgical approach - only exclude the specific problematic model
- Keep other Google models in caching (they work fine)
- Add comment explaining the exclusion with issue reference
- Update test to only exclude the specific model
This targets just the model causing 3+ minute lag while preserving
caching benefits for other Google models that work properly.
* feat: Add safeWriteJson utility for atomic file operations
Implements a robust JSON file writing utility that:
- Prevents concurrent writes to the same file using in-memory locks
- Ensures atomic operations with temporary file and backup strategies
- Handles error cases with proper rollback mechanisms
- Cleans up temporary files even when operations fail
- Provides comprehensive test coverage for success and failure scenarios
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: use safeWriteJson for all JSON file writes
This change refactors all direct JSON file writes to use the safeWriteJson
utility, which implements atomic file writes to prevent data corruption
during write operations.
- Modified safeWriteJson to accept optional replacer and space arguments
- Updated tests to verify correct behavior with the new implementation
Fixes: #722
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: Implement inter-process file locking for safeWriteJson
Replaces the previous in-memory lock in `safeWriteJson` with
`proper-lockfile` to provide robust, cross-process advisory file
locking. This enhances safety when multiple processes might attempt
concurrent writes to the same JSON file.
Key changes:
- Added `proper-lockfile` and `@types/proper-lockfile` dependencies.
- `safeWriteJson` now uses `proper-lockfile.lock()` with configured
retries, staleness checks (31s), and lock update intervals (10s).
- An `onCompromised` handler is included to manage scenarios where
the lock state is unexpectedly altered.
- Logging and comments within `safeWriteJson` have been refined for
clarity, ensuring error logs include backtraces.
- The test suite `safeWriteJson.test.ts` has been significantly
updated to:
- Use real timers (`jest.useRealTimers()`).
- Employ a more comprehensive mock for `fs/promises`.
- Correctly manage file pre-existence for various scenarios.
- Simulate lock contention by mocking `proper-lockfile.lock()`
using `jest.doMock` and a dynamic require for the SUT.
- Verify lock release by checking for the absence of the `.lock`
file.
All tests are passing with these changes.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* feat: implement streaming JSON write in safeWriteJson
Refactor safeWriteJson to use stream-json for memory-efficient JSON serialization:
- Replace in-memory string creation with streaming pipeline
- Add Disassembler and Stringer from stream-json library
- Extract streaming logic to a dedicated helper function
- Add proper-lockfile and stream-json dependencies
This implementation reduces memory usage when writing large JSON objects.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* fix: improve safeWriteJson locking mechanism
- Use file path itself for locking instead of separate lock file
- Improve error handling and clarity of code
- Enhance cleanup of temporary files
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: fix safeWriteJson test failures
- Ensure test file exists before locking
- Add proper mocking for fs.createWriteStream
- Fix test assertions to match expected behavior
- Improve test comments to follow project guidelines
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* test: update tests to work with safeWriteJson
Updated tests to work with safeWriteJson instead of direct fs.writeFile calls:
- Updated importExport.test.ts to expect safeWriteJson calls instead of fs.writeFile
- Fixed McpHub.test.ts by properly mocking fs/promises module:
- Moved jest.mock() to the top of the file before any imports
- Added mock implementations for all fs functions used by safeWriteJson
- Updated the test setup to work with the mocked fs module
All tests now pass successfully.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* refactor: replace JSON.stringify with safeWriteJson for file operations
Replace all non-test instances of JSON.stringify used for writing to JSON files with safeWriteJson to ensure safer file operations with proper locking, error handling, and atomic writes.
- Updated src/services/mcp/McpHub.ts
- Updated src/services/code-index/cache-manager.ts
- Updated src/api/providers/fetchers/modelEndpointCache.ts
- Updated src/api/providers/fetchers/modelCache.ts
- Updated tests to match the new implementation
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
* docs: add rules for using safeWriteJson
Add concise rules for using safeWriteJson instead of JSON.stringify with file operations to ensure atomic writes and prevent data corruption.
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
---------
Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Eric Wheeler <roo-code@z.ewheeler.org>
Co-authored-by: Daniel <57051444+daniel-lxs@users.noreply.github.com>
* Added changeset for my code changes
* Use URL constructor for joining baseUrl and path in litellm.ts
* Restoring Roo dotfiles
* Moved Roo dotfiles to root directory
* Revert this
* Add tests for litellm URL constructor fix
---------
Co-authored-by: Kevin White <kcwhite@ancestry.com>
Co-authored-by: Daniel <57051444+daniel-lxs@users.noreply.github.com>
Co-authored-by: Daniel Riccio <ricciodaniel98@gmail.com>
