Sameer Kankute
c638f45213
Implement Bedrock Guardrail apply_guardrail endpoint support ( #15892 )
...
* Add bedrock support for apply gaurdrails
* Add bedrock support doc
* remove unused variable
* remove unused variable
2025-10-24 10:24:03 -07:00
Byron Grogan
bf47c25de0
[fix] Pass user-defined headers and extra_headers to image-edit calls ( #15811 )
2025-10-23 15:08:10 -07:00
Krish Dholakia
e0c4baf66f
fix(ui/): fix routing for custom server root path ( #15701 )
...
* fix(ui/): fix routing for custom server root path
* fix: fix eslint errors
2025-10-23 13:59:29 -07:00
Jacob Linney
0644c20604
fix(vertex-ai): cost tracking for search spend ( #15859 )
2025-10-23 13:29:10 -07:00
Ishaan Jaff
6d947d7326
[Bug Fix] Exa Search API - ensure request params are sent to Exa AI ( #15855 )
...
* fix filter_out_litellm_params
* use filter_out_litellm_params
* test_filter_out_litellm_params
2025-10-23 11:56:30 -07:00
Ishaan Jaff
511d435f6f
[Bug Fix]: Hooks broken on /bedrock passthrough due to missing metadata ( #15849 )
...
* refactor handle_bedrock_passthrough_router_model
* test_bedrock_router_passthrough_metadata_initialization
2025-10-23 11:52:37 -07:00
Alexsander Hamir
8c5118195d
fix: replace deprecated gemini-1.5-pro-preview-0514 with gemini-2.5-flash-lite in function calling test ( #15852 )
2025-10-23 11:48:58 -07:00
Ishaan Jaffer
ae7b13550e
test_models_by_provider
2025-10-23 09:10:41 -07:00
Ishaan Jaffer
8e65f99ebb
test fix TTS
2025-10-23 08:50:54 -07:00
Ishaan Jaffer
5498a8b099
test_ensure_initialize_azure_sdk_client_always_used
2025-10-23 08:50:39 -07:00
Ishaan Jaffer
74b8a1dbdf
test_aaamodel_prices_and_context_window_json_is_valid
2025-10-23 08:47:08 -07:00
Krish Dholakia
573306f3cd
(feat) Vector Stores: support Vertex AI Search API as vector store through LiteLLM ( #15781 )
...
* feat(vector_stores/): initial commit adding Vertex AI Search API support for litellm
new vector store provider
* feat(vector_store/): use vector store id for vertex ai search api
* fix: transformation.py
cleanup
* fix: implement abstract function
* fix: fix linting error
* fix: main.py
fix check
2025-10-22 18:56:36 -07:00
Ishaan Jaff
3e4b5ef3a5
[Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI ( #15821 )
...
* add search cost tracking
* add cost tracking for tavily tiers
* add search to call types
* add search_provider_cost_per_query
* add cost tracking for search APIs
* add cost tracking search APIs
* docs cost tracking search
* docs search
* fix linting
2025-10-22 17:29:09 -07:00
Ishaan Jaff
ad62a6d3d1
[Feat] Add DataforSEO Search API ( #15817 )
...
* docs google PSE
* add SearchProviders
* add search providers
* add PSE search
* add SearchProviders
* get_provider_search_config
* add Search
* init Search
* add get_http_method on BaseSearch
* fixes for Google PSE
* TestGooglePSESearch
* add DATAFORSEO
* add DataForSEOSearchConfig
* TestDataForSEOSearch
* add DataForSEO
* fix base transform
* fix search
* fix dataforSEO
* docs fix
* fix linting
* fix linting
2025-10-22 16:00:40 -07:00
nuernber
799a2b624a
use proper bedrock model name in health check ( #15808 )
2025-10-22 15:24:57 -07:00
Ishaan Jaff
ec6a5ffa2d
[Fix] Azure AI Speech - Ensure voice is mapped from request body -> SSML body , allow sending role and style ( #15810 )
...
* update map_openai_params
* fix update voice transform
* fix text_to_speech_provider_config
* test_azure_ava_tts_with_custom_voice
* test Azure AVA style, role sent
* _build_express_as_element
* docs custom params
* build LANG
* fix transform
* fix transform
* fix speech
* docs update
* docs azure ai speech
2025-10-22 14:41:11 -07:00
Ishaan Jaff
d91efa7a7b
[Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure ( #15804 )
...
* add ErrorEventError nested field
* test_openai_responses_api_token_limit_error
* test_openai_responses_api_token_limit_error
2025-10-22 14:18:46 -07:00
Sameer Kankute
2ab2d15efc
Fix Token Spend is under budget for passthrough ( #15805 )
2025-10-22 10:55:22 -07:00
Roman G
eac3cba44f
Support for embeddings_by_type Response Format in Bedrock Cohere Embed v1 ( #15707 )
...
* feat(cohere): Enhance embedding transformation to support Bedrock's embeddings by type
* test(cohere): Add unit tests for embedding transformation responses
2025-10-22 09:32:11 -07:00
Ishaan Jaffer
e80bba83e3
test fix
2025-10-22 09:29:04 -07:00
soo-jin.kim
03e1d93199
fix: Apply max_connections configuration to Redis async client ( #15797 )
...
* fix: Apply max_connections configuration to Redis async client
- Add max_connections to available Redis cluster kwargs
- Add connection_pool parameter to get_redis_async_client()
- Pass connection_pool to Redis client if provided
- Prevents Redis connection exhaustion under high load
* test: Add tests for Redis max_connections feature
- Test max_connections is included in cluster kwargs
- Test connection_pool parameter is properly passed to async client
- Test async client works without connection_pool parameter
All 3 tests pass successfully
2025-10-22 09:19:08 -07:00
soo-jin.kim
8050995dbb
fix: Rename configured_cold_storage_logger to cold_storage_custom_logger ( #15798 )
...
- Change variable name in litellm/__init__.py from configured_cold_storage_logger to cold_storage_custom_logger
- Update all references across the codebase to use the new variable name
- This fixes silent failure of cold storage logging due to variable name mismatch
- Configuration files use cold_storage_custom_logger, code should match
Files updated:
- litellm/__init__.py
- litellm/litellm_core_utils/litellm_logging.py
- litellm/proxy/spend_tracking/cold_storage_handler.py
- litellm/responses/litellm_completion_transformation/session_handler.py
- tests/test_litellm/litellm_core_utils/test_litellm_logging.py
- tests/test_litellm/responses/litellm_completion_transformation/test_session_handler.py
2025-10-22 09:17:08 -07:00
Anthony Ivan
5f7a6b49eb
Feat: Allow prompt caching to be used for Anthropic Claude on Databricks ( #15801 )
2025-10-22 09:11:04 -07:00
Sameer Kankute
44495c0117
fix encrypted content error ( #15782 )
2025-10-21 23:29:48 -07:00
Ishaan Jaff
f5a80110c1
[Feat] Add /search endpoint on LiteLLM Gateway ( #15780 )
...
* add SearchProvider
* add SearchToolTypedDict
* add search
* add SearchAPIRouter
* working router level search
* add search to allowed llm / ocr routes
* feat: add search_router
* add routing + proxy for search APIs
* /v1/search/{search_tool_name}
* fix search routing
* feat: parse_search_tools
* clean up sidebar
* docs fix
* router tests for search tools
* docs fix
2025-10-21 19:05:20 -07:00
Ishaan Jaffer
d9b85ab276
fix: rename search_provider
2025-10-21 17:42:18 -07:00
wangjifeng
8cbaec0310
feat: Add imageConfig parameter for gemini-2.5-flash-image ( #15530 )
...
* Add imageConfig parameter support for Vertex AI to enable gemini-2.5-flash-image model requirements
* Add test for imageConfig parameter support in Vertex AI Gemini transformation
2025-10-21 17:08:30 -07:00
Ishaan Jaff
7b939b4558
[Feat] Add EXA AI Search API to LiteLLM ( #15774 )
...
* add BaseSearchConfig
* add BaseSearchConfig
* validate_environment
* fix handlers
* add PerplexitySearchConfig
* add PerplexitySearchConfig
* add LiteLLM Search API module.
* add BaseSearchConfig
* add _build_search_optional_params
* add search_testing
* add BaseSearchTest
* add TestPerplexitySearch
* fix BASE
* fix handler
* add search API
* add to init
* fix: working perplexity search API
* add _hidden_params to search
* add TAVILY to LlmProviders
* add TavilySearchConfig
* add TavilySearchConfig
* TestTavilySearch
* add tavily transform
* TestParallelAISearch
* add LlmProviders
* add ParallelAISearchConfig
* add ParallelAISearchConfig
* ParallelAISearchConfig
* add EXA AI Search API
* add ExaAISearchConfig
* TestExaAISearch
* add get_supported_perplexity_optional_params
* add Exa AI Search API
* add transform_search_request
* add ExaAISearchConfig
* fix linting errors
* transform_search_request
2025-10-21 17:06:23 -07:00
Ishaan Jaff
208f76f8ad
[Feat] Add Parallel AI - Search API ( #15772 )
...
* add BaseSearchConfig
* add BaseSearchConfig
* validate_environment
* fix handlers
* add PerplexitySearchConfig
* add PerplexitySearchConfig
* add LiteLLM Search API module.
* add BaseSearchConfig
* add _build_search_optional_params
* add search_testing
* add BaseSearchTest
* add TestPerplexitySearch
* fix BASE
* fix handler
* add search API
* add to init
* fix: working perplexity search API
* add _hidden_params to search
* add TAVILY to LlmProviders
* add TavilySearchConfig
* add TavilySearchConfig
* TestTavilySearch
* add tavily transform
* TestParallelAISearch
* add LlmProviders
* add ParallelAISearchConfig
* add ParallelAISearchConfig
* ParallelAISearchConfig
2025-10-21 17:00:05 -07:00
Ishaan Jaff
b9f3f9fb79
[Feat] Add Tavily Search API ( #15770 )
...
* add BaseSearchConfig
* add BaseSearchConfig
* validate_environment
* fix handlers
* add PerplexitySearchConfig
* add PerplexitySearchConfig
* add LiteLLM Search API module.
* add BaseSearchConfig
* add _build_search_optional_params
* add search_testing
* add BaseSearchTest
* add TestPerplexitySearch
* fix BASE
* fix handler
* add search API
* add to init
* fix: working perplexity search API
* add _hidden_params to search
* add TAVILY to LlmProviders
* add TavilySearchConfig
* add TavilySearchConfig
* TestTavilySearch
* add tavily transform
2025-10-21 16:59:29 -07:00
wenhua
b0ccc35a9c
fix(ollama): Enhance chunk parsing for empty responses without 'thinking' and improve error logging ( #13333 ) ( #15717 )
2025-10-21 16:59:01 -07:00
Ishaan Jaff
e1cb92862e
[Feat] Add def search() APIs for Web Search - Perplexity API ( #15769 )
...
* add BaseSearchConfig
* add BaseSearchConfig
* validate_environment
* fix handlers
* add PerplexitySearchConfig
* add PerplexitySearchConfig
* add LiteLLM Search API module.
* add BaseSearchConfig
* add _build_search_optional_params
* add search_testing
* add BaseSearchTest
* add TestPerplexitySearch
* fix BASE
* fix handler
* add search API
* add to init
* fix: working perplexity search API
* add _hidden_params to search
2025-10-21 16:58:51 -07:00
Ishaan Jaff
9135e748a0
[Feat ] /ocr - Add mode + Health check support for OCR models ( #15767 )
...
* get_mode_handlers
* use get_mode_handlers
* test_ahealth_check_ocr
* Add OCR mode to test models
* docs OCR Health Checks
* fix connection endpoint
2025-10-21 16:58:37 -07:00
Javier Garcia
b0a3a7c4fb
Add details in docs ( #15721 )
...
* Add details in docs
* add logic to set span attributes and unit tests
* Restore html files
* Remove html files
* Remove html files
2025-10-21 16:57:51 -07:00
Thomas Mildner
1cfc4624c3
[Feat] Add SENTRY_ENVIRONMENT configuration for Sentry integration ( #15760 )
...
* [Feat] Add SENTRY_ENVIRONMENT configuration for Sentry integration and corresponding tests
* [Refactor] Enhance test_sentry_environment by mocking sentry_sdk and improving environment handling
* [Fix] Update default SENTRY_ENVIRONMENT to 'production' and enhance test for Sentry integration
* [Fix] Update test_sentry_environment to verify correct handling of SENTRY_ENVIRONMENT values
* [Fix] Update test_sentry_environment to assert correct handling of production environment
2025-10-21 16:40:55 -07:00
Vinod Singh
d4aadda692
Auth Header Fix for MCP Tool Call ( #15736 )
...
* fixed the Auth header for MCP Tool Call
* Final fix for Auth header
* testcase for mcp_auth_header_extraction, insensitive_alias_matching, insensitive_servername_matching added
2025-10-21 13:58:03 -07:00
YutaSaito
d79bdd491f
feat: add GraySwan Guardrails support ( #15756 )
2025-10-21 11:13:50 -07:00
Talal
46d55bd92a
fix: Add response_type + PKCE parameters to OAuth authorization endpoint ( #15720 )
...
* fix: Add response_type parameter to OAuth authorization endpoint
Fixes #15684
OAuth providers like Google require the response_type parameter during
the authorization flow. This commit adds response_type=code to the
authorization redirect parameters, which is required by the OAuth 2.0
specification (RFC 6749 Section 4.1.1).
Changes:
- Added response_type=code to authorization params in discoverable_endpoints.py
- Added test coverage for the response_type parameter
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com>
* fix oauth flow by forwarding code_challenge and forwarding code_verifier
---------
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 09:43:19 -07:00
Tom Haynes
98f1d63508
use correct otel logger, and normalise otel paths ( #15645 )
2025-10-21 09:16:03 -07:00
Ishaan Jaff
92335d991c
[Feat] Add Azure AVA (Speech AI) Cost Tracking ( #15754 )
...
* add azure/speech/ cost tracking
* test_azure_ava_tts_async
* add azure/speech to model cost map
* docs cost tracking
* docs tts AVA
* add azure/speech/azure-tts
2025-10-20 18:01:51 -07:00
Ishaan Jaffer
60fab591db
rename test files
2025-10-20 18:00:17 -07:00
Ishaan Jaff
157739da01
[Bug]: Fix Incorrect status value in responses api with gemini ( #15753 )
...
* _map_chat_completion_finish_reason_to_responses_status
* test_transform_chat_completion_response_with_reasoning_content
* test_transform_chat_completion_response_output_item_status
2025-10-20 17:58:56 -07:00
Ishaan Jaff
73a23a6c78
[Feat] Add Azure AVA TTS integration ( #15749 )
...
* add AzureBaseIssueTokenHandler
* add BaseTextToSpeechConfig
* async_text_to_speech_handler
* add AzureAVATextToSpeechConfig
* add get_provider_text_to_speech_config
* add AzureAVATextToSpeechConfig
* fixes for base_llm_http_handler
* fix transform_text_to_speech_request
* test_azure_ava_tts_async
* test_azure_ava_tts_async
* fix TextToSpeechRequestData
* fix transform_text_to_speech_request
* add text_to_speech_handler in LLMHttpHandler
* remove old file
* fix transform_text_to_speech_request
* fix dispatch_text_to_speech
* fix azure TTS
* fix AVA TTS
* fix transform
* fix linting
* ci/cd - use one job for audio testing
* fix tests
* fix llm http handler debugging
* unit tests azure tts
* docs Azure speech
* docs fix
* docs azure AVA
* docs azure AVA
* fix handlers
* test_async_realtime_uses_max_size_parameter
2025-10-20 16:52:23 -07:00
Sameer Kankute
1fb798f81d
(Bug) Fix JSON serialization error in Helicone logging by removing OpenTelemetry span from metadata ( #15728 )
...
* remove span object from helicon metadata
* Add test
2025-10-20 08:53:22 -07:00
Timothée Lecomte
3ef9b2015a
feat: read from custom-llm-provider header ( #15528 )
2025-10-18 22:04:53 -07:00
Ishaan Jaff
f55745fc5e
[Fix] Forward anthropic-beta headers to Bedrock, VertexAI ( #15700 )
...
* [Fix] Forward anthropic-beta headers to Bedrock and other cross-provider scenarios (#15623 )
* add_provider_specific_headers_to_request
* fix add_provider_specific_headers_to_request
* test_provider_specific_header_multi_provider
* test_provider_specific_header_in_request
---------
Co-authored-by: Jack Venberg <jack.venberg@rover.com>
2025-10-18 16:26:32 -07:00
Ishaan Jaff
6ab9b0af9f
[Fix] Anthropic cache_control incorrectly applied to all content items instead of last item only ( #15699 )
...
* fix: _safe_insert_cache_control_in_message
* test_anthropic_cache_control_hook_system_message
* docs prompt cache injection
* docs fix
2025-10-18 15:18:08 -07:00
Jason Roberts
c471bf1f16
feat(guardrails): Add content masking and streaming support to PANW Prisma AIRS guardrail ( #15666 )
...
* feat(guardrails): Add content masking and streaming support to PANW Prisma AIRS
- Add mask_request_content and mask_response_content parameters
- Implement content masking for prompts and responses
- Add streaming support with real-time masking
- Add comprehensive test coverage (28 tests)
- Update documentation with masking examples and security notes
* fix(guardrails): Fix PANW Prisma AIRS env var fallback and text completion support
2025-10-18 13:57:51 -07:00
katsuhiro muto
d5e686b3e8
[Fix] Support service_tier in chat completion ( #15693 )
...
* Support service_tier
* fix test
2025-10-18 13:55:54 -07:00
Krish Dholakia
4e141df03a
(feat) Team level model-specific tpm/rpm limits + working key-level validation of tpm/rpm limit when assigned to team ( #15513 )
...
* fix(support-model-specific-tpm/rpm-limits): Allows setting rate limits by tpm/rpm for models by team
* fix(key_management_endpoints.py): enforce guaranteed throughput with key-level model tpm/rpm limits, when team-level tpm/rpm limits are set
* test: add unit testing
* fix: fix minor linting errors
* fix: refactor
2025-10-18 13:14:04 -07:00
