Sameer Kankute
c638f45213
Implement Bedrock Guardrail apply_guardrail endpoint support ( #15892 )
...
* Add bedrock support for apply gaurdrails
* Add bedrock support doc
* remove unused variable
* remove unused variable
2025-10-24 10:24:03 -07:00
Sameer Kankute
b9585b1db5
Update documentation for enable_caching_on_provider_specific_optional_params ( #15885 )
2025-10-24 10:22:27 -07:00
Alexsander Hamir
9338727960
feat(proxy): support absolute RPM/TPM in priority_reservation ( #15813 )
...
* feat(proxy): support absolute RPM/TPM in priority_reservation
Allow priority reservations as absolute values instead of percentages:
- Float: {'prod': 0.75} (75%, existing)
- RPM: {'prod': {'type': 'rpm', 'value': 750}}
- TPM: {'prod': {'type': 'tpm', 'value': 750000}}
Added _convert_to_percent() that converts absolute values to percentages
based on model capacity. Fully backward compatible.
* feat(types): convert priority_reservation Dict to TypedDict
Add PriorityReservationDict TypedDict to replace generic Dict type in priority_reservation configuration.
Changes:
- Add PriorityReservationDict to litellm/types/utils.py
- Update convert_priority_to_percent() signature in rate_limiter_utils.py
- Update litellm.priority_reservation type annotation in __init__.py
Improves IDE autocomplete and type checking for priority reservation configs.
* docs: update dynamic rate limiter priority reservation docs
2025-10-23 18:30:36 -07:00
mubashirosmani
c5fee97850
docs: add OpenAI responses api ( #15868 )
...
* docs: add tip openai page
* added responses api
---------
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
2025-10-23 18:25:59 -07:00
mubashirosmani
09c1ad190e
docs: add tip openai page ( #15866 )
...
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
2025-10-23 18:25:25 -07:00
Ishaan Jaffer
ea8a604188
fix to .debug
2025-10-23 18:08:56 -07:00
Ishaan Jaffer
f4e98f715a
fix linting
2025-10-23 18:07:28 -07:00
Ishaan Jaffer
76d658eee8
fix linting
2025-10-23 18:05:30 -07:00
Ishaan Jaff
5de912375c
[Feat] UI - Add logos for search providers ( #15872 )
...
* add LiteLLM_SearchToolsTable
* init SearchToolRegistry
* fix add SearchToolRegistry
* fix add SearchToolRegistry
* fix handling search tool management
* fix search imports
* fix registry
* init search tools in memory
* fix init tools in mem
* fix TypedDict def
* add new SCHEMA
* bump proxy extras
* add LiteLLM_SearchToolsTable_search_tool_name_key
* bump extras with migration
* fix working CRUD Ops
* fix: _init_search_tools_in_db
* add UI friendly name for search providers
* add ui friendly name for search providers
* add providers available
* working layout
* better layout
* clean add search tool
* update_router_search_tools
* fix remove in memory registry, since router is in mem store
* allow testing search tool connection
* clean create search tool
* add test_search_tool_connection
* fix: _init_search_tools_in_db
* add searchToolQueryCall
* fix icon
* clean tester
* add parallel ai logo
* add exa ai logo
* add google PSE logo
* add tavily logo
* add dataforseo + perplexity
* add parallel ai logo
* UI show logos for search
2025-10-23 18:00:40 -07:00
Ishaan Jaff
fc9aba279e
[Feat] UI - Search Tools, allow adding search tools on UI + testing search ( #15871 )
...
* add LiteLLM_SearchToolsTable
* init SearchToolRegistry
* fix add SearchToolRegistry
* fix add SearchToolRegistry
* fix handling search tool management
* fix search imports
* fix registry
* init search tools in memory
* fix init tools in mem
* fix TypedDict def
* add new SCHEMA
* bump proxy extras
* add LiteLLM_SearchToolsTable_search_tool_name_key
* bump extras with migration
* fix working CRUD Ops
* fix: _init_search_tools_in_db
* add UI friendly name for search providers
* add ui friendly name for search providers
* add providers available
* working layout
* better layout
* clean add search tool
* update_router_search_tools
* fix remove in memory registry, since router is in mem store
* allow testing search tool connection
* clean create search tool
* add test_search_tool_connection
* fix: _init_search_tools_in_db
* add searchToolQueryCall
* fix icon
* clean tester
2025-10-23 17:59:29 -07:00
Ishaan Jaff
d8ea1665c7
[Feat] - [Backend] Search APIs - Allow storing configured Search APIs in DB ( #15862 )
...
* add LiteLLM_SearchToolsTable
* init SearchToolRegistry
* fix add SearchToolRegistry
* fix add SearchToolRegistry
* fix handling search tool management
* fix search imports
* fix registry
* init search tools in memory
* fix init tools in mem
* fix TypedDict def
* add new SCHEMA
* bump proxy extras
* add LiteLLM_SearchToolsTable_search_tool_name_key
* bump extras with migration
* fix working CRUD Ops
* fix: _init_search_tools_in_db
2025-10-23 17:57:49 -07:00
Byron Grogan
bf47c25de0
[fix] Pass user-defined headers and extra_headers to image-edit calls ( #15811 )
2025-10-23 15:08:10 -07:00
Krish Dholakia
e0c4baf66f
fix(ui/): fix routing for custom server root path ( #15701 )
...
* fix(ui/): fix routing for custom server root path
* fix: fix eslint errors
2025-10-23 13:59:29 -07:00
Jacob Linney
0644c20604
fix(vertex-ai): cost tracking for search spend ( #15859 )
2025-10-23 13:29:10 -07:00
Ishaan Jaff
6d947d7326
[Bug Fix] Exa Search API - ensure request params are sent to Exa AI ( #15855 )
...
* fix filter_out_litellm_params
* use filter_out_litellm_params
* test_filter_out_litellm_params
2025-10-23 11:56:30 -07:00
Ishaan Jaff
511d435f6f
[Bug Fix]: Hooks broken on /bedrock passthrough due to missing metadata ( #15849 )
...
* refactor handle_bedrock_passthrough_router_model
* test_bedrock_router_passthrough_metadata_initialization
2025-10-23 11:52:37 -07:00
Alexsander Hamir
8c5118195d
fix: replace deprecated gemini-1.5-pro-preview-0514 with gemini-2.5-flash-lite in function calling test ( #15852 )
2025-10-23 11:48:58 -07:00
Ishaan Jaffer
d53a8b3e8d
Revert "fix(responses-api): simplify reasoning item handling for gpt-5-codex ( #15815 )"
...
This reverts commit cc63cf28ff
2025-10-23 11:17:36 -07:00
Ishaan Jaffer
ae7b13550e
test_models_by_provider
2025-10-23 09:10:41 -07:00
Ishaan Jaffer
8e65f99ebb
test fix TTS
2025-10-23 08:50:54 -07:00
Ishaan Jaffer
5498a8b099
test_ensure_initialize_azure_sdk_client_always_used
2025-10-23 08:50:39 -07:00
Ishaan Jaffer
74b8a1dbdf
test_aaamodel_prices_and_context_window_json_is_valid
2025-10-23 08:47:08 -07:00
Ishaan Jaffer
bfe4167183
bump: version 1.78.7 → 1.78.8
2025-10-23 08:42:03 -07:00
Alexsander Hamir
cc63cf28ff
fix(responses-api): simplify reasoning item handling for gpt-5-codex ( #15815 )
...
Simplifies the _handle_reasoning_item method to only filter out None values
without attempting to construct ResponseReasoningItem objects. This approach:
- Avoids mutation of encrypted_content in reasoning items
- Fixes compatibility with gpt-5-codex and Codex CLI
- Removes dependency on ResponseReasoningItem import
- Simplifies the code by removing complex fallback logic
The previous approach was trying to construct ResponseReasoningItem objects
with required fields (id, summary), which caused issues with encrypted
payloads. The new approach simply filters None values while preserving
all other data intact.
Issue: https://github.com/BerriAI/litellm/issues/13484
Reported-by: rajpathak (OpenAI)
Tested-by: rajpathak (OpenAI Codex CLI)
2025-10-23 08:41:37 -07:00
Krish Dholakia
573306f3cd
(feat) Vector Stores: support Vertex AI Search API as vector store through LiteLLM ( #15781 )
...
* feat(vector_stores/): initial commit adding Vertex AI Search API support for litellm
new vector store provider
* feat(vector_store/): use vector store id for vertex ai search api
* fix: transformation.py
cleanup
* fix: implement abstract function
* fix: fix linting error
* fix: main.py
fix check
2025-10-22 18:56:36 -07:00
Ishaan Jaff
3e4b5ef3a5
[Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI ( #15821 )
...
* add search cost tracking
* add cost tracking for tavily tiers
* add search to call types
* add search_provider_cost_per_query
* add cost tracking for search APIs
* add cost tracking search APIs
* docs cost tracking search
* docs search
* fix linting
2025-10-22 17:29:09 -07:00
Ishaan Jaff
143e314dda
[Feat] Add Google PSE Search Provider ( #15816 )
...
* docs google PSE
* add SearchProviders
* add search providers
* add PSE search
* add SearchProviders
* get_provider_search_config
* add Search
* init Search
* add get_http_method on BaseSearch
* fixes for Google PSE
* TestGooglePSESearch
2025-10-22 16:01:28 -07:00
Ishaan Jaff
ad62a6d3d1
[Feat] Add DataforSEO Search API ( #15817 )
...
* docs google PSE
* add SearchProviders
* add search providers
* add PSE search
* add SearchProviders
* get_provider_search_config
* add Search
* init Search
* add get_http_method on BaseSearch
* fixes for Google PSE
* TestGooglePSESearch
* add DATAFORSEO
* add DataForSEOSearchConfig
* TestDataForSEOSearch
* add DataForSEO
* fix base transform
* fix search
* fix dataforSEO
* docs fix
* fix linting
* fix linting
2025-10-22 16:00:40 -07:00
nuernber
799a2b624a
use proper bedrock model name in health check ( #15808 )
2025-10-22 15:24:57 -07:00
Ishaan Jaff
ec6a5ffa2d
[Fix] Azure AI Speech - Ensure voice is mapped from request body -> SSML body , allow sending role and style ( #15810 )
...
* update map_openai_params
* fix update voice transform
* fix text_to_speech_provider_config
* test_azure_ava_tts_with_custom_voice
* test Azure AVA style, role sent
* _build_express_as_element
* docs custom params
* build LANG
* fix transform
* fix transform
* fix speech
* docs update
* docs azure ai speech
2025-10-22 14:41:11 -07:00
Ishaan Jaff
d91efa7a7b
[Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure ( #15804 )
...
* add ErrorEventError nested field
* test_openai_responses_api_token_limit_error
* test_openai_responses_api_token_limit_error
2025-10-22 14:18:46 -07:00
Sameer Kankute
2ab2d15efc
Fix Token Spend is under budget for passthrough ( #15805 )
2025-10-22 10:55:22 -07:00
Ishaan Jaffer
abe67df95e
refactor large func
2025-10-22 10:04:52 -07:00
Ishaan Jaffer
57a2ec3beb
fix: _extract_fields_recursive
2025-10-22 09:37:29 -07:00
Roman G
eac3cba44f
Support for embeddings_by_type Response Format in Bedrock Cohere Embed v1 ( #15707 )
...
* feat(cohere): Enhance embedding transformation to support Bedrock's embeddings by type
* test(cohere): Add unit tests for embedding transformation responses
2025-10-22 09:32:11 -07:00
Ishaan Jaffer
e80bba83e3
test fix
2025-10-22 09:29:04 -07:00
soo-jin.kim
03e1d93199
fix: Apply max_connections configuration to Redis async client ( #15797 )
...
* fix: Apply max_connections configuration to Redis async client
- Add max_connections to available Redis cluster kwargs
- Add connection_pool parameter to get_redis_async_client()
- Pass connection_pool to Redis client if provided
- Prevents Redis connection exhaustion under high load
* test: Add tests for Redis max_connections feature
- Test max_connections is included in cluster kwargs
- Test connection_pool parameter is properly passed to async client
- Test async client works without connection_pool parameter
All 3 tests pass successfully
2025-10-22 09:19:08 -07:00
soo-jin.kim
8050995dbb
fix: Rename configured_cold_storage_logger to cold_storage_custom_logger ( #15798 )
...
- Change variable name in litellm/__init__.py from configured_cold_storage_logger to cold_storage_custom_logger
- Update all references across the codebase to use the new variable name
- This fixes silent failure of cold storage logging due to variable name mismatch
- Configuration files use cold_storage_custom_logger, code should match
Files updated:
- litellm/__init__.py
- litellm/litellm_core_utils/litellm_logging.py
- litellm/proxy/spend_tracking/cold_storage_handler.py
- litellm/responses/litellm_completion_transformation/session_handler.py
- tests/test_litellm/litellm_core_utils/test_litellm_logging.py
- tests/test_litellm/responses/litellm_completion_transformation/test_session_handler.py
2025-10-22 09:17:08 -07:00
nuernber
69946bb35b
fix the date for sonnet 3.7 in govcloud ( #15800 )
2025-10-22 09:14:07 -07:00
Anthony Ivan
5f7a6b49eb
Feat: Allow prompt caching to be used for Anthropic Claude on Databricks ( #15801 )
2025-10-22 09:11:04 -07:00
Sameer Kankute
44495c0117
fix encrypted content error ( #15782 )
2025-10-21 23:29:48 -07:00
Krish Dholakia
29a97784e7
(feat) Passthrough - set auth on passthrough endpoints, on the UI ( #15778 )
...
* fix(add_pass_through.tsx): allow setting 'auth' to true for passthrough endpoints on the UI
* fix: working update auth on passthrough endpoints + show auth on passthrough table
2025-10-21 23:26:43 -07:00
Ishaan Jaffer
02e34a57d6
anthropic.claude-3-7-sonnet-20240620-v1:0
2025-10-21 19:19:52 -07:00
Ishaan Jaffer
bd0a8a047a
docs search_tools
2025-10-21 19:15:18 -07:00
Ishaan Jaffer
adc503c091
bump: version 1.78.6 → 1.78.7
2025-10-21 19:06:05 -07:00
Ishaan Jaff
f5a80110c1
[Feat] Add /search endpoint on LiteLLM Gateway ( #15780 )
...
* add SearchProvider
* add SearchToolTypedDict
* add search
* add SearchAPIRouter
* working router level search
* add search to allowed llm / ocr routes
* feat: add search_router
* add routing + proxy for search APIs
* /v1/search/{search_tool_name}
* fix search routing
* feat: parse_search_tools
* clean up sidebar
* docs fix
* router tests for search tools
* docs fix
2025-10-21 19:05:20 -07:00
Ishaan Jaffer
d9b85ab276
fix: rename search_provider
2025-10-21 17:42:18 -07:00
Ishaan Jaffer
bea8e13a94
fix: GuardrailConfigModel
2025-10-21 17:11:56 -07:00
wangjifeng
8cbaec0310
feat: Add imageConfig parameter for gemini-2.5-flash-image ( #15530 )
...
* Add imageConfig parameter support for Vertex AI to enable gemini-2.5-flash-image model requirements
* Add test for imageConfig parameter support in Vertex AI Gemini transformation
2025-10-21 17:08:30 -07:00
Ishaan Jaff
7b939b4558
[Feat] Add EXA AI Search API to LiteLLM ( #15774 )
...
* add BaseSearchConfig
* add BaseSearchConfig
* validate_environment
* fix handlers
* add PerplexitySearchConfig
* add PerplexitySearchConfig
* add LiteLLM Search API module.
* add BaseSearchConfig
* add _build_search_optional_params
* add search_testing
* add BaseSearchTest
* add TestPerplexitySearch
* fix BASE
* fix handler
* add search API
* add to init
* fix: working perplexity search API
* add _hidden_params to search
* add TAVILY to LlmProviders
* add TavilySearchConfig
* add TavilySearchConfig
* TestTavilySearch
* add tavily transform
* TestParallelAISearch
* add LlmProviders
* add ParallelAISearchConfig
* add ParallelAISearchConfig
* ParallelAISearchConfig
* add EXA AI Search API
* add ExaAISearchConfig
* TestExaAISearch
* add get_supported_perplexity_optional_params
* add Exa AI Search API
* add transform_search_request
* add ExaAISearchConfig
* fix linting errors
* transform_search_request
2025-10-21 17:06:23 -07:00
