Commit graph

14665 commits

Author SHA1 Message Date
Carlo Alberto Ferraris
8b1424166b
attempt to avoid/minimize deadlocks (#15281)
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-10-24 12:22:38 -07:00
Sameer Kankute
0f9996a4d0
Litellm sameer oct staging (#15806)
* Addd v2/chat support for cohere

* fix streaming

* Use v2_transformation for logging passthrough:

* Use v2_transformation for logging passthrough:

* Add test for checking if document and citation_options is getting passed

* Update the cohere model

* Add cost tracking for vertex ai passthrough batch jobs

* Add full passthrough support

* refactor code according to the comments

* Add passthrough handler

* remove invalid params

* Updated documentation

* Updated documentation

* Updated documentation

* Correct the import

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix documentation for get batch req (#15742)

* Add grounding info to responses API (#15737)

* Add grounding info to responses API

* fix lint errors

* Use typed objects for annotations

* Use typed objects for annotations

* fix mypy error

* Litellm fix json serialize alreting 2 (#15741)

* fix json serializable error for alerts

* Add test

* fix mypt errors

* fix mypt errors

* Add Qwen3 imported model support for AWS Bedrock (#15783)

* Add qwen imported model support

* fix mypy errors

* fix empty user message error (#15784)

* fix typed dict for list

* Add azure supported videos endpoint

* fix mapped tests

* add azure sora models to model map

* Add OpenAI video generation and content retrieval support (#15745)

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix typed dict for list

* fix mypy errors

* move directory

* make v2 chat default

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Revert "Add Azure Video Generation Support with Sora Integration"

* refactor videos repo

* add test

* Add azure openai videos support

* Add azure openai videos support

* Add router endpoint support for videos

* fix mypy error

* add azure models

* fix mapped test

* fix mypy error

* Add proxy router test

* Add proxy router test

* remove deprecated model name from tests

* fix import error

* fix import error

* Add gaurdrail integration in videos endpoint

* Add logging support for videos endpoint

* Add final documentation supporting videos integration

* fix model name and document input

* Update literals to avoid mypy errors

* Remove unused imports and print statements

* revert guardrail support for video generation and video remix

* revert guardrail support for video generation and video remix

* Fix failing mapped and llm translation tests
2025-10-24 12:17:22 -07:00
oroxenberg
c793bd5ba9
Lasso Security Guardrail: Add v3 API Support (#12452)
* 1. add v3 classify
2. add new classifix for masking
3. support same id for the conversation for pre and post
working with duplicates

* clean code, remove some debug and run tests

* update liter errors

* improvment for Code Organization, httpx Error Handling Specificity, Logging Improvements and Type

* transfer test test_lasso_guard_config to the new location

* Fix type hints and linting errors in lasso.py

- Add type: ignore for httpx module when None
- Fix return type issues in _handle_classification and _handle_masking
- Ensure masked_messages is not None before passing to _apply_masking_to_model_response
- Convert LassoResponse to dict for _log_masking_applied call
2025-10-24 11:03:58 -07:00
Sameer Kankute
c638f45213
Implement Bedrock Guardrail apply_guardrail endpoint support (#15892)
* Add bedrock support for apply gaurdrails

* Add bedrock support doc

* remove unused variable

* remove unused variable
2025-10-24 10:24:03 -07:00
Byron Grogan
bf47c25de0
[fix] Pass user-defined headers and extra_headers to image-edit calls (#15811) 2025-10-23 15:08:10 -07:00
Krish Dholakia
e0c4baf66f
fix(ui/): fix routing for custom server root path (#15701)
* fix(ui/): fix routing for custom server root path

* fix: fix eslint errors
2025-10-23 13:59:29 -07:00
Jacob Linney
0644c20604
fix(vertex-ai): cost tracking for search spend (#15859) 2025-10-23 13:29:10 -07:00
Ishaan Jaff
6d947d7326
[Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855)
* fix filter_out_litellm_params

* use filter_out_litellm_params

* test_filter_out_litellm_params
2025-10-23 11:56:30 -07:00
Ishaan Jaff
511d435f6f
[Bug Fix]: Hooks broken on /bedrock passthrough due to missing metadata (#15849)
* refactor handle_bedrock_passthrough_router_model

* test_bedrock_router_passthrough_metadata_initialization
2025-10-23 11:52:37 -07:00
Alexsander Hamir
8c5118195d
fix: replace deprecated gemini-1.5-pro-preview-0514 with gemini-2.5-flash-lite in function calling test (#15852) 2025-10-23 11:48:58 -07:00
Ishaan Jaffer
ae7b13550e test_models_by_provider 2025-10-23 09:10:41 -07:00
Ishaan Jaffer
8e65f99ebb test fix TTS 2025-10-23 08:50:54 -07:00
Ishaan Jaffer
5498a8b099 test_ensure_initialize_azure_sdk_client_always_used 2025-10-23 08:50:39 -07:00
Ishaan Jaffer
74b8a1dbdf test_aaamodel_prices_and_context_window_json_is_valid 2025-10-23 08:47:08 -07:00
Krish Dholakia
573306f3cd
(feat) Vector Stores: support Vertex AI Search API as vector store through LiteLLM (#15781)
* feat(vector_stores/): initial commit adding Vertex AI Search API support for litellm

new vector store provider

* feat(vector_store/): use vector store id for vertex ai search api

* fix: transformation.py

cleanup

* fix: implement abstract function

* fix: fix linting error

* fix: main.py

fix check
2025-10-22 18:56:36 -07:00
Ishaan Jaff
3e4b5ef3a5
[Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI (#15821)
* add search cost tracking

* add cost tracking for tavily tiers

* add search to call types

* add search_provider_cost_per_query

* add cost tracking for search APIs

* add cost tracking search APIs

* docs cost tracking search

* docs search

* fix linting
2025-10-22 17:29:09 -07:00
Ishaan Jaff
ad62a6d3d1
[Feat] Add DataforSEO Search API (#15817)
* docs google PSE

* add SearchProviders

* add search providers

* add PSE search

* add SearchProviders

* get_provider_search_config

* add Search

* init Search

* add get_http_method on BaseSearch

* fixes for Google PSE

* TestGooglePSESearch

* add DATAFORSEO

* add DataForSEOSearchConfig

* TestDataForSEOSearch

* add DataForSEO

* fix base transform

* fix search

* fix dataforSEO

* docs fix

* fix linting

* fix linting
2025-10-22 16:00:40 -07:00
nuernber
799a2b624a
use proper bedrock model name in health check (#15808) 2025-10-22 15:24:57 -07:00
Ishaan Jaff
ec6a5ffa2d
[Fix] Azure AI Speech - Ensure voice is mapped from request body -> SSML body , allow sending role and style (#15810)
* update map_openai_params

* fix update voice transform

* fix text_to_speech_provider_config

* test_azure_ava_tts_with_custom_voice

* test Azure AVA style, role sent

* _build_express_as_element

* docs custom params

* build LANG

* fix transform

* fix transform

* fix speech

* docs update

* docs azure ai speech
2025-10-22 14:41:11 -07:00
Ishaan Jaff
d91efa7a7b
[Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure (#15804)
* add ErrorEventError nested field

* test_openai_responses_api_token_limit_error

* test_openai_responses_api_token_limit_error
2025-10-22 14:18:46 -07:00
Sameer Kankute
2ab2d15efc
Fix Token Spend is under budget for passthrough (#15805) 2025-10-22 10:55:22 -07:00
Roman G
eac3cba44f
Support for embeddings_by_type Response Format in Bedrock Cohere Embed v1 (#15707)
* feat(cohere): Enhance embedding transformation to support Bedrock's embeddings by type

* test(cohere): Add unit tests for embedding transformation responses
2025-10-22 09:32:11 -07:00
Ishaan Jaffer
e80bba83e3 test fix 2025-10-22 09:29:04 -07:00
soo-jin.kim
03e1d93199
fix: Apply max_connections configuration to Redis async client (#15797)
* fix: Apply max_connections configuration to Redis async client

- Add max_connections to available Redis cluster kwargs
- Add connection_pool parameter to get_redis_async_client()
- Pass connection_pool to Redis client if provided
- Prevents Redis connection exhaustion under high load

* test: Add tests for Redis max_connections feature

- Test max_connections is included in cluster kwargs
- Test connection_pool parameter is properly passed to async client
- Test async client works without connection_pool parameter

All 3 tests pass successfully
2025-10-22 09:19:08 -07:00
soo-jin.kim
8050995dbb
fix: Rename configured_cold_storage_logger to cold_storage_custom_logger (#15798)
- Change variable name in litellm/__init__.py from configured_cold_storage_logger to cold_storage_custom_logger
- Update all references across the codebase to use the new variable name
- This fixes silent failure of cold storage logging due to variable name mismatch
- Configuration files use cold_storage_custom_logger, code should match

Files updated:
- litellm/__init__.py
- litellm/litellm_core_utils/litellm_logging.py
- litellm/proxy/spend_tracking/cold_storage_handler.py
- litellm/responses/litellm_completion_transformation/session_handler.py
- tests/test_litellm/litellm_core_utils/test_litellm_logging.py
- tests/test_litellm/responses/litellm_completion_transformation/test_session_handler.py
2025-10-22 09:17:08 -07:00
Anthony Ivan
5f7a6b49eb
Feat: Allow prompt caching to be used for Anthropic Claude on Databricks (#15801) 2025-10-22 09:11:04 -07:00
Sameer Kankute
44495c0117
fix encrypted content error (#15782) 2025-10-21 23:29:48 -07:00
Ishaan Jaff
f5a80110c1
[Feat] Add /search endpoint on LiteLLM Gateway (#15780)
* add SearchProvider

* add SearchToolTypedDict

* add search

* add SearchAPIRouter

* working router level search

* add search to allowed llm / ocr routes

* feat: add search_router

* add routing + proxy for search APIs

* /v1/search/{search_tool_name}

* fix search routing

* feat: parse_search_tools

* clean up sidebar

* docs fix

* router tests for search tools

* docs fix
2025-10-21 19:05:20 -07:00
Ishaan Jaffer
d9b85ab276 fix: rename search_provider 2025-10-21 17:42:18 -07:00
wangjifeng
8cbaec0310
feat: Add imageConfig parameter for gemini-2.5-flash-image (#15530)
* Add imageConfig parameter support for Vertex AI to enable gemini-2.5-flash-image model requirements

* Add test for imageConfig parameter support in Vertex AI Gemini transformation
2025-10-21 17:08:30 -07:00
Ishaan Jaff
7b939b4558
[Feat] Add EXA AI Search API to LiteLLM (#15774)
* add BaseSearchConfig

* add BaseSearchConfig

* validate_environment

* fix handlers

* add PerplexitySearchConfig

* add PerplexitySearchConfig

* add LiteLLM Search API module.

* add BaseSearchConfig

* add _build_search_optional_params

* add search_testing

* add BaseSearchTest

* add TestPerplexitySearch

* fix BASE

* fix handler

* add search API

* add to init

* fix: working perplexity search API

* add _hidden_params to search

* add TAVILY to LlmProviders

* add TavilySearchConfig

* add TavilySearchConfig

* TestTavilySearch

* add tavily transform

* TestParallelAISearch

* add LlmProviders

* add ParallelAISearchConfig

* add ParallelAISearchConfig

* ParallelAISearchConfig

* add EXA AI Search API

* add ExaAISearchConfig

* TestExaAISearch

* add get_supported_perplexity_optional_params

* add Exa AI Search API

* add transform_search_request

* add ExaAISearchConfig

* fix linting errors

* transform_search_request
2025-10-21 17:06:23 -07:00
Ishaan Jaff
208f76f8ad
[Feat] Add Parallel AI - Search API (#15772)
* add BaseSearchConfig

* add BaseSearchConfig

* validate_environment

* fix handlers

* add PerplexitySearchConfig

* add PerplexitySearchConfig

* add LiteLLM Search API module.

* add BaseSearchConfig

* add _build_search_optional_params

* add search_testing

* add BaseSearchTest

* add TestPerplexitySearch

* fix BASE

* fix handler

* add search API

* add to init

* fix: working perplexity search API

* add _hidden_params to search

* add TAVILY to LlmProviders

* add TavilySearchConfig

* add TavilySearchConfig

* TestTavilySearch

* add tavily transform

* TestParallelAISearch

* add LlmProviders

* add ParallelAISearchConfig

* add ParallelAISearchConfig

* ParallelAISearchConfig
2025-10-21 17:00:05 -07:00
Ishaan Jaff
b9f3f9fb79
[Feat] Add Tavily Search API (#15770)
* add BaseSearchConfig

* add BaseSearchConfig

* validate_environment

* fix handlers

* add PerplexitySearchConfig

* add PerplexitySearchConfig

* add LiteLLM Search API module.

* add BaseSearchConfig

* add _build_search_optional_params

* add search_testing

* add BaseSearchTest

* add TestPerplexitySearch

* fix BASE

* fix handler

* add search API

* add to init

* fix: working perplexity search API

* add _hidden_params to search

* add TAVILY to LlmProviders

* add TavilySearchConfig

* add TavilySearchConfig

* TestTavilySearch

* add tavily transform
2025-10-21 16:59:29 -07:00
wenhua
b0ccc35a9c
fix(ollama): Enhance chunk parsing for empty responses without 'thinking' and improve error logging (#13333) (#15717) 2025-10-21 16:59:01 -07:00
Ishaan Jaff
e1cb92862e
[Feat] Add def search() APIs for Web Search - Perplexity API (#15769)
* add BaseSearchConfig

* add BaseSearchConfig

* validate_environment

* fix handlers

* add PerplexitySearchConfig

* add PerplexitySearchConfig

* add LiteLLM Search API module.

* add BaseSearchConfig

* add _build_search_optional_params

* add search_testing

* add BaseSearchTest

* add TestPerplexitySearch

* fix BASE

* fix handler

* add search API

* add to init

* fix: working perplexity search API

* add _hidden_params to search
2025-10-21 16:58:51 -07:00
Ishaan Jaff
9135e748a0
[Feat ] /ocr - Add mode + Health check support for OCR models (#15767)
* get_mode_handlers

* use get_mode_handlers

* test_ahealth_check_ocr

* Add OCR mode to test models

* docs OCR Health Checks

* fix connection endpoint
2025-10-21 16:58:37 -07:00
Javier Garcia
b0a3a7c4fb
Add details in docs (#15721)
* Add details in docs

* add logic to set span attributes and unit tests

* Restore html files

* Remove html files

* Remove html files
2025-10-21 16:57:51 -07:00
Thomas Mildner
1cfc4624c3
[Feat] Add SENTRY_ENVIRONMENT configuration for Sentry integration (#15760)
* [Feat] Add SENTRY_ENVIRONMENT configuration for Sentry integration and corresponding tests

* [Refactor] Enhance test_sentry_environment by mocking sentry_sdk and improving environment handling

* [Fix] Update default SENTRY_ENVIRONMENT to 'production' and enhance test for Sentry integration

* [Fix] Update test_sentry_environment to verify correct handling of SENTRY_ENVIRONMENT values

* [Fix] Update test_sentry_environment to assert correct handling of production environment
2025-10-21 16:40:55 -07:00
Vinod Singh
d4aadda692
Auth Header Fix for MCP Tool Call (#15736)
* fixed the Auth header for MCP Tool Call

* Final fix for Auth header

* testcase for mcp_auth_header_extraction, insensitive_alias_matching, insensitive_servername_matching added
2025-10-21 13:58:03 -07:00
YutaSaito
d79bdd491f
feat: add GraySwan Guardrails support (#15756) 2025-10-21 11:13:50 -07:00
Talal
46d55bd92a
fix: Add response_type + PKCE parameters to OAuth authorization endpoint (#15720)
* fix: Add response_type parameter to OAuth authorization endpoint

Fixes #15684

OAuth providers like Google require the response_type parameter during
the authorization flow. This commit adds response_type=code to the
authorization redirect parameters, which is required by the OAuth 2.0
specification (RFC 6749 Section 4.1.1).

Changes:
- Added response_type=code to authorization params in discoverable_endpoints.py
- Added test coverage for the response_type parameter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix oauth flow by forwarding code_challenge and forwarding code_verifier

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 09:43:19 -07:00
Tom Haynes
98f1d63508
use correct otel logger, and normalise otel paths (#15645) 2025-10-21 09:16:03 -07:00
Ishaan Jaff
92335d991c
[Feat] Add Azure AVA (Speech AI) Cost Tracking (#15754)
* add azure/speech/ cost tracking

* test_azure_ava_tts_async

* add azure/speech to model cost map

* docs cost tracking

* docs tts AVA

* add azure/speech/azure-tts
2025-10-20 18:01:51 -07:00
Ishaan Jaffer
60fab591db rename test files 2025-10-20 18:00:17 -07:00
Ishaan Jaff
157739da01
[Bug]: Fix Incorrect status value in responses api with gemini (#15753)
* _map_chat_completion_finish_reason_to_responses_status

* test_transform_chat_completion_response_with_reasoning_content

* test_transform_chat_completion_response_output_item_status
2025-10-20 17:58:56 -07:00
Ishaan Jaff
73a23a6c78
[Feat] Add Azure AVA TTS integration (#15749)
* add AzureBaseIssueTokenHandler

* add BaseTextToSpeechConfig

* async_text_to_speech_handler

* add AzureAVATextToSpeechConfig

* add get_provider_text_to_speech_config

* add AzureAVATextToSpeechConfig

* fixes for base_llm_http_handler

* fix transform_text_to_speech_request

* test_azure_ava_tts_async

* test_azure_ava_tts_async

* fix TextToSpeechRequestData

* fix transform_text_to_speech_request

* add text_to_speech_handler in LLMHttpHandler

* remove old file

* fix transform_text_to_speech_request

* fix dispatch_text_to_speech

* fix azure TTS

* fix AVA TTS

* fix transform

* fix linting

* ci/cd - use one job for audio testing

* fix tests

* fix llm http handler debugging

* unit tests azure tts

* docs Azure speech

* docs fix

* docs azure AVA

* docs azure AVA

* fix handlers

* test_async_realtime_uses_max_size_parameter
2025-10-20 16:52:23 -07:00
Sameer Kankute
1fb798f81d
(Bug) Fix JSON serialization error in Helicone logging by removing OpenTelemetry span from metadata (#15728)
* remove span object from helicon metadata

* Add test
2025-10-20 08:53:22 -07:00
Timothée Lecomte
3ef9b2015a
feat: read from custom-llm-provider header (#15528) 2025-10-18 22:04:53 -07:00
Ishaan Jaff
f55745fc5e
[Fix] Forward anthropic-beta headers to Bedrock, VertexAI (#15700)
* [Fix] Forward anthropic-beta headers to Bedrock and other cross-provider scenarios (#15623)

* add_provider_specific_headers_to_request

* fix add_provider_specific_headers_to_request

* test_provider_specific_header_multi_provider

* test_provider_specific_header_in_request

---------

Co-authored-by: Jack Venberg <jack.venberg@rover.com>
2025-10-18 16:26:32 -07:00
Ishaan Jaff
6ab9b0af9f
[Fix] Anthropic cache_control incorrectly applied to all content items instead of last item only (#15699)
* fix: _safe_insert_cache_control_in_message

* test_anthropic_cache_control_hook_system_message

* docs prompt cache injection

* docs fix
2025-10-18 15:18:08 -07:00