Commit graph

6027 commits

Author SHA1 Message Date
Ishaan Jaffer
3dffb6bd40 test fixes 2025-10-25 11:20:09 -07:00
Ishaan Jaffer
ec6c166548 _add_azure_related_dynamic_params 2025-10-25 11:11:36 -07:00
Ishaan Jaffer
eb425550a2 test responses API fixes 2025-10-25 10:58:39 -07:00
Ishaan Jaffer
77a12f1a87 test audio fixes 2025-10-25 10:46:08 -07:00
Ishaan Jaffer
1feb13e76a test_audio_speech_litellm 2025-10-25 10:42:47 -07:00
Ishaan Jaffer
d8e5938f54 test_azure_img_gen_health_check 2025-10-25 10:35:47 -07:00
Ishaan Jaffer
778e10119c test_azure_img_gen_health_check 2025-10-25 10:27:10 -07:00
Ishaan Jaffer
caa7da98b7 TestAzureAIOCR 2025-10-25 10:26:41 -07:00
Ishaan Jaffer
0bedf1c0a7 fix tests 2025-10-25 10:19:24 -07:00
Ishaan Jaff
e4d5f00990
[Feat] New Guardrail - Dynamo AI Guardrail (#15920)
* add dynamo types

* fix Dynamo guard

* add dynamo guardrail

* add dynamo ai docs guard

* docs fix

* test dynamo

* test LASSO
2025-10-24 17:11:04 -07:00
Shadi
8a5ff84e49
fixed lasso import config, redis cluster hash tags for test keys (#15917) 2025-10-24 14:31:59 -07:00
Sam Chou
a173232f8c
Fix MLFlow tags - split request_tags into (key, val) if request_tag has colon (#15914)
* Fix mlflow tags - split request_tags into (key, val) if request_tag has colon

* Redundant name: tag_dict -> tags
2025-10-24 12:57:15 -07:00
Carlo Alberto Ferraris
8b1424166b
attempt to avoid/minimize deadlocks (#15281)
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-10-24 12:22:38 -07:00
Sameer Kankute
0f9996a4d0
Litellm sameer oct staging (#15806)
* Addd v2/chat support for cohere

* fix streaming

* Use v2_transformation for logging passthrough:

* Use v2_transformation for logging passthrough:

* Add test for checking if document and citation_options is getting passed

* Update the cohere model

* Add cost tracking for vertex ai passthrough batch jobs

* Add full passthrough support

* refactor code according to the comments

* Add passthrough handler

* remove invalid params

* Updated documentation

* Updated documentation

* Updated documentation

* Correct the import

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix documentation for get batch req (#15742)

* Add grounding info to responses API (#15737)

* Add grounding info to responses API

* fix lint errors

* Use typed objects for annotations

* Use typed objects for annotations

* fix mypy error

* Litellm fix json serialize alreting 2 (#15741)

* fix json serializable error for alerts

* Add test

* fix mypt errors

* fix mypt errors

* Add Qwen3 imported model support for AWS Bedrock (#15783)

* Add qwen imported model support

* fix mypy errors

* fix empty user message error (#15784)

* fix typed dict for list

* Add azure supported videos endpoint

* fix mapped tests

* add azure sora models to model map

* Add OpenAI video generation and content retrieval support (#15745)

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix typed dict for list

* fix mypy errors

* move directory

* make v2 chat default

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Revert "Add Azure Video Generation Support with Sora Integration"

* refactor videos repo

* add test

* Add azure openai videos support

* Add azure openai videos support

* Add router endpoint support for videos

* fix mypy error

* add azure models

* fix mapped test

* fix mypy error

* Add proxy router test

* Add proxy router test

* remove deprecated model name from tests

* fix import error

* fix import error

* Add gaurdrail integration in videos endpoint

* Add logging support for videos endpoint

* Add final documentation supporting videos integration

* fix model name and document input

* Update literals to avoid mypy errors

* Remove unused imports and print statements

* revert guardrail support for video generation and video remix

* revert guardrail support for video generation and video remix

* Fix failing mapped and llm translation tests
2025-10-24 12:17:22 -07:00
oroxenberg
c793bd5ba9
Lasso Security Guardrail: Add v3 API Support (#12452)
* 1. add v3 classify
2. add new classifix for masking
3. support same id for the conversation for pre and post
working with duplicates

* clean code, remove some debug and run tests

* update liter errors

* improvment for Code Organization, httpx Error Handling Specificity, Logging Improvements and Type

* transfer test test_lasso_guard_config to the new location

* Fix type hints and linting errors in lasso.py

- Add type: ignore for httpx module when None
- Fix return type issues in _handle_classification and _handle_masking
- Ensure masked_messages is not None before passing to _apply_masking_to_model_response
- Convert LassoResponse to dict for _log_masking_applied call
2025-10-24 11:03:58 -07:00
Sameer Kankute
c638f45213
Implement Bedrock Guardrail apply_guardrail endpoint support (#15892)
* Add bedrock support for apply gaurdrails

* Add bedrock support doc

* remove unused variable

* remove unused variable
2025-10-24 10:24:03 -07:00
Byron Grogan
bf47c25de0
[fix] Pass user-defined headers and extra_headers to image-edit calls (#15811) 2025-10-23 15:08:10 -07:00
Krish Dholakia
e0c4baf66f
fix(ui/): fix routing for custom server root path (#15701)
* fix(ui/): fix routing for custom server root path

* fix: fix eslint errors
2025-10-23 13:59:29 -07:00
Jacob Linney
0644c20604
fix(vertex-ai): cost tracking for search spend (#15859) 2025-10-23 13:29:10 -07:00
Ishaan Jaff
6d947d7326
[Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855)
* fix filter_out_litellm_params

* use filter_out_litellm_params

* test_filter_out_litellm_params
2025-10-23 11:56:30 -07:00
Ishaan Jaff
511d435f6f
[Bug Fix]: Hooks broken on /bedrock passthrough due to missing metadata (#15849)
* refactor handle_bedrock_passthrough_router_model

* test_bedrock_router_passthrough_metadata_initialization
2025-10-23 11:52:37 -07:00
Alexsander Hamir
8c5118195d
fix: replace deprecated gemini-1.5-pro-preview-0514 with gemini-2.5-flash-lite in function calling test (#15852) 2025-10-23 11:48:58 -07:00
Ishaan Jaffer
ae7b13550e test_models_by_provider 2025-10-23 09:10:41 -07:00
Ishaan Jaffer
8e65f99ebb test fix TTS 2025-10-23 08:50:54 -07:00
Ishaan Jaffer
5498a8b099 test_ensure_initialize_azure_sdk_client_always_used 2025-10-23 08:50:39 -07:00
Ishaan Jaffer
74b8a1dbdf test_aaamodel_prices_and_context_window_json_is_valid 2025-10-23 08:47:08 -07:00
Krish Dholakia
573306f3cd
(feat) Vector Stores: support Vertex AI Search API as vector store through LiteLLM (#15781)
* feat(vector_stores/): initial commit adding Vertex AI Search API support for litellm

new vector store provider

* feat(vector_store/): use vector store id for vertex ai search api

* fix: transformation.py

cleanup

* fix: implement abstract function

* fix: fix linting error

* fix: main.py

fix check
2025-10-22 18:56:36 -07:00
Ishaan Jaff
3e4b5ef3a5
[Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI (#15821)
* add search cost tracking

* add cost tracking for tavily tiers

* add search to call types

* add search_provider_cost_per_query

* add cost tracking for search APIs

* add cost tracking search APIs

* docs cost tracking search

* docs search

* fix linting
2025-10-22 17:29:09 -07:00
Ishaan Jaff
ad62a6d3d1
[Feat] Add DataforSEO Search API (#15817)
* docs google PSE

* add SearchProviders

* add search providers

* add PSE search

* add SearchProviders

* get_provider_search_config

* add Search

* init Search

* add get_http_method on BaseSearch

* fixes for Google PSE

* TestGooglePSESearch

* add DATAFORSEO

* add DataForSEOSearchConfig

* TestDataForSEOSearch

* add DataForSEO

* fix base transform

* fix search

* fix dataforSEO

* docs fix

* fix linting

* fix linting
2025-10-22 16:00:40 -07:00
nuernber
799a2b624a
use proper bedrock model name in health check (#15808) 2025-10-22 15:24:57 -07:00
Ishaan Jaff
ec6a5ffa2d
[Fix] Azure AI Speech - Ensure voice is mapped from request body -> SSML body , allow sending role and style (#15810)
* update map_openai_params

* fix update voice transform

* fix text_to_speech_provider_config

* test_azure_ava_tts_with_custom_voice

* test Azure AVA style, role sent

* _build_express_as_element

* docs custom params

* build LANG

* fix transform

* fix transform

* fix speech

* docs update

* docs azure ai speech
2025-10-22 14:41:11 -07:00
Ishaan Jaff
d91efa7a7b
[Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure (#15804)
* add ErrorEventError nested field

* test_openai_responses_api_token_limit_error

* test_openai_responses_api_token_limit_error
2025-10-22 14:18:46 -07:00
Sameer Kankute
2ab2d15efc
Fix Token Spend is under budget for passthrough (#15805) 2025-10-22 10:55:22 -07:00
Roman G
eac3cba44f
Support for embeddings_by_type Response Format in Bedrock Cohere Embed v1 (#15707)
* feat(cohere): Enhance embedding transformation to support Bedrock's embeddings by type

* test(cohere): Add unit tests for embedding transformation responses
2025-10-22 09:32:11 -07:00
Ishaan Jaffer
e80bba83e3 test fix 2025-10-22 09:29:04 -07:00
soo-jin.kim
03e1d93199
fix: Apply max_connections configuration to Redis async client (#15797)
* fix: Apply max_connections configuration to Redis async client

- Add max_connections to available Redis cluster kwargs
- Add connection_pool parameter to get_redis_async_client()
- Pass connection_pool to Redis client if provided
- Prevents Redis connection exhaustion under high load

* test: Add tests for Redis max_connections feature

- Test max_connections is included in cluster kwargs
- Test connection_pool parameter is properly passed to async client
- Test async client works without connection_pool parameter

All 3 tests pass successfully
2025-10-22 09:19:08 -07:00
soo-jin.kim
8050995dbb
fix: Rename configured_cold_storage_logger to cold_storage_custom_logger (#15798)
- Change variable name in litellm/__init__.py from configured_cold_storage_logger to cold_storage_custom_logger
- Update all references across the codebase to use the new variable name
- This fixes silent failure of cold storage logging due to variable name mismatch
- Configuration files use cold_storage_custom_logger, code should match

Files updated:
- litellm/__init__.py
- litellm/litellm_core_utils/litellm_logging.py
- litellm/proxy/spend_tracking/cold_storage_handler.py
- litellm/responses/litellm_completion_transformation/session_handler.py
- tests/test_litellm/litellm_core_utils/test_litellm_logging.py
- tests/test_litellm/responses/litellm_completion_transformation/test_session_handler.py
2025-10-22 09:17:08 -07:00
Anthony Ivan
5f7a6b49eb
Feat: Allow prompt caching to be used for Anthropic Claude on Databricks (#15801) 2025-10-22 09:11:04 -07:00
Sameer Kankute
44495c0117
fix encrypted content error (#15782) 2025-10-21 23:29:48 -07:00
Ishaan Jaff
f5a80110c1
[Feat] Add /search endpoint on LiteLLM Gateway (#15780)
* add SearchProvider

* add SearchToolTypedDict

* add search

* add SearchAPIRouter

* working router level search

* add search to allowed llm / ocr routes

* feat: add search_router

* add routing + proxy for search APIs

* /v1/search/{search_tool_name}

* fix search routing

* feat: parse_search_tools

* clean up sidebar

* docs fix

* router tests for search tools

* docs fix
2025-10-21 19:05:20 -07:00
Ishaan Jaffer
d9b85ab276 fix: rename search_provider 2025-10-21 17:42:18 -07:00
wangjifeng
8cbaec0310
feat: Add imageConfig parameter for gemini-2.5-flash-image (#15530)
* Add imageConfig parameter support for Vertex AI to enable gemini-2.5-flash-image model requirements

* Add test for imageConfig parameter support in Vertex AI Gemini transformation
2025-10-21 17:08:30 -07:00
Ishaan Jaff
7b939b4558
[Feat] Add EXA AI Search API to LiteLLM (#15774)
* add BaseSearchConfig

* add BaseSearchConfig

* validate_environment

* fix handlers

* add PerplexitySearchConfig

* add PerplexitySearchConfig

* add LiteLLM Search API module.

* add BaseSearchConfig

* add _build_search_optional_params

* add search_testing

* add BaseSearchTest

* add TestPerplexitySearch

* fix BASE

* fix handler

* add search API

* add to init

* fix: working perplexity search API

* add _hidden_params to search

* add TAVILY to LlmProviders

* add TavilySearchConfig

* add TavilySearchConfig

* TestTavilySearch

* add tavily transform

* TestParallelAISearch

* add LlmProviders

* add ParallelAISearchConfig

* add ParallelAISearchConfig

* ParallelAISearchConfig

* add EXA AI Search API

* add ExaAISearchConfig

* TestExaAISearch

* add get_supported_perplexity_optional_params

* add Exa AI Search API

* add transform_search_request

* add ExaAISearchConfig

* fix linting errors

* transform_search_request
2025-10-21 17:06:23 -07:00
Ishaan Jaff
208f76f8ad
[Feat] Add Parallel AI - Search API (#15772)
* add BaseSearchConfig

* add BaseSearchConfig

* validate_environment

* fix handlers

* add PerplexitySearchConfig

* add PerplexitySearchConfig

* add LiteLLM Search API module.

* add BaseSearchConfig

* add _build_search_optional_params

* add search_testing

* add BaseSearchTest

* add TestPerplexitySearch

* fix BASE

* fix handler

* add search API

* add to init

* fix: working perplexity search API

* add _hidden_params to search

* add TAVILY to LlmProviders

* add TavilySearchConfig

* add TavilySearchConfig

* TestTavilySearch

* add tavily transform

* TestParallelAISearch

* add LlmProviders

* add ParallelAISearchConfig

* add ParallelAISearchConfig

* ParallelAISearchConfig
2025-10-21 17:00:05 -07:00
Ishaan Jaff
b9f3f9fb79
[Feat] Add Tavily Search API (#15770)
* add BaseSearchConfig

* add BaseSearchConfig

* validate_environment

* fix handlers

* add PerplexitySearchConfig

* add PerplexitySearchConfig

* add LiteLLM Search API module.

* add BaseSearchConfig

* add _build_search_optional_params

* add search_testing

* add BaseSearchTest

* add TestPerplexitySearch

* fix BASE

* fix handler

* add search API

* add to init

* fix: working perplexity search API

* add _hidden_params to search

* add TAVILY to LlmProviders

* add TavilySearchConfig

* add TavilySearchConfig

* TestTavilySearch

* add tavily transform
2025-10-21 16:59:29 -07:00
wenhua
b0ccc35a9c
fix(ollama): Enhance chunk parsing for empty responses without 'thinking' and improve error logging (#13333) (#15717) 2025-10-21 16:59:01 -07:00
Ishaan Jaff
e1cb92862e
[Feat] Add def search() APIs for Web Search - Perplexity API (#15769)
* add BaseSearchConfig

* add BaseSearchConfig

* validate_environment

* fix handlers

* add PerplexitySearchConfig

* add PerplexitySearchConfig

* add LiteLLM Search API module.

* add BaseSearchConfig

* add _build_search_optional_params

* add search_testing

* add BaseSearchTest

* add TestPerplexitySearch

* fix BASE

* fix handler

* add search API

* add to init

* fix: working perplexity search API

* add _hidden_params to search
2025-10-21 16:58:51 -07:00
Ishaan Jaff
9135e748a0
[Feat ] /ocr - Add mode + Health check support for OCR models (#15767)
* get_mode_handlers

* use get_mode_handlers

* test_ahealth_check_ocr

* Add OCR mode to test models

* docs OCR Health Checks

* fix connection endpoint
2025-10-21 16:58:37 -07:00
Javier Garcia
b0a3a7c4fb
Add details in docs (#15721)
* Add details in docs

* add logic to set span attributes and unit tests

* Restore html files

* Remove html files

* Remove html files
2025-10-21 16:57:51 -07:00
Thomas Mildner
1cfc4624c3
[Feat] Add SENTRY_ENVIRONMENT configuration for Sentry integration (#15760)
* [Feat] Add SENTRY_ENVIRONMENT configuration for Sentry integration and corresponding tests

* [Refactor] Enhance test_sentry_environment by mocking sentry_sdk and improving environment handling

* [Fix] Update default SENTRY_ENVIRONMENT to 'production' and enhance test for Sentry integration

* [Fix] Update test_sentry_environment to verify correct handling of SENTRY_ENVIRONMENT values

* [Fix] Update test_sentry_environment to assert correct handling of production environment
2025-10-21 16:40:55 -07:00