Commit graph

26757 commits

Author SHA1 Message Date
Ishaan Jaffer
778e10119c test_azure_img_gen_health_check 2025-10-25 10:27:10 -07:00
Ishaan Jaffer
caa7da98b7 TestAzureAIOCR 2025-10-25 10:26:41 -07:00
Ishaan Jaffer
0bedf1c0a7 fix tests 2025-10-25 10:19:24 -07:00
Ishaan Jaffer
9ef54ab686 ci/cd run again 2025-10-25 10:09:47 -07:00
Ishaan Jaff
e4d5f00990
[Feat] New Guardrail - Dynamo AI Guardrail (#15920)
* add dynamo types

* fix Dynamo guard

* add dynamo guardrail

* add dynamo ai docs guard

* docs fix

* test dynamo

* test LASSO
2025-10-24 17:11:04 -07:00
Otavio Brito
68b8b66b2f
update vertex ai gemini costs (#15911) 2025-10-24 17:01:56 -07:00
Ishaan Jaffer
a07ed76f54 fix linting error 2025-10-24 14:45:38 -07:00
Ishaan Jaffer
6dee76c580 UI fix linting errors 2025-10-24 14:36:55 -07:00
Shadi
8a5ff84e49
fixed lasso import config, redis cluster hash tags for test keys (#15917) 2025-10-24 14:31:59 -07:00
superpoussin22
bd76d86b1c
Add mistral medium 3 and Codestral 2 on vertex (#15887) 2025-10-24 12:58:47 -07:00
Sam Chou
a173232f8c
Fix MLFlow tags - split request_tags into (key, val) if request_tag has colon (#15914)
* Fix mlflow tags - split request_tags into (key, val) if request_tag has colon

* Redundant name: tag_dict -> tags
2025-10-24 12:57:15 -07:00
Carlo Alberto Ferraris
8b1424166b
attempt to avoid/minimize deadlocks (#15281)
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-10-24 12:22:38 -07:00
Sameer Kankute
0f9996a4d0
Litellm sameer oct staging (#15806)
* Addd v2/chat support for cohere

* fix streaming

* Use v2_transformation for logging passthrough:

* Use v2_transformation for logging passthrough:

* Add test for checking if document and citation_options is getting passed

* Update the cohere model

* Add cost tracking for vertex ai passthrough batch jobs

* Add full passthrough support

* refactor code according to the comments

* Add passthrough handler

* remove invalid params

* Updated documentation

* Updated documentation

* Updated documentation

* Correct the import

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix documentation for get batch req (#15742)

* Add grounding info to responses API (#15737)

* Add grounding info to responses API

* fix lint errors

* Use typed objects for annotations

* Use typed objects for annotations

* fix mypy error

* Litellm fix json serialize alreting 2 (#15741)

* fix json serializable error for alerts

* Add test

* fix mypt errors

* fix mypt errors

* Add Qwen3 imported model support for AWS Bedrock (#15783)

* Add qwen imported model support

* fix mypy errors

* fix empty user message error (#15784)

* fix typed dict for list

* Add azure supported videos endpoint

* fix mapped tests

* add azure sora models to model map

* Add OpenAI video generation and content retrieval support (#15745)

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix typed dict for list

* fix mypy errors

* move directory

* make v2 chat default

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Revert "Add Azure Video Generation Support with Sora Integration"

* refactor videos repo

* add test

* Add azure openai videos support

* Add azure openai videos support

* Add router endpoint support for videos

* fix mypy error

* add azure models

* fix mapped test

* fix mypy error

* Add proxy router test

* Add proxy router test

* remove deprecated model name from tests

* fix import error

* fix import error

* Add gaurdrail integration in videos endpoint

* Add logging support for videos endpoint

* Add final documentation supporting videos integration

* fix model name and document input

* Update literals to avoid mypy errors

* Remove unused imports and print statements

* revert guardrail support for video generation and video remix

* revert guardrail support for video generation and video remix

* Fix failing mapped and llm translation tests
2025-10-24 12:17:22 -07:00
oroxenberg
c793bd5ba9
Lasso Security Guardrail: Add v3 API Support (#12452)
* 1. add v3 classify
2. add new classifix for masking
3. support same id for the conversation for pre and post
working with duplicates

* clean code, remove some debug and run tests

* update liter errors

* improvment for Code Organization, httpx Error Handling Specificity, Logging Improvements and Type

* transfer test test_lasso_guard_config to the new location

* Fix type hints and linting errors in lasso.py

- Add type: ignore for httpx module when None
- Fix return type issues in _handle_classification and _handle_masking
- Ensure masked_messages is not None before passing to _apply_masking_to_model_response
- Convert LassoResponse to dict for _log_masking_applied call
2025-10-24 11:03:58 -07:00
Sameer Kankute
c638f45213
Implement Bedrock Guardrail apply_guardrail endpoint support (#15892)
* Add bedrock support for apply gaurdrails

* Add bedrock support doc

* remove unused variable

* remove unused variable
2025-10-24 10:24:03 -07:00
Sameer Kankute
b9585b1db5
Update documentation for enable_caching_on_provider_specific_optional_params (#15885) 2025-10-24 10:22:27 -07:00
Alexsander Hamir
9338727960
feat(proxy): support absolute RPM/TPM in priority_reservation (#15813)
* feat(proxy): support absolute RPM/TPM in priority_reservation

Allow priority reservations as absolute values instead of percentages:
- Float: {'prod': 0.75} (75%, existing)
- RPM: {'prod': {'type': 'rpm', 'value': 750}}
- TPM: {'prod': {'type': 'tpm', 'value': 750000}}

Added _convert_to_percent() that converts absolute values to percentages
based on model capacity. Fully backward compatible.

* feat(types): convert priority_reservation Dict to TypedDict

Add PriorityReservationDict TypedDict to replace generic Dict type in priority_reservation configuration.

Changes:
- Add PriorityReservationDict to litellm/types/utils.py
- Update convert_priority_to_percent() signature in rate_limiter_utils.py
- Update litellm.priority_reservation type annotation in __init__.py

Improves IDE autocomplete and type checking for priority reservation configs.

* docs: update dynamic rate limiter priority reservation docs
2025-10-23 18:30:36 -07:00
mubashirosmani
c5fee97850
docs: add OpenAI responses api (#15868)
* docs: add tip openai page

* added responses api

---------

Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
2025-10-23 18:25:59 -07:00
mubashirosmani
09c1ad190e
docs: add tip openai page (#15866)
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
2025-10-23 18:25:25 -07:00
Ishaan Jaffer
ea8a604188 fix to .debug 2025-10-23 18:08:56 -07:00
Ishaan Jaffer
f4e98f715a fix linting 2025-10-23 18:07:28 -07:00
Ishaan Jaffer
76d658eee8 fix linting 2025-10-23 18:05:30 -07:00
Ishaan Jaff
5de912375c
[Feat] UI - Add logos for search providers (#15872)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db

* add UI friendly name for search providers

* add ui friendly name for search providers

* add providers available

* working layout

* better layout

* clean add search tool

* update_router_search_tools

* fix remove in memory registry, since router is in mem store

* allow testing search tool connection

* clean create search tool

* add test_search_tool_connection

* fix: _init_search_tools_in_db

* add searchToolQueryCall

* fix icon

* clean tester

* add parallel ai logo

* add exa ai logo

* add google PSE logo

* add tavily logo

* add dataforseo + perplexity

* add parallel ai logo

* UI show logos for search
2025-10-23 18:00:40 -07:00
Ishaan Jaff
fc9aba279e
[Feat] UI - Search Tools, allow adding search tools on UI + testing search (#15871)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db

* add UI friendly name for search providers

* add ui friendly name for search providers

* add providers available

* working layout

* better layout

* clean add search tool

* update_router_search_tools

* fix remove in memory registry, since router is in mem store

* allow testing search tool connection

* clean create search tool

* add test_search_tool_connection

* fix: _init_search_tools_in_db

* add searchToolQueryCall

* fix icon

* clean tester
2025-10-23 17:59:29 -07:00
Ishaan Jaff
d8ea1665c7
[Feat] - [Backend] Search APIs - Allow storing configured Search APIs in DB (#15862)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db
2025-10-23 17:57:49 -07:00
Byron Grogan
bf47c25de0
[fix] Pass user-defined headers and extra_headers to image-edit calls (#15811) 2025-10-23 15:08:10 -07:00
Krish Dholakia
e0c4baf66f
fix(ui/): fix routing for custom server root path (#15701)
* fix(ui/): fix routing for custom server root path

* fix: fix eslint errors
2025-10-23 13:59:29 -07:00
Jacob Linney
0644c20604
fix(vertex-ai): cost tracking for search spend (#15859) 2025-10-23 13:29:10 -07:00
Ishaan Jaff
6d947d7326
[Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855)
* fix filter_out_litellm_params

* use filter_out_litellm_params

* test_filter_out_litellm_params
2025-10-23 11:56:30 -07:00
Ishaan Jaff
511d435f6f
[Bug Fix]: Hooks broken on /bedrock passthrough due to missing metadata (#15849)
* refactor handle_bedrock_passthrough_router_model

* test_bedrock_router_passthrough_metadata_initialization
2025-10-23 11:52:37 -07:00
Alexsander Hamir
8c5118195d
fix: replace deprecated gemini-1.5-pro-preview-0514 with gemini-2.5-flash-lite in function calling test (#15852) 2025-10-23 11:48:58 -07:00
Ishaan Jaffer
d53a8b3e8d Revert "fix(responses-api): simplify reasoning item handling for gpt-5-codex (#15815)"
This reverts commit cc63cf28ff.
2025-10-23 11:17:36 -07:00
Ishaan Jaffer
ae7b13550e test_models_by_provider 2025-10-23 09:10:41 -07:00
Ishaan Jaffer
8e65f99ebb test fix TTS 2025-10-23 08:50:54 -07:00
Ishaan Jaffer
5498a8b099 test_ensure_initialize_azure_sdk_client_always_used 2025-10-23 08:50:39 -07:00
Ishaan Jaffer
74b8a1dbdf test_aaamodel_prices_and_context_window_json_is_valid 2025-10-23 08:47:08 -07:00
Ishaan Jaffer
bfe4167183 bump: version 1.78.7 → 1.78.8 2025-10-23 08:42:03 -07:00
Alexsander Hamir
cc63cf28ff
fix(responses-api): simplify reasoning item handling for gpt-5-codex (#15815)
Simplifies the _handle_reasoning_item method to only filter out None values
without attempting to construct ResponseReasoningItem objects. This approach:
- Avoids mutation of encrypted_content in reasoning items
- Fixes compatibility with gpt-5-codex and Codex CLI
- Removes dependency on ResponseReasoningItem import
- Simplifies the code by removing complex fallback logic

The previous approach was trying to construct ResponseReasoningItem objects
with required fields (id, summary), which caused issues with encrypted
payloads. The new approach simply filters None values while preserving
all other data intact.

Issue: https://github.com/BerriAI/litellm/issues/13484
Reported-by: rajpathak (OpenAI)
Tested-by: rajpathak (OpenAI Codex CLI)
2025-10-23 08:41:37 -07:00
Krish Dholakia
573306f3cd
(feat) Vector Stores: support Vertex AI Search API as vector store through LiteLLM (#15781)
* feat(vector_stores/): initial commit adding Vertex AI Search API support for litellm

new vector store provider

* feat(vector_store/): use vector store id for vertex ai search api

* fix: transformation.py

cleanup

* fix: implement abstract function

* fix: fix linting error

* fix: main.py

fix check
2025-10-22 18:56:36 -07:00
Ishaan Jaff
3e4b5ef3a5
[Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI (#15821)
* add search cost tracking

* add cost tracking for tavily tiers

* add search to call types

* add search_provider_cost_per_query

* add cost tracking for search APIs

* add cost tracking search APIs

* docs cost tracking search

* docs search

* fix linting
2025-10-22 17:29:09 -07:00
Ishaan Jaff
143e314dda
[Feat] Add Google PSE Search Provider (#15816)
* docs google PSE

* add SearchProviders

* add search providers

* add PSE search

* add SearchProviders

* get_provider_search_config

* add Search

* init Search

* add get_http_method on BaseSearch

* fixes for Google PSE

* TestGooglePSESearch
2025-10-22 16:01:28 -07:00
Ishaan Jaff
ad62a6d3d1
[Feat] Add DataforSEO Search API (#15817)
* docs google PSE

* add SearchProviders

* add search providers

* add PSE search

* add SearchProviders

* get_provider_search_config

* add Search

* init Search

* add get_http_method on BaseSearch

* fixes for Google PSE

* TestGooglePSESearch

* add DATAFORSEO

* add DataForSEOSearchConfig

* TestDataForSEOSearch

* add DataForSEO

* fix base transform

* fix search

* fix dataforSEO

* docs fix

* fix linting

* fix linting
2025-10-22 16:00:40 -07:00
nuernber
799a2b624a
use proper bedrock model name in health check (#15808) 2025-10-22 15:24:57 -07:00
Ishaan Jaff
ec6a5ffa2d
[Fix] Azure AI Speech - Ensure voice is mapped from request body -> SSML body , allow sending role and style (#15810)
* update map_openai_params

* fix update voice transform

* fix text_to_speech_provider_config

* test_azure_ava_tts_with_custom_voice

* test Azure AVA style, role sent

* _build_express_as_element

* docs custom params

* build LANG

* fix transform

* fix transform

* fix speech

* docs update

* docs azure ai speech
2025-10-22 14:41:11 -07:00
Ishaan Jaff
d91efa7a7b
[Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure (#15804)
* add ErrorEventError nested field

* test_openai_responses_api_token_limit_error

* test_openai_responses_api_token_limit_error
2025-10-22 14:18:46 -07:00
Sameer Kankute
2ab2d15efc
Fix Token Spend is under budget for passthrough (#15805) 2025-10-22 10:55:22 -07:00
Ishaan Jaffer
abe67df95e refactor large func 2025-10-22 10:04:52 -07:00
Ishaan Jaffer
57a2ec3beb fix: _extract_fields_recursive 2025-10-22 09:37:29 -07:00
Roman G
eac3cba44f
Support for embeddings_by_type Response Format in Bedrock Cohere Embed v1 (#15707)
* feat(cohere): Enhance embedding transformation to support Bedrock's embeddings by type

* test(cohere): Add unit tests for embedding transformation responses
2025-10-22 09:32:11 -07:00
Ishaan Jaffer
e80bba83e3 test fix 2025-10-22 09:29:04 -07:00