Commit graph

26782 commits

Author SHA1 Message Date
Ishaan Jaffer
cff70ece5a test_azure_astreaming_and_function_calling 2025-10-25 12:18:53 -07:00
Ishaan Jaffer
2b16731273 TestAzureOpenAIO3Mini 2025-10-25 12:13:50 -07:00
Krish Dholakia
1543891763
Responses API - support tags in metadata
* fix(ui/): fix routing for custom server root path

* fix: fix eslint errors

* fix(vector_store_pre_call_hook.py): Fix https://github.com/BerriAI/litellm/issues/15724

* fix(responses/main.py): have 'tags' work across metadata + litellm_metadata

* fix: add unit testing
2025-10-25 12:07:54 -07:00
Krish Dholakia
f8d6a6edb9
fix(managed_files.py): don't raise error if managed object is not found + (Feat) Azure AI - Search Vector Stores + (Fix) Batches - “User default_user_id does not have access to the object” when object not in db + (fix) Vector Stores - show config.yaml vector stores on UI (#15873)
* fix(managed_files.py): don't raise error if managed object is not found

* feat(vector_stores): add azure ai search vector store support

Enables direct querying a vector store on azure

* fix(azure/vector_stores): working azure ai search api vector stores

allows azure direct querying on vector stores

* test: update env vars

* docs(docs/): document new azure ai vector store search

* docs(azure_ai_vector_stores.md): add table

* docs: clarify support for 'create' vector stores

* fix(vector_stores/endpoints.py): Fixes https://github.com/BerriAI/litellm/issues/14606

* fix: fix linting errors
2025-10-25 12:06:24 -07:00
Ishaan Jaffer
214c10f6ef test_completion_cost_databricks_embedding 2025-10-25 11:47:03 -07:00
Ishaan Jaffer
d47555760e test_azure_transcribe_model_mapping 2025-10-25 11:42:34 -07:00
Ishaan Jaffer
8c8e53c7ac whisper test fix 2025-10-25 11:33:02 -07:00
Ishaan Jaffer
e227e8c8a0 mv test_whisper 2025-10-25 11:31:55 -07:00
Ishaan Jaffer
74106589d0 test_completion_azure_ai_gpt_4o_with_flexible_api_base 2025-10-25 11:30:51 -07:00
Ishaan Jaffer
c06098c351 test_databricks_embeddings 2025-10-25 11:28:21 -07:00
Ishaan Jaffer
a3febef431 test_azure_ai_request_format 2025-10-25 11:23:02 -07:00
Ishaan Jaffer
3dffb6bd40 test fixes 2025-10-25 11:20:09 -07:00
Ishaan Jaffer
c0555c84c0 1.78.0-stable 2025-10-25 11:14:59 -07:00
Krish Dholakia
ddacaf6c32
(feat) Organizations: allow org admins to create teams on UI + (feat) IBM Guardrails (#15924)
* fix(oldteams.tsx): allow org admin to create team on ui

* fix(oldteams.tsx): show org admin a dropdown of allowed orgs for team creation

* docs(access_control.md): cleanup doc

* feat(ibm_guardrails/): initial commit adding support for ibm guardrails on litellm

allows user to use self-hosted ibm guardrails

* feat(ibm_detector.py): working detector

* docs(ibm_guardrails.md): document new ibm guardrails

* fix: fix linting errors
2025-10-25 11:13:39 -07:00
Ishaan Jaffer
ec6c166548 _add_azure_related_dynamic_params 2025-10-25 11:11:36 -07:00
Ishaan Jaffer
e9466dc27e pt tests fix 2025-10-25 11:08:33 -07:00
Ishaan Jaffer
bbd9d71946 remove bloat file 2025-10-25 11:04:52 -07:00
Ishaan Jaffer
6ef902921d remove log.txt 2025-10-25 11:03:56 -07:00
Ishaan Jaffer
0b41432aeb remove bloat files 2025-10-25 11:03:19 -07:00
Ishaan Jaffer
92186406f5 remove bloat files 2025-10-25 11:02:05 -07:00
Ishaan Jaffer
79258d55f5 ui fix 2025-10-25 11:01:42 -07:00
Ishaan Jaffer
eb425550a2 test responses API fixes 2025-10-25 10:58:39 -07:00
Ishaan Jaffer
77a12f1a87 test audio fixes 2025-10-25 10:46:08 -07:00
Ishaan Jaffer
1feb13e76a test_audio_speech_litellm 2025-10-25 10:42:47 -07:00
Ishaan Jaffer
d8e5938f54 test_azure_img_gen_health_check 2025-10-25 10:35:47 -07:00
Ishaan Jaffer
778e10119c test_azure_img_gen_health_check 2025-10-25 10:27:10 -07:00
Ishaan Jaffer
caa7da98b7 TestAzureAIOCR 2025-10-25 10:26:41 -07:00
Ishaan Jaffer
0bedf1c0a7 fix tests 2025-10-25 10:19:24 -07:00
Ishaan Jaffer
9ef54ab686 ci/cd run again 2025-10-25 10:09:47 -07:00
Ishaan Jaff
e4d5f00990
[Feat] New Guardrail - Dynamo AI Guardrail (#15920)
* add dynamo types

* fix Dynamo guard

* add dynamo guardrail

* add dynamo ai docs guard

* docs fix

* test dynamo

* test LASSO
2025-10-24 17:11:04 -07:00
Otavio Brito
68b8b66b2f
update vertex ai gemini costs (#15911) 2025-10-24 17:01:56 -07:00
Ishaan Jaffer
a07ed76f54 fix linting error 2025-10-24 14:45:38 -07:00
Ishaan Jaffer
6dee76c580 UI fix linting errors 2025-10-24 14:36:55 -07:00
Shadi
8a5ff84e49
fixed lasso import config, redis cluster hash tags for test keys (#15917) 2025-10-24 14:31:59 -07:00
superpoussin22
bd76d86b1c
Add mistral medium 3 and Codestral 2 on vertex (#15887) 2025-10-24 12:58:47 -07:00
Sam Chou
a173232f8c
Fix MLFlow tags - split request_tags into (key, val) if request_tag has colon (#15914)
* Fix mlflow tags - split request_tags into (key, val) if request_tag has colon

* Redundant name: tag_dict -> tags
2025-10-24 12:57:15 -07:00
Carlo Alberto Ferraris
8b1424166b
attempt to avoid/minimize deadlocks (#15281)
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-10-24 12:22:38 -07:00
Sameer Kankute
0f9996a4d0
Litellm sameer oct staging (#15806)
* Addd v2/chat support for cohere

* fix streaming

* Use v2_transformation for logging passthrough:

* Use v2_transformation for logging passthrough:

* Add test for checking if document and citation_options is getting passed

* Update the cohere model

* Add cost tracking for vertex ai passthrough batch jobs

* Add full passthrough support

* refactor code according to the comments

* Add passthrough handler

* remove invalid params

* Updated documentation

* Updated documentation

* Updated documentation

* Correct the import

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix documentation for get batch req (#15742)

* Add grounding info to responses API (#15737)

* Add grounding info to responses API

* fix lint errors

* Use typed objects for annotations

* Use typed objects for annotations

* fix mypy error

* Litellm fix json serialize alreting 2 (#15741)

* fix json serializable error for alerts

* Add test

* fix mypt errors

* fix mypt errors

* Add Qwen3 imported model support for AWS Bedrock (#15783)

* Add qwen imported model support

* fix mypy errors

* fix empty user message error (#15784)

* fix typed dict for list

* Add azure supported videos endpoint

* fix mapped tests

* add azure sora models to model map

* Add OpenAI video generation and content retrieval support (#15745)

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix typed dict for list

* fix mypy errors

* move directory

* make v2 chat default

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Revert "Add Azure Video Generation Support with Sora Integration"

* refactor videos repo

* add test

* Add azure openai videos support

* Add azure openai videos support

* Add router endpoint support for videos

* fix mypy error

* add azure models

* fix mapped test

* fix mypy error

* Add proxy router test

* Add proxy router test

* remove deprecated model name from tests

* fix import error

* fix import error

* Add gaurdrail integration in videos endpoint

* Add logging support for videos endpoint

* Add final documentation supporting videos integration

* fix model name and document input

* Update literals to avoid mypy errors

* Remove unused imports and print statements

* revert guardrail support for video generation and video remix

* revert guardrail support for video generation and video remix

* Fix failing mapped and llm translation tests
2025-10-24 12:17:22 -07:00
oroxenberg
c793bd5ba9
Lasso Security Guardrail: Add v3 API Support (#12452)
* 1. add v3 classify
2. add new classifix for masking
3. support same id for the conversation for pre and post
working with duplicates

* clean code, remove some debug and run tests

* update liter errors

* improvment for Code Organization, httpx Error Handling Specificity, Logging Improvements and Type

* transfer test test_lasso_guard_config to the new location

* Fix type hints and linting errors in lasso.py

- Add type: ignore for httpx module when None
- Fix return type issues in _handle_classification and _handle_masking
- Ensure masked_messages is not None before passing to _apply_masking_to_model_response
- Convert LassoResponse to dict for _log_masking_applied call
2025-10-24 11:03:58 -07:00
Sameer Kankute
c638f45213
Implement Bedrock Guardrail apply_guardrail endpoint support (#15892)
* Add bedrock support for apply gaurdrails

* Add bedrock support doc

* remove unused variable

* remove unused variable
2025-10-24 10:24:03 -07:00
Sameer Kankute
b9585b1db5
Update documentation for enable_caching_on_provider_specific_optional_params (#15885) 2025-10-24 10:22:27 -07:00
Alexsander Hamir
9338727960
feat(proxy): support absolute RPM/TPM in priority_reservation (#15813)
* feat(proxy): support absolute RPM/TPM in priority_reservation

Allow priority reservations as absolute values instead of percentages:
- Float: {'prod': 0.75} (75%, existing)
- RPM: {'prod': {'type': 'rpm', 'value': 750}}
- TPM: {'prod': {'type': 'tpm', 'value': 750000}}

Added _convert_to_percent() that converts absolute values to percentages
based on model capacity. Fully backward compatible.

* feat(types): convert priority_reservation Dict to TypedDict

Add PriorityReservationDict TypedDict to replace generic Dict type in priority_reservation configuration.

Changes:
- Add PriorityReservationDict to litellm/types/utils.py
- Update convert_priority_to_percent() signature in rate_limiter_utils.py
- Update litellm.priority_reservation type annotation in __init__.py

Improves IDE autocomplete and type checking for priority reservation configs.

* docs: update dynamic rate limiter priority reservation docs
2025-10-23 18:30:36 -07:00
mubashirosmani
c5fee97850
docs: add OpenAI responses api (#15868)
* docs: add tip openai page

* added responses api

---------

Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
2025-10-23 18:25:59 -07:00
mubashirosmani
09c1ad190e
docs: add tip openai page (#15866)
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
2025-10-23 18:25:25 -07:00
Ishaan Jaffer
ea8a604188 fix to .debug 2025-10-23 18:08:56 -07:00
Ishaan Jaffer
f4e98f715a fix linting 2025-10-23 18:07:28 -07:00
Ishaan Jaffer
76d658eee8 fix linting 2025-10-23 18:05:30 -07:00
Ishaan Jaff
5de912375c
[Feat] UI - Add logos for search providers (#15872)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db

* add UI friendly name for search providers

* add ui friendly name for search providers

* add providers available

* working layout

* better layout

* clean add search tool

* update_router_search_tools

* fix remove in memory registry, since router is in mem store

* allow testing search tool connection

* clean create search tool

* add test_search_tool_connection

* fix: _init_search_tools_in_db

* add searchToolQueryCall

* fix icon

* clean tester

* add parallel ai logo

* add exa ai logo

* add google PSE logo

* add tavily logo

* add dataforseo + perplexity

* add parallel ai logo

* UI show logos for search
2025-10-23 18:00:40 -07:00
Ishaan Jaff
fc9aba279e
[Feat] UI - Search Tools, allow adding search tools on UI + testing search (#15871)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db

* add UI friendly name for search providers

* add ui friendly name for search providers

* add providers available

* working layout

* better layout

* clean add search tool

* update_router_search_tools

* fix remove in memory registry, since router is in mem store

* allow testing search tool connection

* clean create search tool

* add test_search_tool_connection

* fix: _init_search_tools_in_db

* add searchToolQueryCall

* fix icon

* clean tester
2025-10-23 17:59:29 -07:00
Ishaan Jaff
d8ea1665c7
[Feat] - [Backend] Search APIs - Allow storing configured Search APIs in DB (#15862)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db
2025-10-23 17:57:49 -07:00