Commit graph

8121 commits

Author SHA1 Message Date
yuneng-jiang
5dad3c9708 Merge remote-tracking branch 'origin' into litellm_ui_callback_fix 2025-11-21 16:27:49 -08:00
yuneng-jiang
d785205a70 Merge remote-tracking branch 'origin' into litellm_org_usage 2025-11-21 16:25:55 -08:00
Ishaan Jaffer
4a9f163db1 TestPromptVersionsEndpoint 2025-11-21 16:21:13 -08:00
Ishaan Jaffer
4e8f1d0143 fix prompt manager 2025-11-21 16:17:44 -08:00
Ishaan Jaffer
2226450437 test_ensure_initialize_azure_sdk_client_always_used 2025-11-21 16:15:35 -08:00
yuneng-jiang
0abfb07ab8
Remove UI Session Token from user/info return (#16851) 2025-11-21 16:11:58 -08:00
Ishaan Jaff
8e318dd06c
[Feat] New LLM Provider - Docker Model Runner (#16948)
* add DOCKER_MODEL_RUNNER

* add DockerModelRunnerChatConfig Transorm

* add docker_model_runner

* add docker_model_runner

* docs docker model runner

* add DockerModelRunnerChatConfig

* add docker_model_runner to providers

* test_completion_hits_correct_url_and_body

* fix sidebar

* TestDockerModelRunnerIntegration

* test_completion_with_custom_engine_and_host

* docs docker model runner

* docs fix
2025-11-21 16:09:32 -08:00
Eiliya
d88580fa28
fix(gemini-video): inherit BaseVideoConfig to enable async content response (#16875)
This fix addresses the same issue that was resolved for OpenAI video in PR #16708.

The GeminiVideoConfig class was importing BaseVideoConfig only within TYPE_CHECKING,
causing it to be 'Any' at runtime. This prevented the async_transform_video_content_response
method from being available during video content downloads.

Changes:
- Moved BaseVideoConfig import from TYPE_CHECKING to top-level imports
- Added test_gemini_video_config_has_async_transform() to verify the fix
- Ensures GeminiVideoConfig properly inherits BaseVideoConfig at runtime

Fixes video generation errors for Gemini Veo models:
'GeminiVideoConfig' object has no attribute 'async_transform_video_content_response'
2025-11-21 16:01:21 -08:00
Ishaan Jaff
6ae22908b7
[Feat] Prompt Versioning - Allow specifying prompt version in code (#16929)
* add _get_prompt_data_from_dotprompt_content

* fix pre call hook for prompt template

* fix: get_latest_version_prompt_id

* fix get_latest_version_prompt_id

* test_get_latest_version_prompt_id
2025-11-21 13:58:49 -08:00
colinlin-stripe
f9d8eeaf8e
[stripe] gemini 3 thought signatures in tool call id (#16895)
* though signature tool call id

* [stripe] refactor and tests

* [stripe] remove md and move to factory

* [stripe] remove redudant test

* [stripe] ran black formatting

* [stripe] add thought signature docs

* [stripe] remove unused import
2025-11-21 13:44:53 -08:00
Ishaan Jaff
97d9da93e0
[Feat] Prompt Management - Allow viewing version history (#16901)
* TestPromptRequest

* add prompts/test endpoint for testing prompt

* TestPromptTestEndpoint

* feat: working v1 of this ui

* workig prompt endpoints

* add chat ui for prompts

* add conversation panel

* add init chat ui

* allow clicking edit prompt

* fix use get_base_prompt_id

* add endpoints for viewing prompt versions

* TestPromptVersioning

* add getPromptVersions

* add VersionHistorySidePanel

* allow viewing version history

* add version history
2025-11-21 08:54:52 -08:00
YutaSaito
041ac054b6
feat: allow custom violation message for tool-permission guardrail (#16916) 2025-11-21 08:52:01 -08:00
yuneng-jiang
d672263fe3 Change litellm_model_cost_map to public route 2025-11-20 20:59:31 -08:00
yuneng-jiang
b50790aaad Revert "Expose new model provider map endpoint and use in add model workflow"
This reverts commit 8a4fefc565.
2025-11-20 20:40:08 -08:00
yuneng-jiang
31535ff4b6 Merge with main 2025-11-20 20:24:03 -08:00
yuneng-jiang
6029a24baa Merge remote-tracking branch 'origin' into litellm_org_usage 2025-11-20 20:19:16 -08:00
yuneng-jiang
7225fc066f
Fix key model alias (#16896) 2025-11-20 16:05:49 -08:00
YutaSaito
93affcb732
[Feat] mcp resources support (#16800)
* feat: mcp prompts support

* feat: mcp resources support
2025-11-20 14:53:44 -08:00
Sameer Kankute
e5948770dd
Fix audio transcription cost tracking (#16478) 2025-11-19 20:29:39 -08:00
Sameer Kankute
c3143e388e
Add thought signature support to v1/messages api (#16812)
* Add thought signature support to v1/messages api

* update the thinking level handling logic

* update the thinking level handling logic

* Add streaming support

* fix intalling litellm error
2025-11-19 20:24:31 -08:00
Sebastian
cb843684b8
fix(vertex_ai): add includeThoughts=True for Gemini 3 reasoning_effort (#16838)
Gemini 3 models require 'includeThoughts: True' in the thinkingConfig to return the actual thought text. Previously, using reasoning_effort set the 'thinkingLevel' but missed the boolean flag, resulting in empty reasoning_content.

This fix:
1. Updates `_map_reasoning_effort_to_thinking_level` to include `includeThoughts: True` for low/medium/high.
2. Adds unit tests to verify the config mapping.
2025-11-19 19:14:42 -08:00
Alex Huang
3b6f3e48cb
Fix optional param mapping (#16852)
* Direct string check instead of tuple string inclusion check

* Add test
2025-11-19 19:10:04 -08:00
Cesar Garcia
7d5cb8ebb2
fix(gemini): Add reasoning_content to streaming responses with tools (#16854)
Fixes #16805

When using Gemini models (2.5/3.0) with streaming + tools enabled,
the reasoning_content field was missing from stream chunks, even though
thinking_blocks were present in non-streaming responses.

Changes:
- Convert thinking_blocks to reasoning_content for streaming responses
- Extract "thinking" field from each thinking_block
- Concatenate multiple thinking parts with newlines
- Assign to reasoning_content in chat_completion_message for streaming

Testing:
- Added test_streaming_chunk_with_tool_calls_includes_reasoning_content
- Test verifies reasoning_content appears with tool calls in streaming
- All 39 existing Gemini tests pass
2025-11-19 19:09:37 -08:00
Nigel Kukard
c5c563c302
fix: fixed openai conversion from responses to completions (#16864)
- Fix blank function name in completions response when using native function calling
- Fix Enum name being used instead of Enum value for comparison in chunk conversion
- Added additional tests to cover changes

Thanks to @mcowger for the invaluable assitance with figuring this issue out!

Fixed #16863
2025-11-19 19:02:52 -08:00
Sameer Kankute
6fc7397dde
Add Vertex AI Image Edit Support (#16828)
* Add vertex ai image edit support

* Fix lint errors
2025-11-19 18:39:28 -08:00
idola9
e1005cb9d3
Use LiteLLM key alias as fallback Noma applicationId in NomaGuardrail (#16832)
* Use auth key name if there are no app id in in headers or in extra_data

* use key alias instead of key name

* Fix

* last priority key alias

* Fix

* Add tests
2025-11-19 17:44:23 -08:00
Sameer Kankute
9622829fa1
Fix vector store create issue (#16804) 2025-11-19 16:53:20 -08:00
yuneng-jiang
4b80813fc2 Return 404 when a user is not found 2025-11-19 13:37:51 -08:00
Ishaan Jaff
c7cf18cf67
[Feat] Prompt Management - Allow storing prompt version in DB (#16848)
* test_dotprompt_auto_detection_with_model_only

* fix _auto_detect_prompt_management_logger

* test_dotprompt_with_prompt_version

* add v1, v2 tests

* add _compile_prompt_helper

* fix _compile_prompt_helper

* test_dotprompt_with_prompt_version

* test_dotprompt_with_prompt_version, test_get_prompt_with_version

* add version in schema

* feat add _get_prompt_spec_for_db_prompt

* add _get_prompt_spec_for_db_prompt

* feat add _get_prompt_spec_for_db_prompt

* update prompt table

* add version in prompt DB

* test_get_prompt_spec_for_db_prompt_with_versions
2025-11-19 13:19:56 -08:00
Naki
98dd866b26
feat(github-copilot): Add Responses API support for gpt-5.1-codex model (#16845)
- Implement GithubCopilotResponsesAPIConfig for /responses endpoint
- Add support for models requiring responses API (e.g., gpt-5.1-codex)
- Auto-detect vision requests and set X-Initiator header
- Follow OpenAI Responses API compatibility pattern
- Add comprehensive unit tests (16 tests passing)

Fixes #16820
2025-11-19 13:17:19 -08:00
Ishaan Jaff
3ebe489082
[Feat] Prompt Management - Add support for versioning prompts (#16836)
* test_dotprompt_auto_detection_with_model_only

* fix _auto_detect_prompt_management_logger

* test_dotprompt_with_prompt_version

* add v1, v2 tests

* add _compile_prompt_helper

* fix _compile_prompt_helper

* test_dotprompt_with_prompt_version

* test_dotprompt_with_prompt_version, test_get_prompt_with_version
2025-11-19 13:16:03 -08:00
Chesars
3e58fe42b7 fix: Support response_format parameter in completion -> responses bridge
Fixes #16810

## Problem

When using completion() with models that have mode: "responses" (like o3-pro,
gpt-5-codex), the response_format parameter with JSON schemas was being ignored
or incorrectly handled, causing:
- Large schemas (>512 chars) to fail with "metadata.schema_dict_json: string too long" error
- Structured outputs to be silently dropped
- Users' code to break unexpectedly

## Root Cause

The completion -> responses bridge in
litellm/completion_extras/litellm_responses_transformation/transformation.py
was missing the conversion of response_format (Chat Completion format) to
text.format (Responses API format).

The inverse bridge (responses -> completion) already had this conversion
implemented in commit 29f0ed223a, but the completion -> responses direction
was incomplete.

## Solution

Added _transform_response_format_to_text_format() method that converts:
- response_format with json_schema → text.format with json_schema
- response_format with json_object → text.format with json_object
- response_format with text → text.format with text

Updated transform_request() to detect and convert response_format parameter
before sending to litellm.responses().

## Changes

- Added _transform_response_format_to_text_format() method (lines 592-647)
- Modified transform_request() to handle response_format (lines 199-203)
- Added comprehensive tests to validate the conversion

## Testing

- 5 new unit tests covering all conversion scenarios
- Real API test with OpenAI confirming large schemas (>512 chars) work
- No more metadata.schema_dict_json errors

## Impact

Users can now use completion() with models that have mode: "responses" and:
- Use large JSON schemas without hitting metadata 512 char limit
- Get proper structured outputs
- Have their existing code continue working
2025-11-19 17:07:46 -03:00
yuneng-jiang
dcf7daf629 Allow root to redirect when docs are not on root path 2025-11-19 12:05:41 -08:00
Ishaan Jaff
1f8fe007a1
[Feat] Prompt Management - Allow specifying just prompt_id in a request to a model (#16834)
* test_dotprompt_auto_detection_with_model_only

* fix _auto_detect_prompt_management_logger

* test_dotprompt_auto_detection_with_model_only
2025-11-19 10:20:58 -08:00
Sameer Kankute
29057ba6ad Added support for azure anthopic models via chat completion 2025-11-19 11:55:38 +05:30
Alan Ponnachan
b92cc2b2f9
fix(bedrock): Ensure consistent chunk IDs in Bedrock streaming responses (#16596)
* ensure consistent chunk IDs in streaming responses

* use native conversationId for consistent stream chunk IDs
2025-11-18 20:37:21 -08:00
Alexsander Hamir
2e007505da
fix(spend-logs): trim logged response strings (#16654)
* fix(spend-logs): trim logged response strings

- route spend-log responses through the existing string sanitizer so oversized base64/text fields are truncated before persistence
- add unit tests covering the truncation path and the feature flag

Note: embeddings-specific truncation (numeric vectors) is still pending and will be handled separately.

* remove unnecessary comment

* add: sanitization unit test for embeddings

* fix: simplify sanatization logic

I overcomplicated a simple change for lack of understanding, fixed.
2025-11-18 20:12:49 -08:00
Sameer Kankute
9e93d65ee2
Add extra_body support for response api params from chat completion (#16765) 2025-11-18 20:05:47 -08:00
Ishaan Jaff
c3c6cef6d8
[Fix] Fixes Swagger UI resolver errors for chat completion endpoints caused by Pydantic v2 $defs not being properly exposed in the OpenAPI schema. (#16784)
* fix add_request_body_to_paths

* test_move_defs_to_components
2025-11-18 17:39:28 -08:00
Ishaan Jaff
2880cb45a2
[Fix] AI Gateway Auth - Ensure Team Tags works when using JWT Auth (#16797)
* fix setting team_metadata

* test_team_metadata_with_tags_flows_through_jwt_auth
2025-11-18 17:36:38 -08:00
yuneng-jiang
8a4fefc565 Expose new model provider map endpoint and use in add model workflow 2025-11-18 15:45:35 -08:00
Ishaan Jaff
e2fc225201
[Feat] SSO - Ensure role from SSO provider is used when a user is inserted onto LiteLLM (#16794)
* test_apply_user_info_values_sso_role_takes_precedence

* fix SSO
2025-11-18 15:35:38 -08:00
Sameer Kankute
f3344a0d07
Fix videos tagging (#16770) 2025-11-18 11:37:05 -08:00
yuneng-jiang
f9ec353b80
[Feature] UI - Allow setting base_url in API reference docs (#16674)
* Allow setting base_url in API reference docs

* Add logic to change base url for test key page
2025-11-18 11:27:28 -08:00
Sameer Kankute
acf206bec6
Add Day 0 gemini-3-pro-preview support (#16719)
* Add thinking signature support for gemini

* Add docs related to thinking signature

* remove double base64 import

* fix mypy errors

* fix litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py mypy

* Add new gemini 3 model and features

* Add docs related to gemini 3

* Update gemini 3 pricing

* fix llm translation tests

* fix mapped tests
2025-11-18 09:44:45 -08:00
yuneng-jiang
98365205ac Deduplicate /tag/daily/activity metadata 2025-11-17 23:00:53 -08:00
Andrey
4dbbec233c
Snowflake provider support: added embeddings, PAT, account_id (#15727)
* added oauth mcp to docs

* added azure ai/grok-4 model family

* Revert "added oauth mcp to docs"

This reverts commit 950b7cef44.

* fix: arize ui integration

* need to remove a file

This reverts commit d6c877b73a.

* snowflake support PAT, account_id and embeddings

* format

* test embeddings

* format

* complete test

* fix: add arize from ui

* updated clarifai functions to openai compatible (#15615)

* fix: npm build errors

* update tests

* SnowflakeBaseConfig moved to utils

* rename pat_key => api_ke

* key_type=PAT => 'pat/key'

* fix if api_key is None

* doc update

* doc update

---------

Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Mubashir Osmani <ilikewafflesomcuh@gmail.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: mogith-pn <143642606+mogith-pn@users.noreply.github.com>
2025-11-17 20:27:46 -08:00
YutaSaito
86ada061ec
fix: prompt injection not working (#16701) 2025-11-17 20:04:57 -08:00
pnookala-godaddy
52337961dc
fix(openai-video): inherit BaseVideoConfig to enable async content response (#16708) 2025-11-17 20:04:10 -08:00
John Lathouwers
d93e13be36
OCI Provider - Add oci_endpoint_id Parameter for OCI Dedicated Endpoints #16723 2025-11-17 19:59:34 -08:00