davida-ps
7777aeb695
fixing prompt-security's guardrail implementation ( #19374 )
...
* Consolidated change
* fix(prompt_security): update message processing to persist sanitized files and filter for API calls
* fix per krrishdholakia suggestion
2026-01-21 20:09:40 -08:00
Ishaan Jaff
ab606c9a73
[Feat] Add Structured output for /v1/messages with Anthropic API, Azure Anthropic API, Bedrock Converse ( #19545 )
...
* fix: add AnthropicMessagesRequestOptionalParams
* add _update_headers_with_anthropic_beta
* fix output format tests
* test_structured_output_e2e
* TestAnthropicAPIStructuredOutput
* test_structured_output_e2e
* fix BASE
* TestAzureAnthropicStructuredOutput
* fix: Bedrock Converse
* add nthropic Messages Pass-Through Architecture
* fix: bedrock invoke output_format
* fix: transform_anthropic_messages_request for vertex anthropic
* TestBedrockInvokeStructuredOutput
* docs anthropic vertex
* docs fix
* docs fix
2026-01-21 20:09:18 -08:00
jay prajapati
363b0cc132
fix(azure): preserve content_policy_violation details for images ( #19328 ) ( #19372 )
...
Azure OpenAI Images (DALL·E 3) returns policy violations as a structured payload under body["error"], including inner_error.content_filter_results and revised_prompt.
LiteLLM previously:
- Failed to extract nested error messages (get_error_message only handled body["message"])
- Missed policy violation detection when error strings were generic
- Dropped inner_error details when raising ContentPolicyViolationError
This change:
- Extracts nested Azure error fields (code/type/message + inner_error)
- Detects policy violations via structured error codes
- Passes an OpenAI-style error body + provider_specific_fields to preserve details
Tests:
- python3 -m pytest tests/test_litellm/llms/azure/test_azure_exception_mapping.py
- python3 -m pytest tests/test_litellm/litellm_core_utils/test_exception_mapping_utils.py
Fixes #19328
2026-01-21 20:06:51 -08:00
Sameer Kankute
3794f86af0
Merge pull request #19502 from Chesars/docs/base-model-openai-dated-versions
...
docs(custom_pricing): add base_model usage for OpenAI dated model versions
2026-01-22 09:19:12 +05:30
Sameer Kankute
36f3250016
Merge pull request #19500 from Chesars/fix/audio-model-pricing
...
fix(pricing): correct audio token costs for gpt-4o-audio-preview models
2026-01-22 09:18:55 +05:30
Sameer Kankute
967bc82e53
Merge pull request #19509 from Chesars/add-gpt-audio-models
...
Add gpt-audio and gpt-audio-mini models to pricing
2026-01-22 09:18:11 +05:30
jay prajapati
0e738a5027
fix(mcp): forward static_headers to MCP servers ( #19341 ) ( #19366 )
...
Forward static_headers from /mcp-rest/test/* routes into the MCP client so headers are present during session.initialize() and tool discovery.
Also add a shared merge_mcp_headers() helper to keep header precedence consistent and ensure OpenAPI-to-MCP generated tools include static_headers.
Tests:
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_rest_endpoints.py
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py -k register_openapi_tools_includes_static_headers
Fixes #19341
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-01-21 19:30:55 -08:00
Ishaan Jaff
d5e912322f
[Fix] VertexAI Pass through - Ensure only anthropic betas are forwarded down to LLM API ( #19542 )
...
* fix ALLOWED_VERTEX_AI_PASSTHROUGH_HEADERS
* test_vertex_passthrough_forwards_anthropic_beta_header
* fix test_vertex_passthrough_forwards_anthropic_beta_header
* test_vertex_passthrough_does_not_forward_litellm_auth_token
* fix utils
* Using Anthropic Beta Features on Vertex AI
* test_forward_headers_from_request_x_pass_prefix
2026-01-21 19:12:04 -08:00
yuneng-jiang
d1172562bc
fixing build
2026-01-21 18:29:08 -08:00
yuneng-jiang
f737429ad9
create team model select
2026-01-21 18:21:42 -08:00
yuneng-jiang
9fe2822cf0
Merge pull request #19540 from BerriAI/litellm_ui_e2e_internal_viewer_tests
...
[Infra] UI - E2E Tests: Internal Viewer Sidebar
2026-01-21 18:15:16 -08:00
yuneng-jiang
e406ebf0dd
Model Select for Create Team
2026-01-21 18:14:45 -08:00
yuneng-jiang
6b6785bc4f
Merge pull request #19539 from BerriAI/litellm_models_scope
...
[Feature] Adding Optional scope Param to /models
2026-01-21 17:41:22 -08:00
yuneng-jiang
4497f2de5c
e2e test internal viewer sidebar
2026-01-21 17:39:02 -08:00
yuneng-jiang
e877b08314
Merge pull request #18995 from BerriAI/litellm_e2e_create_key_test
...
[Infra] UI - E2E Test: Create Key With All Team Models
2026-01-21 16:46:49 -08:00
yuneng-jiang
c6b157832b
Merge pull request #19296 from BerriAI/litellm_esca_reissue
...
[Reissue: Fix] /user/new Privilege Escalation
2026-01-21 16:46:34 -08:00
yuneng-jiang
1a0faaff0d
Merge pull request #19468 from BerriAI/litellm_ui_team_select_001
...
[Fix] UI - MCP Servers: MCP Tools Tab Resetting to Overview
2026-01-21 16:46:19 -08:00
yuneng-jiang
6723b30d03
Adding scope to /models
2026-01-21 16:40:31 -08:00
Alexsander Hamir
2620b9f041
Cut chat_completion latency by ~21% by reducing pre-call processing time ( #19535 )
2026-01-21 16:05:29 -08:00
Cesar Garcia
4106d24215
feat: add GMI Cloud provider support ( #19376 )
...
* feat: add GMI Cloud provider support
Add GMI Cloud as an OpenAI-compatible provider with:
- Provider configuration in providers.json
- Documentation page with usage examples
- Model pricing for 16 models (Claude, GPT, DeepSeek, Gemini, etc.)
- Sidebar entry for docs navigation
* Add gmi_cloud to provider_endpoints_support.json
Add provider entry to pass CI validation check that ensures all
providers in openai_like/providers.json are documented.
* Fix provider key: gmi_cloud -> gmi
Match the provider key with providers.json
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-01-21 15:48:15 -08:00
yuneng-jiang
665d3f2f25
Merge pull request #19534 from BerriAI/litellm_ui_sorting_keys_fix_2
...
[Fix] UI - Virtual Keys Table: Sorting Shows Incorrect Entries
2026-01-21 15:09:43 -08:00
yuneng-jiang
a1636640ce
Adding tests
2026-01-21 14:59:56 -08:00
yuneng-jiang
b23e77585f
Fix virtual keys table sorting
2026-01-21 14:49:33 -08:00
YutaSaito
3a49e4362f
Merge pull request #19528 from BerriAI/litellm_fix_langfuse_trace_id
...
[fix] Send litellm_trace_id to Langfuse
2026-01-22 07:08:10 +09:00
Yuta Saito
898cc3ff4f
test: update langfuse trace_id tests to use litellm_trace_id
2026-01-22 06:19:43 +09:00
Yuta Saito
c0007bd418
fix: Send litellm_trace_id to Langfuse to link LiteLLM logs with Langfuse logs
2026-01-22 05:40:16 +09:00
mubashir1osmani
5f7d8486ff
fix lint
2026-01-21 15:35:13 -05:00
YutaSaito
4a14a53ae8
Merge pull request #19469 from BerriAI/litellm_feat_mcp_spendlogs
...
[feat] mcp spendlogs
2026-01-22 05:29:21 +09:00
yuneng-jiang
2a2717a79c
Merge pull request #19525 from BerriAI/litellm_ui_models_pag_change
...
[Feature] UI - Model Page: All Model Tab Pagination
2026-01-21 12:15:18 -08:00
mubashir1osmani
5b64539d12
fix linting errors
2026-01-21 15:09:53 -05:00
Ishaan Jaff
5cb5969a26
[Fix] LiteLLM VertexAI Pass through - ensuring incoming headers are forwarded down to target ( #19524 )
...
* test_vertex_passthrough_forwards_anthropic_beta_header
* add_incoming_headers
2026-01-21 12:01:33 -08:00
yuneng-jiang
0c5f40fffe
fixing build
2026-01-21 11:54:26 -08:00
yuneng-jiang
7cf80a9283
Paginate All Models Tab
2026-01-21 11:53:15 -08:00
mubashir1osmani
410daf6e6d
added tests
2026-01-21 14:41:35 -05:00
yuneng-jiang
aafe6cc107
Merge pull request #19521 from BerriAI/litellm_models_pagination_1
...
[Feature] Paginate /v2/models/info
2026-01-21 11:41:24 -08:00
yuneng-jiang
3075b0e5a2
fixing mypy linting
2026-01-21 11:22:20 -08:00
yuneng-jiang
d0e35751a1
Fixing tests and linting
2026-01-21 11:02:39 -08:00
yuneng-jiang
b5a7d2ab34
Paginating model/info endpoint
2026-01-21 10:44:18 -08:00
John Greek
aa4b0e0149
Fix duplicate test_handler.py filenames causing pytest collection errors ( #19385 )
2026-01-21 08:47:50 -08:00
Chesars
796e93552d
Fix gpt-audio models pricing and add dated snapshots
...
- Fix audio token pricing for gpt-audio ($32/$64 per 1M, not $40/$80)
- Add gpt-audio-2025-08-28 snapshot (OpenAI returns this in responses)
- Add gpt-audio-mini-2025-10-06 and gpt-audio-mini-2025-12-15 snapshots
- Add missing fields: supported_endpoints, supported_modalities,
supported_output_modalities, supports_native_streaming, etc.
2026-01-21 13:05:38 -03:00
Chesars
da8770004c
Add gpt-audio and gpt-audio-mini models to pricing
...
Fixes #19490 - adds missing OpenAI audio models with correct pricing:
gpt-audio:
- Text: $2.50/$10.00 per 1M tokens (input/output)
- Audio: $40/$80 per 1M tokens (input/output)
gpt-audio-mini:
- Text: $0.60/$2.40 per 1M tokens (input/output)
- Audio: $10/$20 per 1M tokens (input/output)
2026-01-21 12:47:10 -03:00
Chesars
f64ff8327d
docs(custom_pricing): add base_model usage for OpenAI dated model versions
...
Document how to use base_model to ensure custom pricing is applied when
OpenAI returns dated model names (e.g., gpt-4o-mini-audio-preview-2024-12-17)
that differ from the configured model name.
2026-01-21 11:18:51 -03:00
Sameer Kankute
95dc14e400
Merge pull request #19389 from BerriAI/litellm_timeout_fix
...
Fix: total timeout is not respected
2026-01-21 19:16:11 +05:30
Chesars
a0cfb56801
fix(pricing): correct audio token costs for gpt-4o-audio-preview models
...
Update audio token pricing for gpt-4o-audio-preview and
gpt-4o-audio-preview-2024-10-01 to match OpenAI's official pricing:
- input_cost_per_audio_token: 0.0001 -> 4e-05 ($40/1M tokens)
- output_cost_per_audio_token: 0.0002 -> 8e-05 ($80/1M tokens)
The previous values were 2.5x higher than OpenAI's actual pricing.
2026-01-21 10:46:03 -03:00
Sameer Kankute
e758dd0a59
Merge pull request #19472 from BerriAI/litellm_fix_chat_completion_responses_streaming
...
Fix: tool call streaming in chat completion bridge
2026-01-21 19:15:53 +05:30
Sameer Kankute
34d4c5d3fc
Fix mypy litellm_timeout_fix
2026-01-21 19:12:44 +05:30
Sameer Kankute
540370a1aa
Merge pull request #19479 from BerriAI/litellm_sarvam_int
...
Add support for sarvam models
2026-01-21 19:03:52 +05:30
Sameer Kankute
cc92ca088d
remove print statements
2026-01-21 19:00:01 +05:30
Sameer Kankute
c26c06d5b5
remove print statements
2026-01-21 18:59:51 +05:30
Sameer Kankute
b6691baf24
Merge pull request #19482 from BerriAI/litellm_anthropic_max_length
...
Filter out unsupported fields from JSON schema for Anthropic's output_format API.
2026-01-21 18:57:16 +05:30
