Krrish Dholakia
ace862189c
test(test_mcp_server_manager.py): add unit testing
2025-10-09 14:52:46 -07:00
Krish Dholakia
e2e0cdde31
Merge pull request #15315 from lcfyi/lcfyi/add-support-for-parallel-tool-calls
...
Fix parallel tool calls in the Anthropic passthrough adapter
2025-10-08 23:01:55 -07:00
Krish Dholakia
d921df1baf
Merge pull request #15343 from BerriAI/litellm_dev_10_08_2025_p1
...
MCP - support converting OpenAPI specs to MCP servers
2025-10-08 19:10:04 -07:00
Krish Dholakia
12a1d081ee
Merge branch 'main' into litellm_dev_09_11_2025_p1
2025-10-08 19:02:58 -07:00
Ishaan Jaffer
c4022ade49
test mapped tests MCP
2025-10-08 18:34:30 -07:00
Ishaan Jaff
97031dc8ee
Fix - (openrouter): move cache_control to content blocks for claude/gemini ( #15345 )
...
* test_openrouter_transform_request_with_cache_control
* fix CacheControlSupportedModels
* test_openrouter_transform_request_with_cache_control_gemini
2025-10-08 17:41:04 -07:00
Pierre-Emmanuel MERCIER
861750790b
feat: add redis ssl and username support ( #11319 )
2025-10-08 16:48:07 -07:00
Krrish Dholakia
cd13e533a2
feat(mcp_server_manager.py): expand allowed/disallowed list
2025-10-08 16:06:17 -07:00
Krrish Dholakia
28698933fc
feat(mcp_server_manager.py): ensure list tools for openapi servers works as expected
2025-10-08 15:59:42 -07:00
Ishaan Jaff
1c56a0d856
[Fix] Watsonx - Apply correct prompt templates for openai/gpt-oss model family ( #15341 )
...
* fix: apply_prompt_template
* Revert "fix: apply_prompt_template"
This reverts commit 3e0e40b4973e80903796
2025-10-08 15:39:36 -07:00
Leslie Cheng
f0c9dfbaf8
Add some tests
2025-10-07 20:57:08 -07:00
Krish Dholakia
13703f289b
Merge pull request #15292 from timelfrink/fix/bedrock-prompt-caching-cost-calculation
...
fix(bedrock): include cacheWriteInputTokens in prompt_tokens calculation
2025-10-07 19:11:45 -07:00
Ishaan Jaffer
e1ab3620ee
fix: mapped tests
2025-10-07 17:55:52 -07:00
Ishaan Jaff
36c971a6fd
[MCP Gateway] QA/Fixes - Ensure Team/Key level enforcement works for MCPs ( #15305 )
...
* fix: _set_object_permission
* fix: _set_object_permission on teams
* fix: _set_object_permission
* fixes for team/key permissions
* statsh: object permission view
* fix: MCPServerPermissions
* fix: _get_team_object_permission
* test mcp checks for permissions
* fix server checks with prefix names
* test_list_tools_strips_prefix_when_matching_permissions
* ruff fix
* docs - refactor MCP
* docs update MCP docs
* docs allowed tools
2025-10-07 17:34:48 -07:00
Ishaan Jaff
7b56ba240e
[MCP Gateway] Litellm mcp fixes team control ( #15304 )
...
* fix: _set_object_permission
* fix: _set_object_permission on teams
* fix: _set_object_permission
* fixes for team/key permissions
* statsh: object permission view
* fix: MCPServerPermissions
2025-10-07 16:48:00 -07:00
Ishaan Jaff
bc26eff98f
Fix: Make PATCH /model/{model_id}/update handle team_id consistently with POST /model/new ( #15297 )
...
* fix: _update_team_model_in_db
* test_patch_model_with_team_id_creates_proper_setup
2025-10-07 14:04:08 -07:00
Ishaan Jaff
07a17d6d6b
[Feat] Proxy CLI - dont store existing key in the URL, store it in the state param ( #15290 )
...
* Feat: CLI Auth fixes for UI SSO
* fix auth.py
* fix test ui sso.py
2025-10-07 12:36:17 -07:00
Tim Elfrink
c5eb22381d
fix(bedrock): include cacheWriteInputTokens in prompt_tokens calculation
...
Fixes #15263
This PR fixes the cost calculation for Bedrock Anthropic models with prompt caching.
**Root Cause:**
PR #9838 incorrectly removed adding `cacheWriteInputTokens` to `prompt_tokens`
for Bedrock, based on the assumption that it would cause double counting (similar
to an Anthropic API issue). However, Bedrock's token structure is different:
- **Bedrock API**: `inputTokens`, `cacheReadInputTokens`, and `cacheWriteInputTokens`
are ALL separate values that should be summed for total input tokens
- **Anthropic API**: Same structure - all three token types are separate
The fix in #9838 was later reverted for Anthropic (correctly re-adding
`cache_creation_input_tokens` to `prompt_tokens`), but Bedrock was never fixed.
**Changes:**
1. Re-add `cacheWriteInputTokens` to `input_tokens` in Bedrock transformation
2. Update test assertions to reflect correct behavior
3. Add regression test for prompt caching cost calculation
4. Fix typo in Anthropic transformation where `cache_creation_tokens` was
incorrectly set to `cache_read_input_tokens`
**Testing:**
- All existing Bedrock transformation tests pass
- New test validates correct cost calculation with prompt caching
- Verified costs are non-negative and accurate
2025-10-07 20:28:46 +02:00
Sameer Kankute
e4892735f0
fix gemini cli by actually streaming the response ( #15264 )
...
* fix gemini cli by actually streaming the response
* fix cost tracking
* fix test
2025-10-07 11:26:39 -07:00
Sameer Kankute
73f96712f5
fix the reasoningresponse id ( #15265 )
2025-10-07 11:24:29 -07:00
Sameer Kankute
c0d0424eb8
Added streaming support for response api streaming image generation ( #15269 )
2025-10-07 08:15:57 -07:00
Krish Dholakia
94a34dd53a
Merge branch 'main' into litellm_dev_10_06_2025_p1
2025-10-06 20:06:30 -07:00
Krish Dholakia
8b357c24c2
Merge pull request #15226 from otaviofbrito/chore/vertex-ai-context-caching
...
Chore/vertex ai context caching
2025-10-06 20:05:24 -07:00
Krish Dholakia
6b4415684e
Merge pull request #15253 from BerriAI/litellm_dev_10_06_2025_p2
...
fix(azure/responses): remove invalid status param from azure call + MCP - support setting CA_BUNDLE_PATH
2025-10-06 20:01:39 -07:00
Sameer Kankute
8d7f39798c
Removed stop param from unsupported azure models ( #15229 )
...
* Removed stop param from unsupported model
* Use better handling for stop method
* Use better handling for stop method
2025-10-06 19:56:18 -07:00
Ishaan Jaff
1d468a59f3
[MCP Gateway] Fine-grained Database Object Storage Control ( #15255 )
...
* add supported_db_objects
* add _should_load_db_object
* add docs on storing MCP objects in DB
* test_should_load_db_object_with_supported_db_objects
* type fix
2025-10-06 19:31:44 -07:00
Ishaan Jaff
930f88078e
[MCP Gateway] Backend - Allow storing allowed tools by team/key ( #15243 )
...
* add mcp_tool_permissions column to LiteLLM_ObjectPermissionTable
* add mcp_tool_permissions
* add LiteLLM_ObjectPermissionTable
* test key/team mgmt
* fix code QA check
* fix check_tool_permission_for_key_team
* test_key_tool_permission_allows_permitted_tool
* test_list_tools_filters_by_key_team_permissions
* fix _get_tools_from_mcp_servers
* fix logger.debug
* fix check_tool_permission_for_key_team
* fix: filter_tools_by_key_team_permissions
* fix _get_team_object_permission
* fix _set_object_permission
* fix handle_update_object_permission_common
* fix add allowed tools to keys
* test add key/team level tool permissions for MCPs
2025-10-06 18:49:32 -07:00
Krrish Dholakia
7f88a3f9c6
fix(mcp/): add ssl certificate settings for mcp clients
...
respect ca bundle path for mcp calls
2025-10-06 18:36:05 -07:00
Krrish Dholakia
d7aab1d5de
test: add unit tests
2025-10-06 13:53:36 -07:00
Sameer Kankute
cd25782359
fix passing headers for gemini ( #15231 )
2025-10-06 12:59:19 -07:00
Otavio Brito
b2a734c1f8
fix ttl tests
2025-10-06 09:30:23 -03:00
Otavio Brito
251d56ba5e
update docs
2025-10-05 23:47:24 -03:00
Javier de la Torre
df232a71f1
feat(snowflake): add function calling support for Snowflake Cortex REST API
...
Add support for function calling (tools) with Snowflake Cortex models that support it (e.g., Claude 3.5 Sonnet).
Changes:
- Add 'tools' and 'tool_choice' to supported OpenAI parameters
- Implement request transformation: OpenAI function format → Snowflake tool_spec format
- Implement response transformation: Snowflake content_list with tool_use → OpenAI tool_calls
- Add tool_choice transformation: OpenAI nested format → Snowflake array format
Request transformation:
- Transform tools from nested {"type": "function", "function": {...}} to Snowflake's {"tool_spec": {"type": "generic", "name": "...", "input_schema": {...}}}
- Transform tool_choice from {"type": "function", "function": {"name": "..."}} to {"type": "tool", "name": ["..."]}
Response transformation:
- Parse Snowflake's content_list array containing tool_use objects
- Extract tool calls with tool_use_id, name, and input
- Convert to OpenAI's tool_calls format with proper JSON serialization
Testing:
- Add 7 unit tests covering request/response transformations
- Add integration test for Responses API with tool calling
- All tests passing
Fixes issue #15218
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-05 13:08:33 +02:00
Krrish Dholakia
63cb2764fe
test: fix raise
2025-10-04 16:11:22 -07:00
Krrish Dholakia
2d1000c461
test: update tests
2025-10-04 15:31:25 -07:00
Krish Dholakia
a9037ab61f
Merge branch 'main' into litellm_dev_10_03_2025_p1
2025-10-04 15:23:31 -07:00
Krish Dholakia
75e3f4e10d
Merge pull request #15120 from BerriAI/litellm_dev_10_01_2025_p2
...
(feat) Support 'guaranteed_throughput' when setting limits on keys belonging to a team
2025-10-04 15:22:47 -07:00
Krrish Dholakia
814e49df06
test: update tests
2025-10-04 14:39:50 -07:00
Ishaan Jaff
ec9cdf783a
Revert "Add streamGenerateContent cost tracking in passthrough ( #15199 )" ( #15202 )
...
This reverts commit 8095de506a
2025-10-04 14:37:02 -07:00
Sameer Kankute
8095de506a
Add streamGenerateContent cost tracking in passthrough ( #15199 )
...
* Add streamgenerate cost tracking for gemini provider
* add cost tracking test
2025-10-04 12:31:25 -07:00
Ishaan Jaffer
3d6342f852
test_twelvelabs_missing_input_type_error
2025-10-04 12:28:09 -07:00
Ishaan Jaffer
c38c5d20cc
test dynamic rate limiter 3
2025-10-04 12:06:13 -07:00
Ishaan Jaff
f78608082c
[Feat] Dynamic Rate Limiter v3 - fixes for detecting saturation + fixes for post saturation behavior ( #15192 )
...
* fix: test case 1, model hits saturation
* fix: _check_rate_limits test case 2
* fix: _get_priority_allocation
* test_default_priority_shared_pool
* fix: No Rate Limiting when low saturatation
* fix: correctly use model_saturation_check
* fixes priority_descriptors
* fix: tune default PriorityReservationSettings
2025-10-04 10:45:17 -07:00
Krish Dholakia
206f67a11e
Merge pull request #15183 from uc4w6c/fix/test_mcp_server
...
test: fix test_mcp_server.py
2025-10-03 21:44:52 -07:00
Krrish Dholakia
3816d3b36f
test: add unit testing
2025-10-03 17:27:49 -07:00
Krrish Dholakia
c043ad6c2b
fix: replace list in for set in
...
faster
2025-10-03 17:17:20 -07:00
Krrish Dholakia
69edc546c0
fix(mcp_server_manager.py): don't return an invalid server id on list servers
...
Prevents user from hitting 'server not found' error, even when they see it being listed on litellm ui
2025-10-03 17:15:15 -07:00
Yuta Saito
359aaa947f
test: fix test_mcp_server.py
2025-10-04 08:18:15 +09:00
Ishaan Jaff
10d6d72ae3
[Feat] VertexAI - Support googlemap grounding in vertex ai ( #15179 )
...
* add VertexToolName
* test_vertex_tool_params
* fix: working maps grounding
* test_gemini_google_maps_tool_simple
* test_vertex_ai_map_google_maps_tool_with_location
* fix # noqa: PLR0915
* _extract_google_maps_retrieval_config
* fixes for linting
* docs: **Google Maps**
2025-10-03 16:07:51 -07:00
Krish Dholakia
3095f43e92
Merge pull request #15007 from TobiMayr/feature/add-max-requests-env-var
...
feature/add max requests env var
2025-10-03 15:30:15 -07:00
