Commit graph

28063 commits

Author SHA1 Message Date
Jonathan Yang
43dd9e4a90
fix: replace deprecated .dict() with .model_dump() in streaming_handler (#17359)
Replace Pydantic v1 `.dict()` method with v2 `.model_dump()` to fix
PydanticDeprecatedSince20 warnings. The `.dict()` method is deprecated
in Pydantic v2 and will be removed in v3.

Fixes #5987
2025-12-02 22:12:55 -08:00
Jonathan Yang
17faea96bb
fix: conditionally pass enable_cleanup_closed to aiohttp TCPConnector (#17367)
* fix: conditionally pass enable_cleanup_closed to aiohttp TCPConnector

Fixes deprecation warning on Python 3.12.7+ and 3.13.1+ where
enable_cleanup_closed is no longer needed since the underlying
CPython SSL connection leak bug was fixed.

See: https://github.com/python/cpython/pull/118960

* chore: add aiohttp source reference to AIOHTTP_NEEDS_CLEANUP_CLOSED
2025-12-02 22:09:57 -08:00
Mariano Hielpos
566adebdec
update model_prices_and_context_window.json (#17376)
* update model_prices_and_context_window.json

* update

* update
2025-12-02 22:06:51 -08:00
Ali Saleh
6b5ad5d5a6
docs: Update Instructions For Phoenix Integration (#17373) 2025-12-02 22:03:54 -08:00
mossbanay
ae633184f7
Add model price & details for Bedrock model global.anthropic.claude-opus-4-5-20251101-v1:0 (#17380) 2025-12-02 22:02:57 -08:00
Matt Greathouse
f22bc0aab2
Support Deepseek 3.2 with Reasoning (#17384)
* Add openrouter/deepseek/deepseek-v3.2

* Added deepseek-provided v3.2

* Allow reasoning effort param for openrouter models that support it

* Added tests
2025-12-02 22:00:19 -08:00
Richard Song
099ccf56a7
Refactor add_schema_to_components to move definitions to components/schemas and add corresponding unit test (#17389) 2025-12-02 21:57:07 -08:00
Ishaan Jaff
427074ac6e
Fix: Datadog callback regression when ddtrace is installed (#17393)
* fix DD agent host logging

* docs fix

* test_datadog_agent_configuration

* test_datadog_ignores_ddtrace_agent_host
2025-12-02 17:27:50 -08:00
yuneng-jiang
451690bf11
Merge pull request #17321 from BerriAI/litellm_ui_user_alias_table
[Feature] Add User Alias Column to Internal User Table
2025-12-02 17:26:49 -08:00
yuneng-jiang
cfedc371d6
Merge pull request #17325 from BerriAI/litellm_ui_key_terminology
[Refactor] Standardize API Key vs Virtual Key in UI
2025-12-02 17:26:31 -08:00
yuneng-jiang
6e3a44b82c
Merge pull request #17397 from BerriAI/litellm_ui_cred_fix
[Fix] Show all credential values on Edit Credential Modal
2025-12-02 17:12:36 -08:00
yuneng-jiang
4063501d69 Show all credential values on Edit Credential Modal 2025-12-02 17:08:17 -08:00
flozonn
31cad8e6e6
feat: Add Nova lite 2 reasoning support with reasoningConfig (#17371) 2025-12-02 16:33:07 -08:00
Ishaan Jaff
10d56e7c46
add new deepseek-v3p2 (#17395) 2025-12-02 16:19:34 -08:00
yuneng-jiang
12d3c4a1d4
Merge pull request #17394 from BerriAI/litellm_ui_org_models_fix
[Fix] UI - Change Edit Team Models Shown to Match Create Team
2025-12-02 16:06:36 -08:00
Krrish Dholakia
8ee298f9c9 fix: remove python3 headers 2025-12-02 16:06:06 -08:00
yuneng-jiang
9867cc6cf9 Change edit team models to match create team models 2025-12-02 15:58:27 -08:00
Krrish Dholakia
7fb2f4730b build: remove duplicate packages 2025-12-02 15:53:10 -08:00
Ishaan Jaff
6c188c5ae2
[Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech (#17391)
* docs vertex tts

* place vertex ai types in file

* use VertexAITextToSpeechConfig

* use vertex_voice_dict

* refactor docs

* docs vertex ai chirp

* TestVertexAITextToSpeechConfig

* new provider vertex ai chirp3

* test_litellm_speech_vertex_ai_chirp

* add vertex_ai/chirp cost trackign
2025-12-02 15:36:23 -08:00
Leslie Cheng
de4ff120eb
🐛 Fix proxy caching between requests in aiohttp transport (#17122)
* write a regression test

* impl fix

* add test for host case

* use the host as cache key
2025-12-02 14:37:45 -08:00
Ishaan Jaff
db6c6eea89
[Docs] Add guide on how to debug gateway error vs provider error (#17387)
* add error diagnosis

* docs error diagnosis
2025-12-02 14:10:00 -08:00
yuneng-jiang
7d4f3f9290
Merge pull request #17382 from BerriAI/litellm_vertex_api_base
[Infra] Add Google Private API Endpoint to Vertex AI fields
2025-12-02 12:26:28 -08:00
yuneng-jiang
8b8f93d508 Add Google Private API Endpoint to Vertex AI fields 2025-12-02 12:24:00 -08:00
yuneng-jiang
61b0dbd726
Merge pull request #17378 from BerriAI/litellm_indent_import_fix
[Fix] Bedrock Guardrail Indent and Import fix
2025-12-02 11:28:12 -08:00
yuneng-jiang
8c1290dcd2 Indent and import fix 2025-12-02 11:26:29 -08:00
kothamah
12530b375f
Litellm bedrock OpenAI model support (#17368)
* Update constants.py

added constants

* Update base_aws_llm.py

added steps

* Update invoke_handler.py

added openai support

* Update base_invoke_transformation.py

added

* Update test_bedrock_completion.py

added
2025-12-02 09:19:53 -08:00
Cesar Garcia
81f4d863ca
docs: add Azure AI Foundry documentation for Claude models (#17104)
* docs: add Azure AI Foundry documentation for Claude models

Add documentation explaining how to use Claude models (Sonnet 4.5,
Haiku 4.5, Opus 4.1) deployed on Azure AI Foundry with LiteLLM.

Azure exposes Claude using Anthropic's native API, so users can use
the existing anthropic/ provider with their Azure endpoint.

Closes #17066

* docs: Add alternative method for Azure AI Foundry using anthropic/ provider

Document that users can use anthropic/ provider with Azure endpoint
as an alternative to the dedicated azure_ai/ provider.
2025-12-02 09:08:10 -08:00
Sameer Kankute
4ac9e4c81c
Merge pull request #17345 from BerriAI/litellm_fix_jwt_auth_route_issue
Add other routes in jwt auth
2025-12-02 22:21:04 +05:30
Sameer Kankute
397aceced8
Merge pull request #17342 from BerriAI/litellm_fix_mcp_auth_header_forwarding
Fix: litellm user auth not passing issue
2025-12-02 22:20:33 +05:30
Sameer Kankute
18a9af3488
Merge pull request #17291 from BerriAI/litellm_fix_correct_attribute_error_code_raise
Fix 500 error for malformed request
2025-12-02 22:17:48 +05:30
Ishaan Jaff
1bb9e1bde8
[Feat] Add vllm batch+files API support (#15823)
* add OPENAI_COMPATIBLE_BATCH_AND_FILES_PROVIDERS

* fix use OPENAI_COMPATIBLE_BATCH_AND_FILES_PROVIDERS

* add _get_batch_job_total_usage_from_file_content

* fixes for vLLM + 12 labs async invoke

* fix: vLLM Batch APIs

* afile_retrieve

* test_hosted_vllm_full_workflow

* fix SERVER_URL for test
2025-12-02 08:41:50 -08:00
Alexsander Hamir
9ff2ecc16d
Fix: update default proxy_batch_write_at number (#17355)
The default is 10 seconds, not 30.
2025-12-02 05:52:00 -08:00
Alexsander Hamir
0acb7f418d
Fix (Docs) - Update default database connection limit #17353
We recommend using 10–20 connections. Our actual default is 10, but the documentation incorrectly listed the default as 100.
2025-12-02 05:35:58 -08:00
AlexsanderHamir
7324905c95 fix: update default database connection number 2025-12-02 05:29:03 -08:00
Sameer Kankute
6d296b1d25 Add other routes in jwt auth 2025-12-02 14:00:34 +05:30
Sameer Kankute
082c8af37f Fix: litellm user auth not passing issue 2025-12-02 11:25:32 +05:30
Krish Dholakia
4c7a988454
Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support (#17338)
* refactor(generic_guardrail_api.py): refactor to update to new guardrail api logic

* refactor: refactor llm api integrations to support passing in text as a list[str] instead of one at a time

* refactor: fix linting errors

* refactor: pass request type to guardrail api

allows request vs. response processing to occur

* feat: pass user api key dict information to the guardrail api

* fix: pass user api key dict information to the guardrail api

* feat: pass litellm call id + trace id, if present

* docs: update docs
2025-12-01 20:11:58 -08:00
YutaSaito
fe41e14951
fix: remove URL format validation for MCP server endpoints (#17270) 2025-12-01 20:08:54 -08:00
Korbinian Koch
6e8e3b30f9
Update Databricks model pricing and add new models (including databricks pricing test). (#17277)
* update databricks pricing and add DBU<>USD test

* Refactor test_databricks_pricing.py

Removed unnecessary sys.path modification and cleaned up comments.
2025-12-01 20:06:47 -08:00
codgician
e09e309371
feat(github-copilot): Add Embedding API support (#17278) 2025-12-01 20:05:28 -08:00
Boxuan Li
89458573a2
Add context window exception mapping for Together AI (#17284) 2025-12-01 20:02:59 -08:00
YutaSaito
da5b81c1ff
feat: add experimental latest-user filtering for Bedrock (#17282)
* feat: add experimental latest-user filtering for Bedrock

* doc: add experimental bedrock latest-message flag
2025-12-01 20:02:28 -08:00
Saar wintrov
860270a792
SSO: Clear sso integration for all users (#17287) 2025-12-01 20:01:36 -08:00
Cesar Garcia
01dfc3561a
Fix AttributeError when metadata is null in request body (#17263) (#17306)
Handle the case where metadata is explicitly set to null/None in the
request body. This was causing a 401 error with "'NoneType' object
has no attribute 'get'" when calling /v1/batches with metadata: null.

The fix uses `or {}` instead of a default dict value since the key
exists but has a None value.
2025-12-01 19:58:27 -08:00
Cesar Garcia
965406c643
feat(provider): add Z.AI (Zhipu AI) as built-in provider (#17307)
* feat(provider): add Z.AI (Zhipu AI) as built-in provider

Add support for Z.AI GLM models as a native OpenAI-compatible provider.

- Add "zai" to openai_compatible_providers list
- Add ZAI enum to LlmProviders
- Add provider URL resolution for https://api.z.ai/api/paas/v4
- Add 8 GLM models with pricing to model cost maps:
  - glm-4.6 (200K context, $0.6/$2.2 per 1M tokens)
  - glm-4.5, glm-4.5v, glm-4.5-x, glm-4.5-air, glm-4.5-airx
  - glm-4-32b-0414-128k
  - glm-4.5-flash (free tier)
- Add unit tests for provider integration

Closes #17289

* docs: add Z.AI provider documentation

- Add zai.md with usage examples, model list, and pricing
- Add to sidebars.js navigation
2025-12-01 19:56:47 -08:00
idola9
71efcb7115
Refactor Noma guardrail to use shared Responses transformation and include system instructions (#17315)
* Support system prompts in noma guardrails

* Use litellm util to covert chat completions to responses api
2025-12-01 19:56:14 -08:00
rioiart
98a244450e
Fix sso users not added to entra synced team (#17331)
* test: add failing tests for SSO user not added to Entra-synced teams bug

Adds tests reproducing the bug where new SSO users with teams=None
(from NewUserResponse) are not added to Entra ID synced teams because
add_missing_team_member() returns early when teams is None.

Tests demonstrate:
- NewUserResponse with teams=None fails to add user to teams (bug)
- LiteLLM_UserTable with teams=[] correctly adds user to teams (control)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: treat None as empty list in add_missing_team_member for new SSO users

Fixed bug where new SSO users logging in via Microsoft SSO were not added
to their Entra-synced teams. The issue was an early return when
user_info.teams is None (default for NewUserResponse). Now treats None
as an empty list so new users are properly added to all their SSO teams.

Location: litellm/proxy/management_endpoints/ui_sso.py:438-440

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-01 19:53:30 -08:00
rioiart
70126d9130
Fix/new org team validate against org (#17333)
* fix: skip user budget/model validation for org-scoped teams

When creating a team with organization_id, budget and model constraints
should be validated against the organization's limits, not the user's
personal limits. This allows org admins with restrictive personal
budgets to create teams within their organization's more generous limits.

Adds 4 unit tests to verify:
- Org-scoped teams bypass user budget validation
- Org-scoped teams bypass user model validation
- Standalone teams still validate against user limits

* fix: enforce user budget/model limits for standalone teams in update_team

- Add user-level budget and model validation to update_team endpoint for standalone teams,
  matching the existing pattern in new_team
- Org-scoped teams correctly bypass user validation and use organization limits instead
- Add 5 new comprehensive tests covering standalone/org team budget/model validation

* fix: Add direct TPM/RPM org limit validation and consolidate user team limit checks

- Add direct TPM/RPM comparison against org limits in _check_org_team_limits()
- Consolidate budget/models/TPM/RPM user validation into _check_user_team_limits() helper
- Ensure user limits only apply to standalone teams (organization_id=None)
- Org-scoped teams now validate TPM/RPM against org limits (not user limits)
- Add 8 tests for TPM/RPM validation scenarios (org and user limits)
- Reduce code duplication between new_team() and update_team()
2025-12-01 19:51:42 -08:00
Ishaan Jaff
1cdfb3da8f
[Bug Fix] - Fix litellm_enterprise ensure imported routes exist (#17337)
* test_enterprise_routes.py

* test_enterprise_routes_all_imports_exist
2025-12-01 19:14:12 -08:00
Sameer Kankute
289c13ca5d
Merge pull request #17260 from abi-jey/main
fix: GA path for azure openai realtime models
2025-12-02 08:34:10 +05:30