* feat: prisma migrate deploy with lock
Author: Mini Jeong <mini.jeong@navercorp.com>
* fix: use redis cache from proxy server
Author: Mini Jeong <mini.jeong@navercorp.com>
* fix: add type checks and fix unit tests for migration lock
- Add DATABASE_URL validation in _create_baseline_migration() and _resolve_all_migrations()
- Fix MyPy type errors by adding None checks before using database_url in subprocess calls
- Add _resolve_all_migrations mock to failing unit tests to prevent filesystem errors
- Apply Black formatting to modified files
Fixes:
- MyPy type errors: database_url could be None when passed to subprocess
- Unit test failures: _resolve_all_migrations tried to create directories in read-only /test path
* fix: resolve MyPy type error in vertex_ai vertex_llm_base
Fix MyPy type checking error where vertex_api_version parameter type
was incompatible with function signature expectation.
* fix: Return 403 exception when calling GET responses api
* fix: added new step into rotate master key function for processing credentials table
* Add redisvl in requirements.txt
* fix: fixed the issue of handling root paths when processing Discovery protected resource metadata and authorization server metadata URLs.
* fix: added additional grant type into oauth_authorization_server response for fixing mcp auth register bad request issue
* fix: added RFC RECOMMENDED property(scopes_supported) to protected resource and authorization server metadata
* fix: removed initialize the tool name to MCP server name mapping(oauth2) on startup for avoiding 401 error
* fix: upgraded mcp sdk depency version for fixing ClosedResourceError
* Use already configured opentelemetry providers
Users that instrument using opentelemetry-instrument can now setup exporters as per their environment.
* Handle all protocols for all telemetry
* Add more tests
* feat(mcp): parallelize tool fetching from multiple MCP servers (#18627)
* feat(mcp): parallelize tool fetching from multiple MCP servers
Replace sequential tool fetching with asyncio.gather() to reduce
client timeouts when using multiple MCP servers.
Changes:
- mcp_server_manager.py: list_tools() now fetches tools in parallel
- server.py: _get_tools_from_mcp_servers() now fetches tools in parallel
Real-world impact (7 MCP servers example):
- Sequential: ~4.5+ seconds (exceeds typical 5-second client timeouts)
- Parallel: ~1.2 seconds (max of all servers)
Fixes#18626
* fix: copy oauth2_headers to avoid shared dict mutation in parallel tasks
* feat: add display_name, model_vendor, and model_version metadata
* added the option of adding langsmith tenant id in the env (#18623)
* fix(router): Validate routing_strategy at startup to fail fast with helpful error. (#18624)
Invalid routing_strategy values (e.g., "simple" instead of "simple-shuffle") previously failed silently, causing confusing "No deployments available" errors downstream. This change adds upfront validation in routing_strategy_init() to:
- Check if the provided strategy matches valid string values or RoutingStrategy enum
- Raise a clear ValueError listing valid options if invalid
- Fail fast at startup instead of at request time
Fixes behavior reported in #11330 where users had to debug cryptic errors.
Valid strategies: simple-shuffle, least-busy, usage-based-routing, latency-based-routing, cost-based-routing, usage-based-routing-v2
Co-authored-by: Flibbert E. Gibbitz <flibbertygibbitz@runelabs.ai>
* Add libsndfile to database Docker image for audio processing (#18612)
The litellm-database Docker image was missing the libsndfile system
library, which is required by the soundfile Python package for audio
file processing. This caused failures when using audio transcription
endpoints that attempt to calculate audio duration.
This adds libsndfile to the runtime dependencies in Dockerfile.database,
consistent with Dockerfile.alpine which already includes this library.
* Fix: Map Gemini cached_tokens to Langfuse cache_read_input_tokens (#18614)
* Fix: Map Gemini cached_tokens to Langfuse cache_read_input_tokens
Fixes#18520
## Problem
Langfuse integration was not capturing cached tokens from Gemini models.
Gemini returns cached tokens in `usage.prompt_tokens_details.cached_tokens`,
but Langfuse only read from top-level `usage.cache_read_input_tokens`
(which only Anthropic populates).
## Solution
Updated langfuse.py to check both locations:
1. First check top-level cache_read_input_tokens (for Anthropic)
2. Then check prompt_tokens_details.cached_tokens (for Gemini, OpenAI, others)
This ensures all providers' cached tokens are properly reported to Langfuse.
## Changes
- Modified litellm/integrations/langfuse/langfuse.py (lines 742-761)
- Added 3 unit tests in tests/test_litellm/integrations/langfuse/test_gemini_cached_tokens.py
- All existing Langfuse tests still pass (11/11)
## Testing
- test_cached_tokens_extraction: Verifies Gemini cached_tokens extraction
- test_cached_tokens_not_present: Backward compatibility (no cached_tokens)
- test_cached_tokens_is_zero: Edge case when cached_tokens = 0
* Refactor: Extract cache token logic into helper function
Address review feedback from @officer47p
- Created _extract_cache_read_input_tokens() helper function
- Reduces code bloat in _log_langfuse_v2 method
- Improves testability and reusability
- All tests still passing (11/11)
* Adding Role Mappings
* Fixing Edit SSO Settings Modal
* feat: add user_mcp_management_mode for view_all visibility
* Fixing tests
* fix: missing mcp_allow_all_ui.png
* docs: add user_mcp_management_mode
* Align responses API streaming hooks with chat pipeline
* Clarify responses API streaming context
* Address review comments
* feat: Add GigaChat provider support (#18564)
* feat: Add GigaChat provider support
Add native support for GigaChat API (Sber AI, Russia's leading LLM).
Supported features:
- Chat completions (sync/async)
- Streaming (sync/async)
- Function calling / Tools
- Structured output via JSON schema (emulated through function calls)
- Image input (base64 and URL)
- Embeddings
Closes#18515
* fix: resolve mypy type errors in GigaChat handler
- Fix _prepare_file_data return type (use 3-tuple for cleaner type flow)
- Add type annotations for lists in _process_content_parts methods
- Add type annotations in _collapse_user_messages
- Use ChatCompletionToolCallChunk for proper tool_use typing
- Add type: ignore[override] for astreaming async generator
* refactor(gigachat): migrate to BaseConfig pattern
* fix: remove unused imports
* fix: resolve mypy type errors
* fix: mypy type errors
* refactor: address review feedback for GigaChat provider
- Remove singleton pattern, reuse litellm HTTPHandler
- Move constants/errors to transformation files, delete common_utils.py
- Add models to model_prices_and_context_window.json
- Fix ssl_verify not passed to HTTP client for embeddings
* docs: update GigaChat documentation with ssl_verify requirement
* Revert "Add redisvl in requirements.txt"
* Put reasoning summary behind feat flag
* fix: model eol
* fix: anthropic claude-3-opus-20240229 EOL
* Revert "fix: model eol"
This reverts commit 5aa1665d79.
* Fix: ImportError: qualifire package is required for QualifireGuardrail. Install it with: pip install qualifire
* fix: test_secret_manager_failure_does_not_block_email
* fix: test_update_ui_settings_allowlisted_value
* fix: test_aaamodel_prices_and_context_window_json_is_valid
* fix: test_all_models_have_display_name
* fix: async def test_bedrock_apply_guardrail_blocked()
* fix: test_databricks_embeddings[True]
* fix:test_anthropic_beta_header
* fix:test_api_error_handling
* fix:mypy mcp management
* Revert "feat(model_cost): add display_name, model_vendor, and model_version metadata to model entries"
* [Feat] New API Endpoint - Responses API (v1/responses/compact) (#18697)
* init transform_compact_response_api_request
* init acompact_responses
* init async_compact_response_api_handler in llm http handler
* init transform_compact_response_api_request for openai
* init acompact_responses
* fix acompact_responses
* add OAI Compact API
* docs responses API Compact
* code qa checks
* test_openai_compact_responses_api
* fix mypy linting
* fix: remove display name
* Add the LITELLM_REASONING_AUTO_SUMMARY in doc
* fix model map
* [UI] - Feat add request provider form on UI (#18704)
* add request provider form
* fix link to github
* add button
* fix link
* fix(streaming): normalize status code extraction to prevent 4xx errors from triggering mid-stream fallback (#18698)
在流式处理错误时，添加状态码标准化逻辑，确保 4xx 客户端错误直接抛出而不是被包装成 MidStreamFallbackError。
- 新增 _normalize_status_code 函数用于从异常对象提取状态码
- 优先从异常的 status_code 属性获取，其次从 response.status_code 获取
- 当映射异常或原始异常的状态码在 400-499 范围内时，直接抛出映射异常
- 添加单元测试验证 Vertex AI 400 错误正确抛出为 BadRequestError
- 确保流式处理中的客户端错误能够正确传播，而不会触发回退机制
---------
Co-authored-by: Eric84626 <lixiannan@gmail.com>
Co-authored-by: Eric84626 <97266539+Eric84626@users.noreply.github.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: mangabits <1457532+mangabits@users.noreply.github.com>
Co-authored-by: Costa Tsaousis <costa@tsaousis.gr>
Co-authored-by: Nik <nikolas.garza5@gmail.com>
Co-authored-by: Shivam Rawat <161387515+shivamrawat1@users.noreply.github.com>
Co-authored-by: FlibbertyGibbitz <seth@evenkeelconsultingllc.com>
Co-authored-by: Flibbert E. Gibbitz <flibbertygibbitz@runelabs.ai>
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: Yuta Saito <uc4w6c@bma.biglobe.ne.jp>
Co-authored-by: LingXuanYin <3546599908@qq.com>
Co-authored-by: YutaSaito <36355491+uc4w6c@users.noreply.github.com>
Co-authored-by: 0717376 <103773680+0717376@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Kris Xia <xiajiayi0506@gmail.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* init guardrails
* init guardrails
* some fixes
* some fixes
* ruff
* some fixes
* some fixes
* some fixes
* some fixes
* some fixes
* some fixes
* docs
Add support for Z.AI GLM-4.7, latest flagship model with enhanced reasoning capabilities.
Changes:
- Add zai/glm-4.7 to model pricing with /bin/bash.60/M input, .20/M output
- Add cached input pricing (/bin/bash.11/M) for GLM-4.7
- Add supports_reasoning flag to enable thinking parameter
- Update ZAIChatConfig to support thinking parameter for models with reasoning
- Update documentation with GLM-4.7 as latest flagship model
- Add cached input column to pricing table (GLM-4.7 only)
- Add tests for GLM-4.7 reasoning support and cost calculation
- Update all examples to use GLM-4.7
Model specifications:
- Context: 200K input, 128K output
- Supports: reasoning, function calling, tool choice, prompt caching
- Pricing: Same as GLM-4.6 with cache support
See: https://docs.z.ai/guides/llm/glm-4.7
Adds log_format parameter supporting json_array (default), ndjson, and single formats. NDJSON format enables webhook integrations like Sumo Logic to parse individual log records at ingest time. Defaults to json_array for backward compatibility.
* Allow get_nested_value dot notation to support escaping for Kubernetes JWT Support
* Add support for team and org alias fields, add docs, tests
* Fix lint issue with max statements in handle jwt logic
Got an error message:
{"error":{"message":"Invalid JSON payload: trailing comma is not allowed: line 8 column 8 (char 141)","type":"invalid_request_error","param":"request_body","code":"400"}}%
Remove 'none' from gpt-5-mini's supported reasoning_effort values in the documentation table. gpt-5-mini does not support reasoning_effort="none", only minimal, low, medium, and high.
- Add OAuth M2M (Machine-to-Machine) authentication via DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET
- Add Databricks SDK auto-auth with automatic credential discovery
- Add sensitive data redaction for secure logging (tokens, API keys, secrets)
- Add custom user_agent parameter for partner attribution in Databricks telemetry
- Support user_agent in LiteLLM Proxy via config.yaml litellm_params
- Add 49 mocked unit tests for all new functionality
- Add 13 E2E tests for real-world validation (skipped in CI)
- Update documentation with new features and examples
