Commit graph

3956 commits

Author SHA1 Message Date
AlexsanderHamir
d432c96baf fix(redis): handle float redis_version from AWS ElastiCache Valkey
AWS ElastiCache Valkey returns redis_version as a float (7.0) instead
of a string ('7.0.0'), causing AttributeError: 'float' object has no
attribute 'split' in async_lpop when parsing version for LPOP count.

Changes:
- Extract version parsing into _parse_redis_major_version() helper
- Add DEFAULT_REDIS_MAJOR_VERSION constant (replaces magic number)
- Support multiple version formats: string, float, int, malformed
- Add comprehensive test coverage for all version format edge cases

Fixes: 'LiteLLM Redis Cache LPOP: - Got exception from REDIS' error
during db_spend_update_job cronjobs
2025-11-13 09:24:47 -08:00
Ishaan Jaff
ca79ebe7d2 UI - Fix regression where Guardrail Entity Could not be selected and entity was not displayed (#16165)
* fix PiiEntityCategoryMap

* fix OpenAIChatCompletionsHandler

* fix lint
2025-11-01 18:02:13 -07:00
Ishaan Jaffer
3c0d530197 test_async_vertexai_response_basic 2025-11-01 10:58:48 -07:00
Ishaan Jaffer
a188e5f8e4 test_litellm_anthropic_prompt_caching_system 2025-11-01 10:51:15 -07:00
Ishaan Jaffer
2608d37e8e test prompt caching ant 2025-11-01 10:36:22 -07:00
Ishaan Jaffer
b41ad66e38 ant test prompt caching 2025-11-01 10:09:01 -07:00
Ishaan Jaffer
a45e2049f3 test_tool_call_with_empty_enum_property 2025-11-01 09:27:05 -07:00
Ishaan Jaffer
978c52291f test_rerank_custom_callbacks 2025-11-01 09:25:29 -07:00
Ishaan Jaffer
bf934c0799 test_anthropic_api_prompt_caching_basic_with_cache_creation 2025-11-01 09:21:42 -07:00
Ishaan Jaffer
14543324af test_team_budget_metrics 2025-11-01 09:21:17 -07:00
Ishaan Jaffer
f2b5a5f832 test google endpoints 2025-10-31 20:50:31 -07:00
Ishaan Jaffer
cd379c1321 test_anthropic_websearch 2025-10-31 20:43:07 -07:00
Ishaan Jaffer
b18cef300a undo test_anthropic_websearch change 2025-10-31 20:39:24 -07:00
Ishaan Jaffer
290e608818 get_computer_tool_beta_header 2025-10-31 20:38:37 -07:00
Ishaan Jaffer
014362cace fix ocr test 2025-10-31 20:32:03 -07:00
Ishaan Jaffer
a964e7f24c test_stream_token_counting_anthropic_with_include_usage 2025-10-31 20:24:58 -07:00
Ishaan Jaffer
2f1e947c44 test_anthropic_api_prompt_caching_basic 2025-10-31 19:32:46 -07:00
Ishaan Jaffer
27a98de600 test_health_and_chat_completion 2025-10-31 19:28:59 -07:00
Ishaan Jaffer
cd9cf2e6bd test fix 2025-10-31 19:23:08 -07:00
Ishaan Jaffer
c918dafb32 test_router_fallbacks_with_custom_model_costs 2025-10-31 19:23:08 -07:00
Ishaan Jaffer
6be9b97d15 test_stream_token_counting_anthropic_with_include_usage 2025-10-31 19:15:07 -07:00
Ishaan Jaffer
6e46824939 test_streaming_response 2025-10-31 19:10:38 -07:00
Ishaan Jaffer
515ae9680e test_anthropic_tool_use 2025-10-31 19:08:32 -07:00
Ishaan Jaffer
f173cb0f54 test_anthropic_text_editor 2025-10-31 19:07:46 -07:00
Ishaan Jaffer
928fddb3be test_anthropic_structured_output 2025-10-31 19:06:45 -07:00
Ishaan Jaffer
7f79abb552 test_aastreaming_tool_calls_valid_json_str 2025-10-31 19:05:31 -07:00
Ishaan Jaffer
2cd57540a4 fix test fixes 2025-10-31 18:31:00 -07:00
Ishaan Jaffer
22eb2f8033 Revert "Python entry-point for CustomLLM subclasses (#15881)"
This reverts commit 559ae96e38.
2025-10-31 18:24:39 -07:00
Ishaan Jaffer
94c2c28f3d claude-sonnet-4-5-20250929 fix 2025-10-31 18:20:52 -07:00
Ishaan Jaffer
159db27d5c fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
Ishaan Jaffer
16a1b57dd5 TestTavilySearch 2025-10-31 18:06:01 -07:00
Ishaan Jaff
c59a0c9681
[Feat] UI - Allow setting cache settings on UI (#16143)
* add LiteLLM_CacheConfig

* add CacheSettingsField

* add UI cache saver

* feat add cache_settings_router

* fix schema

* fix ssl_check_hostname

* refactor into utils

* add groups for field names

* add test_connection in base cache

* add test_connection inredis and redis cluster

* feat _decrypt_db_variables

* add cache settings endpoints

* test_test_cache_connection_calls_cache_test_connection_with_params

* fix: add switch_on_llm_response_caching

* feat use CacheSettingsManager

* feat use CacheSettingsManager

* TestCacheSettingsManager

* fix update_config

* Cache Field test
2025-10-31 17:43:59 -07:00
yuneng-jiang
725671395a
[Feature] UI - Config Guardrails should not be editable and guardrail info fix (#16142)
* UI Config Guardrails should not be editable and guardrails info definition location fix

* Remove unused import

* Added literals for guardrail definition location
2025-10-31 16:29:24 -07:00
Sameer Kankute
689b210c18
Add Xai websearch cost (#16001)
* Add xai websearch cost

* Add test for websearch cost in xai

* remove not required changes
2025-10-30 20:35:34 -07:00
Nicholas Couture
f747a4a38a
fix: Handle multiple rate limit types per descriptor and prevent IndexError (#16039)
* improve descriptor_key handling for multiple and missing rate limit descriptors in parallel request limiter v3

* Add tests for parallel request limiter v3 in proxy hooks
2025-10-30 20:12:54 -07:00
Sameer Kankute
c1369a07ba
Add Add per model group header forwarding for Bedrock Invoke API (#16042) 2025-10-30 20:10:17 -07:00
John Lathouwers
a6f740f28b
Add OCI Signer Authentication. Closes #16048, Closes #15654 (#16064)
* Add OCI Signer Authentication. Closes #16048,  Closes #15654

* Fix linting error

* Remove Recommended, Catch None, Trim Whitespace

* Make method clear re Proxy vs SDK
2025-10-30 19:59:01 -07:00
pale-aura
0d84c11bbe
Fix vercel_ai_gateway entry for glm-4.6 (#16084)
* Fix vercel_ai_gateway entry for glm-4.6

* Fix: add unit test for vercel_ai_gateway/zai/glm-4.6

* Fix:

Add fix for unit test on vercel_ai_gateway/zai/glm-4.6

* Remove print statements in test_vercel_ai_gateway_glm46_cost_math
2025-10-30 19:40:35 -07:00
Geoffray Viossat
b29ba9e3be
feat: handle Deepgram detected language when available (#16093) 2025-10-30 19:19:34 -07:00
Ishaan Jaff
fd073ee682
Litellm docs readme fixes (#16107)
* add provider test

* docs readme.md

* docs providers

* order providers

* test_providers_alphabetically_ordered

* docs endpoint

* fix config
2025-10-30 17:05:32 -07:00
yuneng-jiang
720ba865fb
[Infra] Litellm Backend SSO Changes (#16029)
* SSO Backend changes

* Encrypt and Decrypt, load into os env

* Linting and addressing comments
2025-10-30 14:32:08 -07:00
Sameer Kankute
eed3ad0bdb
Fix: Moderations endpoint now respects api_base configuration parameter (#16087)
* Update moderation to use api base

* Update moderation to use api base

* Fix mypy error
2025-10-30 11:01:26 -07:00
Sameer Kankute
192935120d
Add OpenAI-compatible annotations support for Cohere v2 citations 2025-10-29 19:12:17 -07:00
Talal
5e10ea4136
Improve(mcp): respect X-Forwarded- headers in OAuth endpoints (#16036)
* fix(mcp): respect X-Forwarded-Proto header in OAuth endpoints

When LiteLLM proxy is deployed behind a reverse proxy (like nginx or a load balancer) that terminates SSL/TLS, the proxy receives HTTP requests internally but should expose HTTPS URLs externally. This change detects the X-Forwarded-Proto header and uses it to construct correct redirect URIs and endpoint URLs.

Changes:
- Added X-Forwarded-Proto detection to authorize, token, oauth_protected_resource_mcp, oauth_authorization_server_mcp, and register_client endpoints
- Added comprehensive tests for X-Forwarded-Proto header support across all affected endpoints
- Fixed existing tests to properly mock request.headers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix formatting

* feat(mcp): support X-Forwarded-Host for proxy base URL reconstruction

Extended X-Forwarded-Proto support to also handle X-Forwarded-Host and X-Forwarded-Port headers. This allows LiteLLM to correctly construct redirect URIs and endpoint URLs when deployed behind a reverse proxy that changes the host/port.

Example scenario:
- Internal URL: http://localhost:8888/github/mcp
- External URL: https://proxy.abc.com/github/mcp
- Proxy sets: X-Forwarded-Proto: https, X-Forwarded-Host: proxy.abc.com

Changes:
- Added get_request_base_url() helper function to centralize X-Forwarded-* header handling
- Replaced all inline X-Forwarded-Proto checks with calls to the helper function
- Helper handles X-Forwarded-Proto, X-Forwarded-Host, and X-Forwarded-Port
- Added tests for X-Forwarded-Host scenarios in authorize and token endpoints

Fixes issue where protected resource URL mismatch occurred:
  Error: Protected resource http://proxy.abc.com:8888/github/mcp
  does not match expected https://proxy.abc.com/github/mcp

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: replace Yelp-specific hostnames with generic examples

Changed all references from chatproxy.yelpcorp.com to proxy.example.com in:
- test_proxy_forwarding.py (default host parameter)
- TEST_PROXY_FORWARDING.md (documentation examples)
- discoverable_endpoints.py (docstring example)
- test_discoverable_endpoints.py (test mock data)

This makes the code more generic and suitable for open source.
All 13 tests still passing.

* remove accidentally added files

* fix formatting

* add new test for get_base_url

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-29 19:11:32 -07:00
Jason Roberts
6672250769
feat(guardrails): Add per-request profile overrides to PANW Prisma AIRS (#16069)
- Per-request metadata: profile_name, profile_id, user_ip, app_name
- Full litellm_call_id for traceability
- Optional profile_name (supports API keys with linked profiles)
- Backward compatible
2025-10-29 19:05:59 -07:00
Ishaan Jaffer
044e26081d test_get_request_body_nova_canvas_inference_profile_arn 2025-10-29 18:30:59 -07:00
Ishaan Jaff
aea78b8d1a
[Feat] Add support for Batch API Rate limiting - PR1 adds support for input based rate limits (#16075)
* add count_input_file_usage

* add count_input_file_usage

* fix count_input_file_usage

* _get_batch_job_input_file_usage

* fixes imports

* use _get_batch_job_input_file_usage

* test_batch_rate_limits

* add _check_and_increment_batch_counters

* add get_rate_limiter_for_call_type

* test_batch_rate_limit_multiple_requests

* fixes for batch limits

* fix linting

* fix MYPY linting
2025-10-29 18:28:52 -07:00
Ishaan Jaffer
8a7f39daa4 tes numeric constants 2025-10-29 18:23:38 -07:00
OrionCodeDev
5f52533294
Fix spend tracking for OCR/aOCR requests (log pages_processed + recognize OCRResponse) (#16070)
* fix-ocr-cost

* fix1
2025-10-29 18:11:21 -07:00
Ishaan Jaff
99feefd614
[Feat] Add FAL AI Image Generations on LiteLLM (#16067)
* add fal-ai provider

* fix image_generation_handler

* init FalAIImageGenerationConfig

* init cost_calculator

* init FAL AI

* TestFAL_AI_ImageGeneration

* fix load_custom_provider_entrypoints

* TestFAL_AI_ImageGeneration

* add imagen4 transform FAL AI

* add FAL AI imagen 4 transform

* BaseImageGenTest

* test_fal_ai_image_generation_basic

* add BRIA + Recraft img gen

* add recraft + BRIA

* test_fal_ai_image_generation_basic

* tests for flux PRO v11

* Add FAL AI SD

* test FAL AI SD

* docs FAL AI

* docs fal ai

* Using Model-Specific Parameters

* add fal ai model prices

* add fall_ai JPG logo

* ui fixes FAL AI

* fix linting

* fix linting

* fix bedrock test_get_request_body_stability3

* test_custom_llm_provider_entrypoint
2025-10-29 13:10:51 -07:00