* init guardrails
* init guardrails
* some fixes
* some fixes
* ruff
* some fixes
* some fixes
* some fixes
* some fixes
* some fixes
* some fixes
* docs
* Fix: Map Gemini cached_tokens to Langfuse cache_read_input_tokens
Fixes#18520
## Problem
Langfuse integration was not capturing cached tokens from Gemini models.
Gemini returns cached tokens in `usage.prompt_tokens_details.cached_tokens`,
but Langfuse only read from top-level `usage.cache_read_input_tokens`
(which only Anthropic populates).
## Solution
Updated langfuse.py to check both locations:
1. First check top-level cache_read_input_tokens (for Anthropic)
2. Then check prompt_tokens_details.cached_tokens (for Gemini, OpenAI, others)
This ensures all providers' cached tokens are properly reported to Langfuse.
## Changes
- Modified litellm/integrations/langfuse/langfuse.py (lines 742-761)
- Added 3 unit tests in tests/test_litellm/integrations/langfuse/test_gemini_cached_tokens.py
- All existing Langfuse tests still pass (11/11)
## Testing
- test_cached_tokens_extraction: Verifies Gemini cached_tokens extraction
- test_cached_tokens_not_present: Backward compatibility (no cached_tokens)
- test_cached_tokens_is_zero: Edge case when cached_tokens = 0
* Refactor: Extract cache token logic into helper function
Address review feedback from @officer47p
- Created _extract_cache_read_input_tokens() helper function
- Reduces code bloat in _log_langfuse_v2 method
- Improves testability and reusability
- All tests still passing (11/11)
This extends the previous fix to handle capitalized fields across ALL guardrail types,
including Presidio, Azure, Lakera, Bedrock, etc.
Discovery:
- Database investigation revealed the issue affects multiple guardrail types
- Found 4 affected guardrails in staging: 3 Presidio + 1 Azure
- All had default_action: 'Deny' causing the same validation failures
- The initial fix only covered ToolPermissionGuardrailConfigModel
Root Cause (Deeper):
- LitellmParams inherits from 13+ different guardrail config models
- Models use ConfigDict(extra="allow") allowing any field to be set
- Users can set default_action/on_disallowed_action on ANY guardrail type
- Only ToolPermissionGuardrailConfigModel was validating these fields
Solution:
- Added field validators to LitellmParams class (parent of all guardrails)
- Validators run for ALL guardrail types: Presidio, Azure, Bedrock, Lakera, etc.
- Added comprehensive tests covering multiple guardrail types
Changes:
- litellm/types/guardrails.py:
* Added @field_validator for default_action in LitellmParams
* Added @field_validator for on_disallowed_action in LitellmParams
* Added normalization in LitellmParams.__init__ as backup
* Imported field_validator from pydantic
- tests/test_litellm/types/test_guardrails_case_normalization.py:
* New test file with 7 tests covering multiple guardrail types
* Tests verify Presidio, Azure, Tool Permission, Lakera, Bedrock
* All tests passing
Impact:
- Previous fix: Only tool_permission guardrails protected
- This fix: ALL guardrail types now protected (13+ types)
- Handles both new writes and existing database records
- Tested against actual database with Presidio/Azure guardrails
Testing:
- 7 new cross-guardrail tests (all passing)
- 27 existing tool_permission tests (all passing)
- Verified fix works for real database records
The litellm-database Docker image was missing the libsndfile system
library, which is required by the soundfile Python package for audio
file processing. This caused failures when using audio transcription
endpoints that attempt to calculate audio duration.
This adds libsndfile to the runtime dependencies in Dockerfile.database,
consistent with Dockerfile.alpine which already includes this library.
Invalid routing_strategy values (e.g., "simple" instead of "simple-shuffle") previously failed silently, causing confusing "No deployments available" errors downstream. This change adds upfront validation in routing_strategy_init() to:
- Check if the provided strategy matches valid string values or RoutingStrategy enum
- Raise a clear ValueError listing valid options if invalid
- Fail fast at startup instead of at request time
Fixes behavior reported in #11330 where users had to debug cryptic errors.
Valid strategies: simple-shuffle, least-busy, usage-based-routing, latency-based-routing, cost-based-routing, usage-based-routing-v2
Co-authored-by: Flibbert E. Gibbitz <flibbertygibbitz@runelabs.ai>
This fixes a critical issue where capitalized values in tool_permission guardrail
configurations (e.g., "Deny" instead of "deny") caused Pydantic validation errors
during proxy startup, leading to repeated initialization failures and latency issues.
Problem:
- Users could save guardrails with capitalized values through UI/API
- Data was written to database without validation (e.g., default_action: "Deny")
- On proxy startup, loading from DB triggered strict Pydantic validation
- ValidationError caused guardrail initialization to fail in a retry loop
- This resulted in startup delays and repeated error logging
Root Cause:
- Write path had no case normalization
- Read path enforced strict lowercase Literal validation
- Asymmetry between write and read caused latent data corruption
Solution:
Added field validators to normalize case before Pydantic validation:
1. ToolPermissionRule.decision ("allow"/"deny")
- Normalizes decision field in rules array
2. ToolPermissionGuardrailConfigModel.default_action ("allow"/"deny")
- Normalizes default fallback action
3. ToolPermissionGuardrailConfigModel.on_disallowed_action ("block"/"rewrite")
- Normalizes disallowed tool behavior
4. ToolPermissionGuardrail.__init__ normalization
- Defensive normalization for direct instantiation
- Ensures normalization regardless of code path
Impact:
- Prevents validation errors during guardrail initialization
- Eliminates startup retry loops and latency issues
- Handles existing database records with capitalized values
- Accepts case-insensitive input from all sources (UI, API, direct calls)
- Fully backward compatible with existing lowercase configurations
Testing:
- Added 3 comprehensive tests for case-insensitive handling
- All 27 existing tests still pass
- Tests verify normalization across all affected fields
Files Changed:
- litellm/types/proxy/guardrails/guardrail_hooks/tool_permission.py
Added @field_validator decorators for case normalization
- litellm/proxy/guardrails/guardrail_hooks/tool_permission.py
Added runtime normalization in __init__ method
- tests/test_litellm/proxy/guardrails/guardrail_hooks/test_tool_permission.py
Added case-insensitive validation tests
