* init guardrails
* init guardrails
* some fixes
* some fixes
* ruff
* some fixes
* some fixes
* some fixes
* some fixes
* some fixes
* some fixes
* docs
* Fix: Map Gemini cached_tokens to Langfuse cache_read_input_tokens
Fixes#18520
## Problem
Langfuse integration was not capturing cached tokens from Gemini models.
Gemini returns cached tokens in `usage.prompt_tokens_details.cached_tokens`,
but Langfuse only read from top-level `usage.cache_read_input_tokens`
(which only Anthropic populates).
## Solution
Updated langfuse.py to check both locations:
1. First check top-level cache_read_input_tokens (for Anthropic)
2. Then check prompt_tokens_details.cached_tokens (for Gemini, OpenAI, others)
This ensures all providers' cached tokens are properly reported to Langfuse.
## Changes
- Modified litellm/integrations/langfuse/langfuse.py (lines 742-761)
- Added 3 unit tests in tests/test_litellm/integrations/langfuse/test_gemini_cached_tokens.py
- All existing Langfuse tests still pass (11/11)
## Testing
- test_cached_tokens_extraction: Verifies Gemini cached_tokens extraction
- test_cached_tokens_not_present: Backward compatibility (no cached_tokens)
- test_cached_tokens_is_zero: Edge case when cached_tokens = 0
* Refactor: Extract cache token logic into helper function
Address review feedback from @officer47p
- Created _extract_cache_read_input_tokens() helper function
- Reduces code bloat in _log_langfuse_v2 method
- Improves testability and reusability
- All tests still passing (11/11)
This extends the previous fix to handle capitalized fields across ALL guardrail types,
including Presidio, Azure, Lakera, Bedrock, etc.
Discovery:
- Database investigation revealed the issue affects multiple guardrail types
- Found 4 affected guardrails in staging: 3 Presidio + 1 Azure
- All had default_action: 'Deny' causing the same validation failures
- The initial fix only covered ToolPermissionGuardrailConfigModel
Root Cause (Deeper):
- LitellmParams inherits from 13+ different guardrail config models
- Models use ConfigDict(extra="allow") allowing any field to be set
- Users can set default_action/on_disallowed_action on ANY guardrail type
- Only ToolPermissionGuardrailConfigModel was validating these fields
Solution:
- Added field validators to LitellmParams class (parent of all guardrails)
- Validators run for ALL guardrail types: Presidio, Azure, Bedrock, Lakera, etc.
- Added comprehensive tests covering multiple guardrail types
Changes:
- litellm/types/guardrails.py:
* Added @field_validator for default_action in LitellmParams
* Added @field_validator for on_disallowed_action in LitellmParams
* Added normalization in LitellmParams.__init__ as backup
* Imported field_validator from pydantic
- tests/test_litellm/types/test_guardrails_case_normalization.py:
* New test file with 7 tests covering multiple guardrail types
* Tests verify Presidio, Azure, Tool Permission, Lakera, Bedrock
* All tests passing
Impact:
- Previous fix: Only tool_permission guardrails protected
- This fix: ALL guardrail types now protected (13+ types)
- Handles both new writes and existing database records
- Tested against actual database with Presidio/Azure guardrails
Testing:
- 7 new cross-guardrail tests (all passing)
- 27 existing tool_permission tests (all passing)
- Verified fix works for real database records
The litellm-database Docker image was missing the libsndfile system
library, which is required by the soundfile Python package for audio
file processing. This caused failures when using audio transcription
endpoints that attempt to calculate audio duration.
This adds libsndfile to the runtime dependencies in Dockerfile.database,
consistent with Dockerfile.alpine which already includes this library.
Invalid routing_strategy values (e.g., "simple" instead of "simple-shuffle") previously failed silently, causing confusing "No deployments available" errors downstream. This change adds upfront validation in routing_strategy_init() to:
- Check if the provided strategy matches valid string values or RoutingStrategy enum
- Raise a clear ValueError listing valid options if invalid
- Fail fast at startup instead of at request time
Fixes behavior reported in #11330 where users had to debug cryptic errors.
Valid strategies: simple-shuffle, least-busy, usage-based-routing, latency-based-routing, cost-based-routing, usage-based-routing-v2
Co-authored-by: Flibbert E. Gibbitz <flibbertygibbitz@runelabs.ai>
