* [Feat] Add model parameter to Generic Guardrail API
Add model information to guardrail requests, allowing guardrails to make
model-specific security decisions.
Changes:
- Add `model` field to GenericGuardrailAPIInputs TypedDict
- Add `model` field to GenericGuardrailAPIRequest Pydantic model
- Update OpenAI and Anthropic handlers to pass model from request/response
- Add unit tests for model parameter handling
* [Feat] Add model parameter to all guardrail_translation handlers
Extend model parameter support to all guardrail handlers for consistent
implementation across all endpoint types:
- OpenAI Responses API (input/output + streaming)
- OpenAI Image Generation (input only)
- OpenAI Text Completion (input/output)
- OpenAI Text-to-Speech (input only)
- OpenAI Audio Transcription (output only)
- Cohere Rerank (input only)
- Pass-through Endpoints (input/output)
- MCP Server (input only)
This addresses the review feedback requesting consistent model parameter
handling across all guardrail_translation/handler.py files.
---------
Co-authored-by: Igal Boxerman <igal@pillar.security>
Google announced that Gemini 2.0 Flash and Flash Lite models will be discontinued on March 31, 2026. Updated deprecation_date field for all affected model variants across different providers (vertex_ai, gemini, deepinfra, openrouter, vercel_ai_gateway).
Models updated:
- gemini-2.0-flash (added deprecation date)
- gemini-2.0-flash-001 (updated from 2026-02-05)
- gemini-2.0-flash-lite (added deprecation date)
- gemini-2.0-flash-lite-001 (updated from 2026-02-25)
All variants now correctly reflect the March 31, 2026 shutdown date.
The RAG query endpoint was failing with "Object of type Router is not
JSON serializable" when called through the proxy. This was caused by two
issues:
1. The Router object passed via kwargs was leaking into the request
payload sent to providers like Bedrock, causing JSON serialization
errors.
2. The RAG query pipeline was calling litellm.acompletion() directly
instead of using the router, so virtual model names configured in the
proxy weren't being resolved to actual provider model IDs.
This fix:
- Extracts the router from kwargs and uses router.acompletion() when
available, falling back to litellm.acompletion() otherwise
- Adds "Router" to the list of non-serializable types in
filter_exceptions_from_params as a defensive measure
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Changed initial keyType state to 'llm_api' instead of 'default'
- Updated reset functions (handleOk, handleCancel) to use 'llm_api'
- Updated form initialValue and Select defaultValue to 'llm_api'
Addresses #19492
- Add @lru_cache decorator to get_model_info() and _cached_get_model_info_helper()
- Update _invalidate_model_cost_lowercase_map() to clear these caches when model_cost changes
- Update test to call cache invalidation after modifying litellm.model_cost
Reduces get_model_cost_information from 46% to <1% of request handling time.
* perf: Optimize use_custom_pricing_for_model with set intersection
Cache CustomPricingLiteLLMParams.model_fields.keys() as a module-level
frozenset and use set intersection to reduce loop iterations from 882k
to 90k (only iterating over keys that exist in both sets).
Performance improvement: 84% faster (6.3x speedup)
- Before: 1.17s total, 65µs per call
- After: 0.19s total, 10µs per call
* Use .get() for defensive dictionary access
* perf: Optimize strip_trailing_slash with O(1) index check
Replace rstrip("/") with direct index check for O(1) performance
instead of O(n) string scanning.
Results:
- strip_trailing_slash: 311ms → 13ms (96% faster)
- get_standard_logging_object_payload: 6.11s → 5.80s (5% faster)
* Handle multiple trailing slashes in strip_trailing_slash
Use rstrip for correctness when URL ends with "//" or more,
otherwise use O(1) index check for single trailing slash.
