The RAG query endpoint was failing with "Object of type Router is not
JSON serializable" when called through the proxy. This was caused by two
issues:
1. The Router object passed via kwargs was leaking into the request
payload sent to providers like Bedrock, causing JSON serialization
errors.
2. The RAG query pipeline was calling litellm.acompletion() directly
instead of using the router, so virtual model names configured in the
proxy weren't being resolved to actual provider model IDs.
This fix:
- Extracts the router from kwargs and uses router.acompletion() when
available, falling back to litellm.acompletion() otherwise
- Adds "Router" to the list of non-serializable types in
filter_exceptions_from_params as a defensive measure
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Changed initial keyType state to 'llm_api' instead of 'default'
- Updated reset functions (handleOk, handleCancel) to use 'llm_api'
- Updated form initialValue and Select defaultValue to 'llm_api'
Addresses #19492
- Add @lru_cache decorator to get_model_info() and _cached_get_model_info_helper()
- Update _invalidate_model_cost_lowercase_map() to clear these caches when model_cost changes
- Update test to call cache invalidation after modifying litellm.model_cost
Reduces get_model_cost_information from 46% to <1% of request handling time.
* perf: Optimize use_custom_pricing_for_model with set intersection
Cache CustomPricingLiteLLMParams.model_fields.keys() as a module-level
frozenset and use set intersection to reduce loop iterations from 882k
to 90k (only iterating over keys that exist in both sets).
Performance improvement: 84% faster (6.3x speedup)
- Before: 1.17s total, 65µs per call
- After: 0.19s total, 10µs per call
* Use .get() for defensive dictionary access
* perf: Optimize strip_trailing_slash with O(1) index check
Replace rstrip("/") with direct index check for O(1) performance
instead of O(n) string scanning.
Results:
- strip_trailing_slash: 311ms → 13ms (96% faster)
- get_standard_logging_object_payload: 6.11s → 5.80s (5% faster)
* Handle multiple trailing slashes in strip_trailing_slash
Use rstrip for correctness when URL ends with "//" or more,
otherwise use O(1) index check for single trailing slash.
The generic provider params update logic loop was unintentionaly overwriting `patterns` and `blocked_words` with empty values because these fields are managed by a separate component ContentFilterManager and not available in the main form values.
Changes:
- Excluded `patterns` and `blocked_words` from the generic provider params update loop in guardrail_info.tsx
- Ensured these fields are only added to the update payload when explicitly handled by the ContentFilterManager logic (detecting changes via `useRef`).
- Added a regression test in guardrail_info.test.tsx to verify that patterns are preserved when only the guardrail name is updated.
Fixes#19639
