mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-06 08:16:43 +00:00
16db51e2cf
Semantic cache keys omit the prompt, so every end user behind one virtual key shares a bucket and can be served another user's semantically similar response. Add an opt-in cache_params.semantic_cache_scope (key | end_user) that appends the authenticated end-user id to the tenant scope, read from metadata and litellm_metadata so /v1/chat/completions, /v1/responses and /v1/messages are all covered, falling back to the key scope when no end-user id is present. Expose the setting in the cache settings API and the Admin UI cache settings form Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||