scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102

(feat) working semantic cache on proxy

Browse source
This commit is contained in:
ishaan-jaff 2024-02-06 08:55:25 -08:00 • committed by Krrish Dholakia
parent cde96b8ba0
commit d05984ef13
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
litellm/proxy/proxy_config.yaml
View file

@ -77,7 +77,7 @@ litellm_settings:
cache_params:
type: "redis-semantic"
similarity_threshold: 0.8
redis_semantic_cache_embedding_model: azure-embedding-model
redis_semantic_cache_use_async: True
# cache: True
# setting callback class
# callbacks: custom_callbacks.proxy_handler_instance # sets litellm.callbacks = [proxy_handler_instance]