(docs) litellm semantic caching

This commit is contained in:
ishaan-jaff 2024-02-06 10:54:55 -08:00 committed by Krrish Dholakia
parent b49b37568a
commit 102f20fc03
2 changed files with 2 additions and 1 deletions

View file

@ -104,7 +104,7 @@ litellm.cache = Cache(
host=os.environ["REDIS_HOST"],
port=os.environ["REDIS_PORT"],
password=os.environ["REDIS_PASSWORD"],
similarity_threshold=0.8,
similarity_threshold=0.8, # similarity threshold for cache hits, 0 == no similarity, 1 = exact matches, 0.5 == 50% similarity
redis_semantic_cache_embedding_model="text-embedding-ada-002", # this model is passed to litellm.embedding(), any litellm.embedding() model is supported here
)
response1 = completion(

View file

@ -7,6 +7,7 @@ Cache LLM Responses
LiteLLM supports:
- In Memory Cache
- Redis Cache
- Redis Semantic Cache
- s3 Bucket Cache
## Quick Start - Redis, s3 Cache, Semantic Cache