scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-05 08:07:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 137

docs(caching.md): add doc on enabling caching for just rate limiting features

Browse source
This commit is contained in:
Krrish Dholakia 2024-08-20 22:10:42 -07:00
parent a7578e9f2c
commit de8eae20d9
1 changed files with 18 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
docs/my-website/docs/proxy/caching.md
View file

@ -185,6 +185,8 @@ $ litellm --config /path/to/config.yaml
</Tabs>
## Using Caching - /chat/completions
<Tabs>
@ -230,6 +232,22 @@ curl --location 'http://0.0.0.0:4000/embeddings' \
</TabItem>
</Tabs>
## Set cache for proxy, but not on the actual llm api call
Use this if you just want to enable features like rate limiting, and loadbalancing across multiple instances.
Set `supported_call_types: []` to disable caching on the actual api call.
```yaml
litellm_settings:
cache: True
cache_params:
type: redis
supported_call_types: []
```
## Debugging Caching - `/cache/ping`
LiteLLM Proxy exposes a `/cache/ping` endpoint to test if the cache is working as expected