scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-06 08:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 78

add config for setting up redis cluster

Browse source
This commit is contained in:
Ishaan Jaff 2024-09-07 09:37:23 -07:00
parent 9a9c0e42eb
commit 3bf2c06e06
2 changed files with 44 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

42
docs/my-website/docs/proxy/caching.md
View file

@ -54,6 +54,10 @@ litellm_caching:<hash>
#### Redis Cluster
<Tabs>
<TabItem value="redis-cluster-config" label="Set on config.yaml">
```yaml
model_list:
- model_name: "*"
@ -68,6 +72,44 @@ litellm_settings:
redis_startup_nodes: [{"host": "127.0.0.1", "port": "7001"}]
```
</TabItem>
<TabItem value="redis-env" label="Set on .env">
You can configure redis cluster in your .env by setting `REDIS_CLUSTER_NODES` in your .env
**Example `REDIS_CLUSTER_NODES`** value
```
REDIS_CLUSTER_NODES = "[{"host": "127.0.0.1", "port": "7001"}, {"host": "127.0.0.1", "port": "7003"}, {"host": "127.0.0.1", "port": "7004"}, {"host": "127.0.0.1", "port": "7005"}, {"host": "127.0.0.1", "port": "7006"}, {"host": "127.0.0.1", "port": "7007"}]"
```
:::note
Example python script for setting redis cluster nodes in .env:
```python
# List of startup nodes
startup_nodes = [
{"host": "127.0.0.1", "port": "7001"},
{"host": "127.0.0.1", "port": "7003"},
{"host": "127.0.0.1", "port": "7004"},
{"host": "127.0.0.1", "port": "7005"},
{"host": "127.0.0.1", "port": "7006"},
{"host": "127.0.0.1", "port": "7007"},
]
# set startup nodes in environment variables
os.environ["REDIS_CLUSTER_NODES"] = json.dumps(startup_nodes)
print("REDIS_CLUSTER_NODES", os.environ["REDIS_CLUSTER_NODES"])
```
:::
</TabItem>
</Tabs>
#### TTL
```yaml

2
litellm/tests/test_caching.py
View file

@ -838,6 +838,7 @@ async def test_redis_cache_cluster_init_unit_test():
@pytest.mark.asyncio
@pytest.mark.skip(reason="Local test. Requires running redis cluster locally.")
async def test_redis_cache_cluster_init_with_env_vars_unit_test():
try:
import json
@ -861,6 +862,7 @@ async def test_redis_cache_cluster_init_with_env_vars_unit_test():
# set startup nodes in environment variables
os.environ["REDIS_CLUSTER_NODES"] = json.dumps(startup_nodes)
print("REDIS_CLUSTER_NODES", os.environ["REDIS_CLUSTER_NODES"])
# unser REDIS_HOST, REDIS_PORT, REDIS_PASSWORD
os.environ.pop("REDIS_HOST", None)