mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-10 22:41:41 +00:00
docs: add redis initalization with kwargs
This commit is contained in:
parent
c619569604
commit
cbeaca071f
4 changed files with 28 additions and 1 deletions
|
|
@ -105,6 +105,14 @@ Then simply initialize:
|
|||
litellm.cache = Cache(type="redis")
|
||||
```
|
||||
|
||||
:::info
|
||||
Use `REDIS_*` environment variables as the primary mechanism for configuring all Redis client library parameters. This approach automatically maps environment variables to Redis client kwargs and is the suggested way to toggle Redis settings.
|
||||
:::
|
||||
|
||||
:::warning
|
||||
If you need to pass non-string Redis parameters (integers, booleans, complex objects), avoid `REDIS_*` environment variables as they may fail during Redis client initialization. Instead, pass them directly as kwargs to the `Cache()` constructor.
|
||||
:::
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="gcs" label="gcs-cache">
|
||||
|
|
|
|||
|
|
@ -282,6 +282,10 @@ Set either `REDIS_URL` or the `REDIS_HOST` in your os environment, to enable cac
|
|||
```
|
||||
|
||||
**Additional kwargs**
|
||||
:::info
|
||||
Use `REDIS_*` environment variables to configure all Redis client library parameters. This is the suggested mechanism for toggling Redis settings as it automatically maps environment variables to Redis client kwargs.
|
||||
:::
|
||||
|
||||
You can pass in any additional redis.Redis arg, by storing the variable + value in your os
|
||||
environment, like this:
|
||||
|
||||
|
|
@ -289,6 +293,17 @@ environment, like this:
|
|||
REDIS_<redis-kwarg-name> = ""
|
||||
```
|
||||
|
||||
For example:
|
||||
```shell
|
||||
REDIS_SSL = "True"
|
||||
REDIS_SSL_CERT_REQS = "None"
|
||||
REDIS_CONNECTION_POOL_KWARGS = '{"max_connections": 20}'
|
||||
```
|
||||
|
||||
:::warning
|
||||
**Note**: For non-string Redis parameters (like integers, booleans, or complex objects), avoid using `REDIS_*` environment variables as they may fail during Redis client initialization. Instead, use `cache_kwargs` in your router configuration for such parameters.
|
||||
:::
|
||||
|
||||
[**See how it's read from the environment**](https://github.com/BerriAI/litellm/blob/4d7ff1b33b9991dcf38d821266290631d9bcd2dd/litellm/_redis.py#L40)
|
||||
|
||||
#### Step 3: Run proxy with config
|
||||
|
|
|
|||
|
|
@ -339,7 +339,7 @@ router_settings:
|
|||
| stream_timeout | Optional[float] | The default timeout for a streaming request. If not set, the 'timeout' value is used. |
|
||||
| debug_level | Literal["DEBUG", "INFO"] | The debug level for the logging library in the router. Defaults to "INFO". |
|
||||
| client_ttl | int | Time-to-live for cached clients in seconds. Defaults to 3600. |
|
||||
| cache_kwargs | dict | Additional keyword arguments for the cache initialization. |
|
||||
| cache_kwargs | dict | Additional keyword arguments for the cache initialization. Use this for non-string Redis parameters that may fail when set via `REDIS_*` environment variables. |
|
||||
| routing_strategy_args | dict | Additional keyword arguments for the routing strategy - e.g. lowest latency routing default ttl |
|
||||
| model_group_alias | dict | Model group alias mapping. E.g. `{"claude-3-haiku": "claude-3-haiku-20240229"}` |
|
||||
| num_retries | int | Number of retries for a request. Defaults to 3. |
|
||||
|
|
|
|||
|
|
@ -1333,6 +1333,10 @@ router = Router(model_list: Optional[list] = None,
|
|||
cache_responses=True)
|
||||
```
|
||||
|
||||
:::info
|
||||
When configuring Redis caching in router settings, use `cache_kwargs` to pass additional Redis parameters, especially for non-string values that may fail when set via `REDIS_*` environment variables.
|
||||
:::
|
||||
|
||||
## Pre-Call Checks (Context Window, EU-Regions)
|
||||
|
||||
Enable pre-call checks to filter out:
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue