mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
docs(reliability.md): add default fallbacks to proxy docs
This commit is contained in:
parent
2783953625
commit
06efe28132
1 changed files with 27 additions and 0 deletions
|
|
@ -434,6 +434,33 @@ litellm_settings:
|
|||
|
||||
|
||||
|
||||
### Default Fallbacks
|
||||
|
||||
You can also set default_fallbacks, in case a specific model group is misconfigured / bad.
|
||||
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: gpt-3.5-turbo-small
|
||||
litellm_params:
|
||||
model: azure/chatgpt-v-2
|
||||
api_base: os.environ/AZURE_API_BASE
|
||||
api_key: os.environ/AZURE_API_KEY
|
||||
api_version: "2023-07-01-preview"
|
||||
|
||||
- model_name: claude-opus
|
||||
litellm_params:
|
||||
model: claude-3-opus-20240229
|
||||
api_key: os.environ/ANTHROPIC_API_KEY
|
||||
|
||||
litellm_settings:
|
||||
default_fallbacks: ["claude-opus"]
|
||||
```
|
||||
|
||||
This will default to claude-opus in case any model fails.
|
||||
|
||||
A model-specific fallbacks (e.g. {"gpt-3.5-turbo-small": ["claude-opus"]}) overrides default fallback.
|
||||
|
||||
### Test Fallbacks!
|
||||
|
||||
Check if your fallbacks are working as expected.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue