scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-10 22:41:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 82

doc: update load balancing and routing with enable_pre_call_checks (#18888)

Browse source
This commit is contained in:
Harshit Jain 2026-01-12 16:47:25 +05:30 • committed by GitHub
parent f9430d1cab
commit 45ac107bee
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 15 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
docs/my-website/docs/proxy/load_balancing.md
View file

@ -264,8 +264,15 @@ model_list:
model: azure/gpt-4-fallback
api_key: os.environ/AZURE_API_KEY_2
order: 2 # 👈 Used when order=1 is unavailable
router_settings:
enable_pre_call_checks: true # 👈 Required for 'order' to work
```
:::important
The `order` parameter requires `enable_pre_call_checks: true` in `router_settings`.
:::
If `order=1` deployment is unavailable (e.g., rate-limited), the router falls back to `order=2` deployments.
### When You'll See Load Balancing in Action

9
docs/my-website/docs/routing.md
View file

@ -861,9 +861,13 @@ model_list = [
},
]
router = Router(model_list=model_list)
router = Router(model_list=model_list, enable_pre_call_checks=True) # 👈 Required for 'order' to work
```
:::important
The `order` parameter requires `enable_pre_call_checks=True` to be set on the Router.
:::
</TabItem>
<TabItem value="proxy" label="PROXY">
@ -880,6 +884,9 @@ model_list:
model: azure/gpt-4-fallback
api_key: os.environ/AZURE_API_KEY_2
order: 2 # 👈 Used when order=1 is unavailable
router_settings:
enable_pre_call_checks: true # 👈 Required for 'order' to work
```
</TabItem>