scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-10 22:41:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 83

docs(managed_batches.md): update docs

Browse source
This commit is contained in:
Krrish Dholakia 2025-11-17 18:00:08 -08:00
parent e73ac28178
commit fdb5df1f27
1 changed files with 12 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

13
docs/my-website/docs/proxy/managed_batches.md
View file

@ -260,4 +260,15 @@ print(f"status: {status}")
When a `target_model_names` is specified, the file is written to all deployments that match the `target_model_names`.
No additional infrastructure is required.
No additional infrastructure is required.
## Could the batch be created at the eastus-01 deployment but a subsequent get of the batch could be routed to (a different) eastus2-01 deployment ?
**A.** You can loadbalance b/w multiple models for the initial create batch. Once that's created - we return a file id, which encodes the model deployment used, so it's sticky and only sends any get/delete to that deployment.