diff --git a/docs/my-website/docs/proxy/managed_batches.md b/docs/my-website/docs/proxy/managed_batches.md index 431d313fc18..4bd3b12d3af 100644 --- a/docs/my-website/docs/proxy/managed_batches.md +++ b/docs/my-website/docs/proxy/managed_batches.md @@ -260,4 +260,15 @@ print(f"status: {status}") When a `target_model_names` is specified, the file is written to all deployments that match the `target_model_names`. -No additional infrastructure is required. \ No newline at end of file +No additional infrastructure is required. + +## Could the batch be created at the eastus-01 deployment but a subsequent get of the batch could be routed to (a different) eastus2-01 deployment ? + +**A.** You can loadbalance b/w multiple models for the initial create batch. Once that's created - we return a file id, which encodes the model deployment used, so it's sticky and only sends any get/delete to that deployment. + + + + + + +