From fdb5df1f27d554baf5a65bb8cf6309c1539c309a Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Mon, 17 Nov 2025 18:00:08 -0800 Subject: [PATCH] docs(managed_batches.md): update docs --- docs/my-website/docs/proxy/managed_batches.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/docs/my-website/docs/proxy/managed_batches.md b/docs/my-website/docs/proxy/managed_batches.md index 431d313fc18..4bd3b12d3af 100644 --- a/docs/my-website/docs/proxy/managed_batches.md +++ b/docs/my-website/docs/proxy/managed_batches.md @@ -260,4 +260,15 @@ print(f"status: {status}") When a `target_model_names` is specified, the file is written to all deployments that match the `target_model_names`. -No additional infrastructure is required. \ No newline at end of file +No additional infrastructure is required. + +## Could the batch be created at the eastus-01 deployment but a subsequent get of the batch could be routed to (a different) eastus2-01 deployment ? + +**A.** You can loadbalance b/w multiple models for the initial create batch. Once that's created - we return a file id, which encodes the model deployment used, so it's sticky and only sends any get/delete to that deployment. + + + + + + +