mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-12 08:42:18 +00:00
373fe799e7
* fix(router): send managed batch uploads to batch deployments when a chat deployment has a lower order A purpose=batch upload on a mixed Azure chat and batch group was order-filtered before batch routing narrowed it, so the file landed only on the order-1 chat deployment and the batch create that followed got Azure's 400 invalid_deployment_type. The upload now runs under the same batch routing as the batch create. * fix(router): keep a failed batch upload's cooldown and order fallback on the batch deployments * fix(router): keep a batch upload off chat deployments while its batch deployment cools down * fix(router): keep a refused batch upload's retry off the chat deployment * fix(router): keep a refused batch upload's error and skip chat while the batch deployment is paused * test(router): pin the order fallback for a batch create on a group with no batch deployment * test(integration): expect an Azure batch upload on the batch deployment alone --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| cache | ||
| logger | ||
| model_cost | ||
| redis | ||
| router | ||
| routes | ||
| secrets | ||
| callable.rs | ||
| coercion.rs | ||
| credentials.rs | ||
| db.rs | ||
| diagnostics.rs | ||
| errors.rs | ||
| execution.rs | ||
| http.rs | ||
| lib.rs | ||
| marshal.rs | ||
| model_registry.rs | ||
| preflight.rs | ||
| python_settings.rs | ||
| tokenizer.rs | ||