scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-05 08:07:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 136
litellm/litellm/batch_completion
History
mateo-berri 2708620d6a feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011)
2026-08-04 12:54:39 -07:00
..
main.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
Readme.md (fix) batch_completion fails with bedrock due to extraneous [max_workers] key (#6176) 2024-10-12 14:10:24 +05:30

Readme.md

Implementation of litellm.batch_completion, litellm.batch_completion_models, litellm.batch_completion_models_all_responses

Doc: https://docs.litellm.ai/docs/completion/batching

LiteLLM Python SDK allows you to:

  1. litellm.batch_completion Batch litellm.completion function for a given model.
  2. litellm.batch_completion_models Send a request to multiple language models concurrently and return the response as soon as one of the models responds.
  3. litellm.batch_completion_models_all_responses Send a request to multiple language models concurrently and return a list of responses from all models that respond.