scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-21 00:21:49 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 74
litellm/litellm/batch_completion
History
mateo-berri b604e2b20c refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 
About 35,000 fixes ruff marks safe across 32 rules (UP006/UP045/UP007
modern annotations, UP032 f-strings, SIM114/SIM118, RET501, and
friends), removal of the 1,296 typing imports the rewrite orphaned, and
hand fixes for what the fixers could not see: five star-import
freeloaders of typing names, two F823 late-import annotations, the
/get/config/list introspection crash on types.UnionType, redundant
function-local RoleMappings imports in ui_sso.py that shadowed the
module-level name once the annotation lost its quotes, and one FURB168
tautology.

B009/B010/PIE804/RUF019 are excluded on purpose: their safe fixes
rewrite getattr/setattr/**-splat/key-in-dict escape hatches into forms
basedpyright then rejects (283 new errors measured), so their budgets
stay at base values.

ruff-strict-budget.json drops by 39,579 this commit (39,968 across the
branch) with 28 rules at an actual 0 and 9 more sharply down.
type-discipline-budget.json ratchets LIT002/LIT006/LIT009 down; LIT001
moves to the now-honest total: the checker matches the spelling `set`
but not the alias `Set`, so the 160 typing.Set annotations rewritten to
set[...] were always mutable-set annotations and only now count.
2026-08-01 15:43:29 -07:00
..
main.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
Readme.md (fix) batch_completion fails with bedrock due to extraneous [max_workers] key (#6176) 2024-10-12 14:10:24 +05:30

Readme.md

Implementation of litellm.batch_completion, litellm.batch_completion_models, litellm.batch_completion_models_all_responses

Doc: https://docs.litellm.ai/docs/completion/batching

LiteLLM Python SDK allows you to:

  1. litellm.batch_completion Batch litellm.completion function for a given model.
  2. litellm.batch_completion_models Send a request to multiple language models concurrently and return the response as soon as one of the models responds.
  3. litellm.batch_completion_models_all_responses Send a request to multiple language models concurrently and return a list of responses from all models that respond.