scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-10 08:28:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 108
litellm/litellm/llms
History
Simon aa769fbc69
fix(streaming): surface upstream abort as server errors (LIT-10159) (#48328) 
* fix(streaming): surface upstream abort finish reasons as errors

* fix(streaming): keep SDK chunks with null choices streaming past the abort check

* fix(streaming): treat null choices as empty in the raw SSE abort check

* refactor(streaming): bind raw SSE choices as Final instead of rebinding

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-28 13:18:05 -07:00
..
a2a refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
ai21/chat refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
aiml fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
aiohttp_openai/chat fix(aiohttp_openai): forward service_tier in transform_response (#47263) 2026-10-28 13:14:44 -07:00
amazon_nova refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
anthropic fix(responses)!: answer 400 before counting a lone Responses built-in tool on Claude (#48798) 2026-10-28 12:12:45 -07:00
apiserpent fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
aws_polly fix(aws_polly): read the engine from the bare model name (#47963) 2026-10-23 18:10:51 -07:00
azure fix(azure): share one Entra token request per worker and audit #47348 (#48769) 2026-10-28 12:24:20 -07:00
azure_ai refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
base_llm fix(gemini)!: accept Google's current Managed Agents fields on create and interactions (#48750) 2026-10-28 12:37:59 -07:00
baseten refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
bedrock fix(responses)!: answer 400 before counting a lone Responses built-in tool on Claude (#48798) 2026-10-28 12:12:45 -07:00
bedrock_mantle refactor(llms): rename model-version transformation files to family names (#48509) 2026-10-26 18:53:16 -07:00
bespoke/decisions feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260) 2026-10-14 08:11:11 -07:00
black_forest_labs refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
brave/search chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
bytez refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
cerebras refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
chatgpt fix(chatgpt): derive the upstream session id from prompt_cache_key (#42014) 2026-10-28 10:57:51 -07:00
clarifai/chat refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
claude_code refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
cloudflare fix(cloudflare): clarify chat and decisions endpoints (#47965) (#48265) 2026-10-25 00:29:48 +00:00
codestral/completion refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
codex refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
cohere fix(cohere): map string stop parameter to list of strings for stop_sequences (#47091) 2026-10-27 19:01:54 -07:00
cometapi refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
compactifai refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
custom_httpx fix(anthropic-messages): merge deployment and client anthropic-beta instead of overwriting (#48594) 2026-10-28 18:20:06 +00:00
dashscope refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
databricks refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
dataforseo/search chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
datarobot/chat refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
deepagents refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
deepgram refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
deepinfra refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
deepseek refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
deprecated_providers refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
docker_model_runner/chat fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
duckduckgo/search fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
e2b fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
edenai refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
elevenlabs fix(elevenlabs): map wav and pcm TTS formats to 24kHz output formats (#48455) 2026-10-26 13:42:01 -07:00
empower/chat
exa_ai/search fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
fal_ai refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
fastcrw fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
featherless_ai/chat refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
firecrawl fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
fireworks_ai fix(decisions): preserve missing native usage (#48147) (#48261) 2026-10-24 17:37:38 -07:00
friendliai/chat fix(friendliai): drop litellm-internal message fields before sending (#45092) 2026-10-20 01:22:07 +00:00
galadriel/chat
gdc refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
gemini fix(gemini)!: accept Google's current Managed Agents fields on create and interactions (#48750) 2026-10-28 12:37:59 -07:00
gigachat refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
github/chat
github_copilot refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
google_pse/search chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
gradient_ai/chat refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
groq fix(groq): allow streamed reasoning replies in history (#47931) (#48268) 2026-10-25 00:52:01 +00:00
heroku/chat fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
hosted_vllm feat(hosted_vllm): accept images on decision deployments (#48621) 2026-10-27 12:43:36 -07:00
huggingface fix(huggingface): stop forcing truncate=false on TEI rerank and pass truncate params through (#48706) 2026-10-28 09:27:26 -07:00
hyperbolic refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
inception refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
infinity fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
jina_ai fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
lambda_ai refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
langflow refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
langgraph refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
laya feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260) 2026-10-14 08:11:11 -07:00
lemonade refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
linkup fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
litellm_proxy refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
llamafile/chat refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
lm_studio feat(decisions): support LM Studio decision models (#48448) 2026-10-27 02:24:09 +00:00
manus fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
meta/realtime fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939) 2026-10-16 15:24:40 +00:00
meta_llama/chat feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
microsoft_365_copilot fix(router)!: bound fake-streamed calls by the request timeout, not stream_timeout (#47162) 2026-10-17 22:08:02 +00:00
milvus/vector_stores refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
minimax refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
mistral fix(mistral): replay assistant reasoning as a Mistral thinking chunk (#48778) 2026-10-28 11:41:33 -07:00
modelscope refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
mongodb fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
moonshot/chat fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
morph feat(morph): support open models and cached input pricing (#48676) 2026-10-27 18:06:14 -07:00
nace/decisions feat: add Nace Drex decisions provider (#46523) 2026-10-16 18:07:50 -07:00
nadir/chat refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
nebius feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
nimble fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
nlp_cloud chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
novita/chat fix(openai-compat): send provider attribution headers on the default SDK path (+ Perplexity) (#44291) 2026-10-07 13:21:27 -07:00
nscale/chat refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
nvidia_nim refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
nvidia_riva refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
oci refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
ollama feat(decisions): support Ollama decision models (#48871) 2026-10-28 12:35:26 -07:00
oobabooga fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
openai fix(streaming): surface upstream abort as server errors (LIT-10159) (#48328) 2026-10-28 13:18:05 -07:00
openai_like feat(providers): add ZeroGPU as an OpenAI-compatible provider (#43874) 2026-10-28 12:40:32 -07:00
opencode refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
openrouter fix(decisions): preserve missing native usage (#48147) (#48261) 2026-10-24 17:37:38 -07:00
opensandbox refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
ovhcloud fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
parallel_ai/search perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
pass_through fix(guardrails): exclude litellm_metadata from pass-through guardrail scan text (#48276) 2026-10-26 12:07:58 -07:00
perplexity refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
petals refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
pg_vector/vector_stores refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
pi feat(harness): add pi harness for litellm.agent() (#45467) 2026-10-20 09:01:40 +00:00
predibase fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
ragflow fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
recraft refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
reducto refactor(ocr): remove the Python OCR execution path and require the Rust route (#43081) 2026-09-24 18:18:50 -07:00
replicate refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
runwayml refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
s3_vectors fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
sagemaker refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
sail chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
sambanova fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
sap refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
scaledown fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
scaleway fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
searchapi chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
searxng fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
serper/search fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
snowflake fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
soniox fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
stability refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
strands_decider/decisions feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678) 2026-10-09 22:51:19 -07:00
tavily/search fix(tavily): return raw_content, score, favicon and the response envelope from Tavily search (#48684) 2026-10-27 18:52:06 -07:00
tencent refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
tinyfish/search refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
together_ai fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
tool_loop feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391) 2026-10-03 17:33:01 +00:00
topaz fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
triton refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
typesafe/decisions fix(decisions): read a deployment's own supports_box_questions flag before the cost map (#46696) 2026-10-15 12:34:36 -07:00
v0 refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
valkey perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
vercel_ai_gateway refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
vertex_ai fix(vertex-ai): preserve image size aspect ratios (#48463) 2026-10-28 17:47:47 +00:00
vllm fix(prompt-templates): treat initial_prompt_value and final_prompt_value as optional (#39762) 2026-10-16 10:30:59 +00:00
volcengine refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
voyage refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
wandb chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
watsonx refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
xai feat(search): add native xAI X Search support (#38505) (#48677) 2026-10-28 00:28:51 +00:00
xinference/image_generation feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
you_com fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
zai refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
__init__.py refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
base.py refactor(types): replace Any with proven types in 24 files (#46663) 2026-10-15 06:10:12 -07:00
compaction.py feat(router): native compact-to-fit across conversation APIs (#42074) 2026-09-21 22:52:29 -07:00
custom_llm.py refactor(types): replace Any with proven types in 24 files (#46663) 2026-10-15 06:10:12 -07:00
maritalk.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
oss_decision.py feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260) 2026-10-14 08:11:11 -07:00
README.md

README.md

File Structure

August 27th, 2024

To make it easy to see how calls are transformed for each model/provider:

we are working on moving all supported litellm providers to a folder structure, where folder name is the supported litellm provider name.

Each folder will contain a *_transformation.py file, which has all the request/response transformation logic, making it easy to see how calls are modified.

E.g. cohere/, bedrock/.