scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-11 03:38:38 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 82
litellm/litellm/llms
History
devin-ai-integration[bot] 01b4ffe16b
fix(bedrock_mantle): route Claude chat completions to the native Messages endpoint (#43646) 
* fix(bedrock_mantle): route Claude chat completions to the native Messages endpoint

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock_mantle): price Claude chat on the Mantle row and route region-prefixed ids

A Mantle request always carries a region, so a Claude id with no bedrock_mantle/<region>/ row fell through the model-info lookup to the bare Bedrock row, which the bedrock provider family also matches, and billed about 10 percent under the Mantle price. The lookup now tries the provider's region-free row before the bare model. The Claude route test asserts the Mantle row, a region-prefixed Claude id is covered end to end, and the provider config map references the Mantle config directly.

* test(bedrock_mantle): cover supported params for Claude and open-weight Mantle ids

* test(bedrock_mantle): audit the Claude chat bridge on the integration rig

Adds the deterministic cells from the /audit of the Mantle Claude chat
bridge: wire-level translation on every chat, responses, and messages
route, SigV4 and bearer auth, region prefixes, api_base suffixes,
unsupported params with and without drop_params, malformed model ids,
upstream errors, the response-cache hit, the health check, pricing from
the Mantle row for Claude and non-Claude ids with a bare Bedrock twin,
and chaos cells for a mixed burst, an upstream outage, slow streams,
and a worker kill on an owned two-worker proxy

---------

Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-03 15:24:41 -07:00
..
a2a chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
ai21/chat
aiml fix(cost): honor deployment pricing for image generation (#39311) 2026-09-22 11:40:39 -07:00
aiohttp_openai/chat feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
amazon_nova feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
anthropic fix(anthropic): stop repeating streamed thinking text in the signature chunk (#44127) 2026-10-02 19:38:41 -07:00
apiserpent
aws_polly refactor(typing): replace Any with proven types in 89 more backend files 2026-09-02 23:08:42 +00:00
azure chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
azure_ai chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
base_llm fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of #38210) (#44134) 2026-10-03 19:29:15 +00:00
baseten
bedrock fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of #38210) (#44134) 2026-10-03 19:29:15 +00:00
bedrock_mantle fix(bedrock_mantle): route Claude chat completions to the native Messages endpoint (#43646) 2026-10-03 15:24:41 -07:00
black_forest_labs feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
brave/search feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
bytez fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of #38210) (#44134) 2026-10-03 19:29:15 +00:00
cerebras fix(cerebras): add max_retries and extra_headers to get_supported_openai_params (#36601) 2026-08-25 14:10:56 -07:00
chatgpt fix(chatgpt): preserve requested service tier in Responses calls (#40108) 2026-10-02 10:57:37 -07:00
clarifai/chat feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
claude_code refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
cloudflare feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers (#44236) 2026-10-03 17:38:38 +00:00
codestral/completion refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00
codex feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
cohere refactor(ocr): remove the Python OCR execution path and require the Rust route (#43081) 2026-09-24 18:18:50 -07:00
cometapi fix(cost): honor deployment pricing for image generation (#39311) 2026-09-22 11:40:39 -07:00
compactifai feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
custom_httpx fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of #38210) (#44134) 2026-10-03 19:29:15 +00:00
dashscope chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
databricks fix(streaming): keep the served service_tier on streamed chunks and spend rows (#42870) 2026-09-29 12:54:17 -07:00
dataforseo/search refactor(typing): replace Any with proven types in 42 more backend files 2026-09-02 15:35:01 +00:00
datarobot/chat
deepagents feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391) 2026-10-03 17:33:01 +00:00
deepgram fix(deepgram): forward only the first model and language values to /listen 2026-09-18 21:29:29 +00:00
deepinfra refactor(types): replace Any with precise types across 73 modules 2026-09-01 11:05:02 +00:00
deepseek chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
deprecated_providers feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
docker_model_runner/chat
duckduckgo/search
e2b Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 06:03:33 -07:00
edenai chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
elevenlabs fix(params): keep _litellm_* kwargs out of provider request bodies by construction (#43221) 2026-09-26 15:20:09 -07:00
empower/chat
exa_ai/search fix(exa): fall back to highlights/summary when text is missing (#42213) 2026-10-02 14:15:15 -07:00
fal_ai chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
fastcrw
featherless_ai/chat
firecrawl
fireworks_ai chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
friendliai/chat
galadriel/chat
gdc refactor(types): replace Any with real types across 29 more backend files 2026-09-08 10:14:07 +00:00
gemini feat(interactions): durable cross-pod settlement for background interaction billing (#41955) 2026-10-02 21:08:38 -07:00
gigachat chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
github/chat
github_copilot refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
google_pse/search
gradient_ai/chat
groq chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
heroku/chat
hosted_vllm chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
huggingface feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
hyperbolic
inception
infinity Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 06:03:33 -07:00
jina_ai refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00
lambda_ai
langflow feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
langgraph fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of #38210) (#44134) 2026-10-03 19:29:15 +00:00
laya feat: add Bespoke Nimble gateway and OSS classifier support (#44246) 2026-10-02 19:37:08 -07:00
lemonade feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
linkup
litellm_proxy chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
llamafile/chat
lm_studio
manus
meta/realtime chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
meta_llama/chat
milvus/vector_stores refactor(s3_vectors): embed search queries through the shared vector store executor 2026-09-02 19:26:31 -07:00
minimax refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
mistral chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
modelscope
mongodb chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
moonshot/chat fix(moonshot, together_ai): send the reasoning effort Kimi K3 accepts (#38611) 2026-08-27 20:46:46 -07:00
morph
nadir/chat chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
nebius
nimble chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
nlp_cloud feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
novita/chat
nscale/chat
nvidia_nim chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
nvidia_riva refactor(types): replace Any with real types across 11 more backend files 2026-09-08 14:09:22 +00:00
oci fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of #38210) (#44134) 2026-10-03 19:29:15 +00:00
ollama refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
oobabooga feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
openai fix(guardrails): run unified guardrails on /v1/images/edits (#44195) 2026-10-02 21:14:35 -07:00
openai_like chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
opencode feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
openrouter feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers (#44236) 2026-10-03 17:38:38 +00:00
opensandbox Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 06:03:33 -07:00
ovhcloud
parallel_ai/search style(websearch): apply ruff format to changed files 2026-09-09 08:55:25 -04:00
pass_through refactor(typing): replace Any with proven types in 42 more backend files 2026-09-02 15:35:01 +00:00
perplexity feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers (#44236) 2026-10-03 17:38:38 +00:00
petals feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
pg_vector/vector_stores refactor(s3_vectors): embed search queries through the shared vector store executor 2026-09-02 19:26:31 -07:00
predibase feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
ragflow Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5_r4 2026-09-03 06:44:40 +00:00
recraft fix(cost): honor deployment pricing for image generation (#39311) 2026-09-22 11:40:39 -07:00
reducto refactor(ocr): remove the Python OCR execution path and require the Rust route (#43081) 2026-09-24 18:18:50 -07:00
replicate feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
runwayml fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
s3_vectors fix(s3_vectors): embed registered-store ingests with the store's embedding model 2026-09-19 04:31:15 -07:00
sagemaker fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of #38210) (#44134) 2026-10-03 19:29:15 +00:00
sail chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
sambanova
sap refactor(types): replace Any with proven types in 5 files (#42722) 2026-09-23 03:14:45 -07:00
scaleway refactor: clean up fresh tech debt from 2026-10-02 (#44362) 2026-10-03 02:12:03 -07:00
searchapi
searxng
serper/search
snowflake chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
soniox refactor(typing): replace Any with proven types in 89 more backend files 2026-09-02 23:08:42 +00:00
stability feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
strands_decider/decisions feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers (#44236) 2026-10-03 17:38:38 +00:00
tavily/search
tencent refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
tinyfish/search chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
together_ai chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
tool_loop feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391) 2026-10-03 17:33:01 +00:00
topaz feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
triton feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
typesafe/decisions feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers (#44236) 2026-10-03 17:38:38 +00:00
v0
valkey chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
vercel_ai_gateway refactor: daily fresh tech debt cleanup, rolling PR (#42710) 2026-09-24 04:17:23 -07:00
vertex_ai fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of #38210) (#44134) 2026-10-03 19:29:15 +00:00
vllm
volcengine refactor(types): replace Any with real types across 54 more backend files 2026-09-08 12:41:07 +00:00
voyage Merge remote-tracking branch 'origin/main' into litellm_decrease_anys_opus5_r5 2026-09-21 12:57:38 -07:00
wandb chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
watsonx feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
xai chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
xinference/image_generation
you_com
zai
__init__.py fix(gemini): bill Google Maps grounding as its own SKU 2026-08-26 15:31:27 -07:00
base.py refactor(types): replace Any with real types across 178 backend files 2026-08-27 10:14:58 +00:00
compaction.py feat(router): native compact-to-fit across conversation APIs (#42074) 2026-09-21 22:52:29 -07:00
custom_llm.py refactor(types): replace Any with real types across 178 backend files 2026-08-27 10:14:58 +00:00
maritalk.py
oss_decision.py feat: add Bespoke Nimble gateway and OSS classifier support (#44246) 2026-10-02 19:37:08 -07:00
README.md

README.md

File Structure

August 27th, 2024

To make it easy to see how calls are transformed for each model/provider:

we are working on moving all supported litellm providers to a folder structure, where folder name is the supported litellm provider name.

Each folder will contain a *_transformation.py file, which has all the request/response transformation logic, making it easy to see how calls are modified.

E.g. cohere/, bedrock/.