scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-28 01:32:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 43
litellm/litellm/llms
History
Exact
togear fc0055497c
feat: add configurable provider affinity header mapping (#41033) 
* feat: add configurable provider affinity header mapping

* fix: sync provider affinity API types

* fix: harden provider affinity header mapping

* fix: avoid provider affinity import cycle

* fix: preserve input callback header mutations

* fix: address provider affinity code scanning findings

* fix: satisfy provider affinity type discipline gate

* test: cover omitted pre-call argument isolation

* fix: resolve remaining provider affinity codeql alerts

* fix: redact provider affinity headers after calls

* refactor: drop provider affinity header log redaction

* fix: reject control characters in affinity session ids as a bad request

* chore: regenerate the openapi snapshot on python 3.12 and reuse the session marker constant

* fix(responses): read the affinity session from the named metadata argument

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 13:31:33 -07:00
..
a2a feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
ai21/chat feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
aiml fix(cost): honor deployment pricing for image generation (#39311) 2026-09-22 11:40:39 -07:00
aiohttp_openai/chat feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
amazon_nova feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
anthropic feat(router): native compact-to-fit across conversation APIs (#42074) 2026-09-21 22:52:29 -07:00
apiserpent feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
aws_polly refactor(typing): replace Any with proven types in 89 more backend files 2026-09-02 23:08:42 +00:00
azure fix(realtime): surface an upstream handshake refusal as an error event and policy close (#42388) 2026-09-22 11:31:15 -07:00
azure_ai fix(cost): honor deployment pricing for image generation (#39311) 2026-09-22 11:40:39 -07:00
base_llm fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
baseten feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
bedrock fix(bedrock/claude_platform): strip body params the AWS endpoint rejects (#31203) 2026-09-22 09:57:05 -07:00
bedrock_mantle fix(bedrock): send every Mantle beta in the anthropic-beta header on the bedrock/mantle route (#42376) 2026-09-21 18:52:29 -07:00
black_forest_labs feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
brave/search feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
bytez feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
cerebras fix(cerebras): add max_retries and extra_headers to get_supported_openai_params (#36601) 2026-08-25 14:10:56 -07:00
chatgpt refactor: replace Any with precise types across 54 modules 2026-09-14 11:13:39 +00:00
clarifai/chat feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
cloudflare/chat feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
codestral/completion refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00
cohere feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
cometapi fix(cost): honor deployment pricing for image generation (#39311) 2026-09-22 11:40:39 -07:00
compactifai feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
custom_httpx feat: add configurable provider affinity header mapping (#41033) 2026-09-22 13:31:33 -07:00
dashscope feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
databricks feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
dataforseo/search refactor(typing): replace Any with proven types in 42 more backend files 2026-09-02 15:35:01 +00:00
datarobot/chat chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
deepgram fix(deepgram): forward only the first model and language values to /listen 2026-09-18 21:29:29 +00:00
deepinfra refactor(types): replace Any with precise types across 73 modules 2026-09-01 11:05:02 +00:00
deepseek fix(anthropic): add the per-turn-control beta when a message carries output_config 2026-09-14 22:01:47 -07:00
deprecated_providers feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
docker_model_runner/chat chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
duckduckgo/search feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
e2b Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 06:03:33 -07:00
edenai fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
elevenlabs refactor(typing): replace Any with proven types in 42 more backend files 2026-09-02 15:35:01 +00:00
empower/chat
exa_ai/search feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
fal_ai fix(fal_ai): honour global api_base for image generation and reject non-string reasoning_effort with 400 (#42512) 2026-09-22 13:13:54 -07:00
fastcrw feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
featherless_ai/chat feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
firecrawl feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
fireworks_ai feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
friendliai/chat
galadriel/chat
gdc refactor(types): replace Any with real types across 29 more backend files 2026-09-08 10:14:07 +00:00
gemini fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
gigachat feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
github/chat
github_copilot Merge remote-tracking branch 'origin/main' into litellm_decrease_anys_opus5_r5 2026-09-21 12:57:38 -07:00
google_pse/search feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
gradient_ai/chat feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
groq feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
heroku/chat refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
hosted_vllm fix: address review feedback on Any reduction 2026-09-14 07:31:41 +00:00
huggingface feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
hyperbolic chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
inception chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
infinity Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 06:03:33 -07:00
jina_ai refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00
lambda_ai chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
langflow feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
langgraph feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
lemonade feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
linkup feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
litellm_proxy fix(spend_tracking): honour the global litellm_proxy override when inferring a model group provider 2026-09-17 00:03:01 +00:00
llamafile/chat chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
lm_studio chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
manus feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
meta/realtime fix(vertex_ai): carry turns across stream rotation and route by model info 2026-09-18 15:35:59 -07:00
meta_llama/chat feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
milvus/vector_stores refactor(s3_vectors): embed search queries through the shared vector store executor 2026-09-02 19:26:31 -07:00
minimax refactor(typing): replace Any with proven types in 42 more backend files 2026-09-02 15:35:01 +00:00
mistral feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
modelscope chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
mongodb merge: resolve MongoDB sidecar staging conflicts 2026-09-08 13:59:56 -07:00
moonshot/chat fix(moonshot, together_ai): send the reasoning effort Kimi K3 accepts (#38611) 2026-08-27 20:46:46 -07:00
morph feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
nebius feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
nimble feat(search): add Nimble as a search provider (#36347) 2026-08-14 17:09:58 -07:00
nlp_cloud feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
novita/chat refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
nscale/chat feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
nvidia_nim fix(proxy): reject mixed NIM model groups and strip the deployment model before the group in /nvidia_nim URLs 2026-09-15 23:34:22 +00:00
nvidia_riva refactor(types): replace Any with real types across 11 more backend files 2026-09-08 14:09:22 +00:00
oci feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
ollama feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
oobabooga feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
openai fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
openai_like feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
openrouter feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
opensandbox Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 06:03:33 -07:00
ovhcloud feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
parallel_ai/search style(websearch): apply ruff format to changed files 2026-09-09 08:55:25 -04:00
pass_through refactor(typing): replace Any with proven types in 42 more backend files 2026-09-02 15:35:01 +00:00
perplexity feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
petals feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
pg_vector/vector_stores refactor(s3_vectors): embed search queries through the shared vector store executor 2026-09-02 19:26:31 -07:00
predibase feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
ragflow Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5_r4 2026-09-03 06:44:40 +00:00
recraft fix(cost): honor deployment pricing for image generation (#39311) 2026-09-22 11:40:39 -07:00
reducto refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00
replicate feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
runwayml fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
s3_vectors fix(s3_vectors): embed registered-store ingests with the store's embedding model 2026-09-19 04:31:15 -07:00
sagemaker feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
sambanova feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
sap feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
scaleway/audio_transcription feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
searchapi feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
searxng feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
serper/search feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
snowflake refactor(types): replace Any with real types across 54 more backend files 2026-09-08 12:41:07 +00:00
soniox refactor(typing): replace Any with proven types in 89 more backend files 2026-09-02 23:08:42 +00:00
stability feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
tavily/search feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
tencent fix(tencent): satisfy basedpyright budget in thinking mapping 2026-08-24 18:41:50 -03:00
tinyfish/search fix(search): propagate GET provider HTTP errors (#40779) 2026-09-11 14:07:32 -07:00
together_ai fix(moonshot, together_ai): send the reasoning effort Kimi K3 accepts (#38611) 2026-08-27 20:46:46 -07:00
topaz feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
triton feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
v0 chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
valkey fix(vector-store): route embeddings through router 2026-09-01 15:33:18 -07:00
vercel_ai_gateway refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00
vertex_ai fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
vllm chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
volcengine refactor(types): replace Any with real types across 54 more backend files 2026-09-08 12:41:07 +00:00
voyage Merge remote-tracking branch 'origin/main' into litellm_decrease_anys_opus5_r5 2026-09-21 12:57:38 -07:00
wandb fix(wandb): gate reasoning effort on model capabilities 2026-09-10 20:47:13 +02:00
watsonx feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
xai fix(xai): accept max_completion_tokens as a supported param 2026-09-21 22:30:13 +00:00
xinference/image_generation feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
you_com feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
zai feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
__init__.py fix(gemini): bill Google Maps grounding as its own SKU 2026-08-26 15:31:27 -07:00
base.py refactor(types): replace Any with real types across 178 backend files 2026-08-27 10:14:58 +00:00
compaction.py feat(router): native compact-to-fit across conversation APIs (#42074) 2026-09-21 22:52:29 -07:00
custom_llm.py refactor(types): replace Any with real types across 178 backend files 2026-08-27 10:14:58 +00:00
maritalk.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
README.md

README.md

File Structure

August 27th, 2024

To make it easy to see how calls are transformed for each model/provider:

we are working on moving all supported litellm providers to a folder structure, where folder name is the supported litellm provider name.

Each folder will contain a *_transformation.py file, which has all the request/response transformation logic, making it easy to see how calls are modified.

E.g. cohere/, bedrock/.