scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-11 22:51:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 154
46618 commits 2327 branches 1406 tags 2.4 GiB
Commit graph

1980 commits

Author SHA1 Message Date
Ishaan Jaff
efa782d6d2
[Feat] Add Nvidia NIM Rerank Support (#15152) 
* feat: add NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix routing to nvidia nim

* docs nvidia nim rerank

* TestNvidiaNim

* nvidia nim rerank fixes

* fix rerank

* transform_rerank_response

* Usage with LiteLLM Proxy

* fixes linting

* NvidiaNimRerankConfig.DEFAULT_NIM_RERANK_API_BASE

* fix Custom API Base URL

* fix rerank base

* fix main.py

* fix transform

* fix linting

* map_cohere_rerank_params

* ruff fix

* linting fixes

* ruff fix
 2025-10-02 18:58:52 -07:00
Krish Dholakia
6b3db2fd49
Merge pull request #15140 from niharm/fix-sonnet-4-5-200k-pricing 
Price Fix: Add 200K prices for Sonnet 4.5
 2025-10-02 17:28:47 -07:00
Mubashir Osmani
f2107a189d
add azure_ai grok-4 model family (#15137) 
* added oauth mcp to docs

* added azure ai/grok-4 model family

* Revert "added oauth mcp to docs"

This reverts commit 950b7cef44.
 2025-10-02 14:21:12 -07:00
nihar
cd7acd2eb2 Add 200K prices for Sonnet 4.5 2025-10-02 12:39:44 -07:00
Ishaan Jaff
d9664a3ee4
fix gpt-5-chat-latest on model cost map (#15116) 2025-10-01 14:35:57 -07:00
Uzair Ali
fcfe856e10
Add support for GPT 5 codex models (#14841) 
* Add support for GPT 5 codex models

* lint

* fixes
 2025-09-30 18:44:35 -07:00
Ishaan Jaff
862736e74b
feat: add groq/moonshotai/kimi-k2-instruct-0905 (#15079) 2025-09-30 12:51:21 -07:00
Eddie Richter
f9e98f75a6 Adding max_input_tokens and max_output_tokens 2025-09-30 12:12:24 -06:00
Eddie Richter
ae92404d05 Initial addition of Lemonade provider. 2025-09-30 12:12:24 -06:00
Krish Dholakia
551a171243
Merge pull request #15043 from cedarm/fix-remove-vertex-latest 
fix: remove invalid vertex -latest models
 2025-09-29 21:54:53 -07:00
Ishaan Jaffer
3e474b9e81 fix claude-sonnet-4-5 model cost map 2025-09-29 18:26:29 -07:00
Ishaan Jaff
05955042d5
Add model pricing and context window for claude-sonnet-4-5 (#15049) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
 2025-09-29 15:09:40 -07:00
Ishaan Jaff
038863a1fe
[Feat] Add new claude-sonnet-4-5 model family (#15041) 
* add new claude-sonnet-4-5

* docs fix

* fix tool_use_system_prompt_tokens

* add anthropic.claude-sonnet-4-5-20250929 to bedrock converse models
 2025-09-29 13:09:00 -07:00
Cedar Myers
5357b1e102 fix: remove invalid vertex -latest models 2025-09-29 16:04:33 -04:00
Wenxi Onyx
ad29d2f944 add models served from ollama cloud 2025-09-28 14:06:08 -07:00
Wenxi Onyx
4adb854e51 add models served from ollama cloud 2025-09-28 13:20:35 -07:00
Krish Dholakia
f8a64345b7
Merge pull request #14942 from danielmklein/main 
Fix inconsistent token configs for gpt-5 models
 2025-09-27 07:49:19 -07:00
Nicolas Herment
50f625433d
Revert incorrect changes to sonnet-4 max output tokens (#14933) 2025-09-26 11:19:52 -07:00
Ishaan Jaff
360befa216
[Feat] Add support for Gemini 2.5 Flash and Flash-lite preview models (09-2025 release) (#14948) 
* add gemini-2.5-flash-preview-09-2025

* docs add gemini-2.5-flash-preview-09-2025 model family
 2025-09-26 09:51:38 -07:00
Daniel Klein
de795a4531 Fix inconsistent token configs for gpt-5 models 2025-09-26 09:43:26 -04:00
Toy-97
6c95bd926f
update: DeepInfra model data refresh [2025-09-26] 
Added models:
deepinfra/deepseek-ai/DeepSeek-V3.1-Terminus

Removed models:
deepinfra/zai-org/GLM-4.5-Air

Modified models:
deepinfra/NousResearch/Hermes-3-Llama-3.1-70B:
   - input_cost_per_token: 1.2e-07 → 3e-07

deepinfra/Qwen/Qwen3-32B:
   - output_cost_per_token: 3e-07 → 2.8e-07

deepinfra/Qwen/Qwen3-Next-80B-A3B-Instruct:
   - max_tokens: 4096 → 262144
   - max_output_tokens: 4096 → 262144
   - max_input_tokens: 4096 → 262144

deepinfra/Qwen/Qwen3-Next-80B-A3B-Thinking:
   - max_tokens: 4096 → 262144
   - max_output_tokens: 4096 → 262144
   - max_input_tokens: 4096 → 262144

deepinfra/Qwen/Qwen3-235B-A22B-Instruct-2507:
   - input_cost_per_token: 1.3e-07 → 9e-08

deepinfra/meta-llama/Meta-Llama-3.1-70B-Instruct:
   - input_cost_per_token: 2.3e-07 → 4e-07

deepinfra/google/gemini-2.5-flash:
   - output_cost_per_token: 1.75e-06 → 2.5e-06
   - input_cost_per_token: 2.1e-07 → 3e-07

deepinfra/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo:
   - output_cost_per_token: 2e-08 → 3e-08
   - input_cost_per_token: 1.5e-08 → 2e-08

deepinfra/meta-llama/Llama-3.2-3B-Instruct:
   - output_cost_per_token: 2.4e-08 → 2e-08
   - input_cost_per_token: 1.2e-08 → 2e-08

deepinfra/Sao10K/L3-8B-Lunaris-v1-Turbo:
   - input_cost_per_token: 2e-08 → 4e-08

deepinfra/openai/gpt-oss-120b:
   - input_cost_per_token: 9e-08 → 5e-08

deepinfra/google/gemini-2.5-pro:
   - output_cost_per_token: 7e-06 → 1e-05
   - input_cost_per_token: 8.75e-07 → 1.25e-06

deepinfra/NousResearch/Hermes-3-Llama-3.1-405B:
   - output_cost_per_token: 8e-07 → 1e-06
   - input_cost_per_token: 7e-07 → 1e-06

deepinfra/Qwen/Qwen3-235B-A22B:
   - output_cost_per_token: 6e-07 → 5.4e-07
   - input_cost_per_token: 1.3e-07 → 1.8e-07

deepinfra/nvidia/Llama-3.1-Nemotron-70B-Instruct:
   - output_cost_per_token: 3e-07 → 6e-07
   - input_cost_per_token: 1.2e-07 → 6e-07

deepinfra/meta-llama/Llama-3.3-70B-Instruct-Turbo:
   - output_cost_per_token: 1.2e-07 → 3.9e-07
   - input_cost_per_token: 3.8e-08 → 1.3e-07

deepinfra/deepseek-ai/DeepSeek-V3-0324:
   - input_cost_per_token: 2.8e-07 → 2.5e-07
   - cache_read_input_token_cost: 2.24e-07 → None

deepinfra/mistralai/Mistral-Small-3.2-24B-Instruct-2506:
   - output_cost_per_token: 1e-07 → 2e-07
   - input_cost_per_token: 5e-08 → 7.5e-08

deepinfra/Qwen/Qwen3-235B-A22B-Thinking-2507:
   - output_cost_per_token: 6e-07 → 2.9e-06
   - input_cost_per_token: 1.3e-07 → 3e-07

deepinfra/zai-org/GLM-4.5:
   - output_cost_per_token: 2e-06 → 1.6e-06
   - input_cost_per_token: 5.5e-07 → 4e-07

deepinfra/mistralai/Mixtral-8x7B-Instruct-v0.1:
   - output_cost_per_token: 2.4e-07 → 4e-07
   - input_cost_per_token: 8e-08 → 4e-07

deepinfra/openai/gpt-oss-20b:
   - output_cost_per_token: 1.6e-07 → 1.5e-07

deepinfra/google/gemma-3-27b-it:
   - output_cost_per_token: 1.7e-07 → 1.6e-07
 2025-09-26 20:11:26 +08:00
Krish Dholakia
2f3155c2ee
Merge pull request #14858 from oytunkutrup1/litellm_fix_gpt3.5_price_fix 
GPT-3.5-Turbo price updated.
 2025-09-25 23:47:43 -07:00
Krish Dholakia
6a1be4722e
Merge pull request #14879 from huangyafei/update_price 
Add gpt-5 and gpt-5-codex to OpenRouter cost map
 2025-09-25 23:46:35 -07:00
Ishaan Jaffer
1585750c1b test fix 2025-09-24 21:45:42 -07:00
huangyafei
107c45da9d Add gpt-5-codex to OpenRouter cost map 2025-09-25 11:05:56 +08:00
huangyafei
1b95940d18 Add gpt-5 to OpenRouter cost map 2025-09-25 11:02:54 +08:00
Luis Felipe Salazar Ucros
c6cb36186c
Add sambanova deepseek v3.1 and gpt-oss-120b (#14866) 
* add sambanova deepseek v3.1 and gpt-oss-120b

* add sambanova deepseek v3.1 and gpt-oss-120b
 2025-09-24 10:54:22 -07:00
Mubashir Osmani
0cd91a82d2
Added vertex_ai/qwen models and azure/gpt-5-codex (#14844) 
* added qwen models and gpt-5-codex

* fix flaky test

* fix failing test
 2025-09-24 10:40:00 -07:00
onlylonly
22eef373eb
feat: New model - Add support for Qwen models family & Deepseek 3.1 to Amazon Bedrock (#14845) 
* New model - Add Bedrock deepseek v3.1 model - "deepseek.v3-v1:0"

* New model - Add Bedrock Qwen models - "qwen.qwen3-coder-480b-a35b-v1:0", "qwen.qwen3-235b-a22b-2507-v1:0", "qwen.qwen3-coder-30b-a3b-v1:0", "qwen.qwen3-32b-v1:0"

* fix: add "deepseek.v3-v1:0" in  litellm/model_prices_and_context_window_backup.json
 2025-09-24 09:58:23 -07:00
oytun.kutrup
410e4956c4
GPT-3.5-Turbo price updated. 2025-09-24 15:32:23 +03:00
Krish Dholakia
d5ced7e068
Merge pull request #14639 from danielmklein/main 
fix: update sonnet 4 configs to reflect million-token context window pricing
 2025-09-23 18:06:15 -07:00
Ishaan Jaff
fff1d1a9f9
Add Vertex AI Qwen3 models to pricing and context window (#14828) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
 2025-09-23 16:16:57 -07:00
Ishaan Jaff
8443eff4a2
feat: add xai/grok-4-fast models (#14833) 2025-09-23 16:01:44 -07:00
Krish Dholakia
88f9cad886
Merge pull request #14796 from BerriAI/litellm_anthopic_token_count_issue 
Add service_tier based pricing support for openai[ BOTH Service & Priority Support]
 2025-09-22 22:38:59 -07:00
Sameerlite
d2208023d2 Add service_tier based pricing support for openai 2025-09-23 10:48:27 +05:30
Gagan Raj Chinka
a5ed3446cd
opnerouter/x-ai/grok-4-fast:free" to model_prices_and_context_window.json (#14779) 
* Update model_prices_and_context_window_backup.json

* Update model_prices_and_context_window.json
 2025-09-22 21:29:15 -07:00
Anubhav Singh
fde4dcb0b8
Merge branch 'main' into wandb-inference 2025-09-23 00:14:11 +05:30
Tim Elfrink
282617f2bc Fix gemini-2.5-flash-image-preview model routing 
- Update mode from 'chat' to 'image_generation' for both model variants
- Ensures correct routing to image generation endpoints
- Resolves 400 'request not supported' error for image generation
 2025-09-19 07:58:58 +02:00
Ishaan Jaffer
114d077cc9 fix: model cost map check 2025-09-18 17:52:56 -07:00
Ishaan Jaffer
c1a967992f fix: model cost map check 2025-09-18 17:37:09 -07:00
Ishaan Jaffer
0626affa11 fix: model cost map check 2025-09-18 17:25:28 -07:00
Ishaan Jaff
4c983f985a
[Feat] Add Bedrock Twelve Labs embedding provider support (#14697) 
* fix: add 12 labs to bedrock embedding

* fix: get_bedrock_embedding_provider

* test: test_text_embedding

* fix: 12 labs embedding transform

* fix: refactor 12 labs transform logic

* fix: test_e2e_bedrock_embedding

* fix: test_e2e_bedrock_embedding

* feat: add bedrock twelvelabs pricing

* DOCS: docs bedrock embedding

* DOCS: 12 labs bedrock overview

* fix: bedrock embeddings 12 labs
 2025-09-18 17:16:45 -07:00
Sameer Kankute
36bedc69ff
Add TwelveLabs marengo model (#14674) 2025-09-18 11:21:35 -07:00
Daniel Klein
b8b30775a4 fix: update sonnet 4 configs to reflect million-context-window pricing 2025-09-17 11:33:23 -04:00
Krrish Dholakia
97cc5f55d6 build(model_prices_and_context_window.json): add claude-3-5 haiku, claude-3-5 sonnet, claude-opus 3, claude-haiku 3 "cache_creation_input_token_cost_above_1hr" pricing 2025-09-16 18:58:49 -07:00
Krrish Dholakia
3188ae9281 build(model_prices_and_context_window.json): add claude-opus-4, claude-3-7-sonnet, claude-sonnet-4 "cache_creation_input_token_cost_above_1hr" pricing 2025-09-16 18:56:24 -07:00
Krrish Dholakia
a2bfd3e476 build(model_prices_and_context_window.json): add claude-opus-4-1 "cache_creation_input_token_cost_above_1hr" pricing 2025-09-16 18:53:56 -07:00
Krrish Dholakia
0fa11c3c25 build(model_prices_and_context_window.json): add "cache_creation_input_token_cost_above_1hr" to all claude-3-5-sonnet models 2025-09-16 18:52:37 -07:00
Krrish Dholakia
1c855385c9 build(model_cost): add cache_creation_input_token_cost_above_1hr pricing 2025-09-16 18:43:57 -07:00
xprilion
470e1b7d2d tinyfix json error 2025-09-16 16:47:51 +05:30