scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-11 22:51:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 123
46618 commits 2327 branches 1406 tags 2.4 GiB
Commit graph

1980 commits

Author SHA1 Message Date
Anubhav Singh
a9667e5930
Merge branch 'BerriAI:main' into wandb-inference 2025-09-16 16:46:58 +05:30
Tim Elfrink
30c3e7b3d3
Fix: Bedrock cross-region inference profile cost calculation (#14566) 
* Add tests for Bedrock cross-region inference profile mapping

- Test model mapping lookup works correctly
- Test proxy cost calculation scenario reproduces original issue
- Verify cost calculation returns expected values
- Ensure compatibility with existing test patterns

* Fix Bedrock cross-region inference profile cost calculation

- Add mapping for bedrock/us.anthropic.claude-3-5-haiku-20241022-v1:0
- Sync backup file for local testing consistency
- Resolve proxy spend tracking failures for cross-region profiles
- Maintain identical configuration with standalone profile

Fixes #14458
 2025-09-15 07:10:20 -07:00
Anubhav Singh
67276a8151
Merge branch 'main' into wandb-inference 2025-09-15 18:34:16 +05:30
Elias TOURNEUX
ef9d1ddc40
feat: Add OVHCloud AI Endpoints as a provider 2025-09-12 13:37:03 +02:00
Ishaan Jaff
69ef062f55 fix tiered_pricing test 2025-09-11 19:56:44 -07:00
Ishaan Jaff
dda115cc6d
[Feat] Cost Tracking - Add support for Tiered Cost Tracking for Qwen API (Dashscope) (#14471) 
* add dashscope logo

* docs fix

* docs fix

* fix supports_batch_calling

* fix naming

* fix input_cost_per_audio_token

* use output_cost_per_reasoning_token

* add tiered_pricing in get_model_info

* test fixes

* fix cost calc

* ruff fix
 2025-09-11 18:14:39 -07:00
Ishaan Jaff
258b674dbb fix deepinfra test 2025-09-10 19:39:23 -07:00
xprilion
667481e75b (feat): Add W&B Inference to LiteLLM 2025-09-11 00:07:30 +05:30
Krish Dholakia
e57a05b2dc
Merge pull request #14324 from Toy-97/patch-1 
update: DeepInfra model data refresh [2025-09-08]
 2025-09-09 22:31:44 -07:00
Krrish Dholakia
076b46c806 build: remove end of life bedrock model 2025-09-09 19:49:22 -07:00
Pedro Azevedo
44dc6d3aef
Fix: Add supports_function_calling for GPT OSS in Bedrock provider (#14375) 
* Refactor JSON formatting and remove unnecessary whitespace in model prices and context window

* Fix formatting inconsistencies and remove unnecessary whitespace in model prices JSON
 2025-09-09 13:49:32 -07:00
Krrish Dholakia
3dba103e2c fix: fix typo 2025-09-08 18:13:34 -07:00
Toy-97
890ee1abfa
update: DeepInfra model data refresh [2025-09-08] 
Removed models:
deepinfra/Qwen/Qwen2.5-Coder-32B-Instruct
deepinfra/deepseek-ai/DeepSeek-V3-0324-Turbo
deepinfra/meta-llama/Llama-3.2-90B-Vision-Instruct
deepinfra/meta-llama/Llama-4-Maverick-17B-128E-Instruct-Turbo
deepinfra/meta-llama/Meta-Llama-3-70B-Instruct
deepinfra/mistralai/Devstral-Small-2507
deepinfra/mistralai/Mistral-7B-Instruct-v0.3
deepinfra/mistralai/Mistral-Small-3.1-24B-Instruct-2503

Modified models:
deepinfra/deepseek-ai/DeepSeek-R1-0528:
   - cache_read_input_token_cost: None → 4e-07

deepinfra/google/gemma-3-4b-it:
   - input_cost_per_token: 2e-08 → 4e-08
   - output_cost_per_token: 4e-08 → 8e-08

deepinfra/Qwen/QwQ-32B:
   - input_cost_per_token: 7.5e-08 → 1.5e-07
   - output_cost_per_token: 1.5e-07 → 4e-07

deepinfra/deepseek-ai/DeepSeek-V3-0324:
   - cache_read_input_token_cost: None → 2.24e-07

deepinfra/deepseek-ai/DeepSeek-V3.1:
   - input_cost_per_token: 3e-07 → 2.7e-07
   - cache_read_input_token_cost: None → 2.16e-07

deepinfra/Qwen/Qwen3-Coder-480B-A35B-Instruct-Turbo:
   - cache_read_input_token_cost: 1.5e-07 → 2.4e-07

deepinfra/NousResearch/Hermes-3-Llama-3.1-70B:
   - supports_tool_choice: True → False

deepinfra/deepseek-ai/DeepSeek-R1-Turbo:
   - max_output_tokens: 163840 → 40960
   - max_input_tokens: 163840 → 40960
   - max_tokens: 163840 → 40960
 2025-09-08 19:17:58 +08:00
Krish Dholakia
ba10173ec7
Merge branch 'main' into heroku-llms 2025-09-06 22:10:20 -07:00
Thomas Rehn
d88771ca49 fix: correct output pricing for gemini-2.5-flash-image-preview https://ai.google.dev/gemini-api/docs/pricing#gemini-2.5-flash-image-preview 2025-09-05 16:22:07 +02:00
Krish Dholakia
f67339a86c
Merge pull request #14028 from onlylhf/volcengine-embedding-support 
Add Volcengine embedding module with handler and transformation logic
 2025-09-04 21:01:25 -07:00
Ishaan Jaff
23ae7170d1
[Feat] Allow using Veo Video Generation through LiteLLM Pass through routes (#14228) 
* fix: add follow_redirects=True,

* test_pass_through_with_httpbin_redirect

* cook book veo video

* docs Veo Video Generation with Google AI Studio

* add veo-3.0-generate-preview cost tracking details

* track vertex_video_models
 2025-09-03 18:25:43 -07:00
Harshavardhan K
bc84b411f4
Fix/remove deprecated cerebras gpt oss 20b (#14213) 
* fix: remove deprecated cerebras/openai/gpt-oss-20b model from pricing files

- Remove cerebras/openai/gpt-oss-20b entry from model_prices_and_context_window.json
- Remove cerebras/openai/gpt-oss-20b entry from model_prices_and_context_window_backup.json
- Fixes GitHub issue #13972 where the deprecated model was still being treated as valid
- Only the 120b version is now supported by Cerebras

* fix: remove deprecated cerebras gpt-oss-20b
 2025-09-03 09:46:08 -07:00
Ishaan Jaff
c821f1ddf1
[Feature]: Support GPT-OSS models on vertex ai (#14184) 
* add VertexAIGPTOSSTransformation

* fix: optional_params

* fix: is_vertex_partner_model

* test_partner_models_httpx

* docs GPT oss docs

* test_vertex_ai_gpt_oss_reasoning_effort

* add vertex ai models
 2025-09-02 14:15:26 -07:00
Ishaan Jaff
94cac6fff2 fix: groq health check 2025-09-01 20:08:36 -07:00
Mubashir Osmani
58ac3ecf84
openrouter: added gpt 4.1 model family (#14101) 2025-09-01 15:36:17 -07:00
Krish Dholakia
9e947fce19
Merge pull request #14087 from zainhas/main 
update together models
 2025-09-01 15:07:58 -07:00
Ishaan Jaff
fa4c833e58 fix: supports reasoning 2025-08-30 10:29:08 -07:00
Krish Dholakia
8b11452d9e
Merge branch 'main' into add-vercel-ai-gateway-provider 2025-08-29 22:27:08 -07:00
Zain Hasan
8a5df2eb74
update together models 
- add tool choice for models
- add deepseek v3.1
 2025-08-30 00:44:42 -04:00
Ishaan Jaff
be86861eb8
feat: add gpt-realtime models - gpt-realtime (#14082) 
* feat: add gpt-realtime models

* add input_cost_per_image
 2025-08-29 17:49:47 -07:00
Ishaan Jaff
607f425cc6
[Bug]: Fix Can't set reasoning_effort for DeepSeek-V3.1 on DeepInfra by default (#14053) 
* fix DeepInfraConfig

* track supports reasoning

* fix: get_supported_openai_params

* test_deepseek_supported_openai_params
 2025-08-29 13:15:43 -07:00
Ishaan Jaff
81db36c06c
add xai/grok-code-fast (#14054) 2025-08-28 18:10:04 -07:00
李海峰
2d0a57a719 Add Volcengine embedding module with handler and transformation logic 
- Implemented VolcEngineEmbeddingHandler for synchronous and asynchronous embedding requests.
- Created VolcEngineEmbeddingConfig for transforming requests and responses to/from Volcengine format.
- Added integration tests for embedding functionality, covering various scenarios including error handling and parameter validation.
- Established test structure for Volcengine embedding, ensuring compliance with LiteLLM testing patterns.
- Included comprehensive tests for parameter mapping, request transformation, and response handling.
 2025-08-28 15:05:11 +08:00
Krish Dholakia
c43706061f
Merge branch 'main' into heroku-llms 2025-08-27 22:21:36 -07:00
Ifta Khairul Alam Adil
d9f8eb27c9 fix: enable tool choice support for model prices and context window 
- Updated the "supports_tool_choice" field to true in the model_prices_and_context_window.json file, allowing for tool choice functionality in the specified model.

Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com>
 2025-08-27 19:46:53 +02:00
Ifta Khairul Alam Adil
8808f78ba6 Merge branch 'main' into fix/gemini-imagen-model-name-validation 2025-08-27 15:45:59 +02:00
Krish Dholakia
4613f7821d
Merge pull request #13536 from edwardsamuel/feat/openrouter-gpt-5-pricing 
Add openrouter gpt-5 family models pricing
 2025-08-26 23:16:53 -07:00
Krish Dholakia
201d96b2d2
Merge pull request #13917 from Toy-97/patch-1 
Deepinfra Metadata Update 24082025
 2025-08-26 23:13:46 -07:00
Ifta Khairul Alam Adil
0f5b31fd78 fix: resolve invalid model name error for Gemini Imagen models (#13851) 
- Fix URL construction in Gemini image generation to strip 'gemini/' prefix
- Google AI API expects base model name without the prefix
- Update model references and pricing information for consistency
- Remove outdated image generation pricing entries

Fixes issue where models like 'gemini/imagen-4.0-fast-generate-preview-06-06'
were being rejected by the Google AI API due to incorrect URL formatting.
 2025-08-27 04:17:35 +02:00
Ishaan Jaff
5647757ab3
[Feat] New model gemini-2.5-flash-image-preview (#13979) 
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview
 2025-08-26 15:04:02 -07:00
Krish Dholakia
55c3fd62ac
Add Claude Opus 4 and 4.1 via OpenRouter and Fix Claude Sonnet 4 and 3.7 via OpenRouter in Prices and Context Window Config 
Update model prices and context window
 2025-08-25 23:28:39 -07:00
Krish Dholakia
3803af5ff5
Merge pull request #13566 from kankute-sameer/fix/azure-gpt4-web-search-support 
fix: remove incorrect web search support for azure/gpt-4.1 family
 2025-08-25 23:28:15 -07:00
Teddy Amkie
97e9502f4a Add DeepSeek-v3.1 pricing for Fireworks AI provider 
- Add fireworks_ai/accounts/fireworks/models/deepseek-v3p1 model configuration
- Set context window: 128K input, 8K output tokens
- Pricing: /bin/zsh.56/1M input tokens, .68/1M output tokens
- Supports response schema and tool choice
- Based on DeepSeek API unified pricing effective Sept 2025
 2025-08-25 18:41:49 -07:00
Toy-97
f2bb1ce31e
Deepinfra Metadata Update 24082025 
Deepseek v3.1 Price Dropped
 2025-08-24 08:30:25 +08:00
Ishaan Jaff
9750374081
[Feat] New LLM API - AI/ML API for Image Gen (#13893) 
* add LlmProviders.AIML

* add AIMLChatConfig

* add aiml

* fix AimlImageGenerationConfig

* add AimlImageGenerationConfig

* add cost_calculator

* fixes for AI ML API

* add known AIML Flux image models

* add AIML Cost Calc

* add AI/ML API

* add aiml_models
 2025-08-23 13:12:44 -07:00
Krish Dholakia
49cb9bdbca
Merge pull request #13843 from BerriAI/litellm_dev_08_29_2025_p3 
SSO - Free SSO usage for up to 5 users + remove deprecated dbrx models (dbrx-instruct, llama 3.1)
 2025-08-23 12:06:01 -07:00
Krrish Dholakia
eb7e5e3a71 Revert "build(model_prices_and_context_window.json): remove retired dbrx model" 
This reverts commit 10d49d0174.
 2025-08-23 10:47:44 -07:00
Krrish Dholakia
10d49d0174 build(model_prices_and_context_window.json): remove retired dbrx model 2025-08-23 10:36:52 -07:00
Krrish Dholakia
c270c5a37a build(model_prices_and_context_window.json): remove deprecated model from model cost map 2025-08-23 10:36:52 -07:00
Sameer Kankute
f239a0f78a Add openrouter deepseek/deepseek-chat-v3.1 support 2025-08-22 22:59:09 +05:30
Krish Dholakia
9c4a86e98e
Merge pull request #13868 from kankute-sameer/litellm_feat_voyage_context_3_embedding_model 
[Feat] Add support for voyage-context-3 embedding model
 2025-08-21 23:18:20 -07:00
Toy-97
8ba292f201
fix extra bracket.json 2025-08-22 13:21:41 +08:00
Toy-97
62ae6d8bbc
Update model metadata for deepinfra.json 2025-08-22 13:11:55 +08:00
Ishaan Jaff
5978f25299
add new gemini/imagen-4.0 models (#13876) 2025-08-21 15:32:31 -07:00