Commit graph

1980 commits

Author SHA1 Message Date
Takashi Iwamoto
1969e351fe
Add Amazon Nova Premier (#10520) 2025-05-03 10:14:39 -07:00
Krish Dholakia
9cc39af131
Add vertex ai meta llama 4 support + handle tool call result in content for vertex ai (#10492)
* refactor(vertex_ai/llama): handle response transformation within config

Allows us to handle https://github.com/BerriAI/litellm/issues/10441#issuecomment-2844975599

* fix(vertex_ai/llama): handle tool call in content

Fixes https://github.com/BerriAI/litellm/issues/10441

* fix(vertex_ai/llama): return 'tool_calls' as finish reason if tool call returned

vertex ai returns stop

* feat(vertex_ai/): cost tracking for vertex_ai/meta/llama-4

* ci(test-linting.yml): pin openai version

* build: reorder pinning

* ci(pyproject.toml): limit openai version

temporary patch as new version has linting errors

* ci(pyproject.toml): limit openai version

temporary patch around linting errors

* ci(limit-openai-version): temporary patch

* fix: fix linting errors

* fix: fix linting error

* fix(parallel_request_limiter_v2.py): add team based multi-instance rate limiting

* fix: fix linting errors

* build(pyproject.toml): modify pin

* ci: bump pin
2025-05-01 22:47:06 -07:00
Ruperto A. Martinez
298a3574f4
Add supports_pdf_input: true to Claude 3.7 bedrock models (#9917)
* Add supports_pdf_input: true to Claude 3.7 bedrock models

* update unit test

---------

Co-authored-by: RupertoXTI <rmartinez@xtillion.com>
2025-05-01 14:56:54 -07:00
Krish Dholakia
6ad483dde7
Litellm dev 04 30 2025 p1 (#10462)
* fix(exception_mapping_utils.py): correctly pass through 504 status code

openai also raises a 504 status code

* build(model_prices_and_context_window.json): add gpt-4o-mini-tts to model cost map

Fixes https://github.com/BerriAI/litellm/issues/9591

* fix(cost_calculator.py): fix input cost calculation for gpt-4o-mini-tts

Fixes https://github.com/BerriAI/litellm/issues/9591

* test: testing updates
2025-04-30 22:11:12 -07:00
Krish Dholakia
d783190e04
Update fireworks ai pricing (#10425)
* build(model_prices_and_context_window.json): add fireworks ai new 0-4b pricing tier

* build(model_prices_and_context_window.json): add more fireworks ai models

* test: update testing

* test: testing updates

* test: update test

* test: update test
2025-04-29 20:58:05 -07:00
Krish Dholakia
70accb754e
fix(langsmith.py): respect langsmith batch size param (#10411)
* fix(langsmith.py): respect langsmith batch size param

* build(model_prices_and_context_window.json): add missing tgai  models

Fixes https://github.com/BerriAI/litellm/issues/8921

Fixes https://github.com/BerriAI/litellm/issues/8278
2025-04-29 18:55:22 -07:00
Ishaan Jaff
e06e7078da
[New model] Add openai/computer-use-preview cost tracking / pricing (#10422)
* add computer-use-preview

* fix computer-use-preview
2025-04-29 17:49:11 -07:00
Ishaan Jaff
72de453cc0 fix gemini/gemini-2.5-pro-exp-03-25 2025-04-28 20:24:40 -07:00
Krrish Dholakia
68c74bb342 build(model_prices_and_context_window.json): add gemini/gemini-2.5-pro-exp-03-25
Addresses https://github.com/BerriAI/litellm/discussions/10058
2025-04-26 12:48:08 -07:00
Krrish Dholakia
421011baec build(model_prices_and_context_window.json): update azure o1-mini pricing 2025-04-26 08:05:09 -07:00
Marty Sullivan
11bb622123
add azure/gpt-image-1 pricing (#10327) 2025-04-25 20:18:56 -07:00
o-khytrov
21e27b13e6
Update deprecation dates and prices (#10308) 2025-04-25 09:43:35 -07:00
Ryan Chase
0a2c964db7
adding support for cohere command-a-03-2025 (#10295) 2025-04-24 17:07:29 -07:00
Ishaan Jaff
36ee132514
[Feat] Add gpt-image-1 cost tracking (#10241)
* add gpt-image-1

* add gpt-image-1 example to docs
2025-04-23 12:20:55 -07:00
Ishaan Jaff
2bb51866b1 fix azure/computer-use-preview native streaming 2025-04-22 18:21:06 -07:00
Marty Sullivan
0b63c7a2eb
Model pricing updates for Azure & VertexAI (#10178) 2025-04-20 11:33:45 -07:00
Krish Dholakia
bbfcb1ac7e
Litellm release notes 04 19 2025 (#10169)
* docs(index.md): initial draft release notes

* docs: note all pending docs

* build(model_prices_and_context_window.json): add o3, gpt-4.1, o4-mini pricing

* docs(vllm.md): update vllm doc to show file message type support

* docs(mistral.md): add mistral passthrough route doc

* docs(gemini.md): add gemini thinking to docs

* docs(vertex.md): add thinking/reasoning content for gemini models to docs

* docs(index.md): more links

* docs(index.md): add more links, images

* docs(index.md): cleanup highlights
2025-04-19 17:26:30 -07:00
Ishaan Jaff
3c463f6715 test fix - output_cost_per_reasoning_token was added to model cost map 2025-04-19 10:02:25 -07:00
Krish Dholakia
36308a31be
Gemini-2.5-flash - support reasoning cost calc + return reasoning content (#10141)
* build(model_prices_and_context_window.json): add vertex ai gemini-2.5-flash pricing

* build(model_prices_and_context_window.json): add gemini reasoning token pricing

* fix(vertex_and_google_ai_studio_gemini.py): support counting thinking tokens for gemini

allows accurate cost calc

* fix(utils.py): add reasoning token cost calc to generic cost calc

ensures gemini-2.5-flash cost calculation is accurate

* build(model_prices_and_context_window.json): mark gemini-2.5-flash as 'supports_reasoning'

* feat(gemini/): support 'thinking' + 'reasoning_effort' params + new unit tests

allow controlling thinking effort for gemini-2.5-flash models

* test: update unit testing

* feat(vertex_and_google_ai_studio_gemini.py): return reasoning content if given in gemini response

* test: update model name

* fix: fix ruff check

* test(test_spend_management_endpoints.py): update tests to be less sensitive to new keys / updates to usage object

* fix(vertex_and_google_ai_studio_gemini.py): fix translation
2025-04-19 09:20:52 -07:00
David Emmanuel
de3c2d14bf
Add Gemini Flash 2.5 Preview Model Price and Context Window (#10125)
* Update model_prices_and_context_window_backup.json

* Update model_prices_and_context_window.json
2025-04-18 09:44:46 -07:00
Krrish Dholakia
809eb859cf fix(azure/o_series_transformation.py): fix azure o4 model routing
Fixes https://github.com/BerriAI/litellm/pull/10065#issuecomment-2814015058
2025-04-17 22:58:01 -07:00
Krrish Dholakia
614d80cb1b build(model_prices_and_context_window.json): add azure gpt-4.1 pricing
ensures cost tracking for gpt-4.1 works
2025-04-17 20:09:17 -07:00
Peter Dave Hello
5c078af738
Add OpenAI o3 & 4o-mini (#10065)
Reference:
- https://platform.openai.com/docs/models/o3
- https://platform.openai.com/docs/models/o4-mini
2025-04-16 12:40:13 -07:00
Emerson Gomes
a2bc0c0f36
Fix cost for Phi-4-multimodal output token (#9880) 2025-04-14 14:31:34 -07:00
Ishaan Jaff
24447eb0cd
fix gpt 4.1 costs (#9991) 2025-04-14 12:50:14 -07:00
Krish Dholakia
bbb7541c22
build(model_prices_and_context_window.json): add gpt-4.1 pricing (#9990)
* build(model_prices_and_context_window.json): add gpt-4.1 pricing

* build(model_prices_and_context_window.json): add gpt-4.1-mini and gpt-4.1-nano model support
2025-04-14 12:14:46 -07:00
Nilanjan De
93037ea4d3
fix: correct the cost for 'gemini/gemini-2.5-pro-preview-03-25' (#9896)
* fix: Typo in the cost 'gemini/gemini-2.5-pro-preview-03-25', closes #9854

* chore: update in backup file as well
2025-04-12 08:20:04 -07:00
Ishaan Jaff
f9ce754817
[Feat] Add litellm.supports_reasoning() util to track if an llm supports reasoning (#9923)
* add supports_reasoning for xai models

* add "supports_reasoning": true for o1 series models

* add supports_reasoning util

* add litellm.supports_reasoning

* add supports reasoning for claude 3-7 models

* add deepseek as supports reasoning

* test_supports_reasoning

* add supports reasoning to model group info

* add supports_reasoning

* docs supports reasoning

* fix supports_reasoning test

* "supports_reasoning": false,

* fix test

* supports_reasoning
2025-04-11 17:56:04 -07:00
Ishaan Jaff
2d6ad534bc
[Feat - PR1] Add xAI grok-3 models to LiteLLM (#9920)
* add xai/grok-3-mini-beta, xai/grok-3-beta

* add grok-3-fast-latest models

* supports_response_schema

* fix pricing

* docs xai
2025-04-11 15:12:12 -07:00
Krish Dholakia
78879c68a9
Revert avglogprobs change + Add azure/gpt-4o-realtime-audio cost tracking (#9893)
* test: initial commit fixing gemini logprobs

Fixes https://github.com/BerriAI/litellm/issues/9888

* fix(vertex_and_google_ai_studio.py): Revert avglogprobs change

Fixes https://github.com/BerriAI/litellm/issues/8890

* build(model_prices_and_context_window.json): add gpt-4o-realtime-preview cost to model cost map

Fixes https://github.com/BerriAI/litellm/issues/9814

* test: add cost calculation unit testing

* test: fix test

* test: update test
2025-04-10 21:23:55 -07:00
Krish Dholakia
ac4f32fb1e
Cost tracking for gemini-2.5-pro (#9837)
* build(model_prices_and_context_window.json): add google/gemini-2.0-flash-lite-001 versioned pricing

Closes https://github.com/BerriAI/litellm/issues/9829

* build(model_prices_and_context_window.json): add initial support for 'supported_output_modalities' param

* build(model_prices_and_context_window.json): add initial support for 'supported_output_modalities' param

* build(model_prices_and_context_window.json): add supported endpoints to gemini-2.5-pro

* build(model_prices_and_context_window.json): add gemini 200k+ pricing

* feat(utils.py): support cost calculation for gemini-2.5-pro above 200k tokens

Fixes https://github.com/BerriAI/litellm/issues/9807

* build: test dockerfile change

* build: revert apk change

* ci(config.yml): pip install wheel

* ci: test problematic package first

* ci(config.yml): pip install only binary

* ci: try more things

* ci: test different ml_dtypes version

* ci(config.yml): check ml_dtypes==0.4.0

* ci: test

* ci: cleanup config.yml

* ci: specify ml dtypes in requirements.txt

* ci: remove redisvl depedency (temporary)

* fix: fix linting errors

* test: update test

* test: fix test
2025-04-09 18:48:43 -07:00
Emerson Gomes
d5e362459c
Update Azure Phi-4 pricing (#9862)
Updates Phi-4 family model prices with recently published info
2025-04-09 13:17:00 -07:00
Li Yang
11389535d5
chore: fix haiku cache read pricing per token (#9834) 2025-04-08 16:43:09 -07:00
Krrish Dholakia
367f48004d build(model_prices_and_context_window.json): consistent params 2025-04-08 12:45:33 -07:00
Krish Dholakia
ac9f03beae
Allow passing thinking param to litellm proxy via client sdk + Code QA Refactor on get_optional_params (get correct values) (#9386)
* fix(litellm_proxy/chat/transformation.py): support 'thinking' param

Fixes https://github.com/BerriAI/litellm/issues/9380

* feat(azure/gpt_transformation.py): add azure audio model support

Closes https://github.com/BerriAI/litellm/issues/6305

* fix(utils.py): use provider_config in common functions

* fix(utils.py): add missing provider configs to get_chat_provider_config

* test: fix test

* fix: fix path

* feat(utils.py): make bedrock invoke nova config baseconfig compatible

* fix: fix linting errors

* fix(azure_ai/transformation.py): remove buggy optional param filtering for azure ai

Removes incorrect check for support tool choice when calling azure ai - prevented calling models with response_format unless on litell model cost map

* fix(amazon_cohere_transformation.py): fix bedrock invoke cohere transformation to inherit from coherechatconfig

* test: fix azure ai tool choice mapping

* fix: fix model cost map to add 'supports_tool_choice' to cohere models

* fix(get_supported_openai_params.py): check if custom llm provider in llm providers

* fix(get_supported_openai_params.py): fix llm provider in list check

* fix: fix ruff check errors

* fix: support defs when calling bedrock nova

* fix(factory.py): fix test
2025-04-07 21:04:11 -07:00
Michael Clark
cd0a1e6000
Update model_prices (#9768) 2025-04-05 09:20:01 -07:00
Ishaan Jaff
220fa23d2b watsonx/ibm/granite-3-8b-instruct 2025-04-04 21:46:02 -07:00
Ishaan Jaff
e2bb203075 update watsonx/ibm/granite-3-8b-instruct" 2025-04-04 21:45:04 -07:00
Krrish Dholakia
7cd7bdbd0f build: fix model cost map 2025-04-04 20:48:29 -07:00
caramulrooney
3e9066e91d
Update model_prices_and_context_window.json (#9620)
Add watsonx/ibm/granite-3-8b-instruct
2025-04-04 20:44:06 -07:00
Hugo Liu
08f9e1447b
fix(asr-groq): add groq whisper models to model cost map (#9648)
Co-authored-by: liuhu <liuhu@huami.com>
2025-04-04 20:43:46 -07:00
Tobias Hermann
5785600c4e
[Feat] Add VertexAI gemini-2.0-flash (#9723) 2025-04-02 22:33:23 -07:00
superpoussin22
136f1d60b3
Update model_prices_and_context_window.json add gemini-2.5-pro-exp-03-25 (#9650)
* Update model_prices_and_context_window.json

add gemini-2.5-pro-exp-03-25

* Update model_prices_and_context_window.json

correct tokens limit
2025-03-31 21:31:30 -07:00
Krish Dholakia
ef88792ad3
Revert "Revert "Correct Databricks llama3.3-70b endpoint and add databricks c…" (#9676)
This reverts commit 1e26c594cb.
2025-03-31 21:30:50 -07:00
Krish Dholakia
1e26c594cb
Revert "Correct Databricks llama3.3-70b endpoint and add databricks claude 3.…" (#9668)
This reverts commit dfb838eaff.
2025-03-31 17:02:21 -07:00
Anton Abilov
dfb838eaff
Correct Databricks llama3.3-70b endpoint and add databricks claude 3.7 sonnet endpoint (#9661) 2025-03-31 15:22:12 -07:00
Krish Dholakia
5c107c64dd
Add gemini audio input support + handle special tokens in sagemaker response (#9640)
* fix(internal_user_endpoints.py): cleanup unused variables on beta endpoint

no team/org split on daily user endpoint

* build(model_prices_and_context_window.json): gemini-2.0-flash supports audio input

* feat(gemini/transformation.py): support passing audio input to gemini

* test: fix test

* fix(gemini/transformation.py): support audio input as a url

enables passing google cloud bucket urls

* fix(gemini/transformation.py): support explicitly passing format of file

* fix(gemini/transformation.py): expand support for inferred file types from url

* fix(sagemaker/completion/transformation.py): fix special token error when counting sagemaker tokens

* test: fix import
2025-03-29 19:23:09 -07:00
Krish Dholakia
5ac61a7572
Add bedrock latency optimized inference support (#9623)
* fix(converse_transformation.py): add performanceConfig param support on bedrock

Closes https://github.com/BerriAI/litellm/issues/7606

* fix(converse_transformation.py): refactor to use more flexible single getter for params which are separate config blocks

* test(test_main.py): add e2e mock test for bedrock performance config

* build(model_prices_and_context_window.json): add versioned multimodal embedding

* refactor(multimodal_embeddings/): migrate to config pattern

* feat(vertex_ai/multimodalembeddings): calculate usage for multimodal embedding calls

Enables cost calculation for multimodal embeddings

* feat(vertex_ai/multimodalembeddings): get usage object for embedding calls

ensures accurate cost tracking for vertexai multimodal embedding calls

* fix(embedding_handler.py): remove unused imports

* fix: fix linting errors

* fix: handle response api usage calculation

* test(test_vertex_ai_multimodal_embedding_transformation.py): update tests

* test: mark flaky test

* feat(vertex_ai/multimodal_embeddings/transformation.py): support text+image+video input

* docs(vertex.md): document sending text + image to vertex multimodal embeddings

* test: remove incorrect file

* fix(multimodal_embeddings/transformation.py): fix linting error

* style: remove unused import
2025-03-29 00:23:09 -07:00
Krish Dholakia
c0845fec1f
Add OpenAI gpt-4o-transcribe support (#9517)
* refactor: introduce new transformation config for gpt-4o-transcribe models

* refactor: expose new transformation configs for audio transcription

* ci: fix config yml

* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions

allows gpt-4o and whisper audio transformation to work as expected

* refactor: migrate fireworks ai + deepgram to new transform request pattern

* feat(openai/): working support for gpt-4o-audio-transcribe

* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map

* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`

* fix(get_supported_openai_params.py): fix return

* refactor(deepgram/): migrate unit test to deepgram handler

* refactor: cleanup unused imports

* fix(get_supported_openai_params.py): fix linting error

* test: update test
2025-03-26 23:10:25 -07:00
Krrish Dholakia
109add7946 build(model_prices_and_context_window.json): add gemini multimodal embedding cost 2025-03-26 23:04:24 -07:00