Commit graph

1380 commits

Author SHA1 Message Date
Marty Sullivan
0909eee744
add missing bedrock models (#22810) 2026-03-04 15:13:09 -08:00
Chesars
0e1a633e30 fix: update mode to realtime for gemini-live models
The mode field is used by health checks to determine the correct
check method (WebSocket for realtime vs REST for chat).
2026-03-04 19:43:23 -03:00
Chesars
ddf9598f30 fix: use /v1/realtime for gemini/ provider live model
The gemini/ prefix indicates Google AI Studio, which uses /v1/realtime
endpoint (OpenAI-compatible), not /vertex_ai/live.
2026-03-04 19:43:23 -03:00
Chesars
20a41a67d6 fix: update gemini-live model supported_endpoints to /vertex_ai/live
The gemini-live-2.5-flash-preview-native-audio-09-2025 model only works
with WebSocket (Live API), not REST endpoints. Changed supported_endpoints
from /v1/chat/completions to /vertex_ai/live to reflect the actual
passthrough endpoint available in LiteLLM proxy.
2026-03-04 19:43:23 -03:00
Peter Dave Hello
007bea10b8
Add Support for OpenAI's Chat-GPT 5.3 Chat model (#22693)
Reference:
- https://openai.com/index/gpt-5-3-instant/
- https://developers.openai.com/api/docs/models/gpt-5.3-chat-latest
2026-03-03 20:27:05 -08:00
Sameer Kankute
7a83acf086
Merge pull request #22620 from OiPunk/codex/litellm-22619-deepinfra-function-calling
fix: add missing supports_function_calling for deepinfra models
2026-03-04 08:51:21 +05:30
Varad Khonde
7d65df351f feat(togetherai): add support for TogetherAI Qwen3.5-397B-A17B model 2026-03-03 23:40:41 +05:30
Sameer Kankute
daa0397f30
Merge pull request #22674 from BerriAI/litellm_gemini-3.1-flash-lite-preview
Day 0 gemini 3.1 flash lite preview support
2026-03-03 22:23:17 +05:30
Sameer Kankute
9d06106af0 Fix gemini-3.1-flash-lite-preview for streaming 2026-03-03 22:22:57 +05:30
Sameer Kankute
c3fe4634b6 Add correct pricing for gemini 3.1 flash lite 2026-03-03 22:06:16 +05:30
Cesar Garcia
a0cc550f48
Merge pull request #22665 from Chesars/feat/add-zai-glm-5-model-pricing
feat(models): add zai/glm-5 and zai/glm-5-code to model cost map
2026-03-03 12:17:42 -03:00
Chesars
75518c3ca7 feat(models): add zai/glm-5 and zai/glm-5-code to model cost map
Add native ZhipuAI GLM-5 and GLM-5-Code model entries with pricing
from docs.z.ai/guides/overview/pricing.
2026-03-03 12:03:40 -03:00
Cesar Garcia
dfd7203d7f
Merge pull request #22650 from Chesars/fix/add-missing-dashscope-glm5-pricing
fix(pricing): add 7 missing DashScope model pricing entries
2026-03-03 11:53:30 -03:00
Cesar Garcia
105a40ba4f
Merge pull request #22649 from Chesars/fix/add-missing-openrouter-model-pricing
fix(pricing): add 5 missing OpenRouter model pricing entries
2026-03-03 11:53:07 -03:00
Sameer Kankute
24ec7f882f
Revert "feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model" 2026-03-03 19:35:26 +05:30
Varad Khonde
d66f8bc15d feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model 2026-03-03 19:17:28 +05:30
Chesars
7977126181 fix(pricing): add 7 missing DashScope model pricing entries
Fixes #22646

Adds pricing for DashScope models that were missing from the cost map,
causing $0 spend tracking in the proxy dashboard:

- dashscope/qwen3-max-2026-01-23 (tiered, same as qwen3-max)
- dashscope/qwen3-next-80b-a3b-instruct ($0.15/$1.20 per 1M)
- dashscope/qwen3-next-80b-a3b-thinking ($0.15/$1.20 per 1M)
- dashscope/qwen3-vl-235b-a22b-instruct ($0.40/$1.60 per 1M)
- dashscope/qwen3-vl-235b-a22b-thinking ($0.40/$4.00 per 1M)
- dashscope/qwen3-vl-32b-instruct ($0.16/$0.64 per 1M)
- dashscope/qwen3-vl-32b-thinking ($0.16/$2.87 per 1M)
2026-03-03 09:52:17 -03:00
Chesars
409208771e fix(pricing): add 5 missing OpenRouter model pricing entries
Fixes #22609

Adds pricing for OpenRouter models that were routing correctly but
returning $0 for spend tracking due to missing cost map entries:

- openrouter/anthropic/claude-sonnet-4.6 ($3.00/$15.00 per 1M tokens)
- openrouter/google/gemini-3.1-pro-preview ($2.00/$12.00 per 1M tokens)
- openrouter/openai/gpt-5.1-codex-max ($1.25/$10.00 per 1M tokens)
- openrouter/qwen/qwen3-coder-plus ($1.00/$5.00 per 1M tokens)
- openrouter/z-ai/glm-5 ($0.80/$2.56 per 1M tokens)
2026-03-03 09:24:49 -03:00
Sameer Kankute
851be58775 Add day 0 support of gemini-3.1-flash-lite-preview 2026-03-03 15:07:47 +05:30
liweiguang
81ddf08494 fix: add missing supports_function_calling for deepinfra models
All 55 deepinfra models that had `supports_tool_choice: true` were
missing the `supports_function_calling` flag, causing
`litellm.supports_function_calling()` to incorrectly return False.

Fixes #22619

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 12:12:43 +08:00
Ishaan Jaff
86d5b4c632
feat: add Nebius AI Studio models to model_prices_and_context_window.json (#22614)
Add 30 Nebius AI Studio models covering:
- Text-to-text: DeepSeek (R1, R1-0528, R1-Distill, V3, V3-0324), Meta Llama
  (3.1-8B/70B/405B, 3.3-70B), Qwen (3-235B/32B/30B/14B/4B, 2.5-72B/32B,
  2.5-Coder-7B, QwQ-32B), Mistral Nemo, NousResearch Hermes-3, NVIDIA
  Nemotron Ultra/Super, Google Gemma-3-27B, Llama-Guard-3
- Vision: Qwen2.5-VL-72B, Qwen2-VL-72B, Qwen2-VL-7B
- Embedding: BAAI/bge-en-icl, BAAI/bge-multilingual-gemma2, intfloat/e5-mistral-7b

Pricing sourced from https://nebius.com/prices-ai-studio (base flavor).
Context windows sourced from https://docs.nebius.com/studio/inference/models/

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-02 18:43:07 -08:00
Ishaan Jaff
bfceb7fc3f
feat(perplexity): add embedding support for pplx-embed-v1 models (#22610)
* feat: add Perplexity embedding support (pplx-embed-v1)

Add support for Perplexity AI's embedding models via the LLM HTTP handler:

Models:
- pplx-embed-v1-0.6b (1024 dims, 32K context, $0.004/1M tokens)
- pplx-embed-v1-4b (2560 dims, 32K context, $0.03/1M tokens)

Implementation:
- PerplexityEmbeddingConfig in litellm/llms/perplexity/embedding/
- Registered in ProviderConfigManager, __init__.py lazy imports, main.py dispatch
- Model pricing added to model_prices_and_context_window.json
- Supports dimensions and encoding_format parameters
- Uses base_llm_http_handler.embedding() pattern

Tests:
- 19 unit tests covering transformation, params, URLs, provider config, model info

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* docs: add Perplexity AI embeddings documentation

- Create providers/perplexity_embedding.md with SDK and proxy usage examples
- Convert Perplexity from flat doc to category in sidebars.js
- Category includes existing chat/responses doc + new embeddings doc
- Covers pplx-embed-v1-0.6b and pplx-embed-v1-4b models
- Documents supported parameters (dimensions, encoding_format)
- Includes proxy config and curl examples

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: decode Perplexity base64_int8 embeddings to OpenAI-format float arrays

Perplexity returns embeddings as base64-encoded signed int8 values by default,
not float arrays like OpenAI. This commit adds decoding in
transform_embedding_response so the proxy returns standard OpenAI-compatible
float arrays (normalized to [-1, 1]).

- Added _decode_base64_embedding() static method
- Handles both base64 strings (decoded) and float lists (passthrough)
- Added 3 new tests for base64 decoding + passthrough

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-02 17:37:50 -08:00
Cesar Garcia
835a2c3dc6
Merge pull request #22583 from Chesars/fix/add-bedrock-cache-token-pricing
fix(pricing): add missing cache token pricing for 24 Bedrock Claude models
2026-03-02 18:46:28 -03:00
Cesar Garcia
680b9ee9f2
Merge pull request #22586 from Chesars/fix/update-gemini-deprecation-dates
fix: update Gemini model deprecation dates
2026-03-02 18:45:30 -03:00
Cesar Garcia
a54a1d27d7
Merge pull request #22596 from Chesars/fix/add-dashscope-models-pricing
fix: add missing pricing for dashscope/qwen3.5-plus and dashscope/qwen3-vl-plus
2026-03-02 18:45:08 -03:00
Cesar Garcia
229eb5234d
Merge pull request #22601 from Chesars/fix/update-mistral-models-pricing
feat: add missing Mistral models and update pricing
2026-03-02 18:44:38 -03:00
Chesars
884f7c5e4e fix: update mistral-small-latest to match Small 3.2 specs
mistral-small-latest now points to Small 3.2 (since June 2025).
Updated pricing from $0.10/$0.30 to $0.06/$0.18 per 1M tokens,
context from 32k to 131k, and added vision support to match
mistral-small-3-2-2506.
2026-03-02 18:36:05 -03:00
Chesars
abb7eb250a fix: remove retired Saba model from new entries
Saba was retired on 9/30/2025 per Mistral docs, replaced by Small 3.2.
2026-03-02 18:19:46 -03:00
Chesars
bd822a7a68 fix: add supports_response_schema to Ministral 3 models
Ministral 3 (3B, 8B, 14B) support structured outputs per Mistral docs.
2026-03-02 18:19:02 -03:00
Chesars
619f53d55a feat: add missing Mistral models and update outdated pricing
Add 9 new Mistral models (mistral-large-2512, mistral-medium-3-1-2508,
mistral-small-3-2-2506, ministral-3-3b/8b/14b-2512, saba-2502,
magistral-medium/small-1-2-2509) and update mistral-large-latest,
mistral-large-3, and mistral-medium-latest with correct pricing and
context windows.

Fixes #22585
2026-03-02 18:02:41 -03:00
Chesars
f0e571413d fix: add missing pricing for dashscope/qwen3.5-plus and dashscope/qwen3-vl-plus
Fixes #22591 - These models were missing from the pricing JSON, causing
$0 cost tracking when routed via the dashscope/* wildcard.

Pricing sourced from official Alibaba Cloud Model Studio docs (international tier).
2026-03-02 17:24:19 -03:00
Chesars
ee3475d187 fix: correct gemini/gemini-2.0-flash-lite-preview-02-05 deprecation_date
Update from 2025-12-02 to 2025-12-09 per
https://ai.google.dev/gemini-api/docs/deprecations
2026-03-02 15:54:24 -03:00
Chesars
53dc4ee7ef fix: revert gemini/gemini-2.0-flash-live-001 deprecation_date to 2025-12-09
The June 1 date is for Vertex AI, but this entry is for the Gemini API
where the shutdown date is December 9, 2025 per
https://ai.google.dev/gemini-api/docs/deprecations
2026-03-02 15:52:01 -03:00
Chesars
ad1ab9e874 fix: add deprecation_date for gemini/gemini-3-pro-preview (Gemini API)
Gemini API shuts down gemini-3-pro-preview on 2026-03-09, per
https://ai.google.dev/gemini-api/docs/deprecations
2026-03-02 15:48:00 -03:00
Chesars
9c8620db00 fix: update Gemini model deprecation dates per Google notifications
- gemini-3-pro-preview: add deprecation_date 2026-03-26 (Vertex AI)
- gemini-2.0-flash / flash-001: update to 2026-06-01
- gemini-2.0-flash-lite / lite-001: update to 2026-06-01
- gemini/gemini-2.0-flash-live-001: update to 2026-06-01
- Also updated deepinfra, openrouter, vercel_ai_gateway variants
2026-03-02 15:40:40 -03:00
Chesars
5c4d3d85e5 fix(pricing): add missing cache token pricing for 24 Bedrock Claude models
Bedrock Claude models were missing cache_read_input_token_cost and
cache_creation_input_token_cost fields, causing cache tokens to be
billed at the full input rate instead of the discounted cache rate.

Added pricing using Bedrock's documented multipliers (0.1x for cache
read, 1.25x for cache write) consistent with all existing entries.
2026-03-02 15:02:21 -03:00
Dima-Mediator
13e74dd389
fix(model-map): correct litellm_provider for gemini/gemini-2.5-flash-image (#22156)
The `gemini/gemini-2.5-flash-image` entry had `litellm_provider` set to
`vertex_ai-language-models` instead of `gemini`. This causes a provider
mismatch in `_check_provider_match()` when the model is used via the
Gemini API provider (`custom_llm_provider="gemini"`), resulting in a
noisy error log on every request:

  "This model isn't mapped yet. model=gemini/gemini-2.5-flash-image,
   custom_llm_provider=gemini"

The `vertex_ai/gemini-2.5-flash-image` entry already exists with the
correct `vertex_ai-language-models` provider, and the sibling
`gemini/gemini-2.5-flash-image-preview` entry correctly uses `gemini`.
2026-02-28 16:00:00 -08:00
Cesar Garcia
7d084dfb9d
Merge pull request #20525 from Chesars/docs/opus-4-6-openrouter-and-1m-context
docs: add OpenRouter Opus 4.6 to model map and update Claude Opus 4.6 docs
2026-02-27 19:07:31 -03:00
Chesars
c47d219806 docs: add openrouter/anthropic/claude-opus-4.6 to main model cost map 2026-02-27 19:04:49 -03:00
Chesars
b5cd2820b1 Merge upstream/main into feat/add-openrouter-native-models
Resolve conflict in model_prices_and_context_window.json by keeping both
the new minimax-m2.5 model from upstream and the OpenRouter native models
from this branch.
2026-02-27 18:40:56 -03:00
Cesar Garcia
88c465b5ef
Merge pull request #20102 from Chesars/xai-deprecation-dates
Add deprecation dates for xAI grok-2-vision-1212 and grok-3-mini models
2026-02-27 18:11:53 -03:00
Cesar Garcia
1943e02cc7
Merge pull request #21582 from Chesars/fix/publicai-apertus-function-calling
fix(models): function calling for PublicAI Apertus models
2026-02-27 17:38:03 -03:00
Cesar Garcia
ea5d551525
Merge pull request #22304 from Chesars/feat/add-gpt-realtime-1.5-model
feat(models): add gpt-realtime-1.5 to model cost map
2026-02-27 16:38:16 -03:00
Cesar Garcia
d8fa43a0cc
Merge pull request #22303 from Chesars/feat/add-gpt-audio-1.5-model
feat(models): add gpt-audio-1.5 to model cost map
2026-02-27 16:37:31 -03:00
Chesars
1ca4dd8542 fix: gpt-audio-1.5 only supports /v1/chat/completions endpoint 2026-02-27 16:11:15 -03:00
Chesars
da73e54b1b fix: gpt-realtime-1.5 only supports /v1/realtime endpoint
Remove /v1/chat/completions and /v1/responses from supported_endpoints
and revert the audio model detection change since gpt-realtime-1.5 does
not go through Chat Completions.
2026-02-27 16:10:01 -03:00
Chesars
1f887547f6 feat(models): add gpt-realtime-1.5 to model cost map
New OpenAI realtime model released 2026-02-23. Adds pricing and capability
metadata for gpt-realtime-1.5 (32K context, 4K output, audio/image/text I/O).
Unlike gpt-realtime, this model also supports Chat Completions and Responses
endpoints (not just WebSocket).

Closes #22266
2026-02-27 16:01:09 -03:00
Chesars
1144d05cba feat(models): add gpt-audio-1.5 to model cost map
New OpenAI audio model released 2026-02-23. Adds pricing and capability
metadata for gpt-audio-1.5 (128K context, 16K output, audio I/O).

Closes #22269
2026-02-27 15:57:56 -03:00
Emerson Gomes
f24a41898b feat(vertex): add gemini-3.1-flash-image-preview model DB support
- add gemini-3.1-flash-image-preview + vertex_ai alias entries\n- set pricing to Gemini 3.1 Flash Image Preview rates\n- mirror updates in packaged backup model map\n- update llm cost calc regression test to cover new model
2026-02-26 14:41:59 -06:00
Jins K Joy
296a138195
Adjust input and output cost per token for mistral-small-2503 (#22097)
* Adjust input and output cost per token for mistral-small-2503

Cost per million for mistral-small-2503 is not correct.

In Azure Documentation:
Pay-as-you-go (per 1,000 tokens)
$0.0001
Model input
$0.0003
Model output

* Update input and output cost per token for model
2026-02-26 00:09:14 -08:00