scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8
34478 commits 3745 branches 1395 tags 3.9 GiB
Commit graph

1387 commits

Author SHA1 Message Date
Antti Puurula
20ce6ddbef
Fix apac.anthropic.claude-sonnet-4-6 -> au.anthropic.claude-sonnet-4-6 (#21466) 2026-03-05 15:54:26 -08:00
Sameer Kankute
8dca085640
Merge pull request #22916 from BerriAI/litellm_gpt-5.4_day_0 
Add day 0 support for gpt-5.4
 2026-03-05 23:41:35 +05:30
Sameer Kankute
3b457b5d8e Add day 0 support for gpt-5.4 2026-03-05 23:40:24 +05:30
Sameer Kankute
b9a8d42882 Add day 0 support for gpt-5.4 2026-03-05 23:26:24 +05:30
Sameer Kankute
5183a6e850
Merge pull request #22866 from mubashir1osmani/feat/bedrock-mantle-provider-clean 
feat: bedrock mantle provider
 2026-03-05 18:24:00 +05:30
Sameer Kankute
bb1297fe1b feat(cost): add azure_ai/mistral-document-ai-2512 to model cost map 
Made-with: Cursor
 2026-03-05 16:07:46 +05:30
mubashir1osmani
df7e3aa1e5 feat(provider): add Amazon Bedrock Mantle as a first-class provider 
Adds `bedrock_mantle` provider for Amazon Bedrock's OpenAI-compatible
inference engine (Project Mantle). Previously users had to use this as
a generic openai_compatible provider, which resulted in incorrect pricing
(OpenAI rates instead of Bedrock rates).

Changes:
- New `BedrockMantleChatConfig` extending `OpenAILikeChatConfig`
  - Regional API base: `https://bedrock-mantle.{region}.api.aws/v1`
  - Auth via `BEDROCK_MANTLE_API_KEY` env var
  - Region resolution: BEDROCK_MANTLE_REGION > AWS_REGION > us-east-1
  - Supports reasoning for gpt-oss models
- Added `BEDROCK_MANTLE` to `LlmProviders` enum
- Added 4 models with correct AWS Bedrock pricing to both pricing files:
  - bedrock_mantle/openai.gpt-oss-120b ($0.15/M in, $0.60/M out)
  - bedrock_mantle/openai.gpt-oss-20b ($0.075/M in, $0.30/M out)
  - bedrock_mantle/openai.gpt-oss-safeguard-120b
  - bedrock_mantle/openai.gpt-oss-safeguard-20b
- Wired provider into get_llm_provider_logic, get_supported_openai_params,
  main.py routing, utils.py map_openai_params + ProviderConfigManager,
  and _lazy_imports_registry
- 19 unit tests covering registration, config, provider resolution, pricing

Usage:
  os.environ["BEDROCK_MANTLE_API_KEY"] = "your-key"
  litellm.completion(model="bedrock_mantle/openai.gpt-oss-120b", ...)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-05 00:03:40 -05:00
Marty Sullivan
0909eee744
add missing bedrock models (#22810) 2026-03-04 15:13:09 -08:00
Chesars
0e1a633e30 fix: update mode to realtime for gemini-live models 
The mode field is used by health checks to determine the correct
check method (WebSocket for realtime vs REST for chat).
 2026-03-04 19:43:23 -03:00
Chesars
ddf9598f30 fix: use /v1/realtime for gemini/ provider live model 
The gemini/ prefix indicates Google AI Studio, which uses /v1/realtime
endpoint (OpenAI-compatible), not /vertex_ai/live.
 2026-03-04 19:43:23 -03:00
Chesars
20a41a67d6 fix: update gemini-live model supported_endpoints to /vertex_ai/live 
The gemini-live-2.5-flash-preview-native-audio-09-2025 model only works
with WebSocket (Live API), not REST endpoints. Changed supported_endpoints
from /v1/chat/completions to /vertex_ai/live to reflect the actual
passthrough endpoint available in LiteLLM proxy.
 2026-03-04 19:43:23 -03:00
Peter Dave Hello
007bea10b8
Add Support for OpenAI's Chat-GPT 5.3 Chat model (#22693) 
Reference:
- https://openai.com/index/gpt-5-3-instant/
- https://developers.openai.com/api/docs/models/gpt-5.3-chat-latest
 2026-03-03 20:27:05 -08:00
Sameer Kankute
7a83acf086
Merge pull request #22620 from OiPunk/codex/litellm-22619-deepinfra-function-calling 
fix: add missing supports_function_calling for deepinfra models
 2026-03-04 08:51:21 +05:30
Varad Khonde
7d65df351f feat(togetherai): add support for TogetherAI Qwen3.5-397B-A17B model 2026-03-03 23:40:41 +05:30
Sameer Kankute
daa0397f30
Merge pull request #22674 from BerriAI/litellm_gemini-3.1-flash-lite-preview 
Day 0 gemini 3.1 flash lite preview support
 2026-03-03 22:23:17 +05:30
Sameer Kankute
9d06106af0 Fix gemini-3.1-flash-lite-preview for streaming 2026-03-03 22:22:57 +05:30
Sameer Kankute
c3fe4634b6 Add correct pricing for gemini 3.1 flash lite 2026-03-03 22:06:16 +05:30
Cesar Garcia
a0cc550f48
Merge pull request #22665 from Chesars/feat/add-zai-glm-5-model-pricing 
feat(models): add zai/glm-5 and zai/glm-5-code to model cost map
 2026-03-03 12:17:42 -03:00
Chesars
75518c3ca7 feat(models): add zai/glm-5 and zai/glm-5-code to model cost map 
Add native ZhipuAI GLM-5 and GLM-5-Code model entries with pricing
from docs.z.ai/guides/overview/pricing.
 2026-03-03 12:03:40 -03:00
Cesar Garcia
dfd7203d7f
Merge pull request #22650 from Chesars/fix/add-missing-dashscope-glm5-pricing 
fix(pricing): add 7 missing DashScope model pricing entries
 2026-03-03 11:53:30 -03:00
Cesar Garcia
105a40ba4f
Merge pull request #22649 from Chesars/fix/add-missing-openrouter-model-pricing 
fix(pricing): add 5 missing OpenRouter model pricing entries
 2026-03-03 11:53:07 -03:00
Sameer Kankute
24ec7f882f
Revert "feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model" 2026-03-03 19:35:26 +05:30
Varad Khonde
d66f8bc15d feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model 2026-03-03 19:17:28 +05:30
Chesars
7977126181 fix(pricing): add 7 missing DashScope model pricing entries 
Fixes #22646

Adds pricing for DashScope models that were missing from the cost map,
causing $0 spend tracking in the proxy dashboard:

- dashscope/qwen3-max-2026-01-23 (tiered, same as qwen3-max)
- dashscope/qwen3-next-80b-a3b-instruct ($0.15/$1.20 per 1M)
- dashscope/qwen3-next-80b-a3b-thinking ($0.15/$1.20 per 1M)
- dashscope/qwen3-vl-235b-a22b-instruct ($0.40/$1.60 per 1M)
- dashscope/qwen3-vl-235b-a22b-thinking ($0.40/$4.00 per 1M)
- dashscope/qwen3-vl-32b-instruct ($0.16/$0.64 per 1M)
- dashscope/qwen3-vl-32b-thinking ($0.16/$2.87 per 1M)
 2026-03-03 09:52:17 -03:00
Chesars
409208771e fix(pricing): add 5 missing OpenRouter model pricing entries 
Fixes #22609

Adds pricing for OpenRouter models that were routing correctly but
returning $0 for spend tracking due to missing cost map entries:

- openrouter/anthropic/claude-sonnet-4.6 ($3.00/$15.00 per 1M tokens)
- openrouter/google/gemini-3.1-pro-preview ($2.00/$12.00 per 1M tokens)
- openrouter/openai/gpt-5.1-codex-max ($1.25/$10.00 per 1M tokens)
- openrouter/qwen/qwen3-coder-plus ($1.00/$5.00 per 1M tokens)
- openrouter/z-ai/glm-5 ($0.80/$2.56 per 1M tokens)
 2026-03-03 09:24:49 -03:00
Sameer Kankute
851be58775 Add day 0 support of gemini-3.1-flash-lite-preview 2026-03-03 15:07:47 +05:30
liweiguang
81ddf08494 fix: add missing supports_function_calling for deepinfra models 
All 55 deepinfra models that had `supports_tool_choice: true` were
missing the `supports_function_calling` flag, causing
`litellm.supports_function_calling()` to incorrectly return False.

Fixes #22619

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-03 12:12:43 +08:00
Ishaan Jaff
86d5b4c632
feat: add Nebius AI Studio models to model_prices_and_context_window.json (#22614) 
Add 30 Nebius AI Studio models covering:
- Text-to-text: DeepSeek (R1, R1-0528, R1-Distill, V3, V3-0324), Meta Llama
  (3.1-8B/70B/405B, 3.3-70B), Qwen (3-235B/32B/30B/14B/4B, 2.5-72B/32B,
  2.5-Coder-7B, QwQ-32B), Mistral Nemo, NousResearch Hermes-3, NVIDIA
  Nemotron Ultra/Super, Google Gemma-3-27B, Llama-Guard-3
- Vision: Qwen2.5-VL-72B, Qwen2-VL-72B, Qwen2-VL-7B
- Embedding: BAAI/bge-en-icl, BAAI/bge-multilingual-gemma2, intfloat/e5-mistral-7b

Pricing sourced from https://nebius.com/prices-ai-studio (base flavor).
Context windows sourced from https://docs.nebius.com/studio/inference/models/

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
 2026-03-02 18:43:07 -08:00
Ishaan Jaff
bfceb7fc3f
feat(perplexity): add embedding support for pplx-embed-v1 models (#22610) 
* feat: add Perplexity embedding support (pplx-embed-v1)

Add support for Perplexity AI's embedding models via the LLM HTTP handler:

Models:
- pplx-embed-v1-0.6b (1024 dims, 32K context, $0.004/1M tokens)
- pplx-embed-v1-4b (2560 dims, 32K context, $0.03/1M tokens)

Implementation:
- PerplexityEmbeddingConfig in litellm/llms/perplexity/embedding/
- Registered in ProviderConfigManager, __init__.py lazy imports, main.py dispatch
- Model pricing added to model_prices_and_context_window.json
- Supports dimensions and encoding_format parameters
- Uses base_llm_http_handler.embedding() pattern

Tests:
- 19 unit tests covering transformation, params, URLs, provider config, model info

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* docs: add Perplexity AI embeddings documentation

- Create providers/perplexity_embedding.md with SDK and proxy usage examples
- Convert Perplexity from flat doc to category in sidebars.js
- Category includes existing chat/responses doc + new embeddings doc
- Covers pplx-embed-v1-0.6b and pplx-embed-v1-4b models
- Documents supported parameters (dimensions, encoding_format)
- Includes proxy config and curl examples

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: decode Perplexity base64_int8 embeddings to OpenAI-format float arrays

Perplexity returns embeddings as base64-encoded signed int8 values by default,
not float arrays like OpenAI. This commit adds decoding in
transform_embedding_response so the proxy returns standard OpenAI-compatible
float arrays (normalized to [-1, 1]).

- Added _decode_base64_embedding() static method
- Handles both base64 strings (decoded) and float lists (passthrough)
- Added 3 new tests for base64 decoding + passthrough

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
 2026-03-02 17:37:50 -08:00
Cesar Garcia
835a2c3dc6
Merge pull request #22583 from Chesars/fix/add-bedrock-cache-token-pricing 
fix(pricing): add missing cache token pricing for 24 Bedrock Claude models
 2026-03-02 18:46:28 -03:00
Cesar Garcia
680b9ee9f2
Merge pull request #22586 from Chesars/fix/update-gemini-deprecation-dates 
fix: update Gemini model deprecation dates
 2026-03-02 18:45:30 -03:00
Cesar Garcia
a54a1d27d7
Merge pull request #22596 from Chesars/fix/add-dashscope-models-pricing 
fix: add missing pricing for dashscope/qwen3.5-plus and dashscope/qwen3-vl-plus
 2026-03-02 18:45:08 -03:00
Cesar Garcia
229eb5234d
Merge pull request #22601 from Chesars/fix/update-mistral-models-pricing 
feat: add missing Mistral models and update pricing
 2026-03-02 18:44:38 -03:00
Chesars
884f7c5e4e fix: update mistral-small-latest to match Small 3.2 specs 
mistral-small-latest now points to Small 3.2 (since June 2025).
Updated pricing from $0.10/$0.30 to $0.06/$0.18 per 1M tokens,
context from 32k to 131k, and added vision support to match
mistral-small-3-2-2506.
 2026-03-02 18:36:05 -03:00
Chesars
abb7eb250a fix: remove retired Saba model from new entries 
Saba was retired on 9/30/2025 per Mistral docs, replaced by Small 3.2.
 2026-03-02 18:19:46 -03:00
Chesars
bd822a7a68 fix: add supports_response_schema to Ministral 3 models 
Ministral 3 (3B, 8B, 14B) support structured outputs per Mistral docs.
 2026-03-02 18:19:02 -03:00
Chesars
619f53d55a feat: add missing Mistral models and update outdated pricing 
Add 9 new Mistral models (mistral-large-2512, mistral-medium-3-1-2508,
mistral-small-3-2-2506, ministral-3-3b/8b/14b-2512, saba-2502,
magistral-medium/small-1-2-2509) and update mistral-large-latest,
mistral-large-3, and mistral-medium-latest with correct pricing and
context windows.

Fixes #22585
 2026-03-02 18:02:41 -03:00
Chesars
f0e571413d fix: add missing pricing for dashscope/qwen3.5-plus and dashscope/qwen3-vl-plus 
Fixes #22591 - These models were missing from the pricing JSON, causing
$0 cost tracking when routed via the dashscope/* wildcard.

Pricing sourced from official Alibaba Cloud Model Studio docs (international tier).
 2026-03-02 17:24:19 -03:00
Chesars
ee3475d187 fix: correct gemini/gemini-2.0-flash-lite-preview-02-05 deprecation_date 
Update from 2025-12-02 to 2025-12-09 per
https://ai.google.dev/gemini-api/docs/deprecations
 2026-03-02 15:54:24 -03:00
Chesars
53dc4ee7ef fix: revert gemini/gemini-2.0-flash-live-001 deprecation_date to 2025-12-09 
The June 1 date is for Vertex AI, but this entry is for the Gemini API
where the shutdown date is December 9, 2025 per
https://ai.google.dev/gemini-api/docs/deprecations
 2026-03-02 15:52:01 -03:00
Chesars
ad1ab9e874 fix: add deprecation_date for gemini/gemini-3-pro-preview (Gemini API) 
Gemini API shuts down gemini-3-pro-preview on 2026-03-09, per
https://ai.google.dev/gemini-api/docs/deprecations
 2026-03-02 15:48:00 -03:00
Chesars
9c8620db00 fix: update Gemini model deprecation dates per Google notifications 
- gemini-3-pro-preview: add deprecation_date 2026-03-26 (Vertex AI)
- gemini-2.0-flash / flash-001: update to 2026-06-01
- gemini-2.0-flash-lite / lite-001: update to 2026-06-01
- gemini/gemini-2.0-flash-live-001: update to 2026-06-01
- Also updated deepinfra, openrouter, vercel_ai_gateway variants
 2026-03-02 15:40:40 -03:00
Chesars
5c4d3d85e5 fix(pricing): add missing cache token pricing for 24 Bedrock Claude models 
Bedrock Claude models were missing cache_read_input_token_cost and
cache_creation_input_token_cost fields, causing cache tokens to be
billed at the full input rate instead of the discounted cache rate.

Added pricing using Bedrock's documented multipliers (0.1x for cache
read, 1.25x for cache write) consistent with all existing entries.
 2026-03-02 15:02:21 -03:00
Dima-Mediator
13e74dd389
fix(model-map): correct litellm_provider for gemini/gemini-2.5-flash-image (#22156) 
The `gemini/gemini-2.5-flash-image` entry had `litellm_provider` set to
`vertex_ai-language-models` instead of `gemini`. This causes a provider
mismatch in `_check_provider_match()` when the model is used via the
Gemini API provider (`custom_llm_provider="gemini"`), resulting in a
noisy error log on every request:

  "This model isn't mapped yet. model=gemini/gemini-2.5-flash-image,
   custom_llm_provider=gemini"

The `vertex_ai/gemini-2.5-flash-image` entry already exists with the
correct `vertex_ai-language-models` provider, and the sibling
`gemini/gemini-2.5-flash-image-preview` entry correctly uses `gemini`.
 2026-02-28 16:00:00 -08:00
Cesar Garcia
7d084dfb9d
Merge pull request #20525 from Chesars/docs/opus-4-6-openrouter-and-1m-context 
docs: add OpenRouter Opus 4.6 to model map and update Claude Opus 4.6 docs
 2026-02-27 19:07:31 -03:00
Chesars
c47d219806 docs: add openrouter/anthropic/claude-opus-4.6 to main model cost map 2026-02-27 19:04:49 -03:00
Chesars
b5cd2820b1 Merge upstream/main into feat/add-openrouter-native-models 
Resolve conflict in model_prices_and_context_window.json by keeping both
the new minimax-m2.5 model from upstream and the OpenRouter native models
from this branch.
 2026-02-27 18:40:56 -03:00
Cesar Garcia
88c465b5ef
Merge pull request #20102 from Chesars/xai-deprecation-dates 
Add deprecation dates for xAI grok-2-vision-1212 and grok-3-mini models
 2026-02-27 18:11:53 -03:00
Cesar Garcia
1943e02cc7
Merge pull request #21582 from Chesars/fix/publicai-apertus-function-calling 
fix(models): function calling for PublicAI Apertus models
 2026-02-27 17:38:03 -03:00
Cesar Garcia
ea5d551525
Merge pull request #22304 from Chesars/feat/add-gpt-realtime-1.5-model 
feat(models): add gpt-realtime-1.5 to model cost map
 2026-02-27 16:38:16 -03:00