Commit graph

1881 commits

Author SHA1 Message Date
Mateo Wang
5bcd494e88
Merge pull request #38560 from BerriAI/devin/1787857843-registry-audit-rolling
fix(registry): add Gemini Omni 1.1 Flash, xAI grok-imagine image models, Mistral cache-read pricing, GLM 5.3 Flash + Kimi K2.7 Code entries
2026-08-28 10:18:28 -07:00
mateo-berri
1aa214e69c fix(registry): align gemini-omni-flash-preview limits with the models API
The models API reports 131072 input / 65536 output for the preview model
and the Interactions API accepts 100k tokens but rejects 130k, so the
1,048,576 input limit copied from the docs was wrong.
2026-08-28 09:19:22 -07:00
mateo-berri
02f787308d fix(registry): correct gemini omni, grok-4.20 multi-agent, kimi-k2.7-code entries
Gemini omni 1.1 flash and omni flash preview only answer on the Interactions
API, so both now list /v1beta/interactions as their endpoint and 1.1 flash
gets the 131072 / 65536 limits the models API reports.

grok-4.20-multi-agent and -latest now match the dated entry (mode responses,
/v1/responses only), and all three drop function calling and tool choice
since the API rejects client-side tools outside a beta.

kimi-k2.7-code gets the capability flags kimi-k2.6 carries (tools, reasoning,
JSON mode, image and video input) plus max_output_tokens.

grok-imagine-image-2.0 gets a low quality tier at $0.04 so quality=low is
not billed at the $0.06 default.
2026-08-28 09:17:46 -07:00
Devin AI
2b02c95ff2 fix(registry): add zai/glm-5.3-flash, databricks-glm-5-3-flash, moonshot/kimi-k2.7-code; xai grok-imagine-image-pro deprecation date
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-28 13:13:38 +00:00
Devin AI
f153203ebe Merge remote-tracking branch 'origin/litellm_internal_staging' into devin/1787857843-registry-audit-rolling 2026-08-28 13:04:58 +00:00
mateo-berri
837bcba32d fix(model_prices): add bedrock_mantle gpt-5.5/5.4 272K tiers, align sol with AWS invoice
AWS bills a Bedrock GPT-5.5 or GPT-5.4 prompt past 272K tokens under the long-context usage types for the
whole prompt, at 2x input, 2x cache read, and 1.5x output, and the cost map only had the flat rates, so a
300K prompt was logged at half of what the invoice charges. The map's promo rates for gpt-5.6-sol are 20%
under the $5.50 input, $33.00 output, $0.55 cache read, and $6.88 cache write per million the invoice bills.

Adds the *_above_272k_tokens fields to gpt-5.5 and gpt-5.4, moves sol's base and tier rates to the invoiced
ones, replaces the test that pinned the flat behaviour with one that pins the invoiced numbers, and updates
the sol pins in the mantle transformation tests
2026-08-27 19:31:45 -07:00
Mateo Wang
10cd9259a3
Merge pull request #38100 from FelipeRodriguesGare/bugfix/tencent-thinking-extra-body
fix(tencent): route thinking through extra_body in chat completions
2026-08-27 18:23:11 -07:00
tin-berri
30ff3723b2
feat(model_prices): let a map entry declare its exact reasoning_effort levels (#38481)
Kimi K3 accepts exactly low, high and max, defaults to max, and always thinks.
The map could not say that: medium and high have no supports_*_reasoning_effort
flag because every other reasoning model takes them, so the ten kimi-k3 entries
carried supports_reasoning alone and resolved to unknown. The dashboard then fell
back to a capability-blind level list that deliberately omits max, which is why a
kimi-k3 tier cannot be set to max thinking today.

Add reasoning_effort_levels, an array key in the shape the map already uses for
supported_endpoints and supported_modalities. Where present it is read first and
wins whole; every other entry keeps answering through the per-level flags,
unchanged. It is deliberately a different name from the computed
ModelGroupInfo.supported_reasoning_efforts, which stays derived from a group's
deployments and is never seeded from one deployment's model_info.

The levels are per entry rather than per model, because the deployments differ:
Moonshot, Together, Fireworks and Azure Foundry all forward the level unchanged
and get the model's own low/high/max, while Perplexity documents a six-value
enum it maps down internally and gets that. The /v1/messages degradation chain
consults the same declaration, so the level the map advertises is the level that
path forwards.
2026-08-27 15:38:01 -07:00
Mateo Wang
a6816f0e96
Merge pull request #38486 from BerriAI/litellm_together_glm53_flash
feat(together_ai): add zai-org/GLM-5.3-Flash to the model registry
2026-08-27 15:24:30 -07:00
Mateo Wang
dc1b847c4f
Merge pull request #38280 from BerriAI/litellm_together_cache_pricing
fix(cost): apply Together AI cache read pricing and per-model registry rates
2026-08-27 15:20:09 -07:00
mateo-berri
1665214bbd feat(together_ai): flag prompt caching on GLM-5.3-Flash like its sibling entries 2026-08-27 15:13:42 -07:00
mateo-berri
ae89f9cf74 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_together_glm53_flash 2026-08-27 15:12:35 -07:00
Mateo Wang
a2c814654e
Merge pull request #38449 from BerriAI/litellm_dashscope_qwen_image_3
feat(dashscope): support qwen-image-3.0 and qwen-image-3.0-pro image generation
2026-08-27 15:08:39 -07:00
mateo-berri
dbadee7210 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_together_cache_pricing
# Conflicts:
#	litellm/model_prices_and_context_window_backup.json
#	model_prices_and_context_window.json
#	tests/test_litellm/test_cost_calculator.py
2026-08-27 14:29:46 -07:00
Devin AI
afe61aa0c8 fix(registry): price xai grok-imagine generated images via input_cost_per_image for default calculator
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-27 20:50:00 +00:00
Devin AI
8a12cae14d merge litellm_internal_staging: resolve registry conflicts, defer to upstream gemini 3.5 transcribe entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-27 20:48:24 +00:00
Mateo Wang
67c7b97fd2
Merge pull request #38207 from BerriAI/litellm_registry_audit_bedrock_sol_anthropic_1hr
fix(model_prices): rolling registry audit - verified models and rates for Novita, DeepInfra, W&B, Bedrock Sol, Gemini, Fireworks, Azure gpt-5.6, Mistral, Together
2026-08-27 13:42:51 -07:00
Devin AI
90073864ec fix(registry): add tpm/rpm to new gemini entries per repo convention
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-27 19:21:21 +00:00
Devin AI
bfc1dc73a1 registry audit: add gemini 3.5 transcribe + omni 1.1 flash, xai grok-imagine image models and grok-4.20 aliases, mistral cache-read pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-27 19:11:13 +00:00
mateo-berri
6a766ae4f7 fix(gemini): add tpm and rpm to the gemini-3.5-transcribe registry entries 2026-08-27 11:18:48 -07:00
mateo-berri
ef4c84dc36 feat(gemini): day-0 support for gemini-3.5-transcribe and transcribe-live
Adds a Gemini audio transcription config that maps /v1/audio/transcriptions
onto the Interactions API (speaker attribution and word timestamps land on
the OpenAI verbose_json shape), registers both models with published pricing,
routes text-only Live sessions to TEXT responseModalities so
gemini-3.5-transcribe-live sessions survive, and makes the token-priced
transcription cost path provider-aware instead of hardcoding OpenAI.
2026-08-27 10:08:23 -07:00
mateo-berri
a0689f04c4 fix(model_prices): cap ministral-3-3b at Mistral API's 131072 and mirror Anthropic family flags on new DeepInfra Claude rows 2026-08-27 09:48:19 -07:00
Devin AI
5d26ae0fcd fix(model_prices): absorb Databricks/Z.AI and xAI registry PRs, add Together and Azure deprecation dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-27 13:16:12 +00:00
Devin AI
4b3e82b8a3 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_bedrock_sol_anthropic_1hr 2026-08-27 13:03:14 +00:00
mateo-berri
b4c6e01fcc feat(together_ai): add zai-org/GLM-5.3-Flash to the model registry
Adds pricing (0.15/0.50 per 1M tokens, 0.03 cached read), the 1M context window, and capability flags (tools, parallel tools, tool choice, response schema, reasoning, vision) for Together AI's zai-org/GLM-5.3-Flash, mirrored into the backup cost map, with exact-value regression tests.
2026-08-27 01:12:12 -07:00
Devin AI
f7c9c87280 feat(dashscope): support qwen-image-3.0 and qwen-image-3.0-pro image generation
Register both models, route image requests to the multimodal generation endpoint instead of the chat compatible-mode base, and pass OpenAI n through as DashScope n so multi-image requests return every image.
2026-08-27 01:59:51 +00:00
Mateo Wang
d8595cb647
Merge pull request #38423 from BerriAI/litellm_gemini_latest_cache_read_rates
fix(model_prices): bill gemini -latest/preview alias cache reads at 10% of input
2026-08-26 17:25:24 -07:00
Mateo Wang
1ac39b10ba
Merge pull request #38412 from BerriAI/litellm_fix_gemini_tts_native_audio_rates
fix(cost-map): correct Gemini TTS and native-audio rates
2026-08-26 17:15:46 -07:00
Mateo Wang
b54f7505a3
Merge pull request #38419 from BerriAI/litellm_gemini_live_realtime_cost
fix(cost): price gemini-live-2.5-flash-native-audio realtime sessions
2026-08-26 17:15:41 -07:00
Mateo Wang
4e295e8eb9
Merge pull request #38422 from BerriAI/litellm_gemini35_flashlite_flex_cache_price
fix(model_prices): correct gemini-3.5-flash-lite flex cache-read pricing
2026-08-26 17:11:07 -07:00
mateo-berri
1e1c231076 fix(model_prices): bill gemini -latest/preview alias cache reads at 10% of input 2026-08-26 16:52:09 -07:00
mateo-berri
bd75c38e84 fix(model_prices): scope flash-lite flex cache-read cut to vertex entries 2026-08-26 16:48:32 -07:00
mateo-berri
dca5144dba Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_gemini_maps_grounding_cost
# Conflicts:
#	litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py
2026-08-26 16:27:40 -07:00
mateo-berri
0243c5dee4 fix(model_prices): correct gemini-3.5-flash-lite flex cache-read pricing 2026-08-26 16:25:15 -07:00
mateo-berri
ab1b7bf3b6 fix(cost): price gemini-live-2.5-flash-native-audio realtime sessions
The GA vertex model had no cost map entry, and the realtime cost handler
accepted the router's price-less auto-registered deployment entry for the
session.created model at zero-defaulted rates, so sessions billed 0.0 even
when base_model pointed at the priced preview key. Adds the GA entry at its
published rates and makes the handler fall through zero-defaulted candidates
unless their cost map entry explicitly declares pricing.
2026-08-26 15:53:37 -07:00
mateo-berri
241daa4cb7 Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_gemini_maps_grounding_cost
# Conflicts:
#	type-discipline-budget.json
2026-08-26 15:33:28 -07:00
Mateo Wang
a8f3a74360
Merge pull request #38397 from BerriAI/litellm_deepseek_vision_forwarding
fix: forward image content lists to DeepSeek vision models
2026-08-26 15:32:25 -07:00
mateo-berri
b9a790899a fix(gemini): bill Google Maps grounding as its own SKU
Gemini API Maps-grounded prompts were billed as web search and Vertex AI Maps-grounded prompts were not billed at all. Classify grounding metadata per candidate into web search vs Maps requests, carry a distinct google_maps_grounding_requests usage counter through non-streaming and streaming paths, and price it via the new google_maps_grounding_cost_per_query cost map key with per-query and per-prompt defaults keyed off web_search_billing_unit. Fixes #35906
2026-08-26 15:31:27 -07:00
mateo-berri
d565860f60 fix(cost-map): correct gemini-live native-audio text input rate 2026-08-26 15:15:05 -07:00
mateo-berri
2548e960f1 fix(cost-map): correct Gemini TTS and native-audio rates
Gemini 2.5 Flash Preview TTS, Gemini 2.5 Pro Preview TTS, and the three
gemini-2.5-flash-native-audio entries carried rates copied from the text
models, so audio output was billed 2x to 6x under Google's published
prices. Set the published per-token rates on all ten keys, add
output_cost_per_audio_token to the native-audio entries, and drop the
long-context tier rates Google does not publish for Pro TTS.
2026-08-26 14:43:21 -07:00
mateo-berri
f772cad959 fix(cost-map): backfill prompt_cache_min_tokens for the remaining Claude 4.x re-export entries 2026-08-26 14:31:45 -07:00
mateo-berri
caa97eea22 fix(cost-map): correct prompt_cache_min_tokens for Claude Fable 5 and backfill Anthropic re-export entries 2026-08-26 14:00:39 -07:00
Yucheng Zhu
df7b1f0fce fix: forward image content lists to DeepSeek vision models 2026-08-26 13:08:22 -07:00
Devin AI
7c7af51185 merge: litellm_internal_staging into rolling registry branch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 19:23:15 +00:00
Mateo Wang
74b6149d18
Merge pull request #38370 from BerriAI/litellm_azure_gpt_5_6_cache_write_pricing
fix(pricing): add azure gpt-5.6 cache write rates and correct data zone priority
2026-08-26 12:21:39 -07:00
Devin AI
4456a4407f fix(model_prices): azure gpt-5.6 cache writes, mistral missing models, together cache reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 19:20:14 +00:00
mateo-berri
ece187ea24 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_anthropic_fast_mode_cache_and_response_speed
# Conflicts:
#	tests/test_litellm/test_cost_calculator.py
2026-08-26 12:10:33 -07:00
Mateo Wang
c13b278d79
Merge pull request #38369 from BerriAI/litellm_anthropic_geo_uplift_missing_models
fix(cost-map): add US data residency uplift to claude-sonnet-4-6 and mythos entries
2026-08-26 12:05:25 -07:00
Devin AI
0cc407a02d Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_bedrock_sol_anthropic_1hr 2026-08-26 19:02:56 +00:00
Mateo Wang
80843ae7cb
Merge pull request #38279 from 6matt/litellm_bedrock_converse_gpt5_reasoning_effort
fix(bedrock): map reasoning_effort to reasoning.effort for OpenAI GPT-5.x on Converse
2026-08-26 11:48:18 -07:00