Commit graph

1871 commits

Author SHA1 Message Date
Mateo Wang
10cd9259a3
Merge pull request #38100 from FelipeRodriguesGare/bugfix/tencent-thinking-extra-body
fix(tencent): route thinking through extra_body in chat completions
2026-08-27 18:23:11 -07:00
tin-berri
30ff3723b2
feat(model_prices): let a map entry declare its exact reasoning_effort levels (#38481)
Kimi K3 accepts exactly low, high and max, defaults to max, and always thinks.
The map could not say that: medium and high have no supports_*_reasoning_effort
flag because every other reasoning model takes them, so the ten kimi-k3 entries
carried supports_reasoning alone and resolved to unknown. The dashboard then fell
back to a capability-blind level list that deliberately omits max, which is why a
kimi-k3 tier cannot be set to max thinking today.

Add reasoning_effort_levels, an array key in the shape the map already uses for
supported_endpoints and supported_modalities. Where present it is read first and
wins whole; every other entry keeps answering through the per-level flags,
unchanged. It is deliberately a different name from the computed
ModelGroupInfo.supported_reasoning_efforts, which stays derived from a group's
deployments and is never seeded from one deployment's model_info.

The levels are per entry rather than per model, because the deployments differ:
Moonshot, Together, Fireworks and Azure Foundry all forward the level unchanged
and get the model's own low/high/max, while Perplexity documents a six-value
enum it maps down internally and gets that. The /v1/messages degradation chain
consults the same declaration, so the level the map advertises is the level that
path forwards.
2026-08-27 15:38:01 -07:00
Mateo Wang
a6816f0e96
Merge pull request #38486 from BerriAI/litellm_together_glm53_flash
feat(together_ai): add zai-org/GLM-5.3-Flash to the model registry
2026-08-27 15:24:30 -07:00
Mateo Wang
dc1b847c4f
Merge pull request #38280 from BerriAI/litellm_together_cache_pricing
fix(cost): apply Together AI cache read pricing and per-model registry rates
2026-08-27 15:20:09 -07:00
mateo-berri
1665214bbd feat(together_ai): flag prompt caching on GLM-5.3-Flash like its sibling entries 2026-08-27 15:13:42 -07:00
mateo-berri
ae89f9cf74 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_together_glm53_flash 2026-08-27 15:12:35 -07:00
Mateo Wang
a2c814654e
Merge pull request #38449 from BerriAI/litellm_dashscope_qwen_image_3
feat(dashscope): support qwen-image-3.0 and qwen-image-3.0-pro image generation
2026-08-27 15:08:39 -07:00
mateo-berri
dbadee7210 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_together_cache_pricing
# Conflicts:
#	litellm/model_prices_and_context_window_backup.json
#	model_prices_and_context_window.json
#	tests/test_litellm/test_cost_calculator.py
2026-08-27 14:29:46 -07:00
Mateo Wang
67c7b97fd2
Merge pull request #38207 from BerriAI/litellm_registry_audit_bedrock_sol_anthropic_1hr
fix(model_prices): rolling registry audit - verified models and rates for Novita, DeepInfra, W&B, Bedrock Sol, Gemini, Fireworks, Azure gpt-5.6, Mistral, Together
2026-08-27 13:42:51 -07:00
mateo-berri
6a766ae4f7 fix(gemini): add tpm and rpm to the gemini-3.5-transcribe registry entries 2026-08-27 11:18:48 -07:00
mateo-berri
ef4c84dc36 feat(gemini): day-0 support for gemini-3.5-transcribe and transcribe-live
Adds a Gemini audio transcription config that maps /v1/audio/transcriptions
onto the Interactions API (speaker attribution and word timestamps land on
the OpenAI verbose_json shape), registers both models with published pricing,
routes text-only Live sessions to TEXT responseModalities so
gemini-3.5-transcribe-live sessions survive, and makes the token-priced
transcription cost path provider-aware instead of hardcoding OpenAI.
2026-08-27 10:08:23 -07:00
mateo-berri
a0689f04c4 fix(model_prices): cap ministral-3-3b at Mistral API's 131072 and mirror Anthropic family flags on new DeepInfra Claude rows 2026-08-27 09:48:19 -07:00
Devin AI
5d26ae0fcd fix(model_prices): absorb Databricks/Z.AI and xAI registry PRs, add Together and Azure deprecation dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-27 13:16:12 +00:00
Devin AI
4b3e82b8a3 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_bedrock_sol_anthropic_1hr 2026-08-27 13:03:14 +00:00
mateo-berri
b4c6e01fcc feat(together_ai): add zai-org/GLM-5.3-Flash to the model registry
Adds pricing (0.15/0.50 per 1M tokens, 0.03 cached read), the 1M context window, and capability flags (tools, parallel tools, tool choice, response schema, reasoning, vision) for Together AI's zai-org/GLM-5.3-Flash, mirrored into the backup cost map, with exact-value regression tests.
2026-08-27 01:12:12 -07:00
Devin AI
f7c9c87280 feat(dashscope): support qwen-image-3.0 and qwen-image-3.0-pro image generation
Register both models, route image requests to the multimodal generation endpoint instead of the chat compatible-mode base, and pass OpenAI n through as DashScope n so multi-image requests return every image.
2026-08-27 01:59:51 +00:00
Mateo Wang
d8595cb647
Merge pull request #38423 from BerriAI/litellm_gemini_latest_cache_read_rates
fix(model_prices): bill gemini -latest/preview alias cache reads at 10% of input
2026-08-26 17:25:24 -07:00
Mateo Wang
1ac39b10ba
Merge pull request #38412 from BerriAI/litellm_fix_gemini_tts_native_audio_rates
fix(cost-map): correct Gemini TTS and native-audio rates
2026-08-26 17:15:46 -07:00
Mateo Wang
b54f7505a3
Merge pull request #38419 from BerriAI/litellm_gemini_live_realtime_cost
fix(cost): price gemini-live-2.5-flash-native-audio realtime sessions
2026-08-26 17:15:41 -07:00
Mateo Wang
4e295e8eb9
Merge pull request #38422 from BerriAI/litellm_gemini35_flashlite_flex_cache_price
fix(model_prices): correct gemini-3.5-flash-lite flex cache-read pricing
2026-08-26 17:11:07 -07:00
mateo-berri
1e1c231076 fix(model_prices): bill gemini -latest/preview alias cache reads at 10% of input 2026-08-26 16:52:09 -07:00
mateo-berri
bd75c38e84 fix(model_prices): scope flash-lite flex cache-read cut to vertex entries 2026-08-26 16:48:32 -07:00
mateo-berri
dca5144dba Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_gemini_maps_grounding_cost
# Conflicts:
#	litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py
2026-08-26 16:27:40 -07:00
mateo-berri
0243c5dee4 fix(model_prices): correct gemini-3.5-flash-lite flex cache-read pricing 2026-08-26 16:25:15 -07:00
mateo-berri
ab1b7bf3b6 fix(cost): price gemini-live-2.5-flash-native-audio realtime sessions
The GA vertex model had no cost map entry, and the realtime cost handler
accepted the router's price-less auto-registered deployment entry for the
session.created model at zero-defaulted rates, so sessions billed 0.0 even
when base_model pointed at the priced preview key. Adds the GA entry at its
published rates and makes the handler fall through zero-defaulted candidates
unless their cost map entry explicitly declares pricing.
2026-08-26 15:53:37 -07:00
mateo-berri
241daa4cb7 Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_gemini_maps_grounding_cost
# Conflicts:
#	type-discipline-budget.json
2026-08-26 15:33:28 -07:00
Mateo Wang
a8f3a74360
Merge pull request #38397 from BerriAI/litellm_deepseek_vision_forwarding
fix: forward image content lists to DeepSeek vision models
2026-08-26 15:32:25 -07:00
mateo-berri
b9a790899a fix(gemini): bill Google Maps grounding as its own SKU
Gemini API Maps-grounded prompts were billed as web search and Vertex AI Maps-grounded prompts were not billed at all. Classify grounding metadata per candidate into web search vs Maps requests, carry a distinct google_maps_grounding_requests usage counter through non-streaming and streaming paths, and price it via the new google_maps_grounding_cost_per_query cost map key with per-query and per-prompt defaults keyed off web_search_billing_unit. Fixes #35906
2026-08-26 15:31:27 -07:00
mateo-berri
d565860f60 fix(cost-map): correct gemini-live native-audio text input rate 2026-08-26 15:15:05 -07:00
mateo-berri
2548e960f1 fix(cost-map): correct Gemini TTS and native-audio rates
Gemini 2.5 Flash Preview TTS, Gemini 2.5 Pro Preview TTS, and the three
gemini-2.5-flash-native-audio entries carried rates copied from the text
models, so audio output was billed 2x to 6x under Google's published
prices. Set the published per-token rates on all ten keys, add
output_cost_per_audio_token to the native-audio entries, and drop the
long-context tier rates Google does not publish for Pro TTS.
2026-08-26 14:43:21 -07:00
mateo-berri
f772cad959 fix(cost-map): backfill prompt_cache_min_tokens for the remaining Claude 4.x re-export entries 2026-08-26 14:31:45 -07:00
mateo-berri
caa97eea22 fix(cost-map): correct prompt_cache_min_tokens for Claude Fable 5 and backfill Anthropic re-export entries 2026-08-26 14:00:39 -07:00
Yucheng Zhu
df7b1f0fce fix: forward image content lists to DeepSeek vision models 2026-08-26 13:08:22 -07:00
Devin AI
7c7af51185 merge: litellm_internal_staging into rolling registry branch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 19:23:15 +00:00
Mateo Wang
74b6149d18
Merge pull request #38370 from BerriAI/litellm_azure_gpt_5_6_cache_write_pricing
fix(pricing): add azure gpt-5.6 cache write rates and correct data zone priority
2026-08-26 12:21:39 -07:00
Devin AI
4456a4407f fix(model_prices): azure gpt-5.6 cache writes, mistral missing models, together cache reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 19:20:14 +00:00
mateo-berri
ece187ea24 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_anthropic_fast_mode_cache_and_response_speed
# Conflicts:
#	tests/test_litellm/test_cost_calculator.py
2026-08-26 12:10:33 -07:00
Mateo Wang
c13b278d79
Merge pull request #38369 from BerriAI/litellm_anthropic_geo_uplift_missing_models
fix(cost-map): add US data residency uplift to claude-sonnet-4-6 and mythos entries
2026-08-26 12:05:25 -07:00
Devin AI
0cc407a02d Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_bedrock_sol_anthropic_1hr 2026-08-26 19:02:56 +00:00
Mateo Wang
80843ae7cb
Merge pull request #38279 from 6matt/litellm_bedrock_converse_gpt5_reasoning_effort
fix(bedrock): map reasoning_effort to reasoning.effort for OpenAI GPT-5.x on Converse
2026-08-26 11:48:18 -07:00
Mateo Wang
2d3144c6c8
Merge pull request #38368 from BerriAI/litellm_fix_bedrock_mantle_gpt55_gpt54_context_window
fix(model_prices): raise bedrock_mantle gpt-5.5 and gpt-5.4 max_input_tokens to Mantle's enforced 1050000
2026-08-26 11:42:47 -07:00
mateo-berri
e97a84afcf fix(model_prices): add above-272k priority rates to azure us/eu gpt-5.6 entries 2026-08-26 11:40:19 -07:00
mateo-berri
8307be68c9 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_pr38100_tencent_thinking 2026-08-26 11:26:58 -07:00
mateo-berri
a0d1fef89d fix(anthropic): scale cache costs by fast mode and trust served speed 2026-08-26 11:24:12 -07:00
mateo-berri
b349b9bf50 fix(pricing): add azure gpt-5.6 cache write rates and correct data zone priority 2026-08-26 10:50:44 -07:00
mateo-berri
6416a97a4d fix(model_prices): price 1-hour cache writes on claude-3-haiku and claude-3-opus at 2x input 2026-08-26 10:48:49 -07:00
mateo-berri
a2cd2d8a4b fix(cost-map): add US data residency uplift to claude-sonnet-4-6 and mythos entries 2026-08-26 10:43:43 -07:00
mateo-berri
b97d5e77eb fix(model_prices): raise bedrock_mantle gpt-5.5 and gpt-5.4 max_input_tokens to Mantle's enforced 1050000 2026-08-26 10:42:00 -07:00
Devin AI
07c9812739 fix(model_prices): carry anthropic behavior flags on deepinfra claude entries, move retired together models to deprecated list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 17:18:51 +00:00
Devin AI
d3ede97189 fix(model_prices): keep novita gpt-oss-120b vision flag per provider catalog
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 16:57:26 +00:00