Commit graph

2036 commits

Author SHA1 Message Date
mateo-berri
5825cc7593 chore: merge litellm_internal_staging into litellm_mistral_voxtral_tts_speech
Some checks failed
LiteLLM Rust / rustfmt, clippy, test (push) Has been cancelled
LiteLLM Rust / release wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
2026-09-06 03:24:55 -07:00
Mateo Wang
02522a5441
Merge pull request #39983 from BerriAI/litellm_lit_7081_azure_ai_gpt_6_astra_pricing
feat(cost-map): add azure_ai/gpt-6-astra Foundry pricing
2026-09-06 01:27:22 -07:00
mateo-berri
3dea1ebb32 fix(cost-map): keep the prompt cache breakpoint flag on the foundry gpt-6-astra row
The openai gpt-6-astra card carries supports_prompt_cache_breakpoint, so a Foundry deployment
reported it as true until the azure_ai row took over the lookup. The cache control hook still
honours breakpoints for that deployment through the bare name, so /model/info was the only thing
that changed, and it now agrees with the hook again.
2026-09-05 23:18:25 -07:00
Mateo Wang
0318b4acdc
Merge pull request #30856 from emerzon/litellm_vertex_lyria_models
feat(vertex): add Lyria model support
2026-09-05 23:12:25 -07:00
mateo-berri
11e45ad953 fix(vertex_ai): mark the Lyria 3 catalog entries text-only
`vertex_ai/lyria-3-clip-preview` and `vertex_ai/lyria-3-pro-preview` were
registered with `supports_vision`, `supports_image_input`, and an `image`
modality, which contradicts their `gemini/lyria-3-*` siblings and makes
/model/info advertise image input on text-to-music models.
2026-09-05 23:00:01 -07:00
mateo-berri
6be78fa850 fix(vertex_ai): bill Lyria per generation, not per audio second
Google prices Lyria per generated clip, so every Vertex Lyria entry in the
price map now carries a single output_cost_per_image and both the speech
and the passthrough cost paths read that one field. The old
output_cost_per_second and audio_seconds_per_prediction pair assumed a
30 second clip, which does not match the 32.768 second WAV Vertex returns,
and no other model in the map priced audio that way

Drops max_audio_length_hours and max_audio_per_prompt from the price map,
its schema, the generator, and ModelInfo, since nothing reads them, and
drops the audio_mime_type hidden param for the same reason: the response
already carries the resolved content type on its own header

Folds the per-model bundled catalog lookups into one cached parse of the
local cost map, validated with a TypeAdapter over a ReadOnly TypedDict
2026-09-05 22:34:31 -07:00
mateo-berri
e79f3ec520 fix(cost-map): stop advertising reasoning_effort max on the azure gpt-6-astra rows
Both Azure routes refuse it. A live call to the same deployment through
openai/deployments/gpt-6-astra/chat/completions on api-version 2025-04-01-preview
answers reasoning_effort max with a 400 unsupported_value naming none, low, medium,
high and xhigh as the values it takes, and xhigh returns 200, so azure/gpt-6-astra
and azure/us/gpt-6-astra now match the azure_ai row.
2026-09-05 22:31:32 -07:00
mateo-berri
e8f311429e fix(cost-map): stop advertising reasoning_effort max on azure_ai/gpt-6-astra
Foundry rejects reasoning_effort max on the gpt-6-astra deployment with a 400 that
names none, low, medium, high, and xhigh as the supported values, so the card no
longer lists max. The request path never gated max (only xhigh is opt-in), so this
only changes /model_group/info and router capability gating. The azure/ twin stays
as is because it was not verified on an Azure OpenAI host
2026-09-05 19:42:15 -07:00
mateo-berri
15372967c6 fix(azure_ai): read the azure_ai card for gpt-5 series reasoning effort gates
Foundry deployments of gpt-6-astra reached through azure_ai used the bare OpenAI card
for the reasoning_effort none gates, so temperature and top_p were refused while the
azure_ai card says none is supported. AzureAIStudioConfig now dispatches gpt-5 series
params through AzureAIGPT5Config, which looks capabilities up under the azure_ai/
prefix the way the azure route does

Also carries the search_context_cost_per_query block azure/gpt-6-astra has, adds a
flex service tier cost test that fails at the merge base, and keeps the wildcard test
from stripping azure_ai/gpt-6-astra out of the provider set
2026-09-05 19:06:38 -07:00
Mateo Wang
56a61cf016
Merge pull request #39764 from BerriAI/litellm_govcloud_profiles_lit6421
feat(pricing): add GovCloud pricing for every live but unpriced Bedrock model
2026-09-05 17:15:22 -07:00
mateo-berri
635bb3a209 feat(cost-map): add azure_ai/gpt-6-astra Foundry pricing
A gpt-6-astra deployment on a Foundry project reached through the
azure_ai route had no cost map entry of its own, so it resolved to the
OpenAI gpt-6-astra card: missing from the azure_ai/* wildcard listing,
flex and priority prices and /v1/batch it does not sell, and no none
reasoning effort. Add azure_ai/gpt-6-astra mirroring the
azure/gpt-6-astra Standard Global sheet the way azure_ai/gpt-5.5 mirrors
azure/gpt-5.5, and extend the cost, reasoning-effort, and wildcard
listing tests to the Foundry route.
2026-09-05 17:08:42 -07:00
mateo-berri
8426235290 feat(ocr): add Cohere Parse support for cohere and azure_ai 2026-09-04 21:25:46 -07:00
mateo-berri
036d104533 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_govcloud_profiles_lit6421 2026-09-04 20:08:16 -07:00
Mateo Wang
4af62a38c1
Merge branch 'litellm_internal_staging' into litellm_mistral_voxtral_tts_speech 2026-09-04 20:07:45 -07:00
mateo-berri
51514b9123 fix(cost-map): azure/gpt-6-astra accepts reasoning_effort none on Foundry 2026-09-04 17:29:55 -07:00
mateo-berri
3202963f25 feat(cost-map): add azure/gpt-6-astra and azure/us/gpt-6-astra Foundry pricing 2026-09-04 16:57:40 -07:00
mateo-berri
6d01ed803d Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_mistral_voxtral_tts_speech 2026-09-04 16:45:33 -07:00
mateo
50d6b26a86 fix(registry): mark baseten GLM-5.3 as vision-capable per Baseten vision docs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-04 20:10:58 +00:00
mateo
1f0611a8b9 fix(registry): drop Together MiniMax M2.7 and revert Qwen2.5 7B Turbo pricing, both non-serverless
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-04 19:25:10 +00:00
mateo
7b96a11e5f feat(registry): add OpenRouter catalog gaps, Fireworks DeepSeek V4 Flash Vision, Together MiniMax M2.7 and Qwen2.5 7B Turbo pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-04 19:19:08 +00:00
mateo-berri
bef3585d82 feat(pricing): add GovCloud rows for every live but unpriced Bedrock model
Every model bedrock list-foundation-models and list-inference-profiles
report as live in us-gov-west-1 or us-gov-east-1 now has a priced row:
Claude Fable 5.1 (profile plus in-region), Nemotron Nano 9B (profile plus
in-region), Grok 4.6 (profile plus Mantle in both regions), the us-gov.
Claude 3 Haiku profile in the east, Nova Lite, Micro and the Nova 2
multimodal embeddings in the west, and the Gemma 4 and gpt-oss Mantle
SKUs the GovCloud offer files price. Offer-file rates are used where AWS
publishes them; Claude rows carry the 1.2x GovCloud premium.
2026-09-04 10:03:43 -07:00
mateo
0c29f510bc fix(registry): drop gpt-image-2 text output price, add openrouter minimax-m3 and qwen3.7-plus
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-04 15:03:29 +00:00
mateo
c707f2fe5d fix(model_prices): databricks gpt-5-3-codex is served via the Responses API
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-04 14:32:11 +00:00
mateo
8e83d6d63d fix(model_prices): add Databricks Sep-2026 catalog, Azure gpt-realtime-2.x, per-token realtime image pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-04 14:06:54 +00:00
mateo
08bfdadb10 chore: merge litellm_internal_staging into litellm_registry_audit_2026_09_02, drop the deleted ocr ledger
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-04 04:18:09 +00:00
mateo-berri
43f31b0a4b feat(pricing): add GovCloud Claude Opus 5 and us-gov. inference profile rows 2026-09-03 18:40:59 -07:00
mateo
f292667601 fix(registry): mark gpt-daybreak-*-latest as responses mode to match their Responses-only endpoints
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 21:41:12 +00:00
Mateo Wang
29ea2bd2cd
Merge pull request #39426 from BerriAI/litellm_azure_ai_grok_4_6_cost_map
feat(azure_ai): add grok-4.6 to the model cost map
2026-09-03 14:36:07 -07:00
mateo
00bdfe797a fix(registry): mark gemini-3.5-live-translate-preview as realtime with official token limits
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 21:16:55 +00:00
mateo
5a3a2f3d0a Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_2026_09_02 2026-09-03 21:09:30 +00:00
mateo
eae7b806e3 fix(registry): point Bedrock Qwen3 Coder 480B source at the us-west-2 on-demand price list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 20:49:44 +00:00
mateo-berri
bba75c7ce9 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_mistral_voxtral_tts_speech
# Conflicts:
#	tests/test_litellm/test_cost_calculator.py
#	tests/test_litellm/test_main.py
2026-09-03 13:35:30 -07:00
mateo
080e364d5e fix(registry): carry Anthropic thinking/sampling flags on new Perplexity and OpenRouter Claude entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 20:28:48 +00:00
Mateo Wang
828d561fcc
Merge pull request #39622 from BerriAI/litellm_gpt_6_astra
feat(models): add gpt-6-astra pricing and metadata
2026-09-03 13:18:34 -07:00
mateo
2c4eb693ed fix(model_prices): absorb Baseten GLM-5.3 and OpenRouter live prices, fix Bedrock Qwen3 Coder 480B input price and Gemini Live image price
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 19:53:58 +00:00
mateo-berri
4991d0bf3e fix(models): match gpt-6-astra reasoning effort levels to OpenAI docs
OpenAI documents low, medium, high, xhigh, and max for gpt-6-astra, with no none level, so the entry stops advertising none and starts advertising max.
2026-09-03 12:47:25 -07:00
mateo-berri
897fba08c8 feat(models): add gpt-6-astra pricing and metadata
Adds the OpenAI gpt-6-astra entry to both price files with standard, flex, priority (fast mode), batch, and above-272K long-context rates, and regression tests covering each tier and the batch rates.
2026-09-03 12:47:25 -07:00
mateo
32a3a65312 fix(model_prices): add Lyria 3.5, Perplexity Agent API and OpenRouter first-party models, fix Nebius, Mistral, OpenRouter metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 19:35:30 +00:00
mateo-berri
51d821ae45 fix(cost): bill bedrock_mantle web search at $12 per 1k queries using Bedrock's reported count 2026-09-03 12:21:54 -07:00
mateo
2c63095e83 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_2026_09_02 2026-09-03 19:02:19 +00:00
mateo-berri
9f806be6e8 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_mistral_voxtral_tts_speech
# Conflicts:
#	tests/test_litellm/test_router.py
2026-09-03 11:00:31 -07:00
mateo-berri
a1e58aabe7 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_mantle_web_search 2026-09-03 09:50:42 -07:00
mateo
840173e778 feat(registry): add azure_ai/mistral-ocr-4-0 page and annotation prices from Azure Retail Prices
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 13:48:25 +00:00
mateo
f26407aa8c feat(registry): add azure_ai/MAI-Thinking-1 from Azure Retail Prices and Foundry docs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 13:43:48 +00:00
mateo
55a5f142e6 fix(model_prices): add azure_ai Codestral-2501 and FW-Nemotron-Lightning-3.5, sync Azure and Vertex deprecation dates, fix novita gpt-oss vision flags
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 13:23:02 +00:00
mateo
1a39275cb3 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_2026_09_02 2026-09-03 13:05:42 +00:00
mateo-berri
75e7f4c4a5 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_mistral_voxtral_tts_speech
Resolves the tests/test_litellm/test_main.py collision, where both sides appended a
new test at the end of the file, by keeping both.

Also carries the one-line fix from #39502: staging arrived with a duplicate
embedding_executor kwarg in the Bedrock KB fake handler, which ruff rejects as a
syntax error, so every commit here would otherwise fail lint. The change is byte
identical to #39502, so that PR merges cleanly once it lands.
2026-09-03 00:07:31 -07:00
Emerson Gomes
f18cb0cdb4
fix(vertex): make Lyria routing and billing data-driven 2026-09-02 19:22:42 -05:00
Emerson Gomes
b96844dd0c
feat(vertex): expose Lyria through audio speech 2026-09-02 19:22:15 -05:00
Emerson Gomes
514e9a1ee6
fix(vertex): address lyria review feedback 2026-09-02 19:19:53 -05:00