Commit graph

9 commits

Author SHA1 Message Date
tin-berri
30ff3723b2
feat(model_prices): let a map entry declare its exact reasoning_effort levels (#38481)
Kimi K3 accepts exactly low, high and max, defaults to max, and always thinks.
The map could not say that: medium and high have no supports_*_reasoning_effort
flag because every other reasoning model takes them, so the ten kimi-k3 entries
carried supports_reasoning alone and resolved to unknown. The dashboard then fell
back to a capability-blind level list that deliberately omits max, which is why a
kimi-k3 tier cannot be set to max thinking today.

Add reasoning_effort_levels, an array key in the shape the map already uses for
supported_endpoints and supported_modalities. Where present it is read first and
wins whole; every other entry keeps answering through the per-level flags,
unchanged. It is deliberately a different name from the computed
ModelGroupInfo.supported_reasoning_efforts, which stays derived from a group's
deployments and is never seeded from one deployment's model_info.

The levels are per entry rather than per model, because the deployments differ:
Moonshot, Together, Fireworks and Azure Foundry all forward the level unchanged
and get the model's own low/high/max, while Perplexity documents a six-value
enum it maps down internally and gets that. The /v1/messages degradation chain
consults the same declaration, so the level the map advertises is the level that
path forwards.
2026-08-27 15:38:01 -07:00
mateo-berri
b9a790899a fix(gemini): bill Google Maps grounding as its own SKU
Gemini API Maps-grounded prompts were billed as web search and Vertex AI Maps-grounded prompts were not billed at all. Classify grounding metadata per candidate into web search vs Maps requests, carry a distinct google_maps_grounding_requests usage counter through non-streaming and streaming paths, and price it via the new google_maps_grounding_cost_per_query cost map key with per-query and per-prompt defaults keyed off web_search_billing_unit. Fixes #35906
2026-08-26 15:31:27 -07:00
tin-berri
4307b34aca
fix: omit thinking.type=disabled for always-on thinking Claude models (#37510) 2026-08-21 10:27:26 -07:00
mateo-berri
b39a339b7d fix(vertex_ai): apply regional endpoint uplift to cost tracking 2026-08-19 15:21:06 -07:00
mateo-berri
be594f5984 feat(guardrails): count bedrock guardrail cost against spend and budgets
Price ApplyGuardrail usage units recorded by PR #37225 with a new
bedrock/guardrails entry in the model cost map (regional override via
bedrock/{region}/guardrails), add the per-request guardrail_cost to the
standard logging payload's response_cost and CostBreakdown, surface it in
the x-litellm-response-cost header, and bill blocked requests through the
failure hook so key and team budgets see what AWS bills
2026-08-18 14:16:07 -07:00
mateo
6517c1dc06 fix(cost): support cache_creation_input_token_cost in tiered pricing and make tier selection all-or-nothing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-13 02:18:10 +00:00
mateo-berri
00e8691064 ci: enforce format assertions so calendar-impossible deprecation dates fail validation 2026-07-28 16:11:22 -07:00
mateo-berri
3d01c39d00 ci: tighten deprecation_date pattern to reject impossible months and days 2026-07-28 15:24:24 -07:00
mateo-berri
4556dfa930 ci: publish a generated JSON schema for model_prices_and_context_window.json 2026-07-27 12:11:17 -07:00