* feat(model_prices): add NVIDIA Nemotron 3.5 Lightning on OpenRouter and DeepInfra
Nemotron 3.5 Lightning shipped 2026-08-11 with public per-token pricing on
OpenRouter and DeepInfra at $0.05/M in and $0.20/M out. Without cost map
entries both ids raise "This model isn't mapped yet" and log at zero spend.
* fix(model_prices): stop asserting an output cap for Nemotron 3.5 Lightning
262144 is the native context window, not the output budget, and neither
OpenRouter nor DeepInfra publishes an output cap. Keeps max_input_tokens at the
native 256K window: 1M needs VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 on a self-hosted
deployment, so it is not what these hosted endpoints serve.
* chore(tests): drop the Nemotron 3.5 Lightning metadata test
Requested on the review thread: the cost map entries stand on their own.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
`supports_native_structured_output` was set only on the bare `deepseek.v3.2`
and `zai.glm-5` entries, so the cross-region inference profiles and the
region-pinned ids resolved to None. The flag gates the native
`outputConfig.textFormat` branch in BedrockConverseConfig, so callers
addressing the same model as `us.deepseek.v3.2` or
`bedrock/us-west-2/deepseek.v3.2` silently fell back to synthetic tool
injection. `us.` is the form Bedrock steers callers toward, so the most
common way to reach these models was the one missing the capability.
Adds the flag to the 12 affected ids and keeps the packaged backup in sync.
test_get_model_info_bedrock_models already caught the region-pinned ids, but
it filters on `litellm_provider == "bedrock"` and the cross-region profiles
carry `bedrock_converse`, so reverting just `us.deepseek.v3.2` and
`eu.deepseek.v3.2` left it green. The new parity test covers the prefixed
profiles and fails on exactly that mutation.
Record supports_tool_search on the Bedrock Claude entries in both cost
map files and have _supports_tool_search_on_bedrock read it first via
the provider-resolved capability lookup, keeping the name patterns as a
fallback for ARNs and ids the map cannot resolve. Threads the flag
through ModelInfoBase and drops a dated remark from the pattern list
Adds 'agentcore' as a search provider backed by an AgentCore Gateway MCP web-search target, usable from litellm.search()/`/search` and as a websearch_interception backend. Supports SigV4 (AWS_IAM gateways) and bearer tokens (CUSTOM_JWT gateways) via a new BaseSearchConfig.sign_request hook.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replaces the five launch models with the two that SCX.ai now leads on.
Both are live on api.scx.ai and both were verified against it for tool
calling, json_object and json_schema output, reasoning, prompt caching,
and, for Qwen3.8 Max, image input
Pricing follows SCX's published USD rates. GLM-5.2 lands at $0.55/M input
and $1.9255/M output, tracking the recent GLM-5.2 market repricing;
Qwen3.8 Max at $1.815/M and $5.4461/M sits under the only other seller of
that model, and is the first Qwen3.8 Max entry in the catalog
Also corrects a metadata bug the removed entries carried: they set
max_tokens equal to max_input_tokens, conflating the context window with
the output cap. Both new entries declare a max_output_tokens of 131072,
which is what the endpoint's own validator enforces
The Add Model placeholder moves to scx-ai/GLM-5.2 now that MiniMax-M2.7
is no longer in the catalog
Dated snapshots like o4-mini-2025-04-16 were missing the flex and priority cost keys their base alias carries, so service-tier requests against pinned snapshots were billed at standard rates. Sync the tier keys wherever the snapshot's anchor prices match the base alias, and add a drift regression test.
* fix(bedrock): stop forwarding no-op toolSpec.strict to Converse
`strict: false` is the Chat Completions default, so sending it to Bedrock
Converse communicates nothing the provider does not already assume, while
Bedrock rejects the key by presence rather than by value: any Claude model
routed through its Anthropic-compatible validator 400s with
`tools.0.custom.strict: Extra inputs are not permitted`.
The existing `bedrock_converse_supports_strict_tools` gate only protects
models whose `model_prices_and_context_window.json` entry carries the flag,
which makes every newly released Claude model broken by default until someone
adds it. That is a losing race for a field that carries no information when
false, and it is unrecoverable from the client side on `/v1/responses`, where
the Responses to Chat Completions bridge stamps `strict: false` onto every
function tool even when the caller never sent one. `drop_params` cannot help
there because the caller never supplied the param.
Drop the key when falsy instead. `strict: true` still honors the per-model
gate, so models that accept strict schemas keep the behavior they have today
and the flag keeps doing its job for the values that actually mean something.
* fix(bedrock): flag Claude Sonnet 5 as rejecting toolSpec.strict
Bedrock routes Sonnet 5 through the Anthropic-compatible validator that
rejects `toolSpec.strict`, but its six pricing-map entries never got
`bedrock_converse_supports_strict_tools: false`, so the gate fell back to
forwarding for Anthropic models and every tool call carrying `strict: true`
400'd. Verified live in us-east-1: before this, `strict: true` against
`us.anthropic.claude-sonnet-5` returns
`tools.0.custom.strict: Extra inputs are not permitted`; after, it returns a
real tool call.
Measured the rest of the family the same way rather than trusting the map:
Sonnet 4.5, Sonnet 4.6 and Haiku 4.5 all accept `strict: true`, and Opus 4.8
already carries the flag. Sonnet 5 was the only entry where the map disagreed
with the provider, so it is the only one changed here.
Same shape as the Opus 4.7/4.8 and Sonnet 4 fixes before it.
OpenAI cut Terra 20% and Luna 80% on 2026-07-30; openai and bedrock_mantle
entries already match. Azure global and us/eu data-zone terra/luna rows still
used the pre-cut rates, so spend tracking over-billed those Azure deployments.
Sol is unchanged. Cache-read, priority, and long-context fields scale with the
same multipliers already used for azure gpt-5.6.
Adds search_context_cost_per_query pricing for the 82 OpenAI/Azure models that advertise supports_web_search but had none (gpt-5 family, o-series, deep-research at $0.01/call; gpt-4.1 at $0.025/call), so built-in web search is no longer billed as $0. Also counts web_search_call items in Responses output so N searches bill N times instead of once; usage-count providers (gemini, anthropic, xai, vertex) still route through get_cost_for_web_search_request and are unaffected.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
OpenAI publishes a long-context column on the Flex tier, at half the standard
long-context rate. We had no field for it, so a >272k flex request fell through
to the standard long-context price and billed 2x: Terra $4/$18 instead of
$2/$9, Luna $0.40/$1.80 instead of $0.20/$0.90, Sol $10/$45 instead of $5/$22.50.
Adding the values to the cost map alone does nothing, because get_model_info
builds ModelInfoBase from an explicit kwargs list and silently drops any key
not named there. Declare the four *_above_272k_tokens_flex fields and wire them
through, then add the values for sol, terra, luna, and the gpt-5.6 alias.
That same gap was already swallowing cache_creation_input_token_cost_flex,
_priority, and _above_272k_tokens, which were present in the cost map but never
reached the calculator; they are wired through here too.
Fast mode (ex-Priority) publishes no long-context column, so nothing is added
there rather than deriving a rate by analogy.
AWS rolled out the 2026-07-30 GPT-5.6 price cut the same day, but the
bedrock_mantle entries still carried values derived from the pre-cut OpenAI
base, so Terra billed 1.25x and Luna 5x over the published rate.
Re-derive both from the AWS Bedrock pricing page, which prices in-region
inference at parity with OpenAI's data residency tier (1.1x base). Sol was
not cut and is unchanged.
Also drop tests/test_litellm/test_gpt_5_6_model_metadata.py; its Azure and
openai pricing assertions are covered by test_llm_cost_calc_utils.py.
Bedrock routes Claude Sonnet 5 through the same Anthropic-compatible
validator as Opus 4.7/4.8 and Sonnet 4, which rejects toolSpec.strict
with 'tools.0.custom.strict: Extra inputs are not permitted'. Set
bedrock_converse_supports_strict_tools: false on all six Sonnet 5
entries so the existing gate strips the field, matching the fix shape
of #31582
Co-authored-by: Yaroslav Budyanskiy <y_budyanskiy@wargaming.net>
