* feat(refuseless): add provider and model pricing
* feat(refuseless): expose provider in Add Model catalog
* test(refuseless): verify model input trimming after upstream fix
* fix(refuseless): exclude unavailable free model from catalogs
* feat(abliteration): add model pricing and capabilities
* feat(abliteration): expose provider in Add Model catalog
* chore(abliteration): keep provider entries independent of other additions
* fix(abliteration): correct reasoning capabilities and verify input trimming
* refactor(cost-map): drop the legacy max_tokens field in favor of max_input_tokens and max_output_tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: drop assertions that pinned the retired cost-map max_tokens field
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): price claude_platform calls from the anthropic cost map rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: avoid recursive Claude Platform lookups
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): price claude_platform calls from their own cost map rows
Replace the claude_platform -> anthropic lookup redirect with
bedrock/claude_platform/* rows, and apply the inference_geo multiplier
in the bedrock cost path so US-only inference is priced at 1.1x
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(bedrock): exempt claude_platform rows from the Converse routing guard
The claude_platform/ route is explicit in the row key, so these rows are
not accidental non-Converse Bedrock chat rows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Add OpenCode Go model discovery, protocol routing, session affinity, and published pricing metadata. Cover Chat Completions, Responses, Messages, and Gemini through the existing provider adapters.
Fixes#31568
* feat: add OpenAI 12-hour cache write pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: omit empty cache-write response fields
* style: format cache-write fields
* fix(cost): bill openai 30m and 12h cache writes from the usage split, add gpt-5.6 rates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): apply the 12h long-context write rate only above the token threshold
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost): key cache write rates by TTL instead of growing the rate tuple
_get_token_base_cost returned a positional 6-tuple and every cache lifetime added a slot to it, to the off-peak helper and to every caller. It now returns a TokenBaseCosts dataclass whose cache write rates live in one mapping keyed by TTL. The TTLs come from the ephemeral_*_input_tokens fields on CacheCreationTokenDetails, so supporting a new lifetime only means adding that usage field and its rate key
A lifetime with no rate of its own now always falls back to the cache creation rate after off-peak pricing is applied. Before this the non-tiered 12h rate and the tiered 1h rate fell back to the rate from before off-peak, so the same request billed differently depending on the pricing path
* refactor(cost): hardcode the cache write TTLs as a Literal instead of parsing usage field names
Parsing ephemeral_*_input_tokens names saved nothing, since a new TTL still needs a usage field and provider parsing code. The TTLs are now a CacheWriteTTL Literal with an explicit rate key per TTL
* refactor(cost): resolve cache write lifetime rates to floats at lookup
Decide the 1h and 12h cost map key in the threshold loop, then resolve each
rate once after off-peak with the base write rate as the default, so no None
flows through the cost code. Add an integration case for a 12h write on a
row without a 12h rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost): name every cache write TTL, marking 5m and 30m as base rate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): list every cache write TTL in one rate key table, None meaning the base write rate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): drop base rate entries from the cache write rate table and rely on fallback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): key cache write rates by plain string TTL
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): fill missing cache write TTL rates with the base write rate after off-peak
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): default missing cache write TTL rates to the base write rate inline
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): rename cache write rate key table to make clear it only covers long TTLs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cost): list every cache write TTL and bill writes of an unknown TTL at the base write rate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): resolve tiered cache write TTL rates after off-peak instead of passing None
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): carry cache write TTL rates on TokenRates and drop TokenBaseCosts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): define TokenRates before its first use to drop the string annotation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cost): build the TokenRates TTL default with default_factory and type tiers as Mapping
MappingProxyType({}) as a dataclass default is rejected on Python 3.11, where
mappingproxy is unhashable, so import litellm failed there. The tier type
moves from dict[str, object] to Mapping[str, object] (LIT001), with tier_rate
accepting a Mapping to match.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): type select_tier_for_input tiers as Mapping so tier_rate callers stay typed
The tier returned as a bare dict made every tier_rate call site a partially
unknown argument once tier_rate took Mapping[str, object]. The range bounds are
now validated with a TypeAdapter instead of indexing an unknown list.
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(gemini)!: honor supports_sampling_params on the Gemini and Vertex path and flag Gemini 3.6 Flash, 3.5 Flash-Lite and later
Google ignores temperature, top_p and top_k on Gemini 3.6 Flash, Gemini 3.5
Flash-Lite and every later Gemini release, and says a later generation will
return 400 for them. The model map now carries supports_sampling_params: false
on those rows, and the Gemini and Vertex transformation reads the flag the way
the Anthropic and Bedrock paths already do, dropping the params under
drop_params and otherwise raising the client-side 400 that names drop_params
BREAKING CHANGE: a request that sends temperature, top_p or top_k to a flagged
Gemini model without drop_params now gets a 400 instead of a 200 whose
sampling params Google silently ignored
Fixes#47015
* fix(gemini): gate the passed-through top_k where the request body is built and keep allowed_openai_params working
top_k is a provider-specific kwarg that never reaches the OpenAI param mapping,
so the flag has to be applied while the Gemini request body is built, the same
boundary Anthropic uses. The mapping now skips a flagged sampling param instead
of raising, because the shared unsupported-param check already returns the 400
that names drop_params, and a param the caller lists in allowed_openai_params
is forwarded verbatim as that 400 promises
* fix(gemini): filter top_k out of the generation config instead of mutating optional_params
* test(gemini): type the sampling-flag twin test and keep its locals out of the loop body
* fix(vertex_ai): reject or drop sampling params on batch rows for flagged Gemini models
The batch JSONL wrapper calls map_openai_params directly and skips get_optional_params,
so a flagged model's temperature, top_p and top_k vanished from the uploaded row with no
400. Batch rows now get the same contract as a real-time call: a 400 naming drop_params,
or the knobs dropped when litellm.drop_params is on
* fix(vertex_ai): ignore null sampling knobs on batch rows for flagged Gemini models
A batch row carrying temperature: null, top_p: null or top_k: null means no
override, the same as on a real-time call, so the row is no longer refused
with a 400 and the null knobs are simply not forwarded
* feat(extraction): add Clairic extraction types and Cloudflare provider config
* feat(extraction): add the /v1/extraction route, router wiring, and Clairic billing
* fix(extraction): refuse or drop stream, pass unknown keys, run guardrails, keep the requested model name
Clairic does not stream, so stream=true answers 400 and is dropped and billed under drop_params. Keys LiteLLM does not know pass through to Cloudflare. Guardrails scan state, field instructions and text answers. Frozen decision and extraction responses now carry the requested model name. List fields must name their item type. /health/test_connection and the Add Model page accept the extraction mode.
* fix(extraction): keep upstream model names on decisions, scan images, keep typed values, refuse OpenAI params
Scope the frozen-response model stamp to extraction so /v1/decisions and /v1/systemone return the upstream name as on main. Pass request images to guardrails as data URLs and drop answer spans when a response guardrail rewrites state_text. Field spec values keep their JSON type, so a wrong type answers 400 instead of being coerced. OpenAI generation params answer 400 unless drop_params. The Model Hub shows a /v1/extraction snippet for extraction models.
* fix(extraction): send only Clairic's request fields, count extraction tokens, keep typed answers typed, drop spans a request guardrail made stale, and honor --max_tokens
* fix(extraction): scan number answers in response guardrails, rewrite guardrail fields in linear time, and leave --max_tokens as on main
* fix(extraction): scan float answers as whole numbers so fraction digits never trip a response guardrail
* fix(extraction): round only mean and median for response guardrails so fractional values keep their decimal text
* fix(extraction): count state and fields for budget reservations and scan estimate quantiles in response guardrails
* fix(extraction): send each repeated text to guardrails once and keep the context-window budget reservation
* test(extraction): type request_data in the guardrail test helpers
* feat(proxy): opt-in free-tier allowance for Google Search grounding spend
* fix(proxy): declare the search free tier counter's service target
* test(cost_map): allow search_free_requests in the cost map schema
* fix(proxy): skip cache hits and never free more than the billed cost in the search free tier
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost-map): add volcengine doubao-seed-2-1-pro-260915 and doubao-seed-2-1-lite-260915
Pro row absorbed from #47372, lite row added from the same Volcengine Ark pricing and model list docs
Co-authored-by: donghongfei <1247543758@qq.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost-map): cover volcengine doubao-seed-2-1-pro-260915 row
Co-authored-by: donghongfei <1247543758@qq.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost-map): add undated azure/eu and azure/us rows for o1, o3, o3-mini, o4-mini, gpt-4o, gpt-4o-mini
Absorbed from #47499
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: donghongfei <1247543758@qq.com>
* feat(dashscope): price Qwen implicit context-cache reads separately
* fix(dashscope): normalize implicit cache price strings
* fix(dashscope): price explicit cache hits through an explicit_ override key
Keep cache_read_input_token_cost as the default (implicit) Qwen cache-read
rate, which the cost map already holds, and add
explicit_cache_read_input_token_cost for hits DashScope tags with
cache_type "ephemeral". Untyped and unknown cache types keep the base rate,
so older releases reading the remote cost map keep billing unchanged.
Add explicit cache-read and cache-creation rates to the qwen3.7-max rows.
* fix(types): accept explicit_cache_read_input_token_cost as a constructor keyword
* fix(dashscope): inherit the explicit cache price on custom deployments and keep tier rates all-or-nothing
Deployments that set only input and output prices now inherit
explicit_cache_read_input_token_cost from the cost map, so explicit hits bill
at the explicit rate. A pricing tier without its own explicit rate bills its
own cache_read rate instead of the model-level explicit one, matching how an
off-peak block falls back.
The DashScope calculator now delegates the explicit rate to the shared
resolver, and the register_model warning is back to main's wording.
* fix(dashscope): keep a deployment's own cache-read price for explicit hits and ignore a non-string cache_type
* fix(dashscope): keep cache_type through Responses usage and let a registered cache-read price cover explicit hits
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(search): add DeepSeek as a /v1/search provider
Adds DeepSeekSearchConfig for POST https://api.deepseek.com/v1/tools/search, registers it under search_provider=deepseek, prices deepseek/search at 0.00075 per query and marks search support in provider_endpoints_support.json
* fix(search): reject DeepSeek queries over the 128 character limit before the call
* test(search): escape regex metacharacters in DeepSeek query limit match
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(openai): drop unsupported reasoning effort for gpt-6.1-sol
gpt-6.1-sol accepts low, medium, high, xhigh, and max. none and minimal were still forwarded, because chat completions ignored an explicit supports_none_reasoning_effort=false and the Responses mapper never applied that contract.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(openai): mark the stripped reasoning object as outgoing JSON
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Azure responses reading the OpenAI effort map
gpt-6-astra on Azure accepts reasoning.effort none. The responses gate was checking the OpenAI entry, which rejects it.
Signed-off-by: Paul El Khoury <82139421+pOpO1-9@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(openai): gate every reasoning_effort level from the cost map on chat and Responses
One gate reads the model's cost-map row for whatever effort level a request
asks for, on Chat Completions and the Responses API, for OpenAI and Azure.
xhigh stays opt-in on chat (Azure also keeps none opt-in); every other level
is refused only when its supports_<level>_reasoning_effort flag is false. A
refused level is dropped under drop_params or answered with a local 400, and
Responses keeps the rest of the reasoning object when it drops effort. The
guards that follow judge temperature by the level the provider will see.
Tests move into the mapped test files and derive the expected outcome for
each level from the map row, so a new GPT-6 model needs only its row.
* test(openai): pin gpt-6.1-sol's refused effort levels and prove the row flags alone drive the gate
* fix(cost-map): azure gpt-6-astra refuses none and honors max on Responses, as live calls show
* test(reasoning_effort): pin the astra max and none flips through the adapter and Foundry paths
* fix(reasoning_effort): keep max out of the route gates and correct the gpt-5.3-codex rows
The same model honors max on Responses and refuses it on Chat Completions, so one row flag
cannot gate it on either route. The gates now skip max, the Azure gpt-6-astra rows keep max
off so the /v1/messages adapter degrades to xhigh and /model_group/info stops short of it as
on main, and chat still forwards max for the Responses bridge.
The gpt-5.3-codex rows said none and xhigh off and minimal on; the live model answers the
other way round (2026-10-01), and an opt-out gate would have dropped the levels Codex CLI sends.
* fix(reasoning_effort): bridge function tools to Responses when the chat gate drops none
* docs(reasoning_effort): correct the max advertisement note
* fix(azure_ai): read the Foundry row on the Responses effort gate and turn none off for Azure gpt-6.1-sol
* fix(azure): gate a prefixed Foundry deployment's Responses effort off its row
litellm.responses(model="azure_ai/<deployment>", api_base=<Foundry host>) is
re-routed to the azure provider with the azure_ai/ prefix intact, so the Azure
Responses config saw a prefixed name, skipped it as another provider's
pass-through, and forwarded none and minimal to Azure even under drop_params.
The Azure gpt-5 resolver also read no row at all for a prefixed name.
The Azure Responses gate now detects the family the way the Azure chat gate
does, and an azure_ai/ name on the Azure OpenAI configs reads its azure_ai/ row
when the map has one, else the azure/ twin, the rule the Foundry chat config
already applies to the same deployment.
* fix(cost-map): turn none off on azure/eu/gpt-6-astra like its sibling rows
* docs(reasoning_effort): say how each gate reads none and shorten the module docstring
* fix(azure_ai): call the public effort-level check from the Foundry Responses gate
* test(reasoning_effort): fold the Foundry Responses tests into the mapped file and type the row helpers
---------
Signed-off-by: Paul El Khoury <82139421+pOpO1-9@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
