mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-25 05:54:53 +00:00
2c5c9b8d84
* fix(bedrock): use native structured output on every Claude model whose Bedrock profiles accept it Bedrock accepts outputConfig structured output on every reachable profile of Claude Opus 4.7, 4.8, 5, 5.5, Sonnet 5, 5.5, Haiku 5.5, and Fable 5.1, so json_schema now goes out natively there instead of as a forced tool, which kept Bedrock from returning adaptive thinking. Fable 5 rejects outputConfig, so its keys go back to the tool path. GovCloud keys keep their flags, and the drift tests let GovCloud lag its commercial base but never lead it. * fix(bedrock): clean Claude schemas for native structured output and keep recursive schemas on the old path Bedrock native output rejects minimum/maximum, minItems, uniqueItems, oneOf, additionalProperties true, and recursive or unresolved $ref, all of which the forced tool accepted. Claude schemas on the Bedrock native paths now go through the cleaner the Anthropic native path already uses, and a schema with a recursive or unresolved $ref keeps the forced tool, inline, or drop path it had. * fix(bedrock): keep older installs on the json tool with a Bedrock-only native output key Older installs fetch the model map from main, so flipping supports_native_structured_output moved them onto native output with their old schema handling. The new supports_bedrock_native_structured_output key is ignored by older code. Schemas native output cannot hold (recursive refs, open objects such as dict fields) keep the json tool on the new key, models on the old key keep their current behavior for those shapes, and the Anthropic API, Vertex AI, and Azure keep their current behavior. * chore(cost-map): regenerate the model prices schema for the Bedrock native output key * fix(bedrock): keep native output to profiles every account accepts and route schemas Bedrock rejects to the json tool Only Opus 5.5, Sonnet 5.5, Haiku 5.5 and Fable 5.1 keep the Bedrock native output key, since Opus 4.8 and Sonnet 5 answer 400 on some accounts and Opus 5 garbles escaped characters. Lookaround patterns and string formats outside Bedrock's list now keep the json tool, objects typed as a union with null are closed, and the cleaner keeps minItems of 0 or 1. * fix(bedrock): keep Opus 5.5 and Fable 5.1 on the json tool, since native output stops their adaptive thinking * fix(bedrock): keep shared $defs in native schemas and send refused schemas to the json tool on every native model Native output now keeps $defs instead of inlining each use, so a submodel reused across a schema counts once against Bedrock's optional and union limits. Schemas native output cannot hold use the json tool on Converse and Invoke chat for both native keys, and minItems is only kept when it is an integer 0 or 1 * fix(bedrock): keep the json tool unless thinking is requested and bound native output schemas The Bedrock native output key now applies only when the call asks for thinking, so calls without thinking keep the json tool as before. Schemas past Bedrock's documented union and optional counts, or past a grammar size estimate fitted to live probes, keep the json tool on both map keys. Open objects keep the json tool on the Anthropic API, Vertex AI and Azure AI, where native output returned them empty. The $ref check no longer expands $defs, so doubling definitions no longer stall the request. * fix(bedrock): resolve root-path $defs refs so nested Pydantic models keep native output on Invoke * test(bedrock): cover native structured output and json tool routing through the proxy Integration cells drive the proxy against a scripted Bedrock peer that enforces Bedrock's structured output schema rules, covering thinking with native output on Converse, Converse streaming, Invoke, /v1/responses and /v1/messages, the deployment opt-in, Fable 5, schema cleaning on every native entry, open objects, and malformed schemas. Vertex, Anthropic and Azure AI cells cover open objects on native-capable models and the cleaner's minItems and nullable object handling. The unit table now covers reasoning_effort none and disabled thinking * fix(bedrock): keep native output for numeric formats and wide old-key schemas, and keep thinking in json mode Check string formats only on nodes that can be strings, apply the 64-field cap only to the Bedrock-only key, carry thinking blocks into the json mode message, and send malformed json tool schemas as object schemas. * fix(anthropic): keep native output for dict fields unless the json tool can be forced * fix(anthropic): keep native output for dict fields when the caller sends tools * test(bedrock): close the dangerous-tool-use test schema, since Bedrock native output requires closed objects * test(anthropic): cover forced and unforced open-object routing, json mode thinking, numeric formats and wide schemas Add proxy cells for open-object schemas that keep native output when the json tool cannot be forced (a model that refuses forced tools on the Anthropic API and Vertex AI, thinking, caller tools), for the thinking block a json mode answer keeps on Vertex AI and Bedrock Invoke, and for integer and number formats and 65-field schemas on each Bedrock map key. Point the forced open-object cells at the Anthropic API and Azure AI only, since no Vertex AI model in the cost map takes both native output and a forced tool * fix(bedrock): keep the native output opt-in on the deployment that sets it, and route effort none open objects to native output * test(bedrock): cover deployment-scoped native output opt-in, register_model and effort none open objects
1243 lines
41 KiB
JSON
1243 lines
41 KiB
JSON
{
|
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
"title": "LiteLLM model_prices_and_context_window.json",
|
|
"description": "Schema for LiteLLM's model price and context window registry (https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json). Every top-level key except 'sample_spec' and 'fallback_generalizations' is a model id, optionally prefixed with its provider (e.g. 'azure/gpt-5.4'), mapping to a model entry. All costs are USD per unit. New optional fields are added regularly, so consumers should ignore unknown fields rather than reject them.",
|
|
"type": "object",
|
|
"properties": {
|
|
"sample_spec": {
|
|
"type": "object",
|
|
"description": "Documentation placeholder illustrating the entry shape; not a real model and not schema-conformant (several values are prose)."
|
|
},
|
|
"fallback_generalizations": {
|
|
"type": "object",
|
|
"description": "Regex rules that generalize unknown model ids to known families; not a model entry.",
|
|
"properties": {
|
|
"rules": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "object",
|
|
"properties": {
|
|
"name": {
|
|
"type": "string"
|
|
},
|
|
"pattern": {
|
|
"type": "string"
|
|
},
|
|
"description": {
|
|
"type": "string"
|
|
}
|
|
},
|
|
"required": [
|
|
"name",
|
|
"pattern"
|
|
],
|
|
"additionalProperties": true
|
|
}
|
|
}
|
|
},
|
|
"additionalProperties": false
|
|
}
|
|
},
|
|
"additionalProperties": {
|
|
"$ref": "#/$defs/modelEntry"
|
|
},
|
|
"$defs": {
|
|
"modelEntry": {
|
|
"type": "object",
|
|
"description": "Pricing, limits, and capability flags for one model. Fields other than litellm_provider are optional; boolean capability flags are simply omitted when unknown or false.",
|
|
"required": [
|
|
"litellm_provider"
|
|
],
|
|
"properties": {
|
|
"annotation_cost_per_page": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"annotation_cost_per_page_batches": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"audio_transcription_config": {
|
|
"type": "string"
|
|
},
|
|
"bedrock_converse_supports_strict_tools": {
|
|
"type": "boolean"
|
|
},
|
|
"bedrock_output_config_effort_ceiling": {
|
|
"type": "string",
|
|
"description": "Highest reasoning effort the Bedrock output_config accepts for this model.",
|
|
"enum": [
|
|
"low",
|
|
"medium",
|
|
"high",
|
|
"max",
|
|
"xhigh"
|
|
]
|
|
},
|
|
"cache_creation_input_audio_token_cost": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_creation_input_token_cost": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "USD per token written to the provider's prompt cache."
|
|
},
|
|
"cache_creation_input_token_cost_above_100k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_above_100k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_above_1hr": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_above_1hr_above_100k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_above_1hr_above_200k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_above_200k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_above_272k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_above_272k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_above_272k_tokens_decisions": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_above_272k_tokens_flex": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Flex service-tier rate for the same-named base field."
|
|
},
|
|
"cache_creation_input_token_cost_above_272k_tokens_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"cache_creation_input_token_cost_above_272k_tokens_ultrafast": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_creation_input_token_cost_batches": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_creation_input_token_cost_decisions": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_creation_input_token_cost_flex": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Flex service-tier rate for the same-named base field."
|
|
},
|
|
"cache_creation_input_token_cost_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"cache_creation_input_token_cost_ultrafast": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_read_input_audio_token_cost": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_read_input_image_token_cost": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_read_input_token_cost": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "USD per prompt token served from the provider's prompt cache."
|
|
},
|
|
"cache_read_input_token_cost_above_100k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_read_input_token_cost_above_100k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_read_input_token_cost_above_200k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_read_input_token_cost_above_200k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_read_input_token_cost_above_200k_tokens_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"cache_read_input_token_cost_above_272k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_read_input_token_cost_above_272k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_read_input_token_cost_above_272k_tokens_decisions": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_read_input_token_cost_above_272k_tokens_flex": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Flex service-tier rate for the same-named base field."
|
|
},
|
|
"cache_read_input_token_cost_above_272k_tokens_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"cache_read_input_token_cost_above_272k_tokens_ultrafast": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_read_input_token_cost_above_512k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"cache_read_input_token_cost_balanced": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_read_input_token_cost_batches": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_read_input_token_cost_decisions": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_read_input_token_cost_flex": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Flex service-tier rate for the same-named base field."
|
|
},
|
|
"cache_read_input_token_cost_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"cache_read_input_token_cost_ultrafast": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"citation_cost_per_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"code_interpreter_cost_per_session": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"comment": {
|
|
"type": "string"
|
|
},
|
|
"cost_per_second": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"default_reasoning_effort": {
|
|
"type": "string",
|
|
"description": "Reasoning effort the provider applies when the request omits reasoning_effort. Gates whether a non-default temperature or the top_p/logprobs sampling params are accepted, which hold only when the effort resolves to 'none'.",
|
|
"enum": [
|
|
"none",
|
|
"minimal",
|
|
"low",
|
|
"medium",
|
|
"high",
|
|
"xhigh"
|
|
]
|
|
},
|
|
"deprecation_date": {
|
|
"type": "string",
|
|
"description": "Date the provider deprecates the model, YYYY-MM-DD.",
|
|
"format": "date",
|
|
"pattern": "^\\d{4}-(0[1-9]|1[0-2])-(0[1-9]|[12]\\d|3[01])$"
|
|
},
|
|
"gemini_audio_only_live": {
|
|
"type": "boolean"
|
|
},
|
|
"gemini_native_audio": {
|
|
"type": "boolean"
|
|
},
|
|
"google_maps_grounding_cost_per_query": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "USD per Grounding with Google Maps request; billed per query or per prompt per web_search_billing_unit."
|
|
},
|
|
"guardrail_cost_per_unit": {
|
|
"type": "object",
|
|
"description": "USD cost per billable guardrail unit, keyed by the provider's usage counter name (e.g. Bedrock's contentPolicyUnits).",
|
|
"additionalProperties": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
}
|
|
},
|
|
"input_cost_per_audio_per_second": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_audio_per_second_above_128k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_audio_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_audio_token_batches": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_audio_token_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"input_cost_per_character": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_character_above_128k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_image": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_image_above_128k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_image_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_image_token_batches": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_pixel": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_query": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_request": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_second": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_text_message": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_token": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "USD per prompt token."
|
|
},
|
|
"input_cost_per_token_above_100k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_above_100k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_above_128k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_above_200k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_above_200k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_above_200k_tokens_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"input_cost_per_token_above_272k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_above_272k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_above_272k_tokens_decisions": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_above_272k_tokens_flex": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Flex service-tier rate for the same-named base field."
|
|
},
|
|
"input_cost_per_token_above_272k_tokens_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"input_cost_per_token_above_272k_tokens_ultrafast": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_above_512k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_token_balanced": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_token_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "USD per prompt token via the provider's batch API."
|
|
},
|
|
"input_cost_per_token_cache_hit": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_token_decisions": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_token_flex": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Flex service-tier rate for the same-named base field."
|
|
},
|
|
"input_cost_per_token_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"input_cost_per_token_ultrafast": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_video_per_second": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_video_per_second_above_128k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_video_per_second_above_15s_interval": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_video_per_second_above_8s_interval": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"input_cost_per_video_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_video_token_batches": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_dbu_cost_per_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"litellm_provider": {
|
|
"type": "string",
|
|
"description": "LiteLLM provider slug; one of https://docs.litellm.ai/docs/providers."
|
|
},
|
|
"max_input_tokens": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "Maximum prompt/context tokens the model accepts."
|
|
},
|
|
"max_output_tokens": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "Maximum tokens the model can generate in one response."
|
|
},
|
|
"max_tokens": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "Legacy field: max output tokens if the provider specifies it, else max input tokens."
|
|
},
|
|
"metadata": {
|
|
"type": "object",
|
|
"description": "Free-form notes about the entry (e.g. pricing derivation)."
|
|
},
|
|
"mode": {
|
|
"type": "string",
|
|
"description": "Primary API surface / task type of the model.",
|
|
"enum": [
|
|
"audio_speech",
|
|
"audio_transcription",
|
|
"chat",
|
|
"completion",
|
|
"embedding",
|
|
"evaluation",
|
|
"guardrail",
|
|
"image_edit",
|
|
"image_generation",
|
|
"moderation",
|
|
"ocr",
|
|
"realtime",
|
|
"rerank",
|
|
"responses",
|
|
"search",
|
|
"vector_store",
|
|
"video_generation"
|
|
]
|
|
},
|
|
"ocr_cost_per_credit": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"ocr_cost_per_page": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"ocr_cost_per_page_batches": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"off_peak_pricing": {
|
|
"type": "object",
|
|
"description": "Rates that replace the same-named base fields while the request falls inside the stated UTC windows.",
|
|
"properties": {
|
|
"hours_utc": {
|
|
"description": "UTC \"HH:MM-HH:MM\" window, or a list of them; a window may wrap past midnight.",
|
|
"oneOf": [
|
|
{
|
|
"type": "string",
|
|
"pattern": "^([01]\\d|2[0-3]):[0-5]\\d-([01]\\d|2[0-3]):[0-5]\\d$"
|
|
},
|
|
{
|
|
"type": "array",
|
|
"items": {
|
|
"type": "string",
|
|
"pattern": "^([01]\\d|2[0-3]):[0-5]\\d-([01]\\d|2[0-3]):[0-5]\\d$"
|
|
},
|
|
"minItems": 1
|
|
}
|
|
]
|
|
},
|
|
"windows": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "object",
|
|
"properties": {
|
|
"hours_utc": {
|
|
"description": "UTC \"HH:MM-HH:MM\" window, or a list of them; a window may wrap past midnight.",
|
|
"oneOf": [
|
|
{
|
|
"type": "string",
|
|
"pattern": "^([01]\\d|2[0-3]):[0-5]\\d-([01]\\d|2[0-3]):[0-5]\\d$"
|
|
},
|
|
{
|
|
"type": "array",
|
|
"items": {
|
|
"type": "string",
|
|
"pattern": "^([01]\\d|2[0-3]):[0-5]\\d-([01]\\d|2[0-3]):[0-5]\\d$"
|
|
},
|
|
"minItems": 1
|
|
}
|
|
]
|
|
},
|
|
"weekdays": {
|
|
"type": "array",
|
|
"description": "ISO-8601 weekday numbers (1 = Monday .. 7 = Sunday) or English day names the window applies on.",
|
|
"items": {
|
|
"oneOf": [
|
|
{
|
|
"type": "integer",
|
|
"minimum": 1,
|
|
"maximum": 7
|
|
},
|
|
{
|
|
"type": "string",
|
|
"pattern": "(?i)^(mon|monday|tue|tues|tuesday|wed|wednesday|thu|thur|thurs|thursday|fri|friday|sat|saturday|sun|sunday)$"
|
|
}
|
|
]
|
|
},
|
|
"minItems": 1
|
|
}
|
|
},
|
|
"required": [
|
|
"hours_utc"
|
|
],
|
|
"additionalProperties": false
|
|
},
|
|
"minItems": 1
|
|
},
|
|
"weekday_timezone": {
|
|
"type": "string",
|
|
"description": "IANA zone the weekdays of each window are read on; defaults to UTC."
|
|
},
|
|
"input_cost_per_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_reasoning_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_read_input_token_cost": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_creation_input_token_cost": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
}
|
|
},
|
|
"anyOf": [
|
|
{
|
|
"required": [
|
|
"hours_utc"
|
|
]
|
|
},
|
|
{
|
|
"required": [
|
|
"windows"
|
|
]
|
|
}
|
|
],
|
|
"additionalProperties": false
|
|
},
|
|
"output_cost_per_audio_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_character": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_character_above_128k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_image": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_image_0.5K": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_image_1024": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_image_1536": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_image_1K": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_image_2K": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_image_4K": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_image_512": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_image_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_image_token_batches": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_pixel": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_reasoning_token": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "USD per reasoning/thinking token, when billed separately."
|
|
},
|
|
"output_cost_per_second": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_second_1080p": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_second_2k": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_second_480p": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_second_4k": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_second_720p": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_second_768p": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_token": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "USD per generated token."
|
|
},
|
|
"output_cost_per_token_above_100k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_above_100k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_above_128k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_above_200k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_above_200k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_above_200k_tokens_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"output_cost_per_token_above_272k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_above_272k_tokens_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_above_272k_tokens_decisions": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_above_272k_tokens_flex": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Flex service-tier rate for the same-named base field."
|
|
},
|
|
"output_cost_per_token_above_272k_tokens_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"output_cost_per_token_above_272k_tokens_ultrafast": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_above_512k_tokens": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Rate applied once the prompt exceeds the token threshold in the field name."
|
|
},
|
|
"output_cost_per_token_balanced": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_token_batches": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "USD per generated token via the provider's batch API."
|
|
},
|
|
"output_cost_per_token_decisions": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_token_flex": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Flex service-tier rate for the same-named base field."
|
|
},
|
|
"output_cost_per_token_priority": {
|
|
"type": "number",
|
|
"minimum": 0,
|
|
"description": "Priority service-tier rate for the same-named base field."
|
|
},
|
|
"output_cost_per_token_ultrafast": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_video_per_second": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_video_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_dbu_cost_per_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_vector_size": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "Embedding dimension for embedding models."
|
|
},
|
|
"prompt_cache_min_tokens": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "Smallest prefix the provider will actually cache; absent means the provider default applies."
|
|
},
|
|
"provider_specific_entry": {
|
|
"type": "object",
|
|
"description": "Provider-internal routing hints (e.g. bedrock_invocation_schema)."
|
|
},
|
|
"reasoning_effort_levels": {
|
|
"type": "array",
|
|
"description": "Exact reasoning_effort levels this deployment accepts; wins over supports_* flags.",
|
|
"items": {
|
|
"type": "string",
|
|
"enum": [
|
|
"none",
|
|
"minimal",
|
|
"low",
|
|
"medium",
|
|
"high",
|
|
"xhigh",
|
|
"max"
|
|
]
|
|
}
|
|
},
|
|
"regional_endpoint_uplift_multiplier": {
|
|
"type": "number",
|
|
"minimum": 1,
|
|
"description": "Multiplier applied to all token costs when served from a non-global Vertex AI endpoint (e.g. 1.10 = +10%)."
|
|
},
|
|
"regional_processing_uplift_multiplier_eu": {
|
|
"type": "number",
|
|
"minimum": 1,
|
|
"description": "Multiplier applied to all token costs for EU data residency (e.g. 1.10 = +10%)."
|
|
},
|
|
"regional_processing_uplift_multiplier_us": {
|
|
"type": "number",
|
|
"minimum": 1,
|
|
"description": "Multiplier applied to all token costs for US data residency (e.g. 1.10 = +10%)."
|
|
},
|
|
"rpm": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "Provider default requests-per-minute limit."
|
|
},
|
|
"search_context_cost_per_query": {
|
|
"type": "object",
|
|
"description": "USD cost per web search query, keyed by search context size.",
|
|
"properties": {
|
|
"search_context_size_low": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"search_context_size_medium": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"search_context_size_high": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
}
|
|
},
|
|
"additionalProperties": false
|
|
},
|
|
"source": {
|
|
"type": "string",
|
|
"description": "URL of the provider pricing/model page this entry was taken from."
|
|
},
|
|
"supported_audio_formats": {
|
|
"type": "array",
|
|
"description": "Audio container formats the model can return.",
|
|
"items": {
|
|
"type": "string",
|
|
"enum": [
|
|
"mp3",
|
|
"wav"
|
|
]
|
|
}
|
|
},
|
|
"supported_endpoints": {
|
|
"type": "array",
|
|
"description": "OpenAI-style API routes this model can be called through, e.g. /v1/chat/completions.",
|
|
"items": {
|
|
"type": "string"
|
|
}
|
|
},
|
|
"supported_modalities": {
|
|
"type": "array",
|
|
"description": "Input modalities the model accepts.",
|
|
"items": {
|
|
"type": "string",
|
|
"enum": [
|
|
"text",
|
|
"image",
|
|
"audio",
|
|
"video"
|
|
]
|
|
}
|
|
},
|
|
"supported_output_modalities": {
|
|
"type": "array",
|
|
"description": "Output modalities the model can produce.",
|
|
"items": {
|
|
"type": "string",
|
|
"enum": [
|
|
"text",
|
|
"image",
|
|
"audio",
|
|
"video",
|
|
"code"
|
|
]
|
|
}
|
|
},
|
|
"supported_regions": {
|
|
"type": "array",
|
|
"description": "Cloud regions the model is available in ('global' or region ids).",
|
|
"items": {
|
|
"type": "string"
|
|
}
|
|
},
|
|
"supports_adaptive_thinking": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_anthropic_compaction": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_anthropic_thinking_payload": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_assistant_prefill": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_audio_input": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_audio_output": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_bedrock_converse_tool_result_images": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_bedrock_native_structured_output": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_bedrock_runtime_chat_completions_inline_reasoning": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_bedrock_runtime_chat_completions_response_format": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_bedrock_runtime_chat_completions_tools_with_reasoning": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_box_questions": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_computer_use": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_embedding_image_input": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_fast_mode": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_forced_tool_use": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_function_calling": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_image_input": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_image_size": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_legacy_thinking": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_low_reasoning_effort": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_max_reasoning_effort": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_mid_conversation_system": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_minimal_reasoning_effort": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_multimodal": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_native_streaming": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_native_structured_output": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_none_reasoning_effort": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_nova_canvas_image_edit": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_output_config": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_parallel_function_calling": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_parallel_tool_use_config": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_pdf_input": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_prompt_cache_breakpoint": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_prompt_caching": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_reasoning": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_regex_lookaround": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_response_schema": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_sampling_params": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_speed": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_system_messages": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_thinking_cache_preservation": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_tool_choice": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_tool_search": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_url_context": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_video_input": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_vision": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_web_search": {
|
|
"type": "boolean"
|
|
},
|
|
"supports_xhigh_reasoning_effort": {
|
|
"type": "boolean"
|
|
},
|
|
"thinking_always_on": {
|
|
"type": "boolean"
|
|
},
|
|
"tiered_pricing": {
|
|
"type": "array",
|
|
"description": "Context-length or result-count tiered rates; each tier's costs apply within its range.",
|
|
"items": {
|
|
"type": "object",
|
|
"properties": {
|
|
"range": {
|
|
"type": "array",
|
|
"description": "[min, max] prompt-token span this tier applies to.",
|
|
"items": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"minItems": 2,
|
|
"maxItems": 2
|
|
},
|
|
"max_results_range": {
|
|
"type": "array",
|
|
"description": "[min, max] result-count span this tier applies to (search models).",
|
|
"items": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"minItems": 2,
|
|
"maxItems": 2
|
|
},
|
|
"input_cost_per_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"output_cost_per_reasoning_token": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_read_input_token_cost": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"cache_creation_input_token_cost": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
},
|
|
"input_cost_per_query": {
|
|
"type": "number",
|
|
"minimum": 0
|
|
}
|
|
},
|
|
"additionalProperties": false
|
|
}
|
|
},
|
|
"tpm": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "Provider default tokens-per-minute limit."
|
|
},
|
|
"use_openai_responses_path": {
|
|
"type": "boolean"
|
|
},
|
|
"uses_embed_content": {
|
|
"type": "boolean"
|
|
},
|
|
"vertex_ai_audio_api": {
|
|
"type": "string",
|
|
"enum": [
|
|
"lyria_predict",
|
|
"lyria_interactions"
|
|
]
|
|
},
|
|
"web_search_billing_unit": {
|
|
"type": "string",
|
|
"description": "Whether web search is billed per query or per prompt.",
|
|
"enum": [
|
|
"per_query",
|
|
"per_prompt"
|
|
]
|
|
}
|
|
},
|
|
"additionalProperties": true
|
|
}
|
|
}
|
|
}
|