mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-15 04:19:08 +00:00
feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706)
* feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos
DashScopeVideoConfig maps litellm's OpenAI Videos API onto Alibaba Model
Studio's async task API: create hits
/api/v1/services/aigc/video-generation/video-synthesis and status and content
poll /api/v1/tasks/{task_id}. It is registered for dashscope and for the
qwencloud and qwen_ai_platform brand aliases, following the same
get_dashscope_family_*_config pattern image generation uses. Unlike the
Sora-shaped providers the request body is nested, so prompt and media go under
input and everything else goes under parameters. X-DashScope-Async is always
sent, because DashScope rejects the call outright without it
OpenAI params translate to DashScope's contract. seconds becomes duration, and
size yields both a ratio and a named resolution tier (1280x720 gives 16:9 plus
720P), since sending only the ratio leaves the request on the default 1080P and
silently bills at that rate. input_reference becomes a single media entry,
base64 encoded into a data URI for file inputs. Which media type a model takes
(first_frame or, on the reference-to-video models, reference_image) is read
from provider_specific_entry.dashscope_video_reference_input via
get_model_info rather than hardcoded; models without it predate the media
array and get the flat img_url field. A caller-supplied media array always
wins, as that is the only way to express multi-role combinations
The create call is the only billed call, so it bills what DashScope will
deliver: the requested duration and tier, or the documented 5 second and 1080P
defaults when they are omitted. Smart duration (duration -1) lets the service
pick the length after that call, so it is rejected rather than recorded at
zero, with a 400. Task creation and task lookup both report failures as a 200 carrying
top-level code and message with no output, so that shape is raised instead of
surfacing as a queued video with an empty id. Status responses re-encode the
deployment model from the polled id, so follow-up polls and downloads route to
the same deployment. Timestamps parse as UTC+8. DashScope publishes no list,
delete or remix endpoint, so those raise rather than guess at a URL
The end-to-end tests drive the real avideo_generation() entrypoint through an
httpx MockTransport and assert on the JSON body httpx actually encoded. That is
what catches a mapper that drops input_reference before the request is built,
or a body fragment json cannot serialize, neither of which a test calling the
transform directly would see
Both model_prices maps gain wan3.0-video, wan3.0-video-prime, wan2.7-t2v,
wan2.7-i2v, wan2.7-r2v and the HappyHorse 1.0 and 1.1 t2v, i2v and r2v models,
for dashscope and the qwencloud and qwen_ai_platform aliases, priced per output
second by resolution tier from Model Studio's USD rate cards. dashscope and
qwen_ai_platform default to the Beijing host and use its rate card; qwencloud
defaults to the international host and uses Singapore's. The three
*_API_BASE_VIDEO overrides are documented in BerriAI/litellm-docs#1920
Co-authored-by: AaronHowell <237895480@qq.com>
* fix(dashscope): fail loudly on video responses that cannot be read
Addresses the review on the video generation PR. A body that is not the
documented task shape, an output block missing task_id or task_status, an
undocumented task_status and a missing output all raise DashScopeVideoError
now, instead of defaulting into a queued video with an empty id that callers
poll forever. A JSONDecodeError from a gateway response no longer escapes as
a generic connection error.
task_id and task_status are required on _TaskOutput, and a missing output
raises even when the body carries no code or message.
Download failures move into the 4xx range, since a 5xx has the proxy retry
them: 409 while the task is still running, 400 for a task that failed, 404
for an id that expired. The status a provider reported is kept when it
reported one, so an in-body failure that arrives on a 200 is billed as the
400 it is rather than as a retryable connection error.
characters and image are only dropped when drop_params is set; the params
that are rebuilt into the body or consumed by the caller (model, prompt,
user, extra_*) stay silently skipped, since the proxy stamps user from the
caller's end-user header.
Co-authored-by: AaronHowell <237895480@qq.com>
* fix(dashscope): clear the lint gates after merging main
* fix(dashscope): accept file tuples and paths as video input_reference
video_generation() types input_reference as FileTypes, which includes
(filename, content[, content_type[, headers]]) tuples and pathlib paths,
but the DashScope encoder only read bytes and file objects, so those
image-to-video calls raised before reaching DashScope. Bare strings stay
URLs and are never opened as paths.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(dashscope): snap size to a supported ratio and read file tuples and paths
* test(dashscope): assert whole mapped values and bill against hand-written rates
The tier pricing test read its expected rate from the same map the cost
calculator uses, so a wrong rate passed. Billing is now checked against
fixed rates driven through the status response usage, while the map test
only asserts every supported tier has its own rate. Size mapping tests
compare the full mapped dict, and the single-test wiring class is a
plain function.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(videos): download content through the caller's client and stop retrying dead tasks
Two Cursor Bugbot findings on the DashScope video PR.
The DashScope artifact lives on OSS, so transform_video_content_response makes a
second request. It built its own cached client, discarding an injected client and
its ssl_verify. Thread the client through the content response the way
transform_video_status_retrieve_response already does, and use it in the three
providers that fetch from a second URL (DashScope, RunwayML, fal.ai). The
remaining providers accept it and read the response body, matching how they
already accept an unused client on status retrieve.
A FAILED or CANCELED task reported without a code or message fell past the
in-flight check into the 502 branch, so the proxy retried a task that can never
produce a video. Raise 400 for a terminal status instead.
* fix(dashscope): answer 404, not 409, when a Wan video is downloaded before it finishes
A download that lands while the task is still PENDING or RUNNING answered 409. That
reads as a client error, which was the intent, but litellm's retry predicate counts 409
as retryable, so the proxy polled DashScope twice more before giving up and then counted
the caller's own early download against the model's error rate
404 is what the OpenAI videos API documents for content that is not ready, and it is the
one status in that family the retry predicate leaves alone. The rest of the mapping is
unchanged: 400 for a task the provider reported as failed, 404 for an expired task id,
502 when a succeeded task carries no artifact
The download tests now drive avideo_content and assert the caller sees NotFoundError
rather than an internal server error
* fix(dashscope): route a video poll back to the model group that created the task
The create response encoded the upstream model name into the returned id. Both the
dashscope and qwen_ai_platform prefixes serve the same model names from their own hosts,
so a proxy with a deployment per region had deployments that only the model group told
apart. Encoding the bare model name resolved a poll to whichever deployment the router
lists first, which sent it to the other region's host with that region's key and a task
id that region never issued. That is the reroute the earlier proof records as the
dashscope and qwen_ai_platform poll ids coming back tagged qwencloud
The id now carries the model group the create was routed to, which is what the proxy
resolves a poll back to, and falls back to the model name for a direct SDK call that has
no router
The regression test drives a real router with one deployment per region, under a
different prefix and under the same one, and asserts the poll carries the key of the
region that created the task
* fix(dashscope): read the routed model group from litellm_metadata
The router stamps the model group it picked into litellm_metadata. metadata only
carries a copy when the caller sent no metadata of its own, and every proxy request
sends some, so a proxied create still encoded the bare model name and the poll
resolved to whichever region's deployment the router lists first
The regression test now also creates the video with caller metadata, which fails
without this change
* fix(dashscope): reject unbillable video requests and keep fal and runway on their own clients
* fix(dashscope): show the caller why a video task is gone or not ready instead of the bare upstream body
* fix(dashscope): check video request values before the billed create and honor per-call drop_params
Type check documented DashScope parameters so a bad seed answers 400 instead of a billed failed task,
pass the call's and deployment's drop_params into the video mapping, answer 400 for an expired task id so
the deployment is not cooled down, read DASHSCOPE_API_BASE for video, keep polling SUSPENDED tasks, and
refuse to map size onto models litellm has no price for.
* fix(dashscope): carry the requested model name in video ids so alias and wildcard keys can poll
A key scoped to an alias or to a name a wildcard serves could create a
video and then got a 403 on the poll and the download, because the id
carried the group that served the call (wan3, qwencloud/*) and the key
was never given that name. The id now carries the name the router was
asked for on that hop, and the router resolves an alias, routing group
or wildcard name in an id back to itself when no deployment name or
provider model matches it first
* fix(dashscope): keep a routing group's video polls on the member that created the task
A routing group spans several model groups, so an id carrying the group's name
load-balanced its polls, and some left for another region's host and key with a
task id that region never issued. The id now carries the member that served the
task when the request named a routing group, and keeps the requested name for a
model group, an alias, or a wildcard match.
* fix(dashscope): carry the requested and served model in video ids so group-scoped keys poll in the creating region
* fix(proxy): pin a video's served group only after key and team aliases resolve
* test(integration): audit DashScope video create, poll, download, routing and chaos paths
* fix(proxy): keep catch-all video polls on their provider and count served polls against the requested group
* test(routing): expect a forged served group to be dropped from the polled video id
* test(integration): cover rate-limited routing group polls and catch-all polls of DashScope videos
---------
Co-authored-by: AaronHowell <237895480@qq.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Rick
GitHub
parent
829d50990d
commit
4f3cb48dc1
34 changed files with 5981 additions and 33 deletions
|
|
@ -145,24 +145,35 @@ class BaseVideoConfig(ABC):
|
|||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: LiteLLMLoggingObj,
|
||||
client: "HTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
pass
|
||||
"""
|
||||
Args:
|
||||
raw_response: Raw HTTP response
|
||||
logging_obj: Logging object
|
||||
client: The client that served the request, for providers that download
|
||||
the video from a second URL instead of reading the response body
|
||||
"""
|
||||
|
||||
async def async_transform_video_content_response(
|
||||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: LiteLLMLoggingObj,
|
||||
client: "AsyncHTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
"""
|
||||
Async transform video content download response to bytes.
|
||||
Optional method - providers can override if they need async transformations
|
||||
(e.g., RunwayML for downloading video from CloudFront URL).
|
||||
|
||||
Default implementation falls back to sync transform_video_content_response.
|
||||
Default implementation falls back to sync transform_video_content_response,
|
||||
which takes the sync client, so providers that download through the client
|
||||
have to override this method to receive the async one.
|
||||
|
||||
Args:
|
||||
raw_response: Raw HTTP response
|
||||
logging_obj: Logging object
|
||||
client: The async client that served the request
|
||||
|
||||
Returns:
|
||||
Video content as bytes
|
||||
|
|
|
|||
|
|
@ -7724,6 +7724,7 @@ class BaseLLMHTTPHandler:
|
|||
return video_content_provider_config.transform_video_content_response(
|
||||
raw_response=response,
|
||||
logging_obj=logging_obj,
|
||||
client=sync_httpx_client,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
|
|
@ -7803,6 +7804,7 @@ class BaseLLMHTTPHandler:
|
|||
return await video_content_provider_config.async_transform_video_content_response(
|
||||
raw_response=response,
|
||||
logging_obj=logging_obj,
|
||||
client=async_httpx_client,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@ if TYPE_CHECKING:
|
|||
BaseImageGenerationConfig,
|
||||
)
|
||||
from litellm.llms.base_llm.rerank.transformation import BaseRerankConfig
|
||||
from litellm.llms.base_llm.videos.transformation import BaseVideoConfig
|
||||
|
||||
DASHSCOPE_CHAT_COMPATIBLE_PATH: Final = "/compatible-mode/v1"
|
||||
DASHSCOPE_RERANK_PATH: Final = "/compatible-api/v1/reranks"
|
||||
|
|
@ -89,6 +90,22 @@ def get_dashscope_family_image_generation_config(
|
|||
return DashScopeImageGenerationConfig()
|
||||
|
||||
|
||||
def get_dashscope_family_video_config(
|
||||
custom_llm_provider: str,
|
||||
) -> "BaseVideoConfig":
|
||||
if custom_llm_provider == "qwencloud":
|
||||
from litellm.llms.dashscope.qwencloud import QwenCloudVideoConfig
|
||||
|
||||
return QwenCloudVideoConfig()
|
||||
if custom_llm_provider == "qwen_ai_platform":
|
||||
from litellm.llms.dashscope.qwen_ai_platform import QwenAIPlatformVideoConfig
|
||||
|
||||
return QwenAIPlatformVideoConfig()
|
||||
from litellm.llms.dashscope.videos.transformation import DashScopeVideoConfig
|
||||
|
||||
return DashScopeVideoConfig()
|
||||
|
||||
|
||||
def resolve_dashscope_family_api_key(custom_llm_provider: str, api_key: str | None) -> str | None:
|
||||
if custom_llm_provider == "dashscope":
|
||||
return api_key or get_secret_str("DASHSCOPE_API_KEY")
|
||||
|
|
|
|||
|
|
@ -7,12 +7,14 @@ from .common_utils import resolve_dashscope_family_rerank_api_base
|
|||
from .embed.transformation import DashScopeEmbeddingConfig
|
||||
from .image_generation.transformation import DashScopeImageGenerationConfig
|
||||
from .rerank.transformation import DashScopeRerankConfig
|
||||
from .videos.transformation import DashScopeVideoConfig
|
||||
|
||||
QWEN_AI_PLATFORM_API_BASE: Final = "https://dashscope.aliyuncs.com/compatible-mode/v1"
|
||||
QWEN_AI_PLATFORM_RERANK_API_BASE: Final = "https://dashscope.aliyuncs.com/compatible-api/v1/reranks"
|
||||
QWEN_AI_PLATFORM_IMAGE_API_BASE: Final = (
|
||||
"https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation"
|
||||
)
|
||||
QWEN_AI_PLATFORM_VIDEO_API_BASE: Final = "https://dashscope.aliyuncs.com"
|
||||
|
||||
|
||||
def _resolve_qwen_ai_platform_api_key(api_key: str | None) -> str | None:
|
||||
|
|
@ -63,3 +65,11 @@ class QwenAIPlatformImageGenerationConfig(DashScopeImageGenerationConfig):
|
|||
|
||||
def _resolve_image_api_base(self, image_api_base: str | None) -> str:
|
||||
return image_api_base or get_secret_str("QWEN_AI_PLATFORM_API_BASE_IMAGE") or QWEN_AI_PLATFORM_IMAGE_API_BASE
|
||||
|
||||
|
||||
class QwenAIPlatformVideoConfig(DashScopeVideoConfig):
|
||||
def _resolve_api_key(self, api_key: str | None) -> str:
|
||||
return _require_qwen_ai_platform_api_key(api_key)
|
||||
|
||||
def _resolve_video_api_base(self, video_api_base: str | None) -> str:
|
||||
return video_api_base or get_secret_str("QWEN_AI_PLATFORM_API_BASE_VIDEO") or QWEN_AI_PLATFORM_VIDEO_API_BASE
|
||||
|
|
|
|||
|
|
@ -7,12 +7,14 @@ from .common_utils import resolve_dashscope_family_rerank_api_base
|
|||
from .embed.transformation import DashScopeEmbeddingConfig
|
||||
from .image_generation.transformation import DashScopeImageGenerationConfig
|
||||
from .rerank.transformation import DashScopeRerankConfig
|
||||
from .videos.transformation import DashScopeVideoConfig
|
||||
|
||||
QWENCLOUD_API_BASE: Final = "https://dashscope-intl.aliyuncs.com/compatible-mode/v1"
|
||||
QWENCLOUD_RERANK_API_BASE: Final = "https://dashscope-intl.aliyuncs.com/compatible-api/v1/reranks"
|
||||
QWENCLOUD_IMAGE_API_BASE: Final = (
|
||||
"https://dashscope-intl.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation"
|
||||
)
|
||||
QWENCLOUD_VIDEO_API_BASE: Final = "https://dashscope-intl.aliyuncs.com"
|
||||
|
||||
|
||||
def _resolve_qwencloud_api_key(api_key: str | None) -> str | None:
|
||||
|
|
@ -63,3 +65,11 @@ class QwenCloudImageGenerationConfig(DashScopeImageGenerationConfig):
|
|||
|
||||
def _resolve_image_api_base(self, image_api_base: str | None) -> str:
|
||||
return image_api_base or get_secret_str("QWENCLOUD_API_BASE_IMAGE") or QWENCLOUD_IMAGE_API_BASE
|
||||
|
||||
|
||||
class QwenCloudVideoConfig(DashScopeVideoConfig):
|
||||
def _resolve_api_key(self, api_key: str | None) -> str:
|
||||
return _require_qwencloud_api_key(api_key)
|
||||
|
||||
def _resolve_video_api_base(self, video_api_base: str | None) -> str:
|
||||
return video_api_base or get_secret_str("QWENCLOUD_API_BASE_VIDEO") or QWENCLOUD_VIDEO_API_BASE
|
||||
|
|
|
|||
0
litellm/llms/dashscope/videos/__init__.py
Normal file
0
litellm/llms/dashscope/videos/__init__.py
Normal file
992
litellm/llms/dashscope/videos/transformation.py
Normal file
992
litellm/llms/dashscope/videos/transformation.py
Normal file
|
|
@ -0,0 +1,992 @@
|
|||
"""
|
||||
DashScope (Alibaba Model Studio) async video task API: create, poll and download. DashScope has no list,
|
||||
delete or remix endpoint, so those raise.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import json
|
||||
from collections.abc import Mapping
|
||||
from datetime import datetime
|
||||
from io import BufferedReader, BytesIO
|
||||
from math import log
|
||||
from pathlib import Path, PurePath
|
||||
from types import MappingProxyType
|
||||
from typing import TYPE_CHECKING, Annotated, Final
|
||||
|
||||
import httpx
|
||||
from httpx._types import RequestFiles
|
||||
from pydantic import BaseModel, Field, StrictBool, StrictInt, StrictStr, TypeAdapter, ValidationError
|
||||
|
||||
import litellm
|
||||
from litellm.exceptions import UnsupportedParamsError
|
||||
from litellm.images.utils import ImageEditRequestUtils
|
||||
from litellm.litellm_core_utils.url_utils import encode_url_path_segment
|
||||
from litellm.llms.base_llm.chat.transformation import BaseLLMException
|
||||
from litellm.llms.base_llm.videos.transformation import BaseVideoConfig
|
||||
from litellm.llms.custom_httpx.http_handler import (
|
||||
_get_httpx_client, # pyright: ignore[reportPrivateUsage, reportUnknownVariableType] # house cached-client factory has no public alias and its stub leaves params untyped
|
||||
get_async_httpx_client, # pyright: ignore[reportUnknownVariableType] # factory stub leaves params untyped
|
||||
)
|
||||
from litellm.secret_managers.main import get_secret_str
|
||||
from litellm.types.router import GenericLiteLLMParams
|
||||
from litellm.types.utils import ModelInfo
|
||||
from litellm.types.videos.main import VideoCreateOptionalRequestParams, VideoObject
|
||||
from litellm.types.videos.utils import (
|
||||
decode_video_id_with_provider,
|
||||
encode_video_id_with_provider,
|
||||
extract_original_video_id,
|
||||
)
|
||||
from litellm.utils import get_model_info
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from litellm.litellm_core_utils.litellm_logging import Logging
|
||||
from litellm.llms.custom_httpx.http_handler import AsyncHTTPHandler, HTTPHandler
|
||||
|
||||
|
||||
class DashScopeVideoError(BaseLLMException):
|
||||
pass
|
||||
|
||||
|
||||
class _TaskOutput(BaseModel, frozen=True):
|
||||
task_id: str
|
||||
task_status: str
|
||||
submit_time: str | None = None
|
||||
scheduled_time: str | None = None
|
||||
end_time: str | None = None
|
||||
orig_prompt: str | None = None
|
||||
video_url: str | None = None
|
||||
code: str | None = None
|
||||
message: str | None = None
|
||||
|
||||
|
||||
class _TaskUsage(BaseModel, frozen=True):
|
||||
video_count: int | None = None
|
||||
duration: float | None = None
|
||||
input_video_duration: float | None = None
|
||||
output_video_duration: float | None = None
|
||||
fps: int | None = None
|
||||
SR: int | None = None
|
||||
ratio: str | None = None
|
||||
|
||||
|
||||
class _TaskResponse(BaseModel, frozen=True):
|
||||
output: _TaskOutput | None = None
|
||||
usage: _TaskUsage | None = None
|
||||
request_id: str | None = None
|
||||
code: str | None = None
|
||||
message: str | None = None
|
||||
|
||||
|
||||
DASHSCOPE_VIDEO_DEFAULT_API_BASE: Final = "https://dashscope.aliyuncs.com"
|
||||
DASHSCOPE_VIDEO_SYNTHESIS_PATH: Final = "/api/v1/services/aigc/video-generation/video-synthesis"
|
||||
DASHSCOPE_TASKS_PATH: Final = "/api/v1/tasks"
|
||||
DASHSCOPE_COMPATIBLE_MODE_PATH: Final = "/compatible-mode/v1"
|
||||
|
||||
_EMPTY_PARAMS: Final[dict[str, object]] = {} # mutable-ok: BaseVideoConfig contract empty params
|
||||
|
||||
_TASK_RESPONSE_ADAPTER: Final = TypeAdapter(_TaskResponse)
|
||||
_PARAMETERS_ADAPTER: Final = TypeAdapter(dict[str, object])
|
||||
_FILE_TUPLE_ADAPTER: Final = TypeAdapter(tuple[object, ...])
|
||||
|
||||
_STATUS_MAP: Final = MappingProxyType(
|
||||
{
|
||||
"PENDING": "queued",
|
||||
"RUNNING": "in_progress",
|
||||
"SUCCEEDED": "completed",
|
||||
"FAILED": "failed",
|
||||
"CANCELED": "cancelled",
|
||||
"SUSPENDED": "queued",
|
||||
}
|
||||
)
|
||||
|
||||
_PENDING_STATUSES: Final = frozenset({"PENDING", "RUNNING", "SUSPENDED"})
|
||||
|
||||
_TERMINAL_FAILURE_STATUSES: Final = frozenset({"FAILED", "CANCELED"})
|
||||
|
||||
_UNKNOWN_TASK_STATUS: Final = "UNKNOWN"
|
||||
|
||||
_TASK_NOT_FOUND_MESSAGE: Final = (
|
||||
"Task not found. DashScope task ids expire 24 hours after creation and are only visible in the region that "
|
||||
"created them."
|
||||
)
|
||||
|
||||
_LEGACY_INPUT_KEYS: Final = (
|
||||
"img_url",
|
||||
"first_frame_url",
|
||||
"last_frame_url",
|
||||
"audio_url",
|
||||
)
|
||||
|
||||
_INPUT_KEYS: Final = ("media", "negative_prompt", *_LEGACY_INPUT_KEYS)
|
||||
|
||||
_REFERENCE_INPUT_FIELD: Final = "dashscope_video_reference_input"
|
||||
_MEDIA_REFERENCE_TYPES: Final = frozenset({"first_frame", "reference_image"})
|
||||
|
||||
DASHSCOPE_DEFAULT_DURATION_SECONDS: Final = 5
|
||||
DASHSCOPE_DEFAULT_RESOLUTION: Final = "1080P"
|
||||
DASHSCOPE_SMART_DURATION: Final = -1
|
||||
|
||||
_PARAMETER_KEYS: Final = (
|
||||
"resolution",
|
||||
"ratio",
|
||||
"duration",
|
||||
"audio",
|
||||
"seed",
|
||||
"prompt_extend",
|
||||
"watermark",
|
||||
)
|
||||
|
||||
# Rebuilt into the body by transform_video_create_request, consumed by the http handler, or stamped by the
|
||||
# proxy from the caller's end-user header, so none of them is a DashScope field.
|
||||
_REQUEST_METADATA: Final = frozenset({"model", "prompt", "user", "extra_headers", "extra_query", "extra_body"})
|
||||
|
||||
# DashScope has no counterpart for these, so they are only dropped when the caller opted in.
|
||||
_UNSUPPORTED_BY_DASHSCOPE: Final = frozenset({"characters", "image"})
|
||||
|
||||
_MAPPED_ELSEWHERE: Final = frozenset({*_REQUEST_METADATA, *_UNSUPPORTED_BY_DASHSCOPE, "seconds", "size", "parameters"})
|
||||
|
||||
_CALLER_PARAMETERS_FIELD: Final = "dashscope_parameters"
|
||||
|
||||
_NOT_CALLER_PARAMETERS: Final = frozenset({*_INPUT_KEYS, *_REQUEST_METADATA, "input_reference", "parameters"})
|
||||
|
||||
# DashScope bills a reference video's own seconds on top of the output (usage.duration = input + output).
|
||||
_INPUT_VIDEO_MEDIA_TYPE: Final = "reference_video"
|
||||
|
||||
_TIER_COST_FIELDS: Final = MappingProxyType(
|
||||
{
|
||||
"480P": "output_cost_per_second_480p",
|
||||
"720P": "output_cost_per_second_720p",
|
||||
"1080P": "output_cost_per_second",
|
||||
}
|
||||
)
|
||||
|
||||
_RESOLUTION_TIERS: Final = MappingProxyType({480: "480p", 720: "720p", 1080: "1080p"})
|
||||
|
||||
_RESOLUTION_HEIGHT_TIERS: Final[tuple[tuple[int, str], ...]] = (
|
||||
(600, "480p"),
|
||||
(900, "720p"),
|
||||
)
|
||||
|
||||
_SUPPORTED_RATIOS: Final[tuple[tuple[int, int], ...]] = ((16, 9), (9, 16), (1, 1), (4, 3), (3, 4))
|
||||
|
||||
|
||||
class _KnownParameters(BaseModel, frozen=True):
|
||||
"""A malformed value here can fail the task after DashScope created and billed it, so it is caught first."""
|
||||
|
||||
duration: StrictInt | None = None
|
||||
seed: Annotated[StrictInt, Field(ge=0, le=2_147_483_647)] | None = None
|
||||
audio: StrictBool | None = None
|
||||
prompt_extend: StrictBool | None = None
|
||||
watermark: StrictBool | None = None
|
||||
resolution: StrictStr | None = None
|
||||
ratio: StrictStr | None = None
|
||||
|
||||
|
||||
def _video_error(raw_response: httpx.Response, status_code: int, message: str) -> DashScopeVideoError:
|
||||
"""The provider's own status wins when it reported one. DashScope otherwise answers 200 even for
|
||||
failures, and a 200 has the proxy retry a call the provider already rejected, so the caller's
|
||||
status is used as the floor."""
|
||||
return DashScopeVideoError(
|
||||
status_code=raw_response.status_code if raw_response.status_code >= 400 else status_code,
|
||||
message=message,
|
||||
headers=raw_response.headers,
|
||||
)
|
||||
|
||||
|
||||
def _task_not_found(raw_response: httpx.Response) -> DashScopeVideoError:
|
||||
"""400, since a 404 would put the deployment in cooldown for everyone over one caller's expired id."""
|
||||
return _video_error(raw_response, 400, _TASK_NOT_FOUND_MESSAGE)
|
||||
|
||||
|
||||
def _parse_task_response(raw_response: httpx.Response) -> _TaskResponse:
|
||||
"""A body that is not the documented task shape is a provider-side failure, not a parse error."""
|
||||
try:
|
||||
return _TASK_RESPONSE_ADAPTER.validate_python(raw_response.json())
|
||||
except (json.JSONDecodeError, ValidationError) as e:
|
||||
raise _video_error(raw_response, 502, f"Unexpected DashScope video response: {e}") from e
|
||||
|
||||
|
||||
def _normalized_api_base(api_base: str) -> str:
|
||||
trimmed: Final = api_base.rstrip("/")
|
||||
if trimmed.endswith(DASHSCOPE_COMPATIBLE_MODE_PATH):
|
||||
return trimmed[: -len(DASHSCOPE_COMPATIBLE_MODE_PATH)]
|
||||
return trimmed
|
||||
|
||||
|
||||
def _ratio_from_size(size: str) -> str | None:
|
||||
ratio_width_str, ratio_separator, ratio_height_str = size.partition(":")
|
||||
if ratio_separator:
|
||||
return size if ratio_width_str.isdigit() and ratio_height_str.isdigit() else None
|
||||
width, height = _size_dimensions(size) or (0, 0)
|
||||
if not width or not height:
|
||||
return None
|
||||
ratio_width, ratio_height = min(
|
||||
_SUPPORTED_RATIOS,
|
||||
key=lambda ratio: abs(log((width * ratio[1]) / (height * ratio[0]))),
|
||||
)
|
||||
return f"{ratio_width}:{ratio_height}"
|
||||
|
||||
|
||||
def _size_dimensions(size: str) -> tuple[int, int] | None:
|
||||
width_str, separator, height_str = size.partition("x")
|
||||
if not separator or not (width_str.isdigit() and height_str.isdigit()):
|
||||
return None
|
||||
return int(width_str), int(height_str)
|
||||
|
||||
|
||||
def _resolution_from_size(size: str) -> str | None:
|
||||
"""
|
||||
DashScope bills per second by named tier, so a size must map to its tier or it bills at the 1080P default.
|
||||
"""
|
||||
dimensions: Final = _size_dimensions(size)
|
||||
if dimensions is None:
|
||||
return None
|
||||
shortest_side: Final = min(dimensions)
|
||||
return next(
|
||||
(label.upper() for threshold, label in _RESOLUTION_HEIGHT_TIERS if shortest_side < threshold),
|
||||
"1080P",
|
||||
)
|
||||
|
||||
|
||||
def _duration_param(seconds: object) -> int | None:
|
||||
if isinstance(seconds, bool):
|
||||
return None
|
||||
if isinstance(seconds, int):
|
||||
return seconds
|
||||
if not isinstance(seconds, (str, float)):
|
||||
return None
|
||||
try:
|
||||
parsed: Final = float(seconds)
|
||||
except ValueError:
|
||||
return None
|
||||
return int(parsed) if parsed.is_integer() else None
|
||||
|
||||
|
||||
def _unsupported_value(model: str, message: str) -> UnsupportedParamsError:
|
||||
return UnsupportedParamsError(
|
||||
message=f"{message}, set drop_params=True to ignore it",
|
||||
model=model,
|
||||
llm_provider="dashscope",
|
||||
)
|
||||
|
||||
|
||||
def _duration_fields(seconds: object, model: str, drop_params: bool) -> Mapping[str, object]:
|
||||
if seconds is None:
|
||||
return _EMPTY_PARAMS
|
||||
duration: Final = _duration_param(seconds)
|
||||
if duration is not None:
|
||||
return MappingProxyType({"duration": duration})
|
||||
if drop_params:
|
||||
return _EMPTY_PARAMS
|
||||
raise _unsupported_value(model, f"seconds must be a whole number of seconds, got {seconds!r}")
|
||||
|
||||
|
||||
def _size_fields(size: object, model: str, drop_params: bool) -> Mapping[str, object]:
|
||||
if size is None:
|
||||
return _EMPTY_PARAMS
|
||||
if not _priced_tiers(model):
|
||||
if drop_params:
|
||||
return _EMPTY_PARAMS
|
||||
raise _unsupported_value(
|
||||
model,
|
||||
f"size only maps onto DashScope video models in litellm's cost map, and {model} is not one, so send "
|
||||
"DashScope's own size fields inside parameters",
|
||||
)
|
||||
ratio: Final = _ratio_from_size(size) if isinstance(size, str) else None
|
||||
if ratio is None or not isinstance(size, str):
|
||||
if drop_params:
|
||||
return _EMPTY_PARAMS
|
||||
raise _unsupported_value(model, f"size must be WIDTHxHEIGHT or a WIDTH:HEIGHT ratio, got {size!r}")
|
||||
resolution: Final = _resolution_from_size(size)
|
||||
return MappingProxyType({"ratio": ratio} if resolution is None else {"ratio": ratio, "resolution": resolution})
|
||||
|
||||
|
||||
def _object_fields(value: object) -> Mapping[str, object] | None:
|
||||
if value is None:
|
||||
return _EMPTY_PARAMS
|
||||
try:
|
||||
return _PARAMETERS_ADAPTER.validate_python(value)
|
||||
except ValidationError:
|
||||
return None
|
||||
|
||||
|
||||
def _parameters_fields(parameters: object, model: str, drop_params: bool) -> Mapping[str, object]:
|
||||
fields: Final = _object_fields(parameters)
|
||||
if fields is not None:
|
||||
return fields
|
||||
if drop_params:
|
||||
return _EMPTY_PARAMS
|
||||
raise _unsupported_value(
|
||||
model, f"parameters must be an object of DashScope request fields, got {type(parameters).__name__}"
|
||||
)
|
||||
|
||||
|
||||
def _malformed_parameters(parameters: Mapping[str, object]) -> tuple[tuple[str, str], ...]:
|
||||
try:
|
||||
_ = _KnownParameters.model_validate(dict(parameters))
|
||||
except ValidationError as e:
|
||||
return tuple((str(error["loc"][0]), error["msg"]) for error in e.errors() if error["loc"])
|
||||
return ()
|
||||
|
||||
|
||||
def _drops_params(litellm_params: GenericLiteLLMParams) -> bool:
|
||||
return litellm_params.drop_params is True or litellm.drop_params is True
|
||||
|
||||
|
||||
def _caller_parameters(parameters: Mapping[str, object], extra_body: object) -> Mapping[str, object]:
|
||||
"""
|
||||
The caller's own ``parameters`` block and ``extra_body`` fields go to DashScope as sent, while other unmapped
|
||||
kwargs stay behind because litellm's own call options (drop_params, a tracing span) arrive the same way.
|
||||
"""
|
||||
extra_fields: Final = (
|
||||
_PARAMETERS_ADAPTER.validate_python(extra_body) if isinstance(extra_body, Mapping) else _EMPTY_PARAMS
|
||||
)
|
||||
return MappingProxyType(
|
||||
{key: value for key, value in (*parameters.items(), *extra_fields.items()) if key not in _NOT_CALLER_PARAMETERS}
|
||||
)
|
||||
|
||||
|
||||
class _MediaEntry(BaseModel, frozen=True):
|
||||
type: str | None = None
|
||||
|
||||
|
||||
_MEDIA_ADAPTER: Final = TypeAdapter(tuple[_MediaEntry, ...])
|
||||
|
||||
|
||||
def _has_input_video(media: object) -> bool:
|
||||
try:
|
||||
entries: Final = _MEDIA_ADAPTER.validate_python(media)
|
||||
except ValidationError:
|
||||
return False
|
||||
return any(entry.type == _INPUT_VIDEO_MEDIA_TYPE for entry in entries)
|
||||
|
||||
|
||||
def _dashscope_model_info(model: str) -> ModelInfo | None:
|
||||
try:
|
||||
return get_model_info(model=model, custom_llm_provider="dashscope")
|
||||
except Exception: # noqa: BLE001 # get_model_info raises bare Exception for unmapped models
|
||||
return None
|
||||
|
||||
|
||||
def _priced_tiers(model: str) -> tuple[str, ...] | None:
|
||||
"""
|
||||
A tier the model has no price for is one DashScope does not offer it at, and litellm would bill it at the 1080P
|
||||
base rate. A model missing from the cost map has nothing to check against.
|
||||
"""
|
||||
info: Final = _dashscope_model_info(model)
|
||||
if info is None:
|
||||
return None
|
||||
return tuple(tier for tier, cost_field in _TIER_COST_FIELDS.items() if info.get(cost_field) is not None)
|
||||
|
||||
|
||||
def _is_priced_tier(resolution: object, priced_tiers: tuple[str, ...]) -> bool:
|
||||
return isinstance(resolution, str) and resolution.strip().upper() in priced_tiers
|
||||
|
||||
|
||||
def _read_all_bytes(file_obj: object) -> bytes:
|
||||
if isinstance(file_obj, tuple):
|
||||
file_tuple: Final = _FILE_TUPLE_ADAPTER.validate_python(file_obj)
|
||||
return _read_all_bytes(file_tuple[1] if len(file_tuple) >= 2 else None)
|
||||
if isinstance(file_obj, PurePath):
|
||||
return Path(file_obj).read_bytes()
|
||||
if isinstance(file_obj, (BytesIO, BufferedReader)):
|
||||
current_position: Final = file_obj.tell()
|
||||
file_obj.seek(0)
|
||||
content: Final = file_obj.read()
|
||||
file_obj.seek(current_position)
|
||||
return content
|
||||
if isinstance(file_obj, bytes):
|
||||
return file_obj
|
||||
if isinstance(file_obj, bytearray):
|
||||
return bytes(file_obj)
|
||||
read: Final = getattr(file_obj, "read", None)
|
||||
if callable(read):
|
||||
data: Final = read()
|
||||
if isinstance(data, bytes):
|
||||
return data
|
||||
raise ValueError("input_reference must be a URL string, bytes, a path, a file object, or a file tuple")
|
||||
|
||||
|
||||
def _image_url(image: object) -> str:
|
||||
if isinstance(image, str):
|
||||
return image
|
||||
content: Final = _read_all_bytes(image)
|
||||
content_type: Final = ImageEditRequestUtils.get_image_content_type(content)
|
||||
encoded: Final = base64.b64encode(content).decode("utf-8")
|
||||
return f"data:{content_type};base64,{encoded}"
|
||||
|
||||
|
||||
def _media_reference_type(model: str) -> str | None:
|
||||
"""Models without the field predate the media array and take the image on the flat ``img_url`` field."""
|
||||
info: Final = _dashscope_model_info(model)
|
||||
if info is None:
|
||||
return None
|
||||
provider_specific: Final = info.get("provider_specific_entry")
|
||||
value: Final = provider_specific.get(_REFERENCE_INPUT_FIELD) if isinstance(provider_specific, Mapping) else None
|
||||
return value if isinstance(value, str) and value in _MEDIA_REFERENCE_TYPES else None
|
||||
|
||||
|
||||
def _resolution_label(usage: _TaskUsage | None, requested_resolution: object) -> str | None:
|
||||
if usage is not None and isinstance(usage.SR, int) and not isinstance(usage.SR, bool):
|
||||
tier: Final = _RESOLUTION_TIERS.get(usage.SR)
|
||||
if tier is not None:
|
||||
return tier
|
||||
if isinstance(requested_resolution, str) and requested_resolution.strip():
|
||||
return requested_resolution.strip().lower()
|
||||
return None
|
||||
|
||||
|
||||
def _video_usage(usage: _TaskUsage | None, requested: Mapping[str, object]) -> Mapping[str, object]:
|
||||
"""
|
||||
``usage.duration`` is DashScope's billed duration, including input video seconds, so it beats the request.
|
||||
"""
|
||||
billed_duration: Final = usage.duration if usage is not None else None
|
||||
requested_duration: Final = requested.get("duration")
|
||||
duration_seconds: Final = (
|
||||
float(billed_duration)
|
||||
if isinstance(billed_duration, (int, float)) and not isinstance(billed_duration, bool)
|
||||
else float(requested_duration)
|
||||
if isinstance(requested_duration, (int, float))
|
||||
and not isinstance(requested_duration, bool)
|
||||
and requested_duration > 0
|
||||
else None
|
||||
)
|
||||
resolution: Final = _resolution_label(usage, requested.get("resolution"))
|
||||
return MappingProxyType(
|
||||
{
|
||||
key: value
|
||||
for key, value in (("duration_seconds", duration_seconds), ("video_resolution", resolution))
|
||||
if value is not None
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _timestamp(value: str | None) -> int | None:
|
||||
"""
|
||||
DashScope stamps times in UTC+8 with no offset.
|
||||
"""
|
||||
if not value:
|
||||
return None
|
||||
try:
|
||||
parsed: Final = datetime.strptime(f"{value}+0800", "%Y-%m-%d %H:%M:%S.%f%z")
|
||||
except ValueError:
|
||||
return None
|
||||
return int(parsed.timestamp())
|
||||
|
||||
|
||||
def _error_block(output: _TaskOutput) -> Mapping[str, object] | None:
|
||||
if not (output.code or output.message):
|
||||
return None
|
||||
return MappingProxyType(
|
||||
{key: value for key, value in (("code", output.code), ("message", output.message)) if value is not None}
|
||||
)
|
||||
|
||||
|
||||
def _size_from_usage(usage: _TaskUsage | None) -> str | None:
|
||||
if usage is None or not isinstance(usage.SR, int) or isinstance(usage.SR, bool) or usage.SR <= 0:
|
||||
return None
|
||||
if not usage.ratio or ":" not in usage.ratio:
|
||||
return None
|
||||
width_str, _, height_str = usage.ratio.partition(":")
|
||||
if not (width_str.isdigit() and height_str.isdigit()):
|
||||
return None
|
||||
ratio_width: Final = int(width_str)
|
||||
ratio_height: Final = int(height_str)
|
||||
if not ratio_width or not ratio_height:
|
||||
return None
|
||||
shortest_ratio_side: Final = min(ratio_width, ratio_height)
|
||||
return f"{usage.SR * ratio_width // shortest_ratio_side}x{usage.SR * ratio_height // shortest_ratio_side}"
|
||||
|
||||
|
||||
def _task_output(task: _TaskResponse, raw_response: httpx.Response) -> _TaskOutput:
|
||||
"""Create and lookup both report failures as code and message with no output, so a missing output
|
||||
raises instead of defaulting into a queued video with an empty id."""
|
||||
if task.output is not None:
|
||||
return task.output
|
||||
raise _video_error(raw_response, 400, task.message or task.code or "DashScope video request failed")
|
||||
|
||||
|
||||
def _video_object_from_task(
|
||||
task: _TaskResponse,
|
||||
raw_response: httpx.Response,
|
||||
model: str | None = None,
|
||||
requested: Mapping[str, object] | None = None,
|
||||
) -> VideoObject:
|
||||
output: Final = _task_output(task, raw_response)
|
||||
if output.task_status == _UNKNOWN_TASK_STATUS:
|
||||
raise _task_not_found(raw_response)
|
||||
status: Final = _STATUS_MAP.get(output.task_status)
|
||||
if status is None: # defaulting it would hand back a job the caller polls forever
|
||||
raise _video_error(raw_response, 502, f"Unknown DashScope task status: {output.task_status}")
|
||||
usage: Final = _video_usage(task.usage, requested or _EMPTY_PARAMS)
|
||||
seconds: Final = usage.get("duration_seconds")
|
||||
error_block: Final = _error_block(output)
|
||||
return VideoObject(
|
||||
id=output.task_id,
|
||||
object="video",
|
||||
status=status,
|
||||
created_at=_timestamp(output.submit_time),
|
||||
completed_at=_timestamp(output.end_time) if status == "completed" else None,
|
||||
error=dict(error_block) if error_block is not None else None,
|
||||
seconds=str(seconds) if seconds is not None else None,
|
||||
size=_size_from_usage(task.usage),
|
||||
model=model,
|
||||
usage=dict(usage) if usage else None,
|
||||
)
|
||||
|
||||
|
||||
def _video_url_from_task(task: _TaskResponse, raw_response: httpx.Response) -> str:
|
||||
"""A download before the video is ready is the caller polling too early, not a fault of the deployment, so it
|
||||
answers 400, which litellm neither retries nor counts toward a deployment cooldown. A 404 would cool the
|
||||
deployment down, and a 409 or a 5xx would be retried against a job that is simply not finished."""
|
||||
output: Final = _task_output(task, raw_response)
|
||||
if output.video_url:
|
||||
return output.video_url
|
||||
|
||||
if output.task_status in _PENDING_STATUSES:
|
||||
raise _video_error(
|
||||
raw_response,
|
||||
400,
|
||||
f"Video is still processing (status: {output.task_status}). Please wait and try again.",
|
||||
)
|
||||
if output.code or output.message:
|
||||
raise _video_error(raw_response, 400, f"Video generation failed: {output.message or output.code}")
|
||||
if output.task_status in _TERMINAL_FAILURE_STATUSES:
|
||||
raise _video_error(raw_response, 400, f"Video generation ended as {output.task_status} with no video URL.")
|
||||
if output.task_status == _UNKNOWN_TASK_STATUS:
|
||||
raise _task_not_found(raw_response)
|
||||
raise _video_error(raw_response, 502, "Video URL not found in task response. The task may not have succeeded yet.")
|
||||
|
||||
|
||||
def _logged_litellm_params(logging_obj: object) -> Mapping[str, object]:
|
||||
litellm_params: Final[object] = getattr(logging_obj, "litellm_params", None)
|
||||
try:
|
||||
return _PARAMETERS_ADAPTER.validate_python(litellm_params)
|
||||
except ValidationError:
|
||||
return _EMPTY_PARAMS
|
||||
|
||||
|
||||
def _polled_models(logging_obj: object) -> tuple[str | None, str | None]:
|
||||
"""
|
||||
The proxy routes by the models encoded in the polled id, so they must survive into the returned id. The served
|
||||
group survives only when the router sent this poll to it, so an edited id cannot write any name it likes back.
|
||||
"""
|
||||
video_id: Final = _logged_litellm_params(logging_obj).get("video_id")
|
||||
if not isinstance(video_id, str):
|
||||
return None, None
|
||||
decoded: Final = decode_video_id_with_provider(video_id)
|
||||
served: Final = decoded.get("served_model")
|
||||
routed: Final = _poll_models(logging_obj)
|
||||
return decoded.get("model_id") or None, served if routed is None or served in routed else None
|
||||
|
||||
|
||||
def _poll_models(logging_obj: object) -> tuple[str, str | None] | None:
|
||||
"""
|
||||
A later status or content request checks the key's model access against the model encoded in the id, and
|
||||
routes by it. DashScope serves the same model name from several regional hosts, so only the group that
|
||||
served the task can send its polls back to the region that created it, while a key may be scoped to the
|
||||
alias, routing group or wildcard name it asked for instead. The id carries both names: the one the router
|
||||
was asked for on this hop, and the group that served it when that differs and is not a wildcard. The router
|
||||
stamps both into litellm_metadata, and metadata carries a copy only when the caller sent none.
|
||||
"""
|
||||
params: Final = _logged_litellm_params(logging_obj)
|
||||
stamped: Final = (_stamped_poll_models(params.get(key)) for key in ("litellm_metadata", "metadata"))
|
||||
return next((models for models in stamped if models), None)
|
||||
|
||||
|
||||
def _stamped_poll_models(metadata: object) -> tuple[str, str | None] | None:
|
||||
try:
|
||||
fields: Final = _PARAMETERS_ADAPTER.validate_python(metadata)
|
||||
except ValidationError:
|
||||
return None
|
||||
requested: Final = _id_safe_name(fields.get("model_group"))
|
||||
served: Final = _id_safe_name(fields.get("deployment_model_name"))
|
||||
if requested is None:
|
||||
return None
|
||||
return requested, None if served is None or served == requested or "*" in served else served
|
||||
|
||||
|
||||
def _id_safe_name(name: object) -> str | None:
|
||||
return name if isinstance(name, str) and name and ";" not in name else None
|
||||
|
||||
|
||||
class DashScopeVideoConfig(BaseVideoConfig):
|
||||
def get_supported_openai_params(self, model: str) -> list[str]: # mutable-ok: BaseVideoConfig contract returns list
|
||||
return [
|
||||
"model",
|
||||
"prompt",
|
||||
"input_reference",
|
||||
"seconds",
|
||||
"size",
|
||||
"user",
|
||||
"extra_headers",
|
||||
"media",
|
||||
"resolution",
|
||||
"ratio",
|
||||
"duration",
|
||||
"audio",
|
||||
"seed",
|
||||
"prompt_extend",
|
||||
"watermark",
|
||||
"negative_prompt",
|
||||
"parameters",
|
||||
*_LEGACY_INPUT_KEYS,
|
||||
]
|
||||
|
||||
def map_openai_params(
|
||||
self,
|
||||
video_create_optional_params: VideoCreateOptionalRequestParams,
|
||||
model: str,
|
||||
drop_params: bool,
|
||||
) -> dict[str, object]: # mutable-ok: BaseVideoConfig contract
|
||||
unsupported: Final = next(
|
||||
(
|
||||
key
|
||||
for key, value in video_create_optional_params.items()
|
||||
if key in _UNSUPPORTED_BY_DASHSCOPE and value is not None
|
||||
),
|
||||
None,
|
||||
)
|
||||
if unsupported is not None and not drop_params:
|
||||
raise UnsupportedParamsError(
|
||||
message=f"'{unsupported}' is not supported by DashScope video generation, set drop_params=True to ignore it",
|
||||
model=model,
|
||||
llm_provider="dashscope",
|
||||
)
|
||||
parameters: Final = _parameters_fields(video_create_optional_params.get("parameters"), model, drop_params)
|
||||
caller_parameters: Final = _caller_parameters(parameters, video_create_optional_params.get("extra_body"))
|
||||
passthrough: Final = MappingProxyType(
|
||||
{
|
||||
key: value
|
||||
for key, value in video_create_optional_params.items()
|
||||
if value is not None and key not in _MAPPED_ELSEWHERE
|
||||
}
|
||||
)
|
||||
mapped_params: Final[dict[str, object]] = { # mutable-ok: BaseVideoConfig contract; extra_body merges into it
|
||||
**_size_fields(video_create_optional_params.get("size"), model, drop_params),
|
||||
**_duration_fields(video_create_optional_params.get("seconds"), model, drop_params),
|
||||
**passthrough,
|
||||
**parameters,
|
||||
}
|
||||
if caller_parameters:
|
||||
mapped_params[_CALLER_PARAMETERS_FIELD] = dict(caller_parameters)
|
||||
return mapped_params
|
||||
|
||||
def _resolve_api_key(self, api_key: str | None) -> str:
|
||||
resolved_api_key: Final = api_key or get_secret_str("DASHSCOPE_API_KEY")
|
||||
if resolved_api_key is None:
|
||||
raise ValueError(
|
||||
"DashScope API key is required. Set DASHSCOPE_API_KEY environment variable or pass api_key parameter."
|
||||
)
|
||||
return resolved_api_key
|
||||
|
||||
def _resolve_video_api_base(self, video_api_base: str | None) -> str:
|
||||
return (
|
||||
video_api_base
|
||||
or get_secret_str("DASHSCOPE_API_BASE_VIDEO")
|
||||
or get_secret_str("DASHSCOPE_API_BASE")
|
||||
or DASHSCOPE_VIDEO_DEFAULT_API_BASE
|
||||
)
|
||||
|
||||
def validate_environment(
|
||||
self,
|
||||
headers: dict[str, str], # mutable-ok: BaseVideoConfig contract; handler expects a mutable headers dict
|
||||
model: str,
|
||||
api_key: str | None = None,
|
||||
litellm_params: GenericLiteLLMParams | None = None,
|
||||
) -> dict[str, str]: # mutable-ok: BaseVideoConfig contract
|
||||
resolved_api_key: Final = self._resolve_api_key(
|
||||
api_key
|
||||
or (litellm_params.api_key if litellm_params is not None and litellm_params.api_key else None)
|
||||
or litellm.api_key
|
||||
)
|
||||
|
||||
auth_headers: Final[dict[str, str]] = { # mutable-ok: httpx request headers are a mutable dict
|
||||
"Authorization": f"Bearer {resolved_api_key}",
|
||||
"Content-Type": "application/json",
|
||||
"X-DashScope-Async": "enable",
|
||||
}
|
||||
headers.update(auth_headers)
|
||||
return headers
|
||||
|
||||
def get_complete_url(
|
||||
self,
|
||||
model: str,
|
||||
api_base: str | None,
|
||||
litellm_params: dict[str, object], # mutable-ok: BaseVideoConfig contract
|
||||
) -> str:
|
||||
return _normalized_api_base(self._resolve_video_api_base(api_base))
|
||||
|
||||
def transform_video_create_request(
|
||||
self,
|
||||
model: str,
|
||||
prompt: str,
|
||||
api_base: str,
|
||||
video_create_optional_request_params: dict[str, object], # mutable-ok: BaseVideoConfig contract
|
||||
litellm_params: GenericLiteLLMParams,
|
||||
headers: dict[str, str], # mutable-ok: BaseVideoConfig contract
|
||||
) -> tuple[dict[str, object], RequestFiles, str]: # mutable-ok: BaseVideoConfig contract
|
||||
reference_field, reference_value = self._reference_input(model, video_create_optional_request_params)
|
||||
passthrough_input: Final = tuple(
|
||||
(key, video_create_optional_request_params[key])
|
||||
for key in _INPUT_KEYS
|
||||
if video_create_optional_request_params.get(key) is not None
|
||||
)
|
||||
reference_entry: Final = (
|
||||
((reference_field, reference_value),)
|
||||
if reference_field is not None and reference_field not in frozenset(key for key, _ in passthrough_input)
|
||||
else ()
|
||||
)
|
||||
request_input: Final[dict[str, object]] = dict( # mutable-ok: request body dict, JSON-serialized by the handler
|
||||
(("prompt", prompt), *passthrough_input, *reference_entry)
|
||||
)
|
||||
|
||||
requested_parameters: Final = MappingProxyType(
|
||||
{
|
||||
**{
|
||||
key: video_create_optional_request_params[key]
|
||||
for key in _PARAMETER_KEYS
|
||||
if video_create_optional_request_params.get(key) is not None
|
||||
},
|
||||
**(_object_fields(video_create_optional_request_params.get(_CALLER_PARAMETERS_FIELD)) or _EMPTY_PARAMS),
|
||||
}
|
||||
)
|
||||
if requested_parameters.get("duration") == DASHSCOPE_SMART_DURATION:
|
||||
raise UnsupportedParamsError(
|
||||
message=(
|
||||
"Smart duration (duration=-1) is not supported through litellm: the video is billed when the task "
|
||||
"is created, before DashScope has picked its length. Pass an explicit duration in seconds."
|
||||
),
|
||||
model=model,
|
||||
llm_provider="dashscope",
|
||||
)
|
||||
if _has_input_video(request_input.get("media")):
|
||||
raise UnsupportedParamsError(
|
||||
message=(
|
||||
"Video input (a reference_video media entry) is not supported through litellm: DashScope bills "
|
||||
"the input video's seconds on top of the output, and the video is billed when the task is "
|
||||
"created, before that length is known."
|
||||
),
|
||||
model=model,
|
||||
llm_provider="dashscope",
|
||||
)
|
||||
drop_params: Final = _drops_params(litellm_params)
|
||||
malformed: Final = _malformed_parameters(requested_parameters)
|
||||
if malformed and not drop_params:
|
||||
raise _unsupported_value(model, "; ".join(f"invalid {field}: {message}" for field, message in malformed))
|
||||
resolution: Final = requested_parameters.get("resolution")
|
||||
priced_tiers: Final = _priced_tiers(model) if resolution is not None else None
|
||||
drops_resolution: Final = priced_tiers is not None and not _is_priced_tier(resolution, priced_tiers)
|
||||
if drops_resolution and not drop_params:
|
||||
raise _unsupported_value(
|
||||
model,
|
||||
f"{model} is not offered at resolution {resolution!r}, only at {', '.join(priced_tiers or ())}",
|
||||
)
|
||||
dropped: Final = frozenset({*(field for field, _ in malformed), *(("resolution",) if drops_resolution else ())})
|
||||
parameters: Final[dict[str, object]] = { # mutable-ok: request body dict, JSON-serialized by the handler
|
||||
key: value for key, value in requested_parameters.items() if key not in dropped
|
||||
}
|
||||
|
||||
request_data: Final[dict[str, object]] = { # mutable-ok: request body dict, JSON-serialized by the handler
|
||||
"model": model,
|
||||
"input": request_input,
|
||||
}
|
||||
if parameters:
|
||||
request_data["parameters"] = parameters
|
||||
|
||||
return request_data, (), f"{api_base}{DASHSCOPE_VIDEO_SYNTHESIS_PATH}"
|
||||
|
||||
@staticmethod
|
||||
def _reference_input(
|
||||
model: str,
|
||||
video_create_optional_request_params: Mapping[str, object],
|
||||
) -> tuple[str | None, object]:
|
||||
input_reference: Final = video_create_optional_request_params.get("input_reference")
|
||||
if input_reference is None:
|
||||
return None, None
|
||||
image_url: Final = _image_url(input_reference)
|
||||
media_type: Final = _media_reference_type(model)
|
||||
if media_type is None:
|
||||
return "img_url", image_url
|
||||
# a plain dict because json.dumps cannot serialize a MappingProxyType
|
||||
return "media", (dict[str, object](type=media_type, url=image_url),)
|
||||
|
||||
def transform_video_create_response(
|
||||
self,
|
||||
model: str,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: "Logging",
|
||||
custom_llm_provider: str | None = None,
|
||||
request_data: dict[str, object] | None = None, # mutable-ok: BaseVideoConfig contract
|
||||
) -> VideoObject:
|
||||
task: Final = _parse_task_response(raw_response)
|
||||
requested: Final = self._requested_parameters(request_data)
|
||||
video_obj: Final = _video_object_from_task(task, raw_response, model=model, requested=requested)
|
||||
if custom_llm_provider and video_obj.id:
|
||||
requested_model, served_model = _poll_models(logging_obj) or (model, None)
|
||||
video_obj.id = encode_video_id_with_provider(
|
||||
video_obj.id, custom_llm_provider, requested_model, served_model
|
||||
)
|
||||
return video_obj
|
||||
|
||||
@staticmethod
|
||||
def _requested_parameters(request_data: Mapping[str, object] | None) -> Mapping[str, object]:
|
||||
"""
|
||||
The create call is the only billed one, so an omitted duration or tier bills DashScope's defaults, not zero.
|
||||
"""
|
||||
parameters: Final = (request_data or _EMPTY_PARAMS).get("parameters")
|
||||
requested: Final = (
|
||||
_PARAMETERS_ADAPTER.validate_python(parameters) if isinstance(parameters, Mapping) else _EMPTY_PARAMS
|
||||
)
|
||||
return MappingProxyType(
|
||||
{
|
||||
"duration": DASHSCOPE_DEFAULT_DURATION_SECONDS,
|
||||
"resolution": DASHSCOPE_DEFAULT_RESOLUTION,
|
||||
**requested,
|
||||
}
|
||||
)
|
||||
|
||||
def transform_video_status_retrieve_request(
|
||||
self,
|
||||
video_id: str,
|
||||
api_base: str,
|
||||
litellm_params: GenericLiteLLMParams,
|
||||
headers: dict[str, str], # mutable-ok: BaseVideoConfig contract
|
||||
) -> tuple[str, dict[str, object]]: # mutable-ok: BaseVideoConfig contract
|
||||
return self._task_url(video_id, api_base), _EMPTY_PARAMS
|
||||
|
||||
@staticmethod
|
||||
def _task_url(video_id: str, api_base: str) -> str:
|
||||
original_task_id: Final = extract_original_video_id(video_id)
|
||||
encoded_task_id: Final = encode_url_path_segment(original_task_id, field_name="video_id")
|
||||
return f"{api_base}{DASHSCOPE_TASKS_PATH}/{encoded_task_id}"
|
||||
|
||||
def transform_video_status_retrieve_response(
|
||||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: "Logging",
|
||||
custom_llm_provider: str | None = None,
|
||||
client: "HTTPHandler | None" = None,
|
||||
) -> VideoObject:
|
||||
task: Final = _parse_task_response(raw_response)
|
||||
model, served_model = _polled_models(logging_obj)
|
||||
video_obj: Final = _video_object_from_task(task, raw_response, model=model)
|
||||
if custom_llm_provider and video_obj.id:
|
||||
video_obj.id = encode_video_id_with_provider(video_obj.id, custom_llm_provider, model, served_model)
|
||||
return video_obj
|
||||
|
||||
def transform_video_content_request(
|
||||
self,
|
||||
video_id: str,
|
||||
api_base: str,
|
||||
litellm_params: GenericLiteLLMParams,
|
||||
headers: dict[str, str], # mutable-ok: BaseVideoConfig contract
|
||||
variant: str | None = None,
|
||||
) -> tuple[str, dict[str, object]]: # mutable-ok: BaseVideoConfig contract
|
||||
return self._task_url(video_id, api_base), _EMPTY_PARAMS
|
||||
|
||||
def transform_video_content_response(
|
||||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: "Logging",
|
||||
client: "HTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
video_url: Final = _video_url_from_task(_parse_task_response(raw_response), raw_response)
|
||||
|
||||
httpx_client: Final[HTTPHandler] = client if client is not None else _get_httpx_client()
|
||||
video_response: Final = httpx_client.get(video_url) # pyright: ignore[reportUnknownMemberType] # HTTPHandler.get stub leaves params/headers untyped
|
||||
video_response.raise_for_status()
|
||||
|
||||
return video_response.content
|
||||
|
||||
async def async_transform_video_content_response(
|
||||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: "Logging",
|
||||
client: "AsyncHTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
video_url: Final = _video_url_from_task(_parse_task_response(raw_response), raw_response)
|
||||
|
||||
async_httpx_client: Final[AsyncHTTPHandler] = (
|
||||
client
|
||||
if client is not None
|
||||
else get_async_httpx_client(
|
||||
llm_provider=litellm.LlmProviders.DASHSCOPE,
|
||||
)
|
||||
)
|
||||
video_response: Final = await async_httpx_client.get(video_url) # pyright: ignore[reportUnknownMemberType] # HTTPHandler.get stub leaves params/headers untyped
|
||||
video_response.raise_for_status()
|
||||
|
||||
return video_response.content
|
||||
|
||||
def transform_video_remix_request(
|
||||
self,
|
||||
video_id: str,
|
||||
prompt: str,
|
||||
api_base: str,
|
||||
litellm_params: GenericLiteLLMParams,
|
||||
headers: dict[str, str], # mutable-ok: BaseVideoConfig contract
|
||||
extra_body: dict[str, object] | None = None, # mutable-ok: BaseVideoConfig contract
|
||||
) -> tuple[str, dict[str, object]]: # mutable-ok: BaseVideoConfig contract
|
||||
raise NotImplementedError("Video remix is not supported by DashScope.")
|
||||
|
||||
def transform_video_remix_response(
|
||||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: "Logging",
|
||||
custom_llm_provider: str | None = None,
|
||||
) -> VideoObject:
|
||||
raise NotImplementedError("Video remix is not supported by DashScope.")
|
||||
|
||||
def transform_video_list_request(
|
||||
self,
|
||||
api_base: str,
|
||||
litellm_params: GenericLiteLLMParams,
|
||||
headers: dict[str, str], # mutable-ok: BaseVideoConfig contract
|
||||
after: str | None = None,
|
||||
limit: int | None = None,
|
||||
order: str | None = None,
|
||||
extra_query: dict[str, object] | None = None, # mutable-ok: BaseVideoConfig contract
|
||||
) -> tuple[str, dict[str, object]]: # mutable-ok: BaseVideoConfig contract
|
||||
raise NotImplementedError(
|
||||
"Video list is not supported by DashScope. Retrieve tasks individually by task id within their 24 hour "
|
||||
"retention window."
|
||||
)
|
||||
|
||||
def transform_video_list_response(
|
||||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: "Logging",
|
||||
custom_llm_provider: str | None = None,
|
||||
) -> dict[str, str]: # mutable-ok: BaseVideoConfig contract
|
||||
raise NotImplementedError("Video list is not supported by DashScope.")
|
||||
|
||||
def transform_video_delete_request(
|
||||
self,
|
||||
video_id: str,
|
||||
api_base: str,
|
||||
litellm_params: GenericLiteLLMParams,
|
||||
headers: dict[str, str], # mutable-ok: BaseVideoConfig contract
|
||||
) -> tuple[str, dict[str, object]]: # mutable-ok: BaseVideoConfig contract
|
||||
raise NotImplementedError(
|
||||
"Video delete is not supported by DashScope. Tasks and their artifacts expire 24 hours after creation."
|
||||
)
|
||||
|
||||
def transform_video_delete_response(
|
||||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: "Logging",
|
||||
) -> VideoObject:
|
||||
raise NotImplementedError("Video delete is not supported by DashScope.")
|
||||
|
||||
def get_error_class(
|
||||
self,
|
||||
error_message: str,
|
||||
status_code: int,
|
||||
headers: dict[str, str] | httpx.Headers, # mutable-ok: BaseVideoConfig contract
|
||||
) -> BaseLLMException:
|
||||
return DashScopeVideoError(
|
||||
status_code=status_code,
|
||||
message=error_message,
|
||||
headers=headers,
|
||||
)
|
||||
|
|
@ -124,6 +124,7 @@ class EdenAIVideoConfig(OpenAIVideoConfig):
|
|||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: "LiteLLMLoggingObj",
|
||||
client: "HTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
raw_response.raise_for_status() # the shared GET helpers return error bodies instead of raising
|
||||
return raw_response.content
|
||||
|
|
|
|||
|
|
@ -562,7 +562,12 @@ class FalAIVideoConfig(BaseVideoConfig):
|
|||
raise ValueError(f"fal.ai video result did not include a video URL: {error_message}")
|
||||
raise ValueError("fal.ai video result did not include a video URL")
|
||||
|
||||
def transform_video_content_response(self, raw_response: httpx.Response, logging_obj: object) -> bytes:
|
||||
def transform_video_content_response(
|
||||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: object,
|
||||
client: "HTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
error: Final[str | None] = _result_error(raw_response)
|
||||
if error is not None:
|
||||
raise FalAIVideoError(
|
||||
|
|
@ -580,7 +585,12 @@ class FalAIVideoConfig(BaseVideoConfig):
|
|||
video_response.raise_for_status()
|
||||
return video_response.content
|
||||
|
||||
async def async_transform_video_content_response(self, raw_response: httpx.Response, logging_obj: object) -> bytes:
|
||||
async def async_transform_video_content_response(
|
||||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: object,
|
||||
client: "AsyncHTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
error: Final[str | None] = _result_error(raw_response)
|
||||
if error is not None:
|
||||
raise FalAIVideoError(
|
||||
|
|
|
|||
|
|
@ -478,6 +478,7 @@ class GeminiVideoConfig(BaseVideoConfig):
|
|||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: LiteLLMLoggingObj,
|
||||
client: "HTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
"""
|
||||
Transform the Veo video content download response.
|
||||
|
|
|
|||
|
|
@ -265,6 +265,7 @@ class OpenAIVideoConfig(BaseVideoConfig):
|
|||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: LiteLLMLoggingObj,
|
||||
client: "HTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
"""Transform the OpenAI video content download response."""
|
||||
return raw_response.content
|
||||
|
|
|
|||
|
|
@ -445,6 +445,7 @@ class RunwayMLVideoConfig(BaseVideoConfig):
|
|||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: LiteLLMLoggingObj,
|
||||
client: "HTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
"""
|
||||
Transform the RunwayML video content download response (synchronous).
|
||||
|
|
@ -474,6 +475,7 @@ class RunwayMLVideoConfig(BaseVideoConfig):
|
|||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: LiteLLMLoggingObj,
|
||||
client: "AsyncHTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
"""
|
||||
Transform the RunwayML video content download response (asynchronous).
|
||||
|
|
|
|||
|
|
@ -585,6 +585,7 @@ class VertexAIVideoConfig(BaseVideoConfig, VertexBase):
|
|||
self,
|
||||
raw_response: httpx.Response,
|
||||
logging_obj: LiteLLMLoggingObj,
|
||||
client: "HTTPHandler | None" = None,
|
||||
) -> bytes:
|
||||
"""
|
||||
Transform the Veo video content download response.
|
||||
|
|
|
|||
|
|
@ -17874,6 +17874,281 @@
|
|||
"/v1/images/generations"
|
||||
]
|
||||
},
|
||||
"dashscope/wan3.0-video": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video",
|
||||
"output_cost_per_second": 0.165025,
|
||||
"output_cost_per_second_480p": 0.041256,
|
||||
"output_cost_per_second_720p": 0.082513,
|
||||
"output_cost_per_second_1080p": 0.165025,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 all-in-one video generation (text, first/last frame, multimodal reference, edit, extend). Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/wan3.0-video-prime": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video-prime",
|
||||
"output_cost_per_second": 0.254399,
|
||||
"output_cost_per_second_480p": 0.0636,
|
||||
"output_cost_per_second_720p": 0.127199,
|
||||
"output_cost_per_second_1080p": 0.254399,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 fast tier. Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.1-t2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-t2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 text-to-video. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.1-i2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-i2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 image-to-video from a first frame. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.1-r2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-r2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 reference-to-video, 1 to 9 reference images. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.0-t2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-t2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 text-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.0-i2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-i2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 image-to-video from a first frame. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.0-r2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-r2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 reference-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/wan2.7-t2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-t2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 text-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/wan2.7-i2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-i2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 image-to-video (first frame, first and last frame, continuation). No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/wan2.7-r2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-r2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 reference-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/deepseek-v4-flash": {
|
||||
"cache_read_input_token_cost": 4e-08,
|
||||
"input_cost_per_token": 2e-07,
|
||||
|
|
@ -18826,6 +19101,281 @@
|
|||
"/v1/images/generations"
|
||||
]
|
||||
},
|
||||
"qwencloud/wan3.0-video": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video",
|
||||
"output_cost_per_second": 0.2,
|
||||
"output_cost_per_second_480p": 0.05,
|
||||
"output_cost_per_second_720p": 0.1,
|
||||
"output_cost_per_second_1080p": 0.2,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 all-in-one video generation (text, first/last frame, multimodal reference, edit, extend). Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/wan3.0-video-prime": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video-prime",
|
||||
"output_cost_per_second": 0.28,
|
||||
"output_cost_per_second_480p": 0.068,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.28,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 fast tier. Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.1-t2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-t2v",
|
||||
"output_cost_per_second": 0.18,
|
||||
"output_cost_per_second_480p": 0.07,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.18,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 text-to-video. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.1-i2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-i2v",
|
||||
"output_cost_per_second": 0.18,
|
||||
"output_cost_per_second_480p": 0.07,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.18,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 image-to-video from a first frame. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.1-r2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-r2v",
|
||||
"output_cost_per_second": 0.18,
|
||||
"output_cost_per_second_480p": 0.07,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.18,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 reference-to-video, 1 to 9 reference images. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.0-t2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-t2v",
|
||||
"output_cost_per_second": 0.24,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.24,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 text-to-video. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.0-i2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-i2v",
|
||||
"output_cost_per_second": 0.24,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.24,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 image-to-video from a first frame. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.0-r2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-r2v",
|
||||
"output_cost_per_second": 0.24,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.24,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 reference-to-video. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/wan2.7-t2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-t2v",
|
||||
"output_cost_per_second": 0.15,
|
||||
"output_cost_per_second_720p": 0.1,
|
||||
"output_cost_per_second_1080p": 0.15,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 text-to-video. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/wan2.7-i2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-i2v",
|
||||
"output_cost_per_second": 0.15,
|
||||
"output_cost_per_second_720p": 0.1,
|
||||
"output_cost_per_second_1080p": 0.15,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 image-to-video (first frame, first and last frame, continuation). No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/wan2.7-r2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-r2v",
|
||||
"output_cost_per_second": 0.15,
|
||||
"output_cost_per_second_720p": 0.1,
|
||||
"output_cost_per_second_1080p": 0.15,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 reference-to-video. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/deepseek-v4-flash": {
|
||||
"cache_read_input_token_cost": 4e-08,
|
||||
"input_cost_per_token": 2e-07,
|
||||
|
|
@ -19818,6 +20368,281 @@
|
|||
"/v1/images/generations"
|
||||
]
|
||||
},
|
||||
"qwen_ai_platform/wan3.0-video": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video",
|
||||
"output_cost_per_second": 0.165025,
|
||||
"output_cost_per_second_480p": 0.041256,
|
||||
"output_cost_per_second_720p": 0.082513,
|
||||
"output_cost_per_second_1080p": 0.165025,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 all-in-one video generation (text, first/last frame, multimodal reference, edit, extend). Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/wan3.0-video-prime": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video-prime",
|
||||
"output_cost_per_second": 0.254399,
|
||||
"output_cost_per_second_480p": 0.0636,
|
||||
"output_cost_per_second_720p": 0.127199,
|
||||
"output_cost_per_second_1080p": 0.254399,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 fast tier. Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.1-t2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-t2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 text-to-video. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.1-i2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-i2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 image-to-video from a first frame. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.1-r2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-r2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 reference-to-video, 1 to 9 reference images. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.0-t2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-t2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 text-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.0-i2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-i2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 image-to-video from a first frame. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.0-r2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-r2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 reference-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/wan2.7-t2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-t2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 text-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/wan2.7-i2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-i2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 image-to-video (first frame, first and last frame, continuation). No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/wan2.7-r2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-r2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 reference-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"databricks/databricks-bge-large-en": {
|
||||
"cache_creation_input_token_cost": 1.0003e-07,
|
||||
"cache_read_input_token_cost": 1.0003e-07,
|
||||
|
|
|
|||
|
|
@ -1808,8 +1808,9 @@ async def _team_member_within_fallback_model_budget(
|
|||
|
||||
|
||||
class ProxyBaseLLMRequestProcessing:
|
||||
def __init__(self, data: dict):
|
||||
def __init__(self, data: dict, served_model: str | None = None):
|
||||
self.data = data
|
||||
self.served_model = served_model
|
||||
self._tags_before_guardrails: frozenset[str] | None = None
|
||||
|
||||
@property
|
||||
|
|
@ -2263,6 +2264,8 @@ class ProxyBaseLLMRequestProcessing:
|
|||
if alias_target is not None:
|
||||
self.data["model"] = alias_target
|
||||
|
||||
aliased_model: Final = self.data.get("model")
|
||||
|
||||
litellm_call_id: Final = resolve_litellm_call_id(request.headers.get("x-litellm-call-id"))
|
||||
self.data["litellm_call_id"] = litellm_call_id
|
||||
bind_telemetry_call_id(litellm_call_id)
|
||||
|
|
@ -2361,6 +2364,14 @@ class ProxyBaseLLMRequestProcessing:
|
|||
if "messages" in self.data and self.data["messages"]:
|
||||
logging_obj.update_messages(self.data["messages"])
|
||||
|
||||
if (
|
||||
self.served_model is not None
|
||||
and llm_router is not None
|
||||
and isinstance(aliased_model, str)
|
||||
and self.data.get("model") == aliased_model
|
||||
):
|
||||
self.data["model"] = llm_router.resolve_served_model_name(aliased_model, self.served_model)
|
||||
|
||||
return self.data, logging_obj
|
||||
|
||||
async def _pre_call_with_fallbacks(
|
||||
|
|
|
|||
|
|
@ -272,12 +272,12 @@ async def video_status(
|
|||
# Resolve model_name from model_id if available
|
||||
# This allows the router to automatically inject litellm_params from the model config
|
||||
if model_id_from_decoded and llm_router:
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded)
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded, provider_from_id)
|
||||
if resolved_model:
|
||||
data["model"] = resolved_model
|
||||
|
||||
# Process request using ProxyBaseLLMRequestProcessing
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data)
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data, served_model=decoded.get("served_model"))
|
||||
try:
|
||||
status: Final[object] = await processor.base_process_llm_request(
|
||||
request=request,
|
||||
|
|
@ -372,11 +372,11 @@ async def video_content(
|
|||
# Resolve model_name from model_id if available
|
||||
# This allows the router to automatically inject litellm_params from the model config
|
||||
if model_id_from_decoded and llm_router:
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded)
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded, provider_from_id)
|
||||
if resolved_model:
|
||||
data["model"] = resolved_model
|
||||
# Process request using ProxyBaseLLMRequestProcessing
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data)
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data, served_model=decoded.get("served_model"))
|
||||
try:
|
||||
# Call the video content function directly to get raw bytes
|
||||
video_bytes: Final = await processor.base_process_llm_request(
|
||||
|
|
@ -480,12 +480,12 @@ async def video_remix(
|
|||
# Resolve model_name from model_id if available
|
||||
# This allows the router to automatically inject litellm_params from the model config
|
||||
if model_id_from_decoded and llm_router:
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded)
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded, provider_from_id)
|
||||
if resolved_model:
|
||||
data["model"] = resolved_model
|
||||
|
||||
# Process request using ProxyBaseLLMRequestProcessing
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data)
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data, served_model=decoded.get("served_model"))
|
||||
try:
|
||||
remixed: Final[object] = await processor.base_process_llm_request(
|
||||
request=request,
|
||||
|
|
@ -683,7 +683,7 @@ async def video_get_character(
|
|||
data["custom_llm_provider"] = custom_llm_provider
|
||||
|
||||
if model_id_from_decoded and llm_router:
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded)
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded, provider_from_id)
|
||||
if resolved_model:
|
||||
data["model"] = resolved_model
|
||||
|
||||
|
|
@ -794,11 +794,11 @@ async def video_edit(
|
|||
data["custom_llm_provider"] = custom_llm_provider
|
||||
|
||||
if model_id_from_decoded and llm_router:
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded)
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded, provider_from_id)
|
||||
if resolved_model:
|
||||
data["model"] = resolved_model
|
||||
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data)
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data, served_model=decoded.get("served_model"))
|
||||
try:
|
||||
edited: Final[object] = await processor.base_process_llm_request(
|
||||
request=request,
|
||||
|
|
@ -891,11 +891,11 @@ async def video_extension(
|
|||
data["custom_llm_provider"] = custom_llm_provider
|
||||
|
||||
if model_id_from_decoded and llm_router:
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded)
|
||||
resolved_model: Final = llm_router.resolve_model_name_from_model_id(model_id_from_decoded, provider_from_id)
|
||||
if resolved_model:
|
||||
data["model"] = resolved_model
|
||||
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data)
|
||||
processor: Final = ProxyBaseLLMRequestProcessing(data=data, served_model=decoded.get("served_model"))
|
||||
try:
|
||||
extended: Final[object] = await processor.base_process_llm_request(
|
||||
request=request,
|
||||
|
|
|
|||
|
|
@ -901,7 +901,7 @@ class Router:
|
|||
def _routing_groups(self, value: dict[str, RoutingGroup]) -> None:
|
||||
self.routing_groups = value
|
||||
|
||||
model_names: set = set()
|
||||
model_names: set[str] = set()
|
||||
cache_responses: bool | None = False
|
||||
default_cache_time_seconds: int = 1 * 60 * 60 # 1 hour
|
||||
tenacity = None
|
||||
|
|
@ -12001,7 +12001,9 @@ class Router:
|
|||
"""
|
||||
return candidate_id in self.model_id_to_deployment_index_map
|
||||
|
||||
def resolve_model_name_from_model_id(self, model_id: str | None) -> str | None:
|
||||
def resolve_model_name_from_model_id(
|
||||
self, model_id: str | None, custom_llm_provider: str | None = None
|
||||
) -> str | None:
|
||||
"""
|
||||
Resolve model_name from model_id.
|
||||
|
||||
|
|
@ -12011,11 +12013,14 @@ class Router:
|
|||
Strategy:
|
||||
1. First, check if model_id directly matches a model_name or deployment ID
|
||||
2. If not, search through router's model_list to find a match by litellm_params.model
|
||||
3. Return the model_name if found, None otherwise
|
||||
3. If not, keep model_id when the router routes that name itself (an alias, a routing group, or a wildcard
|
||||
whose deployment serves `custom_llm_provider`)
|
||||
4. Return the model_name if found, None otherwise
|
||||
|
||||
Args:
|
||||
model_id: The model_id extracted from decoded video_id
|
||||
(could be model_name or litellm_params.model value)
|
||||
custom_llm_provider: The provider extracted from the same id
|
||||
|
||||
Returns:
|
||||
model_name if found, None otherwise. If None, the request will fall through
|
||||
|
|
@ -12055,9 +12060,22 @@ class Router:
|
|||
if model_name:
|
||||
return model_name
|
||||
|
||||
# No match found
|
||||
if self.is_recognized_model(model_id):
|
||||
return model_id
|
||||
if custom_llm_provider is not None and self.pattern_router.routes_to_provider(model_id, custom_llm_provider):
|
||||
return model_id
|
||||
return None
|
||||
|
||||
def resolve_served_model_name(self, model_name: str, served_model: str) -> str:
|
||||
"""
|
||||
`served_model` when `model_name` routes to it (the alias target, the routing group member, or the name
|
||||
itself), so a request can return to the group that served an earlier one, else `model_name`.
|
||||
"""
|
||||
target: Final = self.get_model_from_alias(model_name) or model_name
|
||||
routing_group: Final = self.get_routing_group(target)
|
||||
members: Final = routing_group.models if routing_group is not None else ()
|
||||
return served_model if served_model == target or served_model in members else model_name
|
||||
|
||||
def map_team_model(self, team_model_name: str | None, team_id: str) -> str | None:
|
||||
"""
|
||||
Check if team_model_name resolves to team-specific deployments.
|
||||
|
|
|
|||
|
|
@ -7,8 +7,20 @@ import re
|
|||
from re import Match
|
||||
from typing import Final
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, TypeAdapter
|
||||
|
||||
from litellm._logging import verbose_router_logger
|
||||
from litellm.litellm_core_utils.get_llm_provider_logic import inferred_provider
|
||||
from litellm.types.router import LiteLLM_Params
|
||||
|
||||
|
||||
class _RoutedDeployment(BaseModel):
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
litellm_params: LiteLLM_Params
|
||||
|
||||
|
||||
_ROUTED_DEPLOYMENTS: Final = TypeAdapter(tuple[_RoutedDeployment, ...])
|
||||
|
||||
|
||||
class PatternUtils:
|
||||
|
|
@ -160,6 +172,14 @@ class PatternMatchRouter:
|
|||
|
||||
return None # No matching pattern found
|
||||
|
||||
def routes_to_provider(self, request: str, custom_llm_provider: str) -> bool:
|
||||
deployments: Final = _ROUTED_DEPLOYMENTS.validate_python(self.route(request) or ())
|
||||
return any(
|
||||
(deployment.litellm_params.custom_llm_provider or inferred_provider(deployment.litellm_params.model))
|
||||
== custom_llm_provider
|
||||
for deployment in deployments
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def set_deployment_model_name(
|
||||
matched_pattern: Match,
|
||||
|
|
|
|||
|
|
@ -112,6 +112,7 @@ class DecodedVideoId(TypedDict, total=False):
|
|||
custom_llm_provider: str | None
|
||||
model_id: str | None
|
||||
video_id: str
|
||||
served_model: ReadOnly[str | None]
|
||||
|
||||
|
||||
class CharacterObject(LiteLLMBaseModel):
|
||||
|
|
|
|||
|
|
@ -13,6 +13,7 @@ from litellm.types.utils import SpecialEnums
|
|||
from litellm.types.videos.main import DecodedVideoId
|
||||
|
||||
VIDEO_ID_PREFIX: Final = "video_"
|
||||
VIDEO_ID_SERVED_MODEL_PREFIX: Final = "served_model:"
|
||||
CHARACTER_ID_PREFIX: Final = "character_"
|
||||
CHARACTER_ID_TEMPLATE: Final = "litellm:custom_llm_provider:{};model_id:{};character_id:{}"
|
||||
|
||||
|
|
@ -35,7 +36,9 @@ def _add_base64_padding(value: str) -> str:
|
|||
return value
|
||||
|
||||
|
||||
def encode_video_id_with_provider(video_id: str, provider: str, model_id: str | None = None) -> str:
|
||||
def encode_video_id_with_provider(
|
||||
video_id: str, provider: str, model_id: str | None = None, served_model: str | None = None
|
||||
) -> str:
|
||||
"""Encode provider and model_id into video_id using base64."""
|
||||
if not provider or not video_id:
|
||||
return video_id
|
||||
|
|
@ -50,8 +53,9 @@ def encode_video_id_with_provider(video_id: str, provider: str, model_id: str |
|
|||
|
||||
# ID is not encoded (even if it starts with video_), so encode it
|
||||
assembled_id = str(SpecialEnums.LITELLM_MANAGED_VIDEO_COMPLETE_STR.value).format(provider, model_id or "", video_id)
|
||||
served_part: Final = f";{VIDEO_ID_SERVED_MODEL_PREFIX}{served_model}" if served_model else ""
|
||||
|
||||
base64_encoded_id: Final[str] = base64.b64encode(assembled_id.encode("utf-8")).decode("utf-8")
|
||||
base64_encoded_id: Final[str] = base64.b64encode((assembled_id + served_part).encode("utf-8")).decode("utf-8")
|
||||
|
||||
return f"{VIDEO_ID_PREFIX}{base64_encoded_id}"
|
||||
|
||||
|
|
@ -99,10 +103,19 @@ def decode_video_id_with_provider(encoded_video_id: str) -> DecodedVideoId:
|
|||
model_id = model_id_part.replace("model_id:", "")
|
||||
decoded_video_id = video_id_part.replace("video_id:", "")
|
||||
|
||||
served_model: Final = (
|
||||
parts[3].removeprefix(VIDEO_ID_SERVED_MODEL_PREFIX) or None
|
||||
if len(parts) >= 4 and parts[3].startswith(VIDEO_ID_SERVED_MODEL_PREFIX)
|
||||
else None
|
||||
)
|
||||
|
||||
if served_model is None:
|
||||
return DecodedVideoId(custom_llm_provider=custom_llm_provider, model_id=model_id, video_id=decoded_video_id)
|
||||
return DecodedVideoId(
|
||||
custom_llm_provider=custom_llm_provider,
|
||||
model_id=model_id,
|
||||
video_id=decoded_video_id,
|
||||
served_model=served_model,
|
||||
)
|
||||
except Exception as e:
|
||||
verbose_logger.debug("Error decoding video_id '%s': %s", encoded_video_id, e)
|
||||
|
|
|
|||
|
|
@ -9859,6 +9859,16 @@ class ProviderConfigManager:
|
|||
from litellm.llms.vertex_ai.videos.transformation import VertexAIVideoConfig
|
||||
|
||||
return VertexAIVideoConfig()
|
||||
elif provider in (
|
||||
LlmProviders.DASHSCOPE,
|
||||
LlmProviders.QWENCLOUD,
|
||||
LlmProviders.QWEN_AI_PLATFORM,
|
||||
):
|
||||
from litellm.llms.dashscope.common_utils import (
|
||||
get_dashscope_family_video_config,
|
||||
)
|
||||
|
||||
return get_dashscope_family_video_config(provider.value)
|
||||
elif LlmProviders.RUNWAYML == provider:
|
||||
from litellm.llms.runwayml.videos.transformation import RunwayMLVideoConfig
|
||||
|
||||
|
|
|
|||
|
|
@ -225,6 +225,7 @@ def video_generation(
|
|||
model=model,
|
||||
video_generation_provider_config=video_generation_provider_config,
|
||||
video_generation_optional_params=video_generation_optional_params,
|
||||
drop_params=litellm_params.drop_params is True,
|
||||
)
|
||||
|
||||
# Pre Call logging
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ class VideoGenerationRequestUtils:
|
|||
model: str,
|
||||
video_generation_provider_config: BaseVideoConfig,
|
||||
video_generation_optional_params: VideoCreateOptionalRequestParams,
|
||||
drop_params: bool = False,
|
||||
) -> dict:
|
||||
"""
|
||||
Get optional parameters for the video generation API.
|
||||
|
|
@ -30,7 +31,7 @@ class VideoGenerationRequestUtils:
|
|||
mapped_params: Final = video_generation_provider_config.map_openai_params(
|
||||
video_create_optional_params=video_generation_optional_params,
|
||||
model=model,
|
||||
drop_params=litellm.drop_params,
|
||||
drop_params=drop_params or litellm.drop_params is True,
|
||||
)
|
||||
|
||||
# Merge extra_body params if present (for provider-specific parameters)
|
||||
|
|
|
|||
|
|
@ -17874,6 +17874,281 @@
|
|||
"/v1/images/generations"
|
||||
]
|
||||
},
|
||||
"dashscope/wan3.0-video": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video",
|
||||
"output_cost_per_second": 0.165025,
|
||||
"output_cost_per_second_480p": 0.041256,
|
||||
"output_cost_per_second_720p": 0.082513,
|
||||
"output_cost_per_second_1080p": 0.165025,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 all-in-one video generation (text, first/last frame, multimodal reference, edit, extend). Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/wan3.0-video-prime": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video-prime",
|
||||
"output_cost_per_second": 0.254399,
|
||||
"output_cost_per_second_480p": 0.0636,
|
||||
"output_cost_per_second_720p": 0.127199,
|
||||
"output_cost_per_second_1080p": 0.254399,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 fast tier. Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.1-t2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-t2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 text-to-video. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.1-i2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-i2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 image-to-video from a first frame. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.1-r2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-r2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 reference-to-video, 1 to 9 reference images. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.0-t2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-t2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 text-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.0-i2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-i2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 image-to-video from a first frame. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/happyhorse-1.0-r2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-r2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 reference-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/wan2.7-t2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-t2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 text-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/wan2.7-i2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-i2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 image-to-video (first frame, first and last frame, continuation). No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"dashscope/wan2.7-r2v": {
|
||||
"litellm_provider": "dashscope",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-r2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 reference-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/deepseek-v4-flash": {
|
||||
"cache_read_input_token_cost": 4e-08,
|
||||
"input_cost_per_token": 2e-07,
|
||||
|
|
@ -18826,6 +19101,281 @@
|
|||
"/v1/images/generations"
|
||||
]
|
||||
},
|
||||
"qwencloud/wan3.0-video": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video",
|
||||
"output_cost_per_second": 0.2,
|
||||
"output_cost_per_second_480p": 0.05,
|
||||
"output_cost_per_second_720p": 0.1,
|
||||
"output_cost_per_second_1080p": 0.2,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 all-in-one video generation (text, first/last frame, multimodal reference, edit, extend). Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/wan3.0-video-prime": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video-prime",
|
||||
"output_cost_per_second": 0.28,
|
||||
"output_cost_per_second_480p": 0.068,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.28,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 fast tier. Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.1-t2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-t2v",
|
||||
"output_cost_per_second": 0.18,
|
||||
"output_cost_per_second_480p": 0.07,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.18,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 text-to-video. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.1-i2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-i2v",
|
||||
"output_cost_per_second": 0.18,
|
||||
"output_cost_per_second_480p": 0.07,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.18,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 image-to-video from a first frame. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.1-r2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-r2v",
|
||||
"output_cost_per_second": 0.18,
|
||||
"output_cost_per_second_480p": 0.07,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.18,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 reference-to-video, 1 to 9 reference images. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.0-t2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-t2v",
|
||||
"output_cost_per_second": 0.24,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.24,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 text-to-video. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.0-i2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-i2v",
|
||||
"output_cost_per_second": 0.24,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.24,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 image-to-video from a first frame. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/happyhorse-1.0-r2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-r2v",
|
||||
"output_cost_per_second": 0.24,
|
||||
"output_cost_per_second_720p": 0.14,
|
||||
"output_cost_per_second_1080p": 0.24,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 reference-to-video. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/wan2.7-t2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-t2v",
|
||||
"output_cost_per_second": 0.15,
|
||||
"output_cost_per_second_720p": 0.1,
|
||||
"output_cost_per_second_1080p": 0.15,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 text-to-video. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/wan2.7-i2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-i2v",
|
||||
"output_cost_per_second": 0.15,
|
||||
"output_cost_per_second_720p": 0.1,
|
||||
"output_cost_per_second_1080p": 0.15,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 image-to-video (first frame, first and last frame, continuation). No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwencloud/wan2.7-r2v": {
|
||||
"litellm_provider": "qwencloud",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-r2v",
|
||||
"output_cost_per_second": 0.15,
|
||||
"output_cost_per_second_720p": 0.1,
|
||||
"output_cost_per_second_1080p": 0.15,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 reference-to-video. No 480P tier. Singapore list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/deepseek-v4-flash": {
|
||||
"cache_read_input_token_cost": 4e-08,
|
||||
"input_cost_per_token": 2e-07,
|
||||
|
|
@ -19818,6 +20368,281 @@
|
|||
"/v1/images/generations"
|
||||
]
|
||||
},
|
||||
"qwen_ai_platform/wan3.0-video": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video",
|
||||
"output_cost_per_second": 0.165025,
|
||||
"output_cost_per_second_480p": 0.041256,
|
||||
"output_cost_per_second_720p": 0.082513,
|
||||
"output_cost_per_second_1080p": 0.165025,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 all-in-one video generation (text, first/last frame, multimodal reference, edit, extend). Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/wan3.0-video-prime": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan3-0-video-prime",
|
||||
"output_cost_per_second": 0.254399,
|
||||
"output_cost_per_second_480p": 0.0636,
|
||||
"output_cost_per_second_720p": 0.127199,
|
||||
"output_cost_per_second_1080p": 0.254399,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 3.0 fast tier. Billed duration is usage.duration, which adds input video seconds for the edit and extend modes. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.1-t2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-t2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 text-to-video. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.1-i2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-i2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 image-to-video from a first frame. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.1-r2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-1-r2v",
|
||||
"output_cost_per_second": 0.165026,
|
||||
"output_cost_per_second_480p": 0.0618845,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.165026,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.1 reference-to-video, 1 to 9 reference images. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.0-t2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-t2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 text-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.0-i2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-i2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 image-to-video from a first frame. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/happyhorse-1.0-r2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/happyhorse-1-0-r2v",
|
||||
"output_cost_per_second": 0.220034,
|
||||
"output_cost_per_second_720p": 0.123769,
|
||||
"output_cost_per_second_1080p": 0.220034,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "HappyHorse 1.0 reference-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/wan2.7-t2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-t2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 text-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/wan2.7-i2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-i2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "first_frame"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 image-to-video (first frame, first and last frame, continuation). No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform/wan2.7-r2v": {
|
||||
"litellm_provider": "qwen_ai_platform",
|
||||
"mode": "video_generation",
|
||||
"source": "https://www.alibabacloud.com/help/en/model-studio/wan2-7-r2v",
|
||||
"output_cost_per_second": 0.143353,
|
||||
"output_cost_per_second_720p": 0.086012,
|
||||
"output_cost_per_second_1080p": 0.143353,
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"video",
|
||||
"audio"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"video"
|
||||
],
|
||||
"supported_endpoints": [
|
||||
"/v1/videos"
|
||||
],
|
||||
"provider_specific_entry": {
|
||||
"dashscope_video_reference_input": "reference_image"
|
||||
},
|
||||
"metadata": {
|
||||
"comment": "Wan 2.7 reference-to-video. No 480P tier. China (Beijing) list price per output second, USD, matching this prefix's default host. The base rate is the 1080P default tier."
|
||||
}
|
||||
},
|
||||
"databricks/databricks-bge-large-en": {
|
||||
"cache_creation_input_token_cost": 1.0003e-07,
|
||||
"cache_read_input_token_cost": 1.0003e-07,
|
||||
|
|
|
|||
|
|
@ -777,7 +777,8 @@
|
|||
"batches": false,
|
||||
"rerank": false,
|
||||
"a2a": true,
|
||||
"interactions": true
|
||||
"interactions": true,
|
||||
"video_generations": true
|
||||
}
|
||||
},
|
||||
"qwencloud": {
|
||||
|
|
@ -795,7 +796,8 @@
|
|||
"batches": false,
|
||||
"rerank": true,
|
||||
"a2a": true,
|
||||
"interactions": true
|
||||
"interactions": true,
|
||||
"video_generations": true
|
||||
}
|
||||
},
|
||||
"qwen_ai_platform": {
|
||||
|
|
@ -813,7 +815,8 @@
|
|||
"batches": false,
|
||||
"rerank": true,
|
||||
"a2a": true,
|
||||
"interactions": true
|
||||
"interactions": true,
|
||||
"video_generations": true
|
||||
}
|
||||
},
|
||||
"databricks": {
|
||||
|
|
|
|||
207
tests/integration/providers/_dashscope_video_support.py
Normal file
207
tests/integration/providers/_dashscope_video_support.py
Normal file
|
|
@ -0,0 +1,207 @@
|
|||
import base64
|
||||
import json
|
||||
import uuid
|
||||
from collections.abc import Callable, Mapping
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
from hashlib import sha256
|
||||
from typing import Final
|
||||
|
||||
import httpx
|
||||
from integration._support.client import (
|
||||
JSON_OBJECT,
|
||||
Gateway,
|
||||
Scenario,
|
||||
eventually,
|
||||
held,
|
||||
list_value,
|
||||
object_value,
|
||||
string_value,
|
||||
)
|
||||
from integration._support.database import read_rows
|
||||
from integration._support.wire import Reply, Request
|
||||
from openai import AsyncOpenAI, OpenAI
|
||||
from pydantic import JsonValue
|
||||
|
||||
CREATE_PATH: Final = "/api/v1/services/aigc/video-generation/video-synthesis"
|
||||
TASKS_PATH: Final = "/api/v1/tasks/"
|
||||
OSS_PATH: Final = "/oss/"
|
||||
API_KEY: Final = "synthetic-dashscope-key"
|
||||
MODEL: Final = "wan3.0-video"
|
||||
PROMPT: Final = "a red kite climbing over a grey sea at dawn"
|
||||
MP4: Final = b"\x00\x00\x00\x18ftypmp42" + uuid.uuid4().bytes * 4
|
||||
MP4_REPLY: Final = Reply(body=MP4, content_type="video/mp4")
|
||||
SUBMITTED: Final = "2026-10-14 10:00:00.000"
|
||||
FINISHED: Final = "2026-10-14 10:01:30.500"
|
||||
SUBMITTED_EPOCH: Final = int(datetime(2026, 10, 14, 2, 0, 0, tzinfo=timezone.utc).timestamp())
|
||||
FINISHED_EPOCH: Final = int(datetime(2026, 10, 14, 2, 1, 30, tzinfo=timezone.utc).timestamp())
|
||||
|
||||
TaskStatus = Callable[[str, str], Mapping[str, JsonValue]]
|
||||
|
||||
|
||||
def json_reply(body: Mapping[str, JsonValue], status: int = 200) -> Reply:
|
||||
return Reply(status=status, body=json.dumps(body).encode())
|
||||
|
||||
|
||||
def pending(task_id: str, origin: str) -> Mapping[str, JsonValue]:
|
||||
return {"request_id": f"req-{task_id}", "output": {"task_id": task_id, "task_status": "PENDING"}}
|
||||
|
||||
|
||||
def succeeded(task_id: str, origin: str) -> Mapping[str, JsonValue]:
|
||||
return {
|
||||
"request_id": f"req-{task_id}",
|
||||
"output": {
|
||||
"task_id": task_id,
|
||||
"task_status": "SUCCEEDED",
|
||||
"submit_time": SUBMITTED,
|
||||
"end_time": FINISHED,
|
||||
"video_url": f"{origin}{OSS_PATH}{task_id}.mp4",
|
||||
},
|
||||
"usage": {"duration": 5, "SR": 720, "ratio": "16:9", "video_count": 1},
|
||||
}
|
||||
|
||||
|
||||
def unknown(task_id: str, origin: str) -> Mapping[str, JsonValue]:
|
||||
return {"request_id": f"req-{task_id}", "output": {"task_id": task_id, "task_status": "UNKNOWN"}}
|
||||
|
||||
|
||||
def task_api(
|
||||
status: TaskStatus = succeeded, oss: Reply = MP4_REPLY, prefix: str = "task"
|
||||
) -> Callable[[Request], Reply]:
|
||||
def respond(request: Request) -> Reply:
|
||||
if request.target.startswith(OSS_PATH):
|
||||
return oss
|
||||
if request.method == "POST" and request.target == CREATE_PATH:
|
||||
return json_reply(pending(f"{prefix}-{uuid.uuid4().hex}", ""))
|
||||
assert request.method == "GET" and request.target.startswith(TASKS_PATH), (request.method, request.target)
|
||||
task_id: Final = request.target.removeprefix(TASKS_PATH)
|
||||
known: Final = task_id.startswith(f"{prefix}-")
|
||||
return json_reply((status if known else unknown)(task_id, f"http://{request.headers['host']}"))
|
||||
|
||||
return respond
|
||||
|
||||
|
||||
def polled_tasks(requests: tuple[Request, ...]) -> tuple[str, ...]:
|
||||
return tuple(
|
||||
request.target.removeprefix(TASKS_PATH)
|
||||
for request in requests
|
||||
if request.method == "GET" and request.target.startswith(TASKS_PATH)
|
||||
)
|
||||
|
||||
|
||||
def create_bodies(requests: tuple[Request, ...]) -> tuple[dict[str, JsonValue], ...]:
|
||||
return tuple(
|
||||
JSON_OBJECT.validate_json(request.body)
|
||||
for request in requests
|
||||
if request.method == "POST" and request.target == CREATE_PATH
|
||||
)
|
||||
|
||||
|
||||
def decoded(video_id: str) -> Mapping[str, str | None]:
|
||||
from litellm.types.videos.utils import decode_video_id_with_provider
|
||||
|
||||
return {
|
||||
name: value if isinstance(value, str) else None
|
||||
for name, value in decode_video_id_with_provider(video_id).items()
|
||||
}
|
||||
|
||||
|
||||
def task_of(video_id: str) -> str:
|
||||
task: Final = decoded(video_id).get("video_id")
|
||||
assert task is not None, video_id
|
||||
return task
|
||||
|
||||
|
||||
def forged_id(provider: str, model: str, task_id: str, served: str | None) -> str:
|
||||
served_part: Final = f";served_model:{served}" if served else ""
|
||||
plain: Final = f"litellm:custom_llm_provider:{provider};model_id:{model};video_id:{task_id}{served_part}"
|
||||
return f"video_{base64.b64encode(plain.encode()).decode()}"
|
||||
|
||||
|
||||
def rate(provider: str, tier_field: str, model: str = MODEL) -> float:
|
||||
import litellm
|
||||
|
||||
value: Final = litellm.model_cost[f"{provider}/{model}"][tier_field]
|
||||
assert isinstance(value, float), (provider, model, tier_field, value)
|
||||
return value
|
||||
|
||||
|
||||
def key_spend(key: str) -> float:
|
||||
rows: Final = read_rows(
|
||||
'SELECT spend FROM "LiteLLM_VerificationToken" WHERE token = %s', (sha256(key.encode()).hexdigest(),)
|
||||
)
|
||||
assert len(rows) == 1, rows
|
||||
return float(str(rows[0]["spend"]))
|
||||
|
||||
|
||||
def settled_spend(key: str, expected: float) -> float:
|
||||
eventually(lambda: key_spend(key), lambda spend: abs(spend - expected) < 1e-9, seconds=70)
|
||||
return spend_stays(key, expected)
|
||||
|
||||
|
||||
def spend_stays(key: str, expected: float) -> float:
|
||||
return held(lambda: key_spend(key), lambda spend: abs(spend - expected) < 1e-9, holding=4)
|
||||
|
||||
|
||||
def spend_rows(key: str) -> tuple[Mapping[str, JsonValue], ...]:
|
||||
return tuple(
|
||||
read_rows(
|
||||
'SELECT request_id, call_type, spend, model_group, status FROM "LiteLLM_SpendLogs" WHERE api_key = %s',
|
||||
(sha256(key.encode()).hexdigest(),),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def proxy_url(gateway: Gateway) -> str:
|
||||
return str(gateway.client.base_url).rstrip("/")
|
||||
|
||||
|
||||
def _listed_over_a_fresh_connection(gateway: Gateway, _: int) -> frozenset[str]:
|
||||
with httpx.Client(base_url=gateway.client.base_url, timeout=15, trust_env=False) as client:
|
||||
listed: Final = client.get("/v1/models", headers={"Authorization": f"Bearer {gateway.key}"})
|
||||
assert listed.status_code == 200, listed.text
|
||||
entries: Final = list_value(JSON_OBJECT.validate_json(listed.content)["data"])
|
||||
return frozenset(string_value(object_value(entry)["id"]) for entry in entries)
|
||||
|
||||
|
||||
def _every_worker_lists(gateway: Gateway, model: str) -> bool:
|
||||
with ThreadPoolExecutor(max_workers=12) as pool:
|
||||
seen: Final = tuple(pool.map(lambda index: _listed_over_a_fresh_connection(gateway, index), range(24)))
|
||||
return all(model in listed for listed in seen)
|
||||
|
||||
|
||||
def deployed(gateway: Gateway, scenario: Scenario, **parameters: JsonValue) -> str:
|
||||
model: Final = scenario.model(model_info=None, **parameters)
|
||||
eventually(lambda: _every_worker_lists(gateway, model), lambda listed: listed, seconds=90)
|
||||
return model
|
||||
|
||||
|
||||
def openai_client(gateway: Gateway, key: str) -> OpenAI:
|
||||
return OpenAI(
|
||||
api_key=key,
|
||||
base_url=proxy_url(gateway) + "/v1",
|
||||
max_retries=0,
|
||||
http_client=httpx.Client(timeout=60, trust_env=False),
|
||||
)
|
||||
|
||||
|
||||
def async_openai_client(gateway: Gateway, key: str) -> AsyncOpenAI:
|
||||
return AsyncOpenAI(
|
||||
api_key=key,
|
||||
base_url=proxy_url(gateway) + "/v1",
|
||||
max_retries=0,
|
||||
http_client=httpx.AsyncClient(timeout=60, trust_env=False),
|
||||
)
|
||||
|
||||
|
||||
def create(gateway: Gateway, body: Mapping[str, JsonValue], key: str | None = None) -> httpx.Response:
|
||||
return gateway.request("POST", "/v1/videos", {"prompt": PROMPT, **body}, key=key)
|
||||
|
||||
|
||||
def created_id(response: httpx.Response) -> str:
|
||||
assert response.status_code == 200, response.text
|
||||
return string_value(JSON_OBJECT.validate_json(response.content)["id"])
|
||||
|
||||
|
||||
def error_message(response: httpx.Response) -> str:
|
||||
return string_value(object_value(JSON_OBJECT.validate_json(response.content)["error"])["message"])
|
||||
377
tests/integration/providers/test_dashscope_video_chaos.py
Normal file
377
tests/integration/providers/test_dashscope_video_chaos.py
Normal file
|
|
@ -0,0 +1,377 @@
|
|||
import json
|
||||
import threading
|
||||
from collections.abc import Callable, Iterator, Mapping
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
from queue import SimpleQueue
|
||||
from types import MappingProxyType
|
||||
from typing import Final
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
import httpx
|
||||
import psutil
|
||||
import pytest
|
||||
import yaml
|
||||
from integration._support.client import JSON_OBJECT, Gateway, eventually, gateway_from_environment
|
||||
from integration._support.process import OwnedProxy, graceful_stop_seconds, owned_proxy_process
|
||||
from integration._support.wire import Reply, Request, Wire, wire_server
|
||||
from integration.providers._dashscope_video_support import (
|
||||
CREATE_PATH,
|
||||
MODEL,
|
||||
MP4,
|
||||
OSS_PATH,
|
||||
PROMPT,
|
||||
TASKS_PATH,
|
||||
create_bodies,
|
||||
created_id,
|
||||
forged_id,
|
||||
json_reply,
|
||||
polled_tasks,
|
||||
task_api,
|
||||
task_of,
|
||||
)
|
||||
from pydantic import JsonValue
|
||||
|
||||
pytestmark: Final = pytest.mark.xdist_group("dashscope-video-owned-proxies")
|
||||
|
||||
_CHAOS_TIMEOUT: Final = int(4 * graceful_stop_seconds() + 240)
|
||||
_GROUP: Final = "dashscope-video-group"
|
||||
_MEMBER_A: Final = "dashscope-video-a"
|
||||
_MEMBER_B: Final = "dashscope-video-b"
|
||||
_CATCH_ALL_KEY: Final = "catch-all-openai-key"
|
||||
_ROUTED_ENVIRONMENT: Final = (
|
||||
"DASHSCOPE_API_BASE",
|
||||
"DASHSCOPE_API_BASE_VIDEO",
|
||||
"DASHSCOPE_API_KEY",
|
||||
"QWENCLOUD_API_BASE",
|
||||
"QWENCLOUD_API_BASE_VIDEO",
|
||||
"QWENCLOUD_API_KEY",
|
||||
"QWEN_AI_PLATFORM_API_BASE",
|
||||
"QWEN_AI_PLATFORM_API_BASE_VIDEO",
|
||||
"QWEN_AI_PLATFORM_API_KEY",
|
||||
)
|
||||
_CHAT_REPLY: Final = json_reply(
|
||||
{
|
||||
"id": "chatcmpl-dashscope-env",
|
||||
"object": "chat.completion",
|
||||
"created": 1791943200,
|
||||
"model": "qwen-plus",
|
||||
"choices": [{"index": 0, "message": {"role": "assistant", "content": "kite"}, "finish_reason": "stop"}],
|
||||
"usage": {"prompt_tokens": 3, "completion_tokens": 1, "total_tokens": 4},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _config(
|
||||
tmp_path: Path,
|
||||
name: str,
|
||||
model_list: tuple[Mapping[str, JsonValue], ...],
|
||||
router_settings: Mapping[str, JsonValue],
|
||||
litellm_settings: Mapping[str, JsonValue],
|
||||
) -> Path:
|
||||
base: Final = yaml.safe_load(Path("tests/integration/proxy_config.yaml").read_text())
|
||||
path: Final = tmp_path / f"{name}.yaml"
|
||||
path.write_text(
|
||||
yaml.safe_dump(
|
||||
{
|
||||
**base,
|
||||
"model_list": [dict(entry) for entry in model_list],
|
||||
"router_settings": {**base["router_settings"], **router_settings},
|
||||
"litellm_settings": {**base["litellm_settings"], **litellm_settings},
|
||||
}
|
||||
)
|
||||
)
|
||||
return path
|
||||
|
||||
|
||||
def _member(name: str, wire: Wire, key: str) -> Mapping[str, JsonValue]:
|
||||
return {"model_name": name, "litellm_params": {"model": f"dashscope/{MODEL}", "api_base": wire.url, "api_key": key}}
|
||||
|
||||
|
||||
def _create(gateway: Gateway, body: Mapping[str, JsonValue]) -> httpx.Response:
|
||||
return gateway.request("POST", "/v1/videos", {"prompt": PROMPT, **body})
|
||||
|
||||
|
||||
def _completed(gateway: Gateway, video_id: str) -> bool:
|
||||
response: Final = gateway.request("GET", f"/v1/videos/{video_id}")
|
||||
assert response.status_code == 200, response.text
|
||||
return JSON_OBJECT.validate_json(response.content)["status"] == "completed"
|
||||
|
||||
|
||||
def _gated(
|
||||
release: threading.Event, armed: threading.Event, held: SimpleQueue[str], prefix: str
|
||||
) -> Callable[[Request], Reply]:
|
||||
tasks: Final = task_api(prefix=prefix)
|
||||
|
||||
def respond(request: Request) -> Reply:
|
||||
if armed.is_set() and request.method == "GET" and request.target.startswith(TASKS_PATH):
|
||||
held.put(request.target)
|
||||
assert release.wait(timeout=120), "The held polls were never released"
|
||||
return tasks(request)
|
||||
|
||||
return respond
|
||||
|
||||
|
||||
def _fresh_poll(gateway: Gateway, video_id: str) -> int | None:
|
||||
try:
|
||||
with httpx.Client(base_url=gateway.client.base_url, timeout=150, trust_env=False) as client:
|
||||
return client.get(f"/v1/videos/{video_id}", headers={"Authorization": f"Bearer {gateway.key}"}).status_code
|
||||
except httpx.TransportError:
|
||||
return None
|
||||
|
||||
|
||||
def _held_connections(pid: int, wires: tuple[Wire, ...]) -> int:
|
||||
ports: Final = frozenset(urlsplit(wire.url).port for wire in wires)
|
||||
return sum(
|
||||
1
|
||||
for connection in psutil.Process(pid).net_connections(kind="tcp")
|
||||
if connection.status == psutil.CONN_ESTABLISHED and connection.raddr and connection.raddr.port in ports
|
||||
)
|
||||
|
||||
|
||||
def _assert_each_polls_on_its_creating_member(gateway: Gateway, video_ids: tuple[str, ...], a: Wire, b: Wire) -> None:
|
||||
a.drain()
|
||||
b.drain()
|
||||
assert all(_completed(gateway, video_id) for video_id in video_ids)
|
||||
a_tasks: Final = sorted(task_of(video_id) for video_id in video_ids if task_of(video_id).startswith("a-"))
|
||||
b_tasks: Final = sorted(task_of(video_id) for video_id in video_ids if task_of(video_id).startswith("b-"))
|
||||
assert sorted(polled_tasks(a.drain())) == a_tasks
|
||||
assert sorted(polled_tasks(b.drain())) == b_tasks
|
||||
|
||||
|
||||
@pytest.mark.timeout(_CHAOS_TIMEOUT)
|
||||
def test_a_worker_killed_mid_poll_burst_and_a_restart_leave_every_task_polling_its_creating_member(
|
||||
gateway: Gateway, tmp_path: Path
|
||||
) -> None:
|
||||
release: Final = threading.Event()
|
||||
armed: Final = threading.Event()
|
||||
held: Final[SimpleQueue[str]] = SimpleQueue()
|
||||
with (
|
||||
wire_server(_gated(release, armed, held, "a")) as a,
|
||||
wire_server(_gated(release, armed, held, "b")) as b,
|
||||
):
|
||||
path: Final = _config(
|
||||
tmp_path,
|
||||
"dashscope-video-chaos",
|
||||
(
|
||||
_member(_MEMBER_A, a, "key-a"),
|
||||
_member(_MEMBER_B, b, "key-b"),
|
||||
),
|
||||
{
|
||||
"routing_groups": [
|
||||
{"group_name": _GROUP, "models": [_MEMBER_A, _MEMBER_B], "routing_strategy": "simple-shuffle"}
|
||||
]
|
||||
},
|
||||
{},
|
||||
)
|
||||
with owned_proxy_process(gateway, tmp_path, {}, config=path, workers=2) as owned:
|
||||
candidate: Final = owned.gateway
|
||||
workers: Final = eventually(owned.worker_pids, lambda pids: len(pids) == 2, seconds=60)
|
||||
video_ids: Final = tuple(
|
||||
created_id(_create(candidate, {"model": _GROUP, "seconds": "5", "size": "1280x720"})) for _ in range(24)
|
||||
)
|
||||
assert {task_of(video_id)[0] for video_id in video_ids} == {"a", "b"}
|
||||
a.drain()
|
||||
b.drain()
|
||||
armed.set()
|
||||
with ThreadPoolExecutor(max_workers=len(video_ids)) as pool:
|
||||
polls: Final = tuple(pool.submit(_fresh_poll, candidate, video_id) for video_id in video_ids)
|
||||
eventually(held.qsize, lambda size: size == len(video_ids), seconds=60)
|
||||
held_by: Final = MappingProxyType({pid: _held_connections(pid, (a, b)) for pid in workers})
|
||||
assert sum(held_by.values()) == len(video_ids), held_by
|
||||
victim_pid, survivor_pid = sorted(workers, key=held_by.__getitem__)
|
||||
with owned.replacing_worker(victim_pid) as kill:
|
||||
kill.freeze()
|
||||
kill()
|
||||
armed.clear()
|
||||
release.set()
|
||||
statuses: Final = tuple(poll.result() for poll in polls)
|
||||
assert statuses.count(200) == held_by[survivor_pid], (held_by, statuses)
|
||||
assert all(status in (200, None) for status in statuses), statuses
|
||||
_assert_each_polls_on_its_creating_member(candidate, video_ids, a, b)
|
||||
with owned_proxy_process(gateway, tmp_path, {}, config=path, workers=2) as restarted:
|
||||
_assert_each_polls_on_its_creating_member(restarted.gateway, video_ids, a, b)
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def environment_wires() -> Iterator[Mapping[str, Wire]]:
|
||||
with (
|
||||
wire_server(task_api()) as compatible,
|
||||
wire_server(task_api()) as qwencloud,
|
||||
wire_server(task_api()) as platform,
|
||||
):
|
||||
yield MappingProxyType({"dashscope": compatible, "qwencloud": qwencloud, "qwen_ai_platform": platform})
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def catch_all_wire() -> Iterator[Wire]:
|
||||
with wire_server(
|
||||
lambda _request: json_reply({"error": {"message": "catch-all", "type": "invalid_request_error"}}, 404)
|
||||
) as wire:
|
||||
yield wire
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def environment_proxy(
|
||||
environment_wires: Mapping[str, Wire], catch_all_wire: Wire, tmp_path_factory: pytest.TempPathFactory
|
||||
) -> Iterator[OwnedProxy]:
|
||||
directory: Final = tmp_path_factory.mktemp("dashscope-video-environment")
|
||||
path: Final = _config(
|
||||
directory,
|
||||
"dashscope-video-environment",
|
||||
(
|
||||
*(
|
||||
{"model_name": f"env-{provider}", "litellm_params": {"model": f"{provider}/{MODEL}"}}
|
||||
for provider in environment_wires
|
||||
),
|
||||
{
|
||||
"model_name": "*",
|
||||
"litellm_params": {
|
||||
"model": "openai/*",
|
||||
"api_base": f"{catch_all_wire.url}/v1",
|
||||
"api_key": _CATCH_ALL_KEY,
|
||||
},
|
||||
},
|
||||
),
|
||||
{},
|
||||
{"drop_params": True},
|
||||
)
|
||||
with (
|
||||
gateway_from_environment() as gateway,
|
||||
owned_proxy_process(
|
||||
gateway,
|
||||
directory,
|
||||
{
|
||||
"DASHSCOPE_API_BASE": f"{environment_wires['dashscope'].url}/compatible-mode/v1",
|
||||
"QWENCLOUD_API_BASE_VIDEO": environment_wires["qwencloud"].url,
|
||||
"QWEN_AI_PLATFORM_API_BASE_VIDEO": environment_wires["qwen_ai_platform"].url,
|
||||
"DASHSCOPE_API_KEY": "env-dashscope-key",
|
||||
"QWENCLOUD_API_KEY": "env-qwencloud-key",
|
||||
"QWEN_AI_PLATFORM_API_KEY": "env-qwen-ai-platform-key",
|
||||
},
|
||||
config=path,
|
||||
remove_environment=_ROUTED_ENVIRONMENT,
|
||||
) as owned,
|
||||
):
|
||||
yield owned
|
||||
|
||||
|
||||
@pytest.mark.timeout(_CHAOS_TIMEOUT)
|
||||
@pytest.mark.parametrize(
|
||||
("provider", "key"),
|
||||
[
|
||||
("dashscope", "env-dashscope-key"),
|
||||
("qwencloud", "env-qwencloud-key"),
|
||||
("qwen_ai_platform", "env-qwen-ai-platform-key"),
|
||||
],
|
||||
)
|
||||
def test_each_dashscope_prefix_reaches_its_own_environment_base_with_its_own_environment_key(
|
||||
environment_proxy: OwnedProxy, environment_wires: Mapping[str, Wire], provider: str, key: str
|
||||
) -> None:
|
||||
wire: Final = environment_wires[provider]
|
||||
wire.drain()
|
||||
candidate: Final = environment_proxy.gateway
|
||||
video_id: Final = created_id(_create(candidate, {"model": f"env-{provider}", "seconds": "5", "size": "1280x720"}))
|
||||
assert _completed(candidate, video_id)
|
||||
download: Final = candidate.request("GET", f"/v1/videos/{video_id}/content")
|
||||
assert download.status_code == 200 and download.content == MP4, download.text
|
||||
received: Final = wire.drain()
|
||||
assert [(request.method, request.target) for request in received] == [
|
||||
("POST", CREATE_PATH),
|
||||
("GET", f"{TASKS_PATH}{task_of(video_id)}"),
|
||||
("GET", f"{TASKS_PATH}{task_of(video_id)}"),
|
||||
("GET", f"{OSS_PATH}{task_of(video_id)}.mp4"),
|
||||
]
|
||||
assert [request.headers.get("authorization") for request in received[:3]] == [f"Bearer {key}"] * 3
|
||||
assert all(
|
||||
other.drain() == () for name, other in environment_wires.items() if name != provider
|
||||
), "A prefix reached another prefix's environment base"
|
||||
|
||||
|
||||
@pytest.mark.timeout(_CHAOS_TIMEOUT)
|
||||
def test_a_catch_all_over_another_provider_never_takes_a_poll_for_an_id_only_its_pattern_matches(
|
||||
environment_proxy: OwnedProxy, environment_wires: Mapping[str, Wire], catch_all_wire: Wire
|
||||
) -> None:
|
||||
wire: Final = environment_wires["dashscope"]
|
||||
candidate: Final = environment_proxy.gateway
|
||||
task: Final = task_of(created_id(_create(candidate, {"model": "env-dashscope", "seconds": "5"})))
|
||||
polled: Final = forged_id("dashscope", MODEL, task, None)
|
||||
wire.drain()
|
||||
catch_all_wire.drain()
|
||||
assert _completed(candidate, polled)
|
||||
download: Final = candidate.request("GET", f"/v1/videos/{polled}/content")
|
||||
assert download.status_code == 200 and download.content == MP4, download.text
|
||||
assert catch_all_wire.drain() == ()
|
||||
received: Final = wire.drain()
|
||||
assert [(request.method, request.target) for request in received] == [
|
||||
("GET", f"{TASKS_PATH}{task}"),
|
||||
("GET", f"{TASKS_PATH}{task}"),
|
||||
("GET", f"{OSS_PATH}{task}.mp4"),
|
||||
]
|
||||
assert [request.headers.get("authorization") for request in received[:2]] == ["Bearer env-dashscope-key"] * 2
|
||||
|
||||
|
||||
@pytest.mark.timeout(_CHAOS_TIMEOUT)
|
||||
def test_global_drop_params_drops_a_malformed_seconds_and_seed_instead_of_answering_400(
|
||||
environment_proxy: OwnedProxy, environment_wires: Mapping[str, Wire]
|
||||
) -> None:
|
||||
wire: Final = environment_wires["dashscope"]
|
||||
wire.drain()
|
||||
created_id(
|
||||
_create(
|
||||
environment_proxy.gateway,
|
||||
{"model": "env-dashscope", "seconds": "10s", "size": "1280x720", "parameters": {"seed": "abc"}},
|
||||
)
|
||||
)
|
||||
(body,) = create_bodies(wire.drain())
|
||||
assert body == {
|
||||
"model": MODEL,
|
||||
"input": {"prompt": PROMPT},
|
||||
"parameters": {"ratio": "16:9", "resolution": "720P"},
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.timeout(_CHAOS_TIMEOUT)
|
||||
def test_the_video_base_beats_the_chat_base_for_videos_while_chat_keeps_the_chat_base(
|
||||
gateway: Gateway, tmp_path: Path
|
||||
) -> None:
|
||||
def chat(request: Request) -> Reply:
|
||||
assert (request.method, request.target) == ("POST", "/compatible-mode/v1/chat/completions"), request.target
|
||||
return _CHAT_REPLY
|
||||
|
||||
with wire_server(task_api()) as video, wire_server(chat) as chat_wire:
|
||||
path: Final = _config(
|
||||
tmp_path,
|
||||
"dashscope-video-base",
|
||||
(
|
||||
{"model_name": "env-video", "litellm_params": {"model": f"dashscope/{MODEL}"}},
|
||||
{"model_name": "env-chat", "litellm_params": {"model": "dashscope/qwen-plus"}},
|
||||
),
|
||||
{},
|
||||
{},
|
||||
)
|
||||
with owned_proxy_process(
|
||||
gateway,
|
||||
tmp_path,
|
||||
{
|
||||
"DASHSCOPE_API_BASE_VIDEO": video.url,
|
||||
"DASHSCOPE_API_BASE": f"{chat_wire.url}/compatible-mode/v1",
|
||||
"DASHSCOPE_API_KEY": "env-dashscope-key",
|
||||
},
|
||||
config=path,
|
||||
remove_environment=_ROUTED_ENVIRONMENT,
|
||||
) as owned:
|
||||
candidate: Final = owned.gateway
|
||||
video_id: Final = created_id(_create(candidate, {"model": "env-video", "seconds": "5"}))
|
||||
assert _completed(candidate, video_id)
|
||||
completion: Final = candidate.post(
|
||||
"/v1/chat/completions", {"model": "env-chat", "messages": [{"role": "user", "content": "kite?"}]}
|
||||
)
|
||||
assert completion["id"] == "chatcmpl-dashscope-env", completion
|
||||
assert [(request.method, request.target) for request in video.drain()] == [
|
||||
("POST", CREATE_PATH),
|
||||
("GET", f"{TASKS_PATH}{task_of(video_id)}"),
|
||||
]
|
||||
(chat_request,) = chat_wire.drain()
|
||||
assert json.loads(chat_request.body)["model"] == "qwen-plus"
|
||||
assert chat_request.headers["authorization"] == "Bearer env-dashscope-key"
|
||||
584
tests/integration/providers/test_dashscope_video_wire.py
Normal file
584
tests/integration/providers/test_dashscope_video_wire.py
Normal file
|
|
@ -0,0 +1,584 @@
|
|||
import base64
|
||||
import json
|
||||
from collections.abc import Mapping
|
||||
from typing import Final
|
||||
|
||||
import pytest
|
||||
from integration._support.client import JSON_OBJECT, Gateway, object_value
|
||||
from integration._support.wire import Reply, Request, wire_server
|
||||
from integration.providers._dashscope_video_support import (
|
||||
API_KEY,
|
||||
CREATE_PATH,
|
||||
FINISHED_EPOCH,
|
||||
MODEL,
|
||||
MP4,
|
||||
OSS_PATH,
|
||||
PROMPT,
|
||||
SUBMITTED_EPOCH,
|
||||
TASKS_PATH,
|
||||
async_openai_client,
|
||||
create,
|
||||
create_bodies,
|
||||
created_id,
|
||||
deployed,
|
||||
error_message,
|
||||
json_reply,
|
||||
openai_client,
|
||||
pending,
|
||||
rate,
|
||||
settled_spend,
|
||||
spend_stays,
|
||||
task_api,
|
||||
task_of,
|
||||
unknown,
|
||||
)
|
||||
from pydantic import JsonValue
|
||||
|
||||
_PNG: Final = base64.b64decode(
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="
|
||||
)
|
||||
_HUGE: Final = "9" * 5120
|
||||
_720P: Final = "output_cost_per_second_720p"
|
||||
_1080P: Final = "output_cost_per_second"
|
||||
|
||||
|
||||
def _assert_task_wire(requests: tuple[Request, ...], task_id: str, parameters: Mapping[str, JsonValue]) -> None:
|
||||
create_request, poll, content_poll, download = requests
|
||||
assert (create_request.method, create_request.target) == ("POST", CREATE_PATH)
|
||||
assert create_request.headers["authorization"] == f"Bearer {API_KEY}"
|
||||
assert create_request.headers["x-dashscope-async"] == "enable"
|
||||
assert create_request.headers["content-type"] == "application/json"
|
||||
assert json.loads(create_request.body) == {"model": MODEL, "input": {"prompt": PROMPT}, "parameters": parameters}
|
||||
for request in (poll, content_poll):
|
||||
assert (request.method, request.target) == ("GET", f"{TASKS_PATH}{task_id}")
|
||||
assert request.headers["authorization"] == f"Bearer {API_KEY}"
|
||||
assert (download.method, download.target) == ("GET", f"{OSS_PATH}{task_id}.mp4")
|
||||
assert "authorization" not in download.headers
|
||||
|
||||
|
||||
def _assert_completed(status: Mapping[str, JsonValue], video_id: str) -> None:
|
||||
assert status["id"] == video_id, status
|
||||
assert status["status"] == "completed", status
|
||||
assert status["created_at"] == SUBMITTED_EPOCH, status
|
||||
assert status["completed_at"] == FINISHED_EPOCH, status
|
||||
assert status["size"] == "1280x720", status
|
||||
assert status["seconds"] == "5.0", status
|
||||
|
||||
|
||||
def _assert_billed_once_at_720p(key: str, provider: str) -> None:
|
||||
assert rate(provider, _720P) != rate(provider, _1080P)
|
||||
settled_spend(key, 5 * rate(provider, _720P))
|
||||
|
||||
|
||||
def test_dashscope_openai_sdk_sync_create_poll_download_bills_the_720p_tier_once(gateway: Gateway) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
key: Final = scenario.key(models=[model])
|
||||
client: Final = openai_client(gateway, key)
|
||||
video: Final = client.videos.create(
|
||||
model=model,
|
||||
prompt=PROMPT,
|
||||
seconds="5", # pyright: ignore[reportArgumentType] # DashScope takes seconds the OpenAI literal type omits
|
||||
size="1280x720",
|
||||
)
|
||||
assert video.status == "queued", video
|
||||
_assert_billed_once_at_720p(key, "dashscope")
|
||||
status: Final = client.videos.retrieve(video.id)
|
||||
_assert_completed(status.model_dump(), video.id)
|
||||
content: Final = client.videos.download_content(video.id)
|
||||
assert content.content == MP4
|
||||
_assert_task_wire(wire.drain(), task_of(video.id), {"ratio": "16:9", "resolution": "720P", "duration": 5})
|
||||
spend_stays(key, 5 * rate("dashscope", _720P))
|
||||
|
||||
|
||||
async def test_qwencloud_openai_sdk_async_create_poll_download_bills_the_qwencloud_720p_tier_once(
|
||||
gateway: Gateway,
|
||||
) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"qwencloud/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
key: Final = scenario.key(models=[model])
|
||||
client: Final = async_openai_client(gateway, key)
|
||||
video: Final = await client.videos.create(
|
||||
model=model,
|
||||
prompt=PROMPT,
|
||||
seconds="5", # pyright: ignore[reportArgumentType] # DashScope takes seconds the OpenAI literal type omits
|
||||
size="1280x720",
|
||||
)
|
||||
assert video.status == "queued", video
|
||||
assert rate("qwencloud", _720P) != rate("dashscope", _720P)
|
||||
_assert_billed_once_at_720p(key, "qwencloud")
|
||||
status: Final = await client.videos.retrieve(video.id)
|
||||
_assert_completed(status.model_dump(), video.id)
|
||||
content: Final = await client.videos.download_content(video.id)
|
||||
assert content.content == MP4
|
||||
_assert_task_wire(wire.drain(), task_of(video.id), {"ratio": "16:9", "resolution": "720P", "duration": 5})
|
||||
spend_stays(key, 5 * rate("qwencloud", _720P))
|
||||
|
||||
|
||||
def test_qwen_ai_platform_raw_http_create_poll_download_bills_the_720p_tier_once(gateway: Gateway) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(
|
||||
gateway, scenario, model=f"qwen_ai_platform/{MODEL}", api_base=wire.url, api_key=API_KEY
|
||||
)
|
||||
key: Final = scenario.key(models=[model])
|
||||
video_id: Final = created_id(create(gateway, {"model": model, "seconds": 5, "size": "1280x720"}, key))
|
||||
_assert_billed_once_at_720p(key, "qwen_ai_platform")
|
||||
status: Final = gateway.request("GET", f"/v1/videos/{video_id}", key=key)
|
||||
assert status.status_code == 200, status.text
|
||||
_assert_completed(JSON_OBJECT.validate_json(status.content), video_id)
|
||||
content: Final = gateway.request("GET", f"/v1/videos/{video_id}/content", key=key)
|
||||
assert content.status_code == 200, content.text
|
||||
assert content.headers["content-type"].startswith("video/mp4")
|
||||
assert content.content == MP4
|
||||
_assert_task_wire(wire.drain(), task_of(video_id), {"ratio": "16:9", "resolution": "720P", "duration": 5})
|
||||
expected: Final = 5 * rate("qwen_ai_platform", _720P)
|
||||
spend_stays(key, expected)
|
||||
|
||||
|
||||
def test_multipart_input_reference_reaches_dashscope_as_a_first_frame_media_entry(gateway: Gateway) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
response: Final = gateway.request_multipart(
|
||||
"/v1/videos",
|
||||
{"model": model, "prompt": PROMPT, "seconds": "5", "size": "1280x720"},
|
||||
{"input_reference": ("frame.png", _PNG, "image/png")},
|
||||
)
|
||||
created_id(response)
|
||||
(body,) = create_bodies(wire.drain())
|
||||
assert body == {
|
||||
"model": MODEL,
|
||||
"input": {
|
||||
"prompt": PROMPT,
|
||||
"media": [{"type": "first_frame", "url": "data:image/png;base64," + base64.b64encode(_PNG).decode()}],
|
||||
},
|
||||
"parameters": {"ratio": "16:9", "resolution": "720P", "duration": 5},
|
||||
}
|
||||
|
||||
|
||||
def test_caller_parameters_reach_dashscope_as_written_beside_the_mapped_size_and_seconds(gateway: Gateway) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
created_id(
|
||||
create(
|
||||
gateway,
|
||||
{
|
||||
"model": model,
|
||||
"seconds": "8",
|
||||
"size": "720x1280",
|
||||
"negative_prompt": "blurry, low contrast",
|
||||
"parameters": {"prompt_extend": False, "watermark": False, "seed": 42, "shot_type": "multi"},
|
||||
},
|
||||
)
|
||||
)
|
||||
(body,) = create_bodies(wire.drain())
|
||||
assert body == {
|
||||
"model": MODEL,
|
||||
"input": {"prompt": PROMPT, "negative_prompt": "blurry, low contrast"},
|
||||
"parameters": {
|
||||
"ratio": "9:16",
|
||||
"resolution": "720P",
|
||||
"duration": 8,
|
||||
"prompt_extend": False,
|
||||
"watermark": False,
|
||||
"seed": 42,
|
||||
"shot_type": "multi",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def test_omitted_seconds_and_size_bill_dashscopes_five_second_1080p_default(gateway: Gateway) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
key: Final = scenario.key(models=[model])
|
||||
created_id(create(gateway, {"model": model}, key))
|
||||
(body,) = create_bodies(wire.drain())
|
||||
assert body == {"model": MODEL, "input": {"prompt": PROMPT}}
|
||||
settled_spend(key, 5 * rate("dashscope", _1080P))
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("field", "value", "message"),
|
||||
[
|
||||
("seconds", "10s", "seconds must be a whole number of seconds"),
|
||||
("seconds", "", "seconds must be a whole number of seconds"),
|
||||
("seconds", _HUGE, "seconds must be a whole number of seconds"),
|
||||
("seconds", ["5"], "seconds must be a whole number of seconds"),
|
||||
("seconds", True, "seconds must be a whole number of seconds"),
|
||||
("seconds", 2.5, "seconds must be a whole number of seconds"),
|
||||
("size", "abc", "size must be WIDTHxHEIGHT or a WIDTH:HEIGHT ratio"),
|
||||
("size", _HUGE, "size must be WIDTHxHEIGHT or a WIDTH:HEIGHT ratio"),
|
||||
("size", 1280, "size must be WIDTHxHEIGHT or a WIDTH:HEIGHT ratio"),
|
||||
("size", "0x720", "size must be WIDTHxHEIGHT or a WIDTH:HEIGHT ratio"),
|
||||
("parameters", "seed=1", "parameters must be an object of DashScope request fields"),
|
||||
("parameters", [1, 2], "parameters must be an object of DashScope request fields"),
|
||||
("parameters", {"seed": "abc"}, "invalid seed"),
|
||||
("parameters", {"seed": -1}, "invalid seed"),
|
||||
("parameters", {"audio": "yes"}, "invalid audio"),
|
||||
("parameters", {"duration": "5"}, "invalid duration"),
|
||||
],
|
||||
ids=[
|
||||
"seconds-suffixed",
|
||||
"seconds-empty",
|
||||
"seconds-5kb",
|
||||
"seconds-list",
|
||||
"seconds-bool",
|
||||
"seconds-fraction",
|
||||
"size-word",
|
||||
"size-5kb",
|
||||
"size-int",
|
||||
"size-zero-side",
|
||||
"parameters-string",
|
||||
"parameters-list",
|
||||
"parameters-seed-string",
|
||||
"parameters-seed-negative",
|
||||
"parameters-audio-string",
|
||||
"parameters-duration-string",
|
||||
],
|
||||
)
|
||||
def test_malformed_create_field_answers_400_before_any_task_is_created(
|
||||
gateway: Gateway, field: str, value: JsonValue, message: str
|
||||
) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
rejected: Final = create(gateway, {"model": model, "seconds": "5", "size": "1280x720", field: value})
|
||||
assert rejected.status_code == 400, rejected.text
|
||||
assert message in error_message(rejected), rejected.text
|
||||
assert wire.drain() == ()
|
||||
created_id(create(gateway, {"model": model, "seconds": "5", "size": "1280x720"}))
|
||||
assert len(create_bodies(wire.drain())) == 1
|
||||
|
||||
|
||||
def test_the_same_field_sent_twice_takes_the_last_value_and_still_validates(gateway: Gateway) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
raw: Final = (
|
||||
f'{{"model": "{model}", "prompt": "{PROMPT}", "seconds": "5", "seconds": "5s", "size": "1280x720"}}'
|
||||
).encode()
|
||||
rejected: Final = gateway.client.post(
|
||||
"/v1/videos",
|
||||
content=raw,
|
||||
headers={"Authorization": f"Bearer {gateway.key}", "Content-Type": "application/json"},
|
||||
)
|
||||
assert rejected.status_code == 400, rejected.text
|
||||
assert "seconds must be a whole number of seconds" in error_message(rejected)
|
||||
accepted: Final = gateway.client.post(
|
||||
"/v1/videos",
|
||||
content=raw.replace(b'"5s"', b'"6"'),
|
||||
headers={"Authorization": f"Bearer {gateway.key}", "Content-Type": "application/json"},
|
||||
)
|
||||
created_id(accepted)
|
||||
(body,) = create_bodies(wire.drain())
|
||||
assert object_value(body["parameters"])["duration"] == 6
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("body", "drop", "message"),
|
||||
[
|
||||
({"seconds": "-1"}, False, "Smart duration (duration=-1) is not supported"),
|
||||
({"parameters": {"duration": -1}}, True, "Smart duration (duration=-1) is not supported"),
|
||||
(
|
||||
{"media": [{"type": "reference_video", "url": "https://example.com/in.mp4"}]},
|
||||
False,
|
||||
"Video input (a reference_video media entry) is not supported",
|
||||
),
|
||||
(
|
||||
{"media": [{"type": "reference_video", "url": "https://example.com/in.mp4"}]},
|
||||
True,
|
||||
"Video input (a reference_video media entry) is not supported",
|
||||
),
|
||||
],
|
||||
ids=["smart-duration", "smart-duration-drop-params", "reference-video", "reference-video-drop-params"],
|
||||
)
|
||||
def test_requests_billed_before_their_length_is_known_answer_400_even_with_drop_params(
|
||||
gateway: Gateway, body: Mapping[str, JsonValue], drop: bool, message: str
|
||||
) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
rejected: Final = create(gateway, {"model": model, **body, **({"drop_params": True} if drop else {})})
|
||||
assert rejected.status_code == 400, rejected.text
|
||||
assert message in error_message(rejected), rejected.text
|
||||
assert wire.drain() == ()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("provider_model", "size", "message"),
|
||||
[
|
||||
("wan2.7-t2v", "832x480", "wan2.7-t2v is not offered at resolution '480P', only at 720P, 1080P"),
|
||||
("wan2.6-t2v", "1280x720", "size only maps onto DashScope video models in litellm's cost map"),
|
||||
],
|
||||
ids=["unpriced-tier", "unpriced-model"],
|
||||
)
|
||||
def test_a_size_litellm_cannot_bill_answers_400_without_a_task(
|
||||
gateway: Gateway, provider_model: str, size: str, message: str
|
||||
) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(
|
||||
gateway, scenario, model=f"dashscope/{provider_model}", api_base=wire.url, api_key=API_KEY
|
||||
)
|
||||
rejected: Final = create(gateway, {"model": model, "seconds": "5", "size": size})
|
||||
assert rejected.status_code == 400, rejected.text
|
||||
assert message in error_message(rejected), rejected.text
|
||||
assert wire.drain() == ()
|
||||
|
||||
|
||||
def test_a_dashscope_200_carrying_an_error_code_reaches_the_caller_as_400_after_one_attempt(gateway: Gateway) -> None:
|
||||
def respond(request: Request) -> Reply:
|
||||
return json_reply({"request_id": "req-1", "code": "InvalidParameter", "message": "The size is not supported."})
|
||||
|
||||
with wire_server(respond) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
rejected: Final = create(gateway, {"model": model, "seconds": "5", "size": "1280x720"})
|
||||
assert rejected.status_code == 400, rejected.text
|
||||
assert "The size is not supported." in error_message(rejected), rejected.text
|
||||
assert [(request.method, request.target) for request in wire.drain()] == [("POST", CREATE_PATH)]
|
||||
|
||||
|
||||
def test_a_dashscope_invalid_api_key_reaches_the_caller_as_401(gateway: Gateway) -> None:
|
||||
def respond(request: Request) -> Reply:
|
||||
return json_reply(
|
||||
{"request_id": "req-2", "code": "InvalidApiKey", "message": "Invalid API-key provided."}, status=401
|
||||
)
|
||||
|
||||
with wire_server(respond) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
rejected: Final = create(gateway, {"model": model, "seconds": "5", "size": "1280x720"})
|
||||
assert rejected.status_code == 401, rejected.text
|
||||
assert "Invalid API-key provided." in rejected.text
|
||||
assert [(request.method, request.target) for request in wire.drain()] == [("POST", CREATE_PATH)]
|
||||
|
||||
|
||||
def test_an_expired_task_polls_as_400_task_not_found_and_leaves_the_deployment_serving(gateway: Gateway) -> None:
|
||||
with wire_server(task_api(unknown)) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
video_id: Final = created_id(create(gateway, {"model": model, "seconds": "5", "size": "1280x720"}))
|
||||
missing: Final = tuple(
|
||||
gateway.request("GET", path)
|
||||
for path in (f"/v1/videos/{video_id}", f"/v1/videos/{video_id}", f"/v1/videos/{video_id}/content")
|
||||
)
|
||||
assert [response.status_code for response in missing] == [400, 400, 400], [r.text for r in missing]
|
||||
assert all(
|
||||
"Task not found. DashScope task ids expire 24 hours after creation" in error_message(response)
|
||||
for response in missing
|
||||
)
|
||||
created_id(create(gateway, {"model": model, "seconds": "5", "size": "1280x720"}))
|
||||
targets: Final = [(request.method, request.target) for request in wire.drain()]
|
||||
assert targets == [
|
||||
("POST", CREATE_PATH),
|
||||
("GET", f"{TASKS_PATH}{task_of(video_id)}"),
|
||||
("GET", f"{TASKS_PATH}{task_of(video_id)}"),
|
||||
("GET", f"{TASKS_PATH}{task_of(video_id)}"),
|
||||
("POST", CREATE_PATH),
|
||||
]
|
||||
|
||||
|
||||
def test_a_download_before_the_task_finishes_answers_400_without_touching_storage(gateway: Gateway) -> None:
|
||||
with wire_server(task_api(pending)) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
video_id: Final = created_id(create(gateway, {"model": model, "seconds": "5", "size": "1280x720"}))
|
||||
status: Final = gateway.get(f"/v1/videos/{video_id}")
|
||||
assert (status["id"], status["status"]) == (video_id, "queued")
|
||||
early: Final = gateway.request("GET", f"/v1/videos/{video_id}/content")
|
||||
assert early.status_code == 400, early.text
|
||||
assert "Video is still processing (status: PENDING)" in error_message(early)
|
||||
assert not any(request.target.startswith(OSS_PATH) for request in wire.drain())
|
||||
|
||||
|
||||
def _failed(task_id: str, origin: str) -> Mapping[str, JsonValue]:
|
||||
return {
|
||||
"request_id": f"req-{task_id}",
|
||||
"output": {
|
||||
"task_id": task_id,
|
||||
"task_status": "FAILED",
|
||||
"code": "DataInspectionFailed",
|
||||
"message": "Output data may contain inappropriate content.",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def test_a_failed_task_reports_failed_with_dashscopes_error_and_its_download_answers_400(gateway: Gateway) -> None:
|
||||
with wire_server(task_api(_failed)) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
video_id: Final = created_id(create(gateway, {"model": model, "seconds": "5", "size": "1280x720"}))
|
||||
status: Final = gateway.get(f"/v1/videos/{video_id}")
|
||||
assert status["status"] == "failed", status
|
||||
assert status["error"] == {
|
||||
"code": "DataInspectionFailed",
|
||||
"message": "Output data may contain inappropriate content.",
|
||||
}
|
||||
download: Final = gateway.request("GET", f"/v1/videos/{video_id}/content")
|
||||
assert download.status_code == 400, download.text
|
||||
assert "Video generation failed: Output data may contain inappropriate content." in error_message(download)
|
||||
assert not any(request.target.startswith(OSS_PATH) for request in wire.drain())
|
||||
|
||||
|
||||
def _unexpected_status(task_id: str, origin: str) -> Mapping[str, JsonValue]:
|
||||
return {"request_id": f"req-{task_id}", "output": {"task_id": task_id, "task_status": "ARCHIVED"}}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shape", ["unknown-status", "not-json"])
|
||||
def test_a_task_body_dashscope_does_not_document_answers_502(gateway: Gateway, shape: str) -> None:
|
||||
created: Final = task_api()
|
||||
|
||||
def respond(request: Request) -> Reply:
|
||||
if request.method == "POST":
|
||||
return created(request)
|
||||
if shape == "not-json":
|
||||
return Reply(body=b"<html>gateway timeout</html>", content_type="text/html")
|
||||
return json_reply(_unexpected_status(request.target.removeprefix(TASKS_PATH), ""))
|
||||
|
||||
with wire_server(respond) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
video_id: Final = created_id(create(gateway, {"model": model, "seconds": "5", "size": "1280x720"}))
|
||||
polled: Final = gateway.request("GET", f"/v1/videos/{video_id}")
|
||||
assert polled.status_code == 502, polled.text
|
||||
assert (
|
||||
"Unknown DashScope task status: ARCHIVED"
|
||||
if shape == "unknown-status"
|
||||
else "Unexpected DashScope video response"
|
||||
) in error_message(polled)
|
||||
assert len(wire.drain()) == 2
|
||||
|
||||
|
||||
def test_storage_refusing_the_download_reaches_the_caller_as_an_error_and_the_proxy_keeps_serving(
|
||||
gateway: Gateway,
|
||||
) -> None:
|
||||
refused: Final = Reply(status=403, body=b"<Error><Code>AccessDenied</Code></Error>", content_type="application/xml")
|
||||
with wire_server(task_api(oss=refused)) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
video_id: Final = created_id(create(gateway, {"model": model, "seconds": "5", "size": "1280x720"}))
|
||||
download: Final = gateway.request("GET", f"/v1/videos/{video_id}/content")
|
||||
assert download.status_code == 403, download.text
|
||||
assert download.headers["content-type"].startswith("application/json"), download.headers
|
||||
assert "AccessDenied" in error_message(download), download.text
|
||||
assert gateway.get(f"/v1/videos/{video_id}")["status"] == "completed"
|
||||
assert [request.target for request in wire.drain()] == [
|
||||
CREATE_PATH,
|
||||
f"{TASKS_PATH}{task_of(video_id)}",
|
||||
f"{OSS_PATH}{task_of(video_id)}.mp4",
|
||||
f"{TASKS_PATH}{task_of(video_id)}",
|
||||
]
|
||||
|
||||
|
||||
def test_the_proxy_has_no_video_delete_route_so_a_dashscope_task_is_never_touched(gateway: Gateway) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
video_id: Final = created_id(create(gateway, {"model": model, "seconds": "5", "size": "1280x720"}))
|
||||
wire.drain()
|
||||
response: Final = gateway.request("DELETE", f"/v1/videos/{video_id}")
|
||||
assert response.status_code == 405, response.text
|
||||
assert wire.drain() == ()
|
||||
assert gateway.get(f"/v1/videos/{video_id}")["status"] == "completed"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("operation", "message"),
|
||||
[("remix", "Video remix is not supported by DashScope"), ("list", "DashScope API key is required")],
|
||||
)
|
||||
def test_operations_dashscope_has_no_api_for_answer_an_error_and_send_nothing_upstream(
|
||||
gateway: Gateway, operation: str, message: str
|
||||
) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
video_id: Final = created_id(create(gateway, {"model": model, "seconds": "5", "size": "1280x720"}))
|
||||
wire.drain()
|
||||
response: Final = (
|
||||
gateway.request("POST", f"/v1/videos/{video_id}/remix", {"prompt": "now at dusk"})
|
||||
if operation == "remix"
|
||||
else gateway.request("GET", "/v1/videos", params={"custom_llm_provider": "dashscope"})
|
||||
)
|
||||
assert response.status_code >= 400, response.text
|
||||
assert message in error_message(response), response.text
|
||||
assert wire.drain() == ()
|
||||
assert gateway.get(f"/v1/videos/{video_id}")["status"] == "completed"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("level", ["request", "deployment"])
|
||||
def test_drop_params_drops_malformed_fields_and_unpriced_tiers_instead_of_answering_400(
|
||||
gateway: Gateway, level: str
|
||||
) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario,
|
||||
model="dashscope/wan2.7-t2v",
|
||||
api_base=wire.url,
|
||||
api_key=API_KEY,
|
||||
**({"drop_params": True} if level == "deployment" else {}),
|
||||
)
|
||||
key: Final = scenario.key(models=[model])
|
||||
created_id(
|
||||
create(
|
||||
gateway,
|
||||
{
|
||||
"model": model,
|
||||
"seconds": "10s",
|
||||
"size": "832x480",
|
||||
"parameters": {"seed": "abc", "watermark": True},
|
||||
**({"drop_params": True} if level == "request" else {}),
|
||||
},
|
||||
key,
|
||||
)
|
||||
)
|
||||
(body,) = create_bodies(wire.drain())
|
||||
assert body == {
|
||||
"model": "wan2.7-t2v",
|
||||
"input": {"prompt": PROMPT},
|
||||
"parameters": {"ratio": "16:9", "watermark": True},
|
||||
}
|
||||
expected: Final = 5 * rate("dashscope", _1080P, "wan2.7-t2v")
|
||||
settled_spend(key, expected)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"parameters", [{}, None, "missing"], ids=["empty-object", "null", "missing"]
|
||||
)
|
||||
def test_empty_null_and_missing_parameters_send_the_same_task_body(gateway: Gateway, parameters: JsonValue) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
created_id(
|
||||
create(
|
||||
gateway,
|
||||
{"model": model, "seconds": "5", **({} if parameters == "missing" else {"parameters": parameters})},
|
||||
)
|
||||
)
|
||||
(body,) = create_bodies(wire.drain())
|
||||
assert body == {"model": MODEL, "input": {"prompt": PROMPT}, "parameters": {"duration": 5}}
|
||||
|
||||
|
||||
def test_two_identical_creates_are_two_tasks_each_billed_once(gateway: Gateway) -> None:
|
||||
with wire_server(task_api()) as wire, gateway.scenario() as scenario:
|
||||
model: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=wire.url, api_key=API_KEY)
|
||||
key: Final = scenario.key(models=[model])
|
||||
first: Final = created_id(create(gateway, {"model": model, "seconds": "5", "size": "832x480"}, key))
|
||||
second: Final = created_id(create(gateway, {"model": model, "seconds": "5", "size": "832x480"}, key))
|
||||
assert first != second
|
||||
assert task_of(first) != task_of(second)
|
||||
bodies: Final = create_bodies(wire.drain())
|
||||
assert len(bodies) == 2 and bodies[0] == bodies[1]
|
||||
expected: Final = 2 * 5 * rate("dashscope", "output_cost_per_second_480p")
|
||||
settled_spend(key, expected)
|
||||
|
||||
|
||||
async def test_litellm_sdk_sync_and_async_create_poll_and_download_through_dashscope() -> None:
|
||||
import litellm
|
||||
|
||||
with wire_server(task_api()) as wire:
|
||||
created: Final = litellm.video_generation(
|
||||
model=f"dashscope/{MODEL}", prompt=PROMPT, seconds="5", size="1280x720", api_base=wire.url, api_key=API_KEY
|
||||
)
|
||||
status: Final = litellm.video_status(video_id=created.id, api_base=wire.url, api_key=API_KEY)
|
||||
assert (status.id, status.status) == (created.id, "completed")
|
||||
assert litellm.video_content(video_id=created.id, api_base=wire.url, api_key=API_KEY) == MP4
|
||||
async_created: Final = await litellm.avideo_generation(
|
||||
model=f"dashscope/{MODEL}", prompt=PROMPT, seconds="5", size="1280x720", api_base=wire.url, api_key=API_KEY
|
||||
)
|
||||
async_status: Final = await litellm.avideo_status(video_id=async_created.id, api_base=wire.url, api_key=API_KEY)
|
||||
assert (async_status.id, async_status.status) == (async_created.id, "completed")
|
||||
assert await litellm.avideo_content(video_id=async_created.id, api_base=wire.url, api_key=API_KEY) == MP4
|
||||
bodies: Final = create_bodies(wire.drain())
|
||||
assert bodies == tuple(
|
||||
{
|
||||
"model": MODEL,
|
||||
"input": {"prompt": PROMPT},
|
||||
"parameters": {"ratio": "16:9", "resolution": "720P", "duration": 5},
|
||||
}
|
||||
for _ in range(2)
|
||||
)
|
||||
450
tests/integration/routing/test_video_poll_routing_wire.py
Normal file
450
tests/integration/routing/test_video_poll_routing_wire.py
Normal file
|
|
@ -0,0 +1,450 @@
|
|||
import threading
|
||||
import uuid
|
||||
from collections.abc import Callable, Iterator, Mapping
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from contextlib import ExitStack
|
||||
from dataclasses import dataclass
|
||||
from typing import Final
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from integration._support.client import (
|
||||
JSON_OBJECT,
|
||||
Gateway,
|
||||
Scenario,
|
||||
eventually,
|
||||
gateway_from_environment,
|
||||
list_value,
|
||||
object_value,
|
||||
string_value,
|
||||
)
|
||||
from integration._support.wire import Reply, Request, Wire, wire_server
|
||||
from integration.providers._dashscope_video_support import (
|
||||
API_KEY,
|
||||
MODEL,
|
||||
MP4,
|
||||
MP4_REPLY,
|
||||
OSS_PATH,
|
||||
PROMPT,
|
||||
create,
|
||||
create_bodies,
|
||||
created_id,
|
||||
decoded,
|
||||
deployed,
|
||||
error_message,
|
||||
forged_id,
|
||||
json_reply,
|
||||
polled_tasks,
|
||||
proxy_url,
|
||||
rate,
|
||||
settled_spend,
|
||||
task_api,
|
||||
task_of,
|
||||
)
|
||||
from pydantic import JsonValue
|
||||
|
||||
pytestmark: Final = pytest.mark.xdist_group("video-poll-routing")
|
||||
|
||||
_OPENAI_VIDEOS: Final = "/v1/videos"
|
||||
_BURST: Final = 12
|
||||
_OPENAI_KEY: Final = "synthetic-openai-key"
|
||||
|
||||
|
||||
def _openai_videos(prefix: str) -> Callable[[Request], Reply]:
|
||||
def video(video_id: str, status: str) -> Mapping[str, JsonValue]:
|
||||
return {
|
||||
"id": video_id,
|
||||
"object": "video",
|
||||
"model": "sora-2",
|
||||
"status": status,
|
||||
"progress": 100 if status == "completed" else 0,
|
||||
"created_at": 1791943200,
|
||||
"seconds": "4",
|
||||
"size": "720x1280",
|
||||
}
|
||||
|
||||
def respond(request: Request) -> Reply:
|
||||
if request.method == "POST":
|
||||
assert request.target == _OPENAI_VIDEOS, request.target
|
||||
return json_reply(video(f"video_{prefix}{uuid.uuid4().hex}", "queued"))
|
||||
assert request.method == "GET" and request.target.startswith(f"{_OPENAI_VIDEOS}/"), request.target
|
||||
video_id: Final = request.target.removeprefix(f"{_OPENAI_VIDEOS}/").removesuffix("/content")
|
||||
if not video_id.startswith(f"video_{prefix}"):
|
||||
return json_reply(
|
||||
{"error": {"message": f"Video {video_id} not found", "type": "invalid_request_error"}}, 404
|
||||
)
|
||||
if request.target.endswith("/content"):
|
||||
return MP4_REPLY
|
||||
return json_reply(video(video_id, "completed"))
|
||||
|
||||
return respond
|
||||
|
||||
|
||||
def _outage_switch(outage: threading.Event, respond: Callable[[Request], Reply]) -> Callable[[Request], Reply]:
|
||||
def switched(request: Request) -> Reply:
|
||||
return Reply(drop_connection=True) if outage.is_set() else respond(request)
|
||||
|
||||
return switched
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _Rig:
|
||||
gateway: Gateway
|
||||
scenario: Scenario
|
||||
a: Wire
|
||||
b: Wire
|
||||
b_outage: threading.Event
|
||||
first: Wire
|
||||
second: Wire
|
||||
member_a: str
|
||||
member_b: str
|
||||
group: str
|
||||
alias: str
|
||||
openai_first: str
|
||||
openai_second: str
|
||||
openai_alias: str
|
||||
|
||||
def drain(self) -> None:
|
||||
for wire in (self.a, self.b, self.first, self.second):
|
||||
wire.drain()
|
||||
|
||||
|
||||
def _fresh_status(gateway: Gateway, body: Mapping[str, JsonValue]) -> int:
|
||||
with httpx.Client(base_url=proxy_url(gateway), timeout=30, trust_env=False) as client:
|
||||
return client.post(
|
||||
"/v1/videos", json={"prompt": PROMPT, **body}, headers={"Authorization": f"Bearer {gateway.key}"}
|
||||
).status_code
|
||||
|
||||
|
||||
def _burst(gateway: Gateway, body: Mapping[str, JsonValue]) -> frozenset[int]:
|
||||
with ThreadPoolExecutor(max_workers=_BURST) as pool:
|
||||
return frozenset(pool.map(lambda _: _fresh_status(gateway, body), range(_BURST)))
|
||||
|
||||
|
||||
def _settled(gateway: Gateway, body: Mapping[str, JsonValue]) -> None:
|
||||
eventually(
|
||||
lambda: _burst(gateway, body) | _burst(gateway, body), lambda seen: seen == frozenset({200}), seconds=90
|
||||
)
|
||||
|
||||
|
||||
def _current(gateway: Gateway, field: str) -> JsonValue:
|
||||
return object_value(gateway.get("/router/settings")["current_values"]).get(field)
|
||||
|
||||
|
||||
def _router_settings(gateway: Gateway, settings: Mapping[str, JsonValue]) -> None:
|
||||
gateway.post("/config/update", {"router_settings": dict(settings)})
|
||||
|
||||
|
||||
def _without_ours(gateway: Gateway, group: str, aliases: frozenset[str]) -> Mapping[str, JsonValue]:
|
||||
groups: Final = list_value(_current(gateway, "routing_groups") or [])
|
||||
current_aliases: Final = object_value(_current(gateway, "model_group_alias") or {})
|
||||
return {
|
||||
"routing_groups": [entry for entry in groups if object_value(entry)["group_name"] != group],
|
||||
"model_group_alias": {name: target for name, target in current_aliases.items() if name not in aliases},
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def rig() -> Iterator[_Rig]:
|
||||
suffix: Final = uuid.uuid4().hex[:10]
|
||||
b_outage: Final = threading.Event()
|
||||
with (
|
||||
gateway_from_environment() as gateway,
|
||||
wire_server(task_api(prefix="a")) as a,
|
||||
wire_server(_outage_switch(b_outage, task_api(prefix="b"))) as b,
|
||||
wire_server(_openai_videos("first")) as first,
|
||||
wire_server(_openai_videos("second")) as second,
|
||||
gateway.scenario() as scenario,
|
||||
ExitStack() as restore,
|
||||
):
|
||||
member_a: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=a.url, api_key=API_KEY)
|
||||
member_b: Final = deployed(gateway, scenario, model=f"dashscope/{MODEL}", api_base=b.url, api_key=API_KEY)
|
||||
openai_first: Final = deployed(
|
||||
gateway, scenario, model=f"openai/sora-{suffix}-first", api_base=f"{first.url}/v1", api_key=_OPENAI_KEY
|
||||
)
|
||||
openai_second: Final = deployed(
|
||||
gateway, scenario, model=f"openai/sora-{suffix}-second", api_base=f"{second.url}/v1", api_key=_OPENAI_KEY
|
||||
)
|
||||
built: Final = _Rig(
|
||||
gateway=gateway,
|
||||
scenario=scenario,
|
||||
a=a,
|
||||
b=b,
|
||||
b_outage=b_outage,
|
||||
first=first,
|
||||
second=second,
|
||||
member_a=member_a,
|
||||
member_b=member_b,
|
||||
group=f"video-group-{suffix}",
|
||||
alias=f"video-alias-{suffix}",
|
||||
openai_first=openai_first,
|
||||
openai_second=openai_second,
|
||||
openai_alias=f"openai-video-alias-{suffix}",
|
||||
)
|
||||
aliases: Final = frozenset({built.alias, built.openai_alias})
|
||||
kept: Final = _without_ours(gateway, built.group, aliases)
|
||||
restore.callback(lambda: _router_settings(gateway, _without_ours(gateway, built.group, aliases)))
|
||||
_router_settings(
|
||||
gateway,
|
||||
{
|
||||
"routing_groups": [
|
||||
*list_value(kept["routing_groups"]),
|
||||
{"group_name": built.group, "models": [member_a, member_b], "routing_strategy": "simple-shuffle"},
|
||||
],
|
||||
"model_group_alias": {
|
||||
**object_value(kept["model_group_alias"]),
|
||||
built.alias: member_b,
|
||||
built.openai_alias: openai_first,
|
||||
},
|
||||
},
|
||||
)
|
||||
_settled(gateway, {"model": built.openai_alias, "seconds": "4"})
|
||||
built.drain()
|
||||
yield built
|
||||
|
||||
|
||||
def _poll(rig: _Rig, video_id: str, key: str | None = None) -> httpx.Response:
|
||||
return rig.gateway.request("GET", f"/v1/videos/{video_id}", key=key)
|
||||
|
||||
|
||||
def _download(rig: _Rig, video_id: str, key: str | None = None) -> httpx.Response:
|
||||
return rig.gateway.request("GET", f"/v1/videos/{video_id}/content", key=key)
|
||||
|
||||
|
||||
def _assert_served(response: httpx.Response, video_id: str) -> None:
|
||||
assert response.status_code == 200, response.text
|
||||
body: Final = JSON_OBJECT.validate_json(response.content)
|
||||
assert (body["id"], body["status"]) == (video_id, "completed"), body
|
||||
|
||||
|
||||
def test_a_key_scoped_to_an_alias_polls_and_downloads_from_the_aliased_deployment(rig: _Rig) -> None:
|
||||
key: Final = rig.scenario.key(models=[rig.alias])
|
||||
rig.drain()
|
||||
video_id: Final = created_id(create(rig.gateway, {"model": rig.alias, "seconds": "5", "size": "1280x720"}, key))
|
||||
assert {name: decoded(video_id).get(name) for name in ("model_id", "served_model")} == {
|
||||
"model_id": rig.alias,
|
||||
"served_model": rig.member_b,
|
||||
}
|
||||
_assert_served(_poll(rig, video_id, key), video_id)
|
||||
download: Final = _download(rig, video_id, key)
|
||||
assert download.status_code == 200, download.text
|
||||
assert download.content == MP4
|
||||
assert rig.a.drain() == ()
|
||||
task: Final = task_of(video_id)
|
||||
assert polled_tasks(rig.b.drain()) == (task, task)
|
||||
|
||||
|
||||
def test_a_routing_group_sends_every_poll_and_download_back_to_the_member_that_created_the_task(rig: _Rig) -> None:
|
||||
key: Final = rig.scenario.key(models=[rig.group])
|
||||
|
||||
def create_one(_index: int) -> str:
|
||||
return created_id(create(rig.gateway, {"model": rig.group, "seconds": "5", "size": "1280x720"}, key))
|
||||
|
||||
video_ids: Final = eventually(
|
||||
lambda: tuple(create_one(index) for index in range(_BURST)),
|
||||
lambda ids: {task_of(video_id)[0] for video_id in ids} == {"a", "b"},
|
||||
seconds=60,
|
||||
)
|
||||
served: Final = {video_id: decoded(video_id) for video_id in video_ids}
|
||||
assert all(entry["model_id"] == rig.group for entry in served.values()), served
|
||||
assert all(
|
||||
entry["served_model"] == (rig.member_a if task_of(video_id).startswith("a-") else rig.member_b)
|
||||
for video_id, entry in served.items()
|
||||
), served
|
||||
rig.drain()
|
||||
polls: Final = tuple((video_id, _poll(rig, video_id, key), _download(rig, video_id, key)) for video_id in video_ids)
|
||||
for video_id, status, download in polls:
|
||||
_assert_served(status, video_id)
|
||||
assert download.status_code == 200 and download.content == MP4, download.text
|
||||
a_tasks: Final = tuple(task_of(video_id) for video_id in video_ids if task_of(video_id).startswith("a-"))
|
||||
b_tasks: Final = tuple(task_of(video_id) for video_id in video_ids if task_of(video_id).startswith("b-"))
|
||||
assert sorted(polled_tasks(rig.a.drain())) == sorted((*a_tasks, *a_tasks))
|
||||
assert sorted(polled_tasks(rig.b.drain())) == sorted((*b_tasks, *b_tasks))
|
||||
|
||||
|
||||
def test_a_key_rate_limit_on_a_routing_group_counts_its_polls_against_the_group(rig: _Rig) -> None:
|
||||
video_id: Final = created_id(create(rig.gateway, {"model": rig.group, "seconds": "5", "size": "1280x720"}))
|
||||
served, other = (rig.b, rig.a) if task_of(video_id).startswith("b-") else (rig.a, rig.b)
|
||||
key: Final = rig.scenario.key(models=[rig.group], model_rpm_limit={rig.group: 2})
|
||||
rig.drain()
|
||||
polls: Final = tuple(_poll(rig, video_id, key) for _ in range(3))
|
||||
assert [response.status_code for response in polls] == [200, 200, 429], [response.text for response in polls]
|
||||
assert f":{rig.group}. Limit type: requests." in error_message(polls[2]), polls[2].text
|
||||
assert other.drain() == ()
|
||||
assert polled_tasks(served.drain()) == (task_of(video_id), task_of(video_id))
|
||||
|
||||
|
||||
def test_a_key_scoped_to_a_wildcard_route_gets_an_id_without_a_served_group_and_polls_through_the_wildcard(
|
||||
rig: _Rig,
|
||||
) -> None:
|
||||
prefix: Final = f"video-wild-{uuid.uuid4().hex[:8]}"
|
||||
created: Final = rig.gateway.post(
|
||||
"/model/new",
|
||||
{
|
||||
"model_name": f"{prefix}/*",
|
||||
"litellm_params": {"model": "dashscope/*", "api_base": rig.a.url, "api_key": API_KEY},
|
||||
"model_info": {},
|
||||
},
|
||||
)
|
||||
rig.scenario.cleanups.callback(rig.scenario.delete_model, string_value(object_value(created["model_info"])["id"]))
|
||||
requested: Final = f"{prefix}/{MODEL}"
|
||||
_settled(rig.gateway, {"model": requested, "seconds": "4"})
|
||||
rig.drain()
|
||||
key: Final = rig.scenario.key(models=[f"{prefix}/*"])
|
||||
video_id: Final = created_id(create(rig.gateway, {"model": requested, "seconds": "5", "size": "1280x720"}, key))
|
||||
assert decoded(video_id).get("model_id") == requested
|
||||
assert decoded(video_id).get("served_model") is None
|
||||
_assert_served(_poll(rig, video_id, key), video_id)
|
||||
received: Final = rig.a.drain()
|
||||
assert JSON_OBJECT.validate_json(received[0].body)["model"] == MODEL
|
||||
assert polled_tasks(received) == (task_of(video_id),)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("level", ["key", "team"])
|
||||
def test_an_alias_to_one_member_keeps_a_forged_served_member_from_moving_the_poll(rig: _Rig, level: str) -> None:
|
||||
task: Final = f"a-{uuid.uuid4().hex}"
|
||||
forged: Final = forged_id("dashscope", rig.group, task, rig.member_a)
|
||||
team: Final = (
|
||||
rig.scenario.team(models=[rig.group, rig.member_b], model_aliases={rig.group: rig.member_b})
|
||||
if level == "team"
|
||||
else None
|
||||
)
|
||||
key: Final = (
|
||||
rig.scenario.key(team_id=team)
|
||||
if team is not None
|
||||
else rig.scenario.key(models=[rig.group, rig.member_b], aliases={rig.group: rig.member_b})
|
||||
)
|
||||
rig.drain()
|
||||
status: Final = _poll(rig, forged, key)
|
||||
download: Final = _download(rig, forged, key)
|
||||
for response in (status, download):
|
||||
assert response.status_code == 400, response.text
|
||||
assert "Task not found" in error_message(response), response.text
|
||||
assert rig.a.drain() == ()
|
||||
assert polled_tasks(rig.b.drain()) == (task, task)
|
||||
|
||||
|
||||
def test_a_served_group_the_ids_own_model_does_not_route_to_is_ignored(rig: _Rig) -> None:
|
||||
video_id: Final = created_id(create(rig.gateway, {"model": rig.member_a, "seconds": "5", "size": "1280x720"}))
|
||||
task: Final = task_of(video_id)
|
||||
forged: Final = forged_id("dashscope", rig.member_a, task, rig.member_b)
|
||||
rig.drain()
|
||||
_assert_served(_poll(rig, forged), video_id)
|
||||
assert rig.b.drain() == ()
|
||||
assert polled_tasks(rig.a.drain()) == (task,)
|
||||
|
||||
|
||||
def test_an_id_without_a_served_group_still_polls_its_own_deployment(rig: _Rig) -> None:
|
||||
video_id: Final = created_id(create(rig.gateway, {"model": rig.member_a, "seconds": "5", "size": "1280x720"}))
|
||||
assert decoded(video_id).get("served_model") is None
|
||||
legacy: Final = forged_id("dashscope", rig.member_a, task_of(video_id), None)
|
||||
assert legacy == video_id
|
||||
rig.drain()
|
||||
_assert_served(_poll(rig, legacy), legacy)
|
||||
download: Final = _download(rig, legacy)
|
||||
assert download.status_code == 200 and download.content == MP4, download.text
|
||||
received: Final = rig.a.drain()
|
||||
assert polled_tasks(received) == (task_of(video_id), task_of(video_id))
|
||||
assert [request.target for request in received if request.target.startswith(OSS_PATH)] == [
|
||||
f"{OSS_PATH}{task_of(video_id)}.mp4"
|
||||
]
|
||||
|
||||
|
||||
def test_a_forged_served_group_on_an_openai_id_never_reaches_the_other_deployment(rig: _Rig) -> None:
|
||||
video_id: Final = created_id(create(rig.gateway, {"model": rig.openai_first, "seconds": "4"}))
|
||||
original: Final = decoded(video_id)
|
||||
forged: Final = forged_id(
|
||||
"openai", string_value(original["model_id"] or ""), string_value(original["video_id"] or ""), rig.openai_second
|
||||
)
|
||||
rig.drain()
|
||||
status: Final = _poll(rig, forged)
|
||||
assert status.status_code == 200, status.text
|
||||
assert JSON_OBJECT.validate_json(status.content)["status"] == "completed"
|
||||
assert rig.second.drain() == ()
|
||||
assert [(request.method, request.target) for request in rig.first.drain()] == [
|
||||
("GET", f"{_OPENAI_VIDEOS}/{original['video_id']}")
|
||||
]
|
||||
|
||||
|
||||
def test_an_openai_video_created_through_a_narrow_wildcard_polls_through_the_wildcard(rig: _Rig) -> None:
|
||||
prefix: Final = f"sora-{uuid.uuid4().hex[:8]}"
|
||||
created: Final = rig.gateway.post(
|
||||
"/model/new",
|
||||
{
|
||||
"model_name": f"{prefix}-*",
|
||||
"litellm_params": {
|
||||
"model": f"openai/{prefix}-*",
|
||||
"api_base": f"{rig.first.url}/v1",
|
||||
"api_key": _OPENAI_KEY,
|
||||
},
|
||||
"model_info": {},
|
||||
},
|
||||
)
|
||||
rig.scenario.cleanups.callback(rig.scenario.delete_model, string_value(object_value(created["model_info"])["id"]))
|
||||
requested: Final = f"{prefix}-pro"
|
||||
_settled(rig.gateway, {"model": requested, "seconds": "4"})
|
||||
rig.drain()
|
||||
video_id: Final = created_id(create(rig.gateway, {"model": requested, "seconds": "4"}))
|
||||
status: Final = _poll(rig, video_id)
|
||||
assert status.status_code == 200, status.text
|
||||
assert JSON_OBJECT.validate_json(status.content)["status"] == "completed"
|
||||
content: Final = _download(rig, video_id)
|
||||
assert content.status_code == 200 and content.content == MP4, content.text
|
||||
original: Final = decoded(video_id)["video_id"]
|
||||
assert [(request.method, request.target) for request in rig.first.drain()] == [
|
||||
("POST", _OPENAI_VIDEOS),
|
||||
("GET", f"{_OPENAI_VIDEOS}/{original}"),
|
||||
("GET", f"{_OPENAI_VIDEOS}/{original}/content"),
|
||||
]
|
||||
|
||||
|
||||
def test_an_openai_video_created_through_an_alias_polls_and_downloads_from_the_alias_target(rig: _Rig) -> None:
|
||||
key: Final = rig.scenario.key(models=[rig.openai_alias])
|
||||
rig.drain()
|
||||
video_id: Final = created_id(create(rig.gateway, {"model": rig.openai_alias, "seconds": "4"}, key))
|
||||
status: Final = _poll(rig, video_id, key)
|
||||
assert status.status_code == 200, status.text
|
||||
content: Final = _download(rig, video_id, key)
|
||||
assert content.status_code == 200 and content.content == MP4, content.text
|
||||
original: Final = decoded(video_id)["video_id"]
|
||||
assert rig.second.drain() == ()
|
||||
assert [(request.method, request.target) for request in rig.first.drain()] == [
|
||||
("POST", _OPENAI_VIDEOS),
|
||||
("GET", f"{_OPENAI_VIDEOS}/{original}"),
|
||||
("GET", f"{_OPENAI_VIDEOS}/{original}/content"),
|
||||
]
|
||||
|
||||
|
||||
def test_a_member_outage_mid_burst_fails_cleanly_and_every_served_task_polls_on_its_creating_member(
|
||||
rig: _Rig,
|
||||
) -> None:
|
||||
key: Final = rig.scenario.key(models=[rig.group])
|
||||
body: Final = {"model": rig.group, "seconds": "5", "size": "1280x720"}
|
||||
rig.drain()
|
||||
rig.b_outage.set()
|
||||
try:
|
||||
with ThreadPoolExecutor(max_workers=16) as pool:
|
||||
during: Final = tuple(pool.map(lambda _: create(rig.gateway, body, key), range(16)))
|
||||
finally:
|
||||
rig.b_outage.clear()
|
||||
failed: Final = tuple(response for response in during if response.status_code != 200)
|
||||
assert all(error_message(response) for response in failed), [response.text for response in failed]
|
||||
served_during: Final = tuple(created_id(response) for response in during if response.status_code == 200)
|
||||
assert all(task_of(video_id).startswith("a-") for video_id in served_during), served_during
|
||||
assert len(create_bodies(rig.a.drain())) == len(served_during)
|
||||
assert create_bodies(rig.b.drain()) != ()
|
||||
with ThreadPoolExecutor(max_workers=12) as pool:
|
||||
recovered: Final = tuple(pool.map(lambda _: created_id(create(rig.gateway, body, key)), range(24)))
|
||||
assert {task_of(video_id)[0] for video_id in recovered} == {"a", "b"}
|
||||
video_ids: Final = (*served_during, *recovered)
|
||||
rig.drain()
|
||||
with ThreadPoolExecutor(max_workers=12) as pool:
|
||||
polls: Final = tuple(pool.map(lambda video_id: (video_id, _poll(rig, video_id, key)), video_ids))
|
||||
for video_id, response in polls:
|
||||
_assert_served(response, video_id)
|
||||
a_tasks: Final = sorted(task_of(video_id) for video_id in video_ids if task_of(video_id).startswith("a-"))
|
||||
b_tasks: Final = sorted(task_of(video_id) for video_id in video_ids if task_of(video_id).startswith("b-"))
|
||||
assert sorted(polled_tasks(rig.a.drain())) == a_tasks
|
||||
assert sorted(polled_tasks(rig.b.drain())) == b_tasks
|
||||
expected: Final = len(video_ids) * 5 * rate("dashscope", "output_cost_per_second_720p")
|
||||
settled_spend(key, expected)
|
||||
1421
tests/unit/llms/dashscope/test_dashscope_video_transformation.py
Normal file
1421
tests/unit/llms/dashscope/test_dashscope_video_transformation.py
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -8783,6 +8783,76 @@ class TestPerRequestModelGroupAlias:
|
|||
assert merged_for == ["group-b"]
|
||||
|
||||
|
||||
class TestVideoServedModelPin:
|
||||
"""A video id names the group a poll asked for and the group that served the task. The served group may only
|
||||
be pinned when the name left after every alias rewrite routes to it, so an edited id cannot undo a key's alias,
|
||||
and only after the pre-call hooks, so a key's rate limits on that name still count the poll, and never over a
|
||||
model a hook picked."""
|
||||
|
||||
@staticmethod
|
||||
def _router() -> litellm.Router:
|
||||
return litellm.Router(
|
||||
model_list=[
|
||||
{"model_name": name, "litellm_params": {"model": "openai/sora-2", "api_key": "sk-fake"}}
|
||||
for name in ("allowed", "restricted")
|
||||
],
|
||||
routing_groups=[
|
||||
{"group_name": "videos", "models": ["allowed", "restricted"], "routing_strategy": "simple-shuffle"}
|
||||
],
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
("key_aliases", "router_settings", "hook_rewrite", "hooked", "expected"),
|
||||
[
|
||||
({}, None, None, "videos", "restricted"),
|
||||
({"videos": "allowed"}, None, None, "allowed", "allowed"),
|
||||
({}, {"model_group_alias": {"videos": "allowed"}}, None, "allowed", "allowed"),
|
||||
({}, None, "allowed", "videos", "allowed"),
|
||||
],
|
||||
ids=["no-alias-pins-the-served-group", "key-alias-wins", "key-or-team-router-alias-wins", "hook-rewrite-wins"],
|
||||
)
|
||||
async def test_served_group_is_pinned_after_the_hooks_only_when_the_aliased_name_routes_to_it(
|
||||
self, monkeypatch, key_aliases, router_settings, hook_rewrite, hooked, expected
|
||||
):
|
||||
processing_obj = ProxyBaseLLMRequestProcessing(
|
||||
data={"model": "videos", "video_id": "video_abc"}, served_model="restricted"
|
||||
)
|
||||
mock_request = MagicMock(spec=Request)
|
||||
mock_request.headers = {}
|
||||
|
||||
async def mock_add_litellm_data_to_request(*args, **kwargs):
|
||||
return kwargs.get("data", {})
|
||||
|
||||
async def pre_call_hook(user_api_key_dict, data, call_type, skip_guardrails=False):
|
||||
hooked_data = copy.deepcopy(data)
|
||||
return hooked_data if hook_rewrite is None else {**hooked_data, "model": hook_rewrite}
|
||||
|
||||
mock_proxy_logging_obj = MagicMock(spec=ProxyLogging)
|
||||
mock_proxy_logging_obj.pre_call_hook = AsyncMock(side_effect=pre_call_hook)
|
||||
monkeypatch.setattr(
|
||||
litellm.proxy.common_request_processing,
|
||||
"add_litellm_data_to_request",
|
||||
mock_add_litellm_data_to_request,
|
||||
)
|
||||
monkeypatch.setattr("litellm.proxy.proxy_server.prisma_client", MagicMock())
|
||||
mock_proxy_config = MagicMock(spec=ProxyConfig)
|
||||
mock_proxy_config._get_hierarchical_router_settings = AsyncMock(return_value=router_settings)
|
||||
|
||||
returned_data, _ = await processing_obj.common_processing_pre_call_logic(
|
||||
request=mock_request,
|
||||
general_settings={},
|
||||
user_api_key_dict=ProxyUserAPIKeyAuth(api_key="hash", models=[], aliases=key_aliases),
|
||||
proxy_logging_obj=mock_proxy_logging_obj,
|
||||
proxy_config=mock_proxy_config,
|
||||
route_type="avideo_status",
|
||||
llm_router=self._router(),
|
||||
)
|
||||
|
||||
assert mock_proxy_logging_obj.pre_call_hook.call_args.kwargs["data"]["model"] == hooked
|
||||
assert returned_data["model"] == expected
|
||||
|
||||
|
||||
class TestInjectCostIntoUsageDict:
|
||||
@staticmethod
|
||||
def _expected_cost(model, prompt_tokens, completion_tokens):
|
||||
|
|
|
|||
|
|
@ -115,7 +115,7 @@ def harness():
|
|||
|
||||
router = MagicMock(spec=Router)
|
||||
resolve_model = MagicMock(
|
||||
side_effect=lambda model_id: RESOLVED_MODELS.get(model_id)
|
||||
side_effect=lambda model_id, custom_llm_provider=None: RESOLVED_MODELS.get(model_id)
|
||||
)
|
||||
router.resolve_model_name_from_model_id = resolve_model
|
||||
|
||||
|
|
@ -276,7 +276,7 @@ async def test_status__model_encoded_id_full_contract(harness):
|
|||
assert resp is SENTINEL
|
||||
assert harness.route_type() == "avideo_status"
|
||||
# provider comes from the decoded id; model_id resolved to a model name.
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID)
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID, "azure")
|
||||
assert harness.processor_data() == {
|
||||
"video_id": AZURE_VIDEO_ID,
|
||||
"custom_llm_provider": "azure",
|
||||
|
|
@ -357,7 +357,7 @@ async def test_content__model_encoded_id(harness):
|
|||
|
||||
await call_content(harness, AZURE_VIDEO_ID)
|
||||
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID)
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID, "azure")
|
||||
assert harness.processor_data() == {
|
||||
"video_id": AZURE_VIDEO_ID,
|
||||
"custom_llm_provider": "azure",
|
||||
|
|
@ -365,6 +365,28 @@ async def test_content__model_encoded_id(harness):
|
|||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("call", [call_status, call_content])
|
||||
async def test_status_and_content__hand_the_served_member_to_the_request_processor(harness, call):
|
||||
"""The id names the routing group a key may be scoped to, and the member that served the task, which is
|
||||
the only one holding it. The processor pins the member once key and team aliases have been applied."""
|
||||
router = Router(
|
||||
model_list=[
|
||||
{"model_name": name, "litellm_params": {"model": "dashscope/wan3.0-video", "api_key": "k"}}
|
||||
for name in ("wan3-a", "wan3-b")
|
||||
],
|
||||
routing_groups=[{"group_name": "wan3", "models": ["wan3-a", "wan3-b"], "routing_strategy": "simple-shuffle"}],
|
||||
)
|
||||
harness.base_process.return_value = b"x"
|
||||
video_id = encode_video_id_with_provider("task1", "dashscope", "wan3", served_model="wan3-b")
|
||||
|
||||
with patch.object(proxy_server, "llm_router", router):
|
||||
await call(harness, video_id)
|
||||
|
||||
assert harness.processor_data()["model"] == "wan3"
|
||||
assert harness.base_process.call_args.args[0].served_model == "wan3-b"
|
||||
|
||||
|
||||
# =========================================================================== #
|
||||
# POST /v1/videos/edits - video_edit #
|
||||
# =========================================================================== #
|
||||
|
|
@ -389,7 +411,7 @@ async def test_edit__extracts_nested_video_id_full_contract(harness):
|
|||
|
||||
assert resp is SENTINEL
|
||||
assert harness.route_type() == "avideo_edit"
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID)
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID, "azure")
|
||||
# nested video object is popped; its id becomes video_id; provider/model
|
||||
# derived from the encoded id.
|
||||
assert harness.processor_data() == {
|
||||
|
|
@ -534,7 +556,7 @@ async def test_remix__model_encoded_id_full_contract(harness):
|
|||
|
||||
assert resp is SENTINEL
|
||||
assert harness.route_type() == "avideo_remix"
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID)
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID, "azure")
|
||||
assert harness.processor_data() == {
|
||||
"prompt": "new colors",
|
||||
"video_id": AZURE_VIDEO_ID,
|
||||
|
|
@ -644,7 +666,7 @@ async def test_get_character__encoded_id_full_contract(harness):
|
|||
resp = await call_get_character(harness, AZURE_CHARACTER_ID)
|
||||
|
||||
assert harness.route_type() == "avideo_get_character"
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID)
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID, "azure")
|
||||
# character_id decoded to its inner value; provider/model from the encoded id.
|
||||
assert harness.processor_data() == {
|
||||
"character_id": "char_orig",
|
||||
|
|
@ -694,7 +716,7 @@ async def test_extension__extracts_nested_video_id_full_contract(harness):
|
|||
|
||||
assert resp is SENTINEL
|
||||
assert harness.route_type() == "avideo_extension"
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID)
|
||||
harness.resolve_model.assert_called_once_with(VIDEO_MODEL_ID, "azure")
|
||||
assert harness.processor_data() == {
|
||||
"prompt": "continue",
|
||||
"video_id": AZURE_VIDEO_ID,
|
||||
|
|
|
|||
|
|
@ -479,3 +479,19 @@ def test_wildcard_priority_over_deployment_names():
|
|||
assert (
|
||||
deployments[0]["litellm_params"]["api_base"] == "http://localhost:8081/openai"
|
||||
), f"Expected '*' wildcard deployment (8081), got {deployments[0]['litellm_params']['api_base']}"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("litellm_params", "custom_llm_provider", "expected"),
|
||||
[
|
||||
({"model": "openai/*"}, "openai", True),
|
||||
({"model": "openai/*"}, "dashscope", False),
|
||||
({"model": "*", "custom_llm_provider": "dashscope"}, "dashscope", True),
|
||||
],
|
||||
)
|
||||
def test_routes_to_provider_matches_the_provider_the_wildcard_deployment_sends_to(
|
||||
litellm_params, custom_llm_provider, expected
|
||||
):
|
||||
router = PatternMatchRouter()
|
||||
router.add_pattern("*", {"model_name": "*", "litellm_params": litellm_params})
|
||||
assert router.routes_to_provider("wan3.0-video", custom_llm_provider) is expected
|
||||
|
|
|
|||
|
|
@ -2526,6 +2526,21 @@ def test_resolve_model_name_from_model_id():
|
|||
assert result == "bedrock-batch-model"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("model_id", "custom_llm_provider", "expected"),
|
||||
[
|
||||
("sora-2", "openai", "sora-2"),
|
||||
("fal-ai/veo3", "fal_ai", None),
|
||||
("veo-3.0-generate-001", "gemini", None),
|
||||
("sora-2", None, None),
|
||||
],
|
||||
)
|
||||
def test_resolve_model_name_from_model_id_keeps_a_wildcard_name_only_for_the_ids_provider(
|
||||
model_id, custom_llm_provider, expected
|
||||
):
|
||||
router = Router(model_list=[{"model_name": "*", "litellm_params": {"model": "openai/*", "api_key": "sk-fake"}}])
|
||||
assert router.resolve_model_name_from_model_id(model_id, custom_llm_provider) == expected
|
||||
|
||||
def test_get_valid_args():
|
||||
"""Test get_valid_args static method returns valid Router.__init__ arguments"""
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue