mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-16 04:29:31 +00:00
4f3cb48dc1
* feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos
DashScopeVideoConfig maps litellm's OpenAI Videos API onto Alibaba Model
Studio's async task API: create hits
/api/v1/services/aigc/video-generation/video-synthesis and status and content
poll /api/v1/tasks/{task_id}. It is registered for dashscope and for the
qwencloud and qwen_ai_platform brand aliases, following the same
get_dashscope_family_*_config pattern image generation uses. Unlike the
Sora-shaped providers the request body is nested, so prompt and media go under
input and everything else goes under parameters. X-DashScope-Async is always
sent, because DashScope rejects the call outright without it
OpenAI params translate to DashScope's contract. seconds becomes duration, and
size yields both a ratio and a named resolution tier (1280x720 gives 16:9 plus
720P), since sending only the ratio leaves the request on the default 1080P and
silently bills at that rate. input_reference becomes a single media entry,
base64 encoded into a data URI for file inputs. Which media type a model takes
(first_frame or, on the reference-to-video models, reference_image) is read
from provider_specific_entry.dashscope_video_reference_input via
get_model_info rather than hardcoded; models without it predate the media
array and get the flat img_url field. A caller-supplied media array always
wins, as that is the only way to express multi-role combinations
The create call is the only billed call, so it bills what DashScope will
deliver: the requested duration and tier, or the documented 5 second and 1080P
defaults when they are omitted. Smart duration (duration -1) lets the service
pick the length after that call, so it is rejected rather than recorded at
zero, with a 400. Task creation and task lookup both report failures as a 200 carrying
top-level code and message with no output, so that shape is raised instead of
surfacing as a queued video with an empty id. Status responses re-encode the
deployment model from the polled id, so follow-up polls and downloads route to
the same deployment. Timestamps parse as UTC+8. DashScope publishes no list,
delete or remix endpoint, so those raise rather than guess at a URL
The end-to-end tests drive the real avideo_generation() entrypoint through an
httpx MockTransport and assert on the JSON body httpx actually encoded. That is
what catches a mapper that drops input_reference before the request is built,
or a body fragment json cannot serialize, neither of which a test calling the
transform directly would see
Both model_prices maps gain wan3.0-video, wan3.0-video-prime, wan2.7-t2v,
wan2.7-i2v, wan2.7-r2v and the HappyHorse 1.0 and 1.1 t2v, i2v and r2v models,
for dashscope and the qwencloud and qwen_ai_platform aliases, priced per output
second by resolution tier from Model Studio's USD rate cards. dashscope and
qwen_ai_platform default to the Beijing host and use its rate card; qwencloud
defaults to the international host and uses Singapore's. The three
*_API_BASE_VIDEO overrides are documented in BerriAI/litellm-docs#1920
Co-authored-by: AaronHowell <237895480@qq.com>
* fix(dashscope): fail loudly on video responses that cannot be read
Addresses the review on the video generation PR. A body that is not the
documented task shape, an output block missing task_id or task_status, an
undocumented task_status and a missing output all raise DashScopeVideoError
now, instead of defaulting into a queued video with an empty id that callers
poll forever. A JSONDecodeError from a gateway response no longer escapes as
a generic connection error.
task_id and task_status are required on _TaskOutput, and a missing output
raises even when the body carries no code or message.
Download failures move into the 4xx range, since a 5xx has the proxy retry
them: 409 while the task is still running, 400 for a task that failed, 404
for an id that expired. The status a provider reported is kept when it
reported one, so an in-body failure that arrives on a 200 is billed as the
400 it is rather than as a retryable connection error.
characters and image are only dropped when drop_params is set; the params
that are rebuilt into the body or consumed by the caller (model, prompt,
user, extra_*) stay silently skipped, since the proxy stamps user from the
caller's end-user header.
Co-authored-by: AaronHowell <237895480@qq.com>
* fix(dashscope): clear the lint gates after merging main
* fix(dashscope): accept file tuples and paths as video input_reference
video_generation() types input_reference as FileTypes, which includes
(filename, content[, content_type[, headers]]) tuples and pathlib paths,
but the DashScope encoder only read bytes and file objects, so those
image-to-video calls raised before reaching DashScope. Bare strings stay
URLs and are never opened as paths.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(dashscope): snap size to a supported ratio and read file tuples and paths
* test(dashscope): assert whole mapped values and bill against hand-written rates
The tier pricing test read its expected rate from the same map the cost
calculator uses, so a wrong rate passed. Billing is now checked against
fixed rates driven through the status response usage, while the map test
only asserts every supported tier has its own rate. Size mapping tests
compare the full mapped dict, and the single-test wiring class is a
plain function.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(videos): download content through the caller's client and stop retrying dead tasks
Two Cursor Bugbot findings on the DashScope video PR.
The DashScope artifact lives on OSS, so transform_video_content_response makes a
second request. It built its own cached client, discarding an injected client and
its ssl_verify. Thread the client through the content response the way
transform_video_status_retrieve_response already does, and use it in the three
providers that fetch from a second URL (DashScope, RunwayML, fal.ai). The
remaining providers accept it and read the response body, matching how they
already accept an unused client on status retrieve.
A FAILED or CANCELED task reported without a code or message fell past the
in-flight check into the 502 branch, so the proxy retried a task that can never
produce a video. Raise 400 for a terminal status instead.
* fix(dashscope): answer 404, not 409, when a Wan video is downloaded before it finishes
A download that lands while the task is still PENDING or RUNNING answered 409. That
reads as a client error, which was the intent, but litellm's retry predicate counts 409
as retryable, so the proxy polled DashScope twice more before giving up and then counted
the caller's own early download against the model's error rate
404 is what the OpenAI videos API documents for content that is not ready, and it is the
one status in that family the retry predicate leaves alone. The rest of the mapping is
unchanged: 400 for a task the provider reported as failed, 404 for an expired task id,
502 when a succeeded task carries no artifact
The download tests now drive avideo_content and assert the caller sees NotFoundError
rather than an internal server error
* fix(dashscope): route a video poll back to the model group that created the task
The create response encoded the upstream model name into the returned id. Both the
dashscope and qwen_ai_platform prefixes serve the same model names from their own hosts,
so a proxy with a deployment per region had deployments that only the model group told
apart. Encoding the bare model name resolved a poll to whichever deployment the router
lists first, which sent it to the other region's host with that region's key and a task
id that region never issued. That is the reroute the earlier proof records as the
dashscope and qwen_ai_platform poll ids coming back tagged qwencloud
The id now carries the model group the create was routed to, which is what the proxy
resolves a poll back to, and falls back to the model name for a direct SDK call that has
no router
The regression test drives a real router with one deployment per region, under a
different prefix and under the same one, and asserts the poll carries the key of the
region that created the task
* fix(dashscope): read the routed model group from litellm_metadata
The router stamps the model group it picked into litellm_metadata. metadata only
carries a copy when the caller sent no metadata of its own, and every proxy request
sends some, so a proxied create still encoded the bare model name and the poll
resolved to whichever region's deployment the router lists first
The regression test now also creates the video with caller metadata, which fails
without this change
* fix(dashscope): reject unbillable video requests and keep fal and runway on their own clients
* fix(dashscope): show the caller why a video task is gone or not ready instead of the bare upstream body
* fix(dashscope): check video request values before the billed create and honor per-call drop_params
Type check documented DashScope parameters so a bad seed answers 400 instead of a billed failed task,
pass the call's and deployment's drop_params into the video mapping, answer 400 for an expired task id so
the deployment is not cooled down, read DASHSCOPE_API_BASE for video, keep polling SUSPENDED tasks, and
refuse to map size onto models litellm has no price for.
* fix(dashscope): carry the requested model name in video ids so alias and wildcard keys can poll
A key scoped to an alias or to a name a wildcard serves could create a
video and then got a 403 on the poll and the download, because the id
carried the group that served the call (wan3, qwencloud/*) and the key
was never given that name. The id now carries the name the router was
asked for on that hop, and the router resolves an alias, routing group
or wildcard name in an id back to itself when no deployment name or
provider model matches it first
* fix(dashscope): keep a routing group's video polls on the member that created the task
A routing group spans several model groups, so an id carrying the group's name
load-balanced its polls, and some left for another region's host and key with a
task id that region never issued. The id now carries the member that served the
task when the request named a routing group, and keeps the requested name for a
model group, an alias, or a wildcard match.
* fix(dashscope): carry the requested and served model in video ids so group-scoped keys poll in the creating region
* fix(proxy): pin a video's served group only after key and team aliases resolve
* test(integration): audit DashScope video create, poll, download, routing and chaos paths
* fix(proxy): keep catch-all video polls on their provider and count served polls against the requested group
* test(routing): expect a forged served group to be dropped from the polled video id
* test(integration): cover rate-limited routing group polls and catch-all polls of DashScope videos
---------
Co-authored-by: AaronHowell <237895480@qq.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| a2a_protocol | ||
| anthropic_interface | ||
| batch_completion | ||
| batches | ||
| caching | ||
| chat_completions | ||
| completion_extras | ||
| compression | ||
| containers | ||
| decisions | ||
| embeddings | ||
| endpoints | ||
| enterprise | ||
| expected_fine_tuning_api | ||
| expected_responses_api_request | ||
| experimental_mcp_client | ||
| files | ||
| fixtures | ||
| gateway | ||
| google_genai | ||
| harness | ||
| images | ||
| integration_support | ||
| integrations | ||
| interactions | ||
| litellm_core_utils | ||
| litellm_proxy_extras | ||
| llms | ||
| messages | ||
| models | ||
| ocr | ||
| passthrough | ||
| proxy | ||
| rag | ||
| realtime_api | ||
| repositories | ||
| rerank_api | ||
| responses | ||
| router_strategy | ||
| router_utils | ||
| rust_bridge | ||
| sandbox | ||
| secret_managers | ||
| skills | ||
| systemone | ||
| telemetry | ||
| test_router | ||
| tracing | ||
| types | ||
| vector_stores | ||
| videos | ||
| __init__.py | ||
| AGENTS.md | ||
| conftest.py | ||
| create_mock_standard_logging_payload.py | ||
| test_a2a_registry_lookup.py | ||
| test_acompletion_session_reuse_e2e.py | ||
| test_add_deployment_no_master_key.py | ||
| test_aembedding_session_reuse_e2e.py | ||
| test_anthropic_beta_headers_filtering.py | ||
| test_anthropic_skills_transformation.py | ||
| test_assert_ci_coverage.py | ||
| test_assert_workflow_dir_hygiene.py | ||
| test_audio_transcription_rust_bridge.py | ||
| test_auto_update_price_and_context_window_file.py | ||
| test_azure_ad_token_credential_resolution.py | ||
| test_azure_ai_grok_4_3_model_metadata.py | ||
| test_azure_ai_grok_4_6_model_metadata.py | ||
| test_baseten_glm_5_3_model_metadata.py | ||
| test_batch_completion_models_all_responses.py | ||
| test_bedrock_marengo_embed_3_model_metadata.py | ||
| test_build_core_distribution.py | ||
| test_chat_ui_responses_session.py | ||
| test_check_licenses.py | ||
| test_check_mcp_operation_boundary.py | ||
| test_check_migrations_lock_timeout.py | ||
| test_check_migrations_no_data_rewrites.py | ||
| test_check_openapi_docs.py | ||
| test_check_py310_typing_imports.py | ||
| test_check_test_quality.py | ||
| test_check_type_discipline.py | ||
| test_check_unbounded_in_lists.py | ||
| test_check_windows_wheel_install.py | ||
| test_circleci_path_filter.py | ||
| test_circleci_run_migration_tests.py | ||
| test_circleci_rust_toolchain.py | ||
| test_claude_fable_5_config.py | ||
| test_claude_haiku_5_5_config.py | ||
| test_claude_opus_4_6_config.py | ||
| test_claude_opus_4_8_config.py | ||
| test_claude_opus_5_config.py | ||
| test_claude_sonnet_5_config.py | ||
| test_cloudflare_workers_ai_model_metadata.py | ||
| test_completion_timeout_resolution.py | ||
| test_component_entrypoint.py | ||
| test_compression.py | ||
| test_conftest_isolation.py | ||
| test_constants.py | ||
| test_container_router.py | ||
| test_cost_calculation_log_level.py | ||
| test_cost_calculator.py | ||
| test_cost_map_guard.py | ||
| test_count_tokens_public_api.py | ||
| test_dashscope_image_generation.py | ||
| test_daybreak_model_metadata.py | ||
| test_deepseek_model_metadata.py | ||
| test_default_branch.py | ||
| test_detect_changes.py | ||
| test_docker_no_network_on_deploy.py | ||
| test_dockerfile_apk_repository.py | ||
| test_dockerfile_bedrock_realtime_extra.py | ||
| test_dockerfile_non_root.py | ||
| test_drop_params_env_var.py | ||
| test_e2e_egress_sentinel.py | ||
| test_eager_tiktoken_load.py | ||
| test_env_key_doc_gate.py | ||
| test_exception_exports.py | ||
| test_exception_header_preservation.py | ||
| test_exception_mapping_request_attribute.py | ||
| test_fake_openai_endpoint.py | ||
| test_filter_out_litellm_params.py | ||
| test_fireworks_serverless_model_costs.py | ||
| test_gate_slot_lock.py | ||
| test_gemini_3_1_flash_lite_image_pricing.py | ||
| test_gemini_tts_native_audio_pricing.py | ||
| test_get_blog_posts.py | ||
| test_git_hooks.py | ||
| test_gpt_5_4_model_metadata.py | ||
| test_gpt_5_5_model_metadata.py | ||
| test_gpt_image_cost_calculator.py | ||
| test_gpt_realtime_mode.py | ||
| test_groq_streaming_encoding.py | ||
| test_guardrail_exception_status_codes.py | ||
| test_integration_run.py | ||
| test_internal_context.py | ||
| test_lazy_imports.py | ||
| test_lint_base_counts.py | ||
| test_lint_workflow_diff_gates.py | ||
| test_litellm_overhead.py | ||
| test_litellm_params_reserved_keys.py | ||
| test_logging.py | ||
| test_lowest_latency_zero_tokens.py | ||
| test_main.py | ||
| test_main_module_header.py | ||
| test_mistral_large_4_model_metadata.py | ||
| test_mistral_medium_3_5_model_metadata.py | ||
| test_mistral_small_4_0_model_metadata.py | ||
| test_mistral_zai_glm_5_2_model_metadata.py | ||
| test_model_block_unblock.py | ||
| test_model_cost_aliases.py | ||
| test_model_param_helper.py | ||
| test_model_prices_schema.py | ||
| test_model_response_normalization.py | ||
| test_muse_spark_1_1_model_metadata.py | ||
| test_muse_spark_1_2_model_metadata.py | ||
| test_muse_spark_1_3_model_metadata.py | ||
| test_mutation_report.py | ||
| test_nested_drop_params.py | ||
| test_no_top_level_test_invocations.py | ||
| test_non_chat_routes_open_llm_spans.py | ||
| test_openai_embedding_encoding_format_default.py | ||
| test_openai_record_replay_proxy.py | ||
| test_openai_service_tier_long_context_pricing.py | ||
| test_openapi_docs_gate.py | ||
| test_package_layout.py | ||
| test_package_metadata.py | ||
| test_pre_commit_lint.py | ||
| test_prisma_generate_if_needed.py | ||
| test_private_usage_aliases.py | ||
| test_process_helpers.py | ||
| test_project_alias_tracking.py | ||
| test_project_tags_pydantic.py | ||
| test_proxy_auth.py | ||
| test_pydantic_namespaces.py | ||
| test_rag_openai_ingestion.py | ||
| test_rate_limit_error_unification.py | ||
| test_read_rc_version.py | ||
| test_redact_string_in_error_paths.py | ||
| test_redis.py | ||
| test_redis_credential_provider.py | ||
| test_register_model_custom_pricing.py | ||
| test_register_model_zero_cost_persistence.py | ||
| test_replicate_model_key_format.py | ||
| test_responses_api_bridge_non_stream.py | ||
| test_responses_id_security.py | ||
| test_responses_streaming_container_ownership.py | ||
| test_retrieve_batch_bedrock_dispatch.py | ||
| test_router_block_helpers.py | ||
| test_router_exception_redaction.py | ||
| test_router_get_settings.py | ||
| test_router_google_genai.py | ||
| test_router_model_cost_isolation.py | ||
| test_router_order_fallback.py | ||
| test_router_per_deployment_num_retries.py | ||
| test_router_redis_init.py | ||
| test_router_retry_backoff_headers.py | ||
| test_router_retry_non_retryable_errors.py | ||
| test_router_retry_policy_update.py | ||
| test_router_silent_experiment.py | ||
| test_router_streaming_fallback_metadata.py | ||
| test_router_weighted_failover.py | ||
| test_ruff_strict_gate.py | ||
| test_sambanova_model_metadata.py | ||
| test_scaledown_model_prices.py | ||
| test_scheduler.py | ||
| test_secret_redaction.py | ||
| test_seed_tracing_fixtures.py | ||
| test_select_ui_test_scope.py | ||
| test_service_logger.py | ||
| test_setup_wizard.py | ||
| test_shared_session_integration.py | ||
| test_socket_policy.py | ||
| test_ssl_verify_unit.py | ||
| test_stream_chunk_builder_annotations.py | ||
| test_stream_chunk_builder_citations.py | ||
| test_stream_chunk_builder_images.py | ||
| test_streaming_connection_cleanup.py | ||
| test_sync_together_ai_models.py | ||
| test_system_message_format_bug.py | ||
| test_test_quality_gate.py | ||
| test_thinking_enabled.py | ||
| test_together_ai_model_metadata.py | ||
| test_tool_loop.py | ||
| test_type_check_gate.py | ||
| test_type_discipline_gate.py | ||
| test_typesafe_model_metadata.py | ||
| test_unit_shard_missing_paths.py | ||
| test_unit_shard_per_test_timeout.py | ||
| test_utils.py | ||
| test_utils_get_model_info.py | ||
| test_utils_get_optional_params.py | ||
| test_utils_module_docstring.py | ||
| test_uuid_helper.py | ||
| test_vcr_classification.py | ||
| test_vcr_conftest_common_banner.py | ||
| test_vcr_filters.py | ||
| test_vcr_leak_guard.py | ||
| test_vcr_redis_persister.py | ||
| test_vcr_safe_body_matcher.py | ||
| test_version.py | ||
| test_vertex_ai_xai_grok_prompt_caching_metadata.py | ||
| test_video_generation.py | ||
| test_with_dashboard_node.py | ||
| test_ws_vcr.py | ||
| test_xai_grok_4_3_model_metadata.py | ||
| test_xai_responses_auto_routing.py | ||