litellm/tests/unit/test_video_generation.py
yuneng-jiang 7248fd8328
chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338)
* feat(model-catalog): add Rust registry validation (#43136)

* doc

* feat(model-catalog): add Rust registry validation

* test(model-catalog): ignore integration tests

* docs(model-catalog): fix validation note grammar

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
(cherry picked from commit c289d5d6fb)

* test(rust): reorganize core crate tests and split cache and OCR suites (#43177)

* test(rust): group cache tests under cache/ and fold test_ocr.py into ocr/

The two failure cases in test_ocr.py duplicated the upstream-500 and timeout rows
of PUBLIC_FAILURES, so only the file-input encoding case moves to ocr/test_requests.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(rust): split the response cache suite into one file per backend

test_response_cache.py grew to 2400 lines. Each backend now has its own file, shared
fixtures live in cache/conftest.py and shared helpers in support/cache.py. The helpers
alias the private native test handles once, dropping the per-call reportPrivateUsage hits

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* split tokenizer test

* test(core): consolidate route integration tests under tests/ with rstest and wiremock

Moves the public-API OCR route tests out of src/ocr/route.rs and document.rs into
tests/ocr/, split per provider plus lifecycle, machine, and document tests, merging
the duplicated pairs. Messages, audio transcription, and chat completions share one
wiremock-based upstream and recording secret source in tests/support, and gain
table-driven cases for auth, routing, upstream errors, streaming, and declines.
Tests of litellm-llms items move to that crate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(messages): keep the stream relay test independent of the stream head contents

The stream head carries no headers on main, so the relay test asserts the
open-then-deliver order and the relayed body instead of header hand-off.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 10413796c6)

* feat(rust): hand upstream response headers to the native Messages stream (#43178)

* ci: drop the ocr_testing job now that tests/ocr_tests is gone

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ocr): restore the live OCR matrix and the ocr_testing job

The public litellm.ocr / aocr / Router interface is unchanged by the Rust
migration, so the live provider matrix still applies. Drops the stale VCR skip
list for the deleted test_rust_bridge.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(messages): show streamed upstream headers never reach the native stream

The Python handler puts the upstream response headers on the stream's
_hidden_params before the first chunk so the proxy can forward them as
llm_provider-* headers. The native route drops them, and this test fails
on the Rust path while passing on Python.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(messages): hand upstream response headers to the native stream before its first chunk

The Messages route fills MessagesStreamHead from the upstream response and
yields it on Open. The Python driver converts it through the protocol host
and hands it to Stream and SyncStream as their _hidden_params, so a
streamed native call carries additional_headers the same way the Python
handler does and the proxy can forward them as llm_provider-* headers.

The relay contract lives in the core crate test, the hand-off in the
host-python driver test, and the header projection in the route host test,
so the recording-server test that showed the gap is dropped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wip

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 081f73f021)

* feat(testkit): agent clients for Claude Code, Codex and opencode (#43181)

* feat(testkit): install and configure Claude Code, Codex and opencode against a gateway

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* refactor(testkit): derive targets from target-lexicon and split agents behind a trait

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* refactor(testkit): group sources into agent and install folders

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(testkit): split agents into install, configure and drive with semver-aware launch

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* style(testkit): drop a needless borrow

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit 9c10e0f985)

* ci: run migrated unit selections on every event in legacy GHA shards (#43182)

* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: rename fork-flag to unit-flag now that it applies on every event

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 694783ebbe)

* test: move tests/test_litellm root and small trees into tests/unit (#43186)

* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: rename fork-flag to unit-flag now that it applies on every event

* test: move tests/test_litellm root and small trees into tests/unit

Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.

* test: carry tests/test_litellm conftest isolation into tests/unit

Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.

* test: merge, split and prune the moved root and small-tree tests

Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.

* ci: run the moved root and small-tree tests under their legacy flags

Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.

* test: make the new tests/unit directories packages

tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.

* test: scope the unit socket block to tests/unit in shared sessions

The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.

* test: give the shard-script tests their own GITHUB_OUTPUT

They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.

* test: point the router and module-deletion checks at tests/unit

router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit f6882246d4)

* refactor(framer): replace Framer trait with tokio-util codecs (#43193)

* refactor(framer): replace Framer trait with tokio-util codecs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(framer): port SSE and AWS event stream framing to codecs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rust): drop clone on Copy capabilities in messages request test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rust): use field init shorthand in messages request test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 3c93ea1697)

* test: move tests/test_litellm/llms into tests/unit/llms (#43191)

* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: rename fork-flag to unit-flag now that it applies on every event

* test: move tests/test_litellm root and small trees into tests/unit

Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.

* test: carry tests/test_litellm conftest isolation into tests/unit

Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.

* test: merge, split and prune the moved root and small-tree tests

Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.

* ci: run the moved root and small-tree tests under their legacy flags

Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.

* test: make the new tests/unit directories packages

tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.

* test: scope the unit socket block to tests/unit in shared sessions

The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.

* test: move tests/test_litellm/llms into tests/unit/llms

Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.

* test: merge, split and prune the moved llms tests

Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.

* ci: run the moved llms tests under their legacy flags

The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.

* test: make the tests/unit/llms directories packages

Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.

* test: drop script runners and path hacks the llms split left dangling

The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.

* test: give the shard-script tests their own GITHUB_OUTPUT

They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.

* test: point the router and module-deletion checks at tests/unit

router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.

* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path

The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.

* test: keep the job's UNIT_FLAG out of the shard-script tests

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 5e6dc89ba1)

* test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194)

* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: rename fork-flag to unit-flag now that it applies on every event

* test: move tests/test_litellm root and small trees into tests/unit

Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.

* test: carry tests/test_litellm conftest isolation into tests/unit

Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.

* test: merge, split and prune the moved root and small-tree tests

Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.

* ci: run the moved root and small-tree tests under their legacy flags

Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.

* test: make the new tests/unit directories packages

tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.

* test: scope the unit socket block to tests/unit in shared sessions

The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.

* test: move tests/test_litellm/llms into tests/unit/llms

Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.

* test: merge, split and prune the moved llms tests

Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.

* ci: run the moved llms tests under their legacy flags

The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.

* test: make the tests/unit/llms directories packages

Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.

* test: drop script runners and path hacks the llms split left dangling

The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.

* test: give the shard-script tests their own GITHUB_OUTPUT

They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.

* test: point the router and module-deletion checks at tests/unit

router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.

* test: move tests/test_litellm integrations and secret_managers into tests/unit

Rename-only. Mirrors the old paths, including the directory conftests
and the prompt and JSON fixtures. Follow-up commits prune and wire them.

* test: prune and repoint the moved integrations tests

Deletes the 7 audited tests a stronger test in the same tree already
covers, imports the TLS sink helpers from their new conftest path, and
restores os.environ after each integrations test. Some presets write
OTEL_EXPORTER_OTLP_HEADERS straight into os.environ, and without the
legacy tree's test ordering that header leaked into the AgentOps tests.

* ci: run the moved integrations tests under their legacy flag

The integrations GHA shard and a new CircleCI job run the integrations
unit selection. secret_managers joins the misc selection.

* docs: point integrations and secret_managers references at tests/unit

* test: make the moved integrations directories packages

* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path

The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.

* test: keep the job's UNIT_FLAG out of the shard-script tests

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit cf491d1df9)

* fix(tests): drop the repeated UNIT_FLAG key in test_unit_shard_missing_paths (#43212)

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 25de1ab2b0)

* fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim (#43206)

* fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs(jwt): describe header team selection under fallback_to_db_teams

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 7b4fd47c6e)

* test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)

* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: rename fork-flag to unit-flag now that it applies on every event

* test: move tests/test_litellm root and small trees into tests/unit

Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.

* test: carry tests/test_litellm conftest isolation into tests/unit

Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.

* test: merge, split and prune the moved root and small-tree tests

Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.

* ci: run the moved root and small-tree tests under their legacy flags

Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.

* test: make the new tests/unit directories packages

tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.

* test: scope the unit socket block to tests/unit in shared sessions

The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.

* test: move tests/test_litellm/llms into tests/unit/llms

Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.

* test: merge, split and prune the moved llms tests

Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.

* ci: run the moved llms tests under their legacy flags

The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.

* test: make the tests/unit/llms directories packages

Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.

* test: drop script runners and path hacks the llms split left dangling

The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.

* test: give the shard-script tests their own GITHUB_OUTPUT

They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.

* test: point the router and module-deletion checks at tests/unit

router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.

* test: move tests/test_litellm integrations and secret_managers into tests/unit

Rename-only. Mirrors the old paths, including the directory conftests
and the prompt and JSON fixtures. Follow-up commits prune and wire them.

* test: prune and repoint the moved integrations tests

Deletes the 7 audited tests a stronger test in the same tree already
covers, imports the TLS sink helpers from their new conftest path, and
restores os.environ after each integrations test. Some presets write
OTEL_EXPORTER_OTLP_HEADERS straight into os.environ, and without the
legacy tree's test ordering that header leaked into the AgentOps tests.

* ci: run the moved integrations tests under their legacy flag

The integrations GHA shard and a new CircleCI job run the integrations
unit selection. secret_managers joins the misc selection.

* docs: point integrations and secret_managers references at tests/unit

* test: make the moved integrations directories packages

* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path

The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.

* test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit

Rename-only. Mirrors the old paths, including fixtures, the stubtest config
and the native-route wheel script. Two files that collide with existing unit
files are merged in a follow-up commit.

* test: merge, prune and repoint the moved core, routing, responses, caching and rust_bridge tests

Merges the two files that collided with existing unit files, folding the
legacy extra case into test_is_chat_completion_cached_dict, and deletes the
9 audited tests a stronger test in the same file already covers.

Keeps what needs the network in tests/test_litellm: test_tokenizers pulls a
tokenizer from the Hugging Face hub, and the gpt2 and r50k_base tokenizer
cases download their BPE files. The unit core_utils conftest points
TIKTOKEN_CACHE_DIR at litellm's bundled encodings so the rest never depend on
import order to stay offline, and FakeSecretVault moves to a shared module
so both trees can build it.

* ci: run the moved core, routing, responses, caching and rust_bridge tests under their flags

core_utils gets a core-utils flag and CircleCI job, and its GHA shard keeps
the legacy path for the retained network tests. router_utils and
router_strategy join enterprise-routing, responses joins
responses-caching-types (minus responses/mcp, which mcp-integration owns),
caching joins caching-local and rust_bridge joins misc. The redis-compat,
test-rust, stubtest and merge-smoke paths follow the move.

* docs: point the Rust crate references at tests/unit

* test: make the moved core, routing and rust_bridge directories packages

* test: keep the no-loop DualCache batch_get_cache regression test

It runs the sync path outside any event loop, which the inside-loop test
cannot, so a change that picks the Redis client by loop state would only
show up there.

* test: keep the job's UNIT_FLAG out of the shard-script tests

* fix(url_utils): block 192.0.0.0/24 on every Python patch release

* test: move the new budget limiter tests into tests/unit/router_strategy

* test: move the new sentry scrubbing tests into tests/unit/litellm_core_utils

* test: move the new zerobus tests into tests/unit/integrations

* test: make tests/unit/integrations/zerobus a package

* test: load litellm's own tiktoken cache setup once instead of resetting it per test

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit a11a93f44a)

* refactor(rust): move credential inheritance and the SDK limits out of the legacy callback crate into a driver preflight (#43259)

The legacy callback crate carried two rewrites that have nothing to do with the
Logging contract: litellm_credential_name inheritance and the max_budget and
num_retries_per_request checks. Any later callback host would need them
unchanged, which is the smell the crate's AGENTS.md now names. They are now a
Preflight the driver in litellm-host-python runs on the keyword view begin
returned, before the host projects from it, supplied by python-bridge and passed
through run_legacy_call. The call order is unchanged (setup, deployment hook,
credentials, limits) and a rejection still fails the call as a host failure, so
the failure callbacks run as before. The preflight rewrites the adapter's own
copy in place, so no extra dict copy and no new lifecycle method

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit e9491d31b5)

* refactor(http): hand out an owned Client and route all providers through the pool (#43245)

* refactor(messages): take the provider client from the injected HTTP pool

The messages route kept its own process-wide reqwest client, so it ignored
ssl_verify, CA bundles, client certs, proxies and every other setting that
litellm-http resolves. The machine now takes the HttpClientPool and the
call's HttpClientConfig, as OCR does, and the bridge passes its shared pool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(http): hand out an owned Client and move chat, audio and OIDC onto the pool

HttpClientPool now returns litellm_http::Client, a newtype only crates/http
can build, so every provider client carries the resolved TLS, proxy and
timeout settings. Chat completions and audio transcription drop their
process-wide reqwest clients and take the pool and call config like
messages; their 600s ceiling moves to the request. OidcResolver takes its
client instead of building one, and the bridge hands it the pooled one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(secrets): build Google, Azure and CyberArk manager clients from the pool

The native secret managers built bare reqwest clients, so they ignored the
host's TLS and proxy settings. load_native_manager now takes the pool and
the host config and hands each manager a pooled client.

CyberArk's CYBERARK_SSL_VERIFY and CYBERARK_CLIENT_CERT/KEY become an
override on the host config instead of a hand-built client. To express a
certificate and key in separate files, HttpClientConfig::client_certificate
is now a ClientIdentity that is either one PEM or a split pair.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(clippy): only crates/http may build a reqwest client

Fence reqwest::Client, ClientBuilder and the TLS builder methods with
disallowed-types and disallowed-methods so new code takes a
litellm_http::Client from the pool. crates/http is exempt as the one place
clients are built, and testkit as a dev-only installer. Tests move to
litellm_http::Client::plain_for_test or a pooled client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(secrets-cyberark): keep verifying certificates when the host disables it

Python hands CyberArk its own ssl_verify, which wins over the global
setting, so CYBERARK_SSL_VERIFY unset or true still verifies even when the
host sets ssl_verify false. The pooled client copied the host's Disabled
and would send the API key unverified; fall back to the built-in roots
instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python-bridge): treat a missing litellm package as no host HTTP settings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 1a58162630)

* refactor(rust): promote anthropic messages out of experimental_pass_through (#43269)

Co-authored-by: Yujong Lee <yujong@berri.ai>
(cherry picked from commit 2ef3250ec3)

* test: stop CI tests from downloading tokenizer files and images (#43257)

* test: load the embedding base image from a committed 100x100 PNG instead of downloading it

* test: move the volcengine embedding test into tests/unit

* test: check gpt2 and r50k_base tokenizer parity against committed tiktoken reference files

* test: check hub tokenizer selection against an in-memory Hugging Face hub

* test: serve image URLs from respx in the gemini tool-result and format-param tests

* ci: drop the emptied legacy core-utils test path

* test: cover the cohere and anthropic tokenizer paths in the hub tokenizer test

* test: fetch every format-param image through respx and check its bytes reach the request

* test: drop the gpt2 and r50k_base parity tests, which no litellm path uses

* test: drop comments that restate assertions in the format-param test

(cherry picked from commit 2530255624)

* fix(rust): preserve nested optional import failures (#43265)

* fix(rust): preserve nested optional import failures

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(rust): restore Python modules after settings tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit e1a9378093)

* fix(callbacks-legacy-python): traverse and release the retained headers dict (#43274)

* fix(callbacks-legacy-python): traverse and release the retained headers dict

LegacyLogging keeps the headers dict it hands to pre_call and post_call, but
its traverse never reported that edge to the collector and close never dropped
it. A cycle a callback builds through that dict could not be collected, and a
closed call kept the dict alive until the driver dropped the whole adapter.
Visit and clear headers like body, with regression tests for both

* refactor(callbacks-legacy-python): move the test support module into its own file

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
(cherry picked from commit 4eb340abe0)

* test(integration): port langfuse callbacks-in-db coverage to the local harness (#43282)

* test(integration): port langfuse callbacks-in-db coverage to the local harness

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert the langfuse db rows after the exported span

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit e11c3f5815)

* test: finish the non-proxy half of tests/test_litellm (#43281)

* test: move key-gated tests/test_litellm SDK tests into tests/llm_translation and drop empty folders

* test: make token counter and health check unit tests run offline

* ci: point unit shards, rust path filter, Makefile and docs at tests/unit

* docs: fix stale test_litellm run paths in moved llm_translation tests

* fix: correct databricks e2e sys.path depth and contributing example path

---------

Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 99655b6f86)

* refactor(rust): prepare inference and auth foundations for the gateway (#43287)

* refactor(rust): prepare inference and auth foundations

* fix(rust): keep textract operations parsing from kebab-case model names

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 7ae721bf79)

* feat(rust): add config router and gateway crates (#43289)

Co-authored-by: Yujong Lee <yujong@berri.ai>
(cherry picked from commit affb547525)

* test(integration): run the Langfuse DB-callback test on its own scratch database (#43288)

* test(integration): run the Langfuse DB-callback test on its own scratch database

The test from #43282 wrote success_callback=langfuse and the LANGFUSE_* env
into the shared integration LiteLLM_Config. The suite's long-running gateway
reloads that table and only ever adds callbacks, so it kept exporting to the
test's closed Langfuse fake for the rest of the shard even after the rows were
restored. The owned proxy now gets a scratch database, which also removes the
snapshot/restore code. scratch_database moves into _support/database.py so
test_cache_and_quota and this test share one copy, and the stock-config guard
now checks the callback settings instead of the raw YAML text.

* test(integration): include failure_callback in the stock-config Langfuse guard

(cherry picked from commit dd63637322)

* refactor(rust): expand logging and test coverage across gateway and Anthropic messages (#43295)

* refactor(rust): prepare inference and auth foundations

* fix(rust): keep textract operations parsing from kebab-case model names

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* done

* refactor(types): derive Anthropic beta string conversions with Strum

* fix(anthropic): report missing max_tokens as a missing field

* refactor(rust): type Anthropic messages headers and auth after the Python layout

Delete anthropic/messages/headers.rs. Its OAuth handling, credential ladder
and beta merging move to anthropic/common_utils.rs where Python keeps them
(optionally_handle_anthropic_oauth, get_auth_header, _merge_beta_headers),
and the feature beta injection becomes update_headers_with_anthropic_beta on
the messages config, as in Python. The BaseAnthropicMessagesConfig impl is
unchanged apart from the bodies of validate_environment and request_headers

Beta values are now the AnthropicBeta enum and BetaSet, which sort, dedupe
and comma-join by construction. Request params gain typed speed, tools and
context_management through Recognized, so the beta logic matches on enums
instead of string-comparing JSON. OauthToken parses the sk-ant-oat token once
and the chat config shares that detection instead of its own copy

Case-insensitive header helpers move next to has_header in litellm-http.
One deliberate divergence: a Bearer-prefixed OAuth key configured through
api_key or ANTHROPIC_API_KEY is sent with a single Bearer scheme, where
Python would emit "Bearer Bearer"

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* done

* fix(rust): repair test compilation and clippy failures

resolve auth before building the outbound request in prepare tests, give the host hook tests their own error type, and drop the disallowed reqwest client and err().expect() from core tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 90873c46de)

* test(proxy_behavior): scope the management proxy fixture to its package so its spend monitor cannot race the spend tests (#43302)

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 115668f43e)

* fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts (#43294)

* fix(ci): stop five stale or flaky CI reds and retry CyberArk policy-load conflicts

The Langfuse redaction unit test exports to a local OTLP capture instead of
polling Langfuse Cloud through a recorded lookup. The passthrough worker-kill
test only requires spend rows for requests the surviving worker served. The
spend-routes sweep treats the intentional /spend/capture_rate 503 as expected.
CyberArk retries a 409 policy load in Python, Rust and the e2e Conjur helper
instead of reading it as "variable exists". The integration egress guard now
matches the script's own cgroup, so it no longer blocks the CircleCI agent,
which runs as the same user.

* fix(ci): keep the policy-load backoff typed as float

* fix(ci): retry CyberArk policy loads without blocking the event loop and tighten the worker-kill and Langfuse tests

* fix(secrets): load CyberArk policy one request at a time per manager

* test(secrets): pin that non-conflict CyberArk policy failures are not retried

* test(unit): run tests/unit with only an allowlisted host environment

CircleCI's unit job inherits every project env var, so real provider keys,
REDIS_HOST, DATABASE_URL and AWS or Azure credentials reached tests that
assume none are set. Locally, litellm's import-time load_dotenv did the same
from any .env up the tree. The unit conftest now drops every variable outside
a small allowlist and disables dotenv before litellm is imported.

* test(e2e): name a failed search and the stuck batch status instead of misattributing them

The websearch session test read an empty web_search_tool_result_error block as a
successful search, so a failing search tool surfaced as a session billing bug.
The batch cancellation timeout now reports the last status the proxy returned.

* fix(ci): scrub the host environment per unit test instead of for the whole pytest process

GHA shards run tests/unit next to other suites in one process, so the import-time
scrub deleted MCP_TEST_PEER_PYTHON before tests/mcp_tests read it and the MCP
upstream fell back to the SDK2 interpreter. The two websearch tests that called
OpenAI and Perplexity live are removed: tests/unit no longer sees their keys.

* fix(ci): scrub only the host variables present before litellm is imported

The per-test scrub also deleted TIKTOKEN_CACHE_DIR, which litellm sets at import to
its bundled encodings, so tokenizer paths tried to download them and hit the
socket guard. The prisma setup test now passes its own database URL instead of
reading one another test leaked into the process environment.

* fix(ci): stop the order-dependent unit reds and settle logging tasks on their own queue

LoggingWorker marked a task done on whichever queue was current when the callback
finished, so a callback that outlived an event-loop change raised "task_done()
called too many times" or undercounted the new loop's queue. It now settles the
queue the task came from.

The rest are test isolation fixes for failures that only appeared when another
file ran first on the same xdist worker: a replaced user_api_key_cache, breaker
metrics unregistered by prometheus tests, semantic_router's health-check filter on
uvicorn.access, logging tasks carried over from bedrock tests, a Router-written
model_cost entry, and a stray post captured by the langflow test. The token
counter check now asserts bounded chunking instead of wall-clock time.

* test(e2e/ui): wait for the logout redirect before visiting a protected page

Logout revokes the session server-side before clearing cookies and navigating, so an immediate page.goto either ran with the cookie still set or was aborted by the logout redirect (net::ERR_ABORTED).

* test(unit): restore the prometheus metrics config per test and settle logs carried from earlier tests in the a2a cost tests

* test(router): pin the router clock in the usage counter tests so a minute rollover cannot empty the read

* test(e2e/ui): wait for logout to clear the token cookie instead of for a login redirect

* test(integration/mcp): answer the model-info probe another test's proxy sends to the model double

(cherry picked from commit 14f4c34c61)

* test(e2e): assert only litellm-owned batch behavior and move the blank S3 env pin to an integration test (#43321)

(cherry picked from commit e53e67ede5)

* ci: drop main and litellm_* branch filters from the CircleCI litellm-main workflows (#43272)

Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit c822c7fffa)

* test(e2e): accept the otel cost write as a linked root trace (#42931)

* test(e2e): accept the otel cost write as a linked root trace

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): window and fail-closed the linked otel trace read-back

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): poll the otel read-back without recursion

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 3ebf6a1fd8)

* refactor(rust): exclude gateway and config crates from rc backport

* refactor(rust): exclude router crate from rc backport

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
2026-09-26 14:26:59 -07:00

2486 lines
90 KiB
Python

import asyncio
import io
import json
import os
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
import litellm
from litellm.cost_calculator import default_video_cost_calculator
from litellm.integrations.custom_logger import CustomLogger
from litellm.litellm_core_utils.litellm_logging import Logging as LitellmLogging
from litellm.litellm_core_utils.logging_worker import GLOBAL_LOGGING_WORKER
from litellm.llms.custom_httpx.http_handler import AsyncHTTPHandler
from litellm.llms.custom_httpx.llm_http_handler import BaseLLMHTTPHandler
from litellm.llms.gemini.videos.transformation import GeminiVideoConfig
from litellm.llms.openai.videos.transformation import OpenAIVideoConfig
from litellm.types.videos.main import VideoObject, VideoResponse
from litellm.videos import main as videos_main
from litellm.videos.main import (
avideo_generation,
avideo_status,
video_generation,
video_status,
)
class TestVideoGeneration:
"""Test suite for video generation functionality."""
def test_video_generation_basic(self):
"""Test basic video generation functionality."""
# Use mock_response parameter for reliable testing
response = video_generation(
prompt="Show them running around the room",
model="sora-2",
seconds="8",
size="720x1280",
mock_response={
"id": "video_123",
"object": "video",
"status": "queued",
"created_at": 1712697600,
"model": "sora-2",
"size": "720x1280",
"seconds": "8",
},
)
assert isinstance(response, VideoObject)
assert response.id == "video_123"
assert response.status == "queued"
assert response.model == "sora-2"
assert response.size == "720x1280"
assert response.seconds == "8"
def test_video_generation_with_mock_response(self):
"""Test video generation with mock response."""
mock_data = {
"id": "video_456",
"object": "video",
"status": "completed",
"created_at": 1712697600,
"completed_at": 1712697660,
"model": "sora-2",
"size": "1280x720",
"seconds": "10",
}
response = video_generation(
prompt="A beautiful sunset over the ocean",
model="sora-2",
seconds="10",
size="1280x720",
mock_response=mock_data,
)
assert isinstance(response, VideoObject)
assert response.id == "video_456"
assert response.status == "completed"
assert response.model == "sora-2"
assert response.size == "1280x720"
assert response.seconds == "10"
def test_video_generation_async(self):
"""Test async video generation functionality."""
mock_response = VideoObject(
id="video_async_123",
object="video",
status="processing",
created_at=1712697600,
model="sora-2",
progress=50,
)
# Mock the async_video_generation_handler to return the mock_response
async_mock = AsyncMock(return_value=mock_response)
with patch.object(
videos_main.base_llm_http_handler,
"async_video_generation_handler",
async_mock,
):
with patch.object(
videos_main.base_llm_http_handler,
"video_generation_handler",
side_effect=lambda **kwargs: async_mock(**kwargs),
):
import asyncio
async def test_async():
response = await avideo_generation(
prompt="A cat playing with a ball",
model="sora-2",
seconds="5",
size="720x1280",
)
return response
response = asyncio.run(test_async())
assert isinstance(response, VideoObject)
assert response.id == "video_async_123"
assert response.status == "processing"
assert response.progress == 50
def test_video_generation_parameter_validation(self):
"""Test video generation parameter validation."""
# Test with minimal required parameters
response = video_generation(
prompt="Test video",
model="sora-2",
mock_response={
"id": "test",
"object": "video",
"status": "queued",
"created_at": 1712697600,
},
)
assert isinstance(response, VideoObject)
assert response.id == "test"
def test_video_generation_error_handling(self):
"""Test video generation error handling."""
with patch.object(
videos_main.base_llm_http_handler,
"video_generation_handler",
side_effect=Exception("API Error"),
):
with pytest.raises(litellm.APIConnectionError):
video_generation(prompt="Test video", model="sora-2")
def test_video_generation_provider_config(self):
"""Test video generation provider configuration."""
config = OpenAIVideoConfig()
# Test supported parameters
supported_params = config.get_supported_openai_params("sora-2")
assert "prompt" in supported_params
assert "model" in supported_params
assert "seconds" in supported_params
assert "size" in supported_params
def test_video_generation_request_transformation(self):
"""Test video generation request transformation."""
config = OpenAIVideoConfig()
# Test request transformation
data, files, returned_api_base = config.transform_video_create_request(
model="sora-2",
prompt="Test video prompt",
api_base="https://api.openai.com/v1/videos",
video_create_optional_request_params={"seconds": "8", "size": "720x1280"},
litellm_params=MagicMock(),
headers={},
)
assert data["model"] == "sora-2"
assert data["prompt"] == "Test video prompt"
assert data["seconds"] == "8"
assert data["size"] == "720x1280"
assert files == []
assert returned_api_base == "https://api.openai.com/v1/videos"
def test_video_generation_request_decodes_encoded_character_ids(self):
"""Encoded character IDs should be decoded before upstream create-video call."""
from litellm.types.videos.utils import encode_character_id_with_provider
config = OpenAIVideoConfig()
encoded_character_id = encode_character_id_with_provider(
character_id="char_123",
provider="openai",
model_id="sora-2",
)
data, files, returned_api_base = config.transform_video_create_request(
model="sora-2",
prompt="Test video prompt",
api_base="https://api.openai.com/v1/videos",
video_create_optional_request_params={
"seconds": "8",
"size": "720x1280",
"characters": [{"id": encoded_character_id}],
},
litellm_params=MagicMock(),
headers={},
)
assert data["characters"] == [{"id": "char_123"}]
assert files == []
assert returned_api_base == "https://api.openai.com/v1/videos"
def test_video_generation_response_transformation(self):
"""Test video generation response transformation."""
config = OpenAIVideoConfig()
# Mock HTTP response
mock_http_response = MagicMock()
mock_http_response.json.return_value = {
"id": "video_789",
"object": "video",
"status": "completed",
"created_at": 1712697600,
"model": "sora-2",
"size": "1280x720",
"seconds": "12",
}
response = config.transform_video_create_response(
model="sora-2", raw_response=mock_http_response, logging_obj=MagicMock()
)
assert isinstance(response, VideoObject)
assert response.id == "video_789"
assert response.status == "completed"
assert response.model == "sora-2"
def test_video_generation_cost_calculation_unknown_model(self):
"""Test video generation cost calculation for unknown model."""
with pytest.raises(Exception, match="Model not found in cost map"):
default_video_cost_calculator(
model="unknown-model",
duration_seconds=5.0,
custom_llm_provider="openai",
)
def test_video_generation_cost_with_custom_model_info(self):
"""Test that custom model_info pricing is applied for video generation.
When a deployment has custom pricing via model_info, it should be used
instead of looking up the global litellm.model_cost map.
Related: https://github.com/BerriAI/litellm/issues/21907
"""
model_info = {
"output_cost_per_video_per_second": 0.05,
}
cost = default_video_cost_calculator(
model="my-custom-video-model",
duration_seconds=10.0,
model_info=model_info,
)
assert cost == 0.5
def test_video_generation_cost_custom_model_info_fallback_to_per_second(self):
"""Test that output_cost_per_second is used as fallback when
output_cost_per_video_per_second is not set in custom model_info.
Related: https://github.com/BerriAI/litellm/issues/21907
"""
model_info = {
"output_cost_per_second": 0.10,
}
cost = default_video_cost_calculator(
model="my-custom-video-model",
duration_seconds=5.0,
model_info=model_info,
)
assert cost == 0.5
def test_video_generation_cost_1080p_tier_via_default_calculator(self):
"""default_video_cost_calculator uses output_cost_per_second_1080p when requested."""
from litellm.cost_calculator import default_video_cost_calculator
model_info = {
"output_cost_per_second": 0.05,
"output_cost_per_second_1080p": 0.08,
}
cost = default_video_cost_calculator(
model="my-custom-video-model",
duration_seconds=10.0,
model_info=model_info,
video_resolution="1080p",
)
assert cost == 0.8
def test_video_generation_cost_custom_pricing_through_completion_cost(self):
"""Test that custom video pricing flows through completion_cost via litellm_logging_obj.
This tests the full cost calculation path: completion_cost extracts model_info
from litellm_logging_obj.litellm_params.metadata.model_info and passes it to
the video cost calculator.
Related: https://github.com/BerriAI/litellm/issues/21907
"""
from litellm.cost_calculator import completion_cost
# Create mock response with usage containing duration_seconds
mock_response = MagicMock()
mock_response.usage = MagicMock()
mock_response.usage.duration_seconds = 10.0
type(mock_response)._hidden_params = {}
# Create mock litellm_logging_obj with custom pricing
mock_logging_obj = MagicMock()
mock_logging_obj.litellm_params = {
"metadata": {
"model_info": {
"output_cost_per_video_per_second": 0.05,
}
}
}
cost = completion_cost(
completion_response=mock_response,
model="openai/hunyuanvideo",
call_type="create_video",
custom_llm_provider="openai",
custom_pricing=True,
litellm_logging_obj=mock_logging_obj,
)
assert cost == 0.5
def test_completion_cost_video_generation_1080p_tier(self):
"""create_video cost uses output_cost_per_second_1080p when usage.video_resolution is 1080p."""
from litellm.cost_calculator import completion_cost
mock_response = MagicMock()
mock_response.usage = MagicMock()
mock_response.usage.duration_seconds = 10.0
mock_response.usage.video_resolution = "1080p"
type(mock_response)._hidden_params = {}
mock_logging_obj = MagicMock()
mock_logging_obj.litellm_params = {
"metadata": {
"model_info": {
"output_cost_per_second": 0.05,
"output_cost_per_second_1080p": 0.08,
}
}
}
cost = completion_cost(
completion_response=mock_response,
model="gemini/veo-3.1-lite-generate-preview",
call_type="create_video",
custom_llm_provider="gemini",
custom_pricing=True,
litellm_logging_obj=mock_logging_obj,
)
assert abs(cost - 0.8) < 0.001
def test_completion_cost_video_edit_uses_video_calculator(self):
"""video_edit is charged via the same video cost path as create_video."""
from litellm.cost_calculator import completion_cost
mock_response = MagicMock()
mock_response.usage = MagicMock()
mock_response.usage.duration_seconds = 10.0
type(mock_response)._hidden_params = {}
mock_logging_obj = MagicMock()
mock_logging_obj.litellm_params = {
"metadata": {
"model_info": {
"output_cost_per_video_per_second": 0.05,
}
}
}
cost = completion_cost(
completion_response=mock_response,
model="vertex_ai/veo-3.1-generate-001",
call_type="video_edit",
custom_llm_provider="vertex_ai",
custom_pricing=True,
litellm_logging_obj=mock_logging_obj,
)
assert cost == 0.5
def test_completion_cost_video_custom_pricing_under_litellm_metadata(self):
"""Video routes store deployment model_info under litellm_metadata, not metadata.
Regression for https://github.com/BerriAI/litellm/issues/36483: custom video
pricing was silently ignored because completion_cost only read metadata.
"""
from litellm.cost_calculator import completion_cost
mock_response = MagicMock()
mock_response.usage = {"duration_seconds": 10.0}
type(mock_response)._hidden_params = {}
mock_logging_obj = MagicMock()
mock_logging_obj.litellm_params = {
"litellm_metadata": {
"model_info": {
"output_cost_per_video_per_second": 0.18,
}
}
}
cost = completion_cost(
completion_response=mock_response,
model="runwayml/seedance2",
call_type="create_video",
custom_llm_provider="runwayml",
custom_pricing=True,
litellm_logging_obj=mock_logging_obj,
)
assert abs(cost - 1.8) < 0.001
def test_completion_cost_video_uses_provider_reported_cost_without_custom_pricing(self):
"""With no custom pricing, the provider's own reported cost wins over a duration estimate."""
from litellm.cost_calculator import completion_cost
mock_response = MagicMock()
mock_response.usage = {
"duration_seconds": 5.0,
"video_resolution": "720p",
"provider_reported_cost_usd": 0.31,
}
type(mock_response)._hidden_params = {}
cost = completion_cost(
completion_response=mock_response,
model="runwayml/gen4_turbo",
call_type="create_video",
custom_llm_provider="runwayml",
)
assert cost == 0.31
def test_completion_cost_video_custom_pricing_beats_provider_reported_cost(self):
"""Deployment-level custom pricing overrides the provider's reported cost."""
from litellm.cost_calculator import completion_cost
mock_response = MagicMock()
mock_response.usage = {
"duration_seconds": 10.0,
"provider_reported_cost_usd": 0.31,
}
type(mock_response)._hidden_params = {}
mock_logging_obj = MagicMock()
mock_logging_obj.litellm_params = {
"metadata": {
"model_info": {
"output_cost_per_video_per_second": 0.18,
}
}
}
cost = completion_cost(
completion_response=mock_response,
model="runwayml/seedance2",
call_type="create_video",
custom_llm_provider="runwayml",
custom_pricing=True,
litellm_logging_obj=mock_logging_obj,
)
assert abs(cost - 1.8) < 0.001
def test_video_generation_with_files(self):
"""Test video generation with file uploads."""
config = OpenAIVideoConfig()
# Mock file data
mock_file = MagicMock()
mock_file.read.return_value = b"fake_image_data"
data, files, returned_api_base = config.transform_video_create_request(
model="sora-2",
prompt="Test video with image",
api_base="https://api.openai.com/v1/videos",
video_create_optional_request_params={
"input_reference": mock_file,
"seconds": "8",
"size": "720x1280",
},
litellm_params=MagicMock(),
headers={},
)
assert data["model"] == "sora-2"
assert data["prompt"] == "Test video with image"
assert len(files) > 0 # Should have files when input_reference is provided
def test_video_generation_environment_validation(self):
"""Test video generation environment validation."""
config = OpenAIVideoConfig()
# Test environment validation
headers = config.validate_environment(
headers={}, model="sora-2", api_key="test-api-key"
)
assert "Authorization" in headers
assert headers["Authorization"] == "Bearer test-api-key"
def test_video_generation_uses_api_key_from_litellm_params(self):
"""Test that video generation handler uses api_key from litellm_params when function parameter is None."""
handler = BaseLLMHTTPHandler()
config = OpenAIVideoConfig()
# Mock the validate_environment method to capture the api_key passed to it
with patch.object(config, "validate_environment") as mock_validate:
mock_validate.return_value = {"Authorization": "Bearer deployment-api-key"}
# Mock the transform and HTTP client
with patch.object(
config, "transform_video_create_request"
) as mock_transform:
mock_transform.return_value = (
{"model": "sora-2", "prompt": "test"},
[],
"https://api.openai.com/v1/videos",
)
# Mock the transform_video_create_response to avoid needing a real response
with patch.object(
config, "transform_video_create_response"
) as mock_transform_response:
mock_video_object = MagicMock()
mock_video_object.id = "video_123"
mock_video_object.object = "video"
mock_video_object.status = "queued"
mock_transform_response.return_value = mock_video_object
mock_response = MagicMock()
mock_response.json.return_value = {
"id": "video_123",
"object": "video",
"status": "queued",
"created_at": 1712697600,
"model": "sora-2",
}
mock_response.status_code = 200
mock_client = MagicMock()
mock_client.post.return_value = mock_response
with patch(
"litellm.llms.custom_httpx.llm_http_handler._get_httpx_client",
return_value=mock_client,
):
result = handler.video_generation_handler(
model="sora-2",
prompt="test prompt",
video_generation_provider_config=config,
video_generation_optional_request_params={},
custom_llm_provider="openai",
litellm_params={
"api_key": "deployment-api-key",
"api_base": "https://api.openai.com/v1",
},
logging_obj=MagicMock(),
timeout=5.0,
api_key=None, # Function parameter is None
_is_async=False,
)
# Verify validate_environment was called with api_key from litellm_params
mock_validate.assert_called_once()
call_args = mock_validate.call_args
assert call_args.kwargs["api_key"] == "deployment-api-key"
def test_video_generation_url_generation(self):
"""Test video generation URL generation."""
config = OpenAIVideoConfig()
# Test URL generation
url = config.get_complete_url(
model="sora-2", api_base="https://api.openai.com/v1", litellm_params={}
)
assert url == "https://api.openai.com/v1/videos"
def test_video_generation_parameter_mapping(self):
"""Test video generation parameter mapping."""
config = OpenAIVideoConfig()
# Test parameter mapping
mapped_params = config.map_openai_params(
video_create_optional_params={
"seconds": "8",
"size": "720x1280",
"user": "test-user",
},
model="sora-2",
drop_params=False,
)
assert mapped_params["seconds"] == "8"
assert mapped_params["size"] == "720x1280"
assert mapped_params["user"] == "test-user"
def test_video_generation_unsupported_parameters(self):
"""Test video generation with provider-specific parameters via extra_body."""
from litellm.videos.utils import VideoGenerationRequestUtils
# Test that provider-specific parameters can be passed via extra_body
# This allows support for Vertex AI and Gemini specific parameters
result = VideoGenerationRequestUtils.get_optional_params_video_generation(
model="sora-2",
video_generation_provider_config=OpenAIVideoConfig(),
video_generation_optional_params={
"seconds": "8",
"extra_body": {"vertex_ai_param": "value", "gemini_param": "value2"},
},
)
# extra_body params should be merged into the result
assert result["seconds"] == "8"
assert result["vertex_ai_param"] == "value"
assert result["gemini_param"] == "value2"
# extra_body itself should be removed from the result
assert "extra_body" not in result
def test_video_generation_types(self):
"""Test video generation type definitions."""
# Test VideoObject
video_obj = VideoObject(
id="test_id",
object="video",
status="completed",
created_at=1712697600,
model="sora-2",
)
assert video_obj.id == "test_id"
assert video_obj.object == "video"
assert video_obj.status == "completed"
# Test dictionary-like access
assert video_obj["id"] == "test_id"
assert video_obj["status"] == "completed"
assert "id" in video_obj
assert video_obj.get("id") == "test_id"
assert video_obj.get("nonexistent", "default") == "default"
# Test JSON serialization
json_data = video_obj.json()
assert json_data["id"] == "test_id"
assert json_data["object"] == "video"
def test_video_generation_response_types(self):
"""Test video generation response types."""
# Test VideoResponse
video_obj = VideoObject(
id="test_id", object="video", status="completed", created_at=1712697600
)
response = VideoResponse(data=[video_obj])
assert len(response.data) == 1
assert response.data[0].id == "test_id"
# Test dictionary-like access
assert response["data"][0]["id"] == "test_id"
assert "data" in response
assert response.get("data")[0]["id"] == "test_id"
# Test JSON serialization
json_data = response.json()
assert len(json_data["data"]) == 1
assert json_data["data"][0]["id"] == "test_id"
def test_video_status_basic(self):
"""Test basic video status functionality."""
# Use mock_response parameter for reliable testing
response = video_status(
video_id="video_123",
model="sora-2",
mock_response={
"id": "video_123",
"object": "video",
"status": "completed",
"created_at": 1712697600,
"completed_at": 1712697660,
"model": "sora-2",
"progress": 100,
"size": "720x1280",
"seconds": "8",
},
)
assert isinstance(response, VideoObject)
assert response.id == "video_123"
assert response.status == "completed"
assert response.progress == 100
assert response.model == "sora-2"
def test_video_status_with_mock_response(self):
"""Test video status with mock response."""
mock_data = {
"id": "video_456",
"object": "video",
"status": "processing",
"created_at": 1712697600,
"model": "sora-2",
"progress": 75,
"size": "1280x720",
"seconds": "10",
}
response = video_status(
video_id="video_456", model="sora-2", mock_response=mock_data
)
assert isinstance(response, VideoObject)
assert response.id == "video_456"
assert response.status == "processing"
assert response.progress == 75
assert response.model == "sora-2"
def test_video_status_async(self):
"""Test async video status functionality."""
mock_response = VideoObject(
id="video_async_123",
object="video",
status="queued",
created_at=1712697600,
model="sora-2",
progress=0,
)
# Mock the async_video_status_handler to return the mock_response
async_mock = AsyncMock(return_value=mock_response)
with patch.object(
videos_main.base_llm_http_handler, "async_video_status_handler", async_mock
):
with patch.object(
videos_main.base_llm_http_handler,
"video_status_handler",
side_effect=lambda **kwargs: async_mock(**kwargs),
):
import asyncio
async def test_async():
response = await avideo_status(
video_id="video_async_123", model="sora-2"
)
return response
response = asyncio.run(test_async())
assert isinstance(response, VideoObject)
assert response.id == "video_async_123"
assert response.status == "queued"
assert response.progress == 0
def test_video_status_parameter_validation(self):
"""Test video status parameter validation."""
# Test with minimal required parameters
response = video_status(
video_id="test_video_id",
model="sora-2",
mock_response={
"id": "test",
"object": "video",
"status": "completed",
"created_at": 1712697600,
},
)
assert isinstance(response, VideoObject)
assert response.id == "test"
def test_video_status_error_handling(self):
"""Test video status error handling."""
with patch.object(
videos_main.base_llm_http_handler,
"video_status_handler",
side_effect=Exception("API Error"),
):
with pytest.raises(litellm.APIConnectionError):
video_status(video_id="test_video_id", model="sora-2")
def test_video_status_request_transformation(self):
"""Test video status request transformation."""
config = OpenAIVideoConfig()
# Test request transformation
url, data = config.transform_video_status_retrieve_request(
video_id="video_123",
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
)
assert url == "https://api.openai.com/v1/videos/video_123"
assert data == {}
def test_video_status_response_transformation(self):
"""Test video status response transformation."""
config = OpenAIVideoConfig()
# Mock HTTP response
mock_http_response = MagicMock()
mock_http_response.json.return_value = {
"id": "video_789",
"object": "video",
"status": "completed",
"created_at": 1712697600,
"completed_at": 1712697660,
"model": "sora-2",
"progress": 100,
"size": "1280x720",
"seconds": "12",
}
response = config.transform_video_status_retrieve_response(
raw_response=mock_http_response, logging_obj=MagicMock()
)
assert isinstance(response, VideoObject)
assert response.id == "video_789"
assert response.status == "completed"
assert response.progress == 100
assert response.model == "sora-2"
def test_video_status_different_states(self):
"""Test video status with different video states."""
# Test queued state
queued_response = video_status(
video_id="video_queued",
model="sora-2",
mock_response={
"id": "video_queued",
"object": "video",
"status": "queued",
"created_at": 1712697600,
"model": "sora-2",
"progress": 0,
},
)
assert queued_response.status == "queued"
assert queued_response.progress == 0
# Test processing state
processing_response = video_status(
video_id="video_processing",
model="sora-2",
mock_response={
"id": "video_processing",
"object": "video",
"status": "processing",
"created_at": 1712697600,
"model": "sora-2",
"progress": 50,
},
)
assert processing_response.status == "processing"
assert processing_response.progress == 50
# Test completed state
completed_response = video_status(
video_id="video_completed",
model="sora-2",
mock_response={
"id": "video_completed",
"object": "video",
"status": "completed",
"created_at": 1712697600,
"completed_at": 1712697660,
"model": "sora-2",
"progress": 100,
},
)
assert completed_response.status == "completed"
assert completed_response.progress == 100
def test_video_status_with_remix_info(self):
"""Test video status with remix information."""
mock_data = {
"id": "video_remix_123",
"object": "video",
"status": "completed",
"created_at": 1712697600,
"completed_at": 1712697660,
"model": "sora-2",
"progress": 100,
"remixed_from_video_id": "video_original_123",
"size": "720x1280",
"seconds": "8",
}
response = video_status(
video_id="video_remix_123", model="sora-2", mock_response=mock_data
)
assert isinstance(response, VideoObject)
assert response.id == "video_remix_123"
assert response.status == "completed"
assert hasattr(response, "remixed_from_video_id")
assert response.remixed_from_video_id == "video_original_123"
def test_video_status_async_inside_async_function(self):
"""Test that sync video_status works inside async functions (no asyncio.run issues)."""
import asyncio
async def test_sync_in_async():
# This should work without asyncio.run() issues
# Use mock_response parameter for reliable testing
response = video_status(
video_id="video_sync_in_async",
model="sora-2",
mock_response={
"id": "video_sync_in_async",
"object": "video",
"status": "completed",
"created_at": 1712697600,
"model": "sora-2",
"progress": 100,
},
)
return response
response = asyncio.run(test_sync_in_async())
assert isinstance(response, VideoObject)
assert response.id == "video_sync_in_async"
assert response.status == "completed"
def test_video_status_url_construction(self):
"""Test video status URL construction."""
config = OpenAIVideoConfig()
# Test with different API bases
test_cases = [
(
"https://api.openai.com/v1/videos",
"video_123",
"https://api.openai.com/v1/videos/video_123",
),
(
"https://api.openai.com/v1/videos/",
"video_123",
"https://api.openai.com/v1/videos/video_123",
),
(
"https://custom-api.com/v1/videos",
"video_456",
"https://custom-api.com/v1/videos/video_456",
),
]
for api_base, video_id, expected_url in test_cases:
url, data = config.transform_video_status_retrieve_request(
video_id=video_id,
api_base=api_base,
litellm_params=MagicMock(),
headers={},
)
assert url == expected_url
assert data == {}
class TestVideoLogging:
"""Test video generation logging functionality."""
class TestVideoLogger(CustomLogger):
def __init__(self):
self.standard_logging_payload = None
async def async_log_success_event(
self, kwargs, response_obj, start_time, end_time
):
self.standard_logging_payload = kwargs.get("standard_logging_object")
@pytest.mark.asyncio
async def test_video_generation_logging(self):
"""Test that video generation creates proper logging payload with cost tracking.
Note: Uses AsyncMock with side_effect pattern for reliable parallel execution.
"""
custom_logger = self.TestVideoLogger()
litellm.logging_callback_manager._reset_all_callbacks()
await asyncio.wait_for(GLOBAL_LOGGING_WORKER.flush(), timeout=10.0)
litellm.callbacks = [custom_logger]
# Mock video generation response
mock_response = VideoObject(
id="video_test_123",
object="video",
status="queued",
created_at=1712697600,
model="sora-2",
size="720x1280",
seconds="8",
)
# Create async mock function to return the mock_response
async def mock_async_handler(*args, **kwargs):
return mock_response
# Patch the async_video_generation_handler method on base_llm_http_handler
with patch.object(
videos_main.base_llm_http_handler,
"async_video_generation_handler",
side_effect=mock_async_handler,
):
response = await litellm.avideo_generation(
prompt="A cat running in a garden",
model="sora-2",
seconds="8",
size="720x1280",
)
await asyncio.sleep(1) # Allow logging to complete
# Verify logging payload was created
assert custom_logger.standard_logging_payload is not None
payload = custom_logger.standard_logging_payload
# Verify basic logging fields
assert payload["call_type"] == "avideo_generation"
assert payload["status"] == "success"
assert payload["model"] == "sora-2"
assert payload["custom_llm_provider"] == "openai"
# Verify response object is recognized for logging
assert payload["response"] is not None
assert payload["response"]["id"] == "video_test_123"
assert payload["response"]["object"] == "video"
# Verify cost tracking is present (may be 0 in test environment)
assert payload["response_cost"] is not None
# Note: Cost calculation may not work in test environment due to mocking
# The important thing is that the logging payload is created and recognized
def test_openai_transform_video_content_request_empty_params():
"""OpenAI content transform should return empty params to ensure GET is used."""
config = OpenAIVideoConfig()
url, params = config.transform_video_content_request(
video_id="video_123",
api_base="https://api.openai.com/v1/videos",
litellm_params={},
headers={},
)
assert url == "https://api.openai.com/v1/videos/video_123/content"
assert params == {}
@pytest.mark.parametrize(
"variant,expected_suffix",
[
("thumbnail", "?variant=thumbnail"),
("spritesheet", "?variant=spritesheet"),
],
)
def test_openai_transform_video_content_request_with_variant(variant, expected_suffix):
"""OpenAI content transform should append ?variant= when variant is provided."""
config = OpenAIVideoConfig()
url, params = config.transform_video_content_request(
video_id="video_123",
api_base="https://api.openai.com/v1/videos",
litellm_params={},
headers={},
variant=variant,
)
assert url == f"https://api.openai.com/v1/videos/video_123/content{expected_suffix}"
assert params == {}
def test_openai_transform_video_content_request_variant_none_no_query_param():
"""OpenAI content transform should NOT append ?variant= when variant is None."""
config = OpenAIVideoConfig()
url, params = config.transform_video_content_request(
video_id="video_123",
api_base="https://api.openai.com/v1/videos",
litellm_params={},
headers={},
variant=None,
)
assert "variant" not in url
assert url == "https://api.openai.com/v1/videos/video_123/content"
def test_video_content_handler_passes_variant_to_url():
"""HTTP handler should pass variant through to the final URL."""
from litellm.llms.custom_httpx.http_handler import HTTPHandler
from litellm.types.router import GenericLiteLLMParams
if hasattr(litellm, "in_memory_llm_clients_cache"):
litellm.in_memory_llm_clients_cache.flush_cache()
handler = BaseLLMHTTPHandler()
config = OpenAIVideoConfig()
mock_client = MagicMock(spec=HTTPHandler)
mock_response = MagicMock()
mock_response.content = b"thumbnail-bytes"
mock_client.get.return_value = mock_response
with patch(
"litellm.llms.custom_httpx.llm_http_handler._get_httpx_client",
return_value=mock_client,
):
result = handler.video_content_handler(
video_id="video_abc",
video_content_provider_config=config,
custom_llm_provider="openai",
litellm_params=GenericLiteLLMParams(api_base="https://api.openai.com/v1"),
logging_obj=MagicMock(),
timeout=5.0,
api_key="sk-test",
client=mock_client,
_is_async=False,
variant="thumbnail",
)
assert result == b"thumbnail-bytes"
called_url = mock_client.get.call_args.kwargs["url"]
assert (
called_url
== "https://api.openai.com/v1/videos/video_abc/content?variant=thumbnail"
)
def test_video_content_handler_uses_get_for_openai():
"""HTTP handler must use GET (not POST) for OpenAI content download."""
from litellm.llms.custom_httpx.http_handler import HTTPHandler
from litellm.types.router import GenericLiteLLMParams
# Clear the HTTP client cache to prevent test isolation issues
# In CI, a cached real HTTPHandler from a previous test might bypass the mock
if hasattr(litellm, "in_memory_llm_clients_cache"):
litellm.in_memory_llm_clients_cache.flush_cache()
handler = BaseLLMHTTPHandler()
config = OpenAIVideoConfig()
# Use spec=HTTPHandler so isinstance(mock_client, HTTPHandler) returns True,
# ensuring the handler uses our mock directly instead of creating a new client.
mock_client = MagicMock(spec=HTTPHandler)
mock_response = MagicMock()
mock_response.content = b"mp4-bytes"
mock_client.get.return_value = mock_response
# Patch _get_httpx_client to ensure no real HTTP client is created
# This prevents test isolation issues where isinstance check might fail
with patch(
"litellm.llms.custom_httpx.llm_http_handler._get_httpx_client"
) as mock_get_client:
mock_get_client.return_value = mock_client
result = handler.video_content_handler(
video_id="video_abc",
video_content_provider_config=config,
custom_llm_provider="openai",
litellm_params=GenericLiteLLMParams(api_base="https://api.openai.com/v1"),
logging_obj=MagicMock(),
timeout=5.0,
api_key="sk-test",
client=mock_client,
_is_async=False,
)
assert result == b"mp4-bytes"
mock_client.get.assert_called_once()
assert not mock_client.post.called
called_url = mock_client.get.call_args.kwargs["url"]
assert called_url == "https://api.openai.com/v1/videos/video_abc/content"
def test_video_content_respects_api_base_and_api_key_from_kwargs():
"""Test that video_content respects api_base and api_key from kwargs (simulating database entry)."""
from litellm.videos.main import video_content
# Mock the handler to capture litellm_params
captured_litellm_params = None
def capture_litellm_params(*args, **kwargs):
nonlocal captured_litellm_params
captured_litellm_params = kwargs.get("litellm_params")
return b"mp4-bytes"
with patch("litellm.videos.main.base_llm_http_handler") as mock_handler:
mock_handler.video_content_handler = capture_litellm_params
# Call video_content with api_base and api_key in kwargs (simulating database entry)
# This simulates how the router passes model config from database via **kwargs
result = video_content(
video_id="video_test_123",
custom_llm_provider="azure",
api_base="https://test-resource.openai.azure.com/", # Passed via kwargs by router
api_key="test-api-key-from-db", # Passed via kwargs by router
)
# Verify that api_base and api_key from kwargs were included in litellm_params
assert captured_litellm_params is not None
assert (
captured_litellm_params.get("api_base")
== "https://test-resource.openai.azure.com/"
)
assert captured_litellm_params.get("api_key") == "test-api-key-from-db"
assert result == b"mp4-bytes"
def test_openai_video_config_has_async_transform():
"""Ensure OpenAIVideoConfig exposes async_transform_video_content_response at runtime."""
cfg = OpenAIVideoConfig()
assert callable(getattr(cfg, "async_transform_video_content_response", None))
def test_gemini_video_config_has_async_transform():
"""Ensure GeminiVideoConfig exposes async_transform_video_content_response at runtime."""
cfg = GeminiVideoConfig()
assert callable(getattr(cfg, "async_transform_video_content_response", None))
def test_encode_video_id_with_provider_handles_azure_video_prefix():
"""
Test that encode_video_id_with_provider correctly encodes Azure/OpenAI video IDs
that start with 'video_' prefix.
This test verifies the fix for the issue where Azure returns video IDs like
'video_69323201cf6081909263f751f89991e6', which were previously skipped
from encoding, causing video status retrieval to default to 'openai' provider.
"""
from litellm.types.videos.utils import (
decode_video_id_with_provider,
encode_video_id_with_provider,
)
# Test case: Azure returns a video ID starting with 'video_'
raw_azure_video_id = "video_69323201cf6081909263f751f89991e6"
provider = "azure"
model_id = "azure/sora-2"
# Encode the video ID with provider information
encoded_id = encode_video_id_with_provider(
video_id=raw_azure_video_id, provider=provider, model_id=model_id
)
# Verify the ID was encoded (should be different from the original)
assert encoded_id != raw_azure_video_id
assert encoded_id.startswith("video_")
# Decode the encoded ID to verify provider information is preserved
decoded = decode_video_id_with_provider(encoded_id)
assert decoded.get("custom_llm_provider") == provider
assert decoded.get("model_id") == model_id
assert decoded.get("video_id") == raw_azure_video_id
# Verify that encoding an already-encoded ID doesn't double-encode it
encoded_twice = encode_video_id_with_provider(
video_id=encoded_id, provider=provider, model_id=model_id
)
assert encoded_twice == encoded_id # Should return the same encoded ID
class TestVideoListTransformation:
"""Tests for video list request/response transformation with provider ID encoding."""
def test_transform_video_list_response_encodes_first_id_and_last_id(self):
"""Verify that first_id and last_id are encoded with provider metadata."""
config = OpenAIVideoConfig()
mock_http_response = MagicMock()
mock_http_response.json.return_value = {
"object": "list",
"data": [
{
"id": "video_aaa",
"object": "video",
"model": "sora-2",
"status": "completed",
},
{
"id": "video_bbb",
"object": "video",
"model": "sora-2",
"status": "completed",
},
],
"first_id": "video_aaa",
"last_id": "video_bbb",
"has_more": False,
}
result = config.transform_video_list_response(
raw_response=mock_http_response,
logging_obj=MagicMock(),
custom_llm_provider="azure",
)
from litellm.types.videos.utils import decode_video_id_with_provider
# data[].id should be encoded
for item in result["data"]:
decoded = decode_video_id_with_provider(item["id"])
assert decoded["custom_llm_provider"] == "azure"
# first_id and last_id should also be encoded
first_decoded = decode_video_id_with_provider(result["first_id"])
assert first_decoded["custom_llm_provider"] == "azure"
assert first_decoded["video_id"] == "video_aaa"
assert first_decoded["model_id"] == "sora-2"
last_decoded = decode_video_id_with_provider(result["last_id"])
assert last_decoded["custom_llm_provider"] == "azure"
assert last_decoded["video_id"] == "video_bbb"
assert last_decoded["model_id"] == "sora-2"
def test_transform_video_list_response_no_provider_leaves_ids_unchanged(self):
"""When custom_llm_provider is None, all IDs should remain unchanged."""
config = OpenAIVideoConfig()
mock_http_response = MagicMock()
mock_http_response.json.return_value = {
"object": "list",
"data": [
{
"id": "video_aaa",
"object": "video",
"model": "sora-2",
"status": "completed",
},
],
"first_id": "video_aaa",
"last_id": "video_aaa",
"has_more": False,
}
result = config.transform_video_list_response(
raw_response=mock_http_response,
logging_obj=MagicMock(),
custom_llm_provider=None,
)
assert result["data"][0]["id"] == "video_aaa"
assert result["first_id"] == "video_aaa"
assert result["last_id"] == "video_aaa"
def test_transform_video_list_response_missing_pagination_fields(self):
"""first_id / last_id may be absent or null; should not raise."""
config = OpenAIVideoConfig()
mock_http_response = MagicMock()
mock_http_response.json.return_value = {
"object": "list",
"data": [
{
"id": "video_aaa",
"object": "video",
"model": "sora-2",
"status": "completed",
},
],
"has_more": False,
}
result = config.transform_video_list_response(
raw_response=mock_http_response,
logging_obj=MagicMock(),
custom_llm_provider="azure",
)
# data[].id should still be encoded
from litellm.types.videos.utils import decode_video_id_with_provider
decoded = decode_video_id_with_provider(result["data"][0]["id"])
assert decoded["custom_llm_provider"] == "azure"
# first_id / last_id should not be present
assert "first_id" not in result
assert "last_id" not in result
def test_transform_video_list_request_decodes_after_parameter(self):
"""Encoded 'after' cursor should be decoded back to the raw provider ID."""
from litellm.types.videos.utils import encode_video_id_with_provider
config = OpenAIVideoConfig()
raw_id = "video_69888baee890819086dd3366bfc372fe"
encoded_id = encode_video_id_with_provider(raw_id, "azure", "sora-2")
url, params = config.transform_video_list_request(
api_base="https://my-resource.openai.azure.com/openai/v1/videos",
litellm_params=MagicMock(),
headers={},
after=encoded_id,
limit=10,
)
assert params["after"] == raw_id
assert params["limit"] == "10"
def test_transform_video_list_request_passes_through_plain_after(self):
"""A plain (non-encoded) 'after' value should pass through unchanged."""
config = OpenAIVideoConfig()
url, params = config.transform_video_list_request(
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
after="video_plain_id",
)
assert params["after"] == "video_plain_id"
def test_transform_video_list_roundtrip(self):
"""first_id from list response should decode correctly when used as after parameter."""
config = OpenAIVideoConfig()
# Simulate a list response
mock_http_response = MagicMock()
mock_http_response.json.return_value = {
"object": "list",
"data": [
{
"id": "video_aaa",
"object": "video",
"model": "sora-2",
"status": "completed",
},
{
"id": "video_bbb",
"object": "video",
"model": "sora-2",
"status": "completed",
},
],
"first_id": "video_aaa",
"last_id": "video_bbb",
"has_more": True,
}
list_result = config.transform_video_list_response(
raw_response=mock_http_response,
logging_obj=MagicMock(),
custom_llm_provider="azure",
)
# Use the encoded last_id as the 'after' cursor for the next page
_, params = config.transform_video_list_request(
api_base="https://my-resource.openai.azure.com/openai/v1/videos",
litellm_params=MagicMock(),
headers={},
after=list_result["last_id"],
)
# The after param sent to the upstream API should be the raw video ID
assert params["after"] == "video_bbb"
class TestVideoEndpointsProxyLitellmParams:
"""Test that video proxy endpoints (status, content, remix) respect litellm_params from proxy config."""
@pytest.fixture
def client_with_vertex_config(self, monkeypatch):
"""Create a test client with a proxy config that includes Vertex AI model with litellm_params."""
import asyncio
import tempfile
import yaml
from fastapi import FastAPI
from fastapi.testclient import TestClient
from litellm.proxy.proxy_server import (
cleanup_router_config_variables,
initialize,
router,
)
from litellm.proxy.video_endpoints.endpoints import router as video_router
# Clean up any existing router config
cleanup_router_config_variables()
# Create inline config
config = {
"model_list": [
{
"model_name": "vertex-ai-sora-2",
"litellm_params": {
"model": "vertex_ai/veo-2.0-generate-001",
"vertex_project": "test-project-123",
"vertex_location": "global",
"vertex_credentials": "/path/to/test-credentials.json",
},
}
]
}
# Write config to temporary file
with tempfile.NamedTemporaryFile(mode="w", suffix=".yaml", delete=False) as f:
yaml.dump(config, f)
config_fp = f.name
try:
# Initialize the proxy with the test config
app = FastAPI()
asyncio.run(initialize(config=config_fp, debug=True))
app.include_router(router)
app.include_router(video_router)
return TestClient(app)
finally:
# Clean up temporary file
import os
if os.path.exists(config_fp):
os.unlink(config_fp)
@pytest.fixture
def mock_video_generation_response(self):
"""Mock video generation response with encoded video_id."""
from litellm.types.videos.utils import encode_video_id_with_provider
# Create an encoded video_id that includes provider and model_id
original_video_id = "projects/test-project-123/locations/global/publishers/google/models/veo-2.0-generate-001/operations/test-operation-123"
encoded_video_id = encode_video_id_with_provider(
video_id=original_video_id,
provider="vertex_ai",
model_id="veo-2.0-generate-001",
)
return VideoObject(
id=encoded_video_id,
object="video",
status="processing",
created_at=1712697600,
model="vertex_ai/veo-2.0-generate-001",
)
@pytest.fixture
def mock_video_status_response(self):
"""Mock video status response."""
return VideoObject(
id="video_test_123",
object="video",
status="completed",
created_at=1712697600,
completed_at=1712697660,
model="vertex_ai/veo-2.0-generate-001",
progress=100,
)
@pytest.fixture
def mock_video_content_response(self):
"""Mock video content response (raw bytes)."""
return b"fake_video_content_bytes"
@pytest.mark.asyncio
async def test_video_status_respects_litellm_params(
self,
client_with_vertex_config,
mock_video_generation_response,
mock_video_status_response,
):
"""Test that video_status endpoint uses litellm_params from proxy config."""
from unittest.mock import AsyncMock, MagicMock, patch
# Create an encoded video_id
encoded_video_id = mock_video_generation_response.id
# Mock the router instance
mock_router_instance = MagicMock()
mock_router_instance.resolve_model_name_from_model_id.return_value = (
"vertex-ai-sora-2"
)
mock_router_instance.model_names = {"vertex-ai-sora-2"}
mock_router_instance.has_model_id.return_value = False
# Mock route_request to capture the data being passed
# route_request should return a coroutine (not await it), so we return a coroutine
async def mock_route_request_func(*args, **kwargs):
return mock_video_status_response
# Create a coroutine that will be added to tasks
def create_mock_coroutine(*args, **kwargs):
return mock_route_request_func(*args, **kwargs)
with patch("litellm.proxy.proxy_server.llm_router", mock_router_instance):
with patch(
"litellm.proxy.common_request_processing.route_request",
side_effect=create_mock_coroutine,
) as mock_route_request:
# Make request to video_status endpoint
response = client_with_vertex_config.get(
f"/v1/videos/{encoded_video_id}",
headers={"Authorization": "Bearer sk-1234"},
)
# Verify the endpoint was called
assert response.status_code == 200, f"Response: {response.text}"
# Verify that route_request was called
assert mock_route_request.called
call_args = mock_route_request.call_args
# route_request is called with data as a keyword argument
data_passed = (
call_args.kwargs.get("data", {})
if call_args.kwargs
else (
call_args.args[0]
if call_args.args and len(call_args.args) > 0
else {}
)
)
# Verify that model was resolved and added to data
assert data_passed.get("model") == "vertex-ai-sora-2", (
f"Expected model to be 'vertex-ai-sora-2', got '{data_passed.get('model')}'. "
f"Full data: {data_passed}, call_args: {call_args}"
)
# Verify that custom_llm_provider is set from decoded video_id
assert data_passed.get("custom_llm_provider") == "vertex_ai", (
f"Expected custom_llm_provider to be 'vertex_ai', got '{data_passed.get('custom_llm_provider')}'. "
f"Full data: {data_passed}"
)
@pytest.mark.asyncio
async def test_video_content_respects_litellm_params(
self,
client_with_vertex_config,
mock_video_generation_response,
mock_video_content_response,
):
"""Test that video_content endpoint uses litellm_params from proxy config."""
from unittest.mock import AsyncMock, MagicMock, patch
# Create an encoded video_id
encoded_video_id = mock_video_generation_response.id
# Mock the router instance
mock_router_instance = MagicMock()
mock_router_instance.resolve_model_name_from_model_id.return_value = (
"vertex-ai-sora-2"
)
mock_router_instance.model_names = {"vertex-ai-sora-2"}
mock_router_instance.has_model_id.return_value = False
# Mock route_request to capture the data being passed
# route_request should return a coroutine (not await it), so we return a coroutine
async def mock_route_request_func(*args, **kwargs):
return mock_video_content_response
# Create a coroutine that will be added to tasks
def create_mock_coroutine(*args, **kwargs):
return mock_route_request_func(*args, **kwargs)
with patch("litellm.proxy.proxy_server.llm_router", mock_router_instance):
with patch(
"litellm.proxy.common_request_processing.route_request",
side_effect=create_mock_coroutine,
) as mock_route_request:
# Make request to video_content endpoint
response = client_with_vertex_config.get(
f"/v1/videos/{encoded_video_id}/content",
headers={"Authorization": "Bearer sk-1234"},
)
# Verify the endpoint was called
assert response.status_code == 200, f"Response: {response.text}"
# Verify that route_request was called
assert mock_route_request.called
call_args = mock_route_request.call_args
# route_request is called with data as a keyword argument
data_passed = (
call_args.kwargs.get("data", {})
if call_args.kwargs
else (
call_args.args[0]
if call_args.args and len(call_args.args) > 0
else {}
)
)
# Verify that model was resolved and added to data
assert data_passed.get("model") == "vertex-ai-sora-2", (
f"Expected model to be 'vertex-ai-sora-2', got '{data_passed.get('model')}'. "
f"Full data: {data_passed}, call_args: {call_args}"
)
# Verify that custom_llm_provider is correctly set from decoded video_id (not "openai")
assert data_passed.get("custom_llm_provider") == "vertex_ai", (
f"Expected custom_llm_provider to be 'vertex_ai', got '{data_passed.get('custom_llm_provider')}'. "
f"Full data: {data_passed}"
)
@pytest.mark.asyncio
async def test_video_content_preserves_custom_llm_provider_from_decoded_id(
self,
client_with_vertex_config,
mock_video_generation_response,
mock_video_content_response,
):
"""Test that video_content preserves custom_llm_provider from decoded video_id."""
from unittest.mock import AsyncMock, MagicMock, patch
# Create an encoded video_id
encoded_video_id = mock_video_generation_response.id
# Mock the router instance
mock_router_instance = MagicMock()
mock_router_instance.resolve_model_name_from_model_id.return_value = (
"vertex-ai-sora-2"
)
mock_router_instance.model_names = {"vertex-ai-sora-2"}
mock_router_instance.has_model_id.return_value = False
# Mock route_request to capture the data being passed
# route_request should return a coroutine (not await it), so we return a coroutine
async def mock_route_request_func(*args, **kwargs):
return mock_video_content_response
# Create a coroutine that will be added to tasks
def create_mock_coroutine(*args, **kwargs):
return mock_route_request_func(*args, **kwargs)
with patch("litellm.proxy.proxy_server.llm_router", mock_router_instance):
with patch(
"litellm.proxy.common_request_processing.route_request",
side_effect=create_mock_coroutine,
) as mock_route_request:
# Make request to video_content endpoint
response = client_with_vertex_config.get(
f"/v1/videos/{encoded_video_id}/content",
headers={"Authorization": "Bearer sk-1234"},
)
# Verify the endpoint was called
assert response.status_code == 200, f"Response: {response.text}"
# Verify that route_request was called
assert mock_route_request.called
call_args = mock_route_request.call_args
# route_request is called with data as a keyword argument
data_passed = (
call_args.kwargs.get("data", {})
if call_args.kwargs
else (
call_args.args[0]
if call_args.args and len(call_args.args) > 0
else {}
)
)
# Most importantly: verify that custom_llm_provider is "vertex_ai" not "openai"
# This was the bug we fixed - it was defaulting to "openai" before
assert data_passed.get("custom_llm_provider") == "vertex_ai", (
f"Expected custom_llm_provider to be 'vertex_ai', "
f"but got '{data_passed.get('custom_llm_provider')}'. "
f"Full data: {data_passed}, call_args: {call_args}"
)
def test_video_remix_handler_uses_api_key_from_litellm_params():
"""Sync remix handler should fall back to litellm_params api_key when api_key param is None."""
handler = BaseLLMHTTPHandler()
config = OpenAIVideoConfig()
with patch.object(config, "validate_environment") as mock_validate:
mock_validate.return_value = {"Authorization": "Bearer deployment-key"}
with patch.object(config, "transform_video_remix_request") as mock_transform:
mock_transform.return_value = (
"https://api.openai.com/v1/videos/video_123/remix",
{"prompt": "remix it"},
)
with patch.object(config, "transform_video_remix_response") as mock_resp:
mock_resp.return_value = MagicMock()
mock_client = MagicMock()
mock_client.post.return_value = MagicMock(status_code=200)
with patch(
"litellm.llms.custom_httpx.llm_http_handler._get_httpx_client",
return_value=mock_client,
):
handler.video_remix_handler(
video_id="video_123",
prompt="remix it",
video_remix_provider_config=config,
custom_llm_provider="openai",
litellm_params={
"api_key": "deployment-key",
"api_base": "https://api.openai.com/v1",
},
logging_obj=MagicMock(),
timeout=5.0,
api_key=None,
_is_async=False,
)
mock_validate.assert_called_once()
assert mock_validate.call_args.kwargs["api_key"] == "deployment-key"
@pytest.mark.asyncio
async def test_async_video_remix_handler_uses_api_key_from_litellm_params():
"""Async remix handler should fall back to litellm_params api_key when api_key param is None."""
handler = BaseLLMHTTPHandler()
config = OpenAIVideoConfig()
with patch.object(config, "validate_environment") as mock_validate:
mock_validate.return_value = {"Authorization": "Bearer deployment-key"}
with patch.object(config, "transform_video_remix_request") as mock_transform:
mock_transform.return_value = (
"https://api.openai.com/v1/videos/video_123/remix",
{"prompt": "remix it"},
)
with patch.object(config, "transform_video_remix_response") as mock_resp:
mock_resp.return_value = MagicMock()
mock_client = MagicMock(spec=AsyncHTTPHandler)
mock_response = MagicMock(status_code=200)
mock_client.post = AsyncMock(return_value=mock_response)
with patch(
"litellm.llms.custom_httpx.llm_http_handler.get_async_httpx_client",
return_value=mock_client,
):
await handler.async_video_remix_handler(
video_id="video_123",
prompt="remix it",
video_remix_provider_config=config,
custom_llm_provider="openai",
litellm_params={
"api_key": "deployment-key",
"api_base": "https://api.openai.com/v1",
},
logging_obj=MagicMock(),
timeout=5.0,
api_key=None,
)
mock_validate.assert_called_once()
assert mock_validate.call_args.kwargs["api_key"] == "deployment-key"
def test_video_remix_handler_prefers_explicit_api_key():
"""Sync remix handler should prefer explicit api_key over litellm_params."""
handler = BaseLLMHTTPHandler()
config = OpenAIVideoConfig()
with patch.object(config, "validate_environment") as mock_validate:
mock_validate.return_value = {"Authorization": "Bearer explicit-key"}
with patch.object(config, "transform_video_remix_request") as mock_transform:
mock_transform.return_value = (
"https://api.openai.com/v1/videos/video_123/remix",
{"prompt": "remix it"},
)
with patch.object(config, "transform_video_remix_response") as mock_resp:
mock_resp.return_value = MagicMock()
mock_client = MagicMock()
mock_client.post.return_value = MagicMock(status_code=200)
with patch(
"litellm.llms.custom_httpx.llm_http_handler._get_httpx_client",
return_value=mock_client,
):
handler.video_remix_handler(
video_id="video_123",
prompt="remix it",
video_remix_provider_config=config,
custom_llm_provider="openai",
litellm_params={
"api_key": "deployment-key",
"api_base": "https://api.openai.com/v1",
},
logging_obj=MagicMock(),
timeout=5.0,
api_key="explicit-key",
_is_async=False,
)
mock_validate.assert_called_once()
assert mock_validate.call_args.kwargs["api_key"] == "explicit-key"
if __name__ == "__main__":
pytest.main([__file__])
# ===== Tests for new video endpoints (characters, edits, extensions) =====
class TestVideoCreateCharacter:
"""Tests for video_create_character / avideo_create_character."""
def test_video_create_character_transform_request(self):
"""Verify multipart form construction for POST /videos/characters."""
config = OpenAIVideoConfig()
fake_video = b"fake_video_bytes"
url, files_list = config.transform_video_create_character_request(
name="hero",
video=fake_video,
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
)
assert url == "https://api.openai.com/v1/videos/characters"
# Should have (name field) + (video file field) = 2 entries
assert len(files_list) == 2
field_names = [f[0] for f in files_list]
assert "name" in field_names
assert "video" in field_names
def test_video_create_character_sets_video_mimetype(self):
"""Ensure character video upload is sent as video/mp4."""
config = OpenAIVideoConfig()
fake_video = io.BytesIO(b"....ftyp....video-bytes")
fake_video.name = "character.mp4"
_, files_list = config.transform_video_create_character_request(
name="hero",
video=fake_video,
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
)
video_parts = [f for f in files_list if f[0] == "video"]
assert len(video_parts) == 1
video_tuple = video_parts[0][1]
assert video_tuple[0] == "character.mp4"
assert video_tuple[2] == "video/mp4"
def test_video_create_character_transform_response(self):
"""Verify CharacterObject is returned from response."""
from litellm.types.videos.main import CharacterObject
config = OpenAIVideoConfig()
mock_response = MagicMock()
mock_response.json.return_value = {
"id": "char_abc123",
"object": "character",
"created_at": 1712697600,
"name": "hero",
}
result = config.transform_video_create_character_response(
raw_response=mock_response,
logging_obj=MagicMock(),
)
assert isinstance(result, CharacterObject)
assert result.id == "char_abc123"
assert result.name == "hero"
def test_video_create_character_mock_response(self):
"""video_create_character returns CharacterObject on mock_response."""
from litellm.types.videos.main import CharacterObject
from litellm.videos.main import video_create_character
response = video_create_character(
name="hero",
video=b"fake",
mock_response={
"id": "char_abc",
"object": "character",
"created_at": 1712697600,
"name": "hero",
},
)
assert isinstance(response, CharacterObject)
assert response.id == "char_abc"
class TestVideoGetCharacter:
"""Tests for video_get_character / avideo_get_character."""
def test_video_get_character_transform_request(self):
"""Verify URL construction for GET /videos/characters/{character_id}."""
config = OpenAIVideoConfig()
url, params = config.transform_video_get_character_request(
character_id="char_xyz",
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
)
assert url == "https://api.openai.com/v1/videos/characters/char_xyz"
assert params == {}
def test_video_get_character_transform_response(self):
"""Verify CharacterObject is returned from GET response."""
from litellm.types.videos.main import CharacterObject
config = OpenAIVideoConfig()
mock_response = MagicMock()
mock_response.json.return_value = {
"id": "char_xyz",
"object": "character",
"created_at": 1712697600,
"name": "villain",
}
result = config.transform_video_get_character_response(
raw_response=mock_response,
logging_obj=MagicMock(),
)
assert isinstance(result, CharacterObject)
assert result.id == "char_xyz"
assert result.name == "villain"
def test_video_get_character_mock_response(self):
"""video_get_character returns CharacterObject on mock_response."""
from litellm.types.videos.main import CharacterObject
from litellm.videos.main import video_get_character
response = video_get_character(
character_id="char_xyz",
mock_response={
"id": "char_xyz",
"object": "character",
"created_at": 1712697600,
"name": "villain",
},
)
assert isinstance(response, CharacterObject)
assert response.id == "char_xyz"
class TestVideoEdit:
"""Tests for video_edit / avideo_edit."""
def test_video_edit_transform_request(self):
"""Verify JSON body with video.id for POST /videos/edits."""
config = OpenAIVideoConfig()
url, data, files = config.transform_video_edit_request(
prompt="make it brighter",
video_id="video_abc123",
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
)
assert url == "https://api.openai.com/v1/videos/edits"
assert data["prompt"] == "make it brighter"
assert data["video"]["id"] == "video_abc123"
assert files is None
def test_video_edit_transform_request_with_extra_body(self):
"""Extra body params are merged into request data."""
config = OpenAIVideoConfig()
url, data, files = config.transform_video_edit_request(
prompt="darken it",
video_id="video_abc123",
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
extra_body={"resolution": "1080p"},
)
assert data["resolution"] == "1080p"
assert files is None
def test_video_edit_mock_response(self):
"""video_edit returns VideoObject on mock_response."""
from litellm.videos.main import video_edit
response = video_edit(
video_id="video_abc123",
prompt="make it brighter",
mock_response={
"id": "video_edit_001",
"object": "video",
"status": "queued",
"created_at": 1712697600,
},
)
assert isinstance(response, VideoObject)
assert response.id == "video_edit_001"
def test_video_edit_strips_encoded_provider_from_video_id(self):
"""Provider-encoded video IDs are decoded before sending to API."""
from litellm.types.videos.utils import encode_video_id_with_provider
config = OpenAIVideoConfig()
encoded_id = encode_video_id_with_provider("raw_video_id", "openai", None)
url, data, files = config.transform_video_edit_request(
prompt="test",
video_id=encoded_id,
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
)
# The video.id in the request body should be the raw ID, not the encoded one
assert data["video"]["id"] == "raw_video_id"
assert files is None
class TestVideoExtension:
"""Tests for video_extension / avideo_extension."""
def test_video_extension_transform_request(self):
"""Verify JSON body with video.id + seconds for POST /videos/extensions."""
config = OpenAIVideoConfig()
url, data = config.transform_video_extension_request(
prompt="continue the scene",
video_id="video_abc123",
seconds="5",
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
)
assert url == "https://api.openai.com/v1/videos/extensions"
assert data["prompt"] == "continue the scene"
assert data["seconds"] == "5"
assert data["video"]["id"] == "video_abc123"
def test_video_extension_transform_request_with_extra_body(self):
"""Extra body params are merged into request data."""
config = OpenAIVideoConfig()
url, data = config.transform_video_extension_request(
prompt="extend",
video_id="video_abc123",
seconds="10",
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
extra_body={"model": "sora-2"},
)
assert data["model"] == "sora-2"
def test_video_extension_mock_response(self):
"""video_extension returns VideoObject on mock_response."""
from litellm.videos.main import video_extension
response = video_extension(
video_id="video_abc123",
prompt="continue the scene",
seconds="5",
mock_response={
"id": "video_ext_001",
"object": "video",
"status": "queued",
"created_at": 1712697600,
},
)
assert isinstance(response, VideoObject)
assert response.id == "video_ext_001"
def test_video_extension_strips_encoded_provider_from_video_id(self):
"""Provider-encoded video IDs are decoded before sending to API."""
from litellm.types.videos.utils import encode_video_id_with_provider
config = OpenAIVideoConfig()
encoded_id = encode_video_id_with_provider("raw_video_id", "openai", None)
url, data = config.transform_video_extension_request(
prompt="extend",
video_id=encoded_id,
seconds="5",
api_base="https://api.openai.com/v1/videos",
litellm_params=MagicMock(),
headers={},
)
assert data["video"]["id"] == "raw_video_id"
@pytest.fixture
def video_proxy_test_client():
from fastapi import FastAPI
from fastapi.testclient import TestClient
from litellm.proxy.auth.user_api_key_auth import user_api_key_auth
from litellm.proxy.video_endpoints.endpoints import router as video_router
app = FastAPI()
app.include_router(video_router)
app.dependency_overrides[user_api_key_auth] = lambda: MagicMock()
return TestClient(app)
def test_character_id_encode_decode_roundtrip():
from litellm.types.videos.utils import (
decode_character_id_with_provider,
encode_character_id_with_provider,
)
encoded = encode_character_id_with_provider(
character_id="char_raw_123",
provider="vertex_ai",
model_id="veo-2.0-generate-001",
)
decoded = decode_character_id_with_provider(encoded)
assert decoded["character_id"] == "char_raw_123"
assert decoded["custom_llm_provider"] == "vertex_ai"
assert decoded["model_id"] == "veo-2.0-generate-001"
def test_character_id_decode_handles_missing_base64_padding():
from litellm.types.videos.utils import (
decode_character_id_with_provider,
encode_character_id_with_provider,
)
encoded = encode_character_id_with_provider(
character_id="id",
provider="openai",
model_id="gpt-4o",
)
encoded_without_padding = encoded.rstrip("=")
decoded = decode_character_id_with_provider(encoded_without_padding)
assert decoded["character_id"] == "id"
assert decoded["custom_llm_provider"] == "openai"
assert decoded["model_id"] == "gpt-4o"
def test_video_create_character_target_model_names_returns_encoded_id(
video_proxy_test_client,
):
from litellm.proxy.common_request_processing import ProxyBaseLLMRequestProcessing
from litellm.types.videos.utils import decode_character_id_with_provider
captured_data = {}
async def _mock_base_process(self, **kwargs):
captured_data.update(self.data)
return {
"id": "char_upstream_123",
"object": "character",
"created_at": 1712697600,
"name": "hero",
}
with patch.object(
ProxyBaseLLMRequestProcessing,
"base_process_llm_request",
new=_mock_base_process,
):
response = video_proxy_test_client.post(
"/v1/videos/characters",
headers={"Authorization": "Bearer sk-1234"},
files={"video": ("character.mp4", b"fake-video", "video/mp4")},
data={
"name": "hero",
"target_model_names": "vertex-ai-sora-2",
"extra_body": json.dumps({"custom_llm_provider": "vertex_ai"}),
},
)
assert response.status_code == 200, response.text
response_json = response.json()
decoded = decode_character_id_with_provider(response_json["id"])
assert decoded["character_id"] == "char_upstream_123"
assert decoded["custom_llm_provider"] == "vertex_ai"
assert decoded["model_id"] == "vertex-ai-sora-2"
assert captured_data["model"] == "vertex-ai-sora-2"
assert captured_data["custom_llm_provider"] == "vertex_ai"
def test_video_get_character_accepts_encoded_character_id(video_proxy_test_client):
from litellm.proxy.common_request_processing import ProxyBaseLLMRequestProcessing
from litellm.types.videos.utils import (
decode_character_id_with_provider,
encode_character_id_with_provider,
)
captured_data = {}
async def _mock_base_process(self, **kwargs):
captured_data.update(self.data)
return {
"id": "char_upstream_123",
"object": "character",
"created_at": 1712697600,
"name": "hero",
}
encoded_character_id = encode_character_id_with_provider(
character_id="char_upstream_123",
provider="vertex_ai",
model_id="veo-2.0-generate-001",
)
mock_router = MagicMock()
mock_router.resolve_model_name_from_model_id.return_value = "vertex-ai-sora-2"
with patch("litellm.proxy.proxy_server.llm_router", mock_router):
with patch.object(
ProxyBaseLLMRequestProcessing,
"base_process_llm_request",
new=_mock_base_process,
):
response = video_proxy_test_client.get(
f"/v1/videos/characters/{encoded_character_id}",
headers={"Authorization": "Bearer sk-1234"},
)
assert response.status_code == 200, response.text
assert captured_data["character_id"] == "char_upstream_123"
assert captured_data["custom_llm_provider"] == "vertex_ai"
assert captured_data["model"] == "vertex-ai-sora-2"
response_decoded = decode_character_id_with_provider(response.json()["id"])
assert response_decoded["character_id"] == "char_upstream_123"
assert response_decoded["custom_llm_provider"] == "vertex_ai"
assert response_decoded["model_id"] == "veo-2.0-generate-001"
@pytest.mark.parametrize("endpoint", ["/v1/videos/edits", "/v1/videos/extensions"])
def test_edit_and_extension_support_custom_provider_from_extra_body(
video_proxy_test_client, endpoint
):
from litellm.proxy.common_request_processing import ProxyBaseLLMRequestProcessing
captured_data = {}
async def _mock_base_process(self, **kwargs):
captured_data.update(self.data)
return {
"id": "video_resp_123",
"object": "video",
"status": "queued",
"created_at": 1712697600,
}
payload = {
"prompt": "test",
"video": {"id": "video_raw_123"},
"extra_body": {"custom_llm_provider": "vertex_ai"},
}
if endpoint.endswith("extensions"):
payload["seconds"] = "4"
with patch.object(
ProxyBaseLLMRequestProcessing,
"base_process_llm_request",
new=_mock_base_process,
):
response = video_proxy_test_client.post(
endpoint,
headers={"Authorization": "Bearer sk-1234"},
json=payload,
)
assert response.status_code == 200, response.text
assert captured_data["custom_llm_provider"] == "vertex_ai"
@pytest.mark.parametrize(
"handler_name, path, form",
[
(
"video_edit",
"/v1/videos/edits",
{"model": "my-video-model", "prompt": "brighter", "video": "video_123"},
),
(
"video_extension",
"/v1/videos/extensions",
{"model": "my-video-model", "prompt": "continue", "seconds": "4", "video": "video_123"},
),
],
)
@pytest.mark.asyncio
async def test_edit_and_extension_read_cached_body_after_auth_consumes_stream(
handler_name, path, form
):
from urllib.parse import urlencode
from fastapi import Response
from starlette.requests import Request
import litellm.proxy.video_endpoints.endpoints as endpoints
from litellm.proxy._types import ProxyException, UserAPIKeyAuth
from litellm.proxy.common_utils.http_parsing_utils import _read_request_body
body = urlencode(form).encode()
stream = {"sent": False}
async def receive():
if stream["sent"]:
return {"type": "http.request", "body": b"", "more_body": False}
stream["sent"] = True
return {"type": "http.request", "body": body, "more_body": False}
request = Request(
{
"type": "http",
"method": "POST",
"path": path,
"headers": [
(b"content-type", b"application/x-www-form-urlencoded"),
(b"content-length", str(len(body)).encode()),
],
"query_string": b"",
},
receive,
)
await _read_request_body(request=request)
handler = getattr(endpoints, handler_name)
with pytest.raises(ProxyException) as exc_info:
await handler(
request=request,
fastapi_response=Response(),
user_api_key_dict=UserAPIKeyAuth(api_key="sk-1234"),
)
message = str(exc_info.value)
assert "Stream consumed" not in message
assert "my-video-model" in message
@pytest.mark.parametrize("endpoint", ["/v1/videos/edits", "/v1/videos/extensions"])
def test_edit_and_extension_route_with_encoded_video_ids(
video_proxy_test_client, endpoint
):
from litellm.proxy.common_request_processing import ProxyBaseLLMRequestProcessing
from litellm.types.videos.utils import encode_video_id_with_provider
captured_data = {}
async def _mock_base_process(self, **kwargs):
captured_data.update(self.data)
return {
"id": "video_resp_123",
"object": "video",
"status": "queued",
"created_at": 1712697600,
}
encoded_video_id = encode_video_id_with_provider(
video_id="video_raw_123",
provider="vertex_ai",
model_id="veo-2.0-generate-001",
)
payload = {"prompt": "test", "video": {"id": encoded_video_id}}
if endpoint.endswith("extensions"):
payload["seconds"] = "4"
mock_router = MagicMock()
mock_router.resolve_model_name_from_model_id.return_value = "vertex-ai-sora-2"
with patch("litellm.proxy.proxy_server.llm_router", mock_router):
with patch.object(
ProxyBaseLLMRequestProcessing,
"base_process_llm_request",
new=_mock_base_process,
):
response = video_proxy_test_client.post(
endpoint,
headers={"Authorization": "Bearer sk-1234"},
json=payload,
)
assert response.status_code == 200, response.text
assert captured_data["video_id"] == encoded_video_id
assert captured_data["custom_llm_provider"] == "vertex_ai"
assert captured_data["model"] == "vertex-ai-sora-2"