scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 84

Merge pull request #38848 from BerriAI/litellm_e2e-rc-1-99-0-fixture-backports

Browse source 
fix(e2e): backport the vertex realtime and vision image fixture fixes to rc/1.99.0
This commit is contained in:
yuneng-jiang 2026-08-29 16:57:15 -07:00 • committed by GitHub
commit 3d14b70ec0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 20 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

BIN
tests/e2e/llm_translation/fixtures/cat.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 30 KiB

2
tests/e2e/llm_translation/realtime/REALTIME_COVERAGE_MATRIX.md
View file

@ -42,7 +42,7 @@ at call time. The provider table below is the source of truth; edit `PROVIDERS`
| openai | `openai-realtime` | `openai/gpt-realtime-2` |
| azure | `azure-realtime` | `azure/gpt-realtime-2` (GA protocol) |
| gemini | `gemini-realtime` | `gemini/gemini-3.1-flash-live-preview` |
| vertex_ai | `vertex-realtime` | `vertex_ai/gemini-live-2.5-flash-preview-native-audio-09-2025` |
| vertex_ai | `vertex-realtime` | `vertex_ai/gemini-live-2.5-flash-native-audio` |
Bedrock and xai (`xai/grok-4-1-fast-non-reasoning`) are supported by the proxy but
kept commented out in `PROVIDERS` until they pass end-to-end here; re-enable them by

2
tests/e2e/llm_translation/realtime/realtime_client.py
View file

@ -78,7 +78,7 @@ PROVIDERS = (
"vertex_ai",
"vertex-realtime",
LiteLLMParamsBody(
model="vertex_ai/gemini-live-2.5-flash-preview-native-audio-09-2025",
model="vertex_ai/gemini-live-2.5-flash-native-audio",
vertex_location="us-central1",
vertex_credentials="os.environ/VERTEXAI_CREDENTIALS",
),

22
tests/e2e/llm_translation/test_chat_completions_regression_e2e.py
View file

@ -16,7 +16,10 @@ via /model/new (Cohere, Gemini, hosted_vllm), each deleted on teardown.
from __future__ import annotations
import base64
import os
from pathlib import Path
from typing import Final
import pytest
from pydantic import BaseModel
@ -79,18 +82,29 @@ def _streamed_tool_call(events: list[str]) -> tuple[str, str]:
return name, arguments
CAT_IMAGE_URL = "https://upload.wikimedia.org/wikipedia/commons/3/3a/Cat03.jpg"
_FIXTURES_DIR: Final = Path(__file__).parent / "fixtures"
CAT_IMAGE: Final = _FIXTURES_DIR / "cat.jpg"
OPENAI_VISION_BACKEND = "openai/gpt-4o"
# OpenAI caches a shared prompt prefix once it exceeds ~1024 tokens; this is well
# past that, so a repeat call reports cached prompt tokens.
def _cat_image_data_url() -> str:
"""The vision image as a data URL, read from a fixture we own.
An https URL would make every vision run depend on a third-party host staying
up and unthrottled, and a 429 from that host reads as a gateway failure. It also
changes what is under test per provider: litellm downloads the image itself for
bedrock, while openai is handed the link and fetches it from its own servers. A
data URL removes the host and puts both providers on the same bytes."""
return "data:image/jpeg;base64," + base64.b64encode(CAT_IMAGE.read_bytes()).decode()
def _vision_messages() -> list[ChatMessage]:
return [
ChatMessage(
role="user",
content=[
TextContentPart(text="What animal is in this image? Answer in one word."),
ImageContentPart(image_url=ImageUrl(url=CAT_IMAGE_URL)),
ImageContentPart(image_url=ImageUrl(url=_cat_image_data_url())),
],
)
]