mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-11 22:51:28 +00:00
8519d7fc24
* fix: rust ocr tests finally pass * fix: move realtime dir * fix(realtime): normalize azure realtime api_base to host for Foundry endpoints The azure realtime handler appended the realtime path to api_base verbatim, so a Foundry base carrying a project path (.../api/projects/<name>) produced an invalid realtime URL and the websocket handshake hung. Normalize api_base to scheme and host before building the realtime path so both Azure OpenAI and Foundry bases connect Point the e2e realtime azure deployment at the GA gpt-realtime model and stop passing the os.environ refs the realtime path never unwraps, resolving them from the gateway env by name instead. Drop the local docker-compose scaffolding from the tree * test(e2e): add Gateway.list_files and list_fine_tuning_jobs for the discovery suite The discovery endpoints suite calls client.gateway.list_files and list_fine_tuning_jobs, which did not exist on Gateway, so both tests errored with AttributeError before reaching the proxy. Add the two GET wrappers using the existing FileListResponse / FineTuningJobsResponse models * revert(realtime): drop azure realtime api_base host-normalization The azure realtime handshake failure was a config issue, not a litellm bug: the realtime base was set to the Azure AI Foundry project endpoint (.../api/projects/<p>), but the OpenAI-compatible realtime route lives at the resource root. litellm correctly appends the realtime path to whatever base it is given, so pointing the realtime deployment at the resource root is the fix and no core change is needed * fix(ocr): route azure_ai doc-intelligence to its own endpoint at the source get_llm_provider inherits AZURE_AI_API_BASE into api_base for every azure_ai/* OCR model, but Azure Document Intelligence is a separate resource reached via AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT, so doc-intelligence requests went to the wrong host. Stop inheriting the azure_ai base for doc-intelligence models so api_base stays unset and both the rust bridge and the python get_complete_url fall back to the document-intelligence endpoint. This drops the earlier _rust_bridge_api_base reorder, which only covered the rust path and let the env silently override an explicit api_base * refactor(ocr): consolidate azure doc-intelligence detection; keep explicit api_base Extract is_azure_document_intelligence_model as the single source of truth for the azure_ai doc-intelligence sub-route so the check is no longer duplicated across _prepare_ocr_request and _rust_bridge_api_base, and gate the dynamic_api_base suppression on the caller not supplying an api_base so an explicit endpoint is always honoured. Restore xai to the realtime PROVIDERS as a documented disabled entry instead of dropping it silently, and add a regression test pinning doc-intelligence api_base resolution. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Mubashir Osmani <mubashir@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
73 lines
3.4 KiB
Python
73 lines
3.4 KiB
Python
"""Shared pipecat realtime service for the proxy realtime e2e suite.
|
|
|
|
Both pipecat suites drive the proxy through pipecat's GA realtime service. The
|
|
stock ``OpenAIRealtimeLLMService`` sends websocket keepalive pings at its default
|
|
interval, and the LiteLLM proxy does not answer them, so the connection is closed
|
|
with a 1011 before the run completes. ``LiteLLMRealtimeLLMService`` carries the
|
|
three overrides from bot.py needed to talk to the proxy, the keepalive-disabling
|
|
``_connect`` being the load-bearing one for every provider.
|
|
|
|
Importing this module skips the collecting test when pipecat is not installed:
|
|
|
|
uv pip install "pipecat-ai[openai]"
|
|
"""
|
|
|
|
# pipecat is an optional, dynamically typed dependency loaded behind importorskip,
|
|
# so its symbols are Unknown to the type checker; relax those rules for this file.
|
|
# pyright: reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false, reportAttributeAccessIssue=false, reportUntypedBaseClass=false, reportUnknownParameterType=false, reportMissingParameterType=false
|
|
|
|
import pytest
|
|
|
|
pytest.importorskip("pipecat", reason="pipecat-ai not installed")
|
|
|
|
from pipecat.services.openai.realtime.llm import ( # noqa: E402
|
|
OpenAIRealtimeLLMService,
|
|
)
|
|
from websockets.asyncio.client import connect as websocket_connect # noqa: E402
|
|
|
|
|
|
class LiteLLMRealtimeLLMService(OpenAIRealtimeLLMService):
|
|
"""Minimal LiteLLM-aware realtime service for tests.
|
|
|
|
Three overrides carried from bot.py:
|
|
1. _connect - disables websockets keepalive pings (LiteLLM proxy
|
|
does not respond to pings, causing 1011 errors).
|
|
2. _create_response - sends session.update with tools BEFORE history
|
|
items so that Gemini's deferred-setup logic in the
|
|
proxy can include tools in the very first setup
|
|
message it forwards to the backend.
|
|
3. _handle_evt_session_created - immediately marks the session ready
|
|
without waiting for a session.updated echo (the
|
|
LiteLLM Gemini bridge does not send one).
|
|
"""
|
|
|
|
async def _connect(self) -> None:
|
|
if self._websocket:
|
|
return
|
|
try:
|
|
# self.base_url already carries the ?model=<alias> the proxy routes on:
|
|
# the parent __init__ sets self.base_url = f"{base_url}?model={settings.model}"
|
|
# before _connect runs, so passing it through preserves the query param.
|
|
self._websocket = await websocket_connect(
|
|
uri=self.base_url,
|
|
additional_headers={"Authorization": f"Bearer {self.api_key}"},
|
|
ping_interval=None,
|
|
close_timeout=10,
|
|
max_size=None,
|
|
)
|
|
self._receive_task = self.create_task(self._receive_task_handler())
|
|
except Exception as exc:
|
|
await self.push_error(error_msg=f"Error connecting: {exc}", exception=exc)
|
|
self._websocket = None
|
|
|
|
async def _create_response(self) -> None:
|
|
if self._llm_needs_conversation_setup and self._context:
|
|
await self._send_session_update()
|
|
await super()._create_response()
|
|
|
|
async def _handle_evt_session_created(self, evt: object) -> None:
|
|
await self._send_session_update()
|
|
self._api_session_ready = True
|
|
if self._run_llm_when_api_session_ready:
|
|
self._run_llm_when_api_session_ready = False
|
|
await self._create_response()
|