mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
ef5d05f137
* fix(realtime): stop sending a second Gemini Live setup on follow-up session.update Gemini Live (BidiGenerateContent) accepts setup as the first-and-only client message; a second setup closes the socket with 1007 Request contains an invalid argument. The AI Studio Gemini path forwarded every client session.update after the first as a follow-up setup, and GA clients (pipecat) send several while configuring the session, so the second one tore the session down before the first turn. Callers saw silence after the first response, exponential per-turn latency from reconnect/retry churn, and intermittent 1011 errors. Drop subsequent session.updates instead of resending setup, matching what the Vertex subclass already does. Tools and instructions must ride on the first session.update before any conversation content. Adds regression tests covering the plain follow-up, a follow-up that adds tools (the case the previous identical-only dedup still forwarded), and the guardrail create_response=False warning path. * fix(realtime): retry the backend open handshake instead of failing with 1011 The upstream Live API open handshake (e.g. Gemini Live) intermittently hangs; waiting longer never recovers a hung attempt, but a fresh attempt almost always connects in ~1s. The proxy opened the backend websocket once with the default open_timeout and no retry, so a single slow handshake surfaced to the caller as a fatal 1011 internal error and dropped the call. Bound each open attempt with a short open_timeout and retry; a bounded attempt that already timed out spaces out the next try, so no backoff is needed. Deterministic handshake-status rejections (auth/4xx) are not retried, and the retry only ever wraps the open, never a live session. Adds tests for retry-then-succeed, raise-after-max-attempts, and no-retry-on-auth-failure. * fix(realtime): close guardrail bypass + surface handshake status; drop obsolete tests Three review fixes on the Gemini Live realtime path. Transcription-guardrail bypass: Gemini Live rejects a second setup (1007), so once the initial setup is sent the guardrail's automaticActivityDetection.disabled=true can no longer be delivered as a follow-up session.update. With that follow-up now dropped, the model's auto-response stayed enabled and a realtime_input_transcription guardrail was bypassed (the model answered before the proxy could gate the turn). Fold the disable into the one-and-only setup instead: the handler injects it into the auto-sent setup (gemini_live_defer_setup false) and _send_to_backend injects it into the deferred first setup. OpenAI sessions accept follow-up updates and are left untouched. Backend handshake status: the open-retry treated only InvalidStatusCode as deterministic; websockets>=15 raises InvalidStatus for a rejected client handshake, so a 401/403 fell into the broad WebSocketException branch and was retried before the caller closed the client with 1011 instead of the upstream status. Treat both as non-retryable. Obsolete tests: the four tests asserting a follow-up session.update is merged and re-sent as a second setup asserted behavior that crashes Gemini Live with 1007 (verified directly against the API). Removed; the drop is covered by new regression tests. * style(realtime): reformat changed files to ruff line-length 120 Post-merge with litellm_internal_staging, which unified ruff format width to 120 (#31518). The realtime change set was formatted at 88, so the changed lines tripped the whole-file ruff format check. Reformat with ruff 0.15.3 at the repo's 120 width; no logic changes. |
||
|---|---|---|
| .. | ||
| files | ||
| image_edit | ||
| realtime | ||
| videos | ||
| __init__.py | ||
| test_cost_calculator.py | ||
| test_gemini_client_setup.py | ||
| test_gemini_common_utils.py | ||
| test_gemini_image_generation_transformation.py | ||
| test_gemini_tts.py | ||