scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-04 02:31:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 71
litellm/tests/test_litellm/llms/openai/realtime
History
Exact
Sameer Kankute b8635bbc7a
feat(realtime): OpenAI Realtime GA support and beta compatibility (#27110) 
* feat(realtime): OpenAI Realtime GA support and beta compatibility

- Normalize beta-style session.update to GA for upstream OpenAI; optional GA→beta
  event translation when client sends OpenAI-Beta: realtime=v1
- Default upstream WebSocket without OpenAI-Beta; forward header when client opts in
- Extend OpenAI realtime types for GA event names and conversation item shapes
- Relax LiteLLMRealtimeStreamLoggingObject.results to List[Any] for GA events
- Update proxy client_secrets fallback to omit beta header; dashboard RealtimePlayground
- Add unit tests for remap, translation, and beta header helper

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix results

* fix greptile

* Fix mypy issues

* Remove unused class constants _GA_TEXT_DELTA_TYPES and _GA_AUDIO_DELTA_TYPES

These frozensets were defined as class-level constants in realtime_streaming.py
but never referenced anywhere in the codebase. Removing dead code.

Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>

* fix(realtime): use GA-shaped session.update in guardrail injections

The guardrail VAD injection code sent a beta-style session.update with a
flat turn_detection field:

  {"session": {"turn_detection": {"create_response": false}}}

When the upstream OpenAI backend operates in GA mode (no OpenAI-Beta
header forwarded), it requires the nested GA shape:

  {"session": {"type": "realtime", "audio": {"input": {"turn_detection": {"create_response": false}}}}}

The _remap_beta_session_to_ga helper was only applied to client-
originated session.update messages in client_ack_messages. Internally-
generated session.updates (sent via _send_to_backend) in two paths:
  - _handle_raw_backend_message (raw/no provider_config path, line 518)
  - backend_to_client_send_messages provider_config path (line 481)
bypassed the remap, so GA upstreams ignored or rejected them, breaking
audio transcription guardrails for all non-beta clients.

Fix: add _make_disable_auto_response_message() helper that always emits
the correct GA-shaped session.update, and replace both injection sites
with it.

Update existing tests to assert the GA nested shape instead of the old
flat beta shape, and add a new unit test for the helper itself.

Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>

* Log realtime session type

* Fix beta realtime session payloads

* Fix realtime audio format remapping edge case

* Fix Azure realtime beta session shape

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-05-05 16:49:20 -07:00
..
README.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
test_openai_realtime_handler.py feat(realtime): OpenAI Realtime GA support and beta compatibility (#27110) 2026-05-05 16:49:20 -07:00

README.md

OpenAI Realtime Handler Tests

Important Context: additional_headers vs extra_headers

Background

There was confusion about the correct parameter name for passing headers to websockets.connect(). This README documents the resolution for future maintainers.

Timeline of Changes

  1. Dec 5, 2025 - Changed extra_headers → additional_headers (commit 8db7f1b8e4)
  2. Dec 18, 2025 - Changed extra_headers → additional_headers again (PR #17950, commit 9f88d61d10)
  3. Jan 15, 2026 - Upgraded websockets from 13.1.0 → 15.0.1 (commit a3cf178e24, Issue #19089)

The Issue & Resolution

The websockets library changed its API between versions:

  • websockets < 14.0: Used extra_headers parameter ✅
  • websockets >= 14.0: Uses additional_headers parameter ✅

LiteLLM uses websockets 15.0.1 (per uv.lock), which requires additional_headers.

Verification

You can verify the correct parameter name:

uv run python -c "import websockets; import inspect; print(inspect.signature(websockets.connect))"

This shows: additional_headers: 'HeadersLike | None' = None for websockets 15.0.1.

Current Implementation (Correct)

# ✅ Correct for websockets 15.0.1+
await websockets.connect(url, additional_headers={
    "Authorization": f"Bearer {api_key}",
    "OpenAI-Beta": "realtime=v1"
})

Impact

This is NOT just a test fix - this was a critical bug that affected all realtime APIs:

  • OpenAI realtime
  • Azure realtime
  • xAI realtime
  • Any pass-through realtime connections

Using extra_headers with websockets 15.0.1 resulted in:

TypeError: connect() got an unexpected keyword argument 'extra_headers'

For Future Maintainers

If you see test failures related to header parameters:

  1. Check installed websockets version:

    uv run python -c "import websockets; print(websockets.__version__)"

  2. Check uv.lock for the pinned version

  3. Verify the correct parameter:

    • websockets >= 14.0: use additional_headers
    • websockets < 14.0: use extra_headers

  4. Ensure consistency across all files:

    • litellm/llms/openai/realtime/handler.py
    • litellm/llms/azure/realtime/handler.py
    • litellm/llms/custom_httpx/llm_http_handler.py
    • litellm/realtime_api/main.py
    • litellm/proxy/pass_through_endpoints/pass_through_endpoints.py

Current Status (Feb 2026):

  • ✅ websockets version: 15.0.1
  • ✅ Correct parameter: additional_headers
  • ✅ All handlers updated and working