Gap in this PR's own fix, found by auditing the other call paths that reach
_record_input_audio_sample_rate.
The method read the rate only from the GA shape (audio.input.format.rate), on
the stated reasoning that "the beta shape's input_audio_format is a bare codec
name with no rate". The bare codec name does carry a rate. LiteLLM's own type
stub says so:
"The format of input audio. Options are pcm16, g711_ulaw, or g711_alaw. For
pcm16, input audio must be 16-bit PCM at a 24kHz sample rate, single channel
(mono), and little-endian byte order."
OpenAIRealtimeSession.input_audio_format, litellm/types/llms/openai.py:1898
And RealTimeStreaming._remap_beta_session_to_ga already expands it to exactly
that, mapping "pcm16" to {"type": "audio/pcm", "rate": 24000} via
_AUDIO_FORMAT_MAP (realtime_streaming.py:219). That remap runs on every
client session.update when the backend is in GA mode
(realtime_streaming.py:1501), before the payload reaches this config.
So the declared rate for one client payload depended on the OpenAI-Beta header,
which says nothing about sample rates:
no header -> remap runs -> audio/pcm;rate=24000
header set -> remap skipped -> audio/pcm;rate=16000
Same bytes, two labels, and the server resamples against whichever it gets.
The rate is now also read from the flat beta codec name, so both routes agree
at 24000 for pcm16. A rate stated outright in the GA shape still wins; the
name-implied one is only the fallback. Only pcm16 is mapped, because
get_audio_mime_type labels every append as pcm16 regardless, so a rate lifted
from a g711 name would describe bytes with a codec they are not in.
The existing coverage could not catch this: it built the GA dict by hand and
fed it straight to the config, sharing the same assumption as the code under
test. The new test drives the real _remap_beta_session_to_ga instead and
asserts both routes land on the same rate. Verified against the pre-fix source:
assert 'audio/pcm;rate=24000' == 'audio/pcm;rate=16000'
3 tests added, the two new beta ones verified to fail before this change. The
beta case is removed from the "keeps the native default" parametrize list,
where it asserted the 16kHz the remapped path never produces, and replaced
with g711_ulaw.
tests/test_litellm/llms/{gemini,vertex_ai}/realtime plus
tests/test_litellm/litellm_core_utils/test_realtime_streaming.py: 224 passed.
33664d6e0a