Rotating the Speech-to-Text stream at 240 s no longer ends the active turn:
the turn and its billed seconds continue on the new stream, forced at 280 s.
Bound the request and event queues (64 and 256) so a slow peer applies
backpressure instead of growing memory. Route a model to the Chirp realtime
path from its cost-map entry (mode audio_transcription plus /v1/realtime)
instead of a hardcoded name. Return on every branch of the recv and
transform helpers (CodeQL mixed returns), have the shared protocol helper take
the provider's error class so the Meta tests assert MuseProtocolError again,
and pin google-cloud-speech in the ci group so unit shards import it.
Bridge OpenAI Realtime transcription sessions on vertex_ai/chirp_* models to
Google Speech-to-Text v2 StreamingRecognize over gRPC, so partial and final
transcripts stream back while audio is still being sent. Interim results become
delta events, finals become completed events carrying billed seconds, the gRPC
stream rotates at 240 s under Google's five-minute cap with billed time chained
across rotations, and audio is split into 25 KB requests.
The OpenAI transcription protocol helpers move into a shared module that Meta
Muse now uses too, google-cloud-speech ships behind a new stt-vertex-chirp extra
bundled into the proxy runtime, and the cost map lists /v1/realtime for chirp_3.
The bedrock-realtime extra pinned aws-sdk-bedrock-runtime 0.7.x, whose Config and BedrockRuntimeClient surface is gone in 0.11. The handler now resolves AsyncBedrockRuntimeConfig, builds AsyncBedrockRuntimeClient with the awscrt duplex transport, closes the client when the session ends, and tells an absent SDK apart from an installed but unsupported version. Moves the pin to >=0.10.0,<0.12.0 with the awscrt extra
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
maturin already packages non-gitignored files under the Python source
directory of a mixed project, and the built wheel contains
litellm/rust_bridge/_native.pyi without the explicit entry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The sequential version sent one request at a time, so a Redis outage never
reached the concurrency where the failed-tracking alert body actually grows.
This drives the proxy with locust against one model group of three mock
deployments, two failing at order 1 and one serving at order 2, so every
request spends its retries on the failing pair and lands on the serving
deployment through the order-based fallback. Two phases, a healthy baseline
and a CLIENT PAUSE WRITE window, and every request must succeed in both.
Latency, RSS and CPU are reported as p50/p90/p99 per phase rather than
asserted on: RSS and CPU come from psutil on the proxy's process tree, since
a multi-worker proxy serves /metrics from the prometheus multiprocess
collector and that drops the process collector's series. Thresholds stay open
until weekly runs give real baselines.
Co-Authored-By: Claude Code <noreply@anthropic.com>
The selector picked up two suites that can never pass in this stack, so
editing either one turned the check permanently red: the presidio masking
suite calls pytest.fail without an analyzer and anonymizer that up.sh
never starts, and the pipecat audio suite skips itself at import time
unless the NLTK punkt_tab data is present, which nothing installs.
tests/e2e/coverage_registry/test_collector.py had the same problem for a
different reason. Its nested pytest.main autoloads pytest-retry from the
ci group the workflow installs and dies with "INTERNALERROR: no option
named 'filtered_exceptions'", so the collect-only pass now disables that
plugin. The plugin's entry point is pytest-retry, not retry, so the same
one-word fix lands on mutmut's pytest_add_cli_args, where "-p no:retry"
was disabling nothing.
Two smaller holes in the harness: a canary argument the shell never
expanded used to select nothing and let the gate pass green, and a secret
that cannot be represented in both bash and dotenv was rejected without
naming the key.
4.17 was picked on the belief that dnspython only became a core pymongo
dependency there, which is wrong: pymongo has declared dnspython>=1.16.0,<3.0.0
as a core requirement since well before that, so mongodb+srv:// URIs resolve at
4.9 too. The real floor is 4.9, the release AsyncMongoClient landed in, and 4.8
has no AsyncMongoClient at all.
Verified against live Atlas on 4.9: sync and async search, list_search_indexes,
same top hit and score as 4.17. Resolution is unchanged, pymongo 4.17.0 either
way, so this only widens what an existing environment is allowed to bring.
GHSA-jp53-mhqp-8xcg (fixed in 6.16.0), GHSA-23w6-3w8w-8484 and
GHSA-763m-79hh-57f2 (fixed in 6.16.1) flag pypdf 6.15.0 in uv.lock and
keep osv-scan red alongside the tornado advisories. The proxy-runtime
extra now requires pypdf>=6.16.1 and the lock resolves 6.16.2.
Atlas Vector Search has no HTTP query API, since the Data API and HTTPS
Endpoints are end-of-life, so this provider extends BaseDirectVectorStoreConfig
and runs the $vectorSearch aggregation through pymongo rather than shaping an
httpx request. That is the same seam Valkey uses for RESP.
vector_store_id names the Atlas Search index, matching Valkey, with the
database and collection supplied through litellm_params.
pymongo lives in a new optional `mongodb` extra and is imported lazily, so the
base install still pulls no MongoDB driver. The floor is 4.17 because that is
where dnspython became a core dependency instead of the `srv` extra, and Atlas
issues mongodb+srv:// URIs that will not resolve without it.
Clients are cached per connection rather than opened per search. Measured
against Atlas, a fresh client costs ~890ms versus ~80ms warm, so copying the
Valkey open-and-close-per-call pattern would have added ~810ms to every query.
ProxyExtrasDBManager.spend_logs_is_partitioned() (#38452) silently returns
False when psycopg can't be imported, and psycopg was never added to the
extra_proxy install, so every production image lacks it. Schema
reconciliation then generates the unfiltered primary-key rewrite against a
genuinely partitioned LiteLLM_SpendLogs and Postgres rejects it, exactly the
failure the fix was meant to prevent. Ships psycopg via extra_proxy and logs
a warning when it's still missing instead of failing silently.
RestrictedPython 8.3 extended its protected-name validation to cover
positional-only parameters, so sandboxed source can no longer bind a
local named _getattr_, _getitem_, _write_ or _print_ that takes
precedence over the hooks the custom-code guardrail sandbox installs.
8.4 and 8.5 continue in the same direction with safer_getattr and the
Python 3.15 syntax audit.
The floor moves rather than the lock alone so downstream installs of
litellm[proxy] pick up the same behaviour.
