Commit graph

11 commits

Author SHA1 Message Date
Ishaan Jaffer
e8461b5b97
style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
stuxf
a6c30b30bf
build: migrate packaging, CI, and Docker from Poetry to uv (#25007)
* build: migrate packaging metadata to uv

* ci: move automation and local tooling to uv

* docker: migrate image builds and runtime setup to uv

* docs: update install and deployment guidance for uv

* chore: align auxiliary scripts and tests with uv

* test: harden test_litellm isolation

* fix: keep release and health check images self-contained

* build: pin uv tooling and health check deps

* test: isolate bedrock image request formatting from suite state

* test: cover sandbox executor requirements flow

* ci: fix circleci no-op command steps

* ci: fix circleci publish workflow parsing

* fix: stabilize remaining uv migration CI checks

* ci: increase matrix test timeout headroom

* fix: restore published docker and license coverage

* fix: restore proxy runtime build parity

* fix: restore proxy extras parity and venv migrations

* ci: persist uv path across circleci steps

* fix: keep psycopg binary in default test env

* docker: preserve prisma cache across stages

* test: run local proxy checks through uv python

* build: restore runtime deps moved into ci

* build: refresh uv lock after upstream merge

* fix: restore module import in test_check_migration after merge

The conflict resolution imported only the function but the test body
references check_migration as a module throughout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching

- Move google-generativeai, Pillow, tenacity back to ci group (they are
  lazily imported and bloat the base SDK install needlessly)
- Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant
  in Docker where system Node.js is already installed via apk)
- Remove all nodejs-wheel node replacement and venv npm patching blocks
  from Dockerfiles since the wheel is no longer installed
- Add --no-default-groups to CodSpeed benchmark workflow so the benchmark
  environment matches the old minimal pip install footprint
- Apply standard uv two-phase Docker pattern: copy metadata first, install
  deps (cached layer), then copy source and install project
- Replace CircleCI enterprise no-op with proper uv sync command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate uv.lock after removing nodejs-wheel-binaries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): use cache/restore instead of cache to prevent cache poisoning

The old workflow used actions/cache/restore (read-only). The uv migration
changed it to actions/cache (read-write), which zizmor flags as a cache
poisoning risk. Restore the safer read-only variant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert

The setup-uv action enables caching by default, which zizmor flags as a
cache poisoning risk. Disable it since we already use a read-only
cache/restore step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv cache in publish workflow

Silences zizmor cache-poisoning alert. Publishing workflow runs
infrequently on protected branches so caching adds no real benefit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): remove duplicate verbose_logger mock in test_check_migration

The logger was patched twice — first via mocker.patch() then via
mocker.patch.object(autospec=True). The second call fails because
autospec cannot inspect an already-mocked attribute. Remove the
redundant first patch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): free disk space before Docker build in test-server-root-path

The Dockerfile.non_root build ran out of disk on the CI runner. Remove
Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 11:46:23 -07:00
Ishaan Jaff
da4cf4942f
[Feat] Add xAI /realtime API Support - works with LiveKitSDK (#20381)
* init: _realtime_health_check + routing

* refactor: OpenAIRealtime

* refactor: XAI_API_BASE

* feat: XAIRealtime

* init feat: XAIRealtime

* OpenAIRealtime

* TestXAIRealtime

* test fixes

* test OAI

* TEST xAI, OAI

* clean realtime jobs

* refactor

* test XAI

* docs xAI

* fix xAI

* fix lint errors

* test_async_realtime_url_contains_model

* test fix

* document test changes

* _realtime_health_check

* docs xai realtime

* fix handlers

* add additional_headers

* fix
2026-02-03 19:58:28 -08:00
Cesar Garcia
4ad5de10cb
fix(realtime): disable SSL for ws:// WebSocket connections (#19345)
When using http:// api_base (converted to ws://), the websockets library
throws "ssl argument is incompatible with a ws:// URI". Only pass SSL
context for secure wss:// connections.

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-01-19 10:37:41 -08:00
Lucas Sugi
9f88d61d10
fix: Change extra_headers to additional_headers (#17950) 2025-12-18 08:57:20 +05:30
Alexsander Hamir
ca2a27c377
fix: add missing mock attributes in websocket and realtime tests (#16974)
- Add scope and url attributes to WebSocket mock in test_user_api_key_auth_websocket
- Add shared_realtime_ssl_context initialization in realtime handler test
2025-11-22 10:44:23 -08:00
Alexsander Hamir
eb5031da1e
[Perf] Fix bottlenecks degrading realtime endpoint performance (#16670)
* Cache realtime websocket request body

Move the realtime request payload builder out of the websocket handler and wrap it with an LRU cache so repeated connections reuse the same bytes object. This keeps the JSON formatting cost down while bounding memory usage.

* Optimize realtime websocket caching

Refactored /v1/realtime to use cached helpers for both the JSON body and query params, introduced a reusable request-scope template, and optimized header handling to avoid redundant work.

* Refine realtime websocket header handling

* Reuse websocket scope headers in auth

* Refactor realtime request body helper

Move the realtime request body formatter into proxy common utils so it can be reused across modules. Reuse it in the websocket auth flow to share LRU caching and avoid ad hoc byte builders.

* fix: revert to old pattern

The old pattern was necessary, we can just return the optimized function instead.

* Reuse SSL context for realtime

Create a shared SSLContext for OpenAI realtime websocket dials and pass it into websockets.connect so we stop re-reading verify paths on every session.

* feat: reuse shared TLS context for realtime websockets

- add `SHARED_REALTIME_SSL_CONTEXT` helper so all realtime websocket clients share the same TLS settings
- wire the shared context into OpenAI, Azure, custom HTTPX handlers, and realtime health checks
- update realtime tests to assert that the expected SSL context is passed to `websockets.connect`

This keeps TLS configuration consistent and avoids recreating SSL contexts per connection.

* Reuse HTTP SSL context for realtime

Remove the standalone realtime SSL helper, expose a shared context directly from the HTTP handler, and point all realtime websocket clients and tests to it. Add the websocket header comparison tool.

* Lazy-load shared realtime SSL context

Fix circular imports introduced by eagerly instantiating the shared TLS context. Make the HTTP handler lazily create the context and have realtime clients/tests fetch it on demand, keeping configuration consistent without breaking startup.

* add: unit test for realtime LRU caches

* fix: merge conflict with imports
2025-11-22 10:01:02 -08:00
Ishaan Jaff
73a23a6c78
[Feat] Add Azure AVA TTS integration (#15749)
* add AzureBaseIssueTokenHandler

* add BaseTextToSpeechConfig

* async_text_to_speech_handler

* add AzureAVATextToSpeechConfig

* add get_provider_text_to_speech_config

* add AzureAVATextToSpeechConfig

* fixes for base_llm_http_handler

* fix transform_text_to_speech_request

* test_azure_ava_tts_async

* test_azure_ava_tts_async

* fix TextToSpeechRequestData

* fix transform_text_to_speech_request

* add text_to_speech_handler in LLMHttpHandler

* remove old file

* fix transform_text_to_speech_request

* fix dispatch_text_to_speech

* fix azure TTS

* fix AVA TTS

* fix transform

* fix linting

* ci/cd - use one job for audio testing

* fix tests

* fix llm http handler debugging

* unit tests azure tts

* docs Azure speech

* docs fix

* docs azure AVA

* docs azure AVA

* fix handlers

* test_async_realtime_uses_max_size_parameter
2025-10-20 16:52:23 -07:00
Jugal D. Bhatt
aea0605eed
[LLM Translation] Fix Realtime API endpoint for no intent (#13476)
* fix intent params

* Add responses

* fix unrelated test

* test fix - fireworks API endpoint is down

* test fix fireworks ai is having an active outage

* test_completion_cost_databricks

* dbrx fix test API currently not responding

* Update OpenAI Realtime handler to use the correct endpoint and include all query parameters. Adjusted error messages for missing API base and key. Updated health check URL construction to pass model as a query parameter.

* Enhance OpenAI Realtime handler tests to ensure model parameter inclusion in WebSocket URL. Added new tests to verify correct URL construction with model and additional parameters, preventing 'missing_model' errors. Updated existing tests for consistency.

* Remove debug print statements for API base and key in OpenAIRealtime handler to clean up the code.

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-08-14 16:24:14 -07:00
Jugal D. Bhatt
c66b288bcf
[LLM Translation] fix query params for realtime api intent (#12838)
* fix query params for realtime api intent

* fix my py

* Add typed dict

* remove typed dict

* fix comments

* add test

* add test

* added proxt log revert

* add real time q params
2025-07-25 10:09:53 -07:00
Krish Dholakia
69c9d75f20
fix(prometheus.py): pass custom metadata labels in litellm_total_toke… (#11414)
* fix(prometheus.py): pass custom metadata labels in litellm_total_tokens metric

* fix(handler.py): handle /v1 for openai realtime translation

Closes https://github.com/BerriAI/litellm/pull/11398

* fix(prometheus.py): fix incrementing total tokens metric
2025-06-05 00:15:23 -07:00