mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-26 01:12:21 +00:00
581882879d
* feat(ui): add Interactions API support to playground with streaming
Adds /v1beta/interactions as a selectable endpoint in the UI playground.
Uses SSE streaming (stream=true) and parses content.delta events for real-time output.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(interactions): remove forced gemini provider so all providers work via interactions API
Proxy endpoint was hardcoding custom_llm_provider="gemini" before routing,
preventing non-Gemini models from using the litellm_responses bridge.
Also reverts the UI Gemini-only model filter.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(interactions): fix streaming for non-gemini providers via bridge
Two bugs in LiteLLMResponsesInteractionsStreamingIterator:
1. content.delta was emitted without "type":"text" in delta dict, so the
UI type-check always failed and no tokens were displayed
2. First OutputTextDeltaEvent was silently dropped (used to emit content.start
with empty text); fixed by handling ResponsePartAddedEvent for content.start
so text deltas go directly to content.delta
Co-authored-by: Cursor <cursoragent@cursor.com>
* undo unrelated changes
* fix(ui): extract model from top-level field in interactions bridge events
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* test(interactions): remove tautological gemini-provider assertion
The test_no_forced_gemini_provider_in_request_data check only asserted
against dict literals it had just constructed, so it always passed and
did not exercise the create_interaction endpoint. The endpoint
deliberately defaults custom_llm_provider to gemini, so the assertion
was also factually incorrect. Drop the misleading test.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(interactions): use ContentPartAddedEvent and guard interaction.start ordering
- ResponsePartAddedEvent corresponds to reasoning summary parts, not text
content parts. Use ContentPartAddedEvent which is the event emitted before
text output deltas (type response.content_part.added).
- Mirror the OutputTextDeltaEvent ordering guard: if interaction.start has
not been sent yet, emit it first before content.start to honor the
documented event ordering contract.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* test(interactions): cover ContentPartAddedEvent ordering and no-op paths
* fix(tests): treat corrupt VCR cassette payloads as cache miss + use gpt-realtime in OpenAI realtime guardrails test
VCR redis persister was raising UnicodeDecodeError on cached payloads that
fail to UTF-8 decode (e.g. legacy entries written by another version of
the persister), failing tests at fixture setup instead of degrading to a
cache miss. Wrap decode+deserialize in a try/except so corrupt cache
entries are treated as CassetteNotFoundError, surfacing the failure via
the existing _record_cache_failure / VCRCassetteCacheWarning path.
OpenAI shut down gpt-4o-realtime-preview-2024-12-17 (and the entire
gpt-4o-realtime-preview family) on 2026-05-07. The live realtime
guardrails integration test now fails with model_not_found instead of
receiving session.created. Point OPENAI_REALTIME_URL at the current GA
model gpt-realtime, and relax the assertion in
test_text_message_blocked_by_guardrail_no_ai_response to also accept the
model's refusal-to-repeat the block message (gpt-realtime declines
verbatim-repeat instructions, which is still a safe outcome since the
original user message was blocked before reaching OpenAI). The
BLOCKED_PHRASE leak check is preserved as a hard invariant.
* fix(tests): migrate realtime + nvidia_nim rerank tests off shut-down upstream models
OpenAI shut down the entire gpt-4o-realtime-preview family (including the
undated alias) on 2026-05-07. The live realtime tests still connected
with that dead alias and failed with messages_received=1 (an error event
'The model gpt-4o-realtime-preview does not exist' instead of
session.created). Point the live OpenAI realtime tests at gpt-realtime,
the current GA realtime model:
- test_openai_realtime_simple.py: get_model() -> gpt-realtime
- test_openai_realtime.py: test_openai_realtime_direct_call_no_intent and
test_openai_realtime_direct_call_with_intent -> openai/gpt-realtime
Mocked unit tests (test_realtime_query_params_construction,
test_realtime_query_params_use_normalized_model_name) are left as-is:
they never hit the network and assert string plumbing only.
NVIDIA reached end-of-life for the hosted
nvidia/llama-3.2-nv-rerankqa-1b-v2 rerank API on 2026-05-18 with no
published replacement, so the live BaseLLMRerankTest.test_basic_rerank
for nvidia_nim now returns HTTP 410 ('Gone'). NVIDIA's hosted catalog
rotates on a schedule, so swapping in another live model would only
defer the failure. Override test_basic_rerank in TestNvidiaNim to mock
the sync/async HTTP transport (same pattern as
test_nvidia_nim_rerank_ranking_endpoint in this file) and inject a fake
NVIDIA_NIM_API_KEY via monkeypatch. The request/response transformation
and cost calculation stay covered offline.
* test(callbacks): harden flaky proxy callback-leak detector
The proxy callback-leak detector (test_check_num_callbacks_on_lowest_latency)
was failing on this PR with 'abs(85 - 95) <= 4' — a bounded one-time
registration jump caused by switching to latency-based-routing
(+LowestLatencyLoggingHandler, +SlackAlerting). The count then plateaus
under load, so this is pollution from the test's own config update, not a
leak.
Replace the brittle two-sample diff threshold with a sampler that settles
past the deliberate config switch and only flags sustained monotonic
per-type growth, with a terminal-burst confirmation pass for leaks that
would otherwise escape the >=2-interval guard. Normalizes instance
addresses so identical callbacks at different memory locations collapse,
and names the leaking type on failure.
* fix(interactions): preserve first text token when both start events are missing
When OutputTextDeltaEvent arrived before any ResponseCreatedEvent or
ContentPartAddedEvent, the double-fallback path emitted interaction.start
and silently dropped the first delta's text — the second delta's
content.start carried only that chunk's delta, and the first token never
made it to any content.delta event consumed by the UI.
Queue a content.start that carries the first delta's text alongside the
interaction.start emission, and drain pending events before pulling the
next upstream chunk.
* chore(ui): remove unused InteractionOutput/InteractionResponse interfaces
Co-authored-by: Yassin Kortam <yassin@berri.ai>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| e2e_tests | ||
| public | ||
| scripts | ||
| src | ||
| tests | ||
| .env.development | ||
| .env.production | ||
| .eslintrc.json | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc | ||
| build_release_ui.sh | ||
| build_ui.sh | ||
| build_ui_custom_path.sh | ||
| knip.json | ||
| next.config.mjs | ||
| package-lock.json | ||
| package.json | ||
| postcss.config.js | ||
| README.md | ||
| tailwind.config.js | ||
| tailwind.config.ts | ||
| tsconfig.json | ||
| tsconfig.tsbuildinfo | ||
| ui_colors.json | ||
| vitest.config.ts | ||
This is a Next.js project bootstrapped with create-next-app.
Getting Started
First, run the development server:
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.
This project uses next/font to automatically optimize and load Inter, a custom Google Font.
Learn More
To learn more about Next.js, take a look at the following resources:
- Next.js Documentation - learn about Next.js features and API.
- Learn Next.js - an interactive Next.js tutorial.
You can check out the Next.js GitHub repository - your feedback and contributions are welcome!
Deploy on Vercel
The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.
Check out our Next.js deployment documentation for more details.