Vertex AI rejects requests containing both search tools (googleSearch,
enterpriseWebSearch, urlContext) and function declarations with error:
'Multiple tools are supported only when they are all search tools.'
When _merge_tools_from_deployment() combines deployment-level search
tools with user-request function tools (e.g. via MCP), the mixed tool
list causes a 400 error. This fix detects the conflict in _map_function()
and drops search tools, keeping function declarations.
Non-search tools like code_execution and computerUse are preserved.
Fixes#23337
* fix(router): discard oldest entry when trimming latency list in lowest_latency strategy
The lowest_latency routing strategy keeps a rolling window of the most
recent latency and time-to-first-token measurements per deployment. When
the window is full, the strategy was discarding the *newest* value
instead of the oldest, because the trim used
`[: max_latency_list_size - 1]` (keeping indices 0..N-2) rather than
`[1:]` (dropping index 0 and keeping indices 1..N-1).
Since new values are appended at the end, the bug meant the most recent
measurement was always dropped once the list reached capacity. The
routing decisions then relied on stale data (including any early-spike
values that never aged out), and timeout penalties written via
`async_log_failure_event` were silently discarded as well.
Fix the slice in all five call sites (sync + async log_success_event for
both latency and time_to_first_token, and async_log_failure_event for
the timeout penalty) and add regression tests covering each path.
* test(router): cover async TTFT trim path in lowest_latency regression tests
Adds test_ttft_list_trimming_discards_oldest_entry_async, an async
counterpart to test_ttft_list_trimming_discards_oldest_entry that drives
async_log_success_event with a ModelResponse and completion_start_time so
the async time_to_first_token trim branch is actually exercised.
Previously no test touched that code path: the sync TTFT test used
log_success_event, and the async latency test passed a plain dict
response_obj without stream/completion_start_time, so TTFT was never
computed and the async trim was unreached. Verified load-bearing by
reverting only the async TTFT slice — the new test fails and all others
pass.
* format
* fix#25506
* address greptile review feedback
* [Test] UI - Models: Add E2E tests for Add Model flow
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)
Added data-testid attributes to UI components to support stable test selectors.
Tests verified passing 3/3 consecutive runs with zero flakiness.
* address greptile review feedback (greploop iteration 1)
Add cleanup helper to delete models created during tests, preventing
stale data accumulation across repeated test runs.
* fix CI: replace data-testid selectors with text/role-based selectors
The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
* remove unnecessary cleanup helper
The database is freshly seeded on every test run via seed.sql,
so per-test cleanup is not needed.
---------
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
* fix: emit input_json_delta for tool args bundled in first streaming chunk
Some providers (xAI, Gemini) include tool_call function arguments in the
same streaming chunk as the function name/id. The AnthropicStreamWrapper
was discarding the trigger chunk entirely when starting a new content
block, which silently dropped the input_json_delta carrying tool
arguments. This caused tool_use blocks to arrive with empty input {}.
Now queue the processed_chunk after content_block_start when it carries
non-empty input_json_delta data. Backward compatible: providers that send
empty arguments in the first chunk (OpenAI-style) are unaffected since
the condition checks for truthy partial_json.
* test: add tests for input_json_delta emission on bundled tool args
Covers the fix for providers (xAI, Gemini) that bundle tool_call
arguments in the same streaming chunk as the function name/id.
Verifies the AnthropicStreamWrapper emits input_json_delta after
content_block_start, and that empty-arg chunks (OpenAI-style) are
unaffected.
* style: apply Black formatting to streaming_iterator.py
* fix: mirror input_json_delta fix to sync __next__ and add sync tests
* test: make no_extra_delta tests assert explicitly instead of passing silently
### Background
The Gemini batchEmbedContents response handler hardcoded `index=0` for
every embedding in the response. Any consumer relying on the OpenAI-format
`index` field to match embeddings back to inputs would silently get wrong
associations.
### Changes
Use `enumerate` in `process_response` so each embedding gets its
positional index instead of 0.
### Test Plan
Added unit test asserting sequential indices and correct vector ordering
for a 3-element batch response.
The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)
Added data-testid attributes to UI components to support stable test selectors.
Tests verified passing 3/3 consecutive runs with zero flakiness.
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
Reviewer flagged that cleanup failures were silently swallowed and
suggested asserting `delete.ok()`. While thinking through the fix, the
actual question turned out to be "does the cleanup matter at all?" —
and the answer is no.
The e2e runner (`run_e2e.sh`) spins up a fresh postgres container per
invocation and tears it down at the end, so every local and CI run
starts with an empty DB. Playwright retries share the same DB but each
attempt creates a new model with a unique `Date.now()` name and only
queries its own model, so orphans from failed attempts never collide
with later attempts or other tests. Nothing else in the suite reads
the all-models table.
Keeping the cleanup would also turn every write test into an implicit
delete test, coupling responsibilities and inflating runtime — which
is probably why `teams.spec.ts` (create a team), `keys.spec.ts`
(update key limits), etc. all leave their entities in place. Matching
that convention, drop the try/finally block and the `createdModelId`
tracking. 12 lines removed, no behavior change.
Covers the full write-path flow for team-scoped models on the Models +
Endpoints page: create via /model/new, click the row to open the detail
view, click Edit Settings, change TPM/RPM, click Save Changes, assert
the new values render back. Cleans up via /model/delete in finally so
reruns stay deterministic.
Requires store_model_in_db: true in the fixture general_settings so the
proxy accepts /model/new and /model/delete. No existing test in the
dashboard e2e suite reads the all-models table or hits the model CRUD
endpoints, so enabling the flag has no cross-test impact.
The suite was superseded by ui/litellm-dashboard/e2e_tests/ on 2026-04-08
and is no longer referenced by CircleCI, docs, or Makefile targets. Drop
the directory wholesale and remove the orphaned e2e:psql npm script that
pointed at its runner.
Updates the expected header text to "Guardrails Settings" to match
GuardrailSettingsView's rendering, and moves the mock guardrails
from team_info.guardrails (legacy top-level path that nothing
reads) to team_info.metadata.guardrails where the component
actually looks. Also tightens the assertion to verify the
individual guardrail names appear, not just the section header.
Previously these were silently dropped with a verbose warning, which
could break observability integrations without surfacing a clear error.
Now raises ValueError with remediation steps (configure server-side
or pass the resolved value) so callers get immediate, actionable feedback.
Converts GuardrailSettingsView from @tremor/react (Badge, Text) to
antd (Tag, plain spans) as part of the Tremor migration. Also
captures the "no new Tremor imports" rule in CLAUDE.md and expands
the existing note in AGENTS.md with the specific antd equivalents
and the yellow→gold gotcha.
Pulls the Global / Team-specific subsection rendering out of
TeamInfo.tsx into a shared GuardrailSettingsView component with
card and inline variants, used on both the team Overview tab
(inside the existing Tremor Card) and the Team Settings tab read
view. The Global subsection header now carries a GlobalOutlined
icon, and since the icon is load-bearing the edit-form chip
coloring is simplified to a single blue instead of green/blue.
The Team Settings tab's read view listed every team field except
guardrails. Adds a Guardrails entry after Status with the same
Global / Team-specific subsections used on the Overview tab, so
the kill switch state and per-section membership are visible
without entering edit mode.
