* test(e2e): cover customer chat/messages cost + streaming paths
Fills five uncovered P0 registry cells matching the customer's confirmed stack
(OpenAI SDK, Bedrock, /v1/messages) and their per-request cost dependency:
- /v1/messages logs cost that matches the x-litellm-response-cost header (LIT-4076)
- OpenAI /chat/completions streams real content, and a non-streamed call is costed
- Bedrock Converse /chat/completions returns real content non-streamed and streamed
The streaming checks aggregate delta content and parse every chunk as JSON, so a
clean-but-empty stream or a truncated chunk fails instead of passing on a bare 200.
* test(e2e): add tool-use coverage for openai, bedrock converse, anthropic responses
Function-calling regression guards on the paths the customer's agentic SDK usage
exercises: OpenAI and Bedrock Converse /chat/completions, and Anthropic
/v1/responses. The model is forced to call a weather tool and the test asserts the
returned tool call names the function and carries JSON-parseable arguments with the
expected field, so a dropped tool_call or malformed argument JSON fails instead of
passing on a bare 200. Adds a minimal tool_calls field to the response OutMessage.
* test(e2e): cover bedrock converse responses + thinking
Adds llm.responses.bedrock_converse.basic/tool_use and
llm.chat_completions.bedrock_converse.thinking. The thinking test enables extended
thinking and requires reasoning_content plus a real answer, so a path that drops
the reasoning block fails rather than passing.
* test(e2e): cover bedrock embeddings + openai structured output and reasoning
Bedrock Titan embeddings return a real vector; OpenAI structured output must yield
schema-conforming JSON with the correct extracted values (age==42, not just valid
JSON); an OpenAI reasoning call must report reasoning tokens, so a non-reasoning
fallback fails. Adds response_format to ChatBody and reasoning-token details to Usage.
* test(e2e): cover vision + streaming tool calls on openai and bedrock converse
Vision on both providers must describe the image (not just 200); the streamed
OpenAI tool call is reassembled from its fragments and its argument JSON parsed, so
a stream that never completes the call or splits its JSON fails. Extends ChatMessage
content to a typed text/image union.
* test(e2e): cover openai prompt caching hit on repeated large prefix
A repeated large-prefix prompt must report cached prompt tokens on the second call,
so a cache regression that stops reusing the prefix (and silently re-bills full
input) fails here.
* test(e2e): cover openai audio speech + bedrock rerank and image generation
Marks the OpenAI TTS cell and adds Bedrock Titan rerank (top_n honored, scored) and
Bedrock Titan image generation (returns b64/url), the customer's non-chat AWS
surfaces.
* test(e2e): cover end-user (customer) create persistence
mgmt.end_user.new.happy_path: create an end-user via /customer/new and confirm
/customer/info reports it, the end-user-identity surface the customer relies on for
per-customer controls. Adds customer models + management-client methods.
* test(e2e): enforce key model allow-list on the passthrough route
other.auth.passthrough.model_allowlist_enforced: a key scoped to gemini must be
denied a claude call through the anthropic passthrough route (403), so custom-auth
scoping is not bypassable by going through passthrough instead of /chat/completions.
* test(e2e): address Greptile - assert stream data events, correlate messages spend by key
- streaming: assert len(stream_events) > 1 instead of chunks > 1, since chunks
counts the terminal data: [DONE] marker and would pass a single content event
- messages cost: correlate the spend row by the unique scoped key rather than the
Anthropic response id, which need not equal the proxy spend-log request_id
The UI theme and logging-callback read endpoints reported only stored
config while the features resolve their values from the process
environment, so a gateway configured purely through env vars showed
blank settings pages even though branding rendered and callbacks fired.
/get/ui_theme_settings read only litellm_settings.ui_theme_config;
logo_url and favicon_url now fall back to UI_LOGO_PATH and
LITELLM_FAVICON_URL when the stored config leaves them blank.
process_callback (the logging-callbacks block of /get/config/callbacks)
reported every callback env var as unset unless it lived in the config
environment_variables overlay; it now falls back to os.getenv, matching
the slack block. Secret values stay redacted for non-admins via the
existing callback role gate.
Stored values keep winning over the environment, so the UI-driven flow
is unchanged.
Resolves LIT-4667
The Cache Settings page read only the database row, so a response cache
pointed at Redis purely through REDIS_* env vars showed a blank page
while the cache worked. It also masked credentials on read with a
partial-reveal string and re-persisted whatever the form submitted, so
an admin who edited an unrelated field and pressed Save wrote the mask
string over the real Redis password, breaking auth.
GET /cache/settings now overlays the same REDIS_* kwargs the runtime
resolves from when the stored config leaves a field unset, and redacts
credentials with a fixed marker. POST /cache/settings restores the
stored secret behind any credential echoed back as the marker or omitted,
and drops an env-sourced marker rather than persisting it; the response
no longer echoes plaintext credentials. The connection test resolves a
redacted credential back to the stored value the same way. The dashboard
never prefills a credential and drops the marker from the save payload,
mirroring the Coordination Redis tab.
Resolves LIT-4315
Delete Key moved into the key info page's overflow dropdown (#34116) and the
credentials table's row actions moved into a shared DataTable overflow menu, so
both specs were clicking a button that no longer exists. Point them at the menu
items instead.
Add a CredentialsPanel unit test asserting the update payload drops the masked
api key and keeps the edited api base, so that guard is not held up solely by an
e2e a table migration can silently disarm.
The spend-log metadata schema gained a compression_savings key, so the
gcs pubsub v1 payload now carries it. The golden fixture was never
updated, and the comparator flags any key present in the payload but
absent from the fixture, so test_async_gcs_pub_sub_v1 failed on every
run. Pin the key as null rather than adding it to ignored_keys; the
value is deterministic on this path, so ignoring it would leave the
assertion blind to the field entirely.
* fix(e2e): reference client.proxy in mid-conversation native providers test
EndpointsClient exposes the shared ProxyClient as .proxy and has never had a
.gateway attribute, so these two calls raised AttributeError at runtime and
failed the tests/e2e basedpyright zero-error gate for any PR touching e2e
files. Introduced in 23b5b7d199.
* test(e2e): cover 12 non-core LLM coverage registry cells
Raises Non-Core LLMs registry coverage from 24/50 to 36/50 (overall 51.9%
to 54.8%). Four cells were already asserted by existing tests and only
gain their covers marker (openai embeddings, openai image generation,
openai TTS, cohere rerank); one is dual-marked onto the existing
spend-tracking embeddings test rather than duplicated.
New tests: bedrock and vertex embeddings, streaming TTS (asserts chunked
transfer encoding so a buffered body cannot pass), audio transcriptions
via the realtime suite's wav fixture, moderations flag/pass pair, and
files list/retrieve in the batches suite.
Harness: e2e_http.upload generalized to any form model with a
file_content_type override (batches path unchanged), new stream_binary
primitive + BinaryStream for binary chunked responses, transcribe and
moderations client methods, file retrieve/list client methods.
* fix(e2e): close streamed TTS response on error paths and surface the error body
With stream=True a non-2xx response returned with the body unread, keeping
the socket checked out until garbage collection; the sibling
_streaming_outcome already consumes resp.text on error. The response now
closes on every path and BinaryStream carries a bounded error_body so a
failed stream call is triageable.
* test(e2e): assert streamed TTS response carries no content-length
httpbin.org is an external dependency prone to transient 503s (caused the
stage failure); its echo-body assertion also doesn't exercise a real LLM
provider. Point the custom pass-through endpoint at the real Anthropic
Messages API instead. Anthropic doesn't echo headers back, but it gates
real behavior on two of them, which is enough to prove forwarding: a
static x-api-key configured on the endpoint (never supplied by the caller)
must reach upstream or the call 401s, and an invalid x-pass-anthropic-version
sent by the caller must reach upstream with the prefix stripped, which
Anthropic echoes verbatim in its 400 body. Verified live against a local
proxy and the real Anthropic API: valid version returns a real completion,
invalid version returns the exact marker in the 400 body.
* refactor(ui): migrate Tool Policies table onto the shared DataTable
Splits the old components/ToolPolicies.tsx into a data-owning panel, a thin
DataTable consumer and a getToolPoliciesTableColumns module, all under
components/ToolPolicies/. The hand-rolled tremor table, sort dropdowns and
Prev/Next pager are replaced by the shared DataTable in client mode, so
sorting, pagination and filtering now come from TanStack rather than local
state. Search moves to the toolbar global filter and the four facets (input
policy, output policy, team, key) move into a filter drawer; the facets match
exactly instead of by substring, so filtering on "trusted" no longer also
matches "untrusted"
Inline policy editing is preserved. The two policy columns still render a
PolicySelect directly in the row, with the per-row-per-column saving state and
the in-place row update kept in the panel that owns the data
The 15s live-tail poll is removed in favour of the toolbar refresh action, which
takes the auto-refresh out of the write path of the inline edits. The green
live-tail banner goes with it. The panel now reads through React Query with
window-focus and reconnect refetching disabled, so refresh stays manual; that
also removes the effect that previously needed a set-state-in-effect suppression
The metric cards, the Needs Review banner and the detail swap are unchanged.
Review still scrolls to the row when it is on screen, but no longer jumps
across pages, since the paginated order now lives inside the table
Drops the unused userRole prop threaded from the route through the view into
the table, and prunes the suppressions stranded by the file move
* fix(ui): make Tool Policies inline saves safe against concurrent edits and refresh
Two races in the inline policy editing path, both found by review.
Saving state was a single tool name per column, so starting a second row's save
re-enabled the first row while its PATCH was still in flight, and whichever
save finished first cleared the indicator for whichever row was in the slot.
Track the set of tool names currently saving per column instead, so each cell
disables and re-enables on its own request
A list fetch already in flight when a save landed would resolve afterwards and
overwrite the row with its pre-save snapshot, silently reverting a policy the
user had just changed and the server had already accepted. Cancel in-flight
queries before writing the row, which is the documented React Query ordering
for this; the stale response is then discarded and the refresh can be retried
Tightens the test helpers that hid the second bug: policy values are now
compared exactly rather than with toHaveTextContent, which substring-matches
and so let "untrusted" satisfy an assertion for "trusted"
sharp reaches the dashboard only as an optional dependency of next, which
pins it to ^0.34.5. A caret range on a 0.x version cannot resolve past
0.34.x, and every stable next through 16.2.11 still declares that same
range, so there is no transitive path to the 0.35 line. Add an overrides
entry, matching how the other pinned transitives in this package are
already handled.
The dashboard builds with output: "export" and images.unoptimized, so
sharp is never loaded; this keeps the lockfile current rather than
changing runtime behaviour.
The test replaced the whole ./guardrail_info_helpers module with a factory
returning only getGuardrailLogoAndName, so guardrailLogoMap became undefined.
guardrail_garden_data.ts indexes that map at module scope and is reachable
from the panel via guardrail_garden.tsx, so the file failed to collect and
the suite never ran. Spread the real module and override only the stubbed
function.
Also cover the delete flow, which is the only consumer of the stubbed helper
in this component; the mocked table already rendered a delete button that no
test clicked.
Both tables consume the shared DataTable's controlled row-selection API, so they
move together.
Users runs fully server-side (sorting, pagination, filtering) with the page,
sort and filter state lifted to ViewUserDashboard, which now also owns the
detail-view swap that used to live inside the table component. Sort controls are
restricted to the five keys the backend accepts so a header click can no longer
send an invalid sort_by. The hand-rolled checkbox column, select-all and
selectedUsers[] are replaced by controlled rowSelection keyed by user id, and the
per-row icon strip becomes an overflow menu.
Model health checks keep client-side sorting, including the custom status and
timestamp orderings, while pagination moves to the shared footer driven by the
grandparent's page state. Selection is cleared whenever the page changes, since
the rows underneath it are swapped out.
ModelDataTable had no consumers left once HealthCheckComponent stopped using it,
so it is removed along with dead local state it carried.
Mirrors the new test-code-quality.yml step locally so a green pre-commit stays predictive: the sub-second checker fires only when tests/e2e Python files are staged, matching the script's staged-file gating for every other block.
Adds get_external to e2e_http.py for absolute third-party GETs (no proxy base url or auth, same Result classification) and rewires fetch_agent_card through it, dropping the urllib.request escape hatch. Creates tests/code_coverage_tests/check_e2e_no_raw_requests.py, the checker tests/e2e/CLAUDE.md already referenced, and wires it into the code-quality workflow so raw HTTP client imports outside the transport fail CI; pre-existing uses (root conftest liveness probe, claude_code version resolver) are grandfathered and exception-type-only imports stay allowed.
* test(ui): run vitest unit tests in GitHub Actions and fix stale key-info tests
The dashboard's vitest suite only ran on CircleCI; GitHub Actions covered the
UI build, lint and api-types sync but never the unit tests. Add a UI Unit Tests
workflow that runs the suite, sharded across a matrix so the wall-clock is not
bound by a single 4-core runner.
Porting it surfaced 17 pre-existing failures. Adding the block/unblock key
action moved Delete Key and Reset Spend into a "More key actions" dropdown and
introduced a React Query hook; KeyInfoHeader's own test was updated but the two
KeyInfoView test files were not. Reach those actions through the dropdown and
stub the new hook the way the neighbouring hook is already stubbed.
The same refactor had quietly hollowed out assertions that still passed:
"should not show Reset Spend button for regular key owner" queried for a button
role that no longer exists, so it held green regardless of the permission
check. Those now open the menu and assert on the menu item, which fails when
canResetSpend is forced true.
Also add the missing cost-optimization page description; page_utils guards that
every navigable page carries one.
* ci(ui): scope PR runs to changed tests, run the full suite on staging
Running the whole vitest suite on every pull request costs about five minutes,
and none of it is recoverable through parallelism: vitest schedules by file and
create_mcp_server.test.tsx alone accounts for 252s of the 255s total, so shards
and extra cores cannot get under that floor. Measured on this branch, css:false,
pool=threads and isolate=false all landed within noise of the baseline.
Scope pull requests to tests reachable from the diff instead, which takes 11s
here, and keep a full run on pushes to litellm_internal_staging so nothing rots
behind a gap in the module graph. Backend-only pull requests match no test files
and exit zero; --passWithNoTests states that rather than leaning on it being the
current default. The checkout needs full history for --changed to resolve the
base commit.
* test(e2e): add Azure AI Foundry and Anthropic /v1/messages coverage
* test(e2e): add Azure AI Foundry + Anthropic messages coverage for the Rust bridge
* test(e2e): guard against an empty SSE stream in the Azure Foundry tool-use streaming case
JWT auth built UserAPIKeyAuth without user_email even though the resolved
user row and the JWT email claim were both available, so the user_email
label on Prometheus metrics and user_api_key_user_email in
StandardLogging/SpendLogs metadata were always None for JWT traffic.
Plumb user_email through JWTAuthBuilderResult: auth_builder returns the
user row email when set, falling back to the user_email_jwt_field claim
(covers the scope-based proxy-admin path where no user row is loaded).
The JWT branch now stamps it on the proxy-admin return, the standard
valid_token, and the auto-registered virtual key object.
Resolves LIT-4238
Follow-up to the static logo import PR. Types providerLogoMap as
Partial<Record<Providers, string>> so raw string keys and lookups are
compile errors, tightens the resolveLogoSrc passthrough from /_next/ to
/_next/static/ so lookalike backend paths still get root-prefixed, adds
an enum coverage test that locks the exact set of logoless providers,
and makes Logo props a discriminated union so provider and src modes
cannot be mixed and src mode requires a label.
The key-edit form sent budget_limits on every save (the stored windows, or []
when a key has none). The backend treats any budget_limits in a /key/update
request as an admin-only budget change, so a non-admin key owner editing a
non-budget field (models, MCP servers, alias) always hit 403 with "Only proxy
admins, team admins, or org admins can call /key/update".
Only include budget_limits when the user actually changed the budget windows,
mirroring how the same handler already drops an unchanged allowed_routes. The
comparison is on (duration, cap) ignoring the server-owned reset_at and window
order; [] is still sent when the user deletes the last window so clearing keeps
working. No backend or API behavior changes.
* refactor(ui): migrate MCP, callback, guardrail, SSO, and search tool logos to the shared Logo component
Third step of the logo consolidation. Every remaining rogue logo
pattern now renders through Logo: MCP well-known grid and backend
mcp_info.logo_url sites (which previously skipped resolveLogoSrc and
broke under non-root mounts), callback maps in callback_info_helpers
plus the backend-provided variant in settings.tsx, the guardrail map
with the garden dataset now deriving logos from guardrailLogoMap
instead of duplicating them, the SSO map deduped from two verbatim
copies into SSOSettings/constants.ts, search tools' filename guessing
replaced with an explicit static-import map, and the two straggler
sites in EntityUsage and model_info_view.
Static-map path strings become bundled static imports throughout;
backend-provided URLs stay runtime strings resolved via Logo src mode.
MCPLogoSelector still stores stable /ui/assets/logos paths so existing
DB rows keep matching. okta's logo remains an external hotlink pending
a vendored local asset. promptguard.svg drops a mismatched intrinsic
dimension attribute for the Turbopack import parser.
* fix(ui): make resolveLogoSrc idempotent for values already carrying the server root path
Stored mcp_info.logo_url values from sub-path deployments could bake in
the deployment root because the old bare img sites did no resolution.
Prefixing those again produced /litellm/litellm/... and a fallback
avatar. Skip prefixing when the value already starts with the current
normalized root segment; paths whose first segment merely begins with
the root text still get prefixed.
* feat(ui): add react-hook-form + zod form infrastructure
Introduce the shared form layer the dashboard's antd forms will migrate onto,
with no user-visible change yet.
- pin react-hook-form, @hookform/resolvers, and zod (kept on 3.25.76 and
imported via the zod/v4 entrypoint so openai's optional zod ^3 peer still
resolves and npm ci stays clean)
- vendor the base-vega Field family into components/shared/form as forwardRef
components on the repo's cva.config, since base-vega ships no form primitive
and its field source imports class-variance-authority and is React 19 style
- add a FormField bridge that binds a react-hook-form Controller to the Field
layer and wires label, description, and error ids into aria attributes
- add pickDirty, which narrows a submitted body to the top-level keys the user
actually touched so a partial update stops re-sending untouched fields
pickDirty reads dirtiness at the top level because react-hook-form tracks it
per leaf, so an edited array arrives as [true, false] and a cleared list as an
empty array that still carries its default-length dirty markers; the falsy
clear tokens (null, [], {}, 0, false) all survive.
Tests cover the Field primitives, the FormField aria wiring against a live
zod resolver, and pickDirty both as a unit and driven through a real
react-hook-form instance.
* test(ui): lock pickDirty behavior on a pure field-array reorder
react-hook-form compares each array element to its default positionally by
value, so useFieldArray move/swap and a reordered scalar array all mark the
moved indices dirty and pickDirty sends the whole array; a swap of two equal
elements is a value-level no-op and is correctly omitted. Covers the reorder
case a review flagged as untested.
The add_deployment and get_credentials background jobs that keep a multi-pod
deployment in sync with config-in-DB objects (models, credentials, guardrails,
general settings, etc.) polled the database on a hardcoded 30s interval, with
no way to trade convergence latency against DB load.
Expose it as the general_setting proxy_config_reload_interval_seconds (env
PROXY_CONFIG_RELOAD_INTERVAL_SECONDS parsed via get_env_int, default 30),
threaded like the existing proxy_batch_polling_interval knob, and surface it on
the admin general-settings page so it is reachable from the dashboard and
persists to the DB for all pods. Non-positive values are rejected at the UI
(gt=0) and fall back to 30s with a warning on the env/config/DB paths.
