The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)
Added data-testid attributes to UI components to support stable test selectors.
Tests verified passing 3/3 consecutive runs with zero flakiness.
Reviewer flagged that cleanup failures were silently swallowed and
suggested asserting `delete.ok()`. While thinking through the fix, the
actual question turned out to be "does the cleanup matter at all?" —
and the answer is no.
The e2e runner (`run_e2e.sh`) spins up a fresh postgres container per
invocation and tears it down at the end, so every local and CI run
starts with an empty DB. Playwright retries share the same DB but each
attempt creates a new model with a unique `Date.now()` name and only
queries its own model, so orphans from failed attempts never collide
with later attempts or other tests. Nothing else in the suite reads
the all-models table.
Keeping the cleanup would also turn every write test into an implicit
delete test, coupling responsibilities and inflating runtime — which
is probably why `teams.spec.ts` (create a team), `keys.spec.ts`
(update key limits), etc. all leave their entities in place. Matching
that convention, drop the try/finally block and the `createdModelId`
tracking. 12 lines removed, no behavior change.
Covers the full write-path flow for team-scoped models on the Models +
Endpoints page: create via /model/new, click the row to open the detail
view, click Edit Settings, change TPM/RPM, click Save Changes, assert
the new values render back. Cleans up via /model/delete in finally so
reruns stay deterministic.
Requires store_model_in_db: true in the fixture general_settings so the
proxy accepts /model/new and /model/delete. No existing test in the
dashboard e2e suite reads the all-models table or hits the model CRUD
endpoints, so enabling the flag has no cross-test impact.
The suite was superseded by ui/litellm-dashboard/e2e_tests/ on 2026-04-08
and is no longer referenced by CircleCI, docs, or Makefile targets. Drop
the directory wholesale and remove the orphaned e2e:psql npm script that
pointed at its runner.
Updates the expected header text to "Guardrails Settings" to match
GuardrailSettingsView's rendering, and moves the mock guardrails
from team_info.guardrails (legacy top-level path that nothing
reads) to team_info.metadata.guardrails where the component
actually looks. Also tightens the assertion to verify the
individual guardrail names appear, not just the section header.
Converts GuardrailSettingsView from @tremor/react (Badge, Text) to
antd (Tag, plain spans) as part of the Tremor migration. Also
captures the "no new Tremor imports" rule in CLAUDE.md and expands
the existing note in AGENTS.md with the specific antd equivalents
and the yellow→gold gotcha.
Pulls the Global / Team-specific subsection rendering out of
TeamInfo.tsx into a shared GuardrailSettingsView component with
card and inline variants, used on both the team Overview tab
(inside the existing Tremor Card) and the Team Settings tab read
view. The Global subsection header now carries a GlobalOutlined
icon, and since the icon is load-bearing the edit-form chip
coloring is simplified to a single blue instead of green/blue.
The Team Settings tab's read view listed every team field except
guardrails. Adds a Guardrails entry after Status with the same
Global / Team-specific subsections used on the Overview tab, so
the kill switch state and per-section membership are visible
without entering edit mode.
Replaces the flat guardrails list with two subsections under the
Guardrails card, so the global vs. team-specific distinction is
carried by the section headers instead of per-badge markers. The
kill-switch state now renders in place of the Global subsection as
"Bypassed for this team", and the separate "Disable Global
Guardrails" field with its confusing "Disabled - Global guardrails
active" badge is removed.
Addresses a11y feedback — global vs. non-global guardrails were
distinguished only by color (green vs. blue). Adds GlobalOutlined
next to global guardrails in (1) the selected-chip tagRender, (2)
the dropdown OptGroup label, and (3) the team info read view badge.
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
The new effectiveGuardrails computation derived its element type from
`info.metadata?.guardrails`, which is implicitly typed `any[]`, so the
downstream `.map((name) => ...)` callback inferred `name` as implicit
`any` and broke the production typecheck.
Renames the new per-guardrail opt-out field from `disabled_global_guardrails`
to `opted_out_global_guardrails` to eliminate the one-character collision with
the legacy `disable_global_guardrails` boolean kill switch. Adds a type guard
on the new gate so a misnamed bool can't crash the guardrail check. Filters
duplicates out of the team-edit guardrail display for legacy teams that have a
global name persisted in `metadata.guardrails` from before this PR. Drops the
unused `isGuardrailsLoading` and `guardrailsError` destructures left in
AddModelForm after the hook refactor.
Adds Python tests for the new gate behavior (root, litellm_metadata, metadata,
non-matching name, empty list, malformed bool value, opt-in coexistence) and
extends useGuardrails.test.ts to exercise the global / optional partition
logic that the rebuilt hook performs in its `select` transform.
Wires the legacy kill switch and the new opt-out list together in the team
edit form so they can never fall out of sync:
- Toggling the kill switch reactively updates the Guardrails Select via
`onValuesChange` — switch on strips all globals from the selection, switch
off re-adds them. Existing opt-in extras are preserved either way.
- When the switch is on, global options in the Select are individually
disabled (greyed out) so the user can still manage opt-in guardrails but
cannot accidentally re-enable a global the kill switch is bypassing.
- The save handler writes both fields together: `disable_global_guardrails`
reflects the switch, and `opted_out_global_guardrails` is set to either
every global (when the switch is on) or the user's explicit opt-outs.
- `effectiveGuardrails` for the form's initialValues honors the kill switch
on legacy teams so the form opens in a state that matches what the runtime
gate is actually doing — fixes the visual lie where chips appeared active
while the switch was bypassing them.
The backend gate already reads the list as the primary path with the bool
as a fallback, so untouched legacy teams keep working until they get edited,
at which point they migrate naturally.
The team edit form computed its `guardrails` initialValues from
`globalGuardrailNames`, which is empty until `useGuardrails` resolves.
Because Ant Design's `<Form initialValues>` is read once on mount, a
user who clicked Edit before the query resolved would see globals
missing from the Select and could save that stale state, silently
opting the team out of every global guardrail.
Gate the form on `!isGuardrailsLoading` so initialValues always sees
the resolved set.
Replace the team info page's tags-mode guardrails Select with a grouped
multi-select that splits guardrails into "Global" (default_on=true) and
"Other" sections. On submit, derive both metadata fields from the single
selection array:
- metadata.guardrails: non-global opt-ins (additive)
- metadata.disabled_global_guardrails: globals the team has opted out of
The legacy disable_global_guardrails kill-switch toggle stays alongside
the multiselect and remains semantically distinct: it kills all global
guardrails including any added in the future, while the new list lets
new globals auto-apply unless explicitly excluded.
Other changes:
- Switch from a manual useEffect/useState guardrails fetch to the
consolidated useGuardrails hook
- Replace stale "Select existing guardrails or enter new ones" help
text with a single tooltip on each section's info icon
- Render selected chips with green/blue color coding via tagRender so
global vs non-global is visible without opening the dropdown
- Drop the redundant [Global] tag from inside the OptGroup options
(the group label already conveys it)
- Update the read-only display to show the effective set (globals not
opted out + additive opt-ins) with [Global] suffix on globals
Extend useGuardrails to return the full guardrail objects plus derived
globalGuardrailNames / optionalGuardrailNames sets via React Query's
select option, instead of just an array of names. Update its existing
consumer (AddModelForm) to extract names from the new shape.
The previous shape was tailored to AddModelForm's single use case
(populate a Select with names). The team info per-guardrail opt-out
work needs default_on per guardrail to split globals from non-globals,
which the old shape couldn't provide. Consolidating into the existing
hook gives both consumers one source of truth and one React Query
cache entry instead of two parallel fetches.
- useGuardrails.ts: rewrite return type, derive global/optional sets
in select(); preserve the existing query key and auth-gate semantics
- AddModelForm.tsx: extract names from data?.guardrails.map(...)
- AddModelForm.test.tsx: update mock to return the new shape (also
fixes a pre-existing shape mismatch in the mock)
- useGuardrails.test.ts: update 3 assertions to read names via
data?.guardrails.map(...) instead of asserting against the flat array
* feat(guardrails): optional skip system message in unified guardrail inputs
Made-with: Cursor
* feat(dashboard): skip_system_message_in_guardrail in guardrail UI
Add a tri-state control (inherit / yes / no) when creating or editing
guardrails so admins can set litellm_params.skip_system_message_in_guardrail
without YAML. Table edit merges existing litellm_params before PUT to avoid
wiping content-filter and other provider fields.
Document the dashboard flow in the guardrails quick start with a screenshot.
Made-with: Cursor
* fix(guardrails): type structured_messages as AllMessageValues for mypy
Use AllMessageValues in openai_messages_without_system and cast adapter
request messages so GenericGuardrailAPIInputs matches TypedDict.
Made-with: Cursor
Tremor's internal Tooltip component sets a setTimeout that fires after
the jsdom test environment tears down, causing a ReferenceError. Add
afterEach that flushes pending timers before cleanup.
Raise vitest testTimeout from 10s to 30s and drop per-test timeout overrides
across UI unit tests. Group CreateUserButton and TeamInfo tests under nested
describe blocks to make the most flaky suites easier to scan.
Remove the silent try/catch from setSecureItem so OAuth hooks can
surface actionable "enable storage" guidance instead of a cryptic
"state lost" error after the round-trip. Add a local try/catch in
ChatUI where the storage write is non-critical.
- Dockerfile.health_check: HEALTHCHECK now verifies the script is intact
instead of unconditionally exiting 0
- secureStorage.ts: replace deprecated escape/unescape with
encodeURIComponent/decodeURIComponent; don't delete legacy values on
decode failure so in-flight flows can time out naturally
- OAuth callback: add same-origin check before redirecting to stored
return URL
Port security fixes from litellm_v1.82.3.dev.6:
- Use secureStorage (sessionStorage wrapper) instead of raw storage for tokens
- Add URL validation for stored worker URLs to prevent open redirects
- Add same-origin checks before redirecting to stored return URLs
- Harden Dockerfile.health_check with non-root user and exec-form HEALTHCHECK
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
On case-sensitive Linux CI, the old regenerate_key_modal.tsx from main
can coexist with the new RegenerateKeyModal.tsx after merge. The old
modal renders "Copy Virtual Key" while the new one renders "Copy Key".
Use /Copy.*Key/ to match both.
Raise vitest testTimeout from 10s to 30s and drop per-test timeout overrides
across UI unit tests. Group CreateUserButton and TeamInfo tests under nested
describe blocks to make the most flaky suites easier to scan.
calculateNewExpiryTime only handled s/h/d, but the grace-period
validation and backend accept m, w, and mo as well. Entering any of
those in the Expire Key field caused the function to return null,
which then propagated as expires: null in the onKeyUpdate payload —
the parent UI would then render the expiry as "Never" even though
the backend had correctly applied the new expiry.
Extend the suffix check to cover s/m/h/d/w/mo, matching "mo" before
"m" so "1mo" isn't misread as minutes. Also nullish-coalesce the
call site so an unparseable duration falls back to the previous
expiry instead of null. Add parametric tests for each supported
suffix plus a regression test for the null fallback.
The regenerate endpoint returns a GenerateKeyResponse that inherits
max_budget/tpm_limit/rpm_limit from KeyRequestBase, so the API echoes
the existing values back. The previous updatedKeyData layout spread
...response *after* the explicit formValues assignments, which meant
the user's just-submitted edits were silently overwritten by the API
echo before being propagated to the parent via onKeyUpdate.
Reorder so the response spread comes first and the formValues-derived
fields override it, and add a regression test that mocks a response
with stale limits to lock the behavior in. Also drop the two leftover
debug console.log statements.
- Label the key block with a small "Virtual Key" caption so the gray
box is clearly the key container.
- Move the Copy Key action to the modal footer as a primary button
with icon; inline copy icon next to the key is removed.
- Swap the button to "Copied" with a check icon on success instead of
firing a notification — less noisy and keeps feedback in place.
- Disable clicking outside the modal to close (maskClosable=false) so
users must explicitly dismiss via Close or X.
- Enlarge the key text and let its container span the full modal
width.
- Tests updated accordingly, including a new test for the copied-state
swap and the "Virtual Key" label.
Use Flex, Typography.Paragraph (with copyable), and Typography.Text
instead of raw divs + code block + CopyToClipboard wrapper. Drops the
direct react-copy-to-clipboard dependency in this component in favor
of antd's native copyable support.
Also fixes two test issues surfaced when running the e2e locally:
- RegenerateKeyModal.test.tsx no longer mocks react-copy-to-clipboard
(the component no longer imports it), removing the CJS require()
inside an ESM mock factory flagged by Greptile.
- keys.spec.ts scopes the Regenerate and Copy lookups to the modal.
The Regenerate button has an icon whose aria-label ("sync") is
concatenated into the button's accessible name, so an exact-match
lookup on "Regenerate" failed; and the new Paragraph copyable
renders a generic "Copy" button that collided with the other
copyable fields on the key info view.
