* feat(ui): add redesigned sidebar account menu
Introduce SidebarAccountMenu, a sidebar-only account/logout menu built on
shadcn Popover/Switch/Badge/Separator/Button, and wire it into leftnav in
place of the shared UserDropdown. The panel has a LiteLLM header with the
bouncing moon and a clickable version tag, Tier/Role/Email/User ID rows
with copy actions, the five display toggles, and Logout.
UserDropdown is left untouched so the control-plane / chat navbar keeps
its existing menu. The version tag links to the same release notes page
as the navbar tag, and the bouncing icon reuses the existing header
animation gated by the Hide Bouncing Icon toggle.
* test(ui): point account-menu e2e specs at the migrated sidebar menu
The sidebar account menu moved from an antd Dropdown to a Base UI popover
(SidebarAccountMenu), so the login, logout, proxy-logout-url, and internal
user identity specs were still waiting on antd-era locators
(.ant-dropdown, the popupRender wrapper class, the user-dropdown-panel test
id, and a menuitem-role Logout). Point them at the new panel test id
(sidebar-account-menu-panel) and the button-role Logout instead. The logout
behavior is unchanged since both menus call the same useLogout handler.
login and internalUserIdentity used an unscoped "Virtual Keys" text check that now also matches the top-bar breadcrumb, and adminSettings navigated via antd menuitem roles the shadcn nav no longer emits. Scope these to the sidebar's complementary landmark and drive the Settings group by role (expand the button, then click the child link), matching the fixes already applied to the migration and navigation specs
The redesigned sidebar + header shell is now scoped to the ai-gateway dashboard. Agent-control-plane (plugin) mode falls back to the original full-width Navbar, which carries the account menu, so preferences and sign-out stay reachable there; Chat and the public model hub already use that Navbar
Update the e2e specs the antd Menu -> shadcn nav swap broke. Sidebar items are now links/buttons rather than antd menuitems, and the top bar renders a breadcrumb whose current-page item is also a "Virtual Keys" link, so the migration and navigation smoke checks now scope their sidebar lookups to the complementary landmark and expand the new collapsible groups by data-slot. Also harden the collapsed-rail unit test to assert the icon and link rather than the CSS-hidden label
* fix(ui): scope key models dropdown options to the key's team
A teamless key no longer offers the all-team-models option in the create and
edit forms; the backend expands that sentinel to the full proxy model list when
no team is attached, which is rarely what the user intended. A team key no
longer surfaces the all-proxy-models sentinel that leaks in verbatim when the
team's own model list carries it; the dropdown keeps All Team Models plus the
team's individual models.
Adds browser coverage to the management e2e suite: playwright (an optional
dependency behind importorskip) drives the proxy-served dashboard at /ui,
asserts the dropdown options a real user sees for teamless and team keys on
both create and edit, and walks the create modal end to end, reading the
persisted key back through /key/info.
* fix(ui): offer all-proxy-models on teamless keys in the models dropdown
A teamless key has no team allowlist to inherit, so the dropdown now offers All
Proxy Models in place of All Team Models on both the create and edit forms, with
the same exclusive-selection handling. Component and browser e2e tests updated to
pin the swapped option pair; the teamless create case now also walks the modal end
to end and reads the persisted key back through /key/info.
* test(ui): update no-team key creation spec to pick All Proxy Models
The create modal no longer offers All Team Models without a team; the teamless
path now offers All Proxy Models, which is what this spec exercises
* fix(ui): gate All Team Models on the team object being loaded
When a key has a team_id but the teams prop does not yet include the matching team, availableModels stays empty and the models dropdown rendered All Team Models on its own with nothing to compare against. Gate the option on the team object being present so it only appears once team models are known, and add a regression test for the loading state
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): filter all-proxy-models from teamless model fetch in key edit form
The teamless fetch path stored modelAvailableCall results without excludeProxyWideSentinel, so an all-proxy-models entry in the response rendered a second option colliding with the hardcoded All Proxy Models sentinel. Apply the same filter used on the team path and add a regression test asserting the sentinel option is not duplicated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The e2e runs against the real proxy, so a contract drift already fails the test at
runtime; tsc only checks the spec against schema.d.ts, a generated snapshot, so a
backend change with a stale snapshot would pass tsc while the live test still
catches it. The dedicated tsconfig + script + CI step were circular ceremony for
that. Keep the zero-runtime-cost type-only import, which still catches mistakes in
the editor, and make its comment honest about what enforces the contract.
Address an adversarial review of the Loadbalancing e2e:
- The "typed against the backend schema" claim was hollow: nothing type-checked
e2e_tests (the root tsconfig excludes it and no CI step runs tsc), so a
contract drift would compile and run unchanged. Add e2e_tests/tsconfig.json, a
typecheck:e2e script, and a CircleCI step so the schema typing actually gates.
- The two describe blocks both mutate the proxy's shared router_settings, and the
Loadbalancing save echoes the whole settings object, so they could clobber each
other under local fullyParallel. Run the file serially.
- patchRouterSettings swallowed a failed seed, which surfaced later as a
misleading UI timeout. Assert the write succeeded, and rely on the server-side
merge instead of echoing the whole settings object back (drops a cast and a GET).
- Empty routing_groups already reproduces the bug, so drop the non-empty seed and
its model coupling.
Drives the real save flow against a live proxy: seeds a present routing_groups
array (the LIT-4057 trigger) via the typed /config/update contract, changes
num_retries on the Loadbalancing tab, and asserts the POST returns 200 instead
of 422, the success toast appears, and the value still shows after a reload (the
ticket's "refresh shows old values" symptom). The round-trip is typed against the
OpenAPI-generated backend schema (ConfigYAML write, RouterSettingsResponse read)
through a type-only import, so a backend contract drift fails the type check.
Cut the default "Virtual Keys" landing (?page=api-keys) over to a path route at
(dashboard)/api-keys. The dashboard is extracted into a shared ApiKeysDashboard
component used by both the new route and the index's inline render, so there's
no duplication. Adding the MIGRATED_PAGES entry repoints the sidebar item and
redirects ?page=api-keys to /ui/api-keys.
The index is the post-login landing and still hosts the legacy switch for the
not-yet-migrated pages (models, pass-through, usage) plus the invitation flow,
so it stays. The auto-redirect now fires only for an explicit ?page= param,
leaving the bare /ui/ landing to render inline; this keeps the return-URL
handling and the invitation_id flow (both of which run at the bare landing)
intact, where a blanket redirect would have dropped them. The new route uses
useAuthorized for the login gate, matching every other migrated route.
* feat(ui): migrate models page to App Router path route
Cut the Models + Endpoints page over from the legacy ?page=models switch
in (dashboard)/page.tsx to a path route at (dashboard)/models-and-endpoints.
Adding the MIGRATED_PAGES entry repoints the sidebar link and redirects old
?page=models bookmarks to /ui/models-and-endpoints.
ModelsAndEndpointsView already sourced identity from useAuthorized() and its
own data via useModelsInfo(), so the token/keys/modelData/setModelData props
were dead; drop them from ModelDashboardProps (and the parent's now-unused
setModelData state) to sever the last of the shared-state coupling.
* test(ui): scope migration smoke's shell probe to the exact sidebar link
The migration smoke used a loose `locator("a", { hasText: "Virtual Keys" })`
to assert the dashboard shell rendered. The Models + Endpoints page content
itself links to the "Virtual Keys page", so on that route the substring filter
matched two anchors and tripped Playwright strict mode. Match the sidebar link
by its exact accessible name instead, which resolves to just the nav item.
* feat(ui): cut playground over to the /ui/playground path route
Follows the api-reference recipe: the sidebar and deep links route
llm-playground to the path route, ?page=llm-playground redirects, and
the legacy switch arm is deleted. The route's page.tsx was already the
real implementation, so no view extraction was needed.
* refactor(ui): move playground-owned files into its route folder
Per the (dashboard) README convention, page-owned code lives in the
page's folder: chat_ui/compareUI/complianceUI components, the chat
hooks, and the playground-only llm_calls helpers move under
(dashboard)/playground/. Modules with non-playground consumers (chat
message primitives; fetch_models, chat_completion, responses_api) stay
at their lowest common ancestor in src/components/{chat_ui,llm_calls}
because legacy pages still import them. eslint-suppressions entries are
re-keyed to the new paths so the grandfathered baseline still applies.
* test(ui): teach sidebar e2e spec about migrated path routes
The sidebar spec asserted ?page=<key> for every item, which the
playground cutover correctly broke: the sidebar now links to
/ui/playground and the legacy URL redirects there. Drive the expected
URL from the migration fixture (now a page-id -> segment map) so
future cutovers only add a fixture entry. Also wrap one import line
in AgentBuilderView.tsx that the move left unformatted; the changed-
files prettier check flagged it.
* test(ui): add a data-driven App Router migration E2E smoke
Add a growing Playwright smoke for migrated pages: for each segment it deep-links
to the path route, asserts the URL and that the dashboard shell rendered, then
clicks off to a legacy page and asserts navigation still works. Driven by
e2e_tests/fixtures/migratedPages.ts, so adding a page is one line.
Runs in two situations against the same proxy: the default mount (npm run
e2e:migration) and a non-root SERVER_ROOT_PATH mount (npm run e2e:migration:root).
globalSetup now logs in at `${SERVER_ROOT_PATH}/ui/login` so the admin storage
state is valid under a prefix. Seeded with api-reference; append the rest as their
migrations merge.
* test(ui): support headed slow-motion + watch pauses in the migration smoke
Honor SLOWMO in the server-root-path config (the default config already did),
and add an env-gated E2E_WATCH_MS pause so a headed run lingers on each state.
Both are no-ops by default, so CI behavior is unchanged.
* test(ui): make the migration smoke a sidebar-click user journey
Rework the smoke from deep-linking to a real navigation journey: start at the
landing page, click the migrated page in the sidebar (expanding submenus for
nested items), assert the path route rendered, reload it (the check a wrong
server_root_path breaks), bounce to a legacy page and back, and — once two pages
are migrated — navigate directly between two migrated pages. Verifies via URL +
shell render, driven by the same fixture list.
* test(ui): address review on the migration smoke
Escape ROOT and segment before interpolating them into RegExp URL matchers so a
future segment containing regex metacharacters can't silently widen the match.
Make the server-root-path config fail fast when SERVER_ROOT_PATH is unset instead
of silently re-running the default mount and passing without exercising the prefix.
* test(ui): drop unused watch helper and fix stale smoke README
* test(ui): run the migration smoke under a server root path in CI
* test(ui): harden + instrument the server-root-path proxy reboot in CI
* test(ui): run the server-root-path migration smoke as its own CI job
Replace the in-place proxy reboot in e2e_ui_testing with a dedicated
e2e_ui_testing_server_root_path job that boots the proxy once with
SERVER_ROOT_PATH=/litellm, matching how every other proxy variant in the
config gets its own job rather than killing and relaunching the live proxy.
The reboot was failing deterministically: after pkill -9 and relaunch the
prefixed proxy never came back up on :4000 (connection refused), so the smoke
never ran. The readiness step that was supposed to surface the cause could
never reach its boot-log tail because CircleCI runs steps under bash -eo
pipefail and the preceding `curl -sv ... | tail` aborted the step with curl's
exit 7. Booting the proxy as the job's own background step lets any boot crash
land in that step's log instead of being swallowed.
The default e2e_ui_testing job is unchanged aside from dropping the reboot,
prefixed-readiness, and prefixed-smoke steps; the migration smoke still runs at
the root mount there via the default Playwright config.
* refactor(ui): centralize proxy base URL resolution into tested resolver
The API base URL join logic was hand-rolled inside networking.tsx and
re-derived inline at hundreds of call sites, with no test coverage and a
latent double-slash bug when the base carried a trailing slash. This pulls
the join into a single pure resolveApiBase() with full unit coverage and
routes the existing resolution through it, also de-duplicating the env
precedence ladder that was copied in two places.
* test(ui): assert root-path redirect joins prefix exactly once
The existing toContain check accepts a doubled separator; tighten it to a
strict prefix match plus a no-double-slash assertion so a regression in the
resolveApiBase origin+SERVER_ROOT_PATH join is caught end-to-end.
The Next.js admin UI is exported with trailingSlash: true, so the proxy
serves /ui/login at /ui/login/index.html and 308s /ui/login → /ui/login/.
The waitForURL predicate used endsWith("/ui/login"), which never matched
the canonicalized URL and timed out after 15s.
This was masked until the build artifacts were regenerated against the
AuthContext fix: the prior bundles still hit the racy redirect path that
fired before proxyBaseUrl was populated, producing /ui/login (no prefix,
no proxy round-trip, no trailing slash) which fortuitously satisfied the
predicate. The first PR to ship the corrected bundle exposed the
assertion bug.
Switch the predicate to includes("/ui/login"); the prefix assertion below
still validates the SERVER_ROOT_PATH preservation that is the actual
contract under test.
* test(e2e): cover PROXY_LOGOUT_URL redirect on Logout
Env-gated spec mirroring the existing serverRootPathRedirect pattern:
when the proxy is booted with PROXY_LOGOUT_URL set, clicking Logout in
the navbar must navigate to that external URL. The standard run_e2e.sh
exports an empty value so the rest of the suite is unaffected; this
spec self-skips unless the env var is populated.
* test(e2e): run PROXY_LOGOUT_URL spec in the suite + harden logout assertions
Boot the e2e proxy with PROXY_LOGOUT_URL set (job-level env in CircleCI and
run_e2e.sh) so proxyLogoutUrl.spec.ts actually runs instead of self-skipping.
Nothing else in the suite performs a logout, so this only affects the behavior
under test.
Harden the spec to verify the logout flow rather than a URL substring:
- wait for /sso/get/ui_settings before clicking so logoutUrl is populated
(otherwise window.location.href = "" silently reloads same-origin)
- assert a token cookie exists first, and is cleared after logout
- locate the dropdown via getByRole instead of internal antd CSS classes
- stub the external destination and assert on URL origin + path prefix
* test(e2e): assert exact PROXY_LOGOUT_URL on logout redirect
Replace the origin + startsWith(pathname) checks with a single normalized
href comparison. With PROXY_LOGOUT_URL=https://www.example.com the path was
"/", so startsWith("/") matched any path and left path/query/hash
unchecked. Comparing normalized hrefs pins scheme, host, port, path, query
and hash while still tolerating the browser's trailing-slash/default-port
normalization.
* test(e2e): assert internal-user navbar identity is scoped to that user
The existing login.spec.ts only checks the admin's navbar identity.
This adds the symmetric check for the internal user — verifying the
account button + dropdown surface the internal user's email, id, and
role, and that no admin-scoped values leak through.
* test(e2e): harden navbar identity test per review feedback
Locate the user dropdown panel by a data-testid on the popupRender div
instead of Ant Design internal + Tailwind class names, so styling
refactors no longer risk breaking the identity-scoping assertions.
Source the seeded user emails/ids from shared constants (match seed.sql)
instead of hardcoding them inline.
* test(e2e): cover Internal User create-key flow when in no teams
The seeded e2e-internal-user is in two teams, so the "no team" branch
of the Create Key modal — where the team dropdown must render empty —
was unreachable. Seeds a noteam@test.local user and adds a spec that
logs in fresh, opens the modal, and asserts the dropdown has zero
options.
* test(e2e): harden no-team dropdown assertion + add with-teams counterpart
Replace the one-shot count() check with a settled-empty assertion: wait for
the dropdown's loaded "No teams found" state before asserting zero options,
so the test can't pass on a transient empty frame while the team-options
request is still in flight.
Add internalUserWithTeams.spec.ts as the differential partner; it logs in as
the seeded e2e-internal-user (two team memberships) and asserts the dropdown
lists exactly those teams. Without it, the no-team spec's zero-options
assertion would still pass against a regression that empties the dropdown for
every user.
* test(e2e): cover navbar Logout flow as proxy admin
The Logout button under the navbar User dropdown was an uncovered
manual-QA step. This test signs in as admin, opens the dropdown,
clicks Logout, then navigates to a protected page and asserts the
redirect to /ui/login — proving the session was cleared.
* test(e2e): fix logout dropdown trigger and account-menu selector
The button never rendered the literal text "User" (it shows initials +
display name), and the antd Dropdown uses trigger={["click"]}, so the
synthetic mouseover/mouseenter never opened the popup. Open it with a real
click on the button's aria-label ("Account menu — ...").
* test(e2e): cover Internal User key modal, team info, key page
Three previously-uncovered manual-QA paths for the Internal User role:
- Create Key modal — confirm the team dropdown is populated with the
user's teams (verifying the role-scoped UI flow exists).
- Team info page — confirm the Settings/Members tabs are hidden for a
regular team member; only the read-only tabs render.
- Virtual Keys page — confirm the proxy's internal litellm-dashboard
team keys never leak into an internal user's table.
* test(e2e): share clickTeamId helper, strengthen key-filter assertion
Address review feedback on the Internal User e2e spec:
- Extract clickTeamId into helpers/navigation.ts; import in both
internalUser and teams specs instead of duplicating it.
- Anchor the litellm-dashboard absence check on the user's own seeded
key so it cannot pass vacuously against an empty table.
- Drop redundant dismissFeedbackPopup calls (navigateToPage already
dismisses internally).
* test(e2e): cover Internal Viewer nav, key, and team-info gating
Three previously-uncovered manual-QA paths for the Internal Viewer role:
- Nav only renders the read-only sections; admin-only items
(Internal Users, Organizations, Models + Endpoints) stay hidden.
- Virtual Keys page hides Create New Key, and the key detail view
hides Regenerate / Reset Spend / Delete actions.
- Team info page hides Members and Settings tabs for the viewer.
* test(e2e): scope viewer nav to sidebar, strengthen tab assertions
Address review feedback on the Internal Viewer e2e spec:
- Scope the nav test to the sidebar complementary landmark and match
items by link role + accessible name. The prior CSS nav, aside
selector grabbed the top bar (the sidebar is a complementary
landmark, not a <nav> tag), so the assertions never hit the real
nav links.
- Land via navigateToPage so the networkidle wait settles the
role-gated nav before asserting.
- Assert the Virtual Keys tab is visible (was only commented).
- Use toHaveCount(0) for hidden team tabs to match the nav block;
tabs are conditionally rendered, not CSS-hidden.
- Drop redundant dismissFeedbackPopup calls (navigateToPage already
dismisses internally).
* test(e2e): cover Team Admin view + member + key flows
Adds a new spec exercising the previously-uncovered team-admin manual-QA
items: viewing all team keys (including other members'), adding a member,
removing a member, and creating a team key with All Team Models. Also
seeds a dedicated invitee user so the add-member test can run in parallel
with the proxy-admin invite test without colliding on the team roster.
* test(e2e): harden team-admin member specs per review feedback
Address Greptile feedback on the Team Admin spec:
- locate the delete action via getByTestId("delete-member") instead of
the fragile svg/img .last() selector
- match the seeded removable member by user_id (members_with_roles stores
no email, so the roster renders user_id)
- assert exact success-toast strings rather than broad regexes that could
match unrelated "success" text
* test(e2e): cover AI Hub make-public flow and public model_hub_table
Three previously-uncovered manual-QA paths land in one spec:
- Admin opens "Select Models to Make Public", advances through the
multi-step modal, and verifies the success toast.
- AI Hub tab strip exposes Model Hub / Agent Hub / MCP Hub / Skill Hub
— note the manual-QA "Claude Code Plugin Marketplace" label was
renamed to Skill Hub; the test pins the current name.
- Anonymous /ui/model_hub_table loads with the master key as `?key=`
and renders the Model Hub tab. Agent Hub / MCP Hub tabs are
conditional on public data and are not asserted here.
* test(e2e): harden AI Hub make-public + public hub assertions
Address Greptile review:
- Make-public test now asserts "Select All (N)" with N>=1 before clicking,
so a missing-seed-data run surfaces immediately instead of timing out
on the disabled Next button or the success toast.
- Public model_hub_table test dismisses the feedback popup before the
tab visibility assertion, matching the ordering used by navigateToPage
so a popup race can't mask the tab mid-evaluation.
* docs(e2e): explain admin vs public AI Hub tab asymmetry
Greptile flagged the all-4-tabs assertion as a potential CI flake,
inferring from the public-page comment that Agent Hub / MCP Hub might
be data-conditional in the admin view too. They aren't — ModelHubTable
renders all four tabs unconditionally for admins. Document the asymmetry
inline so future readers (and future review passes) don't re-derive it.
* test(e2e): cover add-MCP-server flow via discovery → custom form
The "Add MCP server" manual-QA step was uncovered. This adds a test
that opens the discovery modal, jumps into the custom-server form,
fills name + Streamable HTTP transport + a placeholder URL + None
auth, submits, and verifies both the success toast and the new row.
* test(e2e): apply greptile fixes to MCP add-server test
- Anchor the auth-type Select via its enclosing Collapse panel
("Authentication") instead of the placeholder text. The Form.Item has
no label prop, so the previous `hasText: /auth type/i` filter was
matching via "Select auth type" placeholder copy — fragile.
- Document the intentional lack of teardown, matching the pattern used
in addModel.spec.ts: the e2e runner discards the DB per invocation.
Addresses Greptile P2s on PR #29070.
* test(e2e): scope MCP row assertion to the servers table
Scope the post-create row lookup to `table tbody` so the form modal's
`server_name` input — which still holds the timestamped value during
its close animation — can't satisfy the assertion before the server
actually lands in the list.
* docs(e2e): note MCP coverage scope and link to tracker
This spec only smoke-tests the happy-path Streamable HTTP + None auth
flow. Add a top-of-file comment pointing at E2E_COVERAGE.md so future
contributors can see what's still uncovered (other transports, all
auth types, edit/delete, BYOK, tool list/call, access groups).
* test(e2e): cover Team-BYOK add-model flow as proxy admin
The team-only model + team assignment was an uncovered manual-QA path.
This adds a premium-gated test that toggles Team-BYOK, picks the seeded
E2E Team CRUD, submits, and verifies the model lands in All Models with
the team alias attached.
* test(e2e): apply greptile fixes to Team-BYOK test
- Add the 2s networkidle settle that the sibling addModel tests use —
networkidle fires before the All Models table finishes re-rendering,
so the search input was racing with the render.
- Assert on `models-results-count` before inspecting the table body so
an empty search result fails with a clear "expected results count"
message instead of timing out on a missing row.
Addresses Greptile P2s on PR #29068.
* test(e2e): harden Team-BYOK test against flake and stale state
- Add before/after cleanup that deletes any Cohere model already scoped
to e2e-team-crud via /v2/model/info + /model/delete, so Playwright
retries and local reruns don't accumulate rows.
- Pick the team from the dropdown by role/option name instead of a
global getByText match — avoids matching a previously-rendered tag
elsewhere in the form.
- Scope the "created successfully" assertion to .ant-notification so a
stale toast from an earlier test in the same browser context can't
vacuously satisfy it.
- Tighten the All Models assertion: require a single row that contains
BOTH the cohere model name AND the e2e-team-crud alias, so the
team-less wildcard from the sibling "Add wildcard route" test can't
satisfy the check.
* test(e2e): cover add-fallback flow in Router Settings as proxy admin
The Router Settings → Fallbacks → Add Fallbacks flow was an uncovered
manual-QA path. This adds a test that opens the modal, picks a primary
+ fallback from the seeded mock models, saves, and verifies both render
in the fallback table.
* fix(e2e): make router-fallback test idempotent and pick antd options by text
- Match `.ant-select-item-option` by text instead of `getByTitle(...)` —
FallbackGroupConfig uses `options=` (not <Select.Option> children), so
no `title` attribute is emitted and the title-based selector hangs.
- Add before/after hooks that wipe any fallback for fake-openai-gpt-4 via
/config/update so retries and local reruns don't trip on leftover state.
- Tighten the success assertion to a single tbody row containing BOTH the
primary and the fallback names — pre-existing rows can no longer
vacuously satisfy the check.
- Fix the stale "Three tabs" comment to "Four tabs".
Addresses Greptile P2s on PR #29069.
* fix(e2e): keyboard-select fallback models + correct cleanup endpoint
- Replace mouse-based option clicks with click-to-focus + type + Enter.
FallbackGroupConfig's Selects use `options=` and a custom
getPopupContainer, so locating options via `.ant-select-dropdown`
hit several races: DOM-clicks left antd's popup state stale (the
primary popup then intercepted the fallback click), `getByRole`
matched always-mounted hidden options, and pointer stability fought
the open animation. Typing into the showSearch input narrows the
listbox to one option and Enter selects it cleanly.
- Assert on dialog-side state changes (the active tab adopts the
primary model name; the chain helper shows "1/10 used") instead of
popup contents — these reflect the actual selection landing.
- Cleanup helper now hits /get/config/callbacks (the real endpoint;
/get/callbacks returns 404), so the before/after reset actually
clears prior router_settings.fallbacks state.
* test(ui): e2e cover team model edit + admin identity in navbar
Adds two Playwright tests as part of the manual-QA → e2e migration:
"Edit team model selection" exercises the Settings tab Models multi-select
+ Save Changes flow on a seeded team, and the existing login test now
opens the User dropdown and asserts the role and User ID render — guarding
against regressions where login succeeds but the auth context is empty.
Resolves LIT-3093
* test(ui): restore seeded models in team-edit test so retries don't fail
The 'Edit team model selection' test removed fake-anthropic-claude from
E2E_TEAM_CRUD_ID without restoring it. CI runs with retries: 2 and the seed
script runs once before the suite, so a flake on this test would fail the
retry at the "tag is visible" assertion. Wrap the test in try/finally and
restore the seeded models via /team/update before and after.
* test(e2e): fail loudly if team/update restore call fails
Surfaces the real cause when the master key is wrong or the proxy is
unreachable, instead of silently leaving the team in a stale state and
failing later on the visibility assertion.
* fix(e2e): match navbar account button by aria-label, not non-existent "User" text
The previous trigger filter (hasText: /^User$/) didn't match the rendered
UserDropdown button — its text is the displayName ("Account" for the
master-key admin, an email for SSO users), never "User". The evaluate
call then timed out after 15s in CI. Use the stable aria-label prefix
the component always emits, and click directly since the dropdown is
configured trigger=["click"] (the synthetic hover was unnecessary).
* refactor(ui): extract auth state into AuthContext
Move auth state (token, userID, userRole, accessToken, premiumUser, userEmail,
disabledPersonalKeyCreation, showSSOBanner) out of src/app/page.tsx into a
new AuthProvider at src/contexts/AuthContext.tsx. Wrapped at the root layout
so login/onboarding/dashboard routes all have access via useAuth().
Day 1 foundation for the App Router migration: migrated (dashboard)/X/page.tsx
route entry points won't have a parent passing props, so shared auth state
must live in a context they can read from.
Sub-components are unchanged — they still receive accessToken/userID/userRole
as props from page.tsx (which now reads them from useAuth()). Only the
page.tsx → top-level-page-component handoff is de-drilled; deeper prop
drilling is left for the per-page migration to address.
Net change: -86 lines from page.tsx (state + two effects moved), +5 in
layout.tsx (provider wrap), new AuthContext.tsx (~140 lines), test update
to wrap CreateKeyPage in AuthProvider.
Fixes LIT-3366
Part of LIT-3128
* fix(ui): await getUiConfig before clearing authLoading
The AuthContext refactor flipped authLoading to false synchronously on mount
while letting getUiConfig() run fire-and-forget. On SERVER_ROOT_PATH deployments
this races the unauthenticated login-redirect effect: the redirect fires with
proxyBaseUrl still at its module-init value, sending users to /ui/login instead
of {SERVER_ROOT_PATH}/ui/login.
Restores the original sequencing inside AuthProvider's mount effect and adds a
Playwright spec wired into the existing SERVER_ROOT_PATH workflow matrix. The
spec delays the config endpoint via page.route() to make the race deterministic
across CI runners.
* fix(model-edit): allow clearing custom input/output cost on wildcard deployments
A user-set pricing override on a `/model/*` wildcard deployment could not
be removed: clearing the Input/Output Cost fields in the UI succeeded
visually, but the next read still showed the old values because both
`litellm_params` and `model_info` (mirrored via `SPECIAL_MODEL_INFO_PARAMS`)
retained the original rates.
UI: when the pricing field is touched but left empty, send `null` instead
of dropping it from the payload so the backend sees the clear intent. The
cache-read-cost fallback now guards against `null` as well as `undefined`
so a cleared input cost cannot silently wipe the cache-read override.
Backend: `update_db_model` honors explicit-null clears, but ONLY for
`SPECIAL_MODEL_INFO_PARAMS` (the 4 pricing fields). Restricting the
null-clear path prevents a team-scoped caller from using this codepath to
null out privileged fields like `team_id` or access groups.
Tests cover both clear paths (`litellm_params` and `model_info`), the
SPECIAL_MODEL_INFO_PARAMS mirror, PATCH semantics for omitted fields, and
the security guard that non-pricing nulls don't reach the merged dict.
Resolves LIT-3250
* fix(model-edit): run null-clears after both merges, not interleaved
The previous version cleared `model_info` from inside the litellm_params
merge block, but the subsequent `model_info.update(...)` re-injected the
old pricing because the UI's PATCH carries the full model_info blob with
the stale values still in it. Move the explicit-null clear pass to after
both merges so a model_info passthrough cannot resurrect cleared fields.
Adds a regression test for the realistic UI submit shape (both blobs in
the patch, model_info still holding the old pricing).
* test(e2e): clear-custom-pricing flow with create/delete cleanup
Covers the dashboard model edit form's pricing-clear flow end-to-end:
seeds a deployment with custom input/output pricing, drives the UI to
clear both fields, asserts the outgoing PATCH sends explicit nulls,
and confirms via /v2/model/info that the override is gone from both
litellm_params and model_info.
The dashboard DB persists across this suite, so beforeEach creates a
uniquely-named deployment and afterEach POSTs /model/delete to leave
the DB clean regardless of test outcome.
* fix(model-edit): extend pricing clear to cache_read and cache_write costs
Pre-existing parallel of the wildcard input/output cost bug: cleared
cache_read_input_token_cost and cache_creation_input_token_cost overrides
silently persisted because the UI omitted the key (delete or fallback) and
the backend null-clear allowlist did not cover them.
- types/router.py: add cache_read_input_token_cost and
cache_creation_input_token_cost to SPECIAL_MODEL_INFO_PARAMS, so they are
mirrored between litellm_params and model_info by Deployment.__init__ and
honoured by the null-clear loop in update_db_model.
- model_info_view.tsx: emit explicit null for touched-but-empty cache_read
and cache_write fields. Preserve the input_cost->cache_read mirror only
when cache_read itself was not touched.
- model_management_endpoints.py: update the allowlist comment.
- Tests: three new unit tests for cache clear paths and a preserve check;
the e2e spec now seeds, clears, and asserts null PATCH + key-absence for
all four pricing fields.
* test(e2e): forward LITELLM_LICENSE to UI e2e proxy
The UI e2e job ran without LITELLM_LICENSE, so premium_user was always
false in the issued login JWT and premium-gated UI surfaces (Team-BYOK
Model switch, etc.) couldn't be driven through the UI. Forward the env
var from run_e2e.sh and the CircleCI e2e_ui_testing job, and add a
sanity test that decodes the admin storage state token and asserts
premium_user=true so the wiring fails loudly if it ever regresses.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Update ui/litellm-dashboard/e2e_tests/tests/proxy-admin/license.spec.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* test(ui-e2e): add admin key creation with a specific proxy model
Adds Playwright coverage for creating a key (no team) scoped to a single
proxy model, complementing the existing All-Proxy-Models test. Uses a
DOM-dispatched click on the antd dropdown option since the popup
animation can render the option outside the viewport.
* test(ui-e2e): verify scoped key works against mock /chat/completions
Extend the "Create a key with a specific proxy model" test to extract
the new key from the success modal and POST to /chat/completions for
the scoped model, asserting 200 and the mock response body. Without
this the test could pass even if the model selection failed to register.
* chore(e2e): migrate runner to uv, add All Proxy Models key test
Switches the local e2e runner (run_e2e.sh) from poetry to uv to match
the rest of the repo and CI. Adds a Playwright test for creating an
admin key with no team selected (all-proxy-models flow), a SLOWMO env
hook for headed debugging, and a MIGRATION_TRACKING.md doc that maps
the manual UI QA checklist to e2e tests so future migration work has
a single source of truth.
* chore(e2e): address greptile feedback
- Remove MIGRATION_TRACKING.md (docs belong in litellm-docs repo)
- playwright.config.ts: fall back to 0 when SLOWMO is non-numeric
(parseInt returns NaN, which Playwright accepts silently)
- run_e2e.sh: add --frozen to uv sync for CI determinism
The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)
Added data-testid attributes to UI components to support stable test selectors.
Tests verified passing 3/3 consecutive runs with zero flakiness.
Reviewer flagged that cleanup failures were silently swallowed and
suggested asserting `delete.ok()`. While thinking through the fix, the
actual question turned out to be "does the cleanup matter at all?" —
and the answer is no.
The e2e runner (`run_e2e.sh`) spins up a fresh postgres container per
invocation and tears it down at the end, so every local and CI run
starts with an empty DB. Playwright retries share the same DB but each
attempt creates a new model with a unique `Date.now()` name and only
queries its own model, so orphans from failed attempts never collide
with later attempts or other tests. Nothing else in the suite reads
the all-models table.
Keeping the cleanup would also turn every write test into an implicit
delete test, coupling responsibilities and inflating runtime — which
is probably why `teams.spec.ts` (create a team), `keys.spec.ts`
(update key limits), etc. all leave their entities in place. Matching
that convention, drop the try/finally block and the `createdModelId`
tracking. 12 lines removed, no behavior change.
Covers the full write-path flow for team-scoped models on the Models +
Endpoints page: create via /model/new, click the row to open the detail
view, click Edit Settings, change TPM/RPM, click Save Changes, assert
the new values render back. Cleans up via /model/delete in finally so
reruns stay deterministic.
Requires store_model_in_db: true in the fixture general_settings so the
proxy accepts /model/new and /model/delete. No existing test in the
dashboard e2e suite reads the all-models table or hits the model CRUD
endpoints, so enabling the flag has no cross-test impact.
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
On case-sensitive Linux CI, the old regenerate_key_modal.tsx from main
can coexist with the new RegenerateKeyModal.tsx after merge. The old
modal renders "Copy Virtual Key" while the new one renders "Copy Key".
Use /Copy.*Key/ to match both.
- Label the key block with a small "Virtual Key" caption so the gray
box is clearly the key container.
- Move the Copy Key action to the modal footer as a primary button
with icon; inline copy icon next to the key is removed.
- Swap the button to "Copied" with a check icon on success instead of
firing a notification — less noisy and keeps feedback in place.
- Disable clicking outside the modal to close (maskClosable=false) so
users must explicitly dismiss via Close or X.
- Enlarge the key text and let its container span the full modal
width.
- Tests updated accordingly, including a new test for the copied-state
swap and the "Virtual Key" label.
Use Flex, Typography.Paragraph (with copyable), and Typography.Text
instead of raw divs + code block + CopyToClipboard wrapper. Drops the
direct react-copy-to-clipboard dependency in this component in favor
of antd's native copyable support.
Also fixes two test issues surfaced when running the e2e locally:
- RegenerateKeyModal.test.tsx no longer mocks react-copy-to-clipboard
(the component no longer imports it), removing the CJS require()
inside an ESM mock factory flagged by Greptile.
- keys.spec.ts scopes the Regenerate and Copy lookups to the modal.
The Regenerate button has an icon whose aria-label ("sync") is
concatenated into the button's accessible name, so an exact-match
lookup on "Regenerate" failed; and the new Paragraph copyable
renders a generic "Copy" button that collided with the other
copyable fields on the key info view.
Replace Tremor components in the regenerate key modal with Ant Design
equivalents and move the component to a new PascalCase file. The form
layout now uses Row/Col to place Max Budget, TPM Limit, and RPM Limit
on one row and Expire Key with Grace Period on another, reducing the
vertical footprint. The success view shows an Alert banner, the key
alias as secondary context, and the regenerated key in a monospace
block with an inline primary Copy button.
Also adds unit tests for the new component and updates the existing
Playwright spec to match the new banner and button text.
- Add constants.ts with all required exports (key aliases, team IDs)
- Add fixtures/users.ts with all role definitions and storage paths
- Add fixtures/seed.sql for deterministic test database seeding
- Remove Firefox project from playwright config (only Chromium installed)
- Remove unused variable in teams.spec.ts
- Rename CircleCI job to e2e_ui_testing
Add Playwright E2E tests covering proxy admin team and key management
workflows, with a self-contained test runner and CircleCI integration.
Tests cover: create team, invite user, edit/delete team members, create
key in team, regenerate key, update TPM/RPM limits, delete key, and
verify internal user keys are visible.
Infrastructure: run_e2e.sh builds the UI from source before starting
the proxy, ensuring tests always run against the latest UI changes.
Added data-testid attributes to key UI components for reliable selectors.
The createKey test broke because the Models field changed from required
to optional (PR #22826), removing the asterisk from the accessible name.
The unauthenticated redirect test broke because the login URL now
includes a redirect_to query param. Both selectors now use regex to be
resilient to these kinds of changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
