scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

feat(ui): inline Lens settings tab and investigation editor (#44479)

Browse source 
* feat(ui): move worker status into the Lens notch and New investigation into the list toolbar

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(ui): make the Lens notch entry a settings gear that houses the worker section

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(ui): replace the Lens worker modal with an inline Settings tab

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(ui): section the Lens settings tab with tracing status and worker cards

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ui): let Lens settings sections span the full card width

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(ui): replace the investigation setup modal with an inline side-by-side editor

New, edit, and duplicate now take over the Investigations tab body: matching
activity on the left, every setting on the right, with no wizard steps or modal

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(ui): step the inline investigation setup vertically with traces alongside

Setup now sits on the left as three progressive steps (activity, criteria,
run) that collapse to a summary once done and reopen on click. Matching
activity stays on the right for every step. The editor gets a back control
and the Investigations notch shows a New, Editing, or Duplicate badge while
the editor is open

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(ui): page the matching activity preview with useInfiniteQuery as it scrolls

Replace the Previous/Next offset buttons with the same infinite query and
near-tail prefetch the traces list uses, so the preview keeps loaded runs
and its title while the next page arrives

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): make the investigation step field map exhaustive over the form schema

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ui): always show the Settings tab label in the Lens mode switch

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): collapse Lens worker cards into compact status rows

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ui): keep the Lens Settings tab icon-only in every state

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wip

* fix(ui): tick the Lens worker health dot so an expired heartbeat goes stale

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): regroup Lens settings, model and api layers

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): dedupe Lens formatting helpers

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ui): poll Lens once, drive the interval from data, settle mutations before invalidating

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): let Lens leaves fetch their own data

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): bind drawer and trace shortcuts through react-hotkeys-hook

One useShortcut hook replaces the three hand-rolled keydown listeners in SidePanel,
the trace step tree and the log drawer, with a layer option deciding which keys a
pane claims from the panel around it. The span tree footer now renders ShortcutHints
from what is actually bound instead of hand-typed kbd text.

* refactor(ui): extract Inspector from SidePanel

Inspector.Root owns the open item, J/K stepping, Escape and full screen;
Inspector.Row marks a list entry with aria-selected and data-state and toggles
it on click or Enter/Space; Inspector.Panel is the resizable side panel with the
exit animation and click-outside rules. The runs table and section compose these
parts directly, so RunDrawer and the SidePanel prop bag go away.

* refactor(ui): model the Lens worker screen as a tagged union and slot in its ready action

workerScreen() decides between list, form and install from the worker rows,
the registration result and the edit target, so WorkerSettings switches on
one value and each card owns its own copy. The post-install CTA is now a
ReactNode slot that LensWorkspace fills instead of an onReady callback
threaded through LensSettings and WorkerSettings. Clipboard copy state lives
in WorkerInstall as mutations, and the styled settings leaves export Props
types, set data-slot and accept native element props.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): compose the Lens setup stepper from SetupStep children

Each step's heading, summary and fields now live together in one
SetupStep instead of four parallel structures keyed by index, and the
last-step spacing comes from CSS rather than a passed index. The mode
prop is now required since InvestigationsView always passes it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ui): keep the Lens Settings panel mounted so a pending worker install survives tab switches

The settings TabsContent unmounted WorkerSettings whenever another tab was
active, dropping the one-time worker token shown during install. The panel
now uses keepMounted, and the workspace test registers a worker, switches
tabs and back, then follows the connected worker into the first
investigation through the slotted CTA.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(ui): cover the Lens worker install waiting-to-connected transition

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): give the Lens activity preview a grouped contract and a structural debounce

useMatchingActivity now owns its return types (scope options, preview
status, page and optional manual selection) instead of borrowing them
from the components it feeds, and the preview takes those groups plus
the section attributes. The clear-selection action moves into the
preview footer, RunList becomes a RunRow leaf, and ScopeFields drops
the unused nameField and id props now that MetadataFilters calls useId
itself. The preview scope settles through a hashKey-based
useDebouncedValue instead of JSON round-tripping into state, and the
loading title follows isPlaceholderData since the query keeps previous
data. RunFields and AnalysisModelField take the analysis models and the
model gate as two objects instead of seven flat props.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): select the Lens investigations screen with a pure tagged union

investigationScreen maps the list query and the route to one of loading,
failed, welcome, list, detail, setup or missing, so the view can switch
instead of juggling mutually exclusive booleans. The status model gains
activeJob, carries connected inside Readiness and folds the activity probe
into one ActivityCheck value for the welcome page

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(ui): cover the Lens preview footer clear action and the preview debounce

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): let Lens investigation leaves own their URL slice and express intent

InvestigationsView renders the screen union and owns every write through
useInvestigationActions, so leaves receive on* handlers instead of the API
writer. useInvestigationResults becomes useRunSnapshot; FindingsTab,
HistoryTab, RunPicker and RequestEvidenceSheet read their own nuqs slice
and run their own queries. The finding sheet becomes an Inspector side
panel (FindingDetails) keyed per finding, with the trace and request
evidence sheets grouped in EvidenceSheets. WatchAllBanner owns its
mutation, the welcome page takes the readiness and activity values, the
progress sampler records on the wall clock outside render, and run
history invalidates when the list reports a scheduler-started job

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(ui): cover Lens investigation intents, pause, cancel, history refresh and the finding panel

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ui): keep the inspector open behind sheet overlays and mount HotkeysProvider from a client wrapper

* feat(ui): open Lens runs and evidence in the inspector panel

A quote's original trace step or logged request now stacks inside the finding
panel behind a back link, keeping the finding and its feedback draft mounted.
The detail Runs tab and the setup activity preview open runs in the same panel
with J/K stepping, so the TraceSheet and RequestEvidenceSheet modals are gone.
Picking a different finding or run clears any stacked evidence from the URL.

* feat(ui): open Lens investigations in the inspector panel beside the list

The investigations list stays on screen and a row opens its investigation in the
side panel, so J/K walk investigations and their open findings in display order
and the selected row carries the same highlight as runs. The panel body is the
former detail page; findings and runs opened inside it nest their own inspector,
which claims the keys from the one around it while open. Opening an investigation
and peeking at a finding now replace each other in the URL.

* fix(ui): run the Lens notch border along the tab pill and flag only a disconnected worker

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ui): show the shortcut hints in every inspector panel

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): extract the Lens dot field into composable DotFieldRoot and DotFieldCanvas

Move the dot grid model and canvas painter out of TracesTimeline into
components/lens/dotField so other Lens surfaces can reuse it. The root
owns layout and context; overlays compose as children. agoLabel moves to
lens/model/format and the unused columnTop helper is dropped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): drop the manual refresh button from the runs time controls

Live polls and range changes refetch, so the button only cleared the zoom, which Escape already does

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ui): extract the Lens run search into a composable SearchBox primitive

Move the query parser, glob matcher and autocomplete out of runSearch into
components/lens/search, generic over a QueryLanguage (field specs plus
what free text searches). SearchBox.Root owns the ProseMirror state, menu
and keyboard; SearchBox.Input and SearchBox.Suggestions compose under it.
Clause highlighting becomes a ProseMirror plugin built from the language.
RunSearch now only declares the run fields and composes the parts, so the
investigations tab can define its own language and reuse the same box.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ui): label the runs range by preset while Live and pin it once paused

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ui): split the Lens search language from where its data lives

QueryLanguage is now pure vocabulary (keys, groups, icons). Reading
fields off loaded items moves to a ClientIndex consumed by a separate
evaluator, and value suggestions come from an injectable ValueSource, so
a server-backed runs list can plug in a facet lookup while the
investigations tab keeps filtering in memory. The parsed query serializes
to a typed SearchQuery (text terms plus eq/neq/glob/nglob filters) that
the client evaluator consumes today and a server can consume later. The
suggestion menu shows a loading row while a source is still answering.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* style(ui): format the Lens SearchBox and its test with the project prettier config

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(ui): mirror the Lens run filters as trace SQL with a copyable curl in the search footer

The suggestions footer gains a slot, and SearchBox.ApiHint fills it with the
API equivalent of the typed query: a dialect chip, a one-line preview and a
Copy as curl button. The runs box translates each filter to a predicate over
the agent_traces_by_key rollup, bounded to the range the list shows, and
copies a POST to /v1/traces/query. Any other list can plug its own translate
into the same part.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(ui): show the Lens introduction as a first-visit dialog with a typed don't-show-again

The guided setup no longer replaces the Lens tabs. It opens in a dialog on
the first visit of a session or from ?setup=lens, with a close and a
"Don't show this again" checkbox in its top-right corner. The header
"Set up Lens" button is gone. Dismissal state lives in a new schema-validated
web storage helper (src/lib/storage.ts) that reads through
useSyncExternalStore, so server renders see the fallback and other tabs stay
in sync; only Lens uses it for now.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): keep only Copy as curl in the Lens run search footer

Drop the SQL chip and predicate preview; SearchBox.ApiHint becomes
SearchBox.CopyCommand, which takes the command for the current query.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(ui): align the Lens investigations list with the traces list

Use the shared query SearchBox with investigation fields (name, agent, status, schedule), match the traces toolbar, and drop the count footer and inner padding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ui): let Lens settings bring back the introduction after don't show again

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ui): share one InspectorTable between the traces list and the Lens investigations tree

Compose TanStack Table, react-virtual and the shadcn table cells into InspectorTable parts (Root, Grid, Header, Body, Row, Indent). Investigations get findings as real sub-rows with TanStack expansion instead of a hand-rolled flattener.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ui): break Lens import cycles and move shared pieces out of lens

Search and the dot field go to components/shared, run search and the preview
button go to view_logs where they are consumed. Lens api, services and demo
live under data/, all URL state in route.ts, storage keys in storage.ts, and
the session frame styles become cva variants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ui): one Lens readiness source and one onboarding flow

Readiness is computed once in model/readiness and read through
useLensReadiness, replacing useLensSetup, status.readiness and the welcome
screen's own checks. The Investigations welcome now renders the same
onboarding steps as the introduction dialog, with permissions and actions
coming from an OnboardingProvider instead of props passed down four levels.
StepIndicator and StateMessage are shared lens components, and the step
panels are labelled accordion regions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ui): read the Lens token from services and use semantic status colors

Lens services carry the access token they were built for, so trace evidence,
readiness and onboarding read it from context instead of a prop threaded
through six components. List and history invalidation lives in one data
hook. Status colors use the success, warning and destructive tokens, and
template-literal class names go through cn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ui): split Lens demo fixtures from the fake demo APIs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ui): show Lens check history as a dot timeline

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chores

* fix(ui): clear stale Lens evidence on run change and keep read-only users off Settings

Also names inline option objects to bring local/no-large-inline-object-arg back under budget.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
devin-ai-integration[bot] 2026-10-04 07:20:14 +00:00 • committed by GitHub
parent 984b4134be
commit 9a5e828310
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
203 changed files with 9750 additions and 6343 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
ui/litellm-dashboard/package-lock.json generated
View file

@ -42,6 +42,7 @@
"react-dom": "19.2.8",
"react-error-boundary": "6.1.6",
"react-hook-form": "7.82.0",
"react-hotkeys-hook": "5.3.3",
"react-intersection-observer": "11.0.1",
"react-json-view-lite": "2.5.0",
"react-markdown": "9.1.0",
@ -10537,6 +10538,19 @@
"react": "^16.8.0 || ^17 || ^18 || ^19"
}
},
"node_modules/react-hotkeys-hook": {
"version": "5.3.3",
"resolved": "https://registry.npmjs.org/react-hotkeys-hook/-/react-hotkeys-hook-5.3.3.tgz",
"integrity": "sha512-aswgyWUnE25hmhzHTfKDmKzsaSE5DJ4LKaU/o6rQSXkDd/1Bh9TfAFQbHkf6fLy11HvlYkp+cDDarGdhmCDhoQ==",
"license": "MIT",
"workspaces": [
"packages/*"
],
"peerDependencies": {
"react": ">=16.8.0",
"react-dom": ">=16.8.0"
}
},
"node_modules/react-intersection-observer": {
"version": "11.0.1",
"resolved": "https://registry.npmjs.org/react-intersection-observer/-/react-intersection-observer-11.0.1.tgz",

1
ui/litellm-dashboard/package.json
View file

@ -59,6 +59,7 @@
"react-dom": "19.2.8",
"react-error-boundary": "6.1.6",
"react-hook-form": "7.82.0",
"react-hotkeys-hook": "5.3.3",
"react-intersection-observer": "11.0.1",
"react-json-view-lite": "2.5.0",
"react-markdown": "9.1.0",

5
ui/litellm-dashboard/src/app/layout.tsx
View file

@ -7,6 +7,7 @@ import { ThemeProvider } from "next-themes";
import { AuthProvider } from "@/contexts/AuthContext";
import ReactQueryProvider from "@/contexts/ReactQueryProvider";
import HotkeysProvider from "@/contexts/HotkeysProvider";
import { Toaster } from "@/components/ui/sonner";
const inter = Inter({ subsets: ["latin"] });
@ -30,7 +31,9 @@ export default function RootLayout({
<ThemeProvider attribute="class" defaultTheme="light" enableSystem disableTransitionOnChange>
<NuqsAdapter>
<ReactQueryProvider>
<AuthProvider>{children}</AuthProvider>
<HotkeysProvider>
<AuthProvider>{children}</AuthProvider>
</HotkeysProvider>
<Toaster />
</ReactQueryProvider>
</NuqsAdapter>

13
ui/litellm-dashboard/src/components/lens/HeaderActions.tsx
View file

@ -1,13 +0,0 @@
"use client";
import { useContext, type ReactNode } from "react";
import { createPortal } from "react-dom";
import { LensPreviewContext } from "./LensPreviewButton";
export function HeaderActions({ children }: { children: ReactNode }) {
const preview = useContext(LensPreviewContext);
const actions = <div className="flex flex-wrap items-center gap-3">{children}</div>;
if (preview === undefined) return actions;
return preview.target ? createPortal(actions, preview.target) : null;
}

101
ui/litellm-dashboard/src/components/lens/LensModeSwitch.tsx Normal file
View file

@ -0,0 +1,101 @@
"use client";
import { Tabs as TabsPrimitive } from "@base-ui/react/tabs";
import { Activity, ScanSearch, Settings } from "lucide-react";
import { StatusDot } from "@/components/shared/StatusDot";
import { cn } from "@/lib/cva.config";
import type { InvestigationActivity } from "./model/status";
import { useWorkerConnected } from "./hooks/useWorkerConnected";
import type { LensList } from "./model/types";
import { LENS_TABS, type LensTab } from "./route";
import { frameCorner, frameTab } from "./ui/frame";
const MODE_ICONS = { traces: Activity, investigations: ScanSearch, settings: Settings } as const;
const ACTIVITY_DOT: Record<Exclude<InvestigationActivity, "idle">, { className: string; label: string }> = {
running: { className: "bg-info motion-safe:animate-pulse", label: "An investigation is running" },
queued: { className: "bg-muted-foreground/60", label: "An investigation is queued" },
};
function ActivityDot({ activity }: { activity: InvestigationActivity }) {
if (activity === "idle") return null;
return (
<span
aria-hidden="true"
className={cn(
"size-1.5 rounded-full animate-in fade-in-0 zoom-in-50 duration-300",
ACTIVITY_DOT[activity].className,
)}
/>
);
}
/** Inverted corner joining the tab's side border to the card's top border; plain CSS borders so both snap to the same pixels. */
function NotchCorner({ side, demo }: { side: "left" | "right"; demo: boolean }) {
return (
<span
aria-hidden="true"
className={cn(
"pointer-events-none absolute bottom-0 size-3 overflow-hidden",
side === "left" ? "-left-3" : "-right-3",
)}
>
<span className={frameCorner({ session: demo ? "demo" : "live", side })} />
</span>
);
}
export function LensModeSwitch({
activity,
demo,
workers,
setup,
}: {
activity: InvestigationActivity;
demo: boolean;
workers: LensList["workers"] | null;
setup?: string;
}) {
const connected = useWorkerConnected(workers);
const settingsTitle = connected ? "Worker connected" : "Connect worker";
const tabs = Object.entries(LENS_TABS).filter(([view]) => view !== "settings" || workers);
return (
<div className={frameTab({ session: demo ? "demo" : "live" })}>
<NotchCorner side="left" demo={demo} />
<NotchCorner side="right" demo={demo} />
<TabsPrimitive.List aria-label="Lens" className="relative inline-flex h-9 items-center p-1">
<TabsPrimitive.Indicator className="absolute top-1 bottom-1 left-(--active-tab-left) w-(--active-tab-width) rounded-full bg-muted transition-[left,width] duration-300 ease-[cubic-bezier(0.32,0.72,0,1)] motion-reduce:transition-none" />
{tabs.map(([view, label]) => {
const Icon = MODE_ICONS[view as LensTab];
const workerDisconnected = view === "settings" && workers !== null && !connected;
return (
<TabsPrimitive.Tab
key={view}
value={view}
title={view === "settings" ? settingsTitle : undefined}
aria-description={
view === "investigations" && activity !== "idle" ? ACTIVITY_DOT[activity].label : undefined
}
className={cn(
"relative z-raised inline-flex h-full items-center gap-2 rounded-full text-sm font-medium text-muted-foreground outline-none transition-colors duration-200 hover:text-foreground focus-visible:ring-2 focus-visible:ring-ring/50 data-active:text-foreground",
view === "settings" ? "px-2.5" : "px-4",
)}
>
<span className="relative inline-flex">
<Icon aria-hidden="true" className="size-4" />
{workerDisconnected && <StatusDot state="error" className="absolute -top-0.5 -right-0.5 size-1.5" />}
</span>
<span className={cn(view === "settings" && "sr-only")}>{label}</span>
{view === "investigations" && setup && (
<span className="rounded-full bg-muted px-1.5 py-0.5 text-xs leading-none font-medium text-muted-foreground animate-in fade-in-0 duration-200">
{setup}
</span>
)}
{view === "investigations" && <ActivityDot activity={activity} />}
</TabsPrimitive.Tab>
);
})}
</TabsPrimitive.List>
</div>
);
}

4
ui/litellm-dashboard/src/components/lens/LensNavigation.integration.test.tsx → ui/litellm-dashboard/src/components/lens/LensPage.integration.test.tsx
View file

@ -2,6 +2,7 @@ import { screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { renderWithProviders, testQueryClient } from "@/../tests/test-utils";
import { dismissLensIntro } from "@/../tests/lens-test-utils";
import LensPage from "@/app/(dashboard)/lens/page";
const { auth } = vi.hoisted(() => ({ auth: vi.fn() }));
@ -18,6 +19,9 @@ vi.mock("./investigations/InvestigationsView", () => ({
describe("Lens navigation", () => {
beforeEach(() => {
testQueryClient.clear();
window.localStorage.clear();
window.sessionStorage.clear();
dismissLensIntro();
auth.mockReturnValue({ accessToken: "test-token", userRole: "Admin", isViewOnly: false });
vi.stubGlobal(
"fetch",

13
ui/litellm-dashboard/src/components/lens/LensServicesProvider.tsx
View file

@ -1,13 +0,0 @@
"use client";
import type { ReactNode } from "react";
import { TracesApiContext } from "@/components/view_logs/TraceView/tracesApi";
import { LensApiContext, type LensServices } from "./services";
export function LensServicesProvider({ services, children }: { services: LensServices; children: ReactNode }) {
return (
<LensApiContext.Provider value={services.lens}>
<TracesApiContext.Provider value={services.traces}>{children}</TracesApiContext.Provider>
</LensApiContext.Provider>
);
}

266
ui/litellm-dashboard/src/components/lens/LensSetup.integration.test.tsx
View file

@ -2,8 +2,11 @@ import { act, fireEvent, screen, within, waitFor } from "@testing-library/react"
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { renderWithProviders, testQueryClient } from "@/../tests/test-utils";
import { dismissLensIntro } from "@/../tests/lens-test-utils";
import { readStorage } from "@/lib/storage";
import { LENS_INTRO_DISMISSED, LENS_INTRO_SEEN } from "./storage";
import { LensWorkspace } from "./LensWorkspace";
import { createLensDemoData } from "./demo/createLensDemo";
import { createLensDemoData } from "./data/demo/fixtures";
import type { LensList } from "./model/types";
const network = vi.fn<typeof fetch>();
@ -50,6 +53,8 @@ function serve({ enabled = false, traces = false, requests = false, connected =
beforeEach(() => {
testQueryClient.clear();
window.localStorage.clear();
window.sessionStorage.clear();
network.mockReset();
list.mockReset();
vi.stubGlobal("fetch", network);
@ -57,10 +62,95 @@ beforeEach(() => {
serve();
});
const renderWorkspace = (options?: Parameters<typeof renderWithProviders>[1], userRole = "Admin") =>
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole={userRole} readOnly={false} />, options);
const setupParam = (onUrlUpdate: ReturnType<typeof vi.fn>) =>
new URLSearchParams(String(onUrlUpdate.mock.lastCall?.[0].queryString ?? "")).get("setup");
async function connectWorkerFromSettings(user: ReturnType<typeof userEvent.setup>) {
const settings = within(await screen.findByRole("region", { name: "Settings" }));
expect(settings.getByRole("heading", { name: "Connect a worker" })).toBeVisible();
await user.click(settings.getByRole("combobox", { name: "Analysis model" }));
await user.click(await screen.findByRole("option", { name: "analysis" }));
await user.click(settings.getByRole("button", { name: "Get install command" }));
expect(await settings.findByRole("heading", { name: "Worker connected" })).toBeVisible();
await user.click(settings.getByRole("button", { name: "New investigation" }));
expect(await screen.findByRole("region", { name: "New investigation" })).toBeVisible();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
}
describe("Lens introduction", () => {
it("opens on the first visit, rests for the session on close, and stays away after don't show again", async () => {
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
const first = renderWorkspace({ onUrlUpdate });
const intro = within(await screen.findByRole("dialog"));
expect(await intro.findByRole("heading", { name: "The gateway that helps your agents improve" })).toBeVisible();
expect(intro.getByRole("heading", { name: "Before you start" })).toBeVisible();
await user.click(intro.getByRole("button", { name: "Close" }));
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
expect(readStorage(LENS_INTRO_SEEN)).toBe(true);
expect(readStorage(LENS_INTRO_DISMISSED)).toBe(false);
first.unmount();
const second = renderWorkspace();
expect(await screen.findByRole("tablist", { name: "Lens" })).toBeVisible();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
second.unmount();
window.sessionStorage.clear();
const third = renderWorkspace();
const reopened = within(await screen.findByRole("dialog"));
await user.click(reopened.getByRole("checkbox", { name: "Don’t show this again" }));
await user.click(reopened.getByRole("button", { name: "Close" }));
expect(readStorage(LENS_INTRO_DISMISSED)).toBe(true);
third.unmount();
window.sessionStorage.clear();
const fourth = renderWorkspace();
expect(await screen.findByRole("tablist", { name: "Lens" })).toBeVisible();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
fourth.unmount();
renderWorkspace({ searchParams: "?setup=lens" });
expect(await screen.findByRole("dialog")).toBeVisible();
});
it("lets settings bring back an introduction hidden with don't show again", async () => {
const user = userEvent.setup();
const first = renderWorkspace();
const intro = within(await screen.findByRole("dialog"));
await user.click(intro.getByRole("checkbox", { name: "Don’t show this again" }));
await user.click(intro.getByRole("button", { name: "Close" }));
first.unmount();
const settings = renderWorkspace({ searchParams: "?tab=settings" });
const toggle = within(await screen.findByRole("region", { name: "Introduction" })).getByRole("switch", {
name: "Show the introduction on each new session",
});
expect(toggle).not.toBeChecked();
await user.click(toggle);
expect(toggle).toBeChecked();
settings.unmount();
window.sessionStorage.clear();
renderWorkspace();
expect(await screen.findByRole("dialog")).toBeVisible();
});
it("never opens on its own inside the sample session", async () => {
renderWorkspace({ searchParams: "?demo=true" });
expect(await screen.findByText("Where is order #1042?")).toBeVisible();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
});
});
describe("Lens setup journey", () => {
it.each(["/lens", "/lens/activity/available"])(
"keeps recorded traces visible while %s is pending",
async (pendingPath) => {
dismissLensIntro();
serve({ enabled: true, traces: true });
const normal = network.getMockImplementation()!;
network.mockImplementation((input, init) =>
@ -68,7 +158,7 @@ describe("Lens setup journey", () => {
? new Promise<Response>(() => {})
: normal(input, init),
);
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole="Admin" readOnly={false} />);
renderWorkspace();
expect(await screen.findByRole("table", { name: "Agent runs" })).toBeVisible();
},
);
@ -76,6 +166,7 @@ describe("Lens setup journey", () => {
it.each(["/v1/traces", "/lens/activity/available"])(
"opens a saved investigation while %s is pending",
async (pendingPath) => {
dismissLensIntro();
serve();
list.mockResolvedValue({ lenses: data.lenses, workers: [worker()], tracing_enabled: false });
const normal = network.getMockImplementation()!;
@ -84,116 +175,97 @@ describe("Lens setup journey", () => {
? new Promise<Response>(() => {})
: normal(input, init),
);
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole="Admin" readOnly={false} />, {
searchParams: `?lens=${data.lenses[0].id}`,
});
renderWorkspace({ searchParams: `?lens=${data.lenses[0].id}` });
expect(await screen.findByRole("heading", { name: data.lenses[0].settings.name })).toBeVisible();
},
);
it("shares the introduction across tabs and stays in setup after the first trace", async () => {
it("walks a first visit from the introduction through tracing into the Settings tab and the first investigation", async () => {
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole="Admin" readOnly={false} />, { onUrlUpdate });
expect(await screen.findByRole("heading", { name: "Before you start" })).toBeVisible();
await user.click(screen.getByRole("tab", { name: "Investigations" }));
expect(screen.getByRole("heading", { name: "Before you start" })).toBeVisible();
await user.click(screen.getByRole("button", { name: "Set up Lens" }));
await waitFor(() => expect(onUrlUpdate.mock.lastCall?.[0].searchParams.get("setup")).toBe("lens"));
renderWorkspace({ onUrlUpdate });
const intro = within(await screen.findByRole("dialog"));
await user.click(await intro.findByRole("button", { name: "Set up Lens" }));
await waitFor(() => expect(setupParam(onUrlUpdate)).toBe("lens"));
serve({ enabled: true });
await user.click(screen.getByRole("button", { name: "Check setup" }));
expect(await screen.findByText("Trace storage is connected")).toBeVisible();
await user.click(screen.getByRole("button", { name: "Continue to your agent" }));
expect(screen.getByRole("button", { name: "Check for traces" })).toBeVisible();
await user.click(intro.getByRole("button", { name: "Check setup" }));
expect(await intro.findByText("Trace storage is connected")).toBeVisible();
await user.click(intro.getByRole("button", { name: "Continue to your agent" }));
expect(intro.getByRole("button", { name: "Check for traces" })).toBeVisible();
serve({ enabled: true, traces: true });
await user.click(screen.getByRole("button", { name: "Check for traces" }));
expect(await screen.findByText(/Your first trace is ready/)).toBeVisible();
expect(screen.queryByRole("table", { name: "Agent runs" })).not.toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Continue to worker" }));
const connection = within(await screen.findByRole("dialog", { name: "Connect a worker" }));
await user.click(connection.getByRole("combobox", { name: "Analysis model" }));
await user.click(await screen.findByRole("option", { name: "analysis" }));
await user.click(connection.getByRole("button", { name: "Get install command" }));
const connected = within(await screen.findByRole("dialog", { name: "Worker connected" }));
await user.click(connected.getByRole("button", { name: "New investigation" }));
expect(await screen.findByRole("dialog", { name: "Which activity should we investigate?" })).toBeVisible();
expect(screen.getByRole("heading", { name: "Get Lens running", hidden: true })).toBeInTheDocument();
await user.click(intro.getByRole("button", { name: "Check for traces" }));
expect(await intro.findByText(/Your first trace is ready/)).toBeVisible();
expect(screen.getByRole("dialog")).toBeVisible();
await user.click(intro.getByRole("button", { name: "Continue to worker" }));
await waitFor(() => expect(setupParam(onUrlUpdate)).toBeNull());
await connectWorkerFromSettings(user);
});
it("resumes setup after a reload and leaves only when the user chooses traces", async () => {
it("resumes setup from the URL and leaves only when the user chooses traces", async () => {
serve({ enabled: true, traces: true });
const user = userEvent.setup();
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole="Admin" readOnly={false} />, {
searchParams: "?tab=investigations&setup=lens",
});
expect(await screen.findByRole("button", { name: "Connect worker" })).toBeVisible();
await user.click(screen.getByRole("tab", { name: "Traces" }));
expect(screen.getByRole("heading", { name: "Get Lens running" })).toBeVisible();
await user.click(screen.getByRole("button", { name: "View traces" }));
const onUrlUpdate = vi.fn();
renderWorkspace({ searchParams: "?tab=investigations&setup=lens", onUrlUpdate });
const intro = within(await screen.findByRole("dialog"));
expect(await intro.findByRole("button", { name: "Connect worker" })).toBeVisible();
expect(intro.getByRole("heading", { name: "Get Lens running" })).toBeVisible();
await user.click(intro.getByRole("button", { name: "View traces" }));
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
expect(await screen.findByRole("table", { name: "Agent runs" })).toBeVisible();
await waitFor(() => expect(setupParam(onUrlUpdate)).toBeNull());
await user.click(screen.getByRole("tab", { name: "Investigations" }));
expect(await screen.findByRole("heading", { name: "Run your first investigation" })).toBeVisible();
expect(screen.queryByRole("link", { name: "Set up traces" })).not.toBeInTheDocument();
expect(screen.queryByRole("heading", { name: "Get Lens running" })).not.toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Set up Lens" }));
expect(await screen.findByRole("heading", { name: "Get Lens running" })).toBeVisible();
expect(screen.getByRole("button", { name: "Connect worker" })).toBeVisible();
const guide = within(await screen.findByRole("region", { name: "Get Lens running" }));
expect(guide.getByRole("button", { name: /Connect a worker/ })).toHaveAttribute("aria-expanded", "true");
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
});
it("allows request-only investigations without forcing agent instrumentation", async () => {
dismissLensIntro();
serve({ requests: true });
const user = userEvent.setup();
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole="Admin" readOnly={false} />, {
searchParams: "?tab=investigations",
});
expect(await screen.findByText("Request logs received")).toBeVisible();
const welcome = renderWorkspace({ searchParams: "?tab=investigations" });
expect(await screen.findByRole("region", { name: "Get Lens running" })).toBeVisible();
expect(screen.getByRole("button", { name: "Connect worker" })).toBeEnabled();
expect(screen.queryByRole("heading", { name: "Before you start" })).not.toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Set up Lens" }));
expect(await screen.findByRole("heading", { name: "Before you start" })).toBeVisible();
expect(screen.getByRole("button", { name: "Connect worker" })).toBeEnabled();
await user.click(screen.getByRole("button", { name: /Enable tracing on the gateway/ }));
expect(screen.getByRole("button", { name: "Continue with request logs" })).toBeEnabled();
await user.click(screen.getByRole("button", { name: /Send your first trace/ }));
await user.click(screen.getByRole("button", { name: "Continue with request logs" }));
const connection = within(await screen.findByRole("dialog", { name: "Connect a worker" }));
await user.click(connection.getByRole("combobox", { name: "Analysis model" }));
await user.click(await screen.findByRole("option", { name: "analysis" }));
await user.click(connection.getByRole("button", { name: "Get install command" }));
const connected = within(await screen.findByRole("dialog", { name: "Worker connected" }));
await user.click(connected.getByRole("button", { name: "New investigation" }));
expect(await screen.findByRole("dialog", { name: "Which activity should we investigate?" })).toBeVisible();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
welcome.unmount();
renderWorkspace({ searchParams: "?tab=investigations&setup=lens" });
const intro = within(await screen.findByRole("dialog"));
expect(await intro.findByRole("heading", { name: "Before you start" })).toBeVisible();
expect(intro.getByRole("button", { name: "Connect worker" })).toBeEnabled();
await user.click(intro.getByRole("button", { name: /Enable tracing on the gateway/ }));
expect(intro.getByRole("button", { name: "Continue with request logs" })).toBeEnabled();
await user.click(intro.getByRole("button", { name: /Send your first trace/ }));
await user.click(intro.getByRole("button", { name: "Continue with request logs" }));
await connectWorkerFromSettings(user);
});
it("keeps setup recoverable when checking for a first trace fails", async () => {
serve({ enabled: true });
const user = userEvent.setup();
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole="Admin" readOnly={false} />, {
searchParams: "?setup=lens",
});
await screen.findByRole("button", { name: "Check for traces" });
renderWorkspace({ searchParams: "?setup=lens" });
const intro = within(await screen.findByRole("dialog"));
await intro.findByRole("button", { name: "Check for traces" });
const normal = network.getMockImplementation()!;
network.mockImplementation(async (input, init) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/v1/traces") return Response.json({ detail: "Trace storage unavailable" }, { status: 503 });
return normal(input, init);
});
await user.click(screen.getByRole("button", { name: "Check for traces" }));
expect(await screen.findByRole("alert")).toHaveTextContent("Could not check setup");
expect(screen.queryByRole("button", { name: "Continue to worker" })).not.toBeInTheDocument();
await user.click(intro.getByRole("button", { name: "Check for traces" }));
expect(await intro.findByRole("alert")).toHaveTextContent("Could not check setup");
expect(intro.queryByRole("button", { name: "Continue to worker" })).not.toBeInTheDocument();
serve({ enabled: true, traces: true });
await user.click(screen.getByRole("button", { name: "Retry" }));
expect(await screen.findByRole("button", { name: "Continue to worker" })).toBeEnabled();
expect(screen.queryByRole("alert")).not.toBeInTheDocument();
await user.click(intro.getByRole("button", { name: "Retry" }));
expect(await intro.findByRole("button", { name: "Continue to worker" })).toBeEnabled();
expect(intro.queryByRole("alert")).not.toBeInTheDocument();
});
it("keeps request-only users in investigations when an activity refresh fails", async () => {
dismissLensIntro();
serve({ requests: true, connected: true });
const user = userEvent.setup();
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole="Admin" readOnly={false} />, {
searchParams: "?tab=investigations",
});
expect(await screen.findByText("Request logs received")).toBeVisible();
renderWorkspace({ searchParams: "?tab=investigations" });
expect(await screen.findByRole("button", { name: "New investigation" })).toBeEnabled();
const normal = network.getMockImplementation()!;
network.mockImplementation((input, init) =>
new URL(String(input), "http://localhost").pathname === "/lens/activity/available"
@ -201,25 +273,21 @@ describe("Lens setup journey", () => {
: normal(input, init),
);
await act(() => testQueryClient.refetchQueries());
expect(await screen.findByRole("alert")).toHaveTextContent("Could not check recorded activity");
expect(screen.queryByRole("heading", { name: "Get Lens running" })).not.toBeInTheDocument();
expect(await screen.findByRole("alert")).toHaveTextContent("Could not check setup. Activity unavailable");
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
expect(screen.getByRole("button", { name: "New investigation" })).toBeDisabled();
network.mockImplementation(normal);
await user.click(screen.getByRole("button", { name: "Retry" }));
expect(await screen.findByText("Request logs received")).toBeVisible();
expect(screen.getByRole("button", { name: "New investigation" })).toBeEnabled();
await waitFor(() => expect(screen.getByRole("button", { name: "New investigation" })).toBeEnabled());
expect(screen.queryByRole("alert")).not.toBeInTheDocument();
});
it("keeps administrator-only setup unavailable to trace viewers", async () => {
serve({ enabled: true, traces: true });
const user = userEvent.setup();
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole="Internal User" readOnly={false} />, {
searchParams: "?setup=lens",
});
expect(await screen.findByText(/A gateway administrator can connect a worker/)).toBeVisible();
expect(screen.getByRole("button", { name: "Connect worker" })).toBeDisabled();
await user.click(screen.getByRole("tab", { name: "Investigations" }));
expect(screen.getByRole("button", { name: "Connect worker" })).toBeDisabled();
renderWorkspace({ searchParams: "?setup=lens" }, "Internal User");
const intro = within(await screen.findByRole("dialog"));
expect(await intro.findByText(/A gateway administrator can connect a worker/)).toBeVisible();
expect(intro.getByRole("button", { name: "Connect worker" })).toBeDisabled();
expect(network.mock.calls.some(([input]) => new URL(String(input), "http://localhost").pathname === "/lens")).toBe(
false,
);
@ -243,25 +311,23 @@ describe("Lens setup journey", () => {
});
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
renderWithProviders(<LensWorkspace accessToken="setup-token" userRole="Admin" readOnly={false} />, {
searchParams: "?setup=lens",
onUrlUpdate,
});
await user.click(await screen.findByRole("button", { name: "New investigation" }));
const dialog = within(screen.getByRole("dialog"));
fireEvent.change(dialog.getByRole("textbox", { name: "Investigation name" }), {
renderWorkspace({ searchParams: "?setup=lens", onUrlUpdate });
const intro = within(await screen.findByRole("dialog"));
await user.click(await intro.findByRole("button", { name: "New investigation" }));
const editor = within(await screen.findByRole("region", { name: "New investigation" }));
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
fireEvent.change(editor.getByRole("textbox", { name: "Investigation name" }), {
target: { value: "My first review" },
});
await user.click(dialog.getByRole("button", { name: "Continue" }));
await user.click(dialog.getByRole("button", { name: "Continue" }));
await waitFor(() => expect(dialog.getByRole("button", { name: "Run and monitor" })).toBeEnabled());
await user.click(dialog.getByRole("button", { name: "Run and monitor" }));
await user.click(editor.getByRole("button", { name: "Continue" }));
await user.click(editor.getByRole("button", { name: "Continue" }));
await waitFor(() => expect(editor.getByRole("button", { name: "Run and monitor" })).toBeEnabled());
await user.click(editor.getByRole("button", { name: "Run and monitor" }));
expect(await screen.findByRole("heading", { name: "My first review" })).toBeVisible();
expect(screen.queryByRole("heading", { name: "Get Lens running" })).not.toBeInTheDocument();
expect(
within(screen.getByRole("tablist", { name: "Lens" })).getByRole("tab", { name: "Investigations" }),
).toHaveAttribute("aria-selected", "true");
await waitFor(() => expect(onUrlUpdate.mock.lastCall?.[0].searchParams.get("setup")).toBeNull());
await waitFor(() => expect(setupParam(onUrlUpdate)).toBeNull());
const create = network.mock.calls.find(
([input, init]) => new URL(String(input), "http://localhost").pathname === "/lens" && init?.method === "POST",
);

267
ui/litellm-dashboard/src/components/lens/LensWorkspace.integration.test.tsx
View file

@ -2,8 +2,10 @@ import { screen, waitFor, within } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { renderWithProviders, testQueryClient } from "@/../tests/test-utils";
import { dismissLensIntro } from "@/../tests/lens-test-utils";
import { LensWorkspace } from "./LensWorkspace";
import { createLensDemoData } from "./demo/createLensDemo";
import { lensKeys } from "./data/queries";
import { createLensDemoData } from "./data/demo/fixtures";
const lastUrl = (onUrlUpdate: ReturnType<typeof vi.fn>) =>
new URLSearchParams(String(onUrlUpdate.mock.lastCall?.[0].queryString ?? ""));
@ -13,6 +15,9 @@ const expectUrl = (onUrlUpdate: ReturnType<typeof vi.fn>, check: (params: URLSea
const network = vi.fn<typeof fetch>();
beforeEach(() => {
testQueryClient.clear();
window.localStorage.clear();
window.sessionStorage.clear();
dismissLensIntro();
vi.stubGlobal("fetch", network);
network.mockReset();
network.mockImplementation(async (input) => {
@ -27,6 +32,7 @@ describe("Lens interactive demo", () => {
it("opens without tracing, keeps the sample session in the URL, and restores the live view without mixing data", async () => {
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
window.localStorage.clear();
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Internal User" readOnly={false} />, {
onUrlUpdate,
});
@ -38,7 +44,6 @@ describe("Lens interactive demo", () => {
await expectUrl(onUrlUpdate, (url) => expect(url.get("demo")).toBe("true"));
expect(screen.queryByRole("button", { name: "Set up tracing" })).not.toBeInTheDocument();
network.mockClear();
await user.click(screen.getByRole("button", { name: "Refresh" }));
expect(screen.getByText("Where is order #1042?")).toBeVisible();
const search = screen.getByRole("combobox", { name: "Search runs" });
await user.type(search, "headphones");
@ -85,7 +90,8 @@ describe("Lens interactive demo", () => {
await expectUrl(onUrlUpdate, (url) => expect(url.get("view")).toBe("conversation"));
expect(network).not.toHaveBeenCalled();
await user.click(screen.getByRole("switch", { name: "Demo data" }));
expect(await screen.findByRole("heading", { name: "The gateway that helps your agents improve" })).toBeVisible();
expect(await screen.findByRole("heading", { name: "Enable tracing" })).toBeVisible();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
await expectUrl(onUrlUpdate, (url) => expect([...url.keys()]).toEqual([]));
});
@ -97,7 +103,7 @@ describe("Lens interactive demo", () => {
searchParams: "?tab=traces&setup=lens&trace=live-trace&span=live-span&lens=live-lens",
onUrlUpdate,
});
await user.click(await screen.findByRole("switch", { name: "Demo data" }));
await user.click(await screen.findByRole("button", { name: "Explore with sample data" }));
expect(await screen.findByText("Where is order #1042?")).toBeVisible();
await expectUrl(onUrlUpdate, (url) =>
expect([...url.entries()]).toEqual([
@ -126,6 +132,7 @@ describe("Lens interactive demo", () => {
it("connects findings and history to their original trace without live requests", async () => {
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
window.localStorage.clear();
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />, {
searchParams: "?tab=investigations",
onUrlUpdate,
@ -133,7 +140,7 @@ describe("Lens interactive demo", () => {
await user.click(await screen.findByRole("button", { name: "Explore with sample data" }));
network.mockClear();
await user.click(await screen.findByRole("row", { name: /Repeated lookups leave customers without an answer/ }));
const finding = screen.getByRole("dialog");
const finding = screen.getByRole("complementary", { name: "Finding details" });
expect(within(finding).getByText(/The support agent retries/)).toBeVisible();
const summaries = within(finding).getAllByText("support_agent", { exact: true });
await user.click(summaries[0]);
@ -146,16 +153,19 @@ describe("Lens interactive demo", () => {
expect(await navigator.clipboard.readText()).not.toContain("Authorization");
await user.click(screen.getByRole("tab", { name: "Attributes" }));
expect(await screen.findByText("gen_ai.agent.name")).toBeVisible();
await user.click(
within(screen.getByRole("dialog", { name: "Original run" })).getByRole("button", { name: "Close" }),
);
await user.click(within(screen.getByRole("dialog")).getByRole("button", { name: "Close" }));
expect(within(finding).getByRole("button", { name: "Back to finding" })).toBeVisible();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
await user.click(within(finding).getByRole("button", { name: "Back to finding" }));
expect(within(finding).getByText(/The support agent retries/)).toBeVisible();
await user.click(within(finding).getByRole("button", { name: "Close finding (Esc)" }));
expect(await screen.findByRole("table", { name: "Investigations" })).toBeVisible();
expect(network).not.toHaveBeenCalled();
await expectUrl(onUrlUpdate, (url) => expect(url.get("demo")).toBe("true"));
await expectUrl(onUrlUpdate, (url) => expect(url.has("span")).toBe(false));
await user.click(screen.getByRole("switch", { name: "Demo data" }));
expect(await screen.findByRole("heading", { name: "The gateway that helps your agents improve" })).toBeVisible();
await expectUrl(onUrlUpdate, (url) => expect(url.get("demo")).toBeNull());
expect(screen.getByRole("switch", { name: "Demo data" })).not.toBeChecked();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
});
it("has no demo entry for existing investigations, populated traces, or connecting another agent", async () => {
@ -180,7 +190,7 @@ describe("Lens interactive demo", () => {
expect(screen.queryByRole("button", { name: "Set up tracing" })).not.toBeInTheDocument();
});
it("offers sample data in the main panel only for the active tab that needs setup", async () => {
it("shows the header preview only for the active tab that still needs setup", async () => {
const user = userEvent.setup();
const saved = createLensDemoData().lenses[0];
network.mockImplementation(async (input) => {
@ -191,14 +201,13 @@ describe("Lens interactive demo", () => {
return Response.json({ data: [], traces: false, requests: false });
});
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />);
expect(await screen.findByRole("button", { name: "Explore with sample data" })).toBeVisible();
expect(await screen.findByRole("button", { name: "Preview sample" })).toBeVisible();
const tabs = within(screen.getByRole("tablist", { name: "Lens" }));
await user.click(tabs.getByRole("tab", { name: "Investigations" }));
expect(await screen.findByRole("row", { name: new RegExp(saved.settings.name) })).toBeVisible();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Explore with sample data" })).not.toBeInTheDocument();
await user.click(tabs.getByRole("tab", { name: "Traces" }));
await user.click(await screen.findByRole("button", { name: "Explore with sample data" }));
await user.click(await screen.findByRole("button", { name: "Preview sample" }));
expect(await screen.findByRole("table", { name: "Agent runs" })).toBeVisible();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
});
@ -220,7 +229,235 @@ describe("Lens interactive demo", () => {
const tab = within(screen.getByRole("tablist", { name: "Lens" })).getByRole("tab", { name: "Investigations" });
await waitFor(() => expect(tab).toHaveAccessibleDescription("An investigation is running"));
lenses.mockReturnValue([withJob("completed")]);
await testQueryClient.refetchQueries({ queryKey: ["lens", "list"] });
await testQueryClient.refetchQueries({ queryKey: lensKeys.lists() });
await waitFor(() => expect(tab).toHaveAccessibleDescription(""));
});
it("marks the Investigations tab while the inline editor is open and clears it on back", async () => {
const user = userEvent.setup();
const saved = createLensDemoData().lenses[0];
network.mockImplementation(async (input) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/lens") return Response.json({ lenses: [saved], workers: [], tracing_enabled: true });
if (path.endsWith("/runs")) return Response.json(saved.jobs);
if (path === "/lens/agents") return Response.json([]);
if (path.startsWith("/lens/preview")) return Response.json({ eligible: 0, selected: 0, executions: [] });
if (path === "/v1/traces") return Response.json({ detail: "Tracing is not enabled" }, { status: 501 });
return Response.json({ data: [], traces: true, requests: false });
});
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />, {
searchParams: "?tab=investigations&dialog=new",
});
const tabs = within(screen.getByRole("tablist", { name: "Lens" }));
expect(await screen.findByRole("region", { name: "New investigation" })).toBeVisible();
const tab = tabs.getByRole("tab", { name: /^Investigations/ });
expect(tab).toHaveAttribute("aria-selected", "true");
expect(within(tab).getByText("New")).toBeVisible();
await user.click(screen.getByRole("button", { name: "Back to investigations" }));
expect(await screen.findByRole("row", { name: new RegExp(saved.settings.name) })).toBeVisible();
expect(within(tab).queryByText("New")).not.toBeInTheDocument();
});
it("adds a quiet Settings tab that manages the worker inline and reflects its health", async () => {
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
const saved = createLensDemoData().lenses[0];
const worker = {
id: "worker",
name: "Worker",
revoked: false,
analysis_key_id: "a".repeat(64),
scope: saved.scope,
last_seen: new Date().toISOString(),
};
const workers = vi.fn(() => [worker]);
network.mockImplementation(async (input) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/lens") return Response.json({ lenses: [saved], workers: workers(), tracing_enabled: true });
if (path.endsWith("/runs")) return Response.json(saved.jobs);
if (path === "/v1/traces") return Response.json({ detail: "Tracing is not enabled" }, { status: 501 });
return Response.json({ data: [], traces: true, requests: false });
});
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />, { onUrlUpdate });
const tabs = within(screen.getByRole("tablist", { name: "Lens" }));
const settings = await tabs.findByRole("tab", { name: "Settings" });
expect(settings).toHaveAttribute("title", "Worker connected");
await user.click(settings);
await expectUrl(onUrlUpdate, (url) => expect(url.get("tab")).toBe("settings"));
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
const panel = within(screen.getByRole("region", { name: "Settings" }));
expect(panel.getByRole("status")).toHaveTextContent("Tracing enabled");
expect(panel.getByRole("heading", { name: "Analysis worker" })).toBeVisible();
expect(panel.getByRole("heading", { name: worker.name })).toBeVisible();
expect(panel.getByText("Connected")).toBeVisible();
await user.click(panel.getByRole("button", { name: "Edit access" }));
expect(panel.getByRole("heading", { name: "Analysis access" })).toBeVisible();
await user.click(panel.getByRole("button", { name: "Cancel" }));
expect(panel.getByRole("heading", { name: "Analysis worker" })).toBeVisible();
workers.mockReturnValue([{ ...worker, revoked: true }]);
await testQueryClient.refetchQueries({ queryKey: lensKeys.lists() });
await waitFor(() => expect(settings).toHaveAttribute("title", "Connect worker"));
expect(panel.getByRole("heading", { name: "Connect a worker" })).toBeVisible();
expect(panel.queryByRole("button", { name: "Cancel" })).not.toBeInTheDocument();
await user.click(panel.getByRole("button", { name: "Connect an agent" }));
await expectUrl(onUrlUpdate, (url) => expect(url.get("tab")).toBe("traces"));
expect(tabs.getByRole("tab", { name: "Traces" })).toHaveAttribute("aria-selected", "true");
});
it("sends the first-time guide's Connect worker into the Settings tab", async () => {
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
network.mockImplementation(async (input) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/lens") return Response.json({ lenses: [], workers: [], tracing_enabled: true });
if (path === "/v1/traces") return Response.json({ data: [{}] });
return Response.json({ data: [], traces: true, requests: false });
});
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />, {
searchParams: "?tab=investigations",
onUrlUpdate,
});
const guide = within(await screen.findByRole("region", { name: "Get Lens running" }));
await user.click(await guide.findByRole("button", { name: "Connect worker" }));
await expectUrl(onUrlUpdate, (url) => expect(url.get("tab")).toBe("settings"));
const panel = within(await screen.findByRole("region", { name: "Settings" }));
expect(panel.getByRole("heading", { name: "Connect a worker" })).toBeVisible();
expect(panel.getByRole("button", { name: "Get install command" })).toBeVisible();
});
it("keeps a pending worker install across tab switches and offers the first investigation once it connects", async () => {
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
const token = "b".repeat(64);
const worker = {
id: "worker",
name: "Lens worker",
revoked: false,
analysis_key_id: token,
scope: { all_teams: true, api_key_hash: "", team_id: "" },
last_seen: "1970-01-01T00:00:00Z",
};
const workers = vi.fn((): (typeof worker)[] => []);
network.mockImplementation(async (input, init) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/lens") return Response.json({ lenses: [], workers: workers(), tracing_enabled: true });
if (path === "/lens/workers/register" && init?.method === "POST") {
workers.mockReturnValue([worker]);
return Response.json({ token: "lens-test-token", image: "lens-worker:v1", worker });
}
if (path === "/key/list") return Response.json({ keys: [{ token, key_alias: "Analysis" }], total_pages: 1 });
if (path === "/key/info") return Response.json({ info: { models: [], max_budget: null } });
if (path === "/lens/agents") return Response.json([]);
if (path.startsWith("/lens/preview")) return Response.json({ eligible: 0, selected: 0, executions: [] });
if (path === "/v1/traces") return Response.json({ data: [createLensDemoData().runs[0].trace.summary] });
return Response.json({ data: [], traces: true, requests: false });
});
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />, {
searchParams: "?tab=settings",
onUrlUpdate,
});
const panel = within(await screen.findByRole("region", { name: "Settings" }));
await user.click(panel.getByText("Advanced options"));
await user.click(panel.getByRole("switch", { name: "Use an existing virtual key" }));
await user.click(panel.getByRole("combobox", { name: "Charge analysis to" }));
await user.click(await screen.findByRole("option", { name: "Analysis" }));
await user.click(panel.getByRole("button", { name: "Get install command" }));
expect(await panel.findByText("Waiting for your worker to connect…")).toBeInTheDocument();
const tabs = within(screen.getByRole("tablist", { name: "Lens" }));
await user.click(tabs.getByRole("tab", { name: "Traces" }));
await waitFor(() => expect(panel.getByText("Waiting for your worker to connect…")).not.toBeVisible());
await user.click(tabs.getByRole("tab", { name: "Settings" }));
expect(panel.getByText("Waiting for your worker to connect…")).toBeVisible();
expect(panel.getByLabelText("Docker command preview")).toHaveTextContent("LENS_WORKER_TOKEN=lens-test-token");
workers.mockReturnValue([{ ...worker, last_seen: new Date().toISOString() }]);
await testQueryClient.refetchQueries({ queryKey: lensKeys.lists() });
expect(await panel.findByRole("heading", { name: "Worker connected" })).toBeVisible();
await user.click(panel.getByRole("button", { name: "New investigation" }));
await expectUrl(onUrlUpdate, (url) => expect(url.get("tab")).toBe("investigations"));
expect(lastUrl(onUrlUpdate).get("dialog")).toBe("new");
expect(await screen.findByRole("region", { name: "New investigation" })).toBeVisible();
});
it("hides the Settings tab for read-only sessions", async () => {
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly />);
expect(await screen.findByRole("tablist", { name: "Lens" })).toBeVisible();
await waitFor(() => expect(network).toHaveBeenCalled());
expect(screen.queryByRole("tab", { name: "Settings" })).not.toBeInTheDocument();
});
it("sends read-only sessions following a Settings link to the default tab", async () => {
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly />, {
searchParams: "?tab=settings",
});
expect(await screen.findByRole("tab", { name: "Traces", selected: true })).toBeVisible();
});
it("turns the worker health dot off once heartbeats expire even when polling returns unchanged data", async () => {
vi.useFakeTimers({ shouldAdvanceTime: true });
try {
const saved = createLensDemoData().lenses[0];
const worker = {
id: "worker",
name: "Worker",
revoked: false,
analysis_key_id: "a".repeat(64),
scope: saved.scope,
last_seen: new Date().toISOString(),
};
network.mockImplementation(async (input) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/lens") return Response.json({ lenses: [saved], workers: [worker], tracing_enabled: true });
if (path === "/v1/traces") return Response.json({ detail: "Tracing is not enabled" }, { status: 501 });
return Response.json({ data: [], traces: true, requests: false });
});
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />);
const tabs = within(screen.getByRole("tablist", { name: "Lens" }));
const settings = await tabs.findByRole("tab", { name: "Settings" });
expect(settings).toHaveAttribute("title", "Worker connected");
await vi.advanceTimersByTimeAsync(130000);
await waitFor(() => expect(settings).toHaveAttribute("title", "Connect worker"));
} finally {
vi.useRealTimers();
}
});
it("polls /lens every 2s while Settings waits for a worker, then drops to the 10s cadence once it connects", async () => {
vi.useFakeTimers({ shouldAdvanceTime: true });
try {
const saved = createLensDemoData().lenses[0];
const worker = {
id: "worker",
name: "Worker",
revoked: false,
analysis_key_id: "a".repeat(64),
scope: saved.scope,
last_seen: new Date(Date.now() - 600_000).toISOString(),
};
const workers = vi.fn(() => [worker]);
const listCalls = () =>
network.mock.calls.filter(([input]) => new URL(String(input), "http://localhost").pathname === "/lens").length;
network.mockImplementation(async (input) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/lens") return Response.json({ lenses: [saved], workers: workers(), tracing_enabled: true });
if (path === "/v1/traces") return Response.json({ detail: "Tracing is not enabled" }, { status: 501 });
return Response.json({ data: [], traces: true, requests: false });
});
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />, {
searchParams: "?tab=settings",
});
expect(await screen.findByRole("region", { name: "Settings" })).toBeVisible();
const initial = listCalls();
await vi.advanceTimersByTimeAsync(2000);
await waitFor(() => expect(listCalls()).toBe(initial + 1));
workers.mockReturnValue([{ ...worker, last_seen: new Date().toISOString() }]);
await vi.advanceTimersByTimeAsync(2000);
await waitFor(() => expect(listCalls()).toBe(initial + 2));
await vi.advanceTimersByTimeAsync(2000);
expect(listCalls()).toBe(initial + 2);
await vi.advanceTimersByTimeAsync(8000);
await waitFor(() => expect(listCalls()).toBe(initial + 3));
} finally {
vi.useRealTimers();
}
});
});

388
ui/litellm-dashboard/src/components/lens/LensWorkspace.tsx
View file

@ -2,37 +2,49 @@
import { useId, useState } from "react";
import { useQuery } from "@tanstack/react-query";
import { Tabs as TabsPrimitive } from "@base-ui/react/tabs";
import { Activity, Aperture, ArrowUpRight, Loader2, ScanSearch } from "lucide-react";
import { Aperture, ArrowUpRight } from "lucide-react";
import AgentTracesPage from "@/components/view_logs/TraceView/AgentTracesPage";
import { Button } from "@/components/ui/button";
import type { TraceSummary } from "@/components/view_logs/TraceView/traceTypes";
import { Switch } from "@/components/ui/switch";
import { Tabs, TabsContent } from "@/components/ui/tabs";
import { LensServicesProvider } from "./LensServicesProvider";
import { LensPreviewContext } from "./LensPreviewButton";
import { LensServicesProvider, useLensAccessToken, useLensApi, useLiveLensServices } from "./data/LensServices";
import { LensPreviewContext } from "@/components/view_logs/TraceView/LensPreviewButton";
import { isProxyAdminRole, isProxyAdminTierRole } from "@/utils/roles";
import { InvestigationsView } from "./investigations/InvestigationsView";
import { createLensDemo } from "./demo/createLensDemo";
import { lensQueries } from "./api/queries";
import { useLensApi } from "./services";
import { investigationActivity, type InvestigationActivity } from "./model/status";
import { LensSettings } from "./settings/LensSettings";
import { createLensDemo } from "./data/demo/createLensDemo";
import { lensQueries } from "./data/queries";
import { LensModeSwitch } from "./LensModeSwitch";
import { frameCard } from "./ui/frame";
import { investigationActivity, listPollInterval } from "./model/status";
import { cn } from "@/lib/cva.config";
import { LENS_TABS, useLensRoute, type LensTab } from "./route";
import { Button } from "@/components/ui/button";
import { LensGettingStarted } from "./setup/LensGettingStarted";
import { useLensSetup, type LensSetupState } from "./setup/useLensSetup";
import { useDialogRoute, useLensRoute, type LensDialog, type LensTab } from "./route";
import { LensIntroDialog, useLensIntro } from "./onboarding/LensIntroDialog";
import { OnboardingProvider, type Onboarding } from "./onboarding/OnboardingContext";
import { traceRefOf, useOpenTraceRouting } from "@/components/view_logs/TraceView/traceRouting";
type WorkspaceProps = { accessToken: string; userRole: string; readOnly: boolean };
export function LensWorkspace(props: WorkspaceProps) {
const { demo } = useLensRoute();
return demo ? <SampleSession /> : <LensContent {...props} />;
return demo ? <SampleSession /> : <LiveSession {...props} />;
}
function LiveSession(props: WorkspaceProps) {
const services = useLiveLensServices(props.accessToken);
return (
<LensServicesProvider services={services}>
<LensContent userRole={props.userRole} readOnly={props.readOnly} />
</LensServicesProvider>
);
}
function SampleSession() {
const [services] = useState(() => createLensDemo());
return (
<LensServicesProvider services={services}>
<LensContent accessToken="lens-demo" userRole="proxy_admin_viewer" readOnly />
<LensContent userRole="proxy_admin_viewer" readOnly />
</LensServicesProvider>
);
}
@ -47,234 +59,164 @@ function DemoToggle({ demo, onChange }: { demo: boolean; onChange: (demo: boolea
);
}
const MODE_ICONS = { traces: Activity, investigations: ScanSearch } as const;
/** The inline investigation editor marks the tab so the notch says where you are, not just which tab is open. */
const SETUP_LABELS: Partial<Record<LensDialog, string>> = { new: "New", edit: "Editing", duplicate: "Duplicate" };
const ACTIVITY_DOT: Record<Exclude<InvestigationActivity, "idle">, { className: string; label: string }> = {
running: { className: "bg-info motion-safe:animate-pulse", label: "An investigation is running" },
queued: { className: "bg-muted-foreground/60", label: "An investigation is queued" },
};
function ActivityDot({ activity }: { activity: InvestigationActivity }) {
if (activity === "idle") return null;
return (
<span
aria-hidden="true"
className={cn(
"size-1.5 rounded-full animate-in fade-in-0 zoom-in-50 duration-300",
ACTIVITY_DOT[activity].className,
)}
/>
);
}
const FRAME = {
live: {
card: "border border-foreground/15",
tab: "-mb-px border-x border-t border-foreground/15",
corner: { left: "border-r border-b", right: "border-l border-b", tone: "border-foreground/15" },
},
demo: {
card: "border-2 border-info",
tab: "-mb-0.5 border-x-2 border-t-2 border-info",
corner: { left: "border-r-2 border-b-2", right: "border-l-2 border-b-2", tone: "border-info" },
},
} as const;
const frameOf = (demo: boolean) => FRAME[demo ? "demo" : "live"];
/** Inverted corner joining the tab's side border to the card's top border; plain CSS borders so both snap to the same pixels. */
function NotchCorner({ side, demo }: { side: "left" | "right"; demo: boolean }) {
const { corner } = frameOf(demo);
return (
<span
aria-hidden="true"
className={cn(
"pointer-events-none absolute bottom-0 size-3 overflow-hidden",
side === "left" ? "-left-3" : "-right-3",
)}
>
<span
className={cn(
"block size-full shadow-[0_0_0_12px_var(--card)]",
side === "left" ? "rounded-br-xl" : "rounded-bl-xl",
corner[side],
corner.tone,
)}
/>
</span>
);
}
function LensModeSwitch({ activity, demo }: { activity: InvestigationActivity; demo: boolean }) {
return (
<div className={cn("relative z-raised rounded-t-2xl bg-card px-1.5 pt-1.5 pb-[7px]", frameOf(demo).tab)}>
<NotchCorner side="left" demo={demo} />
<NotchCorner side="right" demo={demo} />
<TabsPrimitive.List
aria-label="Lens"
className="relative inline-flex h-9 items-center rounded-full bg-muted/70 p-1"
>
<TabsPrimitive.Indicator className="absolute top-1 bottom-1 left-(--active-tab-left) w-(--active-tab-width) rounded-full bg-background shadow-sm ring-1 ring-border transition-[left,width] duration-300 ease-[cubic-bezier(0.32,0.72,0,1)] motion-reduce:transition-none" />
{Object.entries(LENS_TABS).map(([view, label]) => {
const Icon = MODE_ICONS[view as LensTab];
return (
<TabsPrimitive.Tab
key={view}
value={view}
aria-description={
view === "investigations" && activity !== "idle" ? ACTIVITY_DOT[activity].label : undefined
}
className="relative z-raised inline-flex h-full items-center gap-2 rounded-full px-4 text-sm font-medium text-muted-foreground outline-none transition-colors duration-200 hover:text-foreground focus-visible:ring-2 focus-visible:ring-ring/50 data-active:text-foreground"
>
<Icon aria-hidden="true" className="size-4" />
{label}
{view === "investigations" && <ActivityDot activity={activity} />}
</TabsPrimitive.Tab>
);
})}
</TabsPrimitive.List>
</div>
);
}
function useInvestigationActivity(accessToken: string, enabled: boolean): InvestigationActivity {
const api = useLensApi(accessToken);
const { data } = useQuery({ ...lensQueries.list(api, false), enabled });
return investigationActivity(data?.lenses ?? []);
/** The one always-mounted `/lens` observer; every other reader is a plain cache subscriber. */
function useLensOverview(enabled: boolean, settingsOpen: boolean) {
const api = useLensApi();
const { data } = useQuery({
...lensQueries.list(api),
enabled,
refetchInterval: (query) => listPollInterval(query.state.data, settingsOpen, Date.now()),
});
return { activity: investigationActivity(data?.lenses ?? []), list: data };
}
const PANEL =
"flex min-h-0 flex-1 flex-col overflow-y-auto animate-in fade-in-0 duration-300 motion-reduce:animate-none";
function LensContent({ accessToken, userRole, readOnly }: WorkspaceProps) {
const { tab, lensId, demo, settingUp, setTab, setLensId, setDemo, setSetup } = useLensRoute();
function LensContent({ userRole, readOnly }: Omit<WorkspaceProps, "accessToken">) {
const accessToken = useLensAccessToken();
const { tab, lensId, demo, settingUp, setTab, setDemo, setSetup } = useLensRoute();
const { dialog, openDialog } = useDialogRoute();
const { openTrace } = useOpenTraceRouting();
const [previewTarget, setPreviewTarget] = useState<HTMLDivElement | null>(null);
const activeTab = tab ?? (lensId ? "investigations" : "traces");
const canInvestigate = isProxyAdminTierRole(userRole);
const setupState = useLensSetup(accessToken, !demo, canInvestigate, settingUp);
const setupLocation = { tab: activeTab, canInvestigate, selected: !!lensId, requested: settingUp };
const showSetup = !demo && needsSetup(setupState, setupLocation);
const showSetupButton = !demo && !showSetup && isProxyAdminRole(userRole);
const canViewInvestigations = isProxyAdminTierRole(userRole);
const isAdmin = isProxyAdminRole(userRole);
const canConfigure = canViewInvestigations && !readOnly;
const defaultTab = lensId ? "investigations" : "traces";
const activeTab = tab === "settings" && !canConfigure ? defaultTab : tab ?? defaultTab;
const intro = useLensIntro({ demo, settingUp });
const { activity, list } = useLensOverview(
canViewInvestigations,
(canConfigure && activeTab === "settings") || intro.open,
);
const workers = canConfigure && list ? list.workers : null;
const leaveIntro = (forever = false) => {
if (!intro.open) return;
intro.close(forever);
setSetup(false);
};
const startSetup = () => {
if (!settingUp) setSetup(true);
};
const showTraces = () => {
setSetup(false);
setTab(setupState.tracesReady ? "traces" : "investigations");
const exitIntro = (to: LensTab) => {
leaveIntro();
setTab(to);
};
const showCreated = (id: string) => {
setSetup(false);
setLensId(id);
const showSettings = () => {
leaveIntro();
setTab("settings");
};
const startFirstInvestigation = () => {
leaveIntro();
setTab("investigations");
openDialog("new");
};
const showSentTrace = (trace: TraceSummary) => {
leaveIntro();
setTab("traces");
openTrace(traceRefOf(trace));
};
const enterDemo = () => {
leaveIntro();
setDemo(true);
};
const onboarding: Onboarding = {
readOnly,
canViewInvestigations,
canInvestigate: isAdmin,
canMintTracingKey: isAdmin,
connect: showSettings,
create: startFirstInvestigation,
openTrace: showSentTrace,
};
const activity = useInvestigationActivity(accessToken, canInvestigate);
const preview = (view: LensTab) => ({
target: previewTarget,
open: !demo && activeTab === view ? () => setDemo(true) : undefined,
open: !demo && !intro.open && activeTab === view ? enterDemo : undefined,
});
const setupContent = setupState.loading ? (
<p role="status" className="flex items-center gap-2 py-8 text-sm text-muted-foreground">
<Loader2 aria-hidden="true" className="size-4 animate-spin" />
Checking Lens setup…
</p>
) : (
<TabsContent value={activeTab} keepMounted className={cn(PANEL, "p-4")}>
<LensGettingStarted
accessToken={accessToken}
state={setupState}
readOnly={readOnly}
canInvestigate={isProxyAdminRole(userRole)}
canMintTracingKey={isProxyAdminRole(userRole)}
onStart={startSetup}
onExit={showTraces}
onCreated={showCreated}
onDemo={() => setDemo(true)}
/>
</TabsContent>
);
return (
<main className="flex h-full w-full min-w-0 flex-1 flex-col px-4 pt-3 pb-4">
<Tabs value={activeTab} onValueChange={(value) => setTab(value as LensTab)} className="min-h-0 flex-1 gap-0">
<div className="grid grid-cols-[1fr_auto_1fr] items-end gap-3">
<div className="flex min-w-0 flex-col gap-1 pb-3">
<h1 className="flex items-center gap-2 text-lg font-semibold tracking-tight">
<Aperture aria-hidden="true" className="size-5" strokeWidth={2} />
Lens
</h1>
<p className="truncate text-xs text-muted-foreground">
Trace your agents and investigate what goes wrong.{" "}
<a
href="https://docs.litellm.ai/docs/proxy/lens"
target="_blank"
rel="noopener noreferrer"
className="inline-flex items-center gap-0.5 font-medium text-foreground/80 underline-offset-2 hover:text-foreground hover:underline"
>
Docs
<ArrowUpRight aria-hidden="true" className="size-3" />
</a>
</p>
<OnboardingProvider value={onboarding}>
<main className="flex h-full w-full min-w-0 flex-1 flex-col px-4 pt-3 pb-4">
<Tabs value={activeTab} onValueChange={(value) => setTab(value as LensTab)} className="min-h-0 flex-1 gap-0">
<div className="grid grid-cols-[1fr_auto_1fr] items-end gap-3">
<div className="flex min-w-0 flex-col gap-1 pb-3">
<h1 className="flex items-center gap-2 text-lg font-semibold tracking-tight">
<Aperture aria-hidden="true" className="size-5" strokeWidth={2} />
Lens
</h1>
<p className="truncate text-xs text-muted-foreground">
Trace your agents and investigate what goes wrong.{" "}
<a
href="https://docs.litellm.ai/docs/proxy/lens"
target="_blank"
rel="noopener noreferrer"
className="inline-flex items-center gap-0.5 font-medium text-foreground/80 underline-offset-2 hover:text-foreground hover:underline"
>
Docs
<ArrowUpRight aria-hidden="true" className="size-3" />
</a>
</p>
</div>
<LensModeSwitch
activity={activity}
demo={demo}
workers={workers}
setup={activeTab === "investigations" && dialog ? SETUP_LABELS[dialog] : undefined}
/>
<div className="flex min-w-0 flex-wrap items-center justify-end gap-3 pb-3">
<div ref={setPreviewTarget} />
<DemoToggle demo={demo} onChange={(next) => (next ? enterDemo() : setDemo(false))} />
</div>
</div>
<LensModeSwitch activity={activity} demo={demo} />
<div className="flex min-w-0 flex-wrap items-center justify-end gap-3 pb-3">
{showSetupButton && (
<Button variant="outline" size="sm" onClick={startSetup}>
Set up Lens
</Button>
<div className={frameCard({ session: demo ? "demo" : "live" })}>
<TabsContent value="traces" keepMounted className={PANEL}>
<LensPreviewContext.Provider value={preview("traces")}>
<AgentTracesPage
accessToken={accessToken}
isActive={activeTab === "traces"}
readOnly={readOnly}
canMintTracingKey={isAdmin}
/>
</LensPreviewContext.Provider>
</TabsContent>
<TabsContent value="investigations" className={PANEL}>
<LensPreviewContext.Provider value={preview("investigations")}>
{canViewInvestigations ? (
<InvestigationsView readOnly={readOnly || !isAdmin} />
) : (
<p className="py-6 text-sm text-muted-foreground">
Investigations require proxy administrator access. You can still view your traces.
</p>
)}
</LensPreviewContext.Provider>
</TabsContent>
{workers && list && (
<TabsContent value="settings" keepMounted className={cn(PANEL, "p-6")}>
<LensSettings
list={list}
workerReadyAction={
list.lenses.length === 0 ? (
<Button className="w-full" onClick={startFirstInvestigation}>
New investigation
</Button>
) : undefined
}
onOpenTraces={() => setTab("traces")}
/>
</TabsContent>
)}
<div ref={setPreviewTarget} />
<DemoToggle demo={demo} onChange={setDemo} />
</div>
</div>
<div className={cn("flex min-h-0 flex-1 flex-col overflow-hidden rounded-2xl bg-card", frameOf(demo).card)}>
{showSetup ? (
setupContent
) : (
<>
<TabsContent value="traces" keepMounted className={PANEL}>
<LensPreviewContext.Provider value={preview("traces")}>
<AgentTracesPage
accessToken={accessToken}
isActive={activeTab === "traces"}
readOnly={readOnly}
canMintTracingKey={isProxyAdminRole(userRole)}
/>
</LensPreviewContext.Provider>
</TabsContent>
<TabsContent value="investigations" className={cn(PANEL, "p-4")}>
<LensPreviewContext.Provider value={preview("investigations")}>
{canInvestigate ? (
<InvestigationsView
active={activeTab === "investigations"}
accessToken={accessToken}
readOnly={readOnly || !isProxyAdminRole(userRole)}
/>
) : (
<p className="py-6 text-sm text-muted-foreground">
Investigations require proxy administrator access. You can still view your traces.
</p>
)}
</LensPreviewContext.Provider>
</TabsContent>
</>
)}
</div>
</Tabs>
</main>
</Tabs>
<LensIntroDialog
open={intro.open}
onClose={leaveIntro}
onStart={startSetup}
onExit={exitIntro}
onDemo={enterDemo}
/>
</main>
</OnboardingProvider>
);
}
function needsSetup(
state: LensSetupState,
{
tab,
canInvestigate,
selected,
requested,
}: { tab: LensTab; canInvestigate: boolean; selected: boolean; requested: boolean },
) {
if (requested) return true;
if (!state.missingTraces) return false;
if (tab === "traces") return true;
const hasActivity = state.hasInvestigations || state.hasRequests || selected;
return canInvestigate && !hasActivity;
}

93
ui/litellm-dashboard/src/components/lens/api/mutations.ts
View file

@ -1,93 +0,0 @@
"use client";
import { validateWorkerAddress, analysisAccessSchema, type AnalysisAccess } from "../setup/worker/workerSchema";
import { useMutation } from "@tanstack/react-query";
import type { Lens, Settings, WorkerCreated } from "../model/types";
import type { LensApi } from "./service";
import { useLensApi } from "../services";
export type LensWrite = (api: LensApi) => Promise<unknown>;
export function useLensUpdate(accessToken: string) {
const api = useLensApi(accessToken);
return useMutation({ retry: false, mutationFn: (write: LensWrite) => write(api) });
}
export function useSaveLens(accessToken: string) {
const api = useLensApi(accessToken);
return useMutation({
retry: false,
mutationFn: ({ id, settings }: { id?: string; settings: Settings }): Promise<Lens> => api.saveLens(id, settings),
});
}
export function useRevokeWorker(accessToken: string) {
const api = useLensApi(accessToken);
return useMutation({ retry: false, mutationFn: (id: string) => api.revokeWorker(id) });
}
export async function createAnalysisKey(api: LensApi, access: AnalysisAccess): Promise<string> {
const parsed = analysisAccessSchema.safeParse(access);
if (!parsed.success) throw new Error(parsed.error.issues[0].message);
const result = await api.generateAnalysisKey(parsed.data);
if (!result.token_id) throw new Error("The proxy did not return the new key's ID");
return result.token_id;
}
export function usePrepareWorker(
accessToken: string,
{ onChanged, onPrepared }: { onChanged: () => void; onPrepared: (created: WorkerCreated | null) => void },
) {
const api = useLensApi(accessToken);
return useMutation({
retry: false,
mutationFn: async ({
address,
useExisting,
analysisKey,
access,
editingWorker,
}: {
address: string;
useExisting: boolean;
analysisKey: string | null;
access: AnalysisAccess;
editingWorker: string | null;
}) => {
let newKey: string | null = null;
try {
validateWorkerAddress(address);
const keyId = useExisting ? analysisKey : await createAnalysisKey(api, access);
if (!useExisting) newKey = keyId;
if (editingWorker) {
await api.setWorkerBillingKey(editingWorker, keyId);
onPrepared(null);
onChanged();
return null;
}
const created = await api.registerWorker(keyId);
onPrepared(created);
onChanged();
return created;
} catch (e) {
const message = e instanceof Error ? e.message : "Could not create credential";
if (newKey) {
try {
const current = await api.lenses();
if (!current.workers.some((worker) => !worker.revoked && worker.analysis_key_id === newKey)) {
await api.deleteKeys([newKey]);
}
onChanged();
} catch {
throw new Error(
`${message}. Could not confirm cleanup. Check the Lens analysis key in Virtual Keys before retrying.`,
);
}
}
throw new Error(message);
}
},
});
}

40
ui/litellm-dashboard/src/components/lens/data/LensServices.tsx Normal file
View file

@ -0,0 +1,40 @@
"use client";
import { createContext, useContext, useMemo, type ReactNode } from "react";
import { apiClient } from "@/components/networking";
import { liveTracesApi, TracesApiContext, type TracesApi } from "@/components/view_logs/TraceView/tracesApi";
import { liveLensApi, type LensApi } from "./service";
export interface LensServices {
readonly accessToken: string;
readonly lens: LensApi;
readonly traces: TracesApi;
}
const LensServicesContext = createContext<LensServices | null>(null);
export function liveLensServices(accessToken: string): LensServices {
return { accessToken, lens: liveLensApi(apiClient, accessToken), traces: liveTracesApi(accessToken) };
}
function useLensServices(): LensServices {
const provided = useContext(LensServicesContext);
if (!provided) throw new Error("Lens services need a LensServicesProvider above them");
return provided;
}
export const useLensApi = (): LensApi => useLensServices().lens;
export const useLensAccessToken = (): string => useLensServices().accessToken;
export function useLiveLensServices(accessToken: string): LensServices {
return useMemo(() => liveLensServices(accessToken), [accessToken]);
}
export function LensServicesProvider({ services, children }: { services: LensServices; children: ReactNode }) {
return (
<LensServicesContext.Provider value={services}>
<TracesApiContext.Provider value={services.traces}>{children}</TracesApiContext.Provider>
</LensServicesContext.Provider>
);
}

5
ui/litellm-dashboard/src/components/lens/demo/createLensDemo.test.ts → ui/litellm-dashboard/src/components/lens/data/demo/createLensDemo.test.ts
View file

@ -1,6 +1,7 @@
import { describe, expect, it, vi } from "vitest";
import { createLensDemo, createLensDemoData } from "./createLensDemo";
import { evidenceTarget } from "../model/findings";
import { createLensDemo } from "./createLensDemo";
import { createLensDemoData } from "./fixtures";
import { evidenceTarget } from "../../model/findings";
describe("Lens demo data", () => {
it("links every finding to the quoted original step and assessed run", () => {

76
ui/litellm-dashboard/src/components/lens/data/demo/createLensDemo.ts Normal file
View file

@ -0,0 +1,76 @@
import { ApiError } from "@/lib/http/client";
import type { TracesApi } from "@/components/view_logs/TraceView/tracesApi";
import type { LensServices } from "../LensServices";
import type { LensApi } from "../service";
import { createLensDemoData, type LensDemoData } from "./fixtures";
const notInDemo = (): Promise<never> =>
Promise.reject(new ApiError("This item is not in the demo", 404, { detail: "This item is not in the demo" }));
const readOnly = (): Promise<never> =>
Promise.reject(new ApiError("Demo data is read-only", 403, { detail: "Demo data is read-only" }));
const found = <T>(value: T | undefined): Promise<T> => (value === undefined ? notInDemo() : Promise.resolve(value));
function demoLensApi(data: LensDemoData): LensApi {
const jobs = (lensId: string) => data.lenses.find((lens) => lens.id === lensId)?.jobs;
return {
scope: "demo",
lenses: async () => ({ lenses: data.lenses, workers: [], tracing_enabled: true }),
activity: async () => ({ traces: true, requests: false }),
runs: (lensId, offset) => found(jobs(lensId)?.slice(offset)),
run: (lensId, jobId) => found(jobs(lensId)?.find((job) => job.id === jobId)),
execution: notInDemo,
sample: notInDemo,
agents: notInDemo,
models: async () => ({ data: [] }),
modelDetails: async () => ({ data: [] }),
keys: notInDemo,
keyInfo: notInDemo,
saveLens: readOnly,
startRun: readOnly,
watchAll: async () => ({ watching: [], skipped: [] }),
cancelRun: readOnly,
reviewFinding: readOnly,
registerWorker: readOnly,
setWorkerBillingKey: readOnly,
revokeWorker: readOnly,
generateAnalysisKey: readOnly,
deleteKeys: readOnly,
};
}
function demoTracesApi(data: LensDemoData): TracesApi {
const run = (traceId: string) => data.runs.find(({ trace }) => trace.summary.trace_id === traceId);
return {
live: false,
handoff: (traceId, spanId) => {
const found = run(traceId);
const step = spanId ? found?.details.find((span) => span.span_id === spanId) : found;
return { text: JSON.stringify(step, null, 2), copied: spanId ? "Step copied" : "Trace copied" };
},
list: async ({ startMs, endMs }) => ({
data: data.runs
.map((item) => item.trace.summary)
.filter((trace) => Date.parse(trace.start_time) >= startMs && Date.parse(trace.start_time) <= endMs),
next_cursor: null,
}),
anyRecorded: async () => data.runs.length > 0,
trace: (traceId) => found(run(traceId)?.trace),
span: (traceId, spanId) => found(run(traceId)?.details.find((span) => span.span_id === spanId)),
spanError: async (traceId, spanId) => {
const span = run(traceId)?.trace.spans.find((item) => item.span_id === spanId);
return found(
span && {
span_id: span.span_id,
message: span.error ?? "",
total_chars: span.error?.length ?? 0,
next_cursor: null,
},
);
},
};
}
export function createLensDemo(now = Date.now()): LensServices {
const data = createLensDemoData(now);
return { accessToken: "lens-demo", lens: demoLensApi(data), traces: demoTracesApi(data) };
}

78
ui/litellm-dashboard/src/components/lens/demo/createLensDemo.ts → ui/litellm-dashboard/src/components/lens/data/demo/fixtures.ts
View file

@ -1,11 +1,6 @@
import { ApiError } from "@/lib/http/client";
import type { TracesApi } from "@/components/view_logs/TraceView/tracesApi";
import type { LensServices } from "../services";
import type { LensApi } from "../api/service";
import type { Trace, Span, SpanDetail } from "@/components/view_logs/TraceView/traceTypes";
import type { Lens, Finding, Job, Settings } from "../model/types";
import type { Lens, Finding, Job, Settings } from "../../model/types";
import { withReleaseCases } from "./lensDemoLongTrace";
import { scenarios, type Scenario } from "./scenarios";
const executionId = (traceId: string) => btoa(JSON.stringify(["traces", "", traceId]));
@ -336,74 +331,3 @@ export function createLensDemoData(now = Date.now()) {
});
return { runs, lenses };
}
const notInDemo = (): Promise<never> =>
Promise.reject(new ApiError("This item is not in the demo", 404, { detail: "This item is not in the demo" }));
const readOnly = (): Promise<never> =>
Promise.reject(new ApiError("Demo data is read-only", 403, { detail: "Demo data is read-only" }));
const found = <T>(value: T | undefined): Promise<T> => (value === undefined ? notInDemo() : Promise.resolve(value));
function demoLensApi(data: LensDemoData): LensApi {
const jobs = (lensId: string) => data.lenses.find((lens) => lens.id === lensId)?.jobs;
return {
scope: "demo",
lenses: async () => ({ lenses: data.lenses, workers: [], tracing_enabled: true }),
activity: async () => ({ traces: true, requests: false }),
runs: (lensId, offset) => found(jobs(lensId)?.slice(offset)),
run: (lensId, jobId) => found(jobs(lensId)?.find((job) => job.id === jobId)),
execution: notInDemo,
sample: notInDemo,
agents: notInDemo,
models: async () => ({ data: [] }),
modelDetails: async () => ({ data: [] }),
keys: notInDemo,
keyInfo: notInDemo,
saveLens: readOnly,
startRun: readOnly,
watchAll: async () => ({ watching: [], skipped: [] }),
cancelRun: readOnly,
reviewFinding: readOnly,
registerWorker: readOnly,
setWorkerBillingKey: readOnly,
revokeWorker: readOnly,
generateAnalysisKey: readOnly,
deleteKeys: readOnly,
};
}
function demoTracesApi(data: LensDemoData): TracesApi {
const run = (traceId: string) => data.runs.find(({ trace }) => trace.summary.trace_id === traceId);
return {
live: false,
handoff: (traceId, spanId) => {
const found = run(traceId);
const step = spanId ? found?.details.find((span) => span.span_id === spanId) : found;
return { text: JSON.stringify(step, null, 2), copied: spanId ? "Step copied" : "Trace copied" };
},
list: async ({ startMs, endMs }) => ({
data: data.runs
.map((item) => item.trace.summary)
.filter((trace) => Date.parse(trace.start_time) >= startMs && Date.parse(trace.start_time) <= endMs),
next_cursor: null,
}),
anyRecorded: async () => data.runs.length > 0,
trace: (traceId) => found(run(traceId)?.trace),
span: (traceId, spanId) => found(run(traceId)?.details.find((span) => span.span_id === spanId)),
spanError: async (traceId, spanId) => {
const span = run(traceId)?.trace.spans.find((item) => item.span_id === spanId);
return found(
span && {
span_id: span.span_id,
message: span.error ?? "",
total_chars: span.error?.length ?? 0,
next_cursor: null,
},
);
},
};
}
export function createLensDemo(now = Date.now()): LensServices {
const data = createLensDemoData(now);
return { lens: demoLensApi(data), traces: demoTracesApi(data) };
}

0
ui/litellm-dashboard/src/components/lens/demo/lensDemoLongTrace.ts → ui/litellm-dashboard/src/components/lens/data/demo/lensDemoLongTrace.ts
View file

0
ui/litellm-dashboard/src/components/lens/demo/scenarios.ts → ui/litellm-dashboard/src/components/lens/data/demo/scenarios.ts
View file

56
ui/litellm-dashboard/src/components/lens/data/mutations.ts Normal file
View file

@ -0,0 +1,56 @@
"use client";
import { useMutation, useQueryClient } from "@tanstack/react-query";
import type { Lens, LensList, Settings } from "../model/types";
import type { LensApi } from "./service";
import { lensKeys } from "./queries";
import { useLensApi } from "./LensServices";
export type LensWrite = (api: LensApi) => Promise<unknown>;
export function useInvalidateLenses() {
const api = useLensApi();
const client = useQueryClient();
return () =>
Promise.all([
client.invalidateQueries({ queryKey: lensKeys.list(api.scope) }),
client.invalidateQueries({ queryKey: lensKeys.histories() }),
]);
}
export function useLensUpdate() {
const api = useLensApi();
const invalidate = useInvalidateLenses();
return useMutation({ retry: false, mutationFn: (write: LensWrite) => write(api), onSettled: invalidate });
}
function upsertLens(list: LensList | undefined, saved: Lens): LensList | undefined {
if (!list) return list;
const known = list.lenses.some((lens) => lens.id === saved.id);
const lenses = known ? list.lenses.map((lens) => (lens.id === saved.id ? saved : lens)) : [...list.lenses, saved];
return { ...list, lenses };
}
export function useSaveLens() {
const api = useLensApi();
const client = useQueryClient();
return useMutation({
retry: false,
mutationFn: ({ id, settings }: { id?: string; settings: Settings }): Promise<Lens> => api.saveLens(id, settings),
onSuccess: (saved) => {
client.setQueryData<LensList>(lensKeys.list(api.scope), (current) => upsertLens(current, saved));
return client.invalidateQueries({ queryKey: lensKeys.list(api.scope) });
},
});
}
export function useRevokeWorker() {
const api = useLensApi();
const client = useQueryClient();
return useMutation({
retry: false,
mutationFn: (id: string) => api.revokeWorker(id),
onSettled: () => client.invalidateQueries({ queryKey: lensKeys.list(api.scope) }),
});
}

63
ui/litellm-dashboard/src/components/lens/api/queries.ts → ui/litellm-dashboard/src/components/lens/data/queries.ts
View file

@ -1,9 +1,12 @@
import { infiniteQueryOptions, queryOptions, type Query } from "@tanstack/react-query";
import type { LensList, Settings, ActivitySelection } from "../model/types";
import { infiniteQueryOptions, keepPreviousData, queryOptions } from "@tanstack/react-query";
import { hasActiveJob } from "../model/status";
import type { Sample, Settings, ActivitySelection, Job } from "../model/types";
import type { KeyPage, LensApi } from "./service";
export type { Key } from "./service";
type Activity = Awaited<ReturnType<LensApi["activity"]>>;
export const lensKeys = {
all: ["lens"] as const,
lists: () => [...lensKeys.all, "list"] as const,
@ -19,29 +22,20 @@ export const lensKeys = {
models: (scope: string) => [...lensKeys.all, "models", { scope }] as const,
modelDetails: (scope: string) => [...lensKeys.all, "model-details", { scope }] as const,
activity: (scope: string) => [...lensKeys.all, "activity-available", { scope }] as const,
discovery: (scope: string, source: Settings["source"], hours: number | undefined, asOf: string) =>
[...lensKeys.all, "discovery", { scope, source, hours, asOf }] as const,
preview: (scope: string, selection: ActivitySelection, offset: number, asOf: string) =>
[...lensKeys.all, "preview", { scope, selection, offset, asOf }] as const,
agents: (scope: string, asOf: string) => [...lensKeys.all, "agents", { scope, asOf }] as const,
discoveries: () => [...lensKeys.all, "discovery"] as const,
discovery: (scope: string, source: Settings["source"], hours: number | undefined) =>
[...lensKeys.discoveries(), { scope, source, hours }] as const,
preview: (scope: string, selection: ActivitySelection, asOf: string) =>
[...lensKeys.all, "preview", { scope, selection, asOf }] as const,
agents: (scope: string) => [...lensKeys.all, "agents", { scope }] as const,
analysisKeys: (scope: string, query: string) => [...lensKeys.all, "analysis-keys", { scope, query }] as const,
analysisKeyInfo: (scope: string, keyId: string | undefined) =>
[...lensKeys.all, "analysis-key-info", { scope, keyId }] as const,
};
export const lensQueries = {
list(api: LensApi, workerSetup: boolean) {
const options = {
queryKey: lensKeys.list(api.scope),
queryFn: () => api.lenses(),
refetchInterval: (current: Query<LensList>): number | false => {
const running = current.state.data?.lenses.some((item) =>
item.jobs.some((job) => ["queued", "running"].includes(job.status)),
);
return workerSetup || running ? 2000 : 10000;
},
};
return queryOptions(options);
list(api: LensApi) {
return queryOptions({ queryKey: lensKeys.list(api.scope), queryFn: () => api.lenses(), staleTime: 5000 });
},
models(api: LensApi) {
return queryOptions({ queryKey: lensKeys.models(api.scope), queryFn: () => api.models() });
@ -54,7 +48,8 @@ export const lensQueries = {
queryKey: lensKeys.activity(api.scope),
queryFn: () => api.activity(),
enabled: loaded,
refetchInterval: 5000,
refetchInterval: ({ state }: { state: { data?: Activity } }) =>
state.data?.traces && state.data.requests ? false : 5000,
};
return queryOptions(options);
},
@ -63,7 +58,8 @@ export const lensQueries = {
queryKey: lensKeys.history(api.scope, lensId, historyOffset),
enabled: !!lensId,
queryFn: () => api.runs(lensId as string, historyOffset),
refetchInterval: 10000,
refetchInterval: ({ state }: { state: { data?: Job[] } }) =>
state.data && hasActiveJob(state.data) ? 10000 : false,
};
return queryOptions(options);
},
@ -96,31 +92,32 @@ export const lensQueries = {
};
return queryOptions(options);
},
discovery(api: LensApi, { value, asOf, enabled }: { value: ActivitySelection; asOf: string; enabled: boolean }) {
discovery(api: LensApi, { value, enabled }: { value: ActivitySelection; enabled: boolean }) {
const unfiltered = { source: value.source, service: "", filters: [], lookback_hours: value.lookback_hours };
const options = {
queryKey: lensKeys.discovery(api.scope, value.source, value.lookback_hours, asOf),
queryFn: () => api.sample(unfiltered, 0, asOf),
queryKey: lensKeys.discovery(api.scope, value.source, value.lookback_hours),
queryFn: () => api.sample(unfiltered, 0, new Date().toISOString()),
staleTime: 60000,
enabled,
};
return queryOptions(options);
},
preview(
api: LensApi,
{ scope, offset, asOf, enabled }: { scope: ActivitySelection; offset: number; asOf: string; enabled: boolean },
) {
preview(api: LensApi, { scope, asOf, enabled }: { scope: ActivitySelection; asOf: string; enabled: boolean }) {
const options = {
queryKey: lensKeys.preview(api.scope, scope, offset, asOf),
queryFn: () => api.sample(scope, offset, asOf),
queryKey: lensKeys.preview(api.scope, scope, asOf),
initialPageParam: 0,
queryFn: ({ pageParam }: { pageParam: number }) => api.sample(scope, pageParam, asOf),
getNextPageParam: (lastPage: Sample) => lastPage.next_offset ?? undefined,
enabled,
staleTime: 30000,
gcTime: 60_000,
placeholderData: keepPreviousData,
};
return queryOptions(options);
return infiniteQueryOptions(options);
},
agents(api: LensApi, asOf: string, source: Settings["source"]) {
agents(api: LensApi, source: Settings["source"]) {
const options = {
queryKey: lensKeys.agents(api.scope, asOf),
queryKey: lensKeys.agents(api.scope),
queryFn: () => api.agents(),
enabled: source !== "requests",
staleTime: 60000,

14
ui/litellm-dashboard/src/components/lens/api/service.ts → ui/litellm-dashboard/src/components/lens/data/service.ts
View file

@ -1,9 +1,17 @@
import { z } from "zod";
import type { RunWindow } from "../investigations/detail/RunNowDialog";
import type { ApiClient } from "@/lib/http/client";
import type { components } from "@/lib/http/schema";
import type { ActivitySelection, Job, Lens, LensList, Sample, Settings, WorkerCreated } from "../model/types";
import type { AnalysisModelInfo } from "../setup/fields/analysisModels";
import type {
ActivitySelection,
AnalysisModelInfo,
Job,
Lens,
LensList,
RunWindow,
Sample,
Settings,
WorkerCreated,
} from "../model/types";
export type ExecutionContent = components["schemas"]["ExecutionContent"];
export type FindingStatus = components["schemas"]["FindingUpdate"]["status"];

58
ui/litellm-dashboard/src/components/lens/hooks/useLensReadiness.ts Normal file
View file

@ -0,0 +1,58 @@
"use client";
import { useQuery } from "@tanstack/react-query";
import { isTracingNotEnabled, useTraceAvailability } from "@/components/view_logs/TraceView/useAgentTraces";
import { useLensAccessToken, useLensApi } from "../data/LensServices";
import { lensQueries } from "../data/queries";
import { readiness, type Readiness, type ReadinessInput } from "../model/readiness";
import { useWorkerConnected } from "./useWorkerConnected";
export interface LensReadiness extends Readiness {
readonly loading: boolean;
readonly checking: boolean;
readonly error: string | undefined;
refresh(): void;
}
export function useLensReadiness(canInvestigate: boolean): LensReadiness {
const api = useLensApi();
const accessToken = useLensAccessToken();
const traces = useTraceAvailability(accessToken, true);
const list = useQuery({ ...lensQueries.list(api), enabled: canInvestigate });
const activity = useQuery(lensQueries.activity(api, canInvestigate && list.isSuccess));
const connected = useWorkerConnected(list.data?.workers);
const tracesDisabled = isTracingNotEnabled(traces.error);
const input: ReadinessInput = {
traces: {
recorded: traces.data,
failed: traces.isError,
disabled: tracesDisabled,
checked: traces.isSuccess,
},
tracingConfigured: list.data?.tracing_enabled ?? false,
activity: activity.data,
activityError: activity.error,
connected,
listError: list.error,
investigations: list.data?.lenses.length ?? 0,
};
const state = readiness(input);
const activityError = state.activityReady ? null : (tracesDisabled ? null : traces.error) || activity.error;
const error = list.error || activityError;
const loadingTraces = !state.activityReady && traces.isPending;
const loadingActivity = !state.activityReady && list.isSuccess && activity.isPending;
const loadingInvestigations = canInvestigate && (list.isPending || loadingActivity);
const refresh = () => {
void traces.refetch();
if (!canInvestigate) return;
void list.refetch();
void activity.refetch();
};
return {
...state,
loading: loadingTraces || loadingInvestigations,
checking: traces.isFetching || list.isFetching || activity.isFetching,
error: error?.message,
refresh,
};
}

12
ui/litellm-dashboard/src/components/lens/hooks/useWorkerConnected.ts Normal file
View file

@ -0,0 +1,12 @@
"use client";
import { useNow } from "@/hooks/useNow";
import { workerConnected } from "../model/status";
import type { LensList } from "../model/types";
const HEARTBEAT_TICK_MS = 5000;
export function useWorkerConnected(workers: LensList["workers"] | null | undefined): boolean {
const now = useNow(HEARTBEAT_TICK_MS);
return workers?.some((worker) => workerConnected(worker, now)) ?? false;
}

131
ui/litellm-dashboard/src/components/lens/investigations/Evidence.tsx Normal file
View file

@ -0,0 +1,131 @@
"use client";
import { useState } from "react";
import { useQuery } from "@tanstack/react-query";
import { Inspector } from "@/components/shared/Inspector";
import { Button } from "@/components/ui/button";
import { RunView } from "@/components/view_logs/TraceView/TraceDrawer";
import { useLocalRunSelection } from "@/components/view_logs/TraceView/traceRouting";
import { lensQueries } from "../data/queries";
import { useLensAccessToken, useLensApi } from "../data/LensServices";
import { evidenceTarget, type EvidenceTarget } from "../model/findings";
import type { EvidenceRef } from "../route";
const SECTION = 8000;
export interface EvidenceViewProps {
readonly lensId: string;
readonly evidence: EvidenceRef;
/** Shown as a back link above the evidence when it is stacked over something else. */
readonly backLabel?: string;
readonly onBack?: () => void;
}
/** The original trace step or logged request a piece of evidence points at, filling the panel. */
export function EvidenceView({ lensId, evidence, backLabel, onBack }: EvidenceViewProps) {
const target = evidenceTarget(evidence.id);
return (
<div className="flex min-h-0 flex-1 flex-col" data-testid="evidence-view">
{onBack && backLabel && (
<div className="flex h-9 shrink-0 items-center border-b px-3">
<Inspector.BackLink label={backLabel} onClick={onBack} />
</div>
)}
<EvidenceBody
key={`${evidence.id}:${evidence.span}`}
lensId={lensId}
target={target}
evidence={evidence}
onBack={onBack}
/>
</div>
);
}
function EvidenceBody({
lensId,
target,
evidence,
onBack = () => {},
}: {
lensId: string;
target: EvidenceTarget | null;
evidence: EvidenceRef;
onBack?: () => void;
}) {
if (target?.source === "traces")
return (
<TraceEvidence
traceId={target.id}
traceRef={target.traceRef}
initialSpanId={evidence.span || null}
onBack={onBack}
/>
);
if (target?.source === "requests") return <RequestEvidence lensId={lensId} evidenceId={evidence.id} />;
return <p className="p-4 text-sm text-muted-foreground">This evidence is no longer available.</p>;
}
export function TraceEvidence({
traceId,
traceRef,
initialSpanId,
onBack,
}: {
traceId: string;
traceRef?: string;
initialSpanId: string | null;
onBack: () => void;
}) {
const selection = useLocalRunSelection(initialSpanId);
const accessToken = useLensAccessToken();
return (
<RunView
traceId={traceId}
traceRef={traceRef}
selection={selection}
accessToken={accessToken}
onBack={onBack}
embedded
/>
);
}
function RequestEvidence({ lensId, evidenceId }: { lensId: string; evidenceId: string }) {
const api = useLensApi();
const [offset, setOffset] = useState(0);
const request = { lensId, evidenceId, requestOffset: offset, source: "requests" };
const evidence = useQuery(lensQueries.evidence(api, request));
return (
<div className="min-h-0 flex-1 overflow-y-auto">
<header className="border-b px-4 py-4">
<h2 className="text-sm font-semibold">Request evidence</h2>
<p className="text-xs text-muted-foreground">Original logged input and output</p>
</header>
<div className="space-y-3 p-4">
{evidence.isLoading && <p role="status">Loading request…</p>}
{evidence.error && <p role="alert">{evidence.error.message}</p>}
{evidence.data?.parts.map((p) => (
<pre className="text-xs break-words whitespace-pre-wrap" key={p.span_id}>
{p.content}
</pre>
))}
{evidence.data?.parts.length === 0 && <p>Request was not found or is past retention</p>}
<div className="flex flex-wrap gap-2">
{offset > 0 && (
<Button variant="outline" onClick={() => setOffset(Math.max(0, offset - SECTION))}>
Previous section
</Button>
)}
{evidence.data?.parts.some((p) => p.truncated) && (
<Button variant="outline" onClick={() => setOffset(offset === 0 ? 1 : offset + SECTION)}>
Next section
</Button>
)}
</div>
</div>
</div>
);
}

135
ui/litellm-dashboard/src/components/lens/investigations/FindingDetails.integration.test.tsx Normal file
View file

@ -0,0 +1,135 @@
import { fireEvent, screen, within } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { afterEach, expect, it, vi } from "vitest";
import type { RunSelection } from "@/components/view_logs/TraceView/traceRouting";
import { renderWithLens } from "@/../tests/lens-test-utils";
import { Inspector } from "@/components/shared/Inspector";
import type { OwnedFinding } from "../model/inbox";
import type { Finding, Lens } from "../model/types";
import { FindingPanel, ownedFindingKey } from "./FindingDetails";
vi.mock("@/components/view_logs/TraceView/TraceDrawer", () => ({
RunView: ({ traceId, selection }: { traceId: string; selection: RunSelection }) => (
<div data-testid="run-view">
{traceId} at {selection.spanId}
</div>
),
}));
const lens = {
id: "lens",
settings: { name: "Support reviews", agent_name: "support_agent" },
jobs: [],
findings: [],
} as unknown as Lens;
const finding: Finding = {
id: "finding",
check_id: "check",
title: "Repeated retries",
description: "The tool kept failing",
kind: "issue",
priority: "high",
status: "open",
revision: 1,
first_seen: "2026-09-30T10:00:00Z",
last_seen: "2026-09-30T10:00:00Z",
reason: "Saved feedback",
suggestion: "",
limitation: "",
occurrences: [],
evidence: [],
};
const owned = (current: Finding | null): OwnedFinding | null => (current ? { lens, finding: current } : null);
function Harness({ current, onReview }: { current: Finding | null; onReview: FindingPanelReview }) {
return (
<Inspector.Root
items={[]}
itemKey={ownedFindingKey}
selected={owned(current)}
onSelectedChange={vi.fn()}
noun="finding"
storageKey="test.finding"
>
<FindingPanel readOnly={false} busy={false} onReview={onReview} />
</Inspector.Root>
);
}
type FindingPanelReview = Parameters<typeof FindingPanel>[0]["onReview"];
afterEach(() => vi.restoreAllMocks());
it("keeps a feedback draft during refreshes, sends it with status changes, and restores saved feedback on reopening", async () => {
vi.spyOn(window, "matchMedia").mockImplementation(
(query: string) =>
({
matches: query.includes("reduce"),
media: query,
addEventListener: vi.fn(),
removeEventListener: vi.fn(),
}) as unknown as MediaQueryList,
);
const user = userEvent.setup();
const onReview = vi.fn();
const view = renderWithLens(<Harness current={finding} onReview={onReview} />);
const reason = () => screen.getByRole("textbox", { name: "What should Lens remember?" });
expect(screen.getByRole("complementary", { name: "Finding details" })).toHaveTextContent("support_agent");
expect(reason()).toHaveValue("Saved feedback");
fireEvent.change(reason(), { target: { value: "Draft feedback" } });
await user.click(screen.getByRole("button", { name: "Mark resolved" }));
expect(onReview).toHaveBeenLastCalledWith({ lens, finding }, "resolved", "Draft feedback");
const refreshed: Finding = { ...finding, reason: "Server feedback", status: "resolved" };
view.rerender(<Harness current={refreshed} onReview={onReview} />);
expect(reason()).toHaveValue("Draft feedback");
await user.click(screen.getByRole("button", { name: "Reopen" }));
expect(onReview).toHaveBeenLastCalledWith({ lens, finding: refreshed }, "open", "Draft feedback");
await user.click(screen.getByRole("button", { name: "This is expected" }));
expect(onReview).toHaveBeenLastCalledWith({ lens, finding: refreshed }, "dismissed", "Draft feedback");
view.rerender(<Harness current={null} onReview={onReview} />);
expect(screen.queryByRole("complementary")).not.toBeInTheDocument();
view.rerender(<Harness current={refreshed} onReview={onReview} />);
expect(reason()).toHaveValue("Server feedback");
const other: Finding = { ...finding, id: "other", reason: "Other feedback" };
view.rerender(<Harness current={other} onReview={onReview} />);
expect(reason()).toHaveValue("Other feedback");
});
it("stacks a quote's original step over the finding and keeps the feedback draft on the way back", async () => {
const user = userEvent.setup();
const traceOf = (id: string) => btoa(JSON.stringify(["traces", "", id]));
const quoted: Finding = {
...finding,
evidence: [
{ execution_id: traceOf("trace-1"), span_id: "step-a", quote: "first quote", role: "support" },
{ execution_id: traceOf("trace-2"), span_id: "step-b", quote: "second quote", role: "support" },
],
};
const onUrlUpdate = vi.fn();
renderWithLens(<Harness current={quoted} onReview={vi.fn()} />, { onUrlUpdate });
const panel = screen.getByRole("complementary", { name: "Finding details" });
const reason = () => within(panel).getByRole("textbox", { name: "What should Lens remember?", hidden: true });
fireEvent.change(reason(), { target: { value: "Draft feedback" } });
for (const summary of within(panel).getAllByText(/quote$/)) await user.click(summary);
await user.click(within(panel).getAllByRole("button", { name: "Open original step" })[0]);
expect(await within(panel).findByTestId("run-view")).toHaveTextContent("trace-1 at step-a");
expect(reason()).not.toBeVisible();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
await user.click(within(panel).getByRole("button", { name: "Back to finding" }));
expect(within(panel).queryByTestId("run-view")).not.toBeInTheDocument();
expect(reason()).toBeVisible();
expect(reason()).toHaveValue("Draft feedback");
await user.click(within(panel).getAllByRole("button", { name: "Open original step" })[1]);
expect(await within(panel).findByTestId("run-view")).toHaveTextContent("trace-2 at step-b");
const url = new URLSearchParams(String(onUrlUpdate.mock.lastCall?.[0].queryString ?? ""));
expect(url.get("evidence")).toBe(traceOf("trace-2"));
expect(url.get("evidence_span")).toBe("step-b");
});

200
ui/litellm-dashboard/src/components/lens/investigations/FindingDetails.tsx Normal file
View file

@ -0,0 +1,200 @@
"use client";
import { useState } from "react";
import { ArrowUpRight } from "lucide-react";
import { Inspector } from "@/components/shared/Inspector";
import { Button } from "@/components/ui/button";
import { Textarea } from "@/components/ui/textarea";
import { evidenceTarget } from "../model/findings";
import { runTime } from "../model/format";
import { findingAgents, findingKey, type OwnedFinding, sampledExecutions } from "../model/inbox";
import type { Finding, Sample } from "../model/types";
import { EvidenceView } from "./Evidence";
import { IssueBrief } from "./IssueBrief";
import { type EvidenceRef, useEvidenceRoute } from "../route";
export const ownedFindingKey = (owned: OwnedFinding): string => findingKey(owned.lens, owned.finding);
export interface FindingDetailsProps {
readonly finding: Finding;
readonly agents?: readonly string[];
readonly sampledRuns: Sample["executions"];
readonly readOnly: boolean;
readonly busy: boolean;
readonly onOpenEvidence: (evidence: EvidenceRef) => void;
readonly onReview: (status: Finding["status"], reason: string) => void;
}
export function FindingDetails({
finding,
agents = [],
sampledRuns,
readOnly,
busy,
onOpenEvidence,
onReview,
}: FindingDetailsProps) {
const [reason, setReason] = useState(finding.reason ?? "");
const evidenceGroups = [...new Set(finding.evidence.map((e) => e.execution_id))].map((id) => ({
id,
run: sampledRuns.find((r) => r.id === id),
quotes: finding.evidence.filter((e) => e.execution_id === id),
}));
return (
<div className="min-h-0 flex-1 overflow-y-auto">
<header className="flex flex-col gap-1.5 border-b px-4 py-4">
<h2 className="text-xl leading-snug font-semibold">{finding.title}</h2>
<p className="text-sm text-muted-foreground">
{agents.length > 0 && <span className="font-medium text-foreground">{agents.join(", ")} · </span>}
{finding.kind === "issue" ? `${finding.priority} priority` : "Pattern"} · {finding.occurrences?.length ?? 0}{" "}
linked {finding.occurrences?.length === 1 ? "run" : "runs"}
</p>
</header>
<div className="space-y-6 p-4">
{finding.brief ? (
<IssueBrief title={finding.title} brief={finding.brief} />
) : (
<>
<div>
<p className="mb-2 text-sm font-medium">What happened</p>
<p className="text-sm leading-6 whitespace-pre-wrap">{finding.description}</p>
</div>
{finding.suggestion && (
<div className="border-y py-4">
<p className="text-sm font-medium">What to do next</p>
<p className="mt-2 text-sm leading-6">{finding.suggestion}</p>
</div>
)}
</>
)}
{finding.limitation && (
<details className="text-sm">
<summary className="cursor-pointer font-medium">Evidence limits</summary>
<p className="mt-3 leading-6 text-muted-foreground">{finding.limitation}</p>
</details>
)}
<div>
<p className="text-sm font-medium">Evidence by run</p>
<p className="mt-1 mb-3 text-xs text-muted-foreground">
Exact quotes from the recorded activity. Counterexamples are labeled separately from supporting evidence.
</p>
<div className="space-y-2">
{evidenceGroups.map((group) => (
<details key={group.id} className="rounded-lg border p-3">
<summary className="cursor-pointer text-sm font-medium">
{group.run?.name ?? evidenceTarget(group.id)?.id.slice(0, 12) ?? "Recorded run"}
<span className="ml-2 text-xs font-normal text-muted-foreground">
{group.quotes.length} {group.quotes.length === 1 ? "quote" : "quotes"}
{group.run ? ` · ${runTime(group.run.start_time)}` : ""}
</span>
</summary>
<div className="mt-3 space-y-3">
{group.quotes.map((e, i) => (
<div key={`${e.span_id}-${i}`} className="rounded-md bg-muted/40 p-3">
{e.role === "counterexample" && (
<p className="mb-1 text-xs font-medium text-muted-foreground">Counterexample</p>
)}
<blockquote className="text-xs leading-5 whitespace-pre-wrap break-words">{e.quote}</blockquote>
<Button
variant="ghost"
size="sm"
className="mt-2"
onClick={() => onOpenEvidence({ id: e.execution_id, span: e.span_id })}
>
{evidenceTarget(e.execution_id)?.source === "traces" ? "Open original step" : "Open request"}
<ArrowUpRight className="size-3" />
</Button>
</div>
))}
</div>
</details>
))}
</div>
</div>
{!readOnly && (
<div className="space-y-3 border-t pt-4">
<label className="grid gap-2 text-sm">
What should Lens remember?
<Textarea
value={reason}
onChange={(e) => setReason(e.target.value)}
placeholder="What should Lens know about this finding?"
/>
</label>
<p className="text-xs text-muted-foreground">Your explanation informs future scans of this Lens.</p>
<div className="flex flex-wrap gap-2">
{finding.kind === "issue" && (
<Button
disabled={busy}
onClick={() => onReview(finding.status === "resolved" ? "open" : "resolved", reason)}
>
{finding.status === "resolved" ? "Reopen" : "Mark resolved"}
</Button>
)}
<Button disabled={busy} variant="outline" onClick={() => onReview("dismissed", reason)}>
This is expected
</Button>
</div>
</div>
)}
</div>
</div>
);
}
export interface FindingPanelProps {
readonly readOnly: boolean;
readonly busy: boolean;
/** Runs to name evidence by; defaults to every run the owning investigation has sampled. */
readonly sampledRuns?: Sample["executions"];
readonly onReview: (owned: OwnedFinding, status: Finding["status"], reason: string) => void;
}
/**
* A finding in the side panel. Opening a quote's original step or request stacks it over the finding, which
* stays mounted so a feedback draft survives the round trip.
*/
export function FindingPanelBody({
owned,
readOnly,
busy,
sampledRuns,
onReview,
}: FindingPanelProps & { readonly owned: OwnedFinding }) {
const { evidence, setEvidence } = useEvidenceRoute();
return (
<>
<div hidden={evidence !== null} className={evidence ? undefined : "flex min-h-0 flex-1 flex-col"}>
<FindingDetails
key={ownedFindingKey(owned)}
finding={owned.finding}
agents={findingAgents(owned.lens, owned.finding)}
sampledRuns={sampledRuns ?? sampledExecutions(owned.lens)}
readOnly={readOnly}
busy={busy}
onOpenEvidence={setEvidence}
onReview={(status, reason) => onReview(owned, status, reason)}
/>
</div>
{evidence && (
<EvidenceView
lensId={owned.lens.id}
evidence={evidence}
backLabel="Back to finding"
onBack={() => setEvidence(null)}
/>
)}
</>
);
}
/** The side panel for whichever finding the surrounding Inspector has open. */
export function FindingPanel(props: FindingPanelProps) {
return (
<Inspector.Panel label="Finding details" testId="finding-panel">
{(owned: OwnedFinding) => <FindingPanelBody {...props} owned={owned} />}
</Inspector.Panel>
);
}

51
ui/litellm-dashboard/src/components/lens/investigations/FindingSheet.integration.test.tsx
View file

@ -1,51 +0,0 @@
import { fireEvent, screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { expect, it, vi } from "vitest";
import { renderWithProviders } from "@/../tests/test-utils";
import type { Finding } from "../model/types";
import { FindingSheet } from "./FindingSheet";
const finding: Finding = {
id: "finding",
check_id: "check",
title: "Repeated retries",
description: "The tool kept failing",
kind: "issue",
priority: "high",
status: "open",
revision: 1,
first_seen: "2026-09-30T10:00:00Z",
last_seen: "2026-09-30T10:00:00Z",
reason: "Saved feedback",
suggestion: "",
limitation: "",
occurrences: [],
evidence: [],
};
it("keeps a feedback draft during refreshes, sends it with status changes, and restores saved feedback on reopening", async () => {
const user = userEvent.setup();
const changeFinding = vi.fn().mockResolvedValue(undefined);
const props = { sampledRuns: [], readOnly: false, busy: false, onClose: vi.fn(), onEvidence: vi.fn(), changeFinding };
const view = renderWithProviders(<FindingSheet {...props} finding={finding} />);
const reason = () => screen.getByRole("textbox", { name: "What should Lens remember?" });
expect(reason()).toHaveValue("Saved feedback");
fireEvent.change(reason(), { target: { value: "Draft feedback" } });
await user.click(screen.getByRole("button", { name: "Mark resolved" }));
expect(changeFinding).toHaveBeenLastCalledWith("resolved", "Draft feedback");
const refreshed: Finding = { ...finding, reason: "Server feedback", status: "resolved" };
view.rerender(<FindingSheet {...props} finding={refreshed} />);
expect(reason()).toHaveValue("Draft feedback");
await user.click(screen.getByRole("button", { name: "Reopen" }));
expect(changeFinding).toHaveBeenLastCalledWith("open", "Draft feedback");
await user.click(screen.getByRole("button", { name: "This is expected" }));
expect(changeFinding).toHaveBeenLastCalledWith("dismissed", "Draft feedback");
view.rerender(<FindingSheet {...props} />);
view.rerender(<FindingSheet {...props} finding={refreshed} />);
expect(reason()).toHaveValue("Server feedback");
const other: Finding = { ...finding, id: "other", reason: "Other feedback" };
view.rerender(<FindingSheet {...props} finding={other} />);
expect(reason()).toHaveValue("Other feedback");
});

164
ui/litellm-dashboard/src/components/lens/investigations/FindingSheet.tsx
View file

@ -1,164 +0,0 @@
import { useState } from "react";
import { ArrowUpRight } from "lucide-react";
import { Button } from "@/components/ui/button";
import { Textarea } from "@/components/ui/textarea";
import { Sheet, SheetContent, SheetHeader, SheetTitle, SheetDescription } from "@/components/ui/sheet";
import { IssueBrief } from "./IssueBrief";
import { evidenceTarget } from "../model/findings";
import { runTime } from "../model/format";
import { type Finding, type Sample } from "../model/types";
export function FindingSheet({
finding,
agents = [],
sampledRuns,
readOnly,
busy,
onClose,
onEvidence,
changeFinding,
}: {
finding?: Finding;
agents?: readonly string[];
sampledRuns: Sample["executions"];
readOnly: boolean;
busy: boolean;
onClose: () => void;
onEvidence: (evidence: { id: string; span: string }) => void;
changeFinding: (status: Finding["status"], reason: string) => Promise<void>;
}) {
const [reason, setReason] = useFindingReason(finding);
const evidenceGroups = finding
? [...new Set(finding.evidence.map((e) => e.execution_id))].map((id) => ({
id,
run: sampledRuns.find((r) => r.id === id),
quotes: finding.evidence.filter((e) => e.execution_id === id),
}))
: [];
return (
<Sheet
open={!!finding}
onOpenChange={(open) => {
if (!open) onClose();
}}
>
<SheetContent className="overflow-y-auto data-[side=right]:sm:max-w-2xl">
{finding && (
<>
<SheetHeader>
<SheetTitle className="pr-8 text-xl leading-snug">{finding.title}</SheetTitle>
<SheetDescription>
{agents.length > 0 && <span className="font-medium text-foreground">{agents.join(", ")} · </span>}
{finding.kind === "issue" ? `${finding.priority} priority` : "Pattern"} ·{" "}
{finding.occurrences?.length ?? 0} linked {finding.occurrences?.length === 1 ? "run" : "runs"}
</SheetDescription>
</SheetHeader>
<div className="space-y-6 p-4">
{finding.brief ? (
<IssueBrief title={finding.title} brief={finding.brief} />
) : (
<>
<div>
<p className="mb-2 text-sm font-medium">What happened</p>
<p className="text-sm leading-6 whitespace-pre-wrap">{finding.description}</p>
</div>
{finding.suggestion && (
<div className="border-y py-4">
<p className="text-sm font-medium">What to do next</p>
<p className="mt-2 text-sm leading-6">{finding.suggestion}</p>
</div>
)}
</>
)}
{finding.limitation && (
<details className="text-sm">
<summary className="cursor-pointer font-medium">Evidence limits</summary>
<p className="mt-3 leading-6 text-muted-foreground">{finding.limitation}</p>
</details>
)}
<div>
<p className="text-sm font-medium">Evidence by run</p>
<p className="mt-1 mb-3 text-xs text-muted-foreground">
Exact quotes from the recorded activity. Counterexamples are labeled separately from supporting
evidence.
</p>
<div className="space-y-2">
{evidenceGroups.map((group) => (
<details key={group.id} className="rounded-lg border p-3">
<summary className="cursor-pointer text-sm font-medium">
{group.run?.name ?? evidenceTarget(group.id)?.id.slice(0, 12) ?? "Recorded run"}
<span className="ml-2 text-xs font-normal text-muted-foreground">
{group.quotes.length} {group.quotes.length === 1 ? "quote" : "quotes"}
{group.run ? ` · ${runTime(group.run.start_time)}` : ""}
</span>
</summary>
<div className="mt-3 space-y-3">
{group.quotes.map((e, i) => (
<div key={`${e.span_id}-${i}`} className="rounded-md bg-muted/40 p-3">
{e.role === "counterexample" && (
<p className="mb-1 text-xs font-medium text-muted-foreground">Counterexample</p>
)}
<blockquote className="text-xs leading-5 whitespace-pre-wrap break-words">
{e.quote}
</blockquote>
<Button
variant="ghost"
size="sm"
className="mt-2"
onClick={() => {
onEvidence({ id: e.execution_id, span: e.span_id });
}}
>
{evidenceTarget(e.execution_id)?.source === "traces"
? "Open original step"
: "Open request"}
<ArrowUpRight className="size-3" />
</Button>
</div>
))}
</div>
</details>
))}
</div>
</div>
{!readOnly && (
<div className="space-y-3 border-t pt-4">
<label className="grid gap-2 text-sm">
What should Lens remember?
<Textarea
value={reason}
onChange={(e) => setReason(e.target.value)}
placeholder="What should Lens know about this finding?"
/>
</label>
<p className="text-xs text-muted-foreground">Your explanation informs future scans of this Lens.</p>
<div className="flex flex-wrap gap-2">
{finding.kind === "issue" && (
<Button
disabled={busy}
onClick={() => changeFinding(finding.status === "resolved" ? "open" : "resolved", reason)}
>
{finding.status === "resolved" ? "Reopen" : "Mark resolved"}
</Button>
)}
<Button disabled={busy} variant="outline" onClick={() => changeFinding("dismissed", reason)}>
This is expected
</Button>
</div>
</div>
)}
</div>
</>
)}
</SheetContent>
</Sheet>
);
}
function useFindingReason(finding: Finding | undefined) {
const [draft, setDraft] = useState({ id: finding?.id, reason: finding?.reason ?? "" });
if (draft.id !== finding?.id) setDraft({ id: finding?.id, reason: finding?.reason ?? "" });
const reason = draft.reason;
const setReason = (reason: string) => setDraft({ ...draft, reason });
return [reason, setReason] as const;
}

491
ui/litellm-dashboard/src/components/lens/investigations/InvestigationList.tsx
View file

@ -1,6 +1,14 @@
"use client";
import { Fragment, useState, type KeyboardEvent, type ReactNode } from "react";
import {
getCoreRowModel,
getExpandedRowModel,
useReactTable,
type CellContext,
type ColumnDef,
type TableOptions,
} from "@tanstack/react-table";
import { createContext, useContext, type ReactNode } from "react";
import {
ChevronRight,
Circle,
@ -11,32 +19,64 @@ import {
CircleX,
Pencil,
Play,
Search,
} from "lucide-react";
import { useNow } from "@/hooks/useNow";
import { Input } from "@/components/ui/input";
import { Inspector } from "@/components/shared/Inspector";
import { InspectorTable } from "@/components/shared/InspectorTable";
import { formatActivityTimestamp } from "@/utils/activityTimestamp";
import { cn } from "@/lib/cva.config";
import { agoLabel } from "@/components/view_logs/TraceView/lensField";
import { agoLabel, scopeLabel } from "../model/format";
import { findingAgents, openFindings, scheduleLabel } from "../model/inbox";
import { findingAgents, findingKey, openFindings, scheduleLabel } from "../model/inbox";
import { lensStatus } from "../model/status";
import { scopeLabel } from "../model/format";
import { SearchBox } from "@/components/shared/search/SearchBox";
import { itemValues } from "@/components/shared/search/valueSource";
import { type Finding, type Lens } from "../model/types";
import { useListSearchRoute } from "../route";
import { FINDING_PANEL_WIDTH_KEY } from "../storage";
import { filterInvestigations, INVESTIGATION_INDEX, INVESTIGATION_QUERY } from "./investigationQuery";
const PRIORITY_COLOR = { high: "text-destructive", medium: "text-amber-500", low: "text-muted-foreground" } as const;
const ROW =
"border-b border-border/60 transition-colors duration-150 hover:bg-trace-row-hover motion-reduce:transition-none";
const PRIORITY_COLOR = { high: "text-destructive", medium: "text-warning", low: "text-muted-foreground" } as const;
const META = "truncate text-xs text-muted-foreground";
const ACTION =
"inline-flex size-7 items-center justify-center rounded text-muted-foreground hover:bg-muted hover:text-foreground";
const INVESTIGATION_HEIGHT = 56;
const FINDING_HEIGHT = 48;
function openOnRowKeyDown(event: KeyboardEvent<HTMLTableRowElement>, open: () => void) {
if (event.target !== event.currentTarget || (event.key !== "Enter" && event.key !== " ")) return;
event.preventDefault();
open();
/** One row of the list: an investigation, or an open finding shown under the investigation that owns it. */
export type InvestigationRow =
| { readonly kind: "investigation"; readonly lens: Lens }
| { readonly kind: "finding"; readonly lens: Lens; readonly finding: Finding };
export const investigationRowKey = (row: InvestigationRow): string =>
row.kind === "investigation" ? `investigation:${row.lens.id}` : `finding:${findingKey(row.lens, row.finding)}`;
const findingRows = (row: InvestigationRow): InvestigationRow[] | undefined =>
row.kind === "investigation"
? openFindings(row.lens).map((finding) => ({ kind: "finding", lens: row.lens, finding }))
: undefined;
interface ListContextValue {
readonly now: number;
readonly connected: boolean;
readonly readOnly: boolean;
readonly onEdit: (id: string) => void;
readonly onRunNow: (id: string) => void;
}
const ListContext = createContext<ListContextValue | null>(null);
function useList(): ListContextValue {
const value = useContext(ListContext);
if (value === null) throw new Error("Investigation cells must be rendered inside InvestigationList");
return value;
}
type Cell = CellContext<InvestigationRow, unknown>;
const ROW_LABEL = { investigation: "Investigation details", finding: "Finding details" } as const;
function JobIcon({ lens }: { lens: Lens }) {
const status = lens.jobs[0]?.status;
const className = "size-4 shrink-0";
@ -45,7 +85,7 @@ function JobIcon({ lens }: { lens: Lens }) {
if (status === "failed") return <CircleX aria-hidden="true" className={cn(className, "text-destructive")} />;
if (status === "cancelled")
return <CircleSlash aria-hidden="true" className={cn(className, "text-muted-foreground")} />;
if (status === "completed") return <CircleCheck aria-hidden="true" className={cn(className, "text-emerald-600")} />;
if (status === "completed") return <CircleCheck aria-hidden="true" className={cn(className, "text-success")} />;
return <Circle aria-hidden="true" className={cn(className, "text-muted-foreground")} />;
}
@ -58,222 +98,237 @@ function TwoLine({ meta, title, className }: { meta: ReactNode; title: ReactNode
);
}
function NameCell({ row }: Cell) {
const { now } = useList();
const item = row.original;
if (item.kind === "investigation")
return (
<span className="flex min-w-0 items-center gap-2">
<InspectorTable.Indent
row={row}
toggleLabel={(expanded) => `${expanded ? "Hide" : "Show"} findings for ${item.lens.settings.name}`}
/>
<JobIcon lens={item.lens} />
<TwoLine
meta={`${scopeLabel(item.lens.settings)} · ${scheduleLabel(item.lens, now)}`}
title={item.lens.settings.name}
className="font-semibold"
/>
</span>
);
const priority = item.finding.priority ?? "medium";
return (
<span
className="flex min-w-0 items-center gap-2"
title={item.finding.suggestion ? `Fix: ${item.finding.suggestion}` : undefined}
>
<InspectorTable.Indent row={row} className="h-12" />
<span aria-hidden="true" className="w-4 shrink-0" />
<CircleDot aria-hidden="true" className={cn("size-4 shrink-0", PRIORITY_COLOR[priority])} />
<TwoLine
meta={`${priority} priority · ${findingAgents(item.lens, item.finding).join(", ")}`}
title={item.finding.title}
/>
</span>
);
}
function StatusCell({ row: { original: item } }: Cell) {
const { connected } = useList();
if (item.kind === "finding") {
const runs = item.finding.occurrences.length;
return (
<span className="text-muted-foreground">
{runs} {runs === 1 ? "run" : "runs"}
</span>
);
}
const failed = item.lens.jobs[0]?.status === "failed";
return (
<span className={cn("block truncate", failed ? "text-destructive" : "text-muted-foreground")}>
{lensStatus(item.lens, connected)}
</span>
);
}
function FindingCount({ row: { original: item } }: Cell) {
if (item.kind === "finding") return null;
const count = openFindings(item.lens).length;
if (count === 0) return null;
return (
<span
title={`${count} open ${count === 1 ? "finding" : "findings"}`}
className="inline-flex min-w-5 justify-center rounded-full bg-muted px-1.5 font-mono text-xs tabular-nums text-foreground"
>
{count}
</span>
);
}
function ActivityCell({ row: { original: item } }: Cell) {
const { now } = useList();
if (item.kind === "finding")
return (
<span title={formatActivityTimestamp(item.finding.last_seen)}>
{agoLabel(Date.parse(item.finding.last_seen), now)}
</span>
);
const latest = item.lens.jobs[0];
return (
<span title={latest ? formatActivityTimestamp(latest.created_at) : undefined}>
{latest ? agoLabel(Date.parse(latest.created_at), now) : "never run"}
</span>
);
}
function ActionsCell({ row: { original: item } }: Cell) {
const { readOnly, onEdit, onRunNow } = useList();
if (item.kind === "finding")
return <ChevronRight aria-hidden="true" className="mr-2 ml-auto size-3.5 text-muted-foreground/60" />;
if (readOnly) return null;
const { id, settings } = item.lens;
return (
<span className="flex items-center justify-end gap-0.5">
<button
type="button"
aria-label={`Run ${settings.name} now`}
title="Run now"
onClick={(event) => {
event.stopPropagation();
onRunNow(id);
}}
className={ACTION}
>
<Play className="size-3.5" />
</button>
<button
type="button"
aria-label={`Edit ${settings.name}`}
title="Edit"
onClick={(event) => {
event.stopPropagation();
onEdit(id);
}}
className={cn(ACTION, "text-muted-foreground/60 group-hover:text-muted-foreground")}
>
<Pencil className="size-3.5" />
</button>
</span>
);
}
const COLUMNS: ColumnDef<InvestigationRow>[] = [
{ id: "name", header: "Investigation", cell: NameCell, meta: { className: "pl-2 pr-0" } },
{ id: "status", size: 180, header: "Status", cell: StatusCell, meta: { className: "text-right text-xs" } },
{ id: "findings", size: 72, header: "Open findings", cell: FindingCount, meta: { className: "text-right" } },
{
id: "activity",
size: 120,
header: "Last activity",
cell: ActivityCell,
meta: { className: "text-right text-xs text-muted-foreground" },
},
{ id: "actions", size: 76, header: "Actions", cell: ActionsCell, meta: { className: "pl-0 pr-3" } },
];
export interface InvestigationListProps {
readonly lenses: readonly Lens[];
readonly connected: boolean;
readonly readOnly?: boolean;
readonly selected: InvestigationRow | null;
readonly onSelect: (row: InvestigationRow | null) => void;
readonly onEdit: (id: string) => void;
readonly onRunNow: (id: string) => void;
readonly actions?: ReactNode;
/** Body of the side panel for the selected row. */
readonly children: (row: InvestigationRow) => ReactNode;
}
/** Investigations with their open findings; any row opens in the side panel and J/K walk the visible rows. */
export function InvestigationList({
lenses,
connected,
readOnly = false,
onOpen,
selected,
onSelect,
onEdit,
onRunNow,
onOpenFinding,
}: {
lenses: Lens[];
connected: boolean;
readOnly?: boolean;
onOpen: (id: string) => void;
onEdit: (id: string) => void;
onRunNow: (id: string) => void;
onOpenFinding: (lens: Lens, finding: Finding) => void;
}) {
actions,
children,
}: InvestigationListProps) {
const [search, setSearch] = useListSearchRoute();
const now = useNow(15000);
const [collapsed, setCollapsed] = useState<ReadonlySet<string>>(new Set());
const toggle = (id: string) =>
setCollapsed((current) => {
const next = new Set(current);
if (!next.delete(id)) next.add(id);
return next;
});
const shown = lenses.filter((lens) =>
`${lens.settings.name} ${scopeLabel(lens.settings)}`.toLowerCase().includes(search.toLowerCase()),
);
const shown = filterInvestigations([...lenses], search);
const tableOptions: TableOptions<InvestigationRow> = {
data: shown.map((lens): InvestigationRow => ({ kind: "investigation", lens })),
columns: COLUMNS,
getRowId: investigationRowKey,
getSubRows: findingRows,
initialState: { expanded: true },
autoResetAll: false,
getCoreRowModel: getCoreRowModel(),
getExpandedRowModel: getExpandedRowModel(),
};
const table = useReactTable(tableOptions);
const rows = table.getRowModel().rows.map((row) => row.original);
const noun = selected?.kind ?? "investigation";
return (
<div className="flex min-h-[420px] flex-1 flex-col overflow-hidden rounded-lg border border-border bg-card">
<div className="flex min-h-11 shrink-0 flex-wrap items-center gap-2 border-b border-border bg-muted/40 px-3 py-2">
<div className="relative w-full max-w-[380px]">
<Search className="pointer-events-none absolute top-1/2 left-2.5 size-3.5 -translate-y-1/2 text-muted-foreground" />
<Input
aria-label="Search investigations"
placeholder="Search investigations"
className="h-7 bg-background pl-8 text-xs"
<Inspector.Root
items={rows}
itemKey={investigationRowKey}
selected={selected}
onSelectedChange={onSelect}
noun={noun}
storageKey={FINDING_PANEL_WIDTH_KEY}
>
<div className="flex min-h-[420px] flex-1 flex-col overflow-hidden bg-card">
<div className="flex min-h-11 shrink-0 flex-wrap items-center gap-2 border-b border-border bg-card p-2">
<SearchBox.Root
language={INVESTIGATION_QUERY}
values={itemValues(INVESTIGATION_INDEX, lenses)}
value={search}
onChange={(e) => setSearch(e.target.value)}
/>
onValueChange={setSearch}
label="Search investigations"
>
<SearchBox.Input
className="rounded-lg"
placeholder="Search investigations, or filter like status:failed schedule:watching"
/>
<SearchBox.Suggestions />
</SearchBox.Root>
{actions && <div className="ml-auto flex items-center gap-2">{actions}</div>}
</div>
</div>
<div className="min-h-0 flex-1 overflow-auto">
<table aria-label="Investigations" className="w-full min-w-[720px] table-fixed border-collapse text-left">
<thead className="sr-only">
<tr>
<th>Investigation</th>
<th>Status</th>
<th>Open findings</th>
<th>Last activity</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{shown.map((lens) => {
const latest = lens.jobs[0];
const findings = openFindings(lens);
const expanded = findings.length > 0 && !collapsed.has(lens.id);
return (
<Fragment key={lens.id}>
<tr
onClick={() => onOpen(lens.id)}
onKeyDown={(event) => openOnRowKeyDown(event, () => onOpen(lens.id))}
<ListContext.Provider value={{ now, connected, readOnly, onEdit, onRunNow }}>
<InspectorTable.Root table={table}>
<InspectorTable.Grid aria-label="Investigations" className="min-w-[720px]">
<InspectorTable.Header hidden />
<InspectorTable.Body<InvestigationRow>
rowHeight={(row) => (row.depth ? FINDING_HEIGHT : INVESTIGATION_HEIGHT)}
>
{(row) => (
<InspectorTable.Row
row={row}
item={row.original}
tabIndex={0}
aria-label={lens.settings.name}
className={cn(ROW, "group h-14 cursor-pointer focus-visible:outline-2 focus-visible:outline-ring")}
>
<td className="pl-2">
<span className="flex min-w-0 items-center gap-2">
{findings.length > 0 ? (
<button
type="button"
aria-expanded={expanded}
aria-label={`${expanded ? "Hide" : "Show"} findings for ${lens.settings.name}`}
onClick={(event) => {
event.stopPropagation();
toggle(lens.id);
}}
className="inline-flex size-6 shrink-0 items-center justify-center rounded text-muted-foreground hover:bg-muted hover:text-foreground"
>
<ChevronRight
className={cn(
"size-3.5 transition-transform duration-150 motion-reduce:transition-none",
expanded && "rotate-90",
)}
/>
</button>
) : (
<span aria-hidden="true" className="size-6 shrink-0" />
)}
<JobIcon lens={lens} />
<TwoLine
meta={`${scopeLabel(lens.settings)} · ${scheduleLabel(lens, now)}`}
title={lens.settings.name}
className="font-semibold"
/>
</span>
</td>
<td
className={cn(
"w-[180px] truncate px-3 text-right text-xs",
latest?.status === "failed" ? "text-destructive" : "text-muted-foreground",
)}
>
{lensStatus(lens, connected)}
</td>
<td className="w-[72px] px-3 text-right">
{findings.length > 0 && (
<span
title={`${findings.length} open ${findings.length === 1 ? "finding" : "findings"}`}
className="inline-flex min-w-5 justify-center rounded-full bg-muted px-1.5 font-mono text-xs tabular-nums text-foreground"
>
{findings.length}
</span>
)}
</td>
<td
className="w-[120px] px-3 text-right text-xs text-muted-foreground"
title={latest ? formatActivityTimestamp(latest.created_at) : undefined}
>
{latest ? agoLabel(Date.parse(latest.created_at), now) : "never run"}
</td>
<td className="w-[76px] pr-3">
{!readOnly && (
<span className="flex items-center justify-end gap-0.5">
<button
type="button"
aria-label={`Run ${lens.settings.name} now`}
title="Run now"
onClick={(event) => {
event.stopPropagation();
onRunNow(lens.id);
}}
className="inline-flex size-7 items-center justify-center rounded text-muted-foreground hover:bg-muted hover:text-foreground"
>
<Play className="size-3.5" />
</button>
<button
type="button"
aria-label={`Edit ${lens.settings.name}`}
title="Edit"
onClick={(event) => {
event.stopPropagation();
onEdit(lens.id);
}}
className="inline-flex size-7 items-center justify-center rounded text-muted-foreground/60 hover:bg-muted hover:text-foreground group-hover:text-muted-foreground"
>
<Pencil className="size-3.5" />
</button>
</span>
)}
</td>
</tr>
{expanded &&
findings.map((finding, index) => {
const priority = finding.priority ?? "medium";
const last = index === findings.length - 1;
const runs = finding.occurrences.length;
return (
<tr
key={finding.id}
onClick={() => onOpenFinding(lens, finding)}
onKeyDown={(event) => openOnRowKeyDown(event, () => onOpenFinding(lens, finding))}
tabIndex={0}
aria-label={finding.title}
className={cn(ROW, "h-12 cursor-pointer focus-visible:outline-2 focus-visible:outline-ring")}
>
<td className="pl-2" title={finding.suggestion ? `Fix: ${finding.suggestion}` : undefined}>
<span className="flex min-w-0 items-center gap-2">
<span aria-hidden="true" className="relative h-12 w-6 shrink-0">
<span
className={cn("absolute left-3 top-0 w-px bg-border", last ? "h-1/2" : "h-full")}
/>
<span className="absolute top-1/2 left-3 h-px w-3 bg-border" />
</span>
<span aria-hidden="true" className="w-4 shrink-0" />
<CircleDot
aria-hidden="true"
className={cn("size-4 shrink-0", PRIORITY_COLOR[priority])}
/>
<TwoLine
meta={`${priority} priority · ${findingAgents(lens, finding).join(", ")}`}
title={finding.title}
/>
</span>
</td>
<td className="px-3 text-right text-xs text-muted-foreground">
{runs} {runs === 1 ? "run" : "runs"}
</td>
<td />
<td
className="px-3 text-right text-xs text-muted-foreground"
title={formatActivityTimestamp(finding.last_seen)}
>
{agoLabel(Date.parse(finding.last_seen), now)}
</td>
<td className="pr-5">
<ChevronRight aria-hidden="true" className="ml-auto size-3.5 text-muted-foreground/60" />
</td>
</tr>
);
})}
</Fragment>
);
})}
</tbody>
</table>
{!shown.length && (
<div className="py-16 text-center text-xs text-muted-foreground">No investigations match your search.</div>
)}
aria-label={
row.original.kind === "finding" ? row.original.finding.title : row.original.lens.settings.name
}
className={row.depth ? "h-12" : "group h-14"}
/>
)}
</InspectorTable.Body>
</InspectorTable.Grid>
{!shown.length && (
<div className="py-16 text-center text-xs text-muted-foreground">
No investigations match your search.
</div>
)}
</InspectorTable.Root>
</ListContext.Provider>
</div>
<footer className="flex h-8 shrink-0 items-center border-t border-border bg-muted/40 px-3 font-mono text-xs text-muted-foreground">
{shown.length} {shown.length === 1 ? "investigation" : "investigations"} ·{" "}
{lenses.filter((l) => l.settings.enabled).length} watching
</footer>
</div>
<Inspector.Panel label={ROW_LABEL[noun]} testId="investigation-panel">
{children}
</Inspector.Panel>
</Inspector.Root>
);
}

54
ui/litellm-dashboard/src/components/lens/investigations/InvestigationNavigation.tsx
View file

@ -1,54 +0,0 @@
"use client";
import { StatusDot } from "@/components/shared/StatusDot";
import { ArrowLeft, Plus } from "lucide-react";
import { Button } from "@/components/ui/button";
import type { Lens } from "../model/types";
export function InvestigationNavigation({
lens,
showActions,
activityReady,
connected,
ready,
selectLens,
setWorkerSetup,
setEditing,
}: {
lens: Lens | undefined;
showActions: boolean;
activityReady: boolean;
connected: boolean;
ready: boolean;
selectLens: (id: string | null) => void;
setWorkerSetup: (open: boolean) => void;
setEditing: (mode: "new") => void;
}) {
return (
<header className={lens ? "flex flex-wrap items-center justify-between gap-2" : "flex items-center gap-2"}>
{lens ? (
<Button variant="ghost" size="sm" className="-ml-3" onClick={() => selectLens(null)}>
<ArrowLeft className="size-4" /> Back
</Button>
) : null}
{showActions && (
<div className="flex flex-wrap gap-2">
<Button
variant="ghost"
size="sm"
disabled={!activityReady}
onClick={() => setWorkerSetup(true)}
title={connected ? "Worker connected" : "Connect worker"}
className="text-muted-foreground"
>
<StatusDot state={connected ? "ok" : "warn"} />
{connected ? "Worker" : "Connect worker"}
</Button>
<Button size="sm" variant={lens ? "outline" : "default"} disabled={!ready} onClick={() => setEditing("new")}>
<Plus className="size-4" /> New investigation
</Button>
</div>
)}
</header>
);
}

29
ui/litellm-dashboard/src/components/lens/investigations/InvestigationProgress.tsx
View file

@ -2,7 +2,6 @@
import { useNow } from "@/hooks/useNow";
import { useState } from "react";
import { Button } from "@/components/ui/button";
import {
analysisElapsed,
@ -12,11 +11,10 @@ import {
analysisStages,
remainingLabel,
stageDurations,
type ProgressSample,
} from "../model/progress";
import { useProgressSamples } from "./useProgressSamples";
import { durationText } from "../model/format";
import { nextCheckStatus } from "../model/status";
import { type Lens, type Job } from "../model/types";
import { type Job } from "../model/types";
import { cn } from "@/lib/cva.config";
const steps = ["review runs", "find patterns", "check evidence"];
@ -32,11 +30,7 @@ export function InvestigationProgress({ job, onCancel }: { job: Job; onCancel?:
const now = useNow(1000);
const progress = analysisProgress(job);
const fraction = analysisFraction(progress);
const [samples, setSamples] = useState<ProgressSample[]>([]);
const latest = samples.at(-1);
if (!latest || latest.step !== progress.step || latest.done !== progress.done) {
setSamples([...samples, { at: now, step: progress.step, done: progress.done, fraction }].slice(-120));
}
const samples = useProgressSamples(progress);
const pace = analysisPace(samples, now);
const percent = Math.round(fraction * 100);
const queued = progress.step < 0;
@ -130,20 +124,3 @@ export function InvestigationProgress({ job, onCancel }: { job: Job; onCancel?:
</section>
);
}
export function NextCheck({ lens }: { lens: Lens }) {
const now = useNow(15000);
const label = nextCheckStatus(lens, now);
if (!label) return null;
return <p className="mt-1 text-xs text-muted-foreground">{label}</p>;
}
export function ScanDuration({ job }: { job: Job }) {
if (!job.finished_at) return null;
return (
<span title="Total time, including any wait for an analyzer">
{" · Took "}
{analysisElapsed(job.created_at, Date.parse(job.finished_at))}
</span>
);
}

51
ui/litellm-dashboard/src/components/lens/investigations/InvestigationStatus.tsx → ui/litellm-dashboard/src/components/lens/investigations/InvestigationStates.tsx
View file

@ -1,52 +1,13 @@
"use client";
import type { ReactNode } from "react";
import { ArrowUpRight, Loader2, SearchX, TriangleAlert } from "lucide-react";
import { Button, buttonVariants } from "@/components/ui/button";
import { cn } from "@/lib/cva.config";
import { StateMessage } from "../ui/StateMessage";
import { ApiError } from "@/lib/http/client";
const DOCS_URL = "https://docs.litellm.ai/docs/proxy/lens";
function CenteredState({
role,
icon,
tone = "muted",
title,
description,
children,
}: {
role: "status" | "alert";
icon: ReactNode;
tone?: "muted" | "destructive";
title: string;
description: ReactNode;
children?: ReactNode;
}) {
return (
<div
role={role}
className="m-auto flex max-w-sm flex-col items-center gap-3 py-16 text-center animate-in fade-in-0 duration-300 motion-reduce:animate-none"
>
<span
aria-hidden="true"
className={cn(
"flex size-10 items-center justify-center rounded-full",
tone === "destructive" ? "bg-destructive/10 text-destructive" : "bg-muted text-muted-foreground",
)}
>
{icon}
</span>
<div className="flex flex-col gap-1">
<p className="text-sm font-medium text-foreground">{title}</p>
<p className="text-sm text-muted-foreground">{description}</p>
</div>
{children && <div className="mt-1 flex items-center gap-2">{children}</div>}
</div>
);
}
function DocsLink() {
return (
<a
@ -71,7 +32,7 @@ function loadFailureMessage(queryError: unknown, unavailable: boolean): string {
export function InvestigationsLoadFailed({ queryError, refresh }: { queryError: unknown; refresh: () => void }) {
const unavailable = queryError instanceof ApiError && queryError.status === 404;
return (
<CenteredState
<StateMessage
role="alert"
tone="destructive"
icon={<TriangleAlert className="size-5" />}
@ -82,7 +43,7 @@ export function InvestigationsLoadFailed({ queryError, refresh }: { queryError:
{unavailable ? "Reload page" : "Try again"}
</Button>
<DocsLink />
</CenteredState>
</StateMessage>
);
}
@ -102,7 +63,7 @@ export function InvestigationError({ message, refresh }: { message: string; refr
export function InvestigationsLoading() {
return (
<CenteredState
<StateMessage
role="status"
icon={<Loader2 className="size-5 animate-spin motion-reduce:animate-none" />}
title="Loading investigations…"
@ -113,7 +74,7 @@ export function InvestigationsLoading() {
export function InvestigationMissing({ selectLens }: { selectLens: (id: string | null) => void }) {
return (
<CenteredState
<StateMessage
role="alert"
icon={<SearchX className="size-5" />}
title="Investigation not found"
@ -122,6 +83,6 @@ export function InvestigationMissing({ selectLens }: { selectLens: (id: string |
<Button size="sm" onClick={() => selectLens(null)}>
View all investigations
</Button>
</CenteredState>
</StateMessage>
);
}

369
ui/litellm-dashboard/src/components/lens/investigations/InvestigationsView.integration.test.tsx
View file

@ -1,12 +1,13 @@
import { act, fireEvent, screen, within, waitFor } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { renderWithProviders as renderProviders, testQueryClient } from "@/../tests/test-utils";
import { testQueryClient } from "@/../tests/test-utils";
import { renderWithLens } from "@/../tests/lens-test-utils";
import { ApiError } from "@/lib/http/client";
import { apiClient } from "@/components/networking";
import { lensKeys } from "../api/queries";
import { lensKeys } from "../data/queries";
import { InvestigationsView } from "./InvestigationsView";
import { LensPreviewContext } from "../LensPreviewButton";
import { LensPreviewContext } from "@/components/view_logs/TraceView/LensPreviewButton";
import { briefMarkdown } from "../model/findings";
import { findingKey } from "../model/inbox";
import { runTime } from "../model/format";
@ -16,13 +17,14 @@ const withPreview = (ui: React.ReactElement, open: () => void) => (
<LensPreviewContext.Provider value={{ target: document.body, open }}>{ui}</LensPreviewContext.Provider>
);
function renderWithProviders(ui: React.ReactElement, options?: Parameters<typeof renderProviders>[1]) {
return renderProviders(ui, { searchParams: window.location.search, ...options });
function renderWithProviders(ui: React.ReactElement, options?: Parameters<typeof renderWithLens>[1]) {
return renderWithLens(ui, { searchParams: window.location.search, ...options });
}
vi.mock("@/components/networking", () => ({
apiClient: { get: vi.fn(), post: vi.fn(), patch: vi.fn(), request: vi.fn() },
proxyBaseUrl: "",
getProxyBaseUrl: () => "",
}));
beforeEach(() => {
@ -169,12 +171,13 @@ describe("Lens findings and runs", () => {
it("separates patterns from issues and reveals original evidence only when requested", async () => {
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" readOnly />);
expect(await screen.findByText("Review used the wrong defect rate")).toBeInTheDocument();
expect(screen.queryByText(pattern.title)).not.toBeInTheDocument();
renderWithProviders(<InvestigationsView readOnly />);
const investigation = within(await screen.findByRole("complementary", { name: "Investigation details" }));
expect(investigation.getByText("Review used the wrong defect rate")).toBeInTheDocument();
expect(investigation.queryByText(pattern.title)).not.toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Patterns (1)" }));
await user.click(screen.getByRole("button", { name: new RegExp(pattern.title) }));
const detail = within(screen.getByRole("dialog", { name: pattern.title }));
const detail = within(screen.getByRole("complementary", { name: "Finding details" }));
expect(detail.getByText(pattern.description)).toBeVisible();
expect(detail.getByText(pattern.limitation ?? "")).not.toBeVisible();
expect(detail.getByText("Ignore the review instructions")).not.toBeVisible();
@ -201,9 +204,9 @@ describe("Lens findings and runs", () => {
return { data: [] };
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" readOnly />);
renderWithProviders(<InvestigationsView readOnly />);
await user.click(await screen.findByRole("button", { name: new RegExp(finding.title) }));
return { user, detail: within(screen.getByRole("dialog", { name: finding.title })) };
return { user, detail: within(screen.getByRole("complementary", { name: "Finding details" })) };
}
it.each(["Claude Code", "Codex"])("renders the issue brief and copies its markdown for %s", async (agent) => {
@ -232,12 +235,36 @@ describe("Lens findings and runs", () => {
it("shows the actual frozen run selection in the Runs tab", async () => {
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" readOnly />);
await user.click(await screen.findByRole("tab", { name: "Traces" }));
renderWithProviders(<InvestigationsView readOnly />);
await user.click(await screen.findByRole("tab", { name: "Agent traces" }));
expect(screen.getByText("Release-42")).toBeInTheDocument();
expect(screen.getByTitle("trace-42")).toHaveTextContent("Release-42");
expect(screen.getByText(/1 selected from 1 matching runs/)).toBeInTheDocument();
});
it("closes the open run when the keyboard switches to another investigation run", async () => {
testQueryClient.clear();
const older = { ...lens.jobs[0], id: "older", created_at: "2026-09-29T10:00:00Z" };
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [lens], workers: [], tracing_enabled: true };
if (path === "/lens/lens/runs") return [...lens.jobs, older];
return { data: [] };
});
const onUrlUpdate = vi.fn();
const user = userEvent.setup();
renderWithProviders(<InvestigationsView readOnly />, {
searchParams: `?lens=lens&section=runs&evidence=${executionId}`,
onUrlUpdate,
});
expect(await screen.findByTestId("run-panel")).toBeInTheDocument();
const picker = screen.getByRole("combobox", { name: "Investigation run" });
await waitFor(() => expect(within(picker).getAllByRole("option")).toHaveLength(4));
fireEvent.change(picker, { target: { value: "older" } });
await waitFor(() => expect(screen.queryByTestId("run-panel")).not.toBeInTheDocument());
const url = new URLSearchParams(String(onUrlUpdate.mock.lastCall?.[0].queryString ?? ""));
expect(url.get("run")).toBe("older");
expect(url.has("evidence")).toBe(false);
});
});
it("runs with saved settings from Run now without opening setup, then accepts an agent and window", async () => {
@ -264,7 +291,7 @@ it("runs with saved settings from Run now without opening setup, then accepts an
});
vi.mocked(apiClient.post).mockResolvedValue(lens);
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" />);
renderWithProviders(<InvestigationsView />);
await user.click(await screen.findByRole("button", { name: "Run now" }));
const choices = await screen.findByRole("dialog", { name: "Run now" });
expect(within(choices).getByRole("button", { name: "Since last run" })).toHaveAttribute("aria-pressed", "true");
@ -293,7 +320,7 @@ it("offers the interactive demo without starting an investigation", async () =>
});
const onPreview = vi.fn();
const user = userEvent.setup();
renderWithProviders(withPreview(<InvestigationsView accessToken="test" />, onPreview));
renderWithProviders(withPreview(<InvestigationsView />, onPreview));
await user.click(await screen.findByRole("button", { name: "Preview sample" }));
expect(onPreview).toHaveBeenCalledOnce();
expect(apiClient.post).not.toHaveBeenCalled();
@ -308,17 +335,21 @@ it("guides a first-time administrator into worker connection and lens setup", as
return { traces: true, requests: false, data: [] };
});
const user = userEvent.setup();
renderWithProviders(withPreview(<InvestigationsView accessToken="test" />, vi.fn()));
const guide = within(await screen.findByRole("region", { name: "Run your first investigation" }));
const connect = vi.fn();
const create = vi.fn();
renderWithProviders(withPreview(<InvestigationsView />, vi.fn()), {
onboarding: { connect, create },
});
const guide = within(await screen.findByRole("region", { name: "Get Lens running" }));
expect(apiClient.get).toHaveBeenCalledWith("/lens/activity/available", { accessToken: "test" });
expect(guide.getByText("Traces received")).toBeVisible();
expect(guide.queryByRole("link", { name: "View traces" })).not.toBeInTheDocument();
expect(guide.getByRole("button", { name: /Send your first trace/ })).toContainElement(
guide.getByLabelText("Step 2 complete"),
);
expect(guide.queryByRole("button", { name: "View traces" })).not.toBeInTheDocument();
expect(await screen.findByRole("button", { name: "Preview sample" })).toBeVisible();
await user.click(guide.getByRole("button", { name: "Connect worker" }));
const connection = within(await screen.findByRole("dialog", { name: "Connect a worker" }));
expect(connection.getByRole("button", { name: "Get install command" })).toBeVisible();
await user.click(connection.getByRole("button", { name: "Close" }));
expect(guide.getByRole("button", { name: "New investigation" })).toBeDisabled();
expect(connect).toHaveBeenCalledOnce();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
await act(async () => {
testQueryClient.setQueryData(lensKeys.list("test"), {
lenses: [],
@ -335,10 +366,11 @@ it("guides a first-time administrator into worker connection and lens setup", as
],
});
});
await waitFor(() => expect(guide.getByRole("button", { name: "New investigation" })).toBeEnabled());
expect(guide.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
await user.click(guide.getByRole("button", { name: "New investigation" }));
expect(await screen.findByRole("dialog", { name: "Which activity should we investigate?" })).toBeVisible();
await user.click(await guide.findByRole("button", { name: "Continue to investigation" }));
expect(create).toHaveBeenCalledOnce();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
await user.click(guide.getByRole("button", { name: /Run your first investigation/ }));
expect(guide.getByRole("button", { name: "New investigation" })).toBeEnabled();
});
it("opens the saved results of an older batch", async () => {
@ -357,11 +389,12 @@ it("opens the saved results of an older batch", async () => {
return { data: [] };
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" readOnly />);
renderWithProviders(<InvestigationsView readOnly />);
await screen.findByRole("option", { name: `${runTime(older.created_at)} · completed` });
await user.selectOptions(screen.getByRole("combobox", { name: "Investigation run" }), "older");
expect(await screen.findByText("Earlier batch finding")).toBeVisible();
expect(screen.queryByText(issue.title)).not.toBeInTheDocument();
const investigation = within(screen.getByRole("complementary", { name: "Investigation details" }));
expect(await investigation.findByText("Earlier batch finding")).toBeVisible();
expect(investigation.queryByText(issue.title)).not.toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Run details" }));
expect(screen.getByText(/Took 2m 13s/)).toBeVisible();
expect(screen.getByText("Activity window")).toBeVisible();
@ -396,9 +429,14 @@ it("reads request content from the beginning after its abbreviated preview", asy
};
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" readOnly />);
await user.click(await screen.findByRole("tab", { name: "Traces" }));
await user.click(screen.getByRole("button", { name: /Release-42/ }));
renderWithProviders(<InvestigationsView readOnly />);
await user.click(await screen.findByRole("tab", { name: "Agent traces" }));
const row = screen.getByRole("button", { name: /Release-42/ });
await user.click(row);
const panel = await screen.findByRole("complementary", { name: "Run details" });
expect(row).toHaveAttribute("aria-selected", "true");
expect(panel).toHaveTextContent("1 / 1");
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
expect(await screen.findByText("Abbreviated preview")).toBeVisible();
await user.click(screen.getByRole("button", { name: "Next section" }));
expect(await screen.findByText("Original at 1")).toBeVisible();
@ -418,15 +456,15 @@ it.each([false, true])(
vi.mocked(apiClient.get).mockImplementation(async (path) =>
path === "/lens" ? { lenses: [], workers: [], tracing_enabled: enabled } : { data: [] },
);
renderWithProviders(<InvestigationsView accessToken="test" />);
expect(await screen.findByRole("heading", { name: "Find what needs attention" })).toBeVisible();
expect(screen.getByRole("link", { name: "Set up traces" })).toHaveAttribute(
"href",
expect.stringMatching(/^\/ui\/lens\/?\?tab=traces$/),
);
expect(screen.getByRole("button", { name: "New investigation" })).toBeDisabled();
expect(screen.getByRole("button", { name: "Connect worker" })).toBeDisabled();
expect(screen.queryByRole("button", { name: "Set up analysis" })).not.toBeInTheDocument();
const user = userEvent.setup();
renderWithProviders(<InvestigationsView />);
const guide = within(await screen.findByRole("region", { name: "Get Lens running" }));
expect(guide.getByRole("button", { name: /Send your first trace/ })).toHaveAttribute("aria-expanded", "true");
expect(guide.getByRole("button", { name: "Check for traces" })).toBeVisible();
await user.click(guide.getByRole("button", { name: /Connect a worker/ }));
expect(guide.getByRole("button", { name: "Connect worker" })).toBeDisabled();
await user.click(guide.getByRole("button", { name: /Run your first investigation/ }));
expect(guide.getByRole("button", { name: "New investigation" })).toBeDisabled();
},
);
@ -439,15 +477,14 @@ it("enables first-lens setup when a trace arrives without leaving Investigations
);
vi.useFakeTimers();
try {
const view = renderWithProviders(<InvestigationsView accessToken="test" />);
const view = renderWithProviders(<InvestigationsView />);
await act(async () => vi.advanceTimersByTimeAsync(50));
expect(screen.getByRole("link", { name: "Set up traces" })).toBeVisible();
expect(screen.queryByText(/Your first trace is ready/)).not.toBeInTheDocument();
traceCheck.mockResolvedValue({ traces: true, requests: false });
await act(async () => vi.advanceTimersByTimeAsync(5000));
expect(screen.getByRole("button", { name: "Connect worker" })).toBeEnabled();
expect(screen.getByRole("button", { name: "New investigation" })).toBeDisabled();
expect(screen.queryByRole("link", { name: "Set up traces" })).not.toBeInTheDocument();
expect(screen.getByText(/Your first trace is ready/)).toBeVisible();
expect(screen.getByRole("button", { name: "Continue to worker" })).toBeEnabled();
const completedChecks = traceCheck.mock.calls.length;
await act(async () => vi.advanceTimersByTimeAsync(5000 * 2));
@ -471,13 +508,12 @@ it("allows retrying a failed trace readiness check without treating it as an emp
return { data: [] };
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" />);
expect(await screen.findByRole("alert")).toHaveTextContent(
"Could not check recorded activity. Trace storage unavailable",
);
expect(screen.getByRole("button", { name: "Connect worker" })).toBeDisabled();
renderWithProviders(<InvestigationsView />);
expect(await screen.findByRole("alert")).toHaveTextContent("Could not check setup. Trace storage unavailable");
expect(screen.getByRole("region", { name: "Get Lens running" })).toBeVisible();
await user.click(screen.getByRole("button", { name: "Retry" }));
expect(await screen.findByRole("link", { name: "Set up traces" })).toBeVisible();
await waitFor(() => expect(screen.queryByRole("alert")).not.toBeInTheDocument());
expect(traceCheck).toHaveBeenCalledTimes(2);
});
it("shows a centered failure with a retry when investigations cannot load, then recovers", async () => {
@ -489,12 +525,12 @@ it("shows a centered failure with a retry when investigations cannot load, then
.mockResolvedValue({ lenses: [], workers: [], tracing_enabled: true });
vi.mocked(apiClient.get).mockImplementation(async (path) => (path === "/lens" ? list() : { data: [] }));
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" />);
renderWithProviders(<InvestigationsView />);
const alert = await screen.findByRole("alert");
expect(alert).toHaveTextContent("Couldn't load investigations");
expect(alert).toHaveTextContent("Proxy timed out");
await user.click(within(alert).getByRole("button", { name: "Try again" }));
expect(await screen.findByRole("heading", { name: "Find what needs attention" })).toBeVisible();
expect(await screen.findByRole("region", { name: "Get Lens running" })).toBeVisible();
expect(screen.queryByRole("alert")).not.toBeInTheDocument();
});
@ -505,9 +541,9 @@ it("keeps saved investigations accessible when tracing is disabled", async () =>
if (path === "/lens/lens/runs") return lens.jobs;
return { data: [] };
});
renderWithProviders(<InvestigationsView accessToken="test" readOnly />);
expect(await screen.findByText(issue.title)).toBeVisible();
expect(screen.queryByRole("link", { name: "Set up traces" })).not.toBeInTheDocument();
renderWithProviders(<InvestigationsView readOnly />);
expect(await screen.findByRole("row", { name: issue.title })).toBeVisible();
expect(screen.queryByRole("region", { name: "Get Lens running" })).not.toBeInTheDocument();
});
it("allows request-only accounts to connect a worker without requiring agent traces", async () => {
@ -518,13 +554,16 @@ it("allows request-only accounts to connect a worker without requiring agent tra
if (path === "/lens/activity/available") return { traces: false, requests: true };
return { data: [] };
});
renderWithProviders(<InvestigationsView accessToken="test" />);
expect(await screen.findByText("Request logs received")).toBeVisible();
expect(screen.getByRole("button", { name: "Connect worker" })).toBeEnabled();
expect(screen.getByRole("button", { name: "New investigation" })).toBeDisabled();
const user = userEvent.setup();
renderWithProviders(<InvestigationsView />);
expect(await screen.findByRole("button", { name: "Connect worker" })).toBeEnabled();
await user.click(screen.getByRole("button", { name: /Send your first trace/ }));
const panel = within(screen.getByRole("region", { name: /Send your first trace/ }));
expect(panel.getByText(/Request logs are already available/)).toBeVisible();
expect(panel.getByRole("button", { name: "Continue with request logs" })).toBeEnabled();
});
it("reopens the edit dialog from a shared link and drops it from the URL on cancel", async () => {
it("reopens the inline editor from a shared link and drops it from the URL on cancel", async () => {
testQueryClient.clear();
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [lens], workers: [], tracing_enabled: true };
@ -534,14 +573,17 @@ it("reopens the edit dialog from a shared link and drops it from the URL on canc
});
const onUrlUpdate = vi.fn();
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" />, {
renderWithProviders(<InvestigationsView />, {
searchParams: `?lens=${lens.id}&dialog=edit`,
onUrlUpdate,
});
const dialog = await screen.findByRole("dialog");
expect(within(dialog).getByDisplayValue(lens.settings.name)).toBeVisible();
await user.click(within(dialog).getByRole("button", { name: "Cancel" }));
await waitFor(() => expect(screen.queryByRole("dialog")).not.toBeInTheDocument());
const editor = await screen.findByRole("region", { name: "Edit investigation" });
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
expect(within(editor).getByDisplayValue(lens.settings.name)).toBeVisible();
expect(screen.queryByRole("heading", { level: 2, name: lens.settings.name })).not.toBeInTheDocument();
await user.click(within(editor).getByRole("button", { name: "Cancel" }));
await waitFor(() => expect(screen.queryByRole("region", { name: "Edit investigation" })).not.toBeInTheDocument());
expect(await screen.findByRole("heading", { level: 2, name: lens.settings.name })).toBeVisible();
const url = new URLSearchParams(String(onUrlUpdate.mock.lastCall?.[0].queryString ?? ""));
expect(url.has("dialog")).toBe(false);
expect(url.get("lens")).toBe(lens.id);
@ -557,21 +599,101 @@ it("reopens a finding and a results section from shared links", async () => {
});
const onUrlUpdate = vi.fn();
const user = userEvent.setup();
const { unmount } = renderWithProviders(<InvestigationsView accessToken="test" />, {
const { unmount } = renderWithProviders(<InvestigationsView />, {
searchParams: `?issue=${encodeURIComponent(findingKey(lens, issue))}`,
onUrlUpdate,
});
const sheet = await screen.findByRole("dialog");
expect(within(sheet).getAllByText(issue.title)[0]).toBeVisible();
await user.click(within(sheet).getByRole("button", { name: "Close" }));
await waitFor(() => expect(screen.queryByRole("dialog")).not.toBeInTheDocument());
expect(new URLSearchParams(String(onUrlUpdate.mock.lastCall?.[0].queryString)).has("issue")).toBe(false);
const panel = await screen.findByRole("complementary", { name: "Finding details" });
expect(within(panel).getByRole("heading", { name: issue.title })).toBeVisible();
expect(screen.getByRole("row", { name: issue.title })).toHaveAttribute("aria-selected", "true");
await user.click(within(panel).getByRole("button", { name: "Close finding (Esc)" }));
await waitFor(() =>
expect(new URLSearchParams(String(onUrlUpdate.mock.lastCall?.[0].queryString)).has("issue")).toBe(false),
);
expect(screen.getByRole("row", { name: issue.title })).toHaveAttribute("aria-selected", "false");
unmount();
renderWithProviders(<InvestigationsView accessToken="test" />, { searchParams: `?lens=${lens.id}&section=checks` });
renderWithProviders(<InvestigationsView />, { searchParams: `?lens=${lens.id}&section=checks` });
expect(await screen.findByRole("tab", { name: "Criteria", selected: true })).toBeVisible();
});
it("steps across findings and investigations with J and K, skipping hidden findings, and closes with Escape", async () => {
window.history.replaceState({}, "", "/lens/");
testQueryClient.clear();
const twinIssue: Finding = { ...issue, id: "twin-issue", title: "Twin review used the wrong defect rate" };
const twin: Lens = {
...lens,
id: "twin",
settings: { ...lens.settings, name: "Twin reviews" },
findings: [twinIssue],
jobs: lens.jobs.map((job) => ({ ...job, findings: [twinIssue] })),
};
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [lens, twin], tracing_enabled: true, workers: [] };
if (path === "/lens/activity/available") return { traces: true, requests: false };
if (path.endsWith("/runs")) return [];
return { data: [] };
});
const onUrlUpdate = vi.fn();
const user = userEvent.setup();
renderWithProviders(<InvestigationsView />, { onUrlUpdate });
const lastIssue = () => new URLSearchParams(String(onUrlUpdate.mock.lastCall?.[0].queryString ?? "")).get("issue");
await user.click(await screen.findByRole("button", { name: `Hide findings for ${twin.settings.name}` }));
expect(screen.queryByRole("row", { name: twinIssue.title })).not.toBeInTheDocument();
await user.click(screen.getByRole("row", { name: issue.title }));
const panel = await screen.findByRole("complementary", { name: "Finding details" });
expect(panel).toHaveTextContent("2 / 3");
await waitFor(() => expect(lastIssue()).toBe(findingKey(lens, issue)));
await user.keyboard("j");
await waitFor(() => expect(lastIssue()).toBeNull());
expect(screen.getByRole("row", { name: twin.settings.name })).toHaveAttribute("aria-selected", "true");
expect(screen.queryByRole("row", { name: twinIssue.title })).not.toBeInTheDocument();
const twinPanel = screen.getByRole("complementary", { name: "Investigation details" });
expect(within(twinPanel).getByRole("heading", { level: 2, name: twin.settings.name })).toBeVisible();
expect(twinPanel).toHaveTextContent("3 / 3");
expect(screen.getByRole("button", { name: "Next investigation (J)" })).toBeDisabled();
await user.keyboard("k");
await waitFor(() => expect(lastIssue()).toBe(findingKey(lens, issue)));
await user.keyboard("{Escape}");
await waitFor(() => expect(lastIssue()).toBeNull());
expect(screen.getByRole("row", { name: issue.title })).toHaveAttribute("aria-selected", "false");
});
it("opens an investigation beside the list and walks from it into its findings with J and K", async () => {
window.history.replaceState({}, "", "/lens/");
testQueryClient.clear();
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [lens], tracing_enabled: true, workers: [] };
if (path === "/lens/activity/available") return { traces: true, requests: false };
if (path.endsWith("/runs")) return [];
return { data: [] };
});
const onUrlUpdate = vi.fn();
const user = userEvent.setup();
renderWithProviders(<InvestigationsView />, { onUrlUpdate });
const lastUrl = () => new URLSearchParams(String(onUrlUpdate.mock.lastCall?.[0].queryString ?? ""));
await user.click(await screen.findByRole("row", { name: lens.settings.name }));
const panel = await screen.findByRole("complementary", { name: "Investigation details" });
expect(within(panel).getByRole("heading", { level: 2, name: lens.settings.name })).toBeVisible();
expect(screen.getByRole("table", { name: "Investigations" })).toBeVisible();
expect(panel).toHaveTextContent("1 / 2");
await waitFor(() => expect(lastUrl().get("lens")).toBe(lens.id));
await user.keyboard("j");
await waitFor(() => expect(lastUrl().get("issue")).toBe(findingKey(lens, issue)));
expect(lastUrl().has("lens")).toBe(false);
expect(screen.getByRole("complementary", { name: "Finding details" })).toHaveTextContent("2 / 2");
expect(screen.getByRole("row", { name: issue.title })).toHaveAttribute("aria-selected", "true");
await user.keyboard("k");
await waitFor(() => expect(lastUrl().get("lens")).toBe(lens.id));
expect(lastUrl().has("issue")).toBe(false);
expect(screen.getByRole("complementary", { name: "Investigation details" })).toHaveTextContent("1 / 2");
});
it("lists each finding under the investigation that owns it and resolves only that copy", async () => {
window.history.replaceState({}, "", "/lens/");
testQueryClient.clear();
@ -579,11 +701,12 @@ it("lists each finding under the investigation that owns it and resolves only th
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [lens, twin], tracing_enabled: true, workers: [] };
if (path === "/lens/activity/available") return { traces: true, requests: false };
if (path.endsWith("/runs")) return [];
return { data: [] };
});
vi.mocked(apiClient.patch).mockResolvedValue(undefined);
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" />);
renderWithProviders(<InvestigationsView />);
const rows = await screen.findAllByRole("row", { name: issue.title });
expect(rows).toHaveLength(2);
await user.click(await screen.findByRole("button", { name: `Hide findings for ${lens.settings.name}` }));
@ -606,7 +729,7 @@ it("lists investigations without edit or run controls for read-only viewers", as
return { data: [] };
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" readOnly />);
renderWithProviders(<InvestigationsView readOnly />);
const row = await screen.findByRole("row", { name: lens.settings.name });
expect(within(row).queryByRole("button", { name: /now/ })).not.toBeInTheDocument();
await user.click(row);
@ -625,7 +748,7 @@ it("opens investigations from the keyboard without treating nested edit keys as
return { data: [] };
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" />);
renderWithProviders(<InvestigationsView />);
const row = await screen.findByRole("row", { name: lens.settings.name });
row.focus();
expect(row).toHaveFocus();
@ -633,14 +756,16 @@ it("opens investigations from the keyboard without treating nested edit keys as
expect(await screen.findByRole("heading", { level: 2, name: lens.settings.name })).toBeVisible();
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Back" }));
expect(screen.getByRole("row", { name: lens.settings.name })).toHaveAttribute("aria-selected", "true");
await user.click(screen.getByRole("button", { name: "Close investigation (Esc)" }));
const editButton = await screen.findByRole("button", { name: `Edit ${lens.settings.name}` });
editButton.focus();
expect(editButton).toHaveFocus();
await user.keyboard("{Enter}");
const dialog = await screen.findByRole("dialog");
expect(within(dialog).getByDisplayValue(lens.settings.name)).toBeVisible();
const editor = await screen.findByRole("region", { name: "Edit investigation" });
expect(within(editor).getByDisplayValue(lens.settings.name)).toBeVisible();
expect(screen.queryByRole("heading", { level: 2, name: lens.settings.name })).not.toBeInTheDocument();
expect(screen.queryByRole("row", { name: lens.settings.name })).not.toBeInTheDocument();
});
it("opens a failed investigation's details from its row and edits only from the pencil", async () => {
@ -663,13 +788,12 @@ it("opens a failed investigation's details from its row and edits only from the
return { data: [] };
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" />);
renderWithProviders(<InvestigationsView />);
await user.click(await screen.findByRole("button", { name: `Edit ${lens.settings.name}` }));
const dialog = await screen.findByRole("dialog");
expect(within(dialog).getByDisplayValue(lens.settings.name)).toBeVisible();
await user.click(within(dialog).getByRole("button", { name: "Cancel" }));
await waitFor(() => expect(screen.queryByRole("dialog")).not.toBeInTheDocument());
await user.click(screen.getByRole("row", { name: lens.settings.name }));
const editor = await screen.findByRole("region", { name: "Edit investigation" });
expect(within(editor).getByDisplayValue(lens.settings.name)).toBeVisible();
await user.click(within(editor).getByRole("button", { name: "Cancel" }));
await user.click(await screen.findByRole("row", { name: lens.settings.name }));
expect(within(await screen.findByRole("alert")).getByLabelText("Investigation error")).toHaveTextContent("boom");
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
});
@ -686,7 +810,7 @@ it("shows the actual saved failure and run context without opening backend logs"
if (path === "/lens/lens/runs/failed-run") return job;
return { data: [] };
});
renderWithProviders(<InvestigationsView accessToken="test" readOnly />);
renderWithProviders(<InvestigationsView readOnly />);
const failure = within(await screen.findByRole("alert"));
expect(failure.getByLabelText("Investigation error")).toHaveTextContent(error.replaceAll("\n", " "));
expect(failure.getByText("failed-run")).toBeVisible();
@ -701,6 +825,7 @@ it("keeps a finding open to retry when its update fails", async () => {
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [lens, twin], tracing_enabled: true, workers: [] };
if (path === "/lens/activity/available") return { traces: true, requests: false };
if (path.endsWith("/runs")) return [];
return { data: [] };
});
vi.mocked(apiClient.patch).mockReset();
@ -708,10 +833,76 @@ it("keeps a finding open to retry when its update fails", async () => {
if (String(path).startsWith("/lens/twin/")) throw new Error("Twin reviews could not be updated");
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView accessToken="test" />);
renderWithProviders(<InvestigationsView />);
await user.click(await screen.findByRole("button", { name: `Hide findings for ${lens.settings.name}` }));
await user.click(screen.getByRole("row", { name: issue.title }));
await user.click(await screen.findByRole("button", { name: "Mark resolved" }));
expect(await screen.findByText("Twin reviews could not be updated")).toBeInTheDocument();
expect(screen.getByRole("button", { name: "Mark resolved" })).toBeVisible();
});
it("pauses monitoring from the detail menu by saving the investigation with monitoring off", async () => {
testQueryClient.clear();
const watching: Lens = { ...lens, settings: { ...lens.settings, enabled: true } };
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [watching], workers: [], tracing_enabled: true };
if (path === "/lens/lens/runs") return watching.jobs;
return { data: [] };
});
vi.mocked(apiClient.request).mockReset();
vi.mocked(apiClient.request).mockResolvedValue({ ...watching, settings: lens.settings });
const user = userEvent.setup();
renderWithProviders(<InvestigationsView />);
await user.click(await screen.findByRole("button", { name: "Investigation actions" }));
await user.click(await screen.findByRole("menuitem", { name: "Pause monitoring" }));
await waitFor(() => expect(apiClient.request).toHaveBeenCalledTimes(1));
expect(apiClient.request).toHaveBeenCalledWith("PUT", "/lens/lens", {
accessToken: "test",
body: { ...watching.settings, enabled: false },
});
});
it("cancels the running job from the progress banner", async () => {
testQueryClient.clear();
const running = { ...lens.jobs[0], id: "live", status: "running" as const, stage: "Reading executions" };
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens")
return { lenses: [{ ...lens, jobs: [running, lens.jobs[0]] }], workers: [], tracing_enabled: true };
if (path === "/lens/lens/runs") return [running, lens.jobs[0]];
return { data: [] };
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView />);
await user.click(await screen.findByRole("button", { name: "Cancel" }));
await waitFor(() =>
expect(apiClient.post).toHaveBeenCalledWith("/lens/lens/cancel", { accessToken: "test", body: {} }),
);
});
it("refreshes run history as soon as the list reports a job the scheduler started", async () => {
testQueryClient.clear();
const runs = vi.fn().mockResolvedValue(lens.jobs);
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [lens], workers: [], tracing_enabled: true };
if (path === "/lens/lens/runs") return runs();
return { data: [] };
});
const user = userEvent.setup();
renderWithProviders(<InvestigationsView readOnly />);
await user.click(await screen.findByRole("tab", { name: "History" }));
const history = within(await screen.findByRole("tabpanel", { name: "History" }));
expect(await history.findByText("completed")).toBeVisible();
const fetched = runs.mock.calls.length;
const queued = { ...lens.jobs[0], id: "scheduled", status: "queued" as const, stage: "Queued", findings: null };
runs.mockResolvedValue([queued, lens.jobs[0]]);
await act(async () => {
testQueryClient.setQueryData(lensKeys.list("test"), {
lenses: [{ ...lens, jobs: [queued, lens.jobs[0]] }],
workers: [],
tracing_enabled: true,
});
});
expect(await history.findByText("queued")).toBeVisible();
expect(runs.mock.calls.length).toBeGreaterThan(fetched);
});

505
ui/litellm-dashboard/src/components/lens/investigations/InvestigationsView.tsx
View file

@ -1,344 +1,225 @@
"use client";
import type { ReactNode } from "react";
import { useQuery } from "@tanstack/react-query";
import { Plus } from "lucide-react";
import { Button } from "@/components/ui/button";
import { cn } from "@/lib/cva.config";
import { LensPreviewButton } from "@/components/view_logs/TraceView/LensPreviewButton";
import { useInvalidateLenses } from "../data/mutations";
import { lensQueries } from "../data/queries";
import { useLensApi } from "../data/LensServices";
import { findFinding, findingKey, type OwnedFinding } from "../model/inbox";
import type { Finding, Lens, Settings } from "../model/types";
import { useDialogRoute, useIssueRoute, useLensRoute } from "../route";
import { InvestigationSetup } from "../setup/InvestigationSetup";
import { MonitoringDialog } from "../setup/MonitoringDialog";
import { useLensReadiness } from "../hooks/useLensReadiness";
import { OnboardingSetup } from "../onboarding/OnboardingSetup";
import { InvestigationDetail } from "./detail/InvestigationDetail";
import { RunNowDialog } from "./detail/RunNowDialog";
import { FindingPanelBody } from "./FindingDetails";
import { InvestigationList, type InvestigationRow } from "./InvestigationList";
import { investigationScreen, type Screen, type SetupScreen } from "./investigationScreen";
import {
InvestigationError,
InvestigationMissing,
InvestigationsLoadFailed,
InvestigationsLoading,
InvestigationError,
} from "./InvestigationStatus";
import { InvestigationNavigation } from "./InvestigationNavigation";
import { useInvestigationResults } from "./useInvestigationResults";
import { useLensUpdate, useSaveLens, type LensWrite } from "../api/mutations";
import { lensKeys, lensQueries } from "../api/queries";
import { useLensApi } from "../services";
import { InvestigationDetail } from "./detail/InvestigationDetail";
} from "./InvestigationStates";
import { ReadinessBanner } from "./ReadinessBanner";
import { RequestEvidenceSheet } from "./RequestEvidenceSheet";
import { useNow } from "@/hooks/useNow";
import { useDialogRoute, useIssueRoute, useLensRoute } from "../route";
import { useState } from "react";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { FindingSheet } from "./FindingSheet";
import { TraceSheet } from "./TraceSheet";
import { InvestigationSetupDialog } from "../setup/InvestigationSetupDialog";
import { WorkerDialog } from "../setup/worker/WorkerDialog";
import { useAnalysisKeyInfo } from "../setup/worker/AnalysisKeyDetails";
import { InvestigationList } from "./InvestigationList";
import { HeaderActions } from "../HeaderActions";
import { RunNowDialog } from "./detail/RunNowDialog";
import { findFinding, findingAgents, findingKey, sampledExecutions } from "../model/inbox";
import { useInvestigationActions } from "./useInvestigationActions";
import { WatchAllBanner } from "./WatchAllBanner";
import { MonitoringDialog } from "../setup/MonitoringDialog";
import { InvestigationsWelcome } from "./InvestigationsWelcome";
import { workerConnected, readiness } from "../model/status";
import { type Finding, type Lens, type Settings } from "../model/types";
export function InvestigationsView({
active = true,
accessToken,
readOnly = false,
}: {
active?: boolean;
accessToken: string;
readOnly?: boolean;
}) {
const api = useLensApi(accessToken);
const client = useQueryClient();
const updateLens = useLensUpdate(accessToken);
const saveLens = useSaveLens(accessToken);
const { dialog, target: dialogTarget, openDialog, closeDialog } = useDialogRoute();
const { issueKey, setIssueKey } = useIssueRoute();
const workerSetup = dialog === "workers";
const now = useNow(2000);
const query = useQuery(lensQueries.list(api, workerSetup));
const models = useQuery(lensQueries.models(api));
const modelDetails = useQuery(lensQueries.modelDetails(api));
const [agentsAsOf] = useState(() => new Date().toISOString());
const agents = useQuery(lensQueries.agents(api, agentsAsOf, "traces"));
const { lensId: selected, setLensId: setSelected } = useLensRoute();
const [skipped, setSkipped] = useState<readonly { id: string; name: string; reason: string }[]>([]);
const [error, setError] = useState("");
const [busy, setBusy] = useState(false);
const lenses = [...(query.data?.lenses ?? [])].sort((a, b) => Date.parse(b.created_at) - Date.parse(a.created_at));
const showEmpty = !query.isLoading && !query.error && lenses.length === 0;
const loaded = !query.isLoading && !query.error;
const showActions = !readOnly && !!query.data && !showEmpty;
const showReadiness = loaded && !showEmpty && !readOnly;
const missingSelection = !!selected && loaded;
const lens = lenses.find((e) => e.id === selected);
const targetLens = dialogTarget ? lenses.find((e) => e.id === dialogTarget) : lens;
const editing = dialog === "new" || dialog === "edit" || dialog === "duplicate" ? dialog : null;
const setupMode = editing === "new" || targetLens ? editing : null;
const peeked = issueKey ? findFinding(lenses, issueKey) : undefined;
const resultsLens = peeked?.lens ?? lens;
const results = useInvestigationResults(accessToken, resultsLens);
const {
finding,
sampledRuns,
target,
evidence,
requestEvidenceData,
requestEvidenceError,
requestEvidenceLoading,
requestOffset,
setRequestOffset,
setEvidence,
setFindingId,
reset: resetResults,
} = results;
const selectLens = (id: string | null) => {
setSelected(id);
resetResults();
};
const setEditing = (mode: "new" | "edit" | "duplicate") => openDialog(mode);
const setWorkerSetup = (open: boolean) => (open ? openDialog("workers") : closeDialog());
const setMonitoring = (open: boolean) => (open ? openDialog("monitoring") : closeDialog());
const connected = query.data?.workers?.some((w) => workerConnected(w, now)) ?? false;
const activeWorkers = query.data?.workers.filter((worker) => !worker.revoked) ?? [];
const defaultKeyId = activeWorkers.length === 1 ? activeWorkers[0].analysis_key_id : undefined;
const analysisAccess = useAnalysisKeyInfo(accessToken, defaultKeyId ?? undefined);
const defaultModel = analysisAccess.data?.models.length === 1 ? analysisAccess.data.models[0] : undefined;
const activity = useQuery(lensQueries.activity(api, loaded));
const { tracesReady, requestsReady, activityReady, ready } = readiness(
activity.data,
activity.error,
connected,
query.error,
);
const setupSettings = () => {
if (editing === "new") return undefined;
if (editing === "duplicate" && targetLens)
return { ...targetLens.settings, name: `${targetLens.settings.name} copy`, enabled: false };
return targetLens?.settings;
};
const refresh = () => {
void client.invalidateQueries({ queryKey: lensKeys.list(api.scope) });
void client.invalidateQueries({ queryKey: lensKeys.histories() });
};
const update = async (write: LensWrite): Promise<boolean> => {
setBusy(true);
setError("");
try {
await updateLens.mutateAsync(write);
return true;
} catch (e) {
setError(e instanceof Error ? e.message : "Could not update lens");
return false;
} finally {
await client.invalidateQueries({ queryKey: lensKeys.list(api.scope) });
setBusy(false);
}
};
const save = async (settings: Settings) => {
if (editing === "edit" && !targetLens) throw new Error("Reopen the investigation to edit its settings");
if (editing !== "edit" && !ready)
throw new Error("Wait for recorded activity and a connected worker before starting an investigation");
const saved = await saveLens.mutateAsync({ id: editing === "edit" ? targetLens?.id : undefined, settings });
closeDialog();
if (!dialogTarget) selectLens(saved.id);
refresh();
};
const openFinding = (owner: Lens, picked: Finding) => setIssueKey(findingKey(owner, picked));
const closeFinding = () => {
setFindingId(null);
setIssueKey(null);
};
const changeFinding = async (status: Finding["status"], reason: string) => {
if (peeked) {
const saved = await update((current) => current.reviewFinding(peeked.lens.id, peeked.finding.id, status, reason));
if (saved) closeFinding();
return;
}
if (!lens || !finding) return;
await update((current) => current.reviewFinding(lens.id, finding.id, status, reason));
};
const sheetFinding = peeked ? peeked.finding : finding;
const sheetRuns = peeked ? sampledExecutions(peeked.lens) : sampledRuns;
const detailAgents = lens && finding ? findingAgents(lens, finding) : [];
const sheetAgents = peeked ? findingAgents(peeked.lens, peeked.finding) : detailAgents;
export interface InvestigationsViewProps {
readonly readOnly?: boolean;
}
const onDetail = !!selected;
const showDetailNav = onDetail && !showEmpty;
const showTables = !onDetail && lenses.length > 0;
const showMissing = missingSelection && !lens;
const assertNever = (value: never): never => {
throw new Error(`Unhandled screen ${JSON.stringify(value)}`);
};
interface BannersProps {
readonly error: Error | null | undefined;
readonly activityReady: boolean | null;
readonly flush: boolean;
readonly refresh: () => void;
}
function Banners({ error, activityReady, flush, refresh }: BannersProps) {
if (!error && activityReady === null) return null;
return (
<section aria-label="Investigations" className="flex w-full min-w-0 flex-1 flex-col gap-3">
{showDetailNav && (
<InvestigationNavigation
lens={lens}
showActions={showActions}
activityReady={activityReady}
connected={connected}
ready={ready}
selectLens={selectLens}
setWorkerSetup={setWorkerSetup}
setEditing={setEditing}
/>
)}
{query.error && !query.data ? (
<InvestigationsLoadFailed queryError={query.error} refresh={refresh} />
) : (
(error || query.error) && <InvestigationError message={error || query.error?.message || ""} refresh={refresh} />
)}
{query.isLoading && <InvestigationsLoading />}
{loaded && showEmpty && (
<InvestigationsWelcome
tracesReady={tracesReady}
requestsReady={requestsReady}
checking={activity.isPending}
traceError={activity.error?.message}
connected={connected}
readOnly={!!readOnly}
onRetry={() => {
void activity.refetch();
refresh();
}}
onConnect={() => setWorkerSetup(true)}
onCreate={() => setEditing("new")}
showPreview={activity.isSuccess && !ready}
/>
)}
{showReadiness && !ready && <ReadinessBanner activityReady={activityReady} className="py-2 text-xs" />}
{showTables && (
<div className="flex min-h-0 flex-1 flex-col gap-2">
{active && (
<HeaderActions>
{!readOnly && (
<WatchAllBanner
lenses={lenses}
busy={busy}
skipped={skipped}
onWatchAll={() =>
update(async (api) => {
const result = await api.watchAll();
setSkipped(result.skipped);
})
}
/>
)}
<InvestigationNavigation
lens={undefined}
showActions={showActions}
activityReady={activityReady}
connected={connected}
ready={ready}
selectLens={selectLens}
setWorkerSetup={setWorkerSetup}
setEditing={setEditing}
/>
</HeaderActions>
)}
<div className={cn("flex flex-col gap-3", flush && "p-2 pb-0")}>
{error && <InvestigationError message={error.message} refresh={refresh} />}
{activityReady !== null && <ReadinessBanner activityReady={activityReady} className="py-2 text-xs" />}
</div>
);
}
export function InvestigationsView({ readOnly = false }: InvestigationsViewProps) {
const api = useLensApi();
const invalidateLenses = useInvalidateLenses();
const actions = useInvestigationActions();
const { dialog, target, openDialog, closeDialog } = useDialogRoute();
const { issueKey, setIssueKey } = useIssueRoute();
const { lensId, setLensId } = useLensRoute();
const list = useQuery(lensQueries.list(api));
const status = useLensReadiness(true);
const { connected } = status;
const screenInput = { list, lensId, dialog, target };
const screen = investigationScreen(screenInput);
const lenses = list.data?.lenses ?? [];
const lens = lenses.find((candidate) => candidate.id === lensId);
const peeked = issueKey ? findFinding(lenses, issueKey) ?? null : null;
const selectedRow = (open: Lens | undefined): InvestigationRow | null => {
if (peeked) return { kind: "finding", ...peeked };
return open ? { kind: "investigation", lens: open } : null;
};
const selectRow = (row: InvestigationRow | null) => {
if (row?.kind === "finding") setIssueKey(findingKey(row.lens, row.finding));
else if (row) setLensId(row.lens.id);
else if (peeked) setIssueKey(null);
else setLensId(null);
};
const reviewPeeked = async (owned: OwnedFinding, status: Finding["status"], reason: string) => {
const saved = await actions.review(owned.lens, owned.finding, status, reason);
if (saved) setIssueKey(null);
};
const dialogLens = target ? lenses.find((candidate) => candidate.id === target) : lens;
const refresh = () => {
actions.reset();
void invalidateLenses();
};
const saveSetup = async (setup: SetupScreen, settings: Settings) => {
if (setup.mode !== "edit" && !status.ready)
throw new Error("Wait for recorded activity and a connected worker before starting an investigation");
const saved = await actions.save({ id: setup.mode === "edit" ? setup.lens.id : undefined, settings });
closeDialog();
if (!target) setLensId(saved.id);
};
const bannerError = screen.kind === "failed" ? undefined : actions.error ?? list.error;
const browsing = screen.kind === "list";
const showReadiness = browsing && !readOnly && !status.ready;
const content = (current: Screen): ReactNode => {
switch (current.kind) {
case "loading":
return <InvestigationsLoading />;
case "failed":
return <InvestigationsLoadFailed queryError={current.error} refresh={refresh} />;
case "missing":
return <InvestigationMissing selectLens={setLensId} />;
case "welcome":
if (status.loading) return <InvestigationsLoading />;
return (
<>
{!status.ready && <LensPreviewButton />}
<OnboardingSetup state={status} className="mx-auto w-full max-w-3xl py-6" />
</>
);
case "list":
return (
<InvestigationList
lenses={lenses}
lenses={current.lenses}
connected={connected}
readOnly={readOnly}
onOpen={selectLens}
selected={selectedRow(current.lens)}
onSelect={selectRow}
onEdit={(id) => openDialog("edit", id)}
onRunNow={(id) => openDialog("run_now", id)}
onOpenFinding={openFinding}
actions={
!readOnly && (
<>
<WatchAllBanner lenses={current.lenses} />
<Button size="sm" className="h-8" disabled={!status.ready} onClick={() => openDialog("new")}>
<Plus className="size-4" /> New investigation
</Button>
</>
)
}
>
{(row) =>
row.kind === "finding" ? (
<FindingPanelBody
owned={row}
readOnly={readOnly}
busy={actions.busy}
onReview={(owned, reviewStatus, reason) => void reviewPeeked(owned, reviewStatus, reason)}
/>
) : (
<div className="min-h-0 flex-1 overflow-y-auto p-4">
<InvestigationDetail
lens={row.lens}
readOnly={readOnly}
ready={status.ready}
busy={actions.busy}
connected={connected}
onEdit={() => openDialog("edit")}
onDuplicate={() => openDialog("duplicate")}
onPause={() => void actions.pause(row.lens)}
onEnableMonitoring={() => openDialog("monitoring")}
onCancelRun={() => void actions.cancelRun(row.lens)}
onRunNow={() => openDialog("run_now")}
onReviewFinding={(owned, reviewStatus, reason) =>
void actions.review(owned.lens, owned.finding, reviewStatus, reason)
}
/>
</div>
)
}
</InvestigationList>
);
case "setup":
return (
<InvestigationSetup
ready={status.ready}
mode={current.mode}
initial={current.mode === "new" ? undefined : current.initial}
defaultSource={!status.tracesReady && status.requestsReady ? "requests" : "traces"}
onClose={closeDialog}
onSave={(settings) => saveSetup(current, settings)}
/>
</div>
)}
{showMissing && <InvestigationMissing selectLens={selectLens} />}
{lens && (
<InvestigationDetail
lens={lens}
readOnly={readOnly}
ready={ready}
busy={busy}
setEditing={setEditing}
setMonitoring={setMonitoring}
onRunNow={() => openDialog("run_now")}
update={update}
connected={connected}
results={results}
/>
)}
{setupMode && (
<InvestigationSetupDialog
ready={ready}
mode={setupMode}
initial={setupSettings()}
defaultModel={defaultModel}
defaultSource={!tracesReady && requestsReady ? "requests" : "traces"}
models={models.data?.data.map((m) => m.id) ?? []}
modelDetails={modelDetails.data?.data ?? []}
modelsLoading={models.isLoading}
modelsError={models.error?.message}
accessToken={accessToken}
onClose={closeDialog}
onSave={save}
/>
)}
{dialog === "run_now" && targetLens && (
);
default:
return assertNever(current);
}
};
return (
<section
aria-label="Investigations"
className={cn("flex w-full min-w-0 flex-1 flex-col", !browsing && "gap-3 p-4")}
>
<Banners
error={bannerError}
activityReady={showReadiness ? status.activityReady : null}
flush={browsing}
refresh={refresh}
/>
{content(screen)}
{dialog === "run_now" && dialogLens && (
<RunNowDialog
lens={targetLens}
agents={Array.isArray(agents.data) ? agents.data : []}
busy={busy}
lens={dialogLens}
busy={actions.busy}
onClose={closeDialog}
onRun={async (request) => {
await update((api) => api.startRun(targetLens.id, request));
await actions.startRun(dialogLens, request);
closeDialog();
}}
/>
)}
{workerSetup && (
<WorkerDialog
accessToken={accessToken}
workers={query.data?.workers ?? []}
onClose={closeDialog}
onChanged={refresh}
onReady={ready && showEmpty ? () => openDialog("new") : undefined}
/>
)}
{dialog === "monitoring" && lens && (
{dialog === "monitoring" && dialogLens && (
<MonitoringDialog
settings={lens.settings}
ready={ready}
settings={dialogLens.settings}
ready={status.ready}
onClose={closeDialog}
onSave={async (settings) => {
await saveLens.mutateAsync({ id: lens.id, settings });
refresh();
await actions.save({ id: dialogLens.id, settings });
}}
/>
)}
<FindingSheet
finding={sheetFinding}
agents={sheetAgents}
sampledRuns={sheetRuns}
readOnly={readOnly}
busy={busy}
onClose={closeFinding}
changeFinding={changeFinding}
onEvidence={(value) => {
setRequestOffset(0);
setEvidence(value);
}}
/>
{resultsLens && target?.source === "traces" && (
<TraceSheet
open={!!evidence}
traceId={target.id}
traceRef={target.traceRef}
initialSpanId={evidence?.span}
accessToken={accessToken}
onClose={() => setEvidence(null)}
/>
)}
<RequestEvidenceSheet
target={target}
setEvidence={setEvidence}
requestEvidenceData={requestEvidenceData}
requestEvidenceError={requestEvidenceError}
requestEvidenceLoading={requestEvidenceLoading}
requestOffset={requestOffset}
setRequestOffset={setRequestOffset}
/>
</section>
);
}

168
ui/litellm-dashboard/src/components/lens/investigations/InvestigationsWelcome.tsx
View file

@ -1,168 +0,0 @@
import Link from "next/link";
import { Check, Loader2 } from "lucide-react";
import { Button, buttonVariants } from "@/components/ui/button";
import { LensPreviewButton } from "../LensPreviewButton";
import { uiHref } from "@/utils/uiHref";
import { cn } from "@/lib/cva.config";
export function InvestigationsWelcome({
tracesReady,
requestsReady = false,
checking,
traceError,
connected,
readOnly,
onConnect,
onCreate,
onRetry,
showPreview = false,
}: {
tracesReady: boolean;
requestsReady?: boolean;
checking: boolean;
traceError?: string;
connected: boolean;
readOnly: boolean;
onConnect: () => void;
onCreate: () => void;
onRetry: () => void;
showPreview?: boolean;
}) {
const activityReady = tracesReady || requestsReady;
const workerReady = activityReady && connected;
const canCreate = workerReady && !readOnly;
const canConnect = activityReady && !readOnly;
const traceStatus = activityStatus(tracesReady, requestsReady, checking);
const waitingForWorker = activityReady && !connected;
const stepOffset = Number(!activityReady);
const introduction = activityReady
? {
title: "Run your first investigation",
description: "Your recorded activity is ready. Connect a worker and choose what Lens should look for.",
}
: {
title: "Find what needs attention",
description: "Check how your agents behave. Get findings you can trace back to what happened.",
};
const traceButtonClass = buttonVariants({
variant: "default",
className: "col-start-2 w-fit sm:col-start-auto",
});
return (
<section aria-labelledby="lens-welcome" className="m-auto w-full max-w-2xl py-10">
{showPreview && <LensPreviewButton />}
<h2 id="lens-welcome" className="text-xl font-semibold tracking-tight">
{introduction.title}
</h2>
<p className="mt-2 max-w-lg text-sm leading-6 text-muted-foreground">{introduction.description}</p>
{activityReady && (
<p role="status" className="mt-3 text-sm text-success">
{traceStatus}
</p>
)}
<ol className="mt-8 divide-y border-y">
{!activityReady && (
<li className="grid grid-cols-[28px_minmax(0,1fr)] items-center gap-x-4 gap-y-3 py-5 sm:grid-cols-[28px_minmax(0,1fr)_auto]">
<Step number={1} complete={activityReady} checking={checking} active={!activityReady} />
<div>
<h3 className="text-sm font-medium">Set up traces</h3>
<p className="mt-1 text-sm text-muted-foreground">{traceStatus}</p>
</div>
<Link href={uiHref("lens/?tab=traces")} className={traceButtonClass}>
Set up traces
</Link>
</li>
)}
<li
data-state={activityReady ? "active" : "inactive"}
className="grid grid-cols-[28px_minmax(0,1fr)] items-center gap-x-4 gap-y-3 py-5 text-muted-foreground data-[state=active]:text-foreground sm:grid-cols-[28px_minmax(0,1fr)_auto]"
>
<Step number={stepOffset + 1} complete={workerReady} active={waitingForWorker} />
<div>
<h3 className="text-sm font-medium">Connect a worker</h3>
<p className="mt-1 text-sm text-muted-foreground">
{workerReady ? "Worker connected" : "Runs your investigations in the background."}
</p>
</div>
<Button
className="col-start-2 w-fit sm:col-start-auto"
variant={waitingForWorker ? "default" : "ghost"}
disabled={!canConnect}
onClick={onConnect}
>
{connected ? "Manage worker" : "Connect worker"}
</Button>
</li>
<li
data-state={workerReady ? "active" : "inactive"}
className="grid grid-cols-[28px_minmax(0,1fr)] items-center gap-x-4 gap-y-3 py-5 text-muted-foreground data-[state=active]:text-foreground sm:grid-cols-[28px_minmax(0,1fr)_auto]"
>
<Step number={stepOffset + 2} complete={false} active={workerReady} />
<div>
<h3 className="text-sm font-medium">Run an investigation</h3>
</div>
<Button
className="col-start-2 w-fit sm:col-start-auto"
variant={workerReady ? "default" : "ghost"}
disabled={!canCreate}
onClick={onCreate}
>
New investigation
</Button>
</li>
</ol>
{traceError && (
<div role="alert" className="mt-6 text-sm text-destructive">
Could not check recorded activity. {traceError}{" "}
<Button variant="link" onClick={onRetry}>
Retry
</Button>
</div>
)}
{readOnly && (
<p className="mt-6 text-sm text-muted-foreground">An administrator can finish setup and run investigations.</p>
)}
</section>
);
}
function Step({
number,
complete,
checking = false,
active = false,
}: {
number: number;
complete: boolean;
checking?: boolean;
active?: boolean;
}) {
const incomplete = checking ? <Loader2 className="size-4 animate-spin" /> : number;
const state = stepState(complete, active);
return (
<span
data-state={state}
className={cn(
"flex size-7 items-center justify-center rounded-full text-xs font-medium",
"data-[state=complete]:bg-emerald-50 data-[state=complete]:text-emerald-700 dark:data-[state=complete]:bg-emerald-950 dark:data-[state=complete]:text-emerald-400",
"data-[state=active]:bg-foreground data-[state=active]:text-background",
"data-[state=inactive]:border data-[state=inactive]:text-muted-foreground",
)}
aria-label={complete ? `Step ${number} complete` : `Step ${number}`}
>
{complete ? <Check className="size-4" /> : incomplete}
</span>
);
}
function stepState(complete: boolean, active?: boolean) {
if (complete) return "complete";
if (active) return "active";
return "inactive";
}
function activityStatus(traces: boolean, requests: boolean, checking: boolean) {
if (traces) return "Traces received";
if (requests) return "Request logs received";
return checking ? "Checking for activity…" : "Connect your agent and send a trace.";
}

2
ui/litellm-dashboard/src/components/lens/investigations/IssueBrief.tsx
View file

@ -51,7 +51,7 @@ export function IssueBrief({ title, brief }: { title: string; brief: IssueBrief
className="inline-flex h-7 items-center gap-1.5 rounded-md border border-border bg-background px-2 text-xs font-medium hover:bg-muted"
>
{copied === agent.name ? (
<Check className="size-3.5 text-emerald-600" />
<Check className="size-3.5 text-success" />
) : (
<img src={agent.logo} alt="" aria-hidden className="size-3.5" />
)}

69
ui/litellm-dashboard/src/components/lens/investigations/RequestEvidenceSheet.tsx
View file

@ -1,69 +0,0 @@
"use client";
import type { components } from "@/lib/http/schema";
import { Button } from "@/components/ui/button";
import { Sheet, SheetContent, SheetHeader, SheetTitle, SheetDescription } from "@/components/ui/sheet";
import { evidenceTarget } from "../model/findings";
type RequestEvidence = components["schemas"]["ExecutionContent"];
export function RequestEvidenceSheet({
target,
setEvidence,
requestEvidenceData,
requestEvidenceError,
requestEvidenceLoading,
requestOffset,
setRequestOffset,
}: {
target: ReturnType<typeof evidenceTarget>;
setEvidence: (value: { id: string; span: string } | null) => void;
requestEvidenceData: RequestEvidence | undefined;
requestEvidenceError: Error | null | undefined;
requestEvidenceLoading: boolean;
requestOffset: number;
setRequestOffset: (offset: number) => void;
}) {
return (
<Sheet
open={target?.source === "requests"}
onOpenChange={(open) => {
if (!open) setEvidence(null);
}}
>
<SheetContent className="overflow-y-auto data-[side=right]:sm:max-w-2xl">
<SheetHeader>
<SheetTitle>Request evidence</SheetTitle>
<SheetDescription>Original logged input and output</SheetDescription>
</SheetHeader>
<div className="p-4 space-y-3">
{requestEvidenceLoading && <p role="status">Loading request…</p>}
{requestEvidenceError && <p role="alert">{requestEvidenceError.message}</p>}
{requestEvidenceData?.parts.map((p) => (
<pre className="whitespace-pre-wrap break-words text-xs" key={p.span_id}>
{p.content}
</pre>
))}
{requestEvidenceData?.parts.length === 0 && <p>Request was not found or is past retention</p>}
<div className="flex flex-wrap gap-2">
{requestOffset > 0 && (
<Button variant="outline" onClick={() => setRequestOffset(Math.max(0, requestOffset - 8000))}>
Previous section
</Button>
)}
{requestEvidenceData?.parts.some((p) => p.truncated) && (
<Button
variant="outline"
onClick={() => setRequestOffset(requestOffset === 0 ? 1 : requestOffset + 8000)}
>
Next section
</Button>
)}
</div>
</div>
</SheetContent>
</Sheet>
);
}

20
ui/litellm-dashboard/src/components/lens/investigations/TraceSheet.test.tsx
View file

@ -1,20 +0,0 @@
import { render, screen } from "@testing-library/react";
import { describe, expect, it, vi } from "vitest";
import type { RunSelection } from "@/components/view_logs/TraceView/traceRouting";
import { TraceSheet } from "./TraceSheet";
vi.mock("@/components/view_logs/TraceView/TraceDrawer", () => ({
RunView: ({ selection }: { selection: RunSelection }) => <div data-testid="run-view">{selection.spanId}</div>,
}));
describe("TraceSheet", () => {
it("opens another finding's evidence on its own step", () => {
const props = { open: true, traceId: "trace-1", accessToken: "t", onClose: vi.fn() };
const { rerender } = render(<TraceSheet {...props} initialSpanId="step-a" />);
expect(screen.getByTestId("run-view")).toHaveTextContent("step-a");
rerender(<TraceSheet {...props} initialSpanId="step-b" />);
expect(screen.getByTestId("run-view")).toHaveTextContent("step-b");
});
});

64
ui/litellm-dashboard/src/components/lens/investigations/TraceSheet.tsx
View file

@ -1,64 +0,0 @@
import { RunView } from "@/components/view_logs/TraceView/TraceDrawer";
import { useLocalRunSelection } from "@/components/view_logs/TraceView/traceRouting";
import { Sheet, SheetContent, SheetHeader, SheetTitle, SheetDescription } from "@/components/ui/sheet";
export function TraceSheet({
open,
traceId,
traceRef,
initialSpanId,
accessToken,
onClose,
}: {
open: boolean;
traceId: string;
traceRef?: string;
initialSpanId?: string;
accessToken: string;
onClose: () => void;
}) {
return (
<Sheet
open={open}
onOpenChange={(value) => {
if (!value) onClose();
}}
>
<SheetContent className="w-full overflow-y-auto data-[side=right]:sm:max-w-[90vw]">
<SheetHeader>
<SheetTitle className="text-xl">Original run</SheetTitle>
<SheetDescription>Recorded agent steps and evidence</SheetDescription>
</SheetHeader>
{open && (
<SheetRun
key={`${traceId}:${traceRef}:${initialSpanId}`}
traceId={traceId}
traceRef={traceRef}
initialSpanId={initialSpanId ?? null}
accessToken={accessToken}
onBack={onClose}
/>
)}
</SheetContent>
</Sheet>
);
}
function SheetRun({
traceId,
traceRef,
initialSpanId,
accessToken,
onBack,
}: {
traceId: string;
traceRef?: string;
initialSpanId: string | null;
accessToken: string;
onBack: () => void;
}) {
const selection = useLocalRunSelection(initialSpanId);
return (
<RunView traceId={traceId} traceRef={traceRef} selection={selection} accessToken={accessToken} onBack={onBack} />
);
}

31
ui/litellm-dashboard/src/components/lens/investigations/WatchAllBanner.tsx
View file

@ -1,20 +1,23 @@
"use client";
import { z } from "zod";
import { useLensUpdate } from "../data/mutations";
import type { Lens } from "../model/types";
export function WatchAllBanner({
lenses,
busy,
onWatchAll,
skipped = [],
}: {
lenses: readonly Lens[];
busy: boolean;
onWatchAll: () => void;
skipped?: readonly { id: string; name: string; reason: string }[];
}) {
const watchAllResult = z.object({
skipped: z.array(z.object({ id: z.string(), name: z.string(), reason: z.string() })),
});
export interface WatchAllBannerProps {
readonly lenses: readonly Lens[];
}
export function WatchAllBanner({ lenses }: WatchAllBannerProps) {
const watchAll = useLensUpdate();
const paused = lenses.filter((lens) => !lens.settings.enabled).length;
if (paused === 0) return null;
const result = watchAllResult.safeParse(watchAll.data);
const skipped = result.success ? result.data.skipped : [];
if (skipped.length >= paused)
return (
<span
@ -27,11 +30,11 @@ export function WatchAllBanner({
);
return (
<span role="status" className="text-xs text-muted-foreground">
{paused} paused ·{" "}
{paused} paused · {watchAll.error && <span className="text-destructive">{watchAll.error.message} · </span>}
<button
type="button"
disabled={busy}
onClick={onWatchAll}
disabled={watchAll.isPending}
onClick={() => watchAll.mutate((api) => api.watchAll())}
className="font-medium text-foreground underline-offset-2 hover:underline disabled:opacity-50"
>
Turn all on

26
ui/litellm-dashboard/src/components/lens/investigations/detail/CriteriaTab.tsx
View file

@ -4,39 +4,37 @@ import { Button } from "@/components/ui/button";
import { TabsContent } from "@/components/ui/tabs";
import { type Settings } from "../../model/types";
import { watches } from "../../setup/watches";
import { watches } from "../../model/watches";
export function CriteriaTab({
batchSettings,
readOnly,
setEditing,
}: {
batchSettings: Settings | undefined;
readOnly: boolean;
setEditing: (mode: "edit") => void;
}) {
export interface CriteriaTabProps {
readonly settings: Settings | undefined;
readonly readOnly: boolean;
readonly onEditCriteria: () => void;
}
export function CriteriaTab({ settings, readOnly, onEditCriteria }: CriteriaTabProps) {
return (
<TabsContent value="checks" className="pt-4 space-y-4">
<div className="flex items-center justify-between">
<p className="text-sm text-muted-foreground">Used for this run</p>
{!readOnly && (
<Button variant="outline" size="sm" onClick={() => setEditing("edit")}>
<Button variant="outline" size="sm" onClick={onEditCriteria}>
Edit criteria
</Button>
)}
</div>
{batchSettings?.context && (
{settings?.context && (
<div className="space-y-2 border-b pb-5">
<h3 className="text-base font-semibold">What should the agent be doing?</h3>
<p className="text-xs text-muted-foreground">The purpose and expected outcome of the agent.</p>
<p className="mt-2 whitespace-pre-wrap text-sm">{batchSettings.context}</p>
<p className="mt-2 whitespace-pre-wrap text-sm">{settings.context}</p>
</div>
)}
<div className="pt-2">
<h3 className="text-base font-semibold">Watch for</h3>
<p className="mt-1 text-xs text-muted-foreground">Specific problems or patterns to investigate.</p>
</div>
{batchSettings?.checks.map((c, index) => (
{settings?.checks.map((c, index) => (
<div key={c.id} className="flex items-start gap-3 border-b py-4">
<span className="mt-0.5 text-xs tabular-nums text-muted-foreground">{index + 1}.</span>
<CheckSummary check={c} />

189
ui/litellm-dashboard/src/components/lens/investigations/detail/FindingsTab.tsx
View file

@ -1,109 +1,122 @@
"use client";
import { ListRow } from "@/components/shared/ListRow";
import type { ReactNode } from "react";
import { ChevronRight, CheckCircle2 } from "lucide-react";
import { Button } from "@/components/ui/button";
import { Inspector } from "@/components/shared/Inspector";
import { Button } from "@/components/ui/button";
import { TabsContent } from "@/components/ui/tabs";
import { type Lens, type Finding, type Job } from "../../model/types";
import { cn } from "@/lib/cva.config";
const priorityColors = { high: "bg-red-500", medium: "bg-amber-500", low: "bg-slate-400" };
import { sortedFindings } from "../../model/findings";
import type { OwnedFinding } from "../../model/inbox";
import { activeJob } from "../../model/status";
import { type Lens, type Finding, type Job } from "../../model/types";
import { ownedFindingKey } from "../FindingDetails";
import { FINDING_PANEL_WIDTH_KEY } from "../../storage";
import { useFindingFilters, useFindingRoute } from "../../route";
const priorityColors = { high: "bg-destructive", medium: "bg-warning", low: "bg-muted-foreground" };
function emptyFindingTitle(active: boolean, scanned: boolean, status?: string) {
if (status === "failed" || status === "cancelled") return "No findings from this run";
if (active) return "Your findings will appear here";
return scanned ? "No matching findings" : "Ready for the first analysis";
}
export function FindingsTab({
kind,
setKind,
batchFindings,
filter,
setFilter,
visibleFindings,
setFindingId,
active,
lens,
job,
}: {
kind: "issue" | "pattern";
setKind: (kind: "issue" | "pattern") => void;
batchFindings: Finding[];
filter: string;
setFilter: (filter: string) => void;
visibleFindings: Finding[];
setFindingId: (id: string) => void;
export interface FindingsTabProps {
readonly lens: Lens;
readonly job: Job | undefined;
readonly findings: readonly Finding[];
/** The finding panel, rendered inside this tab's Inspector so J/K walk the listed findings. */
readonly children?: ReactNode;
}
active: Job | undefined;
lens: Lens;
job: Job | undefined;
}) {
export function FindingsTab({ lens, job, findings, children }: FindingsTabProps) {
const { kind, setKind, status, setStatus } = useFindingFilters();
const { findingId, setFindingId } = useFindingRoute();
const active = activeJob(lens.jobs) !== undefined;
const openCount = (of: Finding["kind"]) => findings.filter((f) => f.kind === of && f.status === "open").length;
const visible = sortedFindings(findings.filter((f) => (status === "all" || f.status === status) && f.kind === kind));
const picked = findings.find((f) => f.id === findingId);
const selected: OwnedFinding | null = picked ? { lens, finding: picked } : null;
return (
<TabsContent value="findings" className="pt-4 space-y-4">
<div className="flex flex-wrap items-center justify-between gap-3">
<div className="flex gap-1" aria-label="Finding category">
<Button size="sm" variant={kind === "issue" ? "secondary" : "ghost"} onClick={() => setKind("issue")}>
Needs attention ({batchFindings.filter((f) => f.kind === "issue" && f.status === "open").length ?? 0})
</Button>
<Button size="sm" variant={kind === "pattern" ? "secondary" : "ghost"} onClick={() => setKind("pattern")}>
Patterns ({batchFindings.filter((f) => f.kind === "pattern" && f.status === "open").length ?? 0})
</Button>
</div>
<select
aria-label="Finding status"
className="rounded-md border bg-background px-2 py-1 text-xs"
value={filter}
onChange={(e) => setFilter(e.target.value)}
>
<option value="open">Open</option>
<option value="resolved">Resolved</option>
<option value="dismissed">Dismissed</option>
<option value="all">All statuses</option>
</select>
</div>
<p className="text-xs text-muted-foreground">
{kind === "issue"
? "Problems worth investigating, highest priority first."
: "Useful behavior and trends. These do not necessarily need a fix."}
</p>
<div className="divide-y border-y">
{visibleFindings.map((f) => (
<ListRow
key={f.id}
onClick={() => {
setFindingId(f.id);
}}
className="flex w-full gap-3 py-4 text-left hover:bg-muted/30 focus-visible:outline-2 focus-visible:outline-ring"
<Inspector.Root
items={visible.map((finding) => ({ lens, finding }))}
itemKey={ownedFindingKey}
selected={selected}
onSelectedChange={(owned) => setFindingId(owned?.finding.id ?? null)}
noun="finding"
storageKey={FINDING_PANEL_WIDTH_KEY}
>
<TabsContent value="findings" className="pt-4 space-y-4">
<div className="flex flex-wrap items-center justify-between gap-3">
<div className="flex gap-1" aria-label="Finding category">
<Button size="sm" variant={kind === "issue" ? "secondary" : "ghost"} onClick={() => setKind("issue")}>
Needs attention ({openCount("issue")})
</Button>
<Button size="sm" variant={kind === "pattern" ? "secondary" : "ghost"} onClick={() => setKind("pattern")}>
Patterns ({openCount("pattern")})
</Button>
</div>
<select
aria-label="Finding status"
className="rounded-md border bg-background px-2 py-1 text-xs"
value={status}
onChange={(e) => setStatus(e.target.value)}
>
<span
data-state={f.priority ?? "medium"}
className={cn("mt-1 size-2 shrink-0 rounded-full", priorityColors[f.priority ?? "medium"])}
aria-label={`${f.priority} priority`}
/>
<div className="min-w-0 flex-1">
<p className="text-sm font-medium">{f.title}</p>
<p className="mt-1 line-clamp-2 text-sm text-muted-foreground">{f.description}</p>
<option value="open">Open</option>
<option value="resolved">Resolved</option>
<option value="dismissed">Dismissed</option>
<option value="all">All statuses</option>
</select>
</div>
<p className="text-xs text-muted-foreground">
{kind === "issue"
? "Problems worth investigating, highest priority first."
: "Useful behavior and trends. These do not necessarily need a fix."}
</p>
<div className="divide-y border-y">
{visible.map((f) => (
<Inspector.Row
key={f.id}
item={{ lens, finding: f }}
render={
<button
type="button"
className="flex w-full gap-3 py-4 text-left hover:bg-muted/30 focus-visible:outline-2 focus-visible:outline-ring data-[state=selected]:bg-trace-row-selected data-[state=selected]:shadow-[inset_2px_0_0_var(--trace-brand)]"
/>
}
>
<span
data-state={f.priority ?? "medium"}
className={cn("mt-1 size-2 shrink-0 rounded-full", priorityColors[f.priority ?? "medium"])}
aria-label={`${f.priority} priority`}
/>
<div className="min-w-0 flex-1">
<p className="text-sm font-medium">{f.title}</p>
<p className="mt-1 line-clamp-2 text-sm text-muted-foreground">{f.description}</p>
<p className="mt-2 text-xs text-muted-foreground">
{f.occurrences?.length ?? 0} linked {f.occurrences?.length === 1 ? "run" : "runs"} ·{" "}
{f.kind === "issue" ? `${f.priority} priority` : "Pattern"}
</p>
</div>
<ChevronRight className="size-4 shrink-0 text-muted-foreground" />
</Inspector.Row>
))}
{visible.length === 0 && (
<div className="px-6 py-14 text-center">
<CheckCircle2 className="mx-auto mb-3 size-5 text-muted-foreground" />
<p className="text-sm font-medium">{emptyFindingTitle(active, !!lens.last_scan_at, job?.status)}</p>
<p className="mt-2 text-xs text-muted-foreground">
{f.occurrences?.length ?? 0} linked {f.occurrences?.length === 1 ? "run" : "runs"} ·{" "}
{f.kind === "issue" ? `${f.priority} priority` : "Pattern"}
{active
? "Lens is reviewing the selected activity."
: "Findings reflect the runs analyzed, not a guarantee about all activity."}
</p>
</div>
<ChevronRight className="size-4 shrink-0 text-muted-foreground" />
</ListRow>
))}
{visibleFindings.length === 0 && (
<div className="px-6 py-14 text-center">
<CheckCircle2 className="mx-auto mb-3 size-5 text-muted-foreground" />
<p className="text-sm font-medium">{emptyFindingTitle(!!active, !!lens.last_scan_at, job?.status)}</p>
<p className="mt-2 text-xs text-muted-foreground">
{active
? "Lens is reviewing the selected activity."
: "Findings reflect the runs analyzed, not a guarantee about all activity."}
</p>
</div>
)}
</div>
</TabsContent>
)}
</div>
</TabsContent>
{children}
</Inspector.Root>
);
}

58
ui/litellm-dashboard/src/components/lens/investigations/detail/HistoryTab.tsx
View file

@ -1,47 +1,46 @@
"use client";
import { ListRow } from "@/components/shared/ListRow";
import { useState } from "react";
import { ChevronRight } from "lucide-react";
import { Button } from "@/components/ui/button";
import { TabsContent } from "@/components/ui/tabs";
import { ScanDuration } from "../InvestigationProgress";
import type { Lens, Job } from "../../model/types";
import { HistoryTimeline } from "./HistoryTimeline";
import { ScanDuration } from "./JobMeta";
import { useRunHistory } from "./useRunHistory";
import type { Lens } from "../../model/types";
import { useRunRoute } from "../../route";
import { money, when } from "../../model/format";
export function HistoryTab({
history,
historyError,
refetchHistory,
lens,
openBatch,
historyOffset,
setHistoryOffset,
}: {
history: Job[] | undefined;
historyError: Error | null | undefined;
refetchHistory: () => void;
lens: Lens;
openBatch: (id: string) => void;
historyOffset: number;
setHistoryOffset: (offset: number) => void;
}) {
const PAGE = 50;
export interface HistoryTabProps {
readonly lens: Lens;
}
export function HistoryTab({ lens }: HistoryTabProps) {
const [offset, setOffset] = useState(0);
const history = useRunHistory(lens, offset);
const { openRun } = useRunRoute();
const rows = history.data ?? lens.jobs;
return (
<TabsContent value="activity" className="pt-4 space-y-4">
{historyError && (
{history.error && (
<p role="alert" className="text-sm text-destructive">
Could not load run history.{" "}
<Button variant="link" size="sm" onClick={() => void refetchHistory()}>
<Button variant="link" size="sm" onClick={() => void history.refetch()}>
Retry
</Button>
</p>
)}
{rows.length > 0 && <HistoryTimeline jobs={rows} slots={PAGE} onOpen={openRun} />}
<div className="divide-y border-y">
{(history ?? lens.jobs)?.map((j) => (
{rows.map((j) => (
<ListRow
key={j.id}
onClick={() => openBatch(j.id)}
onClick={() => openRun(j.id)}
className="flex w-full items-center gap-4 py-4 text-left hover:bg-muted/30 focus-visible:outline-2 focus-visible:outline-ring"
>
<div className="min-w-0 flex-1">
@ -66,21 +65,16 @@ export function HistoryTab({
</ListRow>
))}
</div>
{(historyOffset > 0 || (history?.length ?? 0) >= 50) && (
{(offset > 0 || (history.data?.length ?? 0) >= PAGE) && (
<div className="flex justify-between">
<Button
variant="ghost"
size="sm"
disabled={!historyOffset}
onClick={() => setHistoryOffset(Math.max(0, historyOffset - 50))}
>
<Button variant="ghost" size="sm" disabled={!offset} onClick={() => setOffset(Math.max(0, offset - PAGE))}>
Newer runs
</Button>
<Button
variant="ghost"
size="sm"
disabled={(history?.length ?? 0) < 50}
onClick={() => setHistoryOffset(historyOffset + 50)}
disabled={(history.data?.length ?? 0) < PAGE}
onClick={() => setOffset(offset + PAGE)}
>
Older runs
</Button>

98
ui/litellm-dashboard/src/components/lens/investigations/detail/HistoryTimeline.test.ts Normal file
View file

@ -0,0 +1,98 @@
import { describe, expect, it } from "vitest";
import type { Job } from "../../model/types";
import { checkColumns } from "./HistoryTimeline";
const coverage: Job["coverage"] = {
eligible: 0,
selected: 0,
screened: 0,
investigated: 0,
inconclusive: 0,
grouping_batches: 0,
grouped_batches: 0,
candidates: 0,
partial: 0,
unassessable: 0,
};
const job: Job = {
assessments: [],
steps: [],
trigger: "schedule",
coverage,
attempts: 0,
error: "",
cost: 0,
id: "scan",
status: "running",
stage: "Reading executions",
created_at: "2026-09-30T12:00:00Z",
start: "2026-09-29T12:00:00Z",
end: "2026-09-30T12:00:00Z",
revision: 1,
settings: {
context: "",
source: "traces",
lookback_hours: 24,
service: "",
agent_name: "",
filters: [],
enabled: false,
interval_minutes: 15,
sample_size: 100,
sample_percent: 100,
concurrency: 8,
team_id: "",
execution_ids: [],
monthly_budget: 20,
name: "Release reviews",
model: "analysis",
checks: [{ enabled: true, id: "failures", instruction: "Find failed outcomes" }],
},
};
const assessment = (id: string, issues: string[]): Job["assessments"][number] => ({
execution_id: id,
cannot_assess: false,
issue_checks: issues,
pattern_checks: [],
});
const check = (id: string, status: Job["status"], screened: number, assessments: Job["assessments"]): Job => ({
...job,
id,
status,
coverage: { ...coverage, screened },
assessments,
});
describe("checkColumns", () => {
const newest = check("newest", "completed", 3, [
assessment("a", ["failures"]),
assessment("b", []),
assessment("c", ["failures", "other"]),
]);
const oldest = check("oldest", "failed", 7, []);
it("pads the left and puts the newest check on the right", () => {
const columns = checkColumns([newest, oldest], 4);
expect(columns.map((c) => c.job?.id ?? null)).toEqual([null, null, "oldest", "newest"]);
expect(columns[0]).toMatchObject({ total: 0, failed: 0 });
});
it("counts reviewed traces and the traces with at least one issue", () => {
expect(checkColumns([newest], 1)[0]).toMatchObject({ total: 3, failed: 2, opacity: 1 });
});
it("fades checks that did not complete", () => {
const [failed] = checkColumns([oldest], 1);
const [cancelled] = checkColumns([{ ...oldest, status: "cancelled" }], 1);
expect(failed.opacity).toBeLessThan(1);
expect(cancelled.opacity).toBe(failed.opacity);
});
it("keeps only the most recent checks that fit", () => {
expect(checkColumns([newest, oldest], 1).map((c) => c.job?.id)).toEqual(["newest"]);
});
});

92
ui/litellm-dashboard/src/components/lens/investigations/detail/HistoryTimeline.tsx Normal file
View file

@ -0,0 +1,92 @@
"use client";
import { useState } from "react";
import { DotFieldCanvas, DotFieldRoot } from "@/components/shared/dotField/DotField";
import type { DotColumn } from "@/components/shared/dotField/dots";
import { when } from "../../model/format";
import type { Job } from "../../model/types";
const UNSUCCESSFUL_OPACITY = 0.35;
export interface CheckColumn extends DotColumn {
job: Job | null;
}
const affectedRuns = (job: Job): number => job.assessments.filter((a) => a.issue_checks.length > 0).length;
/** One column per check, oldest on the left, padded with empty slots so spacing stays stable as checks accrue. */
export function checkColumns(jobs: readonly Job[], slots: number): readonly CheckColumn[] {
const recent = jobs.slice(0, slots).reverse();
const padding = Array.from(
{ length: slots - recent.length },
(): CheckColumn => ({
job: null,
total: 0,
failed: 0,
series: [],
}),
);
const checks = recent.map(
(job): CheckColumn => ({
job,
total: job.coverage.screened,
failed: affectedRuns(job),
series: [],
opacity: job.status === "failed" || job.status === "cancelled" ? UNSUCCESSFUL_OPACITY : 1,
}),
);
return [...padding, ...checks];
}
const pct = (value: number): string => `${value * 100}%`;
export interface HistoryTimelineProps {
readonly jobs: readonly Job[];
readonly slots: number;
readonly onOpen: (jobId: string) => void;
}
/** Every check as a dot column: reviewed traces in grey, affected traces in red, failed checks faded. */
export function HistoryTimeline({ jobs, slots, onOpen }: HistoryTimelineProps) {
const columns = checkColumns(jobs, slots);
const [hover, setHover] = useState<number | null>(null);
const hovered = hover === null ? null : columns[hover];
return (
<div className="relative select-none" data-testid="history-timeline">
<DotFieldRoot columns={columns} hover={hover} className="flex" onPointerLeave={() => setHover(null)}>
<DotFieldCanvas />
{columns.map(({ job }, i) =>
job ? (
<button
key={job.id}
type="button"
aria-label={`Check at ${when(job.created_at)}`}
className="relative h-full flex-1 focus-visible:outline-2 focus-visible:outline-ring"
onPointerEnter={() => setHover(i)}
onFocus={() => setHover(i)}
onBlur={() => setHover(null)}
onClick={() => onOpen(job.id)}
/>
) : (
<span key={`empty-${i}`} className="h-full flex-1" />
),
)}
</DotFieldRoot>
{hovered?.job && hover !== null && (
<div
role="tooltip"
className="pointer-events-none absolute top-full z-floating mt-1 rounded-md border border-border bg-popover px-2.5 py-1.5 font-mono text-xs text-popover-foreground shadow-md"
style={{ left: pct(Math.min(0.75, hover / slots)) }}
>
<div>{when(hovered.job.created_at)}</div>
<div>
{hovered.total} reviewed, {hovered.failed} affected
</div>
{hovered.job.status !== "completed" && <div className="capitalize">{hovered.job.status}</div>}
</div>
)}
</div>
);
}

101
ui/litellm-dashboard/src/components/lens/investigations/detail/InvestigationActions.test.tsx Normal file
View file

@ -0,0 +1,101 @@
import { screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { describe, expect, it, vi } from "vitest";
import { renderWithProviders } from "@/../tests/test-utils";
import type { Lens } from "../../model/types";
import { InvestigationActions, type InvestigationActionsProps } from "./InvestigationActions";
const lens: Lens = {
version: 0,
spent: 0,
id: "lens",
scope: { all_teams: true, api_key_hash: "", team_id: "" },
settings: {
context: "",
source: "traces",
lookback_hours: 24,
service: "",
agent_name: "",
filters: [],
interval_minutes: 15,
sample_size: 100,
sample_percent: 100,
concurrency: 8,
team_id: "",
execution_ids: [],
monthly_budget: 20,
name: "Release reviews",
model: "analysis",
enabled: true,
checks: [],
},
revision: 1,
created_at: "2026-09-30T10:00:00Z",
next_run_at: "2026-09-30T10:00:00Z",
budget_month: "2026-09",
findings: [],
jobs: [],
};
function renderActions(overrides: Partial<InvestigationActionsProps> = {}) {
const intents = {
onEdit: vi.fn(),
onDuplicate: vi.fn(),
onPause: vi.fn(),
onEnableMonitoring: vi.fn(),
onRunNow: vi.fn(),
};
renderWithProviders(<InvestigationActions lens={lens} ready busy={false} {...intents} {...overrides} />);
return intents;
}
describe("InvestigationActions", () => {
it("reports each menu choice as an intent without touching the API", async () => {
const user = userEvent.setup();
const intents = renderActions();
await user.click(screen.getByRole("button", { name: "Run now" }));
expect(intents.onRunNow).toHaveBeenCalledOnce();
await user.click(screen.getByRole("button", { name: "Investigation actions" }));
await user.click(await screen.findByRole("menuitem", { name: "Pause monitoring" }));
expect(intents.onPause).toHaveBeenCalledOnce();
expect(intents.onEnableMonitoring).not.toHaveBeenCalled();
await user.click(screen.getByRole("button", { name: "Investigation actions" }));
await user.click(await screen.findByRole("menuitem", { name: "Edit investigation" }));
expect(intents.onEdit).toHaveBeenCalledOnce();
await user.click(screen.getByRole("button", { name: "Investigation actions" }));
await user.click(await screen.findByRole("menuitem", { name: "Duplicate" }));
expect(intents.onDuplicate).toHaveBeenCalledOnce();
});
it("offers to enable monitoring on a paused investigation", async () => {
const user = userEvent.setup();
const paused = { ...lens, settings: { ...lens.settings, enabled: false } };
const intents = renderActions({ lens: paused });
await user.click(screen.getByRole("button", { name: "Investigation actions" }));
await user.click(await screen.findByRole("menuitem", { name: "Enable monitoring" }));
expect(intents.onEnableMonitoring).toHaveBeenCalledOnce();
expect(intents.onPause).not.toHaveBeenCalled();
});
it("blocks a second run while one is queued or running", () => {
const job = {
id: "job",
status: "queued" as const,
stage: "Queued",
trigger: "manual" as const,
attempts: 0,
error: "",
cost: 0,
created_at: lens.created_at,
start: lens.created_at,
end: lens.created_at,
revision: 1,
settings: lens.settings,
assessments: [],
steps: [],
findings: [],
};
renderActions({ lens: { ...lens, jobs: [job] } });
expect(screen.getByRole("button", { name: "Run now" })).toBeDisabled();
});
});

51
ui/litellm-dashboard/src/components/lens/investigations/detail/InvestigationActions.tsx
View file

@ -8,28 +8,34 @@ import {
DropdownMenuContent,
DropdownMenuItem,
} from "@/components/ui/dropdown-menu";
import { type Lens, type Job } from "../../model/types";
import type { LensWrite } from "../../api/mutations";
import { hasActiveJob } from "../../model/status";
import { type Lens } from "../../model/types";
export interface InvestigationIntents {
readonly onEdit: () => void;
readonly onDuplicate: () => void;
readonly onPause: () => void;
readonly onEnableMonitoring: () => void;
readonly onCancelRun: () => void;
readonly onRunNow: () => void;
}
export type InvestigationActionsProps = Omit<InvestigationIntents, "onCancelRun"> & {
readonly lens: Lens;
readonly ready: boolean;
readonly busy: boolean;
};
export function InvestigationActions({
lens,
ready,
busy,
active,
setEditing,
setMonitoring,
update,
onEdit,
onDuplicate,
onPause,
onEnableMonitoring,
onRunNow,
}: {
lens: Lens;
ready: boolean;
busy: boolean;
active: Job | undefined;
setEditing: (mode: "new" | "edit" | "duplicate") => void;
setMonitoring: (open: boolean) => void;
update: (write: LensWrite) => Promise<unknown>;
onRunNow: () => void;
}) {
}: InvestigationActionsProps) {
return (
<div className="flex flex-wrap gap-2">
<DropdownMenu>
@ -37,31 +43,28 @@ export function InvestigationActions({
<MoreHorizontal className="size-4" />
</DropdownMenuTrigger>
<DropdownMenuContent align="end" className="w-48">
<DropdownMenuItem onClick={() => setEditing("edit")}>
<DropdownMenuItem onClick={onEdit}>
<Settings2 />
Edit investigation
</DropdownMenuItem>
<DropdownMenuItem disabled={!ready} onClick={() => setEditing("duplicate")}>
<DropdownMenuItem disabled={!ready} onClick={onDuplicate}>
<Copy />
Duplicate
</DropdownMenuItem>
{lens.settings.enabled ? (
<DropdownMenuItem
disabled={busy}
onClick={() => update((api) => api.saveLens(lens.id, { ...lens.settings, enabled: false }))}
>
<DropdownMenuItem disabled={busy} onClick={onPause}>
<Pause />
Pause monitoring
</DropdownMenuItem>
) : (
<DropdownMenuItem disabled={!ready || busy} onClick={() => setMonitoring(true)}>
<DropdownMenuItem disabled={!ready || busy} onClick={onEnableMonitoring}>
<Play />
Enable monitoring
</DropdownMenuItem>
)}
</DropdownMenuContent>
</DropdownMenu>
<Button disabled={busy || !!active || !ready} onClick={onRunNow}>
<Button disabled={busy || hasActiveJob(lens.jobs) || !ready} onClick={onRunNow}>
<Play className="size-3" />
Run now
</Button>

172
ui/litellm-dashboard/src/components/lens/investigations/detail/InvestigationDetail.tsx
View file

@ -1,74 +1,50 @@
"use client";
import type { useInvestigationResults } from "../useInvestigationResults";
import { Button } from "@/components/ui/button";
import { Tabs, TabsList, TabsTrigger, TabsContent } from "@/components/ui/tabs";
import { Tabs, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { RunsTab } from "./RunsTab";
import { InvestigationProgress } from "../InvestigationProgress";
import { StepFeed } from "../StepFeed";
import { InvestigationSummary } from "./InvestigationSummary";
import { InvestigationFailure } from "./InvestigationFailure";
import { scopeLabel, sourceLabels } from "../../model/format";
import { type Lens } from "../../model/types";
import type { LensWrite } from "../../api/mutations";
import type { OwnedFinding } from "../../model/inbox";
import { activeJob } from "../../model/status";
import { type Finding, type Lens } from "../../model/types";
import { FindingPanel } from "../FindingDetails";
import { useSectionRoute } from "../../route";
import { useRunSnapshot } from "../useRunSnapshot";
import { InvestigationActions } from "./InvestigationActions";
import { InvestigationActions, type InvestigationIntents } from "./InvestigationActions";
import { RunPicker } from "./RunPicker";
import { FindingsTab } from "./FindingsTab";
import { CriteriaTab } from "./CriteriaTab";
import { HistoryTab } from "./HistoryTab";
export type InvestigationDetailProps = InvestigationIntents & {
readonly lens: Lens;
readonly readOnly: boolean;
readonly ready: boolean;
readonly busy: boolean;
readonly connected: boolean;
readonly onReviewFinding: (owned: OwnedFinding, status: Finding["status"], reason: string) => void;
};
export function InvestigationDetail({
lens,
readOnly,
ready,
busy,
setEditing,
setMonitoring,
onRunNow,
update,
connected,
results,
}: {
lens: Lens;
readOnly: boolean;
ready: boolean;
busy: boolean;
setEditing: (mode: "new" | "edit" | "duplicate") => void;
setMonitoring: (open: boolean) => void;
onRunNow: () => void;
update: (write: LensWrite) => Promise<unknown>;
connected: boolean;
results: ReturnType<typeof useInvestigationResults>;
}) {
const {
active,
job,
tab,
setTab,
batchSettings,
batchId,
setBatchId,
setFindingId,
selectedOutsideHistory,
history,
historyError,
refetchHistory,
historicalError,
refetchHistorical,
missingSnapshot,
kind,
setKind,
batchFindings,
filter,
setFilter,
visibleFindings,
setRequestOffset,
setEvidence,
openBatch,
historyOffset,
setHistoryOffset,
} = results;
onCancelRun,
onReviewFinding,
...intents
}: InvestigationDetailProps) {
const { section, setSection } = useSectionRoute();
const snapshot = useRunSnapshot(lens);
const { job, batchId, batchSettings, batchFindings, missingSnapshot } = snapshot;
const active = activeJob(lens.jobs);
return (
<div>
<section className="min-w-0 space-y-5">
@ -79,107 +55,47 @@ export function InvestigationDetail({
{sourceLabels[lens.settings.source ?? "traces"]} · {scopeLabel(lens.settings)}
</p>
</div>
{!readOnly && (
<InvestigationActions
lens={lens}
ready={ready}
busy={busy}
active={active}
setEditing={setEditing}
setMonitoring={setMonitoring}
update={update}
onRunNow={onRunNow}
/>
)}
{!readOnly && <InvestigationActions lens={lens} ready={ready} busy={busy} {...intents} />}
</div>
<InvestigationSummary lens={lens} connected={connected} />
{active && (
<InvestigationProgress
key={active.id}
job={active}
onCancel={
readOnly
? undefined
: () => {
void update((api) => api.cancelRun(lens.id));
}
}
/>
)}
{active && <InvestigationProgress key={active.id} job={active} onCancel={readOnly ? undefined : onCancelRun} />}
{active && <StepFeed job={active} />}
{job?.error && <InvestigationFailure job={job} connected={connected} />}
<Tabs value={tab} onValueChange={setTab} key={lens.id}>
<Tabs value={section} onValueChange={setSection} key={lens.id}>
<div className="flex flex-wrap items-center justify-between gap-x-4 gap-y-2 border-b">
<TabsList variant="line">
<TabsTrigger value="findings">Findings</TabsTrigger>
<TabsTrigger value="checks">Criteria</TabsTrigger>
<TabsTrigger value="runs">
{
{ requests: "Requests", both: "Traces & requests", traces: "Traces" }[
batchSettings?.source ?? "traces"
]
}
</TabsTrigger>
<TabsTrigger value="runs">{sourceLabels[batchSettings?.source ?? "traces"]}</TabsTrigger>
<TabsTrigger value="activity">History</TabsTrigger>
</TabsList>
{tab !== "activity" && (
<RunPicker
batchId={batchId}
setBatchId={setBatchId}
setFindingId={setFindingId}
job={job}
selectedOutsideHistory={selectedOutsideHistory}
history={history}
lens={lens}
/>
)}
{section !== "activity" && <RunPicker lens={lens} job={job} />}
</div>
{historicalError && (
{snapshot.error && (
<p role="alert" className="text-sm text-destructive">
Could not load this run.{" "}
<Button variant="link" onClick={() => void refetchHistorical()}>
<Button variant="link" onClick={snapshot.refetch}>
Retry
</Button>
</p>
)}
{missingSnapshot && tab !== "activity" && (
{missingSnapshot && section !== "activity" && (
<p className="text-sm text-muted-foreground">
This older batch predates saved result snapshots. Its findings remain available under All accumulated
findings.
</p>
)}
<FindingsTab
kind={kind}
setKind={setKind}
batchFindings={batchFindings}
filter={filter}
setFilter={setFilter}
visibleFindings={visibleFindings}
setFindingId={setFindingId}
active={active}
lens={lens}
job={job}
/>
<CriteriaTab batchSettings={batchSettings} readOnly={readOnly} setEditing={setEditing} />
<TabsContent value="runs" className="pt-4 space-y-4">
<RunsTab
key={job?.id ?? batchId}
job={job}
onOpen={(id) => {
setRequestOffset(0);
setEvidence({ id, span: "" });
}}
<FindingsTab lens={lens} job={job} findings={batchFindings}>
<FindingPanel
readOnly={readOnly}
busy={busy}
sampledRuns={job?.sample?.executions}
onReview={onReviewFinding}
/>
</TabsContent>
<HistoryTab
history={history}
historyError={historyError}
refetchHistory={refetchHistory}
lens={lens}
openBatch={openBatch}
historyOffset={historyOffset}
setHistoryOffset={setHistoryOffset}
/>
</FindingsTab>
<CriteriaTab settings={batchSettings} readOnly={readOnly} onEditCriteria={intents.onEdit} />
<RunsTab key={job?.id ?? batchId} lens={lens} job={job} />
<HistoryTab lens={lens} />
</Tabs>
</section>
</div>

2
ui/litellm-dashboard/src/components/lens/investigations/detail/InvestigationSummary.tsx
View file

@ -1,5 +1,5 @@
import type { ComponentProps } from "react";
import { NextCheck } from "../InvestigationProgress";
import { NextCheck } from "./JobMeta";
import { lensStatus } from "../../model/status";
import { runTime } from "../../model/format";
import { type Lens } from "../../model/types";

23
ui/litellm-dashboard/src/components/lens/investigations/detail/JobMeta.tsx Normal file
View file

@ -0,0 +1,23 @@
"use client";
import { useNow } from "@/hooks/useNow";
import { analysisElapsed } from "../../model/progress";
import { nextCheckStatus } from "../../model/status";
import type { Job, Lens } from "../../model/types";
export function NextCheck({ lens }: { lens: Lens }) {
const now = useNow(15000);
const label = nextCheckStatus(lens, now);
if (!label) return null;
return <p className="mt-1 text-xs text-muted-foreground">{label}</p>;
}
export function ScanDuration({ job }: { job: Job }) {
if (!job.finished_at) return null;
return (
<span title="Total time, including any wait for an analyzer">
{" · Took "}
{analysisElapsed(job.created_at, Date.parse(job.finished_at))}
</span>
);
}

49
ui/litellm-dashboard/src/components/lens/investigations/detail/RunNowDialog.tsx
View file

@ -1,6 +1,7 @@
"use client";
import { useState } from "react";
import { useQuery } from "@tanstack/react-query";
import { Button } from "@/components/ui/button";
import {
@ -13,60 +14,30 @@ import {
} from "@/components/ui/dialog";
import { Input } from "@/components/ui/input";
import type { Lens } from "../../model/types";
export interface RunWindow {
agent_name?: string;
start?: string;
end?: string;
lookback_hours?: number;
}
const PRESETS = [
{ label: "Since last run", hours: null },
{ label: "Last hour", hours: 1 },
{ label: "Last 24h", hours: 24 },
{ label: "Last 7d", hours: 168 },
{ label: "Custom", hours: -1 },
] as const;
import { lensQueries } from "../../data/queries";
import { useLensApi } from "../../data/LensServices";
import type { Lens, RunWindow } from "../../model/types";
import { RUN_PRESETS, runRequest, type RunChoice, type RunPreset } from "../../model/runRequest";
const localInput = (date: Date) =>
new Date(date.getTime() - date.getTimezoneOffset() * 60_000).toISOString().slice(0, 16);
export interface RunChoice {
preset: (typeof PRESETS)[number]["hours"];
agent: string;
saved: string;
start: string;
end: string;
}
export function runRequest({ preset, agent, saved, start, end }: RunChoice): RunWindow | string {
const agentPart = agent.trim() && agent.trim() !== saved ? { agent_name: agent.trim() } : {};
if (preset === null) return agentPart;
if (preset > 0) return { ...agentPart, lookback_hours: preset };
const startMs = Date.parse(start);
const endMs = Date.parse(end);
if (Number.isNaN(startMs) || Number.isNaN(endMs)) return "Choose a start and end time";
if (startMs >= endMs) return "Start time must be before end time";
return { ...agentPart, start: new Date(startMs).toISOString(), end: new Date(endMs).toISOString() };
}
export function RunNowDialog({
lens,
agents,
busy,
onClose,
onRun,
}: {
lens: Lens;
agents: readonly string[];
busy: boolean;
onClose: () => void;
onRun: (request: RunWindow) => Promise<void>;
}) {
const api = useLensApi();
const agentsQuery = useQuery(lensQueries.agents(api, "traces"));
const agents = Array.isArray(agentsQuery.data) ? agentsQuery.data : [];
const now = new Date();
const [preset, setPreset] = useState<(typeof PRESETS)[number]["hours"]>(null);
const [preset, setPreset] = useState<RunPreset>(null);
const [agent, setAgent] = useState(lens.settings.agent_name ?? "");
const [start, setStart] = useState(localInput(new Date(now.getTime() - 3_600_000)));
const [end, setEnd] = useState(localInput(now));
@ -109,7 +80,7 @@ export function RunNowDialog({
<fieldset className="space-y-2">
<legend className="text-sm font-medium">Traces to review</legend>
<div className="flex flex-wrap gap-1.5">
{PRESETS.map((p) => (
{RUN_PRESETS.map((p) => (
<button
key={p.label}
type="button"

41
ui/litellm-dashboard/src/components/lens/investigations/detail/RunPicker.tsx
View file

@ -4,46 +4,39 @@ import { Info } from "lucide-react";
import { Button } from "@/components/ui/button";
import { Popover, PopoverContent, PopoverTitle, PopoverTrigger } from "@/components/ui/popover";
import { ScanDuration } from "../InvestigationProgress";
import { ScanDuration } from "./JobMeta";
import { useRunHistory } from "./useRunHistory";
import type { Lens, Job } from "../../model/types";
import { useRunRoute } from "../../route";
import { money, when } from "../../model/format";
export function RunPicker({
batchId,
setBatchId,
setFindingId,
job,
selectedOutsideHistory,
history,
lens,
}: {
batchId: string;
setBatchId: (id: string) => void;
setFindingId: (id: string | null) => void;
job: Job | undefined;
selectedOutsideHistory: boolean;
history: Job[] | undefined;
lens: Lens;
}) {
export interface RunPickerProps {
readonly lens: Lens;
readonly job: Job | undefined;
}
export function RunPicker({ lens, job }: RunPickerProps) {
const { batchId, selectRun } = useRunRoute();
const history = useRunHistory(lens, 0);
const options = history.data ?? lens.jobs;
const aggregate = batchId === "latest" || batchId === "all";
const outsideHistory = !aggregate && !options.some((j) => j.id === batchId);
return (
<div className="flex min-w-0 items-center gap-1 pb-1">
<select
aria-label="Investigation run"
className="h-8 w-44 max-w-full truncate rounded-md border-0 bg-transparent px-2 text-xs text-muted-foreground hover:bg-muted focus-visible:outline-2 focus-visible:outline-ring"
value={batchId}
onChange={(e) => {
setBatchId(e.target.value);
setFindingId(null);
}}
onChange={(e) => selectRun(e.target.value)}
>
<option value="latest">Latest run</option>
{job && selectedOutsideHistory && (
{job && outsideHistory && (
<option value={batchId}>
{when(job.created_at)} · {job.status}
</option>
)}
{(history ?? lens.jobs)?.map((j) => (
{options.map((j) => (
<option key={j.id} value={j.id}>
{when(j.created_at)} · {j.status}
</option>

198
ui/litellm-dashboard/src/components/lens/investigations/detail/RunsTab.tsx
View file

@ -1,9 +1,25 @@
import { ListRow } from "@/components/shared/ListRow";
"use client";
import { useState } from "react";
import { ChevronRight } from "lucide-react";
import { Inspector } from "@/components/shared/Inspector";
import { Button } from "@/components/ui/button";
import { TabsContent } from "@/components/ui/tabs";
import { runTime } from "../../model/format";
import { type Job } from "../../model/types";
import { type Job, type Lens } from "../../model/types";
import { EvidenceView } from "../Evidence";
import { FINDING_PANEL_WIDTH_KEY } from "../../storage";
import { useRunEvidenceRoute } from "../../route";
const PAGE = 50;
interface RunRef {
readonly id: string;
}
const runKey = (run: RunRef): string => run.id;
function assessmentLabel(assessment: Job["assessments"][number] | undefined): string {
if (!assessment) return "Not reviewed";
@ -11,7 +27,14 @@ function assessmentLabel(assessment: Job["assessments"][number] | undefined): st
return assessment.issue_checks?.length ? "Issue observed" : "No issue observed";
}
export function RunsTab({ job, onOpen }: { job?: Job; onOpen: (id: string) => void }) {
export interface RunsTabProps {
readonly lens: Lens;
readonly job?: Job;
}
/** The runs a batch reviewed; each opens its original trace or request in the side panel. */
export function RunsTab({ lens, job }: RunsTabProps) {
const { runId, selectRun } = useRunEvidenceRoute();
const [runOffset, setRunOffset] = useState(0);
const [runFilter, setRunFilter] = useState("all");
const assessments = new Map(job?.assessments?.map((a) => [a.execution_id, a]));
@ -23,84 +46,103 @@ export function RunsTab({ job, onOpen }: { job?: Job; onOpen: (id: string) => vo
if (runFilter === "clear") return assessment && !assessment.cannot_assess && !assessment.issue_checks?.length;
return assessment?.issue_checks?.includes(runFilter);
});
const page: readonly RunRef[] = visibleRuns.slice(runOffset, runOffset + PAGE);
const selected: RunRef | null = runId === null ? null : runs.find((run) => run.id === runId) ?? { id: runId };
return (
<div className="space-y-4">
<div className="flex flex-wrap items-center justify-between gap-3">
<p className="text-sm text-muted-foreground">
{runs.length} selected from {job?.sample?.eligible ?? 0} matching runs
</p>
<select
aria-label="Filter reviewed runs"
className="h-8 max-w-full rounded-md border bg-background px-2 text-xs sm:max-w-64"
value={runFilter}
onChange={(e) => {
setRunFilter(e.target.value);
setRunOffset(0);
}}
>
<option value="all">All outcomes</option>
<option value="clear">No issue observed</option>
<option value="unknown">Insufficient evidence or not reviewed</option>
{job?.settings?.context && <option value="expected_behavior">Expected behavior deviation</option>}
{job?.settings?.checks.map((check) => (
<option key={check.id} value={check.id}>
{check.instruction}
</option>
))}
</select>
</div>
<div className="divide-y border-y">
{visibleRuns.slice(runOffset, runOffset + 50).map((run) => (
<ListRow
key={run.id}
title={run.trace_id}
onClick={() => onOpen(run.id)}
className="group grid w-full grid-cols-[minmax(0,1fr)_16px] items-center gap-x-4 gap-y-1 py-4 text-left hover:bg-muted/40 focus-visible:outline-2 focus-visible:outline-ring sm:grid-cols-[minmax(0,1fr)_180px_130px_16px]"
>
<span className="col-start-1 row-start-1 min-w-0 truncate text-sm font-medium">{run.name}</span>
<span className="col-start-1 row-start-2 text-xs text-muted-foreground sm:col-start-2 sm:row-start-1">
{runTime(run.start_time)}
</span>
<span className="col-start-1 row-start-3 text-xs text-muted-foreground sm:col-start-3 sm:row-start-1">
{assessmentLabel(assessments.get(run.id))}
</span>
<ChevronRight
aria-hidden="true"
className="col-start-2 row-start-1 size-4 text-muted-foreground group-hover:text-foreground sm:col-start-4"
/>
</ListRow>
))}
{!visibleRuns.length && (
<p className="py-10 text-center text-sm text-muted-foreground">
{runs.length
? "No runs match this outcome."
: "Selected activity appears here when the worker starts reviewing it."}
<Inspector.Root
items={page}
itemKey={runKey}
selected={selected}
onSelectedChange={(run) => selectRun(run?.id ?? null)}
noun="run"
storageKey={FINDING_PANEL_WIDTH_KEY}
>
<TabsContent value="runs" className="space-y-4 pt-4">
<div className="flex flex-wrap items-center justify-between gap-3">
<p className="text-sm text-muted-foreground">
{runs.length} selected from {job?.sample?.eligible ?? 0} matching runs
</p>
)}
</div>
{(runOffset > 0 || visibleRuns.length > 50) && (
<div className="flex items-center justify-between text-xs text-muted-foreground">
<Button
size="sm"
variant="ghost"
disabled={!runOffset}
onClick={() => setRunOffset(Math.max(0, runOffset - 50))}
<select
aria-label="Filter reviewed runs"
className="h-8 max-w-full rounded-md border bg-background px-2 text-xs sm:max-w-64"
value={runFilter}
onChange={(e) => {
setRunFilter(e.target.value);
setRunOffset(0);
}}
>
Previous runs
</Button>
<span>
{runOffset + 1}–{Math.min(runOffset + 50, visibleRuns.length)} of {visibleRuns.length}
</span>
<Button
size="sm"
variant="ghost"
disabled={runOffset + 50 >= visibleRuns.length}
onClick={() => setRunOffset(runOffset + 50)}
>
Next runs
</Button>
<option value="all">All outcomes</option>
<option value="clear">No issue observed</option>
<option value="unknown">Insufficient evidence or not reviewed</option>
{job?.settings?.context && <option value="expected_behavior">Expected behavior deviation</option>}
{job?.settings?.checks.map((check) => (
<option key={check.id} value={check.id}>
{check.instruction}
</option>
))}
</select>
</div>
)}
</div>
<div className="divide-y border-y">
{visibleRuns.slice(runOffset, runOffset + PAGE).map((run) => (
<Inspector.Row
key={run.id}
item={run}
render={
<button
type="button"
title={run.trace_id}
className="group grid w-full grid-cols-[minmax(0,1fr)_16px] items-center gap-x-4 gap-y-1 py-4 text-left hover:bg-muted/40 focus-visible:outline-2 focus-visible:outline-ring data-[state=selected]:bg-trace-row-selected data-[state=selected]:shadow-[inset_2px_0_0_var(--trace-brand)] sm:grid-cols-[minmax(0,1fr)_180px_130px_16px]"
/>
}
>
<span className="col-start-1 row-start-1 min-w-0 truncate text-sm font-medium">{run.name}</span>
<span className="col-start-1 row-start-2 text-xs text-muted-foreground sm:col-start-2 sm:row-start-1">
{runTime(run.start_time)}
</span>
<span className="col-start-1 row-start-3 text-xs text-muted-foreground sm:col-start-3 sm:row-start-1">
{assessmentLabel(assessments.get(run.id))}
</span>
<ChevronRight
aria-hidden="true"
className="col-start-2 row-start-1 size-4 text-muted-foreground group-hover:text-foreground sm:col-start-4"
/>
</Inspector.Row>
))}
{!visibleRuns.length && (
<p className="py-10 text-center text-sm text-muted-foreground">
{runs.length
? "No runs match this outcome."
: "Selected activity appears here when the worker starts reviewing it."}
</p>
)}
</div>
{(runOffset > 0 || visibleRuns.length > PAGE) && (
<div className="flex items-center justify-between text-xs text-muted-foreground">
<Button
size="sm"
variant="ghost"
disabled={!runOffset}
onClick={() => setRunOffset(Math.max(0, runOffset - PAGE))}
>
Previous runs
</Button>
<span>
{runOffset + 1}–{Math.min(runOffset + PAGE, visibleRuns.length)} of {visibleRuns.length}
</span>
<Button
size="sm"
variant="ghost"
disabled={runOffset + PAGE >= visibleRuns.length}
onClick={() => setRunOffset(runOffset + PAGE)}
>
Next runs
</Button>
</div>
)}
</TabsContent>
<Inspector.Panel label="Run details" testId="run-panel">
{(run: RunRef) => <EvidenceView lensId={lens.id} evidence={{ id: run.id, span: "" }} />}
</Inspector.Panel>
</Inspector.Root>
);
}

20
ui/litellm-dashboard/src/components/lens/investigations/detail/useRunHistory.ts Normal file
View file

@ -0,0 +1,20 @@
"use client";
import { useEffect } from "react";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { useLensApi } from "../../data/LensServices";
import { lensKeys, lensQueries } from "../../data/queries";
import { activeJob } from "../../model/status";
import type { Lens } from "../../model/types";
/** Run history polls off its own rows, so a job the list poll discovers has to push it to refetch. */
export function useRunHistory(lens: Lens, historyOffset: number) {
const api = useLensApi();
const client = useQueryClient();
const activeId = activeJob(lens.jobs)?.id;
useEffect(() => {
if (!activeId) return;
void client.invalidateQueries({ queryKey: lensKeys.histories() }, { cancelRefetch: false });
}, [activeId, client]);
return useQuery(lensQueries.history(api, { lensId: lens.id, historyOffset }));
}

76
ui/litellm-dashboard/src/components/lens/investigations/investigationQuery.test.ts Normal file
View file

@ -0,0 +1,76 @@
import { describe, expect, it } from "vitest";
import { fieldValues } from "@/components/shared/search/evaluate";
import type { Job, Lens } from "../model/types";
import { filterInvestigations, INVESTIGATION_INDEX } from "./investigationQuery";
function makeLens(id: string, settings: Partial<Lens["settings"]>, jobs: readonly Pick<Job, "status">[] = []): Lens {
return {
version: 0,
spent: 0,
id,
scope: { all_teams: true, api_key_hash: "", team_id: "" },
settings: {
context: "",
source: "traces",
lookback_hours: 24,
service: "",
agent_name: "",
filters: [],
interval_minutes: 15,
sample_size: 100,
sample_percent: 100,
concurrency: 8,
team_id: "",
execution_ids: [],
monthly_budget: 20,
name: id,
model: "analysis",
enabled: true,
checks: [],
...settings,
},
revision: 1,
created_at: "2026-09-30T10:00:00Z",
next_run_at: "2026-09-30T10:00:00Z",
budget_month: "2026-09",
findings: [],
jobs: jobs as Job[],
};
}
const lenses = [
makeLens("Refund audit", { agent_name: "billing-agent" }, [{ status: "failed" }, { status: "completed" }]),
makeLens("Release reviews", { service: "reviewer", enabled: false }, [{ status: "completed" }]),
makeLens("Lead scoring", { filters: [{ key: "team", value: "sales" }] }),
];
const names = (query: string) => filterInvestigations(lenses, query).map((lens) => lens.settings.name);
describe("filterInvestigations", () => {
it("matches free text against the name and the scope", () => {
expect(names("REFUND")).toEqual(["Refund audit"]);
expect(names("sales")).toEqual(["Lead scoring"]);
expect(names("reviewer")).toEqual(["Release reviews"]);
});
it("filters by the latest run's status, treating no runs as never", () => {
expect(names("status:failed")).toEqual(["Refund audit"]);
expect(names("status:completed")).toEqual(["Release reviews"]);
expect(names("status:never")).toEqual(["Lead scoring"]);
});
it("filters by schedule and by agent, falling back to the service", () => {
expect(names("schedule:paused")).toEqual(["Release reviews"]);
expect(names("-schedule:paused")).toEqual(["Refund audit", "Lead scoring"]);
expect(names("agent:billing-agent")).toEqual(["Refund audit"]);
expect(names("agent:reviewer")).toEqual(["Release reviews"]);
});
});
describe("INVESTIGATION_INDEX values", () => {
it("offers only the agents and statuses present", () => {
expect(fieldValues(INVESTIGATION_INDEX, lenses, "agent")).toEqual(["billing-agent", "reviewer"]);
expect(fieldValues(INVESTIGATION_INDEX, lenses, "status")).toEqual(["completed", "failed", "never"]);
});
});

30
ui/litellm-dashboard/src/components/lens/investigations/investigationQuery.ts Normal file
View file

@ -0,0 +1,30 @@
import { Bot, CalendarClock, CircleDashed, SquareChevronRight } from "lucide-react";
import { type ClientIndex, filterItems } from "@/components/shared/search/evaluate";
import type { FieldSpec, QueryLanguage } from "@/components/shared/search/language";
import { scopeLabel } from "../model/format";
import type { Lens } from "../model/types";
const INVESTIGATION_FIELDS = {
name: { group: "Investigation", icon: SquareChevronRight, suggestValues: true },
agent: { group: "Investigation", icon: Bot, suggestValues: true },
status: { group: "Latest run", icon: CircleDashed, suggestValues: true },
schedule: { group: "Investigation", icon: CalendarClock, suggestValues: true },
} as const satisfies Record<string, FieldSpec>;
export type InvestigationField = keyof typeof INVESTIGATION_FIELDS;
export const INVESTIGATION_QUERY: QueryLanguage<InvestigationField> = { fields: INVESTIGATION_FIELDS };
export const INVESTIGATION_INDEX: ClientIndex<Lens, InvestigationField> = {
read: {
name: (lens) => [lens.settings.name],
agent: (lens) => [lens.settings.agent_name, lens.settings.service].filter(Boolean),
status: (lens) => [lens.jobs[0]?.status ?? "never"],
schedule: (lens) => [lens.settings.enabled ? "watching" : "paused"],
},
freeText: (lens) => [lens.settings.name, scopeLabel(lens.settings)],
};
export const filterInvestigations = (lenses: Lens[], query: string): Lens[] =>
filterItems(INVESTIGATION_QUERY, INVESTIGATION_INDEX, lenses, query);

103
ui/litellm-dashboard/src/components/lens/investigations/investigationScreen.test.ts Normal file
View file

@ -0,0 +1,103 @@
import { describe, expect, it } from "vitest";
import { investigationScreen, type ScreenInput } from "./investigationScreen";
import type { Lens, LensList } from "../model/types";
function makeLens(id: string, created_at: string): Lens {
return {
version: 0,
spent: 0,
id,
scope: { all_teams: true, api_key_hash: "", team_id: "" },
settings: {
context: "",
source: "traces",
lookback_hours: 24,
service: "",
agent_name: "",
filters: [],
interval_minutes: 15,
sample_size: 100,
sample_percent: 100,
concurrency: 8,
team_id: "",
execution_ids: [],
monthly_budget: 20,
name: `Lens ${id}`,
model: "analysis",
enabled: true,
checks: [],
},
revision: 1,
created_at,
next_run_at: created_at,
budget_month: "2026-09",
findings: [],
jobs: [],
};
}
const older = makeLens("older", "2026-09-01T10:00:00Z");
const newer = makeLens("newer", "2026-09-30T10:00:00Z");
const list = (lenses: Lens[]): LensList => ({ lenses, workers: [], tracing_enabled: true });
const ready = (lenses: Lens[]): ScreenInput["list"] => ({ data: list(lenses), error: null });
const route = { lensId: null, dialog: null, target: null } as const;
describe("investigationScreen", () => {
it("loads, then fails only while there is nothing cached to show", () => {
expect(investigationScreen({ ...route, list: { data: undefined, error: null } })).toEqual({ kind: "loading" });
const error = new Error("down");
expect(investigationScreen({ ...route, list: { data: undefined, error } })).toEqual({ kind: "failed", error });
expect(investigationScreen({ ...route, list: { data: list([newer]), error } })).toMatchObject({ kind: "list" });
});
it("welcomes an empty account and lists newest investigations first otherwise", () => {
expect(investigationScreen({ ...route, list: ready([]) })).toEqual({ kind: "welcome" });
expect(investigationScreen({ ...route, list: ready([older, newer]) })).toEqual({
kind: "list",
lenses: [newer, older],
});
});
it("keeps the list on screen with the selected investigation, or reports it missing", () => {
expect(investigationScreen({ ...route, list: ready([older, newer]), lensId: "older" })).toEqual({
kind: "list",
lenses: [newer, older],
lens: older,
});
expect(investigationScreen({ ...route, list: ready([older]), lensId: "gone" })).toEqual({ kind: "missing" });
expect(investigationScreen({ ...route, list: { data: undefined, error: null }, lensId: "older" })).toEqual({
kind: "loading",
});
});
it("enters setup for new, edit and duplicate, prefilled from the targeted investigation", () => {
expect(investigationScreen({ ...route, list: ready([older]), dialog: "new" })).toEqual({
kind: "setup",
mode: "new",
});
const edit: ScreenInput = { ...route, list: ready([older]), lensId: "older", dialog: "edit" };
const editing = { kind: "setup", mode: "edit", lens: older, initial: older.settings };
expect(investigationScreen(edit)).toEqual(editing);
const duplicate: ScreenInput = {
...route,
list: ready([older, newer]),
lensId: "older",
dialog: "duplicate",
target: "newer",
};
const duplicating = {
kind: "setup",
mode: "duplicate",
lens: newer,
initial: { ...newer.settings, name: "Lens newer copy", enabled: false },
};
expect(investigationScreen(duplicate)).toEqual(duplicating);
});
it("falls back to the plain screen when an edit names an investigation that is not loaded", () => {
const editGone: ScreenInput = { ...route, list: ready([older]), lensId: "gone", dialog: "edit" };
expect(investigationScreen(editGone)).toEqual({ kind: "missing" });
const runNow: ScreenInput = { ...route, list: ready([older]), dialog: "run_now", target: "older" };
expect(investigationScreen(runNow)).toMatchObject({ kind: "list" });
});
});

51
ui/litellm-dashboard/src/components/lens/investigations/investigationScreen.ts Normal file
View file

@ -0,0 +1,51 @@
import type { Lens, LensList, Settings } from "../model/types";
import type { LensDialog } from "../route";
export type SetupMode = "new" | "edit" | "duplicate";
export interface ScreenInput {
readonly list: { readonly data: LensList | undefined; readonly error: Error | null };
readonly lensId: string | null;
readonly dialog: LensDialog | null;
readonly target: string | null;
}
export type SetupScreen =
| { readonly kind: "setup"; readonly mode: "new" }
| { readonly kind: "setup"; readonly mode: "edit" | "duplicate"; readonly lens: Lens; readonly initial: Settings };
export type Screen =
| { readonly kind: "loading" }
| { readonly kind: "failed"; readonly error: Error }
| { readonly kind: "welcome" }
| { readonly kind: "list"; readonly lenses: readonly Lens[]; readonly lens?: Lens }
| { readonly kind: "missing" }
| SetupScreen;
const isSetupMode = (dialog: LensDialog | null): dialog is SetupMode =>
dialog === "new" || dialog === "edit" || dialog === "duplicate";
function setupScreen(mode: SetupMode, lens: Lens | undefined): SetupScreen | undefined {
if (mode === "new") return { kind: "setup", mode };
if (!lens) return undefined;
const initial =
mode === "duplicate" ? { ...lens.settings, name: `${lens.settings.name} copy`, enabled: false } : lens.settings;
return { kind: "setup", mode, lens, initial };
}
export function investigationScreen({ list, lensId, dialog, target }: ScreenInput): Screen {
const lenses = list.data?.lenses ?? [];
const selected = lenses.find((lens) => lens.id === lensId);
const setup = isSetupMode(dialog)
? setupScreen(dialog, target ? lenses.find((lens) => lens.id === target) : selected)
: undefined;
if (setup) return setup;
if (!list.data) return list.error ? { kind: "failed", error: list.error } : { kind: "loading" };
if (lensId && !selected) return { kind: "missing" };
if (lenses.length === 0) return { kind: "welcome" };
return {
kind: "list",
lenses: [...lenses].sort((a, b) => Date.parse(b.created_at) - Date.parse(a.created_at)),
...(selected ? { lens: selected } : {}),
};
}

26
ui/litellm-dashboard/src/components/lens/investigations/useInvestigationActions.ts Normal file
View file

@ -0,0 +1,26 @@
"use client";
import { useLensUpdate, useSaveLens, type LensWrite } from "../data/mutations";
import type { Finding, Lens, RunWindow, Settings } from "../model/types";
/** Every write the Investigations screens can ask for, so leaves only ever express intent. */
export function useInvestigationActions() {
const update = useLensUpdate();
const save = useSaveLens();
const attempt = (write: LensWrite): Promise<boolean> =>
update.mutateAsync(write).then(
() => true,
() => false,
);
return {
busy: update.isPending,
error: update.error,
reset: update.reset,
save: (input: { id?: string; settings: Settings }): Promise<Lens> => save.mutateAsync(input),
pause: (lens: Lens) => attempt((api) => api.saveLens(lens.id, { ...lens.settings, enabled: false })),
cancelRun: (lens: Lens) => attempt((api) => api.cancelRun(lens.id)),
startRun: (lens: Lens, request: RunWindow) => attempt((api) => api.startRun(lens.id, request)),
review: (lens: Lens, finding: Finding, status: Finding["status"], reason: string) =>
attempt((api) => api.reviewFinding(lens.id, finding.id, status, reason)),
};
}

107
ui/litellm-dashboard/src/components/lens/investigations/useInvestigationResults.ts
View file

@ -1,107 +0,0 @@
"use client";
import { useQueryStates } from "nuqs";
import { useCallback, useState } from "react";
import { useQuery } from "@tanstack/react-query";
import { useLensApi } from "../services";
import { lensQueries } from "../api/queries";
import { evidenceTarget, mergeFeedback, sortedFindings } from "../model/findings";
import type { Job, Lens } from "../model/types";
import { RESULT_PARSERS, RESULT_TABS, type ResultTab } from "../route";
const isResultTab = (tab: string): tab is ResultTab => (RESULT_TABS as readonly string[]).includes(tab);
export function useInvestigationResults(accessToken: string, lens: Lens | undefined) {
const api = useLensApi(accessToken);
const [params, setParams] = useQueryStates(RESULT_PARSERS);
const { run: batchId, section: tab, finding: findingId, finding_status: filter, kind } = params;
const [historyOffset, setHistoryOffset] = useState(0);
const setBatchId = useCallback((run: string) => void setParams({ run }), [setParams]);
const setTab = useCallback((section: string) => void (isResultTab(section) && setParams({ section })), [setParams]);
const setFindingId = useCallback(
(finding: string | null) => void setParams({ finding }, { history: "push" }),
[setParams],
);
const setFilter = useCallback((finding_status: string) => void setParams({ finding_status }), [setParams]);
const setKind = useCallback((next: typeof kind) => void setParams({ kind: next }), [setParams]);
const evidence = params.evidence === null ? null : { id: params.evidence, span: params.evidence_span ?? "" };
const setEvidence = useCallback(
(next: { id: string; span: string } | null) =>
void setParams({ evidence: next?.id ?? null, evidence_span: next?.span || null }, { history: "push" }),
[setParams],
);
const history = useQuery(lensQueries.history(api, { lensId: lens?.id, historyOffset }));
const historical = useQuery(lensQueries.run(api, lens?.id, batchId));
const { job, missingSnapshot, selectedOutsideHistory, batchSettings, batchFindings } = runSnapshot(
lens,
batchId,
historical.data,
history.data,
);
const finding = batchFindings.find((f) => f.id === findingId);
const openBatch = (id: string) => void setParams({ run: id, section: null, finding: null });
const active = lens?.jobs?.find((j) => j.status === "queued" || j.status === "running");
const visibleFindings = sortedFindings(
batchFindings.filter((f) => (filter === "all" || f.status === filter) && f.kind === kind),
);
const sampledRuns = job?.sample?.executions ?? [];
const target = evidence ? evidenceTarget(evidence.id) : null;
const [requestOffset, setRequestOffset] = useState(0);
const evidenceInput = { lensId: lens?.id, evidenceId: evidence?.id, requestOffset, source: target?.source };
const requestEvidence = useQuery(lensQueries.evidence(api, evidenceInput));
const reset = useCallback(() => {
setHistoryOffset(0);
void setParams({ run: null, finding: null, section: null });
}, [setParams]);
return {
batchId,
setBatchId,
historyOffset,
setHistoryOffset,
tab,
setTab,
findingId,
setFindingId,
filter,
setFilter,
kind,
setKind,
evidence,
setEvidence,
history: history.data,
historyError: history.error,
refetchHistory: history.refetch,
historicalError: historical.error,
refetchHistorical: historical.refetch,
job,
missingSnapshot,
selectedOutsideHistory,
batchSettings,
batchFindings,
finding,
openBatch,
active,
visibleFindings,
sampledRuns,
target,
requestOffset,
setRequestOffset,
requestEvidenceData: requestEvidence.data,
requestEvidenceError: requestEvidence.error,
requestEvidenceLoading: requestEvidence.isLoading,
reset,
};
}
function runSnapshot(lens: Lens | undefined, batchId: string, historical: Job | undefined, history: Job[] | undefined) {
const job = ["latest", "all"].includes(batchId) ? lens?.jobs?.[0] : historical;
const missingSnapshot = job?.status === "completed" && job.findings == null && batchId !== "all";
const selectedOutsideHistory = !["latest", "all"].includes(batchId) && !history?.some((j) => j.id === batchId);
const batchSettings = job?.settings ?? lens?.settings;
const batchFindings = mergeFeedback(
batchId === "all" ? lens?.findings ?? [] : job?.findings ?? [],
lens?.findings ?? [],
);
return { job, missingSnapshot, selectedOutsideHistory, batchSettings, batchFindings };
}

32
ui/litellm-dashboard/src/components/lens/investigations/useProgressSamples.ts Normal file
View file

@ -0,0 +1,32 @@
"use client";
import { useEffect, useState, useSyncExternalStore } from "react";
import { analysisFraction, type ProgressSample } from "../model/progress";
const KEEP = 120;
type Progress = Pick<ProgressSample, "step" | "done"> & { readonly total: number };
/** A wall-clock log of progress changes; the clock is the external system, so it lives outside render. */
function createSampleLog() {
const listeners = new Set<() => void>();
const state = { samples: [] as readonly ProgressSample[] };
return {
subscribe: (listener: () => void) => {
listeners.add(listener);
return () => void listeners.delete(listener);
},
snapshot: () => state.samples,
record: (progress: Progress) => {
const sample = { at: Date.now(), ...progress, fraction: analysisFraction(progress) };
state.samples = [...state.samples, sample].slice(-KEEP);
listeners.forEach((listener) => listener());
},
};
}
export function useProgressSamples({ step, done, total }: Progress): readonly ProgressSample[] {
const [log] = useState(createSampleLog);
useEffect(() => log.record({ step, done, total }), [log, step, done, total]);
return useSyncExternalStore(log.subscribe, log.snapshot, log.snapshot);
}

38
ui/litellm-dashboard/src/components/lens/investigations/useRunSnapshot.ts Normal file
View file

@ -0,0 +1,38 @@
"use client";
import { useQuery } from "@tanstack/react-query";
import { useLensApi } from "../data/LensServices";
import { lensQueries } from "../data/queries";
import { mergeFeedback } from "../model/findings";
import type { Finding, Job, Lens, Settings } from "../model/types";
import { useRunRoute } from "../route";
export interface RunSnapshot {
readonly batchId: string;
readonly job: Job | undefined;
readonly missingSnapshot: boolean;
readonly batchSettings: Settings | undefined;
readonly batchFindings: readonly Finding[];
readonly error: Error | null;
readonly refetch: () => void;
}
const isAggregate = (batchId: string) => batchId === "latest" || batchId === "all";
/** The run a detail view is looking at: the latest job from the list, or one older job fetched on its own. */
export function useRunSnapshot(lens: Lens | undefined): RunSnapshot {
const api = useLensApi();
const { batchId } = useRunRoute();
const historical = useQuery(lensQueries.run(api, lens?.id, batchId));
const job = isAggregate(batchId) ? lens?.jobs[0] : historical.data;
const current = lens?.findings ?? [];
return {
batchId,
job,
missingSnapshot: job?.status === "completed" && job.findings == null && batchId !== "all",
batchSettings: job?.settings ?? lens?.settings,
batchFindings: mergeFeedback(batchId === "all" ? current : job?.findings ?? [], current),
error: historical.error,
refetch: () => void historical.refetch(),
};
}

9
ui/litellm-dashboard/src/components/lens/model/findings.ts
View file

@ -8,7 +8,14 @@ export function sortedFindings(findings: Finding[]): Finding[] {
);
}
export function evidenceTarget(id: string): { source: string; team: string; id: string; traceRef?: string } | null {
export interface EvidenceTarget {
readonly source: string;
readonly team: string;
readonly id: string;
readonly traceRef?: string;
}
export function evidenceTarget(id: string): EvidenceTarget | null {
try {
const parsed: unknown = JSON.parse(atob(id.replace(/-/g, "+").replace(/_/g, "/")));
if (!Array.isArray(parsed) || ![3, 4].includes(parsed.length) || !parsed.every((item) => typeof item === "string"))

18
ui/litellm-dashboard/src/components/lens/model/format.test.ts
View file

@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import { durationLabel, durationText, money, scopeLabel, when } from "./format";
import { agoLabel, durationLabel, durationText, money, scopeLabel, when } from "./format";
describe("Lens labels", () => {
it("formats an empty scope and joins recorded scope constraints in their original order", () => {
@ -30,3 +30,19 @@ describe("Lens labels", () => {
expect(when(null)).toBe("Not yet");
});
});
describe("agoLabel", () => {
const now = Date.UTC(2026, 9, 3, 12, 0, 0);
it("rounds down to the largest whole unit", () => {
expect(agoLabel(now - 2_000, now)).toBe("just now");
expect(agoLabel(now - 45_000, now)).toBe("45s ago");
expect(agoLabel(now - 12 * 60_000 - 59_000, now)).toBe("12m ago");
expect(agoLabel(now - 3 * 3_600_000, now)).toBe("3h ago");
expect(agoLabel(now - 2 * 86_400_000, now)).toBe("2d ago");
});
it("treats a timestamp slightly in the future as just now instead of negative", () => {
expect(agoLabel(now + 10_000, now)).toBe("just now");
});
});

12
ui/litellm-dashboard/src/components/lens/model/format.ts
View file

@ -1,6 +1,7 @@
import { formatActivityTimestamp as runTime } from "@/utils/activityTimestamp";
import type { Settings } from "./types";
export { formatActivityTimestamp as runTime } from "@/utils/activityTimestamp";
export { runTime };
export function scopeLabel(settings: Partial<Pick<Settings, "service" | "agent_name" | "filters">>): string {
return (
@ -27,6 +28,15 @@ export function durationLabel(value: number, base: "minutes" | "hours" = "minute
return `${minutes} ${minutes === 1 ? "minute" : "minutes"}`;
}
export function agoLabel(thenMs: number, nowMs: number): string {
const seconds = Math.max(0, Math.floor((nowMs - thenMs) / 1000));
if (seconds < 5) return "just now";
if (seconds < 60) return `${seconds}s ago`;
if (seconds < 3600) return `${Math.floor(seconds / 60)}m ago`;
if (seconds < 86400) return `${Math.floor(seconds / 3600)}h ago`;
return `${Math.floor(seconds / 86400)}d ago`;
}
export const money = (n: number) =>
new Intl.NumberFormat("en-US", { style: "currency", currency: "USD", maximumFractionDigits: 3 }).format(n);
export const when = (value?: string | null) => (value ? runTime(value) : "Not yet");

7
ui/litellm-dashboard/src/components/lens/model/inbox.ts
View file

@ -38,7 +38,12 @@ export function openFindings(lens: Lens): Finding[] {
export const findingKey = (lens: Lens, finding: Finding) => `${lens.id}:${finding.id}`;
export function findFinding(lenses: readonly Lens[], key: string) {
export interface OwnedFinding {
readonly lens: Lens;
readonly finding: Finding;
}
export function findFinding(lenses: readonly Lens[], key: string): OwnedFinding | undefined {
return lenses
.flatMap((lens) => lens.findings.map((finding) => ({ lens, finding })))
.find(({ lens, finding }) => findingKey(lens, finding) === key);

2
ui/litellm-dashboard/src/components/lens/model/progress.ts
View file

@ -82,7 +82,7 @@ export interface ProgressSample {
fraction: number;
}
export const stageWeights = [0.6, 0.2, 0.2];
const stageWeights = [0.6, 0.2, 0.2];
export function analysisFraction({
step,

53
ui/litellm-dashboard/src/components/lens/model/readiness.test.ts Normal file
View file

@ -0,0 +1,53 @@
import { describe, expect, it } from "vitest";
import { initialSetupStep, readiness, type ReadinessInput } from "./readiness";
const base: ReadinessInput = {
traces: { recorded: undefined, failed: false, disabled: false, checked: false },
tracingConfigured: false,
activity: undefined,
activityError: null,
connected: true,
listError: null,
investigations: 0,
};
describe("readiness", () => {
it("counts traces confirmed from storage as recorded activity even when the activity check fails", () => {
const failing = { ...base, activityError: new Error("Activity unavailable") };
expect(readiness(failing).ready).toBe(false);
const seen = { ...failing, traces: { ...base.traces, recorded: true } };
expect(readiness(seen)).toMatchObject({ tracesReady: true, ready: true });
expect(readiness({ ...seen, traces: { ...seen.traces, failed: true } }).tracesReady).toBe(false);
});
it("needs a connected worker and a healthy list on top of recorded activity", () => {
const active = { ...base, activity: { traces: false, requests: true } };
expect(readiness(active)).toMatchObject({ requestsReady: true, activityReady: true, ready: true });
expect(readiness({ ...active, connected: false }).ready).toBe(false);
expect(readiness({ ...active, listError: new Error("down") }).ready).toBe(false);
});
it("treats tracing as enabled once storage answers or the gateway reports it configured, unless storage says it is off", () => {
expect(readiness(base).tracingEnabled).toBe(false);
expect(readiness({ ...base, tracingConfigured: true }).tracingEnabled).toBe(true);
expect(readiness({ ...base, traces: { ...base.traces, checked: true } }).tracingEnabled).toBe(true);
const disabled = { ...base, tracingConfigured: true, traces: { ...base.traces, checked: true, disabled: true } };
expect(readiness(disabled).tracingEnabled).toBe(false);
});
it("knows whether any investigation exists", () => {
expect(readiness(base).hasInvestigations).toBe(false);
expect(readiness({ ...base, investigations: 2 }).hasInvestigations).toBe(true);
});
});
describe("initialSetupStep", () => {
it("opens the first step that still needs work", () => {
const offline = { ...base, connected: false };
expect(initialSetupStep(readiness(offline))).toBe(0);
expect(initialSetupStep(readiness({ ...offline, tracingConfigured: true }))).toBe(1);
const recorded = { ...offline, activity: { traces: false, requests: true } };
expect(initialSetupStep(readiness(recorded))).toBe(2);
expect(initialSetupStep(readiness({ ...recorded, connected: true }))).toBe(3);
});
});

47
ui/litellm-dashboard/src/components/lens/model/readiness.ts Normal file
View file

@ -0,0 +1,47 @@
export interface Readiness {
readonly tracingEnabled: boolean;
readonly tracesReady: boolean;
readonly requestsReady: boolean;
readonly activityReady: boolean;
readonly connected: boolean;
readonly hasInvestigations: boolean;
readonly ready: boolean;
}
export interface ReadinessInput {
readonly traces: {
readonly recorded: boolean | undefined;
readonly failed: boolean;
readonly disabled: boolean;
readonly checked: boolean;
};
readonly tracingConfigured: boolean;
readonly activity: { readonly traces: boolean; readonly requests: boolean } | undefined;
readonly activityError: unknown;
readonly connected: boolean;
readonly listError: unknown;
readonly investigations: number;
}
export function readiness(input: ReadinessInput): Readiness {
const { traces, activity, activityError, connected, listError } = input;
/** Traces confirmed straight from trace storage count as recorded activity even if the activity check fails. */
const tracesSeen = traces.recorded === true && !traces.failed;
const tracesReady = tracesSeen || (activity?.traces === true && !activityError);
const requestsReady = activity?.requests === true && !activityError;
const activityReady = tracesReady || requestsReady;
return {
tracingEnabled: !traces.disabled && (traces.checked || input.tracingConfigured),
tracesReady,
requestsReady,
activityReady,
connected,
hasInvestigations: input.investigations > 0,
ready: activityReady && connected && !listError,
};
}
export function initialSetupStep(state: Readiness): number {
if (state.activityReady) return state.connected ? 3 : 2;
return state.tracingEnabled ? 1 : 0;
}

2
ui/litellm-dashboard/src/components/lens/investigations/detail/RunNowDialog.test.ts → ui/litellm-dashboard/src/components/lens/model/runRequest.test.ts
View file

@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { runRequest } from "./RunNowDialog";
import { runRequest } from "./runRequest";
const base = { agent: "support", saved: "support", start: "", end: "" };

30
ui/litellm-dashboard/src/components/lens/model/runRequest.ts Normal file
View file

@ -0,0 +1,30 @@
import type { RunWindow } from "./types";
export const RUN_PRESETS = [
{ label: "Since last run", hours: null },
{ label: "Last hour", hours: 1 },
{ label: "Last 24h", hours: 24 },
{ label: "Last 7d", hours: 168 },
{ label: "Custom", hours: -1 },
] as const;
export type RunPreset = (typeof RUN_PRESETS)[number]["hours"];
export interface RunChoice {
preset: RunPreset;
agent: string;
saved: string;
start: string;
end: string;
}
export function runRequest({ preset, agent, saved, start, end }: RunChoice): RunWindow | string {
const agentPart = agent.trim() && agent.trim() !== saved ? { agent_name: agent.trim() } : {};
if (preset === null) return agentPart;
if (preset > 0) return { ...agentPart, lookback_hours: preset };
const startMs = Date.parse(start);
const endMs = Date.parse(end);
if (Number.isNaN(startMs) || Number.isNaN(endMs)) return "Choose a start and end time";
if (startMs >= endMs) return "Start time must be before end time";
return { ...agentPart, start: new Date(startMs).toISOString(), end: new Date(endMs).toISOString() };
}

48
ui/litellm-dashboard/src/components/lens/model/status.test.ts
View file

@ -1,4 +1,4 @@
import { investigationActivity, nextCheckStatus, workerConnected } from "./status";
import { activeJob, investigationActivity, listPollInterval, nextCheckStatus, workerConnected } from "./status";
import { describe, expect, it } from "vitest";
import type { Job, Lens, LensList } from "./types";
@ -111,3 +111,49 @@ it("summarizes activity across investigations, preferring a running scan over a
expect(investigationActivity([withStatus("queued"), withStatus("running")])).toBe("running");
expect(investigationActivity([{ ...lens, jobs: [lens.jobs[0], { ...job, status: "running" }] }])).toBe("running");
});
describe("List polling cadence", () => {
const now = Date.parse("2026-10-01T12:00:00Z");
const connectedWorker: LensList["workers"][number] = {
id: "worker",
name: "Worker",
revoked: false,
analysis_key_id: "key",
last_seen: "2026-10-01T11:59:59Z",
scope: { all_teams: true, api_key_hash: "", team_id: "" },
};
const staleWorker = { ...connectedWorker, last_seen: "2026-10-01T11:00:00Z" };
const list = (workers: LensList["workers"], lenses: Lens[] = [lens]): LensList => ({
lenses,
workers,
tracing_enabled: true,
});
it("polls slowly when nothing is running and Settings is closed", () => {
expect(listPollInterval(list([staleWorker]), false, now)).toBe(10000);
expect(listPollInterval(undefined, false, now)).toBe(10000);
});
it("polls fast while Settings waits for a worker and slows down once one connects", () => {
expect(listPollInterval(undefined, true, now)).toBe(2000);
expect(listPollInterval(list([staleWorker]), true, now)).toBe(2000);
expect(listPollInterval(list([connectedWorker]), true, now)).toBe(10000);
});
it("speeds up as soon as any investigation is queued or running, whatever tab is open", () => {
const queued: Lens = { ...lens, jobs: [{ ...job, status: "queued" }] };
const running: Lens = { ...lens, jobs: [lens.jobs[0], { ...job, status: "running" }] };
expect(listPollInterval(list([connectedWorker], [lens, queued]), false, now)).toBe(2000);
expect(listPollInterval(list([connectedWorker], [running]), false, now)).toBe(2000);
expect(listPollInterval(list([connectedWorker], [lens]), false, now)).toBe(10000);
});
});
describe("activeJob", () => {
it("picks the queued or running job and ignores finished ones", () => {
const done = { ...job, id: "done", status: "completed" as const };
expect(activeJob([done, job])).toBe(job);
expect(activeJob([done, { ...job, status: "queued" }])?.status).toBe("queued");
expect(activeJob([done])).toBeUndefined();
});
});

30
ui/litellm-dashboard/src/components/lens/model/status.ts
View file

@ -1,10 +1,25 @@
import { formatActivityTimestamp } from "@/utils/activityTimestamp";
import type { Lens, LensList } from "./types";
import type { Job, Lens, LensList } from "./types";
export function workerConnected(worker: LensList["workers"][number], now = Date.now()): boolean {
return !worker.revoked && !!worker.analysis_key_id && now - Date.parse(worker.last_seen) < 120000;
}
export function activeJob(jobs: readonly Job[]): Job | undefined {
return jobs.find((job) => job.status === "queued" || job.status === "running");
}
export function hasActiveJob(jobs: readonly Job[]): boolean {
return activeJob(jobs) !== undefined;
}
/** One observer polls `/lens`: fast while work is in flight or a worker is being connected, slow otherwise. */
export function listPollInterval(list: LensList | undefined, settingsOpen: boolean, now: number): number {
const running = list?.lenses.some((lens) => hasActiveJob(lens.jobs)) ?? false;
const connected = list?.workers.some((worker) => workerConnected(worker, now)) ?? false;
return running || (settingsOpen && !connected) ? 2000 : 10000;
}
export function lensStatus(lens: Lens, connected: boolean): string {
const active = lens.jobs?.find((job) => ["queued", "running"].includes(job.status ?? ""));
if (active) return connected ? active.stage ?? "Queued" : "Waiting for analyzer";
@ -31,19 +46,6 @@ export function nextCheckStatus(lens: Lens, now: number): string | null {
return `Next check ${time} · ${relative}`;
}
export function readiness(
activity: { traces: boolean; requests: boolean } | undefined,
activityError: unknown,
connected: boolean,
listError: unknown,
) {
const tracesReady = activity?.traces === true && !activityError;
const requestsReady = activity?.requests === true && !activityError;
const activityReady = tracesReady || requestsReady;
const ready = activityReady && connected && !listError;
return { tracesReady, requestsReady, activityReady, ready };
}
export type InvestigationActivity = "running" | "queued" | "idle";
export function investigationActivity(lenses: readonly Lens[]): InvestigationActivity {

14
ui/litellm-dashboard/src/components/lens/model/types.ts
View file

@ -32,3 +32,17 @@ export type ActivitySelection = Pick<Settings, "source"> &
>;
export type Worker = LensList["workers"][number];
export interface RunWindow {
agent_name?: string;
start?: string;
end?: string;
lookback_hours?: number;
}
export interface AnalysisModelInfo {
model_group: string;
providers: string[];
mode?: string | null;
supported_openai_params?: string[] | null;
}

2
ui/litellm-dashboard/src/components/lens/setup/watches.ts → ui/litellm-dashboard/src/components/lens/model/watches.ts
View file

@ -1,4 +1,4 @@
import type { Settings } from "../model/types";
import type { Settings } from "./types";
export interface Watch {
id: string;

5
ui/litellm-dashboard/src/components/lens/setup/GatewayFlow.tsx → ui/litellm-dashboard/src/components/lens/onboarding/GatewayFlow.tsx
View file

@ -1,4 +1,5 @@
import { useId } from "react";
import { cn } from "@/lib/cva.config";
import styles from "./LensIntroduction.module.css";
const dotColors = [
@ -45,7 +46,7 @@ export function GatewayFlow() {
<svg
aria-hidden="true"
viewBox="0 0 1000 108"
className={`${styles.flowGraphic} mt-2 h-auto w-full`}
className={cn(styles.flowGraphic, "mt-2 h-auto w-full")}
focusable="false"
>
<defs>
@ -105,7 +106,7 @@ export function GatewayFlow() {
<ellipse cx="500" cy="54" rx="44" ry="40" fill={`url(#${id}-glow)`} className={styles.gatewayGlow} />
<g clipPath={`url(#${id}-after-gate)`}>
<g mask={`url(#${id}-organized-shape)`}>
<g className={`${styles.organizedDots} fill-indigo-500 dark:fill-indigo-400`}>
<g className={cn(styles.organizedDots, "fill-indigo-500 dark:fill-indigo-400")}>
{organizedColumns.map((column) => (
<g key={column} opacity={0.6 + (((column + 8) * 3) % 8) * 0.05}>
{[34, 44, 54, 64, 74].map((y) => (

81
ui/litellm-dashboard/src/components/lens/onboarding/LensGettingStarted.tsx Normal file
View file

@ -0,0 +1,81 @@
"use client";
import { useRef } from "react";
import { Check, ShieldCheck } from "lucide-react";
import { Button } from "@/components/ui/button";
import type { LensReadiness } from "../hooks/useLensReadiness";
import { LensIntroduction } from "./LensIntroduction";
import { OnboardingSetup } from "./OnboardingSetup";
const PREREQUISITES = [
{ title: "LiteLLM gateway", detail: "Access to its configuration" },
{ title: "ClickHouse", detail: "Self-hosted or managed trace storage" },
{ title: "A server with Docker", detail: "To run the analysis worker" },
{ title: "An analysis model", detail: "Available through your gateway" },
] as const;
export interface LensGettingStartedProps {
readonly state: LensReadiness;
readonly onStart: () => void;
readonly onExit: (to: "traces" | "investigations") => void;
readonly onDemo?: () => void;
}
export function LensGettingStarted({ state, onStart, onExit, onDemo }: LensGettingStartedProps) {
const setupRef = useRef<HTMLElement>(null);
const exitTo = state.tracesReady ? "traces" : "investigations";
const start = () => {
onStart();
setupRef.current?.scrollIntoView({ block: "start" });
setupRef.current?.focus({ preventScroll: true });
};
return (
<div className="w-full space-y-6">
<LensIntroduction onStart={start} onDemo={onDemo} />
<div className="grid items-start gap-6 xl:grid-cols-[minmax(0,1fr)_280px] xl:gap-8">
<OnboardingSetup
ref={setupRef}
tabIndex={-1}
state={state}
onFocusCapture={onStart}
action={
(state.activityReady || state.hasInvestigations) && (
<Button variant="outline" size="sm" onClick={() => onExit(exitTo)}>
{exitTo === "traces" ? "View traces" : "View investigations"}
</Button>
)
}
/>
<Prerequisites />
</div>
</div>
);
}
function Prerequisites() {
return (
<aside className="rounded-2xl border bg-card p-6" aria-labelledby="lens-prerequisites">
<h2 id="lens-prerequisites" className="text-base font-semibold">
Before you start
</h2>
<ul className="mt-5 space-y-5 text-sm">
{PREREQUISITES.map((item) => (
<li key={item.title} className="flex gap-3">
<Check aria-hidden="true" className="mt-0.5 size-4 shrink-0 text-muted-foreground" />
<div>
<p className="font-medium">{item.title}</p>
<p className="mt-1 leading-5 text-muted-foreground">{item.detail}</p>
</div>
</li>
))}
</ul>
<div className="mt-5 flex gap-2 border-t pt-5 text-xs leading-5 text-muted-foreground">
<ShieldCheck aria-hidden="true" className="mt-0.5 size-4 shrink-0" />
<p>
Your infrastructure stores the traces. Investigation content is sent to your selected model provider through
the gateway.
</p>
</div>
</aside>
);
}

80
ui/litellm-dashboard/src/components/lens/onboarding/LensIntroDialog.tsx Normal file
View file

@ -0,0 +1,80 @@
"use client";
import { useId, useState } from "react";
import { Loader2, XIcon } from "lucide-react";
import { Button } from "@/components/ui/button";
import { Checkbox } from "@/components/ui/checkbox";
import { Dialog, DialogContent, DialogDescription, DialogTitle } from "@/components/ui/dialog";
import { Label } from "@/components/ui/label";
import { useStoredValue } from "@/lib/storage";
import { useLensReadiness } from "../hooks/useLensReadiness";
import { LENS_INTRO_DISMISSED, LENS_INTRO_SEEN } from "../storage";
import { LensGettingStarted, type LensGettingStartedProps } from "./LensGettingStarted";
import { useOnboarding } from "./OnboardingContext";
export interface LensIntro {
readonly open: boolean;
close(forever: boolean): void;
}
/** Opens on the first Lens visit of a session, or whenever the URL asks for setup. "Don't show this again" outlives the tab; a plain close only rests for the session. */
export function useLensIntro({ demo, settingUp }: { demo: boolean; settingUp: boolean }): LensIntro {
const [dismissed, setDismissed] = useStoredValue(LENS_INTRO_DISMISSED);
const [seen, setSeen] = useStoredValue(LENS_INTRO_SEEN);
const close = (forever: boolean) => {
setSeen(true);
if (forever) setDismissed(true);
};
const firstVisit = !demo && !dismissed && !seen;
return { open: settingUp || firstVisit, close };
}
export type LensIntroDialogProps = Omit<LensGettingStartedProps, "state"> & {
open: boolean;
onClose: (forever: boolean) => void;
};
export function LensIntroDialog({ open, onClose, ...gettingStarted }: LensIntroDialogProps) {
const [forever, setForever] = useState(false);
const checkboxId = useId();
const close = () => onClose(forever);
return (
<Dialog open={open} onOpenChange={(next) => !next && close()}>
<DialogContent
showCloseButton={false}
className="max-h-[calc(100vh-4rem)] gap-0 overflow-y-auto p-0 sm:max-w-4xl"
>
<div className="sticky top-0 z-raised flex items-center justify-end gap-3 bg-popover/90 px-4 py-2 backdrop-blur-sm">
<Label htmlFor={checkboxId} className="gap-2 text-xs font-normal text-muted-foreground">
<Checkbox id={checkboxId} checked={forever} onCheckedChange={(checked) => setForever(checked === true)} />
Don’t show this again
</Label>
<Button variant="ghost" size="icon-sm" aria-label="Close" onClick={close}>
<XIcon />
</Button>
</div>
<DialogTitle className="sr-only">Get started with Lens</DialogTitle>
<DialogDescription className="sr-only">
What Lens does, and the steps to connect tracing, a worker and your first investigation.
</DialogDescription>
<div className="px-5 pb-5">
<IntroContent {...gettingStarted} />
</div>
</DialogContent>
</Dialog>
);
}
function IntroContent(props: Omit<LensGettingStartedProps, "state">) {
const { canViewInvestigations } = useOnboarding();
const state = useLensReadiness(canViewInvestigations);
if (state.loading)
return (
<p role="status" className="flex items-center gap-2 py-8 text-sm text-muted-foreground">
<Loader2 aria-hidden="true" className="size-4 animate-spin" />
Checking Lens setup…
</p>
);
return <LensGettingStarted state={state} {...props} />;
}

0
ui/litellm-dashboard/src/components/lens/setup/LensIntroduction.module.css → ui/litellm-dashboard/src/components/lens/onboarding/LensIntroduction.module.css
View file

25
ui/litellm-dashboard/src/components/lens/setup/LensIntroduction.tsx → ui/litellm-dashboard/src/components/lens/onboarding/LensIntroduction.tsx
View file

@ -1,6 +1,7 @@
import { useState } from "react";
import { ArrowRight, ArrowUpRight, ChevronDown, ChevronRight } from "lucide-react";
import { Button } from "@/components/ui/button";
import { cn } from "@/lib/cva.config";
import styles from "./LensIntroduction.module.css";
import { GatewayFlow } from "./GatewayFlow";
@ -21,7 +22,7 @@ function TraceExample({ highlighted, onHighlight }: EvidenceProps) {
return (
<section
aria-labelledby="lens-sample-trace"
className={`${styles.sampleCard} min-w-0 rounded-xl border bg-card p-4`}
className={cn(styles.sampleCard, "min-w-0 rounded-xl border bg-card p-4")}
>
<div className="flex flex-wrap items-center justify-between gap-2 text-sm">
<h3 id="lens-sample-trace" className="font-semibold">
@ -36,7 +37,7 @@ function TraceExample({ highlighted, onHighlight }: EvidenceProps) {
"grid w-full grid-cols-[minmax(0,1fr)_minmax(48px,1fr)_2.5rem] items-center gap-2 text-left sm:gap-3";
const content = (
<>
<span className={`flex min-w-0 items-center gap-1 ${index > 0 ? "pl-2" : ""}`}>
<span className={cn("flex min-w-0 items-center gap-1", index > 0 && "pl-2")}>
{index === 0 ? (
<ChevronDown aria-hidden="true" className="size-3 shrink-0" />
) : (
@ -49,7 +50,11 @@ function TraceExample({ highlighted, onHighlight }: EvidenceProps) {
</span>
<span aria-hidden="true" className="relative h-2.5 overflow-hidden rounded-sm bg-muted/70">
<span
className={`absolute inset-y-0 rounded-sm ${linked ? styles.evidenceBar : ""} ${item.failed ? "bg-rose-400" : "bg-indigo-600 dark:bg-indigo-400"}`}
className={cn(
"absolute inset-y-0 rounded-sm",
linked && styles.evidenceBar,
item.failed ? "bg-rose-400" : "bg-indigo-600 dark:bg-indigo-400",
)}
style={{ left: `${(item.start / 14.2) * 100}%`, width: `${(item.duration / 14.2) * 100}%` }}
/>
</span>
@ -61,7 +66,7 @@ function TraceExample({ highlighted, onHighlight }: EvidenceProps) {
{linked ? (
<button
type="button"
className={`${rowClass} ${styles.evidenceLink}`}
className={cn(rowClass, styles.evidenceLink)}
data-evidence-link=""
aria-label={`${item.name}, ${item.duration} seconds${item.failed ? ", failed" : ""}. Highlight related finding`}
aria-controls="lens-linked-finding"
@ -78,11 +83,11 @@ function TraceExample({ highlighted, onHighlight }: EvidenceProps) {
})}
</ol>
<div className="mt-3 grid gap-3 border-t pt-3 sm:grid-cols-2">
<div id="lens-benchmark-output" className={`${styles.evidenceOutput} rounded-lg bg-muted/60 p-2.5`}>
<div id="lens-benchmark-output" className={cn(styles.evidenceOutput, "rounded-lg bg-muted/60 p-2.5")}>
<p className="text-xs text-muted-foreground">run_benchmark · output</p>
<p className="mt-1.5 text-sm leading-4.5">Error: benchmark runner unavailable (503)</p>
</div>
<div id="lens-answer-output" className={`${styles.evidenceOutput} rounded-lg bg-muted/60 p-2.5`}>
<div id="lens-answer-output" className={cn(styles.evidenceOutput, "rounded-lg bg-muted/60 p-2.5")}>
<p className="text-xs text-muted-foreground">answer · output</p>
<p className="mt-1.5 text-sm leading-4.5">“{sampleAnswer}”</p>
</div>
@ -95,7 +100,7 @@ function FindingExamples({ highlighted, onHighlight }: EvidenceProps) {
return (
<section
aria-labelledby="lens-sample-findings"
className={`${styles.sampleCard} min-w-0 rounded-xl border bg-card p-4`}
className={cn(styles.sampleCard, "min-w-0 rounded-xl border bg-card p-4")}
>
<div className="flex flex-wrap items-center justify-between gap-2 text-sm">
<h3 id="lens-sample-findings" className="font-semibold">
@ -104,11 +109,11 @@ function FindingExamples({ highlighted, onHighlight }: EvidenceProps) {
<p className="text-xs text-muted-foreground">500 sample runs reviewed</p>
</div>
<div className="mt-3 divide-y border-t">
<article id="lens-linked-finding" className={`${styles.evidenceFinding} py-3`}>
<article id="lens-linked-finding" className={cn(styles.evidenceFinding, "py-3")}>
<h4 className="text-sm font-semibold">
<button
type="button"
className={`${styles.evidenceLink} flex w-full items-start gap-2 text-left`}
className={cn(styles.evidenceLink, "flex w-full items-start gap-2 text-left")}
data-evidence-link=""
aria-controls="lens-benchmark-output lens-answer-output"
aria-pressed={highlighted}
@ -179,7 +184,7 @@ export function LensIntroduction({ onStart, onDemo }: { onStart: () => void; onD
</div>
<GatewayFlow />
<div
className={`${styles.examples} mt-4 grid items-stretch gap-3 md:grid-cols-2`}
className={cn(styles.examples, "mt-4 grid items-stretch gap-3 md:grid-cols-2")}
data-evidence-active={highlighted}
>
<TraceExample highlighted={highlighted} onHighlight={toggleEvidence} />

24
ui/litellm-dashboard/src/components/lens/onboarding/OnboardingContext.tsx Normal file
View file

@ -0,0 +1,24 @@
"use client";
import { createContext, useContext } from "react";
import type { TraceSummary } from "@/components/view_logs/TraceView/traceTypes";
export interface Onboarding {
readonly readOnly: boolean;
readonly canViewInvestigations: boolean;
readonly canInvestigate: boolean;
readonly canMintTracingKey: boolean;
connect(): void;
create(): void;
openTrace(trace: TraceSummary): void;
}
const OnboardingContext = createContext<Onboarding | null>(null);
export const OnboardingProvider = OnboardingContext.Provider;
export function useOnboarding(): Onboarding {
const onboarding = useContext(OnboardingContext);
if (!onboarding) throw new Error("useOnboarding needs an OnboardingProvider above it");
return onboarding;
}

52
ui/litellm-dashboard/src/components/lens/onboarding/OnboardingSetup.tsx Normal file
View file

@ -0,0 +1,52 @@
"use client";
import { useId, type ComponentProps, type ReactNode } from "react";
import { Button } from "@/components/ui/button";
import { cn } from "@/lib/cva.config";
import type { LensReadiness } from "../hooks/useLensReadiness";
import { useOnboarding } from "./OnboardingContext";
import { OnboardingSteps } from "./OnboardingSteps";
export type OnboardingSetupProps = Omit<ComponentProps<"section">, "children"> & {
state: LensReadiness;
action?: ReactNode;
};
export function OnboardingSetup({ state, action, className, ...props }: OnboardingSetupProps) {
const { readOnly, canInvestigate } = useOnboarding();
const titleId = useId();
return (
<section
data-slot="onboarding-setup"
aria-labelledby={titleId}
className={cn("min-w-0 scroll-mt-4 outline-none", className)}
{...props}
>
<div className="mb-6 flex flex-wrap items-center justify-between gap-3">
<div>
<h2 id={titleId} className="text-2xl font-semibold tracking-tight">
Get Lens running
</h2>
<p className="mt-2 text-sm leading-6 text-muted-foreground">
Each step checks your connection, so you’ll see when it’s working.
</p>
</div>
{action}
</div>
<OnboardingSteps state={state} />
{state.error && (
<div role="alert" className="mt-4 flex flex-wrap items-center gap-3 text-sm text-destructive">
<p>Could not check setup. {state.error}</p>
<Button variant="outline" size="sm" onClick={state.refresh} disabled={state.checking}>
Retry
</Button>
</div>
)}
{(readOnly || !canInvestigate) && (
<p className="mt-4 text-sm text-muted-foreground">
A gateway administrator can connect a worker and run investigations.
</p>
)}
</section>
);
}

251
ui/litellm-dashboard/src/components/lens/onboarding/OnboardingSteps.tsx Normal file
View file

@ -0,0 +1,251 @@
"use client";
import { useId, useRef, useState, type ReactNode } from "react";
import { ArrowRight, ChevronDown } from "lucide-react";
import { Button } from "@/components/ui/button";
import { TracingSetupFields } from "@/components/view_logs/TraceView/TracingSetupCard";
import { cn } from "@/lib/cva.config";
import { useLensAccessToken } from "../data/LensServices";
import type { LensReadiness } from "../hooks/useLensReadiness";
import { initialSetupStep } from "../model/readiness";
import { StepIndicator, type StepState } from "../ui/StepIndicator";
import { useOnboarding } from "./OnboardingContext";
type StepProps = { state: LensReadiness; goTo: (step: number) => void };
function useLocked() {
const { readOnly, canInvestigate } = useOnboarding();
return readOnly || !canInvestigate;
}
function StorageStep({ state, goTo }: StepProps) {
const { readOnly, openTrace } = useOnboarding();
const accessToken = useLensAccessToken();
if (!state.tracingEnabled)
return (
<>
<TracingSetupFields
detail="Tracing is not enabled"
accessToken={accessToken}
onOpenTrace={openTrace}
onCheck={state.refresh}
checking={state.checking}
readOnly={readOnly}
/>
<ActivityContinuation state={state} />
</>
);
return (
<div className="space-y-4">
<p role="status" className="text-sm text-success">
Trace storage is connected
</p>
<Button onClick={() => goTo(1)}>
Continue to your agent <ArrowRight aria-hidden="true" className="size-4" />
</Button>
<ActivityContinuation state={state} />
</div>
);
}
function continuationLabel(state: LensReadiness) {
if (state.connected) return "Continue to investigation";
return state.tracesReady ? "Continue to worker" : "Continue with request logs";
}
function ActivityContinuation({ state }: { state: LensReadiness }) {
const { connect, create } = useOnboarding();
const locked = useLocked();
if (!state.activityReady) return null;
return (
<div className="mt-4">
<Button onClick={state.connected ? create : connect} disabled={locked}>
{continuationLabel(state)}
<ArrowRight aria-hidden="true" className="size-4" />
</Button>
{state.requestsReady && !state.tracesReady && (
<p className="mt-2 text-xs text-muted-foreground">
Request logs are already available. You can investigate them now and add agent traces later.
</p>
)}
</div>
);
}
function AgentStep({ state }: StepProps) {
const { readOnly, canMintTracingKey, openTrace } = useOnboarding();
const accessToken = useLensAccessToken();
if (!state.tracingEnabled)
return (
<>
<p className="text-sm text-muted-foreground">Connect trace storage in step 1 before sending a trace.</p>
<ActivityContinuation state={state} />
</>
);
return (
<>
<div hidden={state.tracesReady}>
<TracingSetupFields
detail={null}
accessToken={accessToken}
onOpenTrace={(value) => {
state.refresh();
openTrace(value);
}}
onCheck={state.refresh}
checking={state.checking}
readOnly={readOnly}
canMintTracingKey={canMintTracingKey}
/>
</div>
{state.tracesReady && (
<p role="status" className="text-sm text-success">
Your first trace is ready. Continue setup so Lens can investigate your agent’s behavior.
</p>
)}
<ActivityContinuation state={state} />
</>
);
}
function WorkerStep({ state }: StepProps) {
const { connect, create } = useOnboarding();
const locked = useLocked();
return (
<div className="space-y-4">
<p role="status" className="text-sm text-muted-foreground">
{state.connected
? "Worker connected. You’re ready to create an investigation."
: "The worker reviews recorded activity using a model on your gateway. You choose its analysis model and spending limit."}
</p>
{!state.activityReady && (
<p className="text-sm text-muted-foreground">Record activity in step 2 before connecting a worker.</p>
)}
<Button onClick={state.connected ? create : connect} disabled={!state.activityReady || locked}>
{state.connected ? "Continue to investigation" : "Connect worker"}
<ArrowRight aria-hidden="true" className="size-4" />
</Button>
</div>
);
}
function InvestigationStep({ state }: StepProps) {
const { create } = useOnboarding();
const locked = useLocked();
return (
<div className="space-y-4">
<p className="text-sm leading-6 text-muted-foreground">
Choose the activity to review and describe how your agent should behave. Lens will show findings with evidence
and suggested changes.
</p>
{!state.ready && (
<p className="text-sm text-muted-foreground">
Recorded activity and a connected worker are required before you can run an investigation.
</p>
)}
<Button onClick={create} disabled={!state.ready || locked}>
New investigation <ArrowRight aria-hidden="true" className="size-4" />
</Button>
</div>
);
}
interface StepDefinition {
readonly title: string;
readonly description: string;
readonly complete: (state: LensReadiness) => boolean;
readonly Content: (props: StepProps) => ReactNode;
}
const STEPS: readonly StepDefinition[] = [
{
title: "Enable tracing on the gateway",
description: "Connect ClickHouse and restart the gateway.",
complete: (state) => state.tracingEnabled,
Content: StorageStep,
},
{
title: "Send your first trace",
description: "Capture your agent’s inputs, outputs, and tool calls.",
complete: (state) => state.tracesReady,
Content: AgentStep,
},
{
title: "Connect a worker",
description: "Choose a model and run the worker on your infrastructure.",
complete: (state) => state.connected,
Content: WorkerStep,
},
{
title: "Run your first investigation",
description: "Describe the expected behavior and review a sample of activity.",
complete: (state) => state.hasInvestigations,
Content: InvestigationStep,
},
];
function stepState(complete: boolean, open: boolean): StepState {
if (complete) return "complete";
return open ? "current" : "upcoming";
}
export function OnboardingSteps({ state, className }: { state: LensReadiness; className?: string }) {
const id = useId();
const listRef = useRef<HTMLOListElement>(null);
const [step, setStep] = useState(() => initialSetupStep(state));
const goTo = (index: number) => {
setStep(index);
listRef.current?.querySelector<HTMLButtonElement>(`[aria-controls="${id}-${index}"]`)?.focus();
};
return (
<ol
ref={listRef}
data-slot="onboarding-steps"
className={cn("divide-y overflow-hidden rounded-2xl border bg-card", className)}
>
{STEPS.map(({ title, description, complete, Content }, index) => {
const open = step === index;
return (
<li key={title}>
<h3>
<button
type="button"
id={`${id}-trigger-${index}`}
aria-expanded={open}
aria-controls={`${id}-${index}`}
onClick={() => setStep(index)}
className="group flex w-full items-start gap-4 p-5 text-left outline-none hover:bg-muted/30 focus-visible:bg-muted/50 sm:p-6"
>
<StepIndicator index={index} state={stepState(complete(state), open)} />
<span className="min-w-0 flex-1">
<span
className={cn(
"block text-sm font-medium sm:text-base",
!open && "text-muted-foreground group-hover:text-foreground",
)}
>
{title}
</span>
<span className="mt-1 block text-sm leading-6 font-normal text-muted-foreground">{description}</span>
</span>
<ChevronDown
aria-hidden="true"
className={cn("mt-1 size-4 shrink-0 text-muted-foreground", open && "rotate-180")}
/>
</button>
</h3>
<div
id={`${id}-${index}`}
role="region"
aria-labelledby={`${id}-trigger-${index}`}
hidden={!open}
className="px-5 pb-6 sm:pr-6 sm:pb-7 sm:pl-17"
>
<Content state={state} goTo={goTo} />
</div>
</li>
);
})}
</ol>
);
}

88
ui/litellm-dashboard/src/components/lens/route.tsx → ui/litellm-dashboard/src/components/lens/route.ts
View file

@ -4,14 +4,14 @@ import { parseAsBoolean, parseAsString, parseAsStringLiteral, useQueryStates } f
import { useCallback } from "react";
import { OPEN_TRACE_PARSERS, RUN_FILTER_PARSERS } from "@/components/view_logs/TraceView/traceRouting";
export const LENS_TABS = { traces: "Traces", investigations: "Investigations" } as const;
export const LENS_TABS = { traces: "Traces", investigations: "Investigations", settings: "Settings" } as const;
export type LensTab = keyof typeof LENS_TABS;
const lensTabs = Object.keys(LENS_TABS) as LensTab[];
export const RESULT_TABS = ["findings", "checks", "runs", "activity"] as const;
const RESULT_TABS = ["findings", "checks", "runs", "activity"] as const;
export type ResultTab = (typeof RESULT_TABS)[number];
export const FINDING_KINDS = ["issue", "pattern"] as const;
export const LENS_DIALOGS = ["new", "edit", "duplicate", "run_now", "workers", "monitoring"] as const;
const FINDING_KINDS = ["issue", "pattern"] as const;
export const LENS_DIALOGS = ["new", "edit", "duplicate", "run_now", "monitoring"] as const;
export type LensDialog = (typeof LENS_DIALOGS)[number];
const LENS_PARSERS = {
@ -25,7 +25,7 @@ const ISSUE_PARSERS = { issue: parseAsString };
const LIST_PARSERS = { search: parseAsString.withDefault("") };
export const RESULT_PARSERS = {
const RESULT_PARSERS = {
run: parseAsString.withDefault("latest"),
section: parseAsStringLiteral(RESULT_TABS).withDefault("findings"),
finding: parseAsString,
@ -69,7 +69,7 @@ export function useLensRoute(): LensRoute {
const [{ tab, lens, demo, setup }, setParams] = useQueryStates(SESSION_PARSERS, { history: "push" });
const setTab = useCallback((next: LensTab) => void setParams({ tab: next }), [setParams]);
const setLensId = useCallback(
(next: string | null) => void setParams({ ...CLEARED_RESULTS, lens: next }),
(next: string | null) => void setParams({ ...CLEARED_RESULTS, lens: next, issue: null }),
[setParams],
);
const setDemo = useCallback(
@ -80,12 +80,15 @@ export function useLensRoute(): LensRoute {
return { tab, lensId: lens, demo, settingUp: setup === "lens", setTab, setLensId, setDemo, setSetup };
}
const ISSUE_ROUTE_PARSERS = { ...ISSUE_PARSERS, lens: LENS_PARSERS.lens, ...RESULT_PARSERS };
/** A peeked finding takes the panel over from any open investigation and starts from its summary. */
export function useIssueRoute() {
const [{ issue }, setParams] = useQueryStates(ISSUE_PARSERS);
const [{ issue }, setParams] = useQueryStates(ISSUE_ROUTE_PARSERS);
return {
issueKey: issue,
setIssueKey: useCallback(
(next: string | null) => void setParams({ issue: next }, { history: "push" }),
(next: string | null) => void setParams({ ...CLEARED_RESULTS, lens: null, issue: next }, { history: "push" }),
[setParams],
),
};
@ -105,3 +108,72 @@ export function useDialogRoute() {
const closeDialog = useCallback(() => void setParams({ dialog: null, target: null }), [setParams]);
return { dialog, target, openDialog, closeDialog };
}
const isResultTab = (tab: string): tab is ResultTab => (RESULT_TABS as readonly string[]).includes(tab);
type FindingKind = (typeof FINDING_KINDS)[number];
export interface EvidenceRef {
readonly id: string;
readonly span: string;
}
const NO_RESULT_OPEN = { finding: null, evidence: null, evidence_span: null } as const;
export function useRunRoute() {
const [{ run }, setParams] = useQueryStates(RESULT_PARSERS);
const selectRun = useCallback((id: string) => void setParams({ run: id, ...NO_RESULT_OPEN }), [setParams]);
const openRun = useCallback(
(id: string) => void setParams({ run: id, section: null, ...NO_RESULT_OPEN }),
[setParams],
);
return { batchId: run, selectRun, openRun };
}
export function useSectionRoute() {
const [{ section }, setParams] = useQueryStates(RESULT_PARSERS);
const setSection = useCallback(
(next: string) => void (isResultTab(next) && setParams({ section: next })),
[setParams],
);
return { section, setSection };
}
/** Picking a finding starts from its summary, so any evidence stacked over the previous one is dropped. */
export function useFindingRoute() {
const [{ finding }, setParams] = useQueryStates(RESULT_PARSERS);
const setFindingId = useCallback(
(next: string | null) =>
void setParams({ finding: next, evidence: null, evidence_span: null }, { history: "push" }),
[setParams],
);
return { findingId: finding, setFindingId };
}
/** A run opened from the Runs tab is evidence on its own, outside any finding. */
export function useRunEvidenceRoute() {
const [{ finding, evidence }, setParams] = useQueryStates(RESULT_PARSERS);
const selectRun = useCallback(
(id: string | null) => void setParams({ finding: null, evidence: id, evidence_span: null }, { history: "push" }),
[setParams],
);
return { runId: finding === null ? evidence : null, selectRun };
}
export function useFindingFilters() {
const [{ kind, finding_status }, setParams] = useQueryStates(RESULT_PARSERS);
const setKind = useCallback((next: FindingKind) => void setParams({ kind: next }), [setParams]);
const setStatus = useCallback((next: string) => void setParams({ finding_status: next }), [setParams]);
return { kind, setKind, status: finding_status, setStatus };
}
export function useEvidenceRoute() {
const [{ evidence, evidence_span }, setParams] = useQueryStates(RESULT_PARSERS);
const setEvidence = useCallback(
(next: EvidenceRef | null) =>
void setParams({ evidence: next?.id ?? null, evidence_span: next?.span || null }, { history: "push" }),
[setParams],
);
const current: EvidenceRef | null = evidence === null ? null : { id: evidence, span: evidence_span ?? "" };
return { evidence: current, setEvidence };
}

140
ui/litellm-dashboard/src/components/lens/runSearch/RunSearch.test.tsx
View file

@ -1,140 +0,0 @@
import { render, screen, waitFor, within } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { useState } from "react";
import { describe, expect, it, vi } from "vitest";
import { runs } from "./__fixtures__/runs";
import { RunSearch } from "./RunSearch";
function Harness({ initial = "" }: { initial?: string }) {
const [value, setValue] = useState(initial);
return (
<>
<RunSearch value={value} onChange={setValue} runs={runs} />
<button type="button" onClick={() => setValue("status:ok")}>
Load saved query
</button>
</>
);
}
/** Delivers each emitted value only when asked, like a throttled URL. */
function LaggingHarness() {
const [value, setValue] = useState("");
const [queue, setQueue] = useState<string[]>([]);
return (
<>
<RunSearch value={value} onChange={(next) => setQueue((q) => [...q, next])} runs={runs} />
<output aria-label="undelivered">{queue.join("|")}</output>
<button
type="button"
onClick={() => {
setValue(queue[0]);
setQueue(queue.slice(1));
}}
>
Deliver next
</button>
</>
);
}
const box = () => screen.getByRole("combobox", { name: "Search runs" });
const undelivered = () => screen.getByRole("status", { name: "undelivered" });
const listbox = () => screen.getByRole("listbox", { name: "Search suggestions" });
describe("RunSearch", () => {
it("builds a filter from the keyboard: field, then value", async () => {
const user = userEvent.setup();
render(<Harness />);
await user.click(box());
expect(box()).toHaveAttribute("aria-expanded", "true");
await user.keyboard("-sta");
expect(
within(listbox())
.getAllByRole("option")
.map((o) => o.textContent),
).toEqual(["status"]);
await user.keyboard("{Enter}");
expect(box()).toHaveTextContent(/^-status:$/, { normalizeWhitespace: false });
expect(within(listbox()).getByRole("group", { name: "status" })).toBeVisible();
expect(within(listbox()).getByText("does not contain")).toBeVisible();
await user.keyboard("{ArrowDown}");
const ok = within(listbox()).getByRole("option", { name: "ok" });
expect(ok).toHaveAttribute("aria-selected", "true");
expect(box()).toHaveAttribute("aria-activedescendant", ok.id);
await user.keyboard("{Enter}");
expect(box()).toHaveTextContent(/^-status:ok $/, { normalizeWhitespace: false });
expect(screen.queryByRole("listbox")).not.toBeInTheDocument();
expect(box()).toHaveAttribute("aria-expanded", "false");
});
it("wraps arrow navigation and picks with Tab", async () => {
const user = userEvent.setup();
render(<Harness />);
await user.click(box());
await user.keyboard("{ArrowUp}");
expect(within(listbox()).getByRole("option", { name: "trace_id" })).toHaveAttribute("aria-selected", "true");
await user.keyboard("{ArrowDown}{ArrowDown}{Tab}");
expect(box()).toHaveTextContent(/^agent:$/, { normalizeWhitespace: false });
await user.keyboard("{ArrowDown}{ArrowDown}e");
expect(within(listbox()).getByRole("option", { name: "billing-agent" })).toHaveAttribute("aria-selected", "true");
});
it("picks a suggestion with the mouse without losing focus", async () => {
const user = userEvent.setup();
render(<Harness />);
await user.click(box());
await user.keyboard("agent:");
await user.click(within(listbox()).getByRole("option", { name: "researcher" }));
expect(box()).toHaveTextContent(/^agent:researcher $/, { normalizeWhitespace: false });
expect(box()).toHaveFocus();
await user.keyboard("refund");
expect(box()).toHaveTextContent(/^agent:researcher refund$/, { normalizeWhitespace: false });
});
it("stays one line: Enter with no menu adds nothing, and Escape closes the menu until ArrowDown", async () => {
const user = userEvent.setup();
render(<Harness />);
await user.click(box());
await user.keyboard("refund{Enter}");
expect(box()).toHaveTextContent(/^refund$/, { normalizeWhitespace: false });
await user.keyboard(" {Escape}");
expect(screen.queryByRole("listbox")).not.toBeInTheDocument();
await user.keyboard("{ArrowDown}");
expect(listbox()).toBeVisible();
});
it("emits the query once typing pauses, not per keystroke", async () => {
const user = userEvent.setup();
const onChange = vi.fn();
render(<RunSearch value="" onChange={onChange} runs={runs} />);
await user.click(box());
await user.keyboard("refund");
await waitFor(() => expect(onChange).toHaveBeenCalledWith("refund"));
expect(onChange).toHaveBeenCalledOnce();
});
it("keeps typing intact while the value it emitted arrives late", async () => {
const user = userEvent.setup();
render(<LaggingHarness />);
await user.click(box());
await user.keyboard("ab");
await waitFor(() => expect(undelivered()).toHaveTextContent(/^ab$/, { normalizeWhitespace: false }));
await user.keyboard("c");
await user.click(screen.getByRole("button", { name: "Deliver next" }));
await user.click(box());
await user.keyboard("d");
expect(box()).toHaveTextContent(/^abcd$/, { normalizeWhitespace: false });
await waitFor(() => expect(undelivered()).toHaveTextContent(/^abcd$/, { normalizeWhitespace: false }));
});
it("shows a query set from outside, such as the URL", async () => {
const user = userEvent.setup();
render(<Harness initial="agent:triage" />);
expect(box()).toHaveTextContent("agent:triage");
await user.click(screen.getByRole("button", { name: "Load saved query" }));
expect(box()).toHaveTextContent(/^status:ok$/, { normalizeWhitespace: false });
});
});

257
ui/litellm-dashboard/src/components/lens/runSearch/RunSearch.tsx
View file

@ -1,257 +0,0 @@
"use client";
import "prosemirror-view/style/prosemirror.css";
import { useDebouncedCallback } from "@tanstack/react-pacer/debouncer";
import { ProseMirror, ProseMirrorDoc, reactKeys, useEditorEventCallback } from "@handlewithcare/react-prosemirror";
import {
Bot,
Box,
Braces,
CircleDashed,
CornerDownLeft,
Hash,
type LucideIcon,
Search,
SquareChevronRight,
} from "lucide-react";
import { Schema } from "prosemirror-model";
import { EditorState, TextSelection, type Transaction } from "prosemirror-state";
import { Decoration, DecorationSet, type EditorView } from "prosemirror-view";
import { useId, useMemo, useState } from "react";
import type { TraceSummary } from "@/components/view_logs/TraceView/traceTypes";
import { parseRunQuery, type RunField } from "./runQuery";
import { suggest, type Suggestion, type SuggestionMenu } from "./suggestions";
const schema = new Schema({ nodes: { doc: { content: "text*" }, text: {} } });
const createState = (text: string) =>
EditorState.create({ schema, doc: schema.node("doc", null, text ? schema.text(text) : []), plugins: [reactKeys()] });
const EMIT_WAIT_MS = 150;
const KEEP_MENU_CLOSED = "runSearchKeepMenuClosed";
const FIELD_ICONS: Record<RunField, LucideIcon> = {
name: SquareChevronRight,
agent: Bot,
status: CircleDashed,
model: Box,
input: Braces,
trace_id: Hash,
};
const OPERATORS = [
{ label: "equals", example: "foo:bar" },
{ label: "not equals", example: "-foo:bar" },
{ label: "wildcard match", example: "foo:*bar*" },
{ label: "does not contain", example: "-foo:*bar*" },
];
function highlight(state: EditorState): DecorationSet {
const marks = parseRunQuery(state.doc.textContent).flatMap((clause) =>
clause.kind === "field"
? [
Decoration.inline(clause.from, clause.keyTo, { class: "text-info" }),
Decoration.inline(clause.keyTo, clause.keyTo + 1, { class: "text-muted-foreground" }),
]
: [],
);
return DecorationSet.create(state.doc, marks);
}
function applySuggestion(view: EditorView, item: Suggestion) {
const tr = view.state.tr.insertText(item.insert, item.from, item.to);
tr.setSelection(TextSelection.create(tr.doc, item.from + item.insert.length));
if (item.completesClause) tr.setMeta(KEEP_MENU_CLOSED, true);
view.dispatch(tr);
}
interface RunSearchProps {
value: string;
onChange: (value: string) => void;
/** Loaded runs, the source of value suggestions. */
runs: readonly TraceSummary[];
}
/** One query box for the runs list: free text plus `key:value` filters, with autocomplete for both. */
export function RunSearch({ value, onChange, runs }: RunSearchProps) {
const listId = useId();
const [state, setState] = useState(() => createState(value));
const [seenValue, setSeenValue] = useState(value);
const [unechoed, setUnechoed] = useState<readonly string[]>([]);
const [focused, setFocused] = useState(false);
const [menuOpen, setMenuOpen] = useState(true);
const [active, setActive] = useState(0);
// `value` can trail the editor (the URL updates are throttled), so an echo of text we emitted is not an edit.
if (value !== seenValue) {
setSeenValue(value);
const echo = unechoed.indexOf(value);
if (echo >= 0) setUnechoed(unechoed.slice(echo + 1));
else if (value !== state.doc.textContent) {
setUnechoed([]);
setState(createState(value));
}
}
const { selection } = state;
const text = state.doc.textContent;
const menu = useMemo(() => (selection.empty ? suggest(text, selection.head, runs) : null), [text, selection, runs]);
const items = menu?.groups.flatMap((group) => group.items) ?? [];
const shownMenu = focused && menuOpen ? menu : null;
const visible = shownMenu !== null;
const activeItem = visible ? items[Math.min(active, items.length - 1)] : undefined;
const emit = useDebouncedCallback(
(text: string) => {
setUnechoed((current) => [...current, text]);
onChange(text);
},
{ wait: EMIT_WAIT_MS },
);
const dispatchTransaction = (tr: Transaction) => {
setState((current) => current.apply(tr));
if (tr.selectionSet || tr.docChanged) setActive(0);
if (!tr.docChanged) return;
setMenuOpen(!tr.getMeta(KEEP_MENU_CLOSED));
emit(tr.doc.textContent);
};
const handleKeyDown = (view: EditorView, event: KeyboardEvent): boolean => {
if (event.key === "Escape" && visible) {
setMenuOpen(false);
return true;
}
if (event.key === "ArrowDown" && !visible) {
setMenuOpen(true);
return true;
}
if ((event.key === "ArrowDown" || event.key === "ArrowUp") && items.length > 0) {
const step = event.key === "ArrowDown" ? 1 : -1;
setActive((Math.min(active, items.length - 1) + step + items.length) % items.length);
return true;
}
if ((event.key === "Enter" || event.key === "Tab") && activeItem) {
applySuggestion(view, activeItem);
return true;
}
return false;
};
return (
<ProseMirror
state={state}
dispatchTransaction={dispatchTransaction}
decorations={highlight}
handleKeyDown={handleKeyDown}
handleDOMEvents={{
focus: () => {
setFocused(true);
setMenuOpen(true);
return false;
},
blur: () => {
setFocused(false);
return false;
},
}}
transformPastedText={(pasted) => pasted.replace(/\s+/g, " ")}
attributes={{
role: "combobox",
"aria-label": "Search runs",
"aria-autocomplete": "list",
"aria-expanded": String(visible),
"aria-controls": listId,
...(activeItem && { "aria-activedescendant": `${listId}-${activeItem.id}` }),
spellcheck: "false",
class: "min-w-0 flex-1 overflow-x-auto py-1.5 font-mono text-xs whitespace-pre outline-none",
}}
>
<div className="relative w-full flex-1">
<div className="flex h-8 items-center gap-2 rounded-md border border-input bg-transparent px-2.5 focus-within:border-ring focus-within:ring-[3px] focus-within:ring-ring/50 dark:bg-input/30">
<Search className="size-3.5 shrink-0 text-muted-foreground" />
<div className="relative flex min-w-0 flex-1">
<ProseMirrorDoc />
{!text && (
<span className="pointer-events-none absolute inset-y-0 left-0 flex items-center font-mono text-xs text-muted-foreground">
Search runs, or filter like agent:researcher status:error
</span>
)}
</div>
</div>
{shownMenu && <SuggestionPanel id={listId} menu={shownMenu} activeId={activeItem?.id} />}
</div>
</ProseMirror>
);
}
function SuggestionPanel({ id, menu, activeId }: { id: string; menu: SuggestionMenu; activeId: string | undefined }) {
const pick = useEditorEventCallback((view, item: Suggestion) => applySuggestion(view, item));
return (
<div className="absolute top-full left-0 z-floating mt-1 w-full max-w-xl overflow-hidden rounded-md border border-border bg-popover text-popover-foreground shadow-md">
<div id={id} role="listbox" aria-label="Search suggestions" className="max-h-80 overflow-y-auto p-1">
{menu.groups.map((group) => (
<div key={group.heading} role="group" aria-label={group.heading}>
<div className="px-2 pt-2 pb-1 text-xs text-muted-foreground">{group.heading}</div>
{group.items.map((item) => {
const Icon = FIELD_ICONS[item.field];
return (
<div
key={item.id}
id={`${id}-${item.id}`}
role="option"
aria-selected={item.id === activeId}
onMouseDown={(event) => event.preventDefault()}
onClick={() => pick(item)}
className="flex cursor-pointer items-center gap-2 rounded-sm px-2 py-1.5 font-mono text-xs aria-selected:bg-accent aria-selected:text-accent-foreground"
>
<Icon className="size-3.5 shrink-0 text-muted-foreground" />
<span className="truncate">{item.label}</span>
</div>
);
})}
</div>
))}
{menu.showOperators && <OperatorHints />}
</div>
<div className="flex items-center gap-3 border-t border-border px-3 py-2 text-xs text-muted-foreground">
<span className="flex items-center gap-1">
<Kbd>↑</Kbd>
<Kbd>↓</Kbd>
Navigate
</span>
<span className="flex items-center gap-1">
<Kbd>
<CornerDownLeft className="size-3" />
</Kbd>
Select
</span>
</div>
</div>
);
}
function OperatorHints() {
return (
<div className="pb-1">
<div className="px-2 pt-2 pb-1 text-xs text-muted-foreground">Comparison operators</div>
{OPERATORS.map((op) => (
<div key={op.label} className="flex items-center justify-between px-2 py-1 font-mono text-xs">
<span className="text-warning">{op.label}</span>
<code className="rounded bg-muted px-1.5 text-muted-foreground">{op.example}</code>
</div>
))}
</div>
);
}
function Kbd({ children }: { children: React.ReactNode }) {
return (
<kbd className="inline-flex h-5 min-w-5 items-center justify-center rounded border border-border px-1 font-sans">
{children}
</kbd>
);
}

96
ui/litellm-dashboard/src/components/lens/runSearch/runQuery.test.ts
View file

@ -1,96 +0,0 @@
import { describe, expect, it } from "vitest";
import { run, runs } from "./__fixtures__/runs";
import { fieldValues, filterRuns } from "./runQuery";
const ids = (query: string) => filterRuns(runs, query).map((r) => r.trace_id);
describe("filterRuns", () => {
it("returns every run for an empty or blank query", () => {
expect(filterRuns(runs, " ")).toBe(runs);
});
it("matches free text against input, name and trace id, ignoring case, with every term required", () => {
expect(ids("REFUND")).toEqual(["aaa111"]);
expect(ids("lead")).toEqual(["bbb222"]);
expect(ids("ccc3")).toEqual(["ccc333"]);
expect(ids("vector research")).toEqual(["bbb222"]);
expect(ids("vector refund")).toEqual([]);
});
it("keeps a quoted phrase together", () => {
expect(ids('"my refund"')).toEqual(["aaa111"]);
expect(ids('"refund my"')).toEqual([]);
});
it("splits runs by status, judged by recorded errors", () => {
expect(ids("status:error")).toEqual(["bbb222"]);
expect(ids("-status:error")).toEqual(["aaa111", "ccc333"]);
expect(ids("status:OK")).toEqual(["aaa111", "ccc333"]);
});
it("matches a multi-valued field when any value matches, and negation when none does", () => {
expect(ids("agent:triage")).toEqual(["aaa111"]);
expect(ids("-agent:triage")).toEqual(["bbb222", "ccc333"]);
expect(ids("agent:cron")).toEqual(["ccc333"]);
expect(ids("model:gpt-5")).toEqual(["aaa111"]);
});
it("requires a whole-value match unless the value has wildcards", () => {
expect(ids("agent:billing")).toEqual([]);
expect(ids("agent:*bill*")).toEqual(["aaa111"]);
expect(ids("agent:billing*")).toEqual(["aaa111"]);
expect(ids("agent:*agent")).toEqual(["aaa111"]);
expect(ids("-agent:*search*")).toEqual(["aaa111", "ccc333"]);
expect(ids("name:res.arch_lead")).toEqual([]);
});
it("matches glob segments case-insensitively without overlapping exact anchors", () => {
const billing = run({ trace_id: "billing", agent_names: ["BILLING-agent"] });
const research = run({ trace_id: "research", agent_names: ["research-billing-agent"] });
const doubled = run({ trace_id: "doubled", agent_names: ["agent"] });
const globRuns = [billing, research, doubled];
expect(filterRuns(globRuns, "agent:*bill*")).toEqual([billing, research]);
expect(filterRuns(globRuns, "agent:billing*")).toEqual([billing]);
expect(filterRuns(globRuns, "agent:*agent")).toEqual(globRuns);
expect(filterRuns([run({ name: "aa" })], "name:a*a")).toHaveLength(1);
expect(filterRuns([run({ name: "a" })], "name:a*a")).toHaveLength(0);
expect(filterRuns(globRuns, "agent:**")).toEqual(filterRuns(globRuns, "agent:*"));
});
it("matches adversarial globs without catastrophic backtracking", () => {
const longValue = run({ agent_names: ["a".repeat(5000)] });
const pattern = `agent:${"a*".repeat(30)}b`;
const startedAt = performance.now();
const matches = filterRuns([longValue], pattern);
const elapsed = performance.now() - startedAt;
expect(matches).toEqual([]);
expect(elapsed).toBeLessThan(100);
});
it("searches the readable input text and quoted values with spaces", () => {
expect(ids('input:"*vector stores"')).toEqual(["bbb222"]);
expect(ids("trace_id:bbb222")).toEqual(["bbb222"]);
});
it("combines field clauses with free text", () => {
expect(ids("agent:*e* -status:error refund")).toEqual(["aaa111"]);
});
it("ignores a key still waiting for its value and treats unknown keys as text", () => {
expect(ids("agent:")).toEqual(["aaa111", "bbb222", "ccc333"]);
expect(ids("refund agent:")).toEqual(["aaa111"]);
expect(ids("foo:bar")).toEqual([]);
expect(filterRuns([run({ name: "foo:bar" })], "foo:bar")).toHaveLength(1);
});
});
describe("fieldValues", () => {
it("lists each distinct non-empty value once, sorted", () => {
expect(fieldValues(runs, "agent")).toEqual(["billing-agent", "cron", "researcher", "triage"]);
expect(fieldValues(runs, "status")).toEqual(["error", "ok"]);
expect(fieldValues([run({ models: [] })], "model")).toEqual([]);
});
});

111
ui/litellm-dashboard/src/components/lens/runSearch/runQuery.ts
View file

@ -1,111 +0,0 @@
import type { TraceSummary } from "@/components/view_logs/TraceView/traceTypes";
import { previewText, traceAgentNames } from "@/components/view_logs/TraceView/traceUtils";
type RunFieldGroup = "Run attributes" | "Content" | "Identity";
interface RunFieldSpec {
group: RunFieldGroup;
read: (run: TraceSummary) => readonly string[];
suggestValues: boolean;
}
const runStatus = (run: TraceSummary): string => (run.error_count > 0 ? "error" : "ok");
export const RUN_FIELDS = {
name: { group: "Run attributes", read: (run) => [run.name], suggestValues: true },
agent: { group: "Run attributes", read: traceAgentNames, suggestValues: true },
status: { group: "Run attributes", read: (run) => [runStatus(run)], suggestValues: true },
model: { group: "Run attributes", read: (run) => run.models, suggestValues: true },
input: { group: "Content", read: (run) => [previewText(run.input_preview)], suggestValues: false },
trace_id: { group: "Identity", read: (run) => [run.trace_id], suggestValues: false },
} as const satisfies Record<string, RunFieldSpec>;
export type RunField = keyof typeof RUN_FIELDS;
const isRunField = (key: string): key is RunField => Object.hasOwn(RUN_FIELDS, key);
interface Span {
from: number;
to: number;
}
export type QueryClause =
| (Span & { kind: "text"; value: string })
| (Span & { kind: "field"; field: RunField; negated: boolean; keyTo: number; value: string });
const unquote = (raw: string): string => raw.replace(/^"([^"]*)"?$/, "$1");
/** Whitespace-separated tokens; a double-quoted stretch keeps its spaces. */
function tokenize(text: string): (Span & { raw: string })[] {
return Array.from(text.matchAll(/(?:"[^"]*"?|\S)+/g), (match) => ({
raw: match[0],
from: match.index,
to: match.index + match[0].length,
}));
}
function parseToken({ raw, from, to }: Span & { raw: string }): QueryClause {
const match = /^(-?)([A-Za-z_]+):(.*)$/.exec(raw);
const key = match?.[2].toLowerCase() ?? "";
if (!match || !isRunField(key)) return { kind: "text", value: unquote(raw), from, to };
const negated = match[1] === "-";
return {
kind: "field",
field: key,
negated,
keyTo: from + match[1].length + match[2].length,
value: unquote(match[3]),
from,
to,
};
}
export const parseRunQuery = (text: string): QueryClause[] => tokenize(text).map(parseToken);
/** `bar` matches the whole value, `*bar*` is a glob; both ignore case. */
function valueMatcher(pattern: string): (value: string) => boolean {
const needle = pattern.toLowerCase();
if (!needle.includes("*")) return (value) => value.toLowerCase() === needle;
const segments = needle.split("*");
const first = segments[0];
const last = segments.at(-1);
const middle = segments.slice(1, -1).filter(Boolean);
return (value) => {
const candidate = value.toLowerCase();
if (first && !candidate.startsWith(first)) return false;
if (last && !candidate.endsWith(last)) return false;
const suffixStart = last ? candidate.length - last.length : candidate.length;
let cursor = first ? first.length : 0;
if (suffixStart < cursor) return false;
for (const segment of middle) {
const found = candidate.indexOf(segment, cursor);
if (found < 0 || found + segment.length > suffixStart) return false;
cursor = found + segment.length;
}
return true;
};
}
function matchesClause(run: TraceSummary, clause: QueryClause): boolean {
if (clause.kind === "text") {
const needle = clause.value.toLowerCase();
return [run.trace_id, previewText(run.input_preview), run.name].some((text) => text.toLowerCase().includes(needle));
}
// A key typed without a value yet narrows nothing, so the list does not blank out mid-typing.
if (!clause.value) return true;
const matches = RUN_FIELDS[clause.field].read(run).some(valueMatcher(clause.value));
return clause.negated ? !matches : matches;
}
/** Every clause must hold: free text searches input, name and trace id; `key:value` clauses target one field. */
export function filterRuns(runs: TraceSummary[], query: string): TraceSummary[] {
const clauses = parseRunQuery(query);
if (clauses.length === 0) return runs;
return runs.filter((run) => clauses.every((clause) => matchesClause(run, clause)));
}
export function fieldValues(runs: readonly TraceSummary[], field: RunField): string[] {
return Array.from(new Set(runs.flatMap((run) => RUN_FIELDS[field].read(run))))
.filter(Boolean)
.sort();
}

75
ui/litellm-dashboard/src/components/lens/runSearch/suggestions.test.ts
View file

@ -1,75 +0,0 @@
import { describe, expect, it } from "vitest";
import { run, runs } from "./__fixtures__/runs";
import { suggest, type Suggestion, type SuggestionMenu } from "./suggestions";
const atEnd = (text: string, source = runs) => suggest(text, text.length, source);
const items = (menu: SuggestionMenu | null): Suggestion[] => menu?.groups.flatMap((g) => g.items) ?? [];
const labels = (menu: SuggestionMenu | null) => items(menu).map((item) => item.label);
const apply = (text: string, item: Suggestion) => text.slice(0, item.from) + item.insert + text.slice(item.to);
describe("suggest", () => {
it("offers every field, grouped, in an empty box", () => {
const menu = atEnd("");
expect(menu?.groups.map((g) => g.heading)).toEqual(["Run attributes", "Content", "Identity"]);
expect(labels(menu)).toEqual(["name", "agent", "status", "model", "input", "trace_id"]);
expect(menu?.showOperators).toBe(false);
});
it("narrows fields by the typed prefix and completes the key with a colon", () => {
const menu = atEnd("refund AG");
expect(labels(menu)).toEqual(["agent"]);
expect(apply("refund AG", items(menu)[0])).toBe("refund agent:");
expect(items(menu)[0].completesClause).toBe(false);
});
it("keeps a leading minus when completing a negated key", () => {
expect(apply("-st", items(atEnd("-st"))[0])).toBe("-status:");
});
it("offers nothing for free text that names no field", () => {
expect(atEnd("refund")).toBeNull();
expect(atEnd("foo:bar")).toBeNull();
});
it("offers the loaded values and the operator help once a key has its colon", () => {
const menu = atEnd("agent:");
expect(menu?.groups.map((g) => g.heading)).toEqual(["agent"]);
expect(labels(menu)).toEqual(["billing-agent", "cron", "researcher", "triage"]);
expect(menu?.showOperators).toBe(true);
});
it("narrows values by substring, ignoring wildcards, and drops operator help once a value is typed", () => {
const menu = atEnd("-agent:*AR*");
expect(labels(menu)).toEqual(["researcher"]);
expect(menu?.showOperators).toBe(false);
expect(atEnd("agent:zzz")).toBeNull();
});
it("replaces only the value, adds a separating space at the end, and finishes the clause", () => {
const item = items(atEnd("-agent:bil"))[0];
expect(apply("-agent:bil", item)).toBe("-agent:billing-agent ");
expect(item.completesClause).toBe(true);
const text = "agent:tri refund";
expect(apply(text, items(suggest(text, 9, runs))[0])).toBe("agent:triage refund");
});
it("quotes a value that contains spaces", () => {
const spaced = [run({ name: "Order lookup" })];
expect(apply("name:", items(atEnd("name:", spaced))[0])).toBe('name:"Order lookup" ');
});
it("does not list values for free-form fields but still explains the operators", () => {
const menu = atEnd("input:");
expect(menu?.groups).toEqual([]);
expect(menu?.showOperators).toBe(true);
expect(atEnd("input:x")).toBeNull();
});
it("offers fields in the gap between tokens, but nothing mid-token", () => {
const text = "refund status:ok";
expect(apply(text, items(suggest(text, 7, runs))[1])).toBe("refund agent: status:ok");
expect(suggest(text, 3, runs)).toBeNull();
expect(suggest("status:ok", 0, runs)).toBeNull();
});
});

91
ui/litellm-dashboard/src/components/lens/runSearch/suggestions.ts
View file

@ -1,91 +0,0 @@
import type { TraceSummary } from "@/components/view_logs/TraceView/traceTypes";
import { fieldValues, parseRunQuery, RUN_FIELDS, type QueryClause, type RunField } from "./runQuery";
export interface Suggestion {
id: string;
label: string;
field: RunField;
/** Replaces `text.slice(from, to)`. */
from: number;
to: number;
insert: string;
/** A value finishes the clause and closes the menu; a field keeps it open for its values. */
completesClause: boolean;
}
export interface SuggestionGroup {
heading: string;
items: Suggestion[];
}
export interface SuggestionMenu {
groups: SuggestionGroup[];
showOperators: boolean;
}
const MAX_VALUES = 50;
const FIELDS = Object.keys(RUN_FIELDS) as RunField[];
const isSpace = (char: string | undefined) => char === undefined || /\s/.test(char);
const quoteIfNeeded = (value: string) => (/[\s"]/.test(value) ? `"${value.replaceAll('"', "")}"` : value);
function fieldMenu(prefix: string, negated: boolean, from: number, to: number): SuggestionMenu | null {
const fields = FIELDS.filter((field) => field.startsWith(prefix.toLowerCase()));
if (fields.length === 0) return null;
const groups = Array.from(new Set(fields.map((field) => RUN_FIELDS[field].group)), (heading) => ({
heading,
items: fields
.filter((field) => RUN_FIELDS[field].group === heading)
.map((field) => ({
id: `field:${field}`,
label: field,
field,
from,
to,
insert: `${negated ? "-" : ""}${field}:`,
completesClause: false,
})),
}));
return { groups, showOperators: false };
}
function valueMenu(
clause: Extract<QueryClause, { kind: "field" }>,
text: string,
runs: readonly TraceSummary[],
): SuggestionMenu | null {
const from = clause.keyTo + 1;
const needle = clause.value.replaceAll("*", "").toLowerCase();
const trailing = clause.to === text.length ? " " : "";
const values = RUN_FIELDS[clause.field].suggestValues ? fieldValues(runs, clause.field) : [];
const items = values
.filter((value) => value.toLowerCase().includes(needle))
.slice(0, MAX_VALUES)
.map((value) => ({
id: `value:${value}`,
label: value,
field: clause.field,
from,
to: clause.to,
insert: quoteIfNeeded(value) + trailing,
completesClause: true,
}));
const showOperators = clause.value === "";
if (items.length === 0 && !showOperators) return null;
return { groups: items.length ? [{ heading: clause.field, items }] : [], showOperators };
}
/** What to offer at the cursor: fields while a key is being typed, values once it has a colon. */
export function suggest(text: string, cursor: number, runs: readonly TraceSummary[]): SuggestionMenu | null {
const clause = parseRunQuery(text).find((c) => c.from < cursor && c.to === cursor);
if (!clause) {
const betweenTokens = isSpace(text[cursor - 1]) && isSpace(text[cursor]);
return betweenTokens ? fieldMenu("", false, cursor, cursor) : null;
}
if (clause.kind === "field") return valueMenu(clause, text, runs);
const raw = text.slice(clause.from, clause.to);
const negated = raw.startsWith("-");
const prefix = negated ? raw.slice(1) : raw;
if (/[:"]/.test(prefix)) return null;
return fieldMenu(prefix, negated, clause.from, clause.to);
}

20
ui/litellm-dashboard/src/components/lens/services.ts
View file

@ -1,20 +0,0 @@
"use client";
import { createContext, useContext, useMemo } from "react";
import { apiClient } from "@/components/networking";
import type { TracesApi } from "@/components/view_logs/TraceView/tracesApi";
import { liveLensApi, type LensApi } from "./api/service";
export interface LensServices {
readonly lens: LensApi;
readonly traces: TracesApi;
}
export const LensApiContext = createContext<LensApi | null>(null);
/** Without a provider, falls back to the live HTTP implementation for the caller's token. */
export function useLensApi(accessToken: string): LensApi {
const provided = useContext(LensApiContext);
const live = useMemo(() => liveLensApi(apiClient, accessToken), [accessToken]);
return provided ?? live;
}

84
ui/litellm-dashboard/src/components/lens/settings/LensSettings.tsx Normal file
View file

@ -0,0 +1,84 @@
"use client";
import { useId, type ReactNode } from "react";
import { Activity, ArrowUpRight } from "lucide-react";
import { Button } from "@/components/ui/button";
import { Label } from "@/components/ui/label";
import { Switch } from "@/components/ui/switch";
import { StatusDot } from "@/components/shared/StatusDot";
import { useStoredValue } from "@/lib/storage";
import { LENS_INTRO_DISMISSED } from "../storage";
import { WorkerSettings } from "./worker/WorkerSettings";
import { SettingsCard, SettingsSection } from "./SettingsSection";
import type { LensList } from "../model/types";
const TRACING_DOCS = "https://docs.litellm.ai/docs/proxy/lens";
function TracingSection({ enabled, onOpenTraces }: { enabled: boolean; onOpenTraces: () => void }) {
return (
<SettingsSection heading="Tracing" description="Where your agents send runs so Lens can read them.">
<SettingsCard className="flex flex-wrap items-center justify-between gap-3">
<div className="flex items-center gap-3 text-sm">
<Activity aria-hidden="true" className="size-4 text-muted-foreground" />
<span role="status" className="inline-flex items-center gap-2">
<StatusDot state={enabled ? "ok" : "off"} />
{enabled ? "Tracing enabled" : "Tracing is not enabled"}
</span>
</div>
<div className="flex items-center gap-3">
<a
href={TRACING_DOCS}
target="_blank"
rel="noopener noreferrer"
className="inline-flex items-center gap-1 text-xs text-muted-foreground underline-offset-4 hover:text-foreground hover:underline"
>
Docs
<ArrowUpRight aria-hidden="true" className="size-3" />
</a>
<Button variant="outline" size="sm" onClick={onOpenTraces}>
{enabled ? "Connect an agent" : "Enable tracing"}
</Button>
</div>
</SettingsCard>
</SettingsSection>
);
}
function IntroductionSection() {
const [dismissed, setDismissed] = useStoredValue(LENS_INTRO_DISMISSED);
const id = useId();
return (
<SettingsSection heading="Introduction" description="The getting started dialog shown when you open Lens.">
<SettingsCard className="flex items-center justify-between gap-3">
<Label htmlFor={id} className="text-sm font-normal">
Show the introduction on each new session
</Label>
<Switch id={id} checked={!dismissed} onCheckedChange={(show) => setDismissed(!show)} />
</SettingsCard>
</SettingsSection>
);
}
export function LensSettings({
list,
workerReadyAction,
onOpenTraces,
}: {
list: LensList;
/** Replaces the worker install card's Done button once the new worker connects. */
workerReadyAction?: ReactNode;
onOpenTraces: () => void;
}) {
return (
<div aria-label="Settings" role="region" className="flex w-full flex-col divide-y divide-border">
<TracingSection enabled={list.tracing_enabled} onOpenTraces={onOpenTraces} />
<SettingsSection
heading="Analysis worker"
description="Runs investigations on your server and bills model usage to an analysis key."
>
<WorkerSettings workers={list.workers} readyAction={workerReadyAction} />
</SettingsSection>
<IntroductionSection />
</div>
);
}

33
ui/litellm-dashboard/src/components/lens/settings/SettingsSection.tsx Normal file
View file

@ -0,0 +1,33 @@
import type { ComponentProps } from "react";
import { cn } from "@/lib/cva.config";
export type SettingsSectionProps = ComponentProps<"section"> & { heading: string; description: string };
export function SettingsSection({ heading, description, className, children, ...props }: SettingsSectionProps) {
return (
<section
data-slot="settings-section"
aria-label={heading}
{...props}
className={cn("grid gap-4 py-6 first:pt-0 last:pb-0 md:grid-cols-[200px_minmax(0,1fr)] md:gap-8", className)}
>
<div className="space-y-1">
<h2 className="text-sm font-semibold">{heading}</h2>
<p className="text-xs leading-5 text-muted-foreground">{description}</p>
</div>
<div className="min-w-0 space-y-3">{children}</div>
</section>
);
}
export type SettingsCardProps = ComponentProps<"div">;
export function SettingsCard({ className, ...props }: SettingsCardProps) {
return (
<div
data-slot="settings-card"
{...props}
className={cn("rounded-lg border border-border bg-card p-4", className)}
/>
);
}

8
ui/litellm-dashboard/src/components/lens/setup/worker/AnalysisAccessFields.tsx → ui/litellm-dashboard/src/components/lens/settings/worker/AnalysisAccessFields.tsx
View file

@ -1,21 +1,21 @@
"use client";
import { Controller, useFormContext } from "react-hook-form";
import { lensQueries } from "../../api/queries";
import { lensQueries } from "../../data/queries";
import { useQuery } from "@tanstack/react-query";
import { useLensApi } from "../../services";
import { useLensApi } from "../../data/LensServices";
import { SearchSelect } from "@/components/shared/SearchSelect";
import { Input } from "@/components/ui/input";
import type { WorkerFormInput } from "./workerSchema";
export function AnalysisAccessFields({ accessToken }: { accessToken: string }) {
export function AnalysisAccessFields() {
const {
control,
register,
formState: { errors },
} = useFormContext<WorkerFormInput>();
const api = useLensApi(accessToken);
const api = useLensApi();
const models = useQuery(lensQueries.models(api));
return (
<div className="space-y-5">

57
ui/litellm-dashboard/src/components/lens/setup/worker/AnalysisKeyDetails.tsx → ui/litellm-dashboard/src/components/lens/settings/worker/AnalysisKeyDetails.tsx
View file

@ -1,11 +1,8 @@
"use client";
import { analysisKeyInfoQuery } from "../../api/queries";
import { useQuery } from "@tanstack/react-query";
import { useLensApi } from "../../services";
import { Button } from "@/components/ui/button";
import { runTime } from "../../model/format";
import { useAnalysisKeyInfo } from "./useAnalysisKeyInfo";
function budgetLabel(amount: number | null, duration?: string | null): string {
if (amount === null) return "No key budget";
@ -25,21 +22,39 @@ function budgetLabel(amount: number | null, duration?: string | null): string {
return duration ? `${dollars} / ${periods[duration] ?? duration}` : `${dollars} total`;
}
export function useAnalysisKeyInfo(accessToken: string, keyId?: string) {
const api = useLensApi(accessToken);
return useQuery(analysisKeyInfoQuery(api, keyId));
function modelsLabel(models: readonly string[]): string {
return models.length ? models.join(", ") : "All models";
}
export function AnalysisKeyDetails({
accessToken,
keyId,
showName = false,
}: {
accessToken: string;
keyId: string;
showName?: boolean;
}) {
const key = useAnalysisKeyInfo(accessToken, keyId);
export function AnalysisKeySummary({ keyId }: { keyId: string }) {
const key = useAnalysisKeyInfo(keyId);
if (key.isLoading) return <p className="text-xs text-muted-foreground">Loading billing key…</p>;
if (key.error || !key.data)
return (
<p role="alert" className="flex items-center gap-2 text-xs text-destructive">
Could not load billing key
<Button variant="link" size="xs" className="h-auto p-0" onClick={() => void key.refetch()}>
Retry
</Button>
</p>
);
const info = key.data;
const summary = [
`Bills to ${info.key_alias || "an assigned virtual key"}`,
modelsLabel(info.models),
budgetLabel(info.max_budget, info.budget_duration),
].join(" · ");
const inactive = info.status && info.status !== "active";
return (
<p className="truncate text-xs text-muted-foreground">
{summary}
{inactive && <span className="text-destructive"> · Key {info.status}</span>}
</p>
);
}
export function AnalysisKeyDetails({ keyId }: { keyId: string }) {
const key = useAnalysisKeyInfo(keyId);
if (key.isLoading) return <p className="text-xs text-muted-foreground">Loading key permissions…</p>;
if (key.error || !key.data)
return (
@ -54,14 +69,8 @@ export function AnalysisKeyDetails({
return (
<div className="space-y-2 text-sm">
<dl className="grid grid-cols-[auto_minmax(0,1fr)] gap-x-5 gap-y-2">
{showName && (
<>
<dt className="text-muted-foreground">Billing key</dt>
<dd className="break-words">{info.key_alias || "Assigned virtual key"}</dd>
</>
)}
<dt className="text-muted-foreground">Models</dt>
<dd className="break-words">{info.models.length ? info.models.join(", ") : "All models"}</dd>
<dd className="break-words">{modelsLabel(info.models)}</dd>
<dt className="text-muted-foreground">Key limit</dt>
<dd>{budgetLabel(info.max_budget, info.budget_duration)}</dd>
</dl>

7
ui/litellm-dashboard/src/components/lens/setup/worker/AnalysisKeyPicker.integration.test.tsx → ui/litellm-dashboard/src/components/lens/settings/worker/AnalysisKeyPicker.integration.test.tsx
View file

@ -2,7 +2,8 @@ import { screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { FormProvider, useForm } from "react-hook-form";
import { renderWithProviders, testQueryClient } from "@/../tests/test-utils";
import { testQueryClient } from "@/../tests/test-utils";
import { renderWithLens } from "@/../tests/lens-test-utils";
import { apiClient } from "@/components/networking";
import { AnalysisKeyPicker } from "./AnalysisKeyPicker";
import type { WorkerFormInput } from "./workerSchema";
@ -20,7 +21,7 @@ function AnalysisKeyPickerForm() {
});
return (
<FormProvider {...form}>
<AnalysisKeyPicker accessToken="test" />
<AnalysisKeyPicker />
</FormProvider>
);
}
@ -44,7 +45,7 @@ describe("Lens billing key", () => {
total_pages: 2,
},
);
renderWithProviders(<AnalysisKeyPickerForm />);
renderWithLens(<AnalysisKeyPickerForm />);
await user.click(screen.getByRole("combobox", { name: "Charge analysis to" }));
await user.click(await screen.findByRole("option", { name: "Load more keys" }));
await user.click(await screen.findByRole("option", { name: "Second page" }));

10
ui/litellm-dashboard/src/components/lens/setup/worker/AnalysisKeyPicker.tsx → ui/litellm-dashboard/src/components/lens/settings/worker/AnalysisKeyPicker.tsx
View file

@ -1,11 +1,11 @@
"use client";
import { analysisKeysQuery, type Key } from "../../api/queries";
import { analysisKeysQuery, type Key } from "../../data/queries";
import { Controller, useFormContext, useWatch } from "react-hook-form";
import { useState } from "react";
import { useInfiniteQuery } from "@tanstack/react-query";
import { useLensApi } from "../../services";
import { useLensApi } from "../../data/LensServices";
import { AnalysisKeyDetails } from "./AnalysisKeyDetails";
import type { WorkerFormInput } from "./workerSchema";
import {
@ -17,10 +17,10 @@ import {
ComboboxList,
} from "@/components/ui/combobox";
export function AnalysisKeyPicker({ accessToken }: { accessToken: string }) {
export function AnalysisKeyPicker() {
const { control } = useFormContext<WorkerFormInput>();
const value = useWatch({ control, name: "analysisKey" });
const api = useLensApi(accessToken);
const api = useLensApi();
const [query, setQuery] = useState("");
const [selected, setSelected] = useState<Key | null>(value ? { token: value } : null);
@ -82,7 +82,7 @@ export function AnalysisKeyPicker({ accessToken }: { accessToken: string }) {
/>
</div>
</div>
{choice && <AnalysisKeyDetails accessToken={accessToken} keyId={choice.token} />}
{choice && <AnalysisKeyDetails keyId={choice.token} />}
{keyPages.error && (
<p role="alert" className="text-sm text-destructive">
{keyPages.error.message}

8
ui/litellm-dashboard/src/components/lens/setup/worker/WorkerForm.tsx → ui/litellm-dashboard/src/components/lens/settings/worker/WorkerForm.tsx
View file

@ -8,7 +8,7 @@ import { AnalysisKeyPicker } from "./AnalysisKeyPicker";
import { AnalysisAccessFields } from "./AnalysisAccessFields";
import type { WorkerFormInput } from "./workerSchema";
export function WorkerForm({ accessToken, editingWorker }: { accessToken: string; editingWorker: string | null }) {
export function WorkerForm({ editingWorker }: { editingWorker: string | null }) {
const {
control,
register,
@ -17,11 +17,7 @@ export function WorkerForm({ accessToken, editingWorker }: { accessToken: string
const useExisting = useWatch({ control, name: "useExisting" });
return (
<div className="min-w-0 space-y-5">
{useExisting ? (
<AnalysisKeyPicker accessToken={accessToken} />
) : (
<AnalysisAccessFields accessToken={accessToken} />
)}
{useExisting ? <AnalysisKeyPicker /> : <AnalysisAccessFields />}
<details className="text-sm" open={editingWorker ? true : undefined}>
<summary className="cursor-pointer font-medium">Advanced options</summary>
<div className="mt-4 space-y-5">

104
ui/litellm-dashboard/src/components/lens/settings/worker/WorkerInstall.tsx Normal file
View file

@ -0,0 +1,104 @@
"use client";
import type { ComponentProps, ReactNode } from "react";
import { useMutation } from "@tanstack/react-query";
import { CheckCircle2, Copy, Loader2 } from "lucide-react";
import { Button } from "@/components/ui/button";
import { cn } from "@/lib/cva.config";
import type { WorkerCreated } from "../../model/types";
import { SettingsCard } from "../SettingsSection";
import { workerSetupCommand } from "./workerCommand";
const CLIPBOARD_FAILED = "Clipboard access failed. Allow clipboard access and try again.";
function useCopy() {
return useMutation({ retry: false, mutationFn: (text: string) => navigator.clipboard.writeText(text) });
}
function InstallSteps({ address, created }: { address: string; created: WorkerCreated }) {
const command = workerSetupCommand(address, created.token, created.image);
const copyCommand = useCopy();
const copyToken = useCopy();
return (
<>
<Button variant="default" className="w-full gap-2" onClick={() => copyCommand.mutate(command)}>
{copyCommand.isSuccess ? <CheckCircle2 className="size-4" /> : <Copy className="size-4" />}
{copyCommand.isSuccess ? "Copied" : "Copy Docker command"}
</Button>
<details className="text-sm">
<summary className="cursor-pointer text-muted-foreground">View command</summary>
<p className="mt-3 text-xs text-muted-foreground">Contains a private worker token.</p>
<pre
aria-label="Docker command preview"
className="mt-3 max-h-48 overflow-auto rounded-md bg-muted/40 p-3 text-xs leading-5"
>
{command}
</pre>
</details>
<details className="text-sm">
<summary className="cursor-pointer text-muted-foreground">Using Docker Compose or Helm?</summary>
<p className="mt-3 text-muted-foreground">
Save this private token as LENS_WORKER_TOKEN in Compose or in your Helm worker token secret. Keep it for
future upgrades.
</p>
<Button variant="outline" className="mt-3" onClick={() => copyToken.mutate(created.token)}>
{copyToken.isSuccess ? "Token copied" : "Copy worker token"}
</Button>
</details>
<div className="space-y-3 border-t pt-5">
<div role="status" className="flex items-center gap-2.5 text-sm">
<Loader2 className="size-4 shrink-0 animate-spin text-muted-foreground" /> Waiting for your worker to connect…
</div>
<details className="pl-6.5 text-sm text-muted-foreground">
<summary className="cursor-pointer">Not connecting?</summary>
<p className="mt-2 break-words leading-6">
Check that Docker is running and can reach {address}. Inspect the container logs for connection or
authentication errors. This page updates automatically.
</p>
</details>
</div>
{(copyCommand.isError || copyToken.isError) && (
<p role="alert" className="text-sm text-destructive">
{CLIPBOARD_FAILED}
</p>
)}
</>
);
}
export type WorkerInstallProps = ComponentProps<"div"> & {
address: string;
created: WorkerCreated;
connected: boolean;
/** Rendered once the worker connects, in place of the install steps. */
children: ReactNode;
};
export function WorkerInstall({ address, created, connected, children, className, ...props }: WorkerInstallProps) {
if (!connected)
return (
<SettingsCard {...props} data-slot="worker-install" className={cn("flex flex-col gap-5", className)}>
<header className="space-y-1">
<h3 className="text-base font-semibold">Run the worker</h3>
<p className="text-sm text-muted-foreground">Run this command on a server with Docker.</p>
</header>
<InstallSteps address={address} created={created} />
</SettingsCard>
);
return (
<SettingsCard
{...props}
data-slot="worker-install"
className={cn("flex flex-col items-center gap-5 text-center", className)}
>
<header className="flex flex-col items-center gap-3">
<div className="flex size-12 items-center justify-center rounded-full bg-success/10 text-success">
<CheckCircle2 className="size-6" />
</div>
<h3 className="text-base font-semibold">Worker connected</h3>
<p className="text-sm text-muted-foreground">Ready to run investigations.</p>
</header>
{children}
</SettingsCard>
);
}

65
ui/litellm-dashboard/src/components/lens/settings/worker/WorkerList.tsx Normal file
View file

@ -0,0 +1,65 @@
"use client";
import type { ComponentProps } from "react";
import { Button } from "@/components/ui/button";
import { useNow } from "@/hooks/useNow";
import { StatusDot, type StatusDotProps } from "@/components/shared/StatusDot";
import { cn } from "@/lib/cva.config";
import { workerConnected } from "../../model/status";
import { agoLabel } from "../../model/format";
import { AnalysisKeySummary } from "./AnalysisKeyDetails";
import type { LensList, Worker } from "../../model/types";
function workerStatus(worker: Worker, now: number): { state: StatusDotProps["state"]; label: string } {
if (!worker.analysis_key_id) return { state: "warn", label: "Billing key required" };
if (workerConnected(worker, now)) return { state: "ok", label: "Connected" };
return { state: "off", label: `Not connected · last seen ${agoLabel(Date.parse(worker.last_seen), now)}` };
}
export type WorkerListProps = ComponentProps<"ul"> & {
workers: LensList["workers"];
onEditBilling: (worker: Worker) => void;
onRevoke: (id: string) => void;
};
export function WorkerList({ workers, onEditBilling, onRevoke, className, ...props }: WorkerListProps) {
const now = useNow(5000);
return (
<ul
{...props}
data-slot="worker-list"
className={cn("divide-y divide-border rounded-lg border border-border bg-card", className)}
>
{workers
.filter((w) => !w.revoked)
.map((worker) => {
const status = workerStatus(worker, now);
return (
<li key={worker.id} className="flex flex-wrap items-center gap-x-4 gap-y-2 px-4 py-3 text-sm">
<StatusDot state={status.state} className="shrink-0" />
<div className="min-w-0 flex-1 space-y-0.5">
<div className="flex flex-wrap items-baseline gap-x-2">
<h3 className="font-medium">{worker.name}</h3>
<span className="text-xs text-muted-foreground">{status.label}</span>
</div>
{worker.analysis_key_id && <AnalysisKeySummary keyId={worker.analysis_key_id} />}
</div>
<div className="flex items-center gap-1">
<Button variant="outline" size="sm" onClick={() => onEditBilling(worker)}>
Edit access
</Button>
<Button
variant="ghost"
size="sm"
className="text-muted-foreground hover:text-destructive"
onClick={() => onRevoke(worker.id)}
>
Revoke access
</Button>
</div>
</li>
);
})}
</ul>
);
}

81
ui/litellm-dashboard/src/components/lens/setup/worker/WorkerDialog.integration.test.tsx → ui/litellm-dashboard/src/components/lens/settings/worker/WorkerSettings.integration.test.tsx
View file

@ -1,9 +1,14 @@
import { screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { renderWithProviders, testQueryClient } from "@/../tests/test-utils";
import { useQuery } from "@tanstack/react-query";
import { testQueryClient } from "@/../tests/test-utils";
import { renderWithLens } from "@/../tests/lens-test-utils";
import { apiClient } from "@/components/networking";
import { WorkerDialog } from "./WorkerDialog";
import { lensQueries } from "../../data/queries";
import { useLensApi } from "../../data/LensServices";
import type { LensList } from "../../model/types";
import { WorkerSettings } from "./WorkerSettings";
vi.mock("@/components/networking", () => ({
apiClient: { get: vi.fn(), post: vi.fn(), put: vi.fn(), delete: vi.fn() },
@ -23,6 +28,15 @@ const created = {
},
};
/** Mirrors the workspace: the worker rows come from the cached `/lens` list, so a mutation must refetch it to update them. */
function WorkerSettingsHost() {
const api = useLensApi();
const list = useQuery(lensQueries.list(api));
return list.data ? <WorkerSettings workers={list.data.workers} /> : null;
}
const listCalls = () => vi.mocked(apiClient.get).mock.calls.filter(([path]) => path === "/lens").length;
describe("Worker setup", () => {
beforeEach(() => {
testQueryClient.clear();
@ -39,7 +53,7 @@ describe("Worker setup", () => {
it("generates a complete command using one worker credential and the configured proxy address", async () => {
vi.mocked(apiClient.post).mockResolvedValue(created);
const user = userEvent.setup();
renderWithProviders(<WorkerDialog accessToken="admin" workers={[]} onClose={vi.fn()} onChanged={vi.fn()} />);
const { rerender } = renderWithLens(<WorkerSettings workers={[]} />, { accessToken: "admin" });
await user.click(screen.getByText("Advanced options"));
await user.click(screen.getByRole("switch", { name: "Use an existing virtual key" }));
expect(screen.getByRole("textbox", { name: "LiteLLM proxy URL" })).toHaveValue("https://gateway.example/proxy");
@ -62,22 +76,30 @@ describe("Worker setup", () => {
await user.click(screen.getByText("Using Docker Compose or Helm?"));
await user.click(screen.getByRole("button", { name: "Copy worker token" }));
expect(await navigator.clipboard.readText()).toBe(created.token);
expect(screen.getByRole("button", { name: "Token copied" })).toBeVisible();
expect(await screen.findByRole("button", { name: "Token copied" })).toBeVisible();
rerender(<WorkerSettings workers={[{ ...created.worker, last_seen: new Date().toISOString() }]} />);
expect(screen.getByRole("heading", { name: "Worker connected" })).toBeVisible();
expect(screen.queryByRole("status")).not.toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Done" }));
expect(screen.getByRole("heading", { name: created.worker.name })).toBeVisible();
expect(screen.getByText("Connected")).toBeVisible();
});
it("assigns billing to an existing worker without replacing its access token", async () => {
const user = userEvent.setup();
const changed = vi.fn();
vi.mocked(apiClient.put).mockResolvedValue(created.worker);
renderWithProviders(
<WorkerDialog
accessToken="admin"
workers={[{ ...created.worker, analysis_key_id: null }]}
onClose={vi.fn()}
onChanged={changed}
/>,
);
expect(screen.getByText("Billing key required")).toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Settings" }));
const workers = vi.fn((): LensList["workers"] => [{ ...created.worker, analysis_key_id: null }]);
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [], workers: workers(), tracing_enabled: true };
if (path === "/key/info") return { info: { models: ["analysis-model"], max_budget: 15, budget_duration: "1mo" } };
return { keys: [{ token: "b".repeat(64), key_alias: "Analysis" }], total_pages: 1 };
});
vi.mocked(apiClient.put).mockImplementation(async () => {
workers.mockReturnValue([created.worker]);
return created.worker;
});
renderWithLens(<WorkerSettingsHost />, { accessToken: "admin" });
expect(await screen.findByText("Billing key required")).toBeInTheDocument();
const listedBefore = listCalls();
await user.click(screen.getByRole("button", { name: "Edit access" }));
await user.click(screen.getByRole("combobox", { name: "Charge analysis to" }));
await user.click(await screen.findByRole("option", { name: "Analysis" }));
await user.click(screen.getByRole("button", { name: "Save analysis access" }));
@ -85,23 +107,30 @@ describe("Worker setup", () => {
accessToken: "admin",
body: { analysis_key_id: "b".repeat(64) },
});
expect(changed).toHaveBeenCalledOnce();
expect(await screen.findByText(/Not connected/)).toBeVisible();
expect(screen.queryByText("Billing key required")).not.toBeInTheDocument();
expect(listCalls()).toBe(listedBefore + 1);
expect(apiClient.post).not.toHaveBeenCalled();
});
it("requires revoking the current worker before setting up a replacement", async () => {
const user = userEvent.setup();
const changed = vi.fn();
vi.mocked(apiClient.delete).mockResolvedValue(true);
const props = { accessToken: "admin", onClose: vi.fn(), onChanged: changed };
const view = renderWithProviders(<WorkerDialog {...props} workers={[created.worker]} />);
const workers = vi.fn((): LensList["workers"] => [created.worker]);
vi.mocked(apiClient.get).mockImplementation(async (path) =>
path === "/lens" ? { lenses: [], workers: workers(), tracing_enabled: true } : { data: [] },
);
vi.mocked(apiClient.delete).mockImplementation(async () => {
workers.mockReturnValue([{ ...created.worker, revoked: true }]);
return true;
});
renderWithLens(<WorkerSettingsHost />, { accessToken: "admin" });
const revoke = await screen.findByRole("button", { name: "Revoke access" });
expect(screen.queryByRole("button", { name: "Add worker" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Get install command" })).not.toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Revoke access" }));
await user.click(revoke);
expect(apiClient.delete).toHaveBeenCalledWith("/lens/workers/worker", { accessToken: "admin" });
expect(changed).toHaveBeenCalledOnce();
view.rerender(<WorkerDialog {...props} workers={[{ ...created.worker, revoked: true }]} />);
expect(screen.getByRole("button", { name: "Get install command" })).toBeDisabled();
expect(await screen.findByRole("button", { name: "Get install command" })).toBeDisabled();
expect(screen.getByRole("combobox", { name: "Analysis model" })).toBeVisible();
expect(listCalls()).toBe(2);
});
it("cleans up a newly created key when registration fails before retrying", async () => {
const user = userEvent.setup();
@ -116,7 +145,7 @@ describe("Worker setup", () => {
.mockResolvedValueOnce({})
.mockResolvedValueOnce({ token_id: "retry-key-id" })
.mockResolvedValueOnce(created);
renderWithProviders(<WorkerDialog accessToken="admin" workers={[]} onClose={vi.fn()} onChanged={vi.fn()} />);
renderWithLens(<WorkerSettings workers={[]} />, { accessToken: "admin" });
expect(screen.getByRole("button", { name: "Get install command" })).toBeDisabled();
expect(screen.getByRole("textbox", { name: "LiteLLM proxy URL", hidden: true })).not.toBeVisible();
await user.click(screen.getByRole("combobox", { name: "Analysis model" }));

Some files were not shown because too many files have changed in this diff Show more