From 0c50286a55e1beba301adabeb0648ac4316f773b Mon Sep 17 00:00:00 2001 From: tin-berri Date: Fri, 21 Aug 2026 14:50:02 -0700 Subject: [PATCH] feat(ui): add per-key Savings tab to key detail page (#37693) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(ui): add per-key Savings tab to key detail page Adds a "Savings" tab to the key detail view, showing the same four metrics and time-series chart as the proxy-wide Cost Optimization view, but scoped to a single API key. For org admins, the tab shows the key's full savings across all requests. Non-admins see only their own requests on the key, with a scope note explaining the limitation. Root cause: userDailyActivityCall and userDailyActivityAggregatedCall never forwarded an api_key query parameter to the backend, even though both handlers already accept and filter by it. Changes: - networking.tsx: Add optional apiKey param to both daily activity call wrappers (appended to variadic options tuple for backward compatibility). - costOptimizationUtils.ts: Extract shared metrics helpers (compressionOf, cachingOf, autorouterOf, savedTokensOf, cacheHitRatio) and shortDate so both UsageTab and KeySavingsTab use the same formulas and prevent divergence. - useDailyActivityRange.ts: Refactor into useScopedDailyActivityRange( accessToken, scope: {userId, apiKey?}) for reuse-by-parameter unbundling. Role resolution stays at the entry point (useDailyActivityRange), not in a scoped caller. Update test expectations for new 6-arg tuple. - UsageTab.tsx: Simplify by importing extracted helpers and SummaryCard component instead of defining them inline. No behavioral change. - key_info_view.tsx: Insert "Savings" tab trigger between "Overview" and "Settings"; wire TabsContent to new KeySavingsTab component with lazy mounting (no keepMounted) to defer daily-activity fetch until tab opened. - NEW: components/shared/SummaryCard.tsx — Shared presenter for four-tile summary row (label + value + hint + optional info popover). Extracted from UsageTab so both surfaces show identical tile layout without CSS divergence. - NEW: components/templates/KeySavingsTab.tsx — Per-key view with admin/ non-admin scope branching, empty-state messaging, same chart toggles and info popovers as UsageTab. - NEW: components/templates/KeySavingsTab.test.tsx — 7 tests covering mount, loading state, empty state, scoping, and scope-note visibility. Authorization: No new permission check. Both backends gate api_key filter by the same user role check that governs the request itself. Non-admins must send their own user_id and can only see their own keys. Tests: 6121 pass (1 pre-existing failure unrelated to this change). Prior art / collision note: - PR #37570 (budgets tab) lands in same TabsList hunks as "Savings" tab, but different tab names so conflict trivial if both merge. - PR #37659 (my own) adds progress/cancelled/cancel to DailyActivityRange, but this PR uses stable three-field interface from staging. * fix(ui): scope spend view by the backend's admin-view contract, not all_admin_roles Greptile flagged org admin handling on the key savings tab. The live bug it described does not fire today: useAuthorized supplies session-role labels and all_admin_roles only carries the raw org_admin spelling, so an org admin was already scoped. That safety was accidental, so replace the predicate with spendScopeUserId / hasProxyWideSpendView in utils/roles.ts, mirroring the backend's user_api_key_has_admin_view (proxy admin and admin viewer only, org admin excluded in both spellings), and use it in both useDailyActivityRange and KeySavingsTab Reclassify the KeySavingsTab render test as an integration test per the repo's unit/integration split, move scope-resolution coverage to roles.test.ts as a full role matrix, use real session-role values instead of raw ones, and assert tile totals against non-empty metrics. Replace the nested ternary in the chart body (frontend-lint error) with flat conditional rendering * fix(ui): show auto-router savings as the fourth key-savings tile Cache hit rate had displaced auto-router savings from the fourth slot, diverging from the org-wide Cost Optimization page's tile order. Match it: Total / Compression / Prompt caching / Auto-router, with cache hit rate as a fifth tile. * fix(ui): drop cache hit rate from the key savings tiles Keep the four tiles this page is meant to show: total, compression, prompt caching, and auto-router savings. * fix(ui): stop an empty api_key from widening a key-scoped activity read The paginated and aggregated daily-activity wrappers disagreed on an empty filter value: the paginated one appended it, the aggregated one coerced it to undefined with || and dropped it. Since the aggregated call is the one tried first, an empty key hash would have silently turned a key-scoped read into a proxy-wide one and reported every key's savings as this key's. Use ?? so both send the filter through and it matches nothing instead. * style(ui): satisfy prettier and the inline-object lint rule in key savings tests * refactor(ui): drop the cacheHitRatio extraction left over from the removed tile * fix(ui): pass daily-activity filters raw so both transports agree at the null boundary * refactor(ui): share the savings tiles and totals between both surfaces The per-key Savings tab and the proxy-wide Cost Optimization tab carried a byte-identical four-tile block, three long metric-definition strings included, and five identical useMemo totals. Both now render SavingsTiles and total through useSavingsTotals, so the donut cannot slice numbers the tile above it disagrees with. * docs(ui): say request, not mount, in the savings tab comment The comment claimed mounting eagerly would fire the rollup sweep, which reads as a claim about the bundle. Only the request is deferred; the module ships with the key page either way. * test(ui): pin the daily-activity args array against the real caller signatures The sibling unit test mocks networking, so it checks the positional array against itself and stays green when the array and a networking signature drift apart. Swapping user_id and api_key in the aggregated signature alone passes there and fails here on user_id=hash-abc. * style(ui): hoist the daily-activity query options out of the call argument The four-property object literal tripped local/no-large-inline-object-arg. The violation predates this branch, which only moved the line into the annotated range, and the rule count drops 550 to 549. --- .../_components/UsageTab.tsx | 81 ++-------- .../_components/costOptimizationUtils.ts | 8 + ...useDailyActivityRange.integration.test.tsx | 35 +++++ .../useDailyActivityRange.test.tsx | 13 +- .../_components/useDailyActivityRange.ts | 43 +++++- .../src/components/networking.test.ts | 82 ++++++++++ .../src/components/networking.tsx | 12 +- .../src/components/shared/SavingsTiles.tsx | 65 ++++++++ .../src/components/shared/SummaryCard.tsx | 50 ++++++ .../KeySavingsTab.integration.test.tsx | 133 ++++++++++++++++ .../components/templates/KeySavingsTab.tsx | 144 ++++++++++++++++++ .../components/templates/key_info_view.tsx | 15 ++ ui/litellm-dashboard/src/utils/roles.test.ts | 47 ++++++ ui/litellm-dashboard/src/utils/roles.ts | 12 ++ 14 files changed, 657 insertions(+), 83 deletions(-) create mode 100644 ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.integration.test.tsx create mode 100644 ui/litellm-dashboard/src/components/shared/SavingsTiles.tsx create mode 100644 ui/litellm-dashboard/src/components/shared/SummaryCard.tsx create mode 100644 ui/litellm-dashboard/src/components/templates/KeySavingsTab.integration.test.tsx create mode 100644 ui/litellm-dashboard/src/components/templates/KeySavingsTab.tsx diff --git a/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/UsageTab.tsx b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/UsageTab.tsx index 530f85dc83b..e19ee7e48b0 100644 --- a/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/UsageTab.tsx +++ b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/UsageTab.tsx @@ -1,19 +1,18 @@ "use client"; import React, { useEffect, useMemo, useState } from "react"; -import { Info } from "lucide-react"; import { AreaChart, BarChart, CustomLegend, DonutChart, SEQUENTIAL_COLOR_RAMP } from "@/components/shared/charts"; import AdvancedDatePicker from "@/components/shared/advanced_date_picker"; import { Card, CardAction, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"; -import { Popover, PopoverContent, PopoverTrigger } from "@/components/ui/popover"; import { Tabs, TabsList, TabsTrigger } from "@/components/ui/tabs"; import useCan from "@/app/(dashboard)/hooks/useCan"; import { getToolSpend, ToolSpendResponse } from "@/components/networking"; -import { SpendMetrics } from "@/components/UsagePage/types"; -import { formatNumberWithCommas } from "@/utils/dataUtils"; import { + autorouterOf, buildDailyToolSeries, + cachingOf, + compressionOf, formatRangeLabel, localIsoDay, MAX_POINTS_WITH_DOTS, @@ -22,11 +21,13 @@ import { SAVINGS_SERIES, SavingsAccumulation, SavingsPoint, + shortDate, toCumulative, topToolsBySpend, usd, withStartAnchor, } from "./costOptimizationUtils"; +import SavingsTiles, { useSavingsTotals } from "@/components/shared/SavingsTiles"; import { DailyActivityRange } from "./useDailyActivityRange"; interface UsageTabProps { @@ -41,42 +42,8 @@ const EMPTY_TOOL_SPEND: ToolSpendResponse = { end_date: null, }; -const shortDate = (iso: string): string => - new Date(`${iso}T00:00:00`).toLocaleDateString("en-US", { month: "short", day: "numeric" }); - const isoDay = (d: Date): string => d.toISOString().slice(0, 10); -const compressionOf = (m: SpendMetrics): number => m.compression_savings_spend ?? 0; -const cachingOf = (m: SpendMetrics): number => m.prompt_caching_savings_spend ?? 0; -const autorouterOf = (m: SpendMetrics): number => m.autorouter_savings_spend ?? 0; -const savedTokensOf = (m: SpendMetrics): number => m.compression_saved_tokens ?? 0; - -const SummaryCard = ({ label, value, hint, info }: { label: string; value: string; hint?: string; info?: string }) => ( - - - {label} - {info && ( - - - - - - {info} - - - )} - - -

{value}

- {hint &&

{hint}

} - - -); - const UsageTab: React.FC = ({ accessToken, activity }) => { const { dateValue, onDateChange, results, loading, isFetchingMore } = activity; @@ -106,11 +73,7 @@ const UsageTab: React.FC = ({ accessToken, activity }) => { const toolSpend = toolSpendState?.key === rangeKey ? toolSpendState.data : null; const toolSpendLoading = toolSpendEnabled && toolSpend === null; - const compressionTotal = useMemo(() => results.reduce((sum, d) => sum + compressionOf(d.metrics), 0), [results]); - const cachingTotal = useMemo(() => results.reduce((sum, d) => sum + cachingOf(d.metrics), 0), [results]); - const autorouterTotal = useMemo(() => results.reduce((sum, d) => sum + autorouterOf(d.metrics), 0), [results]); - const savedTokensTotal = useMemo(() => results.reduce((sum, d) => sum + savedTokensOf(d.metrics), 0), [results]); - const totalSaved = compressionTotal + cachingTotal + autorouterTotal; + const totals = useSavingsTotals(results); const [accumulation, setAccumulation] = useState("cumulative"); @@ -156,9 +119,11 @@ const UsageTab: React.FC = ({ accessToken, activity }) => { SAVINGS_DRIVERS.map(({ name, color }) => ({ driver: name, color, - usd: { Compression: compressionTotal, "Prompt caching": cachingTotal, "Auto-router": autorouterTotal }[name], + usd: { Compression: totals.compression, "Prompt caching": totals.caching, "Auto-router": totals.autorouter }[ + name + ], })).filter((d) => d.usd > 0), - [compressionTotal, cachingTotal, autorouterTotal], + [totals], ); const plottedDriverTotal = useMemo(() => byDriver.reduce((sum, d) => sum + d.usd, 0), [byDriver]); @@ -185,31 +150,7 @@ const UsageTab: React.FC = ({ accessToken, activity }) => { -
- - - - -
+
diff --git a/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/costOptimizationUtils.ts b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/costOptimizationUtils.ts index 71f9c63fe99..7f075e48341 100644 --- a/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/costOptimizationUtils.ts +++ b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/costOptimizationUtils.ts @@ -12,6 +12,14 @@ export const usd = (value: number): string => { export const pct = (ratio: number): string => `${formatNumberWithCommas(ratio * 100, 1)}%`; +export const shortDate = (iso: string): string => + new Date(`${iso}T00:00:00`).toLocaleDateString("en-US", { month: "short", day: "numeric" }); + +export const compressionOf = (m: SpendMetrics): number => m.compression_savings_spend ?? 0; +export const cachingOf = (m: SpendMetrics): number => m.prompt_caching_savings_spend ?? 0; +export const autorouterOf = (m: SpendMetrics): number => m.autorouter_savings_spend ?? 0; +export const savedTokensOf = (m: SpendMetrics): number => m.compression_saved_tokens ?? 0; + export type CacheLeakageDimension = "key" | "model"; export interface CacheLeakageRow { diff --git a/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.integration.test.tsx b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.integration.test.tsx new file mode 100644 index 00000000000..78280e32eed --- /dev/null +++ b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.integration.test.tsx @@ -0,0 +1,35 @@ +import { renderHook, waitFor } from "@testing-library/react"; +import { afterEach, describe, expect, it, vi } from "vitest"; + +import { useScopedDailyActivityRange } from "./useDailyActivityRange"; + +// The sibling unit test mocks networking, so it pins the positional args array against itself and +// would still pass if the array and the two networking signatures drifted apart. Nothing else runs +// the real callers, which is what makes the ordering a comment-enforced rule. This drives the hook +// through the actual query serializer instead, so an argument appended or inserted on one side only +// lands its value on the wrong filter and fails here. +describe("useScopedDailyActivityRange wiring", () => { + const originalFetch = global.fetch; + + afterEach(() => { + global.fetch = originalFetch; + }); + + it("lands each scope field on its own query param through the real networking callers", async () => { + const mockFetch = vi.fn().mockResolvedValue( + new Response(JSON.stringify({ results: [], metadata: {} }), { + status: 200, + headers: { "Content-Type": "application/json" }, + }), + ); + global.fetch = mockFetch; + + renderHook(() => useScopedDailyActivityRange("sk-token", { userId: "u1", apiKey: "hash-abc" })); + + await waitFor(() => expect(mockFetch).toHaveBeenCalled()); + + const url = String(mockFetch.mock.calls[0][0]); + expect(url).toContain("user_id=u1"); + expect(url).toContain("api_key=hash-abc"); + }); +}); diff --git a/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.test.tsx b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.test.tsx index 2229438d844..6c9281060a4 100644 --- a/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.test.tsx +++ b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.test.tsx @@ -33,15 +33,24 @@ describe("useDailyActivityRange", () => { it("queries every user's activity for an admin", () => { renderHook(() => useDailyActivityRange("test-token", "u1", "proxy_admin")); - expect(argsOfLastCall()).toEqual(["test-token", expect.any(Date), expect.any(Date), null, true]); + expect(argsOfLastCall()).toEqual(["test-token", expect.any(Date), expect.any(Date), null, true, null]); }); it("scopes the query to the caller for a non-admin", () => { renderHook(() => useDailyActivityRange("test-token", "u1", "internal_user")); - expect(argsOfLastCall()).toEqual(["test-token", expect.any(Date), expect.any(Date), "u1", true]); + expect(argsOfLastCall()).toEqual(["test-token", expect.any(Date), expect.any(Date), "u1", true, null]); }); + it.each(["org_admin", "Org Admin"])( + "scopes the query to the caller for %s, who has no admin view on this endpoint", + (role) => { + renderHook(() => useDailyActivityRange("test-token", "u1", role)); + + expect(argsOfLastCall()).toEqual(["test-token", expect.any(Date), expect.any(Date), "u1", true, null]); + }, + ); + it("fetches through the single-shot aggregated endpoint first so days never fragment across pages", () => { renderHook(() => useDailyActivityRange("test-token", "u1", "proxy_admin")); diff --git a/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.ts b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.ts index 81ddb6af585..8ed57e36a94 100644 --- a/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.ts +++ b/ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useDailyActivityRange.ts @@ -2,7 +2,7 @@ import { useMemo, useState } from "react"; import { userDailyActivityAggregatedCall, userDailyActivityCall } from "@/components/networking"; import { DailyData } from "@/components/UsagePage/types"; -import { all_admin_roles } from "@/utils/roles"; +import { spendScopeUserId } from "@/utils/roles"; import { usePaginatedDailyActivity } from "@/app/(dashboard)/usage/_components/hooks/usePaginatedDailyActivity"; const THIRTY_DAYS_MS = 30 * 24 * 60 * 60 * 1000; @@ -23,10 +23,23 @@ export interface DailyActivityRange { cancel: () => void; } -export const useDailyActivityRange = ( +/** + * Which slice of daily activity to read. Both fields are passed straight through to the + * endpoint as filters, so the caller — not this hook — decides what the viewer may see. + * + * `userId: null` asks for the whole proxy, which the backend only honours for admins; + * a non-admin must send its own id or the request is rejected. That role decision lives in + * `useDailyActivityRange` below rather than in here, so a caller scoping to one key is not + * silently re-scoped to a user as well. + */ +export interface DailyActivityScope { + userId: string | null; + apiKey?: string | null; +} + +export const useScopedDailyActivityRange = ( accessToken: string | null, - userId: string | null, - userRole: string, + scope: DailyActivityScope, ): DailyActivityRange => { const initialFrom = useMemo(() => new Date(new Date().getTime() - THIRTY_DAYS_MS), []); const initialTo = useMemo(() => new Date(), []); @@ -34,14 +47,19 @@ export const useDailyActivityRange = ( const startTime = dateValue.from ?? null; const endTime = dateValue.to ?? null; - const effectiveUserId = all_admin_roles.includes(userRole) ? null : userId; + const { userId, apiKey = null } = scope; - const { data, loading, isFetchingMore, progress, cancelled, cancel } = usePaginatedDailyActivity({ + const activityQueryOptions = { fetchFn: userDailyActivityCall, aggregatedFetchFn: userDailyActivityAggregatedCall, - args: [accessToken, startTime, endTime, effectiveUserId, true], + // Positional, and read by two functions whose signatures diverge at index 3: the paginated + // call takes `page` there (injected by the hook) and the aggregated one does not. Anything + // appended here must therefore be appended to BOTH networking signatures, in this order. + args: [accessToken, startTime, endTime, userId, true, apiKey], enabled: !!accessToken && !!startTime && !!endTime, - }); + }; + const { data, loading, isFetchingMore, progress, cancelled, cancel } = + usePaginatedDailyActivity(activityQueryOptions); return { dateValue, @@ -54,3 +72,12 @@ export const useDailyActivityRange = ( cancel, }; }; + +export const useDailyActivityRange = ( + accessToken: string | null, + userId: string | null, + userRole: string, +): DailyActivityRange => + useScopedDailyActivityRange(accessToken, { + userId: spendScopeUserId(userRole, userId), + }); diff --git a/ui/litellm-dashboard/src/components/networking.test.ts b/ui/litellm-dashboard/src/components/networking.test.ts index 86fe86307a2..f12a52a0bcf 100644 --- a/ui/litellm-dashboard/src/components/networking.test.ts +++ b/ui/litellm-dashboard/src/components/networking.test.ts @@ -627,3 +627,85 @@ describe("getAutoRouterClassifierDefaultPromptCall", () => { expect(String(mockFetch.mock.calls[1][0])).not.toContain("tier_labels"); }); }); + +describe("daily activity api_key filter", () => { + const originalFetch = global.fetch; + + const captureFetch = () => { + const mockFetch = vi.fn().mockResolvedValue( + new Response(JSON.stringify({ results: [], metadata: {} }), { + status: 200, + headers: { "Content-Type": "application/json" }, + }), + ); + global.fetch = mockFetch; + return mockFetch; + }; + + const requestedUrl = (mockFetch: ReturnType): string => String(mockFetch.mock.calls[0][0]); + + const start = new Date("2025-01-01T00:00:00Z"); + const end = new Date("2025-01-31T00:00:00Z"); + + afterEach(() => { + global.fetch = originalFetch; + }); + + it("sends the key hash as api_key from the paginated caller", async () => { + const mockFetch = captureFetch(); + + await Networking.userDailyActivityCall("sk-key", start, end, 1, null, false, "hash-abc"); + + expect(requestedUrl(mockFetch)).toContain("api_key=hash-abc"); + }); + + it("sends the key hash as api_key from the aggregated caller", async () => { + const mockFetch = captureFetch(); + + await Networking.userDailyActivityAggregatedCall("sk-key", start, end, null, false, "hash-abc"); + + expect(requestedUrl(mockFetch)).toContain("api_key=hash-abc"); + }); + + // The two wrappers serialize the same optional filters through different transports, so an + // absent key has to drop the param in both. Dropping it on one side and sending it empty on + // the other would widen a key-scoped read into an unscoped one. + it.each([ + ["paginated", () => Networking.userDailyActivityCall("sk-key", start, end, 1, null, false, null)], + ["aggregated", () => Networking.userDailyActivityAggregatedCall("sk-key", start, end, null, false, null)], + ])("omits api_key entirely from the %s caller when no key is given", async (_label, call) => { + const mockFetch = captureFetch(); + + await call(); + + expect(requestedUrl(mockFetch)).not.toContain("api_key"); + }); + + // An empty key must not be coerced into "no filter". Dropping it would turn a key-scoped read + // into a proxy-wide one and report every key's savings as this key's, so both callers send it + // through and let the filter match nothing instead. + it.each([ + ["paginated", () => Networking.userDailyActivityCall("sk-key", start, end, 1, null, false, "")], + ["aggregated", () => Networking.userDailyActivityAggregatedCall("sk-key", start, end, null, false, "")], + ])("keeps an empty api_key as a filter rather than widening the %s read", async (_label, call) => { + const mockFetch = captureFetch(); + + await call(); + + expect(requestedUrl(mockFetch)).toContain("api_key="); + }); + + // user_id rides the same two transports and widens the same way, so it gets the same guard. + // The aggregated caller used to drop "" via `||`; without this the two filters could drift + // apart again on one side only. + it.each([ + ["paginated", () => Networking.userDailyActivityCall("sk-key", start, end, 1, "", false, null)], + ["aggregated", () => Networking.userDailyActivityAggregatedCall("sk-key", start, end, "", false, null)], + ])("keeps an empty user_id as a filter rather than widening the %s read", async (_label, call) => { + const mockFetch = captureFetch(); + + await call(); + + expect(requestedUrl(mockFetch)).toContain("user_id="); + }); +}); diff --git a/ui/litellm-dashboard/src/components/networking.tsx b/ui/litellm-dashboard/src/components/networking.tsx index 21ac1a8f5d1..5b6d70b4771 100644 --- a/ui/litellm-dashboard/src/components/networking.tsx +++ b/ui/litellm-dashboard/src/components/networking.tsx @@ -1410,6 +1410,7 @@ export const userDailyActivityCall = async ( page: number = 1, userId: string | null = null, includeCurrentUtcDay: boolean = false, + apiKey: string | null = null, ) => { /** * Get daily user activity on proxy @@ -1423,6 +1424,7 @@ export const userDailyActivityCall = async ( extraQueryParams: { user_id: userId, include_current_utc_day: includeCurrentUtcDay ? "true" : undefined, + api_key: apiKey, }, }); }; @@ -2505,12 +2507,12 @@ export const userDailyActivityAggregatedCall = async ( accessToken: string, startTime: Date, endTime: Date, - ...options: [userId?: string | null, includeCurrentUtcDay?: boolean] + ...options: [userId?: string | null, includeCurrentUtcDay?: boolean, apiKey?: string | null] ) => { /** * Get aggregated daily user activity (no pagination) */ - const [userId = null, includeCurrentUtcDay = false] = options; + const [userId = null, includeCurrentUtcDay = false, apiKey = null] = options; try { const formatDate = (date: Date) => { const year = date.getFullYear(); @@ -2524,8 +2526,12 @@ export const userDailyActivityAggregatedCall = async ( start_date: formatDate(startTime), end_date: formatDate(endTime), timezone: new Date().getTimezoneOffset().toString(), - user_id: userId || undefined, + // Passed raw, matching the paginated caller: both serializers drop null and undefined, + // and both keep "". An empty filter must not vanish, or a request scoped to one user or + // key would silently widen into an unscoped, proxy-wide read. + user_id: userId, include_current_utc_day: includeCurrentUtcDay ? "true" : undefined, + api_key: apiKey, }, }); } catch (error) { diff --git a/ui/litellm-dashboard/src/components/shared/SavingsTiles.tsx b/ui/litellm-dashboard/src/components/shared/SavingsTiles.tsx new file mode 100644 index 00000000000..64294e05ef3 --- /dev/null +++ b/ui/litellm-dashboard/src/components/shared/SavingsTiles.tsx @@ -0,0 +1,65 @@ +"use client"; + +import React, { useMemo } from "react"; + +import SummaryCard from "@/components/shared/SummaryCard"; +import { + autorouterOf, + cachingOf, + compressionOf, + savedTokensOf, + usd, +} from "@/app/(dashboard)/cost-optimization/_components/costOptimizationUtils"; +import { DailyData } from "@/components/UsagePage/types"; +import { formatNumberWithCommas } from "@/utils/dataUtils"; + +// Exported because the by-driver donut has to slice the same numbers the tiles print, and two +// totalling paths over the same rows is how a chart and the tile above it end up disagreeing. +export const useSavingsTotals = (results: DailyData[]) => + useMemo(() => { + const sumOf = (of: (metrics: DailyData["metrics"]) => number) => results.reduce((sum, d) => sum + of(d.metrics), 0); + const compression = sumOf(compressionOf); + const caching = sumOf(cachingOf); + const autorouter = sumOf(autorouterOf); + return { + compression, + caching, + autorouter, + savedTokens: sumOf(savedTokensOf), + total: compression + caching + autorouter, + }; + }, [results]); + +const SavingsTiles = ({ results, isLoading }: { results: DailyData[]; isLoading: boolean }) => { + const totals = useSavingsTotals(results); + + return ( +
+ + + + +
+ ); +}; + +export default SavingsTiles; diff --git a/ui/litellm-dashboard/src/components/shared/SummaryCard.tsx b/ui/litellm-dashboard/src/components/shared/SummaryCard.tsx new file mode 100644 index 00000000000..2924f89c642 --- /dev/null +++ b/ui/litellm-dashboard/src/components/shared/SummaryCard.tsx @@ -0,0 +1,50 @@ +"use client"; + +import React from "react"; +import { Info } from "lucide-react"; + +import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; +import { Popover, PopoverContent, PopoverTrigger } from "@/components/ui/popover"; + +export interface SummaryCardProps { + label: string; + value: string; + hint?: string; + /** Rendered behind an info affordance. Use it for how a figure is derived, not for restating the label. */ + info?: string; +} + +/** + * A labelled figure with an optional hint line and an optional "how is this calculated" popover. + * Shared by the proxy-wide Cost Optimization usage tab and the per-key savings tab so both + * surfaces present the same figures identically. + */ +const slugOf = (label: string): string => label.toLowerCase().replace(/\s+/g, "-"); + +const SummaryCard = ({ label, value, hint, info }: SummaryCardProps) => ( + + + {label} + {info && ( + + + + + + {info} + + + )} + + +

{value}

+ {hint &&

{hint}

} + + +); + +export default SummaryCard; diff --git a/ui/litellm-dashboard/src/components/templates/KeySavingsTab.integration.test.tsx b/ui/litellm-dashboard/src/components/templates/KeySavingsTab.integration.test.tsx new file mode 100644 index 00000000000..c91a79fb2db --- /dev/null +++ b/ui/litellm-dashboard/src/components/templates/KeySavingsTab.integration.test.tsx @@ -0,0 +1,133 @@ +import { describe, it, expect, vi, beforeEach } from "vitest"; +import { render, screen } from "@testing-library/react"; +import KeySavingsTab from "./KeySavingsTab"; +import { DailyData, SpendMetrics } from "@/components/UsagePage/types"; +import * as useScopedDailyActivityRangeModule from "@/app/(dashboard)/cost-optimization/_components/useDailyActivityRange"; + +const metrics = (overrides: Partial): SpendMetrics => ({ + spend: 0, + prompt_tokens: 0, + completion_tokens: 0, + total_tokens: 0, + api_requests: 0, + successful_requests: 0, + failed_requests: 0, + cache_read_input_tokens: 0, + cache_creation_input_tokens: 0, + ...overrides, +}); + +const day = (date: string, overrides: Partial): DailyData => ({ + date, + metrics: metrics(overrides), + breakdown: { + models: {}, + model_groups: {}, + mcp_servers: {}, + providers: {}, + api_keys: {}, + entities: {}, + }, +}); + +const mockActivity = ( + overrides: Partial = {}, +): useScopedDailyActivityRangeModule.DailyActivityRange => ({ + dateValue: { from: new Date("2025-01-01"), to: new Date("2025-01-31") }, + onDateChange: vi.fn(), + results: [] as DailyData[], + loading: false, + isFetchingMore: false, + progress: { currentPage: 1, totalPages: 1 }, + cancelled: false, + cancel: vi.fn(), + ...overrides, +}); + +const scopedRange = () => vi.spyOn(useScopedDailyActivityRangeModule, "useScopedDailyActivityRange"); + +const renderTab = (props: Partial> = {}) => + render( + , + ); + +describe("KeySavingsTab", () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + it("totals each savings driver across the days in range", () => { + const firstDay: Partial = { + compression_savings_spend: 1.5, + prompt_caching_savings_spend: 0.25, + autorouter_savings_spend: 2, + compression_saved_tokens: 400, + cache_read_input_tokens: 300, + prompt_tokens: 1000, + }; + const secondDay: Partial = { + compression_savings_spend: 0.5, + prompt_caching_savings_spend: 0.75, + autorouter_savings_spend: 1, + compression_saved_tokens: 600, + cache_read_input_tokens: 200, + prompt_tokens: 1000, + }; + scopedRange().mockReturnValue( + mockActivity({ results: [day("2025-01-01", firstDay), day("2025-01-02", secondDay)] }), + ); + + renderTab(); + + expect(screen.getByTestId("summary-card-total-saved")).toHaveTextContent("$6.00"); + expect(screen.getByTestId("summary-card-compression-savings")).toHaveTextContent("$2.00"); + expect(screen.getByTestId("summary-card-compression-savings")).toHaveTextContent("1,000 tokens compressed"); + expect(screen.getByTestId("summary-card-prompt-caching-savings")).toHaveTextContent("$1.00"); + expect(screen.getByTestId("summary-card-auto-router-savings")).toHaveTextContent("$3.00"); + }); + + it("separates a key with no traffic from one still loading", () => { + scopedRange().mockReturnValue(mockActivity()); + + const { unmount } = renderTab(); + expect(screen.getByTestId("key-savings-empty")).toHaveTextContent("No usage recorded for this key"); + unmount(); + + scopedRange().mockReturnValue(mockActivity({ loading: true })); + renderTab(); + expect(screen.getByTestId("key-savings-empty")).toHaveTextContent("Loading savings"); + }); + + it("asks the endpoint for this key alone, scoped to the viewer's own rows", () => { + const hook = scopedRange().mockReturnValue(mockActivity()); + + renderTab({ userId: "user-456", userRole: "Internal User" }); + + expect(hook).toHaveBeenCalledWith("test-token", { userId: "user-456", apiKey: "key-abc123" }); + expect(screen.getByTestId("key-savings-scope-note")).toHaveTextContent("Showing your own requests"); + }); + + it("reads the whole key for a proxy admin, with no scope note to contradict it", () => { + const hook = scopedRange().mockReturnValue(mockActivity()); + + renderTab({ userId: "admin-123", userRole: "Admin" }); + + expect(hook).toHaveBeenCalledWith("test-token", { userId: null, apiKey: "key-abc123" }); + expect(screen.queryByTestId("key-savings-scope-note")).not.toBeInTheDocument(); + }); + + it("keeps the scope note for an org admin, whose figures cover only their own requests", () => { + const hook = scopedRange().mockReturnValue(mockActivity()); + + renderTab({ userId: "org-admin-1", userRole: "Org Admin" }); + + expect(hook).toHaveBeenCalledWith("test-token", { userId: "org-admin-1", apiKey: "key-abc123" }); + expect(screen.getByTestId("key-savings-scope-note")).toHaveTextContent("Showing your own requests"); + }); +}); diff --git a/ui/litellm-dashboard/src/components/templates/KeySavingsTab.tsx b/ui/litellm-dashboard/src/components/templates/KeySavingsTab.tsx new file mode 100644 index 00000000000..d499c74a535 --- /dev/null +++ b/ui/litellm-dashboard/src/components/templates/KeySavingsTab.tsx @@ -0,0 +1,144 @@ +"use client"; + +import React, { useMemo, useState } from "react"; + +import { AreaChart, BarChart, CustomLegend } from "@/components/shared/charts"; +import AdvancedDatePicker from "@/components/shared/advanced_date_picker"; +import SavingsTiles from "@/components/shared/SavingsTiles"; +import { Card, CardAction, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"; +import { Tabs, TabsList, TabsTrigger } from "@/components/ui/tabs"; +import { hasProxyWideSpendView, spendScopeUserId } from "@/utils/roles"; +import { + autorouterOf, + cachingOf, + compressionOf, + formatRangeLabel, + localIsoDay, + MAX_POINTS_WITH_DOTS, + SAVINGS_COLORS, + SAVINGS_SERIES, + SavingsAccumulation, + SavingsPoint, + shortDate, + toCumulative, + usd, + withStartAnchor, +} from "@/app/(dashboard)/cost-optimization/_components/costOptimizationUtils"; +import { useScopedDailyActivityRange } from "@/app/(dashboard)/cost-optimization/_components/useDailyActivityRange"; + +interface KeySavingsTabProps { + accessToken: string | null; + /** The key's token hash — what spend rows are keyed by, not the one-time plaintext secret. */ + keyToken: string; + userId: string | null; + userRole: string; +} + +const KeySavingsTab: React.FC = ({ accessToken, keyToken, userId, userRole }) => { + // Proxy admins read the whole key. For anyone else the endpoint applies the caller's own user_id + // alongside the key filter, so the figures cover only that viewer's requests on this key -- said + // plainly in the scope note below rather than left to be misread as the key's total. + const readsWholeKey = hasProxyWideSpendView(userRole); + const activity = useScopedDailyActivityRange(accessToken, { + userId: spendScopeUserId(userRole, userId), + apiKey: keyToken, + }); + + const { dateValue, onDateChange, results, loading, isFetchingMore } = activity; + const startTime = dateValue.from ?? null; + const endTime = dateValue.to ?? null; + + const [accumulation, setAccumulation] = useState("cumulative"); + + // Sort on the raw ISO date before shortDate() drops the year: the rollup arrives newest + // first, and the running total has to accumulate forward in time. + const perInterval = useMemo( + () => + [...results] + .sort((a, b) => a.date.localeCompare(b.date)) + .map((d) => ({ + date: shortDate(d.date), + Compression: compressionOf(d.metrics), + "Prompt caching": cachingOf(d.metrics), + "Auto-router": autorouterOf(d.metrics), + })), + [results], + ); + + const overTime = useMemo(() => { + if (accumulation !== "cumulative") return perInterval; + const startLabel = startTime ? shortDate(localIsoDay(startTime)) : ""; + return withStartAnchor(toCumulative(perInterval), startLabel); + }, [accumulation, perInterval, startTime]); + + const intervalLabel = "Per day"; + const rangeLabel = formatRangeLabel(startTime ?? undefined, endTime ?? undefined); + const savingsSubtitle = [ + accumulation === "cumulative" ? "Running total saved" : `Saved ${intervalLabel.toLowerCase()}`, + rangeLabel && `${rangeLabel} (UTC)`, + ] + .filter(Boolean) + .join(" · "); + + const isLoading = loading || isFetchingMore; + const hasRows = results.length > 0; + const chartProps = { + data: overTime, + index: "date", + categories: SAVINGS_SERIES, + colors: SAVINGS_COLORS, + valueFormatter: usd, + showLegend: false, + }; + + return ( +
+
+ Spend is bucketed by UTC day + +
+ + {!readsWholeKey && ( +

+ Showing your own requests on this key. A key shared across a team will have spend from other members that is + not counted here. +

+ )} + + + + + + Savings + {savingsSubtitle} + + + setAccumulation(value as SavingsAccumulation)}> + + Cumulative + {intervalLabel} + + + + + + {/* Distinguishes "still fetching" from "this key genuinely had no traffic": an empty + chart alone reads as a broken panel, and a $0.00 tile reads as a real zero. */} + {!hasRows && ( +

+ {isLoading ? "Loading savings..." : "No usage recorded for this key in this range."} +

+ )} + {hasRows && accumulation === "cumulative" && ( + + )} + {/* Not stacked: auto-router can go negative on a cold-cache write, and stacking would + draw that below the axis while the rest of the bar still read as the total. */} + {hasRows && accumulation !== "cumulative" && } + + +
+ ); +}; + +export default KeySavingsTab; diff --git a/ui/litellm-dashboard/src/components/templates/key_info_view.tsx b/ui/litellm-dashboard/src/components/templates/key_info_view.tsx index b14820d8afa..5cfa737397b 100644 --- a/ui/litellm-dashboard/src/components/templates/key_info_view.tsx +++ b/ui/litellm-dashboard/src/components/templates/key_info_view.tsx @@ -14,6 +14,7 @@ import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"; import { EntityLink } from "@/components/shared/EntityLink"; import { teamDetailHref } from "@/utils/entityLinks"; import { KeyInfoHeader } from "./KeyInfoHeader"; +import KeySavingsTab from "./KeySavingsTab"; import { useEffect, useState } from "react"; import { isProxyAdminRole, isUserTeamAdminForSingleTeam, rolesWithWriteAccess } from "../../utils/roles"; import { mapDisplayToInternalNames, mapInternalToDisplayNames } from "../callback_info_helpers"; @@ -603,6 +604,9 @@ export default function KeyInfoView({ Overview + + Savings + Settings @@ -738,6 +742,17 @@ export default function KeyInfoView({
+ {/* Savings Panel. No keepMounted: this tab sweeps the daily rollup, and staying mounted + would fire that request on every key page open for people who never look at it. */} + + + + {/* Settings Panel */} diff --git a/ui/litellm-dashboard/src/utils/roles.test.ts b/ui/litellm-dashboard/src/utils/roles.test.ts index b2d7c2c75e5..0170d01d8e6 100644 --- a/ui/litellm-dashboard/src/utils/roles.test.ts +++ b/ui/litellm-dashboard/src/utils/roles.test.ts @@ -2,7 +2,9 @@ import { describe, it, expect } from "vitest"; import { all_admin_roles, effectiveSessionRole, + hasProxyWideSpendView, isAdminRole, + spendScopeUserId, isOrgAdminForAnyOrg, isOrgAdminSessionRole, isProxyAdminRole, @@ -325,4 +327,49 @@ describe("roles", () => { expect(teamListScopeUserId(effectiveSessionRole("org_admin"), SESSION_USER_ID)).toBeNull(); }); }); + + describe("spendScopeUserId", () => { + const SESSION_USER_ID = "user-1234"; + + it.each(["proxy_admin", "proxy_admin_viewer"])( + "drops the user id for %s, whom the daily-activity endpoints let read every user's spend", + (rawRole) => { + expect(hasProxyWideSpendView(effectiveSessionRole(rawRole))).toBe(true); + expect(spendScopeUserId(effectiveSessionRole(rawRole), SESSION_USER_ID)).toBeNull(); + }, + ); + + it.each(["Admin", "Admin Viewer", "proxy_admin", "proxy_admin_viewer"])( + "accepts %s in either the session-role or raw spelling, since all_admin_roles carries both", + (role) => { + expect(spendScopeUserId(role, SESSION_USER_ID)).toBeNull(); + }, + ); + + it.each(["internal_user", "internal_user_viewer", "internal_viewer", "app_user"])( + "scopes %s to its own user id, which is the only one the endpoint authorizes", + (rawRole) => { + expect(spendScopeUserId(effectiveSessionRole(rawRole), SESSION_USER_ID)).toBe(SESSION_USER_ID); + }, + ); + + it("scopes an unknown or absent role rather than asking for the whole proxy", () => { + expect(spendScopeUserId(null, SESSION_USER_ID)).toBe(SESSION_USER_ID); + expect(spendScopeUserId("Undefined Role", SESSION_USER_ID)).toBe(SESSION_USER_ID); + }); + + // The backend's user_api_key_has_admin_view covers PROXY_ADMIN and PROXY_ADMIN_VIEW_ONLY only, + // so an org admin asking for the whole proxy is silently narrowed to its own rows and the + // figures would read as the key's total. Both spellings have to scope, unlike teamListScopeUserId. + it.each(["org_admin", "Org Admin"])("scopes %s, whom the backend does not grant an admin view", (role) => { + expect(hasProxyWideSpendView(role)).toBe(false); + expect(spendScopeUserId(role, SESSION_USER_ID)).toBe(SESSION_USER_ID); + }); + + it("differs from all_admin_roles by exactly org admin, which is the whole point of not reusing it", () => { + const scopedByAllAdminRoles = all_admin_roles.filter((role) => spendScopeUserId(role, SESSION_USER_ID) !== null); + + expect(scopedByAllAdminRoles).toEqual(["org_admin"]); + }); + }); }); diff --git a/ui/litellm-dashboard/src/utils/roles.ts b/ui/litellm-dashboard/src/utils/roles.ts index b6d84b9365d..62a5f02cc39 100644 --- a/ui/litellm-dashboard/src/utils/roles.ts +++ b/ui/litellm-dashboard/src/utils/roles.ts @@ -109,3 +109,15 @@ const sessionRolesWithBroadTeamList: string[] = ["Admin", "Admin Viewer", "Org A export const teamListScopeUserId = (userRole: string | null, userId: string | null): string | null => sessionRolesWithBroadTeamList.includes(userRole ?? "") ? null : userId; + +// Mirrors the backend's `user_api_key_has_admin_view`, which the daily-activity endpoints gate on: +// proxy admins read every user's spend, and everyone else is forced to their own user_id. Org admin +// is deliberately absent, so this is `all_admin_roles` minus org admin rather than a reuse of it. +// Both spellings are listed because that constant carries both and either may reach a call site. +const rolesWithProxyWideSpendView: string[] = ["Admin", "Admin Viewer", "proxy_admin", "proxy_admin_viewer"]; + +export const hasProxyWideSpendView = (userRole: string | null): boolean => + rolesWithProxyWideSpendView.includes(userRole ?? ""); + +export const spendScopeUserId = (userRole: string | null, userId: string | null): string | null => + hasProxyWideSpendView(userRole) ? null : userId;