scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-06 02:48:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

feat(ui): typed openapi-fetch foundation (fetchClient) + first typed caller (useCustomers) (#29884)

Browse source 
* feat(ui): add the typed openapi-fetch client (fetchClient) as the dashboard fetch foundation

Introduces fetchClient (openapi-fetch) bound to schema.d.ts, used inside ordinary TanStack Query hooks so path/query/body types come from the proxy's OpenAPI spec. A small runtime registry feeds the client the base URL and auth header name (registered by networking) and the session token (published by AuthContext), so call sites carry no token plumbing; auth-header injection and ApiError mapping live in openapi-fetch middleware reusing deriveErrorMessage/ApiError from client.ts, and non-2xx maps to a thrown ApiError so query functions just read .data.

The base URL default resolves from NEXT_PUBLIC_BASE_URL so a request still targets the right origin if it fires before networking registers its getter. AuthContext clears accessToken alongside the token on logout so no query fires unauthenticated after the session ends.

Foundation only; callers migrate one at a time, each fully typed, in follow-up changes.

* feat(ui): migrate useCustomers to the typed fetchClient

Converts useCustomers from allEndUsersCall to fetchClient.GET("/customer/list"); the response is typed as LiteLLM_EndUserTable[] from the schema, so the hand-written Customer/CustomersResponse types are deleted. They were also inaccurate (allowed_model_region was string but is "eu"|"us", and a budget_id the table has no field for). No cast; the schema type flows to the one consumer. First caller on the new pattern.

* fix(ui): route typed-client errors through the session-expiry handler

The typed fetchClient middleware threw ApiError without invoking the
handleError side effect that the legacy createApiClient wires via
onError, so a migrated caller hitting an expired key no longer triggered
the auto-logout. Add an error-handler seam to runtime.ts, register
handleError from networking.tsx alongside the base-url/header getters,
and call it in the middleware before throwing so both clients behave the
same. Regression test asserts the handler fires with the derived message
on non-2xx and stays silent on success

* fix(ui): point the customers EndUser type at CustomerResponse

The /customer/list response model was renamed to CustomerResponse on
staging; the merged branch still aliased EndUser to LiteLLM_EndUserTable,
so the exported type and its test mock had drifted from what the schema
actually returns. CustomerResponse is also the accurate shape (it types
allowed_model_region as 'eu' | 'us' and carries budget_id)

* chore(ui): refresh eslint-metrics baseline after staging merge

The recorded baseline predated the litellm_internal_staging merge, so its
no-explicit-any and no-large-inline-object-arg counts were higher than the
merged tree actually has. Regenerate via npm run lint:metrics so the gate
reflects current reality

* refactor(ui): source the typed client token from the session cookie, not AuthContext

The typed client read its bearer from a runtime value that AuthContext pushed
via setAuthToken, but migrated hooks gate enabled on useAuthorized, which
decodes the cookie directly. Two independent derivations of the same cookie with
different timing: on first load the query fires (useAuthorized sees the token)
before AuthContext's async effect publishes it, so the first request goes out
unauthenticated and only succeeds on a React Query retry.

Make the token a registered getter like the base-url and header-name getters,
reading the same cookie useAuthorized decodes, so the client's token and the
gate can't diverge. Revert the AuthContext changes entirely; nothing is pushed
from React state anymore.
This commit is contained in:
ryan-crabbe-berri 2026-07-11 10:50:51 -07:00 • committed by GitHub
parent 80c5217ddc
commit 0bf81e2496
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 300 additions and 304 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
ui/litellm-dashboard/eslint-metrics.json
View file

@ -1,7 +1,7 @@
{
"@typescript-eslint/no-explicit-any": 1977,
"@typescript-eslint/no-explicit-any": 1971,
"complexity": 129,
"local/no-large-inline-object-arg": 508,
"local/no-large-inline-object-arg": 501,
"local/no-long-condition-chain": 234,
"max-depth": 59,
"no-console": 16

16
ui/litellm-dashboard/package-lock.json generated
View file

@ -28,6 +28,7 @@
"moment": "2.30.1",
"next": "16.2.6",
"openai": "4.104.0",
"openapi-fetch": "^0.17.0",
"papaparse": "5.5.3",
"react": "18.3.1",
"react-copy-to-clipboard": "5.1.1",
@ -10534,6 +10535,15 @@
"integrity": "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA==",
"license": "MIT"
},
"node_modules/openapi-fetch": {
"version": "0.17.0",
"resolved": "https://registry.npmjs.org/openapi-fetch/-/openapi-fetch-0.17.0.tgz",
"integrity": "sha512-PsbZR1wAPcG91eEthKhN+Zn92FMHxv+/faECIwjXdxfTODGSGegYv0sc1Olz+HYPvKOuoXfp+0pA2XVt2cI0Ig==",
"license": "MIT",
"dependencies": {
"openapi-typescript-helpers": "^0.1.0"
}
},
"node_modules/openapi-typescript": {
"version": "7.13.0",
"resolved": "https://registry.npmjs.org/openapi-typescript/-/openapi-typescript-7.13.0.tgz",
@ -10555,6 +10565,12 @@
"typescript": "^5.x"
}
},
"node_modules/openapi-typescript-helpers": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/openapi-typescript-helpers/-/openapi-typescript-helpers-0.1.0.tgz",
"integrity": "sha512-OKTGPthhivLw/fHz6c3OPtg72vi86qaMlqbJuVJ23qOvQ+53uw1n7HdmkJFibloF7QEjDrDkzJiOJuockM/ljw==",
"license": "MIT"
},
"node_modules/openapi-typescript/node_modules/supports-color": {
"version": "10.2.2",
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-10.2.2.tgz",

1
ui/litellm-dashboard/package.json
View file

@ -44,6 +44,7 @@
"moment": "2.30.1",
"next": "16.2.6",
"openai": "4.104.0",
"openapi-fetch": "^0.17.0",
"papaparse": "5.5.3",
"react": "18.3.1",
"react-copy-to-clipboard": "5.1.1",

312
ui/litellm-dashboard/src/app/(dashboard)/hooks/customers/useCustomers.test.ts
View file

@ -1,334 +1,104 @@
import { allEndUsersCall } from "@/components/networking";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { renderHook, waitFor } from "@testing-library/react";
import React, { ReactNode } from "react";
import { beforeEach, describe, expect, it, vi } from "vitest";
import type { Customer, CustomersResponse } from "./useCustomers";
import { useCustomers } from "./useCustomers";
import { useCustomers, type EndUser } from "./useCustomers";
// Mock the networking function
vi.mock("@/components/networking", () => ({
allEndUsersCall: vi.fn(),
const mockGet = vi.fn();
vi.mock("@/lib/http/api", () => ({
fetchClient: { GET: (...args: unknown[]) => mockGet(...args) },
}));
// Mock useAuthorized hook - we can override this in individual tests
const mockUseAuthorized = vi.fn();
vi.mock("@/app/(dashboard)/hooks/useAuthorized", () => ({
default: () => mockUseAuthorized(),
}));
// Import actual roles instead of mocking them
// Mock data
const mockCustomers: Customer[] = [
{
user_id: "customer-1",
alias: "Test Customer 1",
spend: 150.5,
blocked: false,
allowed_model_region: "us-east-1",
default_model: "gpt-3.5-turbo",
budget_id: "budget-1",
litellm_budget_table: {
budget_id: "budget-1",
max_budget: 1000,
soft_budget: 800,
max_parallel_requests: 10,
tpm_limit: 1000,
rpm_limit: 100,
model_max_budget: { "gpt-4": 500 },
budget_duration: "monthly",
budget_reset_at: "2024-02-01T00:00:00Z",
created_at: "2024-01-01T00:00:00Z",
created_by: "admin-1",
updated_at: "2024-01-01T00:00:00Z",
updated_by: "admin-1",
},
},
{
user_id: "customer-2",
alias: null,
spend: 0,
blocked: true,
allowed_model_region: null,
default_model: null,
budget_id: null,
litellm_budget_table: null,
},
const mockCustomers: EndUser[] = [
{ user_id: "customer-1", alias: "Test Customer 1", spend: 150.5, blocked: false },
{ user_id: "customer-2", alias: null, spend: 0, blocked: true },
];
const mockCustomersResponse: CustomersResponse = mockCustomers;
const authorized = {
accessToken: "test-access-token",
userRole: "Admin",
userId: "test-user-id",
token: "test-token",
userEmail: "test@example.com",
premiumUser: false,
disabledPersonalKeyCreation: null,
showSSOBanner: false,
};
describe("useCustomers", () => {
let queryClient: QueryClient;
beforeEach(() => {
queryClient = new QueryClient({
defaultOptions: {
queries: {
retry: false,
},
},
});
// Reset all mocks
queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
vi.clearAllMocks();
// Set default mock for useAuthorized (enabled state)
mockUseAuthorized.mockReturnValue({
accessToken: "test-access-token",
userRole: "Admin",
userId: "test-user-id",
token: "test-token",
userEmail: "test@example.com",
premiumUser: false,
disabledPersonalKeyCreation: null,
showSSOBanner: false,
});
mockUseAuthorized.mockReturnValue(authorized);
});
const wrapper = ({ children }: { children: ReactNode }) =>
React.createElement(QueryClientProvider, { client: queryClient }, children);
it("should return customers data when query is successful", async () => {
// Mock successful API call
(allEndUsersCall as any).mockResolvedValue(mockCustomersResponse);
it("fetches /customer/list and returns the typed list on success", async () => {
mockGet.mockResolvedValue({ data: mockCustomers });
const { result } = renderHook(() => useCustomers(), { wrapper });
// Initially loading
expect(result.current.isLoading).toBe(true);
expect(result.current.data).toBeUndefined();
// Wait for success
await waitFor(() => {
expect(result.current.isLoading).toBe(false);
expect(result.current.isSuccess).toBe(true);
});
expect(result.current.data).toEqual(mockCustomersResponse);
expect(result.current.error).toBeNull();
expect(allEndUsersCall).toHaveBeenCalledWith("test-access-token");
expect(allEndUsersCall).toHaveBeenCalledTimes(1);
expect(result.current.data).toEqual(mockCustomers);
expect(mockGet).toHaveBeenCalledWith("/customer/list");
expect(mockGet).toHaveBeenCalledTimes(1);
});
it("should handle error when allEndUsersCall fails", async () => {
const errorMessage = "Failed to fetch customers";
const testError = new Error(errorMessage);
// Mock failed API call
(allEndUsersCall as any).mockRejectedValue(testError);
it("surfaces an error when the request rejects", async () => {
const testError = new Error("Failed to fetch customers");
mockGet.mockRejectedValue(testError);
const { result } = renderHook(() => useCustomers(), { wrapper });
// Initially loading
expect(result.current.isLoading).toBe(true);
// Wait for error
await waitFor(() => {
expect(result.current.isLoading).toBe(false);
expect(result.current.isError).toBe(true);
});
expect(result.current.error).toEqual(testError);
expect(result.current.data).toBeUndefined();
expect(allEndUsersCall).toHaveBeenCalledWith("test-access-token");
expect(allEndUsersCall).toHaveBeenCalledTimes(1);
});
it("should not execute query when accessToken is missing", async () => {
// Mock missing accessToken
mockUseAuthorized.mockReturnValue({
accessToken: null,
userRole: "Admin",
userId: "test-user-id",
token: null,
userEmail: "test@example.com",
premiumUser: false,
disabledPersonalKeyCreation: null,
showSSOBanner: false,
});
it("falls back to an empty list when the response has no body", async () => {
mockGet.mockResolvedValue({ data: undefined });
const { result } = renderHook(() => useCustomers(), { wrapper });
// Query should not execute
expect(result.current.isLoading).toBe(false);
expect(result.current.data).toBeUndefined();
expect(result.current.isFetched).toBe(false);
// API should not be called
expect(allEndUsersCall).not.toHaveBeenCalled();
});
it("should not execute query when userRole is not an admin role", async () => {
// Mock non-admin userRole
mockUseAuthorized.mockReturnValue({
accessToken: "test-access-token",
userRole: "member", // Not in all_admin_roles
userId: "test-user-id",
token: "test-token",
userEmail: "test@example.com",
premiumUser: false,
disabledPersonalKeyCreation: null,
showSSOBanner: false,
});
const { result } = renderHook(() => useCustomers(), { wrapper });
// Query should not execute
expect(result.current.isLoading).toBe(false);
expect(result.current.data).toBeUndefined();
expect(result.current.isFetched).toBe(false);
// API should not be called
expect(allEndUsersCall).not.toHaveBeenCalled();
});
it("should not execute query when userRole is null", async () => {
// Mock null userRole
mockUseAuthorized.mockReturnValue({
accessToken: "test-access-token",
userRole: null,
userId: "test-user-id",
token: "test-token",
userEmail: "test@example.com",
premiumUser: false,
disabledPersonalKeyCreation: null,
showSSOBanner: false,
});
const { result } = renderHook(() => useCustomers(), { wrapper });
// Query should not execute
expect(result.current.isLoading).toBe(false);
expect(result.current.data).toBeUndefined();
expect(result.current.isFetched).toBe(false);
// API should not be called
expect(allEndUsersCall).not.toHaveBeenCalled();
});
it("should not execute query when userRole is empty string", async () => {
// Mock empty string userRole
mockUseAuthorized.mockReturnValue({
accessToken: "test-access-token",
userRole: "",
userId: "test-user-id",
token: "test-token",
userEmail: "test@example.com",
premiumUser: false,
disabledPersonalKeyCreation: null,
showSSOBanner: false,
});
const { result } = renderHook(() => useCustomers(), { wrapper });
// Query should not execute
expect(result.current.isLoading).toBe(false);
expect(result.current.data).toBeUndefined();
expect(result.current.isFetched).toBe(false);
// API should not be called
expect(allEndUsersCall).not.toHaveBeenCalled();
});
it("should not execute query when both accessToken and userRole are missing", async () => {
// Mock both auth values missing
mockUseAuthorized.mockReturnValue({
accessToken: null,
userRole: null,
userId: "test-user-id",
token: null,
userEmail: "test@example.com",
premiumUser: false,
disabledPersonalKeyCreation: null,
showSSOBanner: false,
});
const { result } = renderHook(() => useCustomers(), { wrapper });
// Query should not execute
expect(result.current.isLoading).toBe(false);
expect(result.current.data).toBeUndefined();
expect(result.current.isFetched).toBe(false);
// API should not be called
expect(allEndUsersCall).not.toHaveBeenCalled();
});
it("should execute query when accessToken is present and userRole is Admin", async () => {
// Mock successful API call
(allEndUsersCall as any).mockResolvedValue(mockCustomersResponse);
// Ensure auth values are set (already done in beforeEach)
const { result } = renderHook(() => useCustomers(), { wrapper });
// Wait for query to execute
await waitFor(() => {
expect(result.current.isLoading).toBe(false);
});
expect(allEndUsersCall).toHaveBeenCalledWith("test-access-token");
expect(allEndUsersCall).toHaveBeenCalledTimes(1);
});
it("should execute query when accessToken is present and userRole is proxy_admin", async () => {
// Mock successful API call
(allEndUsersCall as any).mockResolvedValue(mockCustomersResponse);
// Mock proxy_admin role
mockUseAuthorized.mockReturnValue({
accessToken: "test-access-token",
userRole: "proxy_admin",
userId: "test-user-id",
token: "test-token",
userEmail: "test@example.com",
premiumUser: false,
disabledPersonalKeyCreation: null,
showSSOBanner: false,
});
const { result } = renderHook(() => useCustomers(), { wrapper });
// Wait for query to execute
await waitFor(() => {
expect(result.current.isLoading).toBe(false);
});
expect(allEndUsersCall).toHaveBeenCalledWith("test-access-token");
expect(allEndUsersCall).toHaveBeenCalledTimes(1);
});
it("should return empty customers array when API returns empty data", async () => {
// Mock API returning empty customers array
(allEndUsersCall as any).mockResolvedValue([]);
const { result } = renderHook(() => useCustomers(), { wrapper });
// Wait for success
await waitFor(() => {
expect(result.current.isLoading).toBe(false);
expect(result.current.isSuccess).toBe(true);
});
expect(result.current.data).toEqual([]);
expect(allEndUsersCall).toHaveBeenCalledWith("test-access-token");
});
it("should handle network timeout error", async () => {
const timeoutError = new Error("Network timeout");
// Mock network timeout
(allEndUsersCall as any).mockRejectedValue(timeoutError);
it("does not fetch when the access token is missing", () => {
mockUseAuthorized.mockReturnValue({ ...authorized, accessToken: null, token: null });
const { result } = renderHook(() => useCustomers(), { wrapper });
// Wait for error
await waitFor(() => {
expect(result.current.isError).toBe(true);
});
expect(result.current.isFetched).toBe(false);
expect(mockGet).not.toHaveBeenCalled();
});
expect(result.current.error).toEqual(timeoutError);
expect(result.current.data).toBeUndefined();
it("does not fetch when the user is not an admin", () => {
mockUseAuthorized.mockReturnValue({ ...authorized, userRole: "member" });
const { result } = renderHook(() => useCustomers(), { wrapper });
expect(result.current.isFetched).toBe(false);
expect(mockGet).not.toHaveBeenCalled();
});
});

37
ui/litellm-dashboard/src/app/(dashboard)/hooks/customers/useCustomers.ts
View file

@ -1,42 +1,19 @@
import { allEndUsersCall } from "@/components/networking";
import { useQuery } from "@tanstack/react-query";
import { createQueryKeys } from "../common/queryKeysFactory";
import { fetchClient } from "@/lib/http/api";
import { all_admin_roles } from "@/utils/roles";
import useAuthorized from "@/app/(dashboard)/hooks/useAuthorized";
import type { components } from "@/lib/http/schema";
export type EndUser = components["schemas"]["CustomerResponse"];
const customersKeys = createQueryKeys("customers");
export interface Customer {
user_id: string;
alias?: string | null;
spend: number;
blocked: boolean;
allowed_model_region?: string | null;
default_model?: string | null;
budget_id?: string | null;
litellm_budget_table?: {
budget_id: string;
max_budget?: number | null;
soft_budget?: number | null;
max_parallel_requests?: number | null;
tpm_limit?: number | null;
rpm_limit?: number | null;
model_max_budget?: Record<string, unknown> | null;
budget_duration?: string | null;
budget_reset_at?: string | null;
created_at: string;
created_by: string;
updated_at: string;
updated_by: string;
} | null;
}
export type CustomersResponse = Customer[];
export const useCustomers = () => {
const { accessToken, userRole } = useAuthorized();
return useQuery<CustomersResponse>({
return useQuery({
queryKey: customersKeys.list({}),
queryFn: async () => await allEndUsersCall(accessToken!),
queryFn: async () => (await fetchClient.GET("/customer/list")).data ?? [],
enabled: Boolean(accessToken) && all_admin_roles.includes(userRole!),
});
};

14
ui/litellm-dashboard/src/components/networking.tsx
View file

@ -22,7 +22,8 @@ export const getCallbackConfigsCall = async (accessToken: string) => {
* Helper file for calls being made to proxy
*/
import MessageManager from "@/components/molecules/message_manager";
import { clearTokenCookies, storeLoginToken } from "@/utils/cookieUtils";
import { clearTokenCookies, getCookie, storeLoginToken } from "@/utils/cookieUtils";
import { decodeToken } from "@/utils/jwtUtils";
import { TagNewRequest, TagUpdateRequest, TagListResponse, TagInfoResponse } from "./tag_management/types";
import { Team } from "./key_team_helpers/key_list";
import { EmailEventSettingsResponse, EmailEventSettingsUpdateRequest } from "./email_events/types";
@ -38,6 +39,12 @@ import type {
import { MCP_TOOLS_PREVIEW_FORBIDDEN_MESSAGE } from "./mcp_tools/constants";
import { createApiClient, deriveErrorMessage } from "@/lib/http/client";
import { resolveApiBase } from "@/lib/http/resolveApiBase";
import {
registerAuthHeaderNameGetter,
registerAuthTokenGetter,
registerBaseUrlGetter,
registerErrorHandler,
} from "@/lib/http/runtime";
import { serverRootPath, setServerRootPath } from "@/lib/serverRootPath";
export { serverRootPath };
@ -371,6 +378,11 @@ const apiClient = createApiClient({
onError: handleError,
});
registerBaseUrlGetter(getProxyBaseUrl);
registerAuthHeaderNameGetter(getGlobalLitellmHeaderName);
registerAuthTokenGetter(() => decodeToken(getCookie("token"))?.key ?? null);
registerErrorHandler(handleError);
export const makeModelGroupPublic = async (accessToken: string, modelGroups: string[]) => {
const url = proxyBaseUrl ? `${proxyBaseUrl}/model_group/make_public` : `/model_group/make_public`;
const response = await fetch(url, {

104
ui/litellm-dashboard/src/lib/http/api.test.ts Normal file
View file

@ -0,0 +1,104 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { fetchClient } from "./api";
import {
registerAuthHeaderNameGetter,
registerAuthTokenGetter,
registerBaseUrlGetter,
registerErrorHandler,
} from "./runtime";
const jsonResponse = (status: number, body: unknown): Response =>
new Response(JSON.stringify(body), { status, headers: { "Content-Type": "application/json" } });
const capturingFetch = (response: Response) => {
const requests: Request[] = [];
const fetch = vi.fn(async (request: Request) => {
requests.push(request);
return response;
});
return { fetch, requests };
};
describe("typed api client middleware", () => {
beforeEach(() => {
registerBaseUrlGetter(() => "");
registerAuthHeaderNameGetter(() => "Authorization");
registerErrorHandler(() => {});
registerAuthTokenGetter(() => null);
});
afterEach(() => {
vi.restoreAllMocks();
});
it("injects the bearer token under the registered auth header name", async () => {
registerAuthTokenGetter(() => "sk-test");
registerAuthHeaderNameGetter(() => "x-litellm-key");
const { fetch, requests } = capturingFetch(jsonResponse(200, { data: [] }));
await fetchClient.GET("/model_group/info", { fetch });
expect(requests[0].headers.get("x-litellm-key")).toBe("Bearer sk-test");
expect(requests[0].headers.get("Authorization")).toBeNull();
});
it("omits the auth header when no token is set", async () => {
const { fetch, requests } = capturingFetch(jsonResponse(200, { data: [] }));
await fetchClient.GET("/model_group/info", { fetch });
expect(requests[0].headers.get("Authorization")).toBeNull();
});
it("rebases the request onto the registered base url, preserving path and query", async () => {
registerBaseUrlGetter(() => "https://proxy.example.com/");
const { fetch, requests } = capturingFetch(jsonResponse(200, { data: [] }));
await fetchClient.GET("/model_group/info", { fetch, params: { query: { model_group: "gpt-4o" } } });
const url = new URL(requests[0].url);
expect(url.origin).toBe("https://proxy.example.com");
expect(url.pathname).toBe("/model_group/info");
expect(url.searchParams.get("model_group")).toBe("gpt-4o");
});
it("maps a non-2xx response to an ApiError carrying status and the derived message", async () => {
const { fetch } = capturingFetch(jsonResponse(403, { error: { message: "no access" } }));
await expect(fetchClient.GET("/model_group/info", { fetch })).rejects.toMatchObject({
name: "ApiError",
status: 403,
message: "no access",
});
});
it("returns the parsed body on a successful response", async () => {
const body = { data: [{ model_group: "gpt-4o" }] };
const { fetch } = capturingFetch(jsonResponse(200, body));
const { data, error } = await fetchClient.GET("/model_group/info", { fetch });
expect(error).toBeUndefined();
expect(data).toEqual(body);
});
it("reports the derived message to the registered error handler on a non-2xx response", async () => {
const onError = vi.fn();
registerErrorHandler(onError);
const { fetch } = capturingFetch(jsonResponse(401, { error: { message: "Authentication Error - Expired Key" } }));
await expect(fetchClient.GET("/model_group/info", { fetch })).rejects.toBeInstanceOf(Error);
expect(onError).toHaveBeenCalledWith("Authentication Error - Expired Key");
});
it("does not call the error handler on a successful response", async () => {
const onError = vi.fn();
registerErrorHandler(onError);
const { fetch } = capturingFetch(jsonResponse(200, { data: [] }));
await fetchClient.GET("/model_group/info", { fetch });
expect(onError).not.toHaveBeenCalled();
});
});

48
ui/litellm-dashboard/src/lib/http/api.ts Normal file
View file

@ -0,0 +1,48 @@
import createFetchClient, { type Middleware } from "openapi-fetch";
import type { paths } from "./schema";
import { ApiError, deriveErrorMessage } from "./client";
import { getAuthHeaderName, getAuthToken, getRequestBaseUrl, reportError } from "./runtime";
const rebaseUrl = (requestUrl: string, base: string): string => {
const { pathname, search } = new URL(requestUrl);
return `${base.replace(/\/+$/, "")}${pathname}${search}`;
};
const middleware: Middleware = {
onRequest({ request }) {
const base = getRequestBaseUrl();
const next = new Request(base ? rebaseUrl(request.url, base) : request.url, request);
const token = getAuthToken();
if (token) {
next.headers.set(getAuthHeaderName(), `Bearer ${token}`);
}
return next;
},
async onResponse({ response }) {
if (response.ok) return response;
const raw = await response.clone().text();
let body: unknown = raw;
let message: string;
try {
body = JSON.parse(raw);
message = deriveErrorMessage(body);
} catch {
message = raw || `HTTP ${response.status}`;
}
reportError(message);
throw new ApiError(message, response.status, body);
},
};
/**
* The typed, schema-bound HTTP client. Use it inside TanStack Query hooks
* (`fetchClient.GET("/path", { params })`) and for imperative calls; path
* params, query params, and request bodies are inferred from schema.d.ts.
*
* The creation-time base is the current origin so request URLs are absolute; the
* middleware rebases each call onto the runtime base when one is registered (a
* split-origin proxy or worker URL), injects the auth header, and maps non-2xx
* responses to ApiError so query functions can just read `.data`.
*/
export const fetchClient = createFetchClient<paths>({ baseUrl: globalThis.location?.origin ?? "" });
fetchClient.use(middleware);

22
ui/litellm-dashboard/src/lib/http/runtime.test.ts Normal file
View file

@ -0,0 +1,22 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { getAuthHeaderName, getRequestBaseUrl } from "./runtime";
describe("runtime request config defaults", () => {
afterEach(() => {
vi.unstubAllEnvs();
});
it("resolves the default base URL from NEXT_PUBLIC_BASE_URL before a getter is registered", () => {
vi.stubEnv("NEXT_PUBLIC_BASE_URL", "https://proxy.example.com/");
expect(getRequestBaseUrl()).toBe("https://proxy.example.com");
});
it("defaults the base URL to same-origin when NEXT_PUBLIC_BASE_URL is unset", () => {
vi.stubEnv("NEXT_PUBLIC_BASE_URL", "");
expect(getRequestBaseUrl()).toBe("");
});
it("defaults the auth header name to Authorization", () => {
expect(getAuthHeaderName()).toBe("Authorization");
});
});

46
ui/litellm-dashboard/src/lib/http/runtime.ts Normal file
View file

@ -0,0 +1,46 @@
import { resolveApiBase } from "./resolveApiBase";
/**
* Runtime request config the typed client reads on every call. The values are
* mutable at runtime (base URL can switch to a worker origin; the auth header
* name and token come from the logged-in session), and they are owned outside
* this module: networking.tsx registers the base URL / header-name / token
* getters and the error handler. The token getter reads the session cookie, the
* same source useAuthorized decodes, so the client's token and the gate that
* enables a query cannot diverge. Keeping the seam here (not importing from the
* component tree) lets api.ts stay in lib/http without a layering inversion.
*
* The base URL default resolves from NEXT_PUBLIC_BASE_URL so a request still
* hits the right origin if it fires before networking registers its fuller
* getter (which additionally folds in the server root path from the live UI
* config). The auth header name has no build-time source, so it defaults to
* "Authorization" until the session's JWT supplies a custom one.
*/
type Getter<T> = () => T;
let baseUrlGetter: Getter<string> = () => resolveApiBase({ explicitBase: process.env.NEXT_PUBLIC_BASE_URL });
let authHeaderNameGetter: Getter<string> = () => "Authorization";
let authTokenGetter: Getter<string | null> = () => null;
let errorHandler: (message: string) => void = () => {};
export const registerBaseUrlGetter = (getter: Getter<string>): void => {
baseUrlGetter = getter;
};
export const registerAuthHeaderNameGetter = (getter: Getter<string>): void => {
authHeaderNameGetter = getter;
};
export const registerAuthTokenGetter = (getter: Getter<string | null>): void => {
authTokenGetter = getter;
};
export const registerErrorHandler = (handler: (message: string) => void): void => {
errorHandler = handler;
};
export const getRequestBaseUrl = (): string => baseUrlGetter();
export const getAuthHeaderName = (): string => authHeaderNameGetter();
export const getAuthToken = (): string | null => authTokenGetter();
export const reportError = (message: string): void => errorHandler(message);