mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-17 23:51:30 +00:00
Krrishdholakia/UI multi proxy url (#22684)
* feat(ui): Add multi-proxy URL switcher for control plane/data plane architecture Allows users to manage and switch between multiple LiteLLM proxy instances from a single UI, enabling a control plane with multiple independent data planes. Changes: - Create ProxyConnectionContext to manage proxy connections in localStorage - Add ProxySwitcher dropdown in navbar to switch between configured proxies - Add ManageProxiesModal for adding/editing/removing proxy connections - Update useAuthorized to use API key for remote proxies instead of cookie JWT - Disable UI config fetch for remote proxies to prevent URL overwriting - Add setProxyBaseUrl export to allow context to update proxy URL When users switch proxies, the page reloads with the new proxy URL in the global, ensuring all 272 API functions use the correct endpoint. Remote proxies require CORS configured to allow the UI's origin. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: Add UI multi-proxy switcher section to control plane docs Add documentation for the new UI proxy switcher feature with mermaid diagrams showing the architecture flow, sequence diagram for the switching workflow, and independent database topology. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ui): Address Greptile review feedback on proxy switcher - Fix removeConnection to use setProxyBaseUrl(null) instead of setProxyBaseUrl(defaultConn.url), consistent with switchConnection - Replace hardcoded "Admin" role for remote proxies with actual role fetched from /user/info endpoint on the remote proxy Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Krish Dholakia
GitHub
parent
a3aef0d3ea
commit
f2b3beac5d
9 changed files with 722 additions and 13 deletions
|
|
@ -27,7 +27,40 @@ When scaling LiteLLM for production use, you may want to deploy multiple instanc
|
|||
|
||||
### Typical Deployment Scenario
|
||||
|
||||
<Image img={require('../../img/scaling_architecture.png')} />
|
||||
<Image img={require('../../img/scaling_architecture.png')} />
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph CP["Control Plane (Admin Instance)"]
|
||||
UI["Admin UI"]
|
||||
ADMIN_API["Management APIs<br/>/key/*, /user/*, /team/*"]
|
||||
end
|
||||
|
||||
subgraph DP_US["Data Plane - US Region"]
|
||||
WORKER_US["Worker Instance<br/>/chat/completions, /v1/*"]
|
||||
end
|
||||
|
||||
subgraph DP_EU["Data Plane - EU Region"]
|
||||
WORKER_EU["Worker Instance<br/>/chat/completions, /v1/*"]
|
||||
end
|
||||
|
||||
DB[("Shared PostgreSQL<br/>Database")]
|
||||
|
||||
UI -->|"Manage keys, teams,<br/>models, config"| ADMIN_API
|
||||
UI -->|"Switch proxy URL<br/>to view data plane"| WORKER_US
|
||||
UI -->|"Switch proxy URL<br/>to view data plane"| WORKER_EU
|
||||
ADMIN_API --> DB
|
||||
WORKER_US --> DB
|
||||
WORKER_EU --> DB
|
||||
|
||||
USER_US["US Users"] -->|"LLM requests"| WORKER_US
|
||||
USER_EU["EU Users"] -->|"LLM requests"| WORKER_EU
|
||||
ADMIN["Admins"] -->|"Administration"| UI
|
||||
|
||||
style CP fill:#e6f3ff,stroke:#4a90d9
|
||||
style DP_US fill:#e6ffe6,stroke:#4a9950
|
||||
style DP_EU fill:#e6ffe6,stroke:#4a9950
|
||||
```
|
||||
|
||||
### Benefits of This Architecture
|
||||
|
||||
|
|
@ -206,6 +239,92 @@ response = requests.post(
|
|||
)
|
||||
```
|
||||
|
||||
## UI Multi-Proxy Switcher
|
||||
|
||||
The LiteLLM Admin UI supports switching between multiple proxy instances directly from the browser. This allows administrators to manage all data planes from a single control plane UI without needing separate browser tabs or logins.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Admin
|
||||
participant UI as Admin UI
|
||||
participant CP as Control Plane Proxy
|
||||
participant DP as Data Plane Proxy
|
||||
|
||||
Admin->>UI: Open Admin UI
|
||||
UI->>CP: Load dashboard (default proxy)
|
||||
Admin->>UI: Add data plane connection<br/>(URL + API key)
|
||||
UI->>DP: Test connection (/health/readiness)
|
||||
DP-->>UI: OK (v1.x.x)
|
||||
Admin->>UI: Switch to data plane
|
||||
UI->>UI: Save selection to localStorage
|
||||
UI->>UI: Reload page
|
||||
UI->>DP: Load dashboard from data plane
|
||||
Admin->>UI: Switch back to control plane
|
||||
UI->>CP: Load dashboard from control plane
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
1. Log into the Admin UI on your control plane instance
|
||||
2. Click the proxy switcher in the navbar (appears after adding a second connection)
|
||||
3. Select **Manage Connections** to add a data plane
|
||||
4. Enter the data plane's URL and an API key (master key or virtual key)
|
||||
5. Click **Test Connection** to verify connectivity
|
||||
6. Save and switch between proxies using the dropdown
|
||||
|
||||
### Adding a Data Plane Connection
|
||||
|
||||
From the Admin UI navbar, open **Manage Connections** and provide:
|
||||
|
||||
| Field | Description | Example |
|
||||
|-------|-------------|---------|
|
||||
| **Name** | A label for this connection | `Production US-East` |
|
||||
| **Proxy URL** | Full URL of the data plane | `https://us-east.litellm.company.com` |
|
||||
| **API Key** | Master key or virtual key for auth | `sk-...` |
|
||||
|
||||
### Requirements
|
||||
|
||||
- **CORS**: Each data plane proxy must have CORS configured to allow requests from the control plane UI's origin. LiteLLM's FastAPI CORS middleware handles this — ensure `allow_origins` includes your control plane domain.
|
||||
- **API Key**: Since the UI authenticates to remote proxies via API key (not cookie-based SSO), provide a key with sufficient permissions for the operations you need.
|
||||
- **Network Access**: The admin's browser must be able to reach each data plane URL directly.
|
||||
|
||||
### Architecture with Independent Databases
|
||||
|
||||
For deployments where each data plane has its own database (true multi-tenancy), the UI switcher allows viewing and managing each independently:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph CP["Control Plane"]
|
||||
UI["Admin UI<br/>(Proxy Switcher)"]
|
||||
CP_PROXY["Admin Proxy"]
|
||||
CP_DB[("Control Plane DB")]
|
||||
end
|
||||
|
||||
subgraph DP1["Data Plane 1"]
|
||||
DP1_PROXY["Worker Proxy"]
|
||||
DP1_DB[("Data Plane 1 DB")]
|
||||
end
|
||||
|
||||
subgraph DP2["Data Plane 2"]
|
||||
DP2_PROXY["Worker Proxy"]
|
||||
DP2_DB[("Data Plane 2 DB")]
|
||||
end
|
||||
|
||||
UI -->|"Default connection"| CP_PROXY
|
||||
UI -.->|"Switch to DP1"| DP1_PROXY
|
||||
UI -.->|"Switch to DP2"| DP2_PROXY
|
||||
|
||||
CP_PROXY --> CP_DB
|
||||
DP1_PROXY --> DP1_DB
|
||||
DP2_PROXY --> DP2_DB
|
||||
|
||||
style CP fill:#e6f3ff,stroke:#4a90d9
|
||||
style DP1 fill:#fff3e6,stroke:#d9944a
|
||||
style DP2 fill:#fff3e6,stroke:#d9944a
|
||||
```
|
||||
|
||||
Connections are stored in browser `localStorage`, so they persist across sessions and require no backend configuration.
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Virtual Keys](./virtual_keys.md) - Managing API keys and users
|
||||
|
|
|
|||
|
|
@ -1,14 +1,18 @@
|
|||
import { getUiConfig, LiteLLMWellKnownUiConfig } from "@/components/networking";
|
||||
import { useProxyConnection } from "@/contexts/ProxyConnectionContext";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { createQueryKeys } from "../common/queryKeysFactory";
|
||||
|
||||
const uiConfigKeys = createQueryKeys("uiConfig");
|
||||
|
||||
export const useUIConfig = () => {
|
||||
const { isRemoteProxy } = useProxyConnection();
|
||||
|
||||
return useQuery<LiteLLMWellKnownUiConfig>({
|
||||
queryKey: uiConfigKeys.list({}),
|
||||
queryFn: async () => await getUiConfig(),
|
||||
staleTime: 24 * 60 * 60 * 1000, // 24 hours - data rarely changes
|
||||
gcTime: 24 * 60 * 60 * 1000, // 24 hours - keep in cache for 24 hours
|
||||
enabled: !isRemoteProxy, // Don't fetch UI config from remote proxies — it would overwrite proxyBaseUrl
|
||||
});
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,10 +1,11 @@
|
|||
"use client";
|
||||
|
||||
import { getProxyBaseUrl } from "@/components/networking";
|
||||
import { getProxyBaseUrl, getGlobalLitellmHeaderName } from "@/components/networking";
|
||||
import { clearTokenCookies, getCookie } from "@/utils/cookieUtils";
|
||||
import { checkTokenValidity, decodeToken } from "@/utils/jwtUtils";
|
||||
import { useProxyConnection } from "@/contexts/ProxyConnectionContext";
|
||||
import { useRouter } from "next/navigation";
|
||||
import { useEffect, useMemo } from "react";
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import { useUIConfig } from "./uiConfig/useUIConfig";
|
||||
|
||||
function formatUserRole(userRole: string) {
|
||||
|
|
@ -39,37 +40,84 @@ function formatUserRole(userRole: string) {
|
|||
const useAuthorized = () => {
|
||||
const router = useRouter();
|
||||
const { data: uiConfig, isLoading: isUIConfigLoading } = useUIConfig();
|
||||
const { activeConnection, isRemoteProxy } = useProxyConnection();
|
||||
|
||||
const token = typeof document !== "undefined" ? getCookie("token") : null;
|
||||
|
||||
const decoded = useMemo(() => decodeToken(token), [token]);
|
||||
const isTokenValid = useMemo(() => checkTokenValidity(token), [token]);
|
||||
const isLoading = isUIConfigLoading;
|
||||
const isAuthorized = isTokenValid && !uiConfig?.admin_ui_disabled;
|
||||
const isLoading = isUIConfigLoading && !isRemoteProxy;
|
||||
|
||||
// For remote proxies, authorized if we have a stored API key
|
||||
const isRemoteAuthorized = isRemoteProxy && !!activeConnection?.apiKey;
|
||||
const isAuthorized = isRemoteAuthorized || (isTokenValid && !uiConfig?.admin_ui_disabled);
|
||||
|
||||
// Fetch actual user info from remote proxy to get the real role
|
||||
const [remoteUserInfo, setRemoteUserInfo] = useState<{
|
||||
user_role: string | null;
|
||||
user_id: string | null;
|
||||
user_email: string | null;
|
||||
}>({ user_role: null, user_id: null, user_email: null });
|
||||
|
||||
useEffect(() => {
|
||||
if (!isRemoteProxy || !activeConnection?.apiKey) return;
|
||||
|
||||
const fetchRemoteUserInfo = async () => {
|
||||
try {
|
||||
const url = `${getProxyBaseUrl()}/user/info`;
|
||||
const response = await fetch(url, {
|
||||
method: "GET",
|
||||
headers: {
|
||||
[getGlobalLitellmHeaderName()]: `Bearer ${activeConnection.apiKey}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
});
|
||||
if (response.ok) {
|
||||
const data = await response.json();
|
||||
setRemoteUserInfo({
|
||||
user_role: data?.user_role ?? null,
|
||||
user_id: data?.user_id ?? null,
|
||||
user_email: data?.user_email ?? null,
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// Silently fail — role will fall back to "Admin"
|
||||
}
|
||||
};
|
||||
|
||||
fetchRemoteUserInfo();
|
||||
}, [isRemoteProxy, activeConnection?.apiKey]); // eslint-disable-line react-hooks/exhaustive-deps
|
||||
|
||||
// Single useEffect for all redirect logic
|
||||
useEffect(() => {
|
||||
if (isLoading) return;
|
||||
|
||||
if (!isAuthorized) {
|
||||
if (!isAuthorized && !isRemoteProxy) {
|
||||
if (token) {
|
||||
clearTokenCookies();
|
||||
}
|
||||
router.replace(`${getProxyBaseUrl()}/ui/login`);
|
||||
}
|
||||
}, [isLoading, isAuthorized, token, router]);
|
||||
}, [isLoading, isAuthorized, isRemoteProxy, token, router]);
|
||||
|
||||
// For remote proxies, use the stored API key as the access token
|
||||
const effectiveAccessToken = isRemoteProxy
|
||||
? activeConnection?.apiKey ?? null
|
||||
: decoded?.key ?? null;
|
||||
|
||||
return {
|
||||
isLoading,
|
||||
isAuthorized,
|
||||
token: isAuthorized ? token : null,
|
||||
accessToken: decoded?.key ?? null,
|
||||
userId: decoded?.user_id ?? null,
|
||||
userEmail: decoded?.user_email ?? null,
|
||||
userRole: formatUserRole(decoded?.user_role),
|
||||
accessToken: effectiveAccessToken,
|
||||
userId: isRemoteProxy ? (remoteUserInfo.user_id ?? null) : (decoded?.user_id ?? null),
|
||||
userEmail: isRemoteProxy ? (remoteUserInfo.user_email ?? null) : (decoded?.user_email ?? null),
|
||||
userRole: isRemoteProxy
|
||||
? formatUserRole(remoteUserInfo.user_role ?? "proxy_admin")
|
||||
: formatUserRole(decoded?.user_role),
|
||||
premiumUser: decoded?.premium_user ?? null,
|
||||
disabledPersonalKeyCreation: decoded?.disabled_non_admin_personal_key_creation ?? null,
|
||||
showSSOBanner: decoded?.login_method === "username_password",
|
||||
showSSOBanner: isRemoteProxy ? false : decoded?.login_method === "username_password",
|
||||
};
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@
|
|||
import React, { Suspense, useEffect, useState } from "react";
|
||||
import Navbar from "@/components/navbar";
|
||||
import { ThemeProvider } from "@/contexts/ThemeContext";
|
||||
import { ProxyConnectionProvider } from "@/contexts/ProxyConnectionContext";
|
||||
import Sidebar2 from "@/app/(dashboard)/components/Sidebar2";
|
||||
import useAuthorized from "@/app/(dashboard)/hooks/useAuthorized";
|
||||
import { useRouter, useSearchParams } from "next/navigation";
|
||||
|
|
@ -77,7 +78,9 @@ function LayoutContent({ children }: { children: React.ReactNode }) {
|
|||
export default function Layout({ children }: { children: React.ReactNode }) {
|
||||
return (
|
||||
<Suspense fallback={<div className="flex items-center justify-center min-h-screen">Loading...</div>}>
|
||||
<LayoutContent>{children}</LayoutContent>
|
||||
<ProxyConnectionProvider>
|
||||
<LayoutContent>{children}</LayoutContent>
|
||||
</ProxyConnectionProvider>
|
||||
</Suspense>
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,229 @@
|
|||
import { useProxyConnection, ProxyConnection } from "@/contexts/ProxyConnectionContext";
|
||||
import {
|
||||
CheckCircleFilled,
|
||||
CloudServerOutlined,
|
||||
DeleteOutlined,
|
||||
EditOutlined,
|
||||
PlusOutlined,
|
||||
} from "@ant-design/icons";
|
||||
import { Button, Form, Input, List, Modal, Space, Tag, Typography, message } from "antd";
|
||||
import React, { useState } from "react";
|
||||
|
||||
const { Text } = Typography;
|
||||
|
||||
interface ManageProxiesModalProps {
|
||||
open: boolean;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
interface ConnectionForm {
|
||||
name: string;
|
||||
url: string;
|
||||
apiKey: string;
|
||||
}
|
||||
|
||||
const ManageProxiesModal: React.FC<ManageProxiesModalProps> = ({ open, onClose }) => {
|
||||
const { connections, activeConnection, addConnection, updateConnection, removeConnection, testConnection } =
|
||||
useProxyConnection();
|
||||
|
||||
const [form] = Form.useForm<ConnectionForm>();
|
||||
const [editingId, setEditingId] = useState<string | null>(null);
|
||||
const [showForm, setShowForm] = useState(false);
|
||||
const [testing, setTesting] = useState(false);
|
||||
const [testResult, setTestResult] = useState<{ ok: boolean; message: string } | null>(null);
|
||||
|
||||
const resetForm = () => {
|
||||
form.resetFields();
|
||||
setEditingId(null);
|
||||
setShowForm(false);
|
||||
setTestResult(null);
|
||||
};
|
||||
|
||||
const handleTest = async () => {
|
||||
try {
|
||||
const values = await form.validateFields(["url", "apiKey"]);
|
||||
setTesting(true);
|
||||
setTestResult(null);
|
||||
const result = await testConnection(values.url, values.apiKey);
|
||||
if (result.ok) {
|
||||
setTestResult({ ok: true, message: `Connected successfully (v${result.version || "unknown"})` });
|
||||
} else {
|
||||
setTestResult({ ok: false, message: result.error || "Connection failed" });
|
||||
}
|
||||
} catch {
|
||||
// form validation error
|
||||
} finally {
|
||||
setTesting(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleSave = async () => {
|
||||
try {
|
||||
const values = await form.validateFields();
|
||||
if (editingId) {
|
||||
updateConnection(editingId, values);
|
||||
message.success("Connection updated");
|
||||
} else {
|
||||
addConnection(values);
|
||||
message.success("Connection added");
|
||||
}
|
||||
resetForm();
|
||||
} catch {
|
||||
// form validation error
|
||||
}
|
||||
};
|
||||
|
||||
const handleEdit = (conn: ProxyConnection) => {
|
||||
setEditingId(conn.id);
|
||||
setShowForm(true);
|
||||
setTestResult(null);
|
||||
form.setFieldsValue({
|
||||
name: conn.name,
|
||||
url: conn.url,
|
||||
apiKey: conn.apiKey,
|
||||
});
|
||||
};
|
||||
|
||||
const handleDelete = (conn: ProxyConnection) => {
|
||||
Modal.confirm({
|
||||
title: "Remove Connection",
|
||||
content: `Are you sure you want to remove "${conn.name}"?`,
|
||||
okText: "Remove",
|
||||
okType: "danger",
|
||||
onOk: () => {
|
||||
removeConnection(conn.id);
|
||||
message.success("Connection removed");
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
return (
|
||||
<Modal
|
||||
title={
|
||||
<Space>
|
||||
<CloudServerOutlined />
|
||||
Manage Proxy Connections
|
||||
</Space>
|
||||
}
|
||||
open={open}
|
||||
onCancel={() => {
|
||||
resetForm();
|
||||
onClose();
|
||||
}}
|
||||
footer={null}
|
||||
width={600}
|
||||
>
|
||||
<List
|
||||
dataSource={connections}
|
||||
renderItem={(conn) => (
|
||||
<List.Item
|
||||
actions={
|
||||
conn.isDefault
|
||||
? [
|
||||
<Tag key="default" color={conn.id === activeConnection?.id ? "green" : "default"}>
|
||||
{conn.id === activeConnection?.id ? "Active" : "Local"}
|
||||
</Tag>,
|
||||
]
|
||||
: [
|
||||
conn.id === activeConnection?.id && (
|
||||
<Tag key="active" color="green">
|
||||
Active
|
||||
</Tag>
|
||||
),
|
||||
<Button key="edit" type="text" size="small" icon={<EditOutlined />} onClick={() => handleEdit(conn)} />,
|
||||
<Button
|
||||
key="delete"
|
||||
type="text"
|
||||
size="small"
|
||||
danger
|
||||
icon={<DeleteOutlined />}
|
||||
onClick={() => handleDelete(conn)}
|
||||
/>,
|
||||
].filter(Boolean)
|
||||
}
|
||||
>
|
||||
<List.Item.Meta
|
||||
title={
|
||||
<Space>
|
||||
{conn.id === activeConnection?.id && <CheckCircleFilled style={{ color: "#52c41a" }} />}
|
||||
{conn.name}
|
||||
</Space>
|
||||
}
|
||||
description={<Text type="secondary" ellipsis>{conn.url}</Text>}
|
||||
/>
|
||||
</List.Item>
|
||||
)}
|
||||
/>
|
||||
|
||||
{showForm ? (
|
||||
<div style={{ marginTop: 16, padding: 16, border: "1px solid #f0f0f0", borderRadius: 8 }}>
|
||||
<Text strong style={{ marginBottom: 12, display: "block" }}>
|
||||
{editingId ? "Edit Connection" : "Add Connection"}
|
||||
</Text>
|
||||
<Form form={form} layout="vertical" size="small">
|
||||
<Form.Item name="name" label="Name" rules={[{ required: true, message: "Enter a name for this connection" }]}>
|
||||
<Input placeholder="e.g. Production US-East" />
|
||||
</Form.Item>
|
||||
<Form.Item
|
||||
name="url"
|
||||
label="Proxy URL"
|
||||
rules={[
|
||||
{ required: true, message: "Enter the proxy URL" },
|
||||
{ type: "url", message: "Enter a valid URL" },
|
||||
]}
|
||||
>
|
||||
<Input placeholder="e.g. https://litellm-prod.example.com" />
|
||||
</Form.Item>
|
||||
<Form.Item
|
||||
name="apiKey"
|
||||
label="API Key"
|
||||
rules={[{ required: true, message: "Enter an API key for this proxy" }]}
|
||||
tooltip="A master key or virtual key for authenticating with the remote proxy"
|
||||
>
|
||||
<Input.Password placeholder="sk-..." />
|
||||
</Form.Item>
|
||||
|
||||
{testResult && (
|
||||
<div
|
||||
style={{
|
||||
marginBottom: 12,
|
||||
padding: 8,
|
||||
borderRadius: 4,
|
||||
backgroundColor: testResult.ok ? "#f6ffed" : "#fff2f0",
|
||||
border: `1px solid ${testResult.ok ? "#b7eb8f" : "#ffccc7"}`,
|
||||
}}
|
||||
>
|
||||
<Text type={testResult.ok ? "success" : "danger"}>{testResult.message}</Text>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<Space>
|
||||
<Button onClick={handleTest} loading={testing}>
|
||||
Test Connection
|
||||
</Button>
|
||||
<Button type="primary" onClick={handleSave}>
|
||||
{editingId ? "Update" : "Add"}
|
||||
</Button>
|
||||
<Button onClick={resetForm}>Cancel</Button>
|
||||
</Space>
|
||||
</Form>
|
||||
</div>
|
||||
) : (
|
||||
<Button
|
||||
type="dashed"
|
||||
icon={<PlusOutlined />}
|
||||
onClick={() => {
|
||||
setShowForm(true);
|
||||
setEditingId(null);
|
||||
setTestResult(null);
|
||||
}}
|
||||
style={{ marginTop: 16, width: "100%" }}
|
||||
>
|
||||
Add Connection
|
||||
</Button>
|
||||
)}
|
||||
</Modal>
|
||||
);
|
||||
};
|
||||
|
||||
export default ManageProxiesModal;
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
import { useProxyConnection } from "@/contexts/ProxyConnectionContext";
|
||||
import { CloudServerOutlined, CheckCircleFilled, DownOutlined, SettingOutlined } from "@ant-design/icons";
|
||||
import type { MenuProps } from "antd";
|
||||
import { Button, Divider, Dropdown, Space, Tag, Typography } from "antd";
|
||||
import React from "react";
|
||||
|
||||
const { Text } = Typography;
|
||||
|
||||
interface ProxySwitcherProps {
|
||||
onManageClick: () => void;
|
||||
}
|
||||
|
||||
const ProxySwitcher: React.FC<ProxySwitcherProps> = ({ onManageClick }) => {
|
||||
const { connections, activeConnection, switchConnection } = useProxyConnection();
|
||||
|
||||
// Only show when there are multiple connections configured
|
||||
if (connections.length <= 1) return null;
|
||||
|
||||
const items: MenuProps["items"] = [
|
||||
...connections.map((conn) => ({
|
||||
key: conn.id,
|
||||
label: (
|
||||
<Space>
|
||||
{conn.id === activeConnection?.id && <CheckCircleFilled style={{ color: "#52c41a" }} />}
|
||||
<span>{conn.name}</span>
|
||||
{conn.isDefault && <Tag color="blue">Local</Tag>}
|
||||
</Space>
|
||||
),
|
||||
onClick: () => {
|
||||
if (conn.id !== activeConnection?.id) {
|
||||
switchConnection(conn.id);
|
||||
}
|
||||
},
|
||||
})),
|
||||
{ type: "divider" as const },
|
||||
{
|
||||
key: "manage",
|
||||
icon: <SettingOutlined />,
|
||||
label: "Manage Connections",
|
||||
onClick: onManageClick,
|
||||
},
|
||||
];
|
||||
|
||||
return (
|
||||
<Dropdown menu={{ items }} trigger={["click"]}>
|
||||
<Button type="text" size="small">
|
||||
<Space>
|
||||
<CloudServerOutlined />
|
||||
<Text ellipsis style={{ maxWidth: 150 }}>
|
||||
{activeConnection?.name || "Default"}
|
||||
</Text>
|
||||
{activeConnection && !activeConnection.isDefault && <Tag color="orange">Remote</Tag>}
|
||||
<DownOutlined style={{ fontSize: 10 }} />
|
||||
</Space>
|
||||
</Button>
|
||||
</Dropdown>
|
||||
);
|
||||
};
|
||||
|
||||
export default ProxySwitcher;
|
||||
|
|
@ -9,6 +9,8 @@ import Link from "next/link";
|
|||
import React, { useEffect, useState } from "react";
|
||||
import { BlogDropdown } from "./Navbar/BlogDropdown/BlogDropdown";
|
||||
import { CommunityEngagementButtons } from "./Navbar/CommunityEngagementButtons/CommunityEngagementButtons";
|
||||
import ManageProxiesModal from "./Navbar/ProxySwitcher/ManageProxiesModal";
|
||||
import ProxySwitcher from "./Navbar/ProxySwitcher/ProxySwitcher";
|
||||
import UserDropdown from "./Navbar/UserDropdown/UserDropdown";
|
||||
|
||||
interface NavbarProps {
|
||||
|
|
@ -42,6 +44,7 @@ const Navbar: React.FC<NavbarProps> = ({
|
|||
}) => {
|
||||
const baseUrl = getProxyBaseUrl();
|
||||
const [logoutUrl, setLogoutUrl] = useState("");
|
||||
const [showManageProxies, setShowManageProxies] = useState(false);
|
||||
const { logoUrl } = useTheme();
|
||||
const { data: healthData } = useHealthReadiness();
|
||||
const version = healthData?.litellm_version;
|
||||
|
|
@ -73,6 +76,7 @@ const Navbar: React.FC<NavbarProps> = ({
|
|||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<nav className="bg-white border-b border-gray-200 sticky top-0 z-10">
|
||||
<div className="w-full">
|
||||
<div className="flex items-center h-14 px-4">
|
||||
|
|
@ -122,6 +126,9 @@ const Navbar: React.FC<NavbarProps> = ({
|
|||
)}
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center ml-4">
|
||||
<ProxySwitcher onManageClick={() => setShowManageProxies(true)} />
|
||||
</div>
|
||||
{/* Right side nav items */}
|
||||
<div className="flex items-center space-x-5 ml-auto">
|
||||
<CommunityEngagementButtons />
|
||||
|
|
@ -146,6 +153,8 @@ const Navbar: React.FC<NavbarProps> = ({
|
|||
</div>
|
||||
</div>
|
||||
</nav>
|
||||
<ManageProxiesModal open={showManageProxies} onClose={() => setShowManageProxies(false)} />
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -121,6 +121,10 @@ const updateServerRootPath = (receivedServerRootPath: string) => {
|
|||
serverRootPath = receivedServerRootPath;
|
||||
};
|
||||
|
||||
export function setProxyBaseUrl(url: string | null) {
|
||||
proxyBaseUrl = url;
|
||||
}
|
||||
|
||||
export const getProxyBaseUrl = (): string => {
|
||||
if (proxyBaseUrl) {
|
||||
return proxyBaseUrl;
|
||||
|
|
|
|||
233
ui/litellm-dashboard/src/contexts/ProxyConnectionContext.tsx
Normal file
233
ui/litellm-dashboard/src/contexts/ProxyConnectionContext.tsx
Normal file
|
|
@ -0,0 +1,233 @@
|
|||
"use client";
|
||||
|
||||
import React, { createContext, useContext, useState, useEffect, useCallback, ReactNode } from "react";
|
||||
import { getProxyBaseUrl, setProxyBaseUrl, getGlobalLitellmHeaderName } from "@/components/networking";
|
||||
import { getLocalStorageItem, setLocalStorageItem } from "@/utils/localStorageUtils";
|
||||
|
||||
const CONNECTIONS_KEY = "litellm_proxy_connections";
|
||||
const ACTIVE_ID_KEY = "litellm_active_connection_id";
|
||||
|
||||
export interface ProxyConnection {
|
||||
id: string;
|
||||
name: string;
|
||||
url: string;
|
||||
apiKey: string;
|
||||
isDefault: boolean;
|
||||
}
|
||||
|
||||
interface TestConnectionResult {
|
||||
ok: boolean;
|
||||
version?: string;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
interface ProxyConnectionContextType {
|
||||
connections: ProxyConnection[];
|
||||
activeConnection: ProxyConnection | null;
|
||||
addConnection: (conn: Omit<ProxyConnection, "id" | "isDefault">) => void;
|
||||
updateConnection: (id: string, updates: Partial<Pick<ProxyConnection, "name" | "url" | "apiKey">>) => void;
|
||||
removeConnection: (id: string) => void;
|
||||
switchConnection: (id: string) => void;
|
||||
testConnection: (url: string, apiKey: string) => Promise<TestConnectionResult>;
|
||||
isRemoteProxy: boolean;
|
||||
}
|
||||
|
||||
const defaultContextValue: ProxyConnectionContextType = {
|
||||
connections: [],
|
||||
activeConnection: null,
|
||||
addConnection: () => {},
|
||||
updateConnection: () => {},
|
||||
removeConnection: () => {},
|
||||
switchConnection: () => {},
|
||||
testConnection: async () => ({ ok: false, error: "No provider" }),
|
||||
isRemoteProxy: false,
|
||||
};
|
||||
|
||||
const ProxyConnectionContext = createContext<ProxyConnectionContextType>(defaultContextValue);
|
||||
|
||||
export const useProxyConnection = () => {
|
||||
return useContext(ProxyConnectionContext);
|
||||
};
|
||||
|
||||
function generateId(): string {
|
||||
if (typeof crypto !== "undefined" && crypto.randomUUID) {
|
||||
return crypto.randomUUID();
|
||||
}
|
||||
return Math.random().toString(36).substring(2) + Date.now().toString(36);
|
||||
}
|
||||
|
||||
function loadConnections(): ProxyConnection[] {
|
||||
const raw = getLocalStorageItem(CONNECTIONS_KEY);
|
||||
if (!raw) return [];
|
||||
try {
|
||||
return JSON.parse(raw);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function saveConnections(connections: ProxyConnection[]) {
|
||||
setLocalStorageItem(CONNECTIONS_KEY, JSON.stringify(connections));
|
||||
}
|
||||
|
||||
function loadActiveId(): string | null {
|
||||
return getLocalStorageItem(ACTIVE_ID_KEY);
|
||||
}
|
||||
|
||||
function saveActiveId(id: string) {
|
||||
setLocalStorageItem(ACTIVE_ID_KEY, id);
|
||||
}
|
||||
|
||||
function ensureDefaultConnection(connections: ProxyConnection[]): ProxyConnection[] {
|
||||
const hasDefault = connections.some((c) => c.isDefault);
|
||||
if (hasDefault) return connections;
|
||||
|
||||
const defaultUrl = getProxyBaseUrl();
|
||||
const defaultConn: ProxyConnection = {
|
||||
id: "default",
|
||||
name: "Default",
|
||||
url: defaultUrl,
|
||||
apiKey: "",
|
||||
isDefault: true,
|
||||
};
|
||||
return [defaultConn, ...connections];
|
||||
}
|
||||
|
||||
interface ProxyConnectionProviderProps {
|
||||
children: ReactNode;
|
||||
}
|
||||
|
||||
export const ProxyConnectionProvider: React.FC<ProxyConnectionProviderProps> = ({ children }) => {
|
||||
const [connections, setConnections] = useState<ProxyConnection[]>(() => {
|
||||
const loaded = loadConnections();
|
||||
return ensureDefaultConnection(loaded);
|
||||
});
|
||||
|
||||
const [activeId, setActiveId] = useState<string>(() => {
|
||||
const savedId = loadActiveId();
|
||||
if (savedId) return savedId;
|
||||
const defaultConn = connections.find((c) => c.isDefault);
|
||||
return defaultConn?.id ?? "default";
|
||||
});
|
||||
|
||||
const activeConnection = connections.find((c) => c.id === activeId) ?? connections.find((c) => c.isDefault) ?? null;
|
||||
const isRemoteProxy = activeConnection !== null && !activeConnection.isDefault;
|
||||
|
||||
// On mount, set the proxyBaseUrl for non-default connections
|
||||
useEffect(() => {
|
||||
if (activeConnection && !activeConnection.isDefault) {
|
||||
setProxyBaseUrl(activeConnection.url);
|
||||
}
|
||||
}, []); // eslint-disable-line react-hooks/exhaustive-deps
|
||||
|
||||
// Persist connections to localStorage whenever they change
|
||||
useEffect(() => {
|
||||
saveConnections(connections);
|
||||
}, [connections]);
|
||||
|
||||
const addConnection = useCallback((conn: Omit<ProxyConnection, "id" | "isDefault">) => {
|
||||
const newConn: ProxyConnection = {
|
||||
...conn,
|
||||
id: generateId(),
|
||||
isDefault: false,
|
||||
};
|
||||
setConnections((prev) => {
|
||||
const updated = [...prev, newConn];
|
||||
saveConnections(updated);
|
||||
return updated;
|
||||
});
|
||||
}, []);
|
||||
|
||||
const updateConnection = useCallback((id: string, updates: Partial<Pick<ProxyConnection, "name" | "url" | "apiKey">>) => {
|
||||
setConnections((prev) => {
|
||||
const updated = prev.map((c) => (c.id === id ? { ...c, ...updates } : c));
|
||||
saveConnections(updated);
|
||||
return updated;
|
||||
});
|
||||
}, []);
|
||||
|
||||
const removeConnection = useCallback((id: string) => {
|
||||
setConnections((prev) => {
|
||||
const conn = prev.find((c) => c.id === id);
|
||||
if (!conn || conn.isDefault) return prev;
|
||||
const updated = prev.filter((c) => c.id !== id);
|
||||
saveConnections(updated);
|
||||
return updated;
|
||||
});
|
||||
|
||||
// If removing the active connection, switch back to default
|
||||
if (id === activeId) {
|
||||
const defaultConn = connections.find((c) => c.isDefault);
|
||||
if (defaultConn) {
|
||||
saveActiveId(defaultConn.id);
|
||||
// Clear the override so getProxyBaseUrl() falls back to window.location.origin
|
||||
setProxyBaseUrl(null);
|
||||
window.location.reload();
|
||||
}
|
||||
}
|
||||
}, [activeId, connections]);
|
||||
|
||||
const switchConnection = useCallback((id: string) => {
|
||||
if (id === activeId) return;
|
||||
const target = connections.find((c) => c.id === id);
|
||||
if (!target) return;
|
||||
|
||||
saveActiveId(target.id);
|
||||
|
||||
if (target.isDefault) {
|
||||
// Switching back to default — clear the override so getProxyBaseUrl() falls back
|
||||
setProxyBaseUrl(null);
|
||||
} else {
|
||||
setProxyBaseUrl(target.url);
|
||||
}
|
||||
|
||||
window.location.reload();
|
||||
}, [activeId, connections]);
|
||||
|
||||
const testConnection = useCallback(async (url: string, apiKey: string): Promise<TestConnectionResult> => {
|
||||
try {
|
||||
const cleanUrl = url.replace(/\/+$/, "");
|
||||
const response = await fetch(`${cleanUrl}/health/readiness`, {
|
||||
method: "GET",
|
||||
headers: {
|
||||
[getGlobalLitellmHeaderName()]: `Bearer ${apiKey}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json().catch(() => ({}));
|
||||
return { ok: false, error: errorData?.error || `HTTP ${response.status}` };
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
return { ok: true, version: data?.litellm_version };
|
||||
} catch (error: unknown) {
|
||||
const message = error instanceof Error ? error.message : "Connection failed";
|
||||
if (message.includes("Failed to fetch") || message.includes("NetworkError")) {
|
||||
return {
|
||||
ok: false,
|
||||
error: `Connection failed. Ensure the proxy is running and CORS is configured to allow requests from ${window.location.origin}`,
|
||||
};
|
||||
}
|
||||
return { ok: false, error: message };
|
||||
}
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<ProxyConnectionContext.Provider
|
||||
value={{
|
||||
connections,
|
||||
activeConnection,
|
||||
addConnection,
|
||||
updateConnection,
|
||||
removeConnection,
|
||||
switchConnection,
|
||||
testConnection,
|
||||
isRemoteProxy,
|
||||
}}
|
||||
>
|
||||
{children}
|
||||
</ProxyConnectionContext.Provider>
|
||||
);
|
||||
};
|
||||
Loading…
Add table
Reference in a new issue