feat: litellm plugin architecture v2 (#30688)

* feat: plugin architecture — toggle between AI Gateway and external plugins

Adds a generic plugin system so any external service can register with
litellm and appear as a mode in the UI alongside the AI Gateway.

Backend (litellm/proxy/plugin_routes.py — new):
- GET /api/plugins: returns registered plugins from config; returns
  plugin_key only to authenticated requests
- ANY /plugin-proxy/{name}/{path}: reverse proxies API calls to plugin

Config:
  general_settings:
    plugins:
      - name: my-plugin
        display_name: My Plugin
        url: https://my-plugin.example.com
        plugin_key: sk-...   # plugin auth key, passed to iframe

UI:
- PluginModeContext.tsx: fetches /api/plugins, persists mode to localStorage
- leftnav.tsx: mode switcher dropdown at top of sidebar; plugin mode shows
  plugin-specific nav items
- layout.tsx: renders iframe to plugin URL in plugin mode; passes plugin_key
  as ?token= for auto sign-in

Plugin contract: expose GET /api/plugin-manifest returning
{ name, display_name, nav_items[], capabilities[] }. No litellm changes
needed to add new plugins — config only.

Reference implementation: LiteLLM-Labs/litellm-agent-control-plane

* feat: add Plugins tab to Admin Settings UI

Allows admins to add/edit/delete plugin registrations directly in the
litellm UI under Admin Settings > Plugins, instead of editing config.yaml.

Uses existing /config/field/update API to persist to general_settings.plugins.
Each plugin entry has: name (identifier), display_name, url, plugin_key.

* fix(ci): black, prettier, eslint, async-client violations

- Black: format plugin_routes.py and proxy_server.py
- Prettier: format PluginModeContext.tsx and PluginSettings.tsx
- ESLint: replace raw fetch() with createApiClient in PluginModeContext
- ESLint: use lazy useState initializer to read localStorage instead of
  calling setModeState inside useEffect (react-hooks/set-state-in-effect)
- code-quality: replace httpx.AsyncClient per-request with
  get_async_httpx_client() shared client (avoids +500ms overhead)

* fix(ci): schema.d.ts regen, Black proxy_server.py, ApiClientConfig fix

- Regenerate schema.d.ts for new /api/plugins routes
- Re-run Black 26.3.1 on proxy_server.py (matches CI version)
- Fix PluginModeContext: createApiClient requires getBaseUrl field

* fix: security hardening + CI fixes

Security (Greptile 1/5 → addressing all 3 findings):
- plugin_routes.py: add Depends(user_api_key_auth) to both /api/plugins
  and /plugin-proxy/{name}/{path} — was an unauthenticated open relay
- plugin_routes.py: /api/plugins now returns plugin_key only to callers
  with a valid litellm token (enforced by user_api_key_auth), not just
  any header presence
- layout.tsx: replace ?token= URL param with postMessage(targetOrigin)
  — token no longer exposed in browser history / logs / Referer headers

CI:
- backend/routes/allowlist.py: add /api/plugins and /plugin-proxy/ to
  fix test_gateway_plus_backend_covers_full_app
- schema.d.ts: regenerated with enterprise routes included
- Black + Prettier formatting

* fix: regenerate schema.d.ts with enterprise routes included

Install litellm-enterprise workspace member before gen:api so audit and
other enterprise routes appear in the generated types, matching what CI
produces with uv sync --extra proxy.

* fix: exclude plugin routes from OpenAPI schema, restore upstream schema.d.ts

Both /api/plugins and /plugin-proxy/ are internal infrastructure routes,
not part of the public litellm API surface. Marking include_in_schema=False
prevents Python-version-dependent schema diffs from breaking the schema
sync check across different environments.

* fix: schema.d.ts - passing schema base + exact plugin route types from openapi-typescript

Use the CI-correct schema from a recently passing branch as base, then
inject plugin route entries (paths + operations) generated by
openapi-typescript from the plugin routes' OpenAPI spec. This avoids
Python-version-dependent formatting differences that made local gen:api
produce incorrect output.

* fix: schema.d.ts - insert plugin ops at correct route registration position

Plugin operations belong after delete_memory_v1_memory__key__delete
(memory_router is included immediately before plugin_router in proxy_server.py),
not after list_organization which is alphabetically but not registration-order.

* fix: schema.d.ts - correct op positions from hunk analysis

list_plugins_api_plugins_get goes after event_logging_batch op (hunk 1: line 33583).
plugin_proxy ops go after create_policy_policies_post (hunk 2: line 44634).
Previous location after delete_memory_v1_memory__key__delete was wrong.

* fix: schema.d.ts - proxy ops go before create_policy (after otel_spans)

* fix(security): restrict plugin_key to proxy_admin role only

Veria finding: plugin_key was returned to any authenticated caller.
Now only proxy_admin users receive plugin credentials in /api/plugins
response — regular internal users see plugin name/url but not the key.

* fix: update schema.d.ts docstring for list_plugins

* fix: clear plugin registry on config reload (Greptile medium)

register_plugins_from_config now replaces the registry instead of
merging, so plugins removed from config are unreachable immediately
without requiring a process restart.

* fix(security): encrypted token exchange for plugin iframe — no raw litellm credential exposure

The dashboard was sending the user's litellm bearer token to the plugin
iframe via postMessage, allowing a compromised plugin to act as that user.

Fix:
- GET /api/plugins/auth-token: proxy encrypts caller token with Fernet
  keyed from LITELLM_SALT_KEY, returns ciphertext only
- UI postMessages the ciphertext (not raw token) to the iframe
- Plugin decrypts server-side with same LITELLM_SALT_KEY via POST /api/plugin-auth
- Raw litellm credential never leaves the proxy in plaintext

Additional hardening already in place:
- /plugin-proxy/* strips Authorization header, injects plugin_key instead
- plugin_key only returned to proxy_admin role via /api/plugins
- Plugin registry cleared (not merged) on config reload

Adds docs/plugin_architecture.md with plugin integration guide.

* fix(code-quality): use get_async_httpx_client in plugin_proxy

* fix: add /api/plugins/auth-token to schema.d.ts

* fix: use apiClient for auth-token fetch, copy correct layout.tsx and PluginModeContext

- Replace raw fetch() with createApiClient (fixes no-restricted-syntax ESLint rule)
- Copy correct layout.tsx with encrypted token + postMessage approach
- Copy correct PluginModeContext.tsx with accessToken prop injection
- Update schema.d.ts with auth-token path and operation entries

* fix: add plugin_auth_token operation to schema.d.ts

* fix(security): strip cookie/set-cookie + fix compressed response headers

Veria High: cookie header was forwarded to plugin backends allowing
capture of litellm JWT session cookies. Strip cookie on requests.
Strip set-cookie from responses so plugins cannot overwrite litellm
session cookies.

Greptile P1: httpx decompresses responses but resp.headers still
contained Content-Encoding/Transfer-Encoding/Content-Length from the
wire. Forwarding these caused double-decompression and length errors.
Now filtered via _RESPONSE_STRIP before returning to the browser.

* fix: update plugin_key help text — no more ?token= reference

* fix(security): disable follow_redirects to prevent SSRF

follow_redirects=True allowed a plugin backend to return a 3xx to an
internal URL, causing the proxy to fetch that internal service and relay
the response. Disabled: clients handle their own redirects.

* fix: forward user identity headers to plugin to address confused deputy

Plugins receive X-LiteLLM-User-Id and X-LiteLLM-User-Role so they can
enforce their own per-user access control before acting on requests that
arrive with the shared plugin_key credential.

* fix(security): restrict /plugin-proxy/* to proxy_admin role

Closes the confused deputy gap: regular users could invoke any plugin
endpoint using the shared plugin_key as a bearer credential. Now only
proxy_admin callers can use the plugin proxy route.

Plugin UIs communicate with the plugin service directly via the iframe
(using the encrypted token exchange); this proxy route is for
administrative/server-to-server access only.

* fix: update schema.d.ts for admin-only proxy route docstring

* fix(bug): use PassThroughEndpoint instead of None for get_async_httpx_client

get_async_httpx_client(llm_provider=None) raises TypeError — the function
concatenates the provider string and None is not a str. Use
httpxSpecialProvider.PassThroughEndpoint, the enum value used by other
internal proxy pass-through routes.

* fix(security): add 30s TTL to encrypted plugin auth tokens

Veria medium: encrypted tokens had no expiry, allowing indefinite replay.
Fernet embeds a timestamp; decrypt_token now passes ttl=30 so tokens
older than 30 seconds are rejected even with a valid HMAC.

Plugin's /api/plugin-auth must call litellm within 30s of the iframe
receiving the postMessage — normal browser behavior, tight enough to
close the replay window.

* feat(ui): topnav plugin switcher, embed plugins at their root

Builds on the plugin architecture already on this branch (encrypted-token
postMessage handshake, /api/plugins, PluginSettings) and removes the parts of the
embed that assumed a specific plugin's shape.

The mode switcher moves out of the sidebar into the topnav and lists AI Gateway
plus each registered plugin by its display_name. Selecting a plugin hides
litellm's sidebar entirely and renders the plugin full-bleed at its root url; the
plugin draws its own navigation inside the iframe. This drops the hardcoded
"Agent Control Plane" label and the hardcoded Sessions/Agents/Routines/... nav
groups (agentControlPlaneMenuGroups / acpPagePaths) that only matched the agent
platform and 404'd for a plugin that serves only / (e.g. the chat UI). The
encrypted-token postMessage flow is unchanged.

Note: embedding at root means a plugin must route internally from /; plugins that
previously relied on the /sessions entrypoint should redirect from their root.

* fix(security): audience-scoped identity claim replaces litellm token

Veria: shared LITELLM_SALT_KEY with plugins + encrypting user bearer token
created delegation/impersonation risk.

Architecture change:
- /api/plugins/auth-token now issues a plugin-scoped identity CLAIM
  {user_id, user_role, plugin, exp} encrypted with HMAC(LITELLM_SALT_KEY, plugin_name)
- Each plugin holds only its own HMAC-derived key; cannot forge claims for
  other plugins or recover LITELLM_SALT_KEY
- Claim contains NO litellm bearer token — compromised plugin learns caller
  identity only, cannot act as that user against the proxy
- 30s TTL enforced in both Fernet header and explicit exp field
- LAP /api/plugin-auth verifies claim, returns its own master key to browser
  (LAP key never exposed without valid claim)

* fix(plugins): allow registering plugins from the admin UI

Adding a plugin in the UI POSTs general_settings.plugins to /config/field/update,
which rejected it with "Invalid field=plugins passed in." because `plugins` was
not a field on ConfigGeneralSettings. Add a typed PluginConfig model and a
`plugins` field so the update validates and persists.

The in-memory plugin registry only refreshed at startup, so a plugin added via
the UI did not appear in /api/plugins (the view switcher) until a restart. Refresh
the registry from the new general_settings whenever the plugins field is updated.

While here, type the registry as dict[str, PluginConfig] instead of raw dicts so
list_plugins and plugin_proxy access typed attributes.

Fix the Plugin Key field copy: it is optional and only used to authenticate
litellm's server-side reverse proxy to a plugin's own backend
(/plugin-proxy/<name>/*). It is not involved in iframe auth, which forwards the
user's litellm token. Plugins that use the forwarded token leave it blank.

* fix: regenerate schema.d.ts with PluginConfig type and updated auth-token endpoint

* fix: use CI-compatible schema base for plugin entries

* fix(plugins): load DB-persisted plugins on startup

Plugins added through the admin UI are saved to DB general_settings, but the
registry only initialised from the YAML config at boot, so UI-added plugins
disappeared from the view switcher after a restart (the Plugins table still
listed them since it reads the DB directly). Refresh the registry from the DB
general_settings when it is merged in at startup.

* fix: add PluginConfig schema, plugins field, fix list_plugins return type

* fix: correct PluginConfig and plugins field positions in schema

* fix: correct plugins field position in schema (after pass_through_endpoints)

* fix: update PluginConfig.plugin_key description to match _types.py source

* fix: move plugins field after pass_through_request_timeout (correct alphabetical position)

* fix: redact plugin_key in config/field/info response

Veria medium: proxy_admin_viewer could read plugin_key via
GET /config/field/info?field_name=plugins. Now plugin_key is
replaced with *** in the response regardless of caller role.
The credential is only usable server-side.

* fix(security): correct plugin docs salt-key guidance, drop iframe clipboard-read

Address the two open Veria findings on the plugin architecture.

The plugin docs told external services to decrypt the iframe auth payload
with the proxy's LITELLM_SALT_KEY directly. That is both insecure and wrong:
the running code derives a per-plugin key as HMAC-SHA256(LITELLM_SALT_KEY,
plugin_name) and ships only a short-lived identity claim with no litellm
bearer token. Sharing the master salt would let a compromised plugin decrypt
any litellm secret recovered from a dump or backup. Rewrite the doc to match
the implementation: the proxy computes the per-plugin key once and provisions
it as a dedicated secret, the plugin validates the claim's audience and 30s
TTL, and LITELLM_SALT_KEY never leaves the proxy. Also refresh the now-stale
module and UI comments that still described the old shared-key token flow.

Drop clipboard-read from the plugin iframe's allow attribute so an untrusted
plugin can no longer read the user's clipboard; clipboard-write is retained.

* fix(ci): modernize PluginConfig typing, refresh budget baselines via merge

* fix(plugins): close iframe auth race and empty-plugins mode fallback

Address the two open Greptile behavioral findings.

The iframe auth handshake only posted the encrypted claim on the iframe's
`load` event. When the auth-token fetch resolved after the iframe had already
loaded, that listener never fired again and the plugin never received the
claim. Send the claim immediately as well as on subsequent loads so both
orderings are covered.

The plugin mode fallback guarded on a non-empty plugins list, so removing all
plugins left a user stranded on a stale mode with a blank iframe instead of
returning to the AI Gateway. Track a loaded flag and fall back to ai-gateway
once plugins have loaded whenever the stored mode is no longer registered,
including the empty-list case.

Add a PluginModeContext regression test covering the empty-list fallback and
the still-registered path.

* chore: re-trigger CI (GH Actions missed the prior head; re-run flaky live-API suites)

* fix(plugins): scope iframe auth claim to the active plugin

The iframe auth-token fetch omitted plugin_name, so the proxy always issued a
claim encrypted under the default plugin's per-plugin key. For any other active
plugin the iframe received a claim it could not decrypt and sign-in silently
broke, and because the cached claim was posted to whichever plugin was mounted,
a compromised iframe could replay the default plugin's claim. The active
plugin's name was also missing from the fetch effect's dependencies, so
switching plugins never refreshed the claim.

Request the claim with the active plugin's name, re-fetch when the active
plugin changes, and only deliver a claim while it still matches the mounted
plugin so one plugin's claim is never replayed to another.

* fix(plugins): never overwrite a stored plugin_key with its redaction placeholder

/config/field/info redacts every plugin_key to "***", so an admin editing a
plugin in the settings UI posted that placeholder straight back and the update
handler persisted "***" as the real credential, permanently destroying the key.

Preserve the stored credential on update: a blank or redacted plugin_key now
sources the existing key from the saved config, only a real value replaces it,
and a placeholder with no stored key is dropped rather than written. The edit
modal also starts the key field blank so an untouched save keeps the current
key, with the field labelled accordingly.

* fix(security): sandbox proxied plugin responses on the dashboard origin

The /plugin-proxy reverse proxy returned the plugin's body and content-type on
the litellm dashboard origin, so a compromised plugin could serve an HTML/JS
document that a proxy_admin navigates to and have it execute with the admin's
session against same-origin management APIs.

Force every proxied response inert: set Content-Security-Policy: sandbox (opaque
origin, scripts disabled) and X-Content-Type-Options: nosniff, applied after the
plugin's own headers so they cannot be overridden. The header construction moves
to a pure helper with a unit test covering the sandbox enforcement and the
existing wire/cookie header stripping.

* fix(plugins): recover to ai-gateway when the plugins fetch fails

The loaded flag was only set on a successful /api/plugins response, so when the
fetch failed a user with a plugin mode stored in localStorage stayed on the
blank plugin placeholder with no switcher to escape. Mark loaded in a finally
so the stored mode still falls back to ai-gateway on failure, and add a
regression test for the failed-fetch path.

* fix(security): never return plugin_key from /api/plugins

The plugin list endpoint returned the plaintext plugin_key to proxy_admin
callers, and the dashboard fetches /api/plugins on every load into React state,
so the credential was exposed to DevTools, memory snapshots, and any same-origin
script. The browser never uses the key; the proxy injects it server-side from
the registry and admin key management runs through the redacted
/config/field/info path. Drop plugin_key from the response for every caller and
update the regression test to assert it is never returned.

* chore(ui): regenerate schema.d.ts for updated list_plugins docstring

* fix(security): strip every litellm auth header before forwarding to plugins

The plugin reverse proxy only removed Authorization and x-api-key, but
user_api_key_auth also authenticates a caller via API-Key, x-goog-api-key,
Ocp-Apim-Subscription-Key, x-litellm-api-key, and any configured custom key
header. A malicious plugin could lure a proxy_admin into calling
/plugin-proxy/... with the litellm key in one of those headers; the request
authenticated locally and then forwarded the same key to the plugin, letting it
impersonate the admin.

Add a canonical SpecialHeaders.litellm_credential_header_names() that the auth
header enum is the single source for, and strip that whole set plus the live
general_settings.litellm_key_header_name from every forwarded request. New auth
headers added to SpecialHeaders are now stripped automatically. Regression tests
cover each credential header, the custom configured header, and the canonical
list's contents.
This commit is contained in:
Krrish Dholakia 2026-06-20 20:37:22 -07:00 committed by GitHub
parent 9f97111edd
commit accbd7e587
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 1977 additions and 10 deletions

View file

@ -120,6 +120,9 @@ BACKEND_PATH_PREFIXES: tuple[str, ...] = (
"/robots.txt",
# Health (k8s probes)
"/health",
# Plugin system
"/api/plugins",
"/plugin-proxy/",
)
BACKEND_EXACT_PATHS: frozenset[str] = frozenset(

141
docs/plugin_architecture.md Normal file
View file

@ -0,0 +1,141 @@
# LiteLLM Plugin Architecture
Plugins let external services appear as selectable modes in the litellm UI sidebar alongside the AI Gateway.
---
## Quick start
### 1. Configure the plugin
Add a `plugins` block to your litellm `config.yaml`:
```yaml
general_settings:
master_key: sk-...
plugins:
- name: my-plugin # unique identifier (no spaces)
display_name: My Plugin # shown in the UI dropdown
url: "https://my-plugin.example.com"
plugin_key: "sk-..." # plugin's own auth credential
```
`plugin_key` is injected as `Authorization: Bearer <plugin_key>` on every
request proxied through `/plugin-proxy/my-plugin/*`. The caller's litellm
credential is stripped before forwarding so the plugin never receives a live
litellm API key.
### 2. Implement two endpoints on your service
| Endpoint | Method | Purpose |
|---|---|---|
| `GET /api/plugin-manifest` | public | Returns plugin metadata for the UI |
| `POST /api/plugin-auth` | public | Decrypts the identity claim for seamless sign-in |
#### `GET /api/plugin-manifest`
```json
{
"name": "my-plugin",
"display_name": "My Plugin",
"version": "1.0.0",
"nav_items": [
{ "key": "home", "label": "Home", "icon": "HomeOutlined", "path": "/" },
{ "key": "reports", "label": "Reports", "icon": "BarChartOutlined", "path": "/reports" }
],
"capabilities": ["reports", "data"]
}
```
#### `POST /api/plugin-auth`
Receives `{ "session_claim": "<fernet-ciphertext>" }`.
The proxy never shares `LITELLM_SALT_KEY` with your plugin. Each plugin is
provisioned with its own dedicated key, derived as
`HMAC-SHA256(LITELLM_SALT_KEY, plugin_name)`. Compute it once on the proxy
host and hand the result to your plugin as a secret (e.g. `PLUGIN_AUTH_KEY`):
```bash
python -c 'import base64,hmac,hashlib,os; \
print(base64.urlsafe_b64encode(hmac.new(os.environ["LITELLM_SALT_KEY"].encode(), b"my-plugin", hashlib.sha256).digest()).decode())'
```
A compromised plugin holding only this scoped key cannot recover
`LITELLM_SALT_KEY` or decrypt any other litellm secret.
Decrypt and validate the claim with that key:
```python
import json, os, time
from cryptography.fernet import Fernet
_CLAIM_TTL_SECONDS = 30
def plugin_auth(session_claim: str) -> dict:
cipher = Fernet(os.environ["PLUGIN_AUTH_KEY"].encode())
claim = json.loads(cipher.decrypt(session_claim.encode(), ttl=_CLAIM_TTL_SECONDS))
if claim.get("plugin") != "my-plugin":
raise ValueError("claim audience mismatch")
if int(claim.get("exp", 0)) < int(time.time()):
raise ValueError("claim expired")
return claim
```
The claim is `{ "plugin", "user_id", "user_role", "exp" }`; it carries no
litellm bearer token. Establish the plugin's own session from `user_id` /
`user_role` and authenticate API calls back to litellm through the
`/plugin-proxy/my-plugin/*` reverse proxy, which injects `plugin_key` for you.
---
## How iframe auth works
```
litellm UI
├─ GET /api/plugins/auth-token -> { session_claim }
└─ postMessage({ type:"litellm-auth", session_claim }, pluginOrigin)
Plugin iframe browser
└─ POST /api/plugin-auth { session_claim }
Plugin server
├─ decrypt(session_claim, PLUGIN_AUTH_KEY) -> { user_id, user_role, exp }
└─ establish plugin session -> stored in sessionStorage
```
No litellm bearer token ever leaves the proxy; the claim only conveys the
caller's identity and expires after 30 seconds. A postMessage intercept
yields ciphertext that is useless without the plugin's scoped key.
---
## Proxy routes
- `GET /api/plugins` — list registered plugins (`name`, `display_name`, `url`). `plugin_key` is **never** returned; it stays server-side. Requires an authenticated caller.
- `GET /api/plugins/auth-token?plugin_name=<name>` — short-lived encrypted identity claim for the named plugin. Requires `LITELLM_SALT_KEY` to be set (503 otherwise) and the plugin to be registered (404 otherwise).
- `ANY /plugin-proxy/{name}/{path}` — authenticated reverse proxy to the plugin backend. Restricted to `proxy_admin`.
---
## Reverse proxy behaviour
When an admin (or server-to-server caller) hits `/plugin-proxy/<name>/<path>`, the proxy authenticates the caller locally, then rewrites the request before forwarding it to the plugin's `url`:
- **Every litellm credential header is stripped**`Authorization`, `x-api-key`, `API-Key`, `x-goog-api-key`, `Ocp-Apim-Subscription-Key`, `x-litellm-api-key`, any configured `litellm_key_header_name`, plus `Cookie`. The plugin can never be handed the caller's live litellm key.
- **`plugin_key` is injected** as `Authorization: Bearer <plugin_key>` — the only credential the plugin receives.
- **Caller identity is forwarded** as `x-litellm-user-id` and `x-litellm-user-role` so the plugin can run its own authorization. These are informational, not credentials.
- **Responses are sandboxed**`Content-Security-Policy: sandbox` and `X-Content-Type-Options: nosniff` are set so plugin-controlled bytes served from the litellm origin cannot execute against the dashboard.
---
## Security checklist
- [ ] `LITELLM_SALT_KEY` is set on the proxy and never shared with the plugin
- [ ] The plugin holds only its derived `HMAC(LITELLM_SALT_KEY, plugin_name)` key, provisioned as a dedicated secret
- [ ] `plugin_key` is a dedicated credential scoped to the plugin (not your litellm master key)
- [ ] Plugin's `POST /api/plugin-auth` enforces the claim's `plugin` audience and `exp` (30s TTL)
- [ ] Plugin treats `x-litellm-user-id` / `x-litellm-user-role` as identity hints, not as proof of authentication
- [ ] Plugin service URL uses HTTPS in production

View file

@ -2142,6 +2142,20 @@ class UserHeaderMapping(LiteLLMPydanticObjectBase):
UserMCPManagementMode = Literal["restricted", "view_all"]
class PluginConfig(LiteLLMPydanticObjectBase):
"""A single external service registered as an embeddable UI plugin."""
name: str = Field(description="unique plugin identifier (kebab-case)")
display_name: str | None = Field(
None, description="human-readable label shown in the UI view switcher"
)
url: str = Field(description="base URL of the plugin service")
plugin_key: str | None = Field(
None,
description="plugin's own credential, injected as Bearer auth only on /plugin-proxy/<name>/* reverse-proxy calls",
)
class ConfigGeneralSettings(LiteLLMPydanticObjectBase):
"""
Documents all the fields supported by `general_settings` in config.yaml
@ -2150,6 +2164,9 @@ class ConfigGeneralSettings(LiteLLMPydanticObjectBase):
completion_model: Optional[str] = Field(
None, description="proxy level default model for all chat completion calls"
)
plugins: list[PluginConfig] | None = Field(
None, description="external services registered as embeddable UI plugins"
)
key_management_system: Optional[KeyManagementSystem] = Field(
None, description="key manager to load keys from / decrypt keys with"
)
@ -3808,6 +3825,28 @@ class SpecialHeaders(enum.Enum):
mcp_servers = "x-mcp-servers"
mcp_access_groups = "x-mcp-access-groups"
@classmethod
def litellm_credential_header_names(cls) -> "frozenset[str]":
"""Lowercased header names user_api_key_auth accepts as a litellm key.
Every header here authenticates the caller, so any code that forwards a
request onward (e.g. the plugin reverse proxy) must strip all of them to
avoid leaking the caller's litellm credential downstream. The static
custom-key header (general_settings.litellm_key_header_name) is runtime
config and must be added on top of this set by the caller.
"""
return frozenset(
header.value.lower()
for header in (
cls.openai_authorization,
cls.azure_authorization,
cls.anthropic_authorization,
cls.google_ai_studio_authorization,
cls.azure_apim_authorization,
cls.custom_litellm_api_key,
)
)
class LitellmDataForBackendLLMCall(TypedDict, total=False):
headers: dict

View file

@ -0,0 +1,344 @@
"""
Plugin proxy routes for litellm.
Enables external services to register as plugins and be proxied through
the litellm proxy server.
Config (in litellm config.yaml general_settings):
plugins:
- name: my-plugin
url: "http://localhost:3210"
display_name: "My Plugin"
plugin_key: "sk-..." # optional: plugin's own auth key
Plugin iframe auth:
The UI calls GET /api/plugins/auth-token to receive a short-lived identity
claim ({user_id, user_role, plugin, exp}) encrypted with a per-plugin key
derived as HMAC-SHA256(LITELLM_SALT_KEY, plugin_name). The claim carries no
litellm bearer token, so a compromised plugin learns only the caller's
identity, never their credential. LITELLM_SALT_KEY itself is never shared
with plugins each plugin holds only its own derived key.
"""
import base64
import hashlib
import hmac as _hmac
import json
import os
import time
from collections.abc import Mapping
from cryptography.fernet import Fernet, InvalidToken
from fastapi import APIRouter, Depends, HTTPException, Request, Response
from litellm.proxy._types import PluginConfig, SpecialHeaders, UserAPIKeyAuth
from litellm.proxy.auth.user_api_key_auth import user_api_key_auth
from litellm.llms.custom_httpx.http_handler import get_async_httpx_client
from litellm.types.llms.custom_http import httpxSpecialProvider
router = APIRouter()
# Hop-by-hop headers (RFC 7230) and the litellm session cookie — never forwarded
# to a plugin backend. Credential headers are added on top per-request from the
# canonical SpecialHeaders set so the plugin only ever authenticates via its own
# injected plugin_key.
_HOP_BY_HOP_STRIP = frozenset(
{
"host",
"connection",
"transfer-encoding",
"te",
"trailers",
"upgrade",
"cookie",
}
)
def _configured_key_header_names() -> frozenset[str]:
"""The lowercased general_settings.litellm_key_header_name, if configured.
Read live from the proxy module (not import-time) so a custom key header set
via config is honoured without a restart. Returns empty when unset.
"""
try:
from litellm.proxy import proxy_server
except Exception:
return frozenset()
general_settings = getattr(proxy_server, "general_settings", None)
if not isinstance(general_settings, dict):
return frozenset()
name: object = general_settings.get("litellm_key_header_name")
return frozenset({name.lower()}) if isinstance(name, str) and name else frozenset()
def _request_strip_headers() -> frozenset[str]:
"""Headers to drop before forwarding a request to a plugin backend.
Every header user_api_key_auth accepts as a litellm credential is stripped
Authorization, x-api-key, API-Key, x-goog-api-key, Ocp-Apim-Subscription-Key,
x-litellm-api-key, and any configured custom key header so a plugin can
never be handed the caller's live litellm key (confused-deputy escalation).
"""
return (
_HOP_BY_HOP_STRIP
| SpecialHeaders.litellm_credential_header_names()
| _configured_key_header_names()
)
# Headers to strip from plugin RESPONSES before returning to the browser.
# httpx already decompresses and de-chunks the body, so forwarding the wire
# encoding headers causes clients to attempt double-decompression (garbage) or
# incorrect length checks. set-cookie is removed so plugins cannot overwrite
# litellm session cookies.
_RESPONSE_STRIP = {
"content-encoding",
"transfer-encoding",
"content-length",
"set-cookie",
}
def _safe_response_headers(raw: "Mapping[str, str]") -> dict[str, str]:
"""Strip wire-encoding/cookie headers and force proxied responses inert.
Plugin-controlled bytes are served from the litellm dashboard origin, so a
compromised plugin could return an HTML/JS document that executes with the
admin's session against same-origin management APIs. A sandbox CSP forces
the response into an opaque origin with scripts disabled, and nosniff stops
content-type confusion from re-enabling execution. Both are set last so a
plugin cannot override them with its own headers.
"""
return {
**{k: v for k, v in raw.items() if k.lower() not in _RESPONSE_STRIP},
"content-security-policy": "sandbox",
"x-content-type-options": "nosniff",
}
# In-memory plugin registry — populated from general_settings at startup
_plugin_registry: dict[str, PluginConfig] = {}
# ---------------------------------------------------------------------------
# Key derivation — audience-scoped per plugin so compromising one plugin
# cannot be used to forge claims for another. LITELLM_SALT_KEY is NEVER
# shared with plugins; each plugin only receives a key derived from
# HMAC(LITELLM_SALT_KEY, plugin_name) which reveals nothing about the master.
# ---------------------------------------------------------------------------
def _plugin_fernet(plugin_name: str) -> Fernet:
"""Return a Fernet cipher whose key is scoped to a specific plugin.
Key material: HMAC-SHA256(LITELLM_SALT_KEY, plugin_name).
A plugin possessing its own key cannot derive the master salt or
forge claims intended for a different plugin.
"""
salt = os.getenv("LITELLM_SALT_KEY", "").encode()
derived = _hmac.new(salt, plugin_name.encode(), hashlib.sha256).digest()
return Fernet(base64.urlsafe_b64encode(derived))
_CLAIM_TTL_SECONDS = 30 # identity claims expire after 30 s
def issue_plugin_session_claim(
plugin_name: str, user_id: str | None, user_role: str | None
) -> str:
"""Issue a short-lived, audience-scoped identity claim for the plugin.
The claim contains {user_id, user_role, plugin, exp}. Crucially it
contains NO litellm bearer token the plugin can only derive the
caller's identity, not act as them against the proxy.
"""
claim = {
"plugin": plugin_name,
"user_id": user_id or "",
"user_role": user_role or "",
"exp": int(time.time()) + _CLAIM_TTL_SECONDS,
}
return _plugin_fernet(plugin_name).encrypt(json.dumps(claim).encode()).decode()
def verify_plugin_session_claim(plugin_name: str, ciphertext: str) -> dict:
"""Verify and decode a plugin session claim.
Raises ValueError if the HMAC is invalid, the audience is wrong, or
the claim is expired. Returns the decoded claim dict on success.
"""
try:
raw = _plugin_fernet(plugin_name).decrypt(
ciphertext.encode(), ttl=_CLAIM_TTL_SECONDS
)
claim = json.loads(raw)
except (InvalidToken, Exception) as exc:
raise ValueError("Invalid, tampered, or expired plugin session claim") from exc
if claim.get("plugin") != plugin_name:
raise ValueError("Plugin claim audience mismatch")
if int(claim.get("exp", 0)) < int(time.time()):
raise ValueError("Plugin session claim expired")
return claim
# ---------------------------------------------------------------------------
# Config
# ---------------------------------------------------------------------------
def register_plugins_from_config(general_settings: dict[str, object]) -> None:
"""Replace the plugin registry from general_settings.
Replaces (not merges) so plugins removed from config are immediately
unreachable without requiring a process restart.
"""
raw = general_settings.get("plugins")
entries: list[object] = raw if isinstance(raw, list) else []
new_registry = {
p.name: p for p in (PluginConfig.model_validate(entry) for entry in entries)
}
_plugin_registry.clear()
_plugin_registry.update(new_registry)
# ---------------------------------------------------------------------------
# Routes
# ---------------------------------------------------------------------------
@router.get("/api/plugins", tags=["plugins"])
async def list_plugins(
user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
) -> list[dict[str, str]]:
"""Return registered plugins for authenticated UI callers.
plugin_key is never returned the browser never needs it (the proxy injects
it server-side from the registry), and exposing it here would leak the
credential into React state and DevTools. Admin key management goes through
the redacted /config/field/info path instead.
"""
return [
{
"name": plugin.name,
"display_name": plugin.display_name or plugin.name,
"url": plugin.url,
}
for plugin in _plugin_registry.values()
]
@router.get("/api/plugins/auth-token", tags=["plugins"])
async def plugin_auth_token(
user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
plugin_name: str = "litellm-platform-plugin",
) -> dict:
"""Issue a short-lived, audience-scoped plugin session claim.
The claim contains {user_id, user_role, plugin, exp}. It does NOT
contain the caller's litellm bearer token — a compromised plugin can
only learn the caller's identity, not impersonate them against the proxy.
Encrypted with a key derived from HMAC(LITELLM_SALT_KEY, plugin_name),
so each plugin holds only its own key and cannot forge claims for others.
Requires LITELLM_SALT_KEY to be set; returns 503 otherwise.
"""
if not os.getenv("LITELLM_SALT_KEY"):
raise HTTPException(
status_code=503,
detail="LITELLM_SALT_KEY is not configured; plugin iframe auth unavailable.",
)
if plugin_name not in _plugin_registry:
raise HTTPException(
status_code=404, detail=f"Plugin '{plugin_name}' is not registered."
)
user_id = getattr(user_api_key_dict, "user_id", None)
user_role = getattr(user_api_key_dict, "user_role", None)
return {
"session_claim": issue_plugin_session_claim(plugin_name, user_id, user_role)
}
@router.api_route(
"/plugin-proxy/{plugin_name}/{path:path}",
methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD"],
tags=["plugins"],
include_in_schema=False,
)
async def plugin_proxy(
plugin_name: str,
path: str,
request: Request,
user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
) -> Response:
"""Authenticated reverse-proxy to a registered plugin backend.
Restricted to proxy_admin callers the shared plugin_key must not be
usable as a confused-deputy credential by regular users. Plugin UIs
talk to the plugin service directly via the iframe; this route is for
administrative and server-to-server access only.
The caller's litellm credential is stripped and replaced with the
plugin's own plugin_key so plugins never receive a live litellm API key.
"""
if getattr(user_api_key_dict, "user_role", None) != "proxy_admin":
return Response(
content="Plugin proxy access requires proxy_admin role.",
status_code=403,
)
plugin = _plugin_registry.get(plugin_name)
if not plugin:
return Response(
content=f"Plugin '{plugin_name}' not registered",
status_code=404,
)
target_url = f"{plugin.url.rstrip('/')}/{path}"
query = request.url.query
if query:
target_url = f"{target_url}?{query}"
body = await request.body()
# Strip caller credentials and hop-by-hop headers from forwarded request
strip = _request_strip_headers()
forward_headers = {
k: v for k, v in request.headers.items() if k.lower() not in strip
}
# Inject plugin's own credential as upstream auth (if configured)
plugin_key = plugin.plugin_key
if plugin_key:
forward_headers["authorization"] = f"Bearer {plugin_key}"
# Forward caller identity so the plugin can enforce its own access control.
# The plugin MUST NOT trust these as credentials — they are informational.
# The plugin_key above is the only authentication mechanism.
user_id = getattr(user_api_key_dict, "user_id", None)
user_role = getattr(user_api_key_dict, "user_role", None)
if user_id:
forward_headers["x-litellm-user-id"] = str(user_id)
if user_role:
forward_headers["x-litellm-user-role"] = str(user_role)
handler = get_async_httpx_client(
llm_provider=httpxSpecialProvider.PassThroughEndpoint
)
try:
req = handler.client.build_request(
method=request.method,
url=target_url,
headers=forward_headers,
content=body,
)
# Do not follow redirects — a redirect to an internal URL would allow
# the plugin to SSRF the proxy into fetching arbitrary internal services.
resp = await handler.client.send(req, follow_redirects=False)
except Exception:
return Response(
content=f"Cannot connect to plugin '{plugin_name}' at {plugin.url}",
status_code=502,
)
return Response(
content=resp.content,
status_code=resp.status_code,
headers=_safe_response_headers(resp.headers),
)

View file

@ -419,6 +419,10 @@ from litellm.proxy.management_endpoints.workflow_management_endpoints import (
)
from litellm.proxy.management_helpers.audit_logs import create_audit_log_for_update
from litellm.proxy.memory.memory_endpoints import router as memory_router
from litellm.proxy.plugin_routes import (
router as plugin_router,
register_plugins_from_config,
)
from litellm.proxy.middleware.in_flight_requests_middleware import (
InFlightRequestsMiddleware,
)
@ -4502,6 +4506,8 @@ class ProxyConfig:
load_from_azure_key_vault(use_azure_key_vault=use_azure_key_vault)
### ALERTING ###
self._load_alerting_settings(general_settings=general_settings)
### PLUGINS ###
register_plugins_from_config(general_settings)
### CONNECT TO DATABASE ###
database_url = general_settings.get("database_url", None)
if database_url and database_url.startswith("os.environ/"):
@ -5663,6 +5669,10 @@ class ProxyConfig:
llm_router=llm_router,
)
if _general_settings is not None and "plugins" in _general_settings:
general_settings["plugins"] = _general_settings["plugins"]
register_plugins_from_config(general_settings)
async def _reschedule_spend_log_cleanup_job(self):
"""
Reschedule the spend log cleanup job based on current general_settings.
@ -14931,6 +14941,41 @@ async def update_config(
Keep it more precise, to prevent overwrite other values unintentially
"""
_PLUGIN_KEY_REDACTED = "***"
def _preserve_redacted_plugin_keys(incoming: object, existing: object) -> object:
"""Restore real plugin_key values the client never sees.
/config/field/info redacts every plugin_key to ``"***"``, so an admin
editing a plugin posts that placeholder (or a blank, when the UI clears the
field) straight back. Treat a blank or redacted plugin_key as "keep the
stored credential" by sourcing it from the existing config; only a real,
non-redacted value replaces it, and a blank with no stored key drops the
field entirely instead of persisting the placeholder.
"""
if not isinstance(incoming, list):
return incoming
stored_keys = {
p["name"]: p["plugin_key"]
for p in (existing if isinstance(existing, list) else [])
if isinstance(p, dict) and p.get("name") and p.get("plugin_key")
}
def resolve(plugin: object) -> object:
if not isinstance(plugin, dict):
return plugin
key = plugin.get("plugin_key")
if key not in (None, "", _PLUGIN_KEY_REDACTED):
return plugin
name = plugin.get("name")
if name in stored_keys:
return {**plugin, "plugin_key": stored_keys[name]}
return {k: v for k, v in plugin.items() if k != "plugin_key"}
return [resolve(p) for p in incoming]
@router.post(
"/config/field/update",
@ -14997,7 +15042,13 @@ async def update_config_general_settings(
## update db
general_settings[data.field_name] = data.field_value
field_value = data.field_value
if data.field_name == "plugins":
field_value = _preserve_redacted_plugin_keys(
field_value, general_settings.get("plugins")
)
general_settings[data.field_name] = field_value
response = await ConfigRepository(prisma_client).table.upsert(
where={"param_name": "general_settings"},
@ -15008,6 +15059,9 @@ async def update_config_general_settings(
)
await invalidate_config_param("general_settings")
if data.field_name == "plugins":
register_plugins_from_config(general_settings)
return response
@ -15063,9 +15117,19 @@ async def get_config_general_settings(
general_settings = dict(db_general_settings.param_value)
if field_name in general_settings:
return ConfigFieldInfo(
field_name=field_name, field_value=general_settings[field_name]
)
field_value = general_settings[field_name]
# Redact plugin_key from plugin configs so the shared credential
# is never returned even to admin-viewer callers.
if field_name == "plugins" and isinstance(field_value, list):
field_value = [
(
{k: ("***" if k == "plugin_key" else v) for k, v in p.items()}
if isinstance(p, dict)
else p
)
for p in field_value
]
return ConfigFieldInfo(field_name=field_name, field_value=field_value)
else:
raise HTTPException(
status_code=400,
@ -16387,6 +16451,7 @@ app.include_router(model_access_group_management_router)
app.include_router(tag_management_router)
app.include_router(workflow_management_router)
app.include_router(memory_router)
app.include_router(plugin_router)
app.include_router(cost_tracking_settings_router)
app.include_router(router_settings_router)
app.include_router(fallback_management_router)

View file

@ -0,0 +1,232 @@
"""Regression tests for UI-registered embed plugins.
Covers three bugs:
1. `general_settings.plugins` was not a field on ConfigGeneralSettings, so the
admin UI's POST /config/field/update with field_name="plugins" was rejected
with "Invalid field=plugins passed in."
2. The in-memory plugin registry only refreshed at startup, so a plugin added
via the UI did not appear in /api/plugins until a restart.
3. Plugins persisted to DB general_settings were not loaded on startup (the
registry only initialised from the YAML config), so UI-added plugins vanished
after a restart.
"""
import asyncio
from unittest.mock import MagicMock
from litellm.proxy._types import (
ConfigGeneralSettings,
LitellmUserRoles,
PluginConfig,
UserAPIKeyAuth,
)
from litellm.proxy.plugin_routes import list_plugins, register_plugins_from_config
def _admin() -> UserAPIKeyAuth:
return UserAPIKeyAuth(api_key="sk-admin", user_role=LitellmUserRoles.PROXY_ADMIN)
def _non_admin() -> UserAPIKeyAuth:
return UserAPIKeyAuth(api_key="sk-user", user_role=LitellmUserRoles.INTERNAL_USER)
def test_plugins_is_a_valid_general_setting() -> None:
"""The config-update endpoint gates on this exact membership check."""
assert "plugins" in ConfigGeneralSettings.model_fields
def test_config_general_settings_parses_plugin_list() -> None:
"""A list of plugin dicts (what the UI sends) coerces into PluginConfig."""
settings = ConfigGeneralSettings.model_validate(
{
"plugins": [
{
"name": "chat-ui",
"display_name": "Chat UI",
"url": "http://localhost:3300",
},
{
"name": "agent-builder",
"url": "http://127.0.0.1:4010",
"plugin_key": "sk-secret",
},
]
}
)
plugins = settings.plugins
assert plugins is not None
assert [p.name for p in plugins] == ["chat-ui", "agent-builder"]
assert isinstance(plugins[0], PluginConfig)
assert plugins[1].display_name is None
assert plugins[1].plugin_key == "sk-secret"
def test_registered_plugins_appear_in_list_without_restart() -> None:
"""register_plugins_from_config makes UI-added plugins visible immediately,
and replaces (not merges) so removed plugins disappear."""
register_plugins_from_config(
{
"plugins": [
{
"name": "chat-ui",
"display_name": "Chat UI",
"url": "http://localhost:3300",
}
]
}
)
names = [p["name"] for p in asyncio.run(list_plugins(user_api_key_dict=_admin()))]
assert names == ["chat-ui"]
register_plugins_from_config(
{
"plugins": [
{
"name": "chat-ui",
"display_name": "Chat UI",
"url": "http://localhost:3300",
},
{
"name": "agent-builder",
"display_name": "Agent Builder",
"url": "http://127.0.0.1:4010",
},
]
}
)
names = sorted(
p["name"] for p in asyncio.run(list_plugins(user_api_key_dict=_admin()))
)
assert names == ["agent-builder", "chat-ui"]
# Removing a plugin from config drops it from the live list.
register_plugins_from_config({})
assert asyncio.run(list_plugins(user_api_key_dict=_admin())) == []
def test_plugin_key_is_never_returned_to_the_browser() -> None:
"""plugin_key is a credential the UI never needs; /api/plugins must omit it
for every caller, admin included, so it never lands in browser state."""
register_plugins_from_config(
{
"plugins": [
{
"name": "p",
"display_name": "P",
"url": "http://localhost:9",
"plugin_key": "sk-secret",
}
]
}
)
admin_entry = asyncio.run(list_plugins(user_api_key_dict=_admin()))[0]
user_entry = asyncio.run(list_plugins(user_api_key_dict=_non_admin()))[0]
assert "plugin_key" not in admin_entry
assert "plugin_key" not in user_entry
assert admin_entry["url"] == "http://localhost:9"
register_plugins_from_config({})
def test_db_persisted_plugins_load_on_startup() -> None:
"""Plugins saved to DB general_settings must register when the DB config is
merged at startup, not just when present in the YAML file."""
from litellm.proxy.proxy_server import ProxyConfig
register_plugins_from_config({}) # start empty (as if YAML had no plugins)
ProxyConfig()._add_general_settings_from_db_config(
config_data={
"general_settings": {
"plugins": [
{
"name": "db-plugin",
"display_name": "DB Plugin",
"url": "http://localhost:5000",
}
]
}
},
general_settings={},
proxy_logging_obj=MagicMock(),
)
names = [p["name"] for p in asyncio.run(list_plugins(user_api_key_dict=_admin()))]
assert names == ["db-plugin"]
register_plugins_from_config({})
def test_safe_response_headers_sandbox_and_strips_wire_headers() -> None:
"""Proxied plugin responses must be inert and shed wire/cookie headers."""
from litellm.proxy.plugin_routes import _safe_response_headers
out = _safe_response_headers(
{
"content-type": "text/html",
"content-encoding": "gzip",
"content-length": "123",
"set-cookie": "session=abc",
"content-security-policy": "default-src *",
}
)
assert out["content-security-policy"] == "sandbox"
assert out["x-content-type-options"] == "nosniff"
assert out["content-type"] == "text/html"
for stripped in ("content-encoding", "content-length", "set-cookie"):
assert stripped not in out
def test_litellm_credential_header_names_covers_every_auth_header() -> None:
"""The canonical strip set must list every header user_api_key_auth accepts
as a litellm key, so a new auth header can't silently start leaking."""
from litellm.proxy._types import SpecialHeaders
assert SpecialHeaders.litellm_credential_header_names() == {
"authorization",
"api-key",
"x-api-key",
"x-goog-api-key",
"ocp-apim-subscription-key",
"x-litellm-api-key",
}
def test_every_litellm_auth_header_is_stripped_before_forwarding() -> None:
"""A plugin must never receive any header that authenticates against litellm,
only the hop-by-hop set and benign headers are forwarded."""
from litellm.proxy.plugin_routes import _request_strip_headers
strip = _request_strip_headers()
incoming = {
"Authorization": "Bearer sk-litellm",
"API-Key": "sk-litellm",
"X-Api-Key": "sk-litellm",
"X-Goog-Api-Key": "sk-litellm",
"Ocp-Apim-Subscription-Key": "sk-litellm",
"X-Litellm-Api-Key": "sk-litellm",
"Cookie": "litellm_session=abc",
"Accept": "application/json",
"X-Trace-Id": "t-1",
}
forwarded = {k: v for k, v in incoming.items() if k.lower() not in strip}
assert forwarded == {"Accept": "application/json", "X-Trace-Id": "t-1"}
def test_configured_custom_key_header_is_stripped() -> None:
"""A custom general_settings.litellm_key_header_name must also be stripped,
read live so config changes are honoured without a restart."""
from litellm.proxy import proxy_server
from litellm.proxy.plugin_routes import _request_strip_headers
original = getattr(proxy_server, "general_settings", None)
proxy_server.general_settings = {"litellm_key_header_name": "X-My-Tenant-Key"}
try:
assert "x-my-tenant-key" in _request_strip_headers()
finally:
proxy_server.general_settings = original

View file

@ -8326,3 +8326,39 @@ def test_get_config_list_includes_cancel_on_disconnect(monkeypatch):
assert fields["cancel_on_disconnect"]["field_type"] == "Boolean"
finally:
app.dependency_overrides.clear()
def test_preserve_redacted_plugin_keys_keeps_stored_credential():
"""A redacted or blank plugin_key on update must not overwrite the real key."""
from litellm.proxy.proxy_server import _preserve_redacted_plugin_keys
existing = [{"name": "p1", "url": "https://p1", "plugin_key": "sk-real-1"}]
redacted = _preserve_redacted_plugin_keys(
[{"name": "p1", "url": "https://p1-new", "plugin_key": "***"}], existing
)
assert redacted == [
{"name": "p1", "url": "https://p1-new", "plugin_key": "sk-real-1"}
]
blanked = _preserve_redacted_plugin_keys(
[{"name": "p1", "url": "https://p1", "plugin_key": ""}], existing
)
assert blanked[0]["plugin_key"] == "sk-real-1"
def test_preserve_redacted_plugin_keys_sets_new_and_drops_orphan_placeholder():
"""A real new key replaces; a placeholder with no stored key is dropped, never persisted."""
from litellm.proxy.proxy_server import _preserve_redacted_plugin_keys
existing = [{"name": "p1", "url": "https://p1", "plugin_key": "sk-real-1"}]
rotated = _preserve_redacted_plugin_keys(
[{"name": "p1", "url": "https://p1", "plugin_key": "sk-new"}], existing
)
assert rotated[0]["plugin_key"] == "sk-new"
new_plugin = _preserve_redacted_plugin_keys(
[{"name": "p2", "url": "https://p2", "plugin_key": "***"}], existing
)
assert "plugin_key" not in new_plugin[0]

View file

@ -0,0 +1,83 @@
import { describe, it, expect, vi } from "vitest";
import { render, waitFor } from "@testing-library/react";
import { AgentControlPlaneView } from "./layout";
const { getMock } = vi.hoisted(() => ({ getMock: vi.fn(() => Promise.resolve({ session_claim: "claim" })) }));
const pluginModeValue = {
mode: "litellm-platform-plugin" as string,
setMode: vi.fn(),
plugins: [{ name: "litellm-platform-plugin", display_name: "Chat UI", url: "http://localhost:3300" }],
activePlugin: { name: "litellm-platform-plugin", display_name: "Chat UI", url: "http://localhost:3300" } as {
name: string;
display_name: string;
url: string;
} | null,
};
vi.mock("@/contexts/PluginModeContext", () => ({ usePluginMode: () => pluginModeValue }));
vi.mock("@/contexts/AuthContext", () => ({ useAuth: () => ({ accessToken: "sk-test-token" }) }));
vi.mock("@/lib/http/client", () => ({
createApiClient: () => ({ get: getMock }),
}));
vi.mock("@/components/networking", () => ({ getProxyBaseUrl: () => "" }));
describe("AgentControlPlaneView iframe", () => {
it("embeds the plugin at its ROOT url, never a hardcoded subpath like /sessions", () => {
const { container } = render(<AgentControlPlaneView />);
const iframe = container.querySelector("iframe");
expect(iframe).not.toBeNull();
const src = iframe!.getAttribute("src")!;
expect(src).toBe("http://localhost:3300/");
expect(src).not.toContain("/sessions");
// title comes from the plugin's display_name, not a hardcoded label
expect(iframe!.getAttribute("title")).toBe("Chat UI");
});
it("does not double the slash when the plugin url has a trailing slash", () => {
pluginModeValue.activePlugin = {
name: "litellm-platform-plugin",
display_name: "Chat UI",
url: "http://localhost:3300/",
};
const { container } = render(<AgentControlPlaneView />);
expect(container.querySelector("iframe")!.getAttribute("src")).toBe("http://localhost:3300/");
pluginModeValue.activePlugin = {
name: "litellm-platform-plugin",
display_name: "Chat UI",
url: "http://localhost:3300",
};
});
it("does not leak the raw token in the iframe src (token goes via encrypted postMessage)", () => {
const { container } = render(<AgentControlPlaneView />);
expect(container.querySelector("iframe")!.getAttribute("src")).not.toContain("token");
});
it("does not delegate clipboard-read to the untrusted plugin iframe", () => {
const { container } = render(<AgentControlPlaneView />);
const allow = container.querySelector("iframe")!.getAttribute("allow") ?? "";
expect(allow).not.toContain("clipboard-read");
expect(allow).toContain("clipboard-write");
});
it("requests the auth-token claim scoped to the active plugin, not a hardcoded default", async () => {
getMock.mockClear();
pluginModeValue.activePlugin = { name: "reports-plugin", display_name: "Reports", url: "http://localhost:3300" };
render(<AgentControlPlaneView />);
await waitFor(() => expect(getMock).toHaveBeenCalled());
const [path, opts] = getMock.mock.calls[0];
expect(path).toBe("/api/plugins/auth-token");
expect(opts.query).toEqual({ plugin_name: "reports-plugin" });
pluginModeValue.activePlugin = {
name: "litellm-platform-plugin",
display_name: "Chat UI",
url: "http://localhost:3300",
};
});
});

View file

@ -1,6 +1,6 @@
"use client";
import React, { Suspense, useState } from "react";
import React, { Suspense, useState, useRef, useEffect } from "react";
import Navbar from "@/components/navbar";
import LoadingScreen from "@/components/common_components/LoadingScreen";
import { ThemeProvider } from "@/contexts/ThemeContext";
@ -9,6 +9,88 @@ import SidebarProvider from "@/app/(dashboard)/components/SidebarProvider";
import { useRouter, useSearchParams, usePathname } from "next/navigation";
import { DebugWarningBanner } from "@/components/DebugWarningBanner";
import { MIGRATED_PAGES, migratedHref, legacyPageHref, legacyKeyForPathname } from "@/utils/migratedPages";
import { PluginModeProvider, usePluginMode } from "@/contexts/PluginModeContext";
import { createApiClient } from "@/lib/http/client";
import { getProxyBaseUrl } from "@/components/networking";
const pluginApiClient = createApiClient({ getBaseUrl: () => getProxyBaseUrl() ?? "" });
// Wrapper so PluginModeProvider receives the live accessToken from auth context,
// which means plugin data refreshes on login/logout without stale cookie reads.
function PluginModeProviderWithAuth({ children }: { children: React.ReactNode }) {
const { accessToken } = useAuth();
return <PluginModeProvider accessToken={accessToken}>{children}</PluginModeProvider>;
}
export function AgentControlPlaneView() {
const { activePlugin } = usePluginMode();
const activePluginName = activePlugin?.name;
const agentPlatformUrl = activePlugin?.url ?? "";
const { accessToken } = useAuth();
const iframeRef = useRef<HTMLIFrameElement>(null);
const [auth, setAuth] = useState<{ plugin: string; claim: string } | null>(null);
// Fetch a short-lived identity claim scoped to the *active* plugin. The claim
// is encrypted under that plugin's own per-plugin key, so it must be requested
// per plugin and re-fetched when the user switches plugins.
useEffect(() => {
if (!accessToken || !activePluginName) return;
let cancelled = false;
pluginApiClient
.get("/api/plugins/auth-token", { accessToken, query: { plugin_name: activePluginName } })
.then((data: { session_claim?: string }) => {
if (!cancelled && data?.session_claim) setAuth({ plugin: activePluginName, claim: data.session_claim });
})
.catch(() => {});
return () => {
cancelled = true;
};
}, [accessToken, activePluginName]);
// Deliver the claim to the iframe via postMessage, but only while it was issued
// for the plugin currently mounted — never replay one plugin's claim to another.
// targetOrigin is the configured plugin URL — no other origin receives it.
useEffect(() => {
const iframe = iframeRef.current;
if (!iframe || !auth || auth.plugin !== activePluginName || !agentPlatformUrl) return;
const send = () => {
iframe.contentWindow?.postMessage({ type: "litellm-auth", session_claim: auth.claim }, agentPlatformUrl);
};
// Cover both orderings: the iframe may have already fired `load` before the
// claim arrived (send now), or it may load/reload later (send on the event).
send();
iframe.addEventListener("load", send);
return () => iframe.removeEventListener("load", send);
}, [auth, activePluginName, agentPlatformUrl]);
if (!agentPlatformUrl) {
return (
<div className="flex flex-1 items-center justify-center text-gray-500">
<div className="text-center">
<p className="text-lg font-medium mb-2">Plugin</p>
<p className="text-sm">Configure the plugin URL in settings</p>
</div>
</div>
);
}
// Embed the plugin at its root; the plugin renders its own full UI (incl. nav) inside.
return (
<iframe
ref={iframeRef}
src={`${agentPlatformUrl.replace(/\/$/, "")}/`}
style={{
width: "100%",
height: "100%",
border: "none",
flex: 1,
minHeight: "calc(100vh - 56px)",
}}
title={activePlugin?.display_name ?? "Plugin"}
allow="clipboard-write"
/>
);
}
function DashboardShell({ children }: { children: React.ReactNode }) {
const router = useRouter();
@ -16,6 +98,7 @@ function DashboardShell({ children }: { children: React.ReactNode }) {
const pathname = usePathname();
const { accessToken } = useAuth();
const [sidebarCollapsed, setSidebarCollapsed] = useState(false);
const { mode } = usePluginMode();
const page = legacyKeyForPathname(pathname) || searchParams.get("page") || "api-keys";
@ -34,10 +117,18 @@ function DashboardShell({ children }: { children: React.ReactNode }) {
/>
<DebugWarningBanner accessToken={accessToken} />
<div className="flex flex-1">
<div className="mt-2">
<SidebarProvider setPage={navigateToPage} defaultSelectedKey={page} sidebarCollapsed={sidebarCollapsed} />
</div>
<main className="flex-1">{children}</main>
{mode !== "ai-gateway" ? (
<div className="flex-1 flex">
<AgentControlPlaneView />
</div>
) : (
<>
<div className="mt-2">
<SidebarProvider setPage={navigateToPage} defaultSelectedKey={page} sidebarCollapsed={sidebarCollapsed} />
</div>
<main className="flex-1">{children}</main>
</>
)}
</div>
</div>
);
@ -62,7 +153,9 @@ function LayoutContent({ children }: { children: React.ReactNode }) {
export default function Layout({ children }: { children: React.ReactNode }) {
return (
<Suspense fallback={<LoadingScreen />}>
<LayoutContent>{children}</LayoutContent>
<PluginModeProviderWithAuth>
<LayoutContent>{children}</LayoutContent>
</PluginModeProviderWithAuth>
</Suspense>
);
}

View file

@ -25,6 +25,7 @@ import LoggingSettings from "./Settings/AdminSettings/LoggingSettings/LoggingSet
import SSOSettings from "./Settings/AdminSettings/SSOSettings/SSOSettings";
import UISettings from "./Settings/AdminSettings/UISettings/UISettings";
import HashicorpVault from "./Settings/AdminSettings/HashicorpVault/HashicorpVault";
import PluginSettings from "./Settings/AdminSettings/PluginSettings/PluginSettings";
import SSOModals from "./SSOModals";
import UIAccessControlForm from "./UIAccessControlForm";
@ -373,6 +374,11 @@ const AdminPanel: React.FC<AdminPanelProps> = ({ proxySettings }) => {
label: "Hashicorp Vault",
children: <HashicorpVault />,
},
{
key: "plugins",
label: "Plugins",
children: <PluginSettings />,
},
];
return (

View file

@ -0,0 +1,76 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { act, fireEvent, render, screen, waitFor } from "@testing-library/react";
import ViewSwitcher from "./ViewSwitcher";
const { mockUsePluginMode, state } = vi.hoisted(() => {
const state = {
mode: "ai-gateway" as string,
setMode: vi.fn(),
plugins: [] as { name: string; display_name: string; url: string }[],
activePlugin: null as { name: string; display_name: string; url: string } | null,
};
return { mockUsePluginMode: vi.fn(() => state), state };
});
vi.mock("@/contexts/PluginModeContext", () => ({ usePluginMode: mockUsePluginMode }));
describe("ViewSwitcher", () => {
afterEach(() => {
state.mode = "ai-gateway";
state.plugins = [];
state.setMode.mockClear();
});
it("renders nothing when there are no plugins", () => {
const { container } = render(<ViewSwitcher />);
expect(container.firstChild).toBeNull();
});
it("labels the button from the active plugin's display_name and lists AI Gateway + each plugin", async () => {
state.plugins = [
{ name: "litellm-platform-plugin", display_name: "Chat UI", url: "http://localhost:3300" },
{ name: "obs", display_name: "Observability", url: "http://localhost:9000" },
];
state.mode = "litellm-platform-plugin";
render(<ViewSwitcher />);
expect(screen.getByRole("button")).toHaveTextContent("Chat UI");
expect(screen.queryByText("Agent Control Plane")).not.toBeInTheDocument();
act(() => {
fireEvent.click(screen.getByRole("button"));
});
await waitFor(() => expect(screen.getByText("AI Gateway")).toBeInTheDocument());
expect(screen.getByText("Observability")).toBeInTheDocument();
});
it("switches to AI Gateway when that entry is picked", async () => {
state.plugins = [{ name: "litellm-platform-plugin", display_name: "Chat UI", url: "http://localhost:3300" }];
state.mode = "litellm-platform-plugin";
render(<ViewSwitcher />);
act(() => {
fireEvent.click(screen.getByRole("button"));
});
await waitFor(() => expect(screen.getByText("AI Gateway")).toBeInTheDocument());
act(() => {
fireEvent.click(screen.getByText("AI Gateway"));
});
expect(state.setMode).toHaveBeenCalledWith("ai-gateway");
});
it("switches to a plugin by name when its entry is picked", async () => {
state.plugins = [{ name: "litellm-platform-plugin", display_name: "Chat UI", url: "http://localhost:3300" }];
state.mode = "ai-gateway";
render(<ViewSwitcher />);
act(() => {
fireEvent.click(screen.getByRole("button"));
});
await waitFor(() => expect(screen.getByText("Chat UI")).toBeInTheDocument());
act(() => {
fireEvent.click(screen.getByText("Chat UI"));
});
expect(state.setMode).toHaveBeenCalledWith("litellm-platform-plugin");
});
});

View file

@ -0,0 +1,46 @@
import React from "react";
import { Dropdown } from "antd";
import { AppstoreOutlined, CheckOutlined, DownOutlined } from "@ant-design/icons";
import type { MenuProps } from "antd";
import { usePluginMode } from "@/contexts/PluginModeContext";
const GATEWAY = "ai-gateway";
export default function ViewSwitcher() {
const { mode, setMode, plugins } = usePluginMode();
// Only a switcher when there is at least one plugin to switch to.
if (plugins.length === 0) return null;
const activeLabel = plugins.find((p) => p.name === mode)?.display_name ?? "AI Gateway";
const entries = [
{ value: GATEWAY, label: "AI Gateway" },
...plugins.map((p) => ({ value: p.name, label: p.display_name })),
];
const items: MenuProps["items"] = entries.map((e) => ({
key: e.value,
label: (
<div className="flex items-center justify-between gap-6 py-0.5">
<span className="font-medium">{e.label}</span>
{e.value === mode && <CheckOutlined className="text-blue-600" />}
</div>
),
}));
const onClick: MenuProps["onClick"] = ({ key }) => setMode(key);
return (
<Dropdown menu={{ items, onClick, selectedKeys: [mode] }} trigger={["click"]}>
<button
type="button"
className="flex items-center gap-2 rounded-md border border-gray-200 px-2.5 py-1.5 text-sm font-medium text-gray-700 transition-colors hover:bg-gray-50"
>
<AppstoreOutlined className="text-gray-500" />
<span>{activeLabel}</span>
<DownOutlined className="text-[10px] text-gray-400" />
</button>
</Dropdown>
);
}

View file

@ -0,0 +1,173 @@
"use client";
import { useState, useEffect } from "react";
import { Button, Card, Form, Input, Modal, Space, Table, Typography } from "antd";
import { DeleteOutlined, EditOutlined, PlusOutlined } from "@ant-design/icons";
import { getConfigFieldSetting, updateConfigFieldSetting } from "@/components/networking";
import useAuthorized from "@/app/(dashboard)/hooks/useAuthorized";
const { Title, Text, Paragraph } = Typography;
interface Plugin {
name: string;
display_name: string;
url: string;
plugin_key?: string;
}
export default function PluginSettings() {
const { accessToken } = useAuthorized();
const [plugins, setPlugins] = useState<Plugin[]>([]);
const [loading, setLoading] = useState(true);
const [saving, setSaving] = useState(false);
const [modalOpen, setModalOpen] = useState(false);
const [editingIndex, setEditingIndex] = useState<number | null>(null);
const [form] = Form.useForm<Plugin>();
useEffect(() => {
if (!accessToken) return;
getConfigFieldSetting(accessToken, "plugins")
.then((data) => {
const val = data?.field_value;
setPlugins(Array.isArray(val) ? val : []);
})
.catch(() => setPlugins([]))
.finally(() => setLoading(false));
}, [accessToken]);
const save = async (updated: Plugin[]) => {
if (!accessToken) return;
setSaving(true);
try {
await updateConfigFieldSetting(accessToken, "plugins", updated);
setPlugins(updated);
} finally {
setSaving(false);
}
};
const openAdd = () => {
setEditingIndex(null);
form.resetFields();
setModalOpen(true);
};
const openEdit = (idx: number) => {
setEditingIndex(idx);
// plugin_key arrives redacted ("***"); start it blank so an untouched save
// keeps the stored credential instead of overwriting it with the placeholder.
form.setFieldsValue({ ...plugins[idx], plugin_key: "" });
setModalOpen(true);
};
const handleDelete = (idx: number) => {
const updated = plugins.filter((_, i) => i !== idx);
save(updated);
};
const handleOk = async () => {
const values = await form.validateFields();
const updated =
editingIndex !== null ? plugins.map((p, i) => (i === editingIndex ? values : p)) : [...plugins, values];
await save(updated);
setModalOpen(false);
};
const columns = [
{
title: "Name",
dataIndex: "name",
key: "name",
render: (v: string) => <Text code>{v}</Text>,
},
{ title: "Display Name", dataIndex: "display_name", key: "display_name" },
{
title: "URL",
dataIndex: "url",
key: "url",
render: (v: string) => (
<a href={v} target="_blank" rel="noopener noreferrer">
{v}
</a>
),
},
{
title: "Plugin Key",
dataIndex: "plugin_key",
key: "plugin_key",
render: (v?: string) => (v ? <Text code>{"•".repeat(8)}</Text> : <Text type="secondary"></Text>),
},
{
title: "Actions",
key: "actions",
render: (_: unknown, __: Plugin, idx: number) => (
<Space>
<Button icon={<EditOutlined />} size="small" onClick={() => openEdit(idx)} />
<Button icon={<DeleteOutlined />} size="small" danger onClick={() => handleDelete(idx)} />
</Space>
),
},
];
return (
<Card>
<Title level={4}>Plugins</Title>
<Paragraph>
Register external services as plugins. Once added, users can toggle to the plugin from the mode switcher in the
top-left of the sidebar.
</Paragraph>
<Paragraph type="secondary" style={{ fontSize: 12 }}>
Each plugin must expose <Text code>GET /api/plugin-manifest</Text> returning nav items and capabilities.
</Paragraph>
<Button type="primary" icon={<PlusOutlined />} onClick={openAdd} style={{ marginBottom: 16 }}>
Add Plugin
</Button>
<Table dataSource={plugins} columns={columns} rowKey="name" loading={loading} pagination={false} size="small" />
<Modal
title={editingIndex !== null ? "Edit Plugin" : "Add Plugin"}
open={modalOpen}
onOk={handleOk}
onCancel={() => setModalOpen(false)}
confirmLoading={saving}
okText="Save"
>
<Form form={form} layout="vertical" style={{ marginTop: 16 }}>
<Form.Item
name="name"
label="Name (identifier)"
rules={[{ required: true, message: "Required" }]}
extra="Used in URLs and config. No spaces. E.g. litellm-platform-plugin"
>
<Input placeholder="litellm-platform-plugin" />
</Form.Item>
<Form.Item name="display_name" label="Display Name" rules={[{ required: true, message: "Required" }]}>
<Input placeholder="Agent Control Plane" />
</Form.Item>
<Form.Item
name="url"
label="URL"
rules={[
{ required: true, message: "Required" },
{ type: "url", message: "Must be a valid URL" },
]}
extra="Base URL of the plugin service"
>
<Input placeholder="https://your-plugin.example.com" />
</Form.Item>
<Form.Item
name="plugin_key"
label="Plugin Key"
extra="Optional. The plugin's own credential, injected as Authorization: Bearer <key> only when litellm reverse-proxies API calls to the plugin's backend (/plugin-proxy/<name>/*). Leave blank for plugins that use the forwarded litellm user token (e.g. iframe plugins) — that path uses the user's token, not this key."
>
<Input.Password
placeholder={editingIndex !== null ? "Leave blank to keep current key" : "sk-... (optional)"}
/>
</Form.Item>
</Form>
</Modal>
</Card>
);
}

View file

@ -16,6 +16,7 @@ import { CommunityEngagementButtons } from "./Navbar/CommunityEngagementButtons/
import { NAV_PRODUCT_LINK_CLASS } from "./Navbar/navProductLinkClass";
import { NotificationsBell } from "./Navbar/NotificationsBell/NotificationsBell";
import UserDropdown from "./Navbar/UserDropdown/UserDropdown";
import ViewSwitcher from "./Navbar/ViewSwitcher";
import WorkerDropdown from "./Navbar/WorkerDropdown/WorkerDropdown";
interface NavbarProps {
@ -111,6 +112,12 @@ const Navbar: React.FC<NavbarProps> = ({
</div>
</div>
{!isPublicPage && (
<div className="ml-4 flex shrink-0 items-center border-l border-gray-200 pl-4">
<ViewSwitcher />
</div>
)}
<div className="ml-auto flex min-w-0 flex-1 items-center justify-end gap-4">
{showWorkerSwitch && (
<div className="flex shrink-0 items-center">

View file

@ -0,0 +1,64 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { render, screen, waitFor } from "@testing-library/react";
import { PluginModeProvider, usePluginMode } from "./PluginModeContext";
import type { Plugin } from "./PluginModeContext";
const { getMock } = vi.hoisted(() => ({ getMock: vi.fn() }));
vi.mock("@/lib/http/client", () => ({
createApiClient: () => ({ get: getMock }),
}));
vi.mock("@/components/networking", () => ({ getProxyBaseUrl: () => "" }));
function ModeProbe() {
const { mode, activePlugin } = usePluginMode();
return (
<div>
<span data-testid="mode">{mode}</span>
<span data-testid="active">{activePlugin?.name ?? "none"}</span>
</div>
);
}
const renderWithPlugins = (plugins: Plugin[]) => {
getMock.mockResolvedValueOnce(plugins);
return render(
<PluginModeProvider accessToken="sk-test">
<ModeProbe />
</PluginModeProvider>,
);
};
describe("PluginModeProvider effectiveMode fallback", () => {
beforeEach(() => {
vi.clearAllMocks();
localStorage.setItem("litellm_plugin_mode", "my-plugin");
});
it("falls back to ai-gateway once an empty plugins list loads", async () => {
renderWithPlugins([]);
await waitFor(() => expect(getMock).toHaveBeenCalled());
await waitFor(() => expect(screen.getByTestId("mode").textContent).toBe("ai-gateway"));
expect(screen.getByTestId("active").textContent).toBe("none");
});
it("keeps the stored mode when it is still registered", async () => {
renderWithPlugins([{ name: "my-plugin", display_name: "My Plugin", url: "https://p.example.com" }]);
await waitFor(() => expect(screen.getByTestId("active").textContent).toBe("my-plugin"));
expect(screen.getByTestId("mode").textContent).toBe("my-plugin");
});
it("falls back to ai-gateway when the plugins fetch fails, never stranding the user", async () => {
getMock.mockRejectedValueOnce(new Error("network down"));
render(
<PluginModeProvider accessToken="sk-test">
<ModeProbe />
</PluginModeProvider>,
);
await waitFor(() => expect(getMock).toHaveBeenCalled());
await waitFor(() => expect(screen.getByTestId("mode").textContent).toBe("ai-gateway"));
});
});

View file

@ -0,0 +1,95 @@
"use client";
import React, { createContext, useContext, useState, useEffect } from "react";
import { createApiClient } from "@/lib/http/client";
import { getProxyBaseUrl } from "@/components/networking";
export type PluginMode = "ai-gateway" | string; // "ai-gateway" or a registered plugin name
export interface PluginNavItem {
key: string;
label: string;
icon?: string;
path: string;
badge?: boolean;
}
export interface Plugin {
name: string;
display_name: string;
url: string;
plugin_key?: string;
nav_items?: PluginNavItem[];
capabilities?: string[];
}
interface PluginModeContextValue {
mode: PluginMode;
setMode: (mode: PluginMode) => void;
plugins: Plugin[];
activePlugin: Plugin | null;
}
const PluginModeContext = createContext<PluginModeContextValue>({
mode: "ai-gateway",
setMode: () => {},
plugins: [],
activePlugin: null,
});
const STORAGE_KEY = "litellm_plugin_mode";
const pluginApiClient = createApiClient({ getBaseUrl: () => getProxyBaseUrl() ?? "" });
function readStoredMode(): PluginMode {
if (typeof window === "undefined") return "ai-gateway";
return localStorage.getItem(STORAGE_KEY) ?? "ai-gateway";
}
interface PluginModeProviderProps {
children: React.ReactNode;
/** Pass the current access token from the app's auth context. */
accessToken?: string | null;
}
export function PluginModeProvider({ children, accessToken }: PluginModeProviderProps) {
const [mode, setModeState] = useState<PluginMode>(readStoredMode);
const [plugins, setPlugins] = useState<Plugin[]>([]);
const [loaded, setLoaded] = useState(false);
useEffect(() => {
// Re-fetch whenever the auth token changes (handles login/logout cycles)
if (!accessToken) return;
pluginApiClient
.get("/api/plugins", { accessToken })
.then((data: Plugin[]) => {
setPlugins(Array.isArray(data) ? data : []);
})
.catch(() => {})
// Mark loaded even on failure so a stored plugin mode still falls back to
// ai-gateway; otherwise a failed fetch would strand the user on a blank
// plugin view with no switcher to escape.
.finally(() => setLoaded(true));
}, [accessToken]);
// Once plugins have loaded, fall back to ai-gateway if the persisted mode is
// no longer registered — including when the list came back empty (all plugins
// removed). Derived rather than setState-in-effect to avoid cascading renders.
const effectiveMode = mode !== "ai-gateway" && loaded && !plugins.some((p) => p.name === mode) ? "ai-gateway" : mode;
const setMode = (m: PluginMode) => {
setModeState(m);
localStorage.setItem(STORAGE_KEY, m);
};
const activePlugin = plugins.find((p) => p.name === effectiveMode) ?? null;
return (
<PluginModeContext.Provider value={{ mode: effectiveMode, setMode, plugins, activePlugin }}>
{children}
</PluginModeContext.Provider>
);
}
export function usePluginMode() {
return useContext(PluginModeContext);
}

View file

@ -515,6 +515,60 @@ export interface paths {
patch?: never;
trace?: never;
};
"/api/plugins": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* List Plugins
* @description Return registered plugins for authenticated UI callers.
*
* plugin_key is never returned the browser never needs it (the proxy injects
* it server-side from the registry), and exposing it here would leak the
* credential into React state and DevTools. Admin key management goes through
* the redacted /config/field/info path instead.
*/
get: operations["list_plugins_api_plugins_get"];
put?: never;
post?: never;
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/api/plugins/auth-token": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* Plugin Auth Token
* @description Issue a short-lived, audience-scoped plugin session claim.
*
* The claim contains {user_id, user_role, plugin, exp}. It does NOT
* contain the caller's litellm bearer token a compromised plugin can
* only learn the caller's identity, not impersonate them against the proxy.
*
* Encrypted with a key derived from HMAC(LITELLM_SALT_KEY, plugin_name),
* so each plugin holds only its own key and cannot forge claims for others.
*
* Requires LITELLM_SALT_KEY to be set; returns 503 otherwise.
*/
get: operations["plugin_auth_token_api_plugins_auth_token_get"];
put?: never;
post?: never;
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/apply_guardrail": {
parameters: {
query?: never;
@ -8945,6 +8999,106 @@ export interface paths {
patch?: never;
trace?: never;
};
"/plugin-proxy/{plugin_name}/{path}": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* Plugin Proxy
* @description Authenticated reverse-proxy to a registered plugin backend.
*
* Restricted to proxy_admin callers the shared plugin_key must not be
* usable as a confused-deputy credential by regular users. Plugin UIs
* talk to the plugin service directly via the iframe; this route is for
* administrative and server-to-server access only.
*
* The caller's litellm credential is stripped and replaced with the
* plugin's own plugin_key so plugins never receive a live litellm API key.
*/
get: operations["plugin_proxy_plugin_proxy__plugin_name___path__get"];
/**
* Plugin Proxy
* @description Authenticated reverse-proxy to a registered plugin backend.
*
* Restricted to proxy_admin callers the shared plugin_key must not be
* usable as a confused-deputy credential by regular users. Plugin UIs
* talk to the plugin service directly via the iframe; this route is for
* administrative and server-to-server access only.
*
* The caller's litellm credential is stripped and replaced with the
* plugin's own plugin_key so plugins never receive a live litellm API key.
*/
put: operations["plugin_proxy_plugin_proxy__plugin_name___path__put"];
/**
* Plugin Proxy
* @description Authenticated reverse-proxy to a registered plugin backend.
*
* Restricted to proxy_admin callers the shared plugin_key must not be
* usable as a confused-deputy credential by regular users. Plugin UIs
* talk to the plugin service directly via the iframe; this route is for
* administrative and server-to-server access only.
*
* The caller's litellm credential is stripped and replaced with the
* plugin's own plugin_key so plugins never receive a live litellm API key.
*/
post: operations["plugin_proxy_plugin_proxy__plugin_name___path__post"];
/**
* Plugin Proxy
* @description Authenticated reverse-proxy to a registered plugin backend.
*
* Restricted to proxy_admin callers the shared plugin_key must not be
* usable as a confused-deputy credential by regular users. Plugin UIs
* talk to the plugin service directly via the iframe; this route is for
* administrative and server-to-server access only.
*
* The caller's litellm credential is stripped and replaced with the
* plugin's own plugin_key so plugins never receive a live litellm API key.
*/
delete: operations["plugin_proxy_plugin_proxy__plugin_name___path__delete"];
/**
* Plugin Proxy
* @description Authenticated reverse-proxy to a registered plugin backend.
*
* Restricted to proxy_admin callers the shared plugin_key must not be
* usable as a confused-deputy credential by regular users. Plugin UIs
* talk to the plugin service directly via the iframe; this route is for
* administrative and server-to-server access only.
*
* The caller's litellm credential is stripped and replaced with the
* plugin's own plugin_key so plugins never receive a live litellm API key.
*/
options: operations["plugin_proxy_plugin_proxy__plugin_name___path__options"];
/**
* Plugin Proxy
* @description Authenticated reverse-proxy to a registered plugin backend.
*
* Restricted to proxy_admin callers the shared plugin_key must not be
* usable as a confused-deputy credential by regular users. Plugin UIs
* talk to the plugin service directly via the iframe; this route is for
* administrative and server-to-server access only.
*
* The caller's litellm credential is stripped and replaced with the
* plugin's own plugin_key so plugins never receive a live litellm API key.
*/
head: operations["plugin_proxy_plugin_proxy__plugin_name___path__head"];
/**
* Plugin Proxy
* @description Authenticated reverse-proxy to a registered plugin backend.
*
* Restricted to proxy_admin callers the shared plugin_key must not be
* usable as a confused-deputy credential by regular users. Plugin UIs
* talk to the plugin service directly via the iframe; this route is for
* administrative and server-to-server access only.
*
* The caller's litellm credential is stripped and replaced with the
* plugin's own plugin_key so plugins never receive a live litellm API key.
*/
patch: operations["plugin_proxy_plugin_proxy__plugin_name___path__patch"];
trace?: never;
};
"/policies": {
parameters: {
query?: never;
@ -22216,6 +22370,11 @@ export interface components {
* @description Default upstream request timeout in seconds for native and custom pass-through endpoints that use pass_through_request. Defaults to 600 when unset.
*/
pass_through_request_timeout?: number | null;
/**
* Plugins
* @description external services registered as embeddable UI plugins
*/
plugins?: components["schemas"]["PluginConfig"][] | null;
/**
* Reject Clientside Metadata Tags
* @description When set to True, rejects requests that contain client-side 'metadata.tags' to prevent users from influencing budgets by sending different tags. Tags can only be inherited from the API key metadata.
@ -28185,6 +28344,32 @@ export interface components {
*/
name: string;
};
/**
* PluginConfig
* @description A single external service registered as an embeddable UI plugin.
*/
PluginConfig: {
/**
* Display Name
* @description human-readable label shown in the UI view switcher
*/
display_name?: string | null;
/**
* Name
* @description unique plugin identifier (kebab-case)
*/
name: string;
/**
* Plugin Key
* @description plugin's own credential, injected as Bearer auth only on /plugin-proxy/<name>/* reverse-proxy calls
*/
plugin_key?: string | null;
/**
* Url
* @description base URL of the plugin service
*/
url: string;
};
/**
* PluginListItem
* @description Plugin item in list responses.
@ -33488,6 +33673,61 @@ export interface operations {
};
};
};
list_plugins_api_plugins_get: {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
[key: string]: string;
}[];
};
};
};
};
plugin_auth_token_api_plugins_auth_token_get: {
parameters: {
query?: {
plugin_name?: string;
};
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
[key: string]: unknown;
};
};
};
/** @description Validation Error */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["HTTPValidationError"];
};
};
};
};
apply_guardrail_apply_guardrail_post: {
parameters: {
query?: never;
@ -44501,6 +44741,230 @@ export interface operations {
};
};
};
plugin_proxy_plugin_proxy__plugin_name___path__get: {
parameters: {
query?: never;
header?: never;
path: {
plugin_name: string;
path: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
/** @description Validation Error */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["HTTPValidationError"];
};
};
};
};
plugin_proxy_plugin_proxy__plugin_name___path__put: {
parameters: {
query?: never;
header?: never;
path: {
plugin_name: string;
path: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
/** @description Validation Error */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["HTTPValidationError"];
};
};
};
};
plugin_proxy_plugin_proxy__plugin_name___path__post: {
parameters: {
query?: never;
header?: never;
path: {
plugin_name: string;
path: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
/** @description Validation Error */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["HTTPValidationError"];
};
};
};
};
plugin_proxy_plugin_proxy__plugin_name___path__delete: {
parameters: {
query?: never;
header?: never;
path: {
plugin_name: string;
path: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
/** @description Validation Error */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["HTTPValidationError"];
};
};
};
};
plugin_proxy_plugin_proxy__plugin_name___path__options: {
parameters: {
query?: never;
header?: never;
path: {
plugin_name: string;
path: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
/** @description Validation Error */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["HTTPValidationError"];
};
};
};
};
plugin_proxy_plugin_proxy__plugin_name___path__head: {
parameters: {
query?: never;
header?: never;
path: {
plugin_name: string;
path: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
/** @description Validation Error */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["HTTPValidationError"];
};
};
};
};
plugin_proxy_plugin_proxy__plugin_name___path__patch: {
parameters: {
query?: never;
header?: never;
path: {
plugin_name: string;
path: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
/** @description Validation Error */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["HTTPValidationError"];
};
};
};
};
create_policy_policies_post: {
parameters: {
query?: never;