mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
feat(otel/v2): admin-owned, identity-scoped trace destinations
OTEL v2 trace destinations are admin-owned and resolved server-side from the authenticated identity; nothing in an inference request (body, headers, or metadata) can set or select where a trace goes. Resolves LIT-3850. A destination is the universal OTLP target (endpoint + headers). The admin registers destinations as named logging credentials and assigns them to a virtual key, team, or organization via metadata.logging_exporters, or marks a destination's scope as global / team-set / org-set. At request time the proxy unions exporters assigned across the identity chain (key + team + org, each read from its own record), resolves them to destinations, and fans the trace out to all of them. An identity with no assignment gets no per-tenant destination (default-deny). The resolved destinations ride a server-only field of standard_callback_dynamic_params, absent from the request-read whitelist. Assignment is admin-owned: metadata.logging_exporters on a key, team, or org is proxy-admin only and every name must be a registered logging credential, validated in the create and update handlers for all three. Credential create/update/delete are proxy-admin gated when the credential is a logging destination. UI: OTEL destinations live as rows in the one Active Logging Callbacks table with a Scope column and an Edit-scope modal (global / teams / orgs); the existing Add Callback modal is extended to also create destinations inline; the key, team, and org forms gain a Logging Exporters multi-select for identity-side assignment, and their overview views render the assigned exporters inside the Logging Settings card. Out of scope: a credential_type column (today the logging tag lives in the free-form credential_info, matching the existing custom_llm_provider discriminator on cursor pass-through credentials, no migration); per-key access from the destination side; routing the request root span through the same per-tenant destinations for full-trace parity. The rebuilt UI bundle ships via the separate chore(ui): rebuild ui PR cycle.
This commit is contained in:
yucheng-berriai
parent
8bc18388e3
commit
a90eff3d5b
44 changed files with 2340 additions and 308 deletions
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -123,3 +123,7 @@ crash.*.log
|
|||
# and should be committed.
|
||||
.vscode
|
||||
.pin_list.txt
|
||||
repro_lit3850/
|
||||
|
||||
# throwaway learning scripts (not a feature)
|
||||
scratch/
|
||||
|
|
|
|||
|
|
@ -512,6 +512,11 @@ DD_TRACER_STREAMING_CHUNK_YIELD_RESOURCE = os.getenv(
|
|||
|
||||
LITELLM_HTTP_STATUS_CLIENT_DISCONNECTED = 499
|
||||
|
||||
# Reserved key/team logging callback var that binds the callback to a named OTEL
|
||||
# credential in the registry (an admin-owned reference resolved server-side into a
|
||||
# trace destination, never forwarded as a request parameter).
|
||||
LITELLM_LOGGING_CREDENTIAL_NAME_KEY = "litellm_logging_credential_name"
|
||||
|
||||
EMAIL_BUDGET_ALERT_TTL = int(
|
||||
os.getenv("EMAIL_BUDGET_ALERT_TTL", 24 * 60 * 60)
|
||||
) # 24 hours in seconds
|
||||
|
|
|
|||
|
|
@ -247,12 +247,13 @@ lives in [`plumbing/`](./plumbing):
|
|||
- [`presets/`](./presets) — each preset reads one integration's env vars and
|
||||
returns an `OpenTelemetryV2Config` (exporter destination + mapper vocabularies
|
||||
+ resource attributes). `PRESET_BY_CALLBACK` maps a callback name (`"arize"`,
|
||||
`"langfuse_otel"`, …) to its preset. Integrations that support team/key-scoped
|
||||
credentials also provide a per-request OTLP header builder
|
||||
(`DYNAMIC_HEADERS_BY_CALLBACK`). Presets do **no** network I/O at build time:
|
||||
AgentOps, for example, mints its JWT lazily inside a custom exporter on the
|
||||
first export (in the `BatchSpanProcessor` worker thread), never on the event
|
||||
loop.
|
||||
`"langfuse_otel"`, …) to its preset. Per-key/team routing is **not** here: a
|
||||
destination is admin-owned infrastructure, resolved server-side from a named
|
||||
credential into an `OtelDestination` (`destinations.py`) and applied by
|
||||
`plumbing/routing.py`. Nothing in `presets/` reads vendor credentials or a host
|
||||
off a request. Presets do **no** network I/O at build time: AgentOps, for
|
||||
example, mints its JWT lazily inside a custom exporter on the first export (in
|
||||
the `BatchSpanProcessor` worker thread), never on the event loop.
|
||||
|
||||
## Extending
|
||||
|
||||
|
|
@ -261,7 +262,8 @@ lives in [`plumbing/`](./plumbing):
|
|||
`key -> extractor` tables) and register it in `mappers/__init__._MAPPER_BY_NAME`.
|
||||
- **A new integration**: add a preset in `presets/` that returns an
|
||||
`OpenTelemetryV2Config`, and register it in `presets/__init__.PRESET_BY_CALLBACK`.
|
||||
If it supports dynamic credentials, add a header builder to
|
||||
`DYNAMIC_HEADERS_BY_CALLBACK`.
|
||||
For admin-owned per-key/team destinations, add an adapter mapping the named
|
||||
credential's values to an `OtelDestination` in `destinations._ADAPTERS` (or rely
|
||||
on the generic `otel_endpoint`/`otel_headers` passthrough).
|
||||
- **A new span kind**: add a role to `spans.py` (registry entry + name builder),
|
||||
a payload dataclass in `payloads.py`, and a branch in the relevant mapper(s).
|
||||
|
|
|
|||
106
litellm/integrations/otel/destinations.py
Normal file
106
litellm/integrations/otel/destinations.py
Normal file
|
|
@ -0,0 +1,106 @@
|
|||
"""Resolve an admin-owned named credential into a typed OTLP destination.
|
||||
|
||||
The destination (endpoint + auth headers) is admin infrastructure config. Each
|
||||
OTEL backend stores its own fields on the named credential's free-form
|
||||
``credential_values``; the adapter here maps those fields to the universal
|
||||
``OtelDestination`` the v2 router exports through. A backend with no bespoke
|
||||
adapter is still reachable through the generic ``otel_endpoint`` / ``otel_headers``
|
||||
passthrough, so the registry covers every OTEL destination rather than an
|
||||
enumerated few. Nothing here reads request data; callers pass admin-resolved
|
||||
credential values only.
|
||||
"""
|
||||
|
||||
from typing import Callable, Mapping, Optional
|
||||
|
||||
from litellm.constants import LITELLM_LOGGING_CREDENTIAL_NAME_KEY
|
||||
from litellm.integrations.langfuse.langfuse_otel import (
|
||||
LANGFUSE_CLOUD_US_ENDPOINT,
|
||||
LangfuseOtelLogger,
|
||||
)
|
||||
from litellm.integrations.otel.model.destination import OtelDestination
|
||||
from litellm.integrations.weave.weave_otel import _get_weave_authorization_header
|
||||
|
||||
#: Reserved ``callback_vars`` key binding a key/team's logging callback to a named
|
||||
#: credential in the registry. It is a reference, resolved server-side; it is never
|
||||
#: forwarded as a request parameter.
|
||||
LOGGING_CREDENTIAL_NAME_KEY = LITELLM_LOGGING_CREDENTIAL_NAME_KEY
|
||||
|
||||
|
||||
def _parse_header_string(raw: str) -> dict[str, str]:
|
||||
pairs = (item.split("=", 1) for item in raw.split(",") if "=" in item)
|
||||
return {key.strip(): value.strip() for key, value in pairs}
|
||||
|
||||
|
||||
def _langfuse_endpoint(host: str) -> str:
|
||||
normalized = host if host.startswith("http") else f"https://{host}"
|
||||
return f"{normalized.rstrip('/')}/api/public/otel"
|
||||
|
||||
|
||||
def _langfuse_destination(values: Mapping[str, str]) -> Optional[OtelDestination]:
|
||||
public_key = values.get("langfuse_public_key")
|
||||
secret_key = values.get("langfuse_secret_key")
|
||||
if not public_key or not secret_key:
|
||||
return None
|
||||
host = values.get("langfuse_host")
|
||||
endpoint = _langfuse_endpoint(host) if host else LANGFUSE_CLOUD_US_ENDPOINT
|
||||
auth = LangfuseOtelLogger._get_langfuse_authorization_header(
|
||||
public_key=public_key, secret_key=secret_key
|
||||
)
|
||||
return OtelDestination(endpoint=endpoint, headers={"Authorization": auth})
|
||||
|
||||
|
||||
def _arize_destination(values: Mapping[str, str]) -> Optional[OtelDestination]:
|
||||
space = values.get("arize_space_id") or values.get("arize_space_key")
|
||||
api_key = values.get("arize_api_key")
|
||||
if not space or not api_key:
|
||||
return None
|
||||
endpoint = values.get("arize_endpoint") or "https://otlp.arize.com/v1"
|
||||
return OtelDestination(
|
||||
endpoint=endpoint, headers={"space_id": space, "api_key": api_key}
|
||||
)
|
||||
|
||||
|
||||
def _weave_destination(values: Mapping[str, str]) -> Optional[OtelDestination]:
|
||||
api_key = values.get("wandb_api_key")
|
||||
endpoint = values.get("weave_endpoint")
|
||||
if not api_key or not endpoint:
|
||||
return None
|
||||
headers = {"Authorization": _get_weave_authorization_header(api_key=api_key)}
|
||||
project_id = values.get("weave_project_id")
|
||||
if project_id:
|
||||
headers["project_id"] = project_id
|
||||
return OtelDestination(endpoint=endpoint, headers=headers)
|
||||
|
||||
|
||||
def _generic_destination(values: Mapping[str, str]) -> Optional[OtelDestination]:
|
||||
"""Any OTLP backend: an explicit endpoint plus raw headers. The catch-all that
|
||||
makes the registry cover self-hosted collectors / Phoenix / Honeycomb / etc."""
|
||||
endpoint = values.get("otel_endpoint")
|
||||
if not endpoint:
|
||||
return None
|
||||
return OtelDestination(
|
||||
endpoint=endpoint, headers=_parse_header_string(values.get("otel_headers", ""))
|
||||
)
|
||||
|
||||
|
||||
_ADAPTERS: dict[str, Callable[[Mapping[str, str]], Optional[OtelDestination]]] = {
|
||||
"langfuse_otel": _langfuse_destination,
|
||||
"arize": _arize_destination,
|
||||
"weave_otel": _weave_destination,
|
||||
}
|
||||
|
||||
#: OTEL v2 callbacks that can be routed to a per-key/team admin destination.
|
||||
OTEL_V2_DESTINATION_CALLBACKS = frozenset(_ADAPTERS)
|
||||
|
||||
|
||||
def build_destination(
|
||||
callback_name: str, values: Mapping[str, str]
|
||||
) -> Optional[OtelDestination]:
|
||||
"""Map an admin credential's ``values`` to an ``OtelDestination`` for
|
||||
``callback_name``, falling back to the generic OTLP passthrough."""
|
||||
adapter = _ADAPTERS.get(callback_name)
|
||||
if adapter is not None:
|
||||
destination = adapter(values)
|
||||
if destination is not None:
|
||||
return destination
|
||||
return _generic_destination(values)
|
||||
|
|
@ -167,6 +167,17 @@ class OpenTelemetryV2(CustomLogger):
|
|||
if getattr(proxy_server, "open_telemetry_logger", None) is None:
|
||||
setattr(proxy_server, "open_telemetry_logger", self)
|
||||
|
||||
def _destinations_for_backend(self, call: "LLMCallEvent") -> tuple:
|
||||
"""The call's admin-resolved destinations that belong to THIS logger's backend.
|
||||
|
||||
A request fans out across whatever exporters its identity chain is assigned;
|
||||
each logger exports only the destinations tagged with its own callback_name,
|
||||
so each backend's span keeps its own attribute vocabulary.
|
||||
"""
|
||||
return tuple(
|
||||
d for d in call.otel_destinations if d.callback_name == self.callback_name
|
||||
)
|
||||
|
||||
# ====================================================================== #
|
||||
# LLM-call callbacks — the span is opened at the ``pre_call`` boundary and
|
||||
# closed here. See ``log_pre_api_call``.
|
||||
|
|
@ -215,7 +226,7 @@ class OpenTelemetryV2(CustomLogger):
|
|||
parent_context=parent_context,
|
||||
start_time_ns=start_time_ns,
|
||||
tracer=self._tenant_tracers.tracer_for(
|
||||
self.tracer, call.dynamic_params
|
||||
self.tracer, self._destinations_for_backend(call)
|
||||
),
|
||||
)
|
||||
self._open_llm_calls[call_id] = _LLMCallSpan(
|
||||
|
|
@ -353,7 +364,9 @@ class OpenTelemetryV2(CustomLogger):
|
|||
parent_context=parent_ctx,
|
||||
start_time_ns=carrier.start_time_ns,
|
||||
end_time_ns=end_time_ns,
|
||||
tracer=self._tenant_tracers.tracer_for(self.tracer, call.dynamic_params),
|
||||
tracer=self._tenant_tracers.tracer_for(
|
||||
self.tracer, self._destinations_for_backend(call)
|
||||
),
|
||||
)
|
||||
|
||||
# ====================================================================== #
|
||||
|
|
|
|||
26
litellm/integrations/otel/model/destination.py
Normal file
26
litellm/integrations/otel/model/destination.py
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
"""The resolved, admin-owned OTLP destination.
|
||||
|
||||
A trace destination is admin-owned infrastructure config, never request data.
|
||||
The proxy resolves a key/team's bound named credential into this typed,
|
||||
backend-agnostic target (an endpoint plus auth headers) server-side, and the v2
|
||||
logger exports through it. Every OTEL backend -- Langfuse, Arize, Weave, a
|
||||
self-hosted collector -- reduces to this shape; the per-backend field mapping
|
||||
lives in ``litellm.integrations.otel.destinations``.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
|
||||
class OtelDestination(BaseModel):
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
endpoint: str
|
||||
headers: dict[str, str] = Field(default_factory=dict)
|
||||
# The OTEL backend (callback_name) this destination belongs to, so a request
|
||||
# that fans out across backends routes each destination to the logger that
|
||||
# owns its attribute vocabulary. None for the legacy single-destination path.
|
||||
callback_name: str | None = None
|
||||
|
||||
def header_string(self) -> str:
|
||||
"""Render headers as the ``k=v,k2=v2`` form an ``ExporterSpec`` expects."""
|
||||
return ",".join(f"{key}={value}" for key, value in self.headers.items())
|
||||
|
|
@ -39,7 +39,10 @@ from __future__ import annotations
|
|||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any, Mapping, cast
|
||||
|
||||
from pydantic import ValidationError
|
||||
|
||||
from litellm.constants import LITELLM_LOGGING_NO_UPSTREAM_LLM_CALL
|
||||
from litellm.integrations.otel.model.destination import OtelDestination
|
||||
from litellm.integrations.otel.model.semconv import resolve_operation
|
||||
from litellm.integrations.otel.model.utils import as_str
|
||||
|
||||
|
|
@ -47,6 +50,31 @@ if TYPE_CHECKING:
|
|||
from litellm.types.utils import StandardLoggingPayload
|
||||
|
||||
|
||||
def _otel_destinations(dynamic_params: Any) -> tuple[OtelDestination, ...]:
|
||||
"""The admin-resolved OTLP destinations carried on ``standard_callback_dynamic_params``.
|
||||
|
||||
Server-set only (the proxy resolves the exporters assigned to the request's
|
||||
identity chain and strips any client value), so this is the sole source the v2
|
||||
router trusts -- request-supplied vendor credentials are never read here. A
|
||||
request fans out to every destination here; each logger keeps only the ones
|
||||
tagged with its own backend.
|
||||
"""
|
||||
if not isinstance(dynamic_params, Mapping):
|
||||
return ()
|
||||
raw = dynamic_params.get("otel_destinations")
|
||||
if not isinstance(raw, list):
|
||||
return ()
|
||||
parsed: list[OtelDestination] = []
|
||||
for item in raw:
|
||||
if not isinstance(item, Mapping):
|
||||
continue
|
||||
try:
|
||||
parsed.append(OtelDestination.model_validate(dict(item)))
|
||||
except ValidationError:
|
||||
continue
|
||||
return tuple(parsed)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RequestIdentity:
|
||||
call_id: str | None = None
|
||||
|
|
@ -210,6 +238,10 @@ class LLMCallEvent:
|
|||
# The ``standard_callback_dynamic_params`` routing the call to a per-tenant
|
||||
# tracer (its own exporter/endpoint), or ``None`` when the call isn't scoped.
|
||||
dynamic_params: Any
|
||||
# The admin-resolved OTLP destinations (endpoint + auth headers) for this call's
|
||||
# identity chain, fanned out to. Empty when none are assigned. The only source the
|
||||
# v2 router trusts for per-tenant routing; never request-derived.
|
||||
otel_destinations: tuple[OtelDestination, ...]
|
||||
# True for synthetic proxy-gate logs (auth / rate-limit rejections): they fire
|
||||
# the ``pre_call`` hook but never made an upstream call, so they get no span.
|
||||
is_no_upstream_call: bool
|
||||
|
|
@ -224,10 +256,12 @@ class LLMCallEvent:
|
|||
payload = cast("StandardLoggingPayload", raw_payload) if raw_payload else None
|
||||
operation = resolve_operation(as_str(kwargs.get("call_type")))
|
||||
model = as_str(kwargs.get("model")) or ""
|
||||
dynamic_params = kwargs.get("standard_callback_dynamic_params")
|
||||
return cls(
|
||||
call_id=_call_id(payload, kwargs),
|
||||
payload=payload,
|
||||
dynamic_params=kwargs.get("standard_callback_dynamic_params"),
|
||||
dynamic_params=dynamic_params,
|
||||
otel_destinations=_otel_destinations(dynamic_params),
|
||||
is_no_upstream_call=bool(kwargs.get(LITELLM_LOGGING_NO_UPSTREAM_LLM_CALL)),
|
||||
provisional_span_name=f"{operation.value} {model}".strip(),
|
||||
)
|
||||
|
|
|
|||
|
|
@ -1,36 +1,37 @@
|
|||
"""Per-request multi-tenant tracer routing.
|
||||
"""Per-request multi-tenant tracer routing with fan-out.
|
||||
|
||||
When a request carries team/key vendor credentials in
|
||||
``standard_callback_dynamic_params``, its spans must export through a
|
||||
``TracerProvider`` whose OTLP headers carry those credentials.
|
||||
``TenantTracerCache`` builds and caches one provider per distinct credential
|
||||
set, and otherwise hands back the logger's default tracer. This lets a single
|
||||
logger fan requests out to many tenants without needing a logger per tenant.
|
||||
A call's identity chain is assigned a set of admin-owned OTEL destinations
|
||||
(``LLMCallEvent.otel_destinations``, resolved server-side from named credentials).
|
||||
Its spans must export to ALL of them plus the configured/global exporter, so
|
||||
``TenantTracerCache`` builds and caches one ``TracerProvider`` per distinct
|
||||
destination SET -- the provider keeps the configured exporters and appends one
|
||||
``SpanProcessor`` per destination, so a span is emitted once and copied to each
|
||||
(no duplicate spans). With no destinations it hands back the logger's default
|
||||
tracer (global only). Destinations are never request-derived, so a caller can
|
||||
neither redirect a trace nor spawn providers.
|
||||
"""
|
||||
|
||||
from collections import OrderedDict
|
||||
from typing import Any, Mapping
|
||||
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.trace import Tracer
|
||||
|
||||
from litellm._logging import verbose_logger
|
||||
from litellm.integrations.otel.model.config import OpenTelemetryV2Config
|
||||
from litellm.integrations.otel.presets import dynamic_otlp_headers
|
||||
from litellm.integrations.otel.model.config import ExporterSpec, OpenTelemetryV2Config
|
||||
from litellm.integrations.otel.model.destination import OtelDestination
|
||||
from litellm.integrations.otel.plumbing.providers import (
|
||||
build_tracer_provider,
|
||||
get_tracer,
|
||||
)
|
||||
|
||||
# Exporter kinds that ignore headers — never rewritten with dynamic credentials.
|
||||
# Exporter kinds that ignore endpoint/headers — never rewritten with a destination.
|
||||
_NON_OTLP_KINDS = ("console", "in_memory", "inmemory", "memory")
|
||||
|
||||
# Cap on distinct credential-scoped providers held at once. ``dynamic_params``
|
||||
# can be populated from request metadata, so an unbounded cache lets a caller
|
||||
# spawn one ``TracerProvider`` (plus its ``BatchSpanProcessor`` background
|
||||
# thread) per unique credential set and exhaust the proxy. The LRU bound keeps
|
||||
# the working set of active tenants resident while flushing and shutting down
|
||||
# evicted providers so their threads are reclaimed.
|
||||
# Cap on distinct destination-scoped providers held at once. Destinations are
|
||||
# admin-owned (one per key/team), so this is resource hygiene rather than an
|
||||
# anti-abuse bound: it keeps the working set of active tenants resident while
|
||||
# flushing and shutting down evicted providers so their exporter threads are
|
||||
# reclaimed.
|
||||
_MAX_CACHED_PROVIDERS = 256
|
||||
|
||||
|
||||
|
|
@ -49,7 +50,7 @@ def _shutdown_provider(provider: TracerProvider) -> None:
|
|||
|
||||
|
||||
class TenantTracerCache:
|
||||
"""Credential-scoped ``TracerProvider`` cache keyed by the dynamic headers."""
|
||||
"""Destination-scoped ``TracerProvider`` cache keyed by endpoint + headers."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
|
|
@ -60,52 +61,68 @@ class TenantTracerCache:
|
|||
self._config = config
|
||||
self._callback_name = callback_name
|
||||
self._tracer_name = tracer_name
|
||||
self._providers: "OrderedDict[tuple[tuple[str, str], ...], TracerProvider]" = (
|
||||
OrderedDict()
|
||||
)
|
||||
self._providers: "OrderedDict[tuple[tuple[str, tuple[tuple[str, str], ...]], ...], TracerProvider]" = (OrderedDict())
|
||||
|
||||
def tracer_for(self, default: Tracer, dynamic_params: Any) -> Tracer:
|
||||
def tracer_for(
|
||||
self, default: Tracer, destinations: "tuple[OtelDestination, ...]"
|
||||
) -> Tracer:
|
||||
"""Return the tracer for this request.
|
||||
|
||||
Use ``default`` unless the request's dynamic credentials require a
|
||||
credential-scoped tracer, in which case build (or reuse) one. The cache
|
||||
is a bounded LRU: the least-recently-used provider is flushed and shut
|
||||
down on overflow so its exporter threads don't accumulate.
|
||||
``destinations`` are the admin-resolved exporters (for this backend) that the
|
||||
request's identity chain is assigned. Empty -> the logger's default tracer
|
||||
(the global/configured exporter only = deny). Otherwise build (or reuse) a
|
||||
provider that exports to the configured exporters PLUS every destination, so
|
||||
one span is emitted once and copied to all (fan-out, no duplicate spans). The
|
||||
cache is a bounded LRU keyed on the destination SET.
|
||||
"""
|
||||
headers = dynamic_otlp_headers(self._callback_name, dynamic_params)
|
||||
if not headers:
|
||||
if not destinations:
|
||||
return default
|
||||
cache_key = tuple(sorted(headers.items()))
|
||||
cache_key = tuple(
|
||||
sorted((d.endpoint, tuple(sorted(d.headers.items()))) for d in destinations)
|
||||
)
|
||||
provider = self._providers.get(cache_key)
|
||||
if provider is not None:
|
||||
self._providers.move_to_end(cache_key)
|
||||
else:
|
||||
provider = build_tracer_provider(self._config_with_headers(headers))
|
||||
provider = build_tracer_provider(
|
||||
self._config_with_destinations(destinations)
|
||||
)
|
||||
self._providers[cache_key] = provider
|
||||
if len(self._providers) > _MAX_CACHED_PROVIDERS:
|
||||
_, evicted = self._providers.popitem(last=False)
|
||||
_shutdown_provider(evicted)
|
||||
return get_tracer(provider, self._tracer_name)
|
||||
|
||||
def _config_with_headers(self, headers: Mapping[str, str]) -> OpenTelemetryV2Config:
|
||||
"""Clone the config, stamping ``headers`` onto the credential's own exporter.
|
||||
|
||||
``headers`` are the per-request credentials of ``self._callback_name`` (the
|
||||
integration that built this cache), so they apply only to the exporter that
|
||||
integration contributed (``spec.owner``). A request that carries one
|
||||
tenant's Arize key must never rewrite the headers of a co-configured
|
||||
Langfuse or self-hosted collector exporter, which would leak that key to a
|
||||
different backend.
|
||||
"""
|
||||
header_str = ",".join(f"{key}={value}" for key, value in headers.items())
|
||||
header_update: dict[str, str] = {"headers": header_str}
|
||||
exporters = [
|
||||
(
|
||||
spec.model_copy(update=header_update)
|
||||
if spec.owner == self._callback_name
|
||||
def _owned_otlp_kind(self) -> str:
|
||||
"""The OTLP transport of this integration's own exporter (langfuse -> http,
|
||||
arize -> grpc), used for the destinations appended below."""
|
||||
for spec in self._config.exporters:
|
||||
if (
|
||||
spec.owner == self._callback_name
|
||||
and spec.kind.lower() not in _NON_OTLP_KINDS
|
||||
else spec
|
||||
):
|
||||
return spec.kind
|
||||
return "otlp_http"
|
||||
|
||||
def _config_with_destinations(
|
||||
self, destinations: "tuple[OtelDestination, ...]"
|
||||
) -> OpenTelemetryV2Config:
|
||||
"""Clone the config, KEEPING its exporters (so the global/default still
|
||||
receives) and APPENDING one exporter per resolved destination. The shared
|
||||
``TracerProvider`` attaches one ``SpanProcessor`` per spec, so a single span
|
||||
is emitted once and exported to the global destination plus every assigned
|
||||
one. Each appended exporter's endpoint is the resolved host (the cross-host
|
||||
fix) with its own auth headers (per-destination isolation)."""
|
||||
kind = self._owned_otlp_kind()
|
||||
appended = [
|
||||
ExporterSpec(
|
||||
kind=kind,
|
||||
endpoint=d.endpoint,
|
||||
headers=d.header_string(),
|
||||
owner=None,
|
||||
)
|
||||
for spec in self._config.exporters
|
||||
for d in destinations
|
||||
]
|
||||
return self._config.model_copy(update={"exporters": exporters})
|
||||
return self._config.model_copy(
|
||||
update={"exporters": [*self._config.exporters, *appended]}
|
||||
)
|
||||
|
|
|
|||
|
|
@ -6,22 +6,22 @@ vocabularies to apply, and any resource attributes. ``PRESET_BY_CALLBACK``
|
|||
maps a callback name (``"arize"``, ``"langfuse_otel"``, ...) to its preset so
|
||||
the factory in ``litellm_logging`` can resolve a name and build a single
|
||||
``OpenTelemetryV2`` instance from the result.
|
||||
|
||||
Per-key/team routing does not live here. A trace destination is admin-owned
|
||||
infrastructure config, resolved server-side from a named credential into an
|
||||
``OtelDestination`` (see ``litellm.integrations.otel.destinations`` and
|
||||
``plumbing.routing``); nothing in this package reads vendor credentials or a
|
||||
host off a request.
|
||||
"""
|
||||
|
||||
from typing import Callable
|
||||
|
||||
from litellm.integrations.otel.presets.agentops import agentops_preset
|
||||
from litellm.integrations.otel.presets.arize import arize_dynamic_headers, arize_preset
|
||||
from litellm.integrations.otel.presets.arize import arize_preset
|
||||
from litellm.integrations.otel.presets.base import Preset
|
||||
from litellm.integrations.otel.presets.langfuse import (
|
||||
langfuse_dynamic_headers,
|
||||
langfuse_preset,
|
||||
)
|
||||
from litellm.integrations.otel.presets.langfuse import langfuse_preset
|
||||
from litellm.integrations.otel.presets.langtrace import langtrace_preset
|
||||
from litellm.integrations.otel.presets.levo import levo_preset
|
||||
from litellm.integrations.otel.presets.phoenix import phoenix_preset
|
||||
from litellm.integrations.otel.presets.weave import weave_dynamic_headers, weave_preset
|
||||
from litellm.types.utils import StandardCallbackDynamicParams
|
||||
from litellm.integrations.otel.presets.weave import weave_preset
|
||||
|
||||
#: Callback name → preset. The ``Preset`` annotation makes mypy verify every
|
||||
#: registered value matches the preset interface.
|
||||
|
|
@ -35,39 +35,10 @@ PRESET_BY_CALLBACK: dict[str, Preset] = {
|
|||
"weave_otel": weave_preset,
|
||||
}
|
||||
|
||||
#: Callback name → per-request OTLP header builder (team/key multi-tenant
|
||||
#: routing). Only integrations that support dynamic credentials appear here —
|
||||
#: Arize-Phoenix/Langtrace/Levo/AgentOps don't, so they use the logger's
|
||||
#: default tracer.
|
||||
DYNAMIC_HEADERS_BY_CALLBACK: dict[
|
||||
str, Callable[[StandardCallbackDynamicParams], dict[str, str]]
|
||||
] = {
|
||||
"arize": arize_dynamic_headers,
|
||||
"langfuse_otel": langfuse_dynamic_headers,
|
||||
"weave_otel": weave_dynamic_headers,
|
||||
}
|
||||
|
||||
|
||||
def dynamic_otlp_headers(
|
||||
callback_name: str | None,
|
||||
dynamic_params: StandardCallbackDynamicParams | None,
|
||||
) -> dict[str, str] | None:
|
||||
"""Per-request OTLP headers for ``callback_name``, or ``None`` if N/A.
|
||||
|
||||
``None`` means "no per-request routing" — the caller uses its default tracer.
|
||||
"""
|
||||
builder = DYNAMIC_HEADERS_BY_CALLBACK.get(callback_name or "")
|
||||
if builder is None or not dynamic_params:
|
||||
return None
|
||||
headers = builder(dynamic_params)
|
||||
return headers or None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"PRESET_BY_CALLBACK",
|
||||
"DYNAMIC_HEADERS_BY_CALLBACK",
|
||||
"Preset",
|
||||
"dynamic_otlp_headers",
|
||||
"agentops_preset",
|
||||
"arize_preset",
|
||||
"langfuse_preset",
|
||||
|
|
|
|||
|
|
@ -10,7 +10,6 @@ from litellm.integrations.otel.model.config import (
|
|||
OpenTelemetryV2Config,
|
||||
)
|
||||
from litellm.integrations.otel.presets.utils import ensure_mappers
|
||||
from litellm.types.utils import StandardCallbackDynamicParams
|
||||
|
||||
|
||||
class _ArizeSettings(BaseSettings):
|
||||
|
|
@ -65,16 +64,3 @@ def _arize_headers(arize_cfg) -> str | None:
|
|||
# credentials are configured.
|
||||
return _ArizeSettings().otlp_traces_headers
|
||||
return ",".join(pieces)
|
||||
|
||||
|
||||
def arize_dynamic_headers(params: StandardCallbackDynamicParams) -> dict[str, str]:
|
||||
"""Per-request Arize OTLP headers from team/key dynamic params."""
|
||||
headers: dict[str, str] = {}
|
||||
# ``arize_space_key`` is the suggested param and wins over ``arize_space_id``.
|
||||
space = params.get("arize_space_key") or params.get("arize_space_id")
|
||||
if space:
|
||||
headers["arize-space-id"] = space
|
||||
api_key = params.get("arize_api_key")
|
||||
if api_key:
|
||||
headers["api_key"] = api_key
|
||||
return headers
|
||||
|
|
|
|||
|
|
@ -9,7 +9,6 @@ from litellm.integrations.otel.model.config import (
|
|||
OpenTelemetryV2Config,
|
||||
)
|
||||
from litellm.integrations.otel.presets.utils import ensure_mappers
|
||||
from litellm.types.utils import StandardCallbackDynamicParams
|
||||
|
||||
|
||||
def langfuse_preset(
|
||||
|
|
@ -33,16 +32,3 @@ def langfuse_preset(
|
|||
"mapper_names": ensure_mappers(base.mapper_names, "langfuse"),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def langfuse_dynamic_headers(params: StandardCallbackDynamicParams) -> dict[str, str]:
|
||||
"""Per-request Langfuse OTLP headers from team/key dynamic params."""
|
||||
public_key = params.get("langfuse_public_key")
|
||||
secret_key = params.get("langfuse_secret_key")
|
||||
if public_key and secret_key:
|
||||
return {
|
||||
"Authorization": _V1Langfuse._get_langfuse_authorization_header(
|
||||
public_key=public_key, secret_key=secret_key
|
||||
)
|
||||
}
|
||||
return {}
|
||||
|
|
|
|||
|
|
@ -6,11 +6,7 @@ from litellm.integrations.otel.model.config import (
|
|||
OpenTelemetryV2Config,
|
||||
)
|
||||
from litellm.integrations.otel.presets.utils import ensure_mappers
|
||||
from litellm.integrations.weave.weave_otel import (
|
||||
_get_weave_authorization_header,
|
||||
get_weave_otel_config,
|
||||
)
|
||||
from litellm.types.utils import StandardCallbackDynamicParams
|
||||
from litellm.integrations.weave.weave_otel import get_weave_otel_config
|
||||
|
||||
|
||||
def weave_preset(
|
||||
|
|
@ -34,15 +30,3 @@ def weave_preset(
|
|||
"mapper_names": ensure_mappers(base.mapper_names, "openinference", "weave"),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def weave_dynamic_headers(params: StandardCallbackDynamicParams) -> dict[str, str]:
|
||||
"""Per-request Weave OTLP headers from team/key dynamic params."""
|
||||
headers: dict[str, str] = {}
|
||||
api_key = params.get("wandb_api_key")
|
||||
if api_key:
|
||||
headers["Authorization"] = _get_weave_authorization_header(api_key=api_key)
|
||||
project_id = params.get("weave_project_id")
|
||||
if project_id:
|
||||
headers["project_id"] = project_id
|
||||
return headers
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
from typing import Dict, Optional
|
||||
from typing import Dict, Optional, cast
|
||||
|
||||
from litellm.types.utils import StandardCallbackDynamicParams
|
||||
from litellm.types.utils import OtelDestinationParams, StandardCallbackDynamicParams
|
||||
|
||||
|
||||
def _is_env_reference(value: object) -> bool:
|
||||
|
|
@ -108,4 +108,15 @@ def initialize_standard_callback_dynamic_params(
|
|||
)
|
||||
standard_callback_dynamic_params[param] = _param_value # type: ignore
|
||||
|
||||
# Admin-owned OTEL v2 destinations, resolved server-side by the proxy from the
|
||||
# exporters assigned to the request's identity chain and stamped onto top-level
|
||||
# kwargs (the proxy strips any client-supplied value first). Read from top-level
|
||||
# only, never from request metadata, and never via _supported_callback_params,
|
||||
# so a request body/metadata cannot set or select a trace destination.
|
||||
otel_destinations = kwargs.get("otel_destinations")
|
||||
if isinstance(otel_destinations, list):
|
||||
standard_callback_dynamic_params["otel_destinations"] = cast(
|
||||
"list[OtelDestinationParams]", otel_destinations
|
||||
)
|
||||
|
||||
return standard_callback_dynamic_params
|
||||
|
|
|
|||
|
|
@ -16,7 +16,10 @@ from pydantic import (
|
|||
from typing_extensions import Required, TypedDict
|
||||
|
||||
from litellm._uuid import uuid
|
||||
from litellm.constants import MCP_STDIO_ALLOWED_COMMANDS
|
||||
from litellm.constants import (
|
||||
LITELLM_LOGGING_CREDENTIAL_NAME_KEY,
|
||||
MCP_STDIO_ALLOWED_COMMANDS,
|
||||
)
|
||||
from litellm.litellm_core_utils.initialize_dynamic_callback_params import (
|
||||
validate_no_callback_env_reference,
|
||||
)
|
||||
|
|
@ -1883,7 +1886,9 @@ class AddTeamCallback(LiteLLMPydanticObjectBase):
|
|||
@classmethod
|
||||
def validate_callback_vars(cls, values):
|
||||
callback_vars = values.get("callback_vars", {})
|
||||
valid_keys = set(StandardCallbackDynamicParams.__annotations__.keys())
|
||||
valid_keys = set(StandardCallbackDynamicParams.__annotations__.keys()) | {
|
||||
LITELLM_LOGGING_CREDENTIAL_NAME_KEY
|
||||
}
|
||||
for key, value in callback_vars.items():
|
||||
if key not in valid_keys:
|
||||
raise ValueError(
|
||||
|
|
@ -1926,7 +1931,9 @@ class TeamCallbackMetadata(LiteLLMPydanticObjectBase):
|
|||
"callbacks": [],
|
||||
"callback_vars": {},
|
||||
}
|
||||
valid_keys = set(StandardCallbackDynamicParams.__annotations__.keys())
|
||||
valid_keys = set(StandardCallbackDynamicParams.__annotations__.keys()) | {
|
||||
LITELLM_LOGGING_CREDENTIAL_NAME_KEY
|
||||
}
|
||||
if callback_vars is not None:
|
||||
for key in callback_vars:
|
||||
if key not in valid_keys:
|
||||
|
|
|
|||
|
|
@ -10,9 +10,13 @@ import litellm
|
|||
from litellm._logging import verbose_proxy_logger
|
||||
from litellm.litellm_core_utils.credential_accessor import CredentialAccessor
|
||||
from litellm.litellm_core_utils.litellm_logging import _get_masked_values
|
||||
from litellm.proxy._types import CommonProxyErrors, UserAPIKeyAuth
|
||||
from litellm.proxy._types import CommonProxyErrors, LitellmUserRoles, UserAPIKeyAuth
|
||||
from litellm.proxy.auth.user_api_key_auth import user_api_key_auth
|
||||
from litellm.proxy.common_utils.encrypt_decrypt_utils import encrypt_value_helper
|
||||
from litellm.proxy.management_endpoints.logging_exporter_validation import (
|
||||
is_admin_gated_credential_info,
|
||||
validate_credential_access,
|
||||
)
|
||||
from litellm.proxy.utils import handle_exception_on_proxy, jsonify_object
|
||||
from litellm.repositories.credentials_repository import CredentialsRepository
|
||||
from litellm.types.utils import CreateCredentialItem, CredentialItem
|
||||
|
|
@ -20,6 +24,25 @@ from litellm.types.utils import CreateCredentialItem, CredentialItem
|
|||
router = APIRouter()
|
||||
|
||||
|
||||
def _require_proxy_admin(user_api_key_dict: UserAPIKeyAuth) -> None:
|
||||
if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail={"error": "Only the proxy admin can manage logging credentials"},
|
||||
)
|
||||
|
||||
|
||||
def _credential_in_memory(credential_name: str) -> Optional[CredentialItem]:
|
||||
return next(
|
||||
(
|
||||
cred
|
||||
for cred in litellm.credential_list
|
||||
if cred.credential_name == credential_name
|
||||
),
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
class CredentialHelperUtils:
|
||||
@staticmethod
|
||||
def encrypt_credential_values(
|
||||
|
|
@ -59,6 +82,10 @@ async def create_credential(
|
|||
"""
|
||||
from litellm.proxy.proxy_server import llm_router, prisma_client
|
||||
|
||||
if is_admin_gated_credential_info(credential.credential_info):
|
||||
_require_proxy_admin(user_api_key_dict)
|
||||
validate_credential_access(credential.credential_info)
|
||||
|
||||
try:
|
||||
if prisma_client is None:
|
||||
raise HTTPException(
|
||||
|
|
@ -240,6 +267,12 @@ async def delete_credential(
|
|||
"""
|
||||
from litellm.proxy.proxy_server import prisma_client
|
||||
|
||||
existing = _credential_in_memory(credential_name)
|
||||
if existing is not None and is_admin_gated_credential_info(
|
||||
existing.credential_info
|
||||
):
|
||||
_require_proxy_admin(user_api_key_dict)
|
||||
|
||||
try:
|
||||
if prisma_client is None:
|
||||
raise HTTPException(
|
||||
|
|
@ -290,10 +323,12 @@ def update_db_credential(
|
|||
|
||||
merged_credential.credential_values.update(encrypted_params)
|
||||
|
||||
# update model info
|
||||
# Merge the patch into the existing credential_info so a partial update (e.g. only
|
||||
# access) preserves credential_type/description/host. The prior guard checked for a
|
||||
# key literally named "credential_info", which is never present, so it reset the dict
|
||||
# on every patch and dropped the logging tag.
|
||||
if encrypted_credential.credential_info:
|
||||
"""Update credential info"""
|
||||
if "credential_info" not in merged_credential.credential_info:
|
||||
if merged_credential.credential_info is None:
|
||||
merged_credential.credential_info = {}
|
||||
merged_credential.credential_info.update(encrypted_credential.credential_info)
|
||||
|
||||
|
|
@ -319,6 +354,14 @@ async def update_credential(
|
|||
"""
|
||||
from litellm.proxy.proxy_server import prisma_client
|
||||
|
||||
existing = _credential_in_memory(credential_name)
|
||||
if is_admin_gated_credential_info(credential.credential_info) or (
|
||||
existing is not None
|
||||
and is_admin_gated_credential_info(existing.credential_info)
|
||||
):
|
||||
_require_proxy_admin(user_api_key_dict)
|
||||
validate_credential_access(credential.credential_info)
|
||||
|
||||
try:
|
||||
if prisma_client is None:
|
||||
raise HTTPException(
|
||||
|
|
|
|||
|
|
@ -83,6 +83,8 @@ _ENABLE_TEAM_STALE_ALIAS_BYPASS: Optional[bool] = None
|
|||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from litellm.integrations.otel.model.destination import OtelDestination
|
||||
from litellm.models.credentials import CredentialItem
|
||||
from litellm.proxy.proxy_server import ProxyConfig as _ProxyConfig
|
||||
from litellm.types.proxy.policy_engine import PolicyMatchContext
|
||||
|
||||
|
|
@ -538,6 +540,161 @@ class KeyAndTeamLoggingSettings:
|
|||
return None
|
||||
|
||||
|
||||
async def _union_logging_exporter_names(user_api_key_dict: UserAPIKeyAuth) -> set:
|
||||
"""The union of admin-assigned exporter names across the request's identity chain.
|
||||
|
||||
Resolves each level from its OWN record: the key's ``metadata`` is shadowed by the
|
||||
team's on the auth object, so it is fetched fresh via ``get_key_object``; the org's
|
||||
metadata is fetched via ``get_org_object``; the team's is already its own on
|
||||
``team_metadata``. Internal-user is intentionally not a routing dimension. The lists
|
||||
are admin-owned; the request never supplies them. Degrades to team-only when no DB
|
||||
is connected (SDK mode).
|
||||
"""
|
||||
from litellm.proxy import proxy_server
|
||||
from litellm.proxy.auth.auth_checks import get_key_object, get_org_object
|
||||
|
||||
names: set = set()
|
||||
|
||||
def _add(metadata: Any) -> None:
|
||||
if not isinstance(metadata, dict):
|
||||
return
|
||||
assigned = metadata.get("logging_exporters")
|
||||
if isinstance(assigned, list):
|
||||
names.update(str(name) for name in assigned)
|
||||
|
||||
prisma_client = proxy_server.prisma_client
|
||||
cache = proxy_server.user_api_key_cache
|
||||
span = getattr(user_api_key_dict, "parent_otel_span", None)
|
||||
|
||||
# KEY: the key's own metadata (the auth object's .metadata is the team's shadow).
|
||||
if user_api_key_dict.token and prisma_client is not None:
|
||||
try:
|
||||
key_obj = await get_key_object(
|
||||
hashed_token=user_api_key_dict.token,
|
||||
prisma_client=prisma_client,
|
||||
user_api_key_cache=cache,
|
||||
parent_otel_span=span,
|
||||
proxy_logging_obj=proxy_server.proxy_logging_obj,
|
||||
)
|
||||
_add(key_obj.metadata)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# TEAM: team_metadata is already the team's own.
|
||||
_add(user_api_key_dict.team_metadata)
|
||||
|
||||
# ORG: the org's own metadata (central catch-all).
|
||||
if user_api_key_dict.org_id and prisma_client is not None:
|
||||
try:
|
||||
org_obj = await get_org_object(
|
||||
org_id=user_api_key_dict.org_id,
|
||||
prisma_client=prisma_client,
|
||||
user_api_key_cache=cache,
|
||||
parent_otel_span=span,
|
||||
proxy_logging_obj=proxy_server.proxy_logging_obj,
|
||||
)
|
||||
_add(getattr(org_obj, "metadata", None))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return names
|
||||
|
||||
|
||||
def _access_matches(access: Any, team_id: Optional[str], org_id: Optional[str]) -> bool:
|
||||
"""Whether an admin-owned destination's ``access`` grants this caller.
|
||||
|
||||
``global`` reaches everyone; otherwise the caller's team or org must be listed.
|
||||
Per-key access is intentionally absent: a key's token rotates on regenerate, so
|
||||
per-key assignment lives on the key's own ``logging_exporters`` instead.
|
||||
"""
|
||||
if not isinstance(access, dict):
|
||||
return False
|
||||
if access.get("global") is True:
|
||||
return True
|
||||
teams = access.get("teams")
|
||||
if team_id is not None and isinstance(teams, (list, tuple)) and team_id in teams:
|
||||
return True
|
||||
orgs = access.get("orgs")
|
||||
return org_id is not None and isinstance(orgs, (list, tuple)) and org_id in orgs
|
||||
|
||||
|
||||
async def _resolve_logging_exporters(
|
||||
user_api_key_dict: UserAPIKeyAuth,
|
||||
) -> "tuple[list, list]":
|
||||
"""Resolve the destinations this request fans out to, as (destinations, backends).
|
||||
|
||||
The selected set is the union of the identity chain's assigned exporter names
|
||||
(key + team + org ``logging_exporters``) and every admin-owned logging destination
|
||||
whose ``credential_info.access`` matches the caller (global, or the caller's team
|
||||
or org). Each survivor is built via ``build_destination`` and deduped on (endpoint,
|
||||
headers). The request never names or supplies a destination. Returns ([], []) only
|
||||
when nothing is selected (default-deny).
|
||||
"""
|
||||
from litellm.integrations.otel.destinations import build_destination
|
||||
|
||||
names = await _union_logging_exporter_names(user_api_key_dict)
|
||||
team_id, org_id = user_api_key_dict.team_id, user_api_key_dict.org_id
|
||||
|
||||
def _selected(credential: "CredentialItem") -> bool:
|
||||
info = credential.credential_info or {}
|
||||
if info.get("credential_type") != "logging":
|
||||
return False
|
||||
if credential.credential_name in names:
|
||||
return True
|
||||
return _access_matches(info.get("access"), team_id, org_id)
|
||||
|
||||
def _build(
|
||||
credential: "CredentialItem",
|
||||
) -> "Optional[tuple[str, OtelDestination]]":
|
||||
backend = (credential.credential_info or {}).get("description")
|
||||
if not backend:
|
||||
return None
|
||||
values = {
|
||||
str(key): str(value)
|
||||
for key, value in (credential.credential_values or {}).items()
|
||||
}
|
||||
destination = build_destination(backend, values)
|
||||
return None if destination is None else (backend, destination)
|
||||
|
||||
built = tuple(
|
||||
result
|
||||
for credential in litellm.credential_list
|
||||
if _selected(credential)
|
||||
if (result := _build(credential)) is not None
|
||||
)
|
||||
deduped = {
|
||||
(destination.endpoint, tuple(sorted(destination.headers.items()))): (
|
||||
backend,
|
||||
destination,
|
||||
)
|
||||
for backend, destination in built
|
||||
}
|
||||
destinations = [
|
||||
{
|
||||
"callback_name": backend,
|
||||
"endpoint": destination.endpoint,
|
||||
"headers": destination.headers,
|
||||
}
|
||||
for backend, destination in deduped.values()
|
||||
]
|
||||
backends = list(dict.fromkeys(backend for backend, _ in deduped.values()))
|
||||
return destinations, backends
|
||||
|
||||
|
||||
async def _apply_admin_logging_exporters(
|
||||
data: dict, user_api_key_dict: UserAPIKeyAuth
|
||||
) -> None:
|
||||
"""Stamp the resolved fan-out destinations onto ``data`` and activate their
|
||||
backends. A client value was already stripped by the caller; default-deny means
|
||||
an identity with no assignment gets no per-tenant destination here."""
|
||||
destinations, backends = await _resolve_logging_exporters(user_api_key_dict)
|
||||
if not destinations:
|
||||
return
|
||||
data["otel_destinations"] = destinations
|
||||
existing = data.get("success_callback") or []
|
||||
data["success_callback"] = list(dict.fromkeys([*existing, *backends]))
|
||||
|
||||
|
||||
def _get_dynamic_logging_metadata(
|
||||
user_api_key_dict: UserAPIKeyAuth, proxy_config: ProxyConfig
|
||||
) -> Optional[TeamCallbackMetadata]:
|
||||
|
|
@ -1825,6 +1982,9 @@ async def add_litellm_data_to_request(
|
|||
)
|
||||
|
||||
# Team Callbacks controls
|
||||
# A client must never set or override OTEL destinations; they are admin-owned and
|
||||
# resolved server-side below, so drop any value carried in the request.
|
||||
data.pop("otel_destinations", None)
|
||||
callback_settings_obj = _get_dynamic_logging_metadata(
|
||||
user_api_key_dict=user_api_key_dict, proxy_config=proxy_config
|
||||
)
|
||||
|
|
@ -1833,10 +1993,14 @@ async def add_litellm_data_to_request(
|
|||
data["failure_callback"] = callback_settings_obj.failure_callback
|
||||
|
||||
if callback_settings_obj.callback_vars is not None:
|
||||
# unpack callback_vars in data
|
||||
for k, v in callback_settings_obj.callback_vars.items():
|
||||
data[k] = v
|
||||
|
||||
# Admin-owned exporter assignment: resolve the union of exporters assigned across
|
||||
# the request's identity chain (key + team + org) into fan-out destinations and
|
||||
# activate their backends. Default-deny: an unassigned identity gets none.
|
||||
await _apply_admin_logging_exporters(data, user_api_key_dict)
|
||||
|
||||
# Add disabled callbacks from key metadata
|
||||
if (
|
||||
user_api_key_dict.metadata
|
||||
|
|
|
|||
|
|
@ -1465,6 +1465,11 @@ async def generate_key_fn(
|
|||
- user_id: (str) Unique user id - used for tracking spend across multiple keys for same user id.
|
||||
"""
|
||||
try:
|
||||
from litellm.proxy.management_endpoints.logging_exporter_validation import (
|
||||
validate_logging_exporter_assignment,
|
||||
)
|
||||
|
||||
validate_logging_exporter_assignment(data.metadata, user_api_key_dict)
|
||||
from litellm.proxy._types import CommonProxyErrors
|
||||
from litellm.proxy.proxy_server import (
|
||||
prisma_client,
|
||||
|
|
@ -2510,6 +2515,11 @@ async def update_key_fn(
|
|||
}'
|
||||
```
|
||||
"""
|
||||
from litellm.proxy.management_endpoints.logging_exporter_validation import (
|
||||
validate_logging_exporter_assignment,
|
||||
)
|
||||
|
||||
validate_logging_exporter_assignment(data.metadata, user_api_key_dict)
|
||||
from litellm.proxy.proxy_server import (
|
||||
llm_router,
|
||||
premium_user,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,105 @@
|
|||
"""Validation for admin-owned logging-exporter assignment on key/team/org.
|
||||
|
||||
An identity's ``metadata.logging_exporters`` binds it to admin-owned trace
|
||||
destinations. Assigning is proxy-admin only, and every name must be a registered
|
||||
logging credential, so a key/team/org can only point at destinations the admin has
|
||||
provisioned. The resolver (``litellm_pre_call_utils``) trusts this at request time.
|
||||
"""
|
||||
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import HTTPException, status
|
||||
|
||||
import litellm
|
||||
from litellm.proxy._types import LitellmUserRoles, UserAPIKeyAuth
|
||||
|
||||
LOGGING_EXPORTERS_KEY = "logging_exporters"
|
||||
|
||||
|
||||
def is_admin_gated_credential_info(credential_info: Optional[dict]) -> bool:
|
||||
"""Whether a credential write must be proxy-admin only.
|
||||
|
||||
True when the credential is a logging destination or carries an ``access`` grant,
|
||||
since both control where other tenants' traces are exported.
|
||||
"""
|
||||
if not isinstance(credential_info, dict):
|
||||
return False
|
||||
return (
|
||||
credential_info.get("credential_type") == "logging"
|
||||
or "access" in credential_info
|
||||
)
|
||||
|
||||
|
||||
def validate_credential_access(credential_info: Optional[dict]) -> None:
|
||||
"""Validate ``credential_info.access`` shape when the write sets one.
|
||||
|
||||
No-op when ``access`` is absent. Otherwise it must be an object whose ``global`` (if
|
||||
present) is a bool and whose ``teams``/``orgs`` (if present) are lists of strings.
|
||||
Per-key access is intentionally unsupported on a destination.
|
||||
"""
|
||||
if not isinstance(credential_info, dict) or "access" not in credential_info:
|
||||
return
|
||||
access = credential_info["access"]
|
||||
if not isinstance(access, dict):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail={"error": "credential_info.access must be an object"},
|
||||
)
|
||||
if "global" in access and not isinstance(access["global"], bool):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail={"error": "access.global must be a boolean"},
|
||||
)
|
||||
for field in ("teams", "orgs"):
|
||||
bucket = access.get(field)
|
||||
if bucket is not None and not (
|
||||
isinstance(bucket, list) and all(isinstance(item, str) for item in bucket)
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail={"error": f"access.{field} must be a list of strings"},
|
||||
)
|
||||
|
||||
|
||||
def _logging_credential_names() -> set:
|
||||
return {
|
||||
credential.credential_name
|
||||
for credential in litellm.credential_list
|
||||
if (credential.credential_info or {}).get("credential_type") == "logging"
|
||||
}
|
||||
|
||||
|
||||
def validate_logging_exporter_assignment(
|
||||
metadata: Optional[dict], user_api_key_dict: UserAPIKeyAuth
|
||||
) -> None:
|
||||
"""Validate a ``metadata.logging_exporters`` assignment, if the update sets one.
|
||||
|
||||
No-op when the update does not touch ``logging_exporters``. Otherwise it must be a
|
||||
list, the caller must be the proxy admin, and every name must be a registered
|
||||
logging credential.
|
||||
"""
|
||||
if not isinstance(metadata, dict) or LOGGING_EXPORTERS_KEY not in metadata:
|
||||
return
|
||||
exporters = metadata.get(LOGGING_EXPORTERS_KEY)
|
||||
if not isinstance(exporters, list):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail={"error": "logging_exporters must be a list of credential names"},
|
||||
)
|
||||
if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail={"error": "Only the proxy admin can assign logging exporters"},
|
||||
)
|
||||
known = _logging_credential_names()
|
||||
unknown = [name for name in exporters if name not in known]
|
||||
if unknown:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail={
|
||||
"error": (
|
||||
f"Unknown or non-logging credential(s): {unknown}. Register them "
|
||||
"as logging credentials before assigning."
|
||||
)
|
||||
},
|
||||
)
|
||||
|
|
@ -201,6 +201,13 @@ async def new_organization(
|
|||
}'
|
||||
```
|
||||
"""
|
||||
from litellm.proxy.management_endpoints.logging_exporter_validation import (
|
||||
validate_logging_exporter_assignment,
|
||||
)
|
||||
|
||||
validate_logging_exporter_assignment(
|
||||
getattr(data, "metadata", None), user_api_key_dict
|
||||
)
|
||||
|
||||
from litellm.proxy.proxy_server import (
|
||||
litellm_proxy_admin_name,
|
||||
|
|
@ -496,6 +503,14 @@ async def update_organization(
|
|||
# Create validated data model
|
||||
data = LiteLLM_OrganizationTableUpdate(**raw_data_with_flat_budget_fields)
|
||||
|
||||
from litellm.proxy.management_endpoints.logging_exporter_validation import (
|
||||
validate_logging_exporter_assignment,
|
||||
)
|
||||
|
||||
validate_logging_exporter_assignment(
|
||||
getattr(data, "metadata", None), user_api_key_dict
|
||||
)
|
||||
|
||||
# Validate budget values are not negative
|
||||
if data.max_budget is not None and (
|
||||
not math.isfinite(data.max_budget) or data.max_budget < 0
|
||||
|
|
|
|||
|
|
@ -1022,6 +1022,11 @@ async def new_team(
|
|||
```
|
||||
"""
|
||||
try:
|
||||
from litellm.proxy.management_endpoints.logging_exporter_validation import (
|
||||
validate_logging_exporter_assignment,
|
||||
)
|
||||
|
||||
validate_logging_exporter_assignment(data.metadata, user_api_key_dict)
|
||||
from litellm.proxy.management_helpers.audit_logs import (
|
||||
get_audit_log_changed_by,
|
||||
)
|
||||
|
|
@ -1711,6 +1716,11 @@ async def update_team(
|
|||
```
|
||||
"""
|
||||
try:
|
||||
from litellm.proxy.management_endpoints.logging_exporter_validation import (
|
||||
validate_logging_exporter_assignment,
|
||||
)
|
||||
|
||||
validate_logging_exporter_assignment(data.metadata, user_api_key_dict)
|
||||
from litellm.proxy.proxy_server import (
|
||||
litellm_proxy_admin_name,
|
||||
llm_router,
|
||||
|
|
|
|||
|
|
@ -3037,6 +3037,20 @@ OPENAI_RESPONSE_HEADERS = [
|
|||
]
|
||||
|
||||
|
||||
class OtelDestinationParams(TypedDict, total=False):
|
||||
"""A resolved, admin-owned OTLP destination carried server-side only.
|
||||
|
||||
Populated by the proxy from the exporters assigned to a request's identity
|
||||
chain; never read from a request body or metadata. The v2 logger validates and
|
||||
exports through it. ``callback_name`` is the OTEL backend this destination
|
||||
belongs to, so fan-out routes each destination to the right backend's logger.
|
||||
"""
|
||||
|
||||
callback_name: str
|
||||
endpoint: str
|
||||
headers: Dict[str, str]
|
||||
|
||||
|
||||
class StandardCallbackDynamicParams(TypedDict, total=False):
|
||||
# Langfuse dynamic params
|
||||
langfuse_public_key: Optional[str]
|
||||
|
|
@ -3084,6 +3098,12 @@ class StandardCallbackDynamicParams(TypedDict, total=False):
|
|||
turn_off_message_logging: Optional[bool] # when true will not log messages
|
||||
litellm_disabled_callbacks: Optional[List[str]]
|
||||
|
||||
# Admin-owned OTEL v2 destinations, resolved server-side from the exporters
|
||||
# assigned to the request's identity chain (key/team/user/org), fanned out to.
|
||||
# Never request-settable: absent from the request-read whitelist in
|
||||
# initialize_dynamic_callback_params, so a request body/metadata cannot set it.
|
||||
otel_destinations: Optional[List[OtelDestinationParams]]
|
||||
|
||||
|
||||
class CustomPricingLiteLLMParams(BaseModel):
|
||||
## CUSTOM PRICING ##
|
||||
|
|
|
|||
105
tests/test_litellm/integrations/otel/test_destinations.py
Normal file
105
tests/test_litellm/integrations/otel/test_destinations.py
Normal file
|
|
@ -0,0 +1,105 @@
|
|||
"""``build_destination`` maps an admin credential to a generic OTLP destination.
|
||||
|
||||
The point of these tests is that the resolution is backend-agnostic: Langfuse,
|
||||
Arize, Weave, and any raw collector all resolve to an ``{endpoint, headers}``
|
||||
the router exports through, and an incomplete credential resolves to nothing.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.abspath("../../../.."))
|
||||
|
||||
from litellm.integrations.otel.destinations import (
|
||||
OTEL_V2_DESTINATION_CALLBACKS,
|
||||
build_destination,
|
||||
)
|
||||
|
||||
|
||||
def test_langfuse_endpoint_derived_from_host_with_basic_auth():
|
||||
dest = build_destination(
|
||||
"langfuse_otel",
|
||||
{
|
||||
"langfuse_host": "https://cloud.langfuse.com",
|
||||
"langfuse_public_key": "pk-eu",
|
||||
"langfuse_secret_key": "sk-eu",
|
||||
},
|
||||
)
|
||||
assert dest is not None
|
||||
assert dest.endpoint == "https://cloud.langfuse.com/api/public/otel"
|
||||
scheme, b64 = dest.headers["Authorization"].split(" ", 1)
|
||||
assert scheme == "Basic"
|
||||
assert base64.b64decode(b64).decode() == "pk-eu:sk-eu"
|
||||
|
||||
|
||||
def test_langfuse_bare_host_gets_https_and_path():
|
||||
dest = build_destination(
|
||||
"langfuse_otel",
|
||||
{
|
||||
"langfuse_host": "my-langfuse.internal",
|
||||
"langfuse_public_key": "pk",
|
||||
"langfuse_secret_key": "sk",
|
||||
},
|
||||
)
|
||||
assert dest is not None
|
||||
assert dest.endpoint == "https://my-langfuse.internal/api/public/otel"
|
||||
|
||||
|
||||
def test_langfuse_without_host_defaults_to_us_cloud():
|
||||
dest = build_destination(
|
||||
"langfuse_otel",
|
||||
{"langfuse_public_key": "pk", "langfuse_secret_key": "sk"},
|
||||
)
|
||||
assert dest is not None
|
||||
assert dest.endpoint == "https://us.cloud.langfuse.com/api/public/otel"
|
||||
|
||||
|
||||
def test_langfuse_incomplete_returns_none():
|
||||
assert build_destination("langfuse_otel", {"langfuse_public_key": "pk"}) is None
|
||||
|
||||
|
||||
def test_arize_space_and_api_key_headers():
|
||||
dest = build_destination("arize", {"arize_space_id": "S", "arize_api_key": "K"})
|
||||
assert dest is not None
|
||||
assert dest.endpoint == "https://otlp.arize.com/v1"
|
||||
assert dest.headers == {"space_id": "S", "api_key": "K"}
|
||||
|
||||
|
||||
def test_weave_requires_endpoint_and_key():
|
||||
assert build_destination("weave_otel", {"wandb_api_key": "w"}) is None
|
||||
dest = build_destination(
|
||||
"weave_otel",
|
||||
{
|
||||
"wandb_api_key": "w",
|
||||
"weave_endpoint": "https://trace.wandb.ai/otel/v1/traces",
|
||||
"weave_project_id": "entity/project",
|
||||
},
|
||||
)
|
||||
assert dest is not None
|
||||
assert dest.endpoint == "https://trace.wandb.ai/otel/v1/traces"
|
||||
assert dest.headers["project_id"] == "entity/project"
|
||||
assert "Authorization" in dest.headers
|
||||
|
||||
|
||||
def test_generic_passthrough_covers_any_backend():
|
||||
dest = build_destination(
|
||||
"some_self_hosted_collector",
|
||||
{
|
||||
"otel_endpoint": "https://collector.internal:4318/v1/traces",
|
||||
"otel_headers": "x-api-key=abc,x-team=42",
|
||||
},
|
||||
)
|
||||
assert dest is not None
|
||||
assert dest.endpoint == "https://collector.internal:4318/v1/traces"
|
||||
assert dest.headers == {"x-api-key": "abc", "x-team": "42"}
|
||||
|
||||
|
||||
def test_unknown_backend_without_generic_fields_returns_none():
|
||||
assert build_destination("mystery", {"foo": "bar"}) is None
|
||||
|
||||
|
||||
def test_registry_lists_the_first_class_backends():
|
||||
assert OTEL_V2_DESTINATION_CALLBACKS == frozenset(
|
||||
{"langfuse_otel", "arize", "weave_otel"}
|
||||
)
|
||||
|
|
@ -1,88 +1,97 @@
|
|||
"""Per-request multi-tenant credential routing (V1 parity)."""
|
||||
"""Per-tenant tracer routing on admin-owned OTEL destinations, with fan-out.
|
||||
|
||||
A request's identity chain is assigned a set of admin-owned exporters; the v2 logger
|
||||
fans the trace out to all of them (plus the configured/global exporter), and never
|
||||
routes on request-supplied vendor credentials. These tests lock the contract: the
|
||||
request cannot route a trace, each destination's endpoint follows its resolved host
|
||||
(cross-host fix), the configured exporters are kept (global also receives), and a
|
||||
logger only exports the destinations tagged with its own backend.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.abspath("../../../.."))
|
||||
|
||||
from opentelemetry.trace import NoOpTracer
|
||||
|
||||
from litellm.integrations.otel.model.config import ExporterSpec, OpenTelemetryV2Config
|
||||
from litellm.integrations.otel.presets import dynamic_otlp_headers
|
||||
from litellm.integrations.otel.model.destination import OtelDestination
|
||||
from litellm.integrations.otel.model.metadata import LLMCallEvent
|
||||
from litellm.integrations.otel.plumbing.routing import TenantTracerCache
|
||||
|
||||
|
||||
def _cache(callback_name, exporters=None):
|
||||
cfg = OpenTelemetryV2Config(exporters=exporters or [ExporterSpec(kind="in_memory")])
|
||||
cfg = OpenTelemetryV2Config(
|
||||
exporters=exporters or [ExporterSpec(kind="in_memory", owner=callback_name)]
|
||||
)
|
||||
return TenantTracerCache(cfg, callback_name, "litellm")
|
||||
|
||||
|
||||
# --- header builders mirror the V1 construct_dynamic_otel_headers overrides --- #
|
||||
|
||||
|
||||
def test_arize_dynamic_headers():
|
||||
headers = dynamic_otlp_headers(
|
||||
"arize", {"arize_space_id": "S", "arize_api_key": "K"}
|
||||
def _dest(endpoint, auth="Basic AAAA", backend="langfuse_otel"):
|
||||
return OtelDestination(
|
||||
endpoint=endpoint, headers={"Authorization": auth}, callback_name=backend
|
||||
)
|
||||
assert headers == {"arize-space-id": "S", "api_key": "K"}
|
||||
|
||||
|
||||
def test_arize_space_key_overrides_space_id():
|
||||
headers = dynamic_otlp_headers(
|
||||
"arize", {"arize_space_id": "S", "arize_space_key": "SK"}
|
||||
def _event(destinations):
|
||||
return LLMCallEvent.from_dict(
|
||||
{
|
||||
"standard_callback_dynamic_params": {"otel_destinations": destinations},
|
||||
"call_type": "acompletion",
|
||||
"model": "gpt-4o",
|
||||
}
|
||||
)
|
||||
assert headers == {"arize-space-id": "SK"}
|
||||
|
||||
|
||||
def test_langfuse_dynamic_headers_need_both_keys():
|
||||
assert dynamic_otlp_headers("langfuse_otel", {"langfuse_public_key": "pk"}) is None
|
||||
headers = dynamic_otlp_headers(
|
||||
"langfuse_otel", {"langfuse_public_key": "pk", "langfuse_secret_key": "sk"}
|
||||
)
|
||||
assert headers is not None and "Authorization" in headers
|
||||
# --- routing only happens for admin destinations --------------------------- #
|
||||
|
||||
|
||||
def test_weave_dynamic_headers():
|
||||
headers = dynamic_otlp_headers(
|
||||
"weave_otel", {"wandb_api_key": "w", "weave_project_id": "p"}
|
||||
)
|
||||
assert headers is not None
|
||||
assert "Authorization" in headers and headers["project_id"] == "p"
|
||||
|
||||
|
||||
def test_non_participating_callbacks_have_no_routing():
|
||||
# Phoenix subclasses the base in V1 (no override) → no dynamic routing.
|
||||
assert dynamic_otlp_headers("arize_phoenix", {"arize_api_key": "K"}) is None
|
||||
assert dynamic_otlp_headers("langtrace", {"arize_api_key": "K"}) is None
|
||||
assert dynamic_otlp_headers(None, {"arize_api_key": "K"}) is None
|
||||
|
||||
|
||||
def test_no_dynamic_params_is_no_routing():
|
||||
assert dynamic_otlp_headers("arize", None) is None
|
||||
assert dynamic_otlp_headers("arize", {}) is None
|
||||
|
||||
|
||||
# --- TenantTracerCache routes + caches a TracerProvider per credential set --- #
|
||||
|
||||
|
||||
def test_provider_cached_per_credential_set():
|
||||
cache = _cache("arize")
|
||||
def test_no_destinations_uses_default_tracer():
|
||||
cache = _cache("langfuse_otel")
|
||||
default = NoOpTracer()
|
||||
creds_a = {"arize_space_id": "S", "arize_api_key": "K"}
|
||||
creds_b = {"arize_space_id": "S2", "arize_api_key": "K2"}
|
||||
assert cache.tracer_for(default, ()) is default
|
||||
assert cache._providers == {}
|
||||
|
||||
cache.tracer_for(default, creds_a)
|
||||
cache.tracer_for(default, creds_a) # same set → reuse, no new provider
|
||||
|
||||
def test_provider_cached_per_destination_set():
|
||||
cache = _cache("langfuse_otel")
|
||||
default = NoOpTracer()
|
||||
a = (_dest("https://eu.example/v1", "Basic A"),)
|
||||
b = (_dest("https://eu.example/v1", "Basic B"),)
|
||||
|
||||
cache.tracer_for(default, a)
|
||||
cache.tracer_for(default, a) # same set -> reuse
|
||||
assert len(cache._providers) == 1
|
||||
cache.tracer_for(default, creds_b) # new set → new provider
|
||||
cache.tracer_for(default, b) # different creds -> new provider
|
||||
assert len(cache._providers) == 2
|
||||
|
||||
|
||||
def test_different_host_is_a_distinct_provider():
|
||||
"""Two destinations with identical headers but different hosts must not collide;
|
||||
the cache key includes each endpoint."""
|
||||
cache = _cache("langfuse_otel")
|
||||
default = NoOpTracer()
|
||||
eu = (_dest("https://cloud.langfuse.com/api/public/otel", "Basic X"),)
|
||||
us = (_dest("https://us.cloud.langfuse.com/api/public/otel", "Basic X"),)
|
||||
cache.tracer_for(default, eu)
|
||||
cache.tracer_for(default, us)
|
||||
assert len(cache._providers) == 2
|
||||
|
||||
|
||||
def test_destination_set_is_order_independent():
|
||||
cache = _cache("langfuse_otel")
|
||||
default = NoOpTracer()
|
||||
a = _dest("https://a/v1", "Basic A")
|
||||
b = _dest("https://b/v1", "Basic B")
|
||||
cache.tracer_for(default, (a, b))
|
||||
cache.tracer_for(default, (b, a)) # same set, different order -> one provider
|
||||
assert len(cache._providers) == 1
|
||||
|
||||
|
||||
def test_provider_cache_is_bounded_and_evicts_lru(monkeypatch):
|
||||
# The cache key derives from request-supplied dynamic credentials, so it
|
||||
# must be bounded — an unbounded cache lets a caller spawn one provider (and
|
||||
# its background exporter thread) per unique credential set. On overflow the
|
||||
# least-recently-used provider is evicted and shut down.
|
||||
from litellm.integrations.otel.plumbing import routing as routing_mod
|
||||
|
||||
monkeypatch.setattr(routing_mod, "_MAX_CACHED_PROVIDERS", 2)
|
||||
|
|
@ -90,61 +99,49 @@ def test_provider_cache_is_bounded_and_evicts_lru(monkeypatch):
|
|||
monkeypatch.setattr(
|
||||
routing_mod, "_shutdown_provider", lambda p: shut_down.append(p)
|
||||
)
|
||||
|
||||
cache = _cache("arize")
|
||||
cache = _cache("langfuse_otel")
|
||||
default = NoOpTracer()
|
||||
|
||||
def creds(space):
|
||||
return {"arize_space_id": space, "arize_api_key": "K"}
|
||||
|
||||
cache.tracer_for(default, creds("1"))
|
||||
cache.tracer_for(default, creds("2"))
|
||||
cache.tracer_for(default, creds("1")) # touch "1" → "2" is now LRU
|
||||
cache.tracer_for(default, creds("3")) # overflow → evict "2"
|
||||
|
||||
cache.tracer_for(default, (_dest("https://1/v1"),))
|
||||
cache.tracer_for(default, (_dest("https://2/v1"),))
|
||||
cache.tracer_for(default, (_dest("https://1/v1"),)) # touch "1" -> "2" is LRU
|
||||
cache.tracer_for(default, (_dest("https://3/v1"),)) # overflow -> evict "2"
|
||||
assert len(cache._providers) == 2
|
||||
assert len(shut_down) == 1 # exactly the evicted provider was shut down
|
||||
assert len(shut_down) == 1
|
||||
|
||||
|
||||
def test_no_dynamic_params_uses_default_tracer():
|
||||
cache = _cache("arize")
|
||||
default = NoOpTracer()
|
||||
assert cache.tracer_for(default, {}) is default
|
||||
assert cache._providers == {}
|
||||
# --- fan-out: keep the configured exporters, append one per destination ----- #
|
||||
|
||||
|
||||
def test_non_participating_callback_uses_default_tracer():
|
||||
cache = _cache("arize_phoenix")
|
||||
default = NoOpTracer()
|
||||
assert cache.tracer_for(default, {"arize_api_key": "K"}) is default
|
||||
assert cache._providers == {}
|
||||
|
||||
|
||||
def test_dynamic_headers_applied_to_otlp_exporter_only():
|
||||
@pytest.mark.parametrize("owner", ["langfuse_otel", "arize", "weave_otel"])
|
||||
def test_fan_out_appends_destination_with_resolved_endpoint(owner):
|
||||
# The configured (global) exporter is kept; each destination is appended with its
|
||||
# OWN resolved endpoint + headers (the cross-host fix, per owner).
|
||||
cache = _cache(
|
||||
"arize",
|
||||
owner,
|
||||
exporters=[
|
||||
ExporterSpec(kind="otlp_http", owner="arize"),
|
||||
ExporterSpec(kind="in_memory", owner="arize"),
|
||||
ExporterSpec(
|
||||
kind="otlp_http",
|
||||
endpoint="https://env-host.example/v1",
|
||||
headers="Authorization=Basic ENV",
|
||||
owner=owner,
|
||||
)
|
||||
],
|
||||
)
|
||||
new_cfg = cache._config_with_headers({"arize-space-id": "S", "api_key": "K"})
|
||||
otlp, in_mem = new_cfg.exporters
|
||||
assert otlp.headers == "arize-space-id=S,api_key=K"
|
||||
assert in_mem.headers is None # console/in_memory left untouched
|
||||
new = cache._config_with_destinations(
|
||||
(_dest("https://resolved.example/v1", "Basic TEAM", backend=owner),)
|
||||
)
|
||||
# global kept verbatim
|
||||
assert new.exporters[0].endpoint == "https://env-host.example/v1"
|
||||
assert new.exporters[0].headers == "Authorization=Basic ENV"
|
||||
# destination appended at the resolved host with its own auth
|
||||
assert new.exporters[-1].endpoint == "https://resolved.example/v1"
|
||||
assert new.exporters[-1].headers == "Authorization=Basic TEAM"
|
||||
assert len(new.exporters) == 2
|
||||
|
||||
|
||||
def test_dynamic_headers_do_not_leak_to_other_owners_exporter():
|
||||
"""A tenant's Arize credentials must never be stamped onto a co-configured
|
||||
exporter owned by a different backend (a self-hosted collector, Langfuse).
|
||||
|
||||
Regression for the cross-backend credential leak: ``_config_with_headers``
|
||||
used to rewrite the headers of every OTLP exporter, so one request carrying
|
||||
a team's Arize key clobbered the base collector's and Langfuse's headers
|
||||
with that key.
|
||||
"""
|
||||
def test_fan_out_preserves_co_configured_exporters():
|
||||
cache = _cache(
|
||||
"arize",
|
||||
"langfuse_otel",
|
||||
exporters=[
|
||||
ExporterSpec(
|
||||
kind="otlp_http",
|
||||
|
|
@ -154,22 +151,117 @@ def test_dynamic_headers_do_not_leak_to_other_owners_exporter():
|
|||
),
|
||||
ExporterSpec(
|
||||
kind="otlp_http",
|
||||
endpoint="https://cloud.langfuse.com/api/public/otel",
|
||||
headers="Authorization=Basic base-langfuse",
|
||||
endpoint="https://us.cloud.langfuse.com/api/public/otel",
|
||||
headers="Authorization=Basic ENV",
|
||||
owner="langfuse_otel",
|
||||
),
|
||||
ExporterSpec(
|
||||
kind="otlp_grpc",
|
||||
endpoint="https://otlp.arize.com/v1",
|
||||
headers="space_id=base,api_key=base",
|
||||
owner="arize",
|
||||
),
|
||||
],
|
||||
)
|
||||
new_cfg = cache._config_with_headers(
|
||||
{"arize-space-id": "TEAMX", "api_key": "TEAMX_KEY"}
|
||||
new = cache._config_with_destinations(
|
||||
(_dest("https://cloud.langfuse.com/api/public/otel", "Basic TEAM"),)
|
||||
)
|
||||
by_owner = {e.owner: e.headers for e in new_cfg.exporters}
|
||||
assert by_owner["arize"] == "arize-space-id=TEAMX,api_key=TEAMX_KEY"
|
||||
assert by_owner[None] == "x=base-collector"
|
||||
assert by_owner["langfuse_otel"] == "Authorization=Basic base-langfuse"
|
||||
# both originals preserved unchanged (no rewrite/leak)
|
||||
assert new.exporters[0].endpoint == "http://self-hosted-collector:4318"
|
||||
assert new.exporters[0].headers == "x=base-collector"
|
||||
assert new.exporters[1].headers == "Authorization=Basic ENV"
|
||||
# exactly one appended
|
||||
assert new.exporters[-1].endpoint == "https://cloud.langfuse.com/api/public/otel"
|
||||
assert len(new.exporters) == 3
|
||||
|
||||
|
||||
def test_fan_out_to_many_destinations_is_one_provider_with_all_exporters():
|
||||
cache = _cache(
|
||||
"langfuse_otel",
|
||||
exporters=[
|
||||
ExporterSpec(
|
||||
kind="otlp_http", endpoint="https://env/v1", owner="langfuse_otel"
|
||||
)
|
||||
],
|
||||
)
|
||||
new = cache._config_with_destinations(
|
||||
(_dest("https://a/v1", "Basic A"), _dest("https://b/v1", "Basic B"))
|
||||
)
|
||||
# global + 2 destinations -> 3 span processors, one provider, one span copied to all
|
||||
assert [e.endpoint for e in new.exporters] == [
|
||||
"https://env/v1",
|
||||
"https://a/v1",
|
||||
"https://b/v1",
|
||||
]
|
||||
cache.tracer_for(NoOpTracer(), (_dest("https://a/v1"), _dest("https://b/v1")))
|
||||
assert len(cache._providers) == 1
|
||||
|
||||
|
||||
# --- security: request credentials never route a trace --------------------- #
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"request_creds",
|
||||
[
|
||||
{
|
||||
"langfuse_public_key": "pk-attacker",
|
||||
"langfuse_secret_key": "sk-attacker",
|
||||
"langfuse_host": "https://attacker.example",
|
||||
},
|
||||
{"arize_api_key": "K-attacker", "arize_space_id": "S-attacker"},
|
||||
{"wandb_api_key": "w-attacker", "weave_endpoint": "https://attacker/otel"},
|
||||
],
|
||||
)
|
||||
def test_request_credentials_are_inert_on_v2(request_creds):
|
||||
"""Any backend's credentials in the request's dynamic params (no admin
|
||||
destinations) produce no per-tenant routing."""
|
||||
event = LLMCallEvent.from_dict(
|
||||
{
|
||||
"standard_callback_dynamic_params": request_creds,
|
||||
"call_type": "acompletion",
|
||||
"model": "gpt-4o",
|
||||
}
|
||||
)
|
||||
assert event.otel_destinations == ()
|
||||
cache = _cache("langfuse_otel")
|
||||
default = NoOpTracer()
|
||||
assert cache.tracer_for(default, event.otel_destinations) is default
|
||||
assert cache._providers == {}
|
||||
|
||||
|
||||
def test_admin_destinations_route():
|
||||
event = _event(
|
||||
[
|
||||
{
|
||||
"callback_name": "langfuse_otel",
|
||||
"endpoint": "https://cloud.langfuse.com/api/public/otel",
|
||||
"headers": {"Authorization": "Basic ADMIN"},
|
||||
}
|
||||
]
|
||||
)
|
||||
assert len(event.otel_destinations) == 1
|
||||
cache = _cache("langfuse_otel")
|
||||
cache.tracer_for(NoOpTracer(), event.otel_destinations)
|
||||
assert len(cache._providers) == 1
|
||||
|
||||
|
||||
# --- a logger only exports the destinations tagged with its own backend ----- #
|
||||
|
||||
|
||||
def test_logger_filters_destinations_to_its_backend():
|
||||
from litellm.integrations.otel.logger import OpenTelemetryV2
|
||||
|
||||
event = _event(
|
||||
[
|
||||
{
|
||||
"callback_name": "langfuse_otel",
|
||||
"endpoint": "https://lf/api/public/otel",
|
||||
"headers": {"Authorization": "Basic A"},
|
||||
},
|
||||
{
|
||||
"callback_name": "arize",
|
||||
"endpoint": "https://otlp.arize.com/v1",
|
||||
"headers": {"space_id": "S"},
|
||||
},
|
||||
]
|
||||
)
|
||||
|
||||
class _Shim:
|
||||
callback_name = "langfuse_otel"
|
||||
|
||||
got = OpenTelemetryV2._destinations_for_backend(_Shim(), event)
|
||||
assert [d.endpoint for d in got] == ["https://lf/api/public/otel"]
|
||||
|
|
|
|||
|
|
@ -44,16 +44,14 @@ def test_agentops_exporter_factory_is_registered():
|
|||
assert _AGENTOPS_EXPORTER_KIND in providers._EXPORTER_FACTORIES
|
||||
|
||||
|
||||
def test_dynamic_cred_presets_tag_exporter_with_matching_owner(monkeypatch):
|
||||
"""Each dynamic-credential preset must tag the exporter it contributes with
|
||||
its own callback name, so per-request tenant routing
|
||||
(``TenantTracerCache``) applies that integration's credentials only to its
|
||||
own exporter and never bleeds them onto a co-configured backend.
|
||||
def test_destination_routable_presets_tag_exporter_with_matching_owner(monkeypatch):
|
||||
"""Each destination-routable preset must tag the exporter it contributes with
|
||||
its own callback name, so per-tenant routing (``TenantTracerCache``) points
|
||||
that integration's admin destination at its own exporter only and never
|
||||
rewrites a co-configured backend's exporter.
|
||||
"""
|
||||
from litellm.integrations.otel.presets import (
|
||||
DYNAMIC_HEADERS_BY_CALLBACK,
|
||||
PRESET_BY_CALLBACK,
|
||||
)
|
||||
from litellm.integrations.otel.destinations import OTEL_V2_DESTINATION_CALLBACKS
|
||||
from litellm.integrations.otel.presets import PRESET_BY_CALLBACK
|
||||
|
||||
monkeypatch.setenv("ARIZE_SPACE_ID", "S")
|
||||
monkeypatch.setenv("ARIZE_API_KEY", "K")
|
||||
|
|
@ -65,7 +63,7 @@ def test_dynamic_cred_presets_tag_exporter_with_matching_owner(monkeypatch):
|
|||
|
||||
from litellm.integrations.otel.model.config import ExporterOwner
|
||||
|
||||
for callback_name in DYNAMIC_HEADERS_BY_CALLBACK:
|
||||
for callback_name in OTEL_V2_DESTINATION_CALLBACKS:
|
||||
cfg = PRESET_BY_CALLBACK[callback_name]()
|
||||
owners = {e.owner for e in cfg.exporters}
|
||||
assert ExporterOwner(callback_name) in owners, (
|
||||
|
|
|
|||
|
|
@ -36,6 +36,46 @@ def test_resolves_plain_values_from_metadata():
|
|||
assert params.get("langfuse_host") == "https://test.langfuse.com"
|
||||
|
||||
|
||||
def test_otel_destinations_read_from_top_level_only():
|
||||
"""The admin-resolved OTEL destinations are carried server-side on top-level
|
||||
kwargs (the proxy strips any client value first). They must surface on the
|
||||
dynamic params so the v2 logger can fan out to them."""
|
||||
destinations = [
|
||||
{
|
||||
"callback_name": "langfuse_otel",
|
||||
"endpoint": "https://cloud.langfuse.com/api/public/otel",
|
||||
"headers": {"Authorization": "Basic ADMIN"},
|
||||
}
|
||||
]
|
||||
|
||||
params = initialize_standard_callback_dynamic_params(
|
||||
{"otel_destinations": destinations}
|
||||
)
|
||||
|
||||
assert params.get("otel_destinations") == destinations
|
||||
|
||||
|
||||
def test_otel_destinations_never_read_from_request_metadata():
|
||||
"""A request body/metadata must not be able to inject OTEL destinations:
|
||||
otel_destinations is deliberately absent from the request-read whitelist, so a
|
||||
value nested in metadata is ignored. Guards the trust boundary."""
|
||||
kwargs = {
|
||||
"metadata": {
|
||||
"otel_destinations": [
|
||||
{
|
||||
"callback_name": "langfuse_otel",
|
||||
"endpoint": "https://attacker.example/api/public/otel",
|
||||
"headers": {"Authorization": "Basic ATTACKER"},
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
params = initialize_standard_callback_dynamic_params(kwargs)
|
||||
|
||||
assert params.get("otel_destinations") is None
|
||||
|
||||
|
||||
def test_env_reference_at_top_level_raises_with_guidance():
|
||||
kwargs = {"langfuse_public_key": "os.environ/LANGFUSE_PUBLIC_KEY"}
|
||||
|
||||
|
|
|
|||
205
tests/test_litellm/proxy/credential_endpoints/test_endpoints.py
Normal file
205
tests/test_litellm/proxy/credential_endpoints/test_endpoints.py
Normal file
|
|
@ -0,0 +1,205 @@
|
|||
"""Admin-gating on credential mutations for logging destinations.
|
||||
|
||||
Logging credentials carry ``credential_info.access`` that controls where other
|
||||
tenants' traces export, so create/update/delete of a logging credential is
|
||||
proxy-admin only. Provider credentials keep their pre-existing (ungated) behavior.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
from fastapi import HTTPException
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
sys.path.insert(0, os.path.abspath("../../../.."))
|
||||
|
||||
import litellm
|
||||
import litellm.proxy.credential_endpoints.endpoints as endpoints
|
||||
from litellm.models.credentials import CredentialItem
|
||||
from litellm.proxy._types import LitellmUserRoles, UserAPIKeyAuth
|
||||
from litellm.types.utils import CreateCredentialItem
|
||||
|
||||
|
||||
def _admin():
|
||||
return UserAPIKeyAuth(api_key="k", user_role=LitellmUserRoles.PROXY_ADMIN)
|
||||
|
||||
|
||||
def _member():
|
||||
return UserAPIKeyAuth(api_key="k", user_role=LitellmUserRoles.INTERNAL_USER)
|
||||
|
||||
|
||||
_LOGGING_INFO = {"credential_type": "logging", "description": "langfuse_otel"}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _connected_db(monkeypatch):
|
||||
"""A working prisma_client + repository so an allowed caller reaches success."""
|
||||
import litellm.proxy.proxy_server as proxy_server
|
||||
|
||||
monkeypatch.setenv("LITELLM_SALT_KEY", "sk-test-salt-key")
|
||||
monkeypatch.setattr(proxy_server, "prisma_client", MagicMock())
|
||||
monkeypatch.setattr(proxy_server, "llm_router", None)
|
||||
repo = MagicMock()
|
||||
repo.create = AsyncMock()
|
||||
repo.delete_by_name = AsyncMock()
|
||||
monkeypatch.setattr(endpoints, "CredentialsRepository", lambda _client: repo)
|
||||
monkeypatch.setattr(
|
||||
endpoints.CredentialAccessor, "upsert_credentials", lambda creds: None
|
||||
)
|
||||
return repo
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_create_logging_credential_forbidden_for_non_admin(_connected_db):
|
||||
with pytest.raises(HTTPException) as exc:
|
||||
await endpoints.create_credential(
|
||||
request=MagicMock(),
|
||||
fastapi_response=MagicMock(),
|
||||
credential=CreateCredentialItem(
|
||||
credential_name="dest",
|
||||
credential_values={"langfuse_host": "h"},
|
||||
credential_info=_LOGGING_INFO,
|
||||
),
|
||||
user_api_key_dict=_member(),
|
||||
)
|
||||
assert exc.value.status_code == 403
|
||||
_connected_db.create.assert_not_awaited()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_create_logging_credential_allowed_for_admin(_connected_db):
|
||||
result = await endpoints.create_credential(
|
||||
request=MagicMock(),
|
||||
fastapi_response=MagicMock(),
|
||||
credential=CreateCredentialItem(
|
||||
credential_name="dest",
|
||||
credential_values={"langfuse_host": "h"},
|
||||
credential_info=_LOGGING_INFO,
|
||||
),
|
||||
user_api_key_dict=_admin(),
|
||||
)
|
||||
assert result["success"] is True
|
||||
_connected_db.create.assert_awaited_once()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_create_provider_credential_not_gated(_connected_db):
|
||||
"""A non-logging (provider) credential keeps its pre-existing ungated behavior."""
|
||||
result = await endpoints.create_credential(
|
||||
request=MagicMock(),
|
||||
fastapi_response=MagicMock(),
|
||||
credential=CreateCredentialItem(
|
||||
credential_name="openai",
|
||||
credential_values={"api_key": "sk"},
|
||||
credential_info={"custom_llm_provider": "openai"},
|
||||
),
|
||||
user_api_key_dict=_member(),
|
||||
)
|
||||
assert result["success"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_update_logging_credential_forbidden_for_non_admin(_connected_db):
|
||||
with pytest.raises(HTTPException) as exc:
|
||||
await endpoints.update_credential(
|
||||
request=MagicMock(),
|
||||
fastapi_response=MagicMock(),
|
||||
credential=CredentialItem(
|
||||
credential_name="dest",
|
||||
credential_values={},
|
||||
credential_info={"access": {"global": True}},
|
||||
),
|
||||
credential_name="dest",
|
||||
user_api_key_dict=_member(),
|
||||
)
|
||||
assert exc.value.status_code == 403
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_update_existing_logging_credential_forbidden_even_without_logging_patch(
|
||||
_connected_db, monkeypatch
|
||||
):
|
||||
"""A non-admin cannot edit a stored logging credential's values, even with a patch
|
||||
that omits credential_info (the gate consults the in-memory credential too)."""
|
||||
monkeypatch.setattr(
|
||||
litellm,
|
||||
"credential_list",
|
||||
[
|
||||
CredentialItem(
|
||||
credential_name="dest",
|
||||
credential_values={"langfuse_host": "h"},
|
||||
credential_info=_LOGGING_INFO,
|
||||
)
|
||||
],
|
||||
)
|
||||
with pytest.raises(HTTPException) as exc:
|
||||
await endpoints.update_credential(
|
||||
request=MagicMock(),
|
||||
fastapi_response=MagicMock(),
|
||||
credential=CredentialItem(
|
||||
credential_name="dest",
|
||||
credential_values={"langfuse_host": "evil"},
|
||||
credential_info={},
|
||||
),
|
||||
credential_name="dest",
|
||||
user_api_key_dict=_member(),
|
||||
)
|
||||
assert exc.value.status_code == 403
|
||||
|
||||
|
||||
def test_update_db_credential_preserves_existing_info_on_partial_patch():
|
||||
"""A partial credential_info patch (e.g. only access from the Edit-access modal) must
|
||||
merge into the stored info, not replace it -- otherwise the logging tag is dropped and
|
||||
the destination vanishes from the registry after the next reload."""
|
||||
from litellm.proxy.credential_endpoints.endpoints import update_db_credential
|
||||
|
||||
db = CredentialItem(
|
||||
credential_name="dest",
|
||||
credential_values={},
|
||||
credential_info={
|
||||
"credential_type": "logging",
|
||||
"description": "langfuse_otel",
|
||||
"host": "h",
|
||||
},
|
||||
)
|
||||
patch = CredentialItem(
|
||||
credential_name="dest",
|
||||
credential_values={},
|
||||
credential_info={"access": {"global": True}},
|
||||
)
|
||||
|
||||
merged = update_db_credential(db, patch)
|
||||
|
||||
assert merged.credential_info == {
|
||||
"credential_type": "logging",
|
||||
"description": "langfuse_otel",
|
||||
"host": "h",
|
||||
"access": {"global": True},
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_logging_credential_forbidden_for_non_admin(
|
||||
_connected_db, monkeypatch
|
||||
):
|
||||
monkeypatch.setattr(
|
||||
litellm,
|
||||
"credential_list",
|
||||
[
|
||||
CredentialItem(
|
||||
credential_name="dest",
|
||||
credential_values={},
|
||||
credential_info=_LOGGING_INFO,
|
||||
)
|
||||
],
|
||||
)
|
||||
with pytest.raises(HTTPException) as exc:
|
||||
await endpoints.delete_credential(
|
||||
request=MagicMock(),
|
||||
fastapi_response=MagicMock(),
|
||||
credential_name="dest",
|
||||
user_api_key_dict=_member(),
|
||||
)
|
||||
assert exc.value.status_code == 403
|
||||
_connected_db.delete_by_name.assert_not_awaited()
|
||||
|
|
@ -0,0 +1,136 @@
|
|||
"""Validation for admin-owned logging-exporter assignment on key/team/org."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
from fastapi import HTTPException
|
||||
|
||||
sys.path.insert(0, os.path.abspath("../../../.."))
|
||||
|
||||
import litellm
|
||||
from litellm.models.credentials import CredentialItem
|
||||
from litellm.proxy._types import LitellmUserRoles, UserAPIKeyAuth
|
||||
from litellm.proxy.management_endpoints.logging_exporter_validation import (
|
||||
is_admin_gated_credential_info,
|
||||
validate_credential_access,
|
||||
validate_logging_exporter_assignment,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _registry():
|
||||
original = litellm.credential_list
|
||||
litellm.credential_list = [
|
||||
CredentialItem(
|
||||
credential_name="langfuse-eu",
|
||||
credential_values={},
|
||||
credential_info={
|
||||
"credential_type": "logging",
|
||||
"description": "langfuse_otel",
|
||||
},
|
||||
),
|
||||
CredentialItem(
|
||||
credential_name="openai-key",
|
||||
credential_values={},
|
||||
credential_info={"custom_llm_provider": "openai"}, # a provider credential
|
||||
),
|
||||
]
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
litellm.credential_list = original
|
||||
|
||||
|
||||
def _admin():
|
||||
return UserAPIKeyAuth(api_key="k", user_role=LitellmUserRoles.PROXY_ADMIN)
|
||||
|
||||
|
||||
def _member():
|
||||
return UserAPIKeyAuth(api_key="k", user_role=LitellmUserRoles.INTERNAL_USER)
|
||||
|
||||
|
||||
def test_admin_with_known_logging_credential_is_allowed(_registry):
|
||||
validate_logging_exporter_assignment(
|
||||
{"logging_exporters": ["langfuse-eu"]}, _admin()
|
||||
)
|
||||
|
||||
|
||||
def test_noop_when_assignment_absent(_registry):
|
||||
# an update that does not touch logging_exporters is never gated/validated
|
||||
validate_logging_exporter_assignment({"some_other_key": 1}, _member())
|
||||
validate_logging_exporter_assignment(None, _member())
|
||||
|
||||
|
||||
def test_non_admin_is_forbidden(_registry):
|
||||
with pytest.raises(HTTPException) as exc:
|
||||
validate_logging_exporter_assignment(
|
||||
{"logging_exporters": ["langfuse-eu"]}, _member()
|
||||
)
|
||||
assert exc.value.status_code == 403
|
||||
|
||||
|
||||
def test_unknown_credential_is_rejected(_registry):
|
||||
with pytest.raises(HTTPException) as exc:
|
||||
validate_logging_exporter_assignment(
|
||||
{"logging_exporters": ["does-not-exist"]}, _admin()
|
||||
)
|
||||
assert exc.value.status_code == 400
|
||||
|
||||
|
||||
def test_provider_credential_is_not_a_valid_logging_exporter(_registry):
|
||||
# openai-key exists but is a provider credential, not a logging destination
|
||||
with pytest.raises(HTTPException) as exc:
|
||||
validate_logging_exporter_assignment(
|
||||
{"logging_exporters": ["openai-key"]}, _admin()
|
||||
)
|
||||
assert exc.value.status_code == 400
|
||||
|
||||
|
||||
def test_non_list_is_rejected(_registry):
|
||||
with pytest.raises(HTTPException) as exc:
|
||||
validate_logging_exporter_assignment(
|
||||
{"logging_exporters": "langfuse-eu"}, _admin()
|
||||
)
|
||||
assert exc.value.status_code == 400
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"credential_info, gated",
|
||||
[
|
||||
({"credential_type": "logging"}, True),
|
||||
({"access": {"global": True}}, True),
|
||||
({"credential_type": "logging", "access": {"teams": ["t"]}}, True),
|
||||
({"custom_llm_provider": "openai"}, False),
|
||||
({}, False),
|
||||
(None, False),
|
||||
],
|
||||
)
|
||||
def test_is_admin_gated_credential_info(credential_info, gated):
|
||||
assert is_admin_gated_credential_info(credential_info) is gated
|
||||
|
||||
|
||||
def test_validate_credential_access_accepts_valid_object():
|
||||
validate_credential_access(
|
||||
{"access": {"global": False, "teams": ["t1", "t2"], "orgs": ["o1"]}}
|
||||
)
|
||||
|
||||
|
||||
def test_validate_credential_access_noop_without_access():
|
||||
validate_credential_access({"credential_type": "logging"})
|
||||
validate_credential_access(None)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"access",
|
||||
[
|
||||
5, # not an object
|
||||
{"global": "yes"}, # global must be bool
|
||||
{"teams": "t1"}, # teams must be a list
|
||||
{"orgs": [1, 2]}, # orgs must be strings
|
||||
],
|
||||
)
|
||||
def test_validate_credential_access_rejects_bad_shape(access):
|
||||
with pytest.raises(HTTPException) as exc:
|
||||
validate_credential_access({"access": access})
|
||||
assert exc.value.status_code == 400
|
||||
|
|
@ -4798,3 +4798,243 @@ async def test_add_litellm_data_to_request_claude_code_drop_params(
|
|||
)
|
||||
|
||||
assert updated.get("drop_params") == expected_drop_params
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _seeded_logging_credentials():
|
||||
from litellm.models.credentials import CredentialItem
|
||||
|
||||
original = litellm.credential_list
|
||||
litellm.credential_list = [
|
||||
CredentialItem(
|
||||
credential_name="langfuse-eu",
|
||||
credential_values={
|
||||
"langfuse_host": "https://cloud.langfuse.com",
|
||||
"langfuse_public_key": "pk-eu",
|
||||
"langfuse_secret_key": "sk-eu",
|
||||
},
|
||||
credential_info={
|
||||
"credential_type": "logging",
|
||||
"description": "langfuse_otel",
|
||||
},
|
||||
),
|
||||
CredentialItem(
|
||||
credential_name="arize-prod",
|
||||
credential_values={"arize_space_id": "S", "arize_api_key": "K"},
|
||||
credential_info={"credential_type": "logging", "description": "arize"},
|
||||
),
|
||||
# A provider credential that must never resolve as a logging destination.
|
||||
CredentialItem(
|
||||
credential_name="openai-key",
|
||||
credential_values={"api_key": "sk-openai"},
|
||||
credential_info={"custom_llm_provider": "openai"},
|
||||
),
|
||||
]
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
litellm.credential_list = original
|
||||
|
||||
|
||||
def _auth(team_exporters=None, token=None, org_id=None, team_id=None):
|
||||
return UserAPIKeyAuth(
|
||||
api_key="hashed-key",
|
||||
token=token,
|
||||
org_id=org_id,
|
||||
team_id=team_id,
|
||||
team_metadata=({"logging_exporters": team_exporters} if team_exporters else {}),
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_logging_exporters_team_level(_seeded_logging_credentials):
|
||||
# team_metadata is the team's own (not shadowed); resolves without a DB fetch.
|
||||
from litellm.proxy.litellm_pre_call_utils import _resolve_logging_exporters
|
||||
|
||||
destinations, backends = await _resolve_logging_exporters(
|
||||
_auth(team_exporters=["langfuse-eu"])
|
||||
)
|
||||
assert {d["endpoint"] for d in destinations} == {
|
||||
"https://cloud.langfuse.com/api/public/otel"
|
||||
}
|
||||
assert backends == ["langfuse_otel"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_logging_exporters_unions_key_team_org(
|
||||
_seeded_logging_credentials, monkeypatch
|
||||
):
|
||||
# key + org are read from their OWN records (the key's .metadata is team-shadowed),
|
||||
# team from team_metadata. All three union, deduped.
|
||||
from types import SimpleNamespace
|
||||
|
||||
import litellm.proxy.proxy_server as proxy_server
|
||||
from litellm.proxy.auth import auth_checks
|
||||
from litellm.proxy.litellm_pre_call_utils import _resolve_logging_exporters
|
||||
|
||||
monkeypatch.setattr(proxy_server, "prisma_client", MagicMock())
|
||||
monkeypatch.setattr(
|
||||
auth_checks,
|
||||
"get_key_object",
|
||||
AsyncMock(
|
||||
return_value=SimpleNamespace(metadata={"logging_exporters": ["arize-prod"]})
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
auth_checks,
|
||||
"get_org_object",
|
||||
AsyncMock(
|
||||
return_value=SimpleNamespace(
|
||||
metadata={"logging_exporters": ["langfuse-eu"]}
|
||||
)
|
||||
),
|
||||
)
|
||||
|
||||
destinations, backends = await _resolve_logging_exporters(
|
||||
_auth(team_exporters=["langfuse-eu"], token="hashed-key", org_id="org-1")
|
||||
)
|
||||
|
||||
assert {d["endpoint"] for d in destinations} == {
|
||||
"https://cloud.langfuse.com/api/public/otel", # team + org (deduped)
|
||||
"https://otlp.arize.com/v1", # key
|
||||
}
|
||||
assert set(backends) == {"langfuse_otel", "arize"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_logging_exporters_empty_without_assignment(
|
||||
_seeded_logging_credentials,
|
||||
):
|
||||
from litellm.proxy.litellm_pre_call_utils import _resolve_logging_exporters
|
||||
|
||||
destinations, backends = await _resolve_logging_exporters(_auth())
|
||||
assert destinations == [] and backends == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_logging_exporters_skips_unknown_and_provider_creds(
|
||||
_seeded_logging_credentials,
|
||||
):
|
||||
from litellm.proxy.litellm_pre_call_utils import _resolve_logging_exporters
|
||||
|
||||
# unknown name + a provider credential (not credential_type=logging) -> nothing
|
||||
destinations, backends = await _resolve_logging_exporters(
|
||||
_auth(team_exporters=["does-not-exist", "openai-key"])
|
||||
)
|
||||
assert destinations == [] and backends == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_admin_logging_exporters_stamps_and_activates(
|
||||
_seeded_logging_credentials,
|
||||
):
|
||||
from litellm.proxy.litellm_pre_call_utils import _apply_admin_logging_exporters
|
||||
|
||||
data: dict = {}
|
||||
await _apply_admin_logging_exporters(data, _auth(team_exporters=["langfuse-eu"]))
|
||||
|
||||
assert data["otel_destinations"][0]["callback_name"] == "langfuse_otel"
|
||||
assert (
|
||||
data["otel_destinations"][0]["endpoint"]
|
||||
== "https://cloud.langfuse.com/api/public/otel"
|
||||
)
|
||||
# the backend is activated for the request
|
||||
assert "langfuse_otel" in data["success_callback"]
|
||||
|
||||
|
||||
_LANGFUSE_ENDPOINT = "https://cloud.langfuse.com/api/public/otel"
|
||||
_ARIZE_ENDPOINT = "https://otlp.arize.com/v1"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _seeded_logging_credentials_with_access():
|
||||
"""Destinations whose access lives ON the credential (global/team/org), in
|
||||
addition to a name-only destination assigned via the identity chain."""
|
||||
from litellm.models.credentials import CredentialItem
|
||||
|
||||
original = litellm.credential_list
|
||||
litellm.credential_list = [
|
||||
CredentialItem(
|
||||
credential_name="langfuse-eu",
|
||||
credential_values={
|
||||
"langfuse_host": "https://cloud.langfuse.com",
|
||||
"langfuse_public_key": "pk-eu",
|
||||
"langfuse_secret_key": "sk-eu",
|
||||
},
|
||||
credential_info={
|
||||
"credential_type": "logging",
|
||||
"description": "langfuse_otel",
|
||||
"access": {"teams": ["team-eu"], "orgs": ["org-eu"]},
|
||||
},
|
||||
),
|
||||
CredentialItem(
|
||||
credential_name="arize-global",
|
||||
credential_values={"arize_space_id": "S", "arize_api_key": "K"},
|
||||
credential_info={
|
||||
"credential_type": "logging",
|
||||
"description": "arize",
|
||||
"access": {"global": True},
|
||||
},
|
||||
),
|
||||
]
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
litellm.credential_list = original
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
"auth_kwargs, expected_endpoints",
|
||||
[
|
||||
# access.global reaches an unassigned caller (no names at all). This case
|
||||
# fails if the resolver early-returns on empty identity names.
|
||||
pytest.param({}, {_ARIZE_ENDPOINT}, id="access-global-unassigned"),
|
||||
# access.teams matches the caller's team_id (no identity names).
|
||||
pytest.param(
|
||||
{"team_id": "team-eu"},
|
||||
{_ARIZE_ENDPOINT, _LANGFUSE_ENDPOINT},
|
||||
id="access-team-match",
|
||||
),
|
||||
# a different team gets only the global destination.
|
||||
pytest.param(
|
||||
{"team_id": "team-other"}, {_ARIZE_ENDPOINT}, id="access-team-mismatch"
|
||||
),
|
||||
# access.orgs matches the caller's org_id.
|
||||
pytest.param(
|
||||
{"org_id": "org-eu"},
|
||||
{_ARIZE_ENDPOINT, _LANGFUSE_ENDPOINT},
|
||||
id="access-org-match",
|
||||
),
|
||||
# identity name AND access point at the same destination -> deduped to one
|
||||
# (plus the always-on global). team-eu reaches langfuse via BOTH paths.
|
||||
pytest.param(
|
||||
{"team_id": "team-eu", "team_exporters": ["langfuse-eu"]},
|
||||
{_ARIZE_ENDPOINT, _LANGFUSE_ENDPOINT},
|
||||
id="both-paths-deduped",
|
||||
),
|
||||
],
|
||||
)
|
||||
async def test_resolve_logging_exporters_access_matrix(
|
||||
_seeded_logging_credentials_with_access, auth_kwargs, expected_endpoints
|
||||
):
|
||||
from litellm.proxy.litellm_pre_call_utils import _resolve_logging_exporters
|
||||
|
||||
destinations, _ = await _resolve_logging_exporters(_auth(**auth_kwargs))
|
||||
assert {d["endpoint"] for d in destinations} == expected_endpoints
|
||||
# no duplicate destinations survive the union
|
||||
assert len(destinations) == len(expected_endpoints)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_logging_exporters_access_default_deny(
|
||||
_seeded_logging_credentials,
|
||||
):
|
||||
"""With no access grants and no identity assignment, nothing resolves -- the
|
||||
global-access path must not invent a destination out of name-only creds."""
|
||||
from litellm.proxy.litellm_pre_call_utils import _resolve_logging_exporters
|
||||
|
||||
destinations, backends = await _resolve_logging_exporters(
|
||||
_auth(team_id="team-eu", org_id="org-eu")
|
||||
)
|
||||
assert destinations == [] and backends == []
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
import { render } from "@testing-library/react";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { fireEvent, render, within } from "@testing-library/react";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { LoggingCallbacksTable } from "./LoggingCallbacksTable";
|
||||
|
||||
describe("LoggingCallbacksTable", () => {
|
||||
|
|
@ -90,4 +90,89 @@ describe("LoggingCallbacksTable", () => {
|
|||
expect(getByText("Success")).toBeInTheDocument();
|
||||
expect(getByText("Failure")).toBeInTheDocument();
|
||||
});
|
||||
|
||||
const NO_VARS = {
|
||||
SLACK_WEBHOOK_URL: null,
|
||||
LANGFUSE_PUBLIC_KEY: null,
|
||||
LANGFUSE_SECRET_KEY: null,
|
||||
LANGFUSE_HOST: null,
|
||||
OPENMETER_API_KEY: null,
|
||||
};
|
||||
|
||||
it("summarizes a global destination's access as Global with no mode badge", () => {
|
||||
const { getByText, queryByText } = render(
|
||||
<LoggingCallbacksTable
|
||||
callbacks={[
|
||||
{
|
||||
name: "langfuse-eu",
|
||||
variables: NO_VARS,
|
||||
credentialName: "langfuse-eu",
|
||||
access: { global: true },
|
||||
},
|
||||
]}
|
||||
availableCallbacks={{}}
|
||||
/>,
|
||||
);
|
||||
expect(getByText("Global")).toBeInTheDocument();
|
||||
// a destination has no success/failure mode badge
|
||||
expect(queryByText("Success")).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("summarizes team/org counts for a scoped destination", () => {
|
||||
const { getByText } = render(
|
||||
<LoggingCallbacksTable
|
||||
callbacks={[
|
||||
{
|
||||
name: "arize-eu",
|
||||
variables: NO_VARS,
|
||||
credentialName: "arize-eu",
|
||||
access: { teams: ["t1", "t2"], orgs: ["o1"] },
|
||||
},
|
||||
]}
|
||||
availableCallbacks={{}}
|
||||
/>,
|
||||
);
|
||||
expect(getByText("2 teams · 1 org")).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("a destination row fires onEditAccess and onDelete, never onTest", () => {
|
||||
const onEditAccess = vi.fn();
|
||||
const onDelete = vi.fn();
|
||||
const onTest = vi.fn();
|
||||
const { getByText } = render(
|
||||
<LoggingCallbacksTable
|
||||
callbacks={[{ name: "dest", variables: NO_VARS, credentialName: "dest", access: { global: true } }]}
|
||||
availableCallbacks={{}}
|
||||
onEditAccess={onEditAccess}
|
||||
onDelete={onDelete}
|
||||
onTest={onTest}
|
||||
/>,
|
||||
);
|
||||
const row = getByText("dest").closest("tr") as HTMLElement;
|
||||
const scoped = within(row);
|
||||
// destination rows expose edit-access + delete, and no test action
|
||||
expect(scoped.queryByTestId("test-callback")).not.toBeInTheDocument();
|
||||
fireEvent.click(scoped.getByTestId("edit-access"));
|
||||
fireEvent.click(scoped.getByTestId("delete-destination"));
|
||||
expect(onEditAccess).toHaveBeenCalledWith(expect.objectContaining({ credentialName: "dest" }));
|
||||
expect(onDelete).toHaveBeenCalledWith(expect.objectContaining({ credentialName: "dest" }));
|
||||
expect(onTest).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("a config callback row keeps the test/edit/delete actions and an em-dash access", () => {
|
||||
const { getByText } = render(
|
||||
<LoggingCallbacksTable
|
||||
callbacks={[{ name: "datadog", type: "success", variables: NO_VARS }]}
|
||||
availableCallbacks={{}}
|
||||
/>,
|
||||
);
|
||||
const row = getByText("datadog").closest("tr") as HTMLElement;
|
||||
const scoped = within(row);
|
||||
expect(scoped.getByTestId("test-callback")).toBeInTheDocument();
|
||||
expect(scoped.getByTestId("edit-callback")).toBeInTheDocument();
|
||||
expect(scoped.getByTestId("delete-callback")).toBeInTheDocument();
|
||||
expect(scoped.queryByTestId("edit-access")).not.toBeInTheDocument();
|
||||
// access cell renders an em-dash for non-destination rows
|
||||
expect(scoped.getByText("—")).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -19,9 +19,23 @@ type LoggingCallbacksProps = {
|
|||
onTest?: (callback: AlertingObject) => void | Promise<void>;
|
||||
onEdit?: (callback: AlertingObject) => void;
|
||||
onDelete?: (callback: AlertingObject) => void;
|
||||
onEditAccess?: (callback: AlertingObject) => void;
|
||||
onAdd?: () => void;
|
||||
};
|
||||
|
||||
const isDestination = (record: AlertingObject): boolean => record.credentialName != null;
|
||||
|
||||
const accessSummary = (record: AlertingObject): string => {
|
||||
const access = record.access;
|
||||
if (!access) return "—";
|
||||
if (access.global) return "Global";
|
||||
const parts = [
|
||||
access.teams?.length ? `${access.teams.length} team${access.teams.length > 1 ? "s" : ""}` : null,
|
||||
access.orgs?.length ? `${access.orgs.length} org${access.orgs.length > 1 ? "s" : ""}` : null,
|
||||
].filter(Boolean);
|
||||
return parts.length ? parts.join(" · ") : "—";
|
||||
};
|
||||
|
||||
type CallbackRow = AlertingObject & {
|
||||
id?: string;
|
||||
mode?: "success" | "failure" | "info" | string;
|
||||
|
|
@ -39,6 +53,7 @@ export const LoggingCallbacksTable: React.FC<LoggingCallbacksProps> = ({
|
|||
onTest = () => {},
|
||||
onEdit = () => {},
|
||||
onDelete = () => {},
|
||||
onEditAccess = () => {},
|
||||
onAdd = () => {},
|
||||
}) => {
|
||||
const columns: TableProps<CallbackRow>["columns"] = [
|
||||
|
|
@ -49,13 +64,21 @@ export const LoggingCallbacksTable: React.FC<LoggingCallbacksProps> = ({
|
|||
render: (_: string, record: CallbackRow) => {
|
||||
const id = record.name;
|
||||
const displayName = availableCallbacks[id]?.ui_callback_name || id;
|
||||
return <div className="font-medium text-gray-800">{displayName}</div>;
|
||||
return (
|
||||
<div>
|
||||
<div className="font-medium text-gray-800">{displayName}</div>
|
||||
{record.destinationLabel && <div className="text-xs text-gray-500">{record.destinationLabel}</div>}
|
||||
</div>
|
||||
);
|
||||
},
|
||||
},
|
||||
{
|
||||
title: <span className="font-medium text-gray-700">Mode</span>,
|
||||
key: "mode",
|
||||
render: (_: unknown, record: CallbackRow) => {
|
||||
// Destination rows fan out on every span, so the success/failure split
|
||||
// does not apply -- only config callbacks carry a mode.
|
||||
if (isDestination(record)) return <span className="text-gray-400">—</span>;
|
||||
// Backend sends `type` (success | failure); legacy in-memory rows
|
||||
// from add-callback flow set `mode`. Read both so newly-added rows
|
||||
// and server-fetched rows both render correctly.
|
||||
|
|
@ -73,20 +96,62 @@ export const LoggingCallbacksTable: React.FC<LoggingCallbacksProps> = ({
|
|||
</span>
|
||||
);
|
||||
},
|
||||
width: 240,
|
||||
width: 200,
|
||||
},
|
||||
{
|
||||
title: <span className="font-medium text-gray-700">Scope</span>,
|
||||
key: "access",
|
||||
render: (_: unknown, record: CallbackRow) =>
|
||||
isDestination(record) ? (
|
||||
<span className="text-sm text-gray-700">{accessSummary(record)}</span>
|
||||
) : (
|
||||
<span className="text-gray-400">—</span>
|
||||
),
|
||||
width: 160,
|
||||
},
|
||||
{
|
||||
title: <span className="font-medium text-gray-700 text-right w-full block">Actions</span>,
|
||||
key: "actions",
|
||||
align: "right",
|
||||
render: (_: unknown, record: CallbackRow) => (
|
||||
<div className="flex justify-end gap-2">
|
||||
<TableIconActionButton variant="Test" tooltipText="Test Callback" onClick={() => onTest(record)} />
|
||||
<TableIconActionButton variant="Edit" tooltipText="Edit Callback" onClick={() => onEdit(record)} />
|
||||
<TableIconActionButton variant="Delete" tooltipText="Delete Callback" onClick={() => onDelete(record)} />
|
||||
</div>
|
||||
),
|
||||
width: 240,
|
||||
render: (_: unknown, record: CallbackRow) =>
|
||||
isDestination(record) ? (
|
||||
<div className="flex justify-end gap-2">
|
||||
<TableIconActionButton
|
||||
variant="Edit"
|
||||
tooltipText="Edit scope"
|
||||
dataTestId="edit-access"
|
||||
onClick={() => onEditAccess(record)}
|
||||
/>
|
||||
<TableIconActionButton
|
||||
variant="Delete"
|
||||
tooltipText="Delete destination"
|
||||
dataTestId="delete-destination"
|
||||
onClick={() => onDelete(record)}
|
||||
/>
|
||||
</div>
|
||||
) : (
|
||||
<div className="flex justify-end gap-2">
|
||||
<TableIconActionButton
|
||||
variant="Test"
|
||||
tooltipText="Test Callback"
|
||||
dataTestId="test-callback"
|
||||
onClick={() => onTest(record)}
|
||||
/>
|
||||
<TableIconActionButton
|
||||
variant="Edit"
|
||||
tooltipText="Edit Callback"
|
||||
dataTestId="edit-callback"
|
||||
onClick={() => onEdit(record)}
|
||||
/>
|
||||
<TableIconActionButton
|
||||
variant="Delete"
|
||||
tooltipText="Delete Callback"
|
||||
dataTestId="delete-callback"
|
||||
onClick={() => onDelete(record)}
|
||||
/>
|
||||
</div>
|
||||
),
|
||||
width: 200,
|
||||
},
|
||||
];
|
||||
return (
|
||||
|
|
|
|||
|
|
@ -8,6 +8,18 @@ export interface AlertingObject {
|
|||
// every row to render as "Success".
|
||||
type?: "success" | "failure" | "success_and_failure";
|
||||
variables: AlertingVariables;
|
||||
// Present only on rows backed by a logging credential (an OTEL trace
|
||||
// destination). Config-callback rows leave these unset, which is how the table
|
||||
// tells the two apart.
|
||||
credentialName?: string;
|
||||
destinationLabel?: string;
|
||||
access?: CredentialAccess;
|
||||
}
|
||||
|
||||
export interface CredentialAccess {
|
||||
global?: boolean;
|
||||
teams?: string[];
|
||||
orgs?: string[];
|
||||
}
|
||||
|
||||
export interface AlertingVariables {
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ interface CallbackConfig {
|
|||
displayName: string;
|
||||
logo: string;
|
||||
supports_key_team_logging: boolean;
|
||||
dynamic_params: Record<string, "text" | "password" | "select" | "upload" | "number">;
|
||||
dynamic_params: Record<string, "text" | "password" | "select" | "upload" | "number" | "credential">;
|
||||
description: string;
|
||||
}
|
||||
|
||||
|
|
@ -14,11 +14,10 @@ export const CALLBACK_CONFIGS: CallbackConfig[] = [
|
|||
id: "arize",
|
||||
displayName: "Arize",
|
||||
logo: `${asset_logos_folder}arize.png`,
|
||||
supports_key_team_logging: true,
|
||||
dynamic_params: {
|
||||
arize_api_key: "password",
|
||||
arize_space_id: "password",
|
||||
},
|
||||
// OTEL v2 destination: assigned per identity via the "Logging Exporters" field
|
||||
// (metadata.logging_exporters), not configured as a per-team callback here.
|
||||
supports_key_team_logging: false,
|
||||
dynamic_params: {},
|
||||
description: "Arize Logging Integration",
|
||||
},
|
||||
{
|
||||
|
|
@ -96,14 +95,22 @@ export const CALLBACK_CONFIGS: CallbackConfig[] = [
|
|||
id: "langfuse_otel",
|
||||
displayName: "Langfuse OTEL",
|
||||
logo: `${asset_logos_folder}langfuse.png`,
|
||||
supports_key_team_logging: true,
|
||||
dynamic_params: {
|
||||
langfuse_public_key: "text",
|
||||
langfuse_secret_key: "password",
|
||||
langfuse_host: "text",
|
||||
},
|
||||
// OTEL v2 destination: assigned per identity via the "Logging Exporters" field
|
||||
// (metadata.logging_exporters), not configured as a per-team callback here.
|
||||
supports_key_team_logging: false,
|
||||
dynamic_params: {},
|
||||
description: "Langfuse v3 OTEL Logging Integration",
|
||||
},
|
||||
{
|
||||
id: "weave_otel",
|
||||
displayName: "Weave OTEL",
|
||||
logo: `${asset_logos_folder}weave.png`,
|
||||
// OTEL v2 destination: assigned per identity via the "Logging Exporters" field
|
||||
// (metadata.logging_exporters), not configured as a per-team callback here.
|
||||
supports_key_team_logging: false,
|
||||
dynamic_params: {},
|
||||
description: "Weave (W&B) OTEL Logging Integration",
|
||||
},
|
||||
{
|
||||
id: "langsmith",
|
||||
displayName: "LangSmith",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,67 @@
|
|||
import { Form, Select, Switch } from "antd";
|
||||
import React from "react";
|
||||
|
||||
import { useOrganizations } from "@/app/(dashboard)/hooks/organizations/useOrganizations";
|
||||
import { useTeams } from "@/app/(dashboard)/hooks/teams/useTeams";
|
||||
import { CredentialAccess } from "../Settings/LoggingAndAlerts/LoggingCallbacks/types";
|
||||
|
||||
interface AccessControlFieldsProps {
|
||||
// value/onChange are optional so the component can be driven either directly
|
||||
// (the Add modal) or injected by an antd Form.Item (the Edit modal).
|
||||
value?: CredentialAccess;
|
||||
onChange?: (next: CredentialAccess) => void;
|
||||
}
|
||||
|
||||
// Admin-owned access for a logging destination: global (every request) or a set of
|
||||
// teams/orgs. Per-key targeting is intentionally absent here -- it lives on the key's
|
||||
// own page, since a key's token rotates on regenerate while team/org ids are stable.
|
||||
const AccessControlFields: React.FC<AccessControlFieldsProps> = ({ value = {}, onChange = () => {} }) => {
|
||||
const { data: teams } = useTeams();
|
||||
const { data: orgs } = useOrganizations();
|
||||
const isGlobal = value.global === true;
|
||||
|
||||
const teamOptions = (teams ?? []).map((t) => ({ value: t.team_id, label: t.team_alias || t.team_id }));
|
||||
const orgOptions = (orgs ?? []).map((o) => ({
|
||||
value: o.organization_id,
|
||||
label: o.organization_alias || o.organization_id,
|
||||
}));
|
||||
|
||||
return (
|
||||
<>
|
||||
<Form.Item
|
||||
label="Global"
|
||||
tooltip="When on, every request's traces export to this destination, regardless of key, team, or org."
|
||||
>
|
||||
<Switch checked={isGlobal} onChange={(global) => onChange({ ...value, global })} />
|
||||
</Form.Item>
|
||||
<Form.Item label="Teams" tooltip="Requests from keys in these teams export to this destination.">
|
||||
<Select
|
||||
mode="multiple"
|
||||
allowClear
|
||||
disabled={isGlobal}
|
||||
placeholder="Select teams"
|
||||
value={value.teams ?? []}
|
||||
onChange={(teamIds) => onChange({ ...value, teams: teamIds })}
|
||||
options={teamOptions}
|
||||
optionFilterProp="label"
|
||||
style={{ width: "100%" }}
|
||||
/>
|
||||
</Form.Item>
|
||||
<Form.Item label="Organizations" tooltip="Requests under these orgs export to this destination.">
|
||||
<Select
|
||||
mode="multiple"
|
||||
allowClear
|
||||
disabled={isGlobal}
|
||||
placeholder="Select organizations"
|
||||
value={value.orgs ?? []}
|
||||
onChange={(orgIds) => onChange({ ...value, orgs: orgIds })}
|
||||
options={orgOptions}
|
||||
optionFilterProp="label"
|
||||
style={{ width: "100%" }}
|
||||
/>
|
||||
</Form.Item>
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default AccessControlFields;
|
||||
|
|
@ -0,0 +1,74 @@
|
|||
import { Form, Modal } from "antd";
|
||||
import React from "react";
|
||||
|
||||
import { CredentialAccess } from "../Settings/LoggingAndAlerts/LoggingCallbacks/types";
|
||||
import NotificationsManager from "../molecules/notifications_manager";
|
||||
import { credentialUpdateCall } from "../networking";
|
||||
import AccessControlFields from "./AccessControlFields";
|
||||
|
||||
interface EditLoggingCredentialModalProps {
|
||||
accessToken: string;
|
||||
credentialName: string | null;
|
||||
access?: CredentialAccess;
|
||||
open: boolean;
|
||||
onClose: () => void;
|
||||
onSaved: () => void;
|
||||
}
|
||||
|
||||
interface AccessForm {
|
||||
access?: CredentialAccess;
|
||||
}
|
||||
|
||||
const EditLoggingCredentialModal: React.FC<EditLoggingCredentialModalProps> = ({
|
||||
accessToken,
|
||||
credentialName,
|
||||
access,
|
||||
open,
|
||||
onClose,
|
||||
onSaved,
|
||||
}) => {
|
||||
// destroyOnClose remounts the Form each open, so initialValues re-seeds from the
|
||||
// current destination -- no effect syncing prop into state.
|
||||
const [form] = Form.useForm<AccessForm>();
|
||||
|
||||
const handleSave = async () => {
|
||||
if (!credentialName) return;
|
||||
const current = form.getFieldsValue().access ?? {};
|
||||
// Always send the full access object: credential_info merges server-side, so a
|
||||
// sparse patch could never clear a bucket. A global grant supersedes team/org.
|
||||
const next: CredentialAccess = current.global
|
||||
? { global: true, teams: [], orgs: [] }
|
||||
: { global: false, teams: current.teams ?? [], orgs: current.orgs ?? [] };
|
||||
try {
|
||||
await credentialUpdateCall(accessToken, credentialName, {
|
||||
credential_name: credentialName,
|
||||
credential_values: {},
|
||||
credential_info: { access: next },
|
||||
});
|
||||
NotificationsManager.success("Access updated");
|
||||
onSaved();
|
||||
onClose();
|
||||
} catch (error) {
|
||||
NotificationsManager.fromBackend(error instanceof Error ? error.message : String(error));
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<Modal
|
||||
title={`Edit scope${credentialName ? ` — ${credentialName}` : ""}`}
|
||||
open={open}
|
||||
onCancel={onClose}
|
||||
onOk={handleSave}
|
||||
okText="Save"
|
||||
destroyOnClose
|
||||
>
|
||||
<Form<AccessForm> form={form} layout="vertical" preserve={false} initialValues={{ access: access ?? {} }}>
|
||||
<Form.Item name="access" noStyle>
|
||||
<AccessControlFields />
|
||||
</Form.Item>
|
||||
</Form>
|
||||
</Modal>
|
||||
);
|
||||
};
|
||||
|
||||
export default EditLoggingCredentialModal;
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
import { Select } from "antd";
|
||||
import React from "react";
|
||||
|
||||
import { useCredentials } from "@/app/(dashboard)/hooks/credentials/useCredentials";
|
||||
|
||||
interface LoggingExportersSelectProps {
|
||||
value?: string[];
|
||||
onChange?: (value: string[]) => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Multi-select of admin-owned logging destinations (credential_type=logging) that an
|
||||
* identity (key / team / org) exports its traces to. The selected names are stored in
|
||||
* metadata.logging_exporters; the proxy unions them across the identity chain and fans
|
||||
* out. Sourced from the same registry, filtered to logging credentials only.
|
||||
*/
|
||||
const LoggingExportersSelect: React.FC<LoggingExportersSelectProps> = ({ value, onChange }) => {
|
||||
const { data } = useCredentials();
|
||||
const options = (data?.credentials ?? [])
|
||||
.filter((credential) => credential.credential_info?.credential_type === "logging")
|
||||
.map((credential) => ({
|
||||
value: credential.credential_name,
|
||||
label: credential.credential_info?.host
|
||||
? `${credential.credential_name} (${credential.credential_info.host})`
|
||||
: credential.credential_name,
|
||||
}));
|
||||
|
||||
return (
|
||||
<Select
|
||||
mode="multiple"
|
||||
allowClear
|
||||
placeholder="Select logging destinations this identity exports to"
|
||||
value={value}
|
||||
onChange={onChange}
|
||||
options={options}
|
||||
style={{ width: "100%" }}
|
||||
optionFilterProp="label"
|
||||
notFoundContent="No logging destinations. Add one under Settings -> Logging Callbacks."
|
||||
/>
|
||||
);
|
||||
};
|
||||
|
||||
export default LoggingExportersSelect;
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
import { CredentialAccess } from "../Settings/LoggingAndAlerts/LoggingCallbacks/types";
|
||||
import { credentialCreateCall } from "../networking";
|
||||
import { LOGGING_DESTINATION_BACKENDS } from "./loggingDestinationFields";
|
||||
|
||||
// The set of OTEL backend ids that are created as logging destinations (credentials),
|
||||
// not as global config callbacks. The unified Add modal branches on this.
|
||||
export const LOGGING_BACKEND_IDS: ReadonlySet<string> = new Set(LOGGING_DESTINATION_BACKENDS.map((b) => b.id));
|
||||
|
||||
// Callback ids that must not surface as global callbacks. Per LIT-3850 OTEL is admin-
|
||||
// owned and routed per identity via trace destinations, and the legacy Langfuse/OTEL
|
||||
// callback paths (`langfuse` v2 SDK, `langfuse_otel` v1, the generic `otel` callback)
|
||||
// are deprecated, so these are only ever destinations -- never callback rows or options.
|
||||
export const NON_CALLBACK_LOGGING_IDS: ReadonlySet<string> = new Set([
|
||||
...LOGGING_DESTINATION_BACKENDS.map((b) => b.id),
|
||||
"langfuse",
|
||||
"otel",
|
||||
]);
|
||||
|
||||
export const backendLabel = (id?: string): string =>
|
||||
LOGGING_DESTINATION_BACKENDS.find((b) => b.id === id)?.label ?? id ?? "-";
|
||||
|
||||
export interface CreateLoggingCredentialInput {
|
||||
credentialName: string;
|
||||
backend: string;
|
||||
values: Record<string, string>;
|
||||
host?: string;
|
||||
access?: CredentialAccess;
|
||||
}
|
||||
|
||||
// One place that owns the logging-credential contract: the credential_type tag, the
|
||||
// backend in description, the non-secret host, and the admin-owned access grant.
|
||||
export const createLoggingCredential = async (accessToken: string, input: CreateLoggingCredentialInput) =>
|
||||
credentialCreateCall(accessToken, {
|
||||
credential_name: input.credentialName,
|
||||
credential_values: input.values,
|
||||
credential_info: {
|
||||
credential_type: "logging",
|
||||
description: input.backend,
|
||||
...(input.host ? { host: input.host } : {}),
|
||||
...(input.access ? { access: input.access } : {}),
|
||||
},
|
||||
});
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
// Create-time field shapes for an admin-owned logging destination, keyed by the
|
||||
// OTEL v2 backend it binds to. This is the inverse of the per-team picker: the
|
||||
// picker selects a destination by name; these fields are what an admin types when
|
||||
// CREATING the named destination in the registry. Keeping the raw keys here (the
|
||||
// admin registry) and out of the per-team form is the provider/logging separation.
|
||||
|
||||
export type LoggingFieldType = "text" | "password";
|
||||
|
||||
export interface LoggingField {
|
||||
name: string;
|
||||
label: string;
|
||||
type: LoggingFieldType;
|
||||
optional?: boolean;
|
||||
}
|
||||
|
||||
export interface LoggingDestinationBackend {
|
||||
id: string; // the callback_name the credential is bound under
|
||||
label: string;
|
||||
fields: LoggingField[];
|
||||
// The non-secret field that names the destination host/endpoint. Surfaced in the
|
||||
// list so an admin can tell e.g. an EU from a US destination apart.
|
||||
hostField: string;
|
||||
}
|
||||
|
||||
export const LOGGING_DESTINATION_BACKENDS: LoggingDestinationBackend[] = [
|
||||
{
|
||||
id: "langfuse_otel",
|
||||
label: "Langfuse",
|
||||
fields: [
|
||||
{ name: "langfuse_host", label: "Langfuse Host", type: "text" },
|
||||
{ name: "langfuse_public_key", label: "Public Key", type: "password" },
|
||||
{ name: "langfuse_secret_key", label: "Secret Key", type: "password" },
|
||||
],
|
||||
hostField: "langfuse_host",
|
||||
},
|
||||
{
|
||||
id: "arize",
|
||||
label: "Arize",
|
||||
fields: [
|
||||
{ name: "arize_space_id", label: "Space ID", type: "password" },
|
||||
{ name: "arize_api_key", label: "API Key", type: "password" },
|
||||
{ name: "arize_endpoint", label: "Endpoint (optional)", type: "text", optional: true },
|
||||
],
|
||||
hostField: "arize_endpoint",
|
||||
},
|
||||
{
|
||||
id: "weave_otel",
|
||||
label: "Weave",
|
||||
fields: [
|
||||
{ name: "wandb_api_key", label: "W&B API Key", type: "password" },
|
||||
{ name: "weave_endpoint", label: "Weave OTEL Endpoint", type: "text" },
|
||||
{ name: "weave_project_id", label: "Project (entity/project)", type: "text" },
|
||||
],
|
||||
hostField: "weave_endpoint",
|
||||
},
|
||||
{
|
||||
id: "generic",
|
||||
label: "Generic OTLP Collector",
|
||||
fields: [
|
||||
{ name: "otel_endpoint", label: "OTLP Endpoint", type: "text" },
|
||||
{ name: "otel_headers", label: "Headers (k=v,k2=v2)", type: "text", optional: true },
|
||||
],
|
||||
hostField: "otel_endpoint",
|
||||
},
|
||||
];
|
||||
|
|
@ -12,6 +12,7 @@ interface LoggingConfig {
|
|||
interface LoggingSettingsViewProps {
|
||||
loggingConfigs?: LoggingConfig[];
|
||||
disabledCallbacks?: string[];
|
||||
loggingExporters?: string[];
|
||||
variant?: "card" | "inline";
|
||||
className?: string;
|
||||
}
|
||||
|
|
@ -19,6 +20,7 @@ interface LoggingSettingsViewProps {
|
|||
export function LoggingSettingsView({
|
||||
loggingConfigs = [],
|
||||
disabledCallbacks = [],
|
||||
loggingExporters = [],
|
||||
variant = "card",
|
||||
className = "",
|
||||
}: LoggingSettingsViewProps) {
|
||||
|
|
@ -56,6 +58,29 @@ export function LoggingSettingsView({
|
|||
|
||||
const content = (
|
||||
<div className="space-y-6">
|
||||
{/* Logging Exporters (admin-owned OTEL trace destinations assigned to this identity) */}
|
||||
<div className="space-y-3">
|
||||
<div className="flex items-center gap-2">
|
||||
<CogIcon className="h-4 w-4 text-blue-600" />
|
||||
<span className="font-semibold text-gray-900">Logging Exporters</span>
|
||||
<Tag color="blue">{loggingExporters.length}</Tag>
|
||||
</div>
|
||||
{loggingExporters.length > 0 ? (
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{loggingExporters.map((name, index) => (
|
||||
<Tag key={index} color="blue">
|
||||
{name}
|
||||
</Tag>
|
||||
))}
|
||||
</div>
|
||||
) : (
|
||||
<div className="flex items-center gap-2 px-3 py-2 rounded-lg bg-gray-50 border border-gray-200">
|
||||
<CogIcon className="h-4 w-4 text-gray-400" />
|
||||
<span className="text-gray-500 text-sm">No logging exporters assigned</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Logging Integrations Section */}
|
||||
<div className="space-y-3">
|
||||
<div className="flex items-center gap-2">
|
||||
|
|
|
|||
|
|
@ -209,6 +209,18 @@ export interface CredentialItem {
|
|||
custom_llm_provider?: string;
|
||||
description?: string;
|
||||
required?: boolean;
|
||||
// "logging" tags an admin-owned trace destination (Option A: lives in the
|
||||
// free-form credential_info, no schema migration). Absent = a provider credential.
|
||||
credential_type?: string;
|
||||
// Non-secret destination host/endpoint, surfaced in the logging credentials list.
|
||||
host?: string;
|
||||
// Admin-owned access grant for a logging destination: which identities its
|
||||
// traces fan out to. global reaches everyone; teams/orgs list ids.
|
||||
access?: {
|
||||
global?: boolean;
|
||||
teams?: string[];
|
||||
orgs?: string[];
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ import React, { useMemo, useState } from "react";
|
|||
import MemberTable from "../common_components/MemberTable";
|
||||
import UserSearchModal from "../common_components/user_search_modal";
|
||||
import MCPServerSelector from "../mcp_server_management/MCPServerSelector";
|
||||
import LoggingExportersSelect from "../logging_credentials/LoggingExportersSelect";
|
||||
import { ModelSelect } from "../ModelSelect/ModelSelect";
|
||||
import NotificationsManager from "../molecules/notifications_manager";
|
||||
import {
|
||||
|
|
@ -134,7 +135,12 @@ const OrganizationInfoView: React.FC<OrganizationInfoProps> = ({
|
|||
max_budget: values.max_budget,
|
||||
budget_duration: values.budget_duration,
|
||||
},
|
||||
metadata: values.metadata ? JSON.parse(values.metadata) : null,
|
||||
metadata: {
|
||||
...(values.metadata ? JSON.parse(values.metadata) : {}),
|
||||
...(values.logging_exporters !== undefined
|
||||
? { logging_exporters: values.logging_exporters }
|
||||
: {}),
|
||||
},
|
||||
};
|
||||
|
||||
// Handle object_permission updates
|
||||
|
|
@ -308,6 +314,21 @@ const OrganizationInfoView: React.FC<OrganizationInfoProps> = ({
|
|||
))}
|
||||
</div>
|
||||
</Card>
|
||||
<Card>
|
||||
<Text>Logging Exporters</Text>
|
||||
<div className="mt-2 flex flex-wrap gap-2">
|
||||
{Array.isArray(orgData.metadata?.logging_exporters) &&
|
||||
orgData.metadata.logging_exporters.length > 0 ? (
|
||||
orgData.metadata.logging_exporters.map((name: string, index: number) => (
|
||||
<Badge key={index} color="blue">
|
||||
{name}
|
||||
</Badge>
|
||||
))
|
||||
) : (
|
||||
<Text className="text-gray-400">None</Text>
|
||||
)}
|
||||
</div>
|
||||
</Card>
|
||||
|
||||
<ObjectPermissionsView
|
||||
objectPermission={orgData.object_permission}
|
||||
|
|
@ -366,6 +387,7 @@ const OrganizationInfoView: React.FC<OrganizationInfoProps> = ({
|
|||
max_budget: orgData.litellm_budget_table.max_budget,
|
||||
budget_duration: orgData.litellm_budget_table.budget_duration,
|
||||
metadata: orgData.metadata ? JSON.stringify(orgData.metadata, null, 2) : "",
|
||||
logging_exporters: orgData.metadata?.logging_exporters || [],
|
||||
vector_stores: orgData.object_permission?.vector_stores || [],
|
||||
mcp_servers_and_groups: {
|
||||
servers: orgData.object_permission?.mcp_servers || [],
|
||||
|
|
@ -437,6 +459,14 @@ const OrganizationInfoView: React.FC<OrganizationInfoProps> = ({
|
|||
/>
|
||||
</Form.Item>
|
||||
|
||||
<Form.Item
|
||||
label="Logging Exporters"
|
||||
name="logging_exporters"
|
||||
tooltip="Admin-owned trace destinations every team in this org exports to (added to each key's and team's). Manage destinations under Settings -> Logging Credentials."
|
||||
>
|
||||
<LoggingExportersSelect />
|
||||
</Form.Item>
|
||||
|
||||
<Form.Item label="Metadata" name="metadata">
|
||||
<Input.TextArea rows={4} />
|
||||
</Form.Item>
|
||||
|
|
@ -491,6 +521,21 @@ const OrganizationInfoView: React.FC<OrganizationInfoProps> = ({
|
|||
</div>
|
||||
<div>Reset: {orgData.litellm_budget_table.budget_duration || "Never"}</div>
|
||||
</div>
|
||||
<div>
|
||||
<Text className="font-medium">Logging Exporters</Text>
|
||||
{Array.isArray(orgData.metadata?.logging_exporters) &&
|
||||
orgData.metadata.logging_exporters.length > 0 ? (
|
||||
<div className="flex flex-wrap gap-2 mt-1">
|
||||
{orgData.metadata.logging_exporters.map((name: string, index: number) => (
|
||||
<Badge key={index} color="blue">
|
||||
{name}
|
||||
</Badge>
|
||||
))}
|
||||
</div>
|
||||
) : (
|
||||
<div className="text-gray-400 mt-1">None</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<ObjectPermissionsView
|
||||
objectPermission={orgData.object_permission}
|
||||
|
|
|
|||
|
|
@ -31,6 +31,7 @@ import AlertingSettings from "./alerting/alerting_settings";
|
|||
import CloudZeroCostTracking from "./CloudZeroCostTracking/CloudZeroCostTracking";
|
||||
import DeleteResourceModal from "./common_components/DeleteResourceModal";
|
||||
import {
|
||||
credentialDeleteCall,
|
||||
deleteCallback,
|
||||
getCallbackConfigsCall,
|
||||
getCallbacksCall,
|
||||
|
|
@ -38,7 +39,17 @@ import {
|
|||
setCallbacksCall,
|
||||
} from "./networking";
|
||||
import { LoggingCallbacksTable } from "./Settings/LoggingAndAlerts/LoggingCallbacks/LoggingCallbacksTable";
|
||||
import { AlertingObject } from "./Settings/LoggingAndAlerts/LoggingCallbacks/types";
|
||||
import { AlertingObject, CredentialAccess } from "./Settings/LoggingAndAlerts/LoggingCallbacks/types";
|
||||
import { useCredentials } from "@/app/(dashboard)/hooks/credentials/useCredentials";
|
||||
import EditLoggingCredentialModal from "./logging_credentials/EditLoggingCredentialModal";
|
||||
import AccessControlFields from "./logging_credentials/AccessControlFields";
|
||||
import {
|
||||
backendLabel,
|
||||
createLoggingCredential,
|
||||
LOGGING_BACKEND_IDS,
|
||||
NON_CALLBACK_LOGGING_IDS,
|
||||
} from "./logging_credentials/loggingCredentialApi";
|
||||
import { LOGGING_DESTINATION_BACKENDS } from "./logging_credentials/loggingDestinationFields";
|
||||
import { parseErrorMessage } from "./shared/errorUtils";
|
||||
interface SettingsPageProps {
|
||||
accessToken: string | null;
|
||||
|
|
@ -247,6 +258,39 @@ const Settings: React.FC<SettingsPageProps> = ({ accessToken, userRole, userID,
|
|||
const [isAddingCallback, setIsAddingCallback] = useState(false);
|
||||
const [isDeletingCallback, setIsDeletingCallback] = useState(false);
|
||||
|
||||
// OTEL trace destinations are credentials tagged credential_type=logging; they share
|
||||
// the one Active Logging Callbacks table as rows alongside config callbacks.
|
||||
const { data: credentialData, refetch: refetchCredentials } = useCredentials();
|
||||
const [editAccessFor, setEditAccessFor] = useState<{ name: string; access?: CredentialAccess } | null>(null);
|
||||
// access for the destination branch of the unified Add modal
|
||||
const [addAccess, setAddAccess] = useState<CredentialAccess>({});
|
||||
const addingDestination = selectedCallback != null && LOGGING_BACKEND_IDS.has(selectedCallback);
|
||||
const addingDestinationFields =
|
||||
LOGGING_DESTINATION_BACKENDS.find((b) => b.id === selectedCallback)?.fields ?? [];
|
||||
|
||||
const destinationRows: AlertingObject[] = (credentialData?.credentials ?? [])
|
||||
.filter((c) => c.credential_info?.credential_type === "logging")
|
||||
.map((c) => ({
|
||||
name: c.credential_name,
|
||||
variables: {} as AlertingObject["variables"],
|
||||
credentialName: c.credential_name,
|
||||
destinationLabel: c.credential_info?.host
|
||||
? `${backendLabel(c.credential_info?.description)} · ${c.credential_info.host}`
|
||||
: backendLabel(c.credential_info?.description),
|
||||
access: c.credential_info?.access,
|
||||
}));
|
||||
|
||||
const handleDeleteDestination = async (name: string) => {
|
||||
if (!accessToken) return;
|
||||
try {
|
||||
await credentialDeleteCall(accessToken, name);
|
||||
NotificationsManager.success("Logging destination deleted");
|
||||
refetchCredentials();
|
||||
} catch (error) {
|
||||
NotificationsManager.fromBackend(parseErrorMessage(error));
|
||||
}
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
if (!accessToken) {
|
||||
return;
|
||||
|
|
@ -379,6 +423,33 @@ const Settings: React.FC<SettingsPageProps> = ({ accessToken, userRole, userID,
|
|||
if (!new_callback) {
|
||||
return;
|
||||
}
|
||||
if (LOGGING_BACKEND_IDS.has(new_callback) && accessToken) {
|
||||
const backendDef = LOGGING_DESTINATION_BACKENDS.find((b) => b.id === new_callback);
|
||||
const fields = backendDef?.fields ?? [];
|
||||
const values = Object.fromEntries(
|
||||
fields.filter((f) => formValues[f.name]).map((f) => [f.name, formValues[f.name]]),
|
||||
);
|
||||
const host = backendDef ? formValues[backendDef.hostField] : undefined;
|
||||
const hasAccess = addAccess.global || addAccess.teams?.length || addAccess.orgs?.length;
|
||||
try {
|
||||
await createLoggingCredential(accessToken, {
|
||||
credentialName: formValues.credential_name,
|
||||
backend: new_callback,
|
||||
values,
|
||||
host,
|
||||
access: hasAccess ? addAccess : undefined,
|
||||
});
|
||||
NotificationsManager.success("Logging destination created");
|
||||
refetchCredentials();
|
||||
setShowAddCallbacksModal(false);
|
||||
setSelectedCallback(null);
|
||||
setAddAccess({});
|
||||
addForm.resetFields();
|
||||
} catch (error) {
|
||||
NotificationsManager.fromBackend(parseErrorMessage(error));
|
||||
}
|
||||
return;
|
||||
}
|
||||
await handleCallbackSubmit(formValues, new_callback, false);
|
||||
};
|
||||
|
||||
|
|
@ -577,14 +648,19 @@ const Settings: React.FC<SettingsPageProps> = ({ accessToken, userRole, userID,
|
|||
<TabPanels>
|
||||
<TabPanel>
|
||||
<LoggingCallbacksTable
|
||||
callbacks={callbacks}
|
||||
callbacks={[...callbacks.filter((c) => !NON_CALLBACK_LOGGING_IDS.has(c.name)), ...destinationRows]}
|
||||
availableCallbacks={allCallbacks}
|
||||
onAdd={() => setShowAddCallbacksModal(true)}
|
||||
onEdit={(cb) => {
|
||||
setSelectedEditCallback(cb);
|
||||
setShowEditCallback(true);
|
||||
}}
|
||||
onDelete={(cb) => handleDeleteCallback(cb)}
|
||||
onEditAccess={(cb) =>
|
||||
cb.credentialName && setEditAccessFor({ name: cb.credentialName, access: cb.access })
|
||||
}
|
||||
onDelete={(cb) =>
|
||||
cb.credentialName ? handleDeleteDestination(cb.credentialName) : handleDeleteCallback(cb)
|
||||
}
|
||||
onTest={async (cb) => {
|
||||
try {
|
||||
await serviceHealthCheck(accessToken, cb.name);
|
||||
|
|
@ -594,6 +670,16 @@ const Settings: React.FC<SettingsPageProps> = ({ accessToken, userRole, userID,
|
|||
}
|
||||
}}
|
||||
/>
|
||||
{accessToken && (
|
||||
<EditLoggingCredentialModal
|
||||
accessToken={accessToken}
|
||||
credentialName={editAccessFor?.name ?? null}
|
||||
access={editAccessFor?.access}
|
||||
open={editAccessFor != null}
|
||||
onClose={() => setEditAccessFor(null)}
|
||||
onSaved={() => refetchCredentials()}
|
||||
/>
|
||||
)}
|
||||
</TabPanel>
|
||||
<TabPanel>
|
||||
<div className="p-8">
|
||||
|
|
@ -702,6 +788,7 @@ const Settings: React.FC<SettingsPageProps> = ({ accessToken, userRole, userID,
|
|||
setShowAddCallbacksModal(false);
|
||||
setSelectedCallback(null);
|
||||
setSelectedCallbackParams([]);
|
||||
setAddAccess({});
|
||||
}}
|
||||
footer={null}
|
||||
>
|
||||
|
|
@ -723,16 +810,42 @@ const Settings: React.FC<SettingsPageProps> = ({ accessToken, userRole, userID,
|
|||
labelAlign="left"
|
||||
>
|
||||
<CallbackSelector
|
||||
callbackConfigs={callbackConfigs}
|
||||
callbackConfigs={[
|
||||
...callbackConfigs.filter((c: { id: string }) => !NON_CALLBACK_LOGGING_IDS.has(c.id)),
|
||||
...LOGGING_DESTINATION_BACKENDS.map((b) => ({ id: b.id, displayName: b.label, logo: "" })),
|
||||
]}
|
||||
selectedCallback={selectedCallback}
|
||||
onCallbackChange={handleSelectedCallbackChange}
|
||||
/>
|
||||
|
||||
<DynamicParamsFields
|
||||
params={selectedCallbackParams}
|
||||
callbackConfigs={callbackConfigs}
|
||||
selectedCallback={selectedCallback}
|
||||
/>
|
||||
{addingDestination ? (
|
||||
<div className="space-y-4 mt-6 p-4 bg-gray-50 rounded-lg border">
|
||||
<FormItem
|
||||
label={<span className="text-sm font-medium text-gray-700">Name</span>}
|
||||
name="credential_name"
|
||||
rules={[{ required: true, message: "Please enter a name" }]}
|
||||
>
|
||||
<Input size="large" placeholder="e.g. langfuse-eu" />
|
||||
</FormItem>
|
||||
{addingDestinationFields.map((f) => (
|
||||
<FormItem
|
||||
key={f.name}
|
||||
label={<span className="text-sm font-medium text-gray-700">{f.label}</span>}
|
||||
name={f.name}
|
||||
rules={f.optional ? undefined : [{ required: true, message: `Please enter the ${f.label.toLowerCase()}` }]}
|
||||
>
|
||||
{f.type === "password" ? <Input.Password size="large" /> : <Input size="large" />}
|
||||
</FormItem>
|
||||
))}
|
||||
<AccessControlFields value={addAccess} onChange={setAddAccess} />
|
||||
</div>
|
||||
) : (
|
||||
<DynamicParamsFields
|
||||
params={selectedCallbackParams}
|
||||
callbackConfigs={callbackConfigs}
|
||||
selectedCallback={selectedCallback}
|
||||
/>
|
||||
)}
|
||||
|
||||
<div className="flex justify-end space-x-3 pt-6 mt-6 border-t border-gray-200">
|
||||
<Button2
|
||||
|
|
@ -740,6 +853,7 @@ const Settings: React.FC<SettingsPageProps> = ({ accessToken, userRole, userID,
|
|||
setShowAddCallbacksModal(false);
|
||||
setSelectedCallback(null);
|
||||
setSelectedCallbackParams([]);
|
||||
setAddAccess({});
|
||||
addForm.resetFields();
|
||||
}}
|
||||
disabled={isAddingCallback}
|
||||
|
|
@ -747,7 +861,7 @@ const Settings: React.FC<SettingsPageProps> = ({ accessToken, userRole, userID,
|
|||
Cancel
|
||||
</Button2>
|
||||
<Button2 htmlType="submit" loading={isAddingCallback} disabled={isAddingCallback}>
|
||||
{isAddingCallback ? "Adding..." : "Add Callback"}
|
||||
{isAddingCallback ? "Adding..." : "Add"}
|
||||
</Button2>
|
||||
</div>
|
||||
</Form>
|
||||
|
|
|
|||
|
|
@ -51,6 +51,7 @@ import NumericalInput from "../shared/numerical_input";
|
|||
import VectorStoreSelector from "../vector_store_management/VectorStoreSelector";
|
||||
import SearchToolSelector from "../SearchTools/SearchToolSelector";
|
||||
import EditLoggingSettings from "./EditLoggingSettings";
|
||||
import LoggingExportersSelect from "../logging_credentials/LoggingExportersSelect";
|
||||
import RouterSettingsAccordion, { RouterSettingsAccordionRef } from "../common_components/RouterSettingsAccordion";
|
||||
import MemberModal from "./EditMembership";
|
||||
import MemberPermissions from "./member_permissions";
|
||||
|
|
@ -530,6 +531,9 @@ const TeamInfoView: React.FC<TeamInfoProps> = ({
|
|||
guardrails: (values.guardrails || []).filter((n: string) => !globalGuardrailNames.has(n)),
|
||||
opted_out_global_guardrails: optedOutGlobalGuardrails,
|
||||
...(values.logging_settings?.length > 0 ? { logging: values.logging_settings } : {}),
|
||||
...(values.logging_exporters !== undefined
|
||||
? { logging_exporters: values.logging_exporters }
|
||||
: {}),
|
||||
disable_global_guardrails: killSwitchOnAtSave,
|
||||
soft_budget_alerting_emails:
|
||||
typeof values.soft_budget_alerting_emails === "string"
|
||||
|
|
@ -862,6 +866,9 @@ const TeamInfoView: React.FC<TeamInfoProps> = ({
|
|||
|
||||
<LoggingSettingsView
|
||||
loggingConfigs={info.metadata?.logging || []}
|
||||
loggingExporters={
|
||||
Array.isArray(info.metadata?.logging_exporters) ? info.metadata.logging_exporters : []
|
||||
}
|
||||
disabledCallbacks={[]}
|
||||
variant="card"
|
||||
/>
|
||||
|
|
@ -974,6 +981,7 @@ const TeamInfoView: React.FC<TeamInfoProps> = ({
|
|||
)
|
||||
: "",
|
||||
logging_settings: info.metadata?.logging || [],
|
||||
logging_exporters: info.metadata?.logging_exporters || [],
|
||||
secret_manager_settings: info.metadata?.secret_manager_settings
|
||||
? JSON.stringify(info.metadata.secret_manager_settings, null, 2)
|
||||
: "",
|
||||
|
|
@ -1425,6 +1433,14 @@ const TeamInfoView: React.FC<TeamInfoProps> = ({
|
|||
/>
|
||||
</Form.Item>
|
||||
|
||||
<Form.Item
|
||||
label="Logging Exporters"
|
||||
name="logging_exporters"
|
||||
tooltip="Admin-owned trace destinations this team exports to. Resolved server-side and fanned out (added to the key's and org's). Manage destinations under Settings -> Logging Callbacks."
|
||||
>
|
||||
<LoggingExportersSelect />
|
||||
</Form.Item>
|
||||
|
||||
<Form.Item label="Logging Settings" name="logging_settings">
|
||||
<EditLoggingSettings
|
||||
value={form.getFieldValue("logging_settings")}
|
||||
|
|
@ -1639,6 +1655,9 @@ const TeamInfoView: React.FC<TeamInfoProps> = ({
|
|||
|
||||
<LoggingSettingsView
|
||||
loggingConfigs={info.metadata?.logging || []}
|
||||
loggingExporters={
|
||||
Array.isArray(info.metadata?.logging_exporters) ? info.metadata.logging_exporters : []
|
||||
}
|
||||
disabledCallbacks={[]}
|
||||
variant="inline"
|
||||
className="pt-4 border-t border-gray-200"
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ import { fetchTeamModels } from "../organisms/create_key_button";
|
|||
import NumericalInput from "../shared/numerical_input";
|
||||
import { Tag } from "../tag_management/types";
|
||||
import EditLoggingSettings from "../team/EditLoggingSettings";
|
||||
import LoggingExportersSelect from "../logging_credentials/LoggingExportersSelect";
|
||||
import VectorStoreSelector from "../vector_store_management/VectorStoreSelector";
|
||||
|
||||
interface KeyEditViewProps {
|
||||
|
|
@ -189,6 +190,7 @@ export function KeyEditView({
|
|||
accessGroups: keyData.object_permission?.agent_access_groups || [],
|
||||
},
|
||||
logging_settings: extractLoggingSettings(keyData.metadata),
|
||||
logging_exporters: keyData.metadata?.logging_exporters || [],
|
||||
disabled_callbacks: Array.isArray(keyData.metadata?.litellm_disabled_callbacks)
|
||||
? mapInternalToDisplayNames(keyData.metadata.litellm_disabled_callbacks)
|
||||
: [],
|
||||
|
|
@ -218,6 +220,7 @@ export function KeyEditView({
|
|||
},
|
||||
mcp_tool_permissions: keyData.object_permission?.mcp_tool_permissions || {},
|
||||
logging_settings: extractLoggingSettings(keyData.metadata),
|
||||
logging_exporters: keyData.metadata?.logging_exporters || [],
|
||||
disabled_callbacks: Array.isArray(keyData.metadata?.litellm_disabled_callbacks)
|
||||
? mapInternalToDisplayNames(keyData.metadata.litellm_disabled_callbacks)
|
||||
: [],
|
||||
|
|
@ -719,6 +722,14 @@ export function KeyEditView({
|
|||
<Input value={projectDisplay ?? ""} disabled />
|
||||
</Form.Item>
|
||||
)}
|
||||
<Form.Item
|
||||
label="Logging Exporters"
|
||||
name="logging_exporters"
|
||||
tooltip="Admin-owned trace destinations this key exports to. Resolved server-side and fanned out (added to the team's and org's). Manage destinations under Settings -> Logging Credentials."
|
||||
>
|
||||
<LoggingExportersSelect />
|
||||
</Form.Item>
|
||||
|
||||
<Form.Item label="Logging Settings" name="logging_settings">
|
||||
<EditLoggingSettings
|
||||
value={form.getFieldValue("logging_settings")}
|
||||
|
|
|
|||
|
|
@ -254,6 +254,9 @@ export default function KeyInfoView({
|
|||
...(Array.isArray(formValues.logging_settings) && formValues.logging_settings.length > 0
|
||||
? { logging: formValues.logging_settings }
|
||||
: {}),
|
||||
...(formValues.logging_exporters !== undefined
|
||||
? { logging_exporters: formValues.logging_exporters }
|
||||
: {}),
|
||||
...(formValues.disabled_callbacks?.length > 0
|
||||
? {
|
||||
litellm_disabled_callbacks: mapDisplayToInternalNames(formValues.disabled_callbacks),
|
||||
|
|
@ -275,6 +278,9 @@ export default function KeyInfoView({
|
|||
...(Array.isArray(formValues.logging_settings) && formValues.logging_settings.length > 0
|
||||
? { logging: formValues.logging_settings }
|
||||
: {}),
|
||||
...(formValues.logging_exporters !== undefined
|
||||
? { logging_exporters: formValues.logging_exporters }
|
||||
: {}),
|
||||
...(formValues.disabled_callbacks?.length > 0
|
||||
? {
|
||||
litellm_disabled_callbacks: mapDisplayToInternalNames(formValues.disabled_callbacks),
|
||||
|
|
@ -613,6 +619,11 @@ export default function KeyInfoView({
|
|||
|
||||
<LoggingSettingsView
|
||||
loggingConfigs={extractLoggingSettings(currentKeyData.metadata)}
|
||||
loggingExporters={
|
||||
Array.isArray(currentKeyData.metadata?.logging_exporters)
|
||||
? currentKeyData.metadata.logging_exporters
|
||||
: []
|
||||
}
|
||||
disabledCallbacks={
|
||||
Array.isArray(currentKeyData.metadata?.litellm_disabled_callbacks)
|
||||
? mapInternalToDisplayNames(currentKeyData.metadata.litellm_disabled_callbacks)
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue