mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-08 03:08:45 +00:00
feat(lens): simplify deployment and first trace setup (#45230)
* feat(lens): simplify deployment and first trace setup * test(lens): keep setup fixtures within lint budgets * fix(lens): preserve setup state and harden bundled storage startup * fix(lens): handle setup recovery and Helm endpoint boundaries
This commit is contained in:
parent
ed9c02d2d9
commit
dc2a14a656
37 changed files with 1502 additions and 184 deletions
6
.github/workflows/helm_unit_test.yml
vendored
6
.github/workflows/helm_unit_test.yml
vendored
|
|
@ -22,6 +22,12 @@ jobs:
|
|||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Check Lens Compose configuration
|
||||
run: |
|
||||
python3 -m unittest discover -s deploy/lens -p 'test_*.py'
|
||||
python3 deploy/lens/configure.py --version 1.2.3 --env-file "$RUNNER_TEMP/lens.env"
|
||||
docker compose --env-file "$RUNNER_TEMP/lens.env" -f deploy/lens/stack.yaml config --quiet
|
||||
|
||||
- name: Set up Helm 3.11.1
|
||||
uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4.3.1
|
||||
with:
|
||||
|
|
|
|||
|
|
@ -6,21 +6,20 @@ Agent exporters send traces directly to Lens. LiteLLM sends its optional request
|
|||
|
||||
## New local installation
|
||||
|
||||
Install Docker with Compose and Git, then build the gateway and Lens from one checkout:
|
||||
Install Docker with Compose, Python 3.10 or later, and Git. Clone LiteLLM, select a published release that includes Lens, and start the existing Compose stack:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/BerriAI/litellm.git
|
||||
cd litellm
|
||||
export LITELLM_RELEASE_TAG="sha-$(git rev-parse HEAD)"
|
||||
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 24)"
|
||||
export LITELLM_LENS_SERVICE_TOKEN="$(openssl rand -hex 32)"
|
||||
export OPENAI_API_KEY='<your-provider-key>'
|
||||
docker compose -f docker/docker-compose.tracing.yml up -d --build
|
||||
python3 deploy/lens/configure.py --version <release-version>
|
||||
docker compose --env-file deploy/lens/.env -f deploy/lens/stack.yaml up -d --wait
|
||||
```
|
||||
|
||||
Save the generated keys privately and reuse them when restarting or upgrading. This stack binds to localhost and uses development database passwords; use your normal secrets, TLS, backups, and ingress for a hosted deployment
|
||||
The configuration command generates your keys and database passwords once, saves them in `deploy/lens/.env` with owner-only permissions, and preserves them on subsequent runs. Back up this file alongside your database volumes. Both images use the selected release; there is no local image build
|
||||
|
||||
Open `http://localhost:4002/ui/` and sign in as `admin` with `LITELLM_MASTER_KEY`. Under **Lens > Traces > Set up tracing**, generate a tracing key and copy the ingestion URL. Local exporters use `http://localhost:4318`. Model calls keep their existing LiteLLM URL and model key
|
||||
Open `http://localhost:4000/ui/` and sign in as `admin` using `LITELLM_MASTER_KEY` from the saved file. Open **Lens**, select your framework, generate a tracing key, and copy the displayed configuration. The trace endpoint is already filled in. Keep your agent's existing model credentials; the tracing key only authorizes trace uploads
|
||||
|
||||
PostgreSQL and ClickHouse use persistent Docker volumes and have no host ports. The dashboard and trace listener bind to localhost. Use your normal TLS and ingress for a hosted deployment. Stop the stack with `docker compose --env-file deploy/lens/.env -f deploy/lens/stack.yaml down`; omit `-v` to retain data
|
||||
|
||||
Under **Lens > Investigations > Connect worker**, choose an analysis model and monthly budget. The deployed service connects automatically after you save these settings. There is no worker command or second token to copy
|
||||
|
||||
|
|
@ -68,14 +67,30 @@ Lens does not need provider credentials, PostgreSQL credentials, a GPU, or the L
|
|||
|
||||
### Kubernetes with Helm
|
||||
|
||||
Both `helm/litellm` and `helm/litellm-helm` support the Lens service. Keep your existing release, namespace, values, and database configuration. Create two Secrets through your normal secret manager: `litellm-lens-service` with key `service-token`, and `litellm-lens-clickhouse` with key `url`
|
||||
Both `helm/litellm` and `helm/litellm-helm` support Lens. Keep your existing chart, release name, namespace, and values. Add:
|
||||
|
||||
```yaml
|
||||
lensWorker:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
Then run your usual Helm deployment command using the matching published chart. The chart supplies the matching Lens image, generates the shared service secret, starts a single ClickHouse instance with a persistent volume, and connects the services. Your cluster needs a default storage class, or set `lensWorker.clickhouse.storageClassName`. Bundled storage defaults to 20 GiB; set `lensWorker.clickhouse.storage` before installation to choose another size
|
||||
|
||||
When your chart manages an ingress with one hostname, the chart fills in the public tracing address and routes `/lens-ingest` directly to Lens. TLS is detected from `ingress.tls` or an ALB certificate annotation. With custom ingress, multiple hostnames, or TLS terminated elsewhere, set the address explicitly:
|
||||
|
||||
```yaml
|
||||
lensWorker:
|
||||
enabled: true
|
||||
publicUrl: https://<your-litellm-host>/lens-ingest
|
||||
```
|
||||
|
||||
For a dedicated trace hostname, configure `lensWorker.ingress.enabled`, `host`, `className`, and `tls`. Its hostname supplies the public address unless you override `publicUrl`. Internal Lens routes stay private
|
||||
|
||||
To use an existing ClickHouse database and secrets managed by your platform, keep these overrides:
|
||||
|
||||
```yaml
|
||||
lensWorker:
|
||||
enabled: true
|
||||
image:
|
||||
repository: <matching-worker-image-repository>
|
||||
digest: sha256:<matching-worker-image-digest>
|
||||
serviceTokenSecret:
|
||||
name: litellm-lens-service
|
||||
key: service-token
|
||||
|
|
@ -84,21 +99,13 @@ lensWorker:
|
|||
key: url
|
||||
clickhouseDatabase: litellm
|
||||
retentionDays: 14
|
||||
publicUrl: https://<your-litellm-host>/lens-ingest
|
||||
```
|
||||
|
||||
Set `clickhouseDatabase` and `retentionDays` to your existing database and retention before upgrading
|
||||
Supplying `clickhouseSecret.name` uses that database and disables bundled storage. Keep your database name and retention policy. For GitOps tools that render Helm without cluster access, supply both existing secrets so rendering cannot regenerate credentials
|
||||
|
||||
When the chart's main ingress is enabled, it routes `/lens-ingest` directly to Lens. With a custom ingress, add that route yourself. For a dedicated hostname, use `lensWorker.ingress.enabled`, `host`, `className`, and `tls`, and set `publicUrl` to that hostname. The chart connects LiteLLM to Lens internally and gives both services the shared secret
|
||||
Normal Helm upgrades reuse the generated credentials. Secrets are retained on uninstall, and the ClickHouse volume is retained by Kubernetes. Back them up together. Treat changing the database, storage class, or secret reference as an infrastructure change, not a routine version update
|
||||
|
||||
Update your existing component image overrides to matching builds, then use the chart from that checkout:
|
||||
|
||||
```bash
|
||||
helm upgrade --install litellm ./helm/litellm \
|
||||
--namespace litellm -f values.yaml --wait
|
||||
```
|
||||
|
||||
Use `./helm/litellm-helm` if that is your existing chart. `lensWorker.replicaCount` scales ingestion and investigations. Each replica needs access to the same ClickHouse and gateway. Credentials refresh every 30 seconds; a newly created key may briefly receive a retryable 429. Revocations propagate on refresh, and a replica stops accepting traces when its credential snapshot reaches 90 seconds
|
||||
After deployment, open **Lens**. If it was already open, click **Check setup**. The setup section moves to your framework and tracing key when Lens is reachable and storage is ready. Investigation setup asks for the analysis model and budget; the installed service connects automatically
|
||||
|
||||
## Upgrade
|
||||
|
||||
|
|
@ -106,13 +113,9 @@ Upgrade LiteLLM and Lens from the same source commit and release identity. For a
|
|||
|
||||
Keep the same databases, encryption keys, shared service secret, and public ingestion URL. Pause scheduled investigations and finish or cancel active runs, update both images through your usual deployment process, then check ingestion and run an investigation before resuming schedules. Do not run `docker compose down -v`
|
||||
|
||||
When upgrading from the Python worker, replace it with the Rust Lens service, move the existing ClickHouse connection to Lens, and configure the service URLs and secret on LiteLLM. Existing trace data remains in the same ClickHouse database; findings and settings remain in PostgreSQL. Stop the old worker. Generate dedicated tracing keys and change agent exporters to the ingestion URL. A virtual model key no longer authorizes uploads; the old gateway upload endpoints return 410 with setup guidance
|
||||
|
||||
If you retain an explicit `LENS_WORKER_TOKEN`, it remains an optional investigation credential. Normal setup uses the shared service connection and registers one managed worker identity. Configure the analysis model and billing key in the dashboard; provider keys stay on LiteLLM
|
||||
|
||||
## Development
|
||||
|
||||
`make lens-dev` starts LiteLLM, the Rust Lens service, and the hot-reload dashboard. Set `LENS_DEV_PROXY_PORT` and `LENS_DEV_UI_PORT` to change the local ports. For containers, pass the same release identity to both builds. Unversioned or incompatible workers are refused before claiming work
|
||||
`make lens-dev` starts LiteLLM, Lens, and the hot-reload dashboard. Set `LENS_DEV_PROXY_PORT` and `LENS_DEV_UI_PORT` to change the local ports. For containers, pass the same release identity to both builds. Unversioned or incompatible workers are refused before claiming work
|
||||
|
||||
## Configure a lens
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
model_list: []
|
||||
|
||||
general_settings:
|
||||
master_key: os.environ/LITELLM_MASTER_KEY
|
||||
tracing:
|
||||
|
|
|
|||
73
deploy/lens/configure.py
Normal file
73
deploy/lens/configure.py
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import shlex
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
SECRET_NAMES: Final = (
|
||||
"LITELLM_MASTER_KEY",
|
||||
"LITELLM_SALT_KEY",
|
||||
"LITELLM_LENS_SERVICE_TOKEN",
|
||||
"POSTGRES_PASSWORD",
|
||||
"CLICKHOUSE_PASSWORD",
|
||||
)
|
||||
|
||||
|
||||
def environment_content(path: Path) -> str:
|
||||
if not path.exists():
|
||||
return "".join(
|
||||
f"{name}={'sk-' if name.endswith('KEY') else ''}{secrets.token_hex(32)}\n" for name in SECRET_NAMES
|
||||
)
|
||||
saved: Final = path.read_text().splitlines()
|
||||
values: Final = dict(line.split("=", 1) for line in saved if "=" in line)
|
||||
if any(not values.get(name) for name in SECRET_NAMES):
|
||||
raise ValueError(f"{path} is incomplete. Restore your saved credentials before continuing")
|
||||
return "\n".join(line for line in saved if not line.startswith("LITELLM_VERSION=")) + "\n"
|
||||
|
||||
|
||||
def configure(path: Path, version: str) -> None:
|
||||
release: Final = version.removeprefix("v")
|
||||
if not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+(?:[-.][a-zA-Z0-9.-]+)?", release):
|
||||
raise ValueError("Use a published release version, such as 1.82.0 or 1.82.0-nightly")
|
||||
if path.is_symlink():
|
||||
raise ValueError(f"Refusing to replace a symlink: {path}")
|
||||
existing: Final = path.exists()
|
||||
content: Final = environment_content(path)
|
||||
temporary: Final = path.with_name(f".{path.name}.{secrets.token_hex(8)}")
|
||||
descriptor: Final = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
try:
|
||||
with os.fdopen(descriptor, "w") as output:
|
||||
output.write(content + f"LITELLM_VERSION={release}\n")
|
||||
if existing:
|
||||
os.replace(temporary, path)
|
||||
else:
|
||||
os.link(temporary, path)
|
||||
finally:
|
||||
temporary.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser: Final = argparse.ArgumentParser(description="Create or update the configuration for the Lens Compose stack")
|
||||
parser.add_argument("--version", required=True, help="Published LiteLLM release; Lens uses the matching version")
|
||||
parser.add_argument("--env-file", type=Path, default=Path(__file__).with_name(".env"))
|
||||
arguments: Final = parser.parse_args()
|
||||
try:
|
||||
configure(arguments.env_file, arguments.version)
|
||||
except (OSError, ValueError) as error:
|
||||
parser.exit(1, f"Could not configure Lens: {error}\n")
|
||||
sys.stdout.write(
|
||||
f"Saved {arguments.env_file}. Existing keys and database passwords are preserved\n"
|
||||
f"Start with: docker compose --env-file {shlex.quote(str(arguments.env_file))} "
|
||||
"-f deploy/lens/stack.yaml up -d --wait\n"
|
||||
"Open http://localhost:4000/ui/ and sign in as admin with LITELLM_MASTER_KEY from the saved file\n"
|
||||
"Back up this file with your database volumes. Do not commit it\n"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
52
deploy/lens/test_configure.py
Normal file
52
deploy/lens/test_configure.py
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from configure import SECRET_NAMES, configure
|
||||
|
||||
|
||||
class ComposeConfigurationTests(unittest.TestCase):
|
||||
def test_restart_and_upgrade_preserve_private_credentials_and_custom_settings(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / ".env"
|
||||
configure(path, "v1.2.3")
|
||||
original = dict(line.split("=", 1) for line in path.read_text().splitlines())
|
||||
self.assertEqual(path.stat().st_mode & 0o777, 0o600)
|
||||
self.assertEqual(len({original[name] for name in SECRET_NAMES}), len(SECRET_NAMES))
|
||||
self.assertTrue(all(len(original[name]) >= 64 for name in SECRET_NAMES))
|
||||
with path.open("a") as output:
|
||||
output.write("LITELLM_LENS_PUBLIC_URL=https://traces.example/prefix\n")
|
||||
configure(path, "1.2.3")
|
||||
configure(path, "v1.2.4-nightly")
|
||||
updated = dict(line.split("=", 1) for line in path.read_text().splitlines())
|
||||
self.assertEqual({name: updated[name] for name in SECRET_NAMES}, {name: original[name] for name in SECRET_NAMES})
|
||||
self.assertEqual(updated["LITELLM_VERSION"], "1.2.4-nightly")
|
||||
self.assertEqual(updated["LITELLM_LENS_PUBLIC_URL"], "https://traces.example/prefix")
|
||||
self.assertEqual(path.stat().st_mode & 0o777, 0o600)
|
||||
|
||||
def test_incomplete_configuration_is_never_replaced_with_new_database_passwords(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / ".env"
|
||||
original = "POSTGRES_PASSWORD=existing\n"
|
||||
path.write_text(original)
|
||||
with self.assertRaisesRegex(ValueError, "incomplete"):
|
||||
configure(path, "1.2.3")
|
||||
self.assertEqual(path.read_text(), original)
|
||||
|
||||
def test_invalid_release_and_symlink_leave_existing_files_untouched(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / ".env"
|
||||
target = Path(directory) / "saved"
|
||||
target.write_text("preserve")
|
||||
path.symlink_to(target)
|
||||
with self.assertRaisesRegex(ValueError, "symlink"):
|
||||
configure(path, "1.2.3")
|
||||
self.assertEqual(target.read_text(), "preserve")
|
||||
path.unlink()
|
||||
with self.assertRaisesRegex(ValueError, "published release"):
|
||||
configure(path, "1.2.3\nPOSTGRES_PASSWORD=replaced")
|
||||
self.assertFalse(path.exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -371,3 +371,30 @@ shutdown drain window.
|
|||
{{- $_ := set $labels "app.kubernetes.io/name" (printf "%s-lens-worker" (include "litellm.name" . | trunc 51 | trimSuffix "-")) -}}
|
||||
{{- toYaml $labels -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "litellm.lensWorker.serviceTokenSecretName" -}}
|
||||
{{- .Values.lensWorker.serviceTokenSecret.name | default (printf "%s-lens-service" (include "litellm.fullname" .)) -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "litellm.lensWorker.bundledClickhouse" -}}
|
||||
{{- if and .Values.lensWorker.enabled .Values.lensWorker.clickhouse.enabled (not .Values.lensWorker.clickhouseSecret.name) -}}true{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "litellm.lensWorker.publicUrl" -}}
|
||||
{{- if .Values.lensWorker.publicUrl -}}
|
||||
{{- .Values.lensWorker.publicUrl -}}
|
||||
{{- else if .Values.lensWorker.ingress.enabled -}}
|
||||
{{- $tls := or (not (empty .Values.lensWorker.ingress.tls)) (hasKey .Values.lensWorker.ingress.annotations "alb.ingress.kubernetes.io/certificate-arn") -}}
|
||||
{{- printf "%s://%s" (ternary "https" "http" $tls) (required "lensWorker.ingress.host is required" .Values.lensWorker.ingress.host) -}}
|
||||
{{- else if and .Values.ingress.enabled (eq (len .Values.ingress.hosts) 1) -}}
|
||||
{{- $host := required "ingress.hosts[0].host is required" (first .Values.ingress.hosts).host -}}
|
||||
{{- $tls := or (not (empty .Values.ingress.tls)) (hasKey .Values.ingress.annotations "alb.ingress.kubernetes.io/certificate-arn") -}}
|
||||
{{- printf "%s://%s/lens-ingest" (ternary "https" "http" $tls) $host -}}
|
||||
{{- else -}}
|
||||
{{- fail "lensWorker.publicUrl is required when there is no single ingress hostname" -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "litellm.lensWorker.clickhouseName" -}}
|
||||
{{- printf "%s-lens-clickhouse" (include "litellm.fullname" . | trunc 47 | trimSuffix "-") -}}
|
||||
{{- end -}}
|
||||
|
|
|
|||
|
|
@ -60,11 +60,11 @@ spec:
|
|||
- name: LITELLM_LENS_URL
|
||||
value: {{ printf "http://%s-lens-worker:%v" (include "litellm.fullname" .) .Values.lensWorker.service.port | quote }}
|
||||
- name: LITELLM_LENS_PUBLIC_URL
|
||||
value: {{ required "lensWorker.publicUrl is required" .Values.lensWorker.publicUrl | quote }}
|
||||
value: {{ include "litellm.lensWorker.publicUrl" . | quote }}
|
||||
- name: LITELLM_LENS_SERVICE_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ required "lensWorker.serviceTokenSecret.name is required" .Values.lensWorker.serviceTokenSecret.name | quote }}
|
||||
name: {{ include "litellm.lensWorker.serviceTokenSecretName" . | quote }}
|
||||
key: {{ .Values.lensWorker.serviceTokenSecret.key | quote }}
|
||||
{{- end }}
|
||||
{{- include "litellm.proxyEnv" . | nindent 12 }}
|
||||
|
|
|
|||
98
helm/litellm-helm/templates/lens/clickhouse.yaml
Normal file
98
helm/litellm-helm/templates/lens/clickhouse.yaml
Normal file
|
|
@ -0,0 +1,98 @@
|
|||
{{- if include "litellm.lensWorker.bundledClickhouse" . }}
|
||||
{{- $name := include "litellm.lensWorker.clickhouseName" . }}
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
spec:
|
||||
clusterIP: None
|
||||
selector:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/component: lens-clickhouse
|
||||
ports:
|
||||
- name: http
|
||||
port: 8123
|
||||
targetPort: http
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
spec:
|
||||
serviceName: {{ $name }}
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/component: lens-clickhouse
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/component: lens-clickhouse
|
||||
spec:
|
||||
automountServiceAccountToken: false
|
||||
{{- with .Values.imagePullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 101
|
||||
runAsGroup: 101
|
||||
fsGroup: 101
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
containers:
|
||||
- name: clickhouse
|
||||
image: {{ .Values.lensWorker.clickhouse.image | quote }}
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop: [ALL]
|
||||
env:
|
||||
- name: CLICKHOUSE_USER
|
||||
value: default
|
||||
- name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ $name }}
|
||||
key: password
|
||||
- name: CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT
|
||||
value: "1"
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 8123
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /ping
|
||||
port: http
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 60
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /ping
|
||||
port: http
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /ping
|
||||
port: http
|
||||
timeoutSeconds: 3
|
||||
resources:
|
||||
{{- toYaml .Values.lensWorker.clickhouse.resources | nindent 12 }}
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /var/lib/clickhouse
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes: [ReadWriteOnce]
|
||||
{{- if ne .Values.lensWorker.clickhouse.storageClassName nil }}
|
||||
storageClassName: {{ .Values.lensWorker.clickhouse.storageClassName | quote }}
|
||||
{{- end }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.lensWorker.clickhouse.storage | quote }}
|
||||
{{- end }}
|
||||
|
|
@ -45,13 +45,23 @@ spec:
|
|||
- name: LITELLM_LENS_SERVICE_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ required "lensWorker.serviceTokenSecret.name is required" .Values.lensWorker.serviceTokenSecret.name | quote }}
|
||||
name: {{ include "litellm.lensWorker.serviceTokenSecretName" . | quote }}
|
||||
key: {{ .Values.lensWorker.serviceTokenSecret.key | quote }}
|
||||
{{- if include "litellm.lensWorker.bundledClickhouse" . }}
|
||||
- name: CLICKHOUSE_HOST
|
||||
value: {{ include "litellm.lensWorker.clickhouseName" . | quote }}
|
||||
- name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ include "litellm.lensWorker.clickhouseName" . | quote }}
|
||||
key: password
|
||||
{{- else }}
|
||||
- name: CLICKHOUSE_URL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ required "lensWorker.clickhouseSecret.name is required" .Values.lensWorker.clickhouseSecret.name | quote }}
|
||||
key: {{ .Values.lensWorker.clickhouseSecret.key | quote }}
|
||||
{{- end }}
|
||||
- name: CLICKHOUSE_DATABASE
|
||||
value: {{ .Values.lensWorker.clickhouseDatabase | quote }}
|
||||
- name: AGENT_TRACING_RETENTION_DAYS
|
||||
|
|
|
|||
27
helm/litellm-helm/templates/lens/secrets.yaml
Normal file
27
helm/litellm-helm/templates/lens/secrets.yaml
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
{{- if and .Values.lensWorker.enabled (not .Values.lensWorker.serviceTokenSecret.name) }}
|
||||
{{- $name := include "litellm.lensWorker.serviceTokenSecretName" . }}
|
||||
{{- $existing := lookup "v1" "Secret" .Release.Namespace $name }}
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
annotations:
|
||||
helm.sh/resource-policy: keep
|
||||
type: Opaque
|
||||
data:
|
||||
{{ .Values.lensWorker.serviceTokenSecret.key }}: {{ if $existing }}{{ required "Saved Lens service secret is missing its key" (index $existing.data .Values.lensWorker.serviceTokenSecret.key) | quote }}{{ else }}{{ randAlphaNum 64 | b64enc | quote }}{{ end }}
|
||||
{{- end }}
|
||||
{{- if include "litellm.lensWorker.bundledClickhouse" . }}
|
||||
{{- $name := include "litellm.lensWorker.clickhouseName" . }}
|
||||
{{- $existing := lookup "v1" "Secret" .Release.Namespace $name }}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
annotations:
|
||||
helm.sh/resource-policy: keep
|
||||
type: Opaque
|
||||
data:
|
||||
password: {{ if $existing }}{{ required "Saved Lens ClickHouse secret is missing its password" (index $existing.data "password") | quote }}{{ else }}{{ randAlphaNum 64 | b64enc | quote }}{{ end }}
|
||||
{{- end }}
|
||||
84
helm/litellm-helm/tests/lens_endpoint_defaults_tests.yaml
Normal file
84
helm/litellm-helm/tests/lens_endpoint_defaults_tests.yaml
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
suite: Lens endpoint defaults
|
||||
templates:
|
||||
- deployment.yaml
|
||||
- configmap-litellm.yaml
|
||||
set:
|
||||
lensWorker.enabled: true
|
||||
ingress.enabled: true
|
||||
ingress.hosts: [{host: gateway.example, paths: [{path: /, pathType: Prefix}]}]
|
||||
ingress.tls:
|
||||
- hosts: [gateway.example]
|
||||
secretName: tls
|
||||
tests:
|
||||
- it: derives the trace endpoint from the deployment hostname and TLS
|
||||
template: deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: https://gateway.example/lens-ingest
|
||||
- it: preserves an explicitly configured public address
|
||||
set:
|
||||
lensWorker.publicUrl: https://custom.example/prefix
|
||||
template: deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: https://custom.example/prefix
|
||||
- it: uses the dedicated Lens ingress when configured
|
||||
set:
|
||||
lensWorker.ingress.enabled: true
|
||||
lensWorker.ingress.host: traces.example
|
||||
lensWorker.ingress.tls:
|
||||
- hosts: [traces.example]
|
||||
secretName: traces-tls
|
||||
template: deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: https://traces.example
|
||||
- it: uses HTTPS for a dedicated ALB ingress with certificate annotations
|
||||
set:
|
||||
lensWorker.ingress.enabled: true
|
||||
lensWorker.ingress.host: traces.example
|
||||
lensWorker.ingress.className: alb
|
||||
lensWorker.ingress.tls: []
|
||||
lensWorker.ingress.annotations:
|
||||
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:123456789012:certificate/test
|
||||
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'
|
||||
alb.ingress.kubernetes.io/ssl-redirect: "443"
|
||||
template: deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: https://traces.example
|
||||
- it: uses HTTP for a dedicated ingress without its own TLS
|
||||
set:
|
||||
lensWorker.ingress.enabled: true
|
||||
lensWorker.ingress.host: traces.example
|
||||
lensWorker.ingress.tls: []
|
||||
lensWorker.ingress.annotations: {}
|
||||
template: deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: http://traces.example
|
||||
- it: uses HTTP when ingress has no TLS
|
||||
set:
|
||||
ingress.tls: []
|
||||
template: deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: http://gateway.example/lens-ingest
|
||||
45
helm/litellm-helm/tests/lens_saved_secrets_tests.yaml
Normal file
45
helm/litellm-helm/tests/lens_saved_secrets_tests.yaml
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
suite: Lens preserves credentials across Helm upgrades
|
||||
templates:
|
||||
- lens/secrets.yaml
|
||||
set:
|
||||
fullnameOverride: lens-test
|
||||
lensWorker.enabled: true
|
||||
release:
|
||||
namespace: lens
|
||||
name: lens-test
|
||||
upgrade: true
|
||||
kubernetesProvider:
|
||||
scheme:
|
||||
v1/Secret:
|
||||
gvr:
|
||||
version: v1
|
||||
resource: secrets
|
||||
namespaced: true
|
||||
objects:
|
||||
- apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: lens-test-lens-service
|
||||
namespace: lens
|
||||
data:
|
||||
service-token: c2F2ZWQtc2VydmljZS10b2tlbg==
|
||||
- apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: lens-test-lens-clickhouse
|
||||
namespace: lens
|
||||
data:
|
||||
password: c2F2ZWQtZGF0YWJhc2UtcGFzc3dvcmQ=
|
||||
tests:
|
||||
- it: reuses the service credential instead of breaking running services
|
||||
documentIndex: 0
|
||||
asserts:
|
||||
- equal:
|
||||
path: data.service-token
|
||||
value: c2F2ZWQtc2VydmljZS10b2tlbg==
|
||||
- it: reuses the database password instead of locking out stored traces
|
||||
documentIndex: 1
|
||||
asserts:
|
||||
- equal:
|
||||
path: data.password
|
||||
value: c2F2ZWQtZGF0YWJhc2UtcGFzc3dvcmQ=
|
||||
|
|
@ -117,7 +117,7 @@ tests:
|
|||
lensWorker.clickhouseSecret.name: lens-storage
|
||||
asserts:
|
||||
- failedTemplate:
|
||||
errorMessage: lensWorker.publicUrl is required
|
||||
errorMessage: lensWorker.publicUrl is required when there is no single ingress hostname
|
||||
- it: omits Lens connection settings when disabled in deployment.yaml
|
||||
template: deployment.yaml
|
||||
asserts:
|
||||
|
|
|
|||
160
helm/litellm-helm/tests/lens_setup_tests.yaml
Normal file
160
helm/litellm-helm/tests/lens_setup_tests.yaml
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
suite: Lens managed setup
|
||||
set:
|
||||
fullnameOverride: lens-test
|
||||
lensWorker.enabled: true
|
||||
lensWorker.publicUrl: https://traces.example
|
||||
release:
|
||||
namespace: lens
|
||||
name: lens-test
|
||||
templates:
|
||||
- lens/deployment.yaml
|
||||
- lens/clickhouse.yaml
|
||||
- lens/secrets.yaml
|
||||
tests:
|
||||
- it: generates both private credentials for a new installation
|
||||
template: lens/secrets.yaml
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 2
|
||||
- matchRegex:
|
||||
path: data.service-token
|
||||
pattern: '^[A-Za-z0-9+/]{86}==$'
|
||||
documentIndex: 0
|
||||
- matchRegex:
|
||||
path: data.password
|
||||
pattern: '^[A-Za-z0-9+/]{86}==$'
|
||||
documentIndex: 1
|
||||
- equal:
|
||||
path: metadata.annotations["helm.sh/resource-policy"]
|
||||
value: keep
|
||||
- it: connects Lens to its private bundled storage
|
||||
template: lens/deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_HOST
|
||||
value: lens-test-lens-clickhouse
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: lens-test-lens-clickhouse
|
||||
key: password
|
||||
- it: stores traces on a persistent volume with the chosen storage class
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 1
|
||||
set:
|
||||
lensWorker.clickhouse.storage: 40Gi
|
||||
lensWorker.clickhouse.storageClassName: fast
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.volumeClaimTemplates[0].spec
|
||||
value:
|
||||
accessModes: [ReadWriteOnce]
|
||||
storageClassName: fast
|
||||
resources:
|
||||
requests:
|
||||
storage: 40Gi
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: lens-test-lens-clickhouse
|
||||
key: password
|
||||
- it: keeps an existing external database instead of creating a new one
|
||||
template: lens/clickhouse.yaml
|
||||
set:
|
||||
lensWorker.clickhouseSecret.name: external-clickhouse
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
- it: leaves supplied secrets under their existing manager
|
||||
template: lens/secrets.yaml
|
||||
set:
|
||||
lensWorker.clickhouseSecret.name: external-clickhouse
|
||||
lensWorker.serviceTokenSecret.name: external-service
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
- it: creates no credentials or database when Lens is disabled
|
||||
templates:
|
||||
- lens/secrets.yaml
|
||||
- lens/clickhouse.yaml
|
||||
set:
|
||||
lensWorker.enabled: false
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
- it: requires an external database when bundled storage is explicitly disabled
|
||||
template: lens/deployment.yaml
|
||||
set:
|
||||
lensWorker.clickhouse.enabled: false
|
||||
asserts:
|
||||
- failedTemplate:
|
||||
errorMessage: lensWorker.clickhouseSecret.name is required
|
||||
- it: keeps storage names valid and uses the same name for the connection
|
||||
set:
|
||||
fullnameOverride: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
asserts:
|
||||
- matchRegex:
|
||||
path: metadata.name
|
||||
pattern: '^[a-z]([-a-z0-9]{0,61}[a-z0-9])?$'
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 0
|
||||
- matchRegex:
|
||||
path: metadata.name
|
||||
pattern: '^[a-z]([-a-z0-9]{0,61}[a-z0-9])?$'
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 1
|
||||
- equal:
|
||||
path: metadata.name
|
||||
value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 0
|
||||
- equal:
|
||||
path: spec.serviceName
|
||||
value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 1
|
||||
- equal:
|
||||
path: metadata.name
|
||||
value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
template: lens/secrets.yaml
|
||||
documentIndex: 1
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_HOST
|
||||
value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
template: lens/deployment.yaml
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
key: password
|
||||
template: lens/deployment.yaml
|
||||
|
||||
- it: allows cold storage startup and inherits registry credentials
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 1
|
||||
set:
|
||||
imagePullSecrets: [{name: registry-auth}]
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.template.spec.imagePullSecrets
|
||||
value: [{name: registry-auth}]
|
||||
- equal:
|
||||
path: spec.template.spec.containers[0].startupProbe
|
||||
value:
|
||||
httpGet: {path: /ping, port: http}
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 60
|
||||
|
|
@ -667,6 +667,17 @@ lensWorker:
|
|||
serviceTokenSecret:
|
||||
name: ""
|
||||
key: service-token
|
||||
clickhouse:
|
||||
enabled: true
|
||||
image: clickhouse/clickhouse-server:26.9.6.6
|
||||
storage: 20Gi
|
||||
storageClassName: null
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
memory: 2Gi
|
||||
clickhouseDatabase: litellm
|
||||
retentionDays: 14
|
||||
clickhouseSecret:
|
||||
|
|
|
|||
|
|
@ -520,11 +520,11 @@ shutdown drain window.
|
|||
- name: LITELLM_LENS_URL
|
||||
value: {{ printf "http://%s-lens-worker:%v" (include "litellm.fullname" .) .Values.lensWorker.service.port | quote }}
|
||||
- name: LITELLM_LENS_PUBLIC_URL
|
||||
value: {{ required "lensWorker.publicUrl is required" .Values.lensWorker.publicUrl | quote }}
|
||||
value: {{ include "litellm.lensWorker.publicUrl" . | quote }}
|
||||
- name: LITELLM_LENS_SERVICE_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ required "lensWorker.serviceTokenSecret.name is required" .Values.lensWorker.serviceTokenSecret.name | quote }}
|
||||
name: {{ include "litellm.lensWorker.serviceTokenSecretName" . | quote }}
|
||||
key: {{ .Values.lensWorker.serviceTokenSecret.key | quote }}
|
||||
{{- end }}
|
||||
{{- end -}}
|
||||
|
|
@ -534,3 +534,29 @@ shutdown drain window.
|
|||
{{- $_ := set $labels "app.kubernetes.io/name" (printf "%s-lens-worker" (include "litellm.name" . | trunc 51 | trimSuffix "-")) -}}
|
||||
{{- toYaml $labels -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "litellm.lensWorker.serviceTokenSecretName" -}}
|
||||
{{- .Values.lensWorker.serviceTokenSecret.name | default (printf "%s-lens-service" (include "litellm.fullname" .)) -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "litellm.lensWorker.bundledClickhouse" -}}
|
||||
{{- if and .Values.lensWorker.enabled .Values.lensWorker.clickhouse.enabled (not .Values.lensWorker.clickhouseSecret.name) -}}true{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "litellm.lensWorker.publicUrl" -}}
|
||||
{{- if .Values.lensWorker.publicUrl -}}
|
||||
{{- .Values.lensWorker.publicUrl -}}
|
||||
{{- else if .Values.lensWorker.ingress.enabled -}}
|
||||
{{- $tls := or (not (empty .Values.lensWorker.ingress.tls)) (hasKey .Values.lensWorker.ingress.annotations "alb.ingress.kubernetes.io/certificate-arn") -}}
|
||||
{{- printf "%s://%s" (ternary "https" "http" $tls) (required "lensWorker.ingress.host is required" .Values.lensWorker.ingress.host) -}}
|
||||
{{- else if and .Values.ingress.enabled .Values.ingress.host -}}
|
||||
{{- $tls := or (not (empty .Values.ingress.tls)) (hasKey .Values.ingress.annotations "alb.ingress.kubernetes.io/certificate-arn") -}}
|
||||
{{- printf "%s://%s/lens-ingest" (ternary "https" "http" $tls) .Values.ingress.host -}}
|
||||
{{- else -}}
|
||||
{{- fail "lensWorker.publicUrl is required when there is no single ingress hostname" -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "litellm.lensWorker.clickhouseName" -}}
|
||||
{{- printf "%s-lens-clickhouse" (include "litellm.fullname" . | trunc 47 | trimSuffix "-") -}}
|
||||
{{- end -}}
|
||||
|
|
|
|||
98
helm/litellm/templates/lens/clickhouse.yaml
Normal file
98
helm/litellm/templates/lens/clickhouse.yaml
Normal file
|
|
@ -0,0 +1,98 @@
|
|||
{{- if include "litellm.lensWorker.bundledClickhouse" . }}
|
||||
{{- $name := include "litellm.lensWorker.clickhouseName" . }}
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
spec:
|
||||
clusterIP: None
|
||||
selector:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/component: lens-clickhouse
|
||||
ports:
|
||||
- name: http
|
||||
port: 8123
|
||||
targetPort: http
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
spec:
|
||||
serviceName: {{ $name }}
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/component: lens-clickhouse
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/component: lens-clickhouse
|
||||
spec:
|
||||
automountServiceAccountToken: false
|
||||
{{- with .Values.imagePullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 101
|
||||
runAsGroup: 101
|
||||
fsGroup: 101
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
containers:
|
||||
- name: clickhouse
|
||||
image: {{ .Values.lensWorker.clickhouse.image | quote }}
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop: [ALL]
|
||||
env:
|
||||
- name: CLICKHOUSE_USER
|
||||
value: default
|
||||
- name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ $name }}
|
||||
key: password
|
||||
- name: CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT
|
||||
value: "1"
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 8123
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /ping
|
||||
port: http
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 60
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /ping
|
||||
port: http
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /ping
|
||||
port: http
|
||||
timeoutSeconds: 3
|
||||
resources:
|
||||
{{- toYaml .Values.lensWorker.clickhouse.resources | nindent 12 }}
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /var/lib/clickhouse
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes: [ReadWriteOnce]
|
||||
{{- if ne .Values.lensWorker.clickhouse.storageClassName nil }}
|
||||
storageClassName: {{ .Values.lensWorker.clickhouse.storageClassName | quote }}
|
||||
{{- end }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.lensWorker.clickhouse.storage | quote }}
|
||||
{{- end }}
|
||||
|
|
@ -45,13 +45,23 @@ spec:
|
|||
- name: LITELLM_LENS_SERVICE_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ required "lensWorker.serviceTokenSecret.name is required" .Values.lensWorker.serviceTokenSecret.name | quote }}
|
||||
name: {{ include "litellm.lensWorker.serviceTokenSecretName" . | quote }}
|
||||
key: {{ .Values.lensWorker.serviceTokenSecret.key | quote }}
|
||||
{{- if include "litellm.lensWorker.bundledClickhouse" . }}
|
||||
- name: CLICKHOUSE_HOST
|
||||
value: {{ include "litellm.lensWorker.clickhouseName" . | quote }}
|
||||
- name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ include "litellm.lensWorker.clickhouseName" . | quote }}
|
||||
key: password
|
||||
{{- else }}
|
||||
- name: CLICKHOUSE_URL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ required "lensWorker.clickhouseSecret.name is required" .Values.lensWorker.clickhouseSecret.name | quote }}
|
||||
key: {{ .Values.lensWorker.clickhouseSecret.key | quote }}
|
||||
{{- end }}
|
||||
- name: CLICKHOUSE_DATABASE
|
||||
value: {{ .Values.lensWorker.clickhouseDatabase | quote }}
|
||||
- name: AGENT_TRACING_RETENTION_DAYS
|
||||
|
|
|
|||
27
helm/litellm/templates/lens/secrets.yaml
Normal file
27
helm/litellm/templates/lens/secrets.yaml
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
{{- if and .Values.lensWorker.enabled (not .Values.lensWorker.serviceTokenSecret.name) }}
|
||||
{{- $name := include "litellm.lensWorker.serviceTokenSecretName" . }}
|
||||
{{- $existing := lookup "v1" "Secret" .Release.Namespace $name }}
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
annotations:
|
||||
helm.sh/resource-policy: keep
|
||||
type: Opaque
|
||||
data:
|
||||
{{ .Values.lensWorker.serviceTokenSecret.key }}: {{ if $existing }}{{ required "Saved Lens service secret is missing its key" (index $existing.data .Values.lensWorker.serviceTokenSecret.key) | quote }}{{ else }}{{ randAlphaNum 64 | b64enc | quote }}{{ end }}
|
||||
{{- end }}
|
||||
{{- if include "litellm.lensWorker.bundledClickhouse" . }}
|
||||
{{- $name := include "litellm.lensWorker.clickhouseName" . }}
|
||||
{{- $existing := lookup "v1" "Secret" .Release.Namespace $name }}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
annotations:
|
||||
helm.sh/resource-policy: keep
|
||||
type: Opaque
|
||||
data:
|
||||
password: {{ if $existing }}{{ required "Saved Lens ClickHouse secret is missing its password" (index $existing.data "password") | quote }}{{ else }}{{ randAlphaNum 64 | b64enc | quote }}{{ end }}
|
||||
{{- end }}
|
||||
86
helm/litellm/tests/lens_endpoint_defaults_tests.yaml
Normal file
86
helm/litellm/tests/lens_endpoint_defaults_tests.yaml
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
suite: Lens endpoint defaults
|
||||
templates:
|
||||
- gateway/deployment.yaml
|
||||
- gateway/configmap.yaml
|
||||
set:
|
||||
lensWorker.enabled: true
|
||||
ingress.enabled: true
|
||||
ingress.host: gateway.example
|
||||
ingress.tls:
|
||||
- hosts: [gateway.example]
|
||||
secretName: tls
|
||||
tests:
|
||||
- it: derives the trace endpoint from the deployment hostname and TLS
|
||||
template: gateway/deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: https://gateway.example/lens-ingest
|
||||
- it: preserves an explicitly configured public address
|
||||
set:
|
||||
lensWorker.publicUrl: https://custom.example/prefix
|
||||
template: gateway/deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: https://custom.example/prefix
|
||||
- it: uses the dedicated Lens ingress when configured
|
||||
set:
|
||||
lensWorker.ingress.enabled: true
|
||||
lensWorker.ingress.host: traces.example
|
||||
lensWorker.ingress.tls:
|
||||
- hosts: [traces.example]
|
||||
secretName: traces-tls
|
||||
template: gateway/deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: https://traces.example
|
||||
- it: uses HTTPS for a dedicated ALB ingress with certificate annotations
|
||||
set:
|
||||
lensWorker.ingress.enabled: true
|
||||
lensWorker.ingress.host: traces.example
|
||||
lensWorker.ingress.className: alb
|
||||
lensWorker.ingress.tls: []
|
||||
lensWorker.ingress.annotations:
|
||||
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:123456789012:certificate/test
|
||||
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'
|
||||
alb.ingress.kubernetes.io/ssl-redirect: "443"
|
||||
template: gateway/deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: https://traces.example
|
||||
- it: uses HTTP for a dedicated ingress without its own TLS
|
||||
set:
|
||||
lensWorker.ingress.enabled: true
|
||||
lensWorker.ingress.host: traces.example
|
||||
lensWorker.ingress.tls: []
|
||||
lensWorker.ingress.annotations: {}
|
||||
template: gateway/deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: http://traces.example
|
||||
- it: uses HTTP when ingress has no TLS
|
||||
set:
|
||||
ingress.tls: []
|
||||
template: gateway/deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_PUBLIC_URL
|
||||
value: http://gateway.example/lens-ingest
|
||||
values:
|
||||
- ./values/required.yaml
|
||||
47
helm/litellm/tests/lens_saved_secrets_tests.yaml
Normal file
47
helm/litellm/tests/lens_saved_secrets_tests.yaml
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
suite: Lens preserves credentials across Helm upgrades
|
||||
templates:
|
||||
- lens/secrets.yaml
|
||||
set:
|
||||
fullnameOverride: lens-test
|
||||
lensWorker.enabled: true
|
||||
release:
|
||||
namespace: lens
|
||||
name: lens-test
|
||||
upgrade: true
|
||||
kubernetesProvider:
|
||||
scheme:
|
||||
v1/Secret:
|
||||
gvr:
|
||||
version: v1
|
||||
resource: secrets
|
||||
namespaced: true
|
||||
objects:
|
||||
- apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: lens-test-lens-service
|
||||
namespace: lens
|
||||
data:
|
||||
service-token: c2F2ZWQtc2VydmljZS10b2tlbg==
|
||||
- apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: lens-test-lens-clickhouse
|
||||
namespace: lens
|
||||
data:
|
||||
password: c2F2ZWQtZGF0YWJhc2UtcGFzc3dvcmQ=
|
||||
tests:
|
||||
- it: reuses the service credential instead of breaking running services
|
||||
documentIndex: 0
|
||||
asserts:
|
||||
- equal:
|
||||
path: data.service-token
|
||||
value: c2F2ZWQtc2VydmljZS10b2tlbg==
|
||||
- it: reuses the database password instead of locking out stored traces
|
||||
documentIndex: 1
|
||||
asserts:
|
||||
- equal:
|
||||
path: data.password
|
||||
value: c2F2ZWQtZGF0YWJhc2UtcGFzc3dvcmQ=
|
||||
values:
|
||||
- ./values/required.yaml
|
||||
|
|
@ -136,7 +136,7 @@ tests:
|
|||
lensWorker.clickhouseSecret.name: lens-storage
|
||||
asserts:
|
||||
- failedTemplate:
|
||||
errorMessage: lensWorker.publicUrl is required
|
||||
errorMessage: lensWorker.publicUrl is required when there is no single ingress hostname
|
||||
- it: omits Lens connection settings when disabled in gateway/deployment.yaml
|
||||
template: gateway/deployment.yaml
|
||||
asserts:
|
||||
|
|
|
|||
162
helm/litellm/tests/lens_setup_tests.yaml
Normal file
162
helm/litellm/tests/lens_setup_tests.yaml
Normal file
|
|
@ -0,0 +1,162 @@
|
|||
suite: Lens managed setup
|
||||
set:
|
||||
fullnameOverride: lens-test
|
||||
lensWorker.enabled: true
|
||||
lensWorker.publicUrl: https://traces.example
|
||||
release:
|
||||
namespace: lens
|
||||
name: lens-test
|
||||
templates:
|
||||
- lens/deployment.yaml
|
||||
- lens/clickhouse.yaml
|
||||
- lens/secrets.yaml
|
||||
tests:
|
||||
- it: generates both private credentials for a new installation
|
||||
template: lens/secrets.yaml
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 2
|
||||
- matchRegex:
|
||||
path: data.service-token
|
||||
pattern: '^[A-Za-z0-9+/]{86}==$'
|
||||
documentIndex: 0
|
||||
- matchRegex:
|
||||
path: data.password
|
||||
pattern: '^[A-Za-z0-9+/]{86}==$'
|
||||
documentIndex: 1
|
||||
- equal:
|
||||
path: metadata.annotations["helm.sh/resource-policy"]
|
||||
value: keep
|
||||
- it: connects Lens to its private bundled storage
|
||||
template: lens/deployment.yaml
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_HOST
|
||||
value: lens-test-lens-clickhouse
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: lens-test-lens-clickhouse
|
||||
key: password
|
||||
- it: stores traces on a persistent volume with the chosen storage class
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 1
|
||||
set:
|
||||
lensWorker.clickhouse.storage: 40Gi
|
||||
lensWorker.clickhouse.storageClassName: fast
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.volumeClaimTemplates[0].spec
|
||||
value:
|
||||
accessModes: [ReadWriteOnce]
|
||||
storageClassName: fast
|
||||
resources:
|
||||
requests:
|
||||
storage: 40Gi
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: lens-test-lens-clickhouse
|
||||
key: password
|
||||
- it: keeps an existing external database instead of creating a new one
|
||||
template: lens/clickhouse.yaml
|
||||
set:
|
||||
lensWorker.clickhouseSecret.name: external-clickhouse
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
- it: leaves supplied secrets under their existing manager
|
||||
template: lens/secrets.yaml
|
||||
set:
|
||||
lensWorker.clickhouseSecret.name: external-clickhouse
|
||||
lensWorker.serviceTokenSecret.name: external-service
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
- it: creates no credentials or database when Lens is disabled
|
||||
templates:
|
||||
- lens/secrets.yaml
|
||||
- lens/clickhouse.yaml
|
||||
set:
|
||||
lensWorker.enabled: false
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
- it: requires an external database when bundled storage is explicitly disabled
|
||||
template: lens/deployment.yaml
|
||||
set:
|
||||
lensWorker.clickhouse.enabled: false
|
||||
asserts:
|
||||
- failedTemplate:
|
||||
errorMessage: lensWorker.clickhouseSecret.name is required
|
||||
- it: keeps storage names valid and uses the same name for the connection
|
||||
set:
|
||||
fullnameOverride: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
asserts:
|
||||
- matchRegex:
|
||||
path: metadata.name
|
||||
pattern: '^[a-z]([-a-z0-9]{0,61}[a-z0-9])?$'
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 0
|
||||
- matchRegex:
|
||||
path: metadata.name
|
||||
pattern: '^[a-z]([-a-z0-9]{0,61}[a-z0-9])?$'
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 1
|
||||
- equal:
|
||||
path: metadata.name
|
||||
value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 0
|
||||
- equal:
|
||||
path: spec.serviceName
|
||||
value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 1
|
||||
- equal:
|
||||
path: metadata.name
|
||||
value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
template: lens/secrets.yaml
|
||||
documentIndex: 1
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_HOST
|
||||
value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
template: lens/deployment.yaml
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: CLICKHOUSE_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lens-clickhouse
|
||||
key: password
|
||||
template: lens/deployment.yaml
|
||||
- it: allows cold storage startup and inherits registry credentials
|
||||
template: lens/clickhouse.yaml
|
||||
documentIndex: 1
|
||||
set:
|
||||
imagePullSecrets: [{name: registry-auth}]
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.template.spec.imagePullSecrets
|
||||
value: [{name: registry-auth}]
|
||||
- equal:
|
||||
path: spec.template.spec.containers[0].startupProbe
|
||||
value:
|
||||
httpGet: {path: /ping, port: http}
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 60
|
||||
|
||||
values:
|
||||
- ./values/required.yaml
|
||||
|
|
@ -77,13 +77,20 @@ tests:
|
|||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
- it: requires a shared service secret when enabled
|
||||
- it: uses the managed service secret when none is supplied
|
||||
template: lens/deployment.yaml
|
||||
set:
|
||||
lensWorker.enabled: true
|
||||
lensWorker.publicUrl: https://traces.example
|
||||
asserts:
|
||||
- failedTemplate:
|
||||
errorMessage: lensWorker.serviceTokenSecret.name is required
|
||||
- contains:
|
||||
path: spec.template.spec.containers[0].env
|
||||
content:
|
||||
name: LITELLM_LENS_SERVICE_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: RELEASE-NAME-litellm-lens-service
|
||||
key: service-token
|
||||
- it: uses the chart release and a secret without granting Kubernetes access
|
||||
template: lens/deployment.yaml
|
||||
chart:
|
||||
|
|
|
|||
|
|
@ -644,6 +644,17 @@ lensWorker:
|
|||
serviceTokenSecret:
|
||||
name: ""
|
||||
key: service-token
|
||||
clickhouse:
|
||||
enabled: true
|
||||
image: clickhouse/clickhouse-server:26.9.6.6
|
||||
storage: 20Gi
|
||||
storageClassName: null
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
memory: 2Gi
|
||||
clickhouseDatabase: litellm
|
||||
retentionDays: 14
|
||||
clickhouseSecret:
|
||||
|
|
|
|||
|
|
@ -194,16 +194,37 @@ async def service_connection(auth: Auth) -> ServiceConnection:
|
|||
public_url: Final = os.environ.get("LITELLM_LENS_PUBLIC_URL", "").rstrip("/")
|
||||
try:
|
||||
connection: Final = LensConnection.from_env()
|
||||
except ValueError:
|
||||
return ServiceConnection(
|
||||
url=public_url,
|
||||
connected=False,
|
||||
status=ServiceStatus(),
|
||||
configured=bool(os.environ.get("LITELLM_LENS_URL")),
|
||||
release=release_tag(),
|
||||
)
|
||||
try:
|
||||
client: Final = connection.control_client()
|
||||
async with client.stream(
|
||||
"GET", connection.endpoint("/internal/status"), headers=connection.headers, timeout=2
|
||||
) as response:
|
||||
if response.status_code == 200:
|
||||
status: Final = ServiceStatus.model_validate_json(await bounded_response(response, 16 * 1024))
|
||||
return ServiceConnection(url=public_url, connected=True, status=status)
|
||||
return ServiceConnection(
|
||||
url=public_url,
|
||||
connected=True,
|
||||
status=status,
|
||||
configured=True,
|
||||
release=release_tag(),
|
||||
)
|
||||
except (ValueError, RuntimeError, httpx.HTTPError):
|
||||
pass
|
||||
return ServiceConnection(url=public_url, connected=False, status=ServiceStatus())
|
||||
return ServiceConnection(
|
||||
url=public_url,
|
||||
connected=False,
|
||||
status=ServiceStatus(),
|
||||
configured=True,
|
||||
release=release_tag(),
|
||||
)
|
||||
|
||||
|
||||
async def credential_snapshot() -> IngestionSnapshot:
|
||||
|
|
|
|||
|
|
@ -59,6 +59,8 @@ class ServiceConnection(Record):
|
|||
url: str
|
||||
connected: bool
|
||||
status: ServiceStatus
|
||||
configured: bool = False
|
||||
release: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
|
|
|
|||
|
|
@ -1048,6 +1048,7 @@ async def test_service_status_uses_internal_auth_and_only_advertises_the_public_
|
|||
result: Final = await service_connection(UserAPIKeyAuth(user_role=LitellmUserRoles.INTERNAL_USER))
|
||||
assert result.url == "https://traces.example/lens-ingest"
|
||||
assert result.connected is connected
|
||||
assert result.configured is True
|
||||
assert result.status.storage_ready is connected
|
||||
assert route.calls[0].request.headers["Authorization"] == "Bearer " + "x" * 32
|
||||
assert "private storage details" not in result.model_dump_json()
|
||||
|
|
@ -1057,6 +1058,23 @@ async def test_service_status_uses_internal_auth_and_only_advertises_the_public_
|
|||
assert denied.value.status_code == 403
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("url,configured", (("", False), ("http://lens", True)))
|
||||
async def test_service_setup_distinguishes_missing_installation_from_incomplete_configuration(
|
||||
monkeypatch: pytest.MonkeyPatch, url: str, configured: bool
|
||||
) -> None:
|
||||
from litellm.proxy.lens.endpoints import service_connection
|
||||
|
||||
monkeypatch.setenv("LITELLM_LENS_URL", url)
|
||||
monkeypatch.delenv("LITELLM_LENS_SERVICE_TOKEN", raising=False)
|
||||
monkeypatch.setenv("LITELLM_RELEASE_TAG", "v1.2.3")
|
||||
result: Final = await service_connection(UserAPIKeyAuth(user_role=LitellmUserRoles.INTERNAL_USER))
|
||||
assert result.configured is configured
|
||||
assert result.connected is False
|
||||
assert result.release == "v1.2.3"
|
||||
assert result.status.storage_ready is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_credential_snapshot_excludes_expired_keys_and_disables_caching(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
from unittest.mock import AsyncMock
|
||||
|
|
|
|||
|
|
@ -1116,11 +1116,6 @@
|
|||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/usage/_components/components/EntityUsage/EntityUsage.tsx": {
|
||||
"local/no-complex-jsx-arrow": {
|
||||
"count": 2
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/usage/_components/components/UsageAIChatPanel.tsx": {
|
||||
"no-nested-ternary": {
|
||||
"count": 1
|
||||
|
|
@ -1130,17 +1125,11 @@
|
|||
}
|
||||
},
|
||||
"src/app/(dashboard)/usage/_components/components/UsagePageView.tsx": {
|
||||
"local/no-complex-jsx-arrow": {
|
||||
"count": 2
|
||||
},
|
||||
"max-lines": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/purity": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 2
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/users/_components/BulkEditUsers.tsx": {
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import { act, fireEvent, screen, within, waitFor } from "@testing-library/react";
|
||||
import userEvent from "@testing-library/user-event";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { renderWithProviders, testQueryClient } from "@/../tests/test-utils";
|
||||
import { chooseSelectOption, renderWithProviders, testQueryClient } from "@/../tests/test-utils";
|
||||
import { readRequest, requestPath } from "@/../tests/lens-test-utils";
|
||||
import { LensWorkspace } from "./LensWorkspace";
|
||||
import { createLensDemoData } from "./data/demo/fixtures";
|
||||
|
|
@ -19,16 +19,19 @@ const worker = () => ({
|
|||
scope: data.lenses[0].scope,
|
||||
});
|
||||
|
||||
function serve({ enabled = false, traces = false, requests = false, connected = false } = {}) {
|
||||
function serve({ enabled = false, traces = false, requests = false, connected = false, storageReady = true } = {}) {
|
||||
list.mockResolvedValue({ lenses: [], workers: connected ? [worker()] : [], tracing_enabled: enabled });
|
||||
network.mockImplementation(async (input, init) => {
|
||||
const { path, method, body, query } = await readRequest(input, init);
|
||||
if (path === "/lens/service")
|
||||
return Response.json({
|
||||
if (path === "/lens/service") {
|
||||
const service = {
|
||||
url: "https://traces.test",
|
||||
connected: true,
|
||||
status: { storage_ready: true, credentials_ready: true },
|
||||
});
|
||||
configured: enabled,
|
||||
connected: enabled,
|
||||
status: { storage_ready: storageReady, credentials_ready: true },
|
||||
};
|
||||
return Response.json(service);
|
||||
}
|
||||
if (path === "/v1/traces")
|
||||
return enabled
|
||||
? Response.json({ data: traces ? [data.runs[0].trace.summary] : [] })
|
||||
|
|
@ -143,6 +146,19 @@ describe("Lens introduction", () => {
|
|||
});
|
||||
|
||||
describe("Lens setup journey", () => {
|
||||
it("waits for storage readiness before completing installation", async () => {
|
||||
serve({ enabled: true, storageReady: false });
|
||||
const user = userEvent.setup();
|
||||
renderWorkspace();
|
||||
const installation = await screen.findByRole("region", { name: /Install Lens/ });
|
||||
expect(await within(installation).findByText(/trace storage is unavailable/)).toBeVisible();
|
||||
expect(screen.queryByText("Trace storage is connected")).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("combobox", { name: "Your agent framework" })).not.toBeInTheDocument();
|
||||
serve({ enabled: true });
|
||||
await user.click(within(installation).getByRole("button", { name: "Check setup" }));
|
||||
expect(await screen.findByRole("combobox", { name: "Your agent framework" })).toBeVisible();
|
||||
});
|
||||
|
||||
it.each(["/lens", "/lens/activity/available"])(
|
||||
"keeps recorded traces visible while %s is pending",
|
||||
async (pendingPath) => {
|
||||
|
|
@ -192,6 +208,41 @@ describe("Lens setup journey", () => {
|
|||
await connectWorkerFromSettings(user);
|
||||
});
|
||||
|
||||
it("continues to agent setup when a background service check detects the installation", async () => {
|
||||
const user = userEvent.setup();
|
||||
renderWorkspace({ searchParams: "?setup=lens" });
|
||||
const intro = within(await screen.findByRole("region", { name: "Get started with Lens" }));
|
||||
expect(await intro.findByRole("button", { name: "Check setup" })).toBeVisible();
|
||||
serve({ enabled: true });
|
||||
await act(() => testQueryClient.refetchQueries({ queryKey: ["lens-service"] }));
|
||||
await user.click(await intro.findByRole("button", { name: "Continue to your agent" }));
|
||||
expect(await intro.findByRole("combobox", { name: "Your agent framework" })).toBeVisible();
|
||||
expect(intro.getByRole("button", { name: "Generate tracing key" })).toBeEnabled();
|
||||
expect(intro.getByRole("button", { name: "Copy tracing configuration" })).toBeVisible();
|
||||
await chooseSelectOption(user, intro.getByRole("combobox", { name: "Your agent framework" }), "LangGraph");
|
||||
const normal = network.getMockImplementation()!;
|
||||
network.mockImplementation((input, init) =>
|
||||
requestPath(input) === "/lens/tracing/keys"
|
||||
? Promise.resolve(Response.json({ key: "sk-tracing-setup", active: true }))
|
||||
: normal(input, init),
|
||||
);
|
||||
await user.click(intro.getByRole("button", { name: "Generate tracing key" }));
|
||||
expect(await intro.findByText("Your tracing key")).toBeVisible();
|
||||
serve({ enabled: true, storageReady: false });
|
||||
await act(() => testQueryClient.refetchQueries({ queryKey: ["lens-service"] }));
|
||||
const installation = within(await intro.findByRole("region", { name: /Install Lens/ }));
|
||||
expect(await installation.findByText(/trace storage is unavailable/)).toBeVisible();
|
||||
serve({ enabled: true });
|
||||
await act(() => testQueryClient.refetchQueries({ queryKey: ["lens-service"] }));
|
||||
const agent = within(await intro.findByRole("region", { name: /Send your first trace/ }));
|
||||
expect(await agent.findByRole("combobox", { name: "Your agent framework" })).toHaveTextContent("LangGraph");
|
||||
expect(agent.getByText("Your tracing key")).toBeVisible();
|
||||
expect(agent.queryByRole("button", { name: "Generate tracing key" })).not.toBeInTheDocument();
|
||||
serve({ enabled: true, traces: true });
|
||||
await user.click(intro.getByRole("button", { name: "Check for traces" }));
|
||||
expect(await intro.findByRole("button", { name: "Continue to worker" })).toBeEnabled();
|
||||
});
|
||||
|
||||
it("resumes setup from the URL and leaves only when the user chooses traces", async () => {
|
||||
serve({ enabled: true, traces: true });
|
||||
const user = userEvent.setup();
|
||||
|
|
@ -222,7 +273,7 @@ describe("Lens setup journey", () => {
|
|||
const intro = within(await screen.findByRole("region", { name: "Get started with Lens" }));
|
||||
expect(await intro.findByRole("heading", { name: "Before you start" })).toBeVisible();
|
||||
expect(intro.getByRole("button", { name: "Connect worker" })).toBeEnabled();
|
||||
await user.click(intro.getByRole("button", { name: /Enable tracing on the gateway/ }));
|
||||
await user.click(intro.getByRole("button", { name: /Install Lens/ }));
|
||||
expect(intro.getByRole("button", { name: "Continue with request logs" })).toBeEnabled();
|
||||
await user.click(intro.getByRole("button", { name: /Send your first trace/ }));
|
||||
await user.click(intro.getByRole("button", { name: "Continue with request logs" }));
|
||||
|
|
|
|||
|
|
@ -445,7 +445,7 @@ it("guides a first-time administrator into worker connection and lens setup", as
|
|||
expect.objectContaining({ authorization: "Bearer test" }),
|
||||
);
|
||||
expect(guide.queryByRole("button", { name: /Send your first trace/ })).not.toBeInTheDocument();
|
||||
expect(guide.queryByRole("button", { name: /Enable tracing on the gateway/ })).not.toBeInTheDocument();
|
||||
expect(guide.queryByRole("button", { name: /Install Lens/ })).not.toBeInTheDocument();
|
||||
expect(guide.getByRole("button", { name: /Connect a worker/ })).toHaveAttribute("aria-expanded", "true");
|
||||
expect(guide.queryByRole("button", { name: "View traces" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
|
||||
|
|
@ -602,7 +602,9 @@ it.each([false, true])(
|
|||
const user = userEvent.setup();
|
||||
renderWithProviders(<InvestigationsView />);
|
||||
const guide = within(await screen.findByRole("region", { name: "Get Lens running" }));
|
||||
expect(guide.getByRole("button", { name: /Send your first trace/ })).toHaveAttribute("aria-expanded", "true");
|
||||
await waitFor(() =>
|
||||
expect(guide.getByRole("button", { name: /Send your first trace/ })).toHaveAttribute("aria-expanded", "true"),
|
||||
);
|
||||
expect(await guide.findByRole("button", { name: "Check for traces" })).toBeVisible();
|
||||
await user.click(guide.getByRole("button", { name: /Connect a worker/ }));
|
||||
expect(guide.getByRole("button", { name: "Connect worker" })).toBeDisabled();
|
||||
|
|
|
|||
|
|
@ -9,9 +9,9 @@ import { OnboardingSetup } from "./OnboardingSetup";
|
|||
|
||||
const PREREQUISITES = [
|
||||
{ title: "LiteLLM gateway", detail: "Access to its configuration" },
|
||||
{ title: "ClickHouse", detail: "Self-hosted or managed trace storage" },
|
||||
{ title: "A server with Docker", detail: "To run the analysis worker" },
|
||||
{ title: "An analysis model", detail: "Available through your gateway" },
|
||||
{ title: "Trace storage", detail: "Included, or use your own ClickHouse" },
|
||||
{ title: "Docker or Kubernetes", detail: "Use your existing deployment" },
|
||||
{ title: "An analysis model", detail: "For investigations, after tracing is connected" },
|
||||
] as const;
|
||||
|
||||
export interface LensGettingStartedProps {
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
import { useId, useRef, useState, type ReactNode } from "react";
|
||||
import { ArrowRight, ChevronDown } from "lucide-react";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { TracingSetupFields } from "@/components/lens/onboarding/tracing/TracingSetupCard";
|
||||
import { TracingSetupFields, useLensService } from "@/components/lens/onboarding/tracing/TracingSetupCard";
|
||||
import { cn } from "@/lib/cva.config";
|
||||
import { useLensAccessToken } from "../data/LensServices";
|
||||
import type { LensReadiness } from "../hooks/useLensReadiness";
|
||||
|
|
@ -11,17 +11,17 @@ import { initialSetupStep } from "../model/readiness";
|
|||
import { StepIndicator, type StepState } from "../ui/StepIndicator";
|
||||
import { useOnboarding } from "./OnboardingContext";
|
||||
|
||||
type StepProps = { state: LensReadiness; goTo: (step: number) => void };
|
||||
type StepProps = { state: LensReadiness; goTo: (step: number) => void; storageReady: boolean };
|
||||
|
||||
function useLocked() {
|
||||
const { readOnly, canInvestigate } = useOnboarding();
|
||||
return readOnly || !canInvestigate;
|
||||
}
|
||||
|
||||
function StorageStep({ state, goTo }: StepProps) {
|
||||
function StorageStep({ state, goTo, storageReady }: StepProps) {
|
||||
const { readOnly, openTrace } = useOnboarding();
|
||||
const accessToken = useLensAccessToken();
|
||||
if (!state.tracingEnabled)
|
||||
if (!storageReady)
|
||||
return (
|
||||
<>
|
||||
<TracingSetupFields
|
||||
|
|
@ -75,13 +75,6 @@ function ActivityContinuation({ state }: { state: LensReadiness }) {
|
|||
function AgentStep({ state }: StepProps) {
|
||||
const { readOnly, canMintTracingKey, openTrace } = useOnboarding();
|
||||
const accessToken = useLensAccessToken();
|
||||
if (!state.tracingEnabled)
|
||||
return (
|
||||
<>
|
||||
<p className="text-sm text-muted-foreground">Connect trace storage in step 1 before sending a trace.</p>
|
||||
<ActivityContinuation state={state} />
|
||||
</>
|
||||
);
|
||||
return (
|
||||
<>
|
||||
<div hidden={state.tracesReady}>
|
||||
|
|
@ -153,15 +146,15 @@ function InvestigationStep({ state }: StepProps) {
|
|||
interface StepDefinition {
|
||||
readonly title: string;
|
||||
readonly description: string;
|
||||
readonly complete: (state: LensReadiness) => boolean;
|
||||
readonly complete: (state: LensReadiness, storageReady: boolean) => boolean;
|
||||
readonly Content: (props: StepProps) => ReactNode;
|
||||
}
|
||||
|
||||
const STEPS: readonly StepDefinition[] = [
|
||||
{
|
||||
title: "Enable tracing on the gateway",
|
||||
description: "Connect ClickHouse and restart the gateway.",
|
||||
complete: (state) => state.tracingEnabled,
|
||||
title: "Install Lens",
|
||||
description: "Enable Lens in your Helm or Docker deployment.",
|
||||
complete: (_state, storageReady) => storageReady,
|
||||
Content: StorageStep,
|
||||
},
|
||||
{
|
||||
|
|
@ -172,7 +165,7 @@ const STEPS: readonly StepDefinition[] = [
|
|||
},
|
||||
{
|
||||
title: "Connect a worker",
|
||||
description: "Choose a model and run the worker on your infrastructure.",
|
||||
description: "Choose an analysis model and spending limit.",
|
||||
complete: (state) => state.connected,
|
||||
Content: WorkerStep,
|
||||
},
|
||||
|
|
@ -201,7 +194,11 @@ export function OnboardingSteps({
|
|||
const id = useId();
|
||||
const listRef = useRef<HTMLOListElement>(null);
|
||||
const offset = includeTracing ? 0 : 2;
|
||||
const accessToken = useLensAccessToken();
|
||||
const service = useLensService(accessToken);
|
||||
const storageReady = Boolean(service.data?.connected && service.data.status.storage_ready && service.data.url);
|
||||
const [step, setStep] = useState(() => Math.max(offset, initialSetupStep(state)));
|
||||
const visibleStep = !storageReady && step === 1 ? 0 : step;
|
||||
const goTo = (index: number) => {
|
||||
setStep(index);
|
||||
listRef.current?.querySelector<HTMLButtonElement>(`[aria-controls="${id}-${index}"]`)?.focus();
|
||||
|
|
@ -214,7 +211,7 @@ export function OnboardingSteps({
|
|||
>
|
||||
{STEPS.slice(offset).map(({ title, description, complete, Content }, index) => {
|
||||
const stepIndex = index + offset;
|
||||
const open = Math.max(offset, step) === stepIndex;
|
||||
const open = Math.max(offset, visibleStep) === stepIndex;
|
||||
return (
|
||||
<li key={title}>
|
||||
<h3>
|
||||
|
|
@ -226,7 +223,7 @@ export function OnboardingSteps({
|
|||
onClick={() => setStep(stepIndex)}
|
||||
className="group flex w-full items-start gap-4 p-5 text-left outline-none hover:bg-muted/30 focus-visible:bg-muted/50 sm:p-6"
|
||||
>
|
||||
<StepIndicator index={index} state={stepState(complete(state), open)} />
|
||||
<StepIndicator index={index} state={stepState(complete(state, storageReady), open)} />
|
||||
<span className="min-w-0 flex-1">
|
||||
<span
|
||||
className={cn(
|
||||
|
|
@ -251,7 +248,7 @@ export function OnboardingSteps({
|
|||
hidden={!open}
|
||||
className="px-5 pb-6 sm:pr-6 sm:pb-7 sm:pl-17"
|
||||
>
|
||||
<Content state={state} goTo={goTo} />
|
||||
<Content state={state} goTo={goTo} storageReady={storageReady} />
|
||||
</div>
|
||||
</li>
|
||||
);
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import { screen } from "@testing-library/react";
|
||||
import { act, screen } from "@testing-library/react";
|
||||
import userEvent from "@testing-library/user-event";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { chooseSelectOption, renderWithProviders } from "@/../tests/test-utils";
|
||||
import { chooseSelectOption, renderWithProviders, testQueryClient } from "@/../tests/test-utils";
|
||||
import { copyToClipboard } from "@/utils/dataUtils";
|
||||
import { agentTraceCall, apiClient } from "../../../networking";
|
||||
import {
|
||||
|
|
@ -51,14 +51,18 @@ const renderCard = async (
|
|||
|
||||
const network = vi.fn<typeof fetch>();
|
||||
beforeEach(() => {
|
||||
testQueryClient.clear();
|
||||
vi.clearAllMocks();
|
||||
vi.stubGlobal("fetch", network);
|
||||
network.mockResolvedValue(Response.json({}));
|
||||
vi.mocked(apiClient.get).mockResolvedValue({
|
||||
const readyService = {
|
||||
url: "https://traces.test",
|
||||
configured: true,
|
||||
release: "v1.2.3",
|
||||
connected: true,
|
||||
status: { storage_ready: true, credentials_ready: true },
|
||||
});
|
||||
};
|
||||
vi.mocked(apiClient.get).mockResolvedValue(readyService);
|
||||
vi.mocked(apiClient.post).mockResolvedValue({ key: SECRET, active: true });
|
||||
});
|
||||
|
||||
|
|
@ -72,7 +76,6 @@ describe("TracingSetupCard", () => {
|
|||
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
|
||||
expect(network).not.toHaveBeenCalled();
|
||||
expect(card).not.toHaveTextContent("store: clickhouse");
|
||||
await user.click(screen.getByText("Set up manually"));
|
||||
expect(screen.getByText(/^export LITELLM_TRACING_KEY=/)).toBeVisible();
|
||||
expect(card).not.toHaveTextContent(/langsmith/i);
|
||||
});
|
||||
|
|
@ -114,7 +117,6 @@ describe("TracingSetupCard", () => {
|
|||
normalizeWhitespace: false,
|
||||
});
|
||||
|
||||
await user.click(screen.getByText("Set up manually"));
|
||||
expect(screen.getByText(/^pip install opentelemetry-distro/)).toHaveTextContent(
|
||||
"crewai openinference-instrumentation-crewai",
|
||||
);
|
||||
|
|
@ -128,7 +130,6 @@ describe("TracingSetupCard", () => {
|
|||
const user = userEvent.setup();
|
||||
const { card } = await renderCard();
|
||||
await chooseSelectOption(user, screen.getByRole("combobox", { name: "Your agent framework" }), "Vercel AI SDK");
|
||||
await user.click(screen.getByText("Set up manually"));
|
||||
expect(screen.queryByRole("combobox", { name: "Model" })).not.toBeInTheDocument();
|
||||
expect(card).toHaveTextContent("npm install ai @ai-sdk/otel");
|
||||
expect(card).toHaveTextContent('const AGENT_NAME = "research_agent"');
|
||||
|
|
@ -145,7 +146,6 @@ describe("TracingSetupCard", () => {
|
|||
vi.mocked(apiClient.post).mockResolvedValue({ key: SECRET });
|
||||
const { card } = await renderCard();
|
||||
await chooseSelectOption(user, screen.getByRole("combobox", { name: "Your agent framework" }), "Hermes");
|
||||
await user.click(screen.getByText("Set up manually"));
|
||||
expect(screen.queryByRole("combobox", { name: "Model" })).not.toBeInTheDocument();
|
||||
expect(card).toHaveTextContent("Keep your existing model settings");
|
||||
await user.click(screen.getByRole("button", { name: "Generate tracing key" }));
|
||||
|
|
@ -175,7 +175,6 @@ describe("TracingSetupCard", () => {
|
|||
const user = userEvent.setup();
|
||||
vi.mocked(apiClient.post).mockResolvedValue({ key: SECRET });
|
||||
const { card } = await renderCard();
|
||||
await user.click(screen.getByText("Set up manually"));
|
||||
await user.click(screen.getByRole("button", { name: "Generate tracing key" }));
|
||||
|
||||
expect(await screen.findByText("Your tracing key")).toBeVisible();
|
||||
|
|
@ -226,18 +225,92 @@ describe("TracingSetupCard", () => {
|
|||
expect(screen.queryByRole("button", { name: /View trace/ })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("guides proxy setup before agent setup and allows checking readiness", async () => {
|
||||
it("shows matching-version installation instructions without changing the landing design", async () => {
|
||||
const user = userEvent.setup();
|
||||
const onCheck = vi.fn();
|
||||
const missingService = {
|
||||
configured: false,
|
||||
connected: false,
|
||||
release: "v1.2.3",
|
||||
url: "",
|
||||
status: {},
|
||||
};
|
||||
vi.mocked(apiClient.get).mockResolvedValue(missingService);
|
||||
const { card } = await renderCard({ detail: "Agent tracing is not enabled", onCheck });
|
||||
expect(screen.getByRole("heading", { name: "Enable tracing" })).toBeVisible();
|
||||
expect(card).toHaveTextContent("LITELLM_LENS_URL");
|
||||
expect(card).toHaveTextContent("LITELLM_LENS_SERVICE_TOKEN");
|
||||
expect(await screen.findByText("Install Lens")).toBeVisible();
|
||||
expect(card).toHaveTextContent("Use Lens v1.2.3 to match this LiteLLM deployment");
|
||||
expect(screen.getByRole("link", { name: "Helm setup" })).toHaveAttribute(
|
||||
"href",
|
||||
"https://docs.litellm.ai/docs/proxy/lens/deployment#using-helm",
|
||||
);
|
||||
expect(screen.getByRole("link", { name: "Docker setup" })).toHaveAttribute(
|
||||
"href",
|
||||
"https://docs.litellm.ai/docs/proxy/lens/deployment#using-docker",
|
||||
);
|
||||
expect(screen.queryByRole("combobox", { name: "Your agent framework" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Send a test trace" })).not.toBeInTheDocument();
|
||||
const readyService = {
|
||||
configured: true,
|
||||
connected: true,
|
||||
url: "https://traces.test",
|
||||
status: { storage_ready: true },
|
||||
};
|
||||
vi.mocked(apiClient.get).mockResolvedValue(readyService);
|
||||
await user.click(screen.getByRole("button", { name: "Check setup" }));
|
||||
expect(onCheck).toHaveBeenCalledOnce();
|
||||
expect(screen.getByText(/Tracing is still unavailable/)).toBeVisible();
|
||||
expect(await screen.findByRole("combobox", { name: "Your agent framework" })).toBeVisible();
|
||||
});
|
||||
|
||||
it.each([
|
||||
[false, false, "Lens is configured, but LiteLLM cannot reach it"],
|
||||
[true, false, "Lens is connected, but its trace storage is unavailable"],
|
||||
])(
|
||||
"explains a configured service failure without recommending reinstallation",
|
||||
async (connected, storageReady, message) => {
|
||||
const service = {
|
||||
configured: true,
|
||||
connected,
|
||||
url: "https://traces.test",
|
||||
status: { storage_ready: storageReady },
|
||||
};
|
||||
vi.mocked(apiClient.get).mockResolvedValue(service);
|
||||
await renderCard({ detail: "Service unavailable" });
|
||||
expect(await screen.findByText(message, { exact: false })).toBeVisible();
|
||||
expect(screen.queryByText("Install Lens")).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Generate tracing key" })).not.toBeInTheDocument();
|
||||
},
|
||||
);
|
||||
|
||||
it("preserves the framework and uncopied key across a background service outage", async () => {
|
||||
const user = userEvent.setup();
|
||||
await renderCard();
|
||||
await chooseSelectOption(user, screen.getByRole("combobox", { name: "Your agent framework" }), "LangGraph");
|
||||
await user.click(screen.getByRole("button", { name: "Generate tracing key" }));
|
||||
await screen.findByText("Your tracing key");
|
||||
const ready = testQueryClient.getQueryData(["lens-service", "sk-admin"]);
|
||||
const unavailable = {
|
||||
configured: true,
|
||||
connected: false,
|
||||
url: "https://traces.test",
|
||||
status: { storage_ready: false },
|
||||
};
|
||||
act(() => testQueryClient.setQueryData(["lens-service", "sk-admin"], unavailable));
|
||||
expect(await screen.findByText(/Lens is configured, but LiteLLM cannot reach it/)).toBeVisible();
|
||||
act(() => testQueryClient.setQueryData(["lens-service", "sk-admin"], ready));
|
||||
expect(await screen.findByRole("combobox", { name: "Your agent framework" })).toHaveTextContent("LangGraph");
|
||||
expect(screen.getByText("Your tracing key")).toBeVisible();
|
||||
await user.click(screen.getByRole("button", { name: "Copy tracing configuration" }));
|
||||
expect(copyToClipboard).toHaveBeenLastCalledWith(tracingEnvSnippet("https://traces.test", SECRET));
|
||||
expect(apiClient.post).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("shows the copyable configuration without another disclosure and includes the generated key", async () => {
|
||||
const user = userEvent.setup();
|
||||
await renderCard();
|
||||
expect(screen.getByText(/^export LITELLM_TRACING_KEY=/)).toBeVisible();
|
||||
await user.click(screen.getByRole("button", { name: "Generate tracing key" }));
|
||||
await screen.findByText("Your tracing key");
|
||||
await user.click(screen.getByRole("button", { name: "Copy tracing configuration" }));
|
||||
expect(copyToClipboard).toHaveBeenLastCalledWith(tracingEnvSnippet("https://traces.test", SECRET));
|
||||
});
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ import type { TraceSummary } from "../../traces/types";
|
|||
|
||||
const COPIED_RESET_MS = 1500;
|
||||
const DOCS_URL = "https://docs.litellm.ai/docs/proxy/lens";
|
||||
const DEPLOYMENT_URL = `${DOCS_URL}/deployment`;
|
||||
const EXAMPLE_MODEL = "openai/gpt-6.1-sol";
|
||||
const SAMPLE_TRACE_POLL_MS = 1000;
|
||||
export const TRACING_KEY_REQUEST = { name: "Agent tracing" } as const;
|
||||
|
|
@ -77,22 +78,18 @@ export const otlpEndpoints = (proxyUrl: string): readonly (readonly [string, str
|
|||
["Protocol", "OTLP/HTTP (protobuf or JSON)", false],
|
||||
];
|
||||
|
||||
export const PROXY_CONFIG_SNIPPET = [
|
||||
'export LITELLM_LENS_URL="http://lens-worker:4318"',
|
||||
'export LITELLM_LENS_PUBLIC_URL="https://traces.example.com"',
|
||||
'export LITELLM_LENS_SERVICE_TOKEN="<shared service secret>"',
|
||||
].join("\n");
|
||||
|
||||
function CodeBlock({
|
||||
code,
|
||||
display = code,
|
||||
tabs,
|
||||
wrap = false,
|
||||
copyLabel = "Copy",
|
||||
}: {
|
||||
code: string;
|
||||
display?: string;
|
||||
tabs?: React.ReactNode;
|
||||
wrap?: boolean;
|
||||
copyLabel?: string;
|
||||
}) {
|
||||
const [copied, setCopied] = useState(false);
|
||||
useTimeout(() => setCopied(false), copied ? COPIED_RESET_MS : null);
|
||||
|
|
@ -104,7 +101,7 @@ function CodeBlock({
|
|||
<button
|
||||
type="button"
|
||||
onClick={() => void copy()}
|
||||
aria-label="Copy"
|
||||
aria-label={copyLabel}
|
||||
className="ml-auto text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
{copied ? <Check className="size-3.5" /> : <Copy className="size-3.5" />}
|
||||
|
|
@ -412,7 +409,7 @@ const CODING_AGENT_LOGOS: Record<CodingAgent, string> = {
|
|||
};
|
||||
|
||||
function setupTitle(enabled: boolean, connected: boolean) {
|
||||
if (!enabled) return "Enable tracing";
|
||||
if (!enabled) return "Set up Lens";
|
||||
return connected ? "Connect another agent" : "Connect your agent";
|
||||
}
|
||||
|
||||
|
|
@ -425,38 +422,81 @@ interface ConnectAgentProps {
|
|||
onCheck: () => void;
|
||||
readOnly: boolean;
|
||||
canMintTracingKey: boolean;
|
||||
framework: string;
|
||||
setFramework: (framework: string) => void;
|
||||
installer: Installer;
|
||||
setInstaller: (installer: Installer) => void;
|
||||
tracingKey: string | null;
|
||||
setTracingKey: (key: string) => void;
|
||||
}
|
||||
|
||||
function EnableTracing({ checked, checking, onCheck }: { checked: boolean; checking: boolean; onCheck: () => void }) {
|
||||
export function useLensService(accessToken: string) {
|
||||
return useQuery({
|
||||
queryKey: ["lens-service", accessToken],
|
||||
queryFn: () => apiClient.get<components["schemas"]["ServiceConnection"]>("/lens/service", { accessToken }),
|
||||
refetchInterval: 15000,
|
||||
});
|
||||
}
|
||||
|
||||
function EnableTracing({
|
||||
connection,
|
||||
checking,
|
||||
onCheck,
|
||||
}: {
|
||||
connection: components["schemas"]["ServiceConnection"];
|
||||
checking: boolean;
|
||||
onCheck: () => void;
|
||||
}) {
|
||||
const configured = connection.configured ?? connection.connected;
|
||||
let message = "Lens is connected. Set its public tracing address so your agents know where to send traces.";
|
||||
if (!configured) {
|
||||
message =
|
||||
"Enable Lens in your existing Helm or Docker deployment. It runs alongside LiteLLM and stores your traces.";
|
||||
} else if (!connection.connected) {
|
||||
message =
|
||||
"Lens is configured, but LiteLLM cannot reach it. Check that the Lens service is running and both services use the same service secret.";
|
||||
} else if (!connection.status.storage_ready) {
|
||||
message = "Lens is connected, but its trace storage is unavailable. Check the ClickHouse connection.";
|
||||
}
|
||||
return (
|
||||
<>
|
||||
<Step title="Proxy configuration">
|
||||
<p className="mb-3 text-sm leading-6 text-muted-foreground">
|
||||
Run the Lens service with ClickHouse access, then set these variables on LiteLLM and restart it. Use the same
|
||||
service secret on both services.
|
||||
<div className="space-y-4">
|
||||
<p className="text-sm font-medium">{configured ? "Check the Lens connection" : "Install Lens"}</p>
|
||||
<p className="text-sm leading-6 text-muted-foreground">{message}</p>
|
||||
{!configured && (
|
||||
<p className="text-sm text-muted-foreground">
|
||||
{connection.release ? (
|
||||
<>
|
||||
Use Lens <code>{connection.release}</code> to match this LiteLLM deployment.
|
||||
</>
|
||||
) : (
|
||||
"Use Lens from the same release as this LiteLLM deployment."
|
||||
)}{" "}
|
||||
The deployment connects the services and supplies trace storage.
|
||||
</p>
|
||||
<CodeBlock code={PROXY_CONFIG_SNIPPET} tabs={<FileLabel>LiteLLM environment</FileLabel>} />
|
||||
)}
|
||||
<div className="flex flex-wrap gap-3">
|
||||
<a
|
||||
className="mt-3 inline-flex items-center gap-1 text-sm underline underline-offset-4"
|
||||
href={`${DOCS_URL}#configure-an-existing-proxy`}
|
||||
className="inline-flex items-center gap-1 text-sm underline underline-offset-4"
|
||||
href={`${DEPLOYMENT_URL}#using-helm`}
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
>
|
||||
Lens service setup <ArrowUpRight aria-hidden="true" className="size-3.5" />
|
||||
Helm setup <ArrowUpRight aria-hidden="true" className="size-3.5" />
|
||||
</a>
|
||||
<a
|
||||
className="inline-flex items-center gap-1 text-sm underline underline-offset-4"
|
||||
href={`${DEPLOYMENT_URL}#using-docker`}
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
>
|
||||
Docker setup <ArrowUpRight aria-hidden="true" className="size-3.5" />
|
||||
</a>
|
||||
</Step>
|
||||
{checked && !checking && (
|
||||
<p className="mt-4 text-sm text-muted-foreground">
|
||||
Tracing is still unavailable. Check that the configuration was applied to this proxy and it has restarted.
|
||||
</p>
|
||||
)}
|
||||
<div className="mt-6 border-t pt-6">
|
||||
<Button onClick={onCheck} disabled={checking}>
|
||||
{checking && <Loader2 aria-hidden="true" className="size-4 animate-spin" />}
|
||||
{checking ? "Checking…" : "Check setup"}
|
||||
</Button>
|
||||
</div>
|
||||
</>
|
||||
<Button variant="outline" onClick={onCheck} disabled={checking}>
|
||||
{checking && <Loader2 aria-hidden="true" className="size-4 animate-spin" />}
|
||||
{checking ? "Checking…" : "Check setup"}
|
||||
</Button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -498,7 +538,7 @@ function CodingAgentSetup({
|
|||
<div className="p-4">
|
||||
<p className="text-sm leading-6 text-muted-foreground">
|
||||
Run the setup command in your agent’s project. It uses <code className="text-xs">LITELLM_TRACING_KEY</code>{" "}
|
||||
for traces and keeps your model key separate .
|
||||
for traces and keeps your model key separate.
|
||||
</p>
|
||||
<Button className="mt-3" onClick={() => void copy()}>
|
||||
{copied === command ? (
|
||||
|
|
@ -529,17 +569,16 @@ function ConnectAgent({
|
|||
onCheck,
|
||||
readOnly,
|
||||
canMintTracingKey,
|
||||
framework,
|
||||
setFramework,
|
||||
installer,
|
||||
setInstaller,
|
||||
tracingKey,
|
||||
setTracingKey,
|
||||
}: ConnectAgentProps) {
|
||||
const proxyUrl = getProxyBaseUrl().replace(/\/$/, "");
|
||||
const connection = useQuery({
|
||||
queryKey: ["lens-service", accessToken],
|
||||
queryFn: () => apiClient.get<components["schemas"]["ServiceConnection"]>("/lens/service", { accessToken }),
|
||||
refetchInterval: 15000,
|
||||
});
|
||||
const connection = useLensService(accessToken);
|
||||
const traceUrl = connection.data?.url ?? "";
|
||||
const [framework, setFramework] = useState(FRAMEWORKS[0].id);
|
||||
const [installer, setInstaller] = useState<Installer>("pip");
|
||||
const [tracingKey, setTracingKey] = useState<string | null>(null);
|
||||
const guide = FRAMEWORKS.find((f) => f.id === framework) ?? FRAMEWORKS[0];
|
||||
const install = guide.install?.startsWith("pip install ")
|
||||
? PY_INSTALL[installer](guide.install.slice("pip install ".length))
|
||||
|
|
@ -566,16 +605,6 @@ function ConnectAgent({
|
|||
Lens is connected, but ClickHouse is unavailable.
|
||||
</p>
|
||||
)}
|
||||
<Endpoints proxyUrl={traceUrl}>
|
||||
{!readOnly && (
|
||||
<SendTestTrace
|
||||
accessToken={accessToken}
|
||||
traceUrl={traceUrl}
|
||||
tracingKey={tracingKey}
|
||||
onOpenTrace={onOpenTrace}
|
||||
/>
|
||||
)}
|
||||
</Endpoints>
|
||||
<div className="mt-6 space-y-2">
|
||||
<label id="tracing-framework" className="block text-sm font-medium">
|
||||
Your agent framework
|
||||
|
|
@ -610,9 +639,7 @@ function ConnectAgent({
|
|||
<p className="text-sm text-muted-foreground">Ask your proxy admin for a dedicated Lens tracing key.</p>
|
||||
)}
|
||||
</Step>
|
||||
<CodingAgentSetup proxyUrl={proxyUrl} traceUrl={traceUrl} guide={guide} model={model} />
|
||||
<details className="mt-6 border-t pt-6">
|
||||
<summary className="w-fit cursor-pointer text-sm font-medium">Set up manually</summary>
|
||||
<div className="mt-6 border-t pt-6">
|
||||
{install && (
|
||||
<Step title="Install dependencies">
|
||||
<CodeBlock
|
||||
|
|
@ -653,6 +680,7 @@ function ConnectAgent({
|
|||
)}
|
||||
</p>
|
||||
<CodeBlock
|
||||
copyLabel="Copy tracing configuration"
|
||||
code={tracingEnvSnippet(traceUrl, tracingKey)}
|
||||
display={tracingEnvSnippet(traceUrl, tracingKey && maskSecret(tracingKey))}
|
||||
tabs={<FileLabel>Shell</FileLabel>}
|
||||
|
|
@ -676,7 +704,7 @@ function ConnectAgent({
|
|||
</div>
|
||||
)}
|
||||
<a
|
||||
href={`${DOCS_URL}?framework=${guide.id}#send-your-first-trace`}
|
||||
href={`${DOCS_URL}/first-trace?framework=${guide.id}`}
|
||||
className="mt-3 inline-flex items-center gap-1 text-sm underline underline-offset-4"
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
|
|
@ -684,8 +712,18 @@ function ConnectAgent({
|
|||
View in docs <ArrowUpRight aria-hidden="true" className="size-3.5" />
|
||||
</a>
|
||||
</Step>
|
||||
</details>
|
||||
|
||||
</div>
|
||||
<CodingAgentSetup proxyUrl={proxyUrl} traceUrl={traceUrl} guide={guide} model={model} />
|
||||
<Endpoints proxyUrl={traceUrl}>
|
||||
{!readOnly && (
|
||||
<SendTestTrace
|
||||
accessToken={accessToken}
|
||||
traceUrl={traceUrl}
|
||||
tracingKey={tracingKey}
|
||||
onOpenTrace={onOpenTrace}
|
||||
/>
|
||||
)}
|
||||
</Endpoints>
|
||||
<TraceReceipt connected={connected} checked={checked} checking={checking} onCheck={onCheck} />
|
||||
</>
|
||||
);
|
||||
|
|
@ -703,7 +741,6 @@ type TracingSetupProps = {
|
|||
};
|
||||
|
||||
export function TracingSetupFields({
|
||||
detail,
|
||||
accessToken,
|
||||
onOpenTrace,
|
||||
connected = false,
|
||||
|
|
@ -712,29 +749,58 @@ export function TracingSetupFields({
|
|||
readOnly = false,
|
||||
canMintTracingKey = false,
|
||||
}: TracingSetupProps) {
|
||||
const [framework, setFramework] = useState(FRAMEWORKS[0].id);
|
||||
const [installer, setInstaller] = useState<Installer>("pip");
|
||||
const [tracingKey, setTracingKey] = useState<string | null>(null);
|
||||
const [checked, setChecked] = useState(false);
|
||||
const connection = useLensService(accessToken);
|
||||
const check = () => {
|
||||
setChecked(true);
|
||||
void connection.refetch();
|
||||
onCheck?.();
|
||||
};
|
||||
return detail === null ? (
|
||||
if (connection.isPending)
|
||||
return (
|
||||
<p role="status" className="text-sm text-muted-foreground">
|
||||
Checking Lens connection…
|
||||
</p>
|
||||
);
|
||||
if (!connection.data)
|
||||
return (
|
||||
<div className="space-y-3">
|
||||
<p role="alert" className="text-sm text-muted-foreground">
|
||||
Could not check the Lens connection.
|
||||
</p>
|
||||
<Button variant="outline" onClick={check}>
|
||||
Try again
|
||||
</Button>
|
||||
</div>
|
||||
);
|
||||
if (!connection.data.connected || !connection.data.status.storage_ready || !connection.data.url)
|
||||
return <EnableTracing connection={connection.data} checking={checking || connection.isFetching} onCheck={check} />;
|
||||
return (
|
||||
<ConnectAgent
|
||||
framework={framework}
|
||||
setFramework={setFramework}
|
||||
installer={installer}
|
||||
setInstaller={setInstaller}
|
||||
tracingKey={tracingKey}
|
||||
setTracingKey={setTracingKey}
|
||||
accessToken={accessToken}
|
||||
onOpenTrace={onOpenTrace}
|
||||
connected={connected}
|
||||
checked={checked}
|
||||
checking={checking}
|
||||
checking={checking || connection.isFetching}
|
||||
onCheck={check}
|
||||
readOnly={readOnly}
|
||||
canMintTracingKey={canMintTracingKey}
|
||||
/>
|
||||
) : (
|
||||
<EnableTracing checked={checked} checking={checking} onCheck={check} />
|
||||
);
|
||||
}
|
||||
|
||||
export function TracingSetupCard(props: TracingSetupProps) {
|
||||
const enabled = props.detail === null;
|
||||
const connection = useLensService(props.accessToken);
|
||||
const enabled = Boolean(connection.data?.connected && connection.data.status.storage_ready && connection.data.url);
|
||||
|
||||
return (
|
||||
<div className="w-full max-w-3xl pb-8" data-testid="tracing-setup-card">
|
||||
|
|
@ -746,7 +812,7 @@ export function TracingSetupCard(props: TracingSetupProps) {
|
|||
) : (
|
||||
<span aria-hidden="true" className="size-1.5 rounded-full bg-muted-foreground/50" />
|
||||
)}
|
||||
{enabled ? "Tracing enabled" : "Tracing is not enabled"}
|
||||
{enabled ? "Tracing enabled" : "Setup required"}
|
||||
</span>
|
||||
<a
|
||||
className="ml-auto inline-flex shrink-0 items-center gap-1 text-sm underline underline-offset-4"
|
||||
|
|
@ -760,7 +826,7 @@ export function TracingSetupCard(props: TracingSetupProps) {
|
|||
<p className="mt-2 text-sm leading-6 text-muted-foreground">
|
||||
{enabled
|
||||
? "Send your agent’s runs to LiteLLM to see its inputs, outputs, and tool calls."
|
||||
: "Tracing needs a Lens service with ClickHouse access and a connection from LiteLLM."}
|
||||
: "Connect Lens to start recording your agent’s runs."}
|
||||
</p>
|
||||
<TracingSetupFields {...props} />
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -34,6 +34,24 @@ vi.mock("../detail/run/RunView", () => ({
|
|||
|
||||
import { agentTraceListCall, apiClient } from "../../../networking";
|
||||
|
||||
const readyService = {
|
||||
configured: true,
|
||||
url: "https://traces.test",
|
||||
connected: true,
|
||||
status: { storage_ready: true, credentials_ready: true },
|
||||
};
|
||||
const missingService = {
|
||||
configured: false,
|
||||
url: "",
|
||||
connected: false,
|
||||
release: "v1.2.3",
|
||||
status: { storage_ready: false, credentials_ready: false },
|
||||
};
|
||||
const serveService = (ready: boolean) => {
|
||||
const service = ready ? readyService : missingService;
|
||||
vi.mocked(apiClient.get).mockImplementation(async (path) => (path === "/lens/service" ? service : { data: [] }));
|
||||
};
|
||||
|
||||
const runs = (traceList as TracePage).data as TraceSummary[];
|
||||
|
||||
const lastUrl = (onUrlUpdate: ReturnType<typeof vi.fn>) =>
|
||||
|
|
@ -77,11 +95,7 @@ describe("AgentTracesSection", () => {
|
|||
setupIntersectionMocking(vi.fn);
|
||||
testQueryClient.clear();
|
||||
vi.mocked(agentTraceListCall).mockReset();
|
||||
vi.mocked(apiClient.get).mockImplementation(async (path) =>
|
||||
path === "/lens/service"
|
||||
? { url: "https://traces.test", connected: true, status: { storage_ready: true, credentials_ready: true } }
|
||||
: { data: [] },
|
||||
);
|
||||
serveService(true);
|
||||
vi.mocked(apiClient.post).mockImplementation(async (_path, options) => {
|
||||
const body = options?.body as { traces: { trace_id: string; trace_ref?: string }[] };
|
||||
return body.traces.map((trace) => ({ ...trace, finding_count: null }));
|
||||
|
|
@ -158,21 +172,18 @@ describe("AgentTracesSection", () => {
|
|||
expect(agentTraceListCall).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("renders the setup snippet when the proxy answers 501", async () => {
|
||||
it("links to installation when the proxy has no Lens configured", async () => {
|
||||
serveService(false);
|
||||
vi.mocked(agentTraceListCall).mockRejectedValue(
|
||||
new ApiError("Agent tracing is not enabled", 501, { detail: "Agent tracing is not enabled" }),
|
||||
);
|
||||
renderSection();
|
||||
|
||||
const card = await screen.findByTestId("tracing-setup-card");
|
||||
expect(card).toHaveTextContent("Tracing is not enabled");
|
||||
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
|
||||
expect(card).toHaveTextContent("LITELLM_LENS_URL");
|
||||
expect(card).toHaveTextContent("LITELLM_LENS_SERVICE_TOKEN");
|
||||
expect(await screen.findByText("Install Lens")).toBeVisible();
|
||||
expect(screen.getByRole("link", { name: "Helm setup" })).toBeVisible();
|
||||
expect(screen.getByRole("link", { name: "Docker setup" })).toBeVisible();
|
||||
expect(screen.getByRole("button", { name: "Check setup" })).toBeEnabled();
|
||||
expect(card).not.toHaveTextContent(/langsmith/i);
|
||||
expect(card).toHaveTextContent("Lens service setup");
|
||||
expect(card).toHaveTextContent("Run the Lens service");
|
||||
expect(screen.queryByRole("button", { name: "Generate tracing key" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("shows the waiting guide when tracing is on but no runs have arrived", async () => {
|
||||
|
|
@ -180,7 +191,7 @@ describe("AgentTracesSection", () => {
|
|||
renderSection();
|
||||
|
||||
const card = await screen.findByTestId("tracing-setup-card");
|
||||
expect(card).toHaveTextContent("Connect your agent");
|
||||
expect(await screen.findByRole("heading", { name: "Connect your agent" })).toBeVisible();
|
||||
expect(await screen.findByText("Waiting for your first trace")).toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
|
||||
expect(card).not.toHaveTextContent("store: clickhouse");
|
||||
|
|
@ -188,7 +199,7 @@ describe("AgentTracesSection", () => {
|
|||
|
||||
it("keeps the trace list available when traces exist outside the current time window", async () => {
|
||||
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
|
||||
vi.mocked(apiClient.get).mockResolvedValue(traceList);
|
||||
vi.mocked(apiClient.get).mockImplementation(async (path) => (path === "/lens/service" ? readyService : traceList));
|
||||
renderSection();
|
||||
expect(await screen.findByText("No runs in this time range")).toBeVisible();
|
||||
expect(screen.queryByTestId("tracing-setup-card")).not.toBeInTheDocument();
|
||||
|
|
@ -199,10 +210,12 @@ describe("AgentTracesSection", () => {
|
|||
});
|
||||
|
||||
it("checks proxy readiness, waits for an agent, and confirms receipt using actual query results", async () => {
|
||||
serveService(false);
|
||||
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Tracing is not enabled", 501, {}));
|
||||
renderSection();
|
||||
const checkSetup = await screen.findByRole("button", { name: "Check setup" });
|
||||
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
|
||||
serveService(true);
|
||||
fireEvent.click(checkSetup);
|
||||
expect(await screen.findByRole("heading", { name: "Connect your agent" })).toBeVisible();
|
||||
expect(await screen.findByText("Waiting for your first trace")).toBeVisible();
|
||||
|
|
@ -215,6 +228,7 @@ describe("AgentTracesSection", () => {
|
|||
});
|
||||
|
||||
it("keeps setup visible while checking and explains when tracing is still disabled", async () => {
|
||||
serveService(false);
|
||||
const failure = new ApiError("Tracing is not enabled", 501, {});
|
||||
vi.mocked(agentTraceListCall).mockRejectedValue(failure);
|
||||
renderSection();
|
||||
|
|
@ -228,17 +242,19 @@ describe("AgentTracesSection", () => {
|
|||
);
|
||||
fireEvent.click(checkSetup);
|
||||
expect(await screen.findByRole("button", { name: "Checking…" })).toBeDisabled();
|
||||
expect(screen.getByRole("heading", { name: "Enable tracing" })).toBeVisible();
|
||||
expect(screen.getByRole("heading", { name: "Set up Lens" })).toBeVisible();
|
||||
await act(async () => rejectCheck(failure));
|
||||
expect(await screen.findByText(/Tracing is still unavailable/)).toBeVisible();
|
||||
expect(await screen.findByText("Install Lens")).toBeVisible();
|
||||
expect(screen.getByRole("button", { name: "Check setup" })).toBeEnabled();
|
||||
});
|
||||
|
||||
it("keeps received traces and the open drawer visible during subsequent fetches", async () => {
|
||||
serveService(false);
|
||||
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Tracing is not enabled", 501, {}));
|
||||
renderSection();
|
||||
const checkSetup = await screen.findByRole("button", { name: "Check setup" });
|
||||
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
|
||||
serveService(true);
|
||||
fireEvent.click(checkSetup);
|
||||
const rows = await screen.findAllByTestId("agent-trace-row");
|
||||
fireEvent.click(rows[0]);
|
||||
|
|
@ -269,7 +285,7 @@ describe("AgentTracesSection", () => {
|
|||
expect(screen.getByText("No runs in this time range")).toBeVisible();
|
||||
expect(screen.queryByText(/Could not load runs/)).not.toBeInTheDocument();
|
||||
|
||||
vi.mocked(apiClient.get).mockResolvedValue(traceList);
|
||||
vi.mocked(apiClient.get).mockImplementation(async (path) => (path === "/lens/service" ? readyService : traceList));
|
||||
fireEvent.click(screen.getByRole("button", { name: "Retry trace check" }));
|
||||
await waitFor(() => expect(screen.queryByRole("alert")).not.toBeInTheDocument());
|
||||
expect(screen.getByText("No runs in this time range")).toBeVisible();
|
||||
|
|
@ -277,12 +293,13 @@ describe("AgentTracesSection", () => {
|
|||
});
|
||||
|
||||
it("treats a proxy without the trace routes (404) like tracing being off", async () => {
|
||||
serveService(false);
|
||||
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Not Found", 404, { detail: "Not Found" }));
|
||||
renderSection();
|
||||
|
||||
const card = await screen.findByTestId("tracing-setup-card");
|
||||
expect(card).toHaveTextContent("Tracing is not enabled");
|
||||
expect(card).toHaveTextContent("LITELLM_LENS_SERVICE_TOKEN");
|
||||
expect(await screen.findByText("Install Lens")).toBeVisible();
|
||||
expect(card).toHaveTextContent("Use Lens v1.2.3");
|
||||
});
|
||||
|
||||
it("lists uninvestigated runs without presenting tool errors as failures", async () => {
|
||||
|
|
|
|||
10
ui/litellm-dashboard/src/lib/http/schema.d.ts
generated
vendored
10
ui/litellm-dashboard/src/lib/http/schema.d.ts
generated
vendored
|
|
@ -45534,8 +45534,18 @@ export interface components {
|
|||
};
|
||||
/** ServiceConnection */
|
||||
ServiceConnection: {
|
||||
/**
|
||||
* Configured
|
||||
* @default false
|
||||
*/
|
||||
configured: boolean;
|
||||
/** Connected */
|
||||
connected: boolean;
|
||||
/**
|
||||
* Release
|
||||
* @default
|
||||
*/
|
||||
release: string;
|
||||
status: components["schemas"]["ServiceStatus"];
|
||||
/** Url */
|
||||
url: string;
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue